Skip to content

Analyzing Security Events

Kubernetes runtime security often requires monitoring both Falco alerts and Kubernetes audit logs to detect advanced threats. By combining these data sources, you can identify patterns that indicate unauthorized access, lateral movement, or other sophisticated attacks. This section explores techniques for correlating and analyzing these data streams effectively.

Correlating Falco Alerts and Audit Logs

1. Timestamp Alignment for Event Sequencing

Falco alerts and Kubernetes audit logs are often generated asynchronously. To identify potential threats, align events by timestamp. For example, a suspicious container_exec alert might be preceded by an audit log entry showing a user with elevated privileges.

Example: 

# Extract Falco alerts and audit logs, then sort by timestamp
kubectl get auditlog -n kube-system -o json | jq '.items[] | {timestamp: .spec.time, event: "audit"}'
falco --output=json | jq '.alerts[] | {timestamp: .timestamp, event: "falco"}'

2. Event Type Mapping

Map Falco alert types (e.g., container exec, file access) to corresponding Kubernetes audit log actions (e.g., User.authentication, Pod.create). This helps contextualize alerts within the broader system behavior.

Example: 

# Filter audit logs for user authentication events
kubectl get auditlog -n kube-system -o json | jq '.items[] | select(.spec.request.user != "system:serviceaccount")'

3. Contextual Enrichment

Enrich Falco alerts with metadata from audit logs, such as pod names, namespaces, or user identities. This provides deeper visibility into the attack surface.

Example: 

# Join Falco alerts with pod metadata from audit logs
kubectl get auditlog -n kube-system -o json | jq '.items[] | {pod: .spec.request.resource.name, user: .spec.request.user}' | jq -s add

Tools for Combined Analysis

1. Centralized Log Aggregation

Use tools like ELK Stack (Elasticsearch, Logstash, Kibana) or Grafana Loki to aggregate Falco alerts and audit logs. This enables unified querying and visualization.

Example: 

# Forward Falco alerts to Loki via UDP
falco --output=udp://loki:3100

2. SIEM Integration

Integrate with Security Information and Event Management (SIEM) platforms like Splunk or IBM QRadar to automate threat detection workflows. Configure rules that trigger alerts when Falco events correlate with audit log anomalies.

3. Custom Dashboards

Build dashboards in Kibana or Grafana to visualize combined data. For example, track the frequency of container_exec alerts alongside user authentication events.

Advanced Detection Patterns

1. Anomaly Detection

Look for spikes in events like container_exec or file_create that deviate from baseline behavior. Use tools like Prometheus + Grafana to monitor metrics over time.

Example PromQL Query: 

rate(falco_container_exec_total[5m]) > 10

2. User Behavior Analysis

Detect unusual user activity, such as a non-privileged user attempting to access sensitive resources. Cross-reference audit logs with Falco alerts to identify potential privilege escalation.

3. Lateral Movement Indicators

Identify signs of lateral movement, such as repeated exec commands across multiple pods or unexpected API calls to the Kubernetes API server.

Example: 

# Find Falco alerts with repeated pod targets
falco --output=json | jq '.alerts[] | select(.rule == "container exec") | {pod: .container.image, count: length}'

Key takeaways

  • Correlate timestamps and event types to sequence security incidents.
  • Use centralized log tools like Loki or ELK to unify Falco and audit data.
  • Detect anomalies in user behavior or event frequency to spot advanced threats.
  • Enrich alerts with pod and user metadata for deeper forensic analysis.
  • Leverage SIEM platforms to automate threat response workflows.