Audit Log Correlation

Kubernetes audit logs provide detailed records of API interactions, while Falco generates real-time security alerts. Correlating these two data sources enables deeper analysis of runtime security events by combining Falco’s behavioral detection with Kubernetes’ operational context. This integration is critical for identifying malicious activity, troubleshooting false positives, and meeting compliance requirements.

---

1. Prerequisites and Tools

Before correlating Falco alerts with Kubernetes audit logs, ensure: - Falco is deployed in your cluster, configured to output alerts in a structured format (e.g., JSON or Syslog). - Kubernetes audit logs are enabled and exported to a centralized logging system (e.g., Elasticsearch, Fluentd, or cloud-native solutions like AWS CloudWatch or GCP Logging). - A log aggregation pipeline (e.g., ELK stack, Loki, or Prometheus + Grafana) to unify Falco and audit logs.

---

2. Falco Alert Structure

Falco alerts typically include: - timestamp: ISO8601-formatted time. - container.name: Name of the affected container. - proc.name: Process name. - evt.type: Type of event (e.g., container.exec). - output: Human-readable summary of the alert.

Example:

{
  "timestamp": "2023-10-05T14:23:45.678Z",
  "container.name": "nginx",
  "proc.name": "sh",
  "evt.type": "container.exec",
  "output": "A container executed a command with elevated privileges"
}

---

3. Kubernetes Audit Log Structure

Kubernetes audit logs (from the kube-apiserver audit webhook) include: - @timestamp: ISO8601 time. - user.user: User or service account. - request.method: HTTP method (e.g., POST). - request.uri: API endpoint (e.g., /api/v1/namespaces/default/pods). - response.status: HTTP status code.

Example:

{
  "@timestamp": "2023-10-05T14:23:45.678Z",
  "user.user": "system:serviceaccount:default:nginx",
  "request.method": "POST",
  "request.uri": "/api/v1/namespaces/default/pods",
  "response.status": 201
}

---

4. Correlation Strategy

To correlate Falco alerts with Kubernetes audit logs: 1. Timestamp alignment: Use the timestamp field from Falco and @timestamp from audit logs to identify overlapping events. 2. Contextual matching: - Match container.name from Falco with request.uri in audit logs (e.g., /pods/nginx). - Use proc.name or output to infer the action (e.g., exec or create). 3. User/service account mapping: Cross-reference user.user in audit logs with Falco’s container.name or proc.name to identify the actor.

---

5. Implementation Example

Step 1: Configure Falco to Output JSON

Edit falco.yaml to set the output format:

output:
  - type: json
    filename: /var/log/falco/falco.json

Step 2: Aggregate Logs with Fluentd

Use Fluentd to collect both Falco and audit logs:

<match **>
  @type elasticsearch
  host elasticsearch
  port 9200
  logstash_format true
  logstash_prefix falco
</match>

Step 3: Query Correlated Data in Kibana

Use Elasticsearch queries to join Falco and audit logs:

{
  "query": {
    "bool": {
      "must": [
        { "match": { "falco.timestamp": "2023-10-05T14:23:45.678Z" }},
        { "match": { "audit.request.uri": "/pods/nginx" }}
      ]
    }
  }
}

---

6. Use Cases

• Detecting privilege escalation: A Falco alert about container.exec with elevated privileges can be correlated with an audit log showing a POST to /pods with user.user as a high-privilege service account.
• Troubleshooting false positives: Cross-referencing Falco alerts with audit logs can confirm whether a suspicious command was part of legitimate operations (e.g., a scheduled job).

---

Key takeaways

• Correlating Falco alerts with Kubernetes audit logs provides contextual insights into security events.
• Use centralized logging tools (e.g., ELK stack) to unify and analyze both data sources.
• Focus on matching timestamps, container names, and user identities for effective correlation.
• Leverage SIEM or custom queries to detect patterns like privilege escalation or unauthorized access.