Notification Templates

Customizing Alert Notifications with Templates¶

Alertmanager allows you to tailor the content of alerts sent to different notification channels (e.g., email, Slack, PagerDuty) using notification templates. These templates are written in Go's text/template syntax and can dynamically inject alert details, labels, annotations, and custom logic. Below, we walk through creating templates for three common integrations.

📧 Email Templates¶

Email notifications use the email configuration block. Customize the subject and body using Go templates.

Example: Basic Email Template¶

email_configs:
- to: "team@example.com"
  subject: "{{ .Status }}: {{ .Labels.alertname }}"
  body: |
    {{ define "email.body" }}
    **Alert:** {{ .Status }}
    **Alert Name:** {{ .Labels.alertname }}
    **Severity:** {{ .Labels.severity }}
    **Description:** {{ .Annotations.summary }}
    **Labels:**
    {{ range $key, $value := .Labels }}
    - **{{ $key }}:** {{ $value }}
    {{ end }}
    **Annotations:**
    {{ range $key, $value := .Annotations }}
    - **{{ $key }}:** {{ $value }}
    {{ end }}
    {{ end }}

Key Template Variables¶

{{ .Status }}: Alert status (e.g., "firing" or "resolved").
{{ .Labels }}: Map of alert labels (e.g., severity, job).
{{ .Annotations }}: Map of annotations (e.g., summary, description).

💬 Slack Templates¶

Slack notifications use the slack configuration block. Format messages with text and attachments for rich formatting.

Example: Slack Attachment Template¶

slack_configs:
- channel: "#alerts"
  text: "New alert: {{ .Labels.alertname }}"
  attachments:
  - fallback: "Alert: {{ .Labels.alertname }}"
    color: "#FF0000"
    title: "{{ .Status }}"
    text: |
      **Summary:** {{ .Annotations.summary }}
      **Severity:** {{ .Labels.severity }}
      **Labels:**
      {{ range $key, $value := .Labels }}
      - **{{ $key }}:** {{ $value }}
      {{ end }}
    footer: "Alertmanager"

Tips¶

Use {{ .Status }} for alert state (e.g., "firing").
Add fields for structured data (e.g., {{ .Labels }} as key-value pairs).

📡 PagerDuty Templates¶

PagerDuty requires a pagerduty configuration block with summary and details fields.

Example: PagerDuty Summary and Details¶

pagerduty_configs:
- service_key: "your-service-key"
  summary: "Alert: {{ .Labels.alertname }} ({{ .Status }})"
  details: |
    **Severity:** {{ .Labels.severity }}
    **Description:** {{ .Annotations.summary }}
    **Labels:**
    {{ range $key, $value := .Labels }}
    - **{{ $key }}:** {{ $value }}
    {{ end }}

Best Practices¶

Keep summary concise for PagerDuty's UI.
Use details for full contextual information.

🧠 Advanced Template Customization¶

Conditional Logic¶

Use {{ if }} to tailor messages based on alert severity:

{{ if eq .Labels.severity "critical" }}
**CRITICAL ALERT:** {{ .Labels.alertname }}
{{ else }}
**WARNING:** {{ .Labels.alertname }}
{{ end }}

Reusable Templates¶

Define reusable templates in templates/ and reference them:

templates:
- /path/to/custom_template.tmpl

🔍 Testing Templates¶

Local Testing: Use the Alertmanager CLI with a --template-file flag to validate syntax.
Dry Run: Send a test alert via the /api/v1/alerts endpoint with a mock payload.

Key takeaways¶

Use Go templates to dynamically inject alert metadata into notifications.
Structure templates with clear sections (e.g., headers, labels, annotations) for readability.
Tailor templates to each integration's requirements (e.g., Slack's attachments, PagerDuty's summary/details).
Test templates in isolation to avoid formatting errors in production.