Alert Status Management#

Alert status management in Keep covers how individual and bulk status changes flow from the UI through the API to persistence, workflow triggering, and incident resolution. Key concerns are: preventing duplicate submissions, deduplicating incident resolution checks, and efficiently evaluating workflow triggers across large alert batches.

Alert Status Change Flow#

Alert status updates are processed as enrichments — arbitrary key/value metadata attached to an alert fingerprint. A status change sets status (and optionally dismissed/dismissUntil) in the enrichment payload.

Keep supports five statuses: firing, resolved, acknowledged, suppressed, and pending 1.

Single-alert path:
The AlertChangeStatusModal posts to POST /alerts/enrich?dispose_on_new_alert=<bool> with a fingerprint and enrichments body 2. The backend handler enrich_alert delegates to _enrich_alert.

Bulk path:
batch_enrich_alerts handles POST /alerts/batch_enrich. It accepts either an explicit fingerprints list or a CEL expression (never both simultaneously) 3. When cel is given, matching alerts are resolved first via query_last_alerts and their fingerprints extracted 4. After fingerprints are resolved either way, the function calls enrichment_bl.batch_enrich(...) and then:

1. Indexes enriched alerts to Elasticsearch 5
2. Pushes a Pusher poll-alerts event to connected clients 6
3. Triggers workflow evaluation via WorkflowManager.insert_events(...) if applicable 7
4. Checks whether any linked incidents should auto-resolve 8

UI Safety — Duplicate Submission Prevention#

Without protection, double-clicking "Change Status" fires duplicate API requests and shows duplicate success toasts. PR #6622 addresses this with an isSubmitting state flag in AlertChangeStatusModal.

• Both handleChangeStatus (single alert) and handleChangeStatusBatch (bulk) set isSubmitting(true) before making API calls and reset it in a finally block.
• The Cancel and primary action buttons are disabled while isSubmitting is true, preventing re-entry.
• Batch mode includes an early-return guard: if no status is selected, an error toast is shown and the function exits immediately 9.

Dispose on new alert toggle:
The modal also exposes a "Dispose on new alert" switch 10, defaulting to true. When enabled (dispose_on_new_alert=true), the status enrichment is cleared automatically when a new alert with the same fingerprint arrives — preventing a stale manual status from persisting on a re-firing alert.

Backend — Incident Resolution Deduplication#

When a bulk status change resolves alerts, Keep must also check whether their parent incidents should auto-resolve. The logic lives at the end of batch_enrich_alerts.

Current (pre-fix) logic:

for alert in alerts:
    for incident in alert._incidents:
        if incident.resolve_on == ResolveOn.ALL and is_all_alerts_resolved(...):
            incident.status = RESOLVED
            session.add(incident)
        session.commit() # ← called inside nested loop

This causes the same incident to be processed and committed multiple times when multiple enriched alerts belong to the same incident 8.

Post-fix approach (PR #6622):
Build a unique_incidents = {incident.id: incident ...} dict across all alerts, then iterate unique_incidents.values() once — guaranteeing each incident is evaluated at most once — followed by a single session.commit() outside the loop.

The resolution gate uses two conditions: incident.resolve_on == ResolveOn.ALL.value (incident is configured to resolve only when all alerts resolve) and is_all_alerts_resolved(incident, session) returning True 11. Incident data is hydrated beforehand via enrich_alerts_with_incidents(tenant_id, alerts) 12.

Backend — Workflow Caching Across Batch Operations#

After enrichment, if workflows should run, batch_enrich_alerts calls WorkflowManager.insert_events(tenant_id, enriched_alerts_dto). The insert_events method evaluates which workflows should fire for each alert in the batch.

Pre-fix behavior in insert_events: for every event in the batch, all tenant workflows are re-fetched from the database and re-parsed. This creates O(events × workflows) complexity — for a bulk change of 100 alerts across 20 workflows, that is 2,000 parse operations.

Post-fix approach (PR #6622):

• Fetch all workflow models once before the event loop.
• Parse them into a parsed_workflows list (one _get_workflow_from_store() call per workflow).
• Reuse parsed_workflows for every event's trigger evaluation — reducing complexity to O(workflows + events).
• For each workflow that does match and gets queued, a fresh workflow instance is created via another _get_workflow_from_store() call. This ensures that any state mutations from trigger evaluation don't bleed into the actual execution context 9.

Matched workflows are enqueued via scheduler.workflows_to_run.append(...) under a threading lock 13. Trigger evaluation uses CEL (Common Expression Language) with backward-compatibility support for legacy filter formats.

Key Files & Entry Points#

Related PR: #6622 — Fix slow bulk alert status changes and prevent duplicate submissions (open as of 2026-07-08) — the definitive reference for all three optimizations described above.