Alerting System Data Models#

Alert Struct#

The Alert struct models a single alert event generated by the detection engine. It includes the following fields (source):

pub struct Alert {
    pub id: Uuid,
    pub severity: AlertSeverity,
    pub title: String,
    pub description: String,
    pub detection_rule_id: String,
    pub process_record: ProcessRecord,
    pub timestamp: SystemTime,
    pub deduplication_key: String,
    pub context: AlertContext,
}

The struct derives Serialize and Deserialize via serde, supporting JSON serialization and deserialization. This enables easy transmission and storage of alert data.

AlertSeverity#

AlertSeverity is an enum representing the severity of an alert. Possible values are Low, Medium, High, and Critical. It supports string parsing (case-insensitive) and display as lowercase strings. Example:

let sev = "High".parse::<AlertSeverity>().unwrap();
assert_eq!(sev, AlertSeverity::High);
assert_eq!(sev.to_string(), "high");

AlertContext#

AlertContext provides additional context for an alert. It contains a HashMap<String, String> for arbitrary key-value data, a vector of tags, an optional source string, and an optional confidence value (a f64 between 0.0 and 1.0). Builder-style methods allow chaining:

let ctx = AlertContext::new()
    .with_data("user", "alice")
    .with_tag("suspicious")
    .with_source("kernel")
    .with_confidence(0.85).unwrap();

Alert Construction and Methods#

The Alert::new constructor generates a new UUID, sets the timestamp to the current system time, and constructs the deduplication key as <severity>:<detection_rule_id>:<title>. Methods are provided to add context data, tags, source, and confidence (with validation). Alerts can compute their age in seconds and determine if they are "recent" based on a configurable threshold (source).

Deduplication Keys#

The deduplication_key field uniquely identifies an alert to prevent duplicates. It is constructed as <severity>:<detection_rule_id>:<title> at alert creation. This key is used by downstream systems (such as alert sinks) to identify and suppress duplicate alerts within a configurable deduplication window (source).

Detection Engine#

Rule Identification and Matching#

Detection rules are modeled by the DetectionRule struct, which includes fields such as id, name, description, sql_query, severity, enabled, tags, and metadata. Each rule is uniquely identified by its id (source).

The detection engine (DefaultDetectionEngine) manages rules in a HashMap keyed by rule ID. Rules are loaded using load_rule, which validates the rule's SQL before insertion. Rules can be enabled or disabled, and removed by ID.

When executing, the engine evaluates enabled rules against process data using DataFusion-based SQL query execution over redb-backed TableProvider implementations. The SQL query specified in each rule is executed via a DataFusion SessionContext against the event store tables. Matches produce alerts with deduplication and completeness markers. Event predicates use a numeric severity_level field (0=trace, 1=debug, 2=info, 3=warn, 4=error) for ordering; string comparisons like $.severity >= 'high' are avoided as they yield incorrect lexicographic results (source).

Rule Validation#

Rules are validated using the validate_sql method, which ensures that the SQL query is a single SELECT statement, does not use banned functions, and does not exceed limits on projections or joins. Validation performs SQL schema validation and AST analysis to prevent direct database access. The is_valid method checks that the rule has a non-empty name and SQL query and passes SQL validation (source).

SQL-to-IPC Detection Engine (ADR-0006)#

The detection engine uses a two-phase pipeline: validation and planning at rule load time, and DataFusion-based execution per collection cycle. Detection rules travel this strict pipeline where the original SQL dialect never reaches the execution layer.

Architecture Overview#

1. Rule load: sqlparser parses the rule; AST validation enforces SELECT-only statements, an approved-function allowlist, and configurable maximum subquery nesting depth (default: 3). Table references are validated against the schema catalog.
2. Pushdown planning: eligible predicates (equality, inequality, IN, LIKE, REGEXP) and projections are lowered into protobuf DetectionTasks addressed to the owning collectors.
3. Derivation: the remainder of the rule is lowered to derived standard SQL that DataFusion can execute.
4. Execution: each collection cycle, the derived SQL runs in a DataFusion SessionContext whose catalog is populated by redb-backed per-collector TableProvider implementations.
5. Alerting: matches produce alerts with deduplication and completeness markers.

Internal Structure#

The DefaultDetectionEngine uses DataFusion SessionContext and TableProvider implementations rather than holding a direct redb::Database handle. The engine manages rule execution through:

• query_ctx: DataFusion session executing validated/derived SQL
• table_providers: redb per-domain tables exposed through TableProvider implementations
• rule_manager: manages rule lifecycle and validation
• alert_manager: handles alert generation and deduplication
• sql_validator: performs AST validation and schema checking

Event Subscription#

The subscribe_to_events method returns a bounded Receiver<BusEvent> instead of an unbounded receiver. Channel capacity is configurable per topic class, with an overflow policy that drops the oldest events. A monotonic dropped-event counter is surfaced via health metrics to track backpressure and event loss.

Severity Level Convention#

Event predicates use a numeric severity_level field for ordering:

• 0 = trace
• 1 = debug
• 2 = info
• 3 = warn
• 4 = error

This convention avoids errors from string comparison (e.g., 'critical' < 'high' lexicographically), ensuring correct severity-based filtering in detection rules.

Configuration Options#

recent_threshold_seconds#

The AlertingConfig struct defines alerting configuration, including the recent_threshold_seconds field. This field specifies the threshold (in seconds) for considering an alert as "recent". The default is 3600 seconds (1 hour). This value is used by Alert::is_recent_with_config to determine alert recency (source).

The configuration system supports hierarchical overrides: system config file, user config file, environment variables, and embedded defaults. The environment variable {PREFIX}_RECENT_THRESHOLD_SECONDS can override the recent_threshold_seconds value.

Other relevant configuration fields include dedup_window_seconds (deduplication window) and max_alerts_per_minute (rate limiting).

Testing and Validation Practices#

The alerting system includes comprehensive unit tests for alert creation, serialization/deserialization, severity parsing and display, context manipulation, confidence validation, alert age, and recency checks with custom thresholds and configurations. Detection rules are tested for creation, serialization, SQL validation (including rejection of invalid or unsafe queries), and lifecycle operations (enable/disable, tagging, metadata). The detection engine is tested for rule loading, duplicate rule handling, invalid SQL rejection, rule removal, enabling/disabling, and rule execution scenarios (including multiple rules, disabled rules, and no matches) (alert tests, rule tests, engine tests, config tests).

Configuration loading and merging, including environment variable overrides, are also validated by tests to ensure correct application of settings and error handling for invalid configurations.

Example: Creating and Serializing an Alert#

let process = ProcessRecord::new(1234, "test-process".to_owned());
let alert = Alert::new(
    AlertSeverity::High,
    "Suspicious Process Detected",
    "A potentially malicious process was detected",
    "rule-001",
    process,
)
.with_tag("malware")
.with_source("kernel")
.with_confidence(0.95)
.expect("valid confidence");

let json = serde_json::to_string(&alert).unwrap();
let deserialized: Alert = serde_json::from_str(&json).unwrap();
assert_eq!(alert, deserialized);

Example: Checking Alert Recency with Configuration#

let alert = /* ... */;
let config = AlertingConfig::default();
if alert.is_recent_with_config(&config) {
    // Alert is recent (within config.recent_threshold_seconds)
}

Example: Detection Engine Rule Loading and Execution#

let mut engine = DefaultDetectionEngine::new();
let rule = DetectionRule::new(
    "rule-1",
    "Suspicious Process Rule",
    "Detects suspicious processes",
    "SELECT * FROM processes WHERE name LIKE '%suspicious%'",
    "suspicious_process",
    AlertSeverity::High,
);
engine.load_rule(rule).unwrap();

let processes = vec![ProcessRecord::new(1234, "suspicious-process".to_owned())];
let alerts = engine.execute_rules(&processes);
assert!(!alerts.is_empty());