feat(alerts): deterministic metric-transition reducer with manager parity harness

Phase 1 of docs/ALERT_ENGINE_EVOLUTION.md. internal/alerts/reducer is a
pure transition core for the metric-threshold family — hysteresis
trigger/clear, sustained-for delay with dip reset, warning/critical
severity derivation with the percentage 99-cap — characterized from
Manager.checkMetric, with time entering only through the signal's
ObservedAt so every sequence is deterministic and replayable.

reducer_parity_test.go drives the live manager and the reducer through
identical observation sequences (simulated time via the established
pending-backdate trick) and fails on any divergence after every step,
with the manager as the reference. The harness proved itself on its
first run by catching the stale-resolve defect fixed in the previous
commit.

Registers coverage gap alert-lifecycle-contract-coverage. Remaining
slices: offline/confirmation families, ack lifecycle, intent
interaction, shadow-mode runtime feed.
This commit is contained in:
rcourtman 2026-08-26 22:53:10 +01:00
parent 20ea1ea4d3
commit 7c212b906e
5 changed files with 828 additions and 2 deletions

View file

@ -0,0 +1,38 @@
# Alert lifecycle transitions lack a deterministic contract harness
Recorded: 2026-08-26
Origin: alerting audit (2026-08-26) finding 2 and docs/ALERT_ENGINE_EVOLUTION.md
Phase 1.
## Gap
The alert manager's transition semantics (hysteresis trigger/clear,
sustained-for delays, severity derivation, confirmation counts) are
implemented imperatively across per-family check paths and asserted only
by per-unit tests. There is no deterministic, engine-independent statement
of the lifecycle contract, so regressions in the frozen transition core
keep shipping (#1682, #1683, #1553, #1693, #1724) and cross-family
semantic drift is invisible.
## Resolution
Phase 1 of docs/ALERT_ENGINE_EVOLUTION.md: a pure transition reducer
(`internal/alerts/reducer`) that characterizes the manager's semantics
family by family, plus a parity harness (`reducer_parity_test.go` in
`internal/alerts`) that drives both engines through identical observation
sequences and fails on any divergence. The reducer must characterize, not
improve: divergences are reducer bugs unless investigation proves a
manager defect, which is then fixed in the manager first.
First slice (2026-08-26): the metric-threshold family. The harness
immediately caught a real manager defect: `checkMetric` hysteresis
resolution removed alerts by the legacy `<resourceID>-<metric>` ID while
canonical-identity alerts are stored under the canonical state key and the
legacy ID carries no alias, so the removal silently no-oped — a resolved
notification went out while the alert stayed active and re-resolved every
poll (guest per-disk usage alerts were the remaining production caller;
user-visible as the stale-alert class, e.g. #1580). Fixed at the caller
with a focused regression test before the parity slice landed.
Remaining for later slices: offline/confirmation families, ack/snooze
lifecycle, intent-policy interaction, and a shadow-mode runtime feed.

View file

@ -1,6 +1,6 @@
{
"version": "6.0",
"updated_at": "2026-08-08",
"updated_at": "2026-08-26",
"scope": {
"active_repos": [
"pulse",
@ -9787,10 +9787,52 @@
"kind": "file"
}
]
},
{
"id": "alert-lifecycle-contract-coverage",
"summary": "Alert transition semantics (hysteresis, sustained-for delays, severity derivation, confirmations) have no deterministic engine-independent contract harness, so transition-core regressions ship undetected (#1682, #1683, #1553, #1693, #1724) and cross-family drift is invisible. Resolution is Phase 1 of docs/ALERT_ENGINE_EVOLUTION.md: a pure characterization reducer (internal/alerts/reducer) plus a parity harness driving reducer and manager through identical sequences; the first metric-family slice already surfaced and fixed a silent stale-resolve defect in checkMetric.",
"owner": "project-owner",
"status": "triaged",
"recorded_at": "2026-08-26",
"lane_ids": [
"L6",
"L13"
],
"subsystem_ids": [
"alerts"
],
"proposed_resolution": "lane-expansion",
"coverage_impact": 5,
"evidence": [
{
"repo": "pulse",
"path": "docs/release-control/v6/internal/records/alert-lifecycle-contract-coverage-2026-08-26.md",
"kind": "file"
},
{
"repo": "pulse",
"path": "docs/ALERT_ENGINE_EVOLUTION.md",
"kind": "file"
}
]
}
],
"candidate_lanes": [],
"work_claims": [],
"work_claims": [
{
"id": "claude-alert-reducer-coverage-gap-alert-lifecycle-contract-coverage",
"agent_id": "claude-alert-reducer",
"summary": "Phase 1 slice: metric-family reducer + parity harness; includes checkMetric stale-resolve fix",
"target_id": "v6-product-lane-expansion",
"claimed_at": "2026-08-26T21:52:17Z",
"heartbeat_at": "2026-08-26T21:52:17Z",
"expires_at": "2026-08-27T01:52:17Z",
"work_item": {
"kind": "coverage-gap",
"id": "alert-lifecycle-contract-coverage"
}
}
],
"open_decisions": [],
"source_of_truth_file": "docs/release-control/v6/internal/SOURCE_OF_TRUTH.md",
"resolved_decisions": [