The deterministic capacity forecast queried utilization history using the
canonical resource ID (storage-<hash>), but the monitoring layer records
metrics under the metrics-target/source ID (MetricsTarget.ResourceID). For
most storage sources these two IDs diverge, so the forecast query always
found nothing and the feature stayed dormant even for pools with full
ingestion history (Proxmox, TrueNAS, PBS, agent-backed storage).
This splits the two ID roles in the precompute path: a new metricsID field
on patrolStoragePoolRow / patrolPrecomputeStorageSource carries the
metrics-target ID (resolved from SourceID() in the readState branch, or
MetricsTarget.ResourceID in the unified-resource branch) for history
queries, while the canonical id is still used for stamp-matching findings.
Adds a test proving the forecast fires when history exists only under the
metrics-target ID and that the resulting forecast resourceID stays
canonical so StampCapacityForecasts can still match it.
Contract (ai-runtime): the forecast must query history under the
metrics-target ID (StoragePoolView.SourceID()), not the canonical ID; the
forecast's own resourceID stays canonical for stamp-matching.
Refs lane: protection-posture-attention-queue
Capacity forecasts (days-to-full, current usage, trend) were previously
computed deterministically but only fed into LLM prompt text. They never
reached the finding as structured data, so the frontend could not render a
verified urgency signal and the model's speculation was the only thing the
operator saw.
This persists CapacityForecast on Finding (marshal-mirror pair) and stamps
it post-analysis via a service join (StampCapacityForecasts). The forecast
filter now also keeps stable-high (>=80%) pools so the deterministic "no
fill trend" reading wins over model speculation, and fixes isQuiet wrongly
treating stable (-1) trends as filling.
Frontend maps the forecast through the UnifiedFinding view model and renders
a deterministic urgency line (Filling up / Stable / days-to-full · % used)
in the expanded finding detail.
Note: forecasts only populate for resources whose usage is ingested into
metrics history (Proxmox/Ceph storage, nodes, guests). Agent-host storage
(unraid pools) is not yet ingested as a time-series and remains a follow-up
to activate the feature for those pools.
Refs lane: protection-posture-attention-queue
When a Proxmox node has an agent installed, SMART/temperature metrics are
written by the host agent under the agent's disk source ID. But
BuildMetricsTarget preferred the Proxmox source first, and Proxmox API
does not expose detailed SMART data. The mismatch meant the frontend
queried a metrics key that nothing writes, so disk metrics appeared empty.
Move the Agent source check ahead of Proxmox/TrueNAS for
ResourceTypePhysicalDisk, mirroring the existing pattern for
ResourceTypeAgent where agent source already wins over platform sources.
When a disk has a serial number, it is used as the canonical metric ID
regardless of source (PreferredPhysicalDiskMetricID), so the reorder
only affects disks without serials — exactly the case where source
priority determines the key.
Refs #1487
The unified_resources.db grew without bound (2GB reported) because:
1. No VACUUM: DELETE freed rows internally but never shrank the file.
Added auto_vacuum(INCREMENTAL) to the DSN for new databases, plus a
one-time migrateAutoVacuum() that converts existing databases.
reclaimFreePages() now runs after each prune cycle to return freed
pages to the OS via PRAGMA incremental_vacuum.
2. Missing retention: action_lifecycle_events, export_audits, and
loop_reports had no retention at all. Added 90-day retention for
lifecycle/export audits and 30-day for loop_reports, matching the
existing action_audits/resource_changes cadence.
3. Slow cleanup cadence: the retention loop ran every 6h and never on
startup. Reduced to hourly and added an initial prune 30s after
startup so a restart with a bloated DB starts recovering immediately.
Mirrors the proven pattern from metrics.db (auto_vacuum INCREMENTAL +
incremental_vacuum + WAL checkpoint).
Refs #1496
getDockerCommandPayload returned dispatched commands on every report
fetch, causing the agent to re-execute check-updates on every poll
cycle. When the ack also failed, the report was buffered and retried,
creating an infinite loop.
- Only return command payload on the queued->dispatched transition;
subsequent fetches return nil (agent already received it).
- Don't propagate ack errors from handleCheckUpdatesCommand; the report
was delivered and check-updates is fire-and-forget. Command expires
if ack never succeeds.
Refs #1504
When the Proxmox API's meminfo/status payload omits Available, Buffers,
and Cached (common for QEMU guest-agent and node status responses), the
code derived 'available' from Free alone — producing used = Total-Free
which counts reclaimable page cache as used memory (e.g. 94% instead of
the correct 76%).
Guest path (deriveGuestMemInfoAvailable): return 0 when cache metrics
are completely missing and only Free is available, so resolveGuestStatusMemory
tries better sources: guest-agent /proc/meminfo file-read (returns
MemAvailable), RRD memavailable, or the linked host agent.
Node path (resolveNodeMemory): always try RRD memavailable when cache
metrics are missing, not only when effectiveAvailable == 0 — previously
a non-zero Free value blocked the RRD fallback.
Refs #1501
sendViaProviderWithAddresses mutated the shared e.config.Username for
provider-specific defaults (SendGrid, Postmark, SparkPost, Resend).
If concurrent goroutines sent email simultaneously, this was a data
race on the config struct.
Move the resolution into negotiateAuth via a local variable
(resolveProviderUsername helper) so the shared config is never mutated.
The isEditableTarget early-return blocked Cmd+K when focus was in any
input, textarea, or select element. The command palette shortcut must
work from anywhere — it is the primary keyboard navigation path.
Move the Cmd+K check before the editable-target guard.
Unraid's mdResyncAction field can retain its last value (e.g. "check")
after a parity check is canceled, causing Pulse to report a stale sync
action indefinitely. The mdResync/mdResyncPos field is the authoritative
indicator: it drops to 0 when no resync is running. Gate SyncAction on
a non-zero position so the alert clears once the sync actually stops.
Refs #1485
Add deduplicateHistory() to HistoryManager that collapses consecutive
same-alert entries within a 5-minute window. The flapping/re-fire bugs
(lifecycle and stateful paths) created many duplicate history entries
before the cooldown fixes were deployed. This cleanup runs on startup
after loadHistory and periodically alongside cleanOldEntries, so both
existing noise and any future edge-case duplicates are handled.
On this install: 332 entries → 130 (60% reduction).
handleLogin rejected all credentials when ALLOW_ADMIN_BYPASS=1 because
it validated against config AuthUser/AuthPass directly without checking
the bypass flag. This made dev-mode browser testing impossible after a
backend restart with bypass env vars — the API middleware accepted all
requests but the login page could never obtain a session cookie.
When bypass is enabled, accept any credentials and create a session as
'admin'.
Stateful alerts (ZFS pool/device, storage topology) were creating
duplicate history entries every poll cycle because SyncStorageAlertsForInstance
clears alerts between evaluations, making the existing-alert check miss.
Add the same 5-minute re-fire cooldown used in the lifecycle path: when a
stateful alert re-fires within 5 minutes of resolution, reactivate the
original (preserving StartTime) and update the existing history entry's
LastSeen instead of appending a new entry.
When an alert fires, resolves, and re-fires within 5 minutes (the
recently-resolved retention window), the previous implementation created
a new history entry for each fire cycle. For a flapping connection like
an unreachable Proxmox node, this produced hundreds of duplicate entries
in a single night — 302 identical 'pi unreachable' alerts in one case.
The fix checks the recently-resolved map before creating a new history
entry. If the alert was resolved within the cooldown window, it
reactivates the original alert (preserving StartTime) and updates the
existing history entry's LastSeen instead of appending a new one.
Rapid clicks on the copy button accumulated setTimeout callbacks that
continued firing after component unmount. Clear the previous timer
before setting a new one, and clean up on unmount.
Adds an 'Ack all (N)' button next to the '+N related' toggle on grouped
alerts. Acknowledges the primary and all related alerts in one action
using the existing bulkAcknowledge API endpoint, without touching
unrelated alerts.
The patrol_report_finding tool descriptions for evidence, impact, and
recommendation were too permissive, causing the LLM to frequently omit
them. Future patrol runs will produce better-scaffolded findings.
1. Rewrite tool descriptions to emphasize evidence as a trust anchor
that should always be included, and impact as expected whenever the
data supports it.
2. Add 'Authoring Evidence' section to the patrol system prompt with
concrete examples, matching the existing 'Authoring Impact' section.
3. Add test verifying trust scaffolding guidance is in the system prompt.
Addresses checklist L48-50 (trust scaffolding).
Alerts that share a resource ancestor (e.g., ZFS pool + device within
that pool) are now grouped on the alerts overview page. The primary
alert (highest severity, most recent) is shown by default with a
'+N related' toggle that expands to reveal the related alerts.
Grouping uses resourceId nesting: alerts with 3+ path segments are
sub-components and share the group key of their parent (first 2
segments). Alerts on different subsystems remain separate.
Single-alert groups render exactly as before — no visual change when
there's nothing to group.
1. Add Impact field to findingView struct — was completely missing from
the unified findings API response serialization.
2. Expand generateImpact() to cover all ~25 alert types the system
produces. Previously only 7 had curated impact text.
3. Fix naming mismatches: poweredOff/nodeOffline never matched actual
alert types powered-off/host-offline.
4. Expand TypeCategoryMap with all missing alert types.
Existing alerts kept their stale message from when they first fired.
Now the message is regenerated on every evaluation cycle so label
fixes and wording improvements take effect immediately.
getAlertStyles now accepts an optional nodeMatch parameter. Node table
rows pass their node name so storage/topology/disk alerts (which have
a different resourceId than the node) correctly highlight the affected
node row. Previously ZFS pool errors on minipc were invisible on the
overview page.
Alert resource names are now clickable links that navigate to the
appropriate platform overview page (Proxmox, Docker, Machines, etc.)
based on the alert's resource type. Lets users jump directly from an
alert to the affected resource for investigation.
