Commit graph

3311 commits

Author SHA1 Message Date
rcourtman
27cd862df3 Make global logging reconfiguration race-safe 2026-07-04 11:52:20 +01:00
rcourtman
313648b2f1 feat(mcp): add additive fleet-context filtering
get_fleet_context returned the whole registry with no way to narrow it,
so a large fleet produced a payload that exceeded the agent harness's
50KB cap and forced agents to receive or page through healthy resources
that are pure noise from a triage standpoint.

Add optional additive query-param filters — hasFindings, severity,
technology, resourceType — that compose by intersection. All optional
(omitting every filter returns the full fleet, backward compatible);
unknown/unmatched values return 200 with resources: [] (a valid triage
answer, not an error). The hasFindings=true filter is the headline
triage case: show me only what needs attention.

The MCP adapter layer had no GET query-param transport — ProjectCapabilityCall
short-circuited GET after path substitution and dropped all other args.
Close that gap generically: ProjectedCall gains a Query url.Values field
populated from leftover non-path public args for GET/DELETE, and
BuildCapabilityHTTPRequest encodes it onto the request URL. Benefits any
future GET capability with filter args, not just fleet-context. The
resources/list adapter calls fleet-context with empty args and continues
to receive the unfiltered fleet.

The manifest now declares the filters via inputSchema so they are
discoverable through the capability surface the same way add_node's
params are.

- ProjectedCall.Query + GET query-param forwarding (projection.go, http.go)
- fleetContextFilter parsing/matching in HandleFleetContext
- fleetContextInputSchema + InputSchema on the capability
- Regenerated cmd/pulse-mcp/README.md via generate-pulse-intelligence-docs
- Contract updates: api-contracts, ai-runtime, agent-lifecycle, storage-recovery
- Tests: adapter query-forwarding unit tests, 7 filter handler tests
  (hasFindings/severity/technology/resourceType/no-filter/unknown/compose),
  contract pinning inputSchema + end-to-end query forwarding
2026-07-03 18:01:05 +01:00
rcourtman
20c2f27057 feat(mcp): add list_resource_capabilities tool
Agents had no way to discover which governed capabilities a resource
advertises before calling plan_action. The data existed (Resource.Capabilities
with full param schemas) but was only surfaced as count-limited prose
facts inside get_resource_context, which deliberately omits parameter
schemas. This forced agents to guess capabilityName and params.

Add a dedicated structured surface: GET /api/agent/resource-capabilities/{id}
returns the canonical ResourceCapability list (name, type, approval level,
platform, full parameter schemas) for a single resource. A resource with
no advertised capabilities returns 200 with an empty array, the signal
to skip plan_action.

Internal plumbing already existed (registry.Get + Resource.Capabilities,
the same path plan_action validates against); this is the wiring work at
the canonical agent-surface layer. The tools/call dispatch is fully
generic so no mcp.go changes were needed.

Companion to get_resource_context: that tool remains the human-readable
prose summary; this is the structured schema surface for action planning.

- Manifest capability, name/path constants, output schema helper
- Handler mirroring HandleResourceContext error/scope patterns
- Route registration behind monitoring:read
- Activity telemetry mapping to resource_context class
- Regenerated cmd/pulse-mcp/README.md via generate-pulse-intelligence-docs
- Contract updates: api-contracts, ai-runtime, agent-lifecycle, storage-recovery
- Tests: manifest ownership/addressing/table-test, handler happy/empty/404/405,
  contract pinning capability presence/scope/wire-shape
2026-07-03 17:07:41 +01:00
rcourtman
b1fdefdd50 Normalize Docker container CPU capacity
Refs #1293
2026-07-03 11:51:08 +01:00
rcourtman
806fd7409c Show cluster labels in Proxmox node names
Refs #1475
2026-07-03 10:52:18 +01:00
rcourtman
c1d8cd8f11 Prefer agent temperatures over SSH fallback
Refs #1473

Refs #1509
2026-07-03 10:43:28 +01:00
rcourtman
8b1781b013 Ground Patrol run summaries in accepted findings
Refs #1512
2026-07-03 10:15:55 +01:00
rcourtman
875e414b4b Align resource staleness with poll cadence
Refs #1468
2026-07-03 09:05:12 +01:00
rcourtman
6344d17fd1 Allow local Pulse HTTP agent URLs
Refs #1505
2026-07-03 00:24:40 +01:00
rcourtman
282bee7a28 Scope removed host-agent blocks by identity
Refs #1495
2026-07-03 00:02:00 +01:00
rcourtman
477c1e3da6 Alert on expected QEMU guest agent outages
Refs #1508
2026-07-02 23:52:02 +01:00
rcourtman
dcf541784d Bound server state broadcast memory
Refs #1442
2026-07-02 23:27:49 +01:00
rcourtman
dd357c1ef3 Fix hosted relay onboarding test setup
Some checks are pending
Build and Test / Secret Scan (push) Waiting to run
Build and Test / Frontend & Backend (push) Waiting to run
- seed connected relay status before building mobile onboarding payloads
2026-07-02 22:34:37 +01:00
rcourtman
97b9369333 Guard infrastructure import plan approvals
- add candidate import approval and preview gate for node onboarding
- document metrics, availability, and proxy role behavior
- lock config auth reads and preserve storage metrics fallback
2026-07-02 21:59:26 +01:00
rcourtman
6c9b4f8e8d Harden GA telemetry disclosure 2026-07-02 11:50:03 +01:00
rcourtman
df67c01a64 Fix mobile relay runtime startup 2026-07-01 10:38:57 +01:00
rcourtman
c0ac0762da Fix security scan findings
Harden proxy-auth admin role checks, metrics listener exposure, Teams webhook escaping, and dependency lockfiles.
2026-07-01 09:55:35 +01:00
rcourtman
9dc4386318 Attach mock availability checks to services 2026-06-30 21:16:16 +01:00
rcourtman
c759c9a77a Gate mobile pairing codes on relay readiness 2026-06-30 17:12:55 +01:00
rcourtman
229867636d Add resource priority to Patrol queue 2026-06-30 10:37:54 +01:00
rcourtman
7c75b13d2c Carry host power sensor readings 2026-06-30 10:02:49 +01:00
rcourtman
d393ccf310 Add typed NVIDIA GPU stats 2026-06-30 09:43:33 +01:00
rcourtman
fe3c0f3ee3 Collect NVIDIA GPU temperatures 2026-06-30 09:11:18 +01:00
rcourtman
d8de95b91f Show custom provider model catalogs 2026-06-29 23:19:13 +01:00
rcourtman
c8dc45577c Emit TrueNAS pool status incidents 2026-06-29 22:54:19 +01:00
rcourtman
4eac3b535c Accept ping alias for availability targets 2026-06-29 22:43:13 +01:00
rcourtman
597c369a63 Add alert delivery diagnosis endpoint 2026-06-29 22:27:01 +01:00
rcourtman
a801f456d6 Improve Patrol SMART disk evidence 2026-06-29 18:17:24 +01:00
rcourtman
6b7d0b45f9 Add agent fleet diagnostics endpoint 2026-06-29 18:03:46 +01:00
rcourtman
73208a2863 Document state summary integration contract 2026-06-29 17:39:11 +01:00
rcourtman
e103278885 Capture v6 GA release target 2026-06-29 15:26:42 +01:00
rcourtman
1510a73e1f Guard guest disk alert migration identity
Add regression coverage ensuring per-disk guest alert migration cannot remap an active alert from one guest onto another. Refs #1511
2026-06-28 21:41:07 +01:00
rcourtman
fe32dfffb8 Pin update telemetry privacy boundary
Add regression coverage that raw updater failure text cannot escape through the coarse failure category.
2026-06-28 15:40:33 +01:00
rcourtman
d6a21eba98 Add update funnel telemetry
Record anonymous 30-day update attempts, successes, failures, and coarse failure categories from local update history.
2026-06-28 15:19:39 +01:00
rcourtman
85c46cfcd1 Track alert save workers during shutdown 2026-06-28 01:39:12 +01:00
rcourtman
4fc56162f6 Harden hostagent SMART no-device test
Some checks are pending
Build and Test / Secret Scan (push) Waiting to run
Build and Test / Frontend & Backend (push) Waiting to run
2026-06-28 01:01:20 +01:00
rcourtman
8fff04eac5 fix(ai): query capacity forecast history under metrics-target ID not canonical ID
The deterministic capacity forecast queried utilization history using the
canonical resource ID (storage-<hash>), but the monitoring layer records
metrics under the metrics-target/source ID (MetricsTarget.ResourceID). For
most storage sources these two IDs diverge, so the forecast query always
found nothing and the feature stayed dormant even for pools with full
ingestion history (Proxmox, TrueNAS, PBS, agent-backed storage).

This splits the two ID roles in the precompute path: a new metricsID field
on patrolStoragePoolRow / patrolPrecomputeStorageSource carries the
metrics-target ID (resolved from SourceID() in the readState branch, or
MetricsTarget.ResourceID in the unified-resource branch) for history
queries, while the canonical id is still used for stamp-matching findings.

Adds a test proving the forecast fires when history exists only under the
metrics-target ID and that the resulting forecast resourceID stays
canonical so StampCapacityForecasts can still match it.

Contract (ai-runtime): the forecast must query history under the
metrics-target ID (StoragePoolView.SourceID()), not the canonical ID; the
forecast's own resourceID stays canonical for stamp-matching.

Refs lane: protection-posture-attention-queue
2026-06-27 22:18:09 +01:00
rcourtman
d45715d083 feat(ai): surface deterministic capacity forecasts as first-class finding signals
Capacity forecasts (days-to-full, current usage, trend) were previously
computed deterministically but only fed into LLM prompt text. They never
reached the finding as structured data, so the frontend could not render a
verified urgency signal and the model's speculation was the only thing the
operator saw.

This persists CapacityForecast on Finding (marshal-mirror pair) and stamps
it post-analysis via a service join (StampCapacityForecasts). The forecast
filter now also keeps stable-high (>=80%) pools so the deterministic "no
fill trend" reading wins over model speculation, and fixes isQuiet wrongly
treating stable (-1) trends as filling.

Frontend maps the forecast through the UnifiedFinding view model and renders
a deterministic urgency line (Filling up / Stable / days-to-full · % used)
in the expanded finding detail.

Note: forecasts only populate for resources whose usage is ingested into
metrics history (Proxmox/Ceph storage, nodes, guests). Agent-host storage
(unraid pools) is not yet ingested as a time-series and remains a follow-up
to activate the feature for those pools.

Refs lane: protection-posture-attention-queue
2026-06-27 22:03:28 +01:00
rcourtman
f676dea097 fix(monitoring): prefer agent source for physical disk SMART metrics
When a Proxmox node has an agent installed, SMART/temperature metrics are
written by the host agent under the agent's disk source ID. But
BuildMetricsTarget preferred the Proxmox source first, and Proxmox API
does not expose detailed SMART data. The mismatch meant the frontend
queried a metrics key that nothing writes, so disk metrics appeared empty.

Move the Agent source check ahead of Proxmox/TrueNAS for
ResourceTypePhysicalDisk, mirroring the existing pattern for
ResourceTypeAgent where agent source already wins over platform sources.

When a disk has a serial number, it is used as the canonical metric ID
regardless of source (PreferredPhysicalDiskMetricID), so the reorder
only affects disks without serials — exactly the case where source
priority determines the key.

Refs #1487
2026-06-27 20:22:04 +01:00
rcourtman
028e8c8df2 fix(unifiedresources): add space reclamation and retention for all append-only tables
The unified_resources.db grew without bound (2GB reported) because:

1. No VACUUM: DELETE freed rows internally but never shrank the file.
   Added auto_vacuum(INCREMENTAL) to the DSN for new databases, plus a
   one-time migrateAutoVacuum() that converts existing databases.
   reclaimFreePages() now runs after each prune cycle to return freed
   pages to the OS via PRAGMA incremental_vacuum.

2. Missing retention: action_lifecycle_events, export_audits, and
   loop_reports had no retention at all. Added 90-day retention for
   lifecycle/export audits and 30-day for loop_reports, matching the
   existing action_audits/resource_changes cadence.

3. Slow cleanup cadence: the retention loop ran every 6h and never on
   startup. Reduced to hourly and added an initial prune 30s after
   startup so a restart with a bloated DB starts recovering immediately.

Mirrors the proven pattern from metrics.db (auto_vacuum INCREMENTAL +
incremental_vacuum + WAL checkpoint).

Refs #1496
2026-06-27 18:38:56 +01:00
rcourtman
d6aed650b1 fix(docker): stop manual update check from looping on ack failure
getDockerCommandPayload returned dispatched commands on every report
fetch, causing the agent to re-execute check-updates on every poll
cycle. When the ack also failed, the report was buffered and retried,
creating an infinite loop.

- Only return command payload on the queued->dispatched transition;
  subsequent fetches return nil (agent already received it).
- Don't propagate ack errors from handleCheckUpdatesCommand; the report
  was delivered and check-updates is fire-and-forget. Command expires
  if ack never succeeds.

Refs #1504
2026-06-27 18:26:23 +01:00
rcourtman
b5ddbd5606 fix(monitoring): use MemAvailable instead of MemTotal-MemFree for Linux guest/node memory
When the Proxmox API's meminfo/status payload omits Available, Buffers,
and Cached (common for QEMU guest-agent and node status responses), the
code derived 'available' from Free alone — producing used = Total-Free
which counts reclaimable page cache as used memory (e.g. 94% instead of
the correct 76%).

Guest path (deriveGuestMemInfoAvailable): return 0 when cache metrics
are completely missing and only Free is available, so resolveGuestStatusMemory
tries better sources: guest-agent /proc/meminfo file-read (returns
MemAvailable), RRD memavailable, or the linked host agent.

Node path (resolveNodeMemory): always try RRD memavailable when cache
metrics are missing, not only when effectiveAvailable == 0 — previously
a non-zero Free value blocked the RRD fallback.

Refs #1501
2026-06-27 17:41:00 +01:00
rcourtman
22ed97f562 fix(email): eliminate data race on provider username resolution
sendViaProviderWithAddresses mutated the shared e.config.Username for
provider-specific defaults (SendGrid, Postmark, SparkPost, Resend).
If concurrent goroutines sent email simultaneously, this was a data
race on the config struct.

Move the resolution into negotiateAuth via a local variable
(resolveProviderUsername helper) so the shared config is never mutated.
2026-06-27 17:36:51 +01:00
rcourtman
28d2413c71 fix(hostagent): clear stale Unraid sync action when resync position is zero
Unraid's mdResyncAction field can retain its last value (e.g. "check")
after a parity check is canceled, causing Pulse to report a stale sync
action indefinitely. The mdResync/mdResyncPos field is the authoritative
indicator: it drops to 0 when no resync is running. Gate SyncAction on
a non-zero position so the alert clears once the sync actually stops.

Refs #1485
2026-06-27 17:08:02 +01:00
rcourtman
5bc209d919 fix(alerts): deduplicate flapping history entries on load and periodic cleanup
Add deduplicateHistory() to HistoryManager that collapses consecutive
same-alert entries within a 5-minute window. The flapping/re-fire bugs
(lifecycle and stateful paths) created many duplicate history entries
before the cooldown fixes were deployed. This cleanup runs on startup
after loadHistory and periodically alongside cleanOldEntries, so both
existing noise and any future edge-case duplicates are handled.

On this install: 332 entries → 130 (60% reduction).
2026-06-27 16:36:29 +01:00
rcourtman
125959e4e0 fix(dev): login endpoint respects admin bypass mode
handleLogin rejected all credentials when ALLOW_ADMIN_BYPASS=1 because
it validated against config AuthUser/AuthPass directly without checking
the bypass flag. This made dev-mode browser testing impossible after a
backend restart with bypass env vars — the API middleware accepted all
requests but the login page could never obtain a session cookie.

When bypass is enabled, accept any credentials and create a session as
'admin'.
2026-06-27 16:09:23 +01:00
rcourtman
fa7e4711e7 fix(alerts): re-fire cooldown for stateful alerts to prevent duplicate history
Stateful alerts (ZFS pool/device, storage topology) were creating
duplicate history entries every poll cycle because SyncStorageAlertsForInstance
clears alerts between evaluations, making the existing-alert check miss.
Add the same 5-minute re-fire cooldown used in the lifecycle path: when a
stateful alert re-fires within 5 minutes of resolution, reactivate the
original (preserving StartTime) and update the existing history entry's
LastSeen instead of appending a new entry.
2026-06-27 15:58:22 +01:00
rcourtman
40ad4178ba fix(alerts): prevent duplicate history entries for flapping alerts
When an alert fires, resolves, and re-fires within 5 minutes (the
recently-resolved retention window), the previous implementation created
a new history entry for each fire cycle. For a flapping connection like
an unreachable Proxmox node, this produced hundreds of duplicate entries
in a single night — 302 identical 'pi unreachable' alerts in one case.

The fix checks the recently-resolved map before creating a new history
entry. If the alert was resolved within the cooldown window, it
reactivates the original alert (preserving StartTime) and updates the
existing history entry's LastSeen instead of appending a new one.
2026-06-27 15:38:13 +01:00
rcourtman
e197fa3db5 Handle JSON marshal error in agentic question tool response
Unchecked json.Marshal error could silently return empty string instead
of surfacing the encoding failure to the caller.
2026-06-27 15:28:32 +01:00
rcourtman
48cf663237 Strengthen evidence and impact guidance in patrol finding tool
The patrol_report_finding tool descriptions for evidence, impact, and
recommendation were too permissive, causing the LLM to frequently omit
them. Future patrol runs will produce better-scaffolded findings.

1. Rewrite tool descriptions to emphasize evidence as a trust anchor
   that should always be included, and impact as expected whenever the
   data supports it.
2. Add 'Authoring Evidence' section to the patrol system prompt with
   concrete examples, matching the existing 'Authoring Impact' section.
3. Add test verifying trust scaffolding guidance is in the system prompt.

Addresses checklist L48-50 (trust scaffolding).
2026-06-27 14:53:48 +01:00