Fixes#1640. Three defects around POST /api/ai/patrol/readiness on slow
local hardware behind a reverse proxy:
1. The handler ran up to four sequential provider calls (~45s and more on
slow Ollama boxes) while writing nothing to the response, so any
intermediary with a ~30s read timeout severed the request mid-run. The
handler now commits headers up front and streams flushed newline
keepalives every 10s while the evaluation runs, then appends the normal
JSON payload. Leading newlines are insignificant JSON whitespace, so
existing clients parse the response unchanged.
2. A severed connection cancels the request context, and
patrolRuntimeFailureFromError classified the resulting context.Canceled
as a generic "Provider analysis error", blaming the provider and model
for an infrastructure event. Mid-run cancellation is now classified as
the new "interrupted" cause: the overall status and every unfinished
dimension and autonomy mode report not assessed, per-scenario evidence
completed before the interruption is preserved in the returned result,
and the readiness cache keeps the last completed evaluation.
context.DeadlineExceeded keeps its provider-path timeout classification.
3. createAPIErrorFromResponse pre-seeded the error message with the raw
response body, making its non-JSON guard dead code, so full HTML proxy
error pages became Error.message and were rendered into the readiness
result boxes. Non-JSON bodies now surface only when they are short
plain text; anything with markup or excessive length collapses to a
generic status-derived message.
Regression tests: internal/ai/issue1640_readiness_cancellation_test.go,
internal/api/issue1640_readiness_transport_test.go, and
frontend-modern/src/utils/__tests__/apiClient.issue1640.test.ts, all
registered in the subsystem verification registry.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Found while triaging #1643 and #1637: both fields were stored via
system.json and accepted, validated, and echoed by the settings API,
but nothing ever consumed them. The unattended update schedule is owned
entirely by the systemd timer install.sh renders, and no UI control ever
set the fields, so the API persisted a schedule preference that could
never take effect.
Remove the fields from config.Config, SystemSettings, the settings
handlers, and the frontend config type, along with the interval
validation and the .env AUTO_UPDATE_CHECK_INTERVAL rewrite (a legacy
line is now preserved verbatim). Legacy clients that still send the
keys get them silently ignored instead of validated, and a system.json
written before the removal still loads cleanly - both behaviors are
pinned by new tests. User docs no longer describe the phantom schedule
settings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Disabling one metric for one resource required knowing to type -1 into
the override editor (the alert engine treats trigger <= 0 as disabled),
and the global-defaults row could only re-enable a disabled metric by
click, not disable one. Add an On/Off badge next to the threshold input
in the override row editor (desktop and mobile), in the global-defaults
row (desktop and mobile), and in the bulk edit dialog, mirroring the
existing Backup/Snapshot StatusBadge pattern in the same rows. The
toggle writes -1; toggling back on restores the inherit-default state
in override editors and the metric's enabled default in global
defaults. In the desktop row editor the badge swallows mousedown so it
doesn't blur the input, which would save and close the editor before
the click lands.
Also fix the off-value inconsistency found while triaging #1642: the
thresholds help banner told users to type 0, which the edit-state off
detection (=== -1) never recognises. Standardize the advertised
disable value on -1, matching docs/FAQ.md and the engine's actual off
sentinel; read-mode cells keep treating <= 0 as Off so overrides saved
as 0 under the old banner still display correctly.
Closes#1642.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contract-Neutral: UI-only off toggle over existing -1 threshold semantics; no alert truth or identity change
OIDC providers get a server-generated UUID provider ID, so the IdP
redirect URI (/api/oidc/<id>/callback) was undiscoverable in the UI even
though the API already returns it as oidcCallbackUrl. Surface it as a
copyable Callback / Redirect URL on the provider card and in the OIDC
edit modal, mirroring the SAML SP metadata display. Found while triaging
issue #1635.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contract-Neutral: UI-only display of existing oidcCallbackUrl API field in SSO settings; no public contract delta
The Docker host, Docker container, Proxmox node, Kubernetes cluster and
node, TrueNAS system and app, and vSphere host tables rendered their
CPU/memory/disk bars without a thresholds prop, so coloring fell through
to the hardcoded METRIC_THRESHOLDS display defaults (memory 75/85). A
Docker host at 84% memory showed a red bar even when the operator had
raised its alert thresholds — the alert runtime resolved the override
correctly and stayed quiet while the row shouted.
Each table now resolves display thresholds through the alerts activation
store with its runtime scope (agent, docker, node, kubernetes, truenas,
vmware) and the resource's override identity candidates, mirroring the
AgentsMachinesTable pattern. The display resolver gains the kubernetes /
truenas / vmware scopes backed by kubernetesDefaults / truenasDefaults /
vmwareDefaults with factory fallbacks, and the unified platform
candidate chain is exported from alertOverridesModel so the tables bind
overrides to the same rows buildProjectedOverrides indexes.
The alertOverridesModel chunk folded into sharedPlatformPage now that
platform tables import it; bundle baseline regenerated within budget.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
c2c51019d grew the chunk to 7.08 kB gzip, past the 15% headroom over
the 5.94 kB baseline, which has failed the bundle budget on every main
push since. The growth is the intentional probe assignment and source
attribution feature, so re-baseline at the measured size.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Host agents key their identity on the machine-derived agent ID, so MSP
template deployments that clone /etc/machine-id fold two physical
machines at different sites into one host row whose reports overwrite
each other (hostname, report IP, and interfaces flapping between
sites), silently poisoning node-agent linking.
Mirror the Docker host identity-collapse doctrine (#1584) for host
report ingest: track hostname and report-IP revisits per resolved
agent identity inside the monitoring-owned flap window, publish an
active conflict as models.Host.IdentityConflict through unified
resources, and warn on the Machines page. The report IP is tracked
alongside the hostname because template fleets often reuse hostnames
across sites (pve01 at two customers), leaving the address as the only
field that betrays the clone. A one-time hostname rename never
revisits and is not flagged; the conflict clears on its own once only
one machine keeps reporting for the window.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The availability check editor gains a "Run from" control listing
connected agent hosts, locked behind the external_probe entitlement
with the standard feature-gate affordance - visible for
discoverability, disabled without Pro, and never in the way of
saving local checks. Statuses show a "via agent" chip when the
latest observation came from a probe, stale probe reports render as
indeterminate, and clearing an assignment serializes an explicit
empty probeAgentId so the backend merge-decode clears it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add availability_probe_targets and availability_probe_agents to the
telemetry ping - counts only, no agent names or addresses - with the
disclosure table updated in both privacy doc copies. Document the
feature in the availability-checks configuration guide and the
unified agent guide, including the ICMP capability caveat for
containerized probes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add external_probe to the licensing feature enumeration, Pro tier
membership, self-hosted feature catalog, feature map, upgrade matrix
and pricing handoff, with the entitlement contract goldens updated.
The capability is served by the community binary - the entitlement
alone gates it, like relay. No existing free functionality changes:
local availability checks stay ungated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolves the two open Dependabot alerts for GHSA-r28c-9q8g-f849
(source-map auto-loading path traversal, patched in 8.5.18). postcss
is a build-time dependency, so no shipped runtime code was affected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Issue #1621: saving only "intentionally offline" left the SQLite
auto_remediation_policy_json column NULL, and the read path only
normalized the policy inside the non-NULL branch, so the API served
"capabilityNames": null. The section's dirty-state memo spread that
value during render and crashed the expanded row into the route error
boundary. Normalize the policy unconditionally after the scan so the
wire always carries [], type capabilityNames as nullable in the TS API
surface, and guard the spread in the component.
Issue #1622: the inline table-row presentation gated the entire
operator-overrides section on the resource exposing an auto-authorizable
capability, so stopped Docker containers (which only expose start,
never auto-authorized) lost intentionally-offline and
never-auto-remediate exactly when they matter most. Render the section
whenever the resource has an id; the automatic-actions block already
self-gates on eligible capabilities. Also add min-w-0 to the section's
flex text columns and overflow-x-clip to the inline row content box so
the long never-auto-remediate copy wraps instead of escaping the row
border below the lg breakpoint.
Regression coverage: issue-named Go test for the NULL-column read path
and a render test feeding the section the real pre-fix wire payload
with capabilityNames: null.
Fixes#1621Fixes#1622
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The readiness advisor failed capable local models for adapter and probe
defects rather than model incapability (#1624 Ollama, #1614 llama.cpp):
- Send num_ctx sized to the haystack fixtures (clamped to the model's
trained window) so Ollama no longer truncates ~25KB prompts at its
4096-token server default; the trained-window guard alone passed while
the runtime request was being truncated.
- Forward an explicitly pinned temperature 0 instead of dropping it to
Ollama's 0.8 default against a nonce-exact validator (ChatRequest gains
TemperatureSet; Ollama options temperature is now a pointer).
- Raise the probe generation cap from 256 to 2048 tokens so qwen3-style
<think> reasoning cannot exhaust the budget before the tool call, and
surface the provider done_reason when validation fails.
- Synthesise tool-call IDs in the OpenAI-compatible adapter (streaming
finalizer and buffered path) when the server omits them, as llama.cpp
commonly does, mirroring the Ollama adapter instead of failing tool
protocol 0/3 on transport shape.
- Probe with the Patrol loop's 60s stream stall allowance instead of the
12s chat default (chat.PatrolProviderStreamIdleTimeout is now exported).
- Stop discarding probe and validator errors: log them, carry them in a
new PatrolModelReadinessResult.Details field surfaced through the API
snapshot and Settings UI, and keep a transport-level probe failure's
specific diagnosis instead of overwriting it with the generic
capability wording.
Builds on 4a2335ce7, which already reclassifies protocol failure as
"provider connected; Patrol capability not verified".
Fixes#1624Fixes#1614
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
These are pure modules that already had a spec but had never been through a
branch-coverage pass, so their cold arms had never been measured. Counts are
uncovered branches, module scoped, existing spec versus existing plus new.
- resourceTypeCompat 12 -> 0, the legacy type-alias paths: unrecognised type
strings, casing variants and the fallback identity mapping.
- summaryCardInteraction 8 -> 0 and diskDetailPresentation 6 -> 0.
- commercialBillingModel: asUnlimitedLimit, buildHostedCommercialPlanModel and
buildHostedCommercialUsageModel had never been called by any test. All three
are now covered across unlimited and numeric limits and usage at, below and
above the limit.
- alertDetailPresentation 4 -> 0, resourceTypePresentation 2 -> 0,
unifiedResourceTableStateModel 4 -> 0, assistantSlashCommands 4 -> 1.
No source file is modified. Adversarial review returned no rejects but found
the heaviest padding of this run, around twenty-five near-duplicate cases plus
one assertion that mirrored the source's own date formatting. Every one was
removed or replaced with a concrete literal, and each module's coverage was
re-measured as identical afterwards, proving the removed cases carried nothing.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
A second pass over modules whose own specs left only a handful of cold arms.
Counts are uncovered branches, module scoped, existing specs versus existing
plus new.
- diskResourceUtils 6 -> 0, the densest remaining gap in the frontend by
ratio: missing capacity fields, absent SMART evidence and the no-candidate
node arms.
- proxmoxPageModel 6 -> 0, availabilitySettingsModel 5 -> 0 and
standalonePageModel 5 -> 0, all empty-collection guards, absent optional
metadata and validation rejection arms.
- connectionsTableModel 8 -> 2 across eleven existing specs.
truenasPageModel was DISCARDED rather than committed: its remaining eight arms
measured no coverage gain and the agent correctly reported seven of them as
unreachable by construction, so there was nothing honest to land.
No source file is modified. Adversarial review returned no rejects and flagged
eight near-duplicate cases across four files; all eight were removed and the
module coverage re-measured as unchanged, proving they carried nothing.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
Seven modules carrying the biggest measured branch residue in the frontend.
Counts are uncovered branches, module scoped, existing specs versus existing
plus new, re-measured against current main after the TrueNAS topology
discriminator change landed under them.
- resourceDetailDrawerTrueNASModel 28 -> 3, and hasTrueNASDetailSections,
which no test had ever called, is now covered on both verdicts.
- agentMachineTableModel 25 -> 5, missing metrics, absent agent metadata,
blank status strings and the comparator ties.
- workloadSelectors 18 -> 1, comparator ties on every sort key so the
tiebreak arm runs, plus the blank-search and undefined-metric arms.
- resourceDetailDiscoveryModel 18 -> 0, resources with no discovery metadata
and the label and confidence fallbacks.
- proxmoxBackupRecoveryModel 17 -> 0, guests with no backups, unparseable
timestamps and the recency classifier boundaries.
- api/truenas 15 -> 0, real request building and response handling arms.
- webInterfaceUrlFieldModel covers its three never-called functions.
No source file is modified. Adversarial review returned no rejects and flagged
six near-duplicate cases plus two test titles that overclaimed what their
bodies checked. All six were deleted and the coverage re-measured as identical,
proving they carried nothing; both titles were corrected without touching an
assertion.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
Seven modules from the current source drop still had a measured branch
residue their own specs never reached. Counts are uncovered branches,
module scoped, existing specs versus existing plus new.
- resourceDetailMappers 13 -> 0, absent nested metadata, empty arrays and
the unknown-discriminator default arm.
- infrastructureAgentUpdateCommandsModel 9 -> 0 on top of three prior
coverage files.
- stackedMemoryBarModel 9 -> 1, zero and missing totals, over-total and
negative segments, the clamping boundaries.
- api/vmware 9 -> 0 and api/nodes 8 -> 0, request building and response
handling arms rather than mocked-fetch echoes. api/nodes also covers
createHostAgentInstallToken, which had never been called.
- nodeModalModel 8 -> 0, blank fields and the validation rejection arms.
- utils/workloads 8 -> 0.
No source file is modified. Adversarial review returned no rejects and
flagged ten near-duplicate cases across the vmware and workloads files;
nine were deleted and the module coverage re-measured as identical, proving
they carried nothing. The tenth was restored because re-measuring showed it
did carry a branch.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
c77571685 added a per-instance org_switched handler to
createNonSuspendingQuery that only called reset(). reset() bumps the
request generation and writes value, loading, error and resolvedOnce, none
of which the source effect tracks, so the effect cannot re-run and the
handler never issued a new request. A consumer with a constant source and
no polling therefore emptied on an org switch and stayed empty until it
remounted. Two such consumers exist, connectionsSnapshot in
useWorkloadsState and patrol-status in usePatrolIntelligenceState; polling
consumers self-healed on their next tick, which is why this was easy to
miss.
The handler now refetches after resetting, matching the clear-and-refetch
shape useWorkloads, useUnifiedResources and useStorageSummaryCharts already
use. Stale pre-switch responses are still discarded by request generation.
The existing late-resolution test captured a single resolver, which the new
refetch would overwrite, so it now keeps every resolver and settles the
pre-switch one specifically. That keeps it aimed at the stale-response
guarantee it names rather than silently testing the newest request.
4815a5c80 split PBSDiagnostic into canonical fields plus a nested probe
object, moving probe failure text to pbs.probe.error and adding
pbs.stateReason. sanitizeDiagnosticsData spreads the PBS entry and redacts
only host, name, id and the top-level error, so transport strings such as
"dial tcp 192.168.1.20:8007: connect: connection refused" stopped being
IP-redacted. That text used to live in pbs.error and was scrubbed; it now
rode unredacted into the sanitized bundle operators attach to support
requests.
The new fields are spread conditionally so an entry without them keeps its
exact previous shape, which leaves the existing strict-equality coverage
untouched.
Both proofs assert on the serialized bundle rather than on the sanitizer's
source, because the failure mode is a payload field added later that the
sanitizer never learned about. Verified to fail with the redaction removed,
reporting the raw address inside the exported JSON.
599c8e634 changed the TrueNAS pool projection to publish the ZFS data vdev
layout ("mirror", "raidz2", "stripe") as Storage.Topology. Topology is the
closed discriminator vocabulary every storage consumer keys on, and TrueNAS
pools travel as ResourceTypeStorage, so the frontend identifies them solely
by storage.topology === 'pool'. On any appliance whose pools report data
vdevs, every pool therefore fell out of the TrueNAS page, its datasets
orphaned, and per-system pool counts read zero.
Nothing caught it because internal/truenas/fixtures.go carries no VDevs, so
the projection still returned "pool" for every fixture while the producer
side pinned the new "mirror" value. Both suites stayed green and disagreed
with each other.
Pools now publish Topology "pool" again, with the layout in the new
Storage.VDevLayout field. poolVDevLayout returns an empty string when the
native report carries no data vdevs so callers fall back rather than
inventing a layout, and the presentation surfaces that previously read the
layout out of topology now prefer vdevLayout and fall back to topology.
Alert typing was unaffected (unified_eval.go already routes non-dataset
storage to truenas-pool through its default arm); incident metadata carried
the layout string and now carries the discriminator again.
Regression coverage pins the boundary from both sides, driving real vdev
layouts rather than the vdev-less fixtures:
- TestPoolTopologyStaysDiscriminatorAcrossVDevLayouts (producer, five layouts)
- TestStorageTopologyAndVDevLayoutAreDistinctIdentityFields (clone and wire)
- pool identity boundary cases in truenasPageModel.test.ts (consumer)
- topology label cases in storageAdapters.test.ts and the TrueNAS drawer model
A machine monitored only by the Pulse agent was listed under Alerts,
Thresholds, Virtualization Hosts as well as under Machines, because the
section was fed by every resource of type `agent` rather than by
provider-owned virtualization nodes.
The two rows do not share an override identity. The Machines row resolves
through `hostOverrideIdCandidates`, which prefers the agent-derived id, while
the Virtualization Hosts row reads and writes `overridesMap.get(node.id)` on
the bare resource id. A custom threshold saved from the Virtualization Hosts
row therefore landed under a key agent alerting never reads, so the machine
kept alerting on the global default with no visible reason.
The section now takes `virtualizationHostResources`, which excludes standalone
Pulse-agent resources via the existing `isPulseAgentPlatformResource`
classifier already used by the standalone page, platform navigation and the
resource drawer. Provider-owned nodes are unaffected, since they carry
provider-owner platform evidence. Those machines lose nothing: they keep their
Machines row, which is the one alerting honours.
Reported via support by a customer whose cloud host kept firing at the 85%
global agent memory limit despite a 100% override saved on the wrong row.
Adds the subsystem contract deltas and verification artifacts for the
preceding wearout and physical-disk-node fixes, which landed without them.
The contracts now state the rule rather than leaving it implicit in each
caller. Wearout has two absent states and one real zero, -1 is the unreported
sentinel, and 0 is evidence only from a device that reports endurance at all.
storagehealth.WearoutReported is the single authority, and alerting, the
unified risk projection, and the frontend all gate on it. Storage-recovery
records that disk-to-node association is evidence-directional, so a disk with
no Proxmox scope still matches on node name while a scoped disk stays inside
its instance.
Proofs cover the predicate itself, a spent SSD alerting where it previously
stayed silent, absent evidence staying quiet on both the unreported sentinel
and a rotational zero, the view returning the sentinel rather than the struct
zero value, and the presentation helpers agreeing with the server.
Verified: full frontend suite 1061 files / 19405 tests, canonical completion
guard, gofmt, go build ./..., and internal/alerts, internal/monitoring,
internal/storagehealth, internal/unifiedresources.
Physical disks reported by a host agent carry no Proxmox scope, so tightening
matchesPhysicalDiskNode to require an instance on both sides dropped every
agent-reported disk off the Proxmox node it lives on. The Physical Disks node
filter rendered no options at all, grouping lost the node, and metric target
resolution could not find it. Match on node name when the disk side has no
instance evidence, and keep rejecting a Proxmox-scoped disk against a node
outside that instance, so cluster isolation still holds in the direction that
motivated the change.
Wearout had two competing readings of 0. The presentation layer treated it as
no endurance remaining while the alert path treated it as absent and stayed
silent, so a disk could read critical in the UI and never alert. Two sources
also published 0 for a disk that reported nothing: PhysicalDiskView.Wearout
returned the struct zero value on a nil view, as did the nil-view branch of
physicalDiskFromReadStateView. Both now return the documented -1 sentinel.
storagehealth.WearoutReported is the single authority for whether a reading is
evidence at all: -1 is absent, 0 is real only from a device that reports
endurance, and rotational disks never do. Risk assessment, alerting, and the
frontend all gate on it rather than each reinventing the boundary.
Verified: full frontend suite 1061 files / 19402 tests, eslint, theme and
canonical-platform audits, tsc, go build ./..., and internal/alerts,
internal/monitoring, internal/storagehealth, internal/unifiedresources.
Preserve tenant-scoped metadata through partial URL updates and project stable URLs across runtime identities. Use a safe adjacent launch control across overview tables with desktop and mobile regression coverage.
Keep the configured cluster URL as primary and recover discovered member
endpoints asynchronously while it is healthy. Reset member reachability
evidence when the effective address changes.
Refs #1437 and #1493
