Hosts (PVE nodes and host agents) were entirely outside the discovery
fingerprint model. Two consequences, one root:
1. collectFingerprints only covered docker/lxc/vm/k8s, so a host never
appeared in GetChangedResources and automatic discovery never ran for
it. Only the per-resource manual trigger worked, which is exactly the
v6.0.5-rc.3 behaviour reported in #1479.
2. cleanupOrphanedData built its keep-set from the same four types, so
CleanupOrphanedDiscoveries swept every agent:* discovery as an orphan
on the next fingerprint cycle (default 5 minutes). Manually discovered
hosts silently reverted to undiscovered.
Fix: add GenerateHostFingerprint (identity, OS, kernel, arch, tags;
status excluded so online/offline flapping cannot trigger rediscovery),
fingerprint snap.Hosts under the canonical agent:<id>:<id> key, and keep
host/node discovery keys (canonical, hostname-alias, and legacy host:
prefix forms) out of orphan cleanup. Also purge the store's in-memory
cache when an orphaned discovery file is removed, so deletions are not
masked for the cache TTL.
The refreshSuggestedAvailabilityProbeFromState method existed but was
never called, so existing discovery records never received availability
probe suggestions. This adds backfillAvailabilitySuggestions which:
- Triggers from SetReadState (goroutine) and runDiscoveryLoop (ticker)
- Retries with exponential backoff when the state snapshot is empty,
waiting for the monitor to populate Proxmox data before proceeding
- Matches containers by VMID only (was VMID+Node, which failed in
clusters where the discovery targetID differs from the container's
node)
Also extends SuggestAvailabilityProbe with a hostname fallback: when
ServiceType is empty (deep scan not yet run), checks the discovery's
Hostname against webServiceDefaults/tcpServiceDefaults. This covers
containers like jellyfin, grafana, frigate, esphome, zigbee2mqtt,
homeassistant, mqtt, etc. in environments where background AI is
disabled.
Adds zigbee2mqtt and ntfy to webServiceDefaults.
Discovery already identifies services and their default ports. Now it
also suggests an availability probe configuration for each discovered
service with a known web interface (webServiceDefaults) or TCP service
(tcpServiceDefaults). The suggestion appears in the resource drawer as a
card with a 'Monitor availability' button. On approval, it calls the
existing POST /api/availability-targets API — one canonical system, no
second management surface.
Backend:
- New SuggestAvailabilityProbe() generates protocol/port/path from the
same webServiceDefaults map used for URL suggestions, with a
tcpServiceDefaults fallback for databases and message brokers
- New AvailabilityProbeSuggestion type on ResourceDiscovery
- Wired into both discovery paths (DiscoverResource + Docker background)
- Cached discoveries get the suggestion via refreshSuggestedAvailabilityProbe
on read, so existing data picks it up without re-discovery
Frontend:
- AvailabilityProbeSuggestionCard in GuestDrawerOverview with one-click
creation via AvailabilityTargetsAPI.create()
- Card hidden when the resource already has an availability facet
- Added 'https' to AvailabilityProbeProtocol type (backend already
supports it)
The metadata-only abstention is only reached when command scanning is already
enabled but the host agent returned no command output. The old reason text told
the user to 'Enable Pulse Commands' — a toggle that is provably already on when
this message appears — so the guidance was self-contradictory. The reason now
states commands are enabled and points at the actual gap: confirm the host
agent is connected and its API token has the agent:exec scope.
Discovery records for Proxmox guests (VM / system-container) are canonically
keyed by node name + VMID — the form the background fingerprint loop and the
Assistant prefetch already use. But the browser action path addresses guests by
the linked agent UUID (discoveryTarget.agentId, the action-authorization target),
and normalizeDiscoveryRequest was a no-op for guest types — so a UUID-targeted
trigger/lookup stored and read records under a second, divergent key. Result: the
resource drawer reported "Not discovered" for a guest the background loop had
already discovered, and duplicate records accumulated (node-keyed + UUID-keyed).
normalizeDiscoveryRequest now canonicalizes guest targets to the node name:
resolve a linked-agent-UUID target back to its hosting node (node->linked-agent
map, falling back to the agent host's hostname when it is a known node). Both
creation (DiscoverResource) and lookup (GetDiscoveryByResource) funnel through
this chokepoint, so every path converges on the node-name key; the caller already
aliases the original target, so pre-existing UUID-keyed records are still found
and consolidated onto the canonical key on the next run.
The agent UUID is unchanged as the action-authorization target — this governs
only the record key, so the discoveryTarget security contract is untouched.
- Test: TestService_ProxmoxGuestDiscoveryCanonicalizesToNodeKey.
- Live-verified on the dev instance: the Home Assistant LXC drawer readiness went
missing -> fresh (discoveryId system-container:delly:101).
formatSingleDiscovery (the local/full path, FormatForAIContext) reported a
resource's service, access, paths, and ports but not how old the discovery was —
so Ollama/local users lacked the recency signal cloud users just got. Emit a
"Last Discovered: <age>" header fact when the timestamp is known, completing
freshness parity across the cloud-safe and full paths. Omitted when unknown.
Test: TestFormatSingleDiscovery_IncludesFreshness (present when set, absent when
zero).
The pushed cloud-safe operational context told the model a resource's access
pattern, paths, and ports but not how OLD the discovery was — so the Assistant
could present a 2-week-old cached scan as current. For a monitoring assistant,
recency is the most important provenance signal.
FormatCloudSafeContext now appends "Last discovered: <age>" (via the existing
FormatDiscoveryAge helper, previously unused) when the timestamp is known, and
the push-path conversion (cloudSafeOperationalContext) carries UpdatedAt through.
A timestamp is non-identifying, so it adds no PII. Omitted when unknown.
- Tests: FormatCloudSafeContext freshness present/absent; cloudSafeOperationalContext
carries UpdatedAt end of the push conversion (and still emits no PII).
- Contract: ai-runtime cloud-safe context must carry discovery age when known.
Follow-up: same freshness line on the local/full path (formatSingleDiscovery).
Discovery-side enablement for making the Assistant useful on CLOUD models.
Today, sensitive resources route to cloud as a terse redacted summary, so
the Assistant never receives discovery's access context and gives generic
answers — invisibly broken for the majority of users who run cloud AI.
FormatCloudSafeContext returns the operational context the Assistant needs
(service identity, access pattern, config/data/log paths, port numbers)
while omitting PII by construction (no hostname, IP, bind addresses). The
chat sanitizer (Codex's ai/chat + unifiedresources policy lane) can include
this in cloud-routed summaries behind an opt-in, instead of withholding
everything. Local routing keeps using FormatForAIContext (full context).
Tested: includes service/access/paths/ports; rejects hostname + IP.
Cached discoveries from before the surface/fast-path/nested fixes still
counted as 'fresh' by the time-based window, so the panel showed worse
pre-fix data (what 'surely this isnt done?' surfaced on esphome).
Add servicediscovery.DiscoveryEngineVersion (currently 1), stamped onto
every freshly built discovery (LXC/VM and Docker build sites). Unlike
CLIAccessVersion it is NOT auto-upgraded on read, so a missing/older value
reliably means the result predates the current engine. The per-guest panel
shows an amber 're-run for improved results' nudge when engine version is
below current (CURRENT_DISCOVERY_ENGINE_VERSION, kept in sync).
Contract-neutral re: unified-resources (DiscoveryTab is a consumer; this is
a UI nudge + discovery field, no consumption-contract delta) — landed via
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT. Go package + version-stamp test pass;
type-check + eslint clean.
The name fast-path identifies workloads named after their service, but a
workload with a generic name (ct101, db1) still fell through to the model —
which, with a slow reasoning model configured, times out rather than just
being slow. Many such workloads are still identifiable instantly by a
DISTINCTIVE listening port.
Add a Ports field to the identity table (only single-service ports: 8123
HA, 32400 Plex, 5432 postgres, 6379 redis, 1883/8883 mosquitto, 3306
mariadb, 8086 influxdb, 8096 jellyfin, 9090 prometheus, 6052 esphome) and
a second fast-path inferSurfaceIdentityFromPorts that runs after the name
path and before the model. Ambiguous ports (80, 443, 8080, 3000, 5000) are
deliberately excluded — a bad port guess is worse than asking the model.
Parser validated against real ss -tlnp output from LXC 101 (extracts 8123,
ignores ephemeral ports / PIDs / docker-proxy noise → home-assistant).
Unit tests cover distinctive-port matches and ambiguous-only non-matches.
The fast-path nailed identity but emitted the bare-guest access path, so
for a service running in Docker inside an LXC/VM (e.g. Home Assistant
Container) it told the Assistant to 'pct exec' into the guest shell —
wrong: the service is one layer deeper.
Access topology is index-level 'how to reach it' (only a probe can know a
service runs in a nested container), so re-add a LIGHT nested-container
probe (docker ps names+images, not the deep enumeration) to the LXC/VM
surface sets. When a nested container matches the identified service,
layer cli_access: '... docker exec <container> <command>'. Identity stays
instant from the name; the access path is corrected by one cheap command
when the agent is connected, and falls back to bare-guest guidance when not.
Verified live: HA LXC 101 (homeassistant container under Docker, alongside
watchtower) now yields cli_access including 'docker exec homeassistant',
in ~1s. Tests cover the match (HA by name, postgres by image), non-matches
(watchtower, unrelated service, no docker), and the cli_access layering.
Discovery should be instant for the obvious case: a workload named after
its service ('home-assistant', 'frigate', 'mqtt'…) is identified from the
name alone, with no model call and no command scan needed. This is the
'surface index' — identity + how-to-reach — with depth left to the
Assistant's own knowledge and on-demand commands.
Expand knownServiceIdentities from one entry (esphome) to the common
homelab set, and run inferSurfaceIdentity BEFORE the model: on a
name match, build the identity result and skip the (slow reasoning-model)
analysis entirely. Conservative — name signals only, never broad
command-output guesses — so the model is skipped only on an obvious match;
ambiguous workloads still fall through to full analysis.
Verified live on real infra: HA LXC now identifies in ~0s as Home
Assistant (0.9), no model call, even with the agent disconnected —
previously it timed out at 45s on the reasoning model.
Tests: new inferSurfaceIdentity coverage; updated three tests whose
fixtures were named after known services (they now take the fast-path) —
abstention test uses a generic name, repair test expects 0 model calls,
cached test uses a complete identity. Full package green.
Discovery is the index, not the encyclopedia: it needs to quickly answer
'what is this and how do I reach it', then the Assistant supplies
standard-service knowledge and runs commands on demand for specifics.
Trim the guest command sets to surface identity signals only (OS,
hostname, running services, listening ports, top processes for
LXC/VM; OS, processes, ports, env for Docker). Drop the deep
enumeration — installed_packages, config_files, docker_mounts,
hardware/GPU, disk, cron, nested docker_check — which bloated the
evidence payload (and the AI analysis) for no benefit the Assistant
can't get live. Remove the now-unused dockerMountsCommand const and
retire its test; add TestGuestCommandSetsAreSurfaceOnly to pin the
surface intent (verified live: HA LXC went from 13 commands to 5).
Note: full speed also needs a fast identification path (the configured
reasoning model still exceeds the 45s analysis timeout on its own) —
that's the follow-up.
Completes the resource-type matrix in the scenario corpus: LXC, Docker,
VM, and now k8s. A redis-pod cache-loss/restart question that needs
kubectl exec access, redis.conf, the rollout-restart command, and the
memory-limit fact — verified through both chat and remediation packs.
Test-only coverage; full servicediscovery package green.
The corpus covered LXC and Docker workloads but not VMs, which the agent
reaches via the QEMU guest agent (qm guest exec) rather than pct/docker
exec. Add a Plex-on-VM cell — a transcode-failure question that needs the
guest-exec access, the GPU decoder (hardware fact), and the restart
command — verified through both the chat and remediation packs. Test-only
coverage; full servicediscovery package green.
FormatForRemediation surfaced config and log paths but not data paths,
while FormatForAIContext (chat) does. For remediation those matter —
backup targets, disk-full triage, restore points (e.g. a database data
dir or HA's /config/.storage). Add a Data Directories section, matching
the chat pack. Extends the remediation test to assert a data path
reaches it; teeth-checked. Full servicediscovery package green.
Add two common-service cells to the context oracle. Beyond documenting
nginx and MQTT, they pin two code paths the existing cells did not cover:
the read-only bind-mount marker (nginx config mounted read-only) and
security-category fact surfacing (mosquitto auth). Both pass against the
current formatters (no production gap) — regression protection for the
iter4/iter5 mount + fact-filter work. Corpus now 6 cells (HA-LXC,
HA-Docker, postgres, frigate, nginx, mosquitto). Full package green.
FormatForRemediation (the discovery context Patrol/remediation consume)
surfaced CLI access, config/log paths and ONLY hardware facts — so the
context meant for fixing a workload never told you how to restart it or
where to edit its files on the host, the two core fix actions. Add a
'Service Control' section (service-category facts: systemd unit / restart
command) and a 'Bind Mounts (host -> container)' section, matching the
parity FormatForAIContext already has after iters 4-7.
New test asserts the restart command and host bind-mount source reach the
remediation context; teeth-checked. Build + vet + gofmt clean, full
servicediscovery package green.
Iter 5 added service+storage to the surfaced fact categories but the
context pack still capped at the first 5 facts by insertion order — so a
trailing service-control fact (how to restart the workload, the most
actionable one we just started capturing) could be silently dropped,
undermining iter 5-6.
Sort the priority facts by actionability (service > security >
dependency > hardware > version > storage, stable within category)
before capping, and raise the cap 5 -> 8 (still under the analyzer's
12-fact limit). The most useful facts now always survive.
Corpus: add a fact-heavy Frigate cell (6 priority facts, service-control
last). Filter test now asserts cap=8 and that a trailing service fact
sorts first and survives. Both teeth-checked. Full package green.
Iterations 4-5 made the context pack SURFACE Docker mounts and
service/storage facts; this closes the CAPTURE side so the analyzer
actually produces them. The workload analysis prompt asked for config
dirs but never for how to restart/reload the service or the specific
files a user edits. Add: (q9) how the service is managed/restarted; an
instruction to put specific key files (configuration.yaml,
automations.yaml, postgresql.conf) in config_paths rather than just the
parent dir; and an instruction to record the service-control mechanism
as a 'service'-category fact. Directly serves the 'reload my automation'
case.
Test pins both instructions in the built deep prompt; teeth-checked.
Build + vet + gofmt clean, full servicediscovery package green.
filterImportantFacts kept only hardware/dependency/security/version
facts, dropping 'service' and 'storage'. But a service fact (e.g. the
systemd unit) is exactly how the Assistant restarts/reloads a workload,
and a storage fact (the backing dataset/disk) is where its data lives —
neither is redundant with the CLI/path sections, and both are what a
real question like 'the database is slow, restart it' needs. Add both to
the priority categories.
Corpus: add a postgresql LXC cell whose required context includes the
systemd unit and data filesystem. Teeth-checked — the case fails without
the filter change. Build + vet + gofmt clean, full servicediscovery
package green.
Two-part start of the Discovery->Assistant context-completeness work.
1. FormatForAIContext (the context pack Chat/Patrol consume) dropped
DockerMounts, even though the model captures them. A container path
like /config is meaningless for editing or backing up persistent files
without its host source, so the Assistant could not act on a real
request like 'edit my blinds automation on the host'. Surface the
host -> container mapping (with read-only marker).
2. Add scenario_corpus_test.go — the verifiable oracle for the goal:
given a realistic discovered workload, the context pack must surface
everything the Assistant needs to answer a concrete user question with
zero re-explanation. Seeded with Home Assistant (LXC: pct exec +
automations.yaml + log; Docker: docker exec + bind-mount source). The
corpus grows one service-type cell at a time; a missing substring is a
concrete gap to close in the analyzer or formatter.
Teeth-checked: the Docker case fails without change #1. Build + vet clean,
full servicediscovery package green.
When a deep scan was attempted for a workload (container/VM) but produced no
command output — e.g. the host agent rejects exec — the AI analyzer was still
asked to identify the service from metadata alone. It confabulated confident,
false identities: an ESPHome LXC and an influxdb-telegraf LXC were both
"identified" as Pi-hole at 0.95 confidence, with invented facts carrying
fabricated command sources (source: "pihole -v" for a command that never ran)
and a docker exec CLI for an LXC.
When a command scan was attempted for a command-dependent resource type but
yielded nothing, abstain: skip the analyzer entirely and return a not-determined
result (no service, no facts, zero confidence) with guidance to enable Pulse
Commands. Host agents are unaffected (identified from their own metadata), and
deployments without command scanning keep the metadata-only path. Abstaining
also avoids a wasted model call.
Verified live: re-running discovery on the esphome LXC now returns an empty,
zero-confidence result with the correct container CLI instead of fabricated
Pi-hole facts. Adds a regression test.
Dead-code sweep. Functions flagged unreachable by golang.org/x/tools/cmd/deadcode
and confirmed unused across pulse, pulse-enterprise, pulse-pro and pulse-mobile by
adversarial cross-repo verification. Cross-module reachability was checked
explicitly (only pkg/ exported symbols are importable by other modules; internal/
packages and _test.go files are not). go build, go vet and test-compile all pass.
- add a forced run action to pulse_discovery for known resources
- make discovery progress describe model-backed evidence analysis rather than a live Assistant chat
- keep shared select hydration stable for persisted discovery intervals
processFingerprint ran v.Index(i).Interface() then reflect.ValueOf(item),
dropping addressability and making reflect.Call panic on every iteration
with "Container as type *Container". The defer/recover in
collectFingerprints swallowed it, so LXC and VM fingerprints never
landed in the store — change-detection and discovery for those resource
types have been broken since v6.
Pass the slice element's address straight through (.Addr()) so the
generator's pointer receiver gets the right type. Add a regression test
that fails if anyone goes back through .Interface().
- Redirect PVE node lookups to linked Host Agent ID when available.
- Implement deduplication in discovery lists to prefer Host Agent data over redundant Node entries.
- Add fallback mechanism to original Node ID for discovery retrieval ensuring compatibility with legacy data.
- Update data adapters and added comprehensive unit tests for redirection and deduplication logic.
- AI Intelligence endpoints (/api/ai/intelligence/*, /api/ai/forecast/*,
/api/ai/unified/findings, etc.) now require ai:execute scope to prevent
low-privilege tokens from reading sensitive intelligence data
- AI Knowledge endpoints (/api/ai/knowledge/*) now require ai:chat scope
to prevent arbitrary guest data access across the fleet
- AI Debug Context (/api/ai/debug/context) now requires settings:read scope
to prevent system prompt and infrastructure details leakage
- WebSocket origin check now validates peer IP is private when allowing
private network origins, mitigating CSWSH attacks where a malicious page
on the same LAN tries to hijack connections using victim's session cookie
Add deterministic URL suggestion based on service type and external IP:
- Add SuggestedURL field to ResourceDiscovery type (Go + TypeScript)
- Create url_suggestion.go with 60+ service defaults (Jellyfin, Plex,
Home Assistant, Grafana, Proxmox, etc.)
- Support HTTPS services, custom paths (/web, /dashboard/, /admin)
- Fall back to discovered ports for unknown services
- Add UI in DiscoveryTab with "Use this" button to populate URL input
- Add comprehensive unit tests for URL suggestion logic
Suggestion only appears when no custom URL is saved. User clicks
"Use this" to populate the input, then "Save" to confirm.
- Fix visual flash in discovery tab
- Standardize table column widths and UI across Docker, Hosts, Storage, etc.
- Add support for new K8s and Host charts
- Fix Service Discovery tests
- Add AI provider indicator showing local (Ollama) vs cloud (Anthropic/OpenAI) analysis
- Add "What Discovery Does" explanation section before first scan
- Show commands preview before scan so users know what will run
- Add scan details section showing raw command outputs for admins
- Filter sensitive Docker labels (passwords, secrets, tokens) before AI analysis
- Add comprehensive tests for label filtering
This improves sysadmin confidence by making discovery transparent about
what it does, what data it collects, and where that data goes.
- Fix routing for POST/PUT/DELETE on /api/discovery/host/ endpoints
(Go's http.ServeMux was matching the longer prefix before method-specific routes)
- Add HOST-specific AI prompt that focuses on identifying the host OS
rather than services/containers running on it
- Add success message UI after discovery completes
- Fix timing so success appears after data is visible (not during refetch)
- Add error handling and display for failed discoveries
