fetch_rows_remote loaded every row in the window into a list and
json.dumps'd the whole payload on the remote side; on the 1GB
pulse-license droplet a 30-day window over a 104k-row DB got the
process OOM-killed. The remote helper now prints a db_stats header
line and then one JSON line per row while iterating the cursor, so
it never holds the full result set. Local parsing updated to match
and covered by unit tests; fetch_rows_local is unchanged.
Verified end to end against the droplet with --since-days 30.
Contract-Neutral: fetch_rows_remote OOM fix: stream JSON-lines instead of one giant payload; no contract delta, report output unchanged
Production telemetry (7d, 2026-07-17) shows a persistent tail of live
installs on 6.0.0-rc.x and 6.0.3/6.0.4 while 6.0.5 has been stable since
2026-07-09. Root cause for the reachable stranding class: release
selection trusted GitHub's created_at ordering, and this repo interleaves
v5-line maintenance releases with v6 releases (v5.1.36 was created the
day before v6.0.5). Whenever a v5.1.x release is the most recently
created stable:
- stable channel (getLatestReleaseForChannel) returned the first
non-prerelease in list order -> "no update" for every v6 install
until the next v6 release ships
- the RSS rate-limit fallback returned the first feed entry matching the
channel, same failure, both channels
- the unattended updater trusted /releases/latest, which GitHub defines
as most-recently-created stable -> timer no-ops in the same window
All three now select the highest eligible version: the API path tracks
newest stable + newest prerelease in one pass (prerelease = GitHub flag
OR tag shape), the feed fallback picks the max matching the channel, and
pulse-auto-update.sh scans /releases pairing each tag with its own
draft/prerelease flags before the fail-closed shape filter, keeping
/releases/latest as fallback. helm-chart-* tags fail semver parsing and
are dropped everywhere.
Verified not broken (pinned by new tests): rc-channel installs are
offered the newer stable (6.0.0-rc.6 -> 6.0.5) and move onto the next rc
line when one opens (-> 6.1.0-rc.2); the auto-update prerelease filter
(fail-closed is_prerelease_tag, metadata-flag refusal) still refuses
prerelease targets on stable. Go tests fail on the pre-fix code
(returned 5.1.37 / "no update").
Contract: deployment-installability now pins version-max release
selection with proofs in manager_stranded_upgrade_test.go and
pulse_auto_update_test.go.
Every mock-mode boot (e2e/CI managed backends, qual runs, demo containers)
was sending startup/heartbeat pings describing the synthetic fixture fleet,
registering as real installs in the license-server telemetry DB and
dominating the VMware/Kubernetes/TrueNAS adoption aggregates (346 of 6,225
weekly-active installs matched the mock signature, incl. 120 of 227 on
6.1.0-rc.1).
- internal/telemetry: route startup and heartbeat sends through sendEvent,
which drops the ping while mock.IsMockEnabled() is true; checked per event
so runtime mock toggles take effect immediately. Disclosed in the package
doc and both PRIVACY.md copies.
- scripts/telemetry_adoption_report.py: exclude historical mock-fleet rows
(kubernetes_pods = 120×N with vmware_hosts = 7×N, the internal/mock
fixture template) by default, report the excluded row/install counts, and
add --include-mock-fleet to audit them. Versions through 6.1.0-rc.2 keep
pinging the mock fleet until upgraded, so report-side filtering stays
necessary.
- security-privacy contract: record the mock-mode suppression boundary and
the report-side mock-fleet exclusion as subsystem invariants.
- Tests: Go sendEvent suppression/sending pair against an httptest endpoint;
Python signature-matcher and summarize_rows exclusion coverage.
Manifest-backed MCP tools, prompts, and resources with surface affordance contracts; agent capability manifest and governance projection; API contract tests and capability route projection; operations-loop and intelligence-funnel telemetry; release-control subsystem documentation, registry, and tooling; licensing and configuration.
Finishes the deferred half of the v5->v6 parity fix for #1323 (the
pulse-auto-update.sh half landed in 672e81985). The interactive install.sh
update/reinstall flow stopped a running Pulse then called start_pulse, which
tolerates a silent start failure (common on unprivileged LXC) by printing a note
and returning 0 — leaving Pulse stopped under an "installation completed!" message.
- stop_pulse_for_update records whether Pulse was running before the update.
- start_pulse, only when Pulse was running before (PULSE_WAS_ACTIVE), no longer
accepts a silent start failure: it verifies the service became active
(wait_for_service_active, 20s), retries one explicit start, and surfaces a clear
error + diagnostics if it still will not come up. Fresh installs are unchanged
(the flag stays false, so the reassuring container note is kept).
- Wired into all three update/reinstall sites; added a BASH_SOURCE guard so the
installer's functions can be unit-tested without running the installer.
Scope: fixes the #1323 'restart silently failed' case. Does NOT add a binary
rollback (download_pulse deletes bin/pulse.old right after the swap) — a
bad-release rollback is a separate concern.
Test: scripts/tests/test-install-update-resilience.sh (sources install.sh, stubs
systemctl, asserts was-active capture + retry + clear error). Go installtests +
bash -n confirm the guard does not change installer execution.
Back-port the pulse-auto-update.sh half of v5 fix 0f2982ce3 to v6. After a
verified update, if Pulse was running beforehand, wait_for_service_active
polls up to 20s, tries one explicit start, and if it still does not come
back up restores the backup binary/VERSION, restarts, and returns failure
instead of leaving Pulse stopped (common on unprivileged LXC where the
installer's restart silently fails). Adds a BASH_SOURCE guard so the
script can be sourced, plus scripts/tests/test-pulse-auto-update.sh.
Note: the install.sh start_pulse STOPPED_PULSE_SERVICE guard (interactive
installer path) still needs the stop-tracking infrastructure wired into
v6's update flow; tracked as a follow-up.
Adopt replacement managed hot-dev supervisors during listener waits and let mock-mode toggles continue only when the browser entrypoint proves the requested runtime state after a non-clean restart.
Render managed runtime status probes as unavailable instead of 000 when a restricted shell can see loopback listeners but cannot run local HTTP probes. Refs architecture post-RC canonicalization follow-up.
Keep the managed dev runtime on loopback unless PULSE_DEV_LAN=true is set, so installed LAN agents cannot accidentally attach to a developer session. Harden managed status checks for restricted local shells. Refs architecture post-RC canonicalization follow-up.
Two specific platform-page quality issues from the audit:
1. **/kubernetes/overview only had 1 cluster.** Bumping the K8s cluster
count past 1 had been deferred because the prior
monitor-broadcast equivalence test compared the raw snapshot count
to the broadcast count exactly, and broadcast's
`coalesceBroadcastResources` + second-pass coalesce inside
`convertResourcesForBroadcast` legitimately drops merge candidates
that the raw snapshot keeps. Switch the test to compare against
the canonical snapshot count within a ±5% tolerance so future
fixture bumps stay green without loosening any of the test's
exact-name and exact-identity assertions. With that in place, bump
`K8sClusterCount` 1 → 3 in `internal/mock/generator.go`,
`scripts/toggle-mock.sh`, and the matching
`scripts/tests/test-toggle-mock.sh` so the canonical mock estate
ships with production + staging + edge clusters end-to-end.
Live mock survey: k8s-cluster: 3, k8s-deployment: 42, pod: 120,
plus 15 K8s nodes merged onto their agent hosts.
2. **/vmware/storage looked empty under platform-page chrome.**
`resolveStorageSourceKey` was reading only `storage.type` (the
on-disk technology like `vsan`, `vmfs`, `nfs41`, `zfs-pool`) and
never consulted `storage.platform` (the canonical platform key
like `vmware-vsphere` or `truenas`). Source filter chip options
were therefore generated as `vsan`, `vmfs`, `nfs41`, etc., and
`forcedSourceFilter='vmware-vsphere'` had nothing to match.
Prefer the canonical `storage.platform` tag when set, so VMware
datastores group under `vmware-vsphere`, TrueNAS pools group under
`truenas`, PBS datastores under `proxmox-pbs`, etc., for both the
chip options and the embedded platform-page filter.
Browser verification (Playwright, chromium, live mock-mode dev runtime):
- 9 tests pass.
Targeted vitest:
- `src/features/storageBackups` + `src/utils/__tests__/sourcePlatforms.test.ts` +
`src/components/Storage/__tests__/storageSourceOptions.test.ts` (31
files / 141 tests) green.
Go tests:
- `go test ./internal/mock/... ./internal/monitoring/... ./internal/vmware/...`
all green.
Contracts updated:
- `monitoring.md` Shared Boundaries: new K8s multi-cluster default,
±5% tolerance for the broadcast equivalence assertion.
- `deployment-installability.md` Shared Boundaries: toggle-mock.sh /
DefaultConfig parity updated for the K8sClusterCount=3 baseline.
Mock pages were sparse: 3 Proxmox nodes × 3 VMs × 3 LXCs, 2 Docker
hosts × 5 containers, 1 K8s cluster × 3 nodes × 10 pods × 4
deployments. That populated platform pages with handfuls of rows
rather than table density that exercises sorting, grouping, drawers,
and responsive layout.
Bump `internal/mock/generator.go::DefaultConfig` to target a mature
small-to-mid homelab / SMB environment:
- NodeCount: 3 → 5 (matches the curated demo scenario's pve1..pve5
regional naming)
- VMsPerNode: 3 → 6
- LXCsPerNode: 3 → 8
- DockerHostCount: 2 → 5
- DockerContainersPerHost: 5 → 14
- GenericHostCount: 2 → 4
- K8sClusterCount: 1 (unchanged; the curated demo and broadcast
coalesce tests assume a single cluster identity)
- K8sNodesPerCluster: 3 → 5
- K8sPodsPerCluster: 10 → 40
- K8sDeploymentsPerCluster: 4 → 14
Resource survey under the new defaults (live mock backend):
- TOTAL 307 unique resources (was ~50-100)
- app-container: 75, storage: 55, system-container: 44, pod: 40,
vm: 31, physical_disk: 19, agent: 15, k8s-deployment: 14,
docker-host: 5, network-endpoint: 5, pmg: 2, pbs: 1, k8s-cluster: 1
Platform pages now feel populated under mock mode:
- /docker/overview: 5 hosts (was 2)
- /docker/containers: 75 containers (was 13)
- /kubernetes/nodes: 5 (was 3)
- /kubernetes/pods: 40 (was 10)
- /kubernetes/deployments: 14 (was 4)
`internal/mock/demo_scenarios.go` extended to season `local`,
`local-zfs`, and per-node iso/service-pool storage names for pve6 and
beyond, so future NodeCount bumps don't regress the curated demo into
generic "service-pool" labels (a test guard explicitly forbids that
alias). A new `TestDemoScenarioStorageNamingHandlesScaledNodeCount`
covers the scaled-NodeCount path.
`internal/monitoring/monitor_unified_state_test.go` updated to compare
the broadcast count against the coalesced snapshot count rather than
the raw snapshot count — the broadcast path merges resources that
share a canonical host key (K8s nodes onto linked agent hosts), so
larger fixture sizes legitimately produce more merge candidates, and
the prior raw-equality assertion would have broken on any future
fixture growth too. The test still asserts every canonical name and
mock identity it checked before.
`scripts/toggle-mock.sh` (`mock_default_entries`) and the matching
`scripts/tests/test-toggle-mock.sh` assertions are aligned with the
new defaults so `npm run mock:edit` and per-dev `.env` seeding match
the canonical baseline.
Contracts updated:
- `monitoring.md` Shared Boundaries: records the new DefaultConfig
target sizes and the requirement that demo-scenario seasoning stay
aligned with NodeCount changes.
- `deployment-installability.md` Shared Boundaries: records that
`mock_default_entries()` in toggle-mock.sh must stay aligned with
`internal/mock.DefaultConfig` so CLI/toggle/runtime mock densities
never drift apart.
Targeted Go tests:
- `go test ./internal/mock/...` green
- `go test ./internal/monitoring/...` green
Playwright (chromium, live mock-mode dev runtime):
- 9 tests, all pass; populated assertions now hit dense tables (5
hosts, 14+ containers, 40 pods, etc.).
Known remaining fixture gaps (canonical adapter, not config):
- VMware fixture inventory in `internal/vmware/fixtures.go` is
hardcoded at 4 hosts / 6 VMs / 4 datastores; not scaled in this
commit.
- TrueNAS fixture inventory in `internal/truenas/fixtures.go` is
similarly hardcoded; not scaled in this commit.
Fix the local hot-dev backend monitor so a missing Pulse process is counted safely under pipefail, keep backend launch stderr in the debug log, and govern the new runtime helper with focused smoke coverage.
The supervisor's backend health monitor in scripts/hot-dev.sh only
checked whether a `./pulse` process existed via pgrep. If the Go binary
hung, panicked into recovery without exiting, or failed to bind :7655
while the process stayed alive, the monitor saw a healthy process count
and never restarted, leaving the dev frontend talking to a dead listener.
Hit that twice in one session.
Add an HTTP probe of /api/health on the dev port. After two consecutive
5s polls where the process exists but /api/health is unreachable, kill
the process and restart the same way the dead/duplicate branches do.
Cut the poll interval from 10s to 5s so the worst-case unresponsive
window is ~10s instead of ~30s. Bring deployment-installability into
line by naming alive-but-unresponsive recovery as part of the dev-runtime
orchestration boundary, and cover the new branch with static-source
assertions in scripts/tests/test-hot-dev-bg.sh.
