Two failures landed this morning:
- test_gitleaks_ignore.py's historical-fingerprint resolution (added in
d1fa7f38c) needs branch- and tag-reachable history, but the
scripts-and-build job used a depth-1 checkout, so all 37 fingerprints
failed to resolve in CI. Fetch full history like the gitleaks and
changes jobs already do.
- d1f687c0e routed frontend-modern/package.json and package-lock.json to
the new frontend-dependency-security path policy without updating the
manifest expectations pinned in test-hot-dev-bg.sh. Pin the effective
first-match policy per manifest instead of bare membership so the
routing stays asserted and shadowed entries cannot fake coverage.
Contract-Neutral: CI-only fix: full-history checkout for gitleaks fingerprint resolution test and re-pin hot-dev manifest policy expectations to the registry routing d1f687c0e already established; no contract delta
The docs link drift guard (test_repo_docs_link_drift.py) fails CI because
render_release_body.py links docs/CODE_SIGNING_POLICY.md at blob/main.
Release bodies must reference the docs snapshot of the release they
describe, so the installation section now links blob/v{version}, matching
how the rest of the body pins to the version being cut. The unit test pin
follows. Also skip .claude/ in the drift guard so local runs on a shared
tree are not polluted by stale agent worktrees under .claude/worktrees
that CI never checks out.
The mock:on and mock:off scripts used GNU 'sed -i' syntax, which fails on
the default macOS BSD sed. The '||' fallback then appended a fresh
PULSE_MOCK_MODE line on every invocation, so a local .env accumulated three
contradictory entries.
They also targeted the repo-root .env, which hot-dev.sh does not consult
when choosing the data directory. hot-dev.sh reads the canonical flag from
tmp/dev-config/.env, written authoritatively by toggle-mock.sh. The npm
wrappers were therefore a no-op that corrupted .env as a side effect, while
leaving the operator believing mock mode had switched.
Point them at toggle-mock.sh, which handles BSD sed, writes the canonical
file, syncs the runtime env and restarts the managed runtime. Add
mock:status and mock:edit; the hot-dev startup banner already advertised
'npm run mock:edit', which did not exist.
Extend the deployment-installability contract to cover the mock wrappers
alongside the existing repo-root dev entry rules, and add a regression test
asserting the wrappers delegate rather than rewriting the flag inline, and
that every mock command named in the hot-dev banner exists.
Adversarial review of 9db25ba60 found four residual defects in the
auto-update asset install, plus a doc line it left contradicting itself.
- install_auto_update_assets copied the bundled helper into the staged
mktemp file with an unchecked cp, and both call sites invoke the
function under `if !`, which suppresses errexit for its whole body. A
failed copy (ENOSPC, EIO) fell through to
configure_auto_update_script_repo, whose awk emits a lone GITHUB_REPO=
line for empty input, so a shebang-less one-line stub replaced the
working helper with a "script" that only ever exits 0 - silently
disabling unattended updates. Check the cp, and refuse the swap unless
the staged helper is non-empty and starts with #!.
- Both units were rendered with a bare truncating `cat > "$unit"` whose
status was never checked, and the function's last statement is
safe_systemctl daemon-reload, which returns 0 by design. A failing
write truncated a working unit and still reported success. Render each
unit to ${path}.tmp and commit it with a checked rename, so a failure
leaves the installed unit byte-identical.
- The widened ReadWritePaths could not reach deployed boxes: the unit
that grants the write access is itself the file that has to be
rewritten, and on an existing install the sandbox running the
installer excludes /etc/systemd/system and /usr/local/bin (EROFS). The
Go update pipeline cannot carry it either - pulse.service runs as
User=pulse with its own ProtectSystem=strict over the install and
config dirs only. So probe each destination directory up front and,
when one is blocked, re-exec this already-signature-verified installer
through systemd-run with a new internal --repair-auto-update-units
entry point: PID 1 forks the transient unit, so it starts in the host
mount namespace instead of inheriting the sandbox. The installer is
copied into the install dir first because the calling unit's
PrivateTmp=yes hides its /tmp copy from PID 1. The escape needs root
and systemd-run, and never recurses.
- Keep the ReadWritePaths entries as directory grants: every write now
commits with a rename from a sibling staging file, and rename needs
write access on the containing directory, so the file-level entries
systemd would otherwise accept cannot work. Document the tradeoff in
the unit and the subsystem contract instead.
The deployment-installability contract still claimed the update sandbox
leaves "only the install dir, config dir and /tmp" writable, which the
paragraph the same file gained in 9db25ba60 contradicts; the same stale
rationale had been copied into two test comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two bugs from #1630 that combined to take installs down silently:
1. perform_update()'s install-failed rollback branch restored the backup
but never restarted pulse.service. Since the generated
pulse-update.service gates on ExecCondition=systemctl is-active,
every later timer run was then skipped and the install stayed down
until manual intervention. Restart is now guaranteed by a
service_was_active-guarded restart in that branch plus an
ensure_service_restarted RETURN trap so no exit path can miss it
(re-fix of #1323, originally c0b3a0e66, lost in 778a2577b and only
partially restored in 672e81985).
2. install.sh aborted under errexit when writing the /bin/update helper
on a read-only filesystem - after the new binary was installed and
the service stopped, landing in bug 1's no-restart branch. The stock
pulse-update.service uses ProtectSystem=strict, so /bin and
/usr/local/bin are read-only on stock unattended updates; transient
read-only remounts hit the same path. The helper write, PATH
appends, and the /usr/local/bin/pulse symlink are now idempotent and
non-fatal with a warning (install_binary_symlink).
Contract: deployment-installability now pins fail-closed service
availability for unattended updates and non-fatal writes outside the
hardened unit's writable set, with proofs in pulse_auto_update_test.go
and root_install_sh_test.go plus shell regression coverage in
scripts/tests/test-pulse-auto-update.sh (installer-exits-nonzero path)
and scripts/tests/test-install-update-resilience.sh (read-only helper
and symlink paths, verified under set -e, root-safe via ENOTDIR).
Fixes#1630
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fetch_rows_remote loaded every row in the window into a list and
json.dumps'd the whole payload on the remote side; on the 1GB
pulse-license droplet a 30-day window over a 104k-row DB got the
process OOM-killed. The remote helper now prints a db_stats header
line and then one JSON line per row while iterating the cursor, so
it never holds the full result set. Local parsing updated to match
and covered by unit tests; fetch_rows_local is unchanged.
Verified end to end against the droplet with --since-days 30.
Contract-Neutral: fetch_rows_remote OOM fix: stream JSON-lines instead of one giant payload; no contract delta, report output unchanged
Production telemetry (7d, 2026-07-17) shows a persistent tail of live
installs on 6.0.0-rc.x and 6.0.3/6.0.4 while 6.0.5 has been stable since
2026-07-09. Root cause for the reachable stranding class: release
selection trusted GitHub's created_at ordering, and this repo interleaves
v5-line maintenance releases with v6 releases (v5.1.36 was created the
day before v6.0.5). Whenever a v5.1.x release is the most recently
created stable:
- stable channel (getLatestReleaseForChannel) returned the first
non-prerelease in list order -> "no update" for every v6 install
until the next v6 release ships
- the RSS rate-limit fallback returned the first feed entry matching the
channel, same failure, both channels
- the unattended updater trusted /releases/latest, which GitHub defines
as most-recently-created stable -> timer no-ops in the same window
All three now select the highest eligible version: the API path tracks
newest stable + newest prerelease in one pass (prerelease = GitHub flag
OR tag shape), the feed fallback picks the max matching the channel, and
pulse-auto-update.sh scans /releases pairing each tag with its own
draft/prerelease flags before the fail-closed shape filter, keeping
/releases/latest as fallback. helm-chart-* tags fail semver parsing and
are dropped everywhere.
Verified not broken (pinned by new tests): rc-channel installs are
offered the newer stable (6.0.0-rc.6 -> 6.0.5) and move onto the next rc
line when one opens (-> 6.1.0-rc.2); the auto-update prerelease filter
(fail-closed is_prerelease_tag, metadata-flag refusal) still refuses
prerelease targets on stable. Go tests fail on the pre-fix code
(returned 5.1.37 / "no update").
Contract: deployment-installability now pins version-max release
selection with proofs in manager_stranded_upgrade_test.go and
pulse_auto_update_test.go.
Every mock-mode boot (e2e/CI managed backends, qual runs, demo containers)
was sending startup/heartbeat pings describing the synthetic fixture fleet,
registering as real installs in the license-server telemetry DB and
dominating the VMware/Kubernetes/TrueNAS adoption aggregates (346 of 6,225
weekly-active installs matched the mock signature, incl. 120 of 227 on
6.1.0-rc.1).
- internal/telemetry: route startup and heartbeat sends through sendEvent,
which drops the ping while mock.IsMockEnabled() is true; checked per event
so runtime mock toggles take effect immediately. Disclosed in the package
doc and both PRIVACY.md copies.
- scripts/telemetry_adoption_report.py: exclude historical mock-fleet rows
(kubernetes_pods = 120×N with vmware_hosts = 7×N, the internal/mock
fixture template) by default, report the excluded row/install counts, and
add --include-mock-fleet to audit them. Versions through 6.1.0-rc.2 keep
pinging the mock fleet until upgraded, so report-side filtering stays
necessary.
- security-privacy contract: record the mock-mode suppression boundary and
the report-side mock-fleet exclusion as subsystem invariants.
- Tests: Go sendEvent suppression/sending pair against an httptest endpoint;
Python signature-matcher and summarize_rows exclusion coverage.
Manifest-backed MCP tools, prompts, and resources with surface affordance contracts; agent capability manifest and governance projection; API contract tests and capability route projection; operations-loop and intelligence-funnel telemetry; release-control subsystem documentation, registry, and tooling; licensing and configuration.
Finishes the deferred half of the v5->v6 parity fix for #1323 (the
pulse-auto-update.sh half landed in 672e81985). The interactive install.sh
update/reinstall flow stopped a running Pulse then called start_pulse, which
tolerates a silent start failure (common on unprivileged LXC) by printing a note
and returning 0 — leaving Pulse stopped under an "installation completed!" message.
- stop_pulse_for_update records whether Pulse was running before the update.
- start_pulse, only when Pulse was running before (PULSE_WAS_ACTIVE), no longer
accepts a silent start failure: it verifies the service became active
(wait_for_service_active, 20s), retries one explicit start, and surfaces a clear
error + diagnostics if it still will not come up. Fresh installs are unchanged
(the flag stays false, so the reassuring container note is kept).
- Wired into all three update/reinstall sites; added a BASH_SOURCE guard so the
installer's functions can be unit-tested without running the installer.
Scope: fixes the #1323 'restart silently failed' case. Does NOT add a binary
rollback (download_pulse deletes bin/pulse.old right after the swap) — a
bad-release rollback is a separate concern.
Test: scripts/tests/test-install-update-resilience.sh (sources install.sh, stubs
systemctl, asserts was-active capture + retry + clear error). Go installtests +
bash -n confirm the guard does not change installer execution.
Back-port the pulse-auto-update.sh half of v5 fix 0f2982ce3 to v6. After a
verified update, if Pulse was running beforehand, wait_for_service_active
polls up to 20s, tries one explicit start, and if it still does not come
back up restores the backup binary/VERSION, restarts, and returns failure
instead of leaving Pulse stopped (common on unprivileged LXC where the
installer's restart silently fails). Adds a BASH_SOURCE guard so the
script can be sourced, plus scripts/tests/test-pulse-auto-update.sh.
Note: the install.sh start_pulse STOPPED_PULSE_SERVICE guard (interactive
installer path) still needs the stop-tracking infrastructure wired into
v6's update flow; tracked as a follow-up.
Adopt replacement managed hot-dev supervisors during listener waits and let mock-mode toggles continue only when the browser entrypoint proves the requested runtime state after a non-clean restart.
Render managed runtime status probes as unavailable instead of 000 when a restricted shell can see loopback listeners but cannot run local HTTP probes. Refs architecture post-RC canonicalization follow-up.
Keep the managed dev runtime on loopback unless PULSE_DEV_LAN=true is set, so installed LAN agents cannot accidentally attach to a developer session. Harden managed status checks for restricted local shells. Refs architecture post-RC canonicalization follow-up.
Two specific platform-page quality issues from the audit:
1. **/kubernetes/overview only had 1 cluster.** Bumping the K8s cluster
count past 1 had been deferred because the prior
monitor-broadcast equivalence test compared the raw snapshot count
to the broadcast count exactly, and broadcast's
`coalesceBroadcastResources` + second-pass coalesce inside
`convertResourcesForBroadcast` legitimately drops merge candidates
that the raw snapshot keeps. Switch the test to compare against
the canonical snapshot count within a ±5% tolerance so future
fixture bumps stay green without loosening any of the test's
exact-name and exact-identity assertions. With that in place, bump
`K8sClusterCount` 1 → 3 in `internal/mock/generator.go`,
`scripts/toggle-mock.sh`, and the matching
`scripts/tests/test-toggle-mock.sh` so the canonical mock estate
ships with production + staging + edge clusters end-to-end.
Live mock survey: k8s-cluster: 3, k8s-deployment: 42, pod: 120,
plus 15 K8s nodes merged onto their agent hosts.
2. **/vmware/storage looked empty under platform-page chrome.**
`resolveStorageSourceKey` was reading only `storage.type` (the
on-disk technology like `vsan`, `vmfs`, `nfs41`, `zfs-pool`) and
never consulted `storage.platform` (the canonical platform key
like `vmware-vsphere` or `truenas`). Source filter chip options
were therefore generated as `vsan`, `vmfs`, `nfs41`, etc., and
`forcedSourceFilter='vmware-vsphere'` had nothing to match.
Prefer the canonical `storage.platform` tag when set, so VMware
datastores group under `vmware-vsphere`, TrueNAS pools group under
`truenas`, PBS datastores under `proxmox-pbs`, etc., for both the
chip options and the embedded platform-page filter.
Browser verification (Playwright, chromium, live mock-mode dev runtime):
- 9 tests pass.
Targeted vitest:
- `src/features/storageBackups` + `src/utils/__tests__/sourcePlatforms.test.ts` +
`src/components/Storage/__tests__/storageSourceOptions.test.ts` (31
files / 141 tests) green.
Go tests:
- `go test ./internal/mock/... ./internal/monitoring/... ./internal/vmware/...`
all green.
Contracts updated:
- `monitoring.md` Shared Boundaries: new K8s multi-cluster default,
±5% tolerance for the broadcast equivalence assertion.
- `deployment-installability.md` Shared Boundaries: toggle-mock.sh /
DefaultConfig parity updated for the K8sClusterCount=3 baseline.
Mock pages were sparse: 3 Proxmox nodes × 3 VMs × 3 LXCs, 2 Docker
hosts × 5 containers, 1 K8s cluster × 3 nodes × 10 pods × 4
deployments. That populated platform pages with handfuls of rows
rather than table density that exercises sorting, grouping, drawers,
and responsive layout.
Bump `internal/mock/generator.go::DefaultConfig` to target a mature
small-to-mid homelab / SMB environment:
- NodeCount: 3 → 5 (matches the curated demo scenario's pve1..pve5
regional naming)
- VMsPerNode: 3 → 6
- LXCsPerNode: 3 → 8
- DockerHostCount: 2 → 5
- DockerContainersPerHost: 5 → 14
- GenericHostCount: 2 → 4
- K8sClusterCount: 1 (unchanged; the curated demo and broadcast
coalesce tests assume a single cluster identity)
- K8sNodesPerCluster: 3 → 5
- K8sPodsPerCluster: 10 → 40
- K8sDeploymentsPerCluster: 4 → 14
Resource survey under the new defaults (live mock backend):
- TOTAL 307 unique resources (was ~50-100)
- app-container: 75, storage: 55, system-container: 44, pod: 40,
vm: 31, physical_disk: 19, agent: 15, k8s-deployment: 14,
docker-host: 5, network-endpoint: 5, pmg: 2, pbs: 1, k8s-cluster: 1
Platform pages now feel populated under mock mode:
- /docker/overview: 5 hosts (was 2)
- /docker/containers: 75 containers (was 13)
- /kubernetes/nodes: 5 (was 3)
- /kubernetes/pods: 40 (was 10)
- /kubernetes/deployments: 14 (was 4)
`internal/mock/demo_scenarios.go` extended to season `local`,
`local-zfs`, and per-node iso/service-pool storage names for pve6 and
beyond, so future NodeCount bumps don't regress the curated demo into
generic "service-pool" labels (a test guard explicitly forbids that
alias). A new `TestDemoScenarioStorageNamingHandlesScaledNodeCount`
covers the scaled-NodeCount path.
`internal/monitoring/monitor_unified_state_test.go` updated to compare
the broadcast count against the coalesced snapshot count rather than
the raw snapshot count — the broadcast path merges resources that
share a canonical host key (K8s nodes onto linked agent hosts), so
larger fixture sizes legitimately produce more merge candidates, and
the prior raw-equality assertion would have broken on any future
fixture growth too. The test still asserts every canonical name and
mock identity it checked before.
`scripts/toggle-mock.sh` (`mock_default_entries`) and the matching
`scripts/tests/test-toggle-mock.sh` assertions are aligned with the
new defaults so `npm run mock:edit` and per-dev `.env` seeding match
the canonical baseline.
Contracts updated:
- `monitoring.md` Shared Boundaries: records the new DefaultConfig
target sizes and the requirement that demo-scenario seasoning stay
aligned with NodeCount changes.
- `deployment-installability.md` Shared Boundaries: records that
`mock_default_entries()` in toggle-mock.sh must stay aligned with
`internal/mock.DefaultConfig` so CLI/toggle/runtime mock densities
never drift apart.
Targeted Go tests:
- `go test ./internal/mock/...` green
- `go test ./internal/monitoring/...` green
Playwright (chromium, live mock-mode dev runtime):
- 9 tests, all pass; populated assertions now hit dense tables (5
hosts, 14+ containers, 40 pods, etc.).
Known remaining fixture gaps (canonical adapter, not config):
- VMware fixture inventory in `internal/vmware/fixtures.go` is
hardcoded at 4 hosts / 6 VMs / 4 datastores; not scaled in this
commit.
- TrueNAS fixture inventory in `internal/truenas/fixtures.go` is
similarly hardcoded; not scaled in this commit.
Fix the local hot-dev backend monitor so a missing Pulse process is counted safely under pipefail, keep backend launch stderr in the debug log, and govern the new runtime helper with focused smoke coverage.
