The finding-links spec asserted Open related links into the retired
standalone routes from the pre-workbench findings list; those cross-link
affordances were removed deliberately with platform-first navigation,
and the monitor-first workbench feeds findings through a different
pipeline its stubs never reach. The run-history breakdown spec pinned
per-run type counts that the workbench simplified to plain totals; the
underlying count-separation contract (truenas_checked distinct from
agent hosts) lives in the investigation context model and is unit
covered there.
The spec entered through a retired settings route with an inert REST
stub (mention candidates come from websocket state) and the old Expand
Pulse Assistant launcher name. Re-pointed at the TrueNAS platform page
and the Ask Pulse Assistant launcher, it exposes a real regression:
Docker app-containers appear in the @-mention autocomplete but TrueNAS
app-containers no longer do, despite identical resource typing in
websocket state. The spec keeps the contract and skips via fixme until
mention targeting is restored.
The alert-links spec pinned Open related links into the retired
standalone routes, which were removed together with those routes'
cross-link affordances; its stub timestamps had also aged out of the
history window. The stubbed TrueNAS alert now uses fresh timestamps, the
Resource action is scoped to its own row (live mock alerts share the
page), and the contract asserts the surviving canonical handoff: the
resource incidents panel scoped to the alerting resource.
The disk-history contract lived on the retired /storage route with REST
stubs the platform pages never read. It now drives the TrueNAS storage
section against the mock dataset: the SMART-failing sdc disk opens the
resource drawer, and the History tab must request the disk series from
the metrics store under a canonical key (serial when available, the
disk:<node>:<device> composite otherwise - the store serves both, which
is what protects serial-less disks).
The thresholds surface replaced the neutral infrastructure/systems/
containers groupings with per-platform scopes, so the TrueNAS routing
contract inverted by design: TrueNAS systems, pools, datasets, and disks
now live under their own scope button. The spec asserts that scope's
sections against the mock dataset instead of the retired neutral pages.
The TrueNAS storage-links spec asserted cross-links from the retired
/infrastructure card into the retired /workloads, /storage, and /recovery
routes, and the VMware source-filter spec asserted the retired
/infrastructure source dropdown; platform-first navigation replaced both
concepts with the per-platform pages and their section navigation, which
the platform-pages shell spec covers.
The PBS active-task and job-health evidence drawer content this spec
pins still exists in ResourceDetailDrawerOverviewTab, but the
platform-first rework left it unreachable: proxmoxPageModel routes PBS
resources only into the Backups summary servers table with no expansion,
no UnifiedResourceTable lists the resource, and neither the workloads
search nor the command palette can reach it. That is a dropped product
surface, not spec rot, so the spec stays as the contract and skips via
fixme until the drill-in returns.
Platform pages render websocket state, so the offline-node guard's REST
stubs never reached the surface it was checking; it also targeted the
retired /infrastructure route. The mock scenario already forces pve5
(Disaster Recovery B) offline deterministically, which is the exact
state the guard exists for: the spec now asserts on /proxmox that the
offline node stays listed with dashed live metrics while keeping its
web-interface link and detail expansion.
The upgrade-return spec asserted the pre-redesign panel: the plan
comparison summary paragraph, Compare plans link, activation summary
copy, and purchase-state URLs that survive verbatim. The current panel
presents plan cards with View plans links, the activation summary hands
off to Patrol mode selection, purchase-state params are consumed into
notices and normalized out of the URL, and a failed activation lands
directly on the recovery deep-link with the disclosure open. Six of the
seven flows are green against that reality; the unavailable-handoff flow
is marked fixme because it exposes a real defect (rel=noopener defeats
the original-tab reopen and the popup fallback strands users on /),
tracked for a governed product fix together with the still-legacy
checkout redirect paths.
The billing plan panel no longer renders a monitored-system capacity
review block; capacity truth surfaces at the point of adding systems
through the connection dialogs' impact previews, which the consolidated
workspace specs pin. The spec keeps its surviving guard: self-hosted
billing shows no upgrade pressure while monitored-system usage is
unavailable.
Specs 54 and 55 never authenticated; they only ever passed against an
already-authenticated dev session, so on the e2e stack every test landed
on the login screen before reaching the billing panel. Entry helpers now
establish a session first. Both specs still carry assertions against the
pre-redesign billing copy (upgrade arrival, capacity review wording);
that rework is tracked separately - this lands the auth prerequisite that
any version of those assertions needs.
The recovery layout spec pinned the retired standalone /recovery surface
down to its testids and the /api/recovery/series endpoint, none of which
exist anymore; backup activity lives in the Proxmox Backups section now.
The replacement guards the current surface against the same regression
class: picking an activity day narrows the backups table in place, a
year-long range renders its 365 bars without pushing the page into
horizontal overflow, and the PBS servers table keeps its trailing column
inside its wrapper on the default desktop column set.
The license-activation journey (env-gated, skipped without activation
credentials) still asserted the retired Assistant & Patrol heading on
/settings/system-ai; the panel renders Provider & Models now, per
settingsHeaderMeta. Copy-only alignment; the journey itself still needs
a live activation environment to run.
The provider-setup spec asserted the retired Assistant & Patrol shell:
old heading, old enable button, the old setup dialog names, and a Model
Overrides section that no longer renders (per-section overrides moved to
the Patrol, Assistant, and Service Context panels). The contracts are
re-pinned on the current flow: first enable opens the Set up Pulse
Intelligence dialog and submits provider credentials with no hardcoded
model, saved Patrol readiness warnings keep their provider and model
context, and a rejected save through Save provider settings keeps the
preflight recommendation in the failure alert.
The spec asserted the pre-rework AI settings shell (Assistant & Patrol
heading, Enable Assistant and Patrol button, and the retired setup
dialog). The durable contract survives on the current panel: pressing
Enable Pulse Intelligence without a configured provider issues no
settings update, no quickstart copy appears anywhere, and a direct
enable via the API still fails without resurrecting quickstart state.
The disclosure specs pinned pre-rework copy and routes: the telemetry
summary and PRIVACY.md were rewritten (one outbound usage-data scope),
chat action controls moved to the Assistant panel where the autonomous
option is gated on runtime autonomy (stubbed to the unblocked capability
set, since the e2e stack runs the community runtime), and the billing
Terms of Service link sits inside the collapsed manual key recovery
disclosure. With the .md content-type fix the shipped docs now render in
the popup, so the open-doc assertions exercise real navigation again.
The header-framing spec pinned four retired standalone routes; platform
pages intentionally carry no page header, so the contract now covers the
utility surfaces that still render the shared PageHeader (alerts overview,
settings general, and Patrol, whose description follows the current copy),
and the vertical-alignment check pins alerts against Patrol.
Five Patrol tests drove the retired Configure Patrol dialog and the
"Automation:" status strip, both removed with the monitor-first workbench,
so they could only time out on the dead affordances. The durable contracts
are re-pinned where the behavior lives now: a rejected Patrol settings save
surfaces the server reason from the settings panel, a save that returns a
not-ready Patrol model surfaces the readiness blocker with provider and
model context, and the Patrol mode group stays clamped to Watch only with
the Pro modes disabled when the runtime lacks autonomy, even when the
server reports a stale full autonomy level. The scoped-trigger strip
wording tests had no surviving surface; workbench presentation is covered
by the monitor-first workbench spec.
The per-platform connection workspaces under
/settings/infrastructure/platforms/* are retired compatibility paths;
connections now live in the consolidated Connected systems workspace, whose
table reads the unified /api/connections endpoint and whose add dialogs
still speak the unchanged per-platform REST APIs. The TrueNAS and VMware
specs are rewritten against that surface: table listing from
/api/connections, the add dialog's draft Test connection payload, the
monitored-system impact preview copy from monitoredSystemPresentation, the
capacity-denial alert (the server explanation now surfaces as a
notification while the dialog stays open), and the structured
unsupported-vCenter draft-test guidance, which kept its testid and copy in
the ConnectionEditor.
The demo-boundary spec's settings heading follows the same rename
(Infrastructure Operations -> Infrastructure); its remaining retired-route
references are tracked for the demo-contract pass.
On mobile viewports the settings navigation sits behind the Settings
drawer trigger rather than a persistent sidebar, and panel descriptions
are desktop-only copy (hidden sm:block). The spec now opens the drawer
before asserting the shared navigation and search on mobile projects and
scopes the description assertion to desktop, instead of failing on a
sidebar mobile intentionally does not show.
The perf spec measured Infrastructure/Workloads tab transitions, both
retired with the platform-first navigation, so it could only ever time out
looking for the removed infrastructure-page hook. It now measures
Proxmox/Docker primary-tab switches with per-platform ready signals and
keeps the same 2200ms default budgets (medians run 300-600ms on the local
stack). Budget env overrides follow the new direction names.
The standalone /infrastructure, /workloads, /storage, and /recovery routes
were deliberately retired with the platform-first navigation (abb6f86ae);
these specs existed to pin those routes' filter, handoff, and scoped-routing
contracts, so there is no platform-first surface for them to assert. The
subjects that do survive the rework are covered elsewhere: workload drawers
and filters by the platform-page and embedded-workloads specs, and refresh
resilience by the workloads stability spec being ported to the Proxmox page.
The settings shell copy moved (General, Billing & Usage, Provider & Models
and their descriptions); spec 15 now mirrors settingsHeaderMeta instead of
pre-rework titles. The first-run agent handoff pre-provisions the scoped
install token, so spec 11 asserts that contract rather than a Generate
token button, exempts the app shell's Patrol open-work polling from the
no-AI-bootstrap guard (it powers the Patrol nav badge on every route),
treats navigation-aborted auth probes as benign console noise, and targets
/settings/support/reporting directly now that the /operations alias is
retired. Spec 12 scopes migration-notice assertions to the settings content
because the global banner legitimately renders the same copy.
Specs 36/54/55 hardcoded the hot-dev URL (127.0.0.1:5173), which does not
exist in the dockerized stack or CI; they now resolve the shared runtime
base URL. Spec 36 also never authenticated (it only ever ran against an
already-authenticated dev session) and pinned fixture timestamps that aged
out of the history view's default period window.
The visual spec injects an inline stylesheet, which the nonce CSP active
outside hot-dev blocks, so it now runs with bypassCSP. The commercial
cancellation specs require a pre-provisioned live-Stripe fixture bundle
(test clocks, customer and subscription ids) that no CI environment
carries; they skip with the missing variables named instead of failing.
The first-session specs depend on /api/security/dev/reset-first-run, which
is gated on development mode; the test stack never set PULSE_DEV, so every
reset attempt answered 403 in both CI and local compose runs. The only other
effects of the flag in this stack are the localhost AllowedOrigins default
and env-gated dev toggles that remain off.
Playwright storage states persist cookies and localStorage only, while the
primary-API-token flow authenticates through sessionStorage. Storage states
captured after token auth were silently unauthenticated, so whole spec files
landed on the login screen whenever the token path happened to win. Storage
states now always come from the cookie-backed password login, cached once
per run and probed before reuse.
Explicit bearer/X-API-Token requests now go through a cookie-less request
context: the backend intentionally CSRF-rejects mutations that carry a
session cookie without a CSRF token even when the Authorization header is
valid, so token-scope coverage could never pass from a logged-in page.
Session-semantics specs use the new ensureSessionAuthenticated instead of
racing the token path.
First-run resets now refuse the implicitly resolved hot-dev runtime (the pid
fallback with no explicit base URL and no harness-written runtime state);
an ad-hoc spec run wiped the live dev backend's auth twice through that
path. Resets can also recover an instance stuck unauthenticated mid-reset by
re-reading the rotated bootstrap token through docker exec, and login()
retries through the backend's 10-per-minute login limiter instead of failing
on burst runs.
Proving run 28925348032 showed a shard can still hit the 45-minute job
timeout while the suite carries many real failures: shard 4 held the
visual crawl (5.3 minutes per attempt, currently failing) plus dozens of
14-second failing attempts, and with two retries the failure storm
outran the budget even under the 20-failure cap.
Retry once instead of twice on CI, and run the visual crawl only on the
desktop project. The mobile emulation projects keep their dedicated
mobile specs; a per-project 5-minute crawl was the single most expensive
line in the suite.
Every main push since the v6 branch flip was cancelled at the 45-minute
job timeout with no verdict. The flip brought the full 94-spec suite onto
main (the last green run, 2026-06-29, ran only 2 specs on the v5 main),
and it runs sequentially against a release-tagged image whose mock-fixture
gate returns 403 without a demo entitlement. Dozens of specs fail, retry
twice each, and burn the budget: of the 31 minutes of suite time in run
28907574469, 18.8 minutes were failing attempts.
- Add GO_BUILD_TAGS build arg (default release) and build the pulse:test
e2e image with it empty, matching the dev harness the suite is green
under. Shipped images keep the release tag; release-gate behavior keeps
its dedicated -tags release Go tests.
- Shard Playwright 4 ways across a CI matrix (214/202/205/203 tests per
shard) with per-shard report artifacts and an aggregate verdict job.
- Cap CI at 20 failures so an env-broken run reports red in minutes
instead of grinding into a no-verdict cancellation.
tests/integration/README.md and QUICK_START.md referenced the retired
.github/workflows/test-updates.yml; update-flow coverage now runs as
tests/79-update-flow.spec.ts inside the main suite via test-e2e.yml.
The Update Integration Tests workflow lost its Go test
(tests/integration/api) in the v6 release commit and was reduced to a
diagnostic smoke test that duplicated the test-e2e stack boot. Replace
it with tests/79-update-flow.spec.ts in the main suite, which runs via
test-e2e.yml on the same trigger paths:
- stable-channel check returns the mock v99.0.0 release and filters
the v99.1.0-rc.1 prerelease (regression guard for the auto-update
prerelease bug); rc-channel check surfaces the prerelease
- update plan reports honest manual instructions for the docker
deployment with readiness attached
- apply refuses prerelease download URLs on the stable channel (409)
- apply of an unsigned artifact fails closed at SSHSIG verification;
a completed update against the unsigned mock artifact would mean
the pinned-key trust root was bypassed
The old happy-path apply test is intentionally not revived: v6 made
SSHSIG verification against the pinned pulse-installer key mandatory,
so completing an apply would require shipping the real signing key to
the harness or weakening the trust root.
mock-github-server now serves v-prefixed asset names and download
paths like real Pulse releases (pulse-v99.0.0-linux-amd64.tar.gz);
the in-app updater only recognizes v-prefixed versions in download
URLs, so the old unprefixed shape made every apply fail validation
before reaching the paths under test. Unknown non-tarball sidecar
files (e.g. .sshsig) now 404 instead of falling back to tarball bytes.
The spec self-skips when the update check is not served by the mock
server, so managed-local-backend runs are unaffected.
Manifest-backed MCP tools, prompts, and resources with surface affordance contracts; agent capability manifest and governance projection; API contract tests and capability route projection; operations-loop and intelligence-funnel telemetry; release-control subsystem documentation, registry, and tooling; licensing and configuration.
Automates the docs/MSP.md validation checklist and the WEBHOOKS.md
delivery contract against the real server, covering the cross-layer
seams unit tests cannot see (today's two isolation bugs both lived
there):
- org-bound token allowed in its own org, denied 403 in sibling orgs
AND the default org (leaked client-site token must not read the
provider estate)
- dedicated agent-ingest port serves only /api/agents/* (management
paths 404, agent route auth-gated)
- client-org webhook delivery through the instance-wide private-target
allowlist saved in default-org context, with HMAC signature verified
against the documented recipe, X-Pulse-Event-ID idempotency header,
and tenant id/name stamped in the payload
- restart inheritance: after docker restart the persisted allowlist
still applies to lazily recreated tenant monitors (skips when the
docker CLI cannot manage the container)
Compose: pulse-test gains PULSE_AGENT_INGEST_PORT=7656 (+ port map)
and a host-gateway extra_host so the spec's capture listener is
reachable from the container.
golangci-lint run ./... failed on ~190 pre-existing errcheck violations and
5 unformatted files, burying any new regression in noise. Fix all of them:
- Test files that hand-rolled mock-mode set/restore (vmware, truenas, and
friends) now use the canonical setMockModeForTest/testutil.SetMockMode
helper instead of drift copies that ignored SetEnabled errors.
- internal/mock and internal/monitoring tests get package-local
mustSetEnabled/mustSetMockEnabled/mustSetMonitorMockMode helpers that
fail the test on toggle errors.
- pkg/auth/sqlite_manager.go, pkg/metrics/store.go, pkg/server/server.go:
rollbacks in defers use the explicit-discard idiom, migration renames and
rollup commits log failures, the hosted reaper goroutine logs an error
exit, shutdown mock-disable logs failures.
- Remaining test sites check errors with t.Fatalf/t.Errorf or explicitly
discard best-effort calls (restore-chmods, handler-closure unmarshals)
per existing repo style.
- gofmt: internal/api/maintenance_verification.go, internal/ai/demo.go and
three findings test files.
Only dupl findings remain (44 pre-existing production-code duplication
pairs) — those need real refactors, not mechanical fixes.
Full test suites pass for every touched package.
