A guest's unified canonical ID hashed its node-scoped source ID
(instance:node:vmid), so a live migration to another cluster node
re-minted the resource and orphaned every operator-owned row keyed by
the old ID: explicit availability check links (fail-closed by design,
the reported symptom in #1669), alert overrides, operator state, action
audits, manual links, and recovery subjects. VMIDs are unique within a
cluster, so guests now derive their canonical ID from instance+VMID
("proxmox-guest:<instance>:<vmid>") and keep it across migrations. The
guest-metadata half of #1669 was fixed separately at the metadata-store
layer.
Existing installs converge through record-declared succession: ingest
declares the retired node-scoped IDs superseded for every node the
instance currently knows (current names plus native aliases), so rows
orphaned by pre-upgrade migrations also re-key. Successions are now
recorded durably in a canonical_id_successions table, which memoizes
the re-key (steady-state rebuilds re-declare the same eras every tick
without touching SQL) and lets change-journal reads merge retired guest
eras the way pin EraIDs do for hosts. The succession re-key also covers
manual link/exclusion rows.
Availability links resolve retired canonical IDs and old-node source
triples through a registry superseded index plus guest-triple parsing
(persistence keys only, ambiguity fails closed), and the stored
LinkedResourceID re-homes to the current canonical ID on the
alert-migration cadence. Recovery subjects converge on the same
derivation via CanonicalSubjectResourceID, the mapper's registry-miss
fallback, node-independent external guest keys, and the store's startup
backfill, which also sweeps posture rows stranded under retired subject
keys. Metrics history and frontend row identity key off the node-scoped
source ID and are deliberately unchanged.
Public declaration of how AI tools are used to build and maintain
Pulse, linked from the README documentation list and the FAQ.
Automated triage replies now carry a disclosure footer per the
policy this document sets.
Every unattended update created another full config snapshot under
config-backups (or next to the config dir) and nothing ever pruned old
ones, so small root filesystems filled up within days (#1646, reported
on the hardened-unit fallback path where snapshots land under the
install dir). backup_existing now keeps the five newest snapshots and
removes the rest after each successful copy.
Contract-Neutral: installer config backup rotation; shell-only fix, no runtime contract
TrueNAS reserves most RAM for the ZFS ARC and the kernel's MemAvailable
does not count ARC even though it shrinks under memory pressure, so
memory derived from total minus available read ~95% used on every ZFS
system and masked the real pressure (#1668). The realtime telemetry
Pulse already subscribes to carries arc_size, so parse it, subtract the
ARC from effective used the same way node accounting treats buffers and
page cache, and surface it through the existing agent memory cache
field. The TrueNAS systems table stacks the ARC as a reclaimable cache
segment with a Used with ARC cache tooltip row, and the reporting
history query now fetches the arcsize graph so the memory history
subtracts the ARC per point instead of contradicting the current value.
Contract-Neutral: TrueNAS memory accounting treats ZFS ARC as reclaimable cache; existing cache field, no schema change
Guest metadata (web interface URL, tags, notes) is keyed by the
node-scoped guest ID, so a VM or container migrating to another node in
the same cluster orphaned everything attached to it (#1669). VMIDs are
unique within a Proxmox cluster, so when the exact ID and the legacy
formats all miss, an entry for the same instance and VMID under a
different node is the same guest and migrates to the new node-scoped ID.
The explicit availability check link stores the hashed canonical
resource ID, which also embeds the node; that half needs a
node-independent canonical guest identity and is tracked separately.
Contract-Neutral: guest metadata follows node migration; behavioral bug fix, no wire contract or payload change
A hostname change on the same machine and token forked the host module
onto a suffixed identity to stay safe against cloned VMs sharing a
machine ID (#1584), but the fork never healed, so one machine stayed
split into two agent identities forever. In unified installs the
workload modules keep reporting under the base agent ID, so Agent
Doctor flagged both halves with no way to merge them (#1667, Home
Assistant add-on with agent_hostname set after first boot).
Once the pre-rename record has stopped reporting for three health
windows the collision is proven to be a rename, not a clone: the stale
record and the forked identity are superseded and the binding returns
to the base identity. A rename discovered while the old record is
already stale now adopts the base identity without forking at all.
Live clones keep reporting under the old hostname, so they never
match the heal condition and stay forked.
Contract-Neutral: host agent identity rename heal; behavioral bug fix, no wire contract or payload change
A node joining a PVE cluster after setup could never be trusted when the
primary was fingerprint-pinned (#1664). Two independent gaps: cluster
discovery validation only relaxed TLS when client construction failed,
but a pinned-fingerprint mismatch surfaces from the first API call, so
the member was judged not-a-Proxmox-node and its captured fingerprint
discarded; and the cluster client's TOFU refresh only ran for endpoints
that already had a per-endpoint fingerprint, handling rotation but never
first trust. Validation now retries with the member's own captured
fingerprint before rejecting it, TOFU capture runs on first use, and
discovery failures now distinguish DNS, refused, timeout, and TLS
causes in the endpoint error instead of one generic message.
Contract-Neutral: cluster discovery TLS validation bug fix; no wire contract or payload change
install.ps1 set ServicePointManager's certificate callback to a
PowerShell scriptblock. ServicePointManager can invoke that callback on
a worker thread that has no runspace, where a scriptblock delegate
fails closed, so agent downloads died with a TLS error even though the
certificate policy would have accepted them (hit by an rc.6 tester with
Skip TLS verification enabled). The copied install command already
migrated to a compiled validator for exactly this reason in rc.6; the
downloaded script now does the same. Validation policy is unchanged,
with fingerprint pinning first, then insecure, then clean chains, then
the custom CA. Verified with pwsh covering all four policy branches
plus the callback running on a runspace-free worker thread.
Contract-Neutral: installer TLS callback implementation moved to a compiled type; same validation policy, no payload or endpoint change
Console hosts execute pasted input line by line, so the literal
newlines inside the embedded certificate-validator C# broke the copied
command in both Windows PowerShell 5.1 and PowerShell 7 (reported by an
rc.6 tester who had to join it manually before it would run). C# is
whitespace insensitive and the source carries no line comments, so the
type definition now collapses onto the command's single line, and a
test pins every command variant to stay newline free.
Contract-Neutral: copied command formatting only; same PowerShell semantics, no payload or endpoint change
The agent intentionally reports the digest-pinned image sentinel for
image@sha256 references, where there is no tag to resolve against the
registry. Every surface funneled that through the error branch and
rendered a danger-toned Check failed badge for a state that is not a
failure (raised in #1666). The containers table, the images table, and
the container drawer now render a neutral Pinned state with a tooltip
saying why checks do not apply.
Mock containers also gain update check states (current, update
available, digest-pinned) so these badges are exercisable in mock mode
and on the demo, which previously never populated updateStatus at all.
Contract-Neutral: frontend presentation of the existing digest-pinned sentinel plus mock fixture variety; no wire contract change
Every /api/state request and websocket hydrate ran a full unified
registry rebuild while holding the adapter's mutation lock, including
synchronous SQLite writes for change records and identity pins plus
re-reads of overrides and pins on registry construction. On slow
volumes a single rebuild holds the lock for the duration of those
transactions, and every state read queues behind it. #1665 hit exactly
this with the data dir on NFS, where the same instance's metrics logs
show single commits taking 30+ seconds, and the UI sat on /api/state
for minutes.
Read paths now refresh through TryReplaceRegistryForRead. It skips
while the current generation is younger than two seconds, collapsing
same-cycle rebuild storms, and it never queues behind an in-flight
ingest rebuild since that rebuild is already publishing a generation
at least as fresh. Consume-once supplemental payloads are only drained
once a rebuild commits. Cold start still blocks and builds the first
generation so a fresh session cannot render empty. Ingest boundaries
keep rebuilding eagerly through the unchanged populate methods.
Contract-Neutral: internal registry rebuild scheduling on the state read path; no wire contract or payload change
The digest-drift refusal only said the digest no longer matched, which
left no way to tell a genuine plan/image drift from a comparison bug
(#1666 shipped refusals on every classic-overlay2 host for exactly that
reason, and the report had to correlate image stores to get close).
Include the planned digest plus the local image id and repo digest in
the refusal so the action record itself carries the evidence.
Contract-Neutral: agent preflight refusal message detail only; no payload or schema change, error stays within the existing 1024-byte bounded field
The prerelease runbook now tells the operator that a red Definitive
Release Verdict caused only by a promotion-only publish_private_pro_runtime
failure is recovered by rerunning the public run's failed jobs, which
reuses the signed packet the earlier attempt uploaded, and warns against
hand-dispatching a fresh Build Pro Release at the same prefix. Behavior
contracted in 9d2c49f3e.
Contract-Neutral: documentation-only runbook note for promotion-only rerun recovery; contract + installtests proof deltas already landed with the behavior in 9d2c49f3e, this commit changes operator docs only
The paid-runtime R2 prefix was derived from the wall-clock date, and the
enterprise build was always dispatched as a fresh rebuild. Rerunning the
failed publish job after a transient promotion failure therefore hit the
enterprise R2 overwrite guard (same run id, existing prefix) and the
Definitive Release Verdict could never go green by rerun, as on
v6.2.0-rc.6. The prefix date now comes from the run's createdAt, which
is stable across rerun attempts, and the build dispatch passes
reuse_existing_packet=true so pulse-enterprise validates the packet the
earlier attempt uploaded, skips the rebuild, and only the promotion
re-executes against it. The deployment-installability contract now
records rerun recoverability as part of the promotion boundary.
Transport-level failures from httpClient.Do (connection refused/reset,
DNS, TLS, timeouts) were wrapped as plain errors, so they fell through
userFriendlyActivationError's LicenseServerError branch to the generic
contact-support fallback instead of the retryable temporarily-unavailable
message. Users behind flaky networks got a dead-end message, and the
2026-08-01 rc.6 paid-runtime gate failure was misdiagnosed the same way.
Classify Do failures at the client layer as a retryable LicenseServerError
(code license_server_unreachable, StatusCode 0 so revocation/suspension/
migration status gates never match) with the cause reachable via Unwrap so
the status poller's context.Canceled check keeps working. The cloud-paid
transport boundary contract now names this classification. Tests pin the
classification, the cancellation chain, and the end-to-end user-facing
retryable message.
The rc.6 notes were authored at the prepare commit, before the container
update-check backoff, the entitled Pro registry skip, and the two
identity-based PBS row merge changes landed. The notes and per-RC
changelog now describe all of them so the published packet matches the
tagged tip.
Connected systems showed a PBS machine twice (API row plus host-agent row)
whenever the PBS connection was configured with an address the agent never
reports, because PBS identity was limited to the configured name and host.
The PBS poll now captures the hostname the node reports about itself
(GET /nodes, the previously unused client GetNodeName) on
models.PBSInstance.NodeName, and /api/connections includes it in the PBS
connection's host aliases, so directPlatformHostAttachment merges the agent
running on that machine into the PBS row the same way PVE composes from
API-reported node identity. Node-name fetch failure stays partial data and
never fails the poll. Reported by Johannes Strasser (rc.5).
A PBS instance whose host also runs a Pulse agent showed twice under
Settings > Infrastructure > Connected systems: once as the API
connection and once as an agent host, while the equivalent PVE setup
collapses into one API + Agent row. Two defects in the connections
grouping caused this.
The PBS and PMG pollers write pbs-<name>/pmg-<name> unified instance
IDs while connection IDs carry the bare configured name, so
primaryConnectionIDForResource built connection IDs that never matched
and the unified-resource attachment pass could not see either platform.
Beyond that, agent-to-PBS pairing only had the literal host-string
fallback, which fails when the connection is configured with an FQDN or
DNS alias the agent never reports. Attachment now also consumes the
shared top-level-system resolver, whose same-machine contract (machine
identity, short-form hostname equivalence, unique IPs) is what the
Infrastructure surfaces already use to count such a machine as one
system. Groups naming more than one platform connection fail closed,
and Proxmox stays with the LinkedAgentID passes and their
cluster-membership semantics.
Reported by Johannes Strasser against v6.2.0-rc.5 (pbs01/pbs02 each
listed twice).
The registry checker cached rate-limit refusals on the 15 minute
transient-error TTL, and the per-container check runs every collection
cycle, so once a strict registry refused a lookup the agent retried
every 15 minutes. Each refused HEAD still counts against the registry's
allowance, which can hold the limit tripped indefinitely. A design
partner saw exactly this on docker.n8n.io, where the rate limited badge
never cleared.
Cache rate-limited lookups for an hour instead so the allowance can
recover; other transient errors keep the short TTL.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT: behavioral backoff fix; no payload or contract delta
Contract-Neutral: behavioral backoff fix; no payload or contract delta
The Docker agent's registry checker HEADs manifests anonymously. The
Pulse Pro image lives on license.pulserelay.pro, whose token endpoint
requires a license credential the agent does not hold, so the check can
never succeed and every Pro Docker deployment pinned a permanent
"authentication required" badge on its own Pulse container. That
container updates through the broker's digest-pinned commands, not the
generic checker.
Report nothing for that registry instead, the same way a disabled
checker does, so no badge renders. Reported by a design partner on
6.2.0-rc.4.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT: behavioral fix suppressing a structurally impossible check; no payload or contract delta
Contract-Neutral: behavioral fix suppressing a structurally impossible check; no payload or contract delta
The unified resources API serializes most numeric and struct fields with
omitempty, so zero values arrive absent on the wire. Several frontend
types embedded in the resource facets still declared those fields as
required, which is the same class of mismatch that crashed the
Thresholds page in rc.5 (#1663): the type promised presence, fixtures
dutifully supplied it, and the first real payload without the field
broke a consumer.
Swept against internal/unifiedresources/types.go json tags:
- Disk: total/used/free/usage optional (DiskInfo omits them all), and
declare filesystem, the key unified payloads actually carry - the
legacy state payload keeps type. The Thresholds guest-disk projection
now reads filesystem first.
- HostRAIDArray: devices and rebuildPercent optional (HostRAIDMeta
serializes both omitempty).
- PBS job types: vmid/nextRun/removedBytes/error optional to match the
models tags the pbs facet passes through verbatim.
Consumers assuming presence now compute or default instead of
dereferencing: normalizeDiskArray/getResourceDiskSummary return a new
NormalizedDisk type that guarantees solid numerics at the type level,
and the disk/RAID render paths default absent values to zero the same
way the 15bb0c217 fix did.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT: frontend-only type alignment with existing omitempty wire serialization; no wire payload or contract delta
Contract-Neutral: frontend-only type alignment with existing omitempty wire serialization; no wire payload or contract delta
v6.2.0-rc.5 shipped with an empty Proxmox workloads table and a crashing
Thresholds page (#1663) while the assertions that catch exactly that were
failing in CI: spec 64 red in the non-gating probation tier two hours
before the tag, and the gating Core E2E verdict red on the release commit
itself — which the release pipeline never consults. integration_tests is
also skipped entirely for prereleases, so the builds users test shipped
with no integration coverage at all.
Close the hole with a release_smoke job that runs for every cut,
prereleases included, and blocks create_release and the release verdict:
four interaction-free render assertions (Proxmox nodes+workloads, Docker
hosts+containers, Kubernetes clusters+pods, Alert thresholds) against the
mock-mode image built from the verified frontend bundle. The mock fixture
graph now always contains one freshly provisioned zero-used guest
filesystem, so the omitted-zero-numerics wire shape that crashed rc.5
stays exercised on every mock-backed surface. Verified locally: the suite
passes on main and fails on the rc.5 frontend for exactly the two shipped
regressions.
A box with the binary still at /opt/pulse/bin/pulse but /etc/pulse, the
systemd unit and the /usr/local/bin symlink deleted by hand takes the
update path ("Reinstalling version ..."), which assumed the previous
install's environment still existed. With auto-updates enabled the run
died at setup_auto_updates writing system.json into the missing config
dir; without them it printed a success completion while systemctl
enable/start had failed with "Unit pulse.service could not be found",
softened into the unprivileged-container note (#1663).
Both the --version and menu update flows now run setup_directories and
recreate the unit file when it is missing (an existing unit stays
untouched so user customizations survive normal updates), and
start_pulse refuses to report success when the unit does not exist at
all. Documented as a deployment-installability contract invariant.
The unified resources payload serializes guest filesystem numerics with
omitempty, so a disk whose usage is exactly zero arrives with the field
absent. The guest filesystem projection guarded with usage < 0, which
undefined passes, then dereferenced usage.toFixed and took down the whole
Alerts > Thresholds page behind the error boundary (#1663).
Treat absent numerics as zero and derive the percentage from used/total
when the serialized value is missing; a negative usage still means the
poller could not read the filesystem and stays excluded.
The PBS host physical disks change pointed the Proxmox page's forced
platform filter at the proxmox-all aggregate but only taught the storage
source matcher its prefix semantics. The workloads matcher still compared
it for equality against concrete guest scopes (proxmox-pve), so every VM
and LXC failed the filter and the Proxmox overview rendered 'No Proxmox
workloads' on every install (#1663).
Resolve the aggregate in one canonical predicate,
sourcePlatformScopeMatchesFilter, and use it from both the workloads and
storage matchers.
isBlockedFetchIP had the same bypass as the audit webhook validator and the
restricted outbound transport fixed in 70d275288: every net.IP predicate it
uses reads only the literal address bytes, so 64:ff9b::a9fe:a9fe fetched
169.254.169.254 while looking like ordinary global unicast.
Reuse securityutil.EmbeddedIPv4Candidates rather than growing a second
AI-local list of transition prefixes, and hold each embedded destination to
the same policy as the outer address so PULSE_AI_ALLOW_LOOPBACK and
PULSE_AI_ALLOW_PRIVATE_IPS keep working through the wrapper.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The net.IP predicates every SSRF guard is built from read only the literal
address bytes, so an IPv6 transition address smuggles an internal IPv4
destination past all of them. 64:ff9b::a9fe:a9fe reaches 169.254.169.254
while IsLoopback, IsPrivate, IsLinkLocalUnicast and To4 all report an
ordinary public address, defeating both the webhook URL validator and the
restricted outbound transport with the same input.
Add securityutil.EmbeddedIPv4Candidates, which unwraps NAT64 (RFC 6052
well-known and RFC 8215 local-use prefixes), 6to4, Teredo, ISATAP,
IPv4-compatible and IPv4-translated encodings, and run every candidate
through the caller's own policy in both layers. The embedded destination
inherits the outer policy rather than a stricter one, so a NAT64 address
wrapping a permitted public target stays permitted and AllowPrivateIPs /
AllowLoopback relax the embedded check the same way they relax the outer.
Reported by tonghuaroot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
