Fleets on networks numbered from nominally public IP space (issue
#1522: an AD estate on 192.20.0.0/16) cannot pass the agent's
local-network plaintext heuristic, and the only workaround was pointing
a .internal DNS alias at the server, which bypasses the same control
less visibly than a flag would. --allow-plaintext-http
(PULSE_AGENT_ALLOW_PLAINTEXT_HTTP) records process-wide consent once at
agent startup before any module validates a URL, covers every agent
transport including the websocket command channel, warns at startup
that the API token travels in cleartext, defaults closed, and is never
emitted by generated install commands or settable by the server.
A pool built from /dev/disk/by-id/nvme-eui.<hex> references (the
installer's device naming when identical NVMe models share a box)
produced no serial key in the disk-to-pool matcher, and the WWN lookup
never stripped the eui. prefix smartctl reports, so the disk fell back
to the generic Proxmox usage string and showed 'ZFS' instead of its
pool name while an identically-built node showed 'local-zfs' (issue
#1540).
RecoverExecutingActions existed with full test coverage but had no
production caller, so any typed action mid-dispatch across a server
restart (container update, start/stop/restart, host update, storage
cleanup) stayed in the executing state forever and sat in the Actions
inbox as live work, even after the agent persisted its terminal durable
receipt. Reproduced live on the dev instance with a Docker container
update (act_bf77dfe860ad3d8e4e0a91dc8eb83b44).
The router now runs a bounded, serialized recovery pass per organization
from a startup background worker, and again whenever an agent
(re)registers on the agentexec command server via a new registration
notifier, because a receipt-pending attempt can only be reconciled while
the owning agent is connected. Both triggers reuse the existing
query-only reconciliation semantics; nothing gains a resend authority.
Task 07 owns this residual; the api-contracts and agent-lifecycle
subsystem contracts now record the production trigger. The
rg-07-durable-delivery gate suite stays green, and a new router-level
test pins that a receipt-pending executing action completes from the
agent receipt without a second dispatch.
098ba4eaa changed plan resource versions to hash relationship identity
instead of observation stamps, and updated the planner unit tests, but
missed the pinned API contract snapshot: its fixture carries a vm-to-node
relationship, so the resourceVersion (and the actionId, decisionId, and
planHash derived from it) legitimately changed. TestContract_ActionPlanJSONSnapshot
has been red on main since that commit; this re-pins the snapshot to the
deterministic values the new hashing produces.
Docker adapters restamp relationship ObservedAt/LastSeenAt on every
~15s report, and the action planner folded those stamps into the plan's
resource version, so any reviewed action against a relationship-bearing
container (start, stop, restart, and the restored update) drifted to a
409 action_plan_drift before a human could read the review dialog and
click approve. Relationship edges now count by identity (source,
target, type, active, discoverer, metadata), the same
identity-versus-timestamp boundary change emission drew for issue
#1496. Found live: the UI update journey failed with plan drift on
every attempt slower than one report cycle.
v6.1.0-rc.1 retired the legacy update endpoints before a replacement
existed, so the UI's Update button failed with an internal-jargon 410
(issue #1564). This lands the replacement end to end: update_container
is a typed agentexec operation with its own strict codec, durable
receipts, and a request digest bound to the image digest the plan
observed; the unified agent bridges execution to the Docker module's
existing pull/backup/recreate/verify/rollback implementation (which now
reports rollback attempt and outcome); and the container action
executor plans, dispatches, and reconciles the operation with declared
backup/rollback compensation truth. Containers advertise an
admin-approval update capability while an image update with a stated
current digest is detected. The legacy endpoints stay retired but
return actionable copy.
Proven live against a Colima daemon: single-container update, the
issue-1564 shared-network-namespace update, and the full UI journey
(Update button, governed review, approve, run) all completed with the
namespace preserved and the backup retained.
URLs saved through the resource drawer historically landed in the docker
store under the runtime container key, which any stable record (including
an intentionally empty cleared one) outranks in the unified customUrl
projection, and which orphans on container recreation. On report ingest,
copy those records into the stable app-container guest key when it is
missing (cleared links stay cleared), healing saves stranded before the
drawer moved to the stable identity. Also read the runtime key before its
copy-if-missing container-name snapshot so the freshest write wins among
the docker-store fallbacks.
Refs #1556
Thinking models served through OpenAI-compatible endpoints (qwen3 via
Ollama >=0.31, DeepSeek) spend most of a turn streaming per-token
reasoning deltas before any content. The stream reader wrapped the whole
turn in a wall-clock deadline at the configured request timeout, so a
live, visibly-thinking stream was killed mid-thought with 'AI response
timed out before completion' (#1576, second symptom).
The timeout's job is stall detection, not turn budgeting. It now bounds
only how long Pulse waits for the stream to start: the response-header
wait (Ollama holds headers while a cold model loads) and the first-chunk
wait both honor the configured timeout, completing what 1c0648451
started. Once deltas flow, the 12s inter-chunk stall bound and caller
cancellation are the only limits, matching the native Ollama provider's
long-standing design.
Verified live against Ollama 0.31.1 / qwen3:8b: with a 15s configured
timeout, a turn streaming 10k chars of reasoning over 4m17s completes
and delivers the answer; previously it died at the timeout.
Two TrueNAS systems that report the same hostname collapsed into one
flapping resource (#1573, #1575): systemSourceID keyed the system by the
snapshot-reported hostname, every child pool/dataset/app/VM/share/disk
was scoped under it, and the client minted the system's machine key from
the DMI serial with a hostname fallback, so serial-less systems sharing
a hostname (and DR clones sharing a serial) also fully merged in the
identity matcher.
The system source ID now scopes to the connection ID the poller passes
through NewLiveProviderForConnection; the hostname arm survives only for
fixture snapshots, which carry no connection. The ingest identity drops
the machine key entirely (DR clones share DMI serials, and vendor
placeholder serials collide across unrelated machines), the client no
longer falls back to the hostname for MachineID, and ingest skips
identity-pin completion for SourceTrueNAS so a stale pre-fix pin or a
same-named agent host's pin cannot lend the system a machine key and
re-merge what connection scoping keeps apart. Agent.AgentID and the
native metric history keys follow the source ID minus its system:
prefix, so BuildMetricsTarget keeps resolving one series.
Rows minted under the retired hostname-keyed derivation re-key once via
record-declared succession: records name their old canonical IDs in
IngestRecord.SupersededCanonicalIDs and IngestRecords applies the
existing ApplyCanonicalIDSuccessions semantics (operator state and
action audits re-key, the superseded pin drops, never while the old ID
still belongs to a live resource, journal rows are never rewritten).
Alert identities and persisted metric series under old child IDs are
not re-keyed: active alerts re-arm under the new IDs and TrueNAS host
charts are backed by native read-through history.
Ollama's /api/chat returns reasoning in message.thinking, but the native
provider had no field for it: thinking models like qwen3 (the quickstart
default) showed dead air in the Assistant drawer for the whole reasoning
phase. Decode the field, stream it as thinking events (matching the
OpenAI provider's reasoning handling), carry it on the non-streaming
response as ReasoningContent, round-trip prior-turn reasoning on
assistant history messages, and make SupportsThinking tell the truth.
Verified live against Ollama 0.31.1 with qwen3:8b: first stream event is
now thinking (1137 thinking chunks before content), and non-streaming
Chat returns the reasoning text.
The OpenAI-compatible stream reader bounded every chunk wait at 12s.
Local backends (LM Studio, llama.cpp) legitimately spend minutes on
prompt processing before the first SSE chunk, so raising the provider
timeout in Settings changed nothing and Pulse dropped the stream with
'AI response timed out' (discussion #1571). The wait for first bytes now
uses the configured request timeout; the 12s bound still applies to
inter-chunk gaps once the stream is flowing.
A provider that issues no refresh token (offline_access not requested,
e.g. default Authelia) left session.OIDCIssuer unset, so
/api/security/status reported an empty ssoSessionUsername and
X-Auth-Method fell back to plain session. The frontend then skipped the
SSO fast path, and the pre-auth bootstrap short-circuit pinned the user
to the login page after ?oidc=success before the /api/state probe could
run (issue #1574). Stamp issuer/client on every OIDC session (refresh
stays gated on the token being present) and let a completed SSO
callback bypass the bootstrap short-circuit.
dockerUpdateStatusChanged compared the whole DockerUpdateStatusMeta with
reflect.DeepEqual, so the LastChecked stamp refreshed by every periodic
update check emitted a resource_changes row per container per cycle and
flooded unified_resources.db (discussion #1577, same class as #1496).
Only availability, digests, and error text count as change now.
The relay client's HTTP proxy dialed http://127.0.0.1:<FrontendPort> for
every proxied mobile request. With HTTPS_ENABLED the main listener serves
TLS on that port, so Go answered each plaintext dial with a bare
"Client sent an HTTP request to an HTTPS server" 400 - breaking Remote
Access backlog sync (alerts/approvals) on every HTTPS-enabled instance.
A non-loopback BIND_ADDRESS broke the same dial outright.
Route proxied requests through the router's own handler chain in-process
instead, via a streaming-capable RoundTripper (pipe-backed, SSE flush,
panic recovery, loopback RemoteAddr for address-keyed middleware). The
listener's scheme and bind address no longer matter, and the request
traverses exactly the middleware the real listener serves.
Reported by Johannes Strasser (Remote Access thread, 2026-07-14).
Cover the uncovered branches of migrateActionAuditCanonicalization (store.go):
idempotent no-op skip + len(updates)==0 early return, terminal-state exclusion
(expired not in the WHERE clause), mixed-batch skip-vs-update partition, and the
un-normalizable strict-error WARN branch (with an in-test honesty guard that
fails if the fixture stops reaching the strict path). The shipped repro test
covers only the happy path. New _test.go only; no source or existing tests
touched. Adds 4 tests; go test/vet/gofmt green.
Lifecycle transitions compare stored request/plan/origin JSON byte-for-byte
as their concurrent-writer guard. Rows written by v6.0.x re-marshal
differently after read-time normalization (backfilled plan expiry,
legacy-unknown policy decision, approval requirement), so every transition
on them matched zero rows and was swallowed as already-final. The visible
symptom: pre-upgrade pending approvals sit in the Actions inbox forever
with no way to clear them - the review dialog blocks approve/reject by
design for records without policy provenance, and the expiry sweep
silently failed to retire them.
Rewrite non-terminal audit rows to the current canonical JSON shape once
at store open; the existing expiry sweep then moves stale pre-upgrade
approvals to History on the next inbox load. Rows that cannot be expressed
in the current shape are left as stored and logged.
Reported by Johannes Strasser on v6.1.0-rc.1.
Guest alerts are keyed by node-scoped resource IDs, and only metric
threshold alerts were re-homed after a live migration. Lifecycle alerts
(powered-state and future guest kinds) stayed keyed to the old node
forever because their resolvers only look up the new-node key, node
existence GC never fires while the source node is still online, and
nothing else consumes the move.
Generalize the migration helper to match lifecycle spec IDs (which
embed the node-scoped resource ID) as well as node-independent metric
spec IDs, and call it from the canonical lifecycle and stateful
evaluators, the powered-off resolver, and the disabled-threshold early
returns in both metric paths. Guest-wide sweep clears (metric clear on
stop, per-disk cleanup, tag suppression) now match the stable
instance+vmid identity across nodes too.
Reported by Johannes Strasser: a VM live-migrated between Proxmox
nodes stranded its pre-existing active alert with no way to clear it.
The v6.0.5 host-adoption fix (f85009913) only applies when a
re-registering agent matches a top-level instance host, so an agent on a
non-primary cluster member never benefited: its registration created a
standalone instance that ConsolidatePVEInstances folded back into the
cluster, and the fill-empty endpoint merge silently discarded the agent's
fresh address. The member row kept showing the corosync short-DNS host
rebuilt on every re-discovery (the "Install issues with V6" support
thread; a reinstall on v6.0.5 still showed the stale name).
Canonical auto-register now matches cluster member endpoints directly:
address identity against the agent's candidate list first, then an
unambiguous corosync node-name match. The Pulse-verified selected host is
adopted as the member's IPOverride, the durable field re-discovery
preserves and polling prefers, plus the fingerprint captured from that
address. An admin-managed override absent from the candidate list is
preserved, mirroring shouldPreserveExistingAutoRegisterHost. Credential
writes stay restricted to a same-token-identity secret refresh (reinstall
rotates the agent's token in place, so the stored secret is already
invalid) and full promotion onto a credential-less cluster; a member's
distinct per-node token never replaces working cluster credentials, and
no standalone instance is created for consolidation to discard.
When fetchVMFSInfo fails (agent not running, timeout, permission
denied, no filesystems), the builder passed the cluster/resources
numbers through — and PVE always reports 0 used for QEMU — so the UI
rendered a confident "0% (0 B/<allocated>)" for every affected VM
instead of the unavailable dash plus diskStatusReason tooltip. The
agent-disabled path already used the -1 sentinel for exactly this;
apply it on the error path too. The stabilizer can still replace the
sentinel with the previous good sample when recent agent evidence
exists.
Reported by Massimo Simoni (support, 2026-07-13): screenshot showed
every VM at 0% disk with only containers reporting real usage.
A follow-up sent during an active run now offers itself to the running
agentic loop via POST /api/ai/sessions/{id}/steer. Accepted steers join
the loop at its next turn boundary (the abort-check site) as plain user
messages, are announced with a steer_applied stream event so the drawer
settles the pending row, and persist through the end-of-run save. A
steer carries prompt text only: no route, control-level, or autonomy
changes, no turn-budget extension, system sessions rejected, and the
per-session inbox is bounded (steer_backlog overflow). Delivery is not
guaranteed by acceptance: a run that ends first discards the inbox and
the row drains as an ordinary queued turn, so pre-steering queue
semantics remain the fallback. Steering rows lose edit/remove once
accepted.
Chat turns, compaction, and title calls now stamp session_id on their
cost.UsageEvent, and the done event carries session_cost_usd summed from
the operator ledger (cost.Store.SessionCostUSD). The drawer's last-turn
summary appends '$0.12 session' with a sub-cent floor. The figure is
omitted whenever any of the session's models has unknown pricing, and
free local models price known-at-zero, so no figure is ever partial.
Retrying a failed turn re-sent the prompt without removing the persisted
turn, so session history double-recorded the prompt. Session undo now
accepts an expected-prompt guard (a stale retry can never remove a
different turn); retry drops the replaced turn server-side before
re-sending. The latest settled assistant answer gains a hover-revealed
Regenerate button that reuses the same path.
