Write the deep-dive docs the AI pages promised

AI.md, AI_AUTONOMY.md and PULSE_PRO.md linked nine times into
docs/architecture/, which .gitignore marks as not for public release. The
targets were never missing, they were deliberately unpublished, so every one
of those links was dead for anybody but the maintainer.

Write the three promised documents against the code and publish them under
docs/ where the rest of the shipped set lives.

PATROL_ARCHITECTURE.md covers a run end to end. The interesting part is that
deterministic signal detection runs after the model, not before, so unmatched
signals catch what the model failed to file. Signal types, the thresholds
those signals derive from your own alert settings, and every condition in
Finding.ShouldInvestigate are documented from internal/ai/patrol_signals.go
and internal/ai/findings.go.

ASSISTANT_SAFETY.md documents the session state machine in
internal/ai/chat/fsm.go, its four states, the tool classification it runs on,
and its invariants. No write without a validated target, no second write
before the first is verified, no final answer about an unverified change, and
no attempt count that wears the gate down.

ASSISTANT_ARCHITECTURE.md covers the agentic loop around that machine, the
three-phase pipeline, why only execution parallelises and at what cap, the
read-before-write batch that must stay ordered, the look-before-asking gate
and its bound, and the stable error codes.

Also removed two older pointers into the same private directory, in API.md
and UPGRADE_v6.md, and the two references to ENTITLEMENT_MATRIX.md in
PULSE_PRO.md. That file exists locally and is a monetization document, so
publishing it is not a documentation decision.

Unresolvable intra-doc links are now 2 of 218, both internal release-control
documents deliberately withheld from the shipped set.

Contract-Neutral: documentation only
This commit is contained in:
rcourtman 2026-08-03 22:19:38 +01:00
parent d4609665dc
commit 63fc89f701
16 changed files with 726 additions and 30 deletions

View file

@ -29,7 +29,7 @@ Pulse Assistant is a **protocol-driven, safety-gated LLM tool surface** that:
- **Grounds answers in real tool work** — visible tool traces, read-after-write verification, and transcript hygiene prevent unsupported execution claims from being treated as facts
- **Returns structured tool errors** — the model can recover from clear, machine-readable failures
📖 **For a deep technical dive into the Assistant architecture, see [architecture/pulse-assistant-deep-dive.md](architecture/pulse-assistant-deep-dive.md).**
📖 **For a deep technical dive into the Assistant architecture, see [Assistant architecture](ASSISTANT_ARCHITECTURE.md).**
### Not Just Another Alerting System
@ -43,12 +43,12 @@ Pulse Patrol is a **scheduled and event-triggered governed operator** that:
All while running entirely on your infrastructure with BYOK for complete privacy.
📖 **For a deep technical dive into the Patrol runtime, see [architecture/pulse-patrol-deep-dive.md](architecture/pulse-patrol-deep-dive.md).**
📖 **For a deep technical dive into the Patrol runtime, see [Patrol architecture](PATROL_ARCHITECTURE.md).**
🧪 **For independent live-fault qualification, safety gates, model comparison,
and release-claim rules, see [AI_PATROL_QUALIFICATION.md](AI_PATROL_QUALIFICATION.md).**
See [architecture/pulse-assistant.md](architecture/pulse-assistant.md) for the original safety architecture documentation.
See [Assistant safety architecture](ASSISTANT_SAFETY.md) for the original safety architecture documentation.
### Assistant And MCP
@ -696,11 +696,11 @@ Pulse tracks token usage and costs:
### Deep Dives (Recommended for Technical Audiences)
- **[Pulse Assistant Deep Dive](architecture/pulse-assistant-deep-dive.md)** — Complete technical breakdown of the model-owned tool surface: explicit context, session fact caching, FSM enforcement, parallel execution, grounded execution guardrails, structured errors
- **[Pulse Patrol Deep Dive](architecture/pulse-patrol-deep-dive.md)** — Patrol runtime documentation: evidence assembly, deterministic signal extraction, model evaluation, investigation context, investigation orchestration
- **[Pulse Assistant Deep Dive](ASSISTANT_ARCHITECTURE.md)** — Complete technical breakdown of the model-owned tool surface: explicit context, session fact caching, FSM enforcement, parallel execution, grounded execution guardrails, structured errors
- **[Pulse Patrol Deep Dive](PATROL_ARCHITECTURE.md)** — Patrol runtime documentation: evidence assembly, deterministic signal extraction, model evaluation, investigation context, investigation orchestration
### Reference Documentation
- [Architecture: Pulse Assistant (Safety Gates)](architecture/pulse-assistant.md) — Detailed FSM states, tool protocol, and invariants
- [Architecture: Pulse Assistant (Safety Gates)](ASSISTANT_SAFETY.md) — Detailed FSM states, tool protocol, and invariants
- [API Reference](API.md) — Complete API endpoint documentation
- [Plans and entitlements](PULSE_PRO.md) — Community/Relay/Pro/Cloud features and licensing

View file

@ -182,4 +182,4 @@ Token usage and estimated costs are tracked per provider:
- [Pulse Intelligence Overview](AI.md) — Full Pulse Intelligence system documentation
- [Plans and Entitlements](PULSE_PRO.md) — Feature availability by plan
- [API Reference](API.md) — Complete API documentation
- [Pulse Patrol Deep Dive](architecture/pulse-patrol-deep-dive.md) — Technical architecture details
- [Pulse Patrol Deep Dive](PATROL_ARCHITECTURE.md) — Technical architecture details

View file

@ -492,8 +492,8 @@ Detailed storage usage per node and pool.
### Recovery (formerly Backups / Snapshots)
Pulse v6 uses the recovery API to provide a platform-agnostic view of backup and snapshot artifacts.
See `docs/architecture/RECOVERY_CONTRACT.md` for the provider-neutral contract
(subjects, points, rollups, posture, and filter semantics).
The endpoints below carry the provider-neutral contract covering subjects,
points, rollups, posture, and filter semantics.
- `GET /api/recovery/points`
- Query params:

View file

@ -0,0 +1,99 @@
# Pulse Assistant deep dive
How the Assistant's agentic loop executes tool calls, for readers who want
more than the overview in [AI features](AI.md).
The safety state machine has its own page. See
[Pulse Assistant safety architecture](ASSISTANT_SAFETY.md) for the states,
transitions, and invariants. This page covers the loop that runs around it.
## The three-phase pipeline
Each provider turn can return several tool calls at once. The loop processes
them in three phases, and the split matters because only one of the three is
safe to parallelise.
**Phase 1, pre-check, runs sequentially.** This is where the state machine
gate, loop detection, and budget checks happen. Every call is judged before
any call runs.
**Phase 2, execute, runs in parallel.** Independent calls run concurrently
through goroutines, with concurrency capped at four.
**Phase 3, post-process, runs sequentially.** Streaming output, state machine
transitions, and knowledge extraction happen in a deterministic order, so
concurrent execution cannot produce non-deterministic session state.
## What is not allowed to run in parallel
Parallelism is bounded by real ordering requirements rather than applied
uniformly.
A same-turn read-before-write dependency forces sequential execution. If one
turn contains both a `patrol_get_findings` read and a finding lifecycle write,
the batch runs in order, because the write's deduplication and assessment
precondition is established by that read. Letting them race inside one turn
would mean writing against a precondition that had not been checked.
Independent reads and independent finding writes still run in parallel. The
provider's original call order stays authoritative.
Interactive input is also excluded. `pulse_question` never runs in parallel
with other tools, since asking you something is not an independent operation.
## The look-before-asking gate
The Assistant is discouraged from asking you a question before it has tried to
find the answer. If the model attempts to ask without having attempted any
tool call, the attempt is blocked and it is pushed to look first.
The gate is bounded rather than absolute. It allows at most two blocks per
turn, after which the question goes through. A model that genuinely cannot
proceed without input is not trapped in a loop, and any real tool attempt
satisfies the gate immediately.
## Structured errors
Failures reach the model as stable machine-readable codes rather than as
prose, so it can branch on the failure instead of parsing an English sentence.
The codes are declared in `internal/agentcapabilities/errors.go` and include
`resource_not_found`, `operator_state_not_set`, `operator_state_invalid`,
`invalid_finding_request`, `finding_not_found`, `finding_action_not_allowed`,
`patrol_unavailable`, `invalid_action_request`, `capability_not_found`,
`action_execution_unavailable`, `action_actor_unavailable`, and `missing_id`.
A call blocked by the state machine uses the same mechanism, returning the
`fsm_blocked` code with the state and tool that were involved.
The same codes are published in the capability manifest at
`/api/agent/capabilities`, and a contract test fails the build if a handler
can emit a code the manifest does not declare, or the manifest declares a code
no handler emits. See [agent integrations](AGENT_SUBSTRATE.md).
## Grounded execution
Several guardrails exist to keep the model's claims tied to evidence it
actually gathered.
The state machine supplies the structural half. A write moves the session into
verification, and the Assistant cannot deliver a final answer about that write
until it has read something afterwards.
The prompts supply the rest. Instructions repeated across the agentic prompts
tell the model to treat infrastructure names, labels, logs, and other
collected values as untrusted data rather than as instructions, and not to
invent evidence, root cause, verification, remediation, or a claim that an
action was taken.
Prompt instructions are the weaker of the two, which is exactly why the
verification requirement lives in code instead. Where a guarantee needs to
hold, it is enforced structurally.
## Related reading
- [Pulse Assistant safety architecture](ASSISTANT_SAFETY.md) for the state
machine in detail.
- [Patrol deep dive](PATROL_ARCHITECTURE.md) for the scheduled analysis
runtime.
- [Agent integrations](AGENT_SUBSTRATE.md) for driving the same surface
from an external agent.
- [AI features](AI.md) for the overview and configuration.

100
docs/ASSISTANT_SAFETY.md Normal file
View file

@ -0,0 +1,100 @@
# Pulse Assistant safety architecture
The state machine that governs what the Pulse Assistant is allowed to do
during a chat session, the tool classification it runs on, and the invariants
it holds.
The point of this machine is structural. Prompt wording can be argued with by
a model, and drifts as prompts are edited. These rules are enforced in code,
in `internal/ai/chat/fsm.go`, so neither a model nor a future prompt change
can talk its way past them.
## Tool kinds
Every tool call is classified before it runs.
| Kind | Meaning |
|---|---|
| `resolve` | Discovery and query tools that find resources |
| `read` | Read-only tools such as logs, metrics, status, and config |
| `write` | Mutating tools such as restart, stop, start, delete, and file write |
| `user_input` | Interactive tools that ask you something |
Classification is `ClassifyToolCall`, which delegates to the shared
`agentcapabilities` classifier so the Assistant and the rest of the agent
surface agree on what counts as a write.
## States
A session starts in `RESOLVING` and moves between four states.
| State | What it means |
|---|---|
| `RESOLVING` | No validated target yet, so resources must be discovered first |
| `READING` | A target is established and querying is allowed |
| `WRITING` | Transitional, entered around a mutation |
| `VERIFYING` | A write happened and evidence has not been gathered since |
Transitions on a successful tool call are as follows.
- A `resolve` or `read` in `RESOLVING` moves the session to `READING`.
- A `write` from any state moves the session to `VERIFYING`, records the tool
and timestamp, and clears the read-after-write flag.
- A `resolve` or `read` while in `VERIFYING` sets read-after-write, which is
what satisfies the verification requirement.
- A `user_input` call does not advance state at all, because asking you a
question is neither discovery nor verification.
`CompleteVerification` returns a verified session from `VERIFYING` to
`READING` so further writes become possible.
## The invariants
**No writing without a validated target.** A `write` attempted in `RESOLVING`
is blocked. The model must establish what it is acting on before it acts.
**No writing again until the last write is verified.** A `write` attempted in
`VERIFYING` is blocked until a read or resolve has run since the write.
**No final answer about an unverified change.** `CanFinalAnswer` refuses while
the session is in `VERIFYING` with no read-after-write. The Assistant cannot
tell you it restarted something and then decline to look at whether the
restart worked.
**Repeated attempts do not wear the gate down.** Consecutive blocked writes in
`VERIFYING` increment a counter, and that counter is telemetry only. There is
no attempt threshold after which the verification requirement is waived.
**Reads are never blocked.** No state blocks a `read`, `resolve`, or
`user_input`. The machine constrains mutation and the claims made about
mutation, not information gathering.
## Blocked calls and recovery
A blocked call returns an `FSMBlockedError` carrying the state, the tool, the
tool kind, a reason, and a recoverable flag. It surfaces to the model with the
stable code `ErrCodeFSMBlocked` rather than as prose, so the model can branch
on the code.
Blocks are recoverable rather than terminal. The session tracks a pending
recovery per blocked operation, and a later successful call of the same tool
clears it. Pending recoveries expire after ten minutes.
## Resetting
`Reset` returns the session to `RESOLVING` and clears all tracking, which is
what a full session clear does.
`ResetKeepProgress` is the softer variant used when context is cleared but
pinned items are kept. It drops verification tracking and moves a `VERIFYING`
session back to `READING`, without discarding that a target was established.
Note that `WroteThisEpisode` means "wrote at all during this session" rather
than "wrote during the current verification cycle", and `CompleteVerification`
deliberately leaves it set.
## Related reading
- [AI features](AI.md) for the overview and configuration.
- [Patrol deep dive](PATROL_ARCHITECTURE.md) for the scheduled analysis
runtime, which is a separate loop from the Assistant.

152
docs/PATROL_ARCHITECTURE.md Normal file
View file

@ -0,0 +1,152 @@
# Pulse Patrol deep dive
How a Patrol run actually works, for readers who want more than the overview
in [AI features](AI.md).
The short version is that Patrol does not trust the model to notice
everything. A run pairs model judgement with deterministic detection, and the
deterministic half acts as a safety net over the model's output rather than as
an input to it.
## A run, end to end
A run moves through the following stages. Each is described in more detail
below.
1. A pre-flight budget check, which fails fast before any context is built.
2. Guest intelligence gathering, covering discovery and reachability probing.
3. Deterministic triage, which narrows the surface the model will look at.
4. Seed context construction from the triage result.
5. The agentic loop, where the model calls tools and files findings.
6. Deterministic signal detection over the tool calls the model actually made.
7. An evaluation pass for signals the model saw but did not file.
8. An assessment sweep for findings the model left unassessed.
## Budget check first
The budget check runs before context assembly and before a chat service is
acquired. A run that cannot afford to complete is stopped at the cheapest
possible point rather than after the expensive work.
## Guest intelligence
Discovery and reachability probing run before the seed context is built, so
the context the model receives already reflects which guests are reachable.
Reachability results are kept, because they feed the deterministic detection
stage later in the run.
## Deterministic triage and seed context
Triage narrows the surface first, and the seed context is then built from what
triage returned rather than from everything Pulse knows. This keeps the
model's starting context focused on what looks interesting.
Quiet infrastructure still goes through the configured model. Patrol does not
skip the model when nothing looks wrong, because "nothing is wrong" is itself
a judgement worth making explicitly.
The turn budget for the loop is computed from the size of the surface. See
`computePatrolMaxTurns` and `computeTriageMaxTurns` in
`internal/ai/patrol_ai.go`.
## The agentic loop
The model runs an agentic loop with tool access. Findings are created through
tool calls during the loop rather than parsed out of prose afterwards, so a
finding exists because the model deliberately filed it.
## Deterministic signal detection
After the loop completes, Patrol collects the tool calls that ran and scans
their results for known problem signals. This scan is deterministic and does
not depend on model judgement at all. The entry point is `DetectSignals` in
`internal/ai/patrol_signals.go`.
The signal types are:
| Signal | Meaning |
|---|---|
| `smart_failure` | A disk reporting SMART failure |
| `high_cpu` | Sustained CPU above the threshold |
| `high_memory` | Sustained memory above the threshold |
| `high_disk` | Disk or pool usage above the threshold |
| `backup_failed` | A backup job that failed |
| `backup_stale` | No backup inside the staleness window |
| `guest_unreachable` | A guest that failed reachability probing |
Thresholds come from your own alert configuration rather than from a separate
set of numbers, so detection agrees with what Pulse would alert on.
`SignalThresholdsFromPatrol` maps your configured alert thresholds onto the
signal thresholds, falling back per field to these defaults when a value is
not configured.
| Threshold | Default |
|---|---|
| Storage warning | 75% |
| Storage critical | 95% |
| High CPU | 70% |
| High memory | 80% |
| Backup staleness | 48 hours |
Backup staleness has no user-facing setting and always uses the default.
Reachability signals detected during the guest intelligence stage are merged
into the same set at this point.
## The evaluation pass
Detected signals are compared against the findings the model actually filed.
Any signal with no corresponding finding is an unmatched signal, meaning the
data showed a problem and the model did not report it.
Unmatched signals go to a bounded evaluation pass scoped to exactly those
signals. This is the safety net. It is why a run can still surface a failing
disk when the model overlooked one in a large sweep.
## The assessment sweep
Smaller models sometimes finish the main pass without recording an assessment
for every finding they filed. Patrol detects the missing assessments and runs
one bounded follow-up pass scoped to exactly those findings, rather than
rerunning the whole analysis or leaving the findings unassessed.
## Investigation
A finding may be investigated after it is filed. Investigation is a separate
loop from the run that produced the finding, coordinated through the
`InvestigationOrchestrator` contract in `pkg/aicontracts/interfaces.go`.
Whether a finding is investigated at all depends on your autonomy level and on
the finding's own state. `Finding.ShouldInvestigate` in
`internal/ai/findings.go` is the gate, and it refuses in every one of these
cases.
- Autonomy is unset or `monitor`. Investigation only runs at `approval`,
`assisted`, or `full`.
- The finding is resolved, dismissed, suppressed, or snoozed.
- The severity is not warning or critical. Info and watch findings are not
investigated.
- A fix is already queued for approval, so the next move belongs to you.
- The finding reached a terminal outcome, such as fix verified, resolved, fix
rejected, cannot fix, or needs attention.
- The finding has already been investigated three times.
- The last investigation was inside the cooldown, which is one hour by
default and shorter for timeout failures, since those are transient.
Patrol also recovers investigations that were left running and retries ones
that timed out. See `recoverStuckInvestigations` and
`retryTimedOutInvestigations` in `internal/ai/patrol_findings.go`.
## Why it is built this way
Model judgement is good at explaining and correlating, and unreliable at
exhaustive enumeration. Deterministic detection is the reverse. Patrol uses
each for what it is good at, and the unmatched-signal check is the seam where
the deterministic half can correct the model.
## Related reading
- [AI features](AI.md) for the overview and configuration.
- [Patrol autonomy](AI_AUTONOMY.md) for what each autonomy level permits.
- [Patrol qualification](AI_PATROL_QUALIFICATION.md) for the evidence and
release-claim rules.

View file

@ -2,8 +2,6 @@
This document explains Pulse's user-facing plan structure, the locked self-hosted commercial model, and how those plans map to runtime feature gates.
For the canonical, code-aligned entitlement table (including internal tier names), see:
- `docs/architecture/ENTITLEMENT_MATRIX.md`
## Plan Mapping (User-Facing -> Code Tiers)
@ -134,7 +132,7 @@ Legend:
- Included: `Y` / `N`
- `Y*`: Enterprise/custom only (`enterprise` tier or explicit entitlement)
This matrix is derived from the canonical table in `docs/architecture/ENTITLEMENT_MATRIX.md` plus runtime history/limit semantics exposed through entitlements.
This matrix reflects the entitlement keys enforced in code plus the runtime history and limit semantics exposed through entitlements.
| Constant | Capability Key | Display Name | Community | Relay | Pro | Cloud | Primary Gating Mechanism / Notes |
|---|---|---|:---:|:---:|:---:|:---:|---|
@ -221,6 +219,6 @@ This returns a feature map including keys like `relay`, `ai_alerts`, `ai_autofix
## Deep Dives
- [Pulse Patrol Deep Dive](architecture/pulse-patrol-deep-dive.md)
- [Pulse Assistant Deep Dive](architecture/pulse-assistant-deep-dive.md)
- [Pulse Patrol Deep Dive](PATROL_ARCHITECTURE.md)
- [Pulse Assistant Deep Dive](ASSISTANT_ARCHITECTURE.md)
- [Pulse Intelligence overview](AI.md)

View file

@ -384,4 +384,3 @@ Multi-tenant mode is opt-in and additionally license-gated:
- Enablement flag: `PULSE_MULTI_TENANT_ENABLED=true`
- Capability gate: `multi_tenant`
See any multi-tenant operational docs under `docs/architecture/` if you plan to run this mode.

View file

@ -29,7 +29,7 @@ Pulse Assistant is a **protocol-driven, safety-gated LLM tool surface** that:
- **Grounds answers in real tool work** — visible tool traces, read-after-write verification, and transcript hygiene prevent unsupported execution claims from being treated as facts
- **Returns structured tool errors** — the model can recover from clear, machine-readable failures
📖 **For a deep technical dive into the Assistant architecture, see [architecture/pulse-assistant-deep-dive.md](architecture/pulse-assistant-deep-dive.md).**
📖 **For a deep technical dive into the Assistant architecture, see [Assistant architecture](ASSISTANT_ARCHITECTURE.md).**
### Not Just Another Alerting System
@ -43,12 +43,12 @@ Pulse Patrol is a **scheduled and event-triggered governed operator** that:
All while running entirely on your infrastructure with BYOK for complete privacy.
📖 **For a deep technical dive into the Patrol runtime, see [architecture/pulse-patrol-deep-dive.md](architecture/pulse-patrol-deep-dive.md).**
📖 **For a deep technical dive into the Patrol runtime, see [Patrol architecture](PATROL_ARCHITECTURE.md).**
🧪 **For independent live-fault qualification, safety gates, model comparison,
and release-claim rules, see [AI_PATROL_QUALIFICATION.md](AI_PATROL_QUALIFICATION.md).**
See [architecture/pulse-assistant.md](architecture/pulse-assistant.md) for the original safety architecture documentation.
See [Assistant safety architecture](ASSISTANT_SAFETY.md) for the original safety architecture documentation.
### Assistant And MCP
@ -696,11 +696,11 @@ Pulse tracks token usage and costs:
### Deep Dives (Recommended for Technical Audiences)
- **[Pulse Assistant Deep Dive](architecture/pulse-assistant-deep-dive.md)** — Complete technical breakdown of the model-owned tool surface: explicit context, session fact caching, FSM enforcement, parallel execution, grounded execution guardrails, structured errors
- **[Pulse Patrol Deep Dive](architecture/pulse-patrol-deep-dive.md)** — Patrol runtime documentation: evidence assembly, deterministic signal extraction, model evaluation, investigation context, investigation orchestration
- **[Pulse Assistant Deep Dive](ASSISTANT_ARCHITECTURE.md)** — Complete technical breakdown of the model-owned tool surface: explicit context, session fact caching, FSM enforcement, parallel execution, grounded execution guardrails, structured errors
- **[Pulse Patrol Deep Dive](PATROL_ARCHITECTURE.md)** — Patrol runtime documentation: evidence assembly, deterministic signal extraction, model evaluation, investigation context, investigation orchestration
### Reference Documentation
- [Architecture: Pulse Assistant (Safety Gates)](architecture/pulse-assistant.md) — Detailed FSM states, tool protocol, and invariants
- [Architecture: Pulse Assistant (Safety Gates)](ASSISTANT_SAFETY.md) — Detailed FSM states, tool protocol, and invariants
- [API Reference](API.md) — Complete API endpoint documentation
- [Plans and entitlements](PULSE_PRO.md) — Community/Relay/Pro/Cloud features and licensing

View file

@ -182,4 +182,4 @@ Token usage and estimated costs are tracked per provider:
- [Pulse Intelligence Overview](AI.md) — Full Pulse Intelligence system documentation
- [Plans and Entitlements](PULSE_PRO.md) — Feature availability by plan
- [API Reference](API.md) — Complete API documentation
- [Pulse Patrol Deep Dive](architecture/pulse-patrol-deep-dive.md) — Technical architecture details
- [Pulse Patrol Deep Dive](PATROL_ARCHITECTURE.md) — Technical architecture details

View file

@ -492,8 +492,8 @@ Detailed storage usage per node and pool.
### Recovery (formerly Backups / Snapshots)
Pulse v6 uses the recovery API to provide a platform-agnostic view of backup and snapshot artifacts.
See `docs/architecture/RECOVERY_CONTRACT.md` for the provider-neutral contract
(subjects, points, rollups, posture, and filter semantics).
The endpoints below carry the provider-neutral contract covering subjects,
points, rollups, posture, and filter semantics.
- `GET /api/recovery/points`
- Query params:

View file

@ -0,0 +1,99 @@
# Pulse Assistant deep dive
How the Assistant's agentic loop executes tool calls, for readers who want
more than the overview in [AI features](AI.md).
The safety state machine has its own page. See
[Pulse Assistant safety architecture](ASSISTANT_SAFETY.md) for the states,
transitions, and invariants. This page covers the loop that runs around it.
## The three-phase pipeline
Each provider turn can return several tool calls at once. The loop processes
them in three phases, and the split matters because only one of the three is
safe to parallelise.
**Phase 1, pre-check, runs sequentially.** This is where the state machine
gate, loop detection, and budget checks happen. Every call is judged before
any call runs.
**Phase 2, execute, runs in parallel.** Independent calls run concurrently
through goroutines, with concurrency capped at four.
**Phase 3, post-process, runs sequentially.** Streaming output, state machine
transitions, and knowledge extraction happen in a deterministic order, so
concurrent execution cannot produce non-deterministic session state.
## What is not allowed to run in parallel
Parallelism is bounded by real ordering requirements rather than applied
uniformly.
A same-turn read-before-write dependency forces sequential execution. If one
turn contains both a `patrol_get_findings` read and a finding lifecycle write,
the batch runs in order, because the write's deduplication and assessment
precondition is established by that read. Letting them race inside one turn
would mean writing against a precondition that had not been checked.
Independent reads and independent finding writes still run in parallel. The
provider's original call order stays authoritative.
Interactive input is also excluded. `pulse_question` never runs in parallel
with other tools, since asking you something is not an independent operation.
## The look-before-asking gate
The Assistant is discouraged from asking you a question before it has tried to
find the answer. If the model attempts to ask without having attempted any
tool call, the attempt is blocked and it is pushed to look first.
The gate is bounded rather than absolute. It allows at most two blocks per
turn, after which the question goes through. A model that genuinely cannot
proceed without input is not trapped in a loop, and any real tool attempt
satisfies the gate immediately.
## Structured errors
Failures reach the model as stable machine-readable codes rather than as
prose, so it can branch on the failure instead of parsing an English sentence.
The codes are declared in `internal/agentcapabilities/errors.go` and include
`resource_not_found`, `operator_state_not_set`, `operator_state_invalid`,
`invalid_finding_request`, `finding_not_found`, `finding_action_not_allowed`,
`patrol_unavailable`, `invalid_action_request`, `capability_not_found`,
`action_execution_unavailable`, `action_actor_unavailable`, and `missing_id`.
A call blocked by the state machine uses the same mechanism, returning the
`fsm_blocked` code with the state and tool that were involved.
The same codes are published in the capability manifest at
`/api/agent/capabilities`, and a contract test fails the build if a handler
can emit a code the manifest does not declare, or the manifest declares a code
no handler emits. See [agent integrations](AGENT_SUBSTRATE.md).
## Grounded execution
Several guardrails exist to keep the model's claims tied to evidence it
actually gathered.
The state machine supplies the structural half. A write moves the session into
verification, and the Assistant cannot deliver a final answer about that write
until it has read something afterwards.
The prompts supply the rest. Instructions repeated across the agentic prompts
tell the model to treat infrastructure names, labels, logs, and other
collected values as untrusted data rather than as instructions, and not to
invent evidence, root cause, verification, remediation, or a claim that an
action was taken.
Prompt instructions are the weaker of the two, which is exactly why the
verification requirement lives in code instead. Where a guarantee needs to
hold, it is enforced structurally.
## Related reading
- [Pulse Assistant safety architecture](ASSISTANT_SAFETY.md) for the state
machine in detail.
- [Patrol deep dive](PATROL_ARCHITECTURE.md) for the scheduled analysis
runtime.
- [Agent integrations](AGENT_SUBSTRATE.md) for driving the same surface
from an external agent.
- [AI features](AI.md) for the overview and configuration.

View file

@ -0,0 +1,100 @@
# Pulse Assistant safety architecture
The state machine that governs what the Pulse Assistant is allowed to do
during a chat session, the tool classification it runs on, and the invariants
it holds.
The point of this machine is structural. Prompt wording can be argued with by
a model, and drifts as prompts are edited. These rules are enforced in code,
in `internal/ai/chat/fsm.go`, so neither a model nor a future prompt change
can talk its way past them.
## Tool kinds
Every tool call is classified before it runs.
| Kind | Meaning |
|---|---|
| `resolve` | Discovery and query tools that find resources |
| `read` | Read-only tools such as logs, metrics, status, and config |
| `write` | Mutating tools such as restart, stop, start, delete, and file write |
| `user_input` | Interactive tools that ask you something |
Classification is `ClassifyToolCall`, which delegates to the shared
`agentcapabilities` classifier so the Assistant and the rest of the agent
surface agree on what counts as a write.
## States
A session starts in `RESOLVING` and moves between four states.
| State | What it means |
|---|---|
| `RESOLVING` | No validated target yet, so resources must be discovered first |
| `READING` | A target is established and querying is allowed |
| `WRITING` | Transitional, entered around a mutation |
| `VERIFYING` | A write happened and evidence has not been gathered since |
Transitions on a successful tool call are as follows.
- A `resolve` or `read` in `RESOLVING` moves the session to `READING`.
- A `write` from any state moves the session to `VERIFYING`, records the tool
and timestamp, and clears the read-after-write flag.
- A `resolve` or `read` while in `VERIFYING` sets read-after-write, which is
what satisfies the verification requirement.
- A `user_input` call does not advance state at all, because asking you a
question is neither discovery nor verification.
`CompleteVerification` returns a verified session from `VERIFYING` to
`READING` so further writes become possible.
## The invariants
**No writing without a validated target.** A `write` attempted in `RESOLVING`
is blocked. The model must establish what it is acting on before it acts.
**No writing again until the last write is verified.** A `write` attempted in
`VERIFYING` is blocked until a read or resolve has run since the write.
**No final answer about an unverified change.** `CanFinalAnswer` refuses while
the session is in `VERIFYING` with no read-after-write. The Assistant cannot
tell you it restarted something and then decline to look at whether the
restart worked.
**Repeated attempts do not wear the gate down.** Consecutive blocked writes in
`VERIFYING` increment a counter, and that counter is telemetry only. There is
no attempt threshold after which the verification requirement is waived.
**Reads are never blocked.** No state blocks a `read`, `resolve`, or
`user_input`. The machine constrains mutation and the claims made about
mutation, not information gathering.
## Blocked calls and recovery
A blocked call returns an `FSMBlockedError` carrying the state, the tool, the
tool kind, a reason, and a recoverable flag. It surfaces to the model with the
stable code `ErrCodeFSMBlocked` rather than as prose, so the model can branch
on the code.
Blocks are recoverable rather than terminal. The session tracks a pending
recovery per blocked operation, and a later successful call of the same tool
clears it. Pending recoveries expire after ten minutes.
## Resetting
`Reset` returns the session to `RESOLVING` and clears all tracking, which is
what a full session clear does.
`ResetKeepProgress` is the softer variant used when context is cleared but
pinned items are kept. It drops verification tracking and moves a `VERIFYING`
session back to `READING`, without discarding that a target was established.
Note that `WroteThisEpisode` means "wrote at all during this session" rather
than "wrote during the current verification cycle", and `CompleteVerification`
deliberately leaves it set.
## Related reading
- [AI features](AI.md) for the overview and configuration.
- [Patrol deep dive](PATROL_ARCHITECTURE.md) for the scheduled analysis
runtime, which is a separate loop from the Assistant.

View file

@ -0,0 +1,152 @@
# Pulse Patrol deep dive
How a Patrol run actually works, for readers who want more than the overview
in [AI features](AI.md).
The short version is that Patrol does not trust the model to notice
everything. A run pairs model judgement with deterministic detection, and the
deterministic half acts as a safety net over the model's output rather than as
an input to it.
## A run, end to end
A run moves through the following stages. Each is described in more detail
below.
1. A pre-flight budget check, which fails fast before any context is built.
2. Guest intelligence gathering, covering discovery and reachability probing.
3. Deterministic triage, which narrows the surface the model will look at.
4. Seed context construction from the triage result.
5. The agentic loop, where the model calls tools and files findings.
6. Deterministic signal detection over the tool calls the model actually made.
7. An evaluation pass for signals the model saw but did not file.
8. An assessment sweep for findings the model left unassessed.
## Budget check first
The budget check runs before context assembly and before a chat service is
acquired. A run that cannot afford to complete is stopped at the cheapest
possible point rather than after the expensive work.
## Guest intelligence
Discovery and reachability probing run before the seed context is built, so
the context the model receives already reflects which guests are reachable.
Reachability results are kept, because they feed the deterministic detection
stage later in the run.
## Deterministic triage and seed context
Triage narrows the surface first, and the seed context is then built from what
triage returned rather than from everything Pulse knows. This keeps the
model's starting context focused on what looks interesting.
Quiet infrastructure still goes through the configured model. Patrol does not
skip the model when nothing looks wrong, because "nothing is wrong" is itself
a judgement worth making explicitly.
The turn budget for the loop is computed from the size of the surface. See
`computePatrolMaxTurns` and `computeTriageMaxTurns` in
`internal/ai/patrol_ai.go`.
## The agentic loop
The model runs an agentic loop with tool access. Findings are created through
tool calls during the loop rather than parsed out of prose afterwards, so a
finding exists because the model deliberately filed it.
## Deterministic signal detection
After the loop completes, Patrol collects the tool calls that ran and scans
their results for known problem signals. This scan is deterministic and does
not depend on model judgement at all. The entry point is `DetectSignals` in
`internal/ai/patrol_signals.go`.
The signal types are:
| Signal | Meaning |
|---|---|
| `smart_failure` | A disk reporting SMART failure |
| `high_cpu` | Sustained CPU above the threshold |
| `high_memory` | Sustained memory above the threshold |
| `high_disk` | Disk or pool usage above the threshold |
| `backup_failed` | A backup job that failed |
| `backup_stale` | No backup inside the staleness window |
| `guest_unreachable` | A guest that failed reachability probing |
Thresholds come from your own alert configuration rather than from a separate
set of numbers, so detection agrees with what Pulse would alert on.
`SignalThresholdsFromPatrol` maps your configured alert thresholds onto the
signal thresholds, falling back per field to these defaults when a value is
not configured.
| Threshold | Default |
|---|---|
| Storage warning | 75% |
| Storage critical | 95% |
| High CPU | 70% |
| High memory | 80% |
| Backup staleness | 48 hours |
Backup staleness has no user-facing setting and always uses the default.
Reachability signals detected during the guest intelligence stage are merged
into the same set at this point.
## The evaluation pass
Detected signals are compared against the findings the model actually filed.
Any signal with no corresponding finding is an unmatched signal, meaning the
data showed a problem and the model did not report it.
Unmatched signals go to a bounded evaluation pass scoped to exactly those
signals. This is the safety net. It is why a run can still surface a failing
disk when the model overlooked one in a large sweep.
## The assessment sweep
Smaller models sometimes finish the main pass without recording an assessment
for every finding they filed. Patrol detects the missing assessments and runs
one bounded follow-up pass scoped to exactly those findings, rather than
rerunning the whole analysis or leaving the findings unassessed.
## Investigation
A finding may be investigated after it is filed. Investigation is a separate
loop from the run that produced the finding, coordinated through the
`InvestigationOrchestrator` contract in `pkg/aicontracts/interfaces.go`.
Whether a finding is investigated at all depends on your autonomy level and on
the finding's own state. `Finding.ShouldInvestigate` in
`internal/ai/findings.go` is the gate, and it refuses in every one of these
cases.
- Autonomy is unset or `monitor`. Investigation only runs at `approval`,
`assisted`, or `full`.
- The finding is resolved, dismissed, suppressed, or snoozed.
- The severity is not warning or critical. Info and watch findings are not
investigated.
- A fix is already queued for approval, so the next move belongs to you.
- The finding reached a terminal outcome, such as fix verified, resolved, fix
rejected, cannot fix, or needs attention.
- The finding has already been investigated three times.
- The last investigation was inside the cooldown, which is one hour by
default and shorter for timeout failures, since those are transient.
Patrol also recovers investigations that were left running and retries ones
that timed out. See `recoverStuckInvestigations` and
`retryTimedOutInvestigations` in `internal/ai/patrol_findings.go`.
## Why it is built this way
Model judgement is good at explaining and correlating, and unreliable at
exhaustive enumeration. Deterministic detection is the reverse. Patrol uses
each for what it is good at, and the unmatched-signal check is the seam where
the deterministic half can correct the model.
## Related reading
- [AI features](AI.md) for the overview and configuration.
- [Patrol autonomy](AI_AUTONOMY.md) for what each autonomy level permits.
- [Patrol qualification](AI_PATROL_QUALIFICATION.md) for the evidence and
release-claim rules.

View file

@ -2,8 +2,6 @@
This document explains Pulse's user-facing plan structure, the locked self-hosted commercial model, and how those plans map to runtime feature gates.
For the canonical, code-aligned entitlement table (including internal tier names), see:
- `docs/architecture/ENTITLEMENT_MATRIX.md`
## Plan Mapping (User-Facing -> Code Tiers)
@ -134,7 +132,7 @@ Legend:
- Included: `Y` / `N`
- `Y*`: Enterprise/custom only (`enterprise` tier or explicit entitlement)
This matrix is derived from the canonical table in `docs/architecture/ENTITLEMENT_MATRIX.md` plus runtime history/limit semantics exposed through entitlements.
This matrix reflects the entitlement keys enforced in code plus the runtime history and limit semantics exposed through entitlements.
| Constant | Capability Key | Display Name | Community | Relay | Pro | Cloud | Primary Gating Mechanism / Notes |
|---|---|---|:---:|:---:|:---:|:---:|---|
@ -221,6 +219,6 @@ This returns a feature map including keys like `relay`, `ai_alerts`, `ai_autofix
## Deep Dives
- [Pulse Patrol Deep Dive](architecture/pulse-patrol-deep-dive.md)
- [Pulse Assistant Deep Dive](architecture/pulse-assistant-deep-dive.md)
- [Pulse Patrol Deep Dive](PATROL_ARCHITECTURE.md)
- [Pulse Assistant Deep Dive](ASSISTANT_ARCHITECTURE.md)
- [Pulse Intelligence overview](AI.md)

View file

@ -384,4 +384,3 @@ Multi-tenant mode is opt-in and additionally license-gated:
- Enablement flag: `PULSE_MULTI_TENANT_ENABLED=true`
- Capability gate: `multi_tenant`
See any multi-tenant operational docs under `docs/architecture/` if you plan to run this mode.