ouroboros/docs/PERSISTENCE.md

PERSISTENCE.md — durable data-plane inventory

Every durable entity under the runtime data root (DATA_DIR, default ~/Ouroboros/data/), with its schema, migration path, retention and reset semantics (what deleting the entity while the server is stopped does). These contracts are local to each entity; there is no generic persistence framework.

tests/test_persistence_inventory.py scans data-path constructors in ouroboros/, supervisor/, server.py and launcher.py and requires every scanned data-relative path to be covered by a row here (count-anchored both ways).

Shared idioms (the vocabulary the rows use)

• Stamp — the opt-in _schema_version key from ouroboros/contracts/schema_versions.py (ABI-2): missing/invalid reads as legacy 0; existing files are not retrofitted. Some pre-ABI-2 entities carry their own spelling (schema_version, schema, state_version) — noted per row; new stamps use the shared key.
• GC retention — ouroboros/retention.py: one owner knob OUROBOROS_GC_RETENTION_DAYS (default 7, clamped 1–365; legacy per-subsystem keys migrate). Governs subagent worktrees, headless/task drives, task trees, service logs, consumed schedule receipts, confirmed capability probes, delegate recovery/supervision sweeps, code_intel reconcile-failed prunes, and agent media. Memory journals retain full new rows independently of this knob.
• Rotation — supervisor/state.py::rotate_jsonl_log_if_needed: >800 KB → atomic rename to archive/<prefix>_<ts>.jsonl under the append lock. Applied on the supervisor tick to chat.jsonl, progress.jsonl, events.jsonl, tools.jsonl, supervisor.jsonl, task_reflections.jsonl. Chain readers enumerate archive/<stem>_*.jsonl name-sorted (chronological by construction); utils.jsonl_chain_handles is the rotation-race-safe traversal (open-live-first + inode dedup).
• archive/ is durable history, never GC'd (BIBLE P1): readers backfill from rotated segments; no retention sweep touches archive/ and none may be added.
• Atomic writes — atomic_write_json/atomic_write_text (tmp+rename) and update_json_locked (sidecar <file>.lock); JSONL appends go through append_jsonl (O_APPEND + sidecar lock) unless noted.

1. Root files

Path	Writer	Format	schema_version	Retention	Reset
settings.json	ouroboros/config.py save_settings (lock settings.json.lock, integrity guard)	JSON env-key map	none — not needed: migration is per-key inside load_settings (legacy keys migrate on read); external sha256 pin OUROBOROS_SETTINGS_SHA256 makes it immutable when set	fixed-size overwrite	recreated from defaults; ALL owner secrets/modes lost — never delete casually
.ouroboros_isolated_benchmark	EXTERNAL writer — the benchmark launchers (devtools/benchmarks/evolve_smoke.py, devtools/benchmarks/editbench/run_editbench.py, devtools/benchmarks/cybergym/cybergym_server.py) stamp it into their throwaway data root; the runtime only READS it (supervisor/state.py rotate suppression, ouroboros/agent_startup_checks.py)	one-line marker text	none — not needed: presence IS the fact	write-once per benchmark drive; never GC'd	the drive stops declaring itself synthetic: JSONL rotation and the benchmark-only startup carve-outs switch back to live-root behaviour on a throwaway root
settings.json.lock, *.lock sidecars, locks/**	ouroboros/platform_layer.py lock family (+ supervisor/state.py, supervisor/update_merge.py, ouroboros/skill_lifecycle_queue.py)	O_EXCL lockfiles (unlinked on release) or flock files (persistent); plus one transient kernel-lock capability probe file per lock directory (.kernel-lock-probe.<pid>.<ns>, created, locked and unlinked once per process by kernel_file_locks_enforced)	none — not needed	self-healing (mtime/pid staleness) — except a lock whose owner died and whose pid was REUSED by a live process: it reads as alive whoever owns the pid (kill(0) succeeds for a same-uid impostor and answers EPERM for another user's — both mean alive), is never reclaimed by age while the impostor lives, and needs a hand repair meanwhile	zero durable state; deleting while stopped is a no-op

2. state/ — singletons (fixed-size overwrite)

Path	Writer	schema_version	Retention	Reset
state/state.json, state/state.last_good.json	supervisor/state.py (lock locks/state.lock)	_schema_version: 1 (ABI-2, stamp-on-write)	overwrite, totals-only cost projection; legacy keys popped on load	falls back to last_good, then mints defaults; loses owner binding, spend counters, session id
state/queue_snapshot.json	supervisor/queue_snapshot.py	_schema_version: 1 (ABI-2)	overwrite; self-expires (max_age 900 s)	absent = 0 restored; queued-unstarted tasks silently dropped
state/direct_roots.json	supervisor/direct_roots.py — the main loop's off-lock projection of live direct-chat turns, written beside the snapshot on every tick (publish_direct_roots, server.py); queue init hands the previous process's rows to snapshot restore, which fences them, and empties the file in the same step (take_direct_roots, supervisor/queue.py) so a dead process's turns never outlive it	none — not needed: the whole file is re-derived every tick, and an unreadable one reads as no direct roots	overwrite; the rows are whatever the actor registry holds this tick, and a turn mid-admission (its actor lock only ever TRIED) is skipped behind ONE aggregate incomplete fact rather than blocking the loop	absent/empty = no live direct-chat roots are known: a task's peer roster (ouroboros/peer_roster.py) lists only the pooled roots from state/queue_snapshot.json until the next tick rewrites it
state/advisory_review.json	ouroboros/review_state.py (lock locks/advisory_review.lock)	own pre-ABI-2 spelling state_version: 3 (+duplicate schema_version) — kept; _schema_version deliberately avoids this key	bounded on write: runs 10, attempts 50, debts 50; open_obligations coalesced	recreated empty; recorded blocking obligations forgiven, commit gate demands fresh review (fail-closed)
state/scheduled_tasks.json	supervisor/queue_schedules.py	schema_version: 1 — authored at the write seam (_write_scheduled_tasks); legacy files gain it on their next write	consumed once receipts age out past GC retention on the scheduler tick (prune_consumed_once_records); 2 MB WARN = prune broken or live set huge	owner cron/once schedules lost; skill-manifest schedules resync automatically
state/terminal_deliveries.json	supervisor/terminal_delivery.py	own schema_version: 2	bounded: delivered 512, pending 64, replays 5	dedupe + owed-outbox lost: possible double- or never-delivery of one buffered terminal answer
state/cancel_intents.json	ouroboros/cancel_intents.py	own schema_version: 1	self-draining (settled rows leave)	in-flight cancels lost: cancelled-unsettled task revives as pending; forensics survive in supervisor.jsonl
state/update_letter.json	ouroboros/update_letter.py (refresh_after_check — the ONE writer, synchronous inside a FETCHING update check: boot and the Updates panel's check button)	own record shape keyed by the update range (base_sha, target_sha, channel, target ref) — no _schema_version	overwrite per check; the letter is never deleted after the update lands (it outlives its range; project_letter reads applied/other by SHA equality + the recorded target_in_head fact)	absent = no letter: the Updates panel and the agent's Runtime context (official_update_projection) show the check's status alone; the next FETCHING check rewrites it
state/capability_evidence.json	ouroboros/capability_evidence.py	none — accepted (self-healing cache; TTLs on read)	expired probe keys drop at the write seam: failed/unprobeable past their read TTL, confirmed past GC retention (blip-keep evidence survives inside retention); owner acks never expire	recreated; every reviewer window resolves unknown (sends are sized by the full-window assumption and re-probed), owner acks must be re-given; no review authority depends on it
state/evolution_campaign.json	supervisor/evolution_lifecycle.py (CAS under state.lock)	own schema_version: 1 (campaign) / 2 (active_transaction); the active transaction carries the pre-commit commit_intent (reviewed tree + parents) written BEFORE git commit, which boot recovery turns into the commit_receipt a crash never wrote	bounded histories (50)	in-flight self-modification transaction unabsorbable; anti-repeat fingerprints lost
state/evolution_metrics_cache.json	ouroboros/utils.py	own schema: 1 (strictly validated)	one point per git tag, no prune — accepted (derived cache)	pure cache; recomputed from git
state/projects.json, state/project_task_bindings.json	ouroboros/projects_registry.py (sidecar locks)	_schema_version: 2 / 1 (ABI-2)	never age-pruned (owner curates); deletes are durable tombstones	tombstones live here: losing it can resurrect deleted project rooms (marker unlink mitigates)
state/ui_preferences.json	ouroboros/gateway/ui_preferences.py	none — not needed (defaults contract + strict unknown-key rejection; retired keys dropped on write)	bounded (widget_order 200, cursors 1000)	cosmetic; defaults restored
state/server_port	ouroboros/server_entrypoint.py	none — not needed (1-line transport fact)	overwrite; pre-start unlink	readers fall back to default port
state/server_port.bindings.json	ouroboros/server_process.py (record_service_binding, clear_service_binding, shared locked update)	no schema version; exact service keys and measured process identity validated on read	at most main, host_service and local_model; overwrite per observed binding, exact-identity removal on close	current endpoint facts are lost; browser falls back to existing port/custody facts and refuses known endpoints with unconfirmed identity
state/server_process.json	launcher.py	none — PID/path-bound process facts; optional server_host_source distinguishes parent env from saved Settings, absent on old launchers means unknown	one-shot, self-deleting	stray-server reap skipped once; bind-source disclosure becomes unknown, never inferred from value equality
state/auth_secret.key	ouroboros/server_auth.py	none — not needed	write-once	regenerated; one forced owner re-login (accepted)
state/worker_pids.json	supervisor/worker_pool_lifecycle.py	none — not needed (legacy reap path; SSOT is process_ledger)	overwrite ≤ pool size	orphan reap falls to the custody reaper
state/extension_companions.json	ouroboros/extension_companion.py	none — not needed (runtime snapshot)	bounded by live companions	launcher loses force-kill map; leaked child processes possible
state/pending_restart_verify.json	supervisor/evolution_lifecycle.py (write_pending_restart_marker, the one writer — the supervisor's evolution restart and the agent's restart tool in ouroboros/tools/control_runtime.py both go through it)	none — not needed (one-shot marker, consumed by rename to .claimed.<pid>.json)	one-shot	evolution restart refused fail-closed without its receipt
state/advisory_overrides.json	ouroboros/tools/review.py, ouroboros/tools/claude_advisory_review.py	none — not needed (visibility counter; each bypass also durably in events.jsonl)	bounded (recent 10)	bypass counter lost; no gate reads it
state/scope_delivery_migration.json	ouroboros/tools/review_admission.py (disclose_scope_delivery_migration, atomic write)	none — not needed: presence IS the fact (the notice was given)	write-once per install	the one-time review_scope_delivery_migrated notice — a stored bare api scope row now runs a retrieving inspection episode — is emitted again on the next review; no review behaviour depends on the file
state/reviewer_slot_last_execution.json	ouroboros/reviewer_slot_config.py	none — not needed (disclosure, never enforcement)	capped 64, oldest evicted	UI disclosure line lost until next review
state/usage_import_watermark.json	ouroboros/usage_legacy_import.py	none — completion boolean is the contract	write-once	legacy import re-runs, fingerprint-deduped against the ledger (safe)
state/presence_bindings.json	ouroboros/presence_bindings.py (update_json_locked, strict existing dict)	own schema_version: 1 — a mismatched version or a non-dict bindings map is a typed refusal, never a silent migration	overwrite; bounded by the owner-created links	every owner link from a transport room to a behavior skill is lost; presence stops routing those rooms until the links are re-created
state/presence_turn_gate/last-<sha256>.json	ouroboros/presence_runner.py (atomic_write_json, one file per conversation key, written under the conversation lock at the end of an executed turn)	flat JSON: conversation_key, task_id, outcome, message, transport_sends, work_ref, finished_at, delivery; a rebuildable projection, task results and delivery receipts stay canonical	overwrite per executed turn; a cached replay writes only to rebuild a pointer that never landed or names an older turn; the write is best effort (a failure logs a warning, the next replay retries); no age GC (one small file per conversation)	the next turn of that conversation loses the previous-turn fact until the following executed turn, or a replay of the last settled authored turn, rewrites it
state/request_wire_compatibility.json	ouroboros/request_wire_contract.py (update_json_locked)	own schema_version: 1; any other version reads as an empty store	per-profile actions expire on read at REQUEST_WIRE_TTL_SEC (14 days)	learned provider-wire adaptations are forgotten; the next send re-learns them from a fresh provider refusal (one extra failed attempt per route)
state/claudexor_rotation_provisioning.json	ouroboros/claudexor_daemon.py _record_rotation_receipt (atomic text)	none — not needed: a disclosure receipt, never read back for behaviour	last reconcile that actually patched wins	the receipt is lost; provisioning itself is idempotent — the next ensure re-reads the daemon settings and re-POSTs only when the values are absent
state/delegate_terminal_refresh_cursor.json	ouroboros/delegate_terminal.py refresh_recently_settled_terminals (atomic_write_json, best-effort)	none — accepted: a byte offset over the rotated custody chain plus a deferred map, self-grounding (an offset past the chain resets to 0)	overwrite; deferred capped at 500 tasks	the cursor re-grounds at 0 and the one-time historical pass repeats, paced by the 5 MB per-tick scan cap; no terminal evidence is lost
state/extension_generation.json	ouroboros/extension_reconcile_queue.py publish_extension_generation (write-if-changed)	own schema_version: 1	overwrite (one live-extension fingerprint)	an absent marker is NO evidence of divergence (fail-closed): running workers keep their spawn-time extension set until the next publish
state/post_task_evolution_request.json, state/post_task_evolution_counter.json	ouroboros/post_task_evolution.py (atomic publish; the supervisor polls and consumes)	request: own schema_version: 1; counter: none — not needed (a single n)	request is one-shot (consumed by apply_pending_request, deleted by the owner-stop sites); counter overwrites	a pending promotion signal is dropped (no evolution task queued) and the every-N cadence restarts its count; the durable re-arm backstop stays the evolution_owner_stopped state flag, not this file
state/subagent_last_delegation.json	ouroboros/subagent_history.py record_last_delegation (locked atomic, best-effort, canonical data plane; task finalization and common session custody producers)	none — additive disclosure; old single-row receipts remain readable, missing/corrupt data is unknown	latest dated observation per actor, bounded by MAX_CONFIGURED_SUBAGENTS, plus compatible latest receipt; undated occurrence stays unknown and replay does not refresh age	helper history disappears until a new observation; admission is unchanged and full forensics remain in custody/attempt records
state/panic_stop.flag, state/owner_restart_no_resume.flag	ouroboros/server_control.py, server.py (atomic pair)	none — not needed (one-shot flags, consumed on boot)	consumed	absence is the default; next boot auto-resumes

3. state/ — append-only ledgers

Path	Writer	schema/record marker	Retention	Reset
state/usage_attempts.jsonl (+ state/usage_attempts.quarantine.jsonl, state/usage_attempts.lock)	ouroboros/usage_ledger.py single chokepoint (own O_APPEND+fsync under named lock — NOT append_jsonl); ouroboros/usage_compaction.py rewrites it whole (verified candidate, atomic swap) under the same lock	no _schema_version; its own validated contract: dense seq, kind discriminator, state machine, per-row candidate_measurement_kind, attribution physical_attempt_v1; compacted files lead with a stamped usage_baseline header + usage_baseline_group rows — accepted	bounded by compaction (config USAGE_LEDGER_COMPACT_BYTES, 8 MB): terminal non-review attempt chains fold into the baseline block, raw segment archived first (fsync'd) — see docs/USAGE_COMPACTION.md; 20 MB WARN can reflect broken compaction, a large unfoldable residue, or a refused/skipped pass: the name-tier refusal emits usage_ledger_compaction_refused once per process per data root; a policy abort (_Abort) emits usage_ledger_compaction_skipped once per process per (data root, reason); the two snapshot-race exits before archive/swap only log warnings, without a typed event; torn tails — and a charge a compaction swap erased inside its last syscall, read back from the old inode (POSIX) — quarantined, never GC'd	monetary history destroyed, seq restarts, budget fences read $0; watermark survives so legacy import will NOT re-run — deleting the ledger alone is unrecoverable; a ledger reset beside archive/usage_ledger/ makes history queries raise generation newer while an unreferenced segment represents a newer generation and is not a byte-prefix of the live file; after fresh compactions reach the surviving archive generations, those old unreferenced segments are skipped and their attempt IDs remain absent — see history readers; keep or reset the ledger and archive together
state/skill_review_root_tasks.jsonl (+ state/skill_review_root_tasks.gaps.jsonl)	ouroboros/skill_review_history.py (_append_root_task_projection_once: one compact row per root task appended when its skill review lands; a row the projection could not attribute goes to the .gaps.jsonl ledger via _record_root_task_projection_gap)	rows carry usage_attribution_schema: physical_attempt_v1; no version key — accepted (derived index over the per-skill review_history.jsonl, P7)	unbounded append; reads are BOUNDED (skill_readiness.py: 1 MiB / 512 records tail, a truncated or gapped read is disclosed as projection_incomplete) and the startup hot-store check warns past SKILL_REVIEW_ROOT_TASKS_WARN_BYTES	delete with state/; the index is not rebuilt — readiness and the acceptance packet read projection_incomplete until new rows accrue (the per-skill histories keep the truth)
state/process_ledger.jsonl	ouroboros/process_custody.py (spawn chokepoint)	none — downgrade-safe field split (start_time/start_time_boot) is the versioning device — accepted	self-compacting: reapers rewrite survivors-only	prior-generation supervised processes permanently orphaned (fingerprint index lost)
state/evolution_checkpoints.jsonl	ouroboros/evolution_checkpoints.py	schema_version: 1 on every row (+kind on outcome rows)	unbounded append; read bounded (last 200) — accepted (structured solve-capability history is the product)	absorbed/abandoned objectives can be re-proposed (BUG3 regression); the outcome row is DERIVABLE from the resolved campaign transaction, so a row lost to a crash between the two writes is replayed at boot (source: boot_backfill)
state/consciousness_observations.jsonl	none — RETIRED with the observation inbox; nothing writes or reads it	n/a	the alarm clock MOVES a file left by an older version once, unread, to archive/consciousness_observations.jsonl (a name collision gets a _<boot ts> suffix) and never looks again	nothing — the rows are already historical

4. state/ — directories

Path	Writer	schema_version	Retention	Reset
state/extra-ca-bundle/*.pem	ouroboros/net_transport.py (extra_ca_bundle; tmp file + atomic replace)	none — derived (certifi followed by the owner's OUROBOROS_EXTRA_CA_BUNDLE PEM), each file named by the merged bytes' digest	a changed owner file writes a new file; siblings older than a day are pruned	recreated on the next client construction; nothing beyond that process's TLS trust depends on it
state/skills/<name>/ owner state (review.json, review_job.json, grants.json, enabled.json, deps.json, self_authored.json, owner_attestation.json, accepted_rebuttals.json, health.json, uninstalled.json, provenance sidecars, auto_repair.json, presence_profile_state.json)	ouroboros/skill_loader.py, skill_review_runner.py, skill_owner_attestation.py, skill_review_cycles.py, skill_uninstall_state.py, extension_health.py, marketplace/*, ouroboros/gateway/marketplace.py; allowlist SSOT contracts/skill_payload_policy.py	deps.json/self_authored.json/provenance: schema_version: 1; review.json/enabled.json/grants.json/review_job.json/owner_attestation.json/accepted_rebuttals.json: _schema_version: 1 (ABI-2, stamp-on-write — readers keep legacy-0 tolerance, unstamped files never retrofitted); verdict/grant staleness stays pinned by content_hash	no age GC; hub uninstalls write an uninstalled.json tombstone and the startup sweep clears the dead state BY that mark — grants.json survives as owner authority, a reinstall self-heals the tombstone; the gateway's local delete removes the whole state dir	absent state = disabled + pending review + grants revoked (fail-closed); owner_attestation absence invalidates its verdict
state/skills/<name>/review_history.jsonl + review_dispatch/ (legacy review_dispatch.json)	ouroboros/skill_review_history.py	rows carry usage_attribution_schema: physical_attempt_v1; no version key — accepted (derived-counter SSOT, P7)	history unbounded per skill — accepted with BOUNDED reads: every reader windows the 4 MB tail (find_history_job_bounded idiom); lifecycle terminal rows persist their ordinals so counters stay exact inside the window (a group aged past it restarts low — under-counts, never over-blocks); per-skill archive rotation declined (no per-skill archive plane; disclosed)	review-cycle ceiling resets to zero; paid dispatches become free again
state/delegate_project_retirements/<sha256[:24]>.lock	ouroboros/delegate_custody_usage.py (project_retirement_lock: exclusive file lock around one project's settlement/retirement decision; stale after 120 s, owner-aware)	none — not needed (lock file, no payload)	one file per project ever settled; reclaimed as stale by the next holder	delete freely; a live holder re-creates its lock
state/skills/<name>/ transport dirs: extension_calls/, __extension_imports/	ouroboros/extension_process_runner.py, extension_import_staging.py	none — not needed (per-call transport files, staged import trees)	per-call files consumed; import leaves reaped owner-dead+grace	transient; recreated per call
state/delegate_recovery/, state/delegate_recovery_transactions/ (+active.json), state/delegate_supervision/	ouroboros/delegate_recovery.py, delegate_supervision.py (+ startup sweep delegate_state_sweep.py)	own schema: 1 on supervision/transactions; recovery rows fingerprinted, unversioned	terminal+age startup sweep: terminal-status recovery rows (vetoed/adopted), unreferenced transactions and settled-task supervision files past GC retention; live/resumable rows, active.json, no-result tasks and unreadables kept fail-closed; unreadable custody log skips the sweep	interrupted delegated runs cancelled instead of adopted; duplicate wake replay; planned handoffs vetoed
state/delegate_actor_claims/*.lock, state/.payload_delegation_claim.lock	ouroboros/delegate_custody.py, delegate_start_claims.py	none — locks	unlinked on release	no durable state
state/code_intel/<root-sha>/inventory.json	ouroboros/code_intelligence.py	own schema_version: 2 (older/malformed rebuilt silently)	per-repo rewrite in place; stale roots age-pruned at startup by inventory.json mtime past GC retention (pure cache)	pure derived cache; one full re-index
state/extension_reconcile/ (+failed/)	ouroboros/extension_reconcile_queue.py	none — not needed (one-shot markers)	consumed by server loop; after 5 attempts moved to failed/, where markers age-prune past GC retention (the failure fact stays durable in events.jsonl)	pending worker→server reconciles lost; re-toggle heals
state/workspace_executor_processes/	ouroboros/workspace_executor.py	own schema_version: 1 + owner tag	unlink on stop; stale rows filtered at read (pid/cmd-sha)	service processes survive unreaped
state/acceptance_fence_acks/<token>.<req>.json	supervisor/events_worker_reports.py (writer, one file per pooled-worker request), ouroboros/agent.py (the requester reads and unlinks only its own req; direct turns fence in-process and write none)	none — transport ack	inline GC on write (255 newest / 3600 s); a late ack nobody waits for ages out here	the waiting worker re-sends a transition once (a read never), then records a typed supervisor_ack_unavailable unknown — never read as a refusal
state/headless_tasks/<id>/ (child data drives)	ouroboros/headless.py	child state/state.json: schema_version: 1	GC-retention prune at startup (terminal + age; skips artifacts-not-terminal / refs-unpromoted)	in-flight child drives and unpromoted child refs lost
state/cx/ (managed Claudexor runtime)	ouroboros/claudexor_runtime.py	meta files versioned (_NODE_META_SCHEMA_VERSION: 2, pin 1)	deliberate keep-all (rollback selects older pins); staging/displaced temporaries reaped	re-downloaded on demand; nothing durable lost
state/betterleaks/ (managed runtime + cache)	ouroboros/betterleaks_runtime.py	own manifests (schema_version: 1)	keep-all archive cache — accepted (managed runtime)	re-downloaded on demand
state/pycache/, state/python-userbase/	launcher.py, ouroboros/packaged_cli.py, launcher_bootstrap.py (PYTHONPYCACHEPREFIX / PYTHONUSERBASE)	none — caches	no GC; python-userbase is a documented manual-recovery hazard (outranks bundle site-packages)	pycache: always safe (recompile); python-userbase: deletes REAL user-installed deps
state/review_continuations/<task>.json (+ state/review_continuations/archived/**, state/review_continuations/corrupt/** with its .txt reason notes)	ouroboros/task_continuation.py (atomic write; rename to archived/ on retire, replace_atomic to corrupt/ on quarantine)	none — accepted: a typed dataclass with an ownership check (a task_id mismatch raises) and malformed JSON quarantined, never migrated	live payload per blocked task, cleared on resume; a settled, un-resumed continuation is retired to archived/ past RETIRE_SETTLED_CONTINUATION_AFTER_DAYS (7) so history stops riding into every new prompt; archived/ and corrupt/ unbounded — accepted (small typed payloads; the quarantine IS the corruption evidence)	blocked tasks lose their resume pointer: findings, obligations and the commit intent must be re-derived by re-running review
state/skills/<name>/auth_token.json	ouroboros/extension_plugin_api.py mint_skill_token (atomic write, chmod 0600)	none — not needed: the token is bound to content_hash, which IS its staleness contract	one token per skill, rotated on a content-hash change; a transient hash failure reuses or fails closed, never rotates	the next mint issues a new token, so a companion still holding the old one is de-authorized until respawn — do not delete while companions run
state/skills/<name>/repair_admission.json	ouroboros/skill_repair_admission.py record_repair_admission (atomic; the newest admission owns the record)	own schema_version: 1	one record per skill, superseded by the next admitted repair	the repair CAS has nothing to check against and its writes are refused — fail-closed by design, since this record exists to remove exactly that fail-open
state/skills/<name>/jobs/<job>/ (assets/, output/, tmp/)	ouroboros/extension_plugin_api.py skill_job_dir (creates the three children on request; the extension owns the contents)	none — not needed: a workspace, not a record	unbounded: nothing sweeps it, and the gateway's local skill delete is the only path that removes it with the state dir	in-flight extension jobs lose their assets and output; the next call recreates the tree
state/skills/<name>/chat_id_counter.json	ouroboros/gateway/host_service.py allocate_internal_chat_id (atomic, under the in-process counter lock)	none — not needed: range_name plus last/next id	one file per skill; ids descend from A2A_CHAT_ID_MAX	allocation restarts at the top of the range, so a fresh A2A room can reuse a chat id already present in history
state/project_source_locks/	none in this tree — orphan plane seen in live layouts (removed feature leftover)	none	n/a	harmless; nothing reads or recreates it

5. logs/

Path	Writer	Record marker	Retention	Reset
logs/chat.jsonl	supervisor/message_bus.py (+presence, project summaries)	direction + optional type; no version — accepted (projection replayed by chain-aware readers)	rotated 800 KB → archive/chat_*.jsonl; archive chain WARN at 100 MB	newest generation lost; consolidation cursor reports gap (recoverable)
logs/progress.jsonl	supervisor/message_bus.py (+plan review)	type: send_message, is_progress	rotated 800 KB → archive/progress_*.jsonl; 8 MB WARN = rotation broken	current segment lost; readers archive-chain-aware
logs/events.jsonl	~60 modules via append_jsonl (+delegate_custody.emit)	universal type discriminator — accepted (per-type payloads owned by emitters)	rotated 800 KB → archive/events_*.jsonl; custody readers (replay, fault tail-scan, complete_custody_rows, settled-terminal chain cursor, legacy-usage import, swarm rollup, worker-boot verify) are chain-aware; delegated start rows reference their complete raw replay envelope in observability/blobs/** (blob before event), with inline legacy rows still readable; task_received omits only an equal metadata mirror of its top-level task contract; 8 MB live WARN = rotation broken; 100 MB chain WARN = replay degradation	delegated-run custody destroyed (chain incl. archive segments): open runs invisible/unreapable; lineage, citations, legacy-usage source lost
logs/tools.jsonl	ouroboros/loop_tool_execution.py (+budget-drive mirror)	type: tool_call + task lineage (root_task_id/delegation_role from resolve_task_lineage, a direct root is its own root), untruncated args	rotated 800 KB → archive/tools_*.jsonl; tail readers (api_logs_tail, task_events) archive-backfill; 8 MB WARN = rotation broken	untruncated tool record + result_ref pointers lost
logs/supervisor.jsonl	supervisor family, process_custody, gateway control, server shutdown	type (+secondary event_type)	rotated 800 KB → archive/supervisor_*.jsonl + 8 MB tripwire; tail readers (memory.read_jsonl_tail, api_logs_tail) archive-backfill	reap receipts, rescue disclosures, shutdown causes lost
logs/task_reflections.jsonl	ouroboros/reflection.py (+ project-scoped copy under projects/<id>/logs/)	full rows unversioned; pointer rows type: project_reflection_pointer	rotated 800 KB → archive/task_reflections_*.jsonl + 8 MB tripwire; tail-20 read archive-backfills; project-scoped copies follow project retention (never age-pruned)	inter-task memory-carry signal lost
logs/containment_faults.jsonl	ouroboros/delegate_custody.py (mirrored to events.jsonl)	type ∈ CONTAINMENT_FAULT/RESOLVED joined on run_id	unbounded BY DESIGN — read whole so an open fault never ages out — accepted	health invariant degrades to the 4 MB events tail scan (the regression this file fixed)
logs/chat_annotations.jsonl	ouroboros/project_dialogue.py (append_jsonl under the shared sidecar append lock)	type: chat_annotation keyed by (client_message_id, routing_token); latest row per key wins (an owner message keeps one row per routing act; task-authored acts ride synthetic agent-steer:<token> ids)	self-compacting at 800 KB under the append lock: the latest row of every message still present in the chat chain (live chat.jsonl + the 3 newest archive/chat_*.jsonl) is rewritten, the rest is DROPPED — presentation state, so it is not rotated into archive/	annotation cards fall back to their plain chat rows; the one named exception (#198) also loses the durable picker decision-card token, so a pending manual routing choice must be made again
logs/tasks/task_<id>.txt	ouroboros/utils.py log sanitization (write_text, best-effort)	raw oversized task text, no envelope; the log row keeps text_full_path	unbounded: no retention sweep names logs/tasks/	the truncated text in the log row stays; only the spilled full text of oversized task prompts is lost
logs/agent_stdout.log	launcher.py pipe-copy thread	unstructured text	bounded ~8 MB (2 MB × .1..3 backups, rotated by the copy thread)	pre-logging crash output lost; nothing parses it
logs/server.log (+.1..3), logs/launcher.log	stdlib RotatingFileHandler (server.py, launcher.py) with secret-redacting filter; a spawn/forkserver worker importing server.py as __mp_main__ gets a stream handler only (one rotator per file; OUROBOROS_WORKER_START_METHOD=fork inherits the parent's handlers unchanged)	text	bounded ~8 MB (2 MB × 4) — the model citizen	stdlib log history lost; nothing parses it

6. memory/ (Ouroboros cognition — operator read-only)

Path	Writer	Marker	Retention	Reset
memory/identity.md	ouroboros/tools/control_runtime.py (full overwrite, ≥50 chars, shrink notice)	none	unbounded document — accepted (identity is the product)	reseeded default; identity lost (journal keeps history)
memory/scratchpad.md + scratchpad_blocks.json	ouroboros/memory.py (derived, regenerated from blocks under lock)	none	bounded: 10 blocks, eviction journaled first (fail-closed)	regenerated; evicted history in journal
memory/WORLD.md	ouroboros/world_profiler.py (write-once)	none	fixed	regenerates on restart — deletion IS the refresh mechanism
memory/registry.md, memory/deep_review.md	ouroboros/tools/memory_tools.py (section RMW), ouroboros/agent.py (overwrite)	none	unbounded / last-wins — accepted	recreated lazily
memory/dialogue_blocks.json + dialogue_meta.json	ouroboros/consolidator.py, memory_nomination_receipts.py (locked atomic)	pending_knowledge_nominations source-entry IDs; legacy last_unpublished_nominations preserved; era_retry (source hash + effective Light dispatch binding as key + observed route of a not-shorter era)	blocks reduced by era compression (ordinary pass: over 10 blocks, the oldest run of up to 4 SUMMARY blocks before the newest; pressure pass: every complete run, uncapped; an era is never re-compressed — calendar-era recompression is deferred, not implemented — so eras accumulate unbounded); unresolved nomination index unbounded; no tool-level resolver yet, later success never retires old debt	blocks: compressed biography irreproducible; meta: cursor and unpublished-obligation evidence lost
memory/dialogue_summary.md	none — legacy read-only (reader in context.py)	none	frozen	legacy artifact; nothing writes it
memory/knowledge/** (topic .md + index-full.md + patterns.md)	ouroboros/tools/knowledge.py, consolidator.py (index rebuild), reflection.py (patterns CAS rewrite)	none	topic files unbounded — accepted (curated by consolidation); backlog topic merge-only fail-closed	recreated lazily; knowledge lost
memory/*_journal.jsonl, memory/knowledge_history.jsonl, memory/knowledge/patterns_history.jsonl	ouroboros/memory.py, tools/control_runtime.py, tools/knowledge.py, reflection.py — every append through the append_jsonl sidecar-lock seam	scratchpad journal: type rows; others unversioned full-text snapshots; knowledge_history.jsonl source_capture rows carry the host stamp writer/route/writer_input_ref/old_chars/new_chars (rows older than the stamp read unknown), an automatic anchored edit also its authored edits (with basis) and, only when supplied, its summary; historical digested rows retain content_digested: true	complete new old+new snapshots are retained indefinitely; memory_journal_compaction.py is a read-only compatibility entry point, not a source rewriter; existing digest-only rows cannot be restored; the memory_journal_observation startup event gives byte sizes (or missing/unreadable) for the three named journals; scratchpad keeps its eviction journal	deleting the journals loses undo/provenance; eviction/rewrite paths fail closed when journal append fails; historically digested content remains irrecoverable
memory/owner_mailbox/<task>.jsonl + .acks.jsonl	ouroboros/owner_mailbox.py (append-only; revocation appends, reader resolves)	kind discriminator	lifecycle-bounded: unlinked, under the per-task mail lock appends and acknowledgements share, only once the SETTLED canonical row with post-work closed holds every exact unread row AND a verified canonical projection of every input-bearing row's attachments (unread_mailbox.rows/.inputs, task_custody.settle_task_mailbox; the loop thread's seam carries no inputs, the off-loop drive-custody pass and the drive settlement do); a torn read, no result, open post-work, a non-terminal row or an uncarried input keeps it	undelivered owner directives + restart-surviving hurry latch lost; acks lost ⇒ re-delivery

7. Skills payloads, tasks, uploads, projects, services

Path	Writer	Marker	Retention	Reset
skills/{native,clawhub,ouroboroshub,external}/<name>/** (+.staging/, .ouroboros_env/ with cache/ tmp/ home/)	ouroboros/marketplace/*, launcher_bootstrap.py seed, agent self-authoring	provenance sidecars schema_version: 1; env fingerprint.json: 1	no age GC (payloads are installed software); staging rmtree'd per install, crash orphans recognized by name fragments; package caches live with the skill	bucket recreated empty; native seeds NOT re-seeded (deletion intent preserved) except post-bootstrap set; orphaned state/skills/ rows keep grants + tombstone after the startup sweep
state/skills/<name>/dependency_cache/	marketplace/isolated_deps.py via existing installer subprocesses and verified resource downloads	downloaded resources keyed by sha256; package-manager cache formats	reused across payload/environment replacement; follows existing skill-state cleanup	resources are verified/downloaded again and package caches rebuilt
state/skills/<name>/go/, state/skills/<name>/go-cache/	Go compiler launched by tools/skill_exec.py:_run_go_skill, with GOPATH/GOCACHE bound to skill_state_dir	Go-owned cache formats	reused across script runs; follows existing skill-state cleanup	compiler recreates caches; skill payload and review remain unchanged
task_results/<id>.json	ouroboros/task_results.py (locked merge)	_schema_version: 1 (ABI-2); unstamped/future/malformed → quarantine, no conversion — with one carve-out: the boot latch migration re-stamps unstamped rows still at cancel_requested in place (same status, one typed task_result_cancel_latch_admitted event) so a wedged task still reaches its cancelled terminal	UNBOUNDED — one file per task forever, no GC; lifecycle authority is retained deliberately	lifecycle authority lost; drive prunes degrade to age-only; strict authority reads break
task_results/quarantine/	ouroboros/task_result_schema.py (same-dir rename); presence_runner.py reads the retained id before a transport retry	quarantined bytes unchanged; a Presence retry with quarantined authority refuses regeneration	NEVER GC'd (pinned); recovery is manual owner re-stamp	quarantined evidence lost; a previous Presence attempt could be mistaken for a new event
task_results/<id>.custody.lock, task_results/<id>.mail.lock, state/custody_staging/<id>-<token>/, state/custody_trash/<id>-<token>/	ouroboros/task_custody.py (the custody lock every canonical-store publisher takes — copy-back, ref retry, host artifact finalization, mailbox input carry and drive settlement; the mail lock mailbox appends, acknowledgements, mailbox cleanup and the settlement's final check take; private staging copied and verified before publication, then placed create-only; a settled drive moved under the supervisor's queue interlock plus both locks, then deleted)	none — transient	locks released per phase; staging deleted by its settlement; leftovers older than an hour removed by the off-loop drive-custody pass	none: a lost staging copy or trash entry is re-prepared or was already fully in canonical custody
task_results/artifacts/<id>/** (+verification_receipts.jsonl, .directory.*.tmp, .directory.*.json.tmp), task_results/artifact_versions/	ouroboros/artifacts.py, headless.py, outcome_receipt_store.py	artifact and complete directory manifests schema_version: 1; scratch manifest 2	artifact versions bounded (5 per name); artifacts live with their result; directory capture stages ZIP and manifest beside the result, removing owned temporaries on caught failures	deliverable bytes lost; results keep dangling manifests
task_drives/<id>/** (+tmp_scripts/)	ouroboros/headless.py, tools/tool_context.py, tools/shell.py	child stamps as above	settled off the loop thread by the drive-custody pass (terminal + age, default 7 d; task_custody.settle_child_drive); the data/tmp_scripts fallback's hard-kill orphans are swept at startup (when no script can be live), the whole-tree walk for orphaned atomic temps by the first reconcile pass of the generation (sweep_stale_temp_files)	scratch lost; canonical artifacts survive
task_trees/<root>/blackboard.jsonl	ouroboros/task_tree_ledger.py	rows unversioned; snapshot digest schema_version: 1	GC-retention prune at startup (root terminal + age)	swarm coordination facts lost for live trees
state/subagent_worktrees.json (registry; checkouts live OUTSIDE data root)	ouroboros/subagent_worktrees.py	none — malformed → typed refusal (absent = empty is the designed asymmetry) — accepted; file_baseline retains copied binary input identities	prune_orphans (age + missing checkout; skips delegated_exec) + custody-cross-checked snapshot prune (fail-closed on unreadable custody)	permanent leak of checkouts + pinned refs (nothing else names them)
task_results/artifacts/<task>/source_handles/context_checkpoints (focus_source_<reader>-<sha256>.md)	ouroboros/tools/project_journal.py (_retain_focus_source, through artifacts.store_actor_source_bytes) on the canonical root; read by task_finalization.focus_source_projection (digest glob for a historical selector)	native task_source handle on focus.source_handle (size + sha256, verified on read)	write-once, digest-named; lives with the task's artifacts (no separate prune)	a roster's retained_source reads source_unavailable; the focus text itself survives on the task result
task_results/artifacts/<task>/delegated_runs/<run>/engine-files-manifest.json, task_results/artifacts/<task>/delegated_runs/<run>/*-*	ouroboros/delegate_directory.py, existing streamed task-artifact owner	exact engine manifest SHA and per-file before/after digests	canonical result artifacts retained; engine keeps unapplied copy results under its existing disposition/retention owner	loss of complete binary results, baseline references and evidence after execution copy cleanup
uploads/** (+screenshots/, views/, atomic-copy .tmp files)	ouroboros/gateway/files.py through artifacts.copy_artifact_file for chat uploads; tools/browser.py, tools/vision.py, server_owner_routing.py	raw owner bytes; chat ingestion measures size and SHA256 while copying	owner attachments in the uploads/ root: NO retention, owner-explicit delete only; chat upload no longer applies the former 50 MiB cap, while Files-browser and downstream transport limits retain their own contracts; agent-generated screenshots//views/ age out past GC retention at startup	chat attachments dangle (readers skip missing); staged task copies survive
services/<task>/*.log	ouroboros/workspace_executor.py, tools/services.py	none — raw text; archived content becomes observability blob with events.jsonl receipt	GC-retention prune at startup (archive-then-unlink); per-task terminal archive; oversize logs retained live	live tails lost; archived blobs survive
projects/<id>/** (knowledge, journal, workpad, reflections, .project.json marker)	ouroboros/project_facts.py, projects_registry.py	marker unversioned; registry stamped (§2)	NEVER age-pruned (owner curates; delete = durable tombstone)	per-project memory lost; registry row survives, room reappears empty
projects/<id>/knowledge_history.jsonl	ouroboros/knowledge.py through the shared knowledge write lock	append rows retain source topic, revision, operation facts and the host stamp; the former knowledge_journal.jsonl size telemetry beside it (global and project) is no longer written — an existing file is inert	follows the owning project shelf; retained with the project until explicit deletion	history/provenance lost while authored project notes remain
archive/** (rotated segments, rescue/, usage_import/, managed_repo/)	rotation + supervisor/git_ops_rescue.py, usage_legacy_import.py, launcher_bootstrap.py	segments inherit source shape; usage_import carries sha256 sidecar	UNBOUNDED BY DESIGN — durable history, never GC'd (P1) — accepted	memory horizon truncated; rescue copies of uncommitted work destroyed
archive/usage_ledger/segment_*.jsonl	ouroboros/usage_compaction.py (exact pre-compaction ledger bytes, written + fsync'd BEFORE the live swap)	each segment is a whole valid ledger generation; hash-pinned by the live usage_baseline header (source_sha256), chained recursively through each segment's own leading header	UNBOUNDED BY DESIGN — the folded monetary history, never GC'd (P1); read via archived_attempt_ids (tamper-evident, per-attempt joins for the model-send reverse sweep)	folded per-attempt monetary history unrecoverable; live aggregates (baseline block) survive, but seal/attempt joins for folded ids break — the mirror of the ledger row: deleting the ARCHIVE alone under a stamped ledger is a typed chain break on every history question, deleting the LEDGER alone raises generation newer for surviving newer, non-prefix segments; once fresh compactions reach those generations, old unreferenced segments are skipped and their attempt IDs are absent (history readers); reset both together
observability/{calls,blobs,salvaged}/**	ouroboros/observability.py (private 0700/0600, CAS gzip); model-send records beside the call manifests: model_send_seal block + write-once <attempt>.model_send_violation.json typed facts (ouroboros/model_send_seal.py)	call manifests schema_version: 1 + custody/redaction honesty markers; blob refs sha-verified on read; model_send_seal.seal_version: 1 with the canonical_json_v1 basis string	preserved indefinitely BY CONTRACT (the startup census counts, never deletes); the inert OUROBOROS_OBSERVABILITY_RETENTION_DAYS knob is RETIRED (key in RETIRED_SETTING_KEYS)	every recorded result_ref/manifest_ref dangles (strict readers raise); pending delegated request bodies become unknown, while custody identity still blocks replacement and protects snapshots; replay refuses without the recorded body; salvaged outputs unrecoverable; a lost seal on a seam-dispatched attempt surfaces as a typed unlogged_attempt fact at the next startup sweep
claudexor/**	EXTERNAL writer — the claudexord daemon (Ouroboros only mkdirs, appends daemon.log, writes ouroboros-owned.json marker)	marker unversioned	daemon-owned; grows unbounded under our root — disclosed external plane	owner harness logins/profiles lost (fresh device-auth required)
playwright-browsers/	ouroboros/tools/browser.py (vendor install)	none — vendor tree	no GC — accepted (vendor cache)	re-downloaded on next browser use
cache/pip, cache/uv	ouroboros/test_environment.py	package-manager formats; TEST ROOT ONLY, not live data	disposable verification-root lifetime	recreated by explicit dependency setup

Reset ladder (summary)

Always safe (pure caches, recreated): state/pycache, state/code_intel, state/evolution_metrics_cache.json, playwright-browsers/, state/cx, state/betterleaks, lock files, state/server_port. Safe with bounded cost: WORLD.md (regenerates), state/usage_import_watermark.json (safe re-import), ui_preferences.json, auth_secret.key (one re-login). Fail-closed losses (system stays correct, work/authority is forgone): skill state dirs, advisory_review.json, capability_evidence.json, pending_restart_verify.json. Dangerous (authority/history destruction): settings.json, state/usage_attempts.jsonl, task_results/**, logs/events.jsonl, memory/** (including dialogue_meta.json: deletion erases cursor and pending nomination obligations; re-consolidation cannot reconstruct the old IDs), archive/**, observability/**, state/subagent_worktrees.json (leak), claudexor/**, state/python-userbase (real deps).