Second parent is the frozen upstream `ouroboros` head (23ab428f, 407 commits
since the merge base a76961de); first parent is v7.0.0-rc.8 (18b9832e).
Every upstream change lands in v7's owning leaf: S1 transplants keep upstream's
bodies (comments verbatim) under the call-time handle idiom, S2 hand-merges keep
both intents, S3 keeps v7 only with proof (retired 7.0 ABI surfaces, superseded
mechanisms). Per-symbol relocation ledger: docs/archive/v7next/LEDGER_CORRECTIONS.md
(F2 absorption section). Provisional decisions awaiting owner ratification:
D-18 (two-destination symbols), D-19 (acceptance rows follow upstream R2),
D-20 (acceptance_dialogue stays deleted), D-21 (tools/registry.py: facade
import block only).
Docs: upstream ARCHITECTURE/DEVELOPMENT as the base with compact v7 deltas;
bookkeeping moved to docs/archive/v7next. Size-ratchet manifest, domain
manifest and generated inventories regenerated; new leaves: tools/write_shape
walker, gateway/cost_breakdown, tools/core_secret_paths; provider_catalogs.py
and acceptance_dialogue.py removed (v7 owners).
Integrates the moved target (193 commits over the sprint base b9bcc2da,
release 6.114.0 — the DeepSeek landing PR #563 and the docs-consolidation
PR #556 included) into the synthesis branch without rewriting history.
Four files conflicted; each was resolved by substance so BOTH sides' facts
and behaviours survive:
- ouroboros/size_ratchet_manifest.py (generated): UNION of both sides'
BAND_PATHS rationales (the target's ouroboros/gateway/extensions.py,
tests/test_provider_contract_ci.py, tests/test_ui_smoke_project_continuity.py
and web/modules/settings_ui.js beside our acceptance_dialogue.py,
deep_self_review.py and test-suite rows), then regenerated for the merged
tree: ouroboros/gateway/history.py left the band on the target line,
BYTE_DEBT is the live merged size everywhere (loop.py 272905,
tests/test_devtools_benchmarks.py 328068 — below both parents — and
web/modules/chat.js 206949). `scripts/regenerate_size_ratchet.py --check`
is green; no new module, function or byte debt.
- docs/ARCHITECTURE.md and docs/DEVELOPMENT.md: the target's consolidated
structure is the frame; every agentic-review fact of this branch is placed
in the target's section or row — the three deliveries of task acceptance
and deep self-review, the reviewer-row schema (`deep_review` singleton,
roster references, `profile_id`), pacing simplified to the one admission
floor (R52/R55; the EWMA sentences are gone), poll purity, the native
read receipts and BIBLE.md coverage, the retrieving work order and the
CI methodology job. The module-tree rows keep the target's condensed form
extended with our contracts; the full contracts live once in §6 (Task
lifecycle, Review delivery, Deep self-review). Stale target sentences
that our side retired (the `api_chat` acceptance pin, the round cap, the
legacy/default API panel residual) are replaced, never duplicated.
- ouroboros/review_native_episode.py: our side already measures the send
bound as the wire size of the serialized message list, which carries the
WHOLE assistant dict — `reasoning_content` included — so the target's
fix (count replayed reasoning in the fail-closed bound, 295c9062) is
subsumed; the target's regression test passes unchanged. The comment
above the append records the invariant.
Auto-merged both-changed files were checked for silent overlap:
tests/conftest.py gained the same autouse os.environ snapshot/restore
fixture on both sides — the target's tested `_os_environ_isolation` is
kept and our redundant `_restore_process_environment_between_tests` is
dropped (its rationale folded into the surviving docstring); our
gateway-settings binding restore fixture stays. .github/workflows/ci.yml,
ouroboros/config.py, ouroboros/tools/control.py, web/modules/settings_ui.js
and web/modules/reviewer_slots.js carry both sides' changes exactly once
(our deep self-review block sits in the target's new `.reviewer-slots-group`
container like its advisory sibling).
The launcher structural gate (test_every_migrated_launcher_passes_the_structural_gate)
refused the previous commit: reading the host settings file before
admit_benchmark_run() is a refusal with no durable manifest. The metadata is
now rendered FIRST inside the finalize seam — a malformed reviewer panel is a
typed refusal recorded on the durable manifest (stage leaderboard_metadata),
before any submission tree exists — and the two side-effect-ordering comments
describe the real seam (assert_outside_repo is pure; run_root materializes
with the manifest).
Round-2 finding: scope-review models were still declared although scope review
is a commit-time gate that never fires inside a Terminal-Bench task — the same
overstatement as the advisory. Scope rows and defaults are no longer declared
on either the structured or the legacy path; the older assertion that the scope
default "assists the run" pinned that overstatement and is rewritten. The
shrink-only test file's byte debt ratchets down with it.
Review fix batch 5 (wave 2 on 75c78ca2..295c9062; triad-codex
run-18ba6271145f, scope-sol run-7a1f9dd087ee, audit run-1c5c25e16be7).
Critical fixes:
- llm.py: the OpenRouter-branch reasoning_content scrub keyed on value
truthiness, so a legal empty-string echo (or a legacy null) rode the OR
wire; the guard is now key-presence (+falsy-residue regression).
- devtools/benchmarks/common/server_runner.py: DEEPSEEK_ joins
_AUTHORITATIVE_ENV_PREFIXES — an ambient DEEPSEEK_API_KEY survived the
settings-authoritative sweep that promises to strip provider families.
- devtools/benchmarks/programbench: the _active_direct_provider mirror of
config._exclusive_direct_remote_provider_env gains the minimax and
deepseek rows it silently omitted.
- devtools/benchmarks/terminal_bench: _network_preflight gains deepseek
(fixed DEEPSEEK_BASE_URL) and minimax (resolve_minimax_base_url over
MINIMAX_REGION, now forwarded by _container_env) branches — the agent
injected both keys but probed neither endpoint.
- tests/test_deepseek_provider.py: the density-witness regression clears
the process-global _DENSITY_MEMO in a finally (the memo key omits
drive_root, so leftovers poison co-located tests).
Registry-first regressions in the new
tests/test_benchmark_provider_touchpoints.py pin all three benchmark
surfaces against PROVIDER_CREDENTIAL_GROUPS, and both test-side provider
scrub tuples now derive from that registry instead of hardcoding keys
(test_devtools_benchmarks.py stays under its byte-debt ceiling: 328100).
Advisory closures: deepseek rows in the model-catalog and onboarding test
pins (deepseek-only setup, suggestions, profile derivation); honest doc
comments (ci.yml optional-provider rows, pyproject integration marker,
colab collection docstring, server_runtime provider lists, the stale
_EFFORT_CARRYING_PROVIDERS reference in llm.py); the
estimate_message_chars docstring and ARCHITECTURE row now name its real
consumer (local-context compaction proxy) instead of claiming the shared
fit/density basis; size manifest regenerated for the shrunken debt file.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The structured OUROBOROS_REVIEWER_SLOTS is the ONE reviewer configuration
surface. The legacy migration-read block (reviewer_slot_config.py:
_shared_session_route_spec / _legacy_rows / _legacy_config — comma-lists,
phase-5 per-row/advisory route envs, global-effort copying) is DELETED; with
no structured value the loader serves the SHIPPED DEFAULT PANEL: api_chat
triad/scope rows over get_review_models()/get_scope_review_models() (the
derived env plane — identical models to the old legacy defaults on every
config class, incl. single-direct-provider adaptation and a bench launcher's
env override), default advisory row, source="default", historical positional
slot ids (receipts keep lining up) and the unchanged legacy fingerprint
identity for the unconfigured panel (skill-review replay authority survives
the upgrade).
Settings vocabulary: OUROBOROS_REVIEW_MODELS / OUROBOROS_SCOPE_REVIEW_MODELS /
OUROBOROS_SCOPE_REVIEW_MODEL plus the phase-5 route envs
(OUROBOROS_REVIEW_ROUTES / OUROBOROS_SCOPE_REVIEW_ROUTES /
OUROBOROS_ADVISORY_REVIEW_ROUTE) leave SETTINGS_DEFAULTS and join
RETIRED_SETTING_KEYS (ghost purge on load; an install that configured
reviewers only through them gets the default panel — the RC auditor names
this migration, per plan). The derived runtime projection
(project_reviewer_slots_into_env) STAYS, floored from
OPENROUTER_REVIEW_DEFAULTS; get_review_models/get_scope_review_models keep
serving the API-pinned surfaces from that env plane. server_runtime's
provider-defaults migration now normalizes a ghost comma value it is FED but
never INTRODUCES a retired key (the direct-provider review adaptation lives
on the read side); the singular→plural promotion in model_slots is gone
(purged before it could run). gateway/settings candidate probes read the
structured candidate else the live derived config.
Bench templates migrated to structured slots with the SAME models
(continual_learning, gaia, swe_bench_pro base/example/probe/profile; comma
keys dropped everywhere incl. osworld/cybergym/programbench which already
carried structured values); run_tb metadata defaults now come from
OPENROUTER_REVIEW_DEFAULTS; manifests record OUROBOROS_REVIEWER_SLOTS.
tests/test_comma_list_sweep.py is the phase CI gate (ABI-10 hook): no
migration-read branches, retired vocabulary pinned, bench templates clean,
prose rewritten, derived projection alive. ARCHITECTURE deltas same commit.
(cherry picked from commit e7578133758f01eac584e8e6522b3f03c4d09818)
BREAKING (ABI 7.0 window, owner Q10=A verbatim: 'FLOOR/fail_tasks/
deadline-алиасы — удалить'). Both knobs were deprecated one-minor aliases
whose own event announced 'removal: next_major' - this is that major.
- contracts/task_contract.py: VALID_IMPROVEMENT_POLICIES = (fixed, adaptive);
an unknown policy (incl. the retired spelling) normalizes to 'fixed';
stall_rounds_threshold leaves the normalized profile shape (it was
normalized but consumed by nothing).
- task_pacing.py: the deprecated_task_pacing_alias event machinery in
resolve_budget_profile is gone; the until_deadline count-axis lift in
effective_max_improvement_passes is gone, and with it the has_deadline
parameter that existed solely for that branch (callers in task_results
and the rails line updated; the deadline/reserve TIME rail is untouched).
- bench adapters (sanctioned explicitly): programbench schemas.py + README
and swe_bench_pro entrypoint_pro.sh + METHODOLOGY switch to
improvement_policy=fixed - behavior-identical for those runs because their
explicit max_improvement_passes=6 was ALWAYS the binding count axis under
every policy; stall_rounds_threshold=12 dropped (never consumed).
- docs: ARCHITECTURE task_pacing module line and the acceptance passage now
state the removal.
- tests: the aliases' own tests removed (review_cycles alias test + rails A3
clause, two v6544 until_deadline tests); vehicle tests adapted to the
surviving semantics (wallet-authority test now derives its uncapped lane
from the unlimited shared cap; v664 deprecation-noise test now pins that
NO deprecation events are emitted at all; headless CLI forwards 'adaptive';
contract-shape and PB/SWE-Pro expectation pins updated). NEW removal pins
in tests/test_abi5_q10_removals.py: policy tuple, normalize-away shape,
signature no longer takes has_deadline, functional-remnant sweep.
Disclosed consequence: a PRE-7.0 stored root contract whose normalized
profile says until_deadline is judged malformed by the acceptance-wallet
authority (pre-existing unknown-policy behavior); pre-7.0 task-result
history is quarantined wholesale by ABI-2 (Q8=B) in this same release, and
the ABI-7 RC auditor names the migration.
Gates: ruff F clean; size_ratchet 5 passed; affected suites 478+90 passed.
- domains.toml: D14 ten leaves (extension_loader six + skill_review four) and
D08 sixteen leaves mapped; extension_loader/skill_review split rows fully
retired; control/events/queue/workers leaves-rows shrunk to their
hot-deferred remainders (cancel/custody family, D07 half of control).
- D08-pick conflict resolved: the reload_all suite lives in the D14 sibling
test_extension_reload_all.py on this tree, so its worker_main clause got
the D08 adaptation there (worker_process.py owner); the giant kept HEAD.
- test_gaia_events_serializer_carries_web_search_sources re-homed from the
test_devtools_benchmarks giant into tests/test_events_llm_usage.py (its
post-split thematic owner): the byte-debt ratchet correctly refused the
in-place retarget (+79 bytes on a shrink-only file) and the re-home is the
designed pressure valve - the giant shrinks, the pin gains its family.
- LEDGER_CORRECTIONS: coordinator section with the five D13 dispositions
(live protected safety.py deltas incl. the UNROWED _safety_drive_root fix
that must gain a carried row at F5; shell_guards rebind pending with D05
wave; runtime_mode_policy remainder returns with its leaves).
- Quotient report regenerated: 1316 strict module edges, 163 domain edges.
Module side. 22 D08 owners classified: 4 byte-identical across tip/reference
(schedule_contract, supervisor/__init__, active_activity, schedule_time);
6 pure upstream drift - tip bytes stand (promotion_source, tools/followup,
message_bus, queue_transitions, state, task_admission); 4 NEW post-cutoff
upstream modules with no ledger rows - untouched (cognitive_operations,
log_addressing, subagent_task_truth, task_dispatch); 4 monoliths split.
The split: 16 leaves, 142 moved spans, every span transplant-tool proof-green
against git show HEAD:<monolith> (ast=tokens=byte-roundtrip on every symbol,
leaf_invariants=[], unread_declared=[]). supervisor/events.py 4476->1947:
eight handler families (chat_delivery, subagent_admission, schedule_task,
project_routing, coop_checkpoint, budget, worker_reports, runtime_controls).
supervisor/queue.py 1587->1265: queue_schedules on the _queue handle.
supervisor/workers.py 4396->2808: worker_promotion/chat_lane/pool_lifecycle
on the _pool handle + worker_process (the child, no pool state).
tools/control.py 3225->2110 (SHARED D07/D08): only the D08 leaves
control_events/control_routing/control_runtime; D07 rows untouched.
Facades = tip parent - moved spans + re-export block, facade audit green
(every kept span byte-identical to tip, every moved name re-exported).
Drift-probe first per leaf: 4 reference leaves fully byte-true, 12 with
41 upstream-drifted spans re-emitted from tip bytes; no oracle semantics
replayed over drift.
HOT-DEFERRED with evidence (cancel/custody D09-class; upstream 65b5d19f
re-decomposed this ownership): events_task_done family, _handle_cancel_task,
_close_campaign_after_owner_stop, events_evolution_done, queue_snapshot,
queue_timeouts, queue_evolution (upstream's own evolution_lifecycle.py
supersedes), worker_assignment, worker_health. Deferred semantic-delta rows:
D06 events taxonomy (dispatch/EVENT_HANDLERS keep tip bytes), D04 retired
timeout knobs, six retired-name rows. Row 2016 (_handle_schedule_task)
deferred on a mechanism finding: its >300-line FUNCTION_DEBT entry is
(path, qualname)-keyed and this tree's transition validator has no D11
same-qualname relocation rule - the handler stays with its debt key.
Test side: identity suites test_events_extraction (5, pins the deferred
inventory as the F2 work order), test_control_extraction (5),
test_worker_process_extraction (5, reference-verbatim); LEAVES table in
test_module_handle_extraction gains six tool-derived declared-set rows.
Dead-patch class re-pointed to owner leaves mirroring reference adaptations
across 13 files (coop quiescence, schedules, worker_process trio, promote
admission seam -> control_events, pool-disabled probe -> control_routing,
evolution restart claims -> control_runtime alias, budget append_jsonl ->
events_budget, ephemeral-turn source scan -> worker_chat_lane, routing
producers AST scan -> control_routing, worker_main scan -> worker_process,
resource-leak source contracts reference-verbatim); all 15 touched test
files lossless (test-name multisets equal), no new ast-identical dup bodies.
HOT_CODE_PATHS mirrors the 12 hot leaves (D04-block precedent).
size-ratchet manifest regenerated with the official tool (queue.py enters
the 1001-1500 band with rationale; giants shrink in place); ratchet lane
5 passed. ruff check . --select F clean. CI-shape battery on this tree:
parallel 11965 passed rc=0, serial 609 passed rc=0. Import smoke fwd+rev
green; worker_main stays picklable from supervisor.worker_process.
docs/v7next/LEDGER_CORRECTIONS.md: D08 lane section (11 entries).
(cherry picked from commit 1426ea3ae2413580992575f5ab890497469a6657)
Integrates four upstream merges (#337 nested-coordination, #352 cybergym,
#353 mainleak, #358 review-checkpoint). Conflict resolution adopts the
new harness_presentation identity SSOT (SVG marks + branded labels; the
chip's icon field retires) while keeping this branch's chip truth states
- labels become "Codex - 2 ok" style and the review_truth pins move with
them.
Also carries heal part 1 for the target's latent size-ratchet debt (the
drifted tip fails its own blocking CI ratchet lane): config.py returns to
exactly 1600 lines via the settings_integrity extraction (snapshot pin +
write guards, config.X re-exports, deduped verified-read paths),
supervisor/events.py and the benchmarks test drop under their byte caps,
and the cybergym executor sheds its wire layer (typed faults, urllib
transport, telemetry parsing) into cybergym_wire.py.
The pre-implementation gate reviewed an unbounded prose plan with a ~100k-token
governance pack, planning-scout subagents and an Atlas, and every rewritten plan
minted a fresh fingerprint that bought a whole new paid wave from cold reviewers.
Convergence was structurally impossible: the reviewer was asked to author a
competing plan every wave, REVISE_PLAN could not be dispositioned, and nothing
capped the cycles.
plan_task now reviews an INTENTION — the same organ whether the work is code,
research, a deliverable or an action in the world:
- a typed domain-neutral SPEC (goal, in_scope, non_goals, acceptance_claims,
invariants, decisions with rejected alternatives, deferred, affected_resources,
evidence) with host-minted ids that are the only valid `breaks` targets;
- ONE structural fact tiers the governance pack: `constitutional` iff a declared
target resolves under the system repo (never prose, never a plan-kind taxonomy);
- agent-declared evidence, bounded, with EVERY absence named, the runtime data
plane denied outright, and the exploration log redacted through the same SSOT
task acceptance uses;
- typed findings (blocking with a `breaks` id | note | need_evidence) with the
HOST computing the aggregate through adaptive_quorum — no reviewer emits GREEN
as authority, none writes a competing plan;
- ONE owner setting OUROBOROS_REVIEW_MAX_CYCLES (default 2, unlimited available)
bounds paid cycles for plan review, task acceptance (passes = cycles - 1) and
the commit gate's identical-diff attempt cap; an identical envelope replays for
free and a DEGRADED wave costs nothing;
- under blocking, an open plan holds implementation and a spent cap escalates
with a typed review_cycles_exhausted reason and an honest blocked terminal;
under advisory the agent may proceed with the wave open and a loud disclosure.
Deleted: planning scouts, the plan Atlas, plan_class, context_level, the
governance mega-pack, the generative reviewer stance, the hidden 32-wave limit
and the api_chat-only pin. Net effect on the tree is negative.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Phase C of the poltergeist delegation sprint, squash-landed onto the v6.99.0
nanny release. A mutating delegated run never edits the shared tree: the host
provisions a private execution snapshot of the authority target (baseline built
with the sensitive-veto decided before anything is hashed, pinned by a
refs/ouroboros/delegated/ ref), records the durable binding {execution_root,
baseline_sha, target_root, authority_source} on the custody rows BEFORE the
POST, and replays it byte-identically on explicit retry — pending-invocation
recovery carries the full binding so the startup GC (settled && patch_disposed)
never deletes the snapshot holding the child's only work. Terminal
reconciliation captures a settled run's diff through one drive-rooted capture
core, eagerly only at PROVEN terminality; an absent or unreadable close
captures nothing, capture-at-disposition is the retry point (C1-R2), a capture
failing there is the typed INTEGRATE_DELEGATED_CAPTURE_FAILED refusal for both
decisions, and patch_captured means a usable artifact exists (C1-R3 — a failed
manifest never mints the row and reject re-checks before releasing the
snapshot). Nothing lands without the explicit integrate_delegated_patch
apply/reject flow (proven baseline drift, NUL-safe touched paths, cleanup
following the durable disposition row, the protected-path gate scoped to the
Ouroboros body), and open obligations surface via undisposed_patches. Beside
it: SSOT cost projection (accounted_upper_bound_usd beside deprecated
cost_usd, $0-fabrication fixes, honest web cost presentation),
delegated_runs_failed on the evidence receipt, notification chat routing,
byte-accurate argv/env budgeting with --prompt-file transport, and hash-bound
skill repair.
Merge resolution UNIONS phase B's delegate module split (delegate_shared stays
the one refusal/emit/ownership author; delegate_integration imports it),
delegate_answer beside integrate_delegated_patch on every child surface, the
instructions-bearing idempotency digest, and BR2's additive terminal_detail
carry in cancel_and_verify with C's absent-branch no-capture semantics. Size
gates cleared by extraction, no grandfathering: delegate_evidence.py
(task_execution_evidence out of delegate_custody) and synthesis_cost_text.py
(synthesis cost renderers out of agent_task_pipeline), both re-exported
same-object; _delegate_start hands its started payload to one extracted
author.
The formal six-lane exact-SHA gate plus two verified fix rounds (CR1: the
startup GC fails closed over an unreadable custody log, the capture artifact
is readable across a split-drive boundary through a narrow rebind, a durable
apply intent precedes any tree mutation so a crash replay can never record a
false rejection, and delegate_start tells the snapshot truth; CR2: the
AMBIGUOUS apply state gained an explicit owner-acknowledged exit through the
normal disposition guards, a lost prune-skip row is escalated loudly, and a
failed verdict write can no longer strand a pending intent) are squashed into
this landing.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Two new repo-lane editing primitives beyond exact-match edit_text and
full-file write_file, both validated on the included editbench benchmark
before adoption:
- edit_batch: atomic batch of COUNTED exact replacements. Each edit
declares the occurrence count it expects and replaces all of them; any
mismatch aborts the whole batch with per-edit diagnostics. The safe
replace-all: a miscount is an atomic refusal, not a silent corruption.
- apply_patch: context-anchored multi-file patch (V4A-style, no line
numbers; optional @@ anchors; trailing-whitespace fuzzy fallback).
Atomic across all files/hunks with per-hunk diagnostics; parser
tolerates decorative trailing *** on headers (observed model quirk).
write_file repo lane gains two verification rails inherited from an
edit_sketch fast-apply tool that was implemented, benchmarked, and
REJECTED (data in devtools/benchmarks/editbench/README.md): a pre-write
syntax guard for .py/.json (force bypasses with an explicit
SYNTAX_GUARD_BYPASSED disclosure — no silent bypass, P3) and the unified
diff vs the previous version appended to overwrite results (capped with
an explicit truncation count, P1).
Registration follows the New Tool rule across every canonical surface:
safety.py TOOL_POLICY, tool_capabilities (CORE/ACTING envelopes + result
limits; local-readonly and heal lanes deliberately excluded), registry
(_FROZEN_TOOL_MODULES/_WORKSPACE_ALLOWED_TOOLS/_REPO_MUTATION_TOOLS),
outcomes (_ROOT_WRITE_TOOLS so patch/batch-only turns stay acceptance-
review-eligible; _RECOVERY_TOOL_NAMES), smoke EXPECTED_TOOLS, and the
SYSTEM.md/ARCHITECTURE.md/DEVELOPMENT.md/CHECKLISTS.md documentation in
the same commit (P6). The edit_text/write_file descriptions steer tool
selection at the schema source (per DEVELOPMENT's anti-pattern guidance,
not SYSTEM.md prose accretion).
devtools/benchmarks/editbench/ is the evidence and the regression rig:
5 editing tasks (trap-laden rename, surgical edit in a 1691-line module,
whole-function replacement, cross-file move, near-full-file transform),
per-config tool restriction via disabled_tools, deterministic grading,
token/latency/error mining. Migrated under the benchmark admission
contract (admit_benchmark_run/finalize_run_manifest, launcher_audit
MIGRATED_LAUNCHERS); it measures the current working tree by design, so
real runs use the recorded --allow-dirty-seed escape.
Headline numbers (grok-4.5, 3 runs/config, every run solved every task;
full tables in the editbench README): edit_batch finishes the rename
task in ONE tool call vs ~6 for edit_text/write_file; on a weak model
(gemini-3.6-flash) its advantage grows to 1.5-2.5x cheaper/faster; a
free-choice agent picked the right tool per task shape in every run.
No version carriers touched (maintainer assigns the release version).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the v6.82 mobile swipe gestures, make drawer opening clear keyboard state, and keep deeply nested live cards usable in narrow consumers. Set fresh-install triad defaults to Luna, Gemini Flash, and Sonnet while preserving saved settings.
Co-authored-by: Ouroboros <ouroboros@local.mac>
scrub_submission_secrets.py walked `root.rglob("*")` filtered by `p.is_file()` and
wrote with `path.write_text()`. Both follow symlinks, so the tool had two failures,
each demonstrated against the pre-fix code.
A FILE symlink was swept THROUGH: the write landed on the link's target, outside
--root. A pack containing a link to the live settings.json therefore had its real
keys replaced with <REDACTED:...> by the tool whose job is to protect them. `cp -a`
preserves symlinks, so the procedural "run this on a COPY" rule did not help.
A DIRECTORY symlink was worse. rglob does not descend through one, so its contents
were never enumerated — and the verify pass then printed verify_leftovers=0 and
exited 0. The tool affirmatively certified a tree it had never read, for content
reachable under --root and about to be uploaded to a public leaderboard. Silent
non-coverage reported as cleanliness is exactly the class of false claim this
release exists to remove, and here the consequence is a live API key published.
Both are now a hard refusal before a single byte is written, reusing the fail-closed
discipline the --env-passthrough refusal already established: every offending link is
named with its target and its kind, nothing is modified, exit 2. Refusing beats
resolving-and-continuing, because under --root a symlink is either an accident or an
escape and only the operator can say which. With the links removed the tool sweeps
and verifies exactly as before, so this refuses an unsafe shape rather than removing
a capability.
Pre-fix, the new test fails with `assert 0 == 2` — the old tool reported success.
Both defects were invisible to every phase on its own base and appeared only once
the chain was assembled. Neither was caught by the full gate battery, because
neither was pinned by a test — which is the actual finding.
_CACHE_INCLUSIVE_PROMPT_TOKEN_PROVIDERS omitted "anthropic". P6 introduced the set
correctly, while the direct-Anthropic path still reported bare input_tokens. P3
then made that path report prompt_tokens = input + cache_read + cache_creation.
On the merged tree every cache-bearing direct-Anthropic send therefore failed the
membership test and returned early, so P6's token-density learning was vacuous on
the main and heavy slots — precisely the routes it exists to measure — and every
review pack was frozen at the cold-start density. The docstring asserted the
opposite of the tree's behaviour, in a release whose thesis is that artefacts must
not claim what did not happen. GigaChat stays excluded: its precached_prompt_tokens
semantics are still undocumented.
ARCHITECTURE.md stated the migrated-launcher count twice and contradicted itself —
"ELEVEN" from P2/P3 and "ten" from P5, surviving side by side in one 33 KB physical
line that git merged without a conflict marker. The registry holds fourteen. The
first claim is now explicitly historical, the second states the current count, and
a test pins any unqualified count to len(MIGRATED_LAUNCHERS) so a future migration
that forgets the prose fails instead of shipping a document that misdescribes its
own gate. Also repaired the README roll-off sentence, where 6.76.0 had been
appended after the closing clause instead of inside the parenthesised list.
Tests added in both directions for the density gate, which had no coverage at all.
Six reviewed phases land as one release.
Admission is the outer boundary: every migrated launcher records a manifest before
it can touch the filesystem, and finalizes a typed outcome on every path — success,
refusal, crash, and the real exit status. A structural audit enforces that boundary
across all fourteen launchers, together with confinement computed from the active
checkout and a single manifest publisher, judging by effect rather than by callee
name and failing closed on any write form it cannot resolve.
Harness exit codes are no longer trusted as run status: inspect returns zero for an
eval that raised and harbor returns zero for a job whose trials all errored, so the
launchers now read the harness's own artefact and keep "the harness failed", "it
scored nothing" and "it scored honest zeros" distinguishable.
The acceptance dialogue reconciles receipts through one typed identity that is an
equivalence by construction, so a passing check can no longer clear a red it never
addressed. Prompt caching is normalized at every send site and cached calls stop
under-reporting their input. The owner's context mode becomes explicit and
fail-closed, with one enforcement point for every writer of a disk-authored setting.
Deliberate limits are disclosed in each bench's METHODOLOGY.md rather than implied
by silence. Isolated benchmark egress and the multi-lane script generator are
deferred to a later release with restoration patches and carry-forward notes.
Every claim a benchmark run makes about itself is now either enforced before
money is spent or refused with a durable typed record.
Admission is the outer boundary, and that is a tested property, not a
convention: `admit_benchmark_run()` builds the manifest, WRITES it, and only
then lets the seed gate enforce, raising `BenchmarkAdmissionRefused` with the
refusal already on disk. A `finalize_run_manifest()` context manager records a
typed outcome, a typed refusal/error and the REAL exit status on every exit
path, including an escaping exception (an integer `SystemExit.code` is
preserved). All seven migrated launchers route through both seams; none calls
the builder directly; and an `ast` guard in the seam meta-test fails if any
denylisted operation — filesystem assertions, docker, subprocess, state dumps,
writers — precedes admission. The guard was verified to bite: against the
pre-fix tree it reports four violations, two of which no review round had
reached.
Refusals are one family. `BenchmarkAdmissionRefused`, `RuntimeAttestationRefused`
and `SeedShapeRefused` are all `RuntimeError` subclasses carrying a typed
reason, deliberately not `SystemExit` — a `SystemExit`-raising assertion had
made a refusal handler dead code, because `SystemExit` is not an `Exception`
subclass. A behavioural parity test computes what `raise SystemExit(main())`
hands the OS for each launcher and asserts it EQUALS the `exit_code` the
manifest records, so a recorded status can no longer diverge from reality.
`runtime_attestation()` records both facts about a live server — the HTTP
`runtime_version` from the frozen `/api/health` contract and the local
HEAD/VERSION of its checkout — and requires the contracted field: a bare
`version` key now refuses with the non-overridable `runtime_version_absent`, so
an unrelated HTTP server can no longer attest. `OBO_ALLOW_EVOLVED_VOLUME` waives
only `runtime_skew`, in the shell as well as in Python, and commit availability
is decided before skew so a missing commit cannot be mislabelled. Lineage is a
descent check (`merge-base --is-ancestor`), never equality, so an evolution run
legitimately moving HEAD forward is not corruption.
`CAMPAIGN_FATAL_PROVENANCE_REASONS` is a single shared authority consumed by
both SWE-Pro drivers, which now stop the schedule immediately — before any
volume archival — instead of one driver burning every task.
Grading gains a third state: `pass|fail|ungraded` with typed reasons beside the
UNCHANGED headline formula, plus `grade_summary.json`, and the probe
orchestrator reads that artifact instead of scraping log tokens and refuses to
publish a grade that a failed or stale grader produced. ProgramBench's ledger is
append-only at both the run root and the instance directory, skip rows included.
`common/manifests.py::write_json` is atomic and byte-identical, importing
`ouroboros.utils` lazily so the module stays stdlib-only for the container-side
harbor agent, and `openrouter_key_remaining()` reads the authoritative
`limit_remaining`.
Egress isolation was built in this phase and is NOT part of this release: it is
off by default per owner decision, it produced a finding in five separate review
rounds, and round 10's was a direct recurrence of round 5's, so it was extracted
and deferred with its diagnosis preserved. Solve containers run the same open
network as before v6.75.0. The measured evidence stays in METHODOLOGY: the
official SWE-bench Pro harness does not regulate the solve container's network
at all (`--block_network` applies to the EVAL container), and 31 of 287 attempted
instances are musl/Alpine, 84% of which previously produced a non-empty patch.
GAIA, CL-Bench, Terminal-Bench and OSWorld keep their pre-v6.75.0 behaviour here
and migrate in their own phases. No `ouroboros/` runtime behaviour changes.
Reviewed through eleven rounds of the production commit gate (advisory, triad
`claude-fable-5` + `gpt-5.6-sol` + `gemini-3.5-flash` at high effort, scope
`claude-fable-5`): 24 findings, 22 fixed and 2 refuted with the evidence recorded
in the changelog. Per-phase tags are deliberately not created — a single release
tag lands with the campaign synthesis, an owner-approved deviation from the
tag-per-commit norm.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Root cause (PB cmatsuoka__figlet smoke): the agent committed a compiling state, then broke the tree with one last uncommitted edit as the acceptance improvement loop hit its pass cap; the harness ships the LIVE tree, so the verified commit protected nothing — all salvage machinery guards the answer TEXT only. Prompt-only fix (P5) over existing channels (P7): the acceptance rails line marks the last admitted improvement pass (passes_done+1 >= cap, cap>0) as FINAL and, for workspace deliveries (canonical is_workspace_mode() with an attribute fallback), EVERY workspace improvement capsule carries the tree directive — a deadline or cost rail can end the loop between capsules (triad r1) — keeping the tree at a VERIFIED state (rebuild, verify, and commit if the task calls for a commit; revert unverified edits); the 10% deadline flush AND the cost wrap-up note gain one shared commit-NEUTRAL tree sentence (acting self_worktree subagents cannot commit; a moved HEAD fails patch capture closed), byte-identical for non-workspace tasks; the ProgramBench instruction now states the exporter's true submission contract (SOURCE tarball from the CURRENT tree state; .git, root binaries, .ouroboros/ and named build/cache noise excluded at any depth; run ./compile.sh one final time), replacing the false fresh-checkout framing. Tests pin the pacing/rails seam, the workspace gating on both axes, byte-identical non-workspace texts, the real tarball contract, and the instruction truth phrases. P9 carriers bumped to 6.74.4; changelog row added. Disclosed residual (this is a MITIGATION of the incident class, not a closure): a forced tool-less exit — deadline grace or budget stop crossed inside one long round with no pacing note or capsule in the terminal stretch — can still ship an unverified last edit; the structural verification-freshness seam is a filed follow-up pending an owner decision (triad r1-r3, codex full-access review concur).
They imported inspect_ai directly — an optional benchmark dependency absent on CI runners — and failed quick-test with ModuleNotFoundError. The tests now inject a fake inspect_ai.util.sandbox module via monkeypatch, keeping the success-path coverage on every environment. P9 carriers bumped to 6.74.2; changelog row added (oldest patch row rolled off). No runtime code changes.
The v6.74.0 tag CI failed on the deterministic ruff F-rule gate: an unused types.SimpleNamespace import in tests/test_devtools_benchmarks.py (added with the final GAIA staging tests). Import removed; P9 carriers bumped to 6.74.1; changelog patch row added (oldest patch row rolled off). No runtime code changes. Fix-forward per protocol — the published v6.74.0 tag is not re-tagged.
A: verdict-visible improvement capsule (verdict+tier+real blocker via one panel_reason reducer, open obligation ids, money/time/rounds/passes rails line, three real moves); reviewer-authored obligation identity (disposition_kind new|re_raise validated against the host catalog, fail-closed to new, per-pass dedup, reuse-immutable) with surviving rebuttals (previous_disposition/previous_reason/reopened_count); typed dialogue_status (continue_actionable|unreachable_here|stable_disagreement) reduced over ALL contract-valid actors with the panel quorum — termination is reviewer-authored or a real rail, never a host counter.
B: two cache-marked review-prompt segments (governance + task-stable contract), slot label off byte 0, breakpoint cap asserted on the final payload.
C: GAIA sandbox attachment staging (prompt-declared /shared_files channel, exact shared-root lookup, per-file provenance, typed per-sample infra error); CLB acceptance-claims operator patch (receipt-bound claims, knowledge nudge in the steer slot, bounded cost-finality wait); SWE-Pro cumulative shard budget (auto_run derives per_task x scheduled, run_pro seeds from cumulative spend); CLI/PB bounded task_cost_finalized waits for completed/degraded only.
D: light-mode shell guard resolves cwd before judging repo targets (resource-root label false-block fixed); post-task cost publish uses try_get_bridge.
E: generative surface-duty in commit/plan review checklists; all P9 carriers bumped.
Reviewed: 2 adversarial rounds + production triad+scope (PASSED) + codex final (GO); smoke 5x5 on all five benches verified the dialogue/staging/budget mechanics live.
TB2.1 leaderboard static analysis matches config.agents[].name against the
agent name trials report; bare --agent-import-path records name=null, which
can never match (terminal-bench-2-1#121) and is unfixable after upload.
- run_tb.py: generate agent_job_config.json (name = adapter class name() +
import_path + kwargs) and launch harbor with -c instead of bare agent
flags; OUROBOROS_EFFORT_TASK at launch is recorded as the declared
reasoning_effort of the submission key.
- harbor_installed_agent.py: accept the reasoning_effort kwarg and forward
it into the container as OUROBOROS_EFFORT_TASK (declared == effective).
- README/METHODOLOGY: replace the removed 'harbor leaderboard submit' CLI
flow with the verified GitHub-PR flow (terminal-bench-2-1 lb submit,
promotion, /judge, /apply), document the named-config requirement,
trajectory_path pitfalls and the TUS upload fix pointer.
- tests: assert the new -c job-config shape (name/import_path/kwargs/effort).
Generalizable runtime + harness improvements distilled from the 35-FAIL SWE-bench Pro
autopsy, the ProgramBench improvement spec, and the TB2.1 forensics — one minor release,
no benchmark overfit.
Runtime (A/C):
- Scratch is now idempotent/adoptable: an existing untracked in-cwd file can be declared
scratch (sha recorded at declaration, re-declaration idempotent); tracked files and
directories stay blocked. The undeclared-output guard is stat-verified (real filesystem
writes, not string-shape) and, for run_script, audits on EVERY exit path — a write-then-
fail body no longer smuggles an undeclared user_files deliverable.
- Cost axis: budget_profile.cost_hard_stop_pct (0..100; None→50% historical; 0 = NO in-task
cost stop, deadline/rounds/global-gate only; malformed → 50%, never a silent 0). Latched
cost milestones (50/25/10% + wrap-up) replace the old [INFO] nudge. CLI --task-metadata-json
merges into body.metadata with host-owned keys (delegation_role/source) unforgeable.
Bench harness (D/PB/TB):
- SWE-Pro e1v2: /app is the default active workspace (via user_files root), empty memory
default, budget metadata (until_deadline, cost_hard_stop_pct=0) threaded through; bench1
fixes (shard-safe auto_run, k=1, timeline) ported into the repo; the image-load lock and
utility-image use are cross-platform (platform_layer, preflight-pulled, fail-safe restore).
- Workspace acting-integration allowlist (+integrate/compare_subagent_patches) with a
fail-closed self_worktree-under-external-workspace integration guard.
- Protected-artifact policy round-2: structural (operation-identity) false-positive
exceptions for differential-testing harnesses — vcs_diff/stat classifier, spawn-argv=argv[0]
execute, interpreter script/module operand screening, writer-target segmentation — while
read/copy/hash/introspection of the black-box reference stay blocked (incl. alias-separated
reads); pinned by a new permanent policy test.
- ProgramBench instruction v2, until_deadline profile, acceptance claims. TB ffmpeg resolver
chain (platform_layer/imageio-ffmpeg/PATH) + agent-prefix install + cv2 fallback hint.
- 4 verify-guidance clauses in prompts/SYSTEM.md (probe the consumer interface, exercise every
provided input, robust-under-ambiguity, never read hidden graders).
Docs/versions: 6.56.0 across VERSION/pyproject/web/api_types/README/ARCHITECTURE; SWE-Pro +
TB METHODOLOGY (workspace/budget/memory/retry-gate disclosure), PB gate-20 operator method,
pricing sonnet-5, DEVELOPMENT scratch contract.
Reviewed via operator triad (openai/gpt-5.5 + google/gemini-3.5-flash + anthropic/claude-fable-5)
+ scope (anthropic/claude-fable-5) to convergence: scope clean; residual triad items are a
verified reviewer false-positive and the plan-accepted single-parameter budget-threading
tradeoff on a grandfathered core-loop function (advisory-bypass, documented).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The GAIA answer key is cached on the host (metadata.parquet Final answer column +
per-task .jsonld under ~/.cache/inspect_evals and the HF dataset cache), and every
solver runs its agent as a host subprocess with shell access — so the agent can read
the answers off disk (observed: a Codex row ran `find … gaia_dataset … jq
<sample>.jsonld`; the leakage audit caught it). This is a filesystem sibling of the
web-lookup cheat.
Wrap every solver subprocess (Ouroboros + codex/claude/hermes, symmetrically) in
`bwrap --dev-bind / /` with the answer-cache directories masked by empty tmpfs,
leaving the rest of the FS, the network (web search), and CLI config intact. The
inspect scorer runs in the main process outside the wrapper, so scoring is unaffected.
New bwrap_isolate.py (default-on; GAIA_BWRAP_ISOLATE=0 to disable; fails loudly if
bwrap is missing rather than running silently unprotected). Disclosed in METHODOLOGY.
Verified: answer cache invisible inside the wrap; codex/claude solve end-to-end;
the Ouroboros dedicated server binds loopback and runs inside the namespace.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-07-04 GAIA runs caught agents looking up the published answer key
(googling "GAIA benchmark ... answer", downloading metadata.jsonl from mirrors)
and running the weak strict_ddgs retriever instead of native web search — the
latter mislabeled as parity with the native-searching Codex/Claude Code targets.
- Anti-lookup prompt: SSOT GAIA_ANTI_LEAK_INSTRUCTION appended by all four
solvers, worded to avoid the benchmark name / "FINAL ANSWER" marker / the
leak-query regex so it neither primes the model nor self-flags in traces.
- Full trace capture for audit parity: Claude Code -> --output-format
stream-json --verbose (per-sample claude_code_trace.jsonl); Codex -> --json
(codex_trace.jsonl). Both were black boxes before.
- Observability: persist usage.web_search_sources on llm_usage rows in
events.jsonl so the audit can see native (server-side) web-search citations,
which never appear in tools.jsonl (behavior-neutral).
- Audit v2 (audit_leakage.py + new SSOT leak_targets.py): strip prompt
boilerplate before scanning traces (no self-flag); catch quoted "FINAL ANSWER"
queries and dash-named GitHub mirror paths; STRONG gold-verbatim only when the
gold answer came FROM a leak source (plain gold in a normal page is weak);
scan events.jsonl native citations by root_task_id; parse CC/codex JSONL
traces; fix WEB_TOOLS (+skill_exec/start_service, -nonexistent fetch_url/bash).
- Scoring: score_gaia.py --leakage-audit reports leakage_adjusted_accuracy
(STRONG-flagged samples counted incorrect) alongside the authoritative score.
- Docs: METHODOLOGY (quality_openrouter_web is the publishable default; strict_ddgs
is a handicap, not parity; pre-registered zeroing rule; per-harness coverage),
README_harness_compare (who-searches table), ARCHITECTURE (three web-access
mechanisms and their model/observability consequences).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Commit 3/3 of the bench post-mortem sprint (plan smooth-skipping-charm;
commits 1-2 = bf8bde7 v6.54.3, ef786bb v6.54.4). Minor bump per plan.
Shared bench-template scaffold defaults, disclosed in the benchmarks index
and per-bench METHODOLOGY files: OUROBOROS_MAX_WORKERS=4 (same-model
decomposition slots within one task, never best-of-N; TB 2->4, GAIA
--max-workers default 1->4 with explicit 1 as the strict-baseline ablation
and the quality-profile silent 1->5 bump removed, SWE-pro 5->4),
OUROBOROS_SAFETY_MODE=light (disposable jails; deterministic guards stay),
RUNTIME_MODE=pro for container benches (GAIA deliberately stays light: its
solver runs unisolated against a live repo), and claude_code_edit disabled
in every bench solve task (single-model harness measurement). TB raises
_DEADLINE_SAFETY_SEC 30->105 from measured finalization overhead, mirrors
the registry _WEB_TOOLS denylist (youtube_transcript drift fixed, pinned by
a sync test), and the README pins why --all-model keeps review single-model.
ProgramBench e2e runner ported from the colleague's tree, adapted to our
APIs: metadata.budget_profile onto the v6.54.4 contract (footer keys
dropped, reserve pct unit-fixed 0.15->15), solve-model preflight through
migrate_model_value (fail-fast on direct-route legacy ids), atomic
per-instance checkpoints with honored reattach (client poll timeout leaves
the executor alive; the next run skips seed/start and reattaches), explicit
payload-status terminal detection, denominator-preserving ledgers
(skipped_existing rows; skipped = successful for exit code), source-only
submissions (both root binaries excluded by name), idempotent workspace
normalization shared by the prepare-only flow.
New continual_learning/ launcher wraps the EXTERNAL CL-Bench runner
(nothing vendored; fails loudly with obtain instructions), strict-sequential
guard (parallel opt-in is exclusively-stateless-bridge only), standard-path
pointer rows keep the ledger denominator, fidelity gaps of the pinned
external adapter recorded and warned. OSWorld aligned to the official 2.0
protocol (pinned OSWorld-V2@c261cb57, 500-step default, show_result.py
layout, official traj rows), bridge-level final_answer capture with
VM-state-only prompting, preflight verifies the target server's effective
scaffold settings via /api/settings (env cannot configure a --url server)
and refuses the live desktop URL without an explicit flag.
LifelongAgentBench documented as blocked (its 100% run was a no-gold-oracle
artifact). Carriers synced to 6.55.0. Review: 6 triad+scope rounds to a
fully clean verdict (16 confirmed criticals fixed, each with a regression
test); gemini/opus clean in all rounds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
run_gaia.py gains --sample-id (comma-separated ids) to replay a chosen subset of tasks, mirrored into the Inspect argv. _write_manifest now records the EXACT selected ids via a shared _requested_task_ids() helper when --sample-id is set (else the limit-derived level list), so a sample-id rerun no longer writes a manifest claiming the first N level tasks — keeping the audit/denominator contract honest. Focused test covers the manifest selector and the argv lockstep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(improvement_backlog): stub semantic-dedup LLM in groom tests (P6)
_seed_many() seeds items via append_backlog_items(), whose C9.2 semantic-redirect
pre-pass calls semantic_dedup.find_semantic_duplicate_id() once per fingerprint-MISS
with candidates — a real light-model NETWORK call. The seeding runs BEFORE
_patch_groom_llm installs its mock, and that mock only covers chat_observed, not the
detector's own client path. With no API key the call retry-storms for minutes before
failing open to None, making test_groom_backlog_rejects_invented_items ~129s alone
(~40% of the whole suite) and non-deterministic.
Add a module-level autouse fixture that stubs find_semantic_duplicate_id to its own
fail-open default (None = no duplicate — exactly what the doomed call eventually
returns for the distinct seeded items), so the module is network-free and
deterministic. No production change; the dedup contract stays covered by
test_semantic_dedup_v6370.
Effect: test_groom_backlog_rejects_invented_items 129.12s -> 0.02s; whole file
~165s -> 2.37s, all 14 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(core): validate regex up front in _code_search so invalid-regex contract holds for both backends (P3)
_code_search ran the ripgrep path first and returned its formatted result before
ever reaching the Python-fallback re.compile guard. ripgrep accepts some malformed
patterns permissively — an unterminated '[' yields "no matches" instead of erroring
— so an invalid regex like "[invalid" silently returned no-match on the rg path while
only the fallback (rg absent/failed) emitted "⚠️ SEARCH_ERROR: invalid regex". This
made test_code_search_invalid_regex fail whenever ripgrep is present (i.e. always, in
CI and the pre-push preflight), so the suite exited non-zero on every candidate diff
regardless of the change under test.
Compile the regex once up front (regex queries only; literal queries need no check)
and return SEARCH_ERROR on re.error before dispatching to either backend, so both the
rg path and the Python fallback share the same invalid-regex contract.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: guard the conftest repo-root pollution sweep to the xdist controller (xdist-safety)
Under pytest-xdist, pytest_sessionfinish fires on the controller AND every worker against
the SHARED repo root, so the mock-pollution sweep (shutil.rmtree of leaked <MagicMock>
paths + session.exitstatus=1) had workers racing the same rmtree and each independently
failing the run — a non-deterministic, failed-shaped result. Guard the repo-root sweep +
exitstatus mutation behind `if not hasattr(session.config, "workerinput")` so it runs only
on the controller (the single authority); the per-process _PYTEST_DATA_DIR cleanup stays
outside the guard and runs on every process. Serial runs are unaffected (the guard is
always True without -n).
Adds tests/test_conftest_xdist_guard.py (sweep runs on the controller, skipped on a worker).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: parallelize the test suite with pytest-xdist (~9x faster), keep the gate serial
The full-suite CI jobs (quick-test, full-test) now run a PARALLEL pass plus a short SERIAL
pass. Empirically ~270s serial -> ~30s parallel (~9x). The per-commit preflight GATE stays
serial by design (a flaky parallel fail-closed gate manufactures non-deterministic
TESTS_FAILED indistinguishable from a real immune rejection).
- requirements: pytest-xdist + pytest-timeout (a hang-guard).
- pyproject: register the `serial` marker (addopts unchanged).
- ci.yml (quick-test, full-test): two-step. Parallel
`-m "not serial and <default lane exclusions>" -n auto --dist loadscope
--max-worker-restart=0 --timeout=300 --timeout-method=thread`, then serial
`-m "serial and <exclusions>"`. A command-line -m REPLACES the pyproject addopts markexpr,
so the default lane exclusions are repeated and ANDed with the serial split — the union
exactly reproduces the old default suite (4594 parallel + 122 serial = 4716, disjoint).
- conftest: a tryfirst pytest_collection_modifyitems hook marks the real-process/port files
(workspace_executor[+cleanup], process_custody, kill_process_tree_orphans, zombie_prevention,
worker_crash_retry, process_resource_leaks, restart_reconnect, preflight_runner,
services_tool_v2) `serial`; plus an autouse fixture isolating workspace_executor._SERVICES/
_FOREGROUND between tests (a latent ordering bug -n redistribution exposes).
Test-isolation fixes surfaced by running the suite under -n:
- test_task_constraint_tools: 3 bare `sys.modules["...claude_code"] = mock` (no restore ->
polluted the worker's sys.modules -> later SDK-dependent tests failed) -> monkeypatch.setitem.
- test_workspace_executor: poll until the spawned process's command-sha is readable before
registering (the PID-reuse safety check compared the sha recorded at registration vs
recomputed at kill; right after fork+exec the command line is unreadable -> shas diverge ->
kill silently skipped -> flaky), + 5->15s kill-confirmation deadlines.
docs/DEVELOPMENT.md + docs/CHECKLISTS.md: guidance so future tests are parallel-safe or
marked `serial`. Reviewed by adversarial subagents (ship); verified parallel 8/8 + serial 3/3
green, partition exact + disjoint, gate byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* tests+ci: parallel-safety follow-ups (services-global isolation, serial-lane guard, monkeypatch conversions)
Three non-blocking follow-ups from the parallelization review, hardening the
xdist split shipped earlier in this PR:
F1 (tests/conftest.py): extend the autouse _isolate_workspace_executor_globals
fixture to ALSO snapshot/clear/restore ouroboros.tools.services._SERVICES (the
legacy services registry, a separate module-global from workspace_executor's)
under its plain threading.Lock _LOCK. Closes the legacy-services-path
global-leak class generally; raw dict ops only under the lock (no re-entrant
deadlock on the plain Lock), registry-only (never reaps the live Popen
handles), each module lazy-imported under its own guard.
F2 (.github/workflows/ci.yml): add a "Guard non-empty serial marker lane" step
to marker-guards. `pytest --collect-only -m serial` exits 5 on an emptied
_SERIAL_TEST_FILES; set -euo pipefail + tee surfaces it, and a positive anchor
grep on test_workspace_executor.py is the working assertion (the existing
browser-guard's `! grep "no tests collected"` is a dead no-op under -q).
F3 (test_skill_loader / test_iteration2_fixes / test_marketplace_clawhub /
test_devtools_benchmarks): convert remaining bare os.environ / sys.modules
mutations (no save-restore) to auto-reverting monkeypatch.setenv/delenv/setitem,
so the parallel suite has no cross-test env/module leaks. tests/_shared.py's
intentional process-wide SDK mock left untouched.
Verified: parallel lane 4574 pass (-n auto --dist loadscope), serial lane 140
pass, the 4 converted files 159 pass. Two subagent reviews: no regressions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: prune 26 redundant/tautological/obsolete tests (immune coverage retained)
Three-pass audit (finder+skeptic -> 31 empirical verifiers with rename+mutation
experiments -> delete-and-run dry-run) of the 275-file / 4067-test suite found 26
tests whose value is fully backstopped by a stronger survivor or is near-zero:
- tautological / zero-production pins (assert hasattr/callable; inline-simulated
branches; closed-loop regex; TypedDict dict-literal; CPython-only checks);
- proven duplicates with strict-superset twins (test_consolidator, test_commit_gate,
test_cache_optimization, test_review_intent_split, TestGrepRegexHint, ...);
- registry-registration one-liners, all backstopped by test_smoke::test_tool_set_matches
(set-equality vs EXPECTED_TOOLS -- mutation-proven to FAIL on a lost registration);
- dead-feature tests (retired skill_migrations module) + obsolete version regressions
(v636 import now unconditional; README "(N tests)" convention abandoned).
Removes ~47 test functions across 20 files (incl. the whole test_shell_regex_hint.py,
mirrored 1:1 in test_shell_run_shell.py::TestGrepRegexHint, and the whole
TestGoalScopePrecedence class). Also drops 2 now-unused imports + 1 orphaned helper.
KEPT (not redundant): test_smoke::test_git_commit_with_tests_exists -- sole guard of the
post-commit gate seam (a rename experiment proved nothing else catches its loss).
Verified: ruff F clean; collection exit 0 (4703 collected, no emptied class); full
parallel + serial suites green; two subagent reviews confirm exact set + no broken refs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Andrew <andgri200@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The v6.46.0 GAIA adapter could not start its dedicated benchmark server while the
desktop app was running: (1) it bound the default Host Service port 8767, which the
live app holds, crashing startup ("port 8767 busy"); (2) settings_base.json ships
EMPTY provider-key placeholders, which apply_settings_to_env pops from os.environ,
erasing the real keys _sanitized_host_env preserved ("No supported provider configured").
Fix (devtools-only, no runtime-core change):
- _free_port(): auto-pick distinct free ports for BOTH the main server (GAIA_OUROBOROS_URL
/ OUROBOROS_SERVER_PORT) and the Host Service (OUROBOROS_HOST_SERVICE_PORT), so the bench
server coexists with the desktop app and parallel configs never collide on a port.
- _resolve_provider_keys(): inject the REAL keys for the configured models + the pinned web
backend (from os.environ, then data/settings.json) into the rendered settings so the
server detects the provider; keys land only in the isolated, gitignored run dir. 'auto'
web pulls all keys (cascade); a pinned backend only its own; OPENAI_BASE_URL is dropped
for an 'openai' web pin (official web_search needs an empty base_url).
Verified out-of-the-box: plain run_gaia.py started the dedicated server cleanly (free Host
Service port, Supervisor ready, key injected) alongside a running desktop app + two live
L1 runs. New focused test (test_gaia_render_injects_keys_and_free_host_service_port) + all
90 devtools tests green; ruff F clean. devtools-only follow-up to v6.46.0 (no version bump,
matching the devtools-commit precedent).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A GAIA validation audit surfaced harness artifacts that masked Ouroboros's real
capability. This lands the GENERALIZABLE fixes (helping all users, not fitting the
benchmark), a Max-mode generative context-window probe, and a reviewed GAIA adapter.
Runtime:
- Budget exhaustion TERMINALIZES undispatched PENDING tasks (`fail_tasks`,
reason_code=budget_exhausted + task_done event, canonical budget_drive_root,
cancel-requested -> CANCELLED) instead of a silent never-dispatched freeze.
- `search_code` gains a shared wall-clock deadline (rg walk + batch loop + Python
fallback, one budget) and a DISTINCT deadline-truncation signal so a timed-out
search never reads as an authoritative "no matches" (OUROBOROS_SEARCH_CODE_WALL_SEC).
- `web_search` backend pin (OUROBOROS_WEBSEARCH_BACKEND: auto|ddgs|openai|openrouter|
anthropic; openai/ddgs are TRUE pins, no cascade); `ddgs` is now a dependency with
429 backoff so a fixed-model run can force pure retrieval.
- `OUROBOROS_OR_PROVIDER` (resilience/repro presets) routes OpenRouter providers for
rate-limit resilience without losing prompt-cache warmth (gap-merged; never overrides
the require_parameters / reasoning-continuity pins), behind a supports_message_cache_control predicate.
Answer integrity: a tool-count-stamped latch recovers a produced FINAL ANSWER whose
marker a later (un-grounded) round drops, while new tool work invalidates a stale latch
(Q7 no-downgrade); acceptance review on the ADVISORY task surface stops over-failing a
grounded answer (surface-keyed: majority-not-single-FAIL, no solved->DEGRADED on empty
coach). The P3 commit/scope immune gate is untouched.
Security: `user_files` honors OUROBOROS_USER_FILES_ROOT (jailed scratch home; '~'/'~user'
and unnamed deliverables stay IN the jail); the authoritative observability blob is
redacted by default (env-only OUROBOROS_OBSERVABILITY_KEEP_RAW opt-in) with an allowlist
secret matcher that spares cognitive content (sha/commit/uuid/route_fp/model/answer).
Capability: a generative context-window probe (public LLMClient.probe_oversized_context
seam) empirically confirms a route's >=1M window from a FREE over-window reject — only on
the explicit Max toggle/save, never auto-confirming a possibly-paid 200, fail-closed,
bypasses a stale lazy cache; plus a cloud.ru route base_url fix.
Bench: the GAIA adapter (devtools) stages attachments robustly, injects the FINAL ANSWER
protocol, crash-isolates per-sample timeouts (+ server-side ceiling so a hung sample
cannot orphan the shared queue), and exposes per-config knobs (vision slot, review models,
web backend, budget, epochs) with cross-provider credential preservation.
Version carriers bumped 6.45.5 -> 6.46.0; MAX_TOTAL_FUNCTIONS 3520 -> 3524 (reviewed
single-purpose helpers); ARCHITECTURE/README updated. Reviewed to convergence (adversarial
subagents + codex gpt-5.5 + real triad+scope) with a green full suite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keep GAIA run roots valid when their own per-run settings/data env is active, and retry the transient supervisor startup window so the first sample does not flake before producing a structured answer.
WIP checkpoint (local, not pushed) — to be reviewed-merged into repo/ after the re-run.
Fixes two TB2.1 methodology bugs that depressed gpt-5.5's measured score:
1. Network: the adapter set allowed_resources={web:false,network:false} on
disable_agent_web, which blocked `git clone` even though TB2.1 tasks need it
(every task.toml allow_internet=true; verifier clones in-container). Replace
with a declarative `disabled_tools` tool-policy: withhold the agent's own
web_search/browse/browser/analyze_screenshot/vlm_query tools (anti-reward-hack,
== original _WEB_TOOLS set) while leaving shell egress (git/pip/curl) intact.
- task_contract.disabled_tools (additive frozen-ABI field) + normalize_disabled_tools
- registry hides disabled tools from schemas()/core_only/get_schema_by_name/
available_tools (+ dynamic extension/MCP) and blocks them at execute
- /api/tasks + TaskCreateRequest (+ api_types.js mirror) thread the field
- propagates to subagents via the parent-contract spread
- view_image intentionally kept (local image-to-model, outside _WEB_TOOLS)
2. Deadline: Harbor's per-task wall-clock cap was never plumbed into Ouroboros
(deadline_at=""), so pacing/graceful-finalize were dormant and the agent was
hard-killed mid-work with empty results.
- D1: adapter resolves the per-task [agent].timeout_sec from cached task.toml,
subtracts elapsed install/server time + 30s safety, passes timeout_sec ->
deadline_at so the agent paces and self-finalizes a partial before the kill
- D2: surface inherited deadline_at from task_contract into task_metadata so
spawned subagents also pace/finalize
- _capture_current_task_summary records captured_after_cancellation only on a
real interruption (not a terminal infra_failed exit), so the disclosure
ledger no longer misreads a terminal provider_unavailable as a cancellation
Plus the run_tb disclosure-ledger honest taxonomy (pass/provider_infra/cancelled/
genuine) and tests (test_disabled_tools_policy.py, test_run_tb_methodology regression,
adapter terminal-nonzero-exit test) + docs/ARCHITECTURE.md ABI rows.
Reviewed: codex (gpt-5.5) + 7 rounds triad+scope to convergence; only the
version-bump touchpoint remains, intentionally deferred to the eventual merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
