Six reviewed phases land as one release.
Admission is the outer boundary: every migrated launcher records a manifest before
it can touch the filesystem, and finalizes a typed outcome on every path — success,
refusal, crash, and the real exit status. A structural audit enforces that boundary
across all fourteen launchers, together with confinement computed from the active
checkout and a single manifest publisher, judging by effect rather than by callee
name and failing closed on any write form it cannot resolve.
Harness exit codes are no longer trusted as run status: inspect returns zero for an
eval that raised and harbor returns zero for a job whose trials all errored, so the
launchers now read the harness's own artefact and keep "the harness failed", "it
scored nothing" and "it scored honest zeros" distinguishable.
The acceptance dialogue reconciles receipts through one typed identity that is an
equivalence by construction, so a passing check can no longer clear a red it never
addressed. Prompt caching is normalized at every send site and cached calls stop
under-reporting their input. The owner's context mode becomes explicit and
fail-closed, with one enforcement point for every writer of a disk-authored setting.
Deliberate limits are disclosed in each bench's METHODOLOGY.md rather than implied
by silence. Isolated benchmark egress and the multi-lane script generator are
deferred to a later release with restoration patches and carry-forward notes.
Every claim a benchmark run makes about itself is now either enforced before
money is spent or refused with a durable typed record.
Admission is the outer boundary, and that is a tested property, not a
convention: `admit_benchmark_run()` builds the manifest, WRITES it, and only
then lets the seed gate enforce, raising `BenchmarkAdmissionRefused` with the
refusal already on disk. A `finalize_run_manifest()` context manager records a
typed outcome, a typed refusal/error and the REAL exit status on every exit
path, including an escaping exception (an integer `SystemExit.code` is
preserved). All seven migrated launchers route through both seams; none calls
the builder directly; and an `ast` guard in the seam meta-test fails if any
denylisted operation — filesystem assertions, docker, subprocess, state dumps,
writers — precedes admission. The guard was verified to bite: against the
pre-fix tree it reports four violations, two of which no review round had
reached.
Refusals are one family. `BenchmarkAdmissionRefused`, `RuntimeAttestationRefused`
and `SeedShapeRefused` are all `RuntimeError` subclasses carrying a typed
reason, deliberately not `SystemExit` — a `SystemExit`-raising assertion had
made a refusal handler dead code, because `SystemExit` is not an `Exception`
subclass. A behavioural parity test computes what `raise SystemExit(main())`
hands the OS for each launcher and asserts it EQUALS the `exit_code` the
manifest records, so a recorded status can no longer diverge from reality.
`runtime_attestation()` records both facts about a live server — the HTTP
`runtime_version` from the frozen `/api/health` contract and the local
HEAD/VERSION of its checkout — and requires the contracted field: a bare
`version` key now refuses with the non-overridable `runtime_version_absent`, so
an unrelated HTTP server can no longer attest. `OBO_ALLOW_EVOLVED_VOLUME` waives
only `runtime_skew`, in the shell as well as in Python, and commit availability
is decided before skew so a missing commit cannot be mislabelled. Lineage is a
descent check (`merge-base --is-ancestor`), never equality, so an evolution run
legitimately moving HEAD forward is not corruption.
`CAMPAIGN_FATAL_PROVENANCE_REASONS` is a single shared authority consumed by
both SWE-Pro drivers, which now stop the schedule immediately — before any
volume archival — instead of one driver burning every task.
Grading gains a third state: `pass|fail|ungraded` with typed reasons beside the
UNCHANGED headline formula, plus `grade_summary.json`, and the probe
orchestrator reads that artifact instead of scraping log tokens and refuses to
publish a grade that a failed or stale grader produced. ProgramBench's ledger is
append-only at both the run root and the instance directory, skip rows included.
`common/manifests.py::write_json` is atomic and byte-identical, importing
`ouroboros.utils` lazily so the module stays stdlib-only for the container-side
harbor agent, and `openrouter_key_remaining()` reads the authoritative
`limit_remaining`.
Egress isolation was built in this phase and is NOT part of this release: it is
off by default per owner decision, it produced a finding in five separate review
rounds, and round 10's was a direct recurrence of round 5's, so it was extracted
and deferred with its diagnosis preserved. Solve containers run the same open
network as before v6.75.0. The measured evidence stays in METHODOLOGY: the
official SWE-bench Pro harness does not regulate the solve container's network
at all (`--block_network` applies to the EVAL container), and 31 of 287 attempted
instances are musl/Alpine, 84% of which previously produced a non-empty patch.
GAIA, CL-Bench, Terminal-Bench and OSWorld keep their pre-v6.75.0 behaviour here
and migrate in their own phases. No `ouroboros/` runtime behaviour changes.
Reviewed through eleven rounds of the production commit gate (advisory, triad
`claude-fable-5` + `gpt-5.6-sol` + `gemini-3.5-flash` at high effort, scope
`claude-fable-5`): 24 findings, 22 fixed and 2 refuted with the evidence recorded
in the changelog. Per-phase tags are deliberately not created — a single release
tag lands with the campaign synthesis, an owner-approved deviation from the
tag-per-commit norm.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Commit 3/3 of the bench post-mortem sprint (plan smooth-skipping-charm;
commits 1-2 = bf8bde7 v6.54.3, ef786bb v6.54.4). Minor bump per plan.
Shared bench-template scaffold defaults, disclosed in the benchmarks index
and per-bench METHODOLOGY files: OUROBOROS_MAX_WORKERS=4 (same-model
decomposition slots within one task, never best-of-N; TB 2->4, GAIA
--max-workers default 1->4 with explicit 1 as the strict-baseline ablation
and the quality-profile silent 1->5 bump removed, SWE-pro 5->4),
OUROBOROS_SAFETY_MODE=light (disposable jails; deterministic guards stay),
RUNTIME_MODE=pro for container benches (GAIA deliberately stays light: its
solver runs unisolated against a live repo), and claude_code_edit disabled
in every bench solve task (single-model harness measurement). TB raises
_DEADLINE_SAFETY_SEC 30->105 from measured finalization overhead, mirrors
the registry _WEB_TOOLS denylist (youtube_transcript drift fixed, pinned by
a sync test), and the README pins why --all-model keeps review single-model.
ProgramBench e2e runner ported from the colleague's tree, adapted to our
APIs: metadata.budget_profile onto the v6.54.4 contract (footer keys
dropped, reserve pct unit-fixed 0.15->15), solve-model preflight through
migrate_model_value (fail-fast on direct-route legacy ids), atomic
per-instance checkpoints with honored reattach (client poll timeout leaves
the executor alive; the next run skips seed/start and reattaches), explicit
payload-status terminal detection, denominator-preserving ledgers
(skipped_existing rows; skipped = successful for exit code), source-only
submissions (both root binaries excluded by name), idempotent workspace
normalization shared by the prepare-only flow.
New continual_learning/ launcher wraps the EXTERNAL CL-Bench runner
(nothing vendored; fails loudly with obtain instructions), strict-sequential
guard (parallel opt-in is exclusively-stateless-bridge only), standard-path
pointer rows keep the ledger denominator, fidelity gaps of the pinned
external adapter recorded and warned. OSWorld aligned to the official 2.0
protocol (pinned OSWorld-V2@c261cb57, 500-step default, show_result.py
layout, official traj rows), bridge-level final_answer capture with
VM-state-only prompting, preflight verifies the target server's effective
scaffold settings via /api/settings (env cannot configure a --url server)
and refuses the live desktop URL without an explicit flag.
LifelongAgentBench documented as blocked (its 100% run was a no-gold-oracle
artifact). Carriers synced to 6.55.0. Review: 6 triad+scope rounds to a
fully clean verdict (16 confirmed criticals fixed, each with a regression
test); gemini/opus clean in all rounds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add denominator-preserving benchmark manifests and result ledgers across the devtools adapters, tighten Terminal-Bench/Harbor and OSWorld fail-closed preflights, and harden workspace patch artifacts for unborn, invalid, and acting-subagent git baselines.\n\nVerification: focused benchmark/headless/acting suites, isolated full pytest, adversarial review rounds, and real Ouroboros triad/scope review artifacts under bench_runs/devtools_sync_20260607_010949/reviews.
