Two new repo-lane editing primitives beyond exact-match edit_text and
full-file write_file, both validated on the included editbench benchmark
before adoption:
- edit_batch: atomic batch of COUNTED exact replacements. Each edit
declares the occurrence count it expects and replaces all of them; any
mismatch aborts the whole batch with per-edit diagnostics. The safe
replace-all: a miscount is an atomic refusal, not a silent corruption.
- apply_patch: context-anchored multi-file patch (V4A-style, no line
numbers; optional @@ anchors; trailing-whitespace fuzzy fallback).
Atomic across all files/hunks with per-hunk diagnostics; parser
tolerates decorative trailing *** on headers (observed model quirk).
write_file repo lane gains two verification rails inherited from an
edit_sketch fast-apply tool that was implemented, benchmarked, and
REJECTED (data in devtools/benchmarks/editbench/README.md): a pre-write
syntax guard for .py/.json (force bypasses with an explicit
SYNTAX_GUARD_BYPASSED disclosure — no silent bypass, P3) and the unified
diff vs the previous version appended to overwrite results (capped with
an explicit truncation count, P1).
Registration follows the New Tool rule across every canonical surface:
safety.py TOOL_POLICY, tool_capabilities (CORE/ACTING envelopes + result
limits; local-readonly and heal lanes deliberately excluded), registry
(_FROZEN_TOOL_MODULES/_WORKSPACE_ALLOWED_TOOLS/_REPO_MUTATION_TOOLS),
outcomes (_ROOT_WRITE_TOOLS so patch/batch-only turns stay acceptance-
review-eligible; _RECOVERY_TOOL_NAMES), smoke EXPECTED_TOOLS, and the
SYSTEM.md/ARCHITECTURE.md/DEVELOPMENT.md/CHECKLISTS.md documentation in
the same commit (P6). The edit_text/write_file descriptions steer tool
selection at the schema source (per DEVELOPMENT's anti-pattern guidance,
not SYSTEM.md prose accretion).
devtools/benchmarks/editbench/ is the evidence and the regression rig:
5 editing tasks (trap-laden rename, surgical edit in a 1691-line module,
whole-function replacement, cross-file move, near-full-file transform),
per-config tool restriction via disabled_tools, deterministic grading,
token/latency/error mining. Migrated under the benchmark admission
contract (admit_benchmark_run/finalize_run_manifest, launcher_audit
MIGRATED_LAUNCHERS); it measures the current working tree by design, so
real runs use the recorded --allow-dirty-seed escape.
Headline numbers (grok-4.5, 3 runs/config, every run solved every task;
full tables in the editbench README): edit_batch finishes the rename
task in ONE tool call vs ~6 for edit_text/write_file; on a weak model
(gemini-3.6-flash) its advantage grows to 1.5-2.5x cheaper/faster; a
free-choice agent picked the right tool per task shape in every run.
No version carriers touched (maintainer assigns the release version).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the v6.82 mobile swipe gestures, make drawer opening clear keyboard state, and keep deeply nested live cards usable in narrow consumers. Set fresh-install triad defaults to Luna, Gemini Flash, and Sonnet while preserving saved settings.
Co-authored-by: Ouroboros <ouroboros@local.mac>
scrub_submission_secrets.py walked `root.rglob("*")` filtered by `p.is_file()` and
wrote with `path.write_text()`. Both follow symlinks, so the tool had two failures,
each demonstrated against the pre-fix code.
A FILE symlink was swept THROUGH: the write landed on the link's target, outside
--root. A pack containing a link to the live settings.json therefore had its real
keys replaced with <REDACTED:...> by the tool whose job is to protect them. `cp -a`
preserves symlinks, so the procedural "run this on a COPY" rule did not help.
A DIRECTORY symlink was worse. rglob does not descend through one, so its contents
were never enumerated — and the verify pass then printed verify_leftovers=0 and
exited 0. The tool affirmatively certified a tree it had never read, for content
reachable under --root and about to be uploaded to a public leaderboard. Silent
non-coverage reported as cleanliness is exactly the class of false claim this
release exists to remove, and here the consequence is a live API key published.
Both are now a hard refusal before a single byte is written, reusing the fail-closed
discipline the --env-passthrough refusal already established: every offending link is
named with its target and its kind, nothing is modified, exit 2. Refusing beats
resolving-and-continuing, because under --root a symlink is either an accident or an
escape and only the operator can say which. With the links removed the tool sweeps
and verifies exactly as before, so this refuses an unsafe shape rather than removing
a capability.
Pre-fix, the new test fails with `assert 0 == 2` — the old tool reported success.
Both defects were invisible to every phase on its own base and appeared only once
the chain was assembled. Neither was caught by the full gate battery, because
neither was pinned by a test — which is the actual finding.
_CACHE_INCLUSIVE_PROMPT_TOKEN_PROVIDERS omitted "anthropic". P6 introduced the set
correctly, while the direct-Anthropic path still reported bare input_tokens. P3
then made that path report prompt_tokens = input + cache_read + cache_creation.
On the merged tree every cache-bearing direct-Anthropic send therefore failed the
membership test and returned early, so P6's token-density learning was vacuous on
the main and heavy slots — precisely the routes it exists to measure — and every
review pack was frozen at the cold-start density. The docstring asserted the
opposite of the tree's behaviour, in a release whose thesis is that artefacts must
not claim what did not happen. GigaChat stays excluded: its precached_prompt_tokens
semantics are still undocumented.
ARCHITECTURE.md stated the migrated-launcher count twice and contradicted itself —
"ELEVEN" from P2/P3 and "ten" from P5, surviving side by side in one 33 KB physical
line that git merged without a conflict marker. The registry holds fourteen. The
first claim is now explicitly historical, the second states the current count, and
a test pins any unqualified count to len(MIGRATED_LAUNCHERS) so a future migration
that forgets the prose fails instead of shipping a document that misdescribes its
own gate. Also repaired the README roll-off sentence, where 6.76.0 had been
appended after the closing clause instead of inside the parenthesised list.
Tests added in both directions for the density gate, which had no coverage at all.
Six reviewed phases land as one release.
Admission is the outer boundary: every migrated launcher records a manifest before
it can touch the filesystem, and finalizes a typed outcome on every path — success,
refusal, crash, and the real exit status. A structural audit enforces that boundary
across all fourteen launchers, together with confinement computed from the active
checkout and a single manifest publisher, judging by effect rather than by callee
name and failing closed on any write form it cannot resolve.
Harness exit codes are no longer trusted as run status: inspect returns zero for an
eval that raised and harbor returns zero for a job whose trials all errored, so the
launchers now read the harness's own artefact and keep "the harness failed", "it
scored nothing" and "it scored honest zeros" distinguishable.
The acceptance dialogue reconciles receipts through one typed identity that is an
equivalence by construction, so a passing check can no longer clear a red it never
addressed. Prompt caching is normalized at every send site and cached calls stop
under-reporting their input. The owner's context mode becomes explicit and
fail-closed, with one enforcement point for every writer of a disk-authored setting.
Deliberate limits are disclosed in each bench's METHODOLOGY.md rather than implied
by silence. Isolated benchmark egress and the multi-lane script generator are
deferred to a later release with restoration patches and carry-forward notes.
Every claim a benchmark run makes about itself is now either enforced before
money is spent or refused with a durable typed record.
Admission is the outer boundary, and that is a tested property, not a
convention: `admit_benchmark_run()` builds the manifest, WRITES it, and only
then lets the seed gate enforce, raising `BenchmarkAdmissionRefused` with the
refusal already on disk. A `finalize_run_manifest()` context manager records a
typed outcome, a typed refusal/error and the REAL exit status on every exit
path, including an escaping exception (an integer `SystemExit.code` is
preserved). All seven migrated launchers route through both seams; none calls
the builder directly; and an `ast` guard in the seam meta-test fails if any
denylisted operation — filesystem assertions, docker, subprocess, state dumps,
writers — precedes admission. The guard was verified to bite: against the
pre-fix tree it reports four violations, two of which no review round had
reached.
Refusals are one family. `BenchmarkAdmissionRefused`, `RuntimeAttestationRefused`
and `SeedShapeRefused` are all `RuntimeError` subclasses carrying a typed
reason, deliberately not `SystemExit` — a `SystemExit`-raising assertion had
made a refusal handler dead code, because `SystemExit` is not an `Exception`
subclass. A behavioural parity test computes what `raise SystemExit(main())`
hands the OS for each launcher and asserts it EQUALS the `exit_code` the
manifest records, so a recorded status can no longer diverge from reality.
`runtime_attestation()` records both facts about a live server — the HTTP
`runtime_version` from the frozen `/api/health` contract and the local
HEAD/VERSION of its checkout — and requires the contracted field: a bare
`version` key now refuses with the non-overridable `runtime_version_absent`, so
an unrelated HTTP server can no longer attest. `OBO_ALLOW_EVOLVED_VOLUME` waives
only `runtime_skew`, in the shell as well as in Python, and commit availability
is decided before skew so a missing commit cannot be mislabelled. Lineage is a
descent check (`merge-base --is-ancestor`), never equality, so an evolution run
legitimately moving HEAD forward is not corruption.
`CAMPAIGN_FATAL_PROVENANCE_REASONS` is a single shared authority consumed by
both SWE-Pro drivers, which now stop the schedule immediately — before any
volume archival — instead of one driver burning every task.
Grading gains a third state: `pass|fail|ungraded` with typed reasons beside the
UNCHANGED headline formula, plus `grade_summary.json`, and the probe
orchestrator reads that artifact instead of scraping log tokens and refuses to
publish a grade that a failed or stale grader produced. ProgramBench's ledger is
append-only at both the run root and the instance directory, skip rows included.
`common/manifests.py::write_json` is atomic and byte-identical, importing
`ouroboros.utils` lazily so the module stays stdlib-only for the container-side
harbor agent, and `openrouter_key_remaining()` reads the authoritative
`limit_remaining`.
Egress isolation was built in this phase and is NOT part of this release: it is
off by default per owner decision, it produced a finding in five separate review
rounds, and round 10's was a direct recurrence of round 5's, so it was extracted
and deferred with its diagnosis preserved. Solve containers run the same open
network as before v6.75.0. The measured evidence stays in METHODOLOGY: the
official SWE-bench Pro harness does not regulate the solve container's network
at all (`--block_network` applies to the EVAL container), and 31 of 287 attempted
instances are musl/Alpine, 84% of which previously produced a non-empty patch.
GAIA, CL-Bench, Terminal-Bench and OSWorld keep their pre-v6.75.0 behaviour here
and migrate in their own phases. No `ouroboros/` runtime behaviour changes.
Reviewed through eleven rounds of the production commit gate (advisory, triad
`claude-fable-5` + `gpt-5.6-sol` + `gemini-3.5-flash` at high effort, scope
`claude-fable-5`): 24 findings, 22 fixed and 2 refuted with the evidence recorded
in the changelog. Per-phase tags are deliberately not created — a single release
tag lands with the campaign synthesis, an owner-approved deviation from the
tag-per-commit norm.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Root cause (PB cmatsuoka__figlet smoke): the agent committed a compiling state, then broke the tree with one last uncommitted edit as the acceptance improvement loop hit its pass cap; the harness ships the LIVE tree, so the verified commit protected nothing — all salvage machinery guards the answer TEXT only. Prompt-only fix (P5) over existing channels (P7): the acceptance rails line marks the last admitted improvement pass (passes_done+1 >= cap, cap>0) as FINAL and, for workspace deliveries (canonical is_workspace_mode() with an attribute fallback), EVERY workspace improvement capsule carries the tree directive — a deadline or cost rail can end the loop between capsules (triad r1) — keeping the tree at a VERIFIED state (rebuild, verify, and commit if the task calls for a commit; revert unverified edits); the 10% deadline flush AND the cost wrap-up note gain one shared commit-NEUTRAL tree sentence (acting self_worktree subagents cannot commit; a moved HEAD fails patch capture closed), byte-identical for non-workspace tasks; the ProgramBench instruction now states the exporter's true submission contract (SOURCE tarball from the CURRENT tree state; .git, root binaries, .ouroboros/ and named build/cache noise excluded at any depth; run ./compile.sh one final time), replacing the false fresh-checkout framing. Tests pin the pacing/rails seam, the workspace gating on both axes, byte-identical non-workspace texts, the real tarball contract, and the instruction truth phrases. P9 carriers bumped to 6.74.4; changelog row added. Disclosed residual (this is a MITIGATION of the incident class, not a closure): a forced tool-less exit — deadline grace or budget stop crossed inside one long round with no pacing note or capsule in the terminal stretch — can still ship an unverified last edit; the structural verification-freshness seam is a filed follow-up pending an owner decision (triad r1-r3, codex full-access review concur).
They imported inspect_ai directly — an optional benchmark dependency absent on CI runners — and failed quick-test with ModuleNotFoundError. The tests now inject a fake inspect_ai.util.sandbox module via monkeypatch, keeping the success-path coverage on every environment. P9 carriers bumped to 6.74.2; changelog row added (oldest patch row rolled off). No runtime code changes.
The v6.74.0 tag CI failed on the deterministic ruff F-rule gate: an unused types.SimpleNamespace import in tests/test_devtools_benchmarks.py (added with the final GAIA staging tests). Import removed; P9 carriers bumped to 6.74.1; changelog patch row added (oldest patch row rolled off). No runtime code changes. Fix-forward per protocol — the published v6.74.0 tag is not re-tagged.
A: verdict-visible improvement capsule (verdict+tier+real blocker via one panel_reason reducer, open obligation ids, money/time/rounds/passes rails line, three real moves); reviewer-authored obligation identity (disposition_kind new|re_raise validated against the host catalog, fail-closed to new, per-pass dedup, reuse-immutable) with surviving rebuttals (previous_disposition/previous_reason/reopened_count); typed dialogue_status (continue_actionable|unreachable_here|stable_disagreement) reduced over ALL contract-valid actors with the panel quorum — termination is reviewer-authored or a real rail, never a host counter.
B: two cache-marked review-prompt segments (governance + task-stable contract), slot label off byte 0, breakpoint cap asserted on the final payload.
C: GAIA sandbox attachment staging (prompt-declared /shared_files channel, exact shared-root lookup, per-file provenance, typed per-sample infra error); CLB acceptance-claims operator patch (receipt-bound claims, knowledge nudge in the steer slot, bounded cost-finality wait); SWE-Pro cumulative shard budget (auto_run derives per_task x scheduled, run_pro seeds from cumulative spend); CLI/PB bounded task_cost_finalized waits for completed/degraded only.
D: light-mode shell guard resolves cwd before judging repo targets (resource-root label false-block fixed); post-task cost publish uses try_get_bridge.
E: generative surface-duty in commit/plan review checklists; all P9 carriers bumped.
Reviewed: 2 adversarial rounds + production triad+scope (PASSED) + codex final (GO); smoke 5x5 on all five benches verified the dialogue/staging/budget mechanics live.
TB2.1 leaderboard static analysis matches config.agents[].name against the
agent name trials report; bare --agent-import-path records name=null, which
can never match (terminal-bench-2-1#121) and is unfixable after upload.
- run_tb.py: generate agent_job_config.json (name = adapter class name() +
import_path + kwargs) and launch harbor with -c instead of bare agent
flags; OUROBOROS_EFFORT_TASK at launch is recorded as the declared
reasoning_effort of the submission key.
- harbor_installed_agent.py: accept the reasoning_effort kwarg and forward
it into the container as OUROBOROS_EFFORT_TASK (declared == effective).
- README/METHODOLOGY: replace the removed 'harbor leaderboard submit' CLI
flow with the verified GitHub-PR flow (terminal-bench-2-1 lb submit,
promotion, /judge, /apply), document the named-config requirement,
trajectory_path pitfalls and the TUS upload fix pointer.
- tests: assert the new -c job-config shape (name/import_path/kwargs/effort).
Generalizable runtime + harness improvements distilled from the 35-FAIL SWE-bench Pro
autopsy, the ProgramBench improvement spec, and the TB2.1 forensics — one minor release,
no benchmark overfit.
Runtime (A/C):
- Scratch is now idempotent/adoptable: an existing untracked in-cwd file can be declared
scratch (sha recorded at declaration, re-declaration idempotent); tracked files and
directories stay blocked. The undeclared-output guard is stat-verified (real filesystem
writes, not string-shape) and, for run_script, audits on EVERY exit path — a write-then-
fail body no longer smuggles an undeclared user_files deliverable.
- Cost axis: budget_profile.cost_hard_stop_pct (0..100; None→50% historical; 0 = NO in-task
cost stop, deadline/rounds/global-gate only; malformed → 50%, never a silent 0). Latched
cost milestones (50/25/10% + wrap-up) replace the old [INFO] nudge. CLI --task-metadata-json
merges into body.metadata with host-owned keys (delegation_role/source) unforgeable.
Bench harness (D/PB/TB):
- SWE-Pro e1v2: /app is the default active workspace (via user_files root), empty memory
default, budget metadata (until_deadline, cost_hard_stop_pct=0) threaded through; bench1
fixes (shard-safe auto_run, k=1, timeline) ported into the repo; the image-load lock and
utility-image use are cross-platform (platform_layer, preflight-pulled, fail-safe restore).
- Workspace acting-integration allowlist (+integrate/compare_subagent_patches) with a
fail-closed self_worktree-under-external-workspace integration guard.
- Protected-artifact policy round-2: structural (operation-identity) false-positive
exceptions for differential-testing harnesses — vcs_diff/stat classifier, spawn-argv=argv[0]
execute, interpreter script/module operand screening, writer-target segmentation — while
read/copy/hash/introspection of the black-box reference stay blocked (incl. alias-separated
reads); pinned by a new permanent policy test.
- ProgramBench instruction v2, until_deadline profile, acceptance claims. TB ffmpeg resolver
chain (platform_layer/imageio-ffmpeg/PATH) + agent-prefix install + cv2 fallback hint.
- 4 verify-guidance clauses in prompts/SYSTEM.md (probe the consumer interface, exercise every
provided input, robust-under-ambiguity, never read hidden graders).
Docs/versions: 6.56.0 across VERSION/pyproject/web/api_types/README/ARCHITECTURE; SWE-Pro +
TB METHODOLOGY (workspace/budget/memory/retry-gate disclosure), PB gate-20 operator method,
pricing sonnet-5, DEVELOPMENT scratch contract.
Reviewed via operator triad (openai/gpt-5.5 + google/gemini-3.5-flash + anthropic/claude-fable-5)
+ scope (anthropic/claude-fable-5) to convergence: scope clean; residual triad items are a
verified reviewer false-positive and the plan-accepted single-parameter budget-threading
tradeoff on a grandfathered core-loop function (advisory-bypass, documented).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The GAIA answer key is cached on the host (metadata.parquet Final answer column +
per-task .jsonld under ~/.cache/inspect_evals and the HF dataset cache), and every
solver runs its agent as a host subprocess with shell access — so the agent can read
the answers off disk (observed: a Codex row ran `find … gaia_dataset … jq
<sample>.jsonld`; the leakage audit caught it). This is a filesystem sibling of the
web-lookup cheat.
Wrap every solver subprocess (Ouroboros + codex/claude/hermes, symmetrically) in
`bwrap --dev-bind / /` with the answer-cache directories masked by empty tmpfs,
leaving the rest of the FS, the network (web search), and CLI config intact. The
inspect scorer runs in the main process outside the wrapper, so scoring is unaffected.
New bwrap_isolate.py (default-on; GAIA_BWRAP_ISOLATE=0 to disable; fails loudly if
bwrap is missing rather than running silently unprotected). Disclosed in METHODOLOGY.
Verified: answer cache invisible inside the wrap; codex/claude solve end-to-end;
the Ouroboros dedicated server binds loopback and runs inside the namespace.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-07-04 GAIA runs caught agents looking up the published answer key
(googling "GAIA benchmark ... answer", downloading metadata.jsonl from mirrors)
and running the weak strict_ddgs retriever instead of native web search — the
latter mislabeled as parity with the native-searching Codex/Claude Code targets.
- Anti-lookup prompt: SSOT GAIA_ANTI_LEAK_INSTRUCTION appended by all four
solvers, worded to avoid the benchmark name / "FINAL ANSWER" marker / the
leak-query regex so it neither primes the model nor self-flags in traces.
- Full trace capture for audit parity: Claude Code -> --output-format
stream-json --verbose (per-sample claude_code_trace.jsonl); Codex -> --json
(codex_trace.jsonl). Both were black boxes before.
- Observability: persist usage.web_search_sources on llm_usage rows in
events.jsonl so the audit can see native (server-side) web-search citations,
which never appear in tools.jsonl (behavior-neutral).
- Audit v2 (audit_leakage.py + new SSOT leak_targets.py): strip prompt
boilerplate before scanning traces (no self-flag); catch quoted "FINAL ANSWER"
queries and dash-named GitHub mirror paths; STRONG gold-verbatim only when the
gold answer came FROM a leak source (plain gold in a normal page is weak);
scan events.jsonl native citations by root_task_id; parse CC/codex JSONL
traces; fix WEB_TOOLS (+skill_exec/start_service, -nonexistent fetch_url/bash).
- Scoring: score_gaia.py --leakage-audit reports leakage_adjusted_accuracy
(STRONG-flagged samples counted incorrect) alongside the authoritative score.
- Docs: METHODOLOGY (quality_openrouter_web is the publishable default; strict_ddgs
is a handicap, not parity; pre-registered zeroing rule; per-harness coverage),
README_harness_compare (who-searches table), ARCHITECTURE (three web-access
mechanisms and their model/observability consequences).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Commit 3/3 of the bench post-mortem sprint (plan smooth-skipping-charm;
commits 1-2 = bf8bde7 v6.54.3, ef786bb v6.54.4). Minor bump per plan.
Shared bench-template scaffold defaults, disclosed in the benchmarks index
and per-bench METHODOLOGY files: OUROBOROS_MAX_WORKERS=4 (same-model
decomposition slots within one task, never best-of-N; TB 2->4, GAIA
--max-workers default 1->4 with explicit 1 as the strict-baseline ablation
and the quality-profile silent 1->5 bump removed, SWE-pro 5->4),
OUROBOROS_SAFETY_MODE=light (disposable jails; deterministic guards stay),
RUNTIME_MODE=pro for container benches (GAIA deliberately stays light: its
solver runs unisolated against a live repo), and claude_code_edit disabled
in every bench solve task (single-model harness measurement). TB raises
_DEADLINE_SAFETY_SEC 30->105 from measured finalization overhead, mirrors
the registry _WEB_TOOLS denylist (youtube_transcript drift fixed, pinned by
a sync test), and the README pins why --all-model keeps review single-model.
ProgramBench e2e runner ported from the colleague's tree, adapted to our
APIs: metadata.budget_profile onto the v6.54.4 contract (footer keys
dropped, reserve pct unit-fixed 0.15->15), solve-model preflight through
migrate_model_value (fail-fast on direct-route legacy ids), atomic
per-instance checkpoints with honored reattach (client poll timeout leaves
the executor alive; the next run skips seed/start and reattaches), explicit
payload-status terminal detection, denominator-preserving ledgers
(skipped_existing rows; skipped = successful for exit code), source-only
submissions (both root binaries excluded by name), idempotent workspace
normalization shared by the prepare-only flow.
New continual_learning/ launcher wraps the EXTERNAL CL-Bench runner
(nothing vendored; fails loudly with obtain instructions), strict-sequential
guard (parallel opt-in is exclusively-stateless-bridge only), standard-path
pointer rows keep the ledger denominator, fidelity gaps of the pinned
external adapter recorded and warned. OSWorld aligned to the official 2.0
protocol (pinned OSWorld-V2@c261cb57, 500-step default, show_result.py
layout, official traj rows), bridge-level final_answer capture with
VM-state-only prompting, preflight verifies the target server's effective
scaffold settings via /api/settings (env cannot configure a --url server)
and refuses the live desktop URL without an explicit flag.
LifelongAgentBench documented as blocked (its 100% run was a no-gold-oracle
artifact). Carriers synced to 6.55.0. Review: 6 triad+scope rounds to a
fully clean verdict (16 confirmed criticals fixed, each with a regression
test); gemini/opus clean in all rounds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
run_gaia.py gains --sample-id (comma-separated ids) to replay a chosen subset of tasks, mirrored into the Inspect argv. _write_manifest now records the EXACT selected ids via a shared _requested_task_ids() helper when --sample-id is set (else the limit-derived level list), so a sample-id rerun no longer writes a manifest claiming the first N level tasks — keeping the audit/denominator contract honest. Focused test covers the manifest selector and the argv lockstep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(improvement_backlog): stub semantic-dedup LLM in groom tests (P6)
_seed_many() seeds items via append_backlog_items(), whose C9.2 semantic-redirect
pre-pass calls semantic_dedup.find_semantic_duplicate_id() once per fingerprint-MISS
with candidates — a real light-model NETWORK call. The seeding runs BEFORE
_patch_groom_llm installs its mock, and that mock only covers chat_observed, not the
detector's own client path. With no API key the call retry-storms for minutes before
failing open to None, making test_groom_backlog_rejects_invented_items ~129s alone
(~40% of the whole suite) and non-deterministic.
Add a module-level autouse fixture that stubs find_semantic_duplicate_id to its own
fail-open default (None = no duplicate — exactly what the doomed call eventually
returns for the distinct seeded items), so the module is network-free and
deterministic. No production change; the dedup contract stays covered by
test_semantic_dedup_v6370.
Effect: test_groom_backlog_rejects_invented_items 129.12s -> 0.02s; whole file
~165s -> 2.37s, all 14 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(core): validate regex up front in _code_search so invalid-regex contract holds for both backends (P3)
_code_search ran the ripgrep path first and returned its formatted result before
ever reaching the Python-fallback re.compile guard. ripgrep accepts some malformed
patterns permissively — an unterminated '[' yields "no matches" instead of erroring
— so an invalid regex like "[invalid" silently returned no-match on the rg path while
only the fallback (rg absent/failed) emitted "⚠️ SEARCH_ERROR: invalid regex". This
made test_code_search_invalid_regex fail whenever ripgrep is present (i.e. always, in
CI and the pre-push preflight), so the suite exited non-zero on every candidate diff
regardless of the change under test.
Compile the regex once up front (regex queries only; literal queries need no check)
and return SEARCH_ERROR on re.error before dispatching to either backend, so both the
rg path and the Python fallback share the same invalid-regex contract.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: guard the conftest repo-root pollution sweep to the xdist controller (xdist-safety)
Under pytest-xdist, pytest_sessionfinish fires on the controller AND every worker against
the SHARED repo root, so the mock-pollution sweep (shutil.rmtree of leaked <MagicMock>
paths + session.exitstatus=1) had workers racing the same rmtree and each independently
failing the run — a non-deterministic, failed-shaped result. Guard the repo-root sweep +
exitstatus mutation behind `if not hasattr(session.config, "workerinput")` so it runs only
on the controller (the single authority); the per-process _PYTEST_DATA_DIR cleanup stays
outside the guard and runs on every process. Serial runs are unaffected (the guard is
always True without -n).
Adds tests/test_conftest_xdist_guard.py (sweep runs on the controller, skipped on a worker).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: parallelize the test suite with pytest-xdist (~9x faster), keep the gate serial
The full-suite CI jobs (quick-test, full-test) now run a PARALLEL pass plus a short SERIAL
pass. Empirically ~270s serial -> ~30s parallel (~9x). The per-commit preflight GATE stays
serial by design (a flaky parallel fail-closed gate manufactures non-deterministic
TESTS_FAILED indistinguishable from a real immune rejection).
- requirements: pytest-xdist + pytest-timeout (a hang-guard).
- pyproject: register the `serial` marker (addopts unchanged).
- ci.yml (quick-test, full-test): two-step. Parallel
`-m "not serial and <default lane exclusions>" -n auto --dist loadscope
--max-worker-restart=0 --timeout=300 --timeout-method=thread`, then serial
`-m "serial and <exclusions>"`. A command-line -m REPLACES the pyproject addopts markexpr,
so the default lane exclusions are repeated and ANDed with the serial split — the union
exactly reproduces the old default suite (4594 parallel + 122 serial = 4716, disjoint).
- conftest: a tryfirst pytest_collection_modifyitems hook marks the real-process/port files
(workspace_executor[+cleanup], process_custody, kill_process_tree_orphans, zombie_prevention,
worker_crash_retry, process_resource_leaks, restart_reconnect, preflight_runner,
services_tool_v2) `serial`; plus an autouse fixture isolating workspace_executor._SERVICES/
_FOREGROUND between tests (a latent ordering bug -n redistribution exposes).
Test-isolation fixes surfaced by running the suite under -n:
- test_task_constraint_tools: 3 bare `sys.modules["...claude_code"] = mock` (no restore ->
polluted the worker's sys.modules -> later SDK-dependent tests failed) -> monkeypatch.setitem.
- test_workspace_executor: poll until the spawned process's command-sha is readable before
registering (the PID-reuse safety check compared the sha recorded at registration vs
recomputed at kill; right after fork+exec the command line is unreadable -> shas diverge ->
kill silently skipped -> flaky), + 5->15s kill-confirmation deadlines.
docs/DEVELOPMENT.md + docs/CHECKLISTS.md: guidance so future tests are parallel-safe or
marked `serial`. Reviewed by adversarial subagents (ship); verified parallel 8/8 + serial 3/3
green, partition exact + disjoint, gate byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* tests+ci: parallel-safety follow-ups (services-global isolation, serial-lane guard, monkeypatch conversions)
Three non-blocking follow-ups from the parallelization review, hardening the
xdist split shipped earlier in this PR:
F1 (tests/conftest.py): extend the autouse _isolate_workspace_executor_globals
fixture to ALSO snapshot/clear/restore ouroboros.tools.services._SERVICES (the
legacy services registry, a separate module-global from workspace_executor's)
under its plain threading.Lock _LOCK. Closes the legacy-services-path
global-leak class generally; raw dict ops only under the lock (no re-entrant
deadlock on the plain Lock), registry-only (never reaps the live Popen
handles), each module lazy-imported under its own guard.
F2 (.github/workflows/ci.yml): add a "Guard non-empty serial marker lane" step
to marker-guards. `pytest --collect-only -m serial` exits 5 on an emptied
_SERIAL_TEST_FILES; set -euo pipefail + tee surfaces it, and a positive anchor
grep on test_workspace_executor.py is the working assertion (the existing
browser-guard's `! grep "no tests collected"` is a dead no-op under -q).
F3 (test_skill_loader / test_iteration2_fixes / test_marketplace_clawhub /
test_devtools_benchmarks): convert remaining bare os.environ / sys.modules
mutations (no save-restore) to auto-reverting monkeypatch.setenv/delenv/setitem,
so the parallel suite has no cross-test env/module leaks. tests/_shared.py's
intentional process-wide SDK mock left untouched.
Verified: parallel lane 4574 pass (-n auto --dist loadscope), serial lane 140
pass, the 4 converted files 159 pass. Two subagent reviews: no regressions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: prune 26 redundant/tautological/obsolete tests (immune coverage retained)
Three-pass audit (finder+skeptic -> 31 empirical verifiers with rename+mutation
experiments -> delete-and-run dry-run) of the 275-file / 4067-test suite found 26
tests whose value is fully backstopped by a stronger survivor or is near-zero:
- tautological / zero-production pins (assert hasattr/callable; inline-simulated
branches; closed-loop regex; TypedDict dict-literal; CPython-only checks);
- proven duplicates with strict-superset twins (test_consolidator, test_commit_gate,
test_cache_optimization, test_review_intent_split, TestGrepRegexHint, ...);
- registry-registration one-liners, all backstopped by test_smoke::test_tool_set_matches
(set-equality vs EXPECTED_TOOLS -- mutation-proven to FAIL on a lost registration);
- dead-feature tests (retired skill_migrations module) + obsolete version regressions
(v636 import now unconditional; README "(N tests)" convention abandoned).
Removes ~47 test functions across 20 files (incl. the whole test_shell_regex_hint.py,
mirrored 1:1 in test_shell_run_shell.py::TestGrepRegexHint, and the whole
TestGoalScopePrecedence class). Also drops 2 now-unused imports + 1 orphaned helper.
KEPT (not redundant): test_smoke::test_git_commit_with_tests_exists -- sole guard of the
post-commit gate seam (a rename experiment proved nothing else catches its loss).
Verified: ruff F clean; collection exit 0 (4703 collected, no emptied class); full
parallel + serial suites green; two subagent reviews confirm exact set + no broken refs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Andrew <andgri200@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The v6.46.0 GAIA adapter could not start its dedicated benchmark server while the
desktop app was running: (1) it bound the default Host Service port 8767, which the
live app holds, crashing startup ("port 8767 busy"); (2) settings_base.json ships
EMPTY provider-key placeholders, which apply_settings_to_env pops from os.environ,
erasing the real keys _sanitized_host_env preserved ("No supported provider configured").
Fix (devtools-only, no runtime-core change):
- _free_port(): auto-pick distinct free ports for BOTH the main server (GAIA_OUROBOROS_URL
/ OUROBOROS_SERVER_PORT) and the Host Service (OUROBOROS_HOST_SERVICE_PORT), so the bench
server coexists with the desktop app and parallel configs never collide on a port.
- _resolve_provider_keys(): inject the REAL keys for the configured models + the pinned web
backend (from os.environ, then data/settings.json) into the rendered settings so the
server detects the provider; keys land only in the isolated, gitignored run dir. 'auto'
web pulls all keys (cascade); a pinned backend only its own; OPENAI_BASE_URL is dropped
for an 'openai' web pin (official web_search needs an empty base_url).
Verified out-of-the-box: plain run_gaia.py started the dedicated server cleanly (free Host
Service port, Supervisor ready, key injected) alongside a running desktop app + two live
L1 runs. New focused test (test_gaia_render_injects_keys_and_free_host_service_port) + all
90 devtools tests green; ruff F clean. devtools-only follow-up to v6.46.0 (no version bump,
matching the devtools-commit precedent).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A GAIA validation audit surfaced harness artifacts that masked Ouroboros's real
capability. This lands the GENERALIZABLE fixes (helping all users, not fitting the
benchmark), a Max-mode generative context-window probe, and a reviewed GAIA adapter.
Runtime:
- Budget exhaustion TERMINALIZES undispatched PENDING tasks (`fail_tasks`,
reason_code=budget_exhausted + task_done event, canonical budget_drive_root,
cancel-requested -> CANCELLED) instead of a silent never-dispatched freeze.
- `search_code` gains a shared wall-clock deadline (rg walk + batch loop + Python
fallback, one budget) and a DISTINCT deadline-truncation signal so a timed-out
search never reads as an authoritative "no matches" (OUROBOROS_SEARCH_CODE_WALL_SEC).
- `web_search` backend pin (OUROBOROS_WEBSEARCH_BACKEND: auto|ddgs|openai|openrouter|
anthropic; openai/ddgs are TRUE pins, no cascade); `ddgs` is now a dependency with
429 backoff so a fixed-model run can force pure retrieval.
- `OUROBOROS_OR_PROVIDER` (resilience/repro presets) routes OpenRouter providers for
rate-limit resilience without losing prompt-cache warmth (gap-merged; never overrides
the require_parameters / reasoning-continuity pins), behind a supports_message_cache_control predicate.
Answer integrity: a tool-count-stamped latch recovers a produced FINAL ANSWER whose
marker a later (un-grounded) round drops, while new tool work invalidates a stale latch
(Q7 no-downgrade); acceptance review on the ADVISORY task surface stops over-failing a
grounded answer (surface-keyed: majority-not-single-FAIL, no solved->DEGRADED on empty
coach). The P3 commit/scope immune gate is untouched.
Security: `user_files` honors OUROBOROS_USER_FILES_ROOT (jailed scratch home; '~'/'~user'
and unnamed deliverables stay IN the jail); the authoritative observability blob is
redacted by default (env-only OUROBOROS_OBSERVABILITY_KEEP_RAW opt-in) with an allowlist
secret matcher that spares cognitive content (sha/commit/uuid/route_fp/model/answer).
Capability: a generative context-window probe (public LLMClient.probe_oversized_context
seam) empirically confirms a route's >=1M window from a FREE over-window reject — only on
the explicit Max toggle/save, never auto-confirming a possibly-paid 200, fail-closed,
bypasses a stale lazy cache; plus a cloud.ru route base_url fix.
Bench: the GAIA adapter (devtools) stages attachments robustly, injects the FINAL ANSWER
protocol, crash-isolates per-sample timeouts (+ server-side ceiling so a hung sample
cannot orphan the shared queue), and exposes per-config knobs (vision slot, review models,
web backend, budget, epochs) with cross-provider credential preservation.
Version carriers bumped 6.45.5 -> 6.46.0; MAX_TOTAL_FUNCTIONS 3520 -> 3524 (reviewed
single-purpose helpers); ARCHITECTURE/README updated. Reviewed to convergence (adversarial
subagents + codex gpt-5.5 + real triad+scope) with a green full suite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keep GAIA run roots valid when their own per-run settings/data env is active, and retry the transient supervisor startup window so the first sample does not flake before producing a structured answer.
WIP checkpoint (local, not pushed) — to be reviewed-merged into repo/ after the re-run.
Fixes two TB2.1 methodology bugs that depressed gpt-5.5's measured score:
1. Network: the adapter set allowed_resources={web:false,network:false} on
disable_agent_web, which blocked `git clone` even though TB2.1 tasks need it
(every task.toml allow_internet=true; verifier clones in-container). Replace
with a declarative `disabled_tools` tool-policy: withhold the agent's own
web_search/browse/browser/analyze_screenshot/vlm_query tools (anti-reward-hack,
== original _WEB_TOOLS set) while leaving shell egress (git/pip/curl) intact.
- task_contract.disabled_tools (additive frozen-ABI field) + normalize_disabled_tools
- registry hides disabled tools from schemas()/core_only/get_schema_by_name/
available_tools (+ dynamic extension/MCP) and blocks them at execute
- /api/tasks + TaskCreateRequest (+ api_types.js mirror) thread the field
- propagates to subagents via the parent-contract spread
- view_image intentionally kept (local image-to-model, outside _WEB_TOOLS)
2. Deadline: Harbor's per-task wall-clock cap was never plumbed into Ouroboros
(deadline_at=""), so pacing/graceful-finalize were dormant and the agent was
hard-killed mid-work with empty results.
- D1: adapter resolves the per-task [agent].timeout_sec from cached task.toml,
subtracts elapsed install/server time + 30s safety, passes timeout_sec ->
deadline_at so the agent paces and self-finalizes a partial before the kill
- D2: surface inherited deadline_at from task_contract into task_metadata so
spawned subagents also pace/finalize
- _capture_current_task_summary records captured_after_cancellation only on a
real interruption (not a terminal infra_failed exit), so the disclosure
ledger no longer misreads a terminal provider_unavailable as a cancellation
Plus the run_tb disclosure-ledger honest taxonomy (pass/provider_infra/cancelled/
genuine) and tests (test_disabled_tools_policy.py, test_run_tb_methodology regression,
adapter terminal-nonzero-exit test) + docs/ARCHITECTURE.md ABI rows.
Reviewed: codex (gpt-5.5) + 7 rounds triad+scope to convergence; only the
version-bump touchpoint remains, intentionally deferred to the eventual merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The in-container HEAVY lane reads os.environ["OUROBOROS_MODEL_HEAVY"] (config.py:275);
the legacy OUROBOROS_MODEL_CODE env var is a dead no-op after the v6.39 Phase-2 slot
rename (the rename-alias migrate_legacy_slot_keys covers only the settings-file path,
not env vars). Harmless for the single-model run already executed (HEAVY empty -> main
fallback -> gemini) but a silent bug for future ensemble runs where HEAVY != main, as
the neighbor's release plan anticipated. Forward + set OUROBOROS_MODEL_HEAVY instead.
Update two now-stale tests in tests/test_devtools_benchmarks.py that the merge breaks:
- assert OUROBOROS_MODEL_HEAVY (not _CODE) in the container env (slot rename above);
- the default harbor_command must OMIT --agent-setup-timeout-multiplier /
--environment-build-timeout-multiplier: Harbor static_validation rejects them
(static_validation.py _trial_timeout_override_fields), so the cherry-picked Fix-1
leaderboard-faithful default is correct; the old assertions + "methodology-allowed"
comment encoded the now-rejected behavior.
Also documents the post-run disclosure_ledger.json (schema tb_disclosure_ledger.v1) in
the devtools-benchmark paragraph of docs/ARCHITECTURE.md (deferred merge-time doc TODO).
No version bump: devtools dev-infra work; the bump rides the next release.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Naming (B): SSOT `ouroboros/project_naming.py` coins a short human project title via a
bounded LIGHT-model call (P5, no keyword gates; fail-soft to a heuristic; spend recorded
through chat_observed + update_budget_from_usage). A proactive card namer
(`supervisor/workers.py::_spawn_proactive_namer`) names a fresh main-chat card up front —
background thread → `suggested_name` on the result + a `task_named` WS broadcast → the live
card shows the title (UI-buffered if the card races ahead; persisted across reload via
history.py for progress AND task_summary rows). turn-into-project reuses that name with ZERO
extra call; the heuristic `task-…` is gone. ensure_project_scope/agent paths share the SSOT.
Ordering (A): the owner's request sorts to the TOP of the converted project thread —
`_owner_message_send_ts` matched in the originating chat at/before task creation (dup-text
safe), then queued_at / result ts / now. chat_id persisted on the direct-chat RUNNING result.
Self-DoS (#4): a per-(model,use_local)-route `threading.BoundedSemaphore`
(`ouroboros/model_concurrency.py`, `OUROBOROS_MODEL_MAX_CONCURRENCY` from the config SSOT,
default-on, fail-soft, deadline-bounded, cap in the cache key) caps CONCURRENT provider
calls within a process so a task's loop + in-process subagent threads + status pings can't
storm one rate limit. PER-PROCESS (like fallback_cooldown); cross-worker is future work.
Soft join (#7): cancel_task gains a recorded reason; new peek_task inspects a child WITHOUT
absorbing; new discard_child_result(reason) is the EXPLICIT (not prose-parsed) abandon —
both stamp a durable parent_decision the handoff reminder honors, lineage-gated to own
children (cancel fail-closed for constrained profiles). The prose acknowledgement gate is
REMOVED (P5): the reminder is suppressed only by structured decisions/absorption, and a
forced/deadline/provider finalization records orphaned (incl. completed-unread) children
instead of dropping them. peek/discard/cancel extracted to `ouroboros/tools/join_ledger.py`.
Deferrals 1-5 (C): read-only `subagent_projects`/`deliverables` ResourceRoots (never
write/shell, never to subagents); a freshly provisioned genesis root must be empty
(fail-loud); a genesis project emits a typed `deliverable_manifest` on the artifact axis; a
shared `_str_match_replace` gives the data-plane editor the repo editor's match feedback; a
data-plane shrink-guard blocks accidental overwrite/edit truncation (force bypass) across
_data_write/_write_file (single+batch)/_edit_text incl. skill_payload.
Workstreams (E): bench swe_bench_pro/osworld slot keys CODE/FALLBACK/USE_LOCAL_CODE →
HEAVY/FALLBACKS/USE_LOCAL_HEAVY; stale joi-lab catalog URLs → razzant in repo.
Surface: peek_task/discard_child_result tools, OUROBOROS_MODEL_MAX_CONCURRENCY,
subagent_projects/deliverables resource roots, TaskNamedOutbound WS contract. Reviewed:
triad+scope converged (scope pass + gemini/opus SAFE) + codex SAFE TO COMMIT.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dev-infra only (not runtime). Integrates the upstream terminal-bench-fast adapter:
- Apply the upstream harbor patch: terminal_bench/harbor_installed_agent.py env allowlist
(OUROBOROS_MODEL_DEEP_SELF_REVIEW + EFFORT_* keys; tree-sitter-optional requirements) and
run_harbor_smoke.py CLI (-k, agent/environment timeout multipliers, --ouroboros-light-model).
- Replace harness_bench_fast/ouroboros_cli_wrapper.py (54 -> 227 lines: workspace git-init,
per-task logs, model env, --start, retries, --result-json-out, --actor-id, pro+advisory),
rebased to the v6.39 Phase-2 slot rename (OUROBOROS_MODEL_HEAVY / _FALLBACKS, never the
legacy CODE / FALLBACK), and keep honoring the OUROBOROS_BIN env override for --ouroboros-bin.
- Add harness_bench_fast/run_harness_bench_fast.py (orchestrator) with the hardcoded
DEFAULT_BENCH_ROOT parameterized to env HARNESS_BENCH_ROOT or a repo-relative fallback (no
contributor home path).
- Update the coupling test to the new command-shape + Phase-2-slot contract (the old
build_command unit no longer exists).
Verified: py_compile, ruff --select F clean, test_devtools_benchmarks + test_headless_cli
green; reviewed by claudexor codex (gpt-5.5, the OUROBOROS_BIN-override regression it flagged
is fixed). Pre-existing legacy slot keys in other benchmark suites (swe_bench_pro, osworld)
remain functional via the runtime's heavy->main fallback + FALLBACK alias; renaming them is a
separate follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 Blocks 4+6 of the combined release plan (owner-approved single commit):
Block 4 — unix_computer_use (renamed from computer_use; Windows stays a
future separate skill per P7):
- Coordinate normalization: screenshots downscale to fit WXGA and persist
the exact image->input transform; input tools consume the last
screenshot's image space (raw=true bypass); honest approx flags for
multi-display unions (incl. deceptively clean sub-1.0 scales) and
TCC-denied logical-size failures.
- Wayland: session detection, grim capture, ydotool pointer (mask codes
0x40|btn / 0x80|btn), wtype typing; key/hold_key are honestly
unsupported on Wayland (ydotool key takes raw keycodes only — silent
no-op success is refused).
- New actions: left_click_drag, mouse_down/mouse_up, triple_click,
hold_key (macOS pure-modifier-only honesty), cursor_position, wait.
- X11 keys: f1-f16 aliases, super/meta modifiers, original-case
preservation for case-sensitive keysyms in key AND hold_key; '--'
option terminator in type_text.
- ax_tree: macOS set-of-marks of the frontmost window (numbered
role/title/center marks, INPUT-space coordinates) with honest
degradation; Linux degrades to window list.
- Native launcher-seed trust (named, hash-pinned, audited exception —
CHECKLISTS §Skills, plan-approved): the launcher stamps
review.json status=clean (repo_commit_gate/native_seed) at its three
payload-write moments; verdict hash-pinned AFTER seeding and
provenance-BOUND at load (no .seed-origin or unverifiable provenance
=> pending); hash exemption covers ONLY the top-level marker of
native-bucket payloads; one-shot legacy-hash migration cannot
whitewash edits; zero-grant skills (tool/subprocess surface only)
auto-enable only when no explicit owner choice exists (a disable
survives version resync); OUROBOROS_TRUST_NATIVE_SEEDED_SKILLS
default-on opt-out.
Block 6 — bench adapters (devtools):
- run_tb.py: methodology-allowed setup/build timeout multipliers (4x);
per-task multiplier stays 1.0.
- Single-model metric hardening: the container fallback is pinned to the
effective main model in every reachable configuration (kwarg ->
forwarded host model -> packaged default) so SETTINGS_DEFAULTS cannot
resurrect a different fallback model inside the settings-less
container; host fallback is never forwarded.
- max_workers default 2 (plan_task scouts); defensive AgentContext
deadline probe (no-op today, harbor-docs-verified).
Also: SYSTEM.md attempt-cap paragraph (plan-approved surface, v6.30
catch-up doc-sync). Upgrade note: legacy computer_use payloads remain as
reclassified non-launcher skills until manually deleted.
Gauntlet: adversarial pass 1 (GPT+Gemini, 5 blockers fixed) + pass 2
(Fable, SAFE TO COMMIT, 3 minors applied); triad+scope rounds 1-5 with
scope responded every round; round 5 clean (0 criticals).
New SWE-bench-Pro E1v2 harness: Docker-isolated per-task runs with optional
post-task self-evolution, an autonomous multi-task runner with last-good
snapshot/restore, and offline grading. Secrets reach the container only via a
name-only env opt-in (never argv); a refused opt-in / infra skip is surfaced
through the timeline so the runner stops or skips instead of counting a
non-executed task as legitimate; --cadence off disables evolution via the
documented contract; predictions carry the leaderboard model_name_or_path.
English-only operator strings. Removes the superseded evolve_pro path.
Add denominator-preserving benchmark manifests and result ledgers across the devtools adapters, tighten Terminal-Bench/Harbor and OSWorld fail-closed preflights, and harden workspace patch artifacts for unborn, invalid, and acting-subagent git baselines.\n\nVerification: focused benchmark/headless/acting suites, isolated full pytest, adversarial review rounds, and real Ouroboros triad/scope review artifacts under bench_runs/devtools_sync_20260607_010949/reviews.
The v6.20.0 tag CI failed only on full-test (windows-latest); ubuntu and macOS
were green. Two unrelated, test-only failure classes:
- Acting-subagents (this feature): the _make_child_patch test helper wrote
workspace.patch via write_text(), which translates "\n" -> "\r\n" on Windows,
so the file sha256 (read as bytes by integrate_subagent_patch) diverged from
the manifest digest and tripped INTEGRATE_PATCH_CORRUPT. The helper now writes
the exact hashed bytes (write_bytes), matching production (headless.py writes
the patch with open("wb")). Product code unchanged.
- Pre-existing (devtools/executor from d863197, surfaced by the first tag since
the untagged devtools-benchmarks release): test_official_command_builders used
a hardcoded POSIX path literal (now compares str(Path(...))); four
workspace-executor tests assume POSIX path spellings / process-group kill
semantics and are skipped on Windows with explicit reasons. macOS/Linux keep
full coverage.
Version 6.20.0 -> 6.20.1 across all carriers (VERSION, pyproject, package.json,
GATEWAY_CONTRACT_VERSION, ARCHITECTURE header, README badge + relabeled history
row). No runtime behavior change.
