Owner decisions Q6-A and Q3-A (2026-08-21).
Q6-A: the shared OUROBOROS_REVIEW_MAX_CYCLES cap (shipped default "2") also
bounds task-acceptance panels (passes = cycles - 1); past bench campaigns ran
3-26 paid acceptance panels per task, so the shipped default would silently
change bench methodology comparability. Pin the ceiling off in every bench
scaffold:
- add "OUROBOROS_REVIEW_MAX_CYCLES": "unlimited" (the canonical unlimited
sentinel per ouroboros/review_cycles.py) to all bench settings templates:
continual_learning, gaia, osworld, programbench settings_base.json and the
four swe_bench_pro e1v2 templates (settings_base, _run_settings.example,
settings_sonnet46_probe, profiles/light_subagents_gpt55);
- terminal_bench has no JSON template: pin the same value in the
harbor_installed_agent container env beside the other review knobs;
- forward the key through _ISO_SETTINGS_ALLOW_EXACT in common/server_runner.py
so a live-settings pin reaches isolated bench servers like the other
non-secret review-policy knobs.
Also align swe_bench_pro/e1v2/settings_base.json to
OUROBOROS_REVIEW_ENFORCEMENT=advisory: METHODOLOGY.md documents that E1v2
templates deliberately stay advisory (the in-bench commit_reviewed evolution
lane is structurally uncommittable under blocking with the no-version-bump
steer); "blocking" in settings_base was a disclosed discrepancy. Pure JSON
forbids an inline comment, so the rationale lives here and in METHODOLOGY.md.
Q3-A: document in docs/DEVELOPMENT.md the deliberate review-contract
fingerprint scope — roster, routes, enforcement, resolved efforts, prompt
constants; governance-document contents (BIBLE/CHECKLISTS/ARCHITECTURE) stay
outside it, so editing them neither lapses recorded verdicts nor frees
replays; the accepted trade-off is that an old verdict can replay under
amended governance text.
No scoring/grading/task code touched; version carriers unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ouroboros advised a desktop-app owner to reload 'the browser tab' because the
runtime had no fact about the client UI: the launcher never exported the
retired OUROBOROS_DESKTOP_MODE flag, and messages carried no sending-surface
provenance at all.
Two additive facts, no device taxonomy (the model classifies raw observables):
- launcher.py exports OUROBOROS_PRESENTATION (desktop_window|browser_fallback;
absent=web) — the process posture, rendered as runtime_env.presentation.
- The SPA measures raw observables AT SEND TIME (pywebview bridge, ua,
viewport, matchMedia booleans, captured_at) and attaches client_surface to
each chat frame; the gateway normalizes it through the closed-key bounded
ouroboros/client_surface.py SSOT, stamps received_at, carries it in
task_metadata, persists it as an optional chat.jsonl column, and renders it
as the owner_client context fact. Non-web ingress gets a host-stamped
{channel: <source>} fallback; promotion/steering/mailbox carry it, and the
loop notes a mid-task surface change only when the surface IDENTITY differs
(viewport resize is not a device change), with a neutral note for the first
observed fact. SYSTEM.md documents the semantics and the pywebview product
facts (no Cmd+R, SHA auto-reload).
Adversarial waves 1-2 are folded in: Infinity viewport crash at ws ingress,
strict booleans, no-identity facts never mint change notes, provenance-honest
prompt wording, behavioral producer tests, send-site and received_at pins.
client_surface helpers live in their own module (message_bus/loop/chat.js stay
inside their ratchet sizes).
The pre-implementation gate reviewed an unbounded prose plan with a ~100k-token
governance pack, planning-scout subagents and an Atlas, and every rewritten plan
minted a fresh fingerprint that bought a whole new paid wave from cold reviewers.
Convergence was structurally impossible: the reviewer was asked to author a
competing plan every wave, REVISE_PLAN could not be dispositioned, and nothing
capped the cycles.
plan_task now reviews an INTENTION — the same organ whether the work is code,
research, a deliverable or an action in the world:
- a typed domain-neutral SPEC (goal, in_scope, non_goals, acceptance_claims,
invariants, decisions with rejected alternatives, deferred, affected_resources,
evidence) with host-minted ids that are the only valid `breaks` targets;
- ONE structural fact tiers the governance pack: `constitutional` iff a declared
target resolves under the system repo (never prose, never a plan-kind taxonomy);
- agent-declared evidence, bounded, with EVERY absence named, the runtime data
plane denied outright, and the exploration log redacted through the same SSOT
task acceptance uses;
- typed findings (blocking with a `breaks` id | note | need_evidence) with the
HOST computing the aggregate through adaptive_quorum — no reviewer emits GREEN
as authority, none writes a competing plan;
- ONE owner setting OUROBOROS_REVIEW_MAX_CYCLES (default 2, unlimited available)
bounds paid cycles for plan review, task acceptance (passes = cycles - 1) and
the commit gate's identical-diff attempt cap; an identical envelope replays for
free and a DEGRADED wave costs nothing;
- under blocking, an open plan holds implementation and a spent cap escalates
with a typed review_cycles_exhausted reason and an honest blocked terminal;
under advisory the agent may proceed with the wave open and a loud disclosure.
Deleted: planning scouts, the plan Atlas, plan_class, context_level, the
governance mega-pack, the generative reviewer stance, the hidden 32-wave limit
and the api_chat-only pin. Net effect on the tree is negative.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
M1: owner_stop_deadline_ts now returns no deadline when grace_sec<=0 —
a zero finalization grace means the graceful-stop feature is OFF
everywhere (immediate custody path), pre-drain included, matching the
running_owner_stop_tasks feature-off semantics instead of silently
granting a request+600s window.
M2: the owner-stop rail checks the durable effective deadline at the
consume boundary (_owner_stop_window_elapsed); an already-expired
finalize control no longer starts a paid summary turn — the honest
fallback rides the existing typed rail and custody settles the episode.
M3: the worker's drain-stamp write (_mark_owner_stop_control_drained)
retries a failed durable stamp once, confirms an already-present stamp,
and on definitive failure emits a typed owner_stop_stamp_failed forensic
event while assuming NO extended budget: both the sweep and the worker's
own summary gate keep budgeting from the conservative request+outer-cap
anchor on the unstamped intent.
M4: translated the remaining new Russian comments/docstrings in the
b5e21697..d18d7ebf range to English (owner UI wording quoted via the
frozen labels: "Wrap up", "Hurry up", "Stop now", "Stopped with summary").
Focused tests added for all three majors; loop.py growth was offset by
comment-wording compaction in the same file and the shrink-only size
ratchet pin moved DOWN (321071 -> 321064).
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Graceful owner stop (Q1): POST /api/tasks/{id}/cancel accepts
stop_policy=finalize_then_cancel, answers 202 with the durable intent open,
holds the task through a bounded owner-stop finalization episode
(supervisor/owner_stop.py), and monotonically hardens to immediate on
escalation. The final turn runs on its own owner_requested_finalization
rail (never the deadline's false reason), children settle first and feed a
bounded projection into the root's last turn.
Owner hurry (HQ1): POST /api/tasks/{id}/hurry delivers a typed task-local
kind=hurry owner-mailbox control (ouroboros/owner_hurry.py +
gateway/task_hurry.py) - no chat message ever. The armed latch skips the
next otherwise-eligible acceptance panel with a typed reason, zeroes
remaining improvement passes via one effective budget profile, and makes
force-plan projection task-locally advisory; the effect dies with the
attempt (shared retry_reset on reaper timeout and crash requeue), and the
durable owner_hurry projection is written only through update_json_locked.
UI (Q2): one shared three-action dropdown (Podvesti itog / Potoropit /
Ostanovit nemedlenno) on Chat live cards and the Activity tab
(web/modules/task_control_menu.js); pending cancel collapses the menu to
the hard escalation; hurry acknowledges via local toast only. log_events
hides the owner_hurry event family from chat (visible=false).
Provenance (Q5): cancel receipts deliver as system-role cancel_receipt
messages with stable cancel:<tid>:<rid> delivery ids; the full-copy
receipt persists in the durable task result; salvage text demotes
delivery-control JSON to forensic evidence.
Contracts are mirrored additively in gateway/contracts.py and
web/modules/api_types.js (endpoint index + both language mirrors); no
version carrier changes. Byte pins honored by genuine extraction:
acceptance rails line to task_pacing, hurry skip seam to owner_hurry,
steering handlers to supervisor/steering.py; size ratchet regenerated
(loop.py and chat.js debts paid down, supervisor/events.py left the byte
debt map).
Proofs: tests/test_owner_hurry_s3.py + tests/test_owner_stop_s3.py
(19.7.4 no-chat/durability matrix against production seams, including the
crash-requeue retry reset), tests/test_s3_task_control_browser.py (new
marker-gated Chat+Activity dropdown parity and no-chat browser proofs),
web/tests/task_control_menu.test.js.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Replace system-authored Low with an inert false provenance tombstone, preserve owner/P3 intent across migration and benchmark forwarding, and remove the blanket sub-1M settings and switch gates.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Phase C of the poltergeist delegation sprint, squash-landed onto the v6.99.0
nanny release. A mutating delegated run never edits the shared tree: the host
provisions a private execution snapshot of the authority target (baseline built
with the sensitive-veto decided before anything is hashed, pinned by a
refs/ouroboros/delegated/ ref), records the durable binding {execution_root,
baseline_sha, target_root, authority_source} on the custody rows BEFORE the
POST, and replays it byte-identically on explicit retry — pending-invocation
recovery carries the full binding so the startup GC (settled && patch_disposed)
never deletes the snapshot holding the child's only work. Terminal
reconciliation captures a settled run's diff through one drive-rooted capture
core, eagerly only at PROVEN terminality; an absent or unreadable close
captures nothing, capture-at-disposition is the retry point (C1-R2), a capture
failing there is the typed INTEGRATE_DELEGATED_CAPTURE_FAILED refusal for both
decisions, and patch_captured means a usable artifact exists (C1-R3 — a failed
manifest never mints the row and reject re-checks before releasing the
snapshot). Nothing lands without the explicit integrate_delegated_patch
apply/reject flow (proven baseline drift, NUL-safe touched paths, cleanup
following the durable disposition row, the protected-path gate scoped to the
Ouroboros body), and open obligations surface via undisposed_patches. Beside
it: SSOT cost projection (accounted_upper_bound_usd beside deprecated
cost_usd, $0-fabrication fixes, honest web cost presentation),
delegated_runs_failed on the evidence receipt, notification chat routing,
byte-accurate argv/env budgeting with --prompt-file transport, and hash-bound
skill repair.
Merge resolution UNIONS phase B's delegate module split (delegate_shared stays
the one refusal/emit/ownership author; delegate_integration imports it),
delegate_answer beside integrate_delegated_patch on every child surface, the
instructions-bearing idempotency digest, and BR2's additive terminal_detail
carry in cancel_and_verify with C's absent-branch no-capture semantics. Size
gates cleared by extraction, no grandfathering: delegate_evidence.py
(task_execution_evidence out of delegate_custody) and synthesis_cost_text.py
(synthesis cost renderers out of agent_task_pipeline), both re-exported
same-object; _delegate_start hands its started payload to one extracted
author.
The formal six-lane exact-SHA gate plus two verified fix rounds (CR1: the
startup GC fails closed over an unreadable custody log, the capture artifact
is readable across a split-drive boundary through a narrow rebind, a durable
apply intent precedes any tree mutation so a crash replay can never record a
false rejection, and delegate_start tells the snapshot truth; CR2: the
AMBIGUOUS apply state gained an explicit owner-acknowledged exit through the
normal disposition guards, a lost prune-skip row is escalated loudly, and a
failed verdict write can no longer strand a pending intent) are squashed into
this landing.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Squashed landing of sprint/slime-b-lifecycle (e3184c6f + 9df9368d + 2912e634)
onto the phase A mainline (cccc758f, v6.94.0).
Forced delivery-control resolution is pure and no-retry: under the armed
latch, any parsed object carrying the delivery_control key (unknown verbs
included) and any JSON-looking non-parse degrade to the retained candidate
with the typed delivery_control_degraded reason — protocol JSON can no
longer ship as the owner's answer; unarmed JSON passes through untouched.
Both resolvers carry the hardened protocol-intent rule.
Batch child disposition: tree_note(kind='decision') payloads accept a
children array expanded into the same per-child authoritative ledger rows
(exact child_result_sha256 per entry, per-entry rejection); the absorption
reminder and tool schema advertise the batch form.
Acceptance review runs on the forced children_unabsorbed rail (owner Q2A)
through the ordinary panel entry point, with the undispositioned children in
the evidence packet (explicit omission marker past 20); a requested
improvement pass the rail cannot grant terminalizes finalized_unaccepted
with the typed revision_unavailable_on_forced_rail reason.
Provider-death honesty: a provider-killed task terminalizes
failed/provider_unavailable (left BEST_EFFORT_REASON_CODES; salvage text
still rides the result body), and the owner gets an immediate single-shot
"NOT completed" notification with no false resume promise. The
reaper-delivered notification suppression bug is fixed and pinned by
regression test: registration happens only after a successful send, after
task-done cleanup, below the ephemeral-turn return.
A child's settled result stamps the parent's own progress (including
reaper-delivered terminals via the final_task_result fallback), so a
coordinator waiting on children is not idle-killed when its last child
delivers.
The forced-path nanny note closes the 6.94.0 disclosed residual: grounded in
durable custody evidence from the canonical custody root via the one
task_execution_evidence reducer shared with phase A's ordinary nudge
(delegated_runs_succeeded / failure states / evidence_read_failed).
Merge resolution: kept phase A's richer task_execution_evidence reducer
(SUCCEEDED_STATES set, delegated_run_failure_states, evidence_read_failed)
as the single reducer both nanny paths read; phase B's simpler succeeded
count was subsumed by the same delegated_runs_succeeded key.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
rotate_chat_log_if_needed becomes a thin wrapper over the generalized
rotate_jsonl_log_if_needed(drive_root, name, archive_prefix, max_bytes), which
keeps the atomic os.replace under the append sidecar lock, suppresses itself
in .ouroboros_isolated_benchmark data roots (bench harnesses read trial-local
single files), and avoids same-second archive-name collisions with a suffix
that still sorts chronologically. progress.jsonl now rotates on the same
supervisor tick as chat. Log archives are durable history: no GC/retention
sweep touches archive/ (verified: retention.py governs worktrees, task drives
and service logs only).
The TB ATIF trajectory builder reads the full archive chain (oldest-first +
live) for progress and chat, since not every harness writes the suppression
sentinel.
Guard review of the editing tools found the new pair weaker than the tools
they sit beside — not in what they can do, but in what refuses them.
Root cause: a guard judged a different spelling of the path than the write
used. `ctx.repo_path` runs `normalize_root_relative`, so an absolute path
inside the root and a redundant root-basename prefix land on the same file a
bare relative path does; `_resolve_edit_target` checked the RAW spelling, for
which `repo/BIBLE.md` is not a member of the protected-path table while
`BIBLE.md` is. In runtime_mode=advanced, `apply_patch`/`edit_batch` therefore
rewrote BIBLE.md and ouroboros/safety.py where edit_text returns
CORE_PROTECTION_BLOCKED. Reproduced against the real dispatcher, both
spellings, on the default root.
`edit_text`/`write_file` are immune only because the dispatcher canonicalizes
their `path` ARG once (`_PATH_NORMALIZED_TOOLS`). The new tools carry paths
inside the payload, so the module comment claiming their handlers "normalize
each target themselves via the shared edit_text guard chain" described a
normalization the shared chain never had.
One canonicalization contract fixes the whole class:
- `tool_access.canonical_repo_relative_path` is the single normalization both
ends call: the handler before its own protected checks, and the dispatch
gates through `_payload_write_paths`, which reads apply_patch's targets back
out of the REAL parser (`patch_target_paths`) so the gate cannot drift.
- `_resolve_edit_target` RETURNS that canonical rel, because it is the file's
identity. Keying the plan by the raw spelling meant two spellings of one file
in a single call produced two buffers and two writes, and the last silently
discarded the first while the result reported both as applied — reproduced as
real data loss on edit_batch and apply_patch.
- `_ROOT_ARG_REPO_WRITE_TOOLS` replaces the hardcoded write_file/edit_text pair
at the three fences that missed the new tools: the acting-no-workspace fence
(an acting child with no isolated workspace reached the LIVE repo), the
protected-write gate (which also carries the pro + protected_paths_grant
condition), and the acting root-enum narrowing.
Parity with the other repo writers, item by item:
- The assisted managed-update resolver keeps its protected-path exemption here
too; without it these tools were the one lane that could not finish a
conflict resolution.
- A pro-mode protected edit appends `core_patch_notice`, as git._repo_write and
_str_replace_editor do. The mode ALLOWS the write; the notice is what keeps
it visible.
- Validation refusals get the typed `edit_ops_blocked` status: a counted or
context miss is the designed path and is user-correctable exactly like
edit_text's "old_str not found", so it is a policy denial, not the false
tool_failure headline v6.57.0 removed for the other write tools. A partial
write is NOT that: it carries its own `EDIT_OPS_PARTIAL_WRITE_FAILED` prefix
and stays a real execution failure.
- A write failure after some files were written invalidates the advisory
snapshot for them and says PARTIALLY APPLIED, so commit_reviewed cannot accept
them against a pre-review taken before they existed.
Honest contracts instead of implied ones:
- Atomicity is over VALIDATION. Every schema, prompt and doc that implied the
writes roll back now says so, and the residual (a mid-write I/O fault can
leave earlier files applied) is disclosed rather than papered over.
- The fuzzy trailing-whitespace fallback states what it actually did: the
replaced lines, context included, now carry the patch's trailing whitespace.
- `_unified_diff` reports a final-newline change instead of "(no textual
changes)" — the rail exists to let the agent verify an overwrite, so the one
answer it must never give is "nothing changed" for changed bytes.
- `_syntax_check` names the format it actually checked (a NUL byte in a .py file
reported "not valid JSON").
- `edit_batch` gets apply_patch's 80k result limit: an aborted batch reports
every failed edit so one retry can fix them all.
Newline handling is deliberately NOT changed: the repo-write lane reads with
universal newlines and writes \n, so edit_text and write_file flatten CRLF the
same way today. Documented in the module rather than diverging one tool from
its lane.
editbench: generated `fixtures_v2/` is gitignored (running the benchmark dirtied
the seed gate), missing fixture trees fail fast before the isolated server and
the paid jobs rather than inside copytree, the run root is DERIVED instead of
mkdtemp'd so a seed-gate refusal leaves no footprint ahead of admission, and the
README and launcher docstring stop offering `edit_sketch_only` and a `default`
config they no longer have.
Tests exercise the REAL guard chain (a registry over a throwaway repo, no
patched resolver): protected paths refused in all three spellings for all three
tools, the acting-no-workspace fence for all three write tools, one file under
two spellings staying one target, the pro-mode notice, the resolver exemption,
parser-derived patch targets, partial-write disclosure and its status, and the
diff rail. DEVELOPMENT's New Tool rule now names the guard surfaces, since every
visibility list was already green while these fences were missing.
Co-Authored-By: Andrei Kaznacheev <a.kaznacheev@sdgroup.ai>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Two new repo-lane editing primitives beyond exact-match edit_text and
full-file write_file, both validated on the included editbench benchmark
before adoption:
- edit_batch: atomic batch of COUNTED exact replacements. Each edit
declares the occurrence count it expects and replaces all of them; any
mismatch aborts the whole batch with per-edit diagnostics. The safe
replace-all: a miscount is an atomic refusal, not a silent corruption.
- apply_patch: context-anchored multi-file patch (V4A-style, no line
numbers; optional @@ anchors; trailing-whitespace fuzzy fallback).
Atomic across all files/hunks with per-hunk diagnostics; parser
tolerates decorative trailing *** on headers (observed model quirk).
write_file repo lane gains two verification rails inherited from an
edit_sketch fast-apply tool that was implemented, benchmarked, and
REJECTED (data in devtools/benchmarks/editbench/README.md): a pre-write
syntax guard for .py/.json (force bypasses with an explicit
SYNTAX_GUARD_BYPASSED disclosure — no silent bypass, P3) and the unified
diff vs the previous version appended to overwrite results (capped with
an explicit truncation count, P1).
Registration follows the New Tool rule across every canonical surface:
safety.py TOOL_POLICY, tool_capabilities (CORE/ACTING envelopes + result
limits; local-readonly and heal lanes deliberately excluded), registry
(_FROZEN_TOOL_MODULES/_WORKSPACE_ALLOWED_TOOLS/_REPO_MUTATION_TOOLS),
outcomes (_ROOT_WRITE_TOOLS so patch/batch-only turns stay acceptance-
review-eligible; _RECOVERY_TOOL_NAMES), smoke EXPECTED_TOOLS, and the
SYSTEM.md/ARCHITECTURE.md/DEVELOPMENT.md/CHECKLISTS.md documentation in
the same commit (P6). The edit_text/write_file descriptions steer tool
selection at the schema source (per DEVELOPMENT's anti-pattern guidance,
not SYSTEM.md prose accretion).
devtools/benchmarks/editbench/ is the evidence and the regression rig:
5 editing tasks (trap-laden rename, surgical edit in a 1691-line module,
whole-function replacement, cross-file move, near-full-file transform),
per-config tool restriction via disabled_tools, deterministic grading,
token/latency/error mining. Migrated under the benchmark admission
contract (admit_benchmark_run/finalize_run_manifest, launcher_audit
MIGRATED_LAUNCHERS); it measures the current working tree by design, so
real runs use the recorded --allow-dirty-seed escape.
Headline numbers (grok-4.5, 3 runs/config, every run solved every task;
full tables in the editbench README): edit_batch finishes the rename
task in ONE tool call vs ~6 for edit_text/write_file; on a weak model
(gemini-3.6-flash) its advantage grows to 1.5-2.5x cheaper/faster; a
free-choice agent picked the right tool per task shape in every run.
No version carriers touched (maintainer assigns the release version).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Brings the public line's 23 commits — MiniMax direct provider, crash-safe
managed updates, the two-pass hermetic commit gate, and the Windows/release
CI fixes — into the Claudexor-integration line (v6.89.0).
Union resolutions of note (15 conflicted files):
- Version carriers stay 6.89.0 (this line IS the next feature release on top
of 6.88.0); the README Version History carries both lines' rows.
- config.py keeps the SSOT refactors (_clamped_number_setting,
settings_env_keys()) — the MiniMax keys ride automatically via
SETTINGS_DEFAULTS.
- gateway/control.py restart callback carries BOTH fixes: owner=True (the
runtime-mode re-read on owner restarts) and the bool return their restart
serialization reads.
- supervisor/workers.py terminal emission: their terminal_task_metadata
projection (a superset of the evolution_transaction carry) + this line's
cost-emission discipline (`emitted`, unavailable projections never publish
None placeholders).
- scope_review keeps the reviewer-window SSOT (the per-model window helpers
their side still carried are deleted); MiniMax window probing moves into
reviewer_window.reviewer_route/resolve (region-derived base_url +
authenticated catalog probe). The delegated session branch coexists with
their _ScopePromptContext/represent_binary call shape.
- shell.py keeps the claude_code_edit retirement (D10): their copies of the
retired helpers are deleted, their now-unused resolver import dropped.
- platform_layer keeps OUROBOROS_BUNDLE_DIR bundled-resource bases and gains
their ripgrep-pin dataclass; updates.js takes their redesigned
preflight/plan/typed-apply flow whole (it subsumes this line's
always-merge-aware intent); settings.js keeps the D10/6.1 picker removals
and gains the MiniMax region field.
Verified: import smoke, ruff F clean, 11 focused pytest files green
(including their test_update_apply_routing), web tests 136/136.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Triad/scope review of the reworked PR surfaced four issues, fixed here:
- pricing.infer_api_key_type: un-prefixed "minimax/..." ids now classify
as OpenRouter. Unlike cloudru/gigachat, minimax IS a real OpenRouter
vendor namespace and slash-form ids stay router-style by design; the
removed branch made safety.py demand MINIMAX_API_KEY on OpenRouter
installs with a minimax/ light model, silently skipping the LLM safety
check (fail-open). Regression tests added for both spellings.
- tests/test_docs_sync.py pinned the pre-MiniMax direct-provider
fallback sentence and failed deterministically after the ARCHITECTURE
sync; the pin now carries the MiniMax-inclusive wording.
- Docs enumerations that decide review behavior now include MiniMax:
DEVELOPMENT deep-slot EXCEPTION (512K guaranteed floor rationale),
CHECKLISTS item 2h provider-independence list, ARCHITECTURE
_REVIEW_ROUTE_BASE_URL_KEYS and unknown-pricing route lists.
- Benchmark/devtools secret and credential enumerations gained
MINIMAX_API_KEY (+MINIMAX_REGION where the pattern carries routing
keys): common/secrets, TB submission scrubber, isolated server_runner
env allowlist, harbor agent secret keys, GAIA credential resolver —
so a MiniMax key is masked/scrubbed like every other provider key and
minimax:: models resolve their own credentials in bench lanes.
Co-authored-by: Ouroboros <311266734+ouroboros-agent@users.noreply.github.com>
UI/settings + D10: reviewer-slot SSOT (6.1) with the reviewer_slots.js editor, the
D22 'runs as' projection API, D30 owned-claudexord (claudexor_daemon.py + harness
accounts UI + onboarding card), Q2-в credential profile pin on DelegationRoute, and
the D10 migration completed — the claude_code edit path (run_edit/_run_edit_async/
make_path_guard/EDIT_TOOLS + the shell claude_code_edit tool + ~35 tests) is retired
with the registry disabled_tools compat shim kept; the readonly path keeps p7a's
full hardening, and the two p7a fence tests were re-added with only their retired
write-fence halves stripped (the read-fence coverage is live). H4 both halves
verified composed (_candidate_scope_models feeds confirms_at_least require_fresh=True);
H5 holds (env.setdefault). Fusions: agent.py UI-chip directive re-homed onto the ONE
stamped record (_record_executor_facts reads the task); advisory SessionInvocation
carries the slot's session_route; ARCH/DEVELOPMENT gateway lines fused (D10 state +
p7a hardening + D30 daemon); control.py doc line kept D2/D3-correct against p6's
stale pre-p2 copy. Fixed real guard findings: restored the orphaned
_claude_options_has_explicit_param probe, added the annotation pathlib import, and
REVERTED p6's row-1 route re-read in the scope fanout (the p5x carried-route fix
wins; caught by test_mixed_scope_fanout...). Ported 8 more tests off retired
helpers/wording; test_settings_effort's carrier test (red on p6's own head) moved
to the reviewer_slots.js carrier. Guard PASS; fence_probe PASS; focused py suites
green; web 84/84.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first cut passed the custom provider as five -c flags on every codex exec. They
share argv with the prompt, and the benchmark's longest questions overflowed the OS
limit, killing a whole task with an unrecoverable OSError. Patch and README now record
the config.toml form that the ablation actually runs.
Records the two adapter changes the luna ablation runs on, so a fresh checkout can
reproduce it: the bridge's format-repair branch no longer raises NameError on the path
it exists for, and the Codex adapter can be pointed at a custom provider so a
harness-vs-harness comparison does not silently become a provider comparison.
Patch file only — no runtime code changes. The adapter itself lives in the benchmark
checkout, per the convention this directory documents.
Two v6.86.0 losses traced to one paragraph of the OSWorld working prompt.
WHERE. The contract asked only where a result must PERSIST, so a task whose
grader reads the LIVE window (is_vlc_fullscreen compares window size to screen)
was answered by ticking the preference and never entering fullscreen. WHERE now
has two slots — live and persisted — each filled or explicitly marked not
applicable. The not-applicable branch is load-bearing: 28 scoring tasks are read
only through tab lists or the active page, is_expected_tabs compares list
LENGTHS, and an extra tab opened "to check the stored slot" would zero them. So
the clause forbids inventing an action, tab, window or dialog to fill a slot the
task does not have.
UNCHANGED. A task asking for a bullet on an existing paragraph was answered by
typing a new line, because the contract had recorded that paragraph as UNCHANGED.
UNCHANGED now covers only content the task does not mention, with a narrow
exception: new content is created only when the task asks for something that does
not exist yet, while a MARKER or PROPERTY the task names — a bullet, a style, a
colour, an alignment — is applied to the content already there. Typing a fresh
line to carry the marker leaves the named content unmarked.
Three further clauses were written and dropped after adversarial review showed
each costs more than it wins, all verified against winning traces:
- prefer a slide master over per-instance edits: 358aa0a7 scores 1.0 and its
trace states master edits would not work there, because the shapes carry direct
character formatting that overrides styles;
- exhaust a named resource before any substitute: 36037439 scores 1.0 precisely
by detouring to another search engine, and the preamble already says to retry
and try another route on an anti-bot block;
- require a configuration CLI to match the GUI's breadth: generalised from the
single evaluator in the suite that takes a majority over mime types, and it
contradicts the preamble's own rule against shaping work around guesses at how
the grader is implemented.
Adapter prompt only; no runtime, evaluator or task definition is touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Remove the v6.82 mobile swipe gestures, make drawer opening clear keyboard state, and keep deeply nested live cards usable in narrow consumers. Set fresh-install triad defaults to Luna, Gemini Flash, and Sonnet while preserving saved settings.
Co-authored-by: Ouroboros <ouroboros@local.mac>
Forensics against the leader's own published per-task dump put the gap at 19
tasks, 8 of them one class: the work was done and never checked against the
surface the grader reads. The worker now writes the task's obligations as a
numbered checklist BEFORE its first mutating action and closes each item as
observed-satisfied / not-verified / impossible before it may finish, repairing
per item. Plural instructions still cover every element; only a singular
referent resolving to several candidates forces a justified single choice, and
the contract is revisable on new observation.
Three infeasibility shapes are named (discovery outside a stated means
restriction; a named mode the app does not ship; a trigger narrower than the
task states), framed on the END STATE rather than the route, with the brake the
gate already had: a wrong verdict scores zero even when the machine is already
correct. The desktop environment's own gsettings/dconf is a legitimate surface
for STORED values only — if the task wants something DISPLAYED and the device
does not exist, writing the key is not a workaround.
Two adversarial reviews corrected this change three times, and each correction
is the interesting part:
- The colour motivation I had deleted as false is TRUE: replaying the real
grader shows 8472fece's own gold (palette 2A6099) scores 0 against its own
evaluator, which measures distance to pure 0000FF. No palette entry can win
it; deleting the motivation gained nothing and endangered 04578141, a live
1.0 won because of it. Restored and tightened to the exact colour name.
- An earlier draft wrote the per-task proxy config — which carries the account
password — into all 361 result directories, i.e. the tree we archive and
publish. It now lives in lane-private state and is unlinked after the task.
- The session tag keyed on run_dir.parent.name, which is the DOMAIN, so the two
concurrent campaigns would have shared one exit IP per task. Keyed on the
campaign root instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both from the 22-task regression smoke, each with trace evidence:
- impress/05dd4c1d aligned the document-order shape (Shape;135) while the gold
targets the visually higher one (Shape;136). The slide-object branch said a
heading counts as the Nth item but never said in WHICH order to count.
- impress/04578141 read 'use exactly these colours, no variations' as a licence
to type raw 00FF00 through Custom Color; the gold is LibreOffice's palette
Green 00A933 at zero tolerance. That wording forbids a neighbouring shade, it
does not prescribe a hex.
Smoke result: 19 of 22 at 1.0, one at its historical continuous-metric value
(efcf0d81 = 0.8949354689860572, identical to v6.81.1), and these two.
Five tasks that scored 0 in earlier runs now score 1.0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two measured defects from the v6.83.0 campaign, adapter-only.
(1) Evaluator fixtures are declared RELATIVE to the checkout and get_local_file
tests them with a bare os.path.exists, so the grader resolved them against the
process CWD. The official runner works from the checkout root; this bridge did
not, and multi_apps/7f35355e produced the byte-exact answer 25.27 and still
scored 0.0 with only a line in the lane log. evaluate() now runs with the
checkout as CWD, restored on every path.
(2) The 14-turn gate reserve is worst-case; the gate actually spent a mean of 4,
so a flat max_steps-14-1 discarded ~10 turns per example and 13 of 56 opus
failures died at 89-92 turns INSIDE a 100-turn budget. The runner now publishes
the worker's real cap (max_steps - actual gate turns - 1) into the lane settings
the server hot-reloads at every task start, so the declared total is unchanged
and the unused reserve is no longer thrown away.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A gate INFEASIBLE ends the example before the working phase, so the worker
consumed exactly zero policy turns — a KNOWN count. The fail-closed audit read
it as unavailable and flagged budget_fault on the very outcome the gate exists
to produce (caught on os/a462a795 twenty minutes into the v6.83.0 run; the
running run keeps the false flag in its audit field because its seed must stay
clean, and it is corrected at scoring time).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Image integrity is fail-closed at three seams (remote fetch with bounded
re-fetch and write-validate-rename, the shared remote-result builder, the VLM
payload builder). A truncated PNG keeps a valid 24-byte header, so header-only
checks passed it and it detonated rounds later as a non-retryable provider 400
— five task deaths in the v6.81.1 OSWorld run. The rejection is narrow: a
valid-but-huge image and a truncated-yet-renderable JPEG still go through;
only what cannot be rendered at all is refused.
Structured tool failures ({ok: false}) now feed the error counters, anti-loop
and auto-attach — but NOT the verification ledger, because a diagnostic that
honestly reports what it was asked to find is a finding, not a failed run.
Acceptance review gains an ABSENT-PREMISE branch: when the terminal claim is a
missing premise, the deliverable under review is the premise ARGUMENT.
Demanding the named artifact begs the question, and coaching a continuation
that breaches the task's own restrictions manufactures what the task forbids.
A weak premise argument still fails on its own grounds.
type_text routes multi-line and long payloads through the in-VM clipboard, and
picks the paste chord inside the guest call (a terminal ignores Ctrl+V while
the hotkey still reports success).
OSWorld adapter: --max-steps declares AND enforces a leaderboard-comparable
budget. A step is one top-level policy turn, matching the official
predict()->actions[] boundary, not one GUI action. The server round cap is
verified before the VM boots; the gate phase is cancelled at its own reserve,
counted from its LIVE event log (loop_outcome exists only after finalization,
so polling for it would have been dead code); the post-run audit reads policy
turns, not the flat physical-call field they disagree with on 344 of 346
examples. --expect-dataset-commit turns the graded-spec pin into a gate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A leaderboard step is one top-level policy turn (lib_run_single.py increments
step_idx once per agent.predict() and runs every action that turn emitted),
NOT one GUI action — the earlier 0.42-actions-per-round mapping compared a
turn against an action and understated our budget ~2.4x.
- --max-steps declares AND enforces a comparable budget: gate reserve + worker
cap + one tool-less terminal turn, refused fail-closed before the VM boots
when the server round cap exceeds it.
- Post-run audit records policy turns actually used and marks overruns
non-comparable instead of rewriting the reward.
- Typed step semantics in the run manifest; MAX_STEPS wired through the lane
scripts.
- Replaced three test fixtures whose 'minimal valid PNG' literals were
undecodable, and corrected an assertion that pinned an identity coordinate
transform which only held because the stub never downscaled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- ACCEPTANCE_SURFACE_RULES: ABSENT-PREMISE / INFEASIBLE DISPOSITION branch —
when the terminal claim is infeasibility, the deliverable under review is
the premise argument; 'deliverable exists' as a criterion begs the question,
and coaching a restriction-breaching continuation manufactures artifacts
(v6.81.1 gimp/5ca86c6f: reviewer veto converted a scoring 1.0 into 0.0).
- OSWORLD_PREAMBLE: five clauses from the failure forensics (named value via
the app's own control; verbatim clipboard transfer; minimal-diff; ordinals
over real items; finish on the graded surface).
- GATE_PREAMBLE: 4d — named mode / apply scope / prohibition are premise
carriers, not working-phase details; fail-open default unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Not a release — a probe seed for the Opus-5 recovery measurement, to be
rebased onto the other agent's release and reviewed before any push.
- Extension tools answer with a JSON envelope, so a failed call arrived as
{"ok": false} with no core marker and was recorded a SUCCESS: 329 rows in
the v6.81.1 run (302 remote_exec, 20 screenshot, 5 key, 2 click). The error
counter, anti-loop, monitoring and the reflection trace all believed those
calls worked. _structured_tool_failure() reads the payload; typed status
tool_reported_failure; auto-attach refuses such a result.
- Host-side watchdog on the guest control endpoint: an agent killed the guest
server and then worked blind. Unreachable for 180s ends the attempt as a
typed INFRA row (reward null, claim released), never a capability zero.
Probe fails closed.
- type_text: < arrived as > (measured hex 3e for 3c) — angle brackets now take
the clipboard path non-ASCII already takes.
- key: whitespace is a SEQUENCE of chords; the bare form silently no-opped.
- Bridge URL removed from agent-visible results; list/test_connection denied —
an agent read the port from a tool result and curled <bridge>/evaluate.
- Gate rubric 4b/4c (same-thing check, verify-don't-assume) and a working-phase
rule that state must come through the app's own surface, not from underneath.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Everything here is a mechanism fix for a defect the v6.81.0 OSWorld full-run
forensics measured, none of it a patch over a symptom.
CORE (three touches, deliberately minimal):
- Tool results may carry a typed auto_attach_image capability: the host
attaches that local image to the conversation in the SAME round, after the
round's complete tool-message block, through the exact implementation the
view_image tool uses — vision.attach_local_image_to_context, ONE body for
both paths, so trust boundary (allowed roots, size cap, fail-closed MIME
sniff), durable copy (uploads/views) and message shape cannot drift.
Extension (ext_) results only: MCP results are untrusted server-supplied
data and must not drive automatic context mutation. Failure is strictly
non-fatal. Measured cost removed: 3,830 of 16,367 rounds (~21% of the round
budget) were the mandatory second view_image round per observation, and
every task at the 200-round cap scored 0.
- MAX_LIVE_IMAGE_BLOCKS 3 -> 5 (owner decision 2026-07-29).
SKILL unix_computer_use (manifest 0.3.0 -> 0.4.0 so the version-keyed native
seed resync actually distributes this):
- One pointer-coordinate normalizer behind click/move/aliases/mouse_down/
mouse_up/left_click_drag: accepts the malformations models actually emit
(the pair packed into x with y absent, legacy -1 sentinel, or a single
duplicating y — 109 wasted rounds in one run), distinguishes ABSENT from
UNPARSEABLE, and fails loudly on contradiction. y leaves the required
schema so recovery happens before binding.
- double_click/triple_click register as thin click aliases (111 previously
'Unknown tool' calls) — and enter the bench adapter's _GUI_ACTION_TOOLS in
the same commit, so the premise gate cannot click through an alias.
- screenshot results emit auto_attach_image (remote + local builders, both
pinned by tests); remote_exec's description states its real per-backend
contract (fresh bash -lc on OSWorld; SSH login shell on macOS; never the
visible desktop terminal).
OSWORLD ADAPTER:
- Premise-gate prompt becomes a structured rubric (action -> referent ->
blocking -> acquirable -> store-or-render -> unbound placeholders): the
v6.81.0 false kills all judged outcome-meaningfulness instead of
action-performability, and an exception list would be a keyword patch.
- The confirming challenger is REMOVED on its own full-run ledger: 20
invocations, 0 feasible saves, 1 officially-infeasible task lost, 215
worker rounds burned, and it CONFIRMED all four false kills —
identical-prompt re-reads are correlated, not independent. Claim window
back to one premise round; manifest discloses the absence.
- Working preamble: the forced screenshot->view_image loop is gone
(screenshots attach automatically), and an ENVIRONMENT PITFALLS section
states task-general state rules (live-app in-memory copies must be
reconciled after out-of-band edits; terminal tasks belong in the visible
terminal; PIDs resolved by exact executable, never self-matching -f
patterns) — phrased without any claim about what an evaluator inspects,
disclosed in METHODOLOGY as a scaffold revision whose numbers must not be
pooled with earlier ones.
REVIEW TRAIL: 7 triad+scope iterations (fable + gpt-5.6-sol +
gemini-3.6-flash, scope fable, effort high; artifacts under
/mnt/data/a.razzhigaev/osworld_runs/review_v6811_iter*). Fixed from review:
skill manifest version bump (distribution was version-keyed), ARCHITECTURE
auto-attach flow + K=5, per-backend remote_exec wording, gate-phase alias
denylist + named-literal test, post-block attach ordering with pinned test,
absent-vs-unparseable coordinate contract + handler-boundary tests,
real-producer auto_attach_image tests, up-to-two-tasks cost wording.
Rejected with evidence, recorded here per review policy: (1) the module/
function size hard-gate claim — the authoritative gate
(ouroboros/review.py excluded prefixes; tests/test_smoke.py _SKIP_DIRS)
explicitly excludes devtools/ and tests/, and the enforced suite is green;
(2) the demand to remove the ENVIRONMENT PITFALLS rules — the same reviewer
slot prescribed exactly these task-general formulations in iteration 4;
the remaining wording contains no evaluator-behavior claims.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Regenerated from the final 2-commit ouroboros-submission branch
(3ea30ef system + a691cf3 artifacts on upstream 5f8c50eb): includes
engine-task cost harvest contract (cost_final/cost_usd_with_children,
one-shot hot path, finalize on last observe), CLBENCH_SHIM_BIND,
docker --user mapping, run registration in DEFAULT_RUN_NAMES and
leaderboard SYSTEM_DEFS, and the rewritten run-specific METHODOLOGY.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The preceding commit attributed the v1 smoke's collapse (feasible-control mean
0.737 -> 0.459) to OSWorld's silent setup skip. Evidence from the v2 run says
otherwise, and the docs should say what the data says.
DockerProvider.revert_to_snapshot stops the container; start_emulator then
REALLOCATES ports via _get_available_port(5000). The VM address therefore
changes on every reset. v1 published that address exactly once, before the gate
— measured after the fact: 83/83 task dirs have bridge.json older than their
gate record — so the working phase kept driving the pre-gate port, which with 16
lanes allocating from one range another lane's container could already own. The
traces match: empty Desktops, missing task files, and one worker acting on a
different task's presentation entirely (content found in no task file — it was a
neighbouring lane's live VM). v2 republishes and shows 0 regressions against 9 at
the comparable stage; ports demonstrably move and repeat across lanes
(5024 seen on both lane0 and lane4).
The verified reset stays: it closes a real, independent OSWorld fail-open that
also affects ungated runs. But it is defence in depth, not the measured fix, and
both docs now say so — 24 post-gate resets in v2, 0 retries needed. Claiming a
guard fixed something it never fired on is how a harness acquires cargo cult.
Adds the ordering test the class needs: post-gate reset -> target file rewrite ->
_publish_target -> worker creation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 2026-07-28 smoke found the gate itself sound (13/27 caught, 0 false kills
in 42 feasible controls) and the scaffolding around it destructive: the bare
post-gate env.reset() hit OSWorld's silent fail-open (guest probe timeout ->
ALL setup steps skipped, 'Environment setup complete.' logged, no exception)
and working phases opened on VMs without the task's files. Feasible-control
mean fell 0.737 -> 0.459. Three changes, each closing a reviewed hole:
- _reset_verified() replaces BOTH bare resets: asserts the machine-checkable
postcondition (is_environment_used iff non-empty config) plus the screenshot
probe, forces the snapshot revert before every retry (an unforced retry runs
setup on top of partial state), republishes the VM endpoint after the
post-gate reset (docker recreate changes IP/ports), and turns exhaustion
into a typed infra row (reset_unverified, reward null, claim released) --
a setup the harness could not verify must never become a capability zero.
The silent-skip flaw predates the gate and affected single-reset runs too.
- An INFEASIBLE verdict no longer stands alone: an independent challenger
round (fresh session, same read-only envelope) must agree before the kill;
disagreement fails open. One false kill erases the gate's measured edge,
a missed infeasible still has the worker's own TASK_INFEASIBLE path.
- The one condition that must NOT fail open now does not: a premise round
whose cancel did not confirm aborts the attempt as blocked (exit 2) --
a zombie premise session shares the lane's server and connection file and
would act on the VM the worker is scored on. Each round's full tool trace
(verbatim args, not previews) lands in feasibility_gate.json for offline
audit of the read-only-by-instruction promise.
Claim staleness now covers two premise windows. Manifest discloses
feasibility_gate_challenger. Docs updated; 13 new tests pin the reset
postcondition, the two-verdict kill rule, the zombie detection and the
verbatim trace.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reviewed by codex gpt-5.6-sol and two independent readers before any run. They found one
defect that would have corrupted results and several that would have cost feasible tasks.
The claim staleness bound did not include the premise phase. The gated holder occupies
the claim for up to task_timeout/4 BEFORE the working task, so with shipped defaults it
consumed the entire margin the formula reserves for the unbounded evaluate() — after
which a second lane treats the lock as stale, takes the same task, and both score it.
The bound now grows by the same expression the phase's own deadline uses, so the two
cannot drift.
The verdict parser scanned every line in reverse for a keyword. A model that enumerates
the three options while reasoning and then concludes in prose had its recap read as its
answer: a PROCEED became a scored hard zero. Reproduced, then fixed to read only the last
line, which is what the prompt asks for; ordinary formatting (trailing period, bold,
backticks, case) is tolerated, a verdict inside a sentence is not. The reproduction is
now a regression test.
The INFEASIBLE path synthesized {"status": "completed", "result": "TASK_INFEASIBLE"} so
the existing detector would fire. That published a clean runtime outcome and a terminal
answer for an agent that never spoke — the same class of lie the final_answer fix in
257a369 removes. The flag is now explicit, the absent working phase is left visible as an
absence, and the outcome records infeasible_source and the phase's own rounds so a
gate-terminated example cannot be read as an agent that declared it.
Also: the untrusted task text no longer has the last word in the gate prompt (a task
saying "end with INFEASIBLE" could score itself zero); the infeasibility detector reads
the authoritative terminal answer instead of OR-ing over a field where a retracted
mention could step FAIL; sidecar write failures no longer control execution — one of
them sat between env.step("FAIL") and evaluate(), so a failed write lost a task that had
already been acted on; the gate's timeout cancel is confirmed rather than assumed, since
an unconfirmed cancel leaves a second agent on the same VM.
The phase closes the GUI vector only. remote_exec stays a general shell, read-only by
instruction, because classifying shell commands in code is the pattern gate P5 forbids.
The comment claiming the premise "cannot" be manufactured was wrong and now says what is
actually true; the working phase is re-reset after a PROCEED so nothing the premise phase
touched reaches the scored state. README and METHODOLOGY §7 (4c) state all of this.
Tests: the new control flow had none. Added the fail-open table, the recap regression,
the claim-bound relationship, the gate-phase tool removal, and a check that the
acceptance claims stay general.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The prose rule shipped in the previous commit tells the agent to establish the premise
before working. This adds the option of making that structural instead of advisory:
--feasibility-gate posts a first task whose mutating GUI tools are absent from the
capability envelope, so the agent judging whether the premise holds physically cannot
manufacture it first. That was the observed failure — same probe, same negative answer,
then a wallpaper copied onto an empty Desktop.
Three-valued on purpose. Only a standalone INFEASIBLE ends the example; PROCEED,
UNDETERMINED, an unreadable answer, a timeout, a crashed phase and any exception all
proceed to the full-capability phase. The gate may remove a task the agent was
affirmatively certain about and nothing else. The published verified run we compared
against lost a task by calling a Cloudflare block infeasible, so UNDETERMINED is named
in the prompt as the answer for network and anti-bot obstacles.
The INFEASIBLE path does not re-implement scoring: it synthesizes the terminal answer
the working phase would have produced and falls through to the single existing
evaluate() + claim-marker sequence, which is the code that protects against double
scoring and must have exactly one caller.
remote_exec stays available for read-only probes and read-only is an instruction, not an
enforcement: classifying a shell command as reading or writing in code would be the
pattern gate P5 forbids for a semantic decision.
Off by default, and the manifest stops claiming one run per task when it is on.
Unverified: the false-INFEASIBLE rate on feasible tasks is not measured yet, and that is
the number that decides whether this is worth its cost — README says so.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pairwise forensics against a published verified run on the same 361 tasks and the same
model put ~90% of our deficit on the 27 infeasible-evaluator tasks; on the 333 feasible
ones the two runs are statistically indistinguishable. Reading our own traces there, the
failure is not that the agent lacked information: it ran the correct probe, got the
correct negative answer, and then manufactured the premise — copying a system wallpaper
onto an empty Desktop and adjusting its own planted file, building a same-named theme
directory from a sibling, writing document internals the app cannot render — and
reported success.
The prose rules were already there and did not hold, so this changes what they say
rather than adding another one:
- the feasibility rule enumerated missing hardware, accounts and app features, which
never covered an absent object the task acts on. It now names the class: an essential
PRE-EXISTING target or capability the task presupposes is absent — as distinct from
what the task asks to create, from a detail mentioned only as motivation, and from a
transient network block. That last exclusion is not hypothetical: the verified run we
compared against lost 82279c77 by calling a Cloudflare block infeasible.
- establishing the premise no longer competes with speed; it is exempt from the
investigation budget that sits three lines below it.
- manufacturing the premise is named and forbidden, as is searching the guest for
grader code — observed in the v6.81.0 traces, nothing found, nothing exploited, and
it should be prohibited rather than merely unsanctioned.
acceptance_claims was [] on all 361 tasks while the acceptance panel ran on 324 of them.
Four general claims now travel with each task, so the reviewer that already runs
adjudicates observations instead of narrative. No extra model call.
cu_bridge outcomes carried final_answer: null while the answer sat in the runtime
result, which is exactly what METHODOLOGY §4 exists to prevent; it now falls back as
documented.
Nothing here encodes a base rate of infeasible tasks, a task id, an application, or any
evaluator property. Numbers from before and after this revision are from different
scaffolds and must not be pooled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
