mirror of
https://github.com/razzant/ouroboros.git
synced 2026-08-05 00:29:46 +00:00
86dad2255f
Everything here is a mechanism fix for a defect the v6.81.0 OSWorld full-run forensics measured, none of it a patch over a symptom. CORE (three touches, deliberately minimal): - Tool results may carry a typed auto_attach_image capability: the host attaches that local image to the conversation in the SAME round, after the round's complete tool-message block, through the exact implementation the view_image tool uses — vision.attach_local_image_to_context, ONE body for both paths, so trust boundary (allowed roots, size cap, fail-closed MIME sniff), durable copy (uploads/views) and message shape cannot drift. Extension (ext_) results only: MCP results are untrusted server-supplied data and must not drive automatic context mutation. Failure is strictly non-fatal. Measured cost removed: 3,830 of 16,367 rounds (~21% of the round budget) were the mandatory second view_image round per observation, and every task at the 200-round cap scored 0. - MAX_LIVE_IMAGE_BLOCKS 3 -> 5 (owner decision 2026-07-29). SKILL unix_computer_use (manifest 0.3.0 -> 0.4.0 so the version-keyed native seed resync actually distributes this): - One pointer-coordinate normalizer behind click/move/aliases/mouse_down/ mouse_up/left_click_drag: accepts the malformations models actually emit (the pair packed into x with y absent, legacy -1 sentinel, or a single duplicating y — 109 wasted rounds in one run), distinguishes ABSENT from UNPARSEABLE, and fails loudly on contradiction. y leaves the required schema so recovery happens before binding. - double_click/triple_click register as thin click aliases (111 previously 'Unknown tool' calls) — and enter the bench adapter's _GUI_ACTION_TOOLS in the same commit, so the premise gate cannot click through an alias. - screenshot results emit auto_attach_image (remote + local builders, both pinned by tests); remote_exec's description states its real per-backend contract (fresh bash -lc on OSWorld; SSH login shell on macOS; never the visible desktop terminal). OSWORLD ADAPTER: - Premise-gate prompt becomes a structured rubric (action -> referent -> blocking -> acquirable -> store-or-render -> unbound placeholders): the v6.81.0 false kills all judged outcome-meaningfulness instead of action-performability, and an exception list would be a keyword patch. - The confirming challenger is REMOVED on its own full-run ledger: 20 invocations, 0 feasible saves, 1 officially-infeasible task lost, 215 worker rounds burned, and it CONFIRMED all four false kills — identical-prompt re-reads are correlated, not independent. Claim window back to one premise round; manifest discloses the absence. - Working preamble: the forced screenshot->view_image loop is gone (screenshots attach automatically), and an ENVIRONMENT PITFALLS section states task-general state rules (live-app in-memory copies must be reconciled after out-of-band edits; terminal tasks belong in the visible terminal; PIDs resolved by exact executable, never self-matching -f patterns) — phrased without any claim about what an evaluator inspects, disclosed in METHODOLOGY as a scaffold revision whose numbers must not be pooled with earlier ones. REVIEW TRAIL: 7 triad+scope iterations (fable + gpt-5.6-sol + gemini-3.6-flash, scope fable, effort high; artifacts under /mnt/data/a.razzhigaev/osworld_runs/review_v6811_iter*). Fixed from review: skill manifest version bump (distribution was version-keyed), ARCHITECTURE auto-attach flow + K=5, per-backend remote_exec wording, gate-phase alias denylist + named-literal test, post-block attach ordering with pinned test, absent-vs-unparseable coordinate contract + handler-boundary tests, real-producer auto_attach_image tests, up-to-two-tasks cost wording. Rejected with evidence, recorded here per review policy: (1) the module/ function size hard-gate claim — the authoritative gate (ouroboros/review.py excluded prefixes; tests/test_smoke.py _SKIP_DIRS) explicitly excludes devtools/ and tests/, and the enforced suite is green; (2) the demand to remove the ENVIRONMENT PITFALLS rules — the same reviewer slot prescribed exactly these task-general formulations in iteration 4; the remaining wording contains no evaluator-behavior claims. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| benchmarks | ||
| __init__.py | ||
| README.md | ||
Ouroboros Devtools
devtools/ contains operator-side and benchmark support code that should be
versioned with Ouroboros without becoming part of the runtime core.
Rules:
- Generated logs, datasets, run outputs, Docker layers, and secrets do not live here.
- Default benchmark outputs go under
/Users/anton/Ouroboros/bench_runs/. - Runtime modules must not import
devtools. - This is not an immune-system bypass: touched files are reviewed normally.
- Promote code out of
devtoolsonly through a separate reviewed runtime plan.