mirror of
https://github.com/razzant/ouroboros.git
synced 2026-08-05 16:49:47 +00:00
6b831640eb
Two v6.86.0 losses traced to one paragraph of the OSWorld working prompt. WHERE. The contract asked only where a result must PERSIST, so a task whose grader reads the LIVE window (is_vlc_fullscreen compares window size to screen) was answered by ticking the preference and never entering fullscreen. WHERE now has two slots — live and persisted — each filled or explicitly marked not applicable. The not-applicable branch is load-bearing: 28 scoring tasks are read only through tab lists or the active page, is_expected_tabs compares list LENGTHS, and an extra tab opened "to check the stored slot" would zero them. So the clause forbids inventing an action, tab, window or dialog to fill a slot the task does not have. UNCHANGED. A task asking for a bullet on an existing paragraph was answered by typing a new line, because the contract had recorded that paragraph as UNCHANGED. UNCHANGED now covers only content the task does not mention, with a narrow exception: new content is created only when the task asks for something that does not exist yet, while a MARKER or PROPERTY the task names — a bullet, a style, a colour, an alignment — is applied to the content already there. Typing a fresh line to carry the marker leaves the named content unmarked. Three further clauses were written and dropped after adversarial review showed each costs more than it wins, all verified against winning traces: - prefer a slide master over per-instance edits: 358aa0a7 scores 1.0 and its trace states master edits would not work there, because the shapes carry direct character formatting that overrides styles; - exhaust a named resource before any substitute: 36037439 scores 1.0 precisely by detouring to another search engine, and the preamble already says to retry and try another route on an anti-bot block; - require a configuration CLI to match the GUI's breadth: generalised from the single evaluator in the suite that takes a majority over mime types, and it contradicts the preamble's own rule against shaping work around guesses at how the grader is implemented. Adapter prompt only; no runtime, evaluator or task definition is touched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| benchmarks | ||
| __init__.py | ||
| README.md | ||
Ouroboros Devtools
devtools/ contains operator-side and benchmark support code that should be
versioned with Ouroboros without becoming part of the runtime core.
Rules:
- Generated logs, datasets, run outputs, Docker layers, and secrets do not live here.
- Default benchmark outputs go under
/Users/anton/Ouroboros/bench_runs/. - Runtime modules must not import
devtools. - This is not an immune-system bypass: touched files are reviewed normally.
- Promote code out of
devtoolsonly through a separate reviewed runtime plan.