mirror of
https://github.com/razzant/ouroboros.git
synced 2026-08-10 11:03:16 +00:00
fa424eb46f
Context window is no longer a static per-model table: every window claim is sourced, route-fingerprinted Capability Evidence (provider /models metadata, local n_ctx, or an owner acknowledgement) with a status (confirmed/asserted/unprobeable/failed), persisted atomically. Max context mode is fail-closed — it requires >=1M confirmed/asserted evidence for the active route. Changing the model while Max is on stays friction-free: the change succeeds and context auto-downgrades to Low with a plain notice when the new route can't be confirmed >=1M, but a genuine no-connection during the probe is an error (the model is not saved), and a transient provider outage never erases a prior confirmed record. Multi-project: the agent can now CREATE a NAMED project from chat in one LLM-first call (promote_chat_to_task project_name/title; non-ASCII names get a deterministic hash id while the display name is preserved). A main-chat task converts to a project in one click, auto-named from its title/objective (no prompt, no extra LLM call); project-chat follow-up tasks bind to their project so the main chat shows no stray "turn into project" button and instead a calm pointer that opens the project panel; a converted card becomes a calm indigo project identity (no red "error" look); per-project unread dots sort active projects to the top (server-stored last-viewed); the project status/sleep-wake lifecycle was removed. UI: oval (pill) composer with centered controls; per-thread chat scroll restored on tab/ panel switch instead of jumping to the top. Also: real deadline_at finalization + advisory pacing, polyglot tree-sitter code intelligence for non-Python symbols (query_code op=digest; Python stays on stdlib ast), reflection faculty-atrophy doctrine, BIBLE P1 (Capability Evidence) + P8 (faculty atrophy) clauses, and assorted WS9 tool fixes. New surface: POST /api/owner/capability-ack, ouroboros/capability_evidence.py, data/state/capability_evidence.json. Reviewed by triad (gpt-5.5/gemini-3.5-flash/opus-4.8) + scope (gpt-5.5) + claudexor (gpt-5.5) + an independent adversarial multi-agent audit, against the original plans and the owner's raw message transcript; all confirmed defects fixed, remaining findings evidence-rejected or tracked. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| common | ||
| harness_bench_fast | ||
| osworld | ||
| programbench | ||
| swe_bench | ||
| swe_bench_pro | ||
| terminal_bench | ||
| __init__.py | ||
| evolve_smoke.py | ||
| README.md | ||
Ouroboros Benchmark Devtools
This directory contains tracked operator tooling for reproducible benchmark work. These files are reviewed when touched, but are not imported by the runtime core and are not packaged as app runtime code.
Integrations
terminal_bench/— Harbor installed-agent adapter for Terminal-Bench 2.1. Userun_tb.pyfor leaderboard-shaped k-trial runs and submission layout; userun_harbor_smoke.pyfor small local smoke runs.osworld/— OSWorld logs tooling plusrun_step_agent.py, an official env.step-shaped runner that passes VM screenshots as native image attachments to Ouroboros.swe_bench_pro/— SWE-bench Pro patch capture/grading. Frozen prepared repos usepro_predictions.py; persistent evolutionary runs usee1v2/run_pro.py/e1v2/auto_run.py.swe_bench/— standard SWE-bench prediction helpers.programbench/— ProgramBench cleanroom runner.harness_bench_fast/— Ouroboros CLI wrapper and methodology notes for the publicai-forever/harness-bench-fastrunner.common/— shared manifests, result ledgers, safe run roots, secret hygiene, and official command builders.
Output Roots
Write generated run artifacts under an explicit benchmark output root outside
repo/ and outside live runtime data/, typically
/Users/anton/Ouroboros/bench_runs/.... Tests must set
OUROBOROS_BENCH_RUNS_ROOT to a temporary directory so local test runs do not
pollute real benchmark bundles.
Shared Sidecar Schemas
- Run manifests record non-secret provenance: requested task ids where the
benchmark runner exposes them before execution, requested counts/selection
slots for deterministic first-N runs such as Terminal-Bench, exact argv,
official command shape, output paths, model slots, source commit, dirty-state
counts, and hashes. Defaults are adapter-specific (
run_manifest.json,<predictions>.run_manifest.json, orosworld_preflight.run_manifest.json). - Result ledgers are denominator-preserving Ouroboros JSONL files. They record
every requested instance, including setup failures, timeouts, and empty
patches, even when the official benchmark prediction/submission format only
accepts successful rows. Defaults are adapter-specific (
result_index.jsonl,<predictions>.ledger.jsonl, orosworld_preflight.ledger.jsonl).
These sidecars are audit artifacts, not replacement scoring. Official benchmark harnesses and official result files remain the scoring authority.
Methodology Rule
Benchmark changes must be general-purpose harness improvements first. Do not add task-specific answers, hidden verifier knowledge, or resource/timeout overrides that violate a benchmark's official submission rules.