vrr/ouroboros
2
0
Fork 0
mirror of https://github.com/razzant/ouroboros.git synced 2026-08-25 11:01:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
ouroboros/devtools
History
Exact
Anton Razzhigaev 7b2f68fd04 devtools: pin bench review-cycle ceiling; docs: contract-fingerprint scope note 
Owner decisions Q6-A and Q3-A (2026-08-21).

Q6-A: the shared OUROBOROS_REVIEW_MAX_CYCLES cap (shipped default "2") also
bounds task-acceptance panels (passes = cycles - 1); past bench campaigns ran
3-26 paid acceptance panels per task, so the shipped default would silently
change bench methodology comparability. Pin the ceiling off in every bench
scaffold:
- add "OUROBOROS_REVIEW_MAX_CYCLES": "unlimited" (the canonical unlimited
  sentinel per ouroboros/review_cycles.py) to all bench settings templates:
  continual_learning, gaia, osworld, programbench settings_base.json and the
  four swe_bench_pro e1v2 templates (settings_base, _run_settings.example,
  settings_sonnet46_probe, profiles/light_subagents_gpt55);
- terminal_bench has no JSON template: pin the same value in the
  harbor_installed_agent container env beside the other review knobs;
- forward the key through _ISO_SETTINGS_ALLOW_EXACT in common/server_runner.py
  so a live-settings pin reaches isolated bench servers like the other
  non-secret review-policy knobs.

Also align swe_bench_pro/e1v2/settings_base.json to
OUROBOROS_REVIEW_ENFORCEMENT=advisory: METHODOLOGY.md documents that E1v2
templates deliberately stay advisory (the in-bench commit_reviewed evolution
lane is structurally uncommittable under blocking with the no-version-bump
steer); "blocking" in settings_base was a disclosed discrepancy. Pure JSON
forbids an inline comment, so the rationale lives here and in METHODOLOGY.md.

Q3-A: document in docs/DEVELOPMENT.md the deliberate review-contract
fingerprint scope — roster, routes, enforcement, resolved efforts, prompt
constants; governance-document contents (BIBLE/CHECKLISTS/ARCHITECTURE) stay
outside it, so editing them neither lapses recorded verdicts nor frees
replays; the accepted trade-off is that an old verdict can replay under
amended governance text.

No scoring/grading/task code touched; version carriers unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 04:46:26 +00:00
..
benchmarks devtools: pin bench review-cycle ceiling; docs: contract-fingerprint scope note 2026-08-21 04:46:26 +00:00
__init__.py feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
README.md feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00

README.md

Ouroboros Devtools

devtools/ contains operator-side and benchmark support code that should be versioned with Ouroboros without becoming part of the runtime core.

Rules:

  • Generated logs, datasets, run outputs, Docker layers, and secrets do not live here.
  • Default benchmark outputs go under /Users/anton/Ouroboros/bench_runs/.
  • Runtime modules must not import devtools.
  • This is not an immune-system bypass: touched files are reviewed normally.
  • Promote code out of devtools only through a separate reviewed runtime plan.