vrr/ouroboros
2
0
Fork 0
mirror of https://github.com/razzant/ouroboros.git synced 2026-08-04 16:19:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
ouroboros/devtools
History
Exact
Anton Razzhigaev 3b19b16291 docs(terminal_bench): TB2.1 leaderboard submissions are OPEN via harbor CLI 
The README claimed submissions were CLOSED pending a new process (verified
2026-06-30 against the HF PR flow). That is stale: since 2026-06 Harbor Hub
accepts TB2.1 submissions via `harbor auth login` -> `harbor upload` ->
`harbor leaderboard submit` (harbor >= 0.17), and the tbench.ai docs page
still saying "coming soon" is itself stale. This misdirection already caused
two false "submissions are closed" conclusions.

- README: replace the CLOSED status + HF-PR framing with the verified harbor
  CLI flow; document the hard requirements (ATIF trajectory per passing
  trial, trajectories before first upload, jobs become public -> mandatory
  secret scrub, headless auth options, multi-job submissions, content-hash
  task-version checks, dynamic LLM validation); bump harbor 0.13.1 -> 0.17.1.
- METHODOLOGY: add "Leaderboard submission" section (trajectory provenance,
  scrub policy, which runs may be submitted).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:15:22 +00:00
..
benchmarks docs(terminal_bench): TB2.1 leaderboard submissions are OPEN via harbor CLI 2026-07-07 23:15:22 +00:00
__init__.py feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
README.md feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00

README.md

Ouroboros Devtools

devtools/ contains operator-side and benchmark support code that should be versioned with Ouroboros without becoming part of the runtime core.

Rules:

  • Generated logs, datasets, run outputs, Docker layers, and secrets do not live here.
  • Default benchmark outputs go under /Users/anton/Ouroboros/bench_runs/.
  • Runtime modules must not import devtools.
  • This is not an immune-system bypass: touched files are reviewed normally.
  • Promote code out of devtools only through a separate reviewed runtime plan.