vrr/ouroboros
2
0
Fork 0
mirror of https://github.com/razzant/ouroboros.git synced 2026-08-11 11:43:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 15
ouroboros/devtools/benchmarks
History
Exact
Ouroboros d863197cf6 feat(devtools-benchmarks): add official benchmark harnesses and workspace executor
2026-06-06 12:03:30 +03:00
..
common feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
osworld feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
programbench feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
swe_bench feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
swe_bench_pro feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
terminal_bench feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
__init__.py feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00
README.md feat(devtools-benchmarks): add official benchmark harnesses and workspace executor 2026-06-06 12:03:30 +03:00

README.md

Benchmark Devtools

This directory contains thin adapters around official benchmark harnesses. The adapters prepare Ouroboros tasks, capture artifacts, and preserve traces; they do not replace official scoring.

Supported surfaces:

  • ProgramBench: official programbench eval/info and cleanroom submission layout.
  • Terminal-Bench: Harbor installed-agent adapter.
  • SWE-bench and SWE-bench Verified: standard predictions JSONL for the official SWE-bench harness.
  • SWE-bench Pro: official Pro eval wrapper and patch-capture methodology.
  • OSWorld: stop-closed skeleton and logs normalizer only.