Commit graph

1 commit

Author SHA1 Message Date
Alex Jones
7b9cc973ae ci: validate community benchmark submissions on PR
Adds the Community Benchmarks workflow: any PR touching
llmfit-core/data/community/ runs scripts/validate_community_benchmarks.py,
which checks JSON schema conformance (schema.json, draft-07),
path/naming conventions (community/<slug>/<timestamp>-<hash>.json), file
size caps, and cross-field sanity the schema can't express (tps
ordering, plausible hardware bounds, submission timestamps within the
feature's lifetime). Whole-directory validation, so repo integrity is
re-checked on every touch; also runs on push to main.

Seeds the directory with the first genuine submission (Intel Arc 140V /
Core Ultra 7 258V, gemma-3 Q8 via llama-server, 3.8 tok/s) — previously
attempted in #713/#716 — which also exercises the workflow on this PR.

Closes the CI-validation follow-up named in #712.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 11:33:44 +01:00