- CLI spawn tests (calibrate, policy commands/enforcement/e2e) now isolate HOME
to a throwaway dir, so the spawned CLI seeds its model DB deterministically from
the packaged seed instead of reading the host's ~/.llm-checker. This also makes
policy-e2e's fail_count > 0 assertion robust instead of depending on the host
detecting models.
- Jetson nv_tegra_release test pinned process.platform to 'linux' instead of
early-returning (which made it pass vacuously with zero assertions off-Linux).
- hardware-simulation summary now records pass/fail for the matching and
quantization groups too, so Total/Passed cover all assertions (was TPS-only,
giving nonsensical passed+failed != total); added the previously-unexercised
large-model-on-weak-hardware cases (cpuOnly 13b/32b/70b, rtx4090 & m4Pro
32b/70b) and widened the degenerate cpuOnly 70b [1,1] band.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Introduce a complete calibration flow from CLI contract to full runtime execution.\n\n- add calibrate CLI contract with execution modes, benchmark options, and actionable validation errors\n- add strict zod schemas for prompt suites, calibration results, model traces, and policy routing\n- implement full benchmark runner with warmup/measured iterations, metric aggregation, failure isolation, and quality scoring\n- synthesize objective-aware routing policy with guardrails and deterministic fallback ordering\n- add calibration schema/CLI/full-mode tests and wire them into the unified test runner\n\nCloses #40\nCloses #41\nCloses #42\nCloses #43
