github.repository is mixed-case (AlexsJones/llmfit) and OCI repository
names must be lowercase. metadata-action lowercased it implicitly in the
old single-job flow, but the push-by-digest output and imagetools create
use the raw env var — both platform builds failed at export with
"invalid reference format".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The single-job buildx setup compiled the arm64 Rust binary under QEMU
emulation on an x86 runner, dominating the ~50 min image publish and
delaying the amd64 image behind it. Split into per-platform jobs
(linux/amd64 on ubuntu-latest, linux/arm64 on ubuntu-24.04-arm) pushing
by digest, with a merge job assembling the multi-arch manifest — same
tags and platforms, no emulation.
workflow_dispatch now takes a required release tag so an existing
release's image can be rebuilt on demand.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the Community Benchmarks workflow: any PR touching
llmfit-core/data/community/ runs scripts/validate_community_benchmarks.py,
which checks JSON schema conformance (schema.json, draft-07),
path/naming conventions (community/<slug>/<timestamp>-<hash>.json), file
size caps, and cross-field sanity the schema can't express (tps
ordering, plausible hardware bounds, submission timestamps within the
feature's lifetime). Whole-directory validation, so repo integrity is
re-checked on every touch; also runs on push to main.
Seeds the directory with the first genuine submission (Intel Arc 140V /
Core Ultra 7 258V, gemma-3 Q8 via llama-server, 3.8 tok/s) — previously
attempted in #713/#716 — which also exercises the workflow on this PR.
Closes the CI-validation follow-up named in #712.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Raises HuggingFace API rate limits and unlocks gated-model metadata;
the scraper already reads HF_TOKEN from env and warns-and-skips gated
repos without it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the estimate<->reality loop (#112, #119):
- The weekly workflow now refreshes the localmaxxing benchmark cache
(authenticated via LOCALMAXXING_API_KEY) alongside the catalog.
- New calibration test replays every usable cached measurement (single
request, pure-GPU, model in catalog) through estimate_tps using each
preset's bandwidth profile, prints per-preset accuracy, and fails on
systematic bias (median est/measured outside [0.33, 3.0]) — the class
of bug #449 was.
First run against 214 measured rows across 12 presets: overall median
est/measured 0.78 (safely conservative), p10 0.24, p90 1.47. Notable
per-preset finding: RX 7900 XT median 0.28 (3.5x underestimate, n=28)
- follow-up candidate for ROCm bandwidth/efficiency tuning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The doctor report (new in #675) captures raw detection-tool output that
turns hardware bug reports directly into regression-test fixtures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follows up the #585 merge: the schema now lives next to the catalog it
validates, and the workflow/test/docs no longer reference the removed
root data/ copies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Previous attempt added `if: needs.release.result == 'success'` to
publish-python, publish-crate, and update-homebrew, expecting that
to break the transitive skip when sign-windows fails. It didn't.
GitHub Actions rule: an `if:` expression only overrides the implicit
"skip if any ancestor failed/cancelled" behavior when it includes
one of `always()`, `failure()`, or `cancelled()`. A bare expression
that doesn't reference any of those still gets the implicit skip
gate ANDed in.
Verified against the v0.9.28 retroactive workflow_dispatch run
(26470106767): release succeeded, downstream jobs still skipped
despite the new if. Re-adding `always()` to actually take effect.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The release job runs with `if: always()` so a sign-windows failure does
not block the GitHub release. But publish-python, publish-crate, and
update-homebrew only had `needs: release`, so GitHub Actions transitively
skipped them whenever any earlier job (like sign-windows) failed.
This is why the AlexsJones/homebrew-llmfit tap has been pinned to 0.9.23
since v0.9.24 (2026-05-12): five consecutive release runs failed at
sign-windows, which silently skipped the tap update each time.
Add `if: needs.release.result == 'success'` to break the transitive
skip. Also document both Homebrew install paths in README so users on
older macOS can pick the prebuilt-binary tap over the source-build
homebrew-core formula.
Refs #581
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: add AutoRound format detection to prevent misclassification as GGUF
- Add ModelFormat::Autoround enum variant in models.rs
- Update is_prequantized() to include AutoRound
- Fix bit-per-parameter calculation for AutoRound-4bit
- Update scrape_hf_models.py to detect auto-round quant_method
- Correct 3 misclassified AutoRound models in hf_models.json
- Add AutoRound-4bit to SUPPORTED_QUANTS and parse_quant() in plan.rs
- Update pre-quantization comments in fit.rs
- Update test_model_format_prequantized() assertion
Fixes#526
* style: apply cargo fmt formatting
* ci: use npx for web build/test steps to fix Windows PATH issue
* fix: add AutoRound-8bit to quant metrics in models.rs
Add explicit entries for AutoRound-8bit in quant_speed_multiplier,
quant_bytes_per_param, and quant_quality_penalty. Without these,
an 8-bit AutoRound model would incorrectly fall through to the
4-bit defaults (0.5 bpp, quality penalty -5.0).
Fixes Node.js 20 deprecation warning: actions will be forced to Node.js 24
by default on June 2nd, 2026.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(python): scaffold in-repo PyPI packaging
Copied hatch_build.py, src/__init__.py, src/__main__.py, and tests/ from
the external llmfit-pypi repo into llmfit-python/. Created pyproject.toml
at the repo root (adapted from llmfit-pypi) with updated author, paths, and
non-standard source mapping for the gitignore-safe llmfit-python/src/ layout.
Created .github/workflows/pypi.yml to build and publish all 8 platform wheels
on tag push, mirroring the docker.yml trigger pattern.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(python): rewrite hatch_build.py to use in-repo binaries
hatch_build.py previously downloaded release binaries from GitHub at
wheel-build time, verifying SHA256 and fetching license via the GitHub
API (~267 lines). It now expects the pre-compiled binary to already
exist at target/<rust-target>/release/ and reads version and license
directly from Cargo.toml (~174 lines, no network calls).
This enables the new CI split in release.yml: each build matrix job
uploads its binary as a binary-<target> artifact, and a dedicated
publish-python job (ubuntu-latest only) downloads all of them and runs
uv build in a loop over platform tags before publishing to PyPI in a
single step. Wheel failures no longer block binary release assets.
Also removes the now-redundant standalone pypi.yml workflow, adds
license = "MIT" to Cargo.toml workspace.package, and relocates the
Python package source into src/llmfit/ to match hatch layout.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: add Python gitignore entries
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: add Python code quality Makefile targets
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(python): respect LLMFIT_PYTHON_PLATFORM_TAG in editable installs
When the env var is set, look in target/{rust_target}/release/ (via the
renamed _find_binary_for_target) rather than the default host-native
paths (_find_local_binary). Also splits py_target_from_env / py_target
in initialize so the editable branch reuses the already-read value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: add Python CI workflow and tighten Rust CI path filters
- Add python-ci.yml with cross-platform pytest job (gated on llmfit-python/
or pyproject.toml changes via dorny/paths-filter) and a lint/typecheck job
that also triggers on scripts/**/*.py changes
- Exclude llmfit-python/** from Rust CI path triggers
- Replace cryptic Cargo.*o* glob with explicit Cargo.{lock,toml}
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: remove committed .pyc file from tracking
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: normalise ruff lint exception comment formatting
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Ruff formatted these lines into one line
* Fix import ordering
* Move pyproject.toml and uv.lock into llmfit-python/
* Correct path to tested code
* Run checks without installing llmfit itself (which fails)
* chore: bump setup-python and setup-uv to latest versions
Upgrades actions/setup-python v5 → v6 and astral-sh/setup-uv v5 → v8.1.0
to resolve Node.js 20 deprecation warnings in CI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Disallow setting LLMFIT_PYTHON_PLATFORM_TAG on an editable build
* Improve inline comments in binary selection logic
* Move binary version check out of selection logic
* Always check that binary file exists
* Check all binaries before building wheels
* Set the execute bit that was lost by GitHub Actions
* Python CI downloads latest release binary instead of rebuilding Rust
Replaces the cargo build steps in the test job with a download of the
latest release archive from GitHub Releases, verified via SHA256.
LLMFIT_VERSION is set to the downloaded release version so the wheel
build does not conflict with an unreleased Cargo.toml bump.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* publish-python: use release file downloads instead of GHA artifacts
Removes the binary artifact upload/download mechanism. publish-python
now depends on the release job (not build) and downloads each archive
directly from GitHub Releases, verifying its checksum inline via pipe
before extracting the binary and building the wheel.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix typo in ci.yml comment: worfklows -> workflows
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Accept optional v prefix in binary --version output
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Simplify BinaryNotFoundError to inherit only from LlmfitError
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add fmt-check target; use in check instead of fmt
Avoids `make check` unexpectedly modifying files on disk.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Split up checksum file to work around * in sha file on Windows
* Fix Windows zip extraction in Python CI workflow
Use unzip instead of tar to extract .zip archives on Windows, since
Git Bash's tar does not support the zip format.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Make llmfit binary version visible in CI tests
* Prevent multiline echo-on-Windows weirdness
* Consolidate CI workflows and add path-based job gating
Merges python-ci.yml into ci.yml. Adds a changes job using
dorny/paths-filter to detect whether Rust or Python files changed,
then gates fmt/clippy/check/check-python jobs accordingly via
job-level if conditions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Tell uv where the python project is
* Remove outdated sentence about LLMFIT_VERSION from docstring
* Run Python tests with debug binary in editable mode for max reuse with cargo test
* Control the working dir to control test discovery
* Run Python build system integration test when there are no Python changes
* Update comment
* Always run cargo build on macos
* Show contents of target/ (for debugging)
* Debug production of debug builds by cargo test
* Always run cargo build to get a debug build
* Only build debug binary for subsequent pytest
* Revert "Debug production of debug builds by cargo test"
This reverts commit c88158e5ae.
* Revert "Show contents of target/ (for debugging)"
This reverts commit 947cb39988.
* Update comment with corrected description of work reuse
* Move prerequisite debug build step above uv sync step
* fix(ci): use picomatch extglob to exclude llmfit-python from rust filter
The previous config used a leading-! entry (`!llmfit-python/**`) to try
to exclude the Python directory from the rust filter. In dorny/paths-filter's
default 'some' mode, each entry is OR-ed: a leading ! is passed to picomatch
as a pattern that matches anything *not* in llmfit-python/, but the positive
`llmfit-*/**` entry already matched, so the negation had no effect and
rust-changed was always true for Python-only PRs.
Replace the two entries with a single picomatch extglob pattern
`llmfit-!(python)/**`. The !(python) extglob is evaluated against the
segment between `llmfit-` and the path separator, so llmfit-python/ never
matches while all other llmfit-* directories do.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(python): add rust_integration pytest mark for Rust-triggered CI runs
Marks test_binary_runs with @pytest.mark.rust_integration so it runs
in CI when only Rust code has changed, replacing the uv sync --no-dev
integration test step.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: add SignPath code signing for Windows builds
* ci: extract exe before signing to match SignPath PE artifact config
* ci: add SignPath Authenticode signing for Windows builds
- Add sign-windows job to release workflow
- Extract exe, submit to SignPath for signing, repackage into zip
- Release job proceeds even if signing is skipped
- Add SignPath signed badge and banner to README
Interpolating a secret into a shell command line:
cargo publish --token ${{ secrets.CARGO_REGISTRY_TOKEN }}
leaves the token visible in /proc/<pid>/cmdline and `ps` output for the
lifetime of the process. GitHub Actions secret masking redacts log
output only — it cannot scrub the kernel's process table.
cargo reads the CARGO_REGISTRY_TOKEN environment variable automatically;
the cargo book explicitly recommends this over --token for exactly this
reason. GitHub-hosted runners are ephemeral, but defense-in-depth costs
nothing here.
Co-Authored-By: gregkh_clanker_t1000
Updates Cargo.lock to match current workspace versions (v0.9.11) so that
`cargo build --locked` works for downstream packagers (e.g. macports).
Adds a post-processing step to the release-please workflow that runs
`cargo update --workspace` on release PRs, ensuring future version bumps
also update the lock file.
Fixes#476
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GITHUB_TOKEN events don't trigger other workflows, so the Release
workflow never fired on release:published. Using a PAT fixes this.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switches the Release workflow trigger from `push: tags` to the GitHub
`release: published` event, and adds a `workflow_dispatch` input for
manual backfills of an existing tag.
release-please creates tags with GITHUB_TOKEN, which by design does not
trigger downstream workflows. That's why v0.9.4 shipped with only
release notes and no binary assets (#424), while v0.9.3 (tag pushed
manually) worked. Reacting to the `release` event published by
release-please fires the workflow reliably for future releases.
The workflow_dispatch path accepts an existing tag and routes it
through checkout, packaging, asset upload, and the Homebrew tap update
so v0.9.4 can be backfilled after this lands.
Fixes#424
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces manual tag-and-push workflow with release-please, which
auto-creates release PRs with version bumps and changelogs on merge
to main. Existing release.yml build pipeline is unchanged — it still
triggers on v* tags created by release-please.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
