github.repository is mixed-case (AlexsJones/llmfit) and OCI repository
names must be lowercase. metadata-action lowercased it implicitly in the
old single-job flow, but the push-by-digest output and imagetools create
use the raw env var — both platform builds failed at export with
"invalid reference format".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The single-job buildx setup compiled the arm64 Rust binary under QEMU
emulation on an x86 runner, dominating the ~50 min image publish and
delaying the amd64 image behind it. Split into per-platform jobs
(linux/amd64 on ubuntu-latest, linux/arm64 on ubuntu-24.04-arm) pushing
by digest, with a merge job assembling the multi-arch manifest — same
tags and platforms, no emulation.
workflow_dispatch now takes a required release tag so an existing
release's image can be rebuilt on demand.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The REST API and `llmfit system --json` had drifted: the CLI's private
system_json copy in display.rs gained memory_bandwidth_gbps but the
serve_shared copy used by /api/v1/system, MCP, and SSE events never did.
Consumers that treat the API as authoritative (e.g. llmfit-dra) saw GPU
bandwidth zero out and rejected every bandwidth-bounded claim.
Add the field to serve_shared::system_json and make display.rs delegate
to it so the two serializers cannot drift again. Cover the field with
unit tests (known GPU -> value, unknown GPU -> null) and assert its
presence in the existing /api/v1/system route test.
Fixes#747
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On AMD Ryzen AI MAX (Strix Halo) systems missing libdrm_amdgpu.so,
`rocm-smi --showproductname` prints `Card Series: N/A`. llmfit took that
literally as the GPU name, which cascaded into three problems:
- the leaderboard hardware identity became "N/A" (meaningless `bench/n-a-…`
PRs);
- the APU-unify step (which matches a GPU whose name contains "amd"/
"radeon") never matched, so the machine was classed DISCRETE_GPU with the
64 GB shared pool recorded as dedicated VRAM instead of UNIFIED;
- any such machine would keep contributing unusable results.
Detection (hardware.rs):
- Treat rocm-smi "N/A"/empty product names as a generic "AMD GPU" rather
than a literal identity, in both block and tabular parse paths.
- On a recognised unified APU, upgrade a generic GPU name to the APU model
(e.g. "AMD Ryzen AI MAX+ 395 w/ Radeon 8060S (integrated)"), which names
the iGPU and gives a stable leaderboard identity. This also lets the
APU-unify step flag unified memory correctly.
Sharing (share.rs):
- Refuse to submit a stored benchmark whose hardware is still a placeholder
(GPU name "N/A" for a GPU machine, or an unknown CPU for CPU-only). The
community board groups by hardware, so an unidentified machine is noise —
results stay stored locally with a clear message instead.
Verified on a real Strix Halo box: gpu_name now resolves to the APU model,
unified_memory=true, class UNIFIED.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
LM Studio and Docker Model Runner were detected purely by probing
their local HTTP APIs, which are only up while LM Studio's server or
Docker Desktop is running. Installed-but-idle copies rendered as
"not detected" in the TUI provider bar.
Add install-location detection (pure candidate builders + exists()
wrappers): LM Studio via Program Files/LocalAppData exe paths, the
lms CLI, ~/.lmstudio and macOS app bundles; Docker Desktop via the
classic and frontend\ exe layouts, app bundles, /opt/docker-desktop,
~/.docker/desktop and the docker-model CLI plugin. The TUI now shows
a warning-colored "installed (not running)" / "installed (server
off)" state, matching the existing MLX/llama.cpp cached idiom, and
Ollama's header now uses the already-tracked binary_available signal
the same way. llmfit doctor gains a "Provider app installs" section
so future reports carry the evidence.
Fixes#731
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PCI-address colons in the fake sysfs paths are invalid Windows
filenames, so create_dir_all failed the test on windows-latest. The
function under test only reads the tree layout, not the dir name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Discrete Arc cards were detected by name but always reported no VRAM:
the lspci path left vram_gb as None for "the Vulkan fallback to fill
in", but detect_vulkan_gpu_info() only parses device names and never
reads memory heaps, so the merge condition adopting Vulkan VRAM could
never fire. Fit scoring then saw 0 GB and recommended CPU-class models
unless the user overrode memory manually.
Read dedicated VRAM directly from sysfs instead, keyed by the card's
PCI address from lspci -nnD:
- xe driver: sum of tileN/physical_vram_size_bytes under the PCI
device directory
- i915 discrete: drm/cardN/lmem_total_bytes
(mem_info_vram_total is amdgpu-only, which is where the old comment's
"no sysfs VRAM file" claim came from.)
Also fix GPU-name deduplication for the Arc Pro series: is_same_gpu_name
matched Intel entries only on 3-5 digit model numbers, so two-digit Pro
models ("B70") never merged with their Vulkan/Level Zero names — and
worse, fell through to the no-model-number rule that swallowed them into
the iGPU entry. Letter-prefixed model tokens (a770, b580, b70) are now
compared as well.
Verified on a Lunar Lake iGPU machine (integrated path unchanged) and
with fake xe/i915 sysfs trees in tests; dual-card B70 lspci fixture
covers the reporter's setup.
Fixes#609
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unsloth "Dynamic" GGUFs embed a `-UD` marker between the model name and
the quant suffix (e.g. `Qwen3.6-35B-A3B-UD-Q4_K_M.gguf`). `strip_gguf_quant_suffix`
only removed the quant token, so the stem reduced to `qwen3.6-35b-a3b-ud`
and never matched the catalog id `qwen3.6-35b-a3b`.
As a result a model served by llama-server (or present on disk) read as
neither served nor installed: benchmarking reported "installed but not
served by any running provider", and the installed column/detail panel
omitted it.
Strip a trailing `-ud` marker alongside the quant so the stem reduces to
the canonical model name. Fixes both the benchmark-target match
(`tag_matches_model`) and the installed-set detection
(`is_model_installed_llamacpp`), which share this helper. Non-Unsloth
files are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sets DEFAULT_CLIENT_ID to the registered llmfit OAuth App (device flow
enabled; a public client id is not a secret — the device flow needs no
client secret, which is why it was chosen). oauth_client_id() no longer
treats the default as an unregistered placeholder: the shipped id is
used unless LLMFIT_GH_CLIENT_ID overrides it, and an explicitly empty
override opts out of interactive login (restricted CI).
Verified live: with no GITHUB_TOKEN/GH_TOKEN, no cached token, and no
env override, `llmfit bench --share` presents a real GitHub device
code. Completes the last item on #712's pre-merge checklist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the read side of the contribution loop (RFC #710): community
submissions merged under llmfit-core/data/community/ are aggregated by a
new llmfit-core build.rs into an embedded JSON array — a merged
benchmark ships in the next release with no CI step or network fetch.
On hardware identical to a submission's (CPU + GPU fingerprint):
- benchmark page shows the runs as 'llmfit community' rows (distinct
color), pinned below the user's own 'you (local)/(shared)' rows and
deduped against them (your own shared runs also live in the embed)
- fit rows get measured tok/s from a new CommunityBenchIndex, slotted
between the user's own runs and localmaxxing preset medians; a new
MeasuredSource::CommunityLlmfit variant keeps provenance visible
('Measured on Identical Hardware' in the estimate detail)
- community anchors feed estimate calibration, so a fresh install gets
corrected estimates before its user ever benchmarks anything
hardware_payload_matches moves to benchmarks.rs and is shared with the
local store's matches_hardware. Verified end-to-end: with a temp
submission for llama3.1:8b @ 4.0 tok/s and an empty local store
(simulated fresh machine), the fit table shows 4.0 measured
(community_llmfit) and every estimate calibrated x0.37.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the 1.0 banner (kept as a 'previously' link) with the
benchmark-sharing story in all three languages, linking to the
'Contributing benchmarks' guide in docs/cli.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-1.1.0, feat commits no longer drive minor bumps on every release:
versioning switches to always-bump-patch, so the next release is 1.1.1
regardless of commit types. Minor/major releases become deliberate
decisions via a Release-As footer (same mechanism used to cut 1.0.0).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the Community Benchmarks workflow: any PR touching
llmfit-core/data/community/ runs scripts/validate_community_benchmarks.py,
which checks JSON schema conformance (schema.json, draft-07),
path/naming conventions (community/<slug>/<timestamp>-<hash>.json), file
size caps, and cross-field sanity the schema can't express (tps
ordering, plausible hardware bounds, submission timestamps within the
feature's lifetime). Whole-directory validation, so repo integrity is
re-checked on every touch; also runs on push to main.
Seeds the directory with the first genuine submission (Intel Arc 140V /
Core Ultra 7 258V, gemma-3 Q8 via llama-server, 3.8 tok/s) — previously
attempted in #713/#716 — which also exercises the workflow on this PR.
Closes the CI-validation follow-up named in #712.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
