Step 1 — catalog hygiene: remove 100 CI/test-stub entries from
hf_models.json (tiny-random/*, *-tiny-random, tiny-dummy, ci-random-*,
test-*/testing-* repos). These are randomly initialized micro-models
whose names shadow real families — e.g. tiny-random/gemma-3 (9M params)
captured a local 2.5B gemma-3 GGUF and produced an 880 tok/s estimate
for a model that measures 3.8. The scraper now skips them at discovery
and at scrape time (is_test_stub), so they cannot return.
Step 2 — local calibration: one trustworthy benchmark now corrects the
whole estimate column for this machine. Anchors are locally-benched
fits on real catalog models (>= 1B params, dense); the median
measured/estimated ratio (clamped to [0.05, 3.0]) scales every row's
estimate, recorded in estimate_basis.local_calibration and printed in
the estimate-basis detail. Application is idempotent, so post-bench
refreshes never compound.
Stored runs are now also filtered by hardware fingerprint (CPU + GPU
name): measurements from a previous machine configuration neither
override nor calibrate the current one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GGUF source candidates were matched by bare model name only, so models
sharing a name across orgs got linked to quants of the wrong weights:
tiny-random/gemma-3 (9M test stub) linked to mradermacher/gemma-3-GGUF
(a 2.5B model) — downloading 2.5 GB while llmfit predicted 0.0 GB disk.
tiny-random/phi-4 (3M) pointed at quants of microsoft/phi-4 (14.7B).
Scraper: candidate GGUF repos must now declare a base_model tag matching
the source repo. Mismatches are still accepted when the declared base has
~the same parameter count (safetensors total, +-30%) — that's a mirror or
re-upload of identical weights (e.g. unsloth re-uploads pointing at the
canonical upstream), not a wrong link.
Data: audited all 1,656 linked quant repos against the HF API with the
same rules — 19 wrong-model links removed (14 confirmed by parameter
mismatch, 5 unverifiable), 1,510 verified, 295 mirror links kept.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three fixes for metadata loss during catalog regeneration:
- The additive merge with the existing database now backfills
hf_downloads/hf_likes/release_date/languages from the previous
catalog when the fresh entry lacks them. Unauthenticated runs
replace gated models with FALLBACK stubs (0 downloads/likes),
which previously clobbered real values from earlier scrapes.
- _build_discovered_model now fetches full model info when the
listing omits downloads/likes (non-download sort strategies like
trending return listings without these fields), instead of
writing zeros into the catalog.
- infer_languages derives the language for Meta MMS per-language
models (facebook/mms-tts-<iso639-3>) from the repo name, since
these models declare no language metadata via the API.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The model was already picked up by catalog auto-discovery, but had no
gguf_sources (so no download path in the TUI) and was not pinned in
TARGET_MODELS, so it could drop out of future scrapes.
- Add huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated to TARGET_MODELS
- Add MOE_ACTIVE_PARAMS fallback (3B active, matching the base
Qwen/Qwen3.6-35B-A3B it is finetuned from)
- Enrich the existing catalog entry with the mradermacher GGUF source
and align active_parameters with the published 3B figure
Closes#656
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
data/hf_models.json and data/benchmark_cache.json were byte-identical
copies of the files in llmfit-core/data/ (the ones actually embedded via
include_str!), adding ~11 MB of redundant git-tracked data that could
silently drift. Scrapers now write only the canonical llmfit-core/data/
path; scripts and docs updated to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Request cardData in discovery listings instead of fetching each discovered model again. Keep comma-separated multi-license values in the data and split stored license strings when filtering.
Populate model license metadata from Hugging Face card data and license tags. Prefer named licenses when Hugging Face reports a generic other license, and preserve existing license and GGUF source metadata when a refresh response omits it.
- Add MiniMaxAI/MiniMax-M3 to the curated model list (512K context, 128K max output, image input)
- Keep MiniMaxAI/MiniMax-M2.7 as a previous-generation option
- Remove MiniMaxAI/MiniMax-M2.5 from the curated target list
- Update scripts/scrape_hf_models.py to match the new curated order
Co-Authored-By: Octopus <liyuan851277048@icloud.com>
* feat: rescrape model catalog and expand Ollama mappings
- Full rescrape of hf_models.json (1852 -> 5341 models)
- Added 9 curated models to TARGET_MODELS: Devstral, Cohere Command A/R+,
Falcon3-3B, Nemotron-Super-49B, EXAONE-Deep-2.4B/32B
- Added 21 Ollama mappings for Mistral, Qwen, DeepSeek, Falcon, Cohere,
SmolLM2, NVIDIA Nemotron, EXAONE Deep, and OLMo 2
- Preserved gguf_sources from previous data where the rescrape missed them
Supersedes #316.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Rust fit engine already had a precise KV cache formula and an
architecture-aware MoE speed estimator (Tier 1), but both were starved
for data — only 78/1520 models had the required fields, and hidden_size,
vocab_size, and moe_intermediate_size were never populated from the
scraper.
Scraper changes:
- Add extract_arch_metadata() helper that extracts num_hidden_layers,
num_attention_heads, num_key_value_heads, head_dim, hidden_size,
vocab_size, moe_intermediate_size, and shared_expert_intermediate_size
from config.json with text_config fallback for multimodal models
- Apply to both scrape_model() (curated) and _build_discovered_model()
(auto-discovered), fixing the gap where discovered models had no
architecture fields
- Fall back from moe_intermediate_size to intermediate_size for
Mixtral-style models that use a single key for per-expert FFN dim
- Add estimate_params_from_arch() to detect and correct wrong
parameters_raw from FP8/INT4/INT8 repos that report quantized element
counts instead of true parameter count (e.g. unsloth/Qwen3-Coder-Next-FP8
reported 2.1B instead of 79.7B)
- Handle list-type config values (e.g. ERNIE-4.5-VL)
Rust changes:
- Add hidden_size, vocab_size, moe_intermediate_size, and
shared_expert_intermediate_size to HfModelEntry deserialization
- Wire through to LlmModel instead of hardcoded None, enabling the
existing moe_bandwidth_decomposition() Tier 1 path
Results:
- Architecture metadata: 78/1520 (5%) → 1597/1852 (86%)
- MoE Tier 1 speed estimation: 0% → 353/356 (99%)
- Fixes wildly optimistic MoE speed estimates (e.g. Qwen3-Coder-Next-FP8
went from 1606 tok/s to 43 tok/s)
- Model count: 1520 → 1852 (10K min-downloads discovery)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The HuggingFace model scraper was silently broken — the `offset` parameter
is ignored by the HF API, so pagination only ever fetched the first page.
This switches to cursor-based pagination via the Link header, adds multiple
sort strategies (downloads, trendingScore, likes30d), makes the database
additive so historical models are never lost, and allows quantization/repack
orgs (bartowski, unsloth, etc.) since users actually run those variants.
Key changes:
- Fix pagination: use HF API cursor-based pagination (Link header)
- Multi-strategy discovery: downloads + trendingScore + likes30d, merged
- Additive database: existing models retained, fresh data updates in place
- Allow repack orgs: only trl-internal-testing is blocked now
- Config.json fallback: estimate params when safetensors metadata missing
- Discovery enabled by default with 1000 model limit
- Add feature-extraction pipeline for embedding models
- Filter statistics printed every run for transparency
Result: 1520 models (up from 1023), including 364 retained historical.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add AutoRound format detection to prevent misclassification as GGUF
- Add ModelFormat::Autoround enum variant in models.rs
- Update is_prequantized() to include AutoRound
- Fix bit-per-parameter calculation for AutoRound-4bit
- Update scrape_hf_models.py to detect auto-round quant_method
- Correct 3 misclassified AutoRound models in hf_models.json
- Add AutoRound-4bit to SUPPORTED_QUANTS and parse_quant() in plan.rs
- Update pre-quantization comments in fit.rs
- Update test_model_format_prequantized() assertion
Fixes#526
* style: apply cargo fmt formatting
* ci: use npx for web build/test steps to fix Windows PATH issue
* fix: add AutoRound-8bit to quant metrics in models.rs
Add explicit entries for AutoRound-8bit in quant_speed_multiplier,
quant_bytes_per_param, and quant_quality_penalty. Without these,
an 8-bit AutoRound model would incorrectly fall through to the
4-bit defaults (0.5 bpp, quality penalty -5.0).
* chore: Add Qwen3.5 Claude Opus 4.6 reasoning models
* feat: Support parallel models update
* fix: Fix cursor navigate in tui
* feat: Support parallel models enriching
* feat: Support marquee animation for model name column auto if text too long
* chore: Ignore __pycache__
* chore: Remove hf_models backup
* chore: Remove duplicate Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-GGUF in scrape hf models
Add four RWKV7 G1f series models from shoumenchougou GGUF repos.
These are pure RNN/SSM models with no KV cache, 8192 context,
and full llama.cpp compatibility. Architecture "rwkv" is already
handled in models.rs as pure SSM.
Closes#515
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add DeepSeek-V4-Pro, V4-Pro-Base, V4-Flash, and V4-Flash-Base to the
model database. Also fix MoE detection to handle the n_routed_experts
config key used by DeepSeek V4.
Closes#500
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Qwen3.6-27B (dense, 27.8B) and Qwen3.6-35B-A3B (MoE, 36B/3B active)
to the curated model list with GGUF sources from unsloth and ggml-org.
Add hybrid attention layout inference for Qwen3.5/3.6 architectures.
Full re-scrape with --discover -n 2000 brings total to 962 models with
422 GGUF-sourced entries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes#445
- Maverick num_experts was 16 (from generic llama4 fallback), now 128
matching the model name (17B-128E)
- Maverick context_length was 4096 (bare max_position_embeddings),
now 1048576 after applying RoPE scaling factor
- MODELS.md listed both Scout/Maverick as 128k; corrected to 10M/1M
- Scraper detect_moe() now checks text_config for nested MoE fields
(Llama 4 multimodal models store num_local_experts there)
- Scraper infer_context_length() now applies rope_scaling factor when
present, so models using RoPE extension get correct context values
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements all five phases of the roadmap in #417, requested in #411.
Phase 1 (#418): replace the linear KV approximation in models.rs with
a precise per layer formula
kv_bytes = 2 * n_layers * n_kv_heads * head_dim * ctx * dtype_bytes
when num_hidden_layers and head_dim are known on LlmModel. Falls back
to the old linear formula when metadata is missing so the existing
catalog doesn't regress. New fields num_hidden_layers, head_dim, and
attention_layout added to LlmModel.
Phase 1b (#418): scrape num_hidden_layers, num_attention_heads,
num_key_value_heads, head_dim, and hidden_size from each model's
config.json during llmfit update. Cache version bumped to 2 so old
caches re-fetch. Python scraper updated to write the same fields.
Phase 2 (#419): new KvQuant enum (fp16 / fp8 / q8_0 / q4_0 / tq) with
parse() and bytes_per_element(). New --kv-quant flag on plan, new
PlanRequest.kv_quant, new PlanEstimate.kv_quant, plumbed through
evaluate_current and build_path_estimate. CLI prints the active KV in
the plan header. JSON output exposes kv_quant and kv_alternatives with
snake_case labels matching the flag values.
Phase 3 (#420): new AttentionLayout { full, linear } struct with
compressible_fraction(). infer_attention_layout_from_name covers
Qwen3-Next, Jamba, Zamba, Mamba/RWKV; defaults to all-full for plain
dense transformers. effective_attention_layout prefers explicit
metadata, falls back to the heuristic.
Phase 4 (#421): TurboQuant variant gated to GpuBackend::Cuda in
estimate_model_plan; clear error message points at 0xSero/turboquant
and suggests fp8/q8_0/q4_0 for llama.cpp users. CLI prints a warning
on every --kv-quant tq invocation. KvQuantAlternative.note for the TQ
row always says "Experimental: not in upstream vLLM" plus the layer
split when known. The compression only applies to the full attention
slice; linear layers stay at fp16, so hybrid models see honest savings
(~21% on a 10/40 model, not the headline 83%).
Phase 5 (#422): new KvQuantAlternative rows on PlanEstimate showing
per-quant kv_cache_gb, memory_required_gb, savings_fraction, note,
and supported. Rendered as a "KV Cache Alternatives" table in the CLI
plan output and in the TUI plan view (highlighted active row, n/a
marker for unsupported options, per-row notes).
TUI: PlanField gains a KvQuant variant; new plan_kv_quant_input state,
input handling, and Tab cycle position; KV Quant input row in the
plan editor; Active KV line and KV Cache Alternatives section in the
rendered plan output.
Tests: 16 new unit tests covering KvQuant::parse, the precise formula
matching a hand calc on Llama-3.1-8B, KV scaling across fp16/q8/q4,
fallback when metadata is missing, TurboQuant only compressing full
attention layers, honest hybrid savings landing around 21% (not the
83% headline), AttentionLayout::compressible_fraction, name based
hybrid detection, plan errors on TQ + non-CUDA, plan succeeds on TQ
+ CUDA, and TQ row marked unsupported in alternatives off CUDA. All
301 workspace tests pass; no new clippy warnings.
Closes#418#419#420#421#422
Refs #417#411
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add MiniMaxAI/MiniMax-M2.7 to curated models list and manual entries
- Add M2.7 active parameter count for MoE calculations
- Add M2.7 entries to both data/hf_models.json and llmfit-core/data/hf_models.json
- M2.7 is placed before M2.5 as the latest flagship model
- All previous MiniMax models (M2, M2.1, M2.5) retained as alternatives
- test_gguf_source_deserialization — GgufSource JSON round-trips correctly
- test_gguf_sources_default_to_empty — models without gguf_sources in JSON default to []
- test_catalog_popular_models_have_gguf_sources — 5 well-known models (Llama-3.3-70B, Qwen2.5-7B, etc.)
have non-empty gguf_sources in the catalog
- test_catalog_gguf_sources_have_valid_repos — every gguf_source in the catalog has owner/repo format,
non-empty provider, and contains GGUF
- test_catalog_has_significant_gguf_coverage — at least 25% of catalog models have GGUF sources (currently
30%)
providers.rs (7 tests):
- test_hf_name_to_gguf_candidates_generates_common_patterns — heuristic generates bartowski, ggml-org,
TheBloke candidates
- test_hf_name_to_gguf_candidates_strips_owner — strips the Org/ prefix correctly
- test_lookup_gguf_repo_known_mappings — hardcoded mappings resolve for known models
- test_lookup_gguf_repo_unknown_returns_none — unknown models return None
- test_has_gguf_mapping_matches_known_models — boolean check works
- test_gguf_candidates_fallback_covers_major_providers — fallback covers all 3 providers and all end in
-GGUF
- test_gguf_candidates_known_mapping_returns_single — hardcoded mapping returns exactly 1 result
Signed-off-by: AlexsJones <alexsimonjones@gmail.com>
Add ModelFormat enum (Gguf/Awq/Gptq/Mlx/Safetensors) and Vllm inference
runtime. Pre-quantized models are detected from config.json by the scraper
and filtered to CUDA/ROCm only (no Apple Silicon support yet). Dynamic
re-quantization is skipped for fixed-precision formats.
Co-Authored-By: Claude <noreply@anthropic.com>
Incorporates PR #97 by @lichengzhe with merge conflicts resolved.
Adds: DeepSeek V3.2/Speciale, GLM-5, Kimi-K2.5, MiniMax-M2.5,
MiMo-V2-Flash, MiMo-7B-RL, Nemotron-3-Nano, Nemotron-Nano-9B-v2,
Phi-4-reasoning/mini/multimodal, EXAONE-4.0 (32B/1.2B), SmolLM3-3B,
Gemma-3n (E4B/E2B), Qwen3-Coder-Next.
Includes MoE configs, active parameter mappings, Ollama tag mappings,
and comprehensive fallback entries. Resolves conflicts with existing
Qwen 3.5 entries on main.
Co-authored-by: lichengzhe <lichengzhe@users.noreply.github.com>
Signed-off-by: Three Foxes (in a Trenchcoat) <threefoxes53235@gmail.com>
Signed-off-by: Three Foxes (in a Trenchcoat) <threefoxesyes3inatrenchcoat@gmail.com>
Three improvements to GGUF source enrichment:
1. TUI detail view: Added '── GGUF Downloads ──' section showing
providers with 📦 icons, HuggingFace URLs, and download command
hints. Appears between Memory and Notes sections.
2. Default enrichment: --gguf-sources is now enabled by default when
running the scraper. Use --no-gguf-sources to skip (faster scrape).
3. Persistent cache: GGUF source lookups are cached in
data/gguf_sources_cache.json with a 7-day TTL. Subsequent scrapes
hit the cache instead of the HuggingFace API (72 cache hits, 0 API
calls on second run). Cache file is .gitignored.
Trade-off for default enrichment: adds ~30s to first scrape (194 API
calls at 150ms rate limit), but subsequent runs are instant via cache.
The --no-gguf-sources flag preserves the fast path when needed.
Signed-off-by: Three Foxes (in a Trenchcoat) <threefoxesyes3inatrenchcoat@gmail.com>
Add a --gguf-sources flag to the scraper that checks known GGUF providers
(unsloth, bartowski) on HuggingFace for pre-quantized versions of each
model in the database.
Why not add Unsloth as a model provider?
Unsloth publishes repacks/quantizations of existing models, not original
architectures. Adding them as separate model entries would create duplicates
with identical RAM/VRAM requirements. Instead, we link existing models to
their GGUF download sources — this is more actionable for users who want
to actually download and run models locally.
Changes:
- scraper: --gguf-sources flag checks unsloth/ and bartowski/ repos
- models.rs: GgufSource struct + gguf_sources field on LlmModel
- display.rs: show GGUF Downloads section in model info with direct
HuggingFace links and llmfit download command hint
- JSON output: gguf_sources included in fit results
38 models enriched with GGUF sources across both providers.
Signed-off-by: Three Foxes (in a Trenchcoat) <threefoxesyes3inatrenchcoat@gmail.com>
