mirror of
https://github.com/Helldez/BigMoeOnEdge.git
synced 2026-10-03 03:25:42 +00:00
2d4f2b11db
Ling-3.0-flash (127B total, ~5B active, 512 routed experts) becomes one
registry row: standard split expert suffixes, biased top-k (the lfm2moe
pattern), a resident shared expert, leading dense blocks that never bind,
and a NextN/MTP block llama.cpp does not load by default. The hybrid
KDA/MLA attention stack is dense-side llama.cpp machinery, invisible to
the streaming seam.
The submodule moves to fork branch bmoe/expert-ready-hook-ling3: the same
single expert-ready-hook commit, cherry-picked clean onto upstream 3733366
("model : BailingMoE3 Support"). The old fork branch stays, so the commit
the previous pin names remains reachable. The jump crosses 496 upstream
commits and forces three adaptations, all on our side of the seam:
- llama_model_params lost use_mmap for the load_mode enum; the streamer's
required layout is now pinned with LLAMA_LOAD_MODE_MMAP.
- The nextn/MTP tensors are now skipped at load unless load_mtp is set,
and it defaults to off, so --mtp would have built its second context
over a block whose tensors were never loaded. The block is now requested
exactly when speculation asks for it.
- gpt-oss/harmony now declares its thinking tags, so the think-control
probe decides by mechanism: a prefill that moves past the closed span
into further structure of the format itself is binding; one that just
ends at the closing tag is only a suggestion (LFM2.5 stays reported as
uncontrollable).
Byte-identity gates pass on the new base (qwen3moe, gemma4, 4-shard
split). PC smoke on Qwen3.6-35B-A3B matches the recorded baseline on
every applicable cell, --mtp included (17/19 drafts accepted, 3.43 tokens
per verify decode). On device, Ling-3.0-flash loads and streams over its
512 experts; --mtp on it stops at draft rollback because upstream has no
recurrent-state rollback for bailingmoe3 yet (llm_arch_supports_rs_rollback),
a llama.cpp limitation, not a seam one.
App version 0.20.0 (versionCode 35).
|
||
|---|---|---|
| .. | ||
| chat_parse_test.cpp | ||
| CMakeLists.txt | ||
| config_test.cpp | ||
| moe_gates.cpp | ||
| ngram_test.cpp | ||
| think_control_test.cpp | ||