mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-08-25 08:14:24 +00:00
9cd719af21
* dspark: support speculators-format checkpoints (SpecForge exports) Speculators-format DSpark drafts (e.g. SpecForge exports for the Gemma-4-26B-A4B target) differ from the dense DeepSpec checkpoints in three ways: - the config nests the backbone hparams under transformer_layer_config and gives the extract layers as aux_hidden_state_layer_ids - the block is the DFlash 1+N fill-in layout: the anchor slot is a bonus token, not a prediction slot. Written as dflash.bonus_anchor; such drafts build the block and read the mask positions exactly like DFlash (n_max drafts from a 1+n_max block), only the Markov/confidence sampling comes from DSpark - the draft output vocab may be reduced (draft_vocab_size < vocab_size) with a d2t remap table. The converter expands lm_head/markov_w2 back to the full vocab and synthesizes an lm_head bias of -1e9 on the rows the draft cannot produce, so the runtime needs no d2t remapping. Such drafts ship their own (now optional) token_embd/output tensors instead of sharing the target's Verified against gemma4-26b-a4b-dspark: greedy outputs are byte-identical with and without the draft; acceptance 0.46, mean draft len 3.7 (n_max 6). Co-authored-by: desovo7 <942845546@qq.com> Assisted-by: Claude Fable 5 * dspark: fold the speculators draft class into DSparkModel One class now covers every DSpark variant. What used to pick the class is a single flag, because the arch name turns out to be the only thing that separates the two families: SpecForge also exports a flat schema that carries no speculators_* fields yet still uses the 1+N bonus-anchor block, so keying on those fields would silently mis-read its drafts. Also rename i0 to i_first_pred in the draft read loop and the Markov head, and give the head a real bonus_anchor bool instead of testing i0 > 0. Converting the Qwen3-8B DeepSpec draft and both gemma-4 speculators drafts produces byte-identical GGUFs. The one behaviour change is that the markov_head_type check now also covers the DeepSpec checkpoints, which previously skipped it. Co-authored-by: desovo7 <942845546@qq.com> Assisted-by: Claude Opus 5 * dspark: address review comments - rename bonus_anchor to sample_from_anchor (GGUF key and code), matching the checkpoint config field; absent key still means anchor-first - rework the reduced draft vocab to match EAGLE3: d2t is written as I64 absolute target ids and the logits are scattered at runtime, instead of expanding lm_head/markov_w2 and synthesizing an output bias at conversion - move the t2d skip to modify_tensors, like EAGLE3 - drop _is_specforge: the arch name only picks the sample_from_anchor default, embed/lm_head sharing is decided by the draft vocab size - deduplicate the tok_embd create_tensor left behind by the rebase Verified with the RedHat gemma-4-31b speculator draft: greedy output is byte-identical with and without the draft; acceptance 0.26 (n_max 7). Co-authored-by: desovo7 <942845546@qq.com> Assisted-by: Claude Fable 5 * dspark: fold the sample_from_anchor read into the block_size block * dspark: fix flake8 continuation indent * clean up * dspark: key the sample_from_anchor default off the export format Co-authored-by: desovo7 <942845546@qq.com> Assisted-by: Claude Fable * dspark: drop t2d in filter_tensors Co-authored-by: desovo7 <942845546@qq.com> Assisted-by: Claude Fable * dspark: map model.lm_head instead of bypassing the dflash prefix Co-authored-by: desovo7 <942845546@qq.com> Assisted-by: Claude Fable --------- Co-authored-by: desovo7 <942845546@qq.com> Co-authored-by: ruixiang63 <wangruixiang07@outlook.com> |
||
|---|---|---|
| .. | ||
| android | ||
| backend | ||
| development | ||
| multimodal | ||
| ops | ||
| android.md | ||
| autoparser.md | ||
| build-riscv64-spacemit.md | ||
| build-s390x.md | ||
| build.md | ||
| completions.md | ||
| docker.md | ||
| function-calling.md | ||
| install.md | ||
| llguidance.md | ||
| models.md | ||
| multi-gpu.md | ||
| multimodal.md | ||
| ops.md | ||
| preset.md | ||
| release.md | ||
| speculative.md | ||
| xcframework.md | ||