fairydreaming
2b63e0610b
llama : MTP support for DeepSeek V3.2 ( #26457 )
...
* llama : MTP support for DeepSeek V3.2
* model : no need to include MTP layers during DeepSeek V3.2 model type discovery
---------
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
2026-08-03 08:25:01 +02:00
fairydreaming
00f5442cc4
ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer ( #24231 )
...
* ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer
* ggml : remove scale parameters from lightning indexer OP, add f16 mask parameter
* tests : add GGML_OP_LIGHTNING_INDEXER tests
* ggml : bump RPC version
* chore : check if lightning indexer input tensors are not transposed
* tests : count flops instead of bandwidth in lightning indexer test
* chore : add missing const
* chore : whitespace
* ggml : renamed variables in CPU lightning indexer implementation
* ggml : fix lightning indexer mask broadcasting
* tests : tests for lightning indexer mask broadcasting
* chore : whitespace
* llama : use GGML_OP_LIGHTNING_INDEXER in DeepSeek V3.2 and DeepSeek V4 models
---------
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
2026-07-11 11:39:07 +02:00
Georgi Gerganov
7acb4e8cd2
hparams : refactor hparams.n_layer ( #24060 )
...
* hparams : refactor hparams.n_layer
* cont : remove `n_layer_kv()`, use n_layer_all instead
* cont : type consistency
* pi : update SYSTEM.md
* models : fix Step3.5 MTP
* cont : remove duplicate switch cases
* cont : explicitly set `false` to extra layers for `is_swa` and `is_recr`
* cont : fix nextn layer count handling
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2026-06-05 11:09:36 +03:00
fairydreaming
1f0aa2a696
model : support for DeepseekV32ForCausalLM with generic DeepSeek Sparse Attention (DSA) implementation ( #23346 )
...
* llama : support DeepSeek V3.2 model family (with DSA lightning indexer)
* convert : handle DeepseekV32ForCausalLM architecture
* ggml : support for f16 GGML_OP_FILL
* memory : separate hparams argument in llama_kv_cache constructor
* memory : add llama_kv_cache_dsa memory (KV cache + lightning indexer cache)
* llama : support for LLM_ARCH_DEEPSEEK32
* model : llama_model_deepseek32 implementation
* model : merge two scale operations into one in DSA lightning indexer implementation
* chore : remove unused code
* model : support NVFP4 in DeepSeek V3.2
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* memory : refactoring TODO
Co-authored-by: ggerganov <ggerganov@users.noreply.github.com>
---------
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: ggerganov <ggerganov@users.noreply.github.com>
2026-05-29 10:15:17 +02:00
