vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-09 17:34:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
koboldcpp/tools
History
Bartowski 18443257a3
server: add ctx-per-slot (--kv-unified-per-slot) (#24124) 
* Add ctx-per-slot argument for unifid KV cache

* Swap out ctx fractions for ctx pool slots

* Formatting cleanup

* Remove ctx-pool-slots, make ctx-per-slot an int

* refactor it

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-08-27 22:39:14 +02:00
..
batched-bench cmake : add install() for impl libraries + fix apple builds (#23511) 2026-05-22 11:46:26 +03:00
cli llama: model_loader: add TENSOR_READ_LAZY (#27794) 2026-08-27 15:14:34 +02:00
completion llama: model_loader: add TENSOR_READ_LAZY (#27794) 2026-08-27 15:14:34 +02:00
cvector-generator cmake : introduce semantic versioning (#26839) 2026-08-12 14:15:03 +02:00
export-lora docs: fix export-lora --lora-scaled syntax [no release] (#24703) 2026-06-18 16:46:17 +02:00
fit-params fit: also take into account n_streams (#27496) 2026-08-22 16:16:06 +02:00
gguf-split cmake : introduce semantic versioning (#26839) 2026-08-12 14:15:03 +02:00
imatrix imatrix.cpp: Move finite check and only check touched experts (#26861) 2026-08-11 11:18:19 -04:00
llama-bench llama : add --n-cpu-ffn option (#26622) 2026-08-27 11:26:42 +02:00
mtmd args: add --video-* CLI arguments (#24318) 2026-08-27 12:11:12 +02:00
perplexity quant : Optimise memory usage by evicting weights after processing each layer (#22877) 2026-08-18 16:22:32 +02:00
quantize quantize: cap working memory size to avoid loading big tensors onto RAM (#27795) 2026-08-27 18:31:13 +02:00
results libs : rename libcommon -> libllama-common (#21936) 2026-04-17 11:11:46 +03:00
rpc rpc: support apple RDMA as an RPC transport (#26421) 2026-08-25 20:12:15 +03:00
server server: add ctx-per-slot (--kv-unified-per-slot) (#24124) 2026-08-27 22:39:14 +02:00
tokenize tokenize : drop --stdin mutual-exclusion check (#25672) 2026-07-15 18:41:51 +02:00
tts args: add --video-* CLI arguments (#24318) 2026-08-27 12:11:12 +02:00
tuning metal : per-device tuned (Q, NE) for flash-attn vec (#26570) 2026-08-24 19:22:27 +03:00
ui ui: Improve Chat Form Actions UI/UX (models selector, add panel) (#27746) 2026-08-27 14:47:36 +02:00
CMakeLists.txt metal : per-device tuned (Q, NE) for flash-attn vec (#26570) 2026-08-24 19:22:27 +03:00