vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-04-30 20:50:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

memory : remove KV cache size padding (#16812)

Browse source 
* memory : remove KV cache size padding

* cont : restore padding for n_kv tensor shape

* server : use slot context size instead of training context size

* server : simplify context limit logic
This commit is contained in:
Georgi Gerganov 2025-10-28 20:19:44 +02:00 • committed by GitHub
parent a8ca18b4b8
commit 85a7d8677b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 14 additions and 54 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tools/server/tests/unit/test_ctx_shift.py
View file

@ -45,7 +45,7 @@ def test_ctx_shift_enabled():
@pytest.mark.parametrize("n_predict,n_token_output,truncated", [
(64, 64, False),
(-1, 120, True),
(-1, 248, True), # 8 tokens prompt + 248 tokens generated = 256 tokens total
])
def test_ctx_shift_disabled_short_prompt(n_predict: int, n_token_output: int, truncated: bool):
global server