vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-05-02 13:41:15 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

add prefix cache support for kvc2.

Browse source
This commit is contained in:
ouqingliang 2025-06-26 04:57:25 +00:00
parent b154441072
commit 3b4a1c7532
9 changed files with 399 additions and 52 deletions
Download patch file Download diff file Expand all files Collapse all files

2
ktransformers/configs/config.yaml
View file

@ -67,6 +67,6 @@ attn:
page_size: 256
chunk_size: 256
kvc2:
gpu_only: true
gpu_only: false
utilization_percentage: 1.0
cpu_memory_size_GB: 500