vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-05 12:09:48 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
698 commits 40 branches 19 tags 46 MiB
Commit graph

22 commits

Author SHA1 Message Date
djw
3f9bbf1181 support qwen3, dont speak human language 2025-04-28 08:44:47 +00:00
qiyuxinlin
f5287e908a fix no balance_serve import error 2025-04-22 02:11:18 +00:00
dongjw
be84d04253 Fix bug with non-base-multiple chunk_size, update test examples, and resolve issue with writing model_config. Hugging Face URL input is still unsupported. 2025-04-04 15:41:07 +08:00
Atream
e36ddc36a8
Update modeling_deepseek_v3.py 2025-04-03 17:13:06 +08:00
Atream
25cee5810e add balance-serve, support concurrence 2025-03-31 22:55:32 +08:00
Atream
5ec33d046d optimize gguf dequant, save mem, support Q2_K 
use marlin for lm_head, lm_head only calc last token for prefill
extend context window to 19K for DeepSeek-V3/R1 within 24GB VRAM
 2025-02-22 06:13:01 +00:00
ceerrep
bb1cadfff3 Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache 2025-02-17 18:08:04 +08:00
Atream
038bc30888 fix precision bug imported by position_ids in 0.2.0 2025-02-17 09:23:14 +00:00
ceerrep
bb0ccc7b1a feat: add prefix cache for server 2025-02-17 00:10:55 +08:00
12f23eddde
4516282ccc Fix NoneType object has no attribute zero_ 2025-02-15 22:04:45 +08:00
Atream
885a91e7db
Merge pull request #294 from kvcache-ai/feat-fast-MLA 
Feat fast mla
 2025-02-14 19:40:36 +08:00
Atream
1084d4e4b4 linux support triton MLA kernel 2025-02-14 11:38:55 +00:00
Atream
bb35dc5b0d init support for MLA using Attention kernel 2025-02-13 15:01:14 +00:00
Azure
3897f001f5 update FAQ 2025-02-12 08:50:58 +00:00
Azure
907251c743 done support deepseekv3 2025-02-04 15:53:38 +00:00
Azure
f748cd29f0 fix rope; update moegate 2025-02-01 18:05:45 +00:00
Azure
f873558a89 update rope calculation; update modeling.py; update gate for moe 2025-02-01 07:32:21 +00:00
Azure
5a50b34627 fix hard coding caused by rope dim calculation, load from config now 2025-01-31 15:25:50 +00:00
Azure
476b1d8dc6 support deepseekv3; runable but have precition problem 2025-01-31 08:27:24 +00:00
chenxl
4d1d561d28 [feature] release 0.1.3 2024-08-28 16:11:43 +00:00
chenxl
f5f79f5c0e [ADD] support multi-gpu qlen>1 q5_k 2024-08-12 11:41:26 +00:00
chenxl
18c42e67df Initial commit 2024-07-27 16:06:58 +08:00