877 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Atream	261d5d393f	Add files via upload	2025-02-18 14:28:29 +08:00
Azure	2671fc7f4b	Merge pull request #445 from dedfaf/main ... Fix typo in DeepseekR1_V3_tutorial.md Line 174	2025-02-18 14:23:29 +08:00
UnicornChan	ccf0f1910f	Merge pull request #442 from Zhoneym/patch-1 ... Fix `GLIBCXX_3.4.30' not found in Anaconda	2025-02-18 13:58:44 +08:00
liam	07a0555016	⚡ fix device and add test	2025-02-18 12:52:17 +08:00
ceerrep	73d072f609	Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache	2025-02-18 11:44:28 +08:00
ceerrep	2ffb43f974	fix: adapt prefix cache in forward_linux_flashinfer	2025-02-18 11:40:15 +08:00
Xie Weiyu	f029588b61	fix server warmup	2025-02-18 11:39:45 +08:00
Zhoneym	8b221f532a	Fix `GLIBCXX_3.4.30' not found in Anaconda	2025-02-18 11:32:42 +08:00
ceerrep	c70b6f4d5b	fix: use 'cuda:0' by default if torch_device is 'cuda'	2025-02-18 11:15:17 +08:00
dedfaf	3e5c5a2f4a	Fix typo in DeepseekR1_V3_tutorial.md Line 174	2025-02-18 10:57:12 +08:00
Xie Weiyu	9f1da18630	update install.sh	2025-02-17 20:50:19 +08:00
Xie Weiyu	c176e516b5	server mix mla	2025-02-17 20:40:28 +08:00
ceerrep	ee24eb8dc3	fix: fix server for triton kernel	2025-02-17 18:08:45 +08:00
ceerrep	bb1cadfff3	Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache	2025-02-17 18:08:04 +08:00
Atream	038bc30888	fix precision bug imported by position_ids in 0.2.0	2025-02-17 09:23:14 +00:00
xubo	49c6e2fc04	Update README_ZH.md	2025-02-17 16:56:44 +08:00
hoshinohikari	6d9182795c	Fix cmake error caused by lack of environment variables in Windows environment	2025-02-17 15:56:56 +08:00
ceerrep	cd9f7f8f34	fix: server: drop <think> tag in chat template	2025-02-17 14:25:27 +08:00
John W. Leimgruber III	bb39eeb005	Add notes to DeepSeek-R1 tutorial documentation	2025-02-16 13:40:27 -05:00
ceerrep	ca2090d89b	feat: use model name in openai endpoint	2025-02-17 00:27:32 +08:00
ceerrep	5ac266085e	fix: use flash_attn for faster prefill	2025-02-17 00:11:24 +08:00
ceerrep	bb0ccc7b1a	feat: add prefix cache for server	2025-02-17 00:10:55 +08:00
wang jiahao	c515cc49a5	Merge pull request #355 from kvcache-ai/qiyuxinlin-patch-1 ... Update install.md	2025-02-16 15:05:43 +08:00
wang jiahao	24de607be4	Merge pull request #357 from kvcache-ai/qiyuxinlin-patch-2 ... Update SUMMARY.md	2025-02-16 15:05:19 +08:00
Atream	b84524622e	support bf16 read	2025-02-16 06:43:27 +00:00
wang jiahao	76554dd674	Update SUMMARY.md	2025-02-16 14:30:22 +08:00
wang jiahao	56d19d6115	Update install.md	2025-02-16 14:23:56 +08:00
Azure	9f9c3738f9	Merge pull request #354 from Azure-Tang/fix-mockTritonMLA ... [fix] Mock triton mla due to precision issue	2025-02-16 14:06:54 +08:00
Azure	ff6b265e53	Mock triton mla due to precision issue	2025-02-16 06:03:12 +00:00
Atream	c5f036e8a4	Merge pull request #333 from kvcache-ai/feat_experts_gpu ... toy support for experts on GPU, no CUDA Graph	2025-02-15 23:30:24 +08:00
Atream	8ed8eb2a9e	Update FAQ.md	2025-02-15 23:27:35 +08:00
Atream	c189d55bd1	toy support for experts on GPU, no CUDA Graph	2025-02-15 15:16:00 +00:00
wang jiahao	ae8da019c1	Merge pull request #330 from hrz6976/fix-nonetype ... thanks..., I was about to submit and found that you had already modified it. Thank you for your contribution	2025-02-15 22:50:51 +08:00
Azure	56382aa869	Merge pull request #290 from ZhangShuaiyi/dev/check_gguf_path ... ensure that gguf_path argument is a directory.	2025-02-15 22:48:57 +08:00
Azure	c8bf250104	Merge pull request #324 from BiFangKNT/patch-2 ... Update DeepseekR1_V3_tutorial.md	2025-02-15 22:23:09 +08:00
12f23eddde	4516282ccc	Fix NoneType object has no attribute zero_	2025-02-15 22:04:45 +08:00
MuWinds	d3b45d5704	Merge branch 'kvcache-ai:main' into main	2025-02-15 21:57:08 +08:00
ZiWei Yuan	3c6035aa8a	Merge pull request #329 from kvcache-ai/fix_doc ... 📝 fix typo	2025-02-15 21:45:19 +08:00
liam	69b00753c2	📝 fix typo	2025-02-15 21:44:40 +08:00
彼方	bd693b698f	Update DeepseekR1_V3_tutorial.md	2025-02-15 19:43:18 +08:00
UnicornChan	65d73ea3f8	Merge pull request #317 from kvcache-ai/develop-0.2.1 ... [feature] update docker image and entrypoint	2025-02-15 16:19:18 +08:00
ZiWei Yuan	718a71b3a9	Merge pull request #316 from KMSorSMS/main ... 📝 update V0.2.1 Doc	2025-02-15 16:19:00 +08:00
liam	13382f88ab	📝 update V0.2.1 Doc	2025-02-15 16:17:05 +08:00
chenxl	0e4b7a3929	[feature] update docker image and entrypoint	2025-02-15 07:55:33 +00:00
Atream	f9f9f746c0	Merge pull request #315 from kvcache-ai/Atream-add-adapted ... Atream add adapted	2025-02-15 15:45:34 +08:00
Atream	92399283b6	Update attention.py	2025-02-15 15:43:35 +08:00
Atream	d90749d35d	Update triton_attention.py	2025-02-15 15:41:01 +08:00
Shuaiyi	22280bf17f	get dirname if gguf_path is a file	2025-02-15 07:08:22 +00:00
MuWinds	f74c2d1d17	Solve torch.backends.cuda.sdp_kernel() is deprecated.	2025-02-15 12:41:51 +08:00
Azure	1548c99234	Merge pull request #307 from Azure-Tang/main ... [update] Reorganize documentation/README	2025-02-15 10:06:56 +08:00