254 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
ceerrep	ee24eb8dc3	fix: fix server for triton kernel	2025-02-17 18:08:45 +08:00
ceerrep	bb1cadfff3	Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache	2025-02-17 18:08:04 +08:00
Atream	038bc30888	fix precision bug imported by position_ids in 0.2.0	2025-02-17 09:23:14 +00:00
ceerrep	cd9f7f8f34	fix: server: drop <think> tag in chat template	2025-02-17 14:25:27 +08:00
ceerrep	ca2090d89b	feat: use model name in openai endpoint	2025-02-17 00:27:32 +08:00
ceerrep	5ac266085e	fix: use flash_attn for faster prefill	2025-02-17 00:11:24 +08:00
ceerrep	bb0ccc7b1a	feat: add prefix cache for server	2025-02-17 00:10:55 +08:00
wang jiahao	c515cc49a5	Merge pull request #355 from kvcache-ai/qiyuxinlin-patch-1 ... Update install.md	2025-02-16 15:05:43 +08:00
wang jiahao	24de607be4	Merge pull request #357 from kvcache-ai/qiyuxinlin-patch-2 ... Update SUMMARY.md	2025-02-16 15:05:19 +08:00
Atream	b84524622e	support bf16 read	2025-02-16 06:43:27 +00:00
wang jiahao	76554dd674	Update SUMMARY.md	2025-02-16 14:30:22 +08:00
wang jiahao	56d19d6115	Update install.md	2025-02-16 14:23:56 +08:00
Azure	9f9c3738f9	Merge pull request #354 from Azure-Tang/fix-mockTritonMLA ... [fix] Mock triton mla due to precision issue	2025-02-16 14:06:54 +08:00
Azure	ff6b265e53	Mock triton mla due to precision issue	2025-02-16 06:03:12 +00:00
Atream	c5f036e8a4	Merge pull request #333 from kvcache-ai/feat_experts_gpu ... toy support for experts on GPU, no CUDA Graph	2025-02-15 23:30:24 +08:00
Atream	8ed8eb2a9e	Update FAQ.md	2025-02-15 23:27:35 +08:00
Atream	c189d55bd1	toy support for experts on GPU, no CUDA Graph	2025-02-15 15:16:00 +00:00
wang jiahao	ae8da019c1	Merge pull request #330 from hrz6976/fix-nonetype ... thanks..., I was about to submit and found that you had already modified it. Thank you for your contribution	2025-02-15 22:50:51 +08:00
Azure	56382aa869	Merge pull request #290 from ZhangShuaiyi/dev/check_gguf_path ... ensure that gguf_path argument is a directory.	2025-02-15 22:48:57 +08:00
Azure	c8bf250104	Merge pull request #324 from BiFangKNT/patch-2 ... Update DeepseekR1_V3_tutorial.md	2025-02-15 22:23:09 +08:00
12f23eddde	4516282ccc	Fix NoneType object has no attribute zero_	2025-02-15 22:04:45 +08:00
ZiWei Yuan	3c6035aa8a	Merge pull request #329 from kvcache-ai/fix_doc ... 📝 fix typo	2025-02-15 21:45:19 +08:00
liam	69b00753c2	📝 fix typo	2025-02-15 21:44:40 +08:00
彼方	bd693b698f	Update DeepseekR1_V3_tutorial.md	2025-02-15 19:43:18 +08:00
UnicornChan	65d73ea3f8	Merge pull request #317 from kvcache-ai/develop-0.2.1 ... [feature] update docker image and entrypoint	2025-02-15 16:19:18 +08:00
ZiWei Yuan	718a71b3a9	Merge pull request #316 from KMSorSMS/main ... 📝 update V0.2.1 Doc	2025-02-15 16:19:00 +08:00
liam	13382f88ab	📝 update V0.2.1 Doc	2025-02-15 16:17:05 +08:00
chenxl	0e4b7a3929	[feature] update docker image and entrypoint	2025-02-15 07:55:33 +00:00
Atream	f9f9f746c0	Merge pull request #315 from kvcache-ai/Atream-add-adapted ... Atream add adapted	2025-02-15 15:45:34 +08:00
Atream	92399283b6	Update attention.py	2025-02-15 15:43:35 +08:00
Atream	d90749d35d	Update triton_attention.py	2025-02-15 15:41:01 +08:00
Shuaiyi	22280bf17f	get dirname if gguf_path is a file	2025-02-15 07:08:22 +00:00
Azure	1548c99234	Merge pull request #307 from Azure-Tang/main ... [update] Reorganize documentation/README	2025-02-15 10:06:56 +08:00
Azure	227e81b0d3	update zh readme	2025-02-15 01:54:01 +00:00
Azure	ef89b1520b	* Reorganize documentation/README ... * Consolidate the installation section, as it's currently too cluttered * Move the Multi-GPU section to the top-level structure * Add a **detailed** tutorial on registering extra GPU memory with Marlin	2025-02-14 19:58:26 +00:00
Azure	b0b90270d8	Merge pull request #306 from kvcache-ai/revert-305-main ... Revert "[update] Reorganize documentation/README"	2025-02-15 03:44:42 +08:00
Azure	4f4ed36442	Revert "[update] Reorganize documentation/README"	2025-02-15 03:43:48 +08:00
Azure	19d4a50b1c	[update] Reorganize documentation/README ... [update] Reorganize documentation/README	2025-02-15 03:41:43 +08:00
Azure	483182fc3a	fix typo and detail	2025-02-14 19:40:15 +00:00
Azure	823b25eec9	Reorganize documentation/README	2025-02-14 19:08:17 +00:00
Atream	cc8d627e32	Merge pull request #301 from kvcache-ai/fix-cuda-graph-bug ... warm_up before capture	2025-02-14 23:54:43 +08:00
Atream	1946493f2d	warm_up before capture	2025-02-14 15:52:21 +00:00
Atream	cadd55078f	Merge pull request #295 from kvcache-ai/update-wechatgroup ... Update wechatgroup	2025-02-14 19:52:05 +08:00
Atream	e153d78227	Add files via upload	2025-02-14 19:51:11 +08:00
Atream	96e6dff7ac	Delete WeChatGrouop.png	2025-02-14 19:49:14 +08:00
Atream	885a91e7db	Merge pull request #294 from kvcache-ai/feat-fast-MLA ... Feat fast mla	2025-02-14 19:40:36 +08:00
Atream	1084d4e4b4	linux support triton MLA kernel	2025-02-14 11:38:55 +00:00
Azure	6738908699	Merge pull request #280 from Azure-Tang/main ... [fix] Fix incorrect image content in the document	2025-02-14 17:12:14 +08:00
Azure	1b1f417267	Fix incorrect image content in the document	2025-02-14 09:04:22 +00:00
Azure	f4bb374eaf	Merge pull request #254 from Azure-Tang/main ... [Update] Add V3/R1 8 gpu yaml example	2025-02-14 11:54:14 +08:00