309 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Atream	036ae25a89	fix bf16 load, TODO: refactor cpu dequant	2025-02-23 15:37:09 +08:00
Atream	94ab2de3b9	Merge pull request #523 from miaooo0000OOOO/main ... optimize CMake multi core parallel	2025-02-22 17:38:18 +08:00
Atream	72d09f3f6e	Merge pull request #597 from kvcache-ai/feat-more-context ... Feat more context	2025-02-22 17:17:09 +08:00
Atream	f7f1059873	fix merge bug, this branch also padding Marlin	2025-02-22 09:00:09 +00:00
Atream	e90896314c	Merge pull request #577 from JiamingMai/dev ... Fix the link address in the doc install.md	2025-02-22 16:45:41 +08:00
Atream	954796123c	Merge pull request #582 from twobob/patch-1 ... Adjust the installation link to the correct section of docs	2025-02-22 16:44:48 +08:00
Atream	024009675e	Merge branch 'main' into feat-more-context	2025-02-22 06:17:39 +00:00
Atream	5ec33d046d	optimize gguf dequant, save mem, support Q2_K ... use marlin for lm_head, lm_head only calc last token for prefill extend context window to 19K for DeepSeek-V3/R1 within 24GB VRAM	2025-02-22 06:13:01 +00:00
_	5ed441a0f5	Update README.md	2025-02-21 14:15:50 +00:00
JiamingMai	45faddf668	fix the link addresses	2025-02-21 17:53:20 +08:00
Atream	7e1fe256c8	optimize GPU	2025-02-21 05:06:57 +00:00
Azure	25c5bddd08	Merge pull request #506 from makllama/musa ... feat: Support Moore Threads GPU	2025-02-20 22:50:31 +08:00
ZiWei Yuan	1dd84b4a5b	Merge pull request #550 from kvcache-ai/docker_dev ... Docker dev	2025-02-20 22:29:56 +08:00
liam	ed74f91a25	⚡ docker-iamge workflow	2025-02-20 22:27:51 +08:00
liam	a6aee1c54a	Merge branch 'main' of https://github.com/kvcache-ai/ktransformers into main	2025-02-20 22:00:55 +08:00
ZiWei Yuan	64a00651b1	Merge pull request #232 from HandSonic/patch-1 ... feat: add GitHub Actions workflow for building Docker image	2025-02-20 22:00:16 +08:00
liam	97e1dc97f6	⚡ fix .so bug	2025-02-20 21:24:46 +08:00
liam	574b13390b	⚡ fin docker-image	2025-02-20 18:57:47 +08:00
miaooo0000OOOO	0cdc446a19	默认并行编译	2025-02-20 11:09:16 +08:00
Xiaodong Ye	2207f6cd14	feat: Support Moore Threads GPU ... Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>	2025-02-19 18:26:55 +08:00
Atream	b4fb633991	Merge pull request #501 from Zhoneym/main ... Necessary tips for Node.js related issues	2025-02-19 16:46:48 +08:00
Zhoneym	c92a300bd8	Necessary tips for Node.js related issues	2025-02-19 16:37:18 +08:00
Atream	c9eb1f76f6	Merge pull request #384 from ubergarm/add-tutorial-notes ... Add notes to DeepSeek-R1 tutorial documentation	2025-02-19 13:46:49 +08:00
Atream	89f8218a2a	Merge pull request #487 from kvcache-ai/clean_pr ... clean PR code and disable flashinfer	2025-02-19 12:46:48 +08:00
Atream	a529518346	clean PR code and disable flashinfer	2025-02-19 04:42:47 +00:00
Atream	cf4da5fd47	Merge pull request #382 from ceerRep/server-prefix-cache ... fix server and add prefix cache for server	2025-02-19 12:33:36 +08:00
Azure	ea75849d03	Merge pull request #481 from Zhoneym/main ... Modify and add any incorrect or missing content in the `install.md`	2025-02-19 11:19:11 +08:00
Zhoneym	c72291832d	Update install.md	2025-02-19 10:39:43 +08:00
Zhoneym	3015e9dfa7	Modify and add any incorrect or missing content in the install.md file.	2025-02-19 10:01:16 +08:00
ceerrep	584c7d5639	fix: object type for non-streaming response	2025-02-18 23:50:28 +08:00
ceerrep	6d45871de8	fix: workaround return dummy usage	2025-02-18 22:39:49 +08:00
ZiWei Yuan	09f5c5e77e	Merge pull request #476 from kvcache-ai/V0.2.1.post1 ... 🔖 release v0.2.1.post1	2025-02-18 21:48:30 +08:00
liam	75d7d423b0	🔖 release v0.2.1.post1	2025-02-18 20:45:48 +08:00
Azure	a435e1bbcd	Merge pull request #406 from hoshinohikari/main ... Fix cmake error caused by lack of environment variables in Windows environment	2025-02-18 20:09:59 +08:00
ceerrep	db61375f1a	Merge remote-tracking branch 'kvai/main' into server-prefix-cache	2025-02-18 17:51:22 +08:00
Azure	19fd24f461	Merge pull request #412 from haojiubujian1985/delete-unused-link ... Update README_ZH.md	2025-02-18 14:58:10 +08:00
Atream	1c1769a579	Merge pull request #413 from kvcache-ai/fix_precision_MLA ... Fix precision mla	2025-02-18 14:44:50 +08:00
liam	592e13d453	⚡ add mmlu_pro test	2025-02-18 14:43:38 +08:00
Atream	1054ee3ae4	Merge pull request #450 from kvcache-ai/update-WeChatGroup ... Add files via upload	2025-02-18 14:34:07 +08:00
Atream	261d5d393f	Add files via upload	2025-02-18 14:28:29 +08:00
Azure	2671fc7f4b	Merge pull request #445 from dedfaf/main ... Fix typo in DeepseekR1_V3_tutorial.md Line 174	2025-02-18 14:23:29 +08:00
UnicornChan	ccf0f1910f	Merge pull request #442 from Zhoneym/patch-1 ... Fix `GLIBCXX_3.4.30' not found in Anaconda	2025-02-18 13:58:44 +08:00
liam	07a0555016	⚡ fix device and add test	2025-02-18 12:52:17 +08:00
ceerrep	73d072f609	Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache	2025-02-18 11:44:28 +08:00
ceerrep	2ffb43f974	fix: adapt prefix cache in forward_linux_flashinfer	2025-02-18 11:40:15 +08:00
Xie Weiyu	f029588b61	fix server warmup	2025-02-18 11:39:45 +08:00
Zhoneym	8b221f532a	Fix `GLIBCXX_3.4.30' not found in Anaconda	2025-02-18 11:32:42 +08:00
ceerrep	c70b6f4d5b	fix: use 'cuda:0' by default if torch_device is 'cuda'	2025-02-18 11:15:17 +08:00
dedfaf	3e5c5a2f4a	Fix typo in DeepseekR1_V3_tutorial.md Line 174	2025-02-18 10:57:12 +08:00
Xie Weiyu	9f1da18630	update install.sh	2025-02-17 20:50:19 +08:00