322 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
lazymio	07eb712a73	Left out	2025-02-24 21:51:14 +08:00
lazymio	91062a834f	Default values	2025-02-24 21:38:01 +08:00
lazymio	76487c4dcb	Revert repetition_penalty as it is not in API spec	2025-02-24 21:30:03 +08:00
lazymio	05ad288453	Also /chat/completions	2025-02-24 21:08:36 +08:00
lazymio	bf36547f98	Also allow repetition_penalty	2025-02-24 21:07:35 +08:00
lazymio	8704c09192	Allow temperature and top_p from requests	2025-02-24 21:01:33 +08:00
Atream	4b5991e77e	Merge pull request #638 from kvcache-ai/feat-moonlight ... fix KExpertsMarlin on GPU with out CUDA Graph	2025-02-24 17:32:05 +08:00
Atream	f327695079	fix KExpertsMarlin on GPU with out CUDA Graph	2025-02-24 09:30:54 +00:00
Atream	eb039b723d	Merge pull request #621 from kvcache-ai/feat-moonlight ... support moonlight, use ktransformers/optimize/optimize_rules/Moonlight-16B-A3B.yaml	2025-02-23 22:39:08 +08:00
Atream	f5f6c6b95d	update yaml	2025-02-23 14:33:58 +00:00
Atream	e8e02e5ccc	support Moonlight	2025-02-23 14:21:18 +00:00
DDong Jianwei	95d937c51d	tmp	2025-02-23 18:51:42 +08:00
Atream	cdb6f896bb	Merge pull request #612 from kvcache-ai/fix-bf16-load ... fix bf16 load, TODO: refactor cpu dequant	2025-02-23 15:37:23 +08:00
Atream	036ae25a89	fix bf16 load, TODO: refactor cpu dequant	2025-02-23 15:37:09 +08:00
Atream	94ab2de3b9	Merge pull request #523 from miaooo0000OOOO/main ... optimize CMake multi core parallel	2025-02-22 17:38:18 +08:00
Atream	72d09f3f6e	Merge pull request #597 from kvcache-ai/feat-more-context ... Feat more context	2025-02-22 17:17:09 +08:00
Atream	f7f1059873	fix merge bug, this branch also padding Marlin	2025-02-22 09:00:09 +00:00
Atream	e90896314c	Merge pull request #577 from JiamingMai/dev ... Fix the link address in the doc install.md	2025-02-22 16:45:41 +08:00
Atream	954796123c	Merge pull request #582 from twobob/patch-1 ... Adjust the installation link to the correct section of docs	2025-02-22 16:44:48 +08:00
Atream	024009675e	Merge branch 'main' into feat-more-context	2025-02-22 06:17:39 +00:00
Atream	5ec33d046d	optimize gguf dequant, save mem, support Q2_K ... use marlin for lm_head, lm_head only calc last token for prefill extend context window to 19K for DeepSeek-V3/R1 within 24GB VRAM	2025-02-22 06:13:01 +00:00
_	5ed441a0f5	Update README.md	2025-02-21 14:15:50 +00:00
JiamingMai	45faddf668	fix the link addresses	2025-02-21 17:53:20 +08:00
Atream	7e1fe256c8	optimize GPU	2025-02-21 05:06:57 +00:00
Azure	25c5bddd08	Merge pull request #506 from makllama/musa ... feat: Support Moore Threads GPU	2025-02-20 22:50:31 +08:00
ZiWei Yuan	1dd84b4a5b	Merge pull request #550 from kvcache-ai/docker_dev ... Docker dev	2025-02-20 22:29:56 +08:00
liam	ed74f91a25	⚡ docker-iamge workflow	2025-02-20 22:27:51 +08:00
liam	a6aee1c54a	Merge branch 'main' of https://github.com/kvcache-ai/ktransformers into main	2025-02-20 22:00:55 +08:00
ZiWei Yuan	64a00651b1	Merge pull request #232 from HandSonic/patch-1 ... feat: add GitHub Actions workflow for building Docker image	2025-02-20 22:00:16 +08:00
liam	97e1dc97f6	⚡ fix .so bug	2025-02-20 21:24:46 +08:00
liam	574b13390b	⚡ fin docker-image	2025-02-20 18:57:47 +08:00
miaooo0000OOOO	0cdc446a19	默认并行编译	2025-02-20 11:09:16 +08:00
Xiaodong Ye	2207f6cd14	feat: Support Moore Threads GPU ... Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>	2025-02-19 18:26:55 +08:00
Atream	b4fb633991	Merge pull request #501 from Zhoneym/main ... Necessary tips for Node.js related issues	2025-02-19 16:46:48 +08:00
Zhoneym	c92a300bd8	Necessary tips for Node.js related issues	2025-02-19 16:37:18 +08:00
Atream	c9eb1f76f6	Merge pull request #384 from ubergarm/add-tutorial-notes ... Add notes to DeepSeek-R1 tutorial documentation	2025-02-19 13:46:49 +08:00
Atream	89f8218a2a	Merge pull request #487 from kvcache-ai/clean_pr ... clean PR code and disable flashinfer	2025-02-19 12:46:48 +08:00
Atream	a529518346	clean PR code and disable flashinfer	2025-02-19 04:42:47 +00:00
Atream	cf4da5fd47	Merge pull request #382 from ceerRep/server-prefix-cache ... fix server and add prefix cache for server	2025-02-19 12:33:36 +08:00
Azure	ea75849d03	Merge pull request #481 from Zhoneym/main ... Modify and add any incorrect or missing content in the `install.md`	2025-02-19 11:19:11 +08:00
Zhoneym	c72291832d	Update install.md	2025-02-19 10:39:43 +08:00
Zhoneym	3015e9dfa7	Modify and add any incorrect or missing content in the install.md file.	2025-02-19 10:01:16 +08:00
ceerrep	584c7d5639	fix: object type for non-streaming response	2025-02-18 23:50:28 +08:00
ceerrep	6d45871de8	fix: workaround return dummy usage	2025-02-18 22:39:49 +08:00
ZiWei Yuan	09f5c5e77e	Merge pull request #476 from kvcache-ai/V0.2.1.post1 ... 🔖 release v0.2.1.post1	2025-02-18 21:48:30 +08:00
liam	75d7d423b0	🔖 release v0.2.1.post1	2025-02-18 20:45:48 +08:00
Azure	a435e1bbcd	Merge pull request #406 from hoshinohikari/main ... Fix cmake error caused by lack of environment variables in Windows environment	2025-02-18 20:09:59 +08:00
ceerrep	db61375f1a	Merge remote-tracking branch 'kvai/main' into server-prefix-cache	2025-02-18 17:51:22 +08:00
Azure	19fd24f461	Merge pull request #412 from haojiubujian1985/delete-unused-link ... Update README_ZH.md	2025-02-18 14:58:10 +08:00
Atream	1c1769a579	Merge pull request #413 from kvcache-ai/fix_precision_MLA ... Fix precision mla	2025-02-18 14:44:50 +08:00