323 commits

Author	SHA1	Message	Date
liam	c74453d8ca	📝 add doc support and fix bug in qwen2	2025-02-13 16:37:43 +08:00
MorphisZhang	aea4243712	Add optimization config for Deepseek V3/R1 with 4 GPUs	2025-02-13 16:32:28 +08:00
Azure	101db0e9de	Merge branch 'main' into update-yaml	2025-02-12 08:56:03 +00:00
Azure	3897f001f5	update FAQ	2025-02-12 08:50:58 +00:00
ZiWei Yuan	4ae2e81c38	Merge pull request #152 from kvcache-ai/server_support ... Server support	2025-02-12 12:45:13 +08:00
liam	4385e85096	⚡ support force thinking	2025-02-12 12:43:53 +08:00
Azure	0564ac8465	update marlin expert example	2025-02-12 04:11:00 +00:00
liam	6f3a39be08	⚡ update force_think config	2025-02-12 12:10:16 +08:00
liam	e536e1420d	⚡ update force_think	2025-02-12 11:42:55 +08:00
liam	d07087a7e2	⚡ support R1 force thinking	2025-02-11 15:43:41 +08:00
UnicornChan	7527619f53	Merge pull request #122 from kvcache-ai/feat-DeepSeekV3 ... [Feat] add support to DeepSeekV3	2025-02-10 13:54:46 +08:00
liam	83401dbb3b	⚡ ready to publish	2025-02-10 12:29:23 +08:00
unicornchan	c7e6d09068	[feature] update version and github action jobs for package	2025-02-10 01:00:57 +00:00
liam	098602b08f	⚡ v0.2 ongoing	2025-02-09 22:41:14 +08:00
liam	bf1d413be0	Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3	2025-02-08 13:17:10 +08:00
liam	c18ecd7b7f	⚡ add flush print in local_chat output and change default optimize yaml of deepseekv3 to single gpu	2025-02-08 13:15:52 +08:00
RodriMora	b1bff2a405	Added simple /models endpoint to work with frontends that don't allow bypass check like Openweb-ui	2025-02-07 10:30:39 +01:00
Azure	c4d9bc6670	support KExpertsMarlin backend	2025-02-07 05:57:40 +00:00
liam	0262f954c7	Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3	2025-02-06 22:41:25 +08:00
liam	3dca28d23b	⚡ fix moe.cpp int overflow problem	2025-02-06 22:39:16 +08:00
Azure	027b11266c	modify moeinfer param	2025-02-06 14:07:38 +00:00
Azure	ee24a27001	update v3 single gpu rule yaml;	2025-02-04 16:14:35 +00:00
Azure	907251c743	done support deepseekv3	2025-02-04 15:53:38 +00:00
Azure	f748cd29f0	fix rope; update moegate	2025-02-01 18:05:45 +00:00
Azure	f873558a89	update rope calculation; update modeling.py; update gate for moe	2025-02-01 07:32:21 +00:00
Azure	5a50b34627	fix hard coding caused by rope dim calculation, load from config now	2025-01-31 15:25:50 +00:00
Azure	476b1d8dc6	support deepseekv3; runable but have precition problem	2025-01-31 08:27:24 +00:00
liam	04cebec4bb	⚡ rm opt config path default value and fix some config logic bug	2024-11-14 20:02:30 +08:00
liam	c2b4dc805c	🚑️:roll back transformer.py and find that it's multiple chat hsitory have minor accurate error	2024-11-04 14:02:19 +08:00
liam	a148da2cfe	✨: rm sensitive info in config.yaml, add readme of makefile. support old model_path config	2024-11-04 14:02:19 +08:00
anyanqilin	9a2e7057c8	wjh fix change	2024-11-04 14:02:19 +08:00
anyanqilin	a72dc6ed15	wjh change	2024-11-04 14:02:19 +08:00
anyanqilin	2d67016d14	wjh-change	2024-11-04 14:02:19 +08:00
liam	7c94df4bcf	🚑️: back transformer.py bugs version, and fix typo error in local_chat.py	2024-11-04 14:02:19 +08:00
liam	dd1d8667f3	✨: refactor local_chat and fix message slice bug in server	2024-11-04 14:02:19 +08:00
Chen Hongtao	43fc7f44a6	Merge pull request #99 from chenht2022/main ... Adapt Windows	2024-10-09 19:09:58 +08:00
chenht2022	14869b55ad	Adapt Windows	2024-10-09 11:08:32 +00:00
UnicornChan	a81a7ffe21	Merge pull request #77 from TKONIY/fix-prefill-and-generate ... Fix: Wrong type of token list returned by prefill_and_generate	2024-10-09 19:04:27 +08:00
Chen Hongtao	b4904537e3	Merge pull request #83 from sayap/task-queue-cond-var ... Use cond var to avoid busy loop	2024-10-09 18:57:17 +08:00
UnicornChan	43e8848ddd	Merge pull request #86 from xhedit/main ... typo fix: KMisrtal -> KMistral	2024-09-19 10:37:55 +08:00
Azure	3758afb526	fix some dequant function dosen't support multi gpu bug	2024-09-13 08:34:23 +00:00
xhedit	234faf7987	typo fix: KMisrtal -> KMistral	2024-09-12 15:58:01 +00:00
Yap Sok Ann	6666d62237	Use cond var to avoid busy loop	2024-09-11 16:10:54 +07:00
UnicornChan	3ed8a0437f	Merge pull request #72 from sayap/dequantize-iq4-xs ... Support IQ4_XS dequantize	2024-09-06 10:06:10 +08:00
yangshen	ee72cee050	Fix: the tokens return by prefill_and_generate	2024-09-05 05:29:23 +00:00
UnicornChan	be81269e38	Merge pull request #71 from Azure-Tang/main ... [fix] Fix qlen > chunk_size mask is none error	2024-09-02 14:51:05 +08:00
Azure	c55de02f7b	fix qlen > 1000 mask is none error	2024-09-02 02:58:10 +00:00
Yap Sok Ann	be356c1b8d	Support IQ4_XS dequantize	2024-09-02 09:10:19 +07:00
chenxl	49cce0c437	[fix] bugs about Qwen57B, install requirement, Dockerfile	2024-08-30 09:51:32 +00:00
TangJingqi	8747c099f2	update yaml example; update version idx; update docker file	2024-08-29 22:39:20 +08:00