107 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
liam	0262f954c7	Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3	2025-02-06 22:41:25 +08:00
liam	3dca28d23b	⚡ fix moe.cpp int overflow problem	2025-02-06 22:39:16 +08:00
Azure	027b11266c	modify moeinfer param	2025-02-06 14:07:38 +00:00
Azure	ee24a27001	update v3 single gpu rule yaml;	2025-02-04 16:14:35 +00:00
Azure	907251c743	done support deepseekv3	2025-02-04 15:53:38 +00:00
Azure	f748cd29f0	fix rope; update moegate	2025-02-01 18:05:45 +00:00
Azure	f873558a89	update rope calculation; update modeling.py; update gate for moe	2025-02-01 07:32:21 +00:00
Azure	5a50b34627	fix hard coding caused by rope dim calculation, load from config now	2025-01-31 15:25:50 +00:00
Azure	476b1d8dc6	support deepseekv3; runable but have precition problem	2025-01-31 08:27:24 +00:00
UnicornChan	de7e892f72	Merge pull request #115 from KMSorSMS/main ... ⚡ rm opt config path default value and fix some config logic bug	2024-11-14 20:07:12 +08:00
liam	04cebec4bb	⚡ rm opt config path default value and fix some config logic bug	2024-11-14 20:02:30 +08:00
UnicornChan	dddc42038d	Merge pull request #110 from KMSorSMS/main ... refactor local_chat & config setting	2024-11-06 09:44:12 +08:00
liam	28ae7e24ed	✨: finish patch	2024-11-04 14:05:03 +08:00
liam	c2b4dc805c	🚑️:roll back transformer.py and find that it's multiple chat hsitory have minor accurate error	2024-11-04 14:02:19 +08:00
liam	a148da2cfe	✨: rm sensitive info in config.yaml, add readme of makefile. support old model_path config	2024-11-04 14:02:19 +08:00
anyanqilin	9a2e7057c8	wjh fix change	2024-11-04 14:02:19 +08:00
anyanqilin	a72dc6ed15	wjh change	2024-11-04 14:02:19 +08:00
anyanqilin	2d67016d14	wjh-change	2024-11-04 14:02:19 +08:00
liam	7c94df4bcf	🚑️: back transformer.py bugs version, and fix typo error in local_chat.py	2024-11-04 14:02:19 +08:00
liam	dd1d8667f3	✨: refactor local_chat and fix message slice bug in server	2024-11-04 14:02:19 +08:00
UnicornChan	a6a1cc054f	Merge pull request #106 from Azure-Tang/main ... [Fix] Fix readme structure.	2024-10-30 16:36:59 +08:00
TangJingqi	c7d62a67db	update supported models	2024-10-28 14:37:10 +08:00
TangJingqi	d8ddaf0ea0	fix readme	2024-10-28 14:21:12 +08:00
Chen Hongtao	43fc7f44a6	Merge pull request #99 from chenht2022/main ... Adapt Windows	2024-10-09 19:09:58 +08:00
chenht2022	14869b55ad	Adapt Windows	2024-10-09 11:08:32 +00:00
UnicornChan	a81a7ffe21	Merge pull request #77 from TKONIY/fix-prefill-and-generate ... Fix: Wrong type of token list returned by prefill_and_generate	2024-10-09 19:04:27 +08:00
Chen Hongtao	b4904537e3	Merge pull request #83 from sayap/task-queue-cond-var ... Use cond var to avoid busy loop	2024-10-09 18:57:17 +08:00
UnicornChan	43e8848ddd	Merge pull request #86 from xhedit/main ... typo fix: KMisrtal -> KMistral	2024-09-19 10:37:55 +08:00
UnicornChan	49539ac424	Merge pull request #88 from Azure-Tang/main ... [fix] Fix some gpu dequant function doesn't support multi gpu bug	2024-09-15 15:17:05 +08:00
Azure	7953bd9c0d	update readme	2024-09-13 08:47:35 +00:00
Azure	3758afb526	fix some dequant function dosen't support multi gpu bug	2024-09-13 08:34:23 +00:00
xhedit	234faf7987	typo fix: KMisrtal -> KMistral	2024-09-12 15:58:01 +00:00
Yap Sok Ann	6666d62237	Use cond var to avoid busy loop	2024-09-11 16:10:54 +07:00
UnicornChan	3ed8a0437f	Merge pull request #72 from sayap/dequantize-iq4-xs ... Support IQ4_XS dequantize	2024-09-06 10:06:10 +08:00
yangshen	ee72cee050	Fix: the tokens return by prefill_and_generate	2024-09-05 05:29:23 +00:00
UnicornChan	be81269e38	Merge pull request #71 from Azure-Tang/main ... [fix] Fix qlen > chunk_size mask is none error	2024-09-02 14:51:05 +08:00
Azure	c55de02f7b	fix qlen > 1000 mask is none error	2024-09-02 02:58:10 +00:00
Yap Sok Ann	be356c1b8d	Support IQ4_XS dequantize	2024-09-02 09:10:19 +07:00
UnicornChan	022b893819	Merge pull request #67 from UnicornChan/main ... [fix] fix bugs about Qwen2-57B, install requirement, DockerFile	2024-08-30 17:57:49 +08:00
chenxl	49cce0c437	[fix] bugs about Qwen57B, install requirement, Dockerfile	2024-08-30 09:51:32 +00:00
UnicornChan	351698c3b5	Merge pull request #64 from eltociear/patch-1 ... docs: update long_context_introduction.md	2024-08-30 09:03:47 +08:00
chenxl	c80490a95e	[fix] some bugs while package in github action	2024-08-29 19:40:03 +00:00
Ikko Eltociear Ashimine	e961adde15	docs: update long_context_introduction.md ... accuary -> accuracy	2024-08-30 03:34:39 +09:00
UnicornChan	f536a7085f	Merge pull request #62 from Azure-Tang/main ... [Fix] Fix problem that ktransformers cannot offload whole layer in cpu	2024-08-29 23:40:21 +08:00
TangJingqi	8747c099f2	update yaml example; update version idx; update docker file	2024-08-29 22:39:20 +08:00
TangJingqi	6735beb5b6	Fix cannot offload whole layer in cpu	2024-08-29 19:10:14 +08:00
Atream	35d7aed207	Merge pull request #60 from sammcj/patch-1 ... fix(docs): fix broken link	2024-08-29 19:01:16 +08:00
Sam	0b57627bb9	fix(docs): fix broken link	2024-08-29 20:00:55 +10:00
Azure	1dcb8dae5b	Merge pull request #58 from Azure-Tang/main ... [fix] Fix readme datas	2024-08-29 14:39:58 +08:00
TangJingqi	440d827e7c	update readme	2024-08-29 12:04:56 +08:00