877 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
ZiWei Yuan	be13587fe6	Merge branch 'kvcache-ai:main' into main	2025-06-23 21:09:10 +08:00
liam Yuan	22d0d9ccb2	✨ update vendor ZTE name	2025-06-23 21:07:17 +08:00
ZiWei Yuan	06e45fd7d1	Merge pull request #1398 from KMSorSMS/main ... ✨ update vendor support list	2025-06-23 21:04:19 +08:00
liam Yuan	cb77b52c63	✨ update vendor support list	2025-06-23 21:00:01 +08:00
ZiWei Yuan	90888fee0d	Merge pull request #1372 from JaydenChao101/patch-1 ... 包含所有package	2025-06-15 21:08:53 +08:00
Azure	64ec0ec148	Merge pull request #1379 from aubreyli/main ... Load DS-R1-0528 for module is BaseInjectedModule instance	2025-06-11 16:05:04 +08:00
Aubrey Li	b599111b04	Load DS-R1-0528 for module is BaseInjectedModule instance	2025-06-10 11:31:58 +08:00
Jayden Chao	2637d285bf	Update pyproject.toml	2025-06-07 21:28:49 +08:00
wang jiahao	dcba29b291	Merge pull request #1366 from zhouye/main ... Fix kv_b_proj shape for unsloth quantized models	2025-06-07 18:14:00 +08:00
Ye Zhou	255c0fcf3b	Fix kv_b_proj shape for unsloth quantized models	2025-06-05 17:33:11 +08:00
Atream	7071970339	Merge pull request #1343 from zhouye/main ... Mirror #1247 in server mode	2025-06-02 16:07:04 +08:00
Atream	cc367f5814	Merge pull request #1351 from kvcache-ai/load-DeepSeek-0528 ... [Patch] load DeepSeek-R1-0528 and enable CPU GGUF dequant when GPU dequant is not implemented	2025-06-01 19:13:57 +08:00
qiyuxinlin	a6b3243a56	[Patch] lload DeepSeek-R1-0528	2025-05-31 14:19:20 +00:00
Atream	ac48a58cca	Merge pull request #1350 from kvcache-ai/Update-WeChatGroup ... Add files via upload	2025-05-31 13:34:14 +08:00
Atream	44143d972f	Add files via upload	2025-05-31 13:33:36 +08:00
Emmanuel Ferdman	d8bc6402b5	raise exception on device error (#1342) ... * display the unavailable torch device on error * Raise exception on device error --------- Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>	2025-05-29 15:37:41 +08:00
Ye Zhou	00949d5e8d	Mirror #1247 in server mode	2025-05-29 15:30:40 +08:00
aubreyli	ce75fcd7dd	Merge pull request #1337 from liu-shaojun/docker_xpu ... Add Dockerfile and usage guide for XPU support	2025-05-28 14:08:46 +08:00
Shaojun Liu	404ad39a04	docs: add Dockerfile.xpu and GPU driver setup instructions ... - Add Dockerfile.xpu for oneAPI-based container - Create Docker_xpu.md with usage instructions - Update xpu.md to include Docker guide	2025-05-28 13:55:35 +08:00
wang jiahao	0c44f2e211	Merge pull request #1331 from rnwang04/qwen3_xpu_support ... add XPU support for qwen3moe local chat	2025-05-23 10:38:28 +08:00
qiyuxinlin	71a5fc5770	fix local_chat.py chunk_size not effect experts	2025-05-23 02:35:01 +00:00
rnwang04	adc0906967	add XPU support for qwen3moe local chat	2025-05-22 21:01:41 +08:00
Chen Hongtao	25893366b6	Merge pull request #1328 from chenht2022/main ... Fix NaN bug	2025-05-21 11:46:48 +08:00
chenht2022	66453981ff	Fix NaN bug	2025-05-21 03:39:49 +00:00
Atream	7d79735bd0	Merge pull request #1323 from kvcache-ai/Atream-patch-2 ... Update version	2025-05-19 23:21:56 +08:00
Atream	4f78e37625	Update version	2025-05-19 23:21:23 +08:00
Atream	01311d251d	Merge pull request #1320 from aubreyli/no_cuda_graph_err ... VLinearMarlin: padding to input.shape[0] to avoid CUDA error	2025-05-18 02:45:05 -06:00
Aubrey Li	d347aeb518	VLinearMarlin: padding to input.shape[0] to avoid CUDA error ... Fix the following runtime error with --no-use_cuda_graph option Traceback (most recent call last): File "/home/aubrey/miniforge3/envs/kt/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/home/aubrey/miniforge3/envs/kt/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/aubrey/miniforge3/envs/kt/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 282, in run_engine engine.loop() File "/home/aubrey/miniforge3/envs/kt/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 234, in loop self.model_runner.run(self.batch, self.query_manager) File "/home/aubrey/miniforge3/envs/kt/lib/python3.11/site-packages/ktransformers/server/balance_serve/inference/model_runner.py", line 220, in run self.output.logits[0] = self.output.logits[0][self.input[cuda_graph_idx].minibatch.logits_start] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: CUDA error: an illegal memory access was encountered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.	2025-05-18 15:11:37 +08:00
wang jiahao	32f3d7befb	Merge pull request #1307 from kvcache-ai/hyc ... add xpu parameters to install.sh	2025-05-17 15:25:33 +08:00
Alisehen	5b08d5b07b	fix	2025-05-17 07:22:51 +00:00
aubreyli	551ebc91c7	Merge pull request #1313 from rnwang04/update_ipex_llm_version ... fix ipex-llm version to 2.3.0rc1	2025-05-16 13:28:12 +08:00
rnwang04	a56aa45186	fix ipex-llm version to 2.3.0rc1	2025-05-16 12:22:08 +08:00
aubreyli	a3346c9a41	Merge pull request #1311 from kvcache-ai/wx-csy-patch-1 ... XPU.md: fix typos	2025-05-16 08:28:23 +08:00
Shaoyuan CHEN	5d194c5db0	Fix typos	2025-05-15 22:15:55 +08:00
Alisehen	f3b1e36b6a	bug fix	2025-05-15 10:01:51 +00:00
Atream	7faa776659	Merge pull request #1277 from Coekjan/patch-1 ... Fix typo about `GLIBCXX_3.4.32`	2025-05-15 01:58:00 -06:00
Alisehen	edd9efa49e	fix	2025-05-15 07:28:50 +00:00
Alisehen	055680e26c	add flashinfer to cuda device	2025-05-15 07:03:45 +00:00
Alisehen	f3be33a313	add xpu parameters to install.sh	2025-05-15 06:39:02 +00:00
aubreyli	af9472b518	Merge pull request #1306 from aubreyli/xpu-doc ... xpu.md: add device discovery tips	2025-05-15 14:18:47 +08:00
Aubrey Li	72f6d93ffd	xpu.md: add device discovery tips	2025-05-15 14:12:26 +08:00
wang jiahao	8caecf37d8	Merge pull request #1305 from kvcache-ai/update-readme ... fix deduplicate_and_sort cudagraphs	2025-05-15 12:10:20 +08:00
qiyuxinlin	b40f13abeb	fix deduplicate_and_sort cudagraphs	2025-05-15 04:09:34 +00:00
aubreyli	09f0ddc00b	Merge pull request #1303 from rnwang04/fix_typo_and_style ... fix typo and code style, and update setup.py ValueError message	2025-05-15 10:55:58 +08:00
rnwang04	2f6e14a54b	fix md typo, fix code style, and update setup value error message	2025-05-15 10:14:39 +00:00
Atream	07c5f23da5	Merge pull request #1304 from kvcache-ai/Atream-patch-1 ... Update README.md	2025-05-14 20:29:54 -06:00
Atream	d051a14941	Update README.md	2025-05-15 10:29:43 +08:00
wang jiahao	2d3aaef8b6	Merge pull request #1301 from kvcache-ai/update-readme ... update readme	2025-05-14 21:15:55 +08:00
qiyuxinlin	d35d61f6a1	update readme	2025-05-14 13:15:18 +00:00
qiyuxinlin	c3d0ac80c6	update readme	2025-05-14 13:13:10 +00:00