vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-06 20:49:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
112 commits 41 branches 19 tags 46 MiB
Commit graph

112 commits

This branch
This branch
All branches
Author SHA1 Message Date
chenht2022
6b33f41de4 Add V0.3-preview doc 2025-02-09 16:08:16 +00:00
liam
098602b08f ⚡ v0.2 ongoing 2025-02-09 22:41:14 +08:00
liam
bf1d413be0 Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3 2025-02-08 13:17:10 +08:00
liam
c18ecd7b7f ⚡ add flush print in local_chat output and change default optimize yaml of deepseekv3 to single gpu 2025-02-08 13:15:52 +08:00
Azure
c4d9bc6670 support KExpertsMarlin backend 2025-02-07 05:57:40 +00:00
liam
0262f954c7 Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3 2025-02-06 22:41:25 +08:00
liam
3dca28d23b ⚡ fix moe.cpp int overflow problem 2025-02-06 22:39:16 +08:00
Azure
027b11266c modify moeinfer param 2025-02-06 14:07:38 +00:00
Azure
ee24a27001 update v3 single gpu rule yaml; 2025-02-04 16:14:35 +00:00
Azure
907251c743 done support deepseekv3 2025-02-04 15:53:38 +00:00
Azure
f748cd29f0 fix rope; update moegate 2025-02-01 18:05:45 +00:00
Azure
f873558a89 update rope calculation; update modeling.py; update gate for moe 2025-02-01 07:32:21 +00:00
Azure
5a50b34627 fix hard coding caused by rope dim calculation, load from config now 2025-01-31 15:25:50 +00:00
Azure
476b1d8dc6 support deepseekv3; runable but have precition problem 2025-01-31 08:27:24 +00:00
UnicornChan
de7e892f72
Merge pull request #115 from KMSorSMS/main 
⚡ rm opt config path default value and fix some config logic bug
 2024-11-14 20:07:12 +08:00
liam
04cebec4bb ⚡ rm opt config path default value and fix some config logic bug 2024-11-14 20:02:30 +08:00
UnicornChan
dddc42038d
Merge pull request #110 from KMSorSMS/main 
refactor local_chat & config setting
 2024-11-06 09:44:12 +08:00
liam
28ae7e24ed ✨: finish patch 2024-11-04 14:05:03 +08:00
liam
c2b4dc805c 🚑️:roll back transformer.py and find that it's multiple chat hsitory have minor accurate error 2024-11-04 14:02:19 +08:00
liam
a148da2cfe ✨: rm sensitive info in config.yaml, add readme of makefile. support old model_path config 2024-11-04 14:02:19 +08:00
anyanqilin
9a2e7057c8 wjh fix change 2024-11-04 14:02:19 +08:00
anyanqilin
a72dc6ed15 wjh change 2024-11-04 14:02:19 +08:00
anyanqilin
2d67016d14 wjh-change 2024-11-04 14:02:19 +08:00
liam
7c94df4bcf 🚑️: back transformer.py bugs version, and fix typo error in local_chat.py 2024-11-04 14:02:19 +08:00
liam
dd1d8667f3 ✨: refactor local_chat and fix message slice bug in server 2024-11-04 14:02:19 +08:00
UnicornChan
a6a1cc054f
Merge pull request #106 from Azure-Tang/main 
[Fix] Fix readme structure.
 2024-10-30 16:36:59 +08:00
TangJingqi
c7d62a67db update supported models 2024-10-28 14:37:10 +08:00
TangJingqi
d8ddaf0ea0 fix readme 2024-10-28 14:21:12 +08:00
Chen Hongtao
43fc7f44a6
Merge pull request #99 from chenht2022/main 
Adapt Windows
 2024-10-09 19:09:58 +08:00
chenht2022
14869b55ad Adapt Windows 2024-10-09 11:08:32 +00:00
UnicornChan
a81a7ffe21
Merge pull request #77 from TKONIY/fix-prefill-and-generate 
Fix: Wrong type of token list returned by prefill_and_generate
 2024-10-09 19:04:27 +08:00
Chen Hongtao
b4904537e3
Merge pull request #83 from sayap/task-queue-cond-var 
Use cond var to avoid busy loop
 2024-10-09 18:57:17 +08:00
UnicornChan
43e8848ddd
Merge pull request #86 from xhedit/main 
typo fix: KMisrtal -> KMistral
 2024-09-19 10:37:55 +08:00
UnicornChan
49539ac424
Merge pull request #88 from Azure-Tang/main 
[fix] Fix some gpu dequant function doesn't support multi gpu bug
 2024-09-15 15:17:05 +08:00
Azure
7953bd9c0d update readme 2024-09-13 08:47:35 +00:00
Azure
3758afb526 fix some dequant function dosen't support multi gpu bug 2024-09-13 08:34:23 +00:00
xhedit
234faf7987 typo fix: KMisrtal -> KMistral 2024-09-12 15:58:01 +00:00
Yap Sok Ann
6666d62237 Use cond var to avoid busy loop 2024-09-11 16:10:54 +07:00
UnicornChan
3ed8a0437f
Merge pull request #72 from sayap/dequantize-iq4-xs 
Support IQ4_XS dequantize
 2024-09-06 10:06:10 +08:00
yangshen
ee72cee050 Fix: the tokens return by prefill_and_generate 2024-09-05 05:29:23 +00:00
UnicornChan
be81269e38
Merge pull request #71 from Azure-Tang/main 
[fix] Fix qlen > chunk_size mask is none error
 2024-09-02 14:51:05 +08:00
Azure
c55de02f7b fix qlen > 1000 mask is none error 2024-09-02 02:58:10 +00:00
Yap Sok Ann
be356c1b8d Support IQ4_XS dequantize 2024-09-02 09:10:19 +07:00
UnicornChan
022b893819
Merge pull request #67 from UnicornChan/main 
[fix] fix bugs about Qwen2-57B, install requirement, DockerFile
 2024-08-30 17:57:49 +08:00
chenxl
49cce0c437 [fix] bugs about Qwen57B, install requirement, Dockerfile 2024-08-30 09:51:32 +00:00
UnicornChan
351698c3b5
Merge pull request #64 from eltociear/patch-1 
docs: update long_context_introduction.md
 2024-08-30 09:03:47 +08:00
chenxl
c80490a95e [fix] some bugs while package in github action 2024-08-29 19:40:03 +00:00
Ikko Eltociear Ashimine
e961adde15
docs: update long_context_introduction.md 
accuary -> accuracy
 2024-08-30 03:34:39 +09:00
UnicornChan
f536a7085f
Merge pull request #62 from Azure-Tang/main 
[Fix] Fix problem that ktransformers cannot offload whole layer in cpu
 2024-08-29 23:40:21 +08:00
TangJingqi
8747c099f2 update yaml example; update version idx; update docker file 2024-08-29 22:39:20 +08:00