vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-11 07:44:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
877 commits 42 branches 19 tags 46 MiB
Commit graph

877 commits

This branch
This branch
All branches
Author SHA1 Message Date
Azure
3758afb526 fix some dequant function dosen't support multi gpu bug 2024-09-13 08:34:23 +00:00
xhedit
234faf7987 typo fix: KMisrtal -> KMistral 2024-09-12 15:58:01 +00:00
Yap Sok Ann
6666d62237 Use cond var to avoid busy loop 2024-09-11 16:10:54 +07:00
UnicornChan
3ed8a0437f
Merge pull request #72 from sayap/dequantize-iq4-xs 
Support IQ4_XS dequantize
 2024-09-06 10:06:10 +08:00
yangshen
ee72cee050 Fix: the tokens return by prefill_and_generate 2024-09-05 05:29:23 +00:00
UnicornChan
be81269e38
Merge pull request #71 from Azure-Tang/main 
[fix] Fix qlen > chunk_size mask is none error
 2024-09-02 14:51:05 +08:00
Azure
c55de02f7b fix qlen > 1000 mask is none error 2024-09-02 02:58:10 +00:00
Yap Sok Ann
be356c1b8d Support IQ4_XS dequantize 2024-09-02 09:10:19 +07:00
UnicornChan
022b893819
Merge pull request #67 from UnicornChan/main 
[fix] fix bugs about Qwen2-57B, install requirement, DockerFile
 2024-08-30 17:57:49 +08:00
chenxl
49cce0c437 [fix] bugs about Qwen57B, install requirement, Dockerfile 2024-08-30 09:51:32 +00:00
UnicornChan
351698c3b5
Merge pull request #64 from eltociear/patch-1 
docs: update long_context_introduction.md
 2024-08-30 09:03:47 +08:00
chenxl
c80490a95e [fix] some bugs while package in github action 2024-08-29 19:40:03 +00:00
Ikko Eltociear Ashimine
e961adde15
docs: update long_context_introduction.md 
accuary -> accuracy
 2024-08-30 03:34:39 +09:00
UnicornChan
f536a7085f
Merge pull request #62 from Azure-Tang/main 
[Fix] Fix problem that ktransformers cannot offload whole layer in cpu
 2024-08-29 23:40:21 +08:00
TangJingqi
8747c099f2 update yaml example; update version idx; update docker file 2024-08-29 22:39:20 +08:00
TangJingqi
6735beb5b6 Fix cannot offload whole layer in cpu 2024-08-29 19:10:14 +08:00
Atream
35d7aed207
Merge pull request #60 from sammcj/patch-1 
fix(docs): fix broken link
 2024-08-29 19:01:16 +08:00
Sam
0b57627bb9
fix(docs): fix broken link 2024-08-29 20:00:55 +10:00
Azure
1dcb8dae5b
Merge pull request #58 from Azure-Tang/main 
[fix]  Fix readme datas
 2024-08-29 14:39:58 +08:00
TangJingqi
440d827e7c update readme 2024-08-29 12:04:56 +08:00
TangJingqi
abd4214b56 fix readme; adjust param 2024-08-29 10:40:08 +08:00
UnicornChan
233bbb8c55
Merge pull request #57 from UnicornChan/develop-0.1.3 
[feature] release 0.1.3
 2024-08-29 01:57:34 +08:00
chenxl
4d1d561d28 [feature] release 0.1.3 2024-08-28 16:11:43 +00:00
UnicornChan
67f8b370c3
Merge pull request #56 from hyx1999/patch-1 
Update README.md
 2024-08-28 14:30:38 +08:00
_HYX_
ea1143e54e
Update README.md 
Set the reminder to set CUDA_HOME and CUDA_PATH in the README to the "Quick Start" section under "Install CUDA".
 2024-08-28 12:15:38 +08:00
UnicornChan
0f054fe4ff
Merge pull request #52 from UnicornChan/fix-bug-load-config 
Fix: None for load config
 2024-08-22 23:54:17 +08:00
chenxl
b9f0819a86 None for load config 2024-08-22 15:52:25 +00:00
UnicornChan
1f85db3d73
Merge pull request #51 from molamooo/fix-f16-dequantize-device 
[fix] f16 dequantize device ignored
 2024-08-22 16:31:45 +08:00
molamooo
29f4151ebc
[fix] f16 dequantize device ignored 2024-08-22 15:10:06 +08:00
UnicornChan
cbc47d0b68
Merge pull request #48 from Azure-Tang/main 
[fix] Fix bugs about static cache and server param;
 2024-08-21 23:24:46 +08:00
TangJingqi
170b7a6001 fix server don't accept yaml path as param; fix server static cache device problem 2024-08-21 14:19:43 +08:00
Azure
4358722891
Merge pull request #41 from Azure-Tang/main 
Update README
 2024-08-16 15:27:01 +08:00
TangJingqi
4a0f1cbbfa Update readme 2024-08-16 15:22:16 +08:00
Azure
f681eb4a0f
Merge branch 'kvcache-ai:main' into main 2024-08-16 15:21:41 +08:00
Azure
b25ad4ec32
Merge pull request #39 from Azure-Tang/develop-0.1.2 
[fix] fix broken link in tutorial.
 2024-08-16 11:29:11 +08:00
TangJingqi
4f87756c2e fix broken link 2024-08-16 11:10:30 +08:00
TangJingqi
7199699d78 Fix the broken link 2024-08-16 11:00:06 +08:00
Azure
e81fc482ff update README 2024-08-15 16:59:26 +00:00
Azure
6bcec86fa0 update README 2024-08-15 16:54:31 +00:00
UnicornChan
77a34c289c
Merge pull request #36 from kvcache-ai/develop-0.1.2 
Release v0.1.2
 2024-08-15 20:59:50 +08:00
UnicornChan
395cd3e786
Merge pull request #35 from Azure-Tang/develop-0.1.2 
[update] Update readme; Add tutorial
 2024-08-15 20:56:22 +08:00
TangJingqi
de3faaf55d Update readme; add pipeline tutorial; add detailed inject tutorial 2024-08-15 20:42:54 +08:00
UnicornChan
c9bf79299b
Merge pull request #34 from Azure-Tang/develop-0.1.2 
[fix] format classes and files name
 2024-08-15 11:28:19 +08:00
TangJingqi
c47205dce9 fix name 2024-08-15 11:25:12 +08:00
TangJingqi
67043b4b5c [fix] format classes and files name 2024-08-15 10:44:59 +08:00
chenxl
1db4a67dca [feature] add github action for pre compile 2024-08-14 16:54:50 +00:00
Atream
412055d450 [feature] experts can be injected using CPUInfer 
[fix] fix ktransformers interface when use new CUDAGraphRunner
[fix] fix YAML and optimize logic, the top rule has the highest priority
 2024-08-14 16:10:54 +08:00
UnicornChan
80815dbc50
Merge pull request #30 from BITcyman/feature-lc-dequantize_q2k_q3k_gpu 
[feature] support q2_k & q3_k dequantize on gpu
 2024-08-12 21:02:32 +08:00
BITcyman
7c4cb520bd [feature] support q2_k & q3_k dequantize on gpu 2024-08-12 12:53:12 +00:00
chenxl
650c368c18 Merge remote-tracking branch 'upstream/main' into develop-0.1.2 2024-08-12 12:31:49 +00:00