vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-06 20:49:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
82 commits 41 branches 19 tags 46 MiB
Commit graph

82 commits

This branch
This branch
All branches
Author SHA1 Message Date
UnicornChan
a81a7ffe21
Merge pull request #77 from TKONIY/fix-prefill-and-generate 
Fix: Wrong type of token list returned by prefill_and_generate
 2024-10-09 19:04:27 +08:00
Chen Hongtao
b4904537e3
Merge pull request #83 from sayap/task-queue-cond-var 
Use cond var to avoid busy loop
 2024-10-09 18:57:17 +08:00
UnicornChan
43e8848ddd
Merge pull request #86 from xhedit/main 
typo fix: KMisrtal -> KMistral
 2024-09-19 10:37:55 +08:00
UnicornChan
49539ac424
Merge pull request #88 from Azure-Tang/main 
[fix] Fix some gpu dequant function doesn't support multi gpu bug
 2024-09-15 15:17:05 +08:00
Azure
7953bd9c0d update readme 2024-09-13 08:47:35 +00:00
Azure
3758afb526 fix some dequant function dosen't support multi gpu bug 2024-09-13 08:34:23 +00:00
xhedit
234faf7987 typo fix: KMisrtal -> KMistral 2024-09-12 15:58:01 +00:00
Yap Sok Ann
6666d62237 Use cond var to avoid busy loop 2024-09-11 16:10:54 +07:00
UnicornChan
3ed8a0437f
Merge pull request #72 from sayap/dequantize-iq4-xs 
Support IQ4_XS dequantize
 2024-09-06 10:06:10 +08:00
yangshen
ee72cee050 Fix: the tokens return by prefill_and_generate 2024-09-05 05:29:23 +00:00
UnicornChan
be81269e38
Merge pull request #71 from Azure-Tang/main 
[fix] Fix qlen > chunk_size mask is none error
 2024-09-02 14:51:05 +08:00
Azure
c55de02f7b fix qlen > 1000 mask is none error 2024-09-02 02:58:10 +00:00
Yap Sok Ann
be356c1b8d Support IQ4_XS dequantize 2024-09-02 09:10:19 +07:00
UnicornChan
022b893819
Merge pull request #67 from UnicornChan/main 
[fix] fix bugs about Qwen2-57B, install requirement, DockerFile
 2024-08-30 17:57:49 +08:00
chenxl
49cce0c437 [fix] bugs about Qwen57B, install requirement, Dockerfile 2024-08-30 09:51:32 +00:00
UnicornChan
351698c3b5
Merge pull request #64 from eltociear/patch-1 
docs: update long_context_introduction.md
 2024-08-30 09:03:47 +08:00
chenxl
c80490a95e [fix] some bugs while package in github action 2024-08-29 19:40:03 +00:00
Ikko Eltociear Ashimine
e961adde15
docs: update long_context_introduction.md 
accuary -> accuracy
 2024-08-30 03:34:39 +09:00
UnicornChan
f536a7085f
Merge pull request #62 from Azure-Tang/main 
[Fix] Fix problem that ktransformers cannot offload whole layer in cpu
 2024-08-29 23:40:21 +08:00
TangJingqi
8747c099f2 update yaml example; update version idx; update docker file 2024-08-29 22:39:20 +08:00
TangJingqi
6735beb5b6 Fix cannot offload whole layer in cpu 2024-08-29 19:10:14 +08:00
Atream
35d7aed207
Merge pull request #60 from sammcj/patch-1 
fix(docs): fix broken link
 2024-08-29 19:01:16 +08:00
Sam
0b57627bb9
fix(docs): fix broken link 2024-08-29 20:00:55 +10:00
Azure
1dcb8dae5b
Merge pull request #58 from Azure-Tang/main 
[fix]  Fix readme datas
 2024-08-29 14:39:58 +08:00
TangJingqi
440d827e7c update readme 2024-08-29 12:04:56 +08:00
TangJingqi
abd4214b56 fix readme; adjust param 2024-08-29 10:40:08 +08:00
UnicornChan
233bbb8c55
Merge pull request #57 from UnicornChan/develop-0.1.3 
[feature] release 0.1.3
 2024-08-29 01:57:34 +08:00
chenxl
4d1d561d28 [feature] release 0.1.3 2024-08-28 16:11:43 +00:00
UnicornChan
67f8b370c3
Merge pull request #56 from hyx1999/patch-1 
Update README.md
 2024-08-28 14:30:38 +08:00
_HYX_
ea1143e54e
Update README.md 
Set the reminder to set CUDA_HOME and CUDA_PATH in the README to the "Quick Start" section under "Install CUDA".
 2024-08-28 12:15:38 +08:00
UnicornChan
0f054fe4ff
Merge pull request #52 from UnicornChan/fix-bug-load-config 
Fix: None for load config
 2024-08-22 23:54:17 +08:00
chenxl
b9f0819a86 None for load config 2024-08-22 15:52:25 +00:00
UnicornChan
1f85db3d73
Merge pull request #51 from molamooo/fix-f16-dequantize-device 
[fix] f16 dequantize device ignored
 2024-08-22 16:31:45 +08:00
molamooo
29f4151ebc
[fix] f16 dequantize device ignored 2024-08-22 15:10:06 +08:00
UnicornChan
cbc47d0b68
Merge pull request #48 from Azure-Tang/main 
[fix] Fix bugs about static cache and server param;
 2024-08-21 23:24:46 +08:00
TangJingqi
170b7a6001 fix server don't accept yaml path as param; fix server static cache device problem 2024-08-21 14:19:43 +08:00
Azure
4358722891
Merge pull request #41 from Azure-Tang/main 
Update README
 2024-08-16 15:27:01 +08:00
TangJingqi
4a0f1cbbfa Update readme 2024-08-16 15:22:16 +08:00
Azure
f681eb4a0f
Merge branch 'kvcache-ai:main' into main 2024-08-16 15:21:41 +08:00
Azure
b25ad4ec32
Merge pull request #39 from Azure-Tang/develop-0.1.2 
[fix] fix broken link in tutorial.
 2024-08-16 11:29:11 +08:00
TangJingqi
4f87756c2e fix broken link 2024-08-16 11:10:30 +08:00
TangJingqi
7199699d78 Fix the broken link 2024-08-16 11:00:06 +08:00
Azure
e81fc482ff update README 2024-08-15 16:59:26 +00:00
Azure
6bcec86fa0 update README 2024-08-15 16:54:31 +00:00
UnicornChan
77a34c289c
Merge pull request #36 from kvcache-ai/develop-0.1.2 
Release v0.1.2
 2024-08-15 20:59:50 +08:00
UnicornChan
395cd3e786
Merge pull request #35 from Azure-Tang/develop-0.1.2 
[update] Update readme; Add tutorial
 2024-08-15 20:56:22 +08:00
TangJingqi
de3faaf55d Update readme; add pipeline tutorial; add detailed inject tutorial 2024-08-15 20:42:54 +08:00
UnicornChan
c9bf79299b
Merge pull request #34 from Azure-Tang/develop-0.1.2 
[fix] format classes and files name
 2024-08-15 11:28:19 +08:00
TangJingqi
c47205dce9 fix name 2024-08-15 11:25:12 +08:00
TangJingqi
67043b4b5c [fix] format classes and files name 2024-08-15 10:44:59 +08:00