vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-06 20:49:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
60 commits 41 branches 19 tags 46 MiB
Commit graph

60 commits

This branch
This branch
All branches
Author SHA1 Message Date
Sam
0b57627bb9
fix(docs): fix broken link 2024-08-29 20:00:55 +10:00
Azure
1dcb8dae5b
Merge pull request #58 from Azure-Tang/main 
[fix]  Fix readme datas
 2024-08-29 14:39:58 +08:00
TangJingqi
440d827e7c update readme 2024-08-29 12:04:56 +08:00
TangJingqi
abd4214b56 fix readme; adjust param 2024-08-29 10:40:08 +08:00
UnicornChan
233bbb8c55
Merge pull request #57 from UnicornChan/develop-0.1.3 
[feature] release 0.1.3
 2024-08-29 01:57:34 +08:00
chenxl
4d1d561d28 [feature] release 0.1.3 2024-08-28 16:11:43 +00:00
UnicornChan
67f8b370c3
Merge pull request #56 from hyx1999/patch-1 
Update README.md
 2024-08-28 14:30:38 +08:00
_HYX_
ea1143e54e
Update README.md 
Set the reminder to set CUDA_HOME and CUDA_PATH in the README to the "Quick Start" section under "Install CUDA".
 2024-08-28 12:15:38 +08:00
UnicornChan
0f054fe4ff
Merge pull request #52 from UnicornChan/fix-bug-load-config 
Fix: None for load config
 2024-08-22 23:54:17 +08:00
chenxl
b9f0819a86 None for load config 2024-08-22 15:52:25 +00:00
UnicornChan
1f85db3d73
Merge pull request #51 from molamooo/fix-f16-dequantize-device 
[fix] f16 dequantize device ignored
 2024-08-22 16:31:45 +08:00
molamooo
29f4151ebc
[fix] f16 dequantize device ignored 2024-08-22 15:10:06 +08:00
UnicornChan
cbc47d0b68
Merge pull request #48 from Azure-Tang/main 
[fix] Fix bugs about static cache and server param;
 2024-08-21 23:24:46 +08:00
TangJingqi
170b7a6001 fix server don't accept yaml path as param; fix server static cache device problem 2024-08-21 14:19:43 +08:00
Azure
4358722891
Merge pull request #41 from Azure-Tang/main 
Update README
 2024-08-16 15:27:01 +08:00
TangJingqi
4a0f1cbbfa Update readme 2024-08-16 15:22:16 +08:00
Azure
f681eb4a0f
Merge branch 'kvcache-ai:main' into main 2024-08-16 15:21:41 +08:00
Azure
b25ad4ec32
Merge pull request #39 from Azure-Tang/develop-0.1.2 
[fix] fix broken link in tutorial.
 2024-08-16 11:29:11 +08:00
TangJingqi
4f87756c2e fix broken link 2024-08-16 11:10:30 +08:00
TangJingqi
7199699d78 Fix the broken link 2024-08-16 11:00:06 +08:00
Azure
e81fc482ff update README 2024-08-15 16:59:26 +00:00
Azure
6bcec86fa0 update README 2024-08-15 16:54:31 +00:00
UnicornChan
77a34c289c
Merge pull request #36 from kvcache-ai/develop-0.1.2 
Release v0.1.2
 2024-08-15 20:59:50 +08:00
UnicornChan
395cd3e786
Merge pull request #35 from Azure-Tang/develop-0.1.2 
[update] Update readme; Add tutorial
 2024-08-15 20:56:22 +08:00
TangJingqi
de3faaf55d Update readme; add pipeline tutorial; add detailed inject tutorial 2024-08-15 20:42:54 +08:00
UnicornChan
c9bf79299b
Merge pull request #34 from Azure-Tang/develop-0.1.2 
[fix] format classes and files name
 2024-08-15 11:28:19 +08:00
TangJingqi
c47205dce9 fix name 2024-08-15 11:25:12 +08:00
TangJingqi
67043b4b5c [fix] format classes and files name 2024-08-15 10:44:59 +08:00
chenxl
1db4a67dca [feature] add github action for pre compile 2024-08-14 16:54:50 +00:00
Atream
412055d450 [feature] experts can be injected using CPUInfer 
[fix] fix ktransformers interface when use new CUDAGraphRunner
[fix] fix YAML and optimize logic, the top rule has the highest priority
 2024-08-14 16:10:54 +08:00
UnicornChan
80815dbc50
Merge pull request #30 from BITcyman/feature-lc-dequantize_q2k_q3k_gpu 
[feature] support q2_k & q3_k dequantize on gpu
 2024-08-12 21:02:32 +08:00
BITcyman
7c4cb520bd [feature] support q2_k & q3_k dequantize on gpu 2024-08-12 12:53:12 +00:00
chenxl
650c368c18 Merge remote-tracking branch 'upstream/main' into develop-0.1.2 2024-08-12 12:31:49 +00:00
UnicornChan
44f57270c9
Merge pull request #29 from kvcache-ai/fix]-fix-linux-mutex 
Update task_queue.h
 2024-08-12 20:14:04 +08:00
Atream
3c675af61a
Update task_queue.h 2024-08-12 20:06:19 +08:00
chenxl
f5f79f5c0e [ADD] support multi-gpu qlen>1 q5_k 2024-08-12 11:41:26 +00:00
UnicornChan
f293803156
Merge pull request #27 from chenht2022/develop-0.1.2 
[Feature] towards 0.1.2
 2024-08-09 17:57:27 +08:00
UnicornChan
cb7f8e7817
Merge pull request #26 from kvcache-ai/windows 
[fix] linux and windows can all find CPUInfer in current Directory
 2024-08-09 17:57:00 +08:00
chenxl
782a17e4e6 [feature] add bat for windows, update readme 2024-08-09 09:39:42 +00:00
chenht2022
c1cc7d2cd2 1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic. 2024-08-08 09:04:36 +00:00
Atream
1f92f7cc61 [fix] linux and windows can all find CPUInfer in current Directory 2024-08-08 16:02:42 +08:00
UnicornChan
11544ef2b0
Merge pull request #25 from kvcache-ai/windows 
Windows Support
 2024-08-08 16:01:54 +08:00
Atream
0e613b602d [fix] recover fp16 support 2024-08-08 15:42:21 +08:00
chenxl
1d9d397525 fix some bug in compile in linux 2024-08-08 15:34:19 +08:00
Atream
0a2fd52cea support windows support q4_0 and q5_0 dequant on cpu Add CopyRight from pygguf(It was added before, but disappear after merge). Add some TODO in the code. 2024-08-08 15:34:02 +08:00
UnicornChan
442e13bc97
Merge pull request #18 from UnicornChan/update-docker-readme 
update docker.md to support docker pull image
 2024-08-01 15:40:47 +08:00
chenxl
69fa2f4395 update docker.md support docker pull image 2024-08-01 07:39:48 +00:00
UnicornChan
5e83bc0c82
Merge pull request #17 from UnicornChan/feature-support-multi-instruct 
Feature support multi instruct
 2024-08-01 12:29:07 +08:00
chenxl
86ba1336a9 [feature] add support for building docker image 2024-08-01 04:01:00 +00:00
chenxl
112cb3c962 [feature] support python 310 and multi instruction 2024-07-31 13:58:17 +00:00