vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-07 21:19:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
322 commits 41 branches 19 tags 46 MiB
Commit graph

322 commits

This branch
This branch
All branches
Author SHA1 Message Date
liam
107e4be417 ⚡ fix typo 2025-02-10 10:50:40 +08:00
liam
910d8c842a Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktrans_v0.2_dev into feat-DeepSeekV3 2025-02-10 10:15:45 +08:00
liam
cff68532ce ⚡ fix typo 2025-02-10 09:52:48 +08:00
unicornchan
e968fa8d72 [feature] add flash_attn to requirements 2025-02-10 01:52:39 +00:00
liam
fd481af193 ⚡ update v0.3 preview 2025-02-10 09:49:56 +08:00
liam
6dd4fa0e87 ⚡ improve readme 2025-02-10 09:38:26 +08:00
unicornchan
fd8037cda1 Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktrans_v0.2_dev into feat-DeepSeekV3 2025-02-10 01:01:14 +00:00
unicornchan
c7e6d09068 [feature] update version and github action jobs for package 2025-02-10 01:00:57 +00:00
chenht2022
2d684ee96a Small fix 2025-02-09 16:25:43 +00:00
chenht2022
6b33f41de4 Add V0.3-preview doc 2025-02-09 16:08:16 +00:00
liam
098602b08f ⚡ v0.2 ongoing 2025-02-09 22:41:14 +08:00
liam
bf1d413be0 Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3 2025-02-08 13:17:10 +08:00
liam
c18ecd7b7f ⚡ add flush print in local_chat output and change default optimize yaml of deepseekv3 to single gpu 2025-02-08 13:15:52 +08:00
RodriMora
b1bff2a405 Added simple /models endpoint to work with frontends that don't allow bypass check like Openweb-ui 2025-02-07 10:30:39 +01:00
Azure
c4d9bc6670 support KExpertsMarlin backend 2025-02-07 05:57:40 +00:00
liam
0262f954c7 Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3 2025-02-06 22:41:25 +08:00
liam
3dca28d23b ⚡ fix moe.cpp int overflow problem 2025-02-06 22:39:16 +08:00
Azure
027b11266c modify moeinfer param 2025-02-06 14:07:38 +00:00
Azure
ee24a27001 update v3 single gpu rule yaml; 2025-02-04 16:14:35 +00:00
Azure
907251c743 done support deepseekv3 2025-02-04 15:53:38 +00:00
Azure
f748cd29f0 fix rope; update moegate 2025-02-01 18:05:45 +00:00
Azure
f873558a89 update rope calculation; update modeling.py; update gate for moe 2025-02-01 07:32:21 +00:00
Azure
5a50b34627 fix hard coding caused by rope dim calculation, load from config now 2025-01-31 15:25:50 +00:00
Azure
476b1d8dc6 support deepseekv3; runable but have precition problem 2025-01-31 08:27:24 +00:00
UnicornChan
de7e892f72
Merge pull request #115 from KMSorSMS/main 
⚡ rm opt config path default value and fix some config logic bug
 2024-11-14 20:07:12 +08:00
liam
04cebec4bb ⚡ rm opt config path default value and fix some config logic bug 2024-11-14 20:02:30 +08:00
UnicornChan
dddc42038d
Merge pull request #110 from KMSorSMS/main 
refactor local_chat & config setting
 2024-11-06 09:44:12 +08:00
liam
28ae7e24ed ✨: finish patch 2024-11-04 14:05:03 +08:00
liam
c2b4dc805c 🚑️:roll back transformer.py and find that it's multiple chat hsitory have minor accurate error 2024-11-04 14:02:19 +08:00
liam
a148da2cfe ✨: rm sensitive info in config.yaml, add readme of makefile. support old model_path config 2024-11-04 14:02:19 +08:00
anyanqilin
9a2e7057c8 wjh fix change 2024-11-04 14:02:19 +08:00
anyanqilin
a72dc6ed15 wjh change 2024-11-04 14:02:19 +08:00
anyanqilin
2d67016d14 wjh-change 2024-11-04 14:02:19 +08:00
liam
7c94df4bcf 🚑️: back transformer.py bugs version, and fix typo error in local_chat.py 2024-11-04 14:02:19 +08:00
liam
dd1d8667f3 ✨: refactor local_chat and fix message slice bug in server 2024-11-04 14:02:19 +08:00
UnicornChan
a6a1cc054f
Merge pull request #106 from Azure-Tang/main 
[Fix] Fix readme structure.
 2024-10-30 16:36:59 +08:00
TangJingqi
c7d62a67db update supported models 2024-10-28 14:37:10 +08:00
TangJingqi
d8ddaf0ea0 fix readme 2024-10-28 14:21:12 +08:00
Chen Hongtao
43fc7f44a6
Merge pull request #99 from chenht2022/main 
Adapt Windows
 2024-10-09 19:09:58 +08:00
chenht2022
14869b55ad Adapt Windows 2024-10-09 11:08:32 +00:00
UnicornChan
a81a7ffe21
Merge pull request #77 from TKONIY/fix-prefill-and-generate 
Fix: Wrong type of token list returned by prefill_and_generate
 2024-10-09 19:04:27 +08:00
Chen Hongtao
b4904537e3
Merge pull request #83 from sayap/task-queue-cond-var 
Use cond var to avoid busy loop
 2024-10-09 18:57:17 +08:00
UnicornChan
43e8848ddd
Merge pull request #86 from xhedit/main 
typo fix: KMisrtal -> KMistral
 2024-09-19 10:37:55 +08:00
UnicornChan
49539ac424
Merge pull request #88 from Azure-Tang/main 
[fix] Fix some gpu dequant function doesn't support multi gpu bug
 2024-09-15 15:17:05 +08:00
Azure
7953bd9c0d update readme 2024-09-13 08:47:35 +00:00
Azure
3758afb526 fix some dequant function dosen't support multi gpu bug 2024-09-13 08:34:23 +00:00
xhedit
234faf7987 typo fix: KMisrtal -> KMistral 2024-09-12 15:58:01 +00:00
Yap Sok Ann
6666d62237 Use cond var to avoid busy loop 2024-09-11 16:10:54 +07:00
UnicornChan
3ed8a0437f
Merge pull request #72 from sayap/dequantize-iq4-xs 
Support IQ4_XS dequantize
 2024-09-06 10:06:10 +08:00
yangshen
ee72cee050 Fix: the tokens return by prefill_and_generate 2024-09-05 05:29:23 +00:00