vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-07 21:19:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
439 commits 41 branches 19 tags 46 MiB
Commit graph

439 commits

This branch
This branch
All branches
Author SHA1 Message Date
lazymio
b121ca4df8
Fix according to upstream changes 2025-02-27 18:11:35 +08:00
wang jiahao
26f7b4af11
Merge branch 'main' into temperature_top_p_from_request 2025-02-27 18:08:55 +08:00
Azure
1f28f75f55
Merge pull request #717 from kvcache-ai/issue-template 
Update issue templates
 2025-02-27 18:02:34 +08:00
Azure
c61805dd0a
Update issue templates 2025-02-27 17:53:27 +08:00
Atream
50c691297f
Merge pull request #622 from akemimadoka/fix-msvc 
Fix missing macro definition for KTRANSFORMERS_USE_CUDA and <chrono> includes on MSVC
 2025-02-27 17:42:00 +08:00
Atream
0422152cf3
Merge pull request #670 from akemimadoka/fix-win 
Fix RuntimeError on Windows caused by integer overflow in np.prod
 2025-02-27 17:40:27 +08:00
Atream
798e1d0cfa
Merge pull request #532 from xv44586/fix-sse-formatting 
fix: fix SSE formatting
 2025-02-27 12:19:23 +08:00
Atream
f403cde6d4
Merge pull request #650 from ceerRep/main 
feat: basic api key support
 2025-02-27 12:16:53 +08:00
Atream
1d5d5faef6
Merge pull request #626 from cyhasuka/main 
Feat: Clear cache during weight loading to prevent OOM on GPUs with <=8GB VRAM
 2025-02-27 12:13:10 +08:00
Atream
8db6a4d402
Merge branch 'main' into main 2025-02-27 12:12:32 +08:00
wang jiahao
3c8c580580
Merge pull request #691 from swu-hyk/ollama_api_chat 
feat:implementation of chat routing for Ollama
 2025-02-27 11:17:48 +08:00
Azure
ca93cf7548
Merge pull request #702 from Azure-Tang/update-readme 
[UPDATE] Update documents.
 2025-02-26 23:45:24 +08:00
Azure
c05ebb74b1 Update fp8 doc; Update install.md broken link 2025-02-26 15:43:08 +00:00
Atream
3ebe17eb63
Merge pull request #699 from kvcache-ai/Atream-patch-1 
Update DeepseekR1_V3_tutorial.md
 2025-02-26 22:04:45 +08:00
Atream
369f4d917d
Update DeepseekR1_V3_tutorial.md 2025-02-26 22:04:29 +08:00
Atream
9650893adc
Merge pull request #697 from kvcache-ai/fix-yaml 
Update DeepSeek-V3-Chat-multi-gpu-marlin.yaml
 2025-02-26 21:54:01 +08:00
Atream
90eb87b3fc
Update DeepSeek-V3-Chat-multi-gpu-marlin.yaml 2025-02-26 21:53:50 +08:00
swu-hyk
ec7e912fee modify 2025-02-26 19:21:30 +08:00
swu-hyk
68e7df3a25 implementation of chat routing for Ollama 2025-02-26 17:05:00 +08:00
Chen Hongtao
9660b2cc1e
Merge pull request #685 from vproxy-tools/main 
fix numa cpu distribution
 2025-02-26 15:35:19 +08:00
ZiWei Yuan
e7ebb26370
Merge pull request #684 from KMSorSMS/main 
fix dockerfile in devcontainer and fix expert torch
 2025-02-26 15:06:51 +08:00
liam
ffb86c66e3 ⚡ fix experts torch 2025-02-26 15:04:40 +08:00
liam
de082f141c ⚡ fix cd error 2025-02-26 14:54:47 +08:00
wkgcass
b2bff17775 fix numa cpu distribution 
The numa node location would be calculated based on the total number
of worker threads.
So we should always use the actual number of threads instead of using a min() op.
 2025-02-26 14:49:57 +08:00
akemimadoka
8817777e11 Fix RuntimeError on Windows caused by integer overflow in np.prod 2025-02-26 03:50:12 +08:00
Azure
99f6e42113
Merge pull request #668 from KMSorSMS/main 
📝 update benchmark.md
 2025-02-26 00:21:09 +08:00
liam
3ad12751cf 📝 update more detail and fix typo 2025-02-26 00:17:02 +08:00
Azure
31bc990677
Merge pull request #667 from Azure-Tang/update-readme 
[update] Update doc.
 2025-02-26 00:01:46 +08:00
liam
05339ad0ef 📝 update benchmark.md 2025-02-25 23:57:58 +08:00
Azure
bb6920ed72 update doc 2025-02-25 15:46:15 +00:00
ZiWei Yuan
9c71bcb0bb
Merge pull request #665 from KMSorSMS/v0.2.2rc1 
⚡ release v0.2.2rc1
 2025-02-25 22:07:19 +08:00
liam
ddf3339339 ⚡ release v0.2.2rc1 2025-02-25 22:06:36 +08:00
Azure
8333a4d874
Merge pull request #663 from kvcache-ai/develop-0.2.2 
[release]  Release 0.2.2rc.
 2025-02-25 21:47:36 +08:00
Azure
c6e4e1c3c5
Merge pull request #662 from Azure-Tang/support-fp8 
[update] Update readme.
 2025-02-25 21:45:19 +08:00
Azure
91c1619296 Merge branch 'develop-0.2.2' into support-fp8 
Update README.md
 2025-02-25 13:43:26 +00:00
Atream
13974eb264
Update DeepseekR1_V3_tutorial.md 2025-02-25 21:36:52 +08:00
Atream
03f8bc9f79
Update DeepseekR1_V3_tutorial.md add long context 2025-02-25 21:35:31 +08:00
Azure
2c0cce90d0 add fp8 multi gpu yaml example 2025-02-25 13:32:09 +00:00
Atream
d9b2895bd3 Merge branch 'fix-update-flashinfer_wrapper_local_chat' into develop-0.2.2 2025-02-25 12:47:48 +00:00
Atream
477ac28a9c fix-update-flashinfer_wrapper_local_chat 2025-02-25 12:47:31 +00:00
Azure
7e5962af3d fix fp8 multi gpu; update FQA 2025-02-25 10:52:29 +00:00
ZiWei Yuan
89b55052b8
Merge pull request #659 from KMSorSMS/develop-0.2.2 
📝 add benchmark.md
 2025-02-25 17:47:05 +08:00
liam
1b5ac67fca 📝 add benchmark.md 2025-02-25 17:45:17 +08:00
ZiWei Yuan
1aa10e93b3
Merge pull request #658 from KMSorSMS/develop-0.2.2 
⚡ update git ignore add docker dev container
 2025-02-25 17:22:34 +08:00
liam
0ca0b99fab ⚡ update git ignore add docker dev container 2025-02-25 17:22:11 +08:00
Azure
5474be5299 Merge branch 'main' into develop-0.2.2 2025-02-25 09:04:22 +00:00
Azure
021822dd01 update FAQ 2025-02-25 09:02:32 +00:00
Atream
b443c7dfa2
Merge pull request #657 from kvcache-ai/feat-absorb-for-long-prefill 
Feat absorb for long prefill
 2025-02-25 16:53:21 +08:00
Atream
f4c198bd42 support absorb for prefill long context 2025-02-25 08:52:02 +00:00
Azure
050b745a6e
Merge pull request #643 from Azure-Tang/support-fp8 
[feat] Support fp8 linear kernel;
 2025-02-25 16:22:12 +08:00