vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-08 05:29:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #128 from kvcache-ai/doc_add

Browse source 
📝 update
This commit is contained in:
ZiWei Yuan 2025-02-10 22:15:40 +08:00 • committed by GitHub
commit dbaecd0ca5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 34 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
doc/en/DeepseekR1_V3_tutorial.md
View file

@ -1,4 +1,25 @@
<!-- omit in toc -->
# GPT-4/o1-level Local VSCode Copilot on a Desktop with only 24GB VRAM
- [SUMMARY](#summary)
- [Prerequisites](#prerequisites)
- [Bench Result](#bench-result)
- [V0.2](#v02)
- [Settings](#settings)
- [Memory consumption:](#memory-consumption)
- [Benchmark Results](#benchmark-results)
- [V0.3-Preview](#v03-preview)
- [Settings](#settings-1)
- [Memory consumptions:](#memory-consumptions)
- [Benchmark results](#benchmark-results-1)
- [How to Run](#how-to-run)
- [V0.2 Showcase](#v02-showcase)
- [Single socket version (32 cores)](#single-socket-version-32-cores)
- [Dual socket version (64 cores)](#dual-socket-version-64-cores)
- [V0.3 Showcase](#v03-showcase)
- [Dual socket version (64 cores)](#dual-socket-version-64-cores-1)
- [Some Explanations](#some-explanations)
- [FAQ](#faq)
# SUMMARY
> **Fed 10, 2025**: Support DeepseekR1 and V3 on single (24GB VRAM)/multi gpu and 382G DRAM, up to 3~28x speedup.<br>
@ -137,4 +158,6 @@ DeepSeek's MLA operators are highly computationally intensive. While running eve
- Intel AMX Optimization – Our AMX-accelerated kernel is meticulously tuned, running several times faster than existing llama.cpp implementations. We plan to open-source this kernel after cleansing and are considering upstream contributions to llama.cpp.
5. Why Intel CPUs?
Intel is currently the only CPU vendor that supports AMX-like instructions, which delivers significantly better performance compared to AVX-only alternatives.
Intel is currently the only CPU vendor that supports AMX-like instructions, which delivers significantly better performance compared to AVX-only alternatives.
## FAQ
[See detail](./FAQ.md)

10
doc/en/FAQ.md Normal file
View file

@ -0,0 +1,10 @@
# FAQ
## Install
### 1 ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.32' not found
```
in Ubuntu 22.04 installation need to add the:
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install --only-upgrade libstdc++6
```
from-https://github.com/kvcache-ai/ktransformers/issues/117#issuecomment-2647542979