vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-11 07:44:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix local_chat and update balance-serve and SUMMARY doc

Browse source
This commit is contained in:
dongjw 2025-04-03 12:19:43 +08:00
parent 8acb270c90
commit ab0b0f4ea1
3 changed files with 6 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

5
doc/en/balance-serve.md
View file

@ -74,6 +74,9 @@ strings ~/anaconda3/envs/ktransformers/lib/libstdc++.so.6 | grep GLIBCXX
```bash
sudo apt install libtbb-dev libssl-dev libcurl4-openssl-dev libaio1 libaio-dev libfmt-dev libgflags-dev zlib1g-dev patchelf
pip3 install packaging ninja cpufeature numpy openai
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
```
### 3. Build ktransformers
@ -87,7 +90,7 @@ git submodule update --init --recursive
# Install single NUMA dependencies
USE_BALANCE_SERVE=1 bash ./install.sh
# Or Install Dual NUMA dependencies
# For those who have two cpu and 1T RAM（Dual NUMA）:
USE_BALANCE_SERVE=1 USE_NUMA=1 bash ./install.sh
```