vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-14 19:05:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

stick to cu12.1 for linux for now

Browse source
This commit is contained in:
Concedo 2025-06-06 17:38:28 +08:00
parent 9cf32e5fee
commit 8b141d8647
5 changed files with 6 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.github/workflows/kcpp-build-release-linux-cuda12.yaml vendored
View file

@ -10,7 +10,7 @@ on:
env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
KCPP_CUDA: 12.4.0
KCPP_CUDA: 12.1.0
ARCHES_CU12: 1
jobs:

3
.github/workflows/kcpp-build-release-linux.yaml vendored
View file

@ -52,9 +52,6 @@ jobs:
run: |
./koboldcpp.sh dist
- name: Rename file before upload
run: mv dist/koboldcpp-linux-x64-cuda1150 dist/koboldcpp-linux-x64-cuda11
- name: Save artifact
uses: actions/upload-artifact@v4
with:

4
README.md
View file

@ -33,11 +33,11 @@ KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models
- You can also run it using the command line. For info, please check `koboldcpp.exe --help`
## Linux Usage (Precompiled Binary, Recommended)
On modern Linux systems, you should download the `koboldcpp-linux-x64-cuda11` prebuilt PyInstaller binary for greatest compatibility on the **[releases page](https://github.com/LostRuins/koboldcpp/releases/latest)**. Simply download and run the binary (You may have to `chmod +x` it first). If you have a newer device, you can also try the `koboldcpp-linux-x64-cuda12` instead for better speeds.
On modern Linux systems, you should download the `koboldcpp-linux-x64-cuda1150` prebuilt PyInstaller binary for greatest compatibility on the **[releases page](https://github.com/LostRuins/koboldcpp/releases/latest)**. Simply download and run the binary (You may have to `chmod +x` it first). If you have a newer device, you can also try the `koboldcpp-linux-x64-cuda1210` instead for better speeds.
Alternatively, you can also install koboldcpp to the current directory by running the following terminal command:
```
curl -fLo koboldcpp https://github.com/LostRuins/koboldcpp/releases/latest/download/koboldcpp-linux-x64-cuda11 && chmod +x koboldcpp
curl -fLo koboldcpp https://github.com/LostRuins/koboldcpp/releases/latest/download/koboldcpp-linux-x64-cuda1150 && chmod +x koboldcpp
```
After running this command you can launch Koboldcpp from the current directory using `./koboldcpp` in the terminal (for CLI usage, run with `--help`).
Finally, obtain and load a GGUF model. See [here](#Obtaining-a-GGUF-model)

2
environment.yaml
View file

@ -1,6 +1,6 @@
name: koboldcpp
channels:
- nvidia/label/cuda-12.4.0
- nvidia/label/cuda-12.1.0
- conda-forge
- defaults
dependencies:

4
koboldcpp.sh
View file

@ -7,9 +7,9 @@ fi
if [[ ! -f "conda/envs/linux/bin/python" && $KCPP_CUDA != "rocm" || $1 == "rebuild" && $KCPP_CUDA != "rocm" ]]; then
cp environment.yaml environment.tmp.yaml
if [ -n "$KCPP_CUDA" ]; then
sed -i -e "s/nvidia\/label\/cuda-12.4.0/nvidia\/label\/cuda-$KCPP_CUDA/g" environment.tmp.yaml
sed -i -e "s/nvidia\/label\/cuda-12.1.0/nvidia\/label\/cuda-$KCPP_CUDA/g" environment.tmp.yaml
else
KCPP_CUDA=12.4.0
KCPP_CUDA=12.1.0
fi
bin/micromamba create --no-rc --no-shortcuts -r conda -p conda/envs/linux -f environment.tmp.yaml -y
bin/micromamba create --no-rc --no-shortcuts -r conda -p conda/envs/linux -f environment.tmp.yaml -y