mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2025-09-10 17:14:36 +00:00
f6aebb1449
* examples/nix_example.md: fix markdownlint complaints * examples/nix_example.md: add home manager section * examples/nix_example.md: add hardware support section * examples/nix_example.md: rewrite CUDA section --------- Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
70 lines
2.3 KiB
Markdown
70 lines
2.3 KiB
Markdown
# Guide to Nix for KoboldCpp
|
|
|
|
- KoboldCpp is available on Nixpkgs and can be installed by adding just `koboldcpp` to your `environment.systemPackages` *(or it can also be placed in `home.packages`)*.
|
|
|
|
## KoboldCpp Nix - CUDA Support
|
|
|
|
In order to enable NVIDIA CUDA support, you'll need to configure several settings:
|
|
|
|
- Enable required options:
|
|
|
|
```nix
|
|
nixpkgs.config.allowUnfree = true; # Allow proprietary software
|
|
nixpkgs.config.cudaSupport = true; # Enable CUDA functionality
|
|
```
|
|
|
|
- Enable graphics support based on your NixOS version:
|
|
|
|
```nix
|
|
# For NixOS 24.05:
|
|
hardware.opengl.enable = true;
|
|
|
|
# For NixOS 24.11 or unstable:
|
|
hardware.graphics.enable = true;
|
|
```
|
|
|
|
- Set your GPU architecture:
|
|
|
|
```nix
|
|
nixpkgs.config.cudaArches = [ "sm_75" ]; # Example for RTX 2080
|
|
```
|
|
|
|
To find your GPU's architecture code:
|
|
|
|
1. Visit the [NVIDIA Architecture Guide](https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/)
|
|
2. Locate your GPU Architecture
|
|
3. Use the corresponding `sm_XX` code in your configuration
|
|
|
|
## Hardware Support
|
|
|
|
- ✅ Vulkan: Enabled by default on Linux and macOS
|
|
- ✅ Metal: Enabled by default on macOS
|
|
- ❌ ROCm: Not currently available
|
|
|
|
## Example KoboldCpp Nix Configuration
|
|
|
|
```nix
|
|
nixpkgs.config = {
|
|
allowUnfree = true;
|
|
cudaSupport = true;
|
|
cudaArches = [ "sm_75" ];
|
|
};
|
|
# NixOS 24.05
|
|
hardware.opengl.enable = true;
|
|
# NixOS 24.11 or unstable
|
|
# hardware.graphics.enable = true;
|
|
environment.systemPackages = [ pkgs.koboldcpp ];
|
|
# If you're using home-manager to install KoboldCpp
|
|
# home.packages = [ pkgs.koboldcpp ];
|
|
```
|
|
|
|
## KoboldCpp - Home Manager
|
|
|
|
The setup for Home Manager is the same as regular Nix, with one exception regarding Home Manager's instance of nixpkgs. By default, Home Manager manages its own isolated instance of nixpkgs, which has two implications:
|
|
|
|
1. You can keep your private Home Manager nixpkgs instance and simply repeat your `nixpkgs.config` in home manager.
|
|
2. You can set `home-manager.useGlobalPkgs = true;` to copy your module system's nixpkgs instance. This way, you only need to define it in your `configuration.nix`, and Home Manager will "inherit" this configuration.
|
|
|
|
## Getting Help for KoboldCpp Nix
|
|
|
|
- If you face any issues with running KoboldCpp on Nix, please open an issue [here](https://github.com/NixOS/nixpkgs/issues/new?assignees=&labels=0.kind%3A+bug&projects=&template=bug_report.md&title=)
|