From d82d7b60e69ff52a67d535d8aea5205a21246a8f Mon Sep 17 00:00:00 2001
From: DontEatOreo <57304299+DontEatOreo@users.noreply.github.com>
Date: Tue, 26 Nov 2024 10:18:45 +0200
Subject: [PATCH] Refactor Nix section (#1235)

* README.md: remove `nix{3-run,shell}` example

* README.md: better re-word NVIDIA CUDA section for nix

* README.md: remove unneed section in nix

* README.md: add section to open issue regarding nix version on nixpkgs

* README.md: add clarification on how to add KoboldCpp to `home-manager`

* README.md: add example in nix section
---
 README.md | 28 +++++++++++++++++++++++-----
 1 file changed, 23 insertions(+), 5 deletions(-)

diff --git a/README.md b/README.md
index 0e63783c0..830cff5f7 100644
--- a/README.md
+++ b/README.md
@@ -128,12 +128,30 @@ when you can't use the precompiled binary directly, we provide an automated buil
 - These unofficial resources have been contributed by the community, and may be outdated or unmaintained. No official support will be provided for them!
   - Arch Linux Packages: [CUBLAS](https://aur.archlinux.org/packages/koboldcpp-cuda), and [HIPBLAS](https://aur.archlinux.org/packages/koboldcpp-hipblas).
   - Unofficial Dockers: [korewaChino](https://github.com/korewaChino/koboldCppDocker) and [noneabove1182](https://github.com/noneabove1182/koboldcpp-docker)
-  - Nix & NixOS: KoboldCpp is available on Nixpkgs and can be installed by adding just `koboldcpp` to your `environment.systemPackages`.
-    - Make sure to have `nixpkgs.config.allowUnfree`, `hardware.opengl.enable` *(`hardware.graphics.enable` if you're using unstable channel)* and `nixpkgs.config.cudaSupport` set to `true` to enable CUDA.
+  - Nix & NixOS: KoboldCpp is available on Nixpkgs and can be installed by adding just `koboldcpp` to your `environment.systemPackages` *(or it can also be placed in `home.packages`)*.
+    - In order to enable NVIDIA CUDA support, you will need to set `nixpkgs.config.allowUnfree`, `hardware.opengl.enable` (*`hardware.graphics.enable` if you're using 24.11 or the unstable channel*) and `nixpkgs.config.cudaSupport` to `true`, and set `nixpkgs.config.cudaArches` *(e.g., if you have an RTX 2080, you need to set `cudaArches` to `[ "sm_75" ]`)* to your GPU architecture. Find your architecture here: [Matching CUDA arch and CUDA gencode for various NVIDIA architectures](https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/)
+    - Make sure to configure `nixpkgs` settings again/separately for home-manager if `home-manager.useGlobalPkgs` is set to `false`.
     - Metal is enabled by default on macOS, Vulkan support is enabled by default on both Linux and macOS, ROCm support isn't available yet.
-    - You can also use `nix3-run` to use KoboldCpp: `nix run --expr ``with import <nixpkgs> { config = { allowUnfree = true; cudaSupport = true; }; }; koboldcpp`` --impure`
-    - Or use `nix-shell`: `nix-shell --expr 'with import <nixpkgs> { config = { allowUnfree = true; cudaSupport = true; }; }; koboldcpp' --run "koboldcpp" --impure`
-    - Packages (like CLBLast, Vulkan, etc.) can be overridden, please refer to the [17th Nix Pill - Nixpkgs Overriding Packages](https://nixos.org/guides/nix-pills/17-nixpkgs-overriding-packages)
+    - Example Nix Setup:
+    ```nix
+    nixpkgs.config = {
+      allowUnfree = true;
+      cudaSupport = true;
+      cudaArches = [ "sm_75" ];
+    };
+
+    # NixOS 24.05
+    hardware.opengl.enable = true;
+
+    # NixOS 24.11 or unstable
+    # hardware.graphics.enable = true;
+
+    environment.systemPackages = [ pkgs.koboldcpp ];
+
+    # If you're using home-manager to install KoboldCpp
+    # home.packages = [ pkgs.koboldcpp ];
+    ```
+    - If you face any issues with running KoboldCpp on Nix, please open an issue [here](https://github.com/NixOS/nixpkgs/issues/new?assignees=&labels=0.kind%3A+bug&projects=&template=bug_report.md&title=).
 - [GPTLocalhost](https://gptlocalhost.com/demo#KoboldCpp) - KoboldCpp is supported by GPTLocalhost, a local Word Add-in for you to use KoboldCpp in Microsoft Word. A local alternative to "Copilot in Word."
 
 ## Questions and Help Wiki