vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-11 01:24:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

updated readme, memory detection prints

Browse source
This commit is contained in:
Concedo 2025-04-08 20:23:52 +08:00
parent 11c4e7c2c4
commit 8e23a087e7
2 changed files with 5 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -19,7 +19,7 @@ KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models
- Includes multiple modes (chat, adventure, instruct, storywriter) and UI Themes (aesthetic roleplay, classic writer, corporate assistant, messsenger) - Includes multiple modes (chat, adventure, instruct, storywriter) and UI Themes (aesthetic roleplay, classic writer, corporate assistant, messsenger)
- Supports loading Tavern Character Cards, importing many different data formats from various sites, reading or exporting JSON savefiles and persistent stories. - Supports loading Tavern Character Cards, importing many different data formats from various sites, reading or exporting JSON savefiles and persistent stories.
- Many other features including new samplers, regex support, websearch, RAG via TextDB and more. - Many other features including new samplers, regex support, websearch, RAG via TextDB and more.
- Ready-to-use binaries for Windows, MacOS, Linux, Android (via Termux), Colab, Docker, also supports other platforms if self-compiled (like Raspberry PI). - Ready-to-use binaries for Windows, MacOS, Linux. Runs directly with Colab, Docker, also supports other platforms if self-compiled (like Android (via Termux) and Raspberry PI).
- [Need help finding a model? Read this!](https://github.com/LostRuins/koboldcpp/wiki#getting-an-ai-model-file) - [Need help finding a model? Read this!](https://github.com/LostRuins/koboldcpp/wiki#getting-an-ai-model-file)
## Windows Usage (Precompiled Binary, Recommended) ## Windows Usage (Precompiled Binary, Recommended)
@ -31,7 +31,7 @@ KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models
- You can also run it using the command line. For info, please check `koboldcpp.exe --help` - You can also run it using the command line. For info, please check `koboldcpp.exe --help`
## Linux Usage (Precompiled Binary, Recommended) ## Linux Usage (Precompiled Binary, Recommended)
On modern Linux systems, you should download the `koboldcpp-linux-x64-cuda1150` prebuilt PyInstaller binary on the **[releases page](https://github.com/LostRuins/koboldcpp/releases/latest)**. Simply download and run the binary (You may have to `chmod +x` it first). On modern Linux systems, you should download the `koboldcpp-linux-x64-cuda1150` prebuilt PyInstaller binary for greatest compatibility on the **[releases page](https://github.com/LostRuins/koboldcpp/releases/latest)**. Simply download and run the binary (You may have to `chmod +x` it first). If you have a newer device, you can also try the `koboldcpp-linux-x64-cuda1210` instead for better speeds.
Alternatively, you can also install koboldcpp to the current directory by running the following terminal command: Alternatively, you can also install koboldcpp to the current directory by running the following terminal command:
``` ```

10
koboldcpp.py
View file

@ -978,7 +978,6 @@ def fetch_gpu_properties(testCL,testCU,testVK):
FetchedCUdevices = [] FetchedCUdevices = []
FetchedCUdeviceMem = [] FetchedCUdeviceMem = []
FetchedCUfreeMem = [] FetchedCUfreeMem = []
faileddetectvram = False
AMDgpu = None AMDgpu = None
try: # Get NVIDIA GPU names try: # Get NVIDIA GPU names
@ -989,7 +988,6 @@ def fetch_gpu_properties(testCL,testCU,testVK):
except Exception: except Exception:
FetchedCUdeviceMem = [] FetchedCUdeviceMem = []
FetchedCUfreeMem = [] FetchedCUfreeMem = []
faileddetectvram = True
pass pass
if len(FetchedCUdevices)==0: if len(FetchedCUdevices)==0:
try: # Get AMD ROCm GPU names try: # Get AMD ROCm GPU names
@ -1011,7 +1009,6 @@ def fetch_gpu_properties(testCL,testCU,testVK):
except Exception: except Exception:
FetchedCUdeviceMem = [] FetchedCUdeviceMem = []
FetchedCUfreeMem = [] FetchedCUfreeMem = []
faileddetectvram = True
pass pass
lowestcumem = 0 lowestcumem = 0
lowestfreecumem = 0 lowestfreecumem = 0
@ -1030,14 +1027,13 @@ def fetch_gpu_properties(testCL,testCU,testVK):
except Exception: except Exception:
lowestcumem = 0 lowestcumem = 0
lowestfreecumem = 0 lowestfreecumem = 0
faileddetectvram = True
if faileddetectvram:
print("Unable to detect VRAM, please set layers manually.")
MaxMemory[0] = max(lowestcumem,MaxMemory[0]) MaxMemory[0] = max(lowestcumem,MaxMemory[0])
MaxFreeMemory[0] = max(lowestfreecumem,MaxFreeMemory[0]) MaxFreeMemory[0] = max(lowestfreecumem,MaxFreeMemory[0])
if MaxMemory[0] < (1024*1024*256):
print("Unable to detect VRAM, please set layers manually.")
if testVK: if testVK:
try: # Get Vulkan names try: # Get Vulkan names
foundVkGPU = False foundVkGPU = False