vrr/unsloth
2
0
Fork 0
mirror of https://github.com/unslothai/unsloth.git synced 2026-07-09 15:58:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 29

Restore Unsloth CUDA tuning flags in the GPU llama.cpp configure

Browse source 
The CPU/GPU configure split dropped the GGML_CUDA tuning flags from the GPU
branch. Restore GGML_CUDA_FA_ALL_QUANTS=ON, GGML_CUDA_F16=OFF, GGML_CUDA_GRAPHS=OFF,
GGML_CUDA_FORCE_CUBLAS=OFF and GGML_CUDA_PEER_MAX_BATCH_SIZE=8192 inside the
if ($HasNvidiaSmi -and $NvccPath) block so GPU builds match the pre-split baseline.
The CPU-only path is unchanged.
This commit is contained in:
danielhanchen 2026-07-06 10:32:30 +00:00
parent 69d99a44fe
commit e717042454
1 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
studio/setup.ps1
View file

@ -1208,6 +1208,12 @@ if ((Test-Path $LlamaServerBin) -and -not $NeedRebuild) {
$CmakeArgs += "-DCUDAToolkit_ROOT=$CudaToolkitRoot"
$CmakeArgs += "-DCUDA_TOOLKIT_ROOT_DIR=$CudaToolkitRoot"
$CmakeArgs += "-DCMAKE_CUDA_COMPILER=$NvccPath"
# Unsloth-aligned CUDA tuning flags (restored -- keep GPU build behavior unchanged)
$CmakeArgs += '-DGGML_CUDA_FA_ALL_QUANTS=ON'
$CmakeArgs += '-DGGML_CUDA_F16=OFF'
$CmakeArgs += '-DGGML_CUDA_GRAPHS=OFF'
$CmakeArgs += '-DGGML_CUDA_FORCE_CUBLAS=OFF'
$CmakeArgs += '-DGGML_CUDA_PEER_MAX_BATCH_SIZE=8192'
if ($CudaArch) {
# Validate nvcc actually supports this architecture
if (Test-NvccArchSupport -NvccExe $NvccPath -Arch $CudaArch) {