From 01b7daf6b71dd173e64f6606258ddde0e1c0b2c1 Mon Sep 17 00:00:00 2001 From: henk717 Date: Fri, 16 Feb 2024 14:32:30 +0100 Subject: [PATCH] CUDA 12 support for the Conda Runtime (#680) * Remove libculibos dependency This dependency is something that is used to build libcudart which we are also already targeting. The individual file is no longer being distributed with the CUDA12 conda devkit, so we can no longer target it directly. But because all its functionality is inside libcudart we also don't need it. This commit removes the inclusion so that Koboldcpp can be compiled with CUDA12 as distributed by conda. I have tested this on the 1.57 release on CUDA11.5 and CUDA12.1. * Cleanup version definitions The package versions are already controlled by the label, we don't need to define it multiple times for it to work correctly. Removing the separate definitions allows people to easily change which version of CUDA they wish for their system. --- Makefile | 2 +- environment.yaml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index ecb6c9011..892f17e70 100644 --- a/Makefile +++ b/Makefile @@ -141,7 +141,7 @@ endif # it is recommended to use the CMAKE file to build for cublas if you can - will likely work better ifdef LLAMA_CUBLAS CUBLAS_FLAGS = -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/x86_64-linux/include - CUBLASLD_FLAGS = -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L$(CUDA_PATH)/targets/x86_64-linux/lib -L/usr/local/cuda/targets/aarch64-linux/lib -L/usr/lib/wsl/lib + CUBLASLD_FLAGS = -lcuda -lcublas -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L$(CUDA_PATH)/targets/x86_64-linux/lib -L/usr/local/cuda/targets/aarch64-linux/lib -L/usr/lib/wsl/lib CUBLAS_OBJS = ggml-cuda.o ggml_v3-cuda.o ggml_v2-cuda.o ggml_v2-cuda-legacy.o NVCC = nvcc NVCCFLAGS = --forward-unknown-to-host-compiler -use_fast_math diff --git a/environment.yaml b/environment.yaml index cc9e1fcc2..8ffc0ba04 100644 --- a/environment.yaml +++ b/environment.yaml @@ -5,8 +5,8 @@ channels: - defaults dependencies: - python=3.8 - - cuda-nvcc=11.5 - - cuda-libraries-dev=11.5 + - cuda-nvcc + - cuda-libraries-dev - cxx-compiler - gxx=10 - pip