vrr/unsloth
2
0
Fork 0
mirror of https://github.com/unslothai/unsloth.git synced 2026-07-09 15:58:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 29
mirror of https://github.com/unslothai/unsloth
3 commits 357 branches 55 tags 291 MiB
Find a file
Daniel Han-Chen c7d595b8ad requirements
2023-11-30 16:50:44 +11:00
unsloth First upload of Unsloth code 2023-11-30 03:51:54 +11:00
.gitignore Initial commit 2023-11-30 03:50:09 +11:00
LICENSE Initial commit 2023-11-30 03:50:09 +11:00
pyproject.toml First upload of Unsloth code 2023-11-30 03:51:54 +11:00
README.md First upload of Unsloth code 2023-11-30 03:51:54 +11:00
requirements_cu118.txt requirements 2023-11-30 16:50:44 +11:00
requirements_cu121.txt requirements 2023-11-30 16:50:44 +11:00

README.md

Unsloth

2x faster 50% less memory LLM finetuning on a single GPU.

!pip install "unsloth[cu118] @ git+https://github.com/unslothai/unsloth.git" !pip install "unsloth[cu121] @ git+https://github.com/unslothai/unsloth.git"

Google Colab examples

  1. Unsloth fast finetuning example
  2. Original slow finetuning example

Installation instructions

In Google Colab:

!ldconfig /usr/lib64-nvidia
!pip install xformers --index-url https://download.pytorch.org/whl/cu118
!pip install git+https://github.com/danielhanchen/unsloth.git

!ldconfig /usr/lib64-nvidia is necessary (for now) to link CUDA with Python. Possibly a Google Colab linking bug.

For general installations:

  1. Install Xformers OR Flash Attention. Choose 1. Old GPUs use Xformers. New use Flash Attention.
  2. For Xformers, find your Pytorch CUDA version via torch.version.cuda or nvidia-smi.
    • If you have Conda, conda install xformers -c xformers
    • If you have CUDA 11.8, pip install xformers --index-url https://download.pytorch.org/whl/cu118
    • If you have CUDA 12.1, pip install xformers --index-url https://download.pytorch.org/whl/cu121
    • Go to https://github.com/facebookresearch/xformers for other issues.
    • You must have Pytorch 2.1 installed for Xformers. If not, try Flash Attention.
    • Xformers supports all GPUs (Tesla T4 etc).
  3. For Flash Attention, you must have a Ampere, Ada, Hopper GPU (A100, RTX 3090, RTX 4090, H100).
    • Install Flash Attention via pip uninstall -y ninja && pip install ninja then pip install flash-attn --no-build-isolation.
    • Xformers has native support for Flash Attention, so technically installing Xformers is enough.
  4. Then install Unsloth: pip install git+https://github.com/danielhanchen/unsloth.git