vrr/unsloth
2
0
Fork 0
mirror of https://github.com/unslothai/unsloth.git synced 2026-07-10 00:08:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 29

Fix packaging, readme

Browse source
This commit is contained in:
Daniel Han-Chen 2023-12-15 12:57:08 +11:00
parent 4143ca52e1
commit 9c6803a573
2 changed files with 8 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

7
README.md
View file

@ -6,9 +6,10 @@
## 2-5x faster 60% less memory local QLoRA finetuning
* Supports Llama 7b, 13b, 70b, CodeLlama 34b, Mistral 7b, TinyLlama and all Llama archs!
* Mistral 7b [(Colab example)](https://colab.research.google.com/drive/1SKrKGV-BZoU4kv5q3g0jtE_OhRgPtrrQ?usp=sharing) on 1 A100 2.2x faster, uses 62% less VRAM (12.4GB).
* CodeLlama 34b [(Colab example)](https://colab.research.google.com/drive/1gdHyAx8XJsz2yNV-DHvbHjR1iCef5Qmh?usp=sharing) does not OOM is 1.9x faster, uses 32% less VRAM (27GB).
* Kaggle 2 Tesla T4s 5.28x faster on Alpaca. [(Kaggle example)](https://www.kaggle.com/danielhanchen/unsloth-laion-t4-ddp)
* Llama 7b [Colab T4 example](https://colab.research.google.com/drive/1n-fgduZhRUsSjgpqNtVkXA3rSfE7iBdg?usp=sharing) on 1 T4 2x faster, uses 43% less VRAM (8.4GB) LAION dataset. [Alpaca T4 example](https://colab.research.google.com/drive/1oW55fBmwzCOrBVX66RcpptL3a99qWBxb?usp=sharing) 2x faster on 1 T4, using 6.4GB VRAM.
* Mistral 7b [Colab A100 example](https://colab.research.google.com/drive/1SKrKGV-BZoU4kv5q3g0jtE_OhRgPtrrQ?usp=sharing) on 1 A100 2.2x faster, uses 62% less VRAM (12.4GB).
* CodeLlama 34b [Colab example](https://colab.research.google.com/drive/1gdHyAx8XJsz2yNV-DHvbHjR1iCef5Qmh?usp=sharing) does not OOM is 1.9x faster, uses 32% less VRAM (27GB).
* Kaggle 2 Tesla T4s 5.28x faster on Alpaca. [Kaggle example](https://www.kaggle.com/danielhanchen/unsloth-laion-t4-ddp)
* All kernels written in [OpenAI's Triton](https://openai.com/research/triton) language.
* 0% loss in accuracy - no approximation methods - all exact.
* No change of hardware necessary. Supports NVIDIA GPUs since 2018+. Minimum CUDA Compute Capability 7.0 (V100, T4, Titan V, RTX 20, 30, 40x, A100, H100, L40 etc) [Check your GPU](https://developer.nvidia.com/cuda-gpus)

4
pyproject.toml
View file

@ -39,6 +39,7 @@ huggingface = [
"accelerate",
"trl",
"peft",
"packaging",
]
cu118only = [
"xformers @ https://download.pytorch.org/whl/cu118/xformers-0.0.22.post7%2Bcu118-cp39-cp39-manylinux2014_x86_64.whl ; python_version=='3.9'",
@ -70,16 +71,19 @@ cu118_ampere = [
"unsloth[huggingface]",
"bitsandbytes",
"unsloth[cu118only]",
"ninja",
"flash-attn",
]
cu121_ampere = [
"unsloth[huggingface]",
"bitsandbytes",
"unsloth[cu121only]",
"ninja",
"flash-attn",
]
colab_ampere = [
"unsloth[cu118]",
"ninja",
"flash-attn",
]