mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-09 15:58:41 +00:00
mirror of https://github.com/unslothai/unsloth
agentaideepseekdeepseek-r1fine-tuninggemmagemma3llamallama3llmllmsloramistralopenaipytorchqwenqwen3text-to-speechttsunsloth
b8400f40df
* replaced connect with start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix * Studio: build the coding-agent command from the selected server The API keys panel showed a hardcoded `unsloth start claude`. `unsloth start` defaults to 127.0.0.1:8888 and only mints a key for a loopback server, so a non-default port or a tunnel/remote base would target the wrong server or fail to mint. Build the command from the panel base/key (and emit a key for non-loopback), matching the other snippets in the panel. * CLI: keep `unsloth connect` as a hidden alias for `unsloth start` Avoids breaking existing scripts and docs that still call `unsloth connect`. * Tests: stub _unstarted_cleanup in same-task disconnect test The test builds _SameTaskStreamingResponse via __new__, so set the attribute that __call__ now reads. * Match coding-agent command loopback check to the CLI 127.0.0.0/8 rule (#6613) * Keep unsloth_cli.commands.connect importable as a deprecated shim (#6613) * Format the new coding-agents panel strings and import per biome (#6613) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop the unsloth connect alias and shim; unsloth start is the only command (#6613) * Route unsloth connect to unsloth start as a hidden backward-compatible alias (#6613) * Forward unsloth run model-load flags to unsloth start (gguf-variant, context-length, load-in-4bit, tensor-parallel) (#6613) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Session-scope coding agent config in unsloth start Configure each agent for the current session instead of writing the Studio endpoint, key, and default model into the user's own config. Codex, OpenCode, OpenClaw, and Hermes get a private config relocated through their config-path env vars (CODEX_HOME, OPENCODE_CONFIG overlay, OPENCLAW_CONFIG_PATH plus OPENCLAW_STATE_DIR, HERMES_HOME). Claude Code suppresses the attribution header for the session via the CLAUDE_CODE_ATTRIBUTION_HEADER env var plus a --settings overlay, with no ~/.claude write. --launch uses an ephemeral temp dir removed after the agent exits; --no-launch uses a stable Unsloth-owned dir and prints the matching export lines. * Read relocated agent session config in Local Agent Guides CI The contract crosscheck and the openclaw/hermes patch helpers now read each agent's config from the relocated path printed by unsloth start --no-launch (CODEX_HOME, OPENCODE_CONFIG, OPENCLAW_CONFIG_PATH, HERMES_HOME) instead of fixed home paths. The Claude attribution A/B toggles the header for the session only (shipped-config HIT vs vanilla MISS) instead of editing ~/.claude/settings.json. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip the POSIX-only --no-launch parser test on Windows test_no_launch_output_is_parseable mirrors the #6547 bash CI parser, which greps export/unset lines and only runs on Linux/macOS runners. On Windows --no-launch prints PowerShell ($env: / Remove-Item), so the export-line assertion does not apply there. Cross-OS staging CI surfaced this. * Size Claude Code's auto-compact window to the loaded model's context Claude Code auto-compacts against its native (~600k token) window, so against a smaller local model it overflows the server's context (silent truncation) long before it compacts. Set CLAUDE_CODE_AUTO_COMPACT_WINDOW to the loaded model's real context length (the value codex/openclaw already get via model_context_window / contextWindow). Omitted when the model reports no context length. * Pin OpenCode/Hermes context window and set 90% compaction across agents Feed every agent the server-determined sequence length (the value /v1/models reports from runtime_context_length) and a ~90% compaction threshold. OpenCode: a custom-provider model with no limit defaults to context 0, which silently disables auto-compaction, so set limit.context/output and scale the compaction buffer to 10% of the window. Hermes: pin model.context_length (it otherwise falls back to a 256k default when the server's /v1/models omits the field) and set compression.threshold 0.9. Claude: add CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=90 alongside the window. Codex (model_context_window) and OpenClaw (contextWindow) already carried the window and auto-manage off it. * Add `unsloth start pi` recipe Pi was the only agent without a built-in recipe, so the agent-guides CI hand-wrote ~/.pi/agent/models.json. Add a first-class `pi` command mirroring the others: - write_pi_config writes the session-scoped OpenAI-compatible provider config (key in the config, like openclaw/opencode). - pi() launches `pi --provider unsloth --model <id>` (Pi defaults to the google provider, so the provider/model are pinned on the command line) with HOME relocated for the session. Pi has no config-dir env var and resolves ~/.pi off $HOME, so HOME-scoping keeps the user's ~/.pi untouched. Migrate the agent-guides CI off the hand-written config onto the `unsloth start pi --no-launch` path (connection + file-edit), with a crosscheck for the provider api, so the documented recipe is exercised. * Harden unsloth start for Windows and WSL agent launches Address the Codex review on PR 6613: - write_pi_config now pins the loaded contextWindow and a sane maxTokens so Pi compacts instead of overflowing a small Studio context (it otherwise assumes its 128000 default), matching the other agents. - pi() sets USERPROFILE (and HOMEDRIVE/HOMEPATH when present) alongside HOME on native Windows, where Node resolves ~/.pi via USERPROFILE rather than HOME, so the session no longer reads or writes the user's real ~/.pi. - The WSLENV bridge flags path-valued vars with /p so a Windows npm shim under /mnt receives translated paths, while scalar vars (the numeric context window) pass through untranslated. WSLENV is deduped on the bare name. - _print_env prints the launch command with PowerShell-safe quoting so the inline --settings JSON survives copy-paste on native Windows --no-launch. Add tests for the WSLENV path flagging, PowerShell quoting, the Pi context window, and the Pi USERPROFILE relocation. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Set CLAUDE_CODE_NO_FLICKER for the Claude session A local server streams in bursts, so Claude Code's full-screen TUI redraw flickers between tokens. Disable it for the session via CLAUDE_CODE_NO_FLICKER, alongside the other CLAUDE_CODE_* session env knobs. * Add a normalized --yolo flag routed to each agent's auto-approve mode It is easy to forget which agent spells "run tools without prompting" which way, so `unsloth start` now accepts all three spellings as one option (--yolo, --dangerously-skip-permissions, --dangerously-bypass-approvals-and-sandbox) and routes to the agent's own mechanism: - claude: --dangerously-skip-permissions - codex: --dangerously-bypass-approvals-and-sandbox - hermes: --yolo - pi: --approve (Pi's only approval gate is project trust) - opencode: a permission allow block in opencode.json (no CLI flag exists) - openclaw: tools.exec security=full / ask=off / host=gateway (no CLI flag exists) Because the option is parsed by `unsloth start`, the "wrong" spelling for an agent still routes correctly instead of leaking through to the agent and erroring. IS_SANDBOX is deliberately left unset for Claude so its root/sandbox safety gate still applies. Adds routing, cross-routing, and per-config tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix review findings: IPv6 loopback command, pi USERPROFILE under WSL, yolo guard From a 10-reviewer pass over the PR: - studio/frontend agent-command.ts: normalize bracketed IPv6 hosts. URL.hostname returns "[::1]" for http://[::1]:8888, which never matched the "::1" loopback checks, so the copied command embedded the placeholder API key for a local IPv6 server instead of the bare auto-minting command. Now [::1] is treated as loopback like the CLI's is_loopback_url, so the command matches the CLI contract. - pi(): also relocate USERPROFILE (and HOMEDRIVE/HOMEPATH) when running under WSL against a /mnt Windows shim, not just on native Windows. Windows Node resolves ~/.pi via USERPROFILE, and the WSLENV bridge translates the path, so pi no longer falls back to the user's real ~/.pi in that case. - _yolo_command_flags: use .get so a config-based agent (or a typo) yields no flag instead of a latent KeyError. Adds tests for the WSL pi USERPROFILE relocation, the yolo unmapped-agent guard, and that opencode/openclaw --yolo stays config-only (no argv flag). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix round-2 review findings: WSLENV /p upgrade, agent help text - _merge_wslenv now upgrades a user's pre-existing unflagged WSLENV entry (e.g. a bare HOME or USERPROFILE) to the path-translated form (HOME/p) instead of leaving it as-is, so a Windows agent shim under WSL receives the translated session path rather than the raw Linux path. - Generalize the `unsloth start` registration help to list all six agents (was only "Claude Code, Codex"). Adds a test for the WSLENV unflagged-entry upgrade. * Fix round-3 review findings: complete openclaw --yolo, refresh stale copy - openclaw --yolo now also writes the host approvals file (exec-approvals.json with defaults security=full / ask=off / askFallback=full) alongside the tools.exec config. OpenClaw gates tool execution on both layers (the stricter wins), so the config alone could still leave it prompting or denying. Mirrors `openclaw exec-policy preset yolo`. ask=off means nothing is ever prompted, so the runtime socket block is unnecessary. - Studio API panel copy: clarify that a local server auto-mints the key while a remote one embeds it in the command, and add pi to the swap hint. - Local Agent Guides CI: drop the stale "pi has no start.py recipe" note now that all six agents are driven via `unsloth start <agent> --no-launch`. Adds the openclaw approvals-file assertions and a no-yolo openclaw test. * start: parse claude --version with a regex so a format change does not drop optimization flags * start: offer to install a missing agent (prompt then run its install command) * start: auto-start a Studio server for --model when none is running, and stop it on exit * inference: surface an actionable message when llama-server cannot compile a tool grammar * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix review findings: kill the auto-started server tree on Windows; apply the tool-grammar message to the OpenAI passthrough too * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * start: split --model org/repo:variant so a running session is not evicted `unsloth start <agent> --model org/repo:QUANT` failed against an already-running Studio server and, worse, killed whatever model another session had loaded. /v1/models lists a loaded GGUF under its bare repo id (e.g. unsloth/Qwen3-1.7B-GGUF), so _resolve_model never matched the `:QUANT`-suffixed request. It then POSTed /api/inference/load with model_path=org/repo:QUANT, which (a) Hugging Face rejects ("Repo id must use alphanumeric chars, '-', '_' or '.'") and (b) evicts the model the other session was using, so a second 'unsloth start' in a new tmux/terminal tore down the first. Re-running the command then attached to the now-empty server, which is why it 'worked the second time'. Mirror the org/repo:QUANT -> org/repo + --gguf-variant QUANT shorthand that 'unsloth run' and llama.cpp already accept, splitting it in _connect before we match or serve. Matching now resolves against the loaded bare repo id (no spurious reload, no eviction), and any real load uses a valid repo id plus gguf_variant. An explicit --gguf-variant still wins; local paths and Windows drive letters pass through untouched. The auto-serve path likewise spawns 'unsloth run --model org/repo --gguf-variant QUANT'. * start: harden auth-key handling, codex teardown, and CI transcript redaction Three review findings: 1. CI could leak a live key. agent-guides-drive.sh printed the raw 'unsloth start --no-launch' transcript (which carries export UNSLOTH_API_KEY / ANTHROPIC_AUTH_TOKEN lines) to the Actions log on both the failure path and the success path before redact() ran. Add cat_redacted() and use it for those two prints, so the key is scrubbed on the way to the log while the on-disk file stays intact for the env parsing that follows. 2. Outages masqueraded as bad keys. _key_accepted caught a broad Exception and returned False, so a 5xx or timeout while checking a cached key looked like a rejection: it discarded a good key and minted extra ones (local) or reported 'no saved key' (remote). Only treat HTTP 401/403 as a rejection; let other errors propagate so a real outage surfaces. 3. Codex preflight could leave the auto-started server up. _require_gguf_for_codex runs after _connect may have auto-started Studio but before _run installs its teardown finally, so a preflight rejection (e.g. a transformers-backend model) left the server holding the port/GPU until the atexit backstop. Tear it down explicitly at the point of failure. Tests: a 5xx on a saved key surfaces without minting; a non-GGUF codex preflight tears down the auto-served server. * start: fix IPv6/portless studio URLs, Pi config-dir isolation, and Pi install recipe Four review findings: 1. Pi ignored the session config when PI_CODING_AGENT_DIR was already set. Pi's getAgentDir() reads process.env.PI_CODING_AGENT_DIR before falling back to $HOME/.pi/agent, so a value inherited from the user's shell sent Pi to their real config and skipped our provider/key (the HOME relocation alone was not enough). Pin PI_CODING_AGENT_DIR at the session's .pi/agent dir; it is path-valued so the WSL bridge translates it automatically. 2. Pi install hint dropped Pi's documented --ignore-scripts. Pi's README installs with 'npm install -g --ignore-scripts @earendil-works/pi-coding-agent' and notes it needs no install scripts, so accepting the prompt now follows that safe recipe. 3. Auto-start ignored a portless UNSLOTH_STUDIO_URL. unsloth run binds to 'parsed.port or 8888', so http://127.0.0.1 launched the child on 8888 but the health poll (and the returned base) still used port 80, stalling until the startup timeout. Normalize the base to host:8888 (IPv6-safe) before starting and polling. 4. API-panel command mistook IPv6 loopback for the bare default. The bare 'unsloth start' only probes 127.0.0.1:8888 on the IPv4 stack, so http://[::1]:8888 must carry an explicit UNSLOTH_STUDIO_URL. Drop ::1 from the bare-default host set while keeping it a loopback host (URL emitted, no key needed). Tests: PI_CODING_AGENT_DIR is set to the session dir; _effective_base normalizes portless/IPv6 bases; a portless UNSLOTH_STUDIO_URL auto-serves on :8888. * start: apply fresh-review findings across CLI, CI, and the API-panel command From a fresh multi-reviewer pass over the merged head plus the latest Codex bot review: 1. Load knobs now always consult the server. _resolve_model matched on model id alone, so --gguf-variant / --context-length / --no-load-in-4bit / --tensor-parallel were silently ignored whenever the id was already loaded (asking for UD-Q4_K_XL kept a Q8_0 serving). With any explicit knob the CLI defers to /api/inference/load, whose already-loaded dedup answers without reloading when variant and settings match, so a second session running the same command still attaches without evicting the first. 2. OpenCode --yolo and the session model pin now ride in OPENCODE_CONFIG_CONTENT. A project's own opencode.json outranks OPENCODE_CONFIG, so a repo config could silently override the session model and the --yolo permission block; OPENCODE_CONFIG_CONTENT outranks project config. The API key stays in the private file, never in printed env. 3. The --no-launch recipe's last line is a self-contained one-liner (inline VAR=value assignments before the command, conflicting vars blanked). People copy just the last line, and a bare codex/claude there ran against the user's real ~/.codex or Anthropic credentials with zero isolation, e.g. inheriting a pre-existing damaged ~/.codex state DB and blaming the recipe. The CI drive script scrubs the key from the one 'invoking:' echo this adds. 4. The auto-serve log is 0600 and the parent handle is closed. It sat world-readable in the shared tempdir under a predictable name while carrying the minted sk-unsloth- key from the unsloth run banner. 5. _key_accepted fails with a clean message on outages. Non-auth errors (5xx, network, timeout) surfaced as a raw traceback; 401/403 still mean a rejected key. 6. _effective_base strips URL paths, and https loopback targets never auto-serve. http://127.0.0.1:8888/studio polled /studio/api/health (404) and https://127.0.0.1 polled the wrong scheme, both spinning until the 15-minute startup timeout. 7. API-panel command: only literal 127.0.0.1:8888 earns the bare command. localhost can resolve to ::1, which the bare CLI never probes, so it keeps UNSLOTH_STUDIO_URL. 8. CI artifact sweep covers redacted-configs/ and agent-workdir/, not just logs/. Tests: 125 CLI tests pass (new coverage for each fix), 156 backend tests pass, ruff clean. Adds an unsloth connect alias regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * start: hand Pi a clean screen at launch Pi paints inline from wherever the cursor sits: its first render assumes a clean screen instead of clearing or entering the alternate screen itself (current Pi never emits a clear at startup). Launched under unsloth start, that left the session starting mid-scroll beneath the connection output. Clear the screen (click.clear, cross-platform, no-op without a TTY) right before the Studio banner so Pi opens exactly one line down on a clean viewport. Launch path only: --no-launch recipes and piped output are never wiped, and alternate-screen agents are left alone. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * start: auto-override hermes' 64K context floor for small model windows Hermes refuses to initialize when the served model's context window is under 64,000 tokens, and a second copy of the same check rejects the compression model mid-session. write_hermes_config previously pinned the real window, so any small local model (e.g. 40,960) failed at startup with manual config.yaml instructions. For windows below the floor the recipe now claims 65,536 in model.context_length, scales compression.threshold so compaction still fires at 90% of the real window, and sets auxiliary.compression.context_length to cover the mid-session check. Windows at or above the floor keep the exact previous behavior. * ci: install pi with --ignore-scripts, matching the start.py hint The pi cell predates the pi recipe in start.py and still installed the package with lifecycle scripts enabled, so CI stopped exercising the exact command users are prompted to run. npm_retry now passes extra flags through, the pi branch mirrors the install hint verbatim, and the stale no-recipe comment is refreshed. * ci: fail loudly when a relocation var is missing from connect output The empty-string guards ran after appending /config.toml or /config.yaml, so they could never fire: crosscheck_contract silently skipped its contract checks and patch_hermes_tools died on the root path with a bare traceback. Check the raw variable first and guide_fail with the real cause. * staging: 6613 round 6 (https elision, no-launch home reuse, auto-start key fallback) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com> |
||
|---|---|---|
| .github | ||
| images | ||
| scripts | ||
| studio | ||
| tests | ||
| unsloth | ||
| unsloth_cli | ||
| .gitattributes | ||
| .gitignore | ||
| .pre-commit-ci.yaml | ||
| .pre-commit-config.yaml | ||
| build.sh | ||
| cli.py | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| install.ps1 | ||
| install.sh | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| unsloth-cli.py | ||
Unsloth Studio lets you run and train models locally.
Features • Quickstart • Notebooks • Documentation
⚡ Get started
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Windows:
irm https://unsloth.ai/install.ps1 | iex
Community:
⭐ Features
Unsloth Studio (Beta) lets you run and train text, audio, embedding, vision models on Windows, Linux and macOS.
Inference
- Search + download + run models including GGUF, LoRA adapters, safetensors
- Export models: Save or export models to GGUF, 16-bit safetensors and other formats.
- Tool calling: Support for self-healing tool calling and web search
- Code execution: lets LLMs test code in Claude artifacts and sandbox environments
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools with Unsloth
- Auto set inference settings and customize chat templates.
- We work directly with teams behind gpt-oss, Qwen3, Llama 4, Mistral, Gemma 1-3, and Phi-4, where we’ve fixed bugs that improve model accuracy.
- Chat with images, audio, PDFs, code, DOCX and more. Connect API providers (OpenAI, Anthropic) or servers (vLLM, Ollama).
Training
- Train and RL 500+ models up to 2x faster with up to 70% less VRAM, with no accuracy loss.
- Custom Triton and mathematical kernels. See some collabs we did with PyTorch and Hugging Face.
- Data Recipes: Auto-create datasets from PDF, CSV, DOCX etc. Edit data in a visual-node workflow.
- Reinforcement Learning (RL): The most efficient RL library, using 80% less VRAM for GRPO, FP8 etc.
- Supports full fine-tuning, RL, pretraining, 4-bit, 16-bit and, FP8 training.
- Observability: Monitor training live, track loss and GPU usage and customize graphs.
- Multi-GPU training is supported, with major improvements coming soon.
📥 Install
Unsloth can be used in two ways: through Unsloth Studio, the web UI, or through Unsloth Core, the code-based version. Each has different requirements.
Unsloth Studio (web UI)
Unsloth Studio (Beta) works on Windows, Linux, WSL and macOS.
- CPU: Supported for Chat and Data Recipes currently
- NVIDIA: Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
- macOS: Training, MLX and GGUF inference are ALL supported.
- AMD: Chat + Data works. Train with Unsloth Core. Studio support is out soon.
- Multi-GPU: Available now, with a major upgrade on the way
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Use the same command to update.
Windows:
irm https://unsloth.ai/install.ps1 | iex
Use the same command to update.
Launch
unsloth studio -p 8888
For cloud or global access, add -H 0.0.0.0. By default, Unsloth is accessible only locally.
To reach Studio over HTTPS, use unsloth studio --secure. Studio stays bound to localhost and is reached only through a free Cloudflare tunnel, which publishes it at a public https://*.trycloudflare.com URL (it fails closed if the tunnel can't start, so the raw port is never exposed). This makes Studio reachable from the internet, so anyone with the link and API key can use it and run code: keep your API key private (see Remote access below).
Docker
Use our Docker image unsloth/unsloth container. Run:
docker run -d -e JUPYTER_PASSWORD="mypassword" \
-p 8888:8888 -p 8000:8000 -p 2222:22 \
-v $(pwd)/work:/workspace/work \
--gpus all \
unsloth/unsloth
Developer, Nightly, Uninstall
To see developer, nightly and uninstallation etc. instructions, see advanced installation.
Unsloth Core (code-based)
Linux, WSL:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto
Windows:
winget install -e --id Python.Python.3.13
winget install --id=astral-sh.uv -e
uv venv unsloth_env --python 3.13
.\unsloth_env\Scripts\activate
uv pip install unsloth --torch-backend=auto
For Windows, pip install unsloth works only if you have PyTorch installed. Read our Windows Guide.
You can use the same Docker image as Unsloth Studio.
AMD, Intel:
For RTX 50x, B200, 6000 GPUs: uv pip install unsloth --torch-backend=auto. Read our guides for: Blackwell and DGX Spark.
To install Unsloth on AMD and Intel GPUs, follow our AMD Guide and Intel Guide.
📒 Free Notebooks
Train for free with our notebooks. You can use our new free Unsloth Studio notebook to run and train models for free in a web UI. Read our guide. Add dataset, run, then deploy your trained model.
| Model | Free Notebooks | Performance | Memory use |
|---|---|---|---|
| Gemma 4 (E2B) | ▶️ Start for free | 1.5x faster | 50% less |
| Qwen3.5 (4B) | ▶️ Start for free | 1.5x faster | 60% less |
| gpt-oss (20B) | ▶️ Start for free | 2x faster | 70% less |
| Qwen3.5 GSPO | ▶️ Start for free | 2x faster | 70% less |
| gpt-oss (20B): GRPO | ▶️ Start for free | 2x faster | 80% less |
| Qwen3: Advanced GRPO | ▶️ Start for free | 2x faster | 70% less |
| embeddinggemma (300M) | ▶️ Start for free | 2x faster | 20% less |
| Mistral Ministral 3 (3B) | ▶️ Start for free | 1.5x faster | 60% less |
| Llama 3.1 (8B) Alpaca | ▶️ Start for free | 2x faster | 70% less |
| Llama 3.2 Conversational | ▶️ Start for free | 2x faster | 70% less |
| Orpheus-TTS (3B) | ▶️ Start for free | 1.5x faster | 50% less |
- See all our notebooks for: Kaggle, GRPO, TTS, embedding & Vision
- See all our models and all our notebooks
- See detailed documentation for Unsloth here
🦥 Unsloth News
- Connections: Connect any API provider (OpenAI, Anthropic) or server (vLLM, Ollama). Guide
- MTP: Run Qwen3.6 MTP in Unsloth. MTP settings are autoset specific to your hardware. Guide
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools. Guide
- Qwen3.6: Qwen3.6-35B-A3B can now be trained and run in Unsloth Studio. Blog
- Gemma 4: Run and train Google’s new models directly in Unsloth. Blog
- Introducing Unsloth Studio: our new web UI for running and training LLMs. Blog
- Qwen3.5 - 0.8B, 2B, 4B, 9B, 27B, 35-A3B, 112B-A10B are now supported. Guide + notebooks
- Train MoE LLMs 12x faster with 35% less VRAM - DeepSeek, GLM, Qwen and gpt-oss. Blog
- Embedding models: Unsloth now supports ~1.8-3.3x faster embedding fine-tuning. Blog • Notebooks
- New 7x longer context RL vs. all other setups, via our new batching algorithms. Blog
- New RoPE & MLP Triton Kernels & Padding Free + Packing: 3x faster training & 30% less VRAM. Blog
- 500K Context: Training a 20B model with >500K context is now possible on an 80GB GPU. Blog
- FP8 & Vision RL: You can now do FP8 & VLM GRPO on consumer GPUs. FP8 Blog • Vision RL
📥 Advanced Installation
The below advanced instructions are for Unsloth Studio. For Unsloth Core advanced installation, view our docs.
Developer / Nightly / Experimental installs: macOS, Linux, WSL:
The developer install builds from the main branch, which is the latest (nightly) source.
git clone https://github.com/unslothai/unsloth
cd unsloth
./install.sh --local
unsloth studio -p 8888
To install into an isolated location (its own virtual env, auth/, studio.db, cache and llama.cpp build), set UNSLOTH_STUDIO_HOME and pass it again at launch:
UNSLOTH_STUDIO_HOME="$PWD/.studio" ./install.sh --local
UNSLOTH_STUDIO_HOME="$PWD/.studio" unsloth studio -p 8888
Then to update :
cd unsloth && git pull
./install.sh --local
unsloth studio -p 8888
Developer / Nightly / Experimental installs: Windows PowerShell:
The developer install builds from the main branch, which is the latest (nightly) source.
git clone https://github.com/unslothai/unsloth.git
cd unsloth
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888
To install into an isolated location (its own virtual env, auth/, studio.db, cache and llama.cpp build), set UNSLOTH_STUDIO_HOME and pass it again at launch:
$env:UNSLOTH_STUDIO_HOME="$PWD\.studio"; .\install.ps1 --local
$env:UNSLOTH_STUDIO_HOME="$PWD\.studio"; unsloth studio -p 8888
Then to update :
cd unsloth; git pull
.\install.ps1 --local
unsloth studio -p 8888
Remote access: --secure (HTTPS tunnel) vs raw port
By default unsloth studio binds to 127.0.0.1 (this machine only). To reach it from another device, pick one of:
--secure(recommended): serve only through a free Cloudflare HTTPS link. Studio stays bound to localhost and the tunnel provides the public URL; it fails closed (does not start) if the tunnel can't come up, so the raw port is never exposed.
unsloth studio --secure -p 8888
-H 0.0.0.0: bind the raw port on all network interfaces, reachable from anywhere on the network. Only use this on a trusted network.
unsloth studio -H 0.0.0.0 -p 8888
Server-side tools (web search, Python and terminal code execution) run as your user and are on by default. Anyone who can reach the server with the API key can run code on this machine, so keep your API key private and pass --disable-tools when exposing Studio.
Advanced launch options
Installer options can be passed as environment variables. On macOS, Linux and WSL place the variable after the pipe so the shell passes it to sh; on Windows set it with $env: before piping to iex.
Skip PyTorch (GGUF-only mode):
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_NO_TORCH=1 sh
$env:UNSLOTH_NO_TORCH=1; irm https://unsloth.ai/install.ps1 | iex
Pin the Python version:
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_PYTHON=3.12 sh
$env:UNSLOTH_PYTHON='3.12'; irm https://unsloth.ai/install.ps1 | iex
Install to a custom location with UNSLOTH_STUDIO_HOME:
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_STUDIO_HOME=/abs/path sh
$env:UNSLOTH_STUDIO_HOME='C:\path'; irm https://unsloth.ai/install.ps1 | iex
On macOS, the installer defaults to the system certificate store (UV_SYSTEM_CERTS=1) so uv trusts the CAs in your Keychain, needed behind TLS-inspecting proxies (Cisco Umbrella, Zscaler, etc.). Opt out with:
curl -fsSL https://unsloth.ai/install.sh | UV_SYSTEM_CERTS=0 sh
Point the frontend build at a corporate npm mirror/proxy with UNSLOTH_NPM_REGISTRY (for the developer install behind a firewall that blocks registry.npmjs.org):
UNSLOTH_NPM_REGISTRY=https://artifactory.example.com/api/npm/npm/ ./install.sh --local
$env:UNSLOTH_NPM_REGISTRY='https://artifactory.example.com/api/npm/npm/'; .\install.ps1 --local
It is threaded as --registry into the Studio frontend npm/bun installs; the supply-chain locks (7-day min-release-age, exact version pins) stay in force.
Cap Studio's native CPU thread pools on high-core hosts: UNSLOTH_CPU_THREADS=8 unsloth studio -p 8888.
Uninstall
The recommended way to fully remove Unsloth Studio is the matching uninstall script for your OS. It stops any running servers, removes the install dir, the launcher data dir, the desktop shortcut, and any platform-specific entries (macOS .app bundle + Launch Services on Mac; Start Menu, HKCU\Software\Unsloth registry key and user PATH entries on Windows):
- MacOS, WSL, Linux:
curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.sh | sh - Windows (PowerShell):
irm https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.ps1 | iex
If you only want to drop the install dir and keep the launcher/shortcut for a later reinstall, you can instead run rm -rf ~/.unsloth/studio (Mac/Linux/WSL) or Remove-Item -Recurse -Force "$HOME\.unsloth\studio" (Windows). The model cache at ~/.cache/huggingface is not touched by any of these.
For more info, see our docs.
Deleting model files
You can delete old model files either from the bin icon in model search or by removing the relevant cached model folder from the default Hugging Face cache directory. By default, HF uses:
- MacOS, Linux, WSL:
~/.cache/huggingface/hub/ - Windows:
%USERPROFILE%\.cache\huggingface\hub\
💚 Community and Links
| Type | Links |
|---|---|
 Discord |
Join Discord server |
 r/unsloth Reddit |
Join Reddit community |
| 📚 Documentation & Wiki | Read Our Docs |
_logo_late_2025.svg){kind=logo} Twitter (aka X) |
Follow us on X |
| 🔮 Our Models | Unsloth Catalog |
| ✍️ Blog | Read our Blogs |
Citation
You can cite the Unsloth repo as follows:
@software{unsloth,
author = {Daniel Han, Michael Han and Unsloth team},
title = {Unsloth},
url = {https://github.com/unslothai/unsloth},
year = {2023}
}
If you trained a model with 🦥Unsloth, you can use this cool sticker!
License
Unsloth uses a dual-licensing model of Apache 2.0 and AGPL-3.0. The core Unsloth package remains licensed under Apache 2.0, while certain optional components, such as the Unsloth Studio UI are licensed under the open-source license AGPL-3.0.
This structure helps support ongoing Unsloth development while keeping the project open source and enabling the broader ecosystem to continue growing.
Thank You to
- The llama.cpp library that lets users run and save models with Unsloth
- The Hugging Face team and their libraries: transformers and TRL
- The Pytorch and Torch AO team for their contributions
- NVIDIA for their NeMo DataDesigner library and their contributions
- And of course for every single person who has contributed or has used Unsloth!