mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-10 00:08:58 +00:00
mirror of https://github.com/unslothai/unsloth
agentaideepseekdeepseek-r1fine-tuninggemmagemma3llamallama3llmllmsloramistralopenaipytorchqwenqwen3text-to-speechttsunsloth
55043acf50
* [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add SDXL diffusion family (U-Net pipeline support) SDXL is the first U-Net family in the diffusion backend: its denoiser is pipe.unet (UNet2DConditionModel), not a DiT pipe.transformer, and a single-file .safetensors is the whole pipeline rather than a transformer-only file. The backend previously assumed a DiT transformer everywhere, so add the two hooks a U-Net family needs and register SDXL. DiffusionFamily gains denoiser_attr ("transformer" for DiT, "unet" for SDXL) and single_file_is_pipeline (SDXL loads a single file via pipeline_class.from_single_file with the base repo as config, instead of transformer_class.from_single_file plus a companion assembly). _align_vae_dtype now reads the denoiser generically so img2img and inpaint keep the VAE and U-Net dtypes aligned. The non-GGUF trust gate is extended with a short, exact-match, safetensors-only allowlist of official base repos (the SDXL base/refiner and sdxl-turbo), because SDXL ships only as a full pipeline and has no unsloth-hosted GGUF. Local paths stay trusted as before; a random repo, even one that detects as SDXL, is still rejected. The image-conditioned and ControlNet workflows are the standard SDXL pipelines, built around the resident modules via from_pipe like every other family, so SDXL gets txt2img, img2img, inpaint, outpaint, upscale, LoRA and ControlNet. There is no native sd.cpp mapping yet, so the no-GPU route falls back to diffusers. Frontend catalog gains SDXL Base 1.0 and SDXL Turbo entries with SDXL step/guidance defaults (Turbo: few steps, no CFG; base: ~30 steps, real CFG). Tests: new test_diffusion_sdxl.py (family shape, detection, trust allowlist, model kind, U-Net VAE-dtype alignment, LoRA gate) plus loader-branch tests in test_diffusion_backend.py (pipeline-kind from_pretrained, single-file whole-pipeline from_single_file, allowlist accept/reject). Verified live on GPU: sdxl-turbo loads both as a pipeline and as a single file and generates coherent txt2img + img2img. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Images: LoRA free-text Hugging Face entry + recipe round-trip The backend has always accepted a bare Hugging Face repo id (owner/name, or owner/name:weight-file.safetensors) as a LoRA, downloading and applying it. But the picker only rendered when the curated catalog had entries, and the catalog is empty, so there was no UI path to apply any LoRA. Show the LoRA section whenever the loaded model supports LoRA, and replace the curated-only dropdown with a text input: type a Hub repo id, or pick a discovered adapter from a datalist of suggestions when the catalog is populated. Also restore LoRAs when loading a recipe. restoreSettings now parses the recipe's "id:weight" strings (splitting on the last colon, since the id itself may contain one for a specific weight file) back into the selection, so replaying a saved image reproduces its adapters. The generate payload trims hand-typed ids and drops empty / zero-weight rows, and a model swap clears the selection (a LoRA is family-specific) without discarding a free-text pick that is not in the curated list. * Add diffusion LoRA training (SDXL text-to-image) First diffusion training path in Studio: train a LoRA on the SDXL U-Net from an image + caption dataset and export it as a diffusers .safetensors that the existing diffusion LoRA loader (and any diffusers pipeline) can load. core/training/diffusion_lora_trainer.py: - DiffusionLoraConfig with validation/defaults (rank, alpha, targets, lr, steps, grad accumulation, resolution, min-SNR gamma, gradient checkpointing, lr scheduler, seed, mixed precision). - discover_image_caption_pairs: captions from metadata.jsonl / captions.jsonl, per-image .txt/.caption sidecars, or a dreambooth instance_prompt fallback (pure, unit-tested). - run_diffusion_lora_training: the loop -- freeze base, PEFT-wrap the U-Net attention projections, VAE-encode (fp32 VAE to avoid the SDXL fp16 overflow), sample noise + timesteps, predict, MSE loss with optional min-SNR weighting (epsilon / v-prediction), AdamW + get_scheduler + grad accumulation + grad clipping, then export via save_lora_weights. Emits worker-protocol events (model_load_*, progress, complete) and polls should_stop for a clean stop with a partial save. - run_diffusion_training_process: mp.Queue subprocess adapter (event_queue / stop_queue), so the training worker can spawn it; plus a CLI entry point. Only SDXL (U-Net) is trained here; DiT families and the Studio UI form + route wiring are follow-ups. The trainer is decoupled and worker-ready. Tests: test_diffusion_lora_trainer.py covers caption discovery (metadata / sidecar / instance prompt / skip-uncaptioned / errors), config normalisation + validation, the SDXL add-time-ids, and the dict->config adapter. Verified live on GPU: a 60-step SDXL LoRA run lowers the loss, exports a ~45 MB adapter, and loading it back shifts generation from baseline (mean abs pixel diff ~55/255). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * diffusion trainer: emit learning_rate in progress events (Studio pump compatibility) The Studio training pump reads 'learning_rate' from progress events; the diffusion trainer emitted 'lr'. Rename the field (and the CLI reader) so the trainer's events are directly consumable by the existing training status/SSE machinery when it is wired into the worker, without a translation shim. * Wire diffusion LoRA training into the Studio API Make the SDXL LoRA trainer reachable from the app with a small, self-contained job service and JSON routes, deliberately separate from the LLM TrainingBackend (whose lifecycle -- LLM config build, per-run SQLite rows, matplotlib plots, transfer-to-chat- inference -- is text-training specific and would mis-handle a diffusion run). core/training/diffusion_training_service.py: DiffusionTrainingService runs one job at a time -- validate the config cheaply (before any spawn), spawn the trainer subprocess (spawn context, parent-lifetime bound), pump its events (model_load_* / progress / complete / error) into an in-memory status snapshot, and support a clean stop. The subprocess context and target are injectable so the full start -> pump -> status -> complete path is unit-tested without real multiprocessing or torch. routes/training.py: POST /api/train/diffusion/start (400 on a bad config, 409 when a job is already running), POST /api/train/diffusion/stop, GET /api/train/diffusion/status (JSON poll). models/training.py: DiffusionTrainingStartRequest + response schemas mirroring DiffusionLoraConfig, so model_dump() passes straight through. Tests: test_diffusion_training.py -- service happy path, bad-config-before-spawn, concurrent-job rejection, clean stop, crash-without-terminal-event, event transitions; plus route wiring via the FastAPI TestClient (start / 422 / 400 / 409 / status / stop) with a mocked service. The diffusion trainer's progress events already use the field names this path expects. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Images: add a Train LoRA (SDXL) dialog Surface the diffusion training API in the Images page. A "Train LoRA" button in the top bar opens a self-contained dialog to fine-tune an SDXL LoRA on a folder of images: pick the base model, dataset folder, output folder, an optional instance prompt, and the core hyperparameters (steps, rank, resolution, batch, learning rate), then Start. The dialog polls the training status while open and shows a progress bar, step count, live loss, and the saved adapter path, with a Stop button for a clean stop. The dialog is independent of the loaded generation model (training runs in its own subprocess), and prefills the base model with the loaded checkpoint when it is SDXL, else the SDXL base. api.ts gains startDiffusionTraining / stopDiffusionTraining / getDiffusionTrainingStatus plus their types, matching the /api/train/diffusion routes. * Import diffusion training schemas from models.training directly The import-hoist lint flags newly re-exported names in the models/__init__.py hub as unused (it does not treat __all__ membership as a use). Import the three diffusion training schemas straight from models.training in routes/training.py, where they are used in the route annotations and calls, and drop the __init__ re-export. * Remove stray async task scratch outputs committed by mistake * ControlNet: reject filesystem-like ids and do not cache a model past an unload race Two review findings on the ControlNet path: - resolve_controlnet's bare-repo fallback accepted any id with a slash, so a path-shaped id (/tmp/x, ../x) reached from_pretrained as a local directory. Restrict the fallback to a strict owner/name HF repo id shape. - _controlnet_pipe now re-checks the cancel event after the blocking from_pretrained: an unload that raced the download had already cleared the caches, so caching the late module would pin it past the unload. * Pipeline prefetch: fetch only the default torch weights A full-pipeline prefetch kept every repo file outside assets/, so an official repo that ships multiple formats (SDXL Base: fp16 variants, ONNX, OpenVINO, Flax, a top-level single-file twin) downloaded tens of GB from_pretrained never loads. Skip non-torch exports and dtype-variant twins in _pipeline_file_downloaded, and drop a component .bin when the same directory carries a picked safetensors weight (diffusers' own preference). * Diffusion LoRA training: fall back to fp16 when CUDA lacks bf16 The default mixed_precision=bf16 hard-fails on pre-Ampere GPUs (T4 / V100 / RTX 20xx) which have no bf16 compute; check torch.cuda.is_bf16_supported() and drop to fp16 there. * Diffusion training service: join the old pump outside the lock start() joined a finished job's pump thread while holding the service lock, but the pump's final state writes need that same lock, so the join always burned its full timeout and a stale pump could then overwrite the new job's state. Join outside the lock (with a re-check after), and fence _apply_event and the exit handler by process identity so a superseded pump can never touch the current job's state. Adds regression tests for both. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Diffusion LoRA training: harden config handling, cancellation, SDXL conditioning, and safety Addresses review findings on the SDXL LoRA trainer: - Gate the base model with the same trust check as inference (unsloth/*, allowlisted official bases, or a local path) before from_pretrained, so an untrusted remote repo is never fetched or deserialised. - Check the stop signal before the (slow) model load, not only between steps, so a cancel during download is honoured; a stop may carry save=False to cancel without leaving a partial adapter. - Per-sample SDXL add_time_ids from the actual crop (original size + crop offset, with the offset mirrored on horizontal flip) instead of a fixed uncropped-square tensor. - Apply EXIF orientation before resize/crop so rotated photos train upright. - Skip gradient clipping when max_grad_norm <= 0 (the Studio 'disable' value) instead of scaling every gradient to zero. - Coerce Studio config strings/blanks: learning_rate string to float, blank hf_token to anonymous, gradient_checkpointing 'none'/'true'/'unsloth' to bool; reject a zero/negative lora_alpha or learning_rate. - Alias the generic Studio training payload keys (model_name/max_steps/batch_size/lora_r/ lr_scheduler_type/random_seed) onto the diffusion field names. - Mirror the trained adapter into loras/diffusion so the Images LoRA picker discovers it. - Report worker exceptions in both message and error keys so the failure is not lost. Adds regression tests for the config coercion/validation and aliasing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ControlNet: address review findings on the diffusers path - resolve_controlnet enforces catalog family compatibility so a direct API call cannot load a ControlNet built for another family through the wrong pipeline. - Unknown ControlNet ids now surface as a 400 (call site maps FileNotFoundError to ValueError) instead of a generic 500. - strength 0 disables ControlNet entirely, so a no-op selection never pays the download / VRAM cost; the control image is decoded and validated BEFORE the ControlNet is resolved or built, so a malformed image fails fast for the same reason. - ControlNet loads use the base compute dtype (state.dtype is a display string, not a torch.dtype, so it silently fell back to float32) and honor the base offload policy via group offloading instead of forcing the module resident. - Empty/malformed HF token coerced to anonymous access. - Flux Union ControlNet control_mode mapped from the selected control type. - resolve_controlnet drops the unused hf_token/cancel_event params. - ControlNetSpec validates guidance_start <= guidance_end (clean 422). - Images UI ControlNet Select shows its placeholder when nothing is selected. Adds regression tests for family enforcement and the union control-mode map. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Diffusion training API: LLM interlock, pre-spawn VRAM free, path containment, no dropped knobs Four review findings on the diffusion training start path: - It spawned the SDXL trainer without checking the LLM TrainingBackend, so a start while an LLM run was active put two trainers on the same GPU. Add a symmetric interlock: diffusion start returns 409 when LLM training is active, and LLM start refuses while a diffusion job is active. - It went straight to service.start() without freeing GPU residents. Add a pre-spawn free of the export subprocess, the resident Images pipeline (with an arbiter release), and chat models, mirroring the LLM start path. - data_dir / output_dir were passed through unresolved, so Studio-relative names failed and absolute paths bypassed containment. Resolve them with resolve_dataset_path / resolve_output_dir before spawn (400 on an uncontained path). - The request model dropped max_grad_norm and lora_target_modules, so runs that set them trained with defaults. Add both fields. The gemini pump-join deadlock was already fixed earlier (join outside the lock + proc-identity fence). Note: honoring a stop DURING model load is a trainer-loop change owned by the diffusion training engine PR (should_stop polled before the first optimizer step). Adds route + model regression tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Diffusion LoRA: harden resolution, native tag precedence, and diffusers teardown Address review findings on the LoRA path: - resolve_one: normalise a blank/whitespace hf_token to None (anonymous access) and reject a client-supplied weight file with traversal / absolute path. - resolve_specs: convert FileNotFoundError from an unknown/stale id to ValueError so the route returns 400 instead of a generic 500. - _scan_local: disambiguate local adapters that share a stem (foo.safetensors vs foo.gguf) so each is uniquely addressable. - inject_prompt_tags: the backend-validated weight now wins over a user-typed <lora:ALIAS:...> for a selected adapter; unselected user tags are left alone. - diffusers _apply_loras: reject a .gguf adapter with a clear error before touching the pipe (diffusers loads safetensors only). - _unload_locked: drop the explicit unload_lora_weights() on teardown; the pipe is dropped wholesale (freeing adapters), so the previous call could race an in-flight denoise on the same pipe. - Images page: use a stable LoRA key and clear the selection (not just the options) when the catalog refresh fails. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Diffusion: guard trust check against OSError and validate conditioning inputs - _is_trusted_diffusion_repo: wrap Path.exists() so a repo id with invalid characters (or a bare owner/name id) can't raise OSError; treat any failure as not-a-local-path and fall through to the unsloth/ allowlist. validate_load_request still raises the clear FileNotFoundError for a genuinely missing local pick. - generate(): reject mask_image / upscale / reference_images supplied without an input image, and reject reference_images on a family that does not support reference conditioning, instead of silently degrading to txt2img / img2img. * SDXL: reject GGUF up front, skip unused base weights, drop refiner, and harden helpers Addresses review findings on the SDXL family: - Reject a GGUF load for single_file_is_pipeline families (SDXL) in validate_load_request, before the route evicts the current model; SDXL has no transformer-only GGUF variant. - Skip base-repo weight files when a whole-pipeline single file is loaded: from_single_file (config=base) needs only the base config/tokenizer/scheduler, so a local .safetensors no longer triggers a multi-GB base download. - Remove the SDXL refiner from the non-GGUF trust allowlist: it is an img2img-only pipeline but this backend loads every sdxl repo as the base txt2img pipeline. - Normalize a blank/whitespace hf_token to None once in load_pipeline so every load branch degrades to anonymous instead of erroring on a malformed token. - Read the denoiser dtype from a parameter (compile-wrapped modules may lack .dtype) and access state.family.denoiser_attr directly. Adds/updates regression tests for the trust allowlist, GGUF rejection, and base-config filter. * Images: preserve restored LoRAs through model load and never send hidden LoRAs - The LoRA effect cleared the selection on every load->capable transition, which wiped adapters restored from a gallery recipe before the model finished loading. Track the previously-loaded family in a ref and clear only on a real family swap; keep the selection on the initial load and on unload. - Gate the generate payload's loras on loraCapable so a restored selection that is hidden (loaded model does not support LoRA) is never sent to the backend. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Images Train LoRA dialog: token, validation, precision, base-repo prefill, gating, refresh Nine review findings on the SDXL training dialog: - Forward the saved Hub token so a gated/private SDXL base can be trained (the image load flow already sends it). - Re-seed the base-model field from the current default each time the dialog opens; the keep-alive dialog otherwise kept its mount-time default after a model loaded. - Prefill from base_repo (the diffusers pipeline) rather than repo_id, which for a GGUF/single-file SDXL load is the checkpoint path from_pretrained can't open. - Add client-side validation of steps/rank/resolution/batch/learning-rate before the request. - Expose a precision selector (bf16/fp16/fp32) so non-bf16 GPUs can train from the UI, not only the API. - Gate the dialog on the active Images route (active && trainOpen) so switching tabs closes it and stops its polling. - Rescan the LoRA picker when a run completes, so a freshly-trained adapter appears without a model reload. - Cap the dialog height and scroll the body so the Start/Stop footer stays reachable on short viewports. - Correct the copy to not over-promise picker auto-discovery. Freeing the resident Images pipeline before training is handled backend-side in the diffusion training start route. * Merge diffusion-sdxl into diffusion-lora-ux; keep options-only LoRA catch The catalog-refresh .catch from the lower branch clears the selected adapters too, which is right for its catalog-only picker but wrong here: this picker holds free-text HF repo ids that are valid without being in the catalog, so a transient refresh failure must not wipe them. Family swaps still clear the selection and hidden LoRAs are never sent. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Train LoRA dialog: stop suggesting absolute paths the backend rejects The dataset and output placeholders showed /path/to/... examples, but the training routes resolve those fields inside the Studio home and reject absolute paths outside the approved roots, so following the placeholder produced a 400. Use folder-name placeholders and say in the labels and the dialog description where each folder resolves. * Align the VAE to the denoiser's first FLOATING dtype, not its first parameter A GGUF-quantized transformer's leading parameters are packed uint8 storage, so reading next(parameters()).dtype handed nn.Module.to() an integer dtype and every image-conditioned generation on a GGUF model (Qwen-Image-Edit) failed with a 500. Probe the parameters for the first floating dtype, treat an all-integer module as a no-op, and also catch TypeError so an unexpected dtype can never break generation. Regression test included. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Count LR scheduler warmup/decay in optimizer steps, not micro-steps lr_sched.step() runs once per outer optimizer step (after the gradient accumulation inner loop), for train_steps total. The scheduler was configured with num_warmup_steps and num_training_steps multiplied by gradient_accumulation_steps, so with accumulation > 1 a warmup or non-constant schedule stretched past the run and never reached the intended decay. Count both in optimizer steps. * Address Codex review findings on the image-workflows PR Keep diffusion.py importable without torch: the compile/arch patch modules import torch at module level, so import them lazily at their load/unload call sites instead of at module load. This restores the torchless contract so get_diffusion_backend() works on a CPU/native sd.cpp install. Match family reject keywords and aliases as whole path/name segments, not raw substrings, so an unrelated word like edited, edition, or kontextual no longer misroutes or hides a valid base image model, while supported edit families (Qwen-Image-Edit, FLUX Kontext) still resolve. Mirror the same segment matching in the picker task filter. Route FLUX.2-dev native guidance through --guidance like the other FLUX families rather than --cfg-scale. Reject native upscale requests that have no input image. Read image header dimensions and reject over-limit inputs before decoding pixels, so a crafted small-payload image cannot spike memory. Reject an upscale that would shrink the source below its input size. Validate the model_kind against the filename extension before the GPU handoff. Estimate a local diffusers pipeline's size from its on-disk weights so auto memory planning does not skip offload and OOM. Report workflows: [txt2img] from the native backend status so the Create tab stays enabled for a loaded native model. Clamp the outpaint canvas to the backend's 4096px decode limit. Adds regression tests for segment matching and kind/extension validation. * Guard inference loads and worker lifetime against diffusion training Teach the chat and image load guards about an active diffusion (SDXL) LoRA job: a chat load is refused (its footprint cannot be fit-checked against the trainer) and an image load is refused outright, mirroring the existing LLM training guards, so a load can no longer allocate GPU memory alongside the trainer and undo the pre-start cleanup. Bind the diffusion trainer subprocess to the parent's lifetime and scrub the native path lease secret from it by running the child through run_without_native_path_secret, matching the inference/export/LLM workers, so a Studio crash or kill no longer leaves the trainer holding the GPU. Reset in_model_load on the complete and error terminal events: a stop or failure during model loading otherwise leaves the status reporting a stale loading indicator after the job has ended. * Harden diffusion LoRA handling on the diffusers and native paths Reject LoRA on a torch.compile'd diffusers transformer (Speed=default/max): diffusers requires the adapter loaded before compilation, so applying one to the already-compiled module fails with adapter-key mismatches. The status gate now hides the picker and generate raises a clear message instead. Convert a cancelled Hub LoRA download (RuntimeError Cancelled) to the diffusion cancellation sentinel in resolve_specs, so an unload/superseding load during resolution maps to a 409 instead of a generic server error. Drop weight-0 LoRA rows before the native support gate so a request carrying only disabled adapters stays a no-op on families where native LoRA is unsupported, matching the diffusers path. Reject duplicate LoRA ids in the request model: both apply paths suffix colliding names, so a repeated id would stack the same adapter past its per-adapter weight bound. Strip all user-typed <lora:...> prompt tags on the native path (only the selected adapters are materialized in the managed lora-model-dir, so an unselected tag can never resolve), and restore saved LoRA selections from a gallery recipe so restore reproduces a LoRA image. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden ControlNet resolve, gallery metadata, and the control-type picker Check cancellation immediately after a ControlNet from_pretrained and before any device placement, so an unload/eviction that raced the download does not allocate several GB onto the GPU after the load was already cleared. Require a loadable weight or shard index (not just config.json) before a local ControlNet folder is advertised, so an interrupted copy is hidden instead of failing deep in from_pretrained as a generic 500. Do not record a strength-0 ControlNet in the gallery recipe: it is treated as disabled and skipped, so the image is unconditioned and the metadata must not claim a ControlNet was applied. Build the control-type picker from the selected ControlNet's advertised control_types instead of a hardcoded passthrough/canny pair, so a union model with a precomputed depth or pose map sends the correct control_mode. * Address further Codex findings on the image-workflows PR - Persist the actual output image size in the gallery recipe instead of the request sliders: Transform/Inpaint/Edit derive the size from the uploaded image, Extend grows the canvas, and Upscale resizes it, so the sliders recorded (and later restored) the wrong dimensions for those workflows. - Reject a remote '*-GGUF' repo loaded as a full pipeline (no single-file name) in validate_load_request, so the unloadable pick fails before chat is evicted rather than deep in from_pretrained. - Only publish an image-conditioned from_pipe wrapper to the shared aux cache when the load is still current: from_pipe runs under the generate lock but not the state lock, so an unload racing its construction could otherwise cache a wrapper over torn-down modules that a later load would reuse. - Verify the Windows CUDA runtime archive checksum before extracting it, like the main sd-cli archive, so a corrupt or tampered runtime is rejected rather than extracted next to the binary. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refuse non-SDXL base models at diffusion training start The trainer only supports the SDXL U-Net, but a FLUX / Qwen-Image / Z-Image repo or a GGUF filename passed as base_model was accepted and then failed minutes later inside StableDiffusionXLPipeline.from_pretrained with an unrelated-looking error. Add a name-based guard in normalized() so known DiT-family names and .gguf checkpoints are rejected up front, which the API start route surfaces as an immediate 400 with a message that says exactly which bases are trainable. Unrecognisable names still pass through so custom local SDXL checkpoints keep working. * Add diffusion dataset upload and training info endpoints Training an image LoRA required knowing the Studio home layout and copying files onto the server by hand, which is the most confusing step of the whole flow. Two small endpoints fix that: - GET /api/train/diffusion/info reports the datasets and outputs roots plus every dataset folder that contains images (with image/caption counts), so the UI can offer a picker instead of a blind free-text path. - POST /api/train/diffusion/dataset uploads images and optional caption .txt / metadata.jsonl files into a named folder under the datasets root, creating it on first use and accumulating on repeat uploads so large sets can arrive in batches. Names are validated to a single path component and files stream to disk under the same per-upload size cap as LLM dataset uploads. The returned name is a valid data_dir for /diffusion/start. * Rework the Train LoRA dialog into a guided SDXL flow The dialog assumed users knew the Studio home layout and that only SDXL is trainable, and hid both facts behind free-text fields. Restructure it around the three real decisions: - Base model is a dropdown of the trainable SDXL picks (Base 1.0, Turbo, the loaded SDXL pipeline when there is one) with a custom repo/path escape hatch, instead of a bare text field defaulting to a repo id. - Training images come from an in-browser upload (new dataset endpoints) or a picker over existing dataset folders with image/caption counts. No shell access or knowledge of the datasets root is needed any more, and the captioning rules are explained inline. - The output field is now Adapter name and the instance prompt is labelled as the trigger prompt, with a no-captions warning wired to the selected dataset's actual caption count. Hyperparameters collapse behind a training settings toggle since the defaults suit a first run. A completed run says where the adapter went and offers Done / Train another, and the top-bar button gets an icon and a plainer description. The dialog title states the SDXL-only scope and that other families load LoRAs but cannot train them yet. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Validate diffusion training config before freeing the GPU The start route freed resident GPU workloads (export, Images pipeline, chat) before the service validated the config, so a start that was then refused, now including a non-SDXL base model, tore down the user's loaded model for nothing. Run the same cheap normalise pass first; the LLM path already follows this rule via its before_spawn hook. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactor diffusion LoRA training into a family-aware platform Split the SDXL trainer into a shared, architecture-agnostic layer so more model families can be trained without duplicating the plumbing: - New core/training/diffusion_train_common.py holds the config + validation, dataset discovery, event emission, stop protocol, adapter publishing, and a lazy trainer registry (get_trainer). diffusion_lora_trainer.py keeps the SDXL-specific loop and re-exports the moved names so existing imports are unchanged. - The SDXL-only base-model blocklist becomes a positive check: the family is resolved from the base model (or an explicit model_family) via the diffusion family registry, and a known-but-not-yet-trainable family is refused with a clear message. Unknown custom names still default to the SDXL trainer. - DiffusionFamily gains a trainable flag and train_base_repos; SDXL is marked trainable. DiT families flip on when their trainers land. - Trained adapters now write a <name>.json metadata sidecar (family, base model, rank, trigger prompt, ...) that the LoRA scanner reads to family-gate the adapter in the picker instead of showing it as unknown for every model. - The training base-model trust allowlist adds the official FLUX.1-dev, Z-Image-Turbo, and Qwen-Image repos (safetensors-only, no remote code). * Retain diffusion training loss history and expose it in status The training service kept only the latest loss, so a live loss chart could show a single point. Fold each progress event into bounded (step, loss, lr) history arrays (capped at 4000 points, decimated when full) plus the latest throughput and peak VRAM, and record the family / base model / catalog path on completion. The status endpoint returns these as a nested metric_history object the UI can chart directly, and the start request accepts an optional model_family override. * Tests for the diffusion training platform Cover the trainer registry (get_trainer resolves SDXL, unknown family raises), family resolution (explicit model_family validation, resolved_family on the config), the metadata sidecar write + scan read with family gating, and the service loss-history folding (append, bad-point skipping, decimation at cap, family/perf fields) plus the status route nesting metric_history. * Add diffusion dataset labeling and example-import endpoints The Train tab needs to let users caption small datasets in the browser and pull in a ready-made set to see training work end to end, neither of which the upload-only endpoint supported. Add, under /api/train/diffusion/dataset: - GET {name}/images lists every image with its resolved caption (metadata beats a per-image sidecar, matching the trainer's discovery order) so uncaptioned images are visible and flaggable. - GET {name}/image/{filename} serves an image, with ?thumb=<px> returning a cached downscaled JPEG kept in a hidden .thumbs subdir (regenerated when the source is newer) so the labeling grid stays light. - PUT {name}/caption/{filename} writes, or when blank clears, the .txt sidecar; DELETE {name}/image/{filename} removes the image plus its sidecars and thumbnails. - GET dataset-examples lists a curated, license-labelled registry, and POST dataset/import-example materializes one into a dataset folder as numbered images + .txt captions. Two loaders cover the shapes seen in the wild: streaming rows from datasets.load_dataset (dog-example, Tuxemon) and a snapshot + jsonl walk for imagefolder repos whose captions live in a non-standard *.jsonl (the public-domain tarot set). Imports are idempotent and cap the image count. Filenames and dataset names are validated against path traversal and pinned inside the datasets root. * Test diffusion dataset labeling and example-import endpoints Cover caption precedence, thumbnail generation and .thumbs exclusion, caption write/clear, image delete cleanup, path-traversal rejection on names and filenames, and example import with a mocked datasets.load_dataset (files plus sidecars written, idempotent second call, cap respected, load failure mapped to 502). * Add flow-matching DiT LoRA trainers (FLUX.1-dev, Qwen-Image, Z-Image) Extends diffusion LoRA training beyond SDXL to the three popular DiT families via a single shared flow-matching loop parameterised by small per-family specs (loading, prompt/latent encoding, transformer forward, save). Verified against diffusers 0.38.0: - FLUX.1-dev: 2x2 latent packing + image ids, guidance-embed forward, on-the-fly nf4 QLoRA of the 12B transformer (the dev repo is gated, so training needs the user's HF token). - Qwen-Image: 5D VAE latents normalised by the per-channel latents_mean/std, img_shapes forward, prequant nf4 base by default (on-the-fly nf4 for the bf16 base). - Z-Image: list I/O with the reversed timestep convention and a negated prediction, bf16 only. The registry (get_trainer) and DiffusionFamily.trainable / train_base_repos now route these families to the DiT trainer; the SDXL blocklist guard is replaced by a positive family resolution that also rejects GGUF repos (inference-only) and still-unsupported families. Per-family defaults + labels + VRAM notes are exposed via family_train_infos for the Train UI. Memory: caption embeddings are precomputed once and the text encoders freed before the loop; gradient checkpointing (non-reentrant, required for bnb 4-bit) and 8-bit AdamW are on by default. * Speed up + shrink SDXL LoRA training (precompute text embeds, 8-bit AdamW) SDXL re-encoded every caption with both CLIP text encoders on every step (pure waste, since captions are constant) and kept the encoders resident. Precompute each unique caption's embeddings once, then free the text encoders before the loop: numerically identical (embeddings are deterministic and this consumes no torch RNG, so the noise/timestep stream is unchanged) but faster and ~1.5 GB lighter. Default the optimizer to 8-bit AdamW (bitsandbytes) with an fp32 fallback, halving optimizer state with no meaningful LoRA quality cost. Env toggles (UNSLOTH_DIFFUSION_NO_PRECOMPUTE / _FP32_OPTIM) let the accuracy guard A/B the paths. * Expose trainable families in /diffusion/info and preflight gated bases The training info endpoint now returns the trainable model families (name, label, default + allowed base repos, recommended defaults, and a VRAM/access note) so the Train UI can offer a base picker with realistic guidance. The start route preflights a gated base repo (HEAD model_index.json with the user's token) BEFORE freeing resident GPU workloads, so a missing FLUX.1-dev license/token fails fast with an actionable 400 instead of evicting the loaded model and then hitting a confusing mid-load 401. * Tests for DiT trainers, family resolution, info families, gated preflight Cover the DiT spec table, the QLoRA prequant heuristic, the Z-Image bf16-only guard, the gated-repo name check, family resolution now that FLUX/Qwen/Z-Image are trainable (and GGUF repos are rejected as inference-only), the families list in /diffusion/info, and the gated-base 400 preflight that leaves the GPU untouched. * Add diffusion training API client: metrics, families, dataset labeling, examples Extends the Images training client for the Train tab: the status type now carries metric_history (step/loss/lr) plus catalog_path/family/base_model/samples_per_second/ peak_memory_gb; the start request gains model_family; and info gains an optional families list (per-family bases + defaults). Adds typed calls for the dataset labeling and one-click example endpoints: list images with captions, thumbnail URL, write/clear a caption, delete an image, list example datasets, and import an example. * Add diffusion Train panel: config, dataset labeling, live charts, deploy New full-page training workspace for the Images tab. Left column configures the run: model family (FLUX.1-dev, Qwen-Image, Z-Image, SDXL in popularity order, with per-family VRAM/license notes and defaults, backfilled from the backend families list when present), base repo, dataset (existing folder, browser upload, or one-click example import), an in-browser caption labeling grid (per-image thumbnail + caption saved on blur, delete, uncaptioned highlight), adapter name, trigger prompt, and collapsed training settings. Right column shows the live run: progress + loss/avg/speed/peak-VRAM readouts, the reused training loss/LR charts fed from metric_history, and a completion card that deploys the adapter into Create or starts another run. * Wire Create/Train tab switch into the Images page and deploy flow Replaces the Train LoRA dialog with a top-bar Create | Train segmented control next to the model selector. Create renders the existing generation workspace unchanged; Train renders the full-page training panel (unmounted in Create so its polling stops while the backend run and its retained metric history survive a tab switch). Adds a deploy handler: loading the trained adapter's base as a pipeline, queueing the adapter so the LoRA discovery effect applies it once the base is loaded and LoRA-capable for the matching family (with a mismatch warning), seeding the prompt with the trigger, and switching back to Create. Removes the now-unused dialog. * Wrap the DiT training forward in bf16 autocast The fp32 LoRA parameters and the bnb 4-bit base matmuls need a single compute dtype during the forward, exactly like the diffusers dreambooth scripts run under accelerator.autocast. Without it the 4-bit backward on FLUX.1-dev fails with an illegal-address CUBLAS error partway into the first step. Z-Image and Qwen-Image smokes are unaffected and the SDXL path (its own trainer) is untouched. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Train tab example cards and Create/Train tab layout The example-dataset cards used a two-column grid in the ~340px config column, which wrapped titles one word per line and let the long license text overrun into the neighbouring card. Switch to one card per row with a horizontal layout: title with a compact truncated license badge (full text in the tooltip), a two-line clamped description, and the Import button on the right. The Create/Train switch had an icon inside the Train trigger that overhung the pill corner. Drop the icon, make both triggers a fixed equal width so the active pill sits flush in the top bar. * Show only loss and learning-rate charts for diffusion training The Train tab reused the LLM charts section, which also rendered an empty Grad Norm card and an Eval Loss card showing an Evaluation not configured placeholder with a red smear. Neither applies to diffusion LoRA training. Add a diffusion-only two-card view that reuses the loss and learning-rate cards directly with fixed presentation defaults, and note under the loss chart that per-step loss is noisy by design so users read the smoothed line for the trend rather than the raw jitter. * Add a dataset preview strip to the Train tab When a dataset with images is selected, show a strip of up to 8 sampled thumbnails with a +N more tile, so users can see what is in the folder before training. Clicking the strip opens the existing caption review grid. Samples are drawn evenly across the folder and refresh on dataset change or after an upload/import. * Stop example cards from overflowing the Train config column The example-dataset cards still overran the ~340px config column: the license used the Badge component whose baked-in w-fit and whitespace-nowrap ignored the max-width and truncate, and the grid children had the default min-width auto so wide content pushed past the column edge and clipped the Import buttons. Replace the badge with a plain truncating pill span, and give the config column min-w-0 with overflow-x-hidden so nothing escapes its width. * Add Smithsonian Butterflies and Nouns example datasets Two permissive ~100-image sets for the Train tab: huggan/smithsonian_butterflies_subset (CC0, the classic diffusers-docs training set, imported as a subject set with a trigger prompt since its metadata columns are species names not captions) and m1guelpf/nouns (CC0, captioned pixel-art avatars via the text column). Both cap at 100 images. * Paginate the Train tab caption grid with prev/next controls Large example datasets (100+ images) rendered every tile at once, so the caption review grid grew unbounded. Show 24 images per page with < > chevrons and an x-y of N indicator; a new dataset or refresh resets to the first page. * Offer example datasets in the Train dropdown with previews Add an Examples group to the training-images dropdown that imports a curated dataset in one pick, alongside the existing cards. Cards now show up to three preview thumbnails pulled from the public HF datasets-server so the set is visible before download. Hide the trigger prompt when every image already has a caption (a captioned style set needs no trigger), and turn the training-settings toggle into a ghost button with a rotating chevron. * Clamp the training base repo to the selected family The base-model select's state could briefly hold the previous family's repo after a family switch (the reseed effect runs a beat later, and a value with no matching option makes the browser display the first option anyway). The request then carried the stale repo: picking Qwen or Z-Image still sent black-forest-labs/FLUX.1-dev and surfaced FLUX's gated-repo error under the wrong family. Derive an effectiveBase clamped to the current family's repos and use it for the select value, the start request, and the deploy fallback. Also move the Trigger prompt above Adapter name: the trigger describes the dataset, the name only labels the output. * Speed up diffusion LoRA training and cut DiT peak VRAM by a third Perf core for the diffusion trainers, defaults preserving the training math: - Phased model loading: the pipeline now loads without its transformer (conditioning only), captions are encoded and the text encoders freed, the VAE latent cache is built and the VAE freed, and only then does the transformer load. The multi-GB denoiser never shares VRAM with the encoders, cutting measured peak VRAM on B200: FLUX 17.1 -> 10.4 GB, Qwen-Image 19.1 -> 12.8 GB, Z-Image 7.3 -> 4.7 GB. - Latent cache (cache_latents, default on): per-image crop/flip variants (cache_variants, default 4 vs the single frozen variant of the diffusers --cache_latents) store the VAE posterior's affine parameters, so every step still draws a fresh VAE sample; a cached center-crop Z-Image run matches the uncached one at the bf16 nondeterminism floor. - True batching: train_batch_size now actually batches the transformer forward (it was silently 1). nf4 dequant dominates the step cost, so batch 4 lands near batch-1 step time: 4.0x samples/s on Qwen-Image, 3.1x on FLUX, 2.1x on Z-Image, with multi-seed loss envelopes overlapping batch-1. - LR scheduler support in the DiT loop (lr_scheduler / lr_warmup_steps were accepted but ignored); progress events now report the real per-step LR. - TF32 + high fp32 matmul precision under enable_tf32 (default on), snapshot/restored around the run. cudnn.benchmark is scoped to a caller opt-in only: autotuning the fp32 VAE convs doubled peak VRAM on the DiT families for zero steady-state gain. - Vectorized sigma gathering (drops a per-step Python search loop), cached FLUX img_ids/guidance, fused torch AdamW fallback, steady-state samples_per_second (excludes the first-step warmup). - Regional torch.compile plumbing (compile_transformer off/on/auto with eager fallback): auto stays off over a bitsandbytes base where compile is a net loss (27 s warmup, slightly slower steady on Z-Image); it arms automatically for the dense/quantized speed modes that follow. - Stop parity with the LLM trainer: /api/train/diffusion/stop accepts an optional {save} body and the service forwards save=False as a no-save cancel; a new preparing event surfaces cache-build progress. - SDXL trainer gets the same latent cache, perf flags, and fused fallback; its batching, LR schedule, and min-SNR stay as they were. Verified: 83 backend tests green; per-family 30-40 step runs with adapter round-trip generation through the normal LoRA path (FLUX, Qwen-Image, Z-Image all pass). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add base_precision speed modes to DiT training: bf16 2.3-2.6x, int8, fp8 New base_precision config for the DiT trainers: nf4 (unchanged default) | bf16 | int8 | fp8 | auto, advertised per family + per machine through /api/train/diffusion/info (precision_modes, recommended_precision, supports_compile) so the UI can gate the selector. - bf16: dense transformer + regional torch.compile (auto-armed). The measured speed mode: 2.3x nf4 on FLUX (1.81 -> 4.12 steps/s), 2.6x on Z-Image (2.5 -> 6.38 steps/s) on B200, at dense-weight VRAM (FLUX 24.7 GB / Z-Image 13.6 GB peak vs 10.4 / 4.7 for nf4). - int8: torchao weight-only int8 on the frozen base, quantized AFTER add_adapter (quantizing first trips peft 0.18's TorchaoLoraLinear, which is incompatible with the torchao 0.16 config API). Runs eager: inductor rejects the int8 subclass training graph (aliased subclass outputs), so compile is force-disabled for it. - fp8: torchao convert_to_float8_training on the frozen linears (filter skips lora_ modules, proj_out, non-divisible-by-16 dims, pad_inner_dim), applied after add_adapter, compile auto-armed. Works and round-trips, but measured SLOWER than compiled bf16 at LoRA-training shapes (FLUX 3.15 vs 4.12 steps/s; Z-Image similar), so it is an explicit opt-in and auto never picks it. - auto: free VRAM (measured before load) + dense-size table -> bf16 when it fits with headroom, int8 in the middle band, else nf4. Prequant bnb repos always resolve to nf4; dense modes on them are rejected at validation with a pointer to the family's dense base. Two crashes found and fixed along the way: - The cuDNN SDPA backend's training graph fails on the FLUX attention shapes (torch 2.10 + cu130, B200): mha_graph.execute errors, then the context degrades into illegal memory accesses. The perf-flag guard now pins flash/mem-efficient SDPA for the run (mathematically equivalent, snapshot/restored). nf4 escaped it by routing attention differently. - Regional compile now uses dynamic=True (the inference layer's proven default): dynamic=False specialisation fused a gemm_and_bias epilogue that failed with CUBLAS_STATUS_EXECUTION_FAILED on the FLUX training graph; dynamic=True is also faster (Z-Image 3.84 -> 6.38 steps/s). Verified: 98 backend tests green (new test_diffusion_base_precision.py: validation, auto policy table, fp8 filter, compile gating, /info fields); per-mode 40-step runs on FLUX + Z-Image with loss means inside the nf4 envelope and adapter round-trip generation through the normal LoRA path for bf16-, fp8-, and int8-trained adapters. * Clear TF32 flags when enable_tf32 is off so the opt-out is strict fp32 * Address review: auto int8 requires the dense-load transient to fit, dense modes are CUDA-only, auto respects bf16 compute, exact cudnn SDPA restore * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: fp32 latent cache stats + strict-JSON-safe progress floats - Latent caches (DiT + SDXL) now hold the posterior mean/std in fp32 and draw the per-step sample in fp32, casting only the result to the training dtype. This matches the in-loop path (encode fp32 -> sample fp32 -> cast) exactly instead of sampling in bf16; the cache is tiny so the doubled RAM is negligible. - The training service nulls non-finite floats (NaN/Inf loss, avg_loss, learning_rate) at its single ingestion point so status snapshots and persisted run records stay strict-JSON serializable; the metric history skips non-finite loss points. Test covers NaN/Inf progress followed by a finite point. * Address review: gate auto int8 on torchao, scope dense validation to DiT - base_precision="auto" only picks int8 when torchao is importable (the int8 quantize has no runtime fallback, unlike fp8); otherwise the middle band falls back to nf4. Threaded as a parameter so the policy stays pure. - The dense-mode validation (prequant base / bf16 compute) now applies only to DiT families: sdxl ignores base_precision entirely, so a leftover value can no longer fail an SDXL run. The mode-name validity check still runs everywhere. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate int8 and fp8 on a functional torchao import, not find_spec The Windows ROCm torchao import stub satisfies find_spec and even lets from torchao.quantization import quantize_ succeed, but its quantize_ is a no-op: auto would pick int8, leave the transformer dense, and disable compile as if it were quantized. has_functional_torchao imports the exact symbols the int8 path uses and rejects the stub via its sentinel; both the auto picker and the /info advertised modes now use it * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip the sigma-gather test when diffusers is not installed CI runs the backend suite without diffusers; the test checks our index math against the scheduler's own gather, so it skips rather than fails there. * Coerce cache_latents and enable_tf32 string flags in the config dict path The generic Studio config dict path can deliver these flags as strings, and a non-empty string like "false" is truthy, so an opt-out silently no-ops (the latent cache still builds, TF32 stays on). Coerce them the same way gradient_checkpointing already is. * Remove committed runtime scratch artifacts and ignore their dirs logs/ (a 1.3 MB ComfyUI object_info dump plus stale PID files), temp/ (PR body and commit message scratch), and async_task_outputs/ (agent task transcripts) are environment specific runtime artifacts that were committed by accident and carry stale local state into every checkout. Remove them and gitignore the directories so they cannot be re-added. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore pre-Ampere bf16 fail-fast in the DiT trainer The perf rewrite dropped the bf16 capability guard, so a pre-Ampere CUDA device (T4/V100/RTX 20xx) would die deep in model load with an opaque dtype error instead of a clear message. Restores parity with the SDXL trainer. * Size-gate the automatic diffusion latent cache The latent cache holds two fp32 posterior tensors per crop/flip variant per image, pinned on CUDA hosts, so datasets with thousands of images can exhaust host or pinned memory with no fallback. Estimate the cache size from the first real encoded latent and fall back to per-step VAE encoding when it exceeds a 4 GiB budget. UNSLOTH_DIFFUSION_FORCE_LATENT_CACHE bypasses the gate; the existing UNSLOTH_DIFFUSION_NO_LATENT_CACHE opt-out is unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate DiT training precision: deny fp8 for Qwen, gate explicit int8 on torchao, gate advertised dense modes + route on bf16 - normalized() + family_train_infos() mirror the inference fp8 deny for Qwen-Image (activation outliers exceed fp8's range and corrupt the trained result); int8 stays allowed and the UI no longer advertises fp8 for it. - _resolve_base_precision() gates an explicit int8 on a FUNCTIONAL torchao, the same gate auto and /info already apply, so a missing/stub torchao fails fast instead of silently loading dense with compile disabled. - train_precision_modes() gates the dense modes (bf16/int8/fp8/auto) on torch.cuda.is_bf16_supported(), so a non-bf16 CUDA GPU (T4/V100/RTX 20xx) is offered only nf4 instead of a start that evicts resident models and then fails. - start_diffusion_training preflights bf16 support for the DiT families BEFORE _free_gpu_for_diffusion_training(), so any DiT start (nf4 included, since the trainer requires bf16 unconditionally on CUDA) fails fast without eviction. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate DiT training on functional torchao for explicit int8; hide always-400 DiT modes on non-bf16 GPUs The start route preflight only rejected non-bf16 GPUs; an explicit int8 request on a host with a missing or stub torchao passed the preflight, evicted resident GPU workloads, then died in the trainer child (its int8 base quantizer has no fallback). Fold both gates into training_precision_preflight_error so int8-without-torchao fails fast before eviction. Also empty the advertised DiT precision_modes (and surface the reason in vram_note, drop compile) whenever the bf16 preflight would reject the family, so /info never offers an nf4 DiT option the route always 400s. * Reject dense DiT precisions on a CUDA-absent host before eviction; stabilize family-info tests The start-route preflight caught the bf16-GPU and int8-torchao requirements but not the dense precisions' CUDA requirement: on a GPU-less host bf16_unsupported_reason exempts CPU-only, so a bf16/fp8 (or int8-with-torchao) DiT request passed the preflight, evicted resident workloads, then raised only in the trainer child. Add the dense-mode CUDA gate mirroring _resolve_base_precision so the doomed run is rejected up front. Also pin bf16_unsupported_reason in the two positive-path family-info tests so they are deterministic across GPU types (a non-bf16 CUDA box would otherwise empty every DiT family's advertised modes). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|---|---|---|
| .github | ||
| images | ||
| plans | ||
| scripts | ||
| studio | ||
| tests | ||
| unsloth | ||
| unsloth_cli | ||
| .gitattributes | ||
| .gitignore | ||
| .pre-commit-ci.yaml | ||
| .pre-commit-config.yaml | ||
| build.sh | ||
| cli.py | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| install.ps1 | ||
| install.sh | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| unsloth-cli.py | ||
Unsloth Studio lets you run and train models locally.
Features • Quickstart • Notebooks • Documentation
⚡ Get started
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Windows:
irm https://unsloth.ai/install.ps1 | iex
Community:
⭐ Features
Unsloth Studio (Beta) lets you run and train text, audio, embedding, vision models on Windows, Linux and macOS.
Inference
- Search + download + run models including GGUF, LoRA adapters, safetensors
- Export models: Save or export models to GGUF, 16-bit safetensors and other formats.
- Tool calling: Support for self-healing tool calling and web search
- Code execution: lets LLMs test code in Claude artifacts and sandbox environments
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools with Unsloth
- Auto set inference settings and customize chat templates.
- We work directly with teams behind gpt-oss, Qwen3, Llama 4, Mistral, Gemma 1-3, and Phi-4, where we’ve fixed bugs that improve model accuracy.
- Chat with images, audio, PDFs, code, DOCX and more. Connect API providers (OpenAI, Anthropic) or servers (vLLM, Ollama).
Training
- Train and RL 500+ models up to 2x faster with up to 70% less VRAM, with no accuracy loss.
- Custom Triton and mathematical kernels. See some collabs we did with PyTorch and Hugging Face.
- Data Recipes: Auto-create datasets from PDF, CSV, DOCX etc. Edit data in a visual-node workflow.
- Reinforcement Learning (RL): The most efficient RL library, using 80% less VRAM for GRPO, FP8 etc.
- Supports full fine-tuning, RL, pretraining, 4-bit, 16-bit and, FP8 training.
- Observability: Monitor training live, track loss and GPU usage and customize graphs.
- Multi-GPU training is supported, with major improvements coming soon.
📥 Install
Unsloth can be used in two ways: through Unsloth Studio, the web UI, or through Unsloth Core, the code-based version. Each has different requirements.
Unsloth Studio (web UI)
Unsloth Studio (Beta) works on Windows, Linux, WSL and macOS.
- CPU: Supported for Chat and Data Recipes currently
- NVIDIA: Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
- macOS: Training, MLX and GGUF inference are ALL supported.
- AMD: Chat + Data works. Train with Unsloth Core. Studio support is out soon.
- Multi-GPU: Available now, with a major upgrade on the way
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Use the same command to update.
Windows:
irm https://unsloth.ai/install.ps1 | iex
Use the same command to update.
Launch
unsloth studio -p 8888
For cloud or global access, add -H 0.0.0.0. By default, Unsloth is accessible only locally.
To reach Studio over HTTPS, use unsloth studio --secure. Studio stays bound to localhost and is reached only through a free Cloudflare tunnel, which publishes it at a public https://*.trycloudflare.com URL (it fails closed if the tunnel can't start, so the raw port is never exposed). This makes Studio reachable from the internet, so anyone with the link and API key can use it and run code: keep your API key private (see Remote access below).
Docker
Use our Docker image unsloth/unsloth container. Run:
docker run -d -e JUPYTER_PASSWORD="mypassword" \
-p 8888:8888 -p 8000:8000 -p 2222:22 \
-v $(pwd)/work:/workspace/work \
--gpus all \
unsloth/unsloth
Developer, Nightly, Uninstall
To see developer, nightly and uninstallation etc. instructions, see advanced installation.
Unsloth Core (code-based)
Linux, WSL:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto
Windows:
winget install -e --id Python.Python.3.13
winget install --id=astral-sh.uv -e
uv venv unsloth_env --python 3.13
.\unsloth_env\Scripts\activate
uv pip install unsloth --torch-backend=auto
For Windows, pip install unsloth works only if you have PyTorch installed. Read our Windows Guide.
You can use the same Docker image as Unsloth Studio.
AMD, Intel:
For RTX 50x, B200, 6000 GPUs: uv pip install unsloth --torch-backend=auto. Read our guides for: Blackwell and DGX Spark.
To install Unsloth on AMD and Intel GPUs, follow our AMD Guide and Intel Guide.
📒 Free Notebooks
Train for free with our notebooks. You can use our new free Unsloth Studio notebook to run and train models for free in a web UI. Read our guide. Add dataset, run, then deploy your trained model.
| Model | Free Notebooks | Performance | Memory use |
|---|---|---|---|
| Gemma 4 (E2B) | ▶️ Start for free | 1.5x faster | 50% less |
| Qwen3.5 (4B) | ▶️ Start for free | 1.5x faster | 60% less |
| gpt-oss (20B) | ▶️ Start for free | 2x faster | 70% less |
| Qwen3.5 GSPO | ▶️ Start for free | 2x faster | 70% less |
| gpt-oss (20B): GRPO | ▶️ Start for free | 2x faster | 80% less |
| Qwen3: Advanced GRPO | ▶️ Start for free | 2x faster | 70% less |
| embeddinggemma (300M) | ▶️ Start for free | 2x faster | 20% less |
| Mistral Ministral 3 (3B) | ▶️ Start for free | 1.5x faster | 60% less |
| Llama 3.1 (8B) Alpaca | ▶️ Start for free | 2x faster | 70% less |
| Llama 3.2 Conversational | ▶️ Start for free | 2x faster | 70% less |
| Orpheus-TTS (3B) | ▶️ Start for free | 1.5x faster | 50% less |
- See all our notebooks for: Kaggle, GRPO, TTS, embedding & Vision
- See all our models and all our notebooks
- See detailed documentation for Unsloth here
🦥 Unsloth News
- Connections: Connect any API provider (OpenAI, Anthropic) or server (vLLM, Ollama). Guide
- MTP: Run Qwen3.6 MTP in Unsloth. MTP settings are autoset specific to your hardware. Guide
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools. Guide
- Qwen3.6: Qwen3.6-35B-A3B can now be trained and run in Unsloth Studio. Blog
- Gemma 4: Run and train Google’s new models directly in Unsloth. Blog
- Introducing Unsloth Studio: our new web UI for running and training LLMs. Blog
- Qwen3.5 - 0.8B, 2B, 4B, 9B, 27B, 35-A3B, 112B-A10B are now supported. Guide + notebooks
- Train MoE LLMs 12x faster with 35% less VRAM - DeepSeek, GLM, Qwen and gpt-oss. Blog
- Embedding models: Unsloth now supports ~1.8-3.3x faster embedding fine-tuning. Blog • Notebooks
- New 7x longer context RL vs. all other setups, via our new batching algorithms. Blog
- New RoPE & MLP Triton Kernels & Padding Free + Packing: 3x faster training & 30% less VRAM. Blog
- 500K Context: Training a 20B model with >500K context is now possible on an 80GB GPU. Blog
- FP8 & Vision RL: You can now do FP8 & VLM GRPO on consumer GPUs. FP8 Blog • Vision RL
📥 Advanced Installation
The below advanced instructions are for Unsloth Studio. For Unsloth Core advanced installation, view our docs.
Developer / Nightly / Experimental installs: macOS, Linux, WSL:
The developer install builds from the main branch, which is the latest (nightly) source.
git clone https://github.com/unslothai/unsloth
cd unsloth
./install.sh --local
unsloth studio -p 8888
To install into an isolated location (its own virtual env, auth/, studio.db, cache and llama.cpp build), set UNSLOTH_STUDIO_HOME and pass it again at launch:
UNSLOTH_STUDIO_HOME="$PWD/.studio" ./install.sh --local
UNSLOTH_STUDIO_HOME="$PWD/.studio" unsloth studio -p 8888
Then to update :
cd unsloth && git pull
./install.sh --local
unsloth studio -p 8888
Developer / Nightly / Experimental installs: Windows PowerShell:
The developer install builds from the main branch, which is the latest (nightly) source.
git clone https://github.com/unslothai/unsloth.git
cd unsloth
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888
To install into an isolated location (its own virtual env, auth/, studio.db, cache and llama.cpp build), set UNSLOTH_STUDIO_HOME and pass it again at launch:
$env:UNSLOTH_STUDIO_HOME="$PWD\.studio"; .\install.ps1 --local
$env:UNSLOTH_STUDIO_HOME="$PWD\.studio"; unsloth studio -p 8888
Then to update :
cd unsloth; git pull
.\install.ps1 --local
unsloth studio -p 8888
Remote access: --secure (HTTPS tunnel) vs raw port
By default unsloth studio binds to 127.0.0.1 (this machine only). To reach it from another device, pick one of:
--secure(recommended): serve only through a free Cloudflare HTTPS link. Studio stays bound to localhost and the tunnel provides the public URL; it fails closed (does not start) if the tunnel can't come up, so the raw port is never exposed.
unsloth studio --secure -p 8888
-H 0.0.0.0: bind the raw port on all network interfaces, reachable from anywhere on the network. Only use this on a trusted network.
unsloth studio -H 0.0.0.0 -p 8888
Server-side tools (web search, Python and terminal code execution) run as your user and are on by default. Anyone who can reach the server with the API key can run code on this machine, so keep your API key private and pass --disable-tools when exposing Studio.
Advanced launch options
Installer options can be passed as environment variables. On macOS, Linux and WSL place the variable after the pipe so the shell passes it to sh; on Windows set it with $env: before piping to iex.
Skip PyTorch (GGUF-only mode):
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_NO_TORCH=1 sh
$env:UNSLOTH_NO_TORCH=1; irm https://unsloth.ai/install.ps1 | iex
Pin the Python version:
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_PYTHON=3.12 sh
$env:UNSLOTH_PYTHON='3.12'; irm https://unsloth.ai/install.ps1 | iex
Install to a custom location with UNSLOTH_STUDIO_HOME:
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_STUDIO_HOME=/abs/path sh
$env:UNSLOTH_STUDIO_HOME='C:\path'; irm https://unsloth.ai/install.ps1 | iex
On macOS, the installer defaults to the system certificate store (UV_SYSTEM_CERTS=1) so uv trusts the CAs in your Keychain, needed behind TLS-inspecting proxies (Cisco Umbrella, Zscaler, etc.). Opt out with:
curl -fsSL https://unsloth.ai/install.sh | UV_SYSTEM_CERTS=0 sh
Point the frontend build at a corporate npm mirror/proxy with UNSLOTH_NPM_REGISTRY (for the developer install behind a firewall that blocks registry.npmjs.org):
UNSLOTH_NPM_REGISTRY=https://artifactory.example.com/api/npm/npm/ ./install.sh --local
$env:UNSLOTH_NPM_REGISTRY='https://artifactory.example.com/api/npm/npm/'; .\install.ps1 --local
It is threaded as --registry into the Studio frontend npm/bun installs; the supply-chain locks (7-day min-release-age, exact version pins) stay in force.
Cap Studio's native CPU thread pools on high-core hosts: UNSLOTH_CPU_THREADS=8 unsloth studio -p 8888.
Uninstall
The recommended way to fully remove Unsloth Studio is the matching uninstall script for your OS. It stops any running servers, removes the install dir, the launcher data dir, the desktop shortcut, and any platform-specific entries (macOS .app bundle + Launch Services on Mac; Start Menu, HKCU\Software\Unsloth registry key and user PATH entries on Windows):
- MacOS, WSL, Linux:
curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.sh | sh - Windows (PowerShell):
irm https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.ps1 | iex
If you only want to drop the install dir and keep the launcher/shortcut for a later reinstall, you can instead run rm -rf ~/.unsloth/studio (Mac/Linux/WSL) or Remove-Item -Recurse -Force "$HOME\.unsloth\studio" (Windows). The model cache at ~/.cache/huggingface is not touched by any of these.
For more info, see our docs.
Deleting model files
You can delete old model files either from the bin icon in model search or by removing the relevant cached model folder from the default Hugging Face cache directory. By default, HF uses:
- MacOS, Linux, WSL:
~/.cache/huggingface/hub/ - Windows:
%USERPROFILE%\.cache\huggingface\hub\
💚 Community and Links
| Type | Links |
|---|---|
 Discord |
Join Discord server |
 r/unsloth Reddit |
Join Reddit community |
| 📚 Documentation & Wiki | Read Our Docs |
_logo_late_2025.svg){kind=logo} Twitter (aka X) |
Follow us on X |
| 🔮 Our Models | Unsloth Catalog |
| ✍️ Blog | Read our Blogs |
Citation
You can cite the Unsloth repo as follows:
@software{unsloth,
author = {Daniel Han, Michael Han and Unsloth team},
title = {Unsloth},
url = {https://github.com/unslothai/unsloth},
year = {2023}
}
If you trained a model with 🦥Unsloth, you can use this cool sticker!
License
Unsloth uses a dual-licensing model of Apache 2.0 and AGPL-3.0. The core Unsloth package remains licensed under Apache 2.0, while certain optional components, such as the Unsloth Studio UI are licensed under the open-source license AGPL-3.0.
This structure helps support ongoing Unsloth development while keeping the project open source and enabling the broader ecosystem to continue growing.
Thank You to
- The llama.cpp library that lets users run and save models with Unsloth
- The Hugging Face team and their libraries: transformers and TRL
- The Pytorch and Torch AO team for their contributions
- NVIDIA for their NeMo DataDesigner library and their contributions
- And of course for every single person who has contributed or has used Unsloth!