10 commits

Author	SHA1	Message	Date
Daniel Han	85a036a195	Keep an explicitly requested float32 model in float32 without bfloat16 (#7867) ... * Keep an explicitly requested float32 model in float32 without bfloat16 A model loaded with dtype = torch.float32 was being wrapped in float16 autocast on V100/T4 even when the caller set fp16 = False and bf16 = False. The mixed precision block in rl.py reads "neither flag set" as "caller did not choose" and picks for them: use_bf16_amp = (not float16) and _bf16_supported() args.fp16 = not use_bf16_amp so on a GPU without bf16 the answer is always float16. float16 has five exponent bits against float32's eight, so a value the model was loaded wide enough to hold overflows to inf and then NaN. Spark_TTS_(0_5B) is the case that surfaced it: it loads with dtype = torch.float32 ("Spark seems to only work on float32 for now"), sets fp16 = False, bf16 = False, and on a T4 logs nan for every step, then dies at inference inside torch.multinomial, which refuses a distribution containing NaN. The sampler assert is two steps downstream of the cause. Suppressing the autocast for every float32 model is too wide: full finetuning upcasts trainable weights to float32 by itself, and float16 autocast over float32 master weights is the ordinary V100/T4 recipe (issue #4082, tests/python/test_v100_fullft_precision.py). So from_pretrained now records whether float32 was ASKED for in UNSLOTH_USER_FLOAT32, as opposed to arrived at by upcasting, and only an explicit request suppresses the autocast. It is set in both loaders, since FastLanguageModel and FastBaseModel resolve dtype independently, and written on every load so a previous run cannot leave a stale "1" behind. Only the float16 fallback changes. bf16 has float32's exponent range, so a bf16 GPU keeps autocasting and keeps the memory saving. An explicit fp16 = True or bf16 = True is still obeyed, float16 and bfloat16 models are untouched, and UNSLOTH_FORCE_FLOAT32 models still take their earlier branch. tests/test_float32_no_fp16_autocast.py pulls the mixed precision block out of rl.py by AST and executes it against fake args/model objects with the bf16 capability check stubbed, so the no-bf16 path is covered without a V100/T4. No GPU, no network, no trl import. Also, separately: do not let one optional audio dependency kill the whole test session. require_python_package calls sys.exit(1) when a package is missing, and every caller is a test module invoking it at import time, so under pytest that is not a skip of one module -- SystemExit propagates out of collection and the run ends with INTERNALERROR> SystemExit: 1 with no report at all. A missing xcodec2 took out 1000+ unrelated tests. It now skips the module under pytest; outside pytest the exit is unchanged, since these helpers are also used by standalone scripts. test_whisper.py had the same shape for a different reason: it downloads an audio fixture at import and did `assert False` on failure. Wikimedia rate limits and this URL really did answer 429 during a batch run, and an asset we could not fetch says nothing about unsloth, so it skips too. * Record the float32 request on the model, and let GRPO honour a disabled autocast Two real gaps in the float32 gating, plus the provenance problem underneath both. The gate never reached most users. llama, mistral, gemma, gemma2, qwen2 and qwen3 LoRA/QLoRA loads go through FastLanguageModel to dispatch_model.from_pretrained, which is neither of the two loaders that recorded the request, so user_float32 stayed false and the fp16 autocast this branch exists to avoid ran anyway. That is most of the notebooks. The value was also process-global and derived rather than asked for. A program that loads two models before building a trainer described whichever loaded last, and a dtype inferred from a 4bit config's bnb_4bit_compute_dtype was recorded as an explicit model-wide request, which is especially wrong under full finetuning where the loader then turns 4bit off again. Both entry points now read the argument as the caller wrote it, before any normalisation, and the answer is attached to the model. The outermost caller wins, since only it saw the raw argument; the two delegating returns in FastLanguageModel re-stamp it for that reason. Then ACCELERATE_MIXED_PRECISION = 'no' was read as a two-way switch in the GRPO replacements, so 'no' came out as bfloat16 and autocast was entered anyway. torch does not ignore that on a T4 or V100, it raises "Current CUDA Device does not support bfloat16", which is exactly the hardware this branch targets. Verified against torch's own autocast __init__: the check is gated on `enabled`, so disabling autocast is both the correct reading of 'no' and the minimal fix. Full finetuning already set 'no', so this was reachable before this PR too. Tests: 4384 passed against 4369 on the same tree with these changes stashed, identical 1310 environment failures either way, and nothing failing on this branch that is not also failing on its base. * Keep forced float32 on fp16 autocast, size GRPO chunks by the flag, and run the tests off a GPU Three follow-ups on the autocast change. UNSLOTH_FORCE_FLOAT32 exports ACCELERATE_MIXED_PRECISION 'no' as well, and the new gate read only that, so the injected _prepare_inputs header disabled autocast for it while the dtype expression right beside it was still choosing float16. That left Gemma3 and gpt-oss generation in full float32, and it disagreed with the two other consumers, which set _autocast_enabled True for exactly this case. The header honours the flag now. The GRPO chunk autotuner derived dtype_bytes from _autocast_dtype alone, which stays bfloat16 when autocast is off, so it sized hidden states and logits as 16-bit for a forward running in float32. That is half the real figure, on the explicit-float32 and full-finetuning runs this branch is for. The new tests drove torch.amp.autocast(device_type = "cuda") on a box that has CUDA. On a CPU runner torch warns "CUDA is not available. Disabling" and hands back a no-op, so the premise test would not have raised and every enabled case would have read False, failing for reasons unrelated to the code. Measured, then fixed by claiming the device rather than skipping, since a CPU job is where these would most need to run. They pass with the GPUs visible and with CUDA_VISIBLE_DEVICES empty. Tests: 4386 passed, same 1310 environment failures as the branch base, nothing failing here that is not also failing there. * Size the GRPO chunk on the dtype the forward runs in The autocast-off branch assumed float32, but pure bfloat16 full finetuning also disables autocast (ACCELERATE_MIXED_PRECISION='no') while keeping bf16 weights and a bf16 forward. That path got dtype_bytes=32 where main gave 16, cutting the chunk ~1.5x and doubling the forward-pass count on 10-22GB cards. Read lm_head.dtype instead: float32 for an explicit float32 load, bfloat16 here. Retargets the test that pinned the old value. * Apply ruff kwarg-spacing formatting pre-commit.ci's ruff-format-with-kwargs hook reported "files were modified by this hook" and could not push the result back, so the check stayed red. Applied locally instead. Three source-asserting tests had to be made robust rather than re-pinned, since the reformatting is legitimate and will happen again: - test_the_flag_is_recorded_beside_the_dtype counted the literal "self._autocast_enabled = (", which the formatter collapsed onto one line. Now pairs each _autocast_dtype initialiser with a nearby flag assignment by position, so it still catches a genuinely missing one. - test_the_original_error_is_still_reported anchored on a whole f-string literal, which the formatter merged with the hint that follows it. Anchors on the message text alone now; either shape satisfies the intent. - test_the_source_beats_the_version_fallback exposed a real fragility rather than a test problem: the detector matched the literal "kw_only=True", so any whitespace in transformers' own source would have read an install that needs nothing as one that needs patching. Made whitespace-tolerant. * Latch the GRPO autocast decision on the trainer * Add the license header to the new test files * Stamp the forced float32 answer on the model UNSLOTH_FORCE_FLOAT32 is process wide and from_pretrained clears it on every load, so a model loaded after a Gemma3 or gpt-oss trainer was built left '0' behind and GRPO's first generation dropped the float16 autocast that rl.py's 'no' was written expecting, running generation in full float32. The loaders now stamp the answer on the model they loaded, next to the existing float32 request marker, and the trainer reads it from there. Models without the stamp fall back to the environment as before. * Preserve a bf16 trainer when a forced-float32 model was loaded (#7867) * Stamp the forced float32 answer on every loader return path (#7867) * Stamp the float32 answers on the text-diffusion dispatch (#7867) * Apply the repo ruff-format hook to a test touched by this PR * Apply the repo ruff-format hook (ruff 0.6.9) to files this PR touches * Fix the trainer reading the process-wide forced float32 flag for PR #7867 * Fix a missing optional dependency aborting the whole pytest session for PR #7867 require_package exited the process when a system package was absent. The TTS test modules call it at import time, so under pytest the SystemExit escaped collection and ended the run with INTERNALERROR / exit code 3 instead of skipping one file. It now degrades to a module-level skip under pytest, the same treatment require_python_package already had, and keeps the exit for the standalone script callers. test_whisper.py also imported the unsloth runtime unguarded, which is a collection error on the Windows runner where triton is absent; guard it with a module-level skip. * Inherit an outer autocast by omitting dtype, not by a sentinel * Read the forced float32 stamp inside native fast generation * Stamp the full finetuning mode on the model the trainer reads * Keep one missing-dependency helper after the merge with main The merge left this branch's _skip_if_pytest alongside main's _missing_dependency, doing the same job, plus two near-duplicate test files. Fold both into main's helper and its test module, keeping this branch's extra coverage of require_package and of the call at module scope. * Make the generation autocast tests independent of the host having CUDA torch.autocast(device_type = 'cuda') disables itself when CUDA is absent, so reading _enabled off the constructed object measured the runner instead of the branch, and these could never pass on a CPU-only machine, a Mac or Windows. Record the arguments the code asks for instead. Also skip before importing rl_replacements, which needs unsloth_zoo. * Apply the repo's ruff kwarg-spacing formatting * Format these three files with the ruff the hook actually pins The ruff-format-with-kwargs hook installs ruff==0.6.9 through additional_dependencies, and 0.6.9 and 0.15 disagree on where the message goes in a multi-line assert and on how a long lambda signature wraps. I had run the script with the workspace ruff, so pre-commit.ci kept reformatting these back and failing, with nothing to show in the diff. AST is identical on all three. * Drop the nullcontext import this PR no longer uses Left over from the earlier shape of the autocast gating, which now omits dtype instead of passing a sentinel context. Source lint's import-hoist check treats an added-but-unused hoisted import as a blocker, and it is right: nothing in this file references it. * Tighten comments for the float32 dtype gating change --------- Co-authored-by: danielhanchen <unslothshared@gmail.com>	2026-08-09 05:12:02 -07:00
Daniel Han	68e5c3f04b	tests: make pytest tests collect cleanly and drop the deleted QVQ registry mirror (#8206) ... * tests: make `pytest tests` collect cleanly and drop the deleted QVQ registry mirror Collection was interrupted by 17 errors, so nothing after them ran, and tests/test_model_registry.py::test_model_registration[qwen] was red against the live Hub. Registry: - unsloth/QVQ-72B-Preview no longer exists on the Hub. Authenticated as an unsloth org member the API answers 404 and an org-wide search returns only unsloth/QVQ-72B-Preview-bnb-4bit, so it is not private and not gated (a gated repo still serves public metadata). Drop QuantType.NONE from the QVQ meta; the bnb-4bit mirror and the upstream Qwen/QVQ-72B-Preview stay. - The registry check now treats only RepositoryNotFoundError as "missing". Every other hub error (429, 5xx, DNS, timeout) skips the test instead of reporting all 129 models missing. Collection: - tests/saving files with no test items are standalone GPU scripts whose body runs at import, so bare collection downloaded checkpoints, trained and pushed to the Hub. They now call require_opt_in() and are visible skips unless UNSLOTH_RUN_SAVING_SCRIPTS=1. Running them directly is unchanged. - Four filenames contained a dot, which pytest's importer reads as a package separator; renamed to underscores. - tests/test_raw_text.py left its datasets stub in sys.modules for the rest of the session, which broke collection of tests/utils/test_packing.py. The stub is now restored after raw_text is loaded. - The two sentencepiece tests imported the legacy transformers.utils.sentencepiece_model_pb2, which fails on protobuf >= 4. Use convert_slow_tokenizer.import_protobuf(), the accessor the code under test uses. - tests/test_collection_hygiene.py guards all three regressions. tests --collect-only: 7629 collected + 17 errors in 214s (exit 2) -> 7703 collected, 0 errors in 3s (exit 0). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-08 20:03:55 -07:00
Daniel Han	36020c004a	Skip a test module with a missing optional dep instead of exiting pytest (#7971) ... * Skip a test module with a missing optional dep instead of exiting pytest require_package and require_python_package are called at module import time by the four tests/saving/text_to_speech_models files, and they called sys.exit(1) when a package was absent. Under pytest that lands during collection, where SystemExit becomes an INTERNALERROR and aborts the whole session. On a host without xcodec2, snac or soundfile, `pytest tests/` therefore ran zero tests and reported no failures, which reads as a clean run. Removing one offending module does not help: ignoring test_lasa.py just moves the abort to test_orpheus.py on snac. Route both helpers through one place that skips at module level under pytest and keeps the historical exit for the standalone-script path these files also support. Nothing else in the repo calls either helper. CI is unaffected: every workflow runs targeted paths rather than the whole tests/ tree, which is why this was only reachable when running the suite locally. * Tighten comments in the skip-instead-of-exit test helpers * Trim comments in the skip-instead-of-exit tests --------- Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-08-06 00:39:21 -07:00
Daniel Han	a6dc10dad2	Reduce and tighten comments and docstrings across the test suite (#6429) ... * Reduce and tighten comments and docstrings in tests Shorten verbose comments and docstrings across the test suite without changing any test logic. Remove narration that restates the next line, collapse long module and test docstrings to a single line, and drop banner separators. Keep regression context (issue and PR references, run ids), skip reasons, mocking and timing rationale, license headers, lint and type directives, and commented-out code. Comments and docstrings only: an AST signature check confirms no code, assertions, or string literals changed, and the suite byte-compiles cleanly. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-18 01:07:09 -07:00
Daniel Han	187144d4e7	Reduce and tighten code comments and docstrings repo-wide (#6095) ... Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.	2026-06-08 23:09:51 -07:00
Daniel Han	3ce187da02	Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) ... Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.	2026-06-08 04:24:13 -07:00
Daniel Han	ba2897a318	Revert "[FIX] Vllm guided decoding params (#3662)" ... This reverts commit fb4f0fdf56.	2025-12-01 05:43:45 -08:00
Datta Nimmaturi	fb4f0fdf56	[FIX] Vllm guided decoding params (#3662) ... * vllm sampling params fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * do not patch base_trainer * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * seperate vllm fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Apply suggestion from @danielhanchen * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks" This reverts commit 58b483dc0d1790f99580665801d3fa0d7267c533. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks" This reverts commit b2497519659a9f301e7a633795d9efdafdc2b277. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks" This reverts commit de3daaf429f81aceb6632932b0cb1af5149652a8. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2025-12-01 05:42:37 -08:00
Daniel Han	d6bb89ad44	Formatting & bug fixes (#3563) ... * Update rl.py * Fix CE Loss * Versioning * Update loader.py * Update loader.py * extract_model_type_from_config * Model types * Update loader.py * get_transformers_model_type * Update loader.py * Update loader.py * Update loader.py * Update rl.py * Update pyproject.toml * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Versioning * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update vision.py * Update vision.py * Fix DataParallel * Update _utils.py * Update rl.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update mapper.py * Versioning * Update loader.py * Update loader.py * Update rl.py * Versioning * Update _utils.py * Fix auto_mapping * Update loader.py * Update loader.py * Update vision.py * Update vision.py * Update loader.py * Message * Update vision.py * Update loader.py * Update vision.py * cache_implementation * Update vision.py * Update loader.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * Update vision.py * Save max_seq_length * Update _utils.py * Update rl.py * Update vision.py * Update llama.py * Mistral3 vllm (#3349) * [WIP] use vLLM for vision language models * Update README.md Editing icon sizes * Update README.md Updating icon sizes * Update README.md (#2885) * MoE kernels AGPLv3 * versioning * Many bug fixes (#2908) * add deepseek v3 * add deepseek r1 base * add deepseek r1 zero * add deepseek distill llama * add deepseek distill models * remove redundant code when constructing model names * add mistral small to registry * rename model registration methods * rename deepseek registration methods * refactor naming for mistral and phi * add global register models * refactor model registration tests for new registry apis * add model search method * remove deprecated registration api * add quant type test * add registry readme * make llama registration more specific * clear registry when executing individual model registration file * more registry readme updates * Update _auto_install.py * Llama4 * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Synthetic data * Update mapper.py * Xet and Synthetic * Update synthetic.py * Update loader.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update pyproject.toml * Delete .gitignore * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update _utils.py * Update pyproject.toml * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update chat_templates.py * Seasame force float16 / float32 * Fix Seasame * Update loader.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * is_multimodal * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update vision.py * Update vision.py * Update vision.py * UNSLOTH_DISABLE_STATIC_GENERATION * Update vision.py * Auto vision detection * Sesame * Whisper * Update loader.py * Update loader.py * Update loader.py * Update mapper.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update _utils.py * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * logging * Update pyproject.toml * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * logits / temperature * Update rl_replacements.py * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Debugging only * Update llama.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Generic efficient GRPO * Update rl_replacements.py * Update rl_replacements.py * Remove debugging * Update rl_replacements.py * Update rl_replacements.py * Update vision.py * Update llama.py * Update rl_replacements.py * versioning * Update _utils.py * Update vision.py * Update mapper.py * Update loader.py * Update mapper.py * Update vision.py * Update loader.py * Update vision.py * Update loader.py * Update _utils.py * Update vision.py * gradient checkpointing * Gemma 3N fixes * Update loader.py * Versioning * Gemma 3N fixes * Update vision.py * Update vision.py * Update loader.py * Update vision.py * Fix setup.py * setup.py * Prints * Update setup.py * Update setup.py * Update setup.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update vision.py * Update vision.py * Update pyproject.toml * Update vision.py * Update _utils.py * Update __init__.py * Update __init__.py --------- Co-authored-by: jeromeku <jerome.ku@gmail.com> Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com> * silienty skip falcon h1 import is transformers_version < 4.53.0 (#2912) * Dynamically adjust get_per_token_logps function and patch as well (#2911) * add intel gpu with vllm support (#2903) * [bugs] fix for casual mask (#2868) * fix for casual mask * use un_casual in sdpa * add missing mask * fix for type * Explicitly check if xformers exists for attention (#2889) * Update __init__.py * Update llama.py * if mlp doesn't exist in layer module check for feed_forward name for falcon h1 (#2913) * Move inputs to right devices. (#2919) * Move tensors to right devices * fix multi gpu for non mistral models * multi GPU RoPE for gemma2 * Finish up multi GPU inference * Make multiGPU rope a list * Remove unnecessary transfer to CPU * Remove unnecessary move to CPU * Donot move inputs to device yet will be handled separately in another PR * Move inputs to appropriate decoder device * Make device count global variable * Cleanup RoPE device code * Fixup num_gpu to device count * Cleanup device counts * Use device index for RoPE get_cache * Donot typecast * Use tuple instead of list for tensors. Use device index directly * fixup move to device logic * WIP VLM vLLM * Make vLLM patch a function * Add save and load lora functions * Make fast_inference setup depend on the flag * Improve fast inference patching mechanism * Make vision setting depend on checks in fastbasemodel * Check LoRA and vLLM intercompatibility for vision models * Comment pointing to vLLM LoRA check * Improve lora validation on vLLM * Error out on no vLLM and increase max lora rank * Bug fixes (#3017) * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update pyproject.toml * Delete .gitignore * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update _utils.py * Update pyproject.toml * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update chat_templates.py * Seasame force float16 / float32 * Fix Seasame * Update loader.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * is_multimodal * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update vision.py * Update vision.py * Update vision.py * UNSLOTH_DISABLE_STATIC_GENERATION * Update vision.py * Auto vision detection * Sesame * Whisper * Update loader.py * Update loader.py * Update loader.py * Update mapper.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update _utils.py * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * logging * Update pyproject.toml * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * logits / temperature * Update rl_replacements.py * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Debugging only * Update llama.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Generic efficient GRPO * Update rl_replacements.py * Update rl_replacements.py * Remove debugging * Update rl_replacements.py * Update rl_replacements.py * Update vision.py * Update llama.py * Update rl_replacements.py * versioning * Update _utils.py * Update vision.py * Update mapper.py * Update loader.py * Update mapper.py * Update vision.py * Update loader.py * Update vision.py * Update loader.py * Update _utils.py * Update vision.py * gradient checkpointing * Gemma 3N fixes * Update loader.py * Versioning * Gemma 3N fixes * Update vision.py * Update vision.py * Update loader.py * Update vision.py * Fix setup.py * setup.py * Prints * Update setup.py * Update setup.py * Update setup.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update vision.py * Update vision.py * Update pyproject.toml * Update vision.py * Update _utils.py * Update __init__.py * Update __init__.py * Small fixes * Update vision.py * Update vision.py * versioning * Update __init__.py * Update llama.py * Update rl.py * Update rl.py * Update _utils.py * Update vision.py * Update vision.py * compiler stance * Update _utils.py * Update pyproject.toml * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990) This reverts commit 4021da634a. * skip_guard_eval_unsafe fix * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update llama.py * Update llama.py * Fix `quantization_method` * versioning * fix for casual mask (#3011) * [intel] add for intel path for llama.py (#3012) * fix for intel path * remove unuse code * Update unsloth/models/llama.py --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Update llama.py * Fix Gemma 2 (#3024) * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update pyproject.toml * Delete .gitignore * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update _utils.py * Update pyproject.toml * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update chat_templates.py * Seasame force float16 / float32 * Fix Seasame * Update loader.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * is_multimodal * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update vision.py * Update vision.py * Update vision.py * UNSLOTH_DISABLE_STATIC_GENERATION * Update vision.py * Auto vision detection * Sesame * Whisper * Update loader.py * Update loader.py * Update loader.py * Update mapper.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update _utils.py * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * logging * Update pyproject.toml * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * logits / temperature * Update rl_replacements.py * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Debugging only * Update llama.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Generic efficient GRPO * Update rl_replacements.py * Update rl_replacements.py * Remove debugging * Update rl_replacements.py * Update rl_replacements.py * Update vision.py * Update llama.py * Update rl_replacements.py * versioning * Update _utils.py * Update vision.py * Update mapper.py * Update loader.py * Update mapper.py * Update vision.py * Update loader.py * Update vision.py * Update loader.py * Update _utils.py * Update vision.py * gradient checkpointing * Gemma 3N fixes * Update loader.py * Versioning * Gemma 3N fixes * Update vision.py * Update vision.py * Update loader.py * Update vision.py * Fix setup.py * setup.py * Prints * Update setup.py * Update setup.py * Update setup.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update vision.py * Update vision.py * Update pyproject.toml * Update vision.py * Update _utils.py * Update __init__.py * Update __init__.py * Small fixes * Update vision.py * Update vision.py * versioning * Update __init__.py * Update llama.py * Update rl.py * Update rl.py * Update _utils.py * Update vision.py * Update vision.py * compiler stance * Update _utils.py * Update pyproject.toml * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990) This reverts commit 4021da634a. * skip_guard_eval_unsafe fix * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update llama.py * Update llama.py * Fix `quantization_method` * versioning * Update _utils.py * Update _utils.py * Update _utils.py * falcon force float32 on sm<75 machines (#3026) * Fix torch compile issues (#3028) * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update pyproject.toml * Delete .gitignore * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update _utils.py * Update pyproject.toml * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update chat_templates.py * Seasame force float16 / float32 * Fix Seasame * Update loader.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * is_multimodal * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update vision.py * Update vision.py * Update vision.py * UNSLOTH_DISABLE_STATIC_GENERATION * Update vision.py * Auto vision detection * Sesame * Whisper * Update loader.py * Update loader.py * Update loader.py * Update mapper.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update loader.py * Update loader.py * Update loader.py * Update loader.py * Update _utils.py * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update rl.py * logging * Update pyproject.toml * Update rl.py * versioning * Update rl.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * logits / temperature * Update rl_replacements.py * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Debugging only * Update llama.py * Update llama.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Generic efficient GRPO * Update rl_replacements.py * Update rl_replacements.py * Remove debugging * Update rl_replacements.py * Update rl_replacements.py * Update vision.py * Update llama.py * Update rl_replacements.py * versioning * Update _utils.py * Update vision.py * Update mapper.py * Update loader.py * Update mapper.py * Update vision.py * Update loader.py * Update vision.py * Update loader.py * Update _utils.py * Update vision.py * gradient checkpointing * Gemma 3N fixes * Update loader.py * Versioning * Gemma 3N fixes * Update vision.py * Update vision.py * Update loader.py * Update vision.py * Fix setup.py * setup.py * Prints * Update setup.py * Update setup.py * Update setup.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update vision.py * Update vision.py * Update pyproject.toml * Update vision.py * Update _utils.py * Update __init__.py * Update __init__.py * Small fixes * Update vision.py * Update vision.py * versioning * Update __init__.py * Update llama.py * Update rl.py * Update rl.py * Update _utils.py * Update vision.py * Update vision.py * compiler stance * Update _utils.py * Update pyproject.toml * Update pyproject.toml * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Revert "Revert "Add Qwen2.5-VL-32B-Instruct mapping to fix quantized model me…" (#2990) This reverts commit 4021da634a. * skip_guard_eval_unsafe fix * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update synthetic.py * Update llama.py * Update llama.py * Fix `quantization_method` * versioning * Update _utils.py * Update _utils.py * Update _utils.py * check stride * Cleanup * Update rope_embedding.py * Update gemma2.py * Fix `set_stance` * Update pyproject.toml * Update _utils.py * Fixup patch vllm * Disable mllama * Use variables to decide VLM support * Better attn_impl handling * Patch TF protobuf incompatability * Torch 2.8 (#3186) * Fix mamba * Update loader.py * Update vision.py * Update loader.py * Filter vLLM standby logs (#3131) * filter vLLM standby logs * safeguard standby logger patch * Update unsloth/models/_utils.py * Update unsloth/models/_utils.py * Update unsloth/models/_utils.py --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Update loader.py * Add scaler * Update llama.py * Update _utils.py * Versioning * GPT OSS fix * GPT OSS fix * Update loader.py * Update vision.py * Update vision.py * Update loader.py * Update vision.py * Update vision.py * Update llama.py * Update llama.py * Update llama.py * Versioning * Update mapper.py * Update vision.py * Update vision.py * Update vision.py * Upcast norms * Update loader.py * Update vision.py * Upcast layernorms * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update save.py * Update rl.py * Update pyproject.toml * Update rl.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl.py * Update _utils.py * Update __init__.py * Torch 2.8 * Update rl_replacements.py --------- Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com> * Update _auto_install.py * Update pyproject.toml * Update rl.py * Protobuf issue * Update pyproject.toml * Fix extras transformers typo in pyproject.toml * Update _utils.py * Bug fixes (#3195) * Fix mamba * Update loader.py * Update vision.py * Update loader.py * Filter vLLM standby logs (#3131) * filter vLLM standby logs * safeguard standby logger patch * Update unsloth/models/_utils.py * Update unsloth/models/_utils.py * Update unsloth/models/_utils.py --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Update loader.py * Add scaler * Update llama.py * Update _utils.py * Versioning * GPT OSS fix * GPT OSS fix * Update loader.py * Update vision.py * Update vision.py * Update loader.py * Update vision.py * Update vision.py * Update llama.py * Update llama.py * Update llama.py * Versioning * Update mapper.py * Update vision.py * Update vision.py * Update vision.py * Upcast norms * Update loader.py * Update vision.py * Upcast layernorms * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update save.py * Update rl.py * Update pyproject.toml * Update rl.py * Update rl_replacements.py * Update rl.py * Update rl.py * Update rl.py * Update _utils.py * Update __init__.py * Torch 2.8 * Update rl_replacements.py * Update loader.py * UNSLOTH_ENABLE_CCE * Fix * Update loader.py * Update loader.py * Update __init__.py * Update __init__.py * Update __init__.py * Update __init__.py * Import fixes * Update loader.py * Fix aimv2 issue * Update loader.py * Update import_fixes.py * Update import_fixes.py * Update loader.py * Update loader.py * Update loader.py * Upgrade * Update loader.py * Update loader.py * Update loader.py * Update loader.py --------- Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com> * adallow float32 dtype in FastLanguageModel (#3204) * Update loader.py * Update vision.py * Suppress message and use unsloth sampling params * Use trl sampling params for now * Improve error message * fixup quantized fast inference model name * Add mistral 3 support --------- Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: jeromeku <jerome.ku@gmail.com> Co-authored-by: DoubleMathew <mmathew23@gmail.com> Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com> Co-authored-by: parth2510 <parthguptapg7326@gmail.com> * Set padding to 0 * Fix patch * fixup patch (#3359) Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com> * Update vision.py * Versioning * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * MXFP4 dequant * Update loader.py * Update vision.py * load_in_16bit * Update vision.py * Update vision.py * Update vision.py * Update rl.py * Update vision.py * offload_embedding * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update rl_replacements.py * Update loader.py * Fix padding issue * Update pyproject.toml * Update _utils.py * Update pyproject.toml * Update _utils.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * New models * Update llama.py * Versioning * Update _utils.py * Update llama.py * Update _utils.py * Update llama.py * Fix AMD * Update _utils.py * Update llama.py * Update vision.py * DEVICE_TYPE_TORCH * Update __init__.py * Update __init__.py * Update _utils.py * Move DEVICE_TYPE * Update rl_replacements.py * Update loader.py * AMD install script * Move AMD * Update _amd_install.sh * Update pyproject.toml * Update pyproject.toml * Delete _amd_install.sh * Update device_type.py * Update loader.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update tokenizer_utils.py * Versioning * Update pyproject.toml * Update loader.py * Update _utils.py * Update pyproject.toml * Update pyproject.toml * Update _utils.py * Update pyproject.toml * Update _utils.py * Update _utils.py * Update loader.py * Update _utils.py * Update _utils.py * local_files_only * Cut Cross Entropy * Update llama.py * Update vision.py * Update vision.py * Update vision.py * Qwen 3 VL vLLM (#3489) * Update __init__.py * patch_torchao * torchao_logger * Update rl_replacements.py * Fix * Update rl.py * Update rl.py * Update rl.py * Update rl.py * Update _utils.py * Versioning * fbgemm fp8 block quant support (>=1.4.0) (#3531) * fbgemm fp8 block quant support (>=1.4.0) * Verify for fp8 support before proceeding * Use unsloth zoo's Version and improve comments * spacessss * Update vision.py * Update vision.py * Update rl.py * vllm_sampling_params * Update rl.py * Update rl.py * Update rl.py * Add `ruff` pre-commit hook and apply it (#3424) * Add Ruff pre-commit config and workflow * Add kwarg spacing enforcement helper * Apply Ruff formatting * Update fp8.py * Revert ruff on some files * Update * force-exclude = true * Datasets issue * Ruff * Remove mapper * Update mapper.py * Update pyproject.toml --------- Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com> Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com> Co-authored-by: jeromeku <jerome.ku@gmail.com> Co-authored-by: DoubleMathew <mmathew23@gmail.com> Co-authored-by: Lei Zhenyuan <zhenyuan.lei@intel.com> Co-authored-by: parth2510 <parthguptapg7326@gmail.com> Co-authored-by: Dan Saunders <danjsaund@gmail.com>	2025-11-07 06:00:22 -08:00
Roland Tannous	efe2cc43a7	tests for additional merge fix unsloth zoo pr 163 (#2719) ... * tests for additional merge fix unsloth zoo pr 163 * fixed load_dataset indent in mistral perplexity test file	2025-06-11 14:08:41 -07:00