mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-24 00:04:14 +00:00
e0a2bd8317
* Fix Windows Studio launcher updates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the remaining damaged-file reports and launcher recovery for PR #8092 The report this PR targets listed three damaged files. Preserving the launcher fixes one; the other two still fail the update on their own, since _fail_if_install_damaged exits 1 on any finding. Both are produced by our own installer, so the update they fail is the update meant to repair them. einx and torchao both ship a top-level test/conftest.py, and install_python_stack.py force-reinstalls torchao every update, so pip deletes the file and the pinned torchao does not ship it. package-lock.json is rewritten in place by setup.ps1 and setup.sh, which run npm install inside the installed tree; under legacy-peer-deps npm dedupes hoisted entries and the file shrinks below its recorded size, reproduced exactly as 28473 to 27225. Drop both classes while reading RECORD rather than when reporting, so a filtered row also stays out of the ownership tally and the limit budget and cannot crowd out a real finding. Mirrored into the sidecar scanner, whose docstring asks for the two predicates to be kept in sync. Also three fixes to the transaction itself: - Recover from the hardlinked bin/unsloth.exe shim, which survives the old updater's .deleteme unlink. - Warn instead of exiting when the launcher is missing or invalid. An install already broken by the old updater has neither launcher nor .deleteme, and exiting before setup stopped exactly those users from updating. validate_launcher still judges the result. - Gate recovery on validity rather than existence, and treat a failed backup as a missing safety net rather than a fatal error. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve the launcher from the managed venv and keep a good backup Three follow-ups from review. The transaction resolved Scripts from sys.executable, but setup.ps1 installs into STUDIO_HOME/unsloth_studio (setup.ps1:3411). When a pip-installed or checkout CLI drives the update, those differ, so it backed up and --version validated the caller's launcher while the one actually being replaced went unprotected. Resolve the managed venv the same way _studio_deps._managed_root does for the damage scan. __enter__ overwrote the transaction backup unconditionally, guarded only by the two-byte MZ check. A backup outlives __enter__ only when a previous run died before validating, so it holds the last launcher known to run; overwriting it with a PE-shaped but unvalidated canonical file destroyed the only recovery copy. Write a backup only when there is no usable one already. package-lock.json was skipped outright, which dropped its existence check too. npm rewrites it in place but never deletes it, so keep the row and drop only its recorded size. Also add scripts/ to the shared non-runtime roots: unsloth_zoo ships a top-level scripts/, the same squatted-namespace shape as einx's test/, and it has no __init__.py so nothing imports it. * Move the launcher aside for setup, and restore it if nothing replaces it I rejected this on the strength of setup.ps1:4386-4391, which says renaming the running launcher "only ever failed (WinError 32)". That is not right. A probe on windows-latest builds a real console-script package, runs it, and renames the live launcher: the rename succeeds and a replacement can then be written at the freed path. RESULT idle-launcher: RENAME SUCCEEDED RESULT running-launcher: RENAME SUCCEEDED RESULT publish-replacement: WROTE a new launcher at the canonical path main moved the launcher aside before setup (studio.py:3182) and this branch had removed it, so uv could no longer replace Scripts\unsloth.exe. uv only self-replaces its own executable and deletes a third-party console script outright, and the pip fallback then no-ops on the already-satisfied bare unsloth, so the upgrade was silently skipped. Move it aside again, but keep what this branch was written for: when setup publishes no launcher, validate_launcher restores it rather than leaving the venv with none. That was the original bug, where the old updater renamed the launcher away and then deleted its own .deleteme. Restore prefers the backup over the moved-aside copy: the backup is the last launcher known to run, the moved-aside one is only this run's unvalidated canonical file. The mocked harness cannot reproduce a sharing violation, so the tests pin the invariant (the canonical path is free during setup, a recoverable copy always exists) while the CI probe covers the Windows semantics. * Tell a missing launcher from a broken one, and retry restores Two follow-ups, both from the restore path added in 724b274d4. Restoring before the health check could not tell setup publishing nothing from setup publishing something unusable. A zero-byte or non-PE replacement was quietly swapped for the previous launcher, which then passed --version, so the update reported success and deleted its own recovery copies. Sample whether setup published anything before any restore: nothing published and a good restore is the no-op update this transaction exists for, while a launcher setup did write and that cannot run stays a failure even though the previous one goes back. _restore_backup also picked the first candidate passing the two-byte header check and stopped there. Backups are taken after only that check, so an interrupted run can leave a PE-shaped but non-runnable one, and preferring it stranded the working launcher this run had moved aside. Split restoration: _restore_from puts one candidate back, and _restore_runnable walks the candidates until one actually runs. * Restore a runnable launcher on exceptional exit, narrow the exemption __exit__ restored the first PE-shaped candidate, so an interrupted run's non-runnable backup was installed over the working launcher this run had moved aside, and it could undo a restore validate_launcher had just made. It now uses _restore_runnable, which leaves an already-working launcher alone, walks the candidates until one passes --version, and falls back to the best candidate rather than whichever was tried last. The shared-namespace exemption was also too broad. I justified it on the grounds that tests/ and scripts/ ship no __init__.py, which is wrong: PEP 420 makes them importable, and this repo does 'from scripts import ...' itself. Restrict it to distributions Unsloth does not ship, so einx and torchao squatting on a top-level test/ is exempt while our own top-level trees stay checked. * Keep all four recovery copies as runtime candidates _recovery_candidates only offered the backup and the moved-aside copy, so when an interrupted run left a PE-shaped but non-runnable backup and the legacy .deleteme or the PATH shim was still good, the bad backup was accepted on its header alone and the good copy was never reached. The update then failed every time with the broken bytes canonical. All four are candidates now, deduplicated by normalised path, and _restore_runnable walks them until one passes --version. * Move the update lock out of the replaceable venv Resolving Scripts from the managed venv put the lock inside $VenvDir, and setup.ps1:3748 removes that whole directory to rebuild a stale torch. Windows refuses a recursive delete while a handle inside it is open, so an external CLI holding the lock for the whole setup run failed the repair with "Could not remove stale venv". Keep it under the Studio home instead, which is stable and is the right grain anyway: it is what names the managed venv. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| conftest.py | ||
| test_bitsandbytes_kernel_readiness.py | ||
| test_change_system_message.py | ||
| test_conftest_bitsandbytes_preimport.py | ||
| test_construct_chat_template_validation.py | ||
| test_cpo_processor_text_tokenizer.py | ||
| test_cross_platform_parity.py | ||
| test_dpo_vision_processor_passthrough.py | ||
| test_e2e_no_torch_sandbox.py | ||
| test_fast_language_model_text_only.py | ||
| test_fast_model_config_passthrough.py | ||
| test_fast_sentence_transformer_embedding_parity.py | ||
| test_fast_sentence_transformer_redirect_lifecycle.py | ||
| test_flash_attn_install_python_stack.py | ||
| test_get_chat_template_escaping.py | ||
| test_get_lora_parameters_bias_fp8_block_size.py | ||
| test_get_lora_parameters_fp8_block_size.py | ||
| test_gpu_init_ldconfig_guard.py | ||
| test_grpo_ddp_model_config.py | ||
| test_import_without_bitsandbytes.py | ||
| test_install_python_stack.py | ||
| test_install_uv_override_space.py | ||
| test_mlx_public_trainer_api.py | ||
| test_no_torch_filtering.py | ||
| test_orpo_processor_text_tokenizer.py | ||
| test_pad_token_fix.py | ||
| test_patch_trl_rl_trainers_defensive.py | ||
| test_remove_special_tokens_no_bos.py | ||
| test_revision_forwarding.py | ||
| test_studio_import_no_torch.py | ||
| test_studio_runtime_gate.py | ||
| test_to_sharegpt_optional_none.py | ||
| test_tokenizers_and_torch_constraint.py | ||
| test_torchcodec_torch_compat.py | ||
| test_unsloth_run_tool_policy_resolver.py | ||
| test_v100_fullft_precision.py | ||
| test_virustotal_scan.py | ||
| test_vision_lora_targeting.py | ||
| test_windows_arm64_python_choice.py | ||
| test_windows_git_gate.py | ||
| test_windows_installer_concurrency_guard.py | ||
| test_windows_no_torch_setup.py | ||
| test_windows_python_313_8_screen.py | ||
| test_windows_python_venv_hardening.py | ||
| test_windows_setup_output_encoding.py | ||
| test_windows_studio_update_launcher.py | ||
| test_windows_vcredist_download_tls.py | ||