mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-10 00:08:58 +00:00
19 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Leo Borcherding
|
c86165e735
|
Studio Colab: opt-in shareable Cloudflare tunnel link (#6684)
* Studio Colab: add opt-in shareable Cloudflare tunnel link colab.start(cloudflare=True) opts in to a free Cloudflare quick tunnel and shows a trycloudflare.com link above the proxy iframe, reachable from any device. Default OFF: bare start() keeps the in-tab Colab-proxy behavior. run_server suppresses the tunnel on Colab by design, so colab.py starts it directly via cloudflare_tunnel.start_studio_tunnel(); failures degrade to the Colab proxy only. * Studio Colab notebook: surface opt-in cloudflare=True in start cell * Studio Colab: reskin shareable Cloudflare link to match the proxy banner Retrofit _shareable_link_html to reuse the original Colab proxy banner skin from show_link (white card, black border, Unsloth gem, black Open button) instead of the plain dark box, so the shareable Cloudflare link gets the same prominent 'Ready!' treatment. * Studio Colab: address review feedback on Cloudflare tunnel - try/finally around tunnel start + embed + keepalive so a KeyboardInterrupt while the tunnel is starting or the iframe is rendering tears it down instead of orphaning the cloudflared process (Gemini review). - Publish the directly-started tunnel URL onto app.state.cloudflare_url via a new _publish_cloudflare_url helper so /api/health advertises it; otherwise the frontend's API examples fall back to the unreachable raw server_url (Codex P2). _stop_cloudflare_tunnel now also clears it so health stops showing a dead tunnel. - Notebook: make cloudflare=True a replacement for start(), not an addition, since start() blocks and the second call would never run if both are left in (Codex P2). * Studio Colab: gate Cloudflare tunnel on auth + honor opt-out in run_server - Refuse to open the Cloudflare tunnel while the admin still holds its seeded bootstrap password. While requires_password_change is true the server injects that password into same-origin index GETs, and a public tunnel request counts as same-origin, so sharing the link would leak admin access. New _bootstrap_password_pending() gate (fails safe) blocks the tunnel and tells the user to change the password first, then re-run start(cloudflare=True) (P1). - Pass cloudflare=False into run_server so the opt-out holds even when Colab detection fails; this helper is now the sole owner of the tunnel decision, preventing run_server from opening a tunnel on the 0.0.0.0 bind by default (P2). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio Colab: drop duplicate tunnel link log and simplify start cell guidance * Studio Colab: validate /api/health identity before reusing or tunneling a port * Studio Colab: condense verbose docstrings and comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
Daniel Han
|
187144d4e7
|
Reduce and tighten code comments and docstrings repo-wide (#6095)
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison. |
||
|
Daniel Han
|
8292e699e4
|
Studio: make code comments and docstrings more succinct (#6029)
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison. |
||
|
Daniel Han
|
3ce187da02
|
Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079)
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent. |
||
|
Leo Borcherding
|
ee6695118c
|
Fix/studio colab proxy and iframe - Unsloth Studio not loading in Colab (iframe "refused to connect" and wrong URL) (#5844)
* fix(studio/colab): merge iframe+keepalive into start(), add proxy_headers to uvicorn - Move serve_kernel_port_as_iframe and keepalive loop into colab.start() so both run in the same cell execution context, eliminating the race where the proxy URL was shown before the iframe cell had a chance to run - Add a 2s sleep after run_server() before show_link() to give Colab's proxy infrastructure time to register the bound port - Add proxy_headers=True and forwarded_allow_ips="*" to uvicorn Config so X-Forwarded-Proto/Host from Colab's reverse proxy are trusted - Simplify notebook start cell (no more separate iframe cell needed) * fix(studio/colab): fix iframe blocking and server thread crash in Colab Two root causes for the long-standing proxy/iframe breakage: 1. SecurityHeadersMiddleware set X-Frame-Options: DENY and frame-ancestors 'none' unconditionally, blocking serve_kernel_port_as_iframe regardless of server health. Fix: detect Colab via COLAB_BACKEND_URL/COLAB_GPU env vars, relax frame-ancestors to *.prod.colab.dev and omit X-Frame-Options. 2. asyncio.run() in the daemon thread conflicted with nest_asyncio's global patches applied on the main thread, causing the server to crash silently after ready_event fired. Fix: use explicit new_event_loop() + run_until_complete() in the daemon thread to bypass nest_asyncio's asyncio.run patch. Also replace blind time.sleep(2) with a health endpoint poll so the link and iframe are only shown once the server is truly reachable. * fix(studio/colab): use reliable /content + google.colab path for Colab detection COLAB_BACKEND_URL and COLAB_GPU env vars aren't consistently set across all Colab runtime versions. Use /content dir + google.colab package path as a more reliable signal, computed once at module load. * fix(studio/colab): fix port mismatch, health-check silence, and CSP framing Four bugs causing the iframe and URL button to always fail: 1. Port not propagated back: run_server auto-increments when 8888 is taken, but start() kept using the original port for show_link() and serve_kernel_port_as_iframe() — now reads app.state.server_port. 2. Silent health-check failure: the poll loop never checked whether any attempt succeeded; on all-fail it continued and showed a dead link — now exits early with a clear error message. 3. CSP frame-ancestors too narrow: '*.prod.colab.dev' only matches one subdomain level; actual Colab proxy URLs are two levels deep (e.g. foo.region.prod.colab.dev), and the parent frame may also be colab.research.google.com or a sandboxed null-origin output iframe — changed to '*' in Colab mode (single-user sandbox, no security loss). 4. _IS_COLAB detection hardcoded python3.10/3.11 paths: Python 3.12+ Colab runtimes wouldn't match when env vars aren't set — replaced with a glob over python3.*/dist-packages/google/colab. * fix(studio/colab): harden Colab startup against every known failure mode colab.py: - get_colab_url: retry eval_js up to 3x (10s timeout each), validate that result is a real https:// URL containing the port before accepting it; log a clear warning when falling back to localhost - show_link: safe short_url truncation (try/except around str.index so an unexpected URL shape never blocks the link card from rendering); also emit the URL via logger so it's visible in cell text output even if HTML display is suppressed - start: detect "already running" at entry — on cell re-run Studio is still healthy on port 8888; skip re-launch and go straight to show+iframe so the user never ends up with mismatched port state - start: wrap run_server in try/except (SystemExit + Exception) so startup errors surface as readable messages rather than cell crashes - start: check frontend_path/index.html exists, not just the directory - start: remove unused `import sys` - start / keepalive: catch KeyboardInterrupt so interrupting the cell prints a clean "stopped" message instead of a raw traceback - extract _is_studio_healthy() and _show_and_embed() helpers to deduplicate the fast-path and normal-path logic main.py: - _build_csp: in Colab mode, extend script-src to include *.prod.colab.dev and *.googleusercontent.com (Colab injects scripts from these origins into the output iframe scaffolding) - _build_csp: in Colab mode, extend connect-src with blob:, data:, wss://*.prod.colab.dev, and wss://*.googleusercontent.com so WebSocket streams and Colab kernel traffic are not blocked by CSP * fix(studio/colab): fix iframe width responsiveness and height sizing Replace serve_kernel_port_as_iframe with a raw CSS iframe for two reasons: 1. Width responsiveness: serve_kernel_port_as_iframe sets the width as an HTML attribute (width="100%") which Colab's output machinery can bake into a fixed pixel value on first render, causing the Studio to stop following the notebook panel width when it opens/closes or the window resizes. A CSS style property (style="width:100%") participates in normal reflow and always tracks the parent container width. 2. Height sizing: the hardcoded height=1200 was too tall on short monitors (forced outer-page scroll) and wasted space on tall ones. A small JS snippet reads screen.availHeight and sets height to ~82% of the screen, clamped to [600, 1100]px, with a resize listener that re-fits on zoom changes and panel open/close events. Also eliminate the double eval_js call: _show_and_embed now fetches the Colab proxy URL once and passes it to show_link via the new _url kwarg, so google.colab.kernel.proxyPort is only called once per invocation. Falls back to serve_kernel_port_as_iframe if IPython.display.HTML is unavailable for any reason. * fix(studio/colab): fix link button + add fullscreen hover button to iframe Link button: target="_blank" is blocked by Colab's output sandbox. Switch to onclick="window.open(url,'_blank')" which the sandbox allows. Fullscreen: add a small button that appears on hover in the top-right corner of the iframe. Clicking it calls requestFullscreen() on the wrapper div and stretches the iframe to 100vh/100vw. Exits back to normal on fullscreen change. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert(studio/colab): remove fullscreen button * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/colab): address review feedback - Wrap both urlopen calls in with statements to prevent socket/fd leaks - Replace JS resize listener with CSS height:82vh — simpler, responsive, and no risk of leaked window listeners on cell re-runs - Use importlib.util.find_spec("google.colab") instead of a glob path to detect Colab; more robust across Python versions and venv layouts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/colab): fall back to href navigation when window.open is blocked window.open from a cross-origin sandboxed Colab output iframe can be silently blocked by the browser (returns null, no exception). The old code returned false unconditionally, so a blocked popup left the button doing nothing. Now: if window.open succeeds the new tab opens and the href is suppressed; if it returns null the browser follows the href, navigating the output cell to Studio — always does something useful. * fix(studio/colab): remove button, give iframe a branded header bar The "Open Unsloth Studio" button was unreliable in Colab's sandboxed output context regardless of how window.open was called. Since the iframe already loads Studio inline, the button added no value and confused users with a URL that 404s outside the output cell. Replace the separate link card + bare iframe with a single block: a slim black header bar (Unsloth logo + truncated URL) flush on top of the full-height responsive iframe. Cleaner and removes the broken button entirely. * studio: gate uvicorn proxy_headers/forwarded_allow_ips behind _IS_COLAB forwarded_allow_ips="*" was applied unconditionally, so every Studio deployment trusted X-Forwarded-* headers from any client. Only Colab needs that, because its reverse proxy fronts the kernel. For a normal local/standalone Studio this is an unwanted relaxation, especially when bound to 0.0.0.0. Now proxy_headers/forwarded_allow_ips are only set when _IS_COLAB. Standalone runs fall back to uvicorn's defaults (proxy_headers honored from loopback only), restoring the prior security posture, while Colab keeps the wide trust its proxy requires. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Leo Borcherding
|
68965988cf
|
Fix/studio colab button message: Add fallback message for Colab Studio button when proxy URL fails (#4866)
* Add fallback message for Colab Studio button when localhost link doesn't work * Make fallback message darker grey for better readability * Make fallback message bold for better visibility --------- Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com> |
||
Roland Tannous
|
e79a178200
|
Allow install_python_stack to run on Colab (#4633)
* Allow install_python_stack to run on Colab The _COLAB_NO_VENV flag was setting _SKIP_PYTHON_DEPS=true, which skipped both the PyPI version check (needs $VENV_DIR/bin/python) and install_python_stack (uses sys.executable, works without a venv). Introduce a separate _SKIP_VERSION_CHECK flag for the version check, so install_python_stack still runs on Colab. The _SKIP_PYTHON_DEPS flag remains available for the "versions match" fast path. * Remove colab.py workarounds that broke transformers/hf-hub compatibility PR #4601 added _pip_install_backend_deps(), _bootstrap_studio_venv(), and _is_colab() to colab.py as workarounds for install_python_stack being skipped on Colab. These workarounds: - Stripped version constraints from studio.txt and installed into system Python - Upgraded huggingface-hub to >=1.0, breaking Colab's pre-installed transformers which requires huggingface-hub<1.0 With install_python_stack now running on Colab (previous commit), these workarounds are unnecessary — all deps are properly installed by setup.sh. Restore colab.py to its original PR #4237 structure: just get_colab_url(), show_link(), and start(). * Remove --local flag from setup.sh in Colab notebook The --local flag is not needed for the standard Colab flow since install_python_stack now runs on Colab and installs deps from PyPI. |
||
|
Daniel Han
|
baabfa0a6e
|
Fix Colab huggingface-hub conflict, ensurepip fallback, bump to 2026.3.14 (#4603)
* Fix Colab huggingface-hub conflict, ensurepip fallback, bump to 2026.3.14 - colab.py / setup.sh: relax == pins to >= when installing studio.txt on Colab so huggingface-hub does not clobber Colab's bundled version (breaks transformers is_offline_mode import) - install_python_stack.py: when uv is unavailable and pip is missing (uv-created venvs), bootstrap via ensurepip before attempting upgrade - Bump version to 2026.3.14 - Bump installer min version pins to 2026.3.14 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
23eb7fc0a7
|
Fix Colab Studio launch and setup.ps1 box alignment (#4601)
* Fix Colab Studio launch and setup.ps1 box alignment - colab.py: when the Studio venv is missing on Colab, pip-install backend dependencies (structlog, fastapi, etc.) from studio.txt into the current Python instead of failing with ModuleNotFoundError - setup.sh: on Colab without a venv, install backend deps into system Python and skip venv-dependent sections (Python stack update, llama.cpp build) that would otherwise fail - setup.ps1: use PadRight(47) for the done-line so "Setup Complete!" and "Update Complete!" both align with the box border * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Roland Tannous
|
19e9c60a8e
|
Consolidate dual venvs and separate install from update (#4530)
* refactor: consolidate dual venvs into single ~/.unsloth/studio/unsloth_studio
* refactor: separate install.sh (first-time) from setup.sh (smart update with PyPI version check)
* fix: install.sh calls setup.sh directly, keep both setup and update CLI commands
* fix: use importlib.resources.files() directly without _path attribute
* fix: bootstrap uv before pip upgrade to handle uv venvs without pip
* fix: frontend 404 when launched via CLI, add global symlink to ~/.local/bin
* feat: add --local flag to install.sh and unsloth studio update for branch testing
* fix: resolve repo root from script location for --local installs
* feat: add --package flag to install.sh for testing with custom package names
* feat: add --package flag to unsloth studio update
* fix: always nuke venv in install.sh for clean installs
* revert: remove Windows changes, will handle in separate PR
* fix: error when --package is passed without an argument
* revert: restore Windows scripts to current main
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: always explicitly set STUDIO_LOCAL_INSTALL and STUDIO_PACKAGE_NAME env vars
* fix: pass explicit STUDIO_LOCAL_REPO env var for --local installs
* fix: align banner box for Setup vs Update labels
* deprecate: hide 'unsloth studio setup' command, point users to update/install.sh
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: check stdout not stdin for auto-launch detection (curl pipe fix)
* fix: update install URL to unsloth.ai/install.sh
* fix: update install.sh usage comments to unsloth.ai/install.sh
* fix: use --upgrade-package for base deps to preserve existing torch/CUDA installs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: --local install now also installs unsloth-zoo via base.txt before editable overlay
* fix: don't skip base packages for --local installs (editable needs unsloth-zoo)
* refactor: move --local full dep install to install.sh, keep SKIP_STUDIO_BASE for all paths
* feat: add migration support for old .venv and CWD-based installs in setup.sh
* Revert "feat: add migration support for old .venv and CWD-based installs in setup.sh"
This reverts commit
|
||
|
Daniel Han
|
100b8857f2
|
Fix Studio crash on Anaconda/conda-forge Python (#4484)
* Fix Studio crash on Anaconda Python due to platform._sys_version() parse failure
Anaconda and conda-forge modify sys.version to include distributor
metadata between pipe characters, e.g.:
3.12.4 | packaged by Anaconda, Inc. | (main, ...) [MSC v.1929 ...]
Python's platform._sys_version() has a hardcoded regex that cannot
parse this format, raising ValueError. CPython closed this as "not
planned" (cpython#102396) since Anaconda modified the binary.
This breaks the import chain: run.py -> structlog -> rich -> attrs,
which calls platform.python_implementation() at module scope.
Fix: before any library imports, strip the pipe segments, parse the
cleaned version string via the standard parser, and cache the result
under the original sys.version key so all subsequent platform calls
hit the cache.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add defensive fallback for unpaired pipe edge cases in version patch
Address Gemini review suggestion: if the paired-pipe regex leaves
residual pipes (hypothetical single-pipe distributor metadata), fall
back to extracting the version number and the parenthesized build
info directly. Wrap the entire patch in try/except so unexpected
version string formats degrade gracefully instead of crashing the
patch itself.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor into shared _platform_compat module, cover colab.py entrypoint
Address reviewer feedback:
1. Extract the Anaconda/conda-forge sys.version fix into a shared
_platform_compat.py module that wraps platform._sys_version() with
a retry-on-ValueError fallback. This is more robust than cache-seeding
because it handles all future platform._sys_version() calls, not just
the first one.
2. Import the fix from both run.py and colab.py entrypoints, so Studio
no longer crashes on Anaconda Python regardless of the launch path.
3. The wrapper is idempotent (guarded by a flag) and handles edge cases:
paired pipes (Anaconda, conda-forge), unpaired pipes (hypothetical),
and standard CPython strings (no-op since ValueError is never raised).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Replace monkey-patch with cache-prime, fix colab.py duplicate sys.path, cover main.py
- Rewrite _platform_compat.py: replace function-wrapping monkey-patch with
one-shot cache seed (_seed_sys_version_cache). Parses cleaned sys.version
once and seeds platform._sys_version_cache so the stdlib parser never sees
the problematic Anaconda/conda-forge pipe-delimited string. No function
replacement, no idempotency flag, no reload edge cases.
- colab.py: remove duplicate backend_path sys.path insertion after
_bootstrap_studio_venv(). The early insertion (before _platform_compat
import) already covers it. This also fixes backend/ ending up behind
venv site-packages in sys.path ordering.
- run.py: move PYTHONWARNINGS=ignore before _platform_compat import to
preserve original intent of suppressing warnings early.
- main.py: add sys.path + _platform_compat import before route imports,
covering the direct `uvicorn main:app` launch path.
- Add test_platform_compat.py with 7 tests covering Anaconda, conda-forge,
and standard CPython version strings, plus the loggers import chain.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove test_platform_compat.py from PR
* Handle Format B conda-forge version strings with duplicate paren groups
Some conda-forge builds produce sys.version with the build info both
before and after the pipe label (e.g. "3.9.7 (default, ...) | packaged
by conda-forge | (default, ...) \n[GCC 7.5.0]"). After stripping the
pipe segment, two consecutive (...) groups remain, which still fails
platform._sys_version(). Add a second regex pass to drop the duplicate
paren group.
* Guard _sys_version call with try/except to avoid making things worse
If the cleaned version string is still unparseable by the stdlib regex
(e.g. nested parens, exotic multi-pipe formats), silently give up
instead of letting ValueError propagate at import time -- which would
be a worse crash than the original deferred one.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
Manan Shah
|
6f129a214b
|
Fix Install commands for Windows + 1 line installs (#4447)
* One liner setup for unsloth studio * Fix install scripts: system deps, activation bugs, curl/wget support - install.sh: detect platform (macOS/Linux/WSL) and check for missing system dependencies (cmake, git, build-essential, libcurl4-openssl-dev). Prompt user once for permission to install all missing packages via brew (macOS) or sudo apt-get (Linux/WSL). Add wget fallback via download() helper since curl is not always present on minimal Linux installs. Fix nested curl|sh stdin stealing by downloading uv installer to a tempfile first. Replace venv activation (no-op in a pipe subshell) with explicit --python flag for uv pip install and direct venv binary invocation. Add idempotency guard for venv creation. Redirect stdin on unsloth studio setup to prevent pipe consumption. On macOS, check for Xcode Command Line Tools and trigger install if missing. - install.ps1: wrap script body in Install-UnslothStudio function so that errors use return instead of exit (exit kills the terminal when run via irm|iex). Remove activate.ps1 invocation entirely -- use explicit --python path for uv pip install and & $UnslothExe for studio setup. This avoids both the child-scope activation bug (& vs dot-source) and the execution policy error on default Windows systems. Add winget availability check with clear error message. Fix PATH refresh to append registry paths instead of replacing the session PATH. Add uv installer fallback via astral.sh PowerShell script if winget install does not put uv on PATH. Broaden Python version check to accept 3.11-3.13. Add idempotency guard for venv creation. - README.md: add wget one-liner alternative for systems without curl. * Fix Tailwind CSS v4 .gitignore bug on Windows (#4444) - Add .gitignore hiding workaround to setup.ps1 (matching existing setup.sh logic) so venv .gitignore files containing "*" don't prevent Tailwind's oxide scanner from finding .tsx source files - Add CSS size validation to setup.sh, setup.ps1, and build.sh to catch truncated Tailwind builds early - Remove stray force-rebuild overrides that made the "skip build if current" cache check dead code in both setup scripts - Add rm -rf dist to build.sh to force clean rebuilds for wheel packaging * Change default port 8000 to 8888, fix installer bugs, improve UX - Change default Studio port from 8000 to 8888 across all entry points (run.py, studio.py, ui.py, colab.py, vite.config.ts, setup scripts) - Update launch banner: "Launching with studio venv..." to "Launching Unsloth Studio... Please wait..." - Add "Open your web browser" banner and rename labels (Local -> Local Access, External -> Worldwide Web Address) - Fix venv idempotency: check for bin/python instead of just directory existence, clean up partial venvs on retry - Fix build.sh CSS validation: handle empty CSS case that silently bypassed the check with "integer expression expected" - Fix install.sh sudo handling: try apt-get without sudo first (works when root), then escalate with per-package tracking and user prompt - Fix install.ps1: check exit code from studio setup, fail on error - Add pciutils to WSL GGUF build dependencies - Apply same smart apt-get escalation pattern to studio/setup.sh * Use detected Python version for venv, abort on non-apt Linux - install.ps1: detect existing Python 3.11/3.12/3.13 and use that version for venv creation instead of always forcing 3.13 - install.sh: exit with error on non-apt Linux distros when required packages cannot be auto-installed, instead of silently continuing * Make sudo permission prompt more prominent with warning banner * Add Accept [Y/n] sudo prompt to studio/setup.sh for consistency * Fix native command exit code handling and sudo decline flow install.ps1: Add $LASTEXITCODE checks after winget (Python), uv venv, and uv pip install calls. $ErrorActionPreference only catches PowerShell cmdlet errors, not native executable failures. The Python check also handles winget returning non-zero for "already installed". setup.sh: Skip llama-server build when user declines sudo or sudo is unavailable. Previously the script continued to section 8 which would fail with confusing errors (e.g. "gcc: command not found") since build-essential was never installed. * Move rm -rf llama.cpp inside build branch to preserve existing install When _SKIP_GGUF_BUILD is set (user declined sudo or sudo unavailable), the previous rm -rf would destroy an already-working llama-server before the skip check ran. Move it inside the else branch so existing builds are preserved when the rebuild is skipped. --------- Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
1f12ba16df
|
Combine studio setup fixes: frontend caching, venv isolation, Windows CPU support (#4413)
* Allow Windows setup to complete without NVIDIA GPU setup.ps1 previously hard-exited if nvidia-smi was not found, blocking setup entirely on CPU-only or non-NVIDIA machines. The backend already supports CPU and MLX (Apple Silicon) in chat-only GGUF mode, and the Linux/Mac setup.sh handles missing GPUs gracefully. Changes: - Convert the GPU check from a hard exit to a warning - Guard CUDA toolkit installation behind $HasNvidiaSmi - Install CPU-only PyTorch when no GPU is detected - Build llama.cpp without CUDA flags when no GPU is present - Update doc comment to reflect CPU support * Cache frontend build across setup runs Skip the frontend npm install + build if frontend/dist already exists. Previously setup.ps1 nuked node_modules and package-lock.json on every run, and both scripts always rebuilt even when dist/ was already present. On a git clone editable install, the first setup run still builds the frontend as before. Subsequent runs skip it, saving several minutes. To force a rebuild, delete frontend/dist and re-run setup. * Show pip progress for PyTorch download on Windows The torch CUDA wheel is ~2.8 GB and the CPU wheel is ~300 MB. With | Out-Null suppressing all output, the install appeared completely frozen with no feedback. Remove | Out-Null for the torch install lines so pip's download progress bar is visible. Add a size hint so users know the download is expected to take a while. Also moves the Triton success message inside the GPU branch so it only prints when Triton was actually installed. * Guard CUDA env re-sanitization behind GPU check in llama.cpp build The CUDA_PATH re-sanitization block (lines 1020-1033) references $CudaToolkitRoot which is only set when $HasNvidiaSmi is true and the CUDA Toolkit section runs. On CPU-only machines, $CudaToolkitRoot is null, causing Split-Path to throw: Split-Path : Cannot bind argument to parameter 'Path' because it is null. Wrap the entire block in `if ($HasNvidiaSmi -and $CudaToolkitRoot)`. * Rebuild frontend when source files are newer than dist/ Instead of only checking if dist/ exists, compare source file timestamps against the dist/ directory. If any file in frontend/src/ is newer than dist/, trigger a rebuild. This handles the case where a developer pulls new frontend changes and re-runs setup -- stale assets get rebuilt automatically. * Fix cmake not found on Windows after winget install Two issues fixed: 1. After winget installs cmake, Refresh-Environment may not pick up the new PATH entry (MSI PATH changes sometimes need a new shell). Added a fallback that probes cmake's default install locations (Program Files, LocalAppData) and adds the directory to PATH explicitly if found. 2. If cmake is still unavailable when the llama.cpp build starts (e.g. winget failed silently or PATH was not updated), the build now skips gracefully with a [SKIP] warning instead of crashing with "cmake : The term 'cmake' is not recognized". * Fix frontend rebuild detection and decouple oxc-validator install Address review feedback: - Check entire frontend/ directory for changes, not just src/. The build also depends on package.json, vite.config.ts, tailwind.config.ts, public/, and other config files. A change to any of these now triggers a rebuild. - Move oxc-validator npm install outside the frontend build gate in setup.sh so it always runs on setup, matching setup.ps1 which already had it outside the gate. * Show cmake errors on failure and retry CUDA VS integration with elevation Two fixes for issue #4405 (Windows setup fails at cmake configure): 1. cmake configure: capture output and display it on failure instead of piping to Out-Null. When the error mentions "No CUDA toolset found", print a hint about the CUDA VS integration files. 2. CUDA VS integration copy: when the direct Copy-Item fails (needs admin access to write to Program Files), retry with Start-Process -Verb RunAs to prompt for elevation. This is the root cause of the "No CUDA toolset found" cmake failure -- the .targets files that let MSBuild compile .cu files are missing from the VS BuildCustomizations directory. * Address reviewer feedback: cmake PATH persistence, stale cache, torch error check 1. Persist cmake PATH to user registry so Refresh-Environment cannot drop it later in the same setup run. Previously the process-only PATH addition at phase 1 could vanish when Refresh-Environment rebuilt PATH from registry during phase 2/3 installs. 2. Clean stale CMake cache before configure. If a previous run built with CUDA and the user reruns without a GPU (or vice versa), the cached GGML_CUDA value would persist. Now the build dir is removed before configure. 3. Explicitly set -DGGML_CUDA=OFF for CPU-only builds instead of just omitting CUDA flags. This prevents cmake from auto-detecting a partial CUDA installation. 4. Fix CUDA cmake flag indentation -- was misaligned from the original PR, now consistently indented inside the if/else block. 5. Fail hard if pip install torch returns a non-zero exit code instead of silently continuing with a broken environment. * Remove extra CUDA cmake flags to align Windows with Linux build Drop GGML_CUDA_FA_ALL_QUANTS, GGML_CUDA_F16, GGML_CUDA_GRAPHS, GGML_CUDA_FORCE_CUBLAS, and GGML_CUDA_PEER_MAX_BATCH_SIZE flags. The Linux build in setup.sh only sets GGML_CUDA=ON and lets llama.cpp use its defaults for everything else. Keep Windows consistent. * Address reviewer round 2: GPU probe fallback, Triton check, stale binary rebuild 1. GPU detection: fallback to default nvidia-smi install locations (Program Files\NVIDIA Corporation\NVSMI, System32) when nvidia-smi is not on PATH. Prevents silent CPU-only provisioning on machines that have a GPU but a broken PATH. 2. Triton: check $LASTEXITCODE after pip install and print [WARN] on failure instead of unconditional [OK]. 3. Stale llama-server: check CMakeCache.txt for GGML_CUDA setting and rebuild if the existing binary does not match the current GPU mode (e.g. CUDA binary on a now-CPU-only rerun, or vice versa). * Fix frontend rebuild detection and npm dependency issues Addresses reviewer feedback on the frontend caching logic: 1. setup.sh: Fix broken find command that caused exit under pipefail. The piped `find | xargs find -newer` had paths after the expression which GNU find rejects. Replaced with a simpler `find -maxdepth 1 -type f -newer dist/` that checks ALL top-level files (catches index.html, bun.lock, etc. that the extension allowlist missed). 2. setup.sh: Guard oxc-validator npm install behind `command -v npm` check. When the frontend build is skipped (dist/ is cached), Node bootstrap is also skipped, so npm may not be available. 3. setup.ps1: Replace Get-ChildItem -Include with explicit path probing for src/ and public/. PowerShell's -Include without a trailing wildcard silently returns nothing, so src/public changes were never detected. Also check ALL top-level files instead of just .json/.ts/.js/.mjs extensions. * Fix studio setup: venv isolation, centralized .venv_t5, uv targeting - All platforms (including Colab) now create ~/.unsloth/studio/.venv with --without-pip fallback for broken ensurepip environments - Add --python sys.executable to uv pip install in install_python_stack.py so uv targets the correct venv instead of system Python - Centralize .venv_t5 bootstrap in transformers_version.py with proper validation (checks required packages exist, not just non-empty dir) - Replace ~150 lines of duplicated install code across 3 worker files with calls to the shared _ensure_venv_t5_exists() helper - Use uv-if-present with pip fallback; do not install uv at runtime - Add site.addsitedir() shim in colab.py so notebook cells can import studio packages from the venv without system-Python double-install - Update .venv_t5 packages: huggingface_hub 1.3.0->1.7.1, add hf_xet - Bump transformers pin 4.57.1->4.57.6 in requirements + constraints - Add Fast-Install helper to setup.ps1 with uv+pip fallback - Keep Colab-specific completion banner in setup.sh * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix nvidia-smi PATH persistence and cmake requirement for CPU-only 1. Store nvidia-smi as an absolute path ($NvidiaSmiExe) on first detection. All later calls (Get-CudaComputeCapability, Get-PytorchCudaTag, CUDA toolkit detection) use this absolute path instead of relying on PATH. This survives Refresh-Environment which rebuilds PATH from the registry and drops process-only additions. 2. Make cmake fatal for CPU-only installs. CPU-only machines depend entirely on llama-server for GGUF chat mode, so reporting "Setup Complete!" without it is misleading. GPU machines can still skip the llama-server build since they have other inference paths. * Fix broken frontend freshness detection in setup scripts - setup.sh: Replace broken `find | xargs find -newer` pipeline with single `find ... -newer` call. The old pipeline produced "paths must precede expression" errors (silently suppressed by 2>/dev/null), causing top-level config changes to never trigger a rebuild. - setup.sh: Add `command -v npm` guard to oxc-validator block so it does not fail when Node was not installed (build-skip path). - setup.ps1: Replace `Get-ChildItem -Include` (unreliable without -Recurse on PS 5.1) with explicit directory paths for src/ and public/ scanning. - Both: Add *.html to tracked file patterns so index.html (Vite entry point) changes trigger a rebuild. - Both: Use -print -quit instead of piping to head -1 for efficiency. * Fix bugs found during review of PRs #4404, #4400, #4399 - setup.sh: Add || true guard to find command that checks frontend/src and frontend/public dirs, preventing script abort under set -euo pipefail when either directory is missing - colab.py: Use sys.path.insert(0, ...) instead of site.addsitedir() so Studio venv packages take priority over system copies. Add warning when venv is missing instead of silently failing. - transformers_version.py: _venv_t5_is_valid() now checks installed package versions via .dist-info metadata, not just directory presence. Prevents false positives from stale or wrong-version packages. - transformers_version.py: _install_to_venv_t5() now passes --upgrade so pip replaces existing stale packages in the target directory. - setup.ps1: CPU-only PyTorch install uses --index-url for cpu wheel and all install commands use Fast-Install (uv with pip fallback). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix _venv_t5_is_valid dist-info loop exiting after first directory Remove premature break that caused the loop over .dist-info directories to exit after the first match even if it had no METADATA file. Now continues iterating until a valid METADATA is found or all dirs are exhausted. * Capture error output on failure instead of discarding with Out-Null setup.ps1: 6 locations changed from `| Out-Null` to `| Out-String` with output shown on failure -- PyTorch GPU/CPU install, Triton install, venv_t5 package loop, cmake llama-server and llama-quantize builds. transformers_version.py: clean stale .venv_t5 directory before reinstall when validation detects missing or version-mismatched packages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix ModuleNotFoundError when CLI imports studio.backend.core The backend uses bare "from utils.*" imports everywhere, relying on backend/ being on sys.path. Workers and routes add it at startup, but the CLI imports studio.backend.core as a package -- backend/ was never added. Add sys.path setup at the top of core/__init__.py so lazy imports resolve correctly regardless of entry point. Fixes: unsloth inference unsloth/Qwen3-8B "who are you" crashing with "No module named 'utils'" * Fix frontend freshness check to detect all top-level file changes The extension allowlist (*.json, *.ts, *.js, *.mjs, *.html) missed files like bun.lock, so lockfile-only dependency changes could skip the frontend rebuild. Check all top-level files instead. * Add tiktoken to .venv_t5 for Qwen-family tokenizers Qwen models use tiktoken-based tokenizers which fail when routed through the transformers 5.x overlay without tiktoken installed. Add it to the setup scripts (with deps for Windows) and runtime fallback list. Integrates PR #4418. * Fix tiktoken crash in _venv_t5_is_valid and stray brace in setup.ps1 _venv_t5_is_valid() crashed with ValueError on unpinned packages like "tiktoken" (no ==version). Handle by splitting safely and skipping version check for unpinned packages (existence check only). Also remove stray closing brace in setup.ps1 tiktoken install block. --------- Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Leo Borcherding
|
262271a20d
|
Fix/colab comment edits (#4317)
* Removing .precommit config * edited colab comments * studio: update Unsloth_Studio_Colab.ipynb * studio: update Unsloth_Studio_Colab.ipynb * studio: add Colab T4 GPU metadata to force T4 instance * style: update colab popup to black/white theme with gem icon and play button * feat: center landscape image in colab notebook * style: shrink popup to fit content, truncate URL display * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: center landscape image in colab notebook * feat: use GitHub raw URL for studio landscape image in notebook * chore: update colab notebook --------- Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
Roland Tannous
|
47654cb91c | Final cleanup | ||
|
Roland Tannous
|
a2baf80511 | Update license headers | ||
|
Roland Tannous
|
817f2e8dcc | feat: integrate structlog, configure workers for prod logging, and migrate print statements | ||
|
Roland Tannous
|
d882678fe4 | Add AGPL-3.0 SPDX headers to all source files | ||
Leo Borcherding
|
17df5bf3bf |
feat: Add simple 2-cell Colab notebook (no tunnel needed)
- Create studio/backend/colab.py using Colab's built-in proxy - Uses google.colab.kernel.proxyPort() for URL (no cloudflare) - Shows nice clickable link with IPython.display.HTML - Notebook has just 2 cells: setup and start - Much simpler than external tunneling approach |