mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-10 00:08:58 +00:00
* Add --with-llama-cpp-dir flag to install.ps1 and install.sh Users can now pass --with-llama-cpp-dir /path/to/llama.cpp to the installer to skip downloading or building llama.cpp and use a local directory instead. A junction (Windows) or symlink (Linux/macOS) is created at the canonical install location, bypassing both the prebuilt download (Phase 3) and source build (Phase 4) steps in setup.ps1/setup.sh. The path is passed via UNSLOTH_LOCAL_LLAMA_CPP_DIR env var which setup.ps1 and setup.sh read directly. Ported from the idea in unslothai/unsloth#4384, reimplemented against current Studio architecture. * test: add static wiring test for --with-llama-cpp-dir flag Cross-checks install.sh, install.ps1, studio/setup.sh and studio/setup.ps1 so the flag's contract (parse -> UNSLOTH_LOCAL_LLAMA_CPP_DIR env var -> link local dir, skip prebuilt download and source build) can't silently regress. Wired into studio-backend-ci.yml alongside the other tests/sh installer tests. * Address review feedback on --with-llama-cpp-dir flag - setup.ps1: delete an existing junction/symlink via DirectoryInfo.Delete() instead of a recursive remove, which can traverse the link and wipe the user's real llama.cpp directory on PowerShell 5.1. - setup.ps1: short-circuit the build chain when a local dir is linked so CMake never runs inside the user's checkout when it lacks a Windows-layout binary. - install.sh / setup.sh: resolve paths with CDPATH= cd -P so a set CDPATH cannot corrupt the resolved path. - install.sh: seed _WITH_LLAMA_CPP_DIR from UNSLOTH_LOCAL_LLAMA_CPP_DIR so an exported env var (piped-install style) is honored instead of being clobbered. - setup.sh: create the root llama-quantize shim when linking a local source build so GGUF export's check_llama_cpp() still finds it. - setup.sh / setup.ps1: drop a stale link before the custom-home ownership assert so re-runs with the flag stay idempotent. - test: pin the new linked-dir build short-circuit. * Harden --with-llama-cpp-dir against Codex/Gemini review findings - install.sh: error when --with-llama-cpp-dir is the final arg with no path, matching the existing --package/--python post-loop guards (was a silent fallback to the normal prebuilt/source install). - studio/setup.sh: canonicalize LLAMA_CPP_DIR before the self-link no-op compare. _RESOLVED_LOCAL is fully resolved while LLAMA_CPP_DIR was textual, so a symlinked $HOME made the guard miss and the rm -rf could wipe the user's real llama.cpp tree. - studio/setup.sh: make the llama-quantize shim non-fatal; it writes through the link into the user's tree, which may be read-only (shared/CI cache), and under set -e a failed ln aborted an otherwise-good reuse. - studio/setup.ps1: detect a broken junction via Get-Item -Force instead of Test-Path so a dangling link from a prior run is removed and mklink can relink to a new valid directory. - studio/setup.ps1: use Copy-Item -LiteralPath so a source path containing [ ] isn't treated as a wildcard in the junction copy fallback. - tests: update the wiring assertions for the LiteralPath copy and the canonicalized compare. * Validate/reuse local llama.cpp tree and guard the in-use case Addresses the second Codex pass on the --with-llama-cpp-dir flag: - Validate the linked tree before disabling installs (setup.sh + setup.ps1): reusing a local dir skips BOTH the prebuilt download and the source build, so the dir must already contain a runnable llama-server (build/bin on Linux/macOS, build\bin\Release\llama-server.exe on Windows). Bail out with a clear message instead of linking an unbuilt/wrong-platform checkout and leaving Studio with no usable binary. - Treat a canonical-path target as already linked when it holds a build (setup.sh + setup.ps1): point the flag at ~/.unsloth/llama.cpp itself and an existing build is reused (skip prebuilt + source) rather than clobbered by the staged prebuilt installer (which uses os.replace()/replace). An empty canonical dir still falls through to the normal in-place install. - Abort when an in-use llama.cpp can't be removed on Windows (setup.ps1): Remove-Item -ErrorAction SilentlyContinue can silently leave a locked tree in place; detect that and stop with the same active-process message + exit 3 the prebuilt path uses, instead of junctioning over a half-present dir. Left as follow-up (already tracked by the PR author as a non-blocker): the in-app "Update llama.cpp" updater does not yet recognize a local-link install as externally managed; that fix belongs in studio/backend/utils/llama_cpp_update.py. * Accept all backend llama-server layouts in --with-llama-cpp-dir validation The linked-tree validation only accepted build/bin[/Release]/llama-server, but LlamaCppBackend._layout_candidates() resolves a root-level llama-server first, then build/bin, then build/bin/Release on Windows. A `make` build or a flat release extract (binary at the dir root) was therefore rejected with a hard installer failure even though Studio would have run it. Validate the same candidate set the backend uses in both setup scripts, and add wiring-test assertions so the check can't silently narrow again. * Treat --with-llama-cpp-dir local links as externally managed A --with-llama-cpp-dir install junctions/symlinks the canonical llama.cpp dir to the user's own checkout, but two backend paths still treated it as a Studio-owned tree: - The in-app updater (llama_cpp_update) offered and could apply an official prebuilt over the link, writing through it into the user's checkout (or failing) and silently dropping the link the flag created. - Orphan cleanup (LlamaCppBackend._kill_orphaned_servers) resolved the linked root into its kill allowlist, so a llama-server the user launched from the same checkout was classified as ours and killed on startup. Detect the canonical dir being a symlink/junction (reparse point) and treat the install as unmanaged: get_update_status reports unsupported, start_update refuses with reason "local_link", and the linked root is left out of the orphan allowlist. Adds behavioral tests (link vs plain dir, updater refusal, and the spared-vs-killed orphan control). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add behavioral shell test for --with-llama-cpp-dir linking The existing tests/sh/test_with_llama_cpp_dir_flag.sh is a static grep of the scripts. This adds a behavioral test that extracts the real link block from studio/setup.sh (by content anchors, with a self-validating extraction) and runs it against hermetic fake dirs, asserting the outcomes that matter: - an external CMake build links and arms neither the prebuilt download nor the source build - a flat / make tree (root-level llama-server, no build/bin) is accepted too - an unbuilt tree is rejected with a non-zero exit and no link left behind - relinking over a stale link preserves the target's contents (no data loss) - pointing at the canonical path is a no-op reuse, not a self-referential link Symlink-identity checks run only where real symlinks exist (skipped on Windows git-bash copy-mode); the link/skip/no-data-loss checks run everywhere. Wired into studio-backend-ci.yml next to the static test. * Install psutil in backend CI so orphan-cleanup tests run The new orphan-cleanup tests import psutil for the process scan, but the Backend CI deps step installed studio.txt plus a fixed extras list that omits it, so the two tests failed with ModuleNotFoundError. Add psutil to both backend pytest dep steps (kept in shared shape), and guard the import with pytest.importorskip so a minimal env without psutil skips these tests instead of erroring. --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
137 lines
4.9 KiB
Python
137 lines
4.9 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""Behavioral tests for the --with-llama-cpp-dir 'unmanaged local link' contract.
|
|
|
|
When the canonical llama.cpp dir is a symlink (POSIX) / junction (Windows) to a
|
|
user's own checkout, Studio must treat it as externally managed:
|
|
- the in-app updater must not offer or apply a prebuilt over the link
|
|
- orphan cleanup must not kill a llama-server the user launched from that tree
|
|
|
|
These exercise real link behavior rather than grepping the scripts.
|
|
"""
|
|
|
|
import os
|
|
import subprocess
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from utils import llama_cpp_update as u
|
|
from core.inference.llama_cpp import LlamaCppBackend
|
|
|
|
|
|
def _make_link(link: Path, target: Path) -> None:
|
|
"""Create a directory junction (Windows) / symlink (POSIX); neither needs
|
|
elevation."""
|
|
target.mkdir(parents = True, exist_ok = True)
|
|
if os.name == "nt":
|
|
subprocess.run(
|
|
["cmd", "/c", "mklink", "/J", str(link), str(target)],
|
|
check = True,
|
|
capture_output = True,
|
|
text = True,
|
|
)
|
|
else:
|
|
link.symlink_to(target, target_is_directory = True)
|
|
|
|
|
|
def _server_subpath() -> Path:
|
|
return Path(
|
|
"build/bin/Release/llama-server.exe" if os.name == "nt" else "build/bin/llama-server"
|
|
)
|
|
|
|
|
|
class _FakeProc:
|
|
def __init__(self, pid: int, exe: str) -> None:
|
|
self.info = {"pid": pid, "name": "llama-server", "exe": exe}
|
|
self.killed = False
|
|
|
|
def kill(self) -> None:
|
|
self.killed = True
|
|
|
|
|
|
def test_is_external_link_detects_link_vs_plain_dir(tmp_path: Path) -> None:
|
|
plain = tmp_path / "plain"
|
|
plain.mkdir()
|
|
assert u._is_external_link(plain) is False
|
|
|
|
link = tmp_path / "link"
|
|
_make_link(link, tmp_path / "tgt")
|
|
assert u._is_external_link(link) is True
|
|
|
|
|
|
def test_active_install_is_local_link(tmp_path: Path) -> None:
|
|
link = tmp_path / "llama.cpp"
|
|
_make_link(link, tmp_path / "tgt")
|
|
binary = str(link / _server_subpath())
|
|
assert u._active_install_is_local_link(binary) is True
|
|
|
|
# A plain (non-link) llama.cpp dir is Studio-managed, not a local link.
|
|
plain = tmp_path / "plain" / "llama.cpp"
|
|
plain.mkdir(parents = True)
|
|
assert u._active_install_is_local_link(str(plain / _server_subpath())) is False
|
|
|
|
|
|
def test_get_update_status_reports_local_link(tmp_path: Path, monkeypatch) -> None:
|
|
link = tmp_path / "llama.cpp"
|
|
_make_link(link, tmp_path / "tgt")
|
|
monkeypatch.setattr(u, "_find_binary", lambda: str(link / _server_subpath()))
|
|
st = u.get_update_status()
|
|
assert st["supported"] is False
|
|
assert st["update_available"] is False
|
|
assert st["local_link"] is True
|
|
|
|
|
|
def test_start_update_refuses_local_link(tmp_path: Path, monkeypatch) -> None:
|
|
link = tmp_path / "llama.cpp"
|
|
_make_link(link, tmp_path / "tgt")
|
|
monkeypatch.setattr(u, "_find_binary", lambda: str(link / _server_subpath()))
|
|
res = u.start_update()
|
|
assert res["started"] is False
|
|
assert res["reason"] == "local_link"
|
|
|
|
|
|
def _run_orphan_scan(monkeypatch, studio_root: Path, fake: _FakeProc) -> int:
|
|
# psutil drives the cross-platform process scan; skip (rather than error) if a
|
|
# minimal test env lacks it. CI installs it so these tests actually run.
|
|
psutil = pytest.importorskip("psutil")
|
|
|
|
monkeypatch.setattr(
|
|
LlamaCppBackend,
|
|
"_resolved_studio_root_and_is_legacy",
|
|
staticmethod(lambda: (studio_root.resolve(), False)),
|
|
)
|
|
monkeypatch.setattr(LlamaCppBackend, "_reap_recorded_pid", staticmethod(lambda: 0))
|
|
monkeypatch.setattr(psutil, "process_iter", lambda attrs = None: iter([fake]))
|
|
return LlamaCppBackend._kill_orphaned_servers()
|
|
|
|
|
|
def test_orphan_cleanup_spares_local_link_tree(tmp_path: Path, monkeypatch) -> None:
|
|
studio_root = tmp_path / "studio-home"
|
|
studio_root.mkdir()
|
|
external = tmp_path / "external"
|
|
(external / _server_subpath().parent).mkdir(parents = True)
|
|
(external / _server_subpath()).write_text("x")
|
|
_make_link(studio_root / "llama.cpp", external)
|
|
|
|
exe_under_link = str((external / _server_subpath()).resolve())
|
|
fake = _FakeProc(os.getpid() + 777, exe_under_link)
|
|
killed = _run_orphan_scan(monkeypatch, studio_root, fake)
|
|
assert killed == 0
|
|
assert fake.killed is False
|
|
|
|
|
|
def test_orphan_cleanup_kills_under_real_root(tmp_path: Path, monkeypatch) -> None:
|
|
# Control: a real (non-link) managed root still gets its orphan reaped, so
|
|
# the spare-the-link test above is meaningful (not a no-op).
|
|
studio_root = tmp_path / "studio-home"
|
|
bin_dir = studio_root / "llama.cpp" / _server_subpath().parent
|
|
bin_dir.mkdir(parents = True)
|
|
exe = studio_root / "llama.cpp" / _server_subpath()
|
|
exe.write_text("x")
|
|
|
|
fake = _FakeProc(os.getpid() + 888, str(exe.resolve()))
|
|
killed = _run_orphan_scan(monkeypatch, studio_root, fake)
|
|
assert killed == 1
|
|
assert fake.killed is True
|