MLX CI: find llama-cli where save_pretrained_gguf actually installs it (#6777)

* MLX CI: find llama-cli where save_pretrained_gguf actually installs it

The GGUF reload step hardcoded the CWD-relative paths llama.cpp/llama-cli and
llama.cpp/build/bin/llama-cli, but save_pretrained_gguf builds and installs llama.cpp
under unsloth_zoo's LLAMA_CPP_DEFAULT_DIR ($UNSLOTH_LLAMA_CPP_PATH, else
~/.unsloth/llama.cpp), so the reload could not find the binary and failed the Mac M1
job with "llama-cli not found". _find_llama_cli now searches that install directory
(and honors the env override) before falling back to the old CWD layout, with a
recursive glob as a last resort. The search is a strict superset of the previous
paths, so it cannot regress a layout that already worked.

* MLX CI: return an absolute llama-cli path from the locator

Resolve the located binary to an absolute path. If UNSLOTH_LLAMA_CPP_PATH is a
relative directory (e.g. "."), Path(".") / "llama-cli" normalizes to the bare name
"llama-cli", and subprocess.run treats a separator-less argument as a PATH lookup
rather than a file to execute, raising FileNotFoundError. resolve() makes the returned
path absolute so it always runs the intended binary.

* MLX CI: give llama-cli EOF on stdin so GGUF reload cannot hang

With the binary now found, the GGUF reload actually invokes llama-cli and it timed
out after 300s generating 24 tokens on a 270m model, which is a stdin block rather
than slow generation: subprocess.run captured stdout/stderr but left stdin inherited,
so -no-cnv still left llama-cli waiting for interactive input. Pass
stdin=subprocess.DEVNULL so it receives an immediate EOF and runs the single prompt to
completion.

---------

Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
This commit is contained in:
Daniel Han 2026-07-01 00:49:23 -07:00 committed by GitHub
parent 8cc05ac89c
commit bc69dfad08
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -515,14 +515,50 @@ def cmd_reload(args) -> int:
return 0
def _find_llama_cli() -> Path | None:
"""Locate the llama-cli binary save_pretrained_gguf built.
save_pretrained_gguf installs llama.cpp under unsloth_zoo's LLAMA_CPP_DEFAULT_DIR
($UNSLOTH_LLAMA_CPP_PATH or ~/.unsloth/llama.cpp), not the working directory, so
search there first and keep the CWD-relative layout as a fallback.
"""
bases: list[Path] = []
env_dir = os.environ.get("UNSLOTH_LLAMA_CPP_PATH")
if env_dir:
bases.append(Path(env_dir))
try:
from unsloth_zoo.llama_cpp import LLAMA_CPP_DEFAULT_DIR
bases.append(Path(LLAMA_CPP_DEFAULT_DIR))
except Exception:
bases.append(Path.home() / ".unsloth" / "llama.cpp")
bases.append(Path("llama.cpp"))
seen: set[Path] = set()
for base in bases:
if base in seen:
continue
seen.add(base)
for rel in ("llama-cli", "build/bin/llama-cli"):
cand = base / rel
if cand.is_file() and os.access(cand, os.X_OK):
# Absolute: a separator-less relative path would send subprocess
# to a PATH lookup instead of running the file.
return cand.resolve()
# Last resort: the binary may sit under an unexpected build subdir.
if base.is_dir():
for cand in sorted(base.glob("**/llama-cli")):
if cand.is_file() and os.access(cand, os.X_OK):
return cand.resolve()
return None
def _reload_gguf(save_dir: Path, metrics: dict) -> int:
candidates = [
Path("llama.cpp/llama-cli"),
Path("llama.cpp/build/bin/llama-cli"),
]
llama_cli = next((c for c in candidates if c.exists()), None)
llama_cli = _find_llama_cli()
if llama_cli is None:
raise SystemExit(f"llama-cli not found; checked {candidates}")
raise SystemExit(
"llama-cli not found under $UNSLOTH_LLAMA_CPP_PATH, "
"~/.unsloth/llama.cpp, or ./llama.cpp"
)
gguf_files = sorted(save_dir.glob("*.gguf"))
if not gguf_files:
@ -549,6 +585,9 @@ def _reload_gguf(save_dir: Path, metrics: dict) -> int:
capture_output = True,
text = True,
timeout = 300,
# Hand llama-cli an immediate EOF; without it -no-cnv can still leave the
# process blocked reading stdin, which times out instead of generating.
stdin = subprocess.DEVNULL,
)
metrics["llama_cli_returncode"] = proc.returncode