mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-20 14:23:55 +00:00
06220da4d2
* Run two short jobs as background lanes of Lint CI, not on their own runners
Two workflows each held a runner slot on every commit for a few seconds of
read-only checking:
Unsloth load-orchestrator CI :: test ~33s, its own slot
Lockfile supply-chain audit :: audit ~6s, its own slot
Both now run as background lanes inside Lint CI, which has no path filter and
was already going to occupy a runner on every commit. Three slots become one.
Backgrounded rather than appended as steps, so the lanes overlap the ~65s of
lint instead of adding to it: ubuntu-latest has four cores and the lint steps
are single-threaded. Measured locally on the real step bodies, a 5s lane
alongside 5s of foreground work took 7s total, not 10s.
Absorbing a narrower-triggered job into an unfiltered one can only reduce the
slots a commit takes. The absorbed work now runs on commits that would not have
triggered it, on a runner that was going to exist anyway.
Three things about this were not obvious and are worth recording:
* Each lane needs its own venv. The load-orchestrator lane installs fastapi
and friends while the foreground steps use the interpreter Lint CI's own
pip install populated, and two concurrent installs into one site-packages
is a real race.
* The launch has to detach from the step's stdout and stderr. A background
child inherits those pipes and the step is not considered finished while a
writer holds them: launching a 4s lane took 4.0s before the redirect and
0.0s after. Nothing would have gone red, the overlap would just have
silently disappeared.
* A missing exit-status file has to be a failure. A lane that never started
is otherwise indistinguishable from one that passed.
The payloads live in .github/scripts/lane-*.sh, called by both the lane and the
standalone workflow, so there is one definition. studio-load-orchestrator-ci.yml
keeps workflow_dispatch as an escape hatch; its push trigger had no paths filter
at all, so it was running on every commit to main regardless of the four paths
its PR trigger listed. lockfile-audit.yml keeps its nightly schedule, which is a
different check: it re-reads the lockfiles against advisories published since
the last commit, which no commit-triggered run can do.
tests/studio/test_absorbed_lanes_still_run.py covers the silent failure modes.
Eight mutations verified red, including an uncollected lane, a missing sentinel
read as success, and a launch that stops detaching.
* Run the lane guard on workflow-only PRs
The guard covers work moved off its own runner into a background lane, and
every way that regresses -- a lane launched but never collected, a missing exit
status read as success, a launch that stops detaching -- is an edit to
lint-ci.yml or to one of the two absorbed workflows.
No job's paths filter matches a workflow-only change: workflow-trigger-lint
enumerates specific test modules, and Backend CI matches tests/** and its own
YAML but not the workflow files guarded here. So the suite would first be
collected by Backend CI's unfiltered push on main, after the change had merged.
That matters more than usual because every failure mode here is silent. Nothing
goes red when a lane stops being collected; the absorbed suites just quietly
stop being able to fail anything.
* Run the workflow guards in one pytest, and run the ten that ran nowhere
workflow-trigger-lint is the only job in the repo with no paths filter, which
makes it the only job a workflow-only PR is guaranteed to start. Every other
job filters on source paths: Backend CI matches studio/**, tests/**, scripts/**
and its own YAML, but not arbitrary workflow files.
So a guard that reads a workflow file and is not run by this job has a specific
silent hole. The edit it exists to reject is by definition a workflow-only edit,
and on such a PR it is never collected. It is collected later by Backend CI's
unfiltered push on main, after the change has merged. The guard still works, it
just stops being able to block anything.
Ten modules were in that state at once, including the indicator-parallelism
guard, the GGUF phase-independence guard and the Windows small-checks guard,
each of which reads a workflow and can only be broken by editing one. Three
separate review rounds reported one instance each before the pattern was
visible.
They now run in ONE pytest invocation rather than a step per module. Measured
over these 18 modules: 53.9s as a single call against 300.8s as one each,
because this repo's conftest is expensive to import and a step per module pays
it every time. That also removes the reason the list stopped growing: adding a
guard no longer means adding a step.
tests/studio/test_workflow_guards_run_unfiltered.py keeps the list honest, with
two exemptions carrying their reason (PIL and a local utils helper, neither of
which that job installs). It caught itself on its first run, being a
workflow-reading module that was not yet in the list.
Three mutations verified red: a guard dropped from the list, the invocation
split back into two steps, and the job gaining a paths filter.
* Give the consolidated guard job the cores and the budget it now needs
Absorbing 12 more guard modules into one step made that step the job, and the
job still carried a 5 minute budget sized for the old set. Serial the 21
modules are 209s on a 192-core box, which would have failed on timeout on a
slower runner rather than on anything real.
So: pytest-xdist, `-n 4`, and 15 minutes.
Pinned to 4 rather than `auto` on purpose. ubuntu-latest is a 4-core runner so
on CI the two are identical, but `auto` scales to the host and every xdist
worker re-imports this repo's expensive conftest. Measured here: `auto` spawned
192 workers and took 327s, worse than running serially, while `-n 4` is 42s.
A fixed width behaves the same everywhere and cannot be made pathological by
the machine it lands on.
304 tests, identical results serial and parallel.
|
||
|---|---|---|
| .. | ||
| kaggle_studio_ci | ||
| kaggle_t4_ci | ||
| studio_smoke | ||
| agent-guides-drive.sh | ||
| agent-guides-install.sh | ||
| assert-bundle-signed.ps1 | ||
| assert-llama-loads.sh | ||
| assert-nobuild.ps1 | ||
| assert-prompt-cache.sh | ||
| boot-studio-api-only.sh | ||
| ci-connect-prompt.txt | ||
| ci-min-system-prompt.txt | ||
| clean-machine-assert.sh | ||
| clean-machine-env.sh | ||
| clean-machine-install-name-tool.sh | ||
| ensure-docker-daemon.ps1 | ||
| hf-download-with-retry.sh | ||
| interrupt-install.ps1 | ||
| interrupt-install.sh | ||
| interrupted_install_probe.py | ||
| lane-load-orchestrator.sh | ||
| lane-lockfile-audit.sh | ||
| resolve-desktop-release.py | ||
| run-studio-indicator-browser.sh | ||
| run-studio-permission-browser.sh | ||
| serve-unsloth-run.sh | ||
| virgin-windows-install.ps1 | ||
| virgin-windows-probe.ps1 | ||
| wait-for-health.sh | ||