mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-25 08:42:25 +00:00
2043c734c4
Over 400 completed main push runs, 25 of 96 job types execute in under 120s: 1116s of work spread across 25 runners, each queuing for about three hours. Two of them are dealt with here. Security audit :: pytest tests/security 72s exec, 11096s queue Unsloth export capability :: capability (ubuntu-latest) 67s exec, 10642s queue Different problems, so different treatments. The security suite MOVED, onto the Workflow trigger lint runner ------------------------------------------------------------------------ Same argument that put the lockfile and load-orchestrator lanes into Lint CI in #9176: work with a narrow trigger, moved into a job that was going to occupy a runner on this commit anyway, can only reduce the slots a commit takes. Here the trigger widens too, since this host has no paths filter and security-audit.yml's pull_request does, so the suite now runs on every pull request rather than on the ones that touch its paths. This host and NOT Lint CI, where the other lanes went, and that is the whole decision. Lint CI installs shellcheck from apt, so its harden-runner has to permit escalation and an apt mirror; a security gate moved there would run under a policy weaker than the one it has today. Workflow trigger lint's harden-runner block is byte-for-byte identical to the one the job carried in security-audit.yml (block, disable-sudo, the same six endpoints), so nothing about its isolation changes. harden-runner binds per runner, not per step, which is what makes that the deciding constraint rather than a detail. Folded into the existing pytest invocation rather than added as a step of its own. I wrote it as a separate step first, so a security regression would not be reported as a workflow-guard failure, and test_the_guards_run_in_one_pytest_invocation rejected it: one step per module costs about 15s of interpreter and conftest startup, measured in this repo at 53.9s as one invocation against 300.8s as one each. The guard is right and the attribution preference is not worth 15s. pytest and PyYAML are now pinned here to the versions security-audit.yml pinned them to. That suite runs scripts/lint_workflow_triggers.py as a SUBPROCESS and asserts on its exit semantics, so a pytest or PyYAML that resolves differently changes what it is asserting against. The capability ubuntu leg was DELETED, because it was already duplicated ------------------------------------------------------------------------ That workflow's own comment already explains why it has no macOS leg: every test in tests/test_export_capability.py goes through _patch(), which monkeypatches _has_torch, get_device and is_apple_silicon, so a real Mac proves nothing a Linux runner does not -- and studio-backend-ci.yml runs the same file on ubuntu-latest as part of `pytest tests/`. That argument reaches one step further than it was taken. If Backend CI covers the file on Linux, the ubuntu leg HERE is the duplicate too. Checked: the file is not in that job's --ignore list. The import-safety test does not need a torch-free image either; it installs its own builtins.__import__ blocker and drops preloaded torch/unsloth from sys.modules, so it proves the same thing inside Backend CI's fully installed environment. Windows stays. Nothing else in CI runs that file there, and _has_torch's import probe is the per-OS behaviour the job exists for. The guard ------------------------------------------------------------------------ tests/studio/test_short_job_absorption.py, wired into the unfiltered job. Both changes fail silently rather than loudly if they regress, which is what it is for: - the suite still runs somewhere, and no longer runs twice - the absorbing job's harden-runner has not widened past the six endpoints the suite came with, since "the policy is identical" is the entire justification for this host - the absorbing job has not gained a paths filter - capability still has its Windows leg - studio-backend-ci.yml still runs the whole tests/ tree and does not name test_export_capability.py, because one line added to that --ignore list would remove the coverage the ubuntu leg was deleted for, and nothing would turn red Mutation-tested, each failing exactly one test: drop tests/security from the invocation; add one endpoint to the allowlist; add --ignore=tests/test_export_capability.py to Backend CI; remove the Windows leg. Also corrected a docstring in tests/security/test_scan_packages.py that named tests-security and what it installs. It was about to become false. Verification ------------------------------------------------------------------------ tests/security under the host's own -n 4: 409 passed, 6 skipped. scripts/lint_workflow_triggers.py: OK across 41 workflow files. All three workflows still parse; security-audit.yml keeps its other 4 jobs. Net: 7 short ubuntu slots per commit, down to 5. A note for whoever extends this. The census that found these 15 candidates was partly stale and I nearly acted on it: the lockfile and load-orchestrator rows were already absorbed by #9176 and their samples were pre-merge tails, and the two Local Agent Guides rows show ~0s because they are if-gated to schedule and dispatch, which is a skip and not a fast job. Read the trigger before ranking by duration. Of the remaining candidates, Scorecard is blocked by its job-level id-token: write, the Kaggle gate by a downstream needs:, npm-provenance by an audit egress policy plus registry.npmjs.org, and the notransport clean-install lane by a container that asserts several common tools are absent.
114 lines
5.4 KiB
YAML
114 lines
5.4 KiB
YAML
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
|
|
|
|
# Runs studio/backend/tests/test_export_capability.py on Linux, Windows and macOS.
|
|
#
|
|
# export_capability() is per-OS (is_apple_silicon() and the PyTorch-import probe differ per
|
|
# platform) and the export backend must import without PyTorch, so this confirms the gating and
|
|
# import-safety on hosted Windows/macOS. Hosted runners have no GPU/MLX, so a real accelerator
|
|
# export is validated separately. No GPU / model / llama.cpp: the tests mock the probes and block
|
|
# torch/unsloth, so the job installs only a CPU PyTorch plus import deps.
|
|
|
|
name: Unsloth export capability
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'studio/backend/utils/hardware/hardware.py'
|
|
- 'studio/backend/core/export/export.py'
|
|
- 'studio/backend/routes/export.py'
|
|
- 'studio/backend/main.py'
|
|
- 'studio/backend/tests/test_export_capability.py'
|
|
- '.github/workflows/studio-export-capability-ci.yml'
|
|
push:
|
|
branches: [main]
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.ref == 'refs/heads/main' && github.sha || '' }}
|
|
# Latest-only on a PR branch. On main this does less than it reads like: it stops
|
|
# a RUNNING main job being killed, but GitHub cancels any PENDING run in the group
|
|
# the moment a newer one is queued, so a merge burst still leaves only the tip.
|
|
# See studio-backend-ci.yml, which is grouped per commit on main for that reason.
|
|
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
capability:
|
|
name: capability (${{ matrix.os }})
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
# Windows only, and the reason is the same one that already removed the
|
|
# macOS leg: the only command this job runs is `pytest
|
|
# tests/test_export_capability.py`, and every test in that file goes
|
|
# through `_patch()`, which monkeypatches `_has_torch`, `get_device` and
|
|
# `is_apple_silicon`. Nothing in it touches Metal, MLX or any Apple
|
|
# binary, so a real Mac proved nothing a Linux runner did not.
|
|
#
|
|
# That argument reaches one step further than it was taken. It rests on
|
|
# studio-backend-ci.yml running the same file on ubuntu-latest as part of
|
|
# `pytest tests/` -- which it does, the file is not in that job's
|
|
# --ignore list -- and if that is true then the ubuntu leg HERE is the
|
|
# duplicate, not just the macOS one. The import-safety test does not need
|
|
# a torch-free image to be honest either: it installs its own
|
|
# `builtins.__import__` blocker and drops preloaded torch/unsloth from
|
|
# sys.modules, so it proves the same thing inside Backend CI's fully
|
|
# installed environment.
|
|
#
|
|
# Windows stays, because nothing else in CI runs this file there and
|
|
# `_has_torch`'s import probe is the per-OS behaviour the job exists for.
|
|
# Measured 67s of execution behind a 10642s queue, so the ubuntu leg was
|
|
# a runner slot spent re-proving a Linux result.
|
|
os: [windows-latest]
|
|
runs-on: ${{ matrix.os }}
|
|
timeout-minutes: 20
|
|
env:
|
|
# No accelerator on hosted runners; keep detection on the CPU path.
|
|
CUDA_VISIBLE_DEVICES: ""
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
persist-credentials: false
|
|
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
|
# This job pins its dependencies inline below rather than in a requirements
|
|
# file, so the workflow IS the dependency spec and hashing it is what makes the
|
|
# key describe the payload. Unscoped, setup-python hashes dependency files
|
|
# repo-wide, so one unrelated edit invalidates ~700MB per interpreter.
|
|
with:
|
|
python-version: '3.12'
|
|
|
|
- name: Restore the pip cache
|
|
id: pip-cache
|
|
uses: ./.github/actions/pip-cache-restore
|
|
with:
|
|
key-files: |
|
|
.github/workflows/studio-export-capability-ci.yml
|
|
|
|
- name: Upgrade pip
|
|
run: python -m pip install --upgrade pip
|
|
- name: Install CPU PyTorch
|
|
# CPU wheel index so every OS gets a CPU build; keep PyPI as an extra index so torch's
|
|
# transitive deps still resolve (matching the other workflows in this repo).
|
|
run: python -m pip install --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple "torch>=2.4,<2.13"
|
|
- name: Install backend import deps
|
|
# Enough to import utils.hardware and core.export.export; NOT unsloth (needs a GPU, and
|
|
# the import-safety test blocks it) or triton/llama.cpp (Linux-only / native builds).
|
|
run: python -m pip install
|
|
transformers peft accelerate safetensors huggingface_hub datasets
|
|
sentencepiece protobuf fastapi starlette structlog psutil
|
|
python-multipart pydantic httpx "numpy<3" pytest
|
|
- name: Export capability + import-safety tests
|
|
working-directory: studio/backend
|
|
run: python -m pytest tests/test_export_capability.py -q
|
|
|
|
- name: Save the pip cache
|
|
if: always()
|
|
uses: ./.github/actions/pip-cache-save
|
|
with:
|
|
dir: ${{ steps.pip-cache.outputs.dir }}
|
|
key: ${{ steps.pip-cache.outputs.key }}
|
|
cache-hit: ${{ steps.pip-cache.outputs.cache-hit }}
|
|
|