mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-25 08:42:25 +00:00
90a6a236b5
* lint_workflow_triggers: scan .yaml workflows too, and pin security-audit to every PR GitHub Actions loads both .yml and .yaml out of .github/workflows/, but the trigger lint only globbed *.yml, so an evil.yaml carrying pull_request_target would run for real and still lint clean. Also adds a regression test asserting security-audit.yml's pull_request trigger has no paths filter, since the lint job lives inside that workflow. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Host the workflow-trigger lint in its own unfiltered workflow The lint job lived inside security-audit.yml and inherited its triggers, so the gate was only ever as broad as an unrelated heavy audit workflow's filter policy. Move it to workflow-trigger-lint.yml, which triggers on pull_request with no paths filter. pull_request resolves the workflow file from the PR merge ref, so a PR that adds a paths or paths-ignore filter to the host skips the host for its own PR and the gate never reviews the change. A pytest assertion cannot catch that, because the test runs inside the workflow being skipped. The lint now checks its own host instead: it rejects both filter keys on any workflow that runs the script, and fails when no unfiltered host exists at all. That still leaves the tampering PR itself, so CODEOWNERS now covers .github/workflows/ and CODEOWNERS itself, which is the merge-time control. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Detect the lint host from parsed steps, and check effective CODEOWNERS Host detection matched a regex against the raw workflow text, so a commented out '# - run: python3 scripts/lint_workflow_triggers.py' registered as a host. That defeats the fail-closed check: delete the real workflow, leave a comment behind, and --require-host still passes. Read the parsed jobs/steps instead, which also covers multi-line run blocks. The CODEOWNERS guard asserted a matching rule existed somewhere in the file. GitHub applies only the LAST matching pattern, so appending '* @someone-else' would take over while the guard stayed green. Resolve the effective owners for a workflow path and for CODEOWNERS itself, and add a test that the guard fails when a broader rule is appended. * Require the lint host to be unnarrowed and blocking, and check every workflow's owner The host check only rejected paths and paths-ignore. branches, branches-ignore and types skip PRs just as effectively, so a host restricted to another branch counted as unfiltered. Require a bare pull_request: with no configuration at all, which covers those keys and any future one. A host whose lint step or job carries continue-on-error runs but cannot fail, so --require-host passed while findings were advisory. Reject that too. The CODEOWNERS guard probed only the lint host and CODEOWNERS itself, so a narrower trailing rule could take a different workflow away from its owner while both probes still passed. Check every workflow file has some effective owner, keeping the danielhanchen requirement for the two that matter most. Delegating a workflow to another maintainer stays fine; leaving one unowned does not. The rule parser also skipped ownerless lines, which are valid CODEOWNERS and clear ownership, so a bare pattern was an invisible carve-out. * Reject an if-gated lint host, and glob CODEOWNERS directory patterns An if: condition on the lint step or its job skips the lint while the run still succeeds, which is the same defeat as continue-on-error by another key. Reject any if: on a host rather than trying to prove one always true. The CODEOWNERS matcher compared trailing-slash patterns as literal substrings, so a valid trailing rule like '**/workflows/ @someone-else' took the lint host away from its owner while the guard still computed danielhanchen. Match every directory prefix with fnmatch, allowing unanchored patterns to start at any depth, and cover globbed, unanchored and wildcard-segment rules in the regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Require a real lint invocation, reject needs:, and tighten owner matching Host detection accepted any run text containing the script name, so 'echo scripts/lint_workflow_triggers.py' counted as running the gate. Require the script to appear as the argument of a python interpreter at the start of a command. A lint job with needs: is skipped when its prerequisite is skipped, and the workflow still succeeds. Reject needs: on a host, same as if: and continue-on-error. CODEOWNERS wildmatch lets '**/' match zero directories, so '/.github/**/workflows/' overrides the host while fnmatch returned false. Expand '**/' into both forms before matching. Owner tokens are also validated: GitHub cannot request review from a bare word, so a trailing rule naming 'not-an-owner' leaves the path effectively unowned and no longer counts. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject a defanged lint invocation, and match CODEOWNERS globs per segment A host running 'lint_workflow_triggers.py || true' passed every check: it is a real invocation, the trigger is unfiltered, and no metadata flags it, yet findings can never fail the run. Same for a command passing --workflows-dir at somewhere empty, or --no-require-host, which leaves the job green while it gates nothing. Reject failure-masking shell and both neutering flags. The CODEOWNERS matcher used fnmatch, whose '*' consumes '/'. That made '/.github/*' appear to claim nested workflow files, so a valid CODEOWNERS change touching only direct children of .github would have failed the guard. Translate patterns to a regex instead, where '*' stops at a separator, '**' crosses them and '**/' may match zero directories, and only expand directory prefixes for patterns without a wildcard. This replaces the '**/' expansion workaround, so the helper is now wrong in neither direction. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Require python to execute the lint file, and anchor slashed CODEOWNERS patterns 'python3 -c pass scripts/lint_workflow_triggers.py' names the script but runs the -c program and exits 0. Confirmed directly: rc=0, nothing scanned. The old regex allowed arbitrary arguments before the basename, so that counted as a host. Tokenize the command instead and require the script to be python's executed file, walking option flags but stopping at -c or -m. CODEOWNERS anchoring followed gitignore only for a leading slash. A pattern with an internal separator is root-relative too, so 'workflows/lint.yml' was being tried at every depth and appeared to override '.github/workflows/lint.yml'. That failed valid CODEOWNERS changes aimed at a top-level workflows/ directory. A bare 'workflows/' still floats to any depth. Also pins that ordinary invocations keep working: -u, -X with a value, and plain 'python'. * Require the host to run the repo lint as a plain command with no arguments Three separate holes had one shape, so this replaces the growing list of special cases with a single rule: a host runs scripts/lint_workflow_triggers.py as a standalone command, with no arguments. That covers a decoy /tmp/lint_workflow_triggers.py sharing the basename; a pipeline or background job, where the step's exit status need not be the lint's, since the default run: shell is bash -e with no pipefail; and every argument, including --help, which exits 0 before scanning, and the abbreviated --workflows-d that slipped past the old substring denylist. The _MASKED regex and the NEUTERING_FLAGS list are both gone. argparse also now runs with allow_abbrev=False, so --workflows-d is rejected by the script itself rather than only by the host check. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the lint path exactly, require a lone command, and check the step shell Three more ways a host could look wired while running nothing. The path was a suffix match, so a decoy at /tmp/scripts/lint_workflow_triggers.py passed. Require the exact repo-relative token, or the ./ form. Judging lines in isolation cannot tell a call from a definition, so an invocation parked in an uncalled function or a here-document read as enforcing. The step body must now be the lint command and nothing else, which sidesteps shell parsing entirely and makes the separate set +e check redundant. A custom shell template such as bash -c '"{0}" || true' wraps the command and drops its exit status while the run line stays plain. Only bash and sh count, resolved through step, job defaults.run and workflow defaults.run. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Classify publishers by stem, and tighten what counts as running the lint Scanning .yaml made PUBLISH_WORKFLOW_NAMES inconsistent: a rename to release-desktop.yaml would be loaded but no longer classified as a publisher, so a cache key shared with a PR workflow stopped being a finding. Match on the stem instead. This one was introduced by the .yaml change in this PR. Three ways the interpreter was accepted without running the file. The regex matched any command containing python, so /tmp/fakepython passed; require the BASENAME to be a python, keeping any directory prefix. -V, --version, -h and --help before the path make python print and exit 0, confirmed directly, so they now disqualify like -c and -m already did. And working-directory resolves the same plain command to a different file, so a host setting it on the step, the job defaults or the workflow defaults is rejected, alongside the existing shell check. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Allowlist interpreter flags, reject redirecting env, validate the pull_request value Interpreter flags are now an allowlist rather than a denylist. Each round of review found another flag that stops the file running or masks its status, and -i was the latest: it enters the REPL after the script, so on EOF the process exits 0 even though the lint called sys.exit(1). Confirmed directly, rc=0. Only flags that leave run-this-file-and-return-its-status intact are accepted, so an unrecognised flag fails closed instead of needing to be enumerated. BASH_ENV, ENV and PATH redirect the step without the command text changing at all: non-interactive bash sources BASH_ENV before the step script, so an exit 0 there ends the step before the lint runs. Rejected at workflow, job and step scope. pull_request: false was read as unrestricted, because only mappings were inspected. GitHub rejects a non-mapping event configuration and will not load the workflow, so the value must now be bare or a mapping. * Tighten comments on the workflow-trigger lint Comment and docstring wording only, no behaviour change. The blocks grew a clause per review round; this keeps the reason each rule exists and drops the retelling. * Stop trusting PR-controlled interpreters, option values and startup vars Three variants of one mistake: the host check trusted content the PR itself can supply. A basename-only interpreter check accepted ./python3, which a PR can add to the repository root. Require a bare command or an absolute system path. An allowlisted value-taking option had its value consumed unchecked, so a command substitution in it would run before python started. Option values are now rejected for shell syntax like trailing arguments already were. PYTHONPATH, PYTHONHOME and PYTHONSTARTUP join BASH_ENV, ENV and PATH: a sitecustomize.py on PYTHONPATH is imported before the script and can exit 0. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject expansions in the interpreter token and containerized host jobs The trusted-path check read the token before bash expands it, so "/usr/$(...)/python3" passed on basename and prefix while the substitution ran first. Reject shell syntax there, as option values and trailing arguments already were. A job with container: runs its steps in a PR-selected image that controls the shell and environment, which the workflow-level env merge cannot see. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
68 lines
2.9 KiB
Text
68 lines
2.9 KiB
Text
# Inspired from https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS
|
|
|
|
/unsloth/models/loader.py @danielhanchen @mmathew23
|
|
/unsloth/models/llama.py @Datta0 @danielhanchen @mmathew23
|
|
/unsloth/models/rl.py @Datta0 @pluesclues @danielhanchen
|
|
/unsloth/models/rl_replacements.py @Datta0 @pluesclues @danielhanchen
|
|
/unsloth/trainer.py @danielhanchen
|
|
/unsloth/models/sentence_transformer.py @Etherll @danielhanchen
|
|
/unsloth/save.py @danielhanchen
|
|
/unsloth/tokenizer_utils.py @mmathew23 @danielhanchen
|
|
/unsloth/chat_templates.py @danielhanchen
|
|
/unsloth/ollama_template_mappers.py @danielhanchen
|
|
/unsloth/kernels/moe/*.py @Datta0
|
|
/unsloth/import_fixes.py @danielhanchen
|
|
/unsloth/device_type.py @danielhanchen
|
|
/unsloth/_auto_install.py @danielhanchen
|
|
/unsloth/dataprep/*.py @danielhanchen
|
|
/unsloth/kernels/cross_entropy_loss.py @danielhanchen
|
|
/unsloth/kernels/fast_lora.py @danielhanchen
|
|
/unsloth/kernels/flex_attention.py @danielhanchen
|
|
/unsloth/kernels/fp8.py @Datta0
|
|
/unsloth/kernels/geglu.py @danielhanchen
|
|
/unsloth/kernels/layernorm.py @danielhanchen
|
|
/unsloth/kernels/rms_layernorm.py @danielhanchen
|
|
/unsloth/kernels/rope_embedding.py @danielhanchen
|
|
/unsloth/kernels/swiglu.py @danielhanchen
|
|
/unsloth/kernels/utils.py @danielhanchen @Datta0
|
|
/unsloth/models/_utils.py @danielhanchen @mmathew23
|
|
/unsloth/models/cohere.py @danielhanchen
|
|
/unsloth/models/dpo.py @danielhanchen
|
|
/unsloth/models/falcon_h1.py @danielhanchen
|
|
/unsloth/models/gemma.py @danielhanchen
|
|
/unsloth/models/gemma2.py @danielhanchen
|
|
/unsloth/models/glm4_moe.py @Datta0
|
|
/unsloth/models/granite.py @danielhanchen
|
|
/unsloth/models/llama4.py @danielhanchen
|
|
/unsloth/models/loader_utils.py @Datta0 @danielhanchen
|
|
/unsloth/models/mapper.py @danielhanchen
|
|
/unsloth/models/mistral.py @danielhanchen
|
|
/unsloth/models/qwen2.py @danielhanchen
|
|
/unsloth/models/qwen3.py @Datta0
|
|
/unsloth/models/qwen3_moe.py @Datta0
|
|
/unsloth/models/vision.py @mmathew23 @danielhanchen
|
|
/unsloth/utils/attention_dispatch.py @mmathew23
|
|
/unsloth/utils/hf_hub.py @mmathew23
|
|
/unsloth/utils/packing.py @mmathew23
|
|
|
|
/cli/ @Manan17
|
|
/studio/frontend/ @Shine1i @Manan17
|
|
/studio/frontend/public/ @Shine1i
|
|
/studio/backend/
|
|
/studio/backend/core/data_recipe/
|
|
/studio/backend/tests/ @danielhanchen
|
|
/tests/ @danielhanchen
|
|
/scripts/ @danielhanchen
|
|
|
|
# CI definitions. A workflow change can hand a fork PR the base repo's
|
|
# secrets, or skip the very lint that would have caught it, and no CI gate
|
|
# can stop a PR that disables its own gate. Owner review is the control.
|
|
/.github/workflows/ @danielhanchen
|
|
/.github/CODEOWNERS @danielhanchen
|
|
|
|
# Snapshot data for the notebook linter / Colab oracle. Drift in these
|
|
# files changes the pin floor for every Unsloth notebook, so refreshes
|
|
# must be reviewed by the notebook owners directly. CODEOWNERS later
|
|
# wins, so this overrides the broader /scripts/ rule above.
|
|
/scripts/data/colab_*.txt @danielhanchen @shimmyshimmer
|
|
/scripts/data/colab_*.json @danielhanchen @shimmyshimmer
|