unsloth/scripts
Daniel Han 73d9653d5b
scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed (#6552)
* scan_packages: key baseline on matched-code hash

The baseline matched on (package, package-relative file, check), which
excluded the matched code, so a future finding of the same check in the
same file was suppressed regardless of what the code did. A malicious
future version of an already-baselined package could place a payload in
the same file under the same check and pass the enforcing gate.

Key the baseline on a hash of the matched code too. The hash is over the
deduped, sorted set of matched spans with L<NN>: line markers stripped, so
version bumps, line shifts and match reordering stay stable while new or
changed flagged code reopens the finding. Version is left out of the key so
routine dependency bumps do not reopen every entry. The hash is capped and
recomputable from the stored evidence.

Regenerate scan_packages_baseline.json against the current dependency set;
the hf-stack, studio and extras scan shards pass enforcing (no active
CRITICAL or HIGH).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan_packages: refresh baseline for newer unsloth-zoo release

A newer unsloth-zoo published after the first regenerate added
tests/test_mlx_save_export_regressions.py, a benign test fixture
(temporary_location="/tmp/ignored") that trips the /tmp dropper check.
Regenerate the hf-stack shard against the current set so the entry is
allowlisted; studio and extras are unchanged.

* scan_packages: harden baseline loading against malformed JSON

Guard against a non-dict top-level baseline and non-dict entries so a
corrupt or hand-edited allowlist warns and fails closed instead of
crashing with AttributeError, and treat an explicit evidence: null as
empty.

* scan_packages: hash the full match set, keep indentation, strip only the marker

Address the evidence-hash review feedback:
- Capture every matching line, not the first three, so a payload appended
  after existing matches in a baselined file and check reopens the finding
  instead of riding the sample.
- Preserve leading indentation so a flagged line moved out of a guarded block
  reads as changed.
- Strip only each span's prefix up to the first L<NN>: marker, so an L<NN>:
  inside the matched code is kept and a change to it reopens the finding.

Evidence and its hash are stored in full and stay recomputable from the stored
field. Regenerate the baseline; hf-stack, studio and extras pass enforcing with
no active CRITICAL or HIGH.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan_packages: bind baseline evidence to full matched code

Address review feedback on the evidence-hash baseline key:

- Split evidence only on real span delimiters (" | " before an L<NN>:
  marker, or a newline), so a bitwise-or or union type in matched code
  is no longer split apart into separate spans.
- Record matched lines in full (drop the 160-char per-line cap) and
  record every distinct multiline match, so code appended past the cap
  or a second cross-line match reopens the finding instead of riding the
  first one.
- Give the large-JS-bundle and .pth base64-blob findings a content
  digest instead of empty or prefix-only evidence, and record all .pth
  import lines, so a changed bundle, blob or import no longer inherits a
  baselined empty or truncated key.
- Warn when a loaded baseline has entries without evidence_hash so a
  legacy baseline is regenerated rather than silently degraded.

Regenerate scripts/scan_packages_baseline.json against the current dep
set and add regression tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan_packages: harden multiline and duplicate evidence handling

Follow-up hardening so the evidence hash tracks the full matched code:

- For DOTALL patterns that match across lines, record every line the match
  spans (not just the start line), so a change on a continuation line (the
  URL inside a baselined C2 loop, a swapped credential path) reopens the
  finding. A pathological greedy span is bounded to its head line plus a
  digest of the rest.
- Keep duplicate spans in the canonical evidence so a second identical
  matched line in a new code path changes the key instead of deduping away.
- Anchor the evidence prefix to strip only a genuine leading label or
  line-number marker, leaving a marker-like "L<NN>:" inside raw .pth code
  intact.
- Make the legacy-baseline warning explicit that entries without an
  evidence_hash reopen rather than suppress under a coarse key.

Regenerate scripts/scan_packages_baseline.json (same finding set; entries
for same-file repeated checks are now tracked separately) and add tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan_packages: bind every combo and large finding to its full content

Close the remaining asymmetric-evidence gaps so a changed payload cannot
ride a reviewed baseline entry:

- Digest a capped multiline span from the code without line markers, so a
  pure line shift stays stable while a continuation-line change reopens.
- Give the "Unusually large executable .pth" finding a content digest
  instead of keying on byte size and import-line count alone.
- Record both contributing signals for the JS credential+network stealer,
  the shell credential+network and persistence-hook combos, and the hidden
  network+exec docstring payload, so changing the network/exec side reopens.
- Allow punctuation in an evidence label prefix so a "network+exec:" label
  is stripped and line shifts do not change the key.

Regenerate scripts/scan_packages_baseline.json and add tests for each case.

* scan_packages: bind remaining Python combos; key npm baseline on evidence

Python scanner: the openssl+key, anti-analysis, DNS-exfil and base64+exec+blob
combos recorded only one contributing signal, so a changed payload on the other
side could ride a reviewed baseline entry. Each now binds every co-occurring
signal (and the blob is digested, since it can sit on a separate line from the
decode call).

npm scanner: scan_npm_packages.py keyed its allowlist on (package, path,
pattern) only, the same coarse-key bypass the Python scanner just closed. Add an
evidence hash to the key (schema v3, fail-closed on older baselines) and store
full evidence. The committed baseline stays empty by design.

Regenerate scripts/scan_packages_baseline.json and add tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan_npm_packages: bind full blob evidence and harden baseline loader

Follow-up on the npm evidence-hash key:

- _evidence now records every match and, when a snippet is truncated for
  display, appends a digest of the full match. The obfuscated-blob key was
  hashing only the truncated first-match snippet, so a changed payload tail or
  an appended blob in the same package/file/pattern could ride a reviewed entry.
- _load_baseline guards that the root is an object, entries is a list, and each
  entry is a dict before reading it, so a malformed baseline warns and fails
  closed instead of raising AttributeError.

Add tests for a changed blob tail reopening the key and for malformed entries.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan_packages: symmetric baseline-loader guards; bind npm outbound host context

- Python _load_baseline now rejects a non-list "entries" with a warning instead
  of raising TypeError, matching the npm loader.
- npm cred-surface-host (outbound) records the host with its URL path / fetch
  call / host config, so a changed outbound path, headers or body reopens the
  key rather than riding the bare host literal.

Add tests for both.

* scan_npm_packages: migrate v2 baselines and bind host-config outbound context

- _load_baseline now migrates schema v2 entries by recomputing the evidence
  hash from stored evidence (with a legacy warning), matching the Python
  loader, instead of discarding them; only pre-v2 basename schemas are rejected.
- The cred-surface-host (outbound) host-config branch now captures the whole
  line (path, headers, body), so a changed outbound payload on the same
  hostname line reopens the key instead of riding the bare host snippet.

Add tests for v2 migration and the host-config context binding.

* scan packages: bind PEM key bodies and npm windowed evidence to baseline keys

scan_packages: embedded-key findings now pin the full PEM block (BEGIN..END)
via a content digest, so a key body swapped under the same marker reopens the
finding instead of riding the unchanged BEGIN line. Single-line and DER keys
were already bound by their full matched line; marker-only references with no
END block (validation header lists) are unaffected, so the committed baseline
is unchanged.

scan_npm_packages: _evidence now digests the full containing line whenever the
shown snippet is only a window into it (short match on a long line, or a
truncated payload), so a changed payload tail outside the display window
reopens the key. The npm baseline is empty, so this changes no suppressions.

Adds regression tests for both cases.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan packages: bind multi-line evidence and every blob to baseline keys

_extract_evidence now extends each single-line match over its bracket
continuations, so a multi-line call binds its argument lines and a changed
URL or body on a continuation line reopens the key. After the per-line pass it
also records cross-line matches the scan cannot otherwise see (a DOTALL regex,
or a multi-line construct appended under a check that already had a one-line
match), so an appended multiline payload reopens instead of riding the key.

_blob_digest hashes every large base64 blob (not just the first) for the
base64+exec finding and the .pth large-blob finding, so an appended or swapped
second encoded payload reopens; single-blob files keep the same digest.

scan_npm_packages _evidence digests the full logical line (the matched line
plus its bracket-continuation lines), so a multi-line fetch's option and header
lines bind and a changed payload on a following line reopens the outbound key.

Regenerated the Python baseline: same package/file/check set, 24 entries pick
up the wider multi-line evidence. Adds regression tests for each case.

* scan packages: stop giant greedy spans from binding a whole-file digest

When a greedy DOTALL pattern (reverse shell socket...subprocess, C2 loop) has
its anchor tokens far apart, the match span covers the whole file. Digesting
that span bound thousands of unrelated lines, so the evidence hash drifted on
any edit between the anchors (a dependency bump reshuffling the file), which
made a baselined finding reopen on an upstream release. The multiline pass now
skips an oversized span when the per-line pass already bound the signal lines,
so the evidence is the stable matched lines; a genuinely appended multi-line
construct stays under the cap and is still recorded.

Regenerated the Python baseline against Python 3.12 (the version the scan CI
shards run) so the resolved dependency set matches CI. Same package/file/check
set. Adds a regression test.

* scan packages: tighten evidence binding (order, string brackets, span size)

Address review follow-ups on the evidence extraction:

- _canon_evidence keeps discovery (line) order instead of sorting. Line-shift
  stability already comes from stripping the L<NN>: markers, so order stays
  significant and reordering matched lines (a multi-line call's arguments)
  reopens the finding.
- _logical_line_end (Python) and _logical_line_text (npm) blank string literals
  before counting brackets, so a ) inside a string argument does not close the
  logical line early and drop later argument lines.
- The oversized-span skip now only drops a giant whole-file bridge (over 60
  lines); a genuinely appended multi-line construct is recorded so its payload
  reopens, rather than riding an existing one-line match.
- npm _logical_line_text binds the enclosing bracket group, so a host-config
  object whose { is on a prior line binds its path/headers/body lines.

Regenerated the Python baseline (Python 3.12, matching the scan CI shards):
same package/file/check set. Adds regression tests for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan npm packages: normalize and bound the logical-line digest

- _evidence whitespace-normalizes the logical line before digesting (matching
  _evidence_hash), so a formatter-only reindent of the bound continuation lines
  does not change the sha256 suffix and reopen an unchanged finding.
- _logical_line_text follows a bracket group to its close up to a hard 200-line
  cap (digest input only), so a config object longer than the backward window
  still binds its whole tail instead of silently truncating.

Adds regression tests. npm baseline is empty, so no regeneration is needed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan: cap single-line evidence and widen npm opener window

Cap each rendered evidence line at 200 chars in scan_packages.py: a long
or minified one-line file is shown as a bounded prefix plus a sha256 of the
full line, so a packed payload cannot dump unbounded content into the CI
logs or baseline while a change past the cutoff still changes the digest
and reopens the finding. Mirrors how the npm scanner bounds its snippets.

Widen the npm backward opener window (_MAX_CONT_LINES 12 to 200, symmetric
with the forward cap) so a host deep inside a large options object binds
the whole object, not just its own line; a changed path, header, or body on
any property reopens.

Regenerate the Python baseline with Python 3.12: only the protobuf
nspkg.pth and unsloth-zoo compiler.py evidence change, both from the new
line cap; the package/file/check key set is unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* scan: bind all host contexts, deep call continuations, far-back npm openers

Three fail-closed evidence gaps surfaced by review of the previous round.

scan_npm_packages.py: measure the forward bracket-group cap from the matched
line (idx + _MAX_GROUP_LINES) instead of the opener, so an opener found near
the widened backward limit no longer consumes the forward budget and drops
the path, headers, or body that follow the host.

scan_npm_packages.py: _outbound_host_evidence now records every outbound
context form for a host (URL, fetch-context, host-config), claiming each
non-overlapping match in form order, so a separate host-config request added
beside an already-baselined URL changes the evidence and reopens the key.
The common single-context case keeps its existing snippet.

scan_packages.py: follow a matched Python call over its continuations up to a
separate _MAX_CALL_LINES (40), decoupled from the 12-line display threshold,
so a multi-line requests.post( binds its whole argument list in the digest
and a changed body deep in the call reopens; bounded so a miscounted bracket
cannot swallow unrelated code. No baseline change: the current dependency set
has no matched call that closes between 13 and 40 lines, confirmed by a
Python 3.12 regenerate that produced a byte-identical baseline.

* scan: clamp npm depth, pin large bundles, follow backslash and bound .pth dump

Four fail-closed evidence gaps surfaced by review of the previous round.

scan_npm_packages.py: clamp the backward opener scan at depth 0 so a leading
unmatched closer (a preceding block whose opener is outside the backward
window) no longer drives depth negative and masks the real enclosing opener
that follows; a host-config object after such a block now binds and a changed
path reopens.

scan_packages.py: a large JS bundle now pins its whole content even when
another JS heuristic already fired. The bundle digest was only added when no
other finding existed; it is now appended to every finding's evidence on a
large bundle, so an unchanged obfuscation signature no longer lets changed
payload elsewhere ride the matched-line key.

scan_packages.py: _logical_line_end follows explicit backslash line
continuations, so a call split with a backslash before its parenthesis binds
the continuation line (URL/body) instead of returning at the zero-depth API
line.

scan_packages.py: the catch-all .pth import evidence is bounded through
_cap_line (prefix plus a digest of every line) so a large .pth of benign
imports cannot dump the whole member into the logs or baseline while an
appended or swapped import still reopens.

Baseline regenerated with Python 3.12: key set unchanged; one entry
(unsloth-zoo compiler.py) gains the backslash-continued banner lines now
bound by the continuation fix.

* scan: handle multi-line strings, lifecycle bodies, and de-quadratic evidence

Addresses a review round plus a performance audit of the evidence extractor.

Correctness (fail-closed):
- Bind the UNION of the single-line-blanked and multi-line-blanked bracket spans
  in both scanners. The multi-line view blanks a triple-quoted Python string or a
  backtick template literal that spans lines, so a `)` inside such a string no
  longer closes the enclosing call early and drop later arguments. The single-line
  view still counts a payload embedded INSIDE a string, so a dropper that hides a
  call in a string keeps its argument lines bound. Taking the larger span never
  shrinks the binding below either view, avoiding a fail-open regression.
- cred-env-in-lifecycle now pins the whole lifecycle script body via a digest, so
  a changed non-token line (e.g. adding a curl exfil beside the token reference)
  reopens, not just a change on the token line.

Performance / DoS (the scanner runs on attacker-controlled package files up to the
64 MiB / 16 MiB member caps, with no per-file time budget):
- _extract_evidence precomputes newline offsets once and maps match offsets with
  bisect, removing the O(matches) whole-file content.count per match that made the
  finditer fallback quadratic (a crafted minified file went from ~13 s/MiB and
  hours at the cap to linear).
- npm _index_text splits and string-blanks the file once per evidence call instead
  of per match (was O(matches x file) time and allocation).
- Bound evidence output: _MAX_EVIDENCE_SPANS (Python) and _MAX_EVIDENCE_MATCHES
  (npm) fold the remainder into a digest so a file with thousands of matches cannot
  build a multi-megabyte evidence/baseline blob while an added/removed match past
  the cap still changes the key.
- _outbound_host_evidence caps matches per form and bounds the overlap claim so a
  host repeated many times cannot make it quadratic.

No baseline change: a Python 3.12 regenerate is byte-identical (the union equals the
legacy single-line span for every current dependency file; the cap thresholds sit
above the largest real entry), so these are forward-looking hardening with no drift.

* scan: count all overflow matches, bind their context, blank JS regex literals

Follow-ups on the evidence output caps from the previous commit.

- _outbound_host_evidence no longer truncates each pattern's match iterator with
  islice; it iterates every match and runs the overlap dedup only while the
  display list is below the cap (so claimed stays bounded and the check is O(cap)
  per match, not quadratic), folding every match past the cap into the overflow
  digest. A host context beyond the 64th is counted again, so it reopens.
- The overflow digest (both scanners, via a shared _overflow_digest) binds each
  overflow match's logical-line context, not just the regex match text, so a
  changed payload on an over-cap line reopens even with the matched token
  unchanged.
- The multi-line JS blanked view now blanks regex-literal bodies (tracking the
  previous significant char for regex-vs-division and char classes for a literal
  `/` inside `[...]`), so a `)` inside `/)/` no longer closes an outbound call
  early. The bound span is the union of the single-line and multi-line views, so
  an imperfect regex decision only ever grows the span, never shrinks it.
- The Python overflow digest canonicalizes spans (strips L<NN>: markers via
  _canon_evidence) before hashing, restoring line-shift stability for the
  over-cap region.

No baseline change: the overflow branches only trigger above the per-finding caps
(above the largest real entry), and the npm baseline is empty, so a Python 3.12
regenerate is byte-identical.

* scan: refresh baseline for ipython interactiveshell.py span drift

A newer ipython release changed the filesystem-enumeration span in
IPython/core/interactiveshell.py, so its content digest no longer matched the
baselined evidence and the studio scan shard flagged it as a non-baselined
CRITICAL. Regenerated with Python 3.12: only the ipython entry's evidence_hash
changes; the package/file/check key set is unchanged, and a studio enforcing
spot-check exits 0.

* Bound scanner evidence memory: stream overflow spans and cap lifecycle baseline size

scan_packages.py: _extract_evidence no longer materializes a rendered span
per match before slicing at the display cap. Once out holds _MAX_EVIDENCE_SPANS
spans, further spans fold straight into a running digest, so a minified or
padded file with hundreds of thousands of matching lines keeps memory bounded
to the display cap instead of the match count. The fold reproduces
_canon_evidence(" | ".join(overflow)) byte for byte, so the overflow digest and
every baseline key are unchanged.

scan_npm_packages.py: lifecycle-fetch-exec and cred-path-in-lifecycle stored the
entire install script body as evidence, so --write-baseline on a package with a
multi-MiB lifecycle script bloated the baseline JSON. Both now store a bounded
matched snippet plus a body-sha256 digest, matching cred-env-in-lifecycle. The
digest still binds the whole body, so a change to any line reopens the finding.

Adds tests for the streamed overflow bound and the bounded-but-reopens lifecycle
evidence. Baseline unchanged (byte-identical Python evidence; npm baseline empty).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make npm bracket-group scan order-aware so a same-line close-then-open binds

_scan_group counted brackets with a per-line net (opens minus closes), which
collapses intra-line order: a line that closes a prior block and then opens the
host-config object on the same line, e.g. `}); const opts = {`, nets to <= 0, so
the trailing `{` was dropped and the group started at the hostname line. A
changed path/headers on the following lines then hashed to the same evidence and
could ride an existing baseline key.

Replace the net count with an order-aware (L, R) reduction per line (L closers
needing an opener to the left, R openers needing a closer to the right) and apply
it in order in both the backward and forward scans, clamping stray closers at 0.
The trailing opener now stays visible so the whole object binds and a changed
payload reopens. Per-line cost is unchanged (one C-level bracket findall), so the
existing outbound-host evidence is byte-identical on all prior shapes; only the
previously-dropped same-line case changes. Adds a regression test for it.

* Harden scanner evidence: bound memory and bind Python call tails fail-closed

Five fixes across both scanners, none of which change the committed baseline (a
full regen of all three pip shards produced a byte-identical 185-key set).

scan_npm_packages.py: _evidence and _outbound_host_evidence collected every regex
match into a list before applying the 64-match display cap, so a text file under
the size cap that repeats a cheap signal (such as NPM_TOKEN) millions of times
could allocate a huge list of re.Match objects and stall or OOM before the
overflow digest ran. They now stream from finditer and fold overflow as matches
arrive via a shared _fold_overflow_match helper, byte-identical to the prior
digest.

scan_packages.py:
- _extract_evidence kept inserting every unique over-cap span into the seen set
  even after it stopped appending to the display list, so a generated file with
  millions of one-line matches still grew that set unbounded. It now tracks spans
  only while filling the display list (per-line spans are unique by line number,
  so dropping them past the cap cannot miss a dedup).
- _scan_line_end counted brackets with a per-line net, so a continued statement
  that closes on the same line it opens a flagged call (a leading "]" before
  "requests.post(") had the call's open paren cancelled and bound only the opener
  line. It now applies brackets in order via _bracket_lr (leading closers clamp at
  0), matching the npm bracket fix.
- a single-quoted string continued by a trailing backslash was not tracked across
  lines, so a close paren inside the continued string on the next line closed the
  call early; _blank_code_strings now carries the continuation.
- a call with more argument lines than the soft cap was hashed only through the
  cap, so a changed data=/headers tail past it stayed suppressed; a closing call
  is now followed to its real close under a 200-line hard limit (a never-closing
  opener still stops at the 40-line soft cap so it cannot swallow the file).

Adds regression tests for each. npm baseline is empty; the Python baseline is
unchanged (verified byte-identical by regenerating all three shards).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bind giant DOTALL span anchors and add context to constant IOC evidence

Two fail-closed gaps where a changed payload could keep the same evidence hash
and stay suppressed by the baseline.

scan_packages.py: a giant greedy DOTALL span (a cross-line IOC match bridging
more than 60 lines, e.g. RE_TEMP_EXEC matching a /tmp line and a much-later
subprocess line) was dropped entirely once the per-line pass had any match, so an
appended cross-line payload -- a new /tmp line plus a later subprocess line that
share no single line, so the per-line pass never binds them -- produced the same
evidence and rode the key. The span is no longer dropped: it is bound by its head
and tail anchor lines plus a digest over just those (no line numbers, so a pure
line shift is stable). An added or moved anchor reopens the finding, while churn
in the bridged interior stays stable, so this does not reintroduce whole-file
drift. Two baseline entries (multiprocess test, unsloth-zoo scanner file) carry
such a span and are refreshed; a full three-shard regen confirmed only those two
keys change.

scan_npm_packages.py: known-ioc-string and cred-surface-host (always-bad) recorded
only the bare needle/host as evidence, so a reviewed tarball that kept the IOC
string while altering the adjacent fetch/exfil body produced an identical key.
They now bind matched-line context: known-ioc-string via the matched line and its
bracket-group continuation, cred-surface-host (always-bad) via the outbound call
context (path/headers/body, falling back to the bare host when not in an outbound
call). A changed payload on the same call now reopens.

Adds regression tests for each. npm baseline is empty; the Python baseline updates
only the two giant-span entries.

* Hash giant-span interiors, bind exec/eval trigger, JS content, intra-literal whitespace

Four fail-closed gaps where a changed payload could keep the same evidence hash.

scan_packages.py:
- A giant bridged DOTALL span was bound only by its head and tail anchors, so a
  cross-line payload inserted into the bridged interior between unchanged outer
  anchors kept the same key. The whole span content is now digested (via _render),
  so any interior change reopens; a pure line shift stays stable because the digest
  is over the markerless code. Two baseline entries (multiprocess test, unsloth-zoo
  scanner file) carry such a span; with full-interior binding, multiprocess
  resolved at two versions across shards now yields two distinct entries where the
  anchor digest had collapsed them into one.
- The exec/eval-with-hidden-payload findings omitted the visible exec/eval line
  that makes the hidden string executable, so flipping a harmless eval("1+1") to
  exec(__doc__) kept the same key while arming the payload. The trigger line from
  the real-code view is now bound into the evidence.
- check_js_file extracted evidence with the Python-string-aware extractor, which
  does not blank JS backtick template literals, so a template containing a close
  paren closed a call's bracket span early and omitted later option/body lines. The
  full file content digest is now pinned to every JS finding (not just large
  bundles), binding the whole call.

scan_npm_packages.py: the evidence canon collapsed all whitespace via split(),
erasing whitespace inside JS string literals along with harmless indentation, so a
changed request body 'a b' -> 'a  b' kept the same key. A new _canon_preserve_strings
collapses whitespace only OUTSIDE string literals (reindent-stable) while preserving
it INSIDE single/double/backtick literals (intra-payload edits reopen). Used for the
evidence hash and the logical-line digests.

Adds regression tests for each. npm baseline is empty; the Python baseline updates
the two giant-span entries and adds the second multiprocess version's entry.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-01 04:03:59 -07:00
..
data CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
check_frontend_dep_removal.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
check_new_install_scripts.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
enforce_kwargs_spacing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
install_gemma4_mlx.sh Update Install Scripts (#5968) 2026-06-03 05:39:42 -07:00
install_qwen3_6_mlx.sh Update Install Scripts (#5968) 2026-06-03 05:39:42 -07:00
install_rocm_wsl_strixhalo.sh Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
lint_workflow_triggers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
lockfile_supply_chain_audit.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
notebook_to_python.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
notebook_validator.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
run_ruff_format.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
scan_npm_packages.py scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed (#6552) 2026-07-01 04:03:59 -07:00
scan_npm_packages_baseline.json scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed (#6552) 2026-07-01 04:03:59 -07:00
scan_packages.py scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed (#6552) 2026-07-01 04:03:59 -07:00
scan_packages_baseline.json scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed (#6552) 2026-07-01 04:03:59 -07:00
stamp_studio_release.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
sync_allow_scripts_pins.py Studio: auto-sync allowScripts pins after dependency bumps (#6136) 2026-06-10 02:35:37 -07:00
uninstall.ps1 Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) 2026-06-22 03:09:08 -07:00
uninstall.sh Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) 2026-06-22 03:09:08 -07:00
verify_comment_only_diff.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
verify_import_hoist.py CI: fix import-hoist false positive, vision-cache test cwd, llama.cpp CLI smoke (#6598) 2026-06-23 01:16:47 -07:00