unsloth

mirror of https://github.com/unslothai/unsloth.git synced 2026-08-24 00:04:14 +00:00

History

Daniel Han af2fa437a2 Give each xdist worker its own torch.compile cache directory (#9134 ) Inductor's on-disk caches default to one directory per USER, not per process, so the four workers under pytest -n 4 share /tmp/torchinductor_<user> and its fxgraph, aotautograd and Triton subtrees. The upstream recipe is explicit that a common TORCHINDUCTOR_CACHE_DIR is what makes processes SHARE compiled artifacts, so a different value per worker is how they are kept apart. Sharing is not obviously wrong, since the entries are content-addressed. It is still a write-write interaction between processes that nothing here controls, and a cache is exactly the sort of thing that turns a deterministic suite into an intermittent one. This removes the interaction for the price of some recompilation. What it buys, measured rather than assumed, so nobody has to guess later: a full unsloth_zoo run against an empty dedicated cache directory took 578s and left ZERO entries in it. That suite never populates the on-disk cache, so for that step this is neither a saving nor a cost. It is insurance for the suites that do compile, applied wherever tests run in parallel rather than only where a problem has already been seen. It has to run before torch is imported, so it is a module-level import in the two conftests rather than a fixture. TRITON_CACHE_DIR is set explicitly rather than left to follow, because it only derives from the inductor directory when unset and an environment exporting it would keep all four workers on one Triton cache. An explicit TORCHINDUCTOR_CACHE_DIR is split underneath rather than replaced, so a CI path chosen on purpose is respected. A single-process run is left alone entirely. Verified under real xdist, not only in unit tests: four workers report four distinct directories, and the parallel suites still pass.	2026-08-17 18:48:11 -07:00
..
compile_cache_isolation.py	Give each xdist worker its own torch.compile cache directory (#9134 )	2026-08-17 18:48:11 -07:00

Give each xdist worker its own torch.compile cache directory (#9134 )

Inductor's on-disk caches default to one directory per USER, not per process, so
the four workers under pytest -n 4 share /tmp/torchinductor_<user> and its
fxgraph, aotautograd and Triton subtrees. The upstream recipe is explicit that a
common TORCHINDUCTOR_CACHE_DIR is what makes processes SHARE compiled artifacts,
so a different value per worker is how they are kept apart.

Sharing is not obviously wrong, since the entries are content-addressed. It is
still a write-write interaction between processes that nothing here controls, and
a cache is exactly the sort of thing that turns a deterministic suite into an
intermittent one. This removes the interaction for the price of some
recompilation.

What it buys, measured rather than assumed, so nobody has to guess later: a full
unsloth_zoo run against an empty dedicated cache directory took 578s and left
ZERO entries in it. That suite never populates the on-disk cache, so for that
step this is neither a saving nor a cost. It is insurance for the suites that do
compile, applied wherever tests run in parallel rather than only where a problem
has already been seen.

It has to run before torch is imported, so it is a module-level import in the two
conftests rather than a fixture. TRITON_CACHE_DIR is set explicitly rather than
left to follow, because it only derives from the inductor directory when unset
and an environment exporting it would keep all four workers on one Triton cache.
An explicit TORCHINDUCTOR_CACHE_DIR is split underneath rather than replaced, so
a CI path chosen on purpose is respected. A single-process run is left alone
entirely.

Verified under real xdist, not only in unit tests: four workers report four
distinct directories, and the parallel suites still pass.

2026-08-17 18:48:11 -07:00

compile_cache_isolation.py

Give each xdist worker its own torch.compile cache directory (#9134 )

2026-08-17 18:48:11 -07:00