mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-22 23:34:15 +00:00
7541 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
pre-commit-ci[bot]
|
2048b66300 |
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci |
||
shimmyshimmer
|
fb4d8a8460 |
default new installs to HTTPS downloads, and put the choice in Settings
The download transport preference defaulted to "auto", which resolves to Xet on any machine whose hf_xet looks healthy, so in practice everyone downloaded over Xet without ever choosing it. New installs now download over HTTPS: it resumes a cancelled or dropped transfer, and it needs nothing but ordinary TLS. Installs from before this default keep Auto, seeded once from positive evidence of prior use (a row the user wrote, or a manifest from an earlier download) and persisted, so updating changes nobody's downloads. Settings > General gains a Downloads section with an HTTPS / Xet / Auto control, the difference between the two spelled out, the reason Xet is unavailable when hf_xet is missing, and what Auto resolves to on this machine. use_xet defaulted to True in the download schemas, so a scripted caller that stated no transport got Xet whatever the preference said. It is optional now, and a request stating neither field follows the install's setting. |
||
Daniel Han
|
7266f1b29b
|
Stop a finished save's ingest watcher answering the next test's stub (#9200)
Reduced to the one fix of the original three that is still needed. The other two landed by other routes while this sat: the orphan-scan flake was fixed by #9280, and the extensionless `./mmproj-fallback` import is already `.ts` on main. Both are verified identical to main here rather than assumed. On #9280 taking a different approach: it pins `_pid_parent_is_alive` to a constant for the whole test, where this PR stubbed it only for the invented PID and delegated every other PID to the real implementation. That is a real difference in coverage, so I checked whether main lost anything before dropping mine. It did not: `test_reap_recorded_pid_spares_live_server` in test_llama_cpp_wait_for_vram_settle.py exercises the ownership gate for real, against a process the test genuinely owns. So the gate is still covered and #9280's version stands. What is left, and why it is still needed ------------------------------------------------------------------------ `saveMarkdownAsProjectSource` fires `watchIngestion` without awaiting it, and that watcher polls for up to 300 seconds. Watchers from earlier tests therefore outlive them and keep calling the shared fetch stub, which by then belongs to whichever test is running now. The stub answered any `/jobs/` URL and the uploads shared a filename, so a stale watcher emitted a toast byte-identical to the one the running test was waiting on, under a different project id. The test then waited on the toast and read the announce count without polling. The watcher toasts and announces with nothing awaited between them (save-markdown-source.ts, `toast.error(...)` then `announceProjectSourcesUpdated(...)`), so a tick landing in that window saw 1 instead of 2. Each job handler now answers only its own job id and 404s anything else, so a stale watcher dies quietly instead of borrowing the running test's answers. The save under test gets its own filename, and the test polls on the announce, which the watcher does last, then asserts the toast synchronously. Verified ------------------------------------------------------------------------ Gap sweep, inserting a 150 ms delay into the source between the toast and the announce to widen the window the race needs: main's version of the test fails 6 of 6 runs, this version passes 6 of 6. Both pass 6 of 6 on an idle box with no delay, which is why this reads as an intermittent CI failure rather than a reproducible one. Mutation-tested: deleting `announceProjectSourcesUpdated` from the watcher fails this test (9 passed, 1 failed), so it still proves what it did before rather than having been loosened into passing. Full frontend suite: 4087 passed, 0 failed. |
||
|
Daniel Han
|
6b2445bbb3
|
Drop the HunyuanImage GGUF the Hub no longer serves (#9369)
`Model catalog network check` is red on main:
::error::QuantStack/HunyuanImage-2.1-GGUF: HTTP 401 from
https://huggingface.co/api/models/QuantStack/HunyuanImage-2.1-GGUF
-- the repo is missing, renamed or private
model-catalog network check: 1 problem(s)
The check is right and the catalog is wrong, which is what that workflow says a
red run means: "A red run here means the catalog needs editing, not a re-run".
Confirmed against the Hub directly rather than trusting the annotation. The repo
answers 401 anonymously and 404 to an authenticated request, so it is gone rather
than gated or rate-limited. QuantStack itself is fine and still ships its other
GGUF repos (Wan2.2-I2V-A14B, Qwen-Image-Edit-2509, FLUX.1-Kontext-dev among
them), so this is one model withdrawn, not a publisher disappearing.
An entry the Hub cannot serve is worse than no entry: it renders as a one-click
download that fails partway through, which is exactly the case this workflow was
built to find.
No replacement, deliberately
------------------------------------------------------------------------
I looked before removing. unsloth has HunyuanImage-2.1-FP8, which ships no GGUF.
The third-party HunyuanImage GGUF repos on the Hub are a different lineage --
calcuis/hunyuanimage-gguf is 89 files of "lite" and "v2.0" variants, svjack's is
five split files with 39 downloads. Substituting one changes which weights users
download, and that is a product decision rather than a CI fix, so I have not made
it. Say the word and I will wire whichever you prefer.
What this costs
------------------------------------------------------------------------
The quant ladder, not the model. At 24 GB the group's budget is 61.6 GB and the
bf16 is 50 GB, so HunyuanImage 2.1 stays visible and routes to bf16 instead of
being hidden. The check file now asserts that at both ends, so a future change
that drops the group entirely cannot pass quietly; the old assertion pinned the
GGUF as the 24 GB pick and would simply have been deleted with it.
Left alone on purpose
------------------------------------------------------------------------
tests/test_diffusion_more_families.py still parametrises the dead repo id.
detect_family is pure string parsing with no network in it, and someone who
already downloaded that repo still needs their local copy to resolve to the
hunyuanimage-2.1 family. Removing it would drop coverage of the filename-alias
path (QuantStack drops the dash) for no gain.
The three remaining `canonicalId ... is not a real repo` warnings are unrelated
and correctly advisory: those ids are grouping keys that appear in no artifact
list, so by construction they cannot reach a load. The check computes them as
exactly the canonicalIds absent from artifactsByRepo, so that is structural, not
a promise.
Verified
------------------------------------------------------------------------
`npm run catalog:check --network`: "every declared repo, file and gated flag
agrees with the Hub", where it previously reported 1 problem.
Offline `npm run catalog:check`: all assertions passed.
Full frontend suite: 4080 passed, 0 failed.
|
||
|
Daniel Han
|
f40e20274c
|
Say which runtime components an AppImage is missing before its env hygiene (#9370)
`Desktop app clean machine` is red nightly on main, seven AppImage jobs, all with the app's own preflight naming four missing sonames. The tree is not at fault and neither is the test, so this changes only what the verifier says. What is actually happening ------------------------------------------------------------------------ The scheduled run downloads the published release rather than building one. `REL_TAG` is `v0.1.800-beta`, whose Linux asset was uploaded 2026-08-14T15:26Z and never rebuilt. I extracted it: `usr/bin/unsloth-studio` and zero `.so` files, so it resolves webkit2gtk, libsoup, javascriptcoregtk and appindicator from the host. It is a pre-#9113 thin build, provably: the diagnostic it prints lives at line 210 of `build-thin-appimage.sh`, a script #9113 deleted, and the replacement `appimage-apprun.sh` does not contain that string. #9113 ("Desktop: ship a complete Linux AppImage") landed 2026-08-19T13:09Z and also dropped the line that used to install the host runtime into the appimage row, which is correct under a self-contained bundle and is why that row flipped on an unchanged asset. The Aug 18 run was green, Aug 19's red was the ManagedStale incident that #9263 fixed, and Aug 20 is the first scheduled run after #9113. So the asset under test predates the packaging it is being tested against. The first desktop release cut from current main will satisfy these jobs; nothing in the tree needs changing to make that true, and until that release exists this workflow is red for a correct reason. Note this is invisible on a pull request by construction: on `pull_request` the AppImage jobs consume the artifact from `appimage-pr-build`, built from the PR, so #9113 was green when it merged. Only `schedule` and `push` reach the release. The one real defect ------------------------------------------------------------------------ `verify-complete-appimage.sh` run against that asset reports: Complete AppImage does not clear an inherited LD_LIBRARY_PATH It exits 1 correctly, but it fails at the launcher-hygiene check and never reaches the required-component loop, so a bundle that ships NO runtime at all is reported as an environment-variable bug. That is the difference between an hour of diagnosis and a minute of it. The required-component check now runs first and reports every miss instead of exiting on the first, mirroring the AppRun preflight, which names all four sonames at once rather than making the reader rediscover them one launch at a time. On the published asset it now says: Complete AppImage is missing required runtime component: libwebkit2gtk-4.1.so* ... 21 lines ... Complete AppImage is missing 21 of 21 required runtime components; it resolves them from the host Order only. No check is removed and no assertion is weakened. Verified ------------------------------------------------------------------------ `bash -n` clean. A synthetic AppDir with all 21 components present falls through to the launcher-hygiene check exactly as before, so the hygiene checks still run and still fail when they should. Removing only webkit from that AppDir yields `missing 1 of 21`, so the count is real rather than a formatting flourish. tests/security/test_release_desktop_appimage.py: 15 passed. |
||
|
Daniel Han
|
f68cbac1f0
|
Drain the sampling suites on the store's write chains, not on node's loader (#9367)
#9332 fixed these suites by waiting on two observables: the mocked timers, which it counts exactly, and the module loader, which it raced by issuing an import of its own each round. The second one works and is still the wrong thing to assert. The loader probe rests on an assumption about node internals that I recorded in the comment and never verified: that the hooks thread serves requests in order over one port, so a reply to a request issued after the store's cannot arrive first. If that ever stops holding it fails as a stale read, which is precisely the failure mode #9332 existed to abolish -- a wait that comes up short reports "chat A minP: owed 0, shows 0.01" and sends the next reader into the store. The store already tracks the thing worth waiting on. threadSettingsWriteChains holds the live promise chain per thread, and awaitThreadScopedSettingsWrite already awaits one of them by id. What a drain needs is all of the started ones, without knowing which chats the store decided to write, so this adds awaitStartedThreadScopedSettingsWrites. It is deliberately not a flush. A debounce that has not fired yet is left alone, so a caller cannot use it to make a write happen earlier than the store would have; that would let a test pass against a store that never scheduled the write at all. It is bounded at 20 passes rather than looping until the map empties, so a write that keeps rescheduling itself surfaces as a failed assertion instead of a hang. It repeats rather than awaiting one snapshot because a chain that settles can leave a newer one behind it for the same chat. drainMockedTimers now takes that as a caller-supplied `barrier` instead of carrying its own loader probe. The timer counting is unchanged, and so is the exhaustion throw, which is the property that keeps an under-drain from ever again looking like a lost edit. The approach is the one from #9352, which I opened against the same bug in parallel and is closed in favour of this. #9332's mechanism landed first and turned Windows green; this replaces the part of it that was a guess about the runtime with a measurement of the subject. Verified ------------------------------------------------------------------------ Both suites twice on v24.14.0 and twice on v22.23.2, the version setup-node 22 resolves to: compat 16 passed, simulation 18 passed, every run. Full frontend suite 4080 passed, 0 failed. Mutation-tested on v22.23.2, which is the only runtime where this is observable: disabling the barrier and leaving the timer counting in place gives 8 failed, 10 passed. So the barrier is carrying the load, not decorating a wait that already worked. |
||
oobabooga
|
4c5423a36a
|
Studio: stop docked composer from squeezing download panel (#9334)
* Studio: stop docked composer from squeezing download panel * Floor the overlay rail card by card * Measure the rail's cards off the layout box * Tighten the new comments * Drop out-of-flow cards from the rail's measurements |
||
|
Daniel Han
|
2043c734c4
|
Take two ubuntu jobs off their own runners: absorb one, delete the other (#9360)
Over 400 completed main push runs, 25 of 96 job types execute in under 120s: 1116s of work spread across 25 runners, each queuing for about three hours. Two of them are dealt with here. Security audit :: pytest tests/security 72s exec, 11096s queue Unsloth export capability :: capability (ubuntu-latest) 67s exec, 10642s queue Different problems, so different treatments. The security suite MOVED, onto the Workflow trigger lint runner ------------------------------------------------------------------------ Same argument that put the lockfile and load-orchestrator lanes into Lint CI in #9176: work with a narrow trigger, moved into a job that was going to occupy a runner on this commit anyway, can only reduce the slots a commit takes. Here the trigger widens too, since this host has no paths filter and security-audit.yml's pull_request does, so the suite now runs on every pull request rather than on the ones that touch its paths. This host and NOT Lint CI, where the other lanes went, and that is the whole decision. Lint CI installs shellcheck from apt, so its harden-runner has to permit escalation and an apt mirror; a security gate moved there would run under a policy weaker than the one it has today. Workflow trigger lint's harden-runner block is byte-for-byte identical to the one the job carried in security-audit.yml (block, disable-sudo, the same six endpoints), so nothing about its isolation changes. harden-runner binds per runner, not per step, which is what makes that the deciding constraint rather than a detail. Folded into the existing pytest invocation rather than added as a step of its own. I wrote it as a separate step first, so a security regression would not be reported as a workflow-guard failure, and test_the_guards_run_in_one_pytest_invocation rejected it: one step per module costs about 15s of interpreter and conftest startup, measured in this repo at 53.9s as one invocation against 300.8s as one each. The guard is right and the attribution preference is not worth 15s. pytest and PyYAML are now pinned here to the versions security-audit.yml pinned them to. That suite runs scripts/lint_workflow_triggers.py as a SUBPROCESS and asserts on its exit semantics, so a pytest or PyYAML that resolves differently changes what it is asserting against. The capability ubuntu leg was DELETED, because it was already duplicated ------------------------------------------------------------------------ That workflow's own comment already explains why it has no macOS leg: every test in tests/test_export_capability.py goes through _patch(), which monkeypatches _has_torch, get_device and is_apple_silicon, so a real Mac proves nothing a Linux runner does not -- and studio-backend-ci.yml runs the same file on ubuntu-latest as part of `pytest tests/`. That argument reaches one step further than it was taken. If Backend CI covers the file on Linux, the ubuntu leg HERE is the duplicate too. Checked: the file is not in that job's --ignore list. The import-safety test does not need a torch-free image either; it installs its own builtins.__import__ blocker and drops preloaded torch/unsloth from sys.modules, so it proves the same thing inside Backend CI's fully installed environment. Windows stays. Nothing else in CI runs that file there, and _has_torch's import probe is the per-OS behaviour the job exists for. The guard ------------------------------------------------------------------------ tests/studio/test_short_job_absorption.py, wired into the unfiltered job. Both changes fail silently rather than loudly if they regress, which is what it is for: - the suite still runs somewhere, and no longer runs twice - the absorbing job's harden-runner has not widened past the six endpoints the suite came with, since "the policy is identical" is the entire justification for this host - the absorbing job has not gained a paths filter - capability still has its Windows leg - studio-backend-ci.yml still runs the whole tests/ tree and does not name test_export_capability.py, because one line added to that --ignore list would remove the coverage the ubuntu leg was deleted for, and nothing would turn red Mutation-tested, each failing exactly one test: drop tests/security from the invocation; add one endpoint to the allowlist; add --ignore=tests/test_export_capability.py to Backend CI; remove the Windows leg. Also corrected a docstring in tests/security/test_scan_packages.py that named tests-security and what it installs. It was about to become false. Verification ------------------------------------------------------------------------ tests/security under the host's own -n 4: 409 passed, 6 skipped. scripts/lint_workflow_triggers.py: OK across 41 workflow files. All three workflows still parse; security-audit.yml keeps its other 4 jobs. Net: 7 short ubuntu slots per commit, down to 5. A note for whoever extends this. The census that found these 15 candidates was partly stale and I nearly acted on it: the lockfile and load-orchestrator rows were already absorbed by #9176 and their samples were pre-merge tails, and the two Local Agent Guides rows show ~0s because they are if-gated to schedule and dispatch, which is a skip and not a fast job. Read the trigger before ranking by duration. Of the remaining candidates, Scorecard is blocked by its job-level id-token: write, the Kaggle gate by a downstream needs:, npm-provenance by an audit egress policy plus registry.npmjs.org, and the notransport clean-install lane by a container that asserts several common tools are absent. |
||
|
Daniel Han
|
1c3dde199b
|
Drain the sampling debounce long enough for the node CI actually runs (#9332)
* Stop Frontend CI installing Chromium's system libraries through apt every run `playwright install --with-deps chromium` runs its own `apt-get update` inside itself, so it bypassed everything CI has learned about apt: the shared retry helper's 20s transfer cap, APT_ACQUIRE_RETRIES: '0', and the archive cache. The job failed 3 of 8 runs on main. Job 96072994354 (main, 2026-08-19): 9 packages, 21.1 MB, and `fonts-wqy-zenhei [7472 kB]` alone took 5m50s off azure.archive.ubuntu.com. Both 420s attempts died mid-download. That is the same mirror and the same package that took the webkit shards down in #9289. Attempt 2 logged "Need to get 8833 kB/21.1 MB", so apt does resume partials across attempts and still could not finish. Split the way studio-ui-smoke.yml splits it: download the engine, launch it to find out whether the system libraries are actually missing, and run `install-deps` only if they are. ubuntu-latest is a browser-testing image and usually ships them, so the common path now runs no apt at all. The browser and apt-archive cache keys are deliberately identical to the chromium-only shards in studio-ui-smoke.yml (engine token `c`): same image, same Playwright version, same single engine, so the entry is shared rather than duplicated against a budget measured at 99.3% full. The step's authorised worst case doubles with the second helper call, to 2 x (2 x 420s + 125s) = 1930s, so its timeout goes 17m -> 33m and stays under the job's 40m. Both guarded calls are skipped on the common path. Guard: tests/studio/test_playwright_install_avoids_with_deps.py fails the build if `--with-deps` returns to any workflow, and is wired into workflow-trigger-lint, the only job with no paths filter. * Drain the sampling debounce long enough for the node CI actually runs Frontend CI has been red on main since #9055, not intermittently: eight consecutive main runs failed at `Unit tests`, every one on node v22.23.2. The three suites from #9055 wait for a debounced write with a fixed drain -- three rounds of tick(1000) plus six setImmediate turns -- and then assert. Three rounds is enough on node 24, which is what a dev box happens to have, and is not enough on node 22, which `setup-node: 22` resolves to. The same chain drains far fewer continuations per round there, so the write had not landed when the assertion ran. Reproduced by downloading node 22.23.2 and running the suites against both. Measured on the compat suite: rounds 3 10 30 60 failing 7 5 1 0 The compat suite reported it as a missing value (expected 1.37, actual undefined). The simulation suite reported it as an ORDERING violation -- "chat A temperature: owed 0.6, shows 1.37" -- because a scenario whose write has not landed looks exactly like one that wrote the wrong value, which is why this read as a feature bug rather than a slow test. There were three copies of the drain: one per suite plus the shared thread-sampling-world helper the simulations run through. Fixing only the two suites left the simulations red, since their scenarios drain through the helper. The bound is 200, generous rather than tuned to the observed 60, and settle() now takes an optional `until` predicate: it returns as soon as the caller can see the work, and throws naming itself if the condition never holds, so the next slow runtime reports "settle gave up" instead of an assertion on a missing value. Verified: node 22.23.2 compat 16/16 and simulation 18/18 (both were failing); node 24 full frontend suite 4060/4060. Only the simulation suite imports the helper, and no fixed three-round drain remains in tests/. * Raise the scenario drain to 600: 200 was marginal, and Windows needed more At 200 the simulations were 0 failing on one run and 1 on the next on the same machine, and the Windows runner -- slower again -- still had 2 of 120 orderings short. 600 is 0 failing across three consecutive runs, at 107s against 52s. Also records the adaptive version that was tried and is wrong here, so it is not tried again: the rows only change WHEN the write lands, so 'rows have stopped changing' is precisely the pending state being waited through. Quiescence on that observable stops early by construction and scored 4 failures where the fixed bound scored 0. * Stop the settings smoke asserting a tab count a new page invalidates Unskipping the browser smokes surfaced this immediately: the blocked-panel run failed with 'blocking the data panel took the dialog down' while its own report said dialog: True. Nothing had taken the dialog down. The check was if not state["dialog"] or state["nav"] != 12: and the keyboard-shortcuts page had made the nav 13. A stale constant, reading as an error-handling regression. The nav size is now read before the panel is blocked and compared against itself, which is the invariant that was meant: blocking a panel must not collapse the dialog, whatever size the dialog is. The same drift had a quieter half. The smoke's TABS list still had twelve entries, so keyboard-shortcuts had no browser coverage at all and the smoke stayed green without it. It is added here, and tests/studio/test_settings_smoke_covers_every_tab.py pins both directions against settings-dialog.tsx so the next page cannot go uncovered silently. It also checks the workflow's PW_CHUNK_FAIL names a tab that exists -- that value lives in studio-frontend-ci.yml, not in the smoke, and a rename would leave the run blocking nothing while still reporting PASS. Wired into workflow-trigger-lint, the only job with no paths filter, because it reads a workflow. Mutation-tested both ways: dropping the tab from TABS and restoring the literal count each turn it red. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drain the sampling suites on the loader too, and fail loudly when it gives up The fixed round count in the previous commit was still a guess, and the Windows job proved it: the SAME commit at 600 rounds passed one run and failed the next with "A1: 2 violation(s) across 120 orderings", reporting stale reads as lost edits. Raising the number again was not the answer. The pending work has a second half nothing was waiting on. The store's thread-scoped write ends in `await import("../utils/chat-history-storage")` (chat-runtime-store.ts:1326 and :1754), and these suites register() a resolver hook, which routes that import through the hooks thread. Three repeat imports of an already-loaded module: v24.14.0 no hook 1, 1, 1 turns hook registered 1, 1, 1 v22.23.2 no hook 1, 1, 1 turns hook registered 6, 3, 35 That is the whole green-locally / red-on-CI split, and it is why a loaded Windows runner fails what the same commit passed an hour earlier: the pending work is a message to another thread, so its cost is scheduling latency, not instructions. No round count is correct for that. Counting the mocked timers alone does not cover it either, which is worth recording since it is the obvious next idea. With the counter installed and 25 consecutive quiet rounds per drain, 150 macrotask turns of nothing, v22.23.2 still lost 7 orderings across A1 and A3, every one a write that had not landed. So drain on both observables. tests/helpers/mock-timer-drain.ts wraps the MOCKED setTimeout with a counter, giving an exact count of timers scheduled and not yet fired or cleared, and each round also issues its own import and waits for it, so the wait scales with the loader instead of guessing at it. The drain returns when no timer is outstanding and three consecutive rounds neither scheduled nor fired one. With the probe, three quiet rounds is green on v22 and v24 alike. The generous bound stays as a BACKSTOP that THROWS and names what was still outstanding, which is the change that matters most here. An under-drain used to be indistinguishable from the store losing an edit, so it sent the investigation into the store for two rounds. Now it says so: settle: drain exhausted after 2 rounds, with no timer pending but work still scheduling or firing within the last 3 rounds. Nothing read after this point is trustworthy: a queued write has not landed, so the store still shows the PREVIOUS value, which reads as a wrong value rather than a missing one. Fix the work or raise the backstop; do not read this as the store losing an edit. It is also much faster, because it stops when the work is done rather than after 600 rounds regardless. A typical drain now takes 4 rounds; instrumented over 840 drains in A1 the maximum was 4. simulation v24 101.7s -> 30s v22.23.2 106.5s -> 29s compat v24 0.54s -> 0.46s v22.23.2 0.61s -> 0.52s The three copies of the drain shape, one in the world helper and one in each test file, are now one helper. The dead end already recorded is kept next to the new measurements: quiescence on the store ROWS is still wrong, because the rows only change WHEN the write lands. Verified: both suites green twice each on v24.14.0 and on v22.23.2, the version setup-node 22 resolves to. Full frontend suite 4080 passed, 0 failed. The exhaustion throw was confirmed by forcing the backstop to 2, which produces the message above and no wrong-value violation. A deliberately broken sanitizeThreadScopedSettings still produces a real ordering violation on both runtimes, so the drain is not exiting early. Test files only. * Read the workflow and the dialog as UTF-8, not as the platform default test_source_read_encoding caught four read_text() calls this PR added with no encoding. It is right and they are a real defect: the guard reads settings-dialog.tsx, playwright_settings_tabs.py and studio-frontend-ci.yml, and on a Windows runner Path.read_text() uses the ANSI code page, so any non-ASCII byte in any of them raises UnicodeDecodeError. The whole point of this guard is that a settings page can be added without anyone noticing; a guard that cannot be collected on Windows fails the same way. Repo tests (CPU) was otherwise clean: 1 failed, 8781 passed. * Tell a pwsh crash apart from install.ps1 losing its exit code test_the_pwsh_filter_keeps_the_log_clean_and_the_exit_code_intact went red on a hosted ubuntu runner with completely empty stdout and pwsh's own banner: "An error has occurred that was not properly handled. Additional information is shown below. The PowerShell process will exit." The interpreter died; the script never ran. Workflow trigger lint is green on the last 8 main runs and this PR touches no PowerShell, so it is the runner, not the repo. Read as an ordinary failure it says "$LASTEXITCODE did not survive the added pipeline stages, so a failing install.ps1 would leave its step green" -- an accusation against the installer, raised by a run that produced no evidence either way. That is the same shape as the drain in this PR: an environment shortfall wearing the costume of a product regression. _run_pwsh retries only that case, and the distinction is what keeps it honest: a run that reaches the `RC=` line is returned on the first attempt whatever the value, so a genuine regression can never be retried into green. Only a run with no RC= AND the crash banner is retried, because it carries no verdict to preserve. If both attempts crash it fails with a message naming the interpreter, not install.ps1. 17 passed. Exercised _run_pwsh against a script that prints the banner and no RC=: it raises, so the branch is not vacuous. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
149d29752f
|
Fix the two backend guards that #9292 turned red on the 3.13 leg (#9348)
* Stub the two helpers the sliced harnesses started importing Backend CI's "Repo tests (CPU)" has been red on main since |
||
|
Daniel Han
|
0b681d78cf
|
Put back the AppleDouble GGUF filters #9074 reverted, and guard them (#9353)
Some checks are pending
Mac Studio UI + API + Update + Inference CI / Chat UI, API, Update and GGUF inference (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (windows) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (macos) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (extra) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (picker) (push) Waiting to run
Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Backend CI / (Python 3.11) (push) Waiting to run
Backend CI / (Python 3.13) (push) Waiting to run
Backend CI / Repo tests (CPU) (push) Waiting to run
Unsloth export capability / capability (ubuntu-latest) (push) Waiting to run
Unsloth export capability / capability (windows-latest) (push) Waiting to run
Frontend CI / Frontend build + bundle sanity (push) Waiting to run
Frontend CI / Frontend unit tests (Windows) (push) Waiting to run
Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (banner) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (chat) (push) Waiting to run
Unsloth UI CI / Loaded-models indicator (cross-browser) (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection + VC++ round-trip (VS 2026) (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 units, no-VS resolve, VC++ round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Windows Application Control CI / installer survives a denied unsloth.exe (push) Waiting to run
Workflow trigger lint / workflow-trigger lint (pull_request_target / cache-poisoning) (push) Waiting to run
* Put back the AppleDouble GGUF filters #9074 reverted, and guard them Backend CI's 3.13 leg has six failures on main beyond the two in #9348. Three separate causes, all of them in the tests or in a merge resolution rather than in anything a PR meant to change. 1. llama_cpp.py lost every line of #8919 ------------------------------------------------------------------------ #8919, "never pick a macOS AppleDouble sidecar as a GGUF", touched 49 files. |
||
|
Daniel Han
|
3457a57a33
|
Stub the two helpers the sliced harnesses started importing (#9342)
* Stub the two helpers the sliced harnesses started importing
Backend CI's "Repo tests (CPU)" has been red on main since
|
||
|
Daniel Han
|
78a3587427
|
Keep path_utils off PEP 604, which the 3.9 floor gate rejects (#9335)
* Keep path_utils off PEP 604, which the 3.9 floor gate rejects `Core` has been failing on main at 'python floor compatibility (HARD GATE)', all three HF/TRL cells: 36 studio files now evaluate PEP 604 unions on the floor, up from 35 The 36th is studio/backend/utils/paths/path_utils.py, from #8919. That change is right; it just added subject_key: Callable[[str], object] | None = None to a module with no `from __future__ import annotations`, so the annotation is evaluated at import and `X | None` is a TypeError on the declared 3.9 floor. Uses Optional[...], which the module already imports, rather than adding the future import. Five other modules import this one, and deferring every annotation in it is a wider change than the one union needs -- the ratchet's own docstring warns that converting a file wants Studio booted, because FastAPI resolves annotations when it builds endpoints. Verified on main: offenders back to 35 against a debt of 35, the whole test_python39_compatibility.py suite passes (8), and the AppleDouble guards the original change added still pass (9). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
Pascal André
|
489fab4a71
|
Support OpenCode V2 in unsloth start (#9275)
* Support OpenCode V2 in unsloth start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep stable OpenCode guide coverage * Follow the OpenCode V2 stable release * Honor OpenCode V2 policy and server semantics * Fix OpenCode V2 launch command recipes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
28b8880464
|
Studio: compact a chat by resetting the epoch, not by trimming it forever (#9162)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: count sanitized rolling context prompts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: refit rolling context after respawn
* Studio: scope middle truncation to passthrough
* Studio: refit tool prompts after respawn
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: retain later choice truncation metadata
* Studio: report clipping-only context truncation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: expose the turns a rolling fit evicts
Rolling context eviction currently drops turns with no way for a caller to
learn which ones went. Extract the turn grouping so the eviction unit is
reusable, add an identity-based diff of what a fit removed, and let a caller
reserve room for content it intends to add back after fitting.
The reserve deliberately does not affect whether trimming happens at all, only
how far it goes once it is already required, so a conversation that fits today
is still returned untouched.
* Studio: archive evicted turns into a per-thread rag scope
An evicted turn is currently gone for the rest of the session, so the model
will state the conversation began wherever its visible context begins. Keep the
turns the rolling window drops in a searchable scope built on the existing
store, chunker, embedder and hybrid retrieval.
The archive is cumulative: every compaction adds to it and nothing is cleared,
so a later compaction can still find what an earlier one evicted. It lives in
its own scope rather than the thread's document scope, because thread documents
are injected in full on every request and would re-inject the whole history.
Idempotent by content hash, since the same turns are evicted again on every
later request. Every entry point degrades to a no-op rather than raising.
* Studio: recall archived turns on the turn that evicted them
Given only a search tool, a model decides for itself whether to look, and
mostly does not. Measured on MRCR v2, a 35B declined on 56% of rows, scoring
0.099 when it skipped against 0.461 when it searched. Forcing one retrieval on
the compaction turn took tool-only 0.258 to 0.604, and the model then called
the tool on 0% of rows, so the common path costs nothing extra.
Recall fires at most once per request, since the tool loop refits on every
iteration. The tool loop renders it as an ordinary tool exchange through the
builder shared with document auto-inject; the plain path prefixes the latest
user message instead, because it sends no tools array and a tool role without
one breaks strict chat templates.
Only scalar counts join the context_truncated event, so no message content
reaches the wire.
* Studio: let the model search a compacted conversation
Forced recall answers the turn that evicted, but a later turn can refer to
something the forced pass had no reason to fetch. Add search_conversation so
the model can go looking, scoped to the thread's own archive and sharing the
admission slot with document search so the two cannot race for the embedder.
The tool is offered only once a thread has actually had turns archived, so an
ordinary short chat never pays for the schema, and it is classified read-only
so auto mode does not prompt on every call. A matching system-prompt note tells
the model the session was compacted, since otherwise it assumes the
conversation began where its visible context begins.
Deleting a thread now drops its archive rather than leaking a scope per chat.
* Studio: read thread_id defensively when selecting tools
_select_request_tools also serves the token-count request model, which has no
thread_id field, so the archive gate raised there.
* Studio: retrieve archived turns lexically first
Recalling your own conversation is mostly an exact-match problem: a name, a
number, an identifier someone pasted twenty turns ago. Those live or die on
rare-token matching, and hybrid fusion was losing them.
Measured on a 30-turn walkthrough of a 230k-character document at a 16k window,
where every turn shared the same wrapper text. The chunk holding the needle
ranked 3rd lexically at any k, was never returned by dense retrieval at all,
and RRF pushed it to 16th because it had 30 useless dense hits to fuse with.
End to end the model answered with the exact code once lexical leads, and could
not answer at all before.
Dense still fills whatever the lexical pass leaves, for paraphrased recall.
* Studio: keep recall on the branch the user is actually on
Editing an earlier message rewinds a thread and continues down a new branch,
but the archive is append-only and still holds everything the abandoned
continuation produced. Verified against a live build: after rewinding past a
turn, querying its distinctive text still returned it, so the model could be
handed a turn that on this branch never happened.
Recall now drops archived turns that are absent from the thread's saved
transcript. Threads with no saved transcript are left unfiltered, since an API
caller may pass a thread_id without persisting messages and an empty transcript
is absence of evidence rather than evidence the turns are gone.
Containment on a normalised prefix rather than a digest, because the archived
copy is rendered from the inference projection and the saved copy comes back
through the message store.
* Studio: show a persistent compaction notice on the turn that compacted
The only signal that a long chat had been compacted was a toast, which vanishes after
a few seconds and does not survive a reload. A user who scrolls back later has no way
to find out why the model seemed to forget the start of the conversation.
The notice renders from metadata.custom.contextTruncation inside the assistant
message's own container, so it is not part of the conversation sent to the model, is
not editable, and is not exported as content, but it stays attached to the turn it
describes. It reports how many messages were dropped and, when the conversation
archive is on, that they are still searchable and how many passages were recalled.
* Studio: show the compaction notice once, on the turn it started
A thread that has outgrown its window compacts on every turn from then on, not
just the first, so a notice per compacted turn was a notice on every reply for the
rest of the conversation. The user needs telling once.
The notice is now gated on being the first compacted assistant turn in the thread,
found by walking the thread rather than assuming the compacted turns are contiguous
or that this is the last one, so a rollback that removes the turn it was on moves it
to whichever turn now compacts first. The wording follows: it describes the state
the conversation is in from here on, and carries the counts from the turn it began
on in parentheses.
* Studio: make compaction an occasional event instead of every turn
The fit was stateless. The client re-sends the whole saved transcript on every
request, so "keep the newest N tokens" recomputes from scratch each time and slides
forward a turn or two at a time. Measured on a 40-turn thread against an 8k window,
the eviction boundary moved on 12 of 40 turns, which means every few replies quietly
lost a little more of the conversation, llama-server's prefix cache was thrown away
each time the head of the prompt moved, and there was no such thing as a compaction
event to tell the user about.
Two changes make it discrete. The fit now reads back the boundary the thread last
compacted to, from the newest assistant turn's own persisted truncation, and reapplies
it before deciding anything, so nothing new is stored and it survives a restart. And
when the boundary does have to move, the trim takes a further ROLLING_COMPACTION_
HEADROOM_RATIO of the budget out (default 0.25) rather than skimming to the brim, so
the new boundary has room to stay put. Same 40-turn thread, same window: 4 compactions
over 60 turns instead of 14, keeping about 82 percent of the usable budget.
Both are gated on the prompt not already fitting, exactly as the recall reserve is. A
conversation inside its window is never evicted to satisfy either, and a stale boundary
from a branch that was rolled back cannot evict a chat that now fits.
The notice follows: it is shown when dropped_messages rises above the last turn that
reported it, so it appears once per compaction and stays quiet in between.
* Studio: pin that the compaction notice can never become conversation
The notice is a sidecar rendered from metadata, not a message, and the ways that
could quietly stop being true are all one careless edit away: moving it inside
MessagePrimitive.Parts would make it a content part, and everything that walks parts
would then replay it to the model, copy it and export it.
Asserts it renders as a sibling of the content parts, that neither the outbound
message builder nor the assistant replay serialiser reads the key it renders from
(bounded to those function bodies, since the streaming handler reads the same key
legitimately on the way in), that the markdown export does not mention it, and that
it is suppressed while editing so it cannot be typed into the textarea and saved
back as text.
* Studio: say which part is too long when nothing can make a turn fit
If the message just sent is itself bigger than the window, no amount of eviction
helps. The fit already handled that correctly, returning the conversation untouched
so the request reaches llama-server's normal context-length error, but what the user
was then told was actively misleading: the error reports the size of the WHOLE
conversation and advises shortening it, when the history has already been evicted and
the single message is the part that does not fit. Measured at a 4096-token window: a
5000-token message produces 'Message too long: 10290 tokens ... shorten the
conversation', and shortening it cannot possibly work.
The fit now returns a fits:false diagnosis instead of a bare None, carrying what the
conversation could not be reduced below and how much of that is the latest turn.
Every consumer already gated on fits, so this is inert wherever a truncation is
treated as a compaction; the streaming paths now forward it so the client can use it.
The toast reads the diagnosis and, when the latest turn alone exceeds the window,
says so with the numbers instead of offering advice that leads nowhere. The merge
drops the diagnosis once a later refit succeeds, by delete rather than by assigning
undefined, so an ordinary response keeps exactly the shape it had before.
* Studio: fix the review findings on the conversation archive
Fifteen items, each reproduced against the code before changing anything.
Correctness in the request path. The fit reported a successful recall-capable fit
whenever the result was under the prompt budget, but protected messages can stop the
trim reaching the reserve target, and the recall then went in anyway: reproduced at
ctx 8000, the fit accepted at 6900 and recall took the request to 8948, past the
window it had just been made to fit. Recall is now sized from the room the fit
actually obtained, and skipped when there is none. Separately, the sticky boundary
describes the original transcript, so re-applying it on a later tool-loop fit evicted
another boundary-sized block of live history: measured, a second fit dropped 28 of the
30 surviving messages instead of 14, and the summed count persisted an inflated
boundary for the next request. It is now spent after the first fit of a request.
Availability and privacy. enabled() trusted RAG_AVAILABLE, which only records that
import sqlite_vec worked; rag_available() exists because the native vec0 library it
loads is a separate file a venv can lack. On such a machine the fit held a recall
reserve back, evicting extra history, and then both the archive write and the recall
failed, so the user paid for content they never got. And a temporary chat is never
written to studio.db, yet the frontend still sends its thread_id and the request
carries no incognito flag, so its turns were archived to a scope no deletion flow
could reach. Archival now requires the thread to be persisted, which is the same rule
that keeps every archive reachable by a delete. Clear-history and project deletion
drop archives too; only DELETE /threads did.
Archive integrity. The document was committed before its chunks, so a failed chunk
write left an empty row marked completed that document_by_hash then skipped forever;
both now go in one transaction. The live-branch filter accepted a turn on its first
matching line, so editing only the assistant half kept serving the old answer; the
whole turn must be present. Retrieval fetched exactly k before that filter, so stale
turns could starve live ones and recall returned nothing; it over-fetches first. Tool
turns archived only the tool name, which cannot answer what was actually run, so a
bounded rendering of the arguments and the assistant text goes in as well.
The tool surface. Studio always sends an explicit enabled_tools array and has no
reason to name an internal tool, so the allowlist filter removed search_conversation
before the archive gate ran and the tool, plus the compaction nudge gated on it, never
appeared in a Studio chat. It now follows the archive rather than the allowlist. The
forced recall rendered a tool exchange even when the tool was absent from the
catalogue, which is the strict-template hazard the plain path avoids; it picks inline
in that case, and always inline for the final-answer request, which sends no tools at
all. A model-supplied top_k reached a slice as out[:-1] and returned nearly the whole
candidate pool, so it is clamped. Both retrieval tools now share the per-turn search
cap; only the knowledge-base one was counted.
The UI. A fits:false diagnosis is the fitter reporting it could NOT fit, so toasting
that older turns were removed was untrue and burned the once-per-thread flag a later
real compaction needed. The too-long advice compared the latest turn against the raw
context length rather than the prompt budget, so a 3,500-token message in a
4,096-token window was still told to start a new chat, which fails identically.
* Studio: fix the follow-up findings on the archive fixes
Four items, three of them about last round's own fixes.
Archived tool turns had become permanently unrecallable. render_turn now writes
'assistant called X: args' and 'tool result: ...' lines, while assistant-ui persists a
tool call as a structured tool-call content part that the transcript flattener dropped;
with every archived line required to appear in that transcript, no tool turn could ever
match. The transcript now flattens toolName, args and result, and the probe strips the
'assistant called <name>:' label, which is ours rather than the stored message's.
The branch probe compared only the first 160 normalized characters of each line, so an
edit to the tail of a long answer left the stale copy eligible. Ordinary lines are now
compared whole; tool results keep a prefix, since render_turn deliberately truncates
those and the archived copy is not meant to equal the stored one.
The forced recall sized itself by dividing the remaining budget by CHUNK_TOKENS, which
is an embedding-token limit rather than the chat template's cost, and prices none of
the wrappers around the injection. It is now recounted with the same tokenizer the fit
used and dropped if it overshoots, so the estimate can no longer eat the reply reserve.
An omitted top_k on search_conversation defaulted to the clamp ceiling of eight rather
than the configured recall default, so an ordinary search could return eight archived
turns into the protected current exchange that rolling truncation cannot evict.
* Studio: filter conversation recall to the active branch
A thread's stored rows are the whole message DAG. Retry and regenerate keep the
replaced response as a sibling on purpose, so filtering recall against the whole
thread cannot tell a live turn from one the user replaced, and an archived copy of
the abandoned response could be recalled into a branch where it never happened.
Filter against the messages the request was actually sent with instead, which is
one branch by construction, and hand the same branch to search_conversation so the
model cannot ask for what the forced recall refused. Falls back to the thread-wide
blob for a caller with no branch to offer.
Also retunes two respawn-refit fixtures whose windows no longer produced two
compactions after compaction started trimming a headroom margin below the budget.
* Studio: survive a delete mid-archive, and read the boundary off the active branch
Deleting a chat cancels its generation, but cancellation is cooperative and the
chunk-and-embed pass between the archive's liveness check and its commit does not
observe it. A delete landing in that window drops the thread's rows and sweeps its
scope before the commit puts rows back, leaving content the user deleted in a scope
no later delete can reach. Re-check after the commit and drop the scope: the delete
route removes rows first and sweeps archives last, so either order converges.
The sticky compaction boundary had the same thread-wide read as recall did. The
stored rows are the whole DAG ordered by creation time, so after a Retry the newest
assistant turn can be the sibling the user switched away from, and its boundary is
sized for history the active branch does not have. Resolve it against the request's
own messages instead.
* Studio: give the safetensors loop the same conversation-search guards
search_conversation is advertised by thread, not by backend: the tool selector is
shared, so a chat compacted under a GGUF model still offers it after the user
switches to a safetensors one. The safetensors loop had neither guard the GGUF loop
applies to it. It passed no active branch, so a search there fell back to the
thread-wide rows and could answer from a branch Retry left behind, and it capped
only search_knowledge_base, so paraphrased conversation searches could append
archived passages into the protected current exchange on every iteration until the
window failed. The shared set of capped retrieval tools now lives beside the cap.
The forced recall also derived its query from the loop conversation, which on a
later iteration can end with an internal user-role re-prompt rather than anything
the user wrote. It reads the request branch's own latest user turn instead.
* Studio: widen recall past an abandoned branch, and name whose turn overflowed
One over-fetch is not enough for the live-branch filter. Rewinding or retrying a
continuation that had already been compacted leaves enough stale turns to fill any
fixed candidate window, and the whole page is then rejected while the live match
sitting just below it is never examined, so recall reports nothing although the
answer is in the archive. Widen and re-ask instead, stopping as soon as there are
enough live hits, when the archive stops yielding candidates, or at a bound.
The irreducible-fit diagnosis also carried the size of the last message without
saying whose it was. A tool loop refits with the tool result appended, so that turn
is often output the user never wrote and cannot edit, and the client told them to
shorten it. It now reports the role, and the advice splits on it.
* Studio: shrink an over-budget recall, and keep truncated tool turns on their branch
The exact recount is the right gate, but dropping the whole recall when it fails is
the wrong response: with the shipped defaults a full top-K of long turns lands just
over the reserve once the wrappers are priced, which would disable the forced
retrieval on exactly the long conversations it exists for. Halve the number of turns
and re-ask instead, down to one, before giving up.
The live-branch probe keyed its prefix rule off the tool-result label, but a long
tool result is one appended string containing many newlines, so only its first line
carries that label. Continuation lines were compared in full, including the last one,
which is the only line the truncation marker is on and can therefore never appear in
a transcript: every archived tool turn over the cap was rejected as rolled back.
Key off the marker instead, which also covers truncated tool arguments and compares
the full text everywhere else.
* Studio: retire an edited turn's whole archived copy, not just the edited chunk
A turn longer than CHUNK_TOKENS is stored as several chunks of one document, and the
live-branch filter ran per chunk. Editing the second half of a long answer therefore
retired only the chunks carrying the edit, and an untouched earlier chunk of the same
retired turn stayed eligible on its own. The unit that was archived is the turn, which
is what the filter already claimed to enforce.
Validate every chunk of the candidate's document before admitting any of it, cached
per call since candidates from one turn share a document. A query failure falls back
to the per-chunk answer rather than failing the recall.
* Studio: do not hold a recall reserve back for a chat that is never archived
archive_turns refuses a thread with no saved messages, because a temporary chat must
not be persisted into a scope no deletion flow can reach. The reserve did not follow
that rule: it was granted whenever the RAG stack was available, so an incognito chat,
or an API client sending a thread_id without saving anything, paid a full 2,048-token
reserve for content that can never arrive. The fit subtracts the reserve from its trim
target, so that room is bought with evicted history. Measured on a 4K window: 15
tokens of conversation survived a compaction instead of 1,615.
Both now ask one predicate, can_archive, backed by an existence probe rather than a
row load.
* Studio: match archived turns in order, and delete them without sqlite-vec
Independent line membership accepts a turn whose lines were merely rearranged: every
probe still occurs somewhere, so the pre-edit ordering stayed eligible and would be
served back as what happened. Match the probes in order instead.
That only works if both sides agree on the order, and they did not: render_turn writes
a tool call before any assistant text on the same message and the result after it,
while both transcript builders wrote the text first, so a tool turn carrying both was
rejected outright. One flattener now lays out a message the way render_turn does, for
the request shape and the stored shape alike, and it offers both JSON spacings for a
stored call's arguments, whose object form is not the string the model emitted.
Deletion no longer depends on the optional native extension. An archive is only written
while vec0 loads, but the library can stop loading afterwards, and a delete that quietly
did nothing left a deleted conversation's turns on disk to answer again once it loaded.
The fallback removes the text-bearing rows over a metadata connection; the embedding
rows it cannot reach carry no text and resolve through tables that are gone.
* Studio: give the provider loops the branch, and keep the thread across a respawn
The provider tool loops take their catalogue from the same selector as the local ones,
so search_conversation is advertised there too once a thread has an archive, but the
loop passed no active branch and its searches fell back to the whole stored DAG, where
Retry keeps the response it replaced. This is the third loop to need the same wiring.
The respawn retry also dropped the thread. It re-enables context_overflow on purpose,
to refit an already-compacted prompt for a replacement window, so it is the one path
that deliberately compacts a second time: without the thread those extra evictions were
archived nowhere, nothing was recalled in their place, and the fit held back no reserve
and re-applied no boundary.
* Studio: keep the reply that follows a recall, and archive respawn refits
group_turns keeps an assistant tool call, its result and the reply that follows in one
group, and the archiver rejected any group containing one of our own injections. So on
a turn that forced a recall, the model's actual answer was thrown away with it: the
question was archived from its own group and the answer was not, and a later search
could find what was asked and never what was said. The injections come out now and the
rest of the turn stays; retrieved passages are still kept out of the index they came
from.
The two respawn refits inside the tool loop also evicted without archiving. They run
against a smaller replacement window, so those turns are simply gone otherwise. They
archive only, deliberately: no reserve is held back on that path, so injecting a recall
there is what would push the retry back over the window, and the next request can still
recall what this one archived.
* Studio: bound the branch check to one turn, and anchor only what recall injected
The branch check searched one flattened transcript, so a line an edit removed could be
supplied by any later message that happened to repeat the words. Short answers repeat
constantly: an archived "Should I deploy? / No" survived its answer being edited to
"Yes" because a later turn said "No", and the stale pair stayed recallable. The
transcript is now one normalised string per message, and a turn matches only if its
lines appear in order within a run of adjacent messages no longer than the turn itself.
Recall anchoring took the last two messages of the conversation. That is right for the
tool style, which appends a synthetic pair, and wrong for the inline style, which
appends nothing and rewrites the latest user message in place: it also pinned the
assistant turn before it, and with it an eviction unit the fit was entitled to drop,
which can fail a later iteration that would otherwise have fit. The injection now
reports exactly what it added or rewrote, and only that is anchored.
* Studio: validate a turn's chunks against one run of the branch, not each on its own
The chunks of an archived turn are consecutive slices of a single rendering, and each was
checked for itself. That let a turn be reassembled out of parts that never sat together:
the head matching the question and the answer it has now, the tail matching some later
message that happens to repeat the passage an edit removed, so the stale association
stayed recallable. Reproduced before the change, with each chunk passing on its own.
All chunks of the document must now be found within one run of adjacent messages, bounded
by the document's own line count, which is at least the number of messages the turn was
rendered from, so a turn that really is still there always fits.
* Studio: budget conversation searches and disambiguate identical replies
search_conversation clamped the model's top_k only against a fixed ceiling of 8.
Eight chunks is roughly 4,000 tokens once wrapped, and the result lands in the
current tool exchange, which rolling truncation protects and cannot evict, so on
a small context the search itself made the turn unsendable. The GGUF loop now
passes the room the window actually has left, the tool clamps top_k by it, and a
search with no room says so instead of returning a result that cannot be sent.
The sticky compaction boundary took the newest on-branch assistant row, but the
branch check is textual, so two siblings whose replies read the same ("Done.")
are indistinguishable from there, and Retry is exactly what produces them.
Taking the first match applied a boundary measured on a different, deeper branch
and evicted live history. Where the text cannot separate them, the smallest
boundary is now used: too small costs one more compaction, too large loses turns
the branch still has.
* Studio: price a conversation search against the whole prompt
Three gaps in the budget the previous commit introduced.
The GGUF loop subtracted an estimate of the messages alone, leaving the tool
catalogue out of the prompt entirely. A large catalogue is thousands of tokens,
so the request could already be near its budget while the search was still told
there was room for several 500-token chunks. The preflight already prices the
request exactly, catalogue and template included, so the difference between that
count and the estimate of the same messages is carried forward and the estimate
only covers what the loop appends after it.
The safetensors loop advertises and executes search_conversation for a thread
compacted under a GGUF model, but forwarded no budget, so the clamp in the tool
was skipped there. It now passes the room this model has left. With no known
context length the argument is omitted rather than sent as zero, which would
refuse every search.
The sticky boundary filtered candidates with a substring test, which is right
for archived chunks and wrong for whole messages: an abandoned "Done" rode in on
a live "Not done yet", and having no live twin it then decided the boundary
alone. Where any candidate matches a live message exactly, only the exact ones
are considered; where none does, the old behaviour stands, since a stored row is
not always byte-identical to what the client re-sends.
* Studio: keep the overflow diagnosis on the tool path
The irreducible-overflow branch counted the newest message on its own, and a
tool loop reaches that branch with a tool result last. A tool result by itself
is not a conversation: templates that require it to follow its assistant tool
call refuse to render one, and the exception escaped the fit entirely, so the
caller fell back to the untrimmed request and the client was told nothing at
all. That is the one path this diagnosis was added for.
The count is now attempted and falls back to the estimator when the template
refuses. An approximate number is worth more here than a diagnosis that never
arrives.
* Studio: budget the conversation search in the provider loop
The third tool loop forwarded the active branch but no budget, so the clamp in
the tool was skipped there and a model-chosen top_k of 8 could append roughly 4K
tokens to a prompt this loop replays on its next call.
Studio knows no window for an external model: the request carries no context
length and there is no registry to look one up in, and a custom
OpenAI-compatible endpoint can be a small local server. So rather than a
measured budget, this path spends no more than one ordinary recall's worth,
which is the same amount the compaction turn itself is sized for.
* Studio: bound an archived turn to its own messages, and archive it once
Two problems in the archive.
The whole-document branch check bounded the run by the number of LINES the turn
produced, and gave each chunk its own start. A turn is two or three messages
however long it is, so a hundred-line answer got a hundred-message window, and
the tail of an edited answer could be satisfied by a message well outside the
turn. The run is now bounded by the messages the turn was rendered from, counted
from the labels render_turn writes, and the chunks are scanned as one pass: each
continues where the previous one stopped rather than restarting. The cursor
inside a message is deliberately not carried over, because chunks overlap and a
continuation chunk repeats the tail of the one before it.
The hash check that skips an already-archived turn ran long before the insert,
with the embedding pass in between, and the index on (scope, sha256) is not
unique. Two generations compacting the same thread both cleared it and both
wrote, so the turn was stored twice and its copies took two of the few recall
slots. The check is now repeated under a write lock immediately before the
insert. Reproduced with two concurrent archive passes: two documents before, one
after.
* Studio: record how many messages an archived turn came from
The run an archived turn is allowed to occupy on the branch was bounded by
counting the role labels in its rendered text. That counts lines the user wrote
as well as the ones the renderer did: a pasted chat log carries lines that look
exactly the same, and each one widens the run by a message, which is enough for
the message after an edited turn to supply the passage the edit removed.
The group's size is now recorded on the document when it is archived, in a
nullable column added the way the other lazy upgrades are. Archives written
before this have NULL and fall back to the label count, so nothing needs
backfilling to keep working.
* Studio: persist a compaction boundary the next request can use
The boundary was read back from dropped_messages, which counts what each fit
removed from the conversation in front of it. The tool loop refits on every
iteration and the client sums those counts, so a long agent run added the tool
exchanges the turn itself created, and the next request applied the total to its
saved transcript. Reproduced with six tool calls on a 4K window: three fits of 4
summed to 12, on a branch that only ever had 4 evictable messages.
The boundary is now carried separately, measured against the messages the
request was sent with, so it is absolute and re-sending it cannot advance it.
The client keeps the latest value rather than summing, and turns saved before
this fall back to the dropped count, which is the same number for a turn that
fit once.
The recall reserve is also dropped once archiving has failed. sqlite-vec can be
present and the thread saved while the embedder cannot start: archive_turns
swallows that and recall injects nothing, so the room the fit held back was pure
loss on every compaction, and the failure mode forgot more history than having
the feature off. A failed write marks the archive degraded and the next
successful one clears it.
* Studio: count the boundary past the system prompt, and notice it
Two faults in the boundary added in the previous commit.
It stopped at the first message still present, and a Studio request always
starts with a system prompt that a fit never evicts, so every compaction
recorded a boundary of zero. That is the same as having no boundary: the next
request would move the eviction point again and invalidate the prefix cache on
every turn. Instruction messages are now skipped rather than treated as the
front of the branch, and the newest turn is excluded because it is never evicted
and an inline recall rewrites it in place.
The compaction notice still keyed off dropped_messages, which is the
accumulated count. A tool-heavy turn reporting 12 while the boundary moved to 4
set a high-water mark that silenced the next two real advances. It now reads the
boundary, falling back to the dropped count only for turns saved before the
boundary was recorded.
* Studio: tighten comments in the conversation archive and rolling context window
* Studio: give the pasted-text import an extension on this branch too
The node test runner cannot resolve an extensionless relative import, so
delete-chat-files-preference fails to load the preferences store. main fixed
this in
|
||
|
Daniel Han
|
2affd53e1f
|
Studio: recall the latest version of a fact, not the most quotable one (#9161)
* Studio: add rolling context windows for local GGUF chat
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: return rolling context metadata for non-stream chats
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep original context when rolling fit fails
* Studio: keep instruction groups independently protected
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: preserve rolling context metadata across retries
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: count sanitized rolling context prompts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: refit rolling context after respawn
* Studio: scope middle truncation to passthrough
* Studio: refit tool prompts after respawn
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: retain later choice truncation metadata
* Studio: report clipping-only context truncation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: expose the turns a rolling fit evicts
Rolling context eviction currently drops turns with no way for a caller to
learn which ones went. Extract the turn grouping so the eviction unit is
reusable, add an identity-based diff of what a fit removed, and let a caller
reserve room for content it intends to add back after fitting.
The reserve deliberately does not affect whether trimming happens at all, only
how far it goes once it is already required, so a conversation that fits today
is still returned untouched.
* Studio: archive evicted turns into a per-thread rag scope
An evicted turn is currently gone for the rest of the session, so the model
will state the conversation began wherever its visible context begins. Keep the
turns the rolling window drops in a searchable scope built on the existing
store, chunker, embedder and hybrid retrieval.
The archive is cumulative: every compaction adds to it and nothing is cleared,
so a later compaction can still find what an earlier one evicted. It lives in
its own scope rather than the thread's document scope, because thread documents
are injected in full on every request and would re-inject the whole history.
Idempotent by content hash, since the same turns are evicted again on every
later request. Every entry point degrades to a no-op rather than raising.
* Studio: recall archived turns on the turn that evicted them
Given only a search tool, a model decides for itself whether to look, and
mostly does not. Measured on MRCR v2, a 35B declined on 56% of rows, scoring
0.099 when it skipped against 0.461 when it searched. Forcing one retrieval on
the compaction turn took tool-only 0.258 to 0.604, and the model then called
the tool on 0% of rows, so the common path costs nothing extra.
Recall fires at most once per request, since the tool loop refits on every
iteration. The tool loop renders it as an ordinary tool exchange through the
builder shared with document auto-inject; the plain path prefixes the latest
user message instead, because it sends no tools array and a tool role without
one breaks strict chat templates.
Only scalar counts join the context_truncated event, so no message content
reaches the wire.
* Studio: let the model search a compacted conversation
Forced recall answers the turn that evicted, but a later turn can refer to
something the forced pass had no reason to fetch. Add search_conversation so
the model can go looking, scoped to the thread's own archive and sharing the
admission slot with document search so the two cannot race for the embedder.
The tool is offered only once a thread has actually had turns archived, so an
ordinary short chat never pays for the schema, and it is classified read-only
so auto mode does not prompt on every call. A matching system-prompt note tells
the model the session was compacted, since otherwise it assumes the
conversation began where its visible context begins.
Deleting a thread now drops its archive rather than leaking a scope per chat.
* Studio: read thread_id defensively when selecting tools
_select_request_tools also serves the token-count request model, which has no
thread_id field, so the archive gate raised there.
* Studio: retrieve archived turns lexically first
Recalling your own conversation is mostly an exact-match problem: a name, a
number, an identifier someone pasted twenty turns ago. Those live or die on
rare-token matching, and hybrid fusion was losing them.
Measured on a 30-turn walkthrough of a 230k-character document at a 16k window,
where every turn shared the same wrapper text. The chunk holding the needle
ranked 3rd lexically at any k, was never returned by dense retrieval at all,
and RRF pushed it to 16th because it had 30 useless dense hits to fuse with.
End to end the model answered with the exact code once lexical leads, and could
not answer at all before.
Dense still fills whatever the lexical pass leaves, for paraphrased recall.
* Studio: keep recall on the branch the user is actually on
Editing an earlier message rewinds a thread and continues down a new branch,
but the archive is append-only and still holds everything the abandoned
continuation produced. Verified against a live build: after rewinding past a
turn, querying its distinctive text still returned it, so the model could be
handed a turn that on this branch never happened.
Recall now drops archived turns that are absent from the thread's saved
transcript. Threads with no saved transcript are left unfiltered, since an API
caller may pass a thread_id without persisting messages and an empty transcript
is absence of evidence rather than evidence the turns are gone.
Containment on a normalised prefix rather than a digest, because the archived
copy is rendered from the inference projection and the saved copy comes back
through the message store.
* Studio: show a persistent compaction notice on the turn that compacted
The only signal that a long chat had been compacted was a toast, which vanishes after
a few seconds and does not survive a reload. A user who scrolls back later has no way
to find out why the model seemed to forget the start of the conversation.
The notice renders from metadata.custom.contextTruncation inside the assistant
message's own container, so it is not part of the conversation sent to the model, is
not editable, and is not exported as content, but it stays attached to the turn it
describes. It reports how many messages were dropped and, when the conversation
archive is on, that they are still searchable and how many passages were recalled.
* Studio: show the compaction notice once, on the turn it started
A thread that has outgrown its window compacts on every turn from then on, not
just the first, so a notice per compacted turn was a notice on every reply for the
rest of the conversation. The user needs telling once.
The notice is now gated on being the first compacted assistant turn in the thread,
found by walking the thread rather than assuming the compacted turns are contiguous
or that this is the last one, so a rollback that removes the turn it was on moves it
to whichever turn now compacts first. The wording follows: it describes the state
the conversation is in from here on, and carries the counts from the turn it began
on in parentheses.
* Studio: make compaction an occasional event instead of every turn
The fit was stateless. The client re-sends the whole saved transcript on every
request, so "keep the newest N tokens" recomputes from scratch each time and slides
forward a turn or two at a time. Measured on a 40-turn thread against an 8k window,
the eviction boundary moved on 12 of 40 turns, which means every few replies quietly
lost a little more of the conversation, llama-server's prefix cache was thrown away
each time the head of the prompt moved, and there was no such thing as a compaction
event to tell the user about.
Two changes make it discrete. The fit now reads back the boundary the thread last
compacted to, from the newest assistant turn's own persisted truncation, and reapplies
it before deciding anything, so nothing new is stored and it survives a restart. And
when the boundary does have to move, the trim takes a further ROLLING_COMPACTION_
HEADROOM_RATIO of the budget out (default 0.25) rather than skimming to the brim, so
the new boundary has room to stay put. Same 40-turn thread, same window: 4 compactions
over 60 turns instead of 14, keeping about 82 percent of the usable budget.
Both are gated on the prompt not already fitting, exactly as the recall reserve is. A
conversation inside its window is never evicted to satisfy either, and a stale boundary
from a branch that was rolled back cannot evict a chat that now fits.
The notice follows: it is shown when dropped_messages rises above the last turn that
reported it, so it appears once per compaction and stays quiet in between.
* Studio: pin that the compaction notice can never become conversation
The notice is a sidecar rendered from metadata, not a message, and the ways that
could quietly stop being true are all one careless edit away: moving it inside
MessagePrimitive.Parts would make it a content part, and everything that walks parts
would then replay it to the model, copy it and export it.
Asserts it renders as a sibling of the content parts, that neither the outbound
message builder nor the assistant replay serialiser reads the key it renders from
(bounded to those function bodies, since the streaming handler reads the same key
legitimately on the way in), that the markdown export does not mention it, and that
it is suppressed while editing so it cannot be typed into the textarea and saved
back as text.
* Studio: say which part is too long when nothing can make a turn fit
If the message just sent is itself bigger than the window, no amount of eviction
helps. The fit already handled that correctly, returning the conversation untouched
so the request reaches llama-server's normal context-length error, but what the user
was then told was actively misleading: the error reports the size of the WHOLE
conversation and advises shortening it, when the history has already been evicted and
the single message is the part that does not fit. Measured at a 4096-token window: a
5000-token message produces 'Message too long: 10290 tokens ... shorten the
conversation', and shortening it cannot possibly work.
The fit now returns a fits:false diagnosis instead of a bare None, carrying what the
conversation could not be reduced below and how much of that is the latest turn.
Every consumer already gated on fits, so this is inert wherever a truncation is
treated as a compaction; the streaming paths now forward it so the client can use it.
The toast reads the diagnosis and, when the latest turn alone exceeds the window,
says so with the numbers instead of offering advice that leads nowhere. The merge
drops the diagnosis once a later refit succeeds, by delete rather than by assigning
undefined, so an ordinary response keeps exactly the shape it had before.
* Studio: fix the review findings on the conversation archive
Fifteen items, each reproduced against the code before changing anything.
Correctness in the request path. The fit reported a successful recall-capable fit
whenever the result was under the prompt budget, but protected messages can stop the
trim reaching the reserve target, and the recall then went in anyway: reproduced at
ctx 8000, the fit accepted at 6900 and recall took the request to 8948, past the
window it had just been made to fit. Recall is now sized from the room the fit
actually obtained, and skipped when there is none. Separately, the sticky boundary
describes the original transcript, so re-applying it on a later tool-loop fit evicted
another boundary-sized block of live history: measured, a second fit dropped 28 of the
30 surviving messages instead of 14, and the summed count persisted an inflated
boundary for the next request. It is now spent after the first fit of a request.
Availability and privacy. enabled() trusted RAG_AVAILABLE, which only records that
import sqlite_vec worked; rag_available() exists because the native vec0 library it
loads is a separate file a venv can lack. On such a machine the fit held a recall
reserve back, evicting extra history, and then both the archive write and the recall
failed, so the user paid for content they never got. And a temporary chat is never
written to studio.db, yet the frontend still sends its thread_id and the request
carries no incognito flag, so its turns were archived to a scope no deletion flow
could reach. Archival now requires the thread to be persisted, which is the same rule
that keeps every archive reachable by a delete. Clear-history and project deletion
drop archives too; only DELETE /threads did.
Archive integrity. The document was committed before its chunks, so a failed chunk
write left an empty row marked completed that document_by_hash then skipped forever;
both now go in one transaction. The live-branch filter accepted a turn on its first
matching line, so editing only the assistant half kept serving the old answer; the
whole turn must be present. Retrieval fetched exactly k before that filter, so stale
turns could starve live ones and recall returned nothing; it over-fetches first. Tool
turns archived only the tool name, which cannot answer what was actually run, so a
bounded rendering of the arguments and the assistant text goes in as well.
The tool surface. Studio always sends an explicit enabled_tools array and has no
reason to name an internal tool, so the allowlist filter removed search_conversation
before the archive gate ran and the tool, plus the compaction nudge gated on it, never
appeared in a Studio chat. It now follows the archive rather than the allowlist. The
forced recall rendered a tool exchange even when the tool was absent from the
catalogue, which is the strict-template hazard the plain path avoids; it picks inline
in that case, and always inline for the final-answer request, which sends no tools at
all. A model-supplied top_k reached a slice as out[:-1] and returned nearly the whole
candidate pool, so it is clamped. Both retrieval tools now share the per-turn search
cap; only the knowledge-base one was counted.
The UI. A fits:false diagnosis is the fitter reporting it could NOT fit, so toasting
that older turns were removed was untrue and burned the once-per-thread flag a later
real compaction needed. The too-long advice compared the latest turn against the raw
context length rather than the prompt budget, so a 3,500-token message in a
4,096-token window was still told to start a new chat, which fails identically.
* Studio: fix the follow-up findings on the archive fixes
Four items, three of them about last round's own fixes.
Archived tool turns had become permanently unrecallable. render_turn now writes
'assistant called X: args' and 'tool result: ...' lines, while assistant-ui persists a
tool call as a structured tool-call content part that the transcript flattener dropped;
with every archived line required to appear in that transcript, no tool turn could ever
match. The transcript now flattens toolName, args and result, and the probe strips the
'assistant called <name>:' label, which is ours rather than the stored message's.
The branch probe compared only the first 160 normalized characters of each line, so an
edit to the tail of a long answer left the stale copy eligible. Ordinary lines are now
compared whole; tool results keep a prefix, since render_turn deliberately truncates
those and the archived copy is not meant to equal the stored one.
The forced recall sized itself by dividing the remaining budget by CHUNK_TOKENS, which
is an embedding-token limit rather than the chat template's cost, and prices none of
the wrappers around the injection. It is now recounted with the same tokenizer the fit
used and dropped if it overshoots, so the estimate can no longer eat the reply reserve.
An omitted top_k on search_conversation defaulted to the clamp ceiling of eight rather
than the configured recall default, so an ordinary search could return eight archived
turns into the protected current exchange that rolling truncation cannot evict.
* Studio: filter conversation recall to the active branch
A thread's stored rows are the whole message DAG. Retry and regenerate keep the
replaced response as a sibling on purpose, so filtering recall against the whole
thread cannot tell a live turn from one the user replaced, and an archived copy of
the abandoned response could be recalled into a branch where it never happened.
Filter against the messages the request was actually sent with instead, which is
one branch by construction, and hand the same branch to search_conversation so the
model cannot ask for what the forced recall refused. Falls back to the thread-wide
blob for a caller with no branch to offer.
Also retunes two respawn-refit fixtures whose windows no longer produced two
compactions after compaction started trimming a headroom margin below the budget.
* Studio: survive a delete mid-archive, and read the boundary off the active branch
Deleting a chat cancels its generation, but cancellation is cooperative and the
chunk-and-embed pass between the archive's liveness check and its commit does not
observe it. A delete landing in that window drops the thread's rows and sweeps its
scope before the commit puts rows back, leaving content the user deleted in a scope
no later delete can reach. Re-check after the commit and drop the scope: the delete
route removes rows first and sweeps archives last, so either order converges.
The sticky compaction boundary had the same thread-wide read as recall did. The
stored rows are the whole DAG ordered by creation time, so after a Retry the newest
assistant turn can be the sibling the user switched away from, and its boundary is
sized for history the active branch does not have. Resolve it against the request's
own messages instead.
* Studio: give the safetensors loop the same conversation-search guards
search_conversation is advertised by thread, not by backend: the tool selector is
shared, so a chat compacted under a GGUF model still offers it after the user
switches to a safetensors one. The safetensors loop had neither guard the GGUF loop
applies to it. It passed no active branch, so a search there fell back to the
thread-wide rows and could answer from a branch Retry left behind, and it capped
only search_knowledge_base, so paraphrased conversation searches could append
archived passages into the protected current exchange on every iteration until the
window failed. The shared set of capped retrieval tools now lives beside the cap.
The forced recall also derived its query from the loop conversation, which on a
later iteration can end with an internal user-role re-prompt rather than anything
the user wrote. It reads the request branch's own latest user turn instead.
* Studio: widen recall past an abandoned branch, and name whose turn overflowed
One over-fetch is not enough for the live-branch filter. Rewinding or retrying a
continuation that had already been compacted leaves enough stale turns to fill any
fixed candidate window, and the whole page is then rejected while the live match
sitting just below it is never examined, so recall reports nothing although the
answer is in the archive. Widen and re-ask instead, stopping as soon as there are
enough live hits, when the archive stops yielding candidates, or at a bound.
The irreducible-fit diagnosis also carried the size of the last message without
saying whose it was. A tool loop refits with the tool result appended, so that turn
is often output the user never wrote and cannot edit, and the client told them to
shorten it. It now reports the role, and the advice splits on it.
* Studio: shrink an over-budget recall, and keep truncated tool turns on their branch
The exact recount is the right gate, but dropping the whole recall when it fails is
the wrong response: with the shipped defaults a full top-K of long turns lands just
over the reserve once the wrappers are priced, which would disable the forced
retrieval on exactly the long conversations it exists for. Halve the number of turns
and re-ask instead, down to one, before giving up.
The live-branch probe keyed its prefix rule off the tool-result label, but a long
tool result is one appended string containing many newlines, so only its first line
carries that label. Continuation lines were compared in full, including the last one,
which is the only line the truncation marker is on and can therefore never appear in
a transcript: every archived tool turn over the cap was rejected as rolled back.
Key off the marker instead, which also covers truncated tool arguments and compares
the full text everywhere else.
* Studio: retire an edited turn's whole archived copy, not just the edited chunk
A turn longer than CHUNK_TOKENS is stored as several chunks of one document, and the
live-branch filter ran per chunk. Editing the second half of a long answer therefore
retired only the chunks carrying the edit, and an untouched earlier chunk of the same
retired turn stayed eligible on its own. The unit that was archived is the turn, which
is what the filter already claimed to enforce.
Validate every chunk of the candidate's document before admitting any of it, cached
per call since candidates from one turn share a document. A query failure falls back
to the per-chunk answer rather than failing the recall.
* Studio: do not hold a recall reserve back for a chat that is never archived
archive_turns refuses a thread with no saved messages, because a temporary chat must
not be persisted into a scope no deletion flow can reach. The reserve did not follow
that rule: it was granted whenever the RAG stack was available, so an incognito chat,
or an API client sending a thread_id without saving anything, paid a full 2,048-token
reserve for content that can never arrive. The fit subtracts the reserve from its trim
target, so that room is bought with evicted history. Measured on a 4K window: 15
tokens of conversation survived a compaction instead of 1,615.
Both now ask one predicate, can_archive, backed by an existence probe rather than a
row load.
* Studio: match archived turns in order, and delete them without sqlite-vec
Independent line membership accepts a turn whose lines were merely rearranged: every
probe still occurs somewhere, so the pre-edit ordering stayed eligible and would be
served back as what happened. Match the probes in order instead.
That only works if both sides agree on the order, and they did not: render_turn writes
a tool call before any assistant text on the same message and the result after it,
while both transcript builders wrote the text first, so a tool turn carrying both was
rejected outright. One flattener now lays out a message the way render_turn does, for
the request shape and the stored shape alike, and it offers both JSON spacings for a
stored call's arguments, whose object form is not the string the model emitted.
Deletion no longer depends on the optional native extension. An archive is only written
while vec0 loads, but the library can stop loading afterwards, and a delete that quietly
did nothing left a deleted conversation's turns on disk to answer again once it loaded.
The fallback removes the text-bearing rows over a metadata connection; the embedding
rows it cannot reach carry no text and resolve through tables that are gone.
* Studio: give the provider loops the branch, and keep the thread across a respawn
The provider tool loops take their catalogue from the same selector as the local ones,
so search_conversation is advertised there too once a thread has an archive, but the
loop passed no active branch and its searches fell back to the whole stored DAG, where
Retry keeps the response it replaced. This is the third loop to need the same wiring.
The respawn retry also dropped the thread. It re-enables context_overflow on purpose,
to refit an already-compacted prompt for a replacement window, so it is the one path
that deliberately compacts a second time: without the thread those extra evictions were
archived nowhere, nothing was recalled in their place, and the fit held back no reserve
and re-applied no boundary.
* Studio: keep the reply that follows a recall, and archive respawn refits
group_turns keeps an assistant tool call, its result and the reply that follows in one
group, and the archiver rejected any group containing one of our own injections. So on
a turn that forced a recall, the model's actual answer was thrown away with it: the
question was archived from its own group and the answer was not, and a later search
could find what was asked and never what was said. The injections come out now and the
rest of the turn stays; retrieved passages are still kept out of the index they came
from.
The two respawn refits inside the tool loop also evicted without archiving. They run
against a smaller replacement window, so those turns are simply gone otherwise. They
archive only, deliberately: no reserve is held back on that path, so injecting a recall
there is what would push the retry back over the window, and the next request can still
recall what this one archived.
* Studio: bound the branch check to one turn, and anchor only what recall injected
The branch check searched one flattened transcript, so a line an edit removed could be
supplied by any later message that happened to repeat the words. Short answers repeat
constantly: an archived "Should I deploy? / No" survived its answer being edited to
"Yes" because a later turn said "No", and the stale pair stayed recallable. The
transcript is now one normalised string per message, and a turn matches only if its
lines appear in order within a run of adjacent messages no longer than the turn itself.
Recall anchoring took the last two messages of the conversation. That is right for the
tool style, which appends a synthetic pair, and wrong for the inline style, which
appends nothing and rewrites the latest user message in place: it also pinned the
assistant turn before it, and with it an eviction unit the fit was entitled to drop,
which can fail a later iteration that would otherwise have fit. The injection now
reports exactly what it added or rewrote, and only that is anchored.
* Studio: validate a turn's chunks against one run of the branch, not each on its own
The chunks of an archived turn are consecutive slices of a single rendering, and each was
checked for itself. That let a turn be reassembled out of parts that never sat together:
the head matching the question and the answer it has now, the tail matching some later
message that happens to repeat the passage an edit removed, so the stale association
stayed recallable. Reproduced before the change, with each chunk passing on its own.
All chunks of the document must now be found within one run of adjacent messages, bounded
by the document's own line count, which is at least the number of messages the turn was
rendered from, so a turn that really is still there always fits.
* Studio: budget conversation searches and disambiguate identical replies
search_conversation clamped the model's top_k only against a fixed ceiling of 8.
Eight chunks is roughly 4,000 tokens once wrapped, and the result lands in the
current tool exchange, which rolling truncation protects and cannot evict, so on
a small context the search itself made the turn unsendable. The GGUF loop now
passes the room the window actually has left, the tool clamps top_k by it, and a
search with no room says so instead of returning a result that cannot be sent.
The sticky compaction boundary took the newest on-branch assistant row, but the
branch check is textual, so two siblings whose replies read the same ("Done.")
are indistinguishable from there, and Retry is exactly what produces them.
Taking the first match applied a boundary measured on a different, deeper branch
and evicted live history. Where the text cannot separate them, the smallest
boundary is now used: too small costs one more compaction, too large loses turns
the branch still has.
* Studio: price a conversation search against the whole prompt
Three gaps in the budget the previous commit introduced.
The GGUF loop subtracted an estimate of the messages alone, leaving the tool
catalogue out of the prompt entirely. A large catalogue is thousands of tokens,
so the request could already be near its budget while the search was still told
there was room for several 500-token chunks. The preflight already prices the
request exactly, catalogue and template included, so the difference between that
count and the estimate of the same messages is carried forward and the estimate
only covers what the loop appends after it.
The safetensors loop advertises and executes search_conversation for a thread
compacted under a GGUF model, but forwarded no budget, so the clamp in the tool
was skipped there. It now passes the room this model has left. With no known
context length the argument is omitted rather than sent as zero, which would
refuse every search.
The sticky boundary filtered candidates with a substring test, which is right
for archived chunks and wrong for whole messages: an abandoned "Done" rode in on
a live "Not done yet", and having no live twin it then decided the boundary
alone. Where any candidate matches a live message exactly, only the exact ones
are considered; where none does, the old behaviour stands, since a stored row is
not always byte-identical to what the client re-sends.
* Studio: keep the overflow diagnosis on the tool path
The irreducible-overflow branch counted the newest message on its own, and a
tool loop reaches that branch with a tool result last. A tool result by itself
is not a conversation: templates that require it to follow its assistant tool
call refuse to render one, and the exception escaped the fit entirely, so the
caller fell back to the untrimmed request and the client was told nothing at
all. That is the one path this diagnosis was added for.
The count is now attempted and falls back to the estimator when the template
refuses. An approximate number is worth more here than a diagnosis that never
arrives.
* Studio: budget the conversation search in the provider loop
The third tool loop forwarded the active branch but no budget, so the clamp in
the tool was skipped there and a model-chosen top_k of 8 could append roughly 4K
tokens to a prompt this loop replays on its next call.
Studio knows no window for an external model: the request carries no context
length and there is no registry to look one up in, and a custom
OpenAI-compatible endpoint can be a small local server. So rather than a
measured budget, this path spends no more than one ordinary recall's worth,
which is the same amount the compaction turn itself is sized for.
* Studio: bound an archived turn to its own messages, and archive it once
Two problems in the archive.
The whole-document branch check bounded the run by the number of LINES the turn
produced, and gave each chunk its own start. A turn is two or three messages
however long it is, so a hundred-line answer got a hundred-message window, and
the tail of an edited answer could be satisfied by a message well outside the
turn. The run is now bounded by the messages the turn was rendered from, counted
from the labels render_turn writes, and the chunks are scanned as one pass: each
continues where the previous one stopped rather than restarting. The cursor
inside a message is deliberately not carried over, because chunks overlap and a
continuation chunk repeats the tail of the one before it.
The hash check that skips an already-archived turn ran long before the insert,
with the embedding pass in between, and the index on (scope, sha256) is not
unique. Two generations compacting the same thread both cleared it and both
wrote, so the turn was stored twice and its copies took two of the few recall
slots. The check is now repeated under a write lock immediately before the
insert. Reproduced with two concurrent archive passes: two documents before, one
after.
* Studio: record how many messages an archived turn came from
The run an archived turn is allowed to occupy on the branch was bounded by
counting the role labels in its rendered text. That counts lines the user wrote
as well as the ones the renderer did: a pasted chat log carries lines that look
exactly the same, and each one widens the run by a message, which is enough for
the message after an edited turn to supply the passage the edit removed.
The group's size is now recorded on the document when it is archived, in a
nullable column added the way the other lazy upgrades are. Archives written
before this have NULL and fall back to the label count, so nothing needs
backfilling to keep working.
* Studio: persist a compaction boundary the next request can use
The boundary was read back from dropped_messages, which counts what each fit
removed from the conversation in front of it. The tool loop refits on every
iteration and the client sums those counts, so a long agent run added the tool
exchanges the turn itself created, and the next request applied the total to its
saved transcript. Reproduced with six tool calls on a 4K window: three fits of 4
summed to 12, on a branch that only ever had 4 evictable messages.
The boundary is now carried separately, measured against the messages the
request was sent with, so it is absolute and re-sending it cannot advance it.
The client keeps the latest value rather than summing, and turns saved before
this fall back to the dropped count, which is the same number for a turn that
fit once.
The recall reserve is also dropped once archiving has failed. sqlite-vec can be
present and the thread saved while the embedder cannot start: archive_turns
swallows that and recall injects nothing, so the room the fit held back was pure
loss on every compaction, and the failure mode forgot more history than having
the feature off. A failed write marks the archive degraded and the next
successful one clears it.
* Studio: count the boundary past the system prompt, and notice it
Two faults in the boundary added in the previous commit.
It stopped at the first message still present, and a Studio request always
starts with a system prompt that a fit never evicts, so every compaction
recorded a boundary of zero. That is the same as having no boundary: the next
request would move the eviction point again and invalidate the prefix cache on
every turn. Instruction messages are now skipped rather than treated as the
front of the branch, and the newest turn is excluded because it is never evicted
and an inline recall rewrites it in place.
The compaction notice still keyed off dropped_messages, which is the
accumulated count. A tool-heavy turn reporting 12 while the boundary moved to 4
set a high-water mark that silenced the next two real advances. It now reads the
boundary, falling back to the dropped count only for turns saved before the
boundary was recorded.
* Studio: tighten comments in the conversation archive and rolling context window
* Studio: give the pasted-text import an extension on this branch too
The node test runner cannot resolve an extensionless relative import, so
delete-chat-files-preference fails to load the preferences store. main fixed
this in
|
||
|
Daniel Han
|
18b97f8b72
|
Studio: keep and search the turns rolling context evicts (#9074)
* Studio: add rolling context windows for local GGUF chat
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: return rolling context metadata for non-stream chats
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep original context when rolling fit fails
* Studio: keep instruction groups independently protected
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: preserve rolling context metadata across retries
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: count sanitized rolling context prompts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: refit rolling context after respawn
* Studio: scope middle truncation to passthrough
* Studio: refit tool prompts after respawn
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: retain later choice truncation metadata
* Studio: report clipping-only context truncation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: expose the turns a rolling fit evicts
Rolling context eviction currently drops turns with no way for a caller to
learn which ones went. Extract the turn grouping so the eviction unit is
reusable, add an identity-based diff of what a fit removed, and let a caller
reserve room for content it intends to add back after fitting.
The reserve deliberately does not affect whether trimming happens at all, only
how far it goes once it is already required, so a conversation that fits today
is still returned untouched.
* Studio: archive evicted turns into a per-thread rag scope
An evicted turn is currently gone for the rest of the session, so the model
will state the conversation began wherever its visible context begins. Keep the
turns the rolling window drops in a searchable scope built on the existing
store, chunker, embedder and hybrid retrieval.
The archive is cumulative: every compaction adds to it and nothing is cleared,
so a later compaction can still find what an earlier one evicted. It lives in
its own scope rather than the thread's document scope, because thread documents
are injected in full on every request and would re-inject the whole history.
Idempotent by content hash, since the same turns are evicted again on every
later request. Every entry point degrades to a no-op rather than raising.
* Studio: recall archived turns on the turn that evicted them
Given only a search tool, a model decides for itself whether to look, and
mostly does not. Measured on MRCR v2, a 35B declined on 56% of rows, scoring
0.099 when it skipped against 0.461 when it searched. Forcing one retrieval on
the compaction turn took tool-only 0.258 to 0.604, and the model then called
the tool on 0% of rows, so the common path costs nothing extra.
Recall fires at most once per request, since the tool loop refits on every
iteration. The tool loop renders it as an ordinary tool exchange through the
builder shared with document auto-inject; the plain path prefixes the latest
user message instead, because it sends no tools array and a tool role without
one breaks strict chat templates.
Only scalar counts join the context_truncated event, so no message content
reaches the wire.
* Studio: let the model search a compacted conversation
Forced recall answers the turn that evicted, but a later turn can refer to
something the forced pass had no reason to fetch. Add search_conversation so
the model can go looking, scoped to the thread's own archive and sharing the
admission slot with document search so the two cannot race for the embedder.
The tool is offered only once a thread has actually had turns archived, so an
ordinary short chat never pays for the schema, and it is classified read-only
so auto mode does not prompt on every call. A matching system-prompt note tells
the model the session was compacted, since otherwise it assumes the
conversation began where its visible context begins.
Deleting a thread now drops its archive rather than leaking a scope per chat.
* Studio: read thread_id defensively when selecting tools
_select_request_tools also serves the token-count request model, which has no
thread_id field, so the archive gate raised there.
* Studio: retrieve archived turns lexically first
Recalling your own conversation is mostly an exact-match problem: a name, a
number, an identifier someone pasted twenty turns ago. Those live or die on
rare-token matching, and hybrid fusion was losing them.
Measured on a 30-turn walkthrough of a 230k-character document at a 16k window,
where every turn shared the same wrapper text. The chunk holding the needle
ranked 3rd lexically at any k, was never returned by dense retrieval at all,
and RRF pushed it to 16th because it had 30 useless dense hits to fuse with.
End to end the model answered with the exact code once lexical leads, and could
not answer at all before.
Dense still fills whatever the lexical pass leaves, for paraphrased recall.
* Studio: keep recall on the branch the user is actually on
Editing an earlier message rewinds a thread and continues down a new branch,
but the archive is append-only and still holds everything the abandoned
continuation produced. Verified against a live build: after rewinding past a
turn, querying its distinctive text still returned it, so the model could be
handed a turn that on this branch never happened.
Recall now drops archived turns that are absent from the thread's saved
transcript. Threads with no saved transcript are left unfiltered, since an API
caller may pass a thread_id without persisting messages and an empty transcript
is absence of evidence rather than evidence the turns are gone.
Containment on a normalised prefix rather than a digest, because the archived
copy is rendered from the inference projection and the saved copy comes back
through the message store.
* Studio: show a persistent compaction notice on the turn that compacted
The only signal that a long chat had been compacted was a toast, which vanishes after
a few seconds and does not survive a reload. A user who scrolls back later has no way
to find out why the model seemed to forget the start of the conversation.
The notice renders from metadata.custom.contextTruncation inside the assistant
message's own container, so it is not part of the conversation sent to the model, is
not editable, and is not exported as content, but it stays attached to the turn it
describes. It reports how many messages were dropped and, when the conversation
archive is on, that they are still searchable and how many passages were recalled.
* Studio: show the compaction notice once, on the turn it started
A thread that has outgrown its window compacts on every turn from then on, not
just the first, so a notice per compacted turn was a notice on every reply for the
rest of the conversation. The user needs telling once.
The notice is now gated on being the first compacted assistant turn in the thread,
found by walking the thread rather than assuming the compacted turns are contiguous
or that this is the last one, so a rollback that removes the turn it was on moves it
to whichever turn now compacts first. The wording follows: it describes the state
the conversation is in from here on, and carries the counts from the turn it began
on in parentheses.
* Studio: make compaction an occasional event instead of every turn
The fit was stateless. The client re-sends the whole saved transcript on every
request, so "keep the newest N tokens" recomputes from scratch each time and slides
forward a turn or two at a time. Measured on a 40-turn thread against an 8k window,
the eviction boundary moved on 12 of 40 turns, which means every few replies quietly
lost a little more of the conversation, llama-server's prefix cache was thrown away
each time the head of the prompt moved, and there was no such thing as a compaction
event to tell the user about.
Two changes make it discrete. The fit now reads back the boundary the thread last
compacted to, from the newest assistant turn's own persisted truncation, and reapplies
it before deciding anything, so nothing new is stored and it survives a restart. And
when the boundary does have to move, the trim takes a further ROLLING_COMPACTION_
HEADROOM_RATIO of the budget out (default 0.25) rather than skimming to the brim, so
the new boundary has room to stay put. Same 40-turn thread, same window: 4 compactions
over 60 turns instead of 14, keeping about 82 percent of the usable budget.
Both are gated on the prompt not already fitting, exactly as the recall reserve is. A
conversation inside its window is never evicted to satisfy either, and a stale boundary
from a branch that was rolled back cannot evict a chat that now fits.
The notice follows: it is shown when dropped_messages rises above the last turn that
reported it, so it appears once per compaction and stays quiet in between.
* Studio: pin that the compaction notice can never become conversation
The notice is a sidecar rendered from metadata, not a message, and the ways that
could quietly stop being true are all one careless edit away: moving it inside
MessagePrimitive.Parts would make it a content part, and everything that walks parts
would then replay it to the model, copy it and export it.
Asserts it renders as a sibling of the content parts, that neither the outbound
message builder nor the assistant replay serialiser reads the key it renders from
(bounded to those function bodies, since the streaming handler reads the same key
legitimately on the way in), that the markdown export does not mention it, and that
it is suppressed while editing so it cannot be typed into the textarea and saved
back as text.
* Studio: say which part is too long when nothing can make a turn fit
If the message just sent is itself bigger than the window, no amount of eviction
helps. The fit already handled that correctly, returning the conversation untouched
so the request reaches llama-server's normal context-length error, but what the user
was then told was actively misleading: the error reports the size of the WHOLE
conversation and advises shortening it, when the history has already been evicted and
the single message is the part that does not fit. Measured at a 4096-token window: a
5000-token message produces 'Message too long: 10290 tokens ... shorten the
conversation', and shortening it cannot possibly work.
The fit now returns a fits:false diagnosis instead of a bare None, carrying what the
conversation could not be reduced below and how much of that is the latest turn.
Every consumer already gated on fits, so this is inert wherever a truncation is
treated as a compaction; the streaming paths now forward it so the client can use it.
The toast reads the diagnosis and, when the latest turn alone exceeds the window,
says so with the numbers instead of offering advice that leads nowhere. The merge
drops the diagnosis once a later refit succeeds, by delete rather than by assigning
undefined, so an ordinary response keeps exactly the shape it had before.
* Studio: fix the review findings on the conversation archive
Fifteen items, each reproduced against the code before changing anything.
Correctness in the request path. The fit reported a successful recall-capable fit
whenever the result was under the prompt budget, but protected messages can stop the
trim reaching the reserve target, and the recall then went in anyway: reproduced at
ctx 8000, the fit accepted at 6900 and recall took the request to 8948, past the
window it had just been made to fit. Recall is now sized from the room the fit
actually obtained, and skipped when there is none. Separately, the sticky boundary
describes the original transcript, so re-applying it on a later tool-loop fit evicted
another boundary-sized block of live history: measured, a second fit dropped 28 of the
30 surviving messages instead of 14, and the summed count persisted an inflated
boundary for the next request. It is now spent after the first fit of a request.
Availability and privacy. enabled() trusted RAG_AVAILABLE, which only records that
import sqlite_vec worked; rag_available() exists because the native vec0 library it
loads is a separate file a venv can lack. On such a machine the fit held a recall
reserve back, evicting extra history, and then both the archive write and the recall
failed, so the user paid for content they never got. And a temporary chat is never
written to studio.db, yet the frontend still sends its thread_id and the request
carries no incognito flag, so its turns were archived to a scope no deletion flow
could reach. Archival now requires the thread to be persisted, which is the same rule
that keeps every archive reachable by a delete. Clear-history and project deletion
drop archives too; only DELETE /threads did.
Archive integrity. The document was committed before its chunks, so a failed chunk
write left an empty row marked completed that document_by_hash then skipped forever;
both now go in one transaction. The live-branch filter accepted a turn on its first
matching line, so editing only the assistant half kept serving the old answer; the
whole turn must be present. Retrieval fetched exactly k before that filter, so stale
turns could starve live ones and recall returned nothing; it over-fetches first. Tool
turns archived only the tool name, which cannot answer what was actually run, so a
bounded rendering of the arguments and the assistant text goes in as well.
The tool surface. Studio always sends an explicit enabled_tools array and has no
reason to name an internal tool, so the allowlist filter removed search_conversation
before the archive gate ran and the tool, plus the compaction nudge gated on it, never
appeared in a Studio chat. It now follows the archive rather than the allowlist. The
forced recall rendered a tool exchange even when the tool was absent from the
catalogue, which is the strict-template hazard the plain path avoids; it picks inline
in that case, and always inline for the final-answer request, which sends no tools at
all. A model-supplied top_k reached a slice as out[:-1] and returned nearly the whole
candidate pool, so it is clamped. Both retrieval tools now share the per-turn search
cap; only the knowledge-base one was counted.
The UI. A fits:false diagnosis is the fitter reporting it could NOT fit, so toasting
that older turns were removed was untrue and burned the once-per-thread flag a later
real compaction needed. The too-long advice compared the latest turn against the raw
context length rather than the prompt budget, so a 3,500-token message in a
4,096-token window was still told to start a new chat, which fails identically.
* Studio: fix the follow-up findings on the archive fixes
Four items, three of them about last round's own fixes.
Archived tool turns had become permanently unrecallable. render_turn now writes
'assistant called X: args' and 'tool result: ...' lines, while assistant-ui persists a
tool call as a structured tool-call content part that the transcript flattener dropped;
with every archived line required to appear in that transcript, no tool turn could ever
match. The transcript now flattens toolName, args and result, and the probe strips the
'assistant called <name>:' label, which is ours rather than the stored message's.
The branch probe compared only the first 160 normalized characters of each line, so an
edit to the tail of a long answer left the stale copy eligible. Ordinary lines are now
compared whole; tool results keep a prefix, since render_turn deliberately truncates
those and the archived copy is not meant to equal the stored one.
The forced recall sized itself by dividing the remaining budget by CHUNK_TOKENS, which
is an embedding-token limit rather than the chat template's cost, and prices none of
the wrappers around the injection. It is now recounted with the same tokenizer the fit
used and dropped if it overshoots, so the estimate can no longer eat the reply reserve.
An omitted top_k on search_conversation defaulted to the clamp ceiling of eight rather
than the configured recall default, so an ordinary search could return eight archived
turns into the protected current exchange that rolling truncation cannot evict.
* Studio: filter conversation recall to the active branch
A thread's stored rows are the whole message DAG. Retry and regenerate keep the
replaced response as a sibling on purpose, so filtering recall against the whole
thread cannot tell a live turn from one the user replaced, and an archived copy of
the abandoned response could be recalled into a branch where it never happened.
Filter against the messages the request was actually sent with instead, which is
one branch by construction, and hand the same branch to search_conversation so the
model cannot ask for what the forced recall refused. Falls back to the thread-wide
blob for a caller with no branch to offer.
Also retunes two respawn-refit fixtures whose windows no longer produced two
compactions after compaction started trimming a headroom margin below the budget.
* Studio: survive a delete mid-archive, and read the boundary off the active branch
Deleting a chat cancels its generation, but cancellation is cooperative and the
chunk-and-embed pass between the archive's liveness check and its commit does not
observe it. A delete landing in that window drops the thread's rows and sweeps its
scope before the commit puts rows back, leaving content the user deleted in a scope
no later delete can reach. Re-check after the commit and drop the scope: the delete
route removes rows first and sweeps archives last, so either order converges.
The sticky compaction boundary had the same thread-wide read as recall did. The
stored rows are the whole DAG ordered by creation time, so after a Retry the newest
assistant turn can be the sibling the user switched away from, and its boundary is
sized for history the active branch does not have. Resolve it against the request's
own messages instead.
* Studio: give the safetensors loop the same conversation-search guards
search_conversation is advertised by thread, not by backend: the tool selector is
shared, so a chat compacted under a GGUF model still offers it after the user
switches to a safetensors one. The safetensors loop had neither guard the GGUF loop
applies to it. It passed no active branch, so a search there fell back to the
thread-wide rows and could answer from a branch Retry left behind, and it capped
only search_knowledge_base, so paraphrased conversation searches could append
archived passages into the protected current exchange on every iteration until the
window failed. The shared set of capped retrieval tools now lives beside the cap.
The forced recall also derived its query from the loop conversation, which on a
later iteration can end with an internal user-role re-prompt rather than anything
the user wrote. It reads the request branch's own latest user turn instead.
* Studio: widen recall past an abandoned branch, and name whose turn overflowed
One over-fetch is not enough for the live-branch filter. Rewinding or retrying a
continuation that had already been compacted leaves enough stale turns to fill any
fixed candidate window, and the whole page is then rejected while the live match
sitting just below it is never examined, so recall reports nothing although the
answer is in the archive. Widen and re-ask instead, stopping as soon as there are
enough live hits, when the archive stops yielding candidates, or at a bound.
The irreducible-fit diagnosis also carried the size of the last message without
saying whose it was. A tool loop refits with the tool result appended, so that turn
is often output the user never wrote and cannot edit, and the client told them to
shorten it. It now reports the role, and the advice splits on it.
* Studio: shrink an over-budget recall, and keep truncated tool turns on their branch
The exact recount is the right gate, but dropping the whole recall when it fails is
the wrong response: with the shipped defaults a full top-K of long turns lands just
over the reserve once the wrappers are priced, which would disable the forced
retrieval on exactly the long conversations it exists for. Halve the number of turns
and re-ask instead, down to one, before giving up.
The live-branch probe keyed its prefix rule off the tool-result label, but a long
tool result is one appended string containing many newlines, so only its first line
carries that label. Continuation lines were compared in full, including the last one,
which is the only line the truncation marker is on and can therefore never appear in
a transcript: every archived tool turn over the cap was rejected as rolled back.
Key off the marker instead, which also covers truncated tool arguments and compares
the full text everywhere else.
* Studio: retire an edited turn's whole archived copy, not just the edited chunk
A turn longer than CHUNK_TOKENS is stored as several chunks of one document, and the
live-branch filter ran per chunk. Editing the second half of a long answer therefore
retired only the chunks carrying the edit, and an untouched earlier chunk of the same
retired turn stayed eligible on its own. The unit that was archived is the turn, which
is what the filter already claimed to enforce.
Validate every chunk of the candidate's document before admitting any of it, cached
per call since candidates from one turn share a document. A query failure falls back
to the per-chunk answer rather than failing the recall.
* Studio: do not hold a recall reserve back for a chat that is never archived
archive_turns refuses a thread with no saved messages, because a temporary chat must
not be persisted into a scope no deletion flow can reach. The reserve did not follow
that rule: it was granted whenever the RAG stack was available, so an incognito chat,
or an API client sending a thread_id without saving anything, paid a full 2,048-token
reserve for content that can never arrive. The fit subtracts the reserve from its trim
target, so that room is bought with evicted history. Measured on a 4K window: 15
tokens of conversation survived a compaction instead of 1,615.
Both now ask one predicate, can_archive, backed by an existence probe rather than a
row load.
* Studio: match archived turns in order, and delete them without sqlite-vec
Independent line membership accepts a turn whose lines were merely rearranged: every
probe still occurs somewhere, so the pre-edit ordering stayed eligible and would be
served back as what happened. Match the probes in order instead.
That only works if both sides agree on the order, and they did not: render_turn writes
a tool call before any assistant text on the same message and the result after it,
while both transcript builders wrote the text first, so a tool turn carrying both was
rejected outright. One flattener now lays out a message the way render_turn does, for
the request shape and the stored shape alike, and it offers both JSON spacings for a
stored call's arguments, whose object form is not the string the model emitted.
Deletion no longer depends on the optional native extension. An archive is only written
while vec0 loads, but the library can stop loading afterwards, and a delete that quietly
did nothing left a deleted conversation's turns on disk to answer again once it loaded.
The fallback removes the text-bearing rows over a metadata connection; the embedding
rows it cannot reach carry no text and resolve through tables that are gone.
* Studio: give the provider loops the branch, and keep the thread across a respawn
The provider tool loops take their catalogue from the same selector as the local ones,
so search_conversation is advertised there too once a thread has an archive, but the
loop passed no active branch and its searches fell back to the whole stored DAG, where
Retry keeps the response it replaced. This is the third loop to need the same wiring.
The respawn retry also dropped the thread. It re-enables context_overflow on purpose,
to refit an already-compacted prompt for a replacement window, so it is the one path
that deliberately compacts a second time: without the thread those extra evictions were
archived nowhere, nothing was recalled in their place, and the fit held back no reserve
and re-applied no boundary.
* Studio: keep the reply that follows a recall, and archive respawn refits
group_turns keeps an assistant tool call, its result and the reply that follows in one
group, and the archiver rejected any group containing one of our own injections. So on
a turn that forced a recall, the model's actual answer was thrown away with it: the
question was archived from its own group and the answer was not, and a later search
could find what was asked and never what was said. The injections come out now and the
rest of the turn stays; retrieved passages are still kept out of the index they came
from.
The two respawn refits inside the tool loop also evicted without archiving. They run
against a smaller replacement window, so those turns are simply gone otherwise. They
archive only, deliberately: no reserve is held back on that path, so injecting a recall
there is what would push the retry back over the window, and the next request can still
recall what this one archived.
* Studio: bound the branch check to one turn, and anchor only what recall injected
The branch check searched one flattened transcript, so a line an edit removed could be
supplied by any later message that happened to repeat the words. Short answers repeat
constantly: an archived "Should I deploy? / No" survived its answer being edited to
"Yes" because a later turn said "No", and the stale pair stayed recallable. The
transcript is now one normalised string per message, and a turn matches only if its
lines appear in order within a run of adjacent messages no longer than the turn itself.
Recall anchoring took the last two messages of the conversation. That is right for the
tool style, which appends a synthetic pair, and wrong for the inline style, which
appends nothing and rewrites the latest user message in place: it also pinned the
assistant turn before it, and with it an eviction unit the fit was entitled to drop,
which can fail a later iteration that would otherwise have fit. The injection now
reports exactly what it added or rewrote, and only that is anchored.
* Studio: validate a turn's chunks against one run of the branch, not each on its own
The chunks of an archived turn are consecutive slices of a single rendering, and each was
checked for itself. That let a turn be reassembled out of parts that never sat together:
the head matching the question and the answer it has now, the tail matching some later
message that happens to repeat the passage an edit removed, so the stale association
stayed recallable. Reproduced before the change, with each chunk passing on its own.
All chunks of the document must now be found within one run of adjacent messages, bounded
by the document's own line count, which is at least the number of messages the turn was
rendered from, so a turn that really is still there always fits.
* Studio: budget conversation searches and disambiguate identical replies
search_conversation clamped the model's top_k only against a fixed ceiling of 8.
Eight chunks is roughly 4,000 tokens once wrapped, and the result lands in the
current tool exchange, which rolling truncation protects and cannot evict, so on
a small context the search itself made the turn unsendable. The GGUF loop now
passes the room the window actually has left, the tool clamps top_k by it, and a
search with no room says so instead of returning a result that cannot be sent.
The sticky compaction boundary took the newest on-branch assistant row, but the
branch check is textual, so two siblings whose replies read the same ("Done.")
are indistinguishable from there, and Retry is exactly what produces them.
Taking the first match applied a boundary measured on a different, deeper branch
and evicted live history. Where the text cannot separate them, the smallest
boundary is now used: too small costs one more compaction, too large loses turns
the branch still has.
* Studio: price a conversation search against the whole prompt
Three gaps in the budget the previous commit introduced.
The GGUF loop subtracted an estimate of the messages alone, leaving the tool
catalogue out of the prompt entirely. A large catalogue is thousands of tokens,
so the request could already be near its budget while the search was still told
there was room for several 500-token chunks. The preflight already prices the
request exactly, catalogue and template included, so the difference between that
count and the estimate of the same messages is carried forward and the estimate
only covers what the loop appends after it.
The safetensors loop advertises and executes search_conversation for a thread
compacted under a GGUF model, but forwarded no budget, so the clamp in the tool
was skipped there. It now passes the room this model has left. With no known
context length the argument is omitted rather than sent as zero, which would
refuse every search.
The sticky boundary filtered candidates with a substring test, which is right
for archived chunks and wrong for whole messages: an abandoned "Done" rode in on
a live "Not done yet", and having no live twin it then decided the boundary
alone. Where any candidate matches a live message exactly, only the exact ones
are considered; where none does, the old behaviour stands, since a stored row is
not always byte-identical to what the client re-sends.
* Studio: keep the overflow diagnosis on the tool path
The irreducible-overflow branch counted the newest message on its own, and a
tool loop reaches that branch with a tool result last. A tool result by itself
is not a conversation: templates that require it to follow its assistant tool
call refuse to render one, and the exception escaped the fit entirely, so the
caller fell back to the untrimmed request and the client was told nothing at
all. That is the one path this diagnosis was added for.
The count is now attempted and falls back to the estimator when the template
refuses. An approximate number is worth more here than a diagnosis that never
arrives.
* Studio: budget the conversation search in the provider loop
The third tool loop forwarded the active branch but no budget, so the clamp in
the tool was skipped there and a model-chosen top_k of 8 could append roughly 4K
tokens to a prompt this loop replays on its next call.
Studio knows no window for an external model: the request carries no context
length and there is no registry to look one up in, and a custom
OpenAI-compatible endpoint can be a small local server. So rather than a
measured budget, this path spends no more than one ordinary recall's worth,
which is the same amount the compaction turn itself is sized for.
* Studio: bound an archived turn to its own messages, and archive it once
Two problems in the archive.
The whole-document branch check bounded the run by the number of LINES the turn
produced, and gave each chunk its own start. A turn is two or three messages
however long it is, so a hundred-line answer got a hundred-message window, and
the tail of an edited answer could be satisfied by a message well outside the
turn. The run is now bounded by the messages the turn was rendered from, counted
from the labels render_turn writes, and the chunks are scanned as one pass: each
continues where the previous one stopped rather than restarting. The cursor
inside a message is deliberately not carried over, because chunks overlap and a
continuation chunk repeats the tail of the one before it.
The hash check that skips an already-archived turn ran long before the insert,
with the embedding pass in between, and the index on (scope, sha256) is not
unique. Two generations compacting the same thread both cleared it and both
wrote, so the turn was stored twice and its copies took two of the few recall
slots. The check is now repeated under a write lock immediately before the
insert. Reproduced with two concurrent archive passes: two documents before, one
after.
* Studio: record how many messages an archived turn came from
The run an archived turn is allowed to occupy on the branch was bounded by
counting the role labels in its rendered text. That counts lines the user wrote
as well as the ones the renderer did: a pasted chat log carries lines that look
exactly the same, and each one widens the run by a message, which is enough for
the message after an edited turn to supply the passage the edit removed.
The group's size is now recorded on the document when it is archived, in a
nullable column added the way the other lazy upgrades are. Archives written
before this have NULL and fall back to the label count, so nothing needs
backfilling to keep working.
* Studio: persist a compaction boundary the next request can use
The boundary was read back from dropped_messages, which counts what each fit
removed from the conversation in front of it. The tool loop refits on every
iteration and the client sums those counts, so a long agent run added the tool
exchanges the turn itself created, and the next request applied the total to its
saved transcript. Reproduced with six tool calls on a 4K window: three fits of 4
summed to 12, on a branch that only ever had 4 evictable messages.
The boundary is now carried separately, measured against the messages the
request was sent with, so it is absolute and re-sending it cannot advance it.
The client keeps the latest value rather than summing, and turns saved before
this fall back to the dropped count, which is the same number for a turn that
fit once.
The recall reserve is also dropped once archiving has failed. sqlite-vec can be
present and the thread saved while the embedder cannot start: archive_turns
swallows that and recall injects nothing, so the room the fit held back was pure
loss on every compaction, and the failure mode forgot more history than having
the feature off. A failed write marks the archive degraded and the next
successful one clears it.
* Studio: count the boundary past the system prompt, and notice it
Two faults in the boundary added in the previous commit.
It stopped at the first message still present, and a Studio request always
starts with a system prompt that a fit never evicts, so every compaction
recorded a boundary of zero. That is the same as having no boundary: the next
request would move the eviction point again and invalidate the prefix cache on
every turn. Instruction messages are now skipped rather than treated as the
front of the branch, and the newest turn is excluded because it is never evicted
and an inline recall rewrites it in place.
The compaction notice still keyed off dropped_messages, which is the
accumulated count. A tool-heavy turn reporting 12 while the boundary moved to 4
set a high-water mark that silenced the next two real advances. It now reads the
boundary, falling back to the dropped count only for turns saved before the
boundary was recorded.
* Studio: tighten comments in the conversation archive and rolling context window
* Studio: give the pasted-text import an extension on this branch too
The node test runner cannot resolve an extensionless relative import, so
delete-chat-files-preference fails to load the preferences store. main fixed
this in
|
||
alkinun
|
645ca5c7bd
|
Studio: add rolling context windows for local GGUF chat (#8961)
* Studio: add rolling context windows for local GGUF chat * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: return rolling context metadata for non-stream chats * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep original context when rolling fit fails * Studio: keep instruction groups independently protected * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: preserve rolling context metadata across retries * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: count sanitized rolling context prompts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: refit rolling context after respawn * Studio: scope middle truncation to passthrough * Studio: refit tool prompts after respawn * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: retain later choice truncation metadata * Studio: report clipping-only context truncation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: re-pin the permission read the deep-research contract lost to #8686 The contract still asserted the literal `const permissionMode = loadPermissionMode();`, which #8686 removed when it put a chat-scoped override in front of that read. This branch already carries the refactored store, so Repo tests (CPU) cannot pass here or on anything stacked above it. Ported main's replacement rather than dropping the check. The read is still the contract, and it is scoped to the deep-research setter so it cannot be satisfied by the initial-state constant, which is a different property and would keep passing if the read were deleted. * Studio: count a video part as media in the rolling preflight `_inject_video_part` writes llama.cpp's own `input_video` part into the same message list the fit is then handed, and the media predicate did not list it. Video prompts therefore ran the rolling preflight, which is skipped for media precisely because `/apply-template` token counting does not include the sampled video tokens. The prompt could be certified as fitting when it does not, or lose history it never needed to lose, and still fail with context_length_exceeded. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
oobabooga
|
1840f7bd7f
|
Keep xFormers attention masks on the GPU running each layer (#8516) | ||
|
oobabooga
|
8bad21be0d
|
Studio: take common SQLite reads off the event loop thread (#9234)
Some checks are pending
Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Unsloth export capability / capability (ubuntu-latest) (push) Waiting to run
Unsloth export capability / capability (windows-latest) (push) Waiting to run
Frontend CI / Frontend build + bundle sanity (push) Waiting to run
Frontend CI / Frontend unit tests (Windows) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio UI + API + Update + Inference CI / Chat UI, API, Update and GGUF inference (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (windows) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (macos) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (extra) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (picker) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (banner) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (chat) (push) Waiting to run
Unsloth UI CI / Loaded-models indicator (cross-browser) (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection + VC++ round-trip (VS 2026) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Windows Application Control CI / installer survives a denied unsloth.exe (push) Waiting to run
Workflow trigger lint / workflow-trigger lint (pull_request_target / cache-poisoning) (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 units, no-VS resolve, VC++ round-trip (windows-latest) (push) Waiting to run
--------- Co-authored-by: imagineer99 <samleejackson0@gmail.com> |
||
|
oobabooga
|
9a45830e47
|
Studio: follow up #9201 across non-streaming sibling paths (#9312) | ||
sts-change
|
8d6e969378
|
Studio: never pick a macOS AppleDouble sidecar as a GGUF (#8919)
--------- Co-authored-by: Lyxot <longyixing331@gmail.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
Vineeth Sai Varikuntla
|
a6f267a4f5
|
Keep a sampler's std and uuid format across a save and reopen (#9225) | ||
karan yadav
|
dc26127a42
|
Studio: propagate required backend version to repair pipeline (#8610) (#8670)
* Studio: propagate required backend version to repair pipeline (#8610) Fix second-launch infinite repair loop when installed backend version is outdated (#8610). When the Desktop App launches with an installed managed venv whose version is older than expected_backend_version() (e.g. 2026.8.4 < 2026.8.15), preflight flags the install as ManagedStale with desktop_backend_version_outdated. On auto-repair, unsloth studio update ran setup.sh/setup.ps1 from the old venv, which skipped python dependency installation because INSTALLED_VER == LATEST_VER on PyPI or PyPI timeout, leaving the venv unchanged. The installer fallback (install.sh/install.ps1) also lacked version floor pins on standard fresh paths, locking the user in a permanent repair error loop. Key changes: - Pass UNSLOTH_DESKTOP_BACKEND_VERSION from Tauri (update.rs & install.rs) to child process environments. - Force Python dependency pass in setup.sh & setup.ps1 when UNSLOTH_DESKTOP_BACKEND_VERSION is set and INSTALLED_VER < UNSLOTH_DESKTOP_BACKEND_VERSION. - Apply UNSLOTH_DESKTOP_BACKEND_VERSION floor constraint when updating core packages in install_python_stack.py. - Ensure standard fresh install paths in install.sh and install.ps1 use "unsloth>=2026.8.15". - Bump MIN_DESKTOP_BACKEND_VERSION in preflight/version.rs to "2026.8.15". - Add shell and Python unit tests for the fast-path escape and desktop backend version constraint. Fixes #8610 * Narrow desktop repair to version propagation * Handle version suffixes in repair fallback --------- Co-authored-by: imagineer99 <samleejackson0@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> |
||
Lee Jackson
|
92d3f87055
|
Studio: keep health checks responsive during Anthropic generation (#9201) | ||
Etherl
|
e99dfe5a2b
|
Studio: use a custom llama.cpp build (#9292)
* feat(studio): add custom llama.cpp path setting * fix(studio): harden custom llama.cpp selection * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix custom llama.cpp path resolution * Shorten llama cpp path comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Distinguish non-executable llama server paths * Reload after llama server path changes * Refresh llama consumers after path changes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
Indrajit swain
|
b38490f0a2
|
fix(AMD): retry ROCm llama-server with bundled HIP on HIP/ROCR mismatch (#9002)
* Studio: retry ROCm llama-server with bundled HIP on HIP/ROCR mismatch System ROCm on LD_LIBRARY_PATH plus bundled libamdhip64.so dies on hsa_amd_queue_create@ROCR_1 (exit 127). That is not a VRAM miss, so do not flip --fit; retry once without the system prepend instead. Co-authored-by: Cursor <cursoragent@cursor.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: give the ROCm HIP/ROCR fallback its own spawn retry The library-mix retry was sharing the (0, 1) slot with --fit, so a later VRAM crash after bundled HIP could not fit-retry. Keep the two recoveries independent and pin the mix → bundled HIP → --fit on sequence. Co-authored-by: Cursor <cursoragent@cursor.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: widen the ROCm symbol-lookup detector and fix the retry harness #7233 prepends the whole system ROCm dir, so any lib in it can be the one that fails to resolve, not just HIP. Parse glibc's ``symbol lookup error: <object>: undefined symbol: <sym>`` line and match the ROCm family on the object's basename, so librocblas / libhipblaslt reach the same bundle-only retry instead of the generic 127 text plus a --fit retry that cannot load a missing symbol. The message now names the object and symbol the loader reported: it hardcoded hsa_amd_queue_create while the predicate also fired on any rocr_ match. _run_full_offload_spawns recorded every subprocess.Popen, so a host with the rocm_sdk wheel installed also captured the offload-arch probe that discover_current_target_family() shells out to from inside load_model. That landed at index 0 and shifted the assertions onto the wrong process, which is why the test passed in a full-file run and failed on its own. Record only the fake server and delegate the rest. Also guard the empty LD_LIBRARY_PATH case, cover the mix the retry does not fix (both fit-branch guards had no test), and drop three source-text assertions the behavioural tests already prove. * Studio: bound the ROCm loader object and symbol in the failure message Both captures are runs of non-whitespace taken straight from the child, and _drain_stdout keeps an unterminated line whole, so a wrapper on LLAMA_SERVER_PATH printing a megabyte-scale token would put all of it in the API error. _short is the existing bound for exactly this and every other capture in _classify_start_failure_text already uses it. * Studio: tighten the ROCm retry comments Twelve issue citations across four files for one fix, several of them restating the line below. Keep #8998 where it identifies the crash signature and #7233 where it names the prepend being undone, drop the rest, and cut the prose that repeated the code. One was already stale: the ggml test said the ROCm branch "names libamdhip64", which stopped being true when the detector started matching the whole ROCm family. * Studio: keep the bundle-only ROCm correction for every later child The retry repaired one launch and recorded nothing, so each load paid another crashing spawn, and the STT sidecar never recovered at all: it builds its child env through the same _llama_server_env_for_binary and has no retry of its own, leaving dictation broken on the host chat had just been fixed on. Record the build dir once a bundle-only launch reaches health and skip the prepend there from then on. Written only from that proof, so a host the #7233 prepend is right for cannot land in it, and kept in-process so a ROCm or driver upgrade re-tests on restart. * Studio: keep a spaced object path whole in the ROCm symbol-lookup match glibc echoes the object verbatim, so a custom LLAMA_SERVER_PATH under a directory with spaces kept them and \S+? stopped at the first one, dropping the crash into the generic 127 text instead of the bundle-only retry. Split on the ": undefined symbol:" that follows the object, the same way the missing-library branch above already keeps "/opt/My Runtime/libfoo.so" whole. Bounded at PATH_MAX rather than to end of line: no longer path can exist, and an unbounded lazy capture let one hostile line drive the scan quadratically (200 prefixes on a 400 KB line went 3ms -> 803ms; with the bound, 16ms). * Studio: record the ROCm correction from any launch in the load, not the first _did_rocm_retry lived in _spawn_and_wait, but the correction edits the shared env, so it outlives that call: the drafterless, no-flash, arch-fallback and CPU replay paths all call _spawn_and_wait again with the corrected env and a fresh flag. A load that recovered through one of those came up healthy without recording anything, and the STT sidecar kept the prepend it had already proved wrong. Hoist the flag to the load scope. Verified both ways: mix, signal crash, fit retry, then a healthy no-flash launch now records; with the per-call flag that same sequence does not. * Invalidate ROCm preference after runtime replacement --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Leo Borcherding <borchborchmail@gmail.com> Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com> |
||
|
Daniel Han
|
21296c4274
|
Install Studio on a Windows host whose C# compiler cannot run (#9178)
* Install Studio on a Windows host whose C# compiler cannot run
A first launch of the desktop app on Windows 10 died with
[ERROR] Could not create the Studio install lock: (0) : Die Quelldatei
'C:\Windows\TEMP\05po312x.0.cs' konnte nicht gefunden werden.
(1) : using System;
which is C# compiler error CS2001, not a locking problem. Get-StudioFinalPath
compiles a GetFinalPathNameByHandleW helper with Add-Type, and Windows
PowerShell 5.1 (the interpreter install.rs spawns) compiles by writing the
source into %TEMP% and running csc.exe. When that directory cannot hold a file,
or a scanner eats the source before csc opens it, Add-Type throws, and the
throw travelled up Get-StudioPathHash into Enter-StudioInstallMutex, where it
was reported as a lock failure. install.rs never sets or validates TEMP, so the
installer inherits whatever the app was started with.
Three parts:
Probe TMP and TEMP once with a write, read back and delete, and if the
inherited one cannot hold a file, point both at a per-user directory for the
rest of the run. The compiler is not the only thing that stages through there;
so do the Python, uv and VC++ downloads. It is restored on every exit path.
Split Get-StudioFinalPath into a cached native initializer and a compiler-free
resolver. The initializer skips compiling under Constrained Language Mode,
retries once with a private %TEMP%, and then remembers the answer. The fallback
resolves reparse points component-wise from the root, since a link on a parent
component is the ordinary Windows shape. Callers get an Exact flag so
Test-StudioPathEqual can answer "unknown" rather than "different", which the
runtime lock already reads as "take both locks".
Stop the in-use check failing open. Get-RunningStudioVenvProcesses called the
compiled helper inside catch { continue }, so a host that could not compile
found no running processes and would happily overwrite a venv Studio had open.
It now falls back to the process image path and Win32_Process, and says so.
With the native helper present every output is byte-identical to before: the
install mutex name, the runtime lock names, path identity and the process scan.
Fixes #9140
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Put the private temp directory where uninstall already looks
The fallback temp directory outlives the install on purpose: a Studio started
from it inherits it as its own %TEMP%. That means it has to sit somewhere
scripts/uninstall.ps1 reclaims, and LOCALAPPDATA\UnslothStudio was a folder of
my own invention that nothing would ever clean.
The USERPROFILE candidate was worse than litter. ~\.unsloth is removed only when
it is empty, so a leftover ~\.unsloth\temp would have stopped the uninstaller
clearing the directory at all.
Now LOCALAPPDATA\"Unsloth Studio"\temp, which the uninstaller deletes wholesale
as the data dir, and ~\.unsloth\.cache\temp, which is on its explicit sibling
list. Creating the data dir early cannot be mistaken for an install: the desktop
app decides that from find_unsloth_binary(), not from this directory existing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prove the private temp retry recovers, not just that it survives
Surviving a dead compiler is the floor. The retry is supposed to get the native
resolver back, and nothing asserted that it does. Add-Type is stubbed to behave
the way 5.1's CodeDom does, writing the source into %TEMP% and failing while that
cannot hold a file, then calling the real cmdlet once it can, so the type really
is defined on the second attempt.
Pins the whole sequence: one failure then exactly one retry, the type loaded
afterwards, no degraded warning, TMP and TEMP restored to the broken values they
came in with, and the retry directory cleaned up.
* Reclaim a stale junction in the private temp sweep instead of leaking it
Running the sweep on Windows PowerShell 5.1 showed the reparse-point branch
never removing the link: Remove-Item without -Recurse reports the junction
target's contents and refuses as "directory not empty", so a junction left
under the private temp root would sit there forever. Directory.Delete with
recursive:$false removes the reparse point itself and, unlike -Recurse on
5.1, cannot follow it into the target.
* Say what the junction branch actually guards against
Measured on windows-latest under PowerShell 5.1: Remove-Item -Recurse on a
junction removed only the link and left the target alone, while Remove-Item
without -Recurse threw a NullReferenceException that -ErrorAction did not
suppress. The comment claimed the first case as the hazard; the second is
what is reproducible today. Directory.Delete avoids both.
* Make the installer resolver tests hold on Windows too
Running them on windows-latest turned up three things the Linux-only run could
not see. The junction-alias test extracts Get-StudioFinalPath alone, which is a
dispatcher now, so every call in it was undefined and it read as "could not
resolve"; give it the whole resolver chain. The link-shape cases compared a
POSIX fixture against a GetFullPath result, which is drive-rooted and
backslashed on Windows, so compare paths rather than spellings. And utime with
follow_symlinks=False does not exist on Windows, where aging a link any other
way writes through onto the target; skip there and say why.
Resolve-StudioLinkTarget now normalises $Path before the self-reference check,
so a caller passing an unnormalised spelling still trips the guard instead of
being handed a link that points at itself.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore the caller's temp on every early exit, and keep a live owner's directory
Four review findings, all reachable through paths this change itself opened.
Two early throws sit before the try/finally that owns the locks: rejecting a
custom root under --tauri, and --shortcuts-only finding no managed Python.
Resolving a path can redirect TMP and TEMP, so under irm | iex both left the
caller's own session pointed at an installer-owned directory. The tauri branch
restores before it throws; the shortcuts-only block is now a try/finally, which
also covers the tauri return inside it.
The stale sweep treated age as proof a directory was unused. A Studio
autostarted by an earlier install inherits one as its %TEMP% and can outlive the
one-day cutoff without writing to it, and the sweep runs before the runtime
mutex is taken. The owner PID is in the name, so a directory whose owner is
still running is left alone.
The in-use scan resolved the venv and each process image through the same
resolver, which is enough while it is exact. Without the native helper an alias
it cannot canonicalize keeps its own spelling while the process reports the
physical one, so the prefix test found nothing and the install would overwrite a
venv Studio had open. When the identity is inexact it now also compares the two
paths below their roots, and fails closed.
* Fold SUBST aliases, fix the UNC device form, record the real temp owner
Three follow-ups, each on a path the earlier round opened rather than closed.
A SUBST drive kept its own spelling on a compiler-blocked host, so one directory
reached as X:\venv and as its physical target produced two different install
mutexes and hid a running Studio from the in-use scan. The Python runtime gate
resolves it, since Path.resolve does. Measured on windows-latest that subst.exe
is the only source available without a compiler: Get-PSDrive.DisplayRoot,
Win32_LogicalDisk.ProviderName, GetFullPath and Resolve-Path all reveal nothing.
Get-StudioLexicalPath now folds the alias before it walks components, so the
identity, the mutex and the scan all agree with the runtime gate.
Stripping the four-character device prefix from \??\UNC\server\share left
UNC\server\share, which reads as relative and was combined with the link's own
parent. It becomes \\server\share instead.
The PID baked into a ust-<pid>- name is the installer's, and the installer is
gone by the time the next sweep runs; the process that keeps using the directory
is the Studio it autostarted. That one is recorded in owner.pid when the
autostart happens, and the sweep prefers it over the name.
* Keep a mounted folder's volume GUID target rooted
Same trap as the UNC device form, in a different shape. A mounted folder
reports its target as \??\Volume{GUID}\..., and after the four-character strip
Volume{...}\... is not rooted either, so it was combined with the link's own
parent and the resulting identity named a directory that does not exist. It
becomes the extended-length spelling \\?\Volume{GUID}\... instead, which is the
same device path and stays the volume it names.
* Keep the volume GUID prefix through final normalization
The rewrite in the resolver was undone one step later. Resolve-StudioFinalPathInfo
strips a leading extended-length prefix, which is right for \\?\C:\x -- that still
names a drive afterwards -- and wrong for \\?\Volume{GUID}\x, which becomes the
unrooted Volume{GUID}\x. That hashes to a different identity than the same
directory reached by its drive letter, and leaves GetPathRoot empty so the
relaxed process comparison cannot run either. The volume GUID branch is tested
before the general one and keeps the prefix.
* Say only what was measured about the volume GUID root
The previous commit gave two reasons for keeping the prefix and one of them is
false. Measured on Windows PowerShell 5.1: GetPathRoot returns empty for both
\\?\Volume{GUID}\x and Volume{GUID}\x, so keeping the prefix does not restore a
root and does not re-enable the relaxed process comparison. The real and only
reason is IsPathRooted, which is true for the extended form and false for the
bare one, so the link resolver stops combining the target with the link's own
parent and inventing a directory that does not exist.
* Drop the root-relative fallback in the in-use scan
It was written to catch an aliased root the lexical resolver could not
canonicalize, and it was too broad to keep. Without the native helper every path
is inexact, so it compared path tails across unrelated drives: an ordinary
D:\env\python.exe matched a protected C:\env and aborted a legitimate install
as "the managed Python environment is still in use". That is every
compiler-blocked host, which is the population this change exists to serve.
The alias it was written for was SUBST, and that is now folded in
Get-StudioLexicalPath instead, which is the right place and costs no false
positives. A volume reached by GUID and the same volume reached by drive letter
still cannot be matched without the compiler, and a tail match is not a safe
price to pay for it.
* Probe TMP whenever Windows would use it
GetTempPath takes the first of TMP/TEMP that is merely non-empty, so a
whitespace-only TMP is the one Windows and every child process resolve through.
IsNullOrWhiteSpace read that as unset, probed a healthy TEMP, found it fine and
returned, leaving the compile and every later Python, uv and VC++ download
pointed at a path that cannot exist. Only an absent or empty TMP falls through
to TEMP now.
* Tighten the comments on the temp and path resolver fallbacks
Collapse the explanatory prose to one line where it still reads clearly and
drop the restatements, keeping every measured Windows PowerShell 5.1 fact the
code is shaped around. No code changes: the PowerShell token stream is
identical with comments and newlines dropped, and the two test files pass the
comments-only AST check.
* Resolve the data dir the same way the installer does when uninstalling
install.ps1 falls back to GetFolderPath("LocalApplicationData") when
LOCALAPPDATA is unset, which is the service and CI case the fallback exists for,
and puts its private temp under "Unsloth Studio\temp" there. uninstall.ps1
derived the default data dir from the variable alone and skipped the removal
when it was absent, so that tree survived an uninstall on exactly those hosts.
It uses _AppDataRoot now, the same resolver the WebView2 profile removal
already used a few lines below.
* Resolve an inherited temp path before probing it, and pin what was probed
A relative TMP or TEMP (temp, or the drive-relative C:temp) is resolved by
whoever reads it, and the install relocates out of a Windows system directory
later, so the value that was probed could afterwards name somewhere else or
nowhere. The absolute form is now pinned into both variables, and the caller's
own spelling is still what the restore hands back.
Resolving happens BEFORE the probe rather than after, because the two halves of
the probe disagree on relative paths: Test-Path is relative to PowerShell's
location while the .NET file APIs are relative to the process working directory,
and Set-Location moves only the first. Probing the absolute path checks the same
directory the writes will use.
The override now records whether the directory is one this run created, so
owner.pid is written only there and never into the host's own temp.
* Do not resolve a whitespace-only temp value before probing it
Resolving it first turns " " or a tab into the working directory plus that
name, which is creatable on some filesystems, so the probe would manufacture a
junk directory and then trust it as the host's temp. Leaving it untouched lets
the probe reject it, which is what sends the install to a private directory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Resolve a drive-less rooted link target on the link's own drive
A directory symlink can store a target like "\real": rooted as far as
IsPathRooted is concerned, but naming no drive. The compiler-free resolver
therefore skipped anchoring it to the link and handed it to GetFullPath, which
resolves a drive-less path against the PROCESS current drive. A link on D:
could normalize to C:\real, so the fallback mutex and the in-use process scan
would guard a different directory than the one being installed into.
Windows resolves such a target on the link's own volume, so anchor it to
GetPathRoot of the link before normalizing. A volume-GUID spelling has no root
to anchor to (GetPathRoot is empty there, measured on windows-latest), and
those keep today's behaviour rather than a guess.
* Run the Windows guard tests from a file, not a 32 KB command line
The Windows job started failing every test in this file as WinError 206, "The
filename or extension is too long". These scripts embed the whole extracted
helper chain, which has grown past the 32767 character command line cap
Windows enforces, so nothing was being tested on the platform the tests exist
for. Written to a temp .ps1 and run with -File instead, with a BOM so 5.1 does
not read it as ANSI.
test_running_venv_process_is_reported also raced: a six-ping child can exit
before Windows PowerShell 5.1 has finished its cold start and the csc.exe
compile of the native helper, which reads as the in-use scan missing a running
process. Same fix already applied to the 32-bit test: a long-lived child and a
deadline that allows for a slow shell.
* Reclaim both LocalAppData spellings on uninstall
New-StudioPrivateTempDirectory tries $env:LOCALAPPDATA first and falls through
to the LocalApplicationData known folder when that path is set but not usable,
so a non-blank variable does not tell you where "Unsloth Studio\temp" actually
landed. The uninstaller resolved a single root and stopped at the first
non-blank candidate, so on those hosts it removed a directory that was never
used and left the real tree behind.
It now collects both spellings, deduplicated, and stops servers by port file,
adds stop roots and removes the data dir for each. On an ordinary host the two
agree and the list has one entry, which is exactly today's behaviour.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Sweep abandoned private temp directories on a healthy host too
The stale sweep only ran from New-StudioPrivateTempDirectory, which a run with
a usable inherited TMP/TEMP never reaches. So once a host's temp was fixed, by
an ACL correction or a cleaned environment, nothing ever collected what the
degraded runs had left, including the remains of interrupted downloads, and it
sat there until an uninstall.
The candidate roots move into Get-StudioPrivateTempRoots so both paths can
share them, and the healthy path now sweeps each one. A root that does not
exist is a no-op, which is every ordinary host.
* Require the temp probe file to actually be deleted
The usability probe created a file, read it back, then deleted it with the
error suppressed and reported success regardless. A directory that accepts a
file and refuses to give it back, a denied Delete ACE or a scanner sitting on
the handle, therefore passed as usable. That is the shape behind this whole
issue: csc.exe writes its source and its output into the temp directory and
then cleans up, so the private-temp fallback was being skipped for exactly the
hosts that needed it.
The delete is now verified. Retried up to three times first, since a scanner
holding a file for a moment is not the same as a directory that denies
deletion and only the second should cost a healthy host its own temp.
* Run every Windows guard script from a file, and decode its output as utf-8
The command line cap caught the other two spawn sites as well: the Add-Type
fallback tests and the two mutex holder processes still passed their script
inline, so seven tests died as WinError 206 on windows-latest without testing
anything. All of them now go through a temp .ps1 with -File, the same as the
rest of the file.
Output is also decoded as utf-8 with replacement rather than the console
codepage. cp1252 cannot decode what PowerShell writes into an error message,
and the reader thread raised UnicodeDecodeError from inside subprocess.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
yzxcj797
|
a341b96905
|
fix(studio): img2img/inpaint on GGUF image models — from_pipe fallback with no recast (#9186) (#9193)
--------- Co-authored-by: Lyxot <longyixing331@gmail.com> |
||
Michael Han
|
3cb8ecce78
|
Studio: add an edit_file tool so agents stop rewriting whole files (#8753)
* Studio: add an edit_file tool so agents stop rewriting whole files The tool loop had no way to change a file. ALL_TOOLS was web_search, python, terminal, render_html and search_knowledge_base, so the only way to edit anything was a whole-file `cat > f <<'EOF'` through terminal or an open(...).write(...) through python. Both re-send the entire file to change one line, and both lose whatever the model failed to reproduce verbatim. Measured on a 520-line source file: a one-line change costs 7,750 output tokens to rewrite versus 45 to patch. Over five edits with re-reads that is 79,390 tokens against 8,676, which is why tasks that should fit in 64-94K die past 100K. edit_file replaces an exact string. Not a unified diff: models corrupt @@ hunk headers far more often than they mis-copy a literal snippet, and a bad hunk header patches the wrong place instead of failing. A missing or non-unique old_string is a hard error naming the match count and writes nothing, so the retry is "add context" rather than "recover a mangled file". - Preserves CRLF line endings, UTF-8 BOM and file mode. old_string is matched against normalized text, so a snippet with plain newlines still matches a Windows-authored file instead of failing invisibly. - Atomic write via temp file and rename, so an interrupted write cannot leave a source file half-replaced. - Contained to the session workdir, checked on the realpath so a planted symlink cannot reach out. /mnt/data-style habit paths remap exactly as the python shim does. - Under Full access absolute paths resolve, and the schema says so. Otherwise the model assumes it cannot reach a real checkout and falls back to the rewrite precisely where files are largest. - Still prompts in auto mode: python's open(..., "w") already does, so the cheaper tool must not become the quiet way around that. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden edit_file against the cases raised in review Six real defects, each reproduced before the fix and covered by a test. - Receipt was bounded by diff LINES, which bounds nothing when one line is the whole file. A 200KB minified source returned a 400KB receipt, twice what the tool exists to avoid. Characters are now capped per line and over the receipt: the same edit returns 481 chars. - bool("false") is True, and models emit the JSON string, so replace_all as a string turned the multi-match guard off and rewrote every occurrence. The two spellings models actually produce are mapped, anything else is refused. - A FIFO or character device reported st_size 0 and then read forever. This path carries no timeout or cancel event, so the turn could not be recovered. Only regular files are accepted now. - An absolute path inside a workdir that itself sits under a habit prefix (/workspace/repo) had its own prefix stripped and rejoined onto itself, resolving to /workspace/repo/repo/a.py. Paths already inside the workdir skip the remap; habit paths outside it still remap as before. - Two chats sharing a project workspace could both read, both write, and the later rename discarded the earlier edit silently. The bytes the edit was computed from are compared again before the rename. - Containment was checked once at resolve time, leaving the whole read and diff as a window in which a parent could be swapped for a symlink. It is rechecked immediately before the rename. Left as is: an empty old_string still writes a zero-byte file. Refusing every existing target would strand the model, since no other old_string can match an empty file, so nothing could ever write to it. Nothing is lost with no contents, and the mode is carried over by the write. * Studio: give edit_file the terminal glyph, not the globe status_for_tool reports "Editing: name" for edit_file, and toolStatusKind only treats a "Running" prefix as local, so a file edit on this machine showed the globe, the same badge a web search gets. It is as local as python and terminal, so it takes the same glyph. * Studio: bound the edit_file receipt and make creation atomic Second review pass. Four findings, each reproduced before the fix. - The receipt was capped on output but not on what produced it: difflib was fed the whole file and its generator drained into a list. replace_all on a file at the 16MB cap allocated ~500MB and took 1.3s to return 200 characters. difflib now sees only a window around the first change and the generator is consumed lazily. Measured on the same 16MB file: 501MB -> 48MB, 1.3s -> 0.05s; a 600KB file goes 48MB -> 1MB. Hunk headers are shifted back to real file lines, since a receipt pointing at line 3 of a 9000-line file is worse than none. - Creation checked lexists() and then wrote, so two chats sharing a project workspace could both pass the check and the later write drop the earlier file. The absent case is now created with O_EXCL, and filling a zero-byte file goes through the guarded write rather than clobbering blindly. - New files came out 0600: mkstemp makes the temp file private and copymode had no source to copy from. O_EXCL creation takes the usual umask-derived mode, so a group that reads generated files still can. - enabled_tools in the public request schema still listed only web_search, python, terminal and render_html, leaving the new built-in undiscoverable to clients reading the OpenAPI schema, and bypass_permissions described only the python/terminal sandbox. Both now describe edit_file, including that Full access lifts its containment. Eight new tests, 44 in the file. * Stop the edit receipt inventing deletions, refuse non-regular targets and unpaired surrogates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove the file a failed create just published O_EXCL publishes the name before the first byte and the payload goes out a buffer at a time, so ENOSPC or a quota partway through leaves the bytes that fit. Reproduced with a real kernel write failure: a 117780 byte create left 4096 bytes cut mid-token, and the retry the error message asks for is refused for ever, because an empty old_string refuses a non-empty target and no other old_string exists for a file the model never saw. close() can report a failure for data written earlier, so the error can arrive after most of the file is on disk. Unlinking the inode this call created puts the retry back on the create path, and keeps O_EXCL rather than mkstemp, whose 0600 would ignore the umask. * Let edit_file create an empty file Both strings empty is the documented creation call for __init__.py, py.typed and .gitkeep, but the identical-strings no-op check ran first and refused it, so there was no way to write a zero-byte file. Decide creation before that check, and stop reporting one line for a file with none. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim the review commentary --------- Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> |
||
|
Michael Han
|
77bc10c663
|
Studio: price a partial GGUF by what is left to fetch (#8989)
* Studio: price a partial GGUF by what is left to fetch The size beside a partial variant was the variant total, so continuing a sharded download that was already 40 GB in still read "56 GB" and looked like the whole model coming down again. That is the reading behind the reports of the hub re-downloading models people already have. The variants endpoint now reports download_remaining_bytes for a partial: the plan total minus the bytes on disk a transfer can actually reuse. Reuse is per file, so a finished shard counts and an unresumable partial does not, and the card shows "16 GB left" instead of "56 GB". A one-file quant therefore still reads back its full size, because that is what continuing it costs: huggingface_hub 1.18+ refetches an interrupted file from zero, and a single-file quant has no other file to keep. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: price local and offline partial rows too The local and offline listings return before the hub-plan path, so their partial rows carried no remaining figure and fell back to the full total. The on-device card asks for exactly those (preferLocalCache), which is where a partial is most likely to be looked at. The worker writes its manifest before fetching anything, so those rows can be priced from the file list that produced them. Capped at the row's own total, since a manifest counts companions the row's size may not. The on-device card also merges local rows over remote ones, so it now carries the remaining figure through instead of dropping it. * Studio: stop capping a local remainder by the shards on disk A local listing sizes a variant by summing the shards it can see, so an early interruption makes that total smaller than the transfer: three 2 GB shards with one cached advertises 2 GB, and capping the remainder by it reported "2 GB left" when 4 GB had to be fetched. Under-reporting is the one direction this figure must never be wrong in. The cap was there to stop a companion-inclusive remainder reading larger than the row's own size, but no surface shows the two together: a partial row's size chip IS the remainder. So the cap is dropped and the manifest total stands. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Price a partial by the bytes it holds, and credit a shared blob once across case-variant repo dirs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow the transfer in the remaining label while a download runs The live overlay carried only the expected size, so a row that says N left kept whatever the one-time variant fetch had measured, or the full total for a download started after it. A download 90 percent through read as though nothing had moved. The running job already carries its own progress, so derive the remainder from it and keep the fetched figure for every other row. * Take both terms of the live remainder from the job snapshot_progress nets completed_baseline_bytes out of expected_bytes and downloaded_bytes alike, so the job's two counters are consistent with each other and not with the catalog totals. Subtracting the job's transfer from the larger of the two scopes added that baseline straight back: 1 GB reused and 1 GB fetched of a 5 GB plan read 4 GB left rather than 3 GB. The catalog total still drives the size the row reports. * Price a pinned row against the cache root it names A row can name a snapshot in a previous, legacy or default HF cache root, and the request already scopes partial detection and the manifest read to it, but the blob scan always read the active root. Wrong in both directions: shards already in the pinned root earned no credit, and a copy of the same blob in the active root earned credit a resume into the pinned root cannot use, which reported less left than there is. The docstring called counting the active root only the safe direction to be wrong in, and it is for the first half, but not for the second. The fixture stub also had to start honouring an explicit root; ignoring it would have answered the active root for a pinned row and hidden the behaviour the new tests are about. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep a cancelled download on the measured remainder A job's progress is not bytes a resume can reuse: from huggingface_hub 1.18 the partial is process-unique, opened wb and unlinked in a finally, which is how existing_blob_bytes already prices it. Subtracting a dead job's transfer put 1.0 GB left on a cancelled 18 GB download with all 18 GB still to fetch, so the overlay now only prices running jobs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Price a fallback retry off the live transfer counter An XET run that falls back to HTTP re-claims in the same generation with a completed_baseline_bytes recomputed from disk, so the baseline now covers every blob the XET attempt finalized and the new run reports completed_bytes 0 against a shrunken total. resolveProgressUpdate holds the previous reading through that zero, so the card kept the dead run's finalized bytes while its total moved to the retry's scope, and taking the max of the two counters subtracted 3 GB from a 0.5 GB remainder: the row read 0 B left with the transfer barely started. snapshot_progress builds downloaded_bytes as completed plus in-flight and nets the same baseline out of both, so a single reading never has completed above downloaded and the max could only ever fire on a held figure. * Hold a retry's remainder off a held transfer reading resolveProgressUpdate holds the previous downloadedBytes through a poll that reported zero, and the GGUF row forwarded that held figure as the current transfer. An XET run that falls back to HTTP re-claims in the same generation with completed_baseline_bytes recomputed from the finalized blobs on disk, and snapshot_progress nets that baseline out of downloaded_bytes, completed_bytes and the total alike, so the retry's first reading is a legitimate 0 against a shrunken total. Subtracting the held 3 GB from the 0.5 GB that remained read "0 B left" until the retry moved its first byte. Carry the distinction out of resolveProgressUpdate as measuredTransfer and skip the progress-derived remainder on a held reading, falling back to the backend's own download_remaining_bytes the way the terminal row already does. * Carry the held-transfer marker across a reload measuredTransfer marks a reading the poll did not measure, so the remainder is not derived from a stale downloadedBytes. It was not persisted, so a reload restored the stale bytes beside the new run's shrunken expectedBytes with the flag reading undefined, which the selector treats as measured. That is the '0 B left' the guard exists to stop, and on app start it lasts until the first poll returns rather than a single tick. Persist and restore the flag. Absent stays absent, so a record written before this field still means never polled rather than held. * Carry the held-transfer marker through adoption too A restored fallback job is adopted by the next running-status probe, which seeds the persisted byte counters but rebuilt the job without the marker. That restored it as undefined, which reads as measured, so the remainder was derived from the dead run's bytes again and the row read 0 B left. The marker is a statement about those counters, so it travels with them: seededMeasuredTransfer joins carriesOverSeed in adopt-rules, gated on the same condition. Not carrying the seed zeroes the counters, so there is no held figure left and undefined stays the honest answer. * Do not credit a partial that has not been written yet blob_bytes_present read a missing st_blocks and a present zero as the same thing, so a partial set to its final length before its first chunk landed fell through to st_size and was credited whole -- the row read 0 B left on a download that had transferred nothing. Confirm the emptiness with SEEK_DATA rather than inferring it, so a mount that never populates st_blocks keeps the size fallback. * Read a pre-marker record's counters as held, not as measured A record written before measuredTransfer existed cannot say whether its byte counters were measured, so an absent marker there is not the never-polled it means in a current record. An upgrade landing mid-reclaim restored the dead run's held bytes with the guard reading measured, which is the 0 B left the marker exists to stop. Version the persisted state so the migration can tell the two absences apart, and read only the pre-marker one conservatively. * Trim the review commentary --------- Co-authored-by: shimmyshimmer <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> |
||
|
Lee Jackson
|
6fa27e20aa
|
Default preserve thinking on for Qwen3.8 (#9096)
--------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
Michael Han
|
e3260339d8
|
Revert "Say why a Xet download sits at 0% until it finishes (#9159)" (#9293)
This reverts commit
|
||
|
Daniel Han
|
607e8a310d
|
Cache the .deb set webkit needs, so a dead mirror costs a miss not a shard (#9289)
#9283 stopped two of the four Chat UI shards asking for webkit's system libraries at all, which is why extra and picker went from 14 minutes failing to under 6 passing. The shards that genuinely drive webkit -- chat, banner and the cross-browser indicator -- still pay the full price, and it is still the thing taking them down: 0 upgraded, 181 newly installed Need to get 102 MB/114 MB of archives Get:2 .../fonts-wqy-zenhei [7472 kB] -> 4m51s, then the attempt died Seen again today on a staging chat shard, after every other apt lever in this repo had already been pulled. Bounding the wait cannot help: the work is a real 102 MB and the mirror was delivering 7 MB in five minutes. So stop re-downloading it. apt keeps what it installed in /var/cache/apt/archives until something runs `apt-get clean`, so the .debs are already sitting there at the end of a good run. This copies them out, caches them on main, and copies them back in before install-deps on later runs. apt checks each file against its index and re-fetches only what does not match, so a stale entry costs a download rather than a wrong install -- the failure mode is slow, not incorrect. Keyed on the runner image as well as the engine set, because which .deb versions satisfy a dependency set is a property of the image. Saved on main only, the same rule as every other cache here: a PR-ref entry can only be restored by re-runs of that same PR while still counting against the shared budget. The engine guard from #9283 caught this change, which is the second time it has earned its place. Its detector read step NAMES as well as run bodies, and this step is named for the .deb set it holds -- webkit's -- so two chromium-only shards suddenly looked like webkit users. Fixed in the detector rather than by renaming the step: a `uses:` step cannot drive a browser, so only steps that run something should count. |
||
|
Daniel Han
|
32ba55b578
|
Run the zoo suite in parallel, minus the two files that cannot share a worker (#9285)
Some checks are pending
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection + VC++ round-trip (VS 2026) (push) Waiting to run
Unsloth export capability / capability (ubuntu-latest) (push) Waiting to run
Unsloth export capability / capability (windows-latest) (push) Waiting to run
Frontend CI / Frontend build + bundle sanity (push) Waiting to run
Frontend CI / Frontend unit tests (Windows) (push) Waiting to run
Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio UI + API + Update + Inference CI / Chat UI, API, Update and GGUF inference (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (windows) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (macos) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (extra) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (picker) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (banner) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (chat) (push) Waiting to run
Unsloth UI CI / Loaded-models indicator (cross-browser) (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 units, no-VS resolve, VC++ round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Windows Application Control CI / installer survives a denied unsloth.exe (push) Waiting to run
Workflow trigger lint / workflow-trigger lint (pull_request_target / cache-poisoning) (push) Waiting to run
* Run the zoo suite in parallel, minus the two files that cannot share a worker
unsloth runs unsloth_zoo's whole CPU suite in each of three matrix cells, at
about nine minutes a cell. Measured on a staging runner, whole suite:
cell serial -n 4 --dist loadfile
HF=latest + TRL=latest 580s 253s
HF=4.57.6 + TRL<1 517s 229s
HF=default + TRL=default 568s 246s
Two of the three cells agreed exactly, failure sets and skip sets both. The
third produced 14 failures serial does not: 8 in test_mlx_generate.py and 6 in
test_moe_bnb4bit_per_expert_conversions.py.
Both files pass on their own, and pass under xdist on their own. They are
self-contained, so what breaks them is another file running first in the same
worker -- an ordering serial never produces, because serial is alphabetical. The
speedup and the divergence are therefore separable, and the pair runs in a
process of its own where they are reliable.
Being clear about what this is: a mitigation. The suite has cross-file pollution
that alphabetical ordering happens to hide, and the cure belongs in unsloth_zoo,
on whichever file leaks. What these two files prove does not change here -- a
fresh process is what they already get today.
loadfile rather than the default `load`, because 34 of the 236 zoo test files
touch sys.modules or importlib.reload, so tests within a file have to stay on
one worker and in order.
The guard exists because half of this pairing is silent. An ignore with no
serial rerun deletes 51 tests from CI and the job stays green, which is strictly
worse than the 14 failures, since those at least announce themselves. Same shape
as test_backend_ci_parallel_isolation for studio-backend-ci. Mutation-tested four
ways: dropping the serial rerun, dropping an ignore, dropping --dist loadfile,
and losing a deselect in the split -- that last one matters because the three
deselects lived on the single command this change split in two, and losing one
turns a deliberate "deselected" into a failure on a GPU-less runner.
Listed in workflow-trigger-lint because it reads a workflow, so a workflow-only
PR would otherwise never collect it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Correct the evidence: the divergence moves between cells
A second A/B run across all three cells changed what the first one meant.
First run: HF=default diverged, the other two agreed. Second run: HF=latest
diverged, HF=default agreed. Same 14 tests both times, same two files -- 8 in
test_mlx_generate.py, 6 in test_moe_bnb4bit_per_expert_conversions.py.
So it is not a dependency combination, which is what one run made it look like.
It is worker scheduling, and roughly one cell per run draws the losing order.
Four of six cell-runs were clean.
That matters for anyone who re-runs this and sees green: a single clean run
proves nothing here, and would otherwise read as grounds to drop the pin. The
comment and the guard docstring now say so, because the previous wording pointed
at a cell and would have sent the next person looking at transformers and trl
versions.
The files being pinned are unchanged; if anything the case for them is stronger,
since the divergence lands on exactly the same 14 tests each time rather than
wandering.
* Give the xet stall detector a quiet process too
Staging ran the split for real and produced a failure the A/B never did:
FAILED tests/test_hf_xet_fallback.py::test_a_second_buffering_episode_after_a_drain_is_not_a_stall
AssertionError: a child refilling after a drain was killed:
['Download appears stalled (xet transport) -- no progress for 0s']
Nothing was stalled. The file drives a stall detector against the real clock --
27 sub-second sleeps, one of them commented "within the unmeasurable window" --
and under four workers on four cores the test was simply descheduled long enough
for the detector to fire. "No progress for 0s" is the detector saying so.
That is a different fault from the two files already isolated. Theirs is
ordering, which is why a fresh process fixes it; this one is a wall-clock margin,
and no ordering fix helps a margin. Same remedy, different reason, and the
registry records the difference so nobody later "consolidates" the entries and
loses why each is there.
Checked for the rest of the class rather than waiting to meet it on the next
staging round: exactly two files in the suite use sub-second sleeps, and the
other one sleeps 0.01s to widen a race window without asserting on elapsed time,
so it is not exposed to this.
Found only because the split changes how work is distributed, which is worth
noting: the A/B ran the whole suite under xdist and this file passed every time.
Measuring the change is not the same as running it.
* Record why each pinned file needs the quiet process
Both causes are now known and they are not the same, which matters because the
remedy differs. The bnb one is a sys.modules stub that was never taken back out,
and it is fixed upstream in unsloth_zoo#1076 -- so that pin becomes belt and
braces rather than the only thing holding it. The mlx one is an import-order
contract stated in the shim's own docstring, and the clean fix would spoof
platform.system() suite-wide, so the pin is doing real work there and will keep
doing it.
* Correct the mlx explanation: it is not established
I wrote that test_mlx_generate.py fails because the MLX-on-torch shim has to be
installed before any unsloth_zoo MLX module is imported, quoting the shim's own
docstring. It fits the symptom -- eight isinstance failures are what two copies
of a class look like -- but it is wrong.
conftest imports unsloth_zoo, which pulls unsloth_zoo.mlx in before any test
module loads. So the precondition is violated on every run, including every run
that passes, and the file passes on its own. I found this by asserting the
precondition and watching it fail a green run, which is the only reason this
comment is not shipping as fact.
The pin does not change: eight tests fail under xdist and pass serially,
reproducibly, on the same commit. That observation is what justifies it. The
explanation was mine and it did not survive being tested, so it now says so --
a wrong lead in a comment costs the next person more than no lead at all.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Michael Han
|
dc470257b6
|
studio: keep each chat's sampling params and system prompt with the chat (#9055)
* studio: keep each chat's sampling params and system prompt with the chat The composer pills, permission level and retrieval controls already travel with the thread. The sampling params and the system prompt did not, so returning to a chat started under one prompt showed whichever prompt the last chat had, and every chat shared one set of sliders. Those eight join the snapshot: temperature, top P, top K, min P, the two penalties, the system prompt and its variables. They are the only thread-scoped settings that live under params rather than as store fields of their own, so the read and apply paths route them through there and the whole snapshot is still one object on the thread row. An edit made with a chat open lands on that chat; with no chat open it moves the installation defaults, as before. A model's own recommendation is not an edit: both paths that apply one now say so, or loading a model while a chat was open would pin that chat to the model's sampling. Context and checkpoint stay out. They describe the model that loaded rather than the conversation, and a chat restoring a budget the current model cannot hold is the failure that exclusion exists to prevent. * studio: keep a chat's sampling through a model load and a held edit Three follow-ups from review on the per-chat sampling change. A model's recommendation still landed in the live params: fromModelDefaults only moved where the value was persisted. A chat with stored sampling ran the model's instead, and the next unrelated edit snapshotted that over what the chat had. setParams now puts back the keys the chat holds; one it never set still follows the model. The Qwen mode table is the model's recommendation, not a value the user picked, so both paths that apply it are marked. Loading the model and toggling Think reach the same table, so marking one and not the other pinned the chat by the other route. The held-edit flush read the sampling keys as store fields. They sit under params, so it read undefined, the sanitizer dropped them, and an edit made while the chat's snapshot was in flight was lost when the user navigated away. * studio: persist a model's sampling defaults even when the chat pins them Restoring the chat's values before the diff made every pinned key equal on both sides, so it dropped out of changedParams and never reached the installation defaults. One chat pinning temperature left every later new chat on whatever model loaded before it. The restore is a live-store concern only. Persistence diffs the model's own object again, as it did before the restore existed, so the recommendation reaches the defaults while the open chat keeps what it stored. getChangedInferenceParams bumps the mutation versions, so it stays a single call rather than one diff per purpose. * studio: keep a chat's sampling across a model switch, both directions setCheckpoint replays the destination model's remembered params itself, without going through setParams, so the restore never ran there. An external switch has no load after it to correct the result, so the chat adopted that model's prompt and sampling and kept them. The restore now runs on that path too, and persistence still reads the unrestored object so the model's own values reach the installation defaults. The other direction leaked the same way. rememberOutgoingModel snapshots the live params, which inside a chat are that chat's, and on a model with no entry it persists that snapshot whole. Filtering the incoming edit cannot undo a write that already happened, so the outgoing snapshot is now built without the values the open chat owns. What the model already remembered wins over the installation copy, so stepping off a model inside a chat does not flatten a preference the model was given outside one. persistReplayedParams writes the installation defaults on the setCheckpoint path, so it keeps the in-memory copy level the same way setParams does. * Keep a chat's sampling edits through hydration, model loads and remembered-params writes * Keep a held chat edit out of the outgoing model, and keep a default published mid-pairing withoutActiveThreadParams returned early while a chat's read was still out, so switching model in that window snapshotted the open chat's temperature and system prompt into the outgoing model's shared memory. The sibling restore path already read the held value first; this one consulted only the override. Driving the real store put a chat's own sentinel straight into paramsByModel, which the simulation harness already treats as a leak. Second, a model default published inside the pairing window was sent to the settings route but then rolled back in memory from the pre-window sample, so the session and the server disagreed and the stale value was pinned onto the next chat with no snapshot. noteThreadScopedDefaults also returned early when nothing had been applied yet, which is the common case, so the value had nowhere to land. * Put the installation defaults back when a chat is left mid-read Leaving a chat while its settings read is still out writes the edits made in it to that chat's row, but the store kept showing them, so the next chat took them: a snapshot-less chat captured the store as the installation defaults and was pinned with the previous chat's temperature and system prompt. Restore the pre-window values over the committed fields, and only when the chat is actually being left, so a retry after a failed read and a fork both leave the edit on screen. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: tighten the per-chat inference comments Comments only. Collapses the multi-line explanations in the thread-scoped sampling path to their shortest clear form and drops the restatements, keeping the reasons behind the pairing window, the held-edit capture and the outgoing-model filter. --------- Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
ba9e207ac7
|
Scroll into view before a forced click (#9291)
* Scroll into view before a forced click
`Compare tab: send to two panes` failed on a macOS runner with
Locator.click: Element is outside of the viewport
then a cascade of TargetClosedError as the browser came down behind it. The menu
item existed and was found. It was simply below the fold: a Mac runner's window
is shorter than a Linux one and Compare sits at the bottom of a long menu.
`click(force = True)` turns off actionability checks, which is exactly what these
call sites want against a menu whose overlay would otherwise intercept the click.
It also turns off the scroll that brings the element into the viewport, and
Playwright will not click a point it cannot reach. So the flag was doing two
things and only one of them was wanted.
The forced clicks are not new -- they arrived with the composer redesign in #5891
and have always worked on Linux. What changed is #9264, which made the Compare nav
be found reliably instead of sampled for: before that the step often gave up
earlier with "Compare nav not found" and never reached the click. That PR is
correct; it surfaced this rather than causing it, the same way #9283 was going to
surface the picker's context-pin assertion once the shard stopped dying in apt.
click_forced keeps force and adds the scroll back. The scroll is best effort: an
element that cannot be scrolled -- fixed position, zero size -- should still reach
the click and fail there with Playwright's own message, rather than here with a
scrolling one that names the wrong problem.
Applied to all 17 forced clicks across the three drivers, not just the one that
failed. They are the same hazard on the same runners, and the guard would be
worth little if the file it was written for were the only one obeying it. The
guard is a source scan for the bare shape, so a new one fails here rather than
twenty minutes into a Mac job.
Driven against a fake locator: these tests run on browserless lanes, and the
ordering is the whole contract, so a stand-in that records call order tests it
exactly -- scroll before click, force preserved, a failing scroll not swallowing
the click, and a failing click still propagating.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Michael Han
|
334db4875f
|
Studio: add a keyboard shortcuts page and make the existing chords rebindable (#8948)
* Studio: add a keyboard shortcuts page and make the existing chords rebindable Settings gains a Shortcuts tab listing every shortcut, with search, a recorder to change a chord, and controls to unassign or restore a default. The four chords that already existed were hardcoded across four files. They now resolve through one registry, so an edit applies without a reload. * Keep the shortcuts tab across reloads, and decide contested chords by registry order loadInitialTab checked a hand-written list that the new tab was missing from, so picking it and reloading fell back to General. The list is now the source of the SettingsTab union, so the two cannot drift. A chord claimed by two actions is consumed by whichever window listener runs first, which followed mount order and so varied by route. Registry order owns it instead, and the tab names the row that loses. * Clear rebound shortcuts on preference reset, and translate the shortcuts tab in every locale * Lock the auth-route settings guard to its behaviour rather than one spelling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Record nothing for a Super chord off macOS There is nowhere to put Meta on Windows or Linux: matchesBinding rejects an event carrying it, and the recorder wrote mod from ctrlKey alone, so Super+Alt+K persisted and displayed as Alt+K. That assigns the action to a chord the user did not choose, fires it on Alt+K pressed by itself, and leaves the chord they actually pressed unable to match anything. Returning null is the answer the recorder already gives while only modifiers are held, so it simply keeps listening. * Studio: render the sidebar shortcut hints from the live binding The search tooltip and the Settings menu row hard-coded the shipped chords, so after a rebind or a clear in the shortcuts tab they kept telling the user to press a chord that no longer runs anything. Both now read the resolved binding through useShortcutLabel and drop the hint when the action is unassigned. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> |
||
|
Michael Han
|
0488fe094a
|
Studio: stop the desktop health watchdog killing a backend that is busy generating (#8990)
* Studio: stop the desktop health watchdog killing a backend that is busy generating The watchdog probes /api/liveness every 15s with a 10s budget and declares the backend dead after 3 consecutive misses. Startup was the only load it made an allowance for. A host serving a model far larger than it can hold runs at fractions of a token per second, and the loop feeding those streams goes quiet the same way the warm thread's `import torch` makes it go quiet, so ~45s of silence ended a response that was still being produced and put "Server stopped unexpectedly" in front of it. /api/liveness (and /api/health, which the launcher falls back to) now publishes `inference_active` from the active-generation registry, and the watchdog widens its failure budget to 12 while the last answered probe said the backend was generating. Only for probes that time out: a refused connection means the port is gone, and that is still reported at three strikes. Both watchdog paths classify a failed probe. The adopted path folded every transport error into "not verified", so it reads liveness first and classifies there; the ownership probe's own first step is that same GET, so nothing is skipped by returning early, and both markers stay gated on verification. The kill is also logged with the budget it was measured against, so tauri.log says whether a backend was killed while stalled or was already dead. * Classify an adopted stall from the watchdog's own probe, and ask once more before killing a stalled backend * Keep a reused port on the normal watchdog budget An ownership mismatch answers the probe in full: the liveness fetch succeeds and owner_matches_metadata rejects the different root or token, which is a complete answer, not silence. Labelling it a timeout handed a dead adopted backend the 12-strike busy budget, so it lingered about 180 seconds rather than 45 while its old port was already answering for someone else. The probe now says the one thing it can say for certain, and only from a parsed body, so a transport error or a non-2xx still reads as silence and Unmanageable keeps the busy budget. The preflight hunk is test-only: ManagedCapabilityCacheHome set an environment variable without taking PROCESS_ENV_LOCK, racing the reader in with_xdg_data_home, which resolved the real home and collided the three webview-profile tests on one lock file. Measured at 1 failure in 30 runs on the unmodified tree, so it is pre-existing rather than new here, but the new tests shift the schedule enough to surface it. * Re-read the backend before the watchdog spends its verdict Both probes are awaits, and the last-chance one is 30s wide on top of the 10s cycle probe, so up to 40 seconds pass between reading the state at the top of the loop and acting on it. stop_backend takes whatever handle is stored now and carries no generation of its own, so a stop and start landing in that window had the stale watchdog kill the replacement and emit server-crashed over a backend that had just come up. The busy budget makes that window ordinary rather than unlucky: 12 strikes is about 300 seconds of hanging, which is long enough that a manual restart is the expected response. The generation is the identity the probe never carried, so re-read it and exit through the tail, which is generation-guarded already. * Carry the adopted pre-probe answer through the last-chance confirmation The confirmation asked for alive && inference_active, but on the adopted path alive means the ownership re-check verified us, and that check's two extra loopback requests are what a saturated backend drops. Such a backend answered the pre-probe with the busy marker set and was cleared anyway, mid-response. The busy marker now survives an unverified re-check, cleared only by a real takeover, and the confirmation reads it alongside the stall. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Michael Han
|
2515c44714
|
Studio: recover the OpenAI passthrough from a crashed llama-server (#8999)
* Studio: recover the OpenAI passthrough from a crashed llama-server
A crashed llama-server relaunches on a NEW ephemeral port. /v1/messages already
respawns and retries on a connection error; /v1/chat/completions did not, so a
client on the OpenAI surface kept posting to the dead port and got
502 Lost connection to the model server. It may have crashed -- try reloading the model
on every request until the user reloaded the model by hand, while an
Anthropic-API client on the same backend recovered itself.
Both passthroughs post to the same upstream route, so the retry helper is now
shared rather than Anthropic-only, and the OpenAI streaming and non-streaming
paths retry once against the respawned port. The retry only happens before
anything reaches the client, so no output is duplicated, and only when the
process really exited: _respawn_if_dead declines while it is alive, an MTP
fallback already in flight still wins, and a second unreachable upstream after a
relaunch is reported rather than respawned again.
Verified against a real llama-server: load a GGUF, serve a request, SIGKILL the
child, serve again. Before, both paths 502 forever; after, both recover on the
new port.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retry the passthrough only when the upstream connection was lost
httpx.RequestError covers ReadTimeout, so a slow but healthy generation
was resubmitted and the client waited twice the first-token budget while
two admission slots were burnt. Narrow both retry sites to the
NetworkError/RemoteProtocolError set the Anthropic path already uses,
extend the same one-shot recovery to a dispatch failure that surfaces
after the pre-header status window, and recompute the upstream headers
so the retry presents the api key the replacement server was launched
with rather than the pre-crash one.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the OpenAI passthrough stream alive while llama-server respawns
The post-status-window recovery awaited _passthrough_retry_url inline, so the
full load_model replay ran with no SSE output and no disconnect polling. That
loop already emits a keep-alive every five seconds on a committed response,
and a large GGUF reload lasts far longer, so a proxy or client could drop the
stream before the recovered request was submitted. Run the respawn as a task
and pump the same keep-alive and cancel loop while it is pending.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
|
||
|
Michael Han
|
0eb6b6c931
|
Studio: say when a scan folder cannot be read instead of showing no models (#9053)
* Studio: say when a scan folder cannot be read A folder Unsloth is denied looks exactly like an empty one: the scan catches the OSError, logs it, and moves on, so the model list is empty with no reason given. Add-time validation now opens the directory instead of trusting os.access, which reads mode bits only and passes on folders macOS TCC or a Windows ACL still refuses. The check runs after the denylist rules so a denied path is never opened. The scan keeps the error it already caught, and both scan-folders endpoints return it as a per-folder status the dialog shows with the setting that fixes it. No new work on the healthy path: 0.06us per folder per scan, and the folder list is a dict lookup with no syscalls. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: record scan folder status from the Hub inventory scan too The folders dialog reads /api/hub/scan-folders, but the scan behind it is the Hub inventory, not collect_local_models, so nothing was ever recorded for it and every row stayed "ok". Its custom-folder loop now records the same way. That alone is not enough: the Hub's _scan_models_dir catches the OSError itself and returns an empty list, so no exception reaches the loop. So an empty result is now the trigger. One opendir says whether the folder is empty, gone, or refused, and it runs only for a folder that returned no models. A folder that found models still costs nothing, with a test that fails if it ever touches the filesystem. * Studio: catch denied model subdirs, and recheck when the dialog reopens Two gaps in the folder status. A root can list fine while every model under it is denied, on a NAS mount or a drive owned by another user. The scanners skip an unreadable child silently, so that arrives as the same empty list as an empty folder and was reported as ok. The probe now also opens subdirectories, stopping at the first refusal and capped at 64, so the denied-everything case costs one extra open. The row tells the user to fix permissions and reopen the dialog, but nothing rechecked between inventory scans, so the warning stayed up after access was restored. Listing the folders now rechecks the folders marked bad, and only those. A healthy folder is not in the registry, so the list still opens nothing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: probe two levels down, and keep the tests collectable on Windows The child probe descended one level, so <root>/<publisher>/<model> with the model denied still reported ok: both levels above it list fine and the scanners return nothing. It now walks two levels, depth first, on one shared budget of 64 opens. Depth first means a denied mount is found in three opens instead of after every publisher, and the budget bounds the cost whatever the shape of the tree. os.geteuid does not exist on Windows and a skipif condition is evaluated at import, so collecting the test file there raised AttributeError before the os.name check could skip anything. Resolved once into a shared marker, with a test that runs the module body with geteuid removed. * Studio: flag a folder with one denied model, and show status in the picker A folder holding one readable model and one denied model returned the readable one, so the scan looked successful and the denied model was silently absent. The probe now runs whether or not models were found, and reports "partial" in that case so the copy does not contradict the rows on screen by claiming the folder cannot be read. That is a real cost change on the healthy path, so it is measured rather than claimed: 104us for a folder with 8 model dirs, 0.77ms for one with 300, capped by the same 64-open budget. The end-to-end scan stays inside run-to-run variation, and the folder list still opens nothing unless a folder is already marked bad. The old zero-syscall test is replaced by the bound, which is now the guarantee that matters. The inline model selector manages the same folders and rendered only the path, so it shows the status too. * Studio: stop treating an exhausted probe budget as healthy Three fixes. A denied directory past the open budget was reported as ok. Running out of budget means the tail was never looked at, which is not the same as finding it healthy, so it now returns an internal "unknown" that is never recorded and never sent to the UI. That alone would strand a wide folder in a warning it could never clear, so the registry now remembers which directory refused. A recheck opens that one directory, which settles a fixed folder in a single open no matter how wide the folder is or where the denial sat. A folder recorded as partial kept that status after being deleted, because the recheck preserved partial for every non-ok probe. It now only holds partial against a permission result, so missing and unreadable replace it. One permission test was missing the marker that skips it as root and on Windows, where chmod 000 does not deny. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Probe scan folders off the event loop, and stop a vanished model or a Windows device error condemning the folder * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Probe the commit directory inside an HF snapshots folder, and stop calling a shut root partial Two ways the dialog told the user the wrong thing. The cache layout is models--org--name/snapshots/<commit>/, three levels under a registered root, so a probe that stops at two never opens the one directory the weights live in: a denied commit dir made the model vanish from the list while the folder still reported ok, which is the silent empty case this PR exists to remove. The extra level is bought only for a directory named snapshots, since a blanket third level would spend the open budget descending into diffusers component directories. And when the root itself becomes denied, the partial branch restored partial regardless, so a folder none of which can be read kept saying some models in it could not be read, sending the user hunting for one bad model. The cause the probe already returns is the discriminator: it is the root path itself for the root's own refusal and a nested entry path otherwise. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Michael Han
|
588405dce2
|
Desktop: make every drop zone take a drop again (#9036) (#9056)
* Desktop: make every drop zone take a drop again (#9036) Tauri delivers OS file drops window-wide and suppresses the webview's own drop events, so a zone wired only to `onDrop` does nothing in the desktop app: no drag-over border, and the file is silently ignored. Native drop routing (#8265) was only ever adopted by the shared image picker and the create-project dialog. Every other file drop zone still relied on HTML5 handlers that never fire, which is why this reads as intermittent: the same file works when it lands on the chat, covered by the window-wide handler, and does nothing anywhere else. Adds `useNativeFileDrop`, which claims the native drop for an element and returns drag-over state plus the HTML5 handlers the web build still needs, then adopts it in the zones that were dead: - Projects -> Sources, which also had no drag-over styling at all - Data Recipes unstructured seed - Diffusion training images - Video and audio reference pickers Documents upload by lease rather than an inline read, since the native reader only serves media inline, so the recipe seed route now accepts `nativePathLease` the way the RAG upload routes already do. The native path policy accepts video containers so the reference picker can register what it is given. Also stops two silent discards with the same symptom: a claimed zone that refused a drop while disabled, and compare mode disabling the window-wide handler outright. Both now say what happened. `native-dropzone-coverage.test.ts` walks src/ and fails if a zone reads files from a drag payload without either claiming the native drop or explicitly deferring to the window handler. * Desktop: refuse a dropped model in compare too, and validate before reading Two things the first pass got wrong. Keeping the window-wide listener on outside single chat also handed it model drops, so a GGUF dropped on a compare or project view would load and replace the active model. Nothing happened there before, so that is not a change this should be making. The refusal now covers every kind the handler would act on, models included. The recipe seed route also moved its extension check after the read, so a rejected 500 MB upload was pulled into memory first. Back to validating the filename before reading a byte, as it was. * Desktop: size the native video cap to the largest client-side limit 64 MB sat under the reference picker's own 72 MB, so a clip the picker accepts was refused on drop. The cap is a backstop; callers keep theirs. * Desktop: drop the diffusion zone from this pass, and bound the native read Two review findings, both correct. The diffusion dataset zone accepts .bmp, .m4v, .caption and .jsonl, which the chat attachment policy rejects outright, and .txt, which registers but cannot be read inline. Captions beside images are the documented workflow there, so wiring that zone to the attachment path would have uploaded the images and silently lost the captions. It needs its own registration and upload policy, which is more than this belongs to, so it goes back to the picker it had. The recipe seed route also read a dropped path in full before checking any limit, so a multi-gigabyte local file went into backend memory before the 413. It now refuses on the stat and bounds the read by what the block has left, in case the file grows in between. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Desktop: keep a busy drop zone hit-testable, and size the video cap to the raw limit A disabled seed zone carried pointer-events-none, which takes it out of elementFromPoint, so nativeDropTargetAt could not find the target it had just registered. The disabled message was unreachable and the drop fell through to the window handler instead. MAX_NATIVE_VIDEO_BYTES was set to the reference picker's 96 MiB, but that cap bounds the data URL, not the file: the picker's own raw limit is 75497280 bytes. Rust was reading and base64-encoding up to 96 MiB, 128 MiB across the bridge, for clips the picker then rejected. * Match the document-refusal test to the message it now returns * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: attach video to a chat, and say why when it is unavailable (#9057) * Studio: attach video to a chat, and say why when it is unavailable llama.cpp takes video through its OpenAI-compatible chat endpoint as an `input_video` content part, but only when the mmproj declares video, the binary was built with video support and ffmpeg is installed. It reports that verdict at /props under modalities.video. Nothing about the GGUF alone can tell us, so that is what Studio now reads. Frontend: video joins the drop classifier and its own pending queue beside images and audio, and a VideoAttachmentAdapter takes one clip per message from the picker or a drop. When the model cannot take video the adapter names all three possible causes instead of letting llama-server refuse the request later. Backend: video_base64 on the chat request is forwarded whole as an input_video part, since llama-server owns the frame sampling and there is nothing useful to transcode. has_video_input rides the same path as has_audio_input out to the model row. Compare mode is left out on purpose: video_base64 targets the single loaded GGUF, so at most one side could answer. Dropping a clip there now says so rather than ignoring the file. Size caps line up across the three hops (64 MB in the desktop reader, in the composer and in the route) so no hop accepts what the next refuses. * Studio: carry the video capability through, and cover the passthrough path Three review findings, all correct. syncModelCapabilities took has_video_input but never copied it into the row, and /api/models/list omits it for the active GGUF, so the adapter read false after every load and refused video even when /props reported it. The feature did not work in its main path. The tool and response_format passthrough returns before the injection and forwards an explicit field list, so a clip rode along nowhere and the model answered without it. Refused now, the way audio already is there. The size cap floored the base64 inflation, so a clip of exactly the size the composer allows was refused with a 413, and the data URI header was counted against the payload. Padded ceiling, measured after stripping. * Studio: carry the video capability through every hop, and refuse it where it cannot be served Three separate places map backend capability flags onto a model row and each one dropped the video flag: the direct status adoption, the queued-run capability Pick, and (fixed earlier) syncModelCapabilities. The adapter reads that row, so any of them leaves video refused on a model that supports it. Covered by a rule rather than three spot checks. Injection lives in the GGUF branch, so an external provider or a local transformers model answered as if no clip were attached. Both now refuse, as does token counting, which cannot inject the frames it would need to count. The size check also moved ahead of the automatic model switch so an oversized clip does not evict a working model before the 413, and video now votes in the pre-switch projector requirement alongside audio. The video drain's read-failure toast said 'audio', inherited from the audio drain it was cloned from. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: name the attached modality in the pre-switch refusal Adding video to require_vision made the shared rejection reachable for a request carrying only a clip, but its text is fixed at 'image or audio input', so the user who attached a video was told about modalities the request never carried. The label now follows what is attached, and defaults to the existing wording so the image-only callers are unchanged. * Match the document-refusal test to the message it now returns * Send the API key on the /props readback and skip video in the context recount The /props probe went out without an Authorization header, so under UNSLOTH_DIRECT_STREAM=1 llama-server answered 401 and video capability never came back. Context recount already bails on images and audio because toOpenAIMessages has no branch for them; video has the same property and was missing the bail, so the usage bar priced a text-only prompt and stringified megabytes of base64 on the UI thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Send a picked clip under the container its extension names The accept list carries extensions as well as mime types because the browser's answer is unreliable for mkv and some mov files, and the picker takes those files on the extension. Only an empty type was being replaced, so a clip the browser called application/octet-stream kept that type into the attachment, and the request builder recognises a file part only when its mimeType matches ^video/. The clip was attached, sent and dropped, and the model answered as though nothing were there, which is the silent drop this PR exists to remove. The table mirrors the one in native_intents.rs, so a clip read by the desktop reader and one picked in the browser reach the route the same way. --------- Co-authored-by: shimmyshimmer <182633334+shimmyshimmer@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: shimmyshimmer <182633334+shimmyshimmer@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Michael Han
|
f3b0425345
|
studio: show the model a chat was started on, and offer it back (#9081)
* studio: show the model a chat was started on, and offer it back Every thread row has carried modelId since long before this, stamped at creation and read only for an image-compatibility warning. So the model a chat was started on is already recorded for chats that already exist; what was missing was showing it and offering it back. The chat list shows it on each single row, and a chat open on a different model gets a one-line notice with a Switch back button. Compare rows run two models and show none. Deliberately an offer rather than an automatic switch. For a local model, restoring on open would evict whatever is resident and spend a multi-gigabyte load on clicking a row in the sidebar. The button routes through handleCheckpointChange, the picker's own handler, so the confirmations, VRAM checks and external-provider handling are the same ones a manual switch gets. The notice stays quiet when it has nothing to offer: no stamp, already on that model, or a model since deleted and no longer selectable. The row label yields the right-hand slot to the spinner and the unread dot, and the title truncates first, so a long model name cannot squeeze it out. * Lift the chat model notice clear of the header instead of painting it underneath * Reserve the notice's height in the thread viewport so it stops covering the first message * Key the chat's model to the chat, and clear the research panel too Clearing the stored model inside the effect is a frame late: the effect is passive, so the first render for the incoming chat has already committed with the outgoing chat's model, painting the wrong notice and moving the viewport padding under it. Answering only for a matching thread id makes that render impossible rather than brief. The research panel needed the same offset the thread viewport got. The notice is an opaque bar spanning the whole chat content area, the panel column included, so it covered the first 2.25rem of the panel header: measured 36 px of overlap over the title, the status pill and the close button, with elementFromPoint returning the notice at all four. After the offset the overlap is 0 and the hit tests return the intended controls, while a chat with no notice keeps its exact previous geometry because the variable resolves to 0px. * Offer the switch in a chat that was started as New Chat A chat opened as ?new=<nonce> carries no thread in the URL and keeps none after the first send, so the notice never saw the row that send created and switching model in a chat you had just started offered nothing until you navigated away and reopened it. The store does learn the id, published once the visible thread is the one being persisted, but it still holds the previous chat's id for the first render, until ThreadNewChatSwitch blanks it in an effect. Handing that straight over would put the previous chat's notice on this one, so the id is taken only after it has been seen blanked for this nonce. Before the first send it stays an unpersisted local id and there is nothing to offer. * Offset the canvas panel below the chat model notice The notice is an absolute child of the chat content container, so it spans the canvas column too. The panel's top offset was a fixed 90px, which under custom desktop chrome left the notice covering the top 28px of the panel, including the preview/source tabs and the close control. Move both edges by the notice height, as the research panel already does, with 0px whenever no notice is on screen. * Switch back on the picker's metadata, not the id alone ChatModelNotice called handleCheckpointChange with only the model id. The picker never does: it supplies source, isLora, isDownloaded and isGguf for every row it offers. A Hub row or an external model does not need them (/api/models/list carries isGguf, isExternalModelId routes the rest), but a local or fine-tuned row is in neither, so selectModel resolved isGguf false and the /load request dropped n_parallel, n_batch, n_ubatch and llama_extra_args and sized the context down the transformers path. Resolve the same metadata the picker supplies for those rows, and carry the format for a single .gguf file out of the local inventory so it survives. A GGUF directory stays unclaimed on purpose: that row expands to pick a quant and a chat row records only modelId, so sending llama-server flags for a quant nobody chose would be worse than the plain switch. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Michael Han
|
2f80f11ccd
|
Studio: link the Read aloud TTS row to Audio and show the preview generate wait (#9093)
* Studio: link the Read aloud TTS row to Audio and show the preview generate wait The studio TTS row named the model selector without offering a way to reach it, and Preview showed Stop for the whole generate, which read as an idle button. * Route the Read aloud action to Speak, not the mode Audio was left in AudioPage stays mounted off-route and keeps its mode state, so a plain /audio navigation could land on the transcription selector. Send the task as a mode intent and handle a task without a model on the Audio side. * Tighten the comments on the Read aloud TTS row changes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
oobabooga
|
de102032ee
|
Desktop: ship a complete Linux AppImage (#9113)
--------- Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com> |
||
|
oobabooga
|
531bc481a2
|
Studio: incrementally tokenize streaming code fences (#8935)
--------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
6c41897833
|
Cache uv's downloads, which is now the largest single cost in CI (#9271)
* Cache uv's downloads, which is now the largest single cost in CI Re-profiling after the apt work landed changed the ranking completely. The old top step, Install Playwright browsers at 557-671s, is now 91s. What sits at the top instead: 3588s total 92s x39 Install Unsloth (--local, --no-torch) 1925s total 175s x11 Update banner layout regression (Playwright) 1910s total 478s x 4 unsloth_zoo @ main - full pytest (CPU) 1683s total 99s x17 Perform CodeQL Analysis More total time than any test, in 39 job runs, and all of it uv re-resolving and re-downloading the same wheels: nothing carried its cache between runners. UV_CACHE_DIR appears in this repo only in the two clean-machine workflows, which set it in order to delete it. Caching this is safe because of WHAT is cached. uv's cache is content- addressed by URL and hash, so a stale entry cannot serve wrong content -- the worst it can do is miss. That is the difference between this and caching the venv, which would have to reason about an editable overlay (--local installs the repo with -e, so source changes need no reinstall), a moving `unsloth-zoo @ git+main`, and absolute paths baked into console scripts. I looked at the venv first and it is not worth the hazard. restore-keys is deliberate and is correct only here: a near-miss still supplies almost every wheel, which is most of the win, and it is safe precisely because entries are content-addressed. The same fallback on an install cache would be a bug. Saved on main only, like the HF and Playwright caches and for the reason they record: a PR-scoped entry is restorable only by re-runs of that same PR, while every PR can restore from the default branch, so saving on PRs spends a 50 GiB budget measured at 99.3% full once already and evicts main's copy. `uv cache prune --ci` first, so the key does not grow without bound across 39 jobs. Placed in the composite action, so all 39 call sites get it from one definition -- which is what that action exists for. The cold-install lanes are untouched and a test keeps them that way. They do not use this action today; if one ever did, the composite writes UV_CACHE_DIR to $GITHUB_ENV, which outranks a job-level env: for every later step, so a warm cache would silently replace the cold machine those workflows are named after and they would still go green. Nine tests. All five mutants reintroduced and confirmed red: the cache pointed at the venv, saved on every ref, restore-keys removed, UV_CACHE_DIR set after the install, and a cold lane adopting the action. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Michael Han
|
7b58b56228
|
Studio: make the partial download affordance tell the truth (#8982)
* Studio: make the partial download affordance tell the truth The "Partial" badge's tooltip said "Click to continue", but the badge is a status dot with no handler. On the GGUF card it sits inside the quant trigger, so clicking it opens the quantization menu, and picking the partial quant there only selects it. On the safetensors and dataset cards the click does nothing at all. Every tooltip now names the button that acts. The button itself said "Redownload" for a Xet partial, which reads as a full refetch of a repo that may be tens of GB. It is not one: only incomplete blobs are purged, so snapshot_download skips every shard already on disk and Xet reuses CAS bytes through its chunk cache. The label is now "Continue", and the tooltip says completed files are kept. "Continue" was also promised for HTTP partials that nothing can continue. huggingface_hub 1.18 moved the writer to a process-unique .incomplete opened "wb", so an interrupted file is refetched from zero, and Studio pins >= 1.23 on Python 3.10+. The transport capabilities endpoint now reports whether partials are resumable at all, and only a partial that really is gets "Resume" or a pause glyph on the stop control. * Studio: lead the partial hint with the file that restarts The unit of "kept" is the file, and most quants are one file: a 27B Q4_K_M is a single 18 GB GGUF, so continuing its partial fetches all 18 GB again. Only a sharded repo keeps anything, which is why the 56 GB BF16 in #8927 looked like a resume. Leading with what survives promised something the single-file case cannot deliver, so the restart comes first. * Studio: decide Resume from the partial, not from the hub version The resume wording was derived from the row's transport plus whether the installed huggingface_hub resumes at all. Both can say yes about a partial that cannot be reopened: a cache shared with a 1.18+ environment holds nonce-named partials, and this repo's own pins produce that mix, since Python 3.10+ takes hub >= 1.23 and older takes 0.36.2 against one cache. The row promised Resume and the next attempt purged the bytes. Rows now carry partial_resumable, the conjunction of the transport they report and the registry's per-file check, which is what the transport-conflict dialog already used. The label and the tooltip read that; the installed-writer capability stays where it belongs, on the stop control for a running job, whose partial is this machine's own. Anything that cannot answer defaults to false, so no surface can promise a resume it has not verified. * Studio: scope the resume verdict to the variant that asked is_resumable_partial takes a variant but its blob scan is repo-wide, so a sibling quant's reopenable partial answered for a row whose own partial is nonce-named and about to be purged. Latent until now, since every caller passed a repo and no variant. It now intersects the resumable blobs with the ones the variant's manifest claims, which is what the variant transport-status endpoint already did. No manifest means no way to scope, and an unscoped yes is the promise this guards against, so it answers no. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: judge a companion partial by its own marker A GGUF manifest carries the shared mmproj and MTP drafter alongside the quant's own shards, but those blobs answer to .transport.companion, not to the variant marker. So a companion partial left by Xet, or one with no companion marker at all, satisfied the variant's HTTP marker and backed a Resume that prepare_cache_for_transport purges before fetching anything. The verdict now splits the manifest the way the worker splits it for the purge: main hashes are vouched for by the variant marker, companions only count while the companion marker agrees. A blob in neither set backs nothing, so an unrecognised layout understates rather than promises. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope the resume verdict to each cache entry A repo can own several active cache directories at once: on a case-sensitive filesystem models--Org--Model sits beside models--org--model, and iter_active_repo_cache_dirs yields every one of them. is_resumable_partial unioned partial hashes across all of them, then asked read_active_transport_marker / read_active_companion_marker for the first non-null marker found in any. So a marker in one directory could vouch for a partial in another that prepare_cache_for_transport, which judges per entry, was about to purge. The row promised "Resume with HTTP to keep the progress you already have" for bytes that would not survive. The verdict is now decided per entry, the way the purge decides, so a marker only answers for partials sitting beside it. read_active_companion_marker had no other caller and is gone. * Answer the resume question from the row's own cache root partial_resume_available scoped the transport half to the row's cache root and then threw that root away for the resume half, so a row shown from a remembered or custom root was judged by the active root's blobs and the active root's manifest scope. Wrong in both directions, and both reproduced: a legacy .incomplete under the active root made a remembered-root row offer to keep progress the next run purges, and a genuinely resumable variant row read as unresumable because its manifest lives in its own root's digest scope, so the UI offered a redownload over good bytes. The root is keyword-only and defaults to None, so the two callers that mean the active root are unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: shimmyshimmer <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Michael Han
|
b8c931f9a4
|
Studio: size Xet download buffers from free RAM, not total (#9052)
* Studio: size Xet download buffers from free RAM, not total hf_xet's reconstruction buffers are sized from the machine's TOTAL RAM, so a download started while a model is loaded asks for the same multi-GB budget it would ask for on an idle box. Those buffers are the worker's RSS, not reclaimable page cache, so the request and the resident weights add up and the machine swaps. Clamp the zoo-sized budget to a quarter of free RAM by re-asking the zoo about a smaller machine, so buffer, per-file and file count scale together. A quarter of AVAILABLE always exceeds the zoo's eighth of TOTAL on an idle host, so the clamp is unreachable unless RAM is genuinely held: sizing is byte-identical when there is headroom. Also apply the zoo's existing MIN_XET_RAM_BYTES floor to free RAM when picking a transport, so a host too tight for even the clamped floor downloads over HTTP. Fixes #9032 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the free-RAM clamp reduce-only The recompute calls xet_env_overrides directly, without the throttled flag apply_xet_env threads through after a 429, so an un-throttled recompute could hand back a stream ceiling that backoff had lowered. Take the smaller of the two per key. Every derived value is monotonic in total RAM, so the result stays coherent. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: apply the free-RAM gate where Auto is actually resolved The Studio UI never sends transport_mode=auto. effectiveTransportMode() resolves Auto through get_download_transport_capabilities(probe=true) and submits the answer as an explicit xet/http, which resolve_requested_use_xet honours without calling resolve_auto_use_xet. The gate therefore never ran on the primary flow. Move the verdict into a shared free_ram_pressure_reason() and call it from the probe as well, so the UI path and an API caller that sends auto agree. Probe only, so an ordinary browse poll stays read-only and still does not load Zoo. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: let free RAM decide when health has no verdict Both early returns in resolve_auto_use_xet skipped the gate: a zoo without hf_xet_health, and a health probe that raises. Free RAM is read from hf_xet_tuning, a different module, so neither says anything about whether the machine can afford Xet right now. Fold both into one optimistic path that still consults free RAM, and move the registry probe's RAM read outside the health try for the same reason. A health verdict that already demoted keeps its own reason. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: reserve free RAM across concurrently starting Xet workers A worker allocates inside the child, after Popen returns, so free RAM does not move until well after sizing. Four downloads starting together each read the same untouched value and each took a quarter of it, promising the whole machine. Sizing now subtracts what live siblings were already promised, and the reservation is bound to the worker's pid so it frees when that worker exits. The transport gate subtracts it too: the clamp bottoms out at Xet's floor, so enough simultaneous workers would still add up past free RAM, and the next download is better served by HTTP than by Xet at its minimum. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make RAM admission and reservation one critical section The ledger read and the reserve sat in separate critical sections, so concurrent sizings could all read the same total before any of them wrote, which is the overcommit the ledger exists to stop. Hold the lock across the whole decide-and-reserve region; the recompute inside is pure arithmetic on a frozen profile, and the RAM/disk reading stays outside it. The earlier reservation tests started workers sequentially, which never entered that window. The new test races four threads through it. * Probe worker liveness without signalling on Windows, and let UNSLOTH_FORCE_XET override the free-RAM gate * Ask the backend fresh on every Auto download start The probe answer was cached for 30 seconds, so several Auto downloads begun inside that window were all admitted on the same pre-reservation verdict and each submitted an explicit transport_mode=xet, which the start path honours without re-reading free RAM. Measured on a 32 GB box with 5 GB free: four starts promised 4.25 GB of Xet buffer pools, one per worker process, where a live gate admits the first and sends the rest over HTTP. Render polls still answer from the cache; only the probe itself has to be live, and it costs no network probe because the health verdict is memoized upstream. * Credit a live worker's resident memory against its reservation The ledger charged a bound worker's whole buffer promise for its whole lifetime, but the free-RAM reading it is subtracted from has already dropped by whatever that worker allocated, so the same bytes went out twice. Measured with a real child holding real touched RSS: 4.04 GB genuinely free, reported as 2.0 GB, and the next Auto download demoted to HTTP for room that existed. The reservation exists to bridge the window between sizing and allocation, which is what the comment above it says, so credit what has materialised and keep the full promise for a worker that has not bound yet. The xet buffer is drawn through a permit semaphore rather than preallocated, which is why resident memory is the right measure of how much of the promise is already spent. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: shimmyshimmer <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
915fa5473c
|
fix(studio): refuse a hand-set context unified memory cannot hold (#9172)
* fix(studio): refuse a hand-set context unified memory cannot hold The Metal branch of load_model already works out the largest context that fits, but only Auto was ever moved to it. An explicit context was passed through verbatim on the theory that "--fit on" is a backstop. It is not one here: --fit flexes -ngl to spill layers to the host, and on unified memory the host is the same pool, so the spill frees nothing and the launch over-commits. A discrete GPU answers that with an out-of-memory error, a Mac has nothing to fall back on and takes the whole system down. Gate the explicit request on the ceiling the branch already computes and refuse above it, naming the largest context that fits so the user can act on it. Two deliberate exemptions: the 4096 fallback used when KV cannot be sized is a guess rather than a measurement, so it never refuses, and a manual load with a fixed layer count is the user taking the memory budget over, which is how the other two Metal context guards already treat it. UNSLOTH_ALLOW_METAL_CTX_OVERCOMMIT=1 opts out. The refusal is raised after the placement block, not inside it: the broad except there turns any raise into "GPU selection failed" and restores the original request, which is the over-commit being refused. The slider warning said the model "may use system RAM", which is sound advice on a PC and describes something that does not exist on unified memory. Say what actually happens there instead. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Exempt the virtualised Metal device, and correct why --fit is not enough Two findings from simulating the guard across [Windows, Linux, WSL, Mac] x [NVIDIA, AMD, CPU only] against the real load_model. A virtualised Mac was being refused. Studio pins that whole load to CPU behind --device none, because offloaded layers on a paravirtual Metal device produce corrupt output, so it allocates no GPU memory at all and a GPU working-set budget is the wrong yardstick for it. The refusal would have broken loads that work today on a Mac VM, and on the macOS Actions runners, and the message would have described hardware the launch never touches. Host RAM is the real limit there and _host_offload_shortfall_message already prices it. The neighbouring _caller_owns_budget did not cover this: it is read off the request, so it stays False for the Auto load the pin rewrote to manual/0. The stated reason --fit could not be relied on was wrong. It said --fit only moves layers to the host, which frees nothing on unified memory. In fact llama.cpp does reduce an explicit context: common.h defaults fit_params_min_ctx to 4096, and only "-c 0" raises it to UINT32_MAX to disable reduction. The real problem is what it decides from. ggml-metal's free-memory report comes off the device's recommendedMaxWorkingSetSize, which knows nothing about Studio's own resident gigabyte or two, other running apps, or the iogpu wired limit that is the figure being blown, and _apple_metal_memory_budget_bytes exists precisely because of that gap. When the estimate is optimistic the request stands, the launch over-commits wired memory, and wired pages are not reclaimable by Jetsam, so the driver faults and the machine panics. Tests for both, plus the pass-through -c spellings, what a refused reload costs, and that neither route rewrite (path redaction, the unsupported-model relabel) mangles the message. The two slider warnings no longer promise the load will be refused, since the virtualised case above is exempt. They say what is true everywhere on unified memory and what to do about it. * Apply the repo formatters * Measure the ceiling the guard refuses against, and word the warning from unified memory Three holes, two of them in the refusal itself. The fit hands back its own min_ctx floor when nothing fits, not a smaller answer, so a Mac with room for no tested context returned 4096 with a footprint over budget. That landed in the arm meant for "weights alone exceed the budget", left the measured ceiling unset, and skipped the refusal for every explicit context, on exactly the machine where llama.cpp's --fit has nothing left to give either (it will not price below fit_params_min_ctx, which is the same 4096). Re-price from the search's own 256 alignment step before concluding nothing was measured; the weights-over-budget arm returns the request untouched for any min_ctx, so the two states are told apart by whether the re-priced answer is smaller. The fit was also sized through the model's native length, so the ceiling could never exceed it and any request past it was refused as an over-commit even with the memory to hold it. Nothing clamps a request to the native length on the way in: the Extra Arguments box takes a raw --ctx-size and its own placeholder suggests --rope-scaling yarn, and llama.cpp builds the context at the full -c, capping only the per-slot value afterwards. Re-price through the request when it is larger, and adopt that only when the budget vouches for it. max_available_ctx stays sized from the native length, so the slider bound the UI reads does not move. Both were invisible to the tests because every case stubbed the fit with a lambda that has neither a floor nor a native bound. The stub is now optional. Third: device_type is "mac" for every Darwin host, so the new warning wording reached Intel Macs, where a discrete GPU spills to system RAM exactly as the old wording said. /api/health now reports apple_silicon alongside device_type, on the same gate the Metal budget uses, and the two warnings read that instead. Absent means false, which is the old wording, so nothing regresses where it was already right. * Name an encoding on the route read, and keep the log line out of the comment Both found by staging CI on windows-latest, and both are this branch's doing. routes/inference.py carries non-ASCII (the DeepSeek tool-call tokens), and Path.read_text() defaults to cp1252 on Windows, so the new route-message test died on a UnicodeDecodeError there. tests/test_source_read_encoding.py catches this class on Linux too and was already failing. Pass encoding = "utf-8", which is the convention the rest of the test tree follows. The comment above the refusal quoted the placement handler's own log line, and test_tp_vision_regression string-searches load_model's source for that phrase to assert statement ordering. Quoting it moved the match ahead of the try body and failed the ordering assert. Reworded around it, with a note saying why. * Refuse when nothing fits at all, and stop the health test depending on file order The over-budget arm has three states, not two, and the third was still slipping through. When the fit shrinks, the weights themselves fit, and that is the signal separating a real measurement from the weights-alone case. If the floor it shrank to does not fit either, then nothing fits: there is no number to lower to and every explicit context over-commits. That was being treated as unmeasured, so those requests reached llama-server on a host where all of them are the crash this guard exists to stop. Refuse them, with a message that names no ceiling, because inventing one would send the user to a number that hits the same wall. Auto keeps the 4096 floor it has always launched at here: changing what Auto does is a larger claim than this guard makes. The manual-layers and virtualised-device exemptions still apply. Weights alone over budget still never refuses, and the discriminator is now explicit rather than folded into one condition. Separately, the new health test could not survive a full-suite run. Several files in this tree install a plain `loggers` module and a minimal `structlog` into sys.modules, and main imports `loggers.config`, which needs the real package and real structlog. Whichever file sorted first decided whether this one could import main. It now restores both before importing, and skips rather than fails if structlog is genuinely absent. Pre-existing pattern, and the same reason the MLX-repair health test fails in a full-suite run. * Stop the guard tests reading the real machine's free memory The virtualised-device cases passed on a 16 GB runner and failed on a 7 GB macos-14 one, which reads as a Mac-specific bug and is not one. The paravirtual placement is the only one in this harness that reports child_has_no_gpu, which is what gets it past the host-RAM guard's empty-pool early return; that guard then priced the model against the real host, because _launch stubs _apu_ram_shortfall_message but never stubbed _available_system_memory_mib. Host-memory dependent, not OS dependent: a real 8 GB Mac fails the same way. Stub it to None, which is both helpers' documented way of saying the caller does not know the host's memory and must not refuse on it. Verified the file now passes with available memory forced to 3000, 800 and 200 MiB. * Use an unsignalable pid in the fake process, and put back the modules the health test swaps The fake llama-server the guard tests inject is not inert data. load_model adopts whatever pid it is handed into the process-lifetime registry, and teardown signals that pid's process group, so a low placeholder is a live signal target rather than decoration. 4194303 is one below pid_max: validly shaped, names no process, so the identity check drops it and it can never be signalled. The comment says why, because the next person to copy this stub will otherwise reach for a small number too. The health test also displaced the loggers and structlog stubs other files in this tree install and never put them back, which left a global mutation behind for whatever ran next in the same pytest process. It now restores exactly what it removed, on the skip path as well as the success path. * Read the weights-only state off the budget, not off two fits agreeing Both probes are bounded by the same target, so on a model whose native length is already at or under the search's 256 alignment step they return the same number for a reason that has nothing to do with the weights. Inferring "the fit priced nothing" from that agreement left both verdicts unset and let every explicit context through on a host where none of them fit. Reproduced at native == 256 exactly: an explicit 8192 launched where it should have been refused. Also reachable whenever the GGUF carries no context length, since the request itself then becomes the target. Ask the budget instead: weights alone over it is the state the fit cannot price, and that is a property of the weights and the budget, not of two return values tying. Same verdict as before everywhere the old comparison was right. Auto is unchanged, checked rather than assumed: swept the weights band either side of the boundary on both this tree and the merge base and the emitted context matches at every point. * Re-probe below the fit floor when extending a short-native model The above-native extension probe runs at the fit helper's 4096 minimum. When the model's native length is under that floor and the machine has room for something in between, the probe hands back a non-fitting 4096, the footprint check discards it, and the refusal falls back to naming the native-sized cap. On a 2048-native model with room for 3072 the guard then contradicts itself: a request for 3072 launches, and a request for 8192 is refused with "the largest that fits is 2,048", sending the user to a third less context than the machine holds. Re-price under the floor when the floored result fails validation, which is the pass the cap above already makes for the same reason. Guarded on both sides, so it can only raise a ceiling the budget vouches for. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Publish the extended Metal ceiling, not just refuse against it The above-native extension probe raised _apple_measured_ceiling but left max_available_ctx at the model's native length. That value is published as max_context_length, and both amber warnings fire when the loaded context exceeds it, so a 131,072-token request on a 32,768-native model loaded fine and then arrived with "context length exceeds what fits in unified memory (32,768 tokens)" against a context this branch had just measured and allowed. Copy the accepted ceiling into the published bound. The fit is bounded by the request, so this raises the bound to the context that actually loaded and no further, and a refusal (which accepts no ceiling at the request) leaves it where the cap put it. * Tighten comments in the Metal unified memory context guard Collapse the multi line rationales added by this PR to their shortest form that still carries the reason: why the fit helper's 4096 is a floor and not a measurement, why the refusal is raised outside the placement try/except, why the paravirtual CPU path is exempt, and why max_available_ctx is published. Comments, docstrings and whitespace only, no behaviour change. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |