In autoswap mode, a POST to /v1/completions or /v1/chat/completions
carrying a `model` name that is not an entry in the admin dir set
`model_switch_pass = True` before checking the whitelist. No swap was
performed, but the flag suppressed the request-type dispatch below it,
so the text model was never loaded on demand.
The same requests without a `model` field, and every other model type
(stt/tts/embed/music/image), skip that branch entirely and load fine --
which is why only chat was affected, and why sending one model-less
request worked around it. It also recurs after --adminunloadtimeout
fires, since the "nomodel" state is recovered from by that same
dispatch.
Only set the flag on the path that actually issues the reload.
The reasoning budget derived from reasoning_effort never applied to Mistral
models. gpttype_adapter.cpp picks the think delimiters from a switch on the
model architecture, and mistral3 has no case, so it falls back to <think> /
</think>. Those are not vocabulary tokens for Ministral-3, so TokenizeString
returns more than one token each, the expected_start/end_tokens guard clears
all three vectors, and apply_reasoning_budget() returns at its first if.
The parameter is accepted, converted and passed down to the sampler, then
dropped on a size check, with nothing logged.
Adding the mistral3 case arms the budget. [THINK] and [/THINK] are single
vocabulary tokens (ids 34 and 35 on Ministral-3), so the size guard passes.
The thinkformats entry is a separate fix for a separate defect: without it the
thinking block was never split out, so it leaked into content with its [THINK]
marker still in it, instead of going to reasoning_content.
Measured on Ministral-3-14B-Reasoning-2512 (IQ4_XS, ctx 8192, --jinja), 5 real
prompts x 3 samples per cell, max_tokens 3000 (so a 750-token budget at "low"):
reasoning_effort thinking words before thinking words after
none 311 - 2314 7 (the forced-close phrase)
low 340 - 2255 521 - 574
Forced closes: 0/15 before, 14/15 after at "low" and 15/15 at "none". Three
samples per cell because this model's variance at temperature 0.7 spans a
factor of 4 on an identical payload — a single sample per cell cannot tell an
effect from noise.
No regression on a non-reasoning mistral3 model: Ministral-3-8B-Instruct with
reasoning_effort "low" returns finish_reason "stop", a normal answer and zero
forced closes, since apply_reasoning_budget() bails out when the start marker
never appears.
When a tool/MCP result carries an image, the OpenAI-compatible chat
adapter's jinja code path left the base64 payload in the rendered
prompt as plain text (a single 1024x1024 jpeg bloated the context by
~120k tokens), while the legacy path already stripped it via
strip_mcpcontent_of_media.
- format_jinja now strips the base64 from tool-role string content
before rendering, matching the legacy path; the image itself is
still swept out and attached separately.
- sweep_media_from_messages now also recognizes MCP-style image
content blocks (type == "image") inside a content list, so images
delivered that way are attached instead of dropped.
* sd: sync with master-782-b290693
* sd: sync with master-788-8a51eb9
* sd: sync with master-789-5114672
* sd: sync with master-795-87a0177
* sd: expose ref_image_args and make it trigger edit mode
Treat the presence of sdtiledvae separately from its truthiness so the documented zero value survives a launcher config reload. Fixes#2065.
Co-authored-by: LK-Customs <lolikun@sunpraise-mc.com>
* fix(api): don't iterate a string banned_tokens character by character (#2333)
banned_tokens/banned_strings are consumed as a list of substrings. When a
caller supplies a bare string instead, `banned_tokens[:ban_token_max]`
slices it and `for tok in banned_tokens` walks it one character at a time,
so every letter in the value becomes its own banned substring. Banning
common letters like "e" makes generation collapse into garbage, with no
error to point at the cause.
Coerce a string value into a single-element list, matching how the OpenAI
`stop` parameter is already normalized in transform_genparams.
* fix(api): parse a JSON-array string in banned_tokens instead of inerting it
Per review: coercing the string form to a single-element list left the
reported gendefaults case a seemingly-effective no-op. Parse a JSON array
supplied as a string so the reported config bans "exclude" as intended,
and keep the bare-string form ("exclude") working as a single substring ban.
---------
Co-authored-by: Anai-Guo <antai12232931@anaiguo.com>
The two websearch status messages in koboldcpp.py print with no leading
newline, so they run onto the end of the preceding generation-stats line
(which has no trailing newline), e.g.:
...Total:26.31sReturning cached websearch...
Give both the cached-result print and the new-search print a leading
newline so each starts on its own line, matching the approach already
taken elsewhere (adding the missing newline at the offending print
rather than changing the stats line). Fixes#2337.
Co-authored-by: Anai-Guo <antai12232931@anaiguo.com>
* sd: minor API path handling cleanup
* sd: sync with master-749-b11c95a
* sd: use original API parameters at the internal C++ API
* sd: split_mode and auto_fit backend support
* sd: sync with master-758-c674225
* sd: sync with master-765-bb84971
* sd: sync with master-767-885f01a
