mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-08-21 14:25:35 +00:00
In autoswap mode, a POST to /v1/completions or /v1/chat/completions carrying a `model` name that is not an entry in the admin dir set `model_switch_pass = True` before checking the whitelist. No swap was performed, but the flag suppressed the request-type dispatch below it, so the text model was never loaded on demand. The same requests without a `model` field, and every other model type (stt/tts/embed/music/image), skip that branch entirely and load fine -- which is why only chat was affected, and why sending one model-less request worked around it. It also recurs after --adminunloadtimeout fires, since the "nomodel" state is recovered from by that same dispatch. Only set the flag on the path that actually issues the reload.
This commit is contained in:
parent
152e080b6a
commit
9fdd21de1b
1 changed files with 1 additions and 1 deletions
|
|
@ -4982,11 +4982,11 @@ class KcppProxyHandler(http.server.BaseHTTPRequestHandler):
|
|||
is_different_model = True
|
||||
|
||||
if is_different_model or was_auto_unloaded:
|
||||
model_switch_pass = True
|
||||
whitelist = get_current_admindir_list() # see if its an allowed swap
|
||||
if was_auto_unloaded and not model_name:
|
||||
model_name = "initial_model"
|
||||
if is_different_model and (model_name in whitelist):
|
||||
model_switch_pass = True # only claim the request if we really are swapping
|
||||
global_memory["last_active_timestamp"] = datetime.now()
|
||||
global_memory["triggered_sleeping"] = False
|
||||
reqbody = json.dumps({"filename":model_name})
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue