vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-07-10 01:18:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

server: (router) move model downloading to dedicated process (#24834)

Browse source 
* server: real-time model load progress tracking via /models/sse

* update docs

* server: move model download to child process

* rm unused

* fix most problems

* clean up

* nit fixes

* fix test case

* do not detact() thread

* shorter MODEL_DOWNLOAD_TIMEOUT in test

* throttle
This commit is contained in:
Xuan-Son Nguyen 2026-06-22 18:24:04 +02:00 • committed by GitHub
parent 6ee0f65793
commit 721354fbdf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 312 additions and 152 deletions
Download patch file Download diff file Expand all files Collapse all files

13
tools/server/server.cpp
View file

@ -134,6 +134,7 @@ int llama_server(int argc, char ** argv) {
//
// register API routes
server_child child; // only used in non-router mode
server_routes routes(params, ctx_server);
server_tools tools;
@ -254,11 +255,21 @@ int llama_server(int argc, char ** argv) {
ctx_http.post("/tools", ex_wrapper(tools.handle_post));
}
//
// Handle downloading model
//
if (child.is_child() && child.get_mode() == SERVER_CHILD_MODE_DOWNLOAD) {
return child.run_download(params);
} else if (!is_router_server) {
// single-model mode (NOT spawned by router)
common_params_handle_models(params, LLAMA_EXAMPLE_SERVER);
}
//
// Start the server
//
server_child child; // only used in non-router mode
std::function<void()> clean_up;
if (is_router_server) {