vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-10 00:45:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
14404 commits 3 branches 251 tags 856 MiB
Commit graph

1450 commits

Author SHA1 Message Date
Concedo
f57cd915a9 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/ui-publish.yml
#	CODEOWNERS
#	docs/ops.md
#	ggml/CMakeLists.txt
#	ggml/src/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/argsort-ops.c
#	scripts/sync-ggml.last
#	tools/mtmd/tests/test-deepseek-ocr.py
 2026-07-11 11:36:59 +08:00
Concedo
9d8a50378a Merge commit '961e4b26a7' into concedo_experimental 
# Conflicts:
#	.github/workflows/hip-quality-check.yml
#	AGENTS.md
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/concat-ops.c
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.h
#	ggml/src/ggml-hexagon/htp/hmx-fa-kernels.h
#	ggml/src/ggml-hexagon/htp/hmx-queue.c
#	ggml/src/ggml-hexagon/htp/hmx-queue.h
#	ggml/src/ggml-hexagon/htp/hmx-utils.h
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/hvx-utils.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/rope-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-hexagon/htp/worker-pool.c
#	ggml/src/ggml-hexagon/htp/worker-pool.h
#	ggml/src/ggml-hip/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32_q4_0.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32_q8_0.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_mxfp4_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q4_0_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q4_1_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q4_k_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q5_0_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q5_1_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q5_k_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q6_k_f32_ns.cl
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_vec_split.wgsl
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tools/cli/CMakeLists.txt
#	tools/cli/cli.cpp
#	tools/llama-bench/llama-bench.cpp
 2026-07-11 11:13:27 +08:00
Concedo
0f0245161e Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	tests/test-backend-ops.cpp
#	tests/test-quantize-fns.cpp
#	tools/server/README.md
#	tools/ui/src/lib/constants/settings-registry.ts
 2026-07-11 09:20:40 +08:00
Concedo
fc6e197fdd Merge commit '33ca0dcb9d' into concedo_experimental 
# Conflicts:
#	docs/backend/SYCL.md
#	docs/build.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	ggml/src/ggml-cuda/ggml-cuda.cu
#	ggml/src/ggml-hip/CMakeLists.txt
#	ggml/src/ggml-opencl/fa_tune.h
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/flash_attn_f16.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32_q4_0.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32_q8_0.cl
#	ggml/src/ggml-opencl/kernels/mul_mv_f16_f32_l4.cl
#	ggml/src/ggml-sycl/backend.hpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/cpy.cpp
#	ggml/src/ggml-sycl/cpy.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/presets.hpp
#	src/llama-graph.cpp
#	src/llama-kv-cache.cpp
 2026-07-11 08:46:16 +08:00
Xuan-Son Nguyen
22b69b6e92
arg: prevent duplicate spec model downloads (#25527) 2026-07-10 16:53:26 +02:00
Xuan-Son Nguyen
c4ae9a88f8
server: improve tools, remove apply_diff (#25498) 
* server: improve tools, remove apply_diff

* improve edit tool

* add tools_io abstraction

* add tools_io_basic

* fix build

* move utils to class member

* add const
 2026-07-10 11:52:59 +02:00
Xuan-Son Nguyen
3de7dd4c8f
cli: add --output option (#25484) 2026-07-09 19:37:39 +02:00
Xuan-Son Nguyen
c264f65ff9
cli : move to HTTP-based implementation (#24948) 
* cli: move to HTTP-based implementation

* wip

* working

* remote server ok

* cli support router mode

Co-authored-by: Piotr Wilkin <ilintar@gmail.com>

* case: router with only one model

* Apply suggestions from code review

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>

* remove outdated comment

* use destructor instead

* add ftype

* cli-view --> cli-ui

* pimpl

* no more json in header

* nits fixes

* also show model aliases

---------

Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
 2026-07-08 14:52:43 +02:00
rankaiyx
f296fdfbed
common: auto-create prompts-log-dir at argument parsing, so all tools using the flag benefit (#25322) 2026-07-08 09:45:28 +02:00
Georgi Gerganov
c198af4dc2
spec : fix naming, spacing (#25410) 2026-07-07 18:52:30 +03:00
Alex
f5525f7e7a
server : fix draft model fit vs load inconsistency (#25056) 
* fix: draft model fit vs load inconsistency

* refactor(server): unify draft/mtp parameter initialization, model, and context load
- moves speculative init to speculative.cpp
- changes server_context_impl model_dft and ctx_dft to use raw pointers

- fix: don't throttle progress callback when loading draft model
- refactor: rename draft model/ctx load method

* fix: valign
 2026-07-07 17:20:42 +03:00
zhangrunda
c1a411fb1b
common : add missing <fstream> include in common.h (#25220) 
Signed-off-by: zhangrunda <zhangrunda1234@outlook.com>
 2026-07-07 15:23:53 +02:00
o7si
defa95c306
speculative : fix out-of-bounds read in ngram-map on prompt shrink (#23936) 
* speculative : fix out-of-bounds read in ngram-map on prompt shrink

* speculative : fix ngram-map cleanup cutoff after prompt shrink
 2026-07-07 10:25:04 +03:00
shalinib-ibm
ee445f93d8
common: Set optimal default thread count for ppc ( linux as well as AIX) (#25237) 2026-07-07 05:35:20 +08:00
Concedo
e944cca86f Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	scripts/sync_vendor.py
#	src/llama-model-loader.cpp
#	tests/test-backend-ops.cpp
#	tests/test-chat-auto-parser.cpp
#	tests/test-chat.cpp
#	tools/cli/cli.cpp
#	tools/server/README.md
 2026-07-05 11:30:12 +08:00
Piotr Wilkin (ilintar)
2d973636e2
chat: trim messages sent to StepFun parser (fixes long reasoning loops) (#25238) 
* chat: trim messages sent to StepFun parser (fixes long reasoning loops)

* add regression test; remove duplicate template

* chat: trim StepFun content parts before rendering

The StepFun trim workaround ran on the already-rendered messages, where
typed content parts have been concatenated into a single string, so the
per-part whitespace could no longer be reached. Move the trim ahead of
rendering and apply it to content_parts text as well as the string
content and reasoning_content. Adds a content-parts regression test.

Co-Authored-By: Piotr Wilkin <ilintar@gmail.com>
Assisted-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: tarruda <tpadilha84@gmail.com>
 2026-07-03 23:12:11 +02:00
Ruixiang Wang
152d337fad
spec: support spec-draft-p-min in DFlash (#25246) 
* spec: support spec-draft-p-min in DFlash

* dflash: add n_min guard

* dflash: guard both n_min and n_max
 2026-07-03 15:40:06 +02:00
Concedo
56d11ad4e8 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	docs/backend/OPENCL.md
#	ggml/src/ggml-hexagon/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/hex-dma.h
#	ggml/src/ggml-hexagon/htp/hmx-utils.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-base.h
#	ggml/src/ggml-hexagon/htp/hvx-exp.h
#	ggml/src/ggml-hexagon/htp/hvx-sigmoid.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/cvt.cl
#	scripts/snapdragon/adb/run-completion.sh
#	scripts/snapdragon/adb/run-tool.sh
#	scripts/snapdragon/ggml-hexagon-profile.py
#	tests/test-backend-ops.cpp
 2026-07-02 21:42:36 +08:00
Adrien Gallouët
a6647b1a32
common : use hf primary split as model path (#25194) 
Fixes #25181
 2026-07-01 18:33:00 +02:00
Concedo
cb36463e4a Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
 2026-06-30 22:40:37 +08:00
Pascal
799fcc04a5
common,server: handle bracketed IPv6 literals in URL authority (#25140) 
* common,server: handle bracketed IPv6 literals in URL authority

Parse the [host]:port form (RFC 3986) and bracket IPv6 hosts when
formatting a URL authority: listening log, proxy Host header, proxy
log, client rebuild. The per-request remote_addr stays bare.

* common: restore unsupported scheme throw in url parser

Address @ngxson review: keep the explicit reject in port resolution so
the block stays self-contained. Non-http(s) schemes still throw (also
gated at the top of common_http_parse_url).
 2026-06-30 16:16:44 +02:00
Concedo
61ad97cbc1 Merge commit '8c146a8366' into concedo_experimental 
# Conflicts:
#	src/CMakeLists.txt
#	tests/test-llama-archs.cpp
 2026-06-30 22:00:03 +08:00
Adrien Gallouët
6f4f53f2b7
common : dedup preset and cached model entries in /v1/models (#25131)
Some checks failed
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Details
Python Type-Check / python type-check (push) Has been cancelled
Details 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-06-29 17:37:23 +02:00
Concedo
16ef2badf6 fix superfluous filename defines 2026-06-29 21:04:09 +08:00
Concedo
3b867bd4b1 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/release.yml
#	SECURITY.md
#	common/CMakeLists.txt
#	docs/speculative.md
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/cvt.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f16.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
#	ggml/src/ggml-opencl/kernels/set_rows.cl
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-sycl/norm.cpp
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-chat-template.cpp
#	tests/test-chat.cpp
#	tests/test-export-graph-ops.cpp
#	tests/test-jinja.cpp
#	tests/test-llama-archs.cpp
#	tools/rpc/CMakeLists.txt
#	tools/rpc/README.md
 2026-06-29 16:43:44 +08:00
o7si
277a105dc8
common : remove unused regex-partial (#25118)
Some checks are pending
Python Type-Check / python type-check (push) Waiting to run
Details
2026-06-29 08:48:39 +02:00
Xuan-Son Nguyen
b3fed31b99
jinja, chat: add --reasoning-preserve flag (#25105) 
* jinja, chat: add --reasoning-preserve flag

* correct help message
 2026-06-28 23:33:51 +02:00
Aldehir Rojas
c818263f2a
chat : implement minicpm5 parser (#24889) 
* Add minicpm5 tool call parser

* Refactor MiniCPM5 PEG parser per review feedback

* Fix jinja min/max API to match Jinja2

* modify by review

* MiniCPM5: use autoparser for XML tool calls and fix grammar preserved-token triggers

* MiniCPM5: fix streaming tool-arg placeholder and remove alt XML markers

* skip min/max attribute tests in -py mode

* test-jinja: use real expected output for min/max attribute tests

* MiniCPM5: revert shared mapper and history fallbacks per review

Drop streaming tool-arg placeholder workarounds from the generic PEG
mapper and restore strict tool-call argument JSON parsing so MiniCPM5
support stays limited to autoparser/diff-analyzer changes.

* chat : refactor minicpm5 back to dedicated parser

* cont : simplify grammar

* cont : refactor

* cont : fixes

* cont : rename template to openbmb-MiniCPM5-1B.jinja

* cont : add message delimiters

* cont : fix tests

---------

Co-authored-by: zhangtao <zhangtao2@modelbest.cn>
Co-authored-by: 张涛 <>
 2026-06-28 16:53:32 +02:00
Xuan-Son Nguyen
f68a788b0b
jinja: add --dump-prog for debugging (#25086) 
* jinja: add --dump-prog for debugging

* Update common/jinja/runtime.cpp

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

---------

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>
 2026-06-28 15:50:31 +02:00
Ruixiang Wang
d1b34251bc
spec : add DFlash support (#22105) 
* spec: add DFlash v2 support

* dflash: support sliding window attention per layer_types

* docs: add dflash section

---------

Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
 2026-06-28 16:01:34 +03:00
Adrien Gallouët
c1a1c8ee94
common : allow --offline in llama download (#25091) 
Expose the existing --offline flag to `llama download` so a script can
run it to check whether a model is already cached and ready to be served
without touching the network.

Also fix a latent use-after-free in the URL-task on_done callback:
first_path is block-scoped and was captured by reference, but invoked
after the block ends.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-06-28 12:34:11 +02:00
Georgi Gerganov
27c8bb4f63
logs : reduce v2 (#25078) 
* server : reduce logs

* cont : common

* cont : spec

* cont : CMN_ -> COM_
 2026-06-28 08:52:15 +03:00
Concedo
87aeaff675 fix builds 2026-06-27 18:50:05 +08:00
Concedo
8a5b7084f4 fix tools build 2026-06-27 16:48:53 +08:00
Concedo
e27861e14e Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/cann.Dockerfile
#	.devops/cpu.Dockerfile
#	.devops/cuda.Dockerfile
#	.devops/intel.Dockerfile
#	.devops/musa.Dockerfile
#	.devops/openvino.Dockerfile
#	.devops/rocm.Dockerfile
#	.devops/s390x.Dockerfile
#	.devops/vulkan.Dockerfile
#	.devops/zendnn.Dockerfile
#	.github/workflows/build-cache.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/release.yml
#	app/llama.cpp
#	build-xcframework.sh
#	docs/backend/OPENVINO.md
#	ggml/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-openvino/ggml-decoder.cpp
#	ggml/src/ggml-openvino/openvino/op/add_id.cpp
#	ggml/src/ggml-openvino/openvino/op/glu_swiglu.cpp
#	ggml/src/ggml-openvino/openvino/op/mul_mat_id.cpp
#	ggml/src/ggml-openvino/openvino/op/softmax.cpp
#	ggml/src/ggml-openvino/openvino/op_table.cpp
#	ggml/src/ggml-openvino/openvino/op_table.h
#	ggml/src/ggml-sycl/softmax.cpp
#	scripts/sync-ggml.last
#	tests/test-backend-ops.cpp
#	tests/test-quantize-fns.cpp
#	tools/server/CMakeLists.txt
#	tools/ui/src/lib/services/chat.service.ts
 2026-06-27 10:33:29 +08:00
Concedo
4e43c21e58 Merge commit '9d5d882d8c' into concedo_experimental 
# Conflicts:
#	.github/labeler.yml
#	app/CMakeLists.txt
#	app/llama.cpp
#	build-xcframework.sh
#	common/CMakeLists.txt
#	common/download.h
#	docs/backend/SYCL.md
#	docs/backend/snapdragon/CMakeUserPresets.json
#	docs/speculative.md
#	ggml/CMakeLists.txt
#	ggml/include/ggml-sycl.h
#	ggml/src/ggml-hexagon/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/cmake-toolchain.cmake
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/hex-dma.h
#	ggml/src/ggml-hexagon/htp/hex-utils.h
#	ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/htp_iface.idl
#	ggml/src/ggml-hexagon/htp/hvx-base.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/libggml-htp.inf
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/norm.cl
#	ggml/src/ggml-sycl/conv3d.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	scripts/snapdragon/adb/run-completion.sh
#	scripts/snapdragon/adb/run-tool.sh
#	scripts/snapdragon/ggml-hexagon-profile.py
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-thread-safety.cpp
#	tools/llama-bench/llama-bench.cpp
#	tools/mtmd/CMakeLists.txt
#	tools/mtmd/tests/test-deepseek-ocr.py
 2026-06-27 10:18:52 +08:00
Xuan-Son Nguyen
024930c6ad
arg: fix handling --spec-draft-hf and --hf-repo-v (#25043) 
* arg: fix handling --spec-draft-hf and --hf-repo-v

* fix missing mparams.hf_file
 2026-06-26 14:36:03 +02:00
Xuan-Son Nguyen
60bc8866b1
common: refactor model handling (#24980) 
* common: refactor models handling

* remote preset

* cont

* rm skip_download option

* missing header

* fix plan.model_files

* fix --offline case

* move hf_plan to download

* refactor

* rm redundant curr_ex, add comments

* adapt
 2026-06-25 15:17:51 +02:00
Adrien Gallouët
683b04cc4a
app : add the llama download subcommand (#24982) 
* app : add the download command (with llama-download)

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Remove llama-download tool for now

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-06-25 13:36:36 +02:00
Piotr Wilkin (ilintar)
09cedfd699
chat: harden caps check (#24973) 2026-06-25 02:49:22 +02:00
Xuan-Son Nguyen
fb401045cc
common: remove unused json-partial (#24968) 2026-06-24 18:12:16 +02:00
Concedo
579229d157 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	CODEOWNERS
#	README.md
#	ggml/src/ggml-opencl/kernels/gemv_noshuffle_q8_0_f32.cl
#	ggml/src/ggml-sycl/binbcast.cpp
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-vulkan/CMakeLists.txt
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id_vec.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_q_acc.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/quantize_q8.wgsl
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tests/test-sampling.cpp
#	tools/server/README.md
 2026-06-24 23:28:21 +08:00
Xuan-Son Nguyen
75ad0b23ed
server: fix remote preset handling, add test (#24938) 
* server: add test for remote preset

* fix remote preset handling

* fix

* fix test
 2026-06-23 13:28:34 +02:00
Aldehir Rojas
73618f27a8
server: improve user message detection and create checkpoints at every user message (#24176) 
* server : improve message span logic

* cont : cast size_t to int32_t in comparisons

* server : create checkpoints before every user msg

* chat : remove \n in gemma4 delimiters

* chat : merge msg delimiter structs into one

* cont : reword comment

* cont : initialize tokens in delimiter

* cont : add server_tokens::get_raw_tokens() for mtmd

* cont : move message finding to server_tokens and skip mtmd tokens

* cont : update cohere2moe parser

* cont : increase min-step to 8192 and always produce a chkpt for last user message
 2026-06-23 08:27:28 +03:00
Xuan-Son Nguyen
721354fbdf
server: (router) move model downloading to dedicated process (#24834) 
* server: real-time model load progress tracking via /models/sse

* update docs

* server: move model download to child process

* rm unused

* fix most problems

* clean up

* nit fixes

* fix test case

* do not detact() thread

* shorter MODEL_DOWNLOAD_TIMEOUT in test

* throttle
 2026-06-22 18:24:04 +02:00
Concedo
3090ae0bf7 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/s390x.Dockerfile
#	.dockerignore
#	.github/workflows/docker.yml
#	.github/workflows/release.yml
#	docs/android.md
#	ggml/src/ggml-cpu/amx/mmq.cpp
#	ggml/src/ggml-hexagon/htp/ssm-conv.c
#	tests/peg-parser/test-gbnf-generation.cpp
#	tests/test-arg-parser.cpp
#	tests/test-chat.cpp
#	tests/test-jinja.cpp
#	tests/test-json-schema-to-grammar.cpp
#	tools/server/README.md
 2026-06-22 18:23:59 +08:00
Aldehir Rojas
52b3df0023
common/peg : implement ac parser for stricter grammar generation (#24869) 
* common/peg : implement ac parser

* cont : extract functions

* cont : tidy up

* cont : remove a test

* cont : move ac() def
 2026-06-21 16:20:58 -05:00
Sigbjørn Skjæret
bf533823cd
jinja : implement call statement (#24847) 
* implement call statement

* undo unintended change

* de-lambda

* simplify

* move caller context inside function handler
 2026-06-21 14:04:52 +02:00
Georgi Gerganov
8a118ee86c
minor : clean-up whitespaces (#24862) 
[no ci]
 2026-06-21 11:37:12 +03:00
YiChen Lv
d789527482
spec : Support Step3.5/3.7 flash mtp3 (#24340) 
* add mtp_layer_offset + include nextn flags in graph reuse

* add llama_set_mtp_layer_offset + llama_model_n_nextn_layer API

* offset head select + require all MTP blocks

* speculative multi-head process()

* speculative multi-head draft()

* gather outputs via inp_out_ids

* cleanup

* fix core

* minor cleanup

* merged draft_multi_head into draft()

* mtp rename nextn

* Apply suggestions from code review

Co-authored-by: Aman Gupta <amangupta052@gmail.com>

* clean-up comments

* fix for multi seq

* apply suggestions && chain-heads comment

* add a reference for chain_heads discussion

---------

Co-authored-by: Aman Gupta <amangupta052@gmail.com>
 2026-06-21 11:33:18 +03:00