vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-07-22 15:23:34 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
13650 commits 3 branches 249 tags 833 MiB
Commit graph

1145 commits

Author SHA1 Message Date
Concedo
e541554d70 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/ui-build.yml
#	.github/workflows/ui-publish.yml
#	.github/workflows/ui-self-hosted.yml
#	CMakeLists.txt
#	app/CMakeLists.txt
#	app/llama.cpp
#	common/arg.cpp
#	ggml/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	scripts/snapdragon/ggml-hexagon-profile.py
#	scripts/sync-ggml.last
#	src/CMakeLists.txt
#	tests/test-llama-archs.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/mtmd/CMakeLists.txt
#	tools/mtmd/tests/test-deepseek-ocr.py
#	tools/server/README.md
 2026-05-30 01:54:53 +08:00
Concedo
baacd640c0 Merge commit 'dd1557907a' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-vulkan.yml
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
#	tests/test-backend-ops.cpp
#	tests/test-chat-template.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tools/perplexity/perplexity.cpp
#	tools/server/README.md
 2026-05-30 01:40:09 +08:00
Xuan-Son Nguyen
06d26dfdff
download: add option to skip_download (#23059)
Some checks are pending
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Waiting to run
Details
Python check requirements.txt / check-requirements (push) Waiting to run
Details
Python Type-Check / python type-check (push) Waiting to run
Details 
* download: add option to skip_download

* fix

* fix 2

* if file doesn't exist, respect skip_download flag
 2026-05-29 16:30:55 +02:00
Saba Fallah
da3f990a47
mtmd: Add DeepSeekOCR 2 Support (#20975) 
* mtmd: DeepSeek-OCR 2 support, with multi-tile dynamic resolution

* introduced clip_image_f32::add_viewsep

* address PR review

- drop redundant ggml_cpy ops in both deepseekocr versions build
- drop no-op ggml_cont in build_sam
- assert num_image_tokens deepseekocr2
- view_seperator as (1, n_embd) at conversion (for both versions)
- drop redundant ggml_reshape_2d

* Update tools/mtmd/models/deepseekocr2.cpp

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
 2026-05-29 16:13:51 +02:00
Xuan-Son Nguyen
cb47092b00
server: bump timeout to 3600s (#23842) 
* server: bump timeout to 3600s

* nits: change wording
 2026-05-29 10:23:17 +02:00
Adrien Gallouët
98e480a32e
app : move licences to llama-app (#23824) 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-05-29 07:46:11 +02:00
Xuan-Son Nguyen
751ebd17a5
mtmd-debug: add color and rainbow mode (#23829) 
* mtmd-debug: add color and rainbow mode

* fix M_PI

* max_dist
 2026-05-28 20:59:14 +02:00
Xuan-Son Nguyen
c8914ad4f4
mtmd: fix gemma 4 projector pre_norm (#23822) 2026-05-28 20:58:55 +02:00
ValdikSS
2f6c815dc4
ui: fix audio and video modality detection (#23756) 
When model props are fetched asynchronously from the server,
modelPropsVersion is incremented to trigger reactivity, but
only the vision effect was listening to it.
 2026-05-28 17:36:10 +02:00
Saba Fallah
0b56d283bf
mtmd: n_head_kv defaults to n_head (#23782) 
removed AI-generated comment
 2026-05-28 16:44:36 +02:00
Xuan-Son Nguyen
d6be3158e1
mtmd: fix gemma 4 audio rms norm eps (#23815) 
* mtmd: fix gemma 4 audio rms norm eps

* Update tools/mtmd/clip.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-28 16:31:37 +02:00
Mikolaj Kucharski
7fb1e70b59
arg: Add LLAMA_ARG_API_KEY_FILE environment variable for --api-key-file (#23167) 2026-05-28 16:25:40 +02:00
Funtowicz Morgan
0b246862b9
server: minor tweaks to use more cpp features (#23785) 
* misc(server): add default port to impl RAII

* misc(server): register_gcp_compat() can be const

* misc(server): use proper cpp const/auto methods

* misc(server): do not reset a unique_ptr, use make_unique instead to be exception safe
 2026-05-28 14:00:25 +02:00
Markus Tavenrath
d205df6812
server, ui : Add support for HTTP ETags in llama-server (#23701) 
* allow caching of ui elements in llama-server

* use fnv_hash

* Update tools/server/server-http.cpp

etag has to be set always

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
 2026-05-28 12:21:24 +02:00
Adrien Gallouët
48e7eae41c
perplexity : fix format specifier in LOG_ERR (#23788) 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-05-28 10:34:58 +03:00
Concedo
ca942eb172 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/actions/windows-setup-cuda/action.yml
#	.github/workflows/build-3rd-party.yml
#	.github/workflows/build-android.yml
#	.github/workflows/build-apple.yml
#	.github/workflows/build-cann.yml
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/build-ibm.yml
#	.github/workflows/build-msys.yml
#	.github/workflows/build-opencl.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-riscv.yml
#	.github/workflows/build-rpc.yml
#	.github/workflows/build-sanitize.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/build-sycl.yml
#	.github/workflows/build-vulkan.yml
#	.github/workflows/build-webgpu.yml
#	.github/workflows/hip-quality-check.yml
#	.github/workflows/release.yml
#	.github/workflows/server-sanitize.yml
#	.github/workflows/server-self-hosted.yml
#	.github/workflows/server.yml
#	.github/workflows/ui-self-hosted.yml
#	.github/workflows/ui.yml
#	CONTRIBUTING.md
#	ci/run.sh
#	docs/autoparser.md
#	docs/multimodal/granitevision.md
#	examples/model-conversion/README.md
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-vulkan/CMakeLists.txt
#	ggml/src/ggml-vulkan/ggml-vulkan.cpp
#	ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/cpy.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id_gather.wgsl
#	ggml/src/ggml-zendnn/ggml-zendnn.cpp
#	pyproject.toml
#	scripts/snapdragon/adb/run-completion.sh
#	scripts/snapdragon/adb/run-tool.sh
#	scripts/sync_vendor.py
#	tests/gguf-model-data.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/server/README.md
 2026-05-28 15:28:29 +08:00
Georgi Gerganov
6b4e4bd582
common : fix env names to all have LLAMA_ARG_ prefix (#23778) 2026-05-27 14:52:47 +03:00
Radoslav Gerganov
7085492c6f
server : fix the log message when using SSL (#23393) 
When llama-server is started with SSL key and cert, the log says that it
listens on http instead of https. This patch fixes this.
 2026-05-27 08:06:30 +03:00
Concedo
9fbc0cd7dc Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-and-test-snapdragon.yml
#	.github/workflows/build-android.yml
#	.github/workflows/build-cache.yml
#	.github/workflows/build-cann.yml
#	.github/workflows/build-cross.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-riscv.yml
#	.github/workflows/build-sanitize.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/build-sycl.yml
#	.github/workflows/build-vulkan.yml
#	.github/workflows/build.yml
#	.github/workflows/release.yml
#	.github/workflows/server-sanitize.yml
#	ci/run.sh
#	docs/backend/SYCL.md
#	docs/backend/snapdragon/README.md
#	ggml/CMakeLists.txt
#	ggml/src/ggml-cuda/ggml-cuda.cu
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/get-rows-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-utils.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/rope-ops.c
#	ggml/src/ggml-hexagon/htp/set-rows-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
#	scripts/sync-ggml.last
#	tests/gguf-model-data.cpp
#	tests/test-backend-ops.cpp
 2026-05-27 00:26:08 +08:00
Concedo
9204f78926 Merge commit 'ae251b5ff2' into concedo_experimental 
# Conflicts:
#	.github/actions/linux-setup-spacemit/action.yml
#	.github/actions/unarchive-tar/action.yml
#	.github/workflows/build-android.yml
#	.github/workflows/build-cmake-pkg.yml
#	.github/workflows/build-cross.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/build.yml
#	.github/workflows/check-vendor.yml
#	.github/workflows/code-style.yml
#	.github/workflows/editorconfig.yml
#	.github/workflows/pre-tokenizer-hashes.yml
#	.github/workflows/python-check-requirements.yml
#	.github/workflows/python-lint.yml
#	.github/workflows/python-type-check.yml
#	.github/workflows/server-self-hosted.yml
#	.github/workflows/ui-build.yml
#	.github/workflows/ui.yml
#	.github/workflows/update-ops-docs.yml
#	ci/run.sh
#	docs/build-riscv64-spacemit.md
#	examples/convert_legacy_llama.py
#	ggml/cmake/ggml-config.cmake.in
#	ggml/src/CMakeLists.txt
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	scripts/sync_vendor.py
#	tests/test-chat-auto-parser.cpp
#	tests/test-chat.cpp
#	tests/test-gguf.cpp
#	tools/cli/README.md
#	tools/perplexity/perplexity.cpp
#	tools/server/README.md
 2026-05-26 22:01:57 +08:00
Pascal
5a4126adc1
ui: fix stop/continue during an agentic loop (#23356) 2026-05-25 14:18:59 +02:00
Aman Gupta
6c4cbdc70b
server: MTP layer kv-cache should respect draft type ctk (#23646) 2026-05-25 16:46:23 +08:00
Saba Fallah
b96487645c
ui: media attachments before text (#23467) 
* ui: media attachments before text

* fix prettier formatting
 2026-05-25 08:50:41 +02:00
jacekpoplawski
e2ef8fe42c
server: fix checkpoints creation (#22929) 
* common : add common_chat_split_by_role

* cont : fix spans to reach end of message

* server: fix checkpoints creation

- extract message_spans from chat templates
- find the prompt token position before the latest user message
- split prompt batching at that position
- create a context checkpoint before the latest user input
- avoid periodic mid-prompt checkpoints when that position is known
- handle multimodal prompts when mapping text/template positions to server prompt tokens
- add --checkpoint-min-step to control minimum spacing between checkpoints

* cont : clean-up

* Support autoparser detection for message barriers

* server: fix message span delimiter and update docs

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com>
 2026-05-25 08:56:18 +03:00
fairydreaming
6d57c26ef8
perplexity : fix even more integer overflows (#23623) 
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
 2026-05-25 08:12:39 +03:00
Aldehir Rojas
63248fc3e3
cmake : fix ui build (#23592) 
* cmake/ui : add -fPIC to llama-ui static lib

* cmake : rename host compiled embed helper
 2026-05-24 02:37:28 -05:00
Aman Gupta
83eebe9d08
server: add margin for draft model for fit (#23485) 2026-05-24 14:43:08 +08:00
Concedo
8ca4283f55 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/release.yml
#	.github/workflows/server.yml
#	.github/workflows/ui-build.yml
#	.github/workflows/ui-publish.yml
#	CMakeLists.txt
#	docs/autoparser.md
#	docs/backend/snapdragon/CMakeUserPresets.json
#	docs/backend/snapdragon/README.md
#	docs/backend/snapdragon/windows.md
#	docs/function-calling.md
#	examples/model-conversion/scripts/embedding/run-original-model.py
#	ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/cvt.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_mxfp4_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q4_0_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q4_1_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q4_k_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q5_0_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q5_1_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q5_k_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemm_moe_q6_k_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_mxfp4_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_q4_0_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_q4_1_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_q4_k_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_q5_0_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_q5_1_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_q5_k_f32_ns.cl
#	ggml/src/ggml-opencl/kernels/gemv_moe_q6_k_f32_ns.cl
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/gated_delta_net.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-vulkan/CMakeLists.txt
#	ggml/src/ggml-zendnn/CMakeLists.txt
#	ggml/src/ggml-zendnn/ggml-zendnn.cpp
#	requirements/requirements-convert_hf_to_gguf.txt
#	scripts/snapdragon/windows/setup-build.ps1
#	tools/perplexity/perplexity.cpp
 2026-05-24 13:55:44 +08:00
Aldehir Rojas
b22ff4b7b4
cmake/ui : refactor the build (#23352) 2026-05-23 17:08:22 -04:00
Aditya Singh
c0c7e147e7
requirements : bump torch to 2.11.0 (#23503) 
* requirements: relax torch~=2.6.0 to torch>=2.6.0 for convert_hf_to_gguf

The ~=2.6.0 operator resolves to >=2.6.0, <2.7.0, which fails on
PyPI for platform/CPython combinations where 2.6.x is not present.
The accompanying comment already says 'PyTorch 2.6.0 or later', so
the looser >=2.6.0 matches the documented intent and unblocks
pip install -r requirements/requirements-convert_hf_to_gguf.txt.

Fixes #23408

* requirements: bump torch floor to 2.11.0 per maintainer

* requirements: pin torch to ==2.11.0 per project policy

* requirements: pin mtmd torch and torchvision to 2.11.0/0.26.0 per project policy

* requirements: suppress check_requirements pin warning on mtmd

The check_requirements script flags '==' on lines in files matched by
*/**/requirements*.txt. Append the documented suppression comment to the
pinned torch and torchvision lines (and to the s390x platform marker lines)
so the check passes while keeping the pins required by project policy.

* ty: silence Tensor/Module union check on model[0].auto_model

With torch 2.11.0 stubs, nn.Sequential.__getitem__ now returns
Tensor | Module rather than Module, so model[0].auto_model fails ty
on the SentenceTransformer code path. The runtime behavior is
unchanged because SentenceTransformer always wraps a Module at
index 0. Adding a targeted unresolved-attribute ignore keeps the
type-check green without altering behavior. A follow-up issue
tracks typing the variable explicitly.
 2026-05-23 18:24:39 +02:00
Aldehir Rojas
1acee6bf89
server: only parse empty msg if continuing an assistant msg (#23506) 2026-05-22 11:58:15 -04:00
fairydreaming
ef570f6308
perplexity : fix integer overflow (#23496) 
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
 2026-05-22 15:50:44 +03:00
Concedo
632c41a72f Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-apple.yml
#	.github/workflows/build-cmake-pkg.yml
#	.github/workflows/release.yml
#	.pi/gg/SYSTEM.md
#	CMakeLists.txt
#	CODEOWNERS
#	README.md
#	build-xcframework.sh
#	ci/run.sh
#	docs/build.md
#	examples/CMakeLists.txt
#	examples/llama.android/lib/build.gradle.kts
#	ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_tile.wgsl
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-save-load-state.cpp
#	tools/batched-bench/CMakeLists.txt
#	tools/cli/CMakeLists.txt
#	tools/completion/CMakeLists.txt
#	tools/llama-bench/CMakeLists.txt
#	tools/perplexity/CMakeLists.txt
#	tools/quantize/CMakeLists.txt
#	tools/server/CMakeLists.txt
 2026-05-22 20:42:51 +08:00
Georgi Gerganov
bbce619adb
cmake : add install() for impl libraries + fix apple builds (#23511) 
* pi : update

* ci : fix ios build

* ci : fix andoroid

* ci : fix apple builds

* cmake : add install() for impl libraries

Add install(TARGETS <target> LIBRARY) for all -impl libraries that were
changed from STATIC to shared (controlled by BUILD_SHARED_LIBS) in
commit bb28c1fe2. Without this, cmake --install fails to copy the shared
libraries, causing runtime errors like:

  llama-server: error while loading shared libraries: libllama-server-impl.so

Ref: https://github.com/ggml-org/llama.cpp/issues/23494#issuecomment-4512912515

Assisted-by: llama.cpp:local pi

* ci : fix xcframework build
 2026-05-22 11:46:26 +03:00
Georgi Gerganov
bb28c1fe24
cmake : remove STATIC from impl libraries, enable LLAMA_BUILD_APP by default (#23462) 
* cmake : remove STATIC from impl libraries, allow BUILD_SHARED_LIBS control

Remove explicit STATIC from all -impl libraries (server, cli, completion, bench,
batched-bench, fit-params, quantize, perplexity) so BUILD_SHARED_LIBS controls
shared vs static linkage.

Add WINDOWS_EXPORT_ALL_SYMBOLS ON for proper DLL export on Windows.

Assisted-by: llama.cpp:local pi

* cmake : enable LLAMA_BUILD_APP by default

Assisted-by: llama.cpp:local pi

* ci : disable app in build-cmake-pkg.yml
 2026-05-21 21:13:59 +03:00
Concedo
718dc159b6 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	CMakeLists.txt
#	docs/speculative.md
#	ggml/src/ggml-cuda/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-ops.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/rope-ops.c
#	ggml/src/ggml-hexagon/htp/ssm-conv.c
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	scripts/snapdragon/adb/run-bench.sh
#	scripts/snapdragon/adb/run-cli.sh
#	scripts/snapdragon/adb/run-completion.sh
#	scripts/snapdragon/adb/run-mtmd.sh
#	scripts/snapdragon/windows/run-bench.ps1
#	scripts/snapdragon/windows/run-cli.ps1
#	scripts/snapdragon/windows/run-completion.ps1
#	scripts/snapdragon/windows/run-mtmd.ps1
#	src/llama-vocab.cpp
#	tests/test-backend-ops.cpp
#	tools/batched-bench/CMakeLists.txt
#	tools/batched-bench/batched-bench.cpp
#	tools/cli/CMakeLists.txt
#	tools/cli/README.md
#	tools/cli/cli.cpp
#	tools/completion/CMakeLists.txt
#	tools/completion/README.md
#	tools/llama-bench/CMakeLists.txt
#	tools/llama-bench/llama-bench.cpp
#	tools/mtmd/CMakeLists.txt
#	tools/mtmd/tests/test-deepseek-ocr.py
#	tools/mtmd/tests/tests-requirements.txt
#	tools/perplexity/CMakeLists.txt
#	tools/perplexity/perplexity.cpp
#	tools/quantize/CMakeLists.txt
#	tools/server/CMakeLists.txt
#	tools/server/README.md
#	ty.toml
 2026-05-21 23:47:21 +08:00
Concedo
54af9aada9 Merge commit 'e6b4acfe86' into concedo_experimental 
# Conflicts:
#	.devops/cann.Dockerfile
#	.devops/cpu.Dockerfile
#	.devops/cuda.Dockerfile
#	.devops/intel.Dockerfile
#	.devops/musa.Dockerfile
#	.devops/openvino.Dockerfile
#	.devops/rocm.Dockerfile
#	.devops/s390x.Dockerfile
#	.devops/vulkan.Dockerfile
#	tools/mtmd/clip.cpp
#	tools/mtmd/clip.h
 2026-05-21 23:31:32 +08:00
ScrewTSW
b65bb4baae
server: expose prompt token counts in /slots endpoint (#23454) 
Add n_prompt_tokens, n_prompt_tokens_processed, and n_prompt_tokens_cache
to the /slots JSON response. These fields are already tracked internally
but were not exposed, making it impossible for clients to monitor prompt
evaluation progress during processing.
 2026-05-21 13:29:13 +02:00
Aman Gupta
52fb93a2bd
server : free draft/MTP resources on sleep to fix VRAM leak (#23461) 
The destroy() function in server_context_impl only cleaned up the main
model and context (via llama_init.reset()) but did not free the speculative
decoder (spec), draft context (ctx_dft), or draft model (model_dft).

For MTP (Multi-Token Prediction) models, ctx_dft holds GPU-allocated
resources (KV cache, compute buffers) that are not freed when entering
the sleeping state. On each sleep/resume cycle, new resources are
allocated without the old ones being freed, leading to a VRAM leak
that eventually crashes the server with out-of-memory errors.

Fix by explicitly resetting spec, ctx_dft, and model_dft in destroy()
before resetting llama_init, ensuring proper cleanup order to avoid
use-after-free.

ref: https://github.com/ggml-org/llama.cpp/issues/23395

Assisted-by: llama.cpp:local pi
 2026-05-21 16:11:11 +08:00
Pascal
c9021714e8
server: re-inject subcommand when router spawns children under unified binary (#23442) 2026-05-21 10:09:19 +02:00
Adrien Gallouët
1d7ab2b947
app : add batched-bench, fit-params, quantize & perplexity (#23459)
Some checks are pending
Python Type-Check / python type-check (push) Waiting to run
Details 
* app : add batched-bench, fit-params, quantize & perplexity

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Add missing main.cpp

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Add EOL

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-05-21 10:29:44 +03:00
Aleksander Grygier
5e932a1c8d
ui: Improve Git Hooks for UI development (#23403) 
* refactor: Improve Git Hooks for UI development

* fix: Address review comments

* fix: Use absolute git path for `/hooks`

Co-authored-by: Pascal <admin@serveurperso.com>

---------

Co-authored-by: Pascal <admin@serveurperso.com>
 2026-05-21 08:27:50 +02:00
wendadawen
6a257d4463
mtmd, model : merge HunyuanOCR into HunyuanVL and fix OCR vision precision (#23329) 
- HunyuanOCR shares the same HF arch and vision layout as HunyuanVL butwas split into a separate path that skipped the +0.1 bilinear sampler used by the HF reference.
- Collapse OCR into the HUNYUANVL projector + HUNYUAN_VL text arch
 2026-05-21 00:35:37 +02:00
stduhpf
3a479c9132
ui: Add max image size option (#22849) 
* webui: Add max image size option

* remove magic numbers

* support all image formats

* use const

* Move regex to match b64 images to constants

* use SETTINGS_KEYS to get max image resolution setting

* Do not touch the image if already under the size threshold
 2026-05-21 00:00:09 +02:00
Saba Fallah
a8681a0ed2
mtmd : DeepSeek-OCR image processing fixes, img_tool::resize padding refactor (#23345) 
* mtmd : deepseek-ocr fixes, improvements and refactoring

- image processing changes to achieve full parity with Pillow (reference impl)
- SAM mask casting only when flash-attn is on
- SAM refactor (build_sam() extracted so deepseek-ocr-2 can reuse it)
- llama-chat changes to fix server/WebUI issue (new media_markers_first())
- adapted test-chat-template and added test cases for deepseek-ocr
- changed regression test for deepseek-ocr to use CER+chrF scores for ground-truth comparison; removed embedding-model
- ty.toml ignore unresolved-import for tools/mtmd/tests/**

* image-text reordering fix removed

* refactor bool add_padding + pad_rounding enum into a single pad_style enum
 2026-05-20 17:37:10 +02:00
Aleksander Grygier
6ce96713de
feat: Add WAV MIME type variants and improve audio format detection (#23396) 2026-05-20 16:55:24 +02:00
Adrien Gallouët
29f1482221
app : introduce the llama unified executable (#23296) 
* app : introduce the llama unified executable

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Use serve for server

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Hide completion and bench, add help command

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Remove STATIC

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Use -impl targets instead of -lib

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Revert "Remove STATIC"

This reverts commit cc44caccb9902b34a3531633edac911e5b3d65cd.

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-05-20 13:22:22 +02:00
Aleksander Grygier
e6b4acfe86
refactor: Move text attachments up before the message content in chat completions payload (#23406) 2026-05-20 13:04:01 +02:00
Concedo
7d987af23a Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/cann.Dockerfile
#	.devops/cpu.Dockerfile
#	.devops/cuda.Dockerfile
#	.devops/intel.Dockerfile
#	.devops/llama-cli-cann.Dockerfile
#	.devops/musa.Dockerfile
#	.devops/openvino.Dockerfile
#	.devops/rocm.Dockerfile
#	.devops/s390x.Dockerfile
#	.devops/vulkan.Dockerfile
#	.github/ISSUE_TEMPLATE/011-bug-results.yml
#	.github/ISSUE_TEMPLATE/019-bug-misc.yml
#	.github/workflows/build-and-test-snapdragon.yml
#	.github/workflows/docker.yml
#	.github/workflows/server-self-hosted.yml
#	.github/workflows/ui-ci.yml
#	.pi/gg/SYSTEM.md
#	README.md
#	common/arg.cpp
#	docs/backend/SYCL.md
#	docs/backend/snapdragon/CMakeUserPresets.json
#	docs/backend/snapdragon/README.md
#	docs/speculative.md
#	examples/save-load-state/save-load-state.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/rope-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/cvt.cl
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/gated_delta_net.wgsl
#	tools/cli/README.md
#	tools/server/README.md
 2026-05-20 18:48:34 +08:00
Xuan-Son Nguyen
e2b129e1bf
mtmd: fit_params now take into account mmproj (#21489) 
* mtmd: fit_params now take into account mmproj

* rename alloc_compute_meta to reserve_compute_meta

* rm unused functions

* add ggml_backend_dev_t support

* add debug log
 2026-05-20 11:27:44 +02:00