vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-31 10:04:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
14048 commits 3 branches 253 tags 895 MiB
Commit graph

1387 commits

Author SHA1 Message Date
Concedo
6591c33667 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/release.yml
#	app/llama.cpp
#	common/download.cpp
#	docs/backend/SYCL.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	ggml/CMakeLists.txt
#	ggml/src/CMakeLists.txt
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-sycl/CMakeLists.txt
#	ggml/src/ggml-sycl/backend.hpp
#	ggml/src/ggml-sycl/common.cpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/convert.cpp
#	ggml/src/ggml-sycl/dequantize.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/dpct/helper.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/outprod.cpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	tools/server/README.md
 2026-06-18 21:00:52 +08:00
Xuan-Son Nguyen
4b4d13ae72
server: (router) add model management API (#23976) 
* wip

* server: (router) add SSE realtime updates API

* nits

* wip

* add download API

* add download api

* update docs

* add delete endpoint

* fix std::terminate

* fix crash

* fix 2

* add tests

* nits
 2026-06-17 18:04:58 +02:00
Concedo
b8b7763c76 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/openvino.Dockerfile
#	.github/workflows/build-cache.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/release.yml
#	docs/backend/OPENVINO.md
#	docs/backend/SYCL.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/mul_mv_f16_f32_l4.cl
#	ggml/src/ggml-openvino/.clang-format
#	ggml/src/ggml-openvino/CMakeLists.txt
#	ggml/src/ggml-openvino/ggml-decoder.cpp
#	ggml/src/ggml-openvino/ggml-decoder.h
#	ggml/src/ggml-openvino/ggml-openvino-extra.cpp
#	ggml/src/ggml-openvino/ggml-openvino-extra.h
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-openvino/ggml-quants.cpp
#	ggml/src/ggml-openvino/ggml-quants.h
#	ggml/src/ggml-openvino/openvino/decoder.h
#	ggml/src/ggml-openvino/openvino/frontend.h
#	ggml/src/ggml-openvino/openvino/input_model.h
#	ggml/src/ggml-openvino/openvino/node_context.h
#	ggml/src/ggml-openvino/openvino/op/cont.cpp
#	ggml/src/ggml-openvino/openvino/op/cpy.cpp
#	ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp
#	ggml/src/ggml-openvino/openvino/op/get_rows.cpp
#	ggml/src/ggml-openvino/openvino/op/glu_geglu.cpp
#	ggml/src/ggml-openvino/openvino/op/glu_swiglu.cpp
#	ggml/src/ggml-openvino/openvino/op/mulmat.cpp
#	ggml/src/ggml-openvino/openvino/op/permute.cpp
#	ggml/src/ggml-openvino/openvino/op/reshape.cpp
#	ggml/src/ggml-openvino/openvino/op/rms_norm.cpp
#	ggml/src/ggml-openvino/openvino/op/rope.cpp
#	ggml/src/ggml-openvino/openvino/op/set_rows.cpp
#	ggml/src/ggml-openvino/openvino/op/softmax.cpp
#	ggml/src/ggml-openvino/openvino/op/transpose.cpp
#	ggml/src/ggml-openvino/openvino/op/unary_silu.cpp
#	ggml/src/ggml-openvino/openvino/op/view.cpp
#	ggml/src/ggml-openvino/openvino/op_table.cpp
#	ggml/src/ggml-openvino/openvino/op_table.h
#	ggml/src/ggml-openvino/openvino/pass/mark_decompression_convert_constant_folding.h
#	ggml/src/ggml-openvino/openvino/translate_session.cpp
#	ggml/src/ggml-openvino/openvino/translate_session.h
#	ggml/src/ggml-openvino/openvino/utils.cpp
#	ggml/src/ggml-openvino/openvino/utils.h
#	ggml/src/ggml-openvino/utils.cpp
#	ggml/src/ggml-openvino/utils.h
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
 2026-06-17 16:06:00 +08:00
Max Krasnyansky
cda63856b8
common: update logging to enforce max_capacity and optimize queue resizing (#24490) 
* common: update logging to enforce max_capacity and optimize queue resizing logic

* common/log: remove queue expansion logic
 2026-06-17 09:19:11 +03:00
Concedo
382ce55fb7 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/intel.Dockerfile
#	README.md
#	docs/backend/SYCL.md
#	docs/install.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	ggml/src/ggml-sycl/binbcast.cpp
#	ggml/src/ggml-sycl/concat.cpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-sycl/element_wise.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/mmvq.hpp
#	tests/peg-parser/test-gbnf-generation.cpp
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tools/llama-bench/llama-bench.cpp
 2026-06-16 17:55:04 +08:00
Ruixiang Wang
a1824902b5
spec: add backend sampling support for eagle3 (#24655)
Some checks failed
Update Operations Documentation / update-ops-docs (push) Waiting to run
Details
Python Type-Check / python type-check (push) Has been cancelled
Details
2026-06-16 12:05:52 +03:00
Ruixiang Wang
635b65ad7a
spec: add spec metrics mean acceptance length and acceptance rate per position (#24536) 
* spec: add spec metrics mean acceptance length and acceptance per pos

* fix as suggestion

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix as suggestion

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix as suggestion

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix as suggestions

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-06-16 10:23:09 +03:00
Tarek Dakhran
7dad2f1a17
chat : fix LFM2 tool-call parsing double-escaping (#24667) 
* Add escape test cases

* chat : fix LFM2 tool-call parsing double-escaping
 2026-06-15 22:10:09 +02:00
Piotr Wilkin (ilintar)
38d546330a
chat: include full unparsed prompt in debug (#24650) 
message on parse error
 2026-06-15 17:33:54 +02:00
Pascal
581e8eca8b
chat: harden peg-native tool call parsing (#24329) 
* chat: harden peg-native tool call parsing

accept an optional leading type: function field in
build_json_tools_flat_keys so openai style tool calls parse on
templates whose serialization opens on the name field.

return a clean error and log the unparsed fragment on a final peg
parse failure instead of throwing the raw parser position and input.

keep the raw arguments string in func_args_not_string when it is not
valid json instead of aborting the prompt render.

* chat: surface peg-native parse failures

a final peg parse failure threw the raw parser position and input. log
the unparsed fragment and raise a clearer error instead, so a model
output that does not match the expected format no longer fails silently
with an empty assistant turn.

minimal change, no behavior change on successful parses.

* chat: handle openai style tool calls in peg-native

* nits

* common: scope OpenAI wrapper grammar trigger via autoparser flag

* chat: gate type:function parsing leniency on the analysis flag

Thread accept_openai_wrapper from the generator to build_json_tools_flat_keys
so the leading "type": "function" field is accepted only when openai_wrapper_trigger is set.
 2026-06-15 15:37:04 +02:00
Piotr Wilkin (ilintar)
0ae3f450f0
chat: fix an "oldie but goodie" grammar generator bug that surfaced during last changes (#24653) 
* chat: fix an "oldie but goodie" grammar generator bug that surfaced during last changes

* update erroneous case in PEG parser test
 2026-06-15 15:27:47 +02:00
Concedo
bdbcd049b1 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/cpu.Dockerfile
#	.devops/cuda.Dockerfile
#	.devops/intel.Dockerfile
#	.devops/llama-cli-cann.Dockerfile
#	.devops/musa.Dockerfile
#	.devops/openvino.Dockerfile
#	.devops/rocm.Dockerfile
#	.devops/s390x.Dockerfile
#	.devops/vulkan.Dockerfile
#	.devops/zendnn.Dockerfile
#	.github/labeler.yml
#	docs/backend/CUDA-FEDORA.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	examples/sycl/build.sh
#	examples/sycl/win-build-sycl.bat
#	ggml/src/ggml-sycl/common.cpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/cpy.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/presets.hpp
#	ggml/src/ggml-sycl/set_rows.cpp
#	ggml/src/ggml-sycl/softmax.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	grammars/README.md
#	tests/test-backend-ops.cpp
#	tests/test-chat-auto-parser.cpp
#	tests/test-chat.cpp
#	tests/test-jinja.cpp
#	tools/cli/cli.cpp
 2026-06-15 20:00:57 +08:00
Piotr Wilkin (ilintar)
a6dff71270
chat: fix whitespace problems once and for all (#24624) 
* chat: fix whitespace problems once and for all

* Purge trailing spaces from grammar generation

* Revert "Purge trailing spaces from grammar generation"

This reverts commit b0827ecb7d4767f37cefd751b3646f98d5303891.
 2026-06-15 08:27:10 +02:00
Piotr Wilkin (ilintar)
aedb2a5e9c
chat: add dedicated Cohere2MoE (North Code) parser (#24615) 
* chat: add dedicated Cohere2MoE (North Code) parser

* Some renames to make @CISC happy :>
 2026-06-14 20:17:40 +02:00
Sigbjørn Skjæret
acd79d603c
jinja : add count/d/e filter aliases (#24606) 2026-06-14 15:07:31 +02:00
Concedo
ea0351c71a Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/release.yml
#	.github/workflows/ui-build-self-hosted.yml
#	.github/workflows/ui-build.yml
#	.github/workflows/ui-publish.yml
#	.github/workflows/ui-self-hosted.yml
#	.github/workflows/ui.yml
#	.gitignore
#	README.md
#	docs/ops.md
#	docs/ops/Vulkan.csv
#	ggml/CMakeLists.txt
#	scripts/sync-ggml.last
#	scripts/sync_vendor.py
#	scripts/ui-assets.cmake
#	tests/test-jinja.cpp
#	tests/test-llama-archs.cpp
 2026-06-14 11:26:16 +08:00
Concedo
ef5a724ee4 allow building server 2026-06-14 10:31:14 +08:00
Sigbjørn Skjæret
f05cf4676a
jinja : fix negative step slice with start/stop values (#24580) 2026-06-13 18:28:40 +02:00
Sigbjørn Skjæret
341babcf73
jinja : fix split and replace with empty first arg (#24574) 
* fix split and replace with empty first arg

* fix reserve size
 2026-06-13 16:56:59 +02:00
Georgi Gerganov
d8a24ccee2
fit : wrap llama_device_memory_data (#24522) 2026-06-13 08:09:52 +03:00
Xuan-Son Nguyen
e37abd6b5f
mtmd: add batching API (#24384) 
* mtmd: add batching API

* wip

* first working version (gemma4v)

* add arg

* nits

* wire up support_batch()

* fix 0.0 output embd

* fix audio

* nits

* refactor a bit

* nits

* fix non-batching case

* fix comment
 2026-06-13 00:10:29 +02:00
Georgi Gerganov
02182fc5b9
fit : avoid including llama-ext.h in fit.h (#24506) 2026-06-12 15:57:05 +03:00
Concedo
be1750a3b8 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/cuda.Dockerfile
#	.github/workflows/build-sycl.yml
#	.github/workflows/release.yml
#	ggml/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/cvt.cl
#	ggml/src/ggml-opencl/kernels/gated_delta_net.cl
#	ggml/src/ggml-sycl/gated_delta_net.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/gated_delta_net.wgsl
#	scripts/sync-ggml.last
#	tests/test-backend-ops.cpp
#	tests/test-llama-archs.cpp
 2026-06-12 15:39:57 +08:00
Ruixiang Wang
88a39274ec
spec: add EAGLE3 speculative decoding support (#18039)
Some checks failed
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Details
Python check requirements.txt / check-requirements (push) Has been cancelled
Details
Python Type-Check / python type-check (push) Has been cancelled
Details 
* llama : enable layer input extraction

* spec: support eagle3

* eagle3: fix params bug

* eagle3: support Gemma4 eagle3 from RedHatAI

* eagle3: set sync when get features from target

Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com>

* eagle3 : fix ubatch handling in embd_layer_inp extraction and encoder

Co-authored-by: Doğaç Eldenk <dogacel@gmail.com>

* eagle3: adapt to upstream changes

* eagle3: fix rebase issues and adapt to upstream changes

* eagle3:exclude the eagle3 arch from test-llama-archs

* eagle3: fix editorconfig check failures

* eagle3: fix multi-seq issue in d2t vocab mapping

* cont : minor style / clean-up

* spec : remove `common_speculative_setup_draft_model()`

* llama : clean-up unused API

* eagle3: set d2t vocab mapping in decode graph

* cont : assert layer inputs are configured

* hparams : use n_embd_inp instead of n_embd_target_features

* eagle3: make output.weight optional and inherit from target model when needed

* haparams : generic norm-before-residual param

* llama-ext : consistent names

* cont : fix

* hparams : remove target_hidden_size

* cparams : rename output_layer_inp -> embeddings_layer_inp

* arch : reuse ATTN_NORM_2 instead of adding new hidden norm

* llama : clean-up names

* cont : add assert + comment

* Update conversion/llama.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com>
Co-authored-by: Doğaç Eldenk <dogacel@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-06-12 10:21:06 +03:00
Concedo
a860ec0b37 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/winget.yml
 2026-06-10 23:16:06 +08:00
Tarek Dakhran
d2462f8f7a
chat: fix LFM2/LFM2.5 ignoring json_schema (#24377) 
The LFM2 specialized template handler only built a grammar for tool-calling,
silently ignoring json_schema from response_format.
 2026-06-10 14:41:41 +02:00
Concedo
afc8de2c6b Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/cpu.Dockerfile
#	.devops/cuda.Dockerfile
#	.devops/intel.Dockerfile
#	.devops/musa.Dockerfile
#	.devops/openvino.Dockerfile
#	.devops/rocm.Dockerfile
#	.devops/vulkan.Dockerfile
#	.devops/zendnn.Dockerfile
#	.github/workflows/build-webgpu.yml
#	.github/workflows/release.yml
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/binary.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/concat.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/scale.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/unary.wgsl
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-mtmd-c-api.c
#	tools/cli/cli.cpp
#	tools/mtmd/CMakeLists.txt
#	tools/server/README.md
 2026-06-10 17:21:05 +08:00
ddh0
d2e22ed975
speculative : fix "ngram-map-k4v" name in logging (#24253) 
This is a non-functional change.

When using `--spec-type ngram-map-k4v`, the log messages at startup and
runtime say `ngram-map-k`. Added logic in the in the constructor of
`common_speculative_impl_ngram_map_k` to pass the correct
`COMMON_SPECULATIVE_TYPE_NGRAM_MAP_K4V` when `config.key_only` is
`false`.

After this change, the log messages use the correct name.
 2026-06-10 09:31:35 +02:00
Concedo
dd2f966f4b Merge commit 'c2b1518fd4' into concedo_experimental 
# Conflicts:
#	.devops/intel.Dockerfile
#	ggml/CMakeLists.txt
#	scripts/sync-ggml.last
#	tests/test-backend-ops.cpp
#	tests/test-llama-archs.cpp
#	tools/cli/cli.cpp
 2026-06-10 00:56:44 +08:00
Concedo
cd49b19016 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/docker.yml
#	tests/test-chat.cpp
#	tools/mtmd/clip-impl.h
#	tools/server/README.md
 2026-06-10 00:36:26 +08:00
jacekpoplawski
1e912561dd
server: log prompts to directory (#22031) 
* server: log prompts to directory

Add `--log-prompts-dir` to write each prompt to a separate text file in
the specified directory.

* Apply suggestion from @ngxson

---------

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
 2026-06-09 12:09:07 +02:00
fiesh
961e9a3e46
server : do not clear slots without unified KV cache (#24190) 
* Always export idle slots to RAM

Without this, a slot's VRAM cache may not be written to RAM.  If this
slot happens to be busy then later on, this triggers needless
preprocessing in another slot.

* cont : clean-up

---------

Co-authored-by: Christoph Weiss <weiss@wsoptics.de>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-06-09 10:45:16 +03:00
Xuan-Son Nguyen
8f83d6c271
mtmd : add video input support (#24269) 
* wip

* ok: lazy bitmap API

* remember to free lazy text

* wip

* add mtmd_helper_video

* support video input on server (base64 input)

* add MTMD_VIDEO config

* add timestamp

* update CLI

* cli: allow auto-completion for video

* add --video arg

* fix build

* update docs

* rename as suggested
 2026-06-08 14:40:12 +03:00
ddh0
9e3b928fd8
common : relax sampler name matching (#23744) 
* common : relax sampler name matching

Currently, in some cases, the alternative names for samplers (like
`top-k` and `min-p` instead of the canonical `top_k` and `min_p`) are
not always recognized by the `common_sampler_types_from_names` function
in `common/sampling.cpp`.

This PR changes the signature of this function to remove the `bool
allow_alt_names` flag, and removes all occurences of the flag from call
sites. Therefore, the function will now always match all known names.

I also changed the logic of the function to unconditionally check the
provided sampler names against both the canonical and alternative names,
and to be case-insensitive.

This fixes an issue I was seeing wherein samplers specified in the
`llama-server` UI were not recognized as valid when the alternative
names were used.

* add more alt names

* cont. fix

* cast to unsigned char for correctness

* common : unify sampler name mapping

* annotate canonical vs. alt sampler name mappings per @CISC

* Update common/sampling.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* common : auto-generate sampler name aliases per @ngxson

* use merged map for matching

* use `.merge` instead of iterating

* nit: simplify comment

* nit: use insert everywhere, not index assignment

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-06-07 22:48:11 +02:00
Aman Gupta
04eb4c446d
llama : add Gemma4 MTP (#23398) 2026-06-07 20:50:54 +08:00
Sigbjørn Skjæret
8a091c47ab
spec : fix vocab compatibility check (#24256) 2026-06-07 14:43:52 +03:00
Concedo
e40e5c660e Merge commit '5a69c97439' into concedo_experimental 
# Conflicts:
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/concat.cl
#	ggml/src/ggml-opencl/kernels/cpy.cl
#	ggml/src/ggml-opencl/kernels/get_rows.cl
#	ggml/src/ggml-opencl/kernels/mul_mv_q6_k_f32_flat.cl
#	tests/test-chat.cpp
#	tools/mtmd/CMakeLists.txt
#	tools/mtmd/clip.cpp
#	tools/mtmd/clip.h
 2026-06-07 17:46:09 +08:00
konradmb
465b1f0e75
arg: Skip mmproj download when user supplied mmproj (#24239) 2026-06-07 11:18:44 +02:00
Concedo
202aaef40b Merge commit '6effcecd0b' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-msys.yml
#	.pi/gg/SYSTEM.md
#	common/CMakeLists.txt
#	examples/speculative-simple/speculative-simple.cpp
#	ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	scripts/ui-assets.cmake
#	src/llama-model.cpp
#	tools/cli/cli.cpp
#	tools/imatrix/imatrix.cpp
#	tools/quantize/quantize.cpp
#	tools/server/README.md
#	tools/ui/tests/stories/a11y/ChatScreenForm.a11y.stories.svelte
 2026-06-07 17:10:58 +08:00
Tarek Dakhran
98d5e8ba8a
common/chat : fix LFM2/LFM2.5 reasoning round-trip and <think> leak (#24234) 
* common/chat : fix LFM2 reasoning round-trip and stray <think> leak
* Gate by reasoning format and whether the template supports <think>
 2026-06-06 22:39:21 +02:00
Tarek Dakhran
da87e9b612
common/chat : unify and fix LFM2/LFM2.5 tool parser (#24178) 2026-06-05 14:31:56 -05:00
Xuan-Son Nguyen
260862b8ca
arg: fix double mtp downloads (#24128) 2026-06-04 19:23:48 +03:00
Bartowski
e7bcf1c3a8
Move duplicated imatrix code into single common imatrix-loader.cpp (#22445) 
* Deduplicate imatrix loading code

* Add back LLAMA_TRACE, early exit on quantize missing metadata
 2026-06-04 17:45:40 +02:00
Concedo
7fb55b1e32 does not work for e4b 2026-06-04 10:44:40 +08:00
Aman Gupta
166fe29492
qwen35: use post-norm hidden state for MTP (#24025) 
* qwen35: use post-norm hidden state for MTP

* rename pre_norm to nextn

* fix step35
 2026-06-04 01:29:09 +08:00
Ryan Mangeno
e3666269f9
arg : removed unecesary mmproj download when users pass --no-mmproj (#23425) 2026-06-03 08:04:46 +03:00
Daniel Bevenius
0b7154066e
common : fix state save in common_prompt_batch_decode (#23468) 
* common : fix state save in common_prompt_batch_decode

This commit addresses a bug in common_prompt_batch_decode that affects
the session state store/restore in completion.cpp and
save-load-state.cpp.

The motivation for this is that currently the code is saving n-1 tokens
in both the session_tokens and in the KV cache. Then when loading the
session tokens, and if the prompt matches, it would replay the last
saved token (n-1) into the next position, effectively replaying the
same token in the wrong position.

The fix is to store all n tokens in session_tokens, while the memory
state only reflects n-1 processed tokens as the saving happens before
the last token is decoded in common_prompt_batch_decode.

I ran both completion.cpp and save-load-state.cpp with a transformer, a
recurrent, and a hybrid model.

Resolves: https://github.com/ggml-org/llama.cpp/issues/23400

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
 2026-06-02 15:44:15 +02:00
Xuan-Son Nguyen
60130d18f9
server: add SSE ping interval (#24013) 2026-06-02 14:14:55 +02:00
Concedo
13eaa04269 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/nix/package.nix
#	.github/workflows/server.yml
#	docs/development/HOWTO-add-model.md
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/argsort-ops.c
#	ggml/src/ggml-hexagon/htp/concat-ops.c
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-ops.h
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/hvx-utils.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/pad-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/cvt.cl
#	ggml/src/ggml-webgpu/wgsl-shaders/cpy.wgsl
#	scripts/sync_vendor.py
#	src/llama-context.cpp
#	tests/test-backend-sampler.cpp
#	tools/server/README.md
#	tools/ui/tests/stories/ChatScreenForm.a11y.stories.svelte
 2026-06-02 19:28:39 +08:00
Concedo
153214c66a Merge commit '5aba5364d9' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-apple.yml
#	.github/workflows/build-cpu.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-rpc.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/build-vulkan.yml
#	.github/workflows/build-webgpu.yml
#	.github/workflows/server.yml
#	SECURITY.md
#	docs/backend/SYCL.md
#	docs/backend/ZenDNN.md
#	docs/build.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/convert.cpp
#	ggml/src/ggml-sycl/dequantize.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/fattn-common.hpp
#	ggml/src/ggml-sycl/getrows.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/quants.hpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	tests/test-reasoning-budget.cpp
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/mtmd/CMakeLists.txt
 2026-06-02 18:30:40 +08:00