Commit graph

14007 commits

Author SHA1 Message Date
Concedo
382ce55fb7 Merge branch 'upstream' into concedo_experimental
# Conflicts:
#	.devops/intel.Dockerfile
#	README.md
#	docs/backend/SYCL.md
#	docs/install.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	ggml/src/ggml-sycl/binbcast.cpp
#	ggml/src/ggml-sycl/concat.cpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-sycl/element_wise.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/mmvq.hpp
#	tests/peg-parser/test-gbnf-generation.cpp
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tools/llama-bench/llama-bench.cpp
2026-06-16 17:55:04 +08:00
Concedo
05dbd9a195 fix exception on exit 2026-06-16 17:45:56 +08:00
Ruixiang Wang
a1824902b5
spec: add backend sampling support for eagle3 (#24655)
Some checks failed
Update Operations Documentation / update-ops-docs (push) Waiting to run
Python Type-Check / python type-check (push) Has been cancelled
2026-06-16 12:05:52 +03:00
Winston Ma
32120c10e3
vulkan: prefer host-visible memory buffers on UMA devices (#22930)
* implement UMA host-visible memory

* update based on 0cc4m's suggestion
2026-06-16 09:36:52 +02:00
Jeff Bolz
d5fb104293
vulkan: Support gated_delta_net with S_v=16 (#24581) 2026-06-16 09:26:57 +02:00
Ruixiang Wang
635b65ad7a
spec: add spec metrics mean acceptance length and acceptance rate per position (#24536)
* spec: add spec metrics mean acceptance length and acceptance per pos

* fix as suggestion

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix as suggestion

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix as suggestion

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix as suggestions

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-06-16 10:23:09 +03:00
Adrien Gallouët
e3a74b2990
bench : add --offline (#24511)
* bench : add --offline

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Add default

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-06-16 08:26:05 +02:00
Frosty40
ac79caa7ce
sycl: support reordered Q4_K/Q5_K/Q6_K MoE MUL_MAT_ID (#24452)
* sycl: support reordered Q4_K and Q5_K MoE MUL_MAT_ID

Extend reordered-weight handling to fused MoE MUL_MAT_ID for Q4_K and Q5_K expert tensors and add Q5_K reordered DMMV coverage. Unsupported 3D reorder cases now fall back instead of aborting.

* sycl: extend MoE reorder to Q6_K mul_mat_id
2026-06-16 08:35:00 +03:00
Neo Zhang
fdd109883d
[SYCL] Support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND (#24363)
* support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND

* fix conflict

* rebase, support new UT case of repeat, concat
2026-06-16 08:34:29 +03:00
Todd Malsbary
4196b477da
sycl : Make GGML_SYCL_F16=ON the default (#23996)
* Add -cl-fp32-correctly-rounded-divide-sqrt to F16=ON builds

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Make GGML_SYCL_F16=ON the default

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Leave F32 the default

F16 remains explictly set for example and Dockerfile builds.

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Revert changes to examples/sycl/build scripts

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

---------

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
2026-06-16 08:34:02 +03:00
Pascal
ad39ccaa19
vulkan: add col2im_1d op (#24425)
* vulkan: add GGML_OP_COL2IM_1D, follow-up to the CPU op

* vulkan: col2im_1d bounded gather loop instead of full-K scan with modulo

* vulkan: col2im_1d address review from @jeffbolznv

* vulkan: col2im_1d return nullptr for unsupported types, address review from @0cc4m
2026-06-16 06:34:43 +02:00
Tarek Dakhran
7dad2f1a17
chat : fix LFM2 tool-call parsing double-escaping (#24667)
* Add escape test cases

* chat : fix LFM2 tool-call parsing double-escaping
2026-06-15 22:10:09 +02:00
Concedo
197937aab0 cohere think budget 2026-06-16 01:03:24 +08:00
Wagner Bruna
b33042bc18
sd: sync with master-690-3a54597 (#2270)
* sd: sync with master-690-3a54597

* sd: drop Chroma workaround for Flash Attention

Fixed in master-686-1b702a5 .

* sd: skip setting cpu flags (soon to be removed)
2026-06-16 00:33:00 +08:00
Concedo
71801743c4 added support for cohere north mini 2026-06-16 00:24:30 +08:00
Xuan-Son Nguyen
e36a602ba3
mtmd: fix miscounting n_tokens (#24656) 2026-06-15 18:07:14 +02:00
Concedo
093067ec01 support apriel 2026-06-15 23:39:40 +08:00
Piotr Wilkin (ilintar)
38d546330a
chat: include full unparsed prompt in debug (#24650)
message on parse error
2026-06-15 17:33:54 +02:00
Julien Jerphanion
a1eb756c0b
docs: Add instructions to install llama.cpp from conda-forge (#22219)
* docs: Add instructions to install `llama.cpp` from conda-forge

Signed-off-by: Julien Jerphanion <git@jjerphan.xyz>

* Rewording of instructions

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Signed-off-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-06-15 17:12:25 +02:00
Concedo
a91357614a added lfm2 tool call template support 2026-06-15 22:55:24 +08:00
Pascal
581e8eca8b
chat: harden peg-native tool call parsing (#24329)
* chat: harden peg-native tool call parsing

accept an optional leading type: function field in
build_json_tools_flat_keys so openai style tool calls parse on
templates whose serialization opens on the name field.

return a clean error and log the unparsed fragment on a final peg
parse failure instead of throwing the raw parser position and input.

keep the raw arguments string in func_args_not_string when it is not
valid json instead of aborting the prompt render.

* chat: surface peg-native parse failures

a final peg parse failure threw the raw parser position and input. log
the unparsed fragment and raise a clearer error instead, so a model
output that does not match the expected format no longer fails silently
with an empty assistant turn.

minimal change, no behavior change on successful parses.

* chat: handle openai style tool calls in peg-native

* nits

* common: scope OpenAI wrapper grammar trigger via autoparser flag

* chat: gate type:function parsing leniency on the analysis flag

Thread accept_openai_wrapper from the generator to build_json_tools_flat_keys
so the leading "type": "function" field is accepted only when openai_wrapper_trigger is set.
2026-06-15 15:37:04 +02:00
Piotr Wilkin (ilintar)
0ae3f450f0
chat: fix an "oldie but goodie" grammar generator bug that surfaced during last changes (#24653)
* chat: fix an "oldie but goodie" grammar generator bug that surfaced during last changes

* update erroneous case in PEG parser test
2026-06-15 15:27:47 +02:00
Georgi Gerganov
e3cab403bf
mtmd : add post-decode callback (#24645)
Assisted-by: pi:llama.cpp/Qwen3.6-27B
2026-06-15 16:02:05 +03:00
Concedo
bdbcd049b1 Merge branch 'upstream' into concedo_experimental
# Conflicts:
#	.devops/cpu.Dockerfile
#	.devops/cuda.Dockerfile
#	.devops/intel.Dockerfile
#	.devops/llama-cli-cann.Dockerfile
#	.devops/musa.Dockerfile
#	.devops/openvino.Dockerfile
#	.devops/rocm.Dockerfile
#	.devops/s390x.Dockerfile
#	.devops/vulkan.Dockerfile
#	.devops/zendnn.Dockerfile
#	.github/labeler.yml
#	docs/backend/CUDA-FEDORA.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	examples/sycl/build.sh
#	examples/sycl/win-build-sycl.bat
#	ggml/src/ggml-sycl/common.cpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/cpy.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/presets.hpp
#	ggml/src/ggml-sycl/set_rows.cpp
#	ggml/src/ggml-sycl/softmax.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	grammars/README.md
#	tests/test-backend-ops.cpp
#	tests/test-chat-auto-parser.cpp
#	tests/test-chat.cpp
#	tests/test-jinja.cpp
#	tools/cli/cli.cpp
2026-06-15 20:00:57 +08:00
Concedo
be80f5dcbc auto fitting for draft models 2026-06-15 19:36:54 +08:00
Jeff Bolz
9dbc6621ae
vulkan: support more CONCAT types (#24579) 2026-06-15 13:19:21 +02:00
Andrei
6eab47181c
wasm : fix fallback symbol collision (#24639)
Some checks failed
Python Type-Check / python type-check (push) Waiting to run
Update Operations Documentation / update-ops-docs (push) Waiting to run
Python check requirements.txt / check-requirements (push) Has been cancelled
2026-06-15 10:11:59 +03:00
Katostrofik
e3bb1add8c
SYCL: use native subgroup size for K-quant DMMV (#21700) 2026-06-15 10:10:53 +03:00
someoneinjd
d8a3f523c8
sycl: fix soft_max_f32 max reduction (#24451) 2026-06-15 10:10:12 +03:00
Neo Zhang
72be44f1d2
sycl : fix reorder function; add fp32/fp16 in build script (#24578) 2026-06-15 10:08:34 +03:00
Neo Zhang
8872ab5467
sycl : enhance set_rows to support q1_0, mxfp4, nvfp4 (#24564) 2026-06-15 10:01:40 +03:00
Neo Zhang
987fbd821d
[SYCL] add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp (#24584)
* add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp

* update ops.md
2026-06-15 10:01:07 +03:00
Alexey Kopytko
c035ff4902
[SYCL]: Remove per-allocation Level Zero runtime checks (#23399)
* [SYCL] Centralize Level Zero detection in ggml_sycl_init

* use the same wording

* get back the warning

* [SYCL] Remove per-allocation getenv() for GGML_SYCL_ENABLE_LEVEL_ZERO

* bring back the comment

* move it up to make sure devices call the shots

* move the env detection early

* replace g_ggml_sycl_enable_level_zero with a direct call to .ext_oneapi_level_zero

* update the comment

* switch back to g_ggml_sycl_enable_level_zero with a sentinel

* remove the check

* Reduce the diff

* reword, move lower

* move things aroudn

* remove forward declaration if favor of a full replace

* pre-cache results of zeDeviceGetProperties

* put ggml_sycl_get_env back

* replace get_sycl_env with ggml_sycl_get_env

* add whitespace back

* Apply suggestion from @sanmai
2026-06-15 09:58:42 +03:00
Georgi Gerganov
272088b9f2
metal : add repeat bf16 (#24638) 2026-06-15 09:57:16 +03:00
Piotr Wilkin (ilintar)
a6dff71270
chat: fix whitespace problems once and for all (#24624)
* chat: fix whitespace problems once and for all

* Purge trailing spaces from grammar generation

* Revert "Purge trailing spaces from grammar generation"

This reverts commit b0827ecb7d4767f37cefd751b3646f98d5303891.
2026-06-15 08:27:10 +02:00
Pascal
2a6c391a5e
UI/svg block rendering (#24080)
* ui: add svg block visualizer based on allozaur's mermaid PR

* ui: rationalise diagram block styling and pre transforms shared by mermaid and svg

* ui: live render streaming svg blocks

* ui: also render svg authored in xml code fences

* ui: refactor svg block rendering, address review from allozaur

- Move the svg size ceiling and DOMPurify config out of sanitize-svg.ts into /constants.
- Rename the svg-diagram class to svg-block so the name no longer implies diagrams only.
- Replace the svg, xml and svg tag magic strings in the markdown pipeline with shared constants.
- Promote the data-svg-rendered marker and its sibling data attributes to constants.

* ui: render svg blocks in a shadow root for animation and live zoom

Mount each sanitized svg inside an open shadow root so author <style> and
keyframe or smil animations run while staying scoped to the host element.
Relax the sanitizer to forbid only foreignObject and script, which lets
animation, href and external resource refs through for wider compatibility.
Render the inline block and the zoom dialog from the same reactive source,
so a streaming svg keeps drawing live inside the open zoom popup.
2026-06-15 08:11:36 +02:00
leonardHONG
3686e9d643
CUDA: only support F32/F16 for GGML_OP_REPEAT (#24533) 2026-06-15 09:11:00 +03:00
Concedo
7b217c3d7c reduce default draft amount to 4 2026-06-15 11:47:45 +08:00
Concedo
de08137d51 mcp handle relative paths 2026-06-15 11:20:43 +08:00
Concedo
18665a574a adjust max ctx back to 12288, allow img2img with ref images 2026-06-15 11:15:40 +08:00
Concedo
8c54b65495 img2img allow extra images 2026-06-15 10:55:35 +08:00
Masashi Yoshimura
6e9007ae61
ggml-webgpu: improve i-quants mul_mat performance and speed up prefill (#24530)
* Improve prefill speeds for i-quants

* Fix #if defined() usage in preprocessor guards.
2026-06-14 18:15:30 -07:00
Sigbjørn Skjæret
dd4623a74f
convert : fix lora base model arch retrieval (#24621) 2026-06-15 00:55:26 +02:00
franitel
ef8268feee
fix(ui): render thinking/reasoning block content as markdown (#24611)
* fix(ui): render thinking/reasoning block content as markdown

* feat(ui): add toggle setting for thinking block markdown rendering
2026-06-14 22:56:56 +02:00
Nicolas Mowen
5f04dc7ac3
ui: Add HEIC/HEIF image support (#24137)
* Add boilerplate for file types

* Add heic-to and implement conversion

* Load heic library from CDN

* Use jpg instead of png for conversion

* Move const to constants file
2026-06-14 20:42:16 +02:00
Piotr Wilkin (ilintar)
aedb2a5e9c
chat: add dedicated Cohere2MoE (North Code) parser (#24615)
* chat: add dedicated Cohere2MoE (North Code) parser

* Some renames to make @CISC happy :>
2026-06-14 20:17:40 +02:00
Mohammad Athar
8edaca9034
docs : fix typos in CUDA-FEDORA.md and grammars/README.md (#24459) 2026-06-15 01:33:38 +08:00
Alexander Batischev
20c5266f8a
docker: specify registry to simplify Podman builds (#24607) 2026-06-15 01:27:20 +08:00
Pascal
fd5869fb62
UI/mobile keyboard and pwa popup fixes (#24610)
* ui: make mobile layout keyboard-aware via interactive-widget and dvh shell anchor

* ui: fix duplicate PWA refresh popup by scoping the storage check to non-PWA pages
2026-06-14 18:35:00 +02:00
Amos Wong
1fd6dfe9f3
ui : fix ui clipping in mobile due to incorrect height setup (#24605) 2026-06-14 16:15:51 +02:00