Concedo
104b41091b
refactor drafting
2026-06-12 16:48:00 +08:00
Concedo
be1750a3b8
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cuda.Dockerfile
# .github/workflows/build-sycl.yml
# .github/workflows/release.yml
# ggml/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-opencl/kernels/gated_delta_net.cl
# ggml/src/ggml-sycl/gated_delta_net.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/gated_delta_net.wgsl
# scripts/sync-ggml.last
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
2026-06-12 15:39:57 +08:00
Ruixiang Wang
88a39274ec
spec: add EAGLE3 speculative decoding support ( #18039 )
...
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
* llama : enable layer input extraction
* spec: support eagle3
* eagle3: fix params bug
* eagle3: support Gemma4 eagle3 from RedHatAI
* eagle3: set sync when get features from target
Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com>
* eagle3 : fix ubatch handling in embd_layer_inp extraction and encoder
Co-authored-by: Doğaç Eldenk <dogacel@gmail.com>
* eagle3: adapt to upstream changes
* eagle3: fix rebase issues and adapt to upstream changes
* eagle3:exclude the eagle3 arch from test-llama-archs
* eagle3: fix editorconfig check failures
* eagle3: fix multi-seq issue in d2t vocab mapping
* cont : minor style / clean-up
* spec : remove `common_speculative_setup_draft_model()`
* llama : clean-up unused API
* eagle3: set d2t vocab mapping in decode graph
* cont : assert layer inputs are configured
* hparams : use n_embd_inp instead of n_embd_target_features
* eagle3: make output.weight optional and inherit from target model when needed
* haparams : generic norm-before-residual param
* llama-ext : consistent names
* cont : fix
* hparams : remove target_hidden_size
* cparams : rename output_layer_inp -> embeddings_layer_inp
* arch : reuse ATTN_NORM_2 instead of adding new hidden norm
* llama : clean-up names
* cont : add assert + comment
* Update conversion/llama.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com>
Co-authored-by: Doğaç Eldenk <dogacel@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2026-06-12 10:21:06 +03:00
Concedo
3424543568
skip dot2 vulkan shaders for noext build
2026-06-12 15:20:41 +08:00
ZihaoMu
85f99dca8b
ggml: support concat for scalar types at cuda backend ( #24011 )
...
* cuda: support concat for scalar types
* Update concat.cu
* fix metal ci issue
2026-06-12 09:32:44 +03:00
Neo Zhang
099ea76fb4
[SYCL] Fix CI build & release for SYCL backend ( #24387 )
...
* restore SYCL build and release, remove github cache
* modify for test only
* verify the ccache is used
* remove debug code change
* rm duplicate action, update key in ccache
* add action ccache-clear after building in both ubuntu and windows
* set %NUMBER_OF_PROCESSORS% in widnows build
2026-06-12 09:30:24 +03:00
shaofeiqi
ba1df050f3
opencl: add q5_0/q5_1 gemm and gemv kernels for Adreno ( #24319 )
...
* opencl: add q5_0 adreno support
* opencl: add q5_1 adreno support
* opencl: cosmetic fix
---------
Co-authored-by: Li He <lih@qti.qualcomm.com>
2026-06-11 21:43:09 -07:00
wencan
1593d5684d
docker : support specifying the GCC version for CUDA ( #24447 )
2026-06-11 23:12:09 +02:00
Jeff Bolz
4c6595503f
vulkan: ifdef eMesaHoneykrisp (build fix) ( #24479 )
...
Fixes build/CI after #24306 .
2026-06-11 13:22:17 -05:00
Georgi Gerganov
263cc04a54
sync : ggml
2026-06-11 19:34:19 +03:00
Georgi Gerganov
17e59d6209
ggml : bump version to 0.15.0 (ggml/1539)
2026-06-11 19:34:19 +03:00
Concedo
60d0e70d76
RNN lifeboat implementation
2026-06-11 23:40:43 +08:00
Concedo
7d1147bc35
smartcache improvements
2026-06-11 22:48:37 +08:00
Concedo
79331fd382
fixed mtp for qwen
2026-06-11 22:04:31 +08:00
Winston Ma
fdc3db9b65
vulkan: add fast path for contiguous buffer transfers ( #23973 )
Python Type-Check / python type-check (push) Waiting to run
2026-06-11 15:46:25 +02:00
Kevin Liu
1af154a76f
vulkan: use medium matmul tile on Asahi Linux ( #24306 )
...
* vulkan: use medium matmul tile on Asahi Linux
* vulkan: switch Apple detection to Honeykrisp driver id
2026-06-11 15:43:04 +02:00
Xuan-Son Nguyen
18ef86ecec
server: skip unused log lines on router mode ( #24463 )
2026-06-11 11:36:35 +02:00
o7si
1bfbdb134e
vocab : adopt leading TemplateProcessing special token as BOS ( #24428 )
2026-06-11 10:37:23 +03:00
o7si
68f30663cf
vocab : refactor normalizer flags into options struct, add strip_accents ( #24371 )
...
* vocab : refactor normalizer flags into options struct, add strip_accents
* Update src/llama-vocab.h
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update src/llama-vocab.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2026-06-11 10:36:50 +03:00
Concedo
40084bb1f8
mtp drafting added (+1 squashed commits)
...
Squashed commits:
[b0fa1a6de] wip 1
2026-06-11 15:25:49 +08:00
Aldehir Rojas
db94854ff5
server : skip checkpoints beyond pos_next ( #24411 )
...
* server : skip checkpoints beyond pos_next
* cont : update comment + TODO + ref
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-06-11 10:18:12 +03:00
Concedo
10e4b6d5e3
support gemma assistant as a draft model
2026-06-11 12:16:43 +08:00
Adrien Gallouët
ac4cddeb0d
vendor : update LibreSSL to 4.3.2 ( #24397 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-06-10 22:28:03 +02:00
Gaurav Garg
e95dae18d6
Remove padding and multiple D2D copies for MTP ( #24086 )
...
* Make ggml_gated_delta_net take only the initial recurrent state (D, 1, n_seqs) and passes the snapshot count K as an op parameter instead of inferring it from state->ne[1].
Remove the padding hack and copy all emitted snapshots into the recurrent cache with a single strided ggml_cpy
* Make GDN changes in all backends. Address review comments.
* Fix CI build errors
2026-06-10 23:21:16 +05:30
Concedo
a860ec0b37
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/winget.yml
2026-06-10 23:16:06 +08:00
Tarek Dakhran
d2462f8f7a
chat: fix LFM2/LFM2.5 ignoring json_schema ( #24377 )
...
The LFM2 specialized template handler only built a grammar for tool-calling,
silently ignoring json_schema from response_format.
2026-06-10 14:41:41 +02:00
Oliver Simons
fb83cc9a07
CUDA: Fix ssm_scan_f32 data-races ( #24360 )
...
* Add missing syncthreads before resuing cub_temp_storage
__syncthreads() is required before being allowed to resue TempStorage
smem:
https://nvidia.github.io/cccl/unstable/cub/api/classcub_1_1BlockLoad.html#_CPPv4I0EN3cub9BlockLoad4LoadEv20RandomAccessIteratorRA14ItemsPerThread_1Ti
* Add one more missing __syncthreads
Could also double-buffer, but alternative is to simply ensure all
threads have read smem* before writing to it again in the next loop
iteration
* Remove unused smem from ssm_scan_f32
2026-06-10 14:27:08 +02:00
Concedo
e8ead6efaa
increase ctx and gen amt defaults
2026-06-10 18:00:26 +08:00
Concedo
afc8de2c6b
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .github/workflows/build-webgpu.yml
# .github/workflows/release.yml
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/binary.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/concat.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/scale.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/unary.wgsl
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-mtmd-c-api.c
# tools/cli/cli.cpp
# tools/mtmd/CMakeLists.txt
# tools/server/README.md
2026-06-10 17:21:05 +08:00
Sigbjørn Skjæret
039e20a2db
ci : bump komac version ( #24396 )
2026-06-10 09:45:20 +02:00
ddh0
d2e22ed975
speculative : fix "ngram-map-k4v" name in logging ( #24253 )
...
This is a non-functional change.
When using `--spec-type ngram-map-k4v`, the log messages at startup and
runtime say `ngram-map-k`. Added logic in the in the constructor of
`common_speculative_impl_ngram_map_k` to pass the correct
`COMMON_SPECULATIVE_TYPE_NGRAM_MAP_K4V` when `config.key_only` is
`false`.
After this change, the log messages use the correct name.
2026-06-10 09:31:35 +02:00
Concedo
258ca8fd90
fix build
2026-06-10 10:33:06 +08:00
Concedo
8cf8469bb8
reasoningeffort param
2026-06-10 10:30:57 +08:00
Rémy Mathieu
76da2450a4
webui: implement pinned conversations support ( #21387 )
...
* webui: implement pinned conversations support
* webui: linter/prettier pass
* Fix the unused handleMobileSidebarItemClick from the component.
* the search should find pinned conversations as well
Co-authored-by: Pascal <admin@serveurperso.com>
---------
Co-authored-by: Pascal <admin@serveurperso.com>
2026-06-09 21:33:22 +02:00
Aarnav Pai
d73cd07674
graph: Fix granite speech model inference by applying embedding scale when deepstack is not used ( #24357 )
...
* llama-graph : apply embedding scale when deepstack is not used
* nits: remove non-existant hunyuan-vl from the tests
* apply suggestion from @gabe-l-hart
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-06-09 19:46:27 +02:00
Concedo
e839598185
think effort box
2026-06-10 01:32:42 +08:00
Concedo
dd2f966f4b
Merge commit ' c2b1518fd4' into concedo_experimental
...
# Conflicts:
# .devops/intel.Dockerfile
# ggml/CMakeLists.txt
# scripts/sync-ggml.last
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tools/cli/cli.cpp
2026-06-10 00:56:44 +08:00
Concedo
6bae15da71
fix build, remove clip quantize (+1 squashed commits)
...
Squashed commits:
[09ffa906b] fix build, remove clip quantize
2026-06-10 00:42:34 +08:00
Sigbjørn Skjæret
e25a32e98c
ci : fix windows release ( #24369 )
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
2026-06-09 19:42:23 +03:00
Concedo
cd49b19016
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/docker.yml
# tests/test-chat.cpp
# tools/mtmd/clip-impl.h
# tools/server/README.md
2026-06-10 00:36:26 +08:00
Pascal
483609509d
ui: add opt-in run_javascript frontend tool ( #24244 )
...
* ui: add opt-in run_javascript frontend tool
Expose a run_javascript tool to the model, executed entirely in the
browser through the existing agentic loop. Code runs in a Web Worker
inside a sandboxed iframe with an opaque origin, isolated from the
WebUI and its API. Console output, errors and the return value are
fed back as the tool result. The parent enforces a hard timeout by
removing the iframe, which terminates the worker.
Disabled by default, toggle in Settings > Developer.
* ui: address review feedback from allozaur
Use the JsonSchemaType enum for the tool definition parameter types
instead of raw string literals, extending it with STRING and NUMBER.
Move the worker shim and the iframe harness html into their own files
so the service no longer carries inline source blobs.
Replace the remaining magic strings with constants: SANDBOX_EMPTY_OUTPUT
and SANDBOX_TRUNCATION_NOTICE, and reuse NEWLINE_SEPARATOR for joins.
* ui: move sandbox worker shim to a raw imported file
Replace the inline worker template string with a real sandbox-worker.js
imported as raw text, and build the iframe harness from it in
sandbox-harness.ts. The raw worker ships as a string, not a module, so
it is excluded from eslint and the typecheck program.
2026-06-09 18:02:31 +02:00
Concedo
fe13a2989d
migration to mtmd complete
2026-06-09 23:09:45 +08:00
Concedo
0740ce70b7
revert incorrect token count for mtmd
2026-06-09 22:46:17 +08:00
Concedo
ac8e77d82e
Revert "cleanup round 2"
...
This reverts commit c7e39f9c97
2026-06-09 22:33:00 +08:00
Concedo
da0ba5cefd
Revert "cleanup round 3"
...
This reverts commit 0ead9907f0
2026-06-09 22:32:42 +08:00
Saba Fallah
49f3542190
mtmd: build_vit batching ( #24352 )
2026-06-09 16:32:08 +02:00
Concedo
0ead9907f0
cleanup round 3
2026-06-09 21:52:46 +08:00
Concedo
c7e39f9c97
cleanup round 2
2026-06-09 21:49:20 +08:00
Concedo
f5acccad63
cleanup round 1
2026-06-09 21:13:48 +08:00
Jeff Bolz
d6d0ce8215
vulkan: reduce iq1 shared memory usage for mul_mm ( #24287 )
2026-06-09 13:27:38 +02:00
