Concedo
73cc7d9287
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/s390x.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .dockerignore
# .pi/gg/SYSTEM.md
# ggml/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/sync-ggml.last
# scripts/sync_vendor.py
# tools/cli/README.md
# tools/cli/cli.cpp
# tools/mtmd/clip.cpp
# tools/server/README.md
2026-06-20 17:16:19 +08:00
Concedo
84b8856295
Merge commit ' 32eddaf2ea' into concedo_experimental
...
# Conflicts:
# docs/multimodal.md
# docs/preset.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-hexagon/htp/hmx-queue.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/ui-assets.cmake
# tools/export-lora/README.md
# tools/server/CMakeLists.txt
2026-06-20 11:18:24 +08:00
Xuan-Son Nguyen
2b686a9120
server: refactor child --> router communication ( #24821 )
...
* server: refactor child --> router communication
* fix wakeup case
* add docs
* improve update_status()
* nits
2026-06-20 01:02:26 +02:00
Xuan-Son Nguyen
e475fa2b5f
mtmd, arg: fix utf8 handling on windows ( #24779 )
...
* mtmd, arg: fix utf8 handling on windows
* also fix ggml_fopen
* fix build fail
* also fix CLI
2026-06-19 22:28:38 +02:00
Mikolaj Kucharski
fabde3bf51
arg: Add comment line support to --api-key-file ( #23168 )
2026-06-19 17:33:54 +02:00
Xuan-Son Nguyen
8c2d6f6475
server: add --agent arg, remove redundant webui naming compat ( #24801 )
...
* server: add --agent arg, remove redundant webui naming compat
* corrent env
* fix the test
* llama-gen-docs
* nits: wordings
2026-06-19 16:06:13 +02:00
Xuan-Son Nguyen
552258c535
server: (router) rework -hf preset repo ( #24739 )
...
* server: temporary remove HF remote preset
* rework remove preset.ini support
* rm unused get_remote_preset_whitelist()
* print warning
* add docs
* rm stray file
2026-06-18 12:45:23 +02:00
Concedo
ea0351c71a
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/release.yml
# .github/workflows/ui-build-self-hosted.yml
# .github/workflows/ui-build.yml
# .github/workflows/ui-publish.yml
# .github/workflows/ui-self-hosted.yml
# .github/workflows/ui.yml
# .gitignore
# README.md
# docs/ops.md
# docs/ops/Vulkan.csv
# ggml/CMakeLists.txt
# scripts/sync-ggml.last
# scripts/sync_vendor.py
# scripts/ui-assets.cmake
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
2026-06-14 11:26:16 +08:00
Xuan-Son Nguyen
e37abd6b5f
mtmd: add batching API ( #24384 )
...
* mtmd: add batching API
* wip
* first working version (gemma4v)
* add arg
* nits
* wire up support_batch()
* fix 0.0 output embd
* fix audio
* nits
* refactor a bit
* nits
* fix non-batching case
* fix comment
2026-06-13 00:10:29 +02:00
Concedo
afc8de2c6b
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .github/workflows/build-webgpu.yml
# .github/workflows/release.yml
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/binary.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/concat.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/scale.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/unary.wgsl
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-mtmd-c-api.c
# tools/cli/cli.cpp
# tools/mtmd/CMakeLists.txt
# tools/server/README.md
2026-06-10 17:21:05 +08:00
Concedo
dd2f966f4b
Merge commit ' c2b1518fd4' into concedo_experimental
...
# Conflicts:
# .devops/intel.Dockerfile
# ggml/CMakeLists.txt
# scripts/sync-ggml.last
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tools/cli/cli.cpp
2026-06-10 00:56:44 +08:00
jacekpoplawski
1e912561dd
server: log prompts to directory ( #22031 )
...
* server: log prompts to directory
Add `--log-prompts-dir` to write each prompt to a separate text file in
the specified directory.
* Apply suggestion from @ngxson
---------
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
2026-06-09 12:09:07 +02:00
fiesh
961e9a3e46
server : do not clear slots without unified KV cache ( #24190 )
...
* Always export idle slots to RAM
Without this, a slot's VRAM cache may not be written to RAM. If this
slot happens to be busy then later on, this triggers needless
preprocessing in another slot.
* cont : clean-up
---------
Co-authored-by: Christoph Weiss <weiss@wsoptics.de>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-06-09 10:45:16 +03:00
Xuan-Son Nguyen
8f83d6c271
mtmd : add video input support ( #24269 )
...
* wip
* ok: lazy bitmap API
* remember to free lazy text
* wip
* add mtmd_helper_video
* support video input on server (base64 input)
* add MTMD_VIDEO config
* add timestamp
* update CLI
* cli: allow auto-completion for video
* add --video arg
* fix build
* update docs
* rename as suggested
2026-06-08 14:40:12 +03:00
ddh0
9e3b928fd8
common : relax sampler name matching ( #23744 )
...
* common : relax sampler name matching
Currently, in some cases, the alternative names for samplers (like
`top-k` and `min-p` instead of the canonical `top_k` and `min_p`) are
not always recognized by the `common_sampler_types_from_names` function
in `common/sampling.cpp`.
This PR changes the signature of this function to remove the `bool
allow_alt_names` flag, and removes all occurences of the flag from call
sites. Therefore, the function will now always match all known names.
I also changed the logic of the function to unconditionally check the
provided sampler names against both the canonical and alternative names,
and to be case-insensitive.
This fixes an issue I was seeing wherein samplers specified in the
`llama-server` UI were not recognized as valid when the alternative
names were used.
* add more alt names
* cont. fix
* cast to unsigned char for correctness
* common : unify sampler name mapping
* annotate canonical vs. alt sampler name mappings per @CISC
* Update common/sampling.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* common : auto-generate sampler name aliases per @ngxson
* use merged map for matching
* use `.merge` instead of iterating
* nit: simplify comment
* nit: use insert everywhere, not index assignment
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2026-06-07 22:48:11 +02:00
konradmb
465b1f0e75
arg: Skip mmproj download when user supplied mmproj ( #24239 )
2026-06-07 11:18:44 +02:00
Concedo
202aaef40b
Merge commit ' 6effcecd0b' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-msys.yml
# .pi/gg/SYSTEM.md
# common/CMakeLists.txt
# examples/speculative-simple/speculative-simple.cpp
# ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# scripts/ui-assets.cmake
# src/llama-model.cpp
# tools/cli/cli.cpp
# tools/imatrix/imatrix.cpp
# tools/quantize/quantize.cpp
# tools/server/README.md
# tools/ui/tests/stories/a11y/ChatScreenForm.a11y.stories.svelte
2026-06-07 17:10:58 +08:00
Xuan-Son Nguyen
260862b8ca
arg: fix double mtp downloads ( #24128 )
2026-06-04 19:23:48 +03:00
Concedo
7fb55b1e32
does not work for e4b
2026-06-04 10:44:40 +08:00
Ryan Mangeno
e3666269f9
arg : removed unecesary mmproj download when users pass --no-mmproj ( #23425 )
2026-06-03 08:04:46 +03:00
Xuan-Son Nguyen
60130d18f9
server: add SSE ping interval ( #24013 )
2026-06-02 14:14:55 +02:00
Concedo
13eaa04269
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/nix/package.nix
# .github/workflows/server.yml
# docs/development/HOWTO-add-model.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-ops.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/hvx-utils.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/pad-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-webgpu/wgsl-shaders/cpy.wgsl
# scripts/sync_vendor.py
# src/llama-context.cpp
# tests/test-backend-sampler.cpp
# tools/server/README.md
# tools/ui/tests/stories/ChatScreenForm.a11y.stories.svelte
2026-06-02 19:28:39 +08:00
Aleksander Grygier
f8e67fc583
ui: Add Thinking mode toggle with reasoning effort levels + improvements for Chat Form Add Action UI ( #23434 )
...
* feat: Add "Thinking" toggle and status icon + redesign Chat Form Actions Add panel
* test: Update test reference
* fix: Icon
* fix: E2E test command
* fix: wait for greeting h1 to be visible in e2e test
* fix: remove duplicate PDF option in attachment dropdown
* fix: use label-based group toggle to avoid stale references
* refactor: inline MCP server and tool toggles in mobile sheet
* fix: serve correct build directory in e2e playwright config
* feat: add reasoning effort levels selector in model dropdown
* feat: Reasoning effort
* refactor: Make server origin configurable via environment variable
* feat: Add chat template thinking detector utility
* feat: Add thinking support detection to models store
* refactor: Update model selector components with thinking detection and message-specific indicators
* feat: Update chat form components for model selection and thinking support
* feat: Improve Reasoning controls UI
* refactor: Apply suggestions from code review
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
* fix: Model tags
* refactor: Cleanup
* refactor: Remove unneeded components
* refactor: Cleanup
2026-06-02 10:23:19 +02:00
Georgi Gerganov
5dcb711666
speculative : fix n_outputs_max and remove draft-simple auto-enable ( #23988 )
...
* speculative : add common_speculative_n_max helper function
Extract the speculative max-draft-size logic from server_n_outputs_max
into a reusable common_speculative_n_max() function in common/speculative.
Assisted-by: llama.cpp:local pi
* cont : draft context always has n_parallel outputs
* llama : log n_outputs_max
* speculative : remove draft-simple auto-enable
* ci : enable server tests on PRs
2026-06-01 22:26:58 +03:00
Concedo
e541554d70
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/ui-build.yml
# .github/workflows/ui-publish.yml
# .github/workflows/ui-self-hosted.yml
# CMakeLists.txt
# app/CMakeLists.txt
# app/llama.cpp
# common/arg.cpp
# ggml/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/ggml-opencl.cpp
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/sync-ggml.last
# src/CMakeLists.txt
# tests/test-llama-archs.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/mtmd/CMakeLists.txt
# tools/mtmd/tests/test-deepseek-ocr.py
# tools/server/README.md
2026-05-30 01:54:53 +08:00
Concedo
baacd640c0
Merge commit ' dd1557907a' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-vulkan.yml
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# tests/test-backend-ops.cpp
# tests/test-chat-template.cpp
# tests/test-chat.cpp
# tests/test-llama-archs.cpp
# tools/perplexity/perplexity.cpp
# tools/server/README.md
2026-05-30 01:40:09 +08:00
Xuan-Son Nguyen
06d26dfdff
download: add option to skip_download ( #23059 )
...
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Waiting to run
Python check requirements.txt / check-requirements (push) Waiting to run
Python Type-Check / python type-check (push) Waiting to run
* download: add option to skip_download
* fix
* fix 2
* if file doesn't exist, respect skip_download flag
2026-05-29 16:30:55 +02:00
Adrien Gallouët
98e480a32e
app : move licences to llama-app ( #23824 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-05-29 07:46:11 +02:00
Mikolaj Kucharski
7fb1e70b59
arg: Add LLAMA_ARG_API_KEY_FILE environment variable for --api-key-file ( #23167 )
2026-05-28 16:25:40 +02:00
Concedo
ca942eb172
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/actions/windows-setup-cuda/action.yml
# .github/workflows/build-3rd-party.yml
# .github/workflows/build-android.yml
# .github/workflows/build-apple.yml
# .github/workflows/build-cann.yml
# .github/workflows/build-cuda-ubuntu.yml
# .github/workflows/build-ibm.yml
# .github/workflows/build-msys.yml
# .github/workflows/build-opencl.yml
# .github/workflows/build-openvino.yml
# .github/workflows/build-riscv.yml
# .github/workflows/build-rpc.yml
# .github/workflows/build-sanitize.yml
# .github/workflows/build-self-hosted.yml
# .github/workflows/build-sycl.yml
# .github/workflows/build-vulkan.yml
# .github/workflows/build-webgpu.yml
# .github/workflows/hip-quality-check.yml
# .github/workflows/release.yml
# .github/workflows/server-sanitize.yml
# .github/workflows/server-self-hosted.yml
# .github/workflows/server.yml
# .github/workflows/ui-self-hosted.yml
# .github/workflows/ui.yml
# CONTRIBUTING.md
# ci/run.sh
# docs/autoparser.md
# docs/multimodal/granitevision.md
# examples/model-conversion/README.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-vulkan/CMakeLists.txt
# ggml/src/ggml-vulkan/ggml-vulkan.cpp
# ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/cpy.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id_gather.wgsl
# ggml/src/ggml-zendnn/ggml-zendnn.cpp
# pyproject.toml
# scripts/snapdragon/adb/run-completion.sh
# scripts/snapdragon/adb/run-tool.sh
# scripts/sync_vendor.py
# tests/gguf-model-data.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
2026-05-28 15:28:29 +08:00
Georgi Gerganov
6b4e4bd582
common : fix env names to all have LLAMA_ARG_ prefix ( #23778 )
2026-05-27 14:52:47 +03:00
Concedo
9204f78926
Merge commit ' ae251b5ff2' into concedo_experimental
...
# Conflicts:
# .github/actions/linux-setup-spacemit/action.yml
# .github/actions/unarchive-tar/action.yml
# .github/workflows/build-android.yml
# .github/workflows/build-cmake-pkg.yml
# .github/workflows/build-cross.yml
# .github/workflows/build-self-hosted.yml
# .github/workflows/build.yml
# .github/workflows/check-vendor.yml
# .github/workflows/code-style.yml
# .github/workflows/editorconfig.yml
# .github/workflows/pre-tokenizer-hashes.yml
# .github/workflows/python-check-requirements.yml
# .github/workflows/python-lint.yml
# .github/workflows/python-type-check.yml
# .github/workflows/server-self-hosted.yml
# .github/workflows/ui-build.yml
# .github/workflows/ui.yml
# .github/workflows/update-ops-docs.yml
# ci/run.sh
# docs/build-riscv64-spacemit.md
# examples/convert_legacy_llama.py
# ggml/cmake/ggml-config.cmake.in
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# scripts/sync_vendor.py
# tests/test-chat-auto-parser.cpp
# tests/test-chat.cpp
# tests/test-gguf.cpp
# tools/cli/README.md
# tools/perplexity/perplexity.cpp
# tools/server/README.md
2026-05-26 22:01:57 +08:00
jacekpoplawski
e2ef8fe42c
server: fix checkpoints creation ( #22929 )
...
* common : add common_chat_split_by_role
* cont : fix spans to reach end of message
* server: fix checkpoints creation
- extract message_spans from chat templates
- find the prompt token position before the latest user message
- split prompt batching at that position
- create a context checkpoint before the latest user input
- avoid periodic mid-prompt checkpoints when that position is known
- handle multimodal prompts when mapping text/template positions to server prompt tokens
- add --checkpoint-min-step to control minimum spacing between checkpoints
* cont : clean-up
* Support autoparser detection for message barriers
* server: fix message span delimiter and update docs
---------
Co-authored-by: Alde Rojas <hello@alde.dev>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com>
2026-05-25 08:56:18 +03:00
Concedo
718dc159b6
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# CMakeLists.txt
# docs/speculative.md
# ggml/src/ggml-cuda/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# ggml/src/ggml-opencl/ggml-opencl.cpp
# scripts/snapdragon/adb/run-bench.sh
# scripts/snapdragon/adb/run-cli.sh
# scripts/snapdragon/adb/run-completion.sh
# scripts/snapdragon/adb/run-mtmd.sh
# scripts/snapdragon/windows/run-bench.ps1
# scripts/snapdragon/windows/run-cli.ps1
# scripts/snapdragon/windows/run-completion.ps1
# scripts/snapdragon/windows/run-mtmd.ps1
# src/llama-vocab.cpp
# tests/test-backend-ops.cpp
# tools/batched-bench/CMakeLists.txt
# tools/batched-bench/batched-bench.cpp
# tools/cli/CMakeLists.txt
# tools/cli/README.md
# tools/cli/cli.cpp
# tools/completion/CMakeLists.txt
# tools/completion/README.md
# tools/llama-bench/CMakeLists.txt
# tools/llama-bench/llama-bench.cpp
# tools/mtmd/CMakeLists.txt
# tools/mtmd/tests/test-deepseek-ocr.py
# tools/mtmd/tests/tests-requirements.txt
# tools/perplexity/CMakeLists.txt
# tools/perplexity/perplexity.cpp
# tools/quantize/CMakeLists.txt
# tools/server/CMakeLists.txt
# tools/server/README.md
# ty.toml
2026-05-21 23:47:21 +08:00
Gaurav Garg
ad27757261
Move to backend sampling for MTP draft path ( #23287 )
...
* Move to backend sampling for MTP draft path
Run top_k(10) on the draft backend. D2H transfers happen only for the top 10 logits
Make backend sampling more robust and fallback to CPU on failure cases, such as with "-sm tensor" or when a backend doesn't support TOP_K.
* Allow sampler chains to be partially offloaded to backend
* Add --spec-draft-backend-sampling argument. Enabled by default.
2026-05-20 22:34:45 +05:30
Concedo
7d987af23a
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/llama-cli-cann.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/s390x.Dockerfile
# .devops/vulkan.Dockerfile
# .github/ISSUE_TEMPLATE/011-bug-results.yml
# .github/ISSUE_TEMPLATE/019-bug-misc.yml
# .github/workflows/build-and-test-snapdragon.yml
# .github/workflows/docker.yml
# .github/workflows/server-self-hosted.yml
# .github/workflows/ui-ci.yml
# .pi/gg/SYSTEM.md
# README.md
# common/arg.cpp
# docs/backend/SYCL.md
# docs/backend/snapdragon/CMakeUserPresets.json
# docs/backend/snapdragon/README.md
# docs/speculative.md
# examples/save-load-state/save-load-state.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/gated_delta_net.wgsl
# tools/cli/README.md
# tools/server/README.md
2026-05-20 18:48:34 +08:00
Johannes Gäßler
57cb35c886
common: fix --help for --verbosity ( #23278 )
2026-05-19 21:34:04 +02:00
Georgi Gerganov
d14ce3dab4
llama : MTP clean-up ( #23269 )
...
* llama : disable equal splits for recurrent memory with partial rollback
* spec : re-enable p-min with MTP drafts
* spec : re-enable ngram spec in combination with RS rollback
* spec : fix ngram-map-* params
* spec : fix acceptance logic in combined ngram + draft configs
* graph : fix reuse for combined `token` + `embd` batches
* spec : log parameters for each speculative implementation
- add LOG_INF in each constructor with implementation type and parameters
- extract device string logic into common_speculative_get_devices_str()
- move 'adding speculative implementation' log from init into constructors
Assisted-by: llama.cpp:local pi
* spec : extend --spec-default with ngram-map-k4v
Assisted-by: llama.cpp:local pi
* minor : fix n_embd log
* args : update draft.n_max == 3 + regen docs
* spec : relax ngram-mod rejection thold to 0.25 @ 5 low
* logs : improve
* docs : update speculative decoding CLI argument documentation
- Add missing draft model CPU scheduling and tensor override parameters
- Update --spec-type to include all available types (excluding draft-eagle3 WIP)
- Fix default values to match implementation (n_max=3, n_min=0, p_min=0.0)
- Remove deprecated options (spec-draft-ctx-size, spec-draft-replace)
- Add environment variables for new parameters
Assisted-by: llama.cpp:local pi
* arg : step-back on adding k4v to the default spec config
* cont : fix name
2026-05-19 15:32:58 +03:00
Adrien Gallouët
45b455e66f
common : remove hf cache migration ( #23266 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-05-18 17:11:47 +02:00
Concedo
fecf2dc3fa
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/server-self-hosted.yml
# CMakeLists.txt
# CODEOWNERS
# ci/run.sh
# cmake/llama-config.cmake.in
# common/chat.cpp
# examples/sycl/start-svr.sh
# examples/sycl/test.sh
# examples/sycl/win-start-svr.bat
# examples/sycl/win-test.bat
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# ggml/src/ggml-vulkan/CMakeLists.txt
# scripts/wc2wt.sh
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
2026-05-18 21:27:23 +08:00
Rares Vernica
1a68ec9378
server : honor --embd-normalize CLI arg ( #23125 )
...
The --embd-normalize flag was registered only for the embedding and debug
examples, so llama-server rejected it and the /embedding handler used a
hard-coded default of 2 (L2). Add LLAMA_EXAMPLE_SERVER to the flag's
example set and read params.embd_normalize as the handler's default. The
per-request "embd_normalize" body field continues to override.
2026-05-17 09:39:04 +03:00
Concedo
1e828ccabf
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# common/common.cpp
# ggml/CMakeLists.txt
# scripts/sync-ggml.last
# scripts/sync_vendor.py
# src/llama-context.cpp
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
2026-05-17 11:26:18 +08:00
Concedo
9203b6a051
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# .github/workflows/build-self-hosted.yml
# .github/workflows/release.yml
# .github/workflows/server-sanitize.yml
# .github/workflows/server-self-hosted.yml
# .github/workflows/server.yml
# .github/workflows/ui-build.yml
# .github/workflows/ui-ci.yml
# .github/workflows/ui-publish.yml
# .gitignore
# CMakeLists.txt
# CODEOWNERS
# scripts/ui-download.cmake
# scripts/xxd.cmake
# tests/test-backend-ops.cpp
# tests/test-reasoning-budget.cpp
# tools/CMakeLists.txt
# tools/server/CMakeLists.txt
# tools/server/README.md
2026-05-16 22:56:33 +08:00
Aman Gupta
255582687b
llama + spec: MTP Support ( #22673 )
...
* spec: support MTP
* fix batch size
* rename files
* cont : simplify (#7 )
* MTP: clean-up (#9 )
* MTP: clean-up
* review: use llama_context_type instead of llama_graph_type
* review: remove llama_model_has_mtp
* review: fix convert issues
* convert: fix pycheck
* review: formatting
* use `mtp-` for identifying mtp models
* convert: fix mtp conversion
* mtp -> draft-mtp
* remove unused llama_arch
* add need_embd in speculative
* llama: allow partial seq_rm for GDN models for speculative decoding
Currently speculative checkpoint needs to restart from a checkpoint
after some draft tokens are not accepted, this leads to some wastage in
running the target again. This PR adds the ability to rollback upto
`draft_max` by storing the GDN intermediates.
* fix pending state
* vulkan: add GDN partial rollback
* meta: extend check to axis 1
* metal: add GDN partial rollback
Extend the gated delta net kernel to store intermediate states for
partial rollback support on the Metal backend.
- Add K (snapshot slot count) as a function constant
- Read input state from slot 0 of the 3D state tensor
- Write intermediate states to different slots during token loop
- For K=1, maintain backward-compatible single-slot behavior
Ref: 8c05923630#11 )
* server : adjust checkpoint logic
* cont : rm asserts
* server-context: fix early exit
* spec : fix compatibility with n-gram and add TODOs (#13 )
* metal : cleanup
* llama : fix faulty bitwise check in recurrent memory
* server : disable RS-based MTP in combination with other spec types
* spec : add TODOs
* cont : fix comment
* cont : update comment
* common : fix logic for ngram + mtp compat
* llama-memory: enable checkpointing with partial rollback
* cont: add test-case for loading into a dirty ctx
* llama-memory-recurrent: clear rs_idx in clear
* download: fix mtp path
* llama-arch: fix enorm op
* docs: update docs
* conversion: fix type annotations
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-05-16 20:06:23 +08:00
Aleksander Grygier
59778f0196
ui: Restructure repo to use tools/ui folder and ui / UI / llama-ui / LLAMA_UI naming ( #23064 )
...
* webui: Move static build output from `tools/server/public` to `build/ui` directory
* refactor: Move to `tools/ui`
* refactor: rename CMake variables and preprocessor defines
- Rename LLAMA_BUILD_WEBUI -> LLAMA_BUILD_UI (old kept as deprecated)
- Rename LLAMA_USE_PREBUILT_WEBUI -> LLAMA_USE_PREBUILT_UI (old kept as deprecated)
- Backward compat: old vars auto-forward to new ones with DEPRECATION warning
- Rename internal vars: WEBUI_SOURCE -> UI_SOURCE, WEBUI_SOURCE_DIR -> UI_SOURCE_DIR, etc.
- Rename HF bucket: LLAMA_WEBUI_HF_BUCKET -> LLAMA_UI_HF_BUCKET
- Emit both LLAMA_BUILD_WEBUI and LLAMA_BUILD_UI preprocessor defines
- Emit both LLAMA_WEBUI_DEFAULT_ENABLED and LLAMA_UI_DEFAULT_ENABLED
* refactor: rename CLI flags (--webui -> --ui) with backward compat
- Add --ui/--no-ui (old --webui/--no-webui kept as deprecated aliases)
- Add --ui-config (old --webui-config kept as deprecated alias)
- Add --ui-config-file (old --webui-config-file kept as deprecated alias)
- Add --ui-mcp-proxy/--no-ui-mcp-proxy (old --webui-mcp-proxy kept as deprecated)
- Add new env vars: LLAMA_ARG_UI, LLAMA_ARG_UI_CONFIG, LLAMA_ARG_UI_CONFIG_FILE, LLAMA_ARG_UI_MCP_PROXY
- C++ struct fields: params.ui, params.ui_config_json, params.ui_mcp_proxy added alongside old fields
- Backward compat: old fields synced to new ones in g_params_to_internals
* refactor: update C++ server internals with backward compat
- Rename json_webui_settings -> json_ui_settings (both kept in server_context_meta)
- Rename params.webui usage -> params.ui (both synced, old still works)
- JSON API emits both "ui"/"ui_settings" and "webui"/"webui_settings" keys
- Server routes use params.ui_mcp_proxy || params.webui_mcp_proxy
- Preprocessor guards use #if defined(LLAMA_BUILD_UI) || defined(LLAMA_BUILD_WEBUI)
* refactor: rename CI/CD workflows, artifacts, and build script
- Rename webui-build.yml -> ui-build.yml; artifact webui-build -> ui-build
- Rename webui-publish.yml -> ui-publish.yml; var HF_BUCKET_WEBUI_STATIC_OUTPUT -> HF_BUCKET_UI_STATIC_OUTPUT
- Rename server-webui.yml -> server-ui.yml; job webui-build/checks -> ui-build/checks
- Update server.yml: job/artifact refs webui-build -> ui-build
- Update release.yml: all webui-build/publish refs -> ui-build/publish; HF_TOKEN_WEBUI_STATIC_OUTPUT -> HF_TOKEN_UI_STATIC_OUTPUT
- Update server-self-hosted.yml: webui-build -> ui-build
- Update build-self-hosted.yml: HF_WEBUI_VERSION -> HF_UI_VERSION
- Rename webui-download.cmake -> ui-download.cmake (internal refs updated)
- Update labeler.yml: server/webui -> server/ui path label
* docs: update CODEOWNERS and server README docs
- Update CODEOWNERS: team ggml-org/llama-webui -> ggml-org/llama-ui, path /tools/server/webui/ -> /tools/ui/
- Update server README.md: CLI tables show --ui flags with deprecated --webui aliases
- Update server README-dev.md: "WebUI" -> "UI", paths updated to tools/ui/
* fix: Small fixes for UI build
* fix: CMake.txt syntax
* chore: Formatting
* fix: `.editorconfig` for llama-ui
* chore: Formatting
* refactor: Use `APP_NAME` in Error route
* refactor: Cleanup
* refactor: Single migration service
* make llama-ui a linkable target
* fix: UI Build output
* fix: Missing change
* fix: separate llama-ui npm build output into build/tools/ui/dist subfolder + use cmake npm build instead of downloading ui-build.yml artifacts in CI
* refactor: UI workflows cleanup
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-05-16 02:02:40 +02:00
Concedo
cc82c3164e
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/intel.Dockerfile
# .github/workflows/build-cross.yml
# .github/workflows/build-sycl.yml
# .github/workflows/build.yml
# .github/workflows/editorconfig.yml
# .github/workflows/release.yml
# cmake/riscv64-spacemit-linux-gnu-gcc.cmake
# docs/backend/OPENVINO.md
# docs/backend/SYCL.md
# docs/build-riscv64-spacemit.md
# docs/ops.md
# docs/ops/WebGPU.csv
# embd_res/ggml-vocab-qwen35.gguf
# embd_res/ggml-vocab-qwen35.gguf.inp
# embd_res/ggml-vocab-qwen35.gguf.out
# examples/model-conversion/Makefile
# ggml/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-utils.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-utils.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/common.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_tile.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_vec_reduce.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_vec_split.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/unary.wgsl
# ggml/src/ggml-zendnn/CMakeLists.txt
# ggml/src/ggml-zendnn/ggml-zendnn.cpp
# scripts/snapdragon/adb/run-completion.sh
# tests/CMakeLists.txt
# tools/cli/README.md
# tools/completion/README.md
# tools/mtmd/clip-impl.h
# tools/mtmd/clip.cpp
# tools/mtmd/clip.h
# tools/server/README.md
2026-05-14 19:04:04 +08:00
Georgi Gerganov
67b2b7f2f2
logs : reduce ( #23021 )
...
Python Type-Check / python type-check (push) Waiting to run
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* logs : reduce
* args : fix envs
* server : fix build
* common : print verbosity level at start
* server : clean-up logs
* server : print prompt processing timings + sampling params
* minor : whitespaces
2026-05-14 13:05:52 +03:00
Xuan-Son Nguyen
e75cd5efb5
download: do not exit() on error ( #23008 )
2026-05-13 15:14:58 +02:00
Georgi Gerganov
634275fbbb
spec : update CLI arguments for better consistency ( #22964 )
...
* spec : update CLI arguments for better consistency
* cont : fix CLI arg message
2026-05-13 09:15:39 +03:00
Xuan-Son Nguyen
7bfe120c21
mtmd, server, common: expose modalities to /v1/models ( #22952 )
...
* mtmd, server, common: expose modalities to /v1/models
* fix build
* rename to mtmd_caps
2026-05-12 19:08:07 +02:00
