401 commits

Author	SHA1	Message	Date
Concedo	17c0c8d55d	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # README.md # docs/backend/zDNN.md # docs/build.md # docs/ops.md # ggml/CMakeLists.txt # ggml/src/CMakeLists.txt # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-opencl/ggml-opencl.cpp # ggml/src/ggml-rpc/ggml-rpc.cpp # ggml/src/ggml-sycl/convert.cpp # ggml/src/ggml-sycl/ggml-sycl.cpp # src/llama-quant.cpp # tests/test-backend-ops.cpp # tools/llama-bench/llama-bench.cpp # tools/server/README.md	2025-12-07 16:48:38 +08:00
Concedo	7c5d271d6c	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .github/workflows/build.yml # .github/workflows/release.yml # .github/workflows/winget.yml # CMakeLists.txt # CODEOWNERS # CONTRIBUTING.md # cmake/build-info.cmake # docs/ops.md # docs/ops/BLAS.csv # docs/ops/Metal.csv # examples/CMakeLists.txt # examples/save-load-state/save-load-state.cpp # examples/simple-cmake-pkg/README.md # ggml/CMakeLists.txt # ggml/src/CMakeLists.txt # ggml/src/ggml-rpc/ggml-rpc.cpp # ggml/src/ggml-webgpu/ggml-webgpu.cpp # ggml/src/ggml-webgpu/wgsl-shaders/embed_wgsl.py # src/llama-quant.cpp # tests/test-backend-ops.cpp # tools/server/CMakeLists.txt	2025-12-07 16:37:32 +08:00
Sigbjørn Skjæret	22577583a3	common : change --color to accept on/off/auto, default to auto (#17827)	2025-12-07 03:43:50 +01:00
Adrien Gallouët	ef75a89fdb	build : move _WIN32_WINNT definition to headers (#17736) ... Previously, cmake was forcing `_WIN32_WINNT=0x0A00` for MinGW builds, This caused "macro redefined" warnings with toolchains that define the version. This also removes the `GGML_WIN_VER` variable as it is no longer needed. Signed-off-by: Adrien Gallouët <angt@huggingface.co>	2025-12-04 07:04:02 +01:00
Concedo	03cec02a3d	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .github/workflows/build.yml # .github/workflows/release.yml # .github/workflows/winget.yml # CODEOWNERS # README.md # ci/run.sh # docs/build.md # docs/ops.md # docs/ops/Vulkan.csv # ggml/CMakeLists.txt # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-sycl/ggml-sycl.cpp # scripts/sync_vendor.py # src/CMakeLists.txt # tests/test-json-schema-to-grammar.cpp # tests/test-quantize-stats.cpp # tools/server/CMakeLists.txt # tools/server/README.md	2025-12-03 18:56:31 +08:00
Xuan-Son Nguyen	13628d8bdb	server: add --media-path for local media files (#17697) ... * server: add --media-path for local media files * remove unused fn	2025-12-02 22:49:20 +01:00
Xuan-Son Nguyen	ec18edfcba	server: introduce API for serving / loading / unloading multiple models (#17470) ... * server: add model management and proxy * fix compile error * does this fix windows? * fix windows build * use subprocess.h, better logging * add test * fix windows * feat: Model/Router server architecture WIP * more stable * fix unsafe pointer * also allow terminate loading model * add is_active() * refactor: Architecture improvements * tmp apply upstream fix * address most problems * address thread safety issue * address review comment * add docs (first version) * address review comment * feat: Improved UX for model information, modality interactions etc * chore: update webui build output * refactor: Use only the message data `model` property for displaying model used info * chore: update webui build output * add --models-dir param * feat: New Model Selection UX WIP * chore: update webui build output * feat: Add auto-mic setting * feat: Attachments UX improvements * implement LRU * remove default model path * better --models-dir * add env for args * address review comments * fix compile * refactor: Chat Form Submit component * ad endpoint docs * Merge remote-tracking branch 'webui/allozaur/server_model_management_v1_2' into xsn/server_model_maagement_v1_2 Co-authored-by: Aleksander <aleksander.grygier@gmail.com> * feat: Add copy to clipboard to model name in model info dialog * feat: Model unavailable UI state for model selector * feat: Chat Form Actions UI logic improvements * feat: Auto-select model from last assistant response * chore: update webui build output * expose args and exit_code in API * add note * support extra_args on loading model * allow reusing args if auto_load * typo docs * oai-compat /models endpoint * cleaner * address review comments * feat: Use `model` property for displaying the `repo/model-name` naming format * refactor: Attachments data * chore: update webui build output * refactor: Enum imports * feat: Improve Model Selector responsiveness * chore: update webui build output * refactor: Cleanup * refactor: Cleanup * refactor: Formatters * chore: update webui build output * refactor: Copy To Clipboard Icon component * chore: update webui build output * refactor: Cleanup * chore: update webui build output * refactor: UI badges * chore: update webui build output * refactor: Cleanup * refactor: Cleanup * chore: update webui build output * add --models-allow-extra-args for security * nits * add stdin_file * fix merge * fix: Retrieve lost setting after resolving merge conflict * refactor: DatabaseStore -> DatabaseService * refactor: Database, Conversations & Chat services + stores architecture improvements (WIP) * refactor: Remove redundant settings * refactor: Multi-model business logic WIP * chore: update webui build output * feat: Switching models logic for ChatForm or when regenerating messges + modality detection logic * chore: update webui build output * fix: Add `untrack` inside chat processing info data logic to prevent infinite effect * fix: Regenerate * feat: Remove redundant settigns + rearrange * fix: Audio attachments * refactor: Icons * chore: update webui build output * feat: Model management and selection features WIP * chore: update webui build output * refactor: Improve server properties management * refactor: Icons * chore: update webui build output * feat: Improve model loading/unloading status updates * chore: update webui build output * refactor: Improve API header management via utility functions * remove support for extra args * set hf_repo/docker_repo as model alias when posible * refactor: Remove ConversationsService * refactor: Chat requests abort handling * refactor: Server store * tmp webui build * refactor: Model modality handling * chore: update webui build output * refactor: Processing state reactivity * fix: UI * refactor: Services/Stores syntax + logic improvements Refactors components to access stores directly instead of using exported getter functions. This change centralizes store access and logic, simplifying component code and improving maintainability by reducing the number of exported functions and promoting direct store interaction. Removes exported getter functions from `chat.svelte.ts`, `conversations.svelte.ts`, `models.svelte.ts` and `settings.svelte.ts`. * refactor: Architecture cleanup * feat: Improve statistic badges * feat: Condition available models based on modality + better model loading strategy & UX * docs: Architecture documentation * feat: Update logic for PDF as Image * add TODO for http client * refactor: Enhance model info and attachment handling * chore: update webui build output * refactor: Components naming * chore: update webui build output * refactor: Cleanup * refactor: DRY `getAttachmentDisplayItems` function + fix UI * chore: update webui build output * fix: Modality detection improvement for text-based PDF attachments * refactor: Cleanup * docs: Add info comment * refactor: Cleanup * re * refactor: Cleanup * refactor: Cleanup * feat: Attachment logic & UI improvements * refactor: Constants * feat: Improve UI sidebar background color * chore: update webui build output * refactor: Utils imports + move types to `app.d.ts` * test: Fix Storybook mocks * chore: update webui build output * test: Update Chat Form UI tests * refactor: Tooltip Provider from core layout * refactor: Tests to separate location * decouple server_models from server_routes * test: Move demo test to tests/server * refactor: Remove redundant method * chore: update webui build output * also route anthropic endpoints * fix duplicated arg * fix invalid ptr to shutdown_handler * server : minor * rm unused fn * add ?autoload=true|false query param * refactor: Remove redundant code * docs: Update README documentations + architecture & data flow diagrams * fix: Disable autoload on calling server props for the model * chore: update webui build output * fix ubuntu build * fix: Model status reactivity * fix: Modality detection for MODEL mode * chore: update webui build output --------- Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2025-12-01 19:41:04 +01:00
Xuan-Son Nguyen	7733409734	common: improve verbosity level definitions (#17630) ... * common: improve verbosity level definitions * string_format * update autogen docs	2025-12-01 14:38:13 +01:00
Concedo	724763fdec	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .devops/vulkan.Dockerfile # .github/workflows/build.yml # .github/workflows/server.yml # common/common.cpp # examples/batched/README.md # ggml/CMakeLists.txt # ggml/src/CMakeLists.txt # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-cpu/CMakeLists.txt # ggml/src/ggml-cpu/arch-fallback.h # ggml/src/ggml-opencl/ggml-opencl.cpp # scripts/sync-ggml.last # src/CMakeLists.txt # tests/test-backend-ops.cpp # tools/server/CMakeLists.txt	2025-11-25 16:38:07 +08:00
Aaron Teo	877566d512	llama: introduce support for model-embedded sampling parameters (#17120)	2025-11-25 09:56:07 +08:00
Concedo	5248838a05	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .devops/cann.Dockerfile # .github/workflows/build.yml # .github/workflows/release.yml # .gitignore # README.md # common/CMakeLists.txt # docs/ops.md # docs/ops/Vulkan.csv # examples/eval-callback/eval-callback.cpp # ggml/src/ggml-cann/aclnn_ops.cpp # ggml/src/ggml-cpu/CMakeLists.txt # ggml/src/ggml-cpu/arch/x86/repack.cpp # ggml/src/ggml-cpu/kleidiai/kernels.cpp # scripts/sync-ggml.last # src/llama-grammar.cpp # tests/test-backend-ops.cpp # tests/test-chat.cpp # tools/server/CMakeLists.txt	2025-11-22 18:26:13 +08:00
Georgi Gerganov	196f5083ef	common : more accurate sampling timing (#17382) ... * common : more accurate sampling timing * eval-callback : minor fixes * cont : add time_meas impl * cont : fix log msg [no ci] * cont : fix multiple definitions of time_meas * llama-cli : exclude chat template init from time measurement * cont : print percentage of unaccounted time * cont : do not reset timings	2025-11-20 13:40:10 +02:00
LostRuins Concedo	5125c0b879	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .devops/vulkan.Dockerfile # ggml/src/ggml-cpu/CMakeLists.txt # ggml/src/ggml-opencl/ggml-opencl.cpp # ggml/src/ggml-opencl/kernels/set_rows.cl # ggml/src/ggml-vulkan/ggml-vulkan.cpp # ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp # tests/test-backend-ops.cpp # tools/batched-bench/batched-bench.cpp	2025-11-11 17:10:11 +08:00
Georgi Gerganov	f914544b16	batched-bench : add "separate text gen" mode (#17103)	2025-11-10 12:59:29 +02:00
Xuan-Son Nguyen	aa3b7a90b4	arg: add --cache-list argument to list cached models (#17073) ... * arg: add --cache-list argument to list cached models * new manifest naming format * improve naming * Update common/arg.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2025-11-08 21:54:14 +01:00
LostRuins Concedo	d6a2ad8455	still not really working right	2025-11-09 01:57:48 +08:00
LostRuins Concedo	dfb0966ed2	not working	2025-11-08 10:49:10 +08:00
Georgi Gerganov	13b339bcd9	server : do not default to multiple slots with speculative decoding (#17017) ... * server : do not default to multiple slots with speculative decoding * cont : fix	2025-11-05 14:32:55 +02:00
Xuan-Son Nguyen	070ff4d535	mtmd: add --image-min/max-tokens (#16921)	2025-11-03 11:11:18 +01:00
Concedo	7e7da2583e	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # ggml/src/ggml-cuda/CMakeLists.txt # ggml/src/ggml-cuda/common.cuh # ggml/src/ggml-cuda/fattn.cu # ggml/src/ggml-hip/CMakeLists.txt # ggml/src/ggml-musa/CMakeLists.txt	2025-10-12 16:42:51 +08:00
Georgi Gerganov	4b2dae383d	common : update presets (#16504) ... * presets : add --embd-gemma-default and remove old embedding presets * presets : add gpt-oss presets * presets : add vision presets * cont : remove reasoning overrides [no ci] * cont : fix batch size for embedding gemma [no ci]	2025-10-12 09:29:13 +03:00
Concedo	6d8f8cd65b	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # ggml/src/CMakeLists.txt	2025-10-11 10:01:43 +08:00
Georgi Gerganov	d00cbea63c	server : host-memory prompt caching (#16391) ... * minor : code style * server : fix prompt similarity calculation * server : initial host-memory prompt caching * cont * server : refactor * cont * cont : make the server task of the slot const * cont : minor [no ci] * server : cache prompts and checkpoints only for completion tasks * server : improve prompt caching logic * cont : fix check for number of cached prompts [no ci] * server : improve caching logic, add -cram CLI arg * server : print prompt mismatch info * cont : better naming [no ci] * server : improve prompt cache loading logic * server : add option to debug the slot contents (#16482) * server : add option to debug the slot contents * Update tools/server/server.cpp --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co> * server : add option to disable prompt cache --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>	2025-10-09 18:54:51 +03:00
Concedo	5b6ba02167	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .github/workflows/build.yml # ci/run.sh # examples/model-conversion/Makefile # examples/model-conversion/README.md # examples/model-conversion/logits.cpp # examples/model-conversion/requirements.txt # examples/model-conversion/scripts/embedding/convert-model.sh # examples/model-conversion/scripts/embedding/run-converted-model.sh # examples/model-conversion/scripts/embedding/run-original-model.py # examples/model-conversion/scripts/utils/semantic_check.py # ggml/src/ggml-cann/common.h # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-cpu/kleidiai/kernels.cpp # ggml/src/ggml-cpu/kleidiai/kernels.h # ggml/src/ggml-cpu/kleidiai/kleidiai.cpp # ggml/src/ggml-sycl/common.hpp # ggml/src/ggml-sycl/dpct/helper.hpp # ggml/src/ggml-sycl/ggml-sycl.cpp # ggml/src/ggml-sycl/softmax.cpp # ggml/src/ggml-sycl/softmax.hpp # requirements/requirements-all.txt # tests/test-chat-parser.cpp # tools/server/README.md	2025-10-09 23:46:56 +08:00
Pascal	12bbc3fa50	refactor: centralize CoT parsing in backend for streaming mode (#16394) ... * refactor: unify reasoning handling via backend reasoning_content, drop frontend tag parsing - Updated the chat message component to surface backend-supplied reasoning via message.thinking while showing the raw assistant content without inline tag scrubbing - Simplified chat streaming to append content chunks directly, stream reasoning into the message model, and persist any partial reasoning when generation stops - Refactored the chat service SSE handler to rely on server-provided reasoning_content, removing legacy <think> parsing logic - Refreshed Storybook data and streaming flows to populate the thinking field explicitly for static and streaming assistant messages * refactor: implement streaming-aware universal reasoning parser Remove the streaming mode limitation from --reasoning-format by refactoring try_parse_reasoning() to handle incremental parsing of <think> tags across all formats. - Rework try_parse_reasoning() to track whitespace, partial tags, and multiple reasoning segments, allowing proper separation of reasoning_content and content in streaming mode - Parse reasoning tags before tool call handling in content-only and Llama 3.x formats to ensure inline <think> blocks are captured correctly - Change default reasoning_format from 'auto' to 'deepseek' for consistent behavior - Add 'deepseek-legacy' option to preserve old inline behavior when needed - Update CLI help and documentation to reflect streaming support - Add parser tests for inline <think>...</think> segments The parser now continues processing content after </think> closes instead of stopping, enabling proper message.reasoning_content and message.content separation in both streaming and non-streaming modes. Fixes the issue where streaming responses would dump everything (including post-thinking content) into reasoning_content while leaving content empty. * refactor: address review feedback from allozaur - Passed the assistant message content directly to ChatMessageAssistant to drop the redundant derived state in the chat message component - Simplified chat streaming updates by removing unused partial-thinking handling and persisting partial responses straight from currentResponse - Refreshed the ChatMessage stories to cover standard and reasoning scenarios without the old THINK-tag parsing examples Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> * refactor: restore forced reasoning prefix to pass test-chat ([chat] All tests passed) - store the exact sequence seen on input when 'thinking_forced_open' enforces a reasoning block - inject this prefix before the first accumulated segment in 'reasoning_content', then clear it to avoid duplication - repeat the capture on every new 'start_think' detection to properly handle partial/streaming flows * refactor: address review feedback from ngxson * debug: say goodbye to curl -N, hello one-click raw stream - adds a new checkbox in the WebUI to display raw LLM output without backend parsing or frontend Markdown rendering * Update tools/server/webui/src/lib/components/app/chat/ChatMessages/ChatMessage.svelte Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> * webui: add Storybook example for raw LLM output and scope reasoning format toggle per story - Added a Storybook example that showcases the chat message component in raw LLM output mode with the provided trace sample - Updated every ChatMessage story to toggle the disableReasoningFormat setting so the raw-output rendering remains scoped to its own example * npm run format * chat-parser: address review feedback from ngxson Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com> --------- Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>	2025-10-08 23:18:41 +03:00
Concedo	b6f6338bba	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .github/workflows/build-linux-cross.yml # .github/workflows/build.yml # CODEOWNERS # ggml/CMakeLists.txt # ggml/src/ggml-cuda/fattn.cu # ggml/src/ggml-webgpu/CMakeLists.txt # ggml/src/ggml-webgpu/ggml-webgpu.cpp # ggml/src/ggml-webgpu/wgsl-shaders/mul_mat.tmpl.wgsl # tests/test-backend-ops.cpp # tests/test-chat-template.cpp # tools/llama-bench/llama-bench.cpp # tools/rpc/README.md # tools/server/README.md	2025-10-09 01:33:27 +08:00
Gadflyii	3df2244df4	llama : add --no-host to disable host buffers (#16310) ... * implement --no-host to disable host buffer * fix equal_mparams * move no-host enumeration order together with other model params --------- Co-authored-by: slaren <slarengh@gmail.com>	2025-10-06 19:55:53 +02:00
Concedo	1d728bbc89	Merge commit '128d522c04' into concedo_experimental ... # Conflicts: # .github/workflows/build.yml # .github/workflows/release.yml # ggml/src/ggml-vulkan/ggml-vulkan.cpp # tests/test-alloc.cpp # tests/test-chat.cpp	2025-10-04 23:51:22 +08:00
ddh0	f6dcda3900	server : context checkpointing for hybrid and recurrent models (#16382) ... * initial commit for branch 3 * generalize `swa_checkpoint` to `ctx_checkpoint` this extends `llama-server`'s SWA checkpointing logic to include hybrid/recurrent models such as Jamba, Granite * oops * disable debug prints * keep backwards compat with `--swa-checkpoints` Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * update prompt re-processing message * fix off-by-one error per GG * keep `seq_rm` log per GG Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * server : fix checkpoint logic to support recurrent caches * server : cleanup and fixes --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2025-10-03 21:34:51 +03:00
Concedo	b120e107f9	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .clang-tidy # .devops/musa.Dockerfile # .github/workflows/build-linux-cross.yml # .github/workflows/build.yml # .github/workflows/docker.yml # .gitignore # CODEOWNERS # CONTRIBUTING.md # README.md # build-xcframework.sh # ci/README-MUSA.md # ci/run.sh # common/CMakeLists.txt # docs/docker.md # examples/CMakeLists.txt # examples/eval-callback/CMakeLists.txt # examples/model-conversion/Makefile # examples/model-conversion/README.md # examples/model-conversion/logits.cpp # examples/model-conversion/scripts/causal/compare-logits.py # examples/model-conversion/scripts/causal/run-org-model.py # examples/model-conversion/scripts/embedding/compare-embeddings-logits.sh # examples/model-conversion/scripts/embedding/run-converted-model.sh # examples/model-conversion/scripts/embedding/run-original-model.py # examples/model-conversion/scripts/utils/check-nmse.py # examples/model-conversion/scripts/utils/inspect-org-model.py # examples/model-conversion/scripts/utils/semantic_check.py # ggml/CMakeLists.txt # ggml/include/ggml-zdnn.h # ggml/src/ggml-opencl/ggml-opencl.cpp # ggml/src/ggml-opencl/kernels/set_rows.cl # ggml/src/ggml-rpc/ggml-rpc.cpp # ggml/src/ggml-sycl/ggml-sycl.cpp # ggml/src/ggml-sycl/set_rows.cpp # ggml/src/ggml-webgpu/ggml-webgpu.cpp # ggml/src/ggml-zdnn/ggml-zdnn.cpp # tests/CMakeLists.txt # tests/test-backend-ops.cpp # tests/test-quantize-perf.cpp # tests/test-tokenizers-repo.sh # tools/perplexity/perplexity.cpp # tools/server/tests/README.md	2025-09-27 17:09:14 +08:00
Sigbjørn Skjæret	835b2b915c	model : add GroveMoE support (#15510) ... * add GroveMoE support * remove constexpr that fails on certain compilers * revert crude scalar div implementation, use cast * build_attn_inp_kv_unified -> build_attn_inp_kv * fix build_attn * re-apply ffn_exps regex changes	2025-09-25 19:50:28 +02:00
Concedo	3e72aaff5b	Merge commit '8f8f2274ee' into concedo_experimental ... # Conflicts: # .devops/rocm.Dockerfile # .github/workflows/build.yml # .github/workflows/release.yml # CMakeLists.txt # examples/simple/simple.cpp # ggml/src/ggml-cann/common.h # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-opencl/kernels/tsembd.cl # ggml/src/ggml-sycl/binbcast.cpp # ggml/src/ggml-sycl/binbcast.hpp # ggml/src/ggml-sycl/ggml-sycl.cpp # ggml/src/ggml-sycl/tsembd.cpp # ggml/src/ggml-zdnn/ggml-zdnn.cpp # src/llama-model.cpp # tools/batched-bench/CMakeLists.txt # tools/cvector-generator/CMakeLists.txt # tools/export-lora/CMakeLists.txt # tools/gguf-split/CMakeLists.txt # tools/imatrix/CMakeLists.txt # tools/llama-bench/CMakeLists.txt # tools/llama-bench/llama-bench.cpp # tools/main/CMakeLists.txt # tools/main/README.md # tools/mtmd/CMakeLists.txt # tools/perplexity/CMakeLists.txt # tools/perplexity/perplexity.cpp # tools/quantize/CMakeLists.txt # tools/rpc/rpc-server.cpp # tools/run/CMakeLists.txt # tools/run/run.cpp # tools/tokenize/CMakeLists.txt # tools/tts/CMakeLists.txt	2025-09-21 08:58:23 +08:00
jacekpoplawski	8ff206097c	llama-bench: add --n-cpu-moe support (#15952) ... * llama-bench: add --n-cpu-moe support Support --n-cpu-moe in llama-bench the same way it is supported by llama-server.	2025-09-16 16:17:08 +02:00
Sigbjørn Skjæret	b8e09f08b9	model : add grok-2 support (#15539) ... * add grok-2 support * type fix * type fix * type fix * "fix" vocab for invalid sequences * fix expert tensor mapping and spaces in vocab * add chat template * fix norm tensor mapping * rename layer_out_norm to ffn_post_norm * ensure ffn_post_norm is mapped * fix experts merging * remove erroneous FFN_GATE entry * concatenate split tensors and add more metadata * process all expert layers and try cat instead of hstack * add support for community BPE vocab * fix expert feed forward length and ffn_down concat * commit this too * add ffn_up/gate/down, unsure if sequence is right * add ffn_gate/down/up to tensor names * correct residual moe (still not working) * mess-- * fix embedding scale being applied twice * add built in chat template * change beta fast for grok if default value * remove spm vocab in favor of community bpe vocab * change attention temp length metadata type to integer * update attention temp length metadata * remove comment * replace M_SQRT2 with std::sqrt(2) * add yarn metadata, move defaults to hparams	2025-09-14 23:00:59 +02:00
Concedo	1dbd2fc259	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # docs/build-s390x.md # docs/ops.md # docs/ops/zDNN.csv # ggml/include/ggml-zdnn.h # ggml/src/ggml-sycl/binbcast.cpp # ggml/src/ggml-sycl/concat.cpp # ggml/src/ggml-sycl/conv.cpp # ggml/src/ggml-sycl/convert.cpp # ggml/src/ggml-sycl/cpy.cpp # ggml/src/ggml-sycl/dmmv.cpp # ggml/src/ggml-sycl/dpct/helper.hpp # ggml/src/ggml-sycl/element_wise.cpp # ggml/src/ggml-sycl/getrows.cpp # ggml/src/ggml-sycl/ggml-sycl.cpp # ggml/src/ggml-sycl/gla.cpp # ggml/src/ggml-sycl/im2col.cpp # ggml/src/ggml-sycl/mmq.cpp # ggml/src/ggml-sycl/mmvq.cpp # ggml/src/ggml-sycl/norm.cpp # ggml/src/ggml-sycl/rope.cpp # ggml/src/ggml-sycl/set_rows.cpp # ggml/src/ggml-sycl/softmax.cpp # ggml/src/ggml-sycl/tsembd.cpp # ggml/src/ggml-sycl/wkv.cpp # ggml/src/ggml-zdnn/ggml-zdnn-impl.h # ggml/src/ggml-zdnn/ggml-zdnn.cpp # tools/llama-bench/llama-bench.cpp	2025-09-13 12:25:30 +08:00
Eric Curtin	4bf5549269	Add docker protocol support for llama-server model loading (#15790) ... To pull and run models via: llama-server -dr gemma3 Add some validators and sanitizers for Docker Model urls and metadata Signed-off-by: Eric Curtin <eric.curtin@docker.com>	2025-09-12 16:31:50 +01:00
Georgi Gerganov	f088b6a84f	server : adjust prompt similarity thold + add logs (#15913) ... ggml-ci	2025-09-12 17:02:55 +03:00
Concedo	2562129271	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # README.md # ci/run.sh # docs/backend/CANN.md # examples/speculative/speculative.cpp # ggml/CMakeLists.txt # ggml/src/ggml-cann/aclnn_ops.cpp # ggml/src/ggml-cann/common.h # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-cpu/CMakeLists.txt # ggml/src/ggml-opencl/ggml-opencl.cpp # ggml/src/ggml-opencl/kernels/flash_attn_f16.cl # ggml/src/ggml-opencl/kernels/flash_attn_f32.cl # ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl # ggml/src/ggml-webgpu/ggml-webgpu.cpp # ggml/src/gguf.cpp # src/llama-context.cpp # tests/test-sampling.cpp # tools/server/README.md	2025-09-03 17:16:42 +08:00
Georgi Gerganov	0d161f021a	server : enable /slots by default and make it secure (#15630) ... * server : enable /slots by default and make it secure ggml-ci * server : fix tests to pass `--no-slots` when necessary * server : extend /props with info about enabled endpoints	2025-08-31 20:11:58 +03:00
Concedo	7e35954695	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # docs/build.md # docs/function-calling.md # examples/eval-callback/eval-callback.cpp # ggml/CMakeLists.txt # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-cpu/CMakeLists.txt # ggml/src/ggml-cpu/kleidiai/kernels.cpp # ggml/src/ggml-cpu/kleidiai/kernels.h # ggml/src/ggml-cpu/kleidiai/kleidiai.cpp # scripts/compare-llama-bench.py # scripts/server-bench.py # scripts/tool_bench.py # tests/test-chat.cpp # tools/batched-bench/batched-bench.cpp # tools/llama-bench/llama-bench.cpp # tools/server/README.md	2025-08-31 23:33:36 +08:00
Johannes Gäßler	e81b8e4b7f	llama: use FA + max. GPU layers by default (#15434) ... * llama: use max. GPU layers by default, auto -fa * ggml-backend: abort instead of segfault	2025-08-30 16:32:10 +02:00
Concedo	3060dfb99f	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # examples/model-conversion/Makefile # examples/model-conversion/scripts/causal/convert-model.sh # ggml/src/ggml-cann/aclnn_ops.cpp # ggml/src/ggml-cann/common.h # ggml/src/ggml-cann/ggml-cann.cpp # ggml/src/ggml-cuda/CMakeLists.txt # scripts/compare-commits.sh	2025-08-28 23:17:29 +08:00
Sigbjørn Skjæret	84ab83cc0b	model : jina-embeddings-v3 support (#13693) ... * initial jina-embeddings-v3 support * initial jina-embeddings-v3 support * initial jina-embeddings-v3 support * fix vocab parsing with only tokenizer.json * set mask token lstrip attribute * additional unk_token_id fallback just in case [no ci] * revert vocab_size() change [no ci] * merge tensor loading into general bert * rope * add lora embedding and loading (non-functional) * export separate lora ggufs instead * add adapter metadata api * use std::string * convert_hf_to_lora compatibility * fix assert * apply suggestions from review * apply suggestion from review	2025-08-28 15:49:50 +02:00
Concedo	8b8396c30c	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # README.md # docs/build-s390x.md # examples/llama.vim # ggml/src/ggml-cann/aclnn_ops.cpp # ggml/src/ggml-cann/common.h # scripts/compare-llama-bench.py # src/CMakeLists.txt # tests/test-backend-ops.cpp # tools/llama-bench/README.md # tools/llama-bench/llama-bench.cpp # tools/server/README.md	2025-08-23 11:35:28 +08:00
Georgi Gerganov	9ebebef62f	llama : remove KV cache defragmentation logic (#15473) ... ggml-ci	2025-08-22 12:22:13 +03:00
Concedo	90706ddb14	Merge commit 'fec9519802' into concedo_experimental ... # Conflicts: # Makefile # examples/lookahead/README.md # tools/server/CMakeLists.txt	2025-08-21 19:19:20 +08:00
Concedo	1c41c38a6a	Merge branch 'upstream' into concedo_experimental ... # Conflicts: # .devops/cuda.Dockerfile # CODEOWNERS # README.md # ggml/src/ggml-cann/aclnn_ops.cpp # ggml/src/ggml-cann/common.h # ggml/src/ggml-opencl/ggml-opencl.cpp # scripts/sync-ggml-am.sh # scripts/sync-ggml.last # scripts/sync-ggml.sh # tests/test-chat.cpp # tools/batched-bench/batched-bench.cpp # tools/mtmd/clip.h	2025-08-20 20:34:45 +08:00
Jie Fu (傅杰)	ec5ab1a36c	common : fix context shift help message (#15448) ... Signed-off-by: Jie Fu <jiefu@tencent.com>	2025-08-20 13:33:30 +03:00
Georgi Gerganov	d2fcd91cf9	server : disable context shift by default (#15416) ... * server : disable context shift by default ggml-ci * server : make scopr of test parameters local	2025-08-19 16:46:37 +03:00
Xuan-Son Nguyen	e9288e8869	chat : clarify the meaning of reasoning_format (#15408) ... * chat : clarify the meaning of reasoning_format * add link to this PR	2025-08-19 10:29:36 +02:00