vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-07-09 17:08:33 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
11341 commits 3 branches 248 tags 814 MiB
Commit graph

11341 commits

This branch
This branch
All branches
Author SHA1 Message Date
Concedo
7f485e5287 remove CLBlast, part 1 2026-01-23 13:50:12 +08:00
Concedo
4984c9bc16 Merge commit '12a4a47e6a' into concedo_experimental 
# Conflicts:
#	ci/run.sh
#	examples/model-conversion/scripts/causal/run-converted-model-embeddings-logits.sh
#	examples/model-conversion/scripts/causal/run-converted-model.sh
#	examples/model-conversion/scripts/embedding/run-converted-model.sh
#	ggml/src/ggml-cann/ggml-cann.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-zdnn/ggml-zdnn.cpp
#	ggml/src/ggml-zendnn/ggml-zendnn.cpp
#	tests/CMakeLists.txt
#	tests/test-chat-parser.cpp
#	tests/test-chat-peg-parser.cpp
#	tests/test-chat.cpp
#	tools/cli/cli.cpp
 2026-01-21 21:00:44 +08:00
Wagner Bruna
715a76ce5f
sd: sync to master-480-b87fe13 (#1932) 2026-01-21 20:49:35 +08:00
Piotr Wilkin (ilintar)
12a4a47e6a
Fix GLM 4.7 Lite MoE gating func (#18980) 
* Fix GLM 4.7 MoE gating func

* Update src/models/deepseek2.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update src/llama-model.cpp

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
 2026-01-21 12:35:20 +01:00
Concedo
28091dec43 pipeline parallel default enable 2026-01-21 17:57:41 +08:00
Matthieu Coudron
37c35f0e1c
gguf: display strerrno when cant load a model (#18884) 
I've had issues loading models with llama-server:
[44039] E gguf_init_from_file: failed to open GGUF file 'mistral-7b-v0.1.Q8_0.gguf'

and I was sure it could access the file. Seems like --models-dir and
--models-presets dont interact like I thought they would but I salvaged
this snippet that helps troubleshooting
[44039] E gguf_init_from_file: failed to open GGUF file 'mistral-7b-v0.1.Q8_0.gguf' (errno No such file or directory)
 2026-01-21 08:52:46 +02:00
Oliver Simons
5bd341c9a1
CUDA: Fix builds for older CCCL versions by ifdefing strided_iterator (#18964) 
* CUDA: Fix builds for older CCCL versions by ifdefing strided_iterator

Strided iterator was added in [CCCL
3.1](https://github.com/NVIDIA/cccl/releases/tag/v3.1.0), which is packaged into
[CTK
13.1](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id5)

* Unindent as per code review request
 2026-01-21 02:34:29 +01:00
Adrien Gallouët
1c7cf94b22
common, server : use the same User-Agent by default (#18957) 
This commit also ensures that if a custom User-Agent is used, it will be
the only one sent.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-01-20 18:28:43 +01:00
Xuan-Son Nguyen
2c1f199653
cli : fix reasoning responses in CLI (#18961) 
* cli : fix reasoning responses in CLI

* fix build

* fix build (2)
 2026-01-20 18:23:25 +01:00
Concedo
cdd6578a9a esrgan added 2026-01-20 22:10:37 +08:00
Oliver Simons
d1e3556481
CUDA: Replace init_offsets kernel with iterators in cub-based argsort (#18930) 
* CUDA: Replace `init_offsets` with iterators in argsort

This is a QOL improvement, saving us the cost of materializing the
iterator

* Remove unnecessary include from top-k.cu
 2026-01-20 20:11:01 +08:00
Adrien Gallouët
08f3f4a8a3
ggml : cleanup path_str() (#18928) 
- Remove pragmas as `std::codecvt_utf8` is not used.
- Avoid implicit `strlen()`.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-01-20 11:42:49 +01:00
Georgi Gerganov
271191906c
metal : enable FA for MLA heads (#18950) 2026-01-20 12:21:28 +02:00
Daniel Bevenius
7dee9ff59a
convert : use n_groups instead of hardcoded values in reshape (#18929) 
* convert : use n_groups instead of hardcoded values in reshape

This commit modifies the conversion script for NemotronHModel to use
the 'n_groups' hyperparameter, and allow Python to calculate the the
last dimension, using -1, when reshaping the 'mixer.norm.weight' tensor.

* use self.n_group instead of self.hparams["n_groups"]
 2026-01-20 06:55:24 +01:00
Xuan-Son Nguyen
6df686bee6
server : refactor oai_parser_opt, move it to server_chat_params (#18937) 
* server_chat_params

* move chat format into CLI

* use meta whenever possible

* clean up, no more chatml fallback
 2026-01-19 23:28:01 +01:00
ddh0
1706a6d7c6
convert : support Glm4MoeLite (#18936) 
* initial commit for branch

* add glm-4.7-flash, move tokenizer hash

* use `glm4` pretok

* silence flake8 E302 (CI)

* apply review feedback

* add <|user|> as eog

* also add EOG `<|observation|>`

* revert llama-vocab

* inherit vocab from glm4

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2026-01-19 23:09:20 +01:00
Sigbjørn Skjæret
959ecf7f23
jinja : fix undefined keys and attributes and int/float as bool (#18924) 
* fix undefined keys and attributes

* add falsy tests

* as_bool for integers and floats

* more falsy/truthy tests

* --typo
 2026-01-19 20:29:43 +01:00
Sigbjørn Skjæret
4037093c66
ci : run test-jinja -py on high perf [no ci] (#18916) 2026-01-19 20:29:15 +01:00
Lennart Austenfeld
18361c579c
server: fix memory reservations in populate_token_probs (#18787) 2026-01-19 19:13:31 +01:00
Georgi Gerganov
365a3e8c31
ggml : add ggml_build_forward_select (#18550) 
* ggml : add ggml_build_forward_select

* cuda : adapt CUDA graph compat to new feature

* vulkan : update logic to handle command buffer closing

* ggml : check compute for fusion

* ggml : add comment
 2026-01-19 20:03:19 +02:00
Concedo
c9c15749e0 wip on adding esrgan upscaling 2026-01-20 00:35:35 +08:00
Concedo
393791496d flux 2 taesd (+1 squashed commits) 
Squashed commits:

[adfc3f3a2] flux 2 taesd
 2026-01-19 23:47:16 +08:00
Daniel Bevenius
3d55846a5c
model-conversion : add BUILD_DIR variable to run-converted-model scripts (#18927) 
This commit adds a BUILD_DIR variable to the scripts used for running
converted models.

The motivation for this is that currently the `build` directory is
hardcoded and it can be useful to specify a different build directory,
with builds for different configurations.
 2026-01-19 13:12:38 +01:00
Concedo
d827494f17 fix text for vae (+1 squashed commits) 
Squashed commits:

[793caed19] fix text
 2026-01-19 01:50:07 +08:00
Concedo
70f92b12f8 sdxs clamp steps and cfg 2026-01-19 01:07:27 +08:00
Wagner Bruna
10851f223d
sd: sync to master-473-9565c7f (#1927) 
* sd: sync to master-473-9565c7f

* sd: add support for flux2 klein
 2026-01-19 01:04:34 +08:00
Julius Tischbein
287a33017b
llama : Extend fallback, fix fileno for dio file, exclude case that mmap uses dio file (#18887) 2026-01-18 18:35:57 +02:00
Concedo
7f618454ff Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/labeler.yml
#	CODEOWNERS
#	docs/backend/OPENCL.md
#	docs/ops.md
#	docs/ops/CANN.csv
#	docs/ops/WebGPU.csv
#	ggml/src/ggml-blas/CMakeLists.txt
#	ggml/src/ggml-opencl/kernels/mul_mv_q6_k.cl
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/cpy.tmpl.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/set_rows.wgsl
#	tests/test-backend-ops.cpp
 2026-01-18 23:24:29 +08:00
Francisco Herrera
293a1565dc
docs: add linux to index (#18907) 2026-01-18 18:03:35 +08:00
Concedo
3ba3d15fe3 fixed a typo 2026-01-18 16:41:34 +08:00
Llama
95ebfdcde8
Add token ids to logprob data returned by the API (#1928) 
Previously, logprobs only contained the token string
and byte data, as well as the log probability itself.
For workflows that require the token id, translating
from the token bytes to the token id is potentially
costly and unreliable. It is simple and inexpensive
to expose the numeric token ids directly instead.
 2026-01-18 16:30:46 +08:00
Concedo
7b4517c2fe embeddings memory usage regression fix 2026-01-18 16:26:52 +08:00
Xuan-Son Nguyen
fe44d35574
tests : add test-jinja -py option for cross-checking (#18906) 
* tests : add test-jinja -py option or cross-checking

* Update tests/test-jinja.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix + add source

* SandboxedEnvironment

* fix array.map case

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-01-18 08:14:27 +01:00
Concedo
3816391a74 increase logprobs returned to 10 2026-01-18 11:13:42 +08:00
Concedo
22ddad81b9 device override set in gui 2026-01-18 10:54:20 +08:00
Sigbjørn Skjæret
bbcdac0189
jinja : fix object item order (and properly implement dictsort) (#18904) 
* fix object item order

* as_ordered_object

* copy whole object
 2026-01-18 03:40:06 +01:00
Sigbjørn Skjæret
d03c45c9c5
jinja : attribute support for join, map and sort (#18883) 
* support negative array index and default value

* attribute support (int and str) for join, map and sort

* add tests

* update CODEOWNERS

* improve fixme sorting comment
 2026-01-18 02:53:01 +01:00
Sigbjørn Skjæret
10c98cbdf6
jinja : add missing tojson filter for bool (#18900) 
* add missing tojson for bool

* add more literal tests
 2026-01-18 01:05:09 +01:00
Sigbjørn Skjæret
420960ab92
jinja : fix lexing of float literals with sign (#18901) 
* fix lexing of float literals with sign

* add test

* consume_numeric
 2026-01-18 00:57:51 +01:00
Xuan-Son Nguyen
f55b033ae6
jinja: correct member access rule (#18905) 2026-01-18 00:48:55 +01:00
lhez
d1b4757ded
opencl: fix q6_K mv for m=1 (#18893) 2026-01-17 13:50:32 -08:00
Sigbjørn Skjæret
57c0beaed0
ci : add label for jinja changes (#18903) 2026-01-17 21:52:02 +01:00
Georgi Gerganov
2fbde785bc
kv-cache : optimize KQ mask construction (#18842) 
* kv-cache : optimize KQ mask construction

* cont : add explanation + improve

* cont : fix
 2026-01-17 15:42:42 +02:00
Concedo
ac3392d5da updated lite 2026-01-17 21:40:53 +08:00
Concedo
89a205ecc7 bump version 2026-01-17 19:09:14 +08:00
Concedo
62bea5ef4f allow overriding the devices directly 2026-01-17 19:08:06 +08:00
Concedo
21e6ccb8cb updated lite 2026-01-17 15:59:53 +08:00
Concedo
d2b2224b0d vulkan env var always take priority 2026-01-17 10:34:45 +08:00
Concedo
8855a7f52b Merge commit 'c945aaaef2' into concedo_experimental 
# Conflicts:
#	.devops/cann.Dockerfile
#	.github/workflows/build.yml
#	.github/workflows/release.yml
#	README.md
#	common/CMakeLists.txt
#	common/chat.cpp
#	docs/function-calling.md
#	ggml/src/ggml-cann/aclnn_ops.cpp
#	ggml/src/ggml-cann/aclnn_ops.h
#	ggml/src/ggml-cann/common.h
#	ggml/src/ggml-cann/ggml-cann.cpp
#	models/templates/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16.jinja
#	scripts/sync_vendor.py
#	tests/CMakeLists.txt
#	tests/peg-parser/tests.h
#	tests/test-chat-peg-parser.cpp
#	tests/test-chat-template.cpp
#	tests/test-chat.cpp
#	tests/testing.h
#	tools/llama-bench/llama-bench.cpp
 2026-01-17 10:24:03 +08:00
Reese Levine
a89002f07b
ggml webgpu: support for backend sampling (#18880)
Some checks failed
Update Operations Documentation / update-ops-docs (push) Has been cancelled
Details 
* ggml webgpu: add SOFTPLUS unary operator

Implements SOFTPLUS (log(1 + exp(x))) with f16/f32 support. Uses f32
precision for intermediate calculations to prevent f16 overflow.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support
* Follow Vulkan backend numerical stability pattern

* ggml webgpu: add EXPM1 unary operator

Implements EXPM1 (exp(x) - 1) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add FLOOR unary operator

Implements FLOOR (rounds down to nearest integer) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add CEIL unary operator

Implements CEIL (rounds up to nearest integer) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add ROUND unary operator

Implements ROUND (rounds to nearest integer) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add TRUNC unary operator

Implements TRUNC (truncates towards zero) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* docs : update WebGPU support for unary operators (FLOOR, CEIL, ROUND, TRUNC, EXPM1, SOFTPLUS)

* Updates to webgpu get_memory

* Add argmax

* Add argmax,cumsum,sum,sum_rows

* Add necessary CPY/GET_ROWS operators

* Support for argsort using multi-pass strategy

* Update set_rows for i32 indices, move to pre-wgsl

* Port unary operators to pre-wgsl and support FILL

* Implement PAD

* Add support for top-k

* clean up, scope pipeline init mutex

* fix newline

* Add support for log

* Update LOG for better precision, and ops doc

---------

Co-authored-by: Abhijit Ramesh <abhijitramesh2k@gmail.com>
 2026-01-16 16:12:43 -08:00