Concedo
6df4ca13f1
drafting tweak
2026-06-23 20:11:25 +08:00
Concedo
4a7d6dd8a0
alias for draft
2026-06-23 18:43:28 +08:00
Concedo
7fe6fa6fb6
match draft defaults
2026-06-22 21:40:00 +08:00
Concedo
a0f39fe0f5
allow drafting with vision
2026-06-22 20:50:39 +08:00
Concedo
e4771e8e6b
restore draft state before main state, fixes reloading gemma4 assistant
2026-06-22 18:55:26 +08:00
Concedo
f202c0a457
mtp init -2
2026-06-21 10:21:48 +08:00
Concedo
6f4325ac87
support mp3 from api
2026-06-19 10:26:34 +08:00
Concedo
635c45e1a0
fix incorrect mtp layers setting (+1 squashed commits)
...
Squashed commits:
[8dad1a5c0] fix incorrect mtp layers setting
2026-06-18 21:10:28 +08:00
Concedo
197937aab0
cohere think budget
2026-06-16 01:03:24 +08:00
Concedo
a91357614a
added lfm2 tool call template support
2026-06-15 22:55:24 +08:00
Concedo
be80f5dcbc
auto fitting for draft models
2026-06-15 19:36:54 +08:00
Concedo
a06c915eb4
performance optimizations
2026-06-14 17:07:10 +08:00
Concedo
104b41091b
refactor drafting
2026-06-12 16:48:00 +08:00
Concedo
60d0e70d76
RNN lifeboat implementation
2026-06-11 23:40:43 +08:00
Concedo
7d1147bc35
smartcache improvements
2026-06-11 22:48:37 +08:00
Concedo
79331fd382
fixed mtp for qwen
2026-06-11 22:04:31 +08:00
Concedo
40084bb1f8
mtp drafting added (+1 squashed commits)
...
Squashed commits:
[b0fa1a6de] wip 1
2026-06-11 15:25:49 +08:00
Concedo
10e4b6d5e3
support gemma assistant as a draft model
2026-06-11 12:16:43 +08:00
Concedo
afc8de2c6b
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .github/workflows/build-webgpu.yml
# .github/workflows/release.yml
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/binary.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/concat.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/scale.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/unary.wgsl
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-mtmd-c-api.c
# tools/cli/cli.cpp
# tools/mtmd/CMakeLists.txt
# tools/server/README.md
2026-06-10 17:21:05 +08:00
Concedo
6bae15da71
fix build, remove clip quantize (+1 squashed commits)
...
Squashed commits:
[09ffa906b] fix build, remove clip quantize
2026-06-10 00:42:34 +08:00
Concedo
fe13a2989d
migration to mtmd complete
2026-06-09 23:09:45 +08:00
Concedo
0740ce70b7
revert incorrect token count for mtmd
2026-06-09 22:46:17 +08:00
Concedo
ac8e77d82e
Revert "cleanup round 2"
...
This reverts commit c7e39f9c97
2026-06-09 22:33:00 +08:00
Concedo
da0ba5cefd
Revert "cleanup round 3"
...
This reverts commit 0ead9907f0
2026-06-09 22:32:42 +08:00
Concedo
0ead9907f0
cleanup round 3
2026-06-09 21:52:46 +08:00
Concedo
c7e39f9c97
cleanup round 2
2026-06-09 21:49:20 +08:00
Concedo
f5acccad63
cleanup round 1
2026-06-09 21:13:48 +08:00
Concedo
90a14cecf8
mtmd checkpoint 2
2026-06-09 17:55:17 +08:00
Concedo
794a271cfa
upgrade to mtmd checkpoint 1
2026-06-09 16:06:30 +08:00
Concedo
9755b05556
g4ua audio fix
2026-06-05 10:59:24 +08:00
Concedo
b8acf16e73
hacky fix that worked for gemma4 uv
2026-06-05 01:40:08 +08:00
Concedo
2cfcd40fd4
gemma4uv vision still a little buggy
2026-06-05 00:53:23 +08:00
Concedo
7fb55b1e32
does not work for e4b
2026-06-04 10:44:40 +08:00
Concedo
e434b3e61b
kcpp image handling fix
2026-05-26 14:32:53 +08:00
Concedo
ce3aa09b99
cache dir is null
2026-05-23 17:39:09 +08:00
Concedo
4bbbd55be6
rpc implementation is complete
2026-05-23 17:11:30 +08:00
Concedo
f85cc79526
make swa default on models that support it. removed --useswa, added --noswa
2026-05-22 23:38:33 +08:00
Concedo
694e8824c5
mmproj autofit reworked
2026-05-22 20:36:16 +08:00
Concedo
095bf63b58
prep for rpc
2026-05-20 23:29:49 +08:00
Concedo
9203b6a051
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# .github/workflows/build-self-hosted.yml
# .github/workflows/release.yml
# .github/workflows/server-sanitize.yml
# .github/workflows/server-self-hosted.yml
# .github/workflows/server.yml
# .github/workflows/ui-build.yml
# .github/workflows/ui-ci.yml
# .github/workflows/ui-publish.yml
# .gitignore
# CMakeLists.txt
# CODEOWNERS
# scripts/ui-download.cmake
# scripts/xxd.cmake
# tests/test-backend-ops.cpp
# tests/test-reasoning-budget.cpp
# tools/CMakeLists.txt
# tools/server/CMakeLists.txt
# tools/server/README.md
2026-05-16 22:56:33 +08:00
Concedo
80ce8a50b3
allow token bans and eos handling in
2026-05-16 15:20:46 +08:00
askmyteapot
3174fccb83
Fix linker error for kcpp_permit_any_repack ( #2210 )
...
Fixes `gpttype_adapter.lib(gpttype_adapter.obj) : error LNK2019: unresolved external symbol "int kcpp_permit_any_repack"`
It's a `bool`, not an `int`
2026-05-16 08:55:56 +08:00
Concedo
66a7b5e5de
prevent repacking if mmap is used, reduces memory footprint
2026-05-15 16:40:53 +08:00
Concedo
286e62267e
adjust batching eligibility
2026-05-11 21:54:32 +08:00
Concedo
bfaddd7a3b
added support for added memory and gemma and glm prompt fixes for batching mode
2026-05-10 23:39:03 +08:00
Concedo
33ca75d56f
ci for tools upload, minor function reordering
2026-05-10 23:10:43 +08:00
AlpinDale
c03302b670
feat: add a primitive form of continuous batching ( #2167 )
...
* feat: add a primitive form of continuous batching
* fix: deadlock in batching fallback
* fix: windows build
* chore: suppress the contbatch arg from --help
* feat: batch-aware rep_pen_slope
* fix: automatically disable shifting when batching is enabled
* fix: mixed-path state corruption
* fix: attempt to fully separate the two pipelines
* added a semaphore to prevent non-batchable requests from starting while batched requests are running
---------
Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
2026-05-10 17:50:31 +08:00
Concedo
eb30b29d69
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/gguf-publish.yml
# CODEOWNERS
# examples/sycl/test.sh
# pyproject.toml
# tools/mtmd/CMakeLists.txt
# tools/mtmd/README.md
2026-05-08 14:48:57 +08:00
Concedo
950676fdb7
split utils.cpp into 2 files to support sd.cpp
2026-05-04 15:04:12 +08:00
Concedo
8b62e7b667
allow splitmode to be set independently, enable tensor parallelism
2026-05-02 16:41:28 +08:00
