vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-11 17:44:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
7738 commits 3 branches 196 tags 389 MiB
Commit graph

147 commits

Author SHA1 Message Date
Concedo
1b0481f4b1 wip qwen25vl merge 2025-04-27 13:07:07 +08:00
Concedo
36c8db1248 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	examples/llava/clip-impl.h
#	examples/llava/clip.cpp
#	tests/test-arg-parser.cpp
#	tests/test-json-schema-to-grammar.cpp
 2025-04-27 12:51:02 +08:00
Xuan Son Nguyen
89be919988 fix merging problem 2025-04-26 22:54:41 +02:00
Xuan Son Nguyen
82f8e72ecd Merge branch 'master' into qwen25-vl 2025-04-26 22:45:06 +02:00
Xuan-Son Nguyen
4753791e70
clip : improve projector naming (#13118) 
* clip : improve projector naming

* no more kv has_llava_projector

* rm unused kv

* rm more unused
 2025-04-26 22:39:47 +02:00
Xuan Son Nguyen
0c74ea54f5 clean up 2025-04-26 22:37:05 +02:00
HimariO
7e1bb0437a remove attn_window_size from gguf 2025-04-26 20:19:51 +08:00
Concedo
3f545eadbe Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	ggml/src/ggml-rpc/ggml-rpc.cpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	tests/test-backend-ops.cpp
 2025-04-26 09:12:40 +08:00
HimariO
77b144a8e7 replace KEY_FULLATTN_BLK_IDX with KEY_WIN_ATTN_PATTERN 2025-04-26 01:00:00 +08:00
HimariO
f69e9fa04d remove KEY_USE_GLU_MLP, KEY_USE_RMS_NORM 2025-04-26 00:16:27 +08:00
HimariO
caa7e57ec5 add PROJECTOR_TYPE_QWEN2_5_VL 2025-04-26 00:03:02 +08:00
HimariO
a3cd0e52f2 fix attn weight scaling after rebase 2025-04-25 22:12:55 +08:00
HimariO
7f530ac040 remove commented-out code blocks 2025-04-25 22:12:55 +08:00
HimariO
d1af45988a cleaning up 2025-04-25 22:12:26 +08:00
HimariO
2eb32933ea move position id remap out of ggml to avoid int32 cuda operations 2025-04-25 22:12:26 +08:00
HimariO
444e47c088 fix few incorrect tensor memory layout 2025-04-25 22:11:48 +08:00
HimariO
3d5198ee05 handle window attention inputs 2025-04-25 22:11:13 +08:00
HimariO
d9f2d71bc2 implment vision model architecture, gguf convertor 2025-04-25 22:11:13 +08:00
Xuan-Son Nguyen
edb18b6e8f
clip : fix pixtral on some GPU backends (#13097) 
* clip : fix pixtral on some GPU backends

* refactor inp_raw set

* rm outdated comment

* fix dynamic size

* add TODO
 2025-04-25 14:31:42 +02:00
Concedo
6b6597ebf1 allow for single token prompt processing (actual batch size 1) 2025-04-25 16:54:46 +08:00
Xuan-Son Nguyen
13be08daf9
clip : remove boi/eoi embeddings for GLM-edge model (#13081) 2025-04-24 22:17:04 +02:00
Concedo
2f645bb1b4 pixtral is working only on cpu, however the images are distorted 2025-04-24 17:59:47 +08:00
Concedo
28a2723100 merged pixtral support, not fully working 2025-04-24 15:27:02 +08:00
Concedo
8f1edcbdac Merge commit 'dc39a5e7a8' into concedo_experimental 
# Conflicts:
#	README.md
#	SECURITY.md
#	docs/multimodal/MobileVLM.md
#	examples/llava/CMakeLists.txt
#	examples/llava/README.md
#	examples/llava/android/adb_run.sh
#	ggml/CMakeLists.txt
#	ggml/src/CMakeLists.txt
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/rope.cpp
#	ggml/src/ggml-sycl/rope.hpp
 2025-04-24 11:49:08 +08:00
Xuan-Son Nguyen
ecda2ec4b3
mtmd : Support Pixtral 12B (#13065) 
* add pixtral text model (vision is wip)

* cgraph ok, just missing 2D RoPE

* fix bad rebase

* first working version

* fix problem with img_break token

* support dynamic image size

* update docs

* update test script
 2025-04-23 20:21:59 +02:00
Xuan-Son Nguyen
dc39a5e7a8
mtmd : support SmolVLM (version 1 and 2) (#13050) 
* mtmd : support SmolVLM (version 1 and 2)

* correct chat template

* fix n_patches

* scale_factor is an int

* add more models to test
 2025-04-22 16:24:54 +02:00
Concedo
17360a3b32 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/build.yml
#	examples/llava/clip.cpp
 2025-04-20 17:59:58 +08:00
Xuan-Son Nguyen
37b9f0d29d
clip : refactor, add image_manipulation and llava_uhd classes (#13011) 
* clip : refactor, add `image_manipulation` and `llava_uhd`

* refactor llava-1.6 preprocessing

* simplify logic for llava-1.5

* missing include
 2025-04-19 09:15:45 +02:00
Concedo
a6149ad0fc fixed g3 adapter back 2025-04-12 23:17:54 +08:00
Concedo
9f94f62768 fixed segfault 2025-04-12 19:08:27 +08:00
Concedo
7b4254bef9 not working on cpu 2025-04-12 18:55:29 +08:00
Matt Clayton
e59ea539b8
llava: Fix cpu-only clip image encoding sefault (#12907) 
* llava: Fix cpu-only clip image encoding

* clip : no smart ptr for ggml_backend_t

* Fix for backend_ptr push_back

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2025-04-12 07:29:03 +02:00
Concedo
7e1289ade8 fixes for sdcpp 2025-04-12 10:08:23 +08:00
Concedo
a0ae187563 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/docker.yml
#	README.md
#	build-xcframework.sh
#	examples/llava/CMakeLists.txt
#	examples/llava/clip.cpp
#	examples/rpc/rpc-server.cpp
#	examples/run/run.cpp
#	ggml/src/ggml-cann/ggml-cann.cpp
#	scripts/sync-ggml-am.sh
#	scripts/sync-ggml.last
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
 2025-04-12 10:06:47 +08:00
Concedo
ea9bd61e47 Merge commit '64eda5deb9' into concedo_experimental 
# Conflicts:
#	.devops/cuda.Dockerfile
#	.devops/intel.Dockerfile
#	.devops/llama-cli-cann.Dockerfile
#	.devops/musa.Dockerfile
#	.devops/rocm.Dockerfile
#	.devops/vulkan.Dockerfile
#	.github/workflows/build.yml
#	.github/workflows/docker.yml
#	README.md
#	docs/backend/SYCL.md
#	examples/llava/clip.cpp
#	examples/server_embd.py
#	ggml/src/ggml-cann/acl_tensor.cpp
#	ggml/src/ggml-cann/aclnn_ops.cpp
#	ggml/src/ggml-cann/aclnn_ops.h
#	ggml/src/ggml-cann/ggml-cann.cpp
#	src/CMakeLists.txt
#	tests/test-chat-template.cpp
 2025-04-12 08:31:22 +08:00
Xuan-Son Nguyen
0c50923944
clip : use smart pointer (⚠️ breaking change) (#12869) 
* clip : use smart pointers

* fix warmup

* add forward declaration

* misisng include

* fix include (2)

* composite

* simplify batch ptr

* fix conflict
 2025-04-11 12:09:39 +02:00
Xuan-Son Nguyen
8b9cc7cdd8
llava : introduce libmtmd (#12849) 
* wip llava2

* migrated gemma3 to llava2

* add timings

* correct pre/postfix

* fix missing include

* fix compilation unused var warn

* update llava2_tokenize

* change name llava2 --> mtmd

* improve api

* refine helpers

* Update examples/llava/mtmd.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-04-10 22:57:16 +02:00
Xuan-Son Nguyen
65a69e6e1b
clip : do not print ftype (#12832) 2025-04-09 10:09:53 +02:00
Matt Clayton
b32efad2bc
llava: improve clip_ctx destructor to not memleak load_image_size (#12834) 2025-04-08 22:01:58 +02:00
dm4
2dabf759e7
llava: add more helper functions to check projector types in clip context (#12824) 
Signed-off-by: dm4 <sunrisedm4@gmail.com>
 2025-04-08 15:49:13 +02:00
Concedo
88660dd59d merged qwen2.5vl again 2025-04-08 21:32:25 +08:00
Concedo
822cf2430e Merge commit 'f1e3eb4249' into concedo_experimental 
# Conflicts:
#	.github/workflows/build.yml
#	README.md
#	docs/backend/SYCL.md
#	examples/llava/clip.cpp
#	ggml/src/ggml-sycl/CMakeLists.txt
#	ggml/src/ggml-vulkan/cmake/host-toolchain.cmake.in
 2025-04-08 20:48:53 +08:00
Concedo
c58e9a2be3 revert q2.5vl before merge (+1 squashed commits) 
Squashed commits:

[3197ea95] Revert "add tentative support for qwen2.5vl vision from HimariO fork"

This reverts commit 911669087a.
 2025-04-08 20:38:41 +08:00
HimariO
b28ad7ecca fix attn weight scaling after rebase 2025-04-07 22:07:56 +08:00
HimariO
223edef897 remove commented-out code blocks 2025-04-07 21:52:37 +08:00
HimariO
fdae70a832 cleaning up 2025-04-07 21:52:37 +08:00
HimariO
c891300c1e move position id remap out of ggml to avoid int32 cuda operations 2025-04-07 21:52:37 +08:00
HimariO
e18f6a3238 fix few incorrect tensor memory layout 2025-04-07 21:52:37 +08:00
HimariO
9c827814e6 handle window attention inputs 2025-04-07 21:51:18 +08:00
HimariO
9c7cc6de9c implment vision model architecture, gguf convertor 2025-04-07 21:46:06 +08:00