HimariO
f69e9fa04d
remove KEY_USE_GLU_MLP, KEY_USE_RMS_NORM
2025-04-26 00:16:27 +08:00
HimariO
caa7e57ec5
add PROJECTOR_TYPE_QWEN2_5_VL
2025-04-26 00:03:02 +08:00
HimariO
a3cd0e52f2
fix attn weight scaling after rebase
2025-04-25 22:12:55 +08:00
HimariO
7f530ac040
remove commented-out code blocks
2025-04-25 22:12:55 +08:00
HimariO
2de5dc3a14
remove not so often use qwen2vl-cli debug functions
2025-04-25 22:12:55 +08:00
HimariO
91fbdd781d
ignore transformers Qwen2_5_xxx type check
2025-04-25 22:12:26 +08:00
HimariO
d1af45988a
cleaning up
2025-04-25 22:12:26 +08:00
HimariO
2eb32933ea
move position id remap out of ggml to avoid int32 cuda operations
2025-04-25 22:12:26 +08:00
HimariO
444e47c088
fix few incorrect tensor memory layout
2025-04-25 22:11:48 +08:00
HimariO
69b39addd2
add debug utils
2025-04-25 22:11:48 +08:00
HimariO
3d5198ee05
handle window attention inputs
2025-04-25 22:11:13 +08:00
HimariO
d9f2d71bc2
implment vision model architecture, gguf convertor
2025-04-25 22:11:13 +08:00
City
558a764713
Force FP32 compute in GLM4 FFN Down ( #13101 )
...
* Force FP32 compute in cuBLAS GEMM
* Revert "Force FP32 compute in cuBLAS GEMM"
This reverts commit 6efd872732159ab88ee7b3c1d77ba5ebc83079bd.
* Force F32 compute in GLM4 ffn down
* Edit comment to clarify issue
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2025-04-25 14:38:34 +02:00
Xuan-Son Nguyen
edb18b6e8f
clip : fix pixtral on some GPU backends ( #13097 )
...
* clip : fix pixtral on some GPU backends
* refactor inp_raw set
* rm outdated comment
* fix dynamic size
* add TODO
2025-04-25 14:31:42 +02:00
Neo Zhang Jianyu
514c45608f
change the reorder tensor from init to execute OP ( #13003 )
2025-04-25 17:37:51 +08:00
Radoslav Gerganov
553a5c3a9f
rpc : do not wait for response when sending RPC_CMD_SET_TENSOR ( #12943 )
...
RPC_CMD_SET_TENSOR always returns an empty response and we send this 4
times per token. We can improve TG speed if we don't wait for this empty
response.
The performance impact of this change depends on the network latency.
2025-04-25 10:08:08 +03:00
Xuan-Son Nguyen
13be08daf9
clip : remove boi/eoi embeddings for GLM-edge model ( #13081 )
2025-04-24 22:17:04 +02:00
Georgi Gerganov
226251ed56
embeddings : fix batch sizes ( #13076 )
...
ggml-ci
2025-04-24 22:29:22 +03:00
Georgi Gerganov
87616f0680
ggml : fix trailing whitespaces ( #0 )
2025-04-24 17:32:47 +03:00
Georgi Gerganov
63b4911494
sync : ggml
...
ggml-ci
2025-04-24 17:32:47 +03:00
Acly
c6e8cc28c1
ggml : Depthwise 2D convolution (ggml/1152)
...
* ggml-cpu : kernels for faster depthwise 2D convolution
* fix compile: remove static after moving to ops.cpp
* add dilation for depthwise_conv_2d
* review: rename to ggml_conv_2d_dw_direct, remove redundant struct keywords, pass by ref, whitespace
* review: rename depthwise_conv_2d -> conv_2d_dw everywhere
2025-04-24 17:32:47 +03:00
Johannes Gäßler
b10d8bfdb1
CUDA: use switch statements in constexpr functions ( #13095 )
2025-04-24 15:57:10 +02:00
Georgi Gerganov
13b4548877
cmake : do not include ./src as public for libllama ( #13062 )
...
* cmake : do not include ./src as public for libllama
ggml-ci
* cmake : rework tests
ggml-ci
* llguidance : remove unicode include
ggml-ci
* cmake : make c++17 private
ggml-ci
2025-04-24 16:00:10 +03:00
Georgi Gerganov
572b3141d3
clang-tidy : disable warning about missing math parenthesis ( #13091 )
2025-04-24 15:44:05 +03:00
Xuan-Son Nguyen
7c727fbe39
arg : add --no-mmproj-offload ( #13093 )
...
* arg : add --no-mmproj-offload
* Update common/arg.cpp
2025-04-24 14:04:14 +02:00
Xuan-Son Nguyen
80982e815e
arg : clean up handling --mmproj with -hf ( #13082 )
...
* arg : clean up handling --mmproj with -hf
* rm change about no_mmproj
* Revert "rm change about no_mmproj"
This reverts commit 2cac8e0efb629d66c612f137e75d562f94bb9e6c.
* handle no_mmproj explicitly
* skip download mmproj on examples not using it
2025-04-24 12:14:13 +02:00
Georgi Gerganov
7604a7d6b8
metal : fix floating-point range of attention scores in FA kernels ( #13090 )
...
ggml-ci
2025-04-24 10:38:30 +03:00
Eve
b3b6d862cf
vulkan: matmul gcn tuning ( #13016 )
...
* tune matmul for gcn
* this one is more power efficient
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: 0cc4m <picard12@live.de>
* disable this tune for the proprietary driver
---------
Co-authored-by: 0cc4m <picard12@live.de>
2025-04-24 09:18:33 +02:00
pl752
5630406959
llama-mtmd-cli: Sigint rework in mtmd vision example ( #13080 )
...
* Sigint rework in mtmd vision example
* Applied suggestions on mtmd-cli PR
* Forgot to invert one of the conditions
* Update examples/llava/mtmd-cli.cpp
* Removed redundant exit check
---------
Co-authored-by: pl752 <maximpl752@gmail.com>
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
2025-04-23 23:32:35 +02:00
Xuan-Son Nguyen
ecda2ec4b3
mtmd : Support Pixtral 12B ( #13065 )
...
* add pixtral text model (vision is wip)
* cgraph ok, just missing 2D RoPE
* fix bad rebase
* first working version
* fix problem with img_break token
* support dynamic image size
* update docs
* update test script
2025-04-23 20:21:59 +02:00
piDack
eb1776b15a
convert : Append mult-eos,half-rope,bos to GLM4-0414 and Z ( #13021 )
...
* append mult-eos,half-rope,bos to GLM4-0414
* remove unset var
2025-04-23 16:59:14 +02:00
Radoslav Gerganov
2cca6c01e4
rpc : add command line option for number of threads for the CPU backend ( #13060 )
...
closes #13051
2025-04-23 10:32:49 +03:00
Johannes Gäßler
658987cfc9
CUDA: noncont MMVQ + batched bs1 MUL_MAT_ID ( #13014 )
...
* CUDA: noncont MMVQ + batched bs1 MUL_MAT_ID
* fix logic for RoPE support, CUDA graphs
2025-04-22 21:27:40 +02:00
Xuan-Son Nguyen
dc39a5e7a8
mtmd : support SmolVLM (version 1 and 2) ( #13050 )
...
* mtmd : support SmolVLM (version 1 and 2)
* correct chat template
* fix n_patches
* scale_factor is an int
* add more models to test
2025-04-22 16:24:54 +02:00
Georgi Gerganov
ab47dec3d3
security : add note about RPC and server functionality ( #13061 )
...
* security : add note about RPC functionality
* security : add note about llama-server
2025-04-22 16:16:10 +03:00
Georgi Gerganov
7b53389c24
metal : add memory pool for temp allocs ( #12850 )
...
* metal : add memory pool for temp allocs (wip) [no ci]
* cont : free buffers from the heap
* cont : resize heap [no ci]
* cont : refactor heap [no ci]
* cont : heap for each cmd buffer [no ci]
* cont : fix free
* wip
* cont : fix alignment [no ci]
* cont : not working .. [no ci]
* cont : heap allocation now works [no ci]
* cont : use MTLHeapTypePlacement
ggml-ci
* metal : use dynamic MTLHeap allocations
ggml-ci
* metal : add comments
* metal : disable softmax use of mem_pool
ggml-ci
* metal : final touches
2025-04-22 16:15:51 +03:00
Xuan-Son Nguyen
243453533e
llava : update documentations ( #13055 )
...
* llava : update documentations
* fix typo
2025-04-22 10:37:00 +02:00
Diego Devesa
1d735c0b4f
ggml : add SSE 4.2 and x64 base variant for CPUs without AVX ( #12871 )
...
* ggml : add SSE 4.2 variant for CPUs without AVX
* ggml : add x64 base ABI variant
2025-04-21 18:13:51 +02:00
Akarshan Biswas
5368ddda7a
SYCL: Add non-contiguous support in ROPE ( #12993 )
...
ggml-ci
2025-04-21 19:13:30 +05:30
Xuan-Son Nguyen
84a9bf2fc2
mtmd : merge llava, gemma3 and minicpmv CLI into single llama-mtmd-cli ( #13012 )
...
* mtmd : merge `llava-cli` and `gemma3-cli` into single `mtmd-cli`
* support for minicpmv
* remove cpp files of llava and minicpmv
* update hot topics
* mtmd : add not supported msg for qwen2vl
* Update examples/llava/mtmd.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-04-21 15:32:58 +02:00
Xuan-Son Nguyen
2016f07bd1
convert : experimental support for --mmproj flag ( #13023 )
...
* convert : experimental support for `--mmproj` flag
* fix bad ctrl+f replace
* fix style
* split into subclasses TextModel and VisionModel
* rename Mode --> ModelBase
* small fix
* correct CLIP_VISION arch name (because existing GGUF already use it)
* Apply suggestions from code review
Co-authored-by: compilade <git@compilade.net>
* fix Mistral3Model
* fix typo
Co-authored-by: compilade <git@compilade.net>
---------
Co-authored-by: compilade <git@compilade.net>
2025-04-20 23:29:36 +02:00
Jeffrey Morgan
6602304814
llava: fix errors in clip.h on certain compilers ( #13030 )
2025-04-20 12:15:41 +02:00
Jeff Bolz
66168204be
vulkan: support noncontiguous rms_norm ( #13031 )
2025-04-20 10:50:02 +02:00
Jeffrey Morgan
4ba9d711ba
metal: add neg operator ( #13029 )
2025-04-20 08:28:40 +03:00
bandoti
00137157fc
Disable CI cross-compile builds ( #13022 )
2025-04-19 18:05:03 +02:00
Sigbjørn Skjæret
fb28f4f80e
gguf-py : fix upload python package workflow ( #13020 )
2025-04-19 16:26:38 +02:00
Xuan-Son Nguyen
37b9f0d29d
clip : refactor, add image_manipulation and llava_uhd classes ( #13011 )
...
* clip : refactor, add `image_manipulation` and `llava_uhd`
* refactor llava-1.6 preprocessing
* simplify logic for llava-1.5
* missing include
2025-04-19 09:15:45 +02:00
Daniel Tang
6408210082
main : Fix Ctrl+D/newline handling ( #12951 )
...
This restores the behavior from #491 . This does not affect Ctrl+D's ability to
terminate --multiline-input lines (#1040 ).
This also actually implements #587 : "If the user wants the text to end in a
newline, this should be accomplished by explicitly adding a newline by using
\ followed by return, then returning control by pressing return again."
Fixes #12949
2025-04-18 22:02:55 +02:00
Chris Thompson
aff9d107b0
gguf-py : GGUF Editor GUI - Python + Qt6 ( #12930 )
2025-04-18 20:30:41 +02:00
Xuan-Son Nguyen
35370ba945
server : use std::move whenever possible ( #12936 )
...
* server : use std::move whenever possible
* use r-value ref
* Apply suggestions from code review
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* make task creation scoped
* restore std::move
* fix task_id not set correctly
* apply changes from suggestion
Co-authored-by: ggerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-04-18 19:58:12 +02:00
