Concedo
378c3dd40c
updated lite
2025-04-27 01:46:52 +08:00
Concedo
77b9a83956
tryout termux autoinstaller (+1 squashed commits)
...
Squashed commits:
[9aeb5e902] tryout termux autoinstaller (+1 squashed commits)
Squashed commits:
[0e33b5934] tryout termux autoinstaller (+1 squashed commits)
Squashed commits:
[70232ea70] tryout termux autoinstaller (+1 squashed commits)
Squashed commits:
[050770315] tryout termux autoinstaller (+1 squashed commits)
Squashed commits:
[27bfc75a2] tryout termux autoinstaller (+1 squashed commits)
Squashed commits:
[6a32c1f93] tryout termux autoinstaller (+1 squashed commits)
Squashed commits:
[1e53b9d48] tryout termux autoinstaller
2025-04-27 01:27:23 +08:00
Concedo
4dcd215b27
handle explicit null
2025-04-26 13:06:38 +08:00
Concedo
cb1c182673
add more warmup (+1 squashed commits)
...
Squashed commits:
[9578d5352] updated lite
2025-04-26 10:22:09 +08:00
Concedo
4decd6bea1
GLM4 batch clamp
2025-04-26 09:42:17 +08:00
Concedo
3f545eadbe
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-rpc/ggml-rpc.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# tests/test-backend-ops.cpp
2025-04-26 09:12:40 +08:00
kallewoof
7cb815b727
AutoGuess: GLM-4 ( #1502 )
...
* AutoGuess: GLM-4
* add 'chat_start' field to adapters
* GLM-4 fix
2025-04-26 08:47:42 +08:00
Concedo
35dc8387e9
fixed rwkv7 handling
2025-04-26 02:13:06 +08:00
Concedo
5e87c04056
improved memory estimation (+2 squashed commit)
...
Squashed commit:
[3319540f9] mem estimation
[43bad21db] mem estimation
2025-04-26 02:03:09 +08:00
Diego Devesa
295354ea68
llama : fix K-shift with quantized K and BLAS backend ( #13113 )
2025-04-25 19:40:11 +02:00
City
558a764713
Force FP32 compute in GLM4 FFN Down ( #13101 )
...
* Force FP32 compute in cuBLAS GEMM
* Revert "Force FP32 compute in cuBLAS GEMM"
This reverts commit 6efd872732159ab88ee7b3c1d77ba5ebc83079bd.
* Force F32 compute in GLM4 ffn down
* Edit comment to clarify issue
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2025-04-25 14:38:34 +02:00
Xuan-Son Nguyen
edb18b6e8f
clip : fix pixtral on some GPU backends ( #13097 )
...
* clip : fix pixtral on some GPU backends
* refactor inp_raw set
* rm outdated comment
* fix dynamic size
* add TODO
2025-04-25 14:31:42 +02:00
Neo Zhang Jianyu
514c45608f
change the reorder tensor from init to execute OP ( #13003 )
2025-04-25 17:37:51 +08:00
Concedo
6b6597ebf1
allow for single token prompt processing (actual batch size 1)
2025-04-25 16:54:46 +08:00
Radoslav Gerganov
553a5c3a9f
rpc : do not wait for response when sending RPC_CMD_SET_TENSOR ( #12943 )
...
RPC_CMD_SET_TENSOR always returns an empty response and we send this 4
times per token. We can improve TG speed if we don't wait for this empty
response.
The performance impact of this change depends on the network latency.
2025-04-25 10:08:08 +03:00
Xuan-Son Nguyen
13be08daf9
clip : remove boi/eoi embeddings for GLM-edge model ( #13081 )
2025-04-24 22:17:04 +02:00
Georgi Gerganov
226251ed56
embeddings : fix batch sizes ( #13076 )
...
ggml-ci
2025-04-24 22:29:22 +03:00
Concedo
d32d0b382a
glm4 template
2025-04-25 00:41:15 +08:00
Georgi Gerganov
87616f0680
ggml : fix trailing whitespaces ( #0 )
2025-04-24 17:32:47 +03:00
Georgi Gerganov
63b4911494
sync : ggml
...
ggml-ci
2025-04-24 17:32:47 +03:00
Acly
c6e8cc28c1
ggml : Depthwise 2D convolution (ggml/1152)
...
* ggml-cpu : kernels for faster depthwise 2D convolution
* fix compile: remove static after moving to ops.cpp
* add dilation for depthwise_conv_2d
* review: rename to ggml_conv_2d_dw_direct, remove redundant struct keywords, pass by ref, whitespace
* review: rename depthwise_conv_2d -> conv_2d_dw everywhere
2025-04-24 17:32:47 +03:00
Johannes Gäßler
b10d8bfdb1
CUDA: use switch statements in constexpr functions ( #13095 )
2025-04-24 15:57:10 +02:00
Georgi Gerganov
13b4548877
cmake : do not include ./src as public for libllama ( #13062 )
...
* cmake : do not include ./src as public for libllama
ggml-ci
* cmake : rework tests
ggml-ci
* llguidance : remove unicode include
ggml-ci
* cmake : make c++17 private
ggml-ci
2025-04-24 16:00:10 +03:00
Georgi Gerganov
572b3141d3
clang-tidy : disable warning about missing math parenthesis ( #13091 )
2025-04-24 15:44:05 +03:00
Xuan-Son Nguyen
7c727fbe39
arg : add --no-mmproj-offload ( #13093 )
...
* arg : add --no-mmproj-offload
* Update common/arg.cpp
2025-04-24 14:04:14 +02:00
Concedo
25e747e9d8
up version
2025-04-24 18:44:29 +08:00
Xuan-Son Nguyen
80982e815e
arg : clean up handling --mmproj with -hf ( #13082 )
...
* arg : clean up handling --mmproj with -hf
* rm change about no_mmproj
* Revert "rm change about no_mmproj"
This reverts commit 2cac8e0efb629d66c612f137e75d562f94bb9e6c.
* handle no_mmproj explicitly
* skip download mmproj on examples not using it
2025-04-24 12:14:13 +02:00
Concedo
c21c8cd00a
Merge branch 'upstream' into concedo_experimental
2025-04-24 18:00:29 +08:00
Concedo
2f645bb1b4
pixtral is working only on cpu, however the images are distorted
2025-04-24 17:59:47 +08:00
Concedo
f1eb6c4e36
mtmd for debug
2025-04-24 16:27:24 +08:00
Georgi Gerganov
7604a7d6b8
metal : fix floating-point range of attention scores in FA kernels ( #13090 )
...
ggml-ci
2025-04-24 10:38:30 +03:00
Concedo
28a2723100
merged pixtral support, not fully working
2025-04-24 15:27:02 +08:00
Eve
b3b6d862cf
vulkan: matmul gcn tuning ( #13016 )
...
* tune matmul for gcn
* this one is more power efficient
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: 0cc4m <picard12@live.de>
* disable this tune for the proprietary driver
---------
Co-authored-by: 0cc4m <picard12@live.de>
2025-04-24 09:18:33 +02:00
Concedo
8f1edcbdac
Merge commit ' dc39a5e7a8' into concedo_experimental
...
# Conflicts:
# README.md
# SECURITY.md
# docs/multimodal/MobileVLM.md
# examples/llava/CMakeLists.txt
# examples/llava/README.md
# examples/llava/android/adb_run.sh
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/rope.cpp
# ggml/src/ggml-sycl/rope.hpp
2025-04-24 11:49:08 +08:00
pl752
5630406959
llama-mtmd-cli: Sigint rework in mtmd vision example ( #13080 )
...
* Sigint rework in mtmd vision example
* Applied suggestions on mtmd-cli PR
* Forgot to invert one of the conditions
* Update examples/llava/mtmd-cli.cpp
* Removed redundant exit check
---------
Co-authored-by: pl752 <maximpl752@gmail.com>
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
2025-04-23 23:32:35 +02:00
Xuan-Son Nguyen
ecda2ec4b3
mtmd : Support Pixtral 12B ( #13065 )
...
* add pixtral text model (vision is wip)
* cgraph ok, just missing 2D RoPE
* fix bad rebase
* first working version
* fix problem with img_break token
* support dynamic image size
* update docs
* update test script
2025-04-23 20:21:59 +02:00
piDack
eb1776b15a
convert : Append mult-eos,half-rope,bos to GLM4-0414 and Z ( #13021 )
...
* append mult-eos,half-rope,bos to GLM4-0414
* remove unset var
2025-04-23 16:59:14 +02:00
Radoslav Gerganov
2cca6c01e4
rpc : add command line option for number of threads for the CPU backend ( #13060 )
...
closes #13051
2025-04-23 10:32:49 +03:00
Johannes Gäßler
658987cfc9
CUDA: noncont MMVQ + batched bs1 MUL_MAT_ID ( #13014 )
...
* CUDA: noncont MMVQ + batched bs1 MUL_MAT_ID
* fix logic for RoPE support, CUDA graphs
2025-04-22 21:27:40 +02:00
Xuan-Son Nguyen
dc39a5e7a8
mtmd : support SmolVLM (version 1 and 2) ( #13050 )
...
* mtmd : support SmolVLM (version 1 and 2)
* correct chat template
* fix n_patches
* scale_factor is an int
* add more models to test
2025-04-22 16:24:54 +02:00
Concedo
3e8b84b8e5
added support for structured output in chat completions
2025-04-22 22:23:36 +08:00
Georgi Gerganov
ab47dec3d3
security : add note about RPC and server functionality ( #13061 )
...
* security : add note about RPC functionality
* security : add note about llama-server
2025-04-22 16:16:10 +03:00
Georgi Gerganov
7b53389c24
metal : add memory pool for temp allocs ( #12850 )
...
* metal : add memory pool for temp allocs (wip) [no ci]
* cont : free buffers from the heap
* cont : resize heap [no ci]
* cont : refactor heap [no ci]
* cont : heap for each cmd buffer [no ci]
* cont : fix free
* wip
* cont : fix alignment [no ci]
* cont : not working .. [no ci]
* cont : heap allocation now works [no ci]
* cont : use MTLHeapTypePlacement
ggml-ci
* metal : use dynamic MTLHeap allocations
ggml-ci
* metal : add comments
* metal : disable softmax use of mem_pool
ggml-ci
* metal : final touches
2025-04-22 16:15:51 +03:00
Xuan-Son Nguyen
243453533e
llava : update documentations ( #13055 )
...
* llava : update documentations
* fix typo
2025-04-22 10:37:00 +02:00
Concedo
e8b3aeaa28
update some defaults for max length and max ctx
2025-04-22 15:47:01 +08:00
Concedo
6dbee2f2f8
more robust glslc checks, increase default denoise str
2025-04-22 15:19:47 +08:00
Diego Devesa
1d735c0b4f
ggml : add SSE 4.2 and x64 base variant for CPUs without AVX ( #12871 )
...
* ggml : add SSE 4.2 variant for CPUs without AVX
* ggml : add x64 base ABI variant
2025-04-21 18:13:51 +02:00
Concedo
16156f0d86
updated lite
2025-04-21 22:24:33 +08:00
Concedo
6494dce405
handle estimation for multipart gguf (+1 squashed commits)
...
Squashed commits:
[c7b4af92] handle estimation for multipart gguf
2025-04-21 22:07:22 +08:00
Akarshan Biswas
5368ddda7a
SYCL: Add non-contiguous support in ROPE ( #12993 )
...
ggml-ci
2025-04-21 19:13:30 +05:30
