vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-07-10 01:18:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
koboldcpp/ggml/src
History
Hongqiang Wang ebd048fc5e
opencl: flash attention improvement (#25069) 
* opencl: rework FA kernel for f16 and f32

* opencl: flash-attention prefill prepass kernels

- flash_attn_kv_pad_f16    pads the tail KV tile to a BLOCK_N multiple
- flash_attn_mask_pad_f16  pads the matching mask tile
- flash_attn_blk_f16       classifies each KV tile per query block as
                           fully masked / mixed / fully unmasked, so
                           the main kernel can skip fully-masked tiles
                           and the mask lookup for fully-unmasked ones

* opencl: FA kernels for q4_0 and q8_0

* opencl: `set_rows` for f32 to q8_0/q4_0

* opencl: dequant kernels for q4_0 and q8_0

* opencl: add FA tile tuning table with override

* opencl: wire host side for FA

* opencl: q4_0 MoE tensors are also SOA'ed

* opencl: cosmetic fix

* opencl: refactor, also clarify some code paths in comments

* opencl: fix inifity for `-cl-finite-math-only`

---------

Co-authored-by: Li He <lih@qti.qualcomm.com>
2026-06-27 15:36:06 -07:00
..
ggml-blas vulkan: add get/set tensor 2d functions (#22514) 2026-04-30 17:37:13 +02:00
ggml-cann vulkan: add get/set tensor 2d functions (#22514) 2026-04-30 17:37:13 +02:00
ggml-cpu ggml-cpu: fix SVE leftover path in ggml_vec_dot_f32 (#24699) 2026-06-26 10:41:56 +03:00
ggml-cuda [CUDA] Added a cudaMemcpy2DAsync fast path to ggml_cuda_cpy (#25057) 2026-06-27 17:46:21 +05:30
ggml-hexagon hexagon: MUL_MAT and MUL_MAT_ID rework : 32x32 tiled weight repack, kernel-params, cached graphs (#24954) 2026-06-24 12:14:25 -07:00
ggml-hip ggml: backend-agnostic tensor parallelism (experimental) (#19378) 2026-04-09 16:42:19 +02:00
ggml-metal metal : check for BF16 support in concat kernel (#24747) 2026-06-18 09:16:06 +03:00
ggml-musa ggml-cuda: native bf16 flash attention for vec kernel (#20525) 2026-03-22 11:05:51 +01:00
ggml-opencl opencl: flash attention improvement (#25069) 2026-06-27 15:36:06 -07:00
ggml-openvino [CUDA] Added a cudaMemcpy2DAsync fast path to ggml_cuda_cpy (#25057) 2026-06-27 17:46:21 +05:30
ggml-rpc rpc : keep last_graph_uid in the device context (#23273) 2026-05-19 09:42:36 +03:00
ggml-sycl sycl : fix failed ut cases of norm (#25044) 2026-06-27 12:13:43 +03:00
ggml-virtgpu ggml-virtgpu : include missing mutex header (#22810) 2026-05-10 17:32:41 +02:00
ggml-vulkan vulkan: fix step operator for 0 input (#25036) 2026-06-27 10:57:31 +02:00
ggml-webgpu vulkan: support all backend tests for SQR/SQRT/SIN/COS/CLAMP/LEAKY_RELU/NORM (#24582) 2026-06-23 09:48:24 -05:00
ggml-zdnn vulkan: add get/set tensor 2d functions (#22514) 2026-04-30 17:37:13 +02:00
ggml-zendnn ggml-zendnn : fixed naming of matmul function (#20964) 2026-05-27 00:59:35 +02:00
CMakeLists.txt ggml-cpu: Conditionally enable power11 backend based on compiler support (#24687) 2026-06-18 02:45:19 +08:00
ggml-alloc.c ggml-alloc: fix out-of-bounds read in ggml_dyn_tallocr_remove_block (ggml/1492) 2026-05-25 12:38:01 +03:00
ggml-backend-dl.cpp hexagon: enable offloading to Hexagon on Windows on Snapdragon (#19150) 2026-01-29 12:33:21 -08:00
ggml-backend-dl.h hexagon: enable offloading to Hexagon on Windows on Snapdragon (#19150) 2026-01-29 12:33:21 -08:00
ggml-backend-impl.h ggml: backend-agnostic tensor parallelism (experimental) (#19378) 2026-04-09 16:42:19 +02:00
ggml-backend-meta.cpp Remove padding and multiple D2D copies for MTP (#24086) 2026-06-10 23:21:16 +05:30
ggml-backend-reg.cpp ggml : skip already registered backends and devices (#22296) 2026-04-28 10:02:32 +03:00
ggml-backend.cpp sched : reintroduce less synchronizations during split compute (#20793) 2026-06-26 17:18:30 +03:00
ggml-common.h ggml: add Q1_0 1-bit quantization support (CPU) (#21273) 2026-04-06 20:55:21 +02:00
ggml-impl.h ggml: add graph_reused (#21764) 2026-04-16 17:21:28 +08:00
ggml-opt.cpp fix: free ctx_copy in ggml_opt_free to plug per-training-session leak (#21592) 2026-04-08 17:40:15 +02:00
ggml-quants.c ggml : Parallelize quant LUT init (#23595) 2026-05-25 10:15:46 +03:00
ggml-quants.h ggml: add Q1_0 1-bit quantization support (CPU) (#21273) 2026-04-06 20:55:21 +02:00
ggml-threading.cpp
ggml-threading.h
ggml.c mtmd, arg: fix utf8 handling on windows (#24779) 2026-06-19 22:28:38 +02:00
ggml.cpp ggml : Print backtrace on uncaught C++ exceptions (ggml/1232) 2025-06-01 13:43:57 +03:00
gguf.cpp ggml: gguf_init_from_callback and gguf_init_from_buffer (#22341) 2026-05-25 11:33:29 +02:00