koboldcpp/ggml
michaeltrabalka-tech a7cd2f0e98
vulkan: add TQ2_0 (ternary) support (#25850)
* vulkan: TQ2_0 (ternary) support — dequant + dedicated mul_mat_vec + matmul via dequant_funcs

First Vulkan ternary type in ggml. Correctness: OM-125m TQ2_0 vs F16 top-12
logprobs identical to 4 decimals fully offloaded (float dequant path, no Q8_K
activation quant). Speed at 125m ~= F16 (overhead-bound at this scale); the
bandwidth win targets larger BitNet SKUs. MMQ/int-dot path intentionally not
wired yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* tests: enable TQ2_0 in backend-ops type lists

Vulkan now implements TQ2_0 (dequant, mul_mat_vec, mul_mm, get_rows); backends
without support skip via not-supported as usual. TQ1_0 stays disabled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Michael Trabalka <michael.trabalka@sqv.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 08:07:23 +03:00
..
cmake ggml : Parallelize quant LUT init (#23595) 2026-05-25 10:15:46 +03:00
include llama: add default load-mode auto, which avoids mmap on iGPUs (#26081) 2026-08-11 09:20:46 +03:00
src vulkan: add TQ2_0 (ternary) support (#25850) 2026-08-12 08:07:23 +03:00
.gitignore vulkan : cmake integration (#8119) 2024-07-13 18:12:39 +02:00
CMakeLists.txt ggml : bump version to 0.19.0 (ggml/1581) 2026-08-07 17:11:25 +03:00