koboldcpp/docs
Titaniumtown 6509138622
sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (#26779)
Measured on Arc Pro B70 (Battlemage, Level Zero), llama-bench -r 20, two
interleaved rounds, tg128:

    qwen2.5-3B-Instruct Q4_K_M    154.18 -> 158.53 t/s   +2.8%
    gemma-2-2b-it Q4_K_M          162.45 -> 165.62 t/s   +2.0%

llama-batched-bench on qwen2.5-3B, S_TG by batch size:

      B=1   142.72 -> 147.57 t/s    +3.4%
      B=2   243.72 -> 268.26 t/s   +10.1%
      B=4   359.58 -> 398.02 t/s   +10.7%
      B=8   449.75 -> 505.63 t/s   +12.4%
2026-08-14 02:26:23 -04:00
..
android android: fix missing screenshots for Android.md (#18156) 2025-12-19 09:32:04 +02:00
backend sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (#26779) 2026-08-14 02:26:23 -04:00
development mtmd: support Qwen3-TTS (note: breaking change to llama-tts binary) (#26254) 2026-08-04 17:26:15 +02:00
multimodal docs : fix duplicated "the" in granitevision and model-conversion docs (#23767) 2026-05-27 09:34:06 +02:00
ops sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE (#26568) 2026-08-07 08:22:23 +03:00
android.md docs/android.md: Add dependency libandroid-spawn for building in termux (#21812) 2026-06-22 05:48:31 +02:00
autoparser.md common : fix env names to all have LLAMA_ARG_ prefix (#23778) 2026-05-27 14:52:47 +03:00
build-riscv64-spacemit.md ci : update spacemit toolchain url and enhance curl command (#23642) 2026-05-25 10:43:24 +02:00
build-s390x.md docs: update s390x build docs (#19643) 2026-02-16 00:33:34 +08:00
build.md HIP: remove rocWMMA FlashAttention (#26046) 2026-07-24 17:53:54 +02:00
completions.md readme : refresh (#26280) 2026-07-30 16:14:37 +03:00
docker.md sycl : Improve SYCL doc (#23025) 2026-06-04 08:02:54 +03:00
function-calling.md docs: Update documentation with Granite 4.0/4.1 (#23404) 2026-05-22 20:35:46 +08:00
install.md docs: Adapt conda-forge package name (#26229) 2026-07-28 16:51:20 +02:00
llguidance.md llguidance build fixes for Windows (#11664) 2025-02-14 12:46:08 -08:00
models.md readme : refresh (#26280) 2026-07-30 16:14:37 +03:00
multi-gpu.md Write a readme on Multi-GPU usage in llama.cpp (#22729) 2026-05-07 17:48:40 +02:00
multimodal.md mtmd: refactor preprocessor, add mtmd_image_preproc_out (#24736) 2026-06-18 12:04:39 +02:00
ops.md sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE (#26568) 2026-08-07 08:22:23 +03:00
preset.md common: add system-level config file (#26118) 2026-08-13 00:02:27 +02:00
release.md cmake : introduce semantic versioning (#26839) 2026-08-12 14:15:03 +02:00
speculative.md llama : support multi-output backend sampling (#25532) 2026-08-10 16:58:56 +03:00
xcframework.md readme : refresh (#26280) 2026-07-30 16:14:37 +03:00