mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-08-21 22:35:48 +00:00
Measured on Arc Pro B70 (Battlemage, Level Zero), llama-bench -r 20, two
interleaved rounds, tg128:
qwen2.5-3B-Instruct Q4_K_M 154.18 -> 158.53 t/s +2.8%
gemma-2-2b-it Q4_K_M 162.45 -> 165.62 t/s +2.0%
llama-batched-bench on qwen2.5-3B, S_TG by batch size:
B=1 142.72 -> 147.57 t/s +3.4%
B=2 243.72 -> 268.26 t/s +10.1%
B=4 359.58 -> 398.02 t/s +10.7%
B=8 449.75 -> 505.63 t/s +12.4%
|
||
|---|---|---|
| .. | ||
| snapdragon | ||
| VirtGPU | ||
| BLIS.md | ||
| CANN.md | ||
| CUDA-FEDORA.md | ||
| ET.md | ||
| OPENCL.md | ||
| OPENVINO.md | ||
| SYCL.md | ||
| VirtGPU.md | ||
| zDNN.md | ||
| ZenDNN.md | ||