vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-05-05 23:41:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

metal : FA support F32 K and V and head size = 32 (#16531)

Browse source 
* metal : FA support F32 K and V and head size = 32

* graph : remove obsolete comment [no ci]
This commit is contained in:
Georgi Gerganov 2025-10-13 23:07:57 +03:00 • committed by GitHub
parent e38b7c6e9e
commit e60f241eac
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 106 additions and 52 deletions
Download patch file Download diff file Expand all files Collapse all files

1
src/llama-graph.cpp
View file

@ -1323,7 +1323,6 @@ ggml_tensor * llm_graph_context::build_attn_mha(
ggml_tensor * cur;
// TODO: replace hardcoded padding with ggml-provided padding
if (cparams.flash_attn && kq_b == nullptr) {
GGML_ASSERT(kq_b == nullptr && "Flash attention does not support KQ bias yet");