vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-11 17:44:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

batch : add n_used count (#14512)

Browse source 
ggml-ci
This commit is contained in:
Georgi Gerganov 2025-07-04 09:04:59 +03:00 • committed by GitHub
parent 499a8f5a78
commit c79184d2d1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 34 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
src/llama-memory-recurrent.cpp
View file

@ -377,7 +377,8 @@ llama_memory_context_ptr llama_memory_recurrent::init_batch(llama_batch_allocr &
ubatch = balloc.split_equal(n_ubatch);
}
if (ubatch.n_tokens == 0) {
if (balloc.get_n_used() < balloc.get_n_tokens()) {
// failed to find a suitable split
break;
}