vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-11 09:25:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

llama : load MTP tensors only if they are really used (#26296)

Browse source 
* llama : load MTP tensors only if they are really used

* llama : skip loading MTP (if not used) in remaining models that support MTP

---------

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
This commit is contained in:
fairydreaming 2026-07-31 14:57:02 +02:00 • committed by GitHub
parent 6f3c0a790b
commit 82dbc4f017
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 65 additions and 40 deletions
Download patch file Download diff file Expand all files Collapse all files

1
common/common.cpp
View file

@ -1620,6 +1620,7 @@ struct llama_model_params common_model_params_to_llama(common_params & params) {
mparams.progress_callback = params.load_progress_callback;
mparams.progress_callback_user_data = params.load_progress_callback_user_data;
mparams.no_alloc = params.no_alloc;
mparams.load_mtp = std::find(params.speculative.types.begin(), params.speculative.types.end(), COMMON_SPECULATIVE_TYPE_DRAFT_MTP) != params.speculative.types.end();
return mparams;
}