vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-11 01:16:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

llama : load MTP tensors only if they are really used (#26296)

Browse source 
* llama : load MTP tensors only if they are really used

* llama : skip loading MTP (if not used) in remaining models that support MTP

---------

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
This commit is contained in:
fairydreaming 2026-07-31 14:57:02 +02:00 • committed by GitHub
parent 6f3c0a790b
commit 82dbc4f017
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 65 additions and 40 deletions
Download patch file Download diff file Expand all files Collapse all files

2
src/llama.cpp
View file

@ -305,7 +305,7 @@ static std::pair<int, llama_model *> llama_model_load(struct gguf_context * meta
const std::string & fname, std::vector<std::string> & splits, FILE * file, llama_model_params & params) {
try {
llama_model_loader ml(metadata, set_tensor_data, set_tensor_data_ud, fname, splits, file, params.load_mode,
params.check_tensors, params.no_alloc, params.kv_overrides, params.tensor_buft_overrides);
params.check_tensors, params.no_alloc, params.load_mtp, params.kv_overrides, params.tensor_buft_overrides);
ml.print_info();
std::unique_ptr<llama_model> model_ptr(llama_model_create(ml, params));