vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-10 09:04:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

lower topk prefilter token amount to 3k

Browse source
This commit is contained in:
Concedo 2024-10-16 20:39:41 +08:00
parent 80dcfc9db0
commit 7f76425450
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
gpttype_adapter.cpp
View file

@ -1361,8 +1361,8 @@ const std::vector<samplers> & sampler_order, llama_grammar * grammar, float dyna
//dry always first as logits cannot be resorted
sample_dry(n_ctx, dry_penalty_last_n, dry_multiplier, dry_base, dry_allowed_length, dry_sequence_breakers, &candidates_p);
//prefilter to top 5k tokens for improved speed
sample_top_k(&candidates_p, 5000);
//prefilter to top 3k tokens for improved speed
sample_top_k(&candidates_p, 3000);
if (mirostat == 1 || mirostat == 2)
{