vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-15 03:19:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

adjusted down the buf memory allocation now that realloc seems to work

Browse source
This commit is contained in:
Concedo 2023-04-20 17:51:13 +08:00
parent 4605074245
commit 49697d86d8
3 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

2
otherarch/gpt2_v2.cpp
View file

@ -371,7 +371,7 @@ bool gpt2_eval(
const int n_vocab = hparams.n_vocab;
//todo: there is a bug that causes the buffer to oom and I cannot figure it out, hack to increase size for now
static size_t buf_size = 1600u*1024*1024;
static size_t buf_size = 512u*1024*1024;
static void * buf = malloc(buf_size);
if (mem_per_token > 0 && mem_per_token*N*1.6 > buf_size) {