fix compute buffer estimate: reserve 200 MiB VRAM to avoid potential OOM

This commit is contained in:
Li, Zonghang 2025-06-24 20:39:49 +04:00
parent 4dde8458cf
commit 72701ae872

View file

@ -22013,8 +22013,8 @@ void llama_model_compute_buf_size(
// context GPU memory usage, i.e. the initial memory cost of creating a CUDA context,
// even before you launch any kernels or allocate your own buffers.
// this value may vary by GPU and CUDA version, but it's lower than 400 MiB in most cases,
// another 100 MiB is used to prevent accidental OOM.
*gpu_buf += 500 * 1024 * 1024;
// another 200 MiB is used to prevent accidental OOM.
*gpu_buf += 600 * 1024 * 1024;
}
}