vrr/prima.cpp
2
0
Fork 0
mirror of https://github.com/Lizonghang/prima.cpp.git synced 2025-09-10 00:04:34 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'speculative' into dev

Browse source
This commit is contained in:
Li, Zonghang 2025-06-16 13:27:36 +04:00
commit fbbc30c950
6 changed files with 104 additions and 41 deletions
Download patch file Download diff file Expand all files Collapse all files

5
include/llama.h
View file

@ -772,6 +772,11 @@ extern "C" {
LLAMA_API void llama_kv_cache_seq_keep(
struct llama_context * ctx,
llama_seq_id seq_id);
// Notify other nodes to keep only the specified sequence in their KV cache
LLAMA_API void llama_send_kv_cache_seq_keep(
struct llama_context * ctx,
llama_seq_id seq_id);
// Adds relative position "delta" to all tokens that belong to the specified sequence and have positions in [p0, p1)
// If the KV cache is RoPEd, the KV data is updated accordingly: