vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-11 01:24:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

context : perform output reorder lazily upon access after sync (#14853)

Browse source 
* context : perform output reorder after lazily upon access after sync

ggml-ci

* cont : add TODO
This commit is contained in:
Georgi Gerganov 2025-07-24 16:31:48 +03:00 • committed by GitHub
parent 820de57d4f
commit e4868d16d2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 47 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

2
include/llama.h
View file

@ -956,6 +956,7 @@ extern "C" {
// in the order they have appeared in the batch.
// Rows: number of tokens for which llama_batch.logits[i] != 0
// Cols: n_vocab
// TODO: deprecate in favor of llama_get_logits_ith() (ref: https://github.com/ggml-org/llama.cpp/pull/14853#issuecomment-3113143522)
LLAMA_API float * llama_get_logits(struct llama_context * ctx);
// Logits for the ith token. For positive indices, Equivalent to:
@ -970,6 +971,7 @@ extern "C" {
// in the order they have appeared in the batch.
// shape: [n_outputs*n_embd]
// Otherwise, returns NULL.
// TODO: deprecate in favor of llama_get_embeddings_ith() (ref: https://github.com/ggml-org/llama.cpp/pull/14853#issuecomment-3113143522)
LLAMA_API float * llama_get_embeddings(struct llama_context * ctx);
// Get the embeddings for the ith token. For positive indices, Equivalent to: