server: refactor + correctness fixes for metrics (#26920)

Browse source

* server: refactor metrics

* move most fields to server_slot_stats

* cont

* rm result_timings

* tie stats to batch

* cont

* nits: move place in code

* exclude first generated token

* more accurate batch metrics tracking

* n_predict --> n_gen

* metrics_on_prediction

* metrics_flush_idle

* metrics: seperate cache/processed prompt tokens

* refactor server_task_result_metrics

* add test

* nits

* fix flush before reset()

* cont

* rm dead code

* nits

...

This commit is contained in:

Xuan-Son Nguyen 2026-08-13 10:02:01 +02:00 • committed by GitHub

parent e79e4bf660

commit decaf508bb

No known key found for this signature in database

GPG key ID: B5690EEEBB952194

6 changed files with 813 additions and 475 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

637

tools/server/server-context.cpp

View file

File diff suppressed because it is too large Load diff