vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-04 05:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
koboldcpp/examples/llama-eval
History
Georgi Gerganov da46e59cbf
llama-eval : fix crash when answer is None in HTML dump (#25435) 
dict.get("key", default) returns None (not default) when the key
exists but its value is explicitly None. This caused an AttributeError
in _escape_html() when a task errored before grading and answer was
set to None.

Assisted-by: pi:llama.cpp/Qwen3.6-27B
2026-07-08 10:00:03 +03:00
..
llama-eval.py llama-eval : fix crash when answer is None in HTML dump (#25435) 2026-07-08 10:00:03 +03:00
llama-server-simulator.py llama-eval : add per-task summary stats (#23151) 2026-05-19 09:46:05 +03:00
README.md examples : add llama-eval (#21152) 2026-05-12 15:07:00 +03:00
test-simulator.sh examples : add llama-eval (#21152) 2026-05-12 15:07:00 +03:00

README.md

llama-eval

Simple evaluation tool for llama.cpp with support for multiple datasets.

For a full description, usage examples, and sample results, see:

Quick start

# Single server
python3 llama-eval.py \
  --server http://localhost:8033 \
  --model my-model \
  --dataset gsm8k --n_cases 100 \
  --grader-type regex --threads 32

# Multiple servers (comma-separated URLs and thread counts)
python3 llama-eval.py \
  --server http://server1:8033,http://server2:8033 \
  --server-name server1,server2 \
  --threads 16,16 \
  --dataset aime2025 --n_cases 240 \
  --grader-type regex