mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-09 15:58:41 +00:00
9a966adf51
* Studio: trim serving-log noise and surface llama-server engine stats Studio prints one structured line per HTTP request, so the SPA's polling and per-invalidation fan-out bury the lines that matter. - Dedup identical successful GETs within a short window (default 300ms, UNSLOTH_STUDIO_ACCESS_LOG_DEDUP_MS) so a burst logs once. The dedup key includes the query string, so distinct query-driven GETs are not collapsed. Runs after the response is sent, so it adds no request latency; mutations, non-2xx, and loading polls are untouched. - Collapse pure-liveness polls (/api/health, /api/auth/status, /api/inference/status, /api/inference/monitor) to a longer heartbeat (default 10s, UNSLOTH_STUDIO_ACCESS_LOG_POLL_DEDUP_MS). The API monitor console polls /monitor every 1.5s while open. - Translate llama-server's Prometheus /metrics into a periodic vLLM-style engine_stats line (generation/prompt throughput and requests in flight) from a daemon poller, gated on UNSLOTH_STUDIO_ENGINE_STATS. Throughput uses llama-server's predicted_tokens_seconds / prompt_tokens_seconds gauges, with a tokens_predicted_total / prompt_tokens_total counter-delta fallback; it does not use n_decode_total (which counts llama_decode() calls, not tokens). No KV field is emitted, since llama.cpp does not expose kv_cache_usage_ratio. --metrics is added only when probe_server_capabilities reports the binary supports it, so older/custom binaries still load. The poller keeps retrying through transient scrape failures (stop() drives shutdown) and a malformed sample cannot crash its thread. - api_monitor.append_reply: once the preview cap is reached, skip the per-chunk re-concat (avoids O(n^2) on long generations) while still recording the "..." truncation marker for a reply that lands exactly on the cap. - unsloth studio --verbose and unsloth studio run --verbose both restore every per-request log; --verbose before a subcommand is rejected with guidance (matching --secure / --parallel). run --verbose still forwards --log-verbose to llama-server, preserving the pre-existing pass-through verbosity. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| .gitkeep | ||
| __init__.py | ||
| config.py | ||
| handlers.py | ||