mirror of
https://github.com/MoonshotAI/kimi-code.git
synced 2026-07-10 01:39:25 +00:00
42e37eb898
* feat(timing): split TTFT into api-server and client portions Time-to-first-token previously lumped in-process request building (message serialization, param assembly) together with network + server latency, making it impossible to tell whether a slow turn was the client or the API server. Add an `onRequestSent` hook to kosong's GenerateOptions, fired by every provider immediately before it dispatches the network call. The window from request start to dispatch is attributed to the client; the window from dispatch to the first streamed token is attributed to the API server. The split flows through the step.end / turn.step.completed events (and therefore wire.jsonl) and is surfaced in three places: - KIMI_CODE_DEBUG=1: `TTFT: 2.5s (api 2.4s + client 100ms)` - session log: new `llm response` line with the timing breakdown - vis: firstToken/api + firstToken/client rows and timeline label The split is omitted (total only) when a provider does not report the boundary, preserving backward compatibility. * feat(timing): split the decode window into server vs client time Time-to-first-token now reports a client/server split, but the slow part of a long turn is the decode window (inter-token streaming), which was still a single opaque number. Profiling long sessions showed decode throughput halving over a session's lifetime independent of context size, which the synchronous per-chunk stream pipeline can cause: kosong awaits the host callback for every streamed part, so a loaded main thread throttles how fast tokens are pulled off the wire. Account for this directly in the stream loop: the time awaiting the next part (server + network) versus the time spent processing each part in-process (deep copy, host callback, part merge). The split is reported through onStreamEnd and flows through the step.end / turn.step.completed events (and wire.jsonl) into the same three surfaces as the TTFT split: - KIMI_CODE_DEBUG=1: `TPS: 40.0 tok/s (200 tokens in 5.0s; server 4.6s + client 400ms)` - session log: serverDecodeMs / clientConsumeMs on the `llm response` line - vis: streamDuration/server + streamDuration/client rows and timeline label A large, growing client share confirms host-side throttling; a dominant server share points at the server/connection. The per-chunk accounting is wrapped in try/finally so it stays correct across `continue` and aborts, and is omitted when the stream reports nothing. |
||
|---|---|---|
| .. | ||
| kimi-code | ||
| kimi-web | ||
| vis | ||