Large (>=256KB) JSON request bodies were fully JSON.parse'd and pretty-printed
in the body formatter Web Worker, which then crashed (memory/CPU) on huge object
graphs - surfacing "Body formatter worker failed." and sticking at
"Loading body...". Preview truncation only applied to non-JSON bodies.
Now, in preview mode, an over-length text body is truncated to a plain-text
preview (createLogBodyPreviewText) instead of being parsed/pretty-printed, so the
worker never chokes. Full formatting still runs in "full" mode.
The judge is the real body.text length, not sizeBytes (which may be inflated
metadata and should still render its JSON tree when lightweight).
Fixes#1694
normalizeUsageInputTokens decides whether input_tokens already includes the
cached prefix, and it asked the upstream provider protocol first. Both usage
call sites merge the billing headers and the response body into one snapshot
and then normalize that merge once — but on a translated response the two
sources use different conventions:
- x-gateway-billing-* headers restate the upstream provider's own counters
verbatim, so for an OpenAI-compatible upstream they are cache-inclusive.
- The response body is whatever the gateway emitted. An Anthropic body is
cache-exclusive regardless of what it was translated from.
So an Anthropic response served from an OpenAI-compatible upstream had the
body's already-excluded prefix subtracted a second time. Non-streaming
responses were unaffected by luck: the headers win the merge and are
inclusive, making the subtraction correct. Streaming responses carry no
billing headers at all, leaving the body alone to be over-subtracted and then
clamped by Math.max(0, ...) — on a long cached conversation input_tokens
reports 0 on nearly every turn, and the derived cache ratio is pinned at
100%.
Tag each source with a UsageConventionSource and normalize the two separately
before merging them. Reordering the precedence instead is not sufficient: it
moves the defect onto the non-streaming path, which depends on the headers
being reduced.
Raw-trace updates carry no provider protocol, so their billing headers keep
falling back to the request path; only the body side changes there.
Four SSE response transforms decode every upstream chunk in isolation with
chunk.toString(). A UTF-8 character whose bytes straddle a chunk boundary is
therefore decoded twice as two invalid fragments, and each fragment becomes
U+FFFD. The character is destroyed before the block is ever parsed, so no
downstream code can recover it: the JSON still parses, the event still
validates, and the replacement characters are forwarded to the client.
The pending buffer these transforms already keep does not help. It joins
partial SSE *blocks*, not partial *characters* — by the time the bytes reach
it they have already been decoded and lost.
Affected sites, all in the response chain built in gateway/request/pipeline.ts:
codex-patch-bridge.ts transformSseChunk
codex-multi-agent-bridge.ts transformSseChunk
hosted-web-search/response-transform.ts
hostedWebSearchProtocolSseStream
anthropicHostedWebSearchProtocolSseStream
The correct pattern is already in this repository. anthropic-response-model.ts,
the last transform in that same chain, runs the identical block-splitting loop
over a node:string_decoder StringDecoder, which holds back an incomplete
trailing sequence until the next chunk supplies the rest.
Apply that pattern to the other four. Each transform now owns one decoder for
its lifetime, writes every chunk through it, and drains decoder.end() in flush
so a truncated stream still emits what the decoder was holding.
Any non-ASCII output is affected — CJK, Cyrillic, accented Latin, emoji — and
the apply_patch bridge carries file contents, so a corrupted character there is
written to disk.
Windows implements fsync as FlushFileBuffers, which behaves differently
from POSIX in two ways the raw trace spool relied on.
Flushing a directory handle is rejected outright with EPERM. syncDirectory
only tolerated EINVAL, ENOTSUP and EISDIR, so the durability barrier that
follows every rename threw instead of being skipped.
Flushing a file handle requires that handle to carry write access, and both
file flush sites opened their target read-only, so they also failed with
EPERM.
Together these made every durable write fail on Windows: with
observability.requestLogs enabled the sync handler answered 503
spool_unavailable for every bundle and nothing was ever persisted.
Tolerate EPERM alongside the other directory-flush rejections, and reopen
files as "r+" so their flush stays a real barrier rather than a tolerated
error. The tolerated set is extracted into a predicate so a test can pin
down which codes are skipped and, just as importantly, which ones still
propagate.
Fixes#1635
