Commit graph

1 commit

Author SHA1 Message Date
carffuca
cf03af5d31
feat(web-shell): stream-highlight code blocks and fix fence-language aliases (#5869)
* feat(web-shell): stream-highlight code blocks and fix fence-language aliases

- Highlight assistant code fences live as they stream in via @shikijs/stream,
  replacing the per-token whole-block re-tokenize that flickered between plain
  grey and colored and only settled at the end of the turn.
- Fix fence-language aliases (`ts`/`js`/`py`/... and uppercase like `SQL`) that
  previously rendered unhighlighted because the supported-language gate only
  matched canonical Shiki ids.
- Consolidate the static and streaming paths onto a single lazily-created
  JavaScript-regex-engine highlighter (no WASM, on-demand language loading) and
  hand off without a grey flash; bump shiki ^1 -> ^4 and add @shikijs/stream.

* fix(web-shell): address review feedback on streaming code highlight

- Stabilize the `code` renderer via an IsStreamingContext so the
  CodeBlock/StreamingCodeBlock instances are reused (not remounted) across the
  streaming→settled transition, keeping the no-flash hold effective; drop the
  now-redundant COMPONENTS_STREAMING split.
- Log a warning when the streaming highlighter fails to load, instead of
  failing silently to plain text.
- Export `enqueueSuffix` and add unit tests for its no-op/extend/diverge/throw
  branches, plus a streaming→settled hand-off render test.

* fix(web-shell): harden code highlighter (review round 2)

- Guard highlightToHtmlSync with try/catch so a synchronous tokenization error
  falls back to the async path instead of crashing the render tree.
- Dedupe concurrent loadLanguage calls via a pending-languages map (two callers
  could previously both pass the `has` check and double-load).
- Add unit tests for codeHighlighter: highlightToHtml output, the sync
  cold→warm transition, the 30K-char sync bail-out, and concurrent-load dedup.

* fix(web-shell): guard oversized streaming fences + review cleanups

- Fall back to the plain renderer while streaming once the block or its trailing
  line exceeds a threshold (`exceedsStreamingLimit`), so a large single-line /
  minified fence can't pin the UI via @shikijs/stream re-tokenizing the unstable
  current line on every chunk.
- Drop `shell`/`zsh` from SUPPORTED_LANGUAGES — they were dead entries shadowed
  by the alias map (→ bash).
- Tests: add `__resetForTesting` to drop cross-test ordering coupling; add
  highlighter retry / pending-language cleanup tests, `exceedsStreamingLimit`
  unit tests, an oversized-fence fallback test, and a highlighter-load-failure
  fallback test for StreamingCodeBlock.

* refactor(web-shell): stream-highlight code blocks via synchronous per-chunk re-render (drop @shikijs/stream)

---------

Co-authored-by: 易良 <1204183885@qq.com>
2026-06-27 03:26:18 +00:00