Exact

Exact Union RegExp

cogwheel ea82ee164e perf: chat scrolling and markdown rendering (#640) ... * perf: stop shell rebuilds cascading into rows and cut per-flush markdown cost Scrolling: - Give the timeline slivers a delegate with a real shouldRebuild keyed on rowBuilder/entries identity and centerIndex, so ChatPage setState (drag start, keyboard insets, composer resize, pin transitions) rebuilds only the shell, not every mounted row. - Memoize the transcript window, ChatTimelineRenderModel, and rowBuilder by identity in ChatPage; the fresh window list per build also defeated the stable-layout cache's identity fast path. - Replace full MediaQuery.of dependencies with scoped paddingOf/sizeOf. - Raise the streaming cacheExtent from 120px to 600px; the small extent evicted rows that then remounted with a synchronous markdown compile. Markdown: - Memoize buildMarkdownDisplayParts by compiled-document identity; it re-derived one sub-document per block with deep compares on every build. - Add identical() fast paths to CompiledMarkdownDocument and PreparedMarkdownText equality; compare rope segments instead of materializing both sides. - Cache per-message structure-signature fragments by ChatMessage identity instead of rebuilding O(messages x versions) strings per emission. - Gate incremental preparation and the reference-definition strip on one shared line-anchored predicate; a bare "]:" substring no longer forces full re-preparation of the whole message every flush. - Make the streaming split's unsafe-line detection fence-aware and cap freezing at the first raw HTML block instead of keeping the entire document mutable; reuse the fence-close helper instead of compiling a RegExp per fenced block. - Skip the four LaTeX extraction regex passes when content has no $ or backslash; memoize the error-heuristic content scans in the assistant footer; gate profiler map allocations to profiling builds; memoize the code-block line split and render >50k-char code plain. * fix: detect reference definitions past raw HTML blocks in streaming split The single-pass unsafe-line scan returned at the first raw HTML line, so a reference definition appearing after that block was never seen and blocks containing its links could be frozen with the link unresolved. Scan the whole region for definitions and only record the first raw HTML offset as the freeze cap. * fix: distrust fence state after a raw HTML block starts in unsafe-line scan Backtick lines inside raw HTML are content, not fences; an odd count left the fence tracker 'inside' a fence and skipped a later real reference definition, freezing earlier blocks with unresolved links. After the first raw HTML start, check every line for a definition regardless of fence state — at worst more conservative than the whole-document check this replaced. * fix: long-response truncation at completion and follow-ups never arriving Truncation: the streamed buffer is never periodically folded into message state, so /api/chat/completed was built from a stale prefix of the response and the server's echo of that payload truncated the full content when merged back (worst on the HTTP/SSE transport, which reached completion without any terminal flush). Flush the buffer before building the completed payload, and guard the three unguarded overwrite paths (completed echo, replay-gap authoritative recovery, cumulative chat:completion content snapshots) so a strict prefix of already-streamed content is never adopted. Follow-ups: the server emits chat:message:follow_ups only after chat:completion {done:true}, and the per-stream socket subscription is disposed synchronously by that done event, so the streaming handler for follow-ups was unreachable. The passive conversation subscription is the surviving delivery path; apply the pushed payload directly to the target message there instead of relying on a debounced refetch that races the server's own persistence of the suggestions. * fix: address review feedback on follow-ups delivery and splitter - Fall through to the debounced refetch when a pushed follow-ups payload targets a message id not present in local state. - Split the follow-ups envelope parser into a private implementation with a visibleForTesting wrapper, matching file convention. - Detect reference-definition labels containing escaped brackets in the streaming splitter's unsafe-line scan. - Use package:checks in the new follow-ups parser test. * fix: deferred structured-output projections dropping response content The structured-output projector defers full re-projections geometrically (next re-render only at 2x the last projected length) and permanently disables its plain-append fast path once the text contains a backtick, so the visible content can trail the logical content by up to half the response. Two consequences fixed here: - A plain content delta arriving after a deferred projection appended onto the short stale render and flipped structuredOutputIsLatest, which also made the terminal projector finalize bail — permanently dropping the deferred middle of the response on screen and in the persisted echo. appendVisibleAssistantChunk now materializes the full projection (new StructuredOutputStreamingProjector.syncProjectionToLatest) before switching the content basis to plain appends. - handleCompletionDone flushed the buffer before building the completed payload but did not finalize the projector first, so the payload (and the outlet-filter echo derived from it) could carry the stale short render. Also fold the un-flushed streaming buffer into state in _cancelMessageStream (conversation switch / message deletion mid-stream discarded the entire un-synced tail), skipped during provider dispose where state is untouchable. * fix: harden remaining content-adoption paths against divergent server bodies Local and server renders of the same turn wrap reasoning/tool sections in semantic <details> blocks with different attributes (locally injected duration=\"0\" vs the server's real duration), so every raw startsWith/length guard was dead on reasoning turns. Content comparisons now strip rendered semantic details and compare answer bodies: - applyServerContent adopts only when the server's answer body is at least as long as the local one; a snapshot whose raw length grew (long reasoning block) while the answer shrank no longer replaces a complete local answer. - _shouldPreserveLocalAssistantContent (all snapshot adoptions including the reopened-stream reconcile and its buffer rebase) compares stripped bodies. - The completed-echo, replay-gap recovery, and cumulative content-snapshot guards compare stripped bodies, and an echo differing only by details wrappers is a no-op instead of an adoption. Also: - Hermes: a terminal/recovered output that is a strict prefix of the streamed text no longer replaces it (lagging aggregate or incomplete recovery would truncate delivered content). - The local turn echo payload now carries output, files, embeds, usage, sources, statusHistory, followUps, and error: the sync outbox rebuilds the chat blob from these rows and the server merge replaces message objects wholesale, so omitted fields were wiped from the server copy on push. - A stale settled markdown refresh no longer leaves the preparation flag set when nothing newer is queued (indefinite loading skeleton). * fix: address review findings on the unsafe-line scan and echo payload - Remove the backslash overlap in the reference-definition label pattern; the overlapping alternation could backtrack exponentially on long malformed labels, on the UI isolate. - Track <details> bodies opaquely (open/close depth) in the unsafe-line scan, mirroring the block scanner: an unmatched backtick line inside a details body no longer opens a phantom outer fence that hid later reference definitions. - Persist codeExecutions in the local turn-echo payload alongside the other durable server-shape fields. * fix: live-tail freeze/duplication, follow-up persistence, scroll-down jank Live tail (regression from the projection-sync fix): syncProjectionToLatest re-armed the projector's geometric backoff to 2x the full content length while the plain-append transition disabled the append path, so subsequent output snapshots all deferred and the visible tail froze for the rest of the turn. The sync now preserves the backoff threshold. Same-frame handling now also matches the upstream client contract (Chat.svelte): a frame carrying an output snapshot supersedes its own choices delta / content field — Conduit applied the delta first and the snapshot second, duplicating text the snapshot already contained. Follow-ups: pushed suggestions were applied to in-memory state only; the turn echo had been persisted at completion before the event fired, so a conversation switch reloaded the message without them. The passive handler now re-persists the message row after applying the payload. Scroll-down jank: three down-only per-frame costs while returning toward the bottom — the bottom-anchor recompute re-armed a full layout-maintenance pass (row-rect snapshot + pin geometry) on every metrics tick once anchored (now only on anchored-state transitions); pin geometry re-measured three global rects per frame mid-scroll (now skipped until motion settles once reported); and UserScrollNotification(idle) was treated as drag end even though Flutter publishes it at ballistic START, running mode flips and jump-to-latest arming mid-fling (drag end now comes from ScrollEndNotification, which fires at actual rest). * fix: streamed word drops and quote/entity rendering defects Quotes/entities: - Answer text no longer escapes double quotes (element-mode escaping; tags are still neutralized). &quot; escaped into a context the markdown decoder skips — immediately after a backquote, or inside code via the streaming fragment path — surfaced literally on screen. Attribute-mode escaping stays for <details> attribute values. - The plain streaming accumulator was seeded/refilled from the RENDERED (already-escaped) body on reopen/reasoning sync; the next full render escaped it a second time (&amp;quot; decoding once back to a visible &quot;). Plain-content derivation now strips semantic details AND unescapes entities. - Clipboard copy and TTS decode presentation entities back to literal text. API replay deliberately does not (it cannot distinguish model-typed entities from presentation escaping, and the direct bridge has trusted raw replay for fidelity). Missing words: - Whitespace-only deltas were discarded on one transport (trim() guard), gluing words together and losing paragraph breaks. - Whitespace-only semantic text blocks were dropped from full renders and the streaming append delta never re-emits the swallowed prefix — the blank line between a reasoning section and the answer vanished. - Once a backtick/tilde disabled the projector's append path for the turn, geometric backoff left the visible tail up to 50% behind until completion; renders now use a bounded additive step when appends are unavailable. - The SSE parser now mutes same-frame deltas only when the output snapshot parses into renderable blocks, matching the socket path — an output whose items all parse away no longer mutes the delta while rendering nothing. * fix: address review feedback on plain-content whitespace and coverage - The semantic-details strip in plain-content derivation now consumes only the wrapper's own trailing newline instead of \s* plus trim, preserving answer whitespace such as a leading indented code block's indentation. - Regression tests: the additive re-projection schedule for code-bearing streams, and a non-renderable output snapshot not muting the same-frame delta. * fix: match the details parser's exact close token in streaming scans The streaming details trackers accepted '</details >' as a close while the details parser recognizes only the literal '</details>'. A close lookalike inside a streamed details body exited details tracking early, let a body backtick open a phantom outer fence, and hid a valid reference definition after the real close — freezing an earlier reference-style link unresolved. The preparation engine's checkpoint scanner had the same loose pattern and could split prepared content mid-block. Both now match the parser exactly, with a regression test verified to fail against the loose pattern. * fix: reconcile deferred snapshots at terminal finalize and audit lengths finalizeStructuredOutputProjection bailed whenever a plain chunk was the last content-affecting operation; if output snapshots after that chunk had deferred under the re-projection threshold, the deferred tail was dropped from the final content. The finalize now adopts the terminal render unless the accumulated visible text is longer (matching upstream's output-replaces- content contract while preserving delta-only hybrid streams). The done-signal log now records message/rendered/plain lengths so a truncation report can be pinpointed from a single log line: message shorter than rendered points at a lost flush; rendered shorter than plain points at an unrepaired deferred projection. * fix: reconcile terminal projection when plain chunks ended the stream finalizeStructuredOutputProjection bailed whenever a plain chunk was the last content-affecting operation. The terminal snapshot render is authoritative upstream (output replaces content wholesale in Chat.svelte); adopt it unless the accumulated visible text is longer, preserving delta-only hybrid streams. The done-signal log now records message/rendered/plain lengths so a truncation report can be pinpointed from one log line. * fix: render output[] in poll recovery when persisted content is empty OWUI 0.11 never persists a flat content string for a normal completion — the durable body is the output[] item array, so a reasoning turn's raw content is ''. pollServerForMessage ignored output[] entirely: whenever the live socket missed the final frames (buffer caps on long reasoning streams, reconnects), every recovery path polled the server, extracted an empty string, adopted nothing, saw done=true, and finished the turn with the partial local text — permanent tail truncation, reasoning models only. Recovery now renders output[] with the same renderer the snapshot parser uses when flat content is empty. Regression test verified to fail without the fix. * revert: drop the speculative terminal-finalize reconciliation The longer-wins adoption added in 349bb708 was based on a wrong premise: structuredOutputIsLatest is set back to true by every output snapshot, applied or deferred, so the bail it targeted only holds when a plain chunk was the very last content-affecting operation — where the accumulated visible text is already the right terminal value (syncProjectionToLatest keeps it complete). Its regression test passes with and without the change, and adopting the terminal snapshot in that narrow case could drop equal-length delta-only text. Restore the original bail, with a comment explaining why it is correct. * fix: keep the settle transition extent-neutral Measured the streaming-to-settled swap (typing-indicator footer 16+28+4 vs in-card action row 16+32): it is extent-neutral by design for plain, reasoning, and completed-status turns. The residual jump came from status rows whose updates never reported done — the settle filter emptied the list and dropped the entire row (~30px shift on the bottom-anchored viewport), also losing the only description of what the turn did. The last update now stays visible (without the pending spinner) when filtering would otherwise empty the row. A new widget test measures both sides of the swap and asserts extent-neutrality for the plain, scaled, versioned, pending-status, and reasoning cases. Settle-only additions (sources row, a text-scaled version chip) legitimately grow the card; an AnimatedSize wrapper was tried and reverted — RenderAnimatedSize asserts when mutated during this tree's layout pass. * fix: count inline nested details opens like the parser; tighten projector test The streaming split scanner counted details opens with the line-anchored pattern, but the details parser counts complete opening tags anywhere in a line. A nested inline open inside a streamed details body made the first closing tag exit scanner details mode one level early, so a body backtick poisoned fence state and hid later reference definitions from the fallback scan. Depth counting now uses the parser's complete-tag pattern (still crediting a line-leading partial tag so unterminated opens stay mutable); block entry detection stays line-anchored, matching the parser. Also assert the exact terminal projection content in the additive-schedule projector test instead of suffix/length checks that a truncated result could satisfy. * fix: address thermo audit and outside-diff review findings - Persist local-echo and direct/Hermes message payloads in the server shape: citation-shaped sources, snake_case code_executions via the shared converters, and null-stripped files. Client-model shapes broke the OpenWebUI web client's code-execution panels and citations for any chat synced from Conduit. The converters move from api_service.dart to core/utils/openwebui_message_payload.dart, and localEchoRowForMessage is now top-level with a test pinning payload completeness and shape. - Bail out of syncProjectionToLatest when the projector never owned the visible basis (observe-only path): materializing the snapshot there shrank visible content that was deliberately kept as a superset. - Copy user messages verbatim: the assistant clipboard sanitizer decoded entities the user actually typed and stripped definition-shaped lines. - Route replaceVisibleAssistantContent's fallback through initialPlainStreamingContent so cumulative content frames cannot leak details wrappers into the plain accumulator and re-escape later. - Stop treating definition-shaped lines inside a details body as reference definitions in the unsafe-line scan: the parser lifts details bodies into body_markdown compiled as its own document, so they cannot couple frozen and mutable segments (CodeRabbit outside-diff finding). - Deduplicate the semantic-details strip and stale-prefix guards into core/utils/semantic_details.dart; five divergent regex copies and five hand-rolled prefix checks now share one definition. - Extract the follow-ups socket-event parser to features/chat/utils/follow_ups_socket_event.dart and drop the ForTesting shims. - Drop the dead streaming parameter on the chat cache-extent helper, hoist hot-loop regexes in the raw-boundary checkpoint scan, re-resolve the message index before the passive follow-ups re-persist. * style: complete the dart format migration repo-wide The Dart 3.13 formatter had been applied piecemeal to files this branch touched, leaving the repo half-migrated and smearing mechanical churn through functional diffs. One-shot migration of the remaining 214 files; no semantic changes. * fix: count details depth on the raw line, not the dedented candidate The details parser counts open/close tags on the raw line regardless of indentation inside a block, but both scanner depth-tracking sites routed counting through the block-starter candidate, which is null for lines indented four or more columns. An indented nested open was silently dropped, so the first close exited scanner details mode a level early and a body backtick opened a phantom fence that hid later reference definitions. Depth now counts on the raw line (the partial-open carve-out dedents before its anchored check so an indented incomplete tag still holds the block open). * fix: count only complete details tags; keep incomplete entry tags mutable Crediting a partial line-leading <details as an open (added so an unterminated streamed entry tag would not freeze) left scanner depth permanently stale when a body contained a literal <details that never completed: the parser counts only complete tags, so it closed the block while the scanner stayed inside it and swallowed the document-level reference definition after the block. Depth counting is now parser-exact (complete tags only), and the unterminated-entry case is handled structurally: an entry line whose tag has no closing > keeps the tail mutable instead of faking a depth of one.		2026-08-18 01:20:40 +05:30
..
release_notes_validator_test.dart	perf: chat scrolling and markdown rendering (#640)	2026-08-18 01:20:40 +05:30