mirror of
https://github.com/cogwheel0/conduit.git
synced 2026-08-28 13:03:31 +00:00
* perf: stop shell rebuilds cascading into rows and cut per-flush markdown cost
Scrolling:
- Give the timeline slivers a delegate with a real shouldRebuild keyed on
rowBuilder/entries identity and centerIndex, so ChatPage setState (drag
start, keyboard insets, composer resize, pin transitions) rebuilds only
the shell, not every mounted row.
- Memoize the transcript window, ChatTimelineRenderModel, and rowBuilder by
identity in ChatPage; the fresh window list per build also defeated the
stable-layout cache's identity fast path.
- Replace full MediaQuery.of dependencies with scoped paddingOf/sizeOf.
- Raise the streaming cacheExtent from 120px to 600px; the small extent
evicted rows that then remounted with a synchronous markdown compile.
Markdown:
- Memoize buildMarkdownDisplayParts by compiled-document identity; it
re-derived one sub-document per block with deep compares on every build.
- Add identical() fast paths to CompiledMarkdownDocument and
PreparedMarkdownText equality; compare rope segments instead of
materializing both sides.
- Cache per-message structure-signature fragments by ChatMessage identity
instead of rebuilding O(messages x versions) strings per emission.
- Gate incremental preparation and the reference-definition strip on one
shared line-anchored predicate; a bare "]:" substring no longer forces
full re-preparation of the whole message every flush.
- Make the streaming split's unsafe-line detection fence-aware and cap
freezing at the first raw HTML block instead of keeping the entire
document mutable; reuse the fence-close helper instead of compiling a
RegExp per fenced block.
- Skip the four LaTeX extraction regex passes when content has no $ or
backslash; memoize the error-heuristic content scans in the assistant
footer; gate profiler map allocations to profiling builds; memoize the
code-block line split and render >50k-char code plain.
* fix: detect reference definitions past raw HTML blocks in streaming split
The single-pass unsafe-line scan returned at the first raw HTML line, so a
reference definition appearing after that block was never seen and blocks
containing its links could be frozen with the link unresolved. Scan the whole
region for definitions and only record the first raw HTML offset as the
freeze cap.
* fix: distrust fence state after a raw HTML block starts in unsafe-line scan
Backtick lines inside raw HTML are content, not fences; an odd count left the
fence tracker 'inside' a fence and skipped a later real reference definition,
freezing earlier blocks with unresolved links. After the first raw HTML start,
check every line for a definition regardless of fence state — at worst more
conservative than the whole-document check this replaced.
* fix: long-response truncation at completion and follow-ups never arriving
Truncation: the streamed buffer is never periodically folded into message
state, so /api/chat/completed was built from a stale prefix of the response
and the server's echo of that payload truncated the full content when merged
back (worst on the HTTP/SSE transport, which reached completion without any
terminal flush). Flush the buffer before building the completed payload, and
guard the three unguarded overwrite paths (completed echo, replay-gap
authoritative recovery, cumulative chat:completion content snapshots) so a
strict prefix of already-streamed content is never adopted.
Follow-ups: the server emits chat:message:follow_ups only after
chat:completion {done:true}, and the per-stream socket subscription is
disposed synchronously by that done event, so the streaming handler for
follow-ups was unreachable. The passive conversation subscription is the
surviving delivery path; apply the pushed payload directly to the target
message there instead of relying on a debounced refetch that races the
server's own persistence of the suggestions.
* fix: address review feedback on follow-ups delivery and splitter
- Fall through to the debounced refetch when a pushed follow-ups payload
targets a message id not present in local state.
- Split the follow-ups envelope parser into a private implementation with a
visibleForTesting wrapper, matching file convention.
- Detect reference-definition labels containing escaped brackets in the
streaming splitter's unsafe-line scan.
- Use package:checks in the new follow-ups parser test.
* fix: deferred structured-output projections dropping response content
The structured-output projector defers full re-projections geometrically
(next re-render only at 2x the last projected length) and permanently
disables its plain-append fast path once the text contains a backtick, so
the visible content can trail the logical content by up to half the
response. Two consequences fixed here:
- A plain content delta arriving after a deferred projection appended onto
the short stale render and flipped structuredOutputIsLatest, which also
made the terminal projector finalize bail — permanently dropping the
deferred middle of the response on screen and in the persisted echo.
appendVisibleAssistantChunk now materializes the full projection (new
StructuredOutputStreamingProjector.syncProjectionToLatest) before
switching the content basis to plain appends.
- handleCompletionDone flushed the buffer before building the completed
payload but did not finalize the projector first, so the payload (and the
outlet-filter echo derived from it) could carry the stale short render.
Also fold the un-flushed streaming buffer into state in _cancelMessageStream
(conversation switch / message deletion mid-stream discarded the entire
un-synced tail), skipped during provider dispose where state is untouchable.
* fix: harden remaining content-adoption paths against divergent server bodies
Local and server renders of the same turn wrap reasoning/tool sections in
semantic <details> blocks with different attributes (locally injected
duration=\"0\" vs the server's real duration), so every raw startsWith/length
guard was dead on reasoning turns. Content comparisons now strip rendered
semantic details and compare answer bodies:
- applyServerContent adopts only when the server's answer body is at least
as long as the local one; a snapshot whose raw length grew (long reasoning
block) while the answer shrank no longer replaces a complete local answer.
- _shouldPreserveLocalAssistantContent (all snapshot adoptions including the
reopened-stream reconcile and its buffer rebase) compares stripped bodies.
- The completed-echo, replay-gap recovery, and cumulative content-snapshot
guards compare stripped bodies, and an echo differing only by details
wrappers is a no-op instead of an adoption.
Also:
- Hermes: a terminal/recovered output that is a strict prefix of the
streamed text no longer replaces it (lagging aggregate or incomplete
recovery would truncate delivered content).
- The local turn echo payload now carries output, files, embeds, usage,
sources, statusHistory, followUps, and error: the sync outbox rebuilds the
chat blob from these rows and the server merge replaces message objects
wholesale, so omitted fields were wiped from the server copy on push.
- A stale settled markdown refresh no longer leaves the preparation flag set
when nothing newer is queued (indefinite loading skeleton).
* fix: address review findings on the unsafe-line scan and echo payload
- Remove the backslash overlap in the reference-definition label pattern;
the overlapping alternation could backtrack exponentially on long
malformed labels, on the UI isolate.
- Track <details> bodies opaquely (open/close depth) in the unsafe-line
scan, mirroring the block scanner: an unmatched backtick line inside a
details body no longer opens a phantom outer fence that hid later
reference definitions.
- Persist codeExecutions in the local turn-echo payload alongside the other
durable server-shape fields.
* fix: live-tail freeze/duplication, follow-up persistence, scroll-down jank
Live tail (regression from the projection-sync fix): syncProjectionToLatest
re-armed the projector's geometric backoff to 2x the full content length
while the plain-append transition disabled the append path, so subsequent
output snapshots all deferred and the visible tail froze for the rest of the
turn. The sync now preserves the backoff threshold. Same-frame handling now
also matches the upstream client contract (Chat.svelte): a frame carrying an
output snapshot supersedes its own choices delta / content field — Conduit
applied the delta first and the snapshot second, duplicating text the
snapshot already contained.
Follow-ups: pushed suggestions were applied to in-memory state only; the
turn echo had been persisted at completion before the event fired, so a
conversation switch reloaded the message without them. The passive handler
now re-persists the message row after applying the payload.
Scroll-down jank: three down-only per-frame costs while returning toward the
bottom — the bottom-anchor recompute re-armed a full layout-maintenance pass
(row-rect snapshot + pin geometry) on every metrics tick once anchored (now
only on anchored-state transitions); pin geometry re-measured three global
rects per frame mid-scroll (now skipped until motion settles once reported);
and UserScrollNotification(idle) was treated as drag end even though Flutter
publishes it at ballistic START, running mode flips and jump-to-latest
arming mid-fling (drag end now comes from ScrollEndNotification, which fires
at actual rest).
* fix: streamed word drops and quote/entity rendering defects
Quotes/entities:
- Answer text no longer escapes double quotes (element-mode escaping; tags
are still neutralized). " escaped into a context the markdown decoder
skips — immediately after a backquote, or inside code via the streaming
fragment path — surfaced literally on screen. Attribute-mode escaping
stays for <details> attribute values.
- The plain streaming accumulator was seeded/refilled from the RENDERED
(already-escaped) body on reopen/reasoning sync; the next full render
escaped it a second time (&quot; decoding once back to a visible
"). Plain-content derivation now strips semantic details AND
unescapes entities.
- Clipboard copy and TTS decode presentation entities back to literal text.
API replay deliberately does not (it cannot distinguish model-typed
entities from presentation escaping, and the direct bridge has trusted
raw replay for fidelity).
Missing words:
- Whitespace-only deltas were discarded on one transport (trim() guard),
gluing words together and losing paragraph breaks.
- Whitespace-only semantic text blocks were dropped from full renders and
the streaming append delta never re-emits the swallowed prefix — the
blank line between a reasoning section and the answer vanished.
- Once a backtick/tilde disabled the projector's append path for the turn,
geometric backoff left the visible tail up to 50% behind until
completion; renders now use a bounded additive step when appends are
unavailable.
- The SSE parser now mutes same-frame deltas only when the output snapshot
parses into renderable blocks, matching the socket path — an output whose
items all parse away no longer mutes the delta while rendering nothing.
* fix: address review feedback on plain-content whitespace and coverage
- The semantic-details strip in plain-content derivation now consumes only
the wrapper's own trailing newline instead of \s* plus trim, preserving
answer whitespace such as a leading indented code block's indentation.
- Regression tests: the additive re-projection schedule for code-bearing
streams, and a non-renderable output snapshot not muting the same-frame
delta.
* fix: match the details parser's exact close token in streaming scans
The streaming details trackers accepted '</details >' as a close while the
details parser recognizes only the literal '</details>'. A close lookalike
inside a streamed details body exited details tracking early, let a body
backtick open a phantom outer fence, and hid a valid reference definition
after the real close — freezing an earlier reference-style link unresolved.
The preparation engine's checkpoint scanner had the same loose pattern and
could split prepared content mid-block. Both now match the parser exactly,
with a regression test verified to fail against the loose pattern.
* fix: reconcile deferred snapshots at terminal finalize and audit lengths
finalizeStructuredOutputProjection bailed whenever a plain chunk was the
last content-affecting operation; if output snapshots after that chunk had
deferred under the re-projection threshold, the deferred tail was dropped
from the final content. The finalize now adopts the terminal render unless
the accumulated visible text is longer (matching upstream's output-replaces-
content contract while preserving delta-only hybrid streams).
The done-signal log now records message/rendered/plain lengths so a
truncation report can be pinpointed from a single log line: message shorter
than rendered points at a lost flush; rendered shorter than plain points at
an unrepaired deferred projection.
* fix: reconcile terminal projection when plain chunks ended the stream
finalizeStructuredOutputProjection bailed whenever a plain chunk was the
last content-affecting operation. The terminal snapshot render is
authoritative upstream (output replaces content wholesale in Chat.svelte);
adopt it unless the accumulated visible text is longer, preserving
delta-only hybrid streams.
The done-signal log now records message/rendered/plain lengths so a
truncation report can be pinpointed from one log line.
* fix: render output[] in poll recovery when persisted content is empty
OWUI 0.11 never persists a flat content string for a normal completion —
the durable body is the output[] item array, so a reasoning turn's raw
content is ''. pollServerForMessage ignored output[] entirely: whenever the
live socket missed the final frames (buffer caps on long reasoning
streams, reconnects), every recovery path polled the server, extracted an
empty string, adopted nothing, saw done=true, and finished the turn with
the partial local text — permanent tail truncation, reasoning models only.
Recovery now renders output[] with the same renderer the snapshot parser
uses when flat content is empty. Regression test verified to fail without
the fix.
* revert: drop the speculative terminal-finalize reconciliation
The longer-wins adoption added in 349bb708 was based on a wrong premise:
structuredOutputIsLatest is set back to true by every output snapshot,
applied or deferred, so the bail it targeted only holds when a plain chunk
was the very last content-affecting operation — where the accumulated
visible text is already the right terminal value (syncProjectionToLatest
keeps it complete). Its regression test passes with and without the change,
and adopting the terminal snapshot in that narrow case could drop
equal-length delta-only text. Restore the original bail, with a comment
explaining why it is correct.
* fix: keep the settle transition extent-neutral
Measured the streaming-to-settled swap (typing-indicator footer 16+28+4 vs
in-card action row 16+32): it is extent-neutral by design for plain,
reasoning, and completed-status turns. The residual jump came from status
rows whose updates never reported done — the settle filter emptied the list
and dropped the entire row (~30px shift on the bottom-anchored viewport),
also losing the only description of what the turn did. The last update now
stays visible (without the pending spinner) when filtering would otherwise
empty the row.
A new widget test measures both sides of the swap and asserts
extent-neutrality for the plain, scaled, versioned, pending-status, and
reasoning cases. Settle-only additions (sources row, a text-scaled version
chip) legitimately grow the card; an AnimatedSize wrapper was tried and
reverted — RenderAnimatedSize asserts when mutated during this tree's
layout pass.
* fix: count inline nested details opens like the parser; tighten projector test
The streaming split scanner counted details opens with the line-anchored
pattern, but the details parser counts complete opening tags anywhere in a
line. A nested inline open inside a streamed details body made the first
closing tag exit scanner details mode one level early, so a body backtick
poisoned fence state and hid later reference definitions from the fallback
scan. Depth counting now uses the parser's complete-tag pattern (still
crediting a line-leading partial tag so unterminated opens stay mutable);
block entry detection stays line-anchored, matching the parser.
Also assert the exact terminal projection content in the additive-schedule
projector test instead of suffix/length checks that a truncated result
could satisfy.
* fix: address thermo audit and outside-diff review findings
- Persist local-echo and direct/Hermes message payloads in the server
shape: citation-shaped sources, snake_case code_executions via the
shared converters, and null-stripped files. Client-model shapes broke
the OpenWebUI web client's code-execution panels and citations for any
chat synced from Conduit. The converters move from api_service.dart to
core/utils/openwebui_message_payload.dart, and localEchoRowForMessage
is now top-level with a test pinning payload completeness and shape.
- Bail out of syncProjectionToLatest when the projector never owned the
visible basis (observe-only path): materializing the snapshot there
shrank visible content that was deliberately kept as a superset.
- Copy user messages verbatim: the assistant clipboard sanitizer decoded
entities the user actually typed and stripped definition-shaped lines.
- Route replaceVisibleAssistantContent's fallback through
initialPlainStreamingContent so cumulative content frames cannot leak
details wrappers into the plain accumulator and re-escape later.
- Stop treating definition-shaped lines inside a details body as
reference definitions in the unsafe-line scan: the parser lifts details
bodies into body_markdown compiled as its own document, so they cannot
couple frozen and mutable segments (CodeRabbit outside-diff finding).
- Deduplicate the semantic-details strip and stale-prefix guards into
core/utils/semantic_details.dart; five divergent regex copies and five
hand-rolled prefix checks now share one definition.
- Extract the follow-ups socket-event parser to
features/chat/utils/follow_ups_socket_event.dart and drop the
ForTesting shims.
- Drop the dead streaming parameter on the chat cache-extent helper,
hoist hot-loop regexes in the raw-boundary checkpoint scan, re-resolve
the message index before the passive follow-ups re-persist.
* style: complete the dart format migration repo-wide
The Dart 3.13 formatter had been applied piecemeal to files this branch
touched, leaving the repo half-migrated and smearing mechanical churn
through functional diffs. One-shot migration of the remaining 214 files;
no semantic changes.
* fix: count details depth on the raw line, not the dedented candidate
The details parser counts open/close tags on the raw line regardless of
indentation inside a block, but both scanner depth-tracking sites routed
counting through the block-starter candidate, which is null for lines
indented four or more columns. An indented nested open was silently
dropped, so the first close exited scanner details mode a level early
and a body backtick opened a phantom fence that hid later reference
definitions. Depth now counts on the raw line (the partial-open carve-out
dedents before its anchored check so an indented incomplete tag still
holds the block open).
* fix: count only complete details tags; keep incomplete entry tags mutable
Crediting a partial line-leading <details as an open (added so an
unterminated streamed entry tag would not freeze) left scanner depth
permanently stale when a body contained a literal <details that never
completed: the parser counts only complete tags, so it closed the block
while the scanner stayed inside it and swallowed the document-level
reference definition after the block. Depth counting is now parser-exact
(complete tags only), and the unterminated-entry case is handled
structurally: an entry line whose tag has no closing > keeps the tail
mutable instead of faking a depth of one.
806 lines
29 KiB
Dart
806 lines
29 KiB
Dart
import 'dart:async';
|
|
import 'dart:io';
|
|
|
|
import 'package:checks/checks.dart';
|
|
import 'package:conduit/core/database/app_database.dart';
|
|
import 'package:conduit/core/database/database_manager.dart';
|
|
import 'package:conduit/core/models/server_config.dart';
|
|
import 'package:drift/native.dart';
|
|
import 'package:flutter_test/flutter_test.dart';
|
|
import 'package:path/path.dart' as p;
|
|
|
|
import '../../support/gated_close_database.dart';
|
|
|
|
ServerConfig _server(String id) =>
|
|
ServerConfig(id: id, name: 'Server $id', url: 'https://$id.example');
|
|
|
|
void main() {
|
|
late Directory tempDir;
|
|
late List<String> openedFileNames;
|
|
late DatabaseManager manager;
|
|
|
|
/// Mirrors drift_flutter's `driftDatabase(name:)` location:
|
|
/// `<directory>/<name>.sqlite`, but against a temp dir and without
|
|
/// platform channels.
|
|
File fileFor(String fileName) =>
|
|
File(p.join(tempDir.path, '$fileName.sqlite'));
|
|
|
|
setUp(() {
|
|
tempDir = Directory.systemTemp.createTempSync('conduit_db_test');
|
|
openedFileNames = [];
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
openedFileNames.add(fileName);
|
|
return AppDatabase(NativeDatabase(fileFor(fileName)));
|
|
},
|
|
);
|
|
});
|
|
|
|
tearDown(() async {
|
|
await manager.closeActive();
|
|
if (tempDir.existsSync()) {
|
|
tempDir.deleteSync(recursive: true);
|
|
}
|
|
});
|
|
|
|
group('openFor', () {
|
|
test('returns the cached instance for the same server id', () async {
|
|
final first = manager.openFor(_server('alpha'));
|
|
final second = manager.openFor(_server('alpha'));
|
|
check(identical(first, second)).isTrue();
|
|
check(openedFileNames.length).equals(1);
|
|
});
|
|
|
|
test('switching servers closes the previous database', () async {
|
|
final first = manager.openFor(_server('alpha'));
|
|
// Force the lazy executor open so close() has something to tear down.
|
|
await first.customSelect('SELECT 1').get();
|
|
|
|
final second = manager.openFor(_server('beta'));
|
|
check(identical(first, second)).isFalse();
|
|
|
|
// The close is fire-and-forget; poll until the old database refuses
|
|
// work.
|
|
await _waitForClosed(first);
|
|
// The new database stays usable.
|
|
check((await second.customSelect('SELECT 1 AS one').get())).isNotEmpty();
|
|
});
|
|
|
|
test('distinct servers map to distinct database files', () async {
|
|
final first = manager.openFor(_server('alpha'));
|
|
await first.customSelect('SELECT 1').get();
|
|
final second = manager.openFor(_server('beta'));
|
|
await second.customSelect('SELECT 1').get();
|
|
|
|
check(openedFileNames.toSet().length).equals(2);
|
|
check(fileFor(DatabaseManager.fileNameFor('alpha')).existsSync())
|
|
.isTrue();
|
|
check(fileFor(DatabaseManager.fileNameFor('beta')).existsSync()).isTrue();
|
|
});
|
|
|
|
test(
|
|
'rapid switch-back defers until close before opening a new executor',
|
|
() async {
|
|
await manager.closeActive();
|
|
final databases = <String, List<GatedCloseDatabase>>{};
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
final database = GatedCloseDatabase(
|
|
NativeDatabase(fileFor(fileName)),
|
|
)..failClose = false;
|
|
databases.putIfAbsent(fileName, () => []).add(database);
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final alphaFile = DatabaseManager.fileNameFor('alpha');
|
|
final originalAlpha = manager.openFor(_server('alpha'));
|
|
await originalAlpha.customSelect('SELECT 1').get();
|
|
final alphaCloseGate = Completer<void>();
|
|
databases[alphaFile]!.single.closeGate = alphaCloseGate;
|
|
|
|
manager.openFor(_server('beta'));
|
|
await _waitForCloseAttempts(databases[alphaFile]!.single, 1);
|
|
|
|
final deferred = manager.openForIfReady(_server('alpha'));
|
|
check(deferred is DatabaseOpenDeferred).isTrue();
|
|
check(databases[alphaFile]!.length).equals(1);
|
|
|
|
alphaCloseGate.complete();
|
|
await (deferred as DatabaseOpenDeferred).retryAfter;
|
|
|
|
final reopened = manager.openForIfReady(_server('alpha'));
|
|
check(reopened is DatabaseOpenReady).isTrue();
|
|
final reopenedAlpha = (reopened as DatabaseOpenReady).database;
|
|
check(identical(reopenedAlpha, originalAlpha)).isFalse();
|
|
check(databases[alphaFile]!.length).equals(2);
|
|
check((await reopenedAlpha.customSelect('SELECT 1').get()))
|
|
.isNotEmpty();
|
|
},
|
|
);
|
|
});
|
|
|
|
group('lifetime leases', () {
|
|
test(
|
|
'managed provenance survives physical close for detached lease guards',
|
|
() async {
|
|
await manager.closeActive();
|
|
late GatedCloseDatabase database;
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
database = GatedCloseDatabase(NativeDatabase(fileFor(fileName)))
|
|
..failClose = false;
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final opened = manager.openFor(_server('alpha'));
|
|
await opened.customSelect('SELECT 1').get();
|
|
check(manager.serverIdForDatabase(opened)).equals('alpha');
|
|
final closeGate = Completer<void>();
|
|
addTearDown(() {
|
|
if (!closeGate.isCompleted) closeGate.complete();
|
|
});
|
|
database.closeGate = closeGate;
|
|
|
|
final close = manager.closeActive();
|
|
await _waitForCloseAttempts(database, 1);
|
|
|
|
// Operational ownership is already revoked, so a late callback cannot
|
|
// acquire a lease. Stable provenance must nevertheless remain visible
|
|
// or callers could mistake this executor for an unmanaged test seam
|
|
// and issue SQL while/after it closes.
|
|
check(manager.tryAcquireLease(opened)).isNull();
|
|
check(manager.serverIdForDatabase(opened)).equals('alpha');
|
|
|
|
closeGate.complete();
|
|
await close;
|
|
check(manager.tryAcquireLease(opened)).isNull();
|
|
check(manager.serverIdForDatabase(opened)).equals('alpha');
|
|
},
|
|
);
|
|
|
|
test(
|
|
'retired database stays usable until its final lease releases',
|
|
() async {
|
|
final first = manager.openFor(_server('alpha'));
|
|
await first.customSelect('SELECT 1').get();
|
|
final lease = manager.tryAcquireLease(first);
|
|
check(lease).isNotNull();
|
|
|
|
final second = manager.openFor(_server('beta'));
|
|
check(identical(first, second)).isFalse();
|
|
check((await second.customSelect('SELECT 1').get())).isNotEmpty();
|
|
check((await first.customSelect('SELECT 1').get())).isNotEmpty();
|
|
|
|
await lease!.release();
|
|
await _waitForClosed(first);
|
|
check((await second.customSelect('SELECT 1').get())).isNotEmpty();
|
|
},
|
|
);
|
|
|
|
test('switching back reuses a leased retired database', () async {
|
|
final first = manager.openFor(_server('alpha'));
|
|
await first.customSelect('SELECT 1').get();
|
|
final lease = manager.tryAcquireLease(first)!;
|
|
|
|
manager.openFor(_server('beta'));
|
|
final reopened = manager.openFor(_server('alpha'));
|
|
|
|
check(identical(reopened, first)).isTrue();
|
|
check(
|
|
openedFileNames
|
|
.where((name) => name == DatabaseManager.fileNameFor('alpha'))
|
|
.length,
|
|
).equals(1);
|
|
await lease.release();
|
|
// Releasing an active database does not close it under its caller.
|
|
check((await reopened.customSelect('SELECT 1').get())).isNotEmpty();
|
|
|
|
manager.openFor(_server('gamma'));
|
|
await _waitForClosed(first);
|
|
});
|
|
|
|
test(
|
|
'closeActive waits for a leased database and release unblocks it',
|
|
() async {
|
|
final db = manager.openFor(_server('alpha'));
|
|
await db.customSelect('SELECT 1').get();
|
|
final lease = manager.tryAcquireLease(db)!;
|
|
|
|
var closeCompleted = false;
|
|
final close = manager.closeActive().whenComplete(
|
|
() => closeCompleted = true,
|
|
);
|
|
await Future<void>.delayed(Duration.zero);
|
|
check(closeCompleted).isFalse();
|
|
check((await db.customSelect('SELECT 1').get())).isNotEmpty();
|
|
check(manager.openForIfReady(_server('alpha')))
|
|
.isA<DatabaseOpenDeferred>();
|
|
check(manager.tryAcquireLease(db)).isNull();
|
|
|
|
await lease.release();
|
|
await close;
|
|
check(closeCompleted).isTrue();
|
|
await _waitForClosed(db);
|
|
},
|
|
);
|
|
|
|
test('an unmanaged database cannot acquire a manager lease', () async {
|
|
final unmanaged = AppDatabase(NativeDatabase.memory());
|
|
addTearDown(unmanaged.close);
|
|
|
|
check(manager.tryAcquireLease(unmanaged)).isNull();
|
|
});
|
|
|
|
test(
|
|
'lease release and unrelated deletion do not wait for another file close',
|
|
() async {
|
|
await manager.closeActive();
|
|
final databases = <String, GatedCloseDatabase>{};
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
final database = GatedCloseDatabase(
|
|
NativeDatabase(fileFor(fileName)),
|
|
)..failClose = false;
|
|
databases[fileName] = database;
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final alpha = manager.openFor(_server('alpha'));
|
|
await alpha.customSelect('SELECT 1').get();
|
|
final alphaGate = Completer<void>();
|
|
databases[DatabaseManager.fileNameFor('alpha')]!.closeGate = alphaGate;
|
|
addTearDown(() async {
|
|
if (!alphaGate.isCompleted) alphaGate.complete();
|
|
await _waitForClosed(alpha);
|
|
});
|
|
|
|
final beta = manager.openFor(_server('beta'));
|
|
await _waitForCloseAttempts(
|
|
databases[DatabaseManager.fileNameFor('alpha')]!,
|
|
1,
|
|
);
|
|
final betaLease = manager.tryAcquireLease(beta)!;
|
|
final gamma = manager.openFor(_server('gamma'));
|
|
await gamma.customSelect('SELECT 1').get();
|
|
|
|
// Releasing B is immediate bookkeeping, and B's physical close starts
|
|
// independently even though A's different SQLite file is still stuck.
|
|
var releaseCompleted = false;
|
|
final release = betaLease.release().whenComplete(
|
|
() => releaseCompleted = true,
|
|
);
|
|
await Future<void>.delayed(Duration.zero);
|
|
check(releaseCompleted).isTrue();
|
|
await release;
|
|
await _waitForCloseAttempts(
|
|
databases[DatabaseManager.fileNameFor('beta')]!,
|
|
1,
|
|
);
|
|
check(alphaGate.isCompleted).isFalse();
|
|
|
|
// Deleting C likewise awaits only C's exact executor.
|
|
await manager.deleteFor('gamma');
|
|
check(alphaGate.isCompleted).isFalse();
|
|
check(fileFor(DatabaseManager.fileNameFor('gamma')).existsSync())
|
|
.isFalse();
|
|
|
|
alphaGate.complete();
|
|
await _waitForClosed(alpha);
|
|
},
|
|
);
|
|
});
|
|
|
|
group('closeActive', () {
|
|
test('closes and forgets the active database', () async {
|
|
final db = manager.openFor(_server('alpha'));
|
|
await db.customSelect('SELECT 1').get();
|
|
await manager.closeActive();
|
|
await _waitForClosed(db);
|
|
|
|
// Re-opening the same server yields a fresh instance.
|
|
final reopened = manager.openFor(_server('alpha'));
|
|
check(identical(db, reopened)).isFalse();
|
|
check((await reopened.customSelect('SELECT 1').get())).isNotEmpty();
|
|
});
|
|
|
|
test('is a no-op when nothing is open', () async {
|
|
await manager.closeActive();
|
|
});
|
|
|
|
test('propagates failure from its own close attempt', () async {
|
|
await manager.closeActive();
|
|
late GatedCloseDatabase database;
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
database = GatedCloseDatabase(NativeDatabase(fileFor(fileName)));
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final opened = manager.openFor(_server('alpha'));
|
|
await opened.customSelect('SELECT 1').get();
|
|
|
|
await check(manager.closeActive()).throws<StateError>();
|
|
check(database.closeAttempts).equals(1);
|
|
|
|
// A second explicit close retries the exact failed executor; callers do
|
|
// not need to reopen it merely to make cleanup possible.
|
|
database.failClose = false;
|
|
await manager.closeActive();
|
|
check(database.closeAttempts).equals(2);
|
|
|
|
final reopened = manager.openFor(_server('alpha'));
|
|
check(identical(reopened, opened)).isFalse();
|
|
check((await reopened.customSelect('SELECT 1').get())).isNotEmpty();
|
|
database.failClose = false;
|
|
await manager.closeActive();
|
|
});
|
|
|
|
test(
|
|
'concurrent failed-close retries share and report one attempt',
|
|
() async {
|
|
await manager.closeActive();
|
|
late GatedCloseDatabase database;
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
database = GatedCloseDatabase(NativeDatabase(fileFor(fileName)));
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final opened = manager.openFor(_server('alpha'));
|
|
await opened.customSelect('SELECT 1').get();
|
|
await check(manager.closeActive()).throws<StateError>();
|
|
|
|
final retryGate = Completer<void>();
|
|
addTearDown(() {
|
|
if (!retryGate.isCompleted) retryGate.complete();
|
|
});
|
|
database.closeGate = retryGate;
|
|
final firstRetry = manager.closeActive();
|
|
final firstObserved = check(firstRetry).throws<StateError>();
|
|
await _waitForCloseAttempts(database, 2);
|
|
|
|
final secondRetry = manager.closeActive();
|
|
final secondObserved = check(secondRetry).throws<StateError>();
|
|
retryGate.complete();
|
|
|
|
await Future.wait<void>([firstObserved, secondObserved]);
|
|
check(database.closeAttempts).equals(2);
|
|
|
|
database.failClose = false;
|
|
await manager.closeActive();
|
|
check(database.closeAttempts).equals(3);
|
|
},
|
|
);
|
|
|
|
test('active close joins an older failed executor and every waiter settles both', () async {
|
|
await manager.closeActive();
|
|
final databases = <String, GatedCloseDatabase>{};
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
final database = GatedCloseDatabase(
|
|
NativeDatabase(fileFor(fileName)),
|
|
);
|
|
databases[fileName] = database;
|
|
return database;
|
|
},
|
|
);
|
|
final alphaFile = DatabaseManager.fileNameFor('alpha');
|
|
final betaFile = DatabaseManager.fileNameFor('beta');
|
|
|
|
final alpha = manager.openFor(_server('alpha'));
|
|
await alpha.customSelect('SELECT 1').get();
|
|
await check(manager.closeActive()).throws<StateError>();
|
|
final alphaDatabase = databases[alphaFile]!;
|
|
|
|
final beta = manager.openFor(_server('beta'));
|
|
await beta.customSelect('SELECT 1').get();
|
|
final betaDatabase = databases[betaFile]!..failClose = false;
|
|
alphaDatabase.failClose = false;
|
|
final alphaRetryGate = Completer<void>();
|
|
final betaCloseGate = Completer<void>();
|
|
addTearDown(() {
|
|
if (!alphaRetryGate.isCompleted) alphaRetryGate.complete();
|
|
if (!betaCloseGate.isCompleted) betaCloseGate.complete();
|
|
});
|
|
alphaDatabase.closeGate = alphaRetryGate;
|
|
betaDatabase.closeGate = betaCloseGate;
|
|
|
|
var firstCompleted = false;
|
|
final first = manager.closeActive().whenComplete(
|
|
() => firstCompleted = true,
|
|
);
|
|
await _waitForCloseAttempts(alphaDatabase, 2);
|
|
await _waitForCloseAttempts(betaDatabase, 1);
|
|
|
|
var joinedCompleted = false;
|
|
final joined = manager.closeActive().whenComplete(
|
|
() => joinedCompleted = true,
|
|
);
|
|
betaCloseGate.complete();
|
|
await Future<void>.delayed(Duration.zero);
|
|
check(firstCompleted).isFalse();
|
|
check(joinedCompleted).isFalse();
|
|
|
|
alphaRetryGate.complete();
|
|
await Future.wait<void>([first, joined]);
|
|
check(alphaDatabase.closeAttempts).equals(2);
|
|
check(betaDatabase.closeAttempts).equals(1);
|
|
});
|
|
});
|
|
|
|
group('deleteFor', () {
|
|
test(
|
|
'cannot reopen a server while deletion waits for its final lease',
|
|
() async {
|
|
final original = manager.openFor(_server('alpha'));
|
|
await original.customSelect('SELECT 1').get();
|
|
final lease = manager.tryAcquireLease(original)!;
|
|
final base = fileFor(DatabaseManager.fileNameFor('alpha'));
|
|
|
|
final deletion = manager.deleteFor('alpha');
|
|
final duplicateDeletion = manager.deleteFor('alpha');
|
|
check(identical(deletion, duplicateDeletion)).isTrue();
|
|
await Future<void>.delayed(Duration.zero);
|
|
|
|
// Once deleteFor owns the path, stale holders cannot extend its lease
|
|
// set and keep account isolation pending indefinitely.
|
|
check(manager.tryAcquireLease(original)).isNull();
|
|
|
|
AppDatabase? reopened;
|
|
Object? reopenError;
|
|
try {
|
|
reopened = manager.openFor(_server('alpha'));
|
|
await reopened.customSelect('SELECT 1').get();
|
|
} catch (error) {
|
|
reopenError = error;
|
|
}
|
|
|
|
await lease.release();
|
|
await deletion;
|
|
|
|
// Before the deletion tombstone existed, [reopened] was a live second
|
|
// executor here even though its SQLite path had just been unlinked.
|
|
check(reopened != null && !base.existsSync()).isFalse();
|
|
check(reopened).isNull();
|
|
check(reopenError is StateError).isTrue();
|
|
check(base.existsSync()).isFalse();
|
|
|
|
// Once deletion has settled, a clean reopen is allowed and creates a
|
|
// real backing file instead of retaining an executor to an unlinked
|
|
// inode.
|
|
final cleanReopen = manager.openFor(_server('alpha'));
|
|
await cleanReopen.customSelect('SELECT 1').get();
|
|
check(base.existsSync()).isTrue();
|
|
},
|
|
);
|
|
|
|
test(
|
|
'closes the active database and deletes db + journal + wal + shm files',
|
|
() async {
|
|
final db = manager.openFor(_server('alpha'));
|
|
await db.customSelect('SELECT 1').get();
|
|
|
|
final base = fileFor(DatabaseManager.fileNameFor('alpha'));
|
|
check(base.existsSync()).isTrue();
|
|
// Simulate leftover WAL artifacts (present while a database is in WAL
|
|
// mode, and after unclean shutdowns) plus SQLite's rollback journal.
|
|
File('${base.path}-journal').writeAsStringSync('journal');
|
|
File('${base.path}-wal').writeAsStringSync('wal');
|
|
File('${base.path}-shm').writeAsStringSync('shm');
|
|
|
|
await manager.deleteFor('alpha');
|
|
|
|
check(base.existsSync()).isFalse();
|
|
check(File('${base.path}-journal').existsSync()).isFalse();
|
|
check(File('${base.path}-wal').existsSync()).isFalse();
|
|
check(File('${base.path}-shm').existsSync()).isFalse();
|
|
await _waitForClosed(db);
|
|
},
|
|
);
|
|
|
|
test(
|
|
'deletes a non-active server\'s files without touching the active db',
|
|
() async {
|
|
final active = manager.openFor(_server('beta'));
|
|
await active.customSelect('SELECT 1').get();
|
|
|
|
final stale = fileFor(DatabaseManager.fileNameFor('alpha'));
|
|
stale.writeAsStringSync('old db');
|
|
File('${stale.path}-wal').writeAsStringSync('wal');
|
|
|
|
await manager.deleteFor('alpha');
|
|
|
|
check(stale.existsSync()).isFalse();
|
|
check(File('${stale.path}-wal').existsSync()).isFalse();
|
|
check((await active.customSelect('SELECT 1').get())).isNotEmpty();
|
|
},
|
|
);
|
|
|
|
test('is a no-op when no files exist', () async {
|
|
await manager.deleteFor('never-opened');
|
|
});
|
|
|
|
test(
|
|
'retries failed closes without racing deletion or opening a second db',
|
|
() async {
|
|
await manager.closeActive();
|
|
late GatedCloseDatabase database;
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
database = GatedCloseDatabase(NativeDatabase(fileFor(fileName)));
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final opened = manager.openFor(_server('alpha'));
|
|
await opened.customSelect('SELECT 1').get();
|
|
final base = fileFor(DatabaseManager.fileNameFor('alpha'));
|
|
check(base.existsSync()).isTrue();
|
|
|
|
await check(manager.deleteFor('alpha')).throws<StateError>();
|
|
check(base.existsSync()).isTrue();
|
|
check(database.closeAttempts).equals(1);
|
|
|
|
// An actual retry failure is propagated too; the manager does not
|
|
// treat the stale first error as proof that the file is safe to unlink.
|
|
await check(manager.deleteFor('alpha')).throws<StateError>();
|
|
check(base.existsSync()).isTrue();
|
|
check(database.closeAttempts).equals(2);
|
|
|
|
database.failClose = false;
|
|
final retryGate = Completer<void>();
|
|
database.closeGate = retryGate;
|
|
final deletion = manager.deleteFor('alpha');
|
|
final duplicateDeletion = manager.deleteFor('alpha');
|
|
check(identical(deletion, duplicateDeletion)).isTrue();
|
|
await _waitForCloseAttempts(database, 3);
|
|
|
|
// The failed executor remains the file owner while its close retry is
|
|
// in flight, so open cannot create a competing connection.
|
|
check(() => manager.openFor(_server('alpha'))).throws<StateError>();
|
|
check(base.existsSync()).isTrue();
|
|
|
|
retryGate.complete();
|
|
await deletion;
|
|
check(base.existsSync()).isFalse();
|
|
|
|
final cleanReopen = manager.openFor(_server('alpha'));
|
|
check(identical(opened, cleanReopen)).isFalse();
|
|
database.failClose = false;
|
|
check((await cleanReopen.customSelect('SELECT 1').get())).isNotEmpty();
|
|
},
|
|
);
|
|
|
|
test(
|
|
'delete and close callers both observe a shared retry failure',
|
|
() async {
|
|
await manager.closeActive();
|
|
late GatedCloseDatabase database;
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
database = GatedCloseDatabase(NativeDatabase(fileFor(fileName)));
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final opened = manager.openFor(_server('alpha'));
|
|
await opened.customSelect('SELECT 1').get();
|
|
await check(manager.closeActive()).throws<StateError>();
|
|
final base = fileFor(DatabaseManager.fileNameFor('alpha'));
|
|
|
|
final retryGate = Completer<void>();
|
|
addTearDown(() {
|
|
if (!retryGate.isCompleted) retryGate.complete();
|
|
});
|
|
database.closeGate = retryGate;
|
|
final deletion = manager.deleteFor('alpha');
|
|
final deletionObserved = check(deletion).throws<StateError>();
|
|
await _waitForCloseAttempts(database, 2);
|
|
|
|
final close = manager.closeActive();
|
|
final closeObserved = check(close).throws<StateError>();
|
|
retryGate.complete();
|
|
|
|
await Future.wait<void>([deletionObserved, closeObserved]);
|
|
check(database.closeAttempts).equals(2);
|
|
check(base.existsSync()).isTrue();
|
|
|
|
database.failClose = false;
|
|
await manager.deleteFor('alpha');
|
|
check(database.closeAttempts).equals(3);
|
|
check(base.existsSync()).isFalse();
|
|
},
|
|
);
|
|
|
|
test(
|
|
'delete-owned initial close is joined by concurrent close callers',
|
|
() async {
|
|
await manager.closeActive();
|
|
late GatedCloseDatabase database;
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
database = GatedCloseDatabase(NativeDatabase(fileFor(fileName)));
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final opened = manager.openFor(_server('alpha'));
|
|
await opened.customSelect('SELECT 1').get();
|
|
final closeGate = Completer<void>();
|
|
addTearDown(() {
|
|
if (!closeGate.isCompleted) closeGate.complete();
|
|
});
|
|
database.closeGate = closeGate;
|
|
|
|
final deletion = manager.deleteFor('alpha');
|
|
final deletionObserved = check(deletion).throws<StateError>();
|
|
await _waitForCloseAttempts(database, 1);
|
|
|
|
// deleteFor has already removed the executor from the active slot, but
|
|
// closeActive must still join that explicit physical close rather than
|
|
// report success while the deletion-owned attempt is unresolved.
|
|
final concurrentClose = manager.closeActive();
|
|
final closeObserved = check(concurrentClose).throws<StateError>();
|
|
closeGate.complete();
|
|
|
|
await Future.wait<void>([deletionObserved, closeObserved]);
|
|
check(database.closeAttempts).equals(1);
|
|
|
|
// The shared failed executor remains retryable for privacy cleanup.
|
|
database.failClose = false;
|
|
await manager.deleteFor('alpha');
|
|
check(database.closeAttempts).equals(2);
|
|
},
|
|
);
|
|
|
|
test('reuses the exact executor after a background close fails', () async {
|
|
await manager.closeActive();
|
|
late GatedCloseDatabase alphaDatabase;
|
|
var openCount = 0;
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
openCount += 1;
|
|
final database = GatedCloseDatabase(
|
|
NativeDatabase(fileFor(fileName)),
|
|
);
|
|
database.failClose = openCount == 1;
|
|
if (openCount == 1) alphaDatabase = database;
|
|
return database;
|
|
},
|
|
);
|
|
|
|
final opened = manager.openFor(_server('alpha'));
|
|
await opened.customSelect('SELECT 1').get();
|
|
final closeGate = Completer<void>();
|
|
alphaDatabase.closeGate = closeGate;
|
|
manager.openFor(_server('beta'));
|
|
await _waitForCloseAttempts(alphaDatabase, 1);
|
|
|
|
check(() => manager.openFor(_server('alpha'))).throws<StateError>();
|
|
check(openCount).equals(2);
|
|
closeGate.complete();
|
|
|
|
final recovered = await _waitForOpen(manager, 'alpha');
|
|
check(identical(opened, recovered)).isTrue();
|
|
check(openCount).equals(2); // alpha and beta; no second alpha executor.
|
|
check((await recovered.customSelect('SELECT 1').get())).isNotEmpty();
|
|
|
|
alphaDatabase.failClose = false;
|
|
await manager.closeActive();
|
|
});
|
|
});
|
|
|
|
group('fileNameFor', () {
|
|
test('encodes server ids without filename collisions', () {
|
|
final slash = DatabaseManager.fileNameFor('server/a');
|
|
final question = DatabaseManager.fileNameFor('server?a');
|
|
|
|
check(slash == question).isFalse();
|
|
check(slash.startsWith('server_')).isTrue();
|
|
check(RegExp(r'^[A-Za-z0-9_-]+$').hasMatch(slash)).isTrue();
|
|
});
|
|
|
|
test('supports a fixed filename for an independent database', () async {
|
|
await manager.closeActive();
|
|
openedFileNames.clear();
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) {
|
|
openedFileNames.add(fileName);
|
|
return AppDatabase(NativeDatabase(fileFor(fileName)));
|
|
},
|
|
databaseFileName: (_) => 'direct_local_v1',
|
|
);
|
|
|
|
final db = manager.openForServerId('logical-direct-id');
|
|
await db.customSelect('SELECT 1').get();
|
|
|
|
check(openedFileNames).deepEquals(['direct_local_v1']);
|
|
check(fileFor('direct_local_v1').existsSync()).isTrue();
|
|
|
|
await manager.deleteFor('logical-direct-id');
|
|
check(fileFor('direct_local_v1').existsSync()).isFalse();
|
|
});
|
|
|
|
test('rejects two logical ids that resolve to one filename', () async {
|
|
await manager.closeActive();
|
|
manager = DatabaseManager(
|
|
databaseDirectory: () async => tempDir,
|
|
openDatabase: (fileName) =>
|
|
AppDatabase(NativeDatabase(fileFor(fileName))),
|
|
databaseFileName: (_) => 'shared_name',
|
|
);
|
|
|
|
final active = manager.openForServerId('logical-a');
|
|
await active.customSelect('SELECT 1').get();
|
|
|
|
check(() => manager.openForServerId('logical-b')).throws<StateError>();
|
|
await check(manager.deleteFor('logical-b')).throws<StateError>();
|
|
check((await active.customSelect('SELECT 1').get())).isNotEmpty();
|
|
check(fileFor('shared_name').existsSync()).isTrue();
|
|
});
|
|
});
|
|
}
|
|
|
|
/// Polls until [db] rejects queries because its executor was closed.
|
|
Future<void> _waitForClosed(AppDatabase db) async {
|
|
final deadline = DateTime.now().add(const Duration(seconds: 5));
|
|
while (true) {
|
|
try {
|
|
await db.customSelect('SELECT 1').get();
|
|
} catch (_) {
|
|
return; // Closed.
|
|
}
|
|
if (DateTime.now().isAfter(deadline)) {
|
|
throw TimeoutException('database was never closed');
|
|
}
|
|
await Future<void>.delayed(const Duration(milliseconds: 10));
|
|
}
|
|
}
|
|
|
|
Future<void> _waitForCloseAttempts(
|
|
GatedCloseDatabase database,
|
|
int expected,
|
|
) async {
|
|
final deadline = DateTime.now().add(const Duration(seconds: 5));
|
|
while (database.closeAttempts < expected) {
|
|
if (DateTime.now().isAfter(deadline)) {
|
|
throw TimeoutException(
|
|
'database made ${database.closeAttempts} close attempts; '
|
|
'expected $expected',
|
|
);
|
|
}
|
|
await Future<void>.delayed(const Duration(milliseconds: 10));
|
|
}
|
|
}
|
|
|
|
Future<AppDatabase> _waitForOpen(
|
|
DatabaseManager manager,
|
|
String serverId,
|
|
) async {
|
|
final deadline = DateTime.now().add(const Duration(seconds: 5));
|
|
while (true) {
|
|
try {
|
|
return manager.openFor(_server(serverId));
|
|
} on StateError {
|
|
if (DateTime.now().isAfter(deadline)) rethrow;
|
|
await Future<void>.delayed(const Duration(milliseconds: 10));
|
|
}
|
|
}
|
|
}
|