Proves the sync mechanics the editorial-review feature relies on:
server-minted drafts download to clean clients, offline ID collisions
re-ID via the existing client machinery, server-side scene rewrites
auto-download or surface the standard conflict.
* Lock the sync baseline to the server-confirmed hash
Editing a scene through the editor stamps `lastEdited` (an autosave side
effect), and the sync conflict baseline was being re-derived from local
state at mark-dirty time. Because `lastEdited` had already moved, the
recorded baseline disagreed with what the server stored — so a single
client editing its own scene and resyncing raised a phantom conflict on
every sync, with no other device involved.
Stop re-deriving the baseline. The conflict baseline (`originalHash`) is
now the hash the server last confirmed it holds: recorded only on a
successful transfer (the exact bytes uploaded or downloaded) and read
back when an entity next goes dirty. A field that mutates locally after
a sync can no longer taint the baseline.
- ProjectSynchronizationData gains `syncedHashes` (per-entity confirmed
hash). EntityOriginalState.originalHash is nullable: null means the
server never confirmed a hash (new entity, or first edit after upgrade)
so the server skips the conflict check and the baseline self-heals on
that upload.
- SyncJournal.markEntityAsDirty reads the baseline from syncedHashes
rather than taking a recomputed hash; recordSyncedHash sets it.
- EntitySynchronizer.uploadEntity reports the accepted hash via a
callback; the transfer records it on upload and download.
- The upload sources `originalHash` from syncedHashes at transfer time,
not the dirty entry's frozen copy — so a partial sync (some entities
uploaded, a later one failed) can't leave a stale baseline that forges
a phantom conflict on retry.
- FinalizeSync preserves syncedHashes written mid-transfer and prunes
deleted ids (including server-driven deletions) from the map.
- Repositories no longer recompute an entity hash to mark for sync.
Adds an integration regression test: a single client editing a scene
through the editor and resyncing must not raise a conflict (red before
this change, green after).
* Add e2e sync resync scenarios and stabilize the integration harness
High-value round-trip coverage for the sync baseline beyond the
content-edit regression, plus the test-harness fixes the new tests
needed to be reliable.
Tests:
- SyncHashStabilityTest: after a sync, the client's hash for a scene
equals the hash the server stored (baseline and server agree).
- ResyncBaselineScenariosTest: a single client must never raise a
phantom conflict on resync regardless of how the baseline was set —
metadata-only edit, rename, and an edit after a download establishes
the baseline. Each asserts the server's stored hash actually changed,
so a resync that uploaded nothing can't pass vacuously.
Harness (RoundTripTestBase):
- Bind the client IO/Default dispatchers to the same single-threaded
dispatcher as Main. FakeFileSystem is not thread-safe (its open-files
list is a plain MutableList), so the real multi-threaded dispatchers
let concurrent opens across the repos' background scopes corrupt it,
throwing intermittently from findOpenFile — the root of the suite's
order-dependent flakiness.
- newClient() registers each client for teardown-close, so a test that
throws before its own cleanup can't leak a project scope (and its
background coroutine scopes) into the next test.
- Shared syncNoConflict()/serverEntityHash() helpers replace per-file
copies.
- Close the per-test HttpClient in EndToEndTest.tearDown; it was created
every test and never closed, leaking engine threads.
* Add per-entity-type sync resync matrix
The lock-the-baseline fix lives in the shared sync layer, so it must
hold for every entity type. EntityTypeResyncMatrixTest exercises
create → sync → edit → resync (assert no phantom conflict) for notes,
timeline events, and encyclopedia entries; a create → sync → resync for
immutable scene drafts; and a create → sync → delete → resync that
confirms the entity is removed from the server (covering the deletion
path and the synced-hash pruning).
* Fix timeline store race on sync finalize
TimeLineRepository.storeTimeline() read timelineFlow.replayCache.first(),
which throws NoSuchElementException when the timeline was never loaded.
finalizeSync runs it on every sync — including syncs of a project that
never touched the timeline — so whether the async timeline load had
emitted yet was a race that could fail the whole sync. Guard with
firstOrNull (nothing loaded means nothing to store), matching the
pattern correctEventOrder already uses. Surfaced as order-dependent
flakiness once the e2e suite grew.
* Add sync fuzz/property test
A single client driving a seeded random interleaving of editor content
edits, metadata edits, renames, and syncs must never raise a conflict —
there is no other device, so any conflict is a phantom. Catches ordering
edge cases the hand-written scenarios miss (a sync landing between a
content edit and its debounced autosave, a rename between edits). Seeds
are fixed and printed on failure so a failing sequence can be replayed.
* Backfill sync baselines on the first post-upgrade sync
A client upgrading to the lock-the-baseline version has an empty
syncedHashes map — the field did not exist in the old sync.json. With no
baseline a dirty entity uploads with originalHash = null, and the server
skips the conflict check (ServerEntitySynchronizer only conflicts when
originalHash != null), silently overwriting a concurrent edit from
another device.
FetchLocalData now establishes a baseline for every entity that still
lacks one, before any upload reads it: the current local hash for an
in-sync entity (it equals the server's hash for an agreed entity), or
the frozen pre-edit hash carried over from the old dirty list for an
entity that was already dirty at upgrade time. The local-hash source is
safe — if local has silently diverged from the server, the backfilled
baseline is the old hash, so the next edit's upload is correctly flagged
as a conflict rather than a silent overwrite. Existing recorded baselines
are never touched, so it is a one-time, idempotent migration.
Residual: an entity edited for the very first time post-upgrade, before
any sync has run the backfill, still has no recoverable baseline.
* Remove dead SceneDraftRepository.markForSynchronization
Drafts are immutable after creation, so this was never called (its own
comment said so). Removing it leaves syncJournal unused in the repo, so
drop that field and its import too.
* Persist + assert scene content in SceneTimestampsTest
The test set content via onContentChanged + an immediate storeSceneBuffer.
onContentChanged registers the editor buffer asynchronously, so the
storeSceneBuffer found no buffer ("no buffer present"), no-op'd, and the
scene reached the server empty. The test only asserted timestamps, so it
passed while never validating that content survives the round-trip.
Persist via the synchronous storeSceneMarkdownRaw path, and assert the
server-stored entity's content equals the uploaded content so the blind
spot can't silently return.
* Rename IdRepository to IdAllocator
It is a special foundational primative
* Rename SyncDataRepository to SyncJournal
It is a special foundational primative
* Rename GlobalSettingsRepository to GlobalSettingsStore
It is a special foundational primitive
* docs: add Foundation primitives tier to the architecture doc
Document IdAllocator, SyncJournal, and GlobalSettingsStore as a fixed set of
stateful, cross-cutting primitives that the whole data layer may depend on —
acyclic leaves (GlobalSettingsStore <- SyncJournal <- IdAllocator) that named
the dependency reality instead of treating it as a no-sibling violation.
Split the monolithic SceneEditorRepository into single-responsibility pieces:
- SceneRepository — scene tree, structure, ordering, paths, on-disk layout
- SceneContentRepository — in-memory buffers, autosave, dirty tracking
- SceneMetadataRepository — per-scene and project metadata
- SceneEditorService — the component-facing facade that orchestrates the three
and applies the cross-cutting side-effects (statistics, writing activity,
reference index) the repositories deliberately don't reach up to perform
Components and synchronizers now talk to SceneEditorService; the underlying
repositories hold no sibling-repo dependencies. Tests are reorganized to match
the new boundaries, and the scene-editing domain API is documented in
docs/DESIGN_PATTERNS.md.
The previous comment claimed the range assertion proved the edit
"round-tripped instead of being stamped with NOW() server-side." It
doesn't — server-side NOW() during sync falls in the same window.
Replace with an honest description: storeAllBuffers inside prepareForSync
bumps lastEdited right before upload, and the debounced contentFlow can
fire again post-sync, so client-bumped-during-sync and server-stamped-
on-receipt are observationally identical without instrumenting the sync
pipeline. The range check still rules out drops, epoch-0 resets, and
clearly-wrong stamps.
Two failures came up running :integrationTests:jvmTest after the
SQLite-to-Postgres migration.
1. RoundTripTestBase seeded server_config with a raw epoch integer:
INSERT INTO server_config VALUES ('whitelist_enabled', 'false', 1704067200);
server_config.updated_at is now TIMESTAMPTZ; Postgres refused to
coerce the integer. Wrapped the value in to_timestamp(...).
2. SceneTimestampsTest's upload test asserted exact equality between a
pre-sync local lastEdited snapshot and the server's stored value.
SceneEditorRepository runs a debounced (~500ms) auto-save that calls
recordSceneActivity → bumps lastEdited on every Editor-source store,
and the storeAllBuffers pass inside sync's prepareForSync triggers
another. The exact-equality assertion happened to pass against fast
in-memory SQLite because the whole test finished inside the debounce
window; with the slower embedded Postgres it doesn't.
The test's intent is "edit timestamps round-trip to the server, not
stamped with NOW() server-side." Reframed the assertion as "server's
created and lastEdited both fall inside [beforeEdit, afterSync]" —
which is what the test actually proves.
Adds `created` and `lastEdited` timestamps to SceneMetadata, stamped
on every autosave tick in SceneEditorRepository.recordSceneActivity.
StatisticsService rolls them up into a new "Last Edited" tile on the
project home dashboard, and the Android widget reads the same field
from the stats cache.
The timestamps ride the sync wire on ApiProjectEntity.SceneEntity and
participate in EntityHasher.hashScene. On download, local values are
preserved when the server didn't ship them. No conflict resolution —
last write wins.
EntityHasher hash functions had their defaults removed so call sites
can't silently miss a newly added field; this caught a latent bug in
the second markForSynchronization hash call that was relying on
defaults for archived/confirmedReferences/dismissedReferences.
Spins up a real Jetty server and a real headless client in the same JVM,
runs the actual sync protocol over HTTP, and asserts against the shared
FakeFileSystem. Covers seven scenarios: smoke (handshake), client upload,
server download, independent edits, conflict resolved to server, conflict
resolved to client, and client delete.
- New :integrationTests Gradle module with the RoundTripTestBase harness,
HeadlessClient driver, and seven scenario tests.
- :server enables java-test-fixtures so EndToEndTest, E2eTestData, and
SqliteTestDatabase are shared with the new module instead of duplicated.
- CI: build.yml runs the suite on every PR/develop push; prepare-release.yml
adds an integration-tests job that gates all seven package jobs, so a
broken sync protocol blocks the release before any artifact is built.
