vrr/joplock
2
0
Fork 0
mirror of https://github.com/abort-retry-ignore/joplock.git synced 2026-10-19 18:24:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat(joplin): real-client compatibility harness; fix shared-note unsharing, stale share ids and subtree revoke

Browse source 
- integration-tests/: drive the real Joplin CLI against the dev stack (fidelity, guards, nested notebooks, sharing)
- read the share id from jop_share_id: editing a shared note twice used to unshare it for everyone
- subtree share changes re-save descendants through the API so stock Joplin recipients drop/receive them
- resources written with Joplin's default ocr_driver_id
- restore-from-trash regression test; docs (AGENT_GUIDE) incl. the proxy-guard finding
- playwright: HTML-note and E2EE placeholder behaviour in the real UI
This commit is contained in:
igor 2026-10-09 15:13:22 +13:00
parent d0321d595d
commit dda702b6b7
19 changed files with 983 additions and 82 deletions
Download patch file Download diff file Expand all files Collapse all files

39
AGENT_GUIDE.md
View file

@ -150,11 +150,12 @@ Joplin notebooks nest via the folder's `parent_id`. Joplock renders, creates, mo
- **Structure:** not into itself or a descendant (mirrors Joplin's `canNestUnder`). Same-parent move is a no-op.
- **Delete = promote.** Sub-notebooks move up to the deleted notebook's parent; its notes move to that parent (top level: the existing top-level "General", created if missing). Deleting never deletes notes or sub-notebooks. (Old behaviour moved every note to General and orphaned sub-notebooks.)
- **Vaults are top-level leaves.** Cannot contain notebooks, cannot be moved, cannot be created nested (`POST /api/web/vaults` refuses a nested notebook or one with children), cannot end up in a share. The new-notebook modal disables the parent picker when "vault" is ticked. The sync proxy rejects (403) a notebook PUT whose parent is a vault, so an external Joplin client cannot hang plaintext notebooks under a vault.
- **Vaults are top-level leaves.** Cannot contain notebooks, cannot be moved, cannot be created nested (`POST /api/web/vaults` refuses a nested notebook or one with children), cannot end up in a share. The new-notebook modal disables the parent picker when "vault" is ticked. `vaultProxyGuard` also rejects a notebook PUT whose parent is a vault, **but see "Sync-proxy guards do not apply to stock Joplin clients" below: that guard is currently inert for real clients.**
- **Sharing:**
- Only the owner creates/moves/deletes inside a share (recipients get 403).
- A **share root stays top level** (Joplin's rule): moving one under anything is refused ("Stop sharing…"); sharing a nested notebook **lifts it to the top level first** (`prepareForShare`; the share dialog says so); a notebook already inside a share cannot be shared again; a subtree containing a vault cannot be shared.
- `ensureShareIdsOnNotebook` now stamps the **whole subtree** (every sub-notebook and the notes directly in each), writing `share_id`/`is_shared` into the item JSON **and** the `jop_share_id` column (falls back to JSON only if a server lacks the column). Creating a sub-notebook in a share inherits it; moving a subtree into/out of a share re-stamps it and grants/revokes recipient `user_items` (`createShareSync.setSubtreeShare`).
- **Subtree share changes go through the API.** `createShareSync.setSubtreeShare` re-saves every notebook and note of the subtree with the new share fields (`itemWriteService`, lossless thanks to pass-through) so Joplin Server emits change events. Silent database writes are invisible to stock clients: a subtree moved **out** of a share used to stay in the recipient's Joplin app. Items the API cannot rewrite (Joplin-encrypted) get the database sweep only. Verified with a real client recipient in `integration-tests/joplin-sharing.test.js`.
- **Stopping a share clears the dead `share_id` from the owner's whole tree** (notebook, sub-notebooks, notes; JSON and `jop_share_id`), like Joplin's own `unshareFolder`. Left in place, Joplin Server answers any later write of those items with `share not found` (422 — the note could no longer be edited or trashed) and **silently refuses to delete a root folder that still carries a share id** (returns 200, does nothing). Pinned by `playwright-tests/nested-share.spec.js` step 6.
- Recipient access keeps the existing Joplock convention: `user_items.item_id` holds the **`jop_id`** (Joplin Server's own fan-out uses `items.id`, which `itemAccessExpression` does not read).
- `updateFolder` now preserves `created_time`, `user_created_time`, `icon` and `master_key_id` (a rename/move used to reset them).
@ -169,6 +170,38 @@ Joplin notebooks nest via the folder's `parent_id`. Joplock renders, creates, mo
Verified live while building this: a **note created in a shared notebook after the share** is not visible to a Joplock recipient, and a **note moved out of a share stays visible** to the recipient (the note write paths never call grant/revoke; only folder operations do). Fixing means calling `createShareSync.setItemsShare` from the note create/move paths.
## Joplin Field Fidelity (do not regress)
Joplock shares one account with stock Joplin clients, so **a Joplock write must change only what the user edited**. Originally `serializeNote`/`serializeFolder` rebuilt every item from hard-coded values: a single body edit turned a to-do into a plain note, wiped author/source URL/location, flipped an HTML note to markdown, reset `order` and overwrote `source_application`. Measured with the real Joplin CLI synced through Joplock's proxy, then fixed.
### How it works
- **Pass-through serializer** (`app/items/joplinItem.js`): reproduces Joplin's `serialize` / `serialize_format` (canonical key order, ISO times, `\n` escaping, no empty body block). Values come from the item's **stored fields** and only the fields Joplock owns are overridden.
- **Items carry their stored fields.** `mapNoteRow` / `mapFolderRow` attach `fields` (the parsed `items.content` JSON). Every write path passes `existing`, so `serializeNote({ fields: existing.fields, ... })` writes everything else back untouched, including fields Joplock has never heard of (`is_locked`, `extracted_resource_ids`, `user_data`, `application_data`, ...). Defaults (`order = now`, zero geo strings, `markup_language 1`, `source_application net.cozic.joplock-web`) apply **only to brand-new items**; edited items keep their original `source_application`.
- **Joplock-owned fields:** title, body, parent, deleted time, `is_shared`/`share_id`, `updated_time` (always now, sync needs it). `user_updated_time` moves **only on a real title/body edit**; move, trash, restore and share changes preserve it (Joplin's `moveToFolder` semantics).
- **`fields` is internal.** `publicItem()` / `publicItems()` (`routes/_helpers.js`) strip it from JSON API responses. Keep using them on any new endpoint that returns a note or notebook.
- **The server only stores fields its own Joplin library knows** (it drops the rest on upload and when serving), so "unknown to Joplock" fields that can persist are ones a current Joplin writes. Joplin clients themselves also drop unknown fields on read.
- **Share id lives in a column.** Joplin Server moves `share_id` out of the item JSON into `items.jop_share_id` on the first API save. Always read it via `shareIdOfRow()` (column first, JSON fallback) / `shareIdOf()` in SQL. Reading only the JSON made a shared note look unshared after its first edit, so the *second* Joplock edit wrote `share_id: ''` and silently took the note away from every recipient.
### Items Joplock must treat carefully
| Item | Behaviour |
|---|---|
| `markup_language = 2` (HTML note) | Shown **rendered and read-only** (`app/htmlNoteRenderer.js`, allowlist sanitizer on untrusted HTML + TinyMCE's own sanitizing). Form is `data-html-note="1"`: no markdown/rendered switch, no toolbar, title not editable, `tinyMCESyncToTA` is a no-op. The server refuses any body change (403) but still allows move/trash/restore. Never convert it to markdown. |
| Joplin end-to-end encrypted (`jop_encryption_applied = 1`) | **Locked placeholder only.** `e2ee: true`, title/body blanked, no editor form at all. `updateNote`/`updateFolder` throw 403 (`assertNotE2ee`); a live one cannot be hard-deleted; a notebook holding any cannot be deleted (409, checked **before** any write so a delete never half-completes). |
| `is_conflict = 1` (conflict copy) | **Hidden** from lists, counts, headers and search (`conflictFilterSql`), like Joplin's "Conflicts" notebook. Stays untouched, and travels with its notebook when that is deleted (`includeConflicts`). |
| Attachments | `serializeResource` writes Joplin 3.x's OCR fields; `ocr_driver_id` is `1` (Joplin's default; `0` is a legacy quirk). |
### Real-client integration tests (opt-in)
`npm run setup:joplin-cli` installs the real Joplin terminal client under `~/.cache` (not in the repo). `npm run test:joplin` (dev stack up, admin env vars set) drives it headless through Joplock's `/joplin` proxy using a dedicated `compat-client@joplock.test` account (`integration-tests/joplin/harness.js`): `joplin-fidelity`, `joplin-guards`, `joplin-nested`, `joplin-sharing`. They create items in the client, sync, edit/move/trash in Joplock, sync back and compare **field by field** from the client's own SQLite. Raw uploads authenticate with `X-API-AUTH`, like a real client. Joplin Server applies share changes on a 10 s task, so the sharing tests wait ~13 s per step. One test is a tracked `todo` (below).
### Known issues found with the real client (not fixed)
- **Sync-proxy guards do not apply to stock Joplin clients.** `vaultProxyGuard` and `shareProxyGuard` identify the user with `authenticatedUser()`, which reads only Joplock's `sessionId` **cookie**. Joplin apps authenticate with an `X-API-AUTH` header and send no cookie, so both guards "stream through". Verified: a real client wrote a **plaintext note into a vault notebook and created a notebook under a vault**, and nothing stopped it. The vault/share/"no notebook under a vault" proxy protections described elsewhere in this guide therefore only hold for browser-origin traffic. Making them live needs `authenticatedUser` to accept `X-API-AUTH` **and** a fix for a second bug: when a request does authenticate, both guards call `bufferRequest()` on the same stream, so the second read never completes and the request hangs (seen as a hung cookie-authenticated PUT to `/joplin/...`). Behaviour will change for existing clients (403s on vault writes), so treat it as its own change.
- **Collaborator content is invisible to the share owner in Joplock.** Access checks use Joplock's own `user_items` rows keyed by `jop_id`; Joplin Server keys them by `items.id`. A note a collaborator creates in a Joplin app never shows for the owner in Joplock (it does in the owner's Joplin apps). Tracked as the `todo` test in `joplin-sharing.test.js`. Fixing it means changing the access model together with Joplock's own removal paths (otherwise removed users keep access).
- Notes created in a share after sharing do not reach a **Joplock** recipient, and a note moved out of a share stays visible to a Joplock recipient (the note write paths never grant/revoke Joplock's `jop_id` rows; only folder operations do). Stock-client recipients are handled correctly by the server.
## Core Rules
1. Do not modify Joplin Server source for Joplock features unless explicitly approved.
@ -207,7 +240,7 @@ Server (`app/routes/_helpers.js`) rules:
- `assertVaultNoteBodyEncrypted(vaultService, userId, existingParentId, targetParentId, body, noteId, opts)` parses the ciphertext blob and rejects the write when `meta.vault` mismatches the target folder's vault, or `meta.noteId` (if present) mismatches the target note.
- **Vault boundary enforcement** (three layers):
1. **ParentId immutability / no conflict copies**: if a note currently lives in a vault folder, the server rejects any PUT that changes its `parentId` (400: `Vault notes cannot be moved to a different folder`). Vault notes cannot change folder — the folder select is disabled client-side and the server enforces this on every write path. Conflict `createCopy` is also rejected for vault notes: ciphertext is bound to source note id, while copying unlocked DOM content would make a plaintext duplicate.
2. **Ciphertext required inside vaults**: saving to a vault folder always requires an encrypted body. Same-folder saves and sync-proxy writes are both covered. `enforceExistingVault=true` (sync-proxy only) extends this to prevent external Joplin clients from stripping ciphertext by re-parenting.
2. **Ciphertext required inside vaults**: saving to a vault folder always requires an encrypted body. Same-folder saves are covered; sync-proxy writes are covered **only for cookie-authenticated (browser) traffic, not stock Joplin clients** (see "Known issues found with the real client"). `enforceExistingVault=true` (sync-proxy only) extends this to prevent external Joplin clients from stripping ciphertext by re-parenting.
3. **Metadata integrity**: encrypted blobs carry `vault` (folder id) and `noteId` (bound target); both are validated against the destination to prevent cross-vault or cross-note ciphertext smuggling.
- **Client-side**: the folder select (`#editor-folder-select`) is rendered `disabled` for vault-protected notes and stays disabled after unlock — vault notes cannot change parent folder through the UI. The only reachable folder-change path is plain→vault (encrypt on move).
- Every note-write path passes the target `noteId`: `app/routes/api.js` (PUT), `app/routes/fragments.js` (autosave PUT), `app/routes/history.js` (restore), `app/proxy/vaultProxyGuard.js` (sync proxy).
@ -373,6 +406,7 @@ the service that owns them (e.g. `app/settingsService.js` also carries the
### Tests
- `tests/*.test.js` — unit/integration tests (`node:test`; several suites extract real functions out of `public/app.js` and run them under a JSDOM harness)
- Run: `npm test` (i.e. `node --test tests/*.test.js`)
- Real Joplin client compatibility (opt-in, needs the dev stack): `npm run setup:joplin-cli` once, then `npm run test:joplin` (`integration-tests/*.test.js`)
- `playwright-tests/*.spec.js` — browser E2E against a live dev stack; run with `npm run test:ui`
### Deployment
@ -966,6 +1000,7 @@ Recommended inner loop:
## Recently Completed Work
- **Joplin field fidelity + compatibility harness**: pass-through serializer, stored `fields` on every item, HTML notes rendered read-only, Joplin-E2EE items as locked placeholders, conflict copies hidden, share id read from `jop_share_id`, OCR fields on uploads; opt-in real-client tests (`npm run test:joplin`). Also fixed along the way: **restoring a note from the trash returned 404 for every trashed note** (the ownership lookup ignored trashed notes), and **editing a shared note twice in Joplock silently unshared it**. See "Joplin Field Fidelity".
- **Nested notebooks**: Joplin's `parent_id` hierarchy now renders as a tree on desktop (multi-expand, rolled-up counts, ancestors open) and mobile (inline expandable tree), with create-under-parent, move (tree picker), delete-promotes, whole-subtree sharing, and vaults as top-level leaves. `app/items/folderTree.js` + `app/items/folderOps.js`. `updateFolder` also stopped resetting `created_time`/`icon`.
- **Rich-mode uploads now insert through the live editor**: uploading from the modal or the older `#file-upload` picker in rendered mode used to target the hidden `#note-body` textarea, which TinyMCE's own lazy sync overwrote before the debounced save fired — the attachment silently vanished. `_captureUploadInsertTarget()` now captures a TinyMCE caret range (`{mode:'tinymce', rng}`) when the host is visible and the editor is not read-only, and `_insertResourceIntoTinyMCEFromMarkdown()` rebuilds the `<img>`/`<a>` from the saved `![](:/id)` markdown, inserts it with the same `_tinyMCEBlockAttachmentHtml()` padding as drag-drop, and syncs `#note-body` immediately. All three rich-mode insert paths call `_endTinyMCEPostLoadWindow()` so the insert is treated as a real edit instead of post-load echo (otherwise the reconcile re-baselines the hash and the save sees "unchanged"). Read-only (mobile rendered) keeps the textarea/CM fallback. See "Upload behavior" above.
- **flushSave uses `keepalive: true`**: the unload-time flush (visibilitychange, tab close, note-switch navigation) was being aborted mid-flight — Chromium sent a truncated request with no `Cookie` header (401) and the pending edits were lost. Pinned by `tests/saveIdentityGuard.test.js`.

2
app/createServer.js
View file

@ -135,7 +135,7 @@ const createServer = options => {
itemService,
itemWriteService,
vaultService,
shareSync: database ? routeShares.createShareSync({ itemService, database }) : null,
shareSync: database ? routeShares.createShareSync({ itemService, itemWriteService, database }) : null,
});
const upstreamRequestContext = _request => ({

3
app/items/folderOps.js
View file

@ -105,6 +105,7 @@ const createFolderOps = ({ itemService, itemWriteService, vaultService, shareSyn
if (shareFields.shareId && shareSync) {
await shareSync.setSubtreeShare({
ownerId: user.id, folderId: created.id, shareId: shareFields.shareId, previousShareId: '',
sessionId: user.sessionId, requestContext,
});
}
return { id: created.id, parentId: parent, ...shareFields };
@ -168,6 +169,7 @@ const createFolderOps = ({ itemService, itemWriteService, vaultService, shareSyn
if (shareChanges && shareSync) {
await shareSync.setSubtreeShare({
ownerId: user.id, folderId, shareId: newShareId, previousShareId: oldShareId,
sessionId: user.sessionId, requestContext,
});
}
return { changed: true, folder: state.folder, shareChanged: shareChanges };
@ -218,6 +220,7 @@ const createFolderOps = ({ itemService, itemWriteService, vaultService, shareSyn
if (childShareId !== newShareId && shareSync) {
await shareSync.setSubtreeShare({
ownerId: user.id, folderId: child.id, shareId: newShareId, previousShareId: childShareId,
sessionId: user.sessionId, requestContext,
});
}
}

34
app/items/itemService.js
View file

@ -12,6 +12,14 @@ const decodeItemContent = content => {
return JSON.parse(raw);
};
// Joplin Server keeps an item's share id in the `jop_share_id` COLUMN and drops it
// from the stored JSON as soon as the item is saved through the sync API (any save,
// by Joplock or a Joplin app). The JSON copy only exists on items Joplock stamped
// itself and goes stale. Reading only the JSON made a shared item look unshared after
// its first edit, so the next Joplock edit wrote `share_id: ''` and silently took the
// note away from every recipient.
const shareIdOfRow = (row, content) => `${row.jop_share_id || content.share_id || ''}`;
// Items encrypted by Joplin's own end-to-end encryption (not Joplock vaults).
// The stored content is ciphertext: title/body are empty and rewriting the item
// from here would replace the ciphertext with plaintext. They are shown as a
@ -40,8 +48,8 @@ const mapFolderRow = row => {
masterKeyId: content.master_key_id || '',
updatedTime: Number(row.jop_updated_time || content.updated_time || 0),
ownerId: row.owner_id || '',
shareId: content.share_id || '',
isShared: !!(Number(content.is_shared || 0)),
shareId: shareIdOfRow(row, content),
isShared: !!(Number(content.is_shared || 0)) || !!shareIdOfRow(row, content),
};
};
@ -93,8 +101,8 @@ const mapNoteRow = row => {
createdTime: Number(content.created_time || row.created_time || 0),
updatedTime: Number(row.jop_updated_time || content.updated_time || 0),
ownerId: row.owner_id || '',
shareId: content.share_id || '',
isShared: !!(Number(content.is_shared || 0)),
shareId: shareIdOfRow(row, content),
isShared: !!(Number(content.is_shared || 0)) || !!shareIdOfRow(row, content),
};
};
@ -110,8 +118,8 @@ const mapNoteHeaderRow = row => {
deletedTime: Number(row.deleted_time || 0),
updatedTime: Number(row.jop_updated_time || 0),
ownerId: row.owner_id || '',
shareId: row.share_id || '',
isShared: !!Number(row.is_shared || 0),
shareId: `${row.jop_share_id || row.share_id || ''}`,
isShared: !!Number(row.is_shared || 0) || !!(row.jop_share_id || row.share_id),
};
};
@ -176,7 +184,7 @@ const createItemService = database => {
return {
async foldersByUserId(userId) {
const result = await database.query(`
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied, jop_share_id
FROM items
WHERE jop_type = $2${deletedFilterSql('exclude')} AND ${itemAccessExpression()}
ORDER BY LOWER(COALESCE(convert_from(content, 'UTF8')::json->>'title', '')) ASC, created_time ASC
@ -187,7 +195,7 @@ const createItemService = database => {
async folderByUserIdAndJopId(userId, folderId) {
const result = await database.query(`
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied, jop_share_id
FROM items
WHERE jop_type = $2 AND jop_id = $3 AND ${itemAccessExpression()}
LIMIT 1
@ -210,7 +218,7 @@ const createItemService = database => {
}
const result = await database.query(`
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied, jop_share_id
FROM items
${where}
ORDER BY jop_updated_time DESC, created_time DESC
@ -228,6 +236,7 @@ const createItemService = database => {
jop_updated_time,
owner_id,
jop_encryption_applied,
jop_share_id,
COALESCE(convert_from(content, 'UTF8')::json->>'title', '') AS title,
COALESCE((convert_from(content, 'UTF8')::json->>'deleted_time')::bigint, 0) AS deleted_time,
COALESCE(convert_from(content, 'UTF8')::json->>'share_id', '') AS share_id,
@ -296,6 +305,7 @@ const createItemService = database => {
jop_updated_time,
owner_id,
jop_encryption_applied,
jop_share_id,
COALESCE(convert_from(content, 'UTF8')::json->>'title', '') AS title,
COALESCE((convert_from(content, 'UTF8')::json->>'deleted_time')::bigint, 0) AS deleted_time,
COALESCE(convert_from(content, 'UTF8')::json->>'share_id', '') AS share_id,
@ -315,9 +325,9 @@ const createItemService = database => {
if (!termSql) return [];
const limitIdx = 2 + termParams.length + 1;
const result = await database.query(`
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied, jop_share_id
FROM (
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied,
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied, jop_share_id,
${safeJsonExpression("convert_from(content, 'UTF8')")} AS parsed
FROM items
WHERE jop_type = $2 AND ${itemAccessExpression()}
@ -335,7 +345,7 @@ const createItemService = database => {
async noteByUserIdAndJopId(userId, noteId, options = {}) {
const deleted = options.deleted || 'exclude';
const result = await database.query(`
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied
SELECT id, jop_id, jop_parent_id, jop_updated_time, created_time, content, owner_id, jop_encryption_applied, jop_share_id
FROM items
WHERE jop_type = $2 AND jop_id = $3${deletedFilterSql(deleted)} AND ${itemAccessExpression()}
LIMIT 1

2
app/items/itemWriteService.js
View file

@ -150,7 +150,7 @@ const serializeResource = resource => {
ocr_details: '',
ocr_status: 0,
ocr_error: '',
ocr_driver_id: 0,
ocr_driver_id: 1, // Joplin's default (PrintedText); 0 is a legacy quirk it only tolerates
},
overrides,
}),

82
app/routes/shares.js
View file

@ -48,16 +48,21 @@ const inviteesList = data => {
const shareFolderId = share => share && (share.folder_id || share.notebook_id || share.folderId || share.notebookId || '');
// `share_id` of an item, decoded from its JSON content.
// `share_id` of an item.
//
// The authoritative value is the `jop_share_id` COLUMN: Joplin Server moves share_id
// out of the item JSON on every API save, so the JSON copy is absent or stale for any
// item that has been edited. The JSON is only a fallback for items stamped directly in
// the database.
//
// `items` also holds binary resource blobs (jop_type 0, e.g. every uploaded
// image/PDF). Those are not valid UTF-8, so convert_from() raises
// "invalid byte sequence" and aborts the WHOLE statement — and SQL gives no
// "invalid byte sequence" and aborts the WHOLE statement - and SQL gives no
// guarantee that a sibling `jop_type = ...` predicate runs first. The CASE
// guarantees only note/folder rows are ever decoded. (With an unguarded
// expression every share fan-out / revoke silently did nothing once the
// account owned a single uploaded attachment: the callers swallow errors.)
const shareIdOf = (alias = '') => `(CASE WHEN ${alias}jop_type IN (1, 2) THEN convert_from(${alias}content, 'UTF8')::json->>'share_id' END)`;
const shareIdOf = (alias = '') => `(COALESCE(NULLIF(${alias}jop_share_id, ''), CASE WHEN ${alias}jop_type IN (1, 2) THEN convert_from(${alias}content, 'UTF8')::json->>'share_id' END))`;
const newId = () => randomBytes(16).toString('hex');
@ -201,24 +206,59 @@ const revokeRecipientAccess = async (database, shareId, itemIds) => {
// a database (unit tests with fakes).
// shareId the share the items now belong to ('' = none)
// previousShareId the share they belonged to before ('' = none)
const createShareSync = ({ itemService, database }) => ({
// A notebook and everything beneath it changes share.
async setSubtreeShare({ ownerId, folderId, shareId = '', previousShareId = '' }) {
if (!database) return;
const { folderIds, noteIds } = await collectSubtreeItems(itemService, database, ownerId, folderId);
const ids = folderIds.concat(noteIds);
await setShareOnItems(database, ownerId, ids, shareId);
if (previousShareId && previousShareId !== shareId) await revokeRecipientAccess(database, previousShareId, ids);
if (shareId) await grantRecipientAccess(database, ownerId, shareId, folderId);
},
// Individual items (e.g. notes moved out of a deleted notebook) change share.
async setItemsShare({ ownerId, itemIds, shareId = '', previousShareId = '', rootFolderId = '' }) {
if (!database || !itemIds || !itemIds.length) return;
await setShareOnItems(database, ownerId, itemIds, shareId);
if (previousShareId && previousShareId !== shareId) await revokeRecipientAccess(database, previousShareId, itemIds);
if (shareId) await grantRecipientAccess(database, ownerId, shareId, rootFolderId);
},
});
const createShareSync = ({ itemService, itemWriteService, database }) => {
// Re-save every notebook and note of the subtree THROUGH THE API with the new share
// fields. This is what makes stock Joplin clients follow along: Joplin Server only
// adds/removes a recipient's copy (and tells their client to delete it) when it
// sees a change event with previous_share_id, and only API writes create those.
// Direct database writes are silent, so a subtree moved OUT of a share used to
// stay in the recipient's Joplin app. Items the API cannot rewrite (Joplin
// end-to-end encrypted ones) are left to the database stamping below.
const stampThroughApi = async ({ ownerId, folderIds, noteIds, shareId, sessionId, requestContext }) => {
if (!itemWriteService || !sessionId) return;
const fields = { shareId, isShared: !!shareId };
const skipped = [];
for (const id of folderIds) {
const folder = await itemService.folderByUserIdAndJopId(ownerId, id).catch(() => null);
if (!folder || folder.e2ee || (folder.shareId || '') === shareId) continue;
try { await itemWriteService.updateFolder(sessionId, folder, fields, requestContext); } catch (error) { skipped.push(error); }
}
for (const id of noteIds) {
const note = await itemService.noteByUserIdAndJopId(ownerId, id, { deleted: 'all' }).catch(() => null);
if (!note || note.e2ee || (note.shareId || '') === shareId) continue;
try { await itemWriteService.updateNote(sessionId, note, fields, requestContext); } catch (error) { skipped.push(error); }
}
if (skipped.length) throw skipped[0];
};
return {
// A notebook and everything beneath it changes share.
async setSubtreeShare({ ownerId, folderId, shareId = '', previousShareId = '', sessionId, requestContext }) {
if (!database) return;
const { folderIds, noteIds } = await collectSubtreeItems(itemService, database, ownerId, folderId);
const ids = folderIds.concat(noteIds);
// API first (it compares against the current share), then the database sweep
// for anything the API could not write, then Joplock's own recipient rows.
let apiError = null;
try {
await stampThroughApi({ ownerId, folderIds, noteIds, shareId, sessionId, requestContext });
} catch (error) { apiError = error; }
await setShareOnItems(database, ownerId, ids, shareId);
if (previousShareId && previousShareId !== shareId) await revokeRecipientAccess(database, previousShareId, ids);
if (shareId) await grantRecipientAccess(database, ownerId, shareId, folderId);
if (apiError) throw apiError;
},
// Individual items (e.g. notes moved out of a deleted notebook) change share. The
// caller has already written them through the API; this keeps Joplock's own
// recipient rows and the stored JSON in step.
async setItemsShare({ ownerId, itemIds, shareId = '', previousShareId = '', rootFolderId = '' }) {
if (!database || !itemIds || !itemIds.length) return;
await setShareOnItems(database, ownerId, itemIds, shareId);
if (previousShareId && previousShareId !== shareId) await revokeRecipientAccess(database, previousShareId, itemIds);
if (shareId) await grantRecipientAccess(database, ownerId, shareId, rootFolderId);
},
};
};
const createShareUpstream = async (ctx, sessionId, notebookId) => {
// Joplin Server has used both folder_id and notebook_id historically.

2
app/templates/pages.js
View file

@ -10,7 +10,7 @@ const {
} = require('./shared');
const { noteMetaFragment } = require('./fragments');
const ASSET_VERSION = '20261008nestedfolders';
const ASSET_VERSION = '20261009joplinfidelity';
const noteFontFamilyCSS = (settings) => {
const f = settings.noteFontFamily || 'sans';

217
integration-tests/joplin-guards.test.js Normal file
View file

@ -0,0 +1,217 @@
'use strict';
// Items Joplock must never mangle in a Joplin account: HTML notes, end-to-end
// encrypted items and conflict copies - plus attachments, which the client must
// be able to download like any other resource.
const { describe, it, before, after } = require('node:test');
const assert = require('node:assert/strict');
const crypto = require('node:crypto');
const fs = require('node:fs');
const path = require('node:path');
const os = require('node:os');
const { skipReason, sleep, JoplinClient, connectAccount } = require('./joplin/harness');
const skip = skipReason();
const PREFIX = `jc-grd-${Date.now().toString(36)}`;
const id32 = () => crypto.randomBytes(16).toString('hex');
const TS = '2026-01-02T03:04:05.678Z';
// A plain note as a stock client would upload it.
const rawNote = (id, parent, title, body, extra = []) => [
title, '', body, '',
`id: ${id}`, `parent_id: ${parent}`, `created_time: ${TS}`, `updated_time: ${TS}`,
'is_conflict: 0', 'latitude: 0.00000000', 'longitude: 0.00000000', 'altitude: 0.0000', 'author: ', 'source_url: ',
'is_todo: 0', 'todo_due: 0', 'todo_completed: 0', 'source: joplin-desktop', 'source_application: net.cozic.joplin-desktop',
'application_data: ', 'order: 1767323045678', `user_created_time: ${TS}`, `user_updated_time: ${TS}`,
'encryption_cipher_text: ', 'encryption_applied: 0', 'markup_language: 1', 'is_shared: 0', 'share_id: ',
'conflict_original_id: ', 'master_key_id: ', 'user_data: ', 'deleted_time: 0', ...extra, 'type_: 1',
].join('\n');
describe('HTML notes', { skip, concurrency: false }, () => {
let jp; let client;
before(async () => { jp = await connectAccount(); client = await new JoplinClient('html').init(); });
after(async () => { if (client) client.close(); if (jp) await jp.cleanup(PREFIX); });
it('are rendered sanitized and read-only; content edits are refused, moves are not', async () => {
const book = `${PREFIX}-html`;
const title = `${PREFIX}-clip`;
const html = '<h2>Clipped page</h2><p>hello <b>world</b></p><script>alert("x")</script><img src="x" onerror="alert(1)"><a href="javascript:alert(2)">bad</a>';
await client.run('mkbook', book);
await client.run('use', book);
await client.run('mknote', title);
await client.run('set', title, 'markup_language', '2');
await client.run('set', title, 'body', html);
await client.sync();
const n = await client.note(title);
assert.equal(n.markup_language, 2);
// Joplock shows it rendered and read-only, with the dangerous bits gone.
const ed = await jp.ok('GET', `/fragments/editor/${n.id}`);
assert.match(ed.text, /data-html-note="1"/);
assert.match(ed.text, /HTML note/);
const slot = /data-rendered-body="([^"]*)"/.exec(ed.text);
assert.ok(slot, 'rendered body present');
const rendered = slot[1].replace(/&quot;/g, '"').replace(/&lt;/g, '<').replace(/&gt;/g, '>').replace(/&amp;/g, '&').replace(/&#39;/g, "'");
assert.match(rendered, /<h2>Clipped page<\/h2>/);
assert.match(rendered, /<b>world<\/b>/);
assert.doesNotMatch(rendered, /script|onerror|javascript:|alert/i);
// A content edit is refused and the stored HTML is untouched.
const before = await jp.rawItem(n.id);
const refused = await jp.updateNote(n.id, { title, body: '<p>overwritten</p>' });
assert.equal(refused.status, 403, refused.text);
assert.equal(await jp.rawItem(n.id), before, 'item on the server is byte-identical');
// The real autosave path (form PUT) for a MOVE of an HTML note: allowed, body kept.
const dest = await jp.createFolder(`${PREFIX}-html-dest`);
const moved = await jp.request('PUT', `/fragments/editor/${n.id}`, {
form: { title, body: n.body, parentId: dest, currentFolderId: dest },
});
assert.equal(moved.status, 200, moved.text);
await sleep(300);
await client.sync();
const after = await client.note(title);
assert.equal(after.parent_id, dest);
assert.equal(after.markup_language, 2);
assert.equal(after.body, html, 'HTML body byte-for-byte');
});
});
describe('Joplin end-to-end encrypted items', { skip, concurrency: false }, () => {
let jp;
before(async () => { jp = await connectAccount(); });
after(async () => { if (jp) await jp.cleanup(PREFIX); });
// What a client uploads for an encrypted item: ids, parent and the ciphertext only.
const encryptedItem = (type, id, parent) => [
`id: ${id}`, ...(type === 1 ? [`parent_id: ${parent}`] : []),
'encryption_cipher_text: JED01000007fakeciphertextfakeciphertextfakeciphertext',
'encryption_applied: 1',
`type_: ${type}`,
].join('\n');
it('notes show as a locked placeholder and can never be changed from Joplock', async () => {
const folder = await jp.createFolder(`${PREFIX}-e2ee`);
const id = id32();
await jp.putRawItem(id, encryptedItem(1, id, folder));
const before = await jp.rawItem(id);
assert.match(before, /encryption_applied: 1/);
const headers = await jp.noteHeaders();
const h = headers.find(x => x.id === id);
assert.ok(h, 'listed (not silently dropped)');
assert.match(h.title, /Encrypted note/);
const ed = await jp.ok('GET', `/fragments/editor/${id}`);
assert.match(ed.text, /editor-e2ee/);
assert.doesNotMatch(ed.text, /<form|<textarea/);
const put = await jp.updateNote(id, { title: 'overwrite', body: 'plaintext', parentId: folder });
assert.equal(put.status, 403, put.text);
assert.match(put.text, /end-to-end encrypted/);
const trash = await jp.request('DELETE', `/fragments/notes/${id}`);
assert.equal(trash.status, 403, trash.text);
const hard = await jp.request('DELETE', `/api/web/notes/${id}`);
assert.equal(hard.status, 403, hard.text);
assert.equal(await jp.rawItem(id), before, 'ciphertext on the server is byte-identical');
});
it('notebooks show as locked, cannot be renamed, moved or deleted; a parent holding them cannot be deleted half-way', async () => {
const parent = await jp.createFolder(`${PREFIX}-e2ee-parent`);
const encFolder = id32();
await jp.putRawItem(encFolder, [
`id: ${encFolder}`, `parent_id: ${parent}`, 'encryption_cipher_text: JED01000007fakefolderciphertext', 'encryption_applied: 1', 'type_: 2',
].join('\n'));
const folders = await jp.folders();
const f = folders.find(x => x.id === encFolder);
assert.ok(f, 'listed');
assert.match(f.title, /Encrypted notebook/);
const before = await jp.rawItem(encFolder);
assert.equal((await jp.request('PUT', `/api/web/folders/${encFolder}`, { json: { title: 'x' } })).status, 403);
assert.equal((await jp.request('PUT', `/api/web/folders/${encFolder}`, { json: { parentId: '' } })).status, 403);
assert.equal((await jp.request('DELETE', `/api/web/folders/${encFolder}`)).status, 403);
const del = await jp.request('DELETE', `/api/web/folders/${parent}`);
assert.equal(del.status, 409, del.text);
assert.equal(await jp.rawItem(encFolder), before, 'untouched');
assert.ok((await jp.folders()).some(x => x.id === parent), 'the parent was not half-deleted');
});
});
describe('Conflict copies', { skip, concurrency: false }, () => {
let jp; let client;
before(async () => { jp = await connectAccount(); client = await new JoplinClient('cf').init(); });
after(async () => { if (client) client.close(); if (jp) await jp.cleanup(PREFIX); });
it('are hidden from Joplock lists like in Joplin, and travel with a deleted notebook', async () => {
const parent = await jp.createFolder(`${PREFIX}-conf`);
const normal = id32();
const conflict = id32();
await jp.putRawItem(normal, rawNote(normal, parent, `${PREFIX}-normal`, 'normal body'));
await jp.putRawItem(conflict, rawNote(conflict, parent, `${PREFIX}-conflict`, 'conflict body', ['is_conflict: 1']).replace('is_conflict: 0\n', '').replace('conflict_original_id: ', `conflict_original_id: ${normal}`));
const headers = await jp.noteHeaders();
assert.ok(headers.some(h => h.id === normal), 'normal note listed');
assert.ok(!headers.some(h => h.id === conflict), 'conflict note hidden from the note list');
const inFolder = (await jp.ok('GET', `/api/web/notes?folderId=${parent}`)).data.items;
assert.ok(!inFolder.some(n => n.id === conflict));
// the real client still has it (as a conflict)
await client.sync();
const theirs = await client.noteById(conflict);
assert.ok(theirs, 'the client has the conflict copy');
assert.equal(theirs.is_conflict, 1);
// deleting the notebook moves BOTH notes (the hidden one must not be left pointing at nothing)
assert.equal((await jp.request('DELETE', `/api/web/folders/${parent}`)).status, 204);
await sleep(300);
await client.sync();
const normalAfter = await client.noteById(normal);
const conflictAfter = await client.noteById(conflict);
assert.notEqual(normalAfter.parent_id, parent, 'normal note moved out');
assert.notEqual(conflictAfter.parent_id, parent, 'conflict copy moved out too');
assert.equal(conflictAfter.is_conflict, 1, 'and is still a conflict copy');
});
});
describe('Attachments uploaded in Joplock', { skip, concurrency: false }, () => {
let jp; let client; let tmp;
before(async () => { jp = await connectAccount(); client = await new JoplinClient('res').init(); tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'jc-res-')); });
after(async () => { if (client) client.close(); if (jp) await jp.cleanup(PREFIX); if (tmp) fs.rmSync(tmp, { recursive: true, force: true }); });
// 1x1 PNG
const PNG = Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==', 'base64');
it('are downloaded by the client and look exactly like client-made attachments', async () => {
// reference: an attachment made by the real client
const book = `${PREFIX}-res`;
await client.run('mkbook', book);
await client.run('use', book);
await client.run('mknote', `${PREFIX}-res-note`);
const file = path.join(tmp, 'ref.png');
fs.writeFileSync(file, PNG);
await client.run('attach', `${PREFIX}-res-note`, file);
await client.sync();
const [ref] = await client.all('SELECT * FROM resources WHERE filename = ? OR title = ?', ['ref.png', 'ref.png']);
assert.ok(ref, 'client-made resource exists');
// the same image uploaded through Joplock's UI endpoint, linked from a note
const resId = await jp.uploadResource('upl.png', 'image/png', PNG);
const folder = (await jp.folders()).find(f => f.title === book);
await jp.createNote(`${PREFIX}-res-note2`, `![upl.png](:/${resId})`, folder.id);
await sleep(400);
await client.sync();
const mine = (await client.all('SELECT * FROM resources WHERE id = ?', [resId]))[0];
assert.ok(mine, 'the client received the resource record');
assert.equal(mine.mime, 'image/png');
assert.equal(mine.size, PNG.length);
assert.equal(mine.file_extension, 'png');
for (const field of ['encryption_applied', 'encryption_blob_encrypted', 'ocr_status', 'ocr_driver_id', 'ocr_error', 'ocr_text', 'is_shared']) {
assert.equal(mine[field], ref[field], `${field} matches what Joplin itself writes`);
}
const blob = path.join(client.profile, 'resources', `${resId}.png`);
assert.ok(fs.existsSync(blob), 'the client downloaded the attachment');
assert.deepEqual(fs.readFileSync(blob), PNG, 'byte-identical');
});
});

139
integration-tests/joplin-nested.test.js Normal file
View file

@ -0,0 +1,139 @@
'use strict';
// Nested notebooks between Joplock and the real Joplin client, both directions.
const { describe, it, before, after } = require('node:test');
const assert = require('node:assert/strict');
const { skipReason, sleep, JoplinClient, connectAccount } = require('./joplin/harness');
const skip = skipReason();
const PREFIX = `jc-nest-${Date.now().toString(36)}`;
describe('Nested notebooks between Joplock and a Joplin client', { skip, concurrency: false }, () => {
let jp; let client;
before(async () => { jp = await connectAccount(); client = await new JoplinClient('nest').init(); });
after(async () => { if (client) client.close(); if (jp) await jp.cleanup(PREFIX); });
const pull = async () => { await sleep(300); await client.sync(); };
const parentOf = async id => (await jp.folders()).find(f => f.id === id)?.parentId;
it('a tree built in the Joplin client appears nested in Joplock', async () => {
const [a, b, c] = ['a', 'b', 'c'].map(x => `${PREFIX}-t1${x}`);
await client.run('mkbook', a);
await client.run('mkbook', '-p', a, b);
await client.run('mkbook', '-p', b, c);
await client.run('use', c);
await client.run('mknote', `${PREFIX}-t1-leaf`);
await client.sync();
const [fa, fb, fc] = await Promise.all([a, b, c].map(t => client.folder(t)));
assert.equal(fb.parent_id, fa.id, 'the client itself nested them');
const folders = await jp.folders();
const byId = Object.fromEntries(folders.map(f => [f.id, f]));
assert.equal(byId[fa.id].parentId, '');
assert.equal(byId[fb.id].parentId, fa.id);
assert.equal(byId[fc.id].parentId, fb.id);
// and the nav really renders them nested
const nav = (await jp.ok('GET', '/fragments/nav')).text;
assert.match(nav, new RegExp(`data-folder-id="${fc.id}"[^>]*data-parent-id="${fb.id}"[^>]*data-depth="2"`));
const leaf = (await jp.noteHeaders()).find(n => n.title === `${PREFIX}-t1-leaf`);
assert.equal(leaf.parentId, fc.id);
});
it('a tree built in Joplock arrives nested in the Joplin client', async () => {
const a = await jp.createFolder(`${PREFIX}-t2a`);
const b = await jp.createFolder(`${PREFIX}-t2b`, a);
const c = await jp.createFolder(`${PREFIX}-t2c`, b);
await jp.createNote(`${PREFIX}-t2-leaf`, 'leaf body', c);
await pull();
assert.equal((await client.folderById(a)).parent_id, '');
assert.equal((await client.folderById(b)).parent_id, a);
assert.equal((await client.folderById(c)).parent_id, b);
assert.equal((await client.note(`${PREFIX}-t2-leaf`)).parent_id, c);
});
it('moves made in Joplock reach the client, and moves made in the client reach Joplock', async () => {
const a = await jp.createFolder(`${PREFIX}-t3a`);
const b = await jp.createFolder(`${PREFIX}-t3b`);
const c = await jp.createFolder(`${PREFIX}-t3c`, a);
await pull();
// Joplock: c -> under b
assert.equal((await jp.request('PUT', `/api/web/folders/${c}`, { json: { parentId: b } })).status, 200);
await pull();
assert.equal((await client.folderById(c)).parent_id, b);
// client: c -> top level, then under a again
await client.run('mv', c, 'root');
await client.sync();
await sleep(300);
assert.equal(await parentOf(c), '', 'client moved it to the top level');
await client.run('mv', c, a);
await client.sync();
await sleep(300);
assert.equal(await parentOf(c), a);
});
it('both refuse to nest a notebook under its own descendant', async () => {
const a = await jp.createFolder(`${PREFIX}-t4a`);
const b = await jp.createFolder(`${PREFIX}-t4b`, a);
await pull();
await assert.rejects(client.run('mv', a, b), /Cannot move/i, 'Joplin refuses');
const r = await jp.request('PUT', `/api/web/folders/${a}`, { json: { parentId: b } });
assert.equal(r.status, 400, 'Joplock refuses too');
assert.equal(await parentOf(a), '');
});
it('renames keep the nesting and travel both ways', async () => {
const a = await jp.createFolder(`${PREFIX}-t5a`);
const b = await jp.createFolder(`${PREFIX}-t5b`, a);
await pull();
assert.equal((await jp.request('PUT', `/api/web/folders/${b}`, { json: { title: `${PREFIX}-t5b-renamed` } })).status, 200);
await pull();
const rb = await client.folderById(b);
assert.equal(rb.title, `${PREFIX}-t5b-renamed`);
assert.equal(rb.parent_id, a, 'still nested after a Joplock rename');
await client.run('ren', b, `${PREFIX}-t5b-again`);
await client.sync();
await sleep(300);
const jb = (await jp.folders()).find(f => f.id === b);
assert.equal(jb.title, `${PREFIX}-t5b-again`);
assert.equal(jb.parentId, a);
});
it('deleting a middle notebook in Joplock promotes its children and notes; the client converges', async () => {
const a = await jp.createFolder(`${PREFIX}-t6a`);
const b = await jp.createFolder(`${PREFIX}-t6b`, a);
const c = await jp.createFolder(`${PREFIX}-t6c`, b);
const nb = await jp.createNote(`${PREFIX}-t6-note-b`, 'in the middle', b);
const nc = await jp.createNote(`${PREFIX}-t6-note-c`, 'in the leaf', c);
await pull();
assert.equal((await jp.request('DELETE', `/api/web/folders/${b}`)).status, 204);
await pull();
assert.equal(await client.folderById(b), undefined, 'the deleted notebook is gone from the client');
assert.equal((await client.folderById(c)).parent_id, a, 'its child was promoted');
assert.equal((await client.noteById(nb)).parent_id, a, 'its note moved to the parent');
assert.equal((await client.noteById(nc)).parent_id, c, 'the child keeps its own note');
});
it('Joplin 3 trashing a notebook with sub-notebooks removes the whole subtree from Joplock', async () => {
const a = `${PREFIX}-t7a`;
await client.run('mkbook', a);
await client.run('mkbook', '-p', a, `${PREFIX}-t7b`);
await client.run('use', `${PREFIX}-t7b`);
await client.run('mknote', `${PREFIX}-t7-note`);
await client.sync();
const fa = await client.folder(a);
const fb = await client.folder(`${PREFIX}-t7b`);
await sleep(300);
assert.ok((await jp.folders()).some(f => f.id === fb.id), 'visible first');
await client.run('rmbook', '-f', a);
await client.sync();
await sleep(400);
const ids = (await jp.folders()).map(f => f.id);
assert.ok(!ids.includes(fa.id) && !ids.includes(fb.id), 'neither level is listed any more');
assert.ok(!(await jp.noteHeaders()).some(n => n.title === `${PREFIX}-t7-note` && !n.deletedTime), 'the note is not in a live notebook');
});
});

136
integration-tests/joplin-sharing.test.js Normal file
View file

@ -0,0 +1,136 @@
'use strict';
// Shared NESTED notebooks: owner works in Joplock, the recipient is a stock Joplin
// client. Joplin Server applies share changes on a 10 second schedule, so every
// step waits for it exactly like a real session would.
//
// What this guards: a stock client only follows share changes that come with a
// sync change event (an API write). Updating descendants with silent database
// writes used to leave a subtree that was moved OUT of a share sitting in the
// recipient's Joplin app.
const { describe, it, before, after } = require('node:test');
const assert = require('node:assert/strict');
const { skipReason, sleep, Joplock, JoplinClient, ACCOUNT } = require('./joplin/harness');
const skip = skipReason();
const PREFIX = `jc-shr-${Date.now().toString(36)}`;
const SHARE_TASK_MS = 13000; // the server task runs every 10s
describe('Shared nested notebooks reach a stock Joplin client', { skip, concurrency: false }, () => {
let owner; let client; let shareId;
const ids = {};
before(async () => {
owner = await Joplock.login(process.env.PLAYWRIGHT_ADMIN_EMAIL || process.env.JOPLOCK_ADMIN_EMAIL, process.env.PLAYWRIGHT_ADMIN_PASSWORD || process.env.JOPLOCK_ADMIN_PASSWORD);
client = await new JoplinClient('share').init();
});
after(async () => {
if (client) client.close();
if (owner) {
if (shareId) await owner.request('DELETE', `/api/web/shares/${shareId}`);
await owner.cleanup(PREFIX);
}
});
// what the recipient's Joplin app holds for this run
const view = async () => {
await sleep(SHARE_TASK_MS);
await client.sync();
await sleep(400);
await client.sync();
const folders = (await client.all('SELECT id, title, parent_id, share_id FROM folders WHERE title LIKE ?', [`${PREFIX}%`]));
const notes = (await client.all('SELECT id, title, parent_id FROM notes WHERE title LIKE ?', [`${PREFIX}%`]));
return { folders, notes, folderTitles: folders.map(f => f.title.replace(`${PREFIX}-`, '')).sort(), noteTitles: notes.map(n => n.title.replace(`${PREFIX}-`, '')).sort() };
};
it('shares the whole subtree: notebooks keep their nesting and share id, notes come along', async () => {
ids.root = await owner.createFolder(`${PREFIX}-root`);
ids.sub = await owner.createFolder(`${PREFIX}-sub`, ids.root);
ids.leaf = await owner.createFolder(`${PREFIX}-leaf`, ids.sub);
await owner.createNote(`${PREFIX}-note-root`, 'r', ids.root);
ids.noteLeaf = await owner.createNote(`${PREFIX}-note-leaf`, 'l', ids.leaf);
const share = await owner.ok('POST', '/api/web/shares', { json: { notebookId: ids.root } });
shareId = share.data.id;
await owner.ok('POST', `/api/web/shares/${shareId}/invites`, { json: { email: ACCOUNT.email, can_write: 1 } });
const v = await view();
assert.deepEqual(v.folderTitles, ['leaf', 'root', 'sub']);
assert.deepEqual(v.noteTitles, ['note-leaf', 'note-root']);
const byId = Object.fromEntries(v.folders.map(f => [f.id, f]));
assert.equal(byId[ids.root].parent_id, '');
assert.equal(byId[ids.sub].parent_id, ids.root);
assert.equal(byId[ids.leaf].parent_id, ids.sub);
for (const f of v.folders) assert.equal(f.share_id, shareId, `${f.title} carries the share id`);
});
it('a notebook (with sub-notebook and notes) moved INTO the share arrives; moved back OUT it disappears with all of it', async () => {
ids.priv = await owner.createFolder(`${PREFIX}-priv`);
ids.privKid = await owner.createFolder(`${PREFIX}-privkid`, ids.priv);
await owner.createNote(`${PREFIX}-note-priv`, 'p', ids.priv);
await owner.createNote(`${PREFIX}-note-kid`, 'k', ids.privKid);
assert.equal((await owner.request('PUT', `/api/web/folders/${ids.priv}`, { json: { parentId: ids.sub } })).status, 200);
let v = await view();
for (const t of ['priv', 'privkid']) assert.ok(v.folderTitles.includes(t), `${t} arrived`);
for (const t of ['note-priv', 'note-kid']) assert.ok(v.noteTitles.includes(t), `${t} arrived`);
assert.equal((await owner.request('PUT', `/api/web/folders/${ids.priv}`, { json: { parentId: '' } })).status, 200);
v = await view();
for (const t of ['priv', 'privkid']) assert.ok(!v.folderTitles.includes(t), `${t} is gone from the recipient's Joplin app`);
for (const t of ['note-priv', 'note-kid']) assert.ok(!v.noteTitles.includes(t), `${t} is gone from the recipient's Joplin app`);
assert.ok(v.folderTitles.includes('root') && v.folderTitles.includes('leaf'), 'the rest of the share is untouched');
});
it('notebooks and notes created inside the share reach the recipient; edits made in the Joplin app survive an owner edit in Joplock', async () => {
ids.fresh = await owner.createFolder(`${PREFIX}-fresh`, ids.leaf);
await owner.createNote(`${PREFIX}-note-fresh`, 'f', ids.fresh);
let v = await view();
assert.ok(v.folderTitles.includes('fresh') && v.noteTitles.includes('note-fresh'));
// the recipient edits the owner's note in a nested shared notebook, setting fields Joplock does not model
const target = `${PREFIX}-note-leaf`;
await client.run('set', ids.noteLeaf, 'body', 'edited in the recipient joplin app');
await client.run('set', ids.noteLeaf, 'author', 'Recipient Author');
await client.sync();
await sleep(SHARE_TASK_MS);
assert.equal((await owner.note(ids.noteLeaf)).body, 'edited in the recipient joplin app', "the owner sees the recipient's edit");
// the owner edits it in Joplock: the recipient's fields must survive
const r = await owner.updateNote(ids.noteLeaf, { title: target, body: 'edited again by the owner in joplock', parentId: ids.leaf });
assert.equal(r.status, 200, r.text);
await view();
const back = await client.noteById(ids.noteLeaf);
assert.equal(back.body, 'edited again by the owner in joplock');
assert.equal(back.author, 'Recipient Author', "the recipient's author field survives an owner edit in Joplock");
});
// KNOWN GAP (predates nested notebooks): Joplock decides who may see an item from its
// own `user_items` rows keyed by jop_id, while Joplin Server keys them by items.id. A note
// a collaborator creates in a Joplin app therefore never appears for the share OWNER in
// Joplock (it does appear in the owner's Joplin apps). Fixing it means changing the share
// access model together with Joplock's own removal paths, otherwise removed users would
// keep access. Tracked here as a todo so it stays visible without failing the suite.
it('notes a collaborator creates in the share are visible to the owner in Joplock', { todo: 'share access model: jop_id vs items.id user_items rows' }, async () => {
await client.run('use', `${PREFIX}-leaf`);
await client.run('mknote', `${PREFIX}-note-from-recipient`);
await client.sync();
await sleep(SHARE_TASK_MS);
const theirs = (await owner.noteHeaders()).find(h => h.title === `${PREFIX}-note-from-recipient`);
const recipient = await Joplock.login(ACCOUNT.email, ACCOUNT.password);
await recipient.cleanup(`${PREFIX}-note-from-recipient`);
assert.ok(theirs, "the owner sees the collaborator's note in Joplock");
});
it('stopping the share takes every level away from the recipient and leaves the owner able to edit', async () => {
const stopped = await owner.request('DELETE', `/api/web/shares/${shareId}`);
assert.ok(stopped.status < 300, stopped.text);
shareId = null;
const v = await view();
for (const t of ['root', 'sub', 'leaf', 'fresh']) assert.ok(!v.folderTitles.includes(t), `${t} removed from the recipient`);
const ownerFolders = await owner.folders();
for (const id of [ids.root, ids.sub, ids.leaf]) assert.equal(ownerFolders.find(f => f.id === id).shareId || '', '', 'owner side share id cleared');
const rename = await owner.request('PUT', `/api/web/folders/${ids.sub}`, { json: { title: `${PREFIX}-sub-renamed` } });
assert.equal(rename.status, 200, rename.text);
});
});

13
integration-tests/joplin/harness.js
View file

@ -100,6 +100,19 @@ class Joplock {
if (r.status >= 300) throw new Error(`put raw item ${id}: HTTP ${r.status} ${r.text.slice(0, 200)}`);
}
// Upload an attachment the way the browser UI does (POST /fragments/upload).
async uploadResource(filename, mime, data) {
const boundary = `----jcupload${Date.now()}`;
const head = Buffer.from(`--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="${filename}"\r\nContent-Type: ${mime}\r\n\r\n`);
const tail = Buffer.from(`\r\n--${boundary}--\r\n`);
const r = await this.request('POST', '/fragments/upload', {
headers: { 'Content-Type': `multipart/form-data; boundary=${boundary}` },
raw: Buffer.concat([head, data, tail]),
});
if (r.status !== 200) throw new Error(`upload failed: ${r.status} ${r.text.slice(0, 200)}`);
return r.data.resourceId;
}
// Remove everything whose title starts with the given prefix (best effort).
async cleanup(prefix) {
try {

82
package.json
View file

@ -1,43 +1,43 @@
{
"name": "joplock",
"private": true,
"version": "0.1.0",
"description": "Thin-client sidecar web UI for stock Joplin Server",
"main": "server.js",
"scripts": {
"start": "node server.js",
"test": "node --test tests/*.test.js",
"build:cm": "cd cm-build && npm install && npm run build",
"build:hljs": "cd hljs-build && npm install && npm run build",
"build:prism": "cd prism-build && npm install && npm run build",
"test:ui": "playwright test",
"test:ui:headed": "playwright test --headed",
"generate:pwa-assets": "node ./scripts/generatePwaAssets.mjs",
"docker:build": "docker build -t joplock .",
"docker:up": "docker compose up -d",
"docker:down": "docker compose down",
"docker:up:build": "docker compose -f docker-compose.example-full-build.yml up -d --build",
"docker:down:build": "docker compose -f docker-compose.example-full-build.yml down",
"docker:up:dev": "docker compose -f docker-compose.dev.yml up -d --build",
"docker:down:dev": "docker compose -f docker-compose.dev.yml down",
"docker:up:full": "docker compose -f docker-compose.example-full.yml up -d",
"docker:down:full": "docker compose -f docker-compose.example-full.yml down",
"setup:joplin-cli": "bash scripts/setup-joplin-cli.sh",
"test:joplin": "node --test --test-concurrency=1 integration-tests/*.test.js"
},
"dependencies": {
"@adobe/css-tools": "4.4.4",
"@mixmark-io/domino": "2.2.0",
"bcryptjs": "^3.0.3",
"html-entities": "1.4.0",
"markdown-it": "^14.1.1",
"pg": "8.16.3",
"qr-image": "^3.2.0",
"tinymce": "^8.6.0"
},
"devDependencies": {
"@playwright/test": "^1.59.1",
"jsdom": "26.1.0",
"sharp": "^0.34.5"
}
"name": "joplock",
"private": true,
"version": "0.1.0",
"description": "Thin-client sidecar web UI for stock Joplin Server",
"main": "server.js",
"scripts": {
"start": "node server.js",
"test": "node --test tests/*.test.js",
"build:cm": "cd cm-build && npm install && npm run build",
"build:hljs": "cd hljs-build && npm install && npm run build",
"build:prism": "cd prism-build && npm install && npm run build",
"test:ui": "playwright test",
"test:ui:headed": "playwright test --headed",
"generate:pwa-assets": "node ./scripts/generatePwaAssets.mjs",
"docker:build": "docker build -t joplock .",
"docker:up": "docker compose up -d",
"docker:down": "docker compose down",
"docker:up:build": "docker compose -f docker-compose.example-full-build.yml up -d --build",
"docker:down:build": "docker compose -f docker-compose.example-full-build.yml down",
"docker:up:dev": "docker compose -f docker-compose.dev.yml up -d --build",
"docker:down:dev": "docker compose -f docker-compose.dev.yml down",
"docker:up:full": "docker compose -f docker-compose.example-full.yml up -d",
"docker:down:full": "docker compose -f docker-compose.example-full.yml down",
"setup:joplin-cli": "bash scripts/setup-joplin-cli.sh",
"test:joplin": "node --test --test-concurrency=1 integration-tests/*.test.js"
},
"dependencies": {
"@adobe/css-tools": "4.4.4",
"@mixmark-io/domino": "2.2.0",
"bcryptjs": "^3.0.3",
"html-entities": "1.4.0",
"markdown-it": "^14.1.1",
"pg": "8.16.3",
"qr-image": "^3.2.0",
"tinymce": "^8.6.0"
},
"devDependencies": {
"@playwright/test": "^1.59.1",
"jsdom": "26.1.0",
"sharp": "^0.34.5"
}
}

186
playwright-tests/joplin-items.spec.js Normal file
View file

@ -0,0 +1,186 @@
'use strict';
// Joplin items Joplock must treat carefully, in the real UI:
// - HTML notes (markup_language 2): shown rendered and read-only, never rewritten
// - end-to-end encrypted notes: locked placeholder, no editor at all
// The items are uploaded the way a stock Joplin client does (X-API-AUTH, sync API).
const crypto = require('node:crypto');
const { test, expect } = require('@playwright/test');
const { acceptDialogs, login, slug, teardownTestData } = require('./helpers');
const id32 = () => crypto.randomBytes(16).toString('hex');
const TS = '2026-01-02T03:04:05.678Z';
const sessionId = async page => (await page.context().cookies()).find(c => c.name === 'sessionId').value;
// Raw sync-API calls use the standalone `request` fixture (no browser cookies) with only
// X-API-AUTH, like a stock Joplin client. page.request would also send Joplock's
// session cookie, which is not what a Joplin app does.
const putRaw = async (page, request, id, text) => {
const boundary = `----pw${Date.now()}`;
const body = `--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="item.md"\r\nContent-Type: text/markdown\r\n\r\n${text}\r\n--${boundary}--\r\n`;
const res = await request.put(`/joplin/api/items/root:/${id}.md:/content`, {
headers: { 'X-API-AUTH': await sessionId(page), 'Content-Type': `multipart/form-data; boundary=${boundary}` },
data: Buffer.from(body),
});
expect(res.status(), await res.text()).toBeLessThan(300);
};
const getRaw = async (page, request, id) => (await request.get(`/joplin/api/items/root:/${id}.md:/content`, { headers: { 'X-API-AUTH': await sessionId(page) } })).text();
const delRaw = async (page, request, id) => request.delete(`/joplin/api/items/root:/${id}.md:`, { headers: { 'X-API-AUTH': await sessionId(page) } }).catch(() => null);
const api = (page, method, url, body) => page.evaluate(async ([m, u, b]) => {
const res = await fetch(u, { method: m, credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: b === undefined ? undefined : JSON.stringify(b) });
const text = await res.text();
let data = null; try { data = JSON.parse(text); } catch { /* html */ }
return { status: res.status, data, text };
}, [method, url, body]);
const noteProps = (id, parent, extra = []) => [
`id: ${id}`, `parent_id: ${parent}`, `created_time: ${TS}`, `updated_time: ${TS}`,
'is_conflict: 0', 'latitude: 0.00000000', 'longitude: 0.00000000', 'altitude: 0.0000', 'author: ', 'source_url: ',
'is_todo: 0', 'todo_due: 0', 'todo_completed: 0', 'source: joplin-desktop', 'source_application: net.cozic.joplin-desktop',
'application_data: ', 'order: 1767323045678', `user_created_time: ${TS}`, `user_updated_time: ${TS}`,
'encryption_cipher_text: ', 'encryption_applied: 0', ...extra, 'is_shared: 0', 'share_id: ',
'conflict_original_id: ', 'master_key_id: ', 'user_data: ', 'deleted_time: 0', 'type_: 1',
].join('\n');
const openNote = async (page, folderTitle, noteTitle) => {
const folder = page.locator(`#nav-panel .nav-folder[data-folder-title="${folderTitle}"]`).first();
await expect(folder).toBeVisible({ timeout: 15000 });
await folder.locator(':scope > .nav-folder-row .nav-folder-toggle').click();
await folder.locator('.notelist-item', { hasText: noteTitle }).first().click();
};
test.describe('HTML notes', () => {
test('desktop: rendered, sanitized, read-only, and never rewritten', async ({ page, request }, testInfo) => {
test.skip(testInfo.project.name !== 'desktop');
acceptDialogs(page);
const folder = slug('pw-joplinitems');
const noteId = id32();
const title = 'Clipped web page';
const html = '<h2>Clipped heading</h2><p>Plain <b>bold</b> text</p><script>window.__pwx=1</script><img src="x" onerror="window.__pwx=2"><a href="javascript:window.__pwx=3">bad</a>';
await login(page);
try {
const folderId = (await api(page, 'POST', '/api/web/folders', { title: folder, parentId: '' })).data.item.id;
await putRaw(page, request, noteId, `${title}\n\n${html}\n\n${noteProps(noteId, folderId, ['markup_language: 2'])}`);
const before = await getRaw(page, request, noteId);
await page.reload();
await openNote(page, folder, title);
const form = page.locator('#editor-panel #note-editor-form[data-html-note="1"]');
await expect(form).toBeVisible({ timeout: 15000 });
await expect(page.locator('#editor-panel .html-note-banner')).toContainText('HTML note');
await expect(page.locator('#editor-panel #markdown-toggle')).toBeHidden();
await expect(page.locator('#editor-panel #preview-toggle')).toBeHidden();
await expect(page.locator('#editor-panel #editor-toolbar')).toBeHidden();
await expect(page.locator('#editor-panel .editor-title')).toHaveAttribute('contenteditable', 'false');
// rendered in TinyMCE, with the hostile parts gone
const frame = page.frameLocator('iframe.tox-edit-area__iframe');
await expect(frame.locator('body')).toContainText('Clipped heading', { timeout: 15000 });
await expect(frame.locator('h2')).toHaveText('Clipped heading');
await expect(frame.locator('script, [onerror]')).toHaveCount(0);
await expect(frame.locator('a[href^="javascript"]')).toHaveCount(0);
expect(await page.evaluate(() => window.__pwx)).toBeUndefined();
// read-only: typing does nothing and nothing gets saved
await frame.locator('body').click({ force: true });
await page.keyboard.type('SHOULD NOT APPEAR');
await expect(frame.locator('body')).not.toContainText('SHOULD NOT APPEAR');
await page.keyboard.press('Control+b');
await page.waitForTimeout(3500); // longer than the 2s autosave debounce
expect(await getRaw(page, request, noteId)).toBe(before);
// the stored HTML is still HTML, byte for byte
expect(before).toMatch(/markup_language: 2/);
expect(await getRaw(page, request, noteId)).toContain('<script>window.__pwx=1</script>');
} finally {
await delRaw(page, request, noteId);
await teardownTestData(page, { folders: [folder] });
}
});
test('mobile: the HTML note opens rendered with no mode switch', async ({ page, request }, testInfo) => {
test.skip(testInfo.project.name !== 'mobile');
acceptDialogs(page);
const folder = slug('pw-joplinitems');
const noteId = id32();
const title = 'Clipped mobile page';
await login(page);
try {
const folderId = (await api(page, 'POST', '/api/web/folders', { title: folder, parentId: '' })).data.item.id;
await putRaw(page, request, noteId, `${title}\n\n<h3>Mobile heading</h3><p>body</p>\n\n${noteProps(noteId, folderId, ['markup_language: 2'])}`);
await page.reload();
await expect(page.locator('#mobile-app[aria-hidden="false"]')).toBeVisible();
await page.locator('#mobile-folders-body .mobile-folder-row', { hasText: folder }).first().click();
await page.locator('#mobile-notes-body .mobile-note-row', { hasText: title }).first().click();
await expect(page.locator('#mobile-editor-screen.mobile-screen-active')).toBeVisible();
await expect(page.locator('#mobile-editor-body .html-note-banner')).toBeVisible({ timeout: 15000 });
await expect(page.locator('#mobile-md-toggle')).toBeHidden();
await expect(page.locator('#mobile-preview-toggle')).toBeHidden();
await expect(page.frameLocator('iframe.tox-edit-area__iframe').locator('h3')).toHaveText('Mobile heading', { timeout: 15000 });
} finally {
await delRaw(page, request, noteId);
await teardownTestData(page, { folders: [folder] });
}
});
test('switching from an HTML note to a markdown note restores the normal editor', async ({ page, request }, testInfo) => {
test.skip(testInfo.project.name !== 'desktop');
acceptDialogs(page);
const folder = slug('pw-joplinitems');
const htmlId = id32();
await login(page);
try {
const folderId = (await api(page, 'POST', '/api/web/folders', { title: folder, parentId: '' })).data.item.id;
await putRaw(page, request, htmlId, `An html note\n\n<p>html</p>\n\n${noteProps(htmlId, folderId, ['markup_language: 2'])}`);
await api(page, 'POST', '/api/web/notes', { title: 'A markdown note', body: '# md body', parentId: folderId });
await page.reload();
await openNote(page, folder, 'An html note');
await expect(page.locator('#editor-panel #note-editor-form[data-html-note="1"]')).toBeVisible({ timeout: 15000 });
await page.locator('#nav-panel .notelist-item', { hasText: 'A markdown note' }).first().click();
const md = page.locator('#editor-panel #note-editor-form:not([data-html-note])');
await expect(md).toBeVisible({ timeout: 15000 });
await expect(page.locator('#editor-panel .html-note-banner')).toHaveCount(0);
await expect(page.locator('#editor-panel #markdown-toggle')).toBeVisible();
await expect(page.locator('#editor-panel .editor-title')).toHaveAttribute('contenteditable', 'true');
} finally {
await delRaw(page, request, htmlId);
await teardownTestData(page, { folders: [folder] });
}
});
});
test.describe('Joplin end-to-end encrypted notes', () => {
test('desktop: locked placeholder in the list and in the editor, nothing to edit', async ({ page, request }, testInfo) => {
test.skip(testInfo.project.name !== 'desktop');
acceptDialogs(page);
const folder = slug('pw-joplinitems');
const noteId = id32();
await login(page);
try {
const folderId = (await api(page, 'POST', '/api/web/folders', { title: folder, parentId: '' })).data.item.id;
await putRaw(page, request, noteId, [`id: ${noteId}`, `parent_id: ${folderId}`, 'encryption_cipher_text: JED01000007fakeciphertextfakeciphertext', 'encryption_applied: 1', 'type_: 1'].join('\n'));
const before = await getRaw(page, request, noteId);
await page.reload();
const f = page.locator(`#nav-panel .nav-folder[data-folder-title="${folder}"]`).first();
await expect(f).toBeVisible({ timeout: 15000 });
await f.locator(':scope > .nav-folder-row .nav-folder-toggle').click();
const item = f.locator('.notelist-item[data-e2ee="1"]').first();
await expect(item).toBeVisible();
await expect(item).toContainText('Encrypted note');
await expect(item.locator('.note-lock-icon')).toBeVisible();
await item.click();
await expect(page.locator('#editor-panel #editor-e2ee')).toBeVisible({ timeout: 15000 });
await expect(page.locator('#editor-panel')).toContainText('end-to-end encrypted by Joplin');
await expect(page.locator('#editor-panel #note-editor-form')).toHaveCount(0);
await expect(page.locator('#editor-panel textarea')).toHaveCount(0);
await page.waitForTimeout(2500);
expect(await getRaw(page, request, noteId)).toBe(before);
} finally {
await delRaw(page, request, noteId);
await teardownTestData(page, { folders: [folder] });
}
});
});

2
public/service-worker.js
View file

@ -1,4 +1,4 @@
const CACHE_NAME = 'joplock-shell-v80-20261008nestedfolders';
const CACHE_NAME = 'joplock-shell-v81-20261009joplinfidelity';
const STATIC_ASSETS = ['/styles.css', '/htmx.min.js', '/turndown.min.js', '/codemirror.min.js', '/cm-extras.js', '/prism.min.js', '/hljs.min.js', '/manifest.webmanifest', '/icon.svg', '/icon-192.png', '/icon-512.png', '/maskable-icon-192.png', '/maskable-icon-512.png', '/apple-touch-icon.png', '/fonts/CascadiaMono-Regular.woff2', '/fonts/CascadiaMono-Bold.woff2'];
self.addEventListener('install', event => {

4
tests/folderOps.test.js
View file

@ -121,14 +121,14 @@ test('moving a notebook INTO a share stamps the subtree and grants recipients',
const r = await w.ops.moveFolder({ user: USER, folderId: 'pkid', targetParentId: 'sub', requestContext: ctx });
assert.equal(r.shareChanged, true);
assert.deepEqual(w.calls[0], ['updateFolder', 'pkid', { parentId: 'sub', shareId: 'S1', isShared: true }]);
assert.deepEqual(w.sync, [['subtree', { ownerId: 'u1', folderId: 'pkid', shareId: 'S1', previousShareId: '' }]]);
assert.deepEqual(w.sync.map(x => [x[0], x[1].ownerId, x[1].folderId, x[1].shareId, x[1].previousShareId, x[1].sessionId]), [['subtree', 'u1', 'pkid', 'S1', '', 's1']], 'the session is passed so descendants can be re-saved through the API');
});
test('moving a notebook OUT of a share clears the subtree and revokes recipients', async () => {
const w = world({ folders: sharedTree() });
await w.ops.moveFolder({ user: USER, folderId: 'sub', targetParentId: 'private', requestContext: ctx });
assert.deepEqual(w.calls[0], ['updateFolder', 'sub', { parentId: 'private', shareId: '', isShared: false }]);
assert.deepEqual(w.sync, [['subtree', { ownerId: 'u1', folderId: 'sub', shareId: '', previousShareId: 'S1' }]]);
assert.deepEqual(w.sync.map(x => [x[0], x[1].ownerId, x[1].folderId, x[1].shareId, x[1].previousShareId, x[1].sessionId]), [['subtree', 'u1', 'sub', '', 'S1', 's1']]);
});
test('moving within the same share changes the parent only (no share churn)', async () => {

38
tests/itemService.test.js
View file

@ -173,3 +173,41 @@ test('item reads select the encryption column so E2EE items can be recognised',
await svc.noteHeadersByFolder('u', 'f1');
for (const sql of sqls) assert.match(sql, /jop_encryption_applied/);
});
// ── share id: the column is authoritative ──
test('share id comes from the jop_share_id column; the JSON copy is only a fallback (it vanishes after the first API save)', () => {
const afterApiSave = mapNote(row({ title: 'T', body: 'b' /* no share_id: the server stripped it */ }, { jop_share_id: 'SHARE1' }));
assert.equal(afterApiSave.shareId, 'SHARE1');
assert.equal(afterApiSave.isShared, true);
const dbStamped = mapNote(row({ title: 'T', share_id: 'SHARE2', is_shared: 1 }, { jop_share_id: '' }));
assert.equal(dbStamped.shareId, 'SHARE2', 'JSON fallback for items stamped directly in the database');
const both = mapNote(row({ title: 'T', share_id: 'STALE' }, { jop_share_id: 'CURRENT' }));
assert.equal(both.shareId, 'CURRENT', 'the column wins over a stale JSON value');
const none = mapNote(row({ title: 'T' }));
assert.equal(none.shareId, '');
assert.equal(none.isShared, false);
assert.equal(mapFolder(row({ title: 'F' }, { jop_share_id: 'S' })).shareId, 'S');
assert.equal(mapHeader({ jop_id: 'n', jop_parent_id: 'f', title: 't', jop_share_id: 'S' }).shareId, 'S');
});
test('an edit round-trip keeps a shared note shared (the regression: 2nd Joplock edit used to write share_id empty)', () => {
const { serializeNote } = require('../app/items/itemWriteService');
const shared = mapNote(row({ title: 'T', body: 'b' }, { jop_share_id: 'SHARE1' }));
const text = serializeNote({ id: shared.id, title: shared.title, body: 'edited', parentId: shared.parentId, createdTime: 1, isShared: shared.isShared, shareId: shared.shareId, fields: shared.fields }).body;
assert.match(text, /^share_id: SHARE1$/m);
assert.match(text, /^is_shared: 1$/m);
});
test('item reads select the share column', async () => {
const { createItemService } = require('../app/items/itemService');
const sqls = [];
const svc = createItemService({ query: async sql => { sqls.push(sql); return { rows: [] }; } });
await svc.foldersByUserId('u');
await svc.folderByUserIdAndJopId('u', 'f');
await svc.notesByUserId('u');
await svc.noteByUserIdAndJopId('u', 'n');
await svc.noteHeadersByUserId('u');
await svc.noteHeadersByFolder('u', 'f1');
await svc.searchNotes('u', 'x');
for (const sql of sqls) assert.match(sql, /jop_share_id/, sql.slice(0, 90));
});

1
tests/joplinItem.test.js
View file

@ -150,6 +150,7 @@ test('serializeResource writes the Joplin 3.x OCR fields', () => {
const parsed = unserialize(serializeResource({ id: 'r'.repeat(32), title: 'a.png', mime: 'image/png', filename: 'a.png', fileExtension: 'png', size: 10 }).body);
for (const key of ['ocr_text', 'ocr_details', 'ocr_status', 'ocr_error', 'ocr_driver_id']) assert.ok(key in parsed, key);
assert.equal(parsed.ocr_status, '0');
assert.equal(parsed.ocr_driver_id, '1', 'Joplin default (PrintedText), not the legacy 0');
assert.equal(parsed.mime, 'image/png');
assert.equal(parsed.type_, '4');
});

81
tests/shareSync.test.js Normal file
View file

@ -0,0 +1,81 @@
// Share changes on a notebook subtree must reach stock Joplin clients, which only
// react to change events (i.e. API writes), never to silent database updates.
const test = require('node:test');
const assert = require('node:assert/strict');
const { createShareSync } = require('../app/routes/shares');
const world = ({ folders, notes, e2ee = [] }) => {
const apiWrites = [];
const sql = [];
const byFolder = Object.fromEntries(folders.map(f => [f.id, { id: f.id, parentId: f.parentId || '', shareId: f.shareId || '', e2ee: e2ee.includes(f.id) }]));
const byNote = Object.fromEntries(notes.map(n => [n.id, { id: n.id, parentId: n.parentId, shareId: n.shareId || '', e2ee: e2ee.includes(n.id) }]));
const itemService = {
foldersByUserId: async () => Object.values(byFolder),
folderByUserIdAndJopId: async (u, id) => byFolder[id] || null,
noteByUserIdAndJopId: async (u, id) => byNote[id] || null,
};
const itemWriteService = {
updateFolder: async (sid, existing, fields) => { if (existing.e2ee) throw Object.assign(new Error('e2ee'), { statusCode: 403 }); apiWrites.push(['folder', existing.id, fields]); },
updateNote: async (sid, existing, fields) => { if (existing.e2ee) throw Object.assign(new Error('e2ee'), { statusCode: 403 }); apiWrites.push(['note', existing.id, fields]); },
};
const database = {
query: async (text, params) => {
sql.push(text);
if (/SELECT jop_id FROM items WHERE owner_id/.test(text)) return { rows: notes.filter(n => params[1].includes(n.parentId)).map(n => ({ jop_id: n.id })) };
return { rows: [] };
},
};
return { sync: createShareSync({ itemService, itemWriteService, database }), apiWrites, sql };
};
const TREE = [{ id: 'root' }, { id: 'sub', parentId: 'root' }, { id: 'leaf', parentId: 'sub' }, { id: 'other' }];
const NOTES = [{ id: 'n1', parentId: 'root' }, { id: 'n2', parentId: 'leaf' }, { id: 'n3', parentId: 'other' }];
test('moving a subtree INTO a share re-saves every notebook and note below it through the API', async () => {
const w = world({ folders: TREE, notes: NOTES });
await w.sync.setSubtreeShare({ ownerId: 'u', folderId: 'root', shareId: 'S1', previousShareId: '', sessionId: 'sess', requestContext: {} });
assert.deepEqual(w.apiWrites.map(x => [x[0], x[1]]).sort(), [['folder', 'leaf'], ['folder', 'root'], ['folder', 'sub'], ['note', 'n1'], ['note', 'n2']]);
for (const [, , fields] of w.apiWrites) assert.deepEqual(fields, { shareId: 'S1', isShared: true });
assert.ok(!w.apiWrites.some(x => x[1] === 'other' || x[1] === 'n3'), 'unrelated items are not touched');
});
test('moving a subtree OUT of a share clears the share id through the API too (this is what makes Joplin clients drop it)', async () => {
const folders = TREE.map(f => (f.id === 'other' ? f : { ...f, shareId: 'S1' }));
const notes = NOTES.map(n => (n.id === 'n3' ? n : { ...n, shareId: 'S1' }));
const w = world({ folders, notes });
await w.sync.setSubtreeShare({ ownerId: 'u', folderId: 'root', shareId: '', previousShareId: 'S1', sessionId: 'sess', requestContext: {} });
assert.equal(w.apiWrites.length, 5);
for (const [, , fields] of w.apiWrites) assert.deepEqual(fields, { shareId: '', isShared: false });
assert.ok(w.sql.some(t => /DELETE FROM user_items/.test(t)), "Joplock's own recipient rows are revoked as well");
});
test('items already in the target share are not rewritten', async () => {
const w = world({ folders: [{ id: 'root', shareId: 'S1' }, { id: 'sub', parentId: 'root' }], notes: [{ id: 'n1', parentId: 'root', shareId: 'S1' }] });
await w.sync.setSubtreeShare({ ownerId: 'u', folderId: 'root', shareId: 'S1', sessionId: 'sess', requestContext: {} });
assert.deepEqual(w.apiWrites.map(x => x[1]), ['sub']);
});
test('end-to-end encrypted items cannot be re-saved: they are skipped by the API pass and swept by the database', async () => {
const w = world({ folders: [{ id: 'root' }], notes: [{ id: 'n1', parentId: 'root' }, { id: 'enc', parentId: 'root' }], e2ee: ['enc'] });
await w.sync.setSubtreeShare({ ownerId: 'u', folderId: 'root', shareId: 'S1', sessionId: 'sess', requestContext: {} });
assert.ok(!w.apiWrites.some(x => x[1] === 'enc'));
assert.ok(w.sql.some(t => /UPDATE items SET/.test(t)), 'database sweep still runs');
});
test('without a session the database sweep still runs (no API available)', async () => {
const w = world({ folders: [{ id: 'root' }], notes: [{ id: 'n1', parentId: 'root' }] });
await w.sync.setSubtreeShare({ ownerId: 'u', folderId: 'root', shareId: 'S1' });
assert.equal(w.apiWrites.length, 0);
assert.ok(w.sql.some(t => /UPDATE items SET/.test(t)));
});
test('an API failure is reported, but only after the database sweep and recipient rows are done', async () => {
const w = world({ folders: [{ id: 'root' }], notes: [{ id: 'n1', parentId: 'root' }] });
const failing = createShareSync({
itemService: { foldersByUserId: async () => [{ id: 'root', parentId: '' }], folderByUserIdAndJopId: async () => ({ id: 'root', shareId: '' }), noteByUserIdAndJopId: async () => null },
itemWriteService: { updateFolder: async () => { throw new Error('upstream down'); } },
database: { query: async (t, p) => { w.sql.push(t); return { rows: [] }; } },
});
await assert.rejects(failing.setSubtreeShare({ ownerId: 'u', folderId: 'root', shareId: 'S1', sessionId: 's', requestContext: {} }), /upstream down/);
assert.ok(w.sql.some(t => /UPDATE items SET/.test(t)), 'the sweep happened before the error surfaced');
});

2
tests/sharesBinaryItems.test.js
View file

@ -29,6 +29,8 @@ test('shareIdOf() only decodes notes and folders and honours a table alias', ()
const m = /const shareIdOf = \(alias = ''\) => `([^`]+)`;/.exec(src);
assert.ok(m, 'shareIdOf helper must exist');
assert.match(m[1], /CASE WHEN \$\{alias\}jop_type IN \(1, 2\) THEN convert_from\(\$\{alias\}content, 'UTF8'\)::json->>'share_id' END/);
// the authoritative column comes first; the JSON decode stays behind the type guard
assert.match(m[1], /^\(COALESCE\(NULLIF\(\$\{alias\}jop_share_id, ''\), CASE WHEN/);
});
test('every share fan-out / revoke query goes through shareIdOf()', () => {