mirror of
https://github.com/abort-retry-ignore/joplock.git
synced 2026-10-19 10:13:51 +00:00
docs: split the agent guide into a short AGENTS.md and on-demand topic files
AGENT_GUIDE.md was ~33k tokens, loaded in full by every agent that followed AGENTS.md's 'read it first', whatever the task. A third of it was lookups an agent can get from the code (file map, route table, env var table, settings key list, static asset list) and had already gone stale in several places. - AGENTS.md (~1.8k tokens, always loaded): what the project is, a 'before you touch X, read Y' table, the rules that always apply, workflow (including that a push to master publishes the Docker image), and the traps that have each broken real data or a real phone. It keeps the tool-managed graft block as is. - docs/agents/*.md (11 files, read on demand): the hard-won 'why' and 'do not regress' contracts by topic, sliced from the old guide with nothing hand-rewritten. The derivable lookups are dropped; the few non-derivable facts from them (git-ignored dev compose, built bundles, reference.docx) moved to testing-and-dev.md. - pointer comments at the top of the contract-heavy source files, so the rule surfaces at the point of editing - tests/agentDocs.test.js keeps the docs honest: AGENTS.md has a size budget, every topic file is linked and says when to read it, and any file, function or npm script a doc names must still exist - README and two code comments no longer point at AGENT_GUIDE.md
This commit is contained in:
parent
2f7e8f2fef
commit
72e2ff98ce
29 changed files with 1141 additions and 1075 deletions
51
AGENTS.md
51
AGENTS.md
|
|
@ -1,6 +1,55 @@
|
|||
# Agents
|
||||
|
||||
Read [`AGENT_GUIDE.md`](./AGENT_GUIDE.md) first. It is the source of truth for repo conventions, architecture, editor model, mobile shell, settings, debugging workflow, and coding rules.
|
||||
Joplock is a standalone web client for a stock, unmodified Joplin Server: Node (no framework), server-rendered HTML with htmx fragment swaps, and one large client file (`public/app.js`). It reads the shared Postgres directly and writes through Joplin Server's own API, so the same account keeps working from Joplin desktop, mobile and CLI.
|
||||
|
||||
This file is loaded on every task, so it is short on purpose. It gives the rules and says where the **reasons** live. It does not say where code lives: use the code index below (it cannot go stale). The topic files in `docs/agents/` hold the hard-won "why" and "do not regress" rules. **Read the one that matches what you are about to change, before you change it.**
|
||||
|
||||
## Before you touch ... read
|
||||
|
||||
| You are changing | Read first |
|
||||
|---|---|
|
||||
| the editor: CodeMirror/TinyMCE, mode switching, autosave, markdown round-trip, a phantom "Edited" | `docs/agents/editor.md` |
|
||||
| any save path, vaults/encryption, note history/restore, anything reading the editor DOM across a note switch | `docs/agents/vaults-and-save-safety.md` |
|
||||
| how notes/notebooks/resources are serialized or written (anything another Joplin client can see) | `docs/agents/joplin-fidelity.md` |
|
||||
| the notebook tree, drag and drop, sort, pins, sharing, `user_items` | `docs/agents/notebooks-and-sharing.md` |
|
||||
| upload, paste or drop of images/attachments | `docs/agents/editor-uploads.md` |
|
||||
| `public/cm-extras.js` or the CodeMirror bundle | `docs/agents/editor-markdown-extras.md` |
|
||||
| mobile layout, PWA, service worker, viewport/keyboard, static serving, anything iOS | `docs/agents/pwa-mobile.md` |
|
||||
| text expander, AI autocomplete, `/ask` | `docs/agents/ai-expander-ask.md` |
|
||||
| settings, themes, admin, backup/recovery, MFA, debug logging | `docs/agents/settings-and-admin.md` |
|
||||
| server shape, htmx/fragment conventions, routes, data model, or you are unsure where something belongs | `docs/agents/architecture.md` |
|
||||
| tests, Playwright, the dev stack, "my change does not show up" | `docs/agents/testing-and-dev.md` |
|
||||
|
||||
## Rules that always apply
|
||||
|
||||
1. Do not modify Joplin Server source for Joplock features unless explicitly approved.
|
||||
2. The server is authoritative and the browser is ephemeral: no browser-local authoritative storage, and logout must clear client-visible state (shared machines).
|
||||
3. Stay sync-compatible with normal Joplin clients. A Joplock write changes **only what the user edited** (`docs/agents/joplin-fidelity.md`).
|
||||
4. Keep the sidecar API app-oriented; do not expose the raw sync/storage model to the frontend.
|
||||
5. Keep changes minimal. Fix the cause, not the symptom, and add a test that fails without the fix. When you fix mobile, check desktop (and the reverse): they share the editor fragment.
|
||||
|
||||
## Workflow
|
||||
|
||||
- `npm test` before every commit (CI runs it on Node 20; this machine may run a newer Node).
|
||||
- **Pushing to `master` publishes the Docker image**: GitHub Actions runs the tests, then builds and pushes `ghcr.io/abort-retry-ignore/joplock:latest`. Do not commit to `master` or push unless the user asks.
|
||||
- Source edits need `./scripts/rebuild-dev.sh` (the image copies `app/`, `public/` and `server.js` at build time; a container restart is not enough). Browser tests (`npm run test:ui`) need that dev stack and `JOPLOCK_ADMIN_*` in the environment; see `docs/agents/testing-and-dev.md`.
|
||||
- `plans/` is gitignored local history, **not documentation**; `plans/README.md` lists what is stale. Trust the code and these docs over any plan.
|
||||
- Asset versions and the service worker cache name are computed (`app/assetVersion.js`): never hand-edit or "bump" them.
|
||||
- After large changes run `graft build` so the code index matches.
|
||||
|
||||
## Traps that have each broken real data or a real phone
|
||||
|
||||
One line each; the topic file has the detail.
|
||||
|
||||
- **Inline `onclick="foo()"` only sees `window.foo`.** A function works in a unit test and is "not defined" in the browser unless `public/app.js` assigns it to `window` (`tests/inlineHandlersExposed.test.js`).
|
||||
- **Never write one note's text into another.** Debounced saves, async fetches and TinyMCE are shared across note switches; every save path has an identity guard (`vaults-and-save-safety.md`).
|
||||
- **The markdown <-> TinyMCE round trip must be idempotent**, or notes show a phantom "Edited" on open and get re-saved (`editor.md`).
|
||||
- **Do not decode `items.content` for every item in SQL**: it also holds binary resources and one attachment aborts the statement (`notebooks-and-sharing.md`).
|
||||
- **iOS home-screen apps**: never set `apple-mobile-web-app-status-bar-style: black-translucent`; never guess a keyboard from `innerHeight - visualViewport.height`; decide mobile vs desktop from the physical screen, never the window height (`pwa-mobile.md`). Chromium cannot reproduce any of this: measure on a device with `/diag`.
|
||||
- **DOM ids and classes are a contract with inline JS and tests.** Rename them deliberately.
|
||||
- **Floating menus need an opaque background and edge clamping**; pure `:hover` styles go inside `@media (hover: hover)` (`architecture.md`, `pwa-mobile.md`).
|
||||
|
||||
## Code index
|
||||
|
||||
<!-- graft:start -->
|
||||
## Graft — repo context graph
|
||||
|
|
|
|||
1070
AGENT_GUIDE.md
1070
AGENT_GUIDE.md
File diff suppressed because it is too large
Load diff
|
|
@ -142,8 +142,7 @@ The full example is meant as a working reference compose file. Adjust it for you
|
|||
|
||||
### Development
|
||||
|
||||
`AGENT_GUIDE.md` is the source of truth for repo conventions, architecture, the
|
||||
editor model, and debugging workflow. `docs/joplock-db-schema.html` documents the
|
||||
`AGENTS.md` holds the repo rules, and `docs/agents/` the reasons and "do not regress" contracts by topic (it is written for coding agents, short and on-demand). `docs/joplock-db-schema.html` documents the
|
||||
four sidecar tables Joplock adds to the shared database.
|
||||
|
||||
For local work, keep a git-ignored `docker-compose.dev.yml` (Postgres + Joplin
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// Notebook rules and why: docs/agents/notebooks-and-sharing.md
|
||||
|
||||
// Notebook operations that must be consistent across every entry point (the
|
||||
// htmx fragment routes, the JSON API and, on mobile, the same routes again).
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// Tree/sort/pin semantics and why: docs/agents/notebooks-and-sharing.md
|
||||
|
||||
// Pure helpers for Joplin's nested notebooks (a notebook is nested when its
|
||||
// `parent_id` points at another notebook).
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
// A write must change only what the user edited: docs/agents/joplin-fidelity.md (read before changing what is written)
|
||||
const http = require('http');
|
||||
const { randomBytes } = require('crypto');
|
||||
const { serializeItem, MODEL_TYPE_NOTE, MODEL_TYPE_FOLDER, MODEL_TYPE_RESOURCE } = require('./joplinItem');
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// A write must change only what the user edited: docs/agents/joplin-fidelity.md (read before changing serialization)
|
||||
|
||||
// Joplin item serialization, mirroring BaseItem.serialize / serialize_format from
|
||||
// the Joplin client so what Joplock writes is indistinguishable from what a Joplin
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// Sharing model and its known gaps: docs/agents/notebooks-and-sharing.md
|
||||
|
||||
const deriveShareFieldsForMove = (targetFolder) => {
|
||||
const shareId = (targetFolder && targetFolder.shareId) ? targetFolder.shareId : '';
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// Share write guards, and their limits: docs/agents/notebooks-and-sharing.md, docs/agents/joplin-fidelity.md
|
||||
|
||||
/**
|
||||
* shareProxyGuard.js
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// Vault boundary rules, and why this guard is inert for stock Joplin clients today: docs/agents/vaults-and-save-safety.md, docs/agents/joplin-fidelity.md
|
||||
|
||||
/**
|
||||
* vaultProxyGuard.js
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// Static serving (serveFile) and the service worker template: docs/agents/pwa-mobile.md; vault write guard: docs/agents/vaults-and-save-safety.md
|
||||
|
||||
const http = require('http');
|
||||
const fs = require('fs');
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
'use strict';
|
||||
// Sharing model, and why share SQL must never decode items.content for every item: docs/agents/notebooks-and-sharing.md
|
||||
|
||||
const { randomBytes } = require('crypto');
|
||||
const { sendJson, parseBody } = require('./_helpers');
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
// A setting persists only if normalized here AND allow-listed in app/routes/api.js: docs/agents/settings-and-admin.md
|
||||
const validThemes = ['aritim-dark','dark','matrix-amber','matrix-blue','dark-fruit','dark-grey','matrix-orange','matrix-purple','dark-red','dracula','earth','fireball','fruit','grey','light','matrix','nord','oled-dark','solarized-dark','solarized-light','swamp-thing'];
|
||||
|
||||
// Provider templates — preset URL and default model per service.
|
||||
|
|
|
|||
49
docs/agents/ai-expander-ask.md
Normal file
49
docs/agents/ai-expander-ask.md
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
# Text expander, AI autocomplete and /ask
|
||||
|
||||
> Read this before: changing text expansion, AI provider profiles, prose autocomplete or the `/ask` slash command.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Expander / AI Autocomplete
|
||||
|
||||
- Expander entries live in per-user `textExpanders` settings and are configured in Settings -> Expander.
|
||||
- Expander triggers are always on; there is no global autocomplete enable/disable toggle.
|
||||
- Trigger strings are normalized in `app/settingsService.js`, must be non-empty, deduplicated, and are capped at 15 characters.
|
||||
- Expander entry shape is `{ id, trigger, action, profileId, text }`.
|
||||
- `action: 'text'` replaces the trigger with `text`; empty text entries are discarded during normalization.
|
||||
- `action: 'ai'` removes the trigger and launches prose autocomplete. `profileId` selects an AI profile, or falls back to the active profile when blank.
|
||||
- AI autocomplete triggers are not configured in the AI tab. The AI tab owns provider profiles and sentence count; the Expander tab owns trigger strings.
|
||||
- Legacy manual suffix triggers (`double-q`, `triple-space`, `ellipsis`) and robot toggle UI were removed. Do not reintroduce `proseAutocompleteManualTrigger`, `proseAutocompleteManualTriggerOptions`, or `autocompleteEnabled`.
|
||||
- `Ctrl-Space` / `Mod-Space` remains a keyboard shortcut path for manual prose completion, separate from Expander suffix triggers.
|
||||
- Note-link autocomplete (`[[...`) remains separate from AI prose autocomplete.
|
||||
|
||||
## AI Provider Profiles
|
||||
|
||||
- AI provider profiles live in `aiProfiles` and are normalized in `app/settingsService.js`.
|
||||
- Profiles are user-defined; `defaultAiProfiles` is intentionally empty.
|
||||
- Each profile can specify provider, API URL/model, API key, temperature, and active state.
|
||||
- Legacy `openRouterApiKey` / `openRouterModel` are still migrated into an OpenRouter profile for backward compatibility.
|
||||
- `/api/web/ai/prose-complete` accepts optional `profileId` and falls back to the active profile or first keyed profile.
|
||||
- Autocomplete provider requests include `reasoning: { enabled: false }` in the chat completion payload.
|
||||
- Server-side completion post-processing strips repeated prompt prefixes/suffixes, collapses adjacent repeated phrases, limits sentence count, and reports empty-completion diagnostics with `emptyReason`.
|
||||
- Empty completion reasons are `provider-empty`, `provider-repeated-existing-text`, and `trimmed-no-complete-sentence`.
|
||||
|
||||
## `/ask` Slash Command
|
||||
|
||||
- A line starting with `/ask <question>` (must be at column 0, non-empty question after the space/tab) fires a direct Q&A request when Enter is pressed with the caret at the end of that line. Works in both CM6 and TinyMCE.
|
||||
- Detection: `detectAskCommand(lineText)` in `public/app.js`. Trigger wiring: the `Enter` binding in the CM6 `keymap.of([...])` inside `initCM()`, and an `e.key==='Enter'` check at the top of the TinyMCE `editor.on('keydown', ...)` handler (same handler that wires `Ctrl/Cmd-Space`).
|
||||
- Flow: `/ask …` line/block is replaced with a `⏳ Asking…` placeholder immediately, `requestAskCompletion(question, context, profileId)` calls `POST /api/web/ai/ask`, and the placeholder is swapped for the answer (`handleAskInCM`/`handleAskInTinyMCE`). Empty/failed responses restore the original `/ask …` text. There is no accept/dismiss popup for `/ask` — Ctrl+Z is the reject path.
|
||||
- **TinyMCE soft-line detection**: TinyMCE runs with `newline_behavior:'linebreak'`, so pressing Enter inside a paragraph inserts `<br>` rather than starting a new `<p>`. The `/ask` handler therefore looks for the "soft line" between the last `<br>` (or block start) and the caret — not the whole block's text. Firefox `Range.toString()` does NOT emit `\n` for `<br>` (Chromium/WebKit do), so the handler walks the DOM manually collecting text and resetting on each `<br>`. Answer replacement is limited to that soft-line range (`isSoftLine` branch in `handleAskInTinyMCE`) so earlier soft-lines in the same `<p>` are preserved.
|
||||
- Identity guard: both handlers capture `activeEditorForm()` + `_formNoteId(form)` before firing the request and re-check them (plus placeholder presence) before writing the answer, so a note switch during the request drops the result instead of writing into the wrong note.
|
||||
- Disabled entirely inside vault/encrypted notes via `askDisabledForActiveNote()` (checks `form.dataset.encrypted` / `form.dataset.vaultId`) — plaintext context must never be sent to a third-party AI provider from an encrypted note. When the user types `/ask …` and hits Enter in a vault note, `_notifyAskDisabledInVault()` shows a one-shot per-note alert explaining the reason (previously Enter silently fell through to a newline, which looked like a bug).
|
||||
- Server endpoint `POST /api/web/ai/ask` (`app/routes/api.js`, next to `/api/web/ai/prose-complete`) uses a direct-answer system prompt (not the continuation/style-inference prompt used by prose autocomplete), caps `context` at 4000 chars, and uses `max_tokens: 512`. Shares `getActiveProfileFromSettings`/profile resolution with prose autocomplete; no new settings.
|
||||
- Coverage: `tests/askCommand.test.js` (client detection + CM6 handler flow), `tests/createServer.test.js` (endpoint request/response, profile selection, error forwarding).
|
||||
|
||||
|
||||
|
||||
- Expander runtime (`_feedRingBuffer`, `consumePendingTextExpansion`, `runTextExpanderAction`) is in `public/app.js`. Its wiring points are `beforeinput`/`input` listeners inside `initCM()` (source `'cm'`), the old contenteditable-preview activation code (source `'preview'`, now dead), and TinyMCE's `editor.on('keyup')` (rendered mode).
|
||||
- **Markdown mode (CM6) expander is live** (`initCM()` is invoked for real). **Rendered mode (TinyMCE) expander is live for BOTH text and AI triggers**: text triggers via `maybeExpandTextFromTinyMCE()`/`replaceTinyMCETextExpansion()`; AI triggers via `removeTinyMCETriggerForAction()` + `requestTinyMCEProseCompletion()` (prompt from `getTextBeforeCaretTinyMCE()`, insertion via `insertProseCompletionTinyMCE()`). `Ctrl/Cmd-Space` manual AI completion is wired on `editor.on('keydown')` inside TinyMCE too. The `'preview'` contenteditable path is dead (TinyMCE replaced that host). Coverage: `tests/expanderRuntime.test.js` (runtime) + `playwright-tests/ai-rendered.spec.js` (live provider E2E, skips without creds/AI profile).
|
||||
- CodeMirror-mode expansion should inspect the current document suffix rather than relying only on raw DOM input events.
|
||||
- (Historical, if re-wired to TinyMCE's iframe body) triggers can split typed text across text nodes, especially on iOS Safari. Prefer robust text-position/range logic.
|
||||
- The input ring buffer is only for detecting Expander suffix triggers; keep per-keystroke logging minimal.
|
||||
- Client diagnostic logging goes through `POST /api/web/client-log`; it redacts sensitive fields matching text/body/content/password/key/secret/token.
|
||||
- If iPhone behavior differs from desktop, use Docker-visible client logs and remember stale service worker/cache can hide client JS changes.
|
||||
194
docs/agents/architecture.md
Normal file
194
docs/agents/architecture.md
Normal file
|
|
@ -0,0 +1,194 @@
|
|||
# Architecture and conventions
|
||||
|
||||
> Read this before: changing how the server, htmx fragments, routes or the data model fit together, or when you are unsure where something belongs. Also holds the small cross-cutting contracts that have each broken before.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Product Direction
|
||||
|
||||
- Joplin Server stays unmodified
|
||||
- Joplock stays separate project and separate repo
|
||||
- Reuses existing Joplin Server auth/session/user model through sidecar logic
|
||||
- Keeps compatibility with desktop/mobile/CLI clients on same server and same DB
|
||||
- Browser stays thin and untrusted
|
||||
- Shared-browser safety matters: logout should clear client-visible state/cache as much as platform allows
|
||||
- Installable PWA shell, no offline notes/editing
|
||||
- Uses same Postgres database as Joplin Server, no separate app DB
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Stack
|
||||
|
||||
- **Server**: Node.js HTTP server, no framework
|
||||
- **Client**: SSR HTML + htmx fragment swaps + shared browser logic in `public/app.js`
|
||||
- **Editor**: Dual-mode. Markdown mode = CodeMirror 6 (mounted into `#cm-host`); rendered mode = TinyMCE 8. The `#note-body` textarea is the hidden form/sync target for both.
|
||||
- **Code blocks**: Full-screen code modal with a CM6 editor and language picker. Highlighting differs by mode: **markdown mode (CM6)** uses CM6's own `syntaxHighlighting(joplockHighlight)` with the language parsers bundled from `cm-build/`; **rendered mode (TinyMCE)** uses the native `codesample` plugin (PrismJS `.token` spans). highlight.js (`public/hljs.min.js`) is loaded on the page but is only reachable from the dead `#note-preview` path — it highlights nothing live (see the dead-code note below). Rendered mode points TinyMCE's codesample plugin at full `window.Prism` bundle (`public/prism.min.js`) so every language offered by modal has grammar support. Prism token colors are injected into the TinyMCE iframe via `content_style` in `_tinyMCEContentFontStyle()` (the oxide dark content skin ships no `.token` CSS). `codemirror.min.js` and `prism.min.js` are loaded on page before TinyMCE/app.js.
|
||||
- **Autosave**: htmx delayed PUT after typing pause (deferred while modals are open)
|
||||
- **Markdown**: server-side `renderMarkdown()`, client-side Turndown `htmlToMarkdown()`
|
||||
- **Auth**: reuses Joplin Server `sessionId` cookie
|
||||
- **DB access**: reads direct from shared Postgres; writes go through stock Joplin Server API
|
||||
- **Exports**: `POST /api/export/{docx,pdf,html}` in `app/routes/api.js` — DOCX and PDF are rendered server-side with `pandoc` (PDF via `weasyprint` plus pandoc's print CSS), HTML is a single self-contained file with inlined theme CSS, base64 images and base64 attachment links. `public/html-docx.js` is the legacy client-side DOCX path.
|
||||
- Exports work in BOTH editor modes. Rendered mode posts TinyMCE HTML (`format:'html'`); markdown mode posts the markdown (`format:'markdown'`, via `_exportSource()`) and all three endpoints render it with `renderMarkdown()` before the shared strip-links / inline-images pipeline, so images and attachments come out the same in either mode. DOCX with no `format` keeps the legacy pandoc-native markdown input. Pinned by `tests/exportMarkdownMode.test.js` (pandoc stubbed) and the real-container download test in `playwright-tests/md-extras.spec.js`.
|
||||
- **Ops surface**: admin user management (`/admin/users`), `pg_dump` backup/restore (`/admin/backups`, `/admin/restore`), break-glass recovery mode (`/recovery`), and login rate limiting — all served by the same sidecar process.
|
||||
- **Script load order** (`app/templates/pages.js`): `htmx` → `turndown` → `codemirror` → `cm-extras` → `prism` → `tinymce` → `hljs` → `html-docx` → inline config → `app.js` (`defer`). `viewport.js` sits between the inline config and `app.js` (it reads `_joplockConfig.debug`). Every local script and stylesheet carries `?v=<asset version>` (`app/assetVersion.js`, computed from the shipped files; `tests/pwaHead.test.js` enforces it), which is what lets the server serve them immutable.
|
||||
|
||||
### Runtime Shape
|
||||
|
||||
- Initial page load is full SSR HTML from `layoutPage()` in `app/templates/pages.js`
|
||||
- After load, most interactions are fragment-driven via htmx
|
||||
- The browser is intentionally thin: most state is DOM state, form state, or small client-only UI state in `public/app.js`
|
||||
- There is no frontend router and no SPA store
|
||||
- Desktop and mobile share the same server routes and most of the same editor code; mobile is a different screen shell around the same editor fragment
|
||||
|
||||
### Request Flow
|
||||
|
||||
1. Browser hits Joplock
|
||||
2. Joplock validates `sessionId` against Joplin session/user tables
|
||||
3. Fragment endpoints return HTML chunks; htmx swaps DOM
|
||||
4. Writes serialize note/folder/resource and send upstream to stock Joplin Server API
|
||||
|
||||
### Main UI Flow
|
||||
|
||||
1. `GET /` renders the full shell
|
||||
2. Navigation / notes / editor content is loaded from fragment endpoints
|
||||
3. Selecting a folder swaps the notes list or nav tree fragment
|
||||
4. Selecting a note swaps in `editorFragment()`
|
||||
5. Autosave sends `PUT /fragments/editor/:id` with the current form state
|
||||
6. Preview rendering uses `POST /fragments/preview`
|
||||
|
||||
### Fragment Conventions
|
||||
|
||||
- `app/templates/*.js` returns raw HTML strings, not JSX/templates/components
|
||||
- htmx targets are mostly `#nav-panel`, `#notelist-panel`, `#editor-panel`, and mobile-specific targets like `#mobile-editor-body`
|
||||
- Out-of-band swaps are used sparingly; note metadata is one example
|
||||
- Client logic often relies on stable IDs, so be careful renaming DOM IDs used by inline JS
|
||||
|
||||
## Service Responsibilities
|
||||
|
||||
### Stock Joplin Server
|
||||
|
||||
Owns:
|
||||
- login/session/auth source of truth
|
||||
- sync endpoints
|
||||
- canonical storage rules
|
||||
- existing user/session tables
|
||||
|
||||
### Joplock
|
||||
|
||||
Owns:
|
||||
- thin-client UI
|
||||
- sidecar API endpoints
|
||||
- session validation against shared DB
|
||||
- markdown rendering and editor behavior
|
||||
- resource upload/serving
|
||||
- note history snapshots (`joplock_history`)
|
||||
- per-user + admin settings and TOTP seeds (`joplock_settings`)
|
||||
- vault metadata (`joplock_vaults`) and its own session mirror (`joplock_sessions`)
|
||||
- admin user management, DB backup/restore, break-glass recovery
|
||||
- note export (DOCX/PDF/HTML) and the AI provider proxy
|
||||
- PWA shell/assets
|
||||
|
||||
Does not own:
|
||||
- canonical note/folder/resource persistence rules
|
||||
- sync protocol semantics
|
||||
- auth/session source of truth — Joplin's `sessions` table is authoritative; `joplock_sessions` only mirrors `last_seen` for logout/heartbeat
|
||||
- offline-first storage
|
||||
|
||||
### Sidecar Tables (four, no Joplin FKs)
|
||||
|
||||
Joplock owns exactly four tables in the shared database; every cross-reference is
|
||||
an application-level pointer, never an SQL foreign key:
|
||||
|
||||
| Table | Purpose |
|
||||
|---|---|
|
||||
| `joplock_sessions` | `session_id` → `last_seen` mirror of Joplin sessions (logout/heartbeat only) |
|
||||
| `joplock_settings` | per-user settings JSONB, `updated_time`, `totp_seed`; `user_id = '__app__'` is the admin row |
|
||||
| `joplock_vaults` | `user_id` + `folder_id` (unique) with `salt` and `verify` |
|
||||
| `joplock_history` | note snapshots: `note_id`, `user_id`, `title`, `body`, `body_hash`, `saved_time`, indexed by `joplock_history_note_time` |
|
||||
|
||||
Physical schema reference: `docs/joplock-db-schema.html`. Keep it in sync when a
|
||||
column changes — tables are created lazily with `CREATE TABLE IF NOT EXISTS` in
|
||||
the service that owns them (e.g. `app/settingsService.js` also carries the
|
||||
`ALTER TABLE ... ADD COLUMN IF NOT EXISTS totp_seed` migration).
|
||||
|
||||
### How Reads vs Writes Work
|
||||
|
||||
- Reads come from the shared Postgres DB for speed and to match the current server state
|
||||
- Writes do not write directly to Joplin tables; they go through stock Joplin Server APIs
|
||||
- That split is intentional: Joplock can stay lightweight while preserving compatibility with normal Joplin clients
|
||||
- If behavior looks inconsistent after a write, inspect both the sidecar request path and the upstream Joplin API call path
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### Separate repo
|
||||
Joplock lives outside Joplin monorepo. Keep standalone build, test, docs, Docker flow working without Joplin source tree.
|
||||
|
||||
### Shared Postgres database
|
||||
Joplock reads same Postgres database as Joplin Server. No data duplication. Writes still go through Joplin Server API for compatibility and validation.
|
||||
|
||||
### Configurable open mode
|
||||
Notes can open in rendered mode or markdown mode based on the per-user `noteOpenMode` setting (default **markdown**). The stored value may be `preview` (the pre-TinyMCE name, still accepted and preserved by `normalizeSettings`), but note the vocabulary mismatch: `preferredEditorMode()` returns only `'markdown'` or `'rich'`, and `_editorMode` is never `'preview'`. So `noteOpenMode: 'preview'` means "open rendered", and any `_editorMode === 'preview'` branch is dead. Desktop and mobile both respect the same setting. `initEditorPanel` / `_completeUnlock` must call `preferredEditorMode()` (live `_joplockConfig.noteOpenMode`). Do not let `_tinymceReadonlyDefault()` / mobile-shell read-only force rendered mode — that override made Settings → "Open notes in: Markdown" a no-op on tablet/narrow windows. Mobile read-only still applies, but only when the note actually opens in rendered mode. `_joplockConfig` is inlined in `<head>` *before* `app.js`.
|
||||
|
||||
### Shared editor fragment
|
||||
Desktop and mobile do not have separate editor implementations. Both use the same `editorFragment()` and client editor logic; mobile wraps it in a mobile-specific shell and screen navigation layer.
|
||||
|
||||
### Mobile-first navigation without SPA rewrite
|
||||
Mobile uses a folders screen, notes screen, and editor screen implemented in SSR + htmx + inline JS. Do not introduce a client router or framework state layer to solve mobile flow problems.
|
||||
|
||||
## Routes and response contracts
|
||||
|
||||
`app/createServer.js` assembles the server and delegates to the handlers in
|
||||
`app/routes/*.js`, each of which exports a `handle(url, request, response, ctx)`
|
||||
returning `true` when it consumed the request (`app/routes/api.js` additionally
|
||||
exports the export handlers and CSS/URL helpers for tests).
|
||||
|
||||
Routes are not listed here (grep `handle(` in `app/routes/`, or ask the code index); the contracts below are what is not obvious from them.
|
||||
|
||||
A few contracts worth remembering:
|
||||
|
||||
- The editor autosave `PUT /fragments/editor/:id` also sets the
|
||||
`X-Note-Updated-Time` / `X-Note-Conflict` response headers the client relies on
|
||||
(see "Plaintext save identity guard" (`docs/agents/vaults-and-save-safety.md`)).
|
||||
- Mobile note creation is driven by response headers such as `X-Mobile-Note-Id`,
|
||||
consumed in `htmx:afterRequest`.
|
||||
- History restore returns `editorFragment` **inline** (target = the editor
|
||||
container) with OOB swaps, not just the status bar.
|
||||
|
||||
If a UI action appears broken, check:
|
||||
1. Which endpoint it hits
|
||||
2. Which htmx target it swaps
|
||||
3. Which client event handler expects to run after swap/request
|
||||
4. Whether the response includes headers or OOB fragments the client depends on
|
||||
|
||||
## Coding Guidance
|
||||
|
||||
- Keep changes minimal
|
||||
- Preserve sidecar/frontend boundary
|
||||
- `public/app.js` is DOM-contract fragile; validate escaping-heavy changes and stable IDs carefully
|
||||
- **Floating menus must be opaque and stay on screen.** `--bg-elevated` is a ~2% tint in most themes, so it is a *panel lift*, never a popup background: put popups on a solid colour (`--bg-side` / `--bg-editor`; the notebook context menu layers the tint over `--bg-side`). A `position:fixed` popup also cannot be scrolled to, so place it with `positionFolderContextMenu()`-style clamping instead of raw pointer coordinates, or items near the window edge become unreachable. Guarded by `tests/popupBackgrounds.test.js` and `playwright-tests/folder-context-menu.spec.js`.
|
||||
- **Inline handlers only see what `public/app.js` assigns to `window`.** A template `onclick="foo()"` needs `window.foo=foo;`; a function that works in a unit test can still be "not defined" in the browser. `tests/inlineHandlersExposed.test.js` scans every inline handler the templates emit and fails if a called function is not exposed
|
||||
- The code modal lives in `loggedInLayout`, not inside `navigationFragment` or `editorFragment`, so it survives htmx OOB swaps
|
||||
- Be careful with checkbox text handling, `\n`, regex escaping, and DOM-to-markdown round trips
|
||||
- Keep standalone repo paths/docs/scripts correct; avoid reintroducing monorepo assumptions
|
||||
- Prefer changing existing inline helpers over introducing a new abstraction unless there is clear reuse
|
||||
- When fixing mobile behavior, verify desktop is unchanged
|
||||
- When fixing desktop editor behavior, verify mobile still works because both use the same editor fragment
|
||||
- Be cautious with `htmx:afterRequest` assumptions; in htmx 2.x, response headers are often more reliable than old event-property assumptions
|
||||
- If changing vault behavior, verify desktop + mobile, locked + unlocked, existing note + newly-created note, and refresh/restart behavior
|
||||
|
||||
## Debugging: If startup/resume behavior is wrong
|
||||
|
||||
- Check the `/` render path in `app/createServer.js`
|
||||
- Check `resumeLastNote`, `lastNoteId`, and `lastNoteFolderId`
|
||||
- Refresh/restart must not reopen encrypted notes or notes inside vault notebooks
|
||||
|
||||
## Small Contracts That Are Easy To Regress
|
||||
|
||||
Behaviours that are documented nowhere else and have each broken before (verified against the code; the larger contracts have their own sections above).
|
||||
|
||||
- **Vault chrome follows the folder.** `_syncEditorVaultChrome(noteId, inVault, unlocked)` (`public/app.js`) injects/removes the lock-toggle button in `.editor-titlebar` when a note is moved into a vault (called from `_doEncryptNoteInVault`). Vault notes cannot be moved out (see Vault / Encryption Model in `docs/agents/vaults-and-save-safety.md`).
|
||||
- **Live search keeps what you typed.** The sidebar search value is saved around the htmx swap (`_navSearchSavedValue`, captured at `htmx:beforeSwap`) so characters typed while a request is in flight are not lost.
|
||||
- **Search and note lists are paginated.** Desktop and mobile lists end in a "Load more" button (`.notelist-load-more`), not infinite scroll; search is backed by the `pg_trgm` index `ensureIndexes()` creates.
|
||||
- **TinyMCE autosave sync has a fast path and a debug path.** By default `onEdit` only calls `markEdited()` + `scheduleSave()`, and `_lazyTinyMCESyncBeforeSave()` runs once before `formHash()` in `scheduleSave` / `scheduleSaveTitle` / `_activeEditorIsDirty`; with admin debug logging on, every event syncs and logs. `ExecCommand` + `SetContent` are wired so the blocks dropdown and `insertContent` mutations still sync. `flushSave`'s hash-unchanged branch sets `Saved` (without that a phantom `Edited` blocked the sidebar's notebook "+" button).
|
||||
- **Read-only is one function.** `_applyFormReadonly()` locks the title contenteditable, folder select, toolbar buttons/inputs and the CM6 `contentDOM`; `.editor-readonly` in `styles.css` is the visual. On the mobile shell a note that opens in rendered mode is read-only until the pencil (`jop_edit`) is tapped; `_tinymceReadonlyDefault()` and `_tinyMCEToolbarSpec()` read the cached `isMobileShellMode()`.
|
||||
- **The mobile editor body shows an inline spinner** (`.mobile-loading-note`) while a note loads. A `position: fixed` overlay does not work there (it ends up under the shell's fixed layers).
|
||||
- **`renderMarkdown()` strips `hx-*` attributes** from user HTML so a note cannot inject htmx behaviour.
|
||||
31
docs/agents/editor-markdown-extras.md
Normal file
31
docs/agents/editor-markdown-extras.md
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
# Markdown-mode extras (`public/cm-extras.js`)
|
||||
|
||||
> Read this before: changing inline image previews, chips, task checkboxes, folding, tables, paste-as-markdown or the CodeMirror bundle.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Markdown-mode extras (`public/cm-extras.js`)
|
||||
|
||||
`initCM()` spreads `_cmExtraExtensions(C)` (try/catch-guarded: a failure must never stop the editor mounting) into the CM6 extension list. The extras need extra symbols from the bundle (`Decoration`, `WidgetType`, `StateField`, `ViewPlugin`, `showPanel`, `hoverTooltip`, fold APIs, `syntaxTree`, `Prec`, ...) — they are exported from `cm-build/index.js`; **if you add an extra that needs another CM symbol, export it there and rebuild with `npm run build:cm`**.
|
||||
|
||||
User settings (Settings → Markdown editor; `app/settingsService.js`, whitelisted in `app/routes/api.js`, exposed through `_joplockConfig` in `pages.js`, read in `app.js` as `_mdInlineWidgets` / `_mdLivePreview` / `_mdStatusBar` / `_mdFolding`):
|
||||
|
||||
- `mdInlineWidgets` (default ON): image previews, attachment/note-link chips, task checkboxes, code-block background + Copy button.
|
||||
- `mdLivePreview` (default OFF): hide `#`, `**`, `*`, `~~`, backticks and `[](url)` marks except on the heading line / inside the construct the caret touches. Only when the editor has focus.
|
||||
- `mdStatusBar` (default ON, **desktop shell only**: never created in the mobile shell): bottom `showPanel` with word count, reading time, Ln/Col (or selection word count) and an Outline popup.
|
||||
- `mdFolding` (default ON): fold gutter (headings, lists, fenced code) — the theme hides `.cm-gutters` entirely when OFF.
|
||||
- Always on: paste-as-markdown, URL-over-selection → link, table helper, hover info on `:/id` links, and the file-drop claim below.
|
||||
|
||||
How it fits together (details that bit us — keep them):
|
||||
|
||||
- One `StateField` (`modelField`) holds `scanDoc(doc)` output (images, links, tasks, headings, fences; nothing inside fenced code is ever scanned) and provides the **block** image widgets (block decorations cannot come from a `ViewPlugin`). A `ViewPlugin` builds the inline decorations (chips, checkboxes, code lines, live-preview hides) for `view.visibleRanges` only. Docs over ~2 MB skip scanning.
|
||||
- Image rows are block widgets *under* the `` / `<img src=":/id">` line; the source line stays visible and editable. Previews load through `hooks.fetchResourceBlob` into a refcounted blob-URL cache because `/resources/:id` is `Cache-Control: no-store`. Click selects the source reference (Delete removes the image); double-click opens the lightbox.
|
||||
- **Resize writes `<img src=":/id" alt=".." width="N" />`** — byte-identical to the Turndown `joplinImg` rule and understood by `renderMarkdown()`, so rendered mode honours the width and round-trips it. Double-clicking the handle resets to plain ``. Widget positions go stale as text above changes, so `locateImage()` re-finds the line from `view.posAtDOM()` before every edit.
|
||||
- Resize/checkbox edits are ignored when the editor is read-only (`canEdit()` checks `contenteditable="false"`, which is what `_applyFormReadonly` sets).
|
||||
- **CM6 does not repaint its cursor/selection layers when a block widget changes height** (image finishing its load). `relayout()` re-asserts the selection (`addToHistory:false`) after each load; `requestMeasure()` alone leaves the caret painted at a stale y.
|
||||
- **CodeMirror's built-in drop handler reads dropped text-like files (.txt, .svg, .json, ...) and inserts their contents.** The app's own listener uploads them, so the extras claim file drops with a `domEventHandlers({drop})` returning `true` (handlers from extensions run before the built-in). Without it a dropped text file is both pasted AND uploaded.
|
||||
- Paste: the app's capture-phase `paste` listener (`initCM`) uploads image/file clipboard items; the extras' `paste` handler converts rich HTML via `hooks.htmlToMarkdown` = `_pasteHtmlToMarkdown()` (the shared Turndown instance) only when the HTML has real structure (`shouldConvertHtml`), never inside fenced code, never for VS Code clipboards, never on Ctrl/Cmd+Shift+V. (`lang-markdown` also wraps URL-over-selection natively; the extras' version just guarantees it and escapes brackets.)
|
||||
- Download: image previews get a top-right ⬇ button and attachment chips an inline ⬇ (shown once the target resolves to a resource). Both call `hooks.downloadResource` = `downloadResource()`, so behaviour matches rendered mode (action sheet with View/Save on desktop/PWA, direct download on mobile web). The buttons stop propagation so they neither open the lightbox nor select the image source.
|
||||
- Chips replace `[text](:/id)`; the kind (note vs attachment) is resolved lazily via `_resolveInternalLink()` (note headers cache, then `HEAD /resources/:id`). Notes open through the note-list row (`_openNoteById()`), attachments through `_openResourceLightbox()`. Rendered mode still treats every `:/id` link as an attachment — note-link navigation exists only in markdown mode.
|
||||
- Table helper (`Prec.high` keymap): Tab/Shift-Tab align the whole pipe table and move between cells (Tab at the last cell appends a row). Tables inside fences are ignored. **Do not bind Enter**: people type pipe tables row by row, and an "Enter adds a formatted row" handler corrupts what they type next (it broke `table-regression.spec.js`).
|
||||
- No regex lookbehind anywhere in `cm-extras.js` (a parse-time SyntaxError on iOS < 16.4 would take the whole file down) — pinned by a test.
|
||||
- Coverage: `tests/cmExtras.test.js` (pure helpers + real CM6 in jsdom), `playwright-tests/md-extras.spec.js` (real browser: preview/resize/round-trip, chips + hover, checkboxes, tables, paste, status bar/outline/folding, Copy button, live preview).
|
||||
29
docs/agents/editor-uploads.md
Normal file
29
docs/agents/editor-uploads.md
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
# Uploads, paste and drop
|
||||
|
||||
> Read this before: changing note/image/attachment upload, clipboard paste, drag and drop, or the upload size limit.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Upload behavior
|
||||
|
||||
- The upload modal (`openUploadModal()` → `uploadModalFiles()` → `insertUploadedFiles()`) is the primary picker/drag-drop path; it uploads to `/fragments/upload` and inserts into the live TinyMCE document when in rich mode (or the CM6/textarea target when not). On success (all files upload, no errors) the modal auto-dismisses; if any file errors it stays open showing per-file errors.
|
||||
- Drag-and-drop directly onto TinyMCE works via `_uploadFileToTinyMCE()` (inserts `<img data-resource-id>` / `<a data-resource-id>` into the live editor). Markdown-mode drops route through `_uploadFileToCM()` (inserts `` at the CM cursor).
|
||||
- **The upload-modal/picker insert must go through the live editor in rich mode** (it used to fall through to the hidden textarea, which lost the insert). `_captureUploadInsertTarget()` now returns `{ mode: 'tinymce', rng }` when rich mode is active, the TinyMCE host is visible, and the editor is not read-only; `_insertUploadedMarkdown()` then routes to `_insertResourceIntoTinyMCEFromMarkdown(markdown, rng)`, which re-inserts a saved caret range, builds the same `<img>`/`<a>` HTML as the drop path, and inserts it through `_tinyMCEBlockAttachmentHtml()` (so blank-line padding is identical). Read-only editors (mobile rendered mode) still fall through to the textarea/CM branches.
|
||||
- **Programmatic rich-mode inserts must call `_endTinyMCEPostLoadWindow()` first** (`_uploadFileToTinyMCE`, `insertUploadedFiles`, `_insertResourceIntoTinyMCEFromMarkdown`). An insert is a real edit, not load echo: it clears `_tinymcePostLoad`/`_tinymcePostLoadUntil` and sets `_tinymceUserTypedSinceLoad`. Without this, `onEdit` classifies the insert's own events as round-trip echo, the post-load reconcile re-baselines `_savedHash` to the inserted content, and the debounced save then sees "hash unchanged" — the upload never reaches the server.
|
||||
- **Dropped/pasted attachments (image AND document) are padded with a blank line before and after** so a single attachment stays easy to delete even when several are stacked. This padding is *only* about spacing around the inserted resource — it does not change how surrounding typed text is handled.
|
||||
- Markdown mode (CM6): `_uploadFileToCM()` inserts `<pad>` + ref + `\n\n`, where `<pad>` is `''` at the very start of the doc, `\n` if the char before the cursor is already a newline, else `\n\n`. Both images and documents get this. (Plain source blank line; no `md-blank-line` marker needed because the user edits raw text here.)
|
||||
- Rendered mode (TinyMCE): `_tinyMCEBlockAttachmentHtml(editor,inner)` wraps the image/link in its own `<p>` and adds a `<p class="md-blank-line"><br></p>` (the renderer's canonical deletable blank line — see `injectBlankLineBlocks`) before and after, for both images and documents. **Smart**: it skips the leading and/or trailing marker when the caret block is already empty or already adjacent to an existing blank-line paragraph (via `editor.selection.getNode()`), falling back to adding both when the selection API is unavailable (unit tests).
|
||||
- Round-trip safety (rendered mode): `md-blank-line` paragraphs are pre-normalized in `tinymceToMarkdown()` and matched by the `blankLine`/`emptyP` Turndown rules → `\x00BL\x00` sentinel → `\n\n\n` (one extra newline = one blank line that re-renders as an `md-blank-line` `<p>`). Do NOT switch these separators to bare `<div><br></div>` or plain empty `<p></p>` — those get merged/dropped around block-level images and swallow the spacing after a few round-trips.
|
||||
- `_buildMarkdownInsert()` (used by the upload-modal picker's textarea/CM targets) is unchanged — it still adds a single `\n` on each side as needed. It is deliberately NOT part of the blank-line padding change.
|
||||
- Coverage: `tests/cm6MarkdownMode.test.js` (CM padding for image + document; TinyMCE `_tinyMCEBlockAttachmentHtml` markers + smart skip), `tests/previewRoundTrip.test.js` (blank line between stacked image/image, image/doc, doc/image survives render⇄markdown), `tests/appRuntime.test.js` (mode-switch round-trip, no mangling).
|
||||
- Markdown-mode clipboard paste: `initCM()` registers a CAPTURE-phase `paste` listener on `_cmView.dom` that uploads image/file clipboard items via `_uploadFileToCM()` (`_clipboardFilesToUpload()` decides: non-image files always, images unless the clipboard is a text+HTML rich copy such as a spreadsheet). It must be capture-phase on the editor root: CM6's own paste handler only reads `text/plain` and, for an image-only clipboard, would *replace the selection with an empty string*.
|
||||
- Clipboard paste: images are uploaded by TinyMCE's built-in pipeline (`paste_data_images:true` + `automatic_uploads:true` + `images_upload_handler`); non-image clipboard files are handled by an explicit `editor.on('paste', ...)` handler that routes through `_uploadFileToTinyMCE()`.
|
||||
- The Image/Media dialogs' Browse button is wired via `file_picker_callback` to `/fragments/upload`, returning a `/resources/<id>` URL.
|
||||
- All upload paths produce `src="/resources/<id>"` / `href="/resources/<id>"`, which `tinymceToMarkdown()` (the `joplinImg`/`joplinLink` Turndown rules) converts to Joplin `` / `[](:/id)` on save. `data-resource-id` is added by drop/paste/upload-modal paths but is not required for the round-trip (matching is by `src`/`href`).
|
||||
- `uploadFiles()`/`handleFilePicker()` (the older `#file-upload` input path) still exist and batch multi-file selections; image-only uploads must not promote the image filename into the note title.
|
||||
|
||||
## Upload size limit
|
||||
|
||||
- `appSettings.maxUploadMb` (admin setting, default 200, clamped 1–2000) caps upload size. Joplin Server's formidable `maxFileSize` is a hard 200MB ceiling — exceeding it produced opaque 500s after buffering; the limit prevents that.
|
||||
- Server: `app/routes/resources.js` `resolveMaxUploadBytes()` does a fast 413 via `Content-Length` pre-check (before buffering) plus a post-parse guard.
|
||||
- Client: `_maxUploadBytes()` reads `_joplockConfig.maxUploadMb`; `_fileTooLarge()` guards every upload entry point (modal, drop, paste, picker) with a friendly message instead of a failed request.
|
||||
- Admin field: Settings → Admin → Login Security, saved via `/admin/security`.
|
||||
105
docs/agents/editor.md
Normal file
105
docs/agents/editor.md
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
# Editor model
|
||||
|
||||
> Read this before: changing the editor (CodeMirror 6 markdown mode, TinyMCE rendered mode, mode switching, autosave, markdown round-trip) or debugging a phantom "Edited" state.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Architecture (dual-mode: CM6 markdown + TinyMCE rendered)
|
||||
|
||||
The editor supports two modes, both backed by the hidden `<textarea id="note-body">` form field:
|
||||
|
||||
- **Rendered mode = TinyMCE 8.** Persistent singleton (`initPersistentTinyMCE()` in `public/app.js`) mounted on a hidden `<textarea id="tinymce-editor">` that lives outside `.app`/`#mobile-app` in `pages.js` so htmx swaps don't destroy it; a `position:fixed` `#tinymce-host` div is repositioned via `positionTinyMCEHost()` to sit over the `#tinymce-slot` placeholder in `editorFragment()`. TinyMCE content is converted back to markdown via `tinymceToMarkdown()` (Turndown) into `#note-body` on `input`/`change`.
|
||||
- **Markdown mode = CodeMirror 6.** Mounted into `#cm-host` by `mountMarkdownEditor()` → `initCM()`, seeded from `#note-body`. CM6 edits sync into `#note-body` via `cmSyncToTA()` (called from initCM's update listener). `getCM()` returns the live `EditorView`; `cmSetVal()` replaces the CM document.
|
||||
- `codemirror.min.js` is loaded on the page before `app.js` (built from `cm-build/`, `npm run build:cm`), so `window.CM` is available for both the markdown editor and the code-block modal.
|
||||
|
||||
Text-expander is now wired for BOTH modes:
|
||||
- **Markdown mode (CM6)**: `maybeExpandTextFromCM()` on `initCM()` contentDOM input listeners (text + AI triggers).
|
||||
- **Rendered mode (TinyMCE)**: `maybeExpandTextFromTinyMCE()` on `editor.on('keyup')`; inspects the caret text node suffix in the iframe and replaces the trigger via `replaceTinyMCETextExpansion()` (multi-line → `<br>`, then `tinyMCESyncToTA()`). Both `action:'text'` AND `action:'ai'` triggers now fire in rendered mode: AI triggers call `removeTinyMCETriggerForAction()` then `requestTinyMCEProseCompletion()` (builds a prompt from the iframe caret via `getTextBeforeCaretTinyMCE()`, calls `requestProseCompletion()`). The completion is offered in the SAME `note-autocomplete-popup` used by markdown mode — kind `'tinymce-prose'`, accept with Enter/Tab inserts via `insertProseCompletionTinyMCE()` (DOM text nodes, restores a caret bookmark first), Esc discards. `Ctrl/Cmd-Space` inside the iframe is wired on `editor.on('keydown')` (the global `document` keydown can't see iframe keystrokes) and also shows the popup. Popup keys are forwarded from the iframe keydown via `handleRenderPopupKey()` because iframe key events never reach the outer-document listener; popup coords come from `tinyMCECaretCoords()` (iframe caret rect offset by the iframe element rect).
|
||||
|
||||
## Dead code and legacy paths (verified still present)
|
||||
- **Dead `getPV()` / `#note-preview` contenteditable code** still exists in `public/app.js` (superseded by TinyMCE). `getPV()` returns null (the element is never rendered by any template), so every `if(pv){...}` branch in the formatting helpers (`wrapSel`, `insertPfx`, `clearFormat`, `openCodeModal`, `submitCode`, `syncPV`, `replacePVTextExpansion`, etc.) is dead and always falls through to the CM/textarea branch. Harmless. NOT removed because it is threaded through ~30 functions and ripping it out risks regressing the live CM path; do it as a dedicated, well-tested cleanup pass, not a drive-by.
|
||||
- **Image download buttons live in the TinyMCE body, not the dead PV path.** `initResourceImageDownloadButtons`/`activatePV` are legacy-only (PV never renders). The restored button is `initTinyMCEImageDownloadButtons(editor)`, called from `_setTinyMCEContent`'s post-load hook next to `initTinyMCECodeCopyButtons`. Serialization contract (same as the code-copy button): the button is `textContent=''` + `data-mce-bogus="all"` (label via CSS `::after` in `_tinyMCEContentFontStyle()`), so `getContent()` drops it — it can never reach saved markdown, HTML/DOCX/PDF exports. Only the positioning wrap `<span class="preview-img-download-wrap">` survives `getContent()`; `_stripTinymceDownloadChrome()` unwraps it in `tinyMCEContent()` and `tinymceToMarkdown()` so every serialization boundary sees the pre-injection DOM (keeps round-trip hashes stable — no phantom edits). Because the anchor can now sit inside the iframe, `_positionResourceActions` resolves anchor rects via `_anchorRectInHostDoc()` (iframe-viewport rect offset by the iframe element's rect). Coverage: `tests/tinymceImageDownload.test.js`.
|
||||
- **`highlightTinyMCECodeBlocks()` is unreferenced dead code.** It was the hljs pass over the TinyMCE body before Prism/`codesample` took over; every call site was removed but the function is still defined (`public/app.js`). It is the only hljs consumer besides the dead PV path, so removing it (with `highlightCodeBlocks()`) is a safe, self-contained slice of the PV cleanup.
|
||||
|
||||
## Two modes
|
||||
|
||||
- **Markdown mode**: CodeMirror 6 mounted in `#cm-host` is the visible editor; `#note-body` is the hidden sync target. If the CM6 bundle fails to load, `mountMarkdownEditor()` falls back to showing the raw textarea.
|
||||
- **Rendered mode**: TinyMCE (persistent instance, positioned over `#tinymce-slot`) is visible; `tinymceToMarkdown()` converts edited HTML back to markdown on `input`/`change`.
|
||||
|
||||
## Source of truth during editing
|
||||
|
||||
- The hidden textarea `#note-body` is the form field used for saves.
|
||||
- In markdown mode, CM6 changes sync into `#note-body` via `cmSyncToTA()`.
|
||||
- In rendered mode, TinyMCE's `getContent()` is converted via `tinymceToMarkdown()` into `#note-body`.
|
||||
- Switching modes: markdown→rich calls `cmSyncToTA()` then POSTs the markdown to `/fragments/preview` and loads the rendered HTML into TinyMCE; rich→markdown calls `tinyMCESyncToTA()` then mounts CM6 from the textarea. There is no client-side markdown→TinyMCE-HTML converter — that direction round-trips through the server.
|
||||
- File/image uploads should alter markdown first, then refresh rendered preview from markdown; do not treat preview-only DOM insertion as authoritative state.
|
||||
- The title is mirrored between `.editor-title`, hidden title input, and mobile title header when applicable.
|
||||
|
||||
## Save lifecycle
|
||||
|
||||
- `markEdited()` updates UI state to `Edited`
|
||||
- `scheduleSave()` triggers delayed autosave for body/form changes
|
||||
- `scheduleSaveTitle()` is a shorter timer for title changes
|
||||
- If `scheduleSave()` or `scheduleSaveTitle()` sees the same form hash as `_savedHash`, the visible save state should return to `Saved`, not remain `Edited`
|
||||
- `flushSave()` is the forced-save path used before leaving a dirty note; it must also handle vault-note encryption before navigation proceeds
|
||||
- `htmx:afterRequest` on the editor save path transitions UI state back to `Saved`
|
||||
- Offline/request failure paths set status to `Offline`
|
||||
|
||||
## Important fragility points
|
||||
|
||||
- DOM IDs and class names are part of the editor contract with inline JS (`#cm-host`, `#note-body`, `#tinymce-slot`, `#tinymce-host`, `#editor-toolbar`).
|
||||
- Rendered-mode HTML (TinyMCE content) must remain convertible back to markdown with acceptable fidelity via `tinymceToMarkdown()`/Turndown.
|
||||
- Checkbox, code block, and blank-line handling are easy to regress.
|
||||
- **Blank-line markers between blocks are `<p class="md-blank-line"><br></p>`, NOT bare `<div><br></div>`.** `injectBlankLineBlocks()` (`app/markdownRenderer.js`) emits extra blank lines as empty paragraphs because TinyMCE's schema preserves empty `<p>` natively; a bare `<div><br></div>` got normalised/merged/dropped around block-level images, which swallowed spacing between images after a few markdown⇄render round-trips. The Turndown `blankLine` rule (`public/app.js`, and the preview-path copy in `tests/previewRoundTrip.test.js`) matches `P|DIV.md-blank-line`; in `getTurndown()` both `blankLine` and `emptyP` emit the same `\x00BL\x00` sentinel so precedence is moot. **Two TinyMCE quirks made image spacing collapse anyway (both fixed):** (1) **TinyMCE strips the `<br>`** from the marker on `setContent`, leaving an empty `<p class="md-blank-line"></p>` that Turndown drops — so `tinymceToMarkdown()` pre-normalises a **blank** `md-blank-line` paragraph (empty, whitespace/` `-only, or just `<br>`) to the `❤BR❤` sentinel shape before Turndown. **This normalisation MUST stay conditional.** A marker is a real, focusable paragraph in the iframe, so clicking the gap between two blocks puts the caret inside it and typing puts the new text there; rewriting markers unconditionally (the old "empty or not" behaviour) replaced that text with the sentinel and **silently destroyed it** — type a line after a checklist, leave the note, come back, gone. When a marker holds real content, leave the paragraph verbatim: with text present neither the `blankLine` nor the `emptyP` rule matches, so it converts as an ordinary paragraph. Coverage: `tests/appRuntime.test.js` "keeps text typed INTO a blank-line marker (data-loss regression)" + "still collapses a genuinely blank blank-line marker". (2) **Sized/raw-HTML images** (Turndown emits `<img … width=… />` for resized images) are markdown-it *HTML blocks* rendered OUTSIDE any `<p>`; a loose block `<img>` next to markers gets absorbed into an adjacent paragraph by TinyMCE, so `postProcess()` wraps any line that is a lone `<img>` in its own `<p>`. Regression coverage: `tests/appRuntime.test.js` "image spacing …", "sized … survive 6 mode switches", "br-stripped … marker".
|
||||
- The code modal is outside the fragment-swapped editor so it survives swaps; it uses CM6 (`_initCodeModalCM`) which requires `window.CM` (now loaded).
|
||||
- Both markdown mode (CM6) AND rendered mode (TinyMCE) open this same custom full-screen CM6 code modal (`openCodeModal`/`submitCode`) for *editing* the code text/language, NOT TinyMCE's built-in `codesample` dialog. The toolbar uses a custom `jop_code` button; clicking an existing `<pre>` in rendered mode routes through `tinyMCEInsertCodeBlock()` → `openCodeModal()`. On submit in rendered mode, `submitCode()` (TinyMCE branch, `_codeTinyMCE`/`_codeTinyMCEBookmark`) inserts `<pre class="language-x">code</pre>` via `ed.insertContent()` — the `codesample` plugin's `SetContent` handler then highlights it with Prism. Do NOT reintroduce hljs highlighting of rendered-mode blocks; Prism owns rendered-mode coloring. `highlightTinyMCECodeBlocks()` (the old hljs pass over the TinyMCE body) has **no call sites left** — the function body is dead, not deleted, so grep hits are misleading. Do not reintroduce `ed.execCommand('mceCodeSample')` (that opens the built-in dialog).
|
||||
- On htmx editor-panel swap, `_cmView` is destroyed in `htmx:afterSwap` and re-mounted by `initEditorPanel()` (via `mountMarkdownEditor`) on `htmx:afterSettle` when the note opens in markdown mode. Keep that destroy/remount ordering intact.
|
||||
- `#tinymce-host` is `position:fixed` and repositioned via `positionTinyMCEHost()`; if it looks detached, check that function and the `#tinymce-slot` rect, not CSS alone.
|
||||
- **Turndown expels "flanking" whitespace, and it gets that wrong next to atomic children.** `flankingWhitespace()` derives an element's edge whitespace from `node.textContent`, which skips `<img>`/`<br>` (they contribute no text). For `<a><img/> Label</a>` it reports *leading* whitespace even though the whitespace is interior to the produced markdown (` Label`), so `replacementForNode()`'s `content.trim()` cannot remove it — yet it is still prepended. The space was therefore **duplicated on every round-trip and grew one character per note open**, corrupting the stored body. `tinymceToMarkdown()` hides such whitespace behind a sentinel (`_protectInlineLeadingSpace` → `_restoreProtectedSpace`) that **encodes the character code**: these runs are frequently NBSP, and restoring a generic `' '` would itself change the body and keep the note permanently dirty. Do not "simplify" that sentinel back to a plain space, and do not narrow its character class to `[ \t]` — Turndown's `edgeWhitespace` uses `\s`, which matches NBSP.
|
||||
- **Soft breaks inside a blockquote must re-apply the `>` prefix.** `> b\n> c` is ONE quoted paragraph with a soft break, so the renderer emits `<blockquote><p>b<br>c</p></blockquote>`. The `❤BR❤` sentinel is restored to `\n` *after* Turndown has prefixed the lines it emitted, so a naive global `split/join` produced `> b\nc` — the second line escaping the blockquote entirely (real content corruption plus a permanent dirty-on-open diff). The restore is line-aware and carries the leading `>` run across the break; keep it that way. Bare `> ` lines also get their insignificant trailing space stripped.
|
||||
|
||||
## Debugging
|
||||
|
||||
### If toolbar/mode behavior is inconsistent
|
||||
|
||||
- Verify whether the current editor is actually inside `#mobile-editor-body`
|
||||
- Check `syncEditorModeButtons()` and `setEditorMode()`
|
||||
- Check whether the note was initialized with the expected `noteOpenMode`
|
||||
- If switching modes marks the note `Edited`, confirm the current form hash differs from `_savedHash`; unchanged hashes should show `Saved`
|
||||
|
||||
### If title UI drifts
|
||||
|
||||
- Check `.editor-title`
|
||||
- Check hidden input `.editor-title-hidden`
|
||||
- Check `#mobile-editor-title`
|
||||
- Check `autoTitle()` and `syncTitle()`
|
||||
|
||||
### If save-state UI drifts
|
||||
|
||||
- Check `setSaveState()`
|
||||
- Check `#autosave-status`
|
||||
- Check `#mobile-editor-status`
|
||||
- Check htmx save success/failure handlers and upload progress handlers
|
||||
|
||||
### If a note shows "Edited" (or autosaves) immediately on open
|
||||
|
||||
`_savedHash` is snapshotted in `initEditorPanel()` from the **raw server body**, before CM6/TinyMCE has loaded. 820ms after `setContent`, the reconcile timer in `_setTinyMCEContent()` syncs the editor back to markdown and compares hashes. So **any** non-idempotent markdown⇄HTML round-trip looks like a user edit.
|
||||
|
||||
With `debugLogging` on, the reconcile prints exactly which side it took — this is the fastest way to triage:
|
||||
|
||||
| Log line | Meaning |
|
||||
|---|---|
|
||||
| `post-load reconcile: form clean, nothing to do` | Round-trip is idempotent. Save state is not the problem; look elsewhere. |
|
||||
| `post-load reconcile: load normalisation only, re-baselining hash` | Round-trip is lossy but nobody typed. Hash is re-baselined: no `markEdited`, no save. Expected for notes with constructs HTML cannot represent. |
|
||||
| `post-load reconcile: user typed during window, marking edited` | A real edit landed inside the 800ms quiet window and is being saved. Correct behaviour. |
|
||||
|
||||
If you see `user typed during window` when the note was untouched, the `_tinymceUserTypedSinceLoad` flag is being set spuriously — it must only be wired to `keydown`/`paste`/`cut`/`drop`. Never wire it to `input` or `SetContent`: `editor.setContent()` fires both, which reintroduces the phantom edit. Navigation-only keys (arrows, modifiers) must stay filtered out.
|
||||
|
||||
To find out *why* a note is non-idempotent, round-trip its stored body offline:
|
||||
`renderMarkdown(body)` → `tinymceToMarkdown(html)` → diff against `body`. `tests/appRuntime.test.js` has the harness (`makeTurndownCtx()` + `runWithDeps()`) for running the real `public/app.js` functions under JSDOM.
|
||||
|
||||
Known-lossy constructs that are **by design** and will always re-baseline (do not "fix" them by adding markers):
|
||||
|
||||
- an indented ` ``` ` fence loses its indent (HTML cannot carry fence indentation)
|
||||
- encrypted/vault note wrappers strip HTML comments — irrelevant in practice, since locked notes never load plaintext into TinyMCE
|
||||
|
||||
Heading-gap blank lines around ATX headings are no longer lossy: `tinymceToMarkdown(html, prevMd)` keeps Turndown's natural spacing and only falls back to the authored markdown (`prevMd`, what `#note-body` already held for the note) when the fresh conversion differs from it purely in heading-gap shape — collapsing both with `_applyHeadingSpacing()` yields the same text. Authored blank lines around headings survive the switch; untouched compact notes don't gain a phantom "Edited"/save; a real edit always wins (and one-time re-spaces a compact heading note). Do not reintroduce the unconditional `headingGapRe`/`headingLeadRe` collapse in `tinymceToMarkdown()` — it ate blank lines the user typed around headings (reported: add a blank line under a heading in markdown mode, switch to rendered and back, it was gone, and the collapsed body was saved). `_applyHeadingSpacing()` remains for the authored-gap comparator and the legacy preview path (`htmlToMarkdown`). Coverage: `tests/appRuntime.test.js` heading-gap round-trip tests + `playwright-tests/heading-gap-roundtrip.spec.js`.
|
||||
36
docs/agents/joplin-fidelity.md
Normal file
36
docs/agents/joplin-fidelity.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
# Joplin field fidelity
|
||||
|
||||
> Read this before: writing or serializing notes, notebooks or resources, or anything another Joplin client can see. A Joplock write must change only what the user edited.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Joplin Field Fidelity (do not regress)
|
||||
|
||||
Joplock shares one account with stock Joplin clients, so **a Joplock write must change only what the user edited**. Originally `serializeNote`/`serializeFolder` rebuilt every item from hard-coded values: a single body edit turned a to-do into a plain note, wiped author/source URL/location, flipped an HTML note to markdown, reset `order` and overwrote `source_application`. Measured with the real Joplin CLI synced through Joplock's proxy, then fixed.
|
||||
|
||||
### How it works
|
||||
|
||||
- **Pass-through serializer** (`app/items/joplinItem.js`): reproduces Joplin's `serialize` / `serialize_format` (canonical key order, ISO times, `\n` escaping, no empty body block). Values come from the item's **stored fields** and only the fields Joplock owns are overridden.
|
||||
- **Items carry their stored fields.** `mapNoteRow` / `mapFolderRow` attach `fields` (the parsed `items.content` JSON). Every write path passes `existing`, so `serializeNote({ fields: existing.fields, ... })` writes everything else back untouched, including fields Joplock has never heard of (`is_locked`, `extracted_resource_ids`, `user_data`, `application_data`, ...). Defaults (`order = now`, zero geo strings, `markup_language 1`, `source_application net.cozic.joplock-web`) apply **only to brand-new items**; edited items keep their original `source_application`.
|
||||
- **Joplock-owned fields:** title, body, parent, deleted time, `is_shared`/`share_id`, `updated_time` (always now, sync needs it). `user_updated_time` moves **only on a real title/body edit**; move, trash, restore and share changes preserve it (Joplin's `moveToFolder` semantics).
|
||||
- **`fields` is internal.** `publicItem()` / `publicItems()` (`routes/_helpers.js`) strip it from JSON API responses. Keep using them on any new endpoint that returns a note or notebook.
|
||||
- **The server only stores fields its own Joplin library knows** (it drops the rest on upload and when serving), so "unknown to Joplock" fields that can persist are ones a current Joplin writes. Joplin clients themselves also drop unknown fields on read.
|
||||
- **Share id lives in a column.** Joplin Server moves `share_id` out of the item JSON into `items.jop_share_id` on the first API save. Always read it via `shareIdOfRow()` (column first, JSON fallback) / `shareIdOf()` in SQL. Reading only the JSON made a shared note look unshared after its first edit, so the *second* Joplock edit wrote `share_id: ''` and silently took the note away from every recipient.
|
||||
|
||||
### Items Joplock must treat carefully
|
||||
|
||||
| Item | Behaviour |
|
||||
|---|---|
|
||||
| `markup_language = 2` (HTML note) | Shown **rendered and read-only** (`app/htmlNoteRenderer.js`, allowlist sanitizer on untrusted HTML + TinyMCE's own sanitizing). Form is `data-html-note="1"`: no markdown/rendered switch, no toolbar, title not editable, `tinyMCESyncToTA` is a no-op. The server refuses any body change (403) but still allows move/trash/restore. Never convert it to markdown. |
|
||||
| Joplin end-to-end encrypted (`jop_encryption_applied = 1`) | **Locked placeholder only.** `e2ee: true`, title/body blanked, no editor form at all. `updateNote`/`updateFolder` throw 403 (`assertNotE2ee`); a live one cannot be hard-deleted; a notebook holding any cannot be deleted (409, checked **before** any write so a delete never half-completes). |
|
||||
| `is_conflict = 1` (conflict copy) | **Hidden** from lists, counts, headers and search (`conflictFilterSql`), like Joplin's "Conflicts" notebook. Stays untouched, and travels with its notebook when that is deleted (`includeConflicts`). |
|
||||
| Attachments | `serializeResource` writes Joplin 3.x's OCR fields; `ocr_driver_id` is `1` (Joplin's default; `0` is a legacy quirk). |
|
||||
|
||||
### Real-client integration tests (opt-in)
|
||||
|
||||
`npm run setup:joplin-cli` installs the real Joplin terminal client under `~/.cache` (not in the repo). `npm run test:joplin` (dev stack up, admin env vars set) drives it headless through Joplock's `/joplin` proxy using a dedicated `compat-client@joplock.test` account (`integration-tests/joplin/harness.js`): `joplin-fidelity`, `joplin-guards`, `joplin-nested`, `joplin-sharing`. They create items in the client, sync, edit/move/trash in Joplock, sync back and compare **field by field** from the client's own SQLite. Raw uploads authenticate with `X-API-AUTH`, like a real client. Joplin Server applies share changes on a 10 s task, so the sharing tests wait ~13 s per step. One test is a tracked `todo` (below).
|
||||
|
||||
### Known issues found with the real client (not fixed)
|
||||
|
||||
- **Sync-proxy guards do not apply to stock Joplin clients.** `vaultProxyGuard` and `shareProxyGuard` identify the user with `authenticatedUser()`, which reads only Joplock's `sessionId` **cookie**. Joplin apps authenticate with an `X-API-AUTH` header and send no cookie, so both guards "stream through". Verified: a real client wrote a **plaintext note into a vault notebook and created a notebook under a vault**, and nothing stopped it. The vault/share/"no notebook under a vault" proxy protections described elsewhere in this guide therefore only hold for browser-origin traffic. Making them live needs `authenticatedUser` to accept `X-API-AUTH` **and** a fix for a second bug: when a request does authenticate, both guards call `bufferRequest()` on the same stream, so the second read never completes and the request hangs (seen as a hung cookie-authenticated PUT to `/joplin/...`). Behaviour will change for existing clients (403s on vault writes), so treat it as its own change.
|
||||
- **Collaborator content is invisible to the share owner in Joplock.** Access checks use Joplock's own `user_items` rows keyed by `jop_id`; Joplin Server keys them by `items.id`. A note a collaborator creates in a Joplin app never shows for the owner in Joplock (it does in the owner's Joplin apps). Tracked as the `todo` test in `joplin-sharing.test.js`. Fixing it means changing the access model together with Joplock's own removal paths (otherwise removed users keep access).
|
||||
- Notes created in a share after sharing do not reach a **Joplock** recipient, and a note moved out of a share stays visible to a Joplock recipient (the note write paths never grant/revoke Joplock's `jop_id` rows; only folder operations do). Stock-client recipients are handled correctly by the server.
|
||||
134
docs/agents/notebooks-and-sharing.md
Normal file
134
docs/agents/notebooks-and-sharing.md
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
# Notebooks, nesting and sharing
|
||||
|
||||
> Read this before: touching the notebook tree (nesting, drag and drop, sort, pins), notebook create/move/delete rules, or anything about sharing / `user_items`.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Sharing Ownership Model
|
||||
|
||||
- One authoritative tree owned by sharer (`owner_id` never transferred).
|
||||
- Recipients gain access via Joplin `user_items` + accepted `share_users` (auto-accept on invite).
|
||||
- **Vault notebooks cannot be shared**: `POST /api/web/shares` rejects them with 400 `Vault notebooks cannot be shared` (`app/routes/shares.js`).
|
||||
- `share_users.can_write` controls whether recipients can edit shared notes (default `1` — editable). Only the owner can move, delete, or stop-share.
|
||||
- Move into a shared notebook sets `share_id`; move out clears it and drops recipient access to that item.
|
||||
- Revoke/stop sharing removes recipient access; owner keeps folders/notes in place.
|
||||
- Recipients can leave a shared notebook via the share dialog ("Leave notebook" button) → removes their `share_users` row + `user_items` entries.
|
||||
- Only the share owner sees invite/remove/stop controls in the share dialog. Recipients see only their own access status and the Leave button.
|
||||
|
||||
### Shared-note guard layers
|
||||
|
||||
- **Route layer** (`app/routes/fragments.js`, `app/routes/api.js`): `resolveItemShareAccess` → `assertCanWrite`/`assertOwnerForDestructive` on create/update/delete/restore/move. `canWrite` respects `share_users.can_write` for recipients. Owner-only for move/delete regardless of `can_write`. Recipient creates in shared folder blocked with 403.
|
||||
- **Proxy layer** (`app/proxy/shareProxyGuard.js`): inspects PUT and DELETE sync-proxy requests. PUT checks `resolveItemShareAccess` → `canWrite`. DELETE is owner-only. Inherits `noteIdFromItemPath`/`bufferRequest` from vault proxy guard.
|
||||
- **Editor UI** (`app/templates/fragments.js`, `public/app.js`): `editorFragment` accepts `canWrite` param from route handler (queries `share_users.can_write`). When `canWrite=false` renders read-only banner, disables folder select, hides delete button, sets `contenteditable="false"` on title. When `canWrite=true` the editor is fully interactive.
|
||||
- **Share-id propagation** (`app/items/shareAccess.js` for the field derivation, `app/routes/shares.js` for the write): `deriveShareFieldsForMove(targetFolder)` returns `{ shareId, isShared }` from the target folder, so a move sets/clears `share_id`. `ensureShareIdsOnNotebook()` writes `share_id` directly to the items DB content JSON. `createNote`/`updateNote` serialize these fields into Joplin note metadata.
|
||||
- **Share dialog API** (`app/routes/shares.js`): uses Joplin Server's `/api/shares/:id/users` endpoints (not deprecated `/api/share_users`). PATCH/ACCEPT/REJECT/DELETE operations use DB-only writes since Joplin Server's newer API doesn't support mutations on individual share_users. `can_write` column managed via direct `share_users` table updates. The module also exports helpers reused elsewhere: `autoAcceptShareUser`, `populateUserItems`, `ensureShareIdsOnNotebook`, `STATUS_ACCEPTED`/`STATUS_WAITING`.
|
||||
- **Never decode `items.content` for every item in share SQL** (`app/routes/shares.js`): `items` also holds binary resource blobs (`jop_type 0`, every uploaded image/PDF). `convert_from(content,'UTF8')` raises on them and aborts the whole statement — and the routes swallow DB errors, so invites silently stopped fanning out `user_items` (shared notebook never appeared) and revoke/stop-sharing silently left access behind as soon as the owner had one attachment. Read `share_id` only through `shareIdOf()` (a `CASE WHEN jop_type IN (1, 2)` guard; SQL gives no AND/OR evaluation-order guarantee). Pinned by `tests/sharesBinaryItems.test.js`.
|
||||
|
||||
### File map
|
||||
|
||||
| Layer | File |
|
||||
|-------|------|
|
||||
| Access helpers | `app/items/shareAccess.js` |
|
||||
| Share dialog API routes | `app/routes/shares.js` |
|
||||
| Proxy write guard | `app/proxy/shareProxyGuard.js` |
|
||||
| Fragment write gates | `app/routes/fragments.js` |
|
||||
| API write gates | `app/routes/api.js` |
|
||||
| Share dialog template | `app/templates/shares.js` |
|
||||
| Share dialog client | `public/app.js` (openShareDialog, inviteToShare, toggleShareWrite, leaveShareNotebook, etc.) |
|
||||
| Read-only editor | `app/templates/fragments.js` (editorFragment) |
|
||||
| Unit tests | `tests/shareAccess.test.js`, `tests/shareProxyGuard.test.js`, `tests/shareWriteGuards.test.js` |
|
||||
| Playwright tests | `playwright-tests/share-modal.spec.js`, `playwright-tests/share-access.spec.js`, `playwright-tests/share-revoke-move.spec.js` |
|
||||
|
||||
## Nested Notebooks
|
||||
|
||||
Joplin notebooks nest via the folder's `parent_id`. Joplock renders, creates, moves and deletes them on desktop and mobile. There is **no schema change** and no sidecar table: the tree is just `jop_parent_id` on the shared `items` rows, so Joplin desktop/mobile/CLI see exactly the same hierarchy.
|
||||
|
||||
### Where the logic lives
|
||||
|
||||
| Concern | File |
|
||||
|---|---|
|
||||
| Pure tree helpers (build/flatten/descendants/ancestors/roll-up counts/`canNestUnder`/option labels) | `app/items/folderTree.js` |
|
||||
| Create / move / delete / share-eligibility rules, shared by fragment + JSON routes | `app/items/folderOps.js` (`ctx.folderOps`, built in `createServer.js`) |
|
||||
| Subtree share stamping, recipient grant/revoke | `app/routes/shares.js` (`collectSubtreeItems`, `setShareOnItems`, `createShareSync`) |
|
||||
| Sync-proxy guard (no notebooks under a vault) | `app/proxy/vaultProxyGuard.js` (`enforceFolderWrite`) |
|
||||
| Desktop tree markup | `navigationFragment` in `app/templates/fragments.js` |
|
||||
| Mobile tree markup | `mobileFoldersFragment` in `app/templates/mobile.js` |
|
||||
| Pickers | `notebookOptionsHtml` (editor select), `folderPickerOptions` + `GET /fragments/folder-options` (move / new-notebook modals) |
|
||||
| Tests | `tests/folderTree.test.js`, `tests/folderOps.test.js`, `tests/folderRoutes.test.js`, `tests/nestedFoldersUi.test.js`, `playwright-tests/nested-folders.spec.js`, `playwright-tests/nested-share.spec.js` |
|
||||
|
||||
### Tree semantics (keep them)
|
||||
|
||||
- **Siblings keep input order.** `foldersByUserId` already orders by case-folded title; `folderTree` never re-sorts, so flat lists and the virtual "All Notes"/"Trash" rows come out unchanged.
|
||||
- **Defensive like Joplin's `buildTree`:** a folder whose parent is not visible (not shared with this user, deleted) is shown at the top level; a parent cycle never loops (first folder of the cycle in list order becomes a root); depth is capped at `MAX_DEPTH`. Every folder always appears.
|
||||
- **Counts:** the nav and mobile list show **rolled-up** counts (a collapsed parent is honest). `counts` from `folderNoteCountsByUserId` stays the **direct** count and still drives note pagination (`/fragments/folder-notes`), `data-note-count`, and "does this notebook have notes to fetch". Selecting a parent lists only its own notes; "All Notes" is the aggregate.
|
||||
- Sub-notebooks sort/render **before** the notebook's own notes.
|
||||
|
||||
### Desktop DOM contract (easy to break)
|
||||
|
||||
```
|
||||
.nav-folder[data-folder-id][data-parent-id][data-depth][style=--nav-depth:N]
|
||||
> .nav-folder-row
|
||||
> .nav-folder-children (only when it has sub-notebooks; contains more .nav-folder)
|
||||
> .nav-folder-notes[data-folder-id] (this notebook's OWN notes, lazy loaded)
|
||||
```
|
||||
|
||||
- **Never** use `el.querySelector('.nav-folder-notes…')` or descendant CSS (`.nav-folder.collapsed .x`) on a notebook: sub-notebooks are inside it and would match first / leak collapse state. Use `navFolderNotesDiv(el)` (direct child) and the child combinator in CSS (`.nav-folder.collapsed > .nav-folder-notes`).
|
||||
- **Multi-expand, no accordion.** Any number of notebooks can be open. Opening one also opens the notebooks above it (`toggleNavFolder`). `initNavPanel` trusts saved state; with no saved state the selected notebook **and its ancestors** open. Notes are lazy-loaded only for notebooks that are open **and visible** (`navFolderVisible`); expanding a parent loads already-open descendants (`navLoadOpenDescendants`). A notebook with 0 direct notes never requests a notes page.
|
||||
- Saved state is `localStorage['joplock-nav-folders']` (notes-list open).
|
||||
|
||||
### Drag and drop (desktop)
|
||||
|
||||
Drag a notebook onto another to nest it; drop it on the **"Drop here to move to the top level"** strip (only shown while dragging a *nested* notebook) to un-nest it. Hovering a closed notebook that has sub-notebooks for 700 ms opens it so deeper targets are reachable.
|
||||
|
||||
- The template decides who takes part: `navigationFragment` puts `draggable="true" data-dnd="1"` on the row of ordinary notebooks only (not All Notes, Trash, vaults or Joplin-encrypted notebooks). The script reads those attributes; the **server re-validates every move** (`PUT /fragments/folders/:id`), and a refusal is shown with `alert()` (e.g. "Stop sharing this notebook before moving it into another notebook").
|
||||
- Handlers (`navDragStart/Over/Leave/Drop/End`, `navDropAllowed`, `navMoveFolder` in `public/app.js`) are **delegated on `document`**, so they survive htmx re-renders. `dragenter` and `dragover` share one handler. `dragleave` must be judged by position when `relatedTarget` is null (Chromium fires it, with a null target, when the pointer crosses a row's own child elements); clearing the highlight on every `dragleave` made it flicker off. Visual changes happen a tick after `dragstart` because changing layout synchronously there cancels the drag in some browsers.
|
||||
- Desktop only: touch has no HTML5 drag-and-drop, mobile uses the "Move notebook..." sheet. Real-mouse coverage: `playwright-tests/nested-folders.spec.js` ("drag and drop"); pure rules and lifecycle: `tests/nestedFoldersUi.test.js`.
|
||||
|
||||
### Notebook order (A-Z / most recently updated)
|
||||
|
||||
Per-user setting `folderSort` (`alpha` default | `recent`; Settings -> Appearance, plus a quick switch in the sidebar header `#nav-sort-btn` and the mobile folders header `#mobile-sort-btn`, both calling `toggleFolderSort()`; `<body data-folder-sort>` picks which label shows).
|
||||
|
||||
- **What "recent" means** (`sortFoldersByRecent` in `app/items/folderTree.js`, data from `itemService.folderActivityByUserId`): a notebook's rank is the newest note activity **anywhere in its subtree** (so a parent rises with its busiest child, like Joplin's own "updated date" notebook sort), using the note's `user_updated_time` (falls back to the row time) and the notebook's creation time when nothing is in it. Trashed and conflict notes do not count. Because `user_updated_time` only moves on a real title/body edit (see Joplin Field Fidelity in `docs/agents/joplin-fidelity.md`), **moving or trashing a note does not reorder notebooks**.
|
||||
- **Parents always show.** The order is applied to the FLAT list before the tree is built, so reordering can never separate a notebook from its parent; siblings simply sort by their own rolled-up recency, ties keep alphabetical order.
|
||||
- **Where it applies:** the sidebar (`navData` -> `orderFoldersForUser`) and the mobile folders screen. **Pickers are always alphabetical** (editor notebook `<select>`, move / new-notebook dialogs): `sortFoldersByRecent` tags each folder with `alphaRank` and `inAlphabeticalOrder()` restores it, so the database's collation is never re-derived. The activity query only runs for users who chose "recent".
|
||||
- Switching order re-renders the nav via `/fragments/nav`; lazily loaded note lists now receive the open note's id (`navLoadFolderNotes`) so it stays highlighted.
|
||||
- Tests: `tests/folderTree.test.js` (ordering), `tests/folderSort.test.js` (SQL, setting, routes, pickers), `playwright-tests/folder-sort.spec.js`.
|
||||
|
||||
### Pinned notebooks
|
||||
|
||||
Pin a notebook to the top of the list: **Pin to top / Unpin from top** in the desktop context menu (`#folder-ctx-pin`) and the mobile long-press sheet (`#mobile-folder-ctx-pin`); a pin icon marks pinned rows (`.nav-pin-icon`, `.mobile-pin-icon`, `data-pinned="1"`).
|
||||
|
||||
- **Joplock-only, per user.** Stored as `pinnedFolders` in `joplock_settings` (max 100 ids, distinct); changed only through `PUT` / `DELETE /api/web/pinned-folders/:id`. **Nothing is written to the Joplin notebook** (Joplin has no pin concept), so Joplin apps are unaffected, other users do not see your pins, and a recipient can pin a notebook that was shared with them. Pinning a notebook that does not exist is a 404; a successful pin also forgets pins of notebooks that no longer exist (stale ids are otherwise ignored on render). Guarded by a byte-for-byte check of the Joplin item in `playwright-tests/folder-pin.spec.js` and a "no Joplin write" unit test.
|
||||
- **Ordering** (`applyPinnedFirst` in `app/items/folderTree.js`, applied in `orderFoldersForUser` after the A-Z / recent order): pinned notebooks come first among their siblings; among themselves they follow the active order. **Pinning a nested notebook pulls its ancestors up** (rank 1 = "contains a pinned notebook") so it stays near the top and is still drawn under its parents; ancestors are not themselves marked pinned. Like every ordering here it runs on the FLAT list, so a notebook can never be separated from its parent.
|
||||
- **Pickers ignore pins** (editor notebook select, move / new-notebook dialogs): `applyPinnedFirst` keeps/assigns `alphaRank` so `inAlphabeticalOrder()` restores the alphabetical order.
|
||||
- Tests: `tests/folderTree.test.js` (rules), `tests/folderPin.test.js` (setting, endpoints, routes, pickers, client helpers), `playwright-tests/folder-pin.spec.js`.
|
||||
|
||||
### Mobile contract
|
||||
|
||||
- The folders screen is an **inline expandable tree** (not drill-down), so `mobileBack` and the 3-screen stack are untouched. Rows carry `data-folder-id/-parent-id/-depth` and `--m-depth`; rows below the top level render `hidden` and `mobileApplyFolderTree()` reveals them from `localStorage['joplock-mobile-folders']` (sub-notebooks shown; separate from the desktop key). `.mobile-folder-row[hidden]` needs its own CSS rule because `display:flex` beats the UA `[hidden]`.
|
||||
- The chevron (`.mobile-folder-toggle`) is its own tap target and stops propagation; the row tap still calls `mobilePushNotes(id,title)`. **Keep the `mobilePushNotes("id","title")` onclick format**: `wireFolderRowLongPress` parses it. The chevron gutter is only rendered when something is nested.
|
||||
- Long-press sheet gained "New sub-notebook" and "Move notebook…". The note-move sheet (`mobileCtxMove`) reuses the editor `<select>`'s option text, so the tree indentation carries over.
|
||||
- Both expansion keys are cleared on logout (`pages.js`).
|
||||
|
||||
### Rules enforced server side (`folderOps`)
|
||||
|
||||
- **Structure:** not into itself or a descendant (mirrors Joplin's `canNestUnder`). Same-parent move is a no-op.
|
||||
- **Delete = promote.** Sub-notebooks move up to the deleted notebook's parent; its notes move to that parent (top level: the existing top-level "General", created if missing). Deleting never deletes notes or sub-notebooks. (Old behaviour moved every note to General and orphaned sub-notebooks.)
|
||||
- **Vaults are top-level leaves.** Cannot contain notebooks, cannot be moved, cannot be created nested (`POST /api/web/vaults` refuses a nested notebook or one with children), cannot end up in a share. The new-notebook modal disables the parent picker when "vault" is ticked. `vaultProxyGuard` also rejects a notebook PUT whose parent is a vault, **but see "Sync-proxy guards do not apply to stock Joplin clients" in `docs/agents/joplin-fidelity.md`: that guard is currently inert for real clients.**
|
||||
- **Sharing:**
|
||||
- Only the owner creates/moves/deletes inside a share (recipients get 403).
|
||||
- A **share root stays top level** (Joplin's rule): moving one under anything is refused ("Stop sharing…"); sharing a nested notebook **lifts it to the top level first** (`prepareForShare`; the share dialog says so); a notebook already inside a share cannot be shared again; a subtree containing a vault cannot be shared.
|
||||
- `ensureShareIdsOnNotebook` now stamps the **whole subtree** (every sub-notebook and the notes directly in each), writing `share_id`/`is_shared` into the item JSON **and** the `jop_share_id` column (falls back to JSON only if a server lacks the column). Creating a sub-notebook in a share inherits it; moving a subtree into/out of a share re-stamps it and grants/revokes recipient `user_items` (`createShareSync.setSubtreeShare`).
|
||||
- **Subtree share changes go through the API.** `createShareSync.setSubtreeShare` re-saves every notebook and note of the subtree with the new share fields (`itemWriteService`, lossless thanks to pass-through) so Joplin Server emits change events. Silent database writes are invisible to stock clients: a subtree moved **out** of a share used to stay in the recipient's Joplin app. Items the API cannot rewrite (Joplin-encrypted) get the database sweep only. Verified with a real client recipient in `integration-tests/joplin-sharing.test.js`.
|
||||
- **Stopping a share clears the dead `share_id` from the owner's whole tree** (notebook, sub-notebooks, notes; JSON and `jop_share_id`), like Joplin's own `unshareFolder`. Left in place, Joplin Server answers any later write of those items with `share not found` (422 — the note could no longer be edited or trashed) and **silently refuses to delete a root folder that still carries a share id** (returns 200, does nothing). Pinned by `playwright-tests/nested-share.spec.js` step 6.
|
||||
- Recipient access keeps the existing Joplock convention: `user_items.item_id` holds the **`jop_id`** (Joplin Server's own fan-out uses `items.id`, which `itemAccessExpression` does not read).
|
||||
- `updateFolder` now preserves `created_time`, `user_created_time`, `icon` and `master_key_id` (a rename/move used to reset them).
|
||||
|
||||
### Endpoints
|
||||
|
||||
- `PUT /fragments/folders/:id` and `PUT /api/web/folders/:id` accept `title` and/or `parentId` (`''` = top level; both in one write). `POST /fragments/folders` and `POST /api/web/folders` accept `parentId`. `DELETE` promotes.
|
||||
- `GET /fragments/folder-options?exclude=<id>&selected=<id>` → `<option>`s in tree order; the excluded subtree, vaults and foreign shares are `disabled`.
|
||||
- `GET /fragments/nav?folderId=<id>&withSelect=1` highlights/opens a notebook and refreshes the open note's notebook `<select>` (used after a move).
|
||||
|
||||
### Known pre-existing sharing gaps (not caused by nesting; not fixed here)
|
||||
|
||||
Verified live while building this: a **note created in a shared notebook after the share** is not visible to a Joplock recipient, and a **note moved out of a share stays visible** to the recipient (the note write paths never call grant/revoke; only folder operations do). Fixing means calling `createShareSync.setItemsShare` from the note create/move paths.
|
||||
77
docs/agents/pwa-mobile.md
Normal file
77
docs/agents/pwa-mobile.md
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
# PWA and mobile platform
|
||||
|
||||
> Read this before: touching the mobile layout or its screens, the viewport / soft keyboard, the service worker, static serving, iOS behaviour, or `/diag`. iOS findings here come from a real device, not an emulator.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## PWA & Mobile Platform
|
||||
|
||||
Everything about being an installed phone app lives here: which layout a device gets, how the iOS home-screen viewport and soft keyboard behave, what is cached, and how to measure it. **iOS behaviour here was found on a real iPhone Air home-screen install, not in an emulator**: Chromium cannot reproduce the iOS soft keyboard scrolling the document, the status-bar viewport bug, display cutouts or the home indicator. Before changing shell, viewport or keyboard layout, measure on a device with `/diag` (below) and compare numbers, not impressions. The app is an installable PWA with no offline notes: the only thing available offline is a static "you're offline" page.
|
||||
|
||||
### Which shell (mobile vs desktop)
|
||||
`isMobileShellMode()` (`_computeMobileShell`, `public/app.js`) is true for uiMode `mobile`, or in `auto` when the window is <=768px wide, **or** the device is a phone: `(pointer: coarse)` and a physical screen whose short side is <=500 CSS px (`_isPhoneScreen`). The second clause keeps a phone in landscape (812-956px wide) on the mobile shell; it used to get the desktop layout in ~390px of height and rotating reloaded the page (a shell flip reloads). The CSS mirror is the media query `(max-width: 768px), (pointer: coarse) and (max-device-width: 500px), (pointer: coarse) and (max-device-height: 500px)` on the four former `max-width: 768px` blocks in `styles.css`; iOS reports `device-width` as the portrait width in both orientations and Android rotates it, hence both. **Never use the window height** to decide: a soft keyboard shrinks the window (on iOS home-screen apps `innerHeight` shrinks with it), which would flip an iPad to the phone layout and reload the page while typing. Tablets (screen short side >=600) and mouse-driven windows keep the width rule only. The mobile shell is inset by `env(safe-area-inset-left/right)` (landscape notch / Dynamic Island), and `@media (max-height: 500px)` tightens the title bar and toolbar; in landscape with the keyboard up the editor's title bar (`#mobile-editor-header` only, never the search header) steps aside. Covered by `tests/landscapeShell.test.js` and `playwright-tests/mobile-landscape.spec.js` (the iPad case sets the physical screen apart from the window through CDP, because `page.setViewportSize` shrinks the emulated screen too). Mobile/tablet editor behavior should be reasoned about by editor container context, not just viewport width.
|
||||
|
||||
### iOS home-screen status bar (do NOT use black-translucent)
|
||||
Pages must not set `apple-mobile-web-app-status-bar-style` (leave it absent, as herdr-web-ui does). With `black-translucent`, an iOS standalone app sizes `innerHeight`/`100dvh`/`100svh` as *screen height minus the top safe-area inset*, anchored at the top, while `100lvh`/`100vh` still report the full screen. The shell then ends that many points above the bottom edge (a dead band painted in the page background) and every `position: fixed; bottom: …` element (FAB, bottom sheets, backdrops) floats too high. Measured on an iPhone Air: screen 812, `innerHeight` 751, top inset 61. Absent meta = content starts below the status bar, bottom edge lands on the hardware, safe-area-top is 0. iOS bakes the value in at install time, so after changing it the icon must be removed and re-added. `tests/pwaHead.test.js` guards this, that every `apple-touch-startup-image` file exists and matches its media query, and that each `100vh` has a `100dvh` fallback (WebKit reports `100vh`/`100lvh` overflowing in default-style standalone installs). Newer iPhones need their own splash entry in `app/templates/shared.js` + `scripts/generatePwaAssets.mjs` (iPhone Air 420×912 @3x, 16/17 Pro 402×874 @3x are included).
|
||||
|
||||
### Soft keyboard / viewport (`public/viewport.js`)
|
||||
|
||||
Loaded in `<head>` after the inline config and before `app.js`. On a touch device it decides whether a soft keyboard is up (`keyboardVisible`: a typing element is focused, or the keyboard was already up, AND the visual viewport has lost more than 20% of the largest unobstructed height seen; pinch zoom and non-touch never count) and then: sets `html[data-keyboard]` plus `--app-height` / `--app-top`, which `styles.css` applies to `.mobile-app` only in that state (outside it the shell is `inset:0` exactly as before); and, while the mobile shell is showing, puts any document scroll back to (0,0). Reason: iOS scrolls the document to reveal the caret (measured on an iPhone Air home-screen app: `scrollY` 268-342px even with `body{position:fixed}`), which carries the header and the markdown toolbar off the top of the screen; sizing the shell to the visible area and pinning the scroll leaves nothing to scroll. It fires `joplock:viewport` on `document`; `positionTinyMCEHost` uses `#mobile-app`'s bottom (not an `innerHeight` guess) and re-anchors on that event. Never reintroduce `innerHeight - visualViewport.height` keyboard guesses (an iOS home-screen app shrinks `innerHeight` with the keyboard). Kill switch on a device: `localStorage['joplock-viewport'] = 'off'`. With the admin debug flag on it posts `viewport.kb` events (keyboard state, visual viewport, scroll, how often it had to pin, header/toolbar positions on screen) to `/api/web/client-log`; read them with `node scripts/diag-reports.js --event viewport.kb` (flags `HDR-OFF`, `TB-OFF`, `TB-HIDDEN`, `PANNED`, `NOT-PINNED`). `./scripts/rebuild-dev.sh` archives device reports into `.diag-archive/` before recreating the container (a recreate wipes its logs). `/diag?shell=vv` runs this same module. Covered by `tests/viewport.test.js` (pure logic, real iPhone numbers) and `playwright-tests/mobile-toolbar-keyboard.spec.js`.
|
||||
|
||||
### Zoom, touch and hover CSS
|
||||
One `viewportMeta` constant (`app/templates/shared.js`) is used by every page: `width=device-width, initial-scale=1.0, viewport-fit=cover, interactive-widget=resizes-content`. Zoom is allowed (WCAG 1.4.4; `user-scalable=no` also blocks pinch zoom on Android; iOS Safari ignores it anyway). Double-tap zoom on controls is off through `touch-action: manipulation`, and `public/viewport.js` does not pin the page scroll while the user is pinch-zoomed (`shouldPin`). `interactive-widget=resizes-content` only affects Android Chrome. The end of `styles.css` has the touch block (`@media (hover: none) and (pointer: coarse)`: no text-size inflation, tap flash, selection or callout on controls; selection stays on fields and `.cm-content`) and a `prefers-reduced-motion` block (the loading ring keeps turning). Pure `:hover` feedback rules must live inside `@media (hover: hover)`: iOS emulates hover on tap and leaves it stuck (a tapped toolbar button stayed highlighted). Reveal-on-hover affordances that mix `:hover` with another state have `(hover: none)` fallbacks and are exempt. `tests/touchCss.test.js` lints this. `/diag` can reproduce the old meta (`?zoom=0`, `?iw=0`) for comparison.
|
||||
|
||||
### Static asset serving (`serveFile` in `app/routes/_helpers.js`)
|
||||
Compressible text assets (.js .css .json .svg .html .webmanifest, 1 KB to 4 MB) are served brotli/gzip, negotiated from `Accept-Encoding` (q-values respected), with `Vary: Accept-Encoding` and a content-hash `ETag` per encoding (a stale copy revalidates with a 304, not a re-download). Brotli (level 11) and gzip are computed once, asynchronously, and cached in memory per file+mtime; `warmStaticCache(publicDir)` does the root-level files at startup so the first visitor does not wait. A URL carrying `?v=<asset version>` is `immutable` for a year (every script and stylesheet in the app head carries it; `tests/pwaHead.test.js` enforces that); other files are `max-age=300` plus ETag; `service-worker.js` and the manifest are `no-cache`; html is `no-store`; images and fonts stream with a weak validator. Measured on the dev stack: shipped JS+CSS 2.5 MB raw → 0.77 MB (gzip) / 0.67 MB (brotli); cold relaunch with an expired cache on a slow 4G link 7.8s → 2.6s. The reverse proxy in front (Nginx Proxy Manager defaults) does not compress JS/CSS, so this must stay in the app. Covered by `tests/staticServing.test.js`. HTML responses are gzipped separately by `makeSendHtml`.
|
||||
|
||||
### Service worker policy (`public/service-worker.js`)
|
||||
Deliberately small. Navigations go to the network; if that fails the worker serves the static `/offline.html` (inline styles, a Try again button that checks `/health` and reloads, auto-retry on the `online` event). Versioned `.js`/`.css` URLs (`?v=<asset version>`, immutable by construction) are cache-first in a cache named `joplock-shell-<asset version>`; any other Joplock cache is deleted on activate, other apps' caches are left alone. Nothing else is intercepted: **no HTML page, API response, fragment, note or attachment is ever cached** (shared-browser safety; the app has no offline notes). The install step precaches only `/offline.html` and an icon. It used to precache bare URLs that the `?v=` requests never matched (so assets were stored twice) and never cached `app.js`/TinyMCE. Covered by `tests/serviceWorker.test.js` (the real file in a sandbox) and `playwright-tests/pwa-offline.spec.js` (real browser: cached assets, offline launch, recovery). Not done: an in-app "new version, reload" prompt; the new worker takes over immediately (`skipWaiting` + `claim`).
|
||||
|
||||
### Device diagnostics (`/diag`)
|
||||
|
||||
Debug-mode-only page (`app/routes/diag.js`, `app/templates/diag.js`) for measuring viewport, safe-area and soft-keyboard behaviour on a real phone. Linked from Settings → About when debug logging is on. It mocks the mobile shell (header, iframe + input + textarea like the real editors, bottom bar) and posts flat snapshots to `/api/web/client-log`, which land in the container log as `[joplock client] diag.viewport {…}`. Read them with `node scripts/diag-reports.js [--since 30m] [--event diag.viewport|viewport.kb] [--rid <id>] [--json]` (one compact line per report, flags `HDR-OFF` / `BAR-HID` / `BAR-GAP` / `NO-SAFE`). Variants: `?shell=current` (the plain fixed-inset shell, i.e. what iOS does with no help), `?shell=vv` (loads `public/viewport.js`, the production module: shell height follows `visualViewport` while the keyboard is up, page scroll pinned). The viewport meta defaults to the app's own (`viewportMeta`); `?iw=0` drops `interactive-widget=resizes-content` and `?zoom=0` blocks zoom (the old meta) for comparison. Report field names must not match the client-log key filter (`text|body|content|password|key|secret|token`); `tests/diagPage.test.js` enforces that. Use it before changing shell/keyboard layout so the change is compared against real-device numbers, not Chromium emulation (which cannot reproduce the iOS keyboard or notch).
|
||||
|
||||
## Mobile UI Model
|
||||
|
||||
### Shell structure
|
||||
|
||||
- `#mobile-folders-screen`
|
||||
- `#mobile-notes-screen`
|
||||
- `#mobile-editor-screen`
|
||||
|
||||
These screens are shown/hidden by inline JS in `layoutPage()` using class changes, not route changes.
|
||||
|
||||
### Mobile navigation behavior
|
||||
|
||||
- Folder-first flow: folders -> notes -> editor (the folders screen is an inline expandable tree for nested notebooks; see Nested Notebooks in `docs/agents/notebooks-and-sharing.md`)
|
||||
- Search has its own mobile header state
|
||||
- Mobile note creation uses dedicated fragment endpoints and server headers to drive the next UI step
|
||||
- The floating action button is only a mobile affordance; desktop should stay unaffected
|
||||
- FAB visibility should follow screen state directly (`folders` / `notes` visible, `editor` hidden), not only htmx swap side effects
|
||||
- In markdown mode the mobile formatting toolbar (`#editor-toolbar`) is permanent: `setMobileToolbarVisible(false)` is ignored there (only a locked vault hides it). Callers ask to hide it when the viewport handler thinks the keyboard went away, which is wrong on iOS home-screen apps (`innerHeight` shrinks with the visual viewport, so an open keyboard looks closed), and from the rich editor's blur handler after a switch to markdown. Do not reintroduce `innerHeight - visualViewport.height` keyboard guesses; see `/diag` for real-device numbers. Covered by `tests/mobileToolbarVisibility.test.js` and `playwright-tests/mobile-toolbar-keyboard.spec.js`.
|
||||
- The markdown editor's word-count / heading-outline bar (`.cm-jl-status`, `mdStatusBar` setting) is desktop-only: `_cmExtraExtensions` passes `statusBar: _mdStatusBar && !isMobileShellMode()`, so it is never created in the mobile shell (it costs a row under the editor and recounts words on every keystroke). The Outline jump-to-heading popup lives in that bar, so it is not available on mobile either. Covered by `tests/mobileStatusBar.test.js` and `playwright-tests/md-extras.spec.js`.
|
||||
- `#mobile-folders-body` / `#mobile-notes-body` carry `padding-bottom: calc(96px + env(safe-area-inset-bottom))` so the last row (or the notes list's "Load more" button) scrolls clear of the FAB (56px + 20px offset) and the home indicator. Scope it to those two ids: `#mobile-editor-body` shares `.mobile-screen-body` and must not get it. Covered by `playwright-tests/mobile-list-clearance.spec.js`.
|
||||
- Mobile folder rows can include a vault lock button and it must stay inline with the row actions
|
||||
|
||||
### Mobile editor behavior
|
||||
|
||||
- Mobile hides the desktop title bar and uses the mobile header instead
|
||||
- Mobile header mirrors note title and save state
|
||||
- Mode buttons should remain visible and clearly indicate the active mode
|
||||
- Toolbar visibility should be keyed to being inside the mobile editor container, not only screen width
|
||||
- Newly-created empty mobile notes may be discarded on back if still blank/untitled
|
||||
- Locked mobile notes should not reveal plaintext/editor surfaces until unlock
|
||||
|
||||
### Tablet expectations
|
||||
|
||||
- A tablet window wider than 768px gets the **desktop** shell (see "Which shell"); only a narrow window (<=768px) or a phone-sized touch screen gets the mobile shell
|
||||
- Existing note open path and new note open path should behave the same with respect to default open mode, toolbar visibility, and title/save-state UI
|
||||
- When debugging tablet issues, compare the exact htmx target and after-settle path used by new-note vs existing-note opens
|
||||
|
||||
## Debugging
|
||||
|
||||
### If mobile note creation/opening misbehaves
|
||||
|
||||
- Check whether the server response includes the expected mobile header such as `X-Mobile-Note-Id`
|
||||
- Check the `htmx:afterRequest` handler that consumes that header
|
||||
- Compare new-note path vs existing-note path
|
||||
- Check whether the note is in a vault and whether the editor was initialized in locked vs unlocked state
|
||||
99
docs/agents/settings-and-admin.md
Normal file
99
docs/agents/settings-and-admin.md
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
# Settings, themes, admin and ops
|
||||
|
||||
> Read this before: adding a setting or theme, changing the settings page, admin/backup/recovery, MFA, or debug logging.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Storage
|
||||
|
||||
- Settings are stored per-user in `joplock_settings.settings` as JSONB
|
||||
- `app/settingsService.js` owns defaults and normalization
|
||||
- Unknown or invalid values should normalize back to safe defaults
|
||||
|
||||
## What persists a setting
|
||||
|
||||
A setting only persists if it is (a) defaulted/normalized in `app/settingsService.js` and
|
||||
(b) listed in the `allowedKeys` array in `app/routes/api.js` (or handled by a `/admin/*` route
|
||||
for admin settings). Adding a key to only one of the two silently does nothing.
|
||||
|
||||
The list of settings and their defaults is `defaultSettings` / `normalizeSettings` in `app/settingsService.js`; it is not copied here.
|
||||
|
||||
`noteMonospace` is not a separate UI switch: it is derived from `noteFontFamily === 'mono'`
|
||||
(via the `note-body-monospace` class on `<body>`).
|
||||
|
||||
## Debug Logging (runtime-toggleable)
|
||||
|
||||
- `appSettings.debugLogging` is tri-state (`normalizeTristate()` in `app/settingsService.js`): `null` = inherit env `DEBUG`; `true`/`false` = explicit admin override persisted in DB.
|
||||
- `app/createServer.js` keeps `effectiveDebug` in memory; `refreshDebugLogging()` re-reads the DB on startup and after an admin save. `isDebug()` is read dynamically by the request logger and passed into ctx; `_joplockConfig.debug` and the settings-page inline `DEBUG` flag both reflect the effective state.
|
||||
- Toggled via the "Enable debug logging" checkbox in Settings → Admin → Login Security. Applied immediately, no restart. `DEBUG` env in the compose files is only the startup default.
|
||||
|
||||
## The settings page is a full page, not a modal
|
||||
|
||||
- `/settings` is a standalone SSR page (full navigation away from the app), not an htmx fragment or modal. It has its own inline `<script>` and does NOT load `app.js`.
|
||||
- Esc dismisses it (returns to `/`) via `<body onkeydown="...">` — an HTML attribute so it fires before any JS and can't be killed by a script error. It flashes the page background briefly as visual confirmation. Do not move this back into an `addEventListener` inside the IIFE; that proved unreliable (IIFE errors / event-propagation quirks swallowed Esc).
|
||||
- Settings auto-save, so Esc has no unsaved-text concern.
|
||||
|
||||
## Adding a new setting
|
||||
|
||||
1. Add default + normalization in `app/settingsService.js`
|
||||
2. Allow the key in the `allowedKeys` array of `PUT /api/web/settings` in `app/routes/api.js` (or in the matching `/admin/*` route for admin-only settings)
|
||||
3. Add the UI in `settingsPage()` in `app/templates/settings.js`
|
||||
4. If needed, inject the normalized setting into `layoutPage()` / `_joplockConfig` in `app/templates/pages.js` for `public/app.js`
|
||||
5. Nothing to bump: the asset version and service worker cache name are computed from the shipped files (`app/assetVersion.js`)
|
||||
6. Rebuild with `./scripts/rebuild-dev.sh`
|
||||
|
||||
## Adding or editing a theme
|
||||
|
||||
Themes are CSS-only. Each theme is a class on `<body>` that sets a shared set of CSS custom properties.
|
||||
|
||||
**Files to touch:**
|
||||
|
||||
1. **`public/styles.css`** — Add/edit a `.theme-<slug>` block that defines the same set of custom properties used by every other theme.
|
||||
- Minimum properties that must be defined: `--bg`, `--theme-color`, `--bg-side`, `--bg-list`, `--bg-editor`, `--bg-elevated`, `--bg-input`, `--bg-hover`, `--bg-active`, `--overlay`, `--shadow`, `--text`, `--text-dim`, `--text-muted`, `--text-heading`, `--text-on-accent`, `--accent`, `--border`, `--border-focus`, `--danger`, `--toolbar-bg`, `--scrollbar`, `--statusbar-bg`, and `color-scheme` (`light` or `dark`).
|
||||
- `--bg-elevated` in particular is not optional: it is forwarded into the TinyMCE iframe by `_syncTinyMCEThemeVars()` and backs the image-download button background. A theme missing it renders that button transparent.
|
||||
- Keep numbers and hover/active states neutral unless the theme intentionally uses color.
|
||||
- The markdown toolbar and the TinyMCE toolbar both share `color-mix(in srgb, var(--accent) 10%, var(--bg))`. Setting a sensible `--accent` and `--bg` is enough; no extra toolbar work needed.
|
||||
|
||||
2. **`app/settingsService.js`** — Add the theme slug to the `validThemes` array at the top of the file (it is the allowlist that keeps a persisted slug from being normalized back to the default).
|
||||
|
||||
3. **`app/templates/shared.js`** — Add `[<slug>, <displayName>]` to `themeOptions` so it appears in the status bar picker and the settings page.
|
||||
|
||||
4. **`tests/settingsService.test.js`** — Add an assertion that `normalizeSettings({ theme: '<slug>' }).theme` is preserved and not normalized back to the default.
|
||||
|
||||
5. **Service worker / asset version** — nothing to do: `app/assetVersion.js` hashes the shipped files, so new theme CSS changes the version, the `?v=` URLs and the worker's cache name automatically. Add the theme's `--theme-color` in its CSS block (`app/themeColors.js` reads it for the page head; `tests/themeColors.test.js` fails if a selectable theme lacks one).
|
||||
|
||||
6. Rebuild with `./scripts/rebuild-dev.sh`.
|
||||
|
||||
No changes are needed in `pages.js`, `app.js`, or `settings.js`: those all read `themeOptions` or apply `theme-${settings.theme}` dynamically.
|
||||
|
||||
## MFA Notes
|
||||
|
||||
- MFA is per-user, managed via Settings → Security → Two-Factor Authentication.
|
||||
- Each user's TOTP seed is stored in `joplock_settings.totp_seed` in the shared Postgres DB.
|
||||
- No global/shared TOTP seed. The old `JOPLOCK_TOTP_SEED` / `JOPLOCK_TOTP_ISSUER` env vars are removed.
|
||||
- `IGNORE_ADMIN_MFA=true` skips the per-user MFA check at login for the docker-defined admin account (`JOPLOCK_ADMIN_EMAIL`). Other users are unaffected.
|
||||
- Admin can force-enable/disable MFA for any user via the Admin tab (no code required).
|
||||
|
||||
|
||||
## Admin & Ops Surface
|
||||
|
||||
### Who is "admin"
|
||||
|
||||
- Admin is **not** a role grant inside Joplock. `isJoplockAdmin()` in `app/createServer.js` returns true only for the single account whose email equals `JOPLOCK_ADMIN_EMAIL` **and** whose Joplin `users.is_admin` flag is set (`app/auth/sessionService.js`). If `JOPLOCK_ADMIN_EMAIL` is unset, `adminService` is `null` and there is no admin at all — so the Admin tab and the `/admin/*` routes disappear entirely.
|
||||
- Only that account sees the Admin tab on `/settings`; `appSettings` (the `__app__` row) is read only for admins (`app/routes/settings.js`).
|
||||
- `server.js` calls `adminService.ensureAdminUser()` (best-effort, non-blocking) so the account exists in Joplin with the configured password on startup.
|
||||
|
||||
|
||||
### Backup / restore
|
||||
|
||||
- `app/backupService.js` shells out to `pg_dump`/`pg_restore` for the **entire shared database** (Joplin's and Joplock's tables together), writing `<name>.dump` files under `JOPLOCK_BACKUP_DIR` with path-containment checks. Compression is `JOPLOCK_BACKUP_COMPRESSION` (default `zstd:19`) or `JOPLOCK_BACKUP_COMPRESSION_LEVEL` (default `9`).
|
||||
- A restore **replaces the whole database**, Joplin data included. `README.md` documents the operational order (stop/quiesce Joplin Server, stop sync clients).
|
||||
- Backups are only durable if `JOPLOCK_BACKUP_DIR` is on persistent storage.
|
||||
|
||||
### Break-glass recovery
|
||||
|
||||
- `JOPLOCK_RECOVERY_ENABLED=true` + `JOPLOCK_RECOVERY_PASSWORD` expose `/recovery` (`app/routes/recovery.js`, `app/recoveryService.js`): its own login, its own short-lived session (`JOPLOCK_RECOVERY_SESSION_TTL_MINUTES`, default 30), and backup/restore without normal Joplin auth. It is for backup/restore only, not note editing.
|
||||
- Recovery is separate from MFA: it does not consult `joplock_settings.totp_seed`.
|
||||
|
||||
### Login rate limiting
|
||||
|
||||
- `app/auth/rateLimitService.js` is an in-memory, per-credential-attempt limiter; the ceiling is the admin `authRateLimitAttempts` setting (default 20 per 15 minutes). It is not persisted — a Joplock restart clears the counters.
|
||||
116
docs/agents/testing-and-dev.md
Normal file
116
docs/agents/testing-and-dev.md
Normal file
|
|
@ -0,0 +1,116 @@
|
|||
# Testing and the dev stack
|
||||
|
||||
> Read this before: running or writing tests (unit, Playwright), rebuilding the dev stack, or when a code change does not show up.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Verification
|
||||
|
||||
- Run tests: `npm test`
|
||||
- Browser E2E: `npm run test:ui` (needs a live dev stack + admin env vars, see below)
|
||||
- Build image: `npm run docker:build`
|
||||
- Sidecar-only compose: `npm run docker:up`
|
||||
- Full example compose: `npm run docker:up:full`
|
||||
- Full example compose built from source: `npm run docker:up:build` (uses `docker-compose.example-full-build.yml`)
|
||||
- Dev stack: `npm run docker:up:dev` / `./scripts/rebuild-dev.sh` (see "Development Stack")
|
||||
- Rebuild PWA assets: `npm run generate:pwa-assets`
|
||||
|
||||
### Playwright credentials
|
||||
|
||||
- Tests NEVER hardcode credentials. `playwright-tests/helpers.js` resolves the admin account from the environment in this order: `PLAYWRIGHT_ADMIN_EMAIL` → `PLAYWRIGHT_EMAIL` → `JOPLOCK_ADMIN_EMAIL` (and the `*_PASSWORD` equivalents). The dev container sets `JOPLOCK_ADMIN_*`, so the useful tests work against it out of the box.
|
||||
- `login()` calls `requireCredentials()` and fails loudly if none are set. Admin-only specs (`resource-lifecycle`, `auth-rate-limit`) use `hasAdminCredentials()` to `test.skip` when unset. `helpers.js` exports `ADMIN_EMAIL`/`ADMIN_PASSWORD`/`hasAdminCredentials` so specs share one source of truth.
|
||||
|
||||
### Playwright helper robustness (do not simplify these away)
|
||||
|
||||
The shared helpers absorb races in the app's two-phase htmx settling, not test flakiness:
|
||||
|
||||
- `setNoteBody()` clicks whichever markdown toggle is **visible** (`#editor-panel #markdown-toggle:visible`, `#mobile-editor-body #markdown-toggle:visible`, `#mobile-md-toggle:visible`). The mobile shell hides the editor fragment's own MD toggle and shows the header `#mobile-md-toggle` from `pages.js`; a locator that does not filter on `:visible` can click a hidden element and silently inject into a textarea TinyMCE then overwrites.
|
||||
- `setNoteTitle()` waits for the note-creation swaps to land, then sets the title and **verifies it stuck**, retrying up to 8 times. The creation response re-renders the editor with server state (`Untitled note`) after the form first appears, wiping an early title injection.
|
||||
- `deleteNotebook()` waits for the `#nav-panel` swap to settle and retries the delete while a stale/duplicated folder row is still present.
|
||||
- Specs must not depend on a user's editor settings: `md-extras.spec.js` pins the markdown-editor settings in `beforeAll` and restores the account's real values in `afterAll`; other specs read the CodeMirror *document* (`getCM().state.doc`), not `.cm-content` text, because "hide formatting marks" removes `## ` from the DOM of lines the caret is not on. Specs whose flow needs a native `confirm()` (e.g. `deleteNotebook()`) must call `acceptDialogs(page)`; the two-account share specs are desktop-only.
|
||||
- When adding a helper, prefer a shared one in `helpers.js` over a per-spec local; delete helpers only after confirming no spec imports them.
|
||||
|
||||
### Playwright data cleanup (do not leave notes/notebooks/resources behind)
|
||||
|
||||
- **Tests must not leave data in the shared Joplin DB.** Every spec that creates notebooks/notes/resources must clean them up.
|
||||
- Use `teardownTestData(page, { folders, folderPrefixes, titlePrefixes, noteIds })` from `helpers.js` in a `finally` block. It permanently removes the matching notes (trash + purge via `DELETE /fragments/notes/:id` twice), deletes the folders, empties trash, and cleans orphaned resources. It is best-effort (never throws), so it is safe in `finally` even after a failed assertion.
|
||||
- Prefer `{ folders: [folder] }` — purges every note inside the notebook *then* deletes the notebook. (Plain `deleteNotebook()` is NOT enough on its own: deleting a notebook never deletes its notes — they move to the parent notebook, or **General** at the top level — so they leak.)
|
||||
- For notes created outside a dedicated notebook (e.g. mobile "New note" in **All Notes**), capture the id with `getActiveNoteId(page)` and pass `{ noteIds: [id] }`.
|
||||
- A suite-wide safety net runs automatically: `playwright.config.js` `globalTeardown` (`playwright-tests/global-teardown.js`) logs in once after the whole run and purges any leftover test-prefixed folders/notes + orphaned resources (folder name prefixes like `pw-`, `dnd-`, `esc-`, `search-`, `upload-`, `res-lifecycle-`, and known test note-title prefixes). Keep those prefix lists in sync when you add new test naming.
|
||||
- Cleanup relies on `GET /api/web/notes/headers` returning `parentId` (added for this), `DELETE /api/web/folders/:id`, `DELETE /fragments/notes/:id` (trash then purge), `POST /fragments/trash/empty`, and `POST /admin/orphaned-resources/cleanup`.
|
||||
- Verify a change doesn't leak by running a data-creating spec twice and confirming the DB item counts are identical before/after (they must be stable, i.e. zero accumulation).
|
||||
|
||||
### Playwright screenshots
|
||||
|
||||
- Screenshots are captured on every test run (pass or fail) under `test-results/`, named `{testTitle}-{project}-{browser}-{retry}.png`. Configured via `use.screenshot: 'on'` in `playwright.config.js`.
|
||||
- Videos are still `retain-on-failure`, traces `on-first-retry`, and `test-results/` is already git-ignored.
|
||||
|
||||
### Playwright share tests
|
||||
|
||||
Share tests require the admin account and two browser contexts (owner + recipient). The shared reader user is created automatically by `ensureShareTestUsers()` via the admin API.
|
||||
|
||||
```
|
||||
# All share tests (requires live dev stack)
|
||||
npx playwright test playwright-tests/share-*.spec.js --project=desktop
|
||||
|
||||
# Individual specs
|
||||
npx playwright test playwright-tests/share-modal.spec.js --project=desktop
|
||||
npx playwright test playwright-tests/share-access.spec.js --project=desktop
|
||||
npx playwright test playwright-tests/share-revoke-move.spec.js --project=desktop
|
||||
```
|
||||
|
||||
Env vars (same credential chain as other Playwright tests via `JOPLOCK_ADMIN_*`):
|
||||
```
|
||||
JOPLOCK_ADMIN_EMAIL="admin@example.com" JOPLOCK_ADMIN_PASSWORD="..." npx playwright test ...
|
||||
```
|
||||
|
||||
Multi-user fixtures use `test.extend` with per-test `ownerPage`/`readerPage` (separate browser pages). All tests are `desktop`-only and skip on mobile (right-click context menu). Dialogs are auto-accepted by `acceptDialogs()` in `beforeEach`.
|
||||
|
||||
Test data uses `slug('share-...')` prefixes and is cleaned via `teardownTestData` per test. The global teardown also purges `share-*` prefixed folders (add `'share-'` to the prefix list in `global-teardown.js` if missing).
|
||||
|
||||
## Development Stack
|
||||
|
||||
Use the dev compose stack for all development work. It includes Postgres, Joplin Server, and Joplock together.
|
||||
|
||||
- Rebuild Joplock app container after code changes: `./scripts/rebuild-dev.sh`
|
||||
- Start / restart full dev stack: `docker compose -f docker-compose.dev.yml up -d --build`
|
||||
- Stop dev stack: `docker compose -f docker-compose.dev.yml down`
|
||||
|
||||
Do not use the sidecar-only `docker-compose.yml` for development.
|
||||
|
||||
Important:
|
||||
- `docker compose ... restart joplock` is not enough after source edits because the Docker image copies `app/`, `public/`, and `server.js` at build time.
|
||||
- For app code changes, use `./scripts/rebuild-dev.sh` from now on.
|
||||
|
||||
Recommended inner loop:
|
||||
|
||||
1. Edit source
|
||||
2. Rebuild with `./scripts/rebuild-dev.sh`
|
||||
3. Refresh the app
|
||||
4. Check `docker compose -f docker-compose.dev.yml logs --tail=... joplock` if something looks wrong
|
||||
|
||||
## Reference Material
|
||||
|
||||
- Mobile UX reference, only if you have the Joplin monorepo checked out next to this repo: `~/dev/joplin/packages/app-mobile/`
|
||||
- Use it for interaction ideas and behavior parity targets, not as a copy-paste implementation source
|
||||
- Joplock must still fit the SSR + htmx sidecar architecture
|
||||
|
||||
## Debugging: If a code change does not appear in the app
|
||||
|
||||
- Rebuild with `./scripts/rebuild-dev.sh`
|
||||
- Do not rely on `docker compose ... restart joplock` after source edits
|
||||
- If still stale, inspect the built container logs and confirm the right compose stack is running
|
||||
|
||||
## Tests
|
||||
- `tests/*.test.js` — unit/integration tests (`node:test`; several suites extract real functions out of `public/app.js` and run them under a JSDOM harness)
|
||||
- Run: `npm test` (i.e. `node --test tests/*.test.js`)
|
||||
- Real Joplin client compatibility (opt-in, needs the dev stack): `npm run setup:joplin-cli` once, then `npm run test:joplin` (`integration-tests/*.test.js`)
|
||||
- `playwright-tests/*.spec.js` — browser E2E against a live dev stack; run with `npm run test:ui`
|
||||
|
||||
## Configuration, compose files and built bundles
|
||||
|
||||
Facts that are not obvious from the repo layout:
|
||||
|
||||
- **Configuration is inline environment variables in the compose files** (they are commented); there is no committed `.env.example`, and `.env` is git-ignored. The variables themselves are read in `server.js` and the services; do not copy a list of them into docs.
|
||||
- `docker-compose.dev.yml` is the per-developer dev stack (Postgres + Joplin Server + Joplock from source). It is **git-ignored**; if it is missing, recreate it from `docker-compose.example-full-build.yml`. `./scripts/rebuild-dev.sh` and `npm run docker:up:dev` use it. Do not use the sidecar-only `docker-compose.yml` for development.
|
||||
- **Bundles are built, never hand-edited:** `cm-build/` -> `public/codemirror.min.js` (`npm run build:cm`; the markdown extras need extra CodeMirror symbols exported from `cm-build/index.js`), `hljs-build/` -> `public/hljs.min.js` (`npm run build:hljs`), `prism-build/` -> `public/prism.min.js` (`npm run build:prism`). `public/tinymce/` is the TinyMCE 8 npm package copied in. `public/html-docx.js` is the legacy client-side DOCX path. `public/reference.docx` is the pandoc reference document for DOCX styling (regenerate with `scripts/build-reference-docx.sh`).
|
||||
- Icons and splash images are generated: `npm run generate:pwa-assets` (see `docs/agents/pwa-mobile.md`).
|
||||
72
docs/agents/vaults-and-save-safety.md
Normal file
72
docs/agents/vaults-and-save-safety.md
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
# Vaults, encryption and save safety
|
||||
|
||||
> Read this before: touching vaults/encryption, ANY autosave or save path, note history/restore, or code that reads the editor DOM during a note switch. These guards each exist because a real note was overwritten.
|
||||
> This file records *why* and *what not to regress*. It deliberately does not say where code lives (ask the code index, or grep): maps go stale, reasons do not.
|
||||
|
||||
## Vault / Encryption Model
|
||||
|
||||
- Vaults are notebooks/folders with metadata stored in `joplock_vaults`
|
||||
- Titles stay plaintext
|
||||
- Notebook names stay plaintext
|
||||
- Note body ciphertext is stored in normal Joplin note bodies using Joplock markers for compatibility
|
||||
- Browser crypto stays client-side only; server never receives vault passwords
|
||||
- A note inside a vault notebook must be treated as protected even if its stored body is still plaintext during transition states
|
||||
- Locked vault notes render the lock overlay plus hidden editor shells; do not remove the hidden editor DOM because unlock logic depends on it
|
||||
- Unlocking a vault note with markdown as the preferred open mode must seed CM6 from the decrypted `plaintext` argument. Do not call `setEditorMode('markdown')` on unlock: that path `tinyMCESyncToTA()`s leftover/empty TinyMCE content over the just-decrypted body (locked notes skip editor init, so TinyMCE never held this note). `tinyMCESyncToTA()` must also refuse to write when `#tinymce-host` is not visible.
|
||||
- Clicking a vault lock while unlocked should lock immediately and close the open note if it belongs to that vault
|
||||
- Startup/refresh must never auto-resume an encrypted note or a note inside a vault notebook
|
||||
|
||||
### Encrypted-save identity guard (do not remove)
|
||||
|
||||
Encrypted-note autosave is debounced 2s. A hard-won bug was that the timer callback captured the outgoing note's `form`/`noteId`/`vaultId`, but read plaintext from the live DOM (`getTA()`). If the user switched notes during the debounce, plaintext of note B was encrypted with note A's vault key and PUT to note A's URL, silently overwriting A's ciphertext with an encryption of B's body. On next unlock, note A "decrypted" to note B's plaintext.
|
||||
|
||||
Client (`public/app.js`) rules:
|
||||
|
||||
- The encrypted `scheduleSave` override and `buildFlushRequest` MUST read `ta` from the captured form (`form.querySelector('textarea[name="body"], textarea.editor-body')`), never from `getTA()`.
|
||||
- Every encrypted-save path calls `_encryptedSaveIdentityOk(form, expectedNoteId, expectedVaultId)` before hashing, before encrypting, after every `await`, and inside `_triggerEncryptedSave`. Identity checks compare captured `noteId`/`vaultId` against `_formNoteId(form)`, `form.dataset.vaultId`, `form.dataset.encrypted`, `form.isConnected`, `activeEditorForm()`, and `_activeEditorNoteId()`. Abort with a log line if any mismatch.
|
||||
- `encryptForVault(plaintext, vaultId, key, salt, noteId)` embeds `noteId` in the ciphertext blob. Every call passes it.
|
||||
- `htmx:beforeSwap` for `#editor-panel` / `#mobile-editor-body` cancels `_saveTimer` and `_saveTitleTimer` so stale timers can't fire against a fresh note (defence-in-depth alongside the identity guard).
|
||||
|
||||
Server (`app/routes/_helpers.js`) rules:
|
||||
|
||||
- `assertVaultNoteBodyEncrypted(vaultService, userId, existingParentId, targetParentId, body, noteId, opts)` parses the ciphertext blob and rejects the write when `meta.vault` mismatches the target folder's vault, or `meta.noteId` (if present) mismatches the target note.
|
||||
- **Vault boundary enforcement** (three layers):
|
||||
1. **ParentId immutability / no conflict copies**: if a note currently lives in a vault folder, the server rejects any PUT that changes its `parentId` (400: `Vault notes cannot be moved to a different folder`). Vault notes cannot change folder — the folder select is disabled client-side and the server enforces this on every write path. Conflict `createCopy` is also rejected for vault notes: ciphertext is bound to source note id, while copying unlocked DOM content would make a plaintext duplicate.
|
||||
2. **Ciphertext required inside vaults**: saving to a vault folder always requires an encrypted body. Same-folder saves are covered; sync-proxy writes are covered **only for cookie-authenticated (browser) traffic, not stock Joplin clients** (see "Known issues found with the real client" (`docs/agents/joplin-fidelity.md`)). `enforceExistingVault=true` (sync-proxy only) extends this to prevent external Joplin clients from stripping ciphertext by re-parenting.
|
||||
3. **Metadata integrity**: encrypted blobs carry `vault` (folder id) and `noteId` (bound target); both are validated against the destination to prevent cross-vault or cross-note ciphertext smuggling.
|
||||
- **Client-side**: the folder select (`#editor-folder-select`) is rendered `disabled` for vault-protected notes and stays disabled after unlock — vault notes cannot change parent folder through the UI. The only reachable folder-change path is plain→vault (encrypt on move).
|
||||
- Every note-write path passes the target `noteId`: `app/routes/api.js` (PUT), `app/routes/fragments.js` (autosave PUT), `app/routes/history.js` (restore), `app/proxy/vaultProxyGuard.js` (sync proxy).
|
||||
- Legacy blobs without `noteId` still pass (backwards compatible); new writes are note-id-bound.
|
||||
|
||||
Tests must not regress this: an encrypted note's ciphertext blob's `noteId` field must equal the note id it is stored under; a write with a mismatched blob must be rejected with 400 (or 403 via the proxy guard).
|
||||
|
||||
### Plaintext save identity guard (do not remove)
|
||||
|
||||
Plaintext (non-encrypted) autosave has its own cross-note contamination race, fixed after a user report of "note B's body replaced by note A's content". Root cause: the persistent TinyMCE singleton is shared by all rich-mode notes, and several async `/fragments/preview` fetches wrote their result into it with no check that the active note was still the one the fetch was started for. A late preview response for note A landed while note B's form was active; the next TinyMCE→textarea sync copied A's markdown into B's `#note-body`, and the 2s autosave PUT it to `/fragments/editor/B`.
|
||||
|
||||
Client (`public/app.js`) rules:
|
||||
|
||||
- **Provenance stamps**: `_displayedNoteId` (note whose content the user last saw) and `_tinymceContentNoteId` (note whose rendered HTML TinyMCE last loaded) are stamped by `initEditorPanel` and `_setTinyMCEContent(html, noteId)`; `htmx:beforeSwap` for the editor containers clears both. Empty stamps are permissive (defence-in-depth only), non-empty mismatching stamps are hard blocks.
|
||||
- **Guard**: `_plaintextSaveIdentityOk(form)` must pass before any plaintext body PUT: form connected, `hx-put` note id === `data-note-id` === `_displayedNoteId`, form is `activeEditorForm()`, and — in rich mode with the TinyMCE host visible — `_tinymceContentNoteId` matches too. It is enforced in `scheduleSave`, `scheduleSaveTitle`, `buildFlushRequest`, and an `htmx:configRequest` choke point that blocks *any* `hx-put=/fragments/editor/…` request (manual, conflict-button, or `joplock:save`-triggered) fired from a non-active form. The choke point skips `dataset.encrypted==='1'` forms (those are covered by the encrypted guard above).
|
||||
- **Async fetch discipline**: every `/fragments/preview` fetch that writes into the shared editor (`setEditorMode('rich')`, `refreshTinyMCEForActiveNote`) captures the note id before the request and discards the response if the note, mode, or active form changed mid-flight. `tinyMCESyncToTA` and `_lazyTinyMCESyncBeforeSave` refuse to copy TinyMCE content into a textarea whose note doesn't match `_tinymceContentNoteId`.
|
||||
- **Other guarded paths**: `_completeUnlock` aborts if the unlocked note is no longer the active note; late `htmx:afterRequest` save responses from detached/replaced forms don't stamp `snapshotHash` (would mark a switched-to note as "Saved" while dropping its pending edits); `flushSave` success only updates save state when the flushed form is still active.
|
||||
- **flushSave baseUpdatedTime sync (do not remove)**: `flushSave` saves via a raw `fetch()` whose OOB-carrying response body is discarded — unlike the htmx autosave path, nothing would refresh the form's hidden `baseUpdatedTime`. A flush save advances the server clock while the form keeps the old base, so the NEXT autosave PUT trips the server conflict guard and the user sees "A newer version of this note exists on the server" after merely switching tabs/views (visibilitychange → flushSave). Fix: the editor PUT sets an `X-Note-Updated-Time` response header (mirror of the `#editor-sync-state` OOB), and flushSave reads it to refresh the form's `baseUpdatedTime`. flushSave also detects the `X-Note-Conflict` header and surfaces the conflict fragment + banner instead of wrongly marking the editor "Saved".
|
||||
- **flushSave must fetch with `keepalive: true` (do not remove)**: the flush usually fires from `visibilitychange` while the document is being torn down (tab close, note switch via navigation). A plain `fetch()` is aborted mid-flight, Chromium sends a truncated request whose `Cookie` header never reaches the server → 401, and the pending title/body edits are silently lost. `keepalive` lets the request outlive the document. Pinned by `tests/saveIdentityGuard.test.js`.
|
||||
- **Mobile shell conflict participation (do not remove)**: `mobileEditorFragment` re-adds `#editor-sync-state` (with `baseUpdatedTime`) after the titlebar-stripping transform. Without it, mobile saves carry no base → the server skips the conflict check (silent last-write-wins) and `checkNoteFreshness` early-returns (base 0) → the desktop↔mobile switch never detects concurrent changes.
|
||||
- **Shell-scoped banner**: both shells render `#remote-update-bar` with duplicate ids; `showRemoteUpdateBanner`/`dismissRemoteUpdateBanner` resolve the bar via `queryActiveEditor('#remote-update-bar')` first — `getElementById` alone returns the desktop shell's bar, which is `display:none` in the mobile shell (banner invisible exactly when mobile users need it).
|
||||
|
||||
Tests: `tests/saveIdentityGuard.test.js` locks the guard wiring, the flushSave header handling, and the shell-scoped banner; `tests/createServer.test.js` asserts the `X-Note-Updated-Time` header; `tests/templates.test.js` asserts the mobile sync-state; `tests/tinymceOnEditSync.test.js` has a provenance test for the sync refusal.
|
||||
|
||||
### Note-history restore
|
||||
|
||||
- `POST /fragments/history/:noteId/restore/:snapshotId` returns `editorFragment` **inline** (target = `#editor-panel` / `#mobile-editor-body`) with `#autosave-status` and `#nav-panel` as OOB swaps. Do not go back to swapping only `#autosave-status` with an OOB editor-panel — the editor-swap lifecycle (`htmx:afterSwap` destroy CM6, `htmx:afterSettle` reinit) only runs when the request target is the editor container, and without it the restored body doesn't appear until a page refresh.
|
||||
- The client `restoreHistorySnapshot()` cancels `_saveTimer` / `_saveTitleTimer` and clears `_savedHash` before firing the request so a stale autosave from pre-restore edits can't overwrite the restored body.
|
||||
|
||||
## Debugging
|
||||
|
||||
### If vault behavior misbehaves
|
||||
|
||||
- Check whether the folder is marked with `isVault`
|
||||
- Check whether the note is marked with `inVault` / `isEncrypted` / `vaultId`
|
||||
- Check `toggleVaultLock()`, `unlockNote()`, `_completeUnlock()`, and `flushSave()` in `public/app.js`
|
||||
- Check whether the hidden editor shells still exist in locked editor HTML
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
/* joplock client application — extracted from templates.js */
|
||||
/* Server config is passed via window._joplockConfig set inline before this script loads */
|
||||
/* Contracts for this file (editor, saves, vaults, uploads, mobile shell, AI) are in docs/agents/: editor.md, vaults-and-save-safety.md, editor-uploads.md, pwa-mobile.md, ai-expander-ask.md. Read the one that matches your change first. */
|
||||
(function(){
|
||||
var _cfg=window._joplockConfig||{};
|
||||
var _dbg=_cfg.debug||false;
|
||||
|
|
@ -2667,7 +2668,7 @@ function askDisabledForActiveNote(){
|
|||
}
|
||||
// User-visible explanation for why /ask silently no-ops in vault notes. Shown
|
||||
// once per note-open so users don't keep re-typing hoping it will work. The
|
||||
// underlying policy (see AGENT_GUIDE "Ask Slash Command") is that plaintext
|
||||
// underlying policy (see docs/agents/ai-expander-ask.md) is that plaintext
|
||||
// context from an encrypted note must never leave the browser.
|
||||
var _askDisabledNotified={};
|
||||
function _notifyAskDisabledInVault(){
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
/* Joplock markdown-mode (CodeMirror 6) extras.
|
||||
* Contracts and reasons: docs/agents/editor-markdown-extras.md
|
||||
*
|
||||
* Loaded after codemirror.min.js and before app.js. Exposes `window.JoplockMd`:
|
||||
*
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
// Joplock service worker. Policy (AGENT_GUIDE "PWA shell"): never cache anything user-specific.
|
||||
// Joplock service worker. Policy (docs/agents/pwa-mobile.md, "Service worker policy"): never cache anything user-specific.
|
||||
// - navigations: network, and when the network is down a static /offline.html (never a cached
|
||||
// signed-in page; the app has no offline notes)
|
||||
// - versioned scripts/stylesheets (?v=<asset version>, immutable by construction): cache-first, so a
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@
|
|||
* switch the whole thing off with localStorage['joplock-viewport'] = 'off'.
|
||||
*
|
||||
* Pure helpers are exported for node tests (module.exports), like cm-extras.js.
|
||||
* Why this exists and how it is verified: docs/agents/pwa-mobile.md ("Soft keyboard / viewport").
|
||||
*/
|
||||
(function (root) {
|
||||
'use strict';
|
||||
|
|
|
|||
133
tests/agentDocs.test.js
Normal file
133
tests/agentDocs.test.js
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
'use strict';
|
||||
|
||||
// The agent docs are only useful while they are small and true. AGENTS.md is loaded on every task,
|
||||
// so it has a size budget; the topic files in docs/agents/ are read on demand, so each must be
|
||||
// linked from AGENTS.md; and any file, function or npm script a doc names must still exist (a stale
|
||||
// reference actively misleads an agent, which is worse than no doc).
|
||||
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
const root = path.join(__dirname, '..');
|
||||
const read = rel => fs.readFileSync(path.join(root, rel), 'utf8');
|
||||
const agents = read('AGENTS.md');
|
||||
const topicDir = path.join(root, 'docs', 'agents');
|
||||
const topics = fs.readdirSync(topicDir).filter(f => f.endsWith('.md')).sort();
|
||||
const docs = { 'AGENTS.md': agents, ...Object.fromEntries(topics.map(f => [`docs/agents/${f}`, read(`docs/agents/${f}`)])) };
|
||||
|
||||
const ROOTS = '(?:app|public|tests|playwright-tests|integration-tests|scripts|docs|cm-build|hljs-build|prism-build)';
|
||||
|
||||
const globToRegExp = glob => new RegExp(`^${glob.split('*').map(p => p.replace(/[.+?^${}()|[\]\\]/g, '\\$&')).join('[^/]*')}$`);
|
||||
const exists = rel => {
|
||||
if (!rel.includes('*')) return fs.existsSync(path.join(root, rel));
|
||||
const dir = path.join(root, path.dirname(rel));
|
||||
return fs.existsSync(dir) && fs.readdirSync(dir).some(name => globToRegExp(path.basename(rel)).test(name));
|
||||
};
|
||||
|
||||
test('AGENTS.md is small enough to load on every task', () => {
|
||||
const lines = agents.split('\n').length;
|
||||
assert.ok(lines <= 130, `AGENTS.md is ${lines} lines (budget 130): move detail into docs/agents/`);
|
||||
assert.ok(agents.length <= 14000, `AGENTS.md is ${agents.length} chars (budget 14000)`);
|
||||
});
|
||||
|
||||
test('no topic file grows into the next monolith', () => {
|
||||
for (const f of topics) assert.ok(docs[`docs/agents/${f}`].length <= 24000, `docs/agents/${f} is over 24000 chars: split it`);
|
||||
});
|
||||
|
||||
test('every topic file is linked from AGENTS.md, and every docs/agents link resolves', () => {
|
||||
for (const f of topics) assert.ok(agents.includes(`docs/agents/${f}`), `docs/agents/${f} is not linked from AGENTS.md`);
|
||||
for (const [name, text] of Object.entries(docs)) {
|
||||
for (const [, link] of text.matchAll(/docs\/agents\/([\w-]+\.md)/g)) assert.ok(topics.includes(link), `${name} links to docs/agents/${link}, which does not exist`);
|
||||
}
|
||||
});
|
||||
|
||||
test('every topic file says when to read it', () => {
|
||||
for (const f of topics) {
|
||||
const text = docs[`docs/agents/${f}`];
|
||||
assert.match(text, /^# .+\n\n> Read this before: /, `docs/agents/${f} must start with a title and a "Read this before" line`);
|
||||
}
|
||||
});
|
||||
|
||||
test('every repo path a doc names exists', () => {
|
||||
const missing = [];
|
||||
for (const [name, text] of Object.entries(docs)) {
|
||||
for (const [, token] of text.matchAll(/`([^`\n]+)`/g)) {
|
||||
const m = new RegExp(`^(${ROOTS}/[\\w./@*\\-]+?)(?::\\d+)?$`).exec(token.trim());
|
||||
if (m && !exists(m[1])) missing.push(`${name}: ${token}`);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(missing, []);
|
||||
});
|
||||
|
||||
test('every npm script a doc names exists', () => {
|
||||
const scripts = Object.keys(JSON.parse(read('package.json')).scripts);
|
||||
const missing = [];
|
||||
for (const [name, text] of Object.entries(docs)) {
|
||||
for (const [, script] of text.matchAll(/npm run ([\w:-]+)/g)) if (!scripts.includes(script)) missing.push(`${name}: npm run ${script}`);
|
||||
}
|
||||
assert.deepEqual(missing, []);
|
||||
});
|
||||
|
||||
// Source files to look identifiers up in (not the docs themselves, not vendored bundles)
|
||||
const sourceText = (() => {
|
||||
const out = [];
|
||||
const walk = dir => {
|
||||
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
|
||||
const full = path.join(dir, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
if (['node_modules', 'tinymce', 'plans', 'graft', 'test-results', 'playwright-report', 'data', 'vendor'].includes(entry.name) || entry.name.startsWith('.')) continue;
|
||||
walk(full);
|
||||
} else if (/\.(js|mjs|json|css|html)$/.test(entry.name) && !/\.min\.js$/.test(entry.name) && !['html-docx.js'].includes(entry.name)) {
|
||||
out.push(fs.readFileSync(full, 'utf8'));
|
||||
}
|
||||
}
|
||||
};
|
||||
for (const dir of ['app', 'public', 'tests', 'playwright-tests', 'integration-tests', 'scripts']) walk(path.join(root, dir));
|
||||
out.push(read('server.js'), read('package.json'));
|
||||
return out.join('\n');
|
||||
})();
|
||||
|
||||
// Names that are other people's APIs (TinyMCE, CodeMirror, htmx, the browser, Postgres, Node), not ours.
|
||||
const EXTERNAL = new Set([
|
||||
'fetch', 'require', 'handle', 'alert', 'confirm', 'getContent', 'setContent', 'insertContent', 'execCommand', 'mceCodeSample',
|
||||
'convert_from', 'toString', 'focus', 'blur', 'dispatch', 'setSelection', 'scrollIntoView', 'getSelection', 'getBoundingClientRect',
|
||||
'posAtDOM', 'posAtCoords', 'requestMeasure', 'getNode', 'elementFromPoint', 'scrollTo', 'matchMedia', 'getComputedStyle',
|
||||
'addEventListener', 'setDeviceMetricsOverride', 'setViewportSize', 'setOffline', 'newCDPSession', 'newContext', 'toBeVisible',
|
||||
'dragstart', 'dragenter', 'dragover', 'dragleave', 'getElementById', 'querySelector', 'querySelectorAll', 'getAttribute',
|
||||
'pg_dump', 'pg_restore', 'CREATE', 'renderMarkdown', 'markdown', 'htmlToMarkdown',
|
||||
'flankingWhitespace', // Turndown internal (vendored public/turndown.min.js)
|
||||
]);
|
||||
|
||||
test('every function a doc names still exists somewhere in the source', () => {
|
||||
const missing = [];
|
||||
for (const [name, text] of Object.entries(docs)) {
|
||||
for (const [, token] of text.matchAll(/`([A-Za-z_$][\w$.]*)\([^`\n]*\)`/g)) {
|
||||
const fn = token.split('.').pop();
|
||||
if (EXTERNAL.has(fn) || fn.length < 4) continue;
|
||||
if (!new RegExp(`\\b${fn.replace(/\$/g, '\\$')}\\b`).test(sourceText)) missing.push(`${name}: ${fn}()`);
|
||||
}
|
||||
}
|
||||
assert.deepEqual([...new Set(missing)], []);
|
||||
});
|
||||
|
||||
test('source files that point at a topic doc point at one that exists', () => {
|
||||
const bad = [];
|
||||
const scan = rel => {
|
||||
for (const f of fs.readdirSync(path.join(root, rel))) {
|
||||
if (!/\.js$/.test(f) || /\.min\.js$/.test(f)) continue;
|
||||
const text = read(`${rel}/${f}`);
|
||||
for (const [, link] of text.matchAll(/docs\/agents\/([\w-]+\.md)/g)) if (!topics.includes(link)) bad.push(`${rel}/${f}: ${link}`);
|
||||
}
|
||||
};
|
||||
['app', 'app/items', 'app/routes', 'app/proxy', 'app/templates', 'public'].forEach(scan);
|
||||
assert.deepEqual(bad, []);
|
||||
});
|
||||
|
||||
test('nothing still points at the removed AGENT_GUIDE.md', () => {
|
||||
const hits = [];
|
||||
for (const [name, text] of Object.entries({ ...docs, 'README.md': read('README.md') })) if (text.includes('AGENT_GUIDE')) hits.push(name);
|
||||
for (const rel of ['public/app.js', 'public/service-worker.js', 'server.js']) if (read(rel).includes('AGENT_GUIDE')) hits.push(rel);
|
||||
assert.deepEqual(hits, []);
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue