# Joplock Agent Guide ## Purpose This repo owns Joplock, standalone thin-client sidecar web UI for stock Joplin Server. Use this guide when working in this repository. ## Product Direction - Joplin Server stays unmodified - Joplock stays separate project and separate repo - Reuses existing Joplin Server auth/session/user model through sidecar logic - Keeps compatibility with desktop/mobile/CLI clients on same server and same DB - Browser stays thin and untrusted - Shared-browser safety matters: logout should clear client-visible state/cache as much as platform allows - Installable PWA shell, no offline notes/editing - Uses same Postgres database as Joplin Server, no separate app DB ## Architecture Overview ### Stack - **Server**: Node.js HTTP server, no framework - **Client**: SSR HTML + htmx fragment swaps + shared browser logic in `public/app.js` - **Editor**: CodeMirror 6 for markdown editing, contenteditable rendered markdown preview mode - **Code blocks**: Full-screen code modal with CM6 editor and language picker; highlight.js for preview mode syntax highlighting - **Autosave**: htmx delayed PUT after typing pause (deferred while modals are open) - **Markdown**: server-side `renderMarkdown()`, client-side Turndown `htmlToMarkdown()` - **Auth**: reuses Joplin Server `sessionId` cookie - **DB access**: reads direct from shared Postgres; writes go through stock Joplin Server API ### Runtime Shape - Initial page load is full SSR HTML from `layoutPage()` in `app/templates/pages.js` - After load, most interactions are fragment-driven via htmx - The browser is intentionally thin: most state is DOM state, form state, or small client-only UI state in `public/app.js` - There is no frontend router and no SPA store - Desktop and mobile share the same server routes and most of the same editor code; mobile is a different screen shell around the same editor fragment ### Request Flow 1. Browser hits Joplock 2. Joplock validates `sessionId` against Joplin session/user tables 3. Fragment endpoints return HTML chunks; htmx swaps DOM 4. Writes serialize note/folder/resource and send upstream to stock Joplin Server API ### Main UI Flow 1. `GET /` renders the full shell 2. Navigation / notes / editor content is loaded from fragment endpoints 3. Selecting a folder swaps the notes list or nav tree fragment 4. Selecting a note swaps in `editorFragment()` 5. Autosave sends `PUT /fragments/editor/:id` with the current form state 6. Preview rendering uses `POST /fragments/preview` ### Fragment Conventions - `app/templates/**/*.js` returns raw HTML strings, not JSX/templates/components - htmx targets are mostly `#nav-panel`, `#notelist-panel`, `#editor-panel`, and mobile-specific targets like `#mobile-editor-body` - Out-of-band swaps are used sparingly; note metadata is one example - Client logic often relies on stable IDs, so be careful renaming DOM IDs used by inline JS ## Core Rules 1. Do not modify Joplin Server source for Joplock features unless explicitly approved. 2. Server authoritative. Browser ephemeral. 3. Preserve sync compatibility with normal Joplin clients. 4. Do not build browser-local authoritative storage. 5. Keep sidecar API app-oriented. Do not expose raw sync/storage model to frontend. 6. Treat logout as client cleanup event on shared machines. ## Vault / Encryption Model - Vaults are notebooks/folders with metadata stored in `joplock_vaults` - Titles stay plaintext - Notebook names stay plaintext - Note body ciphertext is stored in normal Joplin note bodies using Joplock markers for compatibility - Browser crypto stays client-side only; server never receives vault passwords - A note inside a vault notebook must be treated as protected even if its stored body is still plaintext during transition states - Locked vault notes render the lock overlay plus hidden editor shells; do not remove the hidden editor DOM because unlock logic depends on it - Clicking a vault lock while unlocked should lock immediately and close the open note if it belongs to that vault - Startup/refresh must never auto-resume an encrypted note or a note inside a vault notebook ## Service Responsibilities ### Stock Joplin Server Owns: - login/session/auth source of truth - sync endpoints - canonical storage rules - existing user/session tables ### Joplock Owns: - thin-client UI - sidecar API endpoints - session validation against shared DB - markdown rendering and editor behavior - resource upload/serving - app-specific settings in `joplock_settings` - PWA shell/assets Does not own: - canonical note/folder/resource persistence rules - sync protocol semantics - auth/session source of truth - offline-first storage ## File Map ### Entry / Server - `server.js` — entry point, env wiring, server startup - `app/createServer.js` — server assembly, shared context, full-page `/` render, static serving ### Route Handlers - `app/routes/fragments.js` — desktop/shared fragment routes - `app/routes/mobile.js` — mobile folder/note/search routes - `app/routes/api.js` — JSON API endpoints - `app/routes/auth.js`, `app/routes/settings.js`, `app/routes/admin.js`, `app/routes/history.js`, `app/routes/resources.js` ### Templates / UI - `app/templates/index.js` — central template re-export - `app/templates/pages.js` — full-page layout/login/MFA shells - `app/templates/fragments.js` — nav, editor, search, history, OOB fragments - `app/templates/mobile.js` — mobile folder/note/search fragments - `app/templates/shared.js` — escaping, markdown rendering, title normalization - `app/templates/settings.js` — settings/admin page sections ### Client Runtime - `public/app.js` — shared client logic for editor, autosave, vault flows, mobile screen stack, search, and modals Important subareas: - `settingsPage()` — Settings UI and simple client save helpers - `editorFragment()` — shared editor DOM used by desktop and mobile - `layoutPage()` — logged-in app shell and mobile shell container - `renderMarkdown()` — server-side markdown-to-HTML for preview/render mode - `public/app.js` mobile helpers — folder-first mobile UI, note list, search, editor screen stack ### Auth - `app/auth/cookies.js` — cookie parsing - `app/auth/sessionService.js` — shared DB session lookup - `app/auth/mfaService.js` — env-driven TOTP verification and otpauth/QR generation ### Data - `app/items/itemService.js` — DB reads for folders, notes, search, resources - `app/items/itemWriteService.js` — note/folder/resource serialization and upstream writes - `app/settingsService.js` — Joplock-owned settings table access - `app/vaultService.js` — vault metadata CRUD in `joplock_vaults` ### How Reads vs Writes Work - Reads come from the shared Postgres DB for speed and to match the current server state - Writes do not write directly to Joplin tables; they go through stock Joplin Server APIs - That split is intentional: Joplock can stay lightweight while preserving compatibility with normal Joplin clients - If behavior looks inconsistent after a write, inspect both the sidecar request path and the upstream Joplin API call path ### Static Assets - `public/htmx.min.js` - `public/codemirror.min.js` — CM6 bundle with 11 language parsers (built from `cm-build/`) - `public/hljs.min.js` — highlight.js bundle for preview mode code highlighting (built from `hljs-build/`) - `public/styles.css` - `public/service-worker.js` - `public/manifest.webmanifest` ### Bundle Build Sources - `cm-build/` — CM6 bundle source (`npm install && npm run build` → `public/codemirror.min.js`) - `hljs-build/` — highlight.js bundle source (`npm install && npm run build` → `public/hljs.min.js`) ### Tests - `tests/*.test.js` - Run: `node --test tests/**/*.test.js` ### Deployment - `Dockerfile` - `docker-compose.yml` — sidecar-only example - `docker-compose.example-full.yml` — Postgres + Joplin Server + Joplock example - `.env.example` ## MFA Notes - MFA is per-user, managed via Settings → Security → Two-Factor Authentication. - Each user's TOTP seed is stored in `joplock_settings.totp_seed` in the shared Postgres DB. - No global/shared TOTP seed. The old `JOPLOCK_TOTP_SEED` / `JOPLOCK_TOTP_ISSUER` env vars are removed. - `IGNORE_ADMIN_MFA=true` skips the per-user MFA check at login for the docker-defined admin account (`JOPLOCK_ADMIN_EMAIL`). Other users are unaffected. - Admin can force-enable/disable MFA for any user via the Admin tab (no code required). ## Design Decisions ### Separate repo Joplock lives outside Joplin monorepo. Keep standalone build, test, docs, Docker flow working without Joplin source tree. ### Shared Postgres database Joplock reads same Postgres database as Joplin Server. No data duplication. Writes still go through Joplin Server API for compatibility and validation. ### Configurable open mode Notes can open in rendered mode or markdown mode based on the per-user `noteOpenMode` setting. Desktop and mobile both respect the same setting. ### Shared editor fragment Desktop and mobile do not have separate editor implementations. Both use the same `editorFragment()` and client editor logic; mobile wraps it in a mobile-specific shell and screen navigation layer. ### PWA shell Cache shell/static assets only. Do not cache note/resource/API responses in ways that break shared-browser safety. ### Mobile-first navigation without SPA rewrite Mobile uses a folders screen, notes screen, and editor screen implemented in SSR + htmx + inline JS. Do not introduce a client router or framework state layer to solve mobile flow problems. ### Tablet behavior Tablet still uses the mobile shell in the current responsive design. Mobile/tablet editor behavior should be reasoned about by editor container context, not just viewport width. ## Editor Model ### Two modes - **Markdown mode**: CodeMirror 6 is visible, textarea is sync target - **Rendered mode**: contenteditable preview is visible, Turndown converts edited HTML back to markdown ### Source of truth during editing - The hidden textarea `#note-body` is the form field used for saves - In markdown mode, CodeMirror changes sync into `#note-body` - In rendered mode, preview DOM changes sync back into markdown via `htmlToMarkdown()` - File/image uploads should alter markdown first, then refresh rendered preview from markdown; do not treat preview-only DOM insertion as authoritative state - The title is mirrored between `.editor-title`, hidden title input, and mobile title header when applicable ### Save lifecycle - `markEdited()` updates UI state to `Edited` - `scheduleSave()` triggers delayed autosave for body/form changes - `scheduleSaveTitle()` is a shorter timer for title changes - If `scheduleSave()` or `scheduleSaveTitle()` sees the same form hash as `_savedHash`, the visible save state should return to `Saved`, not remain `Edited` - `flushSave()` is the forced-save path used before leaving a dirty note; it must also handle vault-note encryption before navigation proceeds - `htmx:afterRequest` on the editor save path transitions UI state back to `Saved` - Offline/request failure paths set status to `Offline` ### Upload behavior - The hidden file input `#file-upload` supports multi-select uploads - `handleFilePicker()` snapshots the selected `FileList` before clearing the input so mobile/desktop pickers do not lose files - `uploadFiles()` should batch multi-file selections and avoid mid-batch autosave races that can reload the editor before later files are applied - Desktop and mobile rendered-mode uploads must preserve selection order across multiple images - Image-only uploads must not promote the image filename into the note title; auto-title should ignore image-only first lines ### Important fragility points - DOM IDs and class names are part of the editor contract with inline JS - Preview HTML must remain convertible back to markdown with acceptable fidelity - Checkbox, code block, and blank-line handling are easy to regress - The code modal is outside the fragment-swapped editor so it survives swaps ## Mobile UI Model ### Shell structure - `#mobile-folders-screen` - `#mobile-notes-screen` - `#mobile-editor-screen` These screens are shown/hidden by inline JS in `layoutPage()` using class changes, not route changes. ### Mobile navigation behavior - Folder-first flow: folders -> notes -> editor - Search has its own mobile header state - Mobile note creation uses dedicated fragment endpoints and server headers to drive the next UI step - The floating action button is only a mobile affordance; desktop should stay unaffected - FAB visibility should follow screen state directly (`folders` / `notes` visible, `editor` hidden), not only htmx swap side effects - Mobile folder rows can include a vault lock button and it must stay inline with the row actions ### Mobile editor behavior - Mobile hides the desktop title bar and uses the mobile header instead - Mobile header mirrors note title and save state - Mode buttons should remain visible and clearly indicate the active mode - Toolbar visibility should be keyed to being inside the mobile editor container, not only screen width - Newly-created empty mobile notes may be discarded on back if still blank/untitled - Locked mobile notes should not reveal plaintext/editor surfaces until unlock ### Tablet expectations - Tablet is still in the mobile shell range - Existing note open path and new note open path should behave the same with respect to default open mode, toolbar visibility, and title/save-state UI - When debugging tablet issues, compare the exact htmx target and after-settle path used by new-note vs existing-note opens ## Settings Model ### Storage - Settings are stored per-user in `joplock_settings.settings` as JSONB - `app/settingsService.js` owns defaults and normalization - Unknown or invalid values should normalize back to safe defaults ### Current notable settings - `theme` - `noteFontSize` - `mobileNoteFontSize` - `codeFontSize` - `noteMonospace` - `noteOpenMode` - `resumeLastNote` - `dateFormat` - `datetimeFormat` - `liveSearch` - `confirmTrash` - `autoLogout` - `autoLogoutMinutes` - `encryptionAutoLockMinutes` ### Adding a new setting 1. Add default + normalization in `app/settingsService.js` 2. Allow the key in `/api/web/settings` in `app/createServer.js` 3. Add the UI in `settingsPage()` in `app/templates/settings.js` 4. If needed, inject the normalized setting into `layoutPage()` / `public/app.js` 5. Rebuild with `./scripts/rebuild-dev.sh` ## Route Notes Useful route groups in `app/createServer.js`: - auth pages and login/logout - full page render for `/` - fragment routes for nav, notes, editor, preview - mobile fragment routes for folders, notes, search, mobile note creation - resource upload and resource serving - settings save endpoints - history endpoints If a UI action appears broken, check: 1. Which endpoint it hits 2. Which htmx target it swaps 3. Which client event handler expects to run after swap/request 4. Whether the response includes headers or OOB fragments the client depends on ## Coding Guidance - Keep changes minimal - Preserve sidecar/frontend boundary - `public/app.js` is DOM-contract fragile; validate escaping-heavy changes and stable IDs carefully - The code modal lives in `loggedInLayout`, not inside `navigationFragment` or `editorFragment`, so it survives htmx OOB swaps - Be careful with checkbox text handling, `\n`, regex escaping, and DOM-to-markdown round trips - Keep standalone repo paths/docs/scripts correct; avoid reintroducing monorepo assumptions - Prefer changing existing inline helpers over introducing a new abstraction unless there is clear reuse - When fixing mobile behavior, verify desktop is unchanged - When fixing desktop editor behavior, verify mobile still works because both use the same editor fragment - Be cautious with `htmx:afterRequest` assumptions; in htmx 2.x, response headers are often more reliable than old event-property assumptions - If changing vault behavior, verify desktop + mobile, locked + unlocked, existing note + newly-created note, and refresh/restart behavior ## Debugging Guidance ### If a code change does not appear in the app - Rebuild with `./scripts/rebuild-dev.sh` - Do not rely on `docker compose ... restart joplock` after source edits - If still stale, inspect the built container logs and confirm the right compose stack is running ### If mobile note creation/opening misbehaves - Check whether the server response includes the expected mobile header such as `X-Mobile-Note-Id` - Check the `htmx:afterRequest` handler that consumes that header - Compare new-note path vs existing-note path - Check whether the note is in a vault and whether the editor was initialized in locked vs unlocked state ### If vault behavior misbehaves - Check whether the folder is marked with `isVault` - Check whether the note is marked with `inVault` / `isEncrypted` / `vaultId` - Check `toggleVaultLock()`, `unlockNote()`, `_completeUnlock()`, and `flushSave()` in `public/app.js` - Check whether the hidden editor shells still exist in locked editor HTML ### If startup/resume behavior is wrong - Check the `/` render path in `app/createServer.js` - Check `resumeLastNote`, `lastNoteId`, and `lastNoteFolderId` - Refresh/restart must not reopen encrypted notes or notes inside vault notebooks ### If toolbar/mode behavior is inconsistent - Verify whether the current editor is actually inside `#mobile-editor-body` - Check `syncEditorModeButtons()` and `setEditorMode()` - Check whether the note was initialized with the expected `noteOpenMode` - If switching modes marks the note `Edited`, confirm the current form hash differs from `_savedHash`; unchanged hashes should show `Saved` ### If title UI drifts - Check `.editor-title` - Check hidden input `.editor-title-hidden` - Check `#mobile-editor-title` - Check `autoTitle()` and `syncTitle()` ### If save-state UI drifts - Check `setSaveState()` - Check `#autosave-status` - Check `#mobile-editor-status` - Check htmx save success/failure handlers and upload progress handlers ## Verification - Run tests: `npm test` - Build image: `npm run docker:build` - Sidecar-only compose: `npm run docker:up` - Full example compose: `npm run docker:up:full` ## Development Stack Use the dev compose stack for all development work. It includes Postgres, Joplin Server, and Joplock together. - Rebuild Joplock app container after code changes: `./scripts/rebuild-dev.sh` - Start / restart full dev stack: `docker compose -f docker-compose.dev.yml up -d --build` - Stop dev stack: `docker compose -f docker-compose.dev.yml down` Do not use the sidecar-only `docker-compose.yml` for development. Important: - `docker compose ... restart joplock` is not enough after source edits because the Docker image copies `app/`, `public/`, and `server.js` at build time. - For app code changes, use `./scripts/rebuild-dev.sh` from now on. Recommended inner loop: 1. Edit source 2. Rebuild with `./scripts/rebuild-dev.sh` 3. Refresh the app 4. Check `docker compose -f docker-compose.dev.yml logs --tail=... joplock` if something looks wrong ## Reference Material - Mobile UX reference: `~/dev/joplin/packages/app-mobile/` - Use it for interaction ideas and behavior parity targets, not as a copy-paste implementation source - Joplock must still fit the SSR + htmx sidecar architecture ## Current Baseline - standalone repo at `abort-retry-ignore/joplock` - tests passing in standalone repo - Docker build passing in standalone repo - full example compose verified with alternate free host ports - CI: GitHub Actions builds and pushes image to `ghcr.io` on every push to `master` ## Recently Completed Work - **Lazy nav loading**: folder note lists load on first expand, not on page load - **Search pagination**: `pg_trgm` GIN index, paginated search results with Load More - **Mobile pagination**: paginated note lists on mobile - **Note flash fix**: eliminated redundant `/fragments/preview` fetch on note load - **Search input fix**: value captured at `htmx:beforeSwap` so characters typed during in-flight request are not lost - **Mobile spinner**: inline spinner in editor screen body instead of broken fixed overlay - **Tablet-on-phone fix**: CSS/JS breakpoint raised from 481px to 600px - **Gzip compression**: all HTML responses compressed via Node `zlib` when client sends `Accept-Encoding: gzip` - **hx-* sanitization**: `renderMarkdown()` strips `hx-*` attributes from user HTML to prevent htmx injection - **All Notes fix**: `/fragments/folder-notes` now normalizes `__all_notes__` → `__all__` so the virtual folder loads correctly - **Service worker cache bump**: `v12` forces PWA to fetch fresh CSS/JS after update - **Checkbox styling**: checked items show accent-colored bold icon via `.md-cb-icon` span; icon is styled independently from text using flexbox layout; turndown serializer, click-toggle handler, and new-checkbox inserter all updated to match - **Multi-image uploads**: picker uploads now support multiple files, update markdown as the source of truth, preserve upload order in rendered mode, and refresh preview from markdown after each batch ## Key Conventions - `plans/` is gitignored — do not commit plan files - Do not push to remote unless the user explicitly asks - Run `npm test` before every commit