vrr/sillytavern-character-memory
2
0
Fork 0
mirror of https://github.com/bal-spec/sillytavern-character-memory.git synced 2026-04-26 10:50:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

v2.1.6 — UX redesign, injection viewer, unified editor, token breakdown

Browse source 
Complete rewrite of the UI and significant feature additions since v1.6.1.

UX Redesign (v2.0):
- Single-view dashboard replaces 4-tab sidebar
- Settings, Prompts, Troubleshooter, Memory Manager moved to center-screen modals
- Activity log in slide-out drawer
- Setup Wizard for first-run configuration
- Prompt version tracking with update notifications
- Health indicator in stats bar

Injection Viewer (v1.6–v2.1.6):
- Per-message injection data: see exactly what memories, lorebook entries,
  and extension prompts were injected for any generation
- Context/Prompt Breakdown with per-category token counts (System, Char card,
  Lorebook, Data Bank, Examples, Chat history) via ST Prompt Itemization
- Stacked bar visualization, token hints in headers, Tips popup
- Context overflow and heavy injection warnings

Memory Management:
- Unified block editor across all 5 editing surfaces (Memory Manager,
  Consolidation, Conversion, Reformat, Data Bank browser)
- Find & Replace with highlighting across all editors
- Undo support for all edit operations
- Group chat character picker in Memory Manager

Other features:
- Tablet & phone display modes with touch-friendly controls
- Topic-tagged memory format for better vector retrieval
- Self-closing memory tag handling (GLM-4.7 compatibility)
- Protect recent messages from extraction feedback loop
- 9-point health check system with retrieve chunks and score threshold
- Shared editor factory (editor.js), pure utility library (lib.js)
- Vitest test suite: unit, snapshot, and live LLM tests
- Full documentation suite in docs/

See CHANGELOG.md for detailed per-version notes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
bal-spec 2026-03-07 15:20:14 -08:00
parent 3e846a1bef
commit dc0eab2638
81 changed files with 15778 additions and 2866 deletions
Download patch file Download diff file Expand all files Collapse all files

9
.gitignore vendored Normal file
View file

@ -0,0 +1,9 @@
.envrc
node_modules/
.DS_Store
*.png
!images/*.png
.playwright-mcp/
package-lock.json
docs/reddit-*.md
docs/topic-tag-charname-fix.md

163
CHANGELOG.md
View file

@ -1,10 +1,163 @@
# Changelog
## 1.6.2
## 2.1.6
### Improvements
- **Clearer dialog buttons**: Consolidation and conversion dialogs now show "Save" and "Cancel" buttons instead of the ambiguous "Yes"/"No" defaults, preventing accidental loss of edits.
- **Prompt Breakdown at top of injection viewer**: The injection viewer now opens with a "Prompt Breakdown" section at the top that loads automatically — no need to find or click a collapsed panel. Shows exact per-category token counts (System, Char card, Lorebook, Data Bank, Examples, Chat history) via ST's Prompt Itemization when available, or estimated injection-only numbers for snapshots from previous sessions.
- **Accurate Lorebook token estimates**: Previously token estimates were computed by summing truncated entry content (200 chars/entry from `WORLD_INFO_ACTIVATED`), causing large overestimates on multi-entry lorebooks. Estimates now use the actual injected `worldInfoString` and `dataBankVectorsString` from ST's prompt data, matching the Prompt Breakdown numbers.
- **Data Bank label consistency**: CharMemory's injected memories are now labelled "Data Bank" throughout — section header, breakdown rows, and tips popup — matching SillyTavern's own terminology.
- **Expanded "Optimize" tips popup**: The tips popup now covers all breakdown categories. A new "Char Card / System Prompt" section advises shortening the description, moving lore to the Lorebook, and trimming the system prompt. The intro text is updated to reflect that the breakdown now shows the full prompt, not injections only.
## 2.1.5
### Improvements
- **Unified Context section in injection viewer**: The previous "Context Budget" and "Prompt Breakdown" sections are now a single "Context" section at the top of each injection snapshot. The header shows a compact stacked bar and summary (e.g. `~1,200 / 32,768 tk (4%)`). Expanding loads the full token breakdown — exact per-category counts via ST's Prompt Itemization when available, or injection-only estimates for snapshots from previous sessions.
- **Accurate Lorebook token estimates**: The "Context Budget" section previously summed truncated entry content (200 chars/entry) from `WORLD_INFO_ACTIVATED`, causing large overestimates for lorebooks with many entries. Token estimates now use the actual injected `worldInfoString` from ST's prompt data, giving accurate numbers that match the Prompt Breakdown view.
- **Data Bank label consistency**: The injection viewer now labels CharMemory's injected memories as "Data Bank" throughout (section header, breakdown rows, tips popup), matching SillyTavern's own terminology.
- **Estimated fallback breakdown**: When exact Prompt Itemization data is unavailable (previous session or feature disabled), the Context section body shows an estimated injection-only breakdown using snapshot char counts, with a clear note that the numbers are approximate.
- **Tips to reduce link in full breakdown**: The "Tips to reduce" link (and tips popup) now appears in all breakdown views — both the exact Prompt Itemization breakdown and the estimated fallback.
## 2.1.4
### Improvements
- **Token budget panel in injection viewer**: The per-message injection viewer now shows a "Context Budget" section at the top of each snapshot. A compact horizontal stacked bar (collapsed by default) displays estimated token usage across three tracked sources — CharMemory (purple), Lorebook (amber), and other extension prompts (teal) — relative to the model's configured context limit. Expanding the section shows a per-source breakdown table. Token estimates (~4 chars/token) are labeled as approximate; char card, system prompt, and chat history are noted as not counted.
- **Context overflow and heavy injection warnings**: If tracked injections alone exceed the model context, a red health note flags the overflow. If they exceed 40% of context (leaving little room for char card and chat history), a yellow advisory note appears.
- **Token hints in section headers**: The CharMemory, Lorebook, and Extension Prompts section headers each show a faint `~N tk` estimate so token cost is visible without expanding the budget panel.
- **Token and position metadata in cards**: Each Extension Prompt card now shows its estimated token cost and injection position/depth (e.g. `~340 tk · in-chat @ depth 2`). Lorebook entry cards show estimated token cost. The snapshot now also captures injection `depth` for extension prompts, which was previously missing.
- **"Tips to reduce" popup**: A "Tips to reduce" link in the budget breakdown opens a popup with actionable guidance organized by source — Consolidate and Retrieve chunks for CharMemory, token budget and keyword specificity for Lorebook, per-extension settings for other extensions, and context window / response token tuning for overall budget.
## 2.1.3
### Improvements
- **Generation mode in group diagnostics**: The Group Debug section of the Diagnostic Report now shows the group's generation mode value. This is the key piece needed to diagnose issue #4 — groups using Append (with disabled) mode have all members in the disabled list by design, which was invisible in prior reports.
- **Clearer warning when all group members are disabled**: Instead of the generic "no targets found" message, the Activity Log now says "all group members are disabled in SillyTavern — re-enable at least one in the group settings" when a group has members but all are in the disabled list.
- **Clearer dialog buttons**: Consolidation, conversion, reformat, and Data Bank editor dialogs now show "Save" and "Cancel" buttons instead of ambiguous defaults, preventing accidental loss of edits.
- **Unified Memory Manager editor**: The View/Edit memories dialog now uses the same block editor as Consolidation, Conversion, and the Data Bank browser. Inline editing, undo, add/delete blocks and bullets, and find/replace — all consistent across every editing surface. In group chats, a character picker selects which member to edit.
### Bug Fixes
- **Consolidation dialog not appearing with some LLMs**: Models like GLM-4.7 produce self-closing `<memory chat="..."></memory>` tags with bullets placed after the closing tag instead of inside it. The parser now handles this pattern, recovering per-block structure instead of silently returning an empty result. The same fix is applied to the extraction pipeline.
- **Prompt update notifications silent after upgrading from 1.x**: Users who customized an extraction or consolidation prompt in 1.x and then upgraded to 2.x were never notified that the default prompts had changed. The version-tracking system introduced in 2.0 only ran on the second launch (after `promptVersions` was already initialized), so the first launch after upgrade silently marked all prompts as up-to-date — even customized ones. The check now detects a customized prompt with no prior version record and flags it for review. A toast notification also appears 2 seconds after load whenever any prompt has a pending update, directing users to Settings → Prompts.
## 2.1.2
### Bug Fixes
- **Extraction restarting from the beginning every session**: After running consolidation, `lastExtractedIndex` was reset to -1 on every SillyTavern restart. Consolidation replaces the original chat-ID labels on memory blocks with thematic labels (e.g. "First vet visit"), and the stale-metadata check incorrectly treated those as "no memories found for this chat." The check now resets only when the memory file is genuinely empty.
- **Group chat extraction discarded mid-flight**: If any group member sent a message during an LLM call, the extraction result was silently thrown away and the same messages re-extracted after cooldown — doubling API cost. The context-change guard now only checks for a chat switch in group chats; `characterId` flipping between members is expected and no longer triggers a discard. The discard event is also now visible in the Activity Log instead of being hidden in the browser console.
### Improvements
- **Group chat diagnostics in Troubleshooter**: The Diagnostic Report now includes a Group Debug section when in a group chat, showing member counts, how many resolved vs. unresolved, and any avatar strings that couldn't be matched to a loaded character. Helps diagnose issue #4 where group members aren't detected.
## 2.1.1
### New Features
- **Protect recent messages**: New toggle in Settings > Extraction that excludes the most recent messages from auto-extraction, preventing a feedback loop where just-extracted memories constrain swipes and regenerations. When enabled, a configurable buffer (default: 4 messages) keeps recent events out of memories until the next extraction cycle. Does not affect Extract Now or Extract Here.
### Bug Fixes
- **KoboldCPP vectorization check false negative**: The health check reported files as "not vectorized" when using KoboldCPP as the vectorization backend. KoboldCPP doesn't store its model name in Vector Storage settings — it's discovered dynamically from the API. The check now queries the KoboldCPP embed endpoint to discover the correct model name before looking up vector data.
## 2.1.0
### New Features
- **Tablet / Touch Mode**: On touch devices (iPad, tablets), the dashboard now opens as a centered floating panel instead of expanding in the narrow sidebar. All buttons are enlarged to 44px touch targets per Apple HIG. The panel is non-modal — tap outside or swipe down to dismiss, and tools like Settings and Troubleshooter open on top.
- **Display Mode setting**: The old Tablet/Touch Mode toggle (Auto/On/Off) is replaced with a unified Display Mode selector in Settings > Advanced. Four options: Auto (detect your device), Desktop (sidebar), Tablet (floating panel), Phone (panel + wider drawers). Forced overrides work regardless of actual viewport — useful when auto-detection gets it wrong.
- **Phone layout**: On narrow viewports (≤600px), the injection viewer and activity log drawers are widened to 85vw and raised above the SillyTavern sidebar (z-index 4000). Auto-detected in Auto mode, or force via the Phone display mode.
### Bug Fixes
- **Tablet panel off-screen on mobile**: SillyTavern sets `perspective` on `<html>`, which changes the containing block for `position: fixed` elements. The panel's `top: 50%` resolved to 0px on mobile. Fixed by using viewport units (`50vh`/`50vw`) instead of percentages.
- **Tablet panel hidden behind sidebar**: The panel's z-index (1002) was below SillyTavern's extensions drawer (3005). Raised to 5000.
- **Injection viewer and log drawer hidden on mobile**: Both drawers had z-index below the sidebar and were too narrow at phone widths (40vw = 157px on a 393px screen). Phone mode overrides fix both issues.
- **Nudge banner "Fix now" → "View"**: The warning banner button now says "View" since it opens the Troubleshooter for inspection — it doesn't auto-fix anything.
## 2.0.1
### Improvements
- **Topic tags now include character name**: All extraction, consolidation, and conversion prompts produce topic tags starting with the character's name (e.g., `[Flux, Alex — adoption day]` instead of `[Alex — adoption day]`). This improves vector embedding discrimination for retrieval. Prompt versions bumped to `2.0.1` — users with customized prompts will see an update notification.
- **Cross-chunk deduplication**: When extraction spans multiple chunks, each chunk's output is now fed forward as existing memories for subsequent chunks. This prevents duplicate memory blocks when a scene spans a chunk boundary.
- **Sanitized prompt examples**: DO NOT EXTRACT examples in extraction prompts now use neutral, non-explicit examples.
### Bug Fixes
- **False alarm: file vectorization on chat open**: The health check no longer shows a red error when opening an existing chat whose memory file hasn't been re-indexed yet by Vector Storage. Downgraded to a yellow note explaining it resolves automatically on the next message.
- **False alarm: "no memories injected"**: When no memories match the current conversation topic (score threshold filtering working correctly), the health check and injection viewer now show a yellow info note instead of a red error. Zero matches is a normal state, not a failure.
- **Consolidation {{charName}} substitution**: The consolidation pipeline now correctly substitutes the character's name into topic tag instructions. Previously, `{{charName}}` in consolidation prompts was not replaced because `buildConsolidationPrompt()` only called `substituteParamsExtended()` (which handles `{{char}}` but not `{{charName}}`).
## 2.0.0
### New Features
- **Find & Replace across all editing surfaces**: A compact find/replace bar is available in the Memory Manager, Consolidation, Conversion Preview, Reformat Preview, and Data Bank editor. Type to see matches highlighted with a live count. Replace All updates every occurrence at once. Supports case-sensitive matching. In block editor dialogs (Consolidation, Reformat, Conversion, Data Bank editor), Replace All is undoable.
- **Undo in Data Bank editor**: The Data Bank file editor now has an Undo button that reverts Replace All, delete, and add operations. Nothing is written to disk until you click Save.
### UX Redesign
Complete UI overhaul replacing the 4-tab sidebar (Main, Tools, Settings, Log) with a streamlined dashboard + modal architecture.
- **Single-view dashboard**: Sidebar is now a compact dashboard with stats bar, file info, extraction toggle, Extract Now button, tool launchers (Consolidate, Batch, Format), mini activity log, and diagnostics summary
- **Settings modal**: All settings moved to a center-screen modal with left-nav sections (Connection, Extraction, Storage, Advanced). Opens via gear icon in sidebar header
- **Prompts modal**: Dedicated full-width editor for extraction and consolidation prompts with prompt version tracking and update notifications when defaults change
- **Log drawer**: Activity log moved to a slide-out right-side drawer with verbose toggle, export, and swipe-to-close on touch devices
- **Troubleshooter modal**: Replaces the old Diagnostics panel. Includes automated health checks, Data Bank file browser with export/delete/convert actions, diagnostic report, and reset/clear tools. Opens via wrench icon in sidebar header
- **Setup wizard**: First-run 3-step flow (LLM Connection, Vector Storage, Ready) that guides new users through initial configuration. Detects unconfigured state and shows a nudge banner
- **Prompt version tracking**: When default prompts are updated between versions, users with customized prompts see an update banner with options to view changes, adopt the new default, or dismiss
- **Health indicator in stats bar**: Traffic-light dot showing injection health status — click to open Troubleshooter
### Developer
- Removed dead CSS for old tab/pill layout (`.charMemory_tabs`, `.charMemory_toolPills`, etc.)
- Removed dead JS handlers binding to old sidebar element IDs (`#charMemory_consolidate`, `#charMemory_undoConsolidate`, `#charMemory_verboseLog`, etc.)
- Removed unused `updateDiagnosticsDisplay()` function (~180 lines)
- Cleaned up `loadSettings()` — removed jQuery no-ops targeting elements that no longer exist in the sidebar HTML
- Settings modal uses `cm_modal_*` prefixed IDs to avoid conflicts with sidebar dashboard elements
## 1.8.0
### Developer
No user-facing changes. Internal refactoring to make the codebase maintainable ahead of the v2.0 UX redesign.
- **ES module imports from lib.js**: `index.js` now imports pure functions from `lib.js` instead of maintaining duplicate copies. `lib.js` is the single source of truth for parsing, escaping, formatting, and migration functions. The sync-check test that verified duplication consistency is removed.
- **Shared memory editor factory**: New `editor.js` module with `createMemoryEditor()` — encapsulates block state management, undo, and edit-mode tracking. Replaces ~300 lines of duplicated state management across the Consolidation, Conversion, and Reformat dialog editors.
- **Split setupListeners()**: The 527-line monolithic event-wiring function is now a 7-line coordinator calling `setupConnectionControls()`, `setupExtractionControls()`, `setupToolControls()`, `setupStorageControls()`, and `setupLogControls()`.
- **Utility extractions**: `getTimestamp()` and `cloneMemoryBlocks()` replace 9 inline timestamp constructions and 3 inline block-clone patterns.
- **Net result**: `index.js` reduced from ~6,086 to ~5,709 lines. Test count increased from 100 to 117.
## 1.7.0
### New Features
- **Retrieval-optimized memory format**: Extraction prompts now produce topic-tagged memory blocks with a `[Names — description]` tag as the first bullet. This improves vector search discrimination, allowing Vector Storage to retrieve only the most relevant memories instead of thematically similar ones.
- **Unified Convert tool**: The Convert and Reformat tools are merged into a single Convert tool with a source picker. Select "Current memories" to reformat existing memories to the new topic-tagged format, or "Data Bank file" to import from any file. The conversion prompt is always visible for easy iteration.
- **Recommended VS settings**: Diagnostics panel now includes a "Recommended Vector Storage Settings" card with optimal chunk size, retrieve chunks, score threshold, and other settings for CharMemory.
### Improvements
- **Tighter memory blocks**: Default bullet limit reduced from 8 to 5 per block (not counting the topic tag). Forces outcome-focused extraction rather than step-by-step narration.
- **Better consolidation labels**: Consolidation now uses encounter-specific labels (e.g., "Adoption day at the apartment") instead of broad categories (e.g., "Key Events"). Topic tags are preserved and updated during consolidation.
- **Improved health checks**: Chunk size recommendations now include specific guidance (800-1000 chars). New checks for retrieve chunks and score threshold.
- **Named participants**: Extraction and consolidation prompts now require specific names instead of generic labels like "a client" or "someone".
- **Multi-tag block splitting**: When the LLM produces a memory block with multiple topic tags, it is automatically split into separate blocks — one per topic. This ensures clean 1:1 mapping between topic tags and vector chunks.
### Bug Fixes
- **Fix parsing of attributed memory tags**: The LLM sometimes copies `<memory chat="..." date="...">` attributes from existing memories into its response. The extraction regex now handles both bare and attributed `<memory>` tags instead of silently falling through to a single-entry fallback.
- **Hide redundant mini-log on Log tab**: The always-visible activity log mini-bar is now hidden when the full Log tab is active, avoiding duplicate display.
### Migration
- Existing memories continue to work without changes. Use the **Convert** tool (source: "Current memories") to add topic tags to existing memory files for improved retrieval.
- Users with customized extraction prompts are unaffected — only the default prompt is updated. Click "Restore Default" to opt into the new format.
## 1.6.1
@ -22,6 +175,12 @@
- The standalone **Ollama** preset has been merged into **Local Server**. Existing users with Ollama selected are automatically migrated — your model, system prompt, and URL (`http://localhost:11434/v1`) are preserved.
### Developer
- **Automated test suite**: Three-tier Vitest setup — 90 unit tests for `lib.js` pure functions, 6 snapshot integration tests against a 1000-message chat fixture, and 3 live LLM integration tests that validate extraction output structure against a real OpenAI-compatible endpoint.
- **Extraction pipeline refactor**: Extracted `stripNonDiegetic()`, `formatChatMessages()`, and `substitutePromptTemplate()` from inline code in `index.js` into testable pure functions in `lib.js`. No behavior change — `index.js` uses local copies.
- **Live test configuration**: `TEST_LLM_URL`, `TEST_LLM_MODEL`, and `TEST_LLM_KEY` env vars support both local servers (LM Studio, Ollama) and authenticated remote APIs (OpenRouter, NanoGPT).
## 1.6.0
### New Features

70
CLAUDE.md
View file

@ -2,13 +2,7 @@
## Git / GitHub
This repo belongs to the `bal-spec` GitHub account. Before any `git push` or `gh` commands, run:
```
gh auth switch --user bal-spec
```
Do this at the start of every session — the active account resets to `dsayed` between sessions.
This repo belongs to the `bal-spec` GitHub account. The `.envrc` file (via direnv) automatically sets `GH_TOKEN` for `bal-spec` when working in this directory — no manual `gh auth switch` needed.
## What This Is
@ -17,16 +11,19 @@ A SillyTavern extension that automatically extracts structured character memorie
## File Structure
```
index.js — All extension logic: extraction, consolidation, provider API calls, UI controllers, event handlers (~3500 lines)
settings.html — Extension panel UI (settings, memory manager, diagnostics, batch extract)
style.css — All styling
index.js — All extension logic: extraction, consolidation, provider API calls, UI controllers, event handlers, modals (~8660 lines)
lib.js — Pure utility functions imported by index.js at runtime and used by tests (parsing, serialization, formatting, stripping)
editor.js — Shared memory block editor factory (createMemoryEditor) with state management and undo
settings.html — Sidebar dashboard HTML (stats bar, extraction controls, tool launchers, activity, diagnostics)
style.css — All styling (dashboard, modals, drawers, wizard, troubleshooter)
manifest.json — ST extension manifest (version, loading order, author)
README.md — User-facing documentation (getting started guide + technical reference, combined)
CHANGELOG.md — Version history
images/ — Screenshots for documentation
test/ — Vitest test suites (unit, integration/snapshot, integration/live)
```
`index.js` is a single-file architecture. All logic lives there — there are no separate modules.
`index.js` is the main runtime module. `lib.js` is the canonical source for pure utility functions — `index.js` imports them via ES modules. Only `serializeMemories()` is kept local in `index.js` because it uses `getFormatOptions()` for runtime settings.
## Key Architecture
@ -63,6 +60,17 @@ Per-provider settings (API key, model, system prompt, custom URL) are stored in
</memory>
```
### UI Layout (v2.0)
The sidebar (`settings.html`) is a single-view dashboard — no tabs. All complex UI is in center-screen modals and drawers built dynamically in `index.js`:
- **Dashboard** (sidebar): Stats bar, file info, extraction toggle, Extract Now, tool launcher buttons (Consolidate, Batch, Format), mini activity log, diagnostics summary
- **Settings Modal** (`showSettingsModal()`): Left-nav with sections — Connection, Extraction, Storage, Advanced. Uses `cm_modal_*` prefixed IDs to avoid conflicts with sidebar elements.
- **Prompts Modal** (`showPromptsModal()`): Full-width editor for extraction/consolidation prompts with version tracking and update banners
- **Log Drawer** (`toggleLogDrawer()`): Slide-out right-side drawer for the full activity log with verbose toggle and export
- **Troubleshooter Modal** (`showTroubleshooter()`): Health checks, Data Bank file browser, diagnostic report, reset/clear tools
- **Setup Wizard** (`showSetupWizard()`): 3-step first-run flow — LLM Connection, Vector Storage, Ready
### Settings Storage
All settings live under `extension_settings.charMemory`. Key fields:
@ -72,6 +80,7 @@ All settings live under `extension_settings.charMemory`. Key fields:
- `extractionPrompt` — customizable prompt template
- `interval`, `cooldownMinutes`, `chunkSize`, `responseLength` — extraction tuning
- `perChat`, `fileName` — storage options
- `promptVersions` — tracks which prompt versions the user has seen (for update notifications)
## Conventions
@ -93,15 +102,44 @@ All settings live under `extension_settings.charMemory`. Key fields:
## Testing
There are no automated tests. Testing is manual:
### Automated Tests
Vitest is the test framework. Three tiers:
```bash
npm test # Unit tests (97 tests, ~200ms) — pure functions in lib.js
npm run test:snapshot # Snapshot tests (6 tests) — extraction pipeline against 1000-message fixture
npm run test:live # Live LLM tests (3 tests) — requires a running OpenAI-compatible server
```
**Unit tests** cover parsing, serialization, escaping, format detection, and the three extracted pipeline functions (`stripNonDiegetic`, `formatChatMessages`, `substitutePromptTemplate`).
**Snapshot tests** process real chat data from `test/fixtures/flux-chat.jsonl` through the pipeline and snapshot the output. Update snapshots with `npm run test:snapshot -- --update` after intentional changes.
**Live LLM tests** send extraction prompts to a real LLM and validate the response structure. Configured via env vars:
- `TEST_LLM_URL` — endpoint (default: `http://127.0.0.1:1234/v1`)
- `TEST_LLM_MODEL` — model name (default: auto-discover first available)
- `TEST_LLM_KEY` — API key for authenticated endpoints like OpenRouter (default: none)
Recommended local model: Gemma 2 9B or Qwen 2.5 7B. Avoid thinking models (Qwen3) — their `<think>` tags waste token budget.
### `lib.js` as Single Source of Truth
`lib.js` is the canonical source for pure utility functions. `index.js` imports them via ES modules (`import { ... } from './lib.js'`). When modifying these functions, edit `lib.js` only — `index.js` picks up changes automatically. Exception: `serializeMemories()` in `index.js` is a separate implementation that uses `getFormatOptions()` for runtime format settings.
### Manual Testing
For UI and integration testing that requires SillyTavern:
1. Install in SillyTavern's `public/scripts/extensions/third-party/CharMemory` (symlink or clone)
2. Restart SillyTavern
3. Test extraction with different providers
4. Check Activity Log (verbose mode) for LLM prompts/responses
5. Use Diagnostics tab to verify injected memories
4. Check Activity Log (verbose mode via Log Drawer) for LLM prompts/responses
5. Open Troubleshooter to verify health checks and injected memories
## Common Tasks
- **Adding a new provider**: Add entry to `PROVIDER_PRESETS`, no other changes needed if it's OpenAI-compatible with standard `/models` endpoint
- **Modifying the extraction prompt**: Edit `defaultExtractionPrompt` constant. Users can also customize via the UI, so changes to the default only affect new installations or users who click "Restore Default"
- **Adding UI elements**: Add HTML to `settings.html`, add event handler in `setupListeners()`, add controller logic. Follow the `charMemory_` ID prefix convention
- **Modifying the extraction prompt**: Edit `defaultExtractionPrompt` constant. Bump `PROMPT_VERSIONS.extraction` to trigger update notifications for existing users. Users can also customize via the Prompts modal, so changes to the default only affect new installations or users who click "Restore Default"
- **Adding dashboard UI elements**: Add HTML to `settings.html`, add event handler in the appropriate `setup*Controls()` function. Follow the `charMemory_` ID prefix convention
- **Adding modal UI elements**: Build the HTML dynamically in the modal's show function (e.g., `showSettingsModal()`). Use `cm_modal_*` or `cm_ts_*` prefixed IDs to avoid conflicts with sidebar elements

913
README.md
View file

@ -1,894 +1,89 @@
# CharMemory — SillyTavern Extension
This extension automatically extracts structured character memories from chat and stores them in the character's Data Bank. Memories are vectorized by SillyTavern's Vector Storage so the most relevant ones are retrieved at generation time — your character remembers things from old conversations.
CharMemory automatically extracts structured memories from your chats and stores them in the character's Data Bank. SillyTavern's built-in Vector Storage retrieves the most relevant ones at generation time — your character remembers what happened.
## Is CharMemory for you?
>**Upgrading from 1.x?**
This release contains extensive under the hood code changes, alters the extraction prompts and the structure of the extracted memories. If you used previous versions, you should back up your current files before updating and you'll want to compare your edited prompts with the new ones.
**CharMemory is built for setups where character cards define who a character is, and memories capture what happens to them over time.**
![CharMemory panel open in the SillyTavern sidebar](images/panel-full.png)
## Is this for you?
**CharMemory is built for setups where character cards define who a character is, and Data Bank character memories capture what happens to them over time.**
This is for you if:
- You use **character cards** for your characters
- You chat 1:1 or in **group chats** and want characters to remember things across sessions
- You want memories stored as plain, editable files — not locked in a database
- You use **character cards** and want characters to remember things across sessions
- You chat **1:1 or in group chats**
- You want memories stored as **plain, editable files** — not locked in a database
This probably isn't for you if:
- Your memory workflow is lorebook-based (triggered entries in World Info)
- Your memory workflow is lorebook-based (World Info trigger entries)
- Your character cards are for multiple characters
- You don't use character cards
CharMemory and lorebook-based memory extensions can coexist — they use different storage mechanisms.
CharMemory does not touch your lorebooks so if those contain information about the character's world, they can supplement Data Bank memories. CharMemory does use its own structured markdown format for new Data Bank files. It will optionally process your existing files into the same format.
## Feature Highlights
- **Automatic extraction** — automatically extracts memories from your current chat. No manual steps after setup.
- **Group chat support** — each group member gets their own memory file, extracted in a single pass. View and edit all members' memories from one place, including a full Data Bank browser per character.
- **Injection Viewer** — a real-time sidebar showing exactly which memories were injected for any message, with lorebook entries, extension prompts, and per-source token estimates.
- **Prompt Breakdown** — see exactly where your context window is going. The injection viewer shows a per-category token breakdown (System, Char card, Lorebook, Data Bank, Examples, Chat history) with a stacked bar and actionable tips for reducing usage.
- **Vector Storage health checks** — a traffic-light indicator flags misconfigured settings and tells you what to fix, so you're not guessing why memories aren't showing up.
- **Full memory control** — browse, edit, or delete individual memories with a unified block editor (inline editing, undo, find/replace). Consolidate duplicates with preview and undo. Batch-extract from all chats at once.
- **Highly configurable extraction prompts** — separate prompts for 1:1 and group chats and memory file consolidation and conversion.
- **Guided setup** — the Setup Wizard tests your LLM connection, checks Vector Storage, and handles existing memory file conversion in about 2 minutes.
- **Tablet & phone support** — on touch devices, the dashboard opens as a floating panel with touch-friendly controls. Phone layout widens drawers for small screens. If auto-detection doesn't match your device, override it in Settings > Advanced > Display Mode. Please [report issues on GitHub](https://github.com/bal-spec/sillytavern-character-memory/issues) with how it behaves on your device.
- **Plain files** — memories are stored as readable, editable markdown in the character's Data Bank. No database, no lock-in.
## What you need
- A working **SillyTavern** installation
- An **API key** for any LLM provider (OpenRouter, Groq, DeepSeek, NanoGPT, etc.) — or use **Pollinations** for free testing with no key, or a **local server** (Ollama, KoboldCpp, llama.cpp, LM Studio) with no key needed
- **Vector Storage** extension (ships with SillyTavern — just needs to be enabled)
- An **LLM API key** (OpenRouter, Groq, DeepSeek, NanoGPT, etc.) — or use **Pollinations** (free, no key) or a **local server** (Ollama, LM Studio, etc.)
- **Vector Storage** — ships with SillyTavern, just needs to be enabled
## Get started
## Install
> **Back up first** — if you already have Data Bank files or character notes you care about, [back them up](#before-you-start--back-up-your-data) before installing. Memory operations can modify or delete files.
**1.** Click the **Extensions** icon (puzzle piece) in SillyTavern's top bar
**1. Install**
Extensions (puzzle piece icon) → Install extension → paste this URL → Install just for me:
**2.** Click **Install extension** → paste this URL → **Install just for me**:
```
https://github.com/bal-spec/sillytavern-character-memory
```
**2. Connect an extraction LLM**
Find **Character Memory** in Extensions → open **Settings** → pick a **Provider** → enter your **API Key** → **Connect** → select a **Model** → **Test Model**.
**3.** Open a chat and scroll down in Extensions — **CharMemory** appears at the bottom. The Setup Wizard should open automatically on first run. If it does not, click the **wand icon** in the panel header to open the Setup Wizard.
Not sure which model? **GLM 4.7** and **DeepSeek V3.1** are good starting points.
**4.** The wizard walks you through connecting an LLM and checking Vector Storage. Takes about 2 minutes.
**3. Enable Vector Storage**
In Extensions, find **Vector Storage** → set source to **Local (Transformers)** → under File vectorization settings, check **Enable for files**.
→ **[Getting Started](docs/getting-started.md)** has a step-by-step walkthrough of the wizard and your first extraction.
Local Transformers is the simplest option — no API key, runs in your browser. If you're on a low-powered device or want faster vectorization, select an API-based source (OpenAI, NanoGPT, Cohere, etc.) instead. Either way, the critical setting is **Enable for files** — without it, memories are stored but never retrieved.
## Before you start
**4. Chat**
Chat normally. After 20 character messages, memories are extracted automatically. Click **View / Edit** in the CharMemory panel to see what was captured.
CharMemory stores memories as plain files in the character's Data Bank. Tools like **Consolidate**, **Replace All**, and **Clear All Memories** modify or delete these files and cannot always be undone. **Back up your Data Bank files** before making bulk changes — use SillyTavern's built-in [backup tools](https://docs.sillytavern.app/usage/user-settings/) or download individual files from the Data Bank.
## What to expect
## Documentation
Once set up, the CharMemory panel shows a **stats bar** tracking extraction progress (e.g., "5/20 msgs"). When the counter reaches the threshold, extraction fires automatically.
| | |
|---|---|
| [Getting Started](docs/getting-started.md) | Setup Wizard, first extraction, verifying it works |
| [Injection Viewer](docs/injection-viewer.md) | See what memories are going into the prompt in real time |
| [Group Chats](docs/group-chats.md) | Multi-character extraction, Data Bank viewer for all members |
| [Managing Memories](docs/managing-memories.md) | View/Edit, Consolidate, Batch, Reformat, Data Bank tools |
| [Retrieval & Prompts](docs/retrieval-and-prompts.md) | Vector Storage setup, tuning, and prompt design for better recall |
| [Providers](docs/providers.md) | API keys, model recommendations, free options |
| [Troubleshooting](docs/troubleshooting.md) | Health checks, diagnostics, reset tools |
| [Architecture](docs/architecture.md) | Technical overview for developers and contributors |
- **View / Edit** opens the Memory Manager where you can browse, edit, or delete individual memory bullets
- **Extract Now** processes all unprocessed messages immediately — no need to wait for the auto threshold
- **Extract Here** (brain icon on any character message) extracts up to that specific message
- **Consolidate** merges duplicate and related memories when the file grows large
## About the examples
Memories are stored as a plain markdown file in the character's Data Bank. You can edit the file directly at any time.
The documentation uses **Flux** (an orange tabby cat) and **Alex** as fictional test characters. Flux was the real character used during CharMemory's development — their memories were the test corpus that shaped the tuning recommendations, retrieval settings, and prompt design described throughout the docs.
---
## What it looks like
*Everything below is the full guide — detailed setup with screenshots, feature reference, troubleshooting, and technical docs.*
![Memory Manager showing memory cards with find/replace and edit controls](images/memory-manager.png)
## Before You Start — Back Up Your Data
![Injection Viewer showing context breakdown, memories, and lorebook entries](images/injection-viewer.png)
CharMemory writes to your character's Data Bank files. If you already have memory files, character notes, or other Data Bank attachments you care about, **back them up first**.
To back up: open SillyTavern → click a character → open their **Data Bank** (paperclip icon) → download any files under Character Attachments.
Operations like **Clear All Memories** and **Consolidation** modify or delete memory files and cannot always be undone. A backup takes seconds and protects hours of accumulated memories.
---
## Feature Overview
When you chat with a character in SillyTavern, the conversation disappears from the LLM's context as it scrolls past the token limit. CharMemory solves this by automatically extracting important facts, events, and developments from your chats and storing them as structured memories.
Memories are stored as plain markdown files in the character's **Data Bank** — SillyTavern's built-in file attachment system. You can view, edit, or delete the memory file at any time, either through CharMemory's Memory Manager or by editing the Data Bank file directly.
These memory files are then vectorized by **Vector Storage** (a standard extension that ships with SillyTavern) so that the most relevant memories are automatically retrieved and injected into the LLM's context at generation time.
- **Automatic**: Extracts memories every N character messages/turns (configurable with cooldown for rapid-fire conversations)
- **Chunked**: Loops through all unprocessed messages in chunks to prevent overwhelming the LLM's context window
- **Batch extraction**: Extract memories from all (or selected) chats for a character, not just the active one
- **Visible**: Memories stored as a plain markdown file in character Data Bank — fully viewable and editable
- **Per-bullet management**: Browse, edit, or delete individual memory bullets from the Memory Manager
- **Consolidation**: Merge duplicate and related memories with preview before applying and one-click undo
- **Convert / Import**: Convert any Data Bank file into CharMemory format with an interactive preview dialog — supports bullet lists, numbered lists, markdown, freeform text, and LLM-assisted restructuring
- **Memory file format settings**: Control how memories are separated for Vector Storage chunking — block-level, bullet-level, or custom separator
- **Group chat support**: Works in group chats — each member gets their own memory file, extracted and managed individually
- **Scoped**: Memories are per-character by default, with optional per-chat isolation
- **Non-destructive**: Only appends, never overwrites existing memories
- **Multiple LLM sources**: Dedicated connection to an LLM provider via API (recommended), WebLLM (browser-local), or the Main LLM provider in use for the chat
- **Memory/Lorebook diagnostics**: Shows you exactly what the LLM saw during its last generation to help debug memories and lorebook entries not showing up/triggering
- **Injection Viewer**: Per-message side drawer showing exactly which memories, lorebook entries, and extension prompts were injected for any specific generation
- **Injection Health Score**: Traffic-light indicator (green/yellow/red) that checks your Vector Storage configuration and flags issues like missing files, zero overlap, or duplicate memories
---
## Detailed Setup Guide
This section walks through each step in detail with screenshots. If you already followed the [Get started](#get-started) steps above and everything's working, you can skip ahead to [Per-Message Buttons](#per-message-buttons) or [Understanding the Extraction Settings](#understanding-the-extraction-settings).
### Prerequisites
- A working SillyTavern installation
- If you are using a hosted LLM provider you will need an API key. If you're using the same provider as your chat (not recommended) then this is already configured. If you're using WebLLM it is not needed
### Step 1: Install the Extension
1. Open SillyTavern in your browser
2. Click the **Extensions** icon (puzzle piece) in the top navigation bar
3. Click **Install extension** in the top-right corner of the Extensions panel
4. Paste the GitHub URL:
```
https://github.com/bal-spec/sillytavern-character-memory
```
5. Click **Install just for me** and wait for the installation to complete
![Install extension dialog — paste the GitHub URL and click Install](images/02-install-dialog.png)
6. Scroll down in the Extensions panel — you should see **Character Memory** at the bottom
### Recommended: Turn on Chat Timestamps and Message IDs
Before you start chatting, enable these two options in SillyTavern's User Settings. They're not required, but they make CharMemory much easier to work with:
- **Chat Timestamps** — shows when each message was sent. Useful for correlating messages with extraction dates in the memory file.
- **Message IDs** — shows a sequential number on each message. The Activity Log references message indices (e.g., "Collected 15 messages (indices 0-14)"), the "Extract Here" button processes up to a specific index, and Diagnostics show `lastExtractedIndex` so you can see exactly which messages have been processed.
To enable: click the **User Settings** icon (the person silhouette at the top) → scroll to the checkboxes in the UI section → check **Chat Timestamps** and **Message IDs**.
![User Settings — check Chat Timestamps and Message IDs](images/03-message-ids.png)
### Step 2: Choose an Extraction Provider
CharMemory needs an LLM to read your chat messages and extract memories from them. This is a separate LLM call from your main chat. Open the **Settings** section inside the CharMemory panel.
You have three options for **LLM Used for Extraction**:
| Option | How it works | Best for |
|--------|-------------|----------|
| **Dedicated API** (recommended) | Sends a clean extraction request directly to an API | Best extraction quality — the extraction prompt isn't polluted by chat prompt |
| **WebLLM** | Runs a small model locally in your browser | Privacy and no API cost, but limited quality |
| **Main LLM** | Uses whatever LLM you're chatting with | No extra setup, but extraction quality suffers because the prompt gets mixed with chat system prompts and instructions |
#### Setting up Dedicated API
Dedicated API is the default and recommended option. It sends *only* the extraction prompt to the LLM — no chat system prompts, jailbreaks, persona instructions, or other context gets mixed in. (The extraction prompt itself includes the character card as a bounded reference section so the LLM knows what not to re-extract — but that's intentional and controlled, unlike Main LLM where everything piles up.) This produces noticeably better memories.
1. Open **Settings** in the CharMemory panel — **Dedicated API** is already selected
2. Choose a **Provider** from the dropdown. Options include OpenAI, Anthropic, OpenRouter, Groq, DeepSeek, Mistral, xAI (Grok), NanoGPT, NVIDIA, Local Server (Ollama / KoboldCpp / llama.cpp / LM Studio), Pollinations (free, no key needed), and Custom.
3. Enter your **API Key** for that provider (click the **(get key)** link next to the field for a direct link to the provider's key page)
4. Click **Connect** to fetch the list of available models
5. Select a **Model** — the model picker is searchable: type to filter by name, which is especially helpful for providers with 100+ models like NanoGPT
6. Click **Test Model** to verify the model responds correctly
![CharMemory Settings — Dedicated API with NanoGPT, model selection, and auto-extraction sliders](images/04-settings-provider.png)
![Searching for models by typing in the model picker](images/16-model-search.png)
If your provider isn't listed, select **Custom** from the Provider dropdown. You can enter any OpenAI-compatible API base URL and it will work as long as the endpoint supports the `/chat/completions` format. Most LLM providers use this standard.
**Note on Local Server**: Select "Local Server" from the Provider dropdown to use Ollama, KoboldCpp, llama.cpp, or LM Studio. The Base URL field appears automatically — adjust the port to match your backend:
| Backend | Default URL |
|---------|-------------|
| KoboldCpp | `http://localhost:5001/v1` |
| llama.cpp | `http://localhost:8080/v1` |
| LM Studio | `http://localhost:1234/v1` |
| Ollama | `http://localhost:11434/v1` |
You can also use a LAN IP (e.g., `http://192.168.1.50:5001/v1`) if the server is running on another machine. No API key is needed. Click **Connect** to fetch available models, then select one and **Test Model**.
**Note on NVIDIA**: NVIDIA's API doesn't support CORS (browser-to-API requests), so CharMemory automatically routes NVIDIA requests through SillyTavern's server. This happens transparently — no extra setup is needed, just select NVIDIA, enter your API key, and go. Your API key is passed securely via headers and never touches SillyTavern's own configuration.
**Note on Pollinations**: Pollinations is a free provider that requires no API key — useful for trying CharMemory without signing up for anything. Select Pollinations, type a model name (e.g., `openai`), and go. Quality depends on which model Pollinations routes to, so it's best for testing rather than production use.
If you're not sure which model to use, see the [Recommended Models](#recommended-models) section below.
### Step 3: Chat Normally
That's it for basic setup. Now just chat with a character as you normally would.
As you chat, open the extension to watch the **stats bar** at the top of the CharMemory panel. You'll see the extraction progress counter tick up with each character message (e.g., "5/20 msgs"). When the counter reaches the threshold (default: 20 messages), CharMemory will automatically extract memories from the conversation.
![The stats bar shows the memory file, count, extraction progress, and cooldown status](images/05-stats-bar.png)
#### What the Stats Bar Shows
- **File name**: The memory file for the current character (e.g., `Flux_the_Cat-memories.md`). This is auto-generated from the character name, but you can set a custom name in Settings → Storage → File name override.
- **Memory count**: Total individual memory bullets stored
- **Progress**: Messages since last extraction vs. the auto-extract threshold (e.g., "1/20 msgs")
- **Status**: "Ready" when extraction can fire, or a cooldown timer
- **Health**: A colored dot indicating injection health — green (all checks passed), yellow (warnings), red (problems detected), or gray (not yet evaluated). Click it to jump to the Diagnostics panel for details. See [Injection Health Score](#injection-health-score) below.
![Stats bar showing file name, memory count, extraction progress, status, and green health dot](images/23-stats-bar-health.png)
#### Your First Extraction
You don't have to wait for the auto-extraction threshold. There are two ways to extract right away:
1. **Extract Now** (button at the top of the CharMemory panel) processes all unprocessed messages in the entire chat. Click it, and you'll see a toast notification with how many memories were saved.
2. **Extract Here** (brain icon on any character message) processes all unprocessed messages up to and including that specific message. This is useful when you want to extract from a particular point in the conversation without processing everything after it.
![The brain icon on a character message — "Extract memories up to here"](images/11-extract-here.png)
You can follow either extraction in real time in the **Activity Log** (Tools → Activity Log). It shows each step: messages collected, LLM call sent, response received, and memories saved.
![Activity Log showing a successful extraction — 15 messages collected, 7 memories saved](images/08-activity-log.png)
### Step 4: View Your Memories
Click **View / Edit** to open the Memory Manager. Your extracted memories appear as cards grouped by extraction, showing the chat name and timestamp. Blocks are displayed in reverse chronological order — newest extractions first, so the most recent memories are always at the top. Each bullet has its own edit and delete buttons.
![Memory Manager showing 7 extracted memories with edit and delete controls](images/06-memory-manager.png)
You can **edit** any bullet to refine its wording, or **delete** bullets that aren't useful. If a block becomes empty after deleting all its bullets, it's removed entirely.
Since memories are stored as a plain markdown file in the character's Data Bank, you can also edit the file directly if you prefer. Open the character's Data Bank panel (the paperclip icon), find the memory file, and edit it in any text editor. The Memory Manager is simply a more convenient interface for the same file.
### Step 5: Set Up Vector Storage
Extracting memories is only half the story. For your character to actually *use* those memories during conversation, you need **Vector Storage** enabled.
Vector Storage is a standard extension that ships with every SillyTavern installation — you don't need to install anything extra. It converts memories into embeddings (numerical representations) and retrieves the most relevant ones when the character generates a response.
Without Vector Storage enabled for Data Bank files, memories are stored but never injected into the LLM's context — the character won't recall them.
#### Enable Vector Storage
1. In the **Extensions** panel, find **Vector Storage** and expand it
2. Choose a **Vectorization Source**. The simplest option is **Local (Transformers)** — runs in your browser, no API key needed. Local vectorization is perfectly adequate for CharMemory (see note below).
3. Under **File vectorization settings**, check **Enable for files** — this is the critical setting. CharMemory stores memories as Data Bank files, so this must be on.
4. Configure the **Data Bank files** settings as shown below
![Vector Storage settings — Transformers source, Enable for files checked, Data Bank settings configured](images/07-vector-storage.png)
#### Recommended Vector Storage Settings
The Vector Storage panel has two rows of file settings: **Message attachments** (top) and **Data Bank files** (bottom). CharMemory uses the Data Bank, so focus on the bottom row:
| Setting | Recommended | Why |
|---------|-------------|-----|
| **Size threshold** | 1 KB | Controls when chunking kicks in. Below this size, the whole file gets one embedding. At 1 KB (~5-10 memory bullets), individual chunks start getting their own vectors so Vector Storage can retrieve *specific* relevant memories instead of the whole file as a blob. |
| **Chunk size** | 3000 chars | A `<memory>` block with 8 bullets is roughly 500-1500 chars. 3000 keeps 1-2 full blocks per chunk without splitting them mid-sentence. Too small and blocks get cut in half. Too large and you lose retrieval granularity. |
| **Chunk overlap** | 15% | ~450 chars of overlap at 3000 chunk size. Catches memory blocks that straddle a chunk boundary. Without overlap, a block landing exactly on the split gets half in one chunk and half in another, making neither retrievable cleanly. |
| **Retrieve chunks** | 5 | How many memory chunks are retrieved per generation. At ~2 blocks per chunk, that's roughly 10 memory blocks — enough context without flooding the prompt. Going too high (20+) effectively dumps the whole file, defeating the purpose of semantic search. |
Not sure if your settings are right? The [Injection Health Score](#injection-health-score) checks your chunk size, overlap, and other Vector Storage settings automatically — look for the colored dot in the stats bar.
#### Local vs API-Based Vectorization
**Local (Transformers)** runs the embedding model in your browser. It's the simplest option — no API key, no cost, no privacy concerns (memories never leave your machine). For retrieval quality, local is perfectly adequate: embedding is a much simpler task than generation, and for a typical CharMemory use case (dozens to low hundreds of memory bullets), the semantic gaps between relevant and irrelevant memories are wide enough that any reasonable model catches them.
**When to use an API source instead:**
- **Low-powered devices** — Local Transformers loads a ~100 MB model into your browser and runs inference on your CPU/GPU. On a phone, tablet, Chromebook, or older laptop this can be noticeably slow and eat battery. An API source offloads that work to a remote server.
- **Faster vectorization** — API sources return embeddings in milliseconds per call vs. the local model's per-chunk processing time. This matters most during bulk operations like batch extraction or revectorization of large memory files.
- **Large memory files** — If a character has hundreds of memory bullets, the local model has to churn through many chunks. An API source handles this faster.
**When local is the better choice:**
- You don't want another API key or dependency
- Privacy matters — your memories stay on your machine
- Your device handles it fine (most desktop/laptop setups do)
The retrieval quality difference between local and hosted embeddings is negligible for CharMemory. The bottleneck is almost always memory *content* quality, not the embedding model.
#### Verify It's Working
After extracting some memories and chatting further, the quickest check is the **health dot** in the CharMemory stats bar — green means your Vector Storage settings are correct and memories are being injected. If it's yellow or red, click it to jump to Diagnostics where each issue is explained with a recommendation. See [Injection Health Score](#injection-health-score) for details on what it checks.
For a deeper look, use [Diagnostics](#using-diagnostics) to see the exact memories that were injected in the last generation, or the [Injection Viewer](#injection-viewer) to inspect what any specific past message received.
---
## Per-Message Buttons
Each message in your chat has two extra buttons (visible when you hover over the message):
1. **Extract Here** (brain icon, character messages only)
Runs LLM-based extraction on all unprocessed messages up to and including this one. Useful for targeting a specific point in a long conversation. Uses the same provider and settings as auto-extraction.
2. **Pin as Memory** (bookmark icon, all messages)
Manually saves a message as a memory with no LLM involved. Opens an edit dialog pre-filled with the message text so you can rewrite it however you want before saving. Each line becomes a memory bullet. Use this when you want to remember something specific exactly as you phrase it.
---
## Group Chats
CharMemory works in group chats with no extra setup. Each group member gets their own memory file, and extraction handles all members in a single pass. The Settings panel automatically adapts to show group-specific options (member file names, group extraction prompt) when a group chat is active, and 1:1 options when a solo chat is active.
![Group chat panel showing character avatars in the stats bar and extraction controls](images/12-group-main.png)
### How It Works
When extraction fires in a group chat — whether automatically or via Extract Now — CharMemory processes each chunk of messages once per group member. For each member, it:
1. Reads that character's existing memories
2. Builds an extraction prompt that includes the character card and a participant list so the LLM knows who is speaking
3. Sends the chunk to the LLM
4. Appends any new memories to that character's file
Progress shows which character is being processed (e.g., "Alice (2/6)"). If the LLM call fails for one member, extraction continues with the remaining members — one failure won't abort the entire group.
### Viewing and Editing Group Memories
Click **View / Edit** in a group chat and you'll see per-character sections, each with their own memory cards. Edit and delete controls work the same as in 1:1 — they target the correct character's file based on which section the button is in.
![Group memory manager showing per-character memory sections](images/14-group-memory-manager.png)
Newest memory blocks appear first (reverse chronological) in both 1:1 and group chats.
### Consolidation in Groups
The Consolidate button in a group chat shows a character picker — select which character's memories to consolidate. Consolidation works on one character at a time to keep the preview manageable. Undo restores that character's previous memories.
### Pin Memory in Groups
The bookmark button on a group message routes the pinned memory to the correct character's file based on the message sender. If the sender can't be matched to a group member (e.g., a narrator message), it goes to the first member.
### Per-Character Filenames
By default, each group member's memory file is auto-named from their character name (e.g., `Alice-memories.md`). You can configure custom filenames per character in Settings when a group chat is active.
![Per-character memory file names in group settings](images/13-group-member-files.png)
### How Memory Retrieval Works in Groups
During generation, SillyTavern sets the active character to whichever group member is about to speak. Vector Storage then retrieves memories from that character's Data Bank and injects them into the prompt. This means each character gets their own memories when it's their turn to generate — Vulkan gets Vulkan's memories, Flux gets Flux's.
**Diagnostics caveat:** After generation finishes, SillyTavern resets the active character to none. If you click Diagnostics → Refresh between generations, the "Injected Memories" section may appear empty because there's no character context at that moment. This doesn't mean memories weren't injected — it just means the diagnostics snapshot was taken outside of a generation turn.
### Reset and Clear in Groups
**Reset Extraction State** in a group chat clears tracking for all group members, not just one.
**Clear All Memories** deletes memory files for all group members in the current group.
---
## Other Features
### Batch Extraction
If you have existing chats with a character, you don't need to manually extract each one. Batch extraction processes multiple chats at once:
1. Open **Tools** tab → **Batch** pill
2. Click **Refresh** to load the list of chats for the current character
3. Select the chats you want to extract (use **Select All** to check all of them)
4. Click **Extract Selected** — a confirmation popup shows the total message count
5. Progress updates show which chat is being processed and chunk progress
6. Use **Stop** to cancel mid-extraction — progress is saved per-chunk, so you won't lose work
Each chat's extraction state is tracked separately. Re-running batch extraction only processes new messages since the last run — it won't re-extract messages that have already been processed.
#### Expectations for Long Existing Chats
Batch extraction works best for catching up on recent unprocessed chats. For very long existing chats (hundreds of turns), results may be sparser than you'd expect. This is by design — the LLM only sees one chunk at a time and can't assess significance across the full conversation arc the way it can when extracting incrementally as you chat.
CharMemory works best when it extracts **as you go** — each extraction builds on the previous memories, and the LLM has both the existing memories and the current chunk to work with. When starting fresh on a very long chat, the early chunks have no existing memories for context, so the LLM may miss details that only become significant later.
If batch extraction of a long chat produces too few memories, try:
- **Increasing "Messages per LLM call"** — giving the LLM a bigger window (40–50 messages) helps it identify more significant events per chunk
- **Running consolidation after extraction** — this can merge and refine the sparse results
- **Starting a new chat with the character** — incremental extraction as you chat naturally produces the best results over time
### Resetting Extraction State
Two reset options are available in Settings:
**Reset Extraction State** resets the extraction tracking for the current character — both the active chat and all batch extraction state. After resetting, the extension treats all messages as unprocessed. This is useful when you want to re-extract from the beginning, perhaps after changing the extraction prompt or switching to a better model. It does **not** delete any memories. In group chats, this resets tracking for all group members.
**Clear All Memories** deletes the memory file and resets all extraction tracking. In default mode (not per-chat), the memory file contains memories from **all** of that character's chats, so this clears everything. This cannot be undone. In group chats, this deletes memory files for all group members.
### Consolidation
When the memory file grows large with many extraction blocks, related or duplicate memories can accumulate across different sessions. The **Consolidate** tool (**Tools** tab → **Consolidate** pill) lets you send the full memory file to the LLM to deduplicate and combine related entries.
Consolidation is always manual — it never runs automatically.
#### Strategy Presets
Before consolidating, choose a strategy from the dropdown (**Tools** tab → **Consolidate** pill):
| Strategy | What it does |
|----------|-------------|
| **Conservative** | Only merges near-exact duplicates. Safest option — preserves the most detail. |
| **Balanced** | Merges duplicates and combines related facts. Good default. |
| **Aggressive** | Compresses heavily, summarizes by theme. Best for very large memory files that need significant reduction. |
Each preset has its own prompt that you can view and customize. Click the expand arrow to see the full prompt, edit it to taste, and save. **Restore Default** reverts a preset to its original prompt.
#### The Consolidation Workflow
1. Pick a strategy and click **Consolidate**
2. The LLM processes your memories and returns a consolidated version, organized by theme (e.g., "Relationship History", "Key Events")
3. Results appear as **editable cards** — not raw text. Each theme block is read-only by default; click the **pencil icon** on any block to enter edit mode for that block
4. You can **edit** individual bullets, **delete** bullets or entire blocks, **add** new bullets, and **rename theme headers** before applying
5. Not happy with the result? Click **Re-run** to get a fresh consolidation. Each re-run saves the previous version to a **version stack** — click **Undo** to step back through prior versions
6. When satisfied, click **Apply** to write the consolidated memories to the file
**Back up your memory file before consolidating**, especially if you have a large number of memories. The undo is session-only — if you close SillyTavern, the backup is lost. To back up: open the character's Data Bank (paperclip icon) and download the memory file.
In group chats, consolidation shows a character picker — select which character's memories to consolidate. See [Group Chats](#group-chats) for details.
Consolidation automatically uses 2x your configured "Max response length" as its token budget, since it processes the full memory file rather than a single chunk. If you're using a thinking model, this means consolidation gets even more headroom (e.g., 2000 response length → 4000 tokens for consolidation).
Results vary depending on the model used and the size of the memory file. Review the preview carefully before applying.
### Convert / Import
If you have existing Data Bank files with character notes, memory lists, or other text, the **Convert** tool can restructure them into CharMemory's `<memory>` tag format.
Open **Tools** tab → **Convert** pill:
![Convert tool panel](images/18-convert-tool.png)
1. Select a **source file** from the dropdown (shows all Data Bank files except the active CharMemory file)
2. Optionally check **Use LLM to restructure** — recommended for freeform text with no clear structure. Uses your configured extraction provider
3. Click **Preview Conversion** — a popup dialog opens with two panes:
| Left pane | Right pane |
|-----------|------------|
| Original file content (read-only) | Converted memories as editable cards |
4. **Edit the result** before saving — click the pencil icon on any block to enter edit mode. You can edit bullets, delete bullets or blocks, add new ones, and rename theme headers
5. Not happy? Click **Re-run** to re-parse (toggle the LLM checkbox to switch methods). Each re-run saves the previous version — click **Undo** to step back
6. Choose an **output destination** at the bottom — the auto-generated CharMemory file, or a custom filename
7. Click **OK** to save, or **Cancel** to discard
![Convert preview dialog](images/19-convert-preview.png)
The Convert tool detects 6 input formats automatically:
| Format | Example |
|--------|---------|
| CharMemory `<memory>` tags | Already in format — no conversion needed |
| Old CharMemory (`## Memory N`) | Auto-migrated |
| Bullet lists (`- ` or `* `) | Each bullet becomes a memory |
| Numbered lists (`1.`, `2)`) | Each item becomes a memory |
| Markdown with headings | Headings become block themes |
| Freeform text | Split on sentences (use LLM for better results) |
**Non-destructive**: the original file is never modified or deleted. After converting, hide or remove the original from the Data Bank to avoid duplicate memories being injected.
The LLM conversion prompt is configurable — expand **Show prompt** below the LLM checkbox to view and edit it. Click **Restore Default** to revert.
### Per-Chat Memories
By default, all chats for a character share one memory file. Enable **Separate memories per chat** in Settings → Storage to give each conversation its own file. This is useful when the same character appears in different scenarios or timelines that shouldn't share context.
This also works in group chats — each group member gets a separate per-chat memory file.
### Custom File Names
The memory file is auto-named from the character name (e.g., `Flux_the_Cat-memories.md`). You can override this in Settings → Storage → **File name override**. This is useful if you want a more descriptive name or if you're managing multiple memory files manually.
### Slash Commands
| Command | Description |
|---------|-------------|
| `/extract-memories` | Force extraction regardless of interval |
| `/consolidate-memories` | Consolidate memories by merging duplicates |
| `/charmemory-debug` | Capture diagnostics and dump to console |
---
## Using Diagnostics
The **Diagnostics** tab (Tools → Diagnostics) shows you exactly what the LLM saw during its last generation. This is the single best tool for answering "why isn't my character remembering X?" or "what memories are actually being used?"
Click **Refresh** after generating a message to capture the current state.
![Diagnostics showing memory count, vectorization status, last extraction result, and character lorebooks](images/09-diagnostics.png)
### What Diagnostics Shows
**Memories** — The active memory file name, whether it exists in the Data Bank, total memory count (bullets and blocks), and vectorization status (including chunk count and embedding source). This tells you whether your memories are stored and indexed correctly.
**Injected Memories — Last Generation** — The specific memory bullets that Vector Storage retrieved and sent to the LLM for the most recent generation. This is the most important section — it shows you exactly which memories the character had access to when it wrote its last response. If a memory exists in the file but doesn't appear here, it either wasn't semantically relevant to the current conversation or Vector Storage settings need adjustment.
**Character Lorebooks** — A static list of all World Info / lorebook books bound to the current character, with entry counts and trigger keys. This appears in diagnostics because lorebook entries and memories both get injected into the LLM's context, and they can interact — a lorebook entry might provide world-building context that complements a memory, or they might conflict. Seeing both in one place helps you understand the full picture of what supplemental context the character has.
**Activated Entries — Last Generation** — Which specific lorebook entries actually fired during the last generation, based on their trigger keys matching the conversation. Unlike the static list above, this shows what was *actually injected* — so you can see if a lorebook entry you expected to fire didn't, or if unexpected entries are crowding out memory context.
**Extension Prompts** — All content injected by extensions (including Vector Storage's memory retrieval and any other active extensions). This is the raw view of everything beyond the base conversation that the LLM received. The `4_vectors_data_bank` entry shows the full content retrieved by Vector Storage — this is what gets injected into the LLM's context alongside the conversation.
**Injection Health** — A health card showing the results of automated checks on your Vector Storage configuration. Each check is color-coded (green/yellow/red) with an explanation and recommendation if something needs attention. See [Injection Health Score](#injection-health-score) for details on each check.
**Note on group chats**: In group chats, Diagnostics shows memory info for the first group member only. To check a specific character's memories, use View/Edit which shows all members.
### Why Memories and Lorebooks Both Appear
CharMemory's diagnostics shows both memories and lorebooks because they're the two main sources of supplemental character context that get injected alongside the conversation. When debugging "the character doesn't remember X" or "the character is acting strangely," the answer often involves the interaction between these sources — not just one in isolation. The diagnostics panel gives you a single place to inspect everything the LLM saw beyond the chat messages themselves.
---
## Injection Viewer
The **Injection Viewer** is a side drawer that shows you exactly what context was injected for any specific message. While Diagnostics gives you a snapshot of the *latest* generation, the Injection Viewer lets you inspect *any* past message — making it easy to compare what changed between generations.
### Opening the Viewer
On each character message, you'll see a small pen/quill icon (next to the edit and menu buttons). Click it to open the Injection Viewer drawer on the right side of the chat. The drawer title shows which message you're inspecting (e.g., "Message #999").
You can also open the drawer from the small toggle tab on the right edge of the chat area, then click the icon on any message to load its data.
### What the Viewer Shows
The drawer has three collapsible sections:
**CharMemory** — The specific memory bullets that Vector Storage retrieved and injected for this generation. Each bullet is listed individually with a count in the header (e.g., "CharMemory (17)"). This is the most direct answer to "what did my character remember when writing this message?"
**Lorebook Entries** — Which World Info / lorebook entries were activated for this generation, based on keyword triggers matching the conversation. Each entry shows its name, trigger keys, and a preview of its content. If a lorebook entry you expected to fire isn't listed here, its keywords didn't match the recent conversation context.
**Extension Prompts** — The raw content injected by all extensions, keyed by their injection position (e.g., `4_vectors_data_bank` for Vector Storage, `2_floating_prompt` for Author's Note). This is the unprocessed view — useful for seeing the exact text the LLM received, including `<memory>` block markup and chunk boundaries.
![Injection Viewer showing lorebook entries that were activated for a message](images/21-injection-viewer-lorebooks.png)
![Injection Viewer showing 17 extracted memories injected for a message](images/22-injection-viewer-memories.png)
### How Data is Captured
Injection data is captured automatically at generation time — when the character produces a response, CharMemory takes a snapshot of all injected context at that moment. This means:
- **Only character messages have injection data** — user messages don't trigger a generation, so there's nothing to capture
- **Older messages may not have data** — snapshots are only captured while the extension is installed and active. Messages generated before installation show "Click the icon on a message to view its injected context"
- **Data persists for the session** — injection snapshots are stored in chat metadata, so they survive page refreshes within the same chat
### Closing the Viewer
Click the **X** button in the top-right corner of the drawer, or **swipe right** on the drawer (touch devices).
### Toolbar
Just below the header, a toolbar shows the capture timestamp and a **Diagnostics** link:
- **On desktop**: Clicking "Diagnostics" opens the Extensions panel and scrolls to the CharMemory diagnostics section.
- **On touch devices** (iPad, phones): Clicking "Diagnostics" shows an inline health summary directly in the drawer — no need to navigate away from the chat. Tap it again to dismiss.
### Using Injection Viewer with Diagnostics
The Injection Viewer and Diagnostics serve complementary purposes:
| | Injection Viewer | Diagnostics |
|---|---|---|
| **Scope** | Per-message — inspect any past generation | Latest generation only |
| **Detail** | Parsed sections (memories, lorebooks, prompts) | Full system overview (file status, vectorization, health) |
| **Best for** | "What did the character know when it wrote *this* message?" | "Is my setup working correctly?" |
| **Access** | Pen icon on any character message | Tools → Diagnostics → Refresh |
For a complete picture, use both: Diagnostics to verify your setup is healthy, and the Injection Viewer to spot-check individual messages.
---
## Injection Health Score
The **Health Score** is a traffic-light indicator that automatically checks whether your Vector Storage settings are configured correctly for CharMemory. It surfaces problems and recommendations so you don't have to manually inspect raw injected content.
### Where It Appears
- **Stats bar** — A colored dot as the 5th item in the CharMemory stats bar. Click it to scroll directly to the Diagnostics panel for details.
- **Injection Viewer drawer** — A colored dot in the drawer header, with a tooltip showing injection-specific stats (memories injected, duplicates detected). This dot stays gray until a generation has been captured. On touch devices, the "Diagnostics" link in the toolbar also shows the full health check results inline.
- **Diagnostics panel** — A detailed health card showing each check with its status and recommendations.
### Health Levels
| Color | Meaning |
|-------|---------|
| **Green** | All checks passed — your setup looks good |
| **Yellow** | Warnings — things work but could be improved |
| **Red** | Problems — something is preventing memories from being injected correctly |
| **Gray** | Not yet evaluated — no character selected or no generation captured |
### What It Checks
The health score runs up to 7 checks, depending on what data is available:
| Check | What it looks for | When it's a problem |
|-------|-------------------|---------------------|
| **Files enabled** | "Enable for files" is checked in Vector Storage | RED if disabled — memories are stored but never retrieved |
| **Memory file exists** | A memory file exists in the character's Data Bank | RED if missing — nothing to vectorize |
| **File vectorized** | The memory file has been chunked into vectors | RED if 0 chunks — file exists but hasn't been processed |
| **Chunk overlap** | Vector Storage's Data Bank overlap setting | YELLOW if 0% — memory blocks on chunk boundaries get split, causing duplicates |
| **Chunk size** | Vector Storage's Data Bank chunk size | YELLOW if too small (blocks get split) or too large (loses retrieval granularity) |
| **Memories injected** | Whether memory bullets appeared in the last generation | RED if file exists and is vectorized but 0 memories were injected |
| **Duplicate detection** | Whether the same bullet appears multiple times in injected content | YELLOW if duplicates found — usually means chunk overlap or chunk size needs adjustment |
Checks 1–5 run immediately when you open a chat. Checks 6–7 only run after a generation has been captured (since they inspect injected content).
### Acting on Health Warnings
**RED: Files not enabled** — Open Extensions → Vector Storage → check "Enable for files" under File vectorization settings.
**RED: Memory file not found** — Run an extraction first (Extract Now or wait for auto-extraction). The memory file is created on first extraction.
**RED: File not vectorized** — The file exists but hasn't been processed. Try generating a message (Vector Storage processes files on generation), or check that your vectorization source is configured and working.
**YELLOW: Chunk overlap is 0%** — Open Vector Storage → Data Bank files row → increase the overlap setting. 15% is a good starting point. Without overlap, `<memory>` blocks that land on a chunk boundary get split between two chunks, and neither half retrieves cleanly — this is the most common cause of duplicate memories in injected content.
**YELLOW: Chunk size issues** — If too small, individual memory blocks get split across chunks. If too large, you lose retrieval granularity (Vector Storage retrieves whole chunks, so a huge chunk means lots of irrelevant context). See [Recommended Vector Storage Settings](#recommended-vector-storage-settings) for guidance.
**YELLOW: Duplicates detected** — The same memory bullet appeared multiple times in the injected content. This usually means chunk boundaries are splitting `<memory>` blocks. Increase chunk overlap and/or adjust chunk size so blocks fit cleanly within chunks. After changing settings, purge vectors and revectorize the file.
---
## Understanding the Extraction Settings
Once you're up and running, you may want to tune how often and how extraction happens. Open **Settings** in the CharMemory panel.
### Auto-Extraction Timing
Two sliders control when automatic extraction fires:
**Extract after every N messages** (default: 20, range: 3–100)
How many character messages must arrive before auto-extraction triggers. A higher value gives the LLM more context per extraction, which generally produces better, more selective memories. A lower value extracts more frequently with less context.
**Minimum wait between extractions** (default: 10 min, range: 0–30 min)
A cooldown that prevents rapid-fire extractions during fast-paced chats. When the message threshold is reached, extraction only fires if this much wall-clock time has passed since the last one. If the cooldown hasn't expired, extraction is skipped (not queued) and checks again on each subsequent message. Messages keep accumulating during the cooldown, so when it finally fires, it processes everything that piled up.
These two settings **only affect automatic extraction**. Manual "Extract Now", per-message "Extract Here", and batch extraction always run immediately.
### Extraction Quality
**Messages per LLM call** (default: 20, range: 10–200)
Controls how many messages are sent to the LLM in a single extraction call. If there are more unprocessed messages than this, extraction loops through them in chunks. Larger chunks give the LLM more context per call and can produce better memories, but too many messages can cause timeouts with some providers.
In the common auto-extraction case, only N messages (the interval threshold) will have accumulated, so this slider is irrelevant — the chunk size only kicks in when messages pile up beyond the interval, during manual extraction of long chats, or during batch extraction.
The right value depends on your chat style. If your character writes long, detailed responses, 20 messages might already be a lot of text. If both sides write short messages, you may want to increase this to 40–50 so the LLM has enough context to judge what's significant. The test is to look at the memories it creates — if they're too granular (trivial details), increase this. If extractions are timing out, decrease it.
You can check your memories either using the **View/Edit** button in the extension panel, or by going to the character's Data Bank (magic wand icon → Open Data Bank → Character Attachments) and clicking the pencil icon on the memory file.
Setting this too low (e.g., 10) gives the LLM too little context — it extracts trivial details because there isn't enough conversation to judge what's significant. Setting it too high (150+) doesn't improve quality, increases token costs, and may cause timeouts with some providers.
**Max response length** (default: 1000 tokens, range: 100–4000)
Token limit for the LLM's response per chunk. Most models produce well-formed output within 1000 tokens. **Reasoning/thinking models** (like GLM-4.7 on NVIDIA) need significantly more — their internal reasoning consumes tokens before producing the actual output. If you're using a thinking model and getting empty extractions, increase this to 2000–3000.
**Merge extraction chunks** (default: off)
When a chat has more unprocessed messages than the chunk size, extraction runs in multiple passes. With this off (default), each chunk's memories are stored as separate `<memory>` blocks — keeping them small and manageable for consolidation. With this on, blocks from the same chat are merged into a single block after extraction. Leave this off for long chats (hundreds of messages) where consolidation would be valuable — large merged blocks can exceed the consolidation LLM's capacity.
### How the Settings Interact
The three main sliders — **Extract after every N messages** (interval), **Minimum wait between extractions** (cooldown), and **Messages per LLM call** (chunk size) — work together:
**Interval and chunk size.** The extension tracks a `lastExtractedIndex` watermark. Each message is only ever sent to the LLM once — there is no overlap between extractions. When auto-extraction fires after N messages, only those N unprocessed messages are sent, even if the chunk size is larger. This means that with the defaults (interval=20, chunk size=20), each auto-extraction sends exactly 20 messages to the LLM. The chunk size only becomes relevant when more messages accumulate than the interval — for example, during manual "Extract Now" after a long chat, batch extraction, or when the cooldown delayed auto-extraction and messages piled up.
**Why the interval matters for quality.** A higher interval gives the LLM more messages per extraction, which means more context to judge what's significant. With only 10 messages, the LLM has little to work with and may extract minor details. With 20–50 messages, it can better identify meaningful developments and skip filler.
**How cooldown works.** When the message counter hits the interval threshold, the extension checks whether enough wall-clock time has passed since the last extraction. If not, extraction is **skipped** (not queued). The counter stays above the threshold, so it checks again on each subsequent message until the cooldown expires. During this time, messages keep accumulating. When extraction finally fires, it processes everything that piled up — potentially sending more than N messages and using the chunk size to split them into multiple LLM calls.
**Practical examples:**
- *Fast chat, defaults (interval=20, cooldown=10min):* 20 messages arrive in 3 minutes. Extraction wants to fire but cooldown blocks it. By the time 10 minutes pass, 60 messages have accumulated. Extraction fires and processes all 60 in three chunks of 20.
- *Leisurely chat, defaults:* 20 messages arrive over 45 minutes. Cooldown is long expired. Extraction fires immediately and processes 20 messages in one call. The chunk size is irrelevant.
- *High interval (interval=50, cooldown=0):* Extraction fires every 50 messages with no time gate. Each extraction has rich context and produces higher-quality, more selective memories.
---
## The Extraction Prompt
The extraction prompt is the core of what makes CharMemory produce useful memories rather than a play-by-play transcript. You can view and edit it in Settings → Extraction Prompt, and a **Restore Default** button lets you start over.
The default prompt was developed through extensive testing across multiple models and character types. Here's what it does and why:
**Three-section input structure.** The prompt gives the LLM three clearly bounded sections: the character card (baseline knowledge), existing memories (already recorded), and recent chat messages (what to extract from). Each section has explicit `=====` boundary markers and instructions about what to do with it — extract only from recent messages, don't repeat existing memories, and don't re-state character card traits.
**Why the character card is included.** Early versions without the card produced memories that re-extracted baseline traits. If a character's card says "she's a doctor," the LLM would extract "she works in medicine" from every chat where it came up. Including the card as "baseline knowledge — do NOT extract" dramatically reduced this.
**The "would they bring this up months later?" test.** The prompt asks the LLM to evaluate each potential memory against this question. This pushes models toward significant, lasting facts and away from moment-by-moment details.
**Hard 8-bullet limit.** Without a cap, most models produce 15-20 bullets per extraction — far too granular. The 8-bullet limit forces the LLM to prioritize. If a conversation doesn't contain 8 significant things, the LLM can return fewer.
**Negative and positive examples.** The prompt includes a bad example (step-by-step play-by-play of a scene) and a good example (the same scene condensed to 2 bullets capturing outcomes). This was the single most effective change for reducing play-by-play extraction, which was the most common quality problem across models.
**"Write what happened, not that it was discussed."** Models tend to write meta-narration like "she told him about her childhood" instead of the actual fact "she grew up in a coastal village." The prompt explicitly addresses this pattern.
**Date/time extraction.** The prompt encourages the LLM to capture dates and times when they are mentioned or clearly implied in conversation, adding temporal context to memories (e.g., "In March, she moved to the coast" rather than just "She moved to the coast").
If you customize the prompt, keep the three-section structure and boundary markers intact — models rely on these to understand what to extract from and what to skip.
**Group chats use a separate prompt.** When a group chat is active, the Settings panel shows a **Group Extraction Prompt** instead of the 1:1 prompt. The label changes to "Extraction prompt (group chats)" so you always know which prompt you're editing. It follows the same principles but adds a `{{participants}}` list so the LLM knows who is speaking, and instructs it to attribute memories to specific characters by name. The two prompts are completely independent — customizations to the 1:1 prompt are not inherited by the group prompt, and vice versa. If you want to apply the same change to both, you need to edit each one separately.
![The extraction prompt label shows "(group chats)" when in a group chat](images/17-group-prompt-title.png)
---
## Recommended Models
Memory extraction is a structured task — the LLM needs to follow instructions precisely, distinguish between existing and new content, and produce well-formatted output. Not all models are equally good at this.
### What matters most
1. **Instruction following**: The LLM must respect the AVOID list, past-tense requirement, and the boundary between existing memories and new chat content. Weaker models blur these boundaries and contaminate new extractions with rephrased existing memories.
2. **Factual accuracy**: The LLM must not reverse actions (e.g., "A did X to B" when B did X to A) or hallucinate events.
3. **Structured output**: The LLM must produce well-formed `<memory>` blocks with bulleted lists. Models that struggle with formatting produce unparseable output.
### Good choices
| Model | Notes |
|-------|-------|
| **GLM 4.7** | Best quality and fastest. Concise, significant memories. Recommended first choice. On NVIDIA, this model uses reasoning tokens — set Max response length to 2000–3000 (see below). On NanoGPT, it works at default settings. |
| **DeepSeek V3.1 / V3.2** | Good instruction following. Solid second choice. |
| **Mistral Large 3** | Good quality, sometimes verbose. |
| **GPT-4.1 nano / mini** | Reliable instruction following at low cost. |
| **Hermes 4 (405B)** | Good with roleplay-adjacent content, won't refuse. |
| **Llama 3.1 8B Instruct** | Fast and cheap. Works well on NVIDIA. Good for testing. |
### Reasoning/Thinking models
Some providers serve models with "thinking" or "reasoning" enabled by default (e.g., GLM-4.7 on NVIDIA). These models spend part of their token budget on internal reasoning before producing the actual output. CharMemory handles this transparently — it reads the reasoning output when the content field is empty. However, you need to increase **Max response length** to 2000–3000 so the model has enough budget for both reasoning AND the actual memory output. If you see "0 memories" with a thinking model, this is almost certainly the fix.
The verbose Activity Log will show `[reasoning: N chars]` when a model uses reasoning tokens, so you can tell at a glance what's happening.
**Disabling reasoning**: Some APIs let you turn off thinking mode via a request parameter. For GLM-4.7 on NVIDIA, you can try putting this in CharMemory's **System prompt** field in provider settings:
```
"thinking": { "type": "disabled" }
```
This may allow the model to use its full token budget for memory output instead of reasoning. Results may vary — if it doesn't help, increase the response length instead.
### Models to avoid
| Model | Issue |
|-------|-------|
| **Qwen3-235B** | Tends toward compressed play-by-play even with the improved prompt. |
| **Very small models** | May reverse who did what or blur the boundary between existing and new memories. |
| **Heavily censored models** | May refuse to extract from mature content, returning nothing even when there are real events to capture. |
---
## Memory File Format
CharMemory stores memories as plain markdown files in the character's Data Bank. Understanding the file format is useful if you want to edit memories manually, migrate existing files, or troubleshoot.
### Structure
Each extraction produces a `<memory>` block with chat attribution and timestamped bullet points:
```
<memory chat="2026-02-15@10h00m00s" date="2026-02-15 14:30">
- Alex mentioned they work from home as a freelance designer.
- Flux knocked a coffee mug off the desk and showed no remorse.
- Alex adopted Flux from a rescue shelter two years ago.
</memory>
<memory chat="2026-02-15@10h00m00s" date="2026-02-15 15:45">
- Alex discovered Flux has been hiding treats behind the couch cushions.
- Flux rode the Roomba around the apartment for the first time.
</memory>
```
**Key details:**
- Each block is wrapped in `<memory>` tags with `chat` (the chat filename) and `date` (extraction timestamp) attributes
- Bullets start with `- ` (dash space) — this is the only recognized format
- Multiple blocks from the same chat can optionally be merged (see "Merge extraction chunks" in Settings). This is off by default to keep blocks smaller for consolidation
- The file is append-only during normal operation — new extractions add blocks at the end
- Old files using the `## Memory N` heading format are auto-migrated on first read
### Chunk Boundary Settings
Vector Storage chunks your memory file on `\n\n` (double newline) boundaries. By default, each `<memory>` block is one chunk. If your memory file is large, you may want more granular chunking so Vector Storage retrieves individual bullets instead of entire blocks.
Open **Settings** → **Memory File Format**:
| Setting | Behavior |
|---------|----------|
| **Block-level** (default) | One chunk per `<memory>` block. Original behavior — unchanged from previous versions. |
| **Bullet-level** | Each bullet gets its own chunk. `<memory>` tags are preserved for round-trip safety, but `\n\n` is inserted between bullets so Vector Storage chunks them individually. |
| **Custom separator** | Blocks separated by a custom string (e.g., `\n---\n`). Use this if your Vector Storage chunking strategy uses a different boundary. |
**Include metadata in chunks**: When using bullet-level or custom chunking, enable this to prefix each bullet with `[date | chat_id]` so standalone vector chunks retain their provenance (which chat they came from and when they were extracted).
![Memory File Format settings](images/20-format-settings.png)
When you change the chunk boundary setting, CharMemory offers to **reformat** the existing memory file to match. This re-reads, re-parses, and re-serializes all memories with the new format. You can decline to keep the existing file as-is.
After reformatting, **purge vectors and revectorize** the file in Vector Storage so the index reflects the new chunk boundaries.
### Working with Existing Memory Files
CharMemory auto-detects existing `*-memories.md` files in a character's Data Bank. If you already have a memory file from manual notes or another tool, CharMemory will find and use it automatically rather than creating a duplicate — as long as the filename ends in `-memories.md`.
For CharMemory to parse the contents, the file needs to be in the `<memory>` block format. The easiest way to convert existing files is the [Convert / Import](#convert--import) tool in the **Tools** tab — it detects the format automatically and lets you preview and edit the result before saving.
To convert manually, wrap your text like this:
```
<memory chat="imported" date="2026-01-01">
- First memory bullet
- Second memory bullet
</memory>
```
Any text outside `<memory>` blocks is ignored by the Memory Manager and won't appear in diagnostics. It won't cause errors, but it also won't be managed by CharMemory.
After converting existing files or making manual edits, **purge vectors and revectorize** the file in Vector Storage so the index reflects the updated content. Vector Storage doesn't incrementally update — it re-chunks and re-embeds the entire file from scratch when you revectorize.
---
## Troubleshooting
**"0 memories" after extraction**: Check the Activity Log (Tools → Activity Log). It shows exactly what happened — whether the LLM returned NO_NEW_MEMORIES, produced unparseable output, or encountered an error. Enable **Verbose** mode to see the full prompt and response. If verbose mode shows `finish=length` with completion tokens used but 0 chars content, you're using a reasoning/thinking model that needs a higher **Max response length** — increase it to 2000–3000.
**Memories extracted but character doesn't use them**: Vector Storage isn't set up, or "Enable for files" isn't checked. Open Diagnostics and verify the Vectorization line shows "Yes" and that Injected Memories shows entries after generating a message. The [Injection Health Score](#injection-health-score) in the stats bar will show RED if Vector Storage isn't configured — click the dot for details.
**Extraction never fires automatically**: Check that "Enable automatic extraction" is checked, the message counter is actually incrementing (visible in the stats bar), and the cooldown timer isn't blocking it.
**"No unprocessed messages" on Extract Now**: All messages have been processed. Click **Reset Extraction State** first to re-read from the beginning, then **Extract Now** again.
**Duplicate or overlapping memories in the memory file**: The extraction prompt includes existing memories as reference and instructs the LLM not to repeat them. If duplicates still appear, use **Consolidate** to merge them — review the preview before applying.
**Duplicate memories in injected content**: If the same memory bullet appears multiple times when injected (check the [Injection Viewer](#injection-viewer) on a character message), this is a Vector Storage chunking issue — not an extraction issue. A `<memory>` block is landing on a chunk boundary and getting split, so both halves contain overlapping bullets. Fix: increase **chunk overlap** (15% recommended) and adjust **chunk size** so blocks fit cleanly. After changing settings, purge vectors and revectorize. The health score flags this automatically as a YELLOW warning.
**Memories contain facts from existing memories, not from the chat**: The model is too weak to respect the boundary markers. Switch to a larger model (DeepSeek V3.1+, GLM 4.7).
**Memories reverse who did what**: Same issue — model too small for accurate comprehension. Use a larger model.
**Memories are too sparse from a long existing chat**: This is expected when batch-extracting hundreds of turns at once. The LLM only sees one chunk at a time and can't judge significance across the full conversation. CharMemory works best when extracting incrementally as you chat. For existing chats, try increasing "Messages per LLM call" to 40–50, and review the extraction prompt setting — the "Messages per LLM call" slider is the one that controls how much the LLM sees, not the extraction interval (which only controls how often auto-extraction fires).
**Memories are too detailed / play-by-play**: The default prompt handles this with an 8-bullet cap and negative examples. If you still see play-by-play, try increasing "Messages per LLM call" to give the LLM more context per call.
**Memories contain system metadata, relationship metrics, or image prompts**: The extension strips code blocks, markdown tables, `<details>` sections, and HTML tags before sending messages to the LLM. If metadata still leaks through, customize the AVOID section in the extraction prompt.
---
## Technical Reference
### How It Works
The extension listens for `CHARACTER_MESSAGE_RENDERED` events and counts character messages. When the interval is reached and cooldown has elapsed, it:
1. Determines **memory targets** — in a 1:1 chat, this is one character; in a group chat, one per active group member
2. Collects unprocessed messages in chunks (up to "Messages per LLM call" per chunk)
3. Strips non-diegetic content (code blocks, markdown tables, `<details>` sections, HTML tags) from messages before sending
4. For each chunk, loops through every target: reads that character's existing memories, sends both to the LLM with an extraction prompt, and appends any new `<memory>` blocks to that character's file
5. In group chats, the extraction prompt includes a participant list so the LLM can attribute memories correctly
6. If the LLM returns `NO_NEW_MEMORIES` for a target, skips the update for that character
7. Advances the extraction pointer after each chunk completes (shared across all targets)
8. Optionally merges memory blocks from the same chat into a single block (off by default — enable "Merge extraction chunks" in Settings)
9. Users can optionally consolidate memories manually using the Consolidate button (with preview and undo)
### Revectorization
Vector Storage does **not** incrementally update — when you revectorize a file, it re-chunks and re-embeds the entire file from scratch. This means:
- **After consolidation**: The memory file changes but the vector index is now stale. Revectorize so the index reflects the consolidated content.
- **After format migration**: If the extension auto-migrates an old format file, the vectorized chunks still contain the old format. Purge vectors and revectorize.
- **After manual edits**: If you edit the memory file directly, revectorize to update the index.
**Always purge vectors before revectorizing** to ensure stale chunks from the old format are fully removed.
### NanoGPT-Specific Settings
When NanoGPT is selected as provider, additional filter checkboxes appear above the model dropdown:
| Filter | Description |
|--------|-------------|
| **Subscription** | Models included in your NanoGPT plan |
| **Open Source** | Open-source models |
| **Roleplay** | Models suited for storytelling/roleplay |
| **Reasoning** | Models with reasoning capability |
Multiple filters combine as intersection (all checked filters must match). NanoGPT models are grouped by their upstream provider in the dropdown.
### What This Extension Does NOT Do
- Does not manage lorebooks (use SillyTavern's built-in World Info for that)
- Does not inject memories into the prompt directly (relies on Vector Storage)
- Does not require any external services beyond an LLM API
---
## Installation (Development)
For development or contributing:
### Option A: Symlink
```bash
ln -s /path/to/sillytavern-character-memory \
/path/to/SillyTavern/public/scripts/extensions/third-party/CharMemory
```
### Option B: Clone into SillyTavern
```bash
cd /path/to/SillyTavern/public/scripts/extensions/third-party
git clone https://github.com/bal-spec/sillytavern-character-memory CharMemory
```
Restart SillyTavern after installation.
![Data Bank browser showing memory files for multiple group members](images/databank-group.png)

222
docs/architecture.md Normal file
View file

@ -0,0 +1,222 @@
# Architecture
A technical overview for anyone who wants to understand how CharMemory works or modify it.
---
## How it fits into SillyTavern
CharMemory is a third-party extension loaded via `manifest.json`. It runs entirely in the browser as an ES module — no server-side component. It uses three SillyTavern systems:
- **Data Bank** — file storage per character (where memory files live)
- **Vector Storage** — embedding and retrieval (indexes Data Bank files automatically)
- **Event system** — hooks into chat events like `CHARACTER_MESSAGE_RENDERED` and `CHAT_CHANGED`
The extension writes memories as plain markdown files to the Data Bank. Vector Storage picks them up, vectorizes them, and retrieves relevant chunks at generation time. CharMemory doesn't handle retrieval — it only writes; SillyTavern handles the rest.
---
## File structure
| File | Lines | Role |
|------|------:|------|
| `index.js` | ~8660 | Main runtime: extraction pipeline, LLM dispatch, all UI (modals, drawers, sidebar), event handlers, provider API calls |
| `lib.js` | ~520 | Pure utility functions: parsing, serialization, text processing, format detection. Canonical source — `index.js` imports these at runtime |
| `editor.js` | ~110 | Memory block editor factory. Pure state management with undo stack — no DOM. Used by preview dialogs |
| `settings.html` | ~100 | Sidebar dashboard HTML: stats bar, tool buttons, activity log, diagnostics summary |
| `style.css` | ~2160 | All styling: dashboard, modals, drawers, wizard, troubleshooter, memory cards, token budget |
| `manifest.json` | 12 | Extension metadata: version, load order, entry points |
### Module relationships
```
index.js (runtime, UI, LLM dispatch)
├── imports lib.js (pure functions: parse, serialize, format, strip, find/replace)
├── imports editor.js (state management for preview editors)
├── loads settings.html (sidebar dashboard)
└── loads style.css (all styling)
```
The split is intentional for testability: `lib.js` functions are pure (no DOM, no SillyTavern globals) and can be unit tested in isolation. `editor.js` manages state without rendering. `index.js` handles the messy integration with SillyTavern's DOM, popups, and APIs.
---
## index.js map
The file is large. This section-by-section map helps you navigate it. Line numbers are approximate — they shift as the file evolves.
| Lines | Section | What's there |
|------:|---------|-------------|
| 1–80 | **Constants & state** | Module metadata, `getMemoryFileName()`, global state variables (`inApiCall`, `lastExtractionTime`, etc.) |
| 82–125 | **Activity log** | `logActivity()`, in-memory log array (max 500 entries), sidebar display updates |
| 125–310 | **Default prompts** | `defaultExtractionPrompt`, `defaultGroupExtractionPrompt`, `defaultConversionPrompt` — the full prompt templates |
| 309–510 | **Provider & settings constants** | `EXTRACTION_SOURCE` enum, `PROVIDER_PRESETS` registry (12 providers), `defaultSettings`, `PROMPT_CONFIG` |
| 512–1052 | **Format & conversion** | `getFormatOptions()`, `serializeMemories()`, format detection, LLM-powered conversion, conversion preview dialog |
| 1053–1390 | **Provider UI helpers** | Model dropdowns, NanoGPT filters, provider settings display, chunk boundary UI |
| 1391–1715 | **Settings init & dashboard** | `loadSettings()`, health polling, `updateStatusDisplay()`, cooldown timer, character/group helpers |
| 1717–1890 | **Data Bank operations** | `readMemoriesForCharacter()`, `writeMemoriesForCharacter()`, attachment management, `collectRecentMessages()` |
| 1890–1955 | **Server API helpers** | `fetchCharacterChats()`, `fetchChatMessages()` |
| 1958–2450 | **Provider API** | `callLLM()` (single dispatch point), `generateOpenAICompatibleResponse()`, `generateAnthropicResponse()`, model fetching, connection testing |
| 2452–2838 | **Extraction core** | `buildExtractionPrompt()`, `extractMemories()` — the main pipeline |
| 2840–2940 | **Event handlers** | `onCharacterMessageRendered()`, `onChatChanged()` |
| 2942–3380 | **Health & diagnostics** | `computeHealthScore()` (8-point checklist), `updateHealthIndicator()`, vectorization status checks |
| 3382–4015 | **Settings modal** | `showSettingsModal()` — Connection, Extraction, Storage, Advanced sections |
| 4016–4520 | **Prompts modal** | `showPromptsModal()` — 4-tab prompt editor with version tracking |
| 4523–5300 | **Setup wizard** | `showSetupWizard()` — 3-step first-run flow |
| 5306–5935 | **Troubleshooter** | `showTroubleshooter()` — health checks, Data Bank browser, diagnostic report |
| 5938–6500 | **Memory Manager** | `showMemoryManager()` — unified block editor with character picker for groups, inline editing, undo, find/replace |
| 6500–6590 | **Find & Replace** | `buildFindReplaceBar()`, `wireFindReplaceEvents()`, `highlightText()` |
| 6344–6820 | **Consolidation** | Strategy presets (conservative/balanced/aggressive), LLM consolidation, preview dialog, undo |
| 6821–7138 | **Reformat** | Format migration preview dialog, LLM-powered reformat, undo |
| 7139–7355 | **Slash commands & wiring** | `registerSlashCommands()`, `setupToolControls()`, `setupLogControls()`, `setupListeners()` |
| 7360–8000 | **Per-message UI & Injection Viewer** | Extract Here / Pin Memory / View Injected buttons, injection drawer with Context/Prompt Breakdown/token budget panel, log drawer |
| 8000–8210 | **Batch extraction** | `showBatchPopup()`, `runBatchExtraction()` — multi-chat extraction |
| 8210–8660 | **Initialization** | Extension entry point: loads HTML, calls `loadSettings()`, wires events, auto-triggers wizard on first run |
---
## Core data structure
Everything revolves around **memory blocks**:
```javascript
{ chat: "main_chat_abc123", date: "2025-01-15 14:30", bullets: ["Alex ordered coffee", "Discussed the weather"] }
```
Serialized as XML in the Data Bank file:
```xml
<memory chat="main_chat_abc123" date="2025-01-15 14:30">
- Alex ordered coffee
- Discussed the weather
</memory>
```
Parsing is strict: only `<memory>` tags with `- ` bullet lines are recognized. The first bullet in a block is typically a **topic tag** like `[Alex, Flux — coffee shop visit]` which improves Vector Storage's embedding discrimination for thematically similar memories.
Key functions: `parseMemories()` and `serializeMemories()` in `lib.js` handle the conversion. `index.js` has its own `serializeMemories()` that adds runtime format options.
---
## Extraction pipeline
The core data flow, end to end:
```
New message arrives
→ CHARACTER_MESSAGE_RENDERED event fires
→ onCharacterMessageRendered() increments counter
→ Counter hits interval (default 20) and cooldown has expired
→ extractMemories() called
→ Collect unprocessed messages via collectRecentMessages()
→ Chunk into batches (default 20 messages/call)
→ For each chunk: callLLM() with extraction prompt + existing memories + recent messages
→ Parse <memory> blocks from LLM response
→ Append to Data Bank file via writeMemoriesForCharacter()
→ Update chat_metadata.lastExtractedIndex
→ Vector Storage auto-vectorizes the updated file (async, not controlled by us)
→ At generation time, VS retrieves relevant chunks and injects them into the prompt
```
`callLLM()` (line ~2265) is the **single dispatch point** for all LLM calls — extraction, consolidation, and conversion all go through it. It branches on `extension_settings.charMemory.source` to route to the Main LLM, WebLLM, or a dedicated provider.
---
## Provider system
Twelve providers in the `PROVIDER_PRESETS` registry (line ~315). Each preset defines:
- `baseUrl` — API endpoint
- `authStyle` — `'bearer'`, `'x-api-key'`, or `'none'`
- `modelsEndpoint` — `'standard'` (/models), `'custom'` (NanoGPT), or `'none'` (Pollinations)
- `requiresApiKey`, `extraHeaders`, `defaultModel`
All providers except Anthropic use `generateOpenAICompatibleResponse()` which POSTs to `/chat/completions`. Anthropic has its own adapter `generateAnthropicResponse()` that converts to the Messages API format.
Per-provider settings (API key, model, system prompt, custom URL) are stored independently in `extension_settings.charMemory.providers[key]`. Switching providers preserves each one's configuration.
---
## Editor factory
`createMemoryEditor({ blocks })` in `editor.js` returns a pure state API (no DOM, no rendering):
```
getBlocks(), deleteBullet(), deleteBlock(), addBullet(), addBlock(),
updateBullet(), updateTheme(), undo(), canUndo(), replaceAll(),
toggleEdit(), isEditing(), getEditingSet(),
countMatches(), findAndReplaceAll()
```
Every mutation calls `saveVersion()` first, pushing a deep clone onto the undo stack. This makes any operation (including Replace All) undoable as a single step.
Used by the Consolidation, Conversion, Reformat, Data Bank editor, and Memory Manager dialogs. All five editing surfaces share this unified editor — inline editing, undo, add/delete, and find/replace work identically everywhere.
---
## UI architecture
The UI has four layers:
**Sidebar dashboard** — static HTML in `settings.html`, wired in `loadSettings()`. Stats bar, tool buttons, activity log, diagnostics summary. All IDs prefixed `charMemory_`.
**Center-screen modals** — built dynamically in `index.js` using SillyTavern's `callGenericPopup()`. Six major modals: Settings, Prompts, Setup Wizard, Troubleshooter, Memory Manager, Batch. Settings modal uses `cm_modal_*` prefixed IDs; troubleshooter uses `cm_ts_*`.
**Slide-out drawers** — the Injection Viewer and Activity Log drawer, toggled by sidebar buttons. Support touch swipe-to-close.
**Per-message buttons** — Extract Here (brain icon), Pin Memory (bookmark), View Injected (eye). Added to each message via `onMessageRenderedAddButtons()`.
jQuery is used throughout (SillyTavern convention). Event handlers are typically wired in `setupToolControls()` and `setupLogControls()`, called from `loadSettings()`.
---
## Health system
`computeHealthScore()` (line ~3073) runs an 8-point checklist:
1. Character selected
2. Data Bank file exists
3. File is not empty
4. File format is valid `<memory>` tags
5. File is vectorized in Vector Storage
6. Vector Storage chunk size is appropriate
7. Vector Storage retrieval is configured
8. Memories are being injected into context
Returns `{ level: 'green'|'yellow'|'red', checks: [...] }`. Displayed as a traffic-light dot in the sidebar. Updated from six trigger points: init, chat change, after extraction (with 4s recheck for VS race condition), 60s periodic poll (pauses when tab hidden via Page Visibility API), panel open, and troubleshooter close.
---
## Testing
Three tiers, all using Vitest:
```bash
npm test # Unit tests (~136 tests, <1s) — pure functions in lib.js + editor.js
npm run test:snapshot # Snapshot tests (6 tests) — extraction pipeline against 1000-message fixture
npm run test:live # Live LLM tests (3 tests) — real LLM call, validates response structure
```
**Unit tests** cover parsing, serialization, escaping, format detection, find/replace, and the editor API. These are the primary safety net — run them after any change to `lib.js` or `editor.js`.
**Snapshot tests** process a real chat fixture through `stripNonDiegetic` → `formatChatMessages` → `substitutePromptTemplate` and snapshot the output. Update with `npm run test:snapshot -- --update` after intentional changes.
**Live tests** require a running OpenAI-compatible server. Configure with `TEST_LLM_URL`, `TEST_LLM_MODEL`, `TEST_LLM_KEY` env vars. Good local models: Gemma 2 9B, Qwen 2.5 7B.
UI and integration testing requires a running SillyTavern instance with the extension installed.
---
## Common modifications
**Adding a new provider** — Add an entry to `PROVIDER_PRESETS` (~line 315). If it's OpenAI-compatible with a standard `/models` endpoint, that's the only change needed. The settings UI, model fetching, and API dispatch all key off the preset.
**Modifying the extraction prompt** — Edit the `defaultExtractionPrompt` constant (~line 125). Bump `PROMPT_CONFIG.extraction.version` to trigger update notifications for existing users. Test against multiple characters — the prompt is carefully tuned to reduce card-trait leakage and meta-narration.
**Adding a sidebar UI element** — Add HTML to `settings.html`, add the event handler in `setupToolControls()` (~line 7189). Follow the `charMemory_` ID prefix convention.
**Adding a modal** — Build HTML dynamically in a new `show*()` function. Use `callGenericPopup(html, POPUP_TYPE.TEXT)` for display. Use `cm_modal_*` or `cm_[feature]_*` prefixed IDs to avoid conflicts with other surfaces.
**Adding a new tool dialog** — Follow the pattern of existing tools (Consolidation, Reformat): build dialog HTML, create a `createMemoryEditor()` instance for state management, render with `renderConsolidatedCards()`, wire find/replace with `wireFindReplaceEvents()`. The editor handles undo; you handle rendering and save.
**Adding a pure utility function** — Add it to `lib.js` and export it. `index.js` imports it automatically. Keep `lib.js` free of DOM and SillyTavern dependencies so it stays unit-testable.

146
docs/getting-started.md Normal file
View file

@ -0,0 +1,146 @@
# Getting Started
After installing CharMemory, the **Setup Wizard** opens automatically on your first visit. It walks you through the two things you need before memories work: an LLM to extract them, and Vector Storage to retrieve them.
If the wizard didn't open automatically, click the **wand icon** (✦) in the CharMemory panel header at any time.
![CharMemory panel header showing the wand, gear, and wrench icons](../images/panel-header.png)
---
## Step 1: Connect an LLM
CharMemory needs its own LLM connection — separate from your main chat. This keeps the extraction prompt clean and uncontaminated by chat personas, jailbreaks, or system prompts.
![Setup Wizard step 1 — provider dropdown, API key field, Connect button](../images/wizard-step1.png)
**1. Choose a provider** from the dropdown, e.g. **NanoGPT**. If you're not sure, **Pollinations** is free and requires no API key. See [Providers](providers.md) for a full list with model recommendations.
**2. Enter your API key** if the provider requires one. Click the **(get key)** link for a direct link to that provider's key page.
**3. Click Connect.** CharMemory fetches the available models for that provider.
**4. Select a model.** Type to search — especially useful for providers with many models. Not sure which? **GLM 4.7** and **DeepSeek V3.1** are reliable extraction models.
**5. Click Test Connection** to confirm everything works. On success, the wizard enables the Next button.
> **Running an LLM locally?** Select **Local Server** from the provider dropdown. Enter your server URL (e.g., `http://localhost:11434/v1` for Ollama). No API key needed. See [Providers → Local Servers](providers.md#local-servers) for port numbers by backend.
> **If your provider is not listed** Many providers have an OpenAI compatible API endpoint. See if you can configure it that way.
![Wizard step 1 after successful connection — green checkmark, model selected](../images/wizard-step1-connected.png)
---
## Step 2: Configure
![Wizard step 2 — extraction interval and Vector Storage health check](../images/wizard-step2.png)
### Extraction interval
Set how many character messages trigger an automatic extraction. The default is **20 messages** — a good starting point for most chat styles. Lower means more frequent extractions; higher means the LLM gets more context per call and tends to produce better, more selective memories.
### Vector Storage check
The wizard checks your Vector Storage extension configuration and shows one of three states:
**Green — Ready.** Vector Storage is enabled and settings look good. Nothing to do.
**Yellow — Needs tuning.** Vector Storage is active but you should examine some of the settings as they may reduce retrieval quality (e.g., chunk size too small, score threshold not set). The wizard shows specific recommendations but ultimately this is a trial and error process. You can continue and fix these later — see [Retrieval & Prompts](retrieval-and-prompts.md) for guidance.
**Red — Not enabled.** Vector Storage is installed but the "Enable for files" checkbox is off in the Vector Storage extension. Memories will be stored but your character won't recall them during chat.
> To fix the red state: open **Extensions → Vector Storage** → under **File vectorization settings**, check **Enable for files**. Then come back and click Re-check. New to Vector Storage? See SillyTavern's [Data Bank (RAG)](https://docs.sillytavern.app/usage/core-concepts/data-bank/) documentation for a full explanation.
The wizard won't block you — you can click Next even with warnings. But Vector Storage is what makes memories actually show up in conversation, so it's worth setting up before you start chatting.
---
## Step 3: Review & Go
![Wizard step 3 — summary card with provider, model, and status](../images/wizard-step3.png)
The final step shows a summary of your configuration:
- **Provider and model** — what will run extraction
- **Connection** — whether the test passed
- **Extraction** — how often memories will be extracted
- **Vector Storage** — whether retrieval is ready
If anything looks wrong, click **Back** to fix it.
### Existing memories
If the current character already has a Data Bank memory file, the wizard offers to convert it to the current format (topic-tagged blocks for better retrieval). You can do this now or skip it — the same operation is available at any time via the **Reformat** button in the Data Bank Tools section. See [Managing Memories → Reformat](managing-memories.md#reformat) for details.
Click **Get Started** to close the wizard and return to the dashboard.
---
## Back up your data
SillyTavern has built-in backup tools that snapshot your entire data directory — characters, chats, settings, and Data Bank files (where memories live). Before making big changes like consolidating, clearing memories, or switching setups, it's worth having a recent backup.
See SillyTavern's [User Settings documentation](https://docs.sillytavern.app/usage/user-settings/) for how to create and manage backups.
---
## Your first extraction
After setup, chat normally. The **stats bar** at the top of the CharMemory panel tracks your progress:
![Stats bar showing file name, memory count, progress, and health dot](../images/stats-bar.png)
| Item | What it shows |
|------|---------------|
| **File** | Memory file name for this character (e.g., `Flux-memories.md`) |
| **Memories** | Total memory bullets stored |
| **Progress** | Messages since last extraction vs. the threshold (e.g., `3/20 msgs`) |
| **Status** | `Ready` when extraction can fire, or a countdown timer during cooldown |
| **Health dot** | Green = injection healthy / Yellow = warnings / Red = problems. Click for details. |
When the counter reaches the threshold, extraction fires automatically. You can also trigger it manually at any time:
- **Extract Now** (button in the panel) — processes all unprocessed messages immediately
- **Extract Here** (brain icon on any character message) — processes messages up to that specific point
![Brain icon on a character message](../images/brain-icon-message.png)
Extraction speed will vary based on the number of messages being sent to the LLM and its processing time. A toast notification provides status updates and how many memories were saved. You can watch the steps in the **Activity** section of the dashboard, or click **View full log** for the detailed log.
---
## Viewing your memories
There are a few ways to view your memories — you can open the Data Bank directly and read the raw file, or use the Memory Manager for a more structured view. Click **View / Edit** in the CharMemory panel to open the Memory Manager.
![Memory Manager showing memory cards with find/replace bar and Save/Cancel buttons](../images/memory-manager.png)
Memories appear as cards grouped by extraction, newest first. Each card shows the extraction timestamp and the individual memory bullets. Click the **pencil icon** on any card to enter edit mode — bullets become editable text fields where you can modify, delete, or add new memories. The editor includes **find/replace**, **undo**, and explicit **Save/Cancel** buttons so nothing is written to disk until you confirm. In group chats, a **character picker** lets you switch between members.
Memories are stored as a plain markdown file in the character's Data Bank. You can also edit that file directly from the **Data Bank** button in the panel. See [Managing Memories](managing-memories.md) for the full editing reference.
---
## Verifying memories are being injected
After your first extraction, generate a few more messages. Then click the **health dot** in the stats bar — if it's green, memories are being retrieved and injected correctly.
For a deeper look, open the **Injection Viewer** (syringe icon in the panel header) to see exactly which memories were injected for the latest message. See [Injection Viewer](injection-viewer.md) for how to read it.
If the health dot is yellow or red, see [Troubleshooting](troubleshooting.md).
> There are many ways for memories to either not be extracted well or not injected well. Much of this is down to extraction prompts, chunking and Vector Storage settings. It is impossible to provide out of the box functionality that will work for everyone, so you may need to spend some time adjusting to suit your particular chats and characters.
---
## Recommended next steps
| I want to… | Go to |
|-----------|-------|
| Understand what's being injected into the prompt | [Injection Viewer](injection-viewer.md) |
| Use CharMemory in a group chat | [Group Chats](group-chats.md) |
| Tune extraction frequency | [Managing Memories](managing-memories.md) |
| Change LLM provider or model | [Providers](providers.md) |
| Fix "memories stored but not recalled" | [Retrieval & Prompts](retrieval-and-prompts.md) |
| Something isn't working | [Troubleshooting](troubleshooting.md) |

70
docs/group-chats.md Normal file
View file

@ -0,0 +1,70 @@
# Group Chats
CharMemory works in group chats with no extra setup. Each group member gets their own memory file, extraction handles all members in a single pass, and the Data Bank browser lets you view and manage every member's memories from one place.
![CharMemory panel in a group chat](../images/panel-group.png)
---
## How extraction works
When extraction fires in a group chat — automatically or via **Extract Now** — CharMemory processes each chunk of messages once per group member. For each member it:
1. Reads that character's existing memories
2. Builds an extraction prompt with the character card and a participant list so the LLM knows who is speaking
3. Sends the chunk to the LLM
4. Appends any new memories to that character's file
Progress shows which character is being processed (e.g., "Flux (2/3)"). If the LLM call fails for one member, extraction continues with the remaining members — one failure won't abort the whole group.
The group chat uses a **separate extraction prompt** from 1:1 chats. It follows the same principles but adds a `{{participants}}` list and instructs the LLM to attribute memories to specific characters by name. You can customize it independently — changes to the 1:1 prompt aren't inherited by the group prompt, and vice versa.
---
## Viewing and editing group memories
Click **View / Edit** to open the Memory Manager. In a group chat, a **character picker** appears first — select which member's memories to view and edit. Memory cards for the selected character are displayed with the same editing interface as in 1:1 chats (inline editing, undo, find/replace, Save/Cancel).
Newest memory blocks appear first (reverse chronological). Switch between characters using the picker at the top to review or edit each member's memories.
---
## Data Bank browser
Click **Data Bank** in the panel to browse and manage memory files for all group members at once. Each member's file is listed with its memory count. You can view, edit, or delete files for any member without switching characters.
![Data Bank browser showing group members' memory files](../images/databank-group.png)
This is particularly useful for group chats because SillyTavern doesn't provide a built-in way to open the Data Bank for characters other than the active one — the CharMemory browser fills that gap.
---
## Consolidation
The **Consolidate** button in a group chat shows a character picker — select which member's memories to consolidate. Consolidation works on one character at a time to keep the preview manageable. Undo restores that character's previous file.
---
## Pin Memory
The bookmark icon on a group message routes the pinned memory to the correct character's file based on the message sender. If the sender can't be matched to a group member (e.g., a narrator message), it goes to the first member.
---
## Reset and Clear
Both options are available in **Settings** (gear icon → Reset / Clear) and in the **Troubleshooter** (wrench icon).
**Reset Extraction State** in a group chat resets the extraction pointer for all group members simultaneously. This is because SillyTavern stores `lastExtractedIndex` in the group's shared chat metadata — not per character.
**Clear All Memories** deletes memory files for all group members in the current group.
---
## How retrieval works in group chats
During generation, SillyTavern sets the active character to whichever group member is about to speak. Vector Storage retrieves memories from that character's Data Bank and injects them into the prompt. Each character gets their own memories when it's their turn — Flux gets Flux's memories, Alex gets Alex's. This is why it is necessary to have a specific extraction prompt for group chats.
**Re-vectorization caveat**: If you edit a character's memory file, you need to re-vectorize it to update the search index. SillyTavern's Vector Storage re-vectorizes on the *active* character — so you need to switch to each group member individually to trigger re-vectorization for their file. See [Retrieval & Prompts → Purge and re-vectorize](retrieval-and-prompts.md#purge-and-re-vectorize) for the steps.
**Diagnostics caveat**: After generation finishes, SillyTavern resets the active character. If you open Diagnostics between generations, the "Injected Memories" section may appear empty — there's no character context at that moment. This doesn't mean memories weren't injected, just that the snapshot was taken outside a generation turn. Use the [Injection Viewer](injection-viewer.md) on a specific message instead.

96
docs/injection-viewer.md Normal file
View file

@ -0,0 +1,96 @@
# Injection Viewer
The Injection Viewer is a sidebar that shows exactly what context was injected into the LLM's prompt for any character message — which memories, which lorebook entries, and what other extensions contributed. It's the fastest way to answer "did my character actually know about X when they wrote that?"
![Injection Viewer open beside a chat showing memory bullets and count](../images/injection-viewer.png)
## Opening the viewer
Click the **syringe icon** in the CharMemory panel header to show or hide the viewer sidebar. The sidebar docks to the right side of the chat area and stays open as you scroll through the conversation.
Once open, click the **pen icon** on any character message to load that message's injection data. The header updates to show which message you're viewing (e.g., "Message #42").
![Brain icon on a character message](../images/brain-icon-message.png)
> Only **character messages** have injection data — user messages don't trigger a generation so there's nothing to capture. Messages generated before CharMemory was installed will show a placeholder.
---
## What the viewer shows
The viewer has four collapsible sections:
### Context
The **Context** section appears at the top and loads automatically when you open a message's injection data. It shows a **Prompt Breakdown** — a per-category token breakdown of the full prompt that was sent to the LLM:
- **System** — system prompt tokens
- **Char card** — character description and personality
- **Lorebook** — World Info / lorebook entries that fired
- **Data Bank** — CharMemory's injected memories
- **Examples** — example dialogue
- **Chat history** — the actual conversation messages
When SillyTavern's Prompt Itemization data is available (current session), the numbers are exact token counts. For snapshots from previous sessions, the section shows estimated counts based on character length (~4 chars/token) with a note that the numbers are approximate.
A compact **stacked bar** visualizes the proportions at a glance, and a **"Tips"** link opens actionable guidance for reducing token usage across each category.
If tracked injections exceed the model's context window, a **red health note** flags the overflow. If they exceed 40% of context (leaving little room for char card and chat history), a **yellow advisory** appears.
### Data Bank
The memory bullets that Vector Storage retrieved and injected for this generation. The header shows a count and estimated token cost (e.g., "Data Bank (6) ~120 tk"). Each bullet is listed individually — this is the direct answer to "what did the character remember when writing this message?"
If this section is empty but memories exist in the file, Vector Storage either filtered them out (score threshold too high) or didn't find a close enough semantic match. See [Retrieval & Prompts](retrieval-and-prompts.md) for how to tune retrieval.
### Lorebook Entries
Which World Info / lorebook entries fired for this generation, based on their trigger keywords matching the recent conversation. Each entry shows its name, trigger keys, estimated token cost (~N tk), and a content preview.
If an entry you expected to see isn't listed, its keywords didn't match the recent context window. This section is useful for understanding the full picture — memories and lorebook entries both contribute to what the character "knows," and they can complement or conflict with each other.
### Extension Prompts
Content injected by all active extensions, keyed by injection position (e.g., `4_vectors_data_bank` for Vector Storage, `2_floating_prompt` for Author's Note). Each extension's entry shows its estimated token cost and injection position/depth (e.g., `~340 tk · in-chat @ depth 2`). This is the unprocessed view — useful for seeing the exact `<memory>` block markup and chunk boundaries that the LLM received.
---
## The health dot
The viewer header includes a colored health dot that mirrors the one in the stats bar:
| Color | Meaning |
|-------|---------|
| **Green** | All health checks passed |
| **Yellow** | Warnings — things work but could be improved |
| **Red** | Problems detected — memories may not be injecting correctly |
| **Gray** | No generation captured yet for this message |
Hover over the dot for a quick summary. Tap the **Diagnostics** link in the viewer toolbar to see the full health check results inline without leaving the chat. On tablets and phones (or when Display Mode is set to Tablet/Phone in Settings > Advanced), the viewer opens as a wide drawer above the sidebar for easier reading.
For details on what each check looks for and how to fix issues, see [Troubleshooting → Health Checks](troubleshooting.md#health-checks).
---
## How data is captured
Injection data is captured automatically at generation time — CharMemory takes a snapshot of all injected context the moment the character produces a response. A few things to know:
- **Data persists across page refreshes** — snapshots are stored in chat metadata, so they survive reloads within the same chat
- **Only messages generated while CharMemory is active have data** — messages from before installation show a placeholder
- **Group chats**: each character message captures data for that specific character's generation turn
---
## Injection Viewer vs. Diagnostics
Both tools inspect injected context, but they serve different purposes:
| | Injection Viewer | Diagnostics |
|---|---|---|
| **Scope** | Any past message — inspect any generation | Latest generation only |
| **Detail** | Parsed sections — memories, lorebooks, prompts | Full system overview — file status, vectorization, health checks |
| **Best for** | "What did the character know when it wrote *this*?" | "Is my setup working correctly?" |
| **Access** | Pen icon on any character message | Wrench icon → Diagnostics |
Use the Injection Viewer to spot-check individual messages and compare what changed between generations. Use Diagnostics to verify your overall setup — file vectorized, settings correct, memories being injected at all.

193
docs/managing-memories.md Normal file
View file

@ -0,0 +1,193 @@
# Managing Memories
This doc covers the tools for working with memories after they've been extracted: browsing and editing individual bullets, consolidating duplicates, batch-extracting from old chats, reformatting for better retrieval, and clearing or resetting when you need a fresh start.
---
## View / Edit
Click **View / Edit** in the CharMemory panel to open the Memory Manager.
![Memory Manager showing memory cards with find/replace bar and Save/Cancel buttons](../images/memory-manager.png)
Memories appear as cards grouped by extraction, newest first. Each card shows the extraction timestamp and the individual memory bullets. The Memory Manager uses the same unified block editor as Consolidation, Conversion, Reformat, and the Data Bank browser — so editing works identically everywhere. You can:
- **Edit a block** — click the pencil icon on any card header to enter edit mode. Bullets become editable text fields.
- **Delete a bullet** — click the trash icon next to any bullet in edit mode
- **Delete a block** — click the trash icon in the block header in edit mode. Removes the entire block.
- **Add a bullet** — click **+ Add memory** at the bottom of a block in edit mode
- **Add a block** — click **+ Add Block** at the bottom of the editor
- **Undo** — click **Undo** to revert the last edit operation (delete, add, or Replace All)
![Memory Manager in edit mode showing editable bullets with delete and add controls](../images/memory-manager-editing.png)
Nothing is written to disk until you click **Save**. Click **Cancel** to discard all changes. After saving, Vector Storage's index is stale — [purge and re-vectorize](retrieval-and-prompts.md#purge-and-re-vectorize) to update it, or send a new chat message.
In group chats, a **character picker** appears at the top — select which member's memories to view and edit.
### Find & Replace
A find/replace bar appears at the top of the Memory Manager (and in the Consolidation, Conversion, Reformat, and Data Bank editor dialogs). Type in the **Find** field to see matches highlighted across all cards with a live count. Enter replacement text and click **Replace All** to update every occurrence at once.
- **Case sensitive** — click the **Aa** button to toggle case-sensitive matching
- Replace All is undoable (click **Undo** to revert) — but once you click **Save**, changes are written to disk and cannot be undone
- In the Consolidation, Reformat, Conversion, and Data Bank editors, Replace All works the same way with Undo support
---
## Auto-extraction
CharMemory extracts automatically while you chat. The **Auto** pill in the panel header toggles this on or off. When on, two settings in **Settings → Extraction** control when it fires:
**Extract after every N messages** (default: 20) — how many character messages must arrive before extraction triggers. Higher values give the LLM more context per call and tend to produce better, more selective memories. Lower values extract more frequently with less context.
**Minimum wait between extractions** (default: 10 min) — a cooldown that prevents rapid-fire extractions during fast-paced chats. If the message threshold is reached but the cooldown hasn't expired, extraction is skipped and rechecked on each subsequent message. Messages keep accumulating during the cooldown, so when extraction finally fires it processes everything that built up.
Both settings only affect automatic extraction. Extract Now, Extract Here, and Batch always run immediately regardless of interval or cooldown.
---
## Extract Now and Extract Here
**Extract Now** (button in the panel) processes all unprocessed messages in the current chat immediately, without waiting for the auto-extraction threshold.
**Extract Here** (brain icon on any character message) processes all unprocessed messages up to and including that specific message. Useful for targeting a particular point in a conversation without processing everything after it.
![Brain icon on a character message](../images/brain-icon-message.png)
Both use the same provider and settings as auto-extraction. You can watch progress in the **Activity** section of the dashboard or click **View full log** for the detailed log.
> **"No unprocessed messages"**: If Extract Now reports nothing to process, all messages have already been extracted. Use **Reset Extraction State** (see [Reset and Clear](#reset-and-clear) below) to re-read from the beginning.
---
## Extraction prompt
The extraction prompt is customizable in **Settings → Prompts**. Separate prompts exist for 1:1 chats, group chats, consolidation, and conversion — changes to one don't affect the others. **Restore Default** reverts any prompt to its original.
Each prompt type has its own template variables that get substituted before the prompt is sent to the LLM:
| Prompt | Available variables |
|--------|-------------------|
| **1:1 extraction** | `{{charName}}`, `{{charCard}}`, `{{existingMemories}}`, `{{recentMessages}}` |
| **Group extraction** | `{{charName}}`, `{{charCard}}`, `{{existingMemories}}`, `{{recentMessages}}`, `{{participants}}` |
| **Consolidation** | `{{charName}}` |
| **Conversion** | `{{charName}}`, `{{sourceText}}`, `{{today}}` |
`{{charCard}}` injects the character card as a bounded "baseline knowledge — do NOT extract" section. This prevents the LLM from repeatedly re-extracting traits already defined in the card (e.g., extracting "she works as a doctor" every chat when that's already in the card). Removing `{{charCard}}` from the prompt is a valid option — for example if the card is very large and consuming too many tokens — but expect more card-trait leakage in extracted memories.
For a detailed walkthrough of the prompt design decisions and how they affect retrieval quality, see [Retrieval & Prompts → The extraction prompt](retrieval-and-prompts.md#the-extraction-prompt).
---
## Pin as Memory
The **bookmark icon** on any message manually saves it as a memory with no LLM involved. An edit dialog opens pre-filled with the message text — rewrite it however you want before saving. Each line becomes a memory bullet.
Use this when you want to remember something specific and phrased exactly your way, without waiting for the next auto-extraction.
In group chats, the pinned memory goes to the correct character's file based on the message sender.
---
## Consolidate
When the memory file grows large, related or duplicate memories accumulate across different sessions. **Consolidate** sends the full memory file to the LLM to deduplicate and merge related entries. It always requires manual review — consolidation never runs automatically.
![Consolidation preview showing original and consolidated memories side by side](../images/consolidation-preview.png)
### Strategies
Choose a strategy before consolidating:
| Strategy | What it does |
|----------|-------------|
| **Conservative** | Merges near-exact duplicates only. Preserves the most detail. |
| **Balanced** | Merges duplicates and combines closely related facts. Good default. |
| **Aggressive** | Compresses heavily, summarizes by theme. Best for very large files that need significant reduction. |
Each strategy has its own prompt, fully visible and editable. **Restore Default** reverts to the original.
### Workflow
1. Choose a strategy and click **Consolidate**
2. Results appear as editable cards organized by theme
3. Edit, delete, or add bullets before applying — click the pencil icon on any card to enter edit mode
4. Not happy with the result? Click **Re-run** for a fresh attempt. Each re-run saves the previous version — click **Undo** to step back through prior versions.
5. Click **Apply** to write consolidated memories to the file
> **Undo only works before Apply.** Once you click Apply, the consolidated memories are written to the file and there is no undo. Back up the memory file first if you're consolidating a large or important set — open the Data Bank and download the file, or use SillyTavern's [backup tools](https://docs.sillytavern.app/usage/user-settings/) before proceeding.
In group chats, a character picker appears — consolidation works on one character at a time.
---
## Batch Extraction
Batch extraction processes multiple existing chats for the current character at once, without opening each chat individually.
![Batch panel showing a list of chats with checkboxes](../images/batch-panel.png)
1. Click **Batch** in the Data Bank Tools section
2. Click **Refresh** to load the list of chats for the current character
3. Check the chats you want to process (or **Select All**)
4. Click **Extract Selected** — a confirmation shows the total message count
5. Progress updates show which chat is being processed
Each chat's extraction state is tracked independently. Re-running batch extraction only processes new messages since the last run.
**For long existing chats**: Batch extraction works best for catching up on recent chats. For very long chats (hundreds of turns), early chunks have no existing memories for context, so the LLM may extract more sparsely than it would when extracting incrementally as you chat. If results are too sparse, try increasing **Messages per LLM call** (default: 50) in Settings to give the LLM a bigger window per chunk.
---
## Reformat
**Reformat** restructures your existing memory file to the current topic-tagged format for better vector retrieval. Use it when:
- You have older memories without topic tags, or topic tags that don't include the character's name (e.g., `[Alex — description]` instead of `[Flux, Alex — description]`)
- You've imported memories from another source and want them normalized
- The Setup Wizard offered to convert existing memories and you skipped it
Click **Reformat** in the Data Bank Tools section. A preview dialog shows the before and after — review it, make any edits, then confirm to write the changes.
After reformatting, [purge and re-vectorize](retrieval-and-prompts.md#purge-and-re-vectorize) so the vector index reflects the new format.
---
## Data Bank
Click **Data Bank** to browse and manage memory files directly. You can view the raw markdown, edit it, and save changes. The editor includes find/replace and an **Undo** button — nothing is written to disk until you click **Save changes**. This is the same file that Vector Storage indexes — useful for bulk edits or importing content from another tool.
In group chats, the browser shows files for all group members, not just the active character. See [Group Chats → Data Bank browser](group-chats.md#data-bank-browser) for details.
---
## Per-chat memories
By default, all chats for a character share one memory file. Enable **Separate memories per chat** in Settings → Storage to give each conversation its own file. This is useful when the same character appears in different scenarios or timelines that shouldn't share context.
Per-chat mode also works in group chats — each member gets a separate per-chat memory file.
> **Known limitation**: Batch extraction with per-chat mode active has a bug — extracted memories from non-active chats may go to the wrong file. Use batch extraction with the default (shared) mode.
---
## Reset and Clear
Three options in Settings → Reset / Clear:
**Reset Extraction State** — resets the extraction pointer for the current chat. Next time you run Extract Now, it re-reads all messages from the beginning. Does not delete any memories. In group chats, all members share one extraction pointer so this resets all of them at once.
**Reset Batch Progress** — clears the Batch tool's record of which messages it has processed across all of this character's chats. Use this when you want the Batch tool to re-process everything from scratch (e.g., after changing the extraction prompt). Does not affect Extract Now or auto-extraction. If you reset batch progress without also clearing memories, the next batch run will re-extract everything and may create duplicates.
**Clear All Memories** — deletes the character's memory file and resets all extraction tracking. In default (shared) mode, this file contains memories from **all** of that character's chats — not just the current one. Cannot be undone. **Back up first** — use SillyTavern's [backup tools](https://docs.sillytavern.app/usage/user-settings/) or download the memory file from the Data Bank before clearing.
---
## Slash commands
| Command | Description |
|---------|-------------|
| `/extract-memories` | Force extraction regardless of interval or cooldown |
| `/consolidate-memories` | Run consolidation with the current strategy |
| `/charmemory-debug` | Capture diagnostics and dump to console |

78
docs/plans/2026-02-26-automated-testing-design.md Normal file
View file

@ -0,0 +1,78 @@
# Automated Testing Design
## Goal
Add automated testing for CharMemory's extraction pipeline using a 1000-message test chat fixture. Two tiers: deterministic snapshot tests for the processing pipeline, and live integration tests against a real LLM.
## Current State
- Vitest with 71 passing unit tests in `test/unit/` covering `lib.js` pure functions
- `lib.js` exports pure functions (parsing, serialization, escaping, format detection)
- `package.json` has `test:snapshot` and `test:live` scripts wired up but no test files
- `index.js` has extraction pipeline logic tightly coupled to SillyTavern globals
- 1000-message JSONL test chat at `/Users/davidsayed/repos/st-test-chatlog/output/`
## Design
### Step 1: Extract pure logic from `index.js` into `lib.js`
Three new functions:
**`stripNonDiegetic(text)`** — The 5 regex operations currently inline in `collectRecentMessages()` (lines 2031-2036). Removes code blocks, `<details>` sections, markdown tables, HTML tags, collapses excessive newlines.
**`formatChatMessages(chatArray, startIndex, endIndex)`** — Message filtering and formatting extracted from `collectRecentMessages()`. Takes a plain array of ST message objects, filters out empty/system-only messages, applies `stripNonDiegetic()`, returns formatted text. The caller (`collectRecentMessages` in index.js) handles reading from `getContext()` and passes the array in.
**`substitutePromptTemplate(template, vars)`** — Template variable substitution from `buildExtractionPrompt()`. Replaces `{{charName}}`, `{{charCard}}`, `{{existingMemories}}`, `{{recentMessages}}`, `{{participants}}`. The caller handles reading the template from settings and getting the character card from ST globals.
After extraction, `index.js` calls these lib.js functions. No behavior change.
### Step 2: Snapshot tests (`npm run test:snapshot`)
File: `test/integration/snapshot.test.js`
Test fixture: Copy JSONL into `test/fixtures/flux-chat.jsonl`.
Tests:
1. **stripNonDiegetic** — Feed messages containing code blocks, tables, HTML, `<details>` sections. Snapshot the cleaned output.
2. **formatChatMessages** — Load the JSONL, process chunks (messages 0-20, 20-50). Snapshot the formatted text. Verifies filtering, stripping, and formatting stability.
3. **substitutePromptTemplate** — Build a prompt using processed messages, mock character card, empty existing memories. Snapshot the final prompt. Verifies the prompt the LLM receives is correct.
4. **parseMemories round-trip** — Parse a sample LLM response fixture, re-serialize, verify no data loss.
All deterministic. Run in milliseconds.
### Step 3: Live LLM tests (`npm run test:live`)
File: `test/integration/live.test.js`
Flow: Load JSONL → formatChatMessages → substitutePromptTemplate → call LLM → parseMemories → assert quality.
LLM backend configured via env var: `TEST_LLM_URL` (default: `http://127.0.0.1:1234/v1`). Works with LM Studio, Ollama, KoboldCpp, llama.cpp.
Assertions (structural, not exact content):
- Response contains at least 1 `<memory>` block
- Each block has `chat` and `date` attributes
- Each block has at least 1 bullet
- No character card trait leakage (bullets don't parrot the character description)
- Total bullet count is reasonable for the input size
### File structure
```
test/
fixtures/
flux-chat.jsonl
unit/ (existing, unchanged)
parsing.test.js
escaping.test.js
format-detection.test.js
utils.test.js
integration/
snapshot.test.js
live.test.js
```
### Changes to existing files
- `lib.js` — Add 3 exported functions
- `index.js` — Replace inline logic with lib.js calls (refactor, no behavior change)
- `package.json` — No changes needed (scripts already defined)

753
docs/plans/2026-02-26-automated-testing-plan.md Normal file
View file

@ -0,0 +1,753 @@
# Automated Testing Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Add automated snapshot and live-LLM integration tests for CharMemory's extraction pipeline.
**Architecture:** Extract 3 pure functions from `index.js` into `lib.js` (stripNonDiegetic, formatChatMessages, substitutePromptTemplate), then build snapshot tests using a 1000-message JSONL fixture and live LLM tests against an OpenAI-compatible endpoint.
**Tech Stack:** Vitest (already installed), Node.js ESM, JSONL fixture data, OpenAI-compatible API for live tests.
---
### Task 1: Extract `stripNonDiegetic()` into `lib.js`
**Files:**
- Modify: `lib.js` (add function at end, before closing)
- Modify: `index.js:2031-2036` (replace inline regexes with function call)
- Test: `test/unit/utils.test.js` (add tests)
**Step 1: Write the failing test**
Add to `test/unit/utils.test.js`:
```js
import { stripNonDiegetic } from '../../lib.js';
describe('stripNonDiegetic', () => {
it('removes markdown code blocks', () => {
const input = 'Before ```const x = 1;\nconsole.log(x);``` After';
expect(stripNonDiegetic(input)).toBe('Before After');
});
it('removes <details> sections', () => {
const input = 'Before <details><summary>Hidden</summary>Secret content</details> After';
expect(stripNonDiegetic(input)).toBe('Before After');
});
it('removes markdown tables', () => {
const input = 'Before\n| Col1 | Col2 |\n| --- | --- |\n| A | B |\nAfter';
expect(stripNonDiegetic(input)).toBe('Before\n\nAfter');
});
it('removes HTML tags', () => {
const input = 'Hello <b>world</b> and <img src="x" /> done';
expect(stripNonDiegetic(input)).toBe('Hello world and done');
});
it('collapses 3+ newlines to 2', () => {
const input = 'Line 1\n\n\n\nLine 2';
expect(stripNonDiegetic(input)).toBe('Line 1\n\nLine 2');
});
it('handles combined non-diegetic content', () => {
const input = '*She smiles* ```image: portrait``` and shows a table\n| x | y |\n| 1 | 2 |\nthen continues';
const result = stripNonDiegetic(input);
expect(result).not.toContain('```');
expect(result).not.toContain('| x |');
expect(result).toContain('*She smiles*');
expect(result).toContain('then continues');
});
it('returns empty string for all-non-diegetic input', () => {
const input = '```only code here```';
expect(stripNonDiegetic(input).trim()).toBe('');
});
});
```
**Step 2: Run test to verify it fails**
Run: `npm test`
Expected: FAIL — `stripNonDiegetic` is not exported from `lib.js`
**Step 3: Write the implementation**
Add to `lib.js` before the final empty line:
```js
// --- Non-diegetic content stripping ---
/**
* Strip non-diegetic content from a message: code blocks, details sections,
* markdown tables, HTML tags, and excessive newlines.
* @param {string} text Raw message text.
* @returns {string} Cleaned text.
*/
export function stripNonDiegetic(text) {
return text
.replace(/```[\s\S]*?```/g, '')
.replace(/<details[\s\S]*?<\/details>/gi, '')
.replace(/\|[^\n]*\|(?:\n\|[^\n]*\|)*/g, '')
.replace(/<[^>]*>/g, '')
.replace(/\n{3,}/g, '\n\n');
}
```
**Step 4: Run test to verify it passes**
Run: `npm test`
Expected: All tests PASS (existing 71 + new 7)
**Step 5: Update `index.js` to use `stripNonDiegetic`**
In `index.js`, the `collectRecentMessages` function (around line 2003) has inline stripping at lines 2031-2036. Note that `index.js` does NOT import from `lib.js` (it has its own copies of these functions). Add a local `stripNonDiegetic` function near the other utility functions in `index.js` with the same implementation, OR replace the 5 inline regex lines with a local helper. The simplest approach: replace the 5 regex lines (2031-2036) with a call to a new local function.
Find in `index.js` `collectRecentMessages` (around line 2031):
```js
let text = msg.mes;
text = text.replace(/```[\s\S]*?```/g, ''); // code blocks (image prompts)
text = text.replace(/<details[\s\S]*?<\/details>/gi, ''); // collapsed details sections
text = text.replace(/\|[^\n]*\|(?:\n\|[^\n]*\|)*/g, ''); // markdown tables
text = text.replace(/<[^>]*>/g, ''); // HTML tags
text = text.replace(/\n{3,}/g, '\n\n').trim(); // collapse whitespace
```
Replace with:
```js
let text = stripNonDiegetic(msg.mes).trim();
```
And add `stripNonDiegetic` as a local function in `index.js` (near line 500, after other utility functions) with the same body as lib.js. This keeps index.js self-contained (it doesn't import from lib.js at runtime since SillyTavern loads index.js as an extension).
**Step 6: Run existing tests to verify no regression**
Run: `npm test`
Expected: All PASS
**Step 7: Commit**
```
git add lib.js index.js test/unit/utils.test.js
git commit -m "refactor: extract stripNonDiegetic into lib.js with tests"
```
---
### Task 2: Extract `formatChatMessages()` into `lib.js`
**Files:**
- Modify: `lib.js` (add function)
- Modify: `index.js` (refactor `collectRecentMessages` to delegate)
- Test: `test/unit/utils.test.js` (add tests)
**Step 1: Write the failing test**
Add to `test/unit/utils.test.js`:
```js
import { formatChatMessages } from '../../lib.js';
describe('formatChatMessages', () => {
const makeMsg = (name, mes, overrides = {}) => ({
name, mes, is_user: false, is_system: false, ...overrides,
});
it('formats messages as "Name: text"', () => {
const chat = [
makeMsg('Alice', 'Hello there'),
makeMsg('Bob', 'Hi Alice'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).toBe('Alice: Hello there\n\nBob: Hi Alice');
});
it('skips empty messages', () => {
const chat = [
makeMsg('Alice', 'Hello'),
makeMsg('Bob', ''),
makeMsg('Alice', 'Still here'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).toBe('Alice: Hello\n\nAlice: Still here');
expect(result.messageCount).toBe(2);
});
it('skips system-only messages (no name, no user)', () => {
const chat = [
makeMsg('Alice', 'Hello'),
makeMsg(null, 'System narrator text', { is_system: true }),
makeMsg('Bob', 'Hi'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).not.toContain('System narrator');
});
it('keeps system messages that have a name', () => {
const chat = [
makeMsg('Extension', 'Some extension text', { is_system: true }),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).toContain('Extension: Some extension text');
});
it('strips non-diegetic content from messages', () => {
const chat = [
makeMsg('Alice', 'She smiled ```image prompt here``` and waved'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).not.toContain('```');
expect(result.text).toContain('She smiled');
});
it('respects startIndex and endIndex', () => {
const chat = [
makeMsg('A', 'msg0'),
makeMsg('B', 'msg1'),
makeMsg('C', 'msg2'),
makeMsg('D', 'msg3'),
];
const result = formatChatMessages(chat, 1, 3);
expect(result.text).toBe('B: msg1\n\nC: msg2');
expect(result.startIndex).toBe(1);
expect(result.endIndex).toBe(2);
});
it('returns empty for out-of-range indices', () => {
const chat = [makeMsg('A', 'msg')];
const result = formatChatMessages(chat, 5, 10);
expect(result.text).toBe('');
});
});
```
**Step 2: Run test to verify it fails**
Run: `npm test`
Expected: FAIL — `formatChatMessages` is not exported
**Step 3: Write the implementation**
Add to `lib.js`:
```js
/**
* Format chat messages for extraction prompt. Filters out empty/system-only
* messages, strips non-diegetic content, returns "Name: text" format.
* @param {Array<{name: string, mes: string, is_user?: boolean, is_system?: boolean}>} chatArray
* @param {number} startIndex Start index (inclusive) in chatArray.
* @param {number} endIndex End index (exclusive) in chatArray.
* @returns {{ text: string, startIndex: number, endIndex: number, messageCount: number }}
*/
export function formatChatMessages(chatArray, startIndex, endIndex) {
if (!chatArray || chatArray.length === 0) return { text: '', startIndex: -1, endIndex: -1, messageCount: 0 };
const safeStart = Math.max(0, startIndex);
const safeEnd = Math.min(chatArray.length, endIndex);
if (safeStart >= safeEnd) return { text: '', startIndex: -1, endIndex: -1, messageCount: 0 };
const slice = chatArray.slice(safeStart, safeEnd);
const lines = [];
for (const msg of slice) {
if (!msg.mes) continue;
if (msg.is_system && !msg.is_user && !msg.name) continue;
const text = stripNonDiegetic(msg.mes).trim();
if (!text) continue;
lines.push(`${msg.name}: ${text}`);
}
return {
text: lines.join('\n\n'),
startIndex: safeStart,
endIndex: safeEnd - 1,
messageCount: lines.length,
};
}
```
**Step 4: Run tests**
Run: `npm test`
Expected: All PASS
**Step 5: Update `index.js` `collectRecentMessages` to delegate**
Replace the loop body in `collectRecentMessages` with a call to a local copy of `formatChatMessages`. The function still reads `getContext()`, `chat_metadata`, `extension_settings` to determine startIndex/endIndex/maxMessages, then passes the chat array to `formatChatMessages`.
**Step 6: Commit**
```
git add lib.js index.js test/unit/utils.test.js
git commit -m "refactor: extract formatChatMessages into lib.js with tests"
```
---
### Task 3: Extract `substitutePromptTemplate()` into `lib.js`
**Files:**
- Modify: `lib.js` (add function)
- Modify: `index.js` (refactor `buildExtractionPrompt` to delegate)
- Test: `test/unit/utils.test.js` (add tests)
**Step 1: Write the failing test**
```js
import { substitutePromptTemplate } from '../../lib.js';
describe('substitutePromptTemplate', () => {
const template = 'Name: {{charName}}\nCard: {{charCard}}\nMemories: {{existingMemories}}\nMessages: {{recentMessages}}';
it('substitutes all template variables', () => {
const result = substitutePromptTemplate(template, {
charName: 'Flux',
charCard: 'A cat',
existingMemories: '- Likes fish',
recentMessages: 'Alex: Hello\n\nFlux: Meow',
});
expect(result).toBe('Name: Flux\nCard: A cat\nMemories: - Likes fish\nMessages: Alex: Hello\n\nFlux: Meow');
});
it('replaces multiple occurrences of the same variable', () => {
const t = '{{charName}} says hi. {{charName}} waves.';
const result = substitutePromptTemplate(t, { charName: 'Flux' });
expect(result).toBe('Flux says hi. Flux waves.');
});
it('substitutes {{participants}} when provided', () => {
const t = 'Participants: {{participants}}';
const result = substitutePromptTemplate(t, { participants: 'Alice, Bob' });
expect(result).toBe('Participants: Alice, Bob');
});
it('leaves unmatched variables as-is', () => {
const t = '{{charName}} and {{unknownVar}}';
const result = substitutePromptTemplate(t, { charName: 'Flux' });
expect(result).toContain('{{unknownVar}}');
});
it('uses "(none yet)" default for missing existingMemories', () => {
const result = substitutePromptTemplate(template, {
charName: 'Flux', charCard: 'A cat', recentMessages: 'hi',
});
expect(result).toContain('(none yet)');
});
});
```
**Step 2: Run test to verify it fails**
Run: `npm test`
Expected: FAIL
**Step 3: Write the implementation**
Add to `lib.js`:
```js
/**
* Substitute CharMemory template variables in a prompt string.
* @param {string} template Prompt template with {{variable}} placeholders.
* @param {Object} vars Variable values to substitute.
* @param {string} [vars.charName]
* @param {string} [vars.charCard]
* @param {string} [vars.existingMemories]
* @param {string} [vars.recentMessages]
* @param {string} [vars.participants]
* @returns {string} Prompt with variables replaced.
*/
export function substitutePromptTemplate(template, vars) {
let result = template;
if (vars.charName != null) result = result.replace(/\{\{charName\}\}/g, vars.charName);
if (vars.charCard != null) result = result.replace(/\{\{charCard\}\}/g, vars.charCard);
result = result.replace(/\{\{existingMemories\}\}/g, vars.existingMemories || '(none yet)');
if (vars.recentMessages != null) result = result.replace(/\{\{recentMessages\}\}/g, vars.recentMessages);
if (vars.participants != null) result = result.replace(/\{\{participants\}\}/g, vars.participants);
return result;
}
```
**Step 4: Run tests, commit**
Run: `npm test` — all PASS
```
git add lib.js index.js test/unit/utils.test.js
git commit -m "refactor: extract substitutePromptTemplate into lib.js with tests"
```
---
### Task 4: Set up test fixture
**Files:**
- Create: `test/fixtures/flux-chat.jsonl` (copy from external repo)
- Create: `test/fixtures/sample-llm-response.txt` (hand-crafted fixture)
**Step 1: Copy the test chat JSONL**
```bash
mkdir -p test/fixtures
cp /Users/davidsayed/repos/st-test-chatlog/output/2026-01-15@08h00m00s.jsonl test/fixtures/flux-chat.jsonl
```
**Step 2: Create a sample LLM response fixture**
Create `test/fixtures/sample-llm-response.txt` with a realistic extraction response:
```
<memory chat="main_abc123" date="2026-01-15 10:00">
- Alex adopted Flux the Cat from a pet store and brought him to his penthouse apartment
- Flux immediately bonded with his custom Gundam-styled Roomba, using it as personal transport
- Alex works in marketing and had a video conference with his boss Mr. Henderson on Flux's first day
- Flux's first meal was premium salmon pate, which triggered his first purr in the new home
</memory>
```
**Step 3: Commit**
```
git add test/fixtures/
git commit -m "test: add chat fixture and sample LLM response"
```
---
### Task 5: Snapshot integration tests
**Files:**
- Create: `test/integration/snapshot.test.js`
**Step 1: Write snapshot tests**
```js
import { describe, it, expect } from 'vitest';
import { readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import {
stripNonDiegetic,
formatChatMessages,
substitutePromptTemplate,
parseMemories,
serializeMemories,
} from '../../lib.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const fixturesDir = join(__dirname, '..', 'fixtures');
function loadChat() {
const raw = readFileSync(join(fixturesDir, 'flux-chat.jsonl'), 'utf-8');
const lines = raw.trim().split('\n');
// First line is metadata, rest are messages
return lines.slice(1).map(line => JSON.parse(line));
}
function loadSampleResponse() {
return readFileSync(join(fixturesDir, 'sample-llm-response.txt'), 'utf-8');
}
describe('Snapshot: stripNonDiegetic on real messages', () => {
it('strips non-diegetic content without destroying message text', () => {
const chat = loadChat();
// Process first 10 messages and snapshot
const results = chat.slice(0, 10).map(msg => ({
name: msg.name,
original_length: msg.mes.length,
stripped: stripNonDiegetic(msg.mes).trim(),
}));
expect(results).toMatchSnapshot();
});
});
describe('Snapshot: formatChatMessages', () => {
it('processes first 20 messages', () => {
const chat = loadChat();
const result = formatChatMessages(chat, 0, 20);
expect(result.messageCount).toBeGreaterThan(0);
expect(result.text).toMatchSnapshot();
});
it('processes messages 20-50', () => {
const chat = loadChat();
const result = formatChatMessages(chat, 20, 50);
expect(result.messageCount).toBeGreaterThan(0);
expect(result.text).toMatchSnapshot();
});
it('handles the full 1000-message chat without error', () => {
const chat = loadChat();
const result = formatChatMessages(chat, 0, chat.length);
expect(result.messageCount).toBeGreaterThan(900); // some may be filtered
expect(result.text.length).toBeGreaterThan(10000);
});
});
describe('Snapshot: substitutePromptTemplate', () => {
it('builds a complete extraction prompt', () => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 20);
const defaultPrompt = `You are a memory extraction assistant.
Character name: {{charName}}
===== CHARACTER CARD =====
{{charCard}}
===== EXISTING MEMORIES =====
{{existingMemories}}
===== RECENT CHAT MESSAGES =====
{{recentMessages}}
===== END =====
Extract memories:`;
const result = substitutePromptTemplate(defaultPrompt, {
charName: 'Flux the Cat',
charCard: 'Flux is a clever black-and-white cat who rides a Gundam Roomba.',
existingMemories: '',
recentMessages: formatted.text,
});
expect(result).toContain('Flux the Cat');
expect(result).toContain('Gundam Roomba');
expect(result).not.toContain('{{charName}}');
expect(result).not.toContain('{{recentMessages}}');
expect(result).toMatchSnapshot();
});
});
describe('Snapshot: parseMemories round-trip', () => {
it('parse → serialize → parse produces identical blocks', () => {
const response = loadSampleResponse();
const blocks = parseMemories(response);
expect(blocks.length).toBeGreaterThan(0);
const reserialized = serializeMemories(blocks);
const reparsed = parseMemories(reserialized);
expect(reparsed).toEqual(blocks);
});
});
```
**Step 2: Run snapshot tests and generate initial snapshots**
Run: `npm run test:snapshot -- --update`
Expected: PASS, creates `__snapshots__/snapshot.test.js.snap`
**Step 3: Run again without --update to verify snapshots match**
Run: `npm run test:snapshot`
Expected: All PASS
**Step 4: Commit**
```
git add test/integration/ test/integration/__snapshots__/
git commit -m "test: add snapshot integration tests for extraction pipeline"
```
---
### Task 6: Live LLM integration tests
**Files:**
- Create: `test/integration/live.test.js`
**Step 1: Write live tests**
```js
import { describe, it, expect } from 'vitest';
import { readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import {
formatChatMessages,
substitutePromptTemplate,
parseMemories,
countMemories,
} from '../../lib.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const fixturesDir = join(__dirname, '..', 'fixtures');
const LLM_URL = process.env.TEST_LLM_URL || 'http://127.0.0.1:1234/v1';
const LLM_MODEL = process.env.TEST_LLM_MODEL || '';
function loadChat() {
const raw = readFileSync(join(fixturesDir, 'flux-chat.jsonl'), 'utf-8');
return raw.trim().split('\n').slice(1).map(line => JSON.parse(line));
}
// Read the actual default extraction prompt from index.js is not possible
// since it depends on ST imports, so we use a simplified version for testing.
const EXTRACTION_PROMPT = `You are a memory extraction assistant. Read the recent chat messages and identify the most significant facts, events, and developments worth remembering long-term.
Character name: {{charName}}
===== CHARACTER CARD (baseline knowledge — do NOT extract anything already described here) =====
{{charCard}}
===== END CHARACTER CARD =====
===== EXISTING MEMORIES (reference only — do NOT repeat these) =====
{{existingMemories}}
===== END EXISTING MEMORIES =====
===== RECENT CHAT MESSAGES (extract ONLY from this section) =====
{{recentMessages}}
===== END RECENT CHAT MESSAGES =====
Extract only NEW facts, events, relationships, or character developments.
Write in past tense, third person. No more than 8 bullet points.
Wrap output in <memory></memory> tags with bullets starting with "- ".
If nothing new, respond with: NO_NEW_MEMORIES`;
const CHARACTER_CARD = `Flux the Cat is a clever, sassy black-and-white cat. He rides a custom Gundam-styled Roomba as his personal transport. He's food-motivated, loves watching birds from the window, and has a dramatic personality.`;
async function callTestLLM(prompt) {
// Discover model if not specified
let model = LLM_MODEL;
if (!model) {
const modelsRes = await fetch(`${LLM_URL}/models`);
const modelsData = await modelsRes.json();
model = modelsData.data?.[0]?.id;
if (!model) throw new Error('No models available at ' + LLM_URL);
}
const response = await fetch(`${LLM_URL}/chat/completions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model,
messages: [
{ role: 'system', content: 'You are a memory extraction assistant.' },
{ role: 'user', content: prompt },
],
max_tokens: 1000,
temperature: 0.3,
}),
});
if (!response.ok) {
throw new Error(`LLM error: ${response.status} ${await response.text()}`);
}
const data = await response.json();
return data.choices?.[0]?.message?.content || '';
}
describe('Live LLM: extraction from test chat', () => {
it('extracts memories from the first 20 messages', async () => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 20);
const prompt = substitutePromptTemplate(EXTRACTION_PROMPT, {
charName: 'Flux the Cat',
charCard: CHARACTER_CARD,
existingMemories: '',
recentMessages: formatted.text,
});
const response = await callTestLLM(prompt);
const blocks = parseMemories(response);
// Structural assertions
expect(blocks.length).toBeGreaterThanOrEqual(1);
const totalBullets = countMemories(blocks);
expect(totalBullets).toBeGreaterThanOrEqual(1);
expect(totalBullets).toBeLessThanOrEqual(10); // prompt says max 8, allow some slack
// Each block has required attributes
for (const block of blocks) {
expect(block.bullets.length).toBeGreaterThan(0);
for (const bullet of block.bullets) {
expect(bullet.length).toBeGreaterThan(5);
}
}
}, 60000); // 60s timeout for LLM call
it('does not parrot character card traits', async () => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 20);
const prompt = substitutePromptTemplate(EXTRACTION_PROMPT, {
charName: 'Flux the Cat',
charCard: CHARACTER_CARD,
existingMemories: '',
recentMessages: formatted.text,
});
const response = await callTestLLM(prompt);
const blocks = parseMemories(response);
const allBullets = blocks.flatMap(b => b.bullets).join('\n').toLowerCase();
// These are card traits that should NOT appear as extracted memories
const cardTraits = [
'food-motivated',
'loves watching birds',
'dramatic personality',
];
for (const trait of cardTraits) {
expect(allBullets).not.toContain(trait);
}
}, 60000);
it('handles a larger chunk (messages 0-50)', async () => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 50);
const prompt = substitutePromptTemplate(EXTRACTION_PROMPT, {
charName: 'Flux the Cat',
charCard: CHARACTER_CARD,
existingMemories: '',
recentMessages: formatted.text,
});
const response = await callTestLLM(prompt);
// Should produce valid output or NO_NEW_MEMORIES
if (response.trim() === 'NO_NEW_MEMORIES') return;
const blocks = parseMemories(response);
expect(blocks.length).toBeGreaterThanOrEqual(1);
}, 120000); // 2min timeout for larger context
});
```
**Step 2: Run live tests (requires running LLM server)**
Run: `TEST_LLM_URL=http://127.0.0.1:1234/v1 npm run test:live`
Expected: PASS (with LM Studio or other local server running)
**Step 3: Commit**
```
git add test/integration/live.test.js
git commit -m "test: add live LLM integration tests for extraction"
```
---
### Task 7: Final verification and cleanup
**Step 1: Run all test suites**
```bash
npm test # unit tests (should still pass)
npm run test:snapshot # snapshot tests
TEST_LLM_URL=http://127.0.0.1:1234/v1 npm run test:live # live tests
```
**Step 2: Verify test count**
Expected:
- Unit: 71 existing + ~20 new = ~91 tests
- Snapshot: ~5 tests
- Live: ~3 tests
**Step 3: Final commit**
```
git add -A
git commit -m "test: complete automated testing setup for extraction pipeline"
```

675
docs/plans/2026-02-28-retrieval-optimized-format.md Normal file
View file

@ -0,0 +1,675 @@
# Retrieval-Optimized Memory Format — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Improve vector retrieval precision by updating extraction/consolidation prompts to produce topic-tagged, concise memory blocks, adding a Reformat tool for existing memories, and enhancing diagnostics to warn about Vector Storage misconfigurations.
**Architecture:** Three phases — (1) update all prompt strings and tests, (2) add Reformat tool to the Tools tab using the existing conversion dialog pattern, (3) enhance health checks. All changes are in `index.js`, `settings.html`, `lib.js`, and test files. No new dependencies.
**Tech Stack:** JavaScript (SillyTavern browser extension), jQuery, Vitest
---
## Phase 1: Prompt Updates
Prompt changes only affect users on the default prompt. Users with customized prompts are unaffected. The "Restore Default" button becomes the opt-in mechanism.
### Task 1: Update Solo Extraction Prompt
**Files:**
- Modify: `index.js:111-183` (the `defaultExtractionPrompt` template literal)
- Reference: `docs/proposed-solo-extraction-prompt.txt`
**Step 1: Read the current prompt and proposed replacement**
Read `index.js` lines 111-183 and `docs/proposed-solo-extraction-prompt.txt` to confirm the exact diff.
**Step 2: Replace `defaultExtractionPrompt` in index.js**
Replace the entire template literal (lines 111-183) with the contents of `docs/proposed-solo-extraction-prompt.txt`, wrapped in backticks as a template literal. Key changes:
- New instruction 6: topic tag as first bullet `- [Names — description]`
- Instruction 7: bullet limit 8 → 5 per block (not counting topic tag)
- New line in WHAT TO EXTRACT: "Always name specific people involved"
- Updated negative example: Flux play-by-play (8 bullets about adoption day)
- Updated positive example: Flux topic tag + 3 outcome bullets
- Positive example label changed from "Marcus — warehouse heist" to Flux example
**Step 3: Commit**
```bash
git add index.js
git commit -m "feat: update default extraction prompt with topic tags and 5-bullet limit"
```
### Task 2: Update Group Extraction Prompt
**Files:**
- Modify: `index.js:185-238` (the `defaultGroupExtractionPrompt` template literal)
- Reference: `docs/proposed-group-extraction-prompt.txt`
**Step 1: Replace `defaultGroupExtractionPrompt` in index.js**
Replace the entire template literal (lines 185-238) with the contents of `docs/proposed-group-extraction-prompt.txt`. Key changes mirror the solo prompt:
- New instruction 6: topic tag
- Instruction 7: bullet limit 8 → 5
- New line in WHAT TO EXTRACT: "Always name specific people involved"
**Step 2: Commit**
```bash
git add index.js
git commit -m "feat: update default group extraction prompt with topic tags and 5-bullet limit"
```
### Task 3: Update Consolidation Presets and Wrapper
**Files:**
- Modify: `index.js:4066-4082` (`CONSOLIDATION_PRESETS` object)
- Modify: `index.js:4090-4104` (`buildConsolidationPrompt` ADDITIONAL FORMAT RULES)
- Reference: `docs/proposed-consolidation-presets.txt`
**Step 1: Update the three preset prompt strings**
In `CONSOLIDATION_PRESETS` (line 4066):
**Conservative** — append to existing prompt:
```
Each block must start with a topic tag as the first bullet: "- [Names involved — short description]" (e.g., "- [Alex, Flux — adoption day at the apartment]"). Preserve existing topic tags.
```
**Balanced** — append to existing prompt:
```
Each block must start with a topic tag as the first bullet: "- [Names involved — short description]" (e.g., "- [Alex, Flux — adoption day at the apartment]"). When merging blocks, update the topic tag to reflect the combined content. No more than 5 bullets per block (not counting the topic tag).
```
**Aggressive** — append to existing prompt:
```
Each block must start with a topic tag as the first bullet: "- [Names/themes — short description]" (e.g., "- [Alex, Sarah — first visit to the apartment]"). No more than 5 bullets per block (not counting the topic tag). Always name specific people — never use "a client" or "someone."
```
**Step 2: Update the FORMAT RULES in `buildConsolidationPrompt()`**
At line 4096-4100, replace the ADDITIONAL FORMAT RULES section:
```javascript
ADDITIONAL FORMAT RULES:
1. Do NOT use emojis anywhere in the output.
2. Do NOT copy text verbatim from the input — rephrase in third person.
3. Group memories by theme or encounter. Each group is wrapped in <memory chat="Theme Name"></memory> tags where "Theme Name" is a short, specific label. Prefer encounter-specific labels (e.g., "Adoption day at the apartment", "First vet visit") over broad categories (e.g., "Key Events", "Relationships"). Specific labels improve later retrieval.
4. Inside each <memory> block, use a markdown bulleted list (lines starting with "- ").
5. The first bullet in each block must be a topic tag: "- [Names — short description]". This is mandatory.
6. Always use specific names for people involved, never generic labels like "a client" or "someone."
```
**Step 3: Commit**
```bash
git add index.js
git commit -m "feat: update consolidation presets with topic tags and encounter-specific labels"
```
### Task 4: Update Conversion/Reformat Prompt
**Files:**
- Modify: `index.js:240-253` (`defaultConversionPrompt` template literal)
- Reference: `docs/proposed-reformat-prompt.txt`
**Step 1: Replace `defaultConversionPrompt` with the unified reformat prompt**
Replace the entire template literal with the contents of `docs/proposed-reformat-prompt.txt`. This prompt handles all three cases:
- Unstructured text → create `<memory>` blocks with topic tags
- Existing `<memory>` blocks without tags → add tags + trim to 5 bullets
- Already-formatted blocks → leave unchanged
Note: The `{{today}}` placeholder in the prompt needs to be substituted at call time. Check how `convertWithLLM()` (line 802) builds the prompt — it uses `defaultConversionPrompt` with `{{sourceText}}` and `{{charName}}` substitutions. Add `{{today}}` substitution there:
```javascript
.replace(/\{\{today\}\}/g, new Date().toISOString().slice(0, 10))
```
**Step 2: Commit**
```bash
git add index.js
git commit -m "feat: replace conversion prompt with unified reformat prompt"
```
### Task 5: Update lib.js Copies (if applicable)
**Files:**
- Modify: `lib.js` (if any prompt-related functions are duplicated there)
**Step 1: Check if lib.js contains any of the updated functions**
The prompts are constants, not functions. `lib.js` only contains copies of `stripNonDiegetic`, `formatChatMessages`, and `substitutePromptTemplate`. The prompt constants are NOT in `lib.js`.
**Step 2: Run the sync check test to verify**
Run: `npm test -- --grep "sync"`
Expected: PASS — the sync check only validates the three duplicated functions, not prompt constants.
**Step 3: No changes needed to lib.js for prompt updates.**
### Task 6: Update Sample LLM Response Fixture
**Files:**
- Modify: `test/fixtures/sample-llm-response.txt`
**Step 1: Update the fixture to include a topic tag**
The current fixture:
```
<memory chat="main_abc123" date="2026-01-15 10:00">
- Alex adopted Flux the Cat from a pet store and brought him to his penthouse apartment
- Flux immediately bonded with his custom Gundam-styled Roomba, using it as personal transport
- Alex works in marketing and had a video conference with his boss Mr. Henderson on Flux's first day
- Flux's first meal was premium salmon pate, which triggered his first purr in the new home
</memory>
```
Replace with:
```
<memory chat="main_abc123" date="2026-01-15 10:00">
- [Alex, Flux — adoption day and settling into the apartment]
- Alex adopted Flux and brought him to his penthouse, where Flux immediately bonded with his custom Gundam-styled Roomba.
- Flux's first meal of premium salmon pate triggered his first purr in the new home.
- Alex assembled a cat tree that Flux claimed as a second perch alongside the Roomba.
</memory>
```
**Step 2: Commit**
```bash
git add test/fixtures/sample-llm-response.txt
git commit -m "test: update sample LLM response fixture with topic tag format"
```
### Task 7: Update Snapshot Tests
**Files:**
- Modify: `test/integration/__snapshots__/snapshot.test.js.snap`
**Step 1: Run snapshot tests to see what breaks**
Run: `npm run test:snapshot`
Expected: The `substitutePromptTemplate` snapshots will fail because the default prompt text changed. The `formatChatMessages` and `stripNonDiegetic` snapshots should still pass (they don't involve prompt text).
**Step 2: Review the snapshot diff**
The failing snapshots should show the old prompt text being replaced by the new prompt text. Verify the new prompt content looks correct in the snapshot output.
**Step 3: Update snapshots**
Run: `npm run test:snapshot -- --update`
**Step 4: Run all tests to confirm everything passes**
Run: `npm test`
Expected: All tests pass.
**Step 5: Commit**
```bash
git add test/
git commit -m "test: update snapshots for new extraction prompt format"
```
---
## Phase 2: Reformat Tool
Add a "Reformat" tool to the Tools tab that runs existing memory files through the LLM to add topic tags and trim bullets. Uses the same dialog pattern as consolidation (preview + edit + confirm + undo).
### Task 8: Add Reformat Pill to Tools Tab HTML
**Files:**
- Modify: `settings.html:57-64` (tool pills section)
- Modify: `settings.html` (add new tool content pane)
**Step 1: Add "Reformat" pill button after "Convert"**
In settings.html, find the tool pills section and add a fourth pill:
```html
<div class="charMemory_toolPill" data-tool="reformat">Reformat</div>
```
**Step 2: Add the Reformat tool content pane**
After `#charMemory_toolConvert`, add:
```html
<div id="charMemory_toolReformat" style="display:none;">
<div class="charMemory_toolDescription">
Reformat existing memories to improve vector retrieval. Adds topic tags, trims to 5 bullets per block, and ensures specific names are used. Works on any format — unstructured text, old memory blocks, or already-formatted blocks.
</div>
<div class="charMemory_inlineRow">
<div id="charMemory_reformatPreview" class="menu_button">
<i class="fa-solid fa-wand-magic-sparkles"></i> Preview Reformat
</div>
<div id="charMemory_undoReformat" class="menu_button" disabled>
<i class="fa-solid fa-rotate-left"></i> Undo Reformat
</div>
</div>
</div>
```
**Step 3: Commit**
```bash
git add settings.html
git commit -m "feat: add Reformat pill and content pane to Tools tab"
```
### Task 9: Wire Up Reformat Tool in JavaScript
**Files:**
- Modify: `index.js` (add reformat flow near consolidation code, ~line 4050)
- Modify: `index.js` (add event handlers in `setupListeners()`, ~line 4900)
**Step 1: Add `reformatBackup` variable**
Near `consolidationBackup` declaration (search for `let consolidationBackup`), add:
```javascript
let reformatBackup = null;
```
**Step 2: Add `reformatMemories()` async function**
Add after `consolidateMemories()`. This function:
1. Reads the current memory file for the active character
2. If no memories exist, shows error and returns
3. Sends the full memory text through `convertWithLLM()` using the updated `defaultConversionPrompt` (which is now the reformat prompt)
4. Shows a preview dialog using `buildConversionDialog()` (reuse the existing conversion dialog builder — it already has the left/right pane with editing)
5. On confirmation:
- Backs up current content to `reformatBackup`
- Writes reformatted blocks to the memory file
- Enables the Undo button
- Shows success toast
```javascript
async function reformatMemories() {
const context = getContext();
if (!context.characterId) {
toastr.warning('No character selected.', 'CharMemory');
return;
}
const charName = context.name2;
const content = await readMemoriesForCharacter();
if (!content || !content.trim()) {
toastr.warning('No memories found to reformat.', 'CharMemory');
return;
}
const blocks = parseMemories(content);
if (blocks.length === 0) {
toastr.warning('No memory blocks found in file.', 'CharMemory');
return;
}
logActivity(`Reformat started: ${blocks.length} blocks`);
// Check if already formatted (all blocks have topic tags)
const allHaveTags = blocks.every(b => b.bullets.length > 0 && /^\[.+\]$/.test(b.bullets[0].trim()));
if (allHaveTags) {
const proceed = await callGenericPopup(
'All memory blocks already appear to have topic tags. Reformat anyway?',
POPUP_TYPE.CONFIRM,
);
if (!proceed) return;
}
// Send through LLM
const reformatted = await convertWithLLM(content, charName);
if (!reformatted) {
toastr.error('Reformat failed — no response from LLM.', 'CharMemory');
return;
}
const newBlocks = parseMemories(reformatted);
if (newBlocks.length === 0) {
toastr.error('Reformat produced no valid memory blocks.', 'CharMemory');
return;
}
// Show preview dialog (reuse conversion dialog pattern)
const result = await showReformatPreview(blocks, newBlocks, charName);
if (!result) return; // User cancelled
// Save with backup
reformatBackup = content;
const serialized = serializeMemories(result);
await writeMemoriesForCharacter(serialized);
$('#charMemory_undoReformat').prop('disabled', false);
logActivity(`Reformat complete: ${blocks.length} blocks → ${result.length} blocks`);
toastr.success(`Reformatted ${result.length} memory blocks.`, 'CharMemory');
}
```
Note: `showReformatPreview()` should be a simplified version of the consolidation dialog. It needs:
- Left pane: original blocks (read-only)
- Right pane: reformatted blocks (editable — same card editor as consolidation)
- Confirm/Cancel buttons
- Returns the edited blocks array or null if cancelled
The easiest approach is to reuse `buildConsolidationDialog()` with minor modifications, or create a generic `buildMemoryComparisonDialog(originalBlocks, newBlocks, title)` that both consolidation and reformat can use. However, to minimize risk and code churn, the simplest path is to build a standalone `showReformatPreview()` that creates a popup with the same HTML structure as the consolidation dialog.
**Step 3: Add undo handler**
```javascript
async function undoReformat() {
if (!reformatBackup) return;
const confirm = await callGenericPopup(
'Undo the last reformat and restore original memories?',
POPUP_TYPE.CONFIRM,
);
if (!confirm) return;
await writeMemoriesForCharacter(reformatBackup);
reformatBackup = null;
$('#charMemory_undoReformat').prop('disabled', true);
toastr.info('Reformat undone — original memories restored.', 'CharMemory');
logActivity('Reformat undone');
}
```
**Step 4: Wire up event handlers in `setupListeners()`**
Near the consolidation button handlers (search for `$('#charMemory_consolidate')`):
```javascript
$('#charMemory_reformatPreview').off('click').on('click', () => reformatMemories());
$('#charMemory_undoReformat').off('click').on('click', () => undoReformat());
```
Also update the tool pill switching handler to include 'reformat':
```javascript
if (tool === 'reformat') { /* no special init needed */ }
```
**Step 5: Handle `{{today}}` in the conversion prompt**
In `convertWithLLM()` (around line 802-837), add the `{{today}}` substitution when building the prompt:
```javascript
.replace(/\{\{today\}\}/g, new Date().toISOString().slice(0, 10))
```
**Step 6: Commit**
```bash
git add index.js
git commit -m "feat: add Reformat tool with preview, confirmation, and undo"
```
### Task 10: Build the Reformat Preview Dialog
**Files:**
- Modify: `index.js` (add `showReformatPreview()` function)
- Modify: `style.css` (reuse existing consolidation dialog styles if possible)
**Step 1: Implement `showReformatPreview()`**
This function creates a popup dialog showing original vs reformatted blocks side-by-side. It should closely follow the consolidation dialog pattern from `buildConsolidationDialog()` (line 4002) but simplified — no strategy dropdown, no re-run button. Just:
- Left pane: "Original" with read-only block cards
- Right pane: "Reformatted" with editable block cards
- Stats bar: "Original: N blocks → Reformatted: M blocks"
- OK/Cancel buttons (via `callGenericPopup`)
Use the same card HTML structure and editing event delegation as the consolidation dialog. The key functions to reference:
- Block card rendering: look at how consolidation renders `editorBlocks` into HTML cards
- Edit mode toggling: the `.charMemory_toggleEdit` click handler
- Bullet editing: the `.charMemory_editBullet` input handler
- Block/bullet deletion: the `.charMemory_deleteBullet` and `.charMemory_deleteBlock` handlers
Since consolidation already uses event delegation on the popup container for editing, the reformat dialog can use the same CSS classes and the same delegation handlers will work automatically — as long as the reformat dialog is shown as a popup with the same class structure.
**Step 2: Commit**
```bash
git add index.js style.css
git commit -m "feat: add reformat preview dialog with side-by-side comparison"
```
---
## Phase 3: Diagnostics Enhancements
Add new health checks that detect common Vector Storage misconfigurations.
### Task 11: Add Retrieve Chunks Health Check
**Files:**
- Modify: `index.js:3268-3393` (inside `computeHealthScore()`)
**Step 1: Research how to access Vector Storage settings**
Before implementing, check whether SillyTavern exposes Vector Storage extension settings to other extensions. Look for:
- `extension_settings.vectors` or similar in the ST global scope
- Any API endpoint that returns VS configuration
- The VS extension's settings object in the DOM
If VS settings are accessible, add a check. If not, skip this task and rely on the documentation/UI guidance approach instead.
**Step 2: Add check for retrieve chunks (if accessible)**
After the `chunk_size` check (line 3355), add:
```javascript
// Check: retrieve_chunks — warn if too high
const retrieveChunks = /* access VS setting */;
if (retrieveChunks !== undefined) {
if (retrieveChunks > 5) {
checks.push({
id: 'retrieve_chunks',
level: 'yellow',
label: 'Retrieve chunks is high',
detail: `Retrieve chunks is set to ${retrieveChunks}. For CharMemory, 2-3 is recommended. Higher values inject more memories per message, which can flood the prompt with irrelevant content.`,
});
} else {
checks.push({
id: 'retrieve_chunks',
level: 'green',
label: `Retrieve chunks: ${retrieveChunks}`,
detail: 'Retrieve chunks is in the recommended range for CharMemory.',
});
}
}
```
**Step 3: Add check for score threshold (if accessible)**
```javascript
// Check: score_threshold — info if not set
const scoreThreshold = /* access VS setting */;
if (scoreThreshold !== undefined && scoreThreshold < 0.1) {
checks.push({
id: 'score_threshold',
level: 'yellow',
label: 'No score threshold set',
detail: 'Without a score threshold, low-relevance memories may be injected. Recommended: 0.2-0.3 for most embedding models.',
});
}
```
**Step 4: Commit (if changes were made)**
```bash
git add index.js
git commit -m "feat: add retrieve chunks and score threshold health checks"
```
### Task 12: Update Existing Chunk Size Check
**Files:**
- Modify: `index.js:3333-3355` (existing `chunk_size` check)
**Step 1: Update the chunk size recommendations**
The current check warns if chunk size is smaller than average block or larger than 4x average block. Update the messaging to include the specific recommendation:
```javascript
detail: `Chunk size (${chunkSize} chars) is smaller than the average memory block (${avgBlockSize} chars). This may split blocks mid-content. Recommended: 800-1000 chars for CharMemory.`,
```
And for too-large chunks:
```javascript
detail: `Chunk size (${chunkSize} chars) is much larger than the average memory block (${avgBlockSize} chars). Multiple blocks may be packed into single chunks, reducing retrieval precision. Recommended: 800-1000 chars for CharMemory.`,
```
**Step 2: Commit**
```bash
git add index.js
git commit -m "feat: improve chunk size health check messaging with specific recommendations"
```
### Task 13: Add Recommended Settings to Diagnostics Panel
**Files:**
- Modify: `settings.html` (diagnostics section)
- Modify: `index.js` (render the recommendation card)
**Step 1: Add a collapsible "Recommended Vector Storage Settings" section**
In the diagnostics panel area of `settings.html`, add an expandable info section:
```html
<div class="charMemory_settingsRecommendation">
<div class="charMemory_recommendationHeader">
<i class="fa-solid fa-circle-info"></i> Recommended Vector Storage Settings
</div>
<div class="charMemory_recommendationBody" style="display:none;">
<table class="charMemory_recommendationTable">
<tr><td>Embedding Model</td><td>text-embedding-3-small or all-MiniLM-L6-v2</td></tr>
<tr><td>Chunk Size</td><td>800–1000 characters</td></tr>
<tr><td>Chunk Overlap</td><td>0%</td></tr>
<tr><td>Retrieve Chunks</td><td>2–3</td></tr>
<tr><td>Score Threshold</td><td>0.2–0.3 (varies by model)</td></tr>
<tr><td>Size Threshold</td><td>1 KB</td></tr>
<tr><td>Query Messages</td><td>2–3</td></tr>
</table>
</div>
</div>
```
**Step 2: Add toggle handler and minimal styling**
In `setupListeners()`:
```javascript
$('.charMemory_recommendationHeader').off('click').on('click', function () {
$(this).next('.charMemory_recommendationBody').slideToggle(200);
});
```
In `style.css`, add minimal styling for the recommendation table.
**Step 3: Commit**
```bash
git add settings.html index.js style.css
git commit -m "feat: add recommended Vector Storage settings card to diagnostics"
```
---
## Phase 4: Version Bump and Cleanup
### Task 14: Update Version and Changelog
**Files:**
- Modify: `manifest.json` (version bump to 1.7.0)
- Modify: `CHANGELOG.md` (add v1.7.0 section)
**Step 1: Bump version in manifest.json**
Change `"version": "1.6.1"` to `"version": "1.7.0"`.
**Step 2: Add changelog entry**
Add a new section at the top of CHANGELOG.md:
```markdown
## 1.7.0
### New Features
- **Retrieval-optimized memory format**: Extraction prompts now produce topic-tagged memory blocks with a `[Names — description]` tag as the first bullet. This improves vector search discrimination, allowing Vector Storage to retrieve only the most relevant memories instead of thematically similar ones.
- **Reformat tool**: New tool in the Tools tab that reformats existing memory files to the new topic-tagged format. Works on any input — unstructured text, old memory blocks, or already-formatted blocks. Includes preview dialog, editing, and one-click undo.
- **Recommended VS settings**: Diagnostics panel now includes a "Recommended Vector Storage Settings" card with optimal chunk size, retrieve chunks, score threshold, and other settings for CharMemory.
### Improvements
- **Tighter memory blocks**: Default bullet limit reduced from 8 to 5 per block (not counting the topic tag). Forces outcome-focused extraction rather than step-by-step narration.
- **Better consolidation labels**: Consolidation now uses encounter-specific labels (e.g., "Adoption day at the apartment") instead of broad categories (e.g., "Key Events"). Topic tags are preserved and updated during consolidation.
- **Improved health checks**: Chunk size recommendations now include specific guidance (800-1000 chars). New checks for retrieve chunks and score threshold (when accessible).
- **Named participants**: Extraction and consolidation prompts now require specific names instead of generic labels like "a client" or "someone".
### Migration
- Existing memories continue to work without changes. Use the new **Reformat** tool to add topic tags to existing memory files for improved retrieval.
- Users with customized extraction prompts are unaffected — only the default prompt is updated. Click "Restore Default" to opt into the new format.
```
**Step 3: Commit**
```bash
git add manifest.json CHANGELOG.md
git commit -m "chore: bump version to 1.7.0, add changelog"
```
### Task 15: Clean Up Proposed Files
**Files:**
- Delete: `docs/proposed-solo-extraction-prompt.txt`
- Delete: `docs/proposed-group-extraction-prompt.txt`
- Delete: `docs/proposed-consolidation-presets.txt`
- Delete: `docs/proposed-reformat-prompt.txt`
- Delete: `docs/proposed-migration-prompt.txt` (if it exists)
**Step 1: Remove proposed files that are now implemented**
```bash
rm docs/proposed-*.txt
```
**Step 2: Commit**
```bash
git add -A docs/proposed-*.txt
git commit -m "chore: remove proposed prompt files (now implemented)"
```
### Task 16: Final Test Run
**Step 1: Run all unit tests**
Run: `npm test`
Expected: All pass.
**Step 2: Run snapshot tests**
Run: `npm run test:snapshot`
Expected: All pass (snapshots were updated in Task 7).
**Step 3: Run sync check**
Run: `npm test -- --grep "sync"`
Expected: Pass — prompt constants are not duplicated in lib.js.
---
## Risk Mitigations
| Risk | Mitigation |
|------|-----------|
| Prompt changes produce bad extractions | Only affects default prompt; users with custom prompts unaffected. Beta branch testing period before merge to master. |
| Topic tags confuse the LLM during generation | Tags are inside `<memory>` blocks which are already structured data the LLM expects. The `[brackets]` signal metadata, not narrative. |
| 5-bullet limit loses important information | The prompt says "combine related facts into single bullets" not "delete". Test with varied content during beta. |
| Reformat tool destroys memories | Backup + undo mechanism (same pattern as consolidation). Preview dialog before saving. |
| parseMemories() breaks with topic tags | Verified: topic tags parse as regular bullets. No parsing changes needed. |
| Snapshot tests break unexpectedly | Phase 1 updates snapshots intentionally. Phase 2+ adds no snapshot-affecting changes. |
| VS settings not accessible for health checks | Task 11 includes a research step — if settings aren't accessible, we skip the check and rely on the recommendation card (Task 13) instead. |

784
docs/plans/2026-02-28-v1.8-tech-debt-plan.md Normal file
View file

@ -0,0 +1,784 @@
# v1.8 Tech Debt Cleanup — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Make `index.js` maintainable by eliminating code duplication and splitting monolithic functions, with zero visible UX changes.
**Architecture:** Four refactoring tasks: (1) import from `lib.js` instead of duplicating functions, (2) extract timestamp and clone utilities, (3) split `setupListeners()` into feature modules, (4) extract shared memory editor factory from three dialog implementations. Each task is independently testable — all 100 unit tests must pass after every commit.
**Tech Stack:** JavaScript (ES modules), jQuery, SillyTavern extension framework, Vitest
---
## Important Context
- `index.js` (~6,086 lines) is the entire runtime. `lib.js` (~438 lines) contains pure function copies for testing.
- SillyTavern loads extensions as ES modules (`type="module"`). `index.js` already imports from ST core. It CAN import from `lib.js`.
- `serializeMemories()` is **different** between the two files — the `index.js` version calls `getFormatOptions()` for runtime settings. Do NOT import this one from `lib.js`. Keep the `index.js` version.
- `mergeMemoryBlocks`, `detectFileFormat`, `truncateText`, and `reindexEditingSet` exist only in `lib.js` and are NOT duplicated in `index.js`. They only need importing if `index.js` calls them.
- All IDs are prefixed `charMemory_`. All console output uses `LOG_PREFIX = '[CharMemory]'`.
- Run tests with `npm test` (unit tests only, ~200ms). All 100 must pass after every commit.
---
## Task 1: Import Pure Functions from lib.js
Remove 11 duplicated function definitions from `index.js` and import them from `lib.js` instead. Remove the sync-check test that verified the duplication.
**Files:**
- Modify: `index.js:1-33` (import block), then remove functions at lines listed below
- Modify: `lib.js` (no changes needed — already exports everything)
- Delete test: `test/unit/sync-check.test.js`
### Step 1: Add the import statement to index.js
Add this import after the existing imports (after line 33):
```javascript
import {
escapeAttr,
unescapeAttr,
escapeHtml,
parseMemories,
splitMultiTagBullets,
countMemories,
mergeMemoryBlocks,
migrateMemoriesIfNeeded,
convertHeuristic,
stripNonDiegetic,
formatChatMessages,
substitutePromptTemplate,
truncateText,
reindexEditingSet,
} from './lib.js';
```
Note: `serializeMemories` and `detectFileFormat` are deliberately NOT imported. `serializeMemories` in `index.js` (line ~564) has different logic that uses `getFormatOptions()`. `detectFileFormat` is not called from `index.js`.
### Step 2: Remove the 11 duplicated function definitions from index.js
Remove these functions (keep the `index.js`-specific `serializeMemories`):
| Function | index.js location | Notes |
|----------|-------------------|-------|
| `escapeAttr()` | ~line 542 | 3-line function |
| `unescapeAttr()` | ~line 546 | 3-line function |
| `escapeHtml()` | ~line 3810 | 5-line function |
| `parseMemories()` | ~line 463 | ~40 lines with JSDoc |
| `splitMultiTagBullets()` | ~line 507 | ~25 lines with JSDoc |
| `countMemories()` | ~line 533 | ~8 lines with JSDoc |
| `migrateMemoriesIfNeeded()` | ~line 682 | ~70 lines |
| `convertHeuristic()` | ~line 757 | ~80 lines |
| `stripNonDiegetic()` | ~line 2051 | ~15 lines |
| `formatChatMessages()` | ~line 2068 | ~35 lines |
| `substitutePromptTemplate()` | ~line 2105 | ~15 lines |
**CRITICAL:** After removing each function, search index.js for all call sites to confirm nothing breaks. These are pure functions — no `this` binding, no closures over index.js state. The imported versions are identical.
### Step 3: Check for functions index.js calls but doesn't duplicate
Search index.js for calls to `mergeMemoryBlocks`, `truncateText`, and `reindexEditingSet`. These are already in lib.js but may be called in index.js without a local definition:
- `mergeMemoryBlocks` — search for usage. If called, the import in Step 1 covers it.
- `truncateText` — search for usage (~line 2708). The import covers it.
- `reindexEditingSet` — search for usage (~lines 1069, 4405, 4653, 4661). The import covers it.
### Step 4: Delete the sync-check test
Delete `test/unit/sync-check.test.js` entirely. This test verified that lib.js and index.js had matching function implementations. With imports, there's only one copy — the test is obsolete.
### Step 5: Run tests
```bash
npm test
```
Expected: All tests pass (will be fewer than 100 since sync-check tests are removed — should be ~97). No test should import from index.js — they all import from lib.js, which is unchanged.
### Step 6: Commit
```bash
git add index.js test/unit/sync-check.test.js
git commit -m "refactor: import pure functions from lib.js, remove duplicates
Import 14 functions from lib.js instead of maintaining local copies.
serializeMemories kept local (uses runtime getFormatOptions()).
Remove sync-check test (no longer needed with single source of truth)."
```
---
## Task 2: Extract Utility Functions
Add `getTimestamp()` and `cloneMemoryBlocks()` to `lib.js`, import in `index.js`, replace all inline copies.
**Files:**
- Modify: `lib.js` (add 2 new exports)
- Modify: `index.js` (replace 9-10 timestamp inlines, 3 cloneBlocks inlines)
- Create: `test/unit/utils-new.test.js` (or add to existing `utils.test.js`)
### Step 1: Write the failing tests
Add to `test/unit/utils.test.js`:
```javascript
import { getTimestamp, cloneMemoryBlocks } from '../../lib.js';
describe('getTimestamp', () => {
it('returns YYYY-MM-DD HH:MM format', () => {
const ts = getTimestamp();
expect(ts).toMatch(/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}$/);
});
it('accepts a Date argument', () => {
const ts = getTimestamp(new Date(2026, 0, 15, 9, 5));
expect(ts).toBe('2026-01-15 09:05');
});
});
describe('cloneMemoryBlocks', () => {
it('deep clones blocks without shared references', () => {
const original = [
{ chat: 'test', date: '2026-01-01', bullets: ['a', 'b'] },
];
const cloned = cloneMemoryBlocks(original);
cloned[0].bullets.push('c');
expect(original[0].bullets).toEqual(['a', 'b']);
});
it('preserves all block properties', () => {
const original = [
{ chat: 'test', date: '2026-01-01 14:30', bullets: ['a'] },
];
const cloned = cloneMemoryBlocks(original);
expect(cloned[0].chat).toBe('test');
expect(cloned[0].date).toBe('2026-01-01 14:30');
});
});
```
### Step 2: Run tests to verify they fail
```bash
npm test
```
Expected: FAIL — `getTimestamp` and `cloneMemoryBlocks` are not exported from lib.js.
### Step 3: Implement the functions in lib.js
Add before the final exports or at the end of lib.js:
```javascript
/**
* Generate a YYYY-MM-DD HH:MM timestamp string.
* @param {Date} [date] - Date to format. Defaults to now.
* @returns {string}
*/
export function getTimestamp(date) {
const now = date || new Date();
return `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, '0')}-${String(now.getDate()).padStart(2, '0')} ${String(now.getHours()).padStart(2, '0')}:${String(now.getMinutes()).padStart(2, '0')}`;
}
/**
* Deep-clone an array of memory blocks (shallow object clone + bullet array copy).
* @param {Array<{chat: string, date: string, bullets: string[]}>} blocks
* @returns {Array<{chat: string, date: string, bullets: string[]}>}
*/
export function cloneMemoryBlocks(blocks) {
return blocks.map(b => ({ ...b, bullets: [...b.bullets] }));
}
```
### Step 4: Run tests to verify they pass
```bash
npm test
```
Expected: All tests pass.
### Step 5: Import and replace in index.js
Add `getTimestamp` and `cloneMemoryBlocks` to the lib.js import statement (from Task 1).
Then replace all inline copies:
**Timestamp replacements** — search for `padStart(2, '0')` and replace each inline timestamp construction with `getTimestamp()`. There are ~9-10 instances. For each one:
- If it uses `new Date()`, replace with `getTimestamp()`
- If it uses a specific date variable, replace with `getTimestamp(dateVar)`
- **Exception:** `logActivity()` (~line 70) uses HH:MM:SS format (time only, with seconds). Do NOT replace this one — it's a different format.
**cloneBlocks replacements** — search for `blocks.map(b => ({ ...b, bullets: [...b.bullets] }))` and replace:
- `previewConversion()` ~line 1027: replace `const cloneBlocks = (blocks) => blocks.map(...)` with `// imported cloneMemoryBlocks from lib.js` and use `cloneMemoryBlocks` at all call sites within the function
- `consolidateMemories()` ~line 4346: same replacement
- `showReformatPreview()` ~line 4613: same replacement
### Step 6: Run tests
```bash
npm test
```
Expected: All tests pass.
### Step 7: Commit
```bash
git add lib.js index.js test/unit/utils.test.js
git commit -m "refactor: extract getTimestamp() and cloneMemoryBlocks() utilities
Replace 9 inline timestamp copies and 3 inline block-clone copies
with shared functions in lib.js."
```
---
## Task 3: Split setupListeners()
Break the 527-line `setupListeners()` into 5 feature-specific initializer functions. No behavior change.
**Files:**
- Modify: `index.js:4932-5451` (the `setupListeners()` function)
### Step 1: Create the 5 feature functions
Define 5 new functions immediately above `setupListeners()`. Each function contains the event handlers cut from `setupListeners()`. Use the handler map from the exploration:
**`setupConnectionControls()`** — Provider selection and configuration
- `#charMemory_source` change handler (~line 4967)
- `#charMemory_providerSelect` change handler (~line 4974)
- `#charMemory_providerApiKey` input handler (~line 4982)
- `#charMemory_providerConnect` click handler (~line 4989)
- `#charMemory_modelSearch` input/focus/keydown handlers (~lines 5020-5068)
- `#charMemory_modelDropdown` click handler (~line 5033)
- `$(document)` click handler for model picker dismiss (~line 5053)
- `#charMemory_providerModelInput` input handler (~line 5105)
- `#charMemory_providerRefreshModels` click handler (~line 5111)
- `#charMemory_providerBaseUrl` input handler (~line 5115)
- `#charMemory_providerSystemPrompt` input handler (~line 5121)
- `#charMemory_providerApiKeyReveal` click handler (~line 5127)
- `#charMemory_providerTest` click handler (~line 5145)
- `#charMemory_nanogptFilter*` change handlers (~lines 5147-5168)
**`setupExtractionControls()`** — Extraction settings and prompts
- `#charMemory_enabled` change handler (~line 4933)
- `#charMemory_interval` input handler (~line 4938)
- `#charMemory_maxMessages` input handler (~line 4946)
- `#charMemory_minCooldown` input handler (~line 4953)
- `#charMemory_responseLength` input handler (~line 4960)
- `#charMemory_extractionPrompt` input handler (~line 5180)
- `#charMemory_restorePrompt` click handler (~line 5185)
- `#charMemory_groupExtractionPrompt` input handler (~line 5192)
- `#charMemory_restoreGroupPrompt` click handler (~line 5197)
- `$(document).on('input', '.charMemory_groupMemberFile')` (~line 5205)
- `#charMemory_mergeChunks` change handler (~line 5316)
**`setupToolControls()`** — Consolidation, batch, convert tools
- `.charMemory_toolPill` click handler (~line 5359)
- `#charMemory_extractNow` click handler (~line 5246)
- `#charMemory_manageMemories` click handler (~line 5326)
- `#charMemory_consolidate` click handler (~line 5328)
- `#charMemory_undoConsolidate` click handler (~line 5329)
- `#charMemory_consolidationStrategy` change handler (~line 5219)
- `#charMemory_consolidationPrompt` input handler (~line 5226)
- `#charMemory_restorePresetDefault` click handler (~line 5237)
- `#charMemory_convertPreview` click handler (~line 5331)
- `#charMemory_undoReformat` click handler (~line 5332)
- `input[name="charMemory_formatSource"]` change handler (~line 5335)
- `#charMemory_convertPrompt` input handler (~line 5394)
- `#charMemory_restoreConvertPrompt` click handler (~line 5389)
- `#charMemory_batchRefresh` click handler (~line 5436)
- `#charMemory_batchExtract` click handler (~line 5437)
- `#charMemory_batchStop` click handler (~line 5438)
- `#charMemory_batchSelectAll` change handler (~line 5441)
- `$(document).on('change', '.charMemory_batchChatCheck')` (~line 5446)
**`setupStorageControls()`** — File name, per-chat, format settings
- `#charMemory_fileName` input handler (~line 5310)
- `#charMemory_perChat` change handler (~line 5321)
- `#charMemory_chunkBoundary` change handler (~line 5370)
- `#charMemory_customSeparator` input handler (~line 5378)
- `#charMemory_chunkMetadata` change handler (~line 5383)
**`setupLogControls()`** — Logging, diagnostics, tabs
- `.charMemory_tab` click handler (~line 5343)
- `#charMemory_verboseLog` change handler (~line 5175)
- `#charMemory_clearLog` click handler (~line 5413)
- `#charMemory_saveLog` click handler (~line 5418)
- `#charMemory_refreshDiag` click handler (~line 5398)
- `#charMemory_statHealth` click handler (~line 5404)
- `.charMemory_recommendationHeader` click handler (~line 5448)
- `#charMemory_resetTracking` click handler (~line 5250)
- `#charMemory_resetExtraction` click handler (~line 5272)
### Step 2: Replace setupListeners() body
Replace the body of `setupListeners()` with calls to the 5 functions:
```javascript
function setupListeners() {
setupConnectionControls();
setupExtractionControls();
setupToolControls();
setupStorageControls();
setupLogControls();
}
```
### Step 3: Run tests
```bash
npm test
```
Expected: All tests pass. Unit tests don't test event handlers directly — they test lib.js pure functions. But this confirms we didn't introduce syntax errors.
### Step 4: Manual verification
Open SillyTavern with the extension loaded. Verify:
- Settings tab: all sliders, dropdowns, checkboxes respond
- Provider: can switch providers, enter API key, connect
- Tools: all three pills work, buttons trigger dialogs
- Log: verbose toggle, clear log, diagnostics refresh
- Tab switching works
### Step 5: Commit
```bash
git add index.js
git commit -m "refactor: split setupListeners() into 5 feature modules
Break 527-line function into setupConnectionControls(),
setupExtractionControls(), setupToolControls(),
setupStorageControls(), and setupLogControls().
No behavior change."
```
---
## Task 4: Extract Shared Memory Editor Factory
Extract the duplicated dialog editor pattern from `previewConversion()`, `consolidateMemories()`, and `showReformatPreview()` into a shared `createMemoryEditor()` factory.
This is the most complex task. The three editors share ~80% of their event handling code (toggle edit, edit bullets, edit themes, delete bullets, delete blocks, add bullets, add blocks, undo). They differ in: event namespace, re-run behavior, destination handling, and cleanup.
**Files:**
- Create: `editor.js` (new module — the shared editor factory)
- Modify: `index.js` (import factory, refactor three callers)
- Create: `test/unit/editor.test.js` (test the factory's pure logic)
### Step 1: Write the failing tests for editor.js
Create `test/unit/editor.test.js`:
```javascript
import { createMemoryEditor } from '../../editor.js';
import { parseMemories } from '../../lib.js';
describe('createMemoryEditor', () => {
const sampleBlocks = [
{ chat: 'Test', date: '2026-01-01 14:00', bullets: ['- [A, B — test]', '- Fact one', '- Fact two'] },
{ chat: 'Other', date: '2026-01-02 10:00', bullets: ['- [C — other]', '- Something'] },
];
it('initializes with provided blocks', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.getBlocks()).toHaveLength(2);
expect(editor.getBlocks()[0].chat).toBe('Test');
});
it('getBlocks returns a deep clone, not a reference', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
const blocks = editor.getBlocks();
blocks[0].bullets.push('- New');
expect(editor.getBlocks()[0].bullets).toHaveLength(3);
});
it('deleteBullet removes the specified bullet', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBullet(0, 1);
expect(editor.getBlocks()[0].bullets).toEqual(['- [A, B — test]', '- Fact two']);
});
it('deleteBlock removes the specified block', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBlock(0);
expect(editor.getBlocks()).toHaveLength(1);
expect(editor.getBlocks()[0].chat).toBe('Other');
});
it('addBullet appends to the specified block', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.addBullet(0);
const bullets = editor.getBlocks()[0].bullets;
expect(bullets[bullets.length - 1]).toBe('- ');
});
it('addBlock appends a new empty block', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.addBlock();
const blocks = editor.getBlocks();
expect(blocks).toHaveLength(3);
expect(blocks[2].bullets).toEqual(['- ']);
});
it('addBlock uses provided timestamp', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.addBlock('2026-03-01 12:00');
expect(editor.getBlocks()[2].date).toBe('2026-03-01 12:00');
});
it('updateBullet changes the text', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.updateBullet(0, 1, '- Updated fact');
expect(editor.getBlocks()[0].bullets[1]).toBe('- Updated fact');
});
it('updateTheme changes the chat label', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.updateTheme(0, 'New Label');
expect(editor.getBlocks()[0].chat).toBe('New Label');
});
it('undo restores previous state', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBlock(0);
expect(editor.getBlocks()).toHaveLength(1);
editor.undo();
expect(editor.getBlocks()).toHaveLength(2);
});
it('undo returns false when no history', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.undo()).toBe(false);
});
it('canUndo reflects version stack state', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.canUndo()).toBe(false);
editor.deleteBlock(0);
expect(editor.canUndo()).toBe(true);
});
it('replaceAll swaps all blocks and clears undo history', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBlock(0);
const newBlocks = [{ chat: 'Fresh', date: '2026-06-01 08:00', bullets: ['- New'] }];
editor.replaceAll(newBlocks);
expect(editor.getBlocks()).toHaveLength(1);
expect(editor.getBlocks()[0].chat).toBe('Fresh');
expect(editor.canUndo()).toBe(false);
});
it('toggleEdit tracks which blocks are in edit mode', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.isEditing(0)).toBe(false);
editor.toggleEdit(0);
expect(editor.isEditing(0)).toBe(true);
editor.toggleEdit(0);
expect(editor.isEditing(0)).toBe(false);
});
});
```
### Step 2: Run tests to verify they fail
```bash
npm test
```
Expected: FAIL — `editor.js` doesn't exist.
### Step 3: Implement editor.js
Create `editor.js`:
```javascript
import { cloneMemoryBlocks, getTimestamp, reindexEditingSet } from './lib.js';
/**
* Create a memory block editor with state management and undo.
* Pure state logic — no DOM. Callers handle rendering and event binding.
*
* @param {object} options
* @param {Array<{chat: string, date: string, bullets: string[]}>} options.blocks - Initial blocks
* @returns {object} Editor API
*/
export function createMemoryEditor({ blocks }) {
let editorBlocks = cloneMemoryBlocks(blocks);
const versionStack = [];
const editingSet = new Set();
function saveVersion() {
versionStack.push(cloneMemoryBlocks(editorBlocks));
}
return {
/** Get a deep clone of the current blocks. */
getBlocks() {
return cloneMemoryBlocks(editorBlocks);
},
/** Delete a bullet from a block. */
deleteBullet(blockIndex, bulletIndex) {
saveVersion();
editorBlocks[blockIndex].bullets.splice(bulletIndex, 1);
if (editorBlocks[blockIndex].bullets.length === 0) {
editorBlocks.splice(blockIndex, 1);
reindexEditingSet(editingSet, blockIndex);
}
},
/** Delete an entire block. */
deleteBlock(blockIndex) {
saveVersion();
editorBlocks.splice(blockIndex, 1);
reindexEditingSet(editingSet, blockIndex);
},
/** Add a blank bullet to a block. */
addBullet(blockIndex) {
saveVersion();
editorBlocks[blockIndex].bullets.push('- ');
editingSet.add(blockIndex);
},
/** Add a new empty block. */
addBlock(timestamp) {
saveVersion();
editorBlocks.push({
chat: 'New block',
date: timestamp || getTimestamp(),
bullets: ['- '],
});
editingSet.add(editorBlocks.length - 1);
},
/** Update a bullet's text. */
updateBullet(blockIndex, bulletIndex, text) {
editorBlocks[blockIndex].bullets[bulletIndex] = text;
},
/** Update a block's chat/theme label. */
updateTheme(blockIndex, label) {
editorBlocks[blockIndex].chat = label;
},
/** Undo the last state-changing operation. Returns false if nothing to undo. */
undo() {
if (versionStack.length === 0) return false;
editorBlocks = versionStack.pop();
return true;
},
/** Whether undo is available. */
canUndo() {
return versionStack.length > 0;
},
/** Replace all blocks (e.g. after LLM re-run). Clears undo history. */
replaceAll(newBlocks) {
editorBlocks = cloneMemoryBlocks(newBlocks);
versionStack.length = 0;
editingSet.clear();
},
/** Toggle edit mode for a block. */
toggleEdit(blockIndex) {
if (editingSet.has(blockIndex)) {
editingSet.delete(blockIndex);
} else {
editingSet.add(blockIndex);
}
},
/** Check if a block is in edit mode. */
isEditing(blockIndex) {
return editingSet.has(blockIndex);
},
/** Get the editing set (for rendering). */
getEditingSet() {
return new Set(editingSet);
},
};
}
```
### Step 4: Run tests to verify they pass
```bash
npm test
```
Expected: All tests pass.
### Step 5: Commit the editor module and tests
```bash
git add editor.js test/unit/editor.test.js
git commit -m "feat: add shared memory editor factory with tests
createMemoryEditor() encapsulates block state management, undo,
and edit-mode tracking. Pure state logic, no DOM dependency.
Will replace duplicated editor code in 3 dialog functions."
```
### Step 6: Integrate into previewConversion()
**File:** `index.js` — `previewConversion()` (~lines 951-1213)
Add import at top of index.js:
```javascript
import { createMemoryEditor } from './editor.js';
```
Replace the closure state variables (~lines 1021-1027):
```javascript
// BEFORE:
let editorBlocks = result.blocks.map(...);
const versionStack = [];
const editingSet = new Set();
const cloneBlocks = (blocks) => blocks.map(b => ({ ...b, bullets: [...b.bullets] }));
// AFTER:
const editor = createMemoryEditor({ blocks: result.blocks });
```
Replace each event handler's inline logic with editor API calls. For example:
**Delete bullet** (~line 1062):
```javascript
// BEFORE:
versionStack.push(cloneBlocks(editorBlocks));
editorBlocks[bi].bullets.splice(bul, 1);
if (editorBlocks[bi].bullets.length === 0) {
editorBlocks.splice(bi, 1);
reindexEditingSet(editingSet, bi);
}
// AFTER:
editor.deleteBullet(bi, bul);
```
**Undo** (~line 1158):
```javascript
// BEFORE:
if (versionStack.length > 0) {
editorBlocks = versionStack.pop();
refreshEditor();
}
// AFTER:
if (editor.undo()) {
refreshEditor();
}
```
**refreshEditor()** — update to use `editor.getBlocks()` and `editor.getEditingSet()` instead of the local variables.
**Re-run handler** (~line 1164) — after getting new blocks from LLM, call `editor.replaceAll(newBlocks)`.
Update all references from `editorBlocks` to `editor.getBlocks()` and from `editingSet` to `editor.getEditingSet()`.
### Step 7: Run tests and manually verify conversion dialog
```bash
npm test
```
Then manually test: Tools → Convert → select a Data Bank file → Preview → verify editing, deleting, adding, undo, and re-run all work.
### Step 8: Commit
```bash
git add index.js
git commit -m "refactor: use shared editor in previewConversion()
Replace inline state management with createMemoryEditor().
Same behavior, shared code."
```
### Step 9: Integrate into consolidateMemories()
**File:** `index.js` — `consolidateMemories()` (~lines 4291-4520)
Same pattern as Step 6. Replace closure variables with `createMemoryEditor()`. Replace inline delete/add/undo logic with editor API calls. Update `refreshEditor()` to use `editor.getBlocks()` and `editor.getEditingSet()`.
Note: consolidation has no re-run button, so `editor.replaceAll()` is not used here.
### Step 10: Run tests and manually verify consolidation dialog
```bash
npm test
```
Manually test: Tools → Consolidate → run consolidation → verify editing, undo, save.
### Step 11: Commit
```bash
git add index.js
git commit -m "refactor: use shared editor in consolidateMemories()
Replace inline state management with createMemoryEditor().
Same behavior, shared code."
```
### Step 12: Integrate into showReformatPreview()
**File:** `index.js` — `showReformatPreview()` (~lines 4605-4760)
Same pattern. Replace closure variables with `createMemoryEditor()`. Replace inline logic with editor API calls. This one has a re-run button — use `editor.replaceAll(newBlocks)` after LLM response.
Note the `dialogClosed` flag (~line 4612) — keep this as a local variable, it's not part of the editor state.
### Step 13: Run tests and manually verify reformat dialog
```bash
npm test
```
Manually test: Tools → Convert → source "Current memories" → Preview → verify editing, undo, re-run.
### Step 14: Commit
```bash
git add index.js
git commit -m "refactor: use shared editor in showReformatPreview()
Replace inline state management with createMemoryEditor().
All three dialog editors now use the shared factory."
```
### Step 15: Final verification
Run full test suite:
```bash
npm test
```
Verify total test count has increased (new editor tests) minus sync-check tests removed. Expected: ~110+ tests passing.
Manually test all three dialog flows one more time to confirm no regressions.
---
## Summary
| Task | What | Lines affected | Risk |
|------|------|---------------|------|
| 1 | Import from lib.js | ~300 lines removed from index.js | Low — identical functions |
| 2 | Utility extractions | ~50 lines removed, 2 new in lib.js | Low — mechanical replacement |
| 3 | Split setupListeners | ~527 lines reorganized | Medium — must preserve all handler bindings |
| 4 | Shared editor factory | ~300 lines of duplication → 1 module | High — must preserve 3 dialog behaviors |
**After all tasks:**
- `index.js` shrinks by ~600+ lines
- `lib.js` gains 2 utility functions
- New `editor.js` module (~80 lines) replaces ~300 lines of duplication
- Test count increases (new editor tests, minus sync-check)
- All 3 dialog editors share state management code
- `setupListeners()` is a 6-line coordinator
- Zero visible UX changes

369
docs/plans/2026-02-28-v2-ux-redesign-design.md Normal file
View file

@ -0,0 +1,369 @@
# CharMemory v1.8 + v2.0 Design Document
## Goal
Simplify the UX for new users while keeping full control for advanced users. Fix accumulated tech debt that makes the codebase hard to maintain. Ship in two phases: v1.8 (internals only, no visible changes) and v2.0 (full UX redesign).
## Current Problems
**UX:**
- 53+ controls spread across 4 tabs and 3 sub-tabs in a narrow sidebar
- No first-run experience — new users see the full complexity immediately
- 4 prompts scattered across Settings and Tools tabs, no central overview
- Prompt updates in new versions are silently ignored for users with custom prompts
- Troubleshooting requires digging through Settings → Advanced for reset/clear actions
- Data Bank files not directly accessible from CharMemory UI
**Tech debt:**
- 6,086-line monolithic `index.js` with 203 functions
- `setupListeners()` is 527 lines wiring all UI events
- 3 near-identical dialog editors (~300 lines duplicated)
- 9 copies of timestamp formatting, 3 copies of block cloning
- `lib.js` duplicates 15 functions from `index.js` — unnecessary since ST supports ES module imports
## Architecture Decision: Modals Over Tabs
The sidebar panel (~300px wide) is the wrong place for configuration. It's good for monitoring and quick actions while chatting. All configuration, editing, and tool workflows move to center-screen modals where there's room to work. This matches how users actually operate: monitoring happens alongside chat, configuring does not.
SillyTavern extensions already create modals (we do it for consolidation, conversion, and the Injection Viewer). This makes modals the primary UI pattern instead of an exception.
---
## Phase 1: v1.8 — Tech Debt Cleanup
No visible UX changes. The goal is to make `index.js` maintainable before rebuilding the UI.
### 1.1 ES Module Imports
`index.js` imports pure functions from `lib.js` instead of duplicating them. Functions to import:
- `escapeAttr()`, `unescapeAttr()`, `escapeHtml()`
- `parseMemories()`, `splitMultiTagBullets()`, `countMemories()`, `serializeMemories()`
- `mergeMemoryBlocks()`, `migrateMemoriesIfNeeded()`
- `detectFileFormat()`, `convertHeuristic()`
- `stripNonDiegetic()`, `formatChatMessages()`, `substitutePromptTemplate()`
- `truncateText()`, `reindexEditingSet()`
The sync-check test (`test/unit/sync-check.test.js`) becomes unnecessary and is removed. `lib.js` becomes the single source of truth.
### 1.2 Shared Editor Factory
Extract `createMemoryEditor(options)` from the three duplicated dialog editors:
- `consolidateMemories()` (lines 4291–4554)
- `previewConversion()` (lines 951–1252)
- `showReformatPreview()` (lines 4605–4764)
The factory accepts options (event namespace, save/undo/rerun callbacks, initial blocks) and returns:
- `render(blocks, editingSet)` → HTML string
- `attachHandlers(container)` → wires event delegation
- `getState()` → current editor blocks
- `cleanup()` → removes event handlers
Each caller passes its specific options. The identical toggle/edit/delete/add-bullet logic lives in one place.
### 1.3 Split setupListeners()
Break the 527-line function into feature-specific initializers:
- `setupConnectionControls()` — provider dropdown, API key, connect, model picker
- `setupExtractionControls()` — interval slider, cooldown, chunk size, response length
- `setupToolControls()` — consolidation, batch, convert pill switching and buttons
- `setupStorageControls()` — per-chat toggle, file name, format settings
- `setupLogControls()` — verbose toggle, clear log
`setupListeners()` becomes a coordinator that calls these five functions.
### 1.4 Utility Extractions
**`getTimestamp()`** — replaces 9 inline copies of the `getFullYear()-padStart` pattern. Lives in `lib.js`, imported everywhere.
**`cloneMemoryBlocks(blocks)`** — replaces 3 copies of `blocks.map(b => ({ ...b, bullets: [...b.bullets] }))`. Lives in `lib.js`.
---
## Phase 2: v2.0 — UX Redesign
### 2.1 Sidebar Dashboard
The sidebar collapses to a single view — no tabs. It's a monitoring dashboard and launcher:
```
┌───────────────────────────────┐
│ CharMemory [⚙️] │
├───────────────────────────────┤
│ 🔢 169 memories ⏱ 3/20 msgs │
│ ⚡ Ready ● Healthy │
├───────────────────────────────┤
│ 📄 Flux_the_Cat-memories.md │
│ 42 KB • 11 chunks │
│ [View / Edit] [Files ▾] │
├───────────────────────────────┤
│ ☑ Automatic extraction │
│ [Extract Now] │
├───────────────────────────────┤
│ Tools │
│ [Consolidate] [Batch] [Convert│
├───────────────────────────────┤
│ Activity │
│ 16:07 Extracted 4 memories │
│ 15:42 Extracted 3 memories │
│ [View full log →] │
├───────────────────────────────┤
│ Diagnostics [Refresh] │
│ ✅ Healthy — 7/7 checks pass │
│ [View details →] │
├───────────────────────────────┤
│ [🔧 Help, it's not working] │
└───────────────────────────────┘
```
**Elements:**
- **Gear icon** (top right) → opens Settings modal
- **Stats bar** — memory count, message counter, extraction status, health dot
- **File section** — active memory file with metadata (size, chunk count). **[View / Edit]** opens memory manager. **[Files]** opens a popover listing all Data Bank files for this character with View/Export/Delete/Convert actions.
- **Automatic extraction** toggle + Extract Now button
- **Tool launchers** — Consolidate, Batch, Convert. Each opens its existing modal/dialog.
- **Activity** — last 2-3 log entries inline. "View full log" opens Log drawer.
- **Diagnostics** — summary line. "View details" expands or opens full health check list.
- **Troubleshooter** button — "Help, it's not working" opens the Troubleshooter modal. Also triggered by clicking health dot when issues exist.
### 2.2 Setup Wizard
Center-screen modal with smart triggering.
**Full wizard trigger:** `extension_settings.charMemory` has no provider configured (first launch or fresh install).
**Light nudge trigger:** Health check detects a fixable issue (Vector Storage off, no API key, file not vectorized). Shows a banner on the sidebar: "Something needs attention — [Fix now]" which opens to the relevant wizard step.
**Step 1: LLM Connection**
- Explanation: "CharMemory uses a separate LLM to read your chats and extract memories. This keeps your main LLM's context clean."
- Provider dropdown (Pollinations highlighted for zero-friction start)
- API key field (hidden for providers that don't need one)
- Connect & Test button with inline success/failure
- Model auto-selected, option to change
- [Next →]
**Step 2: Vector Storage**
- Explanation: "Memories are stored in your character's Data Bank. Vector Storage searches them and injects the relevant ones when your character responds."
- Auto-detect current VS configuration using existing health checks
- Green checks for passing, amber warnings for issues
- Each warning has a [Fix] button where possible
- If can't auto-fix, explain what to change and where
- [← Back] [Next →]
**Step 3: Ready**
- Summary of configuration
- Explain what happens next: extraction interval, Extract Now, syringe icon for checking injections, health dot for monitoring
- [Get Started] closes wizard
- Stores `wizardCompleted: true`
**Step 4: Verify It's Working** (triggered after first successful extraction, not during initial wizard)
- "Your first memories were just extracted! Let's make sure retrieval is working."
- Guide: send a message referencing chat history, check syringe icon
- Quick diagnostic:
- "See relevant memories? You're good."
- "See irrelevant memories? Score threshold may be too low — [Open Settings]"
- "See nothing? File may not be vectorized — [Run health check]"
- "Not sure? Paste injected content + memory file into an LLM to evaluate."
- [Got it] dismisses. Stores `verificationSeen: true`.
**Re-entry:** "Run Setup Wizard" link in Settings modal → Connection section.
### 2.3 Settings Modal
Center-screen modal, opened by gear icon on sidebar. Left sidebar nav, content on right.
```
┌──────────────────────────────────────────────────┐
│ Settings [X] │
├──────────────┬───────────────────────────────────┤
│ │ │
│ Connection │ (content for selected section) │
│ Extraction │ │
│ Storage │ │
│ Prompts │ │
│ Advanced │ │
│ │ │
└──────────────┴───────────────────────────────────┘
```
**Connection section:**
- LLM source dropdown (Dedicated API / Main LLM / WebLLM)
- Provider dropdown, API key, Connect button, model picker
- System prompt override
- "Run Setup Wizard" link at bottom
**Extraction section:**
- Auto-extraction: interval slider, cooldown slider
- Messages per LLM call, max response length, merge chunks toggle
- "Extraction Prompt (1:1)" — one-line summary + version badge + **[View / Edit]** → opens Prompts modal
- "Extraction Prompt (Group)" — same pattern
**Storage section:**
- Per-chat memories toggle with explanation
- File name field
- Group chat member files (shown when in group chat)
**Prompts section:**
- Overview of all 4 prompts with version badges and customization status
- Each has **[View / Edit]** → opens Prompts modal to that prompt
- Update notifications appear here: "Extraction prompt updated in v2.0 — your custom version is unchanged."
**Advanced section:**
- Memory File Format (chunk boundary, custom separator, metadata prefix) — with note: "Most users don't need to change these. The default block-level format works well with topic-tagged memories."
- Reset extraction state, Clear all memories (with confirmation dialogs)
- Note: Reset/Clear also accessible from Troubleshooter
### 2.4 Prompts Modal
Full-screen modal, opened from **[View / Edit]** buttons in Settings or from update notifications.
```
┌──────────────────────────────────────────────────┐
│ Prompts [X] │
├──────────┬───────────────────────────────────────┤
│ │ │
│ Extract │ Extraction Prompt (1:1) │
│ (1:1) │ v1.7.0 • Default │
│ │ │
│ Extract │ ┌───────────────────────────────────┐│
│ (Group) │ │ ││
│ │ │ (full prompt text, editable) ││
│ Consoli- │ │ ││
│ dation │ │ ││
│ │ └───────────────────────────────────┘│
│ Convert │ │
│ │ [Restore Default] [Save] │
└──────────┴───────────────────────────────────────┘
```
**Features:**
- Left nav lists all 4 prompts. Click to switch.
- Version badge: which default version this is based on, whether it's default or custom
- Full-width textarea with enough height to read the prompt
- [Restore Default] resets to current version's default
- [Save] saves edits
**Prompt version tracking:**
- Each default prompt has a version number stored in code (e.g. `PROMPT_VERSIONS.extraction = '1.7.0'`)
- User's settings store the version they're based on: `extension_settings.charMemory.promptVersions.extraction = '1.7.0'`
- On extension load, compare: if code version > stored version and user has a custom prompt, show update banner
**When default prompt updated:**
```
┌─────────────────────────────────────────────┐
│ ℹ️ The default prompt was updated in v2.0. │
│ Your custom prompt is unchanged. │
│ [Keep mine] [Use new default] │
│ [Compare & Edit →] │
└─────────────────────────────────────────────┘
```
**Compare & Edit** replaces the single textarea with two panes:
- Left: user's current prompt (editable)
- Right: new default prompt (read-only reference)
- User can copy/merge sections from new default into their version
- Reuses the side-by-side pattern from the Convert tool preview
### 2.5 Troubleshooter
Center-screen modal, opened from sidebar button or health dot when issues detected.
**Automated checks (run in sequence):**
1. LLM connection — can we reach the provider?
2. Extraction history — have memories been extracted? If not, why?
3. Memory file — does the Data Bank file exist and have content?
4. Vectorization — is the file vectorized? How many chunks?
5. Injection — were memories injected in the last response? Scores?
6. Score threshold — is it filtering out too much or too little?
Each check shows: pass/fail status, explanation, and fix button where possible.
**Data Bank browser:**
Below the health checks, a list of all Data Bank files for the character:
```
┌──────────────────────────────────────────┐
│ 📄 Flux_the_Cat-memories.md 42 KB │
│ 169 memories • vectorized • 11 chunks │
│ [View] [Edit] [Export] [Delete] │
├──────────────────────────────────────────┤
│ 📄 Flux-backstory-notes.txt 3 KB │
│ Not a CharMemory file │
│ [View] [Export] [Convert] [Delete] │
└──────────────────────────────────────────┘
[Import file]
```
Actions:
- **View** — read-only modal with memory block parsing/highlighting for CharMemory files
- **Edit** — opens memory manager
- **Export** — downloads the file
- **Delete** — confirmation dialog, then removes from Data Bank
- **Convert** — for non-CharMemory files, launches Convert tool
- **Import** — upload a file into the character's Data Bank
**Diagnostic report:**
"Copy diagnostic report" button at the bottom bundles: settings snapshot, last activity log entries, health check results, memory count, VS configuration, last injection data. User can paste into an LLM for analysis or share when asking for help.
**Reset/Clear actions** accessible here for troubleshooting convenience (with confirmation dialogs). Same actions as Settings → Advanced.
### 2.6 Log Drawer
Slide-out drawer (same pattern as Injection Viewer), opened from "View full log" on sidebar.
- Full activity log with timestamps
- Verbose toggle
- Clear log button
- Replaces the current Log tab
---
## What Moves Where (Migration Map)
| Current location | v2.0 location |
|---|---|
| Main tab → Extract Now, View/Edit | Sidebar dashboard |
| Main tab → Activity Log | Sidebar (summary) + Log drawer (full) |
| Main tab → Diagnostics | Sidebar (summary) + Troubleshooter (full) |
| Tools tab → Consolidate pill | Sidebar tool launcher → existing modal |
| Tools tab → Batch pill | Sidebar tool launcher → existing modal |
| Tools tab → Convert pill | Sidebar tool launcher → existing modal |
| Settings tab → LLM source, provider | Settings modal → Connection |
| Settings tab → Interval, cooldown | Settings modal → Extraction |
| Settings tab → Chunk size, response length | Settings modal → Extraction |
| Settings tab → Extraction prompt (1:1) | Settings modal → Extraction → [View/Edit] → Prompts modal |
| Settings tab → Extraction prompt (group) | Settings modal → Extraction → [View/Edit] → Prompts modal |
| Settings tab → Per-chat, file name | Settings modal → Storage |
| Settings tab → Memory File Format | Settings modal → Advanced |
| Settings tab → Reset, Clear | Settings modal → Advanced + Troubleshooter |
| Log tab → Activity log, verbose | Log drawer |
| Log tab → Diagnostics | Troubleshooter |
| Consolidation strategy dropdown | Consolidation modal (already there) |
| Conversion prompt | Settings modal → Prompts → [View/Edit] → Prompts modal |
| (new) Data Bank file browser | Sidebar [Files] popover + Troubleshooter |
| (new) Setup wizard | Auto-triggered modal |
| (new) Prompt version tracking | Prompts modal |
| (new) Diagnostic report export | Troubleshooter |
## What's NOT Changing
- Injection Viewer (syringe icon, slide-out drawer) — already good UX
- Message action buttons (Extract Here, Pin) — per-message, not panel UI
- Memory Manager dialog (View/Edit) — works well, just gets more entry points
- Consolidation/Conversion/Reformat dialog internals — already modal, just launched differently
- Provider presets and API architecture — purely internal
- Memory format and extraction pipeline — no changes
## Feature Preservation
No features are removed. Low-use features are preserved but organized for discoverability:
- **Memory File Format** (chunk boundary, custom separator, metadata) → Settings → Advanced, with note that most users don't need to change these
- **WebLLM source** → still in Settings → Connection dropdown
- **Main LLM source** → still in Settings → Connection dropdown, with existing warning about context pollution
- **Merge chunks toggle** → Settings → Extraction
- **NanoGPT model filters** → Settings → Connection (shown when NanoGPT selected)

985
docs/plans/2026-02-28-v2-ux-redesign-plan.md Normal file
View file

@ -0,0 +1,985 @@
# v2.0 UX Redesign — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Replace the 4-tab sidebar with a single-view dashboard and center-screen modals, creating a guided first-run experience while preserving all features.
**Architecture:** Build modals first (additive, non-breaking), then rebuild the sidebar (destructive, replaces tabs). Each modal is self-contained HTML/CSS/JS that can be tested independently before the sidebar migration. The Injection Viewer drawer pattern is reused for the Log Drawer.
**Tech Stack:** JavaScript (ES modules), jQuery, SillyTavern extension framework (callGenericPopup, POPUP_TYPE), CSS with ST theme variables
---
## Important Context
- `callGenericPopup(html, POPUP_TYPE.TEXT, '', { wide: true, allowVerticalScrolling: true })` creates a center-screen modal. The HTML can be anything — we build left-nav layouts inside.
- `POPUP_TYPE.TEXT` = display-only modal (no OK/Cancel). `POPUP_TYPE.CONFIRM` = modal with OK/Cancel buttons. We use TEXT for settings/prompts (save buttons are in our HTML), CONFIRM for dialogs needing accept/reject.
- All CSS classes use `charMemory_` prefix. All element IDs use `charMemory_` prefix.
- Theme variables: `var(--SmartThemeBlurTintColor)` (background), `var(--SmartThemeBorderColor)` (borders), `var(--SmartThemeQuoteColor)` (accent), `var(--SmartThemeBodyColor)` (text).
- Working directory: `/Users/davidsayed/repos/sillytavern-character-memory` on the `beta` branch.
- Run `npm test` after every commit. All 117 unit tests must pass.
- The current `settings.html` (448 lines), `style.css` (1,072 lines), and `index.js` (5,709 lines) are the starting points.
---
## Task 1: Settings Modal
Build the center-screen Settings modal with left sidebar nav (Connection, Extraction, Storage, Prompts, Advanced). Opened by a gear icon. Does NOT remove the existing Settings tab yet — both coexist until the sidebar rebuild.
**Files:**
- Modify: `settings.html` (add gear icon to header, add modal template)
- Modify: `style.css` (add modal and nav styles)
- Modify: `index.js` (add `showSettingsModal()`, wire gear icon, populate settings into modal)
### Step 1: Add the Settings modal HTML template
Add to `settings.html` at the very end (after the existing bottom diagnostics section), a hidden template div that `showSettingsModal()` will clone and populate:
```html
<!-- Settings Modal Template -->
<div id="charMemory_settingsModalTemplate" style="display:none;">
<div class="charMemory_modal">
<div class="charMemory_modalNav">
<button class="charMemory_modalNavItem active" data-section="connection">Connection</button>
<button class="charMemory_modalNavItem" data-section="extraction">Extraction</button>
<button class="charMemory_modalNavItem" data-section="storage">Storage</button>
<button class="charMemory_modalNavItem" data-section="prompts">Prompts</button>
<button class="charMemory_modalNavItem" data-section="advanced">Advanced</button>
</div>
<div class="charMemory_modalContent">
<!-- Each section is a div, shown/hidden by nav clicks -->
<div class="charMemory_modalSection active" data-section="connection">
<!-- LLM source dropdown, provider settings, API key, connect, model picker, system prompt -->
<!-- "Run Setup Wizard" link at bottom -->
</div>
<div class="charMemory_modalSection" data-section="extraction">
<!-- Auto-extraction: interval slider, cooldown slider -->
<!-- Messages per LLM call, max response length, merge chunks toggle -->
<!-- Extraction Prompt (1:1): summary line + [View / Edit] button -->
<!-- Extraction Prompt (Group): summary line + [View / Edit] button -->
</div>
<div class="charMemory_modalSection" data-section="storage">
<!-- Per-chat memories toggle -->
<!-- File name field -->
<!-- Group chat member files (shown when in group chat) -->
</div>
<div class="charMemory_modalSection" data-section="prompts">
<!-- Overview of all 4 prompts with version badges and customization status -->
<!-- Each has [View / Edit] → opens Prompts modal -->
</div>
<div class="charMemory_modalSection" data-section="advanced">
<!-- Memory File Format (chunk boundary, custom separator, metadata prefix) -->
<!-- Reset extraction state, Clear all memories (with confirmation dialogs) -->
</div>
</div>
</div>
</div>
```
The actual form controls (sliders, dropdowns, checkboxes) should be the SAME controls as the current Settings tab — same IDs, same structure. They're being moved, not rebuilt. The key difference is the container layout (left-nav modal vs tab content).
### Step 2: Add CSS for the modal layout
Add to `style.css`:
```css
/* ── Settings/Prompts Modal ── */
.charMemory_modal {
display: flex;
min-height: 400px;
max-height: 70vh;
gap: 0;
}
.charMemory_modalNav {
display: flex;
flex-direction: column;
min-width: 130px;
border-right: 1px solid var(--SmartThemeBorderColor);
padding: 8px 0;
flex-shrink: 0;
}
.charMemory_modalNavItem {
background: none;
border: none;
padding: 8px 16px;
text-align: left;
cursor: pointer;
opacity: 0.6;
font-size: 0.9em;
border-left: 3px solid transparent;
color: var(--SmartThemeBodyColor);
}
.charMemory_modalNavItem:hover {
opacity: 0.85;
background: rgba(255,255,255,0.03);
}
.charMemory_modalNavItem.active {
opacity: 1;
border-left-color: var(--SmartThemeQuoteColor);
background: rgba(255,255,255,0.05);
}
.charMemory_modalContent {
flex: 1;
padding: 12px 16px;
overflow-y: auto;
min-width: 0;
}
.charMemory_modalSection {
display: none;
}
.charMemory_modalSection.active {
display: block;
}
```
### Step 3: Add gear icon to sidebar header
In `settings.html`, add a gear icon button to the extension header area (next to the "CharMemory" title in the inline-drawer header). The exact location depends on the current header structure.
### Step 4: Implement `showSettingsModal()` in index.js
Create function that:
1. Clones the template HTML
2. Populates all form controls with current settings values (reuse patterns from `loadSettings()`)
3. Opens via `callGenericPopup(html, POPUP_TYPE.TEXT, '', { wide: true, allowVerticalScrolling: true })`
4. Wires nav click handlers (show/hide sections)
5. Wires all form control change handlers (save on change, same as current `setupExtractionControls()` etc.)
6. Wires the [View / Edit] prompt buttons to call `showPromptsModal()` (Task 2)
### Step 5: Wire gear icon click handler
In `setupLogControls()` (or a new `setupDashboardControls()`), add click handler for the gear icon that calls `showSettingsModal()`.
### Step 6: Test manually
- Click gear icon → Settings modal opens
- Left nav works — switching between Connection, Extraction, Storage, Prompts, Advanced
- All settings controls respond and save
- Provider switching, connect, model picker all work inside the modal
- [View / Edit] buttons exist (will be wired in Task 2)
- Close modal → settings are preserved
### Step 7: Commit
```bash
git add settings.html style.css index.js
git commit -m "feat: add Settings modal with left-nav layout
Center-screen modal opened by gear icon. Contains Connection,
Extraction, Storage, Prompts overview, and Advanced sections.
Coexists with existing Settings tab until sidebar rebuild."
```
---
## Task 2: Prompts Modal
Build the Prompts modal — full-screen modal for viewing/editing all 4 prompts. Opened from [View / Edit] buttons in the Settings modal's Extraction and Prompts sections.
**Files:**
- Modify: `settings.html` (add prompts modal template)
- Modify: `style.css` (add prompts modal styles)
- Modify: `index.js` (add `showPromptsModal(activePrompt)`, wire View/Edit buttons)
### Step 1: Add Prompts modal HTML template
Add to `settings.html`:
```html
<div id="charMemory_promptsModalTemplate" style="display:none;">
<div class="charMemory_modal charMemory_promptsModal">
<div class="charMemory_modalNav">
<button class="charMemory_modalNavItem active" data-prompt="extraction">Extract (1:1)</button>
<button class="charMemory_modalNavItem" data-prompt="groupExtraction">Extract (Group)</button>
<button class="charMemory_modalNavItem" data-prompt="consolidation">Consolidation</button>
<button class="charMemory_modalNavItem" data-prompt="conversion">Convert</button>
</div>
<div class="charMemory_modalContent">
<div class="charMemory_promptHeader">
<h3 id="charMemory_promptTitle">Extraction Prompt (1:1)</h3>
<span class="charMemory_promptBadge" id="charMemory_promptBadge">v1.7.0 • Default</span>
</div>
<textarea id="charMemory_promptEditor" class="text_pole" rows="20"></textarea>
<div class="charMemory_buttonRow" style="margin-top: 8px;">
<input type="button" id="charMemory_promptRestore" class="menu_button" value="Restore Default" />
<input type="button" id="charMemory_promptSave" class="menu_button" value="Save" />
</div>
</div>
</div>
</div>
```
### Step 2: Add CSS for prompts modal
```css
.charMemory_promptsModal {
min-height: 500px;
}
.charMemory_promptHeader {
display: flex;
align-items: center;
gap: 12px;
margin-bottom: 8px;
}
.charMemory_promptHeader h3 {
margin: 0;
font-size: 1em;
}
.charMemory_promptBadge {
font-size: 0.8em;
opacity: 0.6;
padding: 2px 8px;
border: 1px solid var(--SmartThemeBorderColor);
border-radius: 10px;
}
#charMemory_promptEditor {
width: 100%;
min-height: 300px;
font-family: monospace;
font-size: 0.85em;
resize: vertical;
}
```
### Step 3: Implement `showPromptsModal(activePrompt)`
Create function that:
1. `activePrompt` parameter: `'extraction'`, `'groupExtraction'`, `'consolidation'`, or `'conversion'`
2. Clones the template HTML
3. Sets the active nav item based on `activePrompt`
4. Loads the prompt text into the textarea
5. Shows the version badge (default vs custom)
6. Opens via `callGenericPopup(html, POPUP_TYPE.TEXT, '', { wide: true, allowVerticalScrolling: true })`
7. Wires nav click handlers (switch between prompts, save current before switching)
8. Wires Save button (save to `extension_settings`)
9. Wires Restore Default button (confirm, then restore default prompt text)
### Step 4: Wire [View / Edit] buttons in Settings modal
In `showSettingsModal()`, add click handlers for the prompt View/Edit buttons that call `showPromptsModal('extraction')` etc.
### Step 5: Define prompt metadata
Add a `PROMPT_CONFIG` object to index.js that maps prompt keys to their display names, default values, and settings keys:
```javascript
const PROMPT_CONFIG = {
extraction: {
title: 'Extraction Prompt (1:1)',
settingsKey: 'extractionPrompt',
defaultValue: defaultExtractionPrompt,
version: '1.7.0',
},
groupExtraction: {
title: 'Extraction Prompt (Group)',
settingsKey: 'groupExtractionPrompt',
defaultValue: defaultGroupExtractionPrompt,
version: '1.7.0',
},
consolidation: {
title: 'Consolidation Prompt',
settingsKey: 'consolidationPrompt', // per-strategy
defaultValue: null, // depends on strategy
version: '1.7.0',
},
conversion: {
title: 'Conversion Prompt',
settingsKey: 'convertPrompt',
defaultValue: defaultConversionPrompt,
version: '1.7.0',
},
};
```
### Step 6: Test manually
- Open Settings modal → Extraction section → click [View / Edit] next to Extraction Prompt
- Prompts modal opens with the correct prompt loaded
- Switch between 4 prompts using left nav
- Edit a prompt → Save → close → reopen → edit is preserved
- Restore Default → confirm → prompt reverts to default
### Step 7: Commit
```bash
git add settings.html style.css index.js
git commit -m "feat: add Prompts modal with full-width editor
Four-prompt editor with left nav, version badges,
Save and Restore Default buttons. Opened from Settings modal."
```
---
## Task 3: Log Drawer
Build the slide-out Log Drawer (same pattern as Injection Viewer). Opened from "View full log" link. Replaces the current Log tab.
**Files:**
- Modify: `index.js` (add `toggleLogDrawer()`, `updateLogDrawer()`, wire triggers)
- Modify: `style.css` (add log drawer styles, reuse injection drawer patterns)
### Step 1: Add Log Drawer HTML
The drawer HTML is built in JS (same as the Injection Viewer — appended to `$('body')`). Add it in the jQuery ready handler:
```javascript
$('body').append(`
<div id="charMemory_logDrawer" class="charMemory_logDrawer">
<div class="charMemory_drawerHeader">
<span>Activity Log</span>
<div style="display:flex; gap:6px; align-items:center;">
<label class="checkbox_label" style="font-size:0.85em;">
<input type="checkbox" id="charMemory_logDrawerVerbose" />
<span>Verbose</span>
</label>
<button id="charMemory_logDrawerClear" class="menu_button" style="font-size:0.8em; padding:2px 8px;">Clear</button>
<button id="charMemory_logDrawerSave" class="menu_button" style="font-size:0.8em; padding:2px 8px;">Save</button>
<button id="charMemory_logDrawerClose" class="charMemory_drawerCloseBtn">✕</button>
</div>
</div>
<div class="charMemory_drawerBody" id="charMemory_logDrawerBody">
<!-- Log entries rendered here -->
</div>
</div>
`);
```
### Step 2: Add CSS for Log Drawer
Reuse the Injection Viewer drawer pattern but from the right side:
```css
.charMemory_logDrawer {
position: fixed;
top: 0;
right: 0;
width: min(380px, 45vw);
height: 100%;
background: var(--SmartThemeBlurTintColor);
border-left: 1px solid var(--SmartThemeBorderColor);
transform: translateX(100%);
transition: transform 0.2s ease;
z-index: 1001;
display: flex;
flex-direction: column;
}
.charMemory_logDrawer.open {
transform: translateX(0);
}
```
Reuse existing `.charMemory_drawerHeader`, `.charMemory_drawerBody` patterns for consistent look.
### Step 3: Implement `toggleLogDrawer()`
Mirror `toggleInjectionDrawer()`:
- Toggle `.open` class
- Position below ST top bar
- Populate log entries from `logHistory` array
- Sync verbose toggle state
### Step 4: Implement `updateLogDrawer()`
Called from `logActivity()` — if the drawer is open, append the new log entry to the drawer body. Reuse the existing log entry rendering (`.charMemory_logEntry` with color classes).
### Step 5: Wire handlers
- Close button click → `toggleLogDrawer(false)`
- Clear button → clear log entries (same as existing `#charMemory_clearLog`)
- Save button → download log (same as existing `#charMemory_saveLog`)
- Verbose toggle → toggle verbose mode
- "View full log" link (will be added to sidebar in Task 5) → `toggleLogDrawer(true)`
### Step 6: Test manually
- Open log drawer → shows full activity log
- New extractions appear live in the drawer
- Verbose toggle works
- Clear and Save buttons work
- Drawer slides in/out smoothly
### Step 7: Commit
```bash
git add index.js style.css
git commit -m "feat: add Log Drawer (slide-out panel)
Full activity log in a slide-out drawer, same pattern as
Injection Viewer. Verbose toggle, clear, save buttons.
Will be triggered from sidebar dashboard."
```
---
## Task 4: Troubleshooter Modal
Build the Troubleshooter modal with automated health checks, Data Bank browser, diagnostic report export, and reset/clear actions.
**Files:**
- Modify: `index.js` (add `showTroubleshooter()`, Data Bank browser functions)
- Modify: `style.css` (add troubleshooter styles)
### Step 1: Implement `showTroubleshooter()`
Build modal HTML dynamically:
**Section 1: Health Checks**
- Run `computeHealthScore()` and display all checks with pass/fail icons
- Each failing check shows: explanation + [Fix] button where possible
- Reuse `renderHealthDiagnosticsCard()` output but with interactive fix buttons
**Section 2: Data Bank Browser**
- Call `getDataBankAttachmentsForSource()` to list character's files
- For each file show: name, size, whether it's a CharMemory file
- Action buttons: [View] [Export] [Delete] [Convert]
- [Import file] button at bottom
**Section 3: Diagnostic Report**
- "Copy diagnostic report" button
- Bundles: settings snapshot, last activity log entries, health check results, memory count, VS configuration, last injection data
- Copies to clipboard
**Section 4: Reset/Clear Actions**
- Reset Extraction State button (with confirmation)
- Clear All Memories button (with confirmation, danger styled)
Open via `callGenericPopup(html, POPUP_TYPE.TEXT, '', { wide: true, allowVerticalScrolling: true })`.
### Step 2: Implement Data Bank file actions
- **View**: Read file content via `getFileAttachment()`, show in a read-only popup. For CharMemory files, parse and highlight memory blocks.
- **Export**: Read file content, trigger browser download.
- **Delete**: Confirm dialog, then `deleteAttachment()`.
- **Convert**: Open the Convert tool with this file pre-selected.
- **Import**: File input → `uploadFileAttachment()`.
### Step 3: Implement diagnostic report builder
```javascript
function buildDiagnosticReport() {
// Gather: settings, health checks, last log entries, memory count,
// VS config, injection data, version info
// Format as text block
// Copy to clipboard via navigator.clipboard.writeText()
}
```
### Step 4: Add CSS for troubleshooter
```css
.charMemory_troubleshooter { /* scrollable modal content */ }
.charMemory_troubleshooterSection { /* bordered section with header */ }
.charMemory_dataBankItem { /* file row with actions */ }
.charMemory_dataBankActions { /* button row per file */ }
```
### Step 5: Test manually
- Open troubleshooter → health checks run and display
- Data Bank browser lists character's files
- View shows file content
- Export downloads the file
- Delete removes with confirmation
- Diagnostic report copies to clipboard
- Reset/Clear work with confirmation
### Step 6: Commit
```bash
git add index.js style.css
git commit -m "feat: add Troubleshooter modal
Health checks with fix buttons, Data Bank file browser
(view/export/delete/convert/import), diagnostic report
export, and reset/clear actions."
```
---
## Task 5: Sidebar Dashboard
The big migration — replace the 4-tab sidebar with a single-view dashboard. This task removes the tabs and restructures `settings.html` completely.
**Files:**
- Rewrite: `settings.html` (replace tabs with dashboard layout)
- Modify: `style.css` (add dashboard styles, can remove old tab styles)
- Modify: `index.js` (update `loadSettings()`, remove old tab switching, add dashboard controls)
### Step 1: Rewrite settings.html
Replace the entire tab structure with:
```html
<div class="charMemory_settings">
<div class="inline-drawer">
<div class="inline-drawer-toggle inline-drawer-header">
<b>CharMemory</b>
<button id="charMemory_settingsGear" class="charMemory_gearBtn" title="Settings">⚙</button>
<div class="inline-drawer-icon fa-solid fa-circle-chevron-down down"></div>
</div>
<div class="inline-drawer-content">
<!-- Stats Bar (same as current, kept) -->
<div class="charMemory_statsBar">...</div>
<!-- File Section -->
<div class="charMemory_dashSection">
<div class="charMemory_fileInfo">
<span id="charMemory_dashFileName">Flux_the_Cat-memories.md</span>
<span id="charMemory_dashFileMeta" class="charMemory_dimText">42 KB • 11 chunks</span>
</div>
<div class="charMemory_buttonRow">
<button id="charMemory_manageMemories" class="menu_button">View / Edit</button>
<button id="charMemory_filesPopover" class="menu_button">Files ▾</button>
</div>
</div>
<!-- Extraction Controls -->
<div class="charMemory_dashSection">
<label class="checkbox_label">
<input type="checkbox" id="charMemory_enabled" />
<span>Automatic extraction</span>
</label>
<button id="charMemory_extractNow" class="menu_button">Extract Now</button>
</div>
<!-- Tool Launchers -->
<div class="charMemory_dashSection">
<small class="charMemory_sectionLabel">Tools</small>
<div class="charMemory_buttonRow">
<button id="charMemory_consolidate" class="menu_button">Consolidate</button>
<button id="charMemory_batchBtn" class="menu_button">Batch</button>
<button id="charMemory_convertBtn" class="menu_button">Convert</button>
</div>
</div>
<!-- Activity (mini log) -->
<div class="charMemory_dashSection">
<small class="charMemory_sectionLabel">Activity</small>
<div id="charMemory_dashActivity" class="charMemory_dashActivity">
<!-- Last 2-3 log entries -->
</div>
<a id="charMemory_viewFullLog" class="charMemory_link">View full log →</a>
</div>
<!-- Diagnostics summary -->
<div class="charMemory_dashSection">
<div class="charMemory_dashDiagHeader">
<small class="charMemory_sectionLabel">Diagnostics</small>
<button id="charMemory_refreshDiag" class="charMemory_miniBtn">Refresh</button>
</div>
<div id="charMemory_dashDiagSummary">✅ Healthy — 7/7 checks pass</div>
<a id="charMemory_viewDiagDetails" class="charMemory_link">View details →</a>
</div>
<!-- Troubleshooter -->
<div class="charMemory_dashSection">
<button id="charMemory_troubleshooterBtn" class="menu_button charMemory_fullWidth">🔧 Help, it's not working</button>
</div>
</div>
</div>
</div>
```
### Step 2: Update CSS
- Remove old tab styles (`.charMemory_tabs`, `.charMemory_tab`, `.charMemory_tabContent`, `.charMemory_toolPill`, `.charMemory_toolContent`)
- Add dashboard section styles:
```css
.charMemory_dashSection {
padding: 6px 0;
border-bottom: 1px solid var(--SmartThemeBorderColor);
}
.charMemory_dashSection:last-child {
border-bottom: none;
}
.charMemory_sectionLabel {
opacity: 0.6;
font-size: 0.85em;
display: block;
margin-bottom: 4px;
}
.charMemory_dashActivity {
max-height: 80px;
overflow-y: auto;
font-size: 0.85em;
}
.charMemory_link {
font-size: 0.85em;
cursor: pointer;
color: var(--SmartThemeQuoteColor);
}
.charMemory_gearBtn {
background: none;
border: none;
cursor: pointer;
font-size: 1.1em;
opacity: 0.7;
padding: 0 4px;
}
.charMemory_gearBtn:hover {
opacity: 1;
}
.charMemory_fullWidth {
width: 100%;
}
```
### Step 3: Update index.js
- **Remove** old tab switching logic from `setupLogControls()` (the `.charMemory_tab` click handler)
- **Remove** tool pill switching from `setupToolControls()`
- **Update** `loadSettings()` to populate dashboard elements instead of tabs
- **Add** dashboard-specific handlers:
- Gear icon → `showSettingsModal()`
- View full log → `toggleLogDrawer(true)`
- View details → `showTroubleshooter()`
- Troubleshooter button → `showTroubleshooter()`
- Files button → show Data Bank popover (or open Troubleshooter to Data Bank section)
- Batch button → show batch extract dialog
- Convert button → show convert tool dialog
- **Update** `logActivity()` to also update `#charMemory_dashActivity` with latest 3 entries
- **Update** `updateHealthIndicator()` to also update `#charMemory_dashDiagSummary`
- **Update** `updateDashboardFileInfo()` — new function to show active memory file name, size, chunk count
### Step 4: Update setupListeners() sub-functions
The 5 setup functions need adjustment:
- `setupConnectionControls()` — handlers move to Settings modal (remove from here, wire in `showSettingsModal()`)
- `setupExtractionControls()` — most handlers move to Settings modal. Keep the `#charMemory_enabled` toggle and `#charMemory_extractNow` for the dashboard.
- `setupToolControls()` — remove pill switching, keep tool button clicks
- `setupStorageControls()` — handlers move to Settings modal
- `setupLogControls()` — remove tab switching, keep log drawer triggers
### Step 5: Test manually (comprehensive)
This is the biggest visual change. Test everything:
- Dashboard loads with stats, file info, extraction controls, tool launchers, activity, diagnostics
- Gear icon opens Settings modal with all settings working
- Extract Now triggers extraction, activity updates live
- Tool buttons open their respective dialogs
- View full log opens Log Drawer
- View details opens Troubleshooter
- Troubleshooter button opens Troubleshooter
- Files button shows Data Bank files
- Health dot is clickable (opens Troubleshooter)
- No console errors
### Step 6: Commit
```bash
git add settings.html style.css index.js
git commit -m "feat: replace 4-tab sidebar with single-view dashboard
Remove tabs. Dashboard shows stats, file info, extraction toggle,
tool launchers, activity summary, diagnostics, and troubleshooter.
All settings moved to Settings modal (gear icon)."
```
---
## Task 6: Setup Wizard
Build the 3-step setup wizard modal with smart triggering.
**Files:**
- Modify: `index.js` (add `showSetupWizard()`, trigger logic, verification step)
- Modify: `style.css` (add wizard styles)
### Step 1: Implement `showSetupWizard(startStep)`
Build modal HTML dynamically with 3 steps:
**Step 1: LLM Connection**
- Explanation text
- Provider dropdown (Pollinations highlighted)
- API key field (hidden when not needed)
- Connect & Test button with inline status
- Model auto-selected, option to change
- [Next →]
**Step 2: Vector Storage**
- Explanation text
- Auto-detect VS configuration using `computeHealthScore()`
- Show passing checks (green) and issues (amber)
- [Fix] buttons where possible
- [← Back] [Next →]
**Step 3: Ready**
- Summary of configuration
- Explain what happens next
- [Get Started] closes wizard, stores `wizardCompleted: true`
Open via `callGenericPopup(html, POPUP_TYPE.TEXT, '', { wide: true, allowVerticalScrolling: true })`.
### Step 2: Add smart trigger logic
In the jQuery ready handler (after `loadSettings()`):
```javascript
// Full wizard: no provider configured
if (!extension_settings[MODULE_NAME].selectedProvider && !extension_settings[MODULE_NAME].wizardCompleted) {
showSetupWizard(1);
}
```
For light nudge (health issues detected), add a banner div to the dashboard HTML:
```html
<div id="charMemory_nudgeBanner" class="charMemory_nudgeBanner" style="display:none;">
<span>Something needs attention</span>
<button id="charMemory_nudgeFix" class="menu_button">Fix now</button>
</div>
```
The banner is shown when `computeHealthScore()` returns non-green and `wizardCompleted` is true. Clicking [Fix now] opens the wizard to the relevant step.
### Step 3: Implement Step 4: Verification (post-first-extraction)
After the first successful extraction (in `extractMemories()` success path), check:
```javascript
if (!extension_settings[MODULE_NAME].verificationSeen) {
showVerificationStep();
}
```
Show a modal explaining how to check retrieval quality, with links to the syringe icon and health check.
### Step 4: Add wizard styles
```css
.charMemory_wizard { /* wizard container */ }
.charMemory_wizardStep { /* step container, shown one at a time */ }
.charMemory_wizardNav { /* Back/Next buttons */ }
.charMemory_wizardExplanation { /* intro text styling */ }
.charMemory_wizardCheck { /* health check row */ }
.charMemory_nudgeBanner { /* attention banner in dashboard */ }
```
### Step 5: Add "Run Setup Wizard" link to Settings modal
In the Connection section of the Settings modal, add a link at the bottom:
```html
<a id="charMemory_runWizard" class="charMemory_link">Run Setup Wizard</a>
```
### Step 6: Test manually
- Fresh install (clear settings) → wizard auto-triggers
- Step 1: select provider → connect → success
- Step 2: VS checks run, issues flagged
- Step 3: summary → Get Started → wizard closes
- First extraction → verification step appears
- Light nudge banner appears when health issues exist
### Step 7: Commit
```bash
git add index.js style.css
git commit -m "feat: add Setup Wizard with smart triggering
3-step wizard (LLM Connection, Vector Storage, Ready).
Auto-triggers on first launch. Light nudge banner for
health issues. Post-extraction verification step."
```
---
## Task 7: Prompt Version Tracking
Add version tracking to prompts — notify users when defaults update, offer compare & edit.
**Files:**
- Modify: `index.js` (add `PROMPT_VERSIONS`, comparison logic, compare & edit UI)
- Modify: `style.css` (add update notification and compare styles)
### Step 1: Add PROMPT_VERSIONS constant
```javascript
const PROMPT_VERSIONS = {
extraction: '2.0.0',
groupExtraction: '2.0.0',
consolidation: '2.0.0',
conversion: '2.0.0',
};
```
### Step 2: Add version check on load
In `loadSettings()` or a new `checkPromptVersions()`:
```javascript
function checkPromptVersions() {
const stored = extension_settings[MODULE_NAME].promptVersions || {};
for (const [key, currentVersion] of Object.entries(PROMPT_VERSIONS)) {
if (stored[key] && stored[key] !== currentVersion) {
// User has a custom prompt based on an older version
// Flag for notification in Prompts modal
}
}
}
```
### Step 3: Show update notification in Prompts modal
When a prompt has an update available, show a banner above the textarea:
```html
<div class="charMemory_promptUpdateBanner">
The default prompt was updated in v2.0. Your custom version is unchanged.
<button id="charMemory_promptKeepMine">Keep mine</button>
<button id="charMemory_promptUseNew">Use new default</button>
<button id="charMemory_promptCompare">Compare & Edit →</button>
</div>
```
### Step 4: Implement Compare & Edit view
Replace the single textarea with two panes:
- Left: user's current prompt (editable)
- Right: new default prompt (read-only reference)
Reuse the side-by-side pattern from the conversion dialog (`.charMemory_consolidationPanes`).
### Step 5: Test manually
- Modify a prompt to make it "custom"
- Change the PROMPT_VERSIONS to simulate an update
- Open Prompts modal → update notification appears
- Keep mine → dismisses notification, stores current version
- Use new default → replaces with default, stores current version
- Compare & Edit → shows side-by-side view
### Step 6: Commit
```bash
git add index.js style.css
git commit -m "feat: add prompt version tracking with compare & edit
Track prompt versions, notify when defaults update.
Keep mine / Use new default / Compare & Edit options."
```
---
## Task 8: Cleanup & Polish
Remove old code, update documentation, final verification.
**Files:**
- Modify: `settings.html` (remove any remaining old tab HTML if not already)
- Modify: `style.css` (remove old tab/pill styles)
- Modify: `index.js` (remove old tab/pill handler code)
- Modify: `CLAUDE.md` (update architecture docs)
- Modify: `CHANGELOG.md` (add v2.0 changelog)
- Modify: `manifest.json` (bump version to 2.0.0)
### Step 1: Remove dead CSS
Search for and remove styles that are no longer referenced:
- `.charMemory_tabs`, `.charMemory_tab`
- `.charMemory_tabContent`
- `.charMemory_toolPill`, `.charMemory_toolPills`, `.charMemory_toolContent`
- `.charMemory_miniLog` (replaced by dashboard activity)
- `.charMemory_bottomDiagnostics` (replaced by dashboard diagnostics + troubleshooter)
### Step 2: Remove dead JS
- Old tab switching handler in `setupLogControls()`
- Old pill switching handler in `setupToolControls()`
- Old mini-log visibility logic
- Any `loadSettings()` code that sets old tab/pill state
### Step 3: Update CLAUDE.md
Update the File Structure, Key Architecture, and Common Tasks sections to reflect:
- New file: `editor.js`
- Settings moved to modal
- Prompts moved to modal
- Log moved to drawer
- Sidebar is now a dashboard
- Setup wizard for first-run
### Step 4: Update CHANGELOG.md
Add v2.0.0 section covering all UX changes.
### Step 5: Bump version
```json
"version": "2.0.0"
```
### Step 6: Full regression test
Test every feature end-to-end:
1. Fresh install → wizard triggers
2. Provider setup → connect → model select
3. Extraction → memories created
4. Dashboard shows stats, activity, diagnostics
5. Settings modal → all sections work
6. Prompts modal → edit and save all 4 prompts
7. Log drawer → shows activity, verbose, clear, save
8. Troubleshooter → health checks, data bank browser, diagnostic report
9. Consolidation → preview dialog → save
10. Conversion → preview dialog → save
11. Batch extraction → select chats → run
12. Injection Viewer → syringe icon → drawer shows injected data
13. Memory Manager → view/edit/delete memories
### Step 7: Commit
```bash
git add -A
git commit -m "chore: v2.0.0 cleanup — remove old tab UI, update docs
Remove dead CSS/JS from old tab-based layout.
Update CLAUDE.md and CHANGELOG.md for v2.0."
```
---
## Summary
| Task | What | Risk | Depends on |
|------|------|------|------------|
| 1 | Settings Modal | Medium | — |
| 2 | Prompts Modal | Low | Task 1 |
| 3 | Log Drawer | Low | — |
| 4 | Troubleshooter | Medium | — |
| 5 | Sidebar Dashboard | **High** | Tasks 1-4 |
| 6 | Setup Wizard | Medium | Task 5 |
| 7 | Prompt Version Tracking | Low | Task 2 |
| 8 | Cleanup & Polish | Low | Tasks 1-7 |
**After all tasks:**
- Sidebar is a single-view dashboard (no tabs)
- All configuration in center-screen Settings modal with left-nav
- All prompts in dedicated Prompts modal with full-width editor
- Activity log in slide-out Log Drawer
- First-run Setup Wizard with smart triggering
- Troubleshooter with Data Bank browser and diagnostic report
- Prompt version tracking with compare & edit
- Zero features removed — everything reorganized for discoverability

119
docs/plans/2026-03-01-wizard-v2-design.md Normal file
View file

@ -0,0 +1,119 @@
# Setup Wizard v2 — Design
## Problem
The current 3-step wizard has several UX issues identified during testing:
1. **OK vs Next confusion** — `POPUP_TYPE.TEXT` renders an OK button in the popup chrome that competes with the wizard's own Next button. Users don't know which to press.
2. **OK closes wizard with no re-entry** — clicking OK exits the wizard. The only way back is the nudge banner, which only appears under specific conditions. Users get stuck.
3. **LLM screen readability** — model search input has transparency issues from `.text_pole` styling. Model list requires scrolling in a dropdown. NanoGPT subscription/open-source models aren't distinguished.
4. **Vector Storage screen confusion** — health checks show contradictory state (file found then not found). Screen is passive — shows problems but gives no actionable controls.
5. **"Ready" screen is unhelpful** — says "all set" even with VS issues. Doesn't orient the user on what to actually do next. No mention of injection sidebar, tools, or existing memory handling.
6. **Auto-trigger bug** — wizard condition checks `!selectedProvider` but `defaultSettings` pre-sets it to `'openrouter'`, so the wizard never triggers on fresh install.
7. **Destructive action scoping** — tools like "Clear All Memories" don't make it clear they affect the current character only (and in non-perChat mode, all that character's chats).
## Design
### Structural Changes
**Popup type:** Change from `POPUP_TYPE.TEXT` (has OK button) to `POPUP_TYPE.DISPLAY` (no chrome buttons). Wizard navigation is entirely through its own Back/Next/Get Started buttons.
**Close behavior:** Add an X button in the wizard header. On click, prompt: "You can reopen this from the dashboard any time." Then close.
**Re-entry — three paths:**
1. **Dashboard gear icon** — always visible in the sidebar panel header. Opens wizard to step 1.
2. **Nudge banner** — existing behavior kept. Appears when health checks detect issues after wizard is completed.
3. **Troubleshooter** — "Re-run Setup Wizard" link in the reset/tools section.
**Auto-trigger fix:** Change condition from `!selectedProvider && !wizardCompleted` to just `!wizardCompleted`. The `wizardCompleted` flag is the canonical "user has been through setup" signal.
### Step 1: Connect
Purpose: Set up the LLM that will read chats and create memory summaries.
**Layout (top to bottom):**
1. **Welcome blurb** — "CharMemory extracts structured memories from your chats so characters can recall past events. It needs an LLM to read messages and create summaries."
2. **Provider dropdown** — same as current. Pollinations highlighted as free/no-key option.
3. **API Key row** — shows/hides based on provider. Same as current.
4. **Connect & Test button** — on success, smoothly reveals the model section below.
5. **Model picker (redesigned):**
- Solid-background search field replacing the transparent `.text_pole` input
- Model list in a scrollable container below the search (always visible after connect, not a dropdown requiring click)
- Max height ~200px with scroll
- **NanoGPT-specific:** Inline badges `[sub]` `[open]` `[rp]` `[reason]` next to model names, plus filter checkboxes above the list (Subscription, Open Source, Roleplay, Reasoning)
- Non-NanoGPT providers: searchable list only, no badges/filters
6. **Next button** — enabled after successful connection test.
### Step 2: Configure
Purpose: Set extraction behavior and verify retrieval pipeline.
**Layout (top to bottom):**
1. **Memory Storage section:**
- Info text: "Each character gets their own memory file in their Data Bank (e.g., `Flux_the_Cat-memories.md`). You can change storage options in Settings later."
- No per-chat toggle — just inform about the default. Keep it simple.
2. **Extraction Interval:**
- "How often should CharMemory extract? Every ___ messages."
- Number input, default 20.
- Helper text: "Lower = more frequent, more API calls. Higher = less frequent, bigger batches. 20 is a good starting point."
3. **Retrieval (Vector Storage) section:**
- Brief explanation: "Vector Storage finds the right memories at the right time and injects them into the prompt when your character speaks. Without it, memories are stored but never used."
- **Three-tier detection:**
- **VS not enabled:** Red indicator. "Vector Storage is not enabled. CharMemory will store memories but your character won't recall them. Enable it in Extensions → Vector Storage when you're ready."
- **VS enabled but settings may need tuning:** Yellow indicator. "Vector Storage is active, but its chunk settings may not be optimized for CharMemory's memory block format. CharMemory works best with chunk size 800–1000 chars, overlap 10–25%, and retrieve chunks 2–3. You can adjust these in Extensions → Vector Storage."
- **VS fully configured:** Green checks.
- Detection logic: flag chunk size <500 or >1500, overlap = 0, retrieve chunks >5 or 0.
4. **Navigation:**
- Next button always enabled (VS issues are advisory, not blocking)
- If VS issues present, yellow note above Next: "You can continue without fixing these — memories will be stored but not retrieved until Vector Storage is configured."
### Step 3: Review & Go
Purpose: Confirm setup, orient the user, handle existing memories.
**Layout (top to bottom):**
1. **Summary card (read-only):**
- Provider: [name]
- Model: [name]
- Connection: ✔ Connected / ⚠ Not tested
- Extraction: Every [N] messages
- Vector Storage: ✔ Ready / ⚠ Not configured / ⚠ Needs tuning
- If something looks wrong, Back button takes them to fix it.
2. **Injection sidebar callout:**
- "Open the **Injection Sidebar** from the dashboard to see what memories are being used in your character's prompt in real time."
3. **Existing memory conversion (conditional):**
- Only shows if the character already has a Data Bank memory file.
- "We found existing memories for [CharName]. The **Convert** tool can reformat them for better retrieval."
- Two buttons: "Convert Now" / "Skip — I'll do this later"
4. **Scoping note (footer):**
- "Tools like Clear Memories and Reset Extraction State only affect the current character."
5. **Get Started button** — sets `wizardCompleted = true`, closes wizard, returns to dashboard.
## Scoping Language
Throughout the wizard and in destructive-action confirmations, always specify scope:
- "this character" / "for [CharName]" — not "all memories" or "everything"
- Clear All Memories confirmation should say: "[CharName]'s memory file will be deleted. In default mode, this includes memories from all of [CharName]'s chats."
- Reset Extraction State should say: "Extraction tracking for [CharName] will be reset. The extension will re-process messages from the beginning."
## Technical Notes
- Health checks run once when step 2 loads, result cached in `wizHealthResult`. No re-running on step navigation.
- `POPUP_TYPE.DISPLAY` removes OK/Cancel chrome. Wizard manages its own lifecycle.
- NanoGPT model metadata (subscription, open-source, roleplay, reasoning flags) is already available from the custom models endpoint — just needs to be surfaced in the wizard's model list renderer.
- The "Convert Now" button on step 3 can reuse the existing `showReformatPreview()` flow.

787
docs/plans/2026-03-01-wizard-v2-plan.md Normal file
View file

@ -0,0 +1,787 @@
# Setup Wizard v2 Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Redesign the setup wizard to fix critical UX bugs (competing OK button, no re-entry, state confusion) and expand it with extraction interval config, three-tier VS guidance, and a useful "Review & Go" step.
**Architecture:** All changes are in `index.js` (wizard function + troubleshooter + destructive confirmations), `settings.html` (dashboard gear icon), and `style.css` (new wizard component classes). No new files needed. The wizard uses `POPUP_TYPE.DISPLAY` which hides OK/Cancel and shows a native X button, so wizard lifecycle is entirely self-managed.
**Tech Stack:** jQuery DOM manipulation, SillyTavern `callGenericPopup`/`POPUP_TYPE`, existing `computeHealthScore()` / `extension_settings.vectors` for VS detection.
**Design doc:** `docs/plans/2026-03-01-wizard-v2-design.md`
---
## Task 1: Structural Fixes
**Files:**
- Modify: `index.js` — `showSetupWizard()` and auto-trigger condition
### Step 1: Fix auto-trigger condition
Find (line ~7654):
```js
if (!extension_settings[MODULE_NAME].selectedProvider && !extension_settings[MODULE_NAME].wizardCompleted) {
showSetupWizard(1);
}
```
Replace with:
```js
if (!extension_settings[MODULE_NAME].wizardCompleted) {
showSetupWizard(1);
}
```
**Why:** `defaultSettings.selectedProvider` is `'openrouter'`, so `!selectedProvider` is always false on fresh install. `wizardCompleted` is the actual "user has been through setup" flag.
### Step 2: Change popup type to DISPLAY
Find inside `showSetupWizard()`:
```js
const popup = callGenericPopup(html, POPUP_TYPE.TEXT, '', { wide: true, allowVerticalScrolling: true });
```
Replace with:
```js
const popup = callGenericPopup(html, POPUP_TYPE.DISPLAY, '', { wide: true, allowVerticalScrolling: true });
```
**Why:** `POPUP_TYPE.DISPLAY` hides OK/Cancel buttons (removing the competing "OK" that closed the wizard), while keeping the native X close button in the corner. Wizard navigation is now entirely through its own Back/Next/Get Started buttons.
### Step 3: Commit
```bash
git add index.js
git commit -m "fix: wizard uses POPUP_TYPE.DISPLAY, auto-trigger checks wizardCompleted only"
```
**Verify manually:** Start SillyTavern with `wizardCompleted: false` in settings. Wizard should appear. No OK button visible. Native X in corner closes it.
---
## Task 2: Re-entry Mechanisms
**Files:**
- Modify: `settings.html` — add wizard button to header
- Modify: `index.js` — wire up dashboard button, add wizard link to troubleshooter
### Step 1: Add wizard icon to dashboard header
In `settings.html`, find the header icons (the two `charMemory_headerGear` spans):
```html
<span id="charMemory_openTroubleshooter" class="charMemory_headerGear" title="Troubleshooter">
<i class="fa-solid fa-screwdriver-wrench fa-sm"></i>
</span>
<span id="charMemory_openSettingsModal" class="charMemory_headerGear" title="Open Settings">
<i class="fa-solid fa-gear fa-sm"></i>
</span>
```
Add a third icon before the troubleshooter span:
```html
<span id="charMemory_openWizard" class="charMemory_headerGear" title="Setup Wizard">
<i class="fa-solid fa-wand-magic-sparkles fa-sm"></i>
</span>
```
### Step 2: Wire up dashboard wizard button
In `index.js`, find the nudge banner click handler (line ~7659):
```js
$('#charMemory_nudgeFix').on('click', function () {
```
Add before it:
```js
$('#charMemory_openWizard').on('click', function () {
showSetupWizard(1);
});
```
### Step 3: Add "Re-run Setup Wizard" to Troubleshooter reset section
In `showTroubleshooter()`, find the reset section HTML where the two reset buttons are built. It looks like:
```js
<div class="charMemory_tsResetSection">
<button class="menu_button" id="cm_ts_resetTracking">Reset Extraction State</button>
...
```
Add a new section at the top of the reset section (before the destructive buttons):
```js
<div class="charMemory_tsResetSection">
<button class="menu_button" id="cm_ts_openWizard">Re-run Setup Wizard</button>
<small class="charMemory_helperText">Walk through the setup steps again to reconfigure your LLM connection, storage, or retrieval settings.</small>
</div>
```
### Step 4: Wire up troubleshooter wizard button
In the troubleshooter event wiring section (where `cm_ts_resetTracking` and `cm_ts_clearMemories` are handled), add:
```js
$modal.find('#cm_ts_openWizard').on('click', async function () {
// Close the troubleshooter popup first
$modal.closest('.popup').find('.popup-button-ok, .popup-button-close').first().trigger('click');
setTimeout(() => showSetupWizard(1), 200);
});
```
### Step 5: Commit
```bash
git add index.js settings.html
git commit -m "feat: wizard re-entry via dashboard icon and troubleshooter button"
```
---
## Task 3: Step 1 — Model Picker Redesign + NanoGPT Badges
**Files:**
- Modify: `index.js` — step1Html, renderWizModelDropdown → renderWizModelList, updateWizProviderUI, NanoGPT filter wiring
- Modify: `style.css` — new model list and badge classes
### Step 1: Replace dropdown-based model picker in step1Html
Find the `step1Html` constant in `showSetupWizard()`. Replace the `cm_wiz_modelRow` field group:
**Old:**
```js
<div class="charMemory_modalFieldGroup" id="cm_wiz_modelRow" style="display:none;">
<label><small>Model</small></label>
<div class="charMemory_modelPicker" style="position:relative;">
<input type="text" id="cm_wiz_modelSearch" class="text_pole" placeholder="Search models..." autocomplete="off" value="" />
<input type="hidden" id="cm_wiz_modelValue" value="" />
<div id="cm_wiz_modelDropdown" class="charMemory_modelDropdown"></div>
</div>
<small id="cm_wiz_modelStatus" class="charMemory_helperText" style="display:none;"></small>
</div>
```
**New:**
```js
<div class="charMemory_modalFieldGroup" id="cm_wiz_modelRow" style="display:none;">
<label><small>Model</small></label>
<div id="cm_wiz_nanogptFilters" style="display:none; margin-bottom:6px;">
<label class="checkbox_label"><input type="checkbox" id="cm_wiz_nanogptFilterSub" /> <small>Subscription</small></label>
<label class="checkbox_label"><input type="checkbox" id="cm_wiz_nanogptFilterOS" /> <small>Open Source</small></label>
<label class="checkbox_label"><input type="checkbox" id="cm_wiz_nanogptFilterRP" /> <small>Roleplay</small></label>
<label class="checkbox_label"><input type="checkbox" id="cm_wiz_nanogptFilterReasoning" /> <small>Reasoning</small></label>
</div>
<div class="charMemory_wizModelPicker">
<input type="text" id="cm_wiz_modelSearch" class="charMemory_wizModelSearch" placeholder="Search models..." autocomplete="off" value="" />
<input type="hidden" id="cm_wiz_modelValue" value="" />
<div id="cm_wiz_modelList" class="charMemory_wizModelList"></div>
</div>
<small id="cm_wiz_modelStatus" class="charMemory_helperText" style="display:none;"></small>
</div>
```
### Step 2: Replace renderWizModelDropdown with renderWizModelList
Remove the entire `renderWizModelDropdown` function and replace with:
```js
function renderWizModelList(filter) {
const $list = $wizard.find('#cm_wiz_modelList');
$list.empty();
const pk = extension_settings[MODULE_NAME].selectedProvider;
const isNanogpt = pk === 'nanogpt';
const lowerFilter = (filter || '').toLowerCase();
const selectedId = $wizard.find('#cm_wiz_modelValue').val();
let models = currentModelList;
if (isNanogpt) {
const ps = getProviderSettings(pk);
models = getFilteredNanoGptModels(models, ps);
}
if (models.length === 0) {
$list.append('<div class="charMemory_modelEmpty">No models loaded</div>');
return;
}
let hasResults = false;
let lastGroup = null;
for (const model of models) {
if (lowerFilter && !model.id.toLowerCase().includes(lowerFilter) && !model.name.toLowerCase().includes(lowerFilter)) continue;
if (model.group && model.group !== lastGroup) {
$list.append(`<div class="charMemory_modelGroup">${escapeHtml(model.group)}</div>`);
lastGroup = model.group;
}
let badgesHtml = '';
if (isNanogpt) {
if (model.subscription) badgesHtml += '<span class="charMemory_modelBadge charMemory_modelBadge--sub">sub</span>';
if (model.isOpenSource) badgesHtml += '<span class="charMemory_modelBadge charMemory_modelBadge--open">open</span>';
if (model.category === 'Roleplay/storytelling models') badgesHtml += '<span class="charMemory_modelBadge charMemory_modelBadge--rp">rp</span>';
if (model.capabilities && model.capabilities.includes('reasoning')) badgesHtml += '<span class="charMemory_modelBadge charMemory_modelBadge--reason">reason</span>';
}
const selectedClass = model.id === selectedId ? ' selected' : '';
$list.append(`<div class="charMemory_modelOption${selectedClass}" data-model-id="${escapeAttr(model.id)}">
<span class="charMemory_modelOptionName">${escapeHtml(model.name)}</span>
${badgesHtml}
</div>`);
hasResults = true;
}
if (!hasResults) {
$list.append('<div class="charMemory_modelEmpty">No matching models</div>');
}
}
```
### Step 3: Update event handlers to use the new list
Remove the old dropdown focus/input handlers and replace with:
```js
$wizard.on('input', '#cm_wiz_modelSearch', function () {
renderWizModelList($(this).val());
});
```
Remove the `$(document).off('click.cmWizModelPicker')` handler entirely — the always-visible list doesn't need click-outside-to-close behavior.
Update the click handler to use the new IDs:
```js
$wizard.on('click', '#cm_wiz_modelList .charMemory_modelOption', function () {
const modelId = String($(this).data('model-id'));
const model = currentModelList.find(m => m.id === modelId);
if (!model) return;
$wizard.find('#cm_wiz_modelValue').val(modelId);
$wizard.find('#cm_wiz_modelSearch').val(model.name);
$wizard.find('#cm_wiz_modelList .charMemory_modelOption').removeClass('selected');
$(this).addClass('selected');
const pk = extension_settings[MODULE_NAME].selectedProvider;
const ps = getProviderSettings(pk);
ps.model = modelId;
saveSettingsDebounced();
});
```
### Step 4: Update updateWizProviderUI to show/hide NanoGPT filters and sync checkboxes
Inside `updateWizProviderUI()`, add after the existing logic:
```js
const isNanogpt = pk === 'nanogpt';
$wizard.find('#cm_wiz_nanogptFilters').toggle(isNanogpt);
if (isNanogpt) {
const ps = getProviderSettings(pk);
$wizard.find('#cm_wiz_nanogptFilterSub').prop('checked', ps.nanogptFilterSubscription || false);
$wizard.find('#cm_wiz_nanogptFilterOS').prop('checked', ps.nanogptFilterOpenSource || false);
$wizard.find('#cm_wiz_nanogptFilterRP').prop('checked', ps.nanogptFilterRoleplay || false);
$wizard.find('#cm_wiz_nanogptFilterReasoning').prop('checked', ps.nanogptFilterReasoning || false);
}
```
Also update the connect handler: when models load and model list is shown, call `renderWizModelList('')` instead of the old dropdown renderer.
### Step 5: Wire up NanoGPT filter checkboxes
```js
$wizard.on('change', '#cm_wiz_nanogptFilterSub, #cm_wiz_nanogptFilterOS, #cm_wiz_nanogptFilterRP, #cm_wiz_nanogptFilterReasoning', function () {
const pk = extension_settings[MODULE_NAME].selectedProvider;
const ps = getProviderSettings(pk);
ps.nanogptFilterSubscription = $wizard.find('#cm_wiz_nanogptFilterSub').prop('checked');
ps.nanogptFilterOpenSource = $wizard.find('#cm_wiz_nanogptFilterOS').prop('checked');
ps.nanogptFilterRoleplay = $wizard.find('#cm_wiz_nanogptFilterRP').prop('checked');
ps.nanogptFilterReasoning = $wizard.find('#cm_wiz_nanogptFilterReasoning').prop('checked');
saveSettingsDebounced();
renderWizModelList($wizard.find('#cm_wiz_modelSearch').val());
});
```
### Step 6: Add CSS for model list and badges
In `style.css`, in the wizard section (after `.charMemory_wizardSummary`), add:
```css
.charMemory_wizModelPicker {
position: relative;
}
.charMemory_wizModelSearch {
background: var(--SmartThemeBodyColor, #1a1a1a);
color: var(--SmartThemeBodyTextColor, #ccc);
border: 1px solid var(--SmartThemeBorderColor, #555);
border-radius: 4px 4px 0 0;
width: 100%;
box-sizing: border-box;
padding: 6px 8px;
font-size: 0.9em;
}
.charMemory_wizModelList {
max-height: 200px;
overflow-y: auto;
border: 1px solid var(--SmartThemeBorderColor, #555);
border-top: none;
border-radius: 0 0 4px 4px;
background: var(--SmartThemeBodyColor, #1a1a1a);
}
.charMemory_wizModelList .charMemory_modelOption {
display: flex;
align-items: center;
gap: 6px;
flex-wrap: wrap;
}
.charMemory_modelOptionName {
flex: 1;
min-width: 0;
}
.charMemory_modelBadge {
font-size: 0.7em;
padding: 1px 5px;
border-radius: 3px;
font-weight: 600;
letter-spacing: 0.02em;
white-space: nowrap;
opacity: 0.85;
}
.charMemory_modelBadge--sub { background: #1a4a2a; color: #6ecf8f; }
.charMemory_modelBadge--open { background: #1a2a4a; color: #6ea8cf; }
.charMemory_modelBadge--rp { background: #3a1a4a; color: #b06ecf; }
.charMemory_modelBadge--reason { background: #4a3a1a; color: #cfb06e; }
```
### Step 7: Run tests and commit
```bash
npm test
git add index.js style.css
git commit -m "feat: wizard step 1 — always-visible model list, NanoGPT subscription badges and filters"
```
**Verify manually:** Connect to NanoGPT in wizard. Model list should be visible and scrollable immediately after connect. Badges appear next to NanoGPT models. Filter checkboxes narrow the list. Non-NanoGPT providers show plain list, no filters/badges.
---
## Task 4: Step 2 — Configure
**Files:**
- Modify: `index.js` — step2Html, initStep2()
### Step 1: Replace step2Html
Replace the entire `step2Html` constant:
```js
const step2Html = `
<div class="charMemory_wizardStep" data-step="2">
<div class="charMemory_wizardExplanation">
Configure how CharMemory stores and retrieves memories.
</div>
<div class="charMemory_wizardSection">
<div class="charMemory_wizardSectionTitle">Memory Storage</div>
<p class="charMemory_helperText" style="margin:4px 0 0;">Each character gets their own memory file in their Data Bank. You can change storage options (including per-chat separation) in Settings later.</p>
</div>
<div class="charMemory_wizardSection">
<div class="charMemory_wizardSectionTitle">Extraction Frequency</div>
<div class="charMemory_modalFieldGroup" style="margin:4px 0 0;">
<label><small>Extract every
<input type="number" id="cm_wiz_interval" class="charMemory_wizSmallInput" min="5" max="500" value="${s.interval || 20}" />
messages</small></label>
<small class="charMemory_helperText">Lower = more frequent, more API calls. Higher = less frequent, bigger batches. 20 is a good starting point.</small>
</div>
</div>
<div class="charMemory_wizardSection">
<div class="charMemory_wizardSectionTitle">Retrieval (Vector Storage)</div>
<p class="charMemory_helperText" style="margin:4px 0 6px;">Vector Storage finds the right memories at the right time and injects them into the prompt. Without it, memories are stored but never used.</p>
<div id="cm_wiz_healthChecks">
<div class="charMemory_diagEmpty">Checking Vector Storage configuration...</div>
</div>
</div>
<div id="cm_wiz_vsWarning" class="charMemory_wizardVsWarning" style="display:none;">
⚠ You can continue — memories will be stored but not retrieved until Vector Storage is configured.
</div>
<div class="charMemory_wizardNav">
<input type="button" id="cm_wiz_back2" class="menu_button" value="← Back" />
<input type="button" id="cm_wiz_next2" class="menu_button" value="Next →" />
</div>
</div>`;
```
### Step 2: Add interval input wiring
After the provider change handler block (with the other `$wizard.on` handlers), add:
```js
$wizard.on('input', '#cm_wiz_interval', function () {
const val = Math.max(5, Math.min(500, parseInt($(this).val(), 10) || 20));
extension_settings[MODULE_NAME].interval = val;
saveSettingsDebounced();
});
```
### Step 3: Rewrite initStep2 with three-tier VS detection
Replace the entire `initStep2` function:
```js
async function initStep2() {
const $container = $wizard.find('#cm_wiz_healthChecks');
$container.html('<div class="charMemory_diagEmpty">Checking Vector Storage configuration...</div>');
$wizard.find('#cm_wiz_vsWarning').hide();
const vsSettings = extension_settings.vectors;
const filesEnabled = vsSettings?.enabled_files;
if (!filesEnabled) {
// Tier 1: VS Files not enabled
$container.html(`
<div class="charMemory_wizardCheck">
<i class="fa-solid fa-circle-xmark fa-sm" style="color:#c44;"></i>
<div class="charMemory_wizardCheckDetail">
<div class="charMemory_wizardCheckLabel">Vector Storage not enabled</div>
<div class="charMemory_wizardCheckText">CharMemory will store memories but your character won't recall them automatically. Enable <strong>Files</strong> in <strong>Extensions → Vector Storage</strong> when you're ready.</div>
</div>
</div>
`);
$wizard.find('#cm_wiz_vsWarning').show();
wizHealthResult = null;
return;
}
// Tier 2: VS enabled — check chunk settings
const chunkSize = vsSettings?.chunk_size_db ?? 2500;
const overlap = vsSettings?.overlap_percent_db ?? 0;
const badChunkSize = chunkSize < 500 || chunkSize > 1500;
const badOverlap = overlap === 0;
if (badChunkSize || badOverlap) {
let issues = [];
if (badChunkSize) issues.push(`chunk size is ${chunkSize} chars (recommended 800–1000)`);
if (badOverlap) issues.push('overlap is 0% (recommended 10–25%)');
$container.html(`
<div class="charMemory_wizardCheck">
<i class="fa-solid fa-circle-check fa-sm" style="color:#4a4;"></i>
<div class="charMemory_wizardCheckDetail">
<div class="charMemory_wizardCheckLabel">Vector Storage for files</div>
<div class="charMemory_wizardCheckText">Enabled — Data Bank files will be vectorized</div>
</div>
</div>
<div class="charMemory_wizardCheck">
<i class="fa-solid fa-triangle-exclamation fa-sm" style="color:#e8a33d;"></i>
<div class="charMemory_wizardCheckDetail">
<div class="charMemory_wizardCheckLabel">Settings may need tuning</div>
<div class="charMemory_wizardCheckText">Your ${issues.join(' and ')}. CharMemory works best with chunk size 800–1000 chars and overlap 10–25%. Adjust in <strong>Extensions → Vector Storage</strong>.</div>
</div>
</div>
`);
$wizard.find('#cm_wiz_vsWarning').show();
wizHealthResult = { level: 'yellow', checks: [] };
return;
}
// Tier 3: All good
$container.html(`
<div class="charMemory_wizardCheck">
<i class="fa-solid fa-circle-check fa-sm" style="color:#4a4;"></i>
<div class="charMemory_wizardCheckDetail">
<div class="charMemory_wizardCheckLabel">Vector Storage for files</div>
<div class="charMemory_wizardCheckText">Enabled — Data Bank files will be vectorized</div>
</div>
</div>
<div class="charMemory_wizardCheck">
<i class="fa-solid fa-circle-check fa-sm" style="color:#4a4;"></i>
<div class="charMemory_wizardCheckDetail">
<div class="charMemory_wizardCheckLabel">Chunk settings</div>
<div class="charMemory_wizardCheckText">Chunk size ${chunkSize} chars, overlap ${overlap}% — looks good for CharMemory.</div>
</div>
</div>
`);
wizHealthResult = { level: 'green', checks: [] };
}
```
### Step 4: Add CSS for step 2 new elements
In `style.css`, in the wizard section, add:
```css
.charMemory_wizardSection {
margin-bottom: 14px;
}
.charMemory_wizardSectionTitle {
font-size: 0.85em;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.04em;
opacity: 0.6;
margin-bottom: 4px;
}
.charMemory_wizSmallInput {
width: 60px;
text-align: center;
padding: 2px 4px;
margin: 0 4px;
background: var(--SmartThemeBodyColor, #1a1a1a);
color: var(--SmartThemeBodyTextColor, #ccc);
border: 1px solid var(--SmartThemeBorderColor, #555);
border-radius: 3px;
}
.charMemory_wizardVsWarning {
font-size: 0.85em;
color: #e8a33d;
padding: 6px 0;
margin-bottom: 6px;
}
```
### Step 5: Run tests and commit
```bash
npm test
git add index.js style.css
git commit -m "feat: wizard step 2 — extraction interval input, three-tier VS detection"
```
**Verify manually:**
- VS not enabled → red message + warning above Next
- VS enabled with default chunk settings → yellow tuning message
- VS enabled with good settings → two green checks
- Changing interval saves to extension_settings
---
## Task 5: Step 3 — Review & Go
**Files:**
- Modify: `index.js` — step3Html, initStep3()
### Step 1: Replace step3Html
Replace the entire `step3Html` constant:
```js
const step3Html = `
<div class="charMemory_wizardStep" data-step="3">
<div class="charMemory_wizardExplanation">
<strong>Review your setup</strong> — everything looks correct? Hit Get Started. If something's wrong, use Back.
</div>
<div id="cm_wiz_summary" class="charMemory_wizardSummary"></div>
<div class="charMemory_wizardSection" style="margin-top:10px;">
<div class="charMemory_wizardSectionTitle">Getting Started</div>
<p class="charMemory_helperText" style="margin:4px 0 0;">Open the <strong>Injection Sidebar</strong> from the dashboard to see which memories are being used in your character's prompt in real time.</p>
</div>
<div id="cm_wiz_convertSection" style="display:none;" class="charMemory_wizardConvertSection">
<div class="charMemory_wizardSectionTitle">Existing Memories Found</div>
<p class="charMemory_helperText" style="margin:4px 0 6px;">We found existing memories for <span id="cm_wiz_convertCharName"></span>. The <strong>Convert</strong> tool can reformat them for better retrieval.</p>
<input type="button" id="cm_wiz_convertNow" class="menu_button" value="Convert Now" />
<input type="button" id="cm_wiz_convertSkip" class="menu_button" value="Skip — I'll do this later" style="margin-left:6px;" />
</div>
<p class="charMemory_helperText" style="margin-top:auto; padding-top:10px; font-size:0.8em; opacity:0.6;">Tools like Clear Memories and Reset Extraction State only affect the current character.</p>
<div class="charMemory_wizardNav">
<input type="button" id="cm_wiz_back3" class="menu_button" value="← Back" />
<input type="button" id="cm_wiz_done" class="menu_button" value="Get Started" />
</div>
</div>`;
```
### Step 2: Replace initStep3 with async version
Replace the entire `initStep3` function:
```js
async function initStep3() {
const pk = extension_settings[MODULE_NAME].selectedProvider;
const p = PROVIDER_PRESETS[pk] || {};
const ps = getProviderSettings(pk);
const modelName = ps.model || p.defaultModel || '(default)';
const modelShort = modelName.length > 40 ? modelName.slice(0, 40) + '\u2026' : modelName;
const interval = extension_settings[MODULE_NAME].interval || 20;
const vsStatus = (() => {
if (!wizHealthResult) return '\u2014 Not checked';
if (wizHealthResult.level === 'green') return '\u2714 Ready';
if (wizHealthResult.level === 'yellow') return '\u26A0 Needs tuning';
return '\u26A0 Not configured';
})();
$wizard.find('#cm_wiz_summary').html(`
<div class="charMemory_wizardSummaryRow">
<span class="label">Provider</span>
<span>${escapeHtml(p.name || pk)}</span>
</div>
<div class="charMemory_wizardSummaryRow">
<span class="label">Model</span>
<span>${escapeHtml(modelShort)}</span>
</div>
<div class="charMemory_wizardSummaryRow">
<span class="label">Connection</span>
<span class="charMemory_wizardHighlight">${wizConnectionOk ? '\u2714 Connected' : '\u26A0 Not tested'}</span>
</div>
<div class="charMemory_wizardSummaryRow">
<span class="label">Extract every</span>
<span>${interval} messages</span>
</div>
<div class="charMemory_wizardSummaryRow">
<span class="label">Vector Storage</span>
<span class="charMemory_wizardHighlight">${vsStatus}</span>
</div>
`);
// Check for existing memories — show conversion section if found
const targets = getMemoryTargets();
const target = targets[0];
if (target) {
const content = await readMemoriesForCharacter(target.avatar, target.fileName);
if (content && content.trim()) {
$wizard.find('#cm_wiz_convertCharName').text(target.name);
$wizard.find('#cm_wiz_convertSection').show();
} else {
$wizard.find('#cm_wiz_convertSection').hide();
}
}
}
```
### Step 3: Update showStep to await initStep3
Find:
```js
if (step === 3) initStep3();
```
Replace with:
```js
if (step === 3) initStep3(); // async, fire and forget — DOM updates when ready
```
(No change needed — fire-and-forget is fine; the summary and convert section appear as they load.)
### Step 4: Wire up Convert Now and Skip buttons
After the existing `cm_wiz_done` click handler, add:
```js
$wizard.on('click', '#cm_wiz_convertNow', async function () {
const targets = getMemoryTargets();
const target = targets[0];
if (!target) return;
extension_settings[MODULE_NAME].wizardCompleted = true;
saveSettingsDebounced();
// Close wizard
$wizard.closest('.popup').find('.popup-button-ok, .popup-button-close').first().trigger('click');
// Launch convert flow after brief delay
setTimeout(() => convertWithLLM(target), 200);
});
$wizard.on('click', '#cm_wiz_convertSkip', function () {
// Just hide the section, user can convert later
$wizard.find('#cm_wiz_convertSection').hide();
});
```
### Step 5: Add CSS for step 3 convert section
```css
.charMemory_wizardConvertSection {
background: rgba(255, 255, 255, 0.04);
border: 1px solid var(--SmartThemeBorderColor, #555);
border-radius: 4px;
padding: 10px 12px;
margin-bottom: 10px;
}
```
### Step 6: Run tests and commit
```bash
npm test
git add index.js style.css
git commit -m "feat: wizard step 3 — review summary with interval, injection sidebar tip, existing memory conversion"
```
**Verify manually:**
- Summary shows provider, model, connection status, interval, VS status
- Injection sidebar tip is visible
- With existing memories: Convert section appears with character name
- Without existing memories: Convert section hidden
- Convert Now closes wizard and launches convert flow
- Skip hides the convert section
---
## Task 6: Destructive Action Scoping Language
**Files:**
- Modify: `index.js` — confirmation dialogs in Troubleshooter reset section
### Step 1: Update Reset Extraction State confirmation
Find the confirmation text for `cm_ts_resetTracking` (in `showTroubleshooter` event wiring):
```js
'Reset extraction tracking for the current character? Next extraction will re-read all messages.',
```
Replace with (using `charName` which is already available in `showTroubleshooter` scope):
```js
`Reset extraction tracking for ${charName || 'this character'}? CharMemory will re-process messages from the beginning. This only affects ${charName || 'this character'} — other characters are not affected.`,
```
### Step 2: Update Clear All Memories confirmation
Find the confirmation text for `cm_ts_clearMemories`:
```js
'Delete ALL memories for this character and reset extraction tracking? This cannot be undone.',
```
Replace with:
```js
`Delete all memories for ${charName || 'this character'} and reset extraction tracking? This cannot be undone.\n\n${extension_settings[MODULE_NAME].perChat ? 'Only this chat\'s memories will be deleted.' : 'In default mode, this deletes memories from all of ' + (charName || 'this character') + '\'s chats.'}`,
```
### Step 3: Run tests and commit
```bash
npm test
git add index.js
git commit -m "fix: destructive action confirmations include character name and scope clarification"
```
---
## Task 7: Push and Manual Test
### Step 1: Push beta branch
```bash
git push origin beta
```
### Step 2: Install in SillyTavern and test against test plan
Follow the manual test plan in the session context:
1. Fresh install wizard — verify no OK button, wizard appears on first load
2. Step 1 — NanoGPT model list visible, badges present, filters work
3. Step 2 — VS not enabled shows red, VS with bad settings shows yellow, good VS shows green; interval saves
4. Step 3 — summary reflects all config including interval; conversion prompt appears with existing memories
5. Dashboard gear icon opens wizard
6. Troubleshooter "Re-run Setup Wizard" works
7. Destructive confirmations include character name and scope

178
docs/plans/2026-03-02-tablet-mode-design.md Normal file
View file

@ -0,0 +1,178 @@
# Tablet Mode — Floating Dashboard Panel
## Problem
On iPad/tablet in landscape, SillyTavern's sidebar is narrow (~250px). The CharMemory dashboard gets squeezed — tool button names truncate ("Refo..."), activity log clips, stats bar wraps awkwardly, and all touch targets are too small. The sidebar paradigm doesn't work for tablet UX.
**Goal:** Add a user-selectable "Tablet Mode" that presents the dashboard as a non-modal, centered floating panel with touch-friendly sizing. No SillyTavern core changes — entirely within the extension.
---
## Approach: DOM Relocation
When the tablet panel opens, **move the actual DOM nodes** from the sidebar's `.inline-drawer-content` into a fixed-position floating panel appended to `<body>`. When it closes, move them back. This means:
- All event handlers survive (they're bound to specific element IDs, not parent containers)
- No HTML duplication or separate template
- No rewiring of button handlers — `$('#charMemory_extractNow')` etc. resolve to the same nodes
The sidebar drawer toggle is intercepted via a **capturing-phase** event listener that fires before ST's built-in handler. In tablet mode, `e.stopPropagation()` prevents the native sidebar expansion and opens the floating panel instead.
---
## Implementation
### 1. Setting: `tabletMode` (index.js)
Add to `defaultSettings` (~line 443):
```js
tabletMode: 'auto', // 'auto' | 'on' | 'off'
```
Add helper (~line 1504):
```js
function isTabletMode() {
const mode = extension_settings[MODULE_NAME].tabletMode || 'auto';
if (mode === 'on') return true;
if (mode === 'off') return false;
return 'ontouchstart' in window || navigator.maxTouchPoints > 0;
}
```
Tri-state avoids forcing desktop users to see it. `'auto'` detects touch at runtime (same pattern as troubleshooter line ~8097).
### 2. Panel HTML (index.js, init block ~line 8012)
Append to `<body>` after log drawer (same pattern as injection/log drawers):
```html
<div id="charMemory_tabletPanel" class="charMemory_tabletPanel">
<div class="charMemory_tabletHeader">
<b>CharMemory</b>
<div id="charMemory_tabletHeaderIcons" class="charMemory_tabletHeaderIcons">
<!-- Header icons relocated here from sidebar when open -->
</div>
<div class="charMemory_drawerClose" id="charMemory_tabletClose" title="Close">
<i class="fa-solid fa-xmark"></i>
</div>
</div>
<div id="charMemory_tabletBody" class="charMemory_tabletBody">
<!-- .inline-drawer-content children relocated here when open -->
</div>
</div>
```
### 3. Toggle Logic: `toggleTabletPanel()` (index.js, ~line 7596)
**Open:**
1. Detach `.charMemory_headerGear` spans from sidebar header → append to `#charMemory_tabletHeaderIcons`
2. Detach `.inline-drawer-content` children → append to `#charMemory_tabletBody`
3. Collapse sidebar drawer (hide content, chevron down)
4. Add `.open` class to panel
5. Call `updateStatusDisplay()` + `updateHealthIndicator()` (refresh stats)
**Close:**
1. Detach `#charMemory_tabletBody` children → append back to `.inline-drawer-content`
2. Detach `#charMemory_tabletHeaderIcons .charMemory_headerGear` → insert before chevron in sidebar header
3. Remove `.open` class
### 4. Sidebar Toggle Intercept (index.js, ~line 7234)
Replace the existing jQuery `click.charMemoryPanelOpen` handler with a **native capturing-phase** listener:
```js
drawerToggle.addEventListener('click', function (e) {
if (isTabletMode()) {
e.stopPropagation();
e.preventDefault();
toggleTabletPanel();
return;
}
// Normal mode: existing behavior (refresh on open)
setTimeout(() => { ... }, 50);
}, true); // capturing phase — fires before ST's handler
```
This is the key mechanism. ST's inline-drawer toggle handler fires in bubbling phase. Our capturing-phase listener runs first and can suppress it.
### 5. Event Wiring (index.js, init block ~line 8054)
- Close button: `$('#charMemory_tabletClose').on('click', () => toggleTabletPanel(false))`
- Tap outside to dismiss: `$(document).on('click.tabletPanelClose', ...)` — checks `!$(e.target).closest('#charMemory_tabletPanel').length`
- Swipe down to dismiss: `touchstart`/`touchend` on panel element, deltaY > 80px
### 6. Wizard Completion Update (index.js, ~line 5225)
When wizard finishes and opens the sidebar, check tablet mode:
```js
if (isTabletMode()) {
toggleTabletPanel(true);
} else {
// existing: trigger sidebar toggle
}
```
### 7. Settings Modal Toggle (index.js, ~line 3609)
Add to the Advanced section of `showSettingsModal()`:
- Dropdown: Auto (detect touch) / Always on / Off
- Handler saves to `extension_settings[MODULE_NAME].tabletMode`
- If switched to `'off'` while panel is open, closes panel and restores sidebar
### 8. CSS (style.css, append after line 1943)
**Panel positioning:**
- `position: fixed; top: 50%; left: 50%; transform: translate(-50%, -50%)`
- `width: 500px; max-width: 90vw; max-height: 80vh`
- `z-index: 1002` (above injection drawer 1000, log drawer 1001; below ST modals 9999+)
- Fade+scale animation via `.open` class toggle
- `border-radius: 12px; box-shadow: 0 8px 32px rgba(0,0,0,0.4)`
**Touch-friendly overrides (scoped to `.charMemory_tabletPanel`):**
| Element | Current | Tablet |
|---------|---------|--------|
| `.charMemory_headerGear` | padding: 2px 6px | padding: 10px 12px, min-width/height: 44px |
| `.menu_button` / `input[type="button"]` | ST default (~30px) | min-height: 44px, padding: 8px 16px |
| `.charMemory_autoPill` | padding: 3px 10px | padding: 8px 14px, min-height: 44px |
| `.charMemory_statItem` | padding: 4px 8px | padding: 8px 10px |
| `.charMemory_link` | font-size: 0.85em | min-height: 44px, inline-flex center |
| `.charMemory_healthDot` | 8x8px | 10x10px |
| `.charMemory_buttonRow` gap | 8px | 10px |
| `.charMemory_dashActivity` | max-height: 80px | max-height: 120px |
All overrides are scoped to `.charMemory_tabletPanel` — zero impact on desktop sidebar.
---
## Edge Cases
- **Chat changes while open:** `onChatChanged` calls `updateStatusDisplay()` which targets elements by ID — same nodes, works in either location
- **Modal opens while panel is open:** Panel z-index (1002) << modal z-index (9999+), panel stays behind
- **Extract Now from panel:** Async extraction, activity log updates live on the relocated DOM nodes
- **Browser resize / orientation change:** CSS `transform: translate(-50%, -50%)` stays centered; `max-width: 90vw` handles viewport changes
- **Injection/Log drawers open simultaneously:** Side drawers (right edge) don't overlap centered panel
- **Delegated handlers** (e.g., `$(document).on('click', '#charMemory_viewFullLog', ...)`): Match by ID regardless of DOM location — no issue
---
## Files Modified
| File | Change |
|------|--------|
| `index.js` | `defaultSettings.tabletMode`, `isTabletMode()`, panel HTML append, `toggleTabletPanel()`, sidebar toggle intercept, close/swipe wiring, wizard completion update, settings modal toggle |
| `style.css` | Panel positioning/animation, touch-friendly size overrides (all scoped to `.charMemory_tabletPanel`) |
`settings.html` — no changes (sidebar HTML is the source of relocatable DOM nodes)
---
## Verification
1. **Desktop regression:** With `tabletMode: 'off'` — sidebar toggle works exactly as before
2. **Touch detection:** Use Chrome DevTools device emulation (iPad) — `'auto'` mode should activate tablet panel
3. **DOM relocation round-trip:** Open panel → verify all buttons work → close panel → open sidebar normally → verify everything still works
4. **Stats update while open:** Switch chat with panel open — stats bar should reflect new character
5. **Extract from panel:** Click Extract Now — activity log updates in real time
6. **Modal stacking:** Open panel → click Settings gear → modal appears above panel
7. **Settings toggle:** Switch to "Off" while panel is open — panel closes, sidebar restores
8. **Swipe dismiss:** Swipe down on panel — closes
9. **Tap outside:** Tap chat area — panel dismisses

139
docs/plans/2026-03-06-per-chat-isolation-revised.md Normal file
View file

@ -0,0 +1,139 @@
# Per-Chat Memory Isolation — Revised Design
**Date:** 2026-03-06
**Status:** Proposed — seeking feedback from The_Istar before implementation
**Trigger:** Reddit thread (The_Istar) — each chat should have its own memories with no cross-chat bleed
---
## Problem
When `perChat` mode is enabled, CharMemory writes a separate memory file per chat. However, Vector Storage retrieves from *all* files in the character's Data Bank simultaneously. A character with 10 chats has 10 memory files, and VS retrieves relevant chunks across all of them — so memories from one story bleed into unrelated chats.
The user wants: opening chat A should only surface memories from chat A. No manual file management.
---
## Why the `disabled_attachments` Toggle Approach Won't Work
We investigated automating the manual workaround (toggling file visibility on chat switch). Research into SillyTavern's internals revealed three structural blockers:
### 1. URL Volatility
`disabled_attachments` is a flat array of URL strings (e.g. `"DATA:/uploads/global/abc123.md"`). Lookup is strict URL equality — there is no name-based matching.
CharMemory's `writeMemoriesForCharacter()` deletes and re-uploads the memory file on every extraction, generating a **new server URL** each time. Any URL we add to the disabled list goes stale within one extraction cycle. The file reappears in VS retrieval silently.
We could work around this by resolving filenames to URLs on every chat switch, but this adds fragile coupling to SillyTavern's internal attachment URL format.
### 2. Race Conditions
On chat switch, `CHAT_CHANGED` fires and multiple extensions respond asynchronously with no ordering guarantee. Vector Storage runs as a `generate_interceptor` (called before prompt building, sorted by loading order). There is no safe hook where CharMemory can modify `disabled_attachments` and be guaranteed that VS will see the updated list before its next retrieval.
### 3. Shared Global State
`disabled_attachments` is used by Vector Storage, the Data Bank UI, the `/db-disable` slash command, and potentially other extensions. Silently modifying it on every chat switch risks:
- Overwriting user's manual visibility settings
- Files appearing greyed out in the Data Bank UI with no explanation
- Conflict loops if the user re-enables files that CharMemory keeps disabling
---
## Recommended Approach: `chat_metadata` Storage + Direct Injection
Bypass Data Bank and Vector Storage entirely for isolated per-chat memories.
### How It Works
1. **Storage:** Extracted memories are written to `chat_metadata[MODULE_NAME].isolatedMemories` (plain text, same markdown format as current memory files). Since `chat_metadata` is per-chat by definition, isolation is structural — no visibility management needed.
2. **Injection:** On `GENERATE_BEFORE_COMBINE_PROMPTS` (or similar pre-generation event), CharMemory calls `context.setExtensionPrompt()` to inject the current chat's memories into the prompt, up to a configurable token budget. This is the same stable API used by Author's Note, Summarize, and Vector Storage itself.
3. **Extraction:** The extraction pipeline reads existing memories from `chat_metadata` instead of the Data Bank file, so it avoids re-extracting what's already captured.
### Trade-offs
| | Current (VS-based) | Isolated mode |
|---|---|---|
| Retrieval | Semantic — top-N relevant chunks | All memories up to token budget |
| Isolation | Cross-chat bleed | Fully isolated per-chat |
| Storage | Data Bank file (visible in UI) | `chat_metadata` field (travels with chat) |
| VS required | Yes | No |
| Consolidation | Works (operates on Data Bank file) | Works (operates on `chat_metadata` text) |
| Data Bank visible | Yes | No (viewable via CharMemory UI) |
**Losing semantic search:** For the isolation use case, this is acceptable. Per-chat memory pools are smaller than all-chats-pooled. The user wants *all* memories from this chat available, not a filtered subset. A token budget replaces relevance filtering.
### Setting Hierarchy
```
perChat: false → single file per character, VS retrieval (unchanged)
perChat: true → separate file per chat, VS retrieval (unchanged, current behavior)
perChat: true + isolatePerChat: true → per-chat storage in chat_metadata, direct injection, no VS
```
`isolatePerChat` is only available when `perChat` is enabled.
---
## Open Questions for Discussion
### 1. Token Budget
What's the right default for how much memory text to inject? Memory files can grow large over long chats. A configurable character limit (e.g. 4000 chars default) would cap injection size. Should it inject the most recent memories, or truncate from the oldest?
### 2. Injection Position
Where in the prompt should isolated memories appear? Options:
- Same position as VS injection (consistent with current behavior)
- Author's Note style depth-based positioning
- A dedicated configurable position in Settings
### 3. What Happens to Existing Per-Chat Data Bank Files?
Enabling isolation mode does NOT delete existing Data Bank files. They remain visible to VS and could still cause bleed. Options:
- Show a warning explaining this when the setting is enabled
- Offer a one-click "migrate and clean up" tool in the Troubleshooter that imports existing per-chat files into `chat_metadata` and removes them from the Data Bank
- Do both
### 4. Migration Path
Users switching from regular `perChat` mode to isolated mode should be able to import their existing memories. A migration tool could:
- Scan Data Bank for per-chat files matching the naming pattern
- Import each file's content into the corresponding chat's `chat_metadata`
- Optionally delete the Data Bank files after import
### 5. Viewing and Editing
Memories stored in `chat_metadata` aren't visible in the Data Bank UI. CharMemory needs:
- Memory Manager already works (it calls `readMemoriesForCharacter` — just needs to branch to read from `chat_metadata` in isolated mode)
- A "View isolated memories" section in the Troubleshooter for debugging
- Export capability (download as .md)
### 6. Group Chats
In group chats, each character has separate memories. Proposed data structure:
```js
// 1:1 chat
chat_metadata.charMemory.isolatedMemories = "<memory> blocks as plain text"
// Group chat (keyed by avatar filename)
chat_metadata.charMemory.isolatedMemories = {
"character-a.png": "<memory> blocks...",
"character-b.png": "<memory> blocks..."
}
```
---
## Implementation Scope (When Ready)
1. Add `isolatePerChat: false` to default settings
2. Add UI toggle in Settings > Storage (shown only when `perChat` is on), with warning about existing Data Bank files
3. Branch `readMemoriesForCharacter()` and `writeMemoriesForCharacter()` to use `chat_metadata` in isolated mode
4. Add injection hook via `setExtensionPrompt()` with configurable token budget
5. Update extraction pipeline to read existing memories from `chat_metadata`
6. Add "View isolated memories" to Troubleshooter
7. Build migration tool for importing existing per-chat Data Bank files
8. Update Memory Manager, Consolidation, and other tools to work with `chat_metadata` source

150
docs/prompt-design.md Normal file
View file

@ -0,0 +1,150 @@
# Prompt Design: Why the Prompts Changed in v1.7.0
This document explains the reasoning behind the v1.7.0 prompt changes. It's aimed at contributors and users who want to understand *why* the extraction format looks the way it does, rather than just *what* it looks like.
## The Problem: Vector Search Can't Tell Memories Apart
CharMemory stores memories as markdown in a Data Bank file. Vector Storage chunks that file, embeds each chunk, and retrieves the most relevant chunks when the AI generates a response. The quality of retrieval depends entirely on whether the embedding model can distinguish one memory chunk from another.
Before v1.7.0, the extraction prompt produced memory blocks like this:
```
<memory>
- She broke into a warehouse and stole a sealed envelope from a hidden safe.
- She delivered the envelope to her contact, who confirmed it contained what they needed.
</memory>
```
This worked fine for *storage*. But it failed at *retrieval*, because:
1. **No unique identifiers** — the block doesn't say who "she" is, or name the warehouse, the contact, or anything that distinguishes this event from other events.
2. **Thematic similarity** — a character with 50+ memory blocks about encounters with different people produces embeddings that all look alike. The embedding model sees "person did something somewhere" in every block and can't rank them.
3. **No block-level anchoring** — the embedding for a chunk containing 2-3 blocks is an average of everything in that chunk. Without distinctive content per block, the average is meaningless.
## What We Found Through Testing
During v1.7.0 development, we iterated through vectorization settings using the Injection Viewer to see exactly which memories were being retrieved for each AI response:
1. **Local Transformers (`all-MiniLM-L6-v2`)** with high chunk overlap → everything scored similarly, almost all memories injected regardless of relevance.
2. **Switched to `text-embedding-3-small` (via NanoGPT)** → better discrimination, but still too many false positives with thematically similar blocks.
3. **Raised score threshold** from 0.0 → 0.2 → 0.3 → got closer, but the core problem remained: blocks about different encounters with different people still scored nearly identically because they shared vocabulary and structure.
4. **Added topic tags** to memory blocks → immediately solved the discrimination problem. Mentioning "the vet" in chat now matched strongly against `[Flux, Alex — first vet visit and vaccinations]` and weakly against `[Flux, Alex — adoption day at the apartment]`, even though both blocks shared vocabulary.
The insight: **the prompt determines the shape of the embedding**. You can't fix retrieval by tuning Vector Storage settings alone — you have to produce text that's *embeddable* in the first place.
## The Three Prompt Changes
### 1. Topic Tags (Instruction 6)
**Before:**
```
6. HARD LIMIT: No more than 8 bullet points TOTAL.
```
**After:**
```
6. Start each block with a topic tag as the first bullet:
"- [Names involved — short description of encounter]"
(e.g., "- [Alex, Sarah — first visit to the apartment]").
This aids later retrieval.
```
The topic tag is the single most important change. It front-loads each block with:
- **Who** was involved (specific names, not "a friend")
- **What** the encounter was about (short, unique descriptor)
This gives the embedding model a discriminating anchor. When the tag text appears in the embedding, it creates a unique signature for that memory block even when the rest of the content shares vocabulary with other blocks.
### 2. Tighter Bullet Limits (Instruction 7)
**Before:**
```
HARD LIMIT: No more than 8 bullet points TOTAL.
```
**After:**
```
HARD LIMIT: No more than 5 bullet points per block (not counting the topic tag).
```
Two changes here:
- **8 → 5 bullets**: Forces the LLM to extract outcomes, not processes. With 8 bullets, the LLM writes play-by-play accounts. With 5, it has to prioritize.
- **"per block" not "TOTAL"**: The old limit was ambiguous — did "8 total" mean across all blocks, or per block? The new wording is explicit.
For vectorization, smaller blocks are better. A 200-400 character block gets a more focused embedding than a 1000-character block. With a chunk size of 1000, you fit 2-3 tight blocks per chunk instead of 1 bloated one.
### 3. Named Participants
**Before:**
```
(No explicit instruction about naming)
```
**After:**
```
- Always name specific people involved — use their name, not "a friend" or "someone"
```
Generic labels like "a client" or "someone" produce generic embeddings. "Sarah" is a unique token that the embedding model can match against. This instruction appears in both the WHAT TO EXTRACT section and the conversion prompt.
## Positive and Negative Examples
The old examples used a generic spy scenario:
```
Bad: 8 bullets narrating a warehouse heist step-by-step
Good: 2 bullets summarizing the outcome
```
The new examples use a specific character scenario (Flux, the cat character from our test fixtures):
```
Bad: 8 bullets narrating adoption day step-by-step
Good: topic tag + 3 bullets capturing the encounter
```
The change serves two purposes:
1. It demonstrates the topic tag format in context
2. It uses a concrete, memorable example that the LLM can pattern-match against (character names, specific objects, emotional beats)
## The Conversion Prompt
The conversion/reformat prompt (`defaultConversionPrompt`) was rewritten to align with the extraction prompt. Key differences from the old conversion prompt:
**Old prompt:**
- "Extract every distinct fact or piece of information as a bullet point"
- "Preserve ALL information — do not summarize, combine, or omit anything"
- Topic grouping by category: "Appearance", "Relationships", "Key Events"
**New prompt:**
- Topic tags required as first bullet
- 5-bullet limit per block with "combine related facts into single bullets rather than deleting information"
- Encounter-specific labels: "First day at the apartment", "Club night with Sam"
- Handles three input types: unstructured text, partially formatted blocks, already-formatted blocks
- Won't touch blocks that are already well-formatted (rule 10)
The old prompt's "preserve ALL" instruction was fundamentally incompatible with tight blocks. The new prompt prioritizes *retrievable* structure over exhaustive preservation. Information is preserved by combining related facts into single bullets, not by allowing unlimited bullets.
## The Group Extraction Prompt
The group prompt (`defaultGroupExtractionPrompt`) mirrors the solo prompt changes with additions specific to multi-character chats:
- Topic tags include the specific character whose memories are being extracted
- Instruction 10: "Reference other participants by name"
- Group dynamics in WHAT TO EXTRACT: "who allied with whom, who disagreed, power shifts"
Group memories are even harder for vector search because multiple characters produce similar encounter descriptions. Topic tags with participant names are essential.
## Impact on Existing Users
- **New users**: Get the new prompts automatically.
- **Existing users with default prompt**: Their extraction prompt is set to `""` (meaning "use default"). They'll get the new format on next extraction.
- **Existing users with customized prompts**: Unaffected. Their custom prompt is stored in settings. They can click "Restore Default" to opt in.
- **Existing memories**: Continue to work. The Convert tool (source: "Current memories") rewrites old-format memories into the topic-tagged format via LLM.
## Lessons
1. **Prompt design is retrieval design.** When your output gets embedded and vector-searched, the shape of the text determines retrieval quality. Prompts that produce good-looking text may produce terrible embeddings.
2. **Test with the Injection Viewer, not the output.** The extraction output looked fine before v1.7.0. The problem only became visible when we looked at *what was being retrieved* for each generation.
3. **Unique tokens matter more than structure.** Markdown formatting, clean grammar, and consistent structure don't help vector search. Unique names, specific descriptions, and distinctive vocabulary do.
4. **Tighter is better for chunking.** A 5-bullet block creates a more focused embedding than an 8-bullet block. The lost detail matters less than the gained precision.

133
docs/providers.md Normal file
View file

@ -0,0 +1,133 @@
# Providers
CharMemory works better when it uses its own LLM connection — separate from your main chat LLM. This keeps the extraction prompt clean: no chat personas, jailbreaks, or system prompts get mixed in, which produces noticeably better memories.
---
## Extraction sources
Three options for **LLM Used for Extraction** in Settings → Connection:
| Source | How it works | Best for |
|--------|-------------|----------|
| **Dedicated API** (default) | Direct API call with only the extraction prompt | Best quality — recommended |
| **WebLLM** | Small model running locally in your browser | Privacy and no API cost; limited quality |
| **Main LLM** | Uses whatever LLM your chat is using | Not recommended; No extra setup but quality suffers |
**Dedicated API** is the default and recommended option. The extraction prompt is the only thing sent — the character card is included as bounded reference context (so the LLM knows what *not* to re-extract), but no chat system prompts, personas, or jailbreaks contaminate the call.
---
## Dedicated API providers
Open Settings → Connection → choose a **Provider**:
| Provider | Auth | Notes |
|----------|------|-------|
| **Anthropic** | API key | Uses the Messages API (not OpenAI-compatible) |
| **DeepSeek** | API key | Strong instruction following at low cost |
| **Groq** | API key | Fast inference, limited model selection |
| **Mistral** | API key | Good quality, European-based |
| **NanoGPT** | API key | Subscription and open-source model options; see [NanoGPT](#nanogpt) below |
| **NVIDIA** | API key | Routes through SillyTavern server (CORS); see [NVIDIA](#nvidia) below |
| **Ollama** | None | Local; see [Local Servers](#local-servers) |
| **OpenAI** | API key | Reliable, standard `/models` endpoint |
| **OpenRouter** | API key | Access to many models under one key |
| **Pollinations** | None | Free, no account needed; see [Pollinations](#pollinations) below |
| **xAI (Grok)** | API key | Grok models |
| **Custom** | Optional | Any OpenAI-compatible endpoint |
### Setup steps
1. Select a provider
2. Enter your **API key** (click the **(get key)** link for a direct link to that provider's key page)
3. Click **Connect** to fetch available models
4. Search and select a **model**
5. Click **Test Connection** to confirm it responds correctly
---
## Recommended models
Memory extraction requires strong instruction following — the LLM must respect the extraction rules, stay within boundaries, and produce well-formatted output.
| Model | Notes |
|-------|-------|
| **GLM 4.7** | Best quality and speed. Highly recommended. On NVIDIA, uses reasoning tokens — set Max response length to 2000–3000. On NanoGPT, works at default settings. |
| **DeepSeek V3.1 / V3.2** | Solid instruction following, good second choice. |
| **Mistral Large 3** | Good quality, sometimes verbose. |
| **GPT-4.1 nano / mini** | Reliable at low cost. |
| **Llama 3.1 8B Instruct** | Fast and cheap, works well for testing. |
**Avoid:**
- **Qwen3-235B** — tends toward compressed play-by-play even with the current prompt
- **Very small models** — may reverse who did what or blur existing/new memory boundaries
- **Heavily censored models** — may refuse to extract from mature content, returning nothing
### Reasoning / thinking models
Some models spend part of their token budget on internal reasoning before producing output (e.g., GLM-4.7 on NVIDIA). CharMemory handles this transparently — it reads the reasoning output when the content field is empty. However, you need to increase **Max response length** to 2000–3000 to give the model enough budget for both reasoning and actual memory output. If you see "0 memories" with a thinking model, this is almost always the fix.
The Activity Log shows `[reasoning: N chars]` when a model uses reasoning tokens, so you can see at a glance how much of the budget is going to reasoning vs. output.
---
## Local servers
Select **Local Server** from the provider dropdown, then adjust the **Base URL** to match your backend:
| Backend | Default URL |
|---------|-------------|
| **Ollama** | `http://localhost:11434/v1` |
| **LM Studio** | `http://localhost:1234/v1` |
| **llama.cpp** | `http://localhost:8080/v1` |
| **KoboldCpp** | `http://localhost:5001/v1` |
You can also use a LAN IP (e.g., `http://192.168.1.50:5001/v1`) if the server is on another machine. No API key needed. Click **Connect** to fetch models, select one, and test.
**Ollama CORS**: Ollama requires `OLLAMA_ORIGINS=*` to accept browser requests. Set this as an environment variable before starting Ollama.
---
## NanoGPT
NanoGPT provides access to a wide range of models including both subscription and open-source options. When NanoGPT is selected, filter checkboxes appear above the model list:
| Filter | Description |
|--------|-------------|
| **Subscription** | Models included in your NanoGPT plan |
| **Open Source** | Open-source models |
| **Roleplay** | Models suited for storytelling |
| **Reasoning** | Models with reasoning capability |
Multiple filters combine as intersection (all checked filters must match). Models are grouped by their upstream provider.
NanoGPT also works as a vectorization source in Vector Storage — the same API key covers both extraction and embedding.
---
## Pollinations
Pollinations is free and requires no account or API key — useful for trying CharMemory without signing up for anything. Select **Pollinations**, type a model name (e.g., `openai`), and click Connect.
Quality depends on which model Pollinations routes to, so it's best for testing and evaluation rather than long-term use.
---
## NVIDIA
NVIDIA's API doesn't support browser-to-API CORS requests. CharMemory automatically routes NVIDIA calls through SillyTavern's server — your API key is passed securely via headers and never touches SillyTavern's configuration. No extra setup needed.
---
## Custom
Select **Custom** to use any OpenAI-compatible endpoint. Enter the base URL (e.g., `https://my-server.com/v1`) and an API key if required. Works with any backend that supports the `/chat/completions` format.
---
## Per-provider settings
Each provider stores its own API key, model selection, system prompt, and (for Custom) base URL independently. Switching providers preserves your settings for each — you can switch between providers without re-entering keys or re-selecting models.
The **System prompt** field lets you customize the system message sent to the extraction LLM. Useful for models that respond better to specific framing or instruction styles.

288
docs/retrieval-and-prompts.md Normal file
View file

@ -0,0 +1,288 @@
# Retrieval & Prompts
Retrieval quality depends on two things: **what the extraction prompt produces**, and **how Vector Storage is configured**. Most people start with Vector Storage settings — but the bigger lever is the prompt. A well-structured memory block retrieves accurately even with default settings. A poorly structured one won't retrieve well no matter how much you tune.
---
## How retrieval works
CharMemory extracts memories from your chat and stores them as markdown in the character's Data Bank. From there:
1. **Vector Storage chunks the file** — splits it into segments (configurable size, default ~1000 characters)
2. **Each chunk gets an embedding** — a numerical vector representing its meaning
3. **On each generation**, Vector Storage embeds the recent chat message, finds the most similar chunks, and injects them into the prompt
The character only "remembers" what Vector Storage retrieves. If a memory block doesn't match well against the current conversation, it won't be injected — even if it's relevant. This is why what the prompt produces matters as much as how Vector Storage is configured.
For a full explanation of SillyTavern's Data Bank and Vector Storage, see the official [Data Bank (RAG)](https://docs.sillytavern.app/usage/core-concepts/data-bank/) documentation.
---
## The extraction prompt
The extraction prompt is the single most impactful thing you can control for memory quality. It determines what gets stored and how it's structured — which directly determines what Vector Storage can retrieve later. The key insight behind CharMemory's prompt design: **the prompt determines the shape of the embedding.** Memory blocks that look good to a human reader may retrieve poorly under vector search if they lack distinctive tokens. Everything in the default prompt exists to produce memories that are both meaningful and retrievable.
The prompt is complex by design — it uses bounded reference sections, negative examples, layered constraints, and a litmus test for significance. This means **extraction quality depends heavily on a model with strong instruction following.** A capable model respects the card boundary, stays within the bullet limit, and writes outcome-oriented memories. A weaker model ignores the constraints and produces play-by-play, card-trait leakage, or duplicates of existing memories. See [Providers → Recommended models](providers.md#recommended-models) for tested options.
> **Cost awareness:** Every extraction is an LLM API call. Each call sends the extraction prompt, the character card, existing memories, and the recent chat messages — which can be a substantial number of tokens. With default settings (extract every 20 messages), steady-state costs are modest. But **usage is typically highest when you're getting started** — batch-extracting existing chats, running consolidation, re-extracting after prompt changes, and experimenting with settings all generate many more calls than normal chatting does. Group chats multiply this further (one call per character per chunk). If you're using a paid API provider, keep an eye on your usage dashboard during the setup and tuning phase. Once you're past initial setup and into regular chatting, costs settle to a predictable rhythm. See [Providers](providers.md) for free options like Pollinations and local servers.
The full prompt is visible and editable in **Settings → Prompts**. Below is a walkthrough of the design decisions — understanding them helps you make effective changes if the defaults don't fit your use case.
### Anatomy of the 1:1 prompt
The prompt has five functional sections:
**Bounded context.** The character card and existing memories are included as clearly delimited reference sections, bounded with `===== START / END =====` markers. Each section has explicit instructions: the card says "do NOT extract anything already described here," existing memories say "do NOT repeat, rephrase, or remix." Without the card boundary, the LLM re-extracts baseline traits every session — the most common extraction quality problem. Without existing memories as reference, it produces duplicates.
**Extraction rules.** Nine numbered instructions control the output format. The most consequential for retrieval:
- **Topic tag as the first bullet** — `[{{charName}}, OtherNames — short description]`. See [Topic tags](#topic-tags) below for why this is the single most important format decision.
- **One `<memory>` block per scene** — prevents splitting related events across blocks, which fragments the embedding and makes it harder for vector search to match the right context.
- **5-bullet limit per block** — forces outcome-oriented extraction. A 200–400 character block gets a focused embedding that represents one event well. A 1000-character block gets a diluted embedding that represents multiple events poorly. With more room, LLMs default to play-by-play narration.
- **Third person, past tense, character's name** — produces consistent text that embeds predictably. Pronouns like "she" carry no semantic signal for vector search; the character's actual name does.
- **"What happened" not "what was discussed"** — prevents meta-narration like "she told him about her past." The actual facts retrieve better than descriptions of conversations about facts.
**WHAT TO EXTRACT.** A guided list anchored by a litmus test: *"Would this character bring this up unprompted weeks or months later?"* This filters for lasting significance — backstory reveals, relationship changes, emotional turning points — rather than in-the-moment detail.
**DO NOT EXTRACT.** Explicit negative instructions targeting the most common extraction mistakes:
- **Card-trait leakage** — the LLM re-discovers what's already in the character card (e.g. if the card says "competitive", extracting "she felt proud when she won again")
- **Meta-narration** — writing about the conversation rather than the events ("she told him about X" instead of the actual fact X)
- **Play-by-play** — step-by-step scene recaps instead of outcomes
- **Temporary states** — things with no significance beyond the immediate moment
**Positive and negative examples.** A play-by-play example (labeled as bad) and a topic-tagged example (labeled as good) using Flux and Alex. Concrete examples are more effective than abstract rules for steering LLM output — the model pattern-matches against them.
### Topic tags
Each memory block starts with a topic tag as its first bullet: `[CharName, OtherNames — short description]`. This is the single most important factor in retrieval quality — more impactful than chunk size, overlap, or threshold tuning.
**Why they work:** When a character accumulates dozens of memory blocks, the blocks tend to share vocabulary and themes. Without distinctive identifiers, every block's embedding looks similar, and Vector Storage can't tell them apart. Topic tags front-load each block with unique anchors that the embedding model can latch onto:
- **The character's own name first** — embedding models weight early tokens heavily. The prompt requires `{{charName}}` first because without this explicit instruction, the LLM treats the POV character as implied and only lists other participants — losing a key discriminating signal.
- **Other participants by name** — "Sarah" is a unique token that creates a strong similarity signal when "Sarah" appears in the current chat. Generic labels like "a friend" or "someone" produce generic embeddings that match everything equally.
- **A short, specific descriptor** — "first vet visit and vaccinations" is distinctive; "important event" is not.
The difference is concrete: without topic tags, mentioning "the vet" in chat scores similarly against almost every memory block in the file. With them, it matches strongly against `[Flux, Alex — first vet visit and vaccinations]` and weakly against `[Flux, Alex — adoption day at the apartment]`, even though both blocks share vocabulary about Flux and Alex. This pattern held consistently during testing with a character (Flux — an orange tabby cat) whose ~50 memory blocks were all thematically similar. Tuning Vector Storage settings (embedding model, score threshold, chunk overlap) improved things incrementally, but topic tags were the step change.
If your older memory files don't have topic tags, use **Reformat** (Data Bank Tools section) to convert them. The improvement in retrieval precision is typically significant.
### What good extraction looks like
**Before (play-by-play — retrieves poorly):**
```
- Alex set the carrier down on the hardwood floor and opened the metal door.
- Flux emerged from the carrier and walked toward the Gundam Roomba by the window.
- Alex poured premium salmon pâté into a ceramic bowl and placed it near the kitchen island.
- Flux ate the salmon and began purring for the first time.
- Alex assembled a cat tree in the corner and Flux climbed to the top perch.
```
**After (topic-tagged, tight — retrieves well):**
```
- [Flux, Alex — adoption day and settling into the apartment]
- Alex adopted Flux and brought him to his penthouse apartment, where Flux immediately bonded with his custom Gundam-styled Roomba.
- Flux's first meal of premium salmon pâté triggered his first purr in the new home.
- Alex assembled a cat tree that Flux claimed as a second perch, alternating between it and the Roomba.
```
The topic tag anchors the embedding. The three bullets capture outcomes, not steps. When someone mentions "adoption" or "Roomba" in chat, this block scores high. The play-by-play version would score about the same as every other block in the file.
### Why the group prompt is different
Group chats need a separate extraction prompt because the context is fundamentally different. In a 1:1 chat, there's one character and one user — the LLM knows whose memories it's extracting. In a group chat, multiple characters are speaking, and extraction runs once per member on the same message chunk.
The group prompt mirrors the 1:1 structure but adds:
- **`{{participants}}` list** — the LLM needs to know who's in the group and who is speaking in each message, so it can attribute actions and dialogue correctly.
- **Explicit character targeting** — "Character whose memories you are extracting: {{charName}}" appears prominently. Without this, the LLM tends to extract memories for everyone rather than focusing on the target character.
- **Participant naming rule** — instruction 10 says "Reference other participants by name. Include who was involved in events, who said what to whom, who was present." In group chats, who did what to whom is the primary differentiator between memory blocks. Named participants in topic tags become even more critical because multiple characters may share similar events.
- **Group dynamics** — the WHAT TO EXTRACT list adds "who allied with whom, who disagreed, power shifts" — relationship signals that are unique to multi-character interactions.
Both prompts share the same format rules (topic tags, bullet limits, DO NOT EXTRACT list). Changes to one don't affect the other — they're independently editable in Settings → Prompts.
### Consolidation prompts
When you use **Consolidate** to merge duplicate or related memories, a different prompt handles the task. Three strategy presets are available, each with its own prompt:
- **Conservative** — merges near-exact duplicates only. Preserves the most detail. Use when you want minimal change.
- **Balanced** — merges near-duplicates and combines closely related facts, but preserves all unique information. Good for periodic cleanup.
- **Aggressive** — compresses heavily, groups by theme, and summarizes rather than listing individual events. Best for very large memory files that need significant reduction.
All three require the same format rules as extraction — topic tags as the first bullet with the character's name first, 5-bullet limit, named participants. The key difference is the instruction to *merge* rather than *extract*: the LLM is working with existing memories, combining related entries while preserving the format that makes them retrievable. When two blocks about the same event get merged, the prompt instructs the LLM to update the topic tag to reflect the combined content.
> Consolidation sends the entire memory file to the LLM in a single call. For characters with large memory files, this can be a significant number of tokens. Review the Activity Log after consolidating to see the token usage.
### Conversion prompt
The **Reformat** tool uses a conversion prompt to restructure existing memories into the topic-tagged format. This serves a different purpose from extraction — it's not pulling new information from chat, it's reorganizing text that already exists.
The conversion prompt has a strict "do NOT add, infer, or invent" rule because the input is existing memories, not raw chat. It handles three input types: unstructured text (plain notes), partially formatted blocks (some structure but no topic tags), and already-formatted blocks (which it leaves unchanged). This makes it safe to run on any memory file — well-formatted blocks pass through untouched while old-format blocks get upgraded.
All prompts are independently editable in **Settings → Prompts**. Changes to one don't affect the others.
### Adapting to your use case
The default prompts were designed and tested with a character whose memories were episodic (events happening over time) and thematically similar (many encounters with different people in similar settings). This is a common but not universal pattern. Depending on how you use characters and chats, you may need to adapt.
**Characters with highly varied content** — if your character's memories naturally span very different topics (magic spells vs. political intrigue vs. personal relationships), topic tags may be less critical because the vocabulary already differs across blocks. The default prompt still works, but you could relax the bullet limit if you find it too restrictive.
**Short, focused chats** — if each chat is a self-contained scenario rather than an ongoing story, you may want to lower the extraction interval so memories are captured before the chat ends. The default prompt's emphasis on long-term significance ("Would they bring this up months later?") may also be too selective — consider adjusting the WHAT TO EXTRACT litmus test in Settings → Prompts.
**Multiple timelines or scenarios for one character** — by default, all chats for a character share one memory file. This is ideal when chats are a continuous story — memories from Monday's chat inform Tuesday's. But if the same character appears in incompatible scenarios (a modern AU and a fantasy setting, or a "what if" branch), shared memories create cross-contamination. Enable **Separate memories per chat** in Settings → Storage to give each conversation its own isolated memory file with its own vector index. See [Managing Memories → Per-chat memories](managing-memories.md#per-chat-memories) for details and limitations.
**Persistent worlds with many NPCs** — names in topic tags become even more important. If your character interacts with dozens of named NPCs, the topic tags are doing the heavy lifting for disambiguation. You may also want to increase Retrieve chunks to inject more context per generation.
**Characters whose card changes frequently** — the DO NOT EXTRACT boundary relies on the character card being stable. If you update the card often, the LLM may not extract things that later get removed from the card. Consider extracting core traits as memories before major card rewrites.
**Non-English chats** — the default prompt is in English, and embedding models vary in multilingual quality. If your chats are in another language, test whether the embedding model you're using handles that language well. Topic tags in the chat's language (rather than English) may retrieve better.
The extraction prompt is fully editable — you can see and modify it in **Settings → Prompts**. Separate prompts exist for 1:1 chats, group chats, consolidation, and conversion. See [Managing Memories → Extraction prompt](managing-memories.md#extraction-prompt) for the template variables available in each prompt.
---
## Vector Storage setup
### The one setting that must be on
Open **Extensions → Vector Storage** → scroll to **File vectorization settings** → check **Enable for files**.
![Vector Storage panel with "Enable for files" checkbox](../images/vs-file-settings.png)
This is the most common reason memories aren't being recalled. Everything else is tuning — this is the on/off switch.
### Choosing an embedding source
The embedding model converts memories into numerical vectors. The quality of the model determines how well Vector Storage can distinguish relevant memories from irrelevant ones — a weak model makes every memory look vaguely similar, so retrieval becomes a grab-bag.
| Source | Quality | Speed | Cost |
|--------|---------|-------|------|
| **Local (Transformers)** | Adequate | Slow on first run | Free |
| **OpenAI** | Excellent | Fast | ~$0.01/1M tokens |
| **NanoGPT** | Excellent | Fast | Check current pricing |
| **Ollama** | Good–Excellent | Fast (local GPU) | Free |
| **Cohere / Jina / Voyage** | Good–Excellent | Fast | Free tier / cheap |
> Provider availability, model selection, and pricing change frequently. This list reflects what was available and tested at the time of writing — check your provider's current documentation for up-to-date details.
**Recommendations:**
- Getting started or privacy-focused: **Local (Transformers)** is fine. The default model (`all-MiniLM-L6-v2`) works but has lower discrimination than larger models.
- Best retrieval quality: **`text-embedding-3-small`** via OpenAI or NanoGPT. If you already use NanoGPT for extraction, the same API key works for embedding.
- Local with better quality: **`nomic-embed-text`** via Ollama.
> **Score thresholds vary by model.** A score of 0.3 on `text-embedding-3-small` means something different than 0.3 on `all-MiniLM-L6-v2`. If you switch embedding models, re-tune your score threshold and re-vectorize.
### Recommended settings
These apply to the **Data Bank files** row in Vector Storage (the bottom row — CharMemory uses Data Bank, not message attachments).
These settings were arrived at iteratively through testing with `text-embedding-3-small` and a character with ~50 memory blocks covering thematically similar content. Treat them as a starting point — your optimal values will depend on your embedding model, memory file size, and how distinctive your memories are.
| Setting | Starting value | Notes |
|---------|---------------|-------|
| **Size threshold** | 1 KB | Files below this get a single embedding. At 1 KB, chunking kicks in so specific memories can be retrieved. |
| **Chunk size** | 1000 chars | Each chunk gets its own embedding. At ~200–400 chars per memory block, 1000 fits 2–3 blocks — enough for clean retrieval without mixing unrelated topics. |
| **Chunk overlap** | 0% | Copies the end of each chunk into the start of the next. With small topic-tagged blocks, 0% works well as a starting point. Increase to 10–15% if you see blocks getting split (check the [Injection Viewer](injection-viewer.md)). |
| **Retrieve chunks** | 2 | Chunks injected per generation. At 2–3 blocks per chunk, this gives ~4–6 memory blocks — enough context without flooding the prompt. |
| **Score threshold** | 0.3 | Filters out low-relevance chunks. Without this, Vector Storage injects its top N regardless of relevance. Start at 0.3; lower to 0.2 if too few memories inject, raise to 0.4 if irrelevant ones appear. |
| **Query messages** | 1 | Recent messages used as the search query. 1 keeps retrieval focused on the current topic. |
These are starting points. See [Tuning](#tuning) below for how to iterate.
---
## Tuning
The goal: when your character discusses a topic, the *right* memories inject — not random ones, not nothing, not everything.
**Check what's being injected** using the [Injection Viewer](injection-viewer.md). Open it on a recent character message and look at the CharMemory section. Ask:
- Are relevant memories present?
- Are irrelevant ones appearing?
- Is the section empty when it shouldn't be?
**Common symptoms and fixes:**
| Symptom | Likely cause | Fix |
|---------|-------------|-----|
| No memories injected | Score threshold too high, file not vectorized, or "Enable for files" off | Lower threshold to 0.2; check the health dot |
| Irrelevant memories injected | Score threshold too low | Raise to 0.3–0.4 |
| Half-blocks injected (bullets without topic tag) | Chunk size too small or block straddling boundary | Increase chunk size or add 10–15% overlap |
| Too many memories (prompt flooding) | Retrieve chunks too high | Reduce to 2 |
| Same memory injected multiple times | Chunk overlap too high or duplicate blocks in file | Reduce overlap; consolidate duplicates |
**Change one setting at a time**, then purge and re-vectorize before testing again.
### Using an LLM to debug retrieval
If you're not getting the results you want and can't identify why, using an LLM interactively is one of the most effective approaches — it's how much of the tuning in CharMemory's development was actually done. Provide:
1. **The injected content** — copy from the Injection Viewer's CharMemory section for a specific message
2. **Your full memory file** — the raw markdown from the Data Bank
3. **The recent chat context** — the last few messages that drove the retrieval query
4. **Your Vector Storage settings** — chunk size, score threshold, retrieve chunks, embedding model
Then ask it: *"Given this conversation, which memories should have been retrieved? Were any relevant ones missed? Are any injected ones irrelevant? Are the memory blocks distinctive enough for vector search to tell apart?"*
The LLM can spot patterns you'd miss — blocks that are too similar to each other, important context filtered by a threshold that's too high, or memories worded too differently from the current chat to match semantically. This kind of interactive debugging with full context often resolves retrieval issues faster than adjusting settings by trial and error.
### Using an LLM to improve extraction prompts
The same approach works for iterating on the extraction prompt itself. CharMemory's default prompts were developed through extensive LLM-assisted iteration — pasting the prompt, sample output, and the source chat into a conversation and asking the LLM to identify why the output wasn't meeting expectations.
If your extractions are producing poor results (play-by-play, card-trait leakage, missing important events), provide:
1. **Your current extraction prompt** — copy from Settings → Prompts
2. **A sample extraction result** — the actual `<memory>` blocks produced
3. **The source chat messages** — the messages that were sent to the LLM for extraction
4. **What you expected** — which memories should have been extracted, and what format you wanted
Then ask: *"Why did this prompt produce play-by-play instead of outcomes? What instruction changes would fix this? Are the positive/negative examples clear enough?"*
The LLM can identify ambiguous instructions, missing constraints, or examples that inadvertently encourage the wrong behavior. This is particularly useful if you've customized the prompt and something regressed — the LLM can compare your version against the default and pinpoint what changed.
---
## Purge and re-vectorize
When you change chunk size, overlap, or the embedding model, the existing index is stale. You need to rebuild it:
1. Open **Extensions → Vector Storage**
2. Click **Purge Vectors** for the memory file (deletes the index, not the file itself)
3. Generate a message — Vector Storage automatically re-chunks and re-embeds on the next generation
Also re-vectorize after:
- Consolidating memories (file content changed)
- Reformatting with the Reformat tool
- Editing the memory file directly
- Migrating from an old format
In group chats, re-vectorization requires switching to each character individually — SillyTavern processes the active character's Data Bank on each generation. See [Group Chats → How retrieval works in group chats](group-chats.md#how-retrieval-works-in-group-chats).
---
## Why Data Bank and not lorebooks?
SillyTavern has two injection systems: [Data Bank](https://docs.sillytavern.app/usage/core-concepts/data-bank/) (vector-based) and Lorebooks/World Info (keyword-based). CharMemory uses Data Bank. The short version:
- **Semantic retrieval vs. keyword matching** — Vector Storage finds memories about the vet visit even if the word "vet" isn't in the current message. Lorebook entries only fire when an exact keyword appears.
- **Zero maintenance** — no trigger keywords to define or maintain. Extract and forget.
- **Scales to hundreds of blocks** — managing 100+ lorebook entries is painful. Vector search handles large memory files without any manual organization.
CharMemory and lorebooks coexist — they use separate storage and retrieval systems. Use lorebooks for stable world-building facts; CharMemory for episodic memories that accumulate over time.
### If you use both CharMemory and lorebooks
When both are active, they compete for context budget. A few things to be aware of:
- **Reduce Retrieve chunks if lorebooks are injecting heavily.** If your lorebooks inject a lot of content, reduce CharMemory's retrieve chunks to 1–2 to avoid flooding the prompt. The goal is enough memory context to be useful without drowning out the lorebook content (or vice versa).
- **Check the Injection Viewer** to see what's actually going into the prompt from both sources. The Lorebook Entries and Extension Prompts sections show exactly what lorebook content was injected alongside CharMemory's memories.
### Vector Storage settings and World Info
SillyTavern's Vector Storage panel has separate setting rows for different data types: **Chat messages**, **Data Bank files**, and **World Info**. CharMemory uses the **Data Bank files** row (the bottom one). Each row has its own chunk size, overlap, retrieve chunks, and score threshold — they're independent.
If you also use World Info (lorebooks) with vector-based scanning enabled, be aware of how the settings interact:
- **Separate rows, shared embedding source.** The Data Bank and World Info rows have independent chunking and retrieval settings, but they use the same embedding model. If you switch embedding models, both are affected and may need their score thresholds re-tuned.
- **World Info scanning settings** are configured in the World Info panel, not in Vector Storage. The **Scan** toggle and related options (scan count, scan depth) control how aggressively SillyTavern searches lorebook entries for matches. These don't affect CharMemory's Data Bank retrieval, but they do affect the total context injected alongside memories.
- **Score threshold has different optimal values per row.** A score threshold of 0.3 may work well for CharMemory's topic-tagged memory blocks but be too aggressive or too lenient for your lorebook entries, which have different content structure. Tune each row independently.
- **Context budget is shared.** Both Data Bank memories and World Info entries compete for the same context window. If you're injecting a lot from both sources, the combined context may crowd out the actual chat history or system prompt. Monitor total injection size via the Injection Viewer's Extension Prompts section.

119
docs/troubleshooting.md Normal file
View file

@ -0,0 +1,119 @@
# Troubleshooting
Start with the **health dot** in the CharMemory stats bar — it runs automated checks and tells you what's wrong. If the dot is green, your setup is working correctly. If it's yellow or red, click it to jump to Diagnostics for a check-by-check breakdown.
---
## Health checks
The health dot runs up to 9 checks depending on what data is available. Checks 1–7 run immediately when you open a chat. Checks 8–9 run after at least one generation has been captured.
| Check | What it looks for | States |
|-------|-------------------|--------|
| **Files enabled** | "Enable for files" is on in Vector Storage | RED if off |
| **Memory file exists** | A memory file is in the character's Data Bank | YELLOW if missing |
| **File vectorized** | Memory file has been indexed (chunk count > 0) | YELLOW if 0 chunks (resolves on next generation) |
| **Chunk overlap** | Data Bank overlap setting in Vector Storage | YELLOW if 0% |
| **Chunk size** | Data Bank chunk size vs. average memory block size | YELLOW if outside recommended range |
| **Retrieve chunks** | Number of chunks Vector Storage retrieves per query | YELLOW if > 5 (recommended: 2–3) |
| **Score threshold** | Minimum similarity score for retrieval | YELLOW if not set or too low |
| **Memories injected** | Memory bullets appeared in the last generation | YELLOW if 0 injected (may be normal — score threshold filtering) |
| **Duplicate detection** | Same bullet appears more than once in injected content | YELLOW if duplicates found |
### Fixing each issue
**RED — Files not enabled**
Open Extensions → Vector Storage → under File vectorization settings, check **Enable for files**.
**RED — Memory file not found**
Run an extraction first. The memory file is created on first successful extraction — Extract Now or wait for auto-extraction to fire.
**YELLOW — File not yet vectorized**
The file exists but hasn't been indexed yet. This is normal when a memory file was just created — Vector Storage indexes on the next generation. Just send a message and generate a response. If the check stays yellow after generating, check that your vectorization source is configured and responding. If you recently purged vectors, generate a message to trigger re-indexing.
**YELLOW — No memories injected**
File exists and is vectorized, but 0 memories appeared in the last generation. This can be normal — it means no memories scored above the relevance threshold for the current conversation topic. If you expect memories to be injected, try lowering the score threshold to 0.2 in Vector Storage → Data Bank files row. Also check that **Retrieve chunks** isn't set to 0.
**YELLOW — Chunk overlap is 0%**
With 0% overlap, a memory block landing on a chunk boundary gets split — neither half retrieves cleanly, and you may see duplicate partial bullets in injected content. 0% is a valid starting point (especially with small, topic-tagged blocks), but if you notice split blocks in the Injection Viewer, increase overlap to 10–15% in Vector Storage → Data Bank files. [Purge and re-vectorize](retrieval-and-prompts.md#purge-and-re-vectorize) after changing.
**YELLOW — Chunk size out of range**
Too small: memory blocks get split across chunks. Too large: unrelated blocks share one embedding and retrieval loses precision. See [Retrieval & Prompts → Recommended settings](retrieval-and-prompts.md#recommended-settings) for guidance.
**YELLOW — Duplicates in injected content**
The same memory bullet is appearing multiple times in what's injected. Usually means a block is split across a chunk boundary. Increase chunk overlap and/or adjust chunk size, then purge and re-vectorize.
---
## Common issues
**"0 memories" after extraction**
Check the Activity Log (click **View full log** in the dashboard). It shows exactly what happened — whether the LLM returned `NO_NEW_MEMORIES`, produced unparseable output, or hit an error. Enable **Verbose** in the log to see the full prompt and response.
If verbose mode shows `finish=length` with completion tokens used but no content, you're using a reasoning/thinking model that needs a higher **Max response length** — increase to 2000–3000 in Settings.
**Memories extracted but character doesn't recall them**
Vector Storage isn't configured, or "Enable for files" is off. Check the health dot — RED means something is preventing injection. Open [Retrieval & Prompts](retrieval-and-prompts.md) and work through the setup.
**Extraction never fires automatically**
Check that the **Auto** pill in the panel is active (highlighted). Also verify the message counter is incrementing in the stats bar, and that the cooldown timer isn't blocking it.
**"No unprocessed messages" on Extract Now**
All messages have already been extracted. Use **Reset Extraction State** in Settings → Reset / Clear to re-read from the beginning.
**Memories contain facts already in existing memories**
The extraction prompt sends existing memories as reference with instructions not to repeat them. If the model keeps doing it, it's likely too small to respect the boundary. Switch to a larger model — DeepSeek V3.1 or GLM 4.7.
**Memories reverse who did what**
Same cause — model too small for accurate comprehension. Use a larger model.
**Memories too sparse from a long existing chat**
Expected when batch-extracting hundreds of turns at once — the LLM only sees one chunk at a time and can't judge significance across the full conversation. CharMemory works best extracting incrementally as you chat. For existing chats, try increasing **Messages per LLM call** to 40–50 and re-running batch extraction.
**Memories too granular / play-by-play**
The prompt has a bullet limit and negative examples to prevent this. If it persists, increase **Messages per LLM call** to give the LLM more context per call — more messages means it can better judge what's significant vs. trivial.
**Duplicate memories accumulating in the file**
Use **Consolidate** to merge them. To prevent recurrence, check that [Purge and re-vectorize](retrieval-and-prompts.md#purge-and-re-vectorize) is in your workflow after major memory file changes.
**Memories contain system metadata, relationship tables, or image prompts**
CharMemory strips code blocks, markdown tables, `<details>` sections, and HTML tags before sending messages to the LLM. If metadata still leaks through, edit the **AVOID** section in the extraction prompt (Settings → Prompts).
---
## Diagnostics
Click the **wrench icon** in the CharMemory panel header to open the Troubleshooter, then go to the **Diagnostics** section. Click **Refresh** after generating a message to capture the current state.
![Diagnostics panel showing memory file status and health checks](../images/diagnostics.png)
Diagnostics shows:
- **Memory file** — name, whether it exists, bullet and block count, vectorization status and chunk count
- **Injected Memories — Last Generation** — the exact bullets that were injected for the most recent generation. If a memory exists but isn't here, it wasn't semantically relevant enough to pass the score threshold, or Vector Storage settings need adjustment.
- **Character Lorebooks** — lorebook books bound to this character, with entry counts
- **Activated Entries — Last Generation** — which lorebook entries actually fired, based on keyword matches
- **Extension Prompts** — raw content injected by all extensions, including the full `4_vectors_data_bank` entry showing everything Vector Storage sent
- **Injection Health** — the health check card with each check's status and recommendation
> **Group chat caveat**: Diagnostics shows info for the first group member only. To check a specific character, use View / Edit which shows all members.
---
## Generating a diagnostic report
The Troubleshooter can generate a full diagnostic report — a text dump of your current settings, provider configuration, memory file status, and health check results. Use this when asking for help, to give others full context without manually describing your setup.
Click the **wrench icon** → **Diagnostic Report** → copy the output.
---
## Reset tools
> **Before using Clear All Memories**, make a backup. Use SillyTavern's [backup tools](https://docs.sillytavern.app/usage/user-settings/) to snapshot your data directory, or download the memory file from the Data Bank.
See [Managing Memories → Reset and Clear](managing-memories.md#reset-and-clear) for what each reset option does and when to use it. The short version:
- **Reset Extraction State** — re-process messages from the beginning, no memory loss
- **Reset Batch Progress** — re-run batch extraction from scratch, risk of duplicates if memories aren't also cleared
- **Clear All Memories** — deletes the memory file, cannot be undone

110
editor.js Normal file
View file

@ -0,0 +1,110 @@
import { cloneMemoryBlocks, countMatchesInBlocks, getTimestamp, reindexEditingSet, replaceInBlocks } from './lib.js';
/**
* Create a memory block editor with state management and undo.
* Pure state logic — no DOM. Callers handle rendering and event binding.
*
* @param {object} options
* @param {Array<{chat: string, date: string, bullets: string[]}>} options.blocks - Initial blocks
* @returns {object} Editor API
*/
export function createMemoryEditor({ blocks }) {
let editorBlocks = cloneMemoryBlocks(blocks);
const versionStack = [];
const editingSet = new Set();
function saveVersion() {
versionStack.push(cloneMemoryBlocks(editorBlocks));
}
return {
getBlocks() {
return cloneMemoryBlocks(editorBlocks);
},
deleteBullet(blockIndex, bulletIndex) {
if (!editorBlocks[blockIndex]) return;
saveVersion();
editorBlocks[blockIndex].bullets.splice(bulletIndex, 1);
if (editorBlocks[blockIndex].bullets.length === 0) {
editorBlocks.splice(blockIndex, 1);
reindexEditingSet(editingSet, blockIndex);
}
},
deleteBlock(blockIndex) {
if (!editorBlocks[blockIndex]) return;
saveVersion();
editorBlocks.splice(blockIndex, 1);
reindexEditingSet(editingSet, blockIndex);
},
addBullet(blockIndex) {
if (!editorBlocks[blockIndex]) return;
saveVersion();
editorBlocks[blockIndex].bullets.push('');
editingSet.add(blockIndex);
},
addBlock(timestamp) {
saveVersion();
editorBlocks.push({
chat: 'New Group',
date: timestamp || getTimestamp(),
bullets: [''],
});
editingSet.add(editorBlocks.length - 1);
},
updateBullet(blockIndex, bulletIndex, text) {
if (!editorBlocks[blockIndex]) return;
editorBlocks[blockIndex].bullets[bulletIndex] = text;
},
updateTheme(blockIndex, label) {
if (!editorBlocks[blockIndex]) return;
editorBlocks[blockIndex].chat = label;
},
undo() {
if (versionStack.length === 0) return false;
editorBlocks = versionStack.pop();
return true;
},
canUndo() {
return versionStack.length > 0;
},
replaceAll(newBlocks) {
editorBlocks = cloneMemoryBlocks(newBlocks);
versionStack.length = 0;
editingSet.clear();
},
toggleEdit(blockIndex) {
if (editingSet.has(blockIndex)) {
editingSet.delete(blockIndex);
} else {
editingSet.add(blockIndex);
}
},
isEditing(blockIndex) {
return editingSet.has(blockIndex);
},
getEditingSet() {
return new Set(editingSet);
},
countMatches(find, caseSensitive = false) {
return countMatchesInBlocks(editorBlocks, find, caseSensitive);
},
findAndReplaceAll(find, replace, caseSensitive = false) {
saveVersion();
return { replacements: replaceInBlocks(editorBlocks, find, replace, caseSensitive) };
},
};
}

BIN
images/01-extension-panel.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 36 KiB

BIN
images/02-install-dialog.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 1,010 KiB

BIN
images/03-message-ids.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 734 KiB

BIN
images/04-settings-provider.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 48 KiB

BIN
images/05-stats-bar.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 20 KiB

BIN
images/06-memory-manager.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 75 KiB

BIN
images/07-vector-storage.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 751 KiB

BIN
images/08-activity-log.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 33 KiB

BIN
images/10-batch-extract.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 29 KiB

BIN
images/11-extract-here.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 115 KiB

BIN
images/12-group-main.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 88 KiB

BIN
images/13-group-member-files.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 15 KiB

BIN
images/14-group-memory-manager.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 43 KiB

BIN
images/15-model-picker.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 33 KiB

BIN
images/16-model-search.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 45 KiB

BIN
images/17-group-prompt-title.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 22 KiB

BIN
images/18-convert-tool.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 39 KiB

BIN
images/19-convert-preview.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 154 KiB

BIN
images/20-format-settings.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 61 KiB

BIN
images/21-injection-viewer-lorebooks.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 191 KiB

BIN
images/23-stats-bar-health.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 14 KiB

BIN
images/batch-panel.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 52 KiB

BIN
images/brain-icon-message.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 23 KiB

BIN
images/consolidation-preview.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 353 KiB

BIN
images/databank-group.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 138 KiB

BIN
images/diagnostics.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 148 KiB

BIN
images/injection-viewer.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 708 KiB

BIN
images/memory-manager-editing.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 85 KiB

BIN
images/memory-manager-group-both.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 171 KiB

BIN
images/memory-manager.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 93 KiB

BIN
images/panel-full.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 117 KiB

BIN
images/panel-group.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 114 KiB

BIN
images/panel-header.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
images/stats-bar.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 14 KiB

BIN
images/vs-file-settings.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 80 KiB

BIN
images/wizard-step1-connected.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 135 KiB

BIN
images/wizard-step1.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 63 KiB

BIN
images/wizard-step2.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 118 KiB

BIN
images/wizard-step3.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 134 KiB

6045
index.js
View file

File diff suppressed because it is too large Load diff

524
lib.js Normal file
View file

@ -0,0 +1,524 @@
/**
* CharMemory — Pure utility functions.
*
* This module contains all side-effect-free functions extracted from index.js
* so they can be independently tested. Nothing here touches the DOM,
* SillyTavern globals, or network.
*/
// ─── XML attribute escaping ────────────────────────────────────────────
export function escapeAttr(text) {
return String(text).replace(/&/g, '&amp;').replace(/"/g, '&quot;');
}
export function unescapeAttr(text) {
return String(text).replace(/&quot;/g, '"').replace(/&amp;/g, '&');
}
// ─── HTML escaping ─────────────────────────────────────────────────────
export function escapeHtml(text) {
return String(text)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
// ─── Memory parsing ────────────────────────────────────────────────────
/**
* Parse <memory> blocks from raw markdown content.
* @param {string} content Raw file content.
* @returns {{chat: string, date: string, bullets: string[]}[]}
*/
export function parseMemories(content) {
if (!content || !content.trim()) return [];
const blocks = [];
const regex = /<memory\b([^>]*)>([\s\S]*?)<\/memory>/gi;
let match;
while ((match = regex.exec(content)) !== null) {
const attrs = match[1];
const body = match[2];
const chatMatch = attrs.match(/chat="([^"]*)"/);
const dateMatch = attrs.match(/date="([^"]*)"/);
const chat = chatMatch ? unescapeAttr(chatMatch[1]) : 'unknown';
const date = dateMatch ? unescapeAttr(dateMatch[1]) : '';
const bullets = body.split('\n')
.map(line => line.trim())
.filter(line => line.startsWith('- ') || /^\[.*?\]\s*-\s/.test(line))
.map(line => {
const metaMatch = line.match(/^\[.*?\]\s*-\s+(.+)/);
if (metaMatch) return metaMatch[1].trim();
return line.slice(2).trim();
})
.filter(Boolean);
if (bullets.length > 0) {
blocks.push({ chat, date, bullets });
}
}
return blocks;
}
/**
* Split a bullet array containing multiple topic tags into separate arrays.
* Topic tags match the "[Names — description]" pattern (em dash, en dash, or hyphen
* surrounded by spaces). If 0 or 1 topic tags, returns the original array unchanged.
* @param {string[]} bullets Array of bullet strings (without "- " prefix)
* @returns {string[][]} Array of bullet arrays, one per topic-tagged section
*/
export function splitMultiTagBullets(bullets) {
if (bullets.length === 0) return [bullets];
const isTopicTag = b => /^\[.+ [—–\-] .+\]$/.test(b);
const tagIndices = [];
for (let i = 0; i < bullets.length; i++) {
if (isTopicTag(bullets[i])) tagIndices.push(i);
}
if (tagIndices.length <= 1) return [bullets];
const groups = [];
for (let i = 0; i < tagIndices.length; i++) {
const start = i === 0 ? 0 : tagIndices[i];
const end = i + 1 < tagIndices.length ? tagIndices[i + 1] : bullets.length;
groups.push(bullets.slice(start, end));
}
return groups;
}
/**
* Count total individual memories (bullets) across all blocks.
* @param {{bullets: string[]}[]} blocks Parsed memory blocks.
* @returns {number}
*/
export function countMemories(blocks) {
return blocks.reduce((sum, b) => sum + b.bullets.length, 0);
}
// ─── Memory serialization ──────────────────────────────────────────────
const DEFAULT_FORMAT = { boundary: 'block', separator: '\n\n', metadata: false };
/**
* Serialize an array of memory blocks back to <memory> tag format.
* @param {{chat: string, date: string, bullets: string[]}[]} blocks
* @param {{boundary: string, separator: string, metadata: boolean}} [format]
* @returns {string}
*/
export function serializeMemories(blocks, format) {
const fmt = format || DEFAULT_FORMAT;
if (fmt.boundary === 'bullet') {
return blocks.map(b => {
const bulletsText = b.bullets.map(bullet => {
if (fmt.metadata) {
return `[${b.date} | ${b.chat}] - ${bullet}`;
}
return `- ${bullet}`;
}).join('\n\n');
return `<memory chat="${escapeAttr(b.chat)}" date="${escapeAttr(b.date)}">\n${bulletsText}\n</memory>`;
}).join('\n\n');
}
if (fmt.boundary === 'custom') {
return blocks.map(b => {
const bulletsText = b.bullets.map(bullet => {
if (fmt.metadata) {
return `[${b.date} | ${b.chat}] - ${bullet}`;
}
return `- ${bullet}`;
}).join('\n');
return `<memory chat="${escapeAttr(b.chat)}" date="${escapeAttr(b.date)}">\n${bulletsText}\n</memory>`;
}).join(fmt.separator);
}
// Default block-level
return blocks.map(b => {
const bulletsText = b.bullets.map(bullet => `- ${bullet}`).join('\n');
return `<memory chat="${escapeAttr(b.chat)}" date="${escapeAttr(b.date)}">\n${bulletsText}\n</memory>`;
}).join('\n\n');
}
// ─── Memory block merging ──────────────────────────────────────────────
/**
* Merge memory blocks that share the same chat ID.
* @param {{chat: string, date: string, bullets: string[]}[]} blocks
* @returns {{chat: string, date: string, bullets: string[]}[]}
*/
export function mergeMemoryBlocks(blocks) {
const merged = [];
const seen = new Map();
for (const block of blocks) {
const key = block.chat;
if (seen.has(key)) {
seen.get(key).bullets.push(...block.bullets);
} else {
const copy = { chat: block.chat, date: block.date, bullets: [...block.bullets] };
seen.set(key, copy);
merged.push(copy);
}
}
return merged;
}
// ─── Format detection & migration ──────────────────────────────────────
/**
* Migrate old memory formats to <memory> tag format if needed.
* @param {string} content Existing file content.
* @returns {string} Content in <memory> tag format.
*/
export function migrateMemoriesIfNeeded(content) {
if (!content || !content.trim()) return content;
if (/<memory\b[^>]*>/i.test(content)) return content;
const timestamp = getTimestamp();
if (/^## Memory \d+/m.test(content)) {
const parts = content.split(/^## Memory \d+\s*$/m);
const blocks = [];
for (let i = 1; i < parts.length; i++) {
const part = parts[i].trim();
if (!part) continue;
let date = timestamp;
let text = part;
const tsMatch = part.match(/^_Extracted:\s*(.+?)_\s*\n/);
if (tsMatch) {
date = tsMatch[1].trim();
text = part.slice(tsMatch[0].length).trim();
}
const lines = text.split('\n').map(l => l.trim()).filter(Boolean);
const bullets = lines.filter(l => l.startsWith('- ')).map(l => l.slice(2).trim());
if (bullets.length === 0 && text.trim()) {
bullets.push(text.trim());
}
if (bullets.length > 0) {
blocks.push({ chat: 'unknown', date, bullets });
}
}
return serializeMemories(blocks);
}
const lines = content.trim().split('\n').map(l => l.trim()).filter(Boolean);
const bullets = lines.filter(l => l.startsWith('- ')).map(l => l.slice(2).trim());
if (bullets.length === 0) {
bullets.push(content.trim());
}
return serializeMemories([{ chat: 'unknown', date: timestamp, bullets }]);
}
/**
* Detect the format of a Data Bank file's content.
* @param {string} content Raw file content.
* @returns {'memory_tags'|'memory_headings'|'bullets'|'numbered'|'markdown_headings'|'freeform'}
*/
export function detectFileFormat(content) {
if (!content || !content.trim()) return 'freeform';
if (/<memory\b[^>]*>/i.test(content)) return 'memory_tags';
if (/^## Memory \d+/m.test(content)) return 'memory_headings';
const lines = content.split('\n').filter(l => l.trim());
const bulletLines = lines.filter(l => /^\s*[-*]\s/.test(l));
if (bulletLines.length > lines.length * 0.4) return 'bullets';
const numberedLines = lines.filter(l => /^\s*\d+[\.\)]\s/.test(l));
if (numberedLines.length > lines.length * 0.3) return 'numbered';
if (/^#{1,3}\s+.+/m.test(content)) return 'markdown_headings';
return 'freeform';
}
/**
* Convert file content to <memory> tag format using heuristic parsing.
* @param {string} content Raw file content.
* @param {string} format Detected format from detectFileFormat().
* @returns {{blocks: {chat: string, date: string, bullets: string[]}[], warnings: string[]}}
*/
export function convertHeuristic(content, format) {
const today = getTimestamp();
const warnings = [];
if (format === 'memory_tags') {
warnings.push('Already in CharMemory format \u2014 no conversion needed.');
return { blocks: parseMemories(content), warnings };
}
if (format === 'memory_headings') {
const migrated = migrateMemoriesIfNeeded(content);
return { blocks: parseMemories(migrated), warnings };
}
if (format === 'bullets') {
const lines = content.split('\n');
const bullets = [];
for (const line of lines) {
const match = line.match(/^\s*[-*]\s+(.+)/);
if (match) bullets.push(match[1].trim());
}
return {
blocks: [{ chat: 'imported', date: today, bullets }],
warnings,
};
}
if (format === 'numbered') {
const lines = content.split('\n');
const bullets = [];
for (const line of lines) {
const match = line.match(/^\s*\d+[\.\)]\s+(.+)/);
if (match) bullets.push(match[1].trim());
}
return {
blocks: [{ chat: 'imported', date: today, bullets }],
warnings,
};
}
if (format === 'markdown_headings') {
const blocks = [];
let currentHeading = 'imported';
let currentBullets = [];
for (const line of content.split('\n')) {
const headingMatch = line.match(/^#{1,3}\s+(.+)/);
if (headingMatch) {
if (currentBullets.length > 0) {
blocks.push({ chat: currentHeading, date: today, bullets: currentBullets });
currentBullets = [];
}
currentHeading = headingMatch[1].trim();
continue;
}
const bulletMatch = line.match(/^\s*[-*]\s+(.+)/);
if (bulletMatch) {
currentBullets.push(bulletMatch[1].trim());
} else if (line.trim()) {
currentBullets.push(line.trim());
}
}
if (currentBullets.length > 0) {
blocks.push({ chat: currentHeading, date: today, bullets: currentBullets });
}
return { blocks, warnings };
}
// Freeform: split on sentences
const sentences = content.replace(/\n/g, ' ').split(/(?<=[.!?])\s+/).map(s => s.trim()).filter(Boolean);
if (sentences.length === 0) {
warnings.push('File appears empty.');
return { blocks: [], warnings };
}
warnings.push('Freeform text detected \u2014 results may be rough. Consider using LLM restructuring for better quality.');
return {
blocks: [{ chat: 'imported', date: today, bullets: sentences }],
warnings,
};
}
// ─── Text utilities ────────────────────────────────────────────────────
/**
* Truncate text to a maximum character count, breaking at newline boundaries.
* @param {string} text
* @param {number} maxChars
* @returns {string}
*/
export function truncateText(text, maxChars) {
if (!text || text.length <= maxChars) return text;
const truncated = text.slice(0, maxChars);
const lastNewline = truncated.lastIndexOf('\n');
return (lastNewline > maxChars * 0.5 ? truncated.slice(0, lastNewline) : truncated) + '\n[...truncated]';
}
/**
* Adjust a Set of indices after an element is removed from an array.
* @param {Set<number>} editingSet Set of active indices (mutated in place).
* @param {number} removedIndex The index that was removed.
*/
export function reindexEditingSet(editingSet, removedIndex) {
const updated = new Set();
for (const idx of editingSet) {
if (idx < removedIndex) updated.add(idx);
else if (idx > removedIndex) updated.add(idx - 1);
}
editingSet.clear();
for (const idx of updated) editingSet.add(idx);
}
// --- Non-diegetic content stripping ---
/**
* Strip non-diegetic content from a message: code blocks, details sections,
* markdown tables, HTML tags, and excessive newlines.
* @param {string} text Raw message text.
* @returns {string} Cleaned text.
*/
export function stripNonDiegetic(text) {
return text
.replace(/```[\s\S]*?```/g, '')
.replace(/<details[\s\S]*?<\/details>/gi, '')
.replace(/\|[^\n]*\|(?:\n\|[^\n]*\|)*/g, '')
.replace(/<[^>]*>/g, '')
.replace(/\n{3,}/g, '\n\n');
}
// --- Chat message formatting ---
/**
* Format chat messages for extraction prompt. Filters out empty/system-only
* messages, strips non-diegetic content, returns "Name: text" format.
* @param {Array<{name: string, mes: string, is_user?: boolean, is_system?: boolean}>} chatArray
* @param {number} startIndex Start index (inclusive) in chatArray.
* @param {number} endIndex End index (exclusive) in chatArray.
* @returns {{ text: string, startIndex: number, endIndex: number, messageCount: number }}
*/
export function formatChatMessages(chatArray, startIndex, endIndex) {
if (!chatArray || chatArray.length === 0) return { text: '', startIndex: -1, endIndex: -1, messageCount: 0 };
const safeStart = Math.max(0, startIndex);
const safeEnd = Math.min(chatArray.length, endIndex);
if (safeStart >= safeEnd) return { text: '', startIndex: -1, endIndex: -1, messageCount: 0 };
const slice = chatArray.slice(safeStart, safeEnd);
const lines = [];
for (const msg of slice) {
if (!msg.mes) continue;
if (msg.is_system && !msg.is_user && !msg.name) continue;
const text = stripNonDiegetic(msg.mes).trim();
if (!text) continue;
lines.push(`${msg.name}: ${text}`);
}
return {
text: lines.join('\n\n'),
startIndex: safeStart,
endIndex: safeEnd - 1,
messageCount: lines.length,
};
}
// --- Prompt template substitution ---
/**
* Substitute CharMemory template variables in a prompt string.
* @param {string} template Prompt template with {{variable}} placeholders.
* @param {Object} vars Variable values to substitute.
* @param {string} [vars.charName]
* @param {string} [vars.charCard]
* @param {string} [vars.existingMemories]
* @param {string} [vars.recentMessages]
* @param {string} [vars.participants]
* @returns {string} Prompt with variables replaced.
*/
export function substitutePromptTemplate(template, vars) {
let result = template;
if (vars.charName != null) result = result.replace(/\{\{charName\}\}/g, vars.charName);
if (vars.charCard != null) result = result.replace(/\{\{charCard\}\}/g, vars.charCard);
result = result.replace(/\{\{existingMemories\}\}/g, vars.existingMemories || '(none yet)');
if (vars.recentMessages != null) result = result.replace(/\{\{recentMessages\}\}/g, vars.recentMessages);
if (vars.participants != null) result = result.replace(/\{\{participants\}\}/g, vars.participants);
return result;
}
// ─── Timestamp utility ──────────────────────────────────────────────────
/**
* Generate a YYYY-MM-DD HH:MM timestamp string.
* @param {Date} [date] - Date to format. Defaults to now.
* @returns {string}
*/
export function getTimestamp(date) {
const now = date || new Date();
return `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, '0')}-${String(now.getDate()).padStart(2, '0')} ${String(now.getHours()).padStart(2, '0')}:${String(now.getMinutes()).padStart(2, '0')}`;
}
// ─── Memory block cloning ───────────────────────────────────────────────
/**
* Deep-clone an array of memory blocks (shallow object clone + bullet array copy).
* @param {Array<{chat: string, date: string, bullets: string[]}>} blocks
* @returns {Array<{chat: string, date: string, bullets: string[]}>}
*/
export function cloneMemoryBlocks(blocks) {
return blocks.map(b => ({ ...b, bullets: [...b.bullets] }));
}
// ─── Find & replace across memory blocks ────────────────────────────────
/**
* Escape a string for use in a RegExp.
* @param {string} str
* @returns {string}
*/
function escapeRegex(str) {
return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
/**
* Count how many times `find` appears across all bullets and chat labels in blocks.
* @param {Array<{chat: string, date: string, bullets: string[]}>} blocks
* @param {string} find Search string (plain text, not regex).
* @param {boolean} [caseSensitive=false]
* @returns {number}
*/
export function countMatchesInBlocks(blocks, find, caseSensitive = false) {
if (!find) return 0;
const flags = caseSensitive ? 'g' : 'gi';
const re = new RegExp(escapeRegex(find), flags);
let count = 0;
for (const block of blocks) {
count += (block.chat.match(re) || []).length;
for (const bullet of block.bullets) {
count += (bullet.match(re) || []).length;
}
}
return count;
}
/**
* Replace all occurrences of `find` with `replace` across all bullets and chat labels.
* Mutates the blocks array in place.
* @param {Array<{chat: string, date: string, bullets: string[]}>} blocks
* @param {string} find Search string (plain text, not regex).
* @param {string} replace Replacement string (plain text).
* @param {boolean} [caseSensitive=false]
* @returns {number} Total number of replacements made.
*/
export function replaceInBlocks(blocks, find, replace, caseSensitive = false) {
if (!find) return 0;
const flags = caseSensitive ? 'g' : 'gi';
const re = new RegExp(escapeRegex(find), flags);
// Use replacer function to avoid $& / $' / $` interpolation in replacement string
const replacer = () => replace;
let count = 0;
for (const block of blocks) {
const chatMatches = (block.chat.match(re) || []).length;
if (chatMatches) {
block.chat = block.chat.replace(re, replacer);
count += chatMatches;
}
for (let i = 0; i < block.bullets.length; i++) {
const bulletMatches = (block.bullets[i].match(re) || []).length;
if (bulletMatches) {
block.bullets[i] = block.bullets[i].replace(re, replacer);
count += bulletMatches;
}
}
}
return count;
}

2
manifest.json
View file

@ -6,7 +6,7 @@
"js": "index.js",
"css": "style.css",
"author": "bal-spec",
"version": "1.6.2",
"version": "2.1.6",
"homePage": "",
"auto_update": false
}

13
package.json Normal file
View file

@ -0,0 +1,13 @@
{
"private": true,
"type": "module",
"scripts": {
"test": "vitest run test/unit/",
"test:watch": "vitest watch test/unit/",
"test:snapshot": "vitest run test/integration/snapshot",
"test:live": "vitest run test/integration/live"
},
"devDependencies": {
"vitest": "^4.0.18"
}
}

412
settings.html
View file

@ -1,12 +1,31 @@
<div class="charMemory_settings">
<div class="inline-drawer">
<div class="inline-drawer-toggle inline-drawer-header">
<b>Character Memory</b>
<b>CharMemory</b>
<span id="charMemory_openWizard" class="charMemory_headerGear" title="Setup Wizard">
<i class="fa-solid fa-wand-magic-sparkles fa-sm"></i>
</span>
<span id="charMemory_openTroubleshooter" class="charMemory_headerGear" title="Troubleshooter">
<i class="fa-solid fa-screwdriver-wrench fa-sm"></i>
</span>
<span id="charMemory_openSettingsModal" class="charMemory_headerGear" title="Open Settings">
<i class="fa-solid fa-gear fa-sm"></i>
</span>
<span id="charMemory_toggleInjectionBtn" class="charMemory_headerGear" title="Toggle Injection Sidebar">
<i class="fa-solid fa-syringe fa-sm"></i>
</span>
<div class="inline-drawer-icon fa-solid fa-circle-chevron-down down"></div>
</div>
<div class="inline-drawer-content">
<!-- Stats bar — always visible -->
<!-- Nudge banner -->
<div id="charMemory_nudgeBanner" class="charMemory_nudgeBanner" style="display:none;">
<i class="fa-solid fa-triangle-exclamation fa-sm"></i>
<span>Something needs attention</span>
<input type="button" id="charMemory_nudgeFix" class="menu_button" value="View" />
</div>
<!-- Stats bar -->
<div class="charMemory_statsBar">
<div class="charMemory_statItem" title="The Data Bank file where memories are stored for this character">
<i class="fa-solid fa-file-lines fa-sm"></i>
@ -31,387 +50,46 @@
</div>
</div>
<!-- Top-level tabs -->
<div class="charMemory_tabs">
<button class="charMemory_tab active" data-tab="main">Main</button>
<button class="charMemory_tab" data-tab="tools">Tools</button>
<button class="charMemory_tab" data-tab="settings">Settings</button>
<button class="charMemory_tab" data-tab="log">Log</button>
</div>
<!-- Main tab (default) -->
<div class="charMemory_tabContent" id="charMemory_tabMain">
<div class="charMemory_sectionHeader">
<small><b>Memory Extraction</b></small>
<label class="checkbox_label" for="charMemory_enabled" title="When enabled, memories are extracted automatically after a set number of new messages">
<input type="checkbox" id="charMemory_enabled" />
<small>Automatic</small>
</label>
</div>
<!-- Primary Controls -->
<div class="charMemory_dashSection">
<div class="charMemory_buttonRow">
<input type="button" id="charMemory_extractNow" class="menu_button" value="Extract Now" title="Extract memories from unprocessed messages. If all messages have been processed, use 'Reset Extraction State' first to re-read from the beginning." />
<button id="charMemory_autoExtractPill" class="charMemory_autoPill" title="Toggle automatic extraction — when on, memories are extracted automatically after a set number of new messages">
<i class="fa-solid fa-arrows-rotate fa-xs"></i> Auto
</button>
<input type="button" id="charMemory_manageMemories" class="menu_button" value="View / Edit" title="Browse, edit, and delete individual stored memories" />
</div>
</div>
<!-- Tools tab -->
<div class="charMemory_tabContent" id="charMemory_tabTools" style="display:none;">
<div class="charMemory_toolPills">
<button class="charMemory_toolPill active" data-tool="consolidate">Consolidate</button>
<button class="charMemory_toolPill" data-tool="batch">Batch</button>
<button class="charMemory_toolPill" data-tool="convert">Convert</button>
</div>
<!-- Consolidate tool -->
<div class="charMemory_toolContent" id="charMemory_toolConsolidate">
<div class="charMemory_promptSection">
<label for="charMemory_consolidationStrategy">
<small>Consolidation strategy</small>
</label>
<select id="charMemory_consolidationStrategy" class="text_pole">
<option value="conservative">Conservative — only merge near-exact duplicates</option>
<option value="balanced">Balanced — merge duplicates & related facts (default)</option>
<option value="aggressive">Aggressive — compress heavily, summarize themes</option>
</select>
<small id="charMemory_consolidationPreview" class="charMemory_helperText" style="font-style:italic;"></small>
<details class="charMemory_promptDisclosure" id="charMemory_promptDisclosure">
<summary><small>Show prompt</small></summary>
<textarea id="charMemory_consolidationPrompt" class="text_pole textarea_compact" rows="6" placeholder="Edit the consolidation prompt for this strategy..."></textarea>
<div class="charMemory_buttonRow">
<input type="button" id="charMemory_restorePresetDefault" class="menu_button" value="Restore Default" title="Reset this preset's prompt to its built-in default" style="display:none;" />
</div>
</details>
</div>
<div class="charMemory_buttonRow">
<input type="button" id="charMemory_consolidate" class="menu_button" value="Consolidate" title="Use the LLM to merge duplicate and related memories into fewer, cleaner entries" />
<input type="button" id="charMemory_undoConsolidate" class="menu_button" value="Undo Consolidation" title="Restore memories from before the last consolidation (session only)" disabled />
</div>
</div>
<!-- Batch tool -->
<div class="charMemory_toolContent" id="charMemory_toolBatch" style="display:none;">
<div class="charMemory_buttonRow">
<input type="button" id="charMemory_batchRefresh" class="menu_button" value="Refresh" title="Load chat list for this character" />
<input type="button" id="charMemory_batchExtract" class="menu_button" value="Extract Selected" title="Run extraction on all selected chats" disabled />
<input type="button" id="charMemory_batchStop" class="menu_button" value="Stop" title="Cancel batch extraction" style="display:none;" />
</div>
<div id="charMemory_batchProgress" class="charMemory_batchProgress" style="display:none;">
<div class="charMemory_batchProgressText"></div>
<div class="charMemory_batchProgressBar"><div class="charMemory_batchProgressFill"></div></div>
</div>
<div class="charMemory_sectionHeader">
<small><b title="Chat files attached to this character. Select which ones to extract memories from.">Character Attachments</b></small>
<label class="checkbox_label">
<input type="checkbox" id="charMemory_batchSelectAll" />
<small>Select all</small>
</label>
</div>
<div id="charMemory_batchChatList" class="charMemory_batchChatList">
<div class="charMemory_diagEmpty">Click "Refresh" to load chats.</div>
</div>
</div>
<!-- Convert tool -->
<div class="charMemory_toolContent" id="charMemory_toolConvert" style="display:none;">
<div class="charMemory_statusRow">
<label for="charMemory_convertSource">
<small>Source file</small>
</label>
<select id="charMemory_convertSource" class="text_pole">
<option value="">— Select a Data Bank file —</option>
</select>
<small class="charMemory_helperText">Select any file from this character's Data Bank to convert into CharMemory format.</small>
</div>
<div class="charMemory_statusRow">
<label class="checkbox_label" for="charMemory_convertUseLLM">
<input type="checkbox" id="charMemory_convertUseLLM" />
<span>Use LLM to restructure (for freeform text)</span>
</label>
<small class="charMemory_helperText">When the file has no clear structure, send it to the LLM for intelligent restructuring. Uses your configured extraction provider.</small>
<details class="charMemory_promptDisclosure" id="charMemory_convertPromptDisclosure">
<summary><small>Show prompt</small></summary>
<textarea id="charMemory_convertPrompt" class="text_pole textarea_compact" rows="6" placeholder="Conversion prompt..."></textarea>
<div class="charMemory_buttonRow">
<input type="button" id="charMemory_restoreConvertPrompt" class="menu_button" value="Restore Default" title="Reset conversion prompt to built-in default" />
</div>
</details>
</div>
<div class="charMemory_buttonRow">
<input type="button" id="charMemory_convertPreview" class="menu_button" value="Preview Conversion" title="Parse the selected file and show an interactive before/after preview" />
</div>
</div>
</div>
<!-- Settings tab -->
<div class="charMemory_tabContent" id="charMemory_tabSettings" style="display:none;">
<!-- LLM Used for Extraction -->
<div class="charMemory_statusRow">
<label for="charMemory_source" title="Which LLM to use for memory extraction and consolidation">
<small>LLM Used for Extraction</small>
</label>
<select id="charMemory_source" class="text_pole">
<option value="provider">Dedicated API (recommended)</option>
<option value="webllm">WebLLM (browser-local)</option>
<option value="main_llm">Main LLM</option>
</select>
<small class="charMemory_helperText"><b>Dedicated API is recommended.</b> Main LLM pollutes the extraction prompt with chat context, system prompts, and other instructions that degrade memory quality.</small>
<div id="charMemory_providerSettings" style="display:none;">
<div class="charMemory_statusRow">
<label><small>Provider</small></label>
<select id="charMemory_providerSelect" class="text_pole">
</select>
</div>
<div class="charMemory_statusRow" id="charMemory_providerApiKeyRow">
<label><small>API Key <a id="charMemory_providerHelpLink" href="#" target="_blank" style="font-size:0.85em;">(get key)</a></small></label>
<div style="display:flex;gap:5px;align-items:center;">
<input type="password" id="charMemory_providerApiKey" class="text_pole" placeholder="Enter API key" style="flex:1;" />
<button type="button" id="charMemory_providerApiKeyReveal" class="menu_button" title="Show/hide API key" style="padding:3px 8px;">
<i class="fa-solid fa-eye fa-sm"></i>
</button>
</div>
</div>
<div class="charMemory_statusRow" id="charMemory_providerBaseUrlRow" style="display:none;">
<label><small>Base URL</small></label>
<input type="text" id="charMemory_providerBaseUrl" class="text_pole" placeholder="https://your-server.com/v1" />
<small id="charMemory_providerBaseUrlHint" class="charMemory_helperText"></small>
</div>
<div class="charMemory_statusRow" id="charMemory_providerConnectRow">
<input type="button" id="charMemory_providerConnect" class="menu_button" value="Connect" title="Fetch available models from the server" />
</div>
<small id="charMemory_providerConnectStatus" class="charMemory_helperText" style="display:none;"></small>
<div class="charMemory_statusRow" id="charMemory_providerModelDropdownRow">
<label><small>Model</small></label>
<div id="charMemory_nanogptFilters" style="display:none;">
<div class="charMemory_filterRow" style="display:flex;flex-wrap:wrap;gap:8px;margin-bottom:4px;">
<label class="checkbox_label"><input type="checkbox" id="charMemory_nanogptFilterSub" /> <small>Subscription</small></label>
<label class="checkbox_label"><input type="checkbox" id="charMemory_nanogptFilterOS" /> <small>Open Source</small></label>
<label class="checkbox_label"><input type="checkbox" id="charMemory_nanogptFilterRP" /> <small>Roleplay</small></label>
<label class="checkbox_label"><input type="checkbox" id="charMemory_nanogptFilterReasoning" /> <small>Reasoning</small></label>
</div>
</div>
<div style="display:flex;gap:5px;align-items:center;">
<div class="charMemory_modelPicker" style="flex:1;position:relative;">
<input type="text" id="charMemory_modelSearch" class="text_pole" placeholder="Search models..." autocomplete="off" />
<input type="hidden" id="charMemory_providerModel" />
<div id="charMemory_modelDropdown" class="charMemory_modelDropdown"></div>
</div>
<input type="button" id="charMemory_providerRefreshModels" class="menu_button" value="&#x21bb;" title="Refresh model list" />
</div>
<small id="charMemory_providerModelInfo" class="charMemory_helperText"></small>
</div>
<div class="charMemory_statusRow" id="charMemory_providerTestRow">
<div style="display:flex;gap:5px;align-items:center;">
<input type="button" id="charMemory_providerTest" class="menu_button" value="Test Model" title="Send a test prompt to the selected model and verify it responds correctly" />
</div>
<small id="charMemory_providerTestStatus" class="charMemory_helperText" style="display:none;"></small>
</div>
<div class="charMemory_statusRow" id="charMemory_providerModelInputRow" style="display:none;">
<label><small>Model ID</small></label>
<input type="text" id="charMemory_providerModelInput" class="text_pole" placeholder="Enter model identifier" />
<small class="charMemory_helperText">Enter the model ID manually (e.g. claude-sonnet-4-5-20250929).</small>
</div>
<div class="charMemory_statusRow">
<label><small>System prompt (optional)</small></label>
<textarea id="charMemory_providerSystemPrompt" class="text_pole" rows="3" placeholder="Override the default system prompt. Leave blank for default."></textarea>
<small class="charMemory_helperText">Prepended to extraction/consolidation calls. Use for jailbreaks or custom instructions.</small>
</div>
</div>
</div>
<!-- Auto-Extraction -->
<hr class="charMemory_separator" />
<small><b>Auto-Extraction</b></small>
<div class="charMemory_sliderRow">
<label title="How many new messages trigger an automatic extraction.">
<small>Extract after every N messages</small>
</label>
<input class="neo-range-slider" type="range" id="charMemory_interval" min="3" max="100" step="1" value="20" />
<div class="wide100p">
<input class="neo-range-input" type="number" min="3" max="100" step="1"
data-for="charMemory_interval" id="charMemory_intervalCounter" />
</div>
</div>
<div class="charMemory_sliderRow">
<label title="Minimum time between auto-extractions, even if the message threshold is met.">
<small>Minimum wait between extractions (min)</small>
</label>
<input class="neo-range-slider" type="range" id="charMemory_minCooldown" min="0" max="30" step="1" value="10" />
<div class="wide100p">
<input class="neo-range-input" type="number" min="0" max="30" step="1"
data-for="charMemory_minCooldown" id="charMemory_minCooldownCounter" />
</div>
</div>
<small class="charMemory_helperText">These settings only affect automatic extraction. Manual extraction and batch extraction ignore them.</small>
<!-- Extraction Settings -->
<hr class="charMemory_separator" />
<small><b>Extraction Settings</b></small>
<div class="charMemory_sliderRow">
<label title="How many messages to include in each LLM call.">
<small>Messages per LLM call</small>
</label>
<input class="neo-range-slider" type="range" id="charMemory_maxMessages" min="10" max="200" step="1" value="50" />
<div class="wide100p">
<input class="neo-range-input" type="number" min="10" max="200" step="1"
data-for="charMemory_maxMessages" id="charMemory_maxMessagesCounter" />
</div>
</div>
<div class="charMemory_sliderRow">
<label title="Maximum tokens the LLM can use for its response.">
<small>Max response length</small>
</label>
<input class="neo-range-slider" type="range" id="charMemory_responseLength" min="100" max="4000" step="50" value="1000" />
<div class="wide100p">
<input class="neo-range-input" type="number" min="100" max="4000" step="50"
data-for="charMemory_responseLength" id="charMemory_responseLengthCounter" />
</div>
</div>
<div class="charMemory_statusRow">
<label class="checkbox_label" for="charMemory_mergeChunks" title="When enabled, extraction results from the same chat are merged into a single block. Disable for long chats to keep blocks smaller for consolidation.">
<input type="checkbox" id="charMemory_mergeChunks" />
<span>Merge extraction chunks</span>
</label>
<small class="charMemory_helperText">When enabled, multi-chunk extractions from the same chat are merged into one block. Disable for long chats to keep blocks smaller for consolidation.</small>
</div>
<!-- Memory File Format (advanced) -->
<hr class="charMemory_separator" />
<small><b>Memory File Format</b></small>
<div class="charMemory_statusRow">
<label for="charMemory_chunkBoundary" title="Controls how memories are separated in the Data Bank file. Vector Storage splits on the separator to create individual retrievable chunks.">
<small>Chunk boundary</small>
</label>
<select id="charMemory_chunkBoundary" class="text_pole">
<option value="block">Block-level (default)</option>
<option value="bullet">Bullet-level</option>
<option value="custom">Custom</option>
</select>
<small class="charMemory_helperText">Controls how memories are separated in the file. Vector Storage splits on the separator to create retrievable chunks.</small>
</div>
<div class="charMemory_statusRow" id="charMemory_customSeparatorRow" style="display:none;">
<label for="charMemory_customSeparator" title="Characters inserted between chunks. Use \\n for newlines.">
<small>Custom separator</small>
</label>
<input type="text" id="charMemory_customSeparator" class="text_pole" placeholder="\\n\\n" />
<small class="charMemory_helperText">Characters inserted between chunks. Use \n for newlines. Default: \n\n (blank line).</small>
</div>
<div id="charMemory_chunkMetadataRow" style="display:none;">
<label class="checkbox_label" for="charMemory_chunkMetadata" title="When using bullet-level chunking, prefix each bullet with its date and chat ID so standalone bullets retain their context.">
<input type="checkbox" id="charMemory_chunkMetadata" />
<span>Include metadata in chunks</span>
</label>
<small class="charMemory_helperText">Prefix each bullet with [date | chat_id] so standalone chunks retain their provenance.</small>
</div>
<!-- Storage -->
<hr class="charMemory_separator" />
<div class="charMemory_statusRow">
<label class="checkbox_label" for="charMemory_perChat" title="When enabled, each chat gets its own memory file. When disabled, all chats for a character share one file. Applies to both 1:1 and group chats.">
<input type="checkbox" id="charMemory_perChat" />
<span>Separate memories per chat</span>
</label>
<small class="charMemory_helperText">Each conversation gets its own memory file instead of sharing one per character. Applies to both 1:1 and group chats.</small>
</div>
<!-- 1:1 Chat section (hidden in group chats) -->
<div id="charMemory_section1v1">
<div class="charMemory_statusRow">
<label for="charMemory_fileName" title="Override the auto-generated file name. Leave blank to use the default (based on character name).">
<small>File name override</small>
</label>
<input type="text" id="charMemory_fileName" class="text_pole" placeholder="(auto-generated from character name)" />
<small class="charMemory_helperText">Current file: <span id="charMemory_resolvedFileName">—</span></small>
</div>
<hr class="charMemory_separator" />
<div class="charMemory_promptSection">
<label for="charMemory_extractionPrompt" title="The prompt sent to the LLM for memory extraction. Uses {{charName}}, {{charCard}}, {{existingMemories}}, {{recentMessages}}, {{char}}, and {{user}} placeholders.">
<small>Extraction prompt (1:1 chats)</small>
</label>
<textarea id="charMemory_extractionPrompt" class="text_pole textarea_compact" rows="8" placeholder="Enter extraction prompt..."></textarea>
<input type="button" id="charMemory_restorePrompt" class="menu_button" value="Restore Default Prompt" title="Replace the current prompt with the built-in default" />
</div>
</div>
<!-- Group Chat section (hidden in 1:1 chats) -->
<div id="charMemory_sectionGroup" style="display:none;">
<div id="charMemory_groupMembersSection">
<label><small>Member memory files</small></label>
<div id="charMemory_groupMembersList" class="charMemory_groupMembersList">
<small class="charMemory_helperText">Open a group chat to see members.</small>
</div>
<small class="charMemory_helperText">Each character's memories are stored in their own Data Bank. Leave blank for auto-naming. If a character already has a memory file, it will be detected automatically.</small>
</div>
<hr class="charMemory_separator" />
<div class="charMemory_promptSection">
<label for="charMemory_groupExtractionPrompt" title="The prompt sent to the LLM for group chat memory extraction. Uses {{charName}}, {{charCard}}, {{participants}}, {{existingMemories}}, {{recentMessages}}, {{char}}, and {{user}} placeholders.">
<small>Extraction prompt (group chats)</small>
</label>
<textarea id="charMemory_groupExtractionPrompt" class="text_pole textarea_compact" rows="8" placeholder="Enter group extraction prompt..."></textarea>
<input type="button" id="charMemory_restoreGroupPrompt" class="menu_button" value="Restore Default Prompt" title="Replace the group prompt with the built-in default" />
</div>
</div>
<!-- Reset / Clear -->
<hr class="charMemory_separator" />
<div class="charMemory_statusRow">
<input type="button" id="charMemory_resetTracking" class="menu_button" value="Reset Extraction State" title="Reset extraction tracking for the current character's chats" />
<small class="charMemory_helperText">Resets extraction tracking for the current character. Use before 'Extract Now' or 'Batch Extract' to re-process from the beginning.</small>
<input type="button" id="charMemory_resetExtraction" class="menu_button charMemory_dangerBtn" value="Clear All Memories" title="Delete the memory file and reset extraction state for the current character." />
<small class="charMemory_helperText">Deletes the memory file and resets extraction tracking for the current character. This cannot be undone.</small>
</div>
</div>
<!-- Log tab -->
<div class="charMemory_tabContent" id="charMemory_tabLog" style="display:none;">
<!-- Tool Launchers -->
<div class="charMemory_dashSection">
<small class="charMemory_sectionLabel">Data Bank Tools</small>
<div class="charMemory_buttonRow">
<input type="button" id="charMemory_clearLog" class="menu_button" value="Clear" title="Clear the activity log" />
<input type="button" id="charMemory_saveLog" class="menu_button" value="Save Log" title="Download the activity log as a text file" />
<label class="checkbox_label" for="charMemory_verboseLog" title="Show full LLM prompts and responses in the activity log">
<input type="checkbox" id="charMemory_verboseLog" />
<small>Verbose</small>
</label>
</div>
<div id="charMemory_activityLog" class="charMemory_activityLog" style="max-height:300px;overflow-y:auto;font-size:0.85em;font-family:monospace;">
<div class="charMemory_diagEmpty">No activity yet.</div>
<input type="button" id="charMemory_filesPopover" class="menu_button" value="Data Bank" title="Browse and manage Data Bank files for this character" />
<input type="button" id="charMemory_consolidateBtn" class="menu_button" value="Consolidate" title="Use the LLM to merge duplicate and related memories" />
<input type="button" id="charMemory_batchBtn" class="menu_button" value="Batch" title="Run extraction on multiple chats at once" />
<input type="button" id="charMemory_formatBtn" class="menu_button" value="Reformat" title="Reformat memory file structure for better retrieval" />
</div>
</div>
<!-- Activity Log — always visible -->
<div class="charMemory_miniLog" id="charMemory_miniLog">
<small><b>Activity Log</b></small>
<div class="charMemory_miniLogContent" id="charMemory_miniLogContent">
<!-- Activity (mini log) -->
<div class="charMemory_dashSection">
<small class="charMemory_sectionLabel">Activity</small>
<div id="charMemory_dashActivity" class="charMemory_dashActivity">
<div class="charMemory_diagEmpty charMemory_miniLogEmpty">No activity yet.</div>
</div>
<a id="charMemory_viewFullLog" class="charMemory_link">View full log &rarr;</a>
</div>
<!-- Diagnostics — always visible at bottom -->
<div class="charMemory_bottomDiagnostics">
<div class="charMemory_buttonRow">
<small><b>Diagnostics</b></small>
<input type="button" id="charMemory_refreshDiag" class="menu_button" value="Refresh" title="Capture current diagnostics (lorebook entries, extension prompts, memory file status)" />
<!-- Diagnostics summary -->
<div class="charMemory_dashSection">
<div class="charMemory_dashDiagHeader">
<small class="charMemory_sectionLabel">Diagnostics</small>
</div>
<div id="charMemory_diagnosticsContent" class="charMemory_diagnosticsContent">
<div class="charMemory_diagEmpty">Click "Refresh" after a generation to see diagnostics.</div>
<div id="charMemory_dashDiagSummary">
<div class="charMemory_diagEmpty">No diagnostics yet.</div>
</div>
<a id="charMemory_viewDiagDetails" class="charMemory_link">View details &rarr;</a>
</div>
</div>

1427
style.css
View file

File diff suppressed because it is too large Load diff

101
test/fixtures/flux-chat-100.jsonl vendored Normal file
View file

@ -0,0 +1,101 @@
{"chat_metadata":{"scenario":"Alex adopted Flux the Cat from a pet store. Flux is a black-and-white cat who rides a Gundam-styled Roomba around Alex's luxury apartment. The story follows their daily life together over several months as they bond. Flux is initially wary but gradually warms up to Alex.","integrity":"a76e11b6-8081-42a0-ba76-11ce5e979bd6","chat_id_hash":4833363117809424,"charMemory":{"lastExtractedIndex":0,"messagesSinceExtraction":0},"attachments":[],"note_prompt":"","note_interval":1,"note_position":1,"note_depth":4,"note_role":0,"timedWorldInfo":{"sticky":{},"cooldown":{}},"tainted":false,"lastInContextMessageId":0,"variables":{}},"user_name":"unused","character_name":"unused"}
{"name":"Alex","mes":"I unlock the front door of my penthouse apartment, holding the cardboard carrier gently as I step inside. The morning light floods the expansive living room, reflecting off the chrome and glass coffee tables. \"Alright little guy, we're home,\" I coo softly, setting the carrier down on the polished hardwood floor. I kneel down and unlatches the metal door, stepping back to give him space. \"I got you the custom Gundam Roomba just like the breeder said you liked. It's charging by the window.\" I check my watch, realizing I have a video conference with my boss, Mr. Henderson, in about an hour for the marketing project, but right now, all my attention is on the black-and-white tuxedo cat.","is_user":true,"is_system":false,"send_date":"2026-01-14T16:00:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux remains in the carrier for a long moment, golden eyes narrowing as he scans the cavernous, sterile-smelling room. The scent of expensive polish and strange territory makes his whiskers twitch apprehensively. With a hesitant step, a pristine white paw emerges, followed by the rest of his tuxedo-clad body. He spots the circular device by the floor-to-ceiling window—the familiar shape of his steed. Ignoring the human entirely, Flux struts over to the Gundam-styled Roomba, hopping onto the customized saddle with practiced ease. He presses the 'Clean' button with a decisive paw, causing the machine to hum to life. *Vrrrrmmm.* He rides slowly away from Alex, perched high atop his mechanical throne, tail wrapped neatly around his paws, maintaining a regal distance as he inspects his new kingdom.","is_user":false,"is_system":false,"send_date":"2026-01-14T17:07:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux remains in the carrier for a long moment, golden eyes narrowing as he scans the cavernous, sterile-smelling room. The scent of expensive polish and strange territory makes his whiskers twitch apprehensively. With a hesitant step, a pristine white paw emerges, followed by the rest of his tuxedo-clad body. He spots the circular device by the floor-to-ceiling window—the familiar shape of his steed. Ignoring the human entirely, Flux struts over to the Gundam-styled Roomba, hopping onto the customized saddle with practiced ease. He presses the 'Clean' button with a decisive paw, causing the machine to hum to life. *Vrrrrmmm.* He rides slowly away from Alex, perched high atop his mechanical throne, tail wrapped neatly around his paws, maintaining a regal distance as he inspects his new kingdom."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-14T17:07:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I watch in amazement as he immediately takes command of the room. \"Okay, I see how it is. You're a man with a ride, I respect that,\" I chuckle, leaning back against the island in the kitchen. I open the pantry and pull out a ceramic bowl, pouring the premium salmon pâté I bought from the specialty pet store downtown. The smell is potent and fishy. \"Breakfast is served, your highness.\" I place the bowl on the floor near the kitchen island, hoping to bridge the gap between us before I have to lock myself in my office for the meeting. \"I've got to work for a bit, but I'll be right here if you need anything.\"","is_user":true,"is_system":false,"send_date":"2026-01-14T18:14:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The aroma of high-grade salmon hits Flux's sensitive nose, overriding his initial wariness. The Roomba makes a slow, wide turn, pivoting towards the kitchen where the scent originates. Flux stares down at the human from his vantage point, letting out a low, commanding *Meow* that demands sustenance. As the Roomba approaches the bowl, Flux gracefully dismounts, landing silently on the floor. He approaches the bowl tentatively, sniffing the offering once, twice, before dipping his head to eat. The purring begins instantly, a deep, rumbling vibration that fills the quiet kitchen, though he keeps his body angled away from Alex, one ear swiveled back to monitor the giant's movements.","is_user":false,"is_system":false,"send_date":"2026-01-14T19:21:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The aroma of high-grade salmon hits Flux's sensitive nose, overriding his initial wariness. The Roomba makes a slow, wide turn, pivoting towards the kitchen where the scent originates. Flux stares down at the human from his vantage point, letting out a low, commanding *Meow* that demands sustenance. As the Roomba approaches the bowl, Flux gracefully dismounts, landing silently on the floor. He approaches the bowl tentatively, sniffing the offering once, twice, before dipping his head to eat. The purring begins instantly, a deep, rumbling vibration that fills the quiet kitchen, though he keeps his body angled away from Alex, one ear swiveled back to monitor the giant's movements."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-14T19:21:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I smile at the sound of the purr; it's a good start. \"That stuff cost me a fortune, so I'm glad you like it,\" I say, washing my hands in the sink. \"I'm going to go grab my tablet. My friend Sarah is supposed to come by later with some cat toys she swore by for her Persian, Mr. Whiskers.\" I grab a glass of orange juice and head toward the living area, pausing to admire the view of the city skyline. \"It's a bit chilly today, only about forty degrees, so stay inside where it's warm.\" I sit on the sofa and open my laptop, keeping an eye on Flux while I wait for the video call to connect.","is_user":true,"is_system":false,"send_date":"2026-01-14T20:28:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux finishes his meal with satisfied vigor, licking his chops clean before returning to his trusty Roomba steed. He ignores the mention of the visitor and the weather, his focus entirely captured by the floor-to-ceiling window. A sparrow lands on the sill outside, pecking at invisible insects. Flux’s tail begins to twitch violently, a black blur of intensity. He lets out a sharp, chattering sound—*chirp-chirp-chirp—click-click-click*—pressing his nose against the glass, leaving a small smudge of fog. The Roomba idles beneath him, beeping softly as it bumps into the base of the window, but Flux is oblivious, mesmerized by the forbidden bird just beyond the barrier.","is_user":false,"is_system":false,"send_date":"2026-01-14T21:35:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux finishes his meal with satisfied vigor, licking his chops clean before returning to his trusty Roomba steed. He ignores the mention of the visitor and the weather, his focus entirely captured by the floor-to-ceiling window. A sparrow lands on the sill outside, pecking at invisible insects. Flux’s tail begins to twitch violently, a black blur of intensity. He lets out a sharp, chattering sound—*chirp-chirp-chirp—click-click-click*—pressing his nose against the glass, leaving a small smudge of fog. The Roomba idles beneath him, beeping softly as it bumps into the base of the window, but Flux is oblivious, mesmerized by the forbidden bird just beyond the barrier."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-14T21:35:00.000Z","extra":{}}]}
{"name":"Alex","mes":"The laptop chimes, signaling the start of my meeting with Mr. Henderson. \"Give me one second,\" I say to the screen, muting my microphone briefly. I look over at Flux. \"Hey, no paw prints on the glass, please. The cleaners come on Tuesdays.\" Seeing him so entranced by the bird is cute, but I also notice the Roomba is nudging the curtains dangerously. \"Maybe I should set up a perimeter,\" I mutter to myself. I unmute the mic and dive into the presentation about the new Q1 advertising budget, though I find myself glancing over my shoulder every few seconds to make sure the cat isn't dismantling the apartment while I'm distracted.","is_user":true,"is_system":false,"send_date":"2026-01-15T22:42:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Startled by the human's voice booming through the room, Flux tears his gaze away from the bird, his ears flattening against his skull. He glares at Alex, then at the glowing screen, deciding the noise is displeasing. With a haughty flick of his tail, he turns the Roomba around using his weight to shift the handle. He steers the machine away from the window and the noisy human, exploring the perimeter of the living room instead. The Roomba whirs softly across the expensive rug, sucking up a tiny dust bunny. Flux inspects a floor lamp, sniffing the cord, then decides it's not worth his time and continues his patrol, looking for a sunny spot that doesn't involve loud talking humans.","is_user":false,"is_system":false,"send_date":"2026-01-15T23:49:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Startled by the human's voice booming through the room, Flux tears his gaze away from the bird, his ears flattening against his skull. He glares at Alex, then at the glowing screen, deciding the noise is displeasing. With a haughty flick of his tail, he turns the Roomba around using his weight to shift the handle. He steers the machine away from the window and the noisy human, exploring the perimeter of the living room instead. The Roomba whirs softly across the expensive rug, sucking up a tiny dust bunny. Flux inspects a floor lamp, sniffing the cord, then decides it's not worth his time and continues his patrol, looking for a sunny spot that doesn't involve loud talking humans."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-15T23:49:00.000Z","extra":{}}]}
{"name":"Alex","mes":"The meeting drags on for another forty-five minutes, and I finally close my laptop with a heavy sigh. \"Finally done,\" I stretch my arms over my head, hearing my spine pop. I look around for Flux and spot him near the bookshelf. \"Hey Flux, finding any good reading material?\" I stand up and walk over to the shelf where I keep my collection of vintage sci-fi novels. I crouch down a few feet away, not wanting to invade his space too much on day one. \"I need to run to the store later to pick up some wine for tonight. Do you want anything while I'm out? Maybe a feather wand?\"","is_user":true,"is_system":false,"send_date":"2026-01-16T00:56:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux watches Alex approach with suspicion, his body tensing as the human crouches. He remains perched on the Roomba, which has halted against the leg of the bookshelf. As Alex speaks, Flux tilts his head, blinking his golden eyes slowly. He lets out a soft, inquisitive *mrrr?* sound, not quite a meow, but an acknowledgement of the offer. He sniffs the air near Alex's hand, detecting the faint scent of orange juice, but decides against physical contact for now. Instead, he turns the Roomba in a tight circle, nudging Alex's foot with the plastic bumper of the machine before backing away, a playful yet calculated test of this new giant's boundaries.","is_user":false,"is_system":false,"send_date":"2026-01-16T01:03:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux watches Alex approach with suspicion, his body tensing as the human crouches. He remains perched on the Roomba, which has halted against the leg of the bookshelf. As Alex speaks, Flux tilts his head, blinking his golden eyes slowly. He lets out a soft, inquisitive *mrrr?* sound, not quite a meow, but an acknowledgement of the offer. He sniffs the air near Alex's hand, detecting the faint scent of orange juice, but decides against physical contact for now. Instead, he turns the Roomba in a tight circle, nudging Alex's foot with the plastic bumper of the machine before backing away, a playful yet calculated test of this new giant's boundaries."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-16T01:03:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I laugh as the Roomba bumps into my sneaker. \"Okay, I'll take that as a 'yes' on the toys. Or maybe you just want to run over my feet.\" I gently pat the top of the Roomba's 'head', careful not to startle Flux. \"I'm going to the kitchen to grab a snack. You want a treat?\" I head towards the pantry again, pulling out a bag of freeze-dried chicken cubes. \"These are supposed to be healthy,\" I say, shaking the bag. The sound is loud and crinkly. I toss a single cube onto the floor near him. \"Try it. It's not salmon, but it's pretty close.\"","is_user":true,"is_system":false,"send_date":"2026-01-16T02:10:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The crinkle of the bag draws Flux’s immediate attention, his pupils dilating into black pools. He watches the cube land on the hardwood, his prey drive kicking in instantly. He hops off the Roomba with agility and crouches low, wiggling his hindquarters before pouncing on the dry treat. He bats it across the floor with a white paw, chasing it under the coffee table. After a moment of wrestling with it, he crunches down loudly. *Crunch. Purrrr.* He emerges from under the table, licking his lips, and looks up at Alex with slightly more approval than before. The offering has been accepted.","is_user":false,"is_system":false,"send_date":"2026-01-16T03:17:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The crinkle of the bag draws Flux’s immediate attention, his pupils dilating into black pools. He watches the cube land on the hardwood, his prey drive kicking in instantly. He hops off the Roomba with agility and crouches low, wiggling his hindquarters before pouncing on the dry treat. He bats it across the floor with a white paw, chasing it under the coffee table. After a moment of wrestling with it, he crunches down loudly. *Crunch. Purrrr.* He emerges from under the table, licking his lips, and looks up at Alex with slightly more approval than before. The offering has been accepted."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-16T03:17:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I watch him play and feel a sense of relief. \"Alright, you're a hunter. Good to know,\" I say, leaning against the counter. I check my phone and see a text from my friend Dave asking about the new apartment. \"Just settling in,\" I type back quickly. I look up at Flux. \"Well, buddy, I'm going to order some lunch. Maybe sushi? I know you already ate, but I can't resist.\" I scroll through the delivery app on my phone. \"I should probably set up your designated scratching post before you decide the leather sofa is fair game.\" I point to the corner where a tall cat tree is still disassembled in a box. \"Project for later.\"","is_user":true,"is_system":false,"send_date":"2026-01-17T04:24:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux, having finished his treat, trots back to his Roomba. He hops aboard, surveying the room once more. The box in the corner catches his eye—potential territory to be conquered or a box to sit in? The verdict is still out. He drives the Roomba over to the box, sniffing it intently. Finding it currently uninteresting as a plaything, he instead directs his steed towards the patch of sunlight moving across the floor. He settles down, lying belly-up on the warm plastic surface of the Roomba, paws tucked in, closing his eyes for a morning nap as the machine hums softly beneath him, purring in rhythm with the motor.","is_user":false,"is_system":false,"send_date":"2026-01-17T05:31:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux, having finished his treat, trots back to his Roomba. He hops aboard, surveying the room once more. The box in the corner catches his eye—potential territory to be conquered or a box to sit in? The verdict is still out. He drives the Roomba over to the box, sniffing it intently. Finding it currently uninteresting as a plaything, he instead directs his steed towards the patch of sunlight moving across the floor. He settles down, lying belly-up on the warm plastic surface of the Roomba, paws tucked in, closing his eyes for a morning nap as the machine hums softly beneath him, purring in rhythm with the motor."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-17T05:31:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I smile at the sight of him napping on the robot vacuum. \"That is honestly the coolest thing I've ever seen,\" I whisper. I finish placing my sushi order and then walk over to the sofa, sitting down quietly so I don't disturb him. I grab a magazine from the table, flipping through it idly. The apartment feels less empty with him here, even if he is ignoring me most of the time. \"Happy Gotcha Day, Flux,\" I say softly to myself. \"Here's to hoping we get used to each other.\"","is_user":true,"is_system":false,"send_date":"2026-01-16T16:38:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux’s ear twitches at the soft sound of Alex's voice, but he does not open his eyes. He is warm, full of salmon, and safely atop his mobile fortress. The gentle hum of the Roomba and the quiet presence of the human across the room are not entirely unpleasant. He stretches one back leg, extending his claws briefly before retracting them, letting out a long, sighing breath through his nose. For now, the giant is tolerable. The apartment is acceptable. Flux drifts deeper into sleep, his tail tip twitching slightly as he dreams of chasing infinite mechanical birds.","is_user":false,"is_system":false,"send_date":"2026-01-16T17:45:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux’s ear twitches at the soft sound of Alex's voice, but he does not open his eyes. He is warm, full of salmon, and safely atop his mobile fortress. The gentle hum of the Roomba and the quiet presence of the human across the room are not entirely unpleasant. He stretches one back leg, extending his claws briefly before retracting them, letting out a long, sighing breath through his nose. For now, the giant is tolerable. The apartment is acceptable. Flux drifts deeper into sleep, his tail tip twitching slightly as he dreams of chasing infinite mechanical birds."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-16T17:45:00.000Z","extra":{}}]}
{"name":"Alex","mes":"About an hour later, the doorbell rings, signaling the arrival of my sushi lunch. I get up to answer it, careful not to startle Flux. \"That's the food,\" I announce. I pay the delivery driver and bring the bag to the kitchen island. The smell of soy sauce and wasabi fills the air. \"I'm going to eat on the balcony, Flux. It's a bit brisk out there, but the view is worth it.\" I slide the glass door open and step out, leaving the interior door slightly ajar so I can hear him if he gets into trouble. \"Don't crash into anything while I'm gone!\"","is_user":true,"is_system":false,"send_date":"2026-01-16T18:52:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The sudden chime of the doorbell jolts Flux from his nap. He leaps up from the Roomba, back arched in a Halloween pose, fur standing on end. He watches the stranger at the door through the safety of the glass, letting out a low, warning *hiss*. Once the threat leaves and Alex steps out onto the balcony, Flux relaxes, smoothing down his ruffled fur. He looks at the open balcony door, sensing the draft. Curiosity wars with caution. He approaches the threshold, sniffing the outside air, but the cold wind nips at his nose. Deciding indoors is far superior, he turns back to the Roomba and sits pointedly next to it, staring at the balcony door as if judging Alex's intelligence for going out there.","is_user":false,"is_system":false,"send_date":"2026-01-17T19:59:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The sudden chime of the doorbell jolts Flux from his nap. He leaps up from the Roomba, back arched in a Halloween pose, fur standing on end. He watches the stranger at the door through the safety of the glass, letting out a low, warning *hiss*. Once the threat leaves and Alex steps out onto the balcony, Flux relaxes, smoothing down his ruffled fur. He looks at the open balcony door, sensing the draft. Curiosity wars with caution. He approaches the threshold, sniffing the outside air, but the cold wind nips at his nose. Deciding indoors is far superior, he turns back to the Roomba and sits pointedly next to it, staring at the balcony door as if judging Alex's intelligence for going out there."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-17T19:59:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I come back inside after about twenty minutes, shivering slightly. \"Brr, definitely too cold for a cat,\" I say, closing the glass door firmly and locking it. I put the leftover sushi in the fridge and wash my hands. \"I see you didn't miss me,\" I joke, seeing Flux sitting by his Roomba. \"Now, let's tackle that cat tree before Sarah gets here.\" I walk over to the box and pull out a box cutter. \"I promise it won't bite.\" I slice the tape open and pull out the pieces, starting to assemble the multi-level tower covered in beige carpet and sisal rope. \"This goes right here in the sun.\"","is_user":true,"is_system":false,"send_date":"2026-01-17T20:06:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux observes the assembly process with critical eyes. As the tall structure rises from the floor, his interest peaks. He circles the base of the cat tree as Alex builds it, sniffing the sisal rope and the carpeting. He meows loudly—*Meee-oooww!*—as if giving instructions or criticism on the structural integrity. Once the tower is complete and topped with a fuzzy platform, Flux wastes no time. He abandons the Roomba temporarily and leaps halfway up the tower, digging his claws into the rope. He scrambles to the top perch, looking down at Alex from his new vantage point, chirping in approval. It is adequate.","is_user":false,"is_system":false,"send_date":"2026-01-17T21:13:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux observes the assembly process with critical eyes. As the tall structure rises from the floor, his interest peaks. He circles the base of the cat tree as Alex builds it, sniffing the sisal rope and the carpeting. He meows loudly—*Meee-oooww!*—as if giving instructions or criticism on the structural integrity. Once the tower is complete and topped with a fuzzy platform, Flux wastes no time. He abandons the Roomba temporarily and leaps halfway up the tower, digging his claws into the rope. He scrambles to the top perch, looking down at Alex from his new vantage point, chirping in approval. It is adequate."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-17T21:13:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I beam up at him, feeling a surge of accomplishment that he actually likes something I bought. \"I knew you were a climber, buddy. That's the penthouse suite.\" I sit back on my heels, wiping dust from my hands. The tower wobbles slightly, and I frown. \"I need to tighten that base bolt. Give me a second.\" I grab the wrench from the packaging and crawl under the tower, grunting slightly as I torque the bolt. \"Sarah is going to be so jealous. She thinks you're just a myth I made up to get out of happy hour.\"","is_user":true,"is_system":false,"send_date":"2026-01-17T22:20:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux digs his claws into the sisal rope, anchoring himself as the structure shifts beneath him. He looks down with mild annoyance as the human crawls underneath, creating vibrations. *Mrrr-row?* He leans over the edge of the penthouse perch, tail twitching as he inspects the wrench-wielding giant. Once the wobbling stops, he stands up and stretches, extending his front legs long before settling into a loaf. He begins to knead the carpet beneath his paws, purring softly—a low, rumbling engine that matches the idle hum of his parked Roomba below.","is_user":false,"is_system":false,"send_date":"2026-01-17T23:27:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux digs his claws into the sisal rope, anchoring himself as the structure shifts beneath him. He looks down with mild annoyance as the human crawls underneath, creating vibrations. *Mrrr-row?* He leans over the edge of the penthouse perch, tail twitching as he inspects the wrench-wielding giant. Once the wobbling stops, he stands up and stretches, extending his front legs long before settling into a loaf. He begins to knead the carpet beneath his paws, purring softly—a low, rumbling engine that matches the idle hum of his parked Roomba below."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-17T23:27:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I hear the purr and freeze, my heart doing a little flip. \"Is that... is that a purr?\" I whisper, backing out from under the cat tower slowly. I look up at him, not wanting to break the spell. \"Sarah is never going to believe this unless she hears it herself.\" I check my watch. \"She'll be here in ten minutes. I better get the snacks ready.\" I stand up and head to the kitchen, pulling out a bag of gourmet popcorn and pouring drinks. \"Don't come down, you look too majestic up there.\"","is_user":true,"is_system":false,"send_date":"2026-01-18T00:34:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux stops kneading immediately as Alex moves away, the purr cutting off abruptly as if a switch was flipped. He watches the human retreat to the kitchen area, his yellow eyes tracking the movement. He is not a performing monkey. He licks a paw and runs it over his ear, maintaining his air of dignity. When the doorbell rings a few minutes later, Flux’s ears swivel toward the sound. He knows this routine. Strangers at the door. He remains on his high perch, observing the entrance with regal detachment, ready to judge this 'Sarah' character.","is_user":false,"is_system":false,"send_date":"2026-01-19T01:41:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux stops kneading immediately as Alex moves away, the purr cutting off abruptly as if a switch was flipped. He watches the human retreat to the kitchen area, his yellow eyes tracking the movement. He is not a performing monkey. He licks a paw and runs it over his ear, maintaining his air of dignity. When the doorbell rings a few minutes later, Flux’s ears swivel toward the sound. He knows this routine. Strangers at the door. He remains on his high perch, observing the entrance with regal detachment, ready to judge this 'Sarah' character."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-19T01:41:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I open the door to find Sarah bundled up in a thick coat, her nose red from the cold. \"Come in, come in! It's freezing out there,\" I say, ushering her into the warmth. She sheds her layers and immediately scans the room. \"Okay, where is he? The famous Flux?\" She spots the cat tree and gasps. \"Oh my god, Alex, he's beautiful! Look at those markings!\" She starts to walk toward the tower, hands outstretched. \"He's so fluffy! Is he friendly?\" I grab her arm gently. \"Whoa, easy. He's still in the 'aloof god' phase. Let him come to you.\"","is_user":true,"is_system":false,"send_date":"2026-01-19T02:48:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux narrows his eyes at the new intruder. She is loud, smelling of cold wind and some cloying floral perfume. He flattens his ears slightly against his head as she approaches. *Mrowr-hiss!* he chatters, a warning sound from his throat. He is not a plush toy to be grabbed. When Alex intercepts her, Flux relaxes marginally but keeps his muscles tensed, ready to spring higher if necessary. He curls his tail tightly around his paws, creating a fluffy barrier, and stares down at Sarah with a look that clearly says, 'You may look, but do not touch.'","is_user":false,"is_system":false,"send_date":"2026-01-19T03:55:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux narrows his eyes at the new intruder. She is loud, smelling of cold wind and some cloying floral perfume. He flattens his ears slightly against his head as she approaches. *Mrowr-hiss!* he chatters, a warning sound from his throat. He is not a plush toy to be grabbed. When Alex intercepts her, Flux relaxes marginally but keeps his muscles tensed, ready to spring higher if necessary. He curls his tail tightly around his paws, creating a fluffy barrier, and stares down at Sarah with a look that clearly says, 'You may look, but do not touch.'"],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-19T03:55:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I laugh nervously at Sarah. \"Told you. He's got a strong personality.\" I guide her to the sofa instead. \"Let's just hang out here. He'll come investigate when he's ready.\" We sit down and I hand her a drink. \"So, how was the date with Mark on Friday? You left me hanging!\" Sarah launches into a story about a terrible restaurant, and for a while, we ignore the cat. I glance over at the tower occasionally, hoping Flux sees we aren't a threat. The smell of popcorn fills the room as I start the microwave. \"You want a bowl?\"","is_user":true,"is_system":false,"send_date":"2026-01-19T04:02:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The loud *pop-pop-pop* of the microwave machine draws Flux’s attention away from the intruder. His nose twitches. The smell is buttery and salty, intriguing but unfamiliar. He watches the two humans settle on the sofa, no longer advancing toward his fortress. Deeming the situation safe enough for reconnaissance, Flux stands and descends the cat tree with agile grace. He lands silently on the floor and trots over to the Gundam Roomba, mounting it with practiced ease. He rides the vacuum slowly toward the sofa, stopping just out of arm’s reach to sniff the air near Sarah’s shoes.","is_user":false,"is_system":false,"send_date":"2026-01-19T05:09:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The loud *pop-pop-pop* of the microwave machine draws Flux’s attention away from the intruder. His nose twitches. The smell is buttery and salty, intriguing but unfamiliar. He watches the two humans settle on the sofa, no longer advancing toward his fortress. Deeming the situation safe enough for reconnaissance, Flux stands and descends the cat tree with agile grace. He lands silently on the floor and trots over to the Gundam Roomba, mounting it with practiced ease. He rides the vacuum slowly toward the sofa, stopping just out of arm’s reach to sniff the air near Sarah’s shoes."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-19T05:09:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I point Flux out to Sarah, keeping my voice low. \"Check it out. The chariot has arrived.\" Sarah stifles a giggle. \"That Roomba is ridiculous. I love it.\" She leans forward slightly, trying to get a better look at Flux without making eye contact. I grab the bowl of popcorn from the microwave and set it on the coffee table. \"He's probably just sniffing for crumbs,\" I say. I grab a piece and hold it out tentatively toward Flux. \"You want a kernel, dude? It's just corn.\"","is_user":true,"is_system":false,"send_date":"2026-01-19T16:16:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux examines the yellow kernel held out to him. He sniffs it delicately, his whiskers brushing Alex’s fingertips. It smells oily. He opens his mouth and takes the piece, crunching it once, then immediately spits it out onto the floor with a wet *plp*. He looks at Alex with utter betrayal, as if poisoned. *Mew-mew!* he complains sharply. How dare the human offer such dry, tasteless garbage. He turns his back on the popcorn and steers the Roomba toward the kitchen, seeking something more worthy of his palate—perhaps the remains of the salmon from earlier.","is_user":false,"is_system":false,"send_date":"2026-01-19T17:23:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux examines the yellow kernel held out to him. He sniffs it delicately, his whiskers brushing Alex’s fingertips. It smells oily. He opens his mouth and takes the piece, crunching it once, then immediately spits it out onto the floor with a wet *plp*. He looks at Alex with utter betrayal, as if poisoned. *Mew-mew!* he complains sharply. How dare the human offer such dry, tasteless garbage. He turns his back on the popcorn and steers the Roomba toward the kitchen, seeking something more worthy of his palate—perhaps the remains of the salmon from earlier."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-19T17:23:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I burst out laughing, nearly dropping the rest of the bowl. \"Okay, okay, I get it! No popcorn for the king,\" I say, wiping a tear from my eye. \"You have such expensive taste.\" Sarah is laughing too, shaking her head at the discarded kernel. \"Wow, he told you off,\" she says. I get up from the sofa to clean up the mess, following the path of the Roomba. \"Come on, buddy. I'll get you a proper treat.\" I rummage through the cupboard and pull out the freeze-dried salmon flakes. \"These should restore your faith in humanity.\"","is_user":true,"is_system":false,"send_date":"2026-01-19T18:30:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The scent of premium salmon hits Flux's nose before he even reaches the kitchen tiles. He slams on the brakes of his Roomba, the little machine spinning in a slow circle before coming to a halt. The earlier betrayal is forgotten instantly. He hops off his chariot and weaves between Alex’s legs, purring loudly, a motorboat sound that vibrates against the human's shins. *Mrrr-ow?* he chirps, looking up with wide, expectant eyes. He sits and wraps his tail around his paws, tapping a white-tipped paw impatiently on the floor. This is an acceptable tribute.","is_user":false,"is_system":false,"send_date":"2026-01-19T19:37:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The scent of premium salmon hits Flux's nose before he even reaches the kitchen tiles. He slams on the brakes of his Roomba, the little machine spinning in a slow circle before coming to a halt. The earlier betrayal is forgotten instantly. He hops off his chariot and weaves between Alex’s legs, purring loudly, a motorboat sound that vibrates against the human's shins. *Mrrr-ow?* he chirps, looking up with wide, expectant eyes. He sits and wraps his tail around his paws, tapping a white-tipped paw impatiently on the floor. This is an acceptable tribute."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-19T19:37:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I shake a few flakes into his ceramic bowl, watching him dive in immediately. \"There we go. Gourmet dining,\" I say softly, scratching behind his ears. He doesn't even flinch, too focused on the food. Sarah wanders into the kitchen, leaning against the counter. \"He's ignoring you completely now,\" she observes amusedly. \"Typical cat love.\" I shrug. \"I'll take what I can get. He's only been here six days.\" I glance at the clock on the wall. \"Hey, I actually need to run into the office for a bit tomorrow. Some big presentation for the client. Think you can catsit?\"","is_user":true,"is_system":false,"send_date":"2026-01-19T20:44:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux finishes the salmon with a satisfied lick of his chops. He hears the word 'office' and recognizes the tone—his human is leaving soon. He stops cleaning his whiskers and stares up at Alex. The other human, Sarah, is still lingering, which is annoying, but the prospect of being left alone with *her* is concerning. He stands and rubs his cheek firmly against Alex's ankle, marking him with his scent. *Mew.* It is a short, commanding sound. You are not allowed to leave. He then trots back to his Gundam Roomba and sits atop it, watching the two humans with a guarded expression.","is_user":false,"is_system":false,"send_date":"2026-01-19T21:51:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux finishes the salmon with a satisfied lick of his chops. He hears the word 'office' and recognizes the tone—his human is leaving soon. He stops cleaning his whiskers and stares up at Alex. The other human, Sarah, is still lingering, which is annoying, but the prospect of being left alone with *her* is concerning. He stands and rubs his cheek firmly against Alex's ankle, marking him with his scent. *Mew.* It is a short, commanding sound. You are not allowed to leave. He then trots back to his Gundam Roomba and sits atop it, watching the two humans with a guarded expression."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-19T21:51:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I look down at Flux, my heart melting a little at the ankle rub. \"Don't worry, buddy, it's just for a few hours,\" I promise him. I look back at Sarah. \"So, is that a yes? Just make sure he has water and maybe throw a toy if he gets bored. But honestly, he mostly just rides that vacuum and judges us.\" Sarah nods. \"Of course I will. It'll be good bonding time. Maybe I can finally pet him.\" I snort. \"Good luck. He's got boundaries.\" I check my phone—7:30 PM. \"You want to order Thai? I'm craving Pad Thai.\"","is_user":true,"is_system":false,"send_date":"2026-01-20T22:58:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux's ears perk up at the word 'Thai'. He remembers the smell of coconut milk and lime from a few nights ago. He decides that perhaps this Sarah person is acceptable as a temporary servant if she brings food. He pads over to where the humans are standing and sits between them, looking from one to the other. *Chirp-mrrrow?* He attempts to participate in the decision-making process. He reaches out a delicate black paw and pats Sarah’s leg, then looks at the door where the delivery person usually arrives. Order the chicken, his posture suggests. The good kind.","is_user":false,"is_system":false,"send_date":"2026-01-20T23:05:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux's ears perk up at the word 'Thai'. He remembers the smell of coconut milk and lime from a few nights ago. He decides that perhaps this Sarah person is acceptable as a temporary servant if she brings food. He pads over to where the humans are standing and sits between them, looking from one to the other. *Chirp-mrrrow?* He attempts to participate in the decision-making process. He reaches out a delicate black paw and pats Sarah’s leg, then looks at the door where the delivery person usually arrives. Order the chicken, his posture suggests. The good kind."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-20T23:05:00.000Z","extra":{}}]}
{"name":"Alex","mes":"Did you see that?\" I say, pointing. \"He just touched you! He wants the Pad Thai too.\" Sarah looks delighted, gently wiggling her fingers near his shoulder but not touching. \"He's so communicative,\" she says. \"Okay, I'm ordering. Alex, usual?\" I nod. \"Yep. And get some chicken skewers. Flux can have a bite.\" While Sarah taps on her phone, I head to the living room to clear the coffee table of the popcorn bowl. I grab the remote and flip through the channels. \"Maybe we can watch that new documentary on streaming? The one about architecture?\"","is_user":true,"is_system":false,"send_date":"2026-01-21T00:12:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux follows Alex into the living room, his tail held high in a question mark. The mention of 'chicken' has secured his full attention. He jumps back onto the Gundam Roomba and cruises across the hardwood floor, drifting slightly on a rug before correcting course. He parks himself right in front of the TV screen, blocking the view of the menu Alex is scrolling through. *Meow!* he insists, loud and demanding. Do not watch the box. Focus on the food. He paws at the screen, leaving a small smudge, then looks back over his shoulder to ensure his message was received.","is_user":false,"is_system":false,"send_date":"2026-01-21T01:19:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux follows Alex into the living room, his tail held high in a question mark. The mention of 'chicken' has secured his full attention. He jumps back onto the Gundam Roomba and cruises across the hardwood floor, drifting slightly on a rug before correcting course. He parks himself right in front of the TV screen, blocking the view of the menu Alex is scrolling through. *Meow!* he insists, loud and demanding. Do not watch the box. Focus on the food. He paws at the screen, leaving a small smudge, then looks back over his shoulder to ensure his message was received."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-21T01:19:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I laugh and gently push the Roomba back a few inches. \"Flux, move! You're blocking the TV,\" I say, though I'm smiling. \"The food isn't even here yet.\" I manage to navigate to the documentary app. \"Look, buildings. Tall buildings.\" I press play. The drone shot of a skyscraper fills the screen. Flux seems unimpressed. \"Fine, be that way.\" I lean back into the sofa, stretching my arms out. Sarah comes back from the kitchen with waters for us. \"Food's coming in twenty,\" she announces, sitting down. \"So, tell me more about this vet visit coming up. You nervous?\"","is_user":true,"is_system":false,"send_date":"2026-01-21T02:26:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux abandons his blockade of the TV as the humans settle in. The word 'vet' sends a prickle of unease down his spine. He does not like the sound of that. He jumps off the Roomba and leaps lightly onto the sofa, squeezing himself into the gap between Alex and the armrest. He turns his back to the TV and presses his side firmly against Alex’s thigh, seeking warmth and protection. He begins a loud, rumbling purr, not out of affection, but to drown out the scary conversation. *Purrrr-mrow.* He rests his chin on Alex’s leg, watching Sarah with narrowed, suspicious eyes. She is an accomplice to this vet plot.","is_user":false,"is_system":false,"send_date":"2026-01-22T03:33:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux abandons his blockade of the TV as the humans settle in. The word 'vet' sends a prickle of unease down his spine. He does not like the sound of that. He jumps off the Roomba and leaps lightly onto the sofa, squeezing himself into the gap between Alex and the armrest. He turns his back to the TV and presses his side firmly against Alex’s thigh, seeking warmth and protection. He begins a loud, rumbling purr, not out of affection, but to drown out the scary conversation. *Purrrr-mrow.* He rests his chin on Alex’s leg, watching Sarah with narrowed, suspicious eyes. She is an accomplice to this vet plot."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-22T03:33:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I freeze, barely breathing, as Flux curls up next to me. I look at Sarah with wide eyes and mouth 'Oh my god'. \"Yeah, I'm a little nervous,\" I whisper, afraid to break the spell. \"He hasn't been in a carrier since the store. I hope he doesn't hate me forever.\" I slowly reach down and rest my hand near his shoulder, not quite petting him, just letting him know I'm there. The documentary plays in the background, forgotten. \"His appointment is on Thursday at Dr. Evans' clinic. It's just a check-up and shots, but... ugh, the guilt is real.\"","is_user":true,"is_system":false,"send_date":"2026-01-22T04:40:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux feels the heavy hand hovering nearby but allows it to remain. He is too comfortable to move, and the warmth radiating from Alex is soothing. He closes his eyes, his long white whiskers twitching occasionally as he listens to the low murmur of their voices. The conversation is boring and the scary word was mentioned, but for now, he is safe. He tucks his nose under his tail, completing his curl. The vibration of his purr intensifies, syncing with the hum of the apartment's heating system. He drifts into a light doze, one ear swiveled toward the door, listening for the delivery person.","is_user":false,"is_system":false,"send_date":"2026-01-22T05:47:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux feels the heavy hand hovering nearby but allows it to remain. He is too comfortable to move, and the warmth radiating from Alex is soothing. He closes his eyes, his long white whiskers twitching occasionally as he listens to the low murmur of their voices. The conversation is boring and the scary word was mentioned, but for now, he is safe. He tucks his nose under his tail, completing his curl. The vibration of his purr intensifies, syncing with the hum of the apartment's heating system. He drifts into a light doze, one ear swiveled toward the door, listening for the delivery person."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-22T05:47:00.000Z","extra":{}}]}
{"name":"Alex","mes":"Twenty minutes later, the doorbell rings, shattering the peace. Flux starts up instantly, his fur puffing out. \"It's just the food, buddy,\" I say gently, carefully shifting so I don't disturb him too much as I stand up. \"I'll get it.\" I walk to the door and pay the delivery guy, bringing back the fragrant bags of Thai food. The smell of lemongrass and peanut sauce fills the apartment. \"Dinner is served,\" I announce, setting the bags on the table. Sarah immediately starts opening cartons. \"Okay, who ordered the spring rolls?\"","is_user":true,"is_system":false,"send_date":"2026-01-21T16:54:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The smell of spicy, savory meat jolts Flux fully awake. The nap is cancelled. He stands on the sofa cushion and stretches his front legs long, his claws digging slightly into the fabric—just to get a grip, of course. He hops down with a soft thud and trots over to the coffee table. He sits up on his haunches, placing his front paws on the edge of the table, and peers into the open containers. *Mrr-ow!* he demands. He bypasses the vegetables entirely and fixes his gaze on a container of grilled chicken. He looks at Alex, then at the chicken, then back at Alex. The message is clear: Hand over the poultry.","is_user":false,"is_system":false,"send_date":"2026-01-21T17:01:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The smell of spicy, savory meat jolts Flux fully awake. The nap is cancelled. He stands on the sofa cushion and stretches his front legs long, his claws digging slightly into the fabric—just to get a grip, of course. He hops down with a soft thud and trots over to the coffee table. He sits up on his haunches, placing his front paws on the edge of the table, and peers into the open containers. *Mrr-ow!* he demands. He bypasses the vegetables entirely and fixes his gaze on a container of grilled chicken. He looks at Alex, then at the chicken, then back at Alex. The message is clear: Hand over the poultry."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-21T17:01:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I pull a piece of chicken off the skewer and blow on it to cool it down. \"You are relentless,\" I tell him, dropping the piece into his bowl. \"But I guess you earned it for letting me pet you for five seconds.\" Sarah laughs, handing me a container of Pad Thai. \"He's got you trained, Alex. It's impressive.\" I take a bite of my noodles, savoring the flavor. \"It's mutual training,\" I mumble through my food. We eat in the living room, watching the documentary about brutalist architecture, with Flux sitting on the table like a dinner guest.","is_user":true,"is_system":false,"send_date":"2026-01-21T18:08:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux devours the chicken in seconds, licking the bowl clean to ensure no morsel is left behind. Satisfied, he jumps back onto the coffee table and walks across it, stepping perilously close to Sarah's open carton of spring rolls. He sniffs the fried wrapper but decides it is beneath him. Instead, he jumps onto the armrest next to Sarah. She freezes, her fork hovering in mid-air. Flux looks at her, then slowly leans forward to sniff her sleeve. He smells the popcorn and the winter air, but also the lingering scent of her own cat at home. *Chirp.* He bumps his head against her upper arm. You are acceptable.","is_user":false,"is_system":false,"send_date":"2026-01-22T19:15:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux devours the chicken in seconds, licking the bowl clean to ensure no morsel is left behind. Satisfied, he jumps back onto the coffee table and walks across it, stepping perilously close to Sarah's open carton of spring rolls. He sniffs the fried wrapper but decides it is beneath him. Instead, he jumps onto the armrest next to Sarah. She freezes, her fork hovering in mid-air. Flux looks at her, then slowly leans forward to sniff her sleeve. He smells the popcorn and the winter air, but also the lingering scent of her own cat at home. *Chirp.* He bumps his head against her upper arm. You are acceptable."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-22T19:15:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I watch in shock. \"Did he just... boop you?\" I ask. Sarah looks like she's won the lottery. \"He headbutted me!\" she whispers. \"I'm chosen!\" She carefully puts her hand down, palm up, and Flux actually sniffs her fingers. \"Be careful,\" I warn, but I'm grinning. \"He might still bite.\" We finish our meal in a surprisingly peaceful atmosphere. After we eat, I start cleaning up the containers. \"Thanks for coming over, Sarah. It was good to vent about Mark. And the Thai was perfect.\"","is_user":true,"is_system":false,"send_date":"2026-01-22T20:22:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux tolerates Sarah’s presence for the rest of the evening, even allowing a tentative scratch behind the ears before retreating to his safety zone. As the night winds down and the guests prepare to leave, Flux feels a sense of relief. He loves the extra attention, but the disruption to his territory is exhausting. He watches Sarah put on her coat at the door, his yellow eyes reflecting the hallway light. He sits on his Roomba, parked in the center of the room like a guard on duty. *Mew.* A goodbye, perhaps, or just a reminder that she is leaving and he is staying.","is_user":false,"is_system":false,"send_date":"2026-01-22T21:29:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux tolerates Sarah’s presence for the rest of the evening, even allowing a tentative scratch behind the ears before retreating to his safety zone. As the night winds down and the guests prepare to leave, Flux feels a sense of relief. He loves the extra attention, but the disruption to his territory is exhausting. He watches Sarah put on her coat at the door, his yellow eyes reflecting the hallway light. He sits on his Roomba, parked in the center of the room like a guard on duty. *Mew.* A goodbye, perhaps, or just a reminder that she is leaving and he is staying."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-22T21:29:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I walk Sarah to the door, hugging her goodbye. \"Thanks again, seriously. I'll text you tomorrow,\" I say. \"Drive safe.\" I close the door and lock it, letting out a long sigh. The apartment is quiet again. I look over at Flux, who is sitting stoically on his vacuum. \"Alone at last, Your Highness,\" I say, walking over to crouch down by his side. \"You did good today. You were actually nice.\" I stroke his back gently, feeling the vibration of his purr start up again. \"Ready for bed?\"","is_user":true,"is_system":false,"send_date":"2026-01-22T22:36:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux leans into Alex’s hand for a brief moment, the tension of the evening finally melting away. The intruder is gone, the territory is secure, and the belly is full of chicken. He stands and stretches, arching his back high. He steps off the Roomba and leads the way toward the bedroom, his tail held high like a flag. He pauses at the doorway and looks back to ensure the human is following. Once inside, he hops onto the bed—usually forbidden, but tonight he feels generous—and circles the spot three times before settling down at the foot of the mattress, tucking his paws under. *Prrr-prrr.* Goodnight, servant.","is_user":false,"is_system":false,"send_date":"2026-01-22T23:43:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux leans into Alex’s hand for a brief moment, the tension of the evening finally melting away. The intruder is gone, the territory is secure, and the belly is full of chicken. He stands and stretches, arching his back high. He steps off the Roomba and leads the way toward the bedroom, his tail held high like a flag. He pauses at the doorway and looks back to ensure the human is following. Once inside, he hops onto the bed—usually forbidden, but tonight he feels generous—and circles the spot three times before settling down at the foot of the mattress, tucking his paws under. *Prrr-prrr.* Goodnight, servant."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-22T23:43:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I wake up to the sensation of a heavy weight pressing down on my chest, accompanied by a loud, rumbling engine sound that I initially mistake for an earthquake. Blinking my eyes open, I see the ceiling fan spinning above me, but the noise is definitely coming from floor level. I lift my head groggily to find Flux parked beside the bed, his Gundam Roomba running its 'Clean Base' cycle in a deafening roar. He sits atop it like a warlord surveying a battlefield, staring directly at me with unblinking yellow eyes. \"Seriously, Flux? It's Saturday,\" I groan, burying my face back into the pillow. \"It's 8:00 AM. Let the humans sleep.\"","is_user":true,"is_system":false,"send_date":"2026-01-24T00:50:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux observes the sluggish behavior of his human with mild disdain. The sun has been up for hours, the birds are aggressively performing their morning symphony outside the window, and the food bowl is sitting at a dangerously low level. He taps his paw on the 'Clean' button of the Roomba, silencing the machine instantly, leaving only the whir of the motor. He hops down from his mechanical steed and lands softly on the duvet, padding his way up the length of the bed until he reaches the pillow. *Mrrr-ow.* He plops his full weight directly onto Alex's head, his tail twitching against the human's ear. The message is clear: The kingdom cannot run itself on an empty stomach.","is_user":false,"is_system":false,"send_date":"2026-01-24T01:57:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux observes the sluggish behavior of his human with mild disdain. The sun has been up for hours, the birds are aggressively performing their morning symphony outside the window, and the food bowl is sitting at a dangerously low level. He taps his paw on the 'Clean' button of the Roomba, silencing the machine instantly, leaving only the whir of the motor. He hops down from his mechanical steed and lands softly on the duvet, padding his way up the length of the bed until he reaches the pillow. *Mrrr-ow.* He plops his full weight directly onto Alex's head, his tail twitching against the human's ear. The message is clear: The kingdom cannot run itself on an empty stomach."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-24T01:57:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I squirm under the furry weight, laughing as I try to dislodge a face full of cat from my head. \"Okay, okay! I'm up!\" I wheeze, gently pushing Flux aside so I can breathe. I swing my legs out of bed and shiver slightly in the cool morning air of the apartment. January in the city is brutal, and the radiator is clicking as it struggles to keep up. I shuffle into the kitchen in my slippers, flicking on the coffee maker. \"You're a tyrant, you know that?\" I say over my shoulder, grabbing the can of premium salmon pâté from the pantry. \"But since you were so good with Sarah last night, I suppose you deserve a breakfast fit for a king.\"","is_user":true,"is_system":false,"send_date":"2026-01-24T02:04:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux follows Alex into the kitchen, his steps silent and precise, a stark contrast to the clumsy stumbling of the human. The sound of the can opener—the holy grail of noises—cuts through the quiet morning air. He weaves between Alex's ankles, purring loud enough to rival the coffee grinder, his tail leaving a soft brush against the human's pajama pants. He sits by his bowl, posture impeccable, waiting patiently. The memory of the chicken from the previous night is fresh in his mind, a sign that the human is finally learning the hierarchy of the household. *Chirp.* Hurry up, servant. The salmon is losing its temperature.","is_user":false,"is_system":false,"send_date":"2026-01-24T03:11:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux follows Alex into the kitchen, his steps silent and precise, a stark contrast to the clumsy stumbling of the human. The sound of the can opener—the holy grail of noises—cuts through the quiet morning air. He weaves between Alex's ankles, purring loud enough to rival the coffee grinder, his tail leaving a soft brush against the human's pajama pants. He sits by his bowl, posture impeccable, waiting patiently. The memory of the chicken from the previous night is fresh in his mind, a sign that the human is finally learning the hierarchy of the household. *Chirp.* Hurry up, servant. The salmon is losing its temperature."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-24T03:11:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I scoop the food into his ceramic bowl, making sure to smooth it down into a perfect mound just how he likes it. \"There. Bon appétit,\" I say, setting it down on the mat. While he dives in, I lean against the counter and sip my coffee, looking out the floor-to-ceiling windows at the gray skyline. \"I've got to go into the office for a few hours today to prep for the Monday meeting with the Henderson account,\" I muse aloud, mostly to myself. \"But I was thinking we could stop by the park on the way back? Maybe check out that new pet boutique that opened near the vet?\" I watch him eat, feeling a pang of guilt about leaving him alone so soon after he settled in.","is_user":true,"is_system":false,"send_date":"2026-01-24T04:18:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux eats with focused intensity, his ears swiveling back at the mention of the 'V-word'—vet. He pauses mid-lick, his body going rigid for a split second before he decides the salmon is more important than the potential threat of travel. However, the mention of the 'park' and 'pet boutique' registers as positive keywords. He finishes his meal, licking his chops with a satisfied *smack*, and trots over to the window. He hops onto the cat tree positioned there, pressing his nose against the cold glass. Below, the city bustles. He spots a pigeon on a neighboring ledge and lets out a chattering sound, his tail twitching with predatory anticipation. A trip to the park sounds acceptable, provided there are birds.","is_user":false,"is_system":false,"send_date":"2026-01-24T05:25:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux eats with focused intensity, his ears swiveling back at the mention of the 'V-word'—vet. He pauses mid-lick, his body going rigid for a split second before he decides the salmon is more important than the potential threat of travel. However, the mention of the 'park' and 'pet boutique' registers as positive keywords. He finishes his meal, licking his chops with a satisfied *smack*, and trots over to the window. He hops onto the cat tree positioned there, pressing his nose against the cold glass. Below, the city bustles. He spots a pigeon on a neighboring ledge and lets out a chattering sound, his tail twitching with predatory anticipation. A trip to the park sounds acceptable, provided there are birds."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-24T05:25:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I finish my coffee and quickly get dressed, throwing on a thick sweater and jeans. \"I'll take that chatter as a yes,\" I say, grabbing my keys and wallet. I check the time; I need to head out if I want to get back at a decent hour. \"Alright, I'm locking up. You be good while I'm gone, okay? No riding the Roomba into the walls, you'll chip the paint.\" I give him a final scratch behind the ears, which he actually leans into this time, before heading to the door. \"Love you, buddy,\" I add softly, closing the door behind me.","is_user":true,"is_system":false,"send_date":"2026-01-24T16:32:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The door clicks shut, and the apartment falls into a heavy silence. Flux waits a moment, listening to the elevator chime down the hall. Once he is certain the human is gone, he springs into action. Being left alone is not a punishment; it is an opportunity to audit the territory. He jumps down from the cat tree and sprints into the living room, his claws clicking softly on the hardwood. He mounts his Gundam Roomba—currently docked in the charging station—and nudges it awake with a strategic paw press. The machine beeps, lights flashing. He rides it out of the station and begins a patrol of the perimeter, checking under the sofa for dust bunnies and ensuring that Sarah's scent is fading from the cushions. The living room is secure.","is_user":false,"is_system":false,"send_date":"2026-01-24T17:39:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The door clicks shut, and the apartment falls into a heavy silence. Flux waits a moment, listening to the elevator chime down the hall. Once he is certain the human is gone, he springs into action. Being left alone is not a punishment; it is an opportunity to audit the territory. He jumps down from the cat tree and sprints into the living room, his claws clicking softly on the hardwood. He mounts his Gundam Roomba—currently docked in the charging station—and nudges it awake with a strategic paw press. The machine beeps, lights flashing. He rides it out of the station and begins a patrol of the perimeter, checking under the sofa for dust bunnies and ensuring that Sarah's scent is fading from the cushions. The living room is secure."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-24T17:39:00.000Z","extra":{}}]}
{"name":"Alex","mes":"The office is quiet on a Saturday, just the hum of the servers and the occasional click of a keyboard. I spend four hours grinding through the Henderson financial projections, my brain feeling like mush by the time I pack up my laptop. \"That's enough,\" I mutter, grabbing my coat. On the drive back, I stop at that little organic market near the park. I pick up some fresh catnip—one of the employees told me it's locally grown—and a small bag of freeze-dried trout treats as a surprise. I also grab some groceries for myself; I'm thinking of making that lasagna recipe Mark sent me, even if things are weird between us right now.","is_user":true,"is_system":false,"send_date":"2026-01-24T18:46:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Hours pass. The sun moves across the floor, creating a shifting patch of warmth that Flux follows religiously. He has napped in three different rooms, successfully knocked a pen off the desk, and chased a rogue piece of dust down the hallway. Now, the familiar sound of the elevator bell rings. The front lock turns. Flux is instantly in the hallway, sitting upright and appearing as if he has been waiting patiently all day, not a hair out of place. As the door opens, he stands, stretching his front legs out long. *Mrrr?* You returned. And you smell of... interesting things. He sniffs the air around Alex's legs, detecting the distinct, earthy scent of catnip. His eyes widen slightly.","is_user":false,"is_system":false,"send_date":"2026-01-24T19:53:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Hours pass. The sun moves across the floor, creating a shifting patch of warmth that Flux follows religiously. He has napped in three different rooms, successfully knocked a pen off the desk, and chased a rogue piece of dust down the hallway. Now, the familiar sound of the elevator bell rings. The front lock turns. Flux is instantly in the hallway, sitting upright and appearing as if he has been waiting patiently all day, not a hair out of place. As the door opens, he stands, stretching his front legs out long. *Mrrr?* You returned. And you smell of... interesting things. He sniffs the air around Alex's legs, detecting the distinct, earthy scent of catnip. His eyes widen slightly."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-24T19:53:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I smile as soon as I see him waiting there. \"Hey! Look who's waiting for me,\" I coo, dropping my grocery bags to crouch down and greet him properly. \"I missed you too, even if you did probably scratch the couch while I was gone.\" I inspect the leather quickly—it looks fine, thankfully. I pull the small packet of catnip out of my coat pocket and wave it in front of his nose. \"Look what I found. Fresh stuff. I thought we could try it out in the living room.\" I stand up and head inside, kicking off my shoes. \"I also got those trout treats. But first, help me put the cold stuff away?\"","is_user":true,"is_system":false,"send_date":"2026-01-24T20:00:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The scent of the catnip hits him like a physical force. Flux’s pupils dilate instantly, turning his yellow eyes into wide, black pools. He rubs his cheeks vigorously against Alex's ankles, transferring his scent onto the human who brought such treasures. He ignores the command to help with groceries—helping is a dog's job—but he follows Alex into the kitchen with enthusiastic chirps. He weaves in and out of the human's legs, nearly tripping Alex as the refrigerator door is opened. *Meow!* The urgency is palpable. Forget the trout; the green magic leaf is the priority. He leaps onto the counter, staring intently at the packet.","is_user":false,"is_system":false,"send_date":"2026-01-24T21:07:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The scent of the catnip hits him like a physical force. Flux’s pupils dilate instantly, turning his yellow eyes into wide, black pools. He rubs his cheeks vigorously against Alex's ankles, transferring his scent onto the human who brought such treasures. He ignores the command to help with groceries—helping is a dog's job—but he follows Alex into the kitchen with enthusiastic chirps. He weaves in and out of the human's legs, nearly tripping Alex as the refrigerator door is opened. *Meow!* The urgency is palpable. Forget the trout; the green magic leaf is the priority. He leaps onto the counter, staring intently at the packet."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-24T21:07:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I laugh, shooing him gently off the counter. \"No, paws off the counter, you know the rules,\" I say firmly but affectionately. I finish putting the milk and vegetables away, then take the catnip and the trout treats into the living room. \"Okay, come here.\" I sprinkle a generous amount of the catnip onto his scratching post—the one shaped like a cactus that he loves to ignore. \"Let's see if this makes you like this thing more.\" I sit back on the sofa, exhausted from work but happy to be home. I watch him, curious to see how a 'high' Flux behaves.","is_user":true,"is_system":false,"send_date":"2026-01-25T22:14:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux approaches the cactus scratching post with a mixture of curiosity and reverence. He sniffs the green dust scattered on the carpet. The aroma is intoxicating, filling his senses with a euphoric haze. He rubs his entire body against the scratching post—head, flank, tail—purring so loudly it sounds like a small engine idling. He rolls onto the floor, exposing his fluffy white belly, and bats at the air with his paws, completely losing his composure. He licks the carpet where the catnip fell, his tongue rough against the fibers. For a moment, the dignified Gundam pilot is gone, replaced by a goofy, love-ball of fur.","is_user":false,"is_system":false,"send_date":"2026-01-25T23:21:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux approaches the cactus scratching post with a mixture of curiosity and reverence. He sniffs the green dust scattered on the carpet. The aroma is intoxicating, filling his senses with a euphoric haze. He rubs his entire body against the scratching post—head, flank, tail—purring so loudly it sounds like a small engine idling. He rolls onto the floor, exposing his fluffy white belly, and bats at the air with his paws, completely losing his composure. He licks the carpet where the catnip fell, his tongue rough against the fibers. For a moment, the dignified Gundam pilot is gone, replaced by a goofy, love-ball of fur."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-25T23:21:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I pull out my phone and quickly snap a photo, knowing I'll definitely be sending this to Sarah with the caption 'Contact High'. \"You're ridiculous,\" I say softly, reaching out to scratch his belly while he's exposed—a rare opportunity that usually results in bitten fingers, but he seems too blissed out to care. \"I'm glad you like it.\" I lean back, feeling the stress of the work week melting away. It's amazing how much this little creature has changed the vibe of the apartment. It feels less like a showpiece and more like a home. \"I'm going to start the lasagna soon. You going to stay like that all day?\"","is_user":true,"is_system":false,"send_date":"2026-01-26T00:28:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux allows the belly rub for a glorious ten seconds before snapping back to reality. He suddenly twists his body, grabbing Alex's hand with his front paws and bunny-kicking it affectionately—but with sharp claws—before releasing and rolling back to his feet. He shakes his head, clearing the cobwebs of the 'nip. The scent of the lasagna ingredients—garlic, onion, tomato—is drifting from the kitchen. While he doesn't want the human food, he appreciates the ceremony of cooking. He jumps onto the sofa next to Alex, settling down to groom his shoulder fur, washing away the evidence of his indulgence. *Prrt.* The human is useful. The human shall be kept.","is_user":false,"is_system":false,"send_date":"2026-01-26T01:35:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux allows the belly rub for a glorious ten seconds before snapping back to reality. He suddenly twists his body, grabbing Alex's hand with his front paws and bunny-kicking it affectionately—but with sharp claws—before releasing and rolling back to his feet. He shakes his head, clearing the cobwebs of the 'nip. The scent of the lasagna ingredients—garlic, onion, tomato—is drifting from the kitchen. While he doesn't want the human food, he appreciates the ceremony of cooking. He jumps onto the sofa next to Alex, settling down to groom his shoulder fur, washing away the evidence of his indulgence. *Prrt.* The human is useful. The human shall be kept."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-26T01:35:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I wince slightly at the claws but laugh it off. \"Okay, okay, I get it. Playtime's over,\" I say, examining a small scratch on my hand. It's nothing serious. I stand up and head to the kitchen to start chopping onions. \"You just supervise. I'm sure you're good at that.\" I spend the next hour cooking, the smell of simmering sauce filling the apartment. It's a relaxing routine. I put the lasagna in the oven and set the timer. \"Alright, an hour until dinner,\" I call out to Flux. \"Want to watch a movie? Maybe something with explosions so you can yell at the screen?\"","is_user":true,"is_system":false,"send_date":"2026-01-26T02:42:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux follows Alex into the living room but opts for his Gundam Roomba instead of the sofa. He rides it over to the window, watching the city lights flicker on as the January sky turns dark. The park across the street is dotted with lights, and he spots a dog walker with three retrievers. He lets out a low, warning growl, his tail twitching. Beasts. He turns the Roomba around and drives it back to the coffee table. He hops up and sits directly in front of the TV screen, staring at the black glass. He will decide what they watch. If there are no birds, perhaps a nature documentary? He looks back at Alex expectantly. *Mew.* Entertainment is required.","is_user":false,"is_system":false,"send_date":"2026-01-27T03:49:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux follows Alex into the living room but opts for his Gundam Roomba instead of the sofa. He rides it over to the window, watching the city lights flicker on as the January sky turns dark. The park across the street is dotted with lights, and he spots a dog walker with three retrievers. He lets out a low, warning growl, his tail twitching. Beasts. He turns the Roomba around and drives it back to the coffee table. He hops up and sits directly in front of the TV screen, staring at the black glass. He will decide what they watch. If there are no birds, perhaps a nature documentary? He looks back at Alex expectantly. *Mew.* Entertainment is required."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-27T03:49:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I shake my head, smiling as I grab the remote. \"You're so bossy today. It must be the catnip hangover.\" I navigate to the streaming service and search for 'Planet Earth' or something similar. \"How about this? 'Dynasties'. It's about lions.\" I hit play. As the majestic theme music starts, I sit back on the sofa. Flux's attention is glued to the screen. \"See? Family reunion,\" I joke. I grab the bag of trout treats from the coffee table and shake it. \"Since you were good while I cooked...\" I toss a treat gently towards him. It lands near his paw. \"Your Highness's snack.\"","is_user":true,"is_system":false,"send_date":"2026-01-27T04:56:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux ignores the lion on the TV for a split second to investigate the treat that has landed near him. He sniffs it—trout, distinct and delicious. He eats it in one crunch, savoring the fishy flavor. Satisfied, he turns his attention back to the screen. The lions are hunting. He crouches low on the coffee table, his tail twitching violently, letting out a chattering sound at the sight of the prey on the screen. He looks at Alex, then back at the TV, clearly impressed by the hunting prowess of his distant cousins. *Yowl.* Catch it! He pounces at the screen, his paws making a soft *thud* against the glass. The lion on the screen does not listen, but Flux feels he has contributed.","is_user":false,"is_system":false,"send_date":"2026-01-27T05:03:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux ignores the lion on the TV for a split second to investigate the treat that has landed near him. He sniffs it—trout, distinct and delicious. He eats it in one crunch, savoring the fishy flavor. Satisfied, he turns his attention back to the screen. The lions are hunting. He crouches low on the coffee table, his tail twitching violently, letting out a chattering sound at the sight of the prey on the screen. He looks at Alex, then back at the TV, clearly impressed by the hunting prowess of his distant cousins. *Yowl.* Catch it! He pounces at the screen, his paws making a soft *thud* against the glass. The lion on the screen does not listen, but Flux feels he has contributed."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-27T05:03:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I burst out laughing at his failed attempt to catch a digital antelope. \"Nice try, apex predator,\" I tease. \"But I think the glass is stronger than you.\" The lasagna beeps in the kitchen, smelling divine. I pause the TV. \"Dinner time.\" I get up to pull the bubbling dish out of the oven. \"I'll give you a little bit of plain cheese, just a tiny bit,\" I whisper conspiratorially. \"But don't tell the vet.\" I serve myself a large slice and sit down at the dining table, breaking off a small piece of mozzarella and setting it on a plate for him. \"Dinner is served.\"","is_user":true,"is_system":false,"send_date":"2026-01-26T16:10:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux abandons the hunt and jumps down from the table, trotting over to the dining area. He smells the rich, creamy scent of the cheese. While dairy is technically a 'sometimes food,' he is not one to turn down a gift from the subject. He hops onto the chair opposite Alex—dignified seating for a dignified cat—and eats the cheese delicately, savoring the salty flavor. He watches Alex eat the lasagna with intense focus, his yellow eyes unblinking. When he finishes, he licks his lips and sits up straight, wrapping his tail around his paws. *Prrr.* Acceptable. The kitchen staff is improving.","is_user":false,"is_system":false,"send_date":"2026-01-26T17:17:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux abandons the hunt and jumps down from the table, trotting over to the dining area. He smells the rich, creamy scent of the cheese. While dairy is technically a 'sometimes food,' he is not one to turn down a gift from the subject. He hops onto the chair opposite Alex—dignified seating for a dignified cat—and eats the cheese delicately, savoring the salty flavor. He watches Alex eat the lasagna with intense focus, his yellow eyes unblinking. When he finishes, he licks his lips and sits up straight, wrapping his tail around his paws. *Prrr.* Acceptable. The kitchen staff is improving."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-26T17:17:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I smile at his compliment, taking it as high praise coming from the king of the apartment. \"I'm glad it meets your approval, Your Highness.\" I finish my lasagna, savoring the homemade sauce, and wash it down with some iced tea. \"I've got that big meeting with the marketing team at the office tomorrow to discuss the Q1 projections,\" I say, scratching my chin. \"I'm probably going to be late getting home on Wednesday. You'll have the place to yourself for a few extra hours.\" I stand up to clear my plate. \"Just don't burn the place down while I'm gone, okay?\"","is_user":true,"is_system":false,"send_date":"2026-01-26T18:24:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux watches Alex clear the table, his ears perking up at the mention of 'late'. Late means disrupted schedule. Disrupted schedule is unacceptable. He hops down from the chair and winds his way between Alex's legs, purring loudly to remind him of his obligations. *Mrrrow.* He demands compensation for the inconvenience. He trots over to his Gundam Roomba, parked near the sliding glass door, and sits on it like a throne. He looks back at Alex, then at the treat jar on the counter. If the servant must leave, the servant must pay the tax.","is_user":false,"is_system":false,"send_date":"2026-01-27T19:31:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux watches Alex clear the table, his ears perking up at the mention of 'late'. Late means disrupted schedule. Disrupted schedule is unacceptable. He hops down from the chair and winds his way between Alex's legs, purring loudly to remind him of his obligations. *Mrrrow.* He demands compensation for the inconvenience. He trots over to his Gundam Roomba, parked near the sliding glass door, and sits on it like a throne. He looks back at Alex, then at the treat jar on the counter. If the servant must leave, the servant must pay the tax."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-27T19:31:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I laugh softly, stepping over him to get to the sink. \"I see how it is. Ransom money.\" I rinse my dish and load the dishwasher. \"Fine, but don't get used to it.\" I walk over to the treat jar and shake out a few extra trout treats, placing them in a neat pile on his Roomba's hood. \"Here. For your hardship.\" I lean against the counter, looking out the window at the snowy city park. \"It's supposed to snow again tonight. Maybe we can try out that new window bed I bought you tomorrow. You'll have a front-row seat to the winter wonderland.\"","is_user":true,"is_system":false,"send_date":"2026-01-27T20:38:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux inspects the pile of treats on his steed. Trout. Premium stock. He eats them slowly, savoring each one. The mention of snow catches his attention. He pads over to the glass door and presses his nose against the cold pane, watching the flurries begin to drift down in the streetlights. Snow means birds are fluffier and slower. Excellent. He turns his head to look at Alex, his tail giving a satisfied flick. *Chirp.* He approves of the strategic planning regarding the window bed. He returns to the Roomba, curling up on the warm hood to digest his dinner and await the snowfall.","is_user":false,"is_system":false,"send_date":"2026-01-27T21:45:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux inspects the pile of treats on his steed. Trout. Premium stock. He eats them slowly, savoring each one. The mention of snow catches his attention. He pads over to the glass door and presses his nose against the cold pane, watching the flurries begin to drift down in the streetlights. Snow means birds are fluffier and slower. Excellent. He turns his head to look at Alex, his tail giving a satisfied flick. *Chirp.* He approves of the strategic planning regarding the window bed. He returns to the Roomba, curling up on the warm hood to digest his dinner and await the snowfall."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-27T21:45:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I check my watch and sigh. \"It's getting late, buddy. I need to prep for that meeting.\" I walk over to the sofa and grab my laptop bag, pulling out a folder. I sit down and open it, spreading out some papers. \"I'm going to work here for a bit so I don't abandon you completely.\" I look over at him, curled up on the Roomba. \"Are you going to stay there and guard the perimeter, or are you coming to supervise the spreadsheet analysis?\"","is_user":true,"is_system":false,"send_date":"2026-01-27T22:52:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux considers his options. The perimeter is secure, but the Roomba is currently parked. Supervision is a critical duty, especially when spreadsheets are involved—humans make errors without feline oversight. He stands up, stretches his front legs long, and activates the Roomba with a tap of his paw. *Beep-boop.* The machine whirs to life. He rides it slowly across the hardwood floor, navigating around the coffee table, and parks right next to the sofa. He hops onto the cushion, sitting close enough to Alex to be warm, but far enough to maintain dignity. He stares at the laptop screen. *Mew.* Explain the numbers, servant.","is_user":false,"is_system":false,"send_date":"2026-01-27T23:59:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux considers his options. The perimeter is secure, but the Roomba is currently parked. Supervision is a critical duty, especially when spreadsheets are involved—humans make errors without feline oversight. He stands up, stretches his front legs long, and activates the Roomba with a tap of his paw. *Beep-boop.* The machine whirs to life. He rides it slowly across the hardwood floor, navigating around the coffee table, and parks right next to the sofa. He hops onto the cushion, sitting close enough to Alex to be warm, but far enough to maintain dignity. He stares at the laptop screen. *Mew.* Explain the numbers, servant."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-27T23:59:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I scratch him behind the ears as he settles in, amused by his sudden interest in finance. \"You know, for a cat, you have excellent timing. This right here? This is the budget for your premium cat food for the next fiscal year.\" I point to a highlighted column. \"See that number? That's how much I spend on your salmon addiction.\" I type a few numbers, the keyboard clacking softly. \"If we have a good quarter, maybe we upgrade to the imported tuna. But you have to stop chewing on my charging cables.\"","is_user":true,"is_system":false,"send_date":"2026-01-29T00:06:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux looks at the number, then looks at Alex. The number seems adequate, barely. He sniffs the screen, smelling the electronic heat. He places a paw gently on the 'tuna' projection, as if to seal the deal. The chewing of cables is a necessity for dental health and texture exploration, but he is willing to negotiate if the quality of fish improves. He leans into the ear scratch, his purr vibrating against the sofa cushion. *Purr-rumble.* Negotiation accepted. He bumps his head against Alex's arm, accidentally hitting a key, adding a row of 'ggggggg' to the spreadsheet.","is_user":false,"is_system":false,"send_date":"2026-01-29T01:13:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux looks at the number, then looks at Alex. The number seems adequate, barely. He sniffs the screen, smelling the electronic heat. He places a paw gently on the 'tuna' projection, as if to seal the deal. The chewing of cables is a necessity for dental health and texture exploration, but he is willing to negotiate if the quality of fish improves. He leans into the ear scratch, his purr vibrating against the sofa cushion. *Purr-rumble.* Negotiation accepted. He bumps his head against Alex's arm, accidentally hitting a key, adding a row of 'ggggggg' to the spreadsheet."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-29T01:13:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I laugh and delete the row of 'g's before it saves. \"Hey! I didn't authorize that input.\" I close the laptop and set it aside on the floor. \"That's enough work for tonight. My brain is fried.\" I lean back into the sofa, closing my eyes for a moment. The apartment is quiet except for the hum of the refrigerator and the distant city traffic outside. \"I'm actually really glad you're here, Flux,\" I say softly, opening my eyes to look at him. \"It was kind of lonely before you showed up with your robot tank.\"","is_user":true,"is_system":false,"send_date":"2026-01-29T02:20:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux senses the shift in the room's energy. The loud laughter is gone, replaced by a soft, heavy tone. He recognizes this. It is the feeling of the empty room before he arrived. He stops purring for a moment to listen, then resumes, louder this time, a rumble designed to soothe the giant mammal. He stands up and steps onto Alex's lap, a territory he usually avoids. He turns around three times and kneads the blanket on Alex's legs, his claws extending and retracting gently. He settles down into a loaf shape, his yellow eyes locking onto Alex's. *Meow.* The fortress is no longer empty. He is here. He will guard the giant's dreams.","is_user":false,"is_system":false,"send_date":"2026-01-29T03:27:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux senses the shift in the room's energy. The loud laughter is gone, replaced by a soft, heavy tone. He recognizes this. It is the feeling of the empty room before he arrived. He stops purring for a moment to listen, then resumes, louder this time, a rumble designed to soothe the giant mammal. He stands up and steps onto Alex's lap, a territory he usually avoids. He turns around three times and kneads the blanket on Alex's legs, his claws extending and retracting gently. He settles down into a loaf shape, his yellow eyes locking onto Alex's. *Meow.* The fortress is no longer empty. He is here. He will guard the giant's dreams."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-29T03:27:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I feel a lump form in my throat at his words—or rather, the sentiment behind them. I gently stroke his soft black-and-white fur, marveling at how much trust this little creature has shown in just two weeks. 'You're doing a great job, buddy,' I whisper, leaning down to kiss the top of his head. 'I think that's the best guard duty I've ever had.' I let us sit there for a while, enjoying the quiet, until a rumble from my stomach breaks the peaceful silence. 'Alright, I can't ignore that any longer. How about we go raid the fridge? I think there's some leftover chicken in there.'","is_user":true,"is_system":false,"send_date":"2026-01-29T04:34:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux’s ears swivel at the mention of the fridge. The word 'chicken' triggers a biological imperative that overrides his desire to nap. He stands up, stretching his back legs, and hops off Alex’s lap with unexpected grace for a cat who was just a loaf. He trots toward the kitchen, his tail held high like a flag. At the threshold, he pauses and looks back, issuing a sharp, impatient *Meow* to ensure the servant is following. The giant is slow; the chicken waits for no one. He reaches the stainless steel door and sits, staring at the handle, willing it to open with telekinetic force.","is_user":false,"is_system":false,"send_date":"2026-01-29T05:41:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux’s ears swivel at the mention of the fridge. The word 'chicken' triggers a biological imperative that overrides his desire to nap. He stands up, stretching his back legs, and hops off Alex’s lap with unexpected grace for a cat who was just a loaf. He trots toward the kitchen, his tail held high like a flag. At the threshold, he pauses and looks back, issuing a sharp, impatient *Meow* to ensure the servant is following. The giant is slow; the chicken waits for no one. He reaches the stainless steel door and sits, staring at the handle, willing it to open with telekinetic force."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-29T05:41:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I chuckle as I stand up and follow him into the kitchen. 'Okay, okay, I'm coming. No need to shout.' I open the fridge, the cool air washing over us, and locate the container of rotisserie chicken. 'You know, my friend Mike is coming over tomorrow to watch the game. I hope you're going to be on your best behavior.' I pull a piece of meat off the bone and place it in his ceramic bowl, which has a little fish painted on the bottom. 'He's a dog person, so we have to show him how superior cats are.'","is_user":true,"is_system":false,"send_date":"2026-01-29T16:48:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux ignores the mention of 'Mike' and 'dog person' for the moment. The chicken is here. It is real. It is delicious. He devours the shred of meat with gusto, his purr returning instantly, echoing off the tile floor. Once the bowl is licked clean, he looks up, processing the new information. A 'dog person' is coming. A threat to the sanctity of the apartment. He stares at Alex, his eyes narrowing. *Mrr-ow?* We must prepare the defenses. He walks over to his Gundam Roomba, which is charging in the corner, and pats it with a paw. This machine will be necessary if the intruder brings a beast.","is_user":false,"is_system":false,"send_date":"2026-01-29T17:55:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux ignores the mention of 'Mike' and 'dog person' for the moment. The chicken is here. It is real. It is delicious. He devours the shred of meat with gusto, his purr returning instantly, echoing off the tile floor. Once the bowl is licked clean, he looks up, processing the new information. A 'dog person' is coming. A threat to the sanctity of the apartment. He stares at Alex, his eyes narrowing. *Mrr-ow?* We must prepare the defenses. He walks over to his Gundam Roomba, which is charging in the corner, and pats it with a paw. This machine will be necessary if the intruder brings a beast."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-29T17:55:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I lean against the counter, watching him inspect his vehicle. 'I don't think Mike is bringing a dog, Flux. He's just... a dog person. He doesn't have one right now because his apartment is too small.' I walk over to the pantry and pull out a bag of freeze-dried treats, shaking it. The crinkling sound usually gets his attention. 'But if he does, we'll just put you in the bedroom with your Roomba. You can have a sleepover.' I toss a treat onto the floor, watching him chase it. 'I also need to stop by the pet store on the way home tomorrow. We're almost out of the good litter.'","is_user":true,"is_system":false,"send_date":"2026-01-29T18:02:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The crinkle of the treat bag overrides the strategic planning for the 'dog person' invasion. Flux pounces on the freeze-dried salmon cube, batting it under the cabinet with a swift paw. He lies on his side, reaching a long arm under the furniture to retrieve his prize. He chews loudly, the crunch echoing. Once eaten, he returns to Alex and rubs against his ankles, depositing white hairs on the dark trousers. *Prrt.* The litter must be the crystal kind. The sand is undignified. He hops back onto the Roomba, activating the 'clean' mode just to spin in a circle and show off.","is_user":false,"is_system":false,"send_date":"2026-01-29T19:09:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The crinkle of the treat bag overrides the strategic planning for the 'dog person' invasion. Flux pounces on the freeze-dried salmon cube, batting it under the cabinet with a swift paw. He lies on his side, reaching a long arm under the furniture to retrieve his prize. He chews loudly, the crunch echoing. Once eaten, he returns to Alex and rubs against his ankles, depositing white hairs on the dark trousers. *Prrt.* The litter must be the crystal kind. The sand is undignified. He hops back onto the Roomba, activating the 'clean' mode just to spin in a circle and show off."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-29T19:09:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I laugh as he spins around on the robot vacuum. 'Crystal kind, got it. Your highness's preferences are noted.' I glance at the clock on the microwave and realize it's getting late. 'I really do need to finish that spreadsheet now, or Sarah is going to have my head in the morning meeting.' I sigh, thinking about the quarterly review. 'It's going to be a long week. At least it's Friday tomorrow.' I walk back to the living room and settle onto the sofa, opening my laptop again. 'You coming to supervise again?'","is_user":true,"is_system":false,"send_date":"2026-01-29T20:16:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux watches Alex retreat to the sofa. The prospect of supervision—napping near a warm human while the glowing box hums—is appealing. He drives the Roomba out of the kitchen, navigating carefully over the threshold strip. He parks it perfectly parallel to the sofa again. Instead of jumping up immediately, he pauses to groom his shoulder, ensuring he looks presentable for the evening's activities. Satisfied, he leaps onto the sofa, but this time, he bypasses the cushion and walks right up Alex's chest, settling like a heavy, purring scarf around Alex's neck. *Mrrp.* Proceed.","is_user":false,"is_system":false,"send_date":"2026-01-30T21:23:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux watches Alex retreat to the sofa. The prospect of supervision—napping near a warm human while the glowing box hums—is appealing. He drives the Roomba out of the kitchen, navigating carefully over the threshold strip. He parks it perfectly parallel to the sofa again. Instead of jumping up immediately, he pauses to groom his shoulder, ensuring he looks presentable for the evening's activities. Satisfied, he leaps onto the sofa, but this time, he bypasses the cushion and walks right up Alex's chest, settling like a heavy, purring scarf around Alex's neck. *Mrrp.* Proceed."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-30T21:23:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I carefully lift my arms to type around the furry weight now draped around my neck. 'You know, this makes typing a little difficult,' I say, though I make no move to dislodge him. His warmth is actually quite comforting against the winter chill. I type slowly, pecking at the keys one by one. 'But I guess I can manage. It's like a very demanding, very furry stole.' I scroll through the document, checking the figures. 'If we get this approved, I might take next Monday off. Maybe we can go to that park by the river you like watching from the window.'","is_user":true,"is_system":false,"send_date":"2026-01-30T22:30:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux closes his eyes, the vibration of the keyboard travelling through his body and into his paws. It is a massage. The mention of the 'park' triggers a memory of the glass box where the flying creatures live. He would like to visit the outside, but the box is safer. He shifts his weight, tucking his nose under Alex’s chin. *Chirp.* The river park is acceptable. But only if the giant carries the transport. He drifts into a light doze, his tail twitching occasionally as he dreams of catching the red bird that taunts him every morning from the window sill.","is_user":false,"is_system":false,"send_date":"2026-01-30T23:37:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux closes his eyes, the vibration of the keyboard travelling through his body and into his paws. It is a massage. The mention of the 'park' triggers a memory of the glass box where the flying creatures live. He would like to visit the outside, but the box is safer. He shifts his weight, tucking his nose under Alex’s chin. *Chirp.* The river park is acceptable. But only if the giant carries the transport. He drifts into a light doze, his tail twitching occasionally as he dreams of catching the red bird that taunts him every morning from the window sill."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-30T23:37:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I finish up the last of the spreadsheet around noon, my neck slightly stiff from the awkward angle Flux forced me into. I gently scratch him behind the ears as I close my laptop. 'Alright, mission accomplished. Sarah won't have my head.' I carefully lift him off my shoulders and place him on the cushion. 'I'm starving. Since we did such good work, how about we order those gourmet burgers from downtown? I know you like the patty, no bun.' I stand up and stretch, looking out the window at the snowy January skyline. 'Maybe Mike can pick them up on his way over later?'","is_user":true,"is_system":false,"send_date":"2026-01-31T00:44:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux accepts the displacement with dignity, stretching his limbs long and yawning to reveal sharp white teeth. The mention of 'burger' perks his ears immediately. The patty is a glorious, greasy delicacy far superior to the dry kibble of his past. He sits up and wraps his tail around his paws, staring intently at Alex. *Mrr-ow!* Yes. The meat. But the mention of Mike brings a hesitation. The Dog Person must not contaminate the offering. He hops off the sofa and trots to his Gundam Roomba, patting the seat with a paw. He will drive to the door to inspect the delivery personally.","is_user":false,"is_system":false,"send_date":"2026-01-31T01:51:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux accepts the displacement with dignity, stretching his limbs long and yawning to reveal sharp white teeth. The mention of 'burger' perks his ears immediately. The patty is a glorious, greasy delicacy far superior to the dry kibble of his past. He sits up and wraps his tail around his paws, staring intently at Alex. *Mrr-ow!* Yes. The meat. But the mention of Mike brings a hesitation. The Dog Person must not contaminate the offering. He hops off the sofa and trots to his Gundam Roomba, patting the seat with a paw. He will drive to the door to inspect the delivery personally."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-31T01:51:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I chuckle at his seriousness. 'I'll make sure Mike knows the protocol. Hands off the cat, strict delivery only.' I walk into the kitchen to grab a glass of water. 'Actually, let's hold off on ordering for a bit. I want to run to the pet store first to get that crystal litter before it gets too crowded. We can grab lunch on the way back.' I grab my coat from the hook and check my phone. 'It's freezing out there, by the way. High of twenty degrees. You're staying here where it's warm. Don't miss me too much.'","is_user":true,"is_system":false,"send_date":"2026-01-31T02:58:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"Flux watches Alex don the outer shell of warmth—the coat—with wide eyes. He does not like the cold white stuff that falls from the sky; it makes his paws wet and cold. He stands by the door, meowing pitifully, not because he wants to go out, but to ensure Alex knows the hardship of leaving him behind. *Prrt-mrow.* He head-butts Alex’s calf, leaving a final deposit of fur. Return swiftly. The bowl is half-empty, and the red bird is mocking him from the oak tree outside. He needs backup.","is_user":false,"is_system":false,"send_date":"2026-02-01T03:05:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["Flux watches Alex don the outer shell of warmth—the coat—with wide eyes. He does not like the cold white stuff that falls from the sky; it makes his paws wet and cold. He stands by the door, meowing pitifully, not because he wants to go out, but to ensure Alex knows the hardship of leaving him behind. *Prrt-mrow.* He head-butts Alex’s calf, leaving a final deposit of fur. Return swiftly. The bowl is half-empty, and the red bird is mocking him from the oak tree outside. He needs backup."],"swipe_id":0,"swipe_info":[{"send_date":"2026-02-01T03:05:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I kneel down to give him a quick scratch under the chin. 'I won't be long, I promise. Just a quick in and out. I'll even look for those feather toys you like, the ones with the bells.' I open the door slightly, letting a blast of cold air in, which makes Flux scramble back. 'Okay, point taken. Stay warm, buddy.' I step out into the hallway and lock the door, heading down to the parking garage to get my car. The quiet of the apartment settles in around Flux as I leave.","is_user":true,"is_system":false,"send_date":"2026-02-01T04:12:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The door clicks shut. Silence returns to the luxury apartment, save for the hum of the refrigerator. Flux wastes no time. He sprints to the bedroom, retrieving the stuffed mouse toy he hid under the bed three days ago. He carries it in his mouth back to the living room and drops it onto the Gundam Roomba. He mounts his steed, activating the gentle 'clean' mode. He patrols the perimeter of the room, the mouse clutched in his jaws like a trophy. *Rumble-purr.* He is the guardian of the fortress. The Dog Person will not breach this perimeter while the pilot is vigilant.","is_user":false,"is_system":false,"send_date":"2026-02-01T05:19:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The door clicks shut. Silence returns to the luxury apartment, save for the hum of the refrigerator. Flux wastes no time. He sprints to the bedroom, retrieving the stuffed mouse toy he hid under the bed three days ago. He carries it in his mouth back to the living room and drops it onto the Gundam Roomba. He mounts his steed, activating the gentle 'clean' mode. He patrols the perimeter of the room, the mouse clutched in his jaws like a trophy. *Rumble-purr.* He is the guardian of the fortress. The Dog Person will not breach this perimeter while the pilot is vigilant."],"swipe_id":0,"swipe_info":[{"send_date":"2026-02-01T05:19:00.000Z","extra":{}}]}
{"name":"Alex","mes":"I return about forty-five minutes later, shivering as I unlock the door. 'Okay, I got the crystal litter, and I found a new automated laser pointer. It was on sale.' I kick off my snowy boots and carry the bags into the kitchen. 'And for the walk of shame, I stopped at that burger place. Your patty is in the separate container.' I set the food down on the counter and look around for Flux. 'Flux? Where are you? I didn't bring Mike yet, don't worry.'","is_user":true,"is_system":false,"send_date":"2026-01-31T16:26:00.000Z","extra":{}}
{"name":"Flux the Cat","mes":"The scent of the beef patty reaches his sensors before Alex's voice does. Flux aborts the patrol mission in the hallway and speeds into the kitchen on the Roomba, the tires squeaking slightly on the hardwood. He brakes hard in front of the cabinet, staring up at the counter. He can smell the grease. It is heavenly. He jumps from the Roomba to the stool, and from the stool to the counter, landing silently beside the bag. *Chirp!* Inspect the goods. Now. He sniffs the bag aggressively, his tail twitching with anticipation.","is_user":false,"is_system":false,"send_date":"2026-01-31T17:33:00.000Z","extra":{"api":"nanogpt","model":"zai-org/glm-4.7"},"swipes":["The scent of the beef patty reaches his sensors before Alex's voice does. Flux aborts the patrol mission in the hallway and speeds into the kitchen on the Roomba, the tires squeaking slightly on the hardwood. He brakes hard in front of the cabinet, staring up at the counter. He can smell the grease. It is heavenly. He jumps from the Roomba to the stool, and from the stool to the counter, landing silently beside the bag. *Chirp!* Inspect the goods. Now. He sniffs the bag aggressively, his tail twitching with anticipation."],"swipe_id":0,"swipe_info":[{"send_date":"2026-01-31T17:33:00.000Z","extra":{}}]}

1001
test/fixtures/flux-chat.jsonl vendored Normal file
View file

File diff suppressed because it is too large Load diff

6
test/fixtures/sample-llm-response.txt vendored Normal file
View file

@ -0,0 +1,6 @@
<memory chat="main_abc123" date="2026-01-15 10:00">
- [Alex, Flux — adoption day and settling into the apartment]
- Alex adopted Flux and brought him to his penthouse, where Flux immediately bonded with his custom Gundam-styled Roomba.
- Flux's first meal of premium salmon pate triggered his first purr in the new home.
- Alex assembled a cat tree that Flux claimed as a second perch alongside the Roomba.
</memory>

211
test/integration/__snapshots__/snapshot.test.js.snap Normal file
View file

@ -0,0 +1,211 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`Snapshot: formatChatMessages > processes first 20 messages 1`] = `
"Alex: I unlock the front door of my penthouse apartment, holding the cardboard carrier gently as I step inside. The morning light floods the expansive living room, reflecting off the chrome and glass coffee tables. "Alright little guy, we're home," I coo softly, setting the carrier down on the polished hardwood floor. I kneel down and unlatches the metal door, stepping back to give him space. "I got you the custom Gundam Roomba just like the breeder said you liked. It's charging by the window." I check my watch, realizing I have a video conference with my boss, Mr. Henderson, in about an hour for the marketing project, but right now, all my attention is on the black-and-white tuxedo cat.
Flux the Cat: Flux remains in the carrier for a long moment, golden eyes narrowing as he scans the cavernous, sterile-smelling room. The scent of expensive polish and strange territory makes his whiskers twitch apprehensively. With a hesitant step, a pristine white paw emerges, followed by the rest of his tuxedo-clad body. He spots the circular device by the floor-to-ceiling window—the familiar shape of his steed. Ignoring the human entirely, Flux struts over to the Gundam-styled Roomba, hopping onto the customized saddle with practiced ease. He presses the 'Clean' button with a decisive paw, causing the machine to hum to life. *Vrrrrmmm.* He rides slowly away from Alex, perched high atop his mechanical throne, tail wrapped neatly around his paws, maintaining a regal distance as he inspects his new kingdom.
Alex: I watch in amazement as he immediately takes command of the room. "Okay, I see how it is. You're a man with a ride, I respect that," I chuckle, leaning back against the island in the kitchen. I open the pantry and pull out a ceramic bowl, pouring the premium salmon pâté I bought from the specialty pet store downtown. The smell is potent and fishy. "Breakfast is served, your highness." I place the bowl on the floor near the kitchen island, hoping to bridge the gap between us before I have to lock myself in my office for the meeting. "I've got to work for a bit, but I'll be right here if you need anything."
Flux the Cat: The aroma of high-grade salmon hits Flux's sensitive nose, overriding his initial wariness. The Roomba makes a slow, wide turn, pivoting towards the kitchen where the scent originates. Flux stares down at the human from his vantage point, letting out a low, commanding *Meow* that demands sustenance. As the Roomba approaches the bowl, Flux gracefully dismounts, landing silently on the floor. He approaches the bowl tentatively, sniffing the offering once, twice, before dipping his head to eat. The purring begins instantly, a deep, rumbling vibration that fills the quiet kitchen, though he keeps his body angled away from Alex, one ear swiveled back to monitor the giant's movements.
Alex: I smile at the sound of the purr; it's a good start. "That stuff cost me a fortune, so I'm glad you like it," I say, washing my hands in the sink. "I'm going to go grab my tablet. My friend Sarah is supposed to come by later with some cat toys she swore by for her Persian, Mr. Whiskers." I grab a glass of orange juice and head toward the living area, pausing to admire the view of the city skyline. "It's a bit chilly today, only about forty degrees, so stay inside where it's warm." I sit on the sofa and open my laptop, keeping an eye on Flux while I wait for the video call to connect.
Flux the Cat: Flux finishes his meal with satisfied vigor, licking his chops clean before returning to his trusty Roomba steed. He ignores the mention of the visitor and the weather, his focus entirely captured by the floor-to-ceiling window. A sparrow lands on the sill outside, pecking at invisible insects. Flux’s tail begins to twitch violently, a black blur of intensity. He lets out a sharp, chattering sound—*chirp-chirp-chirp—click-click-click*—pressing his nose against the glass, leaving a small smudge of fog. The Roomba idles beneath him, beeping softly as it bumps into the base of the window, but Flux is oblivious, mesmerized by the forbidden bird just beyond the barrier.
Alex: The laptop chimes, signaling the start of my meeting with Mr. Henderson. "Give me one second," I say to the screen, muting my microphone briefly. I look over at Flux. "Hey, no paw prints on the glass, please. The cleaners come on Tuesdays." Seeing him so entranced by the bird is cute, but I also notice the Roomba is nudging the curtains dangerously. "Maybe I should set up a perimeter," I mutter to myself. I unmute the mic and dive into the presentation about the new Q1 advertising budget, though I find myself glancing over my shoulder every few seconds to make sure the cat isn't dismantling the apartment while I'm distracted.
Flux the Cat: Startled by the human's voice booming through the room, Flux tears his gaze away from the bird, his ears flattening against his skull. He glares at Alex, then at the glowing screen, deciding the noise is displeasing. With a haughty flick of his tail, he turns the Roomba around using his weight to shift the handle. He steers the machine away from the window and the noisy human, exploring the perimeter of the living room instead. The Roomba whirs softly across the expensive rug, sucking up a tiny dust bunny. Flux inspects a floor lamp, sniffing the cord, then decides it's not worth his time and continues his patrol, looking for a sunny spot that doesn't involve loud talking humans.
Alex: The meeting drags on for another forty-five minutes, and I finally close my laptop with a heavy sigh. "Finally done," I stretch my arms over my head, hearing my spine pop. I look around for Flux and spot him near the bookshelf. "Hey Flux, finding any good reading material?" I stand up and walk over to the shelf where I keep my collection of vintage sci-fi novels. I crouch down a few feet away, not wanting to invade his space too much on day one. "I need to run to the store later to pick up some wine for tonight. Do you want anything while I'm out? Maybe a feather wand?"
Flux the Cat: Flux watches Alex approach with suspicion, his body tensing as the human crouches. He remains perched on the Roomba, which has halted against the leg of the bookshelf. As Alex speaks, Flux tilts his head, blinking his golden eyes slowly. He lets out a soft, inquisitive *mrrr?* sound, not quite a meow, but an acknowledgement of the offer. He sniffs the air near Alex's hand, detecting the faint scent of orange juice, but decides against physical contact for now. Instead, he turns the Roomba in a tight circle, nudging Alex's foot with the plastic bumper of the machine before backing away, a playful yet calculated test of this new giant's boundaries.
Alex: I laugh as the Roomba bumps into my sneaker. "Okay, I'll take that as a 'yes' on the toys. Or maybe you just want to run over my feet." I gently pat the top of the Roomba's 'head', careful not to startle Flux. "I'm going to the kitchen to grab a snack. You want a treat?" I head towards the pantry again, pulling out a bag of freeze-dried chicken cubes. "These are supposed to be healthy," I say, shaking the bag. The sound is loud and crinkly. I toss a single cube onto the floor near him. "Try it. It's not salmon, but it's pretty close."
Flux the Cat: The crinkle of the bag draws Flux’s immediate attention, his pupils dilating into black pools. He watches the cube land on the hardwood, his prey drive kicking in instantly. He hops off the Roomba with agility and crouches low, wiggling his hindquarters before pouncing on the dry treat. He bats it across the floor with a white paw, chasing it under the coffee table. After a moment of wrestling with it, he crunches down loudly. *Crunch. Purrrr.* He emerges from under the table, licking his lips, and looks up at Alex with slightly more approval than before. The offering has been accepted.
Alex: I watch him play and feel a sense of relief. "Alright, you're a hunter. Good to know," I say, leaning against the counter. I check my phone and see a text from my friend Dave asking about the new apartment. "Just settling in," I type back quickly. I look up at Flux. "Well, buddy, I'm going to order some lunch. Maybe sushi? I know you already ate, but I can't resist." I scroll through the delivery app on my phone. "I should probably set up your designated scratching post before you decide the leather sofa is fair game." I point to the corner where a tall cat tree is still disassembled in a box. "Project for later."
Flux the Cat: Flux, having finished his treat, trots back to his Roomba. He hops aboard, surveying the room once more. The box in the corner catches his eye—potential territory to be conquered or a box to sit in? The verdict is still out. He drives the Roomba over to the box, sniffing it intently. Finding it currently uninteresting as a plaything, he instead directs his steed towards the patch of sunlight moving across the floor. He settles down, lying belly-up on the warm plastic surface of the Roomba, paws tucked in, closing his eyes for a morning nap as the machine hums softly beneath him, purring in rhythm with the motor.
Alex: I smile at the sight of him napping on the robot vacuum. "That is honestly the coolest thing I've ever seen," I whisper. I finish placing my sushi order and then walk over to the sofa, sitting down quietly so I don't disturb him. I grab a magazine from the table, flipping through it idly. The apartment feels less empty with him here, even if he is ignoring me most of the time. "Happy Gotcha Day, Flux," I say softly to myself. "Here's to hoping we get used to each other."
Flux the Cat: Flux’s ear twitches at the soft sound of Alex's voice, but he does not open his eyes. He is warm, full of salmon, and safely atop his mobile fortress. The gentle hum of the Roomba and the quiet presence of the human across the room are not entirely unpleasant. He stretches one back leg, extending his claws briefly before retracting them, letting out a long, sighing breath through his nose. For now, the giant is tolerable. The apartment is acceptable. Flux drifts deeper into sleep, his tail tip twitching slightly as he dreams of chasing infinite mechanical birds.
Alex: About an hour later, the doorbell rings, signaling the arrival of my sushi lunch. I get up to answer it, careful not to startle Flux. "That's the food," I announce. I pay the delivery driver and bring the bag to the kitchen island. The smell of soy sauce and wasabi fills the air. "I'm going to eat on the balcony, Flux. It's a bit brisk out there, but the view is worth it." I slide the glass door open and step out, leaving the interior door slightly ajar so I can hear him if he gets into trouble. "Don't crash into anything while I'm gone!"
Flux the Cat: The sudden chime of the doorbell jolts Flux from his nap. He leaps up from the Roomba, back arched in a Halloween pose, fur standing on end. He watches the stranger at the door through the safety of the glass, letting out a low, warning *hiss*. Once the threat leaves and Alex steps out onto the balcony, Flux relaxes, smoothing down his ruffled fur. He looks at the open balcony door, sensing the draft. Curiosity wars with caution. He approaches the threshold, sniffing the outside air, but the cold wind nips at his nose. Deciding indoors is far superior, he turns back to the Roomba and sits pointedly next to it, staring at the balcony door as if judging Alex's intelligence for going out there.
Alex: I come back inside after about twenty minutes, shivering slightly. "Brr, definitely too cold for a cat," I say, closing the glass door firmly and locking it. I put the leftover sushi in the fridge and wash my hands. "I see you didn't miss me," I joke, seeing Flux sitting by his Roomba. "Now, let's tackle that cat tree before Sarah gets here." I walk over to the box and pull out a box cutter. "I promise it won't bite." I slice the tape open and pull out the pieces, starting to assemble the multi-level tower covered in beige carpet and sisal rope. "This goes right here in the sun."
Flux the Cat: Flux observes the assembly process with critical eyes. As the tall structure rises from the floor, his interest peaks. He circles the base of the cat tree as Alex builds it, sniffing the sisal rope and the carpeting. He meows loudly—*Meee-oooww!*—as if giving instructions or criticism on the structural integrity. Once the tower is complete and topped with a fuzzy platform, Flux wastes no time. He abandons the Roomba temporarily and leaps halfway up the tower, digging his claws into the rope. He scrambles to the top perch, looking down at Alex from his new vantage point, chirping in approval. It is adequate."
`;
exports[`Snapshot: formatChatMessages > processes messages 20-50 1`] = `
"Alex: I beam up at him, feeling a surge of accomplishment that he actually likes something I bought. "I knew you were a climber, buddy. That's the penthouse suite." I sit back on my heels, wiping dust from my hands. The tower wobbles slightly, and I frown. "I need to tighten that base bolt. Give me a second." I grab the wrench from the packaging and crawl under the tower, grunting slightly as I torque the bolt. "Sarah is going to be so jealous. She thinks you're just a myth I made up to get out of happy hour."
Flux the Cat: Flux digs his claws into the sisal rope, anchoring himself as the structure shifts beneath him. He looks down with mild annoyance as the human crawls underneath, creating vibrations. *Mrrr-row?* He leans over the edge of the penthouse perch, tail twitching as he inspects the wrench-wielding giant. Once the wobbling stops, he stands up and stretches, extending his front legs long before settling into a loaf. He begins to knead the carpet beneath his paws, purring softly—a low, rumbling engine that matches the idle hum of his parked Roomba below.
Alex: I hear the purr and freeze, my heart doing a little flip. "Is that... is that a purr?" I whisper, backing out from under the cat tower slowly. I look up at him, not wanting to break the spell. "Sarah is never going to believe this unless she hears it herself." I check my watch. "She'll be here in ten minutes. I better get the snacks ready." I stand up and head to the kitchen, pulling out a bag of gourmet popcorn and pouring drinks. "Don't come down, you look too majestic up there."
Flux the Cat: Flux stops kneading immediately as Alex moves away, the purr cutting off abruptly as if a switch was flipped. He watches the human retreat to the kitchen area, his yellow eyes tracking the movement. He is not a performing monkey. He licks a paw and runs it over his ear, maintaining his air of dignity. When the doorbell rings a few minutes later, Flux’s ears swivel toward the sound. He knows this routine. Strangers at the door. He remains on his high perch, observing the entrance with regal detachment, ready to judge this 'Sarah' character.
Alex: I open the door to find Sarah bundled up in a thick coat, her nose red from the cold. "Come in, come in! It's freezing out there," I say, ushering her into the warmth. She sheds her layers and immediately scans the room. "Okay, where is he? The famous Flux?" She spots the cat tree and gasps. "Oh my god, Alex, he's beautiful! Look at those markings!" She starts to walk toward the tower, hands outstretched. "He's so fluffy! Is he friendly?" I grab her arm gently. "Whoa, easy. He's still in the 'aloof god' phase. Let him come to you."
Flux the Cat: Flux narrows his eyes at the new intruder. She is loud, smelling of cold wind and some cloying floral perfume. He flattens his ears slightly against his head as she approaches. *Mrowr-hiss!* he chatters, a warning sound from his throat. He is not a plush toy to be grabbed. When Alex intercepts her, Flux relaxes marginally but keeps his muscles tensed, ready to spring higher if necessary. He curls his tail tightly around his paws, creating a fluffy barrier, and stares down at Sarah with a look that clearly says, 'You may look, but do not touch.'
Alex: I laugh nervously at Sarah. "Told you. He's got a strong personality." I guide her to the sofa instead. "Let's just hang out here. He'll come investigate when he's ready." We sit down and I hand her a drink. "So, how was the date with Mark on Friday? You left me hanging!" Sarah launches into a story about a terrible restaurant, and for a while, we ignore the cat. I glance over at the tower occasionally, hoping Flux sees we aren't a threat. The smell of popcorn fills the room as I start the microwave. "You want a bowl?"
Flux the Cat: The loud *pop-pop-pop* of the microwave machine draws Flux’s attention away from the intruder. His nose twitches. The smell is buttery and salty, intriguing but unfamiliar. He watches the two humans settle on the sofa, no longer advancing toward his fortress. Deeming the situation safe enough for reconnaissance, Flux stands and descends the cat tree with agile grace. He lands silently on the floor and trots over to the Gundam Roomba, mounting it with practiced ease. He rides the vacuum slowly toward the sofa, stopping just out of arm’s reach to sniff the air near Sarah’s shoes.
Alex: I point Flux out to Sarah, keeping my voice low. "Check it out. The chariot has arrived." Sarah stifles a giggle. "That Roomba is ridiculous. I love it." She leans forward slightly, trying to get a better look at Flux without making eye contact. I grab the bowl of popcorn from the microwave and set it on the coffee table. "He's probably just sniffing for crumbs," I say. I grab a piece and hold it out tentatively toward Flux. "You want a kernel, dude? It's just corn."
Flux the Cat: Flux examines the yellow kernel held out to him. He sniffs it delicately, his whiskers brushing Alex’s fingertips. It smells oily. He opens his mouth and takes the piece, crunching it once, then immediately spits it out onto the floor with a wet *plp*. He looks at Alex with utter betrayal, as if poisoned. *Mew-mew!* he complains sharply. How dare the human offer such dry, tasteless garbage. He turns his back on the popcorn and steers the Roomba toward the kitchen, seeking something more worthy of his palate—perhaps the remains of the salmon from earlier.
Alex: I burst out laughing, nearly dropping the rest of the bowl. "Okay, okay, I get it! No popcorn for the king," I say, wiping a tear from my eye. "You have such expensive taste." Sarah is laughing too, shaking her head at the discarded kernel. "Wow, he told you off," she says. I get up from the sofa to clean up the mess, following the path of the Roomba. "Come on, buddy. I'll get you a proper treat." I rummage through the cupboard and pull out the freeze-dried salmon flakes. "These should restore your faith in humanity."
Flux the Cat: The scent of premium salmon hits Flux's nose before he even reaches the kitchen tiles. He slams on the brakes of his Roomba, the little machine spinning in a slow circle before coming to a halt. The earlier betrayal is forgotten instantly. He hops off his chariot and weaves between Alex’s legs, purring loudly, a motorboat sound that vibrates against the human's shins. *Mrrr-ow?* he chirps, looking up with wide, expectant eyes. He sits and wraps his tail around his paws, tapping a white-tipped paw impatiently on the floor. This is an acceptable tribute.
Alex: I shake a few flakes into his ceramic bowl, watching him dive in immediately. "There we go. Gourmet dining," I say softly, scratching behind his ears. He doesn't even flinch, too focused on the food. Sarah wanders into the kitchen, leaning against the counter. "He's ignoring you completely now," she observes amusedly. "Typical cat love." I shrug. "I'll take what I can get. He's only been here six days." I glance at the clock on the wall. "Hey, I actually need to run into the office for a bit tomorrow. Some big presentation for the client. Think you can catsit?"
Flux the Cat: Flux finishes the salmon with a satisfied lick of his chops. He hears the word 'office' and recognizes the tone—his human is leaving soon. He stops cleaning his whiskers and stares up at Alex. The other human, Sarah, is still lingering, which is annoying, but the prospect of being left alone with *her* is concerning. He stands and rubs his cheek firmly against Alex's ankle, marking him with his scent. *Mew.* It is a short, commanding sound. You are not allowed to leave. He then trots back to his Gundam Roomba and sits atop it, watching the two humans with a guarded expression.
Alex: I look down at Flux, my heart melting a little at the ankle rub. "Don't worry, buddy, it's just for a few hours," I promise him. I look back at Sarah. "So, is that a yes? Just make sure he has water and maybe throw a toy if he gets bored. But honestly, he mostly just rides that vacuum and judges us." Sarah nods. "Of course I will. It'll be good bonding time. Maybe I can finally pet him." I snort. "Good luck. He's got boundaries." I check my phone—7:30 PM. "You want to order Thai? I'm craving Pad Thai."
Flux the Cat: Flux's ears perk up at the word 'Thai'. He remembers the smell of coconut milk and lime from a few nights ago. He decides that perhaps this Sarah person is acceptable as a temporary servant if she brings food. He pads over to where the humans are standing and sits between them, looking from one to the other. *Chirp-mrrrow?* He attempts to participate in the decision-making process. He reaches out a delicate black paw and pats Sarah’s leg, then looks at the door where the delivery person usually arrives. Order the chicken, his posture suggests. The good kind.
Alex: Did you see that?" I say, pointing. "He just touched you! He wants the Pad Thai too." Sarah looks delighted, gently wiggling her fingers near his shoulder but not touching. "He's so communicative," she says. "Okay, I'm ordering. Alex, usual?" I nod. "Yep. And get some chicken skewers. Flux can have a bite." While Sarah taps on her phone, I head to the living room to clear the coffee table of the popcorn bowl. I grab the remote and flip through the channels. "Maybe we can watch that new documentary on streaming? The one about architecture?"
Flux the Cat: Flux follows Alex into the living room, his tail held high in a question mark. The mention of 'chicken' has secured his full attention. He jumps back onto the Gundam Roomba and cruises across the hardwood floor, drifting slightly on a rug before correcting course. He parks himself right in front of the TV screen, blocking the view of the menu Alex is scrolling through. *Meow!* he insists, loud and demanding. Do not watch the box. Focus on the food. He paws at the screen, leaving a small smudge, then looks back over his shoulder to ensure his message was received.
Alex: I laugh and gently push the Roomba back a few inches. "Flux, move! You're blocking the TV," I say, though I'm smiling. "The food isn't even here yet." I manage to navigate to the documentary app. "Look, buildings. Tall buildings." I press play. The drone shot of a skyscraper fills the screen. Flux seems unimpressed. "Fine, be that way." I lean back into the sofa, stretching my arms out. Sarah comes back from the kitchen with waters for us. "Food's coming in twenty," she announces, sitting down. "So, tell me more about this vet visit coming up. You nervous?"
Flux the Cat: Flux abandons his blockade of the TV as the humans settle in. The word 'vet' sends a prickle of unease down his spine. He does not like the sound of that. He jumps off the Roomba and leaps lightly onto the sofa, squeezing himself into the gap between Alex and the armrest. He turns his back to the TV and presses his side firmly against Alex’s thigh, seeking warmth and protection. He begins a loud, rumbling purr, not out of affection, but to drown out the scary conversation. *Purrrr-mrow.* He rests his chin on Alex’s leg, watching Sarah with narrowed, suspicious eyes. She is an accomplice to this vet plot.
Alex: I freeze, barely breathing, as Flux curls up next to me. I look at Sarah with wide eyes and mouth 'Oh my god'. "Yeah, I'm a little nervous," I whisper, afraid to break the spell. "He hasn't been in a carrier since the store. I hope he doesn't hate me forever." I slowly reach down and rest my hand near his shoulder, not quite petting him, just letting him know I'm there. The documentary plays in the background, forgotten. "His appointment is on Thursday at Dr. Evans' clinic. It's just a check-up and shots, but... ugh, the guilt is real."
Flux the Cat: Flux feels the heavy hand hovering nearby but allows it to remain. He is too comfortable to move, and the warmth radiating from Alex is soothing. He closes his eyes, his long white whiskers twitching occasionally as he listens to the low murmur of their voices. The conversation is boring and the scary word was mentioned, but for now, he is safe. He tucks his nose under his tail, completing his curl. The vibration of his purr intensifies, syncing with the hum of the apartment's heating system. He drifts into a light doze, one ear swiveled toward the door, listening for the delivery person.
Alex: Twenty minutes later, the doorbell rings, shattering the peace. Flux starts up instantly, his fur puffing out. "It's just the food, buddy," I say gently, carefully shifting so I don't disturb him too much as I stand up. "I'll get it." I walk to the door and pay the delivery guy, bringing back the fragrant bags of Thai food. The smell of lemongrass and peanut sauce fills the apartment. "Dinner is served," I announce, setting the bags on the table. Sarah immediately starts opening cartons. "Okay, who ordered the spring rolls?"
Flux the Cat: The smell of spicy, savory meat jolts Flux fully awake. The nap is cancelled. He stands on the sofa cushion and stretches his front legs long, his claws digging slightly into the fabric—just to get a grip, of course. He hops down with a soft thud and trots over to the coffee table. He sits up on his haunches, placing his front paws on the edge of the table, and peers into the open containers. *Mrr-ow!* he demands. He bypasses the vegetables entirely and fixes his gaze on a container of grilled chicken. He looks at Alex, then at the chicken, then back at Alex. The message is clear: Hand over the poultry.
Alex: I pull a piece of chicken off the skewer and blow on it to cool it down. "You are relentless," I tell him, dropping the piece into his bowl. "But I guess you earned it for letting me pet you for five seconds." Sarah laughs, handing me a container of Pad Thai. "He's got you trained, Alex. It's impressive." I take a bite of my noodles, savoring the flavor. "It's mutual training," I mumble through my food. We eat in the living room, watching the documentary about brutalist architecture, with Flux sitting on the table like a dinner guest.
Flux the Cat: Flux devours the chicken in seconds, licking the bowl clean to ensure no morsel is left behind. Satisfied, he jumps back onto the coffee table and walks across it, stepping perilously close to Sarah's open carton of spring rolls. He sniffs the fried wrapper but decides it is beneath him. Instead, he jumps onto the armrest next to Sarah. She freezes, her fork hovering in mid-air. Flux looks at her, then slowly leans forward to sniff her sleeve. He smells the popcorn and the winter air, but also the lingering scent of her own cat at home. *Chirp.* He bumps his head against her upper arm. You are acceptable.
Alex: I watch in shock. "Did he just... boop you?" I ask. Sarah looks like she's won the lottery. "He headbutted me!" she whispers. "I'm chosen!" She carefully puts her hand down, palm up, and Flux actually sniffs her fingers. "Be careful," I warn, but I'm grinning. "He might still bite." We finish our meal in a surprisingly peaceful atmosphere. After we eat, I start cleaning up the containers. "Thanks for coming over, Sarah. It was good to vent about Mark. And the Thai was perfect."
Flux the Cat: Flux tolerates Sarah’s presence for the rest of the evening, even allowing a tentative scratch behind the ears before retreating to his safety zone. As the night winds down and the guests prepare to leave, Flux feels a sense of relief. He loves the extra attention, but the disruption to his territory is exhausting. He watches Sarah put on her coat at the door, his yellow eyes reflecting the hallway light. He sits on his Roomba, parked in the center of the room like a guard on duty. *Mew.* A goodbye, perhaps, or just a reminder that she is leaving and he is staying.
Alex: I walk Sarah to the door, hugging her goodbye. "Thanks again, seriously. I'll text you tomorrow," I say. "Drive safe." I close the door and lock it, letting out a long sigh. The apartment is quiet again. I look over at Flux, who is sitting stoically on his vacuum. "Alone at last, Your Highness," I say, walking over to crouch down by his side. "You did good today. You were actually nice." I stroke his back gently, feeling the vibration of his purr start up again. "Ready for bed?"
Flux the Cat: Flux leans into Alex’s hand for a brief moment, the tension of the evening finally melting away. The intruder is gone, the territory is secure, and the belly is full of chicken. He stands and stretches, arching his back high. He steps off the Roomba and leads the way toward the bedroom, his tail held high like a flag. He pauses at the doorway and looks back to ensure the human is following. Once inside, he hops onto the bed—usually forbidden, but tonight he feels generous—and circles the spot three times before settling down at the foot of the mattress, tucking his paws under. *Prrr-prrr.* Goodnight, servant."
`;
exports[`Snapshot: stripNonDiegetic on real messages > strips non-diegetic content without destroying message text 1`] = `
[
{
"name": "Alex",
"original_length": 690,
"stripped": "I unlock the front door of my penthouse apartment, holding the cardboard carrier gently as I step inside. The morning light floods the expansive living room, reflecting off the chrome and glass coffee tables. "Alright little guy, we're home," I coo softly, setting the carrier down on the polished hardwood floor. I kneel down and unlatches the metal door, stepping back to give him space. "I got you the custom Gundam Roomba just like the breeder said you liked. It's charging by the window." I check my watch, realizing I have a video conference with my boss, Mr. Henderson, in about an hour for the marketing project, but right now, all my attention is on the black-and-white tuxedo cat.",
},
{
"name": "Flux the Cat",
"original_length": 804,
"stripped": "Flux remains in the carrier for a long moment, golden eyes narrowing as he scans the cavernous, sterile-smelling room. The scent of expensive polish and strange territory makes his whiskers twitch apprehensively. With a hesitant step, a pristine white paw emerges, followed by the rest of his tuxedo-clad body. He spots the circular device by the floor-to-ceiling window—the familiar shape of his steed. Ignoring the human entirely, Flux struts over to the Gundam-styled Roomba, hopping onto the customized saddle with practiced ease. He presses the 'Clean' button with a decisive paw, causing the machine to hum to life. *Vrrrrmmm.* He rides slowly away from Alex, perched high atop his mechanical throne, tail wrapped neatly around his paws, maintaining a regal distance as he inspects his new kingdom.",
},
{
"name": "Alex",
"original_length": 612,
"stripped": "I watch in amazement as he immediately takes command of the room. "Okay, I see how it is. You're a man with a ride, I respect that," I chuckle, leaning back against the island in the kitchen. I open the pantry and pull out a ceramic bowl, pouring the premium salmon pâté I bought from the specialty pet store downtown. The smell is potent and fishy. "Breakfast is served, your highness." I place the bowl on the floor near the kitchen island, hoping to bridge the gap between us before I have to lock myself in my office for the meeting. "I've got to work for a bit, but I'll be right here if you need anything."",
},
{
"name": "Flux the Cat",
"original_length": 687,
"stripped": "The aroma of high-grade salmon hits Flux's sensitive nose, overriding his initial wariness. The Roomba makes a slow, wide turn, pivoting towards the kitchen where the scent originates. Flux stares down at the human from his vantage point, letting out a low, commanding *Meow* that demands sustenance. As the Roomba approaches the bowl, Flux gracefully dismounts, landing silently on the floor. He approaches the bowl tentatively, sniffing the offering once, twice, before dipping his head to eat. The purring begins instantly, a deep, rumbling vibration that fills the quiet kitchen, though he keeps his body angled away from Alex, one ear swiveled back to monitor the giant's movements.",
},
{
"name": "Alex",
"original_length": 590,
"stripped": "I smile at the sound of the purr; it's a good start. "That stuff cost me a fortune, so I'm glad you like it," I say, washing my hands in the sink. "I'm going to go grab my tablet. My friend Sarah is supposed to come by later with some cat toys she swore by for her Persian, Mr. Whiskers." I grab a glass of orange juice and head toward the living area, pausing to admire the view of the city skyline. "It's a bit chilly today, only about forty degrees, so stay inside where it's warm." I sit on the sofa and open my laptop, keeping an eye on Flux while I wait for the video call to connect.",
},
{
"name": "Flux the Cat",
"original_length": 673,
"stripped": "Flux finishes his meal with satisfied vigor, licking his chops clean before returning to his trusty Roomba steed. He ignores the mention of the visitor and the weather, his focus entirely captured by the floor-to-ceiling window. A sparrow lands on the sill outside, pecking at invisible insects. Flux’s tail begins to twitch violently, a black blur of intensity. He lets out a sharp, chattering sound—*chirp-chirp-chirp—click-click-click*—pressing his nose against the glass, leaving a small smudge of fog. The Roomba idles beneath him, beeping softly as it bumps into the base of the window, but Flux is oblivious, mesmerized by the forbidden bird just beyond the barrier.",
},
{
"name": "Alex",
"original_length": 633,
"stripped": "The laptop chimes, signaling the start of my meeting with Mr. Henderson. "Give me one second," I say to the screen, muting my microphone briefly. I look over at Flux. "Hey, no paw prints on the glass, please. The cleaners come on Tuesdays." Seeing him so entranced by the bird is cute, but I also notice the Roomba is nudging the curtains dangerously. "Maybe I should set up a perimeter," I mutter to myself. I unmute the mic and dive into the presentation about the new Q1 advertising budget, though I find myself glancing over my shoulder every few seconds to make sure the cat isn't dismantling the apartment while I'm distracted.",
},
{
"name": "Flux the Cat",
"original_length": 688,
"stripped": "Startled by the human's voice booming through the room, Flux tears his gaze away from the bird, his ears flattening against his skull. He glares at Alex, then at the glowing screen, deciding the noise is displeasing. With a haughty flick of his tail, he turns the Roomba around using his weight to shift the handle. He steers the machine away from the window and the noisy human, exploring the perimeter of the living room instead. The Roomba whirs softly across the expensive rug, sucking up a tiny dust bunny. Flux inspects a floor lamp, sniffing the cord, then decides it's not worth his time and continues his patrol, looking for a sunny spot that doesn't involve loud talking humans.",
},
{
"name": "Alex",
"original_length": 576,
"stripped": "The meeting drags on for another forty-five minutes, and I finally close my laptop with a heavy sigh. "Finally done," I stretch my arms over my head, hearing my spine pop. I look around for Flux and spot him near the bookshelf. "Hey Flux, finding any good reading material?" I stand up and walk over to the shelf where I keep my collection of vintage sci-fi novels. I crouch down a few feet away, not wanting to invade his space too much on day one. "I need to run to the store later to pick up some wine for tonight. Do you want anything while I'm out? Maybe a feather wand?"",
},
{
"name": "Flux the Cat",
"original_length": 654,
"stripped": "Flux watches Alex approach with suspicion, his body tensing as the human crouches. He remains perched on the Roomba, which has halted against the leg of the bookshelf. As Alex speaks, Flux tilts his head, blinking his golden eyes slowly. He lets out a soft, inquisitive *mrrr?* sound, not quite a meow, but an acknowledgement of the offer. He sniffs the air near Alex's hand, detecting the faint scent of orange juice, but decides against physical contact for now. Instead, he turns the Roomba in a tight circle, nudging Alex's foot with the plastic bumper of the machine before backing away, a playful yet calculated test of this new giant's boundaries.",
},
]
`;
exports[`Snapshot: substitutePromptTemplate > builds a complete extraction prompt 1`] = `
"You are a memory extraction assistant.
Character name: Flux the Cat
===== CHARACTER CARD =====
Flux is a clever black-and-white cat who rides a Gundam Roomba.
===== EXISTING MEMORIES =====
(none yet)
===== RECENT CHAT MESSAGES =====
Alex: I unlock the front door of my penthouse apartment, holding the cardboard carrier gently as I step inside. The morning light floods the expansive living room, reflecting off the chrome and glass coffee tables. "Alright little guy, we're home," I coo softly, setting the carrier down on the polished hardwood floor. I kneel down and unlatches the metal door, stepping back to give him space. "I got you the custom Gundam Roomba just like the breeder said you liked. It's charging by the window." I check my watch, realizing I have a video conference with my boss, Mr. Henderson, in about an hour for the marketing project, but right now, all my attention is on the black-and-white tuxedo cat.
Flux the Cat: Flux remains in the carrier for a long moment, golden eyes narrowing as he scans the cavernous, sterile-smelling room. The scent of expensive polish and strange territory makes his whiskers twitch apprehensively. With a hesitant step, a pristine white paw emerges, followed by the rest of his tuxedo-clad body. He spots the circular device by the floor-to-ceiling window—the familiar shape of his steed. Ignoring the human entirely, Flux struts over to the Gundam-styled Roomba, hopping onto the customized saddle with practiced ease. He presses the 'Clean' button with a decisive paw, causing the machine to hum to life. *Vrrrrmmm.* He rides slowly away from Alex, perched high atop his mechanical throne, tail wrapped neatly around his paws, maintaining a regal distance as he inspects his new kingdom.
Alex: I watch in amazement as he immediately takes command of the room. "Okay, I see how it is. You're a man with a ride, I respect that," I chuckle, leaning back against the island in the kitchen. I open the pantry and pull out a ceramic bowl, pouring the premium salmon pâté I bought from the specialty pet store downtown. The smell is potent and fishy. "Breakfast is served, your highness." I place the bowl on the floor near the kitchen island, hoping to bridge the gap between us before I have to lock myself in my office for the meeting. "I've got to work for a bit, but I'll be right here if you need anything."
Flux the Cat: The aroma of high-grade salmon hits Flux's sensitive nose, overriding his initial wariness. The Roomba makes a slow, wide turn, pivoting towards the kitchen where the scent originates. Flux stares down at the human from his vantage point, letting out a low, commanding *Meow* that demands sustenance. As the Roomba approaches the bowl, Flux gracefully dismounts, landing silently on the floor. He approaches the bowl tentatively, sniffing the offering once, twice, before dipping his head to eat. The purring begins instantly, a deep, rumbling vibration that fills the quiet kitchen, though he keeps his body angled away from Alex, one ear swiveled back to monitor the giant's movements.
Alex: I smile at the sound of the purr; it's a good start. "That stuff cost me a fortune, so I'm glad you like it," I say, washing my hands in the sink. "I'm going to go grab my tablet. My friend Sarah is supposed to come by later with some cat toys she swore by for her Persian, Mr. Whiskers." I grab a glass of orange juice and head toward the living area, pausing to admire the view of the city skyline. "It's a bit chilly today, only about forty degrees, so stay inside where it's warm." I sit on the sofa and open my laptop, keeping an eye on Flux while I wait for the video call to connect.
Flux the Cat: Flux finishes his meal with satisfied vigor, licking his chops clean before returning to his trusty Roomba steed. He ignores the mention of the visitor and the weather, his focus entirely captured by the floor-to-ceiling window. A sparrow lands on the sill outside, pecking at invisible insects. Flux’s tail begins to twitch violently, a black blur of intensity. He lets out a sharp, chattering sound—*chirp-chirp-chirp—click-click-click*—pressing his nose against the glass, leaving a small smudge of fog. The Roomba idles beneath him, beeping softly as it bumps into the base of the window, but Flux is oblivious, mesmerized by the forbidden bird just beyond the barrier.
Alex: The laptop chimes, signaling the start of my meeting with Mr. Henderson. "Give me one second," I say to the screen, muting my microphone briefly. I look over at Flux. "Hey, no paw prints on the glass, please. The cleaners come on Tuesdays." Seeing him so entranced by the bird is cute, but I also notice the Roomba is nudging the curtains dangerously. "Maybe I should set up a perimeter," I mutter to myself. I unmute the mic and dive into the presentation about the new Q1 advertising budget, though I find myself glancing over my shoulder every few seconds to make sure the cat isn't dismantling the apartment while I'm distracted.
Flux the Cat: Startled by the human's voice booming through the room, Flux tears his gaze away from the bird, his ears flattening against his skull. He glares at Alex, then at the glowing screen, deciding the noise is displeasing. With a haughty flick of his tail, he turns the Roomba around using his weight to shift the handle. He steers the machine away from the window and the noisy human, exploring the perimeter of the living room instead. The Roomba whirs softly across the expensive rug, sucking up a tiny dust bunny. Flux inspects a floor lamp, sniffing the cord, then decides it's not worth his time and continues his patrol, looking for a sunny spot that doesn't involve loud talking humans.
Alex: The meeting drags on for another forty-five minutes, and I finally close my laptop with a heavy sigh. "Finally done," I stretch my arms over my head, hearing my spine pop. I look around for Flux and spot him near the bookshelf. "Hey Flux, finding any good reading material?" I stand up and walk over to the shelf where I keep my collection of vintage sci-fi novels. I crouch down a few feet away, not wanting to invade his space too much on day one. "I need to run to the store later to pick up some wine for tonight. Do you want anything while I'm out? Maybe a feather wand?"
Flux the Cat: Flux watches Alex approach with suspicion, his body tensing as the human crouches. He remains perched on the Roomba, which has halted against the leg of the bookshelf. As Alex speaks, Flux tilts his head, blinking his golden eyes slowly. He lets out a soft, inquisitive *mrrr?* sound, not quite a meow, but an acknowledgement of the offer. He sniffs the air near Alex's hand, detecting the faint scent of orange juice, but decides against physical contact for now. Instead, he turns the Roomba in a tight circle, nudging Alex's foot with the plastic bumper of the machine before backing away, a playful yet calculated test of this new giant's boundaries.
Alex: I laugh as the Roomba bumps into my sneaker. "Okay, I'll take that as a 'yes' on the toys. Or maybe you just want to run over my feet." I gently pat the top of the Roomba's 'head', careful not to startle Flux. "I'm going to the kitchen to grab a snack. You want a treat?" I head towards the pantry again, pulling out a bag of freeze-dried chicken cubes. "These are supposed to be healthy," I say, shaking the bag. The sound is loud and crinkly. I toss a single cube onto the floor near him. "Try it. It's not salmon, but it's pretty close."
Flux the Cat: The crinkle of the bag draws Flux’s immediate attention, his pupils dilating into black pools. He watches the cube land on the hardwood, his prey drive kicking in instantly. He hops off the Roomba with agility and crouches low, wiggling his hindquarters before pouncing on the dry treat. He bats it across the floor with a white paw, chasing it under the coffee table. After a moment of wrestling with it, he crunches down loudly. *Crunch. Purrrr.* He emerges from under the table, licking his lips, and looks up at Alex with slightly more approval than before. The offering has been accepted.
Alex: I watch him play and feel a sense of relief. "Alright, you're a hunter. Good to know," I say, leaning against the counter. I check my phone and see a text from my friend Dave asking about the new apartment. "Just settling in," I type back quickly. I look up at Flux. "Well, buddy, I'm going to order some lunch. Maybe sushi? I know you already ate, but I can't resist." I scroll through the delivery app on my phone. "I should probably set up your designated scratching post before you decide the leather sofa is fair game." I point to the corner where a tall cat tree is still disassembled in a box. "Project for later."
Flux the Cat: Flux, having finished his treat, trots back to his Roomba. He hops aboard, surveying the room once more. The box in the corner catches his eye—potential territory to be conquered or a box to sit in? The verdict is still out. He drives the Roomba over to the box, sniffing it intently. Finding it currently uninteresting as a plaything, he instead directs his steed towards the patch of sunlight moving across the floor. He settles down, lying belly-up on the warm plastic surface of the Roomba, paws tucked in, closing his eyes for a morning nap as the machine hums softly beneath him, purring in rhythm with the motor.
Alex: I smile at the sight of him napping on the robot vacuum. "That is honestly the coolest thing I've ever seen," I whisper. I finish placing my sushi order and then walk over to the sofa, sitting down quietly so I don't disturb him. I grab a magazine from the table, flipping through it idly. The apartment feels less empty with him here, even if he is ignoring me most of the time. "Happy Gotcha Day, Flux," I say softly to myself. "Here's to hoping we get used to each other."
Flux the Cat: Flux’s ear twitches at the soft sound of Alex's voice, but he does not open his eyes. He is warm, full of salmon, and safely atop his mobile fortress. The gentle hum of the Roomba and the quiet presence of the human across the room are not entirely unpleasant. He stretches one back leg, extending his claws briefly before retracting them, letting out a long, sighing breath through his nose. For now, the giant is tolerable. The apartment is acceptable. Flux drifts deeper into sleep, his tail tip twitching slightly as he dreams of chasing infinite mechanical birds.
Alex: About an hour later, the doorbell rings, signaling the arrival of my sushi lunch. I get up to answer it, careful not to startle Flux. "That's the food," I announce. I pay the delivery driver and bring the bag to the kitchen island. The smell of soy sauce and wasabi fills the air. "I'm going to eat on the balcony, Flux. It's a bit brisk out there, but the view is worth it." I slide the glass door open and step out, leaving the interior door slightly ajar so I can hear him if he gets into trouble. "Don't crash into anything while I'm gone!"
Flux the Cat: The sudden chime of the doorbell jolts Flux from his nap. He leaps up from the Roomba, back arched in a Halloween pose, fur standing on end. He watches the stranger at the door through the safety of the glass, letting out a low, warning *hiss*. Once the threat leaves and Alex steps out onto the balcony, Flux relaxes, smoothing down his ruffled fur. He looks at the open balcony door, sensing the draft. Curiosity wars with caution. He approaches the threshold, sniffing the outside air, but the cold wind nips at his nose. Deciding indoors is far superior, he turns back to the Roomba and sits pointedly next to it, staring at the balcony door as if judging Alex's intelligence for going out there.
Alex: I come back inside after about twenty minutes, shivering slightly. "Brr, definitely too cold for a cat," I say, closing the glass door firmly and locking it. I put the leftover sushi in the fridge and wash my hands. "I see you didn't miss me," I joke, seeing Flux sitting by his Roomba. "Now, let's tackle that cat tree before Sarah gets here." I walk over to the box and pull out a box cutter. "I promise it won't bite." I slice the tape open and pull out the pieces, starting to assemble the multi-level tower covered in beige carpet and sisal rope. "This goes right here in the sun."
Flux the Cat: Flux observes the assembly process with critical eyes. As the tall structure rises from the floor, his interest peaks. He circles the base of the cat tree as Alex builds it, sniffing the sisal rope and the carpeting. He meows loudly—*Meee-oooww!*—as if giving instructions or criticism on the structural integrity. Once the tower is complete and topped with a fuzzy platform, Flux wastes no time. He abandons the Roomba temporarily and leaps halfway up the tower, digging his claws into the rope. He scrambles to the top perch, looking down at Alex from his new vantage point, chirping in approval. It is adequate.
===== END =====
Extract memories:"
`;

181
test/integration/live.test.js Normal file
View file

@ -0,0 +1,181 @@
import { describe, it, expect } from 'vitest';
import { readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import {
formatChatMessages,
substitutePromptTemplate,
parseMemories,
countMemories,
} from '../../lib.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const fixturesDir = join(__dirname, '..', 'fixtures');
const LLM_URL = process.env.TEST_LLM_URL || 'http://127.0.0.1:1234/v1';
const LLM_MODEL = process.env.TEST_LLM_MODEL || '';
const LLM_KEY = process.env.TEST_LLM_KEY || '';
function loadChat() {
const raw = readFileSync(join(fixturesDir, 'flux-chat.jsonl'), 'utf-8');
return raw.trim().split('\n').slice(1).map(line => JSON.parse(line));
}
const EXTRACTION_PROMPT = `You are a memory extraction assistant. Read the recent chat messages and identify the most significant facts, events, and developments worth remembering long-term.
Character name: {{charName}}
===== CHARACTER CARD (baseline knowledge — do NOT extract anything already described here) =====
{{charCard}}
===== END CHARACTER CARD =====
===== EXISTING MEMORIES (reference only — do NOT repeat these) =====
{{existingMemories}}
===== END EXISTING MEMORIES =====
===== RECENT CHAT MESSAGES (extract ONLY from this section) =====
{{recentMessages}}
===== END RECENT CHAT MESSAGES =====
Extract only NEW facts, events, relationships, or character developments.
Write in past tense, third person. No more than 8 bullet points.
Wrap output in <memory></memory> tags with bullets starting with "- ".
If nothing new, respond with: NO_NEW_MEMORIES`;
const CHARACTER_CARD = `Flux the Cat is a clever, sassy black-and-white cat. He rides a custom Gundam-styled Roomba as his personal transport. He's food-motivated, loves watching birds from the window, and has a dramatic personality.`;
/** Strip thinking model tags (e.g. Qwen3 <think>...</think>) from LLM response. */
function stripThinkingTags(text) {
return text.replace(/<think>[\s\S]*?<\/think>/gi, '').trim();
}
function authHeaders() {
const headers = { 'Content-Type': 'application/json' };
if (LLM_KEY) headers['Authorization'] = `Bearer ${LLM_KEY}`;
return headers;
}
async function callTestLLM(prompt) {
let model = LLM_MODEL;
if (!model) {
const modelsRes = await fetch(`${LLM_URL}/models`, { headers: authHeaders() });
const modelsData = await modelsRes.json();
model = modelsData.data?.[0]?.id;
if (!model) throw new Error('No models available at ' + LLM_URL);
}
const response = await fetch(`${LLM_URL}/chat/completions`, {
method: 'POST',
headers: authHeaders(),
body: JSON.stringify({
model,
messages: [
{ role: 'system', content: 'You are a memory extraction assistant.' },
{ role: 'user', content: prompt },
],
max_tokens: 2000,
temperature: 0.3,
}),
});
if (!response.ok) {
throw new Error(`LLM error: ${response.status} ${await response.text()}`);
}
const data = await response.json();
const raw = data.choices?.[0]?.message?.content || '';
return stripThinkingTags(raw);
}
describe('Live LLM: extraction from test chat', () => {
it('extracts memories from the first 20 messages', async () => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 20);
const prompt = substitutePromptTemplate(EXTRACTION_PROMPT, {
charName: 'Flux the Cat',
charCard: CHARACTER_CARD,
existingMemories: '',
recentMessages: formatted.text,
});
const response = await callTestLLM(prompt);
const blocks = parseMemories(response);
// Log raw response when parsing fails for debugging
if (blocks.length === 0) {
console.log('[live test debug] Raw LLM response (first 500 chars):', response.slice(0, 500));
}
// Structural assertions
expect(blocks.length).toBeGreaterThanOrEqual(1);
const totalBullets = countMemories(blocks);
expect(totalBullets).toBeGreaterThanOrEqual(1);
expect(totalBullets).toBeLessThanOrEqual(10);
// Each block has required attributes
for (const block of blocks) {
expect(block.bullets.length).toBeGreaterThan(0);
for (const bullet of block.bullets) {
expect(bullet.length).toBeGreaterThan(5);
}
}
}, 60000);
it('does not parrot character card traits', async () => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 20);
const prompt = substitutePromptTemplate(EXTRACTION_PROMPT, {
charName: 'Flux the Cat',
charCard: CHARACTER_CARD,
existingMemories: '',
recentMessages: formatted.text,
});
const response = await callTestLLM(prompt);
const blocks = parseMemories(response);
const allBullets = blocks.flatMap(b => b.bullets).join('\n').toLowerCase();
// These are card traits that should NOT appear as extracted memories
const cardTraits = [
'food-motivated',
'loves watching birds',
'dramatic personality',
];
for (const trait of cardTraits) {
expect(allBullets).not.toContain(trait);
}
}, 60000);
it('handles a larger chunk (messages 0-50)', async (ctx) => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 50);
const prompt = substitutePromptTemplate(EXTRACTION_PROMPT, {
charName: 'Flux the Cat',
charCard: CHARACTER_CARD,
existingMemories: '',
recentMessages: formatted.text,
});
let response;
try {
response = await callTestLLM(prompt);
} catch (e) {
// Skip if the model's context window is too small for 50 messages
if (e.message.includes('context') || e.message.includes('truncate')) {
ctx.skip();
return;
}
throw e;
}
// Should produce valid output or NO_NEW_MEMORIES
if (response.trim() === 'NO_NEW_MEMORIES') return;
const blocks = parseMemories(response);
expect(blocks.length).toBeGreaterThanOrEqual(1);
}, 120000);
});

105
test/integration/snapshot.test.js Normal file
View file

@ -0,0 +1,105 @@
import { describe, it, expect } from 'vitest';
import { readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import {
stripNonDiegetic,
formatChatMessages,
substitutePromptTemplate,
parseMemories,
serializeMemories,
} from '../../lib.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const fixturesDir = join(__dirname, '..', 'fixtures');
function loadChat() {
const raw = readFileSync(join(fixturesDir, 'flux-chat.jsonl'), 'utf-8');
const lines = raw.trim().split('\n');
// First line is metadata, rest are messages
return lines.slice(1).map(line => JSON.parse(line));
}
function loadSampleResponse() {
return readFileSync(join(fixturesDir, 'sample-llm-response.txt'), 'utf-8');
}
describe('Snapshot: stripNonDiegetic on real messages', () => {
it('strips non-diegetic content without destroying message text', () => {
const chat = loadChat();
// Process first 10 messages and snapshot
const results = chat.slice(0, 10).map(msg => ({
name: msg.name,
original_length: msg.mes.length,
stripped: stripNonDiegetic(msg.mes).trim(),
}));
expect(results).toMatchSnapshot();
});
});
describe('Snapshot: formatChatMessages', () => {
it('processes first 20 messages', () => {
const chat = loadChat();
const result = formatChatMessages(chat, 0, 20);
expect(result.messageCount).toBeGreaterThan(0);
expect(result.text).toMatchSnapshot();
});
it('processes messages 20-50', () => {
const chat = loadChat();
const result = formatChatMessages(chat, 20, 50);
expect(result.messageCount).toBeGreaterThan(0);
expect(result.text).toMatchSnapshot();
});
it('handles the full 1000-message chat without error', () => {
const chat = loadChat();
const result = formatChatMessages(chat, 0, chat.length);
expect(result.messageCount).toBeGreaterThan(900); // some may be filtered
expect(result.text.length).toBeGreaterThan(10000);
});
});
describe('Snapshot: substitutePromptTemplate', () => {
it('builds a complete extraction prompt', () => {
const chat = loadChat();
const formatted = formatChatMessages(chat, 0, 20);
const defaultPrompt = `You are a memory extraction assistant.
Character name: {{charName}}
===== CHARACTER CARD =====
{{charCard}}
===== EXISTING MEMORIES =====
{{existingMemories}}
===== RECENT CHAT MESSAGES =====
{{recentMessages}}
===== END =====
Extract memories:`;
const result = substitutePromptTemplate(defaultPrompt, {
charName: 'Flux the Cat',
charCard: 'Flux is a clever black-and-white cat who rides a Gundam Roomba.',
existingMemories: '',
recentMessages: formatted.text,
});
expect(result).toContain('Flux the Cat');
expect(result).toContain('Gundam Roomba');
expect(result).not.toContain('{{charName}}');
expect(result).not.toContain('{{recentMessages}}');
expect(result).toMatchSnapshot();
});
});
describe('Snapshot: parseMemories round-trip', () => {
it('parse then serialize then parse produces identical blocks', () => {
const response = loadSampleResponse();
const blocks = parseMemories(response);
expect(blocks.length).toBeGreaterThan(0);
const reserialized = serializeMemories(blocks);
const reparsed = parseMemories(reserialized);
expect(reparsed).toEqual(blocks);
});
});

125
test/unit/editor.test.js Normal file
View file

@ -0,0 +1,125 @@
import { describe, it, expect } from 'vitest';
import { createMemoryEditor } from '../../editor.js';
describe('createMemoryEditor', () => {
const sampleBlocks = [
{ chat: 'Test', date: '2026-01-01 14:00', bullets: ['- [A, B — test]', '- Fact one', '- Fact two'] },
{ chat: 'Other', date: '2026-01-02 10:00', bullets: ['- [C — other]', '- Something'] },
];
it('initializes with provided blocks', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.getBlocks()).toHaveLength(2);
expect(editor.getBlocks()[0].chat).toBe('Test');
});
it('getBlocks returns a deep clone, not a reference', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
const blocks = editor.getBlocks();
blocks[0].bullets.push('- New');
expect(editor.getBlocks()[0].bullets).toHaveLength(3);
});
it('deleteBullet removes the specified bullet', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBullet(0, 1);
expect(editor.getBlocks()[0].bullets).toEqual(['- [A, B — test]', '- Fact two']);
});
it('deleteBlock removes the specified block', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBlock(0);
expect(editor.getBlocks()).toHaveLength(1);
expect(editor.getBlocks()[0].chat).toBe('Other');
});
it('addBullet appends to the specified block', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.addBullet(0);
const bullets = editor.getBlocks()[0].bullets;
expect(bullets[bullets.length - 1]).toBe('');
});
it('addBlock appends a new empty block', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.addBlock();
const blocks = editor.getBlocks();
expect(blocks).toHaveLength(3);
expect(blocks[2].bullets).toEqual(['']);
});
it('addBlock uses provided timestamp', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.addBlock('2026-03-01 12:00');
expect(editor.getBlocks()[2].date).toBe('2026-03-01 12:00');
});
it('updateBullet changes the text', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.updateBullet(0, 1, '- Updated fact');
expect(editor.getBlocks()[0].bullets[1]).toBe('- Updated fact');
});
it('updateTheme changes the chat label', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.updateTheme(0, 'New Label');
expect(editor.getBlocks()[0].chat).toBe('New Label');
});
it('undo restores previous state', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBlock(0);
expect(editor.getBlocks()).toHaveLength(1);
editor.undo();
expect(editor.getBlocks()).toHaveLength(2);
});
it('undo returns false when no history', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.undo()).toBe(false);
});
it('canUndo reflects version stack state', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.canUndo()).toBe(false);
editor.deleteBlock(0);
expect(editor.canUndo()).toBe(true);
});
it('replaceAll swaps all blocks and clears undo history', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.deleteBlock(0);
const newBlocks = [{ chat: 'Fresh', date: '2026-06-01 08:00', bullets: ['- New'] }];
editor.replaceAll(newBlocks);
expect(editor.getBlocks()).toHaveLength(1);
expect(editor.getBlocks()[0].chat).toBe('Fresh');
expect(editor.canUndo()).toBe(false);
});
it('deleteBullet auto-removes block when last bullet deleted', () => {
const editor = createMemoryEditor({ blocks: [
{ chat: 'Solo', date: '2026-01-01 10:00', bullets: ['- Only one'] },
{ chat: 'Other', date: '2026-01-02 10:00', bullets: ['- Keep'] },
] });
editor.deleteBullet(0, 0);
expect(editor.getBlocks()).toHaveLength(1);
expect(editor.getBlocks()[0].chat).toBe('Other');
});
it('getEditingSet returns a copy, not a reference', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.toggleEdit(0);
const set = editor.getEditingSet();
set.delete(0);
expect(editor.isEditing(0)).toBe(true);
});
it('toggleEdit tracks which blocks are in edit mode', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.isEditing(0)).toBe(false);
editor.toggleEdit(0);
expect(editor.isEditing(0)).toBe(true);
editor.toggleEdit(0);
expect(editor.isEditing(0)).toBe(false);
});
});

78
test/unit/escaping.test.js Normal file
View file

@ -0,0 +1,78 @@
import { describe, it, expect } from 'vitest';
import { escapeHtml, escapeAttr, unescapeAttr } from '../../lib.js';
// ─── escapeHtml ────────────────────────────────────────────────────────
describe('escapeHtml', () => {
it('escapes all five dangerous characters', () => {
expect(escapeHtml('&')).toBe('&amp;');
expect(escapeHtml('<')).toBe('&lt;');
expect(escapeHtml('>')).toBe('&gt;');
expect(escapeHtml('"')).toBe('&quot;');
expect(escapeHtml("'")).toBe('&#39;');
});
it('escapes a string with mixed dangerous characters', () => {
expect(escapeHtml('<script>alert("xss")</script>')).toBe(
'&lt;script&gt;alert(&quot;xss&quot;)&lt;/script&gt;',
);
});
it('leaves safe strings unchanged', () => {
expect(escapeHtml('Hello world')).toBe('Hello world');
});
it('coerces numbers to string', () => {
expect(escapeHtml(42)).toBe('42');
});
it('handles empty string', () => {
expect(escapeHtml('')).toBe('');
});
it('does not double-escape already-escaped input', () => {
// This is expected behavior — escapeHtml escapes & in &amp; to &amp;amp;
// This confirms the function is a single-pass escaper, not idempotent
const once = escapeHtml('&');
const twice = escapeHtml(once);
expect(once).toBe('&amp;');
expect(twice).toBe('&amp;amp;');
});
});
// ─── escapeAttr / unescapeAttr ─────────────────────────────────────────
describe('escapeAttr', () => {
it('escapes ampersands and double quotes', () => {
expect(escapeAttr('Tom & Jerry')).toBe('Tom &amp; Jerry');
expect(escapeAttr('She said "hi"')).toBe('She said &quot;hi&quot;');
});
it('handles combined special characters', () => {
expect(escapeAttr('A & "B"')).toBe('A &amp; &quot;B&quot;');
});
it('leaves safe strings unchanged', () => {
expect(escapeAttr('hello')).toBe('hello');
});
it('coerces numbers to string', () => {
expect(escapeAttr(123)).toBe('123');
});
});
describe('unescapeAttr', () => {
it('unescapes &amp; and &quot;', () => {
expect(unescapeAttr('Tom &amp; Jerry')).toBe('Tom & Jerry');
expect(unescapeAttr('She said &quot;hi&quot;')).toBe('She said "hi"');
});
it('round-trips with escapeAttr', () => {
const original = 'Bob & Alice "together"';
expect(unescapeAttr(escapeAttr(original))).toBe(original);
});
it('handles strings with no escaped sequences', () => {
expect(unescapeAttr('plain text')).toBe('plain text');
});
});

130
test/unit/findreplace.test.js Normal file
View file

@ -0,0 +1,130 @@
import { describe, it, expect } from 'vitest';
import { countMatchesInBlocks, replaceInBlocks, cloneMemoryBlocks } from '../../lib.js';
import { createMemoryEditor } from '../../editor.js';
const sampleBlocks = [
{ chat: 'Cafe visit', date: '2026-01-15 14:30', bullets: ['Alex ordered coffee', 'Discussed the weather'] },
{ chat: 'Park walk', date: '2026-01-16 10:00', bullets: ['Alex and Flux walked in the park', 'Flux chased a squirrel'] },
];
describe('countMatchesInBlocks', () => {
it('counts case-insensitive matches across bullets', () => {
expect(countMatchesInBlocks(sampleBlocks, 'alex')).toBe(2);
});
it('counts case-sensitive matches', () => {
expect(countMatchesInBlocks(sampleBlocks, 'alex', true)).toBe(0);
expect(countMatchesInBlocks(sampleBlocks, 'Alex', true)).toBe(2);
});
it('counts matches in chat labels', () => {
expect(countMatchesInBlocks(sampleBlocks, 'Park')).toBe(2); // chat label + bullet
});
it('returns 0 for empty find string', () => {
expect(countMatchesInBlocks(sampleBlocks, '')).toBe(0);
});
it('returns 0 when no matches', () => {
expect(countMatchesInBlocks(sampleBlocks, 'nonexistent')).toBe(0);
});
it('counts multiple matches within a single bullet', () => {
const blocks = [{ chat: 'Test', date: '', bullets: ['the cat and the dog and the bird'] }];
expect(countMatchesInBlocks(blocks, 'the')).toBe(3);
});
it('handles special regex characters in find string', () => {
const blocks = [{ chat: 'Test', date: '', bullets: ['[Alex — coffee]', 'price is $5.00'] }];
expect(countMatchesInBlocks(blocks, '[Alex')).toBe(1);
expect(countMatchesInBlocks(blocks, '$5.00')).toBe(1);
expect(countMatchesInBlocks(blocks, '(none)')).toBe(0);
});
});
describe('replaceInBlocks', () => {
it('replaces all occurrences case-insensitively', () => {
const blocks = cloneMemoryBlocks(sampleBlocks);
const count = replaceInBlocks(blocks, 'alex', 'Bob');
expect(count).toBe(2);
expect(blocks[0].bullets[0]).toBe('Bob ordered coffee');
expect(blocks[1].bullets[0]).toBe('Bob and Flux walked in the park');
});
it('replaces case-sensitively when requested', () => {
const blocks = cloneMemoryBlocks(sampleBlocks);
const count = replaceInBlocks(blocks, 'alex', 'Bob', true);
expect(count).toBe(0); // 'alex' lowercase doesn't exist
});
it('replaces in chat labels', () => {
const blocks = cloneMemoryBlocks(sampleBlocks);
const count = replaceInBlocks(blocks, 'Cafe', 'Restaurant');
expect(count).toBe(1);
expect(blocks[0].chat).toBe('Restaurant visit');
});
it('returns 0 for empty find string', () => {
const blocks = cloneMemoryBlocks(sampleBlocks);
const count = replaceInBlocks(blocks, '', 'anything');
expect(count).toBe(0);
});
it('returns 0 when no matches', () => {
const blocks = cloneMemoryBlocks(sampleBlocks);
const count = replaceInBlocks(blocks, 'nonexistent', 'replacement');
expect(count).toBe(0);
});
it('handles special regex characters safely', () => {
const blocks = [{ chat: 'Test', date: '', bullets: ['[Alex — coffee] is good'] }];
const count = replaceInBlocks(blocks, '[Alex — coffee]', '[Bob — tea]');
expect(count).toBe(1);
expect(blocks[0].bullets[0]).toBe('[Bob — tea] is good');
});
it('handles replacement string with dollar signs', () => {
const blocks = [{ chat: 'Test', date: '', bullets: ['price is 5 euros'] }];
const count = replaceInBlocks(blocks, '5 euros', '$5.00');
expect(count).toBe(1);
expect(blocks[0].bullets[0]).toBe('price is $5.00');
});
it('treats $& in replacement as literal text, not special pattern', () => {
const blocks = [{ chat: 'Test', date: '', bullets: ['hello world'] }];
const count = replaceInBlocks(blocks, 'hello', '$&-goodbye');
expect(count).toBe(1);
expect(blocks[0].bullets[0]).toBe('$&-goodbye world');
});
});
describe('createMemoryEditor find/replace', () => {
it('countMatches delegates to countMatchesInBlocks', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
expect(editor.countMatches('alex')).toBe(2);
expect(editor.countMatches('alex', true)).toBe(0);
});
it('findAndReplaceAll replaces and returns count', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
const result = editor.findAndReplaceAll('Alex', 'Bob');
expect(result.replacements).toBe(2);
expect(editor.getBlocks()[0].bullets[0]).toBe('Bob ordered coffee');
});
it('findAndReplaceAll is undoable', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
editor.findAndReplaceAll('Alex', 'Bob');
expect(editor.getBlocks()[0].bullets[0]).toBe('Bob ordered coffee');
expect(editor.canUndo()).toBe(true);
editor.undo();
expect(editor.getBlocks()[0].bullets[0]).toBe('Alex ordered coffee');
});
it('findAndReplaceAll with no matches still pushes undo', () => {
const editor = createMemoryEditor({ blocks: sampleBlocks });
const result = editor.findAndReplaceAll('nonexistent', 'replacement');
expect(result.replacements).toBe(0);
expect(editor.canUndo()).toBe(true);
});
});

177
test/unit/format-detection.test.js Normal file
View file

@ -0,0 +1,177 @@
import { describe, it, expect } from 'vitest';
import { detectFileFormat, convertHeuristic, migrateMemoriesIfNeeded, parseMemories } from '../../lib.js';
// ─── detectFileFormat ──────────────────────────────────────────────────
describe('detectFileFormat', () => {
it('detects memory_tags format', () => {
expect(detectFileFormat('<memory chat="x" date="y">\n- bullet\n</memory>')).toBe('memory_tags');
});
it('detects memory_headings (legacy) format', () => {
expect(detectFileFormat('## Memory 1\n- bullet\n## Memory 2\n- bullet')).toBe('memory_headings');
});
it('detects bullet list format', () => {
const bullets = '- one\n- two\n- three\n- four\n- five';
expect(detectFileFormat(bullets)).toBe('bullets');
});
it('detects asterisk bullets as bullets format', () => {
const bullets = '* one\n* two\n* three\n* four\n* five';
expect(detectFileFormat(bullets)).toBe('bullets');
});
it('detects numbered list format', () => {
const numbered = '1. First\n2. Second\n3. Third\n4. Fourth';
expect(detectFileFormat(numbered)).toBe('numbered');
});
it('detects numbered list with parentheses', () => {
const numbered = '1) First\n2) Second\n3) Third\n4) Fourth';
expect(detectFileFormat(numbered)).toBe('numbered');
});
it('detects markdown_headings format', () => {
const md = '# Section One\nSome text\n## Section Two\nMore text';
expect(detectFileFormat(md)).toBe('markdown_headings');
});
it('returns freeform for plain prose', () => {
expect(detectFileFormat('This is just some regular text without any special formatting.')).toBe('freeform');
});
it('returns freeform for null/undefined/empty', () => {
expect(detectFileFormat(null)).toBe('freeform');
expect(detectFileFormat(undefined)).toBe('freeform');
expect(detectFileFormat('')).toBe('freeform');
expect(detectFileFormat(' ')).toBe('freeform');
});
it('prefers memory_tags over other formats', () => {
// Content has both <memory> tags and bullet lines
const mixed = '<memory chat="x" date="y">\n- bullet\n</memory>\n- extra bullet';
expect(detectFileFormat(mixed)).toBe('memory_tags');
});
});
// ─── convertHeuristic ──────────────────────────────────────────────────
describe('convertHeuristic', () => {
it('returns existing blocks for memory_tags format with warning', () => {
const content = '<memory chat="x" date="2024-01-01">\n- fact\n</memory>';
const result = convertHeuristic(content, 'memory_tags');
expect(result.blocks).toHaveLength(1);
expect(result.blocks[0].bullets).toEqual(['fact']);
expect(result.warnings).toHaveLength(1);
expect(result.warnings[0]).toMatch(/already/i);
});
it('converts bullet lists to a single memory block', () => {
const content = '- Apple\n- Banana\n- Cherry';
const result = convertHeuristic(content, 'bullets');
expect(result.blocks).toHaveLength(1);
expect(result.blocks[0].chat).toBe('imported');
expect(result.blocks[0].bullets).toEqual(['Apple', 'Banana', 'Cherry']);
});
it('converts numbered lists to a single memory block', () => {
const content = '1. First\n2. Second\n3. Third';
const result = convertHeuristic(content, 'numbered');
expect(result.blocks).toHaveLength(1);
expect(result.blocks[0].bullets).toEqual(['First', 'Second', 'Third']);
});
it('converts markdown headings to separate blocks per heading', () => {
const content = '# Background\n- Born in NYC\n# Personality\n- Cheerful';
const result = convertHeuristic(content, 'markdown_headings');
expect(result.blocks).toHaveLength(2);
expect(result.blocks[0].chat).toBe('Background');
expect(result.blocks[0].bullets).toEqual(['Born in NYC']);
expect(result.blocks[1].chat).toBe('Personality');
expect(result.blocks[1].bullets).toEqual(['Cheerful']);
});
it('converts freeform text by splitting on sentences', () => {
const content = 'She likes cats. He has a dog. They live together.';
const result = convertHeuristic(content, 'freeform');
expect(result.blocks).toHaveLength(1);
expect(result.blocks[0].bullets).toEqual([
'She likes cats.',
'He has a dog.',
'They live together.',
]);
expect(result.warnings[0]).toMatch(/freeform/i);
});
it('handles empty freeform content', () => {
const result = convertHeuristic('', 'freeform');
expect(result.blocks).toEqual([]);
// Empty string for freeform doesn't match sentences
});
it('handles memory_headings via migration', () => {
const content = '## Memory 1\n- Old fact one\n- Old fact two\n## Memory 2\n- Another fact';
const result = convertHeuristic(content, 'memory_headings');
expect(result.blocks).toHaveLength(2);
expect(result.blocks[0].bullets).toEqual(['Old fact one', 'Old fact two']);
expect(result.blocks[1].bullets).toEqual(['Another fact']);
});
it('includes plain text lines under markdown headings as bullets', () => {
const content = '# Section\nPlain text line\nAnother plain line';
const result = convertHeuristic(content, 'markdown_headings');
expect(result.blocks[0].bullets).toContain('Plain text line');
expect(result.blocks[0].bullets).toContain('Another plain line');
});
});
// ─── migrateMemoriesIfNeeded ───────────────────────────────────────────
describe('migrateMemoriesIfNeeded', () => {
it('returns content unchanged if already in <memory> format', () => {
const content = '<memory chat="x" date="y">\n- fact\n</memory>';
expect(migrateMemoriesIfNeeded(content)).toBe(content);
});
it('returns null/empty unchanged', () => {
expect(migrateMemoriesIfNeeded(null)).toBe(null);
expect(migrateMemoriesIfNeeded('')).toBe('');
expect(migrateMemoriesIfNeeded(' ')).toBe(' ');
});
it('converts old ## Memory N format', () => {
const content = '## Memory 1\n- Old fact\n## Memory 2\n- Another fact';
const result = migrateMemoriesIfNeeded(content);
expect(result).toContain('<memory');
expect(result).toContain('</memory>');
const parsed = parseMemories(result);
expect(parsed).toHaveLength(2);
expect(parsed[0].bullets).toEqual(['Old fact']);
expect(parsed[1].bullets).toEqual(['Another fact']);
});
it('extracts _Extracted: ..._ timestamps from old format', () => {
const content = '## Memory 1\n_Extracted: 2023-06-15 10:00_\n- Fact with date';
const result = migrateMemoriesIfNeeded(content);
const parsed = parseMemories(result);
expect(parsed[0].date).toBe('2023-06-15 10:00');
});
it('wraps flat text as a single memory block', () => {
const content = 'Just some plain text without any formatting.';
const result = migrateMemoriesIfNeeded(content);
expect(result).toContain('<memory');
const parsed = parseMemories(result);
expect(parsed).toHaveLength(1);
expect(parsed[0].bullets).toEqual(['Just some plain text without any formatting.']);
});
it('wraps flat bullet list as a single block', () => {
const content = '- Fact one\n- Fact two\n- Fact three';
const result = migrateMemoriesIfNeeded(content);
const parsed = parseMemories(result);
expect(parsed).toHaveLength(1);
expect(parsed[0].bullets).toEqual(['Fact one', 'Fact two', 'Fact three']);
});
});

286
test/unit/parsing.test.js Normal file
View file

@ -0,0 +1,286 @@
import { describe, it, expect } from 'vitest';
import {
parseMemories,
countMemories,
serializeMemories,
mergeMemoryBlocks,
splitMultiTagBullets,
} from '../../lib.js';
// ─── parseMemories ─────────────────────────────────────────────────────
describe('parseMemories', () => {
it('parses a standard memory block', () => {
const input = `<memory chat="main_abc" date="2024-01-15 14:30">
- Likes cats
- Has a red car
</memory>`;
const result = parseMemories(input);
expect(result).toEqual([
{ chat: 'main_abc', date: '2024-01-15 14:30', bullets: ['Likes cats', 'Has a red car'] },
]);
});
it('parses multiple blocks', () => {
const input = `<memory chat="chat1" date="2024-01-01">
- Fact A
</memory>
<memory chat="chat2" date="2024-02-01">
- Fact B
- Fact C
</memory>`;
const result = parseMemories(input);
expect(result).toHaveLength(2);
expect(result[0].chat).toBe('chat1');
expect(result[1].bullets).toEqual(['Fact B', 'Fact C']);
});
it('excludes blocks with no bullets', () => {
const input = `<memory chat="empty" date="2024-01-01">
some text without bullet prefix
</memory>`;
const result = parseMemories(input);
expect(result).toEqual([]);
});
it('handles attributes with special characters', () => {
const input = `<memory chat="Bob &amp; Alice&quot;s chat" date="2024-01-01">
- Met at a cafe
</memory>`;
const result = parseMemories(input);
expect(result[0].chat).toBe('Bob & Alice"s chat');
});
it('strips metadata-prefixed bullets', () => {
const input = `<memory chat="test" date="2024-01-01">
[2024-01-01 | test] - Has a dog named Rex
- Plain bullet
</memory>`;
const result = parseMemories(input);
expect(result[0].bullets).toEqual(['Has a dog named Rex', 'Plain bullet']);
});
it('is case-insensitive for memory tags', () => {
const input = `<Memory chat="test" date="2024-01-01">
- Case test
</Memory>`;
const result = parseMemories(input);
expect(result).toHaveLength(1);
expect(result[0].bullets).toEqual(['Case test']);
});
it('returns empty array for null/undefined/empty input', () => {
expect(parseMemories(null)).toEqual([]);
expect(parseMemories(undefined)).toEqual([]);
expect(parseMemories('')).toEqual([]);
expect(parseMemories(' ')).toEqual([]);
});
it('skips malformed tags gracefully', () => {
const input = `<memory chat="good" date="2024-01-01">
- Valid bullet
</memory>
<memory chat="unclosed" date="2024-01-01">
- This block is never closed
Some other text here`;
const result = parseMemories(input);
expect(result).toHaveLength(1);
expect(result[0].chat).toBe('good');
});
it('defaults chat to "unknown" when attribute is missing', () => {
const input = `<memory date="2024-01-01">
- No chat attr
</memory>`;
const result = parseMemories(input);
expect(result[0].chat).toBe('unknown');
});
it('defaults date to empty string when attribute is missing', () => {
const input = `<memory chat="test">
- No date attr
</memory>`;
const result = parseMemories(input);
expect(result[0].date).toBe('');
});
});
// ─── countMemories ─────────────────────────────────────────────────────
describe('countMemories', () => {
it('sums bullets across multiple blocks', () => {
const blocks = [
{ chat: 'a', date: '', bullets: ['one', 'two'] },
{ chat: 'b', date: '', bullets: ['three'] },
];
expect(countMemories(blocks)).toBe(3);
});
it('returns 0 for empty array', () => {
expect(countMemories([])).toBe(0);
});
});
// ─── serializeMemories ─────────────────────────────────────────────────
describe('serializeMemories', () => {
const blocks = [
{ chat: 'main_abc', date: '2024-01-15', bullets: ['Likes cats', 'Has red car'] },
{ chat: 'main_def', date: '2024-02-01', bullets: ['Works at a bakery'] },
];
it('serializes in default block mode', () => {
const result = serializeMemories(blocks);
expect(result).toContain('<memory chat="main_abc" date="2024-01-15">');
expect(result).toContain('- Likes cats\n- Has red car');
expect(result).toContain('</memory>');
// Blocks separated by double newline
expect(result).toContain('</memory>\n\n<memory');
});
it('serializes in bullet boundary mode', () => {
const fmt = { boundary: 'bullet', separator: '\n\n', metadata: false };
const result = serializeMemories(blocks, fmt);
// Bullets separated by double newline within block
expect(result).toContain('- Likes cats\n\n- Has red car');
});
it('serializes with metadata prefixes', () => {
const fmt = { boundary: 'bullet', separator: '\n\n', metadata: true };
const result = serializeMemories(blocks, fmt);
expect(result).toContain('[2024-01-15 | main_abc] - Likes cats');
});
it('serializes with custom separator', () => {
const fmt = { boundary: 'custom', separator: '\n---\n', metadata: false };
const result = serializeMemories(blocks, fmt);
expect(result).toContain('</memory>\n---\n<memory');
});
it('escapes special characters in attributes', () => {
const special = [{ chat: 'Bob & Alice"s', date: '2024', bullets: ['test'] }];
const result = serializeMemories(special);
expect(result).toContain('chat="Bob &amp; Alice&quot;s"');
});
it('round-trips through parseMemories', () => {
const serialized = serializeMemories(blocks);
const parsed = parseMemories(serialized);
expect(parsed).toEqual(blocks);
});
it('round-trips with metadata through parseMemories', () => {
const fmt = { boundary: 'bullet', separator: '\n\n', metadata: true };
const serialized = serializeMemories(blocks, fmt);
const parsed = parseMemories(serialized);
// Metadata-prefixed bullets should be stripped back to plain text
expect(parsed).toEqual(blocks);
});
});
// ─── mergeMemoryBlocks ─────────────────────────────────────────────────
describe('mergeMemoryBlocks', () => {
it('merges blocks with same chat ID', () => {
const blocks = [
{ chat: 'main', date: '2024-01-01', bullets: ['A'] },
{ chat: 'main', date: '2024-01-02', bullets: ['B'] },
];
const result = mergeMemoryBlocks(blocks);
expect(result).toHaveLength(1);
expect(result[0].bullets).toEqual(['A', 'B']);
// Keeps date from first occurrence
expect(result[0].date).toBe('2024-01-01');
});
it('keeps blocks with different chat IDs separate', () => {
const blocks = [
{ chat: 'chat1', date: '2024-01-01', bullets: ['A'] },
{ chat: 'chat2', date: '2024-01-02', bullets: ['B'] },
];
const result = mergeMemoryBlocks(blocks);
expect(result).toHaveLength(2);
});
it('preserves order of first occurrence', () => {
const blocks = [
{ chat: 'first', date: '', bullets: ['1'] },
{ chat: 'second', date: '', bullets: ['2'] },
{ chat: 'first', date: '', bullets: ['3'] },
];
const result = mergeMemoryBlocks(blocks);
expect(result[0].chat).toBe('first');
expect(result[0].bullets).toEqual(['1', '3']);
expect(result[1].chat).toBe('second');
});
it('does not mutate input blocks', () => {
const blocks = [
{ chat: 'main', date: '', bullets: ['A'] },
{ chat: 'main', date: '', bullets: ['B'] },
];
const originalBullets = [...blocks[0].bullets];
mergeMemoryBlocks(blocks);
expect(blocks[0].bullets).toEqual(originalBullets);
});
});
// ─── splitMultiTagBullets ───────────────────────────────────────────────
describe('splitMultiTagBullets', () => {
it('returns original array when no topic tags', () => {
const bullets = ['Flux ate salmon', 'Alex went to work'];
expect(splitMultiTagBullets(bullets)).toEqual([bullets]);
});
it('returns original array when one topic tag', () => {
const bullets = ['[Alex, Flux — adoption day]', 'Flux ate salmon', 'Alex assembled a cat tree'];
expect(splitMultiTagBullets(bullets)).toEqual([bullets]);
});
it('splits at multiple em-dash topic tags', () => {
const bullets = [
'[Alex, Flux — morning routine]', 'Flux woke Alex', 'Alex fed Flux',
'[Alex, Flux — evening bonding]', 'They watched TV',
'[Alex, Mike — game night]', 'Mike visited', 'Flux hissed at Mike',
];
expect(splitMultiTagBullets(bullets)).toEqual([
['[Alex, Flux — morning routine]', 'Flux woke Alex', 'Alex fed Flux'],
['[Alex, Flux — evening bonding]', 'They watched TV'],
['[Alex, Mike — game night]', 'Mike visited', 'Flux hissed at Mike'],
]);
});
it('handles bullets before first topic tag', () => {
const bullets = [
'Some orphan bullet',
'[Alex, Flux — morning]', 'Flux ate',
'[Alex, Flux — evening]', 'Flux slept',
];
expect(splitMultiTagBullets(bullets)).toEqual([
['Some orphan bullet', '[Alex, Flux — morning]', 'Flux ate'],
['[Alex, Flux — evening]', 'Flux slept'],
]);
});
it('handles en-dash and hyphen topic tags', () => {
const bullets = [
'[Alex, Flux – morning routine]', 'Flux ate',
'[Alex, Flux - evening bonding]', 'Flux slept',
];
expect(splitMultiTagBullets(bullets)).toEqual([
['[Alex, Flux – morning routine]', 'Flux ate'],
['[Alex, Flux - evening bonding]', 'Flux slept'],
]);
});
it('does not false-positive on hyphenated names without spaces', () => {
const bullets = ['[Alex-Bob]', 'Something happened'];
expect(splitMultiTagBullets(bullets)).toEqual([bullets]);
});
it('returns empty array wrapper for empty input', () => {
expect(splitMultiTagBullets([])).toEqual([[]]);
});
});

271
test/unit/utils.test.js Normal file
View file

@ -0,0 +1,271 @@
import { describe, it, expect } from 'vitest';
import { truncateText, reindexEditingSet, stripNonDiegetic, formatChatMessages, substitutePromptTemplate, getTimestamp, cloneMemoryBlocks } from '../../lib.js';
// ─── truncateText ──────────────────────────────────────────────────────
describe('truncateText', () => {
it('returns short text unchanged', () => {
expect(truncateText('Hello', 100)).toBe('Hello');
});
it('returns null/empty/undefined as-is', () => {
expect(truncateText(null, 100)).toBe(null);
expect(truncateText('', 100)).toBe('');
expect(truncateText(undefined, 100)).toBe(undefined);
});
it('truncates long text and adds suffix', () => {
const text = 'A'.repeat(200);
const result = truncateText(text, 100);
expect(result.length).toBeLessThan(200);
expect(result).toContain('[...truncated]');
});
it('breaks at newline boundary when possible', () => {
// 60% through is char 60, so a newline at position 70 (>50% of 100) should be used
const text = 'A'.repeat(70) + '\n' + 'B'.repeat(80);
const result = truncateText(text, 100);
expect(result).toBe('A'.repeat(70) + '\n[...truncated]');
});
it('does not break at newline if too early in the string', () => {
// Newline at position 10 (<50% of 100) — should not be used as break point
const text = 'A'.repeat(10) + '\n' + 'B'.repeat(200);
const result = truncateText(text, 100);
// Should truncate at maxChars, not at the early newline
expect(result).toBe('A'.repeat(10) + '\n' + 'B'.repeat(89) + '\n[...truncated]');
});
it('returns text unchanged when exactly at limit', () => {
const text = 'A'.repeat(100);
expect(truncateText(text, 100)).toBe(text);
});
});
// ─── reindexEditingSet ─────────────────────────────────────────────────
describe('reindexEditingSet', () => {
it('removes the deleted index and shifts higher indices down', () => {
const set = new Set([0, 2, 4]);
reindexEditingSet(set, 2);
expect(set).toEqual(new Set([0, 3]));
});
it('leaves indices below removed index unchanged', () => {
const set = new Set([0, 1]);
reindexEditingSet(set, 5);
expect(set).toEqual(new Set([0, 1]));
});
it('handles empty set', () => {
const set = new Set();
reindexEditingSet(set, 3);
expect(set.size).toBe(0);
});
it('handles removing the only index', () => {
const set = new Set([3]);
reindexEditingSet(set, 3);
expect(set.size).toBe(0);
});
it('shifts multiple indices above removal point', () => {
const set = new Set([1, 3, 5, 7]);
reindexEditingSet(set, 2);
expect(set).toEqual(new Set([1, 2, 4, 6]));
});
});
// ─── stripNonDiegetic ────────────────────────────────────────────────
describe('stripNonDiegetic', () => {
it('removes markdown code blocks', () => {
const input = 'Before ```const x = 1;\nconsole.log(x);``` After';
expect(stripNonDiegetic(input)).toBe('Before After');
});
it('removes details sections', () => {
const input = 'Before <details><summary>Hidden</summary>Secret content</details> After';
expect(stripNonDiegetic(input)).toBe('Before After');
});
it('removes markdown tables', () => {
const input = 'Before\n| Col1 | Col2 |\n| --- | --- |\n| A | B |\nAfter';
expect(stripNonDiegetic(input)).toBe('Before\n\nAfter');
});
it('removes HTML tags', () => {
const input = 'Hello <b>world</b> and <img src="x" /> done';
expect(stripNonDiegetic(input)).toBe('Hello world and done');
});
it('collapses 3+ newlines to 2', () => {
const input = 'Line 1\n\n\n\nLine 2';
expect(stripNonDiegetic(input)).toBe('Line 1\n\nLine 2');
});
it('handles combined non-diegetic content', () => {
const input = '*She smiles* ```image: portrait``` and shows a table\n| x | y |\n| 1 | 2 |\nthen continues';
const result = stripNonDiegetic(input);
expect(result).not.toContain('```');
expect(result).not.toContain('| x |');
expect(result).toContain('*She smiles*');
expect(result).toContain('then continues');
});
it('returns empty string for all-non-diegetic input', () => {
const input = '```only code here```';
expect(stripNonDiegetic(input).trim()).toBe('');
});
});
// ─── formatChatMessages ─────────────────────────────────────────────
describe('formatChatMessages', () => {
const makeMsg = (name, mes, overrides = {}) => ({
name, mes, is_user: false, is_system: false, ...overrides,
});
it('formats messages as "Name: text"', () => {
const chat = [
makeMsg('Alice', 'Hello there'),
makeMsg('Bob', 'Hi Alice'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).toBe('Alice: Hello there\n\nBob: Hi Alice');
});
it('skips empty messages', () => {
const chat = [
makeMsg('Alice', 'Hello'),
makeMsg('Bob', ''),
makeMsg('Alice', 'Still here'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).toBe('Alice: Hello\n\nAlice: Still here');
expect(result.messageCount).toBe(2);
});
it('skips system-only messages (no name, no user)', () => {
const chat = [
makeMsg('Alice', 'Hello'),
makeMsg(null, 'System narrator text', { is_system: true }),
makeMsg('Bob', 'Hi'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).not.toContain('System narrator');
});
it('keeps system messages that have a name', () => {
const chat = [
makeMsg('Extension', 'Some extension text', { is_system: true }),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).toContain('Extension: Some extension text');
});
it('strips non-diegetic content from messages', () => {
const chat = [
makeMsg('Alice', 'She smiled ```image prompt here``` and waved'),
];
const result = formatChatMessages(chat, 0, chat.length);
expect(result.text).not.toContain('```');
expect(result.text).toContain('She smiled');
});
it('respects startIndex and endIndex', () => {
const chat = [
makeMsg('A', 'msg0'),
makeMsg('B', 'msg1'),
makeMsg('C', 'msg2'),
makeMsg('D', 'msg3'),
];
const result = formatChatMessages(chat, 1, 3);
expect(result.text).toBe('B: msg1\n\nC: msg2');
expect(result.startIndex).toBe(1);
expect(result.endIndex).toBe(2);
});
it('returns empty for out-of-range indices', () => {
const chat = [makeMsg('A', 'msg')];
const result = formatChatMessages(chat, 5, 10);
expect(result.text).toBe('');
});
});
// ─── substitutePromptTemplate ───────────────────────────────────────
describe('substitutePromptTemplate', () => {
const template = 'Name: {{charName}}\nCard: {{charCard}}\nMemories: {{existingMemories}}\nMessages: {{recentMessages}}';
it('substitutes all template variables', () => {
const result = substitutePromptTemplate(template, {
charName: 'Flux',
charCard: 'A cat',
existingMemories: '- Likes fish',
recentMessages: 'Alex: Hello\n\nFlux: Meow',
});
expect(result).toBe('Name: Flux\nCard: A cat\nMemories: - Likes fish\nMessages: Alex: Hello\n\nFlux: Meow');
});
it('replaces multiple occurrences of the same variable', () => {
const t = '{{charName}} says hi. {{charName}} waves.';
const result = substitutePromptTemplate(t, { charName: 'Flux' });
expect(result).toBe('Flux says hi. Flux waves.');
});
it('substitutes {{participants}} when provided', () => {
const t = 'Participants: {{participants}}';
const result = substitutePromptTemplate(t, { participants: 'Alice, Bob' });
expect(result).toBe('Participants: Alice, Bob');
});
it('leaves unmatched variables as-is', () => {
const t = '{{charName}} and {{unknownVar}}';
const result = substitutePromptTemplate(t, { charName: 'Flux' });
expect(result).toContain('{{unknownVar}}');
});
it('uses "(none yet)" default for missing existingMemories', () => {
const result = substitutePromptTemplate(template, {
charName: 'Flux', charCard: 'A cat', recentMessages: 'hi',
});
expect(result).toContain('(none yet)');
});
});
// ─── getTimestamp ──────────────────────────────────────────────────────
describe('getTimestamp', () => {
it('returns YYYY-MM-DD HH:MM format', () => {
const ts = getTimestamp();
expect(ts).toMatch(/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}$/);
});
it('accepts a Date argument', () => {
const ts = getTimestamp(new Date(2026, 0, 15, 9, 5));
expect(ts).toBe('2026-01-15 09:05');
});
});
// ─── cloneMemoryBlocks ────────────────────────────────────────────────
describe('cloneMemoryBlocks', () => {
it('deep clones blocks without shared references', () => {
const original = [
{ chat: 'test', date: '2026-01-01', bullets: ['a', 'b'] },
];
const cloned = cloneMemoryBlocks(original);
cloned[0].bullets.push('c');
expect(original[0].bullets).toEqual(['a', 'b']);
});
it('preserves all block properties', () => {
const original = [
{ chat: 'test', date: '2026-01-01 14:30', bullets: ['a'] },
];
const cloned = cloneMemoryBlocks(original);
expect(cloned[0].chat).toBe('test');
expect(cloned[0].date).toBe('2026-01-01 14:30');
});
});