docs: update for v2.1.0 beta, add tests, clean up repo

Docs: - troubleshooting: downgrade file vectorization and no-memories-injected checks from RED to YELLOW (false alarm fixes from v2.0.1) - managing-memories: update topic tag example to include character name - injection-viewer: mention Display Mode setting for tablet/phone - README: add tablet & phone support to Feature Highlights with beta testing call-to-action - getting-started: add backup reminder section - changelog: add nudge banner button rename Tests: - Add escaping.test.js (13 tests) and format-detection.test.js (24 tests) Repo: - Add .gitignore for screenshots, .DS_Store, .playwright-mcp, drafts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 03:39:44 +00:00 · 2026-03-02 15:01:45 -08:00 · 2026-03-02 15:01:45 -08:00 · 0010f0ef98
commit 0010f0ef98
parent dd0a3a73ed
9 changed files with 284 additions and 8 deletions
--- a/.gitignore
+++ b/.gitignore
@ -0,0 +1,9 @@
+.envrc
+node_modules/
+.DS_Store
+*.png
+!images/*.png
+.playwright-mcp/
+package-lock.json
+docs/reddit-*.md
+docs/topic-tag-charname-fix.md
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -13,6 +13,7 @@
 - **Tablet panel off-screen on mobile**: SillyTavern sets `perspective` on `<html>`, which changes the containing block for `position: fixed` elements. The panel's `top: 50%` resolved to 0px on mobile. Fixed by using viewport units (`50vh`/`50vw`) instead of percentages.
 - **Tablet panel hidden behind sidebar**: The panel's z-index (1002) was below SillyTavern's extensions drawer (3005). Raised to 5000.
 - **Injection viewer and log drawer hidden on mobile**: Both drawers had z-index below the sidebar and were too narrow at phone widths (40vw = 157px on a 393px screen). Phone mode overrides fix both issues.
+- **Nudge banner "Fix now" → "View"**: The warning banner button now says "View" since it opens the Troubleshooter for inspection — it doesn't auto-fix anything.

 ## 2.0.1

--- a/README.md
+++ b/README.md
@ -33,6 +33,7 @@ CharMemory does not touch your lorebooks so if those contain information about t
 - **Full memory control** — browse, edit, or delete individual memories. Consolidate duplicates with preview and undo. Batch-extract from all chats at once.
 - **Highly configurable extraction prompts** — separate prompts for 1:1 and group chats and memory file consolidation and conversion.
 - **Guided setup** — the Setup Wizard tests your LLM connection, checks Vector Storage, and handles existing memory file conversion in about 2 minutes.
+- **Tablet & phone support** *(new in 2.1.0 — testing appreciated!)* — on touch devices, the dashboard opens as a floating panel with touch-friendly controls. Phone layout widens drawers for small screens. If auto-detection doesn't match your device, override it in Settings > Advanced > Display Mode. Please [report issues](https://github.com/bal-spec/sillytavern-character-memory/issues) with how it behaves on your device.
 - **Plain files** — memories are stored as readable, editable markdown in the character's Data Bank. No database, no lock-in.

 ## What you need
--- a/docs/getting-started.md
+++ b/docs/getting-started.md
@ -77,6 +77,14 @@ Click **Get Started** to close the wizard and return to the dashboard.

 ---

+## Back up your data
+
+SillyTavern has built-in backup tools that snapshot your entire data directory — characters, chats, settings, and Data Bank files (where memories live). Before making big changes like consolidating, clearing memories, or switching setups, it's worth having a recent backup.
+
+See SillyTavern's [User Settings documentation](https://docs.sillytavern.app/usage/user-settings/) for how to create and manage backups.
+
+---
+
 ## Your first extraction

 After setup, chat normally. The **stats bar** at the top of the CharMemory panel tracks your progress:
--- a/docs/injection-viewer.md
+++ b/docs/injection-viewer.md
@ -51,7 +51,7 @@ The viewer header includes a colored health dot that mirrors the one in the stat
 | **Red** | Problems detected — memories may not be injecting correctly |
 | **Gray** | No generation captured yet for this message |

-Hover over the dot for a quick summary. On touch devices, tap the **Diagnostics** link in the viewer toolbar to see the full health check results inline without leaving the chat.
+Hover over the dot for a quick summary. Tap the **Diagnostics** link in the viewer toolbar to see the full health check results inline without leaving the chat. On tablets and phones (or when Display Mode is set to Tablet/Phone in Settings > Advanced), the viewer opens as a wide drawer above the sidebar for easier reading.

 For details on what each check looks for and how to fix issues, see [Troubleshooting → Health Checks](troubleshooting.md#health-checks).

--- a/docs/managing-memories.md
+++ b/docs/managing-memories.md
@ -137,7 +137,7 @@ Each chat's extraction state is tracked independently. Re-running batch extracti

 **Reformat** restructures your existing memory file to the current topic-tagged format for better vector retrieval. Use it when:

- You have older memories without topic tags (`[Names — description]` as the first bullet)
+- You have older memories without topic tags, or topic tags that don't include the character's name (e.g., `[Alex — description]` instead of `[Flux, Alex — description]`)
 - You've imported memories from another source and want them normalized
 - The Setup Wizard offered to convert existing memories and you skipped it

--- a/docs/troubleshooting.md
+++ b/docs/troubleshooting.md
@ -12,10 +12,10 @@ The health dot runs up to 7 checks depending on what data is available. Checks 1
 |-------|-------------------|--------|
 | **Files enabled** | "Enable for files" is on in Vector Storage | RED if off |
 | **Memory file exists** | A memory file is in the character's Data Bank | RED if missing |
-| **File vectorized** | Memory file has been indexed (chunk count > 0) | RED if 0 chunks |
+| **File vectorized** | Memory file has been indexed (chunk count > 0) | YELLOW if 0 chunks (resolves on next generation) |
 | **Chunk overlap** | Data Bank overlap setting in Vector Storage | YELLOW if 0% |
 | **Chunk size** | Data Bank chunk size in Vector Storage | YELLOW if outside recommended range |
-| **Memories injected** | Memory bullets appeared in the last generation | RED if file exists and vectorized but 0 injected |
+| **Memories injected** | Memory bullets appeared in the last generation | YELLOW if 0 injected (may be normal — score threshold filtering) |
 | **Duplicate detection** | Same bullet appears more than once in injected content | YELLOW if duplicates found |

 ### Fixing each issue
@ -26,11 +26,11 @@ Open Extensions → Vector Storage → under File vectorization settings, check
 **RED — Memory file not found**
 Run an extraction first. The memory file is created on first successful extraction — Extract Now or wait for auto-extraction to fire.

-**RED — File not vectorized**
-The file exists but hasn't been indexed. Try generating a message (Vector Storage indexes on generation), or check that your vectorization source is configured and responding. If you recently purged vectors, just generate a message to trigger re-indexing.
+**YELLOW — File not yet vectorized**
+The file exists but hasn't been indexed yet. This is normal when a memory file was just created — Vector Storage indexes on the next generation. Just send a message and generate a response. If the check stays yellow after generating, check that your vectorization source is configured and responding. If you recently purged vectors, generate a message to trigger re-indexing.

-**RED — Memories not injected**
-File exists and is vectorized, but 0 memories injected. Most likely cause: score threshold is too high. Try lowering it to 0.2 in Vector Storage → Data Bank files row. Also check that **Retrieve chunks** isn't set to 0.
+**YELLOW — No memories injected**
+File exists and is vectorized, but 0 memories appeared in the last generation. This can be normal — it means no memories scored above the relevance threshold for the current conversation topic. If you expect memories to be injected, try lowering the score threshold to 0.2 in Vector Storage → Data Bank files row. Also check that **Retrieve chunks** isn't set to 0.

 **YELLOW — Chunk overlap is 0%**
 With 0% overlap, a memory block landing on a chunk boundary gets split — neither half retrieves cleanly, and you may see duplicate partial bullets in injected content. 0% is a valid starting point (especially with small, topic-tagged blocks), but if you notice split blocks in the Injection Viewer, increase overlap to 10–15% in Vector Storage → Data Bank files. [Purge and re-vectorize](retrieval-and-prompts.md#purge-and-re-vectorize) after changing.
@ -108,6 +108,8 @@ Click the **wrench icon** → **Diagnostic Report** → copy the output.

 ## Reset tools

+> **Before using Clear All Memories**, make a backup. Use SillyTavern's [backup tools](https://docs.sillytavern.app/usage/user-settings/) to snapshot your data directory, or download the memory file from the Data Bank.
+
 See [Managing Memories → Reset and Clear](managing-memories.md#reset-and-clear) for what each reset option does and when to use it. The short version:

 - **Reset Extraction State** — re-process messages from the beginning, no memory loss
--- a/test/unit/escaping.test.js
+++ b/test/unit/escaping.test.js
@ -0,0 +1,78 @@
+import { describe, it, expect } from 'vitest';
+import { escapeHtml, escapeAttr, unescapeAttr } from '../../lib.js';
+
+// ─── escapeHtml ────────────────────────────────────────────────────────
+
+describe('escapeHtml', () => {
+    it('escapes all five dangerous characters', () => {
+        expect(escapeHtml('&')).toBe('&amp;');
+        expect(escapeHtml('<')).toBe('&lt;');
+        expect(escapeHtml('>')).toBe('&gt;');
+        expect(escapeHtml('"')).toBe('&quot;');
+        expect(escapeHtml("'")).toBe('&#39;');
+    });
+
+    it('escapes a string with mixed dangerous characters', () => {
+        expect(escapeHtml('<script>alert("xss")</script>')).toBe(
+            '&lt;script&gt;alert(&quot;xss&quot;)&lt;/script&gt;',
+        );
+    });
+
+    it('leaves safe strings unchanged', () => {
+        expect(escapeHtml('Hello world')).toBe('Hello world');
+    });
+
+    it('coerces numbers to string', () => {
+        expect(escapeHtml(42)).toBe('42');
+    });
+
+    it('handles empty string', () => {
+        expect(escapeHtml('')).toBe('');
+    });
+
+    it('does not double-escape already-escaped input', () => {
+        // This is expected behavior — escapeHtml escapes & in &amp; to &amp;amp;
+        // This confirms the function is a single-pass escaper, not idempotent
+        const once = escapeHtml('&');
+        const twice = escapeHtml(once);
+        expect(once).toBe('&amp;');
+        expect(twice).toBe('&amp;amp;');
+    });
+});
+
+// ─── escapeAttr / unescapeAttr ─────────────────────────────────────────
+
+describe('escapeAttr', () => {
+    it('escapes ampersands and double quotes', () => {
+        expect(escapeAttr('Tom & Jerry')).toBe('Tom &amp; Jerry');
+        expect(escapeAttr('She said "hi"')).toBe('She said &quot;hi&quot;');
+    });
+
+    it('handles combined special characters', () => {
+        expect(escapeAttr('A & "B"')).toBe('A &amp; &quot;B&quot;');
+    });
+
+    it('leaves safe strings unchanged', () => {
+        expect(escapeAttr('hello')).toBe('hello');
+    });
+
+    it('coerces numbers to string', () => {
+        expect(escapeAttr(123)).toBe('123');
+    });
+});
+
+describe('unescapeAttr', () => {
+    it('unescapes &amp; and &quot;', () => {
+        expect(unescapeAttr('Tom &amp; Jerry')).toBe('Tom & Jerry');
+        expect(unescapeAttr('She said &quot;hi&quot;')).toBe('She said "hi"');
+    });
+
+    it('round-trips with escapeAttr', () => {
+        const original = 'Bob & Alice "together"';
+        expect(unescapeAttr(escapeAttr(original))).toBe(original);
+    });
+
+    it('handles strings with no escaped sequences', () => {
+        expect(unescapeAttr('plain text')).toBe('plain text');
+    });
+});
--- a/test/unit/format-detection.test.js
+++ b/test/unit/format-detection.test.js
@ -0,0 +1,177 @@
+import { describe, it, expect } from 'vitest';
+import { detectFileFormat, convertHeuristic, migrateMemoriesIfNeeded, parseMemories } from '../../lib.js';
+
+// ─── detectFileFormat ──────────────────────────────────────────────────
+
+describe('detectFileFormat', () => {
+    it('detects memory_tags format', () => {
+        expect(detectFileFormat('<memory chat="x" date="y">\n- bullet\n</memory>')).toBe('memory_tags');
+    });
+
+    it('detects memory_headings (legacy) format', () => {
+        expect(detectFileFormat('## Memory 1\n- bullet\n## Memory 2\n- bullet')).toBe('memory_headings');
+    });
+
+    it('detects bullet list format', () => {
+        const bullets = '- one\n- two\n- three\n- four\n- five';
+        expect(detectFileFormat(bullets)).toBe('bullets');
+    });
+
+    it('detects asterisk bullets as bullets format', () => {
+        const bullets = '* one\n* two\n* three\n* four\n* five';
+        expect(detectFileFormat(bullets)).toBe('bullets');
+    });
+
+    it('detects numbered list format', () => {
+        const numbered = '1. First\n2. Second\n3. Third\n4. Fourth';
+        expect(detectFileFormat(numbered)).toBe('numbered');
+    });
+
+    it('detects numbered list with parentheses', () => {
+        const numbered = '1) First\n2) Second\n3) Third\n4) Fourth';
+        expect(detectFileFormat(numbered)).toBe('numbered');
+    });
+
+    it('detects markdown_headings format', () => {
+        const md = '# Section One\nSome text\n## Section Two\nMore text';
+        expect(detectFileFormat(md)).toBe('markdown_headings');
+    });
+
+    it('returns freeform for plain prose', () => {
+        expect(detectFileFormat('This is just some regular text without any special formatting.')).toBe('freeform');
+    });
+
+    it('returns freeform for null/undefined/empty', () => {
+        expect(detectFileFormat(null)).toBe('freeform');
+        expect(detectFileFormat(undefined)).toBe('freeform');
+        expect(detectFileFormat('')).toBe('freeform');
+        expect(detectFileFormat('   ')).toBe('freeform');
+    });
+
+    it('prefers memory_tags over other formats', () => {
+        // Content has both <memory> tags and bullet lines
+        const mixed = '<memory chat="x" date="y">\n- bullet\n</memory>\n- extra bullet';
+        expect(detectFileFormat(mixed)).toBe('memory_tags');
+    });
+});
+
+// ─── convertHeuristic ──────────────────────────────────────────────────
+
+describe('convertHeuristic', () => {
+    it('returns existing blocks for memory_tags format with warning', () => {
+        const content = '<memory chat="x" date="2024-01-01">\n- fact\n</memory>';
+        const result = convertHeuristic(content, 'memory_tags');
+        expect(result.blocks).toHaveLength(1);
+        expect(result.blocks[0].bullets).toEqual(['fact']);
+        expect(result.warnings).toHaveLength(1);
+        expect(result.warnings[0]).toMatch(/already/i);
+    });
+
+    it('converts bullet lists to a single memory block', () => {
+        const content = '- Apple\n- Banana\n- Cherry';
+        const result = convertHeuristic(content, 'bullets');
+        expect(result.blocks).toHaveLength(1);
+        expect(result.blocks[0].chat).toBe('imported');
+        expect(result.blocks[0].bullets).toEqual(['Apple', 'Banana', 'Cherry']);
+    });
+
+    it('converts numbered lists to a single memory block', () => {
+        const content = '1. First\n2. Second\n3. Third';
+        const result = convertHeuristic(content, 'numbered');
+        expect(result.blocks).toHaveLength(1);
+        expect(result.blocks[0].bullets).toEqual(['First', 'Second', 'Third']);
+    });
+
+    it('converts markdown headings to separate blocks per heading', () => {
+        const content = '# Background\n- Born in NYC\n# Personality\n- Cheerful';
+        const result = convertHeuristic(content, 'markdown_headings');
+        expect(result.blocks).toHaveLength(2);
+        expect(result.blocks[0].chat).toBe('Background');
+        expect(result.blocks[0].bullets).toEqual(['Born in NYC']);
+        expect(result.blocks[1].chat).toBe('Personality');
+        expect(result.blocks[1].bullets).toEqual(['Cheerful']);
+    });
+
+    it('converts freeform text by splitting on sentences', () => {
+        const content = 'She likes cats. He has a dog. They live together.';
+        const result = convertHeuristic(content, 'freeform');
+        expect(result.blocks).toHaveLength(1);
+        expect(result.blocks[0].bullets).toEqual([
+            'She likes cats.',
+            'He has a dog.',
+            'They live together.',
+        ]);
+        expect(result.warnings[0]).toMatch(/freeform/i);
+    });
+
+    it('handles empty freeform content', () => {
+        const result = convertHeuristic('', 'freeform');
+        expect(result.blocks).toEqual([]);
+        // Empty string for freeform doesn't match sentences
+    });
+
+    it('handles memory_headings via migration', () => {
+        const content = '## Memory 1\n- Old fact one\n- Old fact two\n## Memory 2\n- Another fact';
+        const result = convertHeuristic(content, 'memory_headings');
+        expect(result.blocks).toHaveLength(2);
+        expect(result.blocks[0].bullets).toEqual(['Old fact one', 'Old fact two']);
+        expect(result.blocks[1].bullets).toEqual(['Another fact']);
+    });
+
+    it('includes plain text lines under markdown headings as bullets', () => {
+        const content = '# Section\nPlain text line\nAnother plain line';
+        const result = convertHeuristic(content, 'markdown_headings');
+        expect(result.blocks[0].bullets).toContain('Plain text line');
+        expect(result.blocks[0].bullets).toContain('Another plain line');
+    });
+});
+
+// ─── migrateMemoriesIfNeeded ───────────────────────────────────────────
+
+describe('migrateMemoriesIfNeeded', () => {
+    it('returns content unchanged if already in <memory> format', () => {
+        const content = '<memory chat="x" date="y">\n- fact\n</memory>';
+        expect(migrateMemoriesIfNeeded(content)).toBe(content);
+    });
+
+    it('returns null/empty unchanged', () => {
+        expect(migrateMemoriesIfNeeded(null)).toBe(null);
+        expect(migrateMemoriesIfNeeded('')).toBe('');
+        expect(migrateMemoriesIfNeeded('  ')).toBe('  ');
+    });
+
+    it('converts old ## Memory N format', () => {
+        const content = '## Memory 1\n- Old fact\n## Memory 2\n- Another fact';
+        const result = migrateMemoriesIfNeeded(content);
+        expect(result).toContain('<memory');
+        expect(result).toContain('</memory>');
+        const parsed = parseMemories(result);
+        expect(parsed).toHaveLength(2);
+        expect(parsed[0].bullets).toEqual(['Old fact']);
+        expect(parsed[1].bullets).toEqual(['Another fact']);
+    });
+
+    it('extracts _Extracted: ..._ timestamps from old format', () => {
+        const content = '## Memory 1\n_Extracted: 2023-06-15 10:00_\n- Fact with date';
+        const result = migrateMemoriesIfNeeded(content);
+        const parsed = parseMemories(result);
+        expect(parsed[0].date).toBe('2023-06-15 10:00');
+    });
+
+    it('wraps flat text as a single memory block', () => {
+        const content = 'Just some plain text without any formatting.';
+        const result = migrateMemoriesIfNeeded(content);
+        expect(result).toContain('<memory');
+        const parsed = parseMemories(result);
+        expect(parsed).toHaveLength(1);
+        expect(parsed[0].bullets).toEqual(['Just some plain text without any formatting.']);
+    });
+
+    it('wraps flat bullet list as a single block', () => {
+        const content = '- Fact one\n- Fact two\n- Fact three';
+        const result = migrateMemoriesIfNeeded(content);
+        const parsed = parseMemories(result);
+        expect(parsed).toHaveLength(1);
+        expect(parsed[0].bullets).toEqual(['Fact one', 'Fact two', 'Fact three']);
+    });
+});