fix(memory): scan uncapped when selecting forget candidates (#9530)

* fix(memory): scan uncapped when selecting forget candidates

Recall moved to the uncapped scanner in #8716; forget did not. A document
ranked past the 200-document cap could be recalled and injected into the
prompt but never forgotten.

Forget now scans uncapped, so its candidate universe matches recall's. The
model-selection prompt renders every candidate, so it gets its own bound of
400: literal query matches first, then the most recently modified remainder.
The heuristic fallback keeps scanning the full uncapped list.

Indexer, status, and extraction stay capped on purpose, and the two design
docs that recorded forget as capped now say otherwise.

Refs: #9378

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(memory): give each scope its own share of the forget prompt

Review round 1. The 400-candidate bound ranked both scopes into one recency
budget, so a store whose project entries are all newer than its user entries
seated no user memory at all. The capped scanners this replaced ran per scope,
so each scope always had seats. That made an old user entry unselectable by the
model while recall could still inject it, which is the same asymmetry the PR
set out to close.

Each scope now keeps a 200-candidate quota and whatever a smaller scope leaves
is handed to the other. Within a scope, literal query matches rank first and
both groups are ordered newest first, so truncation is deterministic instead of
scan-order, and the bound logs when it drops candidates.

Also from review: the query normalisation and match predicate are now shared
with selectByHeuristic so the two cannot drift; the user scan gets the
best-effort guard recall.ts and extractionAgentPlanner.ts already carry; and
the docstring and design docs no longer claim an unconditional guarantee the
bound does not provide.

Three tests, each verified against the mutation it is meant to catch: global
ranking drops the user ids, an ascending sort drops the newest filler, and
handing the fallback the bounded list returns 400 of 450 matches.

Refs: #9378

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(memory): bound the unconfirmed forget path and drop the silent scan guard

Review round 2, all suggestions.

MemoryManager.forget passed limit: MAX_SAFE_INTEGER and deletes without
confirmation. With an uncapped scan and a heuristic fallback that substring
matches the whole store, a one-character query matched nearly every entry in
both scopes, where the capped scanners had held that same failure to one scan's
worth of candidates. The limit is now the prompt bound, restoring the old
ceiling.

Round 1 added a best-effort catch on the user scan. That was wrong on two
counts: scan.ts caps after reading and ordering the whole tree, so uncapping
adds no read exposure to justify it, and swallowing the failure made forget
report "no entries matched" for a scope it never read, then act on that answer
by deleting. Reverted, with a comment saying why forget differs from recall
here: a missed injection is recoverable, a missed deletion is not.

normalizeForgetQuery now delegates to normalizeSummary so query matching and
the post-selection re-match cannot drift apart, and one design-doc sentence no
longer implies only semantic matches fall off the bound.

Two tests, each verified against its mutation: the quota split is now exercised
with both scopes over quota, where dropping it to 150 seats 250 project entries
instead of 200; and the delete ceiling fails at 401 removals if the unbounded
limit comes back.

Refs: #9378

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(memory): split forget's deletion seats per scope, and decouple the ceiling

Review round 3.

The deletion ceiling added last round truncated the heuristic fallback in
candidate order, and listIndexedForgetCandidates pushes every user entry ahead
of every project entry. With 450 matching user entries and 50 matching project
ones and the side query down, forget deleted 400 user entries, zero project
ones, and reported success. That is the reachability asymmetry this PR exists
to remove, moved into the delete path. The per-scope allocation the model
prompt already used is now shared with the heuristic, so each scope keeps its
share of the limit and a smaller scope's unused seats go to the other.

The ceiling is also its own constant now rather than an alias of the prompt
bound. Resizing the model prompt is a cost decision and resizing this is a
blast-radius decision; sharing one constant let the first silently widen the
second.

Two tests, each checked against its mutation: the 450-user/50-project shape
returns zero project matches under a plain slice, and oldest-first ranking
inside a scope drops that scope's newest entry from the prompt.

Refs: #9378

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(memory): pin the forget split at a small limit and the heuristic's own order

Cross-review found both new tests mutation-survivable. Every case used a
400 limit, so hard-coding a 200 per-scope quota instead of deriving it from
the budget still passed, and the recency case let the side query succeed, so
it pinned the model prompt's ranking rather than selectByHeuristic's own
comparator.

One case at limit 5 with the side query failing covers both: it asserts the
3/2 split, which only holds if the quota comes from the budget, and that each
scope contributes its newest entry, which fails if the comparator is reversed.
Both mutants verified failing.

Refs: #9378

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(memory): share the forget recency comparator and log a bound deletion

Review round 4, both suggestions.

The mtime comparator was the last thing the model path and the heuristic path
each typed for themselves, after this branch had already hoisted the query
normaliser, the match predicate and the per-scope allocator so the two could
not drift. Each site has its own test, so a one-sided ordering change would
have updated its own test, passed CI, and left the sibling stale. Now one
definition.

The deletion cap also bound silently. The prompt bound warns when it truncates;
the path that actually deletes did not, so a forget that removed 400 of 500
matches reported success and left no record of why recall kept injecting the
rest. It now says so.

No test for the new warning: it is a debug log line, and asserting on it would
pin the wording rather than the behaviour.

Refs: #9378

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Shaojin Wen <shaojin.wensj@alibaba-inc.com>
This commit is contained in:
Harjoth Khara 2026-08-24 07:31:01 +00:00 committed by GitHub
parent c7c4dc80e0
commit dbf7382c8f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 690 additions and 38 deletions

View file

@ -269,3 +269,8 @@ flatters every design; the floor is what makes the headline readable.
- Recall can see older documents outside the shared 200-document scanner cap,
but non-recall callers, including Forget, keep the existing capped scanner.
A broader manageability pass is separate from this recall-only change.
Superseded for Forget: issue #9378 moved Forget to the uncapped scanner, with
a per-scope bound on the model-selection prompt so literal matches in each
scope reach the model first; any match, literal or semantic, ranked past that
bound is not offered to the model and can still be missed. Indexer, Status,
and Extraction remain capped.

View file

@ -83,7 +83,12 @@ is kept.
Forget, Indexer, Status, and Extraction keep the capped scanner. That preserves
their current behavior but means an older document can become recallable before
it becomes manageable by those non-recall flows.
it becomes manageable by those non-recall flows. Superseded for Forget: issue
#9378 moved Forget to the uncapped scanner, with its own per-scope bound on the
model-selection prompt: each scope keeps a 200-candidate quota, unused quota is
redistributed, and literal query matches rank first within a scope. Entries past
that bound are not offered to the model. Indexer, Status, and Extraction remain
capped.
## Failure and compatibility boundaries

View file

@ -105,6 +105,10 @@ function truncate(text: string, maxChars: number): string {
}
async function buildTopicSummaryBlock(projectRoot: string): Promise<string> {
// Deliberately capped, unlike recall (recall.ts) and forget (forget.ts):
// every doc is rendered into the extraction agent's task prompt below, so
// an uncapped scan would grow that prompt without bound. Anything past the
// cap stays reachable — the agent holds read_file/grep/glob/ls.
// User-level scan is best-effort: a read failure on `~/.qwen/memories/`
// must not deny the extraction agent its view of existing project-level
// memories (which it uses to avoid creating duplicates).

View file

@ -11,10 +11,11 @@ import * as path from 'node:path';
import type { Config } from '../config/config.js';
import { runSideQuery } from '../utils/sideQuery.js';
import {
scanAutoMemoryTopicDocuments,
scanUserAutoMemoryTopicDocuments,
scanAllAutoMemoryTopicDocuments,
scanAllUserAutoMemoryTopicDocuments,
} from './scan.js';
import {
forgetManagedAutoMemoryEntries,
forgetManagedAutoMemoryMatches,
selectManagedAutoMemoryForgetCandidates,
} from './forget.js';
@ -31,9 +32,10 @@ vi.mock('../utils/sideQuery.js', () => ({
runSideQuery: vi.fn(),
}));
vi.mock('./scan.js', () => ({
scanAutoMemoryTopicDocuments: vi.fn(),
scanUserAutoMemoryTopicDocuments: vi.fn(),
vi.mock('./scan.js', async (importOriginal) => ({
...(await importOriginal<typeof import('./scan.js')>()),
scanAllAutoMemoryTopicDocuments: vi.fn(),
scanAllUserAutoMemoryTopicDocuments: vi.fn(),
}));
describe('selectManagedAutoMemoryForgetCandidates', () => {
@ -46,8 +48,8 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
vi.resetAllMocks();
vi.mocked(mockConfig.getModel).mockReturnValue('main-model');
vi.mocked(mockConfig.getFastModel).mockReturnValue('fast-model');
vi.mocked(scanUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAutoMemoryTopicDocuments).mockResolvedValue([
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue([
{
type: 'user',
filePath: '/tmp/auto/user/note.md',
@ -85,6 +87,384 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
);
});
it('bounds the model prompt but keeps a matching entry that ranks past the bound', async () => {
// 500 documents: the newest 499 are noise, the oldest one matches the
// query. A plain recency slice would drop it; the bound must not.
const docs = Array.from({ length: 499 }, (_, index) => ({
type: 'reference' as const,
filePath: `/tmp/project/memory/reference/noise-${index}.md`,
relativePath: `reference/noise-${index}.md`,
filename: `noise-${index}.md`,
title: `Noise ${index}`,
description: 'Unrelated',
body: 'Unrelated historical note',
mtimeMs: 1_000 + index,
}));
docs.push({
type: 'reference' as const,
filePath: '/tmp/project/memory/reference/overflow.md',
relativePath: 'reference/overflow.md',
filename: 'overflow.md',
title: 'Overflow',
description: 'Oldest',
body: 'the saved codeword is overflow-zephyr-7040',
mtimeMs: 1,
});
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(docs);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(runSideQuery).mockResolvedValue({ selectedCandidateIds: [] });
await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'overflow-zephyr-7040',
{ config: mockConfig },
);
const options = vi.mocked(runSideQuery).mock.calls[0]?.[1];
const prompt = options?.contents[0]?.parts?.[0]?.text ?? '';
expect(prompt.match(/^id: /gm)).toHaveLength(400);
expect(prompt).toContain('id: project:reference/overflow.md');
// Pins the recency order of the filler: noise-498 is the newest and must
// be kept, noise-0 falls outside the remaining slots and must not be.
expect(prompt).toContain('id: project:reference/noise-498.md');
expect(prompt).not.toContain('id: project:reference/noise-0.md');
});
it('keeps every scope represented in the model prompt when one scope is much newer', async () => {
// 400 project entries, all newer than the 3 user entries, and a query that
// matches none of them literally. A single global recency budget would
// seat 400 project entries and zero user entries, making user memory
// unselectable while recall can still inject it.
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(
Array.from({ length: 400 }, (_, index) => ({
type: 'reference' as const,
filePath: `/tmp/project/memory/reference/proj-${index}.md`,
relativePath: `reference/proj-${index}.md`,
filename: `proj-${index}.md`,
title: `Project ${index}`,
description: 'Unrelated',
body: 'Unrelated project note',
mtimeMs: 10_000 + index,
})),
);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue(
Array.from({ length: 3 }, (_, index) => ({
type: 'user' as const,
filePath: `/tmp/user/memories/user/old-${index}.md`,
relativePath: `user/old-${index}.md`,
filename: `old-${index}.md`,
title: `Old ${index}`,
description: 'Oldest',
body: 'An old cross-project preference',
mtimeMs: index + 1,
})),
);
vi.mocked(runSideQuery).mockResolvedValue({ selectedCandidateIds: [] });
await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'that cross-project preference I mentioned',
{ config: mockConfig },
);
const options = vi.mocked(runSideQuery).mock.calls[0]?.[1];
const prompt = options?.contents[0]?.parts?.[0]?.text ?? '';
expect(prompt.match(/^id: /gm)).toHaveLength(400);
for (let index = 0; index < 3; index++) {
expect(prompt).toContain(`id: user:user/old-${index}.md`);
}
});
it('falls back to the full uncapped candidate list when the model fails', async () => {
const docs = Array.from({ length: 500 }, (_, index) => ({
type: 'reference' as const,
filePath: `/tmp/project/memory/reference/noise-${index}.md`,
relativePath: `reference/noise-${index}.md`,
filename: `noise-${index}.md`,
title: `Noise ${index}`,
description: 'Unrelated',
body: 'Unrelated historical note',
mtimeMs: 1_000 + index,
}));
docs.push({
type: 'reference' as const,
filePath: '/tmp/project/memory/reference/overflow.md',
relativePath: 'reference/overflow.md',
filename: 'overflow.md',
title: 'Overflow',
description: 'Oldest',
body: 'the saved codeword is overflow-zephyr-7040',
mtimeMs: 1,
});
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(docs);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(runSideQuery).mockRejectedValue(new Error('side query failed'));
const result = await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'overflow-zephyr-7040',
{ config: mockConfig },
);
expect(result.strategy).toBe('heuristic');
expect(result.matches.map((match) => match.filePath)).toContain(
'/tmp/project/memory/reference/overflow.md',
);
});
it('bounds how much the unconfirmed forget path can delete at once', async () => {
// forgetManagedAutoMemoryEntries (MemoryManager.forget, the ACP path)
// deletes without confirmation. With an uncapped scan and a heuristic
// fallback that substring-matches the whole store, an unbounded limit
// would let a very short query wipe everything.
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'forget-cap-'));
const originalMemoryBase = process.env['QWEN_CODE_MEMORY_BASE_DIR'];
process.env['QWEN_CODE_MEMORY_BASE_DIR'] = path.join(tempDir, 'memory');
clearAutoMemoryRootCache();
try {
const projectRoot = path.join(tempDir, 'project');
const docsDir = path.join(tempDir, 'docs');
await fs.mkdir(projectRoot, { recursive: true });
await fs.mkdir(docsDir, { recursive: true });
const docs = await Promise.all(
Array.from({ length: 401 }, async (_, index) => {
const body = `forgettable-marker note ${index}`;
const filePath = path.join(docsDir, `doc-${index}.md`);
await fs.writeFile(
filePath,
[
'---',
'type: reference',
`name: Doc ${index}`,
'---',
'',
body,
].join('\n'),
'utf-8',
);
return {
type: 'reference' as const,
filePath,
relativePath: `reference/doc-${index}.md`,
filename: `doc-${index}.md`,
title: `Doc ${index}`,
description: 'Matching',
body,
mtimeMs: 1_000 + index,
};
}),
);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(docs);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(runSideQuery).mockRejectedValue(new Error('side query failed'));
const result = await forgetManagedAutoMemoryEntries(
projectRoot,
'forgettable-marker',
{ config: mockConfig },
);
expect(result.removedEntries).toHaveLength(400);
const survivors = (await fs.readdir(docsDir)).length;
expect(survivors).toBe(1);
} finally {
if (originalMemoryBase === undefined) {
delete process.env['QWEN_CODE_MEMORY_BASE_DIR'];
} else {
process.env['QWEN_CODE_MEMORY_BASE_DIR'] = originalMemoryBase;
}
clearAutoMemoryRootCache();
await fs.rm(tempDir, { recursive: true, force: true });
}
});
it('splits the prompt evenly when both scopes are over quota', async () => {
// Both scopes over the 200 quota, so no spare is redistributed and the
// quota itself decides the split. Mutating the quota changes these counts.
const makeDocs = (scope: 'user' | 'project', dir: string, base: number) =>
Array.from({ length: 300 }, (_, index) => ({
type: (scope === 'user' ? 'user' : 'reference') as 'user' | 'reference',
filePath: `${dir}/doc-${index}.md`,
relativePath: `${scope === 'user' ? 'user' : 'reference'}/doc-${index}.md`,
filename: `doc-${index}.md`,
title: `Doc ${index}`,
description: 'Unrelated',
body: 'Unrelated note',
mtimeMs: base + index,
}));
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue(
makeDocs('user', '/tmp/user/memories/user', 1_000),
);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(
makeDocs('project', '/tmp/project/memory/reference', 500_000),
);
vi.mocked(runSideQuery).mockResolvedValue({ selectedCandidateIds: [] });
await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'something that matches nothing literally',
{ config: mockConfig },
);
const options = vi.mocked(runSideQuery).mock.calls[0]?.[1];
const prompt = options?.contents[0]?.parts?.[0]?.text ?? '';
expect(prompt.match(/^scope: user$/gm)).toHaveLength(200);
expect(prompt.match(/^scope: project$/gm)).toHaveLength(200);
});
it('splits deletion seats per scope when matches exceed the limit', async () => {
// 450 user-scope and 50 project-scope entries all match, the side query is
// down, and the limit is the deletion ceiling. listIndexedForgetCandidates
// pushes user before project, so a plain slice would take 400 user entries
// and zero project ones while reporting a successful forget.
const matching = (
scope: 'user' | 'project',
dir: string,
count: number,
base: number,
) =>
Array.from({ length: count }, (_, index) => ({
type: (scope === 'user' ? 'user' : 'reference') as 'user' | 'reference',
filePath: `${dir}/doc-${index}.md`,
relativePath: `${scope === 'user' ? 'user' : 'reference'}/doc-${index}.md`,
filename: `doc-${index}.md`,
title: `Doc ${index}`,
description: 'Matching',
body: 'the saved codeword is overflow-zephyr-7040',
mtimeMs: base + index,
}));
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue(
matching('user', '/tmp/user/memories/user', 450, 1_000),
);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(
matching('project', '/tmp/project/memory/reference', 50, 1),
);
vi.mocked(runSideQuery).mockRejectedValue(new Error('side query failed'));
const result = await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'overflow-zephyr-7040',
{ config: mockConfig, limit: 400 },
);
expect(result.matches).toHaveLength(400);
const projectMatches = result.matches.filter((match) =>
match.filePath.startsWith('/tmp/project/'),
);
// Every project entry keeps its seat; user scope absorbs the rest.
expect(projectMatches).toHaveLength(50);
});
it('keeps the newest of a scope when its own matches overflow the quota', async () => {
// Both scopes over quota and every entry matches literally, so the ranking
// inside the matched set decides who is seated. Oldest-first would drop the
// newest entries instead.
const matching = (scope: 'user' | 'project', dir: string) =>
Array.from({ length: 300 }, (_, index) => ({
type: (scope === 'user' ? 'user' : 'reference') as 'user' | 'reference',
filePath: `${dir}/doc-${index}.md`,
relativePath: `${scope === 'user' ? 'user' : 'reference'}/doc-${index}.md`,
filename: `doc-${index}.md`,
title: `Doc ${index}`,
description: 'Matching',
body: 'the saved codeword is overflow-zephyr-7040',
mtimeMs: 1_000 + index,
}));
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue(
matching('user', '/tmp/user/memories/user'),
);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(
matching('project', '/tmp/project/memory/reference'),
);
vi.mocked(runSideQuery).mockResolvedValue({ selectedCandidateIds: [] });
await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'overflow-zephyr-7040',
{ config: mockConfig },
);
const options = vi.mocked(runSideQuery).mock.calls[0]?.[1];
const prompt = options?.contents[0]?.parts?.[0]?.text ?? '';
expect(prompt.match(/^scope: user$/gm)).toHaveLength(200);
// doc-299 is the newest of its scope and must be seated; doc-0 the oldest
// and must not be.
expect(prompt).toContain('id: user:user/doc-299.md');
expect(prompt).not.toContain('id: user:user/doc-0.md');
});
it("scales the per-scope split to a small limit and takes each scope's newest", async () => {
// The deletion path with /forget's default-sized limit. Pins two things the
// 400-limit cases cannot: that the split is derived from the budget rather
// than a fixed 200 quota, and the direction of selectByHeuristic's own
// recency comparator (the model path never runs here).
const matching = (scope: 'user' | 'project', dir: string) =>
Array.from({ length: 300 }, (_, index) => ({
type: (scope === 'user' ? 'user' : 'reference') as 'user' | 'reference',
filePath: `${dir}/doc-${index}.md`,
relativePath: `${scope === 'user' ? 'user' : 'reference'}/doc-${index}.md`,
filename: `doc-${index}.md`,
title: `Doc ${index}`,
description: 'Matching',
body: 'the saved codeword is overflow-zephyr-7040',
mtimeMs: 1_000 + index,
}));
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue(
matching('user', '/tmp/user/memories/user'),
);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(
matching('project', '/tmp/project/memory/reference'),
);
vi.mocked(runSideQuery).mockRejectedValue(new Error('side query failed'));
const result = await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'overflow-zephyr-7040',
{ config: mockConfig, limit: 5 },
);
expect(result.strategy).toBe('heuristic');
// 5 seats over 2 scopes: 2 each, then the odd seat to the first scope.
expect(result.matches).toHaveLength(5);
const paths = result.matches.map((match) => match.filePath);
expect(paths.filter((p) => p.startsWith('/tmp/user/'))).toHaveLength(3);
expect(paths.filter((p) => p.startsWith('/tmp/project/'))).toHaveLength(2);
// Newest of each scope, not oldest.
expect(paths).toContain('/tmp/user/memories/user/doc-299.md');
expect(paths).toContain('/tmp/project/memory/reference/doc-299.md');
expect(paths).not.toContain('/tmp/user/memories/user/doc-0.md');
});
it('gives the heuristic fallback the full list, not the bounded one', async () => {
// 450 literal matches with a limit above that: handing the fallback the
// 400-candidate prompt budget instead of the full list would silently
// leave 50 entries undeleted after a model failure.
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue(
Array.from({ length: 450 }, (_, index) => ({
type: 'reference' as const,
filePath: `/tmp/project/memory/reference/match-${index}.md`,
relativePath: `reference/match-${index}.md`,
filename: `match-${index}.md`,
title: `Match ${index}`,
description: 'Matching',
body: 'the saved codeword is overflow-zephyr-7040',
mtimeMs: 1_000 + index,
})),
);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(runSideQuery).mockRejectedValue(new Error('side query failed'));
const result = await selectManagedAutoMemoryForgetCandidates(
'/tmp/project',
'overflow-zephyr-7040',
{ config: mockConfig, limit: 500 },
);
expect(result.strategy).toBe('heuristic');
expect(result.matches).toHaveLength(450);
});
it('wraps the forget query as user data in the selector prompt', async () => {
vi.mocked(runSideQuery).mockResolvedValue({
selectedCandidateIds: [],
@ -105,7 +485,7 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
});
it('indexes user and project candidates with scope-prefixed ids', async () => {
vi.mocked(scanUserAutoMemoryTopicDocuments).mockResolvedValue([
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([
{
type: 'user',
filePath: '/tmp/user/memories/user/note.md',
@ -117,7 +497,7 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
mtimeMs: 2,
},
]);
vi.mocked(scanAutoMemoryTopicDocuments).mockResolvedValue([
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue([
{
type: 'project',
filePath: '/tmp/project/memory/user/note.md',
@ -163,8 +543,8 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
});
it('can select user-level memories through heuristic search', async () => {
vi.mocked(scanAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanUserAutoMemoryTopicDocuments).mockResolvedValue([
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([
{
type: 'user',
filePath: '/tmp/user/memories/user/editor.md',
@ -393,8 +773,8 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
].join('\n'),
'utf-8',
);
vi.mocked(scanAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
const result = await forgetManagedAutoMemoryMatches(
projectRoot,
@ -474,8 +854,8 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
].join('\n'),
'utf-8',
);
vi.mocked(scanAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
const result = await forgetManagedAutoMemoryMatches(
projectRoot,
@ -552,8 +932,8 @@ describe('selectManagedAutoMemoryForgetCandidates', () => {
'utf-8',
);
await fs.mkdir(getUserAutoMemoryIndexPath(), { recursive: true });
vi.mocked(scanAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanUserAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllAutoMemoryTopicDocuments).mockResolvedValue([]);
vi.mocked(scanAllUserAutoMemoryTopicDocuments).mockResolvedValue([]);
const result = await forgetManagedAutoMemoryMatches(
projectRoot,

View file

@ -27,14 +27,48 @@ import {
isUserAutoMemPath,
} from './paths.js';
import {
scanAutoMemoryTopicDocuments,
scanUserAutoMemoryTopicDocuments,
scanAllAutoMemoryTopicDocuments,
scanAllUserAutoMemoryTopicDocuments,
} from './scan.js';
import { ensureAutoMemoryScaffold } from './store.js';
import type { AutoMemoryMetadata, AutoMemoryType } from './types.js';
const debugLogger = createDebugLogger('MEMORY_FORGET');
/**
* Per-scope share of the model-selection prompt. The capped scanners this
* replaced ran per scope, so each scope was guaranteed seats however old its
* entries were relative to the other scope's. Ranking both scopes into one
* global budget would drop that guarantee: on a store whose project entries
* are all newer than its user entries, user memory would get no seats at all
* and become unselectable while recall could still inject it.
*/
const MAX_MODEL_FORGET_CANDIDATES_PER_SCOPE = 200;
/**
* Upper bound on the candidates interpolated into the model-selection prompt.
* The scan is uncapped, but `buildForgetSelectionPrompt` renders every
* candidate, so the prompt needs its own bound. This is the exposure the
* capped scan already permitted at steady state: 200 project + 200 user
* documents at the one-memory-per-file the extraction format writes.
*/
const MAX_MODEL_FORGET_CANDIDATES = MAX_MODEL_FORGET_CANDIDATES_PER_SCOPE * 2;
/**
* Per-call deletion ceiling for `forgetManagedAutoMemoryEntries`, the path
* MemoryManager.forget and ACP take, which deletes without confirmation.
*
* Deliberately its own literal rather than an alias of the prompt bound: the
* two happen to agree today, but resizing the model prompt is a cost decision
* and resizing this is a blast-radius decision. Coupling them would let a
* prompt-sizing change silently widen how much one unconfirmed call can
* delete.
*/
const MAX_UNCONFIRMED_FORGET_DELETIONS = 400;
/** The scopes a forget candidate can come from, in prompt order. */
const FORGET_SCOPES: readonly AutoMemoryStorageScope[] = ['user', 'project'];
export interface AutoMemoryForgetMatch {
topic: AutoMemoryType;
summary: string;
@ -62,6 +96,7 @@ interface IndexedForgetCandidate extends AutoMemoryForgetMatch {
storageScope: AutoMemoryStorageScope;
why?: string;
howToApply?: string;
mtimeMs: number;
}
export type AutoMemoryStorageScope = 'user' | 'project';
@ -94,9 +129,14 @@ async function listIndexedForgetCandidates(
abortSignal?: AbortSignal,
): Promise<IndexedForgetCandidate[]> {
abortSignal?.throwIfAborted();
// Uncapped, to match the recall universe (recall.ts scans uncapped): an
// entry that recall can inject must be one that forget can remove.
const [projectDocs, userDocs] = await Promise.all([
scanAutoMemoryTopicDocuments(projectRoot),
scanUserAutoMemoryTopicDocuments(),
scanAllAutoMemoryTopicDocuments(projectRoot),
// Deliberately NOT best-effort, unlike recall.ts: a scan failure here must
// stay loud. Swallowing it would report "no entries matched" for a scope
// that was never read, and forget acts on that answer by deleting.
scanAllUserAutoMemoryTopicDocuments(),
]);
abortSignal?.throwIfAborted();
const candidates: IndexedForgetCandidate[] = [];
@ -124,6 +164,7 @@ async function listIndexedForgetCandidates(
entryIndex: i,
why: entry.why,
howToApply: entry.howToApply,
mtimeMs: doc.mtimeMs,
});
}
}
@ -164,6 +205,102 @@ function buildForgetSelectionPrompt(
].join('\n');
}
/**
* Shared by the prompt bound and the heuristic so the two cannot drift, and
* delegated to `normalizeSummary` so query matching and the post-selection
* re-match in `forgetManagedAutoMemoryMatches` normalize text identically.
*/
function normalizeForgetQuery(query: string): string {
return normalizeSummary(query);
}
/** Shared by the prompt bound and the heuristic so the two cannot drift. */
function matchesForgetQuery(
candidate: IndexedForgetCandidate,
queryLower: string,
): boolean {
return buildAutoMemoryEntrySearchText(candidate).includes(queryLower);
}
/**
* Shared by the prompt ranking and the heuristic. One definition, because the
* model prompt must rank candidates in the same order the heuristic fallback
* deletes in, and each site has its own test: a one-sided change updates its
* own test, passes CI, and leaves the sibling silently stale.
*/
const byMtimeMsDesc = (a: IndexedForgetCandidate, b: IndexedForgetCandidate) =>
b.mtimeMs - a.mtimeMs;
/** Literal query matches first, then the rest, each newest first. */
function rankScopeForPrompt(
scopeCandidates: IndexedForgetCandidate[],
queryLower: string,
): IndexedForgetCandidate[] {
const matched = scopeCandidates
.filter((candidate) => matchesForgetQuery(candidate, queryLower))
.sort(byMtimeMsDesc);
const rest = scopeCandidates
.filter((candidate) => !matchesForgetQuery(candidate, queryLower))
.sort(byMtimeMsDesc);
return [...matched, ...rest];
}
/**
* Give every scope an equal share of `budget` before letting any scope's
* recency crowd another out, then hand whatever a smaller scope leaves unused
* to the rest. Without this, one scope's entries being uniformly newer takes
* every seat and the other scope becomes unreachable.
*/
function allocatePerScope<T>(rankedByScope: T[][], budget: number): T[] {
const perScope = Math.floor(budget / rankedByScope.length);
const take = rankedByScope.map((scopeRanked) =>
Math.min(scopeRanked.length, perScope),
);
let spare = budget - take.reduce((sum, n) => sum + n, 0);
for (let i = 0; i < rankedByScope.length && spare > 0; i++) {
const extra = Math.min(spare, rankedByScope[i].length - take[i]);
take[i] += extra;
spare -= extra;
}
return rankedByScope.flatMap((scopeRanked, i) =>
scopeRanked.slice(0, take[i]),
);
}
/**
* Bound the model prompt. Each scope keeps its own quota so a scope whose
* entries are all older than the other's still reaches the model, and whatever
* a smaller scope leaves unused is handed to the other. Within a scope,
* literal query matches rank ahead of the rest and both are ordered newest
* first, so truncation is deterministic rather than scan-order.
*
* Entries past the bound are not offered to the model at all. The heuristic
* fallback rescues only the model-failure path: a successful selection that
* returns nothing short-circuits with 'none' and never consults the full list.
*/
function selectModelForgetCandidates(
candidates: IndexedForgetCandidate[],
query: string,
): IndexedForgetCandidate[] {
if (candidates.length <= MAX_MODEL_FORGET_CANDIDATES) {
return candidates;
}
const queryLower = normalizeForgetQuery(query);
const ranked = FORGET_SCOPES.map((scope) =>
rankScopeForPrompt(
candidates.filter((candidate) => candidate.storageScope === scope),
queryLower,
),
);
const selected = allocatePerScope(ranked, MAX_MODEL_FORGET_CANDIDATES);
debugLogger.warn(
`Managed auto-memory forget prompt bounded to ${selected.length} of ` +
`${candidates.length} candidates; entries past the bound cannot be ` +
`selected by the model.`,
);
return selected;
}
async function selectByModel(
candidates: IndexedForgetCandidate[],
query: string,
@ -229,19 +366,37 @@ function selectByHeuristic(
query: string,
limit: number,
): AutoMemoryForgetSelectionResult {
const normalizedQuery = query.replace(/\s+/g, ' ').trim();
const queryLower = normalizedQuery.toLowerCase();
const matches = candidates
.filter((candidate) =>
buildAutoMemoryEntrySearchText(candidate).includes(queryLower),
)
.slice(0, limit)
.map(({ topic, summary, filePath, entryIndex }) => ({
topic,
summary,
filePath,
entryIndex,
}));
const queryLower = normalizeForgetQuery(query);
const matched = candidates.filter((candidate) =>
matchesForgetQuery(candidate, queryLower),
);
if (matched.length > limit) {
// The sibling prompt bound warns when it binds; this one deletes, so
// staying silent leaves no record of why entries recall still injects
// survived a forget that reported success.
debugLogger.warn(
`Managed auto-memory forget matched ${matched.length} entries but the ` +
`limit is ${limit}; ${matched.length - limit} matching entries were ` +
`not deleted.`,
);
}
// Same per-scope split the model path uses. `listIndexedForgetCandidates`
// pushes every user entry before any project entry, so a plain slice hands
// all `limit` deletion seats to user scope once matches exceed it, deleting
// nothing from the other scope while still reporting success.
const matches = allocatePerScope(
FORGET_SCOPES.map((scope) =>
matched
.filter((candidate) => candidate.storageScope === scope)
.sort(byMtimeMsDesc),
),
limit,
).map(({ topic, summary, filePath, entryIndex }) => ({
topic,
summary,
filePath,
entryIndex,
}));
return {
matches,
@ -271,7 +426,7 @@ export async function selectManagedAutoMemoryForgetCandidates(
if (options.config) {
try {
return await selectByModel(
candidates,
selectModelForgetCandidates(candidates, query),
query,
options.config,
limit,
@ -495,7 +650,12 @@ export async function forgetManagedAutoMemoryEntries(
const selection = await selectManagedAutoMemoryForgetCandidates(
projectRoot,
trimmedQuery,
{ ...options, limit: Number.MAX_SAFE_INTEGER },
// Bounded, not MAX_SAFE_INTEGER: this path deletes without confirmation,
// and the heuristic fallback substring-matches the whole store, so a very
// short query matches nearly everything. The capped scanners this replaced
// held the same failure to one scan's worth of candidates; keep that
// ceiling rather than letting an uncapped scan widen it.
{ ...options, limit: MAX_UNCONFIRMED_FORGET_DELETIONS },
);
const result = await forgetManagedAutoMemoryMatches(
projectRoot,

View file

@ -14,7 +14,15 @@ import { runManagedAutoMemoryDream } from './dream.js';
import { planManagedAutoMemoryDreamByAgent } from './dreamAgentPlanner.js';
import { MemoryManager } from './manager.js';
import { rebuildManagedAutoMemoryIndex } from './indexer.js';
import { getAutoMemoryFilePath, getAutoMemoryIndexPath } from './paths.js';
import {
clearAutoMemoryRootCache,
getAutoMemoryFilePath,
getAutoMemoryIndexPath,
} from './paths.js';
import {
forgetManagedAutoMemoryMatches,
selectManagedAutoMemoryForgetCandidates,
} from './forget.js';
import { resolveRelevantAutoMemoryPromptForQuery } from './recall.js';
import { scanAutoMemoryTopicDocuments } from './scan.js';
import { ensureAutoMemoryScaffold } from './store.js';
@ -290,4 +298,94 @@ describe('managed auto-memory lifecycle integration', () => {
);
expect(recall.prompt).toContain('OVERFLOW-ZEPHYR-7040');
});
it('forgets a topic beyond the general 200-document scan cap', async () => {
// Hermetic: forget deletes files, so the user-level scan must never reach
// the real `~/.qwen/memories`.
const originalMemoryBase = process.env['QWEN_CODE_MEMORY_BASE_DIR'];
process.env['QWEN_CODE_MEMORY_BASE_DIR'] = path.join(tempDir, 'memory');
clearAutoMemoryRootCache();
try {
await ensureAutoMemoryScaffold(
projectRoot,
new Date('2026-04-01T00:00:00.000Z'),
);
const referenceDir = path.dirname(
getAutoMemoryFilePath(projectRoot, 'reference/filler-000.md'),
);
await fs.mkdir(referenceDir, { recursive: true });
await Promise.all(
Array.from({ length: 200 }, (_, index) =>
fs.writeFile(
path.join(
referenceDir,
`filler-${String(index).padStart(3, '0')}.md`,
),
[
'---',
'type: reference',
`name: Filler ${index}`,
'description: Unrelated historical note',
'---',
'',
'Unrelated historical note.',
].join('\n'),
'utf-8',
),
),
);
const targetPath = getAutoMemoryFilePath(
projectRoot,
'reference/overflow-target.md',
);
await fs.writeFile(
targetPath,
[
'---',
'type: reference',
'name: Overflow Zephyr Marker',
'description: Unique forget target beyond the general scan cap',
'---',
'',
'The saved codeword is OVERFLOW-ZEPHYR-7040.',
].join('\n'),
'utf-8',
);
// Oldest mtime, so the target ranks 201st and the capped scan drops it.
await fs.utimes(targetPath, new Date(0), new Date(0));
const cappedDocs = await scanAutoMemoryTopicDocuments(projectRoot);
expect(cappedDocs).toHaveLength(200);
expect(cappedDocs.some((doc) => doc.filePath === targetPath)).toBe(false);
// Recall can surface it (uncapped scan), so forget must be able to
// remove it.
const recall = await resolveRelevantAutoMemoryPromptForQuery(
projectRoot,
'What is the overflow zephyr codeword?',
);
expect(recall.selectedDocs.map((doc) => doc.filePath)).toContain(
targetPath,
);
const selection = await selectManagedAutoMemoryForgetCandidates(
projectRoot,
'overflow-zephyr-7040',
);
expect(selection.matches.map((match) => match.filePath)).toContain(
targetPath,
);
await forgetManagedAutoMemoryMatches(projectRoot, selection.matches);
await expect(fs.access(targetPath)).rejects.toThrow();
} finally {
if (originalMemoryBase === undefined) {
delete process.env['QWEN_CODE_MEMORY_BASE_DIR'];
} else {
process.env['QWEN_CODE_MEMORY_BASE_DIR'] = originalMemoryBase;
}
clearAutoMemoryRootCache();
}
});
});