kimi-code/packages/agent-core/test/agent/tool.test.ts
Kai 65d30177ad
feat(agent-core): record llm request trace in wire.jsonl (#1448)
* feat(agent-core): record llm request trace in wire.jsonl

Add three observability record types so every request sent to the model
can be reconstructed from the wire log at the logical-request level:

- llm.tools_snapshot: content-addressed snapshot of the top-level tools
  table as sent (post deferred-strip), written once per unique table
- llm.request: one record per outbound request (retries, strict resends,
  and compaction rounds included) carrying the effective request params
  and hash links to the system prompt and tools snapshot
- mcp.tools_discovered: the server's verbatim tools/list result plus the
  agent's gating (allow-list, collisions), deduplicated by content hash

Observability records never feed state rebuild; replay only restores the
write-dedup cursors. The records/types.ts contract now documents the two
record classes explicitly (persisted is not the same as replayed).

Recording happens at the single Agent.generate choke point. The
LLMRequestLogFields side channel gains kind/projection/maxTokens/
droppedCount, chatWithRetry preserves caller-set fields, and compaction
tags its requests. The vis wire view renders the new record kinds.

* fix(agent-core): record the provider-clamped completion cap in the request trace

The llm.request trace recorded the client-requested budget cap, but
chat-completions providers tighten the actual wire value inside
withMaxCompletionTokens (remaining-context sizing, transport ceilings,
model-default resolution) — with the default budget the clamp is active
on nearly every non-empty-context request, so the recorded value did not
match what was sent.

Providers now expose the effective cap they computed as a readonly
maxCompletionTokens field on the clone, and the recorder reads it from
the effective provider at the Agent.generate choke point. This replaces
the side-channel recomputation, which is removed along with the
appliedCompletionBudgetCap helper.

* fix(agent-core): park pre-replay MCP discovery records and hash the collision outcome

Two wire-hygiene fixes for the mcp.tools_discovered trace:

Parking: the real Session ordering connects MCP servers concurrently with
agent construction, so ToolManager can observe a connected server before
agent.resume() has replayed the wire. Recording at that point bypassed
the restored dedup cursor (duplicating a 1-50KB record on every resume)
and appended a stray metadata record ahead of replay. AgentRecords now
exposes a one-shot opened latch — set when replay completes (after the
migration rewrite flushes) or when the first live record is logged — and
ToolManager parks discoveries until then, re-running the dedup check at
drain time. A frozen range-limited replay never opens; those agents are
transient previews.

Collision hashing: the dedup hash now covers the collision outcome, not
just the raw list and allow-list. Collisions depend on which other
servers hold a sanitized qualified name at registration time, so a
server can re-register with identical tools but a flipped outcome; that
gating change must produce a new record instead of being suppressed.

* fix(agent-core): skip the request trace for pre-flight-aborted calls

Mirror kosong generate()'s pre-flight abort check at the Agent.generate
choke point: a call whose signal is already aborted never reaches the
wire (generate throws before dispatching), so it must not leave an
llm.request/llm.tools_snapshot trace or a diagnostic log line claiming a
request was sent. Recording stays before dispatch for every call that
passes the gate, preserving the crash-safety of the trace.

* chore(agent-core): remove a leftover adaptive-thinking override hook

The adaptiveThinkingOverride option was a temporary local hook explicitly
marked for removal before commit. Nothing passes it, so resolution falls
back to the alias-level adaptiveThinking value in all cases; drop the
option and the dead indirection.

* fix(kosong): derive the exposed completion cap from generation kwargs

maxCompletionTokens was a field stored only by withMaxCompletionTokens,
so caps that reach the wire through other paths were invisible to the
request trace: with completion budgeting disabled via env, Anthropic
still sends the constructor-resolved max_tokens (required by the
Messages API), and constructor-level kwargs like OpenAILegacyOptions
maxTokens were likewise unreported.

Replace the stored field with a getter derived from each provider's
generation kwargs — the single source the request body reads — covering
constructor defaults, direct withGenerationKwargs configuration, and
budget application in one place. Kimi mirrors its request-time legacy
max_tokens alias normalization; openai-legacy reuses the same
normalizeGenerationKwargs the request path uses.

* feat(agent-core): add thinkingKeep passthrough for Kimi providers and update tests
2026-07-07 14:09:19 +08:00

533 lines
26 KiB
TypeScript

import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { ToolCall } from '@moonshot-ai/kosong';
import { describe, expect, it, vi } from 'vitest';
import { budgetToolResultForModel } from '../../src/agent/turn/tool-result-budget';
import { HookEngine } from '../../src/session/hooks';
import type { SessionSubagentHost } from '../../src/session/subagent-host';
import { FLAG_DEFINITIONS, FlagResolver } from '../../src/flags';
import { createFakeKaos } from '../tools/fixtures/fake-kaos';
import { createCommandKaos, testAgent } from './harness/agent';
import { executeTool } from '../tools/fixtures/execute-tool';
const signal = new AbortController().signal;
describe('Agent tools', () => {
it('blocks tools through PreToolUse before permission and emits PostToolUseFailure', async () => {
const execWithEnv = vi.fn().mockRejectedValue(new Error('Bash should not execute'));
const triggered: Array<[string, string, number]> = [];
const hookEngine = new HookEngine(
[
{
event: 'PreToolUse',
matcher: 'Bash',
command: 'node -e "process.stderr.write(\'blocked by PreToolUse\'); process.exit(2)"',
},
{
event: 'PostToolUseFailure',
matcher: 'Bash',
command: 'exit 0',
},
],
{
onTriggered: (event, target, count) => {
triggered.push([event, target, count]);
},
},
);
const ctx = testAgent({
kaos: createFakeKaos({ execWithEnv }),
hookEngine,
});
ctx.configure({ tools: ['Bash'] });
ctx.mockNextResponse({ type: 'text', text: 'I will run Bash.' }, bashCall());
ctx.mockNextResponse({ type: 'text', text: 'The hook blocked Bash.' });
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Try Bash' }] });
await ctx.untilTurnEnd();
expect(execWithEnv).not.toHaveBeenCalled();
expect(triggered).toEqual([
['PreToolUse', 'Bash', 1],
['PostToolUseFailure', 'Bash', 1],
]);
expect(JSON.stringify(ctx.agent.context.data().history)).toContain('blocked by PreToolUse');
});
it('emits PostToolUse after successful tools', async () => {
const triggered: Array<[string, string, number]> = [];
const hookEngine = new HookEngine(
[
{
event: 'PostToolUse',
matcher: 'Bash',
command: 'exit 0',
},
],
{
onTriggered: (event, target, count) => {
triggered.push([event, target, count]);
},
},
);
const ctx = testAgent({
kaos: createCommandKaos('ok'),
hookEngine,
});
ctx.configure({ tools: ['Bash'] });
await ctx.rpc.setPermission({ mode: 'auto' });
ctx.mockNextResponse({ type: 'text', text: 'I will run Bash.' }, bashCall());
ctx.mockNextResponse({ type: 'text', text: 'Bash returned ok.' });
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Run Bash' }] });
await ctx.untilTurnEnd();
expect(triggered).toEqual([['PostToolUse', 'Bash', 1]]);
});
it('uses builtin descriptions on tool call start events', async () => {
const ctx = testAgent({
kaos: createCommandKaos('ok'),
});
ctx.configure({ tools: ['Bash'] });
await ctx.rpc.setPermission({ mode: 'yolo' });
ctx.mockNextResponse({ type: 'text', text: 'I will run Bash.' }, bashCall());
ctx.mockNextResponse({ type: 'text', text: 'Bash returned ok.' });
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Run Bash' }] });
await ctx.untilTurnEnd();
const started = ctx.allEvents.find(
(event) => event.type === '[rpc]' && event.event === 'tool.call.started',
);
expect(started?.args).toMatchObject({
description: 'Running: printf hook-output',
});
});
it('continues after a foreground Agent tool returns a max_tokens failure', async () => {
const completion = Promise.reject(
new Error('Subagent turn failed before completing its final summary: reason=max_tokens.'),
);
void completion.catch(() => undefined);
const subagentHost = {
spawn: vi.fn().mockResolvedValue({
agentId: 'agent-child',
profileName: 'coder',
resumed: false,
completion,
}),
resume: vi.fn(),
} as unknown as SessionSubagentHost;
const ctx = testAgent({ subagentHost });
ctx.configure({ tools: ['Agent'] });
ctx.mockNextResponse({ type: 'text', text: 'I will ask a subagent.' }, agentCall());
ctx.mockNextResponse({
type: 'text',
text: 'The subagent failed with reason=max_tokens, so I will continue in the parent turn.',
});
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Delegate and recover' }] });
await ctx.untilTurnEnd();
expect(subagentHost.spawn).toHaveBeenCalledWith(
expect.objectContaining({
profileName: 'coder',
parentToolCallId: 'call_agent',
prompt: 'Investigate deeply',
description: 'Investigate deeply',
runInBackground: false,
}),
);
expect(ctx.llmCalls).toHaveLength(2);
expect(ctx.allEvents).toContainEqual(
expect.objectContaining({
type: '[rpc]',
event: 'tool.result',
args: expect.objectContaining({
toolCallId: 'call_agent',
isError: true,
output: expect.stringContaining('reason=max_tokens'),
}),
}),
);
expect(JSON.stringify(ctx.llmCalls[1]?.history)).toContain('reason=max_tokens');
});
it('passes text from content-part error outputs to PostToolUseFailure hooks', async () => {
const lookupCall: ToolCall = {
type: 'function',
id: 'call_lookup',
name: 'Lookup',
arguments: '{"query":"moon"}',
};
const resolved: Array<[string, string, string]> = [];
const hookEngine = new HookEngine(
[
{
event: 'PostToolUseFailure',
matcher: 'Lookup',
command: hookErrorMessageAssertCommand('rich failure text'),
},
],
{
onResolved: (event, target, action) => {
resolved.push([event, target, action]);
},
},
);
const ctx = testAgent({ hookEngine });
ctx.configure();
await ctx.rpc.setPermission({ mode: 'auto' });
await ctx.rpc.registerTool({
name: 'Lookup',
description: 'Look up a short test value.',
parameters: {
type: 'object',
properties: {
query: { type: 'string' },
},
required: ['query'],
additionalProperties: false,
},
});
ctx.mockNextResponse({ type: 'text', text: 'I will look it up.' }, lookupCall);
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Look up moon' }] });
await ctx.untilToolCall({
isError: true,
output: [{ type: 'text', text: 'rich failure text' }],
});
ctx.mockNextResponse({ type: 'text', text: 'The lookup failed.' });
await ctx.untilTurnEnd();
await vi.waitFor(() => {
expect(resolved).toEqual([['PostToolUseFailure', 'Lookup', 'allow']]);
});
});
it('uses the active builtin tool set as the LLM visible tools', async () => {
const ctx = testAgent();
ctx.configure({ tools: ['Write', 'Bash'] });
ctx.mockNextResponse({ type: 'text', text: 'ready' });
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Which tools are active?' }] });
await ctx.untilTurnEnd();
expect(ctx.lastLlmInput()).toMatchInlineSnapshot(`
system: <system-prompt>
tools: Bash, Write
messages:
user: text "Which tools are active?"
`);
await ctx.expectResumeMatches();
});
it('disables Bash background mode unless task management tools are active', async () => {
const ctx = testAgent();
ctx.configure({ tools: ['Bash'] });
const bashOnly = ctx.agent.tools.loopTools.find((tool) => tool.name === 'Bash');
expect(bashOnly).toBeDefined();
expect(bashOnly!.description).toContain('Background execution is disabled for this agent.');
expect(bashOnly!.description).not.toContain('the command will be started as a background task');
await expect(
executeTool(bashOnly!, {
turnId: '0',
toolCallId: 'call_bash',
args: { command: 'sleep 10', run_in_background: true, description: 'watch' },
signal,
}),
).resolves.toMatchObject({
isError: true,
output:
'Background execution is not available for this agent because TaskOutput and TaskStop are not enabled.',
});
ctx.agent.tools.setActiveTools(['Bash', 'TaskList', 'TaskOutput', 'TaskStop']);
const managedBash = ctx.agent.tools.loopTools.find((tool) => tool.name === 'Bash');
expect(managedBash).toBeDefined();
expect(managedBash!.description).toContain('run_in_background=true');
});
it('exposes AgentSwarm when a subagent host is available', () => {
const subagentHost = {} as unknown as SessionSubagentHost;
const ctx = testAgent({
subagentHost,
experimentalFlags: new FlagResolver({}, FLAG_DEFINITIONS),
});
ctx.configure({ tools: ['AgentSwarm'] });
expect(ctx.agent.tools.loopTools.some((tool) => tool.name === 'AgentSwarm')).toBe(true);
});
it('routes registered user tools through tool.call request/response', async () => {
const lookupCall: ToolCall = {
type: 'function',
id: 'call_lookup',
name: 'Lookup',
arguments: '{"query":"moon"}',
};
const ctx = testAgent();
ctx.configure();
await ctx.rpc.setPermission({ mode: 'auto' });
await ctx.rpc.registerTool({
name: 'Lookup',
description: 'Look up a short test value.',
parameters: {
type: 'object',
properties: {
query: { type: 'string' },
},
required: ['query'],
additionalProperties: false,
},
});
ctx.mockNextResponse({ type: 'text', text: 'I will look it up.' }, lookupCall);
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Look up moon' }] });
expect(
await ctx.untilToolCall({
content: 'moon-result',
output: 'moon-result',
}),
).toMatchInlineSnapshot(`
[wire] permission.set_mode { "mode": "auto", "time": "<time>" }
[emit] agent.status.updated { "model": "mock-model", "contextTokens": 0, "maxContextTokens": 1000000, "contextUsage": 0, "planMode": false, "swarmMode": false, "permission": "auto" }
[wire] tools.register_user_tool { "name": "Lookup", "description": "Look up a short test value.", "parameters": { "type": "object", "properties": { "query": { "type": "string" } }, "required": [ "query" ], "additionalProperties": false }, "time": "<time>" }
[wire] turn.prompt { "input": [ { "type": "text", "text": "Look up moon" } ], "origin": { "kind": "user" }, "time": "<time>" }
[emit] turn.started { "turnId": 0, "origin": { "kind": "user" } }
[wire] context.append_message { "message": { "role": "user", "content": [ { "type": "text", "text": "Look up moon" } ], "toolCalls": [], "origin": { "kind": "user" } }, "time": "<time>" }
[wire] context.append_message { "message": { "role": "user", "content": [ { "type": "text", "text": "<auto-mode-enter-reminder>" } ], "toolCalls": [], "origin": { "kind": "injection", "variant": "permission_mode" } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.begin", "uuid": "<uuid-1>", "turnId": "0", "step": 1 }, "time": "<time>" }
[emit] turn.step.started { "turnId": 0, "step": 1, "stepId": "<uuid-1>" }
[wire] llm.tools_snapshot { "hash": "3bfeb22e61431247933e79f6ab94e7ca14a127f899bc87e7bbd22594ba9cdb66", "tools": [ { "name": "Lookup", "description": "Look up a short test value.", "parameters": { "type": "object", "properties": { "query": { "type": "string" } }, "required": [ "query" ], "additionalProperties": false } } ], "time": "<time>" }
[wire] llm.request { "kind": "loop", "provider": "kimi", "model": "mock-model", "modelAlias": "mock-model", "thinkingEffort": "off", "maxTokens": 1000000, "toolSelect": false, "systemPromptHash": "ec9c34379c88babbc468ef2f3e0e08cd2f422c8c4a910664fb8bb394d703a575", "toolsHash": "3bfeb22e61431247933e79f6ab94e7ca14a127f899bc87e7bbd22594ba9cdb66", "messageCount": 2, "turnStep": "0.1", "time": "<time>" }
[emit] assistant.delta { "turnId": 0, "delta": "I will look it up." }
[emit] tool.call.delta { "turnId": 0, "toolCallId": "call_lookup", "name": "Lookup", "argumentsPart": "{\\"query\\":\\"moon\\"}" }
[wire] context.append_loop_event { "event": { "type": "content.part", "uuid": "<uuid-2>", "turnId": "0", "step": 1, "stepUuid": "<uuid-1>", "part": { "type": "text", "text": "I will look it up." } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "tool.call", "uuid": "call_lookup", "turnId": "0", "step": 1, "stepUuid": "<uuid-1>", "toolCallId": "call_lookup", "name": "Lookup", "args": { "query": "moon" } }, "time": "<time>" }
[emit] tool.call.started { "turnId": 0, "toolCallId": "call_lookup", "name": "Lookup", "args": { "query": "moon" } }
[emit] toolCall { "turnId": 0, "toolCallId": "call_lookup", "args": { "query": "moon" } }
`);
expect(ctx.lastLlmInput()).toMatchInlineSnapshot(`
system: <system-prompt>
tools: Lookup
messages:
user: text "Look up moon"
user: text <auto-mode-enter-reminder>
`);
ctx.mockNextResponse({ type: 'text', text: 'The lookup result is moon-result.' });
expect(await ctx.untilTurnEnd()).toMatchInlineSnapshot(`
[wire] context.append_loop_event { "event": { "type": "tool.result", "parentUuid": "call_lookup", "toolCallId": "call_lookup", "result": { "output": "moon-result" } }, "time": "<time>" }
[emit] tool.result { "turnId": 0, "toolCallId": "call_lookup", "output": "moon-result" }
[wire] context.append_loop_event { "event": { "type": "step.end", "uuid": "<uuid-1>", "turnId": "0", "step": 1, "usage": { "inputOther": 88, "output": 16, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "tool_use", "messageId": "mock-1" }, "time": "<time>" }
[emit] turn.step.completed { "turnId": 0, "step": 1, "stepId": "<uuid-1>", "usage": { "inputOther": 88, "output": 16, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "tool_use" }
[wire] usage.record { "model": "mock-model", "usage": { "inputOther": 88, "output": 16, "inputCacheRead": 0, "inputCacheCreation": 0 }, "usageScope": "turn", "time": "<time>" }
[emit] agent.status.updated { "model": "mock-model", "contextTokens": 104, "maxContextTokens": 1000000, "contextUsage": 0.000104, "planMode": false, "swarmMode": false, "permission": "auto", "usage": { "byModel": { "mock-model": { "inputOther": 88, "output": 16, "inputCacheRead": 0, "inputCacheCreation": 0 } }, "total": { "inputOther": 88, "output": 16, "inputCacheRead": 0, "inputCacheCreation": 0 }, "currentTurn": { "inputOther": 88, "output": 16, "inputCacheRead": 0, "inputCacheCreation": 0 } } }
[wire] context.append_loop_event { "event": { "type": "step.begin", "uuid": "<uuid-3>", "turnId": "0", "step": 2 }, "time": "<time>" }
[emit] turn.step.started { "turnId": 0, "step": 2, "stepId": "<uuid-3>" }
[wire] llm.request { "kind": "loop", "provider": "kimi", "model": "mock-model", "modelAlias": "mock-model", "thinkingEffort": "off", "maxTokens": 999896, "toolSelect": false, "systemPromptHash": "ec9c34379c88babbc468ef2f3e0e08cd2f422c8c4a910664fb8bb394d703a575", "toolsHash": "3bfeb22e61431247933e79f6ab94e7ca14a127f899bc87e7bbd22594ba9cdb66", "messageCount": 4, "turnStep": "0.2", "time": "<time>" }
[emit] assistant.delta { "turnId": 0, "delta": "The lookup result is moon-result." }
[wire] context.append_loop_event { "event": { "type": "content.part", "uuid": "<uuid-4>", "turnId": "0", "step": 2, "stepUuid": "<uuid-3>", "part": { "type": "text", "text": "The lookup result is moon-result." } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.end", "uuid": "<uuid-3>", "turnId": "0", "step": 2, "usage": { "inputOther": 108, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn", "messageId": "mock-2" }, "time": "<time>" }
[emit] turn.step.completed { "turnId": 0, "step": 2, "stepId": "<uuid-3>", "usage": { "inputOther": 108, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn" }
[wire] usage.record { "model": "mock-model", "usage": { "inputOther": 108, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 }, "usageScope": "turn", "time": "<time>" }
[emit] agent.status.updated { "model": "mock-model", "contextTokens": 120, "maxContextTokens": 1000000, "contextUsage": 0.00012, "planMode": false, "swarmMode": false, "permission": "auto", "usage": { "byModel": { "mock-model": { "inputOther": 196, "output": 28, "inputCacheRead": 0, "inputCacheCreation": 0 } }, "total": { "inputOther": 196, "output": 28, "inputCacheRead": 0, "inputCacheCreation": 0 }, "currentTurn": { "inputOther": 196, "output": 28, "inputCacheRead": 0, "inputCacheCreation": 0 } } }
[emit] turn.ended { "turnId": 0, "reason": "completed" }
`);
expect(ctx.lastLlmInput()).toMatchInlineSnapshot(`
messages:
<last>
assistant: text "I will look it up." calls call_lookup:Lookup { "query": "moon" }
tool[call_lookup]: text "moon-result"
`);
await ctx.rpc.unregisterTool({ name: 'Lookup' });
ctx.mockNextResponse({ type: 'text', text: 'No lookup tool is available.' });
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Can you still use Lookup?' }] });
expect(await ctx.untilTurnEnd()).toMatchInlineSnapshot(`
[wire] tools.unregister_user_tool { "name": "Lookup", "time": "<time>" }
[wire] turn.prompt { "input": [ { "type": "text", "text": "Can you still use Lookup?" } ], "origin": { "kind": "user" }, "time": "<time>" }
[emit] turn.started { "turnId": 1, "origin": { "kind": "user" } }
[wire] context.append_message { "message": { "role": "user", "content": [ { "type": "text", "text": "Can you still use Lookup?" } ], "toolCalls": [], "origin": { "kind": "user" } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.begin", "uuid": "<uuid-5>", "turnId": "1", "step": 1 }, "time": "<time>" }
[emit] turn.step.started { "turnId": 1, "step": 1, "stepId": "<uuid-5>" }
[wire] llm.tools_snapshot { "hash": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945", "tools": [], "time": "<time>" }
[wire] llm.request { "kind": "loop", "provider": "kimi", "model": "mock-model", "modelAlias": "mock-model", "thinkingEffort": "off", "maxTokens": 999880, "toolSelect": false, "systemPromptHash": "ec9c34379c88babbc468ef2f3e0e08cd2f422c8c4a910664fb8bb394d703a575", "toolsHash": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945", "messageCount": 6, "turnStep": "1.1", "time": "<time>" }
[emit] assistant.delta { "turnId": 1, "delta": "No lookup tool is available." }
[wire] context.append_loop_event { "event": { "type": "content.part", "uuid": "<uuid-6>", "turnId": "1", "step": 1, "stepUuid": "<uuid-5>", "part": { "type": "text", "text": "No lookup tool is available." } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.end", "uuid": "<uuid-5>", "turnId": "1", "step": 1, "usage": { "inputOther": 128, "output": 10, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn", "messageId": "mock-3" }, "time": "<time>" }
[emit] turn.step.completed { "turnId": 1, "step": 1, "stepId": "<uuid-5>", "usage": { "inputOther": 128, "output": 10, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn" }
[wire] usage.record { "model": "mock-model", "usage": { "inputOther": 128, "output": 10, "inputCacheRead": 0, "inputCacheCreation": 0 }, "usageScope": "turn", "time": "<time>" }
[emit] agent.status.updated { "model": "mock-model", "contextTokens": 138, "maxContextTokens": 1000000, "contextUsage": 0.000138, "planMode": false, "swarmMode": false, "permission": "auto", "usage": { "byModel": { "mock-model": { "inputOther": 324, "output": 38, "inputCacheRead": 0, "inputCacheCreation": 0 } }, "total": { "inputOther": 324, "output": 38, "inputCacheRead": 0, "inputCacheCreation": 0 }, "currentTurn": { "inputOther": 128, "output": 10, "inputCacheRead": 0, "inputCacheCreation": 0 } } }
[emit] turn.ended { "turnId": 1, "reason": "completed" }
`);
expect(ctx.lastLlmInput()).toMatchInlineSnapshot(`
tools: []
messages:
<last>
assistant: text "The lookup result is moon-result."
user: text "Can you still use Lookup?"
`);
await ctx.expectResumeMatches();
});
it('persists oversized registered tool results before adding them to model context', async () => {
const sessionDir = mkdtempSync(join(tmpdir(), 'tool-result-overflow-'));
try {
const lookupCall: ToolCall = {
type: 'function',
id: 'call_lookup',
name: 'Lookup',
arguments: '{"query":"moon"}',
};
const largeOutput = `${'x'.repeat(60_000)}tail survives`;
const ctx = testAgent({ homedir: sessionDir });
ctx.configure();
await ctx.rpc.setPermission({ mode: 'auto' });
await ctx.rpc.registerTool({
name: 'Lookup',
description: 'Look up a short test value.',
parameters: { type: 'object', properties: {} },
});
ctx.mockNextResponse({ type: 'text', text: 'I will look it up.' }, lookupCall);
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Look up moon' }] });
await ctx.untilToolCall({ output: largeOutput });
ctx.mockNextResponse({ type: 'text', text: 'done' });
await ctx.untilTurnEnd();
const toolText = ctx.compactHistory().find((message) => message.role === 'tool')?.text ?? '';
const outputPath = /^output_path: (.+)$/m.exec(toolText)?.[1];
expect(toolText).toContain('Tool output exceeded 50000 characters');
expect(toolText).not.toContain('tail survives');
expect(outputPath).toBeTruthy();
expect(readFileSync(outputPath!, 'utf8')).toBe(largeOutput);
} finally {
rmSync(sessionDir, { recursive: true, force: true });
}
});
it('does not overwrite saved oversized tool results with repeated call IDs', async () => {
const sessionDir = mkdtempSync(join(tmpdir(), 'tool-result-overflow-'));
try {
const firstOutput = `${'a'.repeat(60_000)}first tail`;
const secondOutput = `${'b'.repeat(60_000)}second tail`;
const first = await budgetToolResultForModel({
homedir: sessionDir,
toolName: 'Lookup',
toolCallId: 'call_lookup',
result: { output: firstOutput },
});
const second = await budgetToolResultForModel({
homedir: sessionDir,
toolName: 'Lookup',
toolCallId: 'call_lookup',
result: { output: secondOutput },
});
const firstPath = savedOutputPath(first.output);
const secondPath = savedOutputPath(second.output);
expect(firstPath).not.toBe(secondPath);
expect(readFileSync(firstPath, 'utf8')).toBe(firstOutput);
expect(readFileSync(secondPath, 'utf8')).toBe(secondOutput);
} finally {
rmSync(sessionDir, { recursive: true, force: true });
}
});
it('keeps oversized tool results intact when no session directory is available', async () => {
const largeOutput = `${'x'.repeat(60_000)}tail survives`;
const result = { output: largeOutput };
const budgeted = await budgetToolResultForModel({
toolName: 'Lookup',
toolCallId: 'call_lookup',
result,
});
expect(budgeted).toBe(result);
expect(budgeted.output).toBe(largeOutput);
});
it('does not save already-truncated tool result previews as full output', async () => {
const sessionDir = mkdtempSync(join(tmpdir(), 'tool-result-overflow-'));
try {
const largeOutput = `${'x'.repeat(60_000)}[...truncated]`;
const result = {
output: largeOutput,
truncated: true,
};
const budgeted = await budgetToolResultForModel({
homedir: sessionDir,
toolName: 'Lookup',
toolCallId: 'call_lookup',
result,
});
expect(budgeted).toBe(result);
expect(budgeted.output).toBe(largeOutput);
expect(budgeted.output).not.toContain('output_path:');
expect(existsSync(join(sessionDir, 'tool-results'))).toBe(false);
} finally {
rmSync(sessionDir, { recursive: true, force: true });
}
});
});
function bashCall(): ToolCall {
return {
type: 'function',
id: 'call_bash',
name: 'Bash',
arguments: '{"command":"printf hook-output","timeout":60}',
};
}
function agentCall(): ToolCall {
return {
type: 'function',
id: 'call_agent',
name: 'Agent',
arguments: JSON.stringify({
prompt: 'Investigate deeply',
description: 'Investigate deeply',
subagent_type: 'coder',
}),
};
}
function savedOutputPath(output: unknown): string {
expect(typeof output).toBe('string');
const outputPath = /^output_path: (.+)$/m.exec(output as string)?.[1];
expect(outputPath).toBeTruthy();
return outputPath!;
}
function hookErrorMessageAssertCommand(expected: string): string {
const script = [
"let input = '';",
"process.stdin.on('data', (chunk) => { input += chunk; });",
"process.stdin.on('end', () => {",
' const payload = JSON.parse(input);',
` if (payload.error?.message === ${JSON.stringify(expected)}) process.exit(0);`,
" console.error(payload.error?.message ?? '<missing>');",
' process.exit(2);',
'});',
].join('');
return `node -e ${JSON.stringify(script)}`;
}