kimi-code/packages/agent-core/test/agent/config.test.ts
liruifengv 108299be3c
refactor!: overhaul thinking config and effort resolution (#1132)
* feat: support multi-level thinking effort switching

- kimi provider: emit thinking.effort in the new wire format; keep reasoning_effort mirrored during the transition
- model catalog: thread support_efforts / default_effort from oauth through to /models
- config schema: add supportEfforts / defaultEffort on model aliases
- TUI: multi-segment thinking control in /model, new /effort command, footer effort display
- switch status uses displayName and distinguishes model vs effort-only changes

* docs: add thinking effort design plans

- thinking-effort-switching.md: implemented multi-level effort switching
- thinking-model-overhaul.md: follow-up refactor plan for the thinking state model

* docs: collapse thinking overhaul plan into a single PR

* refactor!: overhaul thinking config and effort resolution

Replace default_thinking and thinking.mode with a single [thinking] enabled/effort table. ThinkingEffort is now an open string ('off' | 'on' | model-declared effort); effort levels come from each model's support_efforts instead of a fixed enum.

Centralize default and always_thinking clamp logic in resolveThinkingEffort/defaultThinkingEffortFor, and honor an explicitly configured effort when an always_thinking model is forced back on.

TUI keeps a single thinkingEffort field instead of the boolean + level pair; 'on' is normalized to the model default at the UI boundary.

BREAKING CHANGE: default_thinking and thinking.mode are removed from config; migrate to [thinking] enabled/effort.

* refactor: rename residual thinking level wording to effort

Rename comments, error messages, parameter names, the SetThinkingPayload wire field (level -> effort), and TUI local variables so the thinking effort naming is consistent throughout. No behavior change.

* refactor: rename remaining camelCase thinking level identifiers to effort

Rename liveLevel/prevLevel/levelChanged/commitLevel/effectiveLevel to liveEffort/prevEffort/effortChanged/commitEffort/effectiveEffort in the TUI model picker and config commands.

* refactor: eliminate remaining thinking level wording in comments and tests

Rename levelLabel -> effortLabel, EffortSelectorOptions.levels -> efforts, and 'effort level(s)' / 'default level' / 'requested level' wording in comments, error messages, slash-command description, and test titles to effort. Also restore the withThinking(effort) parameter rename in the Kimi provider that was accidentally reverted.

* fix: address codex review feedback on thinking effort handling

- OpenAI thinkingEffortToReasoningEffort and Anthropic clampEffort now normalize 'on' / unrecognized efforts instead of throwing, so boolean non-Kimi models no longer crash on session start.

- ACP resolveCurrentThinkingEnabled treats a non-empty thinking.effort as enabled, matching agent-core's resolveThinkingEffort.

- REST promptThinkingSchema accepts any non-empty effort string so model-declared efforts are not rejected at the API boundary.

* test: align kimi e2e expectations with supportEfforts-gated reasoning_effort

The kimi provider now sends reasoning_effort only when the model declares support_efforts; boolean models (no support_efforts) send only thinking.type. Update the kimi e2e tests to drop the stale reasoning_effort expectation for the boolean test model.

* test: cover [thinking] effort parsing in config.test

Add effort = "high" to the documented [thinking] table in the config parse test and assert config.thinking.effort is resolved, so the new [thinking] effort field has direct parse coverage.

* docs: add thinking test coverage gap analysis

Capture the explore agent's test coverage review for the thinking overhaul PR, including P1/P2 gaps and the two open design questions, for follow-up test additions.

* feat(oauth): parse nested think_efforts from /models response

The /models endpoint now returns effort levels under a nested think_efforts object ({ support, valid_efforts, default_effort }). Parse it preferentially in both managed-kimi-code and open-platform model parsing, falling back to the legacy flat support_efforts / default_effort fields for older servers.

* refactor(oauth): only read nested think_efforts; gate on support=true

Drop the legacy flat support_efforts / default_effort fallback. The think_efforts object is now the single source, and its support flag gates the whole object — when support is not true, valid_efforts and default_effort are ignored entirely.

* chore: remove unused parseStringArray import in open-platform

* docs: finalize thinking effort release notes

Downgrade the changeset to minor with an English summary, drop the version-specific 'added in 1.0.0' info block, and present the deprecated config fields as a table (field / deprecated in 0.21.0 / description).

* refactor: drop temporary refresh toggles and kimi reasoning_effort mirror

Remove the always-true REFRESH_MODELS_ON_PICKER_OPEN / REFRESH_PROVIDER_MODELS_ON_STARTUP toggles and their stale re-enable TODOs, and stop sending reasoning_effort from the kimi provider (thinking.effort is the only wire field now).

* fix(tui): avoid persisting "on" as thinking effort

* fix: preserve persisted thinking effort across login and provider setup

* fix(tui): show actual thinking effort in /status and footer

* test(tui): align message-flow expectations with effort persistence and /status display

* fix(vis): rename thinkingLevel to thinkingEffort in config.update analysis
2026-06-30 22:34:13 +08:00

234 lines
15 KiB
TypeScript

import type { ModelCapability, ProviderConfig, ToolCall } from '@moonshot-ai/kosong';
import { describe, expect, it } from 'vitest';
import type { ResolvedAgentProfile } from '../../src/profile';
import { createCommandKaos, testAgent } from './harness/agent';
import { DEFAULT_TEST_SYSTEM_PROMPT } from './harness/snapshots';
describe('Agent config', () => {
it('exposes provider, system prompt, thinking effort, and model capability updates', async () => {
const ctx = testAgent();
const initialProvider: ProviderConfig = {
type: 'openai',
apiKey: 'sk-initial',
baseUrl: 'https://initial.example/v1',
model: 'gpt-initial',
};
const initialCapability: ModelCapability = {
image_in: true,
video_in: false,
audio_in: false,
thinking: false,
tool_use: true,
max_context_tokens: 128000,
};
ctx.configure({
provider: initialProvider,
modelCapabilities: initialCapability,
});
await expect(ctx.rpc.getConfig({})).resolves.toMatchObject({
provider: initialProvider,
systemPrompt: DEFAULT_TEST_SYSTEM_PROMPT,
thinkingEffort: 'off',
modelCapabilities: initialCapability,
});
const nextProvider: ProviderConfig = {
type: 'kimi',
apiKey: 'sk-next',
baseUrl: 'https://next.example/v1',
model: 'kimi-next',
};
const nextCapability: ModelCapability = {
image_in: true,
video_in: true,
audio_in: false,
thinking: true,
tool_use: true,
max_context_tokens: 262144,
};
ctx.configureRuntimeModel(nextProvider, nextCapability);
ctx.agent.config.update({
systemPrompt: 'Changed profile prompt.',
thinkingEffort: 'high',
});
await expect(ctx.rpc.getConfig({})).resolves.toMatchObject({
provider: nextProvider,
systemPrompt: 'Changed profile prompt.',
thinkingEffort: 'high',
modelCapabilities: nextCapability,
});
await ctx.expectResumeMatches();
});
it('useProfile emits the rendered system prompt and active tools', async () => {
const ctx = testAgent();
ctx.configure();
const profile: ResolvedAgentProfile = {
name: 'test-profile',
systemPrompt: () => 'Profile system prompt.',
tools: ['Bash'],
};
ctx.agent.useProfile(profile);
expect(ctx.newEvents()).toMatchInlineSnapshot(`
[wire] config.update { "profileName": "test-profile", "systemPrompt": "Profile system prompt.", "time": "<time>" }
[emit] agent.status.updated { "model": "mock-model", "contextTokens": 0, "maxContextTokens": 1000000, "contextUsage": 0, "planMode": false, "swarmMode": false, "permission": "manual" }
[wire] tools.set_active_tools { "names": [ "Bash" ], "time": "<time>" }
`);
await ctx.expectResumeMatches();
});
it('useProfile passes additionalDirsInfo to profile system prompts', async () => {
const ctx = testAgent();
ctx.configure();
const profile: ResolvedAgentProfile = {
name: 'context-profile',
systemPrompt: (context) =>
`Prompt with additional dirs: ${context.additionalDirsInfo ?? 'none'}`,
tools: ['Bash'],
};
ctx.agent.useProfile(profile, {
cwdListing: 'cwd listing',
agentsMd: 'agents md',
additionalDirsInfo: '### /extra\nextra-file.txt',
});
expect(ctx.agent.config.systemPrompt).toBe(
'Prompt with additional dirs: ### /extra\nextra-file.txt',
);
ctx.agent.useProfile(profile);
expect(ctx.agent.config.systemPrompt).toBe('Prompt with additional dirs: none');
});
it('config.update with cwd initializes builtin tools', async () => {
const ctx = testAgent();
ctx.configure();
const tools = await ctx.rpc.getTools({});
expect(toolNames(tools)).toEqual(
expect.arrayContaining(['Bash', 'Read', 'Write', 'Edit', 'Grep', 'Glob']),
);
await ctx.expectResumeMatches();
});
it('keeps turn-start config for later steps and applies updates to the next turn', async () => {
const bashCall: ToolCall = {
type: 'function',
id: 'call_bash',
name: 'Bash',
arguments: '{"command":"printf original-result","timeout":60}',
};
const ctx = testAgent({ kaos: createCommandKaos('original-result') });
ctx.configure({ tools: ['Bash'] });
ctx.mockNextResponse({ type: 'text', text: 'I will run Bash.' }, bashCall);
await ctx.rpc.prompt({
input: [{ type: 'text', text: 'Run Bash before config changes' }],
});
expect(await ctx.untilApproval(true)).toMatchInlineSnapshot(`
[wire] turn.prompt { "input": [ { "type": "text", "text": "Run Bash before config changes" } ], "origin": { "kind": "user" }, "time": "<time>" }
[emit] turn.started { "turnId": 0, "origin": { "kind": "user" } }
[wire] context.append_message { "message": { "role": "user", "content": [ { "type": "text", "text": "Run Bash before config changes" } ], "toolCalls": [], "origin": { "kind": "user" } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.begin", "uuid": "<uuid-1>", "turnId": "0", "step": 1 }, "time": "<time>" }
[emit] turn.step.started { "turnId": 0, "step": 1, "stepId": "<uuid-1>" }
[emit] assistant.delta { "turnId": 0, "delta": "I will run Bash." }
[emit] tool.call.delta { "turnId": 0, "toolCallId": "call_bash", "name": "Bash", "argumentsPart": "{\\"command\\":\\"printf original-result\\",\\"timeout\\":60}" }
[wire] context.append_loop_event { "event": { "type": "content.part", "uuid": "<uuid-2>", "turnId": "0", "step": 1, "stepUuid": "<uuid-1>", "part": { "type": "text", "text": "I will run Bash." } }, "time": "<time>" }
[emit] requestApproval { "turnId": 0, "toolCallId": "call_bash", "toolName": "Bash", "action": "Running: printf original-result", "display": { "kind": "command", "command": "printf original-result", "cwd": "<cwd>", "language": "bash" } }
`);
expect(ctx.lastLlmInput()).toMatchInlineSnapshot(`
system: <system-prompt>
tools: Bash
messages:
user: text "Run Bash before config changes"
`);
ctx.configureRuntimeModel({
type: 'kimi',
apiKey: 'test-key',
model: 'changed-model',
});
ctx.agent.config.update({ systemPrompt: 'Changed system prompt.' });
await ctx.rpc.setActiveTools({ names: [] });
ctx.mockNextResponse({ type: 'text', text: 'Still using the original turn config.' });
expect(await ctx.untilTurnEnd()).toMatchInlineSnapshot(`
[wire] permission.record_approval_result { "turnId": 0, "toolCallId": "call_bash", "toolName": "Bash", "action": "Running: printf original-result", "result": { "decision": "approved", "selectedLabel": "approve" }, "time": "<time>" }
[wire] config.update { "modelAlias": "changed-model", "time": "<time>" }
[emit] agent.status.updated { "model": "changed-model", "contextTokens": 0, "maxContextTokens": 1000000, "contextUsage": 0, "planMode": false, "swarmMode": false, "permission": "manual" }
[wire] config.update { "systemPrompt": "Changed system prompt.", "time": "<time>" }
[emit] agent.status.updated { "model": "changed-model", "contextTokens": 0, "maxContextTokens": 1000000, "contextUsage": 0, "planMode": false, "swarmMode": false, "permission": "manual" }
[wire] tools.set_active_tools { "names": [], "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "tool.call", "uuid": "call_bash", "turnId": "0", "step": 1, "stepUuid": "<uuid-1>", "toolCallId": "call_bash", "name": "Bash", "args": { "command": "printf original-result", "timeout": 60 }, "description": "Running: printf original-result", "display": { "kind": "command", "command": "printf original-result", "cwd": "<cwd>", "language": "bash" } }, "time": "<time>" }
[emit] tool.call.started { "turnId": 0, "toolCallId": "call_bash", "name": "Bash", "args": { "command": "printf original-result", "timeout": 60 }, "description": "Running: printf original-result", "display": { "kind": "command", "command": "printf original-result", "cwd": "<cwd>", "language": "bash" } }
[emit] tool.progress { "turnId": 0, "toolCallId": "call_bash", "update": { "kind": "stdout", "text": "original-result" } }
[wire] context.append_loop_event { "event": { "type": "tool.result", "parentUuid": "call_bash", "toolCallId": "call_bash", "result": { "output": "original-result" } }, "time": "<time>" }
[emit] tool.result { "turnId": 0, "toolCallId": "call_bash", "output": "original-result" }
[wire] context.append_loop_event { "event": { "type": "step.end", "uuid": "<uuid-1>", "turnId": "0", "step": 1, "usage": { "inputOther": 9, "output": 23, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "tool_use" }, "time": "<time>" }
[emit] turn.step.completed { "turnId": 0, "step": 1, "stepId": "<uuid-1>", "usage": { "inputOther": 9, "output": 23, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "tool_use" }
[wire] usage.record { "model": "mock-model", "usage": { "inputOther": 9, "output": 23, "inputCacheRead": 0, "inputCacheCreation": 0 }, "usageScope": "turn", "time": "<time>" }
[emit] agent.status.updated { "model": "changed-model", "contextTokens": 32, "maxContextTokens": 1000000, "contextUsage": 0.000032, "planMode": false, "swarmMode": false, "permission": "manual", "usage": { "byModel": { "mock-model": { "inputOther": 9, "output": 23, "inputCacheRead": 0, "inputCacheCreation": 0 } }, "total": { "inputOther": 9, "output": 23, "inputCacheRead": 0, "inputCacheCreation": 0 }, "currentTurn": { "inputOther": 9, "output": 23, "inputCacheRead": 0, "inputCacheCreation": 0 } } }
[wire] context.append_loop_event { "event": { "type": "step.begin", "uuid": "<uuid-3>", "turnId": "0", "step": 2 }, "time": "<time>" }
[emit] turn.step.started { "turnId": 0, "step": 2, "stepId": "<uuid-3>" }
[emit] assistant.delta { "turnId": 0, "delta": "Still using the original turn config." }
[wire] context.append_loop_event { "event": { "type": "content.part", "uuid": "<uuid-4>", "turnId": "0", "step": 2, "stepUuid": "<uuid-3>", "part": { "type": "text", "text": "Still using the original turn config." } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.end", "uuid": "<uuid-3>", "turnId": "0", "step": 2, "usage": { "inputOther": 37, "output": 13, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn" }, "time": "<time>" }
[emit] turn.step.completed { "turnId": 0, "step": 2, "stepId": "<uuid-3>", "usage": { "inputOther": 37, "output": 13, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn" }
[wire] usage.record { "model": "mock-model", "usage": { "inputOther": 37, "output": 13, "inputCacheRead": 0, "inputCacheCreation": 0 }, "usageScope": "turn", "time": "<time>" }
[emit] agent.status.updated { "model": "changed-model", "contextTokens": 50, "maxContextTokens": 1000000, "contextUsage": 0.00005, "planMode": false, "swarmMode": false, "permission": "manual", "usage": { "byModel": { "mock-model": { "inputOther": 46, "output": 36, "inputCacheRead": 0, "inputCacheCreation": 0 } }, "total": { "inputOther": 46, "output": 36, "inputCacheRead": 0, "inputCacheCreation": 0 }, "currentTurn": { "inputOther": 46, "output": 36, "inputCacheRead": 0, "inputCacheCreation": 0 } } }
[emit] turn.ended { "turnId": 0, "reason": "completed" }
`);
expect(ctx.lastLlmInput()).toMatchInlineSnapshot(`
messages:
<last>
assistant: text "I will run Bash." calls call_bash:Bash { "command": "printf original-result", "timeout": 60 }
tool[call_bash]: text "original-result"
`);
ctx.mockNextResponse({ type: 'text', text: 'Now the changed config is active.' });
await ctx.rpc.prompt({ input: [{ type: 'text', text: 'Start a fresh turn' }] });
expect(await ctx.untilTurnEnd()).toMatchInlineSnapshot(`
[wire] turn.prompt { "input": [ { "type": "text", "text": "Start a fresh turn" } ], "origin": { "kind": "user" }, "time": "<time>" }
[emit] turn.started { "turnId": 1, "origin": { "kind": "user" } }
[wire] context.append_message { "message": { "role": "user", "content": [ { "type": "text", "text": "Start a fresh turn" } ], "toolCalls": [], "origin": { "kind": "user" } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.begin", "uuid": "<uuid-5>", "turnId": "1", "step": 1 }, "time": "<time>" }
[emit] turn.step.started { "turnId": 1, "step": 1, "stepId": "<uuid-5>" }
[emit] assistant.delta { "turnId": 1, "delta": "Now the changed config is active." }
[wire] context.append_loop_event { "event": { "type": "content.part", "uuid": "<uuid-6>", "turnId": "1", "step": 1, "stepUuid": "<uuid-5>", "part": { "type": "text", "text": "Now the changed config is active." } }, "time": "<time>" }
[wire] context.append_loop_event { "event": { "type": "step.end", "uuid": "<uuid-5>", "turnId": "1", "step": 1, "usage": { "inputOther": 56, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn" }, "time": "<time>" }
[emit] turn.step.completed { "turnId": 1, "step": 1, "stepId": "<uuid-5>", "usage": { "inputOther": 56, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 }, "finishReason": "end_turn" }
[wire] usage.record { "model": "changed-model", "usage": { "inputOther": 56, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 }, "usageScope": "turn", "time": "<time>" }
[emit] agent.status.updated { "model": "changed-model", "contextTokens": 68, "maxContextTokens": 1000000, "contextUsage": 0.000068, "planMode": false, "swarmMode": false, "permission": "manual", "usage": { "byModel": { "mock-model": { "inputOther": 46, "output": 36, "inputCacheRead": 0, "inputCacheCreation": 0 }, "changed-model": { "inputOther": 56, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 } }, "total": { "inputOther": 102, "output": 48, "inputCacheRead": 0, "inputCacheCreation": 0 }, "currentTurn": { "inputOther": 56, "output": 12, "inputCacheRead": 0, "inputCacheCreation": 0 } } }
[emit] turn.ended { "turnId": 1, "reason": "completed" }
`);
expect(ctx.lastLlmInput()).toMatchInlineSnapshot(`
system: "Changed system prompt."
tools: []
messages:
<last>
assistant: text "Still using the original turn config."
user: text "Start a fresh turn"
`);
await ctx.expectResumeMatches();
});
});
function toolNames(value: unknown): string[] {
if (!Array.isArray(value)) return [];
return value
.map((item) => {
if (item === null || typeof item !== 'object') return null;
const record = item as Record<string, unknown>;
return typeof record['name'] === 'string' ? record['name'] : null;
})
.filter((name): name is string => name !== null);
}