kimi-code/plan/thinking-test-coverage.md
liruifengv 108299be3c
refactor!: overhaul thinking config and effort resolution (#1132)
* feat: support multi-level thinking effort switching

- kimi provider: emit thinking.effort in the new wire format; keep reasoning_effort mirrored during the transition
- model catalog: thread support_efforts / default_effort from oauth through to /models
- config schema: add supportEfforts / defaultEffort on model aliases
- TUI: multi-segment thinking control in /model, new /effort command, footer effort display
- switch status uses displayName and distinguishes model vs effort-only changes

* docs: add thinking effort design plans

- thinking-effort-switching.md: implemented multi-level effort switching
- thinking-model-overhaul.md: follow-up refactor plan for the thinking state model

* docs: collapse thinking overhaul plan into a single PR

* refactor!: overhaul thinking config and effort resolution

Replace default_thinking and thinking.mode with a single [thinking] enabled/effort table. ThinkingEffort is now an open string ('off' | 'on' | model-declared effort); effort levels come from each model's support_efforts instead of a fixed enum.

Centralize default and always_thinking clamp logic in resolveThinkingEffort/defaultThinkingEffortFor, and honor an explicitly configured effort when an always_thinking model is forced back on.

TUI keeps a single thinkingEffort field instead of the boolean + level pair; 'on' is normalized to the model default at the UI boundary.

BREAKING CHANGE: default_thinking and thinking.mode are removed from config; migrate to [thinking] enabled/effort.

* refactor: rename residual thinking level wording to effort

Rename comments, error messages, parameter names, the SetThinkingPayload wire field (level -> effort), and TUI local variables so the thinking effort naming is consistent throughout. No behavior change.

* refactor: rename remaining camelCase thinking level identifiers to effort

Rename liveLevel/prevLevel/levelChanged/commitLevel/effectiveLevel to liveEffort/prevEffort/effortChanged/commitEffort/effectiveEffort in the TUI model picker and config commands.

* refactor: eliminate remaining thinking level wording in comments and tests

Rename levelLabel -> effortLabel, EffortSelectorOptions.levels -> efforts, and 'effort level(s)' / 'default level' / 'requested level' wording in comments, error messages, slash-command description, and test titles to effort. Also restore the withThinking(effort) parameter rename in the Kimi provider that was accidentally reverted.

* fix: address codex review feedback on thinking effort handling

- OpenAI thinkingEffortToReasoningEffort and Anthropic clampEffort now normalize 'on' / unrecognized efforts instead of throwing, so boolean non-Kimi models no longer crash on session start.

- ACP resolveCurrentThinkingEnabled treats a non-empty thinking.effort as enabled, matching agent-core's resolveThinkingEffort.

- REST promptThinkingSchema accepts any non-empty effort string so model-declared efforts are not rejected at the API boundary.

* test: align kimi e2e expectations with supportEfforts-gated reasoning_effort

The kimi provider now sends reasoning_effort only when the model declares support_efforts; boolean models (no support_efforts) send only thinking.type. Update the kimi e2e tests to drop the stale reasoning_effort expectation for the boolean test model.

* test: cover [thinking] effort parsing in config.test

Add effort = "high" to the documented [thinking] table in the config parse test and assert config.thinking.effort is resolved, so the new [thinking] effort field has direct parse coverage.

* docs: add thinking test coverage gap analysis

Capture the explore agent's test coverage review for the thinking overhaul PR, including P1/P2 gaps and the two open design questions, for follow-up test additions.

* feat(oauth): parse nested think_efforts from /models response

The /models endpoint now returns effort levels under a nested think_efforts object ({ support, valid_efforts, default_effort }). Parse it preferentially in both managed-kimi-code and open-platform model parsing, falling back to the legacy flat support_efforts / default_effort fields for older servers.

* refactor(oauth): only read nested think_efforts; gate on support=true

Drop the legacy flat support_efforts / default_effort fallback. The think_efforts object is now the single source, and its support flag gates the whole object — when support is not true, valid_efforts and default_effort are ignored entirely.

* chore: remove unused parseStringArray import in open-platform

* docs: finalize thinking effort release notes

Downgrade the changeset to minor with an English summary, drop the version-specific 'added in 1.0.0' info block, and present the deprecated config fields as a table (field / deprecated in 0.21.0 / description).

* refactor: drop temporary refresh toggles and kimi reasoning_effort mirror

Remove the always-true REFRESH_MODELS_ON_PICKER_OPEN / REFRESH_PROVIDER_MODELS_ON_STARTUP toggles and their stale re-enable TODOs, and stop sending reasoning_effort from the kimi provider (thinking.effort is the only wire field now).

* fix(tui): avoid persisting "on" as thinking effort

* fix: preserve persisted thinking effort across login and provider setup

* fix(tui): show actual thinking effort in /status and footer

* test(tui): align message-flow expectations with effort persistence and /status display

* fix(vis): rename thinkingLevel to thinkingEffort in config.update analysis
2026-06-30 22:34:13 +08:00

12 KiB
Raw Blame History

Thinking 模型重构 — 测试覆盖审查报告

PR #1132分支 support-effort)的测试缺口分析。生成于 2026-06-26供后续补测试用。

总体结论:核心逻辑resolveThinkingEffort、kimi/anthropic provider、ACP toggle、TUI commitEffort覆盖较扎实本次重构引入的若干"归一/兼容"分支存在行为契约未锁定的缺口。没有发现会导致运行时崩溃P0的缺口所有缺口均为行为契约P1或 nice-to-haveP2


两个待确认的设计问题

1. default_thinking / thinking.mode 是「拒绝」还是「静默忽略」?

  • 当前实现:静默忽略 + 下次写入时剥离schema 非 strictpackages/agent-core/src/config/toml.ts 写入时 delete out['mode'] / delete out['default_thinking'])。
  • 计划意图「直接删除不做兼容读取」breaking change。
  • 建议:按"静默忽略 + 写入剥离"补测试锁定P1 第 4 项)。如果要 fail-fast需把 schema 改为 .strict() 并补 fail-fast 测试。

2. commitEffort('on')defaultEffort 不在 supportEfforts 内时返回 defaultEffort,是否有意?

  • 当前实现defaultThinkingEffortForagent-core 和 TUI 内联)都是 model.defaultEffort ?? middleOf(supportEfforts)不校验 defaultEffort 是否在 supportEfforts
  • 为什么合理provider 端会归一(声明外的 effort 不下发 wire effort所以即使 defaultEffort 不在声明里也不会导致 wire 错误。
  • 建议:按"返回 defaultEffort由 provider 归一"补测试锁定P1 第 12 项)。

P1 缺口(行为契约未锁定,建议本 PR 或 follow-up 尽快补)

kosong

openai-common.tsthinkingEffortToReasoningEffort

  • 现有off/low/medium/high/xhigh/max + 未知 'extreme' → undefinedopenai-common-errors.test.ts:314-337)。
  • 缺口'on' 没有显式断言(注释明确把 'on' 列为归一对象,但测试用 'extreme')。
  • 建议:把 it('normalizes unknown effort to undefined') 改成 it.each(['on', 'extreme', 'foo'])

anthropic.tsclampEffort

  • 现有xhigh/max 在 opus-4-5/4-6/4-7/4-8/fable/sonnet/haiku 上的 clamp 与透传、adaptive vs budget、off、low/medium 透传(anthropic.test.ts:1084-1400)。
  • 缺口'on' / 未知 effort'foo')→ clamp 到 'high'anthropic.ts:342-350 的最后 if 分支)完全未覆盖
  • 建议it.each(['on', 'foo']) 在 adaptive 模型(如 claude-opus-4-7)上 → output_config={effort:'high'};在非 adaptive 模型(如 claude-sonnet-4-5)上 → thinking={type:'enabled',budget_tokens:32000} 且无 output_config

google-genai.tswithThinking

  • 现有:非 gemini-3 的 high/offgemini-3 的 off/low/medium/highgetter 反射(google-genai.test.ts:729-836)。
  • 缺口
    • 'on' / 未知 effort'foo')在 gemini-3 上 → 只 include_thoughts:true、不设 thinking_levelgoogle-genai.ts:829-852)。未覆盖。
    • 'on' / 未知 effort 在 非 gemini-3 上 → 只 include_thoughts:true、不设 thinking_budgetgoogle-genai.ts:853-873)。未覆盖。
    • 'xhigh'/'max'gemini-3HIGHfall-through。未覆盖。
  • 建议it.each(['on','foo'])gemini-3-pro-preview 上 → thinking_config={include_thoughts:true}(无 thinking_level);在 gemini-2.5-flash 上 → {include_thoughts:true}(无 thinking_budget)。it.each(['xhigh','max']) 在 gemini-3 上 → thinking_level:'HIGH'

kimi.tswithThinking

  • 现有:非 effort 模型 / effort 模型 / max / off / xhigh/on/foo 不在 supportEfforts / getter / 重复调用(kimi.test.ts:708-799)。
  • 缺口:空 supportEfforts: [](与 undefined 等价)未显式覆盖。
  • 建议:加 createProvider(false, []).withThinking('high')thinking={type:'enabled'}、无 reasoning_effort

agent-core

config/schema.ts — 删除 default_thinking / thinking.mode

  • 现有[thinking] enabled/effort 解析patch merge thinkingdefault_yolo 的 drop-deprecated 范式(configs.test.ts)。
  • 缺口default_thinking(顶层)被忽略/剥离无测试thinking.mode 被忽略/剥离无测试
  • 建议parseConfigString('default_thinking = true\n[thinking]\nmode = "always"\neffort="high"\n')config.thinking 不含 mode、顶层无 defaultThinking,且 writeConfigFile 后文本不含 default_thinking / mode

config/env-model.ts — 删除 KIMI_MODEL_THINKING_MODE / KIMI_MODEL_DEFAULT_THINKING

  • 现有KIMI_MODEL_THINKING_EFFORT 映射;KIMI_MODEL_ADAPTIVE_THINKINGwrite-back 隔离(env-model.test.ts)。
  • 缺口KIMI_MODEL_THINKING_MODE / KIMI_MODEL_DEFAULT_THINKING 被忽略无测试(回归保护,防止未来被误加回)。
  • 建议applyEnvModelConfig(MIN, { KIMI_MODEL_THINKING_MODE:'always', KIMI_MODEL_DEFAULT_THINKING:'high' })config.thinkingundefined(或不含 effort/mode

agent/config/index.tsupdate() clamp 集成

  • 现有always_thinking 'off''on'provider 构建 enabledtoggleable 'off' 保持;切到 always_thinking re-clamp 旧 'off'config-state.test.ts:163-228)。
  • 缺口modelAlias 变化 + 旧 effort='off' + 新 always_thinking 模型 + config.effort='max' → 应 clamp 到 'max'(而非 defaultEffort)。update() 在 modelAlias 分支把 this._thinkingEffort 作为 requested 传入clamp 时读 config?.effort——这条路径未在 ConfigState 集成层测。
  • 建议:先 update({modelAlias: toggleable, thinkingEffort:'off'}),再 update({modelAlias: deep}) 且 kimiConfig 带 thinking:{effort:'max'} → 期望 'max'

session/provider-manager.tssupportEfforts 透传给 kimi provider

  • 现有无直接测试。kimi provider 单元覆盖了 supportEfforts 行为,但接线层provider-manager 把 supportEfforts 写进 kimi provider config未测。
  • 缺口effort-capable alias 经 ProviderManager.resolveProviderConfig → kimi ProviderConfig 应携带 supportEfforts
  • 建议:构造 supportEfforts:['low','high','max'] 的 kimi alias断言 resolveProviderConfig(...).provider 的 supportEfforts 透传(或经 config.provider.thinkingEffort/modelParameters 验证)。

acp-adapter

server.tsresolveCurrentThinkingEnabled

  • 现有:无直接测试。仅经 newSession 间接走 getConfigthinking 字段 → false 分支。
  • 缺口(全 P1
    • getConfig 缺失partial stubfalse
    • thinking.enabled === truetrue=== falsefalse
    • thinking.effort 为非空 stringenabled)→ true本次新增核心分支
    • getConfig 抛错 → false
    • thinking 缺失 / effort:''false
  • 建议:抽一个可直接调用 resolveCurrentThinkingEnabled 的测试(或经 newSession + 不同 getConfig 返回值断言 configOptions.thinking.currentValue),覆盖以上 5 条。

server.tssetupSessionFromExisting resumedThinkingEffort 投影

  • 现有thinkingEffort='high' → toggle onsession-resume.test.ts:142-189)。
  • 缺口thinkingEffort='off'currentValue='off' 未测;thinkingEffort=''(空串)→ off 未测。
  • 建议:加 thinkingEffort:'off'thinkingEffort:'' 两条,断言 thinking.currentValue==='off'

session.tsthinkingOnEffort

  • 现有:默认返回 'on'fixture model 无 supportEfforts/defaultEffort)。
  • 缺口effort-capable modeldefaultEffort='high' 或 middle supportEfforts)→ 返回对应 effort而非 'on'harness 缺失(undefined)→ 'on'
  • 建议makeHarness 加 effort-capable modelsupportEfforts:['low','high','max']defaultEffort:'high'),断言 setThinkingCalls 收到 'high';另加无 harness 的 AcpSession.setThinking(true)'on'

apps/kimi-codeTUI

tui/utils/thinking-config.tsthinkingEffortToConfig / isThinkingOn

  • 现有:无直接单元测试。仅经 cli/provider.test.tskimi-tui-message-flow.test.ts 间接覆盖。
  • 缺口thinkingEffortToConfig('off'){enabled:false}('low'){enabled:true,effort:'low'}('on'){enabled:true,effort:'on'} 无直接单测。
  • 建议:新建 test/tui/utils/thinking-config.test.ts,对 thinkingEffortToConfig / isThinkingOn 做参数化断言。

tui/components/dialogs/model-selector.tscommitEffort

  • 现有'on'→effort 模型 defaultEffort'on'→middle'on'→boolean 'on'model-selector.test.ts)。
  • 缺口commitEffort('on')defaultEffort 不在 supportEfforts 内 → 返回 defaultEffort(即使模型未声明支持)。该行为与 agent-core 一致,但 TUI 层未锁定(见"待确认问题 2")。
  • 建议:加 effortModel(..., ['low','high'], 'max') 非当前模型 → Enter → onSelect 收到 thinking:'max'(锁定现状)。

tui/commands/config.tspersistModelSelection short-circuit

  • 现有runtime 不变但 defaultModel 不同 → 仍写入Alt+S 不持久化(kimi-tui-message-flow.test.ts)。
  • 缺口defaultModelthinking.enabledthinking.effort 三者完全相同 → 返回 false、不调用 setConfigshort-circuit config.ts:467-473)。未直接测。
  • 建议:加一条 getConfig 返回 defaultModel:'k2', thinking:{enabled:true,effort:'on'},选择 k2 + onsetConfig 未被调用,且状态提示 "Already using ..."。

P2 缺口nice-to-have

  • provider.tstype-level 开放字符串断言(const e: ThinkingEffort = 'any-custom-effort')。
  • kimi.tskeep 字段在 withThinking 后保留(withExtraBody({thinking:{keep}}).withThinking(...))。
  • anthropic.tsbudgetTokensForEffort'off'/'xhigh'/'max'throw 分支(生产路径被 clampEffort 保护)。
  • google-genai.ts:非 gemini-3 的 low/medium/xhigh/max budget 矩阵gemini-3 getter 反射。
  • thinking.tsdefaultEffort 不在 supportEfforts 内的语义命名;requested='off' + always_thinking ± config.effort 组合。
  • config-state.test.tsmodelAlias 切换保留非 'off' effortthinkingEffort + modelAlias 同时变化。
  • schema.tsthinking.enabled 非 boolean、thinking.effort 非 string → 报错的类型校验。
  • env-model.tsKIMI_MODEL_THINKING_EFFORT 与已有 enabled 的合并base config 带 thinking:{enabled:true}env 设 effort → {enabled:true, effort})。
  • session-resume.test.tsthinkingEffort 大写/带空格mainConfig 缺 thinkingEffort 的 fallback 断言。
  • session.ts thinkingOnEffortcurrentModelId 不在 catalog → 'on'
  • promptThinkingSchema:非 stringnumber/boolean)→ 拒绝。
  • thinking-config.test.tsisThinkingOn 各分支。
  • commitEffort:非 'on' draft 透传。
  • persistModelSelectiondefaultModel 相同但 effort 不同 → 写入。

建议的补充顺序

本 PR 内补(简单 + 高价值)

  1. openai-commonit.each(['on', 'extreme', 'foo']) 显式含 'on'
  2. anthropic clampEffortit.each(['on', 'foo']) adaptive + budget 两条
  3. schemadefault_thinking / thinking.mode 静默忽略 + 写入剥离
  4. env-modelKIMI_MODEL_THINKING_MODE / KIMI_MODEL_DEFAULT_THINKING 被忽略
  5. ACP resolveCurrentThinkingEnabled5 条分支
  6. TUI thinkingEffortToConfig:直接单元测试(off / on / low
  7. kimi withThinking:空 supportEfforts: []
  8. google-genai withThinking'on' / 未知gemini-3 + 非 gemini-3

follow-up避免本 PR 继续膨胀)

  • provider-manager supportEfforts 透传
  • ConfigState.update() modelAlias 切换 + config.effort 集成
  • ACP resume thinkingEffort='off' / ''
  • commitEffort defaultEffort 不在声明内
  • persistModelSelection short-circuit
  • 全部 P2