kimi-code/packages/node-sdk/test/catalog.test.ts
Kai b5efba7abc
fix: consume the model metadata declared by the models.dev catalog (#2015)
* fix: stop advertising Claude thinking efforts for non-Claude models

Models served over the Anthropic protocol whose names carry no Claude
marker (e.g. a catalog-imported Kimi K3) no longer inherit the latest
Opus effort list, so the model selector stops offering levels the model
does not accept. The models.dev catalog import now also parses
reasoning_options and records the declared effort levels on the model
alias, so K3 offers its real levels (low / high / max).

* fix: consume deprecated, override, and input-limit metadata from the models.dev catalog

- Models declared status=deprecated in the catalog are no longer
  offered for import.
- Per-model provider overrides on gateway providers (an npm package
  targeting an Anthropic SDK plus a usable endpoint) now land as alias
  protocol and base_url, so those models are served over the right
  protocol and endpoint; overrides without a usable URL are skipped.
- A declared limit.input now sizes the context budget instead of the
  larger total context window (e.g. gpt-5: 272k instead of 400k).

The model alias schema gains an optional base_url field (not accepted
in overrides) that Anthropic wire resolution prefers over the
provider-level base URL.

* fix: honor thinking-disable semantics and the OpenAI-compatible fallback in catalog imports

- reasoning_options 'none' is the model's off encoding: off_effort flows
  from the catalog through the model alias to the OpenAI wire providers,
  so turning thinking off sends 'none' instead of omitting the effort
  field; models with effort levels but no way to disable thinking are
  imported as always_thinking and no longer offer an Off option.
- Bare Claude family aliases (e.g. sonnet-latest) recover the inferred
  Anthropic effort profile; v2 comment conventions restored.
- Providers whose SDK the catalog does not type now fall back to the
  OpenAI-compatible wire (with a visible "guessed" note) instead of
  being refused; imports lacking a usable endpoint ask for one
  (--base-url on the CLI, a prompt in the TUI). Proprietary SDKs
  (Amazon Bedrock), unrecognized explicit types, and env-placeholder
  URLs are refused with a clear reason.

* fix: align catalog imports with the reference models.dev consumer

- A JSON null tier in declared effort values is now read as the 'none'
  off-encoding (previously such models were wrongly imported as
  always-thinking with no way to turn reasoning off).
- Alpha-status models are filtered out alongside deprecated ones.
- Models whose per-model provider override targets a wire that cannot be
  expressed per-model (e.g. Claude on google-vertex, whose wire here is
  Gemini-mode Vertex, or gpt entries on an Anthropic provider) are
  skipped instead of being imported under the silently wrong protocol.
- interleaved: true no longer pins reasoning_content: the provider's
  default three-field scan is wider and the pinned key only narrowed
  reasoning parsing for gateways answering with another field name.

* fix: require endpoints for Anthropic-compatible catalog imports and honor --base-url

- catalogProviderNeedsBaseUrl now covers the Anthropic wire: a
  non-official Anthropic-compatible vendor without a concrete catalog
  endpoint (e.g. google-vertex-anthropic) must supply --base-url / the
  TUI prompt instead of silently falling back to the default Anthropic
  endpoint.
- --base-url now takes precedence over the catalog-declared endpoint,
  and an empty --base-url is rejected instead of persisting a blank
  endpoint.

* fix: enforce always-on thinking on every wire and refuse Cohere at import

A model that declares always_thinking (e.g. a catalog-imported gpt-5)
no longer resolves to a dishonest off state via thinking.enabled=false
or an SDK/ACP off request: resolution clamps to the model's default
effort on every wire instead of letting upstream keep reasoning while
the UI reports Off. The Anthropic warn-and-send path for unlisted
effort levels is unchanged. Cohere's proprietary SDK joins Amazon
Bedrock on the import-refusal list instead of being guessed as
OpenAI-compatible.

* fix: harden catalog import edge cases

- An explicit but unrecognized catalog type is now refused before
  npm/id inference, so a future catalog protocol is never silently
  miswired through the OpenAI fallback.
- User-supplied --base-url values for Anthropic-wire providers get the
  same trailing-/v1 normalization as catalog endpoints, avoiding
  /v1/v1/messages requests.
- The TUI import prompt rejects env-placeholder base URLs like the CLI
  does.

* fix: await the floating assertion promise in the catalog add CLI test

* refactor: unify catalog import resolution into a single decision function

Wire-type inference, the OpenAI-compatible fallback, proprietary-SDK
refusal, endpoint adaptation, and the base-URL requirement are now
produced together by resolveCatalogImport, one pure resolver consumed
by both the CLI and the TUI — replacing the cooperating predicates
(inferWireType, isGuessedWireType, catalogProviderNeedsBaseUrl) whose
permutations kept producing edge cases. No behavior change.

* fix: close configured-off clamp hole, keep inferWireType compat, carry same-wire override endpoints

- A configured thinking.effort = "off" no longer bypasses the
  always-on clamp: it is treated as absent and the model default
  applies, mirrored on both engines.
- The previously public inferWireType stays as a deprecated
  compatibility wrapper over resolveCatalogImport so existing SDK
  consumers do not break on a patch release.
- Catalog model overrides that stay on the provider's wire but declare
  their own endpoint now persist it on the alias (and the v1 OpenAI
  wire branches honor alias-level base URLs like the Anthropic branch).

* fix: split total window from input cap and close override/endpoint gaps

- max_context_tokens once again means the total context window (used by
  completion budgeting); a model's declared input limit is tracked as
  max_input_tokens, which compaction, context-splice and usage-ratio
  checks prefer — fixing the over-clamping introduced when the input
  cap was stored as the context budget.
- A catalog endpoint declared only as an env placeholder now always
  produces needs-base-url (official SDK included), so credentials are
  never sent to the public vendor host by default.
- api-only per-model overrides are honored as same-wire endpoint
  changes; overrides targeting another known but inexpressible wire
  (e.g. google-genai on an OpenAI gateway) are skipped; same-wire
  models whose declared endpoint is an unusable placeholder are
  skipped instead of silently rerouted.

* style: drop a function-level comment from the v2 thinking resolver

* chore: consolidate the PR's changesets into two user-facing entries
2026-07-22 01:23:31 +08:00

307 lines
9.8 KiB
TypeScript

import type { KimiConfig } from '@moonshot-ai/agent-core';
import { describe, expect, it, vi } from 'vitest';
import {
applyCatalogProvider,
catalogModelToAlias,
catalogProviderModels,
CatalogFetchError,
fetchCatalog,
type CatalogModel,
} from '../src/catalog';
function catalogResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
status,
headers: { 'Content-Type': 'application/json' },
});
}
const model: CatalogModel = {
id: 'm1',
name: 'M1',
maxOutputSize: 64000,
capability: {
image_in: true,
video_in: false,
audio_in: false,
thinking: true,
tool_use: true,
max_context_tokens: 200000,
},
};
describe('fetchCatalog', () => {
it('fetches and returns the catalog map', async () => {
const catalog = { anthropic: { id: 'anthropic', models: { x: { id: 'x', limit: { context: 1000 } } } } };
const fetchMock = vi.fn(async () => catalogResponse(catalog));
const result = await fetchCatalog('https://x/api.json', {
fetchImpl: fetchMock as unknown as typeof fetch,
});
expect(result).toEqual(catalog);
});
it('throws CatalogFetchError on HTTP error', async () => {
const fetchMock = vi.fn(async () => catalogResponse('no', 500));
await expect(
fetchCatalog('https://x', { fetchImpl: fetchMock as unknown as typeof fetch }),
).rejects.toBeInstanceOf(CatalogFetchError);
});
it('throws on a non-object payload', async () => {
const fetchMock = vi.fn(async () => catalogResponse([1, 2]));
await expect(
fetchCatalog('https://x', { fetchImpl: fetchMock as unknown as typeof fetch }),
).rejects.toThrow(/Unexpected catalog response/);
});
it('sends the given User-Agent, and none by default', async () => {
const fetchMock = vi.fn(async () => catalogResponse({}));
await fetchCatalog(
'https://x/api.json',
{
fetchImpl: fetchMock as unknown as typeof fetch,
userAgent: 'kimi-code-cli/1.2.3',
},
);
const withUa = fetchMock.mock.calls[0] as unknown as [string, RequestInit];
const withUaHeaders = withUa[1].headers as Record<string, string>;
expect(withUaHeaders['User-Agent']).toBe('kimi-code-cli/1.2.3');
expect(withUaHeaders['Accept']).toBe('application/json');
fetchMock.mockClear();
await fetchCatalog('https://x/api.json', {
fetchImpl: fetchMock as unknown as typeof fetch,
});
const withoutUa = fetchMock.mock.calls[0] as unknown as [string, RequestInit];
expect((withoutUa[1].headers as Record<string, string>)['User-Agent']).toBeUndefined();
});
});
describe('catalogModelToAlias', () => {
it('flattens a catalog model capability into alias fields', () => {
expect(catalogModelToAlias('anthropic', model)).toEqual({
provider: 'anthropic',
model: 'm1',
maxContextSize: 200000,
maxOutputSize: 64000,
capabilities: ['image_in', 'thinking', 'tool_use'],
displayName: 'M1',
});
});
});
describe('applyCatalogProvider', () => {
it('writes provider, model aliases, and defaults', () => {
const config = { providers: {} } as KimiConfig;
const result = applyCatalogProvider(config, {
providerId: 'anthropic',
wire: 'anthropic',
baseUrl: 'https://api.anthropic.com',
apiKey: 'sk',
models: [model],
selectedModelId: 'm1',
thinking: true,
});
expect(result.defaultModel).toBe('anthropic/m1');
expect(config.providers['anthropic']).toMatchObject({ type: 'anthropic', apiKey: 'sk' });
expect(config.models?.['anthropic/m1']).toMatchObject({
provider: 'anthropic',
model: 'm1',
maxContextSize: 200000,
});
expect(config.defaultModel).toBe('anthropic/m1');
expect(config.thinking?.enabled).toBe(true);
});
it('writes interleaved reasoning key from a catalog-selected model alias', () => {
const models = catalogProviderModels({
id: 'deepseek',
models: {
'deepseek-v4-pro': {
id: 'deepseek-v4-pro',
name: 'DeepSeek V4 Pro',
family: 'deepseek-thinking',
limit: { context: 1000000, output: 384000 },
reasoning: true,
tool_call: true,
interleaved: { field: 'reasoning_content' },
},
},
});
const config = { providers: {} } as KimiConfig;
applyCatalogProvider(config, {
providerId: 'deepseek',
wire: 'openai',
baseUrl: 'https://api.deepseek.com',
apiKey: 'sk',
models,
selectedModelId: 'deepseek-v4-pro',
thinking: true,
});
expect(config.models?.['deepseek/deepseek-v4-pro']).toMatchObject({
provider: 'deepseek',
model: 'deepseek-v4-pro',
reasoningKey: 'reasoning_content',
});
});
it('writes declared effort levels from reasoning_options into the model alias', () => {
// The models.dev `kimi-for-coding` provider shape for `k3`.
const models = catalogProviderModels({
id: 'kimi-for-coding',
models: {
k3: {
id: 'k3',
name: 'Kimi K3',
limit: { context: 1048576, output: 131072 },
reasoning: true,
reasoning_options: [
{ type: 'toggle' },
{ type: 'effort', values: ['low', 'high', 'max'] },
],
tool_call: true,
modalities: { input: ['text', 'image', 'video'], output: ['text'] },
},
},
});
const config = { providers: {} } as KimiConfig;
applyCatalogProvider(config, {
providerId: 'kimi-for-coding',
wire: 'anthropic',
baseUrl: 'https://api.kimi.com/coding',
apiKey: 'sk',
models,
selectedModelId: 'k3',
thinking: true,
});
expect(config.models?.['kimi-for-coding/k3']).toMatchObject({
provider: 'kimi-for-coding',
model: 'k3',
capabilities: ['image_in', 'video_in', 'thinking', 'tool_use'],
supportEfforts: ['low', 'high', 'max'],
});
});
it('writes per-model protocol/baseUrl overrides and the input-limited context size', () => {
// The zenmux gateway shape: provider defaults to the OpenAI wire, one
// model is served over Anthropic on its own endpoint; plus a gpt-5-style
// input cap below the total context window.
const models = catalogProviderModels({
id: 'gateway',
npm: '@ai-sdk/openai-compatible',
api: 'https://gateway.example.test/api/v1',
models: {
'vendor/claude-model': {
id: 'vendor/claude-model',
name: 'Gateway Claude',
limit: { context: 200000 },
provider: {
npm: '@ai-sdk/anthropic',
api: 'https://gateway.example.test/api/anthropic/v1',
},
},
'vendor/gpt-model': {
id: 'vendor/gpt-model',
limit: { context: 400000, input: 272000, output: 128000 },
},
},
});
const config = { providers: {} } as KimiConfig;
applyCatalogProvider(config, {
providerId: 'gateway',
wire: 'openai',
baseUrl: 'https://gateway.example.test/api/v1',
apiKey: 'sk',
models,
selectedModelId: 'vendor/claude-model',
thinking: false,
});
expect(config.models?.['gateway/vendor/claude-model']).toMatchObject({
provider: 'gateway',
model: 'vendor/claude-model',
protocol: 'anthropic',
baseUrl: 'https://gateway.example.test/api/anthropic',
});
const plain = config.models?.['gateway/vendor/gpt-model'];
expect(plain).toMatchObject({ maxContextSize: 400000, maxInputSize: 272000 });
expect(plain?.protocol).toBeUndefined();
expect(plain?.baseUrl).toBeUndefined();
});
it('maps always-thinking models to always_thinking and carries the off encoding', () => {
const models = catalogProviderModels({
id: 'gateway',
models: {
'gpt-5': {
id: 'gpt-5',
reasoning: true,
reasoning_options: [{ type: 'effort', values: ['low', 'medium', 'high'] }],
limit: { context: 400000, input: 272000 },
},
'grok-4': {
id: 'grok-4',
reasoning: true,
reasoning_options: [{ type: 'effort', values: ['none', 'low', 'medium', 'high'] }],
limit: { context: 256000 },
},
},
});
const config = { providers: {} } as KimiConfig;
applyCatalogProvider(config, {
providerId: 'gateway',
wire: 'openai',
baseUrl: 'https://gateway.example.test/v1',
apiKey: 'sk',
models,
selectedModelId: 'gpt-5',
thinking: true,
});
// No off option: thinking is locked on for a model that always reasons.
expect(config.models?.['gateway/gpt-5']).toMatchObject({
capabilities: ['thinking', 'tool_use'].map((c) => (c === 'thinking' ? 'always_thinking' : c)),
supportEfforts: ['low', 'medium', 'high'],
});
expect(config.models?.['gateway/gpt-5']?.capabilities).not.toContain('thinking');
expect(config.models?.['gateway/gpt-5']?.offEffort).toBeUndefined();
// 'none' becomes the off encoding; the level list stays selectable-only.
expect(config.models?.['gateway/grok-4']).toMatchObject({
capabilities: ['thinking', 'tool_use'],
supportEfforts: ['low', 'medium', 'high'],
offEffort: 'none',
});
});
it('clears stale aliases for the same provider but keeps others', () => {
const config = {
providers: { anthropic: { type: 'anthropic', apiKey: 'old' } },
models: {
'anthropic/stale': { provider: 'anthropic', model: 'stale', maxContextSize: 1 },
'other/keep': { provider: 'other', model: 'keep', maxContextSize: 1 },
},
} as unknown as KimiConfig;
applyCatalogProvider(config, {
providerId: 'anthropic',
wire: 'anthropic',
apiKey: 'new',
models: [model],
selectedModelId: 'm1',
thinking: false,
});
expect(config.models?.['anthropic/stale']).toBeUndefined();
expect(config.models?.['other/keep']).toBeDefined();
});
});