From 474ce289dd39aa42d1a77a9a2e15531aee49aa15 Mon Sep 17 00:00:00 2001 From: Kai Date: Tue, 7 Jul 2026 21:38:43 +0800 Subject: [PATCH 01/24] fix(agent-core): report EXIF-rotated image dimensions and raise edge cap to 3000px (#1460) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(agent-core): report EXIF-rotated image dimensions and raise edge cap to 3000px Image compression now reports original dimensions in the decoded (EXIF-rotated) space, matching the coordinate system of the sent image and of ReadMediaFile region readback; previously portrait JPEGs (orientation 5-8) got swapped width/height in captions. The longest-edge downscale cap rises from 2000px to 3000px, and the default jimp resize path is documented as the anti-aliased area-average one so it is not accidentally switched to a point-sampled interpolation mode. * test: shrink oversized image fixtures to fit CI timeouts The 3600x3600 fixtures introduced for the 3000px edge cap nearly doubled the pixel area jimp has to decode and deflate, pushing the slowest compression tests past the 5s vitest timeout on CI runners. 3600x1800 keeps every fixture over the cap while restoring roughly the workload of the old 2600x2600 fixtures that CI handled comfortably. * test: pin anti-aliased downscale quality with executable guards A 1px checkerboard probe pins the compressor to full-coverage averaging at integer and fractional ratios, with jimp's point-sampled BILINEAR mode kept as the executable aliasing counter-example (it collapses the 50%-gray pattern to solid black at 4:1). Also guards the other classic downscale bugs: transparent-pixel color bleed, mean-brightness drift, iterative recompression degradation, and zero-size collapse on extreme aspect ratios. * fix(agent-core): report decoded EXIF-rotated dimensions in ReadMediaFile notes The media note derived its original-dimensions line from the header sniff, which reports pre-rotation values for EXIF orientation 5-8 JPEGs. The sent image and region readback both live in the decoded (rotated) space, so portrait photos got axis-swapped coordinate guidance. Once a decode has happened — compression or crop — its dimensions now overwrite the sniffed ones. * fix(agent-core): improve handling of EXIF orientation in image dimensions and metadata * fix(agent-core): sniff EXIF orientation and step budget fallback through 2000px Two follow-ups to the EXIF and 3000px-cap changes: sniffImageDimensions now reads the JPEG EXIF Orientation tag (pure header parse, both byte orders) and reports display-space dimensions for orientations 5-8. Passthrough images — never decoded — previously kept the pre-rotation header size in compression results and media read notes, disagreeing with the decoded space that region readback uses. encodeWithinBudget steps the over-budget fallback through 2000px before the 1000px last resort. Raising the cap to 3000px had left a regression window: an image whose 2000px encode fits the byte budget was sent at 1000px where the old 2000px cap used to send it at 2000px. * fix(kimi-code): record pasted image dimensions in display space The TUI paste path recorded attachment and original dimensions from its raw header parser, which ignores EXIF orientation. For a portrait JPEG the submit-time caption then contradicted the sent image's aspect and region readback coordinates were axis-swapped. Dimensions now come from the compression result, which reports display space on both the compressed and passthrough paths; parseImageMeta remains only the format/mime gate. * feat(agent-core): add image compression and crop telemetry Every image ingestion path now reports an image_compress event — outcome (compressed / passthrough fast, guard, unsupported, unhelpful, error), input/output formats, byte and pixel sizes, EXIF transposition, and duration — and region readback reports an image_crop event with a failure classification and the region's share of the original area. Wiring is per call site via a new CompressImageOptions.telemetry option, so the outcome split and timing are measured inside the compressor while each caller only names its source: ReadMediaFile (tool construction, like GrepTool), MCP tool results (McpOutputOptions), server prompt ingestion (ICoreProcessService now exposes the host telemetry client), ACP prompts (session track adapter), and TUI paste (host.track adapter). Properties are numeric/enum only — never paths or content — and a throwing client can never affect the compression result. * fix(agent-core): run the full JPEG quality ladder at fallback sizes The fallback rescales encoded only at quality 20, so a JPEG whose ladder failed at the fitted size collapsed straight to the lowest quality even when the smaller size left budget headroom for a higher rung (the realistic window is the 1000px step, where the 4x pixel drop pays for q80/q60). Each fallback edge now walks the same q80-to-q20 ladder as the fitted size. * test: shrink heavy JPEG fixtures and add explicit timeouts The fallback-ladder test runs ~11 pure-JS JPEG encodes and the EXIF paste test decodes, rotates, and re-encodes a 6.5MP frame; both sat at the edge of the 5s vitest timeout on CI runners. Narrower fixtures cut the pixel area (the ladder test keeps its width above 2000px so the full fallback chain still runs) and explicit 15s timeouts absorb runner variance. * fix(server): scope prompt image compression telemetry to the session The prompt-ingestion image_compress events were emitted with the bare host telemetry client, while every agent-side source inherits a session-scoped client — so prompt_inline/prompt_file events could not be correlated with their session. The route now wraps the client with withTelemetryContext({ sessionId }) like rpc/core-impl does for session telemetry. * chore(changeset): consolidate image compression changesets One entry covering the cap raise and the EXIF dimension fix, listed for both the CLI and the SDK so the SDK changelog's compression description (previously pinned at 2000px) stays accurate. --- .changeset/image-compression-improvements.md | 6 + .../src/tui/controllers/editor-keyboard.ts | 25 +- .../editor-keyboard-image-paste.test.ts | 125 +++- packages/acp-adapter/src/convert.ts | 14 +- packages/acp-adapter/src/session.ts | 8 + packages/acp-adapter/test/cancel.test.ts | 9 +- packages/acp-adapter/test/convert.test.ts | 24 +- packages/agent-core/src/agent/tool/index.ts | 9 +- packages/agent-core/src/index.ts | 1 + packages/agent-core/src/mcp/output.ts | 8 + .../src/services/coreProcess/coreProcess.ts | 8 + .../coreProcess/coreProcessService.ts | 4 + .../src/tools/builtin/file/read-media.ts | 22 +- .../agent-core/src/tools/support/file-type.ts | 62 +- .../src/tools/support/image-compress.ts | 342 ++++++++-- packages/agent-core/test/mcp/output.test.ts | 42 +- .../agent-core/test/tools/file-type.test.ts | 86 +++ .../test/tools/image-compress.test.ts | 613 +++++++++++++++++- .../agent-core/test/tools/read-media.test.ts | 171 ++++- packages/node-sdk/src/index.ts | 1 + packages/server/src/routes/prompts.ts | 20 +- packages/server/test/prompt.e2e.test.ts | 70 +- 22 files changed, 1511 insertions(+), 159 deletions(-) create mode 100644 .changeset/image-compression-improvements.md diff --git a/.changeset/image-compression-improvements.md b/.changeset/image-compression-improvements.md new file mode 100644 index 000000000..70e762223 --- /dev/null +++ b/.changeset/image-compression-improvements.md @@ -0,0 +1,6 @@ +--- +"@moonshot-ai/kimi-code": patch +"@moonshot-ai/kimi-code-sdk": patch +--- + +Raise the image downscale cap from 2000px to 3000px, and fix swapped width/height for EXIF-rotated (portrait) photos in compression captions and media read notes so region readback coordinates map correctly. diff --git a/apps/kimi-code/src/tui/controllers/editor-keyboard.ts b/apps/kimi-code/src/tui/controllers/editor-keyboard.ts index af2d8efd6..e76492f7b 100644 --- a/apps/kimi-code/src/tui/controllers/editor-keyboard.ts +++ b/apps/kimi-code/src/tui/controllers/editor-keyboard.ts @@ -407,8 +407,20 @@ export class EditorKeyboardController { // session's media-originals dir when known, else the temp-dir fallback) // and recorded on the attachment, so submit-time expansion can announce // the compression and point the model at the full-fidelity copy. - const compressed = await compressImageForModel(media.bytes, meta.mime); + const compressed = await compressImageForModel(media.bytes, meta.mime, { + telemetry: { + client: { + track: (event, properties) => + this.host.track(event, properties === undefined ? undefined : { ...properties }), + }, + source: 'tui_paste', + }, + }); const sessionDir = this.host.session?.summary?.sessionDir; + // Dimensions come from the compression result, not parseImageMeta: the + // compressor reports display space (EXIF orientation applied) — the space + // the sent image, the caption, and ReadMediaFile region readback share — + // while parseImageMeta reads the raw pre-rotation header. const attachment = compressed.changed ? this.imageStore.addImage( compressed.data, @@ -421,13 +433,18 @@ export class EditorKeyboardController { meta.mime, sessionDir === undefined ? {} : { dir: sessionMediaOriginalsDir(sessionDir) }, ), - width: meta.width, - height: meta.height, + width: compressed.originalWidth, + height: compressed.originalHeight, byteLength: media.bytes.length, mime: meta.mime, }, ) - : this.imageStore.addImage(media.bytes, meta.mime, meta.width, meta.height); + : this.imageStore.addImage( + media.bytes, + meta.mime, + compressed.width || meta.width, + compressed.height || meta.height, + ); this.host.state.editor.insertTextAtCursor?.(`${attachment.placeholder} `); this.host.state.ui.requestRender(); this.host.track('shortcut_paste', { kind: 'image' }); diff --git a/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts b/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts index 4be348fa2..0fe14bc09 100644 --- a/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts +++ b/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts @@ -37,6 +37,7 @@ vi.mock('#/utils/clipboard/clipboard-image', async (importActual) => { interface PasteHarness { readonly store: ImageAttachmentStore; + readonly track: ReturnType; pasteImage(): Promise; } @@ -45,6 +46,7 @@ function createPasteHarness(options: { sessionDir?: string } = {}): PasteHarness setHistoryFilter: vi.fn() as unknown as (...args: never[]) => unknown, }; const store = new ImageAttachmentStore(); + const track = vi.fn(); const host = { state: { editor, @@ -58,7 +60,7 @@ function createPasteHarness(options: { sessionDir?: string } = {}): PasteHarness ? undefined : { summary: { sessionDir: options.sessionDir } }, btwPanelController: { closeOrCancel: vi.fn(() => false) }, - track: vi.fn(), + track, showError: vi.fn(), openUndoSelector: vi.fn(), cancelRunningShellCommand: vi.fn(), @@ -69,6 +71,7 @@ function createPasteHarness(options: { sessionDir?: string } = {}): PasteHarness return { store, + track, async pasteImage() { const handler = editor['onPasteImage']; if (handler === undefined) throw new Error('onPasteImage handler not installed'); @@ -83,13 +86,50 @@ async function solidPng(width: number, height: number): Promise { ); } +async function solidJpeg(width: number, height: number): Promise { + return new Uint8Array( + await new Jimp({ width, height, color: 0x3366ccff }).getBuffer('image/jpeg', { quality: 90 }), + ); +} + +/** + * Insert a minimal EXIF APP1 segment carrying only an Orientation tag right + * after the JPEG SOI marker (jimp itself never writes EXIF). Mirrors the + * fixture in agent-core's image-compress tests. + */ +function withExifOrientation(jpeg: Uint8Array, orientation: number): Uint8Array { + // TIFF body, little-endian: 8-byte header + IFD0 with a single entry. + const tiff = Buffer.alloc(26); + tiff.write('II', 0, 'latin1'); + tiff.writeUInt16LE(42, 2); + tiff.writeUInt32LE(8, 4); // offset of IFD0 + tiff.writeUInt16LE(1, 8); // one directory entry + tiff.writeUInt16LE(0x0112, 10); // tag: Orientation + tiff.writeUInt16LE(3, 12); // type: SHORT + tiff.writeUInt32LE(1, 14); // count + tiff.writeUInt16LE(orientation, 18); // value, left-aligned in the 4-byte field + tiff.writeUInt32LE(0, 22); // no next IFD + const exifBody = Buffer.concat([Buffer.from('Exif\0\0', 'latin1'), tiff]); + const app1Header = Buffer.alloc(4); + app1Header.writeUInt16BE(0xff_e1, 0); + app1Header.writeUInt16BE(exifBody.length + 2, 2); + return new Uint8Array( + Buffer.concat([ + Buffer.from(jpeg.subarray(0, 2)), // SOI + app1Header, + exifBody, + Buffer.from(jpeg.subarray(2)), + ]), + ); +} + describe('clipboard image paste compression', () => { beforeEach(() => { readClipboardMedia.mockReset(); }); it('downsamples an oversized pasted image before storing it', async () => { - const big = await solidPng(2600, 2600); + const big = await solidPng(3600, 1800); readClipboardMedia.mockResolvedValue({ kind: 'image', bytes: big, mimeType: 'image/png' }); const { store, pasteImage } = createPasteHarness(); @@ -101,18 +141,18 @@ describe('clipboard image paste compression', () => { if (att?.kind !== 'image') throw new Error('expected image attachment'); // Stored metadata reflects the compressed size. - expect(Math.max(att.width, att.height)).toBeLessThanOrEqual(2000); - expect(att.placeholder).toContain('2000×2000'); + expect(Math.max(att.width, att.height)).toBeLessThanOrEqual(3000); + expect(att.placeholder).toContain('3000×1500'); // The stored bytes decode to the compressed dimensions — the thumbnail and // the submitted image both read from these bytes, so they cannot diverge. const dims = parseImageMeta(att.bytes); expect(dims).not.toBeNull(); - expect(Math.max(dims!.width, dims!.height)).toBeLessThanOrEqual(2000); + expect(Math.max(dims!.width, dims!.height)).toBeLessThanOrEqual(3000); }); it('records and persists the pre-compression original for an oversized paste', async () => { - const big = await solidPng(2600, 2600); + const big = await solidPng(3600, 1800); readClipboardMedia.mockResolvedValue({ kind: 'image', bytes: big, mimeType: 'image/png' }); const { store, pasteImage } = createPasteHarness(); @@ -121,8 +161,8 @@ describe('clipboard image paste compression', () => { const att = store.get(1); if (att?.kind !== 'image') throw new Error('expected image attachment'); expect(att.original).toBeDefined(); - expect(att.original?.width).toBe(2600); - expect(att.original?.height).toBe(2600); + expect(att.original?.width).toBe(3600); + expect(att.original?.height).toBe(1800); expect(att.original?.byteLength).toBe(big.length); expect(att.original?.mime).toBe('image/png'); @@ -135,7 +175,7 @@ describe('clipboard image paste compression', () => { it('persists the original into the session media-originals dir when the session is known', async () => { const sessionDir = await mkdtemp(join(tmpdir(), 'kimi-paste-session-')); - const big = await solidPng(2600, 2600); + const big = await solidPng(3600, 1800); readClipboardMedia.mockResolvedValue({ kind: 'image', bytes: big, mimeType: 'image/png' }); const { store, pasteImage } = createPasteHarness({ sessionDir }); @@ -164,4 +204,71 @@ describe('clipboard image paste compression', () => { expect(att.bytes).toBe(small); // identity: no re-encode on the fast path expect(att.original).toBeUndefined(); }); + + it( + 'records an EXIF-rotated compressed original in display space', + async () => { + // Orientation 6 (rotate 90° CW): the header says 3600x400, but the image + // decodes to 400x3600 — the space the compressed bytes and any later + // ReadMediaFile region readback live in. The recorded original (which + // drives the submit-time compression caption) must match that space, or + // the caption contradicts the sent image's aspect and region coordinates + // land axis-swapped. (Kept narrow: pure-JS decode+rotate+encode of a + // larger frame can outlast the test timeout on slow CI runners.) + const portrait = withExifOrientation(await solidJpeg(3600, 400), 6); + readClipboardMedia.mockResolvedValue({ + kind: 'image', + bytes: portrait, + mimeType: 'image/jpeg', + }); + + const { store, pasteImage } = createPasteHarness(); + await pasteImage(); + + const att = store.get(1); + if (att?.kind !== 'image') throw new Error('expected image attachment'); + expect(att.original?.width).toBe(400); + expect(att.original?.height).toBe(3600); + // The compressed attachment itself keeps the portrait aspect. + expect(att.width).toBeLessThan(att.height); + await unlink(att.original!.path!).catch(() => undefined); + }, + 15_000, + ); + + it('stores display-space dimensions for an EXIF-rotated untouched paste', async () => { + // Within budgets → sent byte-for-byte, but the placeholder and metadata + // must still describe the display (rotated) space. + const portrait = withExifOrientation(await solidJpeg(120, 80), 6); + readClipboardMedia.mockResolvedValue({ + kind: 'image', + bytes: portrait, + mimeType: 'image/jpeg', + }); + + const { store, pasteImage } = createPasteHarness(); + await pasteImage(); + + const att = store.get(1); + if (att?.kind !== 'image') throw new Error('expected image attachment'); + expect(att.bytes).toBe(portrait); // fast path — untouched + expect(att.original).toBeUndefined(); + expect(att.width).toBe(80); + expect(att.height).toBe(120); + expect(att.placeholder).toContain('80×120'); + }); + + it('emits image_compress telemetry tagged tui_paste through host.track', async () => { + const big = await solidPng(3600, 1800); + readClipboardMedia.mockResolvedValue({ kind: 'image', bytes: big, mimeType: 'image/png' }); + + const { track, pasteImage } = createPasteHarness(); + await pasteImage(); + + const compressCalls = track.mock.calls.filter(([event]) => event === 'image_compress'); + expect(compressCalls).toHaveLength(1); + const props = compressCalls[0]![1] as Record; + expect(props['source']).toBe('tui_paste'); + expect(props['outcome']).toBe('compressed'); + }); }); diff --git a/packages/acp-adapter/src/convert.ts b/packages/acp-adapter/src/convert.ts index c2a3ca8d7..bf8cdb197 100644 --- a/packages/acp-adapter/src/convert.ts +++ b/packages/acp-adapter/src/convert.ts @@ -5,6 +5,7 @@ import { compressBase64ForModel, persistOriginalImage, type PromptPart, + type TelemetryClient, type ToolInputDisplay, type ToolResultEvent, } from '@moonshot-ai/kimi-code-sdk'; @@ -88,14 +89,23 @@ export function acpBlocksToPromptParts( */ export async function compressPromptImageParts( parts: readonly PromptPart[], - options: { readonly originalsDir?: string | undefined } = {}, + options: { + readonly originalsDir?: string | undefined; + /** Report an `image_compress` event per prompt image (source `acp_prompt`). */ + readonly telemetry?: TelemetryClient | undefined; + } = {}, ): Promise { const out: PromptPart[] = []; for (const part of parts) { if (part.type === 'image_url') { const parsed = parseImageDataUrl(part.imageUrl.url); if (parsed !== null) { - const result = await compressBase64ForModel(parsed.base64, parsed.mimeType); + const result = await compressBase64ForModel(parsed.base64, parsed.mimeType, { + telemetry: + options.telemetry === undefined + ? undefined + : { client: options.telemetry, source: 'acp_prompt' }, + }); if (result.changed) { const originalPath = await persistOriginalImage( Buffer.from(parsed.base64, 'base64'), diff --git a/packages/acp-adapter/src/session.ts b/packages/acp-adapter/src/session.ts index c00cc181b..7fc01f444 100644 --- a/packages/acp-adapter/src/session.ts +++ b/packages/acp-adapter/src/session.ts @@ -737,9 +737,17 @@ export class AcpSession { let parts: readonly PromptPart[]; try { const sessionDir = this.session.summary?.sessionDir; + const track = this.track; parts = await compressPromptImageParts(acpBlocksToPromptParts(blocks), { originalsDir: sessionDir === undefined ? undefined : sessionMediaOriginalsDir(sessionDir), + telemetry: + track === undefined + ? undefined + : { + track: (event, properties) => + track(event, properties === undefined ? undefined : { ...properties }), + }, }); } finally { this.pendingPromptAborts.delete(pending); diff --git a/packages/acp-adapter/test/cancel.test.ts b/packages/acp-adapter/test/cancel.test.ts index e741db135..bbe243ada 100644 --- a/packages/acp-adapter/test/cancel.test.ts +++ b/packages/acp-adapter/test/cancel.test.ts @@ -163,10 +163,11 @@ describe('AcpServer cancel', () => { const { sessionId } = await client.newSession({ cwd: '/tmp/x', mcpServers: [] }); - // A solid 2600×2600 image is small in bytes but slow enough to compress - // that the cancel below reliably lands mid-compression, before any turn. + // A solid 3600×1800 image is small in bytes but slow enough to compress + // that the cancel below reliably lands mid-compression, before any turn + // — while staying safely inside the 5s test timeout on slow CI runners. const data = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ).toString('base64'); const promptP = client.prompt({ @@ -203,7 +204,7 @@ describe('AcpServer cancel', () => { const { sessionId } = await client.newSession({ cwd: '/tmp/x', mcpServers: [] }); const data = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ).toString('base64'); const imageBlock = { type: 'image' as const, data, mimeType: 'image/png' }; diff --git a/packages/acp-adapter/test/convert.test.ts b/packages/acp-adapter/test/convert.test.ts index 79c8b96ac..cb4117f48 100644 --- a/packages/acp-adapter/test/convert.test.ts +++ b/packages/acp-adapter/test/convert.test.ts @@ -338,7 +338,7 @@ describe('compressPromptImageParts', () => { it('downsamples an oversized inline image part and announces the compression', async () => { const originalsDir = await mkdtemp(join(tmpdir(), 'acp-originals-')); - const originalBase64 = await pngBase64(2600, 2600); + const originalBase64 = await pngBase64(3600, 1800); const parts = acpBlocksToPromptParts([imageBlock(originalBase64, 'image/png')]); const compressed = await compressPromptImageParts(parts, { originalsDir }); @@ -348,14 +348,14 @@ describe('compressPromptImageParts', () => { const caption = compressed[0]; if (caption?.type !== 'text') throw new Error('expected a caption text part'); expect(caption.text).toContain('Image compressed'); - expect(caption.text).toContain('2600x2600'); + expect(caption.text).toContain('3600x1800'); const part = compressed[1]; if (part?.type !== 'image_url') throw new Error('expected an image_url part'); const match = /^data:(image\/[a-z]+);base64,(.+)$/.exec(part.imageUrl.url); expect(match).not.toBeNull(); const decoded = await Jimp.fromBuffer(Buffer.from(match![2]!, 'base64')); - expect(Math.max(decoded.width, decoded.height)).toBeLessThanOrEqual(2000); + expect(Math.max(decoded.width, decoded.height)).toBeLessThanOrEqual(3000); // The caption points at a persisted copy of the ORIGINAL bytes, placed in // the provided (session-scoped) originals dir. @@ -367,6 +367,24 @@ describe('compressPromptImageParts', () => { await rm(originalsDir, { recursive: true, force: true }); }); + it('emits image_compress telemetry tagged acp_prompt', async () => { + const originalsDir = await mkdtemp(join(tmpdir(), 'acp-originals-')); + const events: { event: string; props: Record }[] = []; + const parts = acpBlocksToPromptParts([ + imageBlock(await pngBase64(3600, 1800), 'image/png'), + ]); + await compressPromptImageParts(parts, { + originalsDir, + telemetry: { track: (event, props) => events.push({ event, props: { ...props } }) }, + }); + + expect(events).toHaveLength(1); + expect(events[0]!.event).toBe('image_compress'); + expect(events[0]!.props['source']).toBe('acp_prompt'); + expect(events[0]!.props['outcome']).toBe('compressed'); + await rm(originalsDir, { recursive: true, force: true }); + }); + it('passes a within-budget image and text through unchanged', async () => { const parts = acpBlocksToPromptParts([ imageBlock(await pngBase64(32, 32), 'image/png'), diff --git a/packages/agent-core/src/agent/tool/index.ts b/packages/agent-core/src/agent/tool/index.ts index 5ddd0dc3c..f46dcd546 100644 --- a/packages/agent-core/src/agent/tool/index.ts +++ b/packages/agent-core/src/agent/tool/index.ts @@ -316,6 +316,7 @@ export class ToolManager { ); return mcpResultToExecutableOutput(result, qualified, { originalsDir: this.agent.mediaOriginalsDir, + telemetry: this.agent.telemetry, }); }, }; @@ -700,7 +701,13 @@ export class ToolManager { allowBackground, }), (modelCapabilities.image_in || modelCapabilities.video_in) && - new b.ReadMediaFileTool(kaos, workspace, modelCapabilities, videoUploader), + new b.ReadMediaFileTool( + kaos, + workspace, + modelCapabilities, + videoUploader, + this.agent.telemetry, + ), new b.EnterPlanModeTool(this.agent), new b.ExitPlanModeTool(this.agent), // Registered unconditionally: the tool-select flag can flip at runtime diff --git a/packages/agent-core/src/index.ts b/packages/agent-core/src/index.ts index 01e159c0e..b347ac5d6 100644 --- a/packages/agent-core/src/index.ts +++ b/packages/agent-core/src/index.ts @@ -76,6 +76,7 @@ export type { CropImageOptions, CropImageOutcome, ImageCompressionCaptionInput, + ImageCompressionTelemetry, ImageCropRegion, ImageVariantDescription, } from './tools/support/image-compress'; diff --git a/packages/agent-core/src/mcp/output.ts b/packages/agent-core/src/mcp/output.ts index 6c5b5a777..0f77683f9 100644 --- a/packages/agent-core/src/mcp/output.ts +++ b/packages/agent-core/src/mcp/output.ts @@ -29,6 +29,8 @@ import type { ContentPart } from '@moonshot-ai/kosong'; +import type { TelemetryClient } from '#/telemetry'; + import { compressImageContentParts } from '../tools/support/image-compress'; import { persistOriginalImage } from '../tools/support/image-originals'; import type { MCPContentBlock, MCPToolResult } from './types'; @@ -40,6 +42,8 @@ export interface McpOutputOptions { * Falls back to the shared temp-dir cache when absent. */ readonly originalsDir?: string | undefined; + /** Report an `image_compress` event per compressed tool-result image. */ + readonly telemetry?: TelemetryClient | undefined; } // MCP servers can produce arbitrarily large outputs; cap what we feed back to @@ -184,6 +188,10 @@ export async function mcpResultToExecutableOutput( // DATA (never inserted into the parts), so tool output that merely quotes // a caption can never be mistaken for a generated one. const compressed = await compressImageContentParts(budgeted.parts, { + telemetry: + options.telemetry === undefined + ? undefined + : { client: options.telemetry, source: 'mcp_tool_result' }, annotate: { persistOriginal: (bytes, mimeType) => persistOriginalImage( diff --git a/packages/agent-core/src/services/coreProcess/coreProcess.ts b/packages/agent-core/src/services/coreProcess/coreProcess.ts index 2a8b5ff03..73b8b27c7 100644 --- a/packages/agent-core/src/services/coreProcess/coreProcess.ts +++ b/packages/agent-core/src/services/coreProcess/coreProcess.ts @@ -31,6 +31,7 @@ import { createDecorator } from '../../di'; import type { CoreRPC, KimiCoreOptions } from '../../rpc'; +import type { TelemetryClient } from '../../telemetry'; import { type KimiHostIdentity } from '@moonshot-ai/kimi-code-oauth'; export interface CoreProcessServiceOptions extends KimiCoreOptions { @@ -60,6 +61,13 @@ export interface ICoreProcessService { readonly kimiRequestHeaders?: Record | undefined; + /** + * The telemetry client the host wired into `KimiCore` (noop when the host + * supplied none), so daemon-side code — e.g. prompt-ingestion image + * compression — reports through the same sink as core events. + */ + readonly telemetry?: TelemetryClient | undefined; + /** * Resolves once `KimiCore` is fully constructed and the SDK side of the * in-process RPC has been bound. Repeated calls return the cached promise. diff --git a/packages/agent-core/src/services/coreProcess/coreProcessService.ts b/packages/agent-core/src/services/coreProcess/coreProcessService.ts index 58bb34452..c66422bff 100644 --- a/packages/agent-core/src/services/coreProcess/coreProcessService.ts +++ b/packages/agent-core/src/services/coreProcess/coreProcessService.ts @@ -6,6 +6,7 @@ import { createRPC, KimiCore } from '../../rpc'; import { Disposable, registerSingleton, SyncDescriptor } from '../../di'; import type { CoreAPI, CoreRPC, SDKAPI } from '../../rpc'; import type { OAuthTokenProviderResolver } from '../../session/provider-manager'; +import { noopTelemetryClient, type TelemetryClient } from '../../telemetry'; import { createKimiDefaultHeaders, type KimiHostIdentity, @@ -33,6 +34,8 @@ export class CoreProcessService extends Disposable implements ICoreProcessServic public readonly kimiRequestHeaders: Record | undefined; + public readonly telemetry: TelemetryClient; + /** * The in-process `KimiCore` instance. Kept private so daemon-side code can't * grab it and bypass the peer-service indirection. @@ -96,6 +99,7 @@ export class CoreProcessService extends Disposable implements ICoreProcessServic this.kimiRequestHeaders = options.kimiRequestHeaders ?? CoreProcessService._defaultKimiRequestHeaders(env.homeDir, options.identity); + this.telemetry = options.telemetry ?? noopTelemetryClient; // `appVersion` flows into Session records (`app_version`) and tool // call ctx. Prefer explicit > identity.version so callers can pin diff --git a/packages/agent-core/src/tools/builtin/file/read-media.ts b/packages/agent-core/src/tools/builtin/file/read-media.ts index f654829a8..5676cbc40 100644 --- a/packages/agent-core/src/tools/builtin/file/read-media.ts +++ b/packages/agent-core/src/tools/builtin/file/read-media.ts @@ -36,6 +36,7 @@ import { z } from 'zod'; import type { BuiltinTool } from '../../../agent/tool'; import { ToolAccesses } from '../../../loop/tool-access'; import type { ExecutableToolResult, ToolExecution } from '../../../loop/types'; +import type { TelemetryClient } from '../../../telemetry'; import { renderPrompt } from '../../../utils/render-prompt'; import { resolvePathAccessPath } from '../../policies/path-access'; import { MEDIA_SNIFF_BYTES, detectFileType, sniffImageDimensions } from '../../support/file-type'; @@ -44,6 +45,7 @@ import { compressImageForModel, cropImageForModel, formatByteSize, + type ImageCompressionTelemetry, type ImageCropRegion, } from '../../support/image-compress'; import { toInputJsonSchema } from '../../support/input-schema'; @@ -215,11 +217,13 @@ export class ReadMediaFileTool implements BuiltinTool { readonly name = 'ReadMediaFile' as const; readonly description: string; readonly parameters: Record = toInputJsonSchema(ReadMediaFileInputSchema); + private readonly compressTelemetry: ImageCompressionTelemetry | undefined; constructor( private readonly kaos: Kaos, private readonly workspace: WorkspaceConfig, private readonly capabilities: ModelCapability, private readonly videoUploader?: VideoUploader | undefined, + telemetry?: TelemetryClient, ) { if (!capabilities.image_in && !capabilities.video_in) { const skip = new Error('ReadMediaFile requires image_in or video_in capability'); @@ -227,6 +231,8 @@ export class ReadMediaFileTool implements BuiltinTool { throw skip; } this.description = buildDescription(capabilities); + this.compressTelemetry = + telemetry === undefined ? undefined : { client: telemetry, source: 'read_media' }; } resolveExecution(args: ReadMediaFileInput): ToolExecution { @@ -330,6 +336,7 @@ export class ReadMediaFileTool implements BuiltinTool { // full fidelity, so a prior downsampled view can be zoomed into. const outcome = await cropImageForModel(data, fileType.mimeType, args.region, { skipResize: args.full_resolution === true, + telemetry: this.compressTelemetry, }); if (!outcome.ok) { return { isError: true, output: `Cannot read region from "${args.path}": ${outcome.error}` }; @@ -348,8 +355,10 @@ export class ReadMediaFileTool implements BuiltinTool { region: outcome.region, resized: outcome.resized, }; - // The decode knows the true size even when header sniffing does not. - dimensions ??= { width: outcome.originalWidth, height: outcome.originalHeight }; + // The decode is authoritative: it covers formats and nonconforming + // EXIF the header sniff cannot read, and region coordinates live + // in the decoded space, so the note must report it. + dimensions = { width: outcome.originalWidth, height: outcome.originalHeight }; } else if (args.full_resolution === true) { // Native resolution on request — but the provider's per-image byte // ceiling is a hard limit, so refuse explicitly rather than degrade. @@ -382,7 +391,9 @@ export class ReadMediaFileTool implements BuiltinTool { // tokens nor trips the provider's per-image byte ceiling. Best effort: // on any failure compressImageForModel returns the original bytes, so // the read still succeeds with the uncompressed image. - const compressed = await compressImageForModel(data, fileType.mimeType); + const compressed = await compressImageForModel(data, fileType.mimeType, { + telemetry: this.compressTelemetry, + }); const base64 = Buffer.from(compressed.data).toString('base64'); mediaPart = { type: 'image_url', @@ -395,6 +406,11 @@ export class ReadMediaFileTool implements BuiltinTool { byteLength: compressed.finalByteLength, mimeType: compressed.mimeType, }; + if (compressed.changed) { + // Same as the crop path: once a decode happened, its dimensions + // are authoritative over the header sniff. + dimensions = { width: compressed.originalWidth, height: compressed.originalHeight }; + } } } else if (this.videoUploader !== undefined) { mediaPart = await this.videoUploader({ diff --git a/packages/agent-core/src/tools/support/file-type.ts b/packages/agent-core/src/tools/support/file-type.ts index 49b6d1980..ea9058774 100644 --- a/packages/agent-core/src/tools/support/file-type.ts +++ b/packages/agent-core/src/tools/support/file-type.ts @@ -237,6 +237,11 @@ export function sniffMediaFromMagic(data: Buffer | Uint8Array): FileType | null export interface ImageDimensions { readonly width: number; readonly height: number; + /** + * Present (true) when a JPEG EXIF orientation of 5-8 swapped the reported + * width/height into display space. + */ + readonly transposed?: boolean; } /** @@ -247,6 +252,11 @@ export interface ImageDimensions { * after the `WEBP` tag, or the first JPEG SOFn segment). Returns `null` * for formats whose dimensions are not locatable from that region, or * when the supplied buffer is too short to cover it. + * + * JPEG dimensions are reported in DISPLAY space: an EXIF Orientation of + * 5-8 transposes the image at decode time, so the SOF width/height are + * swapped to match what decoders (and this codebase's crop regions and + * compression captions) actually operate in. */ export function sniffImageDimensions(data: Buffer | Uint8Array): ImageDimensions | null { const buf = toBuffer(data); @@ -297,8 +307,10 @@ export function sniffImageDimensions(data: Buffer | Uint8Array): ImageDimensions } // JPEG — scan segment markers for a Start-Of-Frame (SOFn) marker, - // whose payload carries height/width as big-endian uint16. + // whose payload carries height/width as big-endian uint16. An EXIF + // APP1 segment encountered on the way supplies the orientation. if (startsWith(buf, [0xff, 0xd8])) { + let orientation: number | null = null; let offset = 2; while (offset + 9 < buf.length) { if (buf[offset] !== 0xff) { @@ -314,10 +326,11 @@ export function sniffImageDimensions(data: Buffer | Uint8Array): ImageDimensions marker !== 0xc8 && marker !== 0xcc ) { - return { - height: buf.readUInt16BE(offset + 5), - width: buf.readUInt16BE(offset + 7), - }; + const height = buf.readUInt16BE(offset + 5); + const width = buf.readUInt16BE(offset + 7); + return orientation !== null && orientation >= 5 + ? { width: height, height: width, transposed: true } + : { width, height }; } // Standalone markers (RSTn, SOI, EOI) carry no length field. if (marker === 0xd8 || marker === 0xd9 || (marker >= 0xd0 && marker <= 0xd7)) { @@ -326,6 +339,9 @@ export function sniffImageDimensions(data: Buffer | Uint8Array): ImageDimensions } const segmentLength = buf.readUInt16BE(offset + 2); if (segmentLength < 2) break; + if (marker === 0xe1 && orientation === null) { + orientation = readExifOrientation(buf, offset + 4, offset + 2 + segmentLength); + } offset += 2 + segmentLength; } } @@ -333,6 +349,42 @@ export function sniffImageDimensions(data: Buffer | Uint8Array): ImageDimensions return null; } +/** + * Read the Orientation tag (0x0112) out of a JPEG APP1 payload spanning + * [`start`, `end`). Returns 1-8, or `null` when the payload is not EXIF, + * is truncated, or carries no valid orientation. Only IFD0 is examined — + * that is where the tag lives; nothing here follows nested IFDs. + */ +function readExifOrientation(buf: Buffer, start: number, end: number): number | null { + const boundedEnd = Math.min(end, buf.length); + // 'Exif\0\0' preamble, then the TIFF header. + if (start + 6 > boundedEnd || buf.toString('latin1', start, start + 6) !== 'Exif\0\0') { + return null; + } + const tiff = start + 6; + if (tiff + 8 > boundedEnd) return null; + const byteOrder = buf.toString('latin1', tiff, tiff + 2); + const le = byteOrder === 'II'; + if (!le && byteOrder !== 'MM') return null; + const u16 = (offset: number): number => (le ? buf.readUInt16LE(offset) : buf.readUInt16BE(offset)); + const u32 = (offset: number): number => (le ? buf.readUInt32LE(offset) : buf.readUInt32BE(offset)); + if (u16(tiff + 2) !== 42) return null; + const ifd = tiff + u32(tiff + 4); + if (ifd + 2 > boundedEnd) return null; + const entryCount = u16(ifd); + for (let i = 0; i < entryCount; i += 1) { + const entry = ifd + 2 + i * 12; + if (entry + 12 > boundedEnd) return null; + if (u16(entry) === 0x0112) { + // Type SHORT: the value sits in the first two bytes of the 4-byte + // value field, in the TIFF byte order. + const value = u16(entry + 8); + return value >= 1 && value <= 8 ? value : null; + } + } + return null; +} + function getSuffix(path: string): string { const idx = path.lastIndexOf('.'); if (idx === -1) return ''; diff --git a/packages/agent-core/src/tools/support/image-compress.ts b/packages/agent-core/src/tools/support/image-compress.ts index 34f62ecb5..8804eb1c0 100644 --- a/packages/agent-core/src/tools/support/image-compress.ts +++ b/packages/agent-core/src/tools/support/image-compress.ts @@ -28,10 +28,12 @@ import type { ContentPart } from '@moonshot-ai/kosong'; +import type { TelemetryClient } from '#/telemetry'; + import { sniffImageDimensions } from './file-type'; /** Longest-edge ceiling (px). Larger images are scaled down to fit. */ -export const MAX_IMAGE_EDGE_PX = 2000; +export const MAX_IMAGE_EDGE_PX = 3000; /** * Raw-byte budget for a single image. base64 inflates bytes by ~4/3, so a @@ -43,8 +45,13 @@ export const IMAGE_BYTE_BUDGET = 3.75 * 1024 * 1024; /** Progressively lower JPEG quality until the payload fits the byte budget. */ const JPEG_QUALITY_STEPS = [80, 60, 40, 20] as const; -/** Last-ditch longest edge when the budget cannot be met at MAX_IMAGE_EDGE_PX. */ -const FALLBACK_EDGE_PX = 1000; +/** + * Longest-edge step-downs tried when the budget cannot be met at the fitted + * size. The 2000px step preserves the behavior of the previous 2000px cap: + * an image whose 2000px encode fits the budget keeps that resolution + * instead of dropping straight to the 1000px last resort. + */ +const FALLBACK_EDGES_PX = [2000, 1000] as const; /** * Pixel-count ceiling above which we skip compression entirely. A tiny-byte, @@ -79,8 +86,36 @@ export interface CompressImageOptions { readonly byteBudget?: number; /** Override the raw-byte ceiling above which compression is skipped. */ readonly maxDecodeBytes?: number; + /** + * Report an `image_compress` event per compression call (and an + * `image_crop` event per {@link cropImageForModel} call). Absent → silent. + */ + readonly telemetry?: ImageCompressionTelemetry; } +/** Wiring for the optional compression telemetry events. */ +export interface ImageCompressionTelemetry { + readonly client: TelemetryClient; + /** Where the image entered the pipeline, e.g. 'read_media', 'tui_paste'. */ + readonly source: string; +} + +/** + * How a compression call ended, as reported in the `image_compress` event. + * Every `passthrough_*` variant returns the input bytes unchanged: `fast` is + * the within-budgets hot path, `guard` a decode-safety refusal (pixel bomb or + * byte cap), `unsupported` a format the codec cannot re-encode (or empty + * input), `unhelpful` a re-encode that saved neither bytes nor pixels, and + * `error` a decode/encode failure. + */ +type CompressOutcome = + | 'compressed' + | 'passthrough_fast' + | 'passthrough_guard' + | 'passthrough_unsupported' + | 'passthrough_unhelpful' + | 'passthrough_error'; + export interface CompressImageResult { /** Bytes to send: the re-encoded image, or the original when unchanged. */ readonly data: Uint8Array; @@ -90,9 +125,13 @@ export interface CompressImageResult { readonly width: number; /** Pixel height of `data`; falls back to the input size when unknown. */ readonly height: number; - /** Pixel width of the input image; 0 when it cannot be sniffed. */ + /** + * Pixel width of the input image, in display space (EXIF orientation + * applied): the decoded width when re-encoded, the header sniff on + * passthrough (0 when it cannot be determined). + */ readonly originalWidth: number; - /** Pixel height of the input image; 0 when it cannot be sniffed. */ + /** Pixel height of the input image; see {@link originalWidth}. */ readonly originalHeight: number; /** True only when `data` differs from the input bytes. */ readonly changed: boolean; @@ -112,6 +151,7 @@ export async function compressImageForModel( mimeType: string, options: CompressImageOptions = {}, ): Promise { + const startedAt = Date.now(); const maxEdge = options.maxEdge ?? MAX_IMAGE_EDGE_PX; const byteBudget = options.byteBudget ?? IMAGE_BYTE_BUDGET; const maxDecodeBytes = options.maxDecodeBytes ?? MAX_DECODE_BYTES; @@ -129,28 +169,50 @@ export async function compressImageForModel( originalByteLength: bytes.length, finalByteLength: bytes.length, }); + const finish = (outcome: CompressOutcome, result: CompressImageResult): CompressImageResult => { + reportCompressEvent(options.telemetry, { + outcome, + startedAt, + inputMime: normalizedMime, + exifTransposed: dims?.transposed === true, + result, + }); + return result; + }; - if (bytes.length === 0) return passthrough(); + if (bytes.length === 0) return finish('passthrough_unsupported', passthrough()); // Only re-encode formats the codec handles; everything else passes through. - if (!RECODABLE_MIME.has(normalizedMime)) return passthrough(); + if (!RECODABLE_MIME.has(normalizedMime)) return finish('passthrough_unsupported', passthrough()); // Fast path: already within both budgets — no codec load, no allocation. const longestEdge = dims ? Math.max(dims.width, dims.height) : 0; const withinBytes = bytes.length <= byteBudget; const withinEdge = longestEdge > 0 && longestEdge <= maxEdge; - if (withinBytes && (withinEdge || longestEdge === 0)) return passthrough(); + if (withinBytes && (withinEdge || longestEdge === 0)) { + return finish('passthrough_fast', passthrough()); + } // Decompression-bomb guard: refuse to decode absurd pixel counts. The sniff // above gave us the dimensions without decoding, so this costs nothing. - if (dims && dims.width * dims.height > MAX_DECODE_PIXELS) return passthrough(); + if (dims && dims.width * dims.height > MAX_DECODE_PIXELS) { + return finish('passthrough_guard', passthrough()); + } // Refuse to decode very large byte payloads (e.g. a huge or invalid image // from an MCP tool) that would be loaded just to be dropped downstream. - if (bytes.length > maxDecodeBytes) return passthrough(); + if (bytes.length > maxDecodeBytes) return finish('passthrough_guard', passthrough()); try { const { Jimp } = await import('jimp'); const image = await Jimp.fromBuffer(Buffer.from(bytes)); const sourceIsPng = normalizedMime === 'image/png'; + // The decoded bitmap is authoritative for the original size: jimp + // applies EXIF orientation while decoding, and this is the coordinate + // space the encoded result and any later crop region (see + // cropImageForModel, which decodes the same way) actually live in. The + // header sniff also reports display space, but can miss formats or + // nonconforming EXIF that the decoder still handles. + const decodedWidth = image.width; + const decodedHeight = image.height; // Scale so the longest edge fits maxEdge (never enlarges). fitWithinEdge(image, maxEdge); @@ -158,33 +220,33 @@ export async function compressImageForModel( const encoded = await encodeWithinBudget(image, { sourceIsPng, byteBudget, - fallbackEdge: FALLBACK_EDGE_PX, + fallbackEdges: FALLBACK_EDGES_PX, }); // Keep the result when it actually helps: fewer bytes, or fewer pixels // (a smaller image costs fewer vision tokens even if the byte count is // flat, as with near-solid graphics). Otherwise the re-encode bought us // nothing — send the original. - const originalPixels = (dims?.width ?? 0) * (dims?.height ?? 0); + const originalPixels = decodedWidth * decodedHeight; const finalPixels = encoded.width * encoded.height; const shrankBytes = encoded.data.length < bytes.length; - const shrankPixels = originalPixels > 0 && finalPixels < originalPixels; - if (!shrankBytes && !shrankPixels) return passthrough(); + const shrankPixels = finalPixels < originalPixels; + if (!shrankBytes && !shrankPixels) return finish('passthrough_unhelpful', passthrough()); - return { + return finish('compressed', { data: encoded.data, mimeType: encoded.mimeType, width: encoded.width, height: encoded.height, - originalWidth: dims?.width ?? 0, - originalHeight: dims?.height ?? 0, + originalWidth: decodedWidth, + originalHeight: decodedHeight, changed: true, originalByteLength: bytes.length, finalByteLength: encoded.data.length, - }; + }); } catch { // Decode/encode failure — keep the original bytes. - return passthrough(); + return finish('passthrough_error', passthrough()); } } @@ -195,9 +257,13 @@ export interface CompressBase64Result { readonly width: number; /** Pixel height of the (possibly re-encoded) payload; 0 when unknown. */ readonly height: number; - /** Pixel width of the input image; 0 when it cannot be sniffed. */ + /** + * Pixel width of the input image, in display space (EXIF orientation + * applied): the decoded width when re-encoded, the header sniff on + * passthrough (0 when it cannot be determined). + */ readonly originalWidth: number; - /** Pixel height of the input image; 0 when it cannot be sniffed. */ + /** Pixel height of the input image; see {@link originalWidth}. */ readonly originalHeight: number; readonly changed: boolean; readonly originalByteLength: number; @@ -217,10 +283,11 @@ export async function compressBase64ForModel( // Skip very large payloads before allocating: base64 decodes to ~3/4 its // length, so a payload whose decoded size would exceed the cap is passed // through without the Buffer.from allocation (and without touching Jimp). + const startedAt = Date.now(); const maxDecodeBytes = options.maxDecodeBytes ?? MAX_DECODE_BYTES; const approxBytes = Math.floor((base64.length * 3) / 4); if (approxBytes > maxDecodeBytes) { - return { + const result: CompressBase64Result = { base64, mimeType, width: 0, @@ -231,12 +298,20 @@ export async function compressBase64ForModel( originalByteLength: approxBytes, finalByteLength: approxBytes, }; + reportCompressEvent(options.telemetry, { + outcome: 'passthrough_guard', + startedAt, + inputMime: normalizeMime(mimeType), + exifTransposed: false, + result, + }); + return result; } let bytes: Buffer; try { bytes = Buffer.from(base64, 'base64'); } catch { - return { + const result: CompressBase64Result = { base64, mimeType, width: 0, @@ -247,7 +322,16 @@ export async function compressBase64ForModel( originalByteLength: 0, finalByteLength: 0, }; + reportCompressEvent(options.telemetry, { + outcome: 'passthrough_error', + startedAt, + inputMime: normalizeMime(mimeType), + exifTransposed: false, + result, + }); + return result; } + // The event for this call is emitted inside compressImageForModel. const result = await compressImageForModel(bytes, mimeType, options); if (!result.changed) { return { @@ -369,7 +453,10 @@ export interface CompressAnnotateOptions { // ── crop ───────────────────────────────────────────────────────────── -/** Crop rectangle in ORIGINAL-image pixel coordinates. */ +/** + * Crop rectangle in ORIGINAL-image pixel coordinates — the decoded, + * EXIF-rotated space that compression results report as the original size. + */ export interface ImageCropRegion { readonly x: number; readonly y: number; @@ -430,41 +517,50 @@ export async function cropImageForModel( region: ImageCropRegion, options: CropImageOptions = {}, ): Promise { + const startedAt = Date.now(); const maxEdge = options.maxEdge ?? MAX_IMAGE_EDGE_PX; const byteBudget = options.byteBudget ?? IMAGE_BYTE_BUDGET; const maxDecodeBytes = options.maxDecodeBytes ?? MAX_DECODE_BYTES; const normalizedMime = normalizeMime(mimeType); + const fail = (errorKind: CropErrorKind, error: string): CropImageFailure => { + reportCropEvent(options.telemetry, { startedAt, ok: false, errorKind }); + return { ok: false, error }; + }; + const succeed = (result: CropImageSuccess): CropImageSuccess => { + reportCropEvent(options.telemetry, { startedAt, ok: true, result }); + return result; + }; + if (bytes.length === 0) { - return { ok: false, error: 'The image is empty.' }; + return fail('empty', 'The image is empty.'); } if (!RECODABLE_MIME.has(normalizedMime)) { - return { - ok: false, - error: `Cropping is only supported for PNG and JPEG images; got ${mimeType}.`, - }; + return fail( + 'unsupported_format', + `Cropping is only supported for PNG and JPEG images; got ${mimeType}.`, + ); } // NaN slips past every = comparison in the bounds guard below, so gate // on finiteness explicitly rather than surfacing a codec-internal error. if ( ![region.x, region.y, region.width, region.height].every((value) => Number.isFinite(value)) ) { - return { - ok: false, - error: - `Region coordinates must be finite numbers; got x=${String(region.x)}, ` + + return fail( + 'region_invalid', + `Region coordinates must be finite numbers; got x=${String(region.x)}, ` + `y=${String(region.y)}, width=${String(region.width)}, height=${String(region.height)}.`, - }; + ); } const dims = sniffImageDimensions(bytes); if (dims && dims.width * dims.height > MAX_DECODE_PIXELS) { - return { - ok: false, - error: `The image (${String(dims.width)}x${String(dims.height)} pixels) is too large to decode for cropping.`, - }; + return fail( + 'too_large', + `The image (${String(dims.width)}x${String(dims.height)} pixels) is too large to decode for cropping.`, + ); } if (bytes.length > maxDecodeBytes) { - return { ok: false, error: 'The image is too large to decode for cropping.' }; + return fail('too_large', 'The image is too large to decode for cropping.'); } try { @@ -476,12 +572,11 @@ export async function cropImageForModel( const x = Math.floor(region.x); const y = Math.floor(region.y); if (x < 0 || y < 0 || x >= originalWidth || y >= originalHeight || region.width < 1 || region.height < 1) { - return { - ok: false, - error: - `Region (x=${String(region.x)}, y=${String(region.y)}, width=${String(region.width)}, ` + + return fail( + 'out_of_bounds', + `Region (x=${String(region.x)}, y=${String(region.y)}, width=${String(region.width)}, ` + `height=${String(region.height)}) lies outside the ${String(originalWidth)}x${String(originalHeight)} image.`, - }; + ); } const w = Math.min(Math.floor(region.width), originalWidth - x); const h = Math.min(Math.floor(region.height), originalHeight - y); @@ -497,16 +592,15 @@ export async function cropImageForModel( ? await image.getBuffer('image/png', { deflateLevel: 9 }) : await image.getBuffer('image/jpeg', { quality: 90 }); if (buffer.length > byteBudget) { - return { - ok: false, - error: - `The cropped region encodes to ${String(buffer.length)} bytes ` + + return fail( + 'budget', + `The cropped region encodes to ${String(buffer.length)} bytes ` + `(${formatByteSize(buffer.length)}), over the ${String(byteBudget)}-byte ` + `(${formatByteSize(byteBudget)}) per-image limit. ` + 'Choose a smaller region, or allow downscaling.', - }; + ); } - return { + return succeed({ ok: true, data: new Uint8Array(buffer), mimeType: sourceIsPng ? 'image/png' : 'image/jpeg', @@ -518,16 +612,16 @@ export async function cropImageForModel( resized: false, originalByteLength: bytes.length, finalByteLength: buffer.length, - }; + }); } fitWithinEdge(image, maxEdge); const encoded = await encodeWithinBudget(image, { sourceIsPng, byteBudget, - fallbackEdge: FALLBACK_EDGE_PX, + fallbackEdges: FALLBACK_EDGES_PX, }); - return { + return succeed({ ok: true, data: new Uint8Array(encoded.data), mimeType: encoded.mimeType, @@ -539,12 +633,12 @@ export async function cropImageForModel( resized: encoded.width !== w || encoded.height !== h, originalByteLength: bytes.length, finalByteLength: encoded.data.length, - }; + }); } catch (error) { - return { - ok: false, - error: `Failed to decode the image for cropping: ${error instanceof Error ? error.message : String(error)}`, - }; + return fail( + 'decode_failed', + `Failed to decode the image for cropping: ${error instanceof Error ? error.message : String(error)}`, + ); } } @@ -675,7 +769,7 @@ interface EncodedImage { interface EncodeOptions { readonly sourceIsPng: boolean; readonly byteBudget: number; - readonly fallbackEdge: number; + readonly fallbackEdges: readonly number[]; } /** @@ -684,14 +778,17 @@ interface EncodeOptions { * Strategy — prefer the source format so a downscaled screenshot stays lossless * PNG (preserving text and transparency), and only fall back to lossy JPEG when * PNG cannot meet the byte budget: - * - PNG source: PNG at the fitted size → a smaller PNG rescale → JPEG ladder. - * - JPEG source: JPEG quality ladder → a smaller JPEG rescale. + * - PNG source: PNG at the fitted size → smaller PNG rescales, stepping down + * the fallback edges → JPEG ladder. + * - JPEG source: the full quality ladder at the fitted size, then again at + * each fallback edge — a smaller rescale must not skip the high-quality + * rungs its extra pixels just paid for. * * Always returns the smallest buffer it produced, even if no attempt met the * budget — the caller still gates on whether it actually helped. */ async function encodeWithinBudget(image: JimpImage, opts: EncodeOptions): Promise { - const { sourceIsPng, byteBudget, fallbackEdge } = opts; + const { sourceIsPng, byteBudget, fallbackEdges } = opts; let smallest: EncodedImage | null = null; const consider = (data: Buffer, mimeType: string): EncodedImage => { @@ -708,8 +805,9 @@ async function encodeWithinBudget(image: JimpImage, opts: EncodeOptions): Promis if (png.length <= byteBudget) return consider(png, 'image/png'); consider(png, 'image/png'); - // Over budget: a smaller PNG before going lossy. - if (fitWithinEdge(image, fallbackEdge)) { + // Over budget: progressively smaller PNGs before going lossy. + for (const edge of fallbackEdges) { + if (!fitWithinEdge(image, edge)) continue; const smallerPng = await image.getBuffer('image/png', { deflateLevel: 9 }); if (smallerPng.length <= byteBudget) return consider(smallerPng, 'image/png'); consider(smallerPng, 'image/png'); @@ -724,15 +822,20 @@ async function encodeWithinBudget(image: JimpImage, opts: EncodeOptions): Promis return smallest!; } - // JPEG source: quality ladder, then a smaller rescale at the lowest quality. + // JPEG source: quality ladder at the fitted size, then the full ladder + // again at each fallback rescale. for (const quality of JPEG_QUALITY_STEPS) { const jpeg = await image.getBuffer('image/jpeg', { quality }); if (jpeg.length <= byteBudget) return consider(jpeg, 'image/jpeg'); consider(jpeg, 'image/jpeg'); } - if (fitWithinEdge(image, fallbackEdge)) { - const jpeg = await image.getBuffer('image/jpeg', { quality: JPEG_QUALITY_STEPS.at(-1) }); - consider(jpeg, 'image/jpeg'); + for (const edge of fallbackEdges) { + if (!fitWithinEdge(image, edge)) continue; + for (const quality of JPEG_QUALITY_STEPS) { + const jpeg = await image.getBuffer('image/jpeg', { quality }); + if (jpeg.length <= byteBudget) return consider(jpeg, 'image/jpeg'); + consider(jpeg, 'image/jpeg'); + } } return smallest!; @@ -741,6 +844,13 @@ async function encodeWithinBudget(image: JimpImage, opts: EncodeOptions): Promis /** * Scale `image` so its longest edge is at most `edge`, preserving aspect * ratio. No-op (returns false) when the image already fits. + * + * Deliberately passes no `mode`: without one, jimp's default resizer + * downscales with a full-coverage area average (every source pixel + * contributes to the output), which does not alias. The named + * ResizeStrategy modes (BILINEAR, BICUBIC, …) switch to point-sampled + * interpolation that skips source pixels beyond ~2x reduction and produces + * moiré on text and fine patterns — do not "upgrade" this call to one. */ function fitWithinEdge(image: JimpImage, edge: number): boolean { const longest = Math.max(image.width, image.height); @@ -757,3 +867,99 @@ function normalizeMime(mimeType: string): string { const lower = mimeType.trim().toLowerCase(); return lower === 'image/jpg' ? 'image/jpeg' : lower; } + +// ── telemetry ──────────────────────────────────────────────────────── + +/** Failure classification carried by the `image_crop` event. */ +type CropErrorKind = + | 'empty' + | 'unsupported_format' + | 'region_invalid' + | 'too_large' + | 'out_of_bounds' + | 'budget' + | 'decode_failed'; + +/** The subset of a compression result the `image_compress` event reads. */ +interface CompressEventResult { + readonly mimeType: string; + readonly width: number; + readonly height: number; + readonly originalWidth: number; + readonly originalHeight: number; + readonly originalByteLength: number; + readonly finalByteLength: number; +} + +/** + * Emit the `image_compress` event. Properties are all numeric/enum — never + * paths or content — and a throwing client is swallowed so telemetry can + * never affect the compression result. + */ +function reportCompressEvent( + telemetry: ImageCompressionTelemetry | undefined, + input: { + readonly outcome: CompressOutcome; + readonly startedAt: number; + readonly inputMime: string; + readonly exifTransposed: boolean; + readonly result: CompressEventResult; + }, +): void { + if (telemetry === undefined) return; + try { + telemetry.client.track('image_compress', { + source: telemetry.source, + outcome: input.outcome, + input_mime: input.inputMime, + output_mime: normalizeMime(input.result.mimeType), + original_bytes: input.result.originalByteLength, + final_bytes: input.result.finalByteLength, + original_width: input.result.originalWidth, + original_height: input.result.originalHeight, + final_width: input.result.width, + final_height: input.result.height, + exif_transposed: input.exifTransposed, + duration_ms: Date.now() - input.startedAt, + }); + } catch { + // Telemetry must never affect the compression result. + } +} + +/** + * Emit the `image_crop` event. Reports the region as a share of the original + * pixel area rather than raw coordinates. + */ +function reportCropEvent( + telemetry: ImageCompressionTelemetry | undefined, + input: { + readonly startedAt: number; + readonly ok: boolean; + readonly errorKind?: CropErrorKind; + readonly result?: CropImageSuccess; + }, +): void { + if (telemetry === undefined) return; + try { + const { result } = input; + const originalPixels = + result === undefined ? 0 : result.originalWidth * result.originalHeight; + telemetry.client.track('image_crop', { + source: telemetry.source, + ok: input.ok, + error_kind: input.errorKind, + resized: result?.resized, + original_width: result?.originalWidth, + original_height: result?.originalHeight, + region_area_ratio: + result === undefined || originalPixels === 0 + ? undefined + : (result.region.width * result.region.height) / originalPixels, + final_bytes: result?.finalByteLength, + duration_ms: Date.now() - input.startedAt, + }); + } catch { + // Telemetry must never affect the crop outcome. + } +} diff --git a/packages/agent-core/test/mcp/output.test.ts b/packages/agent-core/test/mcp/output.test.ts index 11edc738c..24d2f9fd3 100644 --- a/packages/agent-core/test/mcp/output.test.ts +++ b/packages/agent-core/test/mcp/output.test.ts @@ -8,6 +8,7 @@ import { describe, expect, test } from 'vitest'; import { convertMCPContentBlock, mcpResultToExecutableOutput } from '../../src/mcp/output'; import type { MCPContentBlock, MCPToolResult } from '../../src/mcp/types'; +import type { TelemetryClient } from '../../src/telemetry'; import { sniffImageDimensions } from '../../src/tools/support/file-type'; const MCP_OUTPUT_TRUNCATED_TEXT = @@ -210,6 +211,27 @@ describe('mcpResultToExecutableOutput', () => { return { content, isError }; } + test('emits image_compress telemetry tagged mcp_tool_result', async () => { + const events: { event: string; props: Record }[] = []; + const telemetry: TelemetryClient = { + track: (event, props) => events.push({ event, props: props ?? {} }), + }; + const big = Buffer.from( + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), + ).toString('base64'); + + await mcpResultToExecutableOutput( + result([{ type: 'image', data: big, mimeType: 'image/png' }]), + 'mcp__s__shot', + { telemetry }, + ); + + expect(events).toHaveLength(1); + expect(events[0]!.event).toBe('image_compress'); + expect(events[0]!.props['source']).toBe('mcp_tool_result'); + expect(events[0]!.props['outcome']).toBe('compressed'); + }); + test('collapses a single text part into a plain string', async () => { const out = await mcpResultToExecutableOutput( result([{ type: 'text', text: 'hello' }]), @@ -337,7 +359,7 @@ describe('mcpResultToExecutableOutput', () => { test('downsamples an oversized real image instead of leaving it full-size', async () => { const big = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ).toString('base64'); const out = await mcpResultToExecutableOutput( @@ -353,7 +375,7 @@ describe('mcpResultToExecutableOutput', () => { ); expect(match).not.toBeNull(); const dims = sniffImageDimensions(Buffer.from(match![2]!, 'base64')); - expect(Math.max(dims!.width, dims!.height)).toBeLessThanOrEqual(2000); + expect(Math.max(dims!.width, dims!.height)).toBeLessThanOrEqual(3000); // The image was compressed and kept, not dropped to a notice. const joined = parts.map((p) => (p.type === 'text' ? p.text : '')).join(''); expect(joined).not.toContain('image_url dropped'); @@ -361,7 +383,7 @@ describe('mcpResultToExecutableOutput', () => { test('annotates a downsampled image with a caption note and a readable original', async () => { const bigBytes = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ); const out = await mcpResultToExecutableOutput( @@ -372,7 +394,7 @@ describe('mcpResultToExecutableOutput', () => { // The caption rides the `note` side channel (model-only), keeping its // `` wrapping; the output itself carries only the media. expect(out.note).toContain('Image compressed'); - expect(out.note).toContain('2600x2600'); + expect(out.note).toContain('3600x1800'); const parts = out.output as ContentPart[]; expect(parts.some((p) => p.type === 'text' && p.text.includes('Image compressed'))).toBe( false, @@ -407,7 +429,7 @@ describe('mcpResultToExecutableOutput', () => { test('persists originals into the provided session originals dir', async () => { const dir = await mkdtemp(join(tmpdir(), 'mcp-originals-')); const bigBytes = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ); const out = await mcpResultToExecutableOutput( @@ -432,7 +454,7 @@ describe('mcpResultToExecutableOutput', () => { // to prevent — and orphaning the persisted original. const dir = await mkdtemp(join(tmpdir(), 'mcp-originals-')); const big = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ).toString('base64'); const out = await mcpResultToExecutableOutput( @@ -491,7 +513,7 @@ describe('mcpResultToExecutableOutput', () => { 'sent 50x50 image/jpeg (100 KB). Fine detail may be lost. ' + 'The uncompressed original was not preserved.'; const big = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ).toString('base64'); const out = await mcpResultToExecutableOutput( @@ -502,9 +524,9 @@ describe('mcpResultToExecutableOutput', () => { 'mcp__s__t', ); - // The real caption (2600x2600) rides the note; the quoted one (100x100) + // The real caption (3600x1800) rides the note; the quoted one (100x100) // is tool output and stays where the tool put it. - expect(out.note).toContain('2600x2600'); + expect(out.note).toContain('3600x1800'); expect(out.note).not.toContain('100x100'); const parts = out.output as ContentPart[]; const joined = parts.map((p) => (p.type === 'text' ? p.text : '')).join(''); @@ -517,7 +539,7 @@ describe('mcpResultToExecutableOutput', () => { // into an unclosed fragment. const dir = await mkdtemp(join(tmpdir(), 'mcp-originals-')); const big = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ).toString('base64'); const out = await mcpResultToExecutableOutput( diff --git a/packages/agent-core/test/tools/file-type.test.ts b/packages/agent-core/test/tools/file-type.test.ts index c0587411a..eff175cc2 100644 --- a/packages/agent-core/test/tools/file-type.test.ts +++ b/packages/agent-core/test/tools/file-type.test.ts @@ -382,6 +382,48 @@ function buildJpeg(width: number, height: number): Buffer { return Buffer.concat([soi, app0, sof0]); } +/** + * A minimal EXIF APP1 segment: 'Exif\0\0' + TIFF header + IFD0 holding a + * single Orientation (0x0112) SHORT entry, in the requested byte order. + */ +function exifApp1(orientation: number, byteOrder: 'II' | 'MM'): Buffer { + const le = byteOrder === 'II'; + const tiff = Buffer.alloc(26); + tiff.write(byteOrder, 0, 'latin1'); + const u16 = (value: number, offset: number): void => { + if (le) tiff.writeUInt16LE(value, offset); + else tiff.writeUInt16BE(value, offset); + }; + const u32 = (value: number, offset: number): void => { + if (le) tiff.writeUInt32LE(value, offset); + else tiff.writeUInt32BE(value, offset); + }; + u16(42, 2); + u32(8, 4); // offset of IFD0 + u16(1, 8); // one directory entry + u16(0x0112, 10); // tag: Orientation + u16(3, 12); // type: SHORT + u32(1, 14); // count + u16(orientation, 18); // value, left-aligned in the 4-byte field + u32(0, 22); // no next IFD + const body = Buffer.concat([Buffer.from('Exif\0\0', 'latin1'), tiff]); + const header = Buffer.alloc(4); + header.writeUInt16BE(0xff_e1, 0); + header.writeUInt16BE(body.length + 2, 2); + return Buffer.concat([header, body]); +} + +/** A JPEG whose EXIF APP1 sits between SOI and the remaining segments. */ +function buildJpegWithOrientation( + width: number, + height: number, + orientation: number, + byteOrder: 'II' | 'MM' = 'II', +): Buffer { + const jpeg = buildJpeg(width, height); + return Buffer.concat([jpeg.subarray(0, 2), exifApp1(orientation, byteOrder), jpeg.subarray(2)]); +} + describe('sniffImageDimensions', () => { const cases: ReadonlyArray<{ name: string; @@ -443,6 +485,50 @@ describe('sniffImageDimensions', () => { expect(sniffImageDimensions(data)).toEqual({ width: 100, height: 700 }); }); + describe('JPEG EXIF orientation (dimensions are display-space)', () => { + it.each([5, 6, 7, 8])('swaps width/height for transposing orientation %i', (orientation) => { + // Orientations 5-8 rotate/transpose at decode time: a 120x80 sensor + // frame displays as 80x120. The sniff must report the display space — + // the space decoded images, crop regions, and captions live in. + const data = buildJpegWithOrientation(120, 80, orientation); + expect(sniffImageDimensions(data)).toEqual({ width: 80, height: 120, transposed: true }); + }); + + it.each([1, 2, 3, 4])('keeps width/height for non-transposing orientation %i', (orientation) => { + const data = buildJpegWithOrientation(120, 80, orientation); + expect(sniffImageDimensions(data)).toEqual({ width: 120, height: 80 }); + }); + + it('honors big-endian (MM) TIFF byte order', () => { + const data = buildJpegWithOrientation(120, 80, 6, 'MM'); + expect(sniffImageDimensions(data)).toEqual({ width: 80, height: 120, transposed: true }); + }); + + it('ignores out-of-range orientation values', () => { + expect(sniffImageDimensions(buildJpegWithOrientation(120, 80, 0))).toEqual({ + width: 120, + height: 80, + }); + expect(sniffImageDimensions(buildJpegWithOrientation(120, 80, 9))).toEqual({ + width: 120, + height: 80, + }); + }); + + it('survives a truncated APP1 payload without throwing', () => { + // Declared APP1 length points past the actual TIFF bytes. Whatever + // the sniff returns (unswapped dims or null), it must not throw. + const jpeg = buildJpeg(120, 80); + const app1 = exifApp1(6, 'II'); + const truncated = Buffer.concat([ + jpeg.subarray(0, 2), + app1.subarray(0, 10), + jpeg.subarray(2), + ]); + expect(() => sniffImageDimensions(truncated)).not.toThrow(); + }); + }); + describe('truncated / malformed input returns null without throwing', () => { const malformed: ReadonlyArray<{ name: string; data: Buffer }> = [ { diff --git a/packages/agent-core/test/tools/image-compress.test.ts b/packages/agent-core/test/tools/image-compress.test.ts index 3eb3f32f7..a063327eb 100644 --- a/packages/agent-core/test/tools/image-compress.test.ts +++ b/packages/agent-core/test/tools/image-compress.test.ts @@ -24,9 +24,15 @@ * `` note (dims, sizes, readback path) * - annotate: compressImageContentParts can collect that caption for * each compressed image and persist the original via a callback + * - quality guards: a 1px checkerboard downscales to flat gray (no + * spectral aliasing) at integer and fractional ratios, with jimp's + * point-sampled BILINEAR mode pinned as the aliasing counter-example; + * fully transparent pixels never bleed color into opaque edges; mean + * brightness survives the downscale; recompressing a compressed + * result is a no-op; extreme aspect ratios never collapse to zero */ -import { Jimp } from 'jimp'; +import { Jimp, ResizeStrategy } from 'jimp'; import { describe, expect, it } from 'vitest'; // eslint-disable-next-line import/no-unresolved @@ -42,6 +48,8 @@ import { } from '../../src/tools/support/image-compress'; // eslint-disable-next-line import/no-unresolved import { sniffImageDimensions } from '../../src/tools/support/file-type'; +// eslint-disable-next-line import/no-unresolved +import type { TelemetryClient, TelemetryProperties } from '../../src/telemetry'; // ── fixtures ───────────────────────────────────────────────────────── @@ -76,11 +84,77 @@ async function noisePng(width: number, height: number, alpha = false): Promise { + const image = new Jimp({ width, height, color: 0x000000ff }); + fillXorshiftNoise(image.bitmap.data); + return new Uint8Array(await image.getBuffer('image/png')); +} + +/** JPEG twin of {@link randomNoisePng}, for exercising the JPEG source path. */ +async function randomNoiseJpeg(width: number, height: number): Promise { + const image = new Jimp({ width, height, color: 0x000000ff }); + fillXorshiftNoise(image.bitmap.data); + return new Uint8Array(await image.getBuffer('image/jpeg', { quality: 90 })); +} + +function fillXorshiftNoise(data: Buffer | Uint8Array): void { + let state = 0x9e3779b9; + const next = (): number => { + state ^= (state << 13) >>> 0; + state ^= state >>> 17; + state ^= (state << 5) >>> 0; + state >>>= 0; + return state & 0xff; + }; + for (let i = 0; i < data.length; i += 4) { + data[i] = next(); + data[i + 1] = next(); + data[i + 2] = next(); + data[i + 3] = 0xff; + } +} + async function decodeAlpha(bytes: Uint8Array): Promise { const image = await Jimp.fromBuffer(Buffer.from(bytes)); return image.hasAlpha(); } +/** + * Insert a minimal EXIF APP1 segment carrying only an Orientation tag right + * after the JPEG SOI marker (jimp itself never writes EXIF). + */ +function withExifOrientation(jpeg: Uint8Array, orientation: number): Uint8Array { + // TIFF body, little-endian: 8-byte header + IFD0 with a single entry. + const tiff = Buffer.alloc(26); + tiff.write('II', 0, 'latin1'); + tiff.writeUInt16LE(42, 2); + tiff.writeUInt32LE(8, 4); // offset of IFD0 + tiff.writeUInt16LE(1, 8); // one directory entry + tiff.writeUInt16LE(0x0112, 10); // tag: Orientation + tiff.writeUInt16LE(3, 12); // type: SHORT + tiff.writeUInt32LE(1, 14); // count + tiff.writeUInt16LE(orientation, 18); // value, left-aligned in the 4-byte field + tiff.writeUInt32LE(0, 22); // no next IFD + const exifBody = Buffer.concat([Buffer.from('Exif\0\0', 'latin1'), tiff]); + const app1Header = Buffer.alloc(4); + app1Header.writeUInt16BE(0xff_e1, 0); + app1Header.writeUInt16BE(exifBody.length + 2, 2); + return new Uint8Array( + Buffer.concat([ + Buffer.from(jpeg.subarray(0, 2)), // SOI + app1Header, + exifBody, + Buffer.from(jpeg.subarray(2)), + ]), + ); +} + // ── fast path ──────────────────────────────────────────────────────── describe('compressImageForModel — fast path', () => { @@ -106,15 +180,15 @@ describe('compressImageForModel — fast path', () => { describe('compressImageForModel — dimension cap', () => { it('scales the longest edge down to MAX_IMAGE_EDGE_PX, preserving aspect', async () => { - const png = await solidPng(3000, 1500); + const png = await solidPng(4500, 2250); const result = await compressImageForModel(png, 'image/png'); expect(result.changed).toBe(true); expect(Math.max(result.width, result.height)).toBe(MAX_IMAGE_EDGE_PX); - // 3000x1500 → 2000x1000 (aspect 2:1 preserved). - expect(result.width).toBe(2000); - expect(result.height).toBe(1000); + // 4500x2250 → 3000x1500 (aspect 2:1 preserved). + expect(result.width).toBe(3000); + expect(result.height).toBe(1500); const dims = sniffImageDimensions(result.data); - expect(dims).toEqual({ width: 2000, height: 1000 }); + expect(dims).toEqual({ width: 3000, height: 1500 }); }); it('respects a custom maxEdge', async () => { @@ -128,7 +202,7 @@ describe('compressImageForModel — dimension cap', () => { it('keeps a downscaled opaque PNG lossless (no needless JPEG conversion)', async () => { // A screenshot-like opaque PNG that only needs downscaling must stay PNG so // sharp text is not degraded by JPEG artifacts. - const png = await solidPng(3000, 1500); + const png = await solidPng(4500, 2250); const result = await compressImageForModel(png, 'image/png'); expect(result.changed).toBe(true); expect(result.mimeType).toBe('image/png'); @@ -148,7 +222,7 @@ describe('compressImageForModel — byte budget', () => { }); it('keeps a translucent PNG as PNG when the budget allows', async () => { - const png = await translucentPng(2600, 2600); + const png = await translucentPng(3600, 1800); const result = await compressImageForModel(png, 'image/png'); expect(result.changed).toBe(true); expect(result.mimeType).toBe('image/png'); @@ -163,6 +237,66 @@ describe('compressImageForModel — byte budget', () => { expect(result.mimeType).toBe('image/jpeg'); expect(result.finalByteLength).toBeLessThan(result.originalByteLength); }); + + it('steps down through the 2000px edge before the 1000px fallback', async () => { + // Regression guard for the 3000px cap raise: a PNG whose fitted encode + // is over budget but whose 2000px encode fits must come back at 2000px + // (as it did under the old cap), not skip straight to 1000px. + // The budget is anchored to the actual 2000px encode size (probed with + // an unlimited budget) so the test does not depend on exact deflate + // output sizes. + const png = await randomNoisePng(2400, 600); + const probe = await compressImageForModel(png, 'image/png', { + maxEdge: 2000, + byteBudget: Number.MAX_SAFE_INTEGER, + }); + expect(probe.changed).toBe(true); + expect(probe.mimeType).toBe('image/png'); + expect(Math.max(probe.width, probe.height)).toBe(2000); + // Sanity: the anchor budget must sit below the input size, or the run + // below would pass through on the fast path instead of re-encoding. + expect(probe.finalByteLength + 1024).toBeLessThan(png.length); + + const result = await compressImageForModel(png, 'image/png', { + byteBudget: probe.finalByteLength + 1024, + }); + expect(result.changed).toBe(true); + expect(result.mimeType).toBe('image/png'); + expect(Math.max(result.width, result.height)).toBe(2000); + }); + + it( + 're-runs the JPEG quality ladder at fallback sizes instead of jumping to q20', + async () => { + // A JPEG whose quality ladder fails at every size above 1000px, with the + // budget tuned so that at 1000px a mid-quality (q60) encode fits. The + // fallback must walk the ladder again and return that q60 encode — not + // collapse straight to q20 and needlessly destroy detail. + // The probe replays the implementation's exact resize chain + // (2400 → 2000 → 1000): box-resizing twice does not yield the same + // bitmap as resizing once, and JPEG encoding is deterministic, so the + // probed q60 size matches the implementation's encode byte-for-byte. + // The width must exceed 2000px to exercise the full fallback chain; the + // height is kept small so the ~11 JPEG encodes stay fast on slow CI. + const jpeg = await randomNoiseJpeg(2400, 300); + const probe = await Jimp.fromBuffer(Buffer.from(jpeg)); + probe.resize({ w: 2000, h: 250 }); + probe.resize({ w: 1000, h: 125 }); + const q60Size = (await probe.getBuffer('image/jpeg', { quality: 60 })).length; + const q20Size = (await probe.getBuffer('image/jpeg', { quality: 20 })).length; + expect(q60Size).toBeGreaterThan(q20Size); // sanity: the anchor separates the rungs + + const result = await compressImageForModel(jpeg, 'image/jpeg', { + byteBudget: q60Size + 256, + }); + expect(result.changed).toBe(true); + expect(result.mimeType).toBe('image/jpeg'); + expect(Math.max(result.width, result.height)).toBe(1000); + // The highest quality that fits the budget at 1000px is q60. + expect(result.finalByteLength).toBe(q60Size); + }, + 15_000, + ); }); // ── fallback / robustness ──────────────────────────────────────────── @@ -219,7 +353,7 @@ describe('compressImageForModel — fallback', () => { it('skips compression for payloads over the byte cap without decoding', async () => { // Over the edge (so not the fast path), but capped by maxDecodeBytes. - const png = await solidPng(3000, 100); + const png = await solidPng(3200, 100); const result = await compressImageForModel(png, 'image/png', { maxDecodeBytes: 64 }); expect(result.changed).toBe(false); expect(result.data).toBe(png); // passthrough → Jimp was never called @@ -231,9 +365,9 @@ describe('compressImageForModel — fallback', () => { describe('compressImageForModel — invariants', () => { it('changed always yields a within-cap, decodable payload', async () => { const cases: Uint8Array[] = [ - await solidPng(3000, 1500), + await solidPng(4500, 2250), await noisePng(900, 900), - await translucentPng(2600, 2600), + await translucentPng(3600, 1800), ]; for (const bytes of cases) { const result = await compressImageForModel(bytes, 'image/png'); @@ -276,7 +410,7 @@ describe('compressBase64ForModel', () => { }); it('skips a base64 payload over the byte cap without decoding', async () => { - const png = await solidPng(3000, 100); // over edge, would otherwise compress + const png = await solidPng(3200, 100); // over edge, would otherwise compress const base64 = Buffer.from(png).toString('base64'); const result = await compressBase64ForModel(base64, 'image/png', { maxDecodeBytes: 64 }); expect(result.changed).toBe(false); @@ -300,7 +434,7 @@ describe('compressImageForModel — performance', () => { }); it('compresses a large image within a generous time bound', async () => { - const png = await solidPng(3000, 2000); + const png = await solidPng(4500, 3000); const start = performance.now(); const result = await compressImageForModel(png, 'image/png'); const elapsed = performance.now() - start; @@ -310,7 +444,7 @@ describe('compressImageForModel — performance', () => { it('exposes a sane default budget', () => { expect(IMAGE_BYTE_BUDGET).toBeGreaterThan(0); - expect(MAX_IMAGE_EDGE_PX).toBe(2000); + expect(MAX_IMAGE_EDGE_PX).toBe(3000); }); }); @@ -322,7 +456,7 @@ describe('compressImageContentParts', () => { } it('compresses an oversized inline image part, leaving other parts untouched', async () => { - const big = await solidPng(2600, 2600); + const big = await solidPng(3600, 1800); const parts = [ { type: 'text' as const, text: 'look at this' }, { type: 'image_url' as const, imageUrl: { url: dataUrl('image/png', big) } }, @@ -355,7 +489,7 @@ describe('compressImageContentParts', () => { }); it('keeps an image part id when rewriting the compressed url', async () => { - const big = await solidPng(2600, 2600); + const big = await solidPng(3600, 1800); const parts = [ { type: 'image_url' as const, imageUrl: { url: dataUrl('image/png', big), id: 'att-1' } }, ]; @@ -369,6 +503,36 @@ describe('compressImageContentParts', () => { // ── original-dimension metadata ────────────────────────────────────── +describe('compressImageForModel — EXIF orientation', () => { + it('reports original dimensions in the decoded (EXIF-rotated) space', async () => { + // Orientation 6 (rotate 90° CW): the file header says 120x80, but jimp + // decodes to 80x120 — the space the sent image and any later crop region + // actually live in. The reported original dimensions must match it, not + // the pre-rotation header sniff. + const jpeg = withExifOrientation(await solidJpeg(120, 80), 6); + const result = await compressImageForModel(jpeg, 'image/jpeg', { maxEdge: 64 }); + expect(result.changed).toBe(true); + expect(result.originalWidth).toBe(80); + expect(result.originalHeight).toBe(120); + // The sent image keeps the rotated (portrait) aspect. + expect(result.width).toBeLessThan(result.height); + }); + + it('reports display-space dimensions for an EXIF-rotated passthrough', async () => { + // Within both budgets → no decode ever happens. The header sniff itself + // must account for EXIF orientation so passthrough metadata agrees with + // the space a later region readback (which decodes) will use. + const jpeg = withExifOrientation(await solidJpeg(120, 80), 6); + const result = await compressImageForModel(jpeg, 'image/jpeg'); + expect(result.changed).toBe(false); + expect(result.data).toBe(jpeg); // fast path — not decoded + expect(result.originalWidth).toBe(80); + expect(result.originalHeight).toBe(120); + expect(result.width).toBe(80); + expect(result.height).toBe(120); + }); +}); + describe('compressImageForModel — original dimensions metadata', () => { it('reports original dimensions on passthrough and compressed results', async () => { const small = await solidPng(64, 64); @@ -377,23 +541,23 @@ describe('compressImageForModel — original dimensions metadata', () => { expect(pass.originalWidth).toBe(64); expect(pass.originalHeight).toBe(64); - const big = await solidPng(3000, 1500); + const big = await solidPng(4500, 2250); const shrunk = await compressImageForModel(big, 'image/png'); expect(shrunk.changed).toBe(true); - expect(shrunk.originalWidth).toBe(3000); - expect(shrunk.originalHeight).toBe(1500); - expect(shrunk.width).toBe(2000); + expect(shrunk.originalWidth).toBe(4500); + expect(shrunk.originalHeight).toBe(2250); + expect(shrunk.width).toBe(3000); }); it('reports original dimensions through the base64 wrapper', async () => { - const big = await solidPng(2600, 1300); + const big = await solidPng(3900, 1950); const base64 = Buffer.from(big).toString('base64'); const result = await compressBase64ForModel(base64, 'image/png'); expect(result.changed).toBe(true); - expect(result.originalWidth).toBe(2600); - expect(result.originalHeight).toBe(1300); - expect(result.width).toBe(2000); - expect(result.height).toBe(1000); + expect(result.originalWidth).toBe(3900); + expect(result.originalHeight).toBe(1950); + expect(result.width).toBe(3000); + expect(result.height).toBe(1500); }); }); @@ -464,18 +628,18 @@ describe('cropImageForModel', () => { }); it('downscales an oversized crop to the edge cap by default', async () => { - const png = await solidPng(3000, 1500); + const png = await solidPng(4500, 2250); const result = await cropImageForModel(png, 'image/png', { x: 0, y: 0, - width: 2500, + width: 4000, height: 1200, }); expect(result.ok).toBe(true); if (!result.ok) return; expect(result.resized).toBe(true); expect(Math.max(result.width, result.height)).toBeLessThanOrEqual(MAX_IMAGE_EDGE_PX); - expect(result.region).toEqual({ x: 0, y: 0, width: 2500, height: 1200 }); + expect(result.region).toEqual({ x: 0, y: 0, width: 4000, height: 1200 }); }); it('keeps native resolution with skipResize', async () => { @@ -649,7 +813,7 @@ describe('compressImageContentParts — annotate', () => { } it('collects a caption for a compressed image and persists the original', async () => { - const big = await solidPng(2600, 2600); + const big = await solidPng(3600, 1800); const persisted: { bytes: Uint8Array; mimeType: string }[] = []; const parts = [{ type: 'image_url' as const, imageUrl: { url: dataUrl('image/png', big) } }]; const out = await compressImageContentParts(parts, { @@ -665,7 +829,7 @@ describe('compressImageContentParts — annotate', () => { expect(out.parts).toHaveLength(1); expect(out.parts[0]?.type).toBe('image_url'); expect(out.captions).toHaveLength(1); - expect(out.captions[0]).toContain('2600x2600'); + expect(out.captions[0]).toContain('3600x1800'); expect(out.captions[0]).toContain('/tmp/originals/big.png'); expect(persisted).toHaveLength(1); expect(persisted[0]?.mimeType).toBe('image/png'); @@ -684,7 +848,7 @@ describe('compressImageContentParts — annotate', () => { }); it('captions without a path when persistence fails', async () => { - const big = await solidPng(2600, 2600); + const big = await solidPng(3600, 1800); const parts = [{ type: 'image_url' as const, imageUrl: { url: dataUrl('image/png', big) } }]; const out = await compressImageContentParts(parts, { annotate: { persistOriginal: () => Promise.resolve(null) }, @@ -694,3 +858,390 @@ describe('compressImageContentParts — annotate', () => { expect(out.captions[0]).toMatch(/not preserved/i); }); }); + +// ── downscale quality guards ───────────────────────────────────────── +// +// Downscaling is a resampling operation: input frequencies above the output +// Nyquist limit must be filtered out (averaged), or they fold back as +// low-frequency moiré — spectral aliasing. A 1px checkerboard is the +// worst-case probe: ALL of its energy sits at the input Nyquist frequency, +// so a resampler that skips source pixels turns it into high-contrast +// artifacts, while a correct full-coverage average yields flat ~50% gray. +// These tests pin the compressor to the correct behavior, keep the aliasing +// counter-example executable, and cover the other classic downscale bugs +// (transparent-pixel bleed, brightness drift, iterative degradation, +// degenerate aspect ratios). + +/** 1px checkerboard: every pixel alternates black/white in both axes. */ +async function checkerboardPng(size: number): Promise { + const image = new Jimp({ width: size, height: size, color: 0x000000ff }); + const data = image.bitmap.data; + for (let y = 0; y < size; y += 1) { + for (let x = 0; x < size; x += 1) { + const v = (x + y) % 2 === 0 ? 0 : 255; + const i = (y * size + x) * 4; + data[i] = v; + data[i + 1] = v; + data[i + 2] = v; + data[i + 3] = 0xff; + } + } + return new Uint8Array(await image.getBuffer('image/png')); +} + +interface GrayStats { + readonly min: number; + readonly max: number; + readonly mean: number; +} + +/** Min/max/mean over the red channel (all probes here are grayscale). */ +function grayStats(image: { bitmap: { data: Buffer | Uint8Array } }): GrayStats { + const data = image.bitmap.data; + let min = 255; + let max = 0; + let sum = 0; + for (let i = 0; i < data.length; i += 4) { + const v = data[i]!; + if (v < min) min = v; + if (v > max) max = v; + sum += v; + } + return { min, max, mean: sum / (data.length / 4) }; +} + +describe('compressImageForModel — downscale quality guards', () => { + it('averages a 1px checkerboard to flat gray at an integer ratio (no aliasing)', async () => { + // 2000 → 500 (4:1). Every output pixel covers a 4×4 block holding 8 + // black and 8 white pixels, so a full-coverage average lands on ~127. + // Aliasing would instead show up as black/white patches or moiré bands. + const png = await checkerboardPng(2000); + const result = await compressImageForModel(png, 'image/png', { maxEdge: 500 }); + expect(result.changed).toBe(true); + expect(Math.max(result.width, result.height)).toBe(500); + + const decoded = await Jimp.fromBuffer(Buffer.from(result.data)); + const { min, max } = grayStats(decoded); + expect(min).toBeGreaterThanOrEqual(118); + expect(max).toBeLessThanOrEqual(138); + }); + + it('stays alias-free at a non-integer ratio (fractional pixel coverage)', async () => { + // 2000 → 780 (≈2.56:1). Non-integer ratios are where phase-dependent + // point sampling degrades worst. Fractional window coverage leaves the + // average some mild texture, but nothing may approach black or white. + const png = await checkerboardPng(2000); + const result = await compressImageForModel(png, 'image/png', { maxEdge: 780 }); + expect(result.changed).toBe(true); + + const decoded = await Jimp.fromBuffer(Buffer.from(result.data)); + const { min, max } = grayStats(decoded); + expect(min).toBeGreaterThanOrEqual(90); + expect(max).toBeLessThanOrEqual(165); + }); + + it('control: jimp point-sampled BILINEAR aliases the same input (keeps the probe honest)', async () => { + // Executable counter-example for the constraint documented on + // fitWithinEdge: the named ResizeStrategy modes sample a fixed 2×2 + // neighborhood around the mapped point and skip the rest. At 4:1 the + // sample grid lands on a single checkerboard phase and the 50%-gray + // pattern collapses to solid black — the pattern's energy is entirely + // misrepresented. This proves the two tests above can fail (the probe + // distinguishes resamplers) and pins the library behavior the + // mode-less default call relies on — if jimp ever changes either + // side, revisit the fitWithinEdge comment. + const image = await Jimp.fromBuffer(Buffer.from(await checkerboardPng(2000))); + image.resize({ w: 500, h: 500, mode: ResizeStrategy.BILINEAR }); + const { min, max, mean } = grayStats(image); + // The correct answer is flat ~127 gray (mean ≈ 127, max-min ≈ 0). + // Aliasing shows up as a solid black/white collapse or full-contrast + // banding — far from that answer regardless of sampling phase. + const aliased = mean < 60 || mean > 195 || max - min > 200; + expect(aliased).toBe(true); + }); + + it('never bleeds color from fully transparent pixels into visible ones', async () => { + // Fully transparent pixels still carry RGB values. A resizer that + // blends them into the average tints every transparency edge (halo). + // Probe: a fully transparent BRIGHT RED field around an opaque blue + // square — after a 4:1 downscale no visible pixel may pick up red. + const size = 1600; + const image = new Jimp({ width: size, height: size, color: 0xff000000 }); // red, alpha 0 + const data = image.bitmap.data; + for (let y = 400; y < 1200; y += 1) { + for (let x = 400; x < 1200; x += 1) { + const i = (y * size + x) * 4; + data[i] = 0; + data[i + 1] = 0; + data[i + 2] = 0xff; + data[i + 3] = 0xff; + } + } + const png = new Uint8Array(await image.getBuffer('image/png')); + + const result = await compressImageForModel(png, 'image/png', { maxEdge: 400 }); + expect(result.changed).toBe(true); + expect(result.mimeType).toBe('image/png'); // alpha survives + + const decoded = await Jimp.fromBuffer(Buffer.from(result.data)); + const out = decoded.bitmap.data; + let visible = 0; + for (let i = 0; i < out.length; i += 4) { + if (out[i + 3]! >= 8) { + visible += 1; + expect(out[i]!).toBeLessThanOrEqual(16); // red channel stays ~0 + } + } + expect(visible).toBeGreaterThan(0); // the blue square is still there + }); + + it('preserves mean brightness through the downscale (no energy drift)', async () => { + // A normalized filter keeps the image mean; drift here would indicate + // non-normalized weights (or a broken gamma pipeline stage). + const png = await noisePng(900, 900); + const input = await Jimp.fromBuffer(Buffer.from(png)); + const inputMean = grayStats(input).mean; + + const result = await compressImageForModel(png, 'image/png', { maxEdge: 225 }); + expect(result.changed).toBe(true); + const output = await Jimp.fromBuffer(Buffer.from(result.data)); + expect(Math.abs(grayStats(output).mean - inputMean)).toBeLessThan(3); + }); + + it('recompressing a compressed result is a no-op (no iterative degradation)', async () => { + // Model-bound bytes can re-enter the pipeline (session replay, MCP + // round-trips). Once within budget they must pass through untouched + // instead of being shaved a little smaller on every pass. + const first = await compressImageForModel(await solidPng(4500, 2250), 'image/png'); + expect(first.changed).toBe(true); + + const second = await compressImageForModel(first.data, first.mimeType); + expect(second.changed).toBe(false); + expect(second.data).toBe(first.data); // identity — not even re-decoded + }); + + it('keeps a degenerate aspect ratio at least 1px tall (no zero-size collapse)', async () => { + // 9000×2 scaled to a 3000px edge would round the short side to 0.67px; + // the resizer must clamp to 1, not produce an undecodable 3000×0 image. + const png = await solidPng(9000, 2); + const result = await compressImageForModel(png, 'image/png'); + expect(result.changed).toBe(true); + expect(result.width).toBe(3000); + expect(result.height).toBe(1); + expect(sniffImageDimensions(result.data)).toEqual({ width: 3000, height: 1 }); + }); +}); + +// ── telemetry ──────────────────────────────────────────────────────── + +interface CapturedEvent { + readonly event: string; + readonly props: TelemetryProperties; +} + +function captureTelemetry(): { client: TelemetryClient; events: CapturedEvent[] } { + const events: CapturedEvent[] = []; + return { + client: { track: (event, props) => events.push({ event, props: props ?? {} }) }, + events, + }; +} + +describe('compressImageForModel — telemetry', () => { + it('reports a compressed image with sizes, formats, and duration', async () => { + const { client, events } = captureTelemetry(); + const png = await solidPng(4500, 2250); + const result = await compressImageForModel(png, 'image/png', { + telemetry: { client, source: 'read_media' }, + }); + expect(result.changed).toBe(true); + + expect(events).toHaveLength(1); + const { event, props } = events[0]!; + expect(event).toBe('image_compress'); + expect(props['source']).toBe('read_media'); + expect(props['outcome']).toBe('compressed'); + expect(props['input_mime']).toBe('image/png'); + expect(props['output_mime']).toBe(result.mimeType); + expect(props['original_bytes']).toBe(png.length); + expect(props['final_bytes']).toBe(result.finalByteLength); + expect(props['original_width']).toBe(4500); + expect(props['original_height']).toBe(2250); + expect(props['final_width']).toBe(3000); + expect(props['final_height']).toBe(1500); + expect(props['exif_transposed']).toBe(false); + expect(typeof props['duration_ms']).toBe('number'); + }); + + it('reports the fast path as passthrough_fast', async () => { + const { client, events } = captureTelemetry(); + await compressImageForModel(await solidPng(64, 64), 'image/png', { + telemetry: { client, source: 'tui_paste' }, + }); + expect(events).toHaveLength(1); + expect(events[0]!.props['outcome']).toBe('passthrough_fast'); + expect(events[0]!.props['source']).toBe('tui_paste'); + }); + + it('reports decode guards as passthrough_guard', async () => { + // Decompression-bomb header: 30000×30000 with no pixel data. + const header = Buffer.alloc(24); + Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]).copy(header, 0); + header.writeUInt32BE(13, 8); + header.write('IHDR', 12, 'latin1'); + header.writeUInt32BE(30000, 16); + header.writeUInt32BE(30000, 20); + + const bomb = captureTelemetry(); + await compressImageForModel(new Uint8Array(header), 'image/png', { + telemetry: { client: bomb.client, source: 'mcp_tool_result' }, + }); + expect(bomb.events[0]!.props['outcome']).toBe('passthrough_guard'); + + const byteCap = captureTelemetry(); + await compressImageForModel(await solidPng(3200, 100), 'image/png', { + maxDecodeBytes: 64, + telemetry: { client: byteCap.client, source: 'mcp_tool_result' }, + }); + expect(byteCap.events[0]!.props['outcome']).toBe('passthrough_guard'); + }); + + it('reports non-recodable formats and empty input as passthrough_unsupported', async () => { + const gif = captureTelemetry(); + await compressImageForModel( + new Uint8Array([0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 1, 0, 1, 0]), + 'image/gif', + { telemetry: { client: gif.client, source: 'mcp_tool_result' } }, + ); + expect(gif.events[0]!.props['outcome']).toBe('passthrough_unsupported'); + + const empty = captureTelemetry(); + await compressImageForModel(new Uint8Array(0), 'image/png', { + telemetry: { client: empty.client, source: 'mcp_tool_result' }, + }); + expect(empty.events[0]!.props['outcome']).toBe('passthrough_unsupported'); + }); + + it('reports undecodable bytes as passthrough_error', async () => { + // A tiny corrupt blob would pass through on the fast path (unknown dims, + // small bytes) without ever decoding; to reach the decoder the header + // must claim an over-cap size. 4000×4000 forces a decode of garbage. + const { client, events } = captureTelemetry(); + const corrupt = Buffer.alloc(32); + Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]).copy(corrupt, 0); + corrupt.writeUInt32BE(13, 8); + corrupt.write('IHDR', 12, 'latin1'); + corrupt.writeUInt32BE(4000, 16); + corrupt.writeUInt32BE(4000, 20); + await compressImageForModel(new Uint8Array(corrupt), 'image/png', { + telemetry: { client, source: 'prompt_inline' }, + }); + expect(events[0]!.props['outcome']).toBe('passthrough_error'); + }); + + it('marks EXIF-transposed inputs', async () => { + const { client, events } = captureTelemetry(); + const jpeg = withExifOrientation(await solidJpeg(120, 80), 6); + await compressImageForModel(jpeg, 'image/jpeg', { + maxEdge: 64, + telemetry: { client, source: 'read_media' }, + }); + expect(events[0]!.props['outcome']).toBe('compressed'); + expect(events[0]!.props['exif_transposed']).toBe(true); + }); + + it('reports the base64 early size-skip as passthrough_guard', async () => { + const { client, events } = captureTelemetry(); + const base64 = Buffer.from(await solidPng(3200, 100)).toString('base64'); + await compressBase64ForModel(base64, 'image/png', { + maxDecodeBytes: 64, + telemetry: { client, source: 'prompt_file' }, + }); + expect(events).toHaveLength(1); + expect(events[0]!.props['outcome']).toBe('passthrough_guard'); + expect(events[0]!.props['source']).toBe('prompt_file'); + }); + + it('threads telemetry through compressImageContentParts', async () => { + const { client, events } = captureTelemetry(); + const big = await solidPng(3600, 1800); + const url = `data:image/png;base64,${Buffer.from(big).toString('base64')}`; + await compressImageContentParts([{ type: 'image_url', imageUrl: { url } }], { + telemetry: { client, source: 'mcp_tool_result' }, + }); + expect(events).toHaveLength(1); + expect(events[0]!.event).toBe('image_compress'); + expect(events[0]!.props['outcome']).toBe('compressed'); + expect(events[0]!.props['source']).toBe('mcp_tool_result'); + }); + + it('never lets a throwing telemetry client break compression', async () => { + const throwing: TelemetryClient = { + track: () => { + throw new Error('sink down'); + }, + }; + const png = await solidPng(4500, 2250); + const result = await compressImageForModel(png, 'image/png', { + telemetry: { client: throwing, source: 'read_media' }, + }); + expect(result.changed).toBe(true); // compression outcome unaffected + }); +}); + +describe('cropImageForModel — telemetry', () => { + it('reports a successful crop with the region share of the original', async () => { + const { client, events } = captureTelemetry(); + const png = await solidPng(1000, 500); + const outcome = await cropImageForModel( + png, + 'image/png', + { x: 0, y: 0, width: 500, height: 250 }, + { telemetry: { client, source: 'read_media' } }, + ); + expect(outcome.ok).toBe(true); + + expect(events).toHaveLength(1); + const { event, props } = events[0]!; + expect(event).toBe('image_crop'); + expect(props['source']).toBe('read_media'); + expect(props['ok']).toBe(true); + expect(props['resized']).toBe(false); + expect(props['original_width']).toBe(1000); + expect(props['original_height']).toBe(500); + // 500×250 of 1000×500 → a quarter of the pixels. + expect(props['region_area_ratio']).toBeCloseTo(0.25, 5); + expect(typeof props['duration_ms']).toBe('number'); + expect(typeof props['final_bytes']).toBe('number'); + }); + + it('classifies failures by kind', async () => { + const oob = captureTelemetry(); + await cropImageForModel( + await solidPng(100, 100), + 'image/png', + { x: 200, y: 0, width: 10, height: 10 }, + { telemetry: { client: oob.client, source: 'read_media' } }, + ); + expect(oob.events[0]!.props['ok']).toBe(false); + expect(oob.events[0]!.props['error_kind']).toBe('out_of_bounds'); + + const format = captureTelemetry(); + await cropImageForModel( + new Uint8Array([0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 1, 0, 1, 0]), + 'image/gif', + { x: 0, y: 0, width: 1, height: 1 }, + { telemetry: { client: format.client, source: 'read_media' } }, + ); + expect(format.events[0]!.props['error_kind']).toBe('unsupported_format'); + + const budget = captureTelemetry(); + await cropImageForModel( + await noisePng(900, 900), + 'image/png', + { x: 0, y: 0, width: 900, height: 900 }, + { skipResize: true, byteBudget: 8 * 1024, telemetry: { client: budget.client, source: 'read_media' } }, + ); + expect(budget.events[0]!.props['error_kind']).toBe('budget'); + }); +}); diff --git a/packages/agent-core/test/tools/read-media.test.ts b/packages/agent-core/test/tools/read-media.test.ts index b04adde77..5fd848549 100644 --- a/packages/agent-core/test/tools/read-media.test.ts +++ b/packages/agent-core/test/tools/read-media.test.ts @@ -14,6 +14,7 @@ import { ReadMediaFileTool, } from '../../src/tools/builtin/file/read-media'; import { MEDIA_SNIFF_BYTES, sniffImageDimensions } from '../../src/tools/support/file-type'; +import type { TelemetryClient } from '../../src/telemetry'; import { createFakeKaos, PERMISSIVE_WORKSPACE } from './fixtures/fake-kaos'; import { executeTool } from './fixtures/execute-tool'; @@ -41,6 +42,35 @@ const MP4_HEADER = Buffer.concat([ Buffer.from('mp42isom'), ]); +/** + * Insert a minimal EXIF APP1 segment carrying only an Orientation tag right + * after the JPEG SOI marker (jimp itself never writes EXIF). Mirrors the + * fixture in image-compress.test.ts. + */ +function withExifOrientation(jpeg: Uint8Array, orientation: number): Buffer { + // TIFF body, little-endian: 8-byte header + IFD0 with a single entry. + const tiff = Buffer.alloc(26); + tiff.write('II', 0, 'latin1'); + tiff.writeUInt16LE(42, 2); + tiff.writeUInt32LE(8, 4); // offset of IFD0 + tiff.writeUInt16LE(1, 8); // one directory entry + tiff.writeUInt16LE(0x0112, 10); // tag: Orientation + tiff.writeUInt16LE(3, 12); // type: SHORT + tiff.writeUInt32LE(1, 14); // count + tiff.writeUInt16LE(orientation, 18); // value, left-aligned in the 4-byte field + tiff.writeUInt32LE(0, 22); // no next IFD + const exifBody = Buffer.concat([Buffer.from('Exif\0\0', 'latin1'), tiff]); + const app1Header = Buffer.alloc(4); + app1Header.writeUInt16BE(0xff_e1, 0); + app1Header.writeUInt16BE(exifBody.length + 2, 2); + return Buffer.concat([ + Buffer.from(jpeg.subarray(0, 2)), // SOI + app1Header, + exifBody, + Buffer.from(jpeg.subarray(2)), + ]); +} + function capabilities(overrides: Partial = {}): ModelCapability { return { image_in: true, @@ -58,6 +88,7 @@ function makeReadMediaTool( readonly stat?: Kaos['stat'] | undefined; readonly readBytes?: Kaos['readBytes'] | undefined; readonly modelCapabilities?: ModelCapability | undefined; + readonly telemetry?: TelemetryClient | undefined; } = {}, ): ReadMediaFileTool { const kaos = createFakeKaos({ @@ -68,6 +99,8 @@ function makeReadMediaTool( kaos, PERMISSIVE_WORKSPACE, input.modelCapabilities ?? capabilities(), + undefined, + input.telemetry, ); } @@ -655,9 +688,9 @@ describe('ReadMediaFileTool', () => { it('downsamples an oversized image but reports original dimensions', async () => { const big = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 3600, color: 0x3366ccff }).getBuffer('image/png'), ); - expect(sniffImageDimensions(big)).toEqual({ width: 2600, height: 2600 }); + expect(sniffImageDimensions(big)).toEqual({ width: 3600, height: 3600 }); const tool = makeReadMediaTool({ stat: vi.fn().mockResolvedValue({ ...DEFAULT_STAT, stSize: big.length }), @@ -678,14 +711,136 @@ describe('ReadMediaFileTool', () => { // The image actually sent to the model is downsampled to the edge cap. const sentBytes = Buffer.from(match![2]!, 'base64'); const sentDims = sniffImageDimensions(sentBytes); - expect(Math.max(sentDims!.width, sentDims!.height)).toBeLessThanOrEqual(2000); + expect(Math.max(sentDims!.width, sentDims!.height)).toBeLessThanOrEqual(3000); // The note keeps the ORIGINAL size so coordinate mapping holds. const systemText = noteText(result); - expect(systemText).toContain('2600x2600'); + expect(systemText).toContain('3600x3600'); expect(systemText).toContain(`${String(big.length)} bytes`); }); + it('reports an EXIF-rotated original in the decoded coordinate space', async () => { + // Orientation 6 (rotate 90° CW): the header says 3600x1800, but jimp + // decodes to 1800x3600 — the space the sent image and any region + // readback live in. The note's original size must match that space, + // not the pre-rotation header sniff. + const portrait = withExifOrientation( + new Uint8Array( + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/jpeg', { + quality: 90, + }), + ), + 6, + ); + const tool = makeReadMediaTool({ + stat: vi.fn().mockResolvedValue({ ...DEFAULT_STAT, stSize: portrait.length }), + readBytes: vi.fn().mockResolvedValue(portrait), + }); + + const result = await executeTool(tool, { + turnId: 't1', + toolCallId: 'c_exif', + args: { path: '/workspace/portrait.jpg' }, + signal, + }); + + const systemText = noteText(result); + expect(systemText).toContain('Original dimensions: 1800x3600'); + expect(systemText).toMatch(/downsampled to 1500x3000/); + }); + + it('reports the decoded size for a region read of an EXIF-rotated image', async () => { + // Region coordinates live in the decoded (rotated) space; the note's + // original size must agree with it even when the header sniff succeeds. + const portrait = withExifOrientation( + new Uint8Array( + await new Jimp({ width: 120, height: 80, color: 0x3366ccff }).getBuffer('image/jpeg', { + quality: 90, + }), + ), + 6, + ); + const tool = makeReadMediaTool({ + stat: vi.fn().mockResolvedValue({ ...DEFAULT_STAT, stSize: portrait.length }), + readBytes: vi.fn().mockResolvedValue(portrait), + }); + + const result = await executeTool(tool, { + turnId: 't1', + toolCallId: 'c_exif_region', + args: { path: '/workspace/portrait.jpg', region: { x: 0, y: 0, width: 40, height: 40 } }, + signal, + }); + + expect(noteText(result)).toContain('Original dimensions: 80x120'); + }); + + it('reports display-space dimensions for an EXIF-rotated image sent untouched', async () => { + // Within both budgets the original bytes are sent without decoding; the + // note must still report the display-space size so coordinates derived + // from it agree with a later region readback (which decodes). + const portrait = withExifOrientation( + new Uint8Array( + await new Jimp({ width: 120, height: 80, color: 0x3366ccff }).getBuffer('image/jpeg', { + quality: 90, + }), + ), + 6, + ); + const tool = makeReadMediaTool({ + stat: vi.fn().mockResolvedValue({ ...DEFAULT_STAT, stSize: portrait.length }), + readBytes: vi.fn().mockResolvedValue(portrait), + }); + + const result = await executeTool(tool, { + turnId: 't1', + toolCallId: 'c_exif_untouched', + args: { path: '/workspace/portrait.jpg' }, + signal, + }); + + const systemText = noteText(result); + expect(systemText).toContain('Original dimensions: 80x120'); + expect(systemText).not.toMatch(/downsampled/i); + }); + + it('emits image_compress and image_crop telemetry tagged read_media', async () => { + const events: { event: string; props: Record }[] = []; + const telemetry: TelemetryClient = { + track: (event, props) => events.push({ event, props: props ?? {} }), + }; + const big = Buffer.from( + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), + ); + const tool = makeReadMediaTool({ + stat: vi.fn().mockResolvedValue({ ...DEFAULT_STAT, stSize: big.length }), + readBytes: vi.fn().mockResolvedValue(big), + telemetry, + }); + + await executeTool(tool, { + turnId: 't1', + toolCallId: 'c_tele_read', + args: { path: '/workspace/big.png' }, + signal, + }); + expect(events).toHaveLength(1); + expect(events[0]!.event).toBe('image_compress'); + expect(events[0]!.props['source']).toBe('read_media'); + expect(events[0]!.props['outcome']).toBe('compressed'); + + await executeTool(tool, { + turnId: 't1', + toolCallId: 'c_tele_crop', + args: { path: '/workspace/big.png', region: { x: 0, y: 0, width: 100, height: 100 } }, + signal, + }); + expect(events).toHaveLength(2); + expect(events[1]!.event).toBe('image_crop'); + expect(events[1]!.props['source']).toBe('read_media'); + expect(events[1]!.props['ok']).toBe(true); + }); + describe('region and full_resolution', () => { async function bigPng(width: number, height: number): Promise { return Buffer.from( @@ -725,7 +880,7 @@ describe('ReadMediaFileTool', () => { }); it('announces a downsampled delivery and the region readback in the block', async () => { - const big = await bigPng(2600, 2600); + const big = await bigPng(3600, 3600); const result = await executeTool(toolFor(big), { turnId: 't1', toolCallId: 'c_note', @@ -734,11 +889,11 @@ describe('ReadMediaFileTool', () => { }); const systemText = noteText(result); - expect(systemText).toContain('2600x2600'); + expect(systemText).toContain('3600x3600'); // Wording must not depend on serialization order: some providers keep // the note inline after the media, others flatten tool text and // re-attach the image after it — so no "above"/"below". - expect(systemText).toMatch(/The attached image was downsampled to 2000x2000/); + expect(systemText).toMatch(/The attached image was downsampled to 3000x3000/); expect(systemText).toMatch(/fine detail/i); expect(systemText).toContain('region'); }); @@ -797,7 +952,7 @@ describe('ReadMediaFileTool', () => { }); it('serves full_resolution when the bytes fit the per-image budget', async () => { - const big = await bigPng(2600, 1300); // over the edge cap, tiny in bytes + const big = await bigPng(3900, 1950); // over the edge cap, tiny in bytes const result = await executeTool(toolFor(big), { turnId: 't1', toolCallId: 'c_fullres', diff --git a/packages/node-sdk/src/index.ts b/packages/node-sdk/src/index.ts index 0fb4cc928..d84e395f9 100644 --- a/packages/node-sdk/src/index.ts +++ b/packages/node-sdk/src/index.ts @@ -87,6 +87,7 @@ export type { CompressImageResult, CompressBase64Result, ImageCompressionCaptionInput, + ImageCompressionTelemetry, } from '@moonshot-ai/agent-core'; // Experimental feature flags — types only. Resolved values come from diff --git a/packages/server/src/routes/prompts.ts b/packages/server/src/routes/prompts.ts index 3d089a799..50062f695 100644 --- a/packages/server/src/routes/prompts.ts +++ b/packages/server/src/routes/prompts.ts @@ -13,7 +13,7 @@ import { promptSteerResultSchema, type PromptSubmission, } from '@moonshot-ai/protocol'; -import { IPromptService, AuthModelNotResolvedError, AuthProvisioningRequiredError, AuthTokenMissingError, AuthTokenUnauthorizedError, PromptAlreadyCompletedError, PromptNotFoundError, SessionBusyError, SessionNotFoundError, FileNotFoundError, ICoreProcessService, IEnvironmentService, IFileStore, buildImageCompressionCaption, compressImageForModel, compressBase64ForModel, persistOriginalImage, sessionMediaOriginalsDir, type IInstantiationService, type GetResult } from '@moonshot-ai/agent-core'; +import { IPromptService, AuthModelNotResolvedError, AuthProvisioningRequiredError, AuthTokenMissingError, AuthTokenUnauthorizedError, PromptAlreadyCompletedError, PromptNotFoundError, SessionBusyError, SessionNotFoundError, FileNotFoundError, ICoreProcessService, IEnvironmentService, IFileStore, buildImageCompressionCaption, compressImageForModel, compressBase64ForModel, persistOriginalImage, sessionMediaOriginalsDir, withTelemetryContext, type IInstantiationService, type GetResult, type ImageCompressionTelemetry, type TelemetryClient } from '@moonshot-ai/agent-core'; import { z } from 'zod'; @@ -132,6 +132,12 @@ export function registerPromptsRoutes( const core = a.get(ICoreProcessService); const cacheDir = join(a.get(IEnvironmentService).homeDir, 'cache'); const resolved = await resolvePromptMediaFiles(body, fileStore, cacheDir, { + // Scoped to the session so prompt-ingestion image_compress events + // carry the same context as the per-session agent telemetry. + telemetry: + core.telemetry === undefined + ? undefined + : withTelemetryContext(core.telemetry, { sessionId: session_id }), // Resolved lazily — only when an inline base64 image actually // got compressed — so image-free prompts never pay the lookup. resolveOriginalsDir: async () => { @@ -270,6 +276,8 @@ interface ResolvePromptMediaOptions { * to the shared temp-dir cache. */ readonly resolveOriginalsDir?: () => Promise; + /** Report an `image_compress` event per compressed prompt image. */ + readonly telemetry?: TelemetryClient; } async function resolvePromptMediaFiles( @@ -288,13 +296,17 @@ async function resolvePromptMediaFiles( } return originalsDir; }; + const telemetryFor = (source: string): ImageCompressionTelemetry | undefined => + options.telemetry === undefined ? undefined : { client: options.telemetry, source }; const content: PromptSubmission['content'] = []; for (const part of body.content) { // Inline base64 image: compress the payload in place. This is the same // input-stage step as the file path below, for REST clients that submit an // image as `{ source: { kind: 'base64' } }` instead of uploading a file. if (part.type === 'image' && part.source.kind === 'base64') { - const compressed = await compressBase64ForModel(part.source.data, part.source.media_type); + const compressed = await compressBase64ForModel(part.source.data, part.source.media_type, { + telemetry: telemetryFor('prompt_inline'), + }); if (compressed.changed) { // There is no stored file to point at, so persist the original into // the session's media-originals dir (best effort, temp-dir fallback) @@ -357,7 +369,9 @@ async function resolvePromptMediaFiles( let mediaType = file.meta.media_type; let bytes: Uint8Array = data; if (part.type === 'image') { - const compressed = await compressImageForModel(data, mediaType); + const compressed = await compressImageForModel(data, mediaType, { + telemetry: telemetryFor('prompt_file'), + }); if (compressed.changed) { // The stored file already holds the original bytes — the caption // points straight at it, no extra copy needed. diff --git a/packages/server/test/prompt.e2e.test.ts b/packages/server/test/prompt.e2e.test.ts index 1019715c5..1df6c6e48 100644 --- a/packages/server/test/prompt.e2e.test.ts +++ b/packages/server/test/prompt.e2e.test.ts @@ -32,7 +32,12 @@ import { WebSocket } from 'ws'; import { Jimp } from 'jimp'; import type { Event, PromptSubmission } from '@moonshot-ai/protocol'; -import { IEventService, IPromptService, PromptService } from '@moonshot-ai/agent-core'; +import { + IEventService, + IPromptService, + PromptService, + type TelemetryClient, +} from '@moonshot-ai/agent-core'; import { IRestGateway, startServer, type ServerStartOptions, type RunningServer } from '../src'; import { fixedTokenAuth } from './helpers/serverHarness'; @@ -61,13 +66,14 @@ afterEach(async () => { async function bootDaemon( serviceOverrides?: ServerStartOptions['serviceOverrides'], + telemetry?: TelemetryClient, ): Promise { server = await startServer({ host: '127.0.0.1', port: 0, lockPath, logger: pino({ level: 'silent' }), - coreProcessOptions: { homeDir: bridgeHome }, + coreProcessOptions: { homeDir: bridgeHome, telemetry }, wsGatewayOptions: { pingIntervalMs: 5_000, pongTimeoutMs: 5_000 }, serviceOverrides: [fixedTokenAuth(), ...(serviceOverrides ?? [])], }); @@ -445,7 +451,7 @@ describe('POST /api/v1/sessions/{sid}/prompts — submit validation (W7.2 / Chai const sid = await createSession(r); const bigPng = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ); const upload = buildMultipart({ file: { fieldName: 'file', filename: 'big.png', contentType: 'image/png', data: bigPng }, @@ -479,7 +485,7 @@ describe('POST /api/v1/sessions/{sid}/prompts — submit validation (W7.2 / Chai const sentBytes = Buffer.from(part.source.data, 'base64'); const decoded = await Jimp.fromBuffer(sentBytes); // The model-facing copy is downsampled to the edge cap. - expect(Math.max(decoded.width, decoded.height)).toBeLessThanOrEqual(2000); + expect(Math.max(decoded.width, decoded.height)).toBeLessThanOrEqual(3000); // Compression is announced next to the image, and the caption points at // the stored file (which keeps the original bytes) for readback. @@ -488,7 +494,7 @@ describe('POST /api/v1/sessions/{sid}/prompts — submit validation (W7.2 / Chai throw new Error('expected a compression caption before the image part'); } expect(caption.text).toContain('Image compressed'); - expect(caption.text).toContain('2600x2600'); + expect(caption.text).toContain('3600x1800'); const pathMatch = /saved at "([^"]+)"/.exec(caption.text); expect(pathMatch).not.toBeNull(); const persisted = await readFile(pathMatch![1]!); @@ -516,10 +522,10 @@ describe('POST /api/v1/sessions/{sid}/prompts — submit validation (W7.2 / Chai ]); const sid = await createSession(r); - // Solid 2600×2600: over the edge cap but tiny in bytes, so it stays well + // Solid 3600×1800: over the edge cap but tiny in bytes, so it stays well // under Fastify's inline-JSON limit yet still benefits from downscaling. const base64 = Buffer.from( - await new Jimp({ width: 2600, height: 2600, color: 0x3366ccff }).getBuffer('image/png'), + await new Jimp({ width: 3600, height: 1800, color: 0x3366ccff }).getBuffer('image/png'), ).toString('base64'); const res = await appOf(r).inject({ @@ -538,7 +544,7 @@ describe('POST /api/v1/sessions/{sid}/prompts — submit validation (W7.2 / Chai throw new Error('expected a base64 image part'); } const decoded = await Jimp.fromBuffer(Buffer.from(part.source.data, 'base64')); - expect(Math.max(decoded.width, decoded.height)).toBeLessThanOrEqual(2000); + expect(Math.max(decoded.width, decoded.height)).toBeLessThanOrEqual(3000); // Inline base64 has no stored file, so the original is persisted into the // session's media-originals dir and the caption points there. @@ -559,6 +565,54 @@ describe('POST /api/v1/sessions/{sid}/prompts — submit validation (W7.2 / Chai expect(persisted.equals(Buffer.from(base64, 'base64'))).toBe(true); }); + it('scopes prompt image compression telemetry to the session', async () => { + // The agent-side image_compress sources inherit the session context from + // their per-session telemetry client; the prompt-ingestion sources must + // match, or prompt_inline/prompt_file events cannot be correlated with + // the session they belong to. + const events: { event: string; sessionId: string | null | undefined }[] = []; + const makeClient = (ctx: { sessionId?: string | null }): TelemetryClient => ({ + track: (event) => events.push({ event, sessionId: ctx.sessionId }), + withContext: (patch) => makeClient({ ...ctx, ...patch }), + }); + const r = await bootDaemon( + [ + [ + IPromptService, + createPromptServiceOverride({ + submit: async (_sid, body) => ({ + prompt_id: 'prompt_from_stub', + user_message_id: 'msg_from_stub', + status: 'running', + content: body.content, + created_at: '2026-06-09T00:00:00.000Z', + }), + }), + ], + ], + makeClient({}), + ); + const sid = await createSession(r); + + const base64 = Buffer.from( + await new Jimp({ width: 3600, height: 400, color: 0x3366ccff }).getBuffer('image/png'), + ).toString('base64'); + const res = await appOf(r).inject({ + method: 'POST', + url: `/api/v1/sessions/${sid}/prompts`, + payload: { + content: [ + { type: 'image', source: { kind: 'base64', media_type: 'image/png', data: base64 } }, + ], + }, + }); + expect(envelopeOf(res.json()).code).toBe(0); + + const compress = events.filter((e) => e.event === 'image_compress'); + expect(compress).toHaveLength(1); + expect(compress[0]!.sessionId).toBe(sid); + }); + it('returns 40407 when prompt image file_id is unknown', async () => { let submitted = false; const r = await bootDaemon([ From 150206a6f7027879df954e26736b4baa5d336235 Mon Sep 17 00:00:00 2001 From: Luyu Cheng <2239547+chengluyu@users.noreply.github.com> Date: Tue, 7 Jul 2026 21:51:29 +0800 Subject: [PATCH 02/24] fix: count goal creation turn (#1477) --- .changeset/count-created-goal-turn.md | 5 +++ packages/agent-core/src/agent/turn/index.ts | 7 +++ .../test/harness/goal-session.test.ts | 44 +++++++++++++++++++ 3 files changed, 56 insertions(+) create mode 100644 .changeset/count-created-goal-turn.md diff --git a/.changeset/count-created-goal-turn.md b/.changeset/count-created-goal-turn.md new file mode 100644 index 000000000..8e9ba8ea2 --- /dev/null +++ b/.changeset/count-created-goal-turn.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +Count the turn that starts an autonomous goal toward its goal turn usage. diff --git a/packages/agent-core/src/agent/turn/index.ts b/packages/agent-core/src/agent/turn/index.ts index 95dbb68fa..d77d81029 100644 --- a/packages/agent-core/src/agent/turn/index.ts +++ b/packages/agent-core/src/agent/turn/index.ts @@ -370,6 +370,13 @@ export class TurnFlow { end.event.reason !== 'failed' && end.event.reason !== 'filtered' ) { + // The ordinary turn created or resumed the goal, so it counts as the + // first active goal turn before the continuation driver takes over. + const countedGoal = await this.agent.goal.incrementTurn(); + if (countedGoal?.budget.overBudget === true) { + await this.agent.goal.markBlocked({ reason: 'A configured budget was reached' }); + return end; + } return await this.driveGoal( this.allocateTurnId(), [{ type: 'text', text: GOAL_CONTINUATION_PROMPT }], diff --git a/packages/agent-core/test/harness/goal-session.test.ts b/packages/agent-core/test/harness/goal-session.test.ts index d2d5c0d14..4db52cf07 100644 --- a/packages/agent-core/test/harness/goal-session.test.ts +++ b/packages/agent-core/test/harness/goal-session.test.ts @@ -291,9 +291,53 @@ describe('goal session end-to-end', () => { ); const turnStarts = events.filter((e) => e['type'] === 'turn.started').length; expect(turnStarts).toBeGreaterThanOrEqual(2); + const completionEvent = events.find((event) => { + const change = event['change'] as Record | undefined; + return event['type'] === 'goal.updated' && change?.['kind'] === 'completion'; + }); + expect(completionEvent?.['change']).toMatchObject({ + kind: 'completion', + stats: expect.objectContaining({ turnsUsed: 2 }), + }); expect((await api.getGoal({ agentId: 'main' })).goal).toBeNull(); }); + it('does not start a synthetic continuation when creating the goal exhausts its turn budget', async () => { + const sessionDir = await makeTempDir(); + const events: Array> = []; + const { session, agent, scripted } = await setupSession(sessionDir, events, [ + 'CreateGoal', + 'SetGoalBudget', + ]); + const api = new SessionAPIImpl(session); + + scripted.mockNextResponse({ + type: 'function', + id: 'create', + name: 'CreateGoal', + arguments: JSON.stringify({ objective: 'work' }), + }); + scripted.mockNextResponse({ + type: 'function', + id: 'budget', + name: 'SetGoalBudget', + arguments: JSON.stringify({ value: 1, unit: 'turns' }), + }); + scripted.mockNextResponse({ type: 'text', text: 'Goal created with a one-turn budget.' }); + + agent.turn.prompt([{ type: 'text', text: 'Please start a one-turn goal' }]); + await agent.turn.waitForCurrentTurn(); + + const goal = (await api.getGoal({ agentId: 'main' })).goal; + expect(goal?.status).toBe('blocked'); + expect(goal?.turnsUsed).toBe(1); + expect(scripted.calls).toHaveLength(3); + expect(events.filter((e) => e['type'] === 'turn.started')).toHaveLength(1); + expect(JSON.stringify(agent.context.history)).not.toContain( + 'Continue working toward the active goal', + ); + }); + it('keeps the active turn alive (cancelable) while driving a goal created mid-turn', async () => { const sessionDir = await makeTempDir(); const events: Array> = []; From 6c9abe8cf7765b489ca50a2bb0f9b829fd680a51 Mon Sep 17 00:00:00 2001 From: _Kerman Date: Tue, 7 Jul 2026 22:02:29 +0800 Subject: [PATCH 03/24] feat(kosong): support structured response formats (#1397) --- .../structured-output-provider-format.md | 5 ++ packages/kosong/src/provider.ts | 23 ++++++ packages/kosong/src/providers/anthropic.ts | 23 ++++++ packages/kosong/src/providers/google-genai.ts | 15 ++++ packages/kosong/src/providers/kimi.ts | 19 +++++ .../kosong/src/providers/openai-legacy.ts | 19 +++++ .../kosong/src/providers/openai-responses.ts | 23 ++++++ packages/kosong/test/anthropic.test.ts | 48 +++++++++++- packages/kosong/test/google-genai.test.ts | 64 +++++++++++++++- packages/kosong/test/kimi.test.ts | 37 +++++++++- packages/kosong/test/openai-legacy.test.ts | 37 +++++++++- packages/kosong/test/openai-responses.test.ts | 73 ++++++++++++++++++- 12 files changed, 381 insertions(+), 5 deletions(-) create mode 100644 .changeset/structured-output-provider-format.md diff --git a/.changeset/structured-output-provider-format.md b/.changeset/structured-output-provider-format.md new file mode 100644 index 000000000..625fb130a --- /dev/null +++ b/.changeset/structured-output-provider-format.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kosong": patch +--- + +Add provider-level structured response format support. diff --git a/packages/kosong/src/provider.ts b/packages/kosong/src/provider.ts index be9bb6212..995e79bf7 100644 --- a/packages/kosong/src/provider.ts +++ b/packages/kosong/src/provider.ts @@ -2,6 +2,24 @@ import type { Message, StreamedMessagePart, VideoURLPart } from './message'; import type { Tool } from './tool'; import type { TokenUsage } from './usage'; +export type JsonSchemaObject = Record; + +export interface JsonObjectResponseFormat { + readonly type: 'json_object'; +} + +export interface JsonSchemaResponseFormat { + readonly type: 'json_schema'; + readonly jsonSchema: { + readonly name: string; + readonly schema: JsonSchemaObject; + readonly strict?: boolean | undefined; + readonly description?: string | undefined; + }; +} + +export type ResponseFormat = JsonObjectResponseFormat | JsonSchemaResponseFormat; + /** * Thinking effort passed to {@link ChatProvider.withThinking}. * @@ -118,6 +136,11 @@ export interface GenerateOptions { * each request/retry so providers never retain mutable credential state. */ auth?: ProviderRequestAuth; + /** + * Optional model-output format constraint. Providers map this to their native + * structured-output field when supported. + */ + responseFormat?: ResponseFormat; /** * Host-side instrumentation hook fired immediately before invoking the * provider adapter's generate call. diff --git a/packages/kosong/src/providers/anthropic.ts b/packages/kosong/src/providers/anthropic.ts index 9d38d9e3f..43e653989 100644 --- a/packages/kosong/src/providers/anthropic.ts +++ b/packages/kosong/src/providers/anthropic.ts @@ -12,6 +12,7 @@ import type { FinishReason, GenerateOptions, ProviderRequestAuth, + ResponseFormat, StreamedMessage, ThinkingEffort, } from '#/provider'; @@ -142,6 +143,27 @@ const ANTHROPIC_TOOL_CALL_ID_POLICY: ToolCallIdPolicy = { maxLength: 64, }; +function applyResponseFormat( + kwargs: Record, + format: ResponseFormat | undefined, +): void { + if (format === undefined) return; + if (format.type === 'json_object') { + throw new ChatProviderError( + 'Anthropic provider requires a JSON schema for structured response output.', + ); + } + const outputConfig = + kwargs['output_config'] !== undefined && kwargs['output_config'] !== null + ? { ...(kwargs['output_config'] as Record) } + : {}; + outputConfig['format'] = { + type: 'json_schema', + schema: format.jsonSchema.schema, + }; + kwargs['output_config'] = outputConfig; +} + /** * Per-version default output ceilings sourced from Anthropic's Messages * API model cards (platform.claude.com/docs/en/about-claude/models/overview). @@ -1114,6 +1136,7 @@ export class AnthropicChatProvider implements ChatProvider { if (this._generationKwargs.output_config !== undefined) { kwargs['output_config'] = this._generationKwargs.output_config; } + applyResponseFormat(kwargs, options?.responseFormat); if (this._generationKwargs.contextManagement !== undefined) { kwargs['context_management'] = this._generationKwargs.contextManagement; } diff --git a/packages/kosong/src/providers/google-genai.ts b/packages/kosong/src/providers/google-genai.ts index 0a72b00ea..e39d27d50 100644 --- a/packages/kosong/src/providers/google-genai.ts +++ b/packages/kosong/src/providers/google-genai.ts @@ -11,6 +11,7 @@ import type { FinishReason, GenerateOptions, ProviderRequestAuth, + ResponseFormat, StreamedMessage, ThinkingEffort, } from '#/provider'; @@ -127,6 +128,19 @@ function toolToGoogleGenAI(tool: Tool): GoogleTool { ], }; } + +function applyResponseFormat( + config: Record, + format: ResponseFormat | undefined, +): void { + if (format === undefined) return; + config['responseMimeType'] = 'application/json'; + delete config['responseSchema']; + delete config['responseJsonSchema']; + if (format.type === 'json_schema') { + config['responseJsonSchema'] = format.jsonSchema.schema; + } +} interface GoogleContent { role: string; parts: GooglePart[]; @@ -819,6 +833,7 @@ export class GoogleGenAIChatProvider implements ChatProvider { systemInstruction: systemPrompt, ...(tools.length > 0 ? { tools: tools.map((t) => toolToGoogleGenAI(t)) } : {}), }; + applyResponseFormat(config, options?.responseFormat); try { const client = this._createClient(options?.auth); diff --git a/packages/kosong/src/providers/kimi.ts b/packages/kosong/src/providers/kimi.ts index 30c2a3f99..44f63268c 100644 --- a/packages/kosong/src/providers/kimi.ts +++ b/packages/kosong/src/providers/kimi.ts @@ -6,6 +6,7 @@ import type { GenerateOptions, MaxCompletionTokensOptions, ProviderRequestAuth, + ResponseFormat, StreamedMessage, ThinkingEffort, VideoUploadInput, @@ -200,6 +201,21 @@ function convertTool(tool: Tool): OpenAIToolParam { }, }; } + +function responseFormatToOpenAI(format: ResponseFormat): Record { + if (format.type === 'json_object') { + return { type: 'json_object' }; + } + return { + type: 'json_schema', + json_schema: { + name: format.jsonSchema.name, + schema: format.jsonSchema.schema, + strict: format.jsonSchema.strict, + description: format.jsonSchema.description, + }, + }; +} /** * Extract usage from a streaming chunk. Moonshot may place usage in * `choices[0].usage` in addition to the top-level `usage` field. @@ -505,6 +521,9 @@ export class KimiChatProvider implements ChatProvider { ...requestKwargs, ...(extraBody as Record | undefined), }; + if (options?.responseFormat !== undefined) { + createParams['response_format'] = responseFormatToOpenAI(options.responseFormat); + } if (tools.length > 0) { createParams['tools'] = tools.map((t) => convertTool(t)); diff --git a/packages/kosong/src/providers/openai-legacy.ts b/packages/kosong/src/providers/openai-legacy.ts index c45d8f95c..35d759051 100644 --- a/packages/kosong/src/providers/openai-legacy.ts +++ b/packages/kosong/src/providers/openai-legacy.ts @@ -6,6 +6,7 @@ import type { GenerateOptions, MaxCompletionTokensOptions, ProviderRequestAuth, + ResponseFormat, StreamedMessage, ThinkingEffort, } from '#/provider'; @@ -60,6 +61,21 @@ const OPENAI_CHAT_TOOL_CALL_ID_POLICY: ToolCallIdPolicy = { maxLength: 64, }; +function responseFormatToOpenAI(format: ResponseFormat): Record { + if (format.type === 'json_object') { + return { type: 'json_object' }; + } + return { + type: 'json_schema', + json_schema: { + name: format.jsonSchema.name, + schema: format.jsonSchema.schema, + strict: format.jsonSchema.strict, + description: format.jsonSchema.description, + }, + }; +} + function extractReasoningContent( source: unknown, explicitKey: string | undefined, @@ -571,6 +587,9 @@ export class OpenAILegacyChatProvider implements ChatProvider { stream: this._stream, ...kwargs, }; + if (options?.responseFormat !== undefined) { + createParams['response_format'] = responseFormatToOpenAI(options.responseFormat); + } if (tools.length > 0) { createParams['tools'] = tools.map((t) => toolToOpenAI(t)); diff --git a/packages/kosong/src/providers/openai-responses.ts b/packages/kosong/src/providers/openai-responses.ts index 5549471d6..389d28d72 100644 --- a/packages/kosong/src/providers/openai-responses.ts +++ b/packages/kosong/src/providers/openai-responses.ts @@ -11,6 +11,7 @@ import type { FinishReason, GenerateOptions, ProviderRequestAuth, + ResponseFormat, StreamedMessage, ThinkingEffort, } from '#/provider'; @@ -362,6 +363,22 @@ interface ResponseToolParam { parameters: Record; strict: boolean; } + +function responseFormatToResponsesText(format: ResponseFormat): Record { + if (format.type === 'json_object') { + return { format: { type: 'json_object' } }; + } + return { + format: { + type: 'json_schema', + name: format.jsonSchema.name, + schema: format.jsonSchema.schema, + strict: format.jsonSchema.strict, + description: format.jsonSchema.description, + }, + }; +} + // The Responses API has no input type for video, and only mp3/wav audio can // be inlined as input_file data. Degrade such parts to placeholder text so // the model still learns an attachment existed instead of silently losing it. @@ -1083,6 +1100,12 @@ export class OpenAIResponsesChatProvider implements ChatProvider { if (systemPrompt) { createParams['instructions'] = systemPrompt; } + if (options?.responseFormat !== undefined) { + createParams['text'] = { + ...asRawObject(createParams['text']), + ...responseFormatToResponsesText(options.responseFormat), + }; + } if ( !('responses' in client) || diff --git a/packages/kosong/test/anthropic.test.ts b/packages/kosong/test/anthropic.test.ts index 3a3940495..ca9005439 100644 --- a/packages/kosong/test/anthropic.test.ts +++ b/packages/kosong/test/anthropic.test.ts @@ -1,6 +1,7 @@ import { ChatProviderError } from '#/errors'; import type { ContentPart, Message, StreamedMessagePart, ToolCall } from '#/message'; import { AnthropicChatProvider, resolveDefaultMaxTokens } from '#/providers/anthropic'; +import type { GenerateOptions } from '#/provider'; import type { Tool } from '#/tool'; import { describe, it, expect, vi } from 'vitest'; @@ -63,6 +64,7 @@ async function captureRequestBody( systemPrompt: string, tools: Tool[], history: Message[], + options?: GenerateOptions, ): Promise> { let capturedParams: Record | undefined; let capturedOptions: Record | undefined; @@ -75,7 +77,7 @@ async function captureRequestBody( return Promise.resolve(makeAnthropicResponse()); }); - const stream = await provider.generate(systemPrompt, tools, history); + const stream = await provider.generate(systemPrompt, tools, history, options); for await (const part of stream) { void part; } @@ -405,6 +407,50 @@ describe('AnthropicChatProvider', () => { ]); }); + it('maps json_schema response format to output_config.format', async () => { + const provider = createProvider(); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + const schema = { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + additionalProperties: false, + }; + + const body = await captureRequestBody(provider, '', [], history, { + responseFormat: { + type: 'json_schema', + jsonSchema: { + name: 'contact', + schema, + strict: true, + }, + }, + }); + + expect(body['output_config']).toEqual({ + format: { + type: 'json_schema', + schema, + }, + }); + }); + + it('rejects json_object response format because Anthropic requires a schema', async () => { + const provider = createProvider(); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + + await expect( + provider.generate('', [], history, { + responseFormat: { type: 'json_object' }, + }), + ).rejects.toThrow('Anthropic provider requires a JSON schema for structured response output.'); + }); + it('multi-turn conversation', async () => { const provider = createProvider(); const history: Message[] = [ diff --git a/packages/kosong/test/google-genai.test.ts b/packages/kosong/test/google-genai.test.ts index 772eebc0c..7f1fa81de 100644 --- a/packages/kosong/test/google-genai.test.ts +++ b/packages/kosong/test/google-genai.test.ts @@ -13,6 +13,7 @@ import { GoogleGenAIStreamedMessage, messagesToGoogleGenAIContents, } from '#/providers/google-genai'; +import type { GenerateOptions } from '#/provider'; import type { Tool } from '#/tool'; import { describe, it, expect, vi } from 'vitest'; @@ -50,6 +51,7 @@ async function captureRequestBody( systemPrompt: string, tools: Tool[], history: Message[], + options?: GenerateOptions, ): Promise> { let capturedBody: Record | undefined; @@ -69,7 +71,7 @@ async function captureRequestBody( return Promise.resolve(makeGenerateContentResponse()); }); - const stream = await provider.generate(systemPrompt, tools, history); + const stream = await provider.generate(systemPrompt, tools, history, options); for await (const part of stream) { void part; } @@ -131,6 +133,66 @@ describe('GoogleGenAIChatProvider', () => { expect(config['systemInstruction']).toBe('You are helpful.'); }); + it('maps json_schema response format to response config', async () => { + const provider = createProvider(); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + const schema = { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + additionalProperties: false, + }; + const body = await captureRequestBody(provider, '', [], history, { + responseFormat: { + type: 'json_schema', + jsonSchema: { + name: 'contact', + schema, + strict: true, + }, + }, + }); + + const config = body['config'] as Record; + expect(config['responseMimeType']).toBe('application/json'); + expect(config['responseJsonSchema']).toEqual(schema); + }); + + it('replaces native responseSchema when applying json_schema response format', async () => { + const provider = createProvider().withGenerationKwargs({ + responseSchema: { + type: 'object', + properties: { old: { type: 'string' } }, + }, + }); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + const schema = { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + additionalProperties: false, + }; + + const body = await captureRequestBody(provider, '', [], history, { + responseFormat: { + type: 'json_schema', + jsonSchema: { + name: 'contact', + schema, + strict: true, + }, + }, + }); + + const config = body['config'] as Record; + expect(config['responseSchema']).toBeUndefined(); + expect(config['responseJsonSchema']).toEqual(schema); + }); + it('system messages in history are wrapped and emitted as user content', () => { // Regression: Google GenAI's Content.role only accepts "user" or // "model", so a `system` message sitting in the replay history (from diff --git a/packages/kosong/test/kimi.test.ts b/packages/kosong/test/kimi.test.ts index 2ac66830e..ff9577754 100644 --- a/packages/kosong/test/kimi.test.ts +++ b/packages/kosong/test/kimi.test.ts @@ -2,6 +2,7 @@ import { generate } from '#/generate'; import type { ContentPart, Message, ToolCall } from '#/message'; import { extractUsageFromChunk, KimiChatProvider } from '#/providers/kimi'; import { extractUsage } from '#/providers/openai-common'; +import type { GenerateOptions } from '#/provider'; import type { Tool } from '#/tool'; import { describe, it, expect, vi } from 'vitest'; @@ -51,6 +52,7 @@ async function captureRequestBody( systemPrompt: string, tools: Tool[], history: Message[], + options?: GenerateOptions, ): Promise> { let capturedBody: Record | undefined; @@ -61,7 +63,7 @@ async function captureRequestBody( return Promise.resolve(makeChatCompletionResponse('kimi-k2')); }); - const stream = await provider.generate(systemPrompt, tools, history); + const stream = await provider.generate(systemPrompt, tools, history, options); for await (const part of stream) { void part; } @@ -619,6 +621,39 @@ describe('KimiChatProvider', () => { expect(body['max_tokens']).toBeUndefined(); }); + it('maps json_schema response format to response_format', async () => { + const provider = createProvider(); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + const schema = { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + additionalProperties: false, + }; + const body = await captureRequestBody(provider, '', [], history, { + responseFormat: { + type: 'json_schema', + jsonSchema: { + name: 'contact', + schema, + strict: true, + }, + }, + }); + + expect(body['response_format']).toEqual({ + type: 'json_schema', + json_schema: { + name: 'contact', + schema, + strict: true, + description: undefined, + }, + }); + }); + it('prefers max_completion_tokens when both fields are set', async () => { const provider = createProvider().withGenerationKwargs({ max_completion_tokens: 2048, diff --git a/packages/kosong/test/openai-legacy.test.ts b/packages/kosong/test/openai-legacy.test.ts index c77607f3d..1356261b3 100644 --- a/packages/kosong/test/openai-legacy.test.ts +++ b/packages/kosong/test/openai-legacy.test.ts @@ -1,6 +1,7 @@ import { generate } from '#/generate'; import type { ContentPart, Message, StreamedMessagePart, ToolCall } from '#/message'; import { OpenAILegacyChatProvider } from '#/providers/openai-legacy'; +import type { GenerateOptions } from '#/provider'; import type { Tool } from '#/tool'; import { describe, it, expect, vi } from 'vitest'; @@ -42,6 +43,7 @@ async function captureRequestBody( systemPrompt: string, tools: Tool[], history: Message[], + options?: GenerateOptions, ): Promise> { let capturedBody: Record | undefined; @@ -52,7 +54,7 @@ async function captureRequestBody( return Promise.resolve(makeChatCompletionResponse()); }); - const stream = await provider.generate(systemPrompt, tools, history); + const stream = await provider.generate(systemPrompt, tools, history, options); for await (const part of stream) { void part; } @@ -603,6 +605,39 @@ describe('OpenAILegacyChatProvider', () => { expect(body['max_tokens']).toBe(2048); }); + it('maps json_schema response format to response_format', async () => { + const provider = createProvider(); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + const schema = { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + additionalProperties: false, + }; + const body = await captureRequestBody(provider, '', [], history, { + responseFormat: { + type: 'json_schema', + jsonSchema: { + name: 'contact', + schema, + strict: true, + }, + }, + }); + + expect(body['response_format']).toEqual({ + type: 'json_schema', + json_schema: { + name: 'contact', + schema, + strict: true, + description: undefined, + }, + }); + }); + it('withMaxCompletionTokens sets max_tokens on the cloned provider', async () => { const original = createProvider(); const provider = original.withMaxCompletionTokens(1024); diff --git a/packages/kosong/test/openai-responses.test.ts b/packages/kosong/test/openai-responses.test.ts index 0ea272b7d..7421c8fdf 100644 --- a/packages/kosong/test/openai-responses.test.ts +++ b/packages/kosong/test/openai-responses.test.ts @@ -10,6 +10,7 @@ import { OpenAIResponsesChatProvider, OpenAIResponsesStreamedMessage, } from '#/providers/openai-responses'; +import type { GenerateOptions } from '#/provider'; import type { Tool } from '#/tool'; import { describe, it, expect, vi } from 'vitest'; @@ -45,6 +46,7 @@ async function captureRequestBody( systemPrompt: string, tools: Tool[], history: Message[], + options?: GenerateOptions, ): Promise> { let capturedBody: Record | undefined; @@ -57,7 +59,7 @@ async function captureRequestBody( return Promise.resolve(makeResponsesAPIResponse()); }); - const stream = await provider.generate(systemPrompt, tools, history); + const stream = await provider.generate(systemPrompt, tools, history, options); for await (const part of stream) { void part; } @@ -910,6 +912,75 @@ describe('OpenAIResponsesChatProvider', () => { expect(body['max_output_tokens']).toBe(1024); expect(provider.maxCompletionTokens).toBe(1024); }); + + it('maps json_schema response format to text.format', async () => { + const provider = createProvider(); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + const schema = { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + additionalProperties: false, + }; + const body = await captureRequestBody(provider, '', [], history, { + responseFormat: { + type: 'json_schema', + jsonSchema: { + name: 'contact', + schema, + strict: true, + }, + }, + }); + + expect(body['text']).toEqual({ + format: { + type: 'json_schema', + name: 'contact', + schema, + strict: true, + description: undefined, + }, + }); + }); + + it('preserves existing Responses text options when applying response format', async () => { + const provider = createProvider().withGenerationKwargs({ + text: { verbosity: 'low' }, + }); + const history: Message[] = [ + { role: 'user', content: [{ type: 'text', text: 'Extract contact' }], toolCalls: [] }, + ]; + const schema = { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + additionalProperties: false, + }; + const body = await captureRequestBody(provider, '', [], history, { + responseFormat: { + type: 'json_schema', + jsonSchema: { + name: 'contact', + schema, + strict: true, + }, + }, + }); + + expect(body['text']).toEqual({ + verbosity: 'low', + format: { + type: 'json_schema', + name: 'contact', + schema, + strict: true, + description: undefined, + }, + }); + }); }); describe('reasoning configuration', () => { From 131700097a732b97b3d17c5e2efa1c5a44b013ef Mon Sep 17 00:00:00 2001 From: Luyu Cheng <2239547+chengluyu@users.noreply.github.com> Date: Tue, 7 Jul 2026 22:29:40 +0800 Subject: [PATCH 04/24] fix: clarify goal blocked audit guidance (#1481) --- .changeset/goal-blocked-audit-prompt.md | 5 ++++ .../agent-core/src/agent/injection/goal.ts | 26 +++++++++++++------ packages/agent-core/src/agent/turn/index.ts | 26 ++++++++++++------- .../src/tools/builtin/goal/update-goal.md | 6 ++--- .../src/tools/builtin/goal/update-goal.ts | 4 ++- .../test/agent/injection/goal.test.ts | 12 +++++++++ .../test/harness/goal-session.test.ts | 6 +++++ packages/agent-core/test/tools/goal.test.ts | 17 +++++++++++- 8 files changed, 80 insertions(+), 22 deletions(-) create mode 100644 .changeset/goal-blocked-audit-prompt.md diff --git a/.changeset/goal-blocked-audit-prompt.md b/.changeset/goal-blocked-audit-prompt.md new file mode 100644 index 000000000..4f498ddb1 --- /dev/null +++ b/.changeset/goal-blocked-audit-prompt.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +Tighten goal-mode guidance for blocked and complete status updates. diff --git a/packages/agent-core/src/agent/injection/goal.ts b/packages/agent-core/src/agent/injection/goal.ts index 75f008bcf..b3af1dfce 100644 --- a/packages/agent-core/src/agent/injection/goal.ts +++ b/packages/agent-core/src/agent/injection/goal.ts @@ -147,14 +147,24 @@ function buildGoalReminder(goal: GoalSnapshot): string { 'after completing a useful slice, if material work remains, end the turn normally without ' + 'calling UpdateGoal so the runtime can continue the goal in the next turn. Call UpdateGoal ' + 'with `complete` only when all required work is done, any stated validation has passed, and ' + - 'there is no useful next action. Do not mark complete after only producing a plan, summary, ' + - 'first pass, or partial result. Before calling `complete`, check that every required part of ' + - 'the objective is done, completion criteria are satisfied, requested or expected validation ' + - 'passed or has been reported as impossible, and no known material task remains. Call ' + - 'UpdateGoal with `blocked` only for a genuine impasse: an external condition, required user ' + - 'input, missing credentials or permissions, a persistent technical failure, or an impossible, ' + - 'unsafe, or contradictory objective. Do not use `blocked` because the work is large, hard, ' + - 'slow, uncertain, partial, still needs validation, or needs more goal turns.', + 'there is no useful next action. Completion audit: before calling `complete`, verify the ' + + 'current state against the actual objective and every explicit requirement. Treat weak or ' + + 'indirect evidence as not complete. Do not mark complete after only producing a plan, ' + + 'summary, first pass, or partial result. Do not mark complete merely because a budget is ' + + 'nearly exhausted or you want to stop. Blocked audit: do not call UpdateGoal with `blocked` ' + + 'the first time you hit a blocker. Use `blocked` only for a genuine impasse: an external ' + + 'condition, required user input, missing credentials or permissions, or a persistent ' + + 'technical failure. For those non-terminal blockers, the same blocking condition must ' + + 'repeat for at least 3 consecutive goal turns before you call `blocked`, counting the ' + + 'original/user-triggered turn and automatic continuations. If a previously blocked goal is ' + + 'resumed, treat the resumed run as a fresh blocked audit. Exception: if the objective ' + + 'itself is impossible, unsafe, or contradictory, call UpdateGoal with `blocked` in the same ' + + 'turn; do not run more goal turns just to satisfy the audit. Do not use `blocked` because ' + + 'the work is large, hard, slow, uncertain, incomplete, still needs validation, would ' + + 'benefit from clarification, or needs more goal turns. Once the 3-turn threshold is met ' + + 'and you cannot make meaningful progress without user input or an external-state change, ' + + 'call UpdateGoal with `blocked`; do not keep reporting the blocker while leaving the goal ' + + 'active.', ); return lines.join('\n'); } diff --git a/packages/agent-core/src/agent/turn/index.ts b/packages/agent-core/src/agent/turn/index.ts index d77d81029..54bec107c 100644 --- a/packages/agent-core/src/agent/turn/index.ts +++ b/packages/agent-core/src/agent/turn/index.ts @@ -93,15 +93,23 @@ const GOAL_CONTINUATION_PROMPT = [ 'useful slice, if material work remains, end the turn normally without calling UpdateGoal so', 'the runtime can continue the goal in the next turn. Call UpdateGoal with `complete` only when', 'all required work is done, any stated validation has passed, and there is no useful next', - 'action. Do not mark complete after only producing a plan, summary, first pass, or partial', - 'result. Before calling `complete`, check that every required part of the objective is done,', - 'completion criteria are satisfied, requested or expected validation passed or has been', - 'reported as impossible, and no known material task remains. Call UpdateGoal with `blocked`', - 'only for a genuine impasse: an external condition, required user input, missing credentials', - 'or permissions, a persistent technical failure, or an impossible, unsafe, or contradictory', - 'objective. Do not use `blocked` because the work is large, hard, slow, uncertain, partial,', - 'still needs validation, or needs more goal turns. Do not ask the user for input unless a real', - 'blocker prevents progress.', + 'action. Completion audit: before calling `complete`, verify the current state against the', + 'actual objective and every explicit requirement. Treat weak or indirect evidence as not', + 'complete. Do not mark complete after only producing a plan, summary, first pass, or partial', + 'result. Do not mark complete merely because a budget is nearly exhausted or you want to stop.', + 'Blocked audit: do not call UpdateGoal with `blocked` the first time you hit a blocker. Use', + '`blocked` only for a genuine impasse: an external condition, required user input, missing', + 'credentials or permissions, or a persistent technical failure. For those non-terminal', + 'blockers, the same blocking condition must repeat for at least 3 consecutive goal turns before', + 'you call `blocked`, counting the original/user-triggered turn and automatic continuations.', + 'If a previously blocked goal is resumed, treat the resumed run as a fresh blocked audit.', + 'Exception: if the objective itself is impossible, unsafe, or contradictory, call UpdateGoal', + 'with `blocked` in the same turn; do not run more goal turns just to satisfy the audit. Do not', + 'use `blocked` because the work is large, hard, slow, uncertain, incomplete, still needs', + 'validation, would benefit from clarification, or needs more goal turns. Once the 3-turn', + 'threshold is met and you cannot make meaningful progress without user input or an', + 'external-state change, call UpdateGoal with `blocked`; do not keep reporting the blocker while', + 'leaving the goal active. Do not ask the user for input unless a real blocker prevents progress.', ].join(' '); export class TurnFlow { diff --git a/packages/agent-core/src/tools/builtin/goal/update-goal.md b/packages/agent-core/src/tools/builtin/goal/update-goal.md index 35f7d19c1..7b9aa26d8 100644 --- a/packages/agent-core/src/tools/builtin/goal/update-goal.md +++ b/packages/agent-core/src/tools/builtin/goal/update-goal.md @@ -1,7 +1,7 @@ Set the status of the current goal. This is how you resume, complete, or block an autonomous goal. - `active` — resume a paused or blocked goal when the user explicitly asks you to work on that goal. -- `complete` — the objective is satisfied and any stated validation has passed. The goal ends and a completion summary is recorded. -- `blocked` — a genuine impasse prevents useful progress: an external condition, required user input, missing credentials or permissions, a persistent technical failure, or an impossible, unsafe, or contradictory objective. The goal stops but can be resumed later. Do not use `blocked` because the work is large, hard, slow, uncertain, partial, still needs validation, or needs more goal turns. +- `complete` — the objective is satisfied and any stated validation has passed. The goal ends and a completion summary is recorded. Before using this, verify the current state against the actual objective and every explicit requirement. Treat weak or indirect evidence as not complete. Do not use `complete` merely because a budget is nearly exhausted or you want to stop. +- `blocked` — a genuine impasse prevents useful progress: an external condition, required user input, missing credentials or permissions, a persistent technical failure, or an impossible, unsafe, or contradictory objective. For non-terminal blockers, do not use `blocked` the first time you hit the blocker. The same blocking condition must repeat for at least 3 consecutive goal turns before you call `blocked`, counting the original/user-triggered turn and automatic continuations. If a previously blocked goal is resumed, treat the resumed run as a fresh blocked audit. If the objective itself is impossible, unsafe, or contradictory, call `blocked` in the same turn instead of running more goal turns. Do not use `blocked` because the work is large, hard, slow, uncertain, incomplete, still needs validation, would benefit from clarification, or needs more goal turns. Once the 3-turn threshold is met and you cannot make meaningful progress without user input or an external-state change, call `blocked` instead of leaving the goal active. -Most active goal turns should not call this tool. If you complete one useful slice of work and material work remains, end the turn normally without calling UpdateGoal; the runtime will prompt you to continue in the next goal turn. Call `complete` only when all required work is done, any stated validation has passed, and there is no useful next action. Before calling `complete`, check that every required part of the objective is done, completion criteria are satisfied, requested or expected validation passed or has been reported as impossible, and no known material task remains. Do not call `complete` after only producing a plan, summary, first pass, or partial result. If you call `blocked`, you will be prompted to explain the blocker in your next message. Setting the status is the machine-readable signal; the completion summary or blocker explanation is yours to write in the following message. +Most active goal turns should not call this tool. If you complete one useful slice of work and material work remains, end the turn normally without calling UpdateGoal; the runtime will prompt you to continue in the next goal turn. Call `complete` only when all required work is done, any stated validation has passed, and there is no useful next action. Do not call `complete` after only producing a plan, summary, first pass, or partial result. Call `blocked` only after the blocked audit threshold is met. If you call `blocked`, you will be prompted to explain the blocker in your next message. Setting the status is the machine-readable signal; the completion summary or blocker explanation is yours to write in the following message. diff --git a/packages/agent-core/src/tools/builtin/goal/update-goal.ts b/packages/agent-core/src/tools/builtin/goal/update-goal.ts index 29528cf60..8c4c90481 100644 --- a/packages/agent-core/src/tools/builtin/goal/update-goal.ts +++ b/packages/agent-core/src/tools/builtin/goal/update-goal.ts @@ -25,7 +25,9 @@ export const UpdateGoalToolInputSchema = z .object({ status: z .enum(['active', 'complete', 'blocked']) - .describe('The lifecycle status to set for the current goal.'), + .describe( + 'The lifecycle status to set for the current goal. Use `blocked` for impossible, unsafe, or contradictory objectives, or after the same non-terminal blocking condition repeats for at least 3 consecutive goal turns.', + ), }) .strict(); diff --git a/packages/agent-core/test/agent/injection/goal.test.ts b/packages/agent-core/test/agent/injection/goal.test.ts index ff02efb03..c5a395005 100644 --- a/packages/agent-core/test/agent/injection/goal.test.ts +++ b/packages/agent-core/test/agent/injection/goal.test.ts @@ -176,15 +176,27 @@ describe('GoalInjector content', () => { expect(text).toContain('Goal mode is iterative'); expect(text).toContain('one bounded, useful slice of work'); expect(text).toContain('end the turn normally without calling UpdateGoal'); + expect(text).toContain('Completion audit'); + expect(text).toContain('actual objective and every explicit requirement'); + expect(text).toContain('weak or indirect evidence'); expect(text).toContain('Do not mark complete after only producing a plan'); + expect(text).toContain('budget is nearly exhausted'); }); it('reserves blocked for genuine impasses rather than ordinary unfinished work', async () => { const store = makeStore(); await store.createGoal({ objective: 'finish the migration' }); const text = (await injectOnce(store))!; + expect(text).toContain('Blocked audit'); + expect(text).toContain('do not call UpdateGoal with `blocked` the first time'); expect(text).toContain('only for a genuine impasse'); expect(text).toContain('missing credentials or permissions'); + expect(text).toContain('3 consecutive goal turns'); + expect(text).toContain('fresh blocked audit'); + expect(text).toContain('Exception: if the objective itself is impossible, unsafe, or contradictory'); + expect(text).toContain('do not run more goal turns just to satisfy the audit'); + expect(text).toContain('would benefit from clarification'); + expect(text).toContain('do not keep reporting the blocker while leaving the goal active'); expect(text).toContain('needs more goal turns'); }); diff --git a/packages/agent-core/test/harness/goal-session.test.ts b/packages/agent-core/test/harness/goal-session.test.ts index 4db52cf07..e175e66ee 100644 --- a/packages/agent-core/test/harness/goal-session.test.ts +++ b/packages/agent-core/test/harness/goal-session.test.ts @@ -152,6 +152,12 @@ describe('goal session end-to-end', () => { expect(continuationHistory).toContain('Keep the self-audit brief'); expect(continuationHistory).toContain('do not run another goal turn'); expect(continuationHistory).toContain('end the turn normally without calling UpdateGoal'); + expect(continuationHistory).toContain('Completion audit'); + expect(continuationHistory).toContain('Blocked audit'); + expect(continuationHistory).toContain('3 consecutive goal turns'); + expect(continuationHistory).toContain('fresh blocked audit'); + expect(continuationHistory).toContain('impossible, unsafe, or contradictory'); + expect(continuationHistory).toContain('do not run more goal turns just to satisfy the audit'); expect(continuationHistory).toContain('only for a genuine impasse'); // Terminal UpdateGoal asks the model for one final user-facing summary. diff --git a/packages/agent-core/test/tools/goal.test.ts b/packages/agent-core/test/tools/goal.test.ts index 048ce9b7c..a065796dc 100644 --- a/packages/agent-core/test/tools/goal.test.ts +++ b/packages/agent-core/test/tools/goal.test.ts @@ -294,6 +294,10 @@ describe('UpdateGoalTool', () => { const description = new UpdateGoalTool(fakeAgent()).description.toLowerCase(); // Reserve blocked for genuine impasses, not ordinary unfinished work. expect(description).toContain('genuine impasse'); + expect(description).toContain('3 consecutive goal turns'); + expect(description).toContain('fresh blocked audit'); + expect(description).toContain('impossible, unsafe, or contradictory'); + expect(description).toContain('same turn instead of running more goal turns'); expect(description).toContain('hard, slow'); expect(description).toContain('needs more goal turns'); // UpdateGoal also injects the completion/blocked outcome prompt, so it does @@ -301,11 +305,22 @@ describe('UpdateGoalTool', () => { expect(description).not.toContain('only records the status'); }); + it('exposes the blocked-audit rule in the status parameter schema', () => { + const statusDescription = + ((new UpdateGoalTool(fakeAgent()).parameters as { + properties: Record; + }).properties['status']?.description) ?? ''; + expect(statusDescription).toContain('3 consecutive goal turns'); + expect(statusDescription).toContain('impossible, unsafe, or contradictory objectives'); + }); + it('discourages calling UpdateGoal after a non-terminal work slice', () => { const description = new UpdateGoalTool(fakeAgent()).description; expect(description).toContain('Most active goal turns should not call this tool'); expect(description).toContain('end the turn normally without calling UpdateGoal'); - expect(description).toContain('every required part of the objective is done'); + expect(description).toContain('actual objective and every explicit requirement'); + expect(description).toContain('weak or indirect evidence'); + expect(description).toContain('budget is nearly exhausted'); }); // Keep a capturing context here to prove terminal paths no longer append a From 9b76e5bff631cceaeecb2b0cbc096533c5fdc8cc Mon Sep 17 00:00:00 2001 From: STAR-QUAKE <99738745+starquakee@users.noreply.github.com> Date: Tue, 7 Jul 2026 23:06:29 +0800 Subject: [PATCH 05/24] feat(agent-core): discard loaded tool schemas on compaction (#1471) Align progressive tool disclosure with the discard-on-compaction model: compaction no longer rebuilds loaded dynamic tool schemas. The boundary announcement re-lists every loadable name, the model re-selects what it still needs, and a from-memory call to a no-longer-loaded tool is rejected by preflight with select guidance. This removes the keep-all rebuild and its half-trigger budget heuristics entirely: the post-compaction floor is back to users + summary, which is structurally outside the auto-compaction trigger band, and the guard baseline degenerates to summary + reinjected reminders. Every downstream mechanism already treated the empty loaded set as its consistent base state (ledger scan, pending clear at the compaction boundary, deferred extras, preflight wording), so this is a strict simplification. Co-authored-by: fengchenchen --- .../select-tools-discard-on-compaction.md | 5 + .../agent-core/src/agent/compaction/full.ts | 107 ++++-------------- packages/agent-core/src/agent/tool/index.ts | 15 ++- packages/agent-core/src/loop/run-turn.ts | 2 +- packages/agent-core/src/loop/turn-step.ts | 6 +- .../src/tools/builtin/select-tools.ts | 4 +- .../test/agent/tool-select.e2e.test.ts | 68 ++++++----- 7 files changed, 77 insertions(+), 130 deletions(-) create mode 100644 .changeset/select-tools-discard-on-compaction.md diff --git a/.changeset/select-tools-discard-on-compaction.md b/.changeset/select-tools-discard-on-compaction.md new file mode 100644 index 000000000..70bda32d5 --- /dev/null +++ b/.changeset/select-tools-discard-on-compaction.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +Progressive tool disclosure (`select_tools`, experimental): compaction now discards the loaded tool schemas instead of re-injecting them. After a compaction the boundary announcement re-lists every loadable tool name and the model re-selects what it still needs; a from-memory call to a no-longer-loaded tool is rejected with guidance to select it first. This keeps the post-compaction context at its minimal users+summary floor and removes the schema-rebuild budget heuristics. No effect unless the `tool-select` experimental flag and a `select_tools`-capable model are active. diff --git a/packages/agent-core/src/agent/compaction/full.ts b/packages/agent-core/src/agent/compaction/full.ts index 0bbc12577..ca1f19c8b 100644 --- a/packages/agent-core/src/agent/compaction/full.ts +++ b/packages/agent-core/src/agent/compaction/full.ts @@ -11,7 +11,6 @@ import { type GenerateResult, type Message, type TokenUsage, - type Tool, APIContextOverflowError, APIStatusError, createUserMessage, @@ -20,11 +19,7 @@ import { import type { Agent } from '..'; import type { GenerateOptionsWithRequestLogFields } from '../llm-request-logger'; import type { ContextMessage } from '../context/types'; -import { - collectLoadedDynamicToolNames, - DYNAMIC_TOOL_SCHEMA_VARIANT, - stripDynamicToolContext, -} from '../context/dynamic-tools'; +import { stripDynamicToolContext } from '../context/dynamic-tools'; import { isAbortError } from '../../loop/errors'; import { retryBackoffDelays, @@ -374,65 +369,6 @@ export class FullCompaction { }).trimEnd(); } - /** - * Keep-all rebuild (Phase 1): after compaction folded the history — and the - * dynamic tool schema messages with it — append ONE merged schema message so - * the model keeps calling its loaded tools without re-selecting. Schemas are - * read from the live registry, never copied from the old history, so a - * schema that changed since load self-heals here. Names whose server is - * currently disconnected have no registry schema and are not rebuilt (the - * model re-selects after reconnect); names that survived into the - * post-compaction history (none under today's users+summary rebuild, but - * guarded) are not duplicated. The message goes through the normal - * injection-origin append, so estimation and records pick it up as usual. - * - * Budget guard: the rebuilt floor (users + summary + schemas) is the one - * part of the post-compaction context that compaction itself can never - * shrink — if it lands inside the auto-compaction trigger band, every - * following step re-compacts and rebuilds in a loop. Admit schemas (in name - * order) only while the projected context stays within HALF the compaction - * trigger, so normal turn content still fits before the next compaction; - * anything dropped is simply re-selectable on demand, the same degradation - * as a disconnected server. - */ - private rebuildDynamicToolSchemas(activeBefore: ReadonlySet): void { - if (!this.agent.toolSelectEnabled) return; - if (activeBefore.size === 0) return; - const surviving = collectLoadedDynamicToolNames(this.agent.context.history); - const names = [...activeBefore] - .filter((name) => !surviving.has(name)) - .toSorted((a, b) => a.localeCompare(b)); - const candidates = names - .map((name) => this.agent.tools.getMcpToolSchema(name)) - .filter((tool): tool is NonNullable => tool !== undefined); - if (candidates.length === 0) return; - const tools: Tool[] = []; - let projected = this.tokenCountWithPending; - for (const tool of candidates) { - const toolTokens = estimateTokensForTools([tool]); - // shouldCompact is monotonic in usedSize, so doubling the projected - // size checks "within half the trigger" for both trigger branches - // (ratio and reserved-context). - if (this.strategy.shouldCompact((projected + toolTokens) * 2)) break; - tools.push(tool); - projected += toolTokens; - } - if (tools.length < candidates.length) { - this.agent.log.info('trimmed dynamic tool schema rebuild to stay clear of the compaction trigger', { - kept: tools.length, - dropped: candidates.slice(tools.length).map((tool) => tool.name), - }); - } - if (tools.length === 0) return; - this.agent.context.appendMessage({ - role: 'system', - content: [], - toolCalls: [], - tools, - origin: { kind: 'injection', variant: DYNAMIC_TOOL_SCHEMA_VARIANT }, - }); - } - private postProcessSummary(summary: string): string { const storeData = this.agent.tools.storeData(); const todos = (storeData[TODO_STORE_KEY] as readonly TodoItem[] | undefined) ?? []; @@ -450,11 +386,6 @@ export class FullCompaction { const startedAt = Date.now(); const originalHistory = [...this.agent.context.history]; const tokensBefore = estimateTokensForMessages(originalHistory); - // Loaded-tools snapshot BEFORE the rebuild below folds the history away; - // read here so the keep-all schema rebuild after applyCompaction knows - // what was active. (The ledger scans history, which applyCompaction - // replaces.) - const activeDynamicToolsBefore = new Set(this.agent.tools.loadedDynamicToolNames()); let retryCount = 0; try { await this.triggerPreCompactHook(data, tokensBefore, signal); @@ -491,11 +422,12 @@ export class FullCompaction { // Dynamic-tool protocol context (schema messages, loadable-tools // announcements) is excluded from the summarizer input entirely: it is // protocol state, not conversation — summarizing it wastes tokens and - // risks schema text leaking into the summary. Zero information loss: - // the post-compaction boundary re-announces the manifest and the - // keep-all rebuild re-carries the schemas. Must happen before project() - // (which strips the origin anchor). `originalHistory` itself stays - // untouched for the prefix-race check and `compactedCount`. + // risks schema text leaking into the summary. The post-compaction + // boundary re-announces the manifest; the schemas themselves are + // deliberately dropped (discard-on-compaction) and re-selectable on + // demand. Must happen before project() (which strips the origin + // anchor). `originalHistory` itself stays untouched for the + // prefix-race check and `compactedCount`. let historyForModel: readonly ContextMessage[] = stripDynamicToolContext(originalHistory); let droppedCount = 0; let overflowShrinkCount = 0; @@ -618,7 +550,14 @@ export class FullCompaction { tokensBefore, droppedCount: droppedCount === 0 ? undefined : droppedCount, }); - this.rebuildDynamicToolSchemas(activeDynamicToolsBefore); + // Loaded dynamic tool schemas are deliberately NOT rebuilt: compaction + // discards the loaded set entirely (the boundary announcement re-lists + // every loadable name, and the model re-selects what it still needs). + // Everything downstream already treats the empty loaded set as its + // consistent base state — the ledger scan finds no schema messages, the + // pending set was cleared by applyCompaction, deferred extras drop out + // of the executable table, and a from-memory call is rejected by + // preflight with select guidance. // Telemetry keys are snake_case, but the `context.apply_compaction` // record written below keeps its persisted camelCase field names @@ -638,17 +577,11 @@ export class FullCompaction { ? {} : { input_tokens: inputTotal(usage), output_tokens: usage.output }), }); - // Baseline the "nothing new since compaction" guard on the counter - // that includes the schema rebuild appended above. `result.tokensAfter` - // predates the rebuild (and deliberately keeps its persisted - // users+summary semantics — the rebuild message is accounted through - // the pending-estimate tail, so folding it into tokensAfter would - // double-count on both the live and the restore path). A baseline - // below the actual post-compaction floor would let checkAutoCompaction - // re-trigger even though the compacted shape cannot shrink further. - // compactionWorker raises it once more after injectAfterCompaction so - // the reinjected reminders join the floor too; this earlier capture - // stays as the fallback when reinjection throws. + // Baseline the "nothing new since compaction" guard on the live counter + // (== result.tokensAfter here, since nothing has been appended since + // applyCompaction). compactionWorker raises it once more after + // injectAfterCompaction so the reinjected reminders join the floor; + // this earlier capture stays as the fallback when reinjection throws. this.lastCompactedTokenCount = this.tokenCountWithPending; return result; } catch (error) { diff --git a/packages/agent-core/src/agent/tool/index.ts b/packages/agent-core/src/agent/tool/index.ts index f46dcd546..8803227e0 100644 --- a/packages/agent-core/src/agent/tool/index.ts +++ b/packages/agent-core/src/agent/tool/index.ts @@ -555,8 +555,8 @@ export class ToolManager { * defer-window pending set. History is the single source of truth, so the * ledger survives resume (records replay rebuilds the history), keeps its * state across undo (schema messages have `injection` origin and are not - * undone), and self-heals after compaction (the rebuild message re-carries - * the schemas). + * undone), and empties at compaction (schema messages are discarded with + * the folded history — the model re-selects what it still needs). */ loadedDynamicToolNames(): ReadonlySet { const names = collectLoadedDynamicToolNames(this.agent.context.history); @@ -580,12 +580,11 @@ export class ToolManager { } /** - * Compaction rebuilt the history: from here on the keep-all rebuild message - * (which may have trimmed or skipped schemas — budget guard, disconnected - * servers) is the sole truth about what is still loaded. A pending entry - * surviving past this boundary would report a schema the context no longer - * carries as loaded, and re-selecting it would wrongly answer - * "Already available" instead of injecting. + * Compaction rebuilt the history and discarded every loaded schema with it + * — the loaded set is empty from here on. A pending entry surviving past + * this boundary would report a schema the context no longer carries as + * loaded, and re-selecting it would wrongly answer "Already available" + * instead of injecting. */ onContextCompacted(): void { this.pendingLoadedDynamicTools.clear(); diff --git a/packages/agent-core/src/loop/run-turn.ts b/packages/agent-core/src/loop/run-turn.ts index cb0f328e4..4d55c8579 100644 --- a/packages/agent-core/src/loop/run-turn.ts +++ b/packages/agent-core/src/loop/run-turn.ts @@ -117,7 +117,7 @@ export async function runTurn(input: RunTurnInput): Promise { // Passed through unresolved: the step evaluates it AFTER beforeStep, // next to buildMessages, so the tool table and the request messages // come from the same state (beforeStep can run compaction, which - // trims loaded schemas and rewrites the ledger). + // discards loaded schemas and empties the ledger). buildTools, describeMissingTool, hooks, diff --git a/packages/agent-core/src/loop/turn-step.ts b/packages/agent-core/src/loop/turn-step.ts index 74a3899b8..df33de9a0 100644 --- a/packages/agent-core/src/loop/turn-step.ts +++ b/packages/agent-core/src/loop/turn-step.ts @@ -42,7 +42,7 @@ export interface ExecuteLoopStepDeps { * Per-step tool table builder; wins over the static `tools` snapshot. * Evaluated after `beforeStep`, next to `buildMessages`, so the executable * table and the request messages reflect the same state — `beforeStep` can - * run compaction, which trims loaded dynamic tool schemas. + * run compaction, which discards loaded dynamic tool schemas. */ readonly buildTools?: (() => readonly ExecutableTool[]) | undefined; /** See RunTurnInput.describeMissingTool. */ @@ -90,8 +90,8 @@ export async function executeLoopStep(deps: ExecuteLoopStepDeps): Promise<{ signal.throwIfAborted(); // Resolve the tool table AFTER beforeStep so it reflects the same state as - // the messages built below (beforeStep can run compaction, which trims - // loaded dynamic tool schemas out of the context and the ledger — a table + // the messages built below (beforeStep can run compaction, which discards + // loaded dynamic tool schemas from the context and the ledger — a table // captured earlier would still dispatch a tool the model no longer has). const stepTools = buildTools !== undefined ? buildTools() : tools; const messages = await buildMessages(); diff --git a/packages/agent-core/src/tools/builtin/select-tools.ts b/packages/agent-core/src/tools/builtin/select-tools.ts index f4a610b40..b54c4a083 100644 --- a/packages/agent-core/src/tools/builtin/select-tools.ts +++ b/packages/agent-core/src/tools/builtin/select-tools.ts @@ -94,8 +94,8 @@ export class SelectToolsTool implements BuiltinTool { // sorted by name for byte-stable output. History is never used as a // schema source; an already-loaded name whose registry schema has // since changed is NOT re-injected (no runtime last-wins reliance) — - // the next compaction rebuild or an explicit re-select picks up the - // new schema. + // the stale copy lasts at most until the next compaction discards + // the loaded set, after which a re-select injects the new schema. toLoad.sort((a, b) => a.localeCompare(b)); const tools = toLoad .map((name) => manager.getMcpToolSchema(name)) diff --git a/packages/agent-core/test/agent/tool-select.e2e.test.ts b/packages/agent-core/test/agent/tool-select.e2e.test.ts index 9ef5ec207..e26a0082f 100644 --- a/packages/agent-core/test/agent/tool-select.e2e.test.ts +++ b/packages/agent-core/test/agent/tool-select.e2e.test.ts @@ -490,12 +490,13 @@ describe('disclosure mode — executable table freshness', () => { }); describe('disclosure mode — compaction', () => { - it('filters protocol context from the summarizer input and rebuilds schemas after compaction', async () => { + it('filters protocol context from the summarizer input and discards loaded schemas after compaction', async () => { const ctx = await disclosureAgent(); ctx.mockNextResponse({ type: 'text', text: 'load' }, selectCall('call-1', [GRAFANA_TOOL])); ctx.mockNextResponse({ type: 'text', text: 'ok' }); await runTurn(ctx, 'load it'); + expect(schemaMessages(ctx)).toHaveLength(1); const compacted = new Promise<{ tokensAfter: number }>((resolve) => { ctx.emitter.once('context.apply_compaction', (entry: { args: { tokensAfter: number } }) => { @@ -513,36 +514,42 @@ describe('disclosure mode — compaction', () => { expect(summarizerCall.history.some((m) => m.tools !== undefined)).toBe(false); expect(JSON.stringify(summarizerCall.history)).not.toContain(''); - // Post-compaction context: one rebuild message with the registry schema, - // plus a fresh full announcement — no re-select needed. - const rebuilt = schemaMessages(ctx); - expect(rebuilt).toHaveLength(1); - expect(rebuilt[0]!.tools!.map((t) => t.name)).toEqual([GRAFANA_TOOL]); - expect(rebuilt[0]!.origin).toEqual({ kind: 'injection', variant: 'dynamic_tool_schema' }); + // Post-compaction context: the loaded set is DISCARDED — no schema + // message is rebuilt, the ledger is empty, deferred extras drop out of + // the executable table. The boundary announcement re-lists every + // loadable name so the model re-selects what it still needs. + expect(schemaMessages(ctx)).toHaveLength(0); + expect(ctx.agent.tools.loadedDynamicToolNames().has(GRAFANA_TOOL)).toBe(false); + expect(ctx.agent.tools.loopTools.map((t) => t.name)).not.toContain(GRAFANA_TOOL); expect(ctx.agent.context.history.filter(isLoadableToolsAnnouncement)).toHaveLength(1); - expect(ctx.agent.tools.loadedDynamicToolNames().has(GRAFANA_TOOL)).toBe(true); - expect(ctx.agent.tools.loopTools.map((t) => t.name)).toContain(GRAFANA_TOOL); + expect(historyText(ctx)).toContain(`\n${GRAFANA_TOOL}\n`); - // The "nothing new since compaction" guard must be baselined on the - // true post-compaction floor: summary + rebuild message + the reinjected - // announcement. result.tokensAfter predates all of it, and a baseline - // that misses any re-appended piece would let auto-compaction re-trigger - // against a floor that cannot shrink (each round strips and re-appends - // the same reminders). + // The "nothing new since compaction" guard is baselined on the true + // post-compaction floor: summary + the reinjected announcement (no + // rebuild message anymore). A baseline missing any re-appended piece + // would let auto-compaction re-trigger against a floor that cannot + // shrink (each round strips and re-appends the same reminders). const internals = ctx.agent.fullCompaction as unknown as { lastCompactedTokenCount: number | null; }; const reAnnouncement = ctx.agent.context.history.filter(isLoadableToolsAnnouncement).at(-1)!; expect(internals.lastCompactedTokenCount).toBe( - tokensAfter + estimateTokensForMessage(rebuilt[0]!) + estimateTokensForMessage(reAnnouncement), + tokensAfter + estimateTokensForMessage(reAnnouncement), ); + // Re-selecting after compaction takes the injection branch again (not + // "Already available") — the discard is total, not just cosmetic. + ctx.mockNextResponse({ type: 'text', text: 'reload' }, selectCall('call-2', [GRAFANA_TOOL])); + ctx.mockNextResponse({ type: 'text', text: 'ok' }); + await runTurn(ctx, 'load it again'); + expect(toolResultTexts(ctx)).toContainEqual(`Loaded: ${GRAFANA_TOOL}`); + expect(schemaMessages(ctx)).toHaveLength(1); + expect(ctx.agent.tools.loadedDynamicToolNames().has(GRAFANA_TOOL)).toBe(true); + // The baseline lives strictly within one turn: runOneTurn re-arms it at // every turn boundary, which is what makes cross-turn staleness (undo, // model switches, /clear while idle) structurally impossible. If this // reset ever moves, the guard's staleness analysis must be redone. - ctx.mockNextResponse({ type: 'text', text: 'next turn' }); - await runTurn(ctx, 'anything new'); expect(internals.lastCompactedTokenCount).toBeNull(); }); @@ -590,11 +597,13 @@ describe('disclosure mode — compaction', () => { expect(backCall.tools.map((t) => t.name)).not.toContain('select_tools'); }); - it('trims the schema rebuild instead of re-entering the compaction trigger band', async () => { - // A trigger far below one fat schema: without the rebuild budget guard the - // post-compaction floor (users + summary + schema) would sit permanently - // above the trigger, and every later step would re-compact and rebuild in - // a loop (with the default Infinity per-turn cap, forever). + it('discards heavy loaded schemas at compaction instead of re-entering the trigger band', async () => { + // A trigger far below one fat schema: if compaction carried loaded + // schemas back into the context, the post-compaction floor (users + + // summary + schema) would sit permanently above the trigger and every + // later step would re-compact in a loop (with the default Infinity + // per-turn cap, forever). Discard-on-compaction keeps the floor at + // users + summary, structurally outside the band. const trigger = 2_000; const ctx = testAgent({ experimentalFlags: toolSelectFlagOn(), @@ -638,10 +647,11 @@ describe('disclosure mode — compaction', () => { await ctx.rpc.setPermission({ mode: 'yolo' }); // Step 1 loads the fat schema; step 2's boundary trips the trigger and - // blocks on auto-compaction (consuming the summary mock), which trims the - // rebuild. Step 2 then calls the MCP tool directly — the executable table - // is resolved AFTER the compaction (same state as the messages), so the - // now-unloaded tool must be rejected by preflight, not dispatched. + // blocks on auto-compaction (consuming the summary mock), which discards + // the loaded schema. Step 2 then calls the MCP tool directly — the + // executable table is resolved AFTER the compaction (same state as the + // messages), so the now-unloaded tool must be rejected by preflight, not + // dispatched. const fatCallLog: unknown[] = []; (fatClient as { callTool: unknown }).callTool = async (...args: unknown[]) => { fatCallLog.push(args); @@ -653,12 +663,12 @@ describe('disclosure mode — compaction', () => { ctx.mockNextResponse({ type: 'text', text: 'done' }); await runTurn(ctx, 'load the fat tool'); - // The rebuild was trimmed away: no schema message survives, the ledger is + // The loaded set was discarded: no schema message survives, the ledger is // empty again, and the tool is simply re-selectable on demand. expect(schemaMessages(ctx)).toHaveLength(0); expect(ctx.agent.tools.loadedDynamicToolNames().has(GRAFANA_TOOL)).toBe(false); - // The direct call after the trim was rejected with select guidance and + // The direct call after the discard was rejected with select guidance and // never reached the MCP client. expect(fatCallLog).toHaveLength(0); expect(toolResultTexts(ctx).join('\n')).toContain( From f30781bb273321f3e3bbb548a9d0724ab6299fc6 Mon Sep 17 00:00:00 2001 From: qer Date: Tue, 7 Jul 2026 23:38:20 +0800 Subject: [PATCH 06/24] fix(kimi-code): exit 1 when a headless (-p) turn fails (#1483) Headless (`kimi -p`) failures could exit with code 0 when the event loop drained during the shutdown cleanup (e.g. telemetry's unref'd retry backoff when the network is blocked), because the rejection never reached the process.exit(1) call. Set the failure exit code before any await in both the run-prompt catch and the main catch, and keep the cleanup timeout ref'd so the loop stays alive long enough for the rejection to propagate. --- .changeset/fix-headless-exit-code.md | 5 +++++ apps/kimi-code/src/cli/run-prompt.ts | 7 +++++-- apps/kimi-code/src/main.ts | 10 +++++++++ apps/kimi-code/test/cli/main.test.ts | 31 ++++++++++++++++++++++++++++ 4 files changed, 51 insertions(+), 2 deletions(-) create mode 100644 .changeset/fix-headless-exit-code.md diff --git a/.changeset/fix-headless-exit-code.md b/.changeset/fix-headless-exit-code.md new file mode 100644 index 000000000..f2de03dd8 --- /dev/null +++ b/.changeset/fix-headless-exit-code.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +Fix `kimi -p` runs exiting with code 0 when a turn fails. diff --git a/apps/kimi-code/src/cli/run-prompt.ts b/apps/kimi-code/src/cli/run-prompt.ts index a9221d215..100368203 100644 --- a/apps/kimi-code/src/cli/run-prompt.ts +++ b/apps/kimi-code/src/cli/run-prompt.ts @@ -44,7 +44,11 @@ import { createKimiCodeHostIdentity } from './version'; * * Used to bound shutdown so a wedged cleanup step can't keep a completed * headless run alive, without silently swallowing a cleanup that fails fast. The - * timer is unref'd so it never keeps the loop alive on its own. + * timer stays ref'd so a cleanup step that suspends on an unref'd handle (e.g. + * telemetry's retry backoff when the network is blocked) can't drain the event + * loop and exit 0 before the rejection propagates — the timer keeps the loop + * alive until it fires, then gives the rejection a chance to surface. A wedged + * cleanup is still bounded by `timeoutMs`, so this can't hang the run forever. */ async function raceWithTimeout(promise: Promise, timeoutMs: number): Promise { let timedOut = false; @@ -61,7 +65,6 @@ async function raceWithTimeout(promise: Promise, timeoutMs: number): Promi timedOut = true; resolve(); }, timeoutMs); - timer.unref?.(); }); try { await Promise.race([guarded, timedOutSignal]); diff --git a/apps/kimi-code/src/main.ts b/apps/kimi-code/src/main.ts index a54bfa2bd..860c4423d 100644 --- a/apps/kimi-code/src/main.ts +++ b/apps/kimi-code/src/main.ts @@ -172,6 +172,16 @@ export function main(): void { } }) .catch(async (error: unknown) => { + // Set the failure exit code synchronously, before any `await`. The + // terminal `process.exit(1)` below is our intended exit, but it sits + // behind `await logStartupFailure(...)`; by the time we reach that + // await, the failed run's `finally` cleanup has already torn down its + // ref'd handles (sockets, timers, background tasks). If the event loop + // drains during the await, Node exits on its own with the DEFAULT code + // 0 and `process.exit(1)` never runs — headless (`kimi -p`) failures + // would then exit 0 nondeterministically. Setting `process.exitCode` + // up front makes that drain-exit report failure too. + process.exitCode = 1; const operation = opts.prompt !== undefined ? 'run prompt' : 'start shell'; await logStartupFailure(operation, error); process.stderr.write( diff --git a/apps/kimi-code/test/cli/main.test.ts b/apps/kimi-code/test/cli/main.test.ts index 04d6556cf..31411e8b1 100644 --- a/apps/kimi-code/test/cli/main.test.ts +++ b/apps/kimi-code/test/cli/main.test.ts @@ -32,6 +32,7 @@ const mocks = vi.hoisted(() => { })), initializeCliTelemetry: vi.fn(), handleUpgrade: vi.fn(), + flushDiagnosticLogs: vi.fn(), finalizeHeadlessRun: vi.fn(), log: { info: vi.fn(), @@ -80,6 +81,7 @@ vi.mock('@moonshot-ai/kimi-code-sdk', async () => { mocks.createKimiHarness(...args); return mocks.harness; }, + flushDiagnosticLogs: mocks.flushDiagnosticLogs, KimiHarness: MockKimiHarness, log: mocks.log, }; @@ -215,6 +217,7 @@ describe('main entry command handling', () => { mocks.harness.close.mockResolvedValue(undefined); mocks.shutdownTelemetry.mockResolvedValue(undefined); mocks.handleUpgrade.mockResolvedValue(0); + mocks.flushDiagnosticLogs.mockResolvedValue(undefined); }); it('runs update preflight before starting the shell', async () => { @@ -301,6 +304,34 @@ describe('main entry command handling', () => { expect(typeof forceExitArgs[2]).toBe('function'); }); + it('sets the failure exit code before awaiting startup failure logging', async () => { + const originalExitCode = process.exitCode; + const opts: CLIOptions = { ...defaultOpts(), prompt: 'explain the repo' }; + mocks.validateOptions.mockReturnValue({ options: opts, uiMode: 'print' }); + mocks.runUpdatePreflight.mockResolvedValue('continue'); + mocks.runPrompt.mockRejectedValue(new Error('provider failed')); + mocks.flushDiagnosticLogs.mockImplementation(() => new Promise(() => {})); + const exitSpy = vi.spyOn(process, 'exit').mockImplementation((code?: string | number | null) => { + throw new ExitCalled(Number(code ?? 0)); + }); + + try { + main(); + const programArgs = mocks.createProgram.mock.calls[0] as unknown as unknown[]; + const mainAction = programArgs[1] as (opts: CLIOptions) => void; + mainAction(opts); + + await waitForAssertion(() => { + expect(mocks.flushDiagnosticLogs).toHaveBeenCalledTimes(1); + }); + expect(process.exitCode).toBe(1); + expect(exitSpy).not.toHaveBeenCalled(); + } finally { + exitSpy.mockRestore(); + process.exitCode = originalExitCode; + } + }); + it('keeps shell mode update preflight interactive by default', async () => { const opts = defaultOpts(); mocks.validateOptions.mockReturnValue({ options: opts, uiMode: 'shell' }); From 2206d21327129aa2331b6b159cfce61110b7f94f Mon Sep 17 00:00:00 2001 From: qer Date: Wed, 8 Jul 2026 12:22:28 +0800 Subject: [PATCH 07/24] feat(plugins): add Vercel plugin to marketplace (#1489) --- .changeset/add-vercel-plugin.md | 5 +++++ plugins/marketplace.json | 9 +++++++++ 2 files changed, 14 insertions(+) create mode 100644 .changeset/add-vercel-plugin.md diff --git a/.changeset/add-vercel-plugin.md b/.changeset/add-vercel-plugin.md new file mode 100644 index 000000000..8fd208266 --- /dev/null +++ b/.changeset/add-vercel-plugin.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +Add the Vercel plugin to the bundled plugin marketplace. Run /plugins and select Vercel Plugin to install it. diff --git a/plugins/marketplace.json b/plugins/marketplace.json index 1e514a33a..b35157532 100644 --- a/plugins/marketplace.json +++ b/plugins/marketplace.json @@ -18,6 +18,15 @@ "homepage": "https://github.com/obra/superpowers", "keywords": ["skills", "planning", "tdd", "debugging", "code-review"], "source": "https://github.com/obra/superpowers" + }, + { + "id": "vercel-plugin", + "tier": "curated", + "displayName": "Vercel Plugin", + "description": "Comprehensive Vercel ecosystem plugin — skills, agents, and conventions for the Vercel platform.", + "homepage": "https://vercel.com/docs/agent-resources/vercel-plugin", + "keywords": ["vercel", "deployment", "nextjs", "skills", "agents"], + "source": "https://github.com/vercel/vercel-plugin" } ] } From b30a45efecfa5ece4f4f10f2c5403ba097e7690b Mon Sep 17 00:00:00 2001 From: qer Date: Wed, 8 Jul 2026 13:24:44 +0800 Subject: [PATCH 08/24] feat(web): support Enter key to confirm archive and other dialogs (#1490) --- .changeset/web-confirm-dialog-enter-key.md | 5 +++ .../src/components/dialogs/ConfirmDialog.vue | 45 ++++++++++++++++++- apps/kimi-web/src/components/ui/Dialog.vue | 18 +++++++- 3 files changed, 65 insertions(+), 3 deletions(-) create mode 100644 .changeset/web-confirm-dialog-enter-key.md diff --git a/.changeset/web-confirm-dialog-enter-key.md b/.changeset/web-confirm-dialog-enter-key.md new file mode 100644 index 000000000..c74cc4e05 --- /dev/null +++ b/.changeset/web-confirm-dialog-enter-key.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +web: Press Enter to confirm in archive and other confirmation dialogs. diff --git a/apps/kimi-web/src/components/dialogs/ConfirmDialog.vue b/apps/kimi-web/src/components/dialogs/ConfirmDialog.vue index 803031631..a4ac057d1 100644 --- a/apps/kimi-web/src/components/dialogs/ConfirmDialog.vue +++ b/apps/kimi-web/src/components/dialogs/ConfirmDialog.vue @@ -3,11 +3,19 @@ Dialog (height auto, right-aligned footer). The single confirmation surface for user actions — driven app-wide by useConfirmDialog(). --> diff --git a/apps/kimi-web/src/components/ui/Dialog.vue b/apps/kimi-web/src/components/ui/Dialog.vue index 6624a4f92..a11153bce 100644 --- a/apps/kimi-web/src/components/ui/Dialog.vue +++ b/apps/kimi-web/src/components/ui/Dialog.vue @@ -22,6 +22,9 @@ const props = withDefaults(defineProps<{ /** When false, the body has no padding so the consumer controls layout * (e.g. a full-bleed side-nav). */ padded?: boolean; + /** Element (or selector / resolver) to receive focus when the dialog opens. + * Falls back to the first focusable element, then the dialog panel. */ + initialFocus?: HTMLElement | string | (() => HTMLElement | null | undefined); }>(), { closeOnOverlay: true, closeOnEsc: true, @@ -50,6 +53,18 @@ function focusables(): HTMLElement[] { return panel.value ? Array.from(panel.value.querySelectorAll(FOCUSABLE)) : []; } +function resolveInitialFocus(): HTMLElement | null { + const { initialFocus } = props; + if (!initialFocus) return null; + if (typeof initialFocus === 'function') { + return initialFocus() ?? null; + } + if (typeof initialFocus === 'string') { + return panel.value?.querySelector(initialFocus) ?? null; + } + return panel.value?.contains(initialFocus) ? initialFocus : null; +} + function onKeydown(event: KeyboardEvent) { if (!props.open) return; if (event.key === 'Escape' && props.closeOnEsc) { @@ -87,8 +102,9 @@ watch( openDialogCount.value += 1; previouslyFocused = document.activeElement; await nextTick(); + const initial = resolveInitialFocus(); const list = focusables(); - (list[0] ?? panel.value)?.focus(); + (initial ?? list[0] ?? panel.value)?.focus(); } else { openDialogCount.value = Math.max(0, openDialogCount.value - 1); if (previouslyFocused instanceof HTMLElement) { From 2ad0120c2a5c8383892e4da1ee7c6853926ed365 Mon Sep 17 00:00:00 2001 From: liruifengv Date: Wed, 8 Jul 2026 13:42:22 +0800 Subject: [PATCH 09/24] feat(web): redesign cron reminder as a message bubble (#1480) * feat(web): redesign cron reminder as a message bubble Restyle the cron trigger notice as a right-aligned user-style message bubble that shows the scheduled prompt in full (wrapping across lines), with a small meta row beneath it for the schedule, status, job id and run time. Extract a shared MessageTime component used by both user messages and the cron reminder so the timestamp format and click-to-expand behavior stay consistent, and give the CronCreate/CronList/CronDelete tools distinct calendar icons. * refactor(web): render cron reminders only as standalone turns Remove the embedded cron block path from the web transcript projector so cron reminder fires always render through the standalone right-aligned bubble path. * chore(web): simplify cron redesign changeset --- .changeset/web-cron-redesign.md | 5 + .../kimi-web/src/components/chat/ChatPane.vue | 59 +----- .../src/components/chat/CronNotice.vue | 175 +++++++++--------- .../src/components/chat/MessageTime.vue | 62 +++++++ .../src/components/chatTurnRendering.ts | 7 +- .../src/composables/messagesToTurns.ts | 27 +-- apps/kimi-web/src/lib/icons.ts | 15 ++ apps/kimi-web/src/lib/toolMeta.ts | 4 + apps/kimi-web/src/types.ts | 3 +- apps/kimi-web/test/turn-logic.test.ts | 50 ----- 10 files changed, 182 insertions(+), 225 deletions(-) create mode 100644 .changeset/web-cron-redesign.md create mode 100644 apps/kimi-web/src/components/chat/MessageTime.vue diff --git a/.changeset/web-cron-redesign.md b/.changeset/web-cron-redesign.md new file mode 100644 index 000000000..d338b601c --- /dev/null +++ b/.changeset/web-cron-redesign.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +web: Redesign the scheduled reminder UI. diff --git a/apps/kimi-web/src/components/chat/ChatPane.vue b/apps/kimi-web/src/components/chat/ChatPane.vue index 7103a8cb8..47307bccf 100644 --- a/apps/kimi-web/src/components/chat/ChatPane.vue +++ b/apps/kimi-web/src/components/chat/ChatPane.vue @@ -9,13 +9,13 @@ import Markdown from './Markdown.vue'; import ThinkingBlock from './ThinkingBlock.vue'; import ActivityNotice from './ActivityNotice.vue'; import CronNotice from './CronNotice.vue'; +import MessageTime from './MessageTime.vue'; import AuthMedia from './AuthMedia.vue'; import MoonSpinner from '../ui/MoonSpinner.vue'; import Spinner from '../ui/Spinner.vue'; import Icon from '../ui/Icon.vue'; import Tooltip from '../ui/Tooltip.vue'; import { useConfirmDialog } from '../../composables/useConfirmDialog'; -import { formatMessageTime } from '../../lib/formatMessageTime'; import { copyTextToClipboard } from '../../lib/clipboard'; import { assistantRenderBlocks, @@ -318,27 +318,6 @@ const undoingTurnId = ref(null); let undoFallbackTimer: ReturnType | null = null; const UNDO_FALLBACK_MS = 2500; -// Expanded timestamp state (keyed by turn id) -const expandedTimeTurnIds = ref>(new Set()); -function isTimeExpanded(turnId: string): boolean { - return expandedTimeTurnIds.value.has(turnId); -} -function toggleTime(turnId: string): void { - const next = new Set(expandedTimeTurnIds.value); - if (next.has(turnId)) next.delete(turnId); - else next.add(turnId); - expandedTimeTurnIds.value = next; -} -function displayMessageTime(iso: string, turnId: string): string { - if (isTimeExpanded(turnId)) { - const d = new Date(iso); - if (Number.isNaN(d.getTime())) return iso; - const pad2 = (n: number) => String(n).padStart(2, '0'); - return `${d.getFullYear()}-${pad2(d.getMonth() + 1)}-${pad2(d.getDate())} ${pad2(d.getHours())}:${pad2(d.getMinutes())}`; - } - return formatMessageTime(iso, t('conversation.yesterday')); -} - async function onUndo(turn: ChatTurn): Promise { if ( await confirm({ @@ -611,14 +590,7 @@ function isStreamingRenderBlock(turn: ChatTurn, block: { sourceIndex: number }): - + @@ -642,7 +614,7 @@ function isStreamingRenderBlock(turn: ChatTurn, block: { sourceIndex: number }): - +
@@ -660,7 +632,6 @@ function isStreamingRenderBlock(turn: ChatTurn, block: { sourceIndex: number }): @open-agent="emit('openAgent', $event)" /> -
@@ -855,29 +826,7 @@ function isStreamingRenderBlock(turn: ChatTurn, block: { sourceIndex: number }): margin-top: 2px; margin-right: 4px; } -.u-meta .u-time { - display: inline-flex; - align-items: center; - padding: 2px 5px; - background: none; - border: none; - border-radius: var(--radius-sm); - color: var(--muted); - font: inherit; - font-size: var(--text-base); - line-height: 1; - cursor: pointer; - opacity: 0.7; - transition: opacity 0.12s, color 0.12s, background-color 0.12s; - white-space: nowrap; -} -.u-meta .u-time:hover { - opacity: 1; - color: var(--color-accent); - background: var(--hover); -} -.u-meta .u-edit, -.u-meta .u-time { +.u-meta .u-edit { min-height: 22px; box-sizing: border-box; } diff --git a/apps/kimi-web/src/components/chat/CronNotice.vue b/apps/kimi-web/src/components/chat/CronNotice.vue index e4f8b52e5..4b2084e90 100644 --- a/apps/kimi-web/src/components/chat/CronNotice.vue +++ b/apps/kimi-web/src/components/chat/CronNotice.vue @@ -1,19 +1,23 @@ - + embedded inside an assistant turn's blocks — in both cases it takes the + same text + cron data. --> diff --git a/apps/kimi-web/src/components/chat/MessageTime.vue b/apps/kimi-web/src/components/chat/MessageTime.vue new file mode 100644 index 000000000..daa6a0345 --- /dev/null +++ b/apps/kimi-web/src/components/chat/MessageTime.vue @@ -0,0 +1,62 @@ + + + + + + + diff --git a/apps/kimi-web/src/components/chatTurnRendering.ts b/apps/kimi-web/src/components/chatTurnRendering.ts index 01d13025a..f928adf2d 100644 --- a/apps/kimi-web/src/components/chatTurnRendering.ts +++ b/apps/kimi-web/src/components/chatTurnRendering.ts @@ -2,7 +2,7 @@ // Pure turn-rendering helpers: pure functions of their arguments (no Vue // reactivity, no component state). Shared by ChatPane.vue's template and its // stateful copy/edit helpers. -import type { ChatTurn, CronTurnData, TurnBlock } from '../types'; +import type { ChatTurn, TurnBlock } from '../types'; export function formatTokens(n: number): string { if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(1)}M`; @@ -41,8 +41,7 @@ export type AssistantRenderBlock = | { kind: 'thinking'; thinking: string; sourceIndex: number } | { kind: 'text'; text: string; sourceIndex: number } | { kind: 'tool'; tool: ToolStackItem['tool']; sourceIndex: number } - | { kind: 'tool-stack'; tools: ToolStackItem[] } - | { kind: 'cron'; text: string; cron: CronTurnData; sourceIndex: number }; + | { kind: 'tool-stack'; tools: ToolStackItem[] }; export function rendersToolCard(block: Extract): boolean { return !(block.tool.status === 'ok' && block.tool.media); @@ -86,8 +85,6 @@ export function assistantRenderBlocks(turn: ChatTurn): AssistantRenderBlock[] { rendered.push({ kind: 'thinking', thinking: block.thinking, sourceIndex }); } else if (block.kind === 'text') { rendered.push({ kind: 'text', text: block.text, sourceIndex }); - } else if (block.kind === 'cron') { - rendered.push({ kind: 'cron', text: block.text, cron: block.cron, sourceIndex }); } }); diff --git a/apps/kimi-web/src/composables/messagesToTurns.ts b/apps/kimi-web/src/composables/messagesToTurns.ts index 4293cb0be..8d2993a22 100644 --- a/apps/kimi-web/src/composables/messagesToTurns.ts +++ b/apps/kimi-web/src/composables/messagesToTurns.ts @@ -415,10 +415,6 @@ function buildCronTurn(msg: AppMessage, no: number, kind: 'cron_job' | 'cron_mis return { id: msg.id, role: 'cron', no, text, createdAt: msg.createdAt, cron }; } -function buildCronBlock(msg: AppMessage, kind: 'cron_job' | 'cron_missed'): TurnBlock { - const { text, cron } = buildCronData(msg, kind); - return { kind: 'cron', text, cron }; -} /** * Whether a USER-role message should be shown. Mirrors the TUI's @@ -457,12 +453,6 @@ function continuesAssistantGroup(group: Group | null, promptId: string | undefin ); } -/** True while a tool in this group has been called but not yet resolved — i.e. - * the group is mid-tool-call. A cron injection that arrives now is sandwiched - * between the tool use and its result and must not flush the group. */ -function hasRunningTool(group: Group): boolean { - return group.tools.some((t) => t.status === 'running'); -} /** Extract the plan file path from an ExitPlanMode tool result. The approved * output contains `Plan saved to: `; this survives a page reload (unlike @@ -653,19 +643,10 @@ export function messagesToTurns( // User messages flush the pending group and start a new user turn if (msg.role === 'user') { const cronKind = cronOriginKind(msg); - // A cron injection steered into an in-flight turn can land between a - // tool use and its result in the live event stream. It must NOT flush the - // pending assistant group then — flushing would orphan the next - // tool.result, which only folds into a pending group, leaving the tool - // rendered without output. So embed it as a block only while the group - // has an in-flight tool. Otherwise — a cron at a turn boundary, including - // an idle fire on a REST snapshot that carries no prompt ids (where the - // whole transcript shares one group) — flush and render it as its own - // turn so it doesn't merge into the previous answer. - if (cronKind !== undefined && pendingGroup !== null && hasRunningTool(pendingGroup)) { - pendingGroup.blocks.push(buildCronBlock(msg, cronKind)); - continue; - } + // A cron injection always renders as its own standalone turn: agent-core + // buffers steer input while a turn is in flight and only injects it at the + // turn boundary, so the cron message does not land between a tool use and + // its result in practice. flushGroup(); if (cronKind !== undefined) { turns.push(buildCronTurn(msg, no++, cronKind)); diff --git a/apps/kimi-web/src/lib/icons.ts b/apps/kimi-web/src/lib/icons.ts index 3511af539..e41f0b1e4 100644 --- a/apps/kimi-web/src/lib/icons.ts +++ b/apps/kimi-web/src/lib/icons.ts @@ -26,6 +26,9 @@ import RiArrowRightLine from '~icons/ri/arrow-right-line'; import RiArrowRightSLine from '~icons/ri/arrow-right-s-line'; import RiArrowUpLine from '~icons/ri/arrow-up-line'; import RiBracesLine from '~icons/ri/braces-line'; +import RiCalendarCloseLine from '~icons/ri/calendar-close-line'; +import RiCalendarScheduleLine from '~icons/ri/calendar-schedule-line'; +import RiCalendarTodoLine from '~icons/ri/calendar-todo-line'; import RiChatNewLine from '~icons/ri/chat-new-line'; import RiCheckLine from '~icons/ri/check-line'; import RiCloseLine from '~icons/ri/close-line'; @@ -84,6 +87,9 @@ import RawArrowRightLine from '~icons/ri/arrow-right-line?raw'; import RawArrowRightSLine from '~icons/ri/arrow-right-s-line?raw'; import RawArrowUpLine from '~icons/ri/arrow-up-line?raw'; import RawBracesLine from '~icons/ri/braces-line?raw'; +import RawCalendarCloseLine from '~icons/ri/calendar-close-line?raw'; +import RawCalendarScheduleLine from '~icons/ri/calendar-schedule-line?raw'; +import RawCalendarTodoLine from '~icons/ri/calendar-todo-line?raw'; import RawChatNewLine from '~icons/ri/chat-new-line?raw'; import RawCheckLine from '~icons/ri/check-line?raw'; import RawCloseLine from '~icons/ri/close-line?raw'; @@ -136,6 +142,9 @@ import RawUserLine from '~icons/ri/user-line?raw'; export type IconName = | 'plus' | 'chat-new' + | 'calendar-close' + | 'calendar-schedule' + | 'calendar-todo' | 'close' | 'check' | 'search' @@ -212,6 +221,9 @@ function entry(component: Component, svg: string): IconEntry { export const ICONS: Record = { plus: entry(RiAddLine, RawAddLine), 'chat-new': entry(RiChatNewLine, RawChatNewLine), + 'calendar-close': entry(RiCalendarCloseLine, RawCalendarCloseLine), + 'calendar-schedule': entry(RiCalendarScheduleLine, RawCalendarScheduleLine), + 'calendar-todo': entry(RiCalendarTodoLine, RawCalendarTodoLine), close: entry(RiCloseLine, RawCloseLine), check: entry(RiCheckLine, RawCheckLine), search: entry(RiSearchLine, RawSearchLine), @@ -353,6 +365,9 @@ export const ICON_GROUPS: ReadonlyArray 'check-list', 'bolt', 'git-pull-request', + 'calendar-schedule', + 'calendar-todo', + 'calendar-close', ], ], ['Communication', ['message', 'mail', 'user']], diff --git a/apps/kimi-web/src/lib/toolMeta.ts b/apps/kimi-web/src/lib/toolMeta.ts index 0c7c057e6..6ecfd4c00 100644 --- a/apps/kimi-web/src/lib/toolMeta.ts +++ b/apps/kimi-web/src/lib/toolMeta.ts @@ -93,6 +93,10 @@ const TOOL_GLYPH: Record = { task: 'sparkles', agentswarm: 'git-pull-request', askuserquestion: 'help-circle', + // Cron scheduling tools share a calendar motif: schedule / list / cancel. + croncreate: 'calendar-schedule', + cronlist: 'calendar-todo', + crondelete: 'calendar-close', }; export function toolGlyph(name: string): string { diff --git a/apps/kimi-web/src/types.ts b/apps/kimi-web/src/types.ts index e40b58062..d98b3dbcf 100644 --- a/apps/kimi-web/src/types.ts +++ b/apps/kimi-web/src/types.ts @@ -229,8 +229,7 @@ export interface CronTurnData { export type TurnBlock = | { kind: 'text'; text: string } | { kind: 'thinking'; thinking: string } - | { kind: 'tool'; tool: ToolCall } - | { kind: 'cron'; text: string; cron: CronTurnData }; + | { kind: 'tool'; tool: ToolCall }; export interface ChatTurn { id: string; diff --git a/apps/kimi-web/test/turn-logic.test.ts b/apps/kimi-web/test/turn-logic.test.ts index b28d93b5d..07b09bae7 100644 --- a/apps/kimi-web/test/turn-logic.test.ts +++ b/apps/kimi-web/test/turn-logic.test.ts @@ -320,56 +320,6 @@ describe('messagesToTurns cron', () => { expect(turns).toHaveLength(1); }); - it('embeds an in-turn cron injection as a block and keeps folding the following tool result', () => { - const envelope = - '\n' + - '\nCheck BTC\n\n'; - const turns = messagesToTurns( - [ - message('u1', 'user', [{ type: 'text', text: 'hi' }]), - message('a1', 'assistant', [ - { - type: 'toolUse', - toolCallId: 'tc1', - toolName: 'FetchURL', - input: { url: 'https://example.com' }, - }, - ]), - message('c1', 'user', [{ type: 'text', text: envelope }], { - metadata: { - origin: { - kind: 'cron_job', - jobId: 'j', - cron: '* * * * *', - recurring: true, - coalescedCount: 1, - stale: false, - }, - }, - }), - message('t1', 'tool', [ - { type: 'toolResult', toolCallId: 'tc1', output: 'the price is 62k' }, - ]), - ], - [], - ); - - // user turn + one assistant turn (tool + embedded cron block + result). - // No standalone cron turn, and the tool result is preserved. - expect(turns).toHaveLength(2); - const assistant = turns[1]!; - expect(assistant.role).toBe('assistant'); - expect(assistant.tools?.[0]).toMatchObject({ - id: 'tc1', - status: 'ok', - output: ['the price is 62k'], - }); - expect(assistant.blocks?.find((b) => b.kind === 'cron')).toMatchObject({ - kind: 'cron', - text: 'Check BTC', - cron: { jobId: 'j' }, - }); - }); it('flushes an idle cron fire as its own turn even when no prompt ids are present', () => { const envelope = From 0cc9831a2f79d93903259bd3353e746abac01b67 Mon Sep 17 00:00:00 2001 From: qer Date: Wed, 8 Jul 2026 14:53:29 +0800 Subject: [PATCH 10/24] fix(web): composer model switch also updates global default model (#1491) * fix(web): composer model switch also updates global default model The composer model switcher still switches the active session's model via POST /sessions/{id}/profile (awaited, so the model pill reflects the result), and additionally fires POST /api/v1/config with { default_model } as a fire-and-forget side effect so new sessions inherit the chosen default. The config request is skipped when the model already matches the current default. * fix(web): route ModelPicker overlay selection through the default-model update The overlay opened from the composer's "More models" row (and /model) is a continuation of the same switch flow, so its selection now also bumps the global default model instead of only switching the active session. * fix(web): only persist the default model after a confirmed session switch setModel now returns whether the switch was accepted (true for the draft path), so the composer flow no longer writes a stale or invalid model alias into the global config when the session-level switch failed and rolled back. --- .../web-composer-default-model-switch.md | 5 ++++ apps/kimi-web/src/App.vue | 24 +++++++++++++++++-- .../client/useModelProviderState.ts | 11 ++++++--- 3 files changed, 35 insertions(+), 5 deletions(-) create mode 100644 .changeset/web-composer-default-model-switch.md diff --git a/.changeset/web-composer-default-model-switch.md b/.changeset/web-composer-default-model-switch.md new file mode 100644 index 000000000..e35a51784 --- /dev/null +++ b/.changeset/web-composer-default-model-switch.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +web: The composer model switcher switches the active session's model as before and additionally bumps the global default model, so new sessions inherit the choice. diff --git a/apps/kimi-web/src/App.vue b/apps/kimi-web/src/App.vue index 6c82df986..22f645ef2 100644 --- a/apps/kimi-web/src/App.vue +++ b/apps/kimi-web/src/App.vue @@ -338,7 +338,27 @@ function openLogin(): void { async function handleSelectModel(modelId: string): Promise { showModelPicker.value = false; - await client.setModel(modelId); + // Same semantics as the composer dropdown rows: the overlay is just the + // "more models" continuation of the same flow, so it must also bump the + // global default (see handleComposerSelectModel). + await handleComposerSelectModel(modelId); +} + +async function handleComposerSelectModel(modelId: string): Promise { + // Primary action: switch the active session's model via POST /sessions/{id}/profile + // (same as the model picker overlay). Awaited so the model pill reflects the + // result and failures surface. In the onboarding draft this just stores the + // pick for the first session. + const switched = await client.setModel(modelId); + + // Side effect: also bump the daemon-wide default model via POST /config so + // new sessions inherit the choice. Fire-and-forget — it must not block the UI + // or mask the session switch. Only after a confirmed switch (a stale/invalid + // alias must not become the global default), and skip when it already + // matches the default. + if (switched && modelId !== client.defaultModel.value) { + void client.updateConfig({ defaultModel: modelId }); + } } async function handleAddProvider(input: { type: string; apiKey?: string; baseUrl?: string; defaultModel?: string }): Promise { @@ -759,7 +779,7 @@ function openPr(url: string): void { @archive-session="(id) => client.archiveSession(id)" @compact="client.compact()" @pick-model="openModelPicker()" - @select-model="client.setModel($event)" + @select-model="handleComposerSelectModel($event)" @open-file="openFilePreview($event)" @open-media="openMediaPreview($event)" @open-thinking="openThinkingPanel($event)" diff --git a/apps/kimi-web/src/composables/client/useModelProviderState.ts b/apps/kimi-web/src/composables/client/useModelProviderState.ts index 6eb852e1f..fcd25624f 100644 --- a/apps/kimi-web/src/composables/client/useModelProviderState.ts +++ b/apps/kimi-web/src/composables/client/useModelProviderState.ts @@ -181,8 +181,12 @@ export function useModelProviderState( * can return model '', so the authoritative current model comes from * GET /sessions/{id}/status, which we re-read right after. Optimistically show * the chosen id meanwhile. Never crashes. + * + * Returns whether the switch was accepted (true for the draft path too), so + * callers can gate follow-up persistence (e.g. bumping the global default) on + * a confirmed switch — errors are surfaced here, not thrown. */ - async function setModel(modelId: string): Promise { + async function setModel(modelId: string): Promise { const sid = rawState.activeSessionId; const nextThinking = coerceThinkingForModel(modelById(modelId), rawState.thinking); const prevThinking = rawState.thinking; @@ -191,7 +195,7 @@ export function useModelProviderState( // Remember the pick — startSessionAndSendPrompt applies it at create time. draftModel.value = modelId; applyThinkingLevel(nextThinking); - return; + return true; } // Optimistic: show the chosen model immediately, but remember the previous // one so we can roll back if the switch never reaches the daemon. @@ -217,12 +221,13 @@ export function useModelProviderState( saveThinkingToStorage(prevThinking); } pushOperationFailure('setModel', err, { sessionId: sid }); - return; + return false; } // refreshSessionStatus folds the authoritative current model from /status // back into the session (the profile echo can return ''). Best-effort: a // failure here does not mean the switch failed, so it must not roll back. await refreshSessionStatus(sid); + return true; } /** Toggle whether a model is starred (favorited) in the model picker. */ From b0809ddac833d8d920d95187f7ef64f97bafdbc6 Mon Sep 17 00:00:00 2001 From: qer Date: Wed, 8 Jul 2026 15:03:30 +0800 Subject: [PATCH 11/24] feat(web): prefix skill slash commands with skill: to distinguish them from built-in commands (#1492) --- .changeset/web-skill-slash-prefix.md | 5 +++++ apps/kimi-web/src/App.vue | 15 ++++++++------ .../kimi-web/src/components/chat/Composer.vue | 9 ++++++--- apps/kimi-web/src/lib/slashCommands.ts | 20 ++++++++++++++++--- apps/kimi-web/test/slash-menu.test.ts | 20 ++++++++++++++++--- 5 files changed, 54 insertions(+), 15 deletions(-) create mode 100644 .changeset/web-skill-slash-prefix.md diff --git a/.changeset/web-skill-slash-prefix.md b/.changeset/web-skill-slash-prefix.md new file mode 100644 index 000000000..5ba22488b --- /dev/null +++ b/.changeset/web-skill-slash-prefix.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +web: Show session skills in the slash menu as `/skill:` so they are distinguishable from built-in commands; typing the bare skill name still works. diff --git a/apps/kimi-web/src/App.vue b/apps/kimi-web/src/App.vue index 22f645ef2..eb9131b48 100644 --- a/apps/kimi-web/src/App.vue +++ b/apps/kimi-web/src/App.vue @@ -39,6 +39,7 @@ import ServerAuthDialog from './components/ServerAuthDialog.vue'; import { initServerAuth, onAuthRequired } from './api/daemon/serverAuth'; import type { AppConfig, ThinkingLevel } from './api/types'; import { coerceThinkingForModel, commitLevel, segmentsFor } from './lib/modelThinking'; +import { stripSkillPrefix } from './lib/slashCommands'; import Button from './components/ui/Button.vue'; import IconButton from './components/ui/IconButton.vue'; import Icon from './components/ui/Icon.vue'; @@ -506,13 +507,15 @@ function handleCommand(cmd: string): void { break; default: { // Not a built-in command → treat it as a session skill activation - // (the user picked `/` from the menu, or typed `/ args`). - // The daemon answers an unknown name with skill.not_found, surfaced as a - // warning, so a stray slash is harmless. With no active session, create - // one first (same path as the first prompt) so the activation isn't - // silently dropped on the new-session screen. + // (the user picked `/skill:` from the menu, or typed + // `/ args`). Strip the `skill:` display prefix — the REST API + // takes the bare skill name. The daemon answers an unknown name with + // skill.not_found, surfaced as a warning, so a stray slash is harmless. + // With no active session, create one first (same path as the first + // prompt) so the activation isn't silently dropped on the new-session + // screen. const space = cmd.indexOf(' '); - const name = (space === -1 ? cmd : cmd.slice(0, space)).slice(1); + const name = stripSkillPrefix((space === -1 ? cmd : cmd.slice(0, space)).slice(1)); const args = space === -1 ? undefined : cmd.slice(space + 1).trim() || undefined; if (!name) break; if (!client.activeSessionId.value && client.activeWorkspaceId.value) { diff --git a/apps/kimi-web/src/components/chat/Composer.vue b/apps/kimi-web/src/components/chat/Composer.vue index 4f591349a..e8069b0ec 100644 --- a/apps/kimi-web/src/components/chat/Composer.vue +++ b/apps/kimi-web/src/components/chat/Composer.vue @@ -4,7 +4,7 @@ import { computed, nextTick, onMounted, onUnmounted, ref, watch } from 'vue'; import { useI18n } from 'vue-i18n'; import SlashMenu from './SlashMenu.vue'; import MentionMenu from './MentionMenu.vue'; -import { buildSlashItems, parseSlash } from '../../lib/slashCommands'; +import { buildSlashItems, parseSlash, SKILL_COMMAND_PREFIX } from '../../lib/slashCommands'; import type { FileItem } from './MentionMenu.vue'; import type { ActivationBadges, ConversationStatus, PermissionMode, QueuedPromptView } from '../../types'; import type { AppModel, AppSkill, ThinkingLevel } from '../../api/types'; @@ -304,11 +304,14 @@ function handleSubmit(): void { // If it's a known slash command, keep the optional tail as command input // instead of submitting it as normal chat text. This covers `/goal `, // `/swarm `, `/btw `, slash skills with args, and bare - // commands such as `/model`. + // commands such as `/model`. A hand-typed bare skill name (`/deploy`) also + // resolves to its prefixed menu entry (`/skill:deploy`), mirroring the TUI. if (trimmed) { const parsed = parseSlash(trimmed); const known = parsed - ? buildSlashItems(props.skills).some((item) => item.name === parsed.cmd) + ? buildSlashItems(props.skills).some( + (item) => item.name === parsed.cmd || item.name === `/${SKILL_COMMAND_PREFIX}${parsed.cmd.slice(1)}`, + ) : false; if (parsed && known) { text.value = ''; diff --git a/apps/kimi-web/src/lib/slashCommands.ts b/apps/kimi-web/src/lib/slashCommands.ts index 8eb24ed75..8977f3e24 100644 --- a/apps/kimi-web/src/lib/slashCommands.ts +++ b/apps/kimi-web/src/lib/slashCommands.ts @@ -65,16 +65,30 @@ export function parseSlash(input: string): { cmd: string; arg: string } | null { }; } +/** The prefix marking a slash item as a skill activation (`/skill:`). */ +export const SKILL_COMMAND_PREFIX = 'skill:'; + +/** + * Strip the `skill:` prefix from a slash-command name (with or without the + * leading `/`), returning the bare skill name. Non-prefixed input is returned + * unchanged. + */ +export function stripSkillPrefix(name: string): string { + return name.startsWith(SKILL_COMMAND_PREFIX) ? name.slice(SKILL_COMMAND_PREFIX.length) : name; +} + /** * Build the full slash-item list: built-in commands followed by the session's - * skills (each shown as `/`). Skills carry their raw description and + * skills. Non-builtin skills are shown as `/skill:` so the user can + * tell them apart from built-in commands (mirroring the TUI); builtin-sourced + * skills keep the bare `/`. Skills carry their raw description and * an `isSkill` flag so the caller knows to activate rather than run a command. */ export function buildSlashItems( - skills: ReadonlyArray<{ name: string; description: string }> = [], + skills: ReadonlyArray<{ name: string; description: string; source?: string }> = [], ): SlashCommand[] { const skillItems: SlashCommand[] = skills.map((s) => ({ - name: `/${s.name}`, + name: s.source === 'builtin' ? `/${s.name}` : `/${SKILL_COMMAND_PREFIX}${s.name}`, desc: s.description, isSkill: true, // Keep the selected skill in the composer so arguments can be appended. diff --git a/apps/kimi-web/test/slash-menu.test.ts b/apps/kimi-web/test/slash-menu.test.ts index 9b1d8657b..c270eafb4 100644 --- a/apps/kimi-web/test/slash-menu.test.ts +++ b/apps/kimi-web/test/slash-menu.test.ts @@ -74,11 +74,25 @@ describe('useSlashMenu — update', () => { expect(slash.open.value).toBe(false); }); - it('includes session skills as /', () => { - const { slash } = setup('/', [{ name: 'deploy', description: 'deploy stuff' } as AppSkill]); + it('includes session skills as /skill:', () => { + const { slash } = setup('/', [{ name: 'deploy', description: 'deploy stuff', source: 'project' } as AppSkill]); slash.update(); const names = slash.items.value.map((i) => i.name); - expect(names).toContain('/deploy'); + expect(names).toContain('/skill:deploy'); + }); + + it('keeps builtin-sourced skills unprefixed', () => { + const { slash } = setup('/', [{ name: 'update-config', description: 'edit config', source: 'builtin' } as AppSkill]); + slash.update(); + const names = slash.items.value.map((i) => i.name); + expect(names).toContain('/update-config'); + expect(names).not.toContain('/skill:update-config'); + }); + + it('matches a prefixed skill when filtering by its bare name', () => { + const { slash } = setup('/depl', [{ name: 'deploy', description: 'deploy stuff', source: 'project' } as AppSkill]); + slash.update(); + expect(slash.items.value.map((i) => i.name)).toContain('/skill:deploy'); }); }); From 67b2147d8e8832b1901b669bfec24e4794eafa95 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 8 Jul 2026 15:37:29 +0800 Subject: [PATCH 12/24] ci: release packages (#1468) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- .changeset/add-vercel-plugin.md | 5 --- .changeset/count-created-goal-turn.md | 5 --- .changeset/fix-headless-exit-code.md | 5 --- .changeset/fix-hooks-windows-console-popup.md | 5 --- .changeset/fix-web-ws-reconnect-toast.md | 5 --- .changeset/forbid-model-goal-pauses.md | 5 --- .changeset/goal-blocked-audit-prompt.md | 5 --- .changeset/image-compression-improvements.md | 6 ---- .../migrate-web-icons-to-unplugin-icons.md | 5 --- .../select-tools-discard-on-compaction.md | 5 --- .../structured-output-provider-format.md | 5 --- .../web-composer-default-model-switch.md | 5 --- .changeset/web-confirm-dialog-enter-key.md | 5 --- .changeset/web-cron-redesign.md | 5 --- .changeset/web-skill-slash-prefix.md | 5 --- apps/kimi-code/CHANGELOG.md | 32 +++++++++++++++++++ apps/kimi-code/package.json | 2 +- packages/kosong/CHANGELOG.md | 6 ++++ packages/kosong/package.json | 2 +- packages/node-sdk/CHANGELOG.md | 6 ++++ packages/node-sdk/package.json | 2 +- 21 files changed, 47 insertions(+), 79 deletions(-) delete mode 100644 .changeset/add-vercel-plugin.md delete mode 100644 .changeset/count-created-goal-turn.md delete mode 100644 .changeset/fix-headless-exit-code.md delete mode 100644 .changeset/fix-hooks-windows-console-popup.md delete mode 100644 .changeset/fix-web-ws-reconnect-toast.md delete mode 100644 .changeset/forbid-model-goal-pauses.md delete mode 100644 .changeset/goal-blocked-audit-prompt.md delete mode 100644 .changeset/image-compression-improvements.md delete mode 100644 .changeset/migrate-web-icons-to-unplugin-icons.md delete mode 100644 .changeset/select-tools-discard-on-compaction.md delete mode 100644 .changeset/structured-output-provider-format.md delete mode 100644 .changeset/web-composer-default-model-switch.md delete mode 100644 .changeset/web-confirm-dialog-enter-key.md delete mode 100644 .changeset/web-cron-redesign.md delete mode 100644 .changeset/web-skill-slash-prefix.md diff --git a/.changeset/add-vercel-plugin.md b/.changeset/add-vercel-plugin.md deleted file mode 100644 index 8fd208266..000000000 --- a/.changeset/add-vercel-plugin.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Add the Vercel plugin to the bundled plugin marketplace. Run /plugins and select Vercel Plugin to install it. diff --git a/.changeset/count-created-goal-turn.md b/.changeset/count-created-goal-turn.md deleted file mode 100644 index 8e9ba8ea2..000000000 --- a/.changeset/count-created-goal-turn.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Count the turn that starts an autonomous goal toward its goal turn usage. diff --git a/.changeset/fix-headless-exit-code.md b/.changeset/fix-headless-exit-code.md deleted file mode 100644 index f2de03dd8..000000000 --- a/.changeset/fix-headless-exit-code.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Fix `kimi -p` runs exiting with code 0 when a turn fails. diff --git a/.changeset/fix-hooks-windows-console-popup.md b/.changeset/fix-hooks-windows-console-popup.md deleted file mode 100644 index 46f8b18c7..000000000 --- a/.changeset/fix-hooks-windows-console-popup.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Fix console windows flashing on Windows each time a hook runs. diff --git a/.changeset/fix-web-ws-reconnect-toast.md b/.changeset/fix-web-ws-reconnect-toast.md deleted file mode 100644 index dbf190b92..000000000 --- a/.changeset/fix-web-ws-reconnect-toast.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -web: Fix the connection error toast lingering after the WebSocket reconnects when returning from the background. diff --git a/.changeset/forbid-model-goal-pauses.md b/.changeset/forbid-model-goal-pauses.md deleted file mode 100644 index 3f56a747b..000000000 --- a/.changeset/forbid-model-goal-pauses.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Prevent autonomous goals from being paused by model-reported status updates. diff --git a/.changeset/goal-blocked-audit-prompt.md b/.changeset/goal-blocked-audit-prompt.md deleted file mode 100644 index 4f498ddb1..000000000 --- a/.changeset/goal-blocked-audit-prompt.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Tighten goal-mode guidance for blocked and complete status updates. diff --git a/.changeset/image-compression-improvements.md b/.changeset/image-compression-improvements.md deleted file mode 100644 index 70e762223..000000000 --- a/.changeset/image-compression-improvements.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch -"@moonshot-ai/kimi-code-sdk": patch ---- - -Raise the image downscale cap from 2000px to 3000px, and fix swapped width/height for EXIF-rotated (portrait) photos in compression captions and media read notes so region readback coordinates map correctly. diff --git a/.changeset/migrate-web-icons-to-unplugin-icons.md b/.changeset/migrate-web-icons-to-unplugin-icons.md deleted file mode 100644 index e15b783bd..000000000 --- a/.changeset/migrate-web-icons-to-unplugin-icons.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -web: Compile icons at build time so the bundled web UI only carries the icons it renders. diff --git a/.changeset/select-tools-discard-on-compaction.md b/.changeset/select-tools-discard-on-compaction.md deleted file mode 100644 index 70bda32d5..000000000 --- a/.changeset/select-tools-discard-on-compaction.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Progressive tool disclosure (`select_tools`, experimental): compaction now discards the loaded tool schemas instead of re-injecting them. After a compaction the boundary announcement re-lists every loadable tool name and the model re-selects what it still needs; a from-memory call to a no-longer-loaded tool is rejected with guidance to select it first. This keeps the post-compaction context at its minimal users+summary floor and removes the schema-rebuild budget heuristics. No effect unless the `tool-select` experimental flag and a `select_tools`-capable model are active. diff --git a/.changeset/structured-output-provider-format.md b/.changeset/structured-output-provider-format.md deleted file mode 100644 index 625fb130a..000000000 --- a/.changeset/structured-output-provider-format.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kosong": patch ---- - -Add provider-level structured response format support. diff --git a/.changeset/web-composer-default-model-switch.md b/.changeset/web-composer-default-model-switch.md deleted file mode 100644 index e35a51784..000000000 --- a/.changeset/web-composer-default-model-switch.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -web: The composer model switcher switches the active session's model as before and additionally bumps the global default model, so new sessions inherit the choice. diff --git a/.changeset/web-confirm-dialog-enter-key.md b/.changeset/web-confirm-dialog-enter-key.md deleted file mode 100644 index c74cc4e05..000000000 --- a/.changeset/web-confirm-dialog-enter-key.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -web: Press Enter to confirm in archive and other confirmation dialogs. diff --git a/.changeset/web-cron-redesign.md b/.changeset/web-cron-redesign.md deleted file mode 100644 index d338b601c..000000000 --- a/.changeset/web-cron-redesign.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -web: Redesign the scheduled reminder UI. diff --git a/.changeset/web-skill-slash-prefix.md b/.changeset/web-skill-slash-prefix.md deleted file mode 100644 index 5ba22488b..000000000 --- a/.changeset/web-skill-slash-prefix.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -web: Show session skills in the slash menu as `/skill:` so they are distinguishable from built-in commands; typing the bare skill name still works. diff --git a/apps/kimi-code/CHANGELOG.md b/apps/kimi-code/CHANGELOG.md index 44e5829a8..4cb493e84 100644 --- a/apps/kimi-code/CHANGELOG.md +++ b/apps/kimi-code/CHANGELOG.md @@ -1,5 +1,37 @@ # @moonshot-ai/kimi-code +## 0.23.2 + +### Patch Changes + +- [#1489](https://github.com/MoonshotAI/kimi-code/pull/1489) [`2206d21`](https://github.com/MoonshotAI/kimi-code/commit/2206d21327129aa2331b6b159cfce61110b7f94f) Thanks [@wbxl2000](https://github.com/wbxl2000)! - Add the Vercel plugin to the bundled plugin marketplace. Run /plugins and select Vercel Plugin to install it. + +- [#1477](https://github.com/MoonshotAI/kimi-code/pull/1477) [`150206a`](https://github.com/MoonshotAI/kimi-code/commit/150206a6f7027879df954e26736b4baa5d336235) Thanks [@chengluyu](https://github.com/chengluyu)! - Count the turn that starts an autonomous goal toward its goal turn usage. + +- [#1483](https://github.com/MoonshotAI/kimi-code/pull/1483) [`f30781b`](https://github.com/MoonshotAI/kimi-code/commit/f30781bb273321f3e3bbb548a9d0724ab6299fc6) Thanks [@wbxl2000](https://github.com/wbxl2000)! - Fix `kimi -p` runs exiting with code 0 when a turn fails. + +- [#1466](https://github.com/MoonshotAI/kimi-code/pull/1466) [`063bce2`](https://github.com/MoonshotAI/kimi-code/commit/063bce2a2f52601abaa0d13173ab88371cbbe9ae) Thanks [@liruifengv](https://github.com/liruifengv)! - Fix console windows flashing on Windows each time a hook runs. + +- [#1474](https://github.com/MoonshotAI/kimi-code/pull/1474) [`11c6a37`](https://github.com/MoonshotAI/kimi-code/commit/11c6a37ce030f8e64de5c810da07ec7fab3b0615) Thanks [@wbxl2000](https://github.com/wbxl2000)! - web: Fix the connection error toast lingering after the WebSocket reconnects when returning from the background. + +- [#1476](https://github.com/MoonshotAI/kimi-code/pull/1476) [`d1a964f`](https://github.com/MoonshotAI/kimi-code/commit/d1a964fba9b3dca902ea6f81bacaccc839955c03) Thanks [@chengluyu](https://github.com/chengluyu)! - Prevent autonomous goals from being paused by model-reported status updates. + +- [#1481](https://github.com/MoonshotAI/kimi-code/pull/1481) [`1317000`](https://github.com/MoonshotAI/kimi-code/commit/131700097a732b97b3d17c5e2efa1c5a44b013ef) Thanks [@chengluyu](https://github.com/chengluyu)! - Tighten goal-mode guidance for blocked and complete status updates. + +- [#1460](https://github.com/MoonshotAI/kimi-code/pull/1460) [`474ce28`](https://github.com/MoonshotAI/kimi-code/commit/474ce289dd39aa42d1a77a9a2e15531aee49aa15) Thanks [@RealKai42](https://github.com/RealKai42)! - Raise the image downscale cap from 2000px to 3000px, and fix swapped width/height for EXIF-rotated (portrait) photos in compression captions and media read notes so region readback coordinates map correctly. + +- [#1467](https://github.com/MoonshotAI/kimi-code/pull/1467) [`ee38545`](https://github.com/MoonshotAI/kimi-code/commit/ee385456d0eda380fec067db92c025462db13f5a) Thanks [@liruifengv](https://github.com/liruifengv)! - web: Compile icons at build time so the bundled web UI only carries the icons it renders. + +- [#1471](https://github.com/MoonshotAI/kimi-code/pull/1471) [`9b76e5b`](https://github.com/MoonshotAI/kimi-code/commit/9b76e5bff631cceaeecb2b0cbc096533c5fdc8cc) Thanks [@starquakee](https://github.com/starquakee)! - Progressive tool disclosure (`select_tools`, experimental): compaction now discards the loaded tool schemas instead of re-injecting them. After a compaction the boundary announcement re-lists every loadable tool name and the model re-selects what it still needs; a from-memory call to a no-longer-loaded tool is rejected with guidance to select it first. This keeps the post-compaction context at its minimal users+summary floor and removes the schema-rebuild budget heuristics. No effect unless the `tool-select` experimental flag and a `select_tools`-capable model are active. + +- [#1491](https://github.com/MoonshotAI/kimi-code/pull/1491) [`0cc9831`](https://github.com/MoonshotAI/kimi-code/commit/0cc9831a2f79d93903259bd3353e746abac01b67) Thanks [@wbxl2000](https://github.com/wbxl2000)! - web: The composer model switcher switches the active session's model as before and additionally bumps the global default model, so new sessions inherit the choice. + +- [#1490](https://github.com/MoonshotAI/kimi-code/pull/1490) [`b30a45e`](https://github.com/MoonshotAI/kimi-code/commit/b30a45efecfa5ece4f4f10f2c5403ba097e7690b) Thanks [@wbxl2000](https://github.com/wbxl2000)! - web: Press Enter to confirm in archive and other confirmation dialogs. + +- [#1480](https://github.com/MoonshotAI/kimi-code/pull/1480) [`2ad0120`](https://github.com/MoonshotAI/kimi-code/commit/2ad0120c2a5c8383892e4da1ee7c6853926ed365) Thanks [@liruifengv](https://github.com/liruifengv)! - web: Redesign the scheduled reminder UI. + +- [#1492](https://github.com/MoonshotAI/kimi-code/pull/1492) [`b0809dd`](https://github.com/MoonshotAI/kimi-code/commit/b0809ddac833d8d920d95187f7ef64f97bafdbc6) Thanks [@wbxl2000](https://github.com/wbxl2000)! - web: Show session skills in the slash menu as `/skill:` so they are distinguishable from built-in commands; typing the bare skill name still works. + ## 0.23.1 ### Patch Changes diff --git a/apps/kimi-code/package.json b/apps/kimi-code/package.json index 3d1e8916b..9512d71bb 100644 --- a/apps/kimi-code/package.json +++ b/apps/kimi-code/package.json @@ -1,6 +1,6 @@ { "name": "@moonshot-ai/kimi-code", - "version": "0.23.1", + "version": "0.23.2", "description": "The Starting Point for Next-Gen Agents", "license": "MIT", "author": "Moonshot AI", diff --git a/packages/kosong/CHANGELOG.md b/packages/kosong/CHANGELOG.md index 3ea8fce13..9d71fb022 100644 --- a/packages/kosong/CHANGELOG.md +++ b/packages/kosong/CHANGELOG.md @@ -1,5 +1,11 @@ # @moonshot-ai/kosong +## 0.5.3 + +### Patch Changes + +- [#1397](https://github.com/MoonshotAI/kimi-code/pull/1397) [`6c9abe8`](https://github.com/MoonshotAI/kimi-code/commit/6c9abe8cf7765b489ca50a2bb0f9b829fd680a51) Thanks [@kermanx](https://github.com/kermanx)! - Add provider-level structured response format support. + ## 0.5.2 ### Patch Changes diff --git a/packages/kosong/package.json b/packages/kosong/package.json index 16d09e9db..1c7e19223 100644 --- a/packages/kosong/package.json +++ b/packages/kosong/package.json @@ -1,6 +1,6 @@ { "name": "@moonshot-ai/kosong", - "version": "0.5.2", + "version": "0.5.3", "private": true, "description": "The LLM abstraction layer for modern AI agent applications", "license": "MIT", diff --git a/packages/node-sdk/CHANGELOG.md b/packages/node-sdk/CHANGELOG.md index 08bf9779a..0ad65961c 100644 --- a/packages/node-sdk/CHANGELOG.md +++ b/packages/node-sdk/CHANGELOG.md @@ -1,5 +1,11 @@ # @moonshot-ai/kimi-code-sdk +## 0.13.1 + +### Patch Changes + +- [#1460](https://github.com/MoonshotAI/kimi-code/pull/1460) [`474ce28`](https://github.com/MoonshotAI/kimi-code/commit/474ce289dd39aa42d1a77a9a2e15531aee49aa15) Thanks [@RealKai42](https://github.com/RealKai42)! - Raise the image downscale cap from 2000px to 3000px, and fix swapped width/height for EXIF-rotated (portrait) photos in compression captions and media read notes so region readback coordinates map correctly. + ## 0.13.0 ### Minor Changes diff --git a/packages/node-sdk/package.json b/packages/node-sdk/package.json index e9343b292..ac9877d24 100644 --- a/packages/node-sdk/package.json +++ b/packages/node-sdk/package.json @@ -1,6 +1,6 @@ { "name": "@moonshot-ai/kimi-code-sdk", - "version": "0.13.0", + "version": "0.13.1", "private": true, "description": "TypeScript SDK for the Kimi Code Agent", "license": "MIT", From 2394d013bc8b01a031af95f2927b4ac6647971e0 Mon Sep 17 00:00:00 2001 From: qer Date: Wed, 8 Jul 2026 17:02:00 +0800 Subject: [PATCH 13/24] docs(changelog): sync 0.23.2 from apps/kimi-code/CHANGELOG.md (#1496) --- docs/en/release-notes/changelog.md | 28 ++++++++++++++++++++++++++++ docs/zh/release-notes/changelog.md | 28 ++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/docs/en/release-notes/changelog.md b/docs/en/release-notes/changelog.md index a751feccf..7b98cd3f5 100644 --- a/docs/en/release-notes/changelog.md +++ b/docs/en/release-notes/changelog.md @@ -6,6 +6,34 @@ outline: 2 This page documents the changes in each Kimi Code CLI release. +## 0.23.2 (2026-07-08) + +### Features + +- Add the Vercel plugin to the bundled plugin marketplace. Run `/plugins` and select Vercel Plugin to install it. + +### Bug Fixes + +- Fix `kimi -p` runs exiting with code 0 when a turn fails. +- Prevent autonomous goals from being paused by model-reported status updates. +- Count the turn that starts an autonomous goal toward its turn budget. +- Raise the image downscale cap from 2000px to 3000px, and fix swapped width/height for EXIF-rotated (portrait) photos in compression captions and media read notes so region readback coordinates map correctly. +- web: Fix the connection error toast lingering after the WebSocket reconnects when returning from the background. +- Fix console windows flashing on Windows each time a hook runs. + +### Polish + +- web: Redesign the scheduled reminder UI. +- web: Show session skills in the slash menu as `/skill:` so they are distinguishable from built-in commands; typing the bare skill name still works. +- web: The composer model switcher switches the active session's model as before and additionally bumps the global default model, so new sessions inherit the choice. +- web: Press Enter to confirm in archive and other confirmation dialogs. +- Tighten goal-mode guidance for blocked and complete status updates. +- Progressive tool disclosure (`select_tools`, experimental): compaction now discards the loaded tool schemas instead of re-injecting them, and the model re-selects the tools it still needs afterward. A from-memory call to a no-longer-loaded tool is rejected with guidance to select it first. No effect unless the `tool-select` experimental flag and a `select_tools`-capable model are active. + +### Refactors + +- web: Compile icons at build time so the bundled web UI only carries the icons it renders. + ## 0.23.1 (2026-07-07) ### Bug Fixes diff --git a/docs/zh/release-notes/changelog.md b/docs/zh/release-notes/changelog.md index 89124bdac..bce292cfc 100644 --- a/docs/zh/release-notes/changelog.md +++ b/docs/zh/release-notes/changelog.md @@ -6,6 +6,34 @@ outline: 2 本页记录 Kimi Code CLI 每个版本的变更内容。 +## 0.23.2（2026-07-08） + +### 新功能 + +- 内置插件市场新增 Vercel 插件，运行 `/plugins` 并选择 Vercel Plugin 即可安装。 + +### 修复 + +- 修复 `kimi -p` 在轮次失败时仍以退出码 0 退出的问题。 +- 修复自主目标会被模型上报的状态更新暂停的问题。 +- 修复启动自主目标的轮次未计入其轮次预算的问题。 +- 将图片降采样上限从 2000px 提高到 3000px，并修复 EXIF 旋转（竖拍）照片在压缩说明与媒体读取备注中宽高互换的问题，使区域回读坐标正确对应。 +- web: 修复从后台返回后，WebSocket 重连完成但连接错误提示仍残留的问题。 +- 修复 Windows 上每次运行 hook 时控制台窗口闪烁的问题。 + +### 优化 + +- web: 重新设计定时提醒界面。 +- web: 在斜杠菜单中以 `/skill:` 显示会话技能，便于与内置命令区分；直接输入技能名称仍然可用。 +- web: 输入框的模型切换器在切换当前会话模型的同时，也会更新全局默认模型，使新会话继承该选择。 +- web: 归档等确认对话框支持按 Enter 确认。 +- 优化目标模式对阻塞与完成状态更新的指引。 +- 渐进式工具加载（`select_tools`，实验功能）：压缩后丢弃已加载的工具 schema，由模型重新选择仍需要的工具，使压缩后上下文保持精简；凭记忆调用未再加载的工具会被拒绝，并提示先选择。仅在启用 `tool-select` 实验标志且模型支持 `select_tools` 时生效。 + +### 重构 + +- web: 在构建时编译图标，使打包后的 web UI 仅包含实际渲染的图标。 + ## 0.23.1（2026-07-07） ### 修复 From e83511a7118652a67676bbcfd41148907ad7b8de Mon Sep 17 00:00:00 2001 From: 7Sageer <7sageer@djwcb.cn> Date: Wed, 8 Jul 2026 21:16:44 +0800 Subject: [PATCH 14/24] fix: surface provider auth error for unavailable models (#1506) * fix: surface provider auth error for unavailable models When an OAuth-managed model returns 401 after a forced token refresh, the token is valid but the provider rejected it for that model (the account lacks access). Emit provider.auth_error carrying the provider's message instead of auth.login_required with a misleading "OAuth login expired. Send /login" prompt. * fix(agent-core): preserve provider auth errors through compaction Treat provider.auth_error like auth.login_required in the compaction path so an auth rejection during compaction surfaces the provider's message instead of being wrapped as a generic compaction failure. --- .changeset/fix-model-auth-error-message.md | 5 +++++ packages/agent-core/src/agent/compaction/full.ts | 7 ++++++- packages/agent-core/src/session/provider-manager.ts | 5 +++-- .../agent-core/test/agent/compaction/full.test.ts | 4 ++-- packages/agent-core/test/agent/turn.test.ts | 11 ++++++----- 5 files changed, 22 insertions(+), 10 deletions(-) create mode 100644 .changeset/fix-model-auth-error-message.md diff --git a/.changeset/fix-model-auth-error-message.md b/.changeset/fix-model-auth-error-message.md new file mode 100644 index 000000000..944dbdd6d --- /dev/null +++ b/.changeset/fix-model-auth-error-message.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +Fix a misleading "OAuth login expired" message shown when a model is not available for the current account; the CLI now shows the provider's actual error. diff --git a/packages/agent-core/src/agent/compaction/full.ts b/packages/agent-core/src/agent/compaction/full.ts index ca1f19c8b..005da411f 100644 --- a/packages/agent-core/src/agent/compaction/full.ts +++ b/packages/agent-core/src/agent/compaction/full.ts @@ -595,7 +595,12 @@ export class FullCompaction { thinking_effort: this.agent.config.thinkingEffort, error_type: error instanceof Error ? error.name : 'Unknown', }); - if (isKimiError(error) && error.code === ErrorCodes.AUTH_LOGIN_REQUIRED) throw error; + if ( + isKimiError(error) && + (error.code === ErrorCodes.AUTH_LOGIN_REQUIRED || + error.code === ErrorCodes.PROVIDER_AUTH_ERROR) + ) + throw error; throw new KimiError(ErrorCodes.COMPACTION_FAILED, String(error), { cause: error }); } } diff --git a/packages/agent-core/src/session/provider-manager.ts b/packages/agent-core/src/session/provider-manager.ts index b67aef318..14a6fb5c9 100644 --- a/packages/agent-core/src/session/provider-manager.ts +++ b/packages/agent-core/src/session/provider-manager.ts @@ -203,9 +203,10 @@ export class ProviderManager implements ModelProvider { } catch (error) { if (!(error instanceof APIStatusError) || error.statusCode !== 401) throw error; if (refreshed) { + const reason = error.message.replaceAll('\r', ''); throw new KimiError( - ErrorCodes.AUTH_LOGIN_REQUIRED, - 'OAuth provider credentials were rejected. Send /login to login.', + ErrorCodes.PROVIDER_AUTH_ERROR, + reason.length > 0 ? reason : 'OAuth provider credentials were rejected.', { cause: error, details: { statusCode: error.statusCode, requestId: error.requestId }, diff --git a/packages/agent-core/test/agent/compaction/full.test.ts b/packages/agent-core/test/agent/compaction/full.test.ts index ff9f2ce76..61ab5dd13 100644 --- a/packages/agent-core/test/agent/compaction/full.test.ts +++ b/packages/agent-core/test/agent/compaction/full.test.ts @@ -362,7 +362,7 @@ describe('FullCompaction', () => { expect(messageText(compactionCall?.history[5])).toBe('lookup result'); }); - it('force-refreshes OAuth credentials on compaction 401 and falls back to login_required when replay 401', async () => { + it('force-refreshes OAuth credentials on compaction 401 and treats replay 401 as provider auth error', async () => { const tokenCalls: Array = []; const authKeys: string[] = []; const oauthOptions = oauthTestAgentOptions(async (options) => { @@ -398,7 +398,7 @@ describe('FullCompaction', () => { expect.objectContaining({ event: 'error', args: expect.objectContaining({ - code: 'auth.login_required', + code: 'provider.auth_error', details: expect.objectContaining({ statusCode: 401, requestId: 'req-compact-401', diff --git a/packages/agent-core/test/agent/turn.test.ts b/packages/agent-core/test/agent/turn.test.ts index 5343b221e..68bd19de1 100644 --- a/packages/agent-core/test/agent/turn.test.ts +++ b/packages/agent-core/test/agent/turn.test.ts @@ -1409,7 +1409,7 @@ describe('Agent turn flow', () => { ); }); - it('falls back to login_required when force-refresh and replay both 401', async () => { + it('treats 401 after force-refresh as provider auth error', async () => { const tokenCalls: Array = []; const authKeys: string[] = []; const oauthOptions = oauthAgentOptions( @@ -1447,7 +1447,7 @@ describe('Agent turn flow', () => { args: expect.objectContaining({ reason: 'failed', error: expect.objectContaining({ - code: 'auth.login_required', + code: 'provider.auth_error', details: expect.objectContaining({ statusCode: 401, requestId: 'req-401', @@ -1644,7 +1644,7 @@ describe('Agent turn flow', () => { expect(payloads[1]).toMatchObject({ turnStep: '0.1', attempt: '2/3' }); }); - it('force-refreshes OAuth credentials on video upload 401 and falls back to login_required when replay 401', async () => { + it('force-refreshes OAuth credentials on video upload 401 and surfaces the provider auth error when replay 401', async () => { const tokenCalls: Array = []; const authKeys: string[] = []; const oauthOptions = oauthAgentOptions( @@ -1689,8 +1689,9 @@ describe('Agent turn flow', () => { expect(result.isError).toBe(true); expect(authKeys).toEqual(['fresh-token', 'forced-refresh-token']); expect(tokenCalls).toEqual([undefined, true]); - expect(result.output).toContain('OAuth provider credentials were rejected'); - expect(result.output).toContain('Send /login to login'); + expect(result.output).toContain('Unauthorized'); + expect(result.output).not.toContain('OAuth provider credentials were rejected'); + expect(result.output).not.toContain('Send /login to login'); }); it('cancels an active turn', async () => { From 93c0b7bb7836fa990cd9cd35f6518ed55841d2fe Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 8 Jul 2026 23:18:00 +0800 Subject: [PATCH 15/24] ci: release packages (#1507) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- .changeset/fix-model-auth-error-message.md | 5 ----- apps/kimi-code/CHANGELOG.md | 6 ++++++ apps/kimi-code/package.json | 2 +- 3 files changed, 7 insertions(+), 6 deletions(-) delete mode 100644 .changeset/fix-model-auth-error-message.md diff --git a/.changeset/fix-model-auth-error-message.md b/.changeset/fix-model-auth-error-message.md deleted file mode 100644 index 944dbdd6d..000000000 --- a/.changeset/fix-model-auth-error-message.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@moonshot-ai/kimi-code": patch ---- - -Fix a misleading "OAuth login expired" message shown when a model is not available for the current account; the CLI now shows the provider's actual error. diff --git a/apps/kimi-code/CHANGELOG.md b/apps/kimi-code/CHANGELOG.md index 4cb493e84..9bde39c4a 100644 --- a/apps/kimi-code/CHANGELOG.md +++ b/apps/kimi-code/CHANGELOG.md @@ -1,5 +1,11 @@ # @moonshot-ai/kimi-code +## 0.23.3 + +### Patch Changes + +- [#1506](https://github.com/MoonshotAI/kimi-code/pull/1506) [`e83511a`](https://github.com/MoonshotAI/kimi-code/commit/e83511a7118652a67676bbcfd41148907ad7b8de) Thanks [@7Sageer](https://github.com/7Sageer)! - Fix a misleading "OAuth login expired" message shown when a model is not available for the current account; the CLI now shows the provider's actual error. + ## 0.23.2 ### Patch Changes diff --git a/apps/kimi-code/package.json b/apps/kimi-code/package.json index 9512d71bb..4d9f8df77 100644 --- a/apps/kimi-code/package.json +++ b/apps/kimi-code/package.json @@ -1,6 +1,6 @@ { "name": "@moonshot-ai/kimi-code", - "version": "0.23.2", + "version": "0.23.3", "description": "The Starting Point for Next-Gen Agents", "license": "MIT", "author": "Moonshot AI", From b89fc1a4fbe8c0c3933659cb86b325c82731cf8f Mon Sep 17 00:00:00 2001 From: liruifengv Date: Wed, 8 Jul 2026 23:33:29 +0800 Subject: [PATCH 16/24] docs(changelog): sync 0.23.3 and shorten OAuth error entry (#1509) --- apps/kimi-code/CHANGELOG.md | 2 +- docs/en/release-notes/changelog.md | 6 ++++++ docs/zh/release-notes/changelog.md | 6 ++++++ 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/apps/kimi-code/CHANGELOG.md b/apps/kimi-code/CHANGELOG.md index 9bde39c4a..2a3fc5857 100644 --- a/apps/kimi-code/CHANGELOG.md +++ b/apps/kimi-code/CHANGELOG.md @@ -4,7 +4,7 @@ ### Patch Changes -- [#1506](https://github.com/MoonshotAI/kimi-code/pull/1506) [`e83511a`](https://github.com/MoonshotAI/kimi-code/commit/e83511a7118652a67676bbcfd41148907ad7b8de) Thanks [@7Sageer](https://github.com/7Sageer)! - Fix a misleading "OAuth login expired" message shown when a model is not available for the current account; the CLI now shows the provider's actual error. +- [#1506](https://github.com/MoonshotAI/kimi-code/pull/1506) [`e83511a`](https://github.com/MoonshotAI/kimi-code/commit/e83511a7118652a67676bbcfd41148907ad7b8de) Thanks [@7Sageer](https://github.com/7Sageer)! - Fix a misleading "OAuth login expired" message shown when a model is not available for the current account. ## 0.23.2 diff --git a/docs/en/release-notes/changelog.md b/docs/en/release-notes/changelog.md index 7b98cd3f5..d0549df1c 100644 --- a/docs/en/release-notes/changelog.md +++ b/docs/en/release-notes/changelog.md @@ -6,6 +6,12 @@ outline: 2 This page documents the changes in each Kimi Code CLI release. +## 0.23.3 (2026-07-08) + +### Bug Fixes + +- Fix a misleading "OAuth login expired" message shown when a model is not available for the current account. + ## 0.23.2 (2026-07-08) ### Features diff --git a/docs/zh/release-notes/changelog.md b/docs/zh/release-notes/changelog.md index bce292cfc..444699fdc 100644 --- a/docs/zh/release-notes/changelog.md +++ b/docs/zh/release-notes/changelog.md @@ -6,6 +6,12 @@ outline: 2 本页记录 Kimi Code CLI 每个版本的变更内容。 +## 0.23.3（2026-07-08） + +### 修复 + +- 修复当前账户无法使用某模型时错误显示“OAuth 登录已过期”的问题。 + ## 0.23.2（2026-07-08） ### 新功能 From 735922c291ec3d32d60da6af053f75e1c6179f92 Mon Sep 17 00:00:00 2001 From: liruifengv Date: Thu, 9 Jul 2026 12:30:15 +0800 Subject: [PATCH 17/24] feat(kimi-web): add status-aware browser notifications (#1479) * feat(kimi-web): add approval notification storage key and i18n copy * feat(kimi-web): add approval notification helpers and tests * feat(kimi-web): wire approval notifications and guard completion alerts * fix(kimi-web): extract shouldNotifyCompletion helper and add tests * feat(kimi-web): add approval notification settings toggle * chore(kimi-web): add changeset and tidy notification module comment - Align approval notification tag with spec (kimi-approval-${approvalId}) - Update module header to describe all three notification kinds * fix(kimi-web): make notifications fire reliably - Key completion notification tags by turn (sid + promptId) and question tags by request id, so a stale notification left in the notification center no longer swallows every follow-up alert in the same session - Suppress notifications only while the window is actually focused, not merely visible (document.hasFocus() on top of visibilityState) - Play the attention sound when a tool needs approval, matching the completion and question sounds * chore(kimi-web): simplify changeset --- .changeset/web-approval-notifications.md | 5 + apps/kimi-web/src/App.vue | 2 + .../components/settings/SettingsDialog.vue | 15 ++ .../src/composables/client/useNotification.ts | 114 +++++++++++---- .../client/useSoundNotification.ts | 16 ++- .../src/composables/useKimiWebClient.ts | 75 +++++++--- apps/kimi-web/src/i18n/locales/en/settings.ts | 5 +- apps/kimi-web/src/i18n/locales/zh/settings.ts | 5 +- apps/kimi-web/src/lib/storage.ts | 1 + apps/kimi-web/test/notification-logic.test.ts | 130 +++++++++++++++++- 10 files changed, 320 insertions(+), 48 deletions(-) create mode 100644 .changeset/web-approval-notifications.md diff --git a/.changeset/web-approval-notifications.md b/.changeset/web-approval-notifications.md new file mode 100644 index 000000000..4ffab64dc --- /dev/null +++ b/.changeset/web-approval-notifications.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +web: Add notifications when a tool needs approval, and improve notification reliability. diff --git a/apps/kimi-web/src/App.vue b/apps/kimi-web/src/App.vue index eb9131b48..f41127fb7 100644 --- a/apps/kimi-web/src/App.vue +++ b/apps/kimi-web/src/App.vue @@ -898,6 +898,7 @@ function openPr(url: string): void { :account-model="client.defaultModel.value" :notify="client.notifyOnComplete.value" :notify-question="client.notifyOnQuestion.value" + :notify-approval="client.notifyOnApproval.value" :notify-permission="client.notifyPermission.value" :sound="client.soundOnComplete.value" :conversation-toc="client.conversationToc.value" @@ -910,6 +911,7 @@ function openPr(url: string): void { @set-ui-font-size="client.setUiFontSize($event)" @set-notify="client.setNotifyOnComplete($event)" @set-notify-question="client.setNotifyOnQuestion($event)" + @set-notify-approval="client.setNotifyOnApproval($event)" @set-sound="client.setSoundOnComplete($event)" @set-conversation-toc="client.setConversationToc($event)" @update-config="handleUpdateConfig($event)" diff --git a/apps/kimi-web/src/components/settings/SettingsDialog.vue b/apps/kimi-web/src/components/settings/SettingsDialog.vue index 44d7cf3ca..8f3493a6f 100644 --- a/apps/kimi-web/src/components/settings/SettingsDialog.vue +++ b/apps/kimi-web/src/components/settings/SettingsDialog.vue @@ -32,6 +32,8 @@ const props = defineProps<{ notify: boolean; /** Browser-notification-on-question (needs answer) preference. */ notifyQuestion: boolean; + /** Browser-notification-on-approval preference. */ + notifyApproval: boolean; /** OS permission state ('default' | 'granted' | 'denied') for the hint. */ notifyPermission?: string; /** Play-a-sound-on-completion preference. */ @@ -54,6 +56,7 @@ const emit = defineEmits<{ setUiFontSize: [size: number]; setNotify: [on: boolean]; setNotifyQuestion: [on: boolean]; + setNotifyApproval: [on: boolean]; setSound: [on: boolean]; setConversationToc: [on: boolean]; login: []; @@ -417,6 +420,18 @@ function archiveTime(iso: string): string { @update:model-value="emit('setNotifyQuestion', $event)" />
+
+ + {{ t('settings.notifyOnApproval') }} + {{ t('settings.notifyDenied') }} + + +
{{ t('settings.soundOnComplete') }} ( typeof Notification !== 'undefined' ? Notification.permission : 'denied', ); @@ -61,20 +71,42 @@ function setNotifyOnQuestion(on: boolean): Promise { return setNotifyPref(notifyOnQuestion, STORAGE_KEYS.notifyOnQuestion, on); } -export interface NotifyCompletionCtx { - /** True when the target session is the active one and the page is visible — - in which case we suppress the notification. */ - isActiveAndVisible: boolean; +/** Enable/disable approval notifications. Off by default. */ +function setNotifyOnApproval(on: boolean): Promise { + return setNotifyPref(notifyOnApproval, STORAGE_KEYS.notifyOnApproval, on); +} + +export interface NotifyBaseCtx { + /** True when the user is actually watching the target session: it is the + active session, the page is visible, and the window has focus — in which + case we suppress the notification. */ + isUserWatching: boolean; /** Session title used as the completion notification body and a question-body fallback. */ sessionTitle: string; /** Called when the user clicks the notification (e.g. select the session). */ onClick: () => void; } -export interface NotifyQuestionCtx extends NotifyCompletionCtx { +export interface NotifyCompletionCtx extends NotifyBaseCtx { + /** Prompt id of the finished turn; keys the dedup tag so every turn fires its + own notification while a replayed idle event for the same turn stays + collapsed. Falls back to a per-call unique tag when absent. */ + promptId?: string; +} + +export interface NotifyQuestionCtx extends NotifyBaseCtx { /** Short preview of the question, used as the notification body. Falls back to the session title, then to a generic line when empty. */ questionPreview: string; + /** Unique question request id; used to deduplicate notifications per request. */ + questionId: string; +} + +export interface NotifyApprovalCtx extends NotifyBaseCtx { + /** Tool call name needing approval, used as the notification body. */ + toolName: string; + /** Unique approval request id; used to deduplicate notifications per request. */ + approvalId: string; } export interface NotificationCopy { @@ -111,12 +143,29 @@ export function questionNotificationCopy( }; } +export function approvalNotificationCopy( + sessionTitle: string, + toolName: string, +): NotificationCopy { + return { + title: i18n.global.t('settings.notifyApprovalTitle'), + body: firstText( + toolName, + sessionTitle, + i18n.global.t('settings.notifyApprovalFallback'), + ), + }; +} + /** Shared permission gate + fire. `enabled` is the caller's per-kind preference; - `copy` and `tag` let each kind carry its own text and a per-kind dedup tag - so a completion and a question don't collapse into one notification. */ + `copy` and `tag` let each kind carry its own text and a per-turn/per-request + dedup tag: repeats of the same turn or request collapse into one + notification, while distinct ones each fire (same-tag notifications replace + silently — renotify is unreliable across platforms — so the tag must change + whenever a new alert should pop). */ function maybeNotify( enabled: boolean, - ctx: NotifyCompletionCtx, + ctx: NotifyBaseCtx, copy: NotificationCopy, tag: string, ): void { @@ -135,8 +184,8 @@ function maybeNotify( fire(ctx, copy, tag); } -function fire(ctx: NotifyCompletionCtx, copy: NotificationCopy, tag: string): void { - if (ctx.isActiveAndVisible) return; +function fire(ctx: NotifyBaseCtx, copy: NotificationCopy, tag: string): void { + if (ctx.isUserWatching) return; try { const n = new Notification(copy.title, { body: copy.body, tag, icon: NOTIFICATION_ICON }); n.onclick = () => { @@ -154,24 +203,38 @@ function fire(ctx: NotifyCompletionCtx, copy: NotificationCopy, tag: string): vo } /** Fire a completion notification for a finished session, but only when the - caller says the user isn't already looking at it. */ + caller says the user isn't already looking at it. The tag carries the turn's + prompt id: same-tag notifications replace silently, so without it a stale + notification left in the notification center would swallow every later + turn's alert for that session. */ function maybeNotifyCompletion(sid: string, ctx: NotifyCompletionCtx): void { maybeNotify( notifyOnComplete.value, ctx, completionNotificationCopy(ctx.sessionTitle), - `kimi-complete-${sid}`, + `kimi-complete-${sid}-${ctx.promptId ?? Date.now()}`, ); } /** Fire a notification when a session asks a question, but only when the user explicitly opted into question notifications and isn't already looking. */ -function maybeNotifyQuestion(sid: string, ctx: NotifyQuestionCtx): void { +function maybeNotifyQuestion(ctx: NotifyQuestionCtx): void { maybeNotify( notifyOnQuestion.value, ctx, questionNotificationCopy(ctx.sessionTitle, ctx.questionPreview), - `kimi-question-${sid}`, + `kimi-question-${ctx.questionId}`, + ); +} + +/** Fire a notification when a tool needs approval, but only when the user + explicitly opted into approval notifications and isn't already looking. */ +function maybeNotifyApproval(ctx: NotifyApprovalCtx): void { + maybeNotify( + notifyOnApproval.value, + ctx, + approvalNotificationCopy(ctx.sessionTitle, ctx.toolName), + `kimi-approval-${ctx.approvalId}`, ); } @@ -179,10 +242,13 @@ export function useNotification() { return { notifyOnComplete, notifyOnQuestion, + notifyOnApproval, notifyPermission, setNotifyOnComplete, setNotifyOnQuestion, + setNotifyOnApproval, maybeNotifyCompletion, maybeNotifyQuestion, + maybeNotifyApproval, }; } diff --git a/apps/kimi-web/src/composables/client/useSoundNotification.ts b/apps/kimi-web/src/composables/client/useSoundNotification.ts index db58f6b64..3016c0c6d 100644 --- a/apps/kimi-web/src/composables/client/useSoundNotification.ts +++ b/apps/kimi-web/src/composables/client/useSoundNotification.ts @@ -1,7 +1,9 @@ // apps/kimi-web/src/composables/client/useSoundNotification.ts -// Browser "turn completed" sound: a persisted on/off preference plus a short -// chime synthesized with the WebAudio API (no audio asset, no permission -// prompt). Pure UI action module — it never reads rawState or calls the API. +// Browser attention sound: a persisted on/off preference plus a short chime +// synthesized with the WebAudio API (no audio asset, no permission prompt). +// One chime covers every "the agent needs you" moment — a finished turn, a +// question waiting for an answer, a tool needing approval. Pure UI action +// module — it never reads rawState or calls the API. // // Why the eager "unlock": the sound is most useful when the tab is in the // background (so you hear it while doing something else). But an AudioContext @@ -161,11 +163,19 @@ function maybePlayQuestionSound(): void { playChime(); } +/** Play the attention sound when a tool needs approval, whenever the + preference is on. Same chime as completion: it means "the agent needs you". */ +function maybePlayApprovalSound(): void { + if (!soundOnComplete.value) return; + playChime(); +} + export function useSoundNotification() { return { soundOnComplete, setSoundOnComplete, maybePlayCompletionSound, maybePlayQuestionSound, + maybePlayApprovalSound, }; } diff --git a/apps/kimi-web/src/composables/useKimiWebClient.ts b/apps/kimi-web/src/composables/useKimiWebClient.ts index 2f7e822fc..9b61944bb 100644 --- a/apps/kimi-web/src/composables/useKimiWebClient.ts +++ b/apps/kimi-web/src/composables/useKimiWebClient.ts @@ -30,7 +30,7 @@ import { } from '../lib/storage'; import { createEventBatcher, isRenderEvent } from './client/eventBatcher'; import { useAppearance } from './client/useAppearance'; -import { useNotification } from './client/useNotification'; +import { useNotification, shouldNotifyCompletion } from './client/useNotification'; import { useSoundNotification } from './client/useSoundNotification'; import { useTaskPoller } from './client/useTaskPoller'; import { useModelProviderState } from './client/useModelProviderState'; @@ -860,6 +860,11 @@ function processEvent(appEvent: AppEvent, meta: { sessionId: string; seq: number if (appEvent.type === 'questionRequested') { onQuestionRequested(appEvent.sessionId, appEvent.question); } + + // The agent needs approval for a tool call — surface it so the user comes back. + if (appEvent.type === 'approvalRequested') { + onApprovalRequested(appEvent.sessionId, appEvent.approval); + } } const enqueueEvent = createEventBatcher( @@ -2315,10 +2320,27 @@ const workspaceState = useWorkspaceState(rawState, { fileDiffLoading, }); +/** True when the user is actually watching this session: it is the active + session, the page is visible, and the window has focus. Focus matters on + top of visibility: a window that lost focus to another app often stays + (partially) visible on screen, but the user is working elsewhere and would + miss the moment without a notification. */ +function isUserWatching(sid: string): boolean { + return ( + sid === rawState.activeSessionId && + typeof document !== 'undefined' && + document.visibilityState === 'visible' && + document.hasFocus() + ); +} + function onSessionIdle(sid: string, status: 'idle' | 'aborted'): void { // The turn finished — this session no longer has a prompt in flight. inFlightPromptSessions.delete(sid); rawState.sendingBySession = { ...rawState.sendingBySession, [sid]: false }; + // Capture before the cleanup below drops it — it keys the completion + // notification's dedup tag so each finished turn alerts once. + const finishedPromptId = rawState.promptIdBySession[sid]; // Drop any cached prompt_id so a later skill activation (which has no // prompt_id) doesn't accidentally reuse this stale id for :abort. if (rawState.promptIdBySession[sid] !== undefined) { @@ -2343,16 +2365,20 @@ function onSessionIdle(sid: string, status: 'idle' | 'aborted'): void { } // Browser notification when the user isn't watching this session. - notification.maybeNotifyCompletion(sid, { - isActiveAndVisible: - sid === rawState.activeSessionId && - typeof document !== 'undefined' && - document.visibilityState === 'visible', - sessionTitle: rawState.sessions.find((s) => s.id === sid)?.title ?? '', - onClick: () => { - void workspaceState.selectSession(sid); - }, - }); + // Only real completions notify; aborted turns and turns that ended up + // blocked on approval/question do not fire the generic "Turn finished" alert. + const hasPendingApproval = (rawState.approvalsBySession[sid] ?? []).length > 0; + const hasPendingQuestion = (rawState.questionsBySession[sid] ?? []).length > 0; + if (shouldNotifyCompletion(status, hasPendingApproval, hasPendingQuestion)) { + notification.maybeNotifyCompletion(sid, { + isUserWatching: isUserWatching(sid), + sessionTitle: rawState.sessions.find((s) => s.id === sid)?.title ?? '', + promptId: finishedPromptId, + onClick: () => { + void workspaceState.selectSession(sid); + }, + }); + } // Completion sound — only for real completions (aborted/cancelled turns stay // silent). Plays regardless of visibility so it also reaches a backgrounded tab. @@ -2391,13 +2417,11 @@ function onQuestionRequested(sid: string, question: AppQuestionRequest): void { header && questionText ? `${header}: ${questionText}` : questionText || header; // Browser notification when the user isn't watching this session. - notification.maybeNotifyQuestion(sid, { - isActiveAndVisible: - sid === rawState.activeSessionId && - typeof document !== 'undefined' && - document.visibilityState === 'visible', + notification.maybeNotifyQuestion({ + isUserWatching: isUserWatching(sid), sessionTitle: rawState.sessions.find((s) => s.id === sid)?.title ?? '', questionPreview: preview, + questionId: question.questionId, onClick: () => { void workspaceState.selectSession(sid); }, @@ -2408,6 +2432,23 @@ function onQuestionRequested(sid: string, question: AppQuestionRequest): void { sound.maybePlayQuestionSound(); } +function onApprovalRequested(sid: string, approval: AppApprovalRequest): void { + // Browser notification when the user isn't watching this session. + notification.maybeNotifyApproval({ + isUserWatching: isUserWatching(sid), + sessionTitle: rawState.sessions.find((s) => s.id === sid)?.title ?? '', + toolName: approval.toolName, + approvalId: approval.approvalId, + onClick: () => { + void workspaceState.selectSession(sid); + }, + }); + + // Attention sound — plays regardless of visibility so it also reaches a + // backgrounded tab (same as the completion sound). + sound.maybePlayApprovalSound(); +} + // --------------------------------------------------------------------------- // Composable return // --------------------------------------------------------------------------- @@ -2501,9 +2542,11 @@ export function useKimiWebClient() { setAccent: appearance.setAccent, notifyOnComplete: notification.notifyOnComplete, notifyOnQuestion: notification.notifyOnQuestion, + notifyOnApproval: notification.notifyOnApproval, notifyPermission: notification.notifyPermission, setNotifyOnComplete: notification.setNotifyOnComplete, setNotifyOnQuestion: notification.setNotifyOnQuestion, + setNotifyOnApproval: notification.setNotifyOnApproval, soundOnComplete: sound.soundOnComplete, setSoundOnComplete: sound.setSoundOnComplete, onboarded, diff --git a/apps/kimi-web/src/i18n/locales/en/settings.ts b/apps/kimi-web/src/i18n/locales/en/settings.ts index ec2c8d770..c8aa5a56f 100644 --- a/apps/kimi-web/src/i18n/locales/en/settings.ts +++ b/apps/kimi-web/src/i18n/locales/en/settings.ts @@ -12,12 +12,15 @@ export default { notifications: 'Notifications', notifyOnComplete: 'Notify when a turn completes', notifyOnQuestion: 'Notify when a question needs an answer', - soundOnComplete: 'Play a sound when a turn completes or needs an answer', + notifyOnApproval: 'Notify when a tool needs approval', + soundOnComplete: 'Play a sound when a turn completes, needs an answer, or needs approval', notifyDenied: 'Blocked in browser settings', notifyTitle: 'Kimi Code · Turn finished', notifyQuestionTitle: 'Kimi Code · Needs answer', + notifyApprovalTitle: 'Kimi Code · Approval required', notifyFallback: 'View result', notifyQuestionFallback: 'A question is waiting for your answer', + notifyApprovalFallback: 'A tool needs your approval', account: 'Account', uiFontSize: 'Font size', agentDefaults: 'Agent defaults', diff --git a/apps/kimi-web/src/i18n/locales/zh/settings.ts b/apps/kimi-web/src/i18n/locales/zh/settings.ts index a06a14bdd..d20d6bfa6 100644 --- a/apps/kimi-web/src/i18n/locales/zh/settings.ts +++ b/apps/kimi-web/src/i18n/locales/zh/settings.ts @@ -12,12 +12,15 @@ export default { notifications: '通知', notifyOnComplete: '会话完成时通知', notifyOnQuestion: '待回答时通知', - soundOnComplete: '会话完成或待回答时播放提示音', + notifyOnApproval: '待审批时通知', + soundOnComplete: '会话完成、待回答或待审批时播放提示音', notifyDenied: '已在浏览器设置中被阻止', notifyTitle: 'Kimi Code · 回合完成', notifyQuestionTitle: 'Kimi Code · 待回答', + notifyApprovalTitle: 'Kimi Code · 等待审批', notifyFallback: '点击查看结果', notifyQuestionFallback: '有提问等待你回答', + notifyApprovalFallback: '有工具等待你审批', account: '账户', uiFontSize: '字体大小', agentDefaults: 'Agent 默认值', diff --git a/apps/kimi-web/src/lib/storage.ts b/apps/kimi-web/src/lib/storage.ts index a6d0a1f31..36dab4c61 100644 --- a/apps/kimi-web/src/lib/storage.ts +++ b/apps/kimi-web/src/lib/storage.ts @@ -32,6 +32,7 @@ export const STORAGE_KEYS = { conversationToc: 'kimi-web.beta-toc', notifyOnComplete: 'kimi-web.notify-on-complete', notifyOnQuestion: 'kimi-web.notify-on-question', + notifyOnApproval: 'kimi-web.notify-on-approval', soundOnComplete: 'kimi-web.sound-on-complete', inputHistory: 'kimi-web.input-history', // cross-file diff --git a/apps/kimi-web/test/notification-logic.test.ts b/apps/kimi-web/test/notification-logic.test.ts index f10a31d7d..5b13de8fc 100644 --- a/apps/kimi-web/test/notification-logic.test.ts +++ b/apps/kimi-web/test/notification-logic.test.ts @@ -2,8 +2,10 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest'; import { i18n } from '../src/i18n'; import { STORAGE_KEYS, safeGetString } from '../src/lib/storage'; import { + approvalNotificationCopy, completionNotificationCopy, questionNotificationCopy, + shouldNotifyCompletion, useNotification, } from '../src/composables/client/useNotification'; @@ -41,11 +43,17 @@ function installStorage(storage: Storage): void { // Singleton — module-level refs + setters. The OS Notification API is absent in // the test env, so the *enable* path is a no-op; the disable path and the // load-from-storage defaults are what we exercise here. -const { notifyOnComplete, notifyOnQuestion, setNotifyOnComplete, setNotifyOnQuestion } = useNotification(); -// Captured at import (before beforeEach touches the refs), so these reflect the -// load-from-storage defaults when nothing has been stored yet. +const { + notifyOnComplete, + notifyOnQuestion, + notifyOnApproval, + setNotifyOnComplete, + setNotifyOnQuestion, + setNotifyOnApproval, +} = useNotification(); const importedCompleteDefault = notifyOnComplete.value; const importedQuestionDefault = notifyOnQuestion.value; +const importedApprovalDefault = notifyOnApproval.value; describe('useNotification preferences', () => { beforeEach(() => { @@ -64,6 +72,10 @@ describe('useNotification preferences', () => { expect(importedQuestionDefault).toBe(false); }); + it('approval notifications default to off', () => { + expect(importedApprovalDefault).toBe(false); + }); + it('disabling question notifications persists "0" and updates the ref', () => { void setNotifyOnQuestion(false); expect(notifyOnQuestion.value).toBe(false); @@ -75,6 +87,12 @@ describe('useNotification preferences', () => { expect(notifyOnComplete.value).toBe(false); expect(safeGetString(STORAGE_KEYS.notifyOnComplete)).toBe('0'); }); + + it('disabling approval notifications persists "0" and updates the ref', () => { + void setNotifyOnApproval(false); + expect(notifyOnApproval.value).toBe(false); + expect(safeGetString(STORAGE_KEYS.notifyOnApproval)).toBe('0'); + }); }); describe('notification copy', () => { @@ -110,6 +128,32 @@ describe('notification copy', () => { }); }); + it('uses tool name in approval notifications', () => { + expect(approvalNotificationCopy('Refactor auth flow', 'bash')).toEqual({ + title: 'Kimi Code · Approval required', + body: 'bash', + }); + }); + + it('falls back to session title and then generic approval line', () => { + expect(approvalNotificationCopy('Refactor auth flow', ' ')).toEqual({ + title: 'Kimi Code · Approval required', + body: 'Refactor auth flow', + }); + expect(approvalNotificationCopy(' ', ' ')).toEqual({ + title: 'Kimi Code · Approval required', + body: 'A tool needs your approval', + }); + }); + + it('localizes approval notification copy', () => { + i18n.global.locale.value = 'zh'; + expect(approvalNotificationCopy('', '')).toEqual({ + title: 'Kimi Code · 等待审批', + body: '有工具等待你审批', + }); + }); + it('localizes the notification copy', () => { i18n.global.locale.value = 'zh'; @@ -123,3 +167,83 @@ describe('notification copy', () => { }); }); }); + +describe('shouldNotifyCompletion', () => { + it('returns true only for idle + no pending approval + no pending question', () => { + expect(shouldNotifyCompletion('idle', false, false)).toBe(true); + }); + + it('returns false for aborted', () => { + expect(shouldNotifyCompletion('aborted', false, false)).toBe(false); + }); + + it('returns false when pending approval exists', () => { + expect(shouldNotifyCompletion('idle', true, false)).toBe(false); + }); + + it('returns false when pending question exists', () => { + expect(shouldNotifyCompletion('idle', false, true)).toBe(false); + }); +}); + +// Same-tag notifications replace silently (renotify is unreliable), so the tag +// must be unique per turn/request for follow-up alerts in a session to pop. +describe('notification tags', () => { + class FakeNotification { + static permission = 'granted'; + static fired: Array<{ title: string; tag?: string }> = []; + onclick: (() => void) | null = null; + constructor(title: string, options?: { body?: string; tag?: string; icon?: string }) { + FakeNotification.fired.push({ title, tag: options?.tag }); + } + close(): void {} + } + + const { maybeNotifyCompletion, maybeNotifyQuestion, maybeNotifyApproval } = useNotification(); + const base = { isUserWatching: false, sessionTitle: 'T', onClick: () => {} }; + + beforeEach(() => { + FakeNotification.fired = []; + (globalThis as Record).Notification = FakeNotification; + notifyOnComplete.value = true; + notifyOnQuestion.value = true; + notifyOnApproval.value = true; + }); + + afterEach(() => { + delete (globalThis as Record).Notification; + notifyOnComplete.value = true; + notifyOnQuestion.value = false; + notifyOnApproval.value = false; + }); + + it('completion tags carry the prompt id so each turn in a session alerts', () => { + maybeNotifyCompletion('s1', { ...base, promptId: 'p1' }); + maybeNotifyCompletion('s1', { ...base, promptId: 'p2' }); + expect(FakeNotification.fired.map((f) => f.tag)).toEqual([ + 'kimi-complete-s1-p1', + 'kimi-complete-s1-p2', + ]); + }); + + it('a replayed idle event for the same turn keeps the same tag', () => { + maybeNotifyCompletion('s1', { ...base, promptId: 'p1' }); + maybeNotifyCompletion('s1', { ...base, promptId: 'p1' }); + expect(FakeNotification.fired).toHaveLength(2); + expect(FakeNotification.fired[0]?.tag).toBe(FakeNotification.fired[1]?.tag); + }); + + it('question and approval tags are per-request', () => { + maybeNotifyQuestion({ ...base, questionPreview: 'q', questionId: 'q1' }); + maybeNotifyApproval({ ...base, toolName: 'bash', approvalId: 'a1' }); + expect(FakeNotification.fired.map((f) => f.tag)).toEqual([ + 'kimi-question-q1', + 'kimi-approval-a1', + ]); + }); + + it('suppresses the notification while the user is watching the session', () => { + maybeNotifyCompletion('s1', { ...base, isUserWatching: true, promptId: 'p1' }); + expect(FakeNotification.fired).toHaveLength(0); + }); +}); From ad30a1c6328327729221f9f5fc700b621dfef779 Mon Sep 17 00:00:00 2001 From: Luyu Cheng <2239547+chengluyu@users.noreply.github.com> Date: Thu, 9 Jul 2026 14:55:58 +0800 Subject: [PATCH 18/24] style(web): polish web UI typography and controls (#1502) * style(web): polish sidebar and tool typography - Use UI font and medium weight for sidebar and composer controls - Add reusable shortcut and tool output blocks - Cap long tool output at 50 lines with a scrollbar - Update sidebar show-more copy and muted styling * style(web): refine workspace picker sizing * style(web): align composer mode menus * style(web): tune list and question typography * style(web): reuse shortcut keys in approvals * style(web): size workspace picker from content * feat(web): localize chat status labels * style(web): refine composer toolbar controls * style(web): use complete Inter variable font * style(web): tune sidebar workspace typography * style(web): polish composer and workspace picker * style(web): refine markdown and thinking typography * chore: add web UI polish changeset * fix(web): pin Inter package for Nix build * style(web): polish goal tool calls * style: polish goal mode display * fix: layer latest message pill below menus * style: align goal strip content --- .changeset/web-ui-polish.md | 5 + apps/kimi-web/package.json | 3 +- apps/kimi-web/src/components/SessionRow.vue | 12 +- apps/kimi-web/src/components/Sidebar.vue | 19 +- .../src/components/WorkspaceGroup.vue | 9 +- .../src/components/chat/ApprovalCard.vue | 19 +- .../kimi-web/src/components/chat/ChatDock.vue | 1 + .../src/components/chat/ChatHeader.vue | 24 +- .../kimi-web/src/components/chat/Composer.vue | 400 ++++++++++++++---- .../src/components/chat/ConversationPane.vue | 171 +++++++- .../src/components/chat/GoalStrip.vue | 125 ++++-- .../kimi-web/src/components/chat/Markdown.vue | 21 +- .../src/components/chat/QuestionCard.vue | 22 +- .../src/components/chat/ThinkingBlock.vue | 6 +- .../src/components/chat/ThinkingPanel.vue | 3 +- apps/kimi-web/src/components/chat/ToolRow.vue | 21 +- .../components/chat/tool-calls/EditTool.vue | 16 +- .../chat/tool-calls/GenericTool.vue | 16 +- .../chat/tool-calls/ToolOutputBlock.vue | 42 ++ .../components/mobile/MobileSwitcherSheet.vue | 9 +- apps/kimi-web/src/components/ui/MenuItem.vue | 4 +- .../src/components/ui/ShortcutKey.vue | 24 ++ .../src/composables/useKimiWebClient.ts | 2 +- apps/kimi-web/src/i18n/locales/en/header.ts | 5 + apps/kimi-web/src/i18n/locales/en/sidebar.ts | 4 +- apps/kimi-web/src/i18n/locales/en/status.ts | 9 + apps/kimi-web/src/i18n/locales/en/tools.ts | 15 + apps/kimi-web/src/i18n/locales/zh/composer.ts | 4 +- apps/kimi-web/src/i18n/locales/zh/header.ts | 5 + apps/kimi-web/src/i18n/locales/zh/sidebar.ts | 4 +- apps/kimi-web/src/i18n/locales/zh/status.ts | 13 +- apps/kimi-web/src/i18n/locales/zh/tools.ts | 15 + apps/kimi-web/src/lib/icons.ts | 15 + apps/kimi-web/src/lib/toolMeta.ts | 68 +++ apps/kimi-web/src/main.ts | 3 +- apps/kimi-web/src/style.css | 23 +- apps/kimi-web/src/views/DesignSystemView.vue | 28 +- flake.nix | 2 +- pnpm-lock.yaml | 10 +- 39 files changed, 941 insertions(+), 256 deletions(-) create mode 100644 .changeset/web-ui-polish.md create mode 100644 apps/kimi-web/src/components/chat/tool-calls/ToolOutputBlock.vue create mode 100644 apps/kimi-web/src/components/ui/ShortcutKey.vue diff --git a/.changeset/web-ui-polish.md b/.changeset/web-ui-polish.md new file mode 100644 index 000000000..5a829b6c1 --- /dev/null +++ b/.changeset/web-ui-polish.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +web: Polish the chat UI with Inter typography, localized labels, and tighter sidebar, composer, and menu styling. diff --git a/apps/kimi-web/package.json b/apps/kimi-web/package.json index febbb8626..f7d2f2370 100644 --- a/apps/kimi-web/package.json +++ b/apps/kimi-web/package.json @@ -13,7 +13,8 @@ "check:style": "node scripts/check-style.mjs" }, "dependencies": { - "@fontsource-variable/inter": "^5.2.8", + "@chenglou/pretext": "0.0.8", + "@fontsource-variable/inter": "5.2.8", "@fontsource-variable/jetbrains-mono": "^5.2.8", "@xterm/addon-fit": "^0.11.0", "@xterm/xterm": "^6.0.0", diff --git a/apps/kimi-web/src/components/SessionRow.vue b/apps/kimi-web/src/components/SessionRow.vue index 5d1b8a6c3..2291a2681 100644 --- a/apps/kimi-web/src/components/SessionRow.vue +++ b/apps/kimi-web/src/components/SessionRow.vue @@ -341,8 +341,9 @@ defineExpose({ closeMenu }); .t { color: inherit; - font-size: var(--ui-font-size); - font-weight: var(--weight-regular); + font-size: var(--text-base); + font-weight: 450; + line-height: var(--leading-tight); flex: 1; min-width: 0; overflow: hidden; @@ -353,7 +354,10 @@ defineExpose({ closeMenu }); .ts { color: var(--color-text-faint); font-size: var(--text-xs); - font-family: var(--font-mono); + font-family: var(--font-ui); + font-weight: 475; + font-variant-numeric: tabular-nums; + text-align: right; } /* Trailing action slot: time and kebab share one grid cell (grid-area:1/1). @@ -366,7 +370,7 @@ defineExpose({ closeMenu }); display: inline-grid; flex: none; align-items: center; - justify-items: center; + justify-items: end; } .act .ts, .act .kebab { grid-area: 1 / 1; } diff --git a/apps/kimi-web/src/components/Sidebar.vue b/apps/kimi-web/src/components/Sidebar.vue index 920eb2492..987465dbb 100644 --- a/apps/kimi-web/src/components/Sidebar.vue +++ b/apps/kimi-web/src/components/Sidebar.vue @@ -23,6 +23,7 @@ import IconButton from './ui/IconButton.vue'; import Icon from './ui/Icon.vue'; import Menu from './ui/Menu.vue'; import MenuItem from './ui/MenuItem.vue'; +import ShortcutKey from './ui/ShortcutKey.vue'; import { useConfirmDialog } from '../composables/useConfirmDialog'; const { t } = useI18n(); @@ -599,7 +600,10 @@ onBeforeUnmount(() => { @@ -911,6 +915,7 @@ onBeforeUnmount(() => { color: var(--color-text); font-family: var(--font-ui); font-size: var(--ui-font-size); + font-weight: var(--weight-medium); cursor: pointer; text-align: left; } @@ -950,15 +955,25 @@ onBeforeUnmount(() => { flex: none; } .search-input { + display: inline-flex; + align-items: center; + gap: var(--space-1); flex: 1; min-width: 0; color: var(--color-text); - font-family: var(--mono); + font-family: var(--font-ui); font-size: var(--ui-font-size); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.search-input > span { + overflow: hidden; + text-overflow: ellipsis; +} +.search-label { + font-weight: var(--weight-medium); +} /* Sessions */ .sessions { diff --git a/apps/kimi-web/src/components/WorkspaceGroup.vue b/apps/kimi-web/src/components/WorkspaceGroup.vue index 729707b8e..9e9731929 100644 --- a/apps/kimi-web/src/components/WorkspaceGroup.vue +++ b/apps/kimi-web/src/components/WorkspaceGroup.vue @@ -265,7 +265,7 @@ function onHeaderDragStart(event: DragEvent): void { .gh-name { font-size: var(--ui-font-size-lg); - font-weight: var(--weight-medium); + font-weight: 550; color: var(--color-text); flex: 1; min-width: 0; @@ -276,6 +276,7 @@ function onHeaderDragStart(event: DragEvent): void { } .gh-path { color: var(--color-text-faint); + font-weight: 425; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; @@ -317,10 +318,10 @@ function onHeaderDragStart(event: DragEvent): void { .gh-more.open { color: var(--color-text); background: var(--color-line); } .group-empty { - padding: var(--space-1) var(--space-2) var(--space-1) calc(var(--sb-pad-x) + var(--sb-gutter) + var(--sb-gap)); + padding: var(--space-1) var(--space-2) var(--space-1) calc(var(--sb-pad-x) - var(--space-2) + var(--sb-gutter) + var(--sb-gap)); font-size: var(--text-xs); color: var(--color-text-faint); - font-family: var(--font-mono); + font-family: var(--font-ui); } /* Show-more / show-less — a session-row-shaped compact list control (§07). The empty lead slot mirrors a session row's status gutter, so the label text lands @@ -338,7 +339,7 @@ function onHeaderDragStart(event: DragEvent): void { border: none; border-radius: var(--radius-md); background: transparent; - color: var(--color-text); + color: var(--color-text-muted); font-family: var(--font-ui); font-size: var(--text-xs); text-align: left; diff --git a/apps/kimi-web/src/components/chat/ApprovalCard.vue b/apps/kimi-web/src/components/chat/ApprovalCard.vue index 0448ff261..48047bccb 100644 --- a/apps/kimi-web/src/components/chat/ApprovalCard.vue +++ b/apps/kimi-web/src/components/chat/ApprovalCard.vue @@ -10,6 +10,7 @@ import Badge from '../ui/Badge.vue'; import Button from '../ui/Button.vue'; import IconButton from '../ui/IconButton.vue'; import Icon from '../ui/Icon.vue'; +import ShortcutKey from '../ui/ShortcutKey.vue'; import Tooltip from '../ui/Tooltip.vue'; const props = defineProps<{ @@ -314,20 +315,20 @@ onUnmounted(() => document.removeEventListener('keydown', handleKeydown)); :loading="pendingAction === `option:${opt.label}`" :disabled="busy" @click="approveOption(opt.label)" - >{{ opt.label }}[{{ i + 1 }}] + >{{ opt.label }}[{{ i + 1 }}] - - - + + +
- - - - + + + +
@@ -554,7 +555,7 @@ onUnmounted(() => document.removeEventListener('keydown', handleKeydown)); width: 100%; } .plan-actions { flex-wrap: wrap; } -.k { margin-left: var(--space-2); font: var(--text-xs) var(--font-mono); opacity: .7; } +.k { opacity: .75; } /* ========================================================================= MOBILE (≤640px): the card spans the full chat column, inner previews scroll diff --git a/apps/kimi-web/src/components/chat/ChatDock.vue b/apps/kimi-web/src/components/chat/ChatDock.vue index 4078c1d14..8da9067d2 100644 --- a/apps/kimi-web/src/components/chat/ChatDock.vue +++ b/apps/kimi-web/src/components/chat/ChatDock.vue @@ -251,6 +251,7 @@ defineExpose({ loadForEdit, loadAttachmentsForEdit, focus }); :plan-mode="planMode" :swarm-mode="swarmMode" :goal-mode="goalMode" + :goal="goal" :activation-badges="activationBadges" :models="models" :starred-ids="starredIds" diff --git a/apps/kimi-web/src/components/chat/ChatHeader.vue b/apps/kimi-web/src/components/chat/ChatHeader.vue index 66b603bab..5de3154b9 100644 --- a/apps/kimi-web/src/components/chat/ChatHeader.vue +++ b/apps/kimi-web/src/components/chat/ChatHeader.vue @@ -51,6 +51,25 @@ const behind = computed(() => props.behind ?? 0); const adds = computed(() => props.gitDiffStats?.totalAdditions ?? 0); const dels = computed(() => props.gitDiffStats?.totalDeletions ?? 0); const hasLineStats = computed(() => adds.value > 0 || dels.value > 0); +const PR_STATE_LABEL_KEYS: Record = { + open: 'header.prStatusOpen', + closed: 'header.prStatusClosed', + merged: 'header.prStatusMerged', + draft: 'header.prStatusDraft', +}; + +function normalizedPrState(state: string): string { + return state.trim().toLowerCase().replaceAll('_', '-'); +} + +function prStateClass(state: string): string { + const stateClass = normalizedPrState(state); + return PR_STATE_LABEL_KEYS[stateClass] ? `pr-${stateClass}` : 'pr-unknown'; +} + +function prStateLabel(state: string): string { + return t(PR_STATE_LABEL_KEYS[normalizedPrState(state)] ?? 'header.prStatusUnknown'); +} // --------------------------------------------------------------------------- // More-menu (kebab dropdown) @@ -295,11 +314,11 @@ async function startArchive(): Promise { v-if="pr" type="button" class="ch-pill ch-pr" - :class="`pr-${pr.state}`" + :class="prStateClass(pr.state)" @click="pr && emit('openPr', pr.url)" > - PR #{{ pr.number }} · {{ pr.state }} + PR #{{ pr.number }} · {{ prStateLabel(pr.state) }} @@ -420,6 +439,7 @@ async function startArchive(): Promise { .ch-pr.pr-merged { color: var(--color-done); border-color: var(--color-done-bd); background: var(--color-done-soft); } .ch-pr.pr-closed { color: var(--color-danger); border-color: var(--color-danger-bd); background: var(--color-danger-soft); } .ch-pr.pr-draft { color: var(--color-text-muted); border-color: var(--color-line-strong); background: var(--color-surface-sunken); } +.ch-pr.pr-unknown { color: var(--color-text-muted); border-color: var(--color-line-strong); background: var(--color-surface-sunken); } .ch-pr:hover { border-color: var(--color-line-strong); } /* Fixed more-menu, anchored to the kebab trigger. Surface / items come from diff --git a/apps/kimi-web/src/components/chat/Composer.vue b/apps/kimi-web/src/components/chat/Composer.vue index e8069b0ec..92031302e 100644 --- a/apps/kimi-web/src/components/chat/Composer.vue +++ b/apps/kimi-web/src/components/chat/Composer.vue @@ -1,5 +1,6 @@ + + + + diff --git a/apps/kimi-web/src/components/mobile/MobileSwitcherSheet.vue b/apps/kimi-web/src/components/mobile/MobileSwitcherSheet.vue index da5d73cbd..4a87b0cc0 100644 --- a/apps/kimi-web/src/components/mobile/MobileSwitcherSheet.vue +++ b/apps/kimi-web/src/components/mobile/MobileSwitcherSheet.vue @@ -394,7 +394,7 @@ async function onDeleteWorkspace(ws: WorkspaceView): Promise { } .mgh-name { font-size: var(--ui-font-size-lg); - font-weight: 500; + font-weight: 550; color: var(--color-text); overflow: hidden; text-overflow: ellipsis; @@ -402,6 +402,7 @@ async function onDeleteWorkspace(ws: WorkspaceView): Promise { } .mgh-path { font-size: var(--text-base); + font-weight: 425; color: var(--color-text-faint); overflow: hidden; text-overflow: ellipsis; @@ -430,12 +431,14 @@ async function onDeleteWorkspace(ws: WorkspaceView): Promise { .srow .m { flex: 1; min-width: 0; } .srow .m .t { font-size: var(--text-base); + font-weight: 450; + line-height: var(--leading-tight); color: var(--color-text); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } -.srow.cur .m .t { font-weight: 500; color: var(--color-accent-hover); } +.srow.cur .m .t { color: var(--color-accent-hover); } /* Running indicator — pulse dot in the indent gutter left of the title, mirroring the desktop SessionRow (.t.run::before). */ @@ -471,6 +474,8 @@ async function onDeleteWorkspace(ws: WorkspaceView): Promise { } .srow .m .s { font-size: var(--text-base); + font-weight: 475; + font-variant-numeric: tabular-nums; color: var(--color-text-faint); margin-top: 1px; overflow: hidden; diff --git a/apps/kimi-web/src/components/ui/MenuItem.vue b/apps/kimi-web/src/components/ui/MenuItem.vue index a44bb31d9..cc9cf0d69 100644 --- a/apps/kimi-web/src/components/ui/MenuItem.vue +++ b/apps/kimi-web/src/components/ui/MenuItem.vue @@ -38,9 +38,9 @@ defineEmits<{ click: [event: MouseEvent] }>(); border: none; border-radius: var(--radius-sm); background: transparent; - color: var(--color-text-muted); + color: var(--color-text); font-family: var(--font-ui); - font-size: var(--text-sm); + font-size: var(--text-base); text-align: left; cursor: pointer; transition: background var(--duration-base), color var(--duration-base); diff --git a/apps/kimi-web/src/components/ui/ShortcutKey.vue b/apps/kimi-web/src/components/ui/ShortcutKey.vue new file mode 100644 index 000000000..67c87a182 --- /dev/null +++ b/apps/kimi-web/src/components/ui/ShortcutKey.vue @@ -0,0 +1,24 @@ + + + + diff --git a/apps/kimi-web/src/composables/useKimiWebClient.ts b/apps/kimi-web/src/composables/useKimiWebClient.ts index 9b61944bb..0e2b7adf5 100644 --- a/apps/kimi-web/src/composables/useKimiWebClient.ts +++ b/apps/kimi-web/src/composables/useKimiWebClient.ts @@ -1380,7 +1380,7 @@ function formatTime(iso: string, _status: string): string { const diffD = diffMs / 86400000; if (diffD < 7) return `${Math.round(diffD)}d`; if (diffD < 30) return `${Math.round(diffD / 7)}w`; - if (diffD < 365) return `${Math.round(diffD / 30)}m`; + if (diffD < 365) return `${Math.round(diffD / 30)}mo`; return `${Math.round(diffD / 365)}y`; } catch { return iso; diff --git a/apps/kimi-web/src/i18n/locales/en/header.ts b/apps/kimi-web/src/i18n/locales/en/header.ts index bf046bee4..4273de01d 100644 --- a/apps/kimi-web/src/i18n/locales/en/header.ts +++ b/apps/kimi-web/src/i18n/locales/en/header.ts @@ -10,6 +10,11 @@ export default { gitTooltip: 'Open Files > Changed', detached: 'detached', openPr: 'Open pull request', + prStatusOpen: 'open', + prStatusClosed: 'closed', + prStatusMerged: 'merged', + prStatusDraft: 'draft', + prStatusUnknown: 'unknown', options: 'Options', copySessionId: 'Copy Session ID', renameSession: 'Rename', diff --git a/apps/kimi-web/src/i18n/locales/en/sidebar.ts b/apps/kimi-web/src/i18n/locales/en/sidebar.ts index 3bb705ca3..70adc2e5e 100644 --- a/apps/kimi-web/src/i18n/locales/en/sidebar.ts +++ b/apps/kimi-web/src/i18n/locales/en/sidebar.ts @@ -31,9 +31,9 @@ export default { language: 'Language', daemon: 'Daemon', noSessions: 'No conversations yet', - showMore: 'Load more ({count})', + showMore: 'Load {count} more conversations', showLess: 'Show less', - showAll: 'Show all ({count})', + showAll: 'Show {count} more conversations', loadingMore: 'Loading…', collapseSidebar: 'Collapse sidebar', expandSidebar: 'Expand sidebar', diff --git a/apps/kimi-web/src/i18n/locales/en/status.ts b/apps/kimi-web/src/i18n/locales/en/status.ts index fc4ca8cc4..8b7f02278 100644 --- a/apps/kimi-web/src/i18n/locales/en/status.ts +++ b/apps/kimi-web/src/i18n/locales/en/status.ts @@ -12,23 +12,32 @@ export default { permissionYoloDesc: 'Auto-approve tool actions, but agent may still ask questions', // Plan mode pill planLabel: 'Plan', + planDesc: 'Have the agent make a plan before changing files', planOn: 'on', planOff: 'off', planTooltip: 'Toggle plan mode (research before editing)', // Mode selector (Plan / Goal / Swarm) modesLabel: 'Mode', goalLabel: 'Goal', + goalDesc: 'Track one objective until it is complete', swarmLabel: 'Swarm', + swarmDesc: 'Run parallel agents for broader exploration', modeOff: 'Off', goalPlaceholder: 'What should the agent achieve?', goalStart: 'Start', goalPause: 'Pause', goalResume: 'Resume', goalCancel: 'Cancel', + goalStatusActive: 'Active', + goalStatusPaused: 'Paused', + goalStatusBlocked: 'Blocked', + goalStatusComplete: 'Complete', modeNotSupported: 'Not supported', // Thinking selector thinkingLabel: 'thinking', thinkingTooltip: 'Toggle thinking mode', + thinkingOn: 'On', + thinkingOff: 'Off', starredModels: 'Starred', moreModels: 'More models…', // Status panel diff --git a/apps/kimi-web/src/i18n/locales/en/tools.ts b/apps/kimi-web/src/i18n/locales/en/tools.ts index aaddeb334..13cc18211 100644 --- a/apps/kimi-web/src/i18n/locales/en/tools.ts +++ b/apps/kimi-web/src/i18n/locales/en/tools.ts @@ -13,6 +13,10 @@ export default { task: 'Task', swarm: 'Swarm', ask_user: 'Question', + goal_create: 'Start Goal', + goal_get: 'Read Goal', + goal_budget: 'Set Goal Budget', + goal_update: 'Update Goal', }, swarm: { progress: '{done} / {total}', @@ -32,6 +36,17 @@ export default { created: 'created', todos: '{count} items', }, + goal: { + objectiveWithCriterion: '{objective} · {criterion}', + status: 'Status: {status}', + budget: '{value} {unit}', + turns: '{value} turns', + tokens: '{value} tokens', + milliseconds: '{value} ms', + seconds: '{value} sec', + minutes: '{value} min', + hours: '{value} hr', + }, group: { title: '{count} tool call | {count} tool calls', running: 'running', diff --git a/apps/kimi-web/src/i18n/locales/zh/composer.ts b/apps/kimi-web/src/i18n/locales/zh/composer.ts index 6c1ec1926..719f7adb7 100644 --- a/apps/kimi-web/src/i18n/locales/zh/composer.ts +++ b/apps/kimi-web/src/i18n/locales/zh/composer.ts @@ -22,7 +22,7 @@ export default { emptyConversationTitle: 'Kimi Code', emptyConversation: '还没有消息 —— 在下方输入开始对话', quickStartPlaceholder: '输入消息开始新对话…', - thinkingSuffix: ' · thinking', - thinkingSuffixEffort: ' · thinking: {level}', + thinkingSuffix: ' · 思考', + thinkingSuffixEffort: ' · 思考: {level}', } as const; diff --git a/apps/kimi-web/src/i18n/locales/zh/header.ts b/apps/kimi-web/src/i18n/locales/zh/header.ts index 543cece6c..687845609 100644 --- a/apps/kimi-web/src/i18n/locales/zh/header.ts +++ b/apps/kimi-web/src/i18n/locales/zh/header.ts @@ -10,6 +10,11 @@ export default { gitTooltip: '打开「文件 > 改动」', detached: '游离', openPr: '打开 Pull Request', + prStatusOpen: '已打开', + prStatusClosed: '已关闭', + prStatusMerged: '已合并', + prStatusDraft: '草稿', + prStatusUnknown: '未知', options: '选项', copySessionId: '复制 Session ID', renameSession: '重命名', diff --git a/apps/kimi-web/src/i18n/locales/zh/sidebar.ts b/apps/kimi-web/src/i18n/locales/zh/sidebar.ts index bea56f556..9e92e43bd 100644 --- a/apps/kimi-web/src/i18n/locales/zh/sidebar.ts +++ b/apps/kimi-web/src/i18n/locales/zh/sidebar.ts @@ -31,9 +31,9 @@ export default { language: '语言', daemon: '后台', noSessions: '暂无对话', - showMore: '加载更多 ({count})', + showMore: '加载更多 {count} 个对话', showLess: '收起', - showAll: '展开 ({count})', + showAll: '展开剩余 {count} 个对话', loadingMore: '加载中…', collapseSidebar: '收起侧边栏', expandSidebar: '展开侧边栏', diff --git a/apps/kimi-web/src/i18n/locales/zh/status.ts b/apps/kimi-web/src/i18n/locales/zh/status.ts index 98287e9d5..0256b119b 100644 --- a/apps/kimi-web/src/i18n/locales/zh/status.ts +++ b/apps/kimi-web/src/i18n/locales/zh/status.ts @@ -8,27 +8,36 @@ export default { permissionAuto: '完全自主', permissionYolo: '自动通过', permissionManualDesc: '每个工具操作都需要你手动确认', - permissionAutoDesc: '完全自主运行，agent 自己做决定，不再询问', + permissionAutoDesc: '完全自主运行，智能体自己做决定，不再询问', permissionYoloDesc: '自动批准工具操作，但遇到关键问题仍会询问', // 计划模式 planLabel: '计划', + planDesc: '先让智能体梳理计划，再修改文件', planOn: '开', planOff: '关', planTooltip: '切换计划模式（先调研再修改）', // 模式选择器（计划 / 目标 / Swarm） modesLabel: '模式', goalLabel: '目标', + goalDesc: '持续跟踪一个目标，直到任务完成', swarmLabel: 'Swarm', + swarmDesc: '并行运行多个智能体，适合大范围探索', modeOff: '未启用', - goalPlaceholder: '让 Agent 完成什么目标？', + goalPlaceholder: '让智能体完成什么目标？', goalStart: '开始', goalPause: '暂停', goalResume: '继续', goalCancel: '取消', + goalStatusActive: '进行中', + goalStatusPaused: '已暂停', + goalStatusBlocked: '已阻塞', + goalStatusComplete: '已完成', modeNotSupported: '暂不支持', // 思考强度选择 thinkingLabel: '思考', thinkingTooltip: '切换思考模式', + thinkingOn: '开', + thinkingOff: '关', starredModels: '收藏', moreModels: '更多模型…', // 状态面板 diff --git a/apps/kimi-web/src/i18n/locales/zh/tools.ts b/apps/kimi-web/src/i18n/locales/zh/tools.ts index 0cee01058..f9560e3da 100644 --- a/apps/kimi-web/src/i18n/locales/zh/tools.ts +++ b/apps/kimi-web/src/i18n/locales/zh/tools.ts @@ -13,6 +13,10 @@ export default { task: '任务', swarm: 'Swarm', ask_user: '提问', + goal_create: '启动目标', + goal_get: '读取目标', + goal_budget: '设置目标预算', + goal_update: '更新目标', }, swarm: { progress: '{done} / {total}', @@ -32,6 +36,17 @@ export default { created: '已创建', todos: '{count} 项', }, + goal: { + objectiveWithCriterion: '{objective} · {criterion}', + status: '状态：{status}', + budget: '{value} {unit}', + turns: '{value} 轮', + tokens: '{value} token', + milliseconds: '{value} 毫秒', + seconds: '{value} 秒', + minutes: '{value} 分钟', + hours: '{value} 小时', + }, group: { title: '{count} 个工具调用', running: '运行中', diff --git a/apps/kimi-web/src/lib/icons.ts b/apps/kimi-web/src/lib/icons.ts index e41f0b1e4..a6e1ab5d2 100644 --- a/apps/kimi-web/src/lib/icons.ts +++ b/apps/kimi-web/src/lib/icons.ts @@ -43,6 +43,7 @@ import RiExpandRightLine from '~icons/ri/expand-right-line'; import RiExternalLinkLine from '~icons/ri/external-link-line'; import RiFileAddLine from '~icons/ri/file-add-line'; import RiFileCopyLine from '~icons/ri/file-copy-line'; +import RiFileEditLine from '~icons/ri/file-edit-line'; import RiFileLine from '~icons/ri/file-line'; import RiFileTextLine from '~icons/ri/file-text-line'; import RiFlashlightLine from '~icons/ri/flashlight-line'; @@ -61,6 +62,7 @@ import RiLoginBoxLine from '~icons/ri/login-box-line'; import RiMailLine from '~icons/ri/mail-line'; import RiMessageLine from '~icons/ri/message-line'; import RiMoreLine from '~icons/ri/more-line'; +import RiPauseFill from '~icons/ri/pause-fill'; import RiPencilLine from '~icons/ri/pencil-line'; import RiPlayFill from '~icons/ri/play-fill'; import RiQuestionLine from '~icons/ri/question-line'; @@ -72,6 +74,7 @@ import RiStarFill from '~icons/ri/star-fill'; import RiStarLine from '~icons/ri/star-line'; import RiStopFill from '~icons/ri/stop-fill'; import RiSubtractLine from '~icons/ri/subtract-line'; +import RiTargetLine from '~icons/ri/target-line'; import RiTerminalBoxLine from '~icons/ri/terminal-box-line'; import RiTimeLine from '~icons/ri/time-line'; import RiToolsLine from '~icons/ri/tools-line'; @@ -104,6 +107,7 @@ import RawExpandRightLine from '~icons/ri/expand-right-line?raw'; import RawExternalLinkLine from '~icons/ri/external-link-line?raw'; import RawFileAddLine from '~icons/ri/file-add-line?raw'; import RawFileCopyLine from '~icons/ri/file-copy-line?raw'; +import RawFileEditLine from '~icons/ri/file-edit-line?raw'; import RawFileLine from '~icons/ri/file-line?raw'; import RawFileTextLine from '~icons/ri/file-text-line?raw'; import RawFlashlightLine from '~icons/ri/flashlight-line?raw'; @@ -122,6 +126,7 @@ import RawLoginBoxLine from '~icons/ri/login-box-line?raw'; import RawMailLine from '~icons/ri/mail-line?raw'; import RawMessageLine from '~icons/ri/message-line?raw'; import RawMoreLine from '~icons/ri/more-line?raw'; +import RawPauseFill from '~icons/ri/pause-fill?raw'; import RawPencilLine from '~icons/ri/pencil-line?raw'; import RawPlayFill from '~icons/ri/play-fill?raw'; import RawQuestionLine from '~icons/ri/question-line?raw'; @@ -133,6 +138,7 @@ import RawStarFill from '~icons/ri/star-fill?raw'; import RawStarLine from '~icons/ri/star-line?raw'; import RawStopFill from '~icons/ri/stop-fill?raw'; import RawSubtractLine from '~icons/ri/subtract-line?raw'; +import RawTargetLine from '~icons/ri/target-line?raw'; import RawTerminalBoxLine from '~icons/ri/terminal-box-line?raw'; import RawTimeLine from '~icons/ri/time-line?raw'; import RawToolsLine from '~icons/ri/tools-line?raw'; @@ -177,6 +183,7 @@ export type IconName = | 'folder-solid' | 'file' | 'file-text' + | 'file-edit' | 'file-plus' | 'file-off' | 'image-off' @@ -197,6 +204,8 @@ export type IconName = | 'alert-triangle' | 'clock' | 'sparkles' + | 'target' + | 'pause' | 'play' | 'stop' | 'star' @@ -256,6 +265,7 @@ export const ICONS: Record = { 'folder-solid': entry(RiFolderFill, RawFolderFill), file: entry(RiFileLine, RawFileLine), 'file-text': entry(RiFileTextLine, RawFileTextLine), + 'file-edit': entry(RiFileEditLine, RawFileEditLine), 'file-plus': entry(RiFileAddLine, RawFileAddLine), 'file-off': entry(RiFileLine, RawFileLine), 'image-off': entry(RiImageLine, RawImageLine), @@ -276,6 +286,8 @@ export const ICONS: Record = { 'alert-triangle': entry(RiAlertLine, RawAlertLine), clock: entry(RiTimeLine, RawTimeLine), sparkles: entry(RiSparklingLine, RawSparklingLine), + target: entry(RiTargetLine, RawTargetLine), + pause: entry(RiPauseFill, RawPauseFill), play: entry(RiPlayFill, RawPlayFill), stop: entry(RiStopFill, RawStopFill), star: entry(RiStarFill, RawStarFill), @@ -353,6 +365,7 @@ export const ICON_GROUPS: ReadonlyArray 'folder-solid', 'file', 'file-text', + 'file-edit', 'file-plus', 'file-off', 'image-off', @@ -365,6 +378,7 @@ export const ICON_GROUPS: ReadonlyArray 'check-list', 'bolt', 'git-pull-request', + 'target', 'calendar-schedule', 'calendar-todo', 'calendar-close', @@ -379,6 +393,7 @@ export const ICON_GROUPS: ReadonlyArray 'alert-triangle', 'clock', 'sparkles', + 'pause', 'play', 'stop', 'star', diff --git a/apps/kimi-web/src/lib/toolMeta.ts b/apps/kimi-web/src/lib/toolMeta.ts index 6ecfd4c00..a2b011132 100644 --- a/apps/kimi-web/src/lib/toolMeta.ts +++ b/apps/kimi-web/src/lib/toolMeta.ts @@ -25,6 +25,10 @@ const TOOL_LABEL_KEYS: Record = { task: 'tools.label.task', agentswarm: 'tools.label.swarm', askuserquestion: 'tools.label.ask_user', + creategoal: 'tools.label.goal_create', + getgoal: 'tools.label.goal_get', + setgoalbudget: 'tools.label.goal_budget', + updategoal: 'tools.label.goal_update', }; // --------------------------------------------------------------------------- @@ -60,6 +64,10 @@ const NAME_ALIASES: Record = { subagent: 'task', websearch: 'search', web_search: 'search', + create_goal: 'creategoal', + get_goal: 'getgoal', + set_goal_budget: 'setgoalbudget', + update_goal: 'updategoal', }; export function normalizeToolName(name: string): string { @@ -93,6 +101,10 @@ const TOOL_GLYPH: Record = { task: 'sparkles', agentswarm: 'git-pull-request', askuserquestion: 'help-circle', + creategoal: 'target', + getgoal: 'target', + setgoalbudget: 'target', + updategoal: 'target', // Cron scheduling tools share a calendar motif: schedule / list / cancel. croncreate: 'calendar-schedule', cronlist: 'calendar-todo', @@ -182,6 +194,41 @@ function filePath(d: Record): string | undefined { return str(d.path) ?? str(d.file_path) ?? str(d.filePath) ?? str(d.filename); } +const GOAL_STATUS_KEYS: Record = { + active: 'status.goalStatusActive', + blocked: 'status.goalStatusBlocked', + complete: 'status.goalStatusComplete', +}; + +function goalStatusLabel(value: unknown): string | undefined { + const status = str(value); + if (!status) return undefined; + const key = GOAL_STATUS_KEYS[status]; + return key ? t(key) : status; +} + +function goalBudgetSummary(d: Record): string | undefined { + const value = num(d.value); + const unit = str(d.unit); + if (value === undefined || !unit) return undefined; + switch (unit) { + case 'turns': + return t('tools.goal.turns', { value }); + case 'tokens': + return t('tools.goal.tokens', { value }); + case 'milliseconds': + return t('tools.goal.milliseconds', { value }); + case 'seconds': + return t('tools.goal.seconds', { value }); + case 'minutes': + return t('tools.goal.minutes', { value }); + case 'hours': + return t('tools.goal.hours', { value }); + default: + return t('tools.goal.budget', { value, unit }); + } +} + const BASH_MAX = 64; /** @@ -255,6 +302,27 @@ export function toolSummary(name: string, arg: string, full = false): string { if (items) return c(t('tools.chip.todos', { count: items.length })); return fallback(); } + case 'creategoal': { + if (full) return fallback(); + const objective = str(d.objective); + const criterion = str(d.completionCriterion); + if (objective && criterion) return c(t('tools.goal.objectiveWithCriterion', { objective, criterion })); + return objective ? c(objective) : fallback(); + } + case 'getgoal': { + if (full) return fallback(); + return ''; + } + case 'setgoalbudget': { + if (full) return fallback(); + const summary = goalBudgetSummary(d); + return summary ? c(summary) : fallback(); + } + case 'updategoal': { + if (full) return fallback(); + const status = goalStatusLabel(d.status); + return status ? c(t('tools.goal.status', { status })) : fallback(); + } default: return fallback(); } diff --git a/apps/kimi-web/src/main.ts b/apps/kimi-web/src/main.ts index d546f7ae6..55475a234 100644 --- a/apps/kimi-web/src/main.ts +++ b/apps/kimi-web/src/main.ts @@ -2,7 +2,8 @@ import { createApp } from 'vue'; import App from './App.vue'; import i18n from './i18n'; import { installClientErrorCapture } from './debug/trace'; -import '@fontsource-variable/inter/wght.css'; +import '@fontsource-variable/inter/opsz.css'; +import '@fontsource-variable/inter/opsz-italic.css'; import '@fontsource-variable/jetbrains-mono/wght.css'; import './style.css'; diff --git a/apps/kimi-web/src/style.css b/apps/kimi-web/src/style.css index dda829b72..1f9ae2e50 100644 --- a/apps/kimi-web/src/style.css +++ b/apps/kimi-web/src/style.css @@ -206,9 +206,8 @@ summary { --content-font-size: calc(var(--base-ui-font-size) + 1px); --sidebar-ui-font-size: calc(var(--base-ui-font-size) + 1px); --mono: "JetBrains Mono Variable", "JetBrains Mono", ui-monospace, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; - /* Body/UI font follows the design-system canonical token (system UI font - first; Inter intentionally NOT in the body chain — it stays reserved for - --font-display headings). Mirrors --font-ui so the two can never drift. */ + /* Body/UI font follows the design-system canonical token. Mirrors --font-ui + so the legacy alias can never drift from the current text face. */ --sans: var(--font-ui); color-scheme: light dark; } @@ -454,14 +453,15 @@ html[data-color-scheme="dark"][data-accent="mono"] { --duration-slow: 260ms; /* -- type families ------------------------------------------------------- */ - /* UI/body use the platform's native UI font first (design-system §02); - Inter is intentionally NOT in this chain — it is reserved for - --font-display (optional headings / brand wordmarks) below. */ - --font-ui: -apple-system, BlinkMacSystemFont, "Segoe UI", "PingFang SC", - "Hiragino Sans GB", "Microsoft YaHei", "Source Han Sans SC", "Noto Sans SC", - Roboto, Ubuntu, "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", - "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; - --font-display: "Inter Variable", "Inter", var(--font-ui); + /* UI/body use self-hosted Inter first. CJK and platform system UI families + stay late in the fallback chain so Latin glyphs resolve to Inter while + Chinese text can still fall through to native CJK fonts. */ + --font-ui: "Inter Variable", "Inter", "Helvetica Neue", Arial, + "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", "Source Han Sans SC", + "Noto Sans SC", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, + Ubuntu, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", + "Segoe UI Symbol", "Noto Color Emoji"; + --font-display: var(--font-ui); --font-mono: "JetBrains Mono Variable", "JetBrains Mono", ui-monospace, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; @@ -643,6 +643,7 @@ body { font-size: var(--ui-font-size); font-weight: 400; line-height: 1.6; + font-optical-sizing: auto; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; text-rendering: auto; diff --git a/apps/kimi-web/src/views/DesignSystemView.vue b/apps/kimi-web/src/views/DesignSystemView.vue index 7053cfce0..358d777e2 100644 --- a/apps/kimi-web/src/views/DesignSystemView.vue +++ b/apps/kimi-web/src/views/DesignSystemView.vue @@ -227,18 +227,19 @@ onUnmounted(() => {

All disabled controls use opacity:.5 + cursor:not-allowed uniformly; do not separately grey out or recolor.

Font families

-

Kimi Web uses two font families: --font-ui (UI and body, system fonts first) and --font-mono (code and monospace). Components always reference the variables; do not hard-code font names.

+

Kimi Web uses two font families: --font-ui (UI and body, Inter first) and --font-mono (code and monospace). Components always reference the variables; do not hard-code font names.

-

--font-ui · UI & body (system fonts first)

-

Body and UI use each platform's native UI font — close to the system feel, comfortable for long text and CJK. Fallback chain:

-
--font-ui
--font-ui: -apple-system, BlinkMacSystemFont, "Segoe UI",
+            
--font-ui · UI & body (Inter first)

+            
Body and UI use self-hosted Inter as the primary face. CJK and platform system UI fonts sit late in the fallback chain so Latin glyphs resolve to Inter while Chinese text can fall through to native CJK fonts:

+            
--font-ui
--font-ui: "Inter Variable", "Inter", "Helvetica Neue", Arial,
       "PingFang SC", "Microsoft YaHei", "Noto Sans SC",
-      "Helvetica Neue", Arial, sans-serif,
+      -apple-system, BlinkMacSystemFont, "Segoe UI",
+      Roboto, Ubuntu, sans-serif,
       "Apple Color Emoji", "Segoe UI Emoji", "Noto Color Emoji";

             
    
-              
  • System UI fonts first: SF Pro on macOS / iOS, Segoe UI on Windows.
    • 
-              
  • CJK next: PingFang SC (macOS) / Microsoft YaHei (Windows) / Noto Sans SC (Linux).
    • 
-              
  • Helvetica Neue / Arial / sans-serif as generic fallbacks; emoji fonts at the end.
    • 
+              
  • Inter first: self-hosted Latin UI and body text, loaded through the optical-size normal and italic variable faces.
    • 
+              
  • Western fallbacks next: Helvetica Neue / Arial for environments where Inter cannot load.
    • 
+              
  • CJK and system UI fallbacks late: PingFang SC / Microsoft YaHei / Noto Sans SC, then platform UI fonts and emoji fonts.
    • 
             

 
             
--font-mono · Code & monospace

@@ -251,8 +252,8 @@ onUnmounted(() => {
               FontSourceBundledUsage
               
                 JetBrains Mono@fontsource-variable/jetbrains-mono✓ self-hostedmonospace / code (--font-mono)
-                Inter@fontsource-variable/inter✓ self-hostedoptional: page titles / brand wordmark (--font-display)
-                System UI / CJK fontsoperating system—body / UI (--font-ui), not bundled
+                Inter@fontsource-variable/inter/opsz.css + opsz-italic.css✓ self-hostedUI / body / display (--font-ui, --font-display), wght 100-900, opsz 14-32, normal + italic
+                System UI / CJK fontsoperating system—late fallback for UI / body, not bundled
               
             
             
✓

@@ -262,8 +263,9 @@ onUnmounted(() => {
             
Usage rules

             
    
               
  • Components always use var(--font-ui) / var(--font-mono); do not hard-code font names like 'Inter' / 'JetBrains Mono'.
    • 
-              
  • Body / UI use --font-ui (system fonts first); code / monospace use --font-mono (JetBrains Mono).
    • 
-              
  • Inter is used only for headings / brand scenarios that need a unified look (optional --font-display); it is no longer the body default.
    • 
+              
  • Body / UI use --font-ui (Inter first); code / monospace use --font-mono (JetBrains Mono).
    • 
+              
  • Inter is loaded from the complete optical-size variable faces, including normal and italic styles; font-optical-sizing: auto is enabled globally.
    • 
+              
  • CJK and platform system UI fonts stay late in the --font-ui fallback chain, after Inter and Western fallbacks.
    • 
             

 
             
Type scale & weight

@@ -281,7 +283,7 @@ onUnmounted(() => {
             
-                
+                
diff --git a/flake.nix b/flake.nix
index bdb4158d0..c914a4d56 100644
--- a/flake.nix
+++ b/flake.nix
@@ -152,7 +152,7 @@
               inherit (finalAttrs) pname version src pnpmWorkspaces;
               inherit pnpm;
               fetcherVersion = 3;
-              hash = "sha256-RPjCWL7NqDSKgpHGL16zPlUOfjWN2rkaDY/4GFAD8VA=";
+              hash = "sha256-hUn5Srn3HnEEzU5DLxgjIzFjI0ukM3iSP4QagftEXdE=";
             };
 
             nativeBuildInputs = [
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 7396c9c5c..dab48ecc9 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -166,8 +166,11 @@ importers:
 
   apps/kimi-web:
     dependencies:
+      '@chenglou/pretext':
+        specifier: 0.0.8
+        version: 0.0.8
       '@fontsource-variable/inter':
-        specifier: ^5.2.8
+        specifier: 5.2.8
         version: 5.2.8
       '@fontsource-variable/jetbrains-mono':
         specifier: ^5.2.8
@@ -995,6 +998,9 @@ packages:
   '@chenglou/pretext@0.0.5':
     resolution: {integrity: sha512-A8GZN10REdFGsyuiUgLV8jjPDDFMg5GmgxGWV0I3igxBOnzj+jgz2VMmVD7g+SFyoctfeqHFxbNatKSzVRWtRg==}
 
+  '@chenglou/pretext@0.0.8':
+    resolution: {integrity: sha512-yqm2GMxnPI7VHcHwe84P8ZF0JK/2d2DMKPqMN+s95jQhwDMYYXKVFVJUMEaVWckQStdsjdLav/0Vu+d9YbtGxA==}
+
   '@chevrotain/types@11.1.2':
     resolution: {integrity: sha512-U+HFai5+zmJCkK86QsaJtoITlboZHBqrVketcO2ROv865xfCMSFpELQoz1GkX5GzME8pTa+3kbKrZHQtI0gdbw==}
 
@@ -8077,6 +8083,8 @@ snapshots:
 
   '@chenglou/pretext@0.0.5': {}
 
+  '@chenglou/pretext@0.0.8': {}
+
   '@chevrotain/types@11.1.2': {}
 
   '@colors/colors@1.5.0':

From 9fb19154accf6b6f7abfbf7a9820ccda517bc87e Mon Sep 17 00:00:00 2001
From: Luyu Cheng <2239547+chengluyu@users.noreply.github.com>
Date: Thu, 9 Jul 2026 15:54:50 +0800
Subject: [PATCH 19/24] fix: keep prompt goals running until terminal (#1516)

* fix: keep prompt goals running until terminal

* fix: reject invalid prompt goal commands

* fix: ignore stale prompt goal status checks
---
 .changeset/fix-prompt-goal-mode.md          |   5 +
 apps/kimi-code/src/cli/goal-prompt.ts       |   7 +-
 apps/kimi-code/src/cli/run-prompt.ts        |  70 +++++++++---
 apps/kimi-code/test/cli/goal-prompt.test.ts | 116 ++++++++++++++++++++
 4 files changed, 182 insertions(+), 16 deletions(-)
 create mode 100644 .changeset/fix-prompt-goal-mode.md

diff --git a/.changeset/fix-prompt-goal-mode.md b/.changeset/fix-prompt-goal-mode.md
new file mode 100644
index 000000000..0aa7146f1
--- /dev/null
+++ b/.changeset/fix-prompt-goal-mode.md
@@ -0,0 +1,5 @@
+---
+"@moonshot-ai/kimi-code": patch
+---
+
+Fix prompt-mode goals so they run until completion and report invalid goal commands before sending prompts.
diff --git a/apps/kimi-code/src/cli/goal-prompt.ts b/apps/kimi-code/src/cli/goal-prompt.ts
index 5ab0cfe85..57f223ad4 100644
--- a/apps/kimi-code/src/cli/goal-prompt.ts
+++ b/apps/kimi-code/src/cli/goal-prompt.ts
@@ -46,13 +46,18 @@ const GOAL_PREFIX = /^\/goal(\s|$)/;
  * Parses a headless prompt into a goal-create request, or `undefined` when the
  * prompt is not a `/goal` create command (so the caller runs it as a normal
  * prompt). Non-create goal subcommands are not supported headless and fall
- * through to normal prompt handling.
+ * through to normal prompt handling. Malformed create commands throw instead of
+ * falling through, so validation errors are reported before anything is sent to
+ * the model.
  */
 export function parseHeadlessGoalCreate(prompt: string): HeadlessGoalCreate | undefined {
   const trimmed = prompt.trim();
   if (!GOAL_PREFIX.test(trimmed)) return undefined;
   const args = trimmed.replace(/^\/goal/, '').trim();
   const parsed = parseGoalCommand(args);
+  if (parsed.kind === 'error') {
+    throw new Error(parsed.message);
+  }
   if (parsed.kind !== 'create') return undefined;
   return { objective: parsed.objective, replace: parsed.replace };
 }
diff --git a/apps/kimi-code/src/cli/run-prompt.ts b/apps/kimi-code/src/cli/run-prompt.ts
index 100368203..3bb3a705f 100644
--- a/apps/kimi-code/src/cli/run-prompt.ts
+++ b/apps/kimi-code/src/cli/run-prompt.ts
@@ -234,7 +234,7 @@ async function runHeadlessGoal(
   try {
     // The objective is sent as the normal prompt; goal continuation keeps the
     // turn alive until a terminal state is reached.
-    await runPromptTurn(session, goal.objective, outputFormat, stdout, stderr);
+    await runPromptTurn(session, goal.objective, outputFormat, stdout, stderr, true);
   } finally {
     unsubscribeGoalEvents();
     const snapshot = completedSnapshot ?? (await session.getGoal()).goal;
@@ -432,9 +432,11 @@ function runPromptTurn(
   outputFormat: PromptOutputFormat,
   stdout: PromptOutput,
   stderr: PromptOutput,
+  waitForGoalTerminal = false,
 ): Promise {
   let activeTurnId: number | undefined;
   let activeAgentId: string | undefined;
+  let latestStartedTurnId: number | undefined;
   const outputWriter =
     outputFormat === 'stream-json'
       ? new PromptJsonWriter(stdout)
@@ -469,6 +471,18 @@ function runPromptTurn(
         }
         activeTurnId = event.turnId;
         activeAgentId = event.agentId;
+        latestStartedTurnId = event.turnId;
+        return;
+      }
+      if (
+        waitForGoalTerminal &&
+        event.type === 'goal.updated' &&
+        event.agentId === PROMPT_MAIN_AGENT_ID &&
+        activeTurnId === undefined &&
+        event.snapshot !== null &&
+        event.snapshot.status !== 'active'
+      ) {
+        void finishCompletedTurn();
         return;
       }
       if (
@@ -515,19 +529,29 @@ function runPromptTurn(
           return;
         case 'turn.ended':
           if (event.reason === 'completed') {
-            void (async () => {
-              // Flush the buffered assistant message before draining background
-              // tasks: in stream-json mode the final message is only emitted by
-              // finish(), so a long background wait would otherwise withhold the
-              // main turn's result until the drain settles.
-              outputWriter.flushAssistant();
-              try {
-                await session.waitForBackgroundTasksOnPrint();
-              } catch (error) {
-                log.warn('waitForBackgroundTasksOnPrint failed', { error });
-              }
-              finish();
-            })();
+            outputWriter.flushAssistant();
+            if (waitForGoalTerminal) {
+              const completedTurnId = event.turnId;
+              activeTurnId = undefined;
+              activeAgentId = undefined;
+              void (async () => {
+                try {
+                  const { goal } = await session.getGoal();
+                  if (
+                    activeTurnId !== undefined ||
+                    latestStartedTurnId !== completedTurnId
+                  ) {
+                    return;
+                  }
+                  if (goal?.status === 'active') return;
+                  await finishCompletedTurn();
+                } catch (error) {
+                  finish(error instanceof Error ? error : new Error(String(error)));
+                }
+              })();
+              return;
+            }
+            void finishCompletedTurn();
             return;
           }
           finish(new Error(formatTurnEndedFailure(event)));
@@ -560,6 +584,20 @@ function runPromptTurn(
     session.prompt(prompt).catch((error: unknown) => {
       finish(error instanceof Error ? error : new Error(String(error)));
     });
+
+    async function finishCompletedTurn(): Promise {
+      // Flush the buffered assistant message before draining background tasks:
+      // in stream-json mode the final message is only emitted by finish(), so a
+      // long background wait would otherwise withhold the main turn's result
+      // until the drain settles.
+      outputWriter.flushAssistant();
+      try {
+        await session.waitForBackgroundTasksOnPrint();
+      } catch (error) {
+        log.warn('waitForBackgroundTasksOnPrint failed', { error });
+      }
+      finish();
+    }
   });
 }
 
@@ -610,7 +648,9 @@ class PromptTranscriptWriter implements PromptTurnWriter {
 
   writeToolResult(): void {}
 
-  flushAssistant(): void {}
+  flushAssistant(): void {
+    this.assistantWriter.finish();
+  }
 
   discardAssistant(): void {}
 
diff --git a/apps/kimi-code/test/cli/goal-prompt.test.ts b/apps/kimi-code/test/cli/goal-prompt.test.ts
index 04780bd26..8d75c4721 100644
--- a/apps/kimi-code/test/cli/goal-prompt.test.ts
+++ b/apps/kimi-code/test/cli/goal-prompt.test.ts
@@ -46,6 +46,12 @@ describe('parseHeadlessGoalCreate', () => {
     expect(parseHeadlessGoalCreate('/goal status')).toBeUndefined();
     expect(parseHeadlessGoalCreate('/goal pause')).toBeUndefined();
   });
+
+  it('rejects malformed goal create prompts instead of falling through', () => {
+    expect(() => parseHeadlessGoalCreate(`/goal ${'x'.repeat(4001)}`)).toThrow(
+      'Goal objective is too long',
+    );
+  });
 });
 
 describe('goal summary', () => {
@@ -97,6 +103,7 @@ const mocks = vi.hoisted(() => {
         handler(mainEvent({ type: 'turn.ended', turnId: 1, reason: 'completed' }));
       }
     }),
+    waitForBackgroundTasksOnPrint: vi.fn(async () => {}),
   };
   return {
     session,
@@ -164,6 +171,8 @@ describe('runPrompt headless goal mode', () => {
     mocks.experimentalFeatures = [{ id: 'micro_compaction', enabled: true }];
     mocks.sessions = [];
     mocks.session.createGoal.mockClear();
+    mocks.session.prompt.mockClear();
+    mocks.session.waitForBackgroundTasksOnPrint.mockClear();
     mocks.session.getStatus.mockResolvedValue({ permission: 'auto', model: 'k2' } as never);
     mocks.session.getGoal.mockResolvedValue({ goal: snapshot({ status: 'complete' }) } as never);
   });
@@ -243,6 +252,113 @@ describe('runPrompt headless goal mode', () => {
     expect(mocks.session.prompt).toHaveBeenCalledWith('Ship feature X');
   });
 
+  it('keeps listening across continuation turns until the goal is terminal', async () => {
+    const active = snapshot({ status: 'active', turnsUsed: 1, tokensUsed: 80 });
+    const completed = snapshot({ status: 'complete', turnsUsed: 2, tokensUsed: 160 });
+    mocks.session.getGoal.mockResolvedValueOnce({ goal: active } as never);
+    mocks.session.prompt.mockImplementationOnce(async () => {
+      for (const handler of mocks.eventHandlers) {
+        handler(mocks.mainEvent({ type: 'turn.started', turnId: 1, origin: { kind: 'user' } }));
+        handler(mocks.mainEvent({ type: 'assistant.delta', turnId: 1, delta: '1' }));
+        handler(mocks.mainEvent({ type: 'turn.ended', turnId: 1, reason: 'completed' }));
+      }
+      await Promise.resolve();
+      for (const handler of mocks.eventHandlers) {
+        handler(
+          mocks.mainEvent({
+            type: 'turn.started',
+            turnId: 2,
+            origin: { kind: 'system_trigger', name: 'goal_continuation' },
+          }),
+        );
+        handler(mocks.mainEvent({ type: 'assistant.delta', turnId: 2, delta: '2' }));
+        handler(
+          mocks.mainEvent({
+            type: 'goal.updated',
+            snapshot: completed,
+            change: { kind: 'completion', status: 'complete' },
+          }),
+        );
+        handler(mocks.mainEvent({ type: 'turn.ended', turnId: 2, reason: 'completed' }));
+      }
+    });
+    const stdout = writer();
+    const stderr = writer();
+
+    await runPrompt(opts(), 'test', {
+      stdout,
+      stderr,
+      process: { once: () => {}, off: () => {}, exit: () => undefined as never },
+    });
+
+    expect(stdout.text()).toBe('• 1\n\n• 2\n\n');
+    expect(stderr.text()).toContain('Goal [complete]');
+    expect(stderr.text()).toContain('turns: 2');
+  });
+
+  it('ignores stale goal checks once a continuation turn has started', async () => {
+    const completed = snapshot({ status: 'complete', turnsUsed: 2, tokensUsed: 160 });
+    let resolveFirstGoal: ((value: { goal: null }) => void) | undefined;
+    const firstGoal = new Promise<{ goal: null }>((resolve) => {
+      resolveFirstGoal = resolve;
+    });
+    mocks.session.getGoal
+      .mockImplementationOnce(() => firstGoal as never)
+      .mockResolvedValue({ goal: null } as never);
+    mocks.session.prompt.mockImplementationOnce(async () => {
+      const emit = (event: Record) => {
+        for (const handler of [...mocks.eventHandlers]) {
+          handler(mocks.mainEvent(event));
+        }
+      };
+      emit({ type: 'turn.started', turnId: 1, origin: { kind: 'user' } });
+      emit({ type: 'assistant.delta', turnId: 1, delta: '1' });
+      emit({ type: 'turn.ended', turnId: 1, reason: 'completed' });
+      emit({
+        type: 'turn.started',
+        turnId: 2,
+        origin: { kind: 'system_trigger', name: 'goal_continuation' },
+      });
+      emit({ type: 'assistant.delta', turnId: 2, delta: '2' });
+      emit({
+        type: 'goal.updated',
+        snapshot: completed,
+        change: { kind: 'completion', status: 'complete' },
+      });
+      resolveFirstGoal?.({ goal: null });
+      await Promise.resolve();
+      emit({ type: 'assistant.delta', turnId: 2, delta: ' tail' });
+      emit({ type: 'turn.ended', turnId: 2, reason: 'completed' });
+    });
+    const stdout = writer();
+    const stderr = writer();
+
+    await runPrompt(opts(), 'test', {
+      stdout,
+      stderr,
+      process: { once: () => {}, off: () => {}, exit: () => undefined as never },
+    });
+
+    expect(stdout.text()).toBe('• 1\n\n• 2 tail\n\n');
+    expect(stderr.text()).toContain('Goal [complete]');
+  });
+
+  it('does not send an invalid goal create prompt as a normal prompt', async () => {
+    const stdout = writer();
+    const stderr = writer();
+
+    await expect(
+      runPrompt(opts({ prompt: `/goal ${'x'.repeat(4001)}` }), 'test', {
+        stdout,
+        stderr,
+        process: { once: () => {}, off: () => {}, exit: () => undefined as never },
+      }),
+    ).rejects.toThrow('Goal objective is too long');
+
+    expect(mocks.session.createGoal).not.toHaveBeenCalled();
+    expect(mocks.session.prompt).not.toHaveBeenCalled();
+  });
+
   it('validates the resumed session model before creating a headless goal', async () => {
     mocks.sessions = [{ id: 'ses_goal', workDir: process.cwd() }];
     mocks.session.getStatus.mockResolvedValueOnce({ permission: 'auto', model: '' } as never);

From 173bdfdab1f484ed79927aeaac7dc8116d3fd346 Mon Sep 17 00:00:00 2001
From: Luyu Cheng <2239547+chengluyu@users.noreply.github.com>
Date: Thu, 9 Jul 2026 16:46:57 +0800
Subject: [PATCH 20/24] fix: resume sessions with missing workdir (#1517)

---
 .changeset/fix-missing-workdir-resume.md      |  5 +++++
 packages/agent-core/src/agent/config/index.ts |  2 +-
 .../test/agent/config-state.test.ts           | 19 +++++++++++++++++++
 .../test/tools/fixtures/fake-kaos.ts          |  6 +++---
 4 files changed, 28 insertions(+), 4 deletions(-)
 create mode 100644 .changeset/fix-missing-workdir-resume.md

diff --git a/.changeset/fix-missing-workdir-resume.md b/.changeset/fix-missing-workdir-resume.md
new file mode 100644
index 000000000..187152cf0
--- /dev/null
+++ b/.changeset/fix-missing-workdir-resume.md
@@ -0,0 +1,5 @@
+---
+"@moonshot-ai/kimi-code": patch
+---
+
+Fix resuming sessions whose original working directory no longer exists.
diff --git a/packages/agent-core/src/agent/config/index.ts b/packages/agent-core/src/agent/config/index.ts
index 0eb7c1db1..d4724b6fc 100644
--- a/packages/agent-core/src/agent/config/index.ts
+++ b/packages/agent-core/src/agent/config/index.ts
@@ -48,7 +48,7 @@ export class ConfigState {
     });
     if (changed.cwd) {
       this._cwd = changed.cwd;
-      void this.agent.kaos.chdir(changed.cwd);
+      this.agent.setKaos(this.agent.kaos.withCwd(changed.cwd));
     }
     if (changed.modelAlias) {
       this._modelAlias = changed.modelAlias;
diff --git a/packages/agent-core/test/agent/config-state.test.ts b/packages/agent-core/test/agent/config-state.test.ts
index 32b127950..7a0d169f0 100644
--- a/packages/agent-core/test/agent/config-state.test.ts
+++ b/packages/agent-core/test/agent/config-state.test.ts
@@ -4,8 +4,27 @@ import { emptyUsage } from '@moonshot-ai/kosong';
 import { ProviderManager } from '../../src/session/provider-manager';
 import type { KimiConfig } from '../../src/config';
 import { testAgent } from './harness';
+import { createFakeKaos } from '../tools/fixtures/fake-kaos';
 
 describe('ConfigState model capabilities', () => {
+  it('updates the agent cwd without requiring the directory to exist', () => {
+    const chdir = vi.fn(async () => {
+      throw Object.assign(new Error('missing workspace'), { code: 'ENOENT' });
+    });
+    const ctx = testAgent({
+      kaos: createFakeKaos({
+        getcwd: () => '/workspace',
+        chdir,
+      }),
+    });
+
+    ctx.agent.config.update({ cwd: '/tmp/missing-workdir' });
+
+    expect(ctx.agent.config.cwd).toBe('/tmp/missing-workdir');
+    expect(ctx.agent.kaos.getcwd()).toBe('/tmp/missing-workdir');
+    expect(chdir).not.toHaveBeenCalled();
+  });
+
   it('computes provider and model capabilities from ProviderManager metadata', () => {
     const ctx = testAgent({
       providerManager: new ProviderManager({
diff --git a/packages/agent-core/test/tools/fixtures/fake-kaos.ts b/packages/agent-core/test/tools/fixtures/fake-kaos.ts
index 78613d33a..4384ce348 100644
--- a/packages/agent-core/test/tools/fixtures/fake-kaos.ts
+++ b/packages/agent-core/test/tools/fixtures/fake-kaos.ts
@@ -32,9 +32,9 @@ export function createFakeKaos(
   overrides?: Partial,
   envLayers: readonly Record[] = [],
 ): Kaos {
-  // Hold cwd in a closure so `chdir` (which `config.update({cwd})` now
-  // routes through) can mutate it and later `getcwd()` calls see the
-  // update — mirroring real-kaos semantics without needing a backing fs.
+  // Hold cwd in a closure so tests that call `chdir` directly can mutate it
+  // and later `getcwd()` calls see the update — mirroring real-kaos semantics
+  // without needing a backing fs.
   let cwd = overrides?.getcwd?.() ?? '/workspace';
   const base: Kaos = {
     name: 'fake',

From fe9479d89a22760a5fbe4ef659c0be556c5c3cfd Mon Sep 17 00:00:00 2001
From: Kai 
Date: Thu, 9 Jul 2026 17:06:19 +0800
Subject: [PATCH 21/24] fix: rewrite repeated tool call reminders to redirect
 instead of prohibit (#1518)

The r1/r2/r3 reminders injected into repeated tool results led with
prohibition verdicts and, in r2, echoed the repeated tool name and full
arguments back into the context, reinforcing the very pattern they were
meant to break. Rewrite them to state the situation factually and hand
the model a concrete next action: an expectation-setting sentence for
the next call (r1), a forced decision menu of falsify / ask-user /
conclude (r2), and a final hand-off summary without further tool calls
(r3). Detection, thresholds (3/5/8/12), force-stop, and telemetry are
unchanged.
---
 .../agent-core/src/agent/turn/tool-dedup.ts   | 34 ++++++++-----------
 .../test/agent/turn/tool-dedup.test.ts        | 34 +++++++++----------
 2 files changed, 32 insertions(+), 36 deletions(-)

diff --git a/packages/agent-core/src/agent/turn/tool-dedup.ts b/packages/agent-core/src/agent/turn/tool-dedup.ts
index 31170f8cf..605408777 100644
--- a/packages/agent-core/src/agent/turn/tool-dedup.ts
+++ b/packages/agent-core/src/agent/turn/tool-dedup.ts
@@ -7,32 +7,28 @@ import { canonicalTelemetryArgs } from './canonical-args';
 
 const REMINDER_TEXT_1 =
   '\n\n\n' +
-  'You are repeating the exact same tool call with identical parameters.' +
-  ' Please carefully analyze the previous result. If the task is not yet complete,' +
-  ' try a different method or parameters instead of repeating the same call.' +
+  'The same tool call has been repeated several times in a row. ' +
+  'Before making your next call, write one sentence stating what new information you expect it to produce. ' +
+  'Then act on that sentence: if it names something this result does not already give you, choose the action that best provides it; otherwise, continue with the evidence you already have.' +
   '\n';
 
-function makeReminderText2(toolName: string, repeatCount: number, args: unknown): string {
-  const argsStr = canonicalTelemetryArgs(args);
+function makeReminderText2(repeatCount: number): string {
   return (
     '\n\n\n' +
-    'You have repeatedly called the same tool with identical parameters many times.\n' +
-    'Repeated tool call detected:\n' +
-    `- tool: ${toolName}\n` +
-    `- repeated_times: ${String(repeatCount)}\n` +
-    `- arguments: ${argsStr}\n` +
-    'The previous repeated calls did not make progress. Do not call this exact same tool with the exact same arguments again.\n' +
-    'Carefully inspect the latest tool result and choose a different next action, different parameters, or finish the task if enough evidence has been gathered.' +
+    `The same tool call has now been issued ${String(repeatCount)} times in a row. ` +
+    'Choose exactly one of the following and state your choice before acting:\n' +
+    '(1) Falsification check: run the cheapest test that could conclusively disprove your current approach, if such a test exists.\n' +
+    '(2) Missing input: tell the user precisely what information or decision you need to proceed, and ask for it.\n' +
+    '(3) Conclude: deliver your best result based on the evidence already gathered, listing anything that remains uncertain.' +
     '\n'
   );
 }
 
 const REMINDER_TEXT_3 =
   '\n\n\n' +
-  'You are stuck in a dead end and have repeatedly made the same function call without progress.\n' +
-  'Stop all function calls immediately. Do not call any tool in your next response.\n' +
-  'In analysis, review the current execution state and identify why progress is blocked.\n' +
-  'Then return a text-only summary to the user that reports the current problem, what has already been tried, and what information or decision is needed next.' +
+  'Write your final response now, without any further tool calls. ' +
+  'Cover: the current blocker, each approach you have tried and what it established, and the specific information or decision you need from the user to unblock progress. ' +
+  'Text only.' +
   '\n';
 
 const REPEAT_REMINDER_1_START = 3;
@@ -105,8 +101,8 @@ const DEDUP_PLACEHOLDER_RESULT: ExecutableToolResult = { output: '' };
  * - Cross-step dedup: when the exact same call is repeated consecutively
  *   across steps, the result returned to the model is suffixed with a system
  *   reminder once the streak hits 3. The reminder escalates as the streak
- *   grows: r1 (gentle nudge) from streak 3, r2 (concrete repeat report) from
- *   streak 5, r3 (dead-end stop instruction) from streak 8. From streak 12
+ *   grows: r1 (expectation-setting nudge) from streak 3, r2 (forced decision
+ *   menu) from streak 5, r3 (final hand-off instruction) from streak 8. From streak 12
  *   onward the turn is force-stopped via `{ stopTurn: true }` so the loop
  *   cannot keep spinning on the same call. Force-stop does not flip a
  *   successful tool result into an error — the underlying tool's `isError`
@@ -239,7 +235,7 @@ export class ToolCallDeduplicator {
       finalResult = appendReminder(result, REMINDER_TEXT_3);
       action = 'r3';
     } else if (streak >= REPEAT_REMINDER_2_START) {
-      finalResult = appendReminder(result, makeReminderText2(toolName, streak, args));
+      finalResult = appendReminder(result, makeReminderText2(streak));
       action = 'r2';
     } else if (streak >= REPEAT_REMINDER_1_START) {
       finalResult = appendReminder(result, REMINDER_TEXT_1);
diff --git a/packages/agent-core/test/agent/turn/tool-dedup.test.ts b/packages/agent-core/test/agent/turn/tool-dedup.test.ts
index cff03b9bb..087831645 100644
--- a/packages/agent-core/test/agent/turn/tool-dedup.test.ts
+++ b/packages/agent-core/test/agent/turn/tool-dedup.test.ts
@@ -116,8 +116,8 @@ describe('ToolCallDeduplicator', () => {
         dedup.endStep();
       }
       expect(last!.output as string).toContain('');
-      expect(last!.output as string).toContain('repeating the exact same tool call');
-      expect(last!.output as string).not.toContain('repeated_times');
+      expect(last!.output as string).toContain('what new information you expect');
+      expect(last!.output as string).not.toContain('Choose exactly one');
     });
 
     it('keeps injecting reminder1 at 4 consecutive', async () => {
@@ -129,7 +129,7 @@ describe('ToolCallDeduplicator', () => {
         dedup.endStep();
       }
       expect(last!.output as string).toContain('');
-      expect(last!.output as string).toContain('repeating the exact same tool call');
+      expect(last!.output as string).toContain('what new information you expect');
     });
 
     it('injects reminder2 at exactly 5 consecutive', async () => {
@@ -141,9 +141,9 @@ describe('ToolCallDeduplicator', () => {
         dedup.endStep();
       }
       expect(last!.output as string).toContain('');
-      expect(last!.output as string).toContain('repeated_times: 5');
-      expect(last!.output as string).toContain('tool: Read');
-      expect(last!.output as string).toContain('arguments:');
+      expect(last!.output as string).toContain('issued 5 times in a row');
+      expect(last!.output as string).toContain('Choose exactly one of the following');
+      expect(last!.output as string).toContain('Falsification check');
     });
 
     it.each([6, 7])('keeps injecting reminder2 at %i consecutive', async (streak) => {
@@ -155,8 +155,8 @@ describe('ToolCallDeduplicator', () => {
         dedup.endStep();
       }
       expect(last!.output as string).toContain('');
-      expect(last!.output as string).toContain(`repeated_times: ${String(streak)}`);
-      expect(last!.output as string).toContain('tool: Read');
+      expect(last!.output as string).toContain(`issued ${String(streak)} times in a row`);
+      expect(last!.output as string).toContain('Choose exactly one of the following');
     });
 
     it('injects the dead-end reminder at exactly 8 consecutive', async () => {
@@ -168,7 +168,7 @@ describe('ToolCallDeduplicator', () => {
         dedup.endStep();
       }
       expect(last!.output as string).toContain('');
-      expect(last!.output as string).toContain('stuck in a dead end');
+      expect(last!.output as string).toContain('without any further tool calls');
     });
 
     it('resets streak when a different call is interleaved', async () => {
@@ -214,9 +214,9 @@ describe('ToolCallDeduplicator', () => {
       dedup.endStep();
 
       expect(original.output as string).toContain('');
-      expect(original.output as string).toContain('repeating the exact same tool call');
+      expect(original.output as string).toContain('what new information you expect');
       expect(finalDup.output as string).toContain('');
-      expect(finalDup.output as string).toContain('repeating the exact same tool call');
+      expect(finalDup.output as string).toContain('what new information you expect');
     });
 
     it('same-step spam alone does not trigger reminder', async () => {
@@ -273,7 +273,7 @@ describe('ToolCallDeduplicator', () => {
       const arr = final.output as Array<{ type: string; text: string }>;
       expect(arr).toHaveLength(1);
       expect(arr[0]!.type).toBe('text');
-      expect(arr[0]!.text).toBe('hello' + makeReminderText2('X', 5, { a: 1 }));
+      expect(arr[0]!.text).toBe('hello' + makeReminderText2(5));
     });
 
     it('pushes a new text part when trailing part is non-text', async () => {
@@ -408,8 +408,8 @@ describe('ToolCallDeduplicator', () => {
       const dedup = new ToolCallDeduplicator();
       const last = await runStreak(dedup, 8);
       expect(last.output as string).toContain('');
-      expect(last.output as string).toContain('stuck in a dead end');
-      expect(last.output as string).toContain('Stop all function calls immediately');
+      expect(last.output as string).toContain('Write your final response now');
+      expect(last.output as string).toContain('without any further tool calls');
       // 8 is the reminder threshold, not yet force-stop.
       expect(last.isError).toBeUndefined();
       expect(stopTurnOf(last)).toBeUndefined();
@@ -420,7 +420,7 @@ describe('ToolCallDeduplicator', () => {
       async (streak) => {
         const dedup = new ToolCallDeduplicator();
         const last = await runStreak(dedup, streak);
-        expect(last.output as string).toContain('stuck in a dead end');
+        expect(last.output as string).toContain('Write your final response now');
         expect(last.isError).toBeUndefined();
         expect(stopTurnOf(last)).toBeUndefined();
       },
@@ -429,7 +429,7 @@ describe('ToolCallDeduplicator', () => {
     it('force-stops the turn at exactly 12 consecutive without marking the tool failed', async () => {
       const dedup = new ToolCallDeduplicator();
       const last = await runStreak(dedup, 12);
-      expect(last.output as string).toContain('stuck in a dead end');
+      expect(last.output as string).toContain('Write your final response now');
       // The underlying tool succeeded — force-stop must not flip it to error.
       expect(last.isError).toBeUndefined();
       expect(stopTurnOf(last)).toBe(true);
@@ -459,7 +459,7 @@ describe('ToolCallDeduplicator', () => {
       // The underlying tool was an error — that must survive force-stop.
       expect(last!.isError).toBe(true);
       expect(stopTurnOf(last!)).toBe(true);
-      expect(last!.output as string).toContain('stuck in a dead end');
+      expect(last!.output as string).toContain('Write your final response now');
     });
   });
 

From b91099ed7a2590d1afa4d6e3675671da52b7661c Mon Sep 17 00:00:00 2001
From: liruifengv 
Date: Thu, 9 Jul 2026 17:44:59 +0800
Subject: [PATCH 22/24] feat: display Extra Usage fuel pack balance in /usage
 and /status (#1501)

* feat(oauth): parse boosterWallet extra usage from /usages

* feat(oauth): expose extraUsage on AuthManagedUsageResult

* feat(kimi-code): render Extra Usage section in /usage panel

* fix(kimi-code): address Task 3 review feedback for extra usage section

* feat(kimi-code): render Extra Usage section in /status panel

* feat(kimi-code): wire extraUsage into /usage and /status commands

* chore(extra-usage): address final review findings for fuel pack feature

- Update changeset to cover both kimi-code and kimi-code-sdk packages

- Add parser clamp tests and toolkit null-case test

- Replace 'as never' casts in usage-panel tests

- Wrap long import line in status-panel

* chore: temporarily log /usages raw response for debugging

* fix(oauth): accept BOOSTER balance type and drop debug log

* fix(oauth): drop reset hint from Extra Usage and revert periodEnd parsing

* fix(oauth): treat missing amountLeft as zero extra usage and drop debug log

* revert: keep missing amountLeft defaulting to 0 (fully used)

* feat(extra-usage): show monthly cap usage bar and balance in /usage and /status

* fix(extra-usage): show balance and unlimited marker when no monthly cap

* fix(extra-usage): show monthly used with unlimited marker and balance

* fix(extra-usage): label Used and use English Unlimited

* feat(extra-usage): render balance, monthly used and monthly limit as labeled rows

* fix(extra-usage): move Balance row to the bottom

* fix(extra-usage): format currency values with two decimals for column alignment

* fix(extra-usage): right-align currency values so numbers line up

* fix(extra-usage): align currency symbol and decimal point in usage rows
---
 .changeset/expose-extrausage-toolkit.md       |   6 +
 apps/kimi-code/src/tui/commands/info.ts       |   2 +-
 .../tui/components/messages/status-panel.ts   |  17 ++-
 .../tui/components/messages/usage-panel.ts    | 111 +++++++++++++-
 .../components/messages/status-panel.test.ts  |  38 +++++
 .../components/messages/usage-panel.test.ts   | 138 +++++++++++++++++-
 packages/oauth/src/managed-usage.ts           |  69 ++++++++-
 packages/oauth/src/toolkit.ts                 |   2 +
 packages/oauth/test/managed-usage.test.ts     |  72 ++++++++-
 packages/oauth/test/toolkit.test.ts           |  73 +++++++++
 10 files changed, 517 insertions(+), 11 deletions(-)
 create mode 100644 .changeset/expose-extrausage-toolkit.md

diff --git a/.changeset/expose-extrausage-toolkit.md b/.changeset/expose-extrausage-toolkit.md
new file mode 100644
index 000000000..d31c53ee5
--- /dev/null
+++ b/.changeset/expose-extrausage-toolkit.md
@@ -0,0 +1,6 @@
+---
+"@moonshot-ai/kimi-code": patch
+"@moonshot-ai/kimi-code-sdk": patch
+---
+
+Display Extra Usage (fuel pack) balance in `/usage` and `/status` commands.
diff --git a/apps/kimi-code/src/tui/commands/info.ts b/apps/kimi-code/src/tui/commands/info.ts
index 51ccd3fc1..fd5d397f4 100644
--- a/apps/kimi-code/src/tui/commands/info.ts
+++ b/apps/kimi-code/src/tui/commands/info.ts
@@ -213,5 +213,5 @@ async function loadManagedUsageReport(host: SlashCommandHost): Promise 0) {
+    lines.push('');
+    lines.push(...extraSection);
+  }
+
   return lines;
 }
diff --git a/apps/kimi-code/src/tui/components/messages/usage-panel.ts b/apps/kimi-code/src/tui/components/messages/usage-panel.ts
index 23cef0b27..195860bc3 100644
--- a/apps/kimi-code/src/tui/components/messages/usage-panel.ts
+++ b/apps/kimi-code/src/tui/components/messages/usage-panel.ts
@@ -30,9 +30,19 @@ export interface ManagedUsageRow {
   readonly resetHint?: string;
 }
 
+export interface BoosterWalletInfo {
+  readonly balanceCents: number;
+  readonly totalCents: number;
+  readonly monthlyChargeLimitEnabled: boolean;
+  readonly monthlyChargeLimitCents: number;
+  readonly monthlyUsedCents: number;
+  readonly currency: string;
+}
+
 export interface ManagedUsageReport {
   readonly summary: ManagedUsageRow | null;
   readonly limits: readonly ManagedUsageRow[];
+  readonly extraUsage?: BoosterWalletInfo | null;
 }
 
 export interface UsageReportOptions {
@@ -121,8 +131,7 @@ function buildManagedUsageSection(
     r.limit > 0 ? Math.max(0, Math.min(r.used / r.limit, 1)) : 0;
   const labelWidth = Math.max(10, ...rows.map((r) => r.label.length));
   const pctWidth = Math.max(...rows.map((r) => `${Math.round(usedRatio(r) * 100)}% used`.length));
-  const severityColor = (sev: 'ok' | 'warn' | 'danger'): 'success' | 'warning' | 'error' =>
-    sev === 'danger' ? 'error' : sev === 'warn' ? 'warning' : 'success';
+
   const out: string[] = [accent('Plan usage')];
   for (const row of rows) {
     const ratioUsed = usedRatio(row);
@@ -136,6 +145,91 @@ function buildManagedUsageSection(
   return out;
 }
 
+function severityColor(sev: 'ok' | 'warn' | 'danger'): 'success' | 'warning' | 'error' {
+  return sev === 'danger' ? 'error' : sev === 'warn' ? 'warning' : 'success';
+}
+
+function currencySymbol(currency: string): string {
+  switch (currency.toUpperCase()) {
+    case 'CNY':
+      return '¥';
+    case 'USD':
+      return '$';
+    default:
+      return '';
+  }
+}
+
+interface CurrencyParts {
+  readonly symbol: string;
+  readonly number: string;
+}
+
+function formatCurrencyParts(cents: number, currency: string): CurrencyParts {
+  const symbol = currencySymbol(currency);
+  const main = cents / 100;
+  const formatted = main.toFixed(2);
+  return symbol.length > 0
+    ? { symbol, number: formatted }
+    : { symbol: '', number: `${formatted} ${currency}` };
+}
+
+export function buildExtraUsageSection(
+  extraUsage: BoosterWalletInfo | undefined | null,
+  accent: Colorize,
+  value: Colorize,
+  muted: Colorize,
+): string[] {
+  if (extraUsage === undefined || extraUsage === null) return [];
+
+  const hasMonthlyLimit =
+    extraUsage.monthlyChargeLimitEnabled && extraUsage.monthlyChargeLimitCents > 0;
+
+  const balance = formatCurrencyParts(extraUsage.balanceCents, extraUsage.currency);
+  const used = formatCurrencyParts(extraUsage.monthlyUsedCents, extraUsage.currency);
+  const rows: Array<{ label: string; symbol: string; number: string }> = [];
+  let barLine: string | null = null;
+
+  if (hasMonthlyLimit) {
+    const ratio = Math.max(
+      0,
+      Math.min(extraUsage.monthlyUsedCents / extraUsage.monthlyChargeLimitCents, 1),
+    );
+    const bar = renderProgressBar(ratio, 20);
+    barLine = `  ${currentTheme.fg(severityColor(ratioSeverity(ratio)), bar)}`;
+    const limit = formatCurrencyParts(extraUsage.monthlyChargeLimitCents, extraUsage.currency);
+    rows.push({ label: 'Used this month', ...used });
+    rows.push({ label: 'Monthly limit', ...limit });
+    rows.push({ label: 'Balance', ...balance });
+  } else {
+    rows.push({ label: 'Used this month', ...used });
+    rows.push({ label: 'Monthly limit', symbol: '', number: 'Unlimited' });
+    rows.push({ label: 'Balance', ...balance });
+  }
+
+  // `Used this month` is the longest label; size the column to the widest label
+  // so the currency symbol starts in the same column on every row.
+  const labelWidth = Math.max(...rows.map((r) => r.label.length));
+  // Right-align the numeric part of currency rows against each other so the
+  // decimal points line up (e.g. `¥ 50.00` / `¥200.00`). Text-only rows such as
+  // `Unlimited` carry no currency symbol, so they must not widen the numeric
+  // column — otherwise money values get padded with stray spaces.
+  const numberWidth = Math.max(
+    0,
+    ...rows.filter((r) => r.symbol.length > 0).map((r) => visibleWidth(r.number)),
+  );
+  const row = (label: string, symbol: string, number: string): string => {
+    const cell = symbol.length > 0 ? symbol + number.padStart(numberWidth, ' ') : number;
+    return `  ${muted(label.padEnd(labelWidth, ' '))}  ${value(cell)}`;
+  };
+
+  const lines: string[] = [accent('Extra Usage')];
+  if (barLine !== null) lines.push(barLine);
+  for (const r of rows) lines.push(row(r.label, r.symbol, r.number));
+
+  return lines;
+}
+
 export function buildManagedUsageReportLines(options: ManagedUsageReportLineOptions): string[] {
   const accent = (text: string) => currentTheme.boldFg('primary', text);
   const value = (text: string) => currentTheme.fg('text', text);
@@ -157,8 +251,6 @@ export function buildUsageReportLines(options: UsageReportOptions): string[] {
   const value = (text: string) => currentTheme.fg('text', text);
   const muted = (text: string) => currentTheme.fg('textDim', text);
   const errorStyle = (text: string) => currentTheme.fg('error', text);
-  const severityColor = (sev: 'ok' | 'warn' | 'danger'): 'success' | 'warning' | 'error' =>
-    sev === 'danger' ? 'error' : sev === 'warn' ? 'warning' : 'success';
 
   const lines: string[] = [
     accent('Session usage'),
@@ -197,6 +289,17 @@ export function buildUsageReportLines(options: UsageReportOptions): string[] {
     lines.push(...managedSection);
   }
 
+  const extraSection = buildExtraUsageSection(
+    options.managedUsage?.extraUsage,
+    accent,
+    value,
+    muted,
+  );
+  if (extraSection.length > 0) {
+    lines.push('');
+    lines.push(...extraSection);
+  }
+
   return lines;
 }
 
diff --git a/apps/kimi-code/test/tui/components/messages/status-panel.test.ts b/apps/kimi-code/test/tui/components/messages/status-panel.test.ts
index 7d27be8e2..041860896 100644
--- a/apps/kimi-code/test/tui/components/messages/status-panel.test.ts
+++ b/apps/kimi-code/test/tui/components/messages/status-panel.test.ts
@@ -68,6 +68,44 @@ describe('status panel report lines', () => {
     expect(output).not.toContain('Runtime');
   });
 
+  it('formats extra usage section in status report', () => {
+    const lines = buildStatusReportLines({
+      version: '1.2.3',
+      model: 'k2',
+      workDir: '/tmp/project',
+      sessionId: 'ses-1',
+      sessionTitle: null,
+      thinkingEffort: 'off',
+      permissionMode: 'manual',
+      planMode: false,
+      contextUsage: 0,
+      contextTokens: 0,
+      maxContextTokens: 0,
+      availableModels: {},
+      managedUsage: {
+        summary: null,
+        limits: [],
+        extraUsage: {
+          balanceCents: 15000,
+          totalCents: 20000,
+          monthlyChargeLimitEnabled: true,
+          monthlyChargeLimitCents: 20000,
+          monthlyUsedCents: 5000,
+          currency: 'USD',
+        },
+      },
+    }).map(strip);
+
+    const output = lines.join('\n');
+    expect(output).toContain('Extra Usage');
+    expect(output).toContain('Balance');
+    expect(output).toContain('150.00');
+    expect(output).toContain('Used this month');
+    expect(output).toContain('50.00');
+    expect(output).toContain('Monthly limit');
+    expect(output).toContain('200.00');
+  });
+
   it('falls back to app state and shows status load errors as warnings', () => {
     const lines = buildStatusReportLines({
       version: '1.2.3',
diff --git a/apps/kimi-code/test/tui/components/messages/usage-panel.test.ts b/apps/kimi-code/test/tui/components/messages/usage-panel.test.ts
index ff39cb7e6..cf2598f82 100644
--- a/apps/kimi-code/test/tui/components/messages/usage-panel.test.ts
+++ b/apps/kimi-code/test/tui/components/messages/usage-panel.test.ts
@@ -24,7 +24,7 @@ describe('UsagePanelComponent', () => {
             output: 250,
           },
         },
-      } as never,
+      },
       contextUsage: 0.25,
       contextTokens: 2500,
       maxContextTokens: 10000,
@@ -48,6 +48,142 @@ describe('UsagePanelComponent', () => {
     expect(lines.join('\n')).toContain('resets tomorrow');
   });
 
+  it('formats extra usage with a monthly limit', () => {
+    const lines = buildUsageReportLines({
+      sessionUsage: { byModel: {} },
+      contextUsage: 0,
+      contextTokens: 0,
+      maxContextTokens: 0,
+      managedUsage: {
+        summary: null,
+        limits: [],
+        extraUsage: {
+          balanceCents: 10000,
+          totalCents: 20000,
+          monthlyChargeLimitEnabled: true,
+          monthlyChargeLimitCents: 20000,
+          monthlyUsedCents: 5000,
+          currency: 'USD',
+        },
+      },
+    }).map(strip);
+
+    const output = lines.join('\n');
+    expect(lines).toContain('Extra Usage');
+    expect(output).toContain('Balance');
+    expect(output).toContain('100.00');
+    expect(output).toContain('Used this month');
+    expect(output).toContain('50.00');
+    expect(output).toContain('Monthly limit');
+    expect(output).toContain('200.00');
+    // bar row contains block glyphs but no percentage text
+    expect(output).toContain('░');
+  });
+
+  it('formats extra usage without a monthly limit and omits the progress bar', () => {
+    const lines = buildUsageReportLines({
+      sessionUsage: { byModel: {} },
+      contextUsage: 0,
+      contextTokens: 0,
+      maxContextTokens: 0,
+      managedUsage: {
+        summary: null,
+        limits: [],
+        extraUsage: {
+          balanceCents: 18208,
+          totalCents: 40000,
+          monthlyChargeLimitEnabled: false,
+          monthlyChargeLimitCents: 0,
+          monthlyUsedCents: 21792,
+          currency: 'CNY',
+        },
+      },
+    }).map(strip);
+
+    const output = lines.join('\n');
+    expect(lines).toContain('Extra Usage');
+    expect(output).toContain('Balance');
+    expect(output).toContain('¥182.08');
+    expect(output).toContain('Used this month');
+    expect(output).toContain('¥217.92');
+    expect(output).toContain('Monthly limit');
+    expect(output).toContain('Unlimited');
+    expect(output).not.toContain('░');
+    expect(output).not.toContain('█');
+  });
+
+  it('omits the extra usage section when extraUsage is omitted or null', () => {
+    for (const extraUsage of [undefined, null]) {
+      const lines = buildUsageReportLines({
+        sessionUsage: { byModel: {} },
+        contextUsage: 0,
+        contextTokens: 0,
+        maxContextTokens: 0,
+        managedUsage: { summary: null, limits: [], extraUsage },
+      }).map(strip);
+
+      expect(lines).not.toContain('Extra Usage');
+    }
+  });
+
+  it('formats extra usage with CNY currency', () => {
+    const lines = buildUsageReportLines({
+      sessionUsage: { byModel: {} },
+      contextUsage: 0,
+      contextTokens: 0,
+      maxContextTokens: 0,
+      managedUsage: {
+        summary: null,
+        limits: [],
+        extraUsage: {
+          balanceCents: 10000,
+          totalCents: 20000,
+          monthlyChargeLimitEnabled: true,
+          monthlyChargeLimitCents: 20000,
+          monthlyUsedCents: 5000,
+          currency: 'CNY',
+        },
+      },
+    }).map(strip);
+
+    const output = lines.join('\n');
+    expect(output).toContain('Balance');
+    expect(output).toContain('100.00');
+    expect(output).toContain('Used this month');
+    expect(output).toContain('50.00');
+    expect(output).toContain('Monthly limit');
+    expect(output).toContain('200.00');
+  });
+
+  it('aligns the currency symbol and decimal point across extra usage rows', () => {
+    const lines = buildUsageReportLines({
+      sessionUsage: { byModel: {} },
+      contextUsage: 0,
+      contextTokens: 0,
+      maxContextTokens: 0,
+      managedUsage: {
+        summary: null,
+        limits: [],
+        extraUsage: {
+          balanceCents: 15901,
+          totalCents: 300000,
+          monthlyChargeLimitEnabled: true,
+          monthlyChargeLimitCents: 300000,
+          monthlyUsedCents: 24099,
+          currency: 'CNY',
+        },
+      },
+    }).map(strip);
+
+    const extraRows = lines.filter((line) => line.includes('¥'));
+    expect(extraRows).toHaveLength(3);
+    // The currency symbol stays in one column...
+    expect(new Set(extraRows.map((line) => line.indexOf('¥'))).size).toBe(1);
+    // ...and the right-aligned numeric parts end in the same column, so the
+    // decimal points line up across rows.
+    expect(new Set(extraRows.map((line) => line.length)).size).toBe(1);
+  });
+
   it('wraps preformatted usage lines in a bordered panel', () => {
     const component = new UsagePanelComponent(() => ['Session usage'], 'primary');
     const output = component.render(80).map(strip);
diff --git a/packages/oauth/src/managed-usage.ts b/packages/oauth/src/managed-usage.ts
index 534104a20..928d9008c 100644
--- a/packages/oauth/src/managed-usage.ts
+++ b/packages/oauth/src/managed-usage.ts
@@ -45,14 +45,78 @@ export interface UsageRow {
   readonly resetHint?: string | undefined;
 }
 
+export interface BoosterWalletInfo {
+  /** Remaining balance in whole cents (from balance.amountLeft). */
+  readonly balanceCents: number;
+  /** Total balance in whole cents (from balance.amount). */
+  readonly totalCents: number;
+  /** Whether the user enabled a monthly spending cap. */
+  readonly monthlyChargeLimitEnabled: boolean;
+  /** Monthly spending cap in whole cents; 0 means unlimited. */
+  readonly monthlyChargeLimitCents: number;
+  /** Monthly spend so far in whole cents. */
+  readonly monthlyUsedCents: number;
+  /** ISO currency code, e.g. USD / CNY. */
+  readonly currency: string;
+}
+
 export interface ParsedManagedUsage {
   readonly summary: UsageRow | null;
   readonly limits: UsageRow[];
+  readonly extraUsage: BoosterWalletInfo | null;
+}
+
+const FIXED_POINT_CENTS = 1_000_000;
+
+function fixedPointToCents(value: number): number {
+  const cents = value / FIXED_POINT_CENTS;
+  if (cents > 0 && cents < 1) return 1;
+  return Math.round(cents);
+}
+
+function parseMoney(raw: unknown): { cents: number; currency: string } | null {
+  if (!isRecord(raw)) return null;
+  const cents = toInt(raw['priceInCents']);
+  if (cents === null) return null;
+  const currency = typeof raw['currency'] === 'string' ? raw['currency'] : '';
+  return { cents, currency };
+}
+
+function parseBoosterWallet(raw: unknown): BoosterWalletInfo | null {
+  if (!isRecord(raw)) return null;
+  const balance = raw['balance'];
+  if (!isRecord(balance)) return null;
+  if (balance['type'] !== 'BOOSTER') return null;
+  const amountRaw = toInt(balance['amount']);
+  if (amountRaw === null || amountRaw <= 0) return null;
+  const totalCents = fixedPointToCents(amountRaw);
+  const amountLeftRaw = toInt(balance['amountLeft']);
+  const balanceCents = amountLeftRaw !== null ? fixedPointToCents(amountLeftRaw) : 0;
+
+  const monthlyLimit = parseMoney(raw['monthlyChargeLimit']);
+  const monthlyUsed = parseMoney(raw['monthlyUsed']);
+  const monthlyChargeLimitEnabled = raw['monthlyChargeLimitEnabled'] === true;
+
+  const currency =
+    monthlyLimit && monthlyLimit.currency.length > 0
+      ? monthlyLimit.currency
+      : monthlyUsed && monthlyUsed.currency.length > 0
+        ? monthlyUsed.currency
+        : 'USD';
+
+  return {
+    balanceCents,
+    totalCents,
+    monthlyChargeLimitEnabled,
+    monthlyChargeLimitCents: monthlyLimit?.cents ?? 0,
+    monthlyUsedCents: monthlyUsed?.cents ?? 0,
+    currency,
+  };
 }
 
 export function parseManagedUsagePayload(payload: unknown): ParsedManagedUsage {
   if (typeof payload !== 'object' || payload === null) {
-    return { summary: null, limits: [] };
+    return { summary: null, limits: [], extraUsage: null };
   }
   const rec = payload as Record;
   const summary = toUsageRow(rec['usage'], 'Weekly limit');
@@ -71,7 +135,8 @@ export function parseManagedUsagePayload(payload: unknown): ParsedManagedUsage {
       if (row !== null) limits.push(row);
     }
   }
-  return { summary, limits };
+  const extraUsage = parseBoosterWallet(rec['boosterWallet']);
+  return { summary, limits, extraUsage };
 }
 
 function toUsageRow(raw: unknown, defaultLabel: string): UsageRow | null {
diff --git a/packages/oauth/src/toolkit.ts b/packages/oauth/src/toolkit.ts
index 94dca02bf..0229db7b0 100644
--- a/packages/oauth/src/toolkit.ts
+++ b/packages/oauth/src/toolkit.ts
@@ -97,6 +97,7 @@ export type AuthManagedUsageResult =
       readonly kind: 'ok';
       readonly summary: ParsedManagedUsage['summary'];
       readonly limits: ParsedManagedUsage['limits'];
+      readonly extraUsage: ParsedManagedUsage['extraUsage'];
     }
   | FetchManagedUsageError;
 
@@ -291,6 +292,7 @@ export class KimiOAuthToolkit {
         kind: 'ok',
         summary: result.parsed.summary,
         limits: result.parsed.limits,
+        extraUsage: result.parsed.extraUsage,
       };
     } catch (error) {
       return {
diff --git a/packages/oauth/test/managed-usage.test.ts b/packages/oauth/test/managed-usage.test.ts
index 98d33ee17..f390653bc 100644
--- a/packages/oauth/test/managed-usage.test.ts
+++ b/packages/oauth/test/managed-usage.test.ts
@@ -25,8 +25,8 @@ describe('isManagedKimiCode', () => {
 
 describe('parseManagedUsagePayload', () => {
   it('returns empty when payload is not an object', () => {
-    expect(parseManagedUsagePayload(null)).toEqual({ summary: null, limits: [] });
-    expect(parseManagedUsagePayload('nope')).toEqual({ summary: null, limits: [] });
+    expect(parseManagedUsagePayload(null)).toEqual({ summary: null, limits: [], extraUsage: null });
+    expect(parseManagedUsagePayload('nope')).toEqual({ summary: null, limits: [], extraUsage: null });
   });
 
   it('extracts a summary from the `usage` object', () => {
@@ -74,6 +74,73 @@ describe('parseManagedUsagePayload', () => {
     const parsed = parseManagedUsagePayload({ usage: { used: 1, limit: 10, resetAt: future } });
     expect(parsed.summary?.resetHint).toMatch(/resets in/);
   });
+
+  it('extracts extra usage from boosterWallet.balance', () => {
+    const parsed = parseManagedUsagePayload({
+      usage: { used: 40, limit: 1000, name: 'Weekly limit' },
+      boosterWallet: {
+        id: 'wallet_1',
+        balance: {
+          type: 'BOOSTER',
+          amount: '20000000000',
+          amountLeft: '10000000000',
+          unit: 'UNIT_CURRENCY',
+        },
+        monthlyChargeLimitEnabled: true,
+        monthlyChargeLimit: { currency: 'USD', priceInCents: '20000' },
+        monthlyUsed: { currency: 'USD', priceInCents: '5000' },
+      },
+    });
+    expect(parsed.extraUsage).toEqual({
+      balanceCents: 10000,
+      totalCents: 20000,
+      monthlyChargeLimitEnabled: true,
+      monthlyChargeLimitCents: 20000,
+      monthlyUsedCents: 5000,
+      currency: 'USD',
+    });
+  });
+
+  it('treats missing amountLeft as zero balance', () => {
+    const parsed = parseManagedUsagePayload({
+      usage: { used: 1, limit: 10 },
+      boosterWallet: { balance: { type: 'BOOSTER', amount: '20000000000' } },
+    });
+    expect(parsed.extraUsage).toMatchObject({ totalCents: 20000, balanceCents: 0 });
+  });
+
+  it('defaults monthly limit fields when absent', () => {
+    const parsed = parseManagedUsagePayload({
+      usage: { used: 1, limit: 10 },
+      boosterWallet: {
+        balance: { type: 'BOOSTER', amount: '20000000000', amountLeft: '20000000000' },
+      },
+    });
+    expect(parsed.extraUsage).toEqual({
+      balanceCents: 20000,
+      totalCents: 20000,
+      monthlyChargeLimitEnabled: false,
+      monthlyChargeLimitCents: 0,
+      monthlyUsedCents: 0,
+      currency: 'USD',
+    });
+  });
+
+  it('returns null extra usage when boosterWallet is missing or invalid', () => {
+    expect(parseManagedUsagePayload({ usage: { used: 1, limit: 10 } }).extraUsage).toBeNull();
+    expect(
+      parseManagedUsagePayload({
+        usage: { used: 1, limit: 10 },
+        boosterWallet: { balance: { type: 'OTHER', amount: '100', amountLeft: '50' } },
+      }).extraUsage,
+    ).toBeNull();
+    expect(
+      parseManagedUsagePayload({
+        usage: { used: 1, limit: 10 },
+        boosterWallet: { balance: { type: 'BOOSTER', amount: '0', amountLeft: '0' } },
+      }).extraUsage,
+    ).toBeNull();
+  });
 });
 
 describe('fetchManagedUsage', () => {
@@ -92,6 +159,7 @@ describe('fetchManagedUsage', () => {
       parsed: {
         summary: { label: 'Weekly limit', used: 1, limit: 10 },
         limits: [],
+        extraUsage: null,
       },
     });
 
diff --git a/packages/oauth/test/toolkit.test.ts b/packages/oauth/test/toolkit.test.ts
index 73e091644..6ceaae417 100644
--- a/packages/oauth/test/toolkit.test.ts
+++ b/packages/oauth/test/toolkit.test.ts
@@ -567,6 +567,79 @@ describe('KimiOAuthToolkit', () => {
     expect((await storage.load(storageName))?.accessToken).toBe('fresh-access');
   });
 
+  it('propagates extraUsage from the managed usage response', async () => {
+    const storage = new MemoryTokenStorage();
+    storage.tokens.set('kimi-code', token('access-1'));
+    const fetchImpl = vi.fn(async (_input: unknown, _init?: RequestInit) =>
+      new Response(
+        JSON.stringify({
+          usage: { used: 10, limit: 100, name: 'Weekly limit' },
+          limits: [],
+          boosterWallet: {
+            balance: {
+              type: 'BOOSTER',
+              amount: '20000000000',
+              amountLeft: '10000000000',
+            },
+            monthlyChargeLimitEnabled: true,
+            monthlyChargeLimit: { currency: 'USD', priceInCents: '20000' },
+            monthlyUsed: { currency: 'USD', priceInCents: '5000' },
+          },
+        }),
+        { status: 200, headers: { 'Content-Type': 'application/json' } },
+      ),
+    ) as unknown as typeof fetch;
+    vi.stubGlobal('fetch', fetchImpl);
+    const toolkit = new KimiOAuthToolkit({
+      homeDir: join('/tmp', 'kimi-oauth-toolkit-test'),
+      identity: TEST_IDENTITY,
+      storage,
+      now: () => 100,
+    });
+
+    await expect(toolkit.getManagedUsage()).resolves.toMatchObject({
+      kind: 'ok',
+      summary: { label: 'Weekly limit', used: 10, limit: 100 },
+      limits: [],
+      extraUsage: {
+        balanceCents: 10000,
+        totalCents: 20000,
+        monthlyChargeLimitEnabled: true,
+        monthlyChargeLimitCents: 20000,
+        monthlyUsedCents: 5000,
+        currency: 'USD',
+      },
+    });
+  });
+
+  it('returns null extraUsage when the payload has no boosterWallet', async () => {
+    const storage = new MemoryTokenStorage();
+    storage.tokens.set('kimi-code', token('access-1'));
+    const fetchImpl = vi.fn(async (_input: unknown, _init?: RequestInit) =>
+      new Response(
+        JSON.stringify({
+          usage: { used: 10, limit: 100, name: 'Weekly limit' },
+          limits: [],
+        }),
+        { status: 200, headers: { 'Content-Type': 'application/json' } },
+      ),
+    ) as unknown as typeof fetch;
+    vi.stubGlobal('fetch', fetchImpl);
+    const toolkit = new KimiOAuthToolkit({
+      homeDir: join('/tmp', 'kimi-oauth-toolkit-test'),
+      identity: TEST_IDENTITY,
+      storage,
+      now: () => 100,
+    });
+
+    await expect(toolkit.getManagedUsage()).resolves.toMatchObject({
+      kind: 'ok',
+      summary: { label: 'Weekly limit', used: 10, limit: 100 },
+      limits: [],
+      extraUsage: null,
+    });
+  });
+
   it('removes managed config on logout when an adapter supports cleanup', async () => {
     const storage = new MemoryTokenStorage();
     storage.tokens.set('kimi-code', token('access-1'));

From 1bf2c9afee4643fbf6755f0b92fd60aa14240501 Mon Sep 17 00:00:00 2001
From: Kai 
Date: Thu, 9 Jul 2026 18:05:14 +0800
Subject: [PATCH 23/24] feat: keep image-heavy sessions within provider
 request-size limits (#1508)

* feat(kosong): classify HTTP 413 request-body-too-large as a dedicated error type

* feat(agent-core): lower default image downscale cap to 2000px and make it configurable

* feat(agent-core): strip media to text markers and retry when the compaction request is too large

* feat(agent-core): cap model-initiated image reads with a configurable byte budget

* feat(agent-core): resend with degraded media when the provider rejects the request body as too large

* test(agent-core): add explicit timeouts to encode-heavy image budget tests

* feat: add WebP decoding support with wasm integration

- Introduced a new WebP decoding module using @jsquash/webp's wasm decoder.
- Implemented functions to decode WebP images and check for animated WebP formats.
- Updated image compression tests to include scenarios for WebP handling, including encoding and decoding.
- Enhanced error handling for API request size limits to accommodate various error messages.
- Updated pnpm lockfile to include new dependencies for WebP encoding and decoding.

* chore(changeset): consolidate this PR's entries into one

* fix(nix): update pnpmDeps hash for merged lockfile

* feat(agent-core): refuse HEIC/HEIF reads with platform-matched conversion guidance
---
 .changeset/image-request-size-limits.md       |   6 +
 .../editor-keyboard-image-paste.test.ts       |   6 +-
 docs/en/configuration/config-files.md         |  14 +-
 docs/en/configuration/env-vars.md             |   2 +
 docs/zh/configuration/config-files.md         |  14 +-
 docs/zh/configuration/env-vars.md             |   2 +
 flake.nix                                     |   2 +-
 packages/agent-core/package.json              |   1 +
 .../scripts/generate-webp-dec-wasm.mjs        |  36 ++
 .../agent-core/src/agent/compaction/full.ts   |  59 ++-
 .../agent-core/src/agent/context/index.ts     |  13 +
 .../agent-core/src/agent/context/projector.ts |  55 +++
 .../agent-core/src/agent/records/types.ts     |   5 +-
 packages/agent-core/src/agent/turn/index.ts   |   1 +
 packages/agent-core/src/config/schema.ts      |  22 ++
 packages/agent-core/src/config/toml.ts        |  12 +
 packages/agent-core/src/loop/llm.ts           |   6 +-
 packages/agent-core/src/loop/run-turn.ts      |  22 +-
 packages/agent-core/src/loop/turn-step.ts     | 130 +++++--
 packages/agent-core/src/rpc/core-impl.ts      |   5 +
 .../src/tools/builtin/file/read-media.ts      |  61 +++-
 .../src/tools/support/image-compress.ts       | 248 ++++++++++---
 .../src/tools/support/webp-dec-wasm.ts        |   7 +
 .../src/tools/support/webp-decode.ts          |  86 +++++
 .../compaction/compaction-scenarios.test.ts   |  27 ++
 .../test/agent/compaction/full.test.ts        | 137 +++++++
 .../test/agent/context/projector.test.ts      |  66 +++-
 packages/agent-core/test/agent/turn.test.ts   |  72 ++++
 .../agent-core/test/config/configs.test.ts    |  22 ++
 .../loop/tool-exchange-fallback.e2e.test.ts   | 176 ++++++++-
 .../test/tools/image-compress.test.ts         | 343 ++++++++++++++++--
 .../agent-core/test/tools/read-media.test.ts  | 189 +++++++++-
 packages/kosong/src/errors.ts                 |  47 +++
 packages/kosong/src/index.ts                  |   2 +
 packages/kosong/test/errors.test.ts           |  64 ++++
 pnpm-lock.yaml                                |  94 +----
 36 files changed, 1839 insertions(+), 215 deletions(-)
 create mode 100644 .changeset/image-request-size-limits.md
 create mode 100644 packages/agent-core/scripts/generate-webp-dec-wasm.mjs
 create mode 100644 packages/agent-core/src/tools/support/webp-dec-wasm.ts
 create mode 100644 packages/agent-core/src/tools/support/webp-decode.ts

diff --git a/.changeset/image-request-size-limits.md b/.changeset/image-request-size-limits.md
new file mode 100644
index 000000000..076b0ef78
--- /dev/null
+++ b/.changeset/image-request-size-limits.md
@@ -0,0 +1,6 @@
+---
+"@moonshot-ai/kimi-code": patch
+---
+
+Keep image-heavy sessions within provider request-size limits: model-read images now honor a 256 KB per-image budget and a 2000px downscale cap (configurable via `[image]` in config.toml or `KIMI_IMAGE_*` env vars), oversized WebP is compressed as well, HEIC/HEIF reads are refused with a platform-matched conversion command instead of poisoning the session, and a request-too-large rejection (HTTP 413) now recovers automatically — the request and /compact both retry with older media replaced by text markers instead of failing the session.
+
diff --git a/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts b/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts
index 0fe14bc09..73e05b7ef 100644
--- a/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts
+++ b/apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts
@@ -141,14 +141,14 @@ describe('clipboard image paste compression', () => {
     if (att?.kind !== 'image') throw new Error('expected image attachment');
 
     // Stored metadata reflects the compressed size.
-    expect(Math.max(att.width, att.height)).toBeLessThanOrEqual(3000);
-    expect(att.placeholder).toContain('3000×1500');
+    expect(Math.max(att.width, att.height)).toBeLessThanOrEqual(2000);
+    expect(att.placeholder).toContain('2000×1000');
 
     // The stored bytes decode to the compressed dimensions — the thumbnail and
     // the submitted image both read from these bytes, so they cannot diverge.
     const dims = parseImageMeta(att.bytes);
     expect(dims).not.toBeNull();
-    expect(Math.max(dims!.width, dims!.height)).toBeLessThanOrEqual(3000);
+    expect(Math.max(dims!.width, dims!.height)).toBeLessThanOrEqual(2000);
   });
 
   it('records and persists the pre-compression original for an oversized paste', async () => {
diff --git a/docs/en/configuration/config-files.md b/docs/en/configuration/config-files.md
index f7e893555..303886e36 100644
--- a/docs/en/configuration/config-files.md
+++ b/docs/en/configuration/config-files.md
@@ -87,11 +87,12 @@ Fields in the config file fall into two categories: **top-level scalars** that d
 | `thinking` | `table` | — | Default parameters for Thinking mode → [`thinking`](#thinking) |
 | `loop_control` | `table` | — | Agent loop control parameters → [`loop_control`](#loop_control) |
 | `background` | `table` | — | Background task runtime parameters → [`background`](#background) |
+| `image` | `table` | — | Image compression parameters → [`image`](#image) |
 | `services` | `table` | — | Built-in external service configuration → [`services`](#services) |
 | `permission` | `table` | — | Initial permission rules → [`permission`](#permission) |
 | `hooks` | `array

               
TokenValueUsage

               
--font-ui-apple-system, BlinkMacSystemFont, "Segoe UI", "PingFang SC"…UI & body (system fonts first)
--font-ui"Inter Variable", "Inter", "Helvetica Neue", Arial…UI & body (Inter first)

                 
--font-monoJetBrains Mono…code, tool names, line numbers, diffs

                 
--base-ui-font-size14px user preferenceroot setting that drives UI, reading body, and sidebar font sizes

                 
--content-font-sizecalc(base + 1px)chat Markdown, message bubbles, composer
` | — | Lifecycle hooks; see [Hooks](../customization/hooks.md) |
 
-The following sections cover each of the nested tables in turn: `providers`, `models`, `thinking`, `loop_control`, `background`, `services`, and `permission`.
+The following sections cover each of the nested tables in turn: `providers`, `models`, `thinking`, `loop_control`, `background`, `image`, `services`, and `permission`.
 
 ## `providers`
 
@@ -202,6 +203,17 @@ You can also switch models temporarily without touching the config file — by s
 
 In print mode (`kimi -p ""`), Kimi Code runs a single non-interactive turn and exits as soon as the main agent finishes. If you launch background tasks (for example, concurrent subagents via `Agent(run_in_background=true)`) and need them to run to completion, set `keep_alive_on_exit = true`: the process then waits for every background task to reach a terminal state before exiting, bounded by `print_wait_ceiling_s`. Without it, the single turn ending tears background tasks down with the process.
 
+## `image`
+
+`image` controls how images are compressed before being sent to the model, across every ingestion point (pasted images, `ReadMediaFile` reads, images in MCP tool results, and so on).
+
+| Field | Type | Default | Description |
+| --- | --- | --- | --- |
+| `max_edge_px` | `integer` | `2000` | Longest-edge ceiling in pixels. Larger images are scaled down proportionally to fit; raising it preserves more detail at the cost of larger request bodies |
+| `read_byte_budget` | `integer` | `262144` (256 KB) | Per-image byte budget for images the model reads for itself (`ReadMediaFile` default reads). It bounds the accumulated request-body size when the model keeps screenshotting and reading images; fine detail stays reachable through the `region` parameter, which reads a crop back at full fidelity (`region` and `full_resolution` are not subject to this budget) |
+
+`max_edge_px` can be overridden by the `KIMI_IMAGE_MAX_EDGE_PX` environment variable and `read_byte_budget` by `KIMI_IMAGE_READ_BYTE_BUDGET`; both take higher priority than `config.toml`.
+
 
     
 
     
@@ -792,8 +793,29 @@ function openPr(url: string): void {
       @edit-message="handleEditMessage"
     />
 
+    
+    
+      
+    
+
     
     
 
+    
+    
+
     
      .side { grid-column: 1; }
+.side-handle { grid-column: 2; }
+.app:not(.mobile) > .con { grid-column: 3; }
+.preview-handle { grid-column: 4; }
+
+/* Sidebar toggle — floating button pinned to the top-left corner. On macOS
+   desktop it is resident (rendered in both states beside the traffic lights);
+   on Windows/web it only appears while the sidebar is collapsed (the collapse
+   button lives inside the sidebar header). While collapsed the conversation
+   header pads left so its content clears the button (global block below). */
+.sidebar-toggle-btn {
+  position: absolute;
+  /* Vertically centered in the 48px conversation header. */
+  top: 11px;
+  left: 16px;
+  z-index: var(--z-sticky);
+  /* Fade in on appearance (Windows/web: only rendered while collapsed, so
+     this plays as the sidebar finishes sliding away). macOS disables it. */
+  animation: sidebar-toggle-btn-in 0.18s var(--ease-out) 0.12s backwards;
+  /* Floats over the macOS-desktop window-drag header; keep it clickable. */
+  -webkit-app-region: no-drag;
 }
-/* The collapsed rail occupies track 1; keep the main pane pinned to the
-   conversation track even though the sidebar/handle are display:none. */
-.app.sidebar-collapsed > .con {
-  grid-column: 3;
+/* macOS desktop (hidden title bar): resident beside the floating traffic
+   lights (green light's right edge ≈ 68px; 72 keeps a gap that matches the
+   lights' own 8px rhythm); no entrance animation since it never appears. */
+.app.macos-desktop .sidebar-toggle-btn {
+  left: 72px;
+  animation: none;
+}
+@keyframes sidebar-toggle-btn-in {
+  from { opacity: 0; }
+}
+
+/* Internal-build tag pinned to the app's bottom-right corner (desktop app
+   only — the component renders nothing elsewhere). Informational: never
+   intercepts pointer input. */
+.internal-build-fab {
+  position: absolute;
+  right: var(--space-3);
+  bottom: var(--space-3);
+  z-index: var(--z-sticky);
+  pointer-events: none;
 }
 
 /* Mobile single-column shell: slim top bar (auto) over the full-width
@@ -1208,4 +1267,19 @@ function openPr(url: string): void {
      one continuous line across the layout. */
   --panel-head-h: 48px;
 }
+
+/* Sidebar collapsed (desktop): the conversation header pads left so its
+   content clears the floating sidebar toggle (.sidebar-toggle-btn) — and the
+   macOS traffic lights on desktop builds. Animated in step with the sidebar
+   width transition. Cross-component rule (ChatHeader renders the header), so
+   it lives in this global block. */
+.app:not(.mobile) .chat-header {
+  transition: padding-left 0.28s cubic-bezier(0.4, 0, 0.2, 1);
+}
+.app.sidebar-collapsed .chat-header {
+  padding-left: 52px;
+}
+.app.sidebar-collapsed.macos-desktop .chat-header {
+  padding-left: 108px;
+}
 
diff --git a/apps/kimi-web/src/components/InternalBuildBanner.vue b/apps/kimi-web/src/components/InternalBuildBanner.vue
index e2638aaad..a45748f7b 100644
--- a/apps/kimi-web/src/components/InternalBuildBanner.vue
+++ b/apps/kimi-web/src/components/InternalBuildBanner.vue
@@ -5,8 +5,8 @@ import { isDesktop } from '../lib/desktopFlag';
 
 const { t } = useI18n();
 
-// True only inside the Kimi Desktop app (see desktopFlag.ts). Renders an inline
-// tag meant to sit next to the "Kimi Code" brand in the sidebar header.
+// True only inside the Kimi Desktop app (see desktopFlag.ts). Renders a small
+// tag pinned to the app's bottom-right corner (positioned by App.vue).
 const show = isDesktop;
 
 
diff --git a/apps/kimi-web/src/components/SessionRow.vue b/apps/kimi-web/src/components/SessionRow.vue
index 2291a2681..30158be01 100644
--- a/apps/kimi-web/src/components/SessionRow.vue
+++ b/apps/kimi-web/src/components/SessionRow.vue
@@ -254,7 +254,7 @@ defineExpose({ closeMenu });
           :label="t('sidebar.options')"
           @click.stop="toggleMenu($event)"
         >
-          
+          
         
       
     
@@ -287,22 +287,24 @@ defineExpose({ closeMenu });
 .se {
   /* --sb-* vars come from .side in Sidebar.vue: the title starts at
      --sb-pad-x + --sb-gutter + --sb-gap, exactly under the workspace name.
-     The row is an inset pill: a 6px horizontal margin + 10px padding lands the
-     leading icon at --sb-pad-x (16px), aligned with the workspace header. */
+     The row is an inset pill: the .sessions container's --sb-inset padding +
+     the row's own padding land the leading slot at --sb-pad-x, aligned with
+     the workspace header. */
   display: block;
   margin: 0;
-  padding: var(--space-1) var(--space-2);
-  border-radius: var(--radius-md);
+  padding: 8px var(--space-2);
+  border-radius: var(--radius-sm);
   font-family: var(--font-ui);
   color: var(--color-text);
   cursor: pointer;
   position: relative;
 }
-.se:hover { background: var(--color-surface-sunken); color: var(--color-text); }
+.se:hover { background: var(--sb-hover, var(--color-surface-sunken)); color: var(--color-text); }
+/* Selected: neutral fill (NOT accent-tinted — selection reads as "where I
+   am", the accent stays reserved for actions and status). */
 .se.on {
-  background: var(--color-accent-soft);
-  color: var(--color-accent-hover);
-  box-shadow: inset 0 0 0 1px var(--color-accent-bd);
+  background: var(--color-selected);
+  color: var(--color-text);
 }
 
 .row {
@@ -310,9 +312,9 @@ defineExpose({ closeMenu });
   align-items: center;
   gap: var(--sb-gap, 6px);
   min-width: 0;
-  /* Floor the row at the hover-kebab height (IconButton sm = 26px) so swapping
-     the timestamp for the kebab on hover doesn't grow the row. */
-  min-height: 26px;
+  /* Row height is font-driven: title line-height (13×1.25≈16px) + 2×5px
+     .se padding ≈ 26px. The hover kebab is absolutely positioned (see .act)
+     so it never contributes to row height and can't cause hover jitter. */
 }
 
 .left {
@@ -341,7 +343,7 @@ defineExpose({ closeMenu });
 
 .t {
   color: inherit;
-  font-size: var(--text-base);
+  font-size: var(--ui-font-size-sm);
   font-weight: 450;
   line-height: var(--leading-tight);
   flex: 1;
@@ -356,30 +358,37 @@ defineExpose({ closeMenu });
   font-size: var(--text-xs);
   font-family: var(--font-ui);
   font-weight: 475;
+  line-height: var(--leading-tight);
   font-variant-numeric: tabular-nums;
   text-align: right;
 }
 
-/* Trailing action slot: time and kebab share one grid cell (grid-area:1/1).
-   Both stay in the layout and swap via `visibility` (never display:none), so
-   the slot width = max(time width, IconButton sm 26px) is identical in hover
-   and rest — the badges and title don't reflow, eliminating hover jitter.
-   `.act .kebab` out-specificities IconButton's own display so the hidden
-   default wins. */
+/* Trailing action slot: the relative time (in flow) sets the slot size; the
+   kebab is absolutely positioned over it and swapped via `visibility`, so it
+   contributes neither height (the row stays font-driven) nor width changes
+   (min-width reserves the kebab's footprint, the title doesn't reflow). */
 .act {
-  display: inline-grid;
+  position: relative;
   flex: none;
+  display: inline-flex;
   align-items: center;
-  justify-items: end;
+  justify-content: flex-end;
+  /* Reserve the kebab's width so the trailing slot (and thus the title) never
+     shifts between the time and the kebab, even for short times like "2m". */
+  min-width: 26px;
+}
+.act .kebab {
+  position: absolute;
+  right: 0;
+  top: 50%;
+  transform: translateY(-50%);
+  visibility: hidden;
 }
-.act .ts,
-.act .kebab { grid-area: 1 / 1; }
-.act .kebab { visibility: hidden; }
 .se:hover .act .kebab,
 .act:has(.kebab.open) .kebab { visibility: visible; }
 .se:hover .act .ts,
 .act:has(.kebab.open) .ts { visibility: hidden; }
-.kebab.open { color: var(--color-text); background: var(--color-surface-sunken); }
+.kebab.open { color: var(--color-text); background: var(--sb-hover, var(--color-surface-sunken)); }
 
 /* Fixed + anchored to the ⋯ button via inline style (see positionMenu); the menu
    is teleported to  so the collapsing list's `overflow: hidden` can't clip it. */
@@ -413,15 +422,10 @@ defineExpose({ closeMenu });
 
 .sessions .se {
   margin: 0;
-  border-radius: var(--radius-md);
-  /* Trim the row padding by the inset margin so the title still starts at the
-     same x as the workspace name (whose header has no inset). */
-  padding: var(--space-1) calc(var(--sb-pad-x, 12px) - var(--space-2));
-}
-.sessions .se:hover { background: var(--panel2); }
-.sessions .se.on {
-  background: var(--color-accent-soft);
-  box-shadow: inset 0 0 0 1px var(--color-accent-bd);
+  border-radius: var(--radius-sm);
+  /* Trim the row padding by the container inset so the title still starts at
+     the same x as the workspace name (whose header has no inset). */
+  padding: 8px calc(var(--sb-pad-x, 20px) - var(--sb-inset, 12px));
 }
 .sessions .se .rename-input { border-radius: var(--radius-sm); font-family: var(--sans); }
 .sessions .se .kebab { border-radius: var(--radius-sm); }
diff --git a/apps/kimi-web/src/components/Sidebar.vue b/apps/kimi-web/src/components/Sidebar.vue
index 987465dbb..8c9127e2e 100644
--- a/apps/kimi-web/src/components/Sidebar.vue
+++ b/apps/kimi-web/src/components/Sidebar.vue
@@ -9,21 +9,18 @@ import { serverEndpointLabel } from '../api/config';
 import { copyTextToClipboard } from '../lib/clipboard';
 import {
   loadCollapsedWorkspaces,
-  loadShowWorkspacePaths,
   saveCollapsedWorkspaces,
-  saveShowWorkspacePaths,
 } from '../lib/storage';
 import { moveInOrder, type DropPosition, type WorkspaceSortMode } from '../lib/workspaceOrder';
 import type { Session, WorkspaceGroup as WorkspaceGroupType, WorkspaceView } from '../types';
 import SearchSessionsDialog from './dialogs/SearchSessionsDialog.vue';
 import WorkspaceGroup from './WorkspaceGroup.vue';
-import InternalBuildBanner from './InternalBuildBanner.vue';
 import { isMacosDesktop } from '../lib/desktopFlag';
 import IconButton from './ui/IconButton.vue';
 import Icon from './ui/Icon.vue';
+import Kbd from './ui/Kbd.vue';
 import Menu from './ui/Menu.vue';
 import MenuItem from './ui/MenuItem.vue';
-import ShortcutKey from './ui/ShortcutKey.vue';
 import { useConfirmDialog } from '../composables/useConfirmDialog';
 
 const { t } = useI18n();
@@ -50,6 +47,12 @@ const props = withDefaults(
     unreadBySession?: Record;
     /** Width (px) of the session column, driven by the App resize handle. */
     colWidth?: number;
+    /** True when the sidebar is collapsed: the container animates to width 0
+     *  (content keeps `colWidth` and is clipped), then hides itself. */
+    collapsed?: boolean;
+    /** True while the resize handle is dragged — disables the width transition
+     *  so the sidebar follows the pointer 1:1. */
+    dragging?: boolean;
   }>(),
   {
     activeWorkspace: null,
@@ -58,6 +61,8 @@ const props = withDefaults(
     pendingBySession: () => ({}),
     unreadBySession: () => ({}),
     colWidth: 220,
+    collapsed: false,
+    dragging: false,
   },
 );
 
@@ -84,7 +89,7 @@ const emit = defineEmits<{
 // Session search dialog (Spotlight-style; filters title + last prompt)
 // ---------------------------------------------------------------------------
 const showSearch = ref(false);
-const sessionSearchShortcut = isAppleShortcutPlatform() ? '⌘K' : 'Ctrl K';
+const sessionSearchKeys = isAppleShortcutPlatform() ? ['⌘', 'K'] : ['Ctrl', 'K'];
 
 function openSearch(): void {
   // Sessions are loaded per-workspace (first page only); lazily drain the rest
@@ -111,7 +116,7 @@ function isAppleShortcutPlatform(): boolean {
   return userAgentData?.platform === 'macOS' || userAgentData?.platform === 'iOS';
 }
 
-// Scroll-linked header seam: the .btn-wrap bottom border/shadow only appears
+// Scroll-linked header seam: the .search-wrap bottom border/shadow only appears
 // once the session list has actually scrolled, so an unscrolled list shows no
 // abrupt boundary.
 const sessionsScrolled = ref(false);
@@ -186,18 +191,6 @@ function onLoadMore(id: string): void {
   emit('loadMoreSessions', id);
 }
 
-// ---------------------------------------------------------------------------
-// Workspace path display (toggle in the Workspaces section header)
-// ---------------------------------------------------------------------------
-// Off by default so the list stays compact; turning it on reveals every
-// workspace's root path as a stable subtitle (no hover-induced layout shift).
-const showWorkspacePaths = ref(loadShowWorkspacePaths());
-
-function toggleShowWorkspacePaths(): void {
-  showWorkspacePaths.value = !showWorkspacePaths.value;
-  saveShowWorkspacePaths(showWorkspacePaths.value);
-}
-
 // ---------------------------------------------------------------------------
 // Workspace drag-to-reorder
 // ---------------------------------------------------------------------------
@@ -487,11 +480,6 @@ function chooseSortMode(mode: WorkspaceSortMode): void {
   closeSectionMenu();
 }
 
-function toggleShowWorkspacePathsFromMenu(): void {
-  toggleShowWorkspacePaths();
-  closeSectionMenu();
-}
-
 onBeforeUnmount(() => {
   document.removeEventListener('mousedown', onGhMenuDocClick, true);
   document.removeEventListener('mousedown', onWsMenuDocClick);
@@ -560,54 +548,49 @@ onBeforeUnmount(() => {