feat(agent-core): compress oversized images before sending to the model (#1243)

* feat(agent-core): compress oversized images before sending to the model

Downsample images to a 2000px longest-edge and per-image byte budget at the
single prompt-ingestion chokepoint (the prompt/steer RPC) and on tool results
(ReadMediaFile, MCP), so every client transport — CLI, web, desktop, ACP, SDK —
is covered uniformly inside the core. PNG screenshots stay lossless and only
degrade to JPEG when the byte budget cannot otherwise be met. Best-effort: the
original image is sent unchanged if compression fails.

* fix(agent-core): serialize prompt/steer RPCs to avoid a turn-claim race

The prompt/steer RPC handlers await image compression before turn.launch()
synchronously claims the active turn, so two overlapping calls could both
compress first — letting the faster-to-compress one win the turn and strand the
other on agent_busy. Run these two RPCs through a per-agent serialization chain
so they claim in submit order; cancel and the other RPCs stay immediate.

* fix: update flake.nix pnpmDeps hash for the jimp dependency

Adding jimp to the workspace changed pnpm-lock.yaml, so the pnpmDeps
fixed-output hash was stale and the nix build failed. Update it to the value
the CI nix build reported.

* fix(agent-core): guard image compression against decompression bombs

A tiny-byte, huge-dimension image (e.g. a solid 30000x30000 PNG) would be fully
decoded into a multi-gigabyte bitmap by Jimp before any resize — an OOM vector
the byte budget never catches. Skip compression when the sniffed pixel count
exceeds MAX_DECODE_PIXELS (~100 MP), before the decode; oversized images pass
through uncompressed as they did before compression existed.

* fix(agent-core): cap decode byte size before compressing images

Compression runs before downstream size caps (e.g. the 10MB MCP per-part
limit), so a huge or invalid base64 image from an MCP tool was Buffer.from-
decoded — and handed to Jimp — just to be dropped afterward. Add a
MAX_DECODE_BYTES ceiling (64MB, overridable) checked before the base64 decode
and before Jimp, the byte-side complement to the pixel-count guard; oversized
payloads pass through uncompressed.

* refactor(agent-core): compress images at ingestion, not on the turn RPC

Move image compression off the prompt/steer RPC path and back to each ingestion
site (CLI paste, server upload resolution, ACP conversion; ReadMediaFile and MCP
already compressed at their producers). Compressing on the RPC control path put
an async step before the synchronous turn-claim, which spawned a series of
races: prompt/steer interleaving, and — with a cancel arriving mid-compression —
an ineffective abort that let a cancelled prompt launch anyway.

Treating compression as a pure input-stage transform (done while the content
part is built, before it ever enters the agent loop) removes those races
structurally: rpc.prompt/steer are plain synchronous handlers again, and the
serialization/cancel-window machinery is gone. Records stay compressed, resume
stays consistent, and coverage degrades gracefully (a new client that skips
compression just sends a larger image, as before this feature).

* fix: compress inline base64 prompts and honor ACP cancels mid-compression

Two contained ingestion-site follow-ups:

- server: resolvePromptMediaFiles now also compresses images submitted as an
  inline `{ kind: 'base64' }` source, not just uploaded files, so the REST
  inline-base64 path gets the same downsampling.
- acp-adapter: AcpSession tracks a pending-abort flag while prompt() awaits
  image compression (before any turn exists). A session/cancel in that window
  flips it, so the prompt returns `cancelled` instead of launching a turn the
  client already stopped.

* fix(acp-adapter): cover all concurrent pre-turn prompts on cancel

The pending-abort marker was a single session field, so with two
`session/prompt` requests compressing large inline images at once the later
one overwrote it and a `session/cancel` could mark only one — the other
launched after the client had cancelled. Track a token per in-flight prompt in
a set and flip them all on cancel so every pre-turn prompt is covered.

* chore(node-sdk): declare jimp as a devDependency

The SDK re-exports the image compressor, whose lazy `import('jimp')` (inside
the bundled agent-core code) is inlined into the published dist. jimp was
resolved only transitively via agent-core, so declare it as an explicit build
input here — matching the CLI — to make the bundling reliable rather than
phantom. It stays a devDependency: jimp is bundled, not a runtime dependency.

apps/kimi-code/package.json

@@ -94,6 +94,7 @@
     "chalk": "^5.4.1",
     "cli-highlight": "^2.1.11",
     "commander": "^13.1.0",
+    "jimp": "^1.6.1",
     "pathe": "^2.0.3",
     "postject": "1.0.0-alpha.6",
     "semver": "^7.7.4",

apps/kimi-code/src/tui/controllers/editor-keyboard.ts

@@ -1,4 +1,5 @@
 import type { Session } from '@moonshot-ai/kimi-code-sdk';
+import { compressImageForModel } from '@moonshot-ai/kimi-code-sdk';
 
 import { ClipboardMediaError, readClipboardMedia } from '#/utils/clipboard/clipboard-image';
 import { parseImageMeta } from '#/utils/image/image-mime';
@@ -360,7 +361,19 @@ export class EditorKeyboardController {
 
     const meta = parseImageMeta(media.bytes);
     if (meta === null) return false;
-    const attachment = this.imageStore.addImage(media.bytes, meta.mime, meta.width, meta.height);
+    // Compress at ingestion — a pure data step while building the attachment, so
+    // the stored bytes, the inline thumbnail, the `[image #N (W×H)]` placeholder,
+    // and the submitted image all agree, and the agent core only ever sees an
+    // already-compressed image. Best effort: originals pass through on failure.
+    const compressed = await compressImageForModel(media.bytes, meta.mime);
+    const attachment = compressed.changed
+      ? this.imageStore.addImage(
+          compressed.data,
+          compressed.mimeType,
+          compressed.width,
+          compressed.height,
+        )
+      : this.imageStore.addImage(media.bytes, meta.mime, meta.width, meta.height);
     this.host.state.editor.insertTextAtCursor?.(`${attachment.placeholder} `);
     this.host.state.ui.requestRender();
     this.host.track('shortcut_paste', { kind: 'image' });

apps/kimi-code/test/tui/controllers/editor-keyboard-image-paste.test.ts

@@ -0,0 +1,114 @@
+/**
+ * Clipboard image paste → attachment store, with ingestion-time compression.
+ *
+ * Tests pin:
+ *   - an oversized pasted image is downsampled while building the attachment,
+ *     so the stored bytes, the `[image #N (W×H)]` placeholder, and the eventual
+ *     submitted image all agree on the compressed size
+ *   - a within-budget paste is stored byte-for-byte (fast path)
+ */
+
+import { Jimp } from 'jimp';
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+
+import {
+  EditorKeyboardController,
+  type EditorKeyboardHost,
+} from '#/tui/controllers/editor-keyboard';
+import { ImageAttachmentStore } from '#/tui/utils/image-attachment-store';
+import { parseImageMeta } from '#/utils/image/image-mime';
+
+// vitest hoists vi.mock/vi.hoisted above the imports above, so the mock still
+// applies to the editor-keyboard module that pulls in readClipboardMedia.
+const { readClipboardMedia } = vi.hoisted(() => ({ readClipboardMedia: vi.fn() }));
+
+vi.mock('#/utils/clipboard/clipboard-image', async (importActual) => {
+  const actual = await importActual<typeof import('#/utils/clipboard/clipboard-image')>();
+  return { ...actual, readClipboardMedia };
+});
+
+interface PasteHarness {
+  readonly store: ImageAttachmentStore;
+  pasteImage(): Promise<void>;
+}
+
+function createPasteHarness(): PasteHarness {
+  const editor: Record<string, ((...args: never[]) => unknown) | undefined> = {};
+  const store = new ImageAttachmentStore();
+  const host = {
+    state: {
+      editor,
+      activeDialog: null,
+      appState: { streamingPhase: 'idle', isCompacting: false },
+      footer: { setTransientHint: vi.fn() },
+      ui: { requestRender: vi.fn() },
+    },
+    session: undefined,
+    btwPanelController: { closeOrCancel: vi.fn(() => false) },
+    track: vi.fn(),
+    showError: vi.fn(),
+    openUndoSelector: vi.fn(),
+    cancelRunningShellCommand: vi.fn(),
+  } as unknown as EditorKeyboardHost;
+
+  const controller = new EditorKeyboardController(host, store);
+  controller.install();
+
+  return {
+    store,
+    async pasteImage() {
+      const handler = editor['onPasteImage'];
+      if (handler === undefined) throw new Error('onPasteImage handler not installed');
+      await (handler as () => Promise<boolean>)();
+    },
+  };
+}
+
+async function solidPng(width: number, height: number): Promise<Uint8Array> {
+  return new Uint8Array(
+    await new Jimp({ width, height, color: 0x3366ccff }).getBuffer('image/png'),
+  );
+}
+
+describe('clipboard image paste compression', () => {
+  beforeEach(() => {
+    readClipboardMedia.mockReset();
+  });
+
+  it('downsamples an oversized pasted image before storing it', async () => {
+    const big = await solidPng(2600, 2600);
+    readClipboardMedia.mockResolvedValue({ kind: 'image', bytes: big, mimeType: 'image/png' });
+
+    const { store, pasteImage } = createPasteHarness();
+    await pasteImage();
+
+    expect(store.size()).toBe(1);
+    const att = store.get(1);
+    expect(att?.kind).toBe('image');
+    if (att?.kind !== 'image') throw new Error('expected image attachment');
+
+    // Stored metadata reflects the compressed size.
+    expect(Math.max(att.width, att.height)).toBeLessThanOrEqual(2000);
+    expect(att.placeholder).toContain('2000×2000');
+
+    // The stored bytes decode to the compressed dimensions — the thumbnail and
+    // the submitted image both read from these bytes, so they cannot diverge.
+    const dims = parseImageMeta(att.bytes);
+    expect(dims).not.toBeNull();
+    expect(Math.max(dims!.width, dims!.height)).toBeLessThanOrEqual(2000);
+  });
+
+  it('stores a within-budget paste byte-for-byte', async () => {
+    const small = await solidPng(80, 80);
+    readClipboardMedia.mockResolvedValue({ kind: 'image', bytes: small, mimeType: 'image/png' });
+
+    const { store, pasteImage } = createPasteHarness();
+    await pasteImage();
+
+    const att = store.get(1);
+    if (att?.kind !== 'image') throw new Error('expected image attachment');
+    expect(att.width).toBe(80);
+    expect(att.height).toBe(80);
+    expect(att.bytes).toBe(small); // identity: no re-encode on the fast path
+  });
+});