kimi-code/packages/acp-adapter/test/convert.test.ts
Kai db61c9e2dd
fix: refuse unsupported image formats instead of poisoning sessions (#1536)
* fix: refuse unsupported image formats instead of poisoning sessions

Images in formats providers reject (AVIF, HEIC, BMP, TIFF, ICO) used to
pass through to the API, and the resulting HTTP 400 repeated on every
later turn because the image_url stayed in the session history.

Add a single format policy (accepted set: PNG/JPEG/GIF/WebP) enforced at
every ingestion point: ReadMediaFile refuses with a per-OS conversion
command; MCP tool results, REST uploads, and ACP prompts replace the
image with a text notice; and turn.prompt/steer gates as the last-funnel
backstop so the SDK/RPC path cannot poison a session either. Accepted
MIME aliases (image/jpg, case/whitespace) are forwarded in canonical
form, and data URLs carrying MIME parameters can no longer slip past the
gate. Remote image URLs pass through (no bytes to inspect).

* fix: canonicalize accepted data URLs with MIME parameters

The format gate compared only the MIME token when deciding whether to
rebuild a data URL, so an accepted image carrying MIME parameters
(`data:image/jpeg;charset=utf-8;base64,...`) was forwarded with its
original header. The Anthropic provider splits the data URL and
exact-matches the full header against its whitelist, so the part still
poisoned the session. Rebuild to the byte-exact canonical URL whenever
the original differs, covering aliases, case/whitespace, and parameters
with one comparison.

Addresses review feedback on PR #1536.

* fix: parse data URLs case-insensitively in the image format gate

An uppercase `;BASE64,` marker is legal (RFC 2045 encoding names are
case-insensitive), but the parser required a lowercase match and
returned null, so the gate treated the URL as remote and forwarded it:
an unsupported image could still land in the session history, and the
Anthropic provider's lowercase-only split then threw on every turn.
Match the scheme and marker case-insensitively; the canonical rebuild
emits the lowercase form.

Addresses review feedback on PR #1536.

* fix: harden image format handling against mislabeled and legacy images

Two more ways an unsupported image could reach the provider are closed:

- Bytes, not labels, decide the format. A data-URL image whose declared
  MIME disagrees with its magic bytes (e.g. AVIF bytes an image search
  tool labels image/png) is now gated on the sniffed format at every entry
  point (MCP results, ACP, SDK/RPC prompt, REST inline and file uploads),
  so a mislabel cannot slip past the gate.
- A poisoned image already in the session history no longer kills the
  session: a server image-format 400 (or kosong's client-side image
  rejection) now retries once with every media part replaced by a text
  marker, mirroring the 413 media-degraded recovery. The recovery also
  fires during compaction, and the transient-retry fallback no longer
  burns the retry budget on image-format errors before the dedicated
  recovery can run.

* fix: reject remote image URLs ending in an unsupported extension

Remote image URLs (MCP resource_link, REST `kind: 'url'`) carry no bytes
to sniff, so a link ending in `.avif` (or `.heic`, `.bmp`, `.tiff`,
`.ico`) would pass through and be fetched server-side — and rejected.
Reject such URLs by their path extension instead (query/fragment
ignored, case-insensitive); extensionless or accepted-extension URLs
still pass through to the provider and the 400 recovery.

* fix: tighten image format handling for parameterized MIMEs and recovery scope

Address two review findings on PR #1536:

- A declared media type with parameters (e.g. image/jpeg; charset=utf-8)
  is no longer misread as unsupported: normalizeImageMime now strips
  parameters, matching the data-URL parser, so an accepted image with
  parameters is forwarded instead of dropped.
- The image-format recovery predicate is narrowed to specific
  format/data rejection phrases, so a 400 about image count, size, or
  image-input support no longer triggers a media-stripped resend that
  would let the model answer blind to the user's images.

* fix

* fix: scope image format recovery to images and flag remote SVG URLs

- The media_type/mime_type recovery match now requires the message to
  mention an image, so a video/audio media_type rejection surfaces
  instead of triggering a blind media-stripped resend.
- unsupportedImageMimeFromUrl flags .svg URLs as image/svg+xml without
  touching the shared suffix map (SVG stays text for the file tools),
  so remote SVG images get the intended notice instead of a provider
  rejection.

Addresses review feedback on PR #1536.

* fix: reject remote MCP images by their declared MIME type

An MCP resource_link with an extensionless or signed URL gives the
extension gate nothing to work with, and convertMCPContentBlock was
discarding the declared mimeType — an honestly-declared AVIF/HEIC link
from an image search tool still became an image_url and poisoned the
session. Reject on the declared MIME when the server provides one:
unsupported declarations become a text notice that keeps the URL so the
model can fetch and convert it; accepted declarations pass through as
before.

Addresses review feedback on PR #1536.

* fix: keep image format recovery image-specific and preserve dropped URLs in notices

- Drop the bare `media` alternative from the image-format recovery
  patterns so audio/video media rejections ("unsupported media type",
  "invalid media type") can never be misclassified as image errors and
  blindly media-stripped; every pattern now mentions "image" literally.
- Remote image URLs rejected by their extension now keep the URL in the
  replacement notice (gateImageFormatParts and the REST url path), so the
  model can still fetch and convert the image — matching the declared-MIME
  resource_link path.

Addresses review feedback on PR #1536.

* fix: drop malformed data URLs at ingestion instead of letting them poison the session

A `data:` URL that fails to parse (missing `;base64,` separator, empty
MIME, …) was treated like a remote URL and passed through the format
gate; the provider then rejects it on every turn, and the read-side
media-stripped recovery keeps paying that round-trip until compaction.
Detect unparseable `data:` URLs in gateImageFormatParts and replace them
with a (truncated) notice at ingestion, covering the MCP/ACP/SDK/turn
paths that share the gate.

Addresses review feedback on PR #1536.
2026-07-10 19:36:00 +08:00

457 lines
17 KiB
TypeScript

import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { ContentBlock } from '@agentclientprotocol/sdk';
import { Jimp } from 'jimp';
import { log, type ToolInputDisplay } from '@moonshot-ai/kimi-code-sdk';
import {
acpBlocksToPromptParts,
compressPromptImageParts,
displayBlockToAcpContent,
} from '../src/convert';
const textBlock = (text: string): ContentBlock => ({ type: 'text', text });
const imageBlock = (data: string, mimeType: string): ContentBlock => ({
type: 'image',
data,
mimeType,
});
const audioBlock = (data: string, mimeType: string): ContentBlock => ({
type: 'audio',
data,
mimeType,
});
const resourceLinkBlock = (uri: string, name: string): ContentBlock => ({
type: 'resource_link',
uri,
name,
});
const textResourceBlock = (uri: string, text: string, mimeType?: string): ContentBlock => ({
type: 'resource',
resource: mimeType !== undefined ? { uri, text, mimeType } : { uri, text },
});
const blobResourceBlock = (uri: string, blob: string, mimeType?: string): ContentBlock => ({
type: 'resource',
resource: mimeType !== undefined ? { uri, blob, mimeType } : { uri, blob },
});
describe('acpBlocksToPromptParts', () => {
let warnSpy: ReturnType<typeof vi.spyOn>;
beforeEach(() => {
warnSpy = vi.spyOn(log, 'warn').mockImplementation(() => undefined);
});
afterEach(() => {
warnSpy.mockRestore();
});
it('returns an empty array for an empty input', () => {
expect(acpBlocksToPromptParts([])).toEqual([]);
expect(warnSpy).not.toHaveBeenCalled();
});
it('passes text blocks through as { type: text, text }', () => {
const out = acpBlocksToPromptParts([textBlock('hello'), textBlock('world')]);
expect(out).toEqual([
{ type: 'text', text: 'hello' },
{ type: 'text', text: 'world' },
]);
expect(warnSpy).not.toHaveBeenCalled();
});
it('lifts image blocks into image_url parts with a data URL', () => {
const out = acpBlocksToPromptParts([
textBlock('caption'),
imageBlock('iVBORw0KGgoAAAA', 'image/png'),
]);
expect(out).toEqual([
{ type: 'text', text: 'caption' },
{
type: 'image_url',
imageUrl: { url: 'data:image/png;base64,iVBORw0KGgoAAAA' },
},
]);
expect(warnSpy).not.toHaveBeenCalled();
});
it('emits image and text parts in input order', () => {
const out = acpBlocksToPromptParts([
imageBlock('AAAA', 'image/jpeg'),
textBlock('what is this?'),
]);
expect(out).toEqual([
{
type: 'image_url',
imageUrl: { url: 'data:image/jpeg;base64,AAAA' },
},
{ type: 'text', text: 'what is this?' },
]);
expect(warnSpy).not.toHaveBeenCalled();
});
it('treats raw base64 as opaque — does not strip data: prefixes (documented limitation)', () => {
// Defensive behavior: a caller that pre-wraps the payload as a data URL
// will end up double-wrapped. The ACP spec says `data` is base64, so this
// only affects non-conforming callers.
const out = acpBlocksToPromptParts([
imageBlock('data:image/png;base64,XXXX', 'image/png'),
]);
expect(out).toEqual([
{
type: 'image_url',
imageUrl: { url: 'data:image/png;base64,data:image/png;base64,XXXX' },
},
]);
});
it('drops audio blocks but warns with the dedicated message', () => {
const out = acpBlocksToPromptParts([
textBlock('hi'),
audioBlock('AAAA', 'audio/mpeg'),
]);
expect(out).toEqual([{ type: 'text', text: 'hi' }]);
expect(warnSpy).toHaveBeenCalledTimes(1);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('dropping unsupported audio prompt block'),
expect.objectContaining({ mimeType: 'audio/mpeg' }),
);
});
it('projects file:// resource_link blocks to bare paths', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('file:///a.txt', 'a'),
textBlock('see linked file'),
resourceLinkBlock('file:///b.txt', 'b'),
]);
expect(out).toEqual([
{ type: 'text', text: '/a.txt' },
{ type: 'text', text: 'see linked file' },
{ type: 'text', text: '/b.txt' },
]);
expect(warnSpy).not.toHaveBeenCalled();
});
it('appends a line range to file:// paths when the fragment carries one', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('file:///src/foo.ts#L10', 'foo.ts'),
resourceLinkBlock('file:///src/foo.ts#L10-L20', 'foo.ts'),
resourceLinkBlock('file:///src/foo.ts#L10-20', 'foo.ts'),
resourceLinkBlock('file:///src/foo.ts?line=10', 'foo.ts'),
resourceLinkBlock('file:///src/foo.ts?lines=10-20', 'foo.ts'),
]);
expect(out.map((p) => (p.type === 'text' ? p.text : ''))).toEqual([
'/src/foo.ts:10',
'/src/foo.ts:10-20',
'/src/foo.ts:10-20',
'/src/foo.ts:10',
'/src/foo.ts:10-20',
]);
});
it('URL-decodes file:// paths (spaces, unicode)', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('file:///Users/a%20b/foo.ts', 'foo.ts'),
resourceLinkBlock('file:///Users/%E4%B8%AD%E6%96%87/foo.ts', 'foo.ts'),
]);
expect(out).toEqual([
{ type: 'text', text: '/Users/a b/foo.ts' },
{ type: 'text', text: '/Users/中文/foo.ts' },
]);
});
it('strips the leading slash on Windows file:// drive paths', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('file:///C:/Users/x/foo.ts', 'foo.ts'),
resourceLinkBlock('file:///D:/work/bar.ts#L42', 'bar.ts'),
]);
expect(out).toEqual([
{ type: 'text', text: 'C:/Users/x/foo.ts' },
{ type: 'text', text: 'D:/work/bar.ts:42' },
]);
});
it('preserves non-local file:// hosts as UNC paths', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('file://server/share/project/a.ts#L3', 'a.ts'),
resourceLinkBlock('file://server/share/project/b.ts?lines=10-20', 'b.ts'),
resourceLinkBlock('file://localhost/share/project/c.ts#L3', 'c.ts'),
]);
expect(out).toEqual([
{ type: 'text', text: '//server/share/project/a.ts:3' },
{ type: 'text', text: '//server/share/project/b.ts:10-20' },
{ type: 'text', text: '/share/project/c.ts:3' },
]);
});
it('lowercases UNC hosts so case-variant inputs collapse to one ref', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('file://SERVER/share/project/a.ts#L3', 'a.ts'),
resourceLinkBlock('file://Server/share/project/a.ts#L3', 'a.ts'),
resourceLinkBlock('file://LOCALHOST/share/project/c.ts#L3', 'c.ts'),
]);
expect(out).toEqual([
{ type: 'text', text: '//server/share/project/a.ts:3' },
{ type: 'text', text: '//server/share/project/a.ts:3' },
{ type: 'text', text: '/share/project/c.ts:3' },
]);
});
it('keeps the XML wrapper for non-file:// resource_link schemes', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('zed:///agent/terminal-selection?lines=10', 'Terminal (10 lines)'),
resourceLinkBlock('https://example.com/spec', 'spec'),
]);
expect(out).toEqual([
{
type: 'text',
text:
'<resource_link uri="zed:///agent/terminal-selection?lines=10" name="Terminal (10 lines)" />',
},
{
type: 'text',
text: '<resource_link uri="https://example.com/spec" name="spec" />',
},
]);
});
it('falls back to the XML wrapper for unparseable resource_link uris', () => {
const out = acpBlocksToPromptParts([resourceLinkBlock('not a url', 'weird')]);
expect(out).toEqual([
{ type: 'text', text: '<resource_link uri="not a url" name="weird" />' },
]);
});
it('inlines TextResourceContents as <resource uri>text</resource>', () => {
const out = acpBlocksToPromptParts([
textResourceBlock('file:///hello.md', '# Hello\nworld', 'text/markdown'),
]);
expect(out).toEqual([
{
type: 'text',
text: '<resource uri="file:///hello.md"># Hello\nworld</resource>',
},
]);
expect(warnSpy).not.toHaveBeenCalled();
});
it('drops BlobResourceContents with a dedicated warn', () => {
const out = acpBlocksToPromptParts([
blobResourceBlock('file:///pic.bin', 'AAAA', 'application/octet-stream'),
]);
expect(out).toEqual([]);
expect(warnSpy).toHaveBeenCalledTimes(1);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('dropping blob embedded resource'),
expect.objectContaining({
uri: 'file:///pic.bin',
mimeType: 'application/octet-stream',
}),
);
});
it('escapes XML-special characters in non-file:// resource_link attributes', () => {
const out = acpBlocksToPromptParts([
resourceLinkBlock('https://example.com/a&b', 'name with "quotes" & <angle>'),
]);
expect(out).toEqual([
{
type: 'text',
text:
'<resource_link uri="https://example.com/a&amp;b" name="name with &quot;quotes&quot; &amp; &lt;angle&gt;" />',
},
]);
});
it('emits mixed text + resource_link + embedded text resource in input order', () => {
const out = acpBlocksToPromptParts([
textBlock('header'),
resourceLinkBlock('file:///x', 'x'),
textResourceBlock('file:///y.txt', 'body'),
]);
expect(out).toEqual([
{ type: 'text', text: 'header' },
{ type: 'text', text: '/x' },
{ type: 'text', text: '<resource uri="file:///y.txt">body</resource>' },
]);
expect(warnSpy).not.toHaveBeenCalled();
});
});
describe('displayBlockToAcpContent — plan_review branch (Phase 13.2)', () => {
const planMd = '## Goal\n\nShip the plan_review surface so Zed sees the markdown body.';
it('returns null when block.plan is empty after trimming', () => {
const block: ToolInputDisplay = { kind: 'plan_review', plan: ' \n\t ' };
expect(displayBlockToAcpContent(block)).toBeNull();
});
it('renders the plan markdown alone when no path is set', () => {
const block: ToolInputDisplay = { kind: 'plan_review', plan: planMd };
expect(displayBlockToAcpContent(block)).toEqual({
type: 'content',
content: { type: 'text', text: planMd },
});
});
it('prefixes "Plan saved to: <path>" when block.path is set', () => {
const block: ToolInputDisplay = {
kind: 'plan_review',
plan: planMd,
path: '/tmp/plan.md',
};
expect(displayBlockToAcpContent(block)).toEqual({
type: 'content',
content: {
type: 'text',
text: `Plan saved to: /tmp/plan.md\n\n${planMd}`,
},
});
});
it('preserves the plan body verbatim — no markdown escaping or normalisation', () => {
const richMd = '**bold** & <tag> with "quotes"';
const block: ToolInputDisplay = { kind: 'plan_review', plan: richMd };
const out = displayBlockToAcpContent(block);
expect(out).toEqual({
type: 'content',
content: { type: 'text', text: richMd },
});
});
it('still returns null for an unmapped kind (Phase 5 invariant)', () => {
const cmd: ToolInputDisplay = { kind: 'command', command: 'ls' };
expect(displayBlockToAcpContent(cmd)).toBeNull();
});
});
describe('compressPromptImageParts', () => {
async function pngBase64(width: number, height: number): Promise<string> {
const buf = await new Jimp({ width, height, color: 0x3366ccff }).getBuffer('image/png');
return Buffer.from(buf).toString('base64');
}
it('downsamples an oversized inline image part and announces the compression', async () => {
const originalsDir = await mkdtemp(join(tmpdir(), 'acp-originals-'));
const originalBase64 = await pngBase64(3600, 1800);
const parts = acpBlocksToPromptParts([imageBlock(originalBase64, 'image/png')]);
const compressed = await compressPromptImageParts(parts, { originalsDir });
// A caption precedes the downsampled image so the model knows it is
// looking at a degraded copy and where the original bytes live.
expect(compressed).toHaveLength(2);
const caption = compressed[0];
if (caption?.type !== 'text') throw new Error('expected a caption text part');
expect(caption.text).toContain('Image compressed');
expect(caption.text).toContain('3600x1800');
const part = compressed[1];
if (part?.type !== 'image_url') throw new Error('expected an image_url part');
const match = /^data:(image\/[a-z]+);base64,(.+)$/.exec(part.imageUrl.url);
expect(match).not.toBeNull();
const decoded = await Jimp.fromBuffer(Buffer.from(match![2]!, 'base64'));
expect(Math.max(decoded.width, decoded.height)).toBeLessThanOrEqual(3000);
// The caption points at a persisted copy of the ORIGINAL bytes, placed in
// the provided (session-scoped) originals dir.
const pathMatch = /saved at "([^"]+)"/.exec(caption.text);
expect(pathMatch).not.toBeNull();
expect(pathMatch![1]!.startsWith(originalsDir)).toBe(true);
const persisted = await readFile(pathMatch![1]!);
expect(persisted.equals(Buffer.from(originalBase64, 'base64'))).toBe(true);
await rm(originalsDir, { recursive: true, force: true });
});
it('downsamples to the caller-provided max edge instead of the built-in cap', async () => {
const originalsDir = await mkdtemp(join(tmpdir(), 'acp-originals-'));
const parts = acpBlocksToPromptParts([imageBlock(await pngBase64(3600, 1800), 'image/png')]);
const compressed = await compressPromptImageParts(parts, {
originalsDir,
maxImageEdgePx: 800,
});
const part = compressed[1];
if (part?.type !== 'image_url') throw new Error('expected an image_url part');
const match = /^data:(image\/[a-z]+);base64,(.+)$/.exec(part.imageUrl.url);
expect(match).not.toBeNull();
const decoded = await Jimp.fromBuffer(Buffer.from(match![2]!, 'base64'));
expect(decoded.width).toBe(800);
expect(decoded.height).toBe(400);
await rm(originalsDir, { recursive: true, force: true });
});
it('uses the built-in 2000px cap when no max edge is provided', async () => {
const originalsDir = await mkdtemp(join(tmpdir(), 'acp-originals-'));
const parts = acpBlocksToPromptParts([imageBlock(await pngBase64(3600, 1800), 'image/png')]);
const compressed = await compressPromptImageParts(parts, { originalsDir });
const part = compressed[1];
if (part?.type !== 'image_url') throw new Error('expected an image_url part');
const match = /^data:(image\/[a-z]+);base64,(.+)$/.exec(part.imageUrl.url);
expect(match).not.toBeNull();
const decoded = await Jimp.fromBuffer(Buffer.from(match![2]!, 'base64'));
expect(decoded.width).toBe(2000);
expect(decoded.height).toBe(1000);
await rm(originalsDir, { recursive: true, force: true });
});
it('emits image_compress telemetry tagged acp_prompt', async () => {
const originalsDir = await mkdtemp(join(tmpdir(), 'acp-originals-'));
const events: { event: string; props: Record<string, unknown> }[] = [];
const parts = acpBlocksToPromptParts([
imageBlock(await pngBase64(3600, 1800), 'image/png'),
]);
await compressPromptImageParts(parts, {
originalsDir,
telemetry: { track: (event, props) => events.push({ event, props: { ...props } }) },
});
expect(events).toHaveLength(1);
expect(events[0]!.event).toBe('image_compress');
expect(events[0]!.props['source']).toBe('acp_prompt');
expect(events[0]!.props['outcome']).toBe('compressed');
await rm(originalsDir, { recursive: true, force: true });
});
it('passes a within-budget image and text through unchanged', async () => {
const parts = acpBlocksToPromptParts([
imageBlock(await pngBase64(32, 32), 'image/png'),
textBlock('hi'),
]);
const compressed = await compressPromptImageParts(parts);
expect(compressed).toEqual(parts);
});
it('replaces an image the provider cannot accept with a text notice', async () => {
// An AVIF image must never reach the session history — the provider
// rejects it and every later request would fail. A notice stands in.
const parts = acpBlocksToPromptParts([
textBlock('look at this'),
imageBlock(Buffer.from([1, 2, 3]).toString('base64'), 'image/avif'),
]);
const compressed = await compressPromptImageParts(parts);
expect(compressed).toHaveLength(2);
expect(compressed[0]).toEqual({ type: 'text', text: 'look at this' });
const notice = compressed[1];
if (notice?.type !== 'text') throw new Error('expected a text notice');
expect(notice.text).toContain('image/avif');
});
it('forwards accepted MIME aliases in canonical form', async () => {
// Strict provider whitelists reject the raw `image/jpg` alias — the part
// must land in the session with the canonical MIME.
const base64 = Buffer.from([1, 2, 3]).toString('base64');
const parts = acpBlocksToPromptParts([imageBlock(base64, 'image/jpg')]);
const compressed = await compressPromptImageParts(parts);
expect(compressed).toEqual([
{ type: 'image_url', imageUrl: { url: `data:image/jpeg;base64,${base64}` } },
]);
});
});