mirror of
https://github.com/MoonshotAI/kimi-code.git
synced 2026-08-19 21:56:00 +00:00
refactor(agent-core-v2): carry the context fold cursor in state and converge fold/projection internals (#2875)
* refactor(agent-core-v2): carry the context fold cursor in state and converge fold/projection internals
- ContextModel state is now { messages, fold }: the loop-event fold cursor
(openStepUuid / pending / deferred) lives in the state instead of a
module-level WeakMap keyed by array identity, so wholesale replacements
(undo / clear / compaction / swarm exit) reset it structurally via
EMPTY_FOLD instead of a manual resetFold at five call sites.
- The display transcript and the wire model now share one generic fold
kernel (FoldFrame / FoldEntryAdapter), eliminating the mirrored second
implementation. Events tagged with a non-open step uuid are dropped and
step.end settles only the step it names — defensive in abnormal streams,
identical on well-formed ones (v1 replay unaffected).
- IAgentContextProjectorService converges to project(messages, policy) with
a ProjectionPolicy data object; llmRequester builds the policy from retry
state instead of selecting among four methods.
- Blob rehydrate now also covers messages still deferred in the fold cursor.
- ContextState is deeply frozen at the op boundary to preserve the consumer
immutability the wire's shallow freeze gave the bare array state.
* test(agent-core-v2): move fold parity rationales into the test file header
* docs(agent-core-v2): move fold declaration comments into module headers
* refactor(agent-core-v2): merge FoldFrame into generic ContextState
* refactor(agent-core-v2): compile-enforce part/event handling decisions in context memory
- isVacuousContentPart and dehydrateRecord now switch exhaustively over
ContentPart / LoopRecordedEvent variants, so a new variant fails
compilation until it takes an explicit position
- the transcript/model parity comparator spreads whole messages and masks
only summary content, so new ContextMessage fields join the comparison
automatically
- correct two stale header comments: local message ids persist with
append_message records, and undo's prompt-owned-injection pairing
depends on them after a resume
* refactor(agent-core-v2): converge undo-cut decision in conversationTime
The model Op and the display transcript each walked the undo anchors with
their own loop, and the transcript partially removed the tail when an undo
was blocked (compaction summary / clear floor / too few anchors) while the
model side no-ops at the precheck. Move the walk into conversationTime as
computeUndoCut/computeUndoCutFrom applied destructively by the context.undo
Op and non-destructively by the transcript reducer, so a blocked undo reads
identically on both sides.
Also: make isUndoAnchor exhaustive over origin kinds with a never assertion,
mirrors the compaction result message count via compactionHandoff, and
extend UndoCut with anchorIndex distinguishing the counted anchor from the
injection-extended cut point.
* fix(agent-core-v2): drop removed prompts' injections on multi-turn transcript undo
The transcript's kept-loop retained every injection after the oldest
counted anchor, so with count > 1 a prompt-owned injection of a newer
removed prompt (e.g. an image-compression caption) survived the display
undo while the model Op removed it. Collect the removed anchors' ids on
the same pass and keep only injections not owned by them, so the header's
'prompt-owned ones leave with their prompt' holds for every count.
* refactor(agent-core-v2): accumulate request projection repairs as policy
The llmRequester retry chain kept a RequestProjection union and translated
it into a ProjectionPolicy per attempt; repairs were mutually exclusive,
so a strict resend rejected again for body size or image format either
aborted or silently dropped the strict repair. Retry state is now the
ProjectionPolicy itself: each rejection adds its repair on its own axis
(media: 413 -> degraded -> strip; wire: structure -> strict) without
discarding the other, requestInput's translation layer and the unreachable
snapshot ??= disappear, and the persisted llm.request projection name
derives from the policy (the op enum gains strict-media-degraded /
strict-media-stripped). Also narrows ProjectionPolicy to the variants
actually produced (wire 'strict'; media 'degraded' | { strip }), dropping
the dead 'default'/'keep' literals and their guard.
* refactor(agent-core-v2): derive the visible context window from an append-only log
context.apply_compaction now appends a summary marker carrying the record
fields as CompactionMeta instead of replacing the folded history; the
model-visible window is derived at read time (visibleWindow) with the same
[head, elision?, tail, summary] layout, one deterministic derivation for
live dispatch and replay alike. The record format is unchanged, so v1- and
v2-written sessions keep replaying identically both ways.
- undo maps the visible-window cut back to a log position (the verbatim
legacy-summary edge falls back to the pre-append-only destructive cut)
- replay rehydrate only loads blobs the window derivation can surface
- contextInjector drops position tracking; injection positions become a
read-time scan that splices can never desync
- fullCompaction's safety check moves to the stable-identity log
* fix(agent-core-v2): rehydrate exactly the messages the visible window surfaces
The first append-only rehydrate rule kept pre-marker real user input plus
markers, but a legacyTail derivation keeps window.slice(compactedCount)
visible — assistant/tool media in that range stayed blobref after replay
and could be resent unresolved. Decide survivors by identity membership in
the derived window instead, which covers every derivation branch at once
and skips the unselected pre-marker pool as a bonus.
* fix(agent-core-v2): pop the visible-tail swarm reminder behind a legacy marker
SwarmService.exit decides the pop on the derived window's tail, but the
swarm_mode.exit reducer tested the raw log tail — after a legacyTail
compaction the survivor reminder is visible at the window tail while the
marker is the log tail, so the pop silently skipped and the stale reminder
stayed in the model context. Mirror the visible-tail decision in the
reducer and remove the entry by its stable log identity.
* fix(agent-core-v2): settle open transcript frames when compaction lands mid-fold
An overflow-triggered compaction arrives with the failed attempt's vacuous
partial still open. The transcript appended the summary marker and reset
the fold but left the frame; the retried step's step.begin then settled it
(-1) alongside the new frame (+1), so foldedLength stayed one short of the
model-visible window and kap-server's live-tail merge could duplicate the
retry tail. Settle the frame at the marker through the shared kernel;
recoverFoldedLength recomputes the absolute count right after either way.
* fix(agent-core-v2): settle open frames at the compaction marker
Apply settleModelOpenStep inside context.apply_compaction so a marker only
ever lands on a settled frame: no partial survives a marker and nothing
mutates the log behind one, making the append-only invariant structural
(the identity-prefix check in historySafeToCompact relied on it) instead
of timing-dependent. Mirrors the transcript's settle-at-marker.
Also freeze the derived visible window before caching it (an in-place
consumer mutation now throws instead of silently polluting the shared
cache), and drop the production-unreachable legacy branch of
buildContextCompactionShape so the legacy tail layout lives only in
deriveCompactionWindow.
* refactor(agent-core-v2): tighten naming and comments in context memory internals
- Slim file headers to the package header-only comment convention
- Rename PR-introduced identifiers for clarity: getMessageLog,
ProjectionPolicy.structure, pendingToolCallIds/deferredEntries,
removedEntryCount, deriveVisibleWindowAfterCompaction,
compactedWindowMessageCount
- Extract nextProjectionPolicyForError, removeUndoOwnedEntries and
summarizeProjectionRepairs; name fold intermediates after their
business stage
- Regroup splice-replay tests by topic and unify projection-call
recording in llmRequester tests
* fix(agent-core-v2): preserve bounded context state
* docs(agent-core-v2): restore the domain identity line in the compactionHandoff header
This commit is contained in:
7Sageer
GitHub
parent
2265305e81
commit
02aa24e2f4
10 changed files with 428 additions and 385 deletions
|
|
@ -305,7 +305,7 @@ interface LlmRequestPayload {
|
|||
messageCount: number;
|
||||
turnStep?: string;
|
||||
attempt?: string;
|
||||
projection?: 'strict' | 'media-degraded' | 'media-stripped';
|
||||
projection?: 'strict' | 'media-degraded' | 'media-stripped' | 'strict-media-degraded' | 'strict-media-stripped';
|
||||
droppedCount?: number;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,14 +1,11 @@
|
|||
/**
|
||||
* `contextMemory` domain helper — derives the v1-compatible full-compaction
|
||||
* handoff shape for live rewrites, wire replay, and snapshot reducers.
|
||||
* `contextMemory` domain helper — builds the bounded context window produced
|
||||
* by compaction and exposes the shared user-message selection rules used by
|
||||
* live execution and replay.
|
||||
*
|
||||
* Token budgeting runs through an injectable {@link TokenEstimate}: the live
|
||||
* path (`AgentContextMemoryService.applyCompaction`) passes the estimator
|
||||
* from `IAgentTokenCountingService` (the raw heuristics — the
|
||||
* `[token_counting]` strategy never gates internal estimates); the pure
|
||||
* wire-replay / reducer paths keep the same heuristics — their estimate
|
||||
* fallback only fires when a record lacks `tokensAfter`, so the measured
|
||||
* chain is unaffected.
|
||||
* Estimates token sizes through `kosong`'s contract heuristics (injectable as
|
||||
* `TokenEstimate`) and wraps elision notes through `systemReminder`.
|
||||
* Scope-agnostic.
|
||||
*/
|
||||
|
||||
import { estimateTokens, estimateTokensForMessage, estimateTokensForMessages } from '#/kosong/contract/tokens';
|
||||
|
|
@ -24,7 +21,6 @@ export const COMPACTION_ELISION_VARIANT = 'compaction_elision';
|
|||
|
||||
type MessageLike = ContextMessage;
|
||||
|
||||
/** Injectable token-count estimates; see the file header for who passes what. */
|
||||
export interface TokenEstimate {
|
||||
readonly text: (text: string) => number;
|
||||
readonly message: (message: MessageLike) => number;
|
||||
|
|
@ -51,15 +47,7 @@ export interface ContextCompactionShapeInput {
|
|||
readonly compactedCount: number;
|
||||
readonly tokensBefore: number;
|
||||
readonly tokensAfter?: number;
|
||||
/** Measured output tokens of the compaction LLM exchange — the REAL size of
|
||||
* the generated summary. Preferred over the summary-text estimate in the
|
||||
* `tokensAfter` fallback when present. */
|
||||
readonly summaryOutputTokens?: number;
|
||||
/** Estimated fixed request overhead (system prompt + non-deferred tool
|
||||
* schemas) surviving the compaction; counted into the `tokensAfter`
|
||||
* fallback so the result stays on the same full-request basis as the
|
||||
* measured exchange anchors. Live path only — replay reads the persisted
|
||||
* `tokensAfter` verbatim. */
|
||||
readonly requestOverheadTokens?: number;
|
||||
readonly keptUserMessageCount?: number;
|
||||
readonly keptHeadUserMessageCount?: number;
|
||||
|
|
@ -139,6 +127,7 @@ export function buildContextCompactionShape(
|
|||
};
|
||||
}
|
||||
|
||||
|
||||
export function buildCompactionSummaryText(summary: string): string {
|
||||
const suffix = summary.trim();
|
||||
return `${COMPACTION_SUMMARY_PREFIX}\n${suffix.length > 0 ? suffix : '(no summary available)'}`;
|
||||
|
|
|
|||
|
|
@ -10,7 +10,19 @@
|
|||
import type { ContentPart } from '#/kosong/contract/message';
|
||||
|
||||
export function isVacuousContentPart(part: ContentPart): boolean {
|
||||
if (part.type === 'text') return part.text.trim().length === 0;
|
||||
if (part.type === 'think') return part.encrypted === undefined && part.think.trim().length === 0;
|
||||
return false;
|
||||
switch (part.type) {
|
||||
case 'text':
|
||||
return part.text.trim().length === 0;
|
||||
case 'think':
|
||||
return part.encrypted === undefined && part.think.trim().length === 0;
|
||||
case 'image_url':
|
||||
case 'audio_url':
|
||||
case 'video_url':
|
||||
return false;
|
||||
default: {
|
||||
const exhaustive: never = part;
|
||||
void exhaustive;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,6 +4,14 @@
|
|||
* Defines wire-safe history projections and an opaque snapshot of the media
|
||||
* identities that a provider rejected, allowing later steps to strip only
|
||||
* that content while preserving newly generated recovery media.
|
||||
*
|
||||
* Projection variability is expressed as data: a `ProjectionPolicy` —
|
||||
* `structure: 'strict'` adds the structural repairs strict providers need
|
||||
* (duplicate tool calls dropped, consecutive assistants merged, leading
|
||||
* non-user messages dropped); `media` selects the provider-rejection
|
||||
* fallback (`'degraded'` replaces all but the most recent media with text
|
||||
* markers after an HTTP 413; `{ strip }` replaces exactly the snapshotted
|
||||
* media identities after a rejected-format or still-too-large resend).
|
||||
*/
|
||||
|
||||
import { createDecorator } from '#/_base/di/instantiation';
|
||||
|
|
@ -17,17 +25,19 @@ export interface MediaStripSnapshot {
|
|||
readonly [mediaStripSnapshotBrand]: undefined;
|
||||
}
|
||||
|
||||
export interface ProjectionPolicy {
|
||||
readonly structure?: 'strict';
|
||||
readonly media?: 'degraded' | { readonly strip: MediaStripSnapshot };
|
||||
}
|
||||
|
||||
export interface IAgentContextProjectorService {
|
||||
readonly _serviceBrand: undefined;
|
||||
|
||||
project(messages: readonly ContextMessage[]): readonly Message[];
|
||||
projectStrict(messages: readonly ContextMessage[]): readonly Message[];
|
||||
projectMediaDegraded(messages: readonly ContextMessage[]): readonly Message[];
|
||||
captureMediaStripSnapshot(messages: readonly ContextMessage[]): MediaStripSnapshot;
|
||||
projectMediaStripped(
|
||||
project(
|
||||
messages: readonly ContextMessage[],
|
||||
snapshot?: MediaStripSnapshot,
|
||||
policy?: ProjectionPolicy,
|
||||
): readonly Message[];
|
||||
captureMediaStripSnapshot(messages: readonly ContextMessage[]): MediaStripSnapshot;
|
||||
}
|
||||
|
||||
export const IAgentContextProjectorService = createDecorator<IAgentContextProjectorService>(
|
||||
|
|
|
|||
|
|
@ -15,14 +15,13 @@
|
|||
* repair-dedup signature (`lastRepairSignature`) is registered into
|
||||
* `agentState` (`IAgentStateService`) and read/written through it.
|
||||
*
|
||||
* `projectMediaDegraded` / `projectMediaStripped` are the fallback
|
||||
* projections for the two deterministic provider rejections: media-degraded
|
||||
* (all but the most recent media replaced by text markers) resends after an
|
||||
* HTTP 413 body-size rejection; media-stripped captures every media identity
|
||||
* present when degraded media is still too large or an image format is
|
||||
* rejected, then replaces only that snapshot on later steps so a newly
|
||||
* generated recovery image remains visible. Both are read-side only — the
|
||||
* history keeps its media.
|
||||
* `policy.media` selects the fallback projections for the two deterministic
|
||||
* provider rejections: `'degraded'` (all but the most recent media replaced
|
||||
* by text markers) resends after an HTTP 413 body-size rejection;
|
||||
* `{ strip }` replaces only the snapshotted media identities present when
|
||||
* degraded media is still too large or an image format is rejected, so a
|
||||
* newly generated recovery image remains visible on later steps. Both are
|
||||
* read-side only — the history keeps its media.
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto';
|
||||
|
|
@ -40,6 +39,7 @@ import { ITelemetryService } from '#/app/telemetry/telemetry';
|
|||
import {
|
||||
IAgentContextProjectorService,
|
||||
type MediaStripSnapshot,
|
||||
type ProjectionPolicy,
|
||||
} from './contextProjector';
|
||||
|
||||
export const contextProjectorLastRepairSignatureKey = defineState<string | null>(
|
||||
|
|
@ -66,36 +66,24 @@ export class AgentContextProjectorService implements IAgentContextProjectorServi
|
|||
this.states.set(contextProjectorLastRepairSignatureKey, value);
|
||||
}
|
||||
|
||||
project(messages: readonly ContextMessage[]): readonly Message[] {
|
||||
return this.projectWithTrace(messages, project);
|
||||
}
|
||||
|
||||
projectStrict(messages: readonly ContextMessage[]): readonly Message[] {
|
||||
return this.projectWithTrace(messages, projectStrict);
|
||||
}
|
||||
|
||||
projectMediaDegraded(messages: readonly ContextMessage[]): readonly Message[] {
|
||||
return degradeOlderMediaParts(
|
||||
this.projectWithTrace(messages, project),
|
||||
MEDIA_DEGRADE_KEEP_RECENT,
|
||||
project(
|
||||
messages: readonly ContextMessage[],
|
||||
policy: ProjectionPolicy = {},
|
||||
): readonly Message[] {
|
||||
const projected = this.projectWithTrace(
|
||||
messages,
|
||||
policy.structure === 'strict' ? projectStrict : project,
|
||||
);
|
||||
const media = policy.media;
|
||||
if (media === undefined) return projected;
|
||||
if (media === 'degraded') return degradeOlderMediaParts(projected, MEDIA_DEGRADE_KEEP_RECENT);
|
||||
return stripMediaPartsBySnapshot(projected, media.strip);
|
||||
}
|
||||
|
||||
captureMediaStripSnapshot(messages: readonly ContextMessage[]): MediaStripSnapshot {
|
||||
return captureMediaStripSnapshot(this.projectWithTrace(messages, project));
|
||||
}
|
||||
|
||||
projectMediaStripped(
|
||||
messages: readonly ContextMessage[],
|
||||
snapshot?: MediaStripSnapshot,
|
||||
): readonly Message[] {
|
||||
const projected = this.projectWithTrace(messages, project);
|
||||
return stripMediaPartsBySnapshot(
|
||||
projected,
|
||||
snapshot ?? captureMediaStripSnapshot(projected),
|
||||
);
|
||||
}
|
||||
|
||||
private projectWithTrace(
|
||||
messages: readonly ContextMessage[],
|
||||
fn: (history: readonly ContextMessage[], onAnomaly?: (anomaly: ProjectionAnomaly) => void) => Message[],
|
||||
|
|
@ -121,26 +109,17 @@ export class AgentContextProjectorService implements IAgentContextProjectorServi
|
|||
if (signature === this.lastRepairSignature) return;
|
||||
this.lastRepairSignature = signature;
|
||||
|
||||
let reordered = 0;
|
||||
let synthesized = 0;
|
||||
let droppedOrphan = 0;
|
||||
let duplicateCallsDropped = 0;
|
||||
let duplicateResultsDropped = 0;
|
||||
let leadingDropped = 0;
|
||||
let assistantsMerged = 0;
|
||||
let whitespaceDropped = 0;
|
||||
let vacuousDropped = 0;
|
||||
for (const anomaly of notable) {
|
||||
if (anomaly.kind === 'tool_result_reordered') reordered += 1;
|
||||
else if (anomaly.kind === 'tool_result_synthesized') synthesized += 1;
|
||||
else if (anomaly.kind === 'orphan_tool_result_dropped') droppedOrphan += 1;
|
||||
else if (anomaly.kind === 'duplicate_tool_call_dropped') duplicateCallsDropped += 1;
|
||||
else if (anomaly.kind === 'duplicate_tool_result_dropped') duplicateResultsDropped += 1;
|
||||
else if (anomaly.kind === 'leading_non_user_dropped') leadingDropped += 1;
|
||||
else if (anomaly.kind === 'consecutive_assistants_merged') assistantsMerged += 1;
|
||||
else if (anomaly.kind === 'vacuous_message_dropped') vacuousDropped += 1;
|
||||
else whitespaceDropped += 1;
|
||||
}
|
||||
const {
|
||||
reordered,
|
||||
synthesized,
|
||||
droppedOrphan,
|
||||
duplicateCallsDropped,
|
||||
duplicateResultsDropped,
|
||||
leadingDropped,
|
||||
assistantsMerged,
|
||||
whitespaceDropped,
|
||||
vacuousDropped,
|
||||
} = summarizeProjectionRepairs(notable);
|
||||
const toolCallIds = [
|
||||
...new Set(
|
||||
notable.flatMap((anomaly) => ('toolCallId' in anomaly ? [anomaly.toolCallId] : [])),
|
||||
|
|
@ -183,6 +162,46 @@ type ProjectionAnomaly =
|
|||
| { readonly kind: 'whitespace_text_dropped'; readonly role: string }
|
||||
| { readonly kind: 'vacuous_message_dropped'; readonly role: string };
|
||||
|
||||
interface ProjectionRepairSummary {
|
||||
readonly reordered: number;
|
||||
readonly synthesized: number;
|
||||
readonly droppedOrphan: number;
|
||||
readonly duplicateCallsDropped: number;
|
||||
readonly duplicateResultsDropped: number;
|
||||
readonly leadingDropped: number;
|
||||
readonly assistantsMerged: number;
|
||||
readonly whitespaceDropped: number;
|
||||
readonly vacuousDropped: number;
|
||||
}
|
||||
|
||||
function summarizeProjectionRepairs(
|
||||
anomalies: readonly ProjectionAnomaly[],
|
||||
): ProjectionRepairSummary {
|
||||
const summary = {
|
||||
reordered: 0,
|
||||
synthesized: 0,
|
||||
droppedOrphan: 0,
|
||||
duplicateCallsDropped: 0,
|
||||
duplicateResultsDropped: 0,
|
||||
leadingDropped: 0,
|
||||
assistantsMerged: 0,
|
||||
whitespaceDropped: 0,
|
||||
vacuousDropped: 0,
|
||||
};
|
||||
for (const anomaly of anomalies) {
|
||||
if (anomaly.kind === 'tool_result_reordered') summary.reordered += 1;
|
||||
else if (anomaly.kind === 'tool_result_synthesized') summary.synthesized += 1;
|
||||
else if (anomaly.kind === 'orphan_tool_result_dropped') summary.droppedOrphan += 1;
|
||||
else if (anomaly.kind === 'duplicate_tool_call_dropped') summary.duplicateCallsDropped += 1;
|
||||
else if (anomaly.kind === 'duplicate_tool_result_dropped') summary.duplicateResultsDropped += 1;
|
||||
else if (anomaly.kind === 'leading_non_user_dropped') summary.leadingDropped += 1;
|
||||
else if (anomaly.kind === 'consecutive_assistants_merged') summary.assistantsMerged += 1;
|
||||
else if (anomaly.kind === 'vacuous_message_dropped') summary.vacuousDropped += 1;
|
||||
else summary.whitespaceDropped += 1;
|
||||
}
|
||||
return summary;
|
||||
}
|
||||
|
||||
type OnAnomaly = (anomaly: ProjectionAnomaly) => void;
|
||||
|
||||
export const MEDIA_DEGRADE_KEEP_RECENT = 2;
|
||||
|
|
|
|||
|
|
@ -61,7 +61,7 @@ const llmRequestSchema = z.object({
|
|||
messageCount: z.number(),
|
||||
turnStep: z.string().optional(),
|
||||
attempt: z.string().optional(),
|
||||
projection: z.enum(['strict', 'media-degraded', 'media-stripped']).optional(),
|
||||
projection: z.enum(['strict', 'media-degraded', 'media-stripped', 'strict-media-degraded', 'strict-media-stripped']).optional(),
|
||||
droppedCount: z.number().optional(),
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -7,8 +7,10 @@
|
|||
* folds the completion-token budget into the profile's dialect-free intent
|
||||
* params, then drives a bounded request chain through the `ModelRequester`
|
||||
* resolved from `IModelCatalog`: one primary `requester.request(input, signal,
|
||||
* params)` attempt plus projection rebuilds for request structure or media
|
||||
* compatibility. Before each request the projected messages pass through `media`'s
|
||||
* params)` attempt plus accumulating projection rebuilds — each repeated
|
||||
* provider rejection (request structure, body size, image format) adds its own
|
||||
* repair on top of the ones already applied. Before each request the projected
|
||||
* messages pass through `media`'s
|
||||
* media resolver, which rewrites every `kimi-file://` prompt-media reference
|
||||
* to a provider-acceptable part (an uploaded `ms://` video, an inline base64
|
||||
* `data:` part, or a degradation tag/drop) so the internal reference never
|
||||
|
|
@ -41,6 +43,7 @@ import { IAgentContextMemoryService } from '#/agent/contextMemory/contextMemory'
|
|||
import {
|
||||
IAgentContextProjectorService,
|
||||
type MediaStripSnapshot,
|
||||
type ProjectionPolicy,
|
||||
} from '#/agent/contextProjector/contextProjector';
|
||||
import { IAgentTokenCountingService } from '#/agent/tokenCounting/tokenCounting';
|
||||
import { IAgentProfileService, type ProfileModelContext } from '#/agent/profile/profile';
|
||||
|
|
@ -128,8 +131,6 @@ interface ResolvedLLMRequest {
|
|||
readonly logFields: AgentLLMRequestLogFields;
|
||||
}
|
||||
|
||||
type RequestProjection = 'normal' | 'strict' | 'media-degraded' | 'media-stripped';
|
||||
|
||||
interface LLMRequestLogInput {
|
||||
readonly protocol: Protocol;
|
||||
readonly providerType?: string;
|
||||
|
|
@ -342,41 +343,34 @@ export class AgentLLMRequesterService implements IAgentLLMRequesterService {
|
|||
): Promise<AgentLLMRequestFinish> {
|
||||
this.toolCallIdNormalizer.seedFrom(this.context.get());
|
||||
const shaped = this.toolSelect.shapeHistory(request.messages);
|
||||
let mediaStripSnapshot = this.mediaStripSnapshotForTurn(request.source);
|
||||
const requestInput = (projection: RequestProjection) => {
|
||||
return {
|
||||
const recoveredStrip = this.mediaStripSnapshotForTurn(request.source);
|
||||
let policy: ProjectionPolicy | undefined =
|
||||
recoveredStrip !== undefined
|
||||
? { media: { strip: recoveredStrip } }
|
||||
: this.isRecoveryTurn(this.mediaDegradedTurns, request.source)
|
||||
? { media: 'degraded' }
|
||||
: undefined;
|
||||
const captureMediaStripPolicy = (): { readonly strip: MediaStripSnapshot } => {
|
||||
const snapshot = this.projector.captureMediaStripSnapshot(shaped);
|
||||
this.markMediaStrippedRecoveryTurn(snapshot, request.source);
|
||||
return { strip: snapshot };
|
||||
};
|
||||
const run = async (
|
||||
policy: ProjectionPolicy | undefined,
|
||||
): Promise<AgentLLMRequestFinish> => {
|
||||
onRequestTrace(undefined);
|
||||
const projection = projectionNameOf(policy);
|
||||
const fields =
|
||||
projection === undefined ? request.logFields : { ...request.logFields, projection };
|
||||
const input = {
|
||||
systemPrompt: request.systemPrompt,
|
||||
tools: request.tools,
|
||||
messages:
|
||||
projection === 'strict'
|
||||
? this.projector.projectStrict(shaped)
|
||||
: projection === 'media-degraded'
|
||||
? this.projector.projectMediaDegraded(shaped)
|
||||
: projection === 'media-stripped'
|
||||
? this.projector.projectMediaStripped(
|
||||
shaped,
|
||||
(mediaStripSnapshot ??=
|
||||
this.projector.captureMediaStripSnapshot(shaped)),
|
||||
)
|
||||
: this.projector.project(shaped),
|
||||
};
|
||||
};
|
||||
|
||||
const run = async (projection: RequestProjection): Promise<AgentLLMRequestFinish> => {
|
||||
onRequestTrace(undefined);
|
||||
const projected = requestInput(projection);
|
||||
const input = {
|
||||
...projected,
|
||||
messages: await this.mediaResolver.resolve(
|
||||
projected.messages,
|
||||
this.projector.project(shaped, policy),
|
||||
request.requester,
|
||||
signal,
|
||||
),
|
||||
};
|
||||
const fields =
|
||||
projection === 'normal'
|
||||
? request.logFields
|
||||
: { ...request.logFields, projection };
|
||||
this.warnAboutAnthropicThinkingEffort(request);
|
||||
const logInput: LLMRequestLogInput = {
|
||||
protocol: request.model.protocol,
|
||||
|
|
@ -473,75 +467,77 @@ export class AgentLLMRequesterService implements IAgentLLMRequesterService {
|
|||
};
|
||||
};
|
||||
|
||||
const initialProjection: RequestProjection = mediaStripSnapshot !== undefined
|
||||
? 'media-stripped'
|
||||
: this.isRecoveryTurn(this.mediaDegradedTurns, request.source)
|
||||
? 'media-degraded'
|
||||
: 'normal';
|
||||
let projection: RequestProjection = initialProjection;
|
||||
for (;;) {
|
||||
try {
|
||||
return await run(projection);
|
||||
return await run(policy);
|
||||
} catch (error) {
|
||||
if (signal?.aborted === true) throw error;
|
||||
const raw = unwrapErrorCause(error);
|
||||
if (
|
||||
raw instanceof APIRequestTooLargeError &&
|
||||
(projection === 'normal' || projection === 'media-degraded')
|
||||
) {
|
||||
signal?.throwIfAborted();
|
||||
if (projection === 'normal') {
|
||||
this.log.warn(
|
||||
'provider rejected request as too large; resending with degraded media',
|
||||
{
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
},
|
||||
);
|
||||
this.markRecoveryTurn(this.mediaDegradedTurns, request.source);
|
||||
projection = 'media-degraded';
|
||||
} else {
|
||||
this.log.warn(
|
||||
'provider rejected degraded-media request as too large; resending with rejected media stripped',
|
||||
{
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
},
|
||||
);
|
||||
mediaStripSnapshot = this.projector.captureMediaStripSnapshot(shaped);
|
||||
this.markMediaStrippedRecoveryTurn(mediaStripSnapshot, request.source);
|
||||
projection = 'media-stripped';
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (projection !== 'media-stripped' && isImageFormatError(raw)) {
|
||||
signal?.throwIfAborted();
|
||||
this.log.warn(
|
||||
'provider rejected an image in the request; resending with rejected media stripped',
|
||||
{
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
},
|
||||
);
|
||||
mediaStripSnapshot = this.projector.captureMediaStripSnapshot(shaped);
|
||||
this.markMediaStrippedRecoveryTurn(mediaStripSnapshot, request.source);
|
||||
projection = 'media-stripped';
|
||||
continue;
|
||||
}
|
||||
if (projection === 'normal' && isRecoverableRequestStructureError(raw)) {
|
||||
signal?.throwIfAborted();
|
||||
this.log.warn('provider rejected request structure; resending with strict projection', {
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
});
|
||||
projection = 'strict';
|
||||
continue;
|
||||
}
|
||||
throw error;
|
||||
const nextPolicy = this.nextProjectionPolicyForError(
|
||||
error,
|
||||
policy,
|
||||
request,
|
||||
signal,
|
||||
captureMediaStripPolicy,
|
||||
);
|
||||
if (nextPolicy === undefined) throw error;
|
||||
policy = nextPolicy;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private nextProjectionPolicyForError(
|
||||
error: unknown,
|
||||
policy: ProjectionPolicy | undefined,
|
||||
request: ResolvedLLMRequest,
|
||||
signal: AbortSignal | undefined,
|
||||
captureMediaStripPolicy: () => { readonly strip: MediaStripSnapshot },
|
||||
): ProjectionPolicy | undefined {
|
||||
if (signal?.aborted === true) return undefined;
|
||||
const raw = unwrapErrorCause(error);
|
||||
const media = policy?.media;
|
||||
if (
|
||||
raw instanceof APIRequestTooLargeError &&
|
||||
(media === undefined || media === 'degraded')
|
||||
) {
|
||||
signal?.throwIfAborted();
|
||||
if (media === undefined) {
|
||||
this.log.warn('provider rejected request as too large; resending with degraded media', {
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
});
|
||||
this.markRecoveryTurn(this.mediaDegradedTurns, request.source);
|
||||
return { ...policy, media: 'degraded' };
|
||||
}
|
||||
this.log.warn(
|
||||
'provider rejected degraded-media request as too large; resending with rejected media stripped',
|
||||
{
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
},
|
||||
);
|
||||
return { ...policy, media: captureMediaStripPolicy() };
|
||||
}
|
||||
if (typeof media !== 'object' && isImageFormatError(raw)) {
|
||||
signal?.throwIfAborted();
|
||||
this.log.warn(
|
||||
'provider rejected an image in the request; resending with rejected media stripped',
|
||||
{
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
},
|
||||
);
|
||||
return { ...policy, media: captureMediaStripPolicy() };
|
||||
}
|
||||
if (policy?.structure === undefined && isRecoverableRequestStructureError(raw)) {
|
||||
signal?.throwIfAborted();
|
||||
this.log.warn('provider rejected request structure; resending with strict projection', {
|
||||
model: request.model.name,
|
||||
...request.logFields,
|
||||
});
|
||||
return { ...policy, structure: 'strict' };
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
private normalizeStreamPart(
|
||||
toolCallIds: ToolCallIdResponseNormalizer,
|
||||
part: StreamedMessagePart,
|
||||
|
|
@ -833,13 +829,32 @@ function numberField(fields: AgentLLMRequestLogFields, key: string): number | un
|
|||
return typeof value === 'number' ? value : undefined;
|
||||
}
|
||||
|
||||
function projectionField(
|
||||
fields: AgentLLMRequestLogFields,
|
||||
): 'strict' | 'media-degraded' | 'media-stripped' | undefined {
|
||||
type LlmRequestProjection = NonNullable<LlmRequestPayload['projection']>;
|
||||
|
||||
function projectionNameOf(policy: ProjectionPolicy | undefined): LlmRequestProjection | undefined {
|
||||
if (policy?.structure === 'strict') {
|
||||
if (policy.media === 'degraded') return 'strict-media-degraded';
|
||||
if (typeof policy.media === 'object') return 'strict-media-stripped';
|
||||
return 'strict';
|
||||
}
|
||||
if (policy === undefined) return undefined;
|
||||
if (policy.media === 'degraded') return 'media-degraded';
|
||||
if (typeof policy.media === 'object') return 'media-stripped';
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function projectionField(fields: AgentLLMRequestLogFields): LlmRequestProjection | undefined {
|
||||
const value = fields['projection'];
|
||||
return value === 'strict' || value === 'media-degraded' || value === 'media-stripped'
|
||||
? value
|
||||
: undefined;
|
||||
switch (value) {
|
||||
case 'strict':
|
||||
case 'media-degraded':
|
||||
case 'media-stripped':
|
||||
case 'strict-media-degraded':
|
||||
case 'strict-media-stripped':
|
||||
return value;
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
function fingerprint(content: string): string {
|
||||
|
|
|
|||
|
|
@ -844,6 +844,32 @@ describe('Agent context', () => {
|
|||
expect(withOverhead.messages).toEqual(withoutOverhead.messages);
|
||||
});
|
||||
});
|
||||
|
||||
describe('legacy compaction layout', () => {
|
||||
it('keeps the verbatim summary followed by the uncompacted tail', () => {
|
||||
const history = [userMessage('old'), userMessage('tail')];
|
||||
const legacySummary: ContextMessage = {
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'legacy summary' }],
|
||||
toolCalls: [],
|
||||
origin: { kind: 'compaction_summary' },
|
||||
};
|
||||
const input = {
|
||||
summary: 'legacy summary',
|
||||
legacySummaryMessage: legacySummary,
|
||||
compactedCount: 1,
|
||||
tokensBefore: 100,
|
||||
tokensAfter: 20,
|
||||
legacyTail: true,
|
||||
};
|
||||
|
||||
const shape = buildContextCompactionShape(history, input);
|
||||
|
||||
expect(shape.messages[0]).toBe(legacySummary);
|
||||
expect(shape.messages[1]).toBe(history[1]);
|
||||
expect(shape.messages.map(textOf)).toEqual(['legacy summary', 'tail']);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
function userMessage(text: string, origin?: ContextMessage['origin']): ContextMessage {
|
||||
|
|
|
|||
|
|
@ -137,7 +137,7 @@ describe('projector tool-exchange normalization', () => {
|
|||
}
|
||||
|
||||
function projectStrict(history: readonly ContextMessage[]): readonly Message[] {
|
||||
return projector.projectStrict(history);
|
||||
return projector.project(history, { structure: 'strict' });
|
||||
}
|
||||
|
||||
it('leaves a fully resolved exchange untouched', () => {
|
||||
|
|
@ -656,7 +656,7 @@ describe('projector tool-exchange normalization', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('projectMediaDegraded', () => {
|
||||
describe('project with media: degraded policy', () => {
|
||||
function imageMessage(url: string): ContextMessage {
|
||||
return {
|
||||
role: 'user',
|
||||
|
|
@ -667,13 +667,16 @@ describe('projector tool-exchange normalization', () => {
|
|||
}
|
||||
|
||||
it('keeps the two most recent media parts and replaces older ones with markers', () => {
|
||||
const projected = projector.projectMediaDegraded([
|
||||
imageMessage('data:image/png;base64,OLD1'),
|
||||
user('middle'),
|
||||
imageMessage('data:image/png;base64,OLD2'),
|
||||
imageMessage('data:image/png;base64,KEEP1'),
|
||||
imageMessage('data:image/png;base64,KEEP2'),
|
||||
]);
|
||||
const projected = projector.project(
|
||||
[
|
||||
imageMessage('data:image/png;base64,OLD1'),
|
||||
user('middle'),
|
||||
imageMessage('data:image/png;base64,OLD2'),
|
||||
imageMessage('data:image/png;base64,KEEP1'),
|
||||
imageMessage('data:image/png;base64,KEEP2'),
|
||||
],
|
||||
{ media: 'degraded' },
|
||||
);
|
||||
|
||||
const urls = projected
|
||||
.flatMap((message) => message.content)
|
||||
|
|
@ -690,16 +693,16 @@ describe('projector tool-exchange normalization', () => {
|
|||
});
|
||||
|
||||
it('returns the projected messages untouched when media fits within keep-recent', () => {
|
||||
const projected = projector.projectMediaDegraded([
|
||||
user('text'),
|
||||
imageMessage('data:image/png;base64,AAAA'),
|
||||
]);
|
||||
const projected = projector.project(
|
||||
[user('text'), imageMessage('data:image/png;base64,AAAA')],
|
||||
{ media: 'degraded' },
|
||||
);
|
||||
const allParts = projected.flatMap((message) => message.content);
|
||||
expect(allParts.some((part) => part.type === 'image_url')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('projectMediaStripped', () => {
|
||||
describe('project with media: stripped policy', () => {
|
||||
function imageMessage(url: string, id?: string): ContextMessage {
|
||||
return {
|
||||
role: 'user',
|
||||
|
|
@ -709,8 +712,15 @@ describe('projector tool-exchange normalization', () => {
|
|||
};
|
||||
}
|
||||
|
||||
function projectStripped(
|
||||
history: readonly ContextMessage[],
|
||||
snapshot = projector.captureMediaStripSnapshot(history),
|
||||
): readonly Message[] {
|
||||
return projector.project(history, { media: { strip: snapshot } });
|
||||
}
|
||||
|
||||
it('replaces every media part with a text marker, keeping the surrounding text', () => {
|
||||
const projected = projector.projectMediaStripped([
|
||||
const projected = projectStripped([
|
||||
user('look at these'),
|
||||
imageMessage('data:image/png;base64,AAAA'),
|
||||
{
|
||||
|
|
@ -742,7 +752,7 @@ describe('projector tool-exchange normalization', () => {
|
|||
});
|
||||
|
||||
it('returns the projected messages untouched when there is no media', () => {
|
||||
const projected = projector.projectMediaStripped([user('just text')]);
|
||||
const projected = projectStripped([user('just text')]);
|
||||
expect(projected).toEqual(project([user('just text')]));
|
||||
});
|
||||
|
||||
|
|
@ -750,7 +760,7 @@ describe('projector tool-exchange normalization', () => {
|
|||
const rejected = imageMessage('data:image/png;base64,OLD', 'old-id');
|
||||
const snapshot = projector.captureMediaStripSnapshot([rejected]);
|
||||
|
||||
const projected = projector.projectMediaStripped(
|
||||
const projected = projectStripped(
|
||||
[rejected, imageMessage('data:image/png;base64,NEW', 'new-id')],
|
||||
snapshot,
|
||||
);
|
||||
|
|
@ -776,7 +786,7 @@ describe('projector tool-exchange normalization', () => {
|
|||
orphan,
|
||||
]);
|
||||
|
||||
const projected = projector.projectMediaStripped(
|
||||
const projected = projectStripped(
|
||||
[imageMessage(url, 'orphan-id')],
|
||||
snapshot,
|
||||
);
|
||||
|
|
@ -793,7 +803,7 @@ describe('projector tool-exchange normalization', () => {
|
|||
imageMessage('data:image/png;base64,SAME', 'same-id'),
|
||||
]);
|
||||
|
||||
const projected = projector.projectMediaStripped(
|
||||
const projected = projectStripped(
|
||||
[imageMessage('data:image/png;base64,SAME', 'same-id')],
|
||||
snapshot,
|
||||
);
|
||||
|
|
@ -809,7 +819,7 @@ describe('projector tool-exchange normalization', () => {
|
|||
const url = 'https://example.test/media/image.png';
|
||||
const snapshot = projector.captureMediaStripSnapshot([imageMessage(url, 'old-id')]);
|
||||
|
||||
const projected = projector.projectMediaStripped(
|
||||
const projected = projectStripped(
|
||||
[imageMessage(url, 'new-id')],
|
||||
snapshot,
|
||||
);
|
||||
|
|
|
|||
|
|
@ -1,14 +1,13 @@
|
|||
/**
|
||||
* Scenario: LLM requester uses bounded recovery projections after a
|
||||
* deterministic provider rejection — strict projection for tool-use
|
||||
* adjacency, degraded media followed by full stripping for body-size 413s,
|
||||
* and media stripping for image-format rejections.
|
||||
* Scenario: LLM requests encounter deterministic provider rejections that
|
||||
* require strict tool adjacency, degraded or stripped media, and accumulated
|
||||
* recovery policies across retries.
|
||||
*
|
||||
* Responsibilities: assert retry eligibility, projection order and bounds,
|
||||
* per-turn recovery stickiness, request recording, and usage accounting.
|
||||
* Wiring: real AgentLLMRequesterService with stubbed context memory,
|
||||
* projector, context sizing, profile, model, telemetry, and wire/log services. Run:
|
||||
* pnpm test -- test/agent/llmRequester/llmRequesterService.test.ts
|
||||
* Responsibilities: assert retry eligibility and bounds, projection order,
|
||||
* per-turn recovery state, request recording, normalization, and accounting.
|
||||
* Wiring: real AgentLLMRequesterService with controlled model and stubbed
|
||||
* context, projector, profile, tool, telemetry, state, wire, and log services.
|
||||
* Run: pnpm test -- test/agent/llmRequester/llmRequesterService.test.ts
|
||||
*/
|
||||
|
||||
import { createControlledPromise } from '@antfu/utils';
|
||||
|
|
@ -22,6 +21,7 @@ import type { ContextMessage } from '#/agent/contextMemory/types';
|
|||
import {
|
||||
IAgentContextProjectorService,
|
||||
type MediaStripSnapshot,
|
||||
type ProjectionPolicy,
|
||||
} from '#/agent/contextProjector/contextProjector';
|
||||
import { AgentContextProjectorService } from '#/agent/contextProjector/contextProjectorService';
|
||||
import { AgentLLMRequesterService } from '#/agent/llmRequester/llmRequesterService';
|
||||
|
|
@ -85,6 +85,31 @@ const history: Message[] = [
|
|||
{ role: 'user', content: [{ type: 'text', text: 'hello' }], toolCalls: [] },
|
||||
];
|
||||
|
||||
type ProjectionKind = 'normal' | 'strict' | 'degraded' | 'stripped';
|
||||
|
||||
function classifyProjectionPolicy(policy: ProjectionPolicy | undefined): ProjectionKind {
|
||||
if (typeof policy?.media === 'object') return 'stripped';
|
||||
if (policy?.media === 'degraded') return 'degraded';
|
||||
if (policy?.structure === 'strict') return 'strict';
|
||||
return 'normal';
|
||||
}
|
||||
|
||||
function recordProjectionCalls(): {
|
||||
projector: Pick<IAgentContextProjectorService, 'project'>;
|
||||
calls: ProjectionKind[];
|
||||
} {
|
||||
const calls: ProjectionKind[] = [];
|
||||
return {
|
||||
projector: {
|
||||
project: (messages: readonly ContextMessage[], policy) => {
|
||||
calls.push(classifyProjectionPolicy(policy));
|
||||
return messages;
|
||||
},
|
||||
},
|
||||
calls,
|
||||
};
|
||||
}
|
||||
|
||||
function createRequester(
|
||||
calls: { value: number },
|
||||
firstCallError?: Error | null,
|
||||
|
|
@ -139,15 +164,8 @@ afterEach(() => disposables.dispose());
|
|||
function createService(
|
||||
requester: ModelRequester,
|
||||
projector:
|
||||
| (Pick<IAgentContextProjectorService, 'project' | 'projectStrict'> &
|
||||
Partial<
|
||||
Pick<
|
||||
IAgentContextProjectorService,
|
||||
| 'captureMediaStripSnapshot'
|
||||
| 'projectMediaDegraded'
|
||||
| 'projectMediaStripped'
|
||||
>
|
||||
>)
|
||||
| (Pick<IAgentContextProjectorService, 'project'> &
|
||||
Partial<Pick<IAgentContextProjectorService, 'captureMediaStripSnapshot'>>)
|
||||
| undefined,
|
||||
options: {
|
||||
readonly thinkingLevel?: ThinkingEffort;
|
||||
|
|
@ -185,7 +203,9 @@ function createService(
|
|||
},
|
||||
};
|
||||
const usage = { record: () => undefined, status: () => ({}) };
|
||||
const context = { get: () => options.contextMessages ?? history };
|
||||
const context = {
|
||||
get: () => options.contextMessages ?? history,
|
||||
};
|
||||
const tools = { list: () => [] };
|
||||
const config: Partial<IConfigService> = {
|
||||
get: (() => undefined) as IConfigService['get'],
|
||||
|
|
@ -217,8 +237,6 @@ function createService(
|
|||
} else {
|
||||
ix.stub(IAgentContextProjectorService, {
|
||||
captureMediaStripSnapshot: () => testSnapshot,
|
||||
projectMediaDegraded: projector.project,
|
||||
projectMediaStripped: projector.project,
|
||||
...projector,
|
||||
});
|
||||
}
|
||||
|
|
@ -311,26 +329,15 @@ describe('AgentLLMRequesterService Anthropic effort diagnostics', () => {
|
|||
describe('AgentLLMRequesterService strict resend', () => {
|
||||
it('resends once with strict projection after a recoverable structural 400', async () => {
|
||||
const calls = { value: 0 };
|
||||
let projectCalls = 0;
|
||||
let strictCalls = 0;
|
||||
const { service } = createService(createRequester(calls), {
|
||||
project: (messages: readonly ContextMessage[]) => {
|
||||
projectCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectStrict: (messages: readonly ContextMessage[]) => {
|
||||
strictCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
});
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(createRequester(calls), projection.projector);
|
||||
|
||||
const result = await service.request();
|
||||
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'ok' }]);
|
||||
expect(result.usage).toEqual(emptyUsage());
|
||||
expect(calls.value).toBe(2);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(strictCalls).toBe(1);
|
||||
expect(projection.calls).toEqual(['normal', 'strict']);
|
||||
});
|
||||
|
||||
it('does not resend for non-recoverable errors', async () => {
|
||||
|
|
@ -342,19 +349,13 @@ describe('AgentLLMRequesterService strict resend', () => {
|
|||
throw new APIStatusError(401, 'unauthorized');
|
||||
},
|
||||
});
|
||||
let strictCalls = 0;
|
||||
const { service } = createService(requester, {
|
||||
project: (messages: readonly ContextMessage[]) => messages,
|
||||
projectStrict: (messages: readonly ContextMessage[]) => {
|
||||
strictCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
});
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(requester, projection.projector);
|
||||
|
||||
await expect(service.request()).rejects.toMatchObject({
|
||||
statusCode: 401,
|
||||
});
|
||||
expect(strictCalls).toBe(0);
|
||||
expect(projection.calls).toEqual(['normal']);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -366,78 +367,41 @@ describe('AgentLLMRequesterService media-stripped resend', () => {
|
|||
|
||||
it('resends once with the media-stripped projection after an image-format 400', async () => {
|
||||
const calls = { value: 0 };
|
||||
let projectCalls = 0;
|
||||
let strictCalls = 0;
|
||||
let strippedCalls = 0;
|
||||
const { service } = createService(createRequester(calls, IMAGE_FORMAT_400), {
|
||||
project: (messages: readonly ContextMessage[]) => {
|
||||
projectCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectStrict: (messages: readonly ContextMessage[]) => {
|
||||
strictCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectMediaStripped: (messages: readonly ContextMessage[]) => {
|
||||
strippedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
});
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(createRequester(calls, IMAGE_FORMAT_400), projection.projector);
|
||||
|
||||
const result = await service.request();
|
||||
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'ok' }]);
|
||||
expect(calls.value).toBe(2);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(strictCalls).toBe(0);
|
||||
expect(strippedCalls).toBe(1);
|
||||
expect(projection.calls).toEqual(['normal', 'stripped']);
|
||||
});
|
||||
|
||||
it('keeps later steps of the same turn on the stripped projection', async () => {
|
||||
const calls = { value: 0 };
|
||||
let projectCalls = 0;
|
||||
let strippedCalls = 0;
|
||||
const { service } = createService(createRequester(calls, IMAGE_FORMAT_400), {
|
||||
project: (messages: readonly ContextMessage[]) => {
|
||||
projectCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaStripped: (messages: readonly ContextMessage[]) => {
|
||||
strippedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
});
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(createRequester(calls, IMAGE_FORMAT_400), projection.projector);
|
||||
|
||||
await service.request({ source: { type: 'turn', turnId: 1, step: 1 } });
|
||||
expect(calls.value).toBe(2);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(strippedCalls).toBe(1);
|
||||
expect(projection.calls).toEqual(['normal', 'stripped']);
|
||||
|
||||
await service.request({ source: { type: 'turn', turnId: 1, step: 2 } });
|
||||
expect(calls.value).toBe(3);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(strippedCalls).toBe(2);
|
||||
expect(projection.calls).toEqual(['normal', 'stripped', 'stripped']);
|
||||
});
|
||||
|
||||
it('does not resend for an unrelated 400', async () => {
|
||||
const calls = { value: 0 };
|
||||
let strippedCalls = 0;
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(
|
||||
createRequester(calls, new APIStatusError(400, 'some other validation problem')),
|
||||
{
|
||||
project: (messages: readonly ContextMessage[]) => messages,
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaStripped: (messages: readonly ContextMessage[]) => {
|
||||
strippedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
},
|
||||
projection.projector,
|
||||
);
|
||||
|
||||
await expect(service.request()).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(calls.value).toBe(1);
|
||||
expect(strippedCalls).toBe(0);
|
||||
expect(projection.calls).toEqual(['normal']);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -446,9 +410,7 @@ describe('AgentLLMRequesterService media-degraded resend', () => {
|
|||
|
||||
it('resends once with the media-degraded projection after an HTTP 413', async () => {
|
||||
const calls = { value: 0 };
|
||||
let projectCalls = 0;
|
||||
let degradedCalls = 0;
|
||||
let strippedCalls = 0;
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(
|
||||
createRequester(
|
||||
calls,
|
||||
|
|
@ -456,63 +418,29 @@ describe('AgentLLMRequesterService media-degraded resend', () => {
|
|||
cause: BODY_TOO_LARGE_413,
|
||||
}),
|
||||
),
|
||||
{
|
||||
project: (messages: readonly ContextMessage[]) => {
|
||||
projectCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaDegraded: (messages: readonly ContextMessage[]) => {
|
||||
degradedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectMediaStripped: (messages: readonly ContextMessage[]) => {
|
||||
strippedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
},
|
||||
projection.projector,
|
||||
);
|
||||
|
||||
const result = await service.request();
|
||||
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'ok' }]);
|
||||
expect(calls.value).toBe(2);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(degradedCalls).toBe(1);
|
||||
expect(strippedCalls).toBe(0);
|
||||
expect(projection.calls).toEqual(['normal', 'degraded']);
|
||||
});
|
||||
|
||||
it('falls back to media-stripped when the media-degraded request still receives 413', async () => {
|
||||
const calls = { value: 0 };
|
||||
let projectCalls = 0;
|
||||
let degradedCalls = 0;
|
||||
let strippedCalls = 0;
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(
|
||||
createRequester(calls, BODY_TOO_LARGE_413, [BODY_TOO_LARGE_413]),
|
||||
{
|
||||
project: (messages: readonly ContextMessage[]) => {
|
||||
projectCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaDegraded: (messages: readonly ContextMessage[]) => {
|
||||
degradedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectMediaStripped: (messages: readonly ContextMessage[]) => {
|
||||
strippedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
},
|
||||
projection.projector,
|
||||
);
|
||||
|
||||
const result = await service.request({ source: { type: 'turn', turnId: 1, step: 1 } });
|
||||
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'ok' }]);
|
||||
expect(calls.value).toBe(3);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(degradedCalls).toBe(1);
|
||||
expect(strippedCalls).toBe(1);
|
||||
expect(projection.calls).toEqual(['normal', 'degraded', 'stripped']);
|
||||
});
|
||||
|
||||
it('records repeated-413 recovery projections on the sticky later request', async () => {
|
||||
|
|
@ -521,9 +449,6 @@ describe('AgentLLMRequesterService media-degraded resend', () => {
|
|||
createRequester(calls, BODY_TOO_LARGE_413, [BODY_TOO_LARGE_413]),
|
||||
{
|
||||
project: (messages: readonly ContextMessage[]) => messages,
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaDegraded: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaStripped: (messages: readonly ContextMessage[]) => messages,
|
||||
},
|
||||
);
|
||||
|
||||
|
|
@ -580,62 +505,31 @@ describe('AgentLLMRequesterService media-degraded resend', () => {
|
|||
|
||||
it('stops after the media-stripped request also receives 413', async () => {
|
||||
const calls = { value: 0 };
|
||||
let projectCalls = 0;
|
||||
let degradedCalls = 0;
|
||||
let strippedCalls = 0;
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(
|
||||
createRequester(calls, BODY_TOO_LARGE_413, [BODY_TOO_LARGE_413, BODY_TOO_LARGE_413]),
|
||||
{
|
||||
project: (messages: readonly ContextMessage[]) => {
|
||||
projectCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaDegraded: (messages: readonly ContextMessage[]) => {
|
||||
degradedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectMediaStripped: (messages: readonly ContextMessage[]) => {
|
||||
strippedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
},
|
||||
projection.projector,
|
||||
);
|
||||
|
||||
await expect(
|
||||
service.request({ source: { type: 'turn', turnId: 1, step: 1 } }),
|
||||
).rejects.toBe(BODY_TOO_LARGE_413);
|
||||
expect(calls.value).toBe(3);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(degradedCalls).toBe(1);
|
||||
expect(strippedCalls).toBe(1);
|
||||
expect(projection.calls).toEqual(['normal', 'degraded', 'stripped']);
|
||||
});
|
||||
|
||||
it('keeps later steps of the same turn on the degraded projection', async () => {
|
||||
const calls = { value: 0 };
|
||||
let projectCalls = 0;
|
||||
let degradedCalls = 0;
|
||||
const { service } = createService(createRequester(calls, BODY_TOO_LARGE_413), {
|
||||
project: (messages: readonly ContextMessage[]) => {
|
||||
projectCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaDegraded: (messages: readonly ContextMessage[]) => {
|
||||
degradedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
});
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(createRequester(calls, BODY_TOO_LARGE_413), projection.projector);
|
||||
|
||||
await service.request({ source: { type: 'turn', turnId: 1, step: 1 } });
|
||||
expect(calls.value).toBe(2);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(degradedCalls).toBe(1);
|
||||
expect(projection.calls).toEqual(['normal', 'degraded']);
|
||||
|
||||
await service.request({ source: { type: 'turn', turnId: 1, step: 2 } });
|
||||
expect(calls.value).toBe(3);
|
||||
expect(projectCalls).toBe(1);
|
||||
expect(degradedCalls).toBe(2);
|
||||
expect(projection.calls).toEqual(['normal', 'degraded', 'degraded']);
|
||||
});
|
||||
|
||||
it('does not resend for a plain 400 or a non-413 status', async () => {
|
||||
|
|
@ -644,27 +538,95 @@ describe('AgentLLMRequesterService media-degraded resend', () => {
|
|||
new APIStatusError(422, 'unprocessable'),
|
||||
]) {
|
||||
const calls = { value: 0 };
|
||||
let degradedCalls = 0;
|
||||
const { service } = createService(createRequester(calls, error), {
|
||||
project: (messages: readonly ContextMessage[]) => messages,
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
projectMediaDegraded: (messages: readonly ContextMessage[]) => {
|
||||
degradedCalls += 1;
|
||||
return messages;
|
||||
},
|
||||
});
|
||||
const projection = recordProjectionCalls();
|
||||
const { service } = createService(createRequester(calls, error), projection.projector);
|
||||
|
||||
await expect(service.request()).rejects.toBe(error);
|
||||
expect(calls.value).toBe(1);
|
||||
expect(degradedCalls).toBe(0);
|
||||
expect(projection.calls).toEqual(['normal']);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('AgentLLMRequesterService combined recovery projections', () => {
|
||||
const BODY_TOO_LARGE_413 = new APIRequestTooLargeError(413, 'Request Entity Too Large');
|
||||
const IMAGE_FORMAT_400 = new APIStatusError(
|
||||
400,
|
||||
'unsupported image format: image/avif is not supported',
|
||||
);
|
||||
const STRUCTURAL_400 = new APIStatusError(400, 'messages: `tool_use` ids must be unique');
|
||||
|
||||
function createPolicyRecordingProjector(policies: {
|
||||
policies: (ProjectionPolicy | undefined)[];
|
||||
}): Pick<IAgentContextProjectorService, 'project'> {
|
||||
return {
|
||||
project: (messages: readonly ContextMessage[], policy) => {
|
||||
policies.policies.push(policy);
|
||||
return messages;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
it('accumulates media repairs on top of strict across repeated rejections', async () => {
|
||||
const calls = { value: 0 };
|
||||
const policies: (ProjectionPolicy | undefined)[] = [];
|
||||
const { service, dispatcher, records } = createService(
|
||||
createRequester(calls, STRUCTURAL_400, [BODY_TOO_LARGE_413, BODY_TOO_LARGE_413]),
|
||||
createPolicyRecordingProjector({ policies }),
|
||||
);
|
||||
|
||||
await service.request({ source: { type: 'turn', turnId: 1, step: 1 } });
|
||||
|
||||
expect(calls.value).toBe(4);
|
||||
expect(policies).toEqual([
|
||||
undefined,
|
||||
{ structure: 'strict' },
|
||||
{ structure: 'strict', media: 'degraded' },
|
||||
{ structure: 'strict', media: { strip: expect.anything() } },
|
||||
]);
|
||||
await dispatcher.flush();
|
||||
expect(
|
||||
records.filter((record) => record.type === 'llm.request').map((record) => record['projection']),
|
||||
).toEqual([undefined, 'strict', 'strict-media-degraded', 'strict-media-stripped']);
|
||||
});
|
||||
|
||||
it('strips rejected images on top of strict after an image-format rejection on the strict resend', async () => {
|
||||
const calls = { value: 0 };
|
||||
const policies: (ProjectionPolicy | undefined)[] = [];
|
||||
const { service } = createService(
|
||||
createRequester(calls, STRUCTURAL_400, [IMAGE_FORMAT_400]),
|
||||
createPolicyRecordingProjector({ policies }),
|
||||
);
|
||||
|
||||
await service.request();
|
||||
|
||||
expect(calls.value).toBe(3);
|
||||
expect(policies.map((policy) => policy?.structure)).toEqual([undefined, 'strict', 'strict']);
|
||||
expect(typeof policies[2]?.media).toBe('object');
|
||||
});
|
||||
|
||||
it('applies the strict repair on top of degraded media when a structural 400 follows a 413', async () => {
|
||||
const calls = { value: 0 };
|
||||
const policies: (ProjectionPolicy | undefined)[] = [];
|
||||
const { service } = createService(
|
||||
createRequester(calls, BODY_TOO_LARGE_413, [STRUCTURAL_400]),
|
||||
createPolicyRecordingProjector({ policies }),
|
||||
);
|
||||
|
||||
await service.request();
|
||||
|
||||
expect(calls.value).toBe(3);
|
||||
expect(policies).toEqual([
|
||||
undefined,
|
||||
{ media: 'degraded' },
|
||||
{ structure: 'strict', media: 'degraded' },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('AgentLLMRequesterService trace id', () => {
|
||||
const passthroughProjector = {
|
||||
project: (messages: readonly ContextMessage[]) => messages,
|
||||
projectStrict: (messages: readonly ContextMessage[]) => messages,
|
||||
};
|
||||
|
||||
function createTracedRequester(traceId: string | null): ModelRequester {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue