feat(codemode): sync v2 implementation (#35574)

This commit is contained in:
Aiden Cline 2026-07-06 13:19:21 -05:00 committed by GitHub
parent eb6ff0c1e0
commit d5aa79c73a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
32 changed files with 9544 additions and 6643 deletions

View file

@ -243,12 +243,13 @@ CodeMode executes a deliberately bounded JavaScript subset. It supports:
- Optional chaining, nullish coalescing, templates, spread (arrays, strings, Maps, Sets), and `try`/`catch`.
- Common array, string, number, `Object`, `Math`, and `JSON` operations. Mutating array methods include `push`/`pop`/`shift`/`unshift`/`splice` (removes in place and returns the removed elements)/`fill`/`copyWithin`; array `keys`/`values`/`entries` return **arrays** (matching the Map/Set convention) and work with `for...of` and spread. String methods include `localeCompare` (locale/options arguments ignored), `normalize`, and the `trimLeft`/`trimRight` aliases. `Object.keys` also accepts arrays (index strings, as in JS) and tool references: `Object.keys(tools)` lists the top-level namespaces, including `$codemode`, and `Object.keys(tools.ns)` lists the names at that node (a callable tool enumerates as `[]`; an unknown path is an `UnknownTool` diagnostic). `Object.values`/`Object.entries` on a tool reference fail with a pointer at `Object.keys(tools)` and `tools.$codemode.search`.
- `Date` - `Date.now()`/`Date.parse()`/`Date.UTC()`, `new Date(...)`, the getter methods, and date arithmetic/comparison via the time value. Dates stringify as ISO (`toString` included, for determinism across host timezones).
- Regular expressions - `/literals/` and `new RegExp(...)` with `test`/`exec` (stateful `lastIndex` for `g`), plus string `match`/`matchAll`/`replace`/`replaceAll`/`split`/`search` with patterns. Match results are arrays carrying `index` and named `groups` as own properties (`input` is omitted). Invalid patterns, invalid flags, and missing-`g` calls fail with catchable errors that say what was wrong and how to fix it (escaping hints, the exact `/pattern/g` to write). Patterns run on the host engine, so pathological backtracking is bounded only by the execution timeout. Function replacers are not supported.
- Regular expressions - `/literals/` and `new RegExp(...)` with `test`/`exec` (stateful `lastIndex` for `g`), plus string `match`/`matchAll`/`replace`/`replaceAll`/`split`/`search` with patterns. Match results are arrays carrying `index` and named `groups` as own properties (`input` is omitted). `replace` and `replaceAll` accept function replacers with captures, offset, input, and named groups; callbacks run sequentially, may await tool calls, and have their results coerced to strings. Invalid patterns, invalid flags, and missing-`g` calls fail with catchable errors that say what was wrong and how to fix it (escaping hints, the exact `/pattern/g` to write). Patterns run on the host engine, so pathological backtracking is bounded only by the execution timeout.
- `Map` and `Set` - construction from entries/arrays/strings, `get`/`set`/`add`/`has`/`delete`/`clear`/`size`/`forEach`, and `keys`/`values`/`entries` returning **arrays** (not iterators).
- URL helpers - `URL` resolution and mutation, linked `URLSearchParams`, `URL.canParse`/`URL.parse`, URI and URI-component encoding/decoding, and query parameter construction, lookup, mutation, sorting, callbacks, and materialization. URLSearchParams iteration methods return arrays, matching the Map/Set convention.
- First-class promises - an un-awaited `tools.ns.tool(...)` is a promise value whose call starts immediately on a supervised fiber; `await` resolves it (awaiting a non-promise value is a no-op, and `return tools.ns.tool(...)` resolves like an async-function return). `Promise.all`, `Promise.allSettled`, and `Promise.race` accept any array mixing promises and plain values (built inline, beforehand, or via spread); `Promise.resolve`/`Promise.reject` construct settled promises. `Promise.allSettled` rejection reasons are the same plain `{ name?, message }` data a `catch` binding sees, and `Promise.race` interrupts its losing in-flight calls. At most 8 tool calls run concurrently. When a program completes, still-running un-awaited calls are awaited before the execution ends; a failure from a call that was never awaited surfaces as an unhandled-rejection diagnostic.
- `throw value` and `throw new Error(message)` for explicit program failure. `Error` (and `TypeError`/`RangeError`/`SyntaxError`/`ReferenceError`/`EvalError`/`URIError`) are real constructors, callable with or without `new`; error values are plain `{ name, message }` data that additionally satisfy `instanceof Error` (a specific type matches itself and `Error`, as in JS). Every caught failure - thrown errors, interpreter runtime errors, and tool failures - is `instanceof Error` in a `catch` block; a thrown non-error value (`throw "text"`) is not, matching JS. Caught failures carry the `name` the equivalent real-JS failure would have - `JSON.parse` and invalid regex patterns produce a `SyntaxError` (satisfying `instanceof SyntaxError`), an unknown identifier a `ReferenceError`, assigning to a constant a `TypeError`, a bad `normalize` form a `RangeError`; failures with no specific analogue (including tool failures) are named `"Error"`. `instanceof` also recognizes `Date`, `RegExp`, `Map`, `Set`, `Array`, `Object`, and `Promise`; any other right-hand side is a catchable error.
- `throw value` and `throw new Error(message)` for explicit program failure. `Error` (and `TypeError`/`RangeError`/`SyntaxError`/`ReferenceError`/`EvalError`/`URIError`) are real constructors, callable with or without `new`; error values are plain `{ name, message }` data that additionally satisfy `instanceof Error` (a specific type matches itself and `Error`, as in JS). Every caught failure - thrown errors, interpreter runtime errors, and tool failures - is `instanceof Error` in a `catch` block; a thrown non-error value (`throw "text"`) is not, matching JS. Caught failures carry the `name` the equivalent real-JS failure would have - `JSON.parse` and invalid regex patterns produce a `SyntaxError` (satisfying `instanceof SyntaxError`), an unknown identifier a `ReferenceError`, assigning to a constant a `TypeError`, a bad `normalize` form a `RangeError`; failures with no specific analogue (including tool failures) are named `"Error"`. `instanceof` also recognizes `Date`, `RegExp`, `Map`, `Set`, `URL`, `URLSearchParams`, `Array`, `Object`, and `Promise`; any other right-hand side is a catchable error.
Inside a program, Date/RegExp/Map/Set values stay live everywhere: the internal data checkpoints (`Object.*` helpers, spread, coercion inputs) preserve the instances, so `Object.values({ d: date })[0].getTime()` and a spread copy of an object holding a Map keep working. Only at the host boundary (final result, tool arguments, `JSON.stringify`) do the four value types serialize exactly as `JSON.stringify` would: a Date becomes its ISO string (`null` when invalid) and RegExp/Map/Set become `{}`. Promise values never cross a data boundary: an un-awaited promise in a result or tool argument produces a diagnostic that says to await it, instead of serializing to `{}`.
Inside a program, standard-library values stay live everywhere: the internal data checkpoints (`Object.*` helpers, spread, coercion inputs) preserve the instances, so `Object.values({ d: date })[0].getTime()` and a spread copy of an object holding a Map keep working. Only at the host boundary (final result, tool arguments, `JSON.stringify`) do they serialize exactly as `JSON.stringify` would: Date and URL become strings (an invalid Date becomes `null`), while RegExp, Map, Set, and URLSearchParams become `{}`. Promise values never cross a data boundary: an un-awaited promise in a result or tool argument produces a diagnostic that says to await it, instead of serializing to `{}`.
It does not expose `eval`, dynamic imports, modules, classes, generators, timers, host globals, prototype mutation, custom promise constructors (`new Promise`), promise chaining (`.then`/`.catch`/`.finally` - `await` with `try`/`catch` is the supported style), or arbitrary method calls. Unsupported syntax returns an `UnsupportedSyntax` diagnostic with a source location when available.

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -1,4 +1,4 @@
export * as CodeMode from "./codemode.js"
export * as Tool from "./tool-api.js"
export * as Tool from "./tool.js"
export * as OpenAPI from "./openapi/index.js"
export { ToolError, toolError } from "./tool-error.js"

View file

@ -0,0 +1,200 @@
import type { SafeObject } from "../tool-runtime.js"
import type { SandboxURL } from "../values.js"
export type SourcePosition = {
line: number
column: number
}
export type SourceLocation = {
start: SourcePosition
end: SourcePosition
}
export type AstNode = {
type: string
loc?: SourceLocation
[key: string]: unknown
}
export type ProgramNode = AstNode & {
type: "Program"
body: Array<AstNode>
}
export type Binding = {
mutable: boolean
value: unknown
initialized?: boolean
}
export type StatementResult =
| { kind: "none" }
| { kind: "value"; value: unknown }
| { kind: "return"; value: unknown }
| { kind: "break" }
| { kind: "continue" }
export type MemberReference = {
target: SafeObject | Array<unknown> | SandboxURL
key: string | number
}
export class CodeModeFunction {
constructor(
readonly parameters: ReadonlyArray<AstNode>,
readonly body: AstNode,
readonly capturedScopes: ReadonlyArray<Map<string, Binding>>,
) {}
}
export class IntrinsicReference {
constructor(
readonly receiver: unknown,
readonly name: string,
) {}
}
export class ComputedValue {
constructor(readonly value: unknown) {}
}
export class PromiseNamespace {}
export type PromiseMethodName = "all" | "allSettled" | "race" | "resolve" | "reject"
export class PromiseMethodReference {
constructor(readonly name: PromiseMethodName) {}
}
export type GlobalNamespaceName =
| "Object"
| "Math"
| "JSON"
| "Array"
| "console"
| "Date"
| "RegExp"
| "Map"
| "Set"
| "URL"
| "URLSearchParams"
export class GlobalNamespace {
constructor(readonly name: GlobalNamespaceName) {}
}
export class GlobalMethodReference {
constructor(
readonly namespace: GlobalNamespaceName | "Number" | "String",
readonly name: string,
) {}
}
export class CoercionFunction {
constructor(readonly name: "Number" | "String" | "Boolean" | "parseInt" | "parseFloat") {}
}
export class UriFunction {
constructor(readonly name: "encodeURI" | "encodeURIComponent" | "decodeURI" | "decodeURIComponent") {}
}
export class ProgramThrow {
constructor(readonly value: unknown) {}
}
export class ErrorConstructorReference {
constructor(readonly name: string) {}
}
export type DiagnosticKind =
| "ParseError"
| "UnsupportedSyntax"
| "UnknownTool"
| "InvalidToolInput"
| "InvalidToolOutput"
| "InvalidDataValue"
| "ToolCallLimitExceeded"
| "TimeoutExceeded"
| "ToolFailure"
| "ExecutionFailure"
export const OptionalShortCircuit: unique symbol = Symbol("codemode.optional-short-circuit")
export const supportedSyntaxMessage =
"Supported orchestration syntax: tools.* calls (they return promises - resolve them with await), data literals, destructuring, optional chaining, template literals, conditionals, switch, loops (incl. for...of and for...in over object/array/tools keys), arrow functions, spread, try/catch, array methods (map/filter/find/findIndex/some/every/reduce/flatMap/forEach/sort/slice/concat/indexOf/lastIndexOf/at/flat/reverse/includes/join), string methods (incl. match/matchAll/replace/split with regular expressions), Date/RegExp/Map/Set/URL/URLSearchParams, URI encoding helpers, Object/Math/JSON helpers, captured console.log/warn/error/dir/table, and Promise.all/allSettled/race/resolve/reject over arrays mixing promises and plain values for parallel tool calls (promise chaining with .then/.catch is not supported - use await with try/catch)."
export class InterpreterRuntimeError extends Error {
readonly node?: AstNode
errorName: string = "Error"
constructor(
message: string,
node?: AstNode,
readonly kind: DiagnosticKind = "ExecutionFailure",
readonly suggestions?: ReadonlyArray<string>,
) {
super(message)
this.name = "InterpreterRuntimeError"
if (node) this.node = node
}
as(errorName: string): this {
this.errorName = errorName
return this
}
}
export const unsupportedSyntax = (kind: string, node: AstNode): InterpreterRuntimeError =>
new InterpreterRuntimeError(
`Syntax '${kind}' is not supported in CodeMode. ${supportedSyntaxMessage}`,
node,
"UnsupportedSyntax",
[supportedSyntaxMessage],
)
export const isRecord = (value: unknown): value is Record<string, unknown> => typeof value === "object" && value !== null
export const asNode = (value: unknown, context: string): AstNode => {
if (!isRecord(value) || typeof value.type !== "string") {
throw new InterpreterRuntimeError(`Invalid AST node while reading ${context}.`)
}
return value as AstNode
}
export const getArray = (node: AstNode, key: string): Array<unknown> => {
const value = node[key]
if (!Array.isArray(value)) throw new InterpreterRuntimeError(`Expected '${key}' to be an array.`, node)
return value
}
export const getString = (node: AstNode, key: string): string => {
const value = node[key]
if (typeof value !== "string") throw new InterpreterRuntimeError(`Expected '${key}' to be a string.`, node)
return value
}
export const getBoolean = (node: AstNode, key: string): boolean => {
const value = node[key]
if (typeof value !== "boolean") throw new InterpreterRuntimeError(`Expected '${key}' to be a boolean.`, node)
return value
}
export const getOptionalNode = (node: AstNode, key: string): AstNode | undefined => {
const value = node[key]
if (value === undefined || value === null) return undefined
return asNode(value, key)
}
export const getNode = (node: AstNode, key: string): AstNode => asNode(node[key], key)
export const sourceLocation = (node: AstNode): { readonly line: number; readonly column: number } => ({
line: Math.max(1, (node.loc?.start.line ?? 2) - 1),
column: Math.max(1, (node.loc?.start.column ?? 4) - 3),
})
export const formatLocation = (node?: AstNode): string => {
if (!node?.loc) return ""
const location = sourceLocation(node)
return ` (line ${location.line}, col ${location.column})`
}

File diff suppressed because it is too large Load diff

View file

@ -1,5 +1,5 @@
import { HttpClient } from "effect/unstable/http"
import { Tool, type Definition } from "../tool.js"
import { make, type Definition } from "../tool.js"
import { invoke } from "./runtime.js"
import {
componentDefinitions,
@ -102,7 +102,7 @@ export const fromSpec = (options: Options): Result => {
setTool(
tools,
segments,
Tool.make({
make({
description: operation.description ?? operation.summary ?? `${operation.method} ${path}`,
input: inputSchema(input.fields, definitions),
output: output.value,

View file

@ -46,7 +46,9 @@ export const invoke = (plan: Plan, input: unknown): Effect.Effect<unknown, unkno
)
}
if (json && Option.isNone(decoded)) {
return yield* Effect.fail(toolError(`${plan.operation.method} ${plan.operation.path} returned malformed JSON.`))
return yield* Effect.fail(
toolError(`${plan.operation.method} ${plan.operation.path} returned malformed JSON.`),
)
}
return parsed
})
@ -206,9 +208,8 @@ const buildUrl = (plan: Plan, input: Readonly<Record<string, unknown>>): string
return toolError(`Missing required path parameter '${field.inputName}'.`)
}
const fieldValue = serializeSimple(field, item, (value) =>
encodeURIComponent(value).replace(
/[!'()*]/g,
(character) => `%${character.charCodeAt(0).toString(16).toUpperCase()}`,
encodeURIComponent(value).replace(/[!'()*]/g, (character) =>
`%${character.charCodeAt(0).toString(16).toUpperCase()}`,
),
)
if (fieldValue instanceof ToolError) return fieldValue
@ -270,7 +271,10 @@ const serializeQuery = (
if (value.some((item) => item === undefined || (item !== null && typeof item === "object"))) {
return toolError(`Query parameter '${field.inputName}' contains an unsupported nested value.`)
}
return value.reduce((current, item) => HttpClientRequest.appendUrlParam(current, field.name, String(item)), request)
return value.reduce(
(current, item) => HttpClientRequest.appendUrlParam(current, field.name, String(item)),
request,
)
}
if (isRecord(value) && field.explode) {
return Object.entries(value).reduce<HttpClientRequest.HttpClientRequest | ToolError>((current, [name, item]) => {
@ -285,15 +289,11 @@ const serializeQuery = (
return rendered instanceof ToolError ? rendered : HttpClientRequest.appendUrlParam(request, field.name, rendered)
}
const readResponseBody = (
response: HttpClientResponse.HttpClientResponse,
plan: Plan,
): Effect.Effect<string, ToolError> =>
const readResponseBody = (response: HttpClientResponse.HttpClientResponse, plan: Plan): Effect.Effect<string, ToolError> =>
Effect.gen(function* () {
const contentLength = response.headers["content-length"]
const parsedSize = contentLength === undefined ? undefined : Number.parseInt(contentLength, 10)
const declaredSize =
parsedSize !== undefined && Number.isSafeInteger(parsedSize) && parsedSize >= 0 ? parsedSize : undefined
const declaredSize = parsedSize !== undefined && Number.isSafeInteger(parsedSize) && parsedSize >= 0 ? parsedSize : undefined
if (declaredSize !== undefined && declaredSize > maxResponseBodyBytes) {
return yield* Effect.fail(toolError(`${plan.operation.method} ${plan.operation.path} response exceeds 50 MiB.`))
}
@ -304,9 +304,7 @@ const readResponseBody = (
return Effect.fail(toolError(`${plan.operation.method} ${plan.operation.path} response exceeds 50 MiB.`))
}
if (size + chunk.byteLength > body.byteLength) {
const grown = Buffer.allocUnsafe(
Math.min(maxResponseBodyBytes, Math.max(size + chunk.byteLength, body.byteLength * 2)),
)
const grown = Buffer.allocUnsafe(Math.min(maxResponseBodyBytes, Math.max(size + chunk.byteLength, body.byteLength * 2)))
body.copy(grown, 0, 0, size)
body = grown
}

View file

@ -78,9 +78,7 @@ const isBinaryMediaType = (document: Document, mediaType: string, value: unknown
return isRecord(schema) && schema.format === "binary"
}
const jsonContent = (
content: Record<string, unknown>,
): { readonly mediaType: string; readonly schema: unknown } | undefined => {
const jsonContent = (content: Record<string, unknown>): { readonly mediaType: string; readonly schema: unknown } | undefined => {
const entry = Object.entries(content).find(([mediaType]) => isJsonMediaType(mediaType))
return entry !== undefined && isRecord(entry[1]) ? { mediaType: entry[0], schema: entry[1].schema } : undefined
}
@ -346,7 +344,7 @@ export const operationOutput = (
if (outcomes.length === 0) return { ok: true, value: undefined }
return {
ok: true,
value: withDefinitions(outcomes.length === 1 ? (outcomes[0] ?? {}) : { anyOf: outcomes }, definitions),
value: withDefinitions(outcomes.length === 1 ? outcomes[0] ?? {} : { anyOf: outcomes }, definitions),
}
}
@ -382,9 +380,7 @@ export const operationPath = (
namespaces: ReadonlySet<string>,
): ReadonlyArray<string> => {
const raw = nonEmptyString(operation.operationId)
const segments = (raw === undefined ? [fallbackOperationId(method, path)] : raw.split(".")).map(
sanitizeOperationSegment,
)
const segments = (raw === undefined ? [fallbackOperationId(method, path)] : raw.split(".")).map(sanitizeOperationSegment)
if (isOperationPathAvailable(segments, used, namespaces)) return segments
const conflict = segments.slice(0, -1).findIndex((_, index) => used.has(segments.slice(0, index + 1).join(".")))
if (conflict >= 0 && conflict + 1 < segments.length) {

View file

@ -0,0 +1,51 @@
export const arrayMethods = new Set([
"map",
"filter",
"find",
"findIndex",
"findLast",
"findLastIndex",
"some",
"every",
"includes",
"join",
"reduce",
"reduceRight",
"flatMap",
"forEach",
"sort",
"toSorted",
"slice",
"concat",
"indexOf",
"lastIndexOf",
"at",
"flat",
"reverse",
"toReversed",
"with",
"push",
"pop",
"shift",
"unshift",
"splice",
"fill",
"copyWithin",
"keys",
"values",
"entries",
])
export const mapMethods = new Set(["get", "set", "has", "delete", "clear", "forEach", "keys", "values", "entries"])
export const setMethods = new Set(["add", "has", "delete", "clear", "forEach", "keys", "values", "entries"])
export const spreadItems = (value: unknown): Array<unknown> | undefined => {
if (Array.isArray(value)) return value
if (typeof value === "string") return Array.from(value)
if (value instanceof SandboxMap) return Array.from(value.map.entries(), ([key, item]) => [key, item])
if (value instanceof SandboxSet) return Array.from(value.set.values())
if (value instanceof SandboxURLSearchParams) return Array.from(value.params.entries(), ([key, item]) => [key, item])
return undefined
}
import { SandboxMap, SandboxSet, SandboxURLSearchParams } from "../values.js"

View file

@ -0,0 +1,4 @@
export const consoleMethods = new Set(["log", "info", "debug", "warn", "error", "dir", "table"])
/** Console formatting recursion ceiling; deeper values render as "...". */
export const MAX_CONSOLE_DEPTH = 32

View file

@ -0,0 +1,94 @@
export const dateMethods = new Set([
"getTime",
"valueOf",
"toISOString",
"toJSON",
"toString",
"getFullYear",
"getMonth",
"getDate",
"getDay",
"getHours",
"getMinutes",
"getSeconds",
"getMilliseconds",
"getUTCFullYear",
"getUTCMonth",
"getUTCDate",
"getUTCDay",
"getUTCHours",
"getUTCMinutes",
"getUTCSeconds",
"getUTCMilliseconds",
"getTimezoneOffset",
])
export const dateStatics = new Set(["now", "parse", "UTC"])
export const invokeDateStatic = (name: string, args: Array<unknown>, node: AstNode): number => {
switch (name) {
case "now":
return Date.now()
case "parse":
return Date.parse(coerceToString(args[0]))
case "UTC":
return Date.UTC(...(args.map((arg) => coerceToNumber(arg)) as Parameters<typeof Date.UTC>))
default:
throw new InterpreterRuntimeError(`Date.${name} is not available in CodeMode.`, node)
}
}
export const invokeDateMethod = (value: SandboxDate, name: string, node: AstNode): unknown => {
const hosted = new Date(value.time)
switch (name) {
case "getTime":
case "valueOf":
return value.time
case "toISOString":
if (!Number.isFinite(value.time)) throw new InterpreterRuntimeError("Invalid time value.", node)
return hosted.toISOString()
case "toJSON":
return Number.isFinite(value.time) ? hosted.toISOString() : null
case "toString":
return coerceToString(value)
case "getFullYear":
return hosted.getFullYear()
case "getMonth":
return hosted.getMonth()
case "getDate":
return hosted.getDate()
case "getDay":
return hosted.getDay()
case "getHours":
return hosted.getHours()
case "getMinutes":
return hosted.getMinutes()
case "getSeconds":
return hosted.getSeconds()
case "getMilliseconds":
return hosted.getMilliseconds()
case "getUTCFullYear":
return hosted.getUTCFullYear()
case "getUTCMonth":
return hosted.getUTCMonth()
case "getUTCDate":
return hosted.getUTCDate()
case "getUTCDay":
return hosted.getUTCDay()
case "getUTCHours":
return hosted.getUTCHours()
case "getUTCMinutes":
return hosted.getUTCMinutes()
case "getUTCSeconds":
return hosted.getUTCSeconds()
case "getUTCMilliseconds":
return hosted.getUTCMilliseconds()
case "getTimezoneOffset":
return hosted.getTimezoneOffset()
default:
throw new InterpreterRuntimeError(`Date method '${name}' is not available in CodeMode.`, node)
}
}
import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js"
import { SandboxDate } from "../values.js"
import { coerceToNumber, coerceToString } from "./value.js"

View file

@ -0,0 +1,42 @@
import {
type AstNode,
CodeModeFunction,
InterpreterRuntimeError,
supportedSyntaxMessage,
} from "../interpreter/model.js"
import { copyIn, copyOut } from "../tool-runtime.js"
export const jsonStatics = new Set(["stringify", "parse"])
export const invokeJsonMethod = (name: string, args: Array<unknown>, node: AstNode): unknown => {
if (!jsonStatics.has(name)) throw new InterpreterRuntimeError(`JSON.${name} is not available in CodeMode.`, node)
switch (name) {
case "stringify": {
const replacer = args[1]
if (Array.isArray(replacer) || replacer instanceof CodeModeFunction) {
throw new InterpreterRuntimeError(
"JSON.stringify replacers are not supported in CodeMode.",
node,
"UnsupportedSyntax",
[supportedSyntaxMessage],
)
}
const space = args[2]
const indent = typeof space === "number" || typeof space === "string" ? space : undefined
return JSON.stringify(copyOut(copyIn(args[0], "JSON.stringify value")), null, indent)
}
case "parse": {
const text = args[0]
if (typeof text !== "string") throw new InterpreterRuntimeError("JSON.parse expects a string.", node)
try {
return copyIn(JSON.parse(text), "JSON.parse result")
} catch (error) {
throw new InterpreterRuntimeError(
`JSON.parse received invalid JSON: ${error instanceof Error ? error.message : String(error)}`,
node,
).as("SyntaxError")
}
}
}
throw new InterpreterRuntimeError(`JSON.${name} is not available in CodeMode.`, node)
}

View file

@ -0,0 +1,65 @@
export const mathConstants = new Set(["PI", "E", "LN2", "LN10", "LOG2E", "LOG10E", "SQRT2", "SQRT1_2"])
export const mathMethods = new Set([
"max",
"min",
"abs",
"floor",
"ceil",
"round",
"trunc",
"sign",
"sqrt",
"cbrt",
"pow",
"hypot",
"log",
"log2",
"log10",
"exp",
])
export const invokeMathMethod = (name: string, args: Array<unknown>, node: AstNode): number => {
if (!mathMethods.has(name)) throw new InterpreterRuntimeError(`Math.${name} is not available in CodeMode.`, node)
const nums = args.map((arg) => {
if (typeof arg !== "number") throw new InterpreterRuntimeError(`Math.${name} expects number arguments.`, node)
return arg
})
const [a = Number.NaN, b = Number.NaN] = nums
switch (name) {
case "max":
return Math.max(...nums)
case "min":
return Math.min(...nums)
case "abs":
return Math.abs(a)
case "floor":
return Math.floor(a)
case "ceil":
return Math.ceil(a)
case "round":
return Math.round(a)
case "trunc":
return Math.trunc(a)
case "sign":
return Math.sign(a)
case "sqrt":
return Math.sqrt(a)
case "cbrt":
return Math.cbrt(a)
case "pow":
return Math.pow(a, b)
case "hypot":
return Math.hypot(...nums)
case "log":
return Math.log(a)
case "log2":
return Math.log2(a)
case "log10":
return Math.log10(a)
case "exp":
return Math.exp(a)
}
throw new InterpreterRuntimeError(`Math.${name} is not available in CodeMode.`, node)
}
import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js"

View file

@ -0,0 +1,66 @@
export const numberMethods = new Set(["toFixed", "toPrecision", "toExponential", "toString"])
export const numberConstants = new Set(["MAX_SAFE_INTEGER", "MIN_SAFE_INTEGER", "MAX_VALUE", "MIN_VALUE", "EPSILON"])
export const numberStatics = new Set(["isInteger", "isFinite", "isNaN", "isSafeInteger", "parseInt", "parseFloat"])
export const invokeNumberMethod = (value: number, name: string, args: Array<unknown>, node: AstNode): unknown => {
const optNum = (index: number): number | undefined => {
const arg = args[index]
if (arg === undefined) return undefined
if (typeof arg !== "number") throw new InterpreterRuntimeError(`Number.${name} expects a number argument.`, node)
return arg
}
let result: unknown
switch (name) {
case "toFixed":
result = value.toFixed(optNum(0))
break
case "toExponential":
result = value.toExponential(optNum(0))
break
case "toPrecision": {
const digits = optNum(0)
result = digits === undefined ? value.toString() : value.toPrecision(digits)
break
}
case "toString": {
const radix = optNum(0)
if (radix !== undefined && (radix < 2 || radix > 36)) {
throw new InterpreterRuntimeError("Number.toString radix must be between 2 and 36.", node)
}
result = value.toString(radix)
break
}
default:
throw new InterpreterRuntimeError(`Number method '${name}' is not available in CodeMode.`, node)
}
return boundedData(result, `Number.${name} result`)
}
export const invokeNumberStatic = (name: string, args: Array<unknown>, node: AstNode): unknown => {
const value = args[0]
switch (name) {
case "isInteger":
return Number.isInteger(value)
case "isFinite":
return Number.isFinite(value)
case "isNaN":
return Number.isNaN(value)
case "isSafeInteger":
return Number.isSafeInteger(value)
case "parseInt": {
const radix = args[1]
if (radix !== undefined && typeof radix !== "number") {
throw new InterpreterRuntimeError("Number.parseInt expects a numeric radix.", node)
}
return parseInt(coerceToString(value), radix)
}
case "parseFloat":
return parseFloat(coerceToString(value))
default:
throw new InterpreterRuntimeError(`Number.${name} is not available in CodeMode.`, node)
}
}
import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js"
import { boundedData, coerceToString } from "./value.js"

View file

@ -0,0 +1,77 @@
import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js"
import { isBlockedMember } from "../tool-runtime.js"
import { isSandboxValue, SandboxMap, SandboxURLSearchParams } from "../values.js"
import { boundedData, coerceToString } from "./value.js"
export const objectStatics = new Set(["keys", "values", "entries", "hasOwn", "assign", "fromEntries"])
export const invokeObjectMethod = (name: string, args: Array<unknown>, node: AstNode): unknown => {
if (!objectStatics.has(name)) throw new InterpreterRuntimeError(`Object.${name} is not available in CodeMode.`, node)
const requireObject = (): Record<string, unknown> => {
const value = boundedData(args[0], `Object.${name} input`)
if (isSandboxValue(value)) return {}
if (value === null || typeof value !== "object" || Array.isArray(value)) {
throw new InterpreterRuntimeError(`Object.${name} expects a data object.`, node)
}
return value as Record<string, unknown>
}
const guardedSet = (out: Record<string, unknown>, key: string, item: unknown): void => {
if (isBlockedMember(key)) throw new InterpreterRuntimeError(`Property '${key}' is not available in CodeMode.`, node)
out[key] = item
}
switch (name) {
case "keys": {
const value = boundedData(args[0], "Object.keys input")
if (isSandboxValue(value)) return []
if (Array.isArray(value)) return Object.keys(value)
if (value === null || typeof value !== "object") {
throw new InterpreterRuntimeError("Object.keys expects a data object or array.", node)
}
return Object.keys(value)
}
case "values":
return Object.values(requireObject())
case "entries":
return Object.entries(requireObject()).map(([key, item]) => [key, item])
case "hasOwn":
return Object.hasOwn(requireObject(), String(args[1]))
case "assign": {
const out: Record<string, unknown> = Object.create(null)
for (const source of args) {
if (source === null || source === undefined) continue
const value = boundedData(source, "Object.assign input")
if (isSandboxValue(value)) continue
if (value === null || typeof value !== "object" || Array.isArray(value)) {
throw new InterpreterRuntimeError("Object.assign expects data objects.", node)
}
for (const [key, item] of Object.entries(value)) guardedSet(out, key, item)
}
return out
}
case "fromEntries": {
if (args[0] instanceof SandboxMap) {
const out: Record<string, unknown> = Object.create(null)
for (const [key, item] of args[0].map.entries()) guardedSet(out, coerceToString(key), item)
return out
}
if (args[0] instanceof SandboxURLSearchParams) {
const out: Record<string, unknown> = Object.create(null)
for (const [key, value] of args[0].params.entries()) guardedSet(out, key, value)
return out
}
const pairs = boundedData(args[0], "Object.fromEntries input")
if (!Array.isArray(pairs)) {
throw new InterpreterRuntimeError("Object.fromEntries expects an array of [key, value] pairs.", node)
}
const out: Record<string, unknown> = Object.create(null)
for (const pair of pairs) {
if (!Array.isArray(pair)) {
throw new InterpreterRuntimeError("Object.fromEntries expects [key, value] pairs.", node)
}
guardedSet(out, String(pair[0]), pair[1])
}
return out
}
}
throw new InterpreterRuntimeError(`Object.${name} is not available in CodeMode.`, node)
}

View file

@ -0,0 +1,6 @@
import type { PromiseMethodName } from "../interpreter/model.js"
export const promiseStatics = new Set<PromiseMethodName>(["all", "allSettled", "race", "resolve", "reject"])
/** Maximum number of eagerly forked tool calls that may run concurrently. */
export const TOOL_CALL_CONCURRENCY = 8

View file

@ -0,0 +1,74 @@
export const regexpMethods = new Set(["test", "exec", "toString"])
export const regexpProperties = new Set([
"source",
"flags",
"lastIndex",
"global",
"ignoreCase",
"multiline",
"sticky",
"unicode",
"dotAll",
])
export const regexFailureReason = (error: unknown): string =>
(error instanceof Error ? error.message : String(error)).replace(/^Invalid regular expression:\s*/i, "")
export const escapeRegexHint =
'To match special characters like ( ) [ ] { } + * ? . literally, escape them with a backslash (e.g. "\\\\(") or test for them with String.includes instead.'
export const toHostRegex = (arg: unknown, method: string, node: AstNode, extraFlags = ""): RegExp => {
if (arg instanceof SandboxRegExp) return arg.regex
if (typeof arg === "string") {
try {
return new RegExp(arg, extraFlags)
} catch (error) {
throw new InterpreterRuntimeError(
`String.${method} received the string ${JSON.stringify(arg)}, which is not a valid regular expression pattern (${regexFailureReason(error)}). ${escapeRegexHint}`,
node,
).as("SyntaxError")
}
}
throw new InterpreterRuntimeError(
`String.${method} expects a regular expression (a /pattern/flags literal or new RegExp(...)) or a string pattern, not ${arg === null ? "null" : typeof arg}.`,
node,
)
}
export const matchToValue = (match: RegExpMatchArray): Array<unknown> => {
const result: Array<unknown> = Array.from(match, (group) => group)
if (match.index !== undefined) (result as Record<string, unknown> & Array<unknown>).index = match.index
if (match.groups) {
const groups: SafeObject = Object.create(null) as SafeObject
for (const [key, group] of Object.entries(match.groups)) {
if (!isBlockedMember(key)) groups[key] = group
}
;(result as Record<string, unknown> & Array<unknown>).groups = groups
}
return result
}
export const invokeRegExpMethod = (
value: SandboxRegExp,
name: string,
args: Array<unknown>,
node: AstNode,
): unknown => {
switch (name) {
case "test":
return value.regex.test(coerceToString(args[0]))
case "exec": {
const matched = value.regex.exec(coerceToString(args[0]))
return matched === null ? null : matchToValue(matched)
}
case "toString":
return coerceToString(value)
default:
throw new InterpreterRuntimeError(`RegExp method '${name}' is not available in CodeMode.`, node)
}
}
import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js"
import { isBlockedMember, type SafeObject } from "../tool-runtime.js"
import { SandboxRegExp } from "../values.js"
import { coerceToString } from "./value.js"

View file

@ -0,0 +1,52 @@
export const stringMethods = new Set([
"toLowerCase",
"toUpperCase",
"trim",
"trimStart",
"trimEnd",
"trimLeft",
"trimRight",
"split",
"slice",
"substring",
"substr",
"includes",
"startsWith",
"endsWith",
"indexOf",
"lastIndexOf",
"replace",
"replaceAll",
"repeat",
"padStart",
"padEnd",
"charAt",
"charCodeAt",
"codePointAt",
"at",
"concat",
"toString",
"match",
"matchAll",
"search",
"localeCompare",
"normalize",
])
export const stringStatics = new Set(["fromCharCode", "fromCodePoint"])
export const invokeStringStatic = (name: string, args: Array<unknown>, node: AstNode): unknown => {
const codes = args.map((arg) => {
if (typeof arg !== "number") throw new InterpreterRuntimeError(`String.${name} expects number arguments.`, node)
return arg
})
switch (name) {
case "fromCharCode":
return String.fromCharCode(...codes)
case "fromCodePoint":
return String.fromCodePoint(...codes)
default:
throw new InterpreterRuntimeError(`String.${name} is not available in CodeMode.`, node)
}
}
import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js"

View file

@ -0,0 +1,90 @@
export const urlProperties = new Set([
"href",
"origin",
"protocol",
"username",
"password",
"host",
"hostname",
"port",
"pathname",
"search",
"hash",
])
export const urlWritableProperties = new Set([
"href",
"protocol",
"username",
"password",
"host",
"hostname",
"port",
"pathname",
"search",
"hash",
])
export const urlMethods = new Set(["toString", "toJSON"])
export const urlStatics = new Set(["canParse", "parse"])
export const urlSearchParamsMethods = new Set([
"append",
"delete",
"get",
"getAll",
"has",
"set",
"sort",
"forEach",
"keys",
"values",
"entries",
"toString",
])
export const uriArgument = (value: unknown, label: string): string => coerceToString(boundedData(value, label))
export const invokeUriFunction = (ref: UriFunction, args: Array<unknown>, node: AstNode): string => {
const value = uriArgument(args[0], `${ref.name} input`)
try {
switch (ref.name) {
case "encodeURI":
return encodeURI(value)
case "encodeURIComponent":
return encodeURIComponent(value)
case "decodeURI":
return decodeURI(value)
case "decodeURIComponent":
return decodeURIComponent(value)
}
} catch (error) {
throw new InterpreterRuntimeError(
`${ref.name} received malformed URI data: ${error instanceof Error ? error.message : String(error)}`,
node,
).as("URIError")
}
}
export const urlArgument = (value: unknown, label: string): string =>
value instanceof SandboxURL ? value.url.href : uriArgument(value, label)
export const invokeURLStatic = (name: string, args: Array<unknown>, node: AstNode): unknown => {
if (!urlStatics.has(name)) throw new InterpreterRuntimeError(`URL.${name} is not available in CodeMode.`, node)
if (args.length === 0) throw new InterpreterRuntimeError(`URL.${name} requires a URL argument.`, node).as("TypeError")
const input = urlArgument(args[0], `URL.${name} input`)
const base = args[1] === undefined ? undefined : urlArgument(args[1], `URL.${name} base`)
try {
const url = new URL(input, base)
return name === "canParse" ? true : new SandboxURL(url)
} catch {
return name === "canParse" ? false : null
}
}
export const invokeURLMethod = (value: SandboxURL, name: string, node: AstNode): string => {
if (name === "toString" || name === "toJSON") return value.url.href
throw new InterpreterRuntimeError(`URL method '${name}' is not available in CodeMode.`, node)
}
import { type AstNode, InterpreterRuntimeError, UriFunction } from "../interpreter/model.js"
import { SandboxURL } from "../values.js"
import { boundedData, coerceToString } from "./value.js"

View file

@ -0,0 +1,90 @@
export const errorConstructors = new Set([
"Error",
"TypeError",
"RangeError",
"SyntaxError",
"ReferenceError",
"EvalError",
"URIError",
])
export const valueConstructors = new Set(["Date", "RegExp", "Map", "Set", "URL", "URLSearchParams"])
export const compoundOperators = new Set(["+=", "-=", "*=", "/=", "%=", "**=", "&=", "|=", "^=", "<<=", ">>=", ">>>="])
const ErrorBrand: unique symbol = Symbol("codemode.error")
export const createErrorValue = (name: string, message: string): SafeObject => {
const value = Object.assign(Object.create(null) as SafeObject, { name, message })
Object.defineProperty(value, ErrorBrand, { value: name })
return value
}
export const errorBrandName = (value: unknown): string | undefined =>
value !== null && typeof value === "object"
? ((value as Record<PropertyKey, unknown>)[ErrorBrand] as string | undefined)
: undefined
export const boundedData = (value: unknown, label: string): unknown => copyIn(value, label, true)
export const coerceToString = (value: unknown): string => {
if (value === null) return "null"
if (value === undefined) return "undefined"
if (value instanceof SandboxDate)
return Number.isFinite(value.time) ? new Date(value.time).toISOString() : "Invalid Date"
if (value instanceof SandboxRegExp) return `/${value.regex.source}/${value.regex.flags}`
if (value instanceof SandboxMap) return "[object Map]"
if (value instanceof SandboxSet) return "[object Set]"
if (value instanceof SandboxURL) return value.url.href
if (value instanceof SandboxURLSearchParams) return value.params.toString()
if (typeof value === "object") {
return Array.isArray(value)
? value.map((item) => (item === null || item === undefined ? "" : coerceToString(item))).join(",")
: "[object Object]"
}
return String(value)
}
export const coerceToNumber = (value: unknown): number => {
if (value instanceof SandboxDate) return value.time
if (isSandboxValue(value)) return Number.NaN
return value !== null && typeof value === "object" && !Array.isArray(value) ? Number.NaN : Number(value)
}
export const invokeCoercion = (ref: CoercionFunction, args: Array<unknown>, node: AstNode): unknown => {
const raw = args[0]
if (isSandboxValue(raw)) {
if (ref.name === "Boolean") return true
if (ref.name === "Number") return coerceToNumber(raw)
if (ref.name === "String") return coerceToString(raw)
if (ref.name === "parseInt") return parseInt(coerceToString(raw))
return parseFloat(coerceToString(raw))
}
const value = boundedData(args[0], `${ref.name} input`)
if (ref.name === "Number") return coerceToNumber(value)
if (ref.name === "Boolean") return Boolean(value)
if (ref.name === "parseInt") {
const radix = args[1]
if (radix !== undefined && typeof radix !== "number") {
throw new InterpreterRuntimeError("parseInt expects a numeric radix.", node)
}
return parseInt(coerceToString(value), radix)
}
if (ref.name === "parseFloat") return parseFloat(coerceToString(value))
return coerceToString(value)
}
import {
type AstNode,
CoercionFunction,
InterpreterRuntimeError,
} from "../interpreter/model.js"
import { copyIn, type SafeObject } from "../tool-runtime.js"
import {
isSandboxValue,
SandboxDate,
SandboxMap,
SandboxRegExp,
SandboxSet,
SandboxURL,
SandboxURLSearchParams,
} from "../values.js"

View file

@ -1,2 +0,0 @@
export { isDefinition, make } from "./tool.js"
export type { Definition, JsonSchema, Options, ToolSchema as SchemaType } from "./tool.js"

View file

@ -6,12 +6,18 @@ import {
identifierSegment,
inputProperties,
inputTypeScript,
isDefinition as isToolDefinition,
outputTypeScript,
Tool,
type Definition,
} from "./tool.js"
import { SandboxDate, SandboxMap, SandboxPromise, SandboxRegExp, SandboxSet } from "./values.js"
} from "./tool-schema.js"
import { isDefinition as isToolDefinition, type Definition } from "./tool.js"
import {
SandboxDate,
SandboxMap,
SandboxPromise,
SandboxRegExp,
SandboxSet,
SandboxURL,
SandboxURLSearchParams,
} from "./values.js"
const estimateTokens = (input: string) => Math.max(0, Math.round(input.length / 4))
@ -154,9 +160,9 @@ export const isBlockedMember = (name: string): boolean => blockedMemberNames.has
* Two modes share the walk:
* - **Boundary** (`preserveSandboxValues` false, the default): the host<->sandbox boundary -
* final results, tool-call arguments, `JSON.stringify`. Sandbox value types serialize
* exactly as JSON.stringify would: Date -> ISO string (invalid -> null), RegExp/Map/Set -> {}.
* exactly as JSON.stringify would: Date/URL -> strings, the remaining value types -> {}.
* - **Intra-sandbox checkpoint** (`preserveSandboxValues` true; see `boundedData` in
* codemode.ts): Date/RegExp/Map/Set instances pass through untouched (treated as leaves,
* codemode.ts): standard-library value instances pass through untouched (treated as leaves,
* contents not walked), so values flowing through `Object.*` helpers, coercion inputs, and
* other in-sandbox checkpoints stay fully usable (`.getTime()`, `.has()`, ...).
*
@ -210,7 +216,9 @@ const copyBounded = (
value instanceof SandboxDate ||
value instanceof SandboxRegExp ||
value instanceof SandboxMap ||
value instanceof SandboxSet
value instanceof SandboxSet ||
value instanceof SandboxURL ||
value instanceof SandboxURLSearchParams
) {
return value
}
@ -230,24 +238,30 @@ const copyBounded = (
for (const item of value.values()) wrapped.set.add(copyBounded(item, label, depth + 1, seen, true))
return wrapped
}
if (value instanceof URL) return new SandboxURL(new URL(value.href))
if (value instanceof URLSearchParams) return new SandboxURLSearchParams(new URLSearchParams(value))
}
// Sandbox value types (and their host counterparts, which a host tool may legitimately
// return) serialize exactly as JSON.stringify would at the data boundary: a Date is its
// toJSON() ISO string (invalid -> null), and RegExp/Map/Set have no JSON form beyond {}.
// return) serialize exactly as JSON.stringify would at the data boundary: Date/URL use
// toJSON(), while RegExp/Map/Set/URLSearchParams have no JSON form beyond {}.
if (value instanceof SandboxDate) {
return Number.isFinite(value.time) ? new Date(value.time).toISOString() : null
}
if (value instanceof Date) {
return Number.isFinite(value.getTime()) ? value.toISOString() : null
}
if (value instanceof SandboxURL) return value.url.href
if (value instanceof URL) return value.href
if (
value instanceof SandboxRegExp ||
value instanceof SandboxMap ||
value instanceof SandboxSet ||
value instanceof SandboxURLSearchParams ||
value instanceof RegExp ||
value instanceof Map ||
value instanceof Set
value instanceof Set ||
value instanceof URLSearchParams
) {
return Object.create(null) as SafeObject
}
@ -369,69 +383,70 @@ const termForms = (term: string): Array<string> => {
return forms
}
const makeSearchTool = (searchIndex: ReadonlyArray<SearchEntry>) =>
Tool.make({
description: "Search available Code Mode tools",
input: SearchInput,
output: SearchOutput,
run: (request) =>
Effect.sync(() => {
const query = request.query ?? ""
const offset = request.offset ?? 0
const scoped =
request.namespace === undefined
? searchIndex
: searchIndex.filter((entry) => entry.namespace === request.namespace)
// A query that names one tool path exactly (canonical path or rendered JavaScript
// expression) is a lookup, not a search: return that tool alone.
const trimmed = query.trim()
const pathQuery = trimmed.startsWith("tools.") ? trimmed.slice("tools.".length) : trimmed
const exact =
pathQuery === ""
? undefined
: scoped.find(
(entry) => entry.description.path === pathQuery || toolExpression(entry.description.path) === trimmed,
)
const terms = tokenize(query).map(termForms)
// Additive field-weighted scoring, summed across terms: exact path or path segment
// (20) > path substring (8) > description substring (4) > any searchable text,
// including input parameter names and descriptions (2).
const ranked =
exact !== undefined
? [exact]
: scoped
.map((entry) => {
const path = entry.description.path.toLowerCase()
const description = entry.description.description.toLowerCase()
const score = terms.reduce(
(total, forms) =>
total +
(forms.some((form) => path === form || path.endsWith(`.${form}`)) ? 20 : 0) +
(forms.some((form) => path.includes(form)) ? 8 : 0) +
(forms.some((form) => description.includes(form)) ? 4 : 0) +
(forms.some((form) => entry.searchText.includes(form)) ? 2 : 0),
0,
)
return { entry, score }
})
.filter(({ score }) => terms.length === 0 || score > 0)
.sort(
(left, right) =>
right.score - left.score || left.entry.description.path.localeCompare(right.entry.description.path),
const makeSearchTool = (searchIndex: ReadonlyArray<SearchEntry>): Definition => ({
_tag: "CodeModeTool",
description: "Search available Code Mode tools",
input: SearchInput,
output: SearchOutput,
run: (input) =>
Effect.sync(() => {
const request = input as typeof SearchInput.Type
const query = request.query ?? ""
const offset = request.offset ?? 0
const scoped =
request.namespace === undefined
? searchIndex
: searchIndex.filter((entry) => entry.namespace === request.namespace)
// A query that names one tool path exactly (canonical path or rendered JavaScript
// expression) is a lookup, not a search: return that tool alone.
const trimmed = query.trim()
const pathQuery = trimmed.startsWith("tools.") ? trimmed.slice("tools.".length) : trimmed
const exact =
pathQuery === ""
? undefined
: scoped.find(
(entry) => entry.description.path === pathQuery || toolExpression(entry.description.path) === trimmed,
)
const terms = tokenize(query).map(termForms)
// Additive field-weighted scoring, summed across terms: exact path or path segment
// (20) > path substring (8) > description substring (4) > any searchable text,
// including input parameter names and descriptions (2).
const ranked =
exact !== undefined
? [exact]
: scoped
.map((entry) => {
const path = entry.description.path.toLowerCase()
const description = entry.description.description.toLowerCase()
const score = terms.reduce(
(total, forms) =>
total +
(forms.some((form) => path === form || path.endsWith(`.${form}`)) ? 20 : 0) +
(forms.some((form) => path.includes(form)) ? 8 : 0) +
(forms.some((form) => description.includes(form)) ? 4 : 0) +
(forms.some((form) => entry.searchText.includes(form)) ? 2 : 0),
0,
)
.map(({ entry }) => entry)
const items = ranked.slice(offset, offset + (request.limit ?? defaultSearchLimit)).map(({ description }) => ({
...description,
path: toolExpression(description.path),
}))
const remaining = Math.max(0, ranked.length - offset - items.length)
return {
items,
remaining,
next: remaining > 0 ? { offset: offset + items.length } : null,
}
}),
})
return { entry, score }
})
.filter(({ score }) => terms.length === 0 || score > 0)
.sort(
(left, right) =>
right.score - left.score || left.entry.description.path.localeCompare(right.entry.description.path),
)
.map(({ entry }) => entry)
const items = ranked.slice(offset, offset + (request.limit ?? defaultSearchLimit)).map(({ description }) => ({
...description,
path: toolExpression(description.path),
}))
const remaining = Math.max(0, ranked.length - offset - items.length)
return {
items,
remaining,
next: remaining > 0 ? { offset: offset + items.length } : null,
}
}),
})
const searchDescription = describeDefinition(`${reservedNamespace}.search`, makeSearchTool([]))
@ -590,9 +605,9 @@ export const prepare = <R>(tools: HostTools<R>, catalogBudget = defaultCatalogBu
"",
"## Language",
"",
"Use common JavaScript data operations, functions, control flow, selected standard-library methods, and awaited tool calls.",
"Modules/imports, classes, generators, timers, fetch, eval, prototype access, arbitrary methods, and promise chaining are unavailable. Use Code Mode tools for external operations. Use await with try/catch.",
"Dates serialize to ISO strings at data boundaries; Map/Set/RegExp serialize to `{}`.",
"Use common JavaScript data operations, functions, control flow, selected standard-library methods, and awaited tool calls. Built-ins include Date, RegExp, Map, Set, URL, URLSearchParams, and URI encoding helpers.",
"Modules/imports, classes, generators, timers, fetch, eval, prototype access, unlisted methods, and promise chaining are unavailable. Use Code Mode tools for external operations. Use await with try/catch.",
"Dates and URLs serialize to strings at data boundaries; Map/Set/RegExp/URLSearchParams serialize to `{}`.",
]
const toolSection: Array<string> = [""]

View file

@ -0,0 +1,301 @@
import { JsonPointer, Schema } from "effect"
import type { Definition, JsonSchema, SchemaType } from "./tool.js"
const isEffectSchema = (schema: SchemaType): schema is Schema.Decoder<unknown> & Schema.Top => Schema.isSchema(schema)
const renderLiteral = (value: unknown): string => JSON.stringify(value) ?? "unknown"
/**
* Bare TypeScript identifier - usable unquoted as an object key (and, in the tool runtime,
* with dot access as a tool-path segment). Anything else must be quoted/bracketed.
*/
export const identifierSegment = /^[A-Za-z_$][A-Za-z0-9_$]*$/
/** Renders a property name as a valid TS object key: bare when an identifier, quoted otherwise. */
const renderKey = (name: string): string => (identifierSegment.test(name) ? name : JSON.stringify(name))
const effectNumberSentinel = (schema: JsonSchema) =>
schema.type === "string" &&
Array.isArray(schema.enum) &&
schema.enum.length === 1 &&
(schema.enum[0] === "NaN" || schema.enum[0] === "Infinity" || schema.enum[0] === "-Infinity")
const intersection = (members: ReadonlyArray<string>): string => {
const concrete = members.filter((member) => member !== "unknown")
if (concrete.length === 0) return "unknown"
if (concrete.length === 1) return concrete[0] ?? "unknown"
return concrete.map((member) => (member.includes(" | ") ? `(${member})` : member)).join(" & ")
}
/**
* Recursion ceiling for schema rendering. Object, array, and union recursion all increment
* depth, so this bounds every recursion path - pathological or structurally cyclic schemas
* degrade to `unknown` instead of overflowing the stack (rendering must never throw).
*/
const MAX_RENDER_DEPTH = 8
type RenderContext = {
readonly definitions: Readonly<Record<string, JsonSchema>>
/** Indented, JSDoc-annotated multiline rendering (search results); compact single line otherwise. */
readonly pretty: boolean
}
const hasUnresolvedRef = (
schema: JsonSchema,
definitions: Readonly<Record<string, JsonSchema>>,
seen: ReadonlySet<string> = new Set(),
visited: ReadonlySet<JsonSchema> = new Set(),
): boolean => {
if (visited.has(schema)) return false
const nextVisited = new Set([...visited, schema])
if (schema.$ref !== undefined) {
const segment = schema.$ref.match(/^#\/(?:\$defs|definitions)\/([^/]+)$/)?.[1]
const name = segment === undefined ? undefined : JsonPointer.unescapeToken(segment)
if (name === undefined || definitions[name] === undefined || seen.has(name)) return true
if (hasUnresolvedRef(definitions[name], definitions, new Set([...seen, name]), nextVisited)) return true
}
return [
...(schema.anyOf ?? []),
...(schema.oneOf ?? []),
...(schema.allOf ?? []),
...Object.values(schema.properties ?? {}),
...(schema.items === undefined ? [] : [schema.items]),
...(typeof schema.additionalProperties === "object" ? [schema.additionalProperties] : []),
].some((item) => hasUnresolvedRef(item, definitions, seen, nextVisited))
}
/**
* Schema constraints a TypeScript type cannot express natively but a model benefits from,
* surfaced as JSDoc tags (`@deprecated`, `@default`, `@format`, `@minItems`, `@maxItems`).
*/
const docTags = (schema: JsonSchema): Array<string> => {
const tags: Array<string> = []
if (schema.deprecated === true) tags.push("@deprecated")
if (schema.default !== undefined) {
try {
const rendered = JSON.stringify(schema.default)
if (rendered !== undefined) tags.push(`@default ${rendered}`)
} catch {
// unserializable default: skip rather than emit a broken tag
}
}
if (typeof schema.format === "string") tags.push(`@format ${schema.format}`)
if (typeof schema.minItems === "number") tags.push(`@minItems ${schema.minItems}`)
if (typeof schema.maxItems === "number") tags.push(`@maxItems ${schema.maxItems}`)
return tags
}
/**
* Format a schema `description` plus `tags` as a JSDoc comment at the given indent,
* preserving multi-line text (a single line stays `/** ... *\/`; multiple lines become a
* `*`-prefixed block). `*\/` is neutralized so nothing can close the comment early, and
* blank leading/trailing lines are trimmed. Returns "" (else a trailing newline) so
* callers can prepend it directly to the field line.
*/
const jsdoc = (description: string | undefined, tags: ReadonlyArray<string>, pad: string): string => {
const lines = [...(description === undefined ? [] : description.split("\n")), ...tags].map((line) =>
line.replaceAll("*/", "* /").replace(/\s+$/, ""),
)
while (lines.length > 0 && lines[0]!.trim() === "") lines.shift()
while (lines.length > 0 && lines[lines.length - 1]!.trim() === "") lines.pop()
if (lines.length === 0) return ""
if (lines.length === 1) return `${pad}/** ${lines[0]} */\n`
const body = lines.map((line) => `${pad} *${line === "" ? "" : ` ${line}`}`).join("\n")
return `${pad}/**\n${body}\n${pad} */\n`
}
const renderSchema = (
schema: JsonSchema,
ctx: RenderContext,
depth = 0,
seen: ReadonlySet<string> = new Set(),
): string => {
if (depth > MAX_RENDER_DEPTH) return "unknown"
const nested =
schema.definitions === undefined && schema.$defs === undefined
? ctx
: { ...ctx, definitions: { ...ctx.definitions, ...(schema.definitions ?? {}), ...(schema.$defs ?? {}) } }
if (schema.$ref) {
const segment = schema.$ref.match(/^#\/(?:\$defs|definitions)\/([^/]+)$/)?.[1]
const name = segment === undefined ? undefined : JsonPointer.unescapeToken(segment)
if (!name || !nested.definitions[name] || seen.has(name)) return "unknown"
return intersection([
renderSchema(nested.definitions[name], nested, depth, new Set([...seen, name])),
renderSchema({ ...schema, $ref: undefined }, nested, depth + 1, seen),
])
}
if (schema.const !== undefined) return renderLiteral(schema.const)
if (schema.enum) return schema.enum.map(renderLiteral).join(" | ")
const alternatives = schema.anyOf ?? schema.oneOf
if (alternatives) {
// Effect's number schema emits `anyOf: [{ type: "number" }, { const: "NaN" },
// { const: "Infinity" }, { const: "-Infinity" }]`. Collapse only that artifact;
// real JSON Schema unions such as `string | number` or `number | null` must keep
// every branch.
if (
alternatives.some((item) => item.type === "number") &&
alternatives.every((item) => item.type === "number" || effectNumberSentinel(item))
)
return "number"
// An empty Schema.Struct({}) emits `anyOf: [{ type: "object" }, { type: "array" }]`
// (no properties/items); render the bare shape as {} instead of `{} | Array<unknown>`.
if (
alternatives.length === 2 &&
alternatives[0]?.type === "object" &&
alternatives[0].properties === undefined &&
alternatives[1]?.type === "array" &&
alternatives[1].items === undefined
) {
return "{}"
}
const members = alternatives.map((item) => renderSchema(item, nested, depth + 1, seen))
if (members.some((member) => member === "unknown")) return "unknown"
return intersection([
members.join(" | "),
renderSchema({ ...schema, anyOf: undefined, oneOf: undefined }, nested, depth + 1, seen),
])
}
if (schema.allOf) {
const members = schema.allOf.map((item) => renderSchema(item, nested, depth + 1, seen))
if (schema.allOf.some((item) => hasUnresolvedRef(item, nested.definitions))) return "unknown"
return intersection([renderSchema({ ...schema, allOf: undefined }, nested, depth + 1, seen), ...members])
}
if (Array.isArray(schema.type)) {
return schema.type.map((item) => renderSchema({ ...schema, type: item }, nested, depth + 1, seen)).join(" | ")
}
if (schema.type === "string") return "string"
if (schema.type === "number" || schema.type === "integer") return "number"
if (schema.type === "boolean") return "boolean"
if (schema.type === "null") return "null"
if (schema.type === "array") return `Array<${renderSchema(schema.items ?? {}, nested, depth + 1, seen)}>`
if (schema.type === "object" || schema.properties) {
const required = new Set(schema.required ?? [])
const properties = Object.entries(schema.properties ?? {})
const additional = schema.additionalProperties
const indexType =
additional && typeof additional === "object" ? renderSchema(additional, nested, depth + 1, seen) : undefined
const field = ([name, value]: readonly [string, JsonSchema]) =>
`${renderKey(name)}${required.has(name) ? "" : "?"}: ${renderSchema(value, nested, depth + 1, seen)}`
if (!ctx.pretty) {
const fields = properties.map(field)
if (indexType !== undefined) fields.push(`[key: string]: ${indexType}`)
return fields.length === 0 ? "{}" : `{ ${fields.join("; ")} }`
}
// Pretty: an indented block, each described field preceded by its JSDoc comment.
if (properties.length === 0 && indexType === undefined) return "{}"
const pad = " ".repeat(depth + 1)
const lines = properties.map(
(entry) => `${jsdoc(entry[1].description, docTags(entry[1]), pad)}${pad}${field(entry)},`,
)
if (indexType !== undefined) lines.push(`${pad}[key: string]: ${indexType},`)
return `{\n${lines.join("\n")}\n${" ".repeat(depth)}}`
}
return "unknown"
}
export const toTypeScript = (schema: Schema.Top, decoded = false, pretty = false): string => {
try {
const visible = decoded ? Schema.toType(schema) : schema
const document = Schema.toJsonSchemaDocument(visible) as {
readonly schema: JsonSchema
readonly definitions?: Readonly<Record<string, JsonSchema>>
}
return renderSchema(document.schema, { definitions: document.definitions ?? {}, pretty })
} catch {
return "unknown"
}
}
/** Renders a raw JSON Schema document as a TypeScript type string. */
export const jsonSchemaToTypeScript = (schema: JsonSchema, pretty = false): string => {
try {
return renderSchema(schema, { definitions: { ...(schema.definitions ?? {}), ...(schema.$defs ?? {}) }, pretty })
} catch {
return "unknown"
}
}
/** One input property of a tool, extracted best-effort from its input schema. */
export type InputProperty = {
readonly name: string
readonly description: string | undefined
readonly required: boolean
}
/**
* The property names, descriptions, and required flags of a tool's input schema - the raw
* material for search text. Best-effort: Effect Schemas go through their
* JSON Schema document (the same emission signature rendering uses); JSON Schemas are read
* directly, resolving a trivial top-level `$ref` into `$defs`/`definitions` when present.
* Anything unresolvable yields `[]` (search falls back to path + description).
*/
export const inputProperties = <R>(definition: Definition<R>): Array<InputProperty> => {
try {
const document = isEffectSchema(definition.input)
? (Schema.toJsonSchemaDocument(definition.input) as {
readonly schema: JsonSchema
readonly definitions?: Readonly<Record<string, JsonSchema>>
})
: {
schema: definition.input,
definitions: { ...(definition.input.definitions ?? {}), ...(definition.input.$defs ?? {}) },
}
const definitions = document.definitions ?? {}
let schema = document.schema
if (schema.$ref !== undefined) {
const segment = schema.$ref.match(/^#\/(?:\$defs|definitions)\/([^/]+)$/)?.[1]
const name = segment === undefined ? undefined : JsonPointer.unescapeToken(segment)
const resolved = name === undefined ? undefined : definitions[name]
if (resolved === undefined) return []
schema = resolved
}
const required = new Set(schema.required ?? [])
return Object.entries(schema.properties ?? {}).map(([name, value]) => ({
name,
description: typeof value.description === "string" ? value.description : undefined,
required: required.has(name),
}))
} catch {
return []
}
}
/**
* The model-visible TypeScript type of a tool's input. `pretty` renders an indented
* multiline block with schema descriptions and constraints as JSDoc comments on the
* fields; the default stays the compact single-line form.
*/
export const inputTypeScript = <R>(definition: Definition<R>, pretty = false): string =>
isEffectSchema(definition.input)
? toTypeScript(definition.input, false, pretty)
: jsonSchemaToTypeScript(definition.input, pretty)
/**
* The model-visible TypeScript type of a tool's result; tools without an output schema
* return `unknown`. `pretty` renders the JSDoc-annotated multiline form, as for inputs.
*/
export const outputTypeScript = <R>(definition: Definition<R>, pretty = false): string =>
definition.output === undefined
? "unknown"
: isEffectSchema(definition.output)
? toTypeScript(definition.output, true, pretty)
: jsonSchemaToTypeScript(definition.output, pretty)
/**
* Decodes tool input before `run` is invoked. Effect Schemas validate (throwing on failure);
* JSON-Schema-described inputs pass through unvalidated (render-only).
*/
export const decodeInput = <R>(definition: Definition<R>, value: unknown): unknown =>
isEffectSchema(definition.input) ? Schema.decodeUnknownSync(definition.input)(value) : value
/**
* Decodes a tool result before it is exposed to the program. Effect Schemas validate and
* transform (throwing on failure); JSON Schema outputs and tools without an output schema pass
* the host value through unchanged.
*/
export const decodeOutput = <R>(definition: Definition<R>, value: unknown): unknown =>
definition.output !== undefined && isEffectSchema(definition.output)
? Schema.decodeUnknownSync(definition.output)(value)
: value

View file

@ -1,4 +1,4 @@
import { Effect, JsonPointer, Schema } from "effect"
import { Effect, Schema } from "effect"
/**
* JSON Schema subset accepted for render-only tool schemas.
@ -30,25 +30,25 @@ export type JsonSchema = {
}
/** Either a validating Effect Schema or a render-only JSON Schema document. */
export type ToolSchema = Schema.Decoder<unknown> | JsonSchema
export type SchemaType = Schema.Decoder<unknown> | JsonSchema
/** Schema-backed tool definition consumed by a CodeMode tool tree. */
export type Definition<R = never> = {
readonly _tag: "CodeModeTool"
readonly description: string
readonly input: ToolSchema
readonly output: ToolSchema | undefined
readonly input: SchemaType
readonly output: SchemaType | undefined
readonly run: (input: unknown) => Effect.Effect<unknown, unknown, R>
}
/** The value `run` receives: the decoded type for Effect Schemas, `unknown` for JSON Schemas. */
export type InputType<S> = S extends Schema.Decoder<unknown> ? S["Type"] : unknown
type InputType<S> = S extends Schema.Decoder<unknown> ? S["Type"] : unknown
/** The value `run` returns: the encoded type for Effect Schemas, `unknown` otherwise. */
export type ResultType<S> = S extends Schema.Decoder<unknown> ? S["Encoded"] : unknown
type ResultType<S> = S extends Schema.Decoder<unknown> ? S["Encoded"] : unknown
/** Options for defining one CodeMode tool. */
export type Options<I extends ToolSchema, O extends ToolSchema | undefined, R = never> = {
export type Options<I extends SchemaType, O extends SchemaType | undefined, R = never> = {
readonly description: string
readonly input: I
readonly output?: O
@ -58,305 +58,6 @@ export type Options<I extends ToolSchema, O extends ToolSchema | undefined, R =
export const isDefinition = <R = never>(value: unknown): value is Definition<R> =>
typeof value === "object" && value !== null && "_tag" in value && value._tag === "CodeModeTool"
const isEffectSchema = (schema: ToolSchema): schema is Schema.Decoder<unknown> & Schema.Top => Schema.isSchema(schema)
const renderLiteral = (value: unknown): string => JSON.stringify(value) ?? "unknown"
/**
* Bare TypeScript identifier - usable unquoted as an object key (and, in the tool runtime,
* with dot access as a tool-path segment). Anything else must be quoted/bracketed.
*/
export const identifierSegment = /^[A-Za-z_$][A-Za-z0-9_$]*$/
/** Renders a property name as a valid TS object key: bare when an identifier, quoted otherwise. */
const renderKey = (name: string): string => (identifierSegment.test(name) ? name : JSON.stringify(name))
const effectNumberSentinel = (schema: JsonSchema) =>
schema.type === "string" &&
Array.isArray(schema.enum) &&
schema.enum.length === 1 &&
(schema.enum[0] === "NaN" || schema.enum[0] === "Infinity" || schema.enum[0] === "-Infinity")
const intersection = (members: ReadonlyArray<string>): string => {
const concrete = members.filter((member) => member !== "unknown")
if (concrete.length === 0) return "unknown"
if (concrete.length === 1) return concrete[0] ?? "unknown"
return concrete.map((member) => (member.includes(" | ") ? `(${member})` : member)).join(" & ")
}
/**
* Recursion ceiling for schema rendering. Object, array, and union recursion all increment
* depth, so this bounds every recursion path - pathological or structurally cyclic schemas
* degrade to `unknown` instead of overflowing the stack (rendering must never throw).
*/
const MAX_RENDER_DEPTH = 8
type RenderContext = {
readonly definitions: Readonly<Record<string, JsonSchema>>
/** Indented, JSDoc-annotated multiline rendering (search results); compact single line otherwise. */
readonly pretty: boolean
}
const hasUnresolvedRef = (
schema: JsonSchema,
definitions: Readonly<Record<string, JsonSchema>>,
seen: ReadonlySet<string> = new Set(),
visited: ReadonlySet<JsonSchema> = new Set(),
): boolean => {
if (visited.has(schema)) return false
const nextVisited = new Set([...visited, schema])
if (schema.$ref !== undefined) {
const segment = schema.$ref.match(/^#\/(?:\$defs|definitions)\/([^/]+)$/)?.[1]
const name = segment === undefined ? undefined : JsonPointer.unescapeToken(segment)
if (name === undefined || definitions[name] === undefined || seen.has(name)) return true
if (hasUnresolvedRef(definitions[name], definitions, new Set([...seen, name]), nextVisited)) return true
}
return [
...(schema.anyOf ?? []),
...(schema.oneOf ?? []),
...(schema.allOf ?? []),
...Object.values(schema.properties ?? {}),
...(schema.items === undefined ? [] : [schema.items]),
...(typeof schema.additionalProperties === "object" ? [schema.additionalProperties] : []),
].some((item) => hasUnresolvedRef(item, definitions, seen, nextVisited))
}
/**
* Schema constraints a TypeScript type cannot express natively but a model benefits from,
* surfaced as JSDoc tags (`@deprecated`, `@default`, `@format`, `@minItems`, `@maxItems`).
*/
const docTags = (schema: JsonSchema): Array<string> => {
const tags: Array<string> = []
if (schema.deprecated === true) tags.push("@deprecated")
if (schema.default !== undefined) {
try {
const rendered = JSON.stringify(schema.default)
if (rendered !== undefined) tags.push(`@default ${rendered}`)
} catch {
// unserializable default: skip rather than emit a broken tag
}
}
if (typeof schema.format === "string") tags.push(`@format ${schema.format}`)
if (typeof schema.minItems === "number") tags.push(`@minItems ${schema.minItems}`)
if (typeof schema.maxItems === "number") tags.push(`@maxItems ${schema.maxItems}`)
return tags
}
/**
* Format a schema `description` plus `tags` as a JSDoc comment at the given indent,
* preserving multi-line text (a single line stays `/** ... *\/`; multiple lines become a
* `*`-prefixed block). `*\/` is neutralized so nothing can close the comment early, and
* blank leading/trailing lines are trimmed. Returns "" (else a trailing newline) so
* callers can prepend it directly to the field line.
*/
const jsdoc = (description: string | undefined, tags: ReadonlyArray<string>, pad: string): string => {
const lines = [...(description === undefined ? [] : description.split("\n")), ...tags].map((line) =>
line.replaceAll("*/", "* /").replace(/\s+$/, ""),
)
while (lines.length > 0 && lines[0]!.trim() === "") lines.shift()
while (lines.length > 0 && lines[lines.length - 1]!.trim() === "") lines.pop()
if (lines.length === 0) return ""
if (lines.length === 1) return `${pad}/** ${lines[0]} */\n`
const body = lines.map((line) => `${pad} *${line === "" ? "" : ` ${line}`}`).join("\n")
return `${pad}/**\n${body}\n${pad} */\n`
}
const renderSchema = (
schema: JsonSchema,
ctx: RenderContext,
depth = 0,
seen: ReadonlySet<string> = new Set(),
): string => {
if (depth > MAX_RENDER_DEPTH) return "unknown"
const nested =
schema.definitions === undefined && schema.$defs === undefined
? ctx
: { ...ctx, definitions: { ...ctx.definitions, ...(schema.definitions ?? {}), ...(schema.$defs ?? {}) } }
if (schema.$ref) {
const segment = schema.$ref.match(/^#\/(?:\$defs|definitions)\/([^/]+)$/)?.[1]
const name = segment === undefined ? undefined : JsonPointer.unescapeToken(segment)
if (!name || !nested.definitions[name] || seen.has(name)) return "unknown"
return intersection([
renderSchema(nested.definitions[name], nested, depth, new Set([...seen, name])),
renderSchema({ ...schema, $ref: undefined }, nested, depth + 1, seen),
])
}
if (schema.const !== undefined) return renderLiteral(schema.const)
if (schema.enum) return schema.enum.map(renderLiteral).join(" | ")
const alternatives = schema.anyOf ?? schema.oneOf
if (alternatives) {
// Effect's number schema emits `anyOf: [{ type: "number" }, { const: "NaN" },
// { const: "Infinity" }, { const: "-Infinity" }]`. Collapse only that artifact;
// real JSON Schema unions such as `string | number` or `number | null` must keep
// every branch.
if (
alternatives.some((item) => item.type === "number") &&
alternatives.every((item) => item.type === "number" || effectNumberSentinel(item))
)
return "number"
// An empty Schema.Struct({}) emits `anyOf: [{ type: "object" }, { type: "array" }]`
// (no properties/items); render the bare shape as {} instead of `{} | Array<unknown>`.
if (
alternatives.length === 2 &&
alternatives[0]?.type === "object" &&
alternatives[0].properties === undefined &&
alternatives[1]?.type === "array" &&
alternatives[1].items === undefined
) {
return "{}"
}
const members = alternatives.map((item) => renderSchema(item, nested, depth + 1, seen))
if (members.some((member) => member === "unknown")) return "unknown"
return intersection([
members.join(" | "),
renderSchema({ ...schema, anyOf: undefined, oneOf: undefined }, nested, depth + 1, seen),
])
}
if (schema.allOf) {
const members = schema.allOf.map((item) => renderSchema(item, nested, depth + 1, seen))
if (schema.allOf.some((item) => hasUnresolvedRef(item, nested.definitions))) return "unknown"
return intersection([renderSchema({ ...schema, allOf: undefined }, nested, depth + 1, seen), ...members])
}
if (Array.isArray(schema.type)) {
return schema.type.map((item) => renderSchema({ ...schema, type: item }, nested, depth + 1, seen)).join(" | ")
}
if (schema.type === "string") return "string"
if (schema.type === "number" || schema.type === "integer") return "number"
if (schema.type === "boolean") return "boolean"
if (schema.type === "null") return "null"
if (schema.type === "array") return `Array<${renderSchema(schema.items ?? {}, nested, depth + 1, seen)}>`
if (schema.type === "object" || schema.properties) {
const required = new Set(schema.required ?? [])
const properties = Object.entries(schema.properties ?? {})
const additional = schema.additionalProperties
const indexType =
additional && typeof additional === "object" ? renderSchema(additional, nested, depth + 1, seen) : undefined
const field = ([name, value]: readonly [string, JsonSchema]) =>
`${renderKey(name)}${required.has(name) ? "" : "?"}: ${renderSchema(value, nested, depth + 1, seen)}`
if (!ctx.pretty) {
const fields = properties.map(field)
if (indexType !== undefined) fields.push(`[key: string]: ${indexType}`)
return fields.length === 0 ? "{}" : `{ ${fields.join("; ")} }`
}
// Pretty: an indented block, each described field preceded by its JSDoc comment.
if (properties.length === 0 && indexType === undefined) return "{}"
const pad = " ".repeat(depth + 1)
const lines = properties.map(
(entry) => `${jsdoc(entry[1].description, docTags(entry[1]), pad)}${pad}${field(entry)},`,
)
if (indexType !== undefined) lines.push(`${pad}[key: string]: ${indexType},`)
return `{\n${lines.join("\n")}\n${" ".repeat(depth)}}`
}
return "unknown"
}
export const toTypeScript = (schema: Schema.Top, decoded = false, pretty = false): string => {
try {
const visible = decoded ? Schema.toType(schema) : schema
const document = Schema.toJsonSchemaDocument(visible) as {
readonly schema: JsonSchema
readonly definitions?: Readonly<Record<string, JsonSchema>>
}
return renderSchema(document.schema, { definitions: document.definitions ?? {}, pretty })
} catch {
return "unknown"
}
}
/** Renders a raw JSON Schema document as a TypeScript type string. */
export const jsonSchemaToTypeScript = (schema: JsonSchema, pretty = false): string => {
try {
return renderSchema(schema, { definitions: { ...(schema.definitions ?? {}), ...(schema.$defs ?? {}) }, pretty })
} catch {
return "unknown"
}
}
/** One input property of a tool, extracted best-effort from its input schema. */
export type InputProperty = {
readonly name: string
readonly description: string | undefined
readonly required: boolean
}
/**
* The property names, descriptions, and required flags of a tool's input schema - the raw
* material for search text. Best-effort: Effect Schemas go through their
* JSON Schema document (the same emission signature rendering uses); JSON Schemas are read
* directly, resolving a trivial top-level `$ref` into `$defs`/`definitions` when present.
* Anything unresolvable yields `[]` (search falls back to path + description).
*/
export const inputProperties = <R>(definition: Definition<R>): Array<InputProperty> => {
try {
const document = isEffectSchema(definition.input)
? (Schema.toJsonSchemaDocument(definition.input) as {
readonly schema: JsonSchema
readonly definitions?: Readonly<Record<string, JsonSchema>>
})
: {
schema: definition.input,
definitions: { ...(definition.input.definitions ?? {}), ...(definition.input.$defs ?? {}) },
}
const definitions = document.definitions ?? {}
let schema = document.schema
if (schema.$ref !== undefined) {
const segment = schema.$ref.match(/^#\/(?:\$defs|definitions)\/([^/]+)$/)?.[1]
const name = segment === undefined ? undefined : JsonPointer.unescapeToken(segment)
const resolved = name === undefined ? undefined : definitions[name]
if (resolved === undefined) return []
schema = resolved
}
const required = new Set(schema.required ?? [])
return Object.entries(schema.properties ?? {}).map(([name, value]) => ({
name,
description: typeof value.description === "string" ? value.description : undefined,
required: required.has(name),
}))
} catch {
return []
}
}
/**
* The model-visible TypeScript type of a tool's input. `pretty` renders an indented
* multiline block with schema descriptions and constraints as JSDoc comments on the
* fields; the default stays the compact single-line form.
*/
export const inputTypeScript = <R>(definition: Definition<R>, pretty = false): string =>
isEffectSchema(definition.input)
? toTypeScript(definition.input, false, pretty)
: jsonSchemaToTypeScript(definition.input, pretty)
/**
* The model-visible TypeScript type of a tool's result; tools without an output schema
* return `unknown`. `pretty` renders the JSDoc-annotated multiline form, as for inputs.
*/
export const outputTypeScript = <R>(definition: Definition<R>, pretty = false): string =>
definition.output === undefined
? "unknown"
: isEffectSchema(definition.output)
? toTypeScript(definition.output, true, pretty)
: jsonSchemaToTypeScript(definition.output, pretty)
/**
* Decodes tool input before `run` is invoked. Effect Schemas validate (throwing on failure);
* JSON-Schema-described inputs pass through unvalidated (render-only).
*/
export const decodeInput = <R>(definition: Definition<R>, value: unknown): unknown =>
isEffectSchema(definition.input) ? Schema.decodeUnknownSync(definition.input)(value) : value
/**
* Decodes a tool result before it is exposed to the program. Effect Schemas validate and
* transform (throwing on failure); JSON Schema outputs and tools without an output schema pass
* the host value through unchanged.
*/
export const decodeOutput = <R>(definition: Definition<R>, value: unknown): unknown =>
definition.output !== undefined && isEffectSchema(definition.output)
? Schema.decodeUnknownSync(definition.output)(value)
: value
/**
* Defines one schema-described tool available to a CodeMode program through `tools.*`.
*
@ -384,7 +85,7 @@ export const decodeOutput = <R>(definition: Definition<R>, value: unknown): unkn
* })
* ```
*/
export const make = <I extends ToolSchema, const O extends ToolSchema | undefined = undefined, R = never>(
export const make = <I extends SchemaType, const O extends SchemaType | undefined = undefined, R = never>(
options: Options<I, O, R>,
): Definition<R> => ({
_tag: "CodeModeTool",
@ -393,6 +94,3 @@ export const make = <I extends ToolSchema, const O extends ToolSchema | undefine
output: options.output,
run: (input) => options.run(input as InputType<I>),
})
/** Constructors for schema-backed tools exposed inside CodeMode programs. */
export const Tool = { make, isDefinition }

View file

@ -27,8 +27,23 @@ export class SandboxSet {
readonly set = new Set<unknown>()
}
export const isSandboxValue = (value: unknown): value is SandboxDate | SandboxRegExp | SandboxMap | SandboxSet =>
export class SandboxURLSearchParams {
constructor(readonly params: URLSearchParams) {}
}
export class SandboxURL {
readonly searchParams: SandboxURLSearchParams
constructor(readonly url: URL) {
this.searchParams = new SandboxURLSearchParams(url.searchParams)
}
}
export const isSandboxValue = (
value: unknown,
): value is SandboxDate | SandboxRegExp | SandboxMap | SandboxSet | SandboxURL | SandboxURLSearchParams =>
value instanceof SandboxDate ||
value instanceof SandboxRegExp ||
value instanceof SandboxMap ||
value instanceof SandboxSet
value instanceof SandboxSet ||
value instanceof SandboxURL ||
value instanceof SandboxURLSearchParams

View file

@ -655,9 +655,11 @@ describe("CodeMode public contract", () => {
for (const missing of ["Modules/imports", "classes", "generators", "fetch", "promise chaining"]) {
expect(instructions).toContain(missing)
}
expect(instructions).toContain("URL, URLSearchParams, and URI encoding helpers")
expect(instructions).not.toContain("host globals")
expect(instructions).toContain("Use Code Mode tools for external operations")
expect(instructions).toContain(
"Dates serialize to ISO strings at data boundaries; Map/Set/RegExp serialize to `{}`.",
"Dates and URLs serialize to strings at data boundaries; Map/Set/RegExp/URLSearchParams serialize to `{}`.",
)
})

View file

@ -93,10 +93,16 @@
},
"role": {
"type": "string",
"enum": ["admin", "member"]
"enum": [
"admin",
"member"
]
}
},
"required": ["name", "email"],
"required": [
"name",
"email"
],
"additionalProperties": false
}
}
@ -137,7 +143,9 @@
"type": "integer"
}
},
"required": ["query"],
"required": [
"query"
],
"additionalProperties": false
}
},
@ -208,10 +216,17 @@
},
"role": {
"type": "string",
"enum": ["admin", "member"]
"enum": [
"admin",
"member"
]
}
},
"required": ["id", "name", "email"],
"required": [
"id",
"name",
"email"
],
"additionalProperties": false
}
},

File diff suppressed because it is too large Load diff

View file

@ -1,8 +1,8 @@
import { describe, expect, test } from "bun:test"
import { Effect, Layer, Option } from "effect"
import { HttpClient, HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
import { CodeMode, OpenAPI } from "../src/index.js"
import { inputTypeScript, outputTypeScript, Tool } from "../src/tool.js"
import { CodeMode, OpenAPI, Tool } from "../src/index.js"
import { inputTypeScript, outputTypeScript } from "../src/tool-schema.js"
const baseUrl = "http://localhost:4096"
type Document = OpenAPI.Document
@ -54,9 +54,7 @@ const json = (value: unknown, status = 200) =>
const singleOperation = (operation: Record<string, unknown>, method = "get"): Document => ({
openapi: "3.1.0",
paths: {
"/test": { [method]: { operationId: "test", responses: { 200: { description: "Success" } }, ...operation } },
},
paths: { "/test": { [method]: { operationId: "test", responses: { 200: { description: "Success" } }, ...operation } } },
})
describe("OpenAPI.fromSpec", () => {
@ -96,12 +94,7 @@ describe("OpenAPI.fromSpec", () => {
const remove = toolAt(api.tools, "users.remove")
expect(api.skipped).toEqual([])
if (
!Tool.isDefinition(get) ||
!Tool.isDefinition(create) ||
!Tool.isDefinition(search) ||
!Tool.isDefinition(remove)
) {
if (!Tool.isDefinition(get) || !Tool.isDefinition(create) || !Tool.isDefinition(search) || !Tool.isDefinition(remove)) {
throw new Error("happy-path fixture did not generate every operation")
}
expect(inputTypeScript(get)).toBe(
@ -117,8 +110,7 @@ describe("OpenAPI.fromSpec", () => {
const result = await Effect.runPromise(
CodeMode.make({ tools: { api: api.tools } })
.execute(
`
.execute(`
const user = await tools.api.users.get({
userId: "user-1",
include: ["profile", "permissions"],
@ -136,8 +128,7 @@ describe("OpenAPI.fromSpec", () => {
})
const removed = await tools.api.users.remove({ userId: "user-1" })
return { user, created, summary, removed }
`,
)
`)
.pipe(Effect.provide(client.layer)),
)
@ -491,9 +482,9 @@ describe("OpenAPI.fromSpec", () => {
expect(url.searchParams.get("nullable")).toBe("null")
expect(url.searchParams.get("constructor")).toBe("safe")
expect(client.requests[0]!.headers.meta).toBe("a=b,c=d")
await expect(Effect.runPromise(tool.run({ keys: [undefined] }).pipe(Effect.provide(client.layer)))).rejects.toThrow(
"unsupported nested value",
)
await expect(
Effect.runPromise(tool.run({ keys: [undefined] }).pipe(Effect.provide(client.layer))),
).rejects.toThrow("unsupported nested value")
})
test("skips unsupported parameter encodings and malformed security", () => {
@ -595,10 +586,7 @@ describe("OpenAPI.fromSpec", () => {
test("applies authentication carriers without prototype or collision loss", async () => {
const client = recordingClient(() => json({ ok: true }))
const authenticated = (
security: ReadonlyArray<Record<string, ReadonlyArray<string>>>,
schemes: Record<string, unknown>,
) =>
const authenticated = (security: ReadonlyArray<Record<string, ReadonlyArray<string>>>, schemes: Record<string, unknown>) =>
OpenAPI.fromSpec({
baseUrl,
spec: { ...singleOperation({}), security, components: { securitySchemes: schemes } },
@ -614,10 +602,13 @@ describe("OpenAPI.fromSpec", () => {
expect(new URL(client.requests[0]!.url).searchParams.get("__proto__")).toBe("secret")
const duplicate = toolAt(
authenticated([{ first: [], second: [] }], {
first: { type: "apiKey", in: "header", name: "x-key" },
second: { type: "apiKey", in: "header", name: "x-key" },
}).tools,
authenticated(
[{ first: [], second: [] }],
{
first: { type: "apiKey", in: "header", name: "x-key" },
second: { type: "apiKey", in: "header", name: "x-key" },
},
).tools,
"test",
)
if (!Tool.isDefinition(duplicate)) throw new Error("duplicate auth tool was not generated")
@ -642,7 +633,8 @@ describe("OpenAPI.fromSpec", () => {
},
},
auth: {
resolve: ({ name }) => Effect.succeed(name === "bearer" ? { type: "bearer", token: "secret" } : undefined),
resolve: ({ name }) =>
Effect.succeed(name === "bearer" ? { type: "bearer", token: "secret" } : undefined),
},
})
const alternativeTool = toolAt(alternative.tools, "test")
@ -774,7 +766,9 @@ describe("OpenAPI.fromSpec", () => {
const oversized = recordingClient(
() => new Response(null, { headers: { "content-length": String(50 * 1024 * 1024 + 1) } }),
)
const malformed = recordingClient(() => new Response("{", { headers: { "content-type": "application/json" } }))
const malformed = recordingClient(
() => new Response("{", { headers: { "content-type": "application/json" } }),
)
const chunked = recordingClient(() => new Response(new Uint8Array(50 * 1024 * 1024 + 1)))
await expect(Effect.runPromise(tool.run({}).pipe(Effect.provide(oversized.layer)))).rejects.toThrow(

View file

@ -1,7 +1,7 @@
import { describe, expect, test } from "bun:test"
import { Effect, Schema } from "effect"
import { CodeMode } from "../src/index.js"
import { Tool, inputTypeScript, jsonSchemaToTypeScript, outputTypeScript } from "../src/tool.js"
import { CodeMode, Tool } from "../src/index.js"
import { inputTypeScript, jsonSchemaToTypeScript, outputTypeScript } from "../src/tool-schema.js"
// A raw JSON Schema tool in the shape an MCP adapter produces: render-only input schema
// whose property descriptions and constraints must surface as JSDoc in pretty signatures.

View file

@ -1,12 +1,12 @@
import { describe, expect, test } from "bun:test"
import { Effect } from "effect"
import { Effect, Schema } from "effect"
import { CodeMode, Tool } from "../src/index.js"
// Standard-library value types: Date, RegExp, Map, Set. Programs use them as ordinary JS;
// intra-sandbox checkpoints (Object.* helpers, spread, coercion inputs) preserve the live
// values, while at the host boundary (final result, tool arguments, JSON.stringify) they
// serialize exactly as JSON.stringify would: Date -> ISO string (invalid -> null),
// RegExp/Map/Set -> {}.
// URL -> href, and RegExp/Map/Set/URLSearchParams -> {}.
const run = (code: string) => Effect.runPromise(CodeMode.execute({ code, tools: {} }))
const value = async (code: string) => {
const result = await run(code)
@ -149,6 +149,67 @@ describe("RegExp", () => {
expect(await value(`return "hi bob".replace(/b(o)b/, "[$1]")`)).toBe("hi [o]")
})
test("function replacers receive captures, offsets, input, and named groups", async () => {
expect(
await value(`
const seen = []
const output = "a1b22".replace(/(\\d)(\\d)?/g, (match, first, second, offset, input) => {
seen.push([match, first, second === undefined, offset, input])
return Number(match) * 2
})
return { output, seen }
`),
).toEqual({
output: "a2b44",
seen: [
["1", "1", true, 1, "a1b22"],
["22", "2", false, 3, "a1b22"],
],
})
expect(
await value(`
return "red-blue".replace(
/(?<left>[a-z]+)-(?<right>[a-z]+)/,
(match, left, right, offset, input, groups) => groups.right + ":" + groups.left,
)
`),
).toBe("blue:red")
})
test("function replacers support string searches, zero-length matches, and result coercion", async () => {
expect(await value(`return "banana".replace("na", (match, offset, input) => "[" + offset + "]")`)).toBe("ba[2]na")
expect(await value(`return "ab".replaceAll("", (match, offset) => offset)`)).toBe("0a1b2")
expect(await value(`return "😀".replaceAll(/(?:)/gu, (match, offset) => "[" + offset + "]")`)).toBe("[0]😀[2]")
expect(
await value(`return "123".replace(/\\d/g, (match) => match === "1" ? 7 : match === "2" ? null : { n: 3 })`),
).toBe("7null[object Object]")
})
test("function replacers can await effectful tool calls", async () => {
const decorate = Tool.make({
description: "Decorate a string",
input: Schema.String,
output: Schema.String,
run: (input) => Effect.succeed(`[${input}]`),
})
const result = await Effect.runPromise(
CodeMode.execute({
tools: { host: { decorate } },
code: `return "a1b22".replace(/\\d+/g, async (match) => await tools.host.decorate(match))`,
}),
)
expect(result.ok && result.value).toBe("a[1]b[22]")
const missingAwait = await Effect.runPromise(
CodeMode.execute({
tools: { host: { decorate } },
code: `return "a1".replace(/\\d/, (match) => tools.host.decorate(match))`,
}),
)
expect(!missingAwait.ok && missingAwait.error.kind).toBe("InvalidDataValue")
expect(!missingAwait.ok && missingAwait.error.message).toContain("un-awaited Promise")
})
test("replaceAll without the g flag is a catchable error", async () => {
expect(await value(`try { "a".replaceAll(/a/, "b"); return "no" } catch { return "caught" }`)).toBe("caught")
})
@ -208,6 +269,165 @@ describe("RegExp", () => {
})
})
describe("URL and URI helpers", () => {
test("encodes and decodes complete URIs and URI components", async () => {
expect(
await value(`
return [
encodeURI("https://example.test/a b?q=a/b"),
encodeURIComponent("a b/c?"),
decodeURI("https://example.test/a%20b?q=a/b"),
decodeURIComponent("a%20b%2Fc%3F"),
["a b", "c/d"].map(encodeURIComponent),
]
`),
).toEqual([
"https://example.test/a%20b?q=a/b",
"a%20b%2Fc%3F",
"https://example.test/a b?q=a/b",
"a b/c?",
["a%20b", "c%2Fd"],
])
expect(
await value(`try { decodeURIComponent("%zz"); return false } catch (error) { return error instanceof URIError }`),
).toBe(true)
})
test("resolves and mutates URLs with linked search parameters", async () => {
expect(
await value(`
const url = new URL("../users?id=old#top", "https://user:pass@example.com:8443/api/v1/")
url.pathname = "/items/a b"
url.searchParams.set("id", "a b")
url.searchParams.append("tag", "x/y")
url.hash = "part 1"
return {
href: url.href,
origin: url.origin,
host: url.host,
pathname: url.pathname,
search: url.search,
id: url.searchParams.get("id"),
string: String(url),
json: url.toJSON(),
instances: [
url instanceof URL,
url.searchParams instanceof URLSearchParams,
url.searchParams === url.searchParams,
],
}
`),
).toEqual({
href: "https://user:pass@example.com:8443/items/a%20b?id=a+b&tag=x%2Fy#part%201",
origin: "https://example.com:8443",
host: "example.com:8443",
pathname: "/items/a%20b",
search: "?id=a+b&tag=x%2Fy",
id: "a b",
string: "https://user:pass@example.com:8443/items/a%20b?id=a+b&tag=x%2Fy#part%201",
json: "https://user:pass@example.com:8443/items/a%20b?id=a+b&tag=x%2Fy#part%201",
instances: [true, true, true],
})
})
test("URLSearchParams supports records, pairs, mutation, callbacks, and materialization", async () => {
expect(
await value(`
const params = new URLSearchParams([["tag", "b"], ["tag", "a"], ["q", "a b"]])
const seen = []
params.forEach((value, key) => seen.push(key + "=" + value))
params.delete("tag", "b")
params.append("tag", "c")
params.sort()
return {
text: params.toString(),
size: params.size,
tags: params.getAll("tag"),
has: params.has("tag", "c"),
entries: Array.from(params),
object: Object.fromEntries(params),
record: new URLSearchParams({ page: 2, filter: "open" }).toString(),
seen,
}
`),
).toEqual({
text: "q=a+b&tag=a&tag=c",
size: 3,
tags: ["a", "c"],
has: true,
entries: [
["q", "a b"],
["tag", "a"],
["tag", "c"],
],
object: { q: "a b", tag: "c" },
record: "page=2&filter=open",
seen: ["tag=b", "tag=a", "q=a b"],
})
})
test("URL parsing failures are catchable and values use native JSON forms", async () => {
expect(
await value(`
const parsed = URL.parse("/users", "https://example.test/api/")
let invalidIsTypeError = false
try { new URL("not relative without a base") } catch (error) { invalidIsTypeError = error instanceof TypeError }
return {
canParse: URL.canParse("/users", "https://example.test/api/"),
cannotParse: URL.canParse("not relative without a base"),
parsed: parsed.href,
invalidIsTypeError,
boundary: [new URL("https://example.test/a"), new URLSearchParams("q=one")],
json: JSON.stringify({ url: new URL("https://example.test/a"), params: new URLSearchParams("q=one") }),
}
`),
).toEqual({
canParse: true,
cannotParse: false,
parsed: "https://example.test/users",
invalidIsTypeError: true,
boundary: ["https://example.test/a", {}],
json: '{"url":"https://example.test/a","params":{}}',
})
})
test("distinguishes omitted URL arguments from explicit undefined", async () => {
expect(
await value(`
function throwsTypeError(run) {
try { run(); return false } catch (error) { return error instanceof TypeError }
}
const params = new URLSearchParams()
const required = [
() => params.append(),
() => params.delete(),
() => params.get(),
() => params.getAll(),
() => params.has(),
() => params.set(),
() => params.forEach(),
].map(throwsTypeError)
params.append(undefined, undefined)
return {
construct: throwsTypeError(() => new URL()),
canParse: throwsTypeError(() => URL.canParse()),
parse: throwsTypeError(() => URL.parse()),
explicitUndefined: new URL(undefined, "https://example.test/base/").href,
params: params.toString(),
required,
}
`),
).toEqual({
construct: true,
canParse: true,
parse: true,
explicitUndefined: "https://example.test/base/undefined",
params: "undefined=undefined",
required: [true, true, true, true, true, true, true],
})
})
})
describe("Map", () => {
test("get/set/has/size with chaining", async () => {
expect(