vrr/opencode
2
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-18 06:03:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

fix(codemode): improve prompting (#35509)

Browse source
This commit is contained in:
Aiden Cline 2026-07-06 09:19:04 -05:00 • committed by GitHub
parent 175d200e53
commit 7f008df79d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 407 additions and 362 deletions
Download patch file Download diff file Expand all files Collapse all files

27
packages/codemode/README.md
View file

@ -179,14 +179,14 @@ Supported bearer, basic, header, and query authentication follows OpenAPI `secur
## Discovery
The agent-tool instructions use a budgeted catalog. Every tool namespace is always listed with its tool count regardless of budget, and as many complete tool signatures (each with a one-line description) as fit an estimated-token budget are inlined. Selection is round-robin across namespaces for fairness: in each round (namespaces alphabetical), every namespace still holding un-inlined tools attempts to place its next-cheapest signature line against the shared budget, and a namespace whose next line does not fit drops out while the others keep going - so every namespace gets some representation before any namespace gets everything. The instructions state exactly how comprehensive the list is, both overall (`COMPLETE list` vs `PARTIAL - N of M shown`) and per namespace (`(3 tools)`, `(3 tools, 1 shown)`, `(3 tools, none shown)`).
The agent-tool instructions use a budgeted catalog. Every tool namespace is always listed with its tool count regardless of budget, and as many complete, JSDoc-annotated tool signatures (each with a one-line description) as fit an estimated-token budget are inlined. Schema field descriptions and tags are part of each signature's measured cost. Selection is round-robin across namespaces for fairness: in each round (namespaces alphabetical), every namespace still holding un-inlined tools attempts to place its next-cheapest signature against the shared budget, and a namespace whose next signature does not fit drops out while the others keep going - so every namespace gets some representation before any namespace gets everything. The instructions state exactly how comprehensive the list is, both overall (`COMPLETE list` vs `PARTIAL - N of M shown`) and per namespace (`(3 tools)`, `(3 tools, 1 shown)`, `(3 tools, none shown)`).
The default budget is 2,000 estimated tokens (characters / 4, the same heuristic OpenCode uses). Override it when constructing a runtime:
The catalog-entry budget defaults to 2,000 estimated tokens (characters / 4, the same heuristic OpenCode uses). It applies only to full tool entries shown in the catalog; fixed instructions and namespace summaries are not counted. Override it when constructing a runtime:
```ts
const runtime = CodeMode.make({
tools,
discovery: { maxInlineCatalogTokens: 6_000 },
discovery: { catalogBudget: 6_000 },
})
```
@ -199,30 +199,37 @@ const matches = await tools.$codemode.search({
query: "order status",
namespace: "orders", // optional: scope to one top-level namespace
limit: 10,
offset: 0,
})
```
`search` performs deterministic, additive field-weighted matching. The query is tokenized (camelCase boundaries split; every non-alphanumeric character is a separator; empties and `*` are dropped), and each term scores every tool: exact path or path-segment match (20), path substring (8), description substring (4), and searchable-text substring (2). Each term also carries naive singular variants (trailing `s`/`es` stripped), and a field check passes when the term or any variant matches - so a plural query term (`issues`) still finds a tool whose text only says `issue`, without changing the weights. The searchable text also includes the input schema's property names and their description strings, so a query naming a parameter finds its tool, and substring matching means partial words match. Scores sum across terms; matches are sorted by score (ties broken alphabetically by path) and capped at `limit` results (default 10).
`search` performs deterministic, additive field-weighted matching. The query is tokenized (camelCase boundaries split; every non-alphanumeric character is a separator; empties and `*` are dropped), and each term scores every tool: exact path or path-segment match (20), path substring (8), description substring (4), and searchable-text substring (2). Each term also carries naive singular variants (trailing `s`/`es` stripped), and a field check passes when the term or any variant matches - so a plural query term (`issues`) still finds a tool whose text only says `issue`, without changing the weights. The searchable text also includes the input schema's property names and their description strings, so a query naming a parameter finds its tool, and substring matching means partial words match. Scores sum across terms; matches are sorted by score (ties broken alphabetically by path), then sliced from the zero-based `offset` (default 0) to the configured `limit` (default 10). `remaining` counts matches after the current page. `next` is `{ offset }` when another page exists and `null` on the final page; spread it into the original request to preserve its query, namespace, and limit.
Each result contains the path, description, and generated TypeScript signature, so no second lookup is needed. The result signature is the pretty, JSDoc-annotated multiline form: each described input/output field carries its schema `description` as a `/** ... */` comment, and constraints TypeScript cannot express ride along as tags (`@deprecated`, `@default`, `@format`, `@minItems`, `@maxItems`). The inline catalog in the instructions keeps the compact single-line form.
```ts
const request = { query: "order status", namespace: "orders", limit: 10 }
const page = await tools.$codemode.search(request)
const nextPage = page.next ? await tools.$codemode.search({ ...request, ...page.next }) : undefined
```
Each result contains the path, description, and the same generated TypeScript signature used by the inline catalog, so no second lookup is needed. Signatures use the JSDoc-annotated multiline form: each described input/output field carries its schema `description` as a `/** ... */` comment, and constraints TypeScript cannot express ride along as tags (`@deprecated`, `@default`, `@format`, `@minItems`, `@maxItems`).
```ts
tools.github.list_issues(input: {
/** Repository owner */
owner: string
owner: string,
/** Cursor from the previous response's pageInfo */
after?: string
after?: string,
/**
* Results per page
* @default 30
*/
perPage?: number
perPage?: number,
}): Promise<unknown>
```
Result paths are rendered as JavaScript expressions rooted at `tools` (`tools.orders.lookup`, or `tools.context7["resolve-library-id"]` for non-identifier segments), so each `path` is directly usable as the call site. An empty query browses the catalog alphabetically by path; combined with `namespace` (`{ query: "", namespace: "orders" }`) it lists everything in that namespace. A query that names one tool path exactly (canonical path, `tools.`-prefixed path, or rendered JavaScript expression) is treated as a lookup and returns that tool alone.
The instructions are structured markdown, ordered so the workflow sits at the top and the catalog at the bottom: a `## Workflow` section with numbered steps (find a tool via search when the catalog is partial, or pick from the inlined list when it is complete; call the exact path as-is; `JSON.parse` string results; return only the needed fields), a `## Rules` section holding only guidance the workflow does not already cover (only listed/search-result tools exist inside `tools`; filter and aggregate collections in code; treat `Promise<unknown>` results as shapeless until verified; run independent calls through `Promise.all`; enumerate `tools` with `Object.keys`/`for...in`; browse a namespace via search when it is advertised), a short `## Syntax` section that assumes standard JavaScript and names only what is unusual (TypeScript annotations stripped; the data-boundary serialization of Date/Map/Set/RegExp) or missing (classes, generators, `for await...of`, `.then`/`.catch`/`.finally`), and the budgeted `## Available tools` catalog. Example call forms use explicit `<namespace>.<tool>`/`<field>` placeholders - never a real or fabricated tool name.
The instructions are structured markdown, ordered so the workflow sits at the top and the catalog at the bottom: a `## Workflow` section with numbered steps (find a tool via search when the catalog is partial, or pick from the inlined list when it is complete; call the exact path as-is; return only the needed fields), a `## Rules` section holding only guidance the workflow does not already cover (only listed/search-result Code Mode tools and internal runtime tools exist inside `tools`; filter and aggregate collections in code; narrow `Promise<unknown>` results at runtime; run independent calls through `Promise.all`; enumerate `tools` with `Object.keys`/`for...in`; browse a namespace and paginate search results when search is advertised), a short `## Language` section that identifies the runtime as a restricted JavaScript orchestration language and names its major unavailable capabilities, and the budgeted `## Available tools` catalog. Example call forms use explicit `<namespace>.<tool>`/`<field>` placeholders - never a real or fabricated tool name.
A host cannot define its own `$codemode` top-level namespace.
@ -234,7 +241,7 @@ CodeMode executes a deliberately bounded JavaScript subset. It supports:
- `if`, conditional expressions, `switch`, `for`, `for...of` (arrays, strings, Maps, Sets), `for...in` (own keys of plain objects, index strings of arrays, and namespace/tool names of `tools` references - anything else is an error suggesting `for...of` or `Object.keys`, rather than real JS's surprising behavior of indices for strings and zero iterations for Maps/Sets), `while`, and `do...while`.
- Arrow functions and function declarations with closures, defaults, rest parameters, and destructuring.
- Optional chaining, nullish coalescing, templates, spread (arrays, strings, Maps, Sets), and `try`/`catch`.
- Common array, string, number, `Object`, `Math`, and `JSON` operations. Mutating array methods include `push`/`pop`/`shift`/`unshift`/`splice` (removes in place and returns the removed elements)/`fill`/`copyWithin`; array `keys`/`values`/`entries` return **arrays** (matching the Map/Set convention) and work with `for...of` and spread. String methods include `localeCompare` (locale/options arguments ignored), `normalize`, and the `trimLeft`/`trimRight` aliases. `Object.keys` also accepts arrays (index strings, as in JS) and tool references: `Object.keys(tools)` lists the top-level namespaces and `Object.keys(tools.ns)` the names at that node (a callable tool enumerates as `[]`; an unknown path is an `UnknownTool` diagnostic). `Object.values`/`Object.entries` on a tool reference fail with a pointer at `Object.keys(tools)` and `tools.$codemode.search`.
- Common array, string, number, `Object`, `Math`, and `JSON` operations. Mutating array methods include `push`/`pop`/`shift`/`unshift`/`splice` (removes in place and returns the removed elements)/`fill`/`copyWithin`; array `keys`/`values`/`entries` return **arrays** (matching the Map/Set convention) and work with `for...of` and spread. String methods include `localeCompare` (locale/options arguments ignored), `normalize`, and the `trimLeft`/`trimRight` aliases. `Object.keys` also accepts arrays (index strings, as in JS) and tool references: `Object.keys(tools)` lists the top-level namespaces, including `$codemode`, and `Object.keys(tools.ns)` lists the names at that node (a callable tool enumerates as `[]`; an unknown path is an `UnknownTool` diagnostic). `Object.values`/`Object.entries` on a tool reference fail with a pointer at `Object.keys(tools)` and `tools.$codemode.search`.
- `Date` - `Date.now()`/`Date.parse()`/`Date.UTC()`, `new Date(...)`, the getter methods, and date arithmetic/comparison via the time value. Dates stringify as ISO (`toString` included, for determinism across host timezones).
- Regular expressions - `/literals/` and `new RegExp(...)` with `test`/`exec` (stateful `lastIndex` for `g`), plus string `match`/`matchAll`/`replace`/`replaceAll`/`split`/`search` with patterns. Match results are arrays carrying `index` and named `groups` as own properties (`input` is omitted). Invalid patterns, invalid flags, and missing-`g` calls fail with catchable errors that say what was wrong and how to fix it (escaping hints, the exact `/pattern/g` to write). Patterns run on the host engine, so pathological backtracking is bounded only by the execution timeout. Function replacers are not supported.
- `Map` and `Set` - construction from entries/arrays/strings, `get`/`set`/`add`/`has`/`delete`/`clear`/`size`/`forEach`, and `keys`/`values`/`entries` returning **arrays** (not iterators).

102
packages/codemode/codemode.md
View file

@ -65,17 +65,24 @@ From issue #34787 and design discussion. Do not relitigate these casually.
### Discovery / search
- **Search only - no separate `describe`.** `tools.$codemode.search({ query?, namespace?,
limit? })` over the final tool tree, owned by this package.
- Search result item shape: `{ path, description, signature }` in an `{ items, total }`
wrapper. The `signature` string embeds the full input/output TypeScript types - in search
results it is the pretty, JSDoc-annotated multiline form (Fix 7), so per-field schema
`description`s and constraints (`@default`, `@format`, `@deprecated`, `@minItems`,
`@maxItems`) ride along as field comments. The original spec's separate `input`/`output`
limit?, offset? })` over the final tool tree, owned by this package.
- Search result item shape: `{ path, description, signature }` in an
`{ items, remaining, next }`
wrapper. The `signature` string embeds the full input/output TypeScript types and uses the
same pretty, JSDoc-annotated multiline form in inline catalogs and search results, so
per-field schema `description`s and constraints (`@default`, `@format`, `@deprecated`,
`@minItems`, `@maxItems`) ride along as field comments. The original spec's separate `input`/`output`
raw-schema fields are deliberately NOT added: shapes are already fully expressed in the
TypeScript signature and schema annotations now arrive as JSDoc - intent satisfied, letter
deviated. Result `path`s render a JavaScript expression rooted at `tools` (for example
`tools.github.list_issues` or `tools.context7["resolve-library-id"]`) so each is directly
usable as the call site; the internal `ToolDescription.path` stays unprefixed.
- `offset` is zero-based and defaults to 0. `remaining` counts matches after the current page;
`next` is `{ offset }` when another page exists and `null` on the final page.
- Search is an internal `Tool.make` definition backed by Effect input/output schemas. Its
validation, output checking, call observation, and TypeScript signature use the same path as
host-provided schema tools. Host-only catalog preparation keeps internal tools out of their
own search index; only conditional advertisement remains special.
- Default limit: **10** (done). Exact-path lookup goes through search too: a query equal to a
canonical tool path, `tools.`-prefixed path, or rendered JavaScript expression returns that
tool alone (done).
@ -251,7 +258,7 @@ output limit; return a smaller value]`; logs keep leading lines within the remai
truncation is now the only result-size mechanism.)
- **Search polish**: default limit 12 -> **10** (`defaultSearchLimit`); exact-path lookup - a
trimmed query equal to one tool path (optionally `tools.`-prefixed) returns that tool alone
(`total: 1`), bypassing ranking. Tokenization/ranking/shape unchanged.
(`remaining: 0`, `next: null`), bypassing ranking. Tokenization/ranking/shape unchanged.
### Wave 3 - OpenCode MCP adapter (done)
@ -312,10 +319,10 @@ Instructions are now the budgeted-catalog + prompting-guidance form; verified e2
real MCP config. Package still 101 tests / 0 fail; opencode adapter suites still 34 + 16; both
packages typecheck clean.
- **Budgeted catalog** (`discoveryPlan` in `tool-runtime.ts`): the all-or-nothing
- **Budgeted catalog** (`prepare` in `tool-runtime.ts`): the all-or-nothing
inline/search modes are gone - `DiscoveryMode` deleted, `CodeMode.DiscoveryOptions` is just
`{ maxInlineCatalogBytes? }` (default 16,000 UTF-8 bytes; later converted to
`maxInlineCatalogTokens`, default 4,000 estimated tokens - see Post-wave fixes). Port of
`catalogBudget`, default 4,000 estimated tokens - see Post-wave fixes). Port of
the old opencode
`describe()` `PREVIEW_BUDGET` algorithm, adapted to `ToolDescription`: every namespace is
ALWAYS listed with its tool count; full signature lines
@ -439,9 +446,9 @@ adapter needed **no changes**.
defeating discovery. Fixes, all in this package:
- `ToolRuntime.make` now returns a `keys(path)` capability (`namespaceKeys` in
`tool-runtime.ts`) threaded into the `Interpreter` alongside `invoke` - the interpreter
still never holds the host tool tree. `Object.keys(tools)` yields the top-level namespace
names (never `$codemode`, which is virtual - but `Object.keys(tools.$codemode)` yields
`["search"]`), `Object.keys(tools.ns)` the names at that node; a callable tool leaf
still never holds the callable tool tree. `Object.keys(tools)` yields the top-level namespace
names, including the internally registered `$codemode`; `Object.keys(tools.$codemode)` yields
`["search"]`, and `Object.keys(tools.ns)` yields names at that node; a callable tool leaf
enumerates as `[]` (like `Object.keys` of a JS function); an unknown path throws an
`UnknownTool` diagnostic suggesting `Object.keys(tools)` and `$codemode.search` (matching
call-time unknown-tool behavior rather than silently returning `[]`).
@ -459,7 +466,7 @@ adapter needed **no changes**.
- `supportedSyntaxMessage`, the instructions loops line, and README "Supported Programs"
mention the new surface; tests in `test/enumeration.test.ts` (14, incl. the exact
transcript program) plus one adapter-level assertion that `Object.keys(tools)` returns
MCP server names.
MCP server and CodeMode namespace names.
- **Search ranking, namespace scoping, prefixed result paths (done).**
Motivation: the Wave 4 e2e run showed a model retrying calls because search-result paths
@ -480,7 +487,7 @@ adapter needed **no changes**.
An empty query now browses ALPHABETICALLY by path (was declaration order). Kept:
`{ path, description, signature }` result items, default limit 10, exact-path instant
lookup, input validation errors.
- **Namespace scoping**: `tools.$codemode.search({ query?, namespace?, limit? })` -
- **Namespace scoping**: `tools.$codemode.search({ query?, namespace?, limit?, offset? })` -
`namespace` (validated as a string when provided) filters `SearchEntry`s to one top-level
namespace before ranking; `{ query: "", namespace: "github" }` lists that namespace
alphabetically. `searchSignature` updated.
@ -489,7 +496,7 @@ adapter needed **no changes**.
segments), directly usable as the call site. Internal `ToolDescription.path` stays
unprefixed; only the search RESULT items are rendered this way. Exact-path queries accept
canonical paths and rendered expressions.
- **Instructions** (`discoveryPlan`): an explicit calling-convention line and a browse
- **Instructions** (`prepare`): an explicit calling-convention line and a browse
hint on the search advertisement (both since absorbed into the `## Rules` section by
the instructions restructure below).
- **Tests**: package search/discovery tests updated (prefixed paths, alphabetical browse)
@ -499,28 +506,23 @@ adapter needed **no changes**.
- **Instructions restructure: markdown sections, placeholder-only call forms (done).**
The flat prose instructions (which mixed a real catalog tool with fabricated result
fields in the worked example) are replaced by structured markdown in `discoveryPlan`,
fields in the worked example) are replaced by structured markdown in `prepare`,
ordered so the workflow sits at the top (the least likely part of a long description to
be truncated or skimmed away) and the catalog at the bottom (the per-section content
described here was later condensed by Fix 8 - Workflow/Rules deduped, Syntax inverted):
- **Intro** (2 lines): "Write a CodeMode program... Return code only." + "Execute
JavaScript in a confined runtime with access to the tools listed below under
`tools.*`." (the second line drops the tools clause when the tree is empty).
- **`## Workflow`**: numbered steps - find a tool via `tools.$codemode.search` -> read
the `{ path, description, signature }` matches -> call by path -> `typeof res ===
"string" ? JSON.parse(res) : res` -> return only the needed fields. When the catalog is
COMPLETE the search/read steps collapse into "Pick a tool from the list under
`## Available tools`" and the steps renumber (4 instead of 5).
- **`## Rules`**: call-by-exact-path; TEXT-is-JSON -> JSON.parse; return small (never raw
payloads); filter/aggregate large collections in code instead of per-item round-trips;
described here was later condensed by Fix 8 and the language-accuracy pass):
- **Intro**: identifies the language as restricted JavaScript for calling tools rather than
a general-purpose runtime.
- **`## Workflow`**: with a partial catalog, return search results from one execution, then
copy a selected path into the next execution. With a complete catalog, pick and call an
inlined signature, then return only the needed fields.
- **`## Rules`**: narrow unknown results at runtime; filter/aggregate large collections in code instead of per-item round-trips;
console.log/warn/error/dir/table for intermediates; `Promise.all` parallelism (no
.then/.catch - await + try/catch); `Object.keys(tools)`/`for...in` enumeration;
browse-one-namespace via search (PARTIAL only); and host-side media handling (files/
images never enter the program; a media-only call yields a small text marker - wording
verified against the adapter's `toSandboxResult`/`mediaMarker`).
- **`## Syntax`**: the dense syntax lines unchanged, minus the Promise.all and console
lines (moved into Rules) and the `for (const ns in tools)` fragment (redundant with
the enumeration rule).
- **`## Language`**: a concise positive capability summary plus the major unavailable
runtime capabilities and the data-boundary serialization note.
- **`## Available tools`**: the budgeted catalog unchanged, with the COMPLETE/PARTIAL
header merged into the section heading (no trailing colon); the search-signature
advertisement follows when PARTIAL (its description-reading and browse clauses moved
@ -532,7 +534,7 @@ adapter needed **no changes**.
appear anywhere in the instructions. Zero tools keep "No tools are currently
available." under minimal sections (intro + Syntax + Available tools).
- **Tests**: the package worked-example test replaced by section-structure/placeholder
assertions (section order; JSON.parse + return-small rules present; no
assertions (section order; unknown-result + return-small rules present; no
`total_count`/`list_issues`/real-tool example lines; browse hint only when PARTIAL;
zero-tool minimal sections) - 156 pass / 0 fail; adapter suites gain the same
assertions on the built description (still 35 + 16, green).
@ -542,9 +544,9 @@ budget; namespaces must always be present):
- `src/token.ts` added: copy of `@opencode-ai/core/util/token` (`round(chars / 4)`), so
the package stays dependency-free; keep in sync if the core heuristic changes.
- `CodeMode.DiscoveryOptions.maxInlineCatalogBytes` -> `maxInlineCatalogTokens` (default 4,000
- `CodeMode.DiscoveryOptions.maxInlineCatalogBytes` -> `catalogBudget` (default 4,000
estimated tokens ~ the old 16,000 bytes at 4 chars/token - behavior parity, not a size
reduction). `discoveryPlan` charges `estimate(catalogLine(tool))` per line; cheapest-first
reduction). `prepare` charges `estimate(catalogLine(tool))` per line; cheapest-first
- stop-on-first-miss unchanged at the time (stop-on-first-miss replaced by round-robin in
Fix 8). Namespace stub lines were and remain unbudgeted - every
namespace always appears with its tool count, even at budget 0 (asserted in package and
@ -672,10 +674,8 @@ along). All in `tool-runtime.ts`; no interpreter changes.
interfaces/type aliases are stripped and TS **enums actually work** (transpileModule
compiles them to an IIFE the interpreter runs), hence enums deliberately unmentioned.
`supportedSyntaxMessage` (the in-diagnostic text in `codemode.ts`) is untouched.
- **Workflow/Rules deduped**: the call-by-exact-path, JSON.parse-string-results, and
return-small content now lives ONLY in the numbered Workflow steps (with their
compliance-driving justifications inline: "most tools return JSON as a string", "raw
payloads get truncated and waste context"); Rules keeps only bullets adding new
- **Workflow/Rules deduped**: the call-by-exact-path and return-small content now lives ONLY
in the numbered Workflow steps; Rules keeps only bullets adding new
content - filter/aggregate collections in code, console.\* intermediates (logs ride
back), Promise.all parallelism, Object.keys/for...in enumeration, browse-namespace
(PARTIAL only), and the media rule compressed to one line. The no-.then/.catch
@ -690,7 +690,7 @@ along). All in `tool-runtime.ts`; no interpreter changes.
(332 -> 176), Syntax 453 -> 188 (419 -> 174); fixed prose total 1,005 -> 610 (927 -> 562),
~ 40% reduction with no behavioral content dropped. Workflow grew slightly because it
absorbed the deduped parse/return-small justifications.
- **Round-robin namespace inlining** (`discoveryPlan`): the ported stop-on-first-miss
- **Round-robin namespace inlining** (`prepare`): the ported stop-on-first-miss
behavior (alphabetically-late namespaces starved to "none shown" while an early
namespace inlines everything) is replaced by round-robin fairness - in each round
(namespaces alphabetical), every namespace still holding un-inlined tools attempts to
@ -708,9 +708,9 @@ along). All in `tool-runtime.ts`; no interpreter changes.
the four field checks passes when ANY form matches. Weights, exact-path lookup, and
namespace scoping untouched. A true plural path match still outranks a singular-only
description match (path substring 8 + searchable 2 > description 4 + searchable 2).
- **Tests**: package instruction/structure assertions updated to the new text; new
syntax-section test (leads with "Standard modern JavaScript works", names the
verified not-supported list, keeps the data-boundary note); the budget-exhaustion
- **Tests**: package instruction/structure assertions updated to the new text; the
language-section test rejects full-runtime wording, names major unavailable capabilities,
and keeps the data-boundary note; the budget-exhaustion
test rewritten to assert the new fairness (alpha.expensive not fitting must NOT
prevent beta.cheap from showing: PARTIAL 2 of 3, `- beta (1 tool)` fully shown); new
plural/singular test (query "issues" finds a singular-only tool; ranking still
@ -724,7 +724,7 @@ along). All in `tool-runtime.ts`; no interpreter changes.
**Fix 9 - prompting trims per user review of Fix 8** (user reviewed the condensed
instructions and directed further cuts):
- Default `maxInlineCatalogTokens` 4,000 -> **2,000** (user wants ~2k tokens of signatures
- Default `catalogBudget` 4,000 -> **2,000** (user wants ~2k tokens of signatures
auto-inlined; round-robin fairness from Fix 8 spreads it across all namespaces).
- Console rule and files/images rule DROPPED from `## Rules`. Replaced by a single
`unknown`-treatment warning: "A result typed `Promise<unknown>` has no guaranteed
@ -733,9 +733,9 @@ instructions and directed further cuts):
return the same data; the prompt stays console-neutral, neither for nor against.)
The media-stripping MECHANISM is unchanged and still tested; only the prose about it
is gone - the `[N images attached]` marker is self-explanatory in context.
- Kept as-is per user: the JSON.parse workflow step (maps to the original motivating
transcript failure; NOT copied from prior art - see section 5 note), the browse-namespace rule
(undecided), no no-fetch/ambient-authority rule added (proposed, not approved).
- Later revised: unconditional JSON parsing was removed because text results are not
necessarily JSON. The browse-namespace rule remains; the language section now states that
ambient `fetch` is unavailable and external operations go through Code Mode tools.
- Explicitly REJECTED for now: auto-parsing JSON-looking text results at the adapter
boundary ("could get weird" - type flips, program-sees vs tool-sent divergence). Logged
as a next-iteration follow-up below.
@ -1004,11 +1004,19 @@ focused interpreter-surface pass rather than picked off piecemeal.
- [x] Sandbox values nested inside logged containers print `[CodeMode reference]`
(`console.log({ m: map })`) - could deep-format instead.
### Next iteration: optional search input boundary
- [ ] `SearchInput` uses Effect's exact `optionalKey`, so an omitted field is accepted but an
explicitly present `undefined` field is rejected. The previous handwritten validator
treated explicit `undefined` as omission. Decide whether search should preserve that
convenience locally or whether all tool arguments should adopt JSON-style undefined
normalization; do not broaden `copyOut` semantics solely to fix search.
### Next iteration: text-result handling (deliberate follow-up, user-directed)
- [ ] Revisit how MCP text results reach the program. Today: `structuredContent` when the
server sends it, else joined text as a plain string (the program JSON.parses it,
guided by a workflow step). Considered and deferred: (a) conservative boundary
server sends it, else joined text as a plain string. Programs narrow unknown results
before use; the prompt no longer recommends unconditional JSON parsing. Considered and deferred: (a) conservative boundary
auto-parse (text starting with `{`/`[` that parses cleanly becomes an object) -
rejected for now as potentially confusing (type flips; program sees something other
than what the tool sent); (b) raw-envelope passthrough with the envelope shape

18
packages/codemode/src/codemode.ts
View file

@ -38,12 +38,12 @@ export type ExecutionLimits = {
/** Controls how much of the tool catalog is inlined in agent instructions. */
export type DiscoveryOptions = {
/**
* Estimated-token budget (chars/4, default 2000) for inlined full tool signatures in agent
* instructions. Signatures that fit are inlined round-robin across namespaces; every
* namespace is always listed with its tool count regardless of budget, and
* Approximate budget, in estimated tokens (chars/4, default 2000), for full tool entries in
* the instruction catalog. Tool entries are selected round-robin across namespaces. Fixed
* instructions and namespace summaries do not count toward the budget, and
* `tools.$codemode.search` is always registered.
*/
readonly maxInlineCatalogTokens?: number
readonly catalogBudget?: number
}
type ToolTree<R = never> = {
@ -3921,8 +3921,8 @@ const executeWithLimits = <const Tools extends Record<string, unknown>>(
const tools = ToolRuntime.make(
(options.tools ?? {}) as HostTools<Services<Tools>>,
limits.maxToolCalls,
hooks,
searchIndex,
hooks,
)
const logs: Array<string> = []
const logged = () => (logs.length > 0 ? { logs: [...logs] } : {})
@ -4061,10 +4061,10 @@ export const make = <const Tools extends Record<string, unknown> = {}>(
const tools = (options.tools ?? {}) as HostTools<Services<Tools>>
ToolRuntime.assertValidTools(tools)
const limits = resolveExecutionLimits(options.limits)
const discovery = ToolRuntime.discoveryPlan(tools, options.discovery?.maxInlineCatalogTokens)
const executeProgram = (code: string) => executeWithLimits<Tools>({ ...options, code }, limits, discovery.searchIndex)
const catalog = discovery.catalog
const instructions = discovery.instructions
const prepared = ToolRuntime.prepare(tools, options.discovery?.catalogBudget)
const executeProgram = (code: string) => executeWithLimits<Tools>({ ...options, code }, limits, prepared.searchIndex)
const catalog = prepared.catalog
const instructions = prepared.instructions
return {
catalog: () => catalog,

321
packages/codemode/src/tool-runtime.ts
View file

@ -1,4 +1,4 @@
import { Cause, Effect } from "effect"
import { Cause, Effect, Schema } from "effect"
import { ToolError, toolError } from "./tool-error.js"
import {
decodeInput as decodeToolInput,
@ -75,10 +75,26 @@ export type ToolDescription = {
export type SafeObject = Record<string, unknown>
const reservedNamespace = "$codemode"
const defaultMaxInlineCatalogTokens = 2_000
const defaultCatalogBudget = 2_000
const defaultSearchLimit = 10
const searchSignature =
"tools.$codemode.search({ query?: string, namespace?: string, limit?: number }): Promise<{ items: Array<{ path: string; description: string; signature: string }>; total: number }>"
const PositiveInt = Schema.Int.check(Schema.isGreaterThan(0))
const NonNegativeInt = Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))
const SearchInput = Schema.Struct({
query: Schema.optionalKey(Schema.String),
namespace: Schema.optionalKey(Schema.String),
limit: Schema.optionalKey(PositiveInt),
offset: Schema.optionalKey(NonNegativeInt),
})
const SearchItem = Schema.Struct({
path: Schema.String,
description: Schema.String,
signature: Schema.String,
})
const SearchOutput = Schema.Struct({
items: Schema.Array(SearchItem),
remaining: NonNegativeInt,
next: Schema.NullOr(Schema.Struct({ offset: NonNegativeInt })),
})
const toolExpression = (path: string) =>
"tools" +
path
@ -294,15 +310,17 @@ const definitions = <R>(
return entries
}
const describeDefinition = <R>(path: string, definition: Definition<R>): ToolDescription => ({
path,
description: definition.description,
signature: `${toolExpression(path)}(input: ${inputTypeScript(definition, true)}): Promise<${outputTypeScript(definition, true)}>`,
})
const visibleDefinitions = <R>(tools: HostTools<R>) =>
definitions(tools).map(({ path, definition }) => ({
path,
definition,
description: {
path,
description: definition.description,
signature: `${toolExpression(path)}(input: ${inputTypeScript(definition)}): Promise<${outputTypeScript(definition)}>`,
},
description: describeDefinition(path, definition),
}))
export const catalog = <R>(tools: HostTools<R>): ReadonlyArray<ToolDescription> =>
@ -316,11 +334,6 @@ export type DiscoveryPlan = {
export type SearchEntry = {
readonly description: ToolDescription
/**
* JSDoc-annotated multiline signature shown on search-result items; the compact
* single-line form (inline catalog lines) stays in `description.signature`.
*/
readonly signature: string
/** Top-level namespace (first path segment), matched by the search `namespace` option. */
readonly namespace: string
/** Lowercased path + description + input property names/descriptions, for substring matching. */
@ -354,8 +367,75 @@ const termForms = (term: string): Array<string> => {
return forms
}
const makeSearchTool = (searchIndex: ReadonlyArray<SearchEntry>): Definition => ({
_tag: "CodeModeTool",
description: "Search available Code Mode tools",
input: SearchInput,
output: SearchOutput,
run: (input) =>
Effect.sync(() => {
const request = input as typeof SearchInput.Type
const query = request.query ?? ""
const offset = request.offset ?? 0
const scoped =
request.namespace === undefined
? searchIndex
: searchIndex.filter((entry) => entry.namespace === request.namespace)
// A query that names one tool path exactly (canonical path or rendered JavaScript
// expression) is a lookup, not a search: return that tool alone.
const trimmed = query.trim()
const pathQuery = trimmed.startsWith("tools.") ? trimmed.slice("tools.".length) : trimmed
const exact =
pathQuery === ""
? undefined
: scoped.find(
(entry) => entry.description.path === pathQuery || toolExpression(entry.description.path) === trimmed,
)
const terms = tokenize(query).map(termForms)
// Additive field-weighted scoring, summed across terms: exact path or path segment
// (20) > path substring (8) > description substring (4) > any searchable text,
// including input parameter names and descriptions (2).
const ranked =
exact !== undefined
? [exact]
: scoped
.map((entry) => {
const path = entry.description.path.toLowerCase()
const description = entry.description.description.toLowerCase()
const score = terms.reduce(
(total, forms) =>
total +
(forms.some((form) => path === form || path.endsWith(`.${form}`)) ? 20 : 0) +
(forms.some((form) => path.includes(form)) ? 8 : 0) +
(forms.some((form) => description.includes(form)) ? 4 : 0) +
(forms.some((form) => entry.searchText.includes(form)) ? 2 : 0),
0,
)
return { entry, score }
})
.filter(({ score }) => terms.length === 0 || score > 0)
.sort(
(left, right) =>
right.score - left.score || left.entry.description.path.localeCompare(right.entry.description.path),
)
.map(({ entry }) => entry)
const items = ranked.slice(offset, offset + (request.limit ?? defaultSearchLimit)).map(({ description }) => ({
...description,
path: toolExpression(description.path),
}))
const remaining = Math.max(0, ranked.length - offset - items.length)
return {
items,
remaining,
next: remaining > 0 ? { offset: offset + items.length } : null,
}
}),
})
const searchDescription = describeDefinition(`${reservedNamespace}.search`, makeSearchTool([]))
const catalogLine = (tool: ToolDescription) => {
// Inline catalog lines use only a compact first line; full text stays in search results.
// Keep the tool description concise; the full schema documentation remains in the signature.
const line = tool.description.split("\n", 1)[0]!.trim()
const description = line.length > 120 ? line.slice(0, 119) + "..." : line
return description === "" ? ` - ${tool.signature}` : ` - ${tool.signature} // ${description}`
@ -363,7 +443,6 @@ const catalogLine = (tool: ToolDescription) => {
const toSearchEntry = <R>(path: string, definition: Definition<R>, description: ToolDescription): SearchEntry => ({
description,
signature: `${toolExpression(path)}(input: ${inputTypeScript(definition, true)}): Promise<${outputTypeScript(definition, true)}>`,
namespace: path.split(".", 1)[0]!,
searchText: [
path,
@ -388,7 +467,7 @@ export const assertValidTools = <R>(tools: HostTools<R>): void => {
/**
* Budgeted catalog: every namespace is always listed with its tool count; full call
* signatures are inlined against the `maxInlineCatalogTokens` budget (estimated tokens,
* signatures are inlined against the `catalogBudget` (estimated tokens,
* chars/4) round-robin across namespaces - in each round (namespaces alphabetical), every
* namespace still holding un-inlined tools attempts to place its next-cheapest line, and
* a namespace whose next line does not fit is done while the others keep going - so every
@ -397,12 +476,9 @@ export const assertValidTools = <R>(tools: HostTools<R>): void => {
* namespace. Namespace stub lines are never budgeted: every namespace appears with its
* tool count even at budget 0.
*/
export const discoveryPlan = <R>(
tools: HostTools<R>,
maxInlineCatalogTokens = defaultMaxInlineCatalogTokens,
): DiscoveryPlan => {
if (!Number.isSafeInteger(maxInlineCatalogTokens) || maxInlineCatalogTokens < 0) {
throw new RangeError("discovery.maxInlineCatalogTokens must be a non-negative safe integer")
export const prepare = <R>(tools: HostTools<R>, catalogBudget = defaultCatalogBudget): DiscoveryPlan => {
if (!Number.isSafeInteger(catalogBudget) || catalogBudget < 0) {
throw new RangeError("discovery.catalogBudget must be a non-negative safe integer")
}
const visible = visibleDefinitions(tools)
const described = visible.map(({ description }) => description)
@ -437,7 +513,7 @@ export const discoveryPlan = <R>(
for (const selection of active) {
const tool = selection.queue[0]!
const cost = estimateTokens(catalogLine(tool))
if (used + cost > maxInlineCatalogTokens) continue
if (used + cost > catalogBudget) continue
selection.queue.shift()
selection.picked.add(tool)
used += cost
@ -458,13 +534,14 @@ export const discoveryPlan = <R>(
// catalog at the bottom. Example call forms use placeholders - never a real or fabricated
// tool name - and show both dot and bracket notation so non-identifier names are not normalized.
const intro = [
"Write a CodeMode program to answer the request. Return code only.",
empty
? "Execute JavaScript in a confined runtime."
? "This is a restricted JavaScript language for calling tools, not a general-purpose runtime."
: complete
? "Execute JavaScript in a confined runtime. Inside this program, `tools` contains only the host-provided tools listed below; surrounding agent tools are not available unless listed here."
: "Execute JavaScript in a confined runtime. Inside this program, `tools` contains only the host-provided tools listed or searchable below; surrounding agent tools are not available unless listed here.",
...(empty ? [] : ["Do not infer or normalize tool names; use only exact signatures shown below or returned by search."]),
? "This is a restricted JavaScript language for calling tools, not a general-purpose runtime. Inside the confined interpreter, `tools` contains the Code Mode tools listed below and internal runtime tools; surrounding agent tools are not available."
: "This is a restricted JavaScript language for calling tools, not a general-purpose runtime. Inside the confined interpreter, `tools` contains the Code Mode tools listed or searchable below and internal runtime tools; surrounding agent tools are not available.",
...(empty
? []
: ["Do not infer or normalize tool names; use only exact signatures shown below or returned by search."]),
]
// The search step exists only when search is advertised (PARTIAL catalog); a COMPLETE
@ -478,16 +555,12 @@ export const discoveryPlan = <R>(
...(complete
? [
"1. Pick a tool from the list under `## Available tools` - each line is the exact call signature; use it as-is rather than guessing segments.",
'2. Call it using the exact signature shown; bracket notation and quotes are part of the path.',
'3. Parse text results: `const data = typeof res === "string" ? JSON.parse(res) : res` - most tools return JSON as a string.',
"4. Return only the fields you need: `return { <field>: data.<field> }` - raw payloads get truncated and waste context.",
"2. Call it using the exact signature shown: `const result = await tools.<namespace>.<tool>(input)`; bracket notation and quotes are part of the path.",
"3. Return only the fields you need from structured results; narrow unknown results before reading fields, and avoid returning large raw payloads.",
]
: [
'1. If the exact signature is not listed below, first search: `const { items } = await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
"2. Read the matches: each item is `{ path, description, signature }` - read the description before using an unfamiliar tool.",
"3. Call the result's `path` as-is; bracket notation and quotes are part of the path.",
'4. Parse text results: `const data = typeof res === "string" ? JSON.parse(res) : res` - most tools return JSON as a string.',
"5. Return only the fields you need: `return { <field>: data.<field> }` - raw payloads get truncated and waste context.",
'1. If needed, discover tools: `return await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
"2. In the next execution, copy a returned path exactly, call it, and return only the needed fields.",
]),
]
@ -498,24 +571,26 @@ export const discoveryPlan = <R>(
"## Rules",
"",
complete
? "- Only tools listed here are available inside `tools`; tools from the surrounding agent/runtime are not implicitly exposed."
: "- Only tools listed here or returned by `tools.$codemode.search` are available inside `tools`; tools from the surrounding agent/runtime are not implicitly exposed.",
? "- Only Code Mode tools listed here and internal runtime tools are available; surrounding agent tools are not implicitly exposed."
: "- Only Code Mode tools listed here or returned by `tools.$codemode.search` and internal runtime tools are available; surrounding agent tools are not implicitly exposed.",
"- Filter, aggregate, and transform collections in code - never return them raw or call a tool per item across messages.",
"- A result typed `Promise<unknown>` has no guaranteed shape - verify what actually came back before relying on its fields.",
"- A result typed `Promise<unknown>` may be structured data or text. Before reading fields, check that it is a non-null object and not an array; otherwise handle the returned text or primitive directly.",
'- Run independent calls in parallel: `await Promise.all(items.map((item) => tools.<namespace>.<tool>(item)))`, or use `tools.<namespace>["tool-name"](item)` when the listed signature uses bracket notation.',
"- `Object.keys(tools)` lists namespaces; `Object.keys(tools.<namespace>)` lists its tools; `for...in` works on both.",
...(complete
? []
: ['- Browse one namespace: `await tools.$codemode.search({ query: "", namespace: "<name>" })`.']),
: [
'- Browse one namespace: `await tools.$codemode.search({ query: "", namespace: "<name>" })`.',
"- If search returns `next`, repeat the same search with `offset: next.offset`.",
]),
]
const syntax = [
const language = [
"",
"## Syntax",
"## Language",
"",
"Standard modern JavaScript works: functions/closures, destructuring, template literals, loops, try/catch, spread, optional chaining, the usual Array/String/Object/Math/JSON methods, plus Date, RegExp, Map, Set, and Promise.all/allSettled/race/resolve/reject.",
"TypeScript type annotations are allowed and stripped before execution (decorators are not supported).",
"Not supported (each fails with a message naming the alternative): classes, generators, for await...of, .then/.catch/.finally (use await with try/catch).",
"Use common JavaScript data operations, functions, control flow, selected standard-library methods, and awaited tool calls.",
"Modules/imports, classes, generators, timers, fetch, eval, prototype access, arbitrary methods, and promise chaining are unavailable. Use Code Mode tools for external operations. Use await with try/catch.",
"Dates serialize to ISO strings at data boundaries; Map/Set/RegExp serialize to `{}`.",
]
@ -544,11 +619,11 @@ export const discoveryPlan = <R>(
for (const tool of group) if (picked.has(tool)) toolSection.push(catalogLine(tool))
}
if (!complete) {
toolSection.push("", "Search returns complete callable signatures:", `- ${searchSignature}`)
toolSection.push("", "Search returns complete callable signatures:", `- ${searchDescription.signature}`)
}
}
const lines = [...intro, ...workflow, ...rules, ...syntax, ...toolSection]
const lines = [...intro, ...workflow, ...rules, ...language, ...toolSection]
return {
catalog: described,
instructions: lines.join("\n"),
@ -557,20 +632,13 @@ export const discoveryPlan = <R>(
}
/**
* The enumerable names at one node of the host tool tree - namespace names at the root,
* The enumerable names at one node of the callable tool tree - namespace names at the root,
* tool/namespace names below - powering `Object.keys(tools)` and `for...in` over tool
* references. A callable tool is a leaf and enumerates as `[]` (like `Object.keys` of a
* function in JS). An unknown path is an `UnknownTool` error pointing at the working
* discovery idioms, mirroring how calling an unknown tool fails.
*/
const namespaceKeys = <R>(
tools: HostTools<R>,
path: ReadonlyArray<string>,
searchEnabled: boolean,
): ReadonlyArray<string> => {
// The reserved discovery namespace is virtual (never present in the host tree); enumerate
// it explicitly so `Object.keys(tools.$codemode)` matches the callable surface.
if (searchEnabled && path.length === 1 && path[0] === reservedNamespace) return ["search"]
const namespaceKeys = <R>(tools: HostTools<R>, path: ReadonlyArray<string>): ReadonlyArray<string> => {
let value: HostTool<R> | Definition<R> | HostTools<R> = tools
for (const segment of path) {
if (
@ -579,15 +647,9 @@ const namespaceKeys = <R>(
isDefinition(value) ||
!Object.hasOwn(value, segment)
) {
throw new ToolRuntimeError(
"UnknownTool",
`Unknown tool namespace '${path.join(".")}'.`,
searchEnabled
? [
"Object.keys(tools) lists the available namespaces; tools.$codemode.search({ query }) finds described tools.",
]
: ["Object.keys(tools) lists the available namespaces."],
)
throw new ToolRuntimeError("UnknownTool", `Unknown tool namespace '${path.join(".")}'.`, [
"Object.keys(tools) lists the available namespaces; tools.$codemode.search({ query }) finds described tools.",
])
}
value = value[segment] as HostTool<R> | Definition<R> | HostTools<R>
}
@ -595,11 +657,7 @@ const namespaceKeys = <R>(
return Object.keys(value)
}
const resolve = <R>(
tools: HostTools<R>,
path: ReadonlyArray<string>,
searchEnabled: boolean,
): HostTool<R> | Definition<R> => {
const resolve = <R>(tools: HostTools<R>, path: ReadonlyArray<string>): HostTool<R> | Definition<R> => {
let value: HostTool<R> | Definition<R> | HostTools<R> = tools
for (const segment of path) {
@ -609,11 +667,9 @@ const resolve = <R>(
isDefinition(value) ||
!Object.hasOwn(value, segment)
) {
throw new ToolRuntimeError(
"UnknownTool",
`Unknown tool '${path.join(".")}'.`,
searchEnabled ? ["Use tools.$codemode.search({ query }) to find available described tools."] : [],
)
throw new ToolRuntimeError("UnknownTool", `Unknown tool '${path.join(".")}'.`, [
"Use tools.$codemode.search({ query }) to find available described tools.",
])
}
value = value[segment] as HostTool<R> | Definition<R> | HostTools<R>
}
@ -629,7 +685,7 @@ export type ToolRuntime<R = never> = {
readonly root: ToolReference
readonly calls: Array<ToolCall>
readonly invoke: (path: ReadonlyArray<string>, args: Array<unknown>) => Effect.Effect<unknown, unknown, R>
/** Enumerable namespace/tool names at one node of the host tool tree; see `namespaceKeys`. */
/** Enumerable namespace/tool names at one node of the callable tool tree; see `namespaceKeys`. */
readonly keys: (path: ReadonlyArray<string>) => ReadonlyArray<string>
}
@ -637,11 +693,14 @@ export const make = <R>(
tools: HostTools<R>,
/** Undefined means unlimited tool calls. */
maxToolCalls: number | undefined,
searchIndex: ReadonlyArray<SearchEntry>,
hooks?: ToolCallHooks<R>,
searchIndex?: ReadonlyArray<SearchEntry>,
): ToolRuntime<R> => {
const calls: Array<ToolCall> = []
const searchEnabled = searchIndex !== undefined
const callableTools = {
...tools,
[reservedNamespace]: { search: makeSearchTool(searchIndex) },
}
// Wraps the settling portion of a tool call so onToolCallEnd observes success and failure
// symmetrically. Interruption (e.g. the execution timeout) fires neither outcome.
@ -680,7 +739,7 @@ export const make = <R>(
return {
root: new ToolReference([]),
calls,
keys: (path) => namespaceKeys(tools, path, searchEnabled),
keys: (path) => namespaceKeys(callableTools, path),
invoke: (path, args) =>
Effect.gen(function* () {
const name = path.join(".")
@ -691,107 +750,7 @@ export const make = <R>(
recordCall(call)
return calls.length - 1
}).pipe(Effect.tap((index) => hooks?.onToolCallStart?.({ index, name, input }) ?? Effect.void))
if (name === "$codemode.search") {
if (!searchEnabled) throw new ToolRuntimeError("UnknownTool", `Unknown tool '${name}'.`)
const input = externalArgs[0]
if (externalArgs.length !== 1 || input === null || typeof input !== "object" || Array.isArray(input)) {
throw new ToolRuntimeError(
"InvalidToolInput",
"tools.$codemode.search expects { query?: string; namespace?: string; limit?: number }.",
)
}
const request = input as { query?: unknown; namespace?: unknown; limit?: unknown }
if (request.query !== undefined && typeof request.query !== "string") {
throw new ToolRuntimeError(
"InvalidToolInput",
"tools.$codemode.search query must be a string when provided.",
)
}
if (request.namespace !== undefined && typeof request.namespace !== "string") {
throw new ToolRuntimeError(
"InvalidToolInput",
"tools.$codemode.search namespace must be a string when provided.",
)
}
if (
request.limit !== undefined &&
(typeof request.limit !== "number" || !Number.isSafeInteger(request.limit) || request.limit <= 0)
) {
throw new ToolRuntimeError(
"InvalidToolInput",
"tools.$codemode.search limit must be a positive safe integer when provided.",
)
}
const query = typeof request.query === "string" ? request.query : ""
const namespace = typeof request.namespace === "string" ? request.namespace : undefined
const index = yield* recordAndObserve(request)
return yield* observeEnd(
Effect.try({
try: () => {
const limit = typeof request.limit === "number" ? request.limit : defaultSearchLimit
const scoped =
namespace === undefined ? searchIndex : searchIndex.filter((entry) => entry.namespace === namespace)
// A query that names one tool path exactly (canonical path or rendered
// JavaScript expression) is a lookup, not a search: return that tool alone.
const trimmed = query.trim()
const pathQuery = trimmed.startsWith("tools.") ? trimmed.slice("tools.".length) : trimmed
const exact =
pathQuery === ""
? undefined
: scoped.find(
(entry) =>
entry.description.path === pathQuery || toolExpression(entry.description.path) === trimmed,
)
const terms = tokenize(query).map(termForms)
// Additive field-weighted scoring, summed across terms: exact path or path
// segment (20) > path substring (8) > description substring (4) > any
// searchable text, incl. input parameter names/descriptions (2). Each term
// matches a field when any of its forms (the term or a singular variant)
// does. An empty query browses everything, alphabetical by path.
const ranked =
exact !== undefined
? [exact]
: scoped
.map((entry) => {
const path = entry.description.path.toLowerCase()
const description = entry.description.description.toLowerCase()
const score = terms.reduce(
(total, forms) =>
total +
(forms.some((form) => path === form || path.endsWith(`.${form}`)) ? 20 : 0) +
(forms.some((form) => path.includes(form)) ? 8 : 0) +
(forms.some((form) => description.includes(form)) ? 4 : 0) +
(forms.some((form) => entry.searchText.includes(form)) ? 2 : 0),
0,
)
return { entry, score }
})
.filter(({ score }) => terms.length === 0 || score > 0)
.sort(
(left, right) =>
right.score - left.score ||
left.entry.description.path.localeCompare(right.entry.description.path),
)
.map(({ entry }) => entry)
// Result paths are rendered as JavaScript expressions so each `path` is
// directly usable as the call site (`await tools.github.list({ ... })` or
// `await tools.ns["dashed-name"]({ ... })`). The signature is the pretty,
// JSDoc-annotated form (schema descriptions and constraints ride along as
// field comments).
const items = ranked.slice(0, limit).map(({ description, signature }) => ({
...description,
path: toolExpression(description.path),
signature,
}))
return copyIn({ items, total: ranked.length }, "Result from tool '$codemode.search'")
},
catch: (cause) => cause,
}),
{ index, name, input: request },
)
}
const tool = resolve(tools, path, searchEnabled)
const tool = resolve(callableTools, path)
let describedInput: unknown
if (isDefinition(tool)) {
if (externalArgs.length !== 1)

4
packages/codemode/src/tool-schema.ts
View file

@ -187,9 +187,9 @@ const renderSchema = (
if (properties.length === 0 && indexType === undefined) return "{}"
const pad = " ".repeat(depth + 1)
const lines = properties.map(
(entry) => `${jsdoc(entry[1].description, docTags(entry[1]), pad)}${pad}${field(entry)}`,
(entry) => `${jsdoc(entry[1].description, docTags(entry[1]), pad)}${pad}${field(entry)},`,
)
if (indexType !== undefined) lines.push(`${pad}[key: string]: ${indexType}`)
if (indexType !== undefined) lines.push(`${pad}[key: string]: ${indexType},`)
return `{\n${lines.join("\n")}\n${" ".repeat(depth)}}`
}
return "unknown"

196
packages/codemode/test/codemode.test.ts
View file

@ -426,7 +426,7 @@ describe("CodeMode schema flexibility", () => {
{
path: "adapter.call",
description: "Call an adapter-described tool",
signature: "tools.adapter.call(input: { id: string; count?: number }): Promise<unknown>",
signature: "tools.adapter.call(input: {\n id: string,\n count?: number,\n}): Promise<unknown>",
},
])
@ -459,7 +459,7 @@ describe("CodeMode schema flexibility", () => {
{
path: "users.lookup",
description: "Look up a user",
signature: "tools.users.lookup(input: { login: string }): Promise<{ login: string; id: number }>",
signature: "tools.users.lookup(input: {\n login: string,\n}): Promise<{\n login: string,\n id: number,\n}>",
},
])
@ -475,7 +475,7 @@ describe("CodeMode schema flexibility", () => {
run: () => Effect.succeed("pong"),
})
const runtime = CodeMode.make({ tools: { net: { ping } } })
expect(runtime.catalog()[0]?.signature).toBe("tools.net.ping(input: { host: string }): Promise<unknown>")
expect(runtime.catalog()[0]?.signature).toBe("tools.net.ping(input: {\n host: string,\n}): Promise<unknown>")
const result = await Effect.runPromise(runtime.execute(`return await tools.net.ping({ host: "example.test" })`))
expect(result.ok).toBe(true)
@ -512,19 +512,19 @@ describe("CodeMode public contract", () => {
{
path: "orders.lookup",
description: "Look up an order by ID",
signature: "tools.orders.lookup(input: { id: string }): Promise<{ id: string; status: string }>",
signature: "tools.orders.lookup(input: {\n id: string,\n}): Promise<{\n id: string,\n status: string,\n}>",
},
])
expect(runtime.instructions()).toContain("Available tools (COMPLETE list")
expect(runtime.instructions()).toContain("- orders (1 tool)")
expect(runtime.instructions()).toContain(
" - tools.orders.lookup(input: { id: string }): Promise<{ id: string; status: string }> // Look up an order by ID",
" - tools.orders.lookup(input: {\n id: string,\n}): Promise<{\n id: string,\n status: string,\n}> // Look up an order by ID",
)
// A fully inlined catalog does not advertise search in the instructions...
expect(runtime.instructions()).not.toMatch(/\$codemode/)
// ...but the search tool stays registered, so a speculative call still works. Search
// results carry the pretty multiline signature; the inline catalog stays compact.
// ...but the search tool stays registered, so a speculative call still works with the
// same signature as the inline catalog.
const result = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({ query: "order" })`))
expect(result.ok).toBe(true)
if (result.ok) {
@ -533,10 +533,12 @@ describe("CodeMode public contract", () => {
{
path: "tools.orders.lookup",
description: "Look up an order by ID",
signature: "tools.orders.lookup(input: {\n id: string\n}): Promise<{\n id: string\n status: string\n}>",
signature:
"tools.orders.lookup(input: {\n id: string,\n}): Promise<{\n id: string,\n status: string,\n}>",
},
],
total: 1,
remaining: 0,
next: null,
})
}
})
@ -554,11 +556,11 @@ describe("CodeMode public contract", () => {
{
path: "context7.resolve-library-id",
description: "Resolve a library ID",
signature: 'tools.context7["resolve-library-id"](input: { libraryName: string }): Promise<string>',
signature: 'tools.context7["resolve-library-id"](input: {\n libraryName: string,\n}): Promise<string>',
},
])
expect(runtime.instructions()).toContain(
'tools.context7["resolve-library-id"](input: { libraryName: string }): Promise<string>',
'tools.context7["resolve-library-id"](input: {\n libraryName: string,\n}): Promise<string>',
)
const search = await Effect.runPromise(
@ -571,10 +573,11 @@ describe("CodeMode public contract", () => {
{
path: 'tools.context7["resolve-library-id"]',
description: "Resolve a library ID",
signature: 'tools.context7["resolve-library-id"](input: {\n libraryName: string\n}): Promise<string>',
signature: 'tools.context7["resolve-library-id"](input: {\n libraryName: string,\n}): Promise<string>',
},
],
total: 1,
remaining: 0,
next: null,
})
}
@ -588,7 +591,7 @@ describe("CodeMode public contract", () => {
runtime.execute(`return await tools.$codemode.search({ query: 'tools.context7["resolve-library-id"]' })`),
)
expect(exact.ok).toBe(true)
if (exact.ok) expect((exact.value as { total: number }).total).toBe(1)
if (exact.ok) expect(exact.value).toMatchObject({ remaining: 0, next: null })
})
test("instructions use markdown sections with placeholder-only call forms", () => {
@ -597,23 +600,26 @@ describe("CodeMode public contract", () => {
// Sections in order: workflow at the top, catalog at the bottom.
expect(instructions).toContain("## Workflow")
expect(instructions).toContain("## Rules")
expect(instructions).toContain("## Syntax")
expect(instructions).toContain("## Language")
expect(instructions.indexOf("## Workflow")).toBeLessThan(instructions.indexOf("## Rules"))
expect(instructions.indexOf("## Rules")).toBeLessThan(instructions.indexOf("## Syntax"))
expect(instructions.indexOf("## Syntax")).toBeLessThan(instructions.indexOf("\n## Available tools (COMPLETE list"))
// The workflow carries the result-shape guidance; Rules only add content beyond it.
expect(instructions).toContain(
'`const data = typeof res === "string" ? JSON.parse(res) : res` - most tools return JSON as a string',
expect(instructions.indexOf("## Rules")).toBeLessThan(instructions.indexOf("## Language"))
expect(instructions.indexOf("## Language")).toBeLessThan(
instructions.indexOf("\n## Available tools (COMPLETE list"),
)
expect(instructions).not.toContain("JSON.parse(res)")
expect(instructions).toContain("Return only the fields you need")
expect(instructions).toContain("raw payloads get truncated and waste context")
expect(instructions).toContain("avoid returning large raw payloads")
expect(instructions).toContain("Do not infer or normalize tool names")
expect(instructions).toContain("bracket notation and quotes are part of the path")
expect(instructions).toContain("surrounding agent tools are not available unless listed here")
expect(instructions).toContain("Only tools listed here are available inside `tools`")
expect(instructions).toContain("surrounding agent tools are not available")
expect(instructions).toContain("Only Code Mode tools listed here and internal runtime tools")
// Placeholders use generic namespace/tool/field names only - no fabricated real tools
// and no real catalog tools cherry-picked into example lines.
expect(instructions).toContain("`return { <field>: data.<field> }`")
expect(instructions).toContain("`const result = await tools.<namespace>.<tool>(input)`")
expect(instructions).toContain("Return only the fields you need from structured results")
expect(instructions).toContain("check that it is a non-null object and not an array")
expect(instructions).not.toContain("result.<field>")
expect(instructions).not.toContain("data.<field>")
expect(instructions).not.toContain("total_count")
expect(instructions).not.toContain("list_issues")
expect(instructions).not.toContain("tools.orders.lookup({")
@ -621,36 +627,35 @@ describe("CodeMode public contract", () => {
expect(instructions).toContain("1. Pick a tool from the list under `## Available tools`")
expect(instructions).not.toContain("Browse one namespace")
const partial = CodeMode.make({ tools, discovery: { maxInlineCatalogTokens: 0 } }).instructions()
const partial = CodeMode.make({ tools, discovery: { catalogBudget: 0 } }).instructions()
// PARTIAL: the workflow starts with search (with query-style guidance that is clearly
// a query string, never a tool name) and the browse-namespace rule appears.
expect(partial).toContain(
'1. If the exact signature is not listed below, first search: `const { items } = await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
'1. If needed, discover tools: `return await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
)
expect(partial).toContain("In the next execution, copy a returned path exactly")
expect(partial).toContain(
"Only tools listed here or returned by `tools.$codemode.search` are available inside `tools`",
"Only Code Mode tools listed here or returned by `tools.$codemode.search` and internal runtime tools",
)
expect(partial).toContain(
'- Browse one namespace: `await tools.$codemode.search({ query: "", namespace: "<name>" })`.',
)
expect(partial).toContain("repeat the same search with `offset: next.offset`")
expect(partial).toContain(" limit?: number,\n offset?: number,")
expect(partial).not.toContain("total_count")
expect(partial).not.toContain("tools.orders.lookup({")
})
test("the syntax section names what is unusual or missing, not an allowlist", () => {
test("the language section describes the restricted runtime without overclaiming", () => {
const instructions = CodeMode.make({ tools }).instructions()
// Models already know JavaScript; the section leads with that.
expect(instructions).toContain("Standard modern JavaScript works")
expect(instructions).toContain("TypeScript type annotations are allowed and stripped before execution")
// The not-supported list is derived from (and verified against) the interpreter.
expect(instructions).toContain("Not supported")
for (const missing of ["classes", "generators", "for await...of", ".then/.catch/.finally"]) {
expect(instructions).toContain("restricted JavaScript language for calling tools")
expect(instructions).toContain("not a general-purpose runtime")
expect(instructions).not.toContain("Standard modern JavaScript works")
expect(instructions).not.toContain("TypeScript type annotations")
for (const missing of ["Modules/imports", "classes", "generators", "fetch", "promise chaining"]) {
expect(instructions).toContain(missing)
}
// Implemented by the DSL-expansion pass, so no longer listed as missing.
expect(instructions).not.toContain("instanceof Error")
expect(instructions).not.toContain("splice")
// The data-boundary note survives.
expect(instructions).toContain("Use Code Mode tools for external operations")
expect(instructions).toContain(
"Dates serialize to ISO strings at data boundaries; Map/Set/RegExp serialize to `{}`.",
)
@ -660,7 +665,7 @@ describe("CodeMode public contract", () => {
const runtime = CodeMode.make({})
const instructions = runtime.instructions()
expect(instructions).toContain("No tools are currently available.")
expect(instructions).toContain("## Syntax")
expect(instructions).toContain("## Language")
expect(instructions).toContain("## Available tools")
expect(instructions).not.toContain("## Workflow")
expect(instructions).not.toContain("## Rules")
@ -682,7 +687,7 @@ describe("CodeMode public contract", () => {
})
const runtime = CodeMode.make({
tools: { thread: { uploadFile: upload, generateImage: generate }, orders: { lookup } },
discovery: { maxInlineCatalogTokens: 0 },
discovery: { catalogBudget: 0 },
})
expect(runtime.instructions()).toContain(
"Available tools (PARTIAL - 0 of 3 shown; find the rest with tools.$codemode.search)",
@ -707,15 +712,16 @@ describe("CodeMode public contract", () => {
{
path: "tools.thread.uploadFile",
description: "Upload one readable local file to the current Discord thread",
signature: "tools.thread.uploadFile(input: {\n path: string\n}): Promise<{\n sent: boolean\n}>",
signature: "tools.thread.uploadFile(input: {\n path: string,\n}): Promise<{\n sent: boolean,\n}>",
},
{
path: "tools.thread.generateImage",
description: "Generate an image and upload it to the current Discord thread",
signature: "tools.thread.generateImage(input: {\n prompt: string\n}): Promise<{\n sent: boolean\n}>",
signature: "tools.thread.generateImage(input: {\n prompt: string,\n}): Promise<{\n sent: boolean,\n}>",
},
],
total: 2,
remaining: 0,
next: null,
})
expect(result.toolCalls).toStrictEqual([{ name: "$codemode.search" }])
@ -761,9 +767,14 @@ describe("CodeMode public contract", () => {
const browse = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({})`))
expect(browse.ok).toBe(true)
if (browse.ok) {
const value = browse.value as { items: Array<{ path: string }>; total: number }
const value = browse.value as {
items: Array<{ path: string }>
remaining: number
next: { offset: number } | null
}
expect(value.items).toHaveLength(10)
expect(value.total).toBe(14)
expect(value.remaining).toBe(4)
expect(value.next).toStrictEqual({ offset: 10 })
}
for (const query of ["many.tool13", "tools.many.tool13"]) {
@ -777,10 +788,11 @@ describe("CodeMode public contract", () => {
{
path: "tools.many.tool13",
description: "Numbered tool 13",
signature: "tools.many.tool13(input: {\n id: string\n}): Promise<string>",
signature: "tools.many.tool13(input: {\n id: string,\n}): Promise<string>",
},
],
total: 1,
remaining: 0,
next: null,
})
}
}
@ -807,8 +819,8 @@ describe("CodeMode public contract", () => {
)
expect(browse.ok).toBe(true)
if (browse.ok) {
const value = browse.value as { items: Array<{ path: string }>; total: number }
expect(value.total).toBe(2)
const value = browse.value as { items: Array<{ path: string }>; remaining: number }
expect(value.remaining).toBe(0)
expect(value.items.map((item) => item.path)).toStrictEqual([
"tools.github.create_issue",
"tools.github.list_issues",
@ -821,8 +833,8 @@ describe("CodeMode public contract", () => {
)
expect(scoped.ok).toBe(true)
if (scoped.ok) {
const value = scoped.value as { items: Array<{ path: string }>; total: number }
expect(value.total).toBe(1)
const value = scoped.value as { items: Array<{ path: string }>; remaining: number }
expect(value.remaining).toBe(0)
expect(value.items[0]?.path).toBe("tools.linear.list_issues")
}
@ -858,8 +870,8 @@ describe("CodeMode public contract", () => {
)
expect(byParameter.ok).toBe(true)
if (byParameter.ok) {
const value = byParameter.value as { items: Array<{ path: string }>; total: number }
expect(value.total).toBe(1)
const value = byParameter.value as { items: Array<{ path: string }>; remaining: number }
expect(value.remaining).toBe(0)
expect(value.items[0]?.path).toBe("tools.files.upload")
}
@ -869,8 +881,8 @@ describe("CodeMode public contract", () => {
)
expect(bySubstring.ok).toBe(true)
if (bySubstring.ok) {
const value = bySubstring.value as { items: Array<{ path: string }>; total: number }
expect(value.total).toBe(1)
const value = bySubstring.value as { items: Array<{ path: string }>; remaining: number }
expect(value.remaining).toBe(0)
expect(value.items[0]?.path).toBe("tools.files.upload")
}
})
@ -898,8 +910,8 @@ describe("CodeMode public contract", () => {
)
expect(plural.ok).toBe(true)
if (plural.ok) {
const value = plural.value as { items: Array<{ path: string }>; total: number }
expect(value.total).toBe(1)
const value = plural.value as { items: Array<{ path: string }>; remaining: number }
expect(value.remaining).toBe(0)
expect(value.items[0]?.path).toBe("tools.tracker.fetch_all")
}
@ -907,8 +919,8 @@ describe("CodeMode public contract", () => {
const ranked = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({ query: "issues" })`))
expect(ranked.ok).toBe(true)
if (ranked.ok) {
const value = ranked.value as { items: Array<{ path: string }>; total: number }
expect(value.total).toBe(2)
const value = ranked.value as { items: Array<{ path: string }>; remaining: number }
expect(value.remaining).toBe(0)
expect(value.items.map((item) => item.path)).toStrictEqual([
"tools.github.list_issues",
"tools.tracker.fetch_all",
@ -934,13 +946,33 @@ describe("CodeMode public contract", () => {
const browse = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({})`))
expect(browse.ok).toBe(true)
if (browse.ok) {
const value = browse.value as { items: Array<{ path: string }>; total: number }
const value = browse.value as { items: Array<{ path: string }>; remaining: number; next: unknown }
expect(value.items.map((item) => item.path)).toStrictEqual([
"tools.alpha.aardvark",
"tools.alpha.beta",
"tools.zeta.last",
])
expect(value.remaining).toBe(0)
expect(value.next).toBeNull()
}
const middle = await Effect.runPromise(
runtime.execute(`return await tools.$codemode.search({ limit: 1, offset: 1 })`),
)
expect(middle.ok).toBe(true)
if (middle.ok) {
expect(middle.value).toMatchObject({
items: [{ path: "tools.alpha.beta" }],
remaining: 1,
next: { offset: 2 },
})
}
const exhausted = await Effect.runPromise(
runtime.execute(`return await tools.$codemode.search({ limit: 1, offset: 3 })`),
)
expect(exhausted.ok).toBe(true)
if (exhausted.ok) expect(exhausted.value).toStrictEqual({ items: [], remaining: 0, next: null })
})
test("inlines round-robin across namespaces so one expensive namespace cannot starve the rest", () => {
@ -965,7 +997,7 @@ describe("CodeMode public contract", () => {
// other namespaces from inlining (beta already got its line in the same round).
const runtime = CodeMode.make({
tools: { alpha: { cheap, expensive }, beta: { cheap } },
discovery: { maxInlineCatalogTokens: 40 },
discovery: { catalogBudget: 40 },
})
const instructions = runtime.instructions()
@ -973,14 +1005,38 @@ describe("CodeMode public contract", () => {
"Available tools (PARTIAL - 2 of 3 shown; find the rest with tools.$codemode.search)",
)
expect(instructions).toContain("- alpha (2 tools, 1 shown)")
expect(instructions).toContain(" - tools.alpha.cheap(input: { q: string }): Promise<string> // Cheap")
expect(instructions).toContain(" - tools.alpha.cheap(input: {\n q: string,\n}): Promise<string> // Cheap")
expect(instructions).not.toContain("tools.alpha.expensive(")
// Fully shown namespaces read cleanly (no "shown" annotation).
expect(instructions).toContain("- beta (1 tool)")
expect(instructions).toContain(" - tools.beta.cheap(input: { q: string }): Promise<string> // Cheap")
expect(instructions).toContain(" - tools.beta.cheap(input: {\n q: string,\n}): Promise<string> // Cheap")
expect(instructions).toMatch(/\$codemode\.search/)
})
test("charges inline JSDoc against the catalog token budget", () => {
const documented = Tool.make({
description: "Look up a record",
input: {
type: "object",
properties: {
id: { type: "string", description: "A detailed identifier description. ".repeat(20) },
},
required: ["id"],
} as const,
run: () => Effect.succeed("ok"),
})
const runtime = CodeMode.make({
tools: { records: { lookup: documented } },
discovery: { catalogBudget: 40 },
})
expect(runtime.catalog()[0]?.signature).toContain("/** A detailed identifier description.")
expect(runtime.instructions()).toContain(
"Available tools (PARTIAL - 0 of 1 shown; find the rest with tools.$codemode.search)",
)
expect(runtime.instructions()).not.toContain("tools.records.lookup(input:")
})
test("decodes tool input and output before exposing either side", async () => {
const observed: Array<unknown> = []
const transformed = Tool.make({
@ -1031,17 +1087,27 @@ describe("CodeMode public contract", () => {
expect(() => CodeMode.execute({ code: "return 1", limits: { maxToolCalls: -1 } })).toThrow(RangeError)
expect(() => CodeMode.execute({ code: "return 1", limits: { maxOutputBytes: -1 } })).toThrow(RangeError)
expect(() => CodeMode.make({ tools, discovery: { maxInlineCatalogTokens: -1 } })).toThrow(RangeError)
expect(() => CodeMode.make({ tools, discovery: { catalogBudget: -1 } })).toThrow(RangeError)
const result = await Effect.runPromise(
CodeMode.make({
tools,
discovery: { maxInlineCatalogTokens: 0 },
discovery: { catalogBudget: 0 },
}).execute(`return await tools.$codemode.search({ query: "order", limit: 0.5 })`),
)
expect(result.ok).toBe(false)
if (result.ok) return
expect(result.error.kind).toBe("InvalidToolInput")
for (const offset of [-1, 0.5, Number.MAX_SAFE_INTEGER + 1, "1"]) {
const invalidOffset = await Effect.runPromise(
CodeMode.make({ tools }).execute(
`return await tools.$codemode.search({ query: "order", offset: ${JSON.stringify(offset)} })`,
),
)
expect(invalidOffset.ok).toBe(false)
if (!invalidOffset.ok) expect(invalidOffset.error.kind).toBe("InvalidToolInput")
}
})
test("enforces the tool-call limit as a diagnostic", async () => {

8
packages/codemode/test/enumeration.test.ts
View file

@ -41,7 +41,7 @@ describe("Object.keys over tool references", () => {
const namespaces = Object.keys(tools)
return { namespaces, count: namespaces.length }
`),
).toEqual({ namespaces: ["github", "memory", "playwright"], count: 3 })
).toEqual({ namespaces: ["github", "memory", "playwright", "$codemode"], count: 4 })
})
test("enumerates tool names at a nested namespace", async () => {
@ -52,7 +52,7 @@ describe("Object.keys over tool references", () => {
expect(await value(`return Object.keys(tools.github.list_issues)`)).toEqual([])
})
test("the virtual discovery namespace enumerates its callable surface", async () => {
test("the internal discovery namespace enumerates its callable surface", async () => {
expect(await value(`return Object.keys(tools.$codemode)`)).toEqual(["search"])
})
@ -137,7 +137,7 @@ describe("for...in", () => {
).toBe("only")
})
test("enumerates namespaces and tools from the host tool tree", async () => {
test("enumerates namespaces and tools from the callable tool tree", async () => {
expect(
await value(`
const names = []
@ -146,7 +146,7 @@ describe("for...in", () => {
}
return names
`),
).toEqual(["github.list_issues", "github.get_issue", "memory.search", "playwright.navigate"])
).toEqual(["github.list_issues", "github.get_issue", "memory.search", "playwright.navigate", "$codemode.search"])
})
test("unsupported values fail with a hint at for...of and Object.keys", async () => {

91
packages/codemode/test/signature.test.ts
View file

@ -40,21 +40,21 @@ describe("pretty signature rendering", () => {
[
"{",
" /** Repository owner */",
" owner: string",
" owner: string,",
" /** Cursor from the previous response's pageInfo */",
" after?: string",
" after?: string,",
" /**",
" * Results per page",
" * @default 30",
" */",
" perPage?: number",
" perPage?: number,",
" /**",
" * Filter by labels",
" * @minItems 1",
" * @maxItems 10",
" */",
" labels?: Array<string>",
' state?: "open" | "closed"',
" labels?: Array<string>,",
' state?: "open" | "closed",',
"}",
].join("\n"),
)
@ -83,18 +83,24 @@ describe("pretty signature rendering", () => {
true,
)
expect(pretty).toBe(
["{", " /** Search filter */", " filter?: {", " /** Issue state */", " state?: string", " }", "}"].join(
"\n",
),
[
"{",
" /** Search filter */",
" filter?: {",
" /** Issue state */",
" state?: string,",
" },",
"}",
].join("\n"),
)
})
test("Effect Schema annotations become JSDoc on input and output fields", () => {
expect(inputTypeScript(lookupOrder, true)).toBe(
["{", " /** Order identifier */", " id: string", " verbose?: boolean", "}"].join("\n"),
["{", " /** Order identifier */", " id: string,", " verbose?: boolean,", "}"].join("\n"),
)
expect(outputTypeScript(lookupOrder, true)).toBe(
["{", " /** Current order status */", " status: string", "}"].join("\n"),
["{", " /** Current order status */", " status: string,", "}"].join("\n"),
)
})
@ -129,7 +135,7 @@ describe("pretty signature rendering", () => {
{ type: "object", properties: { size: { type: "number", default: 1n } } },
true,
)
expect(pretty).toBe(["{", " size?: number", "}"].join("\n"))
expect(pretty).toBe(["{", " size?: number,", "}"].join("\n"))
})
test("neutralizes */ inside descriptions so nothing closes the comment early", () => {
@ -150,7 +156,7 @@ describe("pretty signature rendering", () => {
true,
)
expect(pretty).toBe(
["{", " /**", " * First line", " *", " * Second line", " */", " query?: string", "}"].join("\n"),
["{", " /**", " * First line", " *", " * Second line", " */", " query?: string,", "}"].join("\n"),
)
})
@ -235,12 +241,12 @@ describe("non-identifier property names render as quoted keys", () => {
expect(jsonSchemaToTypeScript(rawSchema, true)).toBe(
[
"{",
' "123"?: number',
' "foo-bar"?: string',
' "@type": string',
' "123"?: number,',
' "foo-bar"?: string,',
' "@type": string,',
" /** Dotted name */",
' "x.y"?: number',
" plain?: boolean",
' "x.y"?: number,',
" plain?: boolean,",
"}",
].join("\n"),
)
@ -259,7 +265,7 @@ describe("non-identifier property names render as quoted keys", () => {
})
expect(inputTypeScript(tool)).toContain('"foo-bar"?: string')
expect(outputTypeScript(tool)).toBe('{ "content-type": string }')
expect(outputTypeScript(tool, true)).toBe(["{", ' "content-type": string', "}"].join("\n"))
expect(outputTypeScript(tool, true)).toBe(["{", ' "content-type": string,', "}"].join("\n"))
})
test("Effect Schema structs with non-identifier field names quote too", () => {
@ -269,7 +275,7 @@ describe("non-identifier property names render as quoted keys", () => {
run: () => Effect.succeed(null),
})
expect(inputTypeScript(tool)).toBe('{ "foo-bar": string; plain?: number }')
expect(inputTypeScript(tool, true)).toBe(["{", ' "foo-bar": string', " plain?: number", "}"].join("\n"))
expect(inputTypeScript(tool, true)).toBe(["{", ' "foo-bar": string,', " plain?: number,", "}"].join("\n"))
})
})
@ -308,10 +314,7 @@ describe("union schemas render every alternative", () => {
test("allOf renders intersections with parenthesized union members", () => {
const schema = {
allOf: [
{ type: "object", properties: { id: { type: "string" } } },
{ type: ["string", "null"] },
],
allOf: [{ type: "object", properties: { id: { type: "string" } } }, { type: ["string", "null"] }],
} as const
expect(jsonSchemaToTypeScript(schema)).toBe("{ id?: string } & (string | null)")
})
@ -321,7 +324,9 @@ describe("union schemas render every alternative", () => {
"unknown",
)
expect(
jsonSchemaToTypeScript({ allOf: [{ type: "string" }, { allOf: [{ $ref: "https://example.com/external.json" }] }] }),
jsonSchemaToTypeScript({
allOf: [{ type: "string" }, { allOf: [{ $ref: "https://example.com/external.json" }] }],
}),
).toBe("unknown")
expect(
jsonSchemaToTypeScript({
@ -333,7 +338,7 @@ describe("union schemas render every alternative", () => {
})
})
describe("pretty signatures in search results", () => {
describe("JSDoc signatures in catalogs and search results", () => {
const runtime = CodeMode.make({ tools: { github: { list_issues: listIssues }, orders: { lookup: lookupOrder } } })
const search = async (query: string) => {
@ -342,7 +347,7 @@ describe("pretty signatures in search results", () => {
)
expect(result.ok).toBe(true)
if (!result.ok) throw new Error("search failed")
return result.value as { items: Array<{ path: string; signature: string }>; total: number }
return result.value as { items: Array<{ path: string; signature: string }>; remaining: number }
}
test("a raw JSON Schema (MCP-style) tool's result signature carries field JSDoc and tags", async () => {
@ -352,21 +357,21 @@ describe("pretty signatures in search results", () => {
[
"tools.github.list_issues(input: {",
" /** Repository owner */",
" owner: string",
" owner: string,",
" /** Cursor from the previous response's pageInfo */",
" after?: string",
" after?: string,",
" /**",
" * Results per page",
" * @default 30",
" */",
" perPage?: number",
" perPage?: number,",
" /**",
" * Filter by labels",
" * @minItems 1",
" * @maxItems 10",
" */",
" labels?: Array<string>",
' state?: "open" | "closed"',
" labels?: Array<string>,",
' state?: "open" | "closed",',
"}): Promise<unknown>",
].join("\n"),
)
@ -380,26 +385,26 @@ describe("pretty signatures in search results", () => {
[
"tools.orders.lookup(input: {",
" /** Order identifier */",
" id: string",
" verbose?: boolean",
" id: string,",
" verbose?: boolean,",
"}): Promise<{",
" /** Current order status */",
" status: string",
" status: string,",
"}>",
].join("\n"),
)
}
})
test("the inline catalog line for the same tool stays single-line compact", () => {
test("the inline catalog uses the same JSDoc signatures", async () => {
const instructions = runtime.instructions()
expect(instructions).toContain(
' - tools.github.list_issues(input: { owner: string; after?: string; perPage?: number; labels?: Array<string>; state?: "open" | "closed" }): Promise<unknown> // List issues in a repository',
)
expect(instructions).toContain(
" - tools.orders.lookup(input: { id: string; verbose?: boolean }): Promise<{ status: string }> // Look up an order",
)
expect(instructions).not.toContain("/**")
const github = (await search("list issues repository")).items.find(
({ path }) => path === "tools.github.list_issues",
)!
const orders = (await search("look up order")).items.find(({ path }) => path === "tools.orders.lookup")!
expect(instructions).toContain(` - ${github.signature} // List issues in a repository`)
expect(instructions).toContain(` - ${orders.signature} // Look up an order`)
expect(instructions).toContain("/** Repository owner */")
})
})
@ -422,7 +427,7 @@ describe("non-identifier tool paths", () => {
const instructions = runtime.instructions()
expect(instructions).toContain(
'tools.context7["resolve-library-id"](input: { query: string; libraryName: string }): Promise<unknown>',
'tools.context7["resolve-library-id"](input: {\n query: string,\n libraryName: string,\n}): Promise<unknown>',
)
expect(instructions).toContain("Do not infer or normalize tool names")
expect(instructions).toContain("bracket notation and quotes are part of the path")

2
packages/core/test/mcp.test.ts
View file

@ -183,7 +183,7 @@ it.effect("advertises MCP output schemas to Code Mode", () =>
yield* waitForTool(registry, "execute")
const execute = (yield* toolDefinitions(registry)).find((tool) => tool.name === "execute")
expect(execute?.description).toContain("tools.demo.search(input: {}): Promise<{ ok: boolean }>")
expect(execute?.description).toContain("tools.demo.search(input: {}): Promise<{\n ok: boolean,\n}>")
}),
)