diff --git a/packages/codemode/codemode.md b/packages/codemode/codemode.md index 2ced2df49a1..155f3cd13b9 100644 --- a/packages/codemode/codemode.md +++ b/packages/codemode/codemode.md @@ -121,9 +121,14 @@ closing. Timeout and external interruption cancel immediately instead. ### Confirmed defects -- [ ] Return rejected promises for invalid `Promise.all`/`allSettled`/`race` inputs instead of throwing during the call. - [ ] Align handler callability with the values CodeMode reports as functions, or document the narrower callback allowlist. For example, unsupported constructor-like callables are currently treated as absent handlers. +- [ ] Run synchronous Promise reactions to completion. Reactions currently start in FIFO order but can interleave when + Effect auto-yields a long synchronous handler. +- [ ] Give `Promise.race` settlement the same reaction turn as native promises. Its result currently settles before an + independently queued reaction in several Test262 ordering cases. +- [ ] Drain queued reactions before classifying rejected sources as unhandled. A handler attached by an already-queued + reaction can currently lose to execution-boundary rejection reporting. ### Deliberate deviations and open decisions @@ -152,6 +157,7 @@ closing. Timeout and external interruption cancel immediately instead. - Promise reactions and plain, pending, or settled `await` continuations start in deterministic FIFO order. Nested reactions preserve enqueue order, while an async reaction can suspend without blocking the next queued reaction. - Awaiting the same promise twice settles it once. +- Sparse combinator inputs consume holes as explicit `undefined` values. ### Missing coverage @@ -182,7 +188,7 @@ represent accurately rather than guessing semantics. | Keep an owned tree-walking interpreter. | The product need is bounded tool orchestration, not arbitrary JavaScript. Owning the language surface keeps authority and behavior explicit. | | Treat schemas as the model-facing interface. | Signatures drive correct calls; Effect Schema also provides the runtime validation boundary, while JSON Schema supports adapter interoperability. | | Keep authority host-owned. | CodeMode can only confine programs to supplied tools. The host chooses those tools, and each tool enforces its own authorization and side-effect policy. | -| Use progressive catalog disclosure plus search. | Large tool sets should not consume the prompt, but every namespace must remain discoverable and speculative search calls should remain valid. | +| Use progressive catalog disclosure plus search. | Large tool sets should not consume the prompt, but every namespace must remain discoverable and speculative search calls must remain valid. | | Start tool promises eagerly and supervise them. | This preserves normal call-time parallelism while giving each call run-once settlement and interruption safety. | | Keep files outside the sandbox value space. | Models should compose structured data without routing binary payloads through generated code or context. | | Treat `execute` as the model-facing invocation boundary. | Nested calls are implementation details of one orchestration program. Reusing the outer context and bounding only the final result preserves complete intermediate data without inventing durable child-call identities. | diff --git a/packages/codemode/src/interpreter/runtime.ts b/packages/codemode/src/interpreter/runtime.ts index 429bb4f5eaa..1b98e3d9035 100644 --- a/packages/codemode/src/interpreter/runtime.ts +++ b/packages/codemode/src/interpreter/runtime.ts @@ -2270,11 +2270,17 @@ class Interpreter { }) } - const items = Array.isArray(args[0]) ? args[0] : spreadItems(args[0]) + // Promise combinators consume arrays through their iterator, which visits holes as + // explicit undefined values instead of preserving sparse positions like Array.map. + const items = Array.isArray(args[0]) ? [...args[0]] : spreadItems(args[0]) if (items === undefined) { - throw new InterpreterRuntimeError( - `Promise.${ref.name} expects an array of promises or plain values (e.g. Promise.${ref.name}(items.map((item) => tools.ns.tool(item)))).`, - node, + return this.createPromise( + Effect.fail( + new InterpreterRuntimeError( + `Promise.${ref.name} expects an array of promises or plain values (e.g. Promise.${ref.name}(items.map((item) => tools.ns.tool(item)))).`, + node, + ).as("TypeError"), + ), ) } diff --git a/packages/codemode/test/promise-test262.test.ts b/packages/codemode/test/promise-test262.test.ts new file mode 100644 index 00000000000..27306a071eb --- /dev/null +++ b/packages/codemode/test/promise-test262.test.ts @@ -0,0 +1,334 @@ +/* + * Portions adapted from Test262 at revision 250f204f23a9249ff204be2baec29600faae7b75: + * - test/built-ins/Promise/all/S25.4.4.1_A1.1_T1.js + * - test/built-ins/Promise/all/S25.4.4.1_A2.1_T1.js + * - test/built-ins/Promise/all/S25.4.4.1_A2.3_T1.js + * - test/built-ins/Promise/all/S25.4.4.1_A2.3_T2.js + * - test/built-ins/Promise/all/S25.4.4.1_A2.3_T3.js + * - test/built-ins/Promise/all/S25.4.4.1_A7.1_T1.js + * - test/built-ins/Promise/all/S25.4.4.1_A8.2_T1.js + * - test/built-ins/Promise/all/S25.4.4.1_A8.2_T2.js + * - test/built-ins/Promise/all/iter-arg-is-number-reject.js + * - test/built-ins/Promise/all/resolve-non-thenable.js + * - test/built-ins/Promise/allSettled/is-function.js + * - test/built-ins/Promise/allSettled/returns-promise.js + * - test/built-ins/Promise/allSettled/resolves-empty-array.js + * - test/built-ins/Promise/allSettled/resolves-to-array.js + * - test/built-ins/Promise/allSettled/resolved-all-fulfilled.js + * - test/built-ins/Promise/allSettled/resolved-all-mixed.js + * - test/built-ins/Promise/allSettled/resolved-all-rejected.js + * - test/built-ins/Promise/allSettled/iter-arg-is-number-reject.js + * - test/built-ins/Promise/allSettled/resolve-non-thenable.js + * - test/built-ins/Promise/race/S25.4.4.3_A1.1_T1.js + * - test/built-ins/Promise/race/S25.4.4.3_A7.1_T1.js + * - test/built-ins/Promise/race/S25.4.4.3_A7.1_T2.js + * - test/built-ins/Promise/race/S25.4.4.3_A7.1_T3.js + * - test/built-ins/Promise/race/S25.4.4.3_A7.2_T1.js + * - test/built-ins/Promise/race/S25.4.4.3_A7.3_T1.js + * - test/built-ins/Promise/race/S25.4.4.3_A7.3_T2.js + * - test/built-ins/Promise/race/iter-arg-is-number-reject.js + * - test/built-ins/Promise/resolve/S25.4.4.5_A1.1_T1.js + * - test/built-ins/Promise/resolve/S25.4.4.5_A2.1_T1.js + * - test/built-ins/Promise/resolve/resolve-non-obj.js + * - test/built-ins/Promise/resolve/resolve-non-thenable.js + * - test/built-ins/Promise/reject/S25.4.4.4_A1.1_T1.js + * - test/built-ins/Promise/reject/S25.4.4.4_A2.1_T1.js + * - test/built-ins/Array/from/from-array.js + * - test/language/statements/async-function/declaration-returns-promise.js + * - test/language/statements/async-function/evaluation-body-that-returns.js + * - test/language/statements/async-function/evaluation-body-that-returns-after-await.js + * - test/language/statements/async-function/evaluation-body-that-throws.js + * - test/language/statements/async-function/evaluation-body-that-throws-after-await.js + * + * Copyright 2014 Cubane Canada, Inc. All rights reserved. + * Copyright (C) 2016 the V8 project authors. All rights reserved. + * Copyright (C) 2019 Leo Balter. All rights reserved. + * Copyright (C) 2018 Rick Waldron. All rights reserved. + * Copyright 2015 Microsoft Corporation. All rights reserved. + * Copyright 2016 Microsoft, Inc. All rights reserved. + * Test262 portions are governed by the BSD license in LICENSE.test262. + */ +import { describe, expect, test } from "bun:test" +import { Effect } from "effect" +import { CodeMode } from "../src/index.js" + +const value = async (code: string) => { + const result = await Effect.runPromise(CodeMode.execute({ code, tools: {} })) + if (!result.ok) throw new Error(`expected success, got ${result.error.kind}: ${result.error.message}`) + return result.value +} + +describe("Test262 Promise static adaptations", () => { + test("Promise statics are callable", async () => { + expect( + await value( + `return [typeof Promise.all, typeof Promise.allSettled, typeof Promise.race, typeof Promise.resolve, typeof Promise.reject]`, + ), + ).toEqual(["function", "function", "function", "function", "function"]) + }) + + test("invalid combinator inputs return rejected TypeError promises", async () => { + expect( + await value(` + const observe = async (run) => { + let returned = false + try { + const promise = run() + returned = promise instanceof Promise + await promise + return [returned, "fulfilled"] + } catch (error) { + return [returned, error.name] + } + } + return await Promise.all([ + observe(() => Promise.all(42)), + observe(() => Promise.allSettled(42)), + observe(() => Promise.race(42)), + ]) + `), + ).toEqual([ + [true, "TypeError"], + [true, "TypeError"], + [true, "TypeError"], + ]) + }) + + test("Promise.all returns a promise and creates a fresh empty array", async () => { + expect( + await value(` + const input = [] + const promise = Promise.all(input) + const result = await promise + return [promise instanceof Promise, result instanceof Array, result.length, result !== input] + `), + ).toEqual([true, true, 0, true]) + }) + + test("Promise.all preserves values and input order", async () => { + expect( + await value(` + const first = { id: 1 } + const second = { id: 2 } + const result = await Promise.all([Promise.resolve(3), first, Promise.resolve(second)]) + return [result.length, result[0], result[1] === first, result[2] === second] + `), + ).toEqual([3, 3, true, true]) + }) + + test("Promise.all adopts rejection from either input position", async () => { + expect( + await value(` + const observe = async (promise) => { + try { await promise; return "fulfilled" } catch (reason) { return reason } + } + return await Promise.all([ + observe(Promise.all([Promise.reject(1), Promise.resolve(2)])), + observe(Promise.all([Promise.resolve(1), Promise.reject(2)])), + ]) + `), + ).toEqual([1, 2]) + }) + + test("Promise.all treats sparse positions as undefined values", async () => { + expect( + await value(` + const input = [] + input[1] = 1 + const result = await Promise.all(input) + return [result.length, result[0] === undefined, result[1]] + `), + ).toEqual([2, true, 1]) + }) + + test("Promise.allSettled returns a promise and a fresh array", async () => { + expect( + await value(` + const input = [] + const promise = Promise.allSettled(input) + const result = await promise + return [promise instanceof Promise, result instanceof Array, result.length, result !== input] + `), + ).toEqual([true, true, 0, true]) + }) + + test("Promise.allSettled preserves order and settlement shapes", async () => { + expect( + await value(` + const reason = { id: 4 } + const result = await Promise.allSettled([ + Promise.resolve(1), + Promise.reject(2), + 3, + Promise.reject(reason), + ]) + return [result, result.map((item) => Object.keys(item))] + `), + ).toEqual([ + [ + { status: "fulfilled", value: 1 }, + { status: "rejected", reason: 2 }, + { status: "fulfilled", value: 3 }, + { status: "rejected", reason: { id: 4 } }, + ], + [ + ["status", "value"], + ["status", "reason"], + ["status", "value"], + ["status", "reason"], + ], + ]) + }) + + test("Promise.allSettled preserves input order across settlement timing", async () => { + expect( + await value(` + const slow = async () => { await Promise.resolve(); return "slow" } + return await Promise.allSettled([slow(), Promise.resolve("fast")]) + `), + ).toEqual([ + { status: "fulfilled", value: "slow" }, + { status: "fulfilled", value: "fast" }, + ]) + }) + + test("Promise.allSettled adopts a non-thenable object", async () => { + expect( + await value(` + const object = { id: 1 } + const result = await Promise.allSettled([object]) + return [result, result[0].value === object] + `), + ).toEqual([[{ status: "fulfilled", value: { id: 1 } }], true]) + }) + + test("Promise.allSettled treats sparse positions as undefined values", async () => { + expect( + await value(` + const input = [] + input[1] = 1 + const result = await Promise.allSettled(input) + return [result.length, result[0].status, result[0].value === undefined, result[1]] + `), + ).toEqual([2, "fulfilled", true, { status: "fulfilled", value: 1 }]) + }) + + test("Promise.race returns a promise and settles from its first input", async () => { + expect( + await value(` + const rejected = Promise.reject(2) + const fulfilled = Promise.resolve(1) + const promise = Promise.race([fulfilled, rejected]) + return [promise instanceof Promise, await promise] + `), + ).toEqual([true, 1]) + }) + + test("Promise.race preserves primitive fulfillment and rejection", async () => { + expect( + await value(` + const fulfilled = await Promise.race([23]) + let rejected + try { await Promise.race([Promise.reject(7)]) } catch (reason) { rejected = reason } + return [fulfilled, rejected] + `), + ).toEqual([23, 7]) + }) + + test("Promise.race observes later fulfillment and rejection", async () => { + expect( + await value(` + const slow = async () => { await Promise.resolve(); await Promise.resolve(); return 1 } + const rejectSoon = async () => { await Promise.resolve(); throw 2 } + try { await Promise.race([slow(), rejectSoon()]); return "fulfilled" } catch (reason) { return reason } + `), + ).toBe(2) + }) + + test("Promise.race selects fulfilled and rejected inputs by settlement order", async () => { + expect( + await value(` + const delayed = async (value) => { await Promise.resolve(); return value } + const first = await Promise.race([Promise.resolve(1), Promise.resolve(2)]) + const second = await Promise.race([Promise.resolve(1), delayed(9)]) + const third = await Promise.race([delayed(9), Promise.resolve(2)]) + let fourth + try { await Promise.race([Promise.reject(1), Promise.resolve(2)]) } catch (reason) { fourth = reason } + return [first, second, third, fourth] + `), + ).toEqual([1, 1, 2, 1]) + }) + + test("Promise.race treats a sparse first position as undefined", async () => { + expect( + await value(` + const input = [] + input[1] = 1 + return (await Promise.race(input)) === undefined + `), + ).toBe(true) + }) + + test("Promise.resolve adopts values and nested promises", async () => { + expect( + await value(` + const object = { id: 1 } + return [ + await Promise.resolve(42), + await Promise.resolve(Promise.resolve("nested")), + (await Promise.resolve(object)) === object, + ] + `), + ).toEqual([42, "nested", true]) + }) + + test("Promise.resolve preserves promise identity", async () => { + expect( + await value(` + const promise = Promise.resolve(1) + const identities = new Map([[promise, "same"]]) + return identities.get(Promise.resolve(promise)) + `), + ).toBe("same") + }) + + test("Promise.reject preserves primitive and object reasons", async () => { + expect( + await value(` + const object = { reason: true } + const reasons = [undefined, null, false, true, 0, "", 42, object] + const observe = async (reason) => { + try { await Promise.reject(reason); return false } catch (caught) { return caught === reason } + } + return await Promise.all(reasons.map(observe)) + `), + ).toEqual([true, true, true, true, true, true, true, true]) + }) +}) + +describe("Test262 async function adaptations", () => { + test("async functions return promises and adopt returned values", async () => { + expect( + await value(` + const plain = async () => 42 + const afterAwait = async () => { await Promise.resolve(); return 43 } + const first = plain() + const second = afterAwait() + return [first instanceof Promise, second instanceof Promise, await first, await second] + `), + ).toEqual([true, true, 42, 43]) + }) + + test("async functions reject for throws before and after await", async () => { + expect( + await value(` + const throwsBefore = async () => { throw 1 } + const throwsAfter = async () => { await Promise.resolve(); throw 2 } + const observe = async (promise) => { + try { await promise; return "fulfilled" } catch (reason) { return reason } + } + return await Promise.all([observe(throwsBefore()), observe(throwsAfter())]) + `), + ).toEqual([1, 2]) + }) +}) diff --git a/packages/codemode/test/promise.test.ts b/packages/codemode/test/promise.test.ts index ef96cdeca55..4b45eabe1ee 100644 --- a/packages/codemode/test/promise.test.ts +++ b/packages/codemode/test/promise.test.ts @@ -92,33 +92,14 @@ describe("first-class promise values", () => { } const first = load(1) const second = load(2) - return [first instanceof Promise, second instanceof Promise, await Promise.all([first, second])] + return await Promise.all([first, second]) `), ).toEqual([ - true, - true, - [ - [1, 1], - [2, 2], - ], + [1, 1], + [2, 2], ]) }) - test("async function errors reject instead of throwing at the call site", async () => { - expect( - await value(` - const fail = async () => { throw new Error("boom") } - const promise = fail() - try { - await promise - return "no" - } catch (error) { - return error.message - } - `), - ).toBe("boom") - }) - test("an un-awaited tool call starts eagerly, in call order, before any await", async () => { const trace = makeTrace() const result = await value( @@ -403,10 +384,6 @@ describe("Promise.all over arbitrary arrays", () => { expect(trace.maxActive).toBeLessThanOrEqual(8) }) - test("resolves the empty array", async () => { - expect(await value(`return await Promise.all([])`)).toEqual([]) - }) - test("rejects with the first failure, catchable in-program", async () => { expect( await value(` @@ -564,25 +541,10 @@ describe("Promise.race", () => { }) describe("Promise.resolve / Promise.reject", () => { - test("resolve wraps plain values and passes promises through", async () => { - expect(await value(`return await Promise.resolve(42)`)).toBe(42) - expect(await value(`return await Promise.resolve(Promise.resolve("nested"))`)).toBe("nested") + test("resolve adopts a tool-call promise", async () => { expect(await value(`return await Promise.resolve(tools.host.sleepy({ id: 3 }))`)).toBe(3) }) - test("reject produces a promise whose await throws the reason", async () => { - expect( - await value(` - try { - await Promise.reject("nope") - return "no" - } catch (e) { - return e - } - `), - ).toBe("nope") - }) - test("an abandoned rejected promise surfaces as an unhandled rejection", async () => { const diagnostic = await error(` Promise.reject(new Error("boom")) @@ -602,15 +564,12 @@ describe("Promise combinator values", () => { const settled = Promise.allSettled([Promise.resolve(3)]) const race = Promise.race([Promise.resolve(4)]) return [ - all instanceof Promise, - settled instanceof Promise, - race instanceof Promise, await all.then((values) => values.join(",")), await settled.then((values) => values[0].value), await race.then((value) => value + 1), ] `), - ).toEqual([true, true, true, "1,2", 3, 5]) + ).toEqual(["1,2", 3, 5]) }) }) diff --git a/packages/codemode/test/test262-promise.md b/packages/codemode/test/test262-promise.md new file mode 100644 index 00000000000..5c008b94142 --- /dev/null +++ b/packages/codemode/test/test262-promise.md @@ -0,0 +1,40 @@ +# Test262 Promise Coverage + +The Promise tests adapt Test262 at revision `250f204f23a9249ff204be2baec29600faae7b75`. The upstream Promise tree +contains 729 files; 459 are under the eight API directories corresponding to CodeMode's five Promise statics and three +chaining methods. The executable suite currently cites 39 distinct Promise, Array, and async-function sources. + +This is coverage of CodeMode's confined Promise surface, not a claim of ECMAScript or Test262 conformance. One upstream +file may contain both adapted and inapplicable assertions, so a cited source means only that the represented assertions +were adapted. `LICENSE.test262` contains the upstream BSD terms. + +## Covered Surface + +- `Promise.all`, `Promise.allSettled`, and `Promise.race` result types, ordering, sparse positions, mixed values, and + rejection propagation. +- `Promise.resolve` value adoption and promise identity. +- `Promise.reject` reason preservation. +- Async-function promise creation, returned-value adoption, and throws before and after `await`. + +The Promise PR's handwritten tests remain authoritative for tool-call concurrency, execution-scoped lifetime, +cancellation, output boundaries, model-safe tool failures, unhandled-rejection diagnostics, and chaining behavior. + +## Exclusions + +- `new Promise`, externally captured resolving functions, and executor semantics. +- Promise subclasses, constructors, species, realms, proxies, property descriptors, and function metadata. +- Custom thenables, poisoned `then` accessors, and arbitrary iterable or iterator-closing behavior. +- `Promise.any`, `Promise.try`, `Promise.withResolvers`, `Promise.allKeyed`, and `Promise.allSettledKeyed`. +- Exact native microtask behavior for `.then`, `.catch`, and `.finally`; those are exercised separately by the Promise + PR's chaining tests and audit. +- Exact native unhandled-rejection timing. CodeMode drains execution-owned work and reports safe diagnostics at its + execution boundary. + +## Observed Differences + +- Promise combinators accept arrays plus CodeMode's supported spreadable collections. The model guidance currently + describes arrays, so this remains a contract decision rather than conformance coverage. +- `Promise.race([])` returns an actionable error instead of a permanently pending promise, which cannot usefully finish + a bounded CodeMode execution. +- Promise identity is preserved, but CodeMode intentionally rejects general binary operators over Promise references; + identity is tested through `Map` keys instead of `===`.