test(scripts): copy only the native compiler parts each fixture uses (#163374)
Some checks are pending
ClawSweeper Dispatch / dispatch (push) Waiting to run
CodeQL / Security High (actions) (push) Waiting to run
CodeQL / Security High (process-exec-boundary) (push) Waiting to run
Control UI Locale Refresh / resolve-base (push) Waiting to run
Control UI Locale Refresh / Verify generated PR App permissions (push) Blocked by required conditions
Control UI Locale Refresh / Refresh (push) Blocked by required conditions
Control UI Locale Refresh / Commit control UI locale refresh (push) Blocked by required conditions
Docs Sync Publish Repo / sync-publish-repo (push) Waiting to run
Docs / docs (push) Waiting to run
Native App Locale Refresh / Refresh native fr (push) Blocked by required conditions
Native App Locale Refresh / Refresh native hi (push) Blocked by required conditions
Native App Locale Refresh / Refresh native id (push) Blocked by required conditions
Plugin NPM Release / Seal prepared plugin npm release (push) Blocked by required conditions
Plugin NPM Release / Trusted publisher OIDC exchange (push) Blocked by required conditions
Plugin NPM Release / approve_plugins_npm_release (push) Blocked by required conditions
CodeQL / Security High (channel-runtime-boundary) (push) Waiting to run
CodeQL / Security High (core-auth-secrets) (push) Waiting to run
CodeQL / Security High (mcp-process-tool-boundary) (push) Waiting to run
CodeQL / Security High (network-ssrf-boundary) (push) Waiting to run
CodeQL / Security High (plugin-trust-boundary) (push) Waiting to run
Native App Locale Refresh / Refresh native pl (push) Blocked by required conditions
Native App Locale Refresh / Refresh native pt-BR (push) Blocked by required conditions
Native App Locale Refresh / Refresh native ru (push) Blocked by required conditions
Native App Locale Refresh / Refresh native sv (push) Blocked by required conditions
Native App Locale Refresh / Refresh native th (push) Blocked by required conditions
Native App Locale Refresh / Refresh native tr (push) Blocked by required conditions
Native App Locale Refresh / Refresh native uk (push) Blocked by required conditions
Native App Locale Refresh / Refresh native vi (push) Blocked by required conditions
Native App Locale Refresh / Refresh native zh-CN (push) Blocked by required conditions
Native App Locale Refresh / Refresh native zh-TW (push) Blocked by required conditions
Native App Locale Refresh / Commit native locale refresh (push) Blocked by required conditions
Native App Locale Refresh / resolve-base (push) Waiting to run
Native App Locale Refresh / Verify generated PR App permissions (push) Blocked by required conditions
Native App Locale Refresh / Refresh native ar (push) Blocked by required conditions
Native App Locale Refresh / Refresh native de (push) Blocked by required conditions
Native App Locale Refresh / Refresh native es (push) Blocked by required conditions
Native App Locale Refresh / Refresh native fa (push) Blocked by required conditions
Native App Locale Refresh / Refresh native it (push) Blocked by required conditions
Native App Locale Refresh / Refresh native ja-JP (push) Blocked by required conditions
Native App Locale Refresh / Refresh native ko (push) Blocked by required conditions
Native App Locale Refresh / Refresh native nl (push) Blocked by required conditions
Node Runtime Conformance / scope (push) Waiting to run
Node Runtime Conformance / TypeScript contracts (push) Blocked by required conditions
Node Runtime Conformance / Rust workspace (push) Blocked by required conditions
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Waiting to run
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Blocked by required conditions
Plugin Init Scaffold Validation / Validate provider scaffold (push) Blocked by required conditions
Plugin Init Scaffold Validation / scope (push) Waiting to run
Plugin NPM Release / preview_plugins_npm (push) Waiting to run
Plugin NPM Release / Validate release publish approval (push) Blocked by required conditions
Plugin NPM Release / preview_plugin_pack (push) Blocked by required conditions
Plugin NPM Release / Preflight plugin npm package () (push) Blocked by required conditions
Plugin NPM Release / Publish plugin npm package () (push) Blocked by required conditions
Vitest Cache Warm / dependencies (push) Waiting to run
Plugin NPM Release / verify_plugins_npm (push) Blocked by required conditions
Vitest Cache Warm / warm (push) Waiting to run
Workflow Sanity / actionlint (push) Waiting to run
Workflow Sanity / generated-doc-baselines (push) Waiting to run

materializeNativeCompiler copied the whole native typescript package into
every fixture. 482 of its 493 files are dist/ (the typescript/unstable/*
JavaScript API) and vendor/ (that API's JSON-RPC transport), which only
toolchain identity hashing and fixture-resolved API imports read.

Per consumer:
- changed-lanes: root-test lint is boundary-free and parses config with the
  repository's own API, so the copy was never read; drop it.
- write-plugin-sdk-entry-dts / write-unified-entry-dts: createFixture already
  copies the same package with its platform binary since the TypeScript 7
  cutover; drop the redundant overlay.
- run-tsgo-worktree, ownership --version preflight, Windows long-path
  resolution: launcher-only copies (javaScriptApi: false).
- interceptor fixtures that exec a real compiler: exec the installed binary
  (resolveInstalledNativeCompiler); the lint-consumption case keeps its full
  fixture-local install and binary because preparation hashes it and admits
  default-library reads inside the checkout.
- shard concurrency: its synthetic compiler needs no install.

Build-artifact, extension-boundary, native-declaration, prepare/check
boundary, and tsdown-config fixtures keep the full copy. Copies stay on the
filtered close-on-exec cpSync path; assertions and timeouts are unchanged.
This commit is contained in:
Peter Steinberger 2026-10-02 10:13:53 -07:00 • committed by GitHub
parent 935563c38d
commit 4e3ab7babc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 52 additions and 37 deletions

View file

@ -7,18 +7,18 @@ import fs from "node:fs";
* Without a filter, Node 24 copies directories through std::filesystem, which
* ignores `mode` and opens copies without O_CLOEXEC. A sibling Vitest thread
* that forks during the copy keeps the file writable, so a later execve of it
* fails with ETXTBSY. The filter keeps new files on libuv's close-on-exec
* fails with ETXTBSY. Any filter keeps new files on libuv's close-on-exec
* copyFileSync; the clone mode does the same for overwritten files.
*/
export function copyTreeCloseOnExec(
source: string,
destination: string,
options: { dereference?: boolean } = {},
options: { dereference?: boolean; filter?: (source: string) => boolean } = {},
): void {
fs.cpSync(source, destination, {
...options,
recursive: true,
mode: fs.constants.COPYFILE_FICLONE,
filter: () => true,
filter: options.filter ?? (() => true),
});
}

View file

@ -39,7 +39,6 @@ import { findTypecheckInertPaths } from "../../scripts/lib/typecheck-inert.mts";
import { resolveTestNodeExecPath } from "../../src/test-utils/node-process.js";
import { useAutoCleanupTempDirTracker } from "../helpers/temp-dir.js";
import { createNestedGitEnv } from "../helpers/temp-repo.js";
import { materializeNativeCompiler } from "./native-boundary-fixture.js";
import { preparedScriptWrapperEnv } from "./prepared-script-wrapper.test-support.js";
const tempDirs = useAutoCleanupTempDirTracker(afterEach);
@ -823,7 +822,6 @@ describe("scripts/changed-lanes", () => {
"fails real changed-check lint for $name and passes after repair",
({ count, extension, otherPaths }) => {
const { dir, run } = createRootTestLintFixture();
materializeNativeCompiler(dir);
const targets = Array.from(
{ length: count },
(_, index) => `test/root-lint-${index}.test.${extension}`,

View file

@ -23,6 +23,7 @@ import { installDistArtifactScripts as installScripts } from "./dist-artifact-fi
import {
materializeNativeCompiler,
overrideNativeFixtureExecutable,
resolveInstalledNativeCompiler,
} from "./native-boundary-fixture.js";
import { createFixture as createDeclarationFixture } from "./tsdown-declaration-fixture.js";
@ -97,11 +98,9 @@ function createCheckout(prefix = "openclaw-dist-owner-") {
return root;
}
function installCompiler(root: string, afterEmit = "") {
function installCompiler(root: string, afterEmit = "", native = resolveInstalledNativeCompiler()) {
const launcher = path.join(root, "node_modules/.bin/tsgo");
fs.rmSync(launcher, { force: true });
const native = materializeNativeCompiler(root);
fs.unlinkSync(launcher);
const compiler = write(
root,
"node_modules/.bin/tsgo",
@ -806,7 +805,7 @@ describe.skipIf(process.platform === "win32")("dist artifact ownership", () => {
async ({ owner, unjoined }, { signal }) => {
await withProcesses(async ({ start }) => {
const root = createCheckout();
materializeNativeCompiler(root);
materializeNativeCompiler(root, { javaScriptApi: false });
const ownerPath = write(root, ".artifacts/dist-artifacts.lock/owner.json", owner);
if (unjoined) {
write(root, ".artifacts/dist-artifacts.lock/unjoined", "unverified cleanup");
@ -1056,10 +1055,10 @@ describe.skipIf(process.platform === "win32")("dist artifact ownership", () => {
await withProcesses(async ({ checkpoint, waitEvent, start }) => {
const root = createCheckout();
installScripts(root, ["run-tsgo-core-test-shards.mts", "run-tsgo.mts"], {
compiler: false,
dependencies: ["@openclaw/fs-safe"],
});
fs.unlinkSync(path.join(root, "scripts/tsx.mjs"));
fs.unlinkSync(path.join(root, "node_modules/.bin/tsgo"));
const compiler = write(
root,
"node_modules/.bin/tsgo",
@ -1097,7 +1096,8 @@ describe.skipIf(process.platform === "win32")("dist artifact ownership", () => {
}) => {
await withProcesses(async ({ checkpoint, waitEvent, start }) => {
const root = createCheckout();
installCompiler(root);
// Preparation hashes and loads the fixture's own compiler install.
installCompiler(root, "", materializeNativeCompiler(root));
// Entrypoints resolve this fixture as their checkout. SDK and plugin
// sources let the lint consumer distinguish the narrow preparation mode.
installScripts(

View file

@ -5,6 +5,19 @@ import path from "node:path";
import { copyTreeCloseOnExec } from "../helpers/close-on-exec-copy.js";
const require = createRequire(import.meta.url);
const platformPackage = `@typescript/typescript-${process.platform}-${process.arch}`;
const nativeExecutable = process.platform === "win32" ? "tsc.exe" : "tsc";
function resolvePlatformPackageDir() {
// The platform binary belongs to the native compiler's optional dependencies.
const nativeRequire = createRequire(require.resolve("typescript/package.json"));
return path.dirname(nativeRequire.resolve(`${platformPackage}/package.json`));
}
/** The installed binary, for fixtures that intercept compiler launches themselves. */
export function resolveInstalledNativeCompiler() {
return path.join(resolvePlatformPackageDir(), "lib", nativeExecutable);
}
/** Availability only; integration assertions still verify the actual kernel scope. */
export function hasSemanticTestBackend(): boolean {
@ -25,12 +38,13 @@ export function hasSemanticTestBackend(): boolean {
}
}
/** Native receipts and default libraries must belong to the fixture's own install. */
export function materializeNativeCompiler(rootDir: string) {
/**
* Native receipts and default libraries must belong to the fixture's own install.
* Fixtures that only launch the compiler can omit its JavaScript API (`dist/`,
* `vendor/`); toolchain identity and fixture-resolved API imports need it.
*/
export function materializeNativeCompiler(rootDir: string, { javaScriptApi = true } = {}) {
const root = fs.realpathSync.native(rootDir);
const platformPackage = `@typescript/typescript-${process.platform}-${process.arch}`;
// The platform binary belongs to the native compiler's optional dependencies.
const nativeRequire = createRequire(require.resolve("typescript/package.json"));
const modules = path.join(root, "node_modules");
fs.mkdirSync(modules, { recursive: true });
if (fs.realpathSync.native(modules) !== modules) {
@ -44,12 +58,20 @@ export function materializeNativeCompiler(rootDir: string) {
fs.unlinkSync(target);
}
}
for (const name of ["typescript", platformPackage]) {
const owner = name === platformPackage ? nativeRequire : require;
const source = path.dirname(owner.resolve(`${name}/package.json`));
for (const [name, source] of [
["typescript", path.dirname(require.resolve("typescript/package.json"))],
[platformPackage, resolvePlatformPackageDir()],
] as const) {
const omitted =
name === "typescript" && !javaScriptApi
? [path.join(source, "dist"), path.join(source, "vendor")]
: [];
const destination = path.join(root, "node_modules", name);
fs.mkdirSync(path.dirname(destination), { recursive: true });
copyTreeCloseOnExec(source, destination, { dereference: true });
copyTreeCloseOnExec(source, destination, {
dereference: true,
filter: (file) => !omitted.includes(file),
});
}
const bin = path.join(root, "node_modules/.bin/tsgo");
fs.mkdirSync(path.dirname(bin), { recursive: true });
@ -57,13 +79,7 @@ export function materializeNativeCompiler(rootDir: string) {
if (process.platform === "win32") {
fs.writeFileSync(`${bin}.cmd`, '@node "%~dp0..\\typescript\\bin\\tsc" %*\r\n');
}
return path.join(
root,
"node_modules",
platformPackage,
"lib",
process.platform === "win32" ? "tsc.exe" : "tsc",
);
return path.join(root, "node_modules", platformPackage, "lib", nativeExecutable);
}
/** Intercept a fixture compiler process without changing the production resolver. */

View file

@ -7,6 +7,7 @@ import { useAutoCleanupTempDirTracker } from "../helpers/temp-dir.js";
import {
materializeNativeCompiler,
overrideNativeFixtureExecutable,
resolveInstalledNativeCompiler,
writeNativeFixtureFile,
} from "./native-boundary-fixture.js";
@ -79,13 +80,17 @@ function createLinkedCheckoutFixture() {
return { primary, root, git };
}
function installCheckoutTools(root: string) {
const native = materializeNativeCompiler(root);
function linkCheckoutTools(root: string) {
for (const name of ["tsx", "@openclaw/fs-safe"]) {
const target = path.join(root, "node_modules", name);
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.symlinkSync(path.join(sourceRoot, "node_modules", name), target, "junction");
}
}
function installCheckoutTools(root: string) {
const native = materializeNativeCompiler(root, { javaScriptApi: false });
linkCheckoutTools(root);
return native;
}
@ -107,7 +112,8 @@ describe("run-tsgo linked worktree entry", () => {
it("selects its own root compiler from src while preserving relative project semantics", () => {
const { primary, root } = createLinkedCheckoutFixture();
installCheckoutTools(primary);
const native = installCheckoutTools(root);
linkCheckoutTools(root);
const native = resolveInstalledNativeCompiler();
const write = (file: string, text: string) => writeNativeFixtureFile(root, file, text);
const compilerOptions = {
module: "NodeNext",
@ -136,7 +142,7 @@ process.exitCode = result.status ?? 1;
);
fs.chmodSync(path.join(root, "native-compiler.mjs"), 0o755);
const launcher = path.join(root, "node_modules/.bin/tsgo");
fs.unlinkSync(launcher);
fs.mkdirSync(path.dirname(launcher), { recursive: true });
fs.symlinkSync("../../native-compiler.mjs", launcher, "file");
if (process.platform === "win32") {
write("node_modules/.bin/tsgo.cmd", '@node "%~dp0tsgo" %*\r\n');

View file

@ -223,7 +223,7 @@ console.log("standalone package boundary verified");
"nested-install-".repeat(5),
);
fs.mkdirSync(root, { recursive: true });
const native = materializeNativeCompiler(root);
const native = materializeNativeCompiler(root, { javaScriptApi: false });
expect(native.length).toBeGreaterThanOrEqual(248);
const require = createRequire(path.join(root, "package.json"));
const getExePath: { default: () => string } = require(

View file

@ -7,7 +7,6 @@ import {
publicPluginSdkEntrypoints,
} from "../../scripts/lib/plugin-sdk-entries.mts";
import { requireNodeTool } from "../helpers/node-toolchain.js";
import { materializeNativeCompiler } from "./native-boundary-fixture.js";
import {
createDeclarationFixture as createFixture,
createDeclarationTest,
@ -145,7 +144,6 @@ describe("write-plugin-sdk-entry-dts", { timeout: WRITER_TEST_TIMEOUT_MS }, () =
}) =>
command.lifetime.run(async () => {
const { root, write, writeDeclarations, production, qa } = createFixture(command);
materializeNativeCompiler(root);
expect(production.toSorted()).toEqual(
publicPluginSdkEntrypoints.map((entry) => `plugin-sdk/${entry}`).toSorted(),
);
@ -237,7 +235,6 @@ describe("write-plugin-sdk-entry-dts", { timeout: WRITER_TEST_TIMEOUT_MS }, () =
write: writeRelocated,
writeDeclarations: writeRelocatedDeclarations,
} = createFixture(command);
materializeNativeCompiler(relocated);
writeRelocatedDeclarations("after");
fs.rmSync(path.join(relocated, "contracts/before.ts"));
writeRelocated("test/unrelated.test.ts", "export const test = 2;\n");

View file

@ -3,7 +3,6 @@ import path from "node:path";
import { describe, expect, vi } from "vitest";
import { TSDOWN_NON_SDK_DTS_CONFIG_GROUPS } from "../../scripts/lib/tsdown-config-groups.mts";
import { resolveTsdownDeclarationGeneratorInputs } from "../../scripts/lib/tsdown-declaration-generator-inputs.mts";
import { materializeNativeCompiler } from "./native-boundary-fixture.js";
import {
createDeclarationFixture as createFixture,
createDeclarationTest,
@ -173,7 +172,6 @@ describe("write-unified-entry-dts", () => {
const { root, write, production, declarations } = await measurePhase("fixture", () =>
createFixture(command, TSDOWN_NON_SDK_DTS_CONFIG_GROUPS),
);
await measurePhase("native-compiler-fixture", () => materializeNativeCompiler(root));
expect(Object.values(declarations).every((entries) => entries.length > 0)).toBe(true);
expect(production).toHaveLength(Object.values(declarations).flat().length);
write("extensions/fixture-a/runtime-only.js", 'export const runtimeOnly = "runtime";');