mirror of
https://github.com/openclaw/openclaw.git
synced 2026-10-03 01:29:56 +00:00
feat(tooling): collect local compiler performance evidence (#156836)
* feat(tooling): add opt-in compiler performance evidence * test(tooling): narrow compiler metrics artifact paths * style(tooling): use braced compiler metrics guards * chore(tooling): integrate main compiler ownership * style(ci): format tsgo compiler version lookup * feat(ci): integrate compiler metrics with current runner Preserve the current prepared compiler owner, Kysely preparation, static diagnostics, signal callbacks and output joins while retaining opt-in metrics. Cover combined CI diagnostics and metrics for successful and diagnostic compiler exits. The previous focused tests and profiles remain bound to the older composition. This integration is source-reviewed; exact-head hosted qualification is pending. Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
This commit is contained in:
parent
e055d835cd
commit
c0c8fc9a39
6 changed files with 757 additions and 3 deletions
|
|
@ -134,6 +134,77 @@ Missing active translations or resources, active placeholder drift, invalid
|
|||
artifact syntax, and other generated-output differences remain blocking. Generator sync
|
||||
and the standalone Android and Apple checks retain their strict behavior.
|
||||
|
||||
### Opt-in compiler evidence
|
||||
|
||||
Keep `pnpm check:timed` (or `check:changed --timed`) for stage timings and
|
||||
`pnpm tsgo:profile <graph> --deep` for deliberate multi-pass graph/pprof analysis.
|
||||
For evidence from the compiler invocation you are already running, set
|
||||
`OPENCLAW_TSGO_METRICS_DIR=.artifacts/tsgo-metrics`. Each `run-tsgo` invocation
|
||||
writes a separate JSON artifact on ordinary local developer machines; neither
|
||||
metrics nor `OPENCLAW_TSGO_PPROF_DIR` requires `OPENCLAW_LOCAL_CHECK_MODE=throttled`,
|
||||
CI, or a server-specific setup. Unset or blank metrics means no metrics imports, probes,
|
||||
files, or additional output on the normal path. This also works for test shards
|
||||
and compiler stages reached through the timed check wrappers. It does not add a
|
||||
compiler invocation or change the compiler arguments, limits, deadline, signals,
|
||||
or cleanup policy. Metrics write failures warn without replacing compiler results.
|
||||
|
||||
Artifacts record the effective command and exit/error/signal, managed wall time
|
||||
(including cleanup, excluding evidence I/O and artifact-ownership admission),
|
||||
revision, tracked-dirty status, installed native compiler/Node versions, lockfile
|
||||
digest, OS, and effective Go limits. Unknown provenance is `null`; tracked-dirty
|
||||
status does not account for untracked inputs. Commands and paths can contain
|
||||
private local information: inspect and scrub artifacts before sharing them.
|
||||
|
||||
On Linux, opt-in sampling reads the compiler process's `/proc` CPU counters
|
||||
(all threads) and RSS high-water mark every 100 ms. CPU and peak RSS are explicitly
|
||||
**sampled lower bounds**, not exact end-of-process totals or process-tree memory.
|
||||
They can miss the final interval; very short runs or restricted procfs can have no
|
||||
usable samples. Missing statistics are `null` with a reason, never zero-filled.
|
||||
macOS and Windows report unsupported resource sampling; wall time and provenance
|
||||
remain available. No wrapper process changes signal ownership.
|
||||
|
||||
An explicit `--tsBuildInfoFile` supplies before/after SHA-256 evidence. Its presence
|
||||
is not a cache hit: `hit` stays `unknown`, and the OS page cache is `uncontrolled`.
|
||||
Changed, readable JSON build metadata supplies total, root, and non-root/transitive
|
||||
file counts, including library/declaration inputs. Unchanged, absent, oversized
|
||||
(over 32 MiB), or unsupported metadata leaves counts unavailable rather than
|
||||
attributing a stale graph to this run. Implicit cache paths and solution-build
|
||||
caches are not inferred. No `--listFiles`, `--showConfig`, or diagnostics pass is
|
||||
launched to fill a missing field. Use the existing `tsgo:profile` tool when you
|
||||
intentionally need those additional passes.
|
||||
|
||||
For a bounded comparison, use the same frozen install, source revision, resource
|
||||
limits, machine, and project for three absent-build-info/reuse pairs:
|
||||
|
||||
```bash
|
||||
mkdir -p .artifacts
|
||||
benchmark_dir=$(mktemp -d .artifacts/tsgo-benchmark.XXXXXX)
|
||||
for repeat in 1 2 3; do
|
||||
pair="$benchmark_dir/$repeat"
|
||||
mkdir -p "$pair"
|
||||
OPENCLAW_TSGO_METRICS_DIR="$pair/cold" node scripts/run-tsgo.mjs \
|
||||
-p tsconfig.ui.json --incremental --tsBuildInfoFile "$pair/cache.tsbuildinfo" || break
|
||||
OPENCLAW_TSGO_METRICS_DIR="$pair/warm" node scripts/run-tsgo.mjs \
|
||||
-p tsconfig.ui.json --incremental --tsBuildInfoFile "$pair/cache.tsbuildinfo" || break
|
||||
done
|
||||
```
|
||||
|
||||
Honor the host's existing build lock and resource policy around this command tree;
|
||||
do not run the pairs concurrently. Stop and investigate any nonzero compiler exit
|
||||
before interpreting timing. “Cold” here means only that this pair's build-info file
|
||||
was absent; it does not mean cold OS/dependency caches. “Warm” means reuse was
|
||||
attempted with identical inputs, not a proven cache hit. Compare medians and the
|
||||
range, report all exits and missing fields, and keep before/after revisions and
|
||||
lockfile/toolchain digests beside the results. Do not delete shared caches, drop OS
|
||||
caches, or relax resource caps to manufacture a favorable comparison. The recipe
|
||||
creates only a new benchmark directory and leaves normal caches untouched.
|
||||
|
||||
For CPU/heap investigation, reuse `OPENCLAW_TSGO_PPROF_DIR` with a distinct directory
|
||||
per measured invocation, or use `tsgo:profile --deep`. Profiling changes measurement
|
||||
overhead: enable it for both comparison sides or keep it outside timing pairs.
|
||||
The evidence records the effective `--pprofDir`; it does not manage or delete those
|
||||
profiles. These measurements are developer evidence, not a CI pass/fail threshold.
|
||||
|
||||
The Gateway watch regression check starts its idle CPU window only after readiness
|
||||
and the settle period. Startup and early-exit failures still fail the check. Missing
|
||||
CPU samples from an otherwise valid window fail measurement; whole-run CPU is
|
||||
|
|
|
|||
315
scripts/lib/tsgo-performance.mts
Normal file
315
scripts/lib/tsgo-performance.mts
Normal file
|
|
@ -0,0 +1,315 @@
|
|||
// Opt-in evidence for the existing compiler process, never another compiler pass.
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import fs from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { readFlagValue } from "./arg-utils.mts";
|
||||
import { runManagedCommand } from "./managed-child-process.mts";
|
||||
import { findRepoRoot } from "./repo-root.mjs";
|
||||
|
||||
type Command = Parameters<typeof runManagedCommand>[0];
|
||||
const SAMPLE_INTERVAL_MS = 100;
|
||||
const MAX_METADATA_BYTES = 32 * 1024 * 1024;
|
||||
|
||||
function readFile(file: string | undefined): Buffer | null {
|
||||
if (!file) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
if (fs.statSync(file).size > MAX_METADATA_BYTES) {
|
||||
return null;
|
||||
}
|
||||
return fs.readFileSync(file);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function digest(bytes: Buffer | null) {
|
||||
return bytes === null ? null : createHash("sha256").update(bytes).digest("hex");
|
||||
}
|
||||
|
||||
/** Linux stat accounts for all threads; status VmHWM is the process RSS high-water mark. */
|
||||
export function parseTsgoProcessSample(stat: string, status: string, ticksPerSecond: number) {
|
||||
// comm can contain spaces and parentheses. Fields after its final ')' start at field 3.
|
||||
const fields = stat
|
||||
.slice(stat.lastIndexOf(")") + 2)
|
||||
.trim()
|
||||
.split(/\s+/u);
|
||||
const userTicks = Number(fields[11]);
|
||||
const systemTicks = Number(fields[12]);
|
||||
const startTicks = fields[19];
|
||||
const rssKiB = /^VmHWM:\s+(\d+)\s+kB$/mu.exec(status)?.[1];
|
||||
if (
|
||||
!/^\d+$/u.test(startTicks ?? "") ||
|
||||
!Number.isFinite(userTicks) ||
|
||||
userTicks < 0 ||
|
||||
!Number.isFinite(systemTicks) ||
|
||||
systemTicks < 0 ||
|
||||
!Number.isFinite(ticksPerSecond) ||
|
||||
ticksPerSecond <= 0
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
startTicks,
|
||||
cpuMs: ((userTicks + systemTicks) / ticksPerSecond) * 1000,
|
||||
peakRssBytes: rssKiB === undefined ? null : Number(rssKiB) * 1024,
|
||||
};
|
||||
}
|
||||
|
||||
export function createTsgoResourceSampler({
|
||||
platform = process.platform,
|
||||
ticksPerSecond,
|
||||
read = (file: string) => fs.readFileSync(file, "utf8"),
|
||||
}: {
|
||||
platform?: NodeJS.Platform;
|
||||
ticksPerSecond: number | null;
|
||||
read?: (file: string) => string;
|
||||
}) {
|
||||
let startTicks: string | undefined;
|
||||
let samples = 0;
|
||||
let cpuMs: number | null = null;
|
||||
let peakRssBytes: number | null = null;
|
||||
const unavailableReason =
|
||||
platform !== "linux"
|
||||
? "unsupported-platform"
|
||||
: ticksPerSecond === null
|
||||
? "clock-tick-frequency-unavailable"
|
||||
: null;
|
||||
return {
|
||||
sample(pid: number) {
|
||||
if (unavailableReason || ticksPerSecond === null) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const sample = parseTsgoProcessSample(
|
||||
read(`/proc/${pid}/stat`),
|
||||
read(`/proc/${pid}/status`),
|
||||
ticksPerSecond,
|
||||
);
|
||||
if (!sample || (startTicks !== undefined && startTicks !== sample.startTicks)) {
|
||||
return;
|
||||
}
|
||||
startTicks ??= sample.startTicks;
|
||||
samples += 1;
|
||||
cpuMs = Math.max(cpuMs ?? 0, sample.cpuMs);
|
||||
if (sample.peakRssBytes !== null) {
|
||||
peakRssBytes = Math.max(peakRssBytes ?? 0, sample.peakRssBytes);
|
||||
}
|
||||
} catch {
|
||||
// Short-lived children, restricted procfs, and exit races leave missing samples.
|
||||
}
|
||||
},
|
||||
result() {
|
||||
return {
|
||||
source: "linux-procfs",
|
||||
scope: "compiler-process-all-threads-not-descendants",
|
||||
accuracy: "sampled-lower-bound",
|
||||
intervalMs: SAMPLE_INTERVAL_MS,
|
||||
samples,
|
||||
cpuMs,
|
||||
peakRssBytes,
|
||||
unavailableReason:
|
||||
unavailableReason ?? (samples === 0 ? "process-samples-unavailable" : null),
|
||||
peakRssUnavailableReason:
|
||||
peakRssBytes === null
|
||||
? (unavailableReason ??
|
||||
(samples === 0 ? "process-samples-unavailable" : "rss-high-water-unavailable"))
|
||||
: null,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Counts only metadata written by this invocation; unchanged input is not fresh graph evidence. */
|
||||
export function summarizeTsgoBuildInfo(before: Buffer | null, after: Buffer | null) {
|
||||
const missing = (reason: string) => ({
|
||||
totalFiles: null,
|
||||
rootFiles: null,
|
||||
transitiveFiles: null,
|
||||
unavailableReason: reason,
|
||||
});
|
||||
if (!after) {
|
||||
return missing("build-info-unavailable");
|
||||
}
|
||||
if (before?.equals(after)) {
|
||||
return missing("build-info-unchanged");
|
||||
}
|
||||
try {
|
||||
const info = JSON.parse(after.toString("utf8"));
|
||||
if (
|
||||
!Array.isArray(info.fileNames) ||
|
||||
!info.fileNames.every((name: unknown) => typeof name === "string") ||
|
||||
!Array.isArray(info.root)
|
||||
) {
|
||||
return missing("unsupported-build-info");
|
||||
}
|
||||
const ranges: Array<[number, number]> = [];
|
||||
for (const entry of info.root) {
|
||||
const [start, end] = Array.isArray(entry) && entry.length === 2 ? entry : [entry, entry];
|
||||
if (
|
||||
!Number.isInteger(start) ||
|
||||
!Number.isInteger(end) ||
|
||||
start < 1 ||
|
||||
end < start ||
|
||||
end > info.fileNames.length
|
||||
) {
|
||||
return missing("unsupported-build-info");
|
||||
}
|
||||
ranges.push([start, end]);
|
||||
}
|
||||
ranges.sort(([left], [right]) => left - right);
|
||||
let rootFiles = 0;
|
||||
let previousEnd = 0;
|
||||
for (const [start, end] of ranges) {
|
||||
rootFiles += Math.max(0, end - Math.max(start - 1, previousEnd));
|
||||
previousEnd = Math.max(previousEnd, end);
|
||||
}
|
||||
return {
|
||||
totalFiles: info.fileNames.length,
|
||||
rootFiles,
|
||||
transitiveFiles: info.fileNames.length - rootFiles,
|
||||
unavailableReason: null,
|
||||
};
|
||||
} catch {
|
||||
return missing("unsupported-build-info");
|
||||
}
|
||||
}
|
||||
|
||||
function provenance(cwd: string, env: NodeJS.ProcessEnv) {
|
||||
const root = findRepoRoot(cwd) ?? cwd;
|
||||
const git = (args: string[]) =>
|
||||
spawnSync("git", args, {
|
||||
cwd: root,
|
||||
encoding: "utf8",
|
||||
stdio: ["ignore", "pipe", "ignore"],
|
||||
timeout: 2_000,
|
||||
});
|
||||
const revision = git(["rev-parse", "HEAD"]);
|
||||
const dirty = git(["diff", "--quiet", "HEAD", "--"]);
|
||||
let compilerVersion: string | null = null;
|
||||
try {
|
||||
const require = createRequire(path.join(root, "package.json"));
|
||||
compilerVersion =
|
||||
JSON.parse(fs.readFileSync(require.resolve("typescript/package.json"), "utf8")).version ??
|
||||
null;
|
||||
} catch {
|
||||
/* A missing package manifest is not a compiler failure. */
|
||||
}
|
||||
return {
|
||||
revision: revision.status === 0 ? revision.stdout.trim() : null,
|
||||
trackedDirty: dirty.status === 0 ? false : dirty.status === 1 ? true : null,
|
||||
lockfileSha256: digest(readFile(path.join(root, "pnpm-lock.yaml"))),
|
||||
compilerVersion,
|
||||
nodeVersion: process.version,
|
||||
platform: process.platform,
|
||||
arch: process.arch,
|
||||
osRelease: os.release(),
|
||||
resources: Object.fromEntries(
|
||||
["OPENCLAW_LOCAL_CHECK_MODE", "GOMAXPROCS", "GOGC", "GOMEMLIMIT"].map((key) => [
|
||||
key,
|
||||
env[key] ?? null,
|
||||
]),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/** Keep the managed child as the sole signal/deadline/cleanup owner. Metrics never replace its result. */
|
||||
export async function runMeasuredTsgoCommand(command: Command, directory: string) {
|
||||
const cwd = command.cwd ?? process.cwd();
|
||||
let artifact: string;
|
||||
let inputs: ReturnType<typeof provenance>;
|
||||
try {
|
||||
const destination = path.resolve(cwd, directory);
|
||||
fs.mkdirSync(destination, { recursive: true });
|
||||
artifact = path.join(destination, `tsgo-${randomUUID()}.json`);
|
||||
inputs = provenance(cwd, command.env ?? process.env);
|
||||
} catch {
|
||||
console.error("[tsgo-metrics] evidence setup unavailable; running compiler without metrics");
|
||||
return runManagedCommand(command);
|
||||
}
|
||||
const args = command.args ?? [];
|
||||
const buildInfoArg = readFlagValue(args, "--tsBuildInfoFile");
|
||||
const buildInfo = buildInfoArg ? path.resolve(cwd, buildInfoArg) : undefined;
|
||||
const before = readFile(buildInfo);
|
||||
const ticks =
|
||||
process.platform === "linux"
|
||||
? spawnSync("getconf", ["CLK_TCK"], {
|
||||
encoding: "utf8",
|
||||
stdio: ["ignore", "pipe", "ignore"],
|
||||
timeout: 2_000,
|
||||
})
|
||||
: null;
|
||||
const ticksPerSecond =
|
||||
ticks?.status === 0 && Number(ticks.stdout) > 0 ? Number(ticks.stdout) : null;
|
||||
const sampler = createTsgoResourceSampler({ ticksPerSecond });
|
||||
let timer: ReturnType<typeof setInterval> | undefined;
|
||||
let exitCode: number | null = null;
|
||||
let exitSignal: NodeJS.Signals | null = null;
|
||||
let forwardedSignal: NodeJS.Signals | null = null;
|
||||
let errorCode: string | null = null;
|
||||
const startedAt = new Date().toISOString();
|
||||
const started = performance.now();
|
||||
try {
|
||||
exitCode = await runManagedCommand({
|
||||
...command,
|
||||
onReady(child) {
|
||||
const sample = () => {
|
||||
if (child.pid && child.exitCode === null && child.signalCode === null) {
|
||||
sampler.sample(child.pid);
|
||||
}
|
||||
};
|
||||
sample();
|
||||
if (process.platform === "linux" && ticksPerSecond !== null) {
|
||||
timer = setInterval(sample, SAMPLE_INTERVAL_MS);
|
||||
timer.unref();
|
||||
}
|
||||
child.once("exit", (_code, signal) => {
|
||||
exitSignal = signal;
|
||||
clearInterval(timer);
|
||||
});
|
||||
command.onReady?.(child);
|
||||
},
|
||||
onSignal(signal) {
|
||||
forwardedSignal = signal;
|
||||
command.onSignal?.(signal);
|
||||
},
|
||||
});
|
||||
return exitCode;
|
||||
} catch (error) {
|
||||
errorCode =
|
||||
error && typeof error === "object" && "code" in error ? String(error.code) : "command-failed";
|
||||
throw error;
|
||||
} finally {
|
||||
const wallMs = performance.now() - started;
|
||||
clearInterval(timer);
|
||||
try {
|
||||
const after = readFile(buildInfo);
|
||||
const evidence = {
|
||||
schemaVersion: 1,
|
||||
startedAt,
|
||||
...inputs,
|
||||
command: { bin: command.bin, args, cwd, timeoutMs: command.timeoutMs ?? null },
|
||||
outcome: { exitCode, exitSignal, forwardedSignal, errorCode },
|
||||
wallMs,
|
||||
wallScope: "managed-invocation-including-cleanup-excluding-evidence-io",
|
||||
resources: { policy: inputs.resources, ...sampler.result() },
|
||||
cache: {
|
||||
buildInfoFile: buildInfoArg ?? null,
|
||||
beforeSha256: digest(before),
|
||||
afterSha256: digest(after),
|
||||
hit: "unknown",
|
||||
osPageCache: "uncontrolled",
|
||||
},
|
||||
graph: { source: "changed-tsbuildinfo", ...summarizeTsgoBuildInfo(before, after) },
|
||||
pprofDir: readFlagValue(args, "--pprofDir") ?? null,
|
||||
};
|
||||
fs.writeFileSync(artifact, `${JSON.stringify(evidence, null, 2)}\n`, { flag: "wx" });
|
||||
} catch {
|
||||
console.error("[tsgo-metrics] could not write evidence; compiler outcome preserved");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -110,7 +110,7 @@ export async function runPreparedTsgoCommand(
|
|||
let capturedBytes = 0;
|
||||
let overflow = false;
|
||||
let interrupted = false;
|
||||
const code = await runManagedCommand({
|
||||
const managedCommand = {
|
||||
...command,
|
||||
args: capture ? [...command.args, "--pretty", "false"] : command.args,
|
||||
requireProcessTreeExit: process.platform !== "win32",
|
||||
|
|
@ -145,7 +145,14 @@ export async function runPreparedTsgoCommand(
|
|||
},
|
||||
}
|
||||
: {}),
|
||||
});
|
||||
};
|
||||
const metricsDir = command.env.OPENCLAW_TSGO_METRICS_DIR?.trim();
|
||||
// Keep CI output capture and cleanup callbacks on the same managed invocation.
|
||||
const code = metricsDir
|
||||
? await (
|
||||
await import("./lib/tsgo-performance.mts")
|
||||
).runMeasuredTsgoCommand(managedCommand, metricsDir)
|
||||
: await runManagedCommand(managedCommand);
|
||||
await Promise.all(forwarding);
|
||||
const stdout = Buffer.concat(outputs[0]!).toString("utf8");
|
||||
const stderr = Buffer.concat(outputs[1]!).toString("utf8");
|
||||
|
|
|
|||
|
|
@ -226,6 +226,25 @@ describe("local-check-runtime", () => {
|
|||
expect(env.GOMEMLIMIT).toBe("5GiB");
|
||||
});
|
||||
|
||||
it("enables opt-in profiling on ordinary local machines without a throttled mode", () => {
|
||||
const { args, env } = applyLocalTsgoPolicy(
|
||||
["-p", "tsconfig.ui.json"],
|
||||
{ OPENCLAW_TSGO_PPROF_DIR: ".artifacts/profiles" },
|
||||
ROOMY_HOST,
|
||||
);
|
||||
expect(args).toEqual([
|
||||
"-p",
|
||||
"tsconfig.ui.json",
|
||||
"--declaration",
|
||||
"false",
|
||||
"--pprofDir",
|
||||
".artifacts/profiles",
|
||||
]);
|
||||
expect(env.OPENCLAW_LOCAL_CHECK_MODE).toBeUndefined();
|
||||
expect(env.GOMAXPROCS).toBeUndefined();
|
||||
expect(env.GOMEMLIMIT).toBeUndefined();
|
||||
});
|
||||
|
||||
it("keeps explicit tsgo declaration flags intact", () => {
|
||||
const env = makeEnv({ OPENCLAW_LOCAL_CHECK_MODE: "full" });
|
||||
const longFlag = applyLocalTsgoPolicy(["--declaration"], env, ROOMY_HOST);
|
||||
|
|
|
|||
|
|
@ -43,6 +43,43 @@ it("runs the installed compiler version through the real tsgo wrapper", () => {
|
|||
expect(result.stdout.trim()).toBe(`Version ${nativePackage.version}`);
|
||||
}, 30_000);
|
||||
|
||||
it("keeps compiler output unchanged with opt-in metrics and emits no metrics by default", () => {
|
||||
const cwd = createTempDir("run-tsgo-metrics-");
|
||||
const {
|
||||
OPENCLAW_TSGO_METRICS_DIR: _unset,
|
||||
OPENCLAW_LOCAL_CHECK_MODE: _mode,
|
||||
...baseEnv
|
||||
} = process.env;
|
||||
for (const enabled of [false, true]) {
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[path.resolve("scripts/run-tsgo.mjs"), "--version"],
|
||||
{
|
||||
encoding: "utf8",
|
||||
timeout: 25_000,
|
||||
env: { ...baseEnv, ...(enabled ? { OPENCLAW_TSGO_METRICS_DIR: cwd } : {}) },
|
||||
},
|
||||
);
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.status).toBe(0);
|
||||
expect(result.stdout.trim()).toMatch(/^Version /u);
|
||||
expect(result.stderr).toBe("");
|
||||
expect(fs.readdirSync(cwd)).toHaveLength(enabled ? 1 : 0);
|
||||
if (enabled) {
|
||||
const [artifact] = fs.readdirSync(cwd);
|
||||
if (!artifact) {
|
||||
throw new Error("Missing compiler metrics artifact");
|
||||
}
|
||||
const evidence = JSON.parse(fs.readFileSync(path.join(cwd, artifact), "utf8"));
|
||||
expect(`Version ${evidence.compilerVersion}`).toBe(result.stdout.trim());
|
||||
expect(evidence.outcome.exitCode).toBe(0);
|
||||
expect(evidence.command.args).toContain("--version");
|
||||
expect(evidence.cache.hit).toBe("unknown");
|
||||
expect(evidence.resources.policy.OPENCLAW_LOCAL_CHECK_MODE).toBeNull();
|
||||
}
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
it.each([false, true])(
|
||||
"refuses a shared install without creating dependency links (linked=%s)",
|
||||
(linked) => {
|
||||
|
|
@ -425,9 +462,14 @@ describe.skipIf(process.platform === "win32")("run-tsgo watchdog", () => {
|
|||
cwd: string,
|
||||
timeoutMs: string | undefined,
|
||||
onBeforeReap?: (pid: number | undefined) => void,
|
||||
metricsDir?: string,
|
||||
) {
|
||||
const { OPENCLAW_TSGO_TIMEOUT_MS: _unset, ...inheritedEnv } = process.env;
|
||||
const baseEnv = { ...inheritedEnv, OPENCLAW_CI_STATIC_EVIDENCE: "1" };
|
||||
const baseEnv = {
|
||||
...inheritedEnv,
|
||||
OPENCLAW_CI_STATIC_EVIDENCE: "1",
|
||||
OPENCLAW_TSGO_METRICS_DIR: metricsDir,
|
||||
};
|
||||
try {
|
||||
return spawnSync(
|
||||
process.execPath,
|
||||
|
|
@ -451,6 +493,57 @@ describe.skipIf(process.platform === "win32")("run-tsgo watchdog", () => {
|
|||
}
|
||||
}
|
||||
|
||||
it.each([0, 2])(
|
||||
"preserves CI diagnostics and completion with metrics enabled (exit %s)",
|
||||
(exitCode) => {
|
||||
const cwd = createTempDir("run-tsgo-ci-metrics-");
|
||||
const metricsDir = path.join(cwd, "metrics");
|
||||
const diagnostic =
|
||||
exitCode === 2 ? "src/fixture.ts(1,1): error TS2322: Invalid fixture value.\n" : "";
|
||||
writeFakeTsgo(
|
||||
cwd,
|
||||
`#!/usr/bin/env node
|
||||
process.stdout.write(${JSON.stringify(diagnostic)});
|
||||
process.exitCode = ${exitCode};
|
||||
`,
|
||||
);
|
||||
|
||||
const result = runFakeTsgo(cwd, undefined, undefined, metricsDir);
|
||||
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.status).toBe(exitCode);
|
||||
const leafPrefix = "[ci-static:tsgo:leaf] ";
|
||||
const completionPrefix = "[ci-static:tsgo:completion] ";
|
||||
const lines = result.stdout.trim().split("\n");
|
||||
const leaves = lines.filter((line) => line.startsWith(leafPrefix));
|
||||
const completions = lines.filter((line) => line.startsWith(completionPrefix));
|
||||
expect(leaves).toHaveLength(1);
|
||||
expect(completions).toHaveLength(1);
|
||||
const leaf = JSON.parse(leaves[0]!.slice(leafPrefix.length));
|
||||
expect(leaf).toMatchObject({
|
||||
version: 1,
|
||||
config: "tsconfig.extensions.json",
|
||||
exitCode,
|
||||
stdout: diagnostic,
|
||||
stderr: "",
|
||||
});
|
||||
const completion = JSON.parse(completions[0]!.slice(completionPrefix.length));
|
||||
expect(completion).toMatchObject({ planned: 1, completed: 1, leaves: [leaf.id] });
|
||||
expect(result.stdout).toBe(
|
||||
`${diagnostic}${leafPrefix}${JSON.stringify(leaf)}\n${completionPrefix}${JSON.stringify(completion)}\n`,
|
||||
);
|
||||
const artifacts = fs.readdirSync(metricsDir);
|
||||
expect(artifacts).toHaveLength(1);
|
||||
const metrics = JSON.parse(fs.readFileSync(path.join(metricsDir, artifacts[0]!), "utf8"));
|
||||
expect(metrics.outcome).toMatchObject({ exitCode, errorCode: null });
|
||||
expect(metrics.command.args).toEqual(
|
||||
expect.arrayContaining(["-p", "tsconfig.extensions.json"]),
|
||||
);
|
||||
expect(metrics.command.args.slice(-2)).toEqual(["--pretty", "false"]);
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
it("rejects and drains compiler descendants left after a successful leader exit", async ({
|
||||
signal,
|
||||
}) => {
|
||||
|
|
|
|||
249
test/scripts/tsgo-performance.test.ts
Normal file
249
test/scripts/tsgo-performance.test.ts
Normal file
|
|
@ -0,0 +1,249 @@
|
|||
import { EventEmitter } from "node:events";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { runManagedCommand } from "../../scripts/lib/managed-child-process.mts";
|
||||
import {
|
||||
createTsgoResourceSampler,
|
||||
parseTsgoProcessSample,
|
||||
runMeasuredTsgoCommand,
|
||||
summarizeTsgoBuildInfo,
|
||||
} from "../../scripts/lib/tsgo-performance.mts";
|
||||
import { useAutoCleanupTempDirTracker } from "../helpers/temp-dir.js";
|
||||
|
||||
vi.mock("../../scripts/lib/managed-child-process.mts", () => ({ runManagedCommand: vi.fn() }));
|
||||
const tempDirs = useAutoCleanupTempDirTracker(afterEach);
|
||||
afterEach(() => vi.restoreAllMocks());
|
||||
|
||||
function stat({ user = 10, system = 5, start = "123" } = {}) {
|
||||
const fields = Array<string>(20).fill("0");
|
||||
fields[0] = "R";
|
||||
fields[11] = String(user);
|
||||
fields[12] = String(system);
|
||||
fields[19] = start;
|
||||
return `42 (tsgo (worker)) ${fields.join(" ")}`;
|
||||
}
|
||||
|
||||
it("parses all-thread CPU and kernel RSS high-water observations without confusing comm fields", () => {
|
||||
expect(parseTsgoProcessSample(stat(), "VmHWM:\t1024 kB\n", 100)).toEqual({
|
||||
startTicks: "123",
|
||||
cpuMs: 150,
|
||||
peakRssBytes: 1048576,
|
||||
});
|
||||
expect(parseTsgoProcessSample("garbled", "", 100)).toBeNull();
|
||||
expect(parseTsgoProcessSample(stat(), "", 0)).toBeNull();
|
||||
expect(parseTsgoProcessSample(stat(), "", 100)?.peakRssBytes).toBeNull();
|
||||
});
|
||||
|
||||
it.each(["darwin", "win32"] as const)(
|
||||
"reports resource statistics unavailable on %s without reading procfs",
|
||||
(platform) => {
|
||||
const read = vi.fn();
|
||||
const sampler = createTsgoResourceSampler({ platform, ticksPerSecond: 100, read });
|
||||
sampler.sample(42);
|
||||
expect(read).not.toHaveBeenCalled();
|
||||
expect(sampler.result()).toMatchObject({
|
||||
cpuMs: null,
|
||||
peakRssBytes: null,
|
||||
samples: 0,
|
||||
unavailableReason: "unsupported-platform",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps lower bounds across exit races and refuses a reused PID", () => {
|
||||
const read = vi
|
||||
.fn()
|
||||
.mockReturnValueOnce(stat())
|
||||
.mockReturnValueOnce("VmHWM: 1024 kB\n")
|
||||
.mockImplementationOnce(() => {
|
||||
throw new Error("ENOENT");
|
||||
})
|
||||
.mockReturnValueOnce(stat({ user: 500, start: "456" }))
|
||||
.mockReturnValueOnce("VmHWM: 4096 kB\n");
|
||||
const sampler = createTsgoResourceSampler({ platform: "linux", ticksPerSecond: 100, read });
|
||||
sampler.sample(42);
|
||||
sampler.sample(42);
|
||||
sampler.sample(42);
|
||||
expect(sampler.result()).toMatchObject({
|
||||
cpuMs: 150,
|
||||
peakRssBytes: 1048576,
|
||||
samples: 1,
|
||||
accuracy: "sampled-lower-bound",
|
||||
});
|
||||
const missing = createTsgoResourceSampler({ platform: "linux", ticksPerSecond: null, read });
|
||||
missing.sample(42);
|
||||
expect(missing.result()).toMatchObject({
|
||||
cpuMs: null,
|
||||
unavailableReason: "clock-tick-frequency-unavailable",
|
||||
});
|
||||
});
|
||||
|
||||
it("explains a missing RSS high-water mark separately from available CPU", () => {
|
||||
const sampler = createTsgoResourceSampler({
|
||||
platform: "linux",
|
||||
ticksPerSecond: 100,
|
||||
read: (file) => (file.endsWith("stat") ? stat() : ""),
|
||||
});
|
||||
sampler.sample(42);
|
||||
expect(sampler.result()).toMatchObject({
|
||||
cpuMs: 150,
|
||||
peakRssBytes: null,
|
||||
unavailableReason: null,
|
||||
peakRssUnavailableReason: "rss-high-water-unavailable",
|
||||
});
|
||||
});
|
||||
|
||||
it("counts roots and transitive inputs only from fresh compiler metadata", () => {
|
||||
const info = Buffer.from(
|
||||
JSON.stringify({ fileNames: ["a.ts", "b.ts", "c.ts", "lib.d.ts"], root: [[1, 2], 2] }),
|
||||
);
|
||||
expect(summarizeTsgoBuildInfo(null, info)).toEqual({
|
||||
rootFiles: 2,
|
||||
transitiveFiles: 2,
|
||||
totalFiles: 4,
|
||||
unavailableReason: null,
|
||||
});
|
||||
expect(summarizeTsgoBuildInfo(info, info)).toMatchObject({
|
||||
totalFiles: null,
|
||||
unavailableReason: "build-info-unchanged",
|
||||
});
|
||||
expect(summarizeTsgoBuildInfo(null, null)).toMatchObject({
|
||||
totalFiles: null,
|
||||
unavailableReason: "build-info-unavailable",
|
||||
});
|
||||
for (const unsupported of [
|
||||
"not JSON",
|
||||
'{"fileNames":[],"root":[[1,999999999]]}',
|
||||
'{"fileNames":[]}',
|
||||
]) {
|
||||
expect(summarizeTsgoBuildInfo(null, Buffer.from(unsupported))).toMatchObject({
|
||||
rootFiles: null,
|
||||
unavailableReason: "unsupported-build-info",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe("managed compiler evidence", () => {
|
||||
function fixture() {
|
||||
const cwd = tempDirs.make("tsgo-performance-");
|
||||
fs.writeFileSync(path.join(cwd, "package.json"), '{"private":true}');
|
||||
const directory = path.join(cwd, "metrics");
|
||||
const command = {
|
||||
bin: "tsgo",
|
||||
args: [
|
||||
"-p",
|
||||
"fixture.json",
|
||||
"--tsBuildInfoFile",
|
||||
"cache.tsbuildinfo",
|
||||
"--pprofDir",
|
||||
"profiles",
|
||||
],
|
||||
cwd,
|
||||
env: {},
|
||||
timeoutMs: 10_000,
|
||||
requireProcessTreeExit: true,
|
||||
};
|
||||
return {
|
||||
cwd,
|
||||
directory,
|
||||
command,
|
||||
evidence: () => {
|
||||
const [artifact] = fs.readdirSync(directory);
|
||||
if (!artifact) {
|
||||
throw new Error("Missing compiler metrics artifact");
|
||||
}
|
||||
return JSON.parse(fs.readFileSync(path.join(directory, artifact), "utf8"));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
it("preserves command, output policy, callbacks and numeric failure while recording fresh graph/cache evidence", async () => {
|
||||
const { cwd, directory, command, evidence } = fixture();
|
||||
const onReady = vi.fn();
|
||||
vi.mocked(runManagedCommand).mockImplementationOnce(async (options) => {
|
||||
expect(options).toMatchObject(command);
|
||||
expect(options.stdio).toBeUndefined();
|
||||
const child = Object.assign(new EventEmitter(), {
|
||||
pid: undefined,
|
||||
exitCode: null,
|
||||
signalCode: null,
|
||||
});
|
||||
options.onReady?.(child as Parameters<NonNullable<typeof options.onReady>>[0]);
|
||||
fs.writeFileSync(
|
||||
path.join(cwd, "cache.tsbuildinfo"),
|
||||
'{"fileNames":["a.ts","lib.d.ts"],"root":[1]}',
|
||||
);
|
||||
child.emit("exit", 2, null);
|
||||
return 2;
|
||||
});
|
||||
expect(await runMeasuredTsgoCommand({ ...command, onReady }, directory)).toBe(2);
|
||||
expect(onReady).toHaveBeenCalledOnce();
|
||||
expect(evidence()).toMatchObject({
|
||||
outcome: { exitCode: 2, errorCode: null },
|
||||
command: { args: command.args },
|
||||
pprofDir: "profiles",
|
||||
graph: { totalFiles: 2, rootFiles: 1, transitiveFiles: 1 },
|
||||
cache: { beforeSha256: null, hit: "unknown", osPageCache: "uncontrolled" },
|
||||
});
|
||||
expect(evidence().wallMs).toBeGreaterThanOrEqual(0);
|
||||
expect(evidence().cache.afterSha256).toMatch(/^[a-f0-9]{64}$/u);
|
||||
});
|
||||
|
||||
it("records forwarded and child signals while leaving the managed result authoritative", async () => {
|
||||
const { directory, command, evidence } = fixture();
|
||||
const onSignal = vi.fn();
|
||||
vi.mocked(runManagedCommand).mockImplementationOnce(async (options) => {
|
||||
const child = Object.assign(new EventEmitter(), { exitCode: null, signalCode: null });
|
||||
options.onReady?.(child as Parameters<NonNullable<typeof options.onReady>>[0]);
|
||||
options.onSignal?.("SIGTERM");
|
||||
child.emit("exit", null, "SIGTERM");
|
||||
return 143;
|
||||
});
|
||||
expect(await runMeasuredTsgoCommand({ ...command, onSignal }, directory)).toBe(143);
|
||||
expect(onSignal).toHaveBeenCalledWith("SIGTERM");
|
||||
expect(evidence().outcome).toMatchObject({
|
||||
exitCode: 143,
|
||||
exitSignal: "SIGTERM",
|
||||
forwardedSignal: "SIGTERM",
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["ETIMEDOUT", "ENOENT", "EPROCESSGROUP_CLEANUP_FAILED"])(
|
||||
"never replaces %s with an evidence result",
|
||||
async (code) => {
|
||||
const { directory, command, evidence } = fixture();
|
||||
const error = Object.assign(new Error("compiler failure"), { code });
|
||||
vi.mocked(runManagedCommand).mockRejectedValueOnce(error);
|
||||
await expect(runMeasuredTsgoCommand(command, directory)).rejects.toBe(error);
|
||||
expect(evidence().outcome).toMatchObject({ exitCode: null, errorCode: code });
|
||||
},
|
||||
);
|
||||
|
||||
it("does not turn unavailable evidence output into a compiler failure or rerun", async () => {
|
||||
const { cwd, command } = fixture();
|
||||
const warn = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
vi.mocked(runManagedCommand).mockResolvedValueOnce(7);
|
||||
const calls = vi.mocked(runManagedCommand).mock.calls.length;
|
||||
expect(await runMeasuredTsgoCommand(command, path.join(cwd, "package.json", "metrics"))).toBe(
|
||||
7,
|
||||
);
|
||||
expect(vi.mocked(runManagedCommand).mock.calls.length - calls).toBe(1);
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining("without metrics"));
|
||||
});
|
||||
|
||||
it("writes separate artifacts for repeated invocations and preserves outcomes if final writing fails", async () => {
|
||||
const { directory, command } = fixture();
|
||||
vi.mocked(runManagedCommand).mockResolvedValue(0);
|
||||
await runMeasuredTsgoCommand(command, directory);
|
||||
await runMeasuredTsgoCommand(command, directory);
|
||||
expect(fs.readdirSync(directory)).toHaveLength(2);
|
||||
const warn = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
vi.mocked(runManagedCommand).mockImplementationOnce(async () => {
|
||||
fs.rmSync(directory, { recursive: true });
|
||||
return 9;
|
||||
});
|
||||
expect(await runMeasuredTsgoCommand(command, directory)).toBe(9);
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining("compiler outcome preserved"));
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue