mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-09-11 19:46:21 +00:00
* ci: host PR evidence outside Git Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * ci: pin node and ossutil paths in privileged OSS uploads The publish-verify evidence upload resolved `node` through a PATH prefixed with $RUNNER_TEMP — the one directory PR code can write to via the verify container's bind mount — so a planted fake `node` there won interpreter resolution inside the step that holds CI_BOT_PAT and the OSS config (PATH hijack). Resolve node under the inherited clean PATH and run ossutil from a fresh job-private copy of the sha256-verified binary installed above instead of resolving either through $RUNNER_TEMP. Apply the same shape to web-shell-visuals-publish.yml so the pattern cannot regress if that job ever moves to a persistent pool. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * ci: make OSS setup best-effort and pair bucket/base-URL defaults Mirror qwen-triage.yml in web-shell-visuals-publish.yml: mark the ossutil install and credential-config steps continue-on-error so a setup failure cannot abort the no-image marker-comment path (the upload itself stays loud when images are present). In both publishers, derive the default ALIYUN_OSS_PUBLIC_BASE_URL from ALIYUN_OSS_BUCKET so overriding only one of the two vars cannot post comment links that 404 against (or show stale objects from) the other bucket. Re-record both workflow sizes in the baseline. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * ci: enrich evidence-upload warning with local preconditions The single "Failed to upload evidence images" warning could not distinguish its causes: the ossutil install and credential-config steps run with continue-on-error and render green when they fail, so a missing binary or config looks identical to a transient upload blip or rotated keys. Probe both preconditions into the warning (ossutil=ok/MISSING, config=ok/MISSING) so the investigator sees the gap without trawling three steps of logs. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * test: enforce the uploader flag contract and pin the production arm The evidence-hosting stub shifted --config away unvalidated, and every harness run sets VERIFY_ASSETS_UPLOADER, so the production `node "$uploader"` arm and its --config flag were never exercised: dropping --config from the workflow kept every test green while the real uploader exits 1 on it, silently degrading every /verify report to text-only. Make the stub reject a missing/empty --bucket, --config or --prefix, and pin the production invocation shape (inherited-PATH node resolution plus all three ossutil flags). Mutation-probed: with --config removed from the workflow both tests now fail. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * test: pin the ossutil credential lifecycle in both publishers Nothing pinned the credential lifecycle this PR adds — the sha256-checked ossutil install, the credential-config step, and the if:always() cleanup — so a future edit dropping any of them (or the always() condition) would leave every test green while the OSS key pair persists in $RUNNER_TEMP on the persistent ecs-qwen pool. Add shape assertions for all three steps in qwen-triage.yml and web-shell-visuals-publish.yml, mirroring the release/desktop wiring pins in install-script.test.js and desktop-oss-workflow.test.js. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * docs: align assets-repo guidance with the self-targeting guard The PR-review workflow now blanks a QWEN_REVIEW_ASSETS_REPO that equals the repository under review, but the user doc, the bundled review skill, and the parseAssetsRepo unset-error still recommended pointing it at the repo under review — so a maintainer following the published guidance landed in the "not set" refusal path, and the refusal message re-recommended the exact value the guard had just rejected. Update all three surfaces to the new contract: a dedicated external image-host repository (or a fork/scratch repo), with unset or self-targeting values deliberately degrading to prose and local artifact paths. Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> * test: execute the visuals hosting block against a stub uploader The workflow-level assertions covered the OSS upload path with shape regexes only, so a --prefix that drifts away from RAW_BASE (every preview URL 404s) or a dropped --config line (the uploader hard-fails before any comment posts) both left the suite green. Extract the hosting block and run it against a stub uploader that mirrors the real flag contract and records the upload destination, in the same pattern as the verify path in scripts/tests/qwen-triage-workflow.test.js: the images arm must land the staged files under $STUB_ROOT/$ALIYUN_OSS_BUCKET/pr-assets/web-shell-visuals/<pr>/<sha> with the exact --bucket/--config/--prefix wiring, and RAW_BASE must equal that layout; the no-change arm must never invoke the uploader. Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> * fix(ci): gate ossutil consumers on verified install Prevent the persistent publish runner from executing a stale binary under RUNNER_TEMP when the best-effort install step fails. Pin both the install outcome gate and the hardened PATH invocation in focused workflow tests. Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(ci): grant publisher checkout access Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * test(ci): pin OSS publisher failure paths Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * test(ci): exercise production OSS dispatch Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(ci): isolate OSS publisher on hosted runner * fix(ci): sync size baseline and stale runner comments Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> * fix(ci): give each visuals publish a fresh OSS prefix The web-shell prefix was pr-assets/web-shell-visuals/<pr>/<head-sha>, so a re-run for the same head wrote back over the object keys the already-posted comment referenced. GitHub serves comment images through a caching proxy, so reviewers would keep seeing the previous run's screenshots at a URL whose bytes had changed. The Git-backed design got cache-busting free from the per-run commit SHA; add the run id back to restore it (the verify lane already carries <run-id>-<attempt>). Also: - Resolve the destination as ALIYUN_OSS_PR_ASSETS_BUCKET -> ALIYUN_OSS_BUCKET -> qwen-code-assets in both publishers, with the public base URL derived from whichever wins. Unset, this is byte-for-byte today's behaviour; set, it moves untrusted PR-derived evidence off the bucket that also serves release, desktop, and live-host downloads without a workflow change. - Sparse-check out /package.json alongside the uploader: it is ESM in a .js file, and package.json declares engines >= 22.0 while neither publisher pins a Node, so without the "type": "module" marker the uploader only parses on the Node versions that infer module syntax. - Require the credential file, not just a green install outcome, before dispatching the /verify upload. The configure step is continue-on-error, so a missing or rotated secret otherwise cost three retry backoffs per image before degrading to the same text-only report. - Drop the stale "runs on the persistent ECS pool" premise from the clear-stale-downloads step; publish-verify is on ubuntu-latest now. * fix(ci): retry the ossutil download in both PR publishers The release syncs download ossutil with no retry, which is fine there: a red run is rerun by hand. Here the consequence differs. A CDN blip in the web-shell publisher leaves no ossutil for the upload, and since the upload aborts the step under set -e, the PR loses its preview comment entirely rather than degrading; in the verify publisher it silently costs the report its evidence images. Three retries with --retry-all-errors covers the transient class without changing any success path. Also reword the unset-QWEN_REVIEW_ASSETS_REPO refusal. It ended with a flat claim about the PR-review workflow blanking a self-targeting designation, which reads as a non sequitur to the far more common reader: someone running the CLI locally who simply never set the variable. Keep the diagnostic — a maintainer whose repository variable IS set needs it — but scope it to CI and say why self-targeting is discouraged in the first place. * fix(ci): include run attempt in the web-shell OSS asset prefix The workflow_run id is stable across re-run attempts, so a maintainer re-run of the visuals publish overwrote the exact object keys an already-posted comment references (camo keeps rendering attempt-1 shots). Append run_attempt to the prefix, mirroring the verify lane, and cover same-runId attempt-1-vs-attempt-2 in the hosting harness. Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> * fix(ci): close out PR evidence hosting review findings - The web-shell visuals prefix gains the run attempt: a re-run keeps the run id and only increments the attempt, so the old prefix let a same-head re-run write back over the object keys an already-posted comment references (camo keeps serving the stale screenshots). Mirrors the verify lane <run-id>-<attempt>; the re-run test now models same-run attempt 1 vs 2, not just distinct run ids. - The QWEN_REVIEW_ASSETS_REPO self-targeting guard now trims and compares case-insensitively before the CLI reads the value, so padded or case-shifted self-references degrade like an unset one. - Both publisher checkouts pin ref: github.sha (the default-branch head on their events) instead of relying on default resolution. - The ossutil download retry budget fits the 10-minute job cap in both publishers (~6.1 min worst case; was ~20 min). - Upload attempts are bounded via OSS_UPLOAD_ATTEMPT_TIMEOUT_MS so a stalled ossutil degrades to the text-only report / re-triggerable publish instead of burning the job cap; release syncs stay unbounded (the knob is opt-in). - Fix the serving-model comment (the extension allowlist, not the magic bytes, is what pins the served Content-Type) and pin what earlier suites left unpinned: sparse-checkout lists, bucket-to-URL derivation, curl retry flags, persist-credentials, and the ossutil isolation lines. Both uploader stubs now require the --config file to exist, so config drift turns the harnesses red. - runHostingBlock tears down its mkdtemp fixture instead of leaking it on every call. # Conflicts: # .github/scripts/web-shell-visuals-publish.test.mjs # .github/workflows/web-shell-visuals-publish.yml # docs/design/2026-08-25-pr-evidence-oss-hosting.md * fix(ci): gate visuals-publish Configure step on ossutil install outcome The Install ossutil step keeps continue-on-error: true so a failed install must not block the no-image marker-comment path, but the unguarded Configure Aliyun OSS credentials step still ran and failed confusingly on the missing binary. Give the install step the install-ossutil id and gate Configure on its success outcome, matching the qwen-triage.yml twins. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> --------- Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> Co-authored-by: Shaojin Wen <shaojin.wensj@alibaba-inc.com>
201 lines
5.2 KiB
JavaScript
201 lines
5.2 KiB
JavaScript
#!/usr/bin/env node
|
|
|
|
/**
|
|
* @license
|
|
* Copyright 2026 Qwen Team
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*/
|
|
|
|
import path from 'node:path';
|
|
import { spawnSync } from 'node:child_process';
|
|
import { fail, isMainModule, readOptionValue } from './release-script-utils.js';
|
|
|
|
const MAX_UPLOAD_ATTEMPTS = 3;
|
|
const INITIAL_BACKOFF_MS = 2000;
|
|
const ATTEMPT_TIMEOUT_ENV = 'OSS_UPLOAD_ATTEMPT_TIMEOUT_MS';
|
|
|
|
if (isMainModule(import.meta.url)) {
|
|
try {
|
|
main(process.argv.slice(2));
|
|
} catch (error) {
|
|
console.error(error instanceof Error ? error.message : error);
|
|
process.exitCode = 1;
|
|
}
|
|
}
|
|
|
|
function main(argv) {
|
|
const args = parseUploadArgs(argv);
|
|
if (args.help) {
|
|
printUsage();
|
|
return;
|
|
}
|
|
uploadAssets(args, { attemptTimeoutMs: resolveAttemptTimeoutMs() });
|
|
}
|
|
|
|
// A stalled ossutil (a black-hole socket that accepts but never progresses)
|
|
// must not hang the caller forever: the PR publishers run inside a
|
|
// 10-minute job cap, and one stuck upload would burn the cap and lose the
|
|
// whole report/comment. The bound is opt-in via env so the release syncs —
|
|
// which move much larger files — keep today's unbounded behaviour exactly.
|
|
function resolveAttemptTimeoutMs() {
|
|
const raw = process.env[ATTEMPT_TIMEOUT_ENV];
|
|
if (!raw) {
|
|
return 0;
|
|
}
|
|
const value = Number(raw);
|
|
if (!Number.isInteger(value) || value < 0) {
|
|
fail(
|
|
`${ATTEMPT_TIMEOUT_ENV} must be a non-negative integer of milliseconds, got ${JSON.stringify(raw)}.`,
|
|
);
|
|
}
|
|
return value;
|
|
}
|
|
|
|
function printUsage() {
|
|
console.log(`Usage: node scripts/upload-aliyun-oss-assets.js [options] ASSET...
|
|
|
|
Uploads local assets to a public Aliyun OSS prefix via ossutil.
|
|
|
|
Options:
|
|
--bucket NAME OSS bucket name.
|
|
--config PATH ossutil config path.
|
|
--prefix PREFIX Destination object prefix.
|
|
-h, --help Show this help message.
|
|
`);
|
|
}
|
|
|
|
function parseUploadArgs(argv) {
|
|
const args = {
|
|
assets: [],
|
|
bucket: '',
|
|
config: '',
|
|
help: false,
|
|
prefix: '',
|
|
};
|
|
|
|
for (let index = 0; index < argv.length; index += 1) {
|
|
const arg = argv[index];
|
|
if (arg === '--help' || arg === '-h') {
|
|
args.help = true;
|
|
continue;
|
|
}
|
|
if (arg === '--bucket') {
|
|
args.bucket = readOptionValue(argv, index, arg);
|
|
index += 1;
|
|
continue;
|
|
}
|
|
if (arg === '--config') {
|
|
args.config = readOptionValue(argv, index, arg);
|
|
index += 1;
|
|
continue;
|
|
}
|
|
if (arg === '--prefix') {
|
|
args.prefix = readOptionValue(argv, index, arg).replace(/\/+$/, '');
|
|
index += 1;
|
|
continue;
|
|
}
|
|
if (arg.startsWith('-')) {
|
|
fail(`Unknown option: ${arg}`);
|
|
}
|
|
args.assets.push(arg);
|
|
}
|
|
|
|
if (args.help) {
|
|
return args;
|
|
}
|
|
if (!args.bucket) {
|
|
fail('--bucket requires a value');
|
|
}
|
|
if (!args.config) {
|
|
fail('--config requires a value');
|
|
}
|
|
if (!args.prefix) {
|
|
fail('--prefix requires a value');
|
|
}
|
|
if (args.assets.length === 0) {
|
|
fail('At least one ASSET path is required');
|
|
}
|
|
|
|
return args;
|
|
}
|
|
|
|
function uploadAssets(
|
|
{ assets, bucket, config, prefix },
|
|
{
|
|
ossutilCommand = 'ossutil',
|
|
ossutilCommandArgs = [],
|
|
attemptTimeoutMs = 0,
|
|
} = {},
|
|
) {
|
|
for (const asset of assets) {
|
|
const key = `${prefix}/${path.basename(asset)}`;
|
|
uploadWithRetry(asset, bucket, key, config, {
|
|
ossutilCommand,
|
|
ossutilCommandArgs,
|
|
attemptTimeoutMs,
|
|
});
|
|
}
|
|
}
|
|
|
|
function uploadWithRetry(
|
|
asset,
|
|
bucket,
|
|
key,
|
|
config,
|
|
{ ossutilCommand, ossutilCommandArgs, attemptTimeoutMs },
|
|
) {
|
|
for (let attempt = 1; attempt <= MAX_UPLOAD_ATTEMPTS; attempt += 1) {
|
|
const result = spawnSync(
|
|
ossutilCommand,
|
|
[
|
|
...ossutilCommandArgs,
|
|
'cp',
|
|
asset,
|
|
`oss://${bucket}/${key}`,
|
|
'-c',
|
|
config,
|
|
'-f',
|
|
'--acl',
|
|
'public-read',
|
|
],
|
|
{
|
|
stdio: 'inherit',
|
|
// 0 disables the bound (the release syncs' default); a positive
|
|
// value SIGKILLs an attempt that stalls past it instead of letting
|
|
// it hang the caller out to the job cap.
|
|
...(attemptTimeoutMs > 0
|
|
? { timeout: attemptTimeoutMs, killSignal: 'SIGKILL' }
|
|
: {}),
|
|
},
|
|
);
|
|
|
|
// A timeout surfaces as error.code ETIMEDOUT and a killed child — that
|
|
// is a retryable attempt failure, not a spawn error to rethrow.
|
|
if (result.error && result.error.code !== 'ETIMEDOUT') {
|
|
throw result.error;
|
|
}
|
|
if (result.status === 0) {
|
|
return;
|
|
}
|
|
if (attempt < MAX_UPLOAD_ATTEMPTS) {
|
|
const delayMs = INITIAL_BACKOFF_MS * 2 ** (attempt - 1);
|
|
console.warn(
|
|
`Upload attempt ${attempt}/${MAX_UPLOAD_ATTEMPTS} failed for ${path.basename(asset)}${
|
|
result.error ? ' (timed out)' : ''
|
|
}, retrying in ${delayMs / 1000}s...`,
|
|
);
|
|
sleepSync(delayMs);
|
|
}
|
|
}
|
|
fail(
|
|
`ossutil failed after ${MAX_UPLOAD_ATTEMPTS} attempts while uploading ${asset}`,
|
|
);
|
|
}
|
|
|
|
// Cross-platform synchronous sleep. `spawnSync('sleep', ...)` is unavailable
|
|
// on Windows runners; Atomics.wait blocks the current thread without spawning.
|
|
function sleepSync(ms) {
|
|
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
|
|
}
|
|
|
|
export { parseUploadArgs, resolveAttemptTimeoutMs, uploadAssets };
|