fix(ci): tell a triage action crash apart from a silent agent (#7418)

Both no-response cases produced the same error, and its advice — "check
the 'Run Qwen Triage' step stderr above for diagnostics" — pointed at
diagnostics that do not exist for one of them: the action installs the
CLI with `npm --silent`, so an install failure prints an exit code and
nothing else. Observed on a PR whose triage died at exit 243 during that
install, leaving a maintainer told to read an empty log.

The two causes need opposite responses, and steps.triage.outcome already
distinguishes them:

- action failed  -> no model call happened, nothing about the PR can
                    explain it, and the fix is to re-run the job
- action succeeded, empty summary -> the agent ran and returned nothing,
                    which IS worth reading the step output for

Measured baseline for the retry advice: of 19 executed triage runs, 4
failed, 3 of them on unrelated PRs, and one PR both succeeded and failed
within an hour on the same head.

The outcome is passed through env like RESPONSE already is, and the
script is replayed under bash for both branches.

Co-authored-by: wenshao <wenshao@example.com>
This commit is contained in:
Shaojin Wen 2026-07-21 19:07:11 +08:00 committed by GitHub
parent 32ef628f95
commit 837358f637
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 64 additions and 1 deletions

View file

@ -4,6 +4,7 @@
* SPDX-License-Identifier: Apache-2.0
*/
import { spawnSync } from 'node:child_process';
import { readFileSync } from 'node:fs';
import { describe, expect, it } from 'vitest';
@ -124,6 +125,53 @@ describe('qwen-triage tmux workflow', () => {
expect(checkStep).toContain('if [[ -z "${RESPONSE}"');
});
it('tells an action crash apart from a silent agent, and replays both', () => {
const checkStep = step('Check triage response');
// The outcome must arrive through env like RESPONSE does — inlining the
// expression into the script would be the injection shape this step
// already avoids for RESPONSE.
expect(checkStep).toContain(
"TRIAGE_OUTCOME: '${{ steps.triage.outcome }}'",
);
expect(checkStep).not.toContain('TRIAGE_OUTCOME="${{');
const body = checkStep.match(/run: \|-\n([\s\S]*)$/)?.[1];
expect(body).toBeTruthy();
const script = body.replace(/^ {10}/gm, '');
const run = (env) => {
const proc = spawnSync('bash', ['-c', script], {
env: {
...process.env,
RESPONSE: '',
TRIAGE_OUTCOME: 'success',
...env,
},
encoding: 'utf8',
});
return { status: proc.status, out: `${proc.stdout}${proc.stderr}` };
};
// A crashed action: no model call happened, so nothing about the PR can
// explain it and the guidance must say "re-run", not "read the log" —
// the install runs under `npm --silent`, so there is no log to read.
const crashed = run({ TRIAGE_OUTCOME: 'failure' });
expect(crashed.status).not.toBe(0);
expect(crashed.out).toContain('Triage did not start');
expect(crashed.out).toContain('re-run the failed job');
expect(crashed.out).not.toContain('Triage silent failure');
// A completed action with no summary IS worth reading the step output for.
const silent = run({ TRIAGE_OUTCOME: 'success' });
expect(silent.status).not.toBe(0);
expect(silent.out).toContain('Triage silent failure');
expect(silent.out).toContain('model or prompt problem');
expect(silent.out).not.toContain('Triage did not start');
// A real response still passes, and 'null' still counts as no response.
expect(run({ RESPONSE: 'triaged' }).status).toBe(0);
expect(run({ RESPONSE: 'null' }).status).not.toBe(0);
});
it('notifies the author when a manual triage re-run posts no review', () => {
const notifyStep = step('Notify silent triage re-run');