vrr/unsloth
2
0
Fork 0
mirror of https://github.com/unslothai/unsloth.git synced 2026-08-23 15:53:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 29

Studio CI: make a failing browser smoke say why, and stop it skipping the build gates (#8983)

Browse source 
* Fix two module resolution failures in the frontend test suite

* Studio CI: make a failing browser smoke say why, and stop it skipping the build gates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix the third Windows-only path failure in the frontend test suite

* Drop the #8980 content this branch no longer needs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Snapshot the vite tail before printing it, and put the startup bundle gate ahead of the smokes

The failure dump iterated the live deque the drain thread is still appending to,
while writing each line to stdout, so a vite server that was still talking during
the dump raised deque mutated during iteration and dropped the tail in exactly the
noisy failure the dump was added for. list() of a deque is atomic; take it first.

Startup bundle budget still ran after the browser smokes, and every step carries an
implicit success(), so a red smoke skipped it. It only needs dist/, so it moves up
with the other build gates.

* Tighten comments in the smoke diagnostics changes

* Upload the failing smoke's own report, not every report but that one

The failure upload globbed logs/playwright-*, which four of the five browser
smokes write. playwright_settings_tabs.py writes logs/settings_tabs_report.json
and, for the blocked-chunk arm, logs/settings_tabs_blocked_report.json. So when
either settings smoke failed the artifact contained the reports of the smokes
that had passed and not the one that had just failed, which is the opposite of
what this upload is for.

Add both names to the upload path, and guard it: the new test reads every
logs/ path the wired-up smokes actually write out of their own source and
fails if one is not matched by an upload pattern. Shown red on the bare glob
first, naming logs/settings_tabs_report.json.

* Upload the non-blocking smoke's report on the runs where it is the point

The stream-pacing smoke is continue-on-error, which rewrites its CONCLUSION
to success while leaving its OUTCOME as failure. The artifact upload was
gated on a bare failure(), so on the runs where that smoke was the only thing
that failed -- exactly the runs where its report is the whole point -- the
upload was skipped and logs/playwright-stream-pacing went nowhere.

Give the step an id and OR its raw outcome into the upload condition.

Guarded: the new test walks every continue-on-error browser smoke and fails
if it has no id, or if its outcome is not named in the upload condition.
Shown red two independent ways first, reverting the condition to bare
failure() and separately deleting the step id, each naming the step.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
This commit is contained in:
Daniel Han 2026-08-18 04:42:41 -07:00 • committed by GitHub
parent 58403dd001
commit aa32c1861c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 329 additions and 83 deletions
Download patch file Download diff file Expand all files Collapse all files

170
.github/workflows/studio-frontend-ci.yml vendored
View file

@ -162,78 +162,6 @@ jobs:
run: npm test
- name: Install Chromium for browser smokes
working-directory: ${{ github.workspace }}
run: |
python3 -m pip install 'playwright>=1.45,<2' pytest
python3 -m playwright install --with-deps chromium
# Cheap and browserless, but it imports the harnesses, which import playwright, so it has
# to sit after the install. Covers the Windows teardown branch no runner here executes.
- name: Dev-server lifecycle tests
working-directory: ${{ github.workspace }}
run: |
python3 -m pytest tests/studio/test_playwright_server_lifecycle.py \
tests/studio/test_autoscroll_harness_contract.py \
tests/studio/test_heavy_thread_harness_contract.py \
tests/studio/test_heavy_thread_measurement_integrity.py \
tests/studio/test_heavy_thread_gap_contract.py -q
- name: Browser smoke for ANSI tool output
working-directory: ${{ github.workspace }}
env:
PW_BROWSER: chromium
run: python3 tests/studio/playwright_strip_ansi_smoke.py
# Each harness owns its vite dev server, like the ANSI smoke above. Starting one here
# would background npm, so $! is the wrapper and killing it orphans the node child.
- name: Browser smoke for chat autoscroll
working-directory: ${{ github.workspace }}
run: python3 tests/studio/playwright_chat_autoscroll.py
- name: Browser smoke for research freeze
working-directory: ${{ github.workspace }}
run: python3 tests/studio/playwright_research_freeze.py
# Two small sizes and one repetition on Chromium: enough to prove the fixture still
# renders every kind of content it claims to and that the curve still rises, which is all
# a PR gate can afford. The measurement this harness exists for is the three-engine,
# three-size, three-repetition run, which takes tens of minutes and belongs on a runner
# asked for it deliberately, not on every frontend PR.
- name: Browser smoke for heavy-thread interaction cost
working-directory: ${{ github.workspace }}
env:
SMOKE_HEAVY_CHARS: '25000,100000'
SMOKE_HEAVY_ENGINES: chromium
SMOKE_HEAVY_REPEATS: '1'
PW_ART_DIR: logs/playwright_heavy_thread
run: python3 tests/studio/playwright_heavy_thread.py
# The settings panels are fetched on first view, so only a browser can answer whether
# every tab renders and deep-opens land (and abandoned ones do not come back).
- name: Browser smoke for the settings tab panels
working-directory: ${{ github.workspace }}
run: python3 tests/studio/playwright_settings_tabs.py
# A panel that cannot be fetched is new with lazy loading, and nothing above the
# root-mounted dialog catches, so unguarded it unmounts Studio, not one panel.
- name: Browser smoke for a settings panel that cannot load
working-directory: ${{ github.workspace }}
env:
PW_CHUNK_FAIL: data
PW_PORT: '5400'
PW_OUT: logs/settings_tabs_blocked_report.json
run: python3 tests/studio/playwright_settings_tabs.py
# Reports, does not gate, for now. The budgets inside are calibrated on a
# developer machine, and this measures a CPU-bound render on a shared runner
# under 6x throttling, so a budget set from one box is a flake waiting to
# happen. Tighten it from observed runs here and drop continue-on-error, the
# way the startup profile did.
- name: Browser smoke for chat stream pacing
working-directory: ${{ github.workspace }}
continue-on-error: true
run: python3 tests/studio/playwright_stream_pacing.py
- name: Build
run: npm run build
@ -265,6 +193,104 @@ jobs:
- name: Startup bundle budget
run: npm run bundle:check
# Smokes go last: every step carries an implicit success(), so running them ahead of
# the build gates let one red smoke skip the build and all three bundle assertions.
# Costs nothing here, since each smoke starts its own vite server and reads no dist/.
- name: Install Chromium for browser smokes
working-directory: ${{ github.workspace }}
run: |
python3 -m pip install 'playwright>=1.45,<2' pytest
python3 -m playwright install --with-deps chromium
# Browserless, but imports the harnesses (hence playwright), so it sits after the
# install. Covers the Windows teardown branch no runner here executes.
- name: Dev-server lifecycle tests
working-directory: ${{ github.workspace }}
run: |
python3 -m pytest tests/studio/test_playwright_server_lifecycle.py \
tests/studio/test_autoscroll_harness_contract.py \
tests/studio/test_heavy_thread_harness_contract.py \
tests/studio/test_heavy_thread_measurement_integrity.py \
tests/studio/test_heavy_thread_gap_contract.py -q
- name: Browser smoke for ANSI tool output
working-directory: ${{ github.workspace }}
env:
PW_BROWSER: chromium
run: python3 tests/studio/playwright_strip_ansi_smoke.py
# Each harness owns its vite server: starting one here backgrounds npm, so $! is the
# wrapper and killing it orphans the node child.
- name: Browser smoke for chat autoscroll
working-directory: ${{ github.workspace }}
run: python3 tests/studio/playwright_chat_autoscroll.py
- name: Browser smoke for research freeze
working-directory: ${{ github.workspace }}
run: python3 tests/studio/playwright_research_freeze.py
# Two small sizes and one repetition on Chromium: enough to prove the fixture still
# renders every kind of content it claims to and that the curve still rises, which is all
# a PR gate can afford. The measurement this harness exists for is the three-engine,
# three-size, three-repetition run, which takes tens of minutes and belongs on a runner
# asked for it deliberately, not on every frontend PR.
- name: Browser smoke for heavy-thread interaction cost
working-directory: ${{ github.workspace }}
env:
SMOKE_HEAVY_CHARS: '25000,100000'
SMOKE_HEAVY_ENGINES: chromium
SMOKE_HEAVY_REPEATS: '1'
PW_ART_DIR: logs/playwright_heavy_thread
run: python3 tests/studio/playwright_heavy_thread.py
# The settings panels are fetched on first view, so only a browser can answer whether
# every tab renders and deep-opens land (and abandoned ones do not come back).
- name: Browser smoke for the settings tab panels
working-directory: ${{ github.workspace }}
run: python3 tests/studio/playwright_settings_tabs.py
# A panel that cannot be fetched is new with lazy loading, and nothing above the
# root-mounted dialog catches, so unguarded it unmounts Studio, not one panel.
- name: Browser smoke for a settings panel that cannot load
working-directory: ${{ github.workspace }}
env:
PW_CHUNK_FAIL: data
PW_PORT: '5400'
PW_OUT: logs/settings_tabs_blocked_report.json
run: python3 tests/studio/playwright_settings_tabs.py
# Reports, does not gate, for now. The budgets inside are calibrated on a
# developer machine, and this measures a CPU-bound render on a shared runner
# under 6x throttling, so a budget set from one box is a flake waiting to
# happen. Tighten it from observed runs here and drop continue-on-error, the
# way the startup profile did.
- name: Browser smoke for chat stream pacing
id: stream_pacing
working-directory: ${{ github.workspace }}
continue-on-error: true
run: python3 tests/studio/playwright_stream_pacing.py
# Screenshot and serialized DOM the harnesses write on failure; small, kept only then.
#
# `failure()` alone is not enough. The stream-pacing smoke is `continue-on-error`, which
# rewrites its CONCLUSION to success while leaving its OUTCOME as failure, so on the runs
# where it is the only thing that failed -- the runs where its report is the whole point --
# `failure()` is false and the report went nowhere. Check its raw outcome too.
- name: Upload browser smoke artifacts
if: failure() || steps.stream_pacing.outcome == 'failure'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: studio-frontend-smoke-artifacts
# Four of the five smokes write logs/playwright-<name>; the settings smoke writes a
# JSON report instead, under its own name, so a bare playwright-* glob uploaded
# everything EXCEPT the report of the smoke that had just failed.
path: |
logs/playwright-*
logs/settings_tabs_report.json
logs/settings_tabs_blocked_report.json
retention-days: 3
if-no-files-found: ignore
- name: Biome (non-blocking until accumulated drift is cleared)
continue-on-error: true
run: npm run biome:check

22
tests/studio/_playwright_robust.py
View file

@ -431,6 +431,28 @@ def is_benign_console_error(msg: str) -> bool:
return any(p in msg for p in BENIGN_CONSOLE_ERROR_PATTERNS)
def echo_browser_errors(page: Any, info: Callable[[str], None]) -> None:
"""Print what the browser knows, live, as it happens.
A harness that only asserts on the DOM cannot tell an entry module that threw
from one that is merely slow: both end as an `expect(...)` timeout on a locator
that was never created, under an empty CI log. The smokes each own a throwaway
page, so printing straight through beats collecting for a caller to forward.
"""
page.on("pageerror", lambda e: info(f"pageerror: {e}"))
page.on(
"console",
lambda m: info(f"console.{m.type}: {m.text}") if m.type == "error" else None,
)
page.on("requestfailed", lambda r: info(f"requestfailed: {r.url} {r.failure}"))
# Vite reloads the page after re-optimizing a late-discovered dep, unmounting the
# tree mid-assertion. Name it if it happens.
page.on(
"framenavigated",
lambda f: info(f"navigated: {f.url}") if f is page.main_frame else None,
)
# ─────────────────────────────────────────────────────────────────────
# Diagnostic dump.
# ─────────────────────────────────────────────────────────────────────

2
tests/studio/playwright_chat_autoscroll.py
View file

@ -45,6 +45,7 @@ from playwright.sync_api import sync_playwright
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _playwright_robust import ( # noqa: E402
chromium_launch_args,
echo_browser_errors,
start_vite,
stop_process,
wait_for_smoke_page,
@ -140,6 +141,7 @@ def run() -> dict:
lambda route: route.fulfill(status = 200, content_type = "application/json", body = "{}"),
)
page = context.new_page()
echo_browser_errors(page, info)
page.goto(f"{BASE}/smoke-autoscroll.html", wait_until = "domcontentloaded")
page.wait_for_function("() => Boolean(window.__autoscroll)", timeout = 30_000)
cdp = context.new_cdp_session(page)

2
tests/studio/playwright_research_freeze.py
View file

@ -37,6 +37,7 @@ from playwright.sync_api import sync_playwright
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _playwright_robust import ( # noqa: E402
chromium_launch_args,
echo_browser_errors,
start_vite,
stop_process,
wait_for_smoke_page,
@ -184,6 +185,7 @@ def run() -> dict:
lambda route: route.fulfill(status = 200, content_type = "application/json", body = "{}"),
)
page = context.new_page()
echo_browser_errors(page, info)
page.goto(f"{BASE}/smoke-research.html", wait_until = "domcontentloaded")
page.wait_for_function("() => Boolean(window.__research)", timeout = 30_000)
cdp = context.new_cdp_session(page)

50
tests/studio/playwright_strip_ansi_smoke.py
View file

@ -7,14 +7,17 @@ from __future__ import annotations
import os
import subprocess
import sys
from pathlib import Path
from playwright.sync_api import expect, sync_playwright
from playwright.sync_api import Page, expect, sync_playwright
sys.path.insert(0, str(Path(__file__).resolve().parent))
from _playwright_robust import ( # noqa: E402
chromium_launch_args,
dump_diagnostics,
echo_browser_errors,
start_vite,
stop_process,
wait_for_smoke_page,
@ -41,6 +44,24 @@ def info(msg: str) -> None:
print(f"[ansi-smoke] {msg}", flush = True)
def dump(page: Page, vite: subprocess.Popen[str] | None) -> None:
"""Write down what the page actually was, since CI keeps no live browser.
`dump_diagnostics` records the browser side (screenshot, URL, body excerpt). The
dev server's own output is the other half: a transform error or a forced reload
is reported there and nowhere else.
"""
dump_diagnostics(page, ART, "smoke-ansi-failure", info = info)
if vite is not None:
info("vite tail:")
# Snapshot first: the drain thread is still appending, and printing releases the
# GIL, so lazy iteration raises "deque mutated during iteration" and loses the
# tail in the noisy failure it exists for. `list()` runs in C, so it is atomic.
for line in list(getattr(vite, "vite_tail", [])) or ["(no output)"]:
info(f" {line.rstrip()}")
info(f"artifacts in {ART}")
def main() -> None:
ART.mkdir(parents = True, exist_ok = True)
if OWNS_SERVER:
@ -57,17 +78,24 @@ def main() -> None:
launch_args = chromium_launch_args() if browser_name == "chromium" else []
browser = browser_type.launch(headless = True, args = launch_args)
page = browser.new_page()
page.goto(f"{BASE}/smoke-ansi.html", wait_until = "networkidle")
page.screenshot(path = str(ART / "smoke-ansi.png"), full_page = True)
echo_browser_errors(page, info)
try:
page.goto(f"{BASE}/smoke-ansi.html", wait_until = "networkidle")
page.screenshot(path = str(ART / "smoke-ansi.png"), full_page = True)
for section in SECTIONS:
pane = page.locator(f'section[data-smoke="{section}"] pre').first
expect(pane).to_be_visible(timeout = 15_000)
text = pane.inner_text()
info(f"{section} text: {text!r}")
assert text == "file.txt\nerror", f"{section} rendered unexpected text: {text!r}"
assert ESC not in text, f"{section} still contains ESC"
assert "[32m" not in text, f"{section} still shows SGR garbage"
for section in SECTIONS:
pane = page.locator(f'section[data-smoke="{section}"] pre').first
expect(pane).to_be_visible(timeout = 15_000)
text = pane.inner_text()
info(f"{section} text: {text!r}")
assert (
text == "file.txt\nerror"
), f"{section} rendered unexpected text: {text!r}"
assert ESC not in text, f"{section} still contains ESC"
assert "[32m" not in text, f"{section} still shows SGR garbage"
except Exception:
dump(page, vite)
raise
info("all production panes rendered clean text (no ANSI escapes)")
browser.close()

68
tests/studio/test_autoscroll_harness_contract.py
View file

@ -10,6 +10,7 @@ budget, an unasserted click count, and the two guards below), so the rule is pin
than left to review.
"""
import types
from pathlib import Path
@ -72,6 +73,73 @@ def test_research_freeze_keeps_a_hit_tested_click_in_the_report_phase() -> None:
assert 'results["report"]["clicks_registered"]' in main
def test_harnesses_report_why_the_page_failed() -> None:
# A thrown entry module and a merely slow one both end as a timeout on a locator that
# was never created. Run 31935573269 was that: 15s of nothing, no console, no page
# error, no server output, on 7 of the 8 runs that reached this step.
for name in (
"playwright_chat_autoscroll.py",
"playwright_research_freeze.py",
"playwright_strip_ansi_smoke.py",
):
assert "echo_browser_errors(page, info)" in source(
name
), f"{name} discards pageerror and console.error, so a crashed page reads as a timeout"
def test_ansi_smoke_keeps_the_failed_page_and_the_server_output() -> None:
# The live log dies with the runner; the screenshot, body excerpt and vite's own
# transform errors are what remains.
text = source("playwright_strip_ansi_smoke.py")
assert "dump(page, vite)" in text, "the assertions do not run under the dump"
assert "dump_diagnostics(page, ART" in text
assert 'getattr(vite, "vite_tail"' in text, "vite's output is dropped on failure"
def test_the_ansi_dump_survives_a_vite_server_that_is_still_talking(tmp_path, monkeypatch) -> None:
# A daemon thread appends to the tail deque for as long as vite lives, and the dump
# runs before the server stops. Iterating it live while printing (stdout releases the
# GIL) raises "deque mutated during iteration", losing the tail in the one case it was
# added for: a reload or transform storm.
import importlib.util
import threading
from collections import deque
import pytest
pytest.importorskip("playwright")
monkeypatch.setenv("PW_ART_DIR", str(tmp_path / "art"))
spec = importlib.util.spec_from_file_location(
"_ansi_smoke_under_test", STUDIO_TESTS / "playwright_strip_ansi_smoke.py"
)
assert spec is not None and spec.loader is not None
smoke = importlib.util.module_from_spec(spec)
spec.loader.exec_module(smoke)
tail: deque[str] = deque(maxlen = 20)
for index in range(tail.maxlen or 20):
tail.append(f"vite line {index}")
vite = types.SimpleNamespace(vite_tail = tail)
stop = threading.Event()
def keep_talking() -> None:
index = 0
while not stop.is_set():
tail.append(f"[vite] page reload {index}")
index += 1
talker = threading.Thread(target = keep_talking, daemon = True)
talker.start()
try:
for _ in range(5):
# `page` is unused by the tail print and dump_diagnostics is best-effort,
# so a stub reaches the loop.
smoke.dump(types.SimpleNamespace(), vite)
finally:
stop.set()
talker.join(timeout = 5)
def test_stream_pacing_asserts_its_long_task_probe_measured_something() -> None:
# longTaskMs is the metric the budgets turn on, and it is 0 both when the render is free
# and when the observer never ran: `observe({type: "longtask"})` aborts silently on an

98
tests/studio/test_playwright_suites_run_in_ci.py
View file

@ -14,6 +14,7 @@ failure: the list of what CI runs drifting behind the directory it runs from.
"""
import re
from fnmatch import fnmatch
from pathlib import Path
import yaml
@ -209,6 +210,35 @@ def test_the_linux_job_still_drives_all_three_browser_engines():
assert "chromium" not in disabled and "webkit" not in disabled
def test_no_build_gate_sits_behind_a_browser_smoke():
"""A smoke failure must not decide whether the build gates report.
Every step carries an implicit `if: success()`, so a job stops at its first failing
step and skips the rest. The ANSI smoke is intermittently red, and while it ran ahead
of them the build and the three bundle assertions never reported at all on those runs.
The smokes each start their own vite dev server and read nothing out of `dist/`, so
they belong last. Asserted by step index, since the ordering is the whole guarantee.
"""
document = yaml.safe_load(
(REPO / ".github" / "workflows" / "studio-frontend-ci.yml").read_text(encoding = "utf-8")
)
names = [str(step.get("name", "")) for step in document["jobs"]["build"]["steps"]]
gates = [
"Build",
"Built bundle must not contain Unsloth's unstable_Provider call site",
"Bundle size budget (75 MB)",
"Startup bundle budget",
]
missing = [gate for gate in gates if gate not in names]
assert not missing, f"renamed or deleted build gates: {missing}; update this list"
first_smoke = min(index for index, name in enumerate(names) if name.startswith("Browser smoke"))
late = [gate for gate in gates if names.index(gate) > first_smoke]
assert not late, (
f"{late} run after {names[first_smoke]!r}, so a red browser smoke skips them and the "
f"checks that decide whether the app ships never report. Move them above the smokes."
)
def test_the_scan_reads_the_workflows_it_claims_to():
"""A scan that read nothing would pass both checks above on anything."""
assert len(DRIVERS) > 10, f"only found {len(DRIVERS)} drivers; the glob is wrong"
@ -225,3 +255,71 @@ def test_the_scan_reads_the_workflows_it_claims_to():
"the composite action's own contents are not in the text, so a driver launched "
"from inside one would read as an orphan"
)
def test_every_smoke_report_is_covered_by_the_failure_upload():
"""A smoke that fails must have its own diagnostic in the artifact.
The upload runs `if: failure()`, so the ONE report worth having is the one the
smoke that just failed wrote. Four of the five write `logs/playwright-<name>`;
the settings smoke writes a JSON report under its own name, so a bare
`logs/playwright-*` path uploaded every report except that one.
"""
workflow = yaml.safe_load(
(REPO / ".github" / "workflows" / "studio-frontend-ci.yml").read_text(encoding = "utf-8")
)
steps = workflow["jobs"]["build"]["steps"]
upload = next(s for s in steps if s.get("name") == "Upload browser smoke artifacts")
patterns = [line.strip() for line in str(upload["with"]["path"]).splitlines() if line.strip()]
# Every logs/ path the smokes CI runs actually write, read from their source.
run = " ".join(str(s.get("run", "")) for s in steps)
smokes = [d for d in DRIVERS if d.name in run]
assert len(smokes) >= 5, f"expected the browser smokes to be wired up, found {len(smokes)}"
uncovered = []
for driver in smokes:
text = driver.read_text(encoding = "utf-8")
for out in sorted(set(re.findall(r'"(logs/[^"]+)"', text))):
stem = out.split("%")[0].split("{")[0]
if not any(fnmatch(stem, p) or stem.startswith(p.rstrip("*")) for p in patterns):
uncovered.append(f"{driver.name} -> {out}")
assert not uncovered, (
f"these smoke reports are not in the failure upload: {uncovered}; a failing smoke "
f"would upload every report except its own. Upload patterns: {patterns}"
)
def test_a_continue_on_error_smoke_can_still_upload_its_report():
"""`failure()` cannot see a smoke that is allowed to fail.
`continue-on-error: true` rewrites a step's CONCLUSION to success while leaving its
OUTCOME as failure, so a bare `if: failure()` upload is skipped on exactly the runs
where the non-blocking smoke is the only thing that failed, which is when its report
is the whole point. Each such smoke must be named in the upload condition.
"""
workflow = yaml.safe_load(
(REPO / ".github" / "workflows" / "studio-frontend-ci.yml").read_text(encoding = "utf-8")
)
steps = workflow["jobs"]["build"]["steps"]
upload = next(s for s in steps if s.get("name") == "Upload browser smoke artifacts")
condition = str(upload.get("if", ""))
lenient = [
s
for s in steps
if s.get("continue-on-error") and str(s.get("name", "")).startswith("Browser smoke")
]
assert lenient, "expected at least one continue-on-error browser smoke; did one get renamed?"
unseen = []
for step in lenient:
step_id = step.get("id")
if not step_id:
unseen.append(f"{step['name']!r} has no id, so the upload cannot reference it")
elif f"steps.{step_id}.outcome" not in condition:
unseen.append(f"{step['name']!r} (id {step_id}) is not in the upload condition")
assert not unseen, (
f"{unseen}; a continue-on-error smoke that fails alone leaves conclusion=success, so "
f"`{condition}` skips the upload and its report is lost."
)