mirror of https://github.com/openclaw/openclaw.git synced 2026-10-03 01:29:56 +00:00

History

Peter Steinberger 0e7564ac11 fix: reuse prepared catalogs and repair Gateway E2E fixtures (#136915 ) * test(agents): model confirmed source replies in announce fixtures The successful announce fixture supplied only aggregate message-tool evidence, which no longer proves a committed final reply to the source conversation. Supply the receipt owned by the embedded runner without changing delivery policy or requester-wake sibling rerouting. Proof: the format suite failed 21 of 87 tests at both `c92d0ae9c0` and `a41af7eb71`; all 87 pass with the corrected fixture. * fix(models): reuse prepared catalogs when browsing inventory Inventory callers forced full discovery on every browse while trying to refresh auth-stale content. Give the prepared catalog owner a distinct stale refresh intent, preserve explicit forced refresh, and leave ordinary turn reads on their existing facts. Apply the policy to commands, CLI, and RPC. Correct the Ollama fixture response contract and await full catalog readiness. Verify discovered-only models survive callbacks and RPC while discovery is unavailable, and verify explicit refresh still reaches the provider. Proof: before the fix, the real Gateway made 8 discovery calls after a 3-call warmup, and the owner policy regressions failed. Prepared catalog and auth reload owner tests now pass (43 tests). * test(sandbox): verify writable private workspace isolation The Docker scenario treated workspaceAccess=none as a read-only private workspace. The sandbox mount owner intentionally provides private writable storage while hiding the host and agent workspaces. Assert a private write/read succeeds and the same-named host sentinel stays unchanged. Keep read-only write rejection and read-write host persistence. Use a unique temporary workspace in the owner test so ambient /tmp skills cannot add unexpected mounts. Proof: OrbStack Docker reproduced the old assertion on successful exit zero; the complete none/read-only/read-write scenario now passes. * test(acpx): use the plugin-owned adapter launcher The cancellation fixture hardcoded a root node_modules adapter path, but acpx owns that dependency. This caused deterministic MODULE_NOT_FOUND startup failures in both release validation runs, not an agent startup flake. Remove the override so the canonical plugin launcher resolves its adapter; retain the synthetic Codex app-server process through CODEX_PATH. Proof: the real installed adapter process passes cancellation authorization, rejects stale replacements without interrupting them, and preserves queued successors when the preceding run is cancelled.		2026-09-02 21:24:39 -07:00
..
convex-credential-broker	chore(deps): refresh seven-day eligible packages (#135177 )	2026-09-02 18:43:03 -07:00
scenarios	fix: reuse prepared catalogs and repair Gateway E2E fixtures (#136915 )	2026-09-02 21:24:39 -07:00
frontier-harness-plan.md	chore: migrate active GPT-5.5 references to GPT-5.6 (#104452 )	2026-07-11 06:30:57 -07:00
maturity-scores.yaml	refactor(canvas): make the panel a widget presenter (#126030 )	2026-08-19 08:21:07 -07:00
README.md	Convert QA scenarios to YAML files (#92915 )	2026-06-14 17:31:18 -07:00
scenarios.md	Convert QA scenarios to YAML files (#92915 )	2026-06-14 17:31:18 -07:00

README.md

QA Scenarios

Seed QA assets for the private qa-lab extension.

Files:

scenarios/index.yaml - canonical QA scenario pack, kickoff mission, and operator identity.
scenarios/<theme>/*.yaml - one runnable scenario per YAML file.
frontier-harness-plan.md - big-model bakeoff and tuning loop for harness work.
convex-credential-broker/ - standalone Convex v1 lease broker for pooled live credentials.

Key workflow:

qa suite is the executable frontier subset / regression loop.
qa manual is the scoped personality and style probe after the executable subset is green.
qa coverage prints the scenario coverage inventory from scenario YAML.

Operator workflows:

Use the openclaw-qa-testing skill for QA Lab live lanes, Convex credential pool operations, and WhatsApp live credential setup/replacement.

Keep this folder in git. Add new scenarios here before wiring them into automation.