qwen-code/integration-tests/cli
易良 0c5a4deabd
test(integration): migrate flaky E2E tests to fake-openai-server (#7934)
* test(integration): migrate flaky E2E tests to fake-openai-server

Migrate 39 real-model test cases to use deterministic fake-openai-server
scripting, eliminating model output variance as a failure source.

- tool-control.test.ts: 23 cases migrated (tool filtering, permission
  denial, canUseTool routing, priority rules)
- abort-and-lifecycle.test.ts: 15 cases migrated (abort mechanics,
  lifecycle, cleanup, debug output)
- list_directory.test.ts: 1 case migrated (CLI TestRig with env injection)

channel-plugin.test.ts (3 cases) is intentionally left on real model —
it tests the full WebSocket→AcpBridge→model pipeline and belongs in the
nightly smoke layer per #7616.

Closes #7616

* test(ci): move channel-plugin to nightly + relax assertions

channel-plugin.test.ts tests the full WebSocket→AcpBridge→model pipeline
with real model inference. Its assertions on specific model output
('4', 'pineapple', '50') are inherently non-deterministic.

- Exclude from post-merge E2E jobs (Linux + macOS)
- Add dedicated nightly-only job with continue-on-error
- Relax assertions: verify response is non-empty rather than matching
  specific model output content
- Add retry: 2 to each test case

* fix(test): use streaming chunks for abort test, restore comment

- abort-and-lifecycle 'should handle abort during query execution':
  switch from non-streaming content to contentChunks so the abort
  signal reliably lands while the response is still streaming
- channel-plugin: restore existing comment per AGENTS.md convention

* test(integration): address fake server review feedback

* test(integration): trim fake server review assertions

* test(integration): tighten fake server cleanup

* test(integration): fix permanently-failing stdin-close case and harden list_directory against user settings

The stdin-close case's fake server needle contained a raw quote that
JSON.stringify-escaped transcripts can never match, and the scripted
write_file was rejected by prior-read enforcement; the script now keys
off a quote-free marker and reads before writing. list_directory now
pins auth via CLI flags so a developer's ~/.qwen/settings.json cannot
silently route the run to a real model endpoint.

* test(integration): tighten fake server review regressions

* test(integration): guard abort assertions

* test(integration): fix abort timer race and dead coreTools assertions

* test(integration): address fake server review findings

* test(integration): isolate fake fast model settings

* test(e2e): simplify isolated test setup

---------

Co-authored-by: qwen-code-dev-bot <qwen-code-dev-bot@users.noreply.github.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: Shaojin Wen <shaojin.wensj@alibaba-inc.com>
2026-07-29 11:42:32 +00:00
..
_daemon-benchmark-helpers.ts test(cli): add daemon startup benchmark (#5825) 2026-06-25 06:56:13 +00:00
_daemon-harness.ts feat(mcp): reconcile MCP servers live on settings change (#5561) 2026-06-25 14:50:37 +00:00
_daemon-perf-report.ts feat(daemon): merge daemon-mode feature batch into main (#4490) 2026-06-12 00:34:49 +08:00
_first-output-benchmark.test.ts fix(test): Restore first-output benchmark measurement validity and correct its artifact schema (#7820) 2026-07-28 06:51:03 +00:00
_first-output-benchmark.ts fix(test): Restore first-output benchmark measurement validity and correct its artifact schema (#7820) 2026-07-28 06:51:03 +00:00
acp-cron.test.ts feat(core): enable loop/cron tools by default (#4950) 2026-06-10 23:21:53 +08:00
acp-integration.test.ts fix(acp): hide discontinued oauth model (#7522) 2026-07-23 17:21:12 +00:00
cron-tools.test.ts feat(core): enable loop/cron tools by default (#4950) 2026-06-10 23:21:53 +08:00
daemon-invocation-context.test.ts feat(core): propagate trusted daemon invocation context (#7279) 2026-07-23 06:49:11 +00:00
edit.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
extensions-install.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
file-system.test.ts fix(cli): keep v5 settings migration idempotent (#5676) 2026-06-23 01:28:48 +08:00
gen-ai-telemetry.test.ts feat(core): add ARMS session user ID (#7921) 2026-07-28 11:10:38 +00:00
json-output.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
json-schema.test.ts fix(test): repair stale --json-schema integration assertion (#4075) 2026-05-12 12:00:57 +08:00
list_directory.test.ts test(integration): migrate flaky E2E tests to fake-openai-server (#7934) 2026-07-29 11:42:32 +00:00
mcp_server_cyclic_schema.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
mock-acp-typecheck.test.ts feat(daemon): merge daemon-mode feature batch into main (#4490) 2026-06-12 00:34:49 +08:00
monitor.test.ts ci(e2e): stabilize MCP/CLI flows and cancel stale main runs (#4039) 2026-05-12 16:09:30 +08:00
notebook-edit.test.ts feat(core): add NotebookEdit tool for Jupyter notebooks 2026-05-21 00:06:15 +08:00
qwen-config-dir.test.ts revert(core): revert Protocol enum & model-identity decoupling (#5089) (#5745) 2026-06-23 16:32:38 +08:00
qwen-daemon-first-output-benchmark.test.ts fix(test): Restore first-output benchmark measurement validity and correct its artifact schema (#7820) 2026-07-28 06:51:03 +00:00
qwen-daemon-loadtest.test.ts feat(daemon): merge daemon-mode feature batch into main (#4490) 2026-06-12 00:34:49 +08:00
qwen-daemon-startup-benchmark.test.ts test(cli): add daemon startup benchmark (#5825) 2026-06-25 06:56:13 +00:00
qwen-daemon-vs-cli-benchmark.test.ts feat(daemon): merge daemon-mode feature batch into main (#4490) 2026-06-12 00:34:49 +08:00
qwen-serve-baseline.test.ts refactor(cli): Remove serve bridge re-export shims (#5955) 2026-06-28 13:20:44 +00:00
qwen-serve-channel-workers.test.ts fix(cli): Preserve channel startup failure details (#6950) 2026-07-16 01:17:19 +00:00
qwen-serve-client-mcp.test.ts fix(ci): cover release integration regressions (#5994) 2026-06-29 11:54:11 +00:00
qwen-serve-routes.test.ts feat(web-shell): add monitor task details (#7817) 2026-07-27 12:30:17 +00:00
qwen-serve-streaming.test.ts fix(acp): clear inherited sandboxSessionId for each new ACP session (#7443) 2026-07-21 16:11:45 +00:00
read_many_files.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
run_shell_command.test.ts ci(e2e): stabilize MCP/CLI flows and cancel stale main runs (#4039) 2026-05-12 16:09:30 +08:00
save_memory.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
settings-migration.test.ts revert(core): revert Protocol enum & model-identity decoupling (#5089) (#5745) 2026-06-23 16:32:38 +08:00
simple-mcp-server.test.ts fix(ci): restore release integration env controls (#5121) 2026-06-15 15:56:29 +08:00
sleep-interception.test.ts test(integration): harden flaky sleep-interception e2e against skipped tool calls (#4936) 2026-06-11 00:08:26 +08:00
stdin-context.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
telemetry.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
todo_write.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
tool-search.test.ts feat(core): enable loop/cron tools by default (#4950) 2026-06-10 23:21:53 +08:00
utf-bom-encoding.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00
write_file.test.ts refactor(tests): reorganize integration tests by execution mode 2026-03-29 05:49:17 +00:00