feat(SDK) Add Python SDK implementation for #3010 (#3494)

* Codex worktree snapshot: startup-cleanup

Co-authored-by: Codex

* Add Python SDK real smoke test

Adds a repository-only real E2E smoke script for the Python SDK, plus npm and developer documentation entry points.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): address review findings — bugs, type safety, and test coverage

- Fix prepare_spawn_info: JS files now use "node" instead of sys.executable
- Fix protocol.py: correct total=False misuse on 7 TypedDicts (required fields were optional)
- Fix query.py: add _closed guard in _ensure_started, suppress exceptions in close()
- Fix sync_query.py: prevent close() deadlock, add context manager, add timeouts
- Fix transport.py: handle malformed JSON lines, add _closed guard in start()
- Fix validation.py: use uuid.RFC_4122 instead of magic UUID
- Fix __init__.py: export TextBlock, widen query_sync signature
- Remove dead code: ensure_not_aborted, write_json_line, _thread_error
- Add 12 new tests (29 → 41): context managers, JSON skip, closed guards, spawn info, timeouts

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): address wenshao review — session_id, bool validation, debug stderr

- Fix continue_session=True generating a wrong random session_id
- Add _as_optional_bool helper for strict type validation on bool fields
- Default debug stderr to sys.stderr when no custom callback is provided

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): address remaining wenshao review feedback

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* test(cli): harden settings dialog restart prompt test

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): review fixes — UUID compat, stderr fallback, sync cleanup

- Remove UUID version restriction to support v6/v7/v8 (RFC 9562)
- Always write to sys.stderr when stderr callback raises (was silent when debug=False)
- Prevent duplicate _STOP sentinel in SyncQuery.close() via _stop_sent flag
- Add ruff format --check to CI workflow
- Fix smoke_real.py version guard: fail early before imports instead of NameError
- Apply ruff format to existing files

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): remaining review fixes — exit_code attr, guard strictness, sync timeout

- Add exit_code attribute to ProcessExitError for programmatic access
- Strengthen is_control_response/is_control_cancel guards to require
  payload fields, preventing misrouting of malformed messages
- Expose control_request_timeout property on Query so SyncQuery uses
  the configured timeout instead of a hardcoded 30s default
- Use dataclasses.replace() instead of direct mutation on frozen-style
  QueryOptions in query() factory
- Add ResourceWarning in SyncQuery.__del__ when not properly closed

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): add exit_code default and guard __del__ against partial GC

- Give ProcessExitError.exit_code a default value (-1) so user code can
  construct the exception with just a message string
- Wrap SyncQuery.__del__ in try/except AttributeError to prevent crashes
  when the object is partially garbage-collected

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): review fixes — resource leak, type safety, CI matrix, docs

- Fix SyncQuery.__del__ to call close() on GC instead of only warning
- Replace hasattr duck-type check with isinstance(prompt, AsyncIterable)
- Type-validate permission_mode/auth_type in QueryOptions.from_mapping
- Use TypeGuard return types on all is_sdk_*/is_control_* predicates
- Add 5s margin to sync wrapper timeouts to prevent error type masking
- Expand CI matrix to test Python 3.10, 3.11, 3.12
- Change ProcessExitError.exit_code default from -1 to None
- Add stderr to docs QueryOptions listing
- Update README sync example to use context manager pattern

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): preserve iterator exhaustion state and suppress detached task warning

- Add _exhausted flag to Query.__anext__ and SyncQuery.__next__ so
  repeated iteration after end-of-stream raises Stop(Async)Iteration
  instead of blocking forever.
- Remove re-raise in _initialize() to prevent asyncio
  "Task exception was never retrieved" warning on detached tasks;
  the error is already surfaced via _finish_with_error().

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): reject mcp_servers at validation time and add iterator/init tests

- Reject mcp_servers in validate_query_options() with a clear error
  instead of advertising MCP support to the CLI and then failing at
  runtime when mcp_message arrives.
- Remove dead mcp_servers branch from _initialize().
- Add tests for async/sync iterator exhaustion, detached init task
  warning suppression, and mcp_servers validation.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(sdk-python): fix ruff lint errors in new tests

- Use ControlRequestTimeoutError instead of bare Exception (B017)
- Fix import sorting for stdlib vs third-party (I001)
- Break long line to stay within 88-char limit (E501)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* style(sdk-python): apply ruff format to new tests

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

---------

Co-authored-by: jinye.djy <jinye.djy@alibaba-inc.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This commit is contained in:
jinye 2026-04-25 07:02:58 +08:00 committed by GitHub
parent 202be6ec7d
commit e384338145
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 4676 additions and 14 deletions

View file

@ -0,0 +1,400 @@
from __future__ import annotations
import os
import stat
import textwrap
from pathlib import Path
import pytest
@pytest.fixture()
def fake_qwen_path(tmp_path: Path) -> str:
script_path = tmp_path / "fake_qwen.py"
script_path.write_text(
textwrap.dedent(
"""
#!/usr/bin/env python3
import argparse
import json
import sys
import uuid
def send(message):
sys.stdout.write(json.dumps(message, separators=(",", ":")) + "\\n")
sys.stdout.flush()
def parse_user_content(message):
payload = message.get("message", {})
content = payload.get("content", "")
if isinstance(content, str):
return content
if isinstance(content, list):
text_parts = []
for block in content:
if isinstance(block, dict) and block.get("type") == "text":
text_parts.append(str(block.get("text", "")))
return " ".join(text_parts)
return str(content)
def build_system_message():
return {
"type": "system",
"subtype": "init",
"uuid": session_id,
"session_id": session_id,
"cwd": ".",
"tools": ["Read", "Edit", "Bash"],
"mcp_servers": [],
"model": state["model"],
"permission_mode": state["permission_mode"],
"qwen_code_version": "fake-1.0.0",
"capabilities": {
"canSetModel": True,
"canSetPermissionMode": True,
},
}
def build_assistant_message(text):
return {
"type": "assistant",
"uuid": str(uuid.uuid4()),
"session_id": session_id,
"message": {
"id": str(uuid.uuid4()),
"type": "message",
"role": "assistant",
"model": state["model"],
"content": [
{
"type": "text",
"text": text,
}
],
"usage": {
"input_tokens": 1,
"output_tokens": 1,
},
},
"parent_tool_use_id": None,
}
def build_result_message(result_text):
return {
"type": "result",
"subtype": "success",
"uuid": str(uuid.uuid4()),
"session_id": session_id,
"is_error": False,
"duration_ms": 5,
"duration_api_ms": 1,
"num_turns": 1,
"result": result_text,
"usage": {
"input_tokens": 1,
"output_tokens": 1,
},
"permission_denials": [],
}
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument("--model")
parser.add_argument("--approval-mode")
parser.add_argument("--include-partial-messages", action="store_true")
parser.add_argument("--session-id")
parser.add_argument("--resume")
parser.add_argument(
"--continue",
dest="continue_session",
action="store_true",
)
args, _ = parser.parse_known_args()
session_id = (
args.resume
or args.session_id
or (
"aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee"
if args.continue_session
else str(uuid.uuid4())
)
)
state = {
"model": args.model or "coder-model",
"permission_mode": args.approval_mode or "default",
"include_partial": bool(args.include_partial_messages),
}
pending_permission = None
pending_unknown_control = None
for line in sys.stdin:
line = line.strip()
if not line:
continue
message = json.loads(line)
msg_type = message.get("type")
if msg_type == "control_request":
request_id = message["request_id"]
request = message["request"]
subtype = request.get("subtype")
if subtype == "initialize":
send(
{
"type": "control_response",
"response": {
"subtype": "success",
"request_id": request_id,
"response": {},
},
}
)
send(build_system_message())
elif subtype == "set_model":
state["model"] = request["model"]
send(
{
"type": "control_response",
"response": {
"subtype": "success",
"request_id": request_id,
"response": {},
},
}
)
send(build_system_message())
elif subtype == "set_permission_mode":
state["permission_mode"] = request["mode"]
send(
{
"type": "control_response",
"response": {
"subtype": "success",
"request_id": request_id,
"response": {},
},
}
)
send(build_system_message())
elif subtype == "interrupt":
send(
{
"type": "control_response",
"response": {
"subtype": "success",
"request_id": request_id,
"response": {},
},
}
)
elif subtype == "supported_commands":
send(
{
"type": "control_response",
"response": {
"subtype": "success",
"request_id": request_id,
"response": {
"commands": [
"initialize",
"interrupt",
"set_model",
"set_permission_mode",
]
},
},
}
)
elif subtype == "mcp_server_status":
send(
{
"type": "control_response",
"response": {
"subtype": "success",
"request_id": request_id,
"response": {"servers": []},
},
}
)
else:
send(
{
"type": "control_response",
"response": {
"subtype": "error",
"request_id": request_id,
"error": f"unsupported request: {subtype}",
},
}
)
elif msg_type == "user":
prompt = parse_user_content(message)
if "exit nonzero" in prompt:
sys.exit(9)
if "request unknown control" in prompt:
request_id = str(uuid.uuid4())
pending_unknown_control = {
"request_id": request_id,
"prompt": prompt,
}
send(
{
"type": "control_request",
"request_id": request_id,
"request": {
"subtype": "something_new",
"payload": {},
},
}
)
continue
if "use tool" in prompt or "create file" in prompt:
tool_use_id = str(uuid.uuid4())
send(
{
"type": "assistant",
"uuid": str(uuid.uuid4()),
"session_id": session_id,
"message": {
"id": str(uuid.uuid4()),
"type": "message",
"role": "assistant",
"model": state["model"],
"content": [
{
"type": "tool_use",
"id": tool_use_id,
"name": "write_file",
"input": {
"path": "demo.txt",
"content": "hello",
},
}
],
"usage": {"input_tokens": 1, "output_tokens": 1},
},
"parent_tool_use_id": None,
}
)
request_id = str(uuid.uuid4())
pending_permission = {
"request_id": request_id,
"tool_use_id": tool_use_id,
"prompt": prompt,
}
send(
{
"type": "control_request",
"request_id": request_id,
"request": {
"subtype": "can_use_tool",
"tool_name": "write_file",
"tool_use_id": tool_use_id,
"input": {"path": "demo.txt", "content": "hello"},
"permission_suggestions": [
{"type": "allow", "label": "Allow write"}
],
"blocked_path": None,
},
}
)
continue
if state["include_partial"]:
send(
{
"type": "stream_event",
"uuid": str(uuid.uuid4()),
"session_id": session_id,
"event": {
"type": "content_block_delta",
"index": 0,
"delta": {"type": "text_delta", "text": "partial"},
},
"parent_tool_use_id": None,
}
)
send(build_assistant_message(f"Echo: {prompt}"))
send(build_result_message(f"done: {prompt}"))
elif msg_type == "control_response":
payload = message.get("response", {})
request_id = payload.get("request_id")
if (
pending_unknown_control
and request_id == pending_unknown_control["request_id"]
):
if payload.get("subtype") != "error":
sys.exit(3)
prompt = pending_unknown_control["prompt"]
pending_unknown_control = None
send(
build_assistant_message(
f"Unknown control handled for: {prompt}"
)
)
send(build_result_message(f"unknown-control: {prompt}"))
continue
if (
pending_permission
and request_id == pending_permission["request_id"]
):
prompt = pending_permission["prompt"]
tool_use_id = pending_permission["tool_use_id"]
pending_permission = None
behavior = "deny"
if payload.get("subtype") == "success":
response_payload = payload.get("response") or {}
behavior = response_payload.get("behavior", "deny")
is_allowed = behavior == "allow"
send(
{
"type": "user",
"session_id": session_id,
"message": {
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": tool_use_id,
"is_error": not is_allowed,
"content": "ok" if is_allowed else "denied",
}
],
},
"parent_tool_use_id": tool_use_id,
}
)
send(build_assistant_message(f"tool handled: {prompt}"))
send(build_result_message(f"tool-result: {prompt}"))
continue
"""
).strip()
+ "\n",
encoding="utf-8",
)
script_path.chmod(script_path.stat().st_mode | stat.S_IEXEC)
return str(script_path)
@pytest.fixture(autouse=True)
def disable_history_expansion() -> None:
# No-op fixture used as explicit marker for deterministic test env.
os.environ.setdefault("PYTHONUTF8", "1")

View file

@ -0,0 +1,276 @@
from __future__ import annotations
import asyncio
from collections.abc import AsyncIterator, Callable
from typing import Any
import pytest
from qwen_code_sdk import (
ProcessExitError,
SDKUserMessage,
is_sdk_assistant_message,
is_sdk_partial_assistant_message,
is_sdk_result_message,
is_sdk_system_message,
is_sdk_user_message,
query,
)
CONTINUED_SESSION_ID = "aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee"
VALID_UUID = "123e4567-e89b-12d3-a456-426614174000"
RESUME_UUID = "223e4567-e89b-12d3-a456-426614174000"
async def _collect_messages(result: Any) -> list[dict[str, Any]]:
messages: list[dict[str, Any]] = []
async for message in result:
messages.append(message)
return messages
async def _wait_for(predicate: Callable[[], bool], timeout: float = 2.0) -> None:
loop = asyncio.get_running_loop()
deadline = loop.time() + timeout
while loop.time() < deadline:
if predicate():
return
await asyncio.sleep(0.01)
raise AssertionError("timed out waiting for expected SDK state")
def _tool_result_error_flag(message: dict[str, Any]) -> bool:
content = message["message"]["content"]
assert isinstance(content, list)
return bool(content[0]["is_error"])
@pytest.mark.asyncio
async def test_single_turn_query(fake_qwen_path: str) -> None:
result = query(
"hello world",
{
"path_to_qwen_executable": fake_qwen_path,
},
)
messages = await _collect_messages(result)
assistant = next(
message for message in messages if is_sdk_assistant_message(message)
)
final = next(message for message in messages if is_sdk_result_message(message))
assert assistant["message"]["content"][0]["text"] == "Echo: hello world"
assert final["result"] == "done: hello world"
await result.close()
@pytest.mark.asyncio
async def test_include_partial_messages(fake_qwen_path: str) -> None:
result = query(
"stream partial",
{
"path_to_qwen_executable": fake_qwen_path,
"include_partial_messages": True,
},
)
messages = await _collect_messages(result)
partial = next(
message for message in messages if is_sdk_partial_assistant_message(message)
)
assert partial["event"]["type"] == "content_block_delta"
await result.close()
@pytest.mark.asyncio
async def test_default_permission_callback_denies_tool_use(fake_qwen_path: str) -> None:
result = query(
"use tool now",
{
"path_to_qwen_executable": fake_qwen_path,
},
)
messages = await _collect_messages(result)
tool_result = next(
message
for message in messages
if is_sdk_user_message(message)
and isinstance(message["message"]["content"], list)
)
assert _tool_result_error_flag(tool_result) is True
await result.close()
@pytest.mark.asyncio
async def test_permission_callback_can_allow_tool_use(fake_qwen_path: str) -> None:
async def can_use_tool(
tool_name: str,
tool_input: dict[str, Any],
context: dict[str, Any],
) -> dict[str, Any]:
assert tool_name == "write_file"
assert tool_input["path"] == "demo.txt"
assert context["suggestions"][0]["type"] == "allow"
return {"behavior": "allow", "updatedInput": tool_input}
result = query(
"create file with use tool",
{
"path_to_qwen_executable": fake_qwen_path,
"can_use_tool": can_use_tool,
},
)
messages = await _collect_messages(result)
tool_result = next(
message
for message in messages
if is_sdk_user_message(message)
and isinstance(message["message"]["content"], list)
)
assert _tool_result_error_flag(tool_result) is False
await result.close()
@pytest.mark.asyncio
async def test_unknown_control_requests_are_rejected(fake_qwen_path: str) -> None:
result = query(
"request unknown control",
{
"path_to_qwen_executable": fake_qwen_path,
},
)
messages = await _collect_messages(result)
final = next(message for message in messages if is_sdk_result_message(message))
assert final["result"] == "unknown-control: request unknown control"
await result.close()
@pytest.mark.asyncio
async def test_dynamic_controls_and_status(fake_qwen_path: str) -> None:
release_input = asyncio.Event()
async def prompts() -> AsyncIterator[SDKUserMessage]:
yield {
"type": "user",
"session_id": VALID_UUID,
"message": {
"role": "user",
"content": "first turn",
},
"parent_tool_use_id": None,
}
await release_input.wait()
result = query(
prompts(),
{
"path_to_qwen_executable": fake_qwen_path,
"session_id": VALID_UUID,
},
)
messages: list[dict[str, Any]] = []
async def consume() -> list[dict[str, Any]]:
async for message in result:
messages.append(message)
return messages
collector = asyncio.create_task(consume())
await _wait_for(lambda: any(is_sdk_result_message(message) for message in messages))
assert await result.supported_commands() == {
"commands": [
"initialize",
"interrupt",
"set_model",
"set_permission_mode",
]
}
assert await result.mcp_server_status() == {"servers": []}
await result.set_model("new-model")
await result.set_permission_mode("plan")
release_input.set()
await collector
system_messages = [
message for message in messages if is_sdk_system_message(message)
]
assert any(message["model"] == "new-model" for message in system_messages)
assert any(message["permission_mode"] == "plan" for message in system_messages)
await result.close()
@pytest.mark.asyncio
async def test_session_id_resume_and_continue(fake_qwen_path: str) -> None:
explicit = query(
"hello explicit",
{
"path_to_qwen_executable": fake_qwen_path,
"session_id": VALID_UUID,
},
)
explicit_messages = await _collect_messages(explicit)
assert explicit.get_session_id() == VALID_UUID
assert all(message["session_id"] == VALID_UUID for message in explicit_messages)
await explicit.close()
resumed = query(
"hello resume",
{
"path_to_qwen_executable": fake_qwen_path,
"resume": RESUME_UUID,
},
)
resumed_messages = await _collect_messages(resumed)
assert resumed.get_session_id() == RESUME_UUID
assert all(message["session_id"] == RESUME_UUID for message in resumed_messages)
await resumed.close()
continued = query(
"hello continue",
{
"path_to_qwen_executable": fake_qwen_path,
"continue_session": True,
},
)
continued_messages = await _collect_messages(continued)
assert continued.get_session_id() == CONTINUED_SESSION_ID
assert any(
message["session_id"] == CONTINUED_SESSION_ID for message in continued_messages
)
await continued.close()
@pytest.mark.asyncio
async def test_non_zero_process_exit_is_propagated(fake_qwen_path: str) -> None:
result = query(
"please exit nonzero",
{
"path_to_qwen_executable": fake_qwen_path,
},
)
with pytest.raises(ProcessExitError, match="code 9"):
await _collect_messages(result)
await result.close()
@pytest.mark.asyncio
async def test_async_context_manager(fake_qwen_path: str) -> None:
async with query(
"hello context",
{
"path_to_qwen_executable": fake_qwen_path,
},
) as result:
messages = await _collect_messages(result)
assert result.is_closed()
final = next(m for m in messages if is_sdk_result_message(m))
assert final["result"] == "done: hello context"

View file

@ -0,0 +1,82 @@
from __future__ import annotations
import threading
import time
import pytest
import qwen_code_sdk.sync_query as sync_query_module
from qwen_code_sdk import is_sdk_result_message, query_sync
from qwen_code_sdk.sync_query import SyncQuery
def test_sync_query_single_turn(fake_qwen_path: str) -> None:
result = query_sync(
"hello sync",
{
"path_to_qwen_executable": fake_qwen_path,
},
)
commands = result.supported_commands()
messages = list(result)
assert commands["commands"][0] == "initialize"
assert any(
is_sdk_result_message(message) and message["result"] == "done: hello sync"
for message in messages
)
result.close()
result.close()
def test_sync_query_bootstrap_failure_cleans_up_loop_thread(
monkeypatch: pytest.MonkeyPatch,
) -> None:
def raising_query(*args: object, **kwargs: object) -> object:
raise RuntimeError("bootstrap failed")
monkeypatch.setattr(sync_query_module, "query", raising_query)
baseline_threads = {
thread.ident
for thread in threading.enumerate()
if thread.name == "qwen-sdk-sync-loop"
}
with pytest.raises(RuntimeError, match="bootstrap failed"):
SyncQuery("hello")
deadline = time.time() + 1.0
while time.time() < deadline:
active_threads = {
thread.ident
for thread in threading.enumerate()
if thread.name == "qwen-sdk-sync-loop"
}
if active_threads == baseline_threads:
break
time.sleep(0.01)
active_threads = {
thread.ident
for thread in threading.enumerate()
if thread.name == "qwen-sdk-sync-loop"
}
assert active_threads == baseline_threads
def test_sync_query_context_manager(fake_qwen_path: str) -> None:
with query_sync(
"hello context",
{
"path_to_qwen_executable": fake_qwen_path,
},
) as result:
messages = list(result)
assert any(
is_sdk_result_message(m) and m["result"] == "done: hello context"
for m in messages
)
assert result.is_closed()