mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-07-19 22:14:08 +00:00
* feat(cua-driver): vendor trycua/cua driver with 1000-normalized coordinate support
Vendor libs/cua-driver from trycua/cua into packages/cua-driver as the
basis for qwen-code's computer-use backend, adding an opt-in relative
(1000x1000 normalized) coordinate mode for Qwen-VL clients.
- coord_norm.rs: 0-1000 <-> pixel conversion, per-(pid,window_id) size
cache, tools/list description rewrite (TDD, 27 tests)
- ToolRegistry: normalized field + invoke input/output hooks
- protocol.rs: system-instruction coordinate wording switched by mode
- serve.rs: daemon list path description rewrite (input_schema aware)
- main.rs: CUA_DRIVER_RS_COORDINATE_SPACE env seed
Default coordinate_space=pixels => zero behavior change for existing
pixel clients. Set CUA_DRIVER_RS_COORDINATE_SPACE=normalized_1000 to
enable. Excludes rust/target build output.
* feat(cua-driver): make normalized coordinate scale configurable
Add CUA_DRIVER_RS_COORDINATE_SCALE (default 1000) so the normalization
full-scale can absorb the Qwen 999-vs-1000 cookbook ambiguity without a
recompile. norm_to_px/px_to_norm now take an explicit scale; denormalize_args
reads the process-wide COORDINATE_SCALE seeded once at startup from env.
* ci(cua-driver): add cross-platform release workflow for vendored driver
Standalone GitHub Action that builds, signs, and releases the vendored
cua-driver under packages/cua-driver. Adapted from upstream trycua/cua
cd-rust-cua-driver.yml:
- macOS: universal binary (lipo arm64+x86_64), codesigned + notarized into
CuaDriver.app using qwen-code's existing secrets (MAC_CSC_LINK cert +
App Store Connect API key notarization); Developer ID identity is
auto-discovered from the imported cert.
- Linux: x86_64 + arm64, built in debian:11 for a glibc 2.31 floor.
- Windows: x86_64 + arm64, unsigned (no EV cert, matches upstream).
- Release: softprops/action-gh-release on cua-driver-rs-v* tags or manual
dispatch, prerelease.
Triggered by tag push (cua-driver-rs-v*) or workflow_dispatch.
* chore(cua-driver): rebrand vendored driver as qwen-cua-driver
Rename the vendored trycua/cua driver so the fork installs and runs
independently of any upstream trycua install:
- binary cua-driver -> qwen-cua-driver
- bundle CuaDriver.app -> QwenCuaDriver.app
- bundle id com.trycua.driver -> com.qwencode.cua-driver
Updates the cargo/uia manifests, Info.plist, bundle/proxy launch paths,
permission/health-report wording, the install/build scripts, and the
cross-platform release workflow.
* feat(cua-driver): finish relative-coordinate mode — toggle, scale, zoom/move_cursor
- CUA_DRIVER_RS_COORDINATE_SPACE is now a 1/0 toggle (via is_env_truthy);
default off keeps pixel mode byte-identical to upstream.
- Thread CUA_DRIVER_RS_COORDINATE_SCALE through every coordinate surface
(was hardcoded 1000): input denormalization already used it; now the
rewritten screenshot dims, the tool/param descriptions, and the agent
instructions track the configured scale too.
- Normalize zoom (window basis) and move_cursor (screen basis) inputs and
rewrite their descriptions, alongside click/double_click/right_click/drag.
- Fix zoom on downscaled (Retina) windows: apply the get_window_state resize
ratio so the crop lands on the region the agent saw. Normalized mode only;
pixel-mode zoom unchanged.
All coordinate behavior stays gated on the normalized flag, so the default
(pixels) path is unchanged from upstream.
* chore(cua-driver): add upstream-sync script (git subtree unusable here)
`git subtree split --prefix=libs/cua-driver` hangs on a commit deep in
trycua/cua's history, so the subtree add/pull workflow isn't usable for
the vendored driver (and a pull would re-split + re-hang every time).
Add scripts/sync-from-upstream.sh instead: it git-diffs two upstream refs
(never walks the full history, so it dodges the hang), reprefixes the
libs/cua-driver delta to packages/cua-driver, and `git apply --reject`s it
on top of our local changes — conflicts land as *.rej for manual fixup.
Record the vendored version in .vendored-from and document the migration +
sync method in the design doc.
* chore(cua-driver): exclude vendored driver from qwen-code ESLint
The vendored packages/cua-driver tree carries upstream JS (e.g. the
test-harness Electron app) that doesn't follow qwen-code's lint rules and
fails CI. It is not a workspace package (no package.json) and is not
qwen-code TypeScript, so add it to eslint.config.js global ignores —
alongside packages/desktop/** — the standard treatment for vendored code.
* fix(cua-driver): let start_session revive an idle-reaped session
Ports the fix from upstream trycua/cua#2035 into the vendored driver.
When a session is reaped for idleness, a subsequent start_session with the
same id failed instead of resuming it. Revive the ended session in place so
the agent can continue rather than getting a hard error.
* fix(cua-driver): retry daemon socket writes on EAGAIN
Ports the fix from upstream trycua/cua#2036 into the vendored driver.
A non-blocking daemon socket can return EAGAIN/EWOULDBLOCK mid-write when the
peer's receive buffer is momentarily full. The driver treated that as fatal
and dropped the connection. Add a bounded retry/poll loop (mirror of the
read-side socket_io helper) so transient back-pressure no longer kills the
session; only a real timeout or hard error fails the write.
* fix(cua-driver/linux): stop reporting bare "Clicked" for X11 synthetic clicks
Ports the fix from upstream trycua/cua#2025 into the vendored driver.
On X11, clicks are delivered via XSendEvent synthetic events, which many
toolkits (GTK/SDL/Allegro) ignore because send_event is set. The driver still
reported a flat success ("Clicked"), masking that nothing happened. Report
the synthetic-delivery caveat honestly so the agent can fall back instead of
assuming the click landed.
(platform-linux crate is not built on macOS; verified by clean upstream apply
and covered by upstream + release-workflow Linux CI.)
* fix(cua-driver/windows): list empty-/null-title top-level windows
Ports the fix from upstream trycua/cua#2021 into the vendored driver.
list_windows filtered out any top-level window whose title was empty or null,
so legitimate targets (splash screens, some Electron/game windows, tool
windows) were invisible to the agent and unclickable. Include empty-title
windows, using class name / process as a fallback label.
(platform-windows crate is not built on macOS; verified by clean upstream
apply and covered by upstream + release-workflow Windows CI.)
* chore(cua-driver): track cherry-picked upstream PRs; fix vendored-from
The vendored copy is actually at cua-driver-rs-v0.6.7 (workspace version and
all 0.6.7->0.6.8 delta files confirm it), but .vendored-from had drifted to
0.6.8 during an earlier sync-script trial whose code delta was not kept. Left
as-is it would make a future sync diff 0.6.8->newer and silently skip the real
0.6.7->0.6.8 fixes. Correct it back to 0.6.7.
Also record the four not-yet-merged upstream PRs we carry as cherry-picks
(trycua/cua#2021/#2025/#2035/#2036) in .vendored-patches.md, and have
sync-from-upstream.sh point at it so the next sync reconciles them.
* ci(cua-driver): satisfy repo yamllint on the release workflow
The vendored-driver release workflow tripped 114 quoted-strings violations
under the repo's .yamllint (quote-type: single, required). Single-quote all
string scalars to match every other workflow in .github/workflows.
While reformatting, the release-notes body also got its paragraph blank lines
collapsed and still referenced the old CUA_DRIVER_RS_COORDINATE_SPACE=
normalized_1000 value — restore the blank lines and update it to the current
0/1 toggle (default 0 = off; optional CUA_DRIVER_RS_COORDINATE_SCALE=1000).
* chore(cua-driver): sync vendored driver to cua-driver-rs-v0.6.8
First real run of scripts/sync-from-upstream.sh: it 3-way-applied the upstream
0.6.7->0.6.8 delta onto our local fork. 10/12 files applied cleanly; the 2
rejects (install.ps1, _install-rust.sh) were already-applied baked-version
bumps (0.6.6->0.6.7, our copies were already at 0.6.7), i.e. no real conflict.
0.6.8 brings: Wayland input path (platform-linux), linux health_report +
overlay tweaks, a platform-macos build.rs step, and dependency bumps. Version
moved to 0.6.8 across the workspace.
Verified our work survived the sync untouched: the relative-coordinate shim
(coord_norm/protocol) and all four cherry-picked PRs (socket_io/session +
linux/windows) are intact — in particular the 0.6.8 edit to platform-linux
tools/impl_.rs landed alongside our #2025 change with no collision. macOS
cargo check + 132 core tests green. (platform-linux/windows + the binary
integration test build only on their own runners; upstream CI covers those.)
* ci(cua-driver): add a dry_run gate to the release workflow
Mirror the desktop-release / release dry-run pattern: a workflow_dispatch
dry_run boolean input (default true). The cross-platform build + package jobs
always run and upload their artifacts; the GitHub Release job now publishes
only on a tag push or an explicit dry_run=false dispatch.
Lets us rehearse the whole build/package pipeline (dry_run=true, notarize=false)
and inspect the produced artifacts without cutting a release. A branch push
(no tag, not a dispatch) likewise builds without releasing.
318 lines
11 KiB
Python
318 lines
11 KiB
Python
#!/usr/bin/env python3
|
|
"""Build script to download platform-specific cua-driver binary and create wheel.
|
|
|
|
This script:
|
|
1. Detects the current platform
|
|
2. Downloads the appropriate cua-driver-rs binary from GitHub releases
|
|
3. Places it in src/cua_driver/bin/
|
|
4. Builds the wheel with hatchling
|
|
|
|
Usage:
|
|
python build_wheel.py [--version VERSION]
|
|
"""
|
|
|
|
import argparse
|
|
import hashlib
|
|
import os
|
|
import platform
|
|
import shutil
|
|
import subprocess
|
|
import sys
|
|
import tarfile
|
|
import urllib.request
|
|
import zipfile
|
|
from pathlib import Path
|
|
|
|
|
|
def get_platform_info(arch_override: str = None):
|
|
"""Determine platform and architecture for binary selection.
|
|
|
|
Args:
|
|
arch_override: Optional architecture override (e.g., 'arm64', 'x86_64', 'universal')
|
|
"""
|
|
system = platform.system().lower()
|
|
|
|
if arch_override:
|
|
# Normalize arch_override to handle common aliases/casing
|
|
arch_lower = arch_override.lower()
|
|
if arch_lower in ("x86_64", "amd64", "x64"):
|
|
arch = "x86_64"
|
|
elif arch_lower in ("arm64", "aarch64"):
|
|
arch = "arm64"
|
|
elif arch_lower == "universal":
|
|
arch = "universal"
|
|
else:
|
|
raise ValueError(f"Unsupported architecture override: {arch_override}")
|
|
else:
|
|
machine = platform.machine().lower()
|
|
# Normalize architecture names
|
|
if machine in ("x86_64", "amd64"):
|
|
arch = "x86_64"
|
|
elif machine in ("arm64", "aarch64"):
|
|
arch = "arm64"
|
|
else:
|
|
raise ValueError(f"Unsupported architecture: {machine}")
|
|
|
|
# Map to cua-driver-rs release naming
|
|
if system == "darwin":
|
|
# macOS uses universal binary
|
|
return "darwin", "universal"
|
|
elif system == "linux":
|
|
return "linux", arch
|
|
elif system == "windows":
|
|
return "windows", arch
|
|
else:
|
|
raise ValueError(f"Unsupported platform: {system}")
|
|
|
|
|
|
def get_release_url(version: str, platform_name: str, arch: str) -> tuple[str, list[str]]:
|
|
"""Get the GitHub release URL and binary names for the platform.
|
|
|
|
Returns:
|
|
Tuple of (download_url, list_of_binary_names_in_archive)
|
|
"""
|
|
base_url = f"https://github.com/trycua/cua/releases/download/cua-driver-rs-v{version}"
|
|
|
|
if platform_name == "darwin":
|
|
# Universal binary tarball
|
|
filename = f"cua-driver-rs-{version}-darwin-universal-binary.tar.gz"
|
|
binary_names = ["cua-driver"]
|
|
elif platform_name == "linux":
|
|
filename = f"cua-driver-rs-{version}-linux-{arch}-binary.tar.gz"
|
|
binary_names = ["cua-driver"]
|
|
elif platform_name == "windows":
|
|
filename = f"cua-driver-rs-{version}-windows-{arch}-binary.zip"
|
|
# Windows includes both main executable and UIAccess worker
|
|
binary_names = ["cua-driver.exe", "cua-driver-uia.exe"]
|
|
else:
|
|
raise ValueError(f"Unknown platform: {platform_name}")
|
|
|
|
return f"{base_url}/{filename}", binary_names
|
|
|
|
|
|
def verify_sha256(file_path: Path, expected_sha256: str) -> None:
|
|
"""Verify file matches expected SHA256 hash."""
|
|
h = hashlib.sha256()
|
|
with open(file_path, "rb") as f:
|
|
for chunk in iter(lambda: f.read(8192), b""):
|
|
h.update(chunk)
|
|
actual = h.hexdigest()
|
|
if actual != expected_sha256:
|
|
raise ValueError(
|
|
f"SHA256 mismatch for {file_path.name}: expected {expected_sha256}, got {actual}"
|
|
)
|
|
|
|
|
|
def get_expected_sha256(version: str, archive_name: str) -> str:
|
|
"""Fetch and parse checksums.txt from GitHub release."""
|
|
checksums_url = f"https://github.com/trycua/cua/releases/download/cua-driver-rs-v{version}/checksums.txt"
|
|
print(f"Fetching checksums from {checksums_url}...")
|
|
|
|
try:
|
|
with urllib.request.urlopen(checksums_url) as response:
|
|
content = response.read().decode("utf-8")
|
|
|
|
for line in content.splitlines():
|
|
line = line.strip()
|
|
# Skip empty lines, comments, and headers
|
|
if not line or line.startswith("#") or "Checksums" in line or line.startswith("```"):
|
|
continue
|
|
# Parse "SHA256 filename" format
|
|
parts = line.split()
|
|
if len(parts) >= 2:
|
|
sha, name = parts[0], parts[-1]
|
|
if name == archive_name:
|
|
return sha
|
|
|
|
raise ValueError(f"SHA256 for {archive_name} not found in checksums.txt")
|
|
except Exception as e:
|
|
raise RuntimeError(f"Failed to fetch or parse checksums: {e}")
|
|
|
|
|
|
def download_file(url: str, dest: Path, expected_sha256: str) -> None:
|
|
"""Download a file with progress indication and SHA256 verification."""
|
|
print(f"Downloading {url}...")
|
|
try:
|
|
with urllib.request.urlopen(url) as response:
|
|
total_size = int(response.headers.get("content-length", 0))
|
|
dest.parent.mkdir(parents=True, exist_ok=True)
|
|
|
|
with open(dest, "wb") as f:
|
|
downloaded = 0
|
|
block_size = 8192
|
|
while True:
|
|
chunk = response.read(block_size)
|
|
if not chunk:
|
|
break
|
|
f.write(chunk)
|
|
downloaded += len(chunk)
|
|
if total_size > 0:
|
|
percent = (downloaded / total_size) * 100
|
|
print(f" {percent:.1f}% ({downloaded}/{total_size} bytes)", end="\r")
|
|
|
|
print(f"\nDownloaded to {dest}")
|
|
|
|
# Verify SHA256
|
|
print("Verifying SHA256 checksum...")
|
|
verify_sha256(dest, expected_sha256)
|
|
print("[OK] Checksum verified")
|
|
|
|
except Exception as e:
|
|
if dest.exists():
|
|
dest.unlink()
|
|
raise RuntimeError(f"Failed to download {url}: {e}")
|
|
|
|
|
|
def extract_binaries(archive_path: Path, binary_names: list[str], dest_dir: Path) -> list[Path]:
|
|
"""Extract the binaries from the archive.
|
|
|
|
Args:
|
|
archive_path: Path to the archive file
|
|
binary_names: List of binary names to extract
|
|
dest_dir: Destination directory
|
|
|
|
Returns:
|
|
List of paths to extracted binaries
|
|
"""
|
|
dest_dir.mkdir(parents=True, exist_ok=True)
|
|
extracted_paths = []
|
|
|
|
if archive_path.suffix == ".zip":
|
|
with zipfile.ZipFile(archive_path, "r") as zf:
|
|
for binary_name in binary_names:
|
|
dest_path = dest_dir / binary_name
|
|
# Find the binary in the zip (exact match or ends with /<binary_name>)
|
|
for name in zf.namelist():
|
|
# Match exact name or path ending with /binary_name
|
|
if name == binary_name or name.endswith(f"/{binary_name}"):
|
|
with zf.open(name) as src, open(dest_path, "wb") as dst:
|
|
shutil.copyfileobj(src, dst)
|
|
extracted_paths.append(dest_path)
|
|
print(f"Extracted binary to {dest_path}")
|
|
break
|
|
else:
|
|
raise ValueError(f"Binary {binary_name} not found in {archive_path}")
|
|
else:
|
|
# .tar.gz
|
|
with tarfile.open(archive_path, "r:gz") as tf:
|
|
for binary_name in binary_names:
|
|
# The -binary tarballs have the binary at the root
|
|
tf.extract(binary_name, dest_dir)
|
|
dest_path = dest_dir / binary_name
|
|
extracted_paths.append(dest_path)
|
|
print(f"Extracted binary to {dest_path}")
|
|
|
|
# Make executable on Unix
|
|
if sys.platform != "win32":
|
|
for path in extracted_paths:
|
|
os.chmod(path, 0o755)
|
|
|
|
return extracted_paths
|
|
|
|
|
|
def build_wheel(package_dir: Path, target_arch: str = None) -> None:
|
|
"""Build the wheel using hatchling.
|
|
|
|
Args:
|
|
package_dir: Directory containing pyproject.toml
|
|
target_arch: Target architecture for cross-compilation (x86_64, arm64)
|
|
"""
|
|
print("\nBuilding wheel...")
|
|
|
|
env = os.environ.copy()
|
|
|
|
# Set platform tag override for cross-compilation
|
|
if target_arch:
|
|
system = platform.system().lower()
|
|
if system == "windows":
|
|
# Override wheel platform tag for Windows cross-compilation
|
|
if target_arch == "arm64":
|
|
env["_PYTHON_HOST_PLATFORM"] = "win-arm64"
|
|
elif target_arch == "x86_64":
|
|
env["_PYTHON_HOST_PLATFORM"] = "win-amd64"
|
|
# Linux and macOS don't need overrides for our use case
|
|
# (macOS uses universal binary, Linux builds on native arch)
|
|
|
|
subprocess.run(
|
|
[sys.executable, "-m", "build", "--wheel"],
|
|
cwd=package_dir,
|
|
env=env,
|
|
check=True,
|
|
)
|
|
print("Wheel built successfully!")
|
|
|
|
|
|
def main():
|
|
parser = argparse.ArgumentParser(description="Build cua-driver Python wheel with bundled binary")
|
|
parser.add_argument(
|
|
"--version",
|
|
default="0.5.1",
|
|
help="cua-driver-rs version to download (default: 0.5.1)",
|
|
)
|
|
parser.add_argument(
|
|
"--arch",
|
|
help="Architecture override (e.g., 'arm64', 'x86_64', 'universal')",
|
|
)
|
|
parser.add_argument(
|
|
"--skip-download",
|
|
action="store_true",
|
|
help="Skip download and use existing binary in bin/ (for local testing)",
|
|
)
|
|
args = parser.parse_args()
|
|
|
|
# Determine paths
|
|
script_dir = Path(__file__).parent
|
|
bin_dir = script_dir / "src" / "cua_driver" / "bin"
|
|
download_dir = script_dir / "downloads"
|
|
|
|
if not args.skip_download:
|
|
# Get platform info and download URL
|
|
platform_name, arch = get_platform_info(args.arch)
|
|
print(f"Building for {platform_name}-{arch}")
|
|
|
|
url, binary_names = get_release_url(args.version, platform_name, arch)
|
|
archive_name = url.split("/")[-1]
|
|
archive_path = download_dir / archive_name
|
|
|
|
# Get expected SHA256 from checksums.txt
|
|
expected_sha256 = get_expected_sha256(args.version, archive_name)
|
|
|
|
# Download the release archive (or verify cached)
|
|
if not archive_path.exists():
|
|
download_file(url, archive_path, expected_sha256)
|
|
else:
|
|
print(f"Using cached archive: {archive_path}")
|
|
# Verify cached archive too
|
|
print("Verifying cached archive SHA256...")
|
|
verify_sha256(archive_path, expected_sha256)
|
|
print("[OK] Cached archive checksum verified")
|
|
|
|
# Extract binaries
|
|
extract_binaries(archive_path, binary_names, bin_dir)
|
|
else:
|
|
print("Skipping download (using existing binary)")
|
|
|
|
# Verify main binary exists
|
|
expected_binary = "cua-driver.exe" if sys.platform == "win32" else "cua-driver"
|
|
binary_path = bin_dir / expected_binary
|
|
if not binary_path.exists():
|
|
raise FileNotFoundError(
|
|
f"Binary not found at {binary_path}. "
|
|
f"Run without --skip-download or place binary manually."
|
|
)
|
|
|
|
print(f"\nBinary ready at: {binary_path}")
|
|
print(f"Binary size: {binary_path.stat().st_size / 1024 / 1024:.2f} MB")
|
|
|
|
# List all binaries in bin directory
|
|
print(f"\nAll binaries in {bin_dir}:")
|
|
for binary in bin_dir.iterdir():
|
|
if binary.is_file():
|
|
print(f" - {binary.name} ({binary.stat().st_size / 1024 / 1024:.2f} MB)")
|
|
|
|
# Build the wheel (pass target arch for cross-compilation)
|
|
build_wheel(script_dir, target_arch=arch if not args.skip_download else None)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|