mirror of
https://github.com/zed-industries/zed.git
synced 2026-07-19 04:36:23 +00:00
10f501d700
Summary: - Add the `zed-eval` Python CLI for Modal/Harbor/Pier benchmark orchestration, including content-addressed remote builds, run/suite management, reporting, rejudge, baseline, and cleanup workflows. - Extend `eval-cli` for remote evals with provider/model overrides and step/tool-call metrics in `result.json`. - Add install/source-run helper scripts so `zed-eval` can be installed or run from the checkout without manually setting `PYTHONPATH`. - Harden the remote harness wrappers around exit-code preservation, archive extraction, custom secret wiring, and Harbor/Pier option parity, with regression coverage. Testing: - Using the CLI for two weeks - `PYTHONPATH=crates/eval_cli python3 -m compileall -q crates/eval_cli/zed_eval` - `uv run --project crates/eval_cli/zed_eval python -m unittest discover -s crates/eval_cli/zed_eval/tests` - `bash -n crates/eval_cli/script/install-zed-eval crates/eval_cli/script/zed-eval` - `cargo check -p eval_cli` - `cargo fmt --package eval_cli -- --check` - `cargo test -p eval_cli --no-run` - `./script/clippy -p eval_cli` Release Notes: - N/A
21 lines
618 B
Bash
Executable file
21 lines
618 B
Bash
Executable file
#!/usr/bin/env bash
|
|
#
|
|
# Run the zed-eval Python CLI from this checkout without manually setting
|
|
# PYTHONPATH or installing the tool globally.
|
|
#
|
|
# Usage:
|
|
# crates/eval_cli/script/zed-eval doctor
|
|
|
|
set -euo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
|
PACKAGE_DIR="$REPO_ROOT/crates/eval_cli/zed_eval"
|
|
|
|
if ! command -v uv >/dev/null 2>&1; then
|
|
echo "error: uv is required to run zed-eval from source" >&2
|
|
echo "Install uv from https://docs.astral.sh/uv/getting-started/installation/" >&2
|
|
exit 1
|
|
fi
|
|
|
|
exec uv run --project "$PACKAGE_DIR" zed-eval "$@"
|