mirror of
https://github.com/zed-industries/zed.git
synced 2026-07-10 00:13:29 +00:00
10f501d700
Summary: - Add the `zed-eval` Python CLI for Modal/Harbor/Pier benchmark orchestration, including content-addressed remote builds, run/suite management, reporting, rejudge, baseline, and cleanup workflows. - Extend `eval-cli` for remote evals with provider/model overrides and step/tool-call metrics in `result.json`. - Add install/source-run helper scripts so `zed-eval` can be installed or run from the checkout without manually setting `PYTHONPATH`. - Harden the remote harness wrappers around exit-code preservation, archive extraction, custom secret wiring, and Harbor/Pier option parity, with regression coverage. Testing: - Using the CLI for two weeks - `PYTHONPATH=crates/eval_cli python3 -m compileall -q crates/eval_cli/zed_eval` - `uv run --project crates/eval_cli/zed_eval python -m unittest discover -s crates/eval_cli/zed_eval/tests` - `bash -n crates/eval_cli/script/install-zed-eval crates/eval_cli/script/zed-eval` - `cargo check -p eval_cli` - `cargo fmt --package eval_cli -- --check` - `cargo test -p eval_cli --no-run` - `./script/clippy -p eval_cli` Release Notes: - N/A
27 lines
847 B
Bash
Executable file
27 lines
847 B
Bash
Executable file
#!/usr/bin/env bash
|
|
#
|
|
# Install or update the zed-eval Python CLI as an editable uv tool.
|
|
#
|
|
# Usage:
|
|
# crates/eval_cli/script/install-zed-eval
|
|
#
|
|
# After this, `zed-eval` is available on PATH (assuming uv's tool bin directory
|
|
# is on PATH) and tracks changes in crates/eval_cli/zed_eval without reinstalling.
|
|
|
|
set -euo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
|
PACKAGE_DIR="$REPO_ROOT/crates/eval_cli/zed_eval"
|
|
|
|
if ! command -v uv >/dev/null 2>&1; then
|
|
echo "error: uv is required to install zed-eval" >&2
|
|
echo "Install uv from https://docs.astral.sh/uv/getting-started/installation/" >&2
|
|
exit 1
|
|
fi
|
|
|
|
uv tool install --editable "$PACKAGE_DIR" --force
|
|
|
|
echo ""
|
|
echo "Installed zed-eval from $PACKAGE_DIR"
|
|
echo "If your shell cannot find 'zed-eval', run: uv tool update-shell"
|