Follow-up to #57772.
While comparing the `edit_file_tool_streaming/medium_insertions`
benchmark in Criterion and xctrace, I noticed the DP scoring loop was
spending most of its time on inlined `Matrix::get`/`Matrix::set` access
in `StreamingDiff::push_new`.
This changes the loop to split the previous and current score columns
once per DP column and then index those column slices directly. This
keeps the scoring algorithm the same, but hoists the column offset
calculation out of the inner loop and gives the compiler clearer
aliasing information between the immutable previous column and mutable
current column.
In xctrace, the hottest DP scoring line dropped from 10,407 samples
before to 9,409 samples after on
`edit_file_tool_streaming/medium_insertions`, and total
`StreamingDiff::push_new` leaf samples dropped from 10,800 to 9,822.
This is consistently faster in the end-to-end `edit_file_tool`
benchmark, and gives a small speed-up for the targeted workload without
changing the scoring algorithm.
### Results
Benchmarked with `release-fast` against a clean baseline using
Criterion’s `before_slice` baseline.
#### `streaming_diff_push_new`
| Fixture | Before median | After median | Change |
| --- | ---: | ---: | ---: |
| `tiny_function_rewrite` | `7.6317 ms` | `6.4951 ms` | 10.8% faster |
| `small_function_rewrite` | `45.961 ms` | `44.839 ms` | No significant
change |
| `medium_many_small_changes` | `99.694 ms` | `104.83 ms` | 5.2% slower
|
| `medium_insertions` | `95.909 ms` | `83.093 ms` | 13.5% faster |
#### `edit_file_tool_streaming`
| Fixture | Before median | After median | Change |
| --- | ---: | ---: | ---: |
| `tiny_function_rewrite` | `1.1110 ms` | `1.0617 ms` | 7.1% faster |
| `small_function_rewrite` | `2.4554 ms` | `2.2751 ms` | 7.1% faster |
| `medium_many_small_changes` | `87.272 ms` | `77.065 ms` | 11.5% faster
|
| `medium_insertions` | `92.469 ms` | `87.478 ms` | 5.9% faster |
The lower-level `streaming_diff_push_new` benchmark is mixed, but the
end-to-end `edit_file_tool_streaming` workload improves across all
fixtures.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Release Notes:
- N/A or Added/Fixed/Improved ...
## Motivation
Miniprofs of Zed dropping frames showed foreground thread stalls of
711.56 ms, 641.01 ms, and 221.99 ms from the `edit_file_tool`. The new
benchmark isolated `StreamingDiff::push_new` as one of the expensive
phases so this PR aims to speed it up so we can avoid dropping more
frames in the future.
## Benchmark methodology
I had an agent create a Criterion suite with four deterministic Rust
edit fixtures: tiny localized rewrite, small localized rewrite, many
small changes, and helper block insertions.
Benchmarked with:
```sh
cargo bench -p streaming_diff --bench streaming_diff --profile release-fast -- --warm-up-time 1 --measurement-time 2
```
The benchmark binary was also recorded under `xctrace` CPU Counters to
inspect CPU samples before and after the change.
## Results
`StreamingDiff::push_new` improved across all fixtures:
| Benchmark | Before | After | Improvement |
|---|---:|---:|---:|
| `tiny_function_rewrite` | ~10.81 ms | ~6.91 ms | ~36% faster |
| `small_function_rewrite` | ~51.02 ms | ~35.39 ms | ~31% faster |
| `medium_many_small_changes` | ~130.71 ms | ~83.83 ms | ~36% faster |
| `medium_insertions` | ~120.52 ms | ~79.90 ms | ~34% faster |
The `xctrace` baseline showed samples in `Hash::hash`,
`RandomState::hash_one`, `HashMap::insert`, and
`RawTable::reserve_rehash`. After replacing the map with two `Vec<u32>`
row buffers, the hash table frames disappeared from the top samples.
The speedup comes from removing hash work from the DP inner loop and
replacing scattered hash table probes with contiguous row buffer access
(better cache hits 🏎️). The trace did not include direct cache miss
counts, but this layout is likely more cache friendly because it
replaces hash table access with a continuous line of memory (the
`Vec`s).
Release Notes:
- N/A
Extracts and cleans up GPUI's scheduler code into a new `scheduler`
crate, making it pluggable by external runtimes. This will enable
deterministic integration testing with cloud components by providing a
unified test scheduler across Zed and backend code. In Zed, it will
replace the existing GPUI scheduler for consistent async task management
across platforms.
## Changes
- **Core Implementation**: `TestScheduler` with seed-based
randomization, session tracking (`SessionId`), and foreground/background
task separation for reproducible testing.
- **Executors**: `ForegroundExecutor` (!Send, thread-local) and
`BackgroundExecutor` (Send, with blocking/timeout support) as
GPUI-compatible wrappers.
- **Clock and Timer**: Controllable `TestClock` and future-based `Timer`
for time-sensitive tests.
- **Testing APIs**: `once()`, `with_seed()`, and `many()` methods for
configurable test runs.
- **Dependencies**: Added `async-task`, `chrono`, `futures`, etc., with
updates to `Cargo.toml` and lock file.
## Benefits
- **Integration Testing**: Facilitates reliable async tests involving
cloud sessions, reducing flakiness via deterministic execution.
- **Pluggability**: Trait-based design (`Scheduler`) allows easy
integration into non-GPUI runtimes while maintaining GPUI compatibility.
- **Cleanup**: Refactors GPUI scheduler logic for clarity, correctness
(no `unwrap()`, proper error handling), and extensibility.
Follows Rust guidelines; run `./script/clippy` for verification.
- [x] Define and test a core scheduler that we think can power our cloud
code and GPUI
- [ ] Replace GPUI's scheduler
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
This pull request introduces a new tool for streaming edits. The
short-term goal is for this tool to replace the existing `EditFileTool`,
but we want to get this out the door as soon as possible so that we can
start testing it.
`StreamingEditFileTool` is mutually exclusive with `EditFileTool`. It
will be enabled by default for anyone who has the `agent-stream-edits`
feature flag, as well as people that set `assistant.stream_edits` to
`true` in their settings.
### Implementation
Streaming is achieved by requesting a completion while the `edit_file`
tool gets called. We invoke the model by taking the existing
conversation with the agent and appending a prompt specifically tailored
for editing. In that prompt, we ask the model to produce a stream of
`<old_text>`/`<new_text>` tags. As the model streams text in, we
incrementally parse it and start editing as soon as we can.
### Evals
Note that, as part of this pull request, I also defined some new evals
that I used to drive the behavior of the recursive LLM call. To run
them, use this command:
```bash
cargo test --package=assistant_tools --features eval -- eval_extract_handle_command_output
```
Or comment out the `#[cfg_attr(not(feature = "eval"), ignore)]` macro.
I recommend running them one at a time, because right now we don't
really have a way of orchestrating of all these evals. I think we should
invest into that effort once the new agent panel goes live.
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
This PR extracts the streaming diff implementation to its own
`streaming_diff` crate.
It was duplicated between `assistant` and `assistant2`, but their
implementations were exactly the same (and I don't see a reason why they
would need to diverge).
Release Notes:
- N/A
