Commit graph

6 commits

Author SHA1 Message Date
Max Brunsfeld
d4e89f9587
Add edit prediction evals that test related excerpt usage (#50256)
I've also fixed a race condition with the programmatic context retrieval
in the CLI, which was causing no excerpts to be fetched for the Rust
examples.

Release Notes:

- N/A
2026-02-26 15:36:29 -08:00
Max Brunsfeld
3da9adc4e7
Edit prediction: add some typescript evals, fix a teacher model failure mode (#50141)
In some evals, the teacher produced hallucinations, seemingly due to
context rot. This makes the zeta prompt crate's budgeted rendering
usable by the teacher, so that it can truncate the list of excerpts.

I've also cleaned up the implementation of zeta_prompt's
`format_related_files_within_budget`, and changed the behavior so that
it filters the the excerpts by priority but renders the files in their
original order.

Release Notes:

- N/A
2026-02-25 15:47:08 -08:00
Oleksiy Syvokon
67d9e7d011
ep: Fix teacher parser and other fixes (#49863)
Release Notes:

- N/A
2026-02-23 07:18:04 +00:00
Oleksiy Syvokon
3129d7e6b5
ep: Parse "user accepted prediction" markers in evals (#49598)
Also:
- Add two evals
- Remove duplicated Example 6 from the teacher prompt 

Release Notes:

- N/A
2026-02-19 13:01:07 +00:00
Oleksiy Syvokon
d4af86001c
ep: Update teacher prompt (#49489)
* More conservative predictions for prose
* Explain "user accepted prediction" in the teacher prompt
* Sonnet 4.6 support
* Don't strip comments in teacher prompt's edit history

Release Notes:

- N/A
2026-02-18 15:26:47 +00:00
Max Brunsfeld
6cbeb84880
Tune edit prediction teacher to leave fewer blank spots in predictions (#49315)
Release Notes:

- N/A
2026-02-16 16:38:43 -08:00