Agus Zubiaga ce1a674eba eval: Fine-grained assertions (#29246) ... - Support programmatic examples ([example](17feb260a0/crates/eval/src/examples/file_search.rs)) - Combine data-driven example declarations into a single `.toml` file ([example](17feb260a0/crates/eval/src/examples/find_and_replace_diff_card.toml)) - Run judge on individual assertions (previously called "criteria") - Report judge and programmatic assertions in one combined table Note: We still need to work on concept naming <img width=400 src="https://github.com/user-attachments/assets/fc719c93-467f-412b-8d47-68821bd8a5f5"> Release Notes: - N/A --------- Co-authored-by: Richard Feldman <oss@rtfeldman.com> Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com> Co-authored-by: Thomas Mickley-Doyle <tmickleydoyle@gmail.com>		2025-04-22 23:58:58 -03:00
..
activity_indicator	Git activity indicator (#28204)	2025-04-07 18:10:01 +00:00
agent	eval: Fine-grained assertions (#29246)	2025-04-22 23:58:58 -03:00
anthropic	Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)	2025-04-19 23:26:29 +00:00
askpass	Remove netcat dependency (#28920)	2025-04-16 19:46:36 -04:00
assets	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
assistant	Add ability to attach rules as context (#29109)	2025-04-21 20:16:51 +00:00
assistant_context_editor	agent: Support pasting images as context (#29177)	2025-04-22 09:01:01 +00:00
assistant_settings	Actually run the eval and fix a hang when retrieving outline (#28547)	2025-04-11 00:01:33 +00:00
assistant_slash_command	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
assistant_slash_commands	Add ability to attach rules as context (#29109)	2025-04-21 20:16:51 +00:00
assistant_tool	Streaming tool calls (#29179)	2025-04-21 22:28:32 +00:00
assistant_tools	eval: Fine-grained assertions (#29246)	2025-04-22 23:58:58 -03:00
audio	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
auto_update	windows: Implement AutoUpdater (#25734)	2025-04-14 10:36:31 -07:00
auto_update_helper	Update block diagnostics (#28006)	2025-04-15 09:35:13 -06:00
auto_update_ui	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
aws_http_client	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
bedrock	Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)	2025-04-19 23:26:29 +00:00
breadcrumbs	breadcrumbs: Update multibuffer to match singleton (#28267)	2025-04-07 20:26:55 +00:00
buffer_diff	Fix diff recalculation hang (#28377)	2025-04-10 22:58:41 +00:00
call	Fix deafening new participants (#28330)	2025-04-08 16:01:27 +00:00
channel	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
cli	Try to weak-link ScreenCaptureKit always (#28585)	2025-04-11 17:38:14 +00:00
client	Add new action to run agent eval (#29158)	2025-04-21 21:30:21 -07:00
clock	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
collab	debugger: Open debugger panel on session startup (#29186)	2025-04-22 19:35:47 -04:00
collab_ui	Rename regex search tool to grep and accept an include glob pattern (#29100)	2025-04-20 00:53:30 +00:00
collections	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
command_palette	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
command_palette_hooks	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
component	Add example agent tool preview (#28984)	2025-04-17 17:29:19 +00:00
component_preview	component preview: Focus search input immediately upon opening (#29155)	2025-04-21 11:28:49 -03:00
context_server	agent: Return ToolResult from run inside Tool (#28763)	2025-04-15 14:28:09 +00:00
context_server_settings	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
copilot	Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)	2025-04-19 23:26:29 +00:00
credentials_provider	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
dap	debugger: Open debugger panel on session startup (#29186)	2025-04-22 19:35:47 -04:00
dap_adapters	debugger: More tidy up for SSH (#28993)	2025-04-21 16:00:03 +00:00
db	debugger_ui: Move DEBUGGER_PANEL_PREFIX out of db (#28768)	2025-04-15 14:59:42 +00:00
debugger_tools	debugger: Open debugger panel on session startup (#29186)	2025-04-22 19:35:47 -04:00
debugger_ui	debugger: Open debugger panel on session startup (#29186)	2025-04-22 19:35:47 -04:00
deepseek	Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)	2025-04-19 23:26:29 +00:00
diagnostics	Fix duplicated multi-buffer excerpts (#29193)	2025-04-22 05:25:09 +00:00
docs_preprocessor	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
editor	editor: Use quantize score for code completions sort + Add code completions tests (#29182)	2025-04-23 07:23:34 +05:30
eval	eval: Fine-grained assertions (#29246)	2025-04-22 23:58:58 -03:00
extension	extension: Use heck instead of convert_case for snake_case check (#28608)	2025-04-11 22:01:25 +00:00
extension_api	zed_extension_api: Release v0.4.0 (#29237)	2025-04-22 21:07:52 +00:00
extension_cli	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
extension_host	zed_extension_api: Release v0.4.0 (#29237)	2025-04-22 21:07:52 +00:00
extensions_ui	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
feature_flags	agent: Add websearch tool (#28621)	2025-04-16 19:25:00 +02:00
feedback	feedback: Update issue template URL (#28790)	2025-04-15 21:36:30 -04:00
file_finder	agent: Refine language model selector (#28597)	2025-04-11 23:02:50 +00:00
file_icons	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
fs	Implement dragging external files to remote projects (#28987)	2025-04-17 11:06:56 -07:00
fsevent	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
fuzzy	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
git	Add support for remote branches to the branch picker (#28978)	2025-04-17 16:13:02 +00:00
git_hosting_providers	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
git_ui	Fix duplicated multi-buffer excerpts (#29193)	2025-04-22 05:25:09 +00:00
go_to_line	vim: Fix character count in visual line mode (#28669)	2025-04-17 22:14:44 -06:00
google_ai	agent: Add support for google gemini 2.5 flash preview (#29205)	2025-04-22 09:37:12 +00:00
gpui	Adjust image cache APIs to enable ElementState based APIs (#29243)	2025-04-22 22:08:28 +00:00
gpui_macros	Rename regex search tool to grep and accept an include glob pattern (#29100)	2025-04-20 00:53:30 +00:00
gpui_tokio	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
html_to_markdown	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
http_client	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
http_client_tls	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
icons	agent: Support pasting images as context (#29177)	2025-04-22 09:01:01 +00:00
image_viewer	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
indexed_docs	Add --user-data-dir CLI flag and propose renaming support_dir to data_dir (#26886)	2025-04-10 21:16:43 +00:00
inline_completion	Show edit predictions usage in status bar menu (#29046)	2025-04-18 14:15:19 +00:00
inline_completion_button	inline_completion_button: Show the usage limits returned from the API (#29239)	2025-04-22 21:16:54 +00:00
install_cli	install_cli: Show feedback when installing CLI from welcome screen (#28532)	2025-04-11 01:47:40 +05:30
journal	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
language	language: Fix language_scope_at for markdown code comments (#29230)	2025-04-23 01:20:25 +05:30
language_extension	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
language_model	agent: Add support for google gemini 2.5 flash preview (#29205)	2025-04-22 09:37:12 +00:00
language_model_selector	inline assistant: Fix model picker (#29136)	2025-04-21 01:12:57 +00:00
language_models	gemini: Add support for passing images as part of the prompt (#29203)	2025-04-22 09:05:46 +00:00
language_selector	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
language_tools	gpui: Introduce PlatformKeyboardLayout trait for human-friendly keyboard layout names (#29049)	2025-04-19 22:23:03 +08:00
languages	Add new action to run agent eval (#29158)	2025-04-21 21:30:21 -07:00
livekit_api	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
livekit_client	Fix deafening new participants (#28330)	2025-04-08 16:01:27 +00:00
lmstudio	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
lsp	Add support for insert_text_mode of a completion (#28171)	2025-04-07 10:35:11 -06:00
markdown	Streaming tool calls (#29179)	2025-04-21 22:28:32 +00:00
markdown_preview	Use buffer size for markdown preview (#29172)	2025-04-21 19:29:21 -04:00
media	chore: Make objc a workspace level crate (#28258)	2025-04-07 18:46:09 +00:00
menu	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
migrator	agent: Migrate tool names in settings (#29168)	2025-04-21 17:03:42 +00:00
mistral	Switch from open-codestral-mamba to codestral-latest for default mistral model (#29104)	2025-04-19 23:29:36 +00:00
multi_buffer	Fix duplicated multi-buffer excerpts (#29193)	2025-04-22 05:25:09 +00:00
node_runtime	Add --user-data-dir CLI flag and propose renaming support_dir to data_dir (#26886)	2025-04-10 21:16:43 +00:00
notifications	Merge Component and ComponentPreview trait (#28365)	2025-04-08 16:09:06 -06:00
ollama	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
open_ai	Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)	2025-04-19 23:26:29 +00:00
outline	editor: Refactor EditorMode::Full (#28546)	2025-04-10 23:22:27 +00:00
outline_panel	outline_panel: Rename outline_panel::Open to outline_panel::OpenSelectedEntry (#28890)	2025-04-17 01:44:00 +05:30
panel	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
paths	Implement basic support for VS Code debug configurations (#29160)	2025-04-22 14:24:09 +00:00
picker	agent: Refine language model selector (#28597)	2025-04-11 23:02:50 +00:00
prettier	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
project	editor: Use quantize score for code completions sort + Add code completions tests (#29182)	2025-04-23 07:23:34 +05:30
project_panel	agent: Make directory context display update on rename (#29189)	2025-04-22 01:44:31 +00:00
project_symbols	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
prompt_library	Add ability to attach rules as context (#29109)	2025-04-21 20:16:51 +00:00
prompt_store	Add ability to attach rules as context (#29109)	2025-04-21 20:16:51 +00:00
proto	debugger: More tidy up for SSH (#28993)	2025-04-21 16:00:03 +00:00
recent_projects	debugger: More tidy up for SSH (#28993)	2025-04-21 16:00:03 +00:00
refineable	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
release_channel	Fix handling of --system-specs argument so it happens before Application::new (#29240)	2025-04-22 21:32:32 +00:00
remote	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
remote_server	Fix handling of --system-specs argument so it happens before Application::new (#29240)	2025-04-22 21:32:32 +00:00
repl	Upgrade async-tungstenite to tokio (#26193)	2025-04-08 09:17:08 -06:00
reqwest_client	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
rich_text	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
rope	chore: Bump Rust version to 1.86 (#28021)	2025-04-03 23:32:50 +02:00
rpc	rpc: Remove llm module in favor of zed_llm_client (#28900)	2025-04-16 20:22:44 +00:00
schema_generator	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
search	Rename regex search tool to grep and accept an include glob pattern (#29100)	2025-04-20 00:53:30 +00:00
semantic_index	agent: Attach thread ID and prompt ID to telemetry events (#29069)	2025-04-18 20:41:02 +00:00
semantic_version	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
session	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
settings	gpui: Introduce PlatformKeyboardLayout trait for human-friendly keyboard layout names (#29049)	2025-04-19 22:23:03 +08:00
settings_ui	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
snippet	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
snippet_provider	snippet_provider: Use proper casing of VsCode in identifiers (#29038)	2025-04-18 12:11:54 +00:00
snippets_ui	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
sqlez	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
sqlez_macros	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
story	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
storybook	Merge Component and ComponentPreview trait (#28365)	2025-04-08 16:09:06 -06:00
streaming_diff	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
sum_tree	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
supermaven	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
supermaven_api	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
tab_switcher	tab_switcher: Add keybind to close tab tooltip (#27212)	2025-04-08 15:57:36 -06:00
task	Implement basic support for VS Code debug configurations (#29160)	2025-04-22 14:24:09 +00:00
tasks_ui	debugger: More tidy up for SSH (#28993)	2025-04-21 16:00:03 +00:00
telemetry	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
telemetry_events	Add new action to run agent eval (#29158)	2025-04-21 21:30:21 -07:00
terminal	debugger: More tidy up for SSH (#28993)	2025-04-21 16:00:03 +00:00
terminal_view	debugger: More tidy up for SSH (#28993)	2025-04-21 16:00:03 +00:00
text	format: Re-implement support for formatting with code actions that contain commands (#28392)	2025-04-09 01:53:54 +00:00
theme	Add icon for Vyper files (#28307)	2025-04-09 10:49:39 -04:00
theme_extension	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
theme_importer	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
theme_selector	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
time_format	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
title_bar	project: Show detached head commit SHA in branch pickers (#29007)	2025-04-18 04:23:56 +05:30
toolchain_selector	toolchain: Respect currently focused file when querying toolchains (#28875)	2025-04-16 19:05:57 +02:00
ui	ui: Add .color method to the Switch (#29074)	2025-04-21 10:56:42 -03:00
ui_input	Merge Component and ComponentPreview trait (#28365)	2025-04-08 16:09:06 -06:00
ui_macros	Merge Component and ComponentPreview trait (#28365)	2025-04-08 16:09:06 -06:00
ui_prompt	markdown: Don't retain MarkdownStyle in favor of using MarkdownElement directly (#28255)	2025-04-07 19:03:24 +00:00
util	Fix zed sometimes stopping by using setsid on interactive shells (#29070)	2025-04-18 15:04:26 -06:00
util_macros	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
vim	vim: Add cursor shape settings for each vim mode (#28636)	2025-04-21 18:42:04 -06:00
vim_mode_setting	Add workspace-hack (#27277)	2025-04-02 13:26:34 -07:00
web_search	agent: Add websearch tool (#28621)	2025-04-16 19:25:00 +02:00
web_search_providers	agent: Add websearch tool (#28621)	2025-04-16 19:25:00 +02:00
welcome	install_cli: Show feedback when installing CLI from welcome screen (#28532)	2025-04-11 01:47:40 +05:30
workspace	debugger: Open debugger panel on session startup (#29186)	2025-04-22 19:35:47 -04:00
worktree	Systematically optimize agentic editing performance (#28961)	2025-04-19 02:47:59 +00:00
zed	debugger: Open debugger panel on session startup (#29186)	2025-04-22 19:35:47 -04:00
zed_actions	Add ability to attach rules as context (#29109)	2025-04-21 20:16:51 +00:00
zeta	Add new action to run agent eval (#29158)	2025-04-21 21:30:21 -07:00
zlog	zlog: Add default filters (#29244)	2025-04-22 18:54:56 -04:00
zlog_settings	zlog: Use zlog as default log implementation (#28612)	2025-04-14 14:17:07 +00:00