OpenAI-compatible providers currently cannot reuse Zed's OpenAI Chat
Completions transport unless they also adopt the exact OpenAI request
and response types. OpenRouter therefore maintained its own copy of
request construction, authentication, status handling, response reading,
and server-sent event framing.
This change extracts that mechanical transport into two provider-neutral
functions in `open_ai`: one for streaming requests and one for
non-streaming requests. They accept any serializable request envelope,
preserve custom headers and provider names, return untyped JSON for
provider-specific decoding, and retain typed failures for serialization,
request construction, HTTP transport, response reading, and
deserialization. The existing OpenAI entry points remain as
compatibility wrappers, so existing callers keep the same API and
behavior.
The abstraction deliberately stops at the wire boundary. OpenRouter
continues to own its request and response schemas, attribution headers,
routing controls, cache placement, and API-specific error
interpretation. It now adapts the shared framed stream into those
OpenRouter types instead of implementing a second HTTP and server-sent
event stack.
Moving OpenRouter onto the shared path also requires the ordinary Chat
Completions schema and event mapper to preserve compatible metadata that
OpenRouter already emits. This includes structured reasoning details
needed for replay, fragmented reasoning accumulation, prompt-cache read
and write usage, and thought signatures attached to tool calls. The
stream exposes `[DONE]` explicitly rather than treating it as
indistinguishable from an unexpected end of the response body.
OpenRouter's routing session identifier is hashed before transmission so
Zed's internal thread identifier is not exposed.
The diff is larger than the extracted transport alone because the shared
API is additive, the compatibility metadata must be represented in the
common wire types and event mapper, and the provider-specific adapter
remains intentionally independent. Roughly four hundred added lines are
focused transport, metadata, error-classification, attribution, caching,
and privacy tests. The provider-level OpenRouter implementation becomes
smaller while preserving its existing behavior.
Testing performed:
- `cargo test -p open_ai`
- `cargo test -p open_router`
- `cargo nextest run -p language_models open_router`
- `cargo check -p edit_prediction -p edit_prediction_cli`
- `cargo fmt --all -- --check`
- `./script/clippy -p open_ai -p open_router -p language_models -p
edit_prediction_cli`
- `cargo machete`
Release Notes:
- Improved OpenRouter reasoning continuity and request privacy.
---------
Co-authored-by: Eric Holk <eric@zed.dev>
Closes https://github.com/zed-industries/zed/issues/52576
## Overview
Adds prompt caching support for Anthropic Claude models when accessed
via OpenRouter.
This mirrors Zed's native Anthropic provider by using explicit per-block
`cache_control` breakpoints:
- A long-lived 1-hour breakpoint on the system message's last text
block, covering the tools and system prefix.
- A default 5-minute breakpoint on the last `cache: true` conversation
message.
The implementation intentionally avoids OpenRouter's top-level automatic
`cache_control` field so routing remains available across
Anthropic-compatible providers including Anthropic, Bedrock, and Vertex
AI. It also adds `session_id` sticky routing from the request thread ID
and maps OpenRouter cache usage fields into Zed's token usage
accounting.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Release Notes:
- Fixed Anthropic prompt caching when using OpenRouter.
---------
Co-authored-by: Anant Goel <anant@zed.dev>
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [ ] Unsafe blocks (if any) have justifying comments
- [ ] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [ ] Performance impact has been considered and is acceptable
`list_models` reads the response body once up front, then the error
branch read it
*again* into a shadowing `body`. Since the stream was already drained,
the second read
returned `""`, so error parsing always failed and we surfaced an empty
message instead
of the real API error.
This removes the redundant second read so the error branch uses the body
we already have.
Release Notes:
- Fixed OpenRouter API errors showing empty messages when fetching the
model list
Signed-off-by: Zhiwei Liang <zhiwei.liang@zliang.me>
Add a `custom_headers` setting to each HTTP-based language model
provider
(Anthropic, Bedrock, DeepSeek, Google, LM Studio, Mistral, Ollama,
OpenAI,
OpenAI-compatible, OpenCode, OpenRouter, Vercel AI Gateway, and xAI) so
users
can attach extra headers to every outgoing request. Headers managed by
Zed
(authentication, content-type, etc.) cannot be overridden and are
skipped with
a warning.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Release Notes:
- Added support for configuring custom HTTP headers on language model
providers via `language_models.<provider>.custom_headers`.
Signed-off-by: Aurabindo Pillai <mail@aurabindo.in>
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Release Notes:
- N/A
For long threads we will spend more and more time cloning the messages
just to save them to the database, as we need a copy of everything to do
so asynchronously. Messages are really expensive to clone though and we
accumulate a lot of them really fast, so even for smaller threads we
start seeing pauses in the millisecond range. The fix to this is fairly
simple though, we never mutate the messages once pushed to the vec, so
just Arc them.
This PR also slightly changes `UserMessage` to be a bit faster to clone
as well.
Release Notes:
- Fixed a cause of stutters when interacting with the agent
- `language_model` no longer depends on provider-specific crates such as
`anthropic` and `open_ai` (inverted dependency)
- `language_model_core` was extracted from `language_model` which
contains the types for the provider-specific crates to convert to/from.
- `gpui::SharedString` has been extracted into its own crate (still
exposed by `gpui`), so `language_model_core` and provider API crates
don't have to depend on `gpui`.
- Removes some unnecessary `&'static str` | `SharedString` -> `String`
-> `SharedString` conversions across the codebase.
- Extracts the core logic of the cloud `LanguageModelProvider` into its
own crate with simpler dependencies.
Release Notes:
- N/A
---------
Co-authored-by: John Tur <john-tur@outlook.com>
Closes#37525
By default, thread summary uses default_fast_model (if set), otherwise
default_model, which resolves to openrouter/auto for openrouter
provider. This may cause the summary to be generated by a different
model than the one used by the agent, potentially leading — in cases
such as Claude Opus 4.5 — to summary costs exceeding main agent
execution costs.
The current logic in registry.rs prioritizes default_fast_model over
default_model, which overrides the user-selected model (assigned only to
default_model). Setting default_fast_model = None for the OpenRouter
provider preserves the fallback to openrouter/auto when no model is
chosen, while respecting the user's explicit model selection when one is
provided.
```rust
pub fn set_default_model(&mut self, model: Option<ConfiguredModel>, cx: &mut Context<Self>) {
match (self.default_model.as_ref(), model.as_ref()) {
(Some(old), Some(new)) if old.is_same_as(new) => {}
(None, None) => {}
_ => cx.emit(Event::DefaultModelChanged),
}
self.default_fast_model = maybe!({
let provider = &model.as_ref()?.provider;
let fast_model = provider.default_fast_model(cx)?;
Some(ConfiguredModel {
provider: provider.clone(),
model: fast_model,
})
}); // This sets default fast model (in our case openrouter/auto)
self.default_model = model; //This sets default_model to user selected model
}
```
And latter on :
```rust
pub fn thread_summary_model(&self) -> Option<ConfiguredModel> {
#[cfg(debug_assertions)]
if std::env::var("ZED_SIMULATE_NO_LLM_PROVIDER").is_ok() {
return None;
}
self.thread_summary_model
.clone()
.or_else(|| self.default_fast_model.clone()) // We pick fast_model over default model here
.or_else(|| self.default_model.clone())
}
```
Which results in user choice being ignored.
Proposed behavior:
Use the model explicitly selected by the user in Zed agent
configuration.
If no model is specified, fall back to the configured default.
The resolution is to set in : provider/open_router.rs
```rust
fn default_fast_model(&self, _cx: &App) -> Option<Arc<dyn LanguageModel>> {
None
}
```
This will have a consequence of default_fast_model not being provided
and falling back to user choice - but once the fast model is set via for
example a configuration property - the default_fast_model is picked over
default_model
Release Notes:
- open_router: Use user's default model when comments and thread summary
We've been considering removing workspace-hack for a couple reasons:
- Lukas ran into a situation where its build script seemed to be causing
spurious rebuilds. This seems more likely to be a cargo bug than an
issue with workspace-hack itself (given that it has an empty build
script), but we don't necessarily want to take the time to hunt that
down right now.
- Marshall mentioned hakari interacts poorly with automated crate
updates (in our case provided by rennovate) because you'd need to have
`cargo hakari generate && cargo hakari manage-deps` after their changes
and we prefer to not have actions that make commits.
Currently removing workspace-hack causes our workspace to grow from
~1700 to ~2000 crates being built (depending on platform), which is
mainly a problem when you're building the whole workspace or running
tests across the the normal and remote binaries (which is where
feature-unification nets us the most sharing). It doesn't impact
incremental times noticeably when you're just iterating on `-p zed`, and
we'll hopefully get these savings back in the future when
rust-lang/cargo#14774 (which re-implements the functionality of hakari)
is finished.
Release Notes:
- N/A
Co-Authored-By: Ben K <ben@zed.dev>
Co-Authored-By: Anthony <anthony@zed.dev>
Co-Authored-By: Mikayla <mikayla@zed.dev>
Release Notes:
- settings: Major internal changes to settings. The primary user-facing
effect is that some settings which did not make sense in project
settings files are no-longer read from there. (For example the inline
blame settings)
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: Mikayla Maki <mikayla.c.maki@gmail.com>
Co-authored-by: Anthony <anthony@zed.dev>
Supersedes: #34500
Also this will allow to fix this: #35386 without the UX changes but
providers can now be control through settings as well within zed.
Just rebased the latest main and docs added. Added @AurelienTollard as
co-author as it was started by him everything else remains the same from
original PR.
Release Notes:
- Added ability to control Provider Routing for OpenRouter models from
settings.
Co-authored-by: Aurelien Tollard <tollard.aurelien1999@gmail.com>
This PR switches the OpenRouter integration from fetching all models to
fetching only the models specified in the user's account preferences.
This will help improve the experience
**The Problem**
The previous implementation used the `/models` endpoint, which returned
an exhaustive list of all models supported by OpenRouter. This resulted
in a long and cluttered model selection dropdown in Zed, making it
difficult for users to find the models they actually use.
**The Solution**
We now use the `/models/user` endpoint. This API call returns a curated
list based on the models and providers the user has selected in their
[OpenRouter dashboard](https://openrouter.ai/models).
Ref: [OpenRouter API Docs for User-Filtered
Models](https://openrouter.ai/docs/api-reference/list-models-filtered-by-user-provider-preferences)
Release Notes:
- language_models: Support OpenRouter user preferences for available
models
Improves the error handling for openrouter and adds automatic retry like
anthropic for few of the status codes.
Release Notes:
- Improves error messages for Openrouter provider
- Automatic retry when rate limited or Server error from Openrouter
This prevents the common footgun of copy/pasting an API key
starting/ending with extra newlines, which would lead to a "bad request"
error.
Closes#37038
Release Notes:
- agent: Support pasting language model API keys that contain newlines.
Closes#34314
This PR resolves an issue where serde(untagged) caused Rust None values
to serialize as null, which OpenRouter's Mistral API (when tool_choice
is present) incorrectly interprets as a defined value, leading to a 400
error. By replacing serde(untagged) with serde(snake_case), None values
are now correctly omitted from the serialized JSON, fixing the problem.
P.S. A separate PR will address serde(untagged) usage for other
providers, as null is not expected for them either.
Release Notes:
- Fix ToolChoice getting serialized to null on OpenRouter
Did some bit cleanup of code for loading models for settings as that is
not required as we are fetching all the models from openrouter so it's
better to maintain one source of truth
Release Notes:
- Add thinking support to OpenRouter provider
Previously we were using a mix of `u32` and `usize`, e.g. `max_tokens:
usize, max_output_tokens: Option<u32>` in the same `struct`.
Although [tiktoken](https://github.com/openai/tiktoken) uses `usize`,
token counts should be consistent across targets (e.g. the same model
doesn't suddenly get a smaller context window if you're compiling for
wasm32), and these token counts could end up getting serialized using a
binary protocol, so `usize` is not the right choice for token counts.
I chose to standardize on `u64` over `u32` because we don't store many
of them (so the extra size should be insignificant) and future models
may exceed `u32::MAX` tokens.
Release Notes:
- N/A
We push the usage data whenever we receive it from the provider to make
sure the counting is correct after the turn has ended.
- [x] Ollama
- [x] Copilot
- [x] Mistral
- [x] OpenRouter
- [x] LMStudio
Put all the changes into a single PR open to move these to separate PR
if that makes the review and testing easier.
Release Notes:
- N/A
- [x] Manual Testing(Tested this with Qwen2.5 VL 32B Instruct (free) and
Llama 4 Scout (free), Llama 4 Maverick (free). Llama models have some
issues in write profile due to one of the in built tools schema, so I
tested it with minimal profile.
Closes #ISSUE
Release Notes:
- Add image support to OpenRouter models
---------
Signed-off-by: Umesh Yadav <umesh4257@gmail.com>
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
This pull request adds full integration with OpenRouter, allowing users
to access a wide variety of language models through a single API key.
**Implementation Details:**
* **Provider Registration:** Registers OpenRouter as a new language
model provider within the application's model registry. This includes UI
for API key authentication, token counting, streaming completions, and
tool-call handling.
* **Dedicated Crate:** Adds a new `open_router` crate to manage
interactions with the OpenRouter HTTP API, including model discovery and
streaming helpers.
* **UI & Configuration:** Extends workspace manifests, the settings
schema, icons, and default configurations to surface the OpenRouter
provider and its settings within the UI.
* **Readability:** Reformats JSON arrays within the settings files for
improved readability.
**Design Decisions & Discussion Points:**
* **Code Reuse:** I leveraged much of the existing logic from the
`openai` provider integration due to the significant similarities
between the OpenAI and OpenRouter API specifications.
* **Default Model:** I set the default model to `openrouter/auto`. This
model automatically routes user prompts to the most suitable underlying
model on OpenRouter, providing a convenient starting point.
* **Model Population Strategy:**
* <strike>I've implemented dynamic population of available models by
querying the OpenRouter API upon initialization.
* Currently, this involves three separate API calls: one for all models,
one for tool-use models, and one for models good at programming.
* The data from the tool-use API call sets a `tool_use` flag for
relevant models.
* The data from the programming models API call is used to sort the
list, prioritizing coding-focused models in the dropdown.</strike>
* <strike>**Feedback Welcome:** I acknowledge this multi-call approach
is API-intensive. I am open to feedback and alternative implementation
suggestions if the team believes this can be optimized.</strike>
* **Update: Now this has been simplified to one api call.**
* **UI/UX Considerations:**
* <strike>Authentication Method: Currently, I've implemented the
standard API key input in settings, similar to other providers like
OpenAI/Anthropic. However, OpenRouter also supports OAuth 2.0 with PKCE.
This could offer a potentially smoother, more integrated setup
experience for users (e.g., clicking a button to authorize instead of
copy-pasting a key). Should we prioritize implementing OAuth PKCE now,
or perhaps add it as an alternative option later?</strike>(PKCE is not
straight forward and complicated so skipping this for now. So that we
can add the support and work on this later.)
* <strike>To visually distinguish models better suited for programming,
I've considered adding a marker (e.g., `</>` or `🧠`) next to their
names. Thoughts on this proposal?</strike>. (This will require a changes
and discussion across model provider. This doesn't fall under the scope
of current PR).
* OpenRouter offers 300+ models. The current implementation loads all of
them. **Feedback Needed:** Should we refine this list or implement more
sophisticated filtering/categorization for better usability?
**Motivation:**
This integration directly addresses one of the most highly upvoted
feature requests/discussions within the Zed community. Adding OpenRouter
support significantly expands the range of AI models accessible to
users.
I welcome feedback from the Zed team on this implementation and the
design choices made. I am eager to refine this feature and make it
available to users.
ISSUES: https://github.com/zed-industries/zed/discussions/16576
Release Notes:
- Added support for OpenRouter as a language model provider.
---------
Signed-off-by: Umesh Yadav <umesh4257@gmail.com>
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
