mirror of
https://github.com/zed-industries/zed.git
synced 2026-07-10 00:13:29 +00:00
original uri https://github.com/zed-industries/zed
https://zed.dev
4b119fc547
Hi there, I'm Celina from Hugging Face! Opening this PR to add [llama.cpp](https://llama.app) as a model provider # Objective Today Zed users running llama.cpp have to fall back to the generic OpenAI-compatible provider, which means no auto-discovery (the router mode (`llama serve`) discovers models from the cache and loads them on demand) and manual configuration of every model and its capabilities. This PR makes `llama.cpp` a first-class provider with the same auto-discovery experience. ## Solution - Add a `llama_cpp` client crate with the OpenAI-compatible chat types (`/v1/chat/completions`, including `reasoning_content`) and the discovery types (`/v1/models`, `/props`), mirroring the existing `ollama` crate. - Add the provider in `crates/language_models/src/provider/llama_cpp.rs`, modeled on the Ollama provider (settings, configuration view, event mapping). - Auto discover served models and their context length and tool/vision support from `/props`. Set `auto_discover: false` to list models manually instead. - An unloaded model can't be inspected without loading it, so it is listed with optimistic defaults (large context, tools enabled) and is usable from the first message; its real context length and tool support are filled in once it loads. These live behind a shared map, so a model already selected in an open conversation picks them up without being re-selected. - Show load progress. The provider subscribes to `/models/sse` and surfaces each model's load progress (e.g. "Loading weights 42%") in its display name, reconciling stale labels against `/v1/models`. Builds without `/models/sse` degrade gracefully - no progress, and no capability refresh after the initial discovery. - Add settings (`api_url`, `auto_discover`, `available_models` with per-model `max_tokens` / `supports_tools` / `supports_images`, `context_window`, `custom_headers`), the provider icon, a `default.json` entry, and documentation under "Use a Local Model". No new dependencies: the crate reuses existing workspace dependencies, and shared state uses `std::sync::RwLock`. ## Testing - Unit tests in both new crates cover wire/response parsing, model discovery for single-model and router shapes, the cold-start optimistic defaults, the in-place capability refresh once a model loads, and the `/models/sse` event handling (state changes, load failure, load progress). - Built Zed locally and ran it against a local `llama serve` router: confirmed models are discovered without manual configuration, that the first message works before a model has finished loading, that load progress is shown in the model's display name while it loads, and that the reported context length and tool support refine to the model's real values once it finishes loading. - Platforms: tested on macOS (Apple Silicon). ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase The generation speed (tokens/sec) depends on the machine you're running the model, here it's a Apple M3 Max 64GB running a 4-bit quant of https://huggingface.co/Qwen/Qwen3.5-35B-A3B. For the load progress status, make sure to upgrade your llama.cpp version to the latest build. https://github.com/user-attachments/assets/0254f6ef-abe9-42ed-810b-ef1a5b8fa3bd --- Release Notes: - Added llama.cpp as a language model provider --------- Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com> |
||
|---|---|---|
| .agents/skills | ||
| .cargo | ||
| .cloudflare | ||
| .config | ||
| .factory | ||
| .github | ||
| .zed | ||
| assets | ||
| ci | ||
| crates | ||
| docs | ||
| extensions | ||
| legal | ||
| nix | ||
| script | ||
| tooling | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| .prettierrc | ||
| .rules | ||
| AGENTS.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CLAUDE.md | ||
| clippy.toml | ||
| CODE_OF_CONDUCT.md | ||
| compose.yml | ||
| CONTRIBUTING.md | ||
| debug.plist | ||
| default.nix | ||
| Dockerfile-collab | ||
| Dockerfile-collab.dockerignore | ||
| Dockerfile-cross.dockerignore | ||
| Dockerfile-distros | ||
| Dockerfile-distros.dockerignore | ||
| flake.lock | ||
| flake.nix | ||
| GEMINI.md | ||
| LICENSE-APACHE | ||
| LICENSE-GPL | ||
| livekit.yaml | ||
| lychee.toml | ||
| Procfile | ||
| Procfile.web | ||
| README.md | ||
| renovate.json | ||
| REVIEWERS.conl | ||
| rust-toolchain.toml | ||
| rustfmt.toml | ||
| shell.nix | ||
| typos.toml | ||
Zed
Welcome to Zed, a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
Installation
On macOS, Linux, and Windows you can download Zed directly or install Zed via your local package manager (macOS/Linux/Windows).
Other platforms are not yet available:
- Web (tracking discussion)
Developing Zed
Contributing
See CONTRIBUTING.md for ways you can contribute to Zed.
Also... we're hiring! Check out our jobs page for open roles.
Licensing
Zed source code is licensed primarily under GPL-3.0-or-later, with Apache-2.0 components where marked.
License information for third party dependencies must be correctly provided for CI to pass.
We use cargo-about to automatically comply with open source licenses. If CI is failing, check the following:
- Is it showing a
no license specifiederror for a crate you've created? If so, addpublish = falseunder[package]in your crate's Cargo.toml. - Is the error
failed to satisfy license requirementsfor a dependency? If so, first determine what license the project has and whether this system is sufficient to comply with this license's requirements. If you're unsure, ask a lawyer. Once you've verified that this system is acceptable add the license's SPDX identifier to theacceptedarray inscript/licenses/zed-licenses.toml. - Is
cargo-aboutunable to find the license for a dependency? If so, add a clarification field at the end ofscript/licenses/zed-licenses.toml, as specified in the cargo-about book.
Sponsorship
Zed is developed by Zed Industries, Inc., a for-profit company.
If you’d like to financially support the project, you can do so via GitHub Sponsors. Sponsorships go directly to Zed Industries and are used as general company revenue. There are no perks or entitlements associated with sponsorship.