zed/crates/language_models
Eric Holk 984bf4d466
Some checks are pending
Congratsbot / check-author (push) Waiting to run
Congratsbot / congrats (push) Blocked by required conditions
deploy_nightly_docs / deploy_docs (push) Waiting to run
run_tests / orchestrate (push) Waiting to run
run_tests / check_style (push) Waiting to run
run_tests / clippy_windows (push) Blocked by required conditions
run_tests / clippy_linux (push) Blocked by required conditions
run_tests / clippy_mac (push) Blocked by required conditions
run_tests / clippy_mac_x86_64 (push) Blocked by required conditions
run_tests / run_tests_windows (push) Blocked by required conditions
run_tests / run_tests_linux (push) Blocked by required conditions
run_tests / run_tests_mac (push) Blocked by required conditions
run_tests / miri_scheduler (push) Blocked by required conditions
run_tests / doctests (push) Blocked by required conditions
run_tests / check_workspace_binaries (push) Blocked by required conditions
run_tests / build_visual_tests_binary (push) Blocked by required conditions
run_tests / check_wasm (push) Blocked by required conditions
run_tests / check_dependencies (push) Blocked by required conditions
run_tests / check_docs (push) Blocked by required conditions
run_tests / check_licenses (push) Blocked by required conditions
run_tests / check_scripts (push) Blocked by required conditions
run_tests / check_postgres_and_protobuf_migrations (push) Blocked by required conditions
run_tests / extension_tests (push) Blocked by required conditions
run_tests / tests_pass (push) Blocked by required conditions
Improve explicit compaction for Anthropic models (#62139)
Explicit compaction (`LanguageModel::compact`) was previously
implemented only for OpenAI-routed cloud models, which use a dedicated
compact operation proxied through `/completions/compact`. Anthropic
models only compacted automatically when a request's `compact_at_tokens`
trigger was crossed, leaving consumers without a provider-backed way to
request compaction immediately.

Anthropic has no compact-on-demand operation, but the `compact_20260112`
context-management edit provides the necessary pieces: the lowest
trigger the API accepts, 50,000 input tokens, combined with
`pause_after_compaction`, which stops the response after the compaction
block. Explicit compaction requests remove tools so the internal
summarizer must produce replacement context while retaining Anthropic's
default summarization prompt. The resulting readable summary, optional
opaque provider state, and usage are collected consistently.

This implements explicit compaction for both direct and hosted Anthropic
models. Hosted requests use the normal `/completions` endpoint; gateway
support for forwarding the compaction fields landed in
zed-industries/cloud#3216. Models expose the 50,000-token minimum so
callers can disable explicit compaction below the provider's floor.
Calls made below that floor still fail if the stream produces no
finalized compaction context.

The existing OpenAI compact operation is moved into a provider-specific
helper without changing its request, endpoint, response handling, or
provider-state ownership.

Testing:

- `cargo nextest run -p anthropic -p language_models_cloud -p
language_models`
- `./script/clippy -p anthropic -p language_model -p
language_models_cloud -p language_models`
- `cargo fmt --all --check`
- `git diff --check`

Release Notes:

- N/A

---------

Co-authored-by: Anant Goel <anant@zed.dev>
2026-08-17 04:27:38 +00:00
..
src Improve explicit compaction for Anthropic models (#62139) 2026-08-17 04:27:38 +00:00
Cargo.toml agent: Share compatible Chat Completions infrastructure (#62652) 2026-08-14 22:42:27 +00:00
LICENSE-GPL