Add Cohere provider (#986)
Some checks are pending
CI / Ban suppressions and legacy annotations (push) Waiting to run
CI / pytest (push) Waiting to run
CI / ruff-check (push) Waiting to run
CI / ruff-format (push) Waiting to run
CI / ty (push) Waiting to run

## Problem

FCC does not expose Cohere's OpenAI-compatible chat models, so users
with Cohere keys cannot route Claude, Codex, or messaging prompts
through Cohere.

## Changes

| Before | After |
| --- | --- |
| Provider catalog did not include Cohere. | Provider catalog includes
Cohere with `COHERE_API_KEY`, `COHERE_PROXY`, admin status, and smoke
model wiring. |
| Requests could not target Cohere's compatibility API. |
`providers/cohere` routes OpenAI-chat requests to Cohere's compatibility
API with Cohere-specific request policy. |
| Docs and templates omitted Cohere setup. | README, `.env.example`, and
architecture docs document Cohere setup and ownership. |
| Cohere behavior had no regression coverage. | Provider, runtime,
admin, config, smoke, and catalog tests cover Cohere integration. |

<!-- greptile_comment -->

<details open><summary><h3>Greptile Summary</h3></summary>

This PR adds Cohere as a new OpenAI-compatible chat provider. The main
changes are:

- Cohere provider metadata in the catalog, settings, Admin UI manifest,
and runtime factory.
- A new `CohereProvider` using the shared OpenAI chat transport with
Cohere-specific request shaping.
- Cohere API key, proxy, smoke model, README, architecture, and
environment template updates.
- Tests for Admin config, settings, provider catalog order, smoke
config, runtime creation, and Cohere request/stream behavior.
- Version and lockfile updates for the new provider feature.
</details>

<h3>Confidence Score: 5/5</h3>

Safe to merge with minimal risk.

No functional, security, or contract issues were identified. Cohere is
consistently wired through settings, catalog metadata, factory creation,
Admin config, smoke defaults, docs, versioning, and targeted tests. The
implemented Cohere `reasoning_effort` values match the Compatibility API
behavior checked during review.

No files require special attention.

<details><summary><h3><a href="https://www.greptile.com/trex"><img
alt="T-Rex"
src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg"
height="20" align="absmiddle"></a> T-Rex Logs</h3></summary>

**What T-Rex did**
- Validated the provider runtime handling of Cohere requests, including
the request body policy, streaming parsing, and default base URL and API
key behavior.
- Verified that the runtime descriptor wiring and provider config
proxy/key behavior pass in the general contract validation.
- Confirmed the admin/config smoke contract artifact shows the Cohere
environment settings, admin config masking, feature/provider catalog
contracts, and smoke configuration passing.
- Compared the initial -01-before.log and the clean -02-after.log
captures to confirm the same scoped commands are present and both exit
with code 0.

<a
href="https://app.greptile.com/trex/runs/13309850/artifacts"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifactsDark.svg?v=4"><source
media="(prefers-color-scheme: light)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=4"><img
alt="View all artifacts"
src="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=4"></picture></a>

<sub><a href="https://www.greptile.com/trex"><img alt="T-Rex"
src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg"
height="14" align="absmiddle"></a> Ran code and verified through
T-Rex</sub>
</details>

<details open><summary><h3>Important Files Changed</h3></summary>

| Filename | Overview |
|----------|----------|
| providers/cohere/client.py | Implements Cohere request shaping over
shared OpenAI chat transport, including allowed extra body and reasoning
mapping; no issues found. |
| config/provider_catalog.py | Registers Cohere with credential, proxy,
base URL, transport, and capability metadata; no issues found. |
| providers/runtime/factory.py | Wires Cohere into runtime provider
factory dispatch; no issues found. |
| config/settings.py | Adds Cohere API key and proxy settings aliases;
no issues found. |
| api/admin_config/provider_manifest.py | Adds Cohere API key
labeling/description through catalog-derived Admin fields; no issues
found. |
| smoke/lib/config.py | Adds Cohere default smoke model and credential
detection; no issues found. |
| tests/providers/test_cohere.py | Adds request-policy and streaming
adapter tests for the Cohere provider; no issues found. |
| tests/providers/test_provider_runtime.py | Adds Cohere descriptor,
config build, and factory instantiation coverage; no issues found. |
| README.md | Adds Cohere setup instructions and updates provider
counts/order; no issues found. |
| pyproject.toml | Bumps package version for the new provider feature;
no issues found. |
| uv.lock | Updates the lockfile package version to match
`pyproject.toml`; no issues found. |

</details>

<details open><summary><h3>Sequence Diagram</h3></summary>

<a href="#gh-light-mode-only">

```mermaid
%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant User as User/Admin config
participant Catalog as Provider catalog/settings
participant Factory as Runtime factory
participant Cohere as CohereProvider
participant Transport as OpenAI chat transport
participant API as Cohere Compatibility API

User->>Catalog: "Configure MODEL=cohere/... and COHERE_API_KEY"
Catalog->>Factory: Build ProviderConfig for provider_id cohere
Factory->>Cohere: Instantiate CohereProvider(config)
Cohere->>Transport: Build chat body with Cohere policy
Transport->>API: "POST /chat/completions stream=true"
API-->>Transport: Streaming OpenAI-compatible chunks
Transport-->>User: Anthropic SSE events
```

</a>
<a href="#gh-dark-mode-only">

```mermaid
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant User as User/Admin config
participant Catalog as Provider catalog/settings
participant Factory as Runtime factory
participant Cohere as CohereProvider
participant Transport as OpenAI chat transport
participant API as Cohere Compatibility API

User->>Catalog: "Configure MODEL=cohere/... and COHERE_API_KEY"
Catalog->>Factory: Build ProviderConfig for provider_id cohere
Factory->>Cohere: Instantiate CohereProvider(config)
Cohere->>Transport: Build chat body with Cohere policy
Transport->>API: "POST /chat/completions stream=true"
API-->>Transport: Streaming OpenAI-compatible chunks
Transport-->>User: Anthropic SSE events
```

</a>
</details>

<sub>Reviews (1): Last reviewed commit: ["Add Cohere
provider"](7956802968)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=41887611)</sub>

<!-- /greptile_comment -->
This commit is contained in:
Ali Khokhar 2026-07-05 01:01:19 -07:00 committed by GitHub
parent d4683bf3f6
commit 9a17d1ed0a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 574 additions and 22 deletions

View file

@ -42,6 +42,10 @@ AI_GATEWAY_API_KEY=""
HUGGINGFACE_API_KEY=""
# Cohere Config (OpenAI-compatible Chat Completions at api.cohere.ai/compatibility/v1)
COHERE_API_KEY=""
# Z.ai Config (Anthropic-compatible Messages at api.z.ai/api/anthropic/v1)
ZAI_API_KEY=""
@ -81,7 +85,7 @@ OLLAMA_BASE_URL="http://localhost:11434"
# All Claude model requests are mapped to these models, plain model is fallback
# Format: provider_type/model/name
# Valid providers: "nvidia_nim" | "open_router" | "gemini" | "deepseek" | "mistral" | "mistral_codestral" | "opencode" | "opencode_go" | "vercel" | "huggingface" | "wafer" | "kimi" | "minimax" | "cerebras" | "groq" | "fireworks" | "cloudflare" | "zai" | "lmstudio" | "llamacpp" | "ollama"
# Valid providers: "nvidia_nim" | "open_router" | "gemini" | "deepseek" | "mistral" | "mistral_codestral" | "opencode" | "opencode_go" | "vercel" | "huggingface" | "cohere" | "wafer" | "kimi" | "minimax" | "cerebras" | "groq" | "fireworks" | "cloudflare" | "zai" | "lmstudio" | "llamacpp" | "ollama"
MODEL_OPUS=
MODEL_SONNET=
MODEL_HAIKU=
@ -105,6 +109,7 @@ FCC_SMOKE_MODEL_OPENCODE=
FCC_SMOKE_MODEL_OPENCODE_GO=
FCC_SMOKE_MODEL_VERCEL=
FCC_SMOKE_MODEL_HUGGINGFACE=
FCC_SMOKE_MODEL_COHERE=
FCC_SMOKE_MODEL_ZAI=
FCC_SMOKE_MODEL_FIREWORKS=
FCC_SMOKE_MODEL_CLOUDFLARE=
@ -141,6 +146,7 @@ OPENCODE_PROXY=""
OPENCODE_GO_PROXY=""
VERCEL_AI_GATEWAY_PROXY=""
HUGGINGFACE_PROXY=""
COHERE_PROXY=""
ZAI_PROXY=""
FIREWORKS_PROXY=""
CLOUDFLARE_PROXY=""

View file

@ -371,10 +371,11 @@ where supported, and returning Anthropic SSE strings to the service layer.
Provider-specific inputs that do not apply to other upstreams, such as
Cloudflare's account ID, stay in that provider's factory/client instead of being
added to shared `ProviderConfig`.
Gateway providers such as Vercel AI Gateway and Hugging Face stay thin when
their documented OpenAI-compatible Chat Completions behavior matches shared
transport policy; provider-specific gateway options pass through request
`extra_body`.
Gateway providers such as Vercel AI Gateway, Hugging Face, and Cohere stay thin
when their documented OpenAI-compatible Chat Completions behavior matches shared
transport policy. Provider-specific gateway quirks, such as Cohere's supported
`reasoning_effort` values and unsupported compatibility fields, stay in that
provider package.
### Adding A Provider

View file

@ -57,7 +57,7 @@ Free Claude Code routes Anthropic Messages API traffic from Claude Code (CLI and
- Drop-in proxy for Claude Code's Anthropic API calls (`/v1/messages`, `/v1/models`).
- Drop-in proxy for Codex via the OpenAI Responses API (`/v1/responses`).
- `fcc-claude` and `fcc-codex` launchers that read the current Admin UI port and auth token each time they start.
- 21 provider backends: NVIDIA NIM, OpenRouter, Google AI Studio (Gemini), DeepSeek, Mistral La Plateforme, Mistral Codestral, OpenCode Zen, OpenCode Go, Vercel AI Gateway, Hugging Face Inference Providers, Wafer, Kimi, MiniMax, Cerebras Inference, Groq, Fireworks AI, Cloudflare, Z.ai, LM Studio, llama.cpp, and Ollama.
- 22 provider backends: NVIDIA NIM, OpenRouter, Google AI Studio (Gemini), DeepSeek, Mistral La Plateforme, Mistral Codestral, OpenCode Zen, OpenCode Go, Vercel AI Gateway, Hugging Face Inference Providers, Cohere, Wafer, Kimi, MiniMax, Cerebras Inference, Groq, Fireworks AI, Cloudflare, Z.ai, LM Studio, llama.cpp, and Ollama.
- Per-model routing for Claude Code: send Opus, Sonnet, Haiku, and fallback traffic to different providers.
- Native Claude Code `/model` picker support through the proxy's `/v1/models` endpoint (see [Model Picker](#model-picker)).
- Native Codex `/model` picker support when launched through `fcc-codex`, using a generated local model catalog.
@ -275,7 +275,17 @@ If your existing repo `.env` or `~/.fcc/.env` uses the old voice setting `HF_TOK
Browse models at [Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers/).
### 11. [Wafer](https://wafer.ai/)
### 11. [Cohere](https://cohere.com/)
Create a Cohere API key at [dashboard.cohere.com/api-keys](https://dashboard.cohere.com/api-keys), then paste it into `COHERE_API_KEY` in the Admin UI.
Set `MODEL` to a Cohere model slug such as `cohere/command-a-plus-05-2026`.
Cohere routes through its OpenAI-compatible Compatibility API at `https://api.cohere.ai/compatibility/v1`. FCC keeps Cohere-specific request shaping in the Cohere provider: unsupported compatibility fields are removed, Cohere-compatible structured-output knobs are accepted through `extra_body`, and Claude thinking maps to Cohere's documented `reasoning_effort` values.
Browse models at [Cohere models](https://docs.cohere.com/docs/models).
### 12. [Wafer](https://wafer.ai/)
Get a key from [wafer.ai](https://wafer.ai). In the Admin UI, paste it into `WAFER_API_KEY`, then set `MODEL` to a Wafer Pass model such as `wafer/DeepSeek-V4-Pro`.
@ -288,7 +298,7 @@ Popular examples:
This provider uses Wafer's Anthropic-compatible endpoint at `https://pass.wafer.ai/v1/messages`.
### 12. [Kimi](https://platform.moonshot.ai/)
### 13. [Kimi](https://platform.moonshot.ai/)
Get a key at [platform.moonshot.ai/console/api-keys](https://platform.moonshot.ai/console/api-keys).
@ -298,7 +308,7 @@ This provider calls Kimi's **Anthropic-compatible** Messages API (`https://api.m
Browse models at [platform.moonshot.ai](https://platform.moonshot.ai).
### 13. [MiniMax](https://platform.minimax.io/)
### 14. [MiniMax](https://platform.minimax.io/)
Get a key from [MiniMax](https://platform.minimax.io/user-center/basic-information/interface-key).
@ -306,7 +316,7 @@ In the Admin UI, paste it into `MINIMAX_API_KEY`, then set `MODEL` to a MiniMax
This provider calls MiniMax's **Anthropic-compatible** Messages API (`https://api.minimax.io/anthropic/v1/messages`). `MiniMax-M3` is the recommended default because MiniMax documents controllable Anthropic thinking for that model; other MiniMax models remain discoverable through the provider model list.
### 14. [Cerebras Inference](https://inference-docs.cerebras.ai/quickstart)
### 15. [Cerebras Inference](https://inference-docs.cerebras.ai/quickstart)
Sign up and create an API key in the [Cerebras Cloud Console](https://cloud.cerebras.ai) (see [Quickstart](https://inference-docs.cerebras.ai/quickstart)).
@ -314,7 +324,7 @@ In the Admin UI, set `CEREBRAS_API_KEY`, then route with `MODEL` such as `cerebr
Cerebras exposes an OpenAI-compatible API at `https://api.cerebras.ai/v1` ([OpenAI compatibility](https://inference-docs.cerebras.ai/resources/openai)). Non-standard request fields should go in `extra_body` when using the OpenAI client; see the same page. For reasoning models and parameters, see [Reasoning](https://inference-docs.cerebras.ai/capabilities/reasoning). This proxy follows other OpenAI-compat adapters for thinking via `reasoning_content` when Claude-style thinking is enabled.
### 15. [Groq](https://console.groq.com/)
### 16. [Groq](https://console.groq.com/)
Get an API key at [console.groq.com/keys](https://console.groq.com/keys).
@ -326,7 +336,7 @@ Reasoning-heavy models expose extra knobs documented under [Groq reasoning](http
Browse models at [console.groq.com/docs/models](https://console.groq.com/docs/models).
### 16. [Fireworks AI](https://fireworks.ai/)
### 17. [Fireworks AI](https://fireworks.ai/)
Get an API key at [fireworks.ai/account/api-keys](https://fireworks.ai/account/api-keys).
@ -336,7 +346,7 @@ Fireworks exposes an **Anthropic-compatible** Messages API at `https://api.firew
Browse models at [fireworks.ai/models](https://fireworks.ai/models).
### 17. [Cloudflare](https://developers.cloudflare.com/workers-ai/)
### 18. [Cloudflare](https://developers.cloudflare.com/workers-ai/)
Create a Cloudflare API token and copy your account ID from the Cloudflare dashboard.
@ -344,7 +354,7 @@ In the Admin UI, set `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`, then se
This provider calls Cloudflare's account-scoped **OpenAI-compatible** Chat Completions API at `https://api.cloudflare.com/client/v4/accounts/<account_id>/ai/v1/chat/completions`. Use literal Workers AI model IDs, including the `@cf/` prefix when the catalog model includes it.
### 18. [Z.ai](https://z.ai/)
### 19. [Z.ai](https://z.ai/)
Get an API key at [Z.ai/manage-apikey/apikey-list](https://z.ai/manage-apikey/apikey-list).
@ -359,13 +369,13 @@ Popular examples:
Browse models at [Z.ai](https://z.ai).
### 19. [LM Studio](https://lmstudio.ai/)
### 20. [LM Studio](https://lmstudio.ai/)
Start LM Studio's local server and load a model. In the Admin UI, keep or update `LM_STUDIO_BASE_URL`, then set `MODEL` to the model identifier shown by LM Studio, prefixed with `lmstudio/`.
Prefer models with tool-use support for Claude Code workflows.
### 20. [llama.cpp](https://github.com/ggml-org/llama.cpp)
### 21. [llama.cpp](https://github.com/ggml-org/llama.cpp)
Start `llama-server` with an Anthropic-compatible `/v1/messages` endpoint and enough context for Claude Code requests.
@ -373,7 +383,7 @@ In the Admin UI, keep or update `LLAMACPP_BASE_URL`, then set `MODEL` to the loc
For local coding models, context size matters. If llama.cpp returns HTTP 400 for normal Claude Code requests, increase `--ctx-size` and verify the model/server build supports the requested features.
### 21. [Ollama](https://ollama.com/)
### 22. [Ollama](https://ollama.com/)
Run Ollama and pull a model:
@ -386,7 +396,7 @@ In the Admin UI, keep or update `OLLAMA_BASE_URL`, then set `MODEL` to the same
`OLLAMA_BASE_URL` is the Ollama server root; do not append `/v1`. Example model slugs include `ollama/llama3.1` and `ollama/llama3.1:8b`.
### 22. Mix Providers By Model Tier
### 23. Mix Providers By Model Tier
Each model tier can use a different provider by setting `MODEL_OPUS`, `MODEL_SONNET`, and `MODEL_HAIKU` in the Admin UI. Leave a tier blank to inherit `MODEL`. These tier overrides apply to Claude model names that contain `opus`, `sonnet`, or `haiku`. Codex uses the Admin `MODEL` default through `fcc-codex` unless a session requests a provider-prefixed slug directly.
@ -600,7 +610,7 @@ Important pieces:
- Responses requests convert to Anthropic Messages internally, then share the same model router, normalizer, and provider adapters.
- `fcc-codex` registers a custom `fcc` provider that points Codex at the local proxy's `/v1/responses` endpoint.
- Model routing resolves Claude model names to `MODEL_OPUS`, `MODEL_SONNET`, `MODEL_HAIKU`, or `MODEL`.
- NIM, Gemini, DeepSeek, Mistral, Codestral, OpenCode Zen, OpenCode Go, Vercel AI Gateway, Hugging Face, Cerebras, Groq, and Cloudflare use OpenAI chat streaming translated into Anthropic SSE.
- NIM, Gemini, DeepSeek, Mistral, Codestral, OpenCode Zen, OpenCode Go, Vercel AI Gateway, Hugging Face, Cohere, Cerebras, Groq, and Cloudflare use OpenAI chat streaming translated into Anthropic SSE.
- Wafer, OpenRouter, Kimi, MiniMax, Fireworks AI, Z.ai, LM Studio, llama.cpp, and Ollama use Anthropic Messages style transports where applicable (with provider-specific quirks and model-list URLs).
- The proxy normalizes thinking blocks, tool calls, token usage metadata, and provider errors into the shape each client expects.
- Request optimizations answer trivial Claude Code probes locally to save latency and quota.

View file

@ -568,6 +568,12 @@ _NON_PROVIDER_FIELDS: tuple[ConfigFieldSpec, ...] = (
"smoke",
advanced=True,
),
ConfigFieldSpec(
"FCC_SMOKE_MODEL_COHERE",
"Smoke Cohere Model",
"smoke",
advanced=True,
),
ConfigFieldSpec(
"FCC_SMOKE_MODEL_ZAI",
"Smoke Z.ai Model",

View file

@ -44,6 +44,10 @@ _PROVIDER_FIELD_OVERRIDES: dict[str, dict[str, Any]] = {
"for local Whisper model downloads when voice notes need gated models."
),
},
"COHERE_API_KEY": {
"label": "Cohere API Key",
"description": "Cohere API key for the OpenAI-compatible Compatibility API.",
},
"ZAI_API_KEY": {
"label": "Z.ai API Key",
"description": "Z.ai Coding Plan API key.",

View file

@ -31,6 +31,7 @@ OPENCODE_DEFAULT_BASE = "https://opencode.ai/zen/v1"
OPENCODE_GO_DEFAULT_BASE = "https://opencode.ai/zen/go/v1"
VERCEL_AI_GATEWAY_DEFAULT_BASE = "https://ai-gateway.vercel.sh/v1"
HUGGINGFACE_DEFAULT_BASE = "https://router.huggingface.co/v1"
COHERE_DEFAULT_BASE = "https://api.cohere.ai/compatibility/v1"
# Z.ai Anthropic-compatible Messages API (not OpenAI Coding Plan chat completions).
ZAI_DEFAULT_BASE = "https://api.z.ai/api/anthropic/v1"
# Google AI Studio Gemini API OpenAI-compat layer (not Vertex AI).
@ -166,6 +167,17 @@ PROVIDER_CATALOG: dict[str, ProviderDescriptor] = {
proxy_attr="huggingface_proxy",
capabilities=("chat", "streaming", "tools", "thinking", "rate_limit"),
),
"cohere": ProviderDescriptor(
provider_id="cohere",
display_name="Cohere",
transport_type="openai_chat",
credential_env="COHERE_API_KEY",
credential_url="https://dashboard.cohere.com/api-keys",
credential_attr="cohere_api_key",
default_base_url=COHERE_DEFAULT_BASE,
proxy_attr="cohere_proxy",
capabilities=("chat", "streaming", "tools", "thinking", "rate_limit"),
),
"wafer": ProviderDescriptor(
provider_id="wafer",
display_name="Wafer",
@ -326,7 +338,7 @@ PROVIDER_CATALOG: dict[str, ProviderDescriptor] = {
# Key order:
# NVIDIA NIM first (README default), DeepSeek fourth, OpenCode gateways adjacent,
# Vercel / Hugging Face follow gateway-style remotes, then native Anthropic
# Vercel / Hugging Face / Cohere follow gateway-style remotes, then native Anthropic
# remotes and locals per project plan (github.com/cheahjs/free-llm-api-resources
# Free Providers TOC as rough guide beyond fixed slots).
# ``SUPPORTED_PROVIDER_IDS`` inherits this insertion order for UI and error-message listing.

View file

@ -52,6 +52,9 @@ class Settings(BaseSettings):
# ==================== Hugging Face Inference Providers ====================
huggingface_api_key: str = Field(default="", validation_alias="HUGGINGFACE_API_KEY")
# ==================== Cohere Compatibility API ====================
cohere_api_key: str = Field(default="", validation_alias="COHERE_API_KEY")
# ==================== Z.ai Config ====================
zai_api_key: str = Field(default="", validation_alias="ZAI_API_KEY")
@ -135,6 +138,7 @@ class Settings(BaseSettings):
default="", validation_alias="VERCEL_AI_GATEWAY_PROXY"
)
huggingface_proxy: str = Field(default="", validation_alias="HUGGINGFACE_PROXY")
cohere_proxy: str = Field(default="", validation_alias="COHERE_PROXY")
zai_proxy: str = Field(default="", validation_alias="ZAI_PROXY")
fireworks_proxy: str = Field(default="", validation_alias="FIREWORKS_PROXY")
cloudflare_proxy: str = Field(default="", validation_alias="CLOUDFLARE_PROXY")

View file

@ -0,0 +1,7 @@
"""Cohere Compatibility API OpenAI-compatible adapter."""
from providers.defaults import COHERE_DEFAULT_BASE
from .client import CohereProvider
__all__ = ["COHERE_DEFAULT_BASE", "CohereProvider"]

View file

@ -0,0 +1,88 @@
"""Cohere provider implementation (OpenAI-compatible chat completions)."""
from collections.abc import Mapping
from copy import deepcopy
from typing import Any
from providers.base import ProviderConfig
from providers.defaults import COHERE_DEFAULT_BASE
from providers.exceptions import InvalidRequestError
from providers.transports.openai_chat import (
OpenAIChatRequestPolicy,
OpenAIChatTransport,
build_openai_chat_request_body,
)
_ALLOWED_EXTRA_BODY_KEYS = frozenset(
{
"frequency_penalty",
"presence_penalty",
"response_format",
"seed",
}
)
_REQUEST_POLICY = OpenAIChatRequestPolicy(
provider_name="COHERE",
strip_message_names=True,
unsupported_body_keys=frozenset(
{
"audio",
"logit_bias",
"metadata",
"modalities",
"n",
"parallel_tool_calls",
"prediction",
"service_tier",
"store",
"top_logprobs",
}
),
)
class CohereProvider(OpenAIChatTransport):
"""Cohere Compatibility API at ``https://api.cohere.ai/compatibility/v1``."""
def __init__(self, config: ProviderConfig):
super().__init__(
config,
provider_name="COHERE",
base_url=config.base_url or COHERE_DEFAULT_BASE,
api_key=config.api_key,
)
def _build_request_body(
self, request: Any, thinking_enabled: bool | None = None
) -> dict:
return build_openai_chat_request_body(
request,
thinking_enabled=self._is_thinking_enabled(request, thinking_enabled),
policy=_REQUEST_POLICY,
postprocessors=(_apply_cohere_request_quirks,),
)
def _apply_cohere_request_quirks(
body: dict[str, Any], request_data: Any, thinking_enabled: bool
) -> None:
_merge_allowed_extra_body(body, getattr(request_data, "extra_body", None))
body["reasoning_effort"] = "high" if thinking_enabled else "none"
def _merge_allowed_extra_body(body: dict[str, Any], extra_body: Any) -> None:
if extra_body in (None, {}):
return
if not isinstance(extra_body, Mapping):
raise InvalidRequestError("Cohere extra_body must be an object when provided.")
unsupported = sorted(
str(key) for key in extra_body if key not in _ALLOWED_EXTRA_BODY_KEYS
)
if unsupported:
raise InvalidRequestError(
"Cohere extra_body supports only these keys: "
f"{sorted(_ALLOWED_EXTRA_BODY_KEYS)}. Unsupported: {unsupported}"
)
body.update({str(key): deepcopy(value) for key, value in extra_body.items()})

View file

@ -4,6 +4,7 @@ from config.provider_catalog import (
CEREBRAS_DEFAULT_BASE,
CLOUDFLARE_AI_REST_ROOT,
CODESTRAL_DEFAULT_BASE,
COHERE_DEFAULT_BASE,
DEEPSEEK_DEFAULT_BASE,
GEMINI_DEFAULT_BASE,
GROQ_DEFAULT_BASE,
@ -27,6 +28,7 @@ __all__ = (
"CEREBRAS_DEFAULT_BASE",
"CLOUDFLARE_AI_REST_ROOT",
"CODESTRAL_DEFAULT_BASE",
"COHERE_DEFAULT_BASE",
"DEEPSEEK_DEFAULT_BASE",
"GEMINI_DEFAULT_BASE",
"GROQ_DEFAULT_BASE",

View file

@ -107,6 +107,12 @@ def _create_huggingface(config: ProviderConfig, _settings: Settings) -> BaseProv
return HuggingFaceProvider(config)
def _create_cohere(config: ProviderConfig, _settings: Settings) -> BaseProvider:
from providers.cohere import CohereProvider
return CohereProvider(config)
def _create_zai(config: ProviderConfig, _settings: Settings) -> BaseProvider:
from providers.zai import ZaiProvider
@ -154,6 +160,7 @@ PROVIDER_FACTORIES: dict[str, ProviderFactory] = {
"opencode_go": _create_opencode_go,
"vercel": _create_vercel,
"huggingface": _create_huggingface,
"cohere": _create_cohere,
"wafer": _create_wafer,
"kimi": _create_kimi,
"minimax": _create_minimax,

View file

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "free-claude-code"
version = "3.0.0"
version = "3.1.0"
description = "Middleware between Claude Code CLI (Anthropic API) and NVIDIA NIM"
readme = "README.md"
requires-python = ">=3.14.0"

View file

@ -55,6 +55,7 @@ PROVIDER_SMOKE_DEFAULT_MODELS: dict[str, str] = {
"opencode_go": "opencode_go/minimax-m2.7",
"vercel": "vercel/openai/gpt-5.5",
"huggingface": "huggingface/openai/gpt-oss-120b:fastest",
"cohere": "cohere/command-a-plus-05-2026",
"zai": "zai/glm-5.1",
"gemini": "gemini/models/gemini-3.1-flash-lite",
"groq": "groq/llama-3.3-70b-versatile",
@ -258,6 +259,8 @@ class SmokeConfig:
return bool(self.settings.vercel_ai_gateway_api_key.strip())
if provider == "huggingface":
return bool(self.settings.huggingface_api_key.strip())
if provider == "cohere":
return bool(self.settings.cohere_api_key.strip())
if provider == "zai":
return bool(self.settings.zai_api_key.strip())
if provider == "gemini":

View file

@ -340,6 +340,31 @@ def test_admin_apply_writes_huggingface_key_and_masks_preview(monkeypatch, tmp_p
assert "HUGGINGFACE_API_KEY=hf-secret" in text
def test_admin_apply_writes_cohere_key_and_masks_preview(monkeypatch, tmp_path):
_set_home(monkeypatch, tmp_path)
_clear_process_config(monkeypatch)
app = create_app(lifespan_enabled=False)
response = _local_client(app).post(
"/admin/api/config/apply",
json={
"values": {
"MODEL": "cohere/command-a-plus-05-2026",
"COHERE_API_KEY": "cohere-secret",
}
},
)
assert response.status_code == 200
body = response.json()
assert body["applied"] is True
assert "COHERE_API_KEY=********" in body["env_preview"]
env_file = tmp_path / ".fcc" / ".env"
text = env_file.read_text(encoding="utf-8")
assert "MODEL=cohere/command-a-plus-05-2026" in text
assert "COHERE_API_KEY=cohere-secret" in text
def test_admin_apply_preserves_hidden_diagnostics_and_smoke_values(
monkeypatch, tmp_path
):

View file

@ -327,6 +327,16 @@ class TestSettings:
assert settings.huggingface_proxy == "http://proxy.test:8080"
assert not hasattr(settings, "hf_token")
def test_cohere_settings_from_env(self, monkeypatch):
"""Cohere key and proxy env vars load into settings."""
from config.settings import Settings
monkeypatch.setenv("COHERE_API_KEY", "cohere-key")
monkeypatch.setenv("COHERE_PROXY", "http://proxy.test:8080")
settings = Settings()
assert settings.cohere_api_key == "cohere-key"
assert settings.cohere_proxy == "http://proxy.test:8080"
def test_legacy_hf_token_env_is_ignored(self, monkeypatch):
"""HF_TOKEN is migrated by startup config migration, not read by Settings."""
from config.settings import Settings
@ -919,6 +929,7 @@ class TestPerModelMapping:
parse_provider_type("huggingface/openai/gpt-oss-120b:fastest")
== "huggingface"
)
assert parse_provider_type("cohere/command-a-plus-05-2026") == "cohere"
assert parse_provider_type("gemini/models/gemini-3.1-flash-lite") == "gemini"
assert parse_provider_type("groq/llama-3.3-70b-versatile") == "groq"
assert parse_provider_type("cerebras/llama3.1-8b") == "cerebras"
@ -948,6 +959,9 @@ class TestPerModelMapping:
parse_model_name("huggingface/openai/gpt-oss-120b:fastest")
== "openai/gpt-oss-120b:fastest"
)
assert parse_model_name("cohere/command-a-plus-05-2026") == (
"command-a-plus-05-2026"
)
assert (
parse_model_name("gemini/models/gemini-3.1-flash-lite")
== "models/gemini-3.1-flash-lite"

View file

@ -6,6 +6,7 @@ from providers.base import BaseProvider
from providers.cerebras import CerebrasProvider
from providers.cloudflare import CloudflareProvider
from providers.codestral import CodestralProvider
from providers.cohere import CohereProvider
from providers.deepseek import DeepSeekProvider
from providers.fireworks import FireworksProvider
from providers.gemini import GeminiProvider
@ -96,6 +97,7 @@ def test_provider_and_platform_registries_include_advertised_builtins() -> None:
"opencode_go": OpenCodeProvider,
"vercel": VercelProvider,
"huggingface": HuggingFaceProvider,
"cohere": CohereProvider,
"zai": ZaiProvider,
"gemini": GeminiProvider,
"groq": GroqProvider,

View file

@ -13,6 +13,7 @@ _EXPECTED_PROVIDER_ORDER: tuple[str, ...] = (
"opencode_go",
"vercel",
"huggingface",
"cohere",
"wafer",
"kimi",
"minimax",

View file

@ -37,6 +37,7 @@ def _settings(**overrides):
"opencode_api_key": "",
"vercel_ai_gateway_api_key": "",
"huggingface_api_key": "",
"cohere_api_key": "",
"zai_api_key": "",
"gemini_api_key": "",
"groq_api_key": "",
@ -226,6 +227,22 @@ def test_huggingface_provider_configuration_uses_api_key(monkeypatch) -> None:
assert models[0].full_model == PROVIDER_SMOKE_DEFAULT_MODELS["huggingface"]
def test_cohere_provider_configuration_uses_api_key(monkeypatch) -> None:
monkeypatch.delenv("FCC_SMOKE_MODEL_COHERE", raising=False)
config = _smoke_config(
settings=_settings(
model="ollama/llama3.1",
ollama_base_url="",
cohere_api_key="cohere-key",
)
)
assert config.has_provider_configuration("cohere")
models = config.provider_smoke_models()
assert models[0].provider == "cohere"
assert models[0].full_model == PROVIDER_SMOKE_DEFAULT_MODELS["cohere"]
def test_provider_smoke_model_override_accepts_model_name_without_prefix(
monkeypatch,
) -> None:

View file

@ -0,0 +1,314 @@
"""Tests for Cohere Compatibility API provider."""
from contextlib import asynccontextmanager
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from providers.base import ProviderConfig
from providers.cohere import COHERE_DEFAULT_BASE, CohereProvider
from providers.exceptions import InvalidRequestError
class MockMessage:
def __init__(self, role, content):
self.role = role
self.content = content
class MockRequest:
def __init__(self, **kwargs):
self.model = "command-a-plus-05-2026"
self.messages = [MockMessage("user", "Hello")]
self.max_tokens = 100
self.temperature = 0.5
self.top_p = 0.9
self.system = "System prompt"
self.stop_sequences = None
self.tools = []
self.thinking = MagicMock()
self.thinking.enabled = True
for key, value in kwargs.items():
setattr(self, key, value)
@pytest.fixture
def cohere_config():
return ProviderConfig(
api_key="test_cohere_key",
base_url=COHERE_DEFAULT_BASE,
rate_limit=10,
rate_window=60,
enable_thinking=True,
)
@pytest.fixture(autouse=True)
def mock_rate_limiter():
@asynccontextmanager
async def _slot():
yield
with patch("providers.transports.openai_chat.transport.GlobalRateLimiter") as mock:
instance = mock.get_scoped_instance.return_value
async def _passthrough(fn, *args, **kwargs):
return await fn(*args, **kwargs)
instance.execute_with_retry = AsyncMock(side_effect=_passthrough)
instance.concurrency_slot.side_effect = _slot
yield instance
@pytest.fixture
def cohere_provider(cohere_config):
return CohereProvider(cohere_config)
def test_default_base_url_constant():
assert COHERE_DEFAULT_BASE == "https://api.cohere.ai/compatibility/v1"
def test_init_uses_default_base_url_and_api_key(cohere_config):
with patch("providers.transports.openai_chat.transport.AsyncOpenAI") as mock_openai:
provider = CohereProvider(cohere_config)
assert provider._api_key == "test_cohere_key"
assert provider._base_url == COHERE_DEFAULT_BASE
mock_openai.assert_called_once()
def test_init_strips_trailing_slash(cohere_config):
config = cohere_config.model_copy(update={"base_url": f"{COHERE_DEFAULT_BASE}/"})
with patch("providers.transports.openai_chat.transport.AsyncOpenAI"):
provider = CohereProvider(config)
assert provider._base_url == COHERE_DEFAULT_BASE
def test_build_request_body_sanitizes_documented_unsupported_fields(cohere_provider):
with patch(
"providers.transports.openai_chat.request_policy.build_base_request_body"
) as mock_convert:
mock_convert.return_value = {
"model": "command-a-plus-05-2026",
"messages": [{"role": "user", "name": "alice", "content": "hi"}],
"max_tokens": 42,
"store": True,
"metadata": {"trace": "abc"},
"logit_bias": {"1": -100},
"top_logprobs": 2,
"n": 4,
"modalities": ["text"],
"prediction": {"type": "content", "content": "x"},
"audio": {"voice": "alloy"},
"service_tier": "auto",
"parallel_tool_calls": True,
}
body = cohere_provider._build_request_body(MockRequest())
assert body["messages"][0].get("name") is None
assert body["max_tokens"] == 42
assert "max_completion_tokens" not in body
for key in (
"audio",
"logit_bias",
"metadata",
"modalities",
"n",
"parallel_tool_calls",
"prediction",
"service_tier",
"store",
"top_logprobs",
):
assert key not in body
def test_build_request_body_maps_thinking_enabled_to_reasoning_high(cohere_provider):
body = cohere_provider._build_request_body(MockRequest())
assert body["reasoning_effort"] == "high"
def test_build_request_body_preserves_replayed_reasoning_content(cohere_provider):
with patch(
"providers.transports.openai_chat.request_policy.build_base_request_body"
) as mock_convert:
mock_convert.return_value = {
"model": "command-a-plus-05-2026",
"messages": [
{
"role": "assistant",
"content": "answer",
"reasoning_content": "hidden chain",
}
],
}
body = cohere_provider._build_request_body(MockRequest())
assert body["messages"] == [
{
"role": "assistant",
"content": "answer",
"reasoning_content": "hidden chain",
}
]
assert body["reasoning_effort"] == "high"
def test_build_request_body_maps_thinking_disabled_to_reasoning_none():
provider = CohereProvider(
ProviderConfig(
api_key="test_cohere_key",
base_url=COHERE_DEFAULT_BASE,
rate_limit=10,
rate_window=60,
enable_thinking=False,
)
)
body = provider._build_request_body(MockRequest())
assert body["reasoning_effort"] == "none"
def test_build_request_body_promotes_allowed_extra_body(cohere_provider):
req = MockRequest(
extra_body={
"frequency_penalty": 0.1,
"presence_penalty": 0.2,
"response_format": {"type": "json_object"},
"seed": 123,
}
)
body = cohere_provider._build_request_body(req)
assert body["frequency_penalty"] == 0.1
assert body["presence_penalty"] == 0.2
assert body["response_format"] == {"type": "json_object"}
assert body["seed"] == 123
assert "extra_body" not in body
def test_build_request_body_rejects_unsupported_extra_body(cohere_provider):
req = MockRequest(extra_body={"documents": [{"text": "x"}]})
with pytest.raises(InvalidRequestError, match="Unsupported"):
cohere_provider._build_request_body(req)
@pytest.mark.asyncio
async def test_stream_response_text(cohere_provider):
mock_chunk = MagicMock()
mock_chunk.choices = [
MagicMock(
delta=MagicMock(
content="Hello from Cohere",
reasoning_content=None,
tool_calls=None,
),
finish_reason="stop",
)
]
mock_chunk.usage = MagicMock(completion_tokens=5, prompt_tokens=10)
async def mock_stream():
yield mock_chunk
with patch.object(
cohere_provider._client.chat.completions, "create", new_callable=AsyncMock
) as mock_create:
mock_create.return_value = mock_stream()
events = [
event async for event in cohere_provider.stream_response(MockRequest())
]
assert any(
'"text_delta"' in event and "Hello from Cohere" in event for event in events
)
@pytest.mark.asyncio
async def test_stream_response_tool_call(cohere_provider):
mock_tc = MagicMock()
mock_tc.index = 0
mock_tc.id = "call_1"
mock_tc.function = MagicMock()
mock_tc.function.name = "Read"
mock_tc.function.arguments = '{"file_path":"a.py"}'
mock_chunk = MagicMock()
mock_chunk.choices = [
MagicMock(
delta=MagicMock(content=None, reasoning_content=None, tool_calls=[mock_tc]),
finish_reason="tool_calls",
)
]
mock_chunk.usage = MagicMock(completion_tokens=5, prompt_tokens=10)
async def mock_stream():
yield mock_chunk
with patch.object(
cohere_provider._client.chat.completions, "create", new_callable=AsyncMock
) as mock_create:
mock_create.return_value = mock_stream()
events = [
event async for event in cohere_provider.stream_response(MockRequest())
]
assert any(
'"content_block_start"' in event and '"tool_use"' in event for event in events
)
assert any(
'"input_json_delta"' in event and "file_path" in event for event in events
)
@pytest.mark.asyncio
async def test_stream_response_reasoning_content(cohere_provider):
mock_chunk = MagicMock()
mock_chunk.choices = [
MagicMock(
delta=MagicMock(
content=None,
reasoning_content="Thinking via Cohere",
tool_calls=None,
),
finish_reason="stop",
)
]
mock_chunk.usage = MagicMock(completion_tokens=2, prompt_tokens=10)
async def mock_stream():
yield mock_chunk
with patch.object(
cohere_provider._client.chat.completions, "create", new_callable=AsyncMock
) as mock_create:
mock_create.return_value = mock_stream()
events = [
event async for event in cohere_provider.stream_response(MockRequest())
]
assert any(
'"thinking_delta"' in event and "Thinking via Cohere" in event
for event in events
)
@pytest.mark.asyncio
async def test_cleanup(cohere_provider):
cohere_provider._client = AsyncMock()
await cohere_provider.cleanup()
cohere_provider._client.close.assert_called_once()

View file

@ -6,6 +6,7 @@ import pytest
from config.nim import NimSettings
from config.provider_catalog import (
COHERE_DEFAULT_BASE,
MINIMAX_DEFAULT_BASE,
PROVIDER_CATALOG,
ZAI_DEFAULT_BASE,
@ -14,6 +15,7 @@ from config.provider_ids import SUPPORTED_PROVIDER_IDS
from providers.cerebras import CerebrasProvider
from providers.cloudflare import CloudflareProvider
from providers.codestral import CodestralProvider
from providers.cohere import CohereProvider
from providers.deepseek import DeepSeekProvider
from providers.exceptions import UnknownProviderTypeError
from providers.fireworks import FireworksProvider
@ -51,6 +53,7 @@ def _make_settings(**overrides):
mock.opencode_api_key = "test_opencode_key"
mock.vercel_ai_gateway_api_key = "test_vercel_key"
mock.huggingface_api_key = "test_huggingface_key"
mock.cohere_api_key = "test_cohere_key"
mock.zai_api_key = "test_zai_key"
mock.lm_studio_base_url = "http://localhost:1234/v1"
mock.llamacpp_base_url = "http://localhost:8080/v1"
@ -69,6 +72,7 @@ def _make_settings(**overrides):
mock.opencode_go_proxy = ""
mock.vercel_ai_gateway_proxy = ""
mock.huggingface_proxy = ""
mock.cohere_proxy = ""
mock.zai_proxy = ""
mock.fireworks_proxy = ""
mock.fireworks_api_key = "test_fireworks_key"
@ -226,6 +230,16 @@ def test_huggingface_descriptor_uses_openai_chat_router() -> None:
assert "thinking" in descriptor.capabilities
def test_cohere_descriptor_uses_openai_chat_compatibility_api() -> None:
descriptor = PROVIDER_CATALOG["cohere"]
assert descriptor.transport_type == "openai_chat"
assert descriptor.default_base_url == COHERE_DEFAULT_BASE
assert descriptor.credential_env == "COHERE_API_KEY"
assert descriptor.proxy_attr == "cohere_proxy"
assert "thinking" in descriptor.capabilities
def test_build_provider_config_vercel_uses_gateway_key_and_proxy() -> None:
descriptor = PROVIDER_CATALOG["vercel"]
settings = _make_settings(
@ -252,6 +266,19 @@ def test_build_provider_config_huggingface_uses_api_key_and_proxy() -> None:
assert config.proxy == "http://proxy.test:8080"
def test_build_provider_config_cohere_uses_api_key_and_proxy() -> None:
descriptor = PROVIDER_CATALOG["cohere"]
settings = _make_settings(
cohere_api_key="cohere-token",
cohere_proxy="http://proxy.test:8080",
)
config = build_provider_config(descriptor, settings)
assert config.api_key == "cohere-token"
assert config.proxy == "http://proxy.test:8080"
def test_create_provider_uses_native_openrouter_by_default():
with patch("httpx.AsyncClient"):
provider = create_provider("open_router", _make_settings())
@ -269,6 +296,7 @@ def test_create_provider_instantiates_each_builtin():
cloudflare_account_id="test_cloudflare_account",
vercel_ai_gateway_api_key="test_vercel_key",
huggingface_api_key="test_huggingface_key",
cohere_api_key="test_cohere_key",
kimi_api_key="test_kimi_key",
)
cases = {
@ -288,6 +316,7 @@ def test_create_provider_instantiates_each_builtin():
"opencode_go": OpenCodeProvider,
"vercel": VercelProvider,
"huggingface": HuggingFaceProvider,
"cohere": CohereProvider,
"zai": ZaiProvider,
"gemini": GeminiProvider,
"groq": GroqProvider,

2
uv.lock generated
View file

@ -561,7 +561,7 @@ wheels = [
[[package]]
name = "free-claude-code"
version = "3.0.0"
version = "3.1.0"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },