fix(llm): enable Venice cache routing and custom provider overrides
Some checks are pending
Build And Publish Docker Images / plan (push) Waiting to run
Build And Publish Docker Images / build (push) Blocked by required conditions

Venice and Agent Zero API missed automatic cache hints because the transport
only recognized a fixed set of provider routes. Default both to stable
prompt_cache_key routing and expose a0_prompt_cache_mode through the existing
Additional parameters field for custom providers.

Support cache keys, cache_control markers, automatic detection, and disabling
automatic hints. Preserve explicit caller keys, keep marked requests on Chat
Completions, and strip the internal control before either provider API call.
Document the configuration and add regression and installed-SDK HTTP coverage.

Validation: 316 targeted tests passed in the live Docker framework runtime;
git diff --check passed. Full suite and upstream cache-hit measurements were
not run.
This commit is contained in:
Alessandro 2026-10-13 15:48:32 +02:00
parent c946b3c347
commit f72945e70c
9 changed files with 191 additions and 11 deletions

View file

@ -33,6 +33,7 @@
- Show `Use separate Vision Model` immediately below `Supports Vision` while Main vision is enabled, not inside Advanced Settings; describe the disabled state as using Main's native vision.
- In model overviews, render the effective Vision Model as a text-only `Vision override / Provider / Model` child aligned with Main's provider column, not as an icon-bearing peer row.
- Changing a model provider in the settings UI must clear `api_base` and `kwargs` because both may be provider-specific.
- Chat, Vision, and Utility Additional parameters expose `a0_prompt_cache_mode` overrides through existing kwargs; keep cache-mode help out of embedding fields. The shared transport validates and strips this control.
- Repair provider-specific model-config aliases at the model-config read/build boundary; keep provider-specific repairs out of provider-agnostic core wrappers such as `models.py`.
- The `Agent.read_prompt/end` Anthropic hook adapts only the two bundled thoughts instructions in main/communication/solving prompt results when the effective main provider or model name contains `anthropic`, or the model name contains `claude` (case-insensitive, regardless of provider). Keep prompt files, JSON fields, history, custom wording, and other models unchanged; standalone communication reads must match main-prompt rendering for native protocol projection. Verify with `tests/test_anthropic_thoughts_prompt.py`.
- `modelConfig.createPresetEditor()` owns local preset drafts, row actions, and stable UI-only row keys so deletion or renaming cannot rebind nested model fields.

View file

@ -36,6 +36,12 @@ The normal plugin resolution order selects the most specific available reference
- The preset editor exposes the shared API key for each selected provider and saves key changes separately from secret-free preset definitions.
- Project Settings selects from the same global preset definitions.
## Cache Routing
Venice and Agent Zero API send a stable `prompt_cache_key` derived from the model, leading instructions, and tool definitions. This is a routing hint; cache availability and retention remain provider-controlled. Venice documents it in its [Chat Completions API](https://docs.venice.ai/api-reference/endpoint/chat/completions).
In a chat model's **Advanced Settings → Additional parameters**, set `a0_prompt_cache_mode=prompt_cache_key` or `a0_prompt_cache_mode=cache_control` to match a custom provider's supported format. `auto` keeps the existing provider detection; `off` disables automatic hints. Explicit keys, including `extra_body` keys, remain unchanged. Marker mode keeps marked requests on Chat Completions and uses the existing stable message/tool boundaries.
## Initial Presets And Migration
The startup migration converts the prior global full model configuration into `Default`, preserves existing global presets, promotes distinct scoped full configurations and legacy project presets into uniquely named global presets, and rewrites scoped config files to selection-only JSON. Original files receive a `.pre-unified-presets.bak` backup before replacement; the migration is idempotent.

View file

@ -325,6 +325,7 @@
<div class="field-title">Additional parameters</div>
<div class="field-description">
Any other parameters supported by <a href='https://docs.litellm.ai/docs/set_keys' target='_blank'>LiteLLM</a>. Format is KEY=VALUE on individual lines. Values can be JSON objects, arrays, numbers, lowercase true/false/null, or text. Quote strings to keep values such as "true" as text.
<div x-show="modelType !== 'embedding'">For custom providers, set <code>a0_prompt_cache_mode=prompt_cache_key</code> or <code>a0_prompt_cache_mode=cache_control</code> to match the provider's cache support. The default is <code>auto</code>; <code>off</code> disables automatic cache hints.</div>
</div>
</div>
<div class="field-control">