vrr/agent-zero
2
0
Fork 0
mirror of https://github.com/agent0ai/agent-zero.git synced 2026-07-09 17:08:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
agent-zero/helpers/litellm_transport.py.dox.md
Alessandro 038f88848d Fallback when Responses mock parses SSE 
Detect LiteLLM JSONDecodeError failures caused by the Responses mock streaming iterator trying to parse a real SSE stream.

Fall back to Chat Completions before any output is emitted, keeping custom OpenAI-compatible proxy prefixes provider-neutral.

Add a regression test for SSE-shaped JSON decode failures and update the transport DOX contract.

Verified with: PYTHONPATH="/home/eclypso/a0/agent-zero" conda run -n a0 pytest tests/test_stream_tool_early_stop.py tests/test_responses_architecture.py -q; PYTHONPATH="/home/eclypso/a0/agent-zero" conda run -n a0 python -m py_compile helpers/litellm_transport.py; docker exec -i -w /a0 9c436228a4a2 /opt/venv-a0/bin/python -m py_compile helpers/litellm_transport.py; git diff --check.
2026-07-02 17:29:40 +02:00

2.9 KiB
Raw Permalink Blame History

litellm_transport.py DOX

Purpose

  • Own Agent Zero's LiteLLM transport adapter for Chat Completions and Responses API calls.
  • Normalize Agent Zero model-call kwargs into provider-safe LiteLLM requests.
  • Preserve canonical response metadata for history, provider-state continuation, and fallback decisions.

Ownership

  • litellm_transport.py owns the runtime implementation.
  • litellm_transport.py.dox.md owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
  • Classes:
  • TransportMode
  • TransportRecovery
  • TransportPolicy
  • LiteLLMTransport
  • ChatCompletionsTransport
  • ResponsesTransport
  • ResponsesEventParser
  • Top-level functions include transport cache reset, request normalization, parsing, prompt-cache preparation, and response/error classifiers.

Runtime Contracts

  • Keep provider selection and provider-specific defaults outside this helper; callers pass a resolved LiteLLM model name and kwargs.
  • Strip Agent Zero internal kwargs before sending requests to LiteLLM.
  • Do not send orphan tool controls when no tools are present; strict OpenAI-compatible servers can reject empty tools arrays.
  • Normalize function tool parameter schemas with an explicit object properties field before Responses requests so OpenAI-compatible chat backends reached through LiteLLM can validate them.
  • Prefer Responses API when configured, but fallback to Chat Completions when the provider does not support Responses.
  • Fall back to Chat Completions when a Responses request is rejected before any output by an endpoint-specific or shape-specific Bad Request indicating the provider cannot parse Responses payloads.
  • Fall back to Chat Completions when a Responses endpoint fails before output with an endpoint-specific server error, proxy path-unavailable error, or LiteLLM proxy-extra import error.
  • Fall back to Chat Completions when LiteLLM's Responses mock streaming path tries to JSON-decode a real SSE stream before any output.
  • Preserve Chat Completions tool calls from both non-streaming responses and streaming deltas as canonical LLMResult function-call items.
  • Preserve provider-state metadata when Responses API calls succeed, and fall back to local replay when provider state is unsupported.
  • Keep prompt-cache markers only for providers that accept them.

Work Guidance

  • Add provider-agnostic request cleanup here when multiple OpenAI-compatible providers can benefit.
  • Treat fallback behavior as a shared transport contract, not a provider registry.
  • Keep tool conversion symmetric between Chat Completions and Responses requests.

Verification

  • Run pytest tests/test_stream_tool_early_stop.py tests/test_responses_architecture.py -q after changing transport normalization or fallback behavior.
  • Run local-provider smoke checks when changing OpenAI-compatible request cleanup.

Child DOX Index

No child DOX files.