feat(llm): introduce lightweight circuit breaker to prevent rate-limit bans and resource exhaustion (#2095)

This commit is contained in:
Jin 2026-04-12 17:48:40 +08:00 committed by GitHub
parent 979a461af5
commit 4d4ddb3d3f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 356 additions and 2 deletions

View file

@ -883,3 +883,20 @@ checkpointer:
# use: my_package:MyGuardrailProvider
# config:
# key: value
# ============================================================================
# Circuit Breaker Configuration
# ============================================================================
# Circuit breaker for LLM calls prevents repeated requests to a failing provider.
# When the failure threshold is reached, subsequent calls fast-fail until recovery.
#
# This is useful for:
# - Avoiding rate-limit bans during provider outages
# - Reducing resource exhaustion from retry loops
# - Gracefully degrading when LLM services are unavailable
# circuit_breaker:
# # Number of consecutive failures before opening the circuit (default: 5)
# failure_threshold: 5
# # Time in seconds before attempting to recover (default: 60)
# recovery_timeout_sec: 60