vrr/zed
2
0
Fork 0
mirror of https://github.com/zed-industries/zed.git synced 2026-10-19 02:54:11 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

bedrock: Add GPT-6 Astra and Claude Fable 5.1 (#64201)

Browse source 
# Objective

Add GPT-6 Astra and Claude Fable 5.1 as built-in Amazon Bedrock models.

## Solution

- Use `bedrock-runtime` Converse for both models. Astra's Mantle
endpoint offers Responses and Chat Completions; Converse lets it reuse
Zed's Bedrock client. Fable 5.1 uses the same endpoint and API as Fable
5.
- Add the documented model IDs, geo/global profiles, token limits, and
capabilities. Astra has a 1,050,000-token context window; Fable 5.1 has
1,000,000. Both allow 128,000 output tokens.
- Handle Astra's reasoning fields, encrypted replay, and tool-result
images. Apply Fable 5.1's adaptive thinking, binding controls, and
tool-choice restrictions.
- Limit inference routing to each model's supported source Regions, keep
model-specific reasoning out of other Bedrock models' histories, group
parallel tool results before their images, and report Fable and Astra
refusals.

## Testing

- `cargo test -p bedrock -p language_models --lib`: 33 Bedrock and 116
language-model tests passed. The provider suite was rerun after the
final refusal-handling change.
- `./script/clippy -p bedrock -p language_models`, `cargo fmt -p bedrock
-p language_models -- --check`, and `cd docs && npx --yes prettier@3.5.0
--check src/`: passed.
- Live `bedrock-runtime` Converse/ConverseStream in `us-east-1`: Astra
and Fable 5.1 responded successfully. Astra's streamed encrypted
reasoning replayed on a second turn. Astra accepted forced tool use;
Fable 5.1 rejected it as expected.
- A live Astra request with two parallel tool results followed by two
sibling images succeeded. `aws bedrock list-inference-profiles`
confirmed the commercial US and EU profiles tested.
- GovCloud profile routing was not tested: no GovCloud credentials were
available. Zed's GUI was not run.

## References

- [GPT-6 Astra Bedrock model
card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html)
- [Claude Fable 5.1 Bedrock model
card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-fable-5-1.html)

Release Notes:

- Added GPT-6 Astra and Claude Fable 5.1 to the Amazon Bedrock provider.

---------

Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
This commit is contained in:
Ibrahim Khan 2026-10-09 13:43:27 +00:00 • committed by GitHub
parent e778eec483
commit 089abd6917
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 848 additions and 95 deletions
Download patch file Download diff file Expand all files Collapse all files

81
crates/bedrock/src/bedrock.rs
View file

@ -189,6 +189,11 @@ pub enum Thinking {
},
Adaptive {
effort: BedrockAdaptiveThinkingEffort,
#[serde(default, skip_serializing_if = "Option::is_none")]
binding_controls_beta: Option<String>,
},
Reasoning {
effort: BedrockAdaptiveThinkingEffort,
},
/// Explicitly turns thinking off. Required by Claude Opus 5, where
/// adaptive thinking runs by default when the `thinking` field is
@ -199,7 +204,7 @@ pub enum Thinking {
}
/// Converts the request's thinking configuration into the
/// `additionalModelRequestFields` entries understood by Anthropic models on
/// `additionalModelRequestFields` entries understood by the model on
/// the Converse API.
fn thinking_request_fields(thinking: &Thinking) -> HashMap<String, Document> {
let mut fields = HashMap::new();
@ -221,17 +226,31 @@ fn thinking_request_fields(thinking: &Thinking) -> HashMap<String, Document> {
Thinking::Enabled {
budget_tokens: None,
} => {}
Thinking::Adaptive { effort } => {
fields.insert(
"thinking".to_string(),
Document::from(HashMap::from([
("type".to_string(), Document::String("adaptive".to_string())),
(
"display".to_string(),
Document::String("summarized".to_string()),
),
])),
);
Thinking::Adaptive {
effort,
binding_controls_beta,
} => {
let mut thinking_fields = HashMap::from([
("type".to_string(), Document::String("adaptive".to_string())),
(
"display".to_string(),
Document::String("summarized".to_string()),
),
]);
if let Some(beta_header) = binding_controls_beta {
// Fable 5.1 binds replayed thinking to the prefix, which Zed can change.
thinking_fields.insert(
"block_binding".to_string(),
value_to_aws_document(&serde_json::json!({
"prefix_mismatch_behavior": "drop_block"
})),
);
fields.insert(
"anthropic_beta".to_string(),
Document::Array(vec![Document::String(beta_header.clone())]),
);
}
fields.insert("thinking".to_string(), Document::Object(thinking_fields));
fields.insert(
"output_config".to_string(),
Document::from(HashMap::from([(
@ -240,6 +259,12 @@ fn thinking_request_fields(thinking: &Thinking) -> HashMap<String, Document> {
)])),
);
}
Thinking::Reasoning { effort } => {
fields.insert(
"reasoning".to_string(),
value_to_aws_document(&serde_json::json!({"effort": effort.as_str()})),
);
}
Thinking::Disabled => {
// On Claude Opus 5 omitting the `thinking` field means adaptive
// thinking runs by default, so turning it off requires this
@ -317,6 +342,7 @@ mod tests {
fn test_adaptive_thinking_serializes_effort_in_output_config() {
let fields = thinking_request_fields(&Thinking::Adaptive {
effort: BedrockAdaptiveThinkingEffort::XHigh,
binding_controls_beta: None,
});
let thinking = fields.get("thinking").expect("thinking field");
@ -359,4 +385,35 @@ mod tests {
});
assert!(fields.is_empty());
}
#[test]
fn test_gpt_6_astra_uses_openai_reasoning_fields() {
let fields = thinking_request_fields(&Thinking::Reasoning {
effort: BedrockAdaptiveThinkingEffort::High,
});
assert_eq!(
aws_document_to_value(&Document::Object(fields)),
serde_json::json!({"reasoning": {"effort": "high"}})
);
}
#[test]
fn test_adaptive_thinking_binding_controls() {
let fields = thinking_request_fields(&Thinking::Adaptive {
effort: BedrockAdaptiveThinkingEffort::High,
binding_controls_beta: Some("thinking-binding-controls-2026-08-01".into()),
});
assert_eq!(
aws_document_to_value(&Document::Object(fields)),
serde_json::json!({
"thinking": {
"type": "adaptive",
"display": "summarized",
"block_binding": {"prefix_mismatch_behavior": "drop_block"}
},
"output_config": {"effort": "high"},
"anthropic_beta": ["thinking-binding-controls-2026-08-01"]
})
);
}
}

228
crates/bedrock/src/models.rs
View file

@ -35,6 +35,9 @@ pub enum BedrockModelMode {
AdaptiveThinking {
effort: BedrockAdaptiveThinkingEffort,
},
Reasoning {
effort: BedrockAdaptiveThinkingEffort,
},
}
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
@ -59,6 +62,13 @@ pub struct BedrockThinkingConfig {
#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq, EnumIter)]
pub enum ConverseModel {
// Anthropic Claude 4+ models
#[serde(
rename = "claude-fable-5-1",
alias = "claude-fable-5-1-latest",
alias = "claude-fable-5-1-thinking",
alias = "claude-fable-5-1-thinking-latest"
)]
ClaudeFable5_1,
#[serde(
rename = "claude-fable-5",
alias = "claude-fable-5-latest",
@ -192,7 +202,9 @@ pub enum ConverseModel {
#[serde(rename = "nova-2-lite")]
Nova2Lite,
// OpenAI GPT OSS models
// OpenAI models
#[serde(rename = "gpt-6-astra")]
Gpt6Astra,
#[serde(rename = "gpt-oss-20b")]
GptOss20B,
#[serde(rename = "gpt-oss-120b")]
@ -257,7 +269,11 @@ impl ConverseModel {
}
pub fn from_id(id: &str) -> anyhow::Result<Self> {
if id.starts_with("claude-fable-5") {
if id == "gpt-6-astra" {
Ok(Self::Gpt6Astra)
} else if id.starts_with("claude-fable-5-1") {
Ok(Self::ClaudeFable5_1)
} else if id.starts_with("claude-fable-5") {
Ok(Self::ClaudeFable5)
} else if id.starts_with("claude-opus-5") {
Ok(Self::ClaudeOpus5)
@ -288,6 +304,7 @@ impl ConverseModel {
pub fn id(&self) -> &str {
match self {
Self::ClaudeFable5_1 => "claude-fable-5-1",
Self::ClaudeFable5 => "claude-fable-5",
Self::ClaudeOpus5 => "claude-opus-5",
Self::ClaudeOpus4_8 => "claude-opus-4-8",
@ -321,6 +338,7 @@ impl ConverseModel {
Self::NovaPro => "nova-pro",
Self::NovaPremier => "nova-premier",
Self::Nova2Lite => "nova-2-lite",
Self::Gpt6Astra => "gpt-6-astra",
Self::GptOss20B => "gpt-oss-20b",
Self::GptOss120B => "gpt-oss-120b",
Self::NemotronSuper3_120B => "nemotron-super-3-120b",
@ -342,6 +360,7 @@ impl ConverseModel {
pub fn request_id(&self) -> &str {
match self {
Self::ClaudeFable5_1 => "anthropic.claude-fable-5-1",
Self::ClaudeFable5 => "anthropic.claude-fable-5",
Self::ClaudeOpus5 => "anthropic.claude-opus-5",
Self::ClaudeOpus4_8 => "anthropic.claude-opus-4-8",
@ -375,6 +394,7 @@ impl ConverseModel {
Self::NovaPro => "amazon.nova-pro-v1:0",
Self::NovaPremier => "amazon.nova-premier-v1:0",
Self::Nova2Lite => "amazon.nova-2-lite-v1:0",
Self::Gpt6Astra => "openai.gpt-6-astra",
Self::GptOss20B => "openai.gpt-oss-20b-1:0",
Self::GptOss120B => "openai.gpt-oss-120b-1:0",
Self::NemotronSuper3_120B => "nvidia.nemotron-super-3-120b",
@ -396,6 +416,7 @@ impl ConverseModel {
pub fn display_name(&self) -> &str {
match self {
Self::ClaudeFable5_1 => "Claude Fable 5.1",
Self::ClaudeFable5 => "Claude Fable 5",
Self::ClaudeOpus5 => "Claude Opus 5",
Self::ClaudeOpus4_8 => "Claude Opus 4.8",
@ -429,6 +450,7 @@ impl ConverseModel {
Self::NovaPro => "Amazon Nova Pro",
Self::NovaPremier => "Amazon Nova Premier",
Self::Nova2Lite => "Amazon Nova 2 Lite",
Self::Gpt6Astra => "GPT-6 Astra",
Self::GptOss20B => "GPT OSS 20B",
Self::GptOss120B => "GPT OSS 120B",
Self::NemotronSuper3_120B => "Nemotron Super 3 120B",
@ -452,7 +474,8 @@ impl ConverseModel {
pub fn max_token_count(&self) -> u64 {
match self {
Self::ClaudeFable5
Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeOpus4_7
@ -478,6 +501,7 @@ impl ConverseModel {
Self::NovaLite | Self::NovaPro => 300_000,
Self::NovaPremier => 1_000_000,
Self::Nova2Lite => 300_000,
Self::Gpt6Astra => 1_050_000,
Self::GptOss20B | Self::GptOss120B => 128_000,
Self::NemotronSuper3_120B | Self::NemotronNano3_30B => 262_000,
Self::MiniMaxM2 | Self::MiniMaxM2_1 | Self::MiniMaxM2_5 => 196_000,
@ -490,7 +514,8 @@ impl ConverseModel {
pub fn max_output_tokens(&self) -> u64 {
match self {
Self::ClaudeFable5
Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeOpus4_7
@ -519,6 +544,7 @@ impl ConverseModel {
| Self::Qwen3CoderNext
| Self::Qwen3Coder480B => 8_192,
Self::NovaLite | Self::NovaPro | Self::NovaPremier | Self::Nova2Lite => 5_000,
Self::Gpt6Astra => 128_000,
Self::GptOss20B | Self::GptOss120B => 16_000,
Self::NemotronSuper3_120B | Self::NemotronNano3_30B => 131_000,
Self::MiniMaxM2 | Self::MiniMaxM2_1 | Self::MiniMaxM2_5 => 98_000,
@ -543,7 +569,8 @@ impl ConverseModel {
pub fn supports_tool_use(&self) -> bool {
match self {
Self::ClaudeFable5
Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeOpus4_7
@ -572,6 +599,7 @@ impl ConverseModel {
Self::GLM5 | Self::GLM4_7 | Self::GLM4_7Flash => true,
Self::KimiK2Thinking | Self::KimiK2_5 => true,
Self::DeepSeekR1 | Self::DeepSeekV3_1 | Self::DeepSeekV3_2 => true,
Self::Gpt6Astra => true,
Self::Custom {
supports_tool_use, ..
} => supports_tool_use.unwrap_or(false),
@ -581,7 +609,8 @@ impl ConverseModel {
pub fn supports_images(&self) -> bool {
match self {
Self::ClaudeFable5
Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeOpus4_7
@ -597,6 +626,7 @@ impl ConverseModel {
Self::PixtralLarge => true,
Self::Qwen3VL235B => true,
Self::KimiK2_5 => true,
Self::Gpt6Astra => true,
Self::Custom {
supports_images, ..
} => supports_images.unwrap_or(false),
@ -606,7 +636,8 @@ impl ConverseModel {
pub fn supports_caching(&self) -> bool {
match self {
Self::ClaudeFable5
Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeOpus4_7
@ -628,7 +659,9 @@ impl ConverseModel {
pub fn supports_thinking(&self) -> bool {
match self {
Self::ClaudeFable5
Self::Gpt6Astra
| Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeOpus4_7
@ -647,7 +680,8 @@ impl ConverseModel {
pub fn supports_adaptive_thinking(&self) -> bool {
match self {
Self::ClaudeFable5
Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeOpus4_7
@ -663,9 +697,11 @@ impl ConverseModel {
pub fn supports_xhigh_adaptive_thinking(&self) -> bool {
match self {
Self::ClaudeFable5 | Self::ClaudeOpus5 | Self::ClaudeOpus4_8 | Self::ClaudeSonnet5 => {
true
}
Self::ClaudeFable5_1
| Self::ClaudeFable5
| Self::ClaudeOpus5
| Self::ClaudeOpus4_8
| Self::ClaudeSonnet5 => true,
Self::Custom { thinking, .. } => {
thinking.as_ref().is_some_and(|thinking| thinking.has_xhigh)
}
@ -690,7 +726,11 @@ impl ConverseModel {
};
}
if self.supports_adaptive_thinking() {
if matches!(self, Self::Gpt6Astra) {
BedrockModelMode::Reasoning {
effort: BedrockAdaptiveThinkingEffort::Medium,
}
} else if self.supports_adaptive_thinking() {
BedrockModelMode::AdaptiveThinking {
effort: BedrockAdaptiveThinkingEffort::default(),
}
@ -714,6 +754,68 @@ impl ConverseModel {
return Ok(model_id.into());
}
if matches!(self, Self::Gpt6Astra | Self::ClaudeFable5_1) {
if matches!(self, Self::ClaudeFable5_1)
&& matches!(region, "us-gov-east-1" | "us-gov-west-1")
{
return Ok(format!("us-gov.{model_id}"));
}
let in_us_profile = matches!(
region,
"us-east-1"
| "us-east-2"
| "us-west-1"
| "us-west-2"
| "ca-central-1"
| "ca-west-1"
);
let supports_global = in_us_profile
|| matches!(
region,
"eu-central-1"
| "eu-north-1"
| "eu-west-1"
| "eu-west-2"
| "eu-west-3"
| "ap-northeast-1"
| "ap-northeast-2"
| "ap-northeast-3"
| "ap-south-1"
| "ap-southeast-1"
| "ap-southeast-2"
| "sa-east-1"
)
|| matches!(self, Self::ClaudeFable5_1)
&& matches!(
region,
"eu-central-2"
| "eu-south-1"
| "eu-south-2"
| "ap-east-2"
| "ap-south-2"
| "ap-southeast-3"
| "ap-southeast-4"
| "ap-southeast-5"
| "ap-southeast-6"
| "ap-southeast-7"
| "il-central-1"
| "me-central-1"
| "me-south-1"
| "af-south-1"
| "mx-central-1"
);
if !supports_global {
anyhow::bail!("Unsupported Region {region}");
}
let prefix = if in_us_profile && !allow_global {
"us"
} else {
"global"
};
return Ok(format!("{prefix}.{model_id}"));
}
let supports_global = matches!(
self,
Self::ClaudeFable5
@ -1060,6 +1162,106 @@ impl MantleModel {
#[cfg(test)]
mod tests {
use super::*;
use strum::IntoEnumIterator;
#[test]
fn test_gpt_6_astra_and_fable_5_1_metadata() -> anyhow::Result<()> {
for (model, id, request_id, display_name, context_window, supports_caching) in [
(
ConverseModel::Gpt6Astra,
"gpt-6-astra",
"openai.gpt-6-astra",
"GPT-6 Astra",
1_050_000,
false,
),
(
ConverseModel::ClaudeFable5_1,
"claude-fable-5-1",
"anthropic.claude-fable-5-1",
"Claude Fable 5.1",
1_000_000,
true,
),
] {
assert_eq!(model.id(), id);
assert_eq!(model.request_id(), request_id);
assert_eq!(model.display_name(), display_name);
assert_eq!(model.max_token_count(), context_window);
assert_eq!(model.max_output_tokens(), 128_000);
assert!(model.supports_tool_use());
assert!(model.supports_images());
assert!(model.supports_thinking());
assert_eq!(model.supports_caching(), supports_caching);
assert_eq!(ConverseModel::from_id(id)?, model);
for (region, allow_global, prefix) in [
("us-east-1", false, "us"),
("ca-central-1", false, "us"),
("eu-west-1", false, "global"),
("us-east-1", true, "global"),
] {
assert_eq!(
model.cross_region_inference_id(region, allow_global)?,
format!("{prefix}.{request_id}")
);
}
}
assert_eq!(
ConverseModel::from_id("claude-fable-5-1-thinking")?,
ConverseModel::ClaudeFable5_1
);
Ok(())
}
#[test]
fn test_new_model_inference_profiles_match_source_regions() -> anyhow::Result<()> {
assert_eq!(
ConverseModel::ClaudeFable5_1.cross_region_inference_id("us-gov-west-1", true)?,
"us-gov.anthropic.claude-fable-5-1"
);
assert_eq!(
ConverseModel::ClaudeFable5_1.cross_region_inference_id("af-south-1", false)?,
"global.anthropic.claude-fable-5-1"
);
for region in ["af-south-1", "eu-central-2", "ap-east-2", "us-gov-west-1"] {
assert!(
ConverseModel::Gpt6Astra
.cross_region_inference_id(region, true)
.is_err(),
"Astra has no profile in {region}"
);
}
Ok(())
}
#[test]
fn test_builtin_converse_models_have_unique_ids_and_sane_token_limits() {
use std::collections::HashSet;
let mut ids = HashSet::new();
let mut request_ids = HashSet::new();
for model in
ConverseModel::iter().filter(|model| !matches!(model, ConverseModel::Custom { .. }))
{
assert!(
ids.insert(model.id().to_string()),
"duplicate ConverseModel id: {}",
model.id()
);
assert!(
request_ids.insert(model.request_id().to_string()),
"duplicate ConverseModel request_id: {}",
model.request_id()
);
assert!(
model.max_output_tokens() <= model.max_token_count(),
"{} has max_output_tokens ({}) greater than max_token_count ({})",
model.id(),
model.max_output_tokens(),
model.max_token_count()
);
}
}
#[test]
fn test_builtin_mantle_models_use_responses_protocol() {

634
crates/language_models/src/provider/bedrock.rs
View file

@ -216,6 +216,9 @@ pub enum ModelMode {
AdaptiveThinking {
effort: bedrock::BedrockAdaptiveThinkingEffort,
},
Reasoning {
effort: bedrock::BedrockAdaptiveThinkingEffort,
},
}
impl From<ModelMode> for BedrockModelMode {
@ -224,6 +227,7 @@ impl From<ModelMode> for BedrockModelMode {
ModelMode::Default => BedrockModelMode::Default,
ModelMode::Thinking { budget_tokens } => BedrockModelMode::Thinking { budget_tokens },
ModelMode::AdaptiveThinking { effort } => BedrockModelMode::AdaptiveThinking { effort },
ModelMode::Reasoning { effort } => BedrockModelMode::Reasoning { effort },
}
}
}
@ -234,6 +238,7 @@ impl From<BedrockModelMode> for ModelMode {
BedrockModelMode::Default => ModelMode::Default,
BedrockModelMode::Thinking { budget_tokens } => ModelMode::Thinking { budget_tokens },
BedrockModelMode::AdaptiveThinking { effort } => ModelMode::AdaptiveThinking { effort },
BedrockModelMode::Reasoning { effort } => ModelMode::Reasoning { effort },
}
}
}
@ -981,15 +986,19 @@ async fn build_converse_client(
}
fn converse_language_model(model: &ConverseModel) -> LanguageModel {
let is_gpt_6_astra = matches!(model, ConverseModel::Gpt6Astra);
let reasoning_model = converse_reasoning_model(model.request_id());
LanguageModel {
supports_tools: model.supports_tool_use(),
supports_images: model.supports_images(),
supports_thinking: model.supports_thinking(),
// Astra and Fable 5.1 always reason, so only offer effort levels.
supports_disabling_thinking: reasoning_model.is_none(),
refusal_fallback_model_id: model
.id()
.starts_with(anthropic::FABLE_MODEL_ID_PREFIX)
.then_some(anthropic::FABLE_FALLBACK_MODEL_ID),
supported_effort_levels: if model.supports_adaptive_thinking() {
supported_effort_levels: if model.supports_adaptive_thinking() || is_gpt_6_astra {
vec![
language_model::LanguageModelEffortLevel {
name: "Low".into(),
@ -999,12 +1008,12 @@ fn converse_language_model(model: &ConverseModel) -> LanguageModel {
language_model::LanguageModelEffortLevel {
name: "Medium".into(),
value: "medium".into(),
is_default: false,
is_default: is_gpt_6_astra,
},
language_model::LanguageModelEffortLevel {
name: "High".into(),
value: "high".into(),
is_default: true,
is_default: !is_gpt_6_astra,
},
language_model::LanguageModelEffortLevel {
name: "XHigh".into(),
@ -1019,7 +1028,9 @@ fn converse_language_model(model: &ConverseModel) -> LanguageModel {
]
.into_iter()
.filter(|effort_level| {
effort_level.value != "xhigh" || model.supports_xhigh_adaptive_thinking()
effort_level.value != "xhigh"
|| model.supports_xhigh_adaptive_thinking()
|| is_gpt_6_astra
})
.collect()
} else {
@ -1028,7 +1039,10 @@ fn converse_language_model(model: &ConverseModel) -> LanguageModel {
// Add support for None - we'll filter tool calls at response
tool_choice_support: LanguageModelToolChoiceSupport {
auto: model.supports_tool_use(),
any: model.supports_tool_use(),
any: model.supports_tool_use()
&& anthropic::supports_forced_tool_use(
reasoning_model.as_ref().unwrap_or(model).id(),
),
none: model.supports_tool_use(),
},
supports_streaming_tools: true,
@ -1091,6 +1105,7 @@ impl BedrockLanguageModelProvider {
};
let request = self.stream_bedrock_request(request, cx);
let model = config.clone();
let executor = cx.background_executor().clone();
let future = request_limiter.stream(async move {
let response = request.await.map_err(|err| match err {
@ -1156,7 +1171,7 @@ impl BedrockLanguageModelProvider {
other => LanguageModelCompletionError::Other(anyhow!(other)),
})?;
let events = language_model::stream_in_background(
map_to_language_model_completion_events(response).boxed(),
map_to_language_model_completion_events(response, model).boxed(),
executor,
);
@ -2040,6 +2055,52 @@ fn deny_tool_use_events(
})
}
/// Matches by Bedrock model ID, so custom entries such as
/// `us.anthropic.claude-fable-5-1` get the same handling as the built-in model.
fn converse_reasoning_model(model_id: &str) -> Option<ConverseModel> {
[ConverseModel::Gpt6Astra, ConverseModel::ClaudeFable5_1]
.into_iter()
.find(|model| model_id.ends_with(model.request_id()))
}
fn prepare_bedrock_content(
content: Vec<BedrockInnerContent>,
tool_result_images_as_siblings: bool,
) -> Vec<BedrockInnerContent> {
let mut prepared = Vec::with_capacity(content.len());
let mut images = Vec::new();
for block in content {
match block {
BedrockInnerContent::ToolResult(mut result) if tool_result_images_as_siblings => {
// OpenAI expects parallel tool results before their sibling images.
// <https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ToolResultContentBlock.html>
let mut has_images = false;
for content in std::mem::take(&mut result.content) {
match content {
BedrockToolResultContentBlock::Image(image) => {
has_images = true;
images.push(BedrockInnerContent::Image(image))
}
other => result.content.push(other),
}
}
if has_images && result.content.is_empty() {
result.content.push(BedrockToolResultContentBlock::Text(
"See the attached image output.".into(),
));
}
prepared.push(BedrockInnerContent::ToolResult(result));
}
other => {
prepared.append(&mut images);
prepared.push(other);
}
}
}
prepared.append(&mut images);
prepared
}
pub fn into_bedrock(
request: LanguageModelRequest,
model: String,
@ -2058,6 +2119,13 @@ pub fn into_bedrock(
anyhow::bail!("Bedrock does not support custom tools");
}
let reasoning_model = converse_reasoning_model(&model);
let is_fable_5_1 = reasoning_model == Some(ConverseModel::ClaudeFable5_1);
let is_gpt_6_astra = reasoning_model == Some(ConverseModel::Gpt6Astra);
if is_fable_5_1 && request.tool_choice == Some(LanguageModelToolChoice::Any) {
anyhow::bail!("Claude Fable 5.1 does not support forced tool use");
}
let mut new_messages: Vec<BedrockMessage> = Vec::new();
let mut system_message = String::new();
@ -2070,6 +2138,18 @@ pub fn into_bedrock(
continue;
}
let reasoning_source = message
.reasoning_details
.as_ref()
.and_then(|details| details.get("bedrock_model"))
.and_then(Value::as_str);
let replay_reasoning = match reasoning_source {
Some(source) => reasoning_model
.as_ref()
.is_some_and(|model| model.request_id() == source),
None => reasoning_model.is_none(),
};
match message.role {
Role::User | Role::Assistant => {
let mut bedrock_message_content: Vec<BedrockInnerContent> = message
@ -2085,6 +2165,9 @@ pub fn into_bedrock(
}
MessageContent::Compaction(_) => None,
MessageContent::Thinking { text, signature } => {
if !replay_reasoning {
return None;
}
if model.contains(ConverseModel::DeepSeekR1.request_id()) {
// DeepSeekR1 doesn't support thinking blocks
// And the AWS API demands that you strip them
@ -2108,13 +2191,22 @@ pub fn into_bedrock(
))
}
MessageContent::RedactedThinking(blob) => {
if !replay_reasoning {
return None;
}
if model.contains(ConverseModel::DeepSeekR1.request_id()) {
// DeepSeekR1 doesn't support thinking blocks
// And the AWS API demands that you strip them
return None;
}
use base64::Engine;
let bytes = base64::engine::general_purpose::STANDARD
.decode(blob.as_bytes())
.context("Invalid base64 in Bedrock reasoning history")
.log_err()?;
let redacted =
BedrockThinkingBlock::RedactedContent(BedrockBlob::new(blob));
BedrockThinkingBlock::RedactedContent(BedrockBlob::new(bytes));
Some(BedrockInnerContent::ReasoningContent(redacted))
}
@ -2222,6 +2314,8 @@ pub fn into_bedrock(
}
})
.collect();
bedrock_message_content =
prepare_bedrock_content(bedrock_message_content, is_gpt_6_astra);
if message.cache && supports_caching && !bedrock_message_content.is_empty() {
bedrock_message_content.push(BedrockInnerContent::CachePoint(
CachePointBlock::builder()
@ -2351,41 +2445,58 @@ pub fn into_bedrock(
}
}
let thinking = if request.thinking_allowed {
match thinking_mode {
BedrockModelMode::Thinking { budget_tokens } => {
Some(bedrock::Thinking::Enabled { budget_tokens })
}
BedrockModelMode::AdaptiveThinking {
effort: default_effort,
} => {
let effort = request
.thinking_effort
.as_deref()
.and_then(|e| match e {
"low" => Some(bedrock::BedrockAdaptiveThinkingEffort::Low),
"medium" => Some(bedrock::BedrockAdaptiveThinkingEffort::Medium),
"high" => Some(bedrock::BedrockAdaptiveThinkingEffort::High),
"xhigh" => Some(bedrock::BedrockAdaptiveThinkingEffort::XHigh),
"max" => Some(bedrock::BedrockAdaptiveThinkingEffort::Max),
_ => None,
})
.unwrap_or(default_effort);
Some(bedrock::Thinking::Adaptive { effort })
}
BedrockModelMode::Default => None,
let selected_effort = |default_effort| {
// Astra and Fable 5.1 keep reasoning enabled, so suppressed requests use the default
// effort, matching the Anthropic and OpenAI providers.
// <https://developers.openai.com/api/docs/guides/latest-model?model=gpt-6-astra>
// <https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-fable-5-1.html>
if !request.thinking_allowed {
return default_effort;
}
} else if model.contains(ConverseModel::ClaudeOpus5.request_id()) {
// On Claude Opus 5, omitting the `thinking` field no longer means
// "off": the model runs adaptive thinking by default, so features
// that suppress thinking (e.g. inline assist) must opt out
// explicitly. Earlier Claude models treat omission as "off" and must
// keep omitting the field. No effort accompanies the opt-out because
// `disabled` combined with effort `xhigh`/`max` is a 400.
// <https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-opus-5.html>
Some(bedrock::Thinking::Disabled)
} else {
request
.thinking_effort
.as_deref()
.and_then(|effort| match effort {
"low" => Some(bedrock::BedrockAdaptiveThinkingEffort::Low),
"medium" => Some(bedrock::BedrockAdaptiveThinkingEffort::Medium),
"high" => Some(bedrock::BedrockAdaptiveThinkingEffort::High),
"xhigh" => Some(bedrock::BedrockAdaptiveThinkingEffort::XHigh),
"max" => Some(bedrock::BedrockAdaptiveThinkingEffort::Max),
_ => None,
})
.unwrap_or(default_effort)
};
let thinking = match thinking_mode {
BedrockModelMode::Reasoning { effort } => Some(bedrock::Thinking::Reasoning {
effort: selected_effort(effort),
}),
BedrockModelMode::AdaptiveThinking { effort }
if request.thinking_allowed || is_fable_5_1 =>
{
Some(bedrock::Thinking::Adaptive {
effort: selected_effort(effort),
binding_controls_beta: model
.rsplit_once("anthropic.")
.filter(|(_, model_id)| anthropic::binds_thinking_blocks_to_prefix(model_id))
.map(|_| anthropic::THINKING_BINDING_CONTROLS_BETA_HEADER.to_string()),
})
}
BedrockModelMode::Thinking { budget_tokens } if request.thinking_allowed => {
Some(bedrock::Thinking::Enabled { budget_tokens })
}
_ if !request.thinking_allowed
&& model.contains(ConverseModel::ClaudeOpus5.request_id()) =>
{
// Opus 5 defaults to adaptive thinking, so turning it off requires `disabled`.
// <https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-opus-5.html>
Some(bedrock::Thinking::Disabled)
}
_ => None,
};
let temperature = if is_fable_5_1 || is_gpt_6_astra {
None
} else {
request.temperature.or(default_temperature)
};
Ok(bedrock::Request {
@ -2397,7 +2508,7 @@ pub fn into_bedrock(
thinking,
metadata: None,
stop_sequences: Vec::new(),
temperature: request.temperature.or(default_temperature),
temperature,
top_k: None,
top_p: None,
guardrail_identifier,
@ -2407,6 +2518,7 @@ pub fn into_bedrock(
pub fn map_to_language_model_completion_events(
events: Pin<Box<dyn Send + Stream<Item = Result<BedrockStreamingResponse, anyhow::Error>>>>,
model: ConverseModel,
) -> impl Stream<Item = Result<LanguageModelCompletionEvent, LanguageModelCompletionError>> {
struct RawToolUse {
id: String,
@ -2417,16 +2529,29 @@ pub fn map_to_language_model_completion_events(
struct State {
events: Pin<Box<dyn Send + Stream<Item = Result<BedrockStreamingResponse, anyhow::Error>>>>,
tool_uses_by_index: HashMap<i32, RawToolUse>,
redacted_thinking_by_index: HashMap<i32, Vec<u8>>,
reasoning_model: Option<String>,
emitted_tool_use: bool,
}
let model = converse_reasoning_model(model.request_id()).unwrap_or(model);
let report_refusals = matches!(
model,
ConverseModel::ClaudeFable5_1 | ConverseModel::ClaudeFable5 | ConverseModel::Gpt6Astra
);
let preserve_redacted_thinking = matches!(
model,
ConverseModel::Gpt6Astra | ConverseModel::ClaudeFable5_1
);
let initial_state = State {
events,
tool_uses_by_index: HashMap::default(),
redacted_thinking_by_index: HashMap::default(),
reasoning_model: preserve_redacted_thinking.then(|| model.request_id().to_string()),
emitted_tool_use: false,
};
futures::stream::unfold(initial_state, |mut state| async move {
futures::stream::unfold(initial_state, move |mut state| async move {
match state.events.next().await {
Some(event_result) => match event_result {
Ok(event) => {
@ -2478,12 +2603,22 @@ pub fn map_to_language_model_completion_events(
}))
}
ReasoningContentBlockDelta::RedactedContent(redacted) => {
let content = String::from_utf8(redacted.into_inner())
.unwrap_or("REDACTED".to_string());
Some(Ok(LanguageModelCompletionEvent::Thinking {
text: content,
signature: None,
}))
if preserve_redacted_thinking {
// A reasoning block can span multiple binary deltas.
state
.redacted_thinking_by_index
.entry(cb_delta.content_block_index)
.or_default()
.extend(redacted.into_inner());
None
} else {
let content = String::from_utf8(redacted.into_inner())
.unwrap_or("REDACTED".to_string());
Some(Ok(LanguageModelCompletionEvent::Thinking {
text: content,
signature: None,
}))
}
}
_ => None,
},
@ -2502,28 +2637,46 @@ pub fn map_to_language_model_completion_events(
}
None
}
ConverseStreamOutput::MessageStart(_) => None,
ConverseStreamOutput::ContentBlockStop(cb_stop) => state
.tool_uses_by_index
.remove(&cb_stop.content_block_index)
.map(|tool_use| {
state.emitted_tool_use = true;
let input = parse_tool_arguments(&tool_use.input_json)
.unwrap_or_else(|_| Value::Object(Default::default()));
Ok(LanguageModelCompletionEvent::ToolUse(
LanguageModelToolUse {
id: tool_use.id.into(),
name: tool_use.name.into(),
is_input_complete: true,
raw_input: tool_use.input_json,
input: language_model::LanguageModelToolUseInput::Json(
input,
),
thought_signature: None,
},
ConverseStreamOutput::MessageStart(_) => {
state.reasoning_model.as_ref().map(|id| {
Ok(LanguageModelCompletionEvent::ReasoningDetails(
serde_json::json!({ "bedrock_model": id }),
))
})
}
ConverseStreamOutput::ContentBlockStop(cb_stop) => state
.redacted_thinking_by_index
.remove(&cb_stop.content_block_index)
.map(|data| {
use base64::Engine;
Ok(LanguageModelCompletionEvent::RedactedThinking {
data: base64::engine::general_purpose::STANDARD.encode(data),
})
})
.or_else(|| {
state
.tool_uses_by_index
.remove(&cb_stop.content_block_index)
.map(|tool_use| {
state.emitted_tool_use = true;
let input = parse_tool_arguments(&tool_use.input_json)
.unwrap_or_else(|_| Value::Object(Default::default()));
Ok(LanguageModelCompletionEvent::ToolUse(
LanguageModelToolUse {
id: tool_use.id.into(),
name: tool_use.name.into(),
is_input_complete: true,
raw_input: tool_use.input_json,
input:
language_model::LanguageModelToolUseInput::Json(
input,
),
thought_signature: None,
},
))
})
}),
ConverseStreamOutput::Metadata(cb_meta) => cb_meta.usage.map(|metadata| {
Ok(LanguageModelCompletionEvent::UsageUpdate(TokenUsage {
@ -2540,7 +2693,12 @@ pub fn map_to_language_model_completion_events(
}))
}),
ConverseStreamOutput::MessageStop(message_stop) => {
let stop_reason = if state.emitted_tool_use {
let stop_reason = if report_refusals
&& (message_stop.stop_reason == StopReason::ContentFiltered
|| message_stop.stop_reason.as_str() == "refusal")
{
language_model::StopReason::Refusal
} else if state.emitted_tool_use {
// Some models (e.g. Kimi) send EndTurn even when
// they've made tool calls. Trust the content over
// the stop reason.
@ -3328,6 +3486,342 @@ mod tests {
assert!(error.to_string().contains("timed out"), "{error}");
}
#[test]
fn test_gpt_6_astra_and_fable_5_1_defaults() -> Result<()> {
for (model, expected_thinking) in [
(
ConverseModel::Gpt6Astra,
serde_json::json!({"Reasoning": {"effort": "Medium"}}),
),
(
ConverseModel::ClaudeFable5_1,
serde_json::json!({"Adaptive": {
"effort": "High", "binding_controls_beta": "thinking-binding-controls-2026-08-01"
}}),
),
] {
for thinking_allowed in [true, false] {
let request = into_bedrock(
LanguageModelRequest {
thinking_allowed,
..Default::default()
},
model.cross_region_inference_id("us-east-1", false)?,
model.default_temperature(),
model.max_output_tokens(),
model.thinking_mode(),
model.supports_caching(),
model.supports_tool_use(),
None,
None,
)?;
assert_eq!(
serde_json::to_value(request.thinking)?,
expected_thinking,
"{} with thinking_allowed: {thinking_allowed}",
model.id()
);
assert_eq!(request.temperature, None);
}
assert!(!converse_language_model(&model).supports_disabling_thinking);
}
assert!(converse_language_model(&ConverseModel::ClaudeOpus4_8).supports_disabling_thinking);
Ok(())
}
#[test]
fn test_new_models_replay_only_their_own_encrypted_reasoning() -> Result<()> {
use bedrock::bedrock_client::types::{
ContentBlockDeltaEvent, ContentBlockStopEvent, MessageStartEvent,
};
let response = futures::stream::iter(vec![
Ok(ConverseStreamOutput::MessageStart(
MessageStartEvent::builder()
.role(bedrock::BedrockRole::Assistant)
.build()?,
)),
Ok(ConverseStreamOutput::ContentBlockDelta(
ContentBlockDeltaEvent::builder()
.content_block_index(0)
.delta(ContentBlockDelta::ReasoningContent(
ReasoningContentBlockDelta::RedactedContent(BedrockBlob::new([0, 255])),
))
.build()?,
)),
Ok(ConverseStreamOutput::ContentBlockStop(
ContentBlockStopEvent::builder()
.content_block_index(0)
.build()?,
)),
]);
let events = futures::executor::block_on(
map_to_language_model_completion_events(Box::pin(response), ConverseModel::Gpt6Astra)
.collect::<Vec<_>>(),
)
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
let details = events.iter().find_map(|event| match event {
LanguageModelCompletionEvent::ReasoningDetails(value) => Some(Arc::new(value.clone())),
_ => None,
});
let redacted = events.iter().find_map(|event| match event {
LanguageModelCompletionEvent::RedactedThinking { data } => Some(data.clone()),
_ => None,
});
assert!(
details.is_some(),
"the stored reasoning must identify its model"
);
let redacted = redacted.expect("binary reasoning must survive streaming");
let request_for = |model: ConverseModel, details: Option<Arc<Value>>, content| {
into_bedrock(
LanguageModelRequest {
messages: vec![LanguageModelRequestMessage {
role: Role::Assistant,
content,
cache: false,
reasoning_details: details,
}],
..Default::default()
},
model.cross_region_inference_id("us-east-1", false)?,
model.default_temperature(),
model.max_output_tokens(),
model.thinking_mode(),
model.supports_caching(),
model.supports_tool_use(),
None,
None,
)
};
let own = request_for(
ConverseModel::Gpt6Astra,
details.clone(),
vec![MessageContent::RedactedThinking(redacted.clone())],
)?;
assert!(matches!(
own.messages[0].content().first(),
Some(BedrockInnerContent::ReasoningContent(
BedrockThinkingBlock::RedactedContent(data)
)) if data.as_ref() == [0, 255]
));
for (model, source) in [
(ConverseModel::ClaudeFable5_1, details.clone()),
(ConverseModel::Gpt6Astra, None),
// The refusal fallback sends tagged history to a model without reasoning replay.
(ConverseModel::ClaudeOpus4_8, details.clone()),
] {
let request = request_for(
model,
source,
vec![
MessageContent::RedactedThinking(redacted.clone()),
MessageContent::Thinking {
text: "foreign thinking".into(),
signature: Some("foreign signature".into()),
},
MessageContent::Text("Keep this answer".into()),
],
)?;
assert_eq!(request.messages[0].content().len(), 1);
assert!(matches!(
request.messages[0].content()[0],
BedrockInnerContent::Text(_)
));
}
let malformed = request_for(
ConverseModel::Gpt6Astra,
details,
vec![
MessageContent::RedactedThinking("not base64!".into()),
MessageContent::Text("Keep this answer".into()),
],
)?;
assert_eq!(malformed.messages[0].content().len(), 1);
Ok(())
}
#[test]
fn test_astra_parallel_tool_results_precede_their_images() -> Result<()> {
let results = (1..=2)
.map(|index| {
let mut content = Vec::new();
if index == 1 {
content.push(LanguageModelToolResultContent::Text("Result 1".into()));
}
content.push(LanguageModelToolResultContent::Image(
language_model::LanguageModelImage {
source: "iVBORw0KGgo=".into(),
},
));
MessageContent::ToolResult(language_model::LanguageModelToolResult {
tool_use_id: format!("tool-{index}").into(),
tool_name: "view_image".into(),
is_error: false,
content,
output: None,
})
})
.collect();
let request = into_bedrock(
LanguageModelRequest {
messages: vec![LanguageModelRequestMessage {
role: Role::User,
content: results,
cache: false,
reasoning_details: None,
}],
..Default::default()
},
"us.openai.gpt-6-astra".into(),
None,
128_000,
BedrockModelMode::Default,
false,
true,
None,
None,
)?;
let kinds = request.messages[0]
.content()
.iter()
.map(|content| match content {
BedrockInnerContent::ToolResult(_) => "result",
BedrockInnerContent::Image(_) => "image",
_ => "other",
})
.collect::<Vec<_>>();
assert_eq!(kinds, ["result", "result", "image", "image"]);
assert!(matches!(
&request.messages[0].content()[1],
BedrockInnerContent::ToolResult(result)
if matches!(result.content().first(), Some(BedrockToolResultContentBlock::Text(_)))
));
Ok(())
}
#[test]
fn test_converse_refusals_and_fable_forced_tool_use() -> Result<()> {
let fable = ConverseModel::ClaudeFable5_1;
let earlier_fable = ConverseModel::ClaudeFable5;
assert!(!converse_language_model(&fable).tool_choice_support.any);
assert!(
converse_language_model(&earlier_fable)
.tool_choice_support
.any
);
let forced = into_bedrock(
LanguageModelRequest {
tool_choice: Some(LanguageModelToolChoice::Any),
..Default::default()
},
fable.cross_region_inference_id("us-east-1", false)?,
fable.default_temperature(),
fable.max_output_tokens(),
fable.thinking_mode(),
fable.supports_caching(),
fable.supports_tool_use(),
None,
None,
);
assert!(forced.is_err());
for model in [fable, earlier_fable, ConverseModel::Gpt6Astra] {
for reason in [StopReason::from("refusal"), StopReason::ContentFiltered] {
let response = futures::stream::iter(vec![Ok(ConverseStreamOutput::MessageStop(
bedrock::bedrock_client::types::MessageStopEvent::builder()
.stop_reason(reason)
.build()?,
))]);
let events = futures::executor::block_on(
map_to_language_model_completion_events(Box::pin(response), model.clone())
.collect::<Vec<_>>(),
)
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
assert_eq!(
events,
[LanguageModelCompletionEvent::Stop(
language_model::StopReason::Refusal
)]
);
}
}
Ok(())
}
#[test]
fn test_custom_fable_5_1_entry_matches_the_built_in_model() -> Result<()> {
use bedrock::bedrock_client::types::MessageStartEvent;
let custom = ConverseModel::Custom {
name: "us.anthropic.claude-fable-5-1".into(),
max_tokens: 1_000_000,
display_name: None,
max_output_tokens: Some(128_000),
default_temperature: None,
cache_configuration: None,
supports_tool_use: Some(true),
supports_images: Some(true),
thinking: Some(bedrock::BedrockThinkingConfig {
adaptive: true,
has_xhigh: true,
budget_tokens: None,
}),
};
assert!(!converse_language_model(&custom).tool_choice_support.any);
let response = futures::stream::iter(vec![Ok(ConverseStreamOutput::MessageStart(
MessageStartEvent::builder()
.role(bedrock::BedrockRole::Assistant)
.build()?,
))]);
let events = futures::executor::block_on(
map_to_language_model_completion_events(Box::pin(response), custom.clone())
.collect::<Vec<_>>(),
)
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
let details = events.iter().find_map(|event| match event {
LanguageModelCompletionEvent::ReasoningDetails(value) => Some(Arc::new(value.clone())),
_ => None,
});
let request = into_bedrock(
LanguageModelRequest {
messages: vec![LanguageModelRequestMessage {
role: Role::Assistant,
content: vec![
MessageContent::Thinking {
text: "thinking".into(),
signature: Some("signature".into()),
},
MessageContent::Text("answer".into()),
],
cache: false,
reasoning_details: details,
}],
..Default::default()
},
custom.cross_region_inference_id("us-east-1", false)?,
custom.default_temperature(),
custom.max_output_tokens(),
custom.thinking_mode(),
custom.supports_caching(),
custom.supports_tool_use(),
None,
None,
)?;
assert!(matches!(
request.messages[0].content().first(),
Some(BedrockInnerContent::ReasoningContent(_))
));
Ok(())
}
#[test]
fn test_thinking_disallowed_sends_explicit_opt_out_only_on_opus_5() {
// Claude Opus 5 runs adaptive thinking by default when the `thinking`