vrr/ruvector
2
0
Fork 0
mirror of https://github.com/ruvnet/RuVector.git synced 2026-05-26 07:44:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

fix(adr-115): force HTTP/1.1 and disable connection pooling for Common Crawl

Browse source 
Common Crawl CDX servers have issues with HTTP/2 and connection reuse:
- Force HTTP/1.1 with http1_only() to avoid protocol issues
- Disable connection pooling (pool_max_idle_per_host=0) since CC closes connections
- Add tcp_nodelay for lower latency
This commit is contained in:
Reuven 2026-03-17 01:25:02 -04:00
parent b5c0ddbf96
commit 38b79dc6c0
1 changed files with 3 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
crates/mcp-brain-server/src/pipeline.rs
View file

@ -611,7 +611,9 @@ impl CommonCrawlAdapter {
http: reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(120)) // Increased for CDX latency
.connect_timeout(std::time::Duration::from_secs(30))
.pool_idle_timeout(std::time::Duration::from_secs(90))
.pool_max_idle_per_host(0) // Disable connection pooling (Common Crawl closes connections)
.http1_only() // Force HTTP/1.1 (Common Crawl CDX doesn't handle HTTP/2 well)
.tcp_nodelay(true)
.user_agent("RuVector-Brain/1.0 (pi.ruv.io; +https://github.com/ruvnet/ruvector)")
.build()
.expect("Failed to build reqwest client"),