- Google Search: Google's sorry wall is recognised by host and path, not a
"/sorry" substring, so a result linking to a page with "sorry" in its URL
no longer burns the wave's exit (goto and the captcha check share
captcha.is_sorry_url).
- Walmart: a redirect still pending after the last allowed hop is a refused
try, so the next exit is tried instead of reading an empty page.
- Amazon: a hedge whose sticky lease is unavailable falls back to the
gateway instead of spending the call's hedge budget on nothing.
- Fixtures: live tracking and session tokens scrubbed from the captured
pages (Indeed, Amazon, Google, Google Maps, TikTok, YouTube).
- plugin-host.md: shared_access() everywhere, and the Reddit and Instagram
gaps now describe their fail-fast latches.
- Fail fast and rotate on Google's sorry redirect instead of following it;
hedge slow requests onto a second exit (1 rotation per refused exit).
- Subtitles from YouTube's own player on the warm session, alongside the
watch fetch and across videos: a video with subtitles went from 13 s to 1 s.
- Channel tabs fetched together without the redundant sort chip; the
playlist stops at its limit; comments read reply pages and use /next.
- JSON read with raw_decode and orjson, one parse per page.
- Fixes: search terms dropped next to urls, truncated search text, rounded
views and likes, SEO keywords as hashtags, duration formats, verified
badges, reply pagination, None counts, a translation request without
subtitles, and relative times.
- The agent reads every video in the text; transcripts share the cap.
YOUTUBE_KEEP_IDLE=1 keeps warm sessions across calls (off by default).
keygen.py held Keygen Cloud's address and an empty account id, so the host could
not start with a paid plugin, and could not reach the self-hosted Keygen CE that
holds production's licenses. It now reads KEYGEN_API_URL (the accounts endpoint,
Keygen Cloud's when unset), KEYGEN_ACCOUNT_ID and KEYGEN_HOST, with the backend's
names and meanings, so a deployment gives both the same values. With KEYGEN_HOST
set it sends Host and X-Forwarded-Proto: https, as CE needs.
A 4xx or a redirect from Keygen refuses the key as invalid, with a warning in the
log, instead of failing the request with a 500. A reply with no code is no
answer, so the 24-hour grace still applies.
proprietary/youtube/ is the backend's YouTube scraper behind two tools at
/youtube/mcp, scrape and comments, licensed like Reddit, with progress per
item from an on_item callback. youtube-transcript-api, fake-useragent and
requests join the host for its subtitles. The backend's tests came with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
