vrr/SurfSense
2
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-11-01 11:21:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
SurfSense/plugins/remote/proprietary/scraping/tests
History
Exact
DESKTOP-RTLN3BA\$punk 2fb720cd03 feat(plugin-host): plugin lifespans, one license cache, and results the agent reads 
- A plugin's server.py may expose a lifespan: the host enters it at startup,
  warm-ups run in the background, and scraping/lifecycle.py closes the shared
  sessions, browsers and API clients once at shutdown.
- One license cache per process, stale-while-revalidate, instead of one per
  plugin: the app's 5-minute check no longer pays a Keygen round trip per
  plugin. Refusals and the 24 h grace work as before.
- Kept API clients for Keygen, SearXNG and the captcha solver, which now polls
  every second.
- mcp_server/results.py: tool_result puts each item in the text as one
  compact JSON line, within 32 KB. opencode reads only a result's text when
  there is any, so the agent never saw prices, page text or reviews. A request
  with x-surfsense-text-only: 1 gets no structured copy.
- mcp_server/progress.py: stage progress, so a long call never sits silent.
- GET and DELETE on /<id>/mcp answer 405: the stateless host has no stream to
  offer, and each GET held an idle SSE connection open per plugin.
2026-10-14 05:56:11 -07:00
..
fixtures feat(plugins): one browser pool for every scraper plugin 2026-10-14 05:56:11 -07:00
__init__.py feat(plugins): run SurfSense's Reddit scraper as its own plugin on the plugin host 2026-10-12 00:48:27 +05:30
_local_http.py feat(plugins): a fail-fast proxied transport, keep-alive sessions and ordered fan-out 2026-10-14 05:56:10 -07:00
test_api_clients.py feat(plugin-host): plugin lifespans, one license cache, and results the agent reads 2026-10-14 05:56:11 -07:00
test_browser_pool.py feat(plugins): one browser pool for every scraper plugin 2026-10-14 05:56:11 -07:00
test_captcha_config.py feat(plugins): run SurfSense's Google Search scraper as its own plugin 2026-10-12 04:55:11 +05:30
test_captcha_solvers.py feat(plugin-host): plugin lifespans, one license cache, and results the agent reads 2026-10-14 05:56:11 -07:00
test_concurrency.py feat(plugins): a fail-fast proxied transport, keep-alive sessions and ordered fan-out 2026-10-14 05:56:10 -07:00
test_dataimpulse_provider.py feat(plugins): a fail-fast proxied transport, keep-alive sessions and ordered fan-out 2026-10-14 05:56:10 -07:00
test_http.py feat(plugins): a fail-fast proxied transport, keep-alive sessions and ordered fan-out 2026-10-14 05:56:10 -07:00
test_lifecycle.py feat(plugin-host): plugin lifespans, one license cache, and results the agent reads 2026-10-14 05:56:11 -07:00
test_nodemaven_provider.py feat(plugins): a fail-fast proxied transport, keep-alive sessions and ordered fan-out 2026-10-14 05:56:10 -07:00
test_registry.py refactor(plugins): drop the custom proxy provider; name the vendor or scrape direct 2026-10-12 02:26:02 +05:30
test_rotation.py feat(plugins): run SurfSense's Reddit scraper as its own plugin on the plugin host 2026-10-12 00:48:27 +05:30
test_sessions.py feat(plugins): a fail-fast proxied transport, keep-alive sessions and ordered fan-out 2026-10-14 05:56:10 -07:00
test_settle.py feat(plugins): one browser pool for every scraper plugin 2026-10-14 05:56:11 -07:00