nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-03-16 10:07:26 +00:00

100 lines
1 KiB
Text

80legs
AfD-Verbotsverfahren
AhrefsBot
AllSubmitter
Anarchie
Ankit
Aspiegel
AwarioRssBot
AwarioSmartBot
BackWeb
Badass
BetaBot
Blow
Bolt
BotALot
Bytespider
CheTeam
Custo
DSearch
Devil
Ebingbong
EroCrawler
ExtractorPro
Freeuploader
FrontPage
Fyrebot
GPTBot
GalaxyBot
Google-Extended
GrabNet
GrapeFX
HEADMasterSEO
HTTP::Lite
HTTrack
Hloader
IDBTE4M
Iria
Jorgee
Larbin
LeechGet
LibWeb
Libwhisker
Likse
LinkScan
LinqiaScrapeBot
Lipperhey Spider
Lmspider
Ltx71
MSIECrawler
Masscan
Meanpathbot
Mozlila
Navroad
NetSpider
Nibbler
NimbleCrawler
PECL::HTTP
PageScorer
PageThing.com
Pandalytics
Proximic
PyCurl
RSSingBot
RankurBot
ReGet
SEOlyticsCrawler
SEOprofiler
SeekportBot
SemrushBot
SeobilityBot
SiteCheckerBotCrawler
SiteExplorer
Sitebeam
Siteimprove
Sitevigil
SlySearch
Stripper
Thumbor
Trendictionbot
URLy Warning
VCI
Vagabondo
Wallpapers/3.0
WebGo IS
WebSauger
Webshag
anthropic-ai
autoemailspider
com.plumanalytics
crawl.sogou.com
heritrix
linkdexbot
lwp-request
mediawords
openai
serpstatbot
sysscan
tAkeOut
webgains-bot
xpymep1.exe