nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-02-25 22:04:45 +00:00

100 lines
1.1 KiB
Text

360Spider
404checker
Anthill
AwarioSmartBot
BackWeb
Backlink-Ceck
BacklinkCrawler
BuiltBotTough
BuiltWith
Bullseye
BuzzSumo
CSHttp
CheTeam
Cloud mapping
Cocolyzebot
Collector
Craftbot
Crawling at Home Project
CrunchBot
DataCha0s
Discoverybot
DomainStatsBot
Express WebPictures
ExtLinksBot
ExtractorPro
Firefox/7.0
Flunky
GPTBot
HaosouSpider
Harvest
IlseBot
Image Sucker
InternetSeer
JennyBot
JikeSpider
Jorgee
LNSpiderguy
Libwhisker
LinqiaMetadataDownloaderBot
MIDown tool
MIIxpc
Mail.RU_Bot
MarkWatch
Meanpathbot
Microsoft URL Control
Mozlila
NetSpider
Page Analyzer
PeoplePal
Picscout
PleaseCrawl
Pockey
PyCurl
Quick-Crawler
RankActive
RankingBot
RebelMouse
RedesScrapy
RocketCrawler
SEOkicks-Robot
SMTBot
SalesIntelligent
Scrapy
ScrepyBot
Seekport
SemrushBot
Snake
Snoopy
Sociscraper
Spanner
Spinn3r
Szukacz
VidibleScraper
WWW-Collector-E
Web Fuck
Web Sauger
WebBandit
WebCopier
WebEnhancer
WebImageCollector
Whack
Xaldon WebSpider
Zade
Zeus
adscanner
autoemailspider
backlink-check
eCatch
facebookscraper
heritrix
internetVista monitor
muhstik-scan
page scorer
scan.lol
seocompany.store
seoscanners
sexsearcher
sp_auditbot
trendiction.de
zgrab