nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-12-25 22:04:50 +00:00

100 lines
1.1 KiB
Text

404enemy
ASPSeek
Aboundexbot
Ankit
BDFetch
Backlink-Ceck
Barkrowler
Battleztar Bazinga
BlackWidow
BuzzSumo
Bytespider
CCBot
CazoodleBot
Craftbot
CrazyWebCrawler
CrunchBot
Curious
Digincore
Dirbuster
Disco
DnBCrawler-Analytics
DomainStatsBot
FemtosearchBot
Freeuploader
GrabNet
Grabber
Grafula
HTMLparser
InfoTekies
Intelliseek
InterGET
InternetMeasurement
Keybot Translation-Search-Machine
Keyword Density
LinqiaScrapeBot
Ltx71
MJ12bot
MSFrontPage
MSIECrawler
Mb2345Browser
MegaIndex.ru
Microsoft Data Access
Musobot
NetLyzer
NetSpider
Nikto
Nutch
PHPCrawl
PageAnalyzer
PageScorer
Panscient
PeoplePal
Piepmatz
Pimonster
RankActive
RankActiveLinkBot
RankingBot
Rankivabot
RankurBot
RepoMonkey
SEOkicks-Robot
ScanAlert
Scrapy
Semrush
Seomoz
Shodan
Snoopy
Sottopop
Spbot
Stripper
SuperBot
Suzuran
Toweyabot
Typhoeus
VCI
Vagabondo
WPScan
WWW-Collector-E
Web Sucker
WebGo IS
WebLeacher
Whatweb
YaK
Zeus
ZumBot
ZyBorg
awario.com
clark-crawler
crawler.feedback
instabid
ips-agent
linkdexbot
lwp-request
pcBrowser
probe-image-size
seoscanners
tAkeOut
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler