nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-04-24 12:19:07 +00:00

100 lines
1.1 KiB
Text

ALittle Client
Acunetix
Aipbot
Anthill
AwarioBot
Black Hole
BlackWidow
BotALot
Buck
BuiltWith
CSHttp
Crawling at Home Project
CrunchBot
CyotekWebCopy
DnBCrawler-Analytics
DomCopBot
Exabot
ExtLinksBot
GalaxyBot
Go!Zilla
Go-Ahead-Got-It
Grafula
HTMLparser
Haansoft
Heritrix
IDBTE4M
InfoNaviRobot
Joomla
Kenjin Spider
Keyword Density
LWP::Simple
LibWeb
Libwhisker
Lightspeedsystems
MIDown tool
MSIECrawler
MTRobot
Mag-Net
Masscan
Mata Hari
MicroMessenger
Microsoft Data Access
Microsoft URL Control
Msrabot
NPbot
Net Vampire
Netcraft
PeoplePal
Picsearch
Piepmatz
PleaseCrawl
ProPowerBot
ProWebWalker
ReGet
Scrapy
SenutoBot
Site Sucker
SiteCheckerBotCrawler
SiteExplorer
SiteSucker
Sitevigil
Snake
Sottopop
Spanner
Spinn3r
Sqworm
Steeler
SuperBot
SurveyBot
Thumbor
Toata
TurnitinBot
URLy.Warning
UnisterBot
Voil
Voltron
WWW-Mechanize
Web Fetch
WebLeacher
WebmasterWorldForumBot
WebsiteQuester
Woobot
Wprecon
Xenu
Zeus
ZoomBot
ZyBorg
arquivo.pt
autoemailspider
cohere-ai
com.plumanalytics
heritrix
internetVista monitor
iubenda-radar
mediawords
ripz
s1z.ru
webmeup-crawler
webpros.com
xpymep1.exe