nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2023-09-23 10:07:10 +00:00

100 lines
1.1 KiB
Text

360Spider
ADmantX
Aboundex
AfD-Verbotsverfahren
AhrefsBot
BackDoorBot
Bolt
BotALot
BuiltWith
CODE87
CazoodleBot
CheTeam
Citoid
Copyscape
Crawling at Home Project
CrunchBot
CyotekWebCopy
Disco
DnyzBot
DomCopBot
DomainSigmaCrawler
EasyDL
Extractor
Firefox/7.0
FyberSpider
GT::WWW
Grafula
Harvest
Heritrix
HonoluluBot
InfoNaviRobot
Iria
Jyxobot
Keybot Translation-Search-Machine
Kozmosbot
Larbin
Leap
LinqiaScrapeBot
Magnet
Mail.RU_Bot
Majestic12
MarkMonitor
MauiBot
Mr.4x3
Name Intelligence
Navroad
Net Vampire
NextGenSearchBot
OrangeBot
OrangeSpider
PageGrabber
Pandalytics
Petalbot
Probethenet
PyCurl
ReGet
Recorder
SEOkicks
SEOlyticsCrawler
SalesIntelligent
Searchestate
SeekportBot
Seomoz
Site Sucker
SiteSucker
SlySearch
Snoopy
Sociscraper
Sosospider
Sottopop
SpankBot
Sqlmap
Suzuran
Swiftbot
T8Abot
TeleportPro
TheNomad
Trendiction
VB Project
VeriCiteCrawler
WWW-Mechanize
WallpapersHD
Web Collage
WebFetch
Website Quester
Webster
Whack
Wprecon
Xaldon WebSpider
arquivo.pt
autoemailspider
dataforseo.com
heritrix
lwp-request
openai.com
page scorer
polaris version
satoristudio.net
scalaj-http
ubermetrics-technologies.com