nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-09-04 22:07:28 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404checker
ALittle Client
ASPSeek
Acunetix
AiHitBot
AllSubmitter
AlphaBot
Anarchy
Anthill
BackDoorBot
BackStreet
BacklinkCrawler
Bandit
Barkrowler
BlackWidow
Blow
Bolt
Brandwatch
Buddy
BunnySlippers
Calculon
CazoodleBot
Cegbfeieh
CensysInspect
CherryPicker
Chlooe
Claritybot
Cliqzbot
Cloud mapping
Collector
Copyscape
CrunchBot
Curious
DBLBot
Digincore
Dispatch
DittoSpyder
DnyzBot
DomCopBot
DomainAppender
DomainStatsBot
Domains Project
Dotbot
Download Wonder
EMail Wolf
EasyDL
EirGrabber
Evil
Exabot
ExtLinksBot
ExtractorPro
EyeNetIE
FHscan
FemtosearchBot
FlashGet
Flunky
Foobot
Freeuploader
FrontPage
Fyrebot
G-i-g-a-b-o-t
GT::WWW
Genieo
GetRight
Getintent
Go-Ahead-Got-It
Gotit
GrabNet
Grabber
GridBot
HEADMasterSEO
HTTrack
HaosouSpider
Harvest
Humanlinks
IDBTE4M
IDBot
IRLbot
Iblog
IlseBot
Image Fetch
Indy Library
Intelliseek
Internet Ninja
InternetSeer
Iskanie
JOC Web Spider
JamesBOT
JustView
Kenjin Spider
Keyword Density
Kozmosbot
LNSpiderguy
Larbin
LibWeb
Libwhisker
LieBaoFast
LinkScan
LinkWalker
Linkbot
LinkpadBot
LinqiaScrapeBot
Lipperhey
Litemage_walker
Ltx71
MFC_Tear_Sample
Mag-Net
Majestic SEO
Majestic12
Masscan
MeanPath Bot
MegaIndex.ru
Moblie Safari
Mojolicious
Mozlila
Msrabot
Musobot
NICErsPRO
NearSite
Needle
Nessus
NetMechanic
Netcraft
Netvibes
Niki-bot
NimbleCrawler
Nimbostratus
Nuclei
Octopus
Offline Explorer
Offline Navigator
OpenVAS
Openvas
OrangeBot
OrangeSpider
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
PageScorer
PageThing.com
Pandalytics
Panscient
Pavuk
Pi-Monster
Picsearch
PleaseCrawl
Pockey
ProWebWalker
Probethenet
Psbot
Pump
PyCurl
Quick-Crawler
RSSingBot
RankingBot2
Reaper
Recorder
RocketCrawler
SEOkicks
SISTRIX
SMTBot
SalesIntelligent
Scrapy
SearchmetricsBot
Semrush
SemrushBot
Seomoz
Site Sucker
Sitebeam
Siteimprove
Sociscraper
Sogou web spider
Sosospider
Spammen
SpankBot
Spbot
SputnikBot
Sqworm
Steeler
SurveyBot
Swiftbot
Teleport
Telesphoreo
TheNomad
Thumbor
Toata
Tracemyfile
True_Robot
Turnitin
TurnitinBot
Typhoeus
URLy.Warning
Vacuum
Vagabondo
VelenPublicWebCrawler
VidibleScraper
Voil
WASALive-Bot
WEBDAV
WPScan
WWW-Collector-E
Web Enhancer
Web Sucker
WebGo IS
WebReaper
WebSucker
WebWhacker
Whacker
Who.is Bot
Widow
Wonderbot
Zitebot
ZmEu
ZoominfoBot
ZumBot
ZyBorg
arquivo.pt
autoemailspider
cah.io.community
check1.exe
clark-crawler
coccocbot-web
crawler4j
dataforseo.com
demandbase-bot
eCatch
ips-agent
lwp-request
lwp-trivial
magpie-crawler
meanpathbot
muhstik-scan
page scorer
pcBrowser
plumanalytics
polaris version
ripz
satoristudio.net
siteripz
spyfu
sysscan
trendiction.de
voyagerx.com
webpros.com
webprosbot
x09Mozilla
zgrab