nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-08-15 05:24:10 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404enemy
Aboundex
Aboundexbot
AhrefsBot
AllSubmitter
Alligator
AlphaBot
Ankit
Anthill
Aspiegel
BBBike
BLEXBot
BackDoorBot
BackStreet
BackWeb
Backlink-Ceck
Battleztar Bazinga
BetaBot
Bigfoot
BlackWidow
BuiltBotTough
BuiltWith
CATExplorador
CODE87
CazoodleBot
Cegbfeieh
CheTeam
CheeseBot
ChinaClaw
Chlooe
Cocolyzebot
Collector
Cosmos
Craftbot
CrunchBot
DSearch
DTS Agent
DataCha0s
Deusu
Digincore
Dirbuster
Discobot
DomCopBot
DomainCrawler
DomainStatsBot
Domains Project
EMail Wolf
EirGrabber
Express WebPictures
ExtractorPro
Ezooms
FHscan
FemtosearchBot
FlashGet
FyberSpider
Fyrebot
GT::WWW
GetRight
Getintent
Go!Zilla
GoZilla
Gotit
GrabNet
Grabber
Grafula
HEADMasterSEO
HMView
HTMLparser
HaosouSpider
Havij
Humanlinks
HybridBot
IRLbot
Iblog
IndeedBot
InfoNaviRobot
Internet Ninja
Iskanie
JOC Web Spider
Jbrofuzz
JennyBot
JetCar
Joomla
Jorgee
JustView
Kinza
LNSpiderguy
LeechGet
LexiBot
Lftp
LinkWalker
Linkbot
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
MJ12bot
MSFrontPage
Mag-Net
Magnet
MarkWatch
MauiBot
Mb2345Browser
MeanPath Bot
Microsoft Data Access
Mojeek
Mojolicious
Mozlila
Mr.4x3
Musobot
Nessus
NetAnts
NetSpider
Netcraft
Netvibes
Nibbler
Niki-bot
Nikto
NimbleCrawler
Nimbostratus
Ninja
Nmap
Nutch
Offline Navigator
OpenLinkProfiler
Openfind
OrangeBot
OrangeSpider
PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer
PageScorer
PageThing.com
Picsearch
Pixray
PleaseCrawl
ProPowerBot
ProWebWalker
Psbot
Pump
QueryN Metasearch
Quick-Crawler
RankFlex
RankingBot
Rankivabot
RankurBot
RealDownload
RedesScrapy
Ripper
Rogerbot
SEOkicks-Robot
SEOstats
SMTBot
ScanAlert
Scrapy
Screaming
ScrepyBot
SentiBot
SeoSiteCheckup
SeobilityBot
Shodan
Site Sucker
SiteCheckerBotCrawler
Sitebeam
SmartDownload
Sogou web spider
Sosospider
Spanner
Spbot
Sqlmap
Sqlworm
Sqworm
Stripper
SuperBot
Surfbot
Swiftbot
T0PHackTeam
Telesoft
Telesphoreo
The Intraformant
Thumbor
TightTwatBot
Tracemyfile
Trendictionbot
True_Robot
Turingos
TurnitinBot
TwengaBot
URLy.Warning
Upflow
Vacuum
VeriCiteCrawler
VidibleScraper
WASALive-Bot
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
WeSEE
Web Collage
Web Enhancer
Web Fuck
Web Pix
Web Sauger
Web Sucker
WebCopier
WebFetch
WebGo IS
WebStripper
WebWhacker
Webalta
WebmasterWorldForumBot
WebsiteExtractor
Whack
Whacker
Wonderbot
Xaldon WebSpider
Zade
Zermelo
Zeus
ZoomBot
ZumBot
ZyBorg
arquivo.pt
backlink-check
demandbase-bot
evc-batch
gopher
heritrix
instabid
internetVista monitor
lwp-trivial
meanpathbot
mediawords
netEstate NE Crawler
pcBrowser
plumanalytics
probe-image-size
satoristudio.net
scan.lol
sexsearcher
sogouspider
sp_auditbot
sysscan
tAkeOut
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
zauba.io
zgrab