nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-07-06 10:09:16 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404checker
80legs
ADmantX
AIBOT
Aboundexbot
Acunetix
Alexibot
Anarchy
Ankit
Apexoo
Aspiegel
Asterias
Attach
BackStreet
Backlink-Ceck
Badass
Battleztar Bazinga
BetaBot
Bitacle
BlowFish
Boardreader
Bolt
Brandwatch
Buck
BuiltWith
Bullseye
Copier
Cosmos
Crescent
CrunchBot
DIIbot
DSearch
DTS Agent
DatabaseDriverMysqli
Demon
Deusu
Disco
Discoverybot
DomainAppender
DomainSigmaCrawler
DomainStatsBot
Domains Project
Download Wonder
EMail Siphon
EMail Wolf
Evil
Exabot
Extractor
ExtractorPro
EyeNetIE
Fimap
Firefox/7.0
FlashGet
Flunky
Foobot
FrontPage
FyberSpider
GT::WWW
GetRight
Gigabot
Go!Zilla
GrabNet
Grafula
GridBot
HEADMasterSEO
HaosouSpider
Harvest
IDBTE4M
IDBot
IlseBot
Image Fetch
IndeedBot
InfoNaviRobot
InfoTekies
Intelliseek
InterGET
Internet Ninja
InternetSeer
Iskanie
IstellaBot
JetCar
Jetty
JikeSpider
Joomla
Jorgee
Jyxobot
Keyword Density
Kozmosbot
LeechFTP
LeechGet
Libwhisker
LieBaoFast
LinkWalker
LinksManager
LinqiaRSSBot
Lipperhey
Lipperhey Spider
Lmspider
MIIxpc
MJ12bot
MSFrontPage
MTRobot
Magnet
Majestic SEO
Mass Downloader
MauiBot
Mb2345Browser
MeanPath Bot
MegaIndex.ru
Microsoft URL Control
Minefield
Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mr.4x3
Name Intelligence
Nameprotect
Navroad
NearSite
Nessus
NetMechanic
NetSpider
NextGenSearchBot
Nibbler
Nikto
NimbleCrawler
Nimbostratus
Ninja
Nuclei
Offline Explorer
OnCrawl
Openvas
OrangeBot
OrangeSpider
PECL::HTTP
PHPCrawl
Page Analyzer
PageAnalyzer
PeoplePal
Petalbot
Pi-Monster
Picscout
Picsearch
Piepmatz
PleaseCrawl
Probethenet
Psbot
Pu_iN
PxBroker
QueryN Metasearch
Quick-Crawler
RankingBot2
RankurBot
RealDownload
RebelMouse
Recorder
RedesScrapy
RepoMonkey
Ripper
SEOkicks-Robot
SEOlyticsCrawler
SEOstats
Scanbot
ScoutJet
SearchmetricsBot
SemanticJuice
SeobilityBot
Shodan
SiteExplorer
SiteLockSpider
Sitevigil
Snapbot
SocialRankIOBot
Sogou web spider
Sosospider
Spanner
Spbot
Spinn3r
SputnikBot
Sqlmap
Sqlworm
Sqworm
SuperBot
SuperHTTP
Surfbot
Szukacz
T0PHackTeam
Telesphoreo
The Intraformant
Thumbor
Titan
Toata
Trendiction
Turingos
Turnitin
UnisterBot
Upflow
VB Project
VidibleScraper
WISENutbot
WPScan
WWW-Collector-E
WallpapersHD
Web Enhancer
WebEnhancer
WebFetch
WebImageCollector
WebZIP
WebmasterWorldForumBot
Website Quester
Webster
Whatweb
WiseGuys Robot
Wonderbot
YoudaoBot
Zauba
Zitebot
ZoominfoBot
ZumBot
arquivo-web-crawler
arquivo.pt
backlink-check
cognitiveseo
crawler.feedback
heritrix
ips-agent
lwp-trivial
meanpathbot
mediawords
netEstate NE Crawler
plumanalytics
s1z.ru
satoristudio.net
scan.lol
seocompany.store
seoscanners
serpstatbot
sexsearcher
sitechecker.pro
sogouspider
sp_auditbot
tAkeOut
trendiction.com
ubermetrics-technologies.com
webmeup-crawler
webpros.com
webprosbot
x09Mozilla
zauba.io