nginx-ultimate-bad-bot-blocker/.dev-tools/test_units/random-bots-for-whitelist-test.list
2020-02-08 21:49:48 +02:00

250 lines
2.6 KiB
Text

80legs
Abonti
Aboundex
ADmantX
AfD-Verbotsverfahren
AhrefsBot
AIBOT
Alexibot
Alligator
AllSubmitter
Apexoo
Asterias
BackDoorBot
Backlink-Ceck
backlink-check
BackStreet
BackWeb
Badass
Bandit
Barkrowler
BDCbot
BDFetch
BlowFish
Boardreader
Brandprotect
Calculon
CCBot
CheeseBot
CherryPicker
CheTeam
Chlooe
Claritybot
Cliqzbot
Cloud mapping
Collector
com.plumanalytics
CopyRightCheck
crawler4j
CrazyWebCrawler
Crescent
CrunchBot
CSHttp
DataCha0s
DBLBot
demandbase-bot
Devil
DIIbot
Discobot
Dispatch
DomainSigmaCrawler
Dotbot
ECCP/1.0
EirGrabber
EroCrawler
evc-batch
Exabot
Extractor
Extreme Picture Finder
Ezooms
facebookscraper
FemtosearchBot
Fimap
Freeuploader
Genieo
GermCrawler
GetWeb
Gigablast
G-i-g-a-b-o-t
GrapeFX
GrapeshotCrawler
GT::WWW
HaosouSpider
Harvest
heritrix
Hloader
HTMLparser
HybridBot
Id-search
IlseBot
Image Sucker
IndeedBot
InfoNaviRobot
instabid
InterGET
Internet Ninja
InternetSeer
IRLbot
IstellaBot
JamesBOT
Jbrofuzz
JetCar
Jetty
Jyxobot
Kenjin Spider
Kozmosbot
LeechFTP
LeechGet
Likse
LinkScan
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey
Litemage_walker
lwp-request
LWP::Simple
lwp-trivial
magpie-crawler
Mail.RU_Bot
MarkMonitor
MarkWatch
Masscan
MauiBot
meanpathbot
Mediatoolkitbot
mediawords
MegaIndex.ru
Metauri
Microsoft URL Control
Mojeek
MSFrontPage
MSIECrawler
Msrabot
Nameprotect
Nessus
Netcraft
NetLyzer
NetMechanic
NICErsPRO
NimbleCrawler
Nimbostratus
Nmap
Nutch
oBot
Octopus
OnCrawl
OpenLinkProfiler
OutfoxBot
Page Analyzer
page scorer
PageScorer
Pavuk
PictureFinder
PleaseCrawl
plumanalytics
Pockey
POE-Component-Client-HTTP
Probethenet
ProWebWalker
Psbot
PyCurl
QueryN Metasearch
Quick-Crawler
RankActive
RankActiveLinkBot
Rankivabot
RankurBot
RealDownload
Reaper
Recorder
Rogerbot
RSSingBot
SalesIntelligent
Scanbot
ScreenerBot
Searchestate
SearchmetricsBot
Semrush
SEOkicks
SEOlyticsCrawler
Seomoz
SEOprofiler
seoscanners
SeoSiteCheckup
SEOstats
serpstatbot
Sitebeam
SiteExplorer
SiteSnagger
Sitevigil
SmartDownload
Snake
Snapbot
Sociscraper
Sogou web spider
SpaceBison
Spbot
SputnikBot
spyfu
Steeler
Stripper
Swiftbot
T8Abot
tAkeOut
Teleport
TeleportPro
Telesoft
Telesphorep
TheNomad
TightTwatBot
Toweyabot
Tracemyfile
Trendiction
Trendictionbot
trendiction.com
trendiction.de
TwengaBot
Vagabondo
VB Project
VCI
VeriCiteCrawler
Virusdie
VoidEYE
Voil
Wallpapers/3.0
WASALive-Bot
WebAuto
WebBandit
WebFetch
Web Fetch
Web Fuck
WebGo IS
WebReaper
Webshag
WebsiteQuester
Webster
WebStripper
WebWhacker
WebZIP
WeSEE
Whack
Whacker
WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Wotbox
Wprecon
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
x09Mozilla
x22Mozilla
Xaldon_WebSpider
Xenu
xpymep1.exe
YoudaoBot
zauba.io
Zermelo
Zeus