nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-08-22 10:09:29 +00:00

250 lines
2.6 KiB
Text

ADmantX
AhrefsBot
AllSubmitter
Anarchy99
Aspiegel
Asterias
Atomseobot
AwarioRssBot
BDCbot
BDFetch
BLEXBot
BackStreet
BackWeb
Barkrowler
Bigfoot
Black Hole
Blow
BlowFish
Boardreader
Brandprotect
BuiltWith
BuzzSumo
CSHttp
CensysInspect
CheTeam
Cloud mapping
Cosmos
Crescent
CrunchBot
Curious
Custo
DBLBot
DIIbot
DSearch
Demon
Deusu
Devil
Discobot
Discoverybot
Dispatch
DnBCrawler-Analytics
DnyzBot
DomainStatsBot
Dotbot
Drip
EMail Siphon
Ecxi
Evil
Exabot
Express WebPictures
ExtractorPro
FDM
FHscan
Fimap
Firefox/7.0
FlashGet
Flunky
Foobot
Freeuploader
Fuzz
FyberSpider
Fyrebot
GalaxyBot
GermCrawler
Getintent
Gigabot
GoZilla
Grabber
GrapeshotCrawler
GridBot
HMView
HTMLparser
HaosouSpider
Harvest
Hloader
IDBot
IRLbot
Id-search
Indy Library
InfoNaviRobot
InfoTekies
InterGET
Internet Ninja
Iria
Joomla
Jorgee
Kinza
Larbin
LeechFTP
Libwhisker
Lightspeedsystems
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider
MFC_Tear_Sample
MSFrontPage
MTRobot
Mag-Net
Mail.RU_Bot
Majestic SEO
Majestic12
Masscan
Mata Hari
MegaIndex.ru
Metauri
Microsoft Data Access
Minefield
Mister PiX
MolokaiBot
Msrabot
Musobot
NPbot
Name Intelligence
Navroad
NearSite
Nessus
NetSpider
Nettrack
NextGenSearchBot
Ninja
Not
Nutch
Octopus
OnCrawl
OpenLinkProfiler
OrangeBot
OutclicksBot
POE-Component-Client-HTTP
Page Analyzer
PageThing.com
Panscient
Papa Foto
PictureFinder
Piepmatz
PleaseCrawl
Pockey
PyCurl
RankActiveLinkBot
RankFlex
RealDownload
Recorder
Ripper
Rogerbot
SEOkicks
SEOstats
SMTBot
SalesIntelligent
ScanAlert
Scanbot
ScoutJet
Scrapy
Screaming
ScreenerBot
ScrepyBot
Searchestate
SearchmetricsBot
SemanticJuice
SemrushBot
SeobilityBot
Seomoz
Site Sucker
SiteCheckerBotCrawler
SiteSnagger
SiteSucker
Sitevigil
SmartDownload
SocialRankIOBot
Sottopop
Spammen
Spanner
Sqlworm
Sqworm
Steeler
Stripper
Sucker
Sucuri
SuperBot
Suzuran
Szukacz
Telesoft
Telesphoreo
Telesphorep
TheNomad
TightTwatBot
Tracemyfile
Turnitin
TwengaBot
Typhoeus
URLy Warning
V-BOT
Vacuum
VidibleScraper
VoidEYE
Voil
Voltron
WASALive-Bot
WISENutbot
WWW-Collector-E
WWW::Mechanize
Wallpapers
Wallpapers/3.0
WallpapersHD
Web Auto
Web Enhancer
Web Pix
Web Sucker
WebBandit
WebCopier
WebEnhancer
WebReaper
WebSauger
WebWhacker
Webalta
Website Quester
Whack
Who.is Bot
WinHTTrack
Wprecon
Xaldon WebSpider
Xaldon_WebSpider
Xenu
YoudaoBot
Zauba
Zeus
Zitebot
ZmEu
ZumBot
ZyBorg
cah.io.community
check1.exe
crawl.sogou.com
crawler.feedback
domainsproject.org
heritrix
isitwp.com
linkdexbot
lwp-request
magpie-crawler
mediawords
seocompany.store
sexsearcher
siteripz
sp_auditbot
spyfu
sysscan
ubermetrics-technologies.com
x09Mozilla
xpymep1.exe