nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-02-27 10:10:26 +00:00

250 lines
2.6 KiB
Text

01h4x.com
360Spider
404checker
80legs
Aboundex
AiHitBot
Anarchie
Anarchy
Anarchy99
Anthill
Asterias
BBBike
BDFetch
BackDoorBot
BackWeb
Badass
Bandit
Barkrowler
Bigfoot
Bitacle
Black Hole
Blow
Boardreader
Bolt
BotALot
Buddy
BuiltWith
BunnySlippers
CCBot
CSHttp
Calculon
Cegbfeieh
CheTeam
Claritybot
Cloud mapping
Cogentbot
Crawling at Home Project
CrazyWebCrawler
Crescent
CyotekWebCopy
DTS Agent
DataCha0s
Deusu
Digincore
DigitalPebble
Disco
Discobot
DittoSpyder
DomainAppender
DomainSigmaCrawler
Download Wonder
EMail Siphon
Ebingbong
EroCrawler
Evil
Exabot
ExtLinksBot
ExtractorPro
Ezooms
Fimap
Firefox/7.0
FlashGet
Flunky
Foobot
Freeuploader
GalaxyBot
GetRight
GetWeb
Go!Zilla
Go-Ahead-Got-It
GoZilla
GrabNet
GridBot
Haansoft
HaosouSpider
Havij
Hloader
Id-search
Image Fetch
InfoNaviRobot
InfoTekies
InternetSeer
Iskanie
Joomla
Keyword Density
Kozmosbot
Lanshanbot
LeechFTP
LeechGet
LexiBot
Lftp
LieBaoFast
LinkextractorPro
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
Lipperhey Spider
Ltx71
MFC_Tear_Sample
MJ12bot
MQQBrowser
MTRobot
Mag-Net
Majestic SEO
Majestic12
Masscan
Mata Hari
Mb2345Browser
Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri
Minefield
Mr.4x3
Msrabot
Name Intelligence
Nameprotect
NearSite
Net Vampire
NetLyzer
Netcraft
Nettrack
NextGenSearchBot
Niki-bot
Nikto
Nimbostratus
Nmap
Not
Octopus
Offline Explorer
Offline Navigator
OpenLinkProfiler
OrangeBot
OutclicksBot
OutfoxBot
POE-Component-Client-HTTP
Page Analyzer
Pandalytics
PeoplePal
Pi-Monster
Picscout
PleaseCrawl
Pockey
ProWebWalker
Psbot
PxBroker
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActiveLinkBot
RankurBot
Re-re
ReGet
RedesScrapy
Ripper
RocketCrawler
SEOkicks
SEOprofiler
SEOstats
ScoutJet
Scrapy
ScrepyBot
Searchestate
SearchmetricsBot
Seekport
Semrush
SentiBot
SiteLockSpider
SiteSnagger
SiteSucker
Sitevigil
SlySearch
Snapbot
Snoopy
Spbot
Spinn3r
Sqlworm
Sucuri
SurveyBot
Teleport
Telesoft
Tracemyfile
Trendiction
Trendictionbot
True_Robot
Turingos
TwengaBot
Twice
Typhoeus
URLy.Warning
V-BOT
VB Project
Vacuum
VidibleScraper
Voltron
WISENutbot
WWW::Mechanize
Wallpapers
WeSEE
Web Auto
Web Sucker
WebBandit
WebCopier
WebEnhancer
WebFuck
WebGo IS
WebLeacher
WebStripper
WebWhacker
WebZIP
Webalta
WebmasterWorldForumBot
Website Quester
WebsiteExtractor
WebsiteQuester
Whack
Who.is Bot
WiseGuys Robot
Woobot
Zermelo
Zitebot
ZmEu
ZoomBot
ZoominfoBot
ZumBot
ZyBorg
adscanner
autoemailspider
clark-crawler
crawler.feedback
dataforseo.com
domainsproject.org
evc-batch
heritrix
instabid
isitwp.com
iubenda-radar
lwp-trivial
muhstik-scan
netEstate NE Crawler
page scorer
s1z.ru
scalaj-http
seocompany.store
seoscanners
sysscan
trendiction.com
ubermetrics-technologies.com
webgains-bot
x22Mozilla