nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-03-03 10:09:35 +00:00

250 lines
2.6 KiB
Text

01h4x.com
360Spider
404checker
AIBOT
Aboundex
Acunetix
AhrefsBot
AiHitBot
Alexibot
Alligator
AlphaBot
Anarchie
Ankit
Aspiegel
Atomseobot
Attach
BDFetch
BLEXBot
BackStreet
Barkrowler
Battleztar Bazinga
BetaBot
Black Hole
BlackWidow
Blackboard
BotALot
Buck
BuiltBotTough
CATExplorador
Calculon
CazoodleBot
ChinaClaw
Cloud mapping
Cocolyzebot
Cogentbot
CopyRightCheck
Cosmos
CrazyWebCrawler
Curious
Custo
CyotekWebCopy
DSearch
Dirbuster
Discobot
Discoverybot
DnBCrawler-Analytics
Dotbot
Dragonfly
EMail Siphon
EasyDL
EroCrawler
Exabot
FemtosearchBot
FrontPage
Fuzz
Fyrebot
Genieo
GermCrawler
Gigabot
Go!Zilla
GoZilla
Gotit
Grabber
GrapeshotCrawler
GridBot
HMView
HTTP::Lite
HTTrack
Harvest
Havij
IlseBot
Image Sucker
InfoTekies
Intelliseek
Internet Ninja
Iria
JOC Web Spider
JetCar
Jetty
Joomla
Jyxobot
Kinza
Lanshanbot
LeechFTP
LeechGet
LexiBot
LibWeb
Libwhisker
LinkScan
Linkbot
Linkdexbot
LinksManager
LinqiaMetadataDownloaderBot
Lipperhey Spider
MFC_Tear_Sample
MIDown tool
MJ12bot
MQQBrowser
MSFrontPage
MSIECrawler
Magnet
Majestic SEO
MarkWatch
Mass Downloader
Meanpathbot
Mediatoolkitbot
MegaIndex.ru
MicroMessenger
Microsoft URL Control
Minefield
Mister PiX
Moblie Safari
MolokaiBot
Mozlila
Msrabot
Name Intelligence
Nameprotect
Nessus
NetLyzer
NetMechanic
NetSpider
Nettrack
Netvibes
Nmap
Nuclei
Nutch
Offline Navigator
Openvas
OrangeBot
OutfoxBot
PHPCrawl
PageThing.com
Papa Foto
Pavuk
PeoplePal
ProWebWalker
Probethenet
Psbot
PxBroker
Quick-Crawler
RankingBot2
Rankivabot
RealDownload
RepoMonkey
Ripper
RocketCrawler
Rogerbot
SEOlyticsCrawler
SISTRIX
SMTBot
Scanbot
Scrapy
ScrepyBot
Searchestate
Seekport
SemanticJuice
SemrushBot
SentiBot
SeoSiteCheckup
Siphon
Site Sucker
SiteLockSpider
Siteimprove
Sitevigil
SmartDownload
Snake
Snapbot
Sosospider
SpaceBison
Spanner
Spbot
Spinn3r
Sqworm
Steeler
Stripper
Sucker
Sucuri
Szukacz
Teleport
Telesphoreo
Telesphorep
TheNomad
Thumbor
TightTwatBot
Titan
Trendiction
Turnitin
TurnitinBot
Twice
V-BOT
Vacuum
VidibleScraper
Virusdie
Voil
Voltron
WBSearchBot
WISENutbot
WWW-Collector-E
WeSEE
Web Fetch
Web Pix
WebCollage
WebEnhancer
WebFetch
WebImageCollector
WebLeacher
Webalta
WebmasterWorldForumBot
Website Quester
Webster
Whack
Whatweb
Widow
YoudaoBot
Zade
Zauba
ZmEu
ZumBot
ZyBorg
arquivo-web-crawler
backlink-check
cah.io.community
coccocbot
com.plumanalytics
crawler.feedback
demandbase-bot
domainsproject.org
eCatch
evc-batch
gopher
internetVista monitor
ips-agent
isitwp.com
lwp-request
mediawords
netEstate NE Crawler
ripz
s1z.ru
scalaj-http
seoscanners
sogouspider
sp_auditbot
spyfu
trendiction.com
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
x09Mozilla
x22Mozilla
xpymep1.exe