nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-05-10 10:09:20 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404checker
ASPSeek
Aboundex
AhrefsBot
Alexibot
Anarchie
Aspiegel
Attach
AwarioRssBot
BDCbot
BDFetch
BackWeb
Backlink-Ceck
Badass
BatchFTP
Battleztar Bazinga
BetaBot
Bigfoot
Bitacle
Boardreader
Brandprotect
Buck
Buddy
BuiltBotTough
Bullseye
CCBot
CSHttp
Cegbfeieh
CheeseBot
ChinaClaw
Claritybot
Cloud mapping
Cocolyzebot
Copier
Copyscape
Cosmos
Craftbot
Crescent
CrunchBot
CyotekWebCopy
DBLBot
DataCha0s
DatabaseDriverMysqli
Demon
Dirbuster
Discobot
Dispatch
DomCopBot
DomainAppender
DomainCrawler
Domains Project
ECCP/1.0
EMail Wolf
Extractor
ExtractorPro
Ezooms
FHscan
FemtosearchBot
Firefox/7.0
FlashGet
Flunky
G-i-g-a-b-o-t
GalaxyBot
Genieo
Getintent
Go!Zilla
GoZilla
Grabber
GrapeshotCrawler
HMView
HTTP::Lite
Haansoft
HaosouSpider
HonoluluBot
IDBTE4M
IDBot
IRLbot
Iblog
Image Fetch
Intelliseek
InterGET
JennyBot
JikeSpider
Joomla
Jorgee
JustView
Keyword Density
Lanshanbot
Larbin
Leap
LexiBot
Lftp
Libwhisker
LinkWalker
Linkbot
LinkpadBot
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Spider
Litemage_walker
MFC_Tear_Sample
MIDown tool
MIIxpc
MQQBrowser
MSIECrawler
Mag-Net
Magnet
Majestic SEO
MeanPath Bot
MegaIndex.ru
MicroMessenger
Mojeek
MolokaiBot
Msrabot
NICErsPRO
Navroad
Needle
Nessus
NetLyzer
NetZIP
Netvibes
NextGenSearchBot
Niki-bot
Nimbostratus
Nutch
Offline Navigator
OnCrawl
Page Analyzer
PageScorer
Papa Foto
Pavuk
Petalbot
Pi-Monster
Pimonster
Pixray
PleaseCrawl
ProPowerBot
ProWebWalker
QueryN Metasearch
RankingBot2
RealDownload
RebelMouse
RepoMonkey
RocketCrawler
Rogerbot
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SEOstats
ScanAlert
Scanbot
Screaming
ScrepyBot
Semrush
SeoSiteCheckup
SeobilityBot
Shodan
Siphon
SiteExplorer
SiteLockSpider
SiteSucker
Siteimprove
Snapbot
SocialRankIOBot
Sociscraper
Sogou web spider
Sosospider
SpaceBison
Spammen
Spanner
Spbot
Sucuri
SurveyBot
Teleport
Telesphoreo
Telesphorep
Thumbor
Toweyabot
Tracemyfile
True_Robot
Turingos
Turnitin
TurnitinBot
Typhoeus
V-BOT
VB Project
Vacuum
Vagabondo
VelenPublicWebCrawler
VidibleScraper
Virusdie
VoidEYE
WASALive-Bot
WBSearchBot
WISENutbot
WWW::Mechanize
WallpapersHD
WeSEE
Web Collage
Web Pix
WebEnhancer
WebFetch
WebImageCollector
WebLeacher
WebPix
WebReaper
WebWhacker
Webshag
WebsiteExtractor
WebsiteQuester
Whacker
Xaldon WebSpider
YoudaoBot
Zeus
Zitebot
ZoomBot
ZoominfoBot
adscanner
archive.org_bot
arquivo.pt
backlink-check
cognitiveseo
crawler4j
domainsproject.org
eCatch
instabid
internetVista monitor
isitwp.com
lwp-request
meanpathbot
mediawords
polaris version
probe-image-size
s1z.ru
satoristudio.net
scalaj-http
seobility
seoscanners
seostar
sitechecker.pro
sp_auditbot
spyfu
tAkeOut
trendiction.com
webmeup-crawler
webprosbot
x09Mozilla
zgrab