nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-02-05 22:07:20 +00:00

250 lines
2.7 KiB
Text

01h4x.com
360Spider
404checker
80legs
AIBOT
ASPSeek
Aboundex
Aipbot
Alexibot
AlphaBot
Anarchie
Anarchy99
Asterias
Attach
BBBike
BDCbot
BackWeb
BacklinkCrawler
Bitacle
Blackboard
BlowFish
Bolt
BotALot
Brandprotect
Buddy
Bullseye
CCBot
CODE87
CSHttp
CensysInspect
ChinaClaw
Claritybot
Cloud mapping
Cocolyzebot
Cogentbot
Crawling at Home Project
CrunchBot
DSearch
DataCha0s
Devil
DigitalPebble
Discoverybot
DnBCrawler-Analytics
DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Domains Project
Dotbot
Download Wonder
Dragonfly
ECCP/1.0
EasyDL
EroCrawler
Evil
Exabot
Express WebPictures
ExtLinksBot
Extractor
Extreme Picture Finder
FHscan
Foobot
Freeuploader
FyberSpider
GT::WWW
GalaxyBot
GermCrawler
GetRight
GetWeb
Gigabot
Go-Ahead-Got-It
GridBot
HTTP::Lite
HTTrack
Havij
HonoluluBot
Id-search
Image Fetch
Image Sucker
IndeedBot
Indy Library
InfoNaviRobot
Intelliseek
IstellaBot
JetCar
JustView
Kozmosbot
LWP::Simple
Lanshanbot
Larbin
Leap
LeechFTP
Lftp
LibWeb
Likse
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
MIDown tool
MIIxpc
MJ12bot
MSIECrawler
MTRobot
Magnet
Mail.RU_Bot
Majestic12
MarkWatch
MeanPath Bot
Meanpathbot
MegaIndex.ru
Metauri
Microsoft URL Control
Moblie Safari
Mojeek
Mojolicious
MolokaiBot
Morfeus Fucking Scanner
Mozlila
NPbot
Name Intelligence
Navroad
Needle
Nessus
NetZIP
Netcraft
Nimbostratus
Nutch
OnCrawl
OpenLinkProfiler
OutclicksBot
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber
PageScorer
Papa Foto
PeoplePal
Picsearch
Piepmatz
Pimonster
Pixray
Pockey
ProPowerBot
Psbot
Pump
Quick-Crawler
RankFlex
Rankivabot
Re-re
Reaper
Recorder
Rogerbot
SEOlyticsCrawler
SISTRIX
SMTBot
SalesIntelligent
ScanAlert
Semrush
SemrushBot
SeoSiteCheckup
SeobilityBot
Seomoz
SiteLockSpider
SiteSnagger
Sitebeam
Siteimprove
SmartDownload
Snake
Snapbot
SocialRankIOBot
Sociscraper
Sogou web spider
SpaceBison
Spbot
Sqlworm
Sqworm
SuperHTTP
Surfbot
Teleport
Thumbor
TightTwatBot
Toata
Toweyabot
Trendiction
True_Robot
Turingos
Typhoeus
URLy Warning
VelenPublicWebCrawler
VoidEYE
Voltron
WASALive-Bot
WEBDAV
Web Fuck
Web Sauger
WebAuto
WebBandit
WebEnhancer
WebGo IS
WebLeacher
WebPix
WebStripper
WebSucker
WebZIP
Webalta
WebmasterWorldForumBot
Website Quester
WebsiteExtractor
Whack
Whacker
Whatweb
Who.is Bot
Wonderbot
Wprecon
Xaldon WebSpider
YoudaoBot
Zade
Zauba
Zermelo
Zeus
adscanner
archive.org_bot
backlink-check
clark-crawler
cognitiveseo
com.plumanalytics
crawler.feedback
crawler4j
domainsproject.org
facebookscraper
instabid
ips-agent
lwp-request
mediawords
muhstik-scan
netEstate NE Crawler
page scorer
plumanalytics
probe-image-size
ripz
seoscanners
sogouspider
spyfu
trendiction.de
ubermetrics-technologies.com
webgains-bot
webmeup-crawler
webpros.com
x09Mozilla
xpymep1.exe