nginx-ultimate-bad-bot-blocker/.dev-tools/test_units/random-bots-for-whitelist-test.list
2019-11-06 21:21:01 +02:00

250 lines
2.6 KiB
Text

404enemy
Aboundexbot
ADmantX
AhrefsBot
AllSubmitter
AlphaBot
archive.org_bot
ASPSeek
Asterias
AwarioSmartBot
backlink-check
BDCbot
BDFetch
BuiltWith
Bullseye
CATExplorador
CCBot
Cegbfeieh
ChinaClaw
Cliqzbot
coccocbot-web
Cogentbot
cognitiveseo
com.plumanalytics
Cosmos
Craftbot
crawler4j
Crescent
CrunchBot
CSHttp
Curious
Demon
Devil
DigitalPebble
Discobot
Dispatch
DomainAppender
Dragonfly
DSearch
EirGrabber
evc-batch
Evil
Extractor
ExtractorPro
Extreme Picture Finder
EyeNetIE
FDM
FemtosearchBot
Firefox/7.0
Foobot
FyberSpider
Fyrebot
GermCrawler
GetRight
GetWeb
Go-Ahead-Got-It
GoZilla
Go!Zilla
GrapeFX
GrapeshotCrawler
GT::WWW
HaosouSpider
Harvest
Havij
HEADMasterSEO
heritrix
Hloader
HMView
HTTrack
Image Fetch
Indy Library
instabid
InterGET
Iskanie
Jbrofuzz
JennyBot
JOC Web Spider
Joomla
JustView
Jyxobot
Kenjin Spider
Keyword Density
Kozmosbot
Lanshanbot
LeechGet
LibWeb
Linkdexbot
LinkScan
LinksManager
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Lmspider
LNSpiderguy
lwp-request
LWP::Simple
Mag-Net
magpie-crawler
Mail.RU_Bot
Majestic12
Majestic SEO
Majestic-SEO
Masscan
Mata Hari
meanpathbot
MeanPath Bot
Mediatoolkitbot
mediawords
MFC_Tear_Sample
Microsoft Data Access
MIIxpc
MJ12bot
Mojeek
Mojolicious
MSIECrawler
Name Intelligence
Navroad
Nessus
NetAnts
Netcraft
netEstate NE Crawler
Nettrack
NextGenSearchBot
NimbleCrawler
Nimbostratus
Nutch
oBot
Octopus
Offline Explorer
OnCrawl
Openfind
OpenLinkProfiler
OrangeBot
OutfoxBot
page scorer
Pavuk
pcBrowser
Picscout
PictureFinder
Pi-Monster
Pixray
PleaseCrawl
plumanalytics
ProPowerBot
Psbot
PyCurl
QueryN Metasearch
Quick-Crawler
RankActive
RealDownload
Reaper
RebelMouse
ReGet
RepoMonkey
RocketCrawler
Rogerbot
RSSingBot
SBIder
ScanAlert
Scanbot
scan.lol
ScoutJet
ScreenerBot
SearchmetricsBot
Semrush
SemrushBot
SEOlyticsCrawler
Seomoz
seoscanners
SEOstats
Shodan
SiteExplorer
Siteimprove
SiteSnagger
SmartDownload
Snake
Snapbot
SocialRankIOBot
Sogou web spider
Sosospider
Sottopop
Spammen
sp_auditbot
SputnikBot
spyfu
Sqlmap
Sqlworm
Sqworm
Stripper
SuperBot
Surfbot
SurveyBot
Suzuran
sysscan
T8Abot
Teleport
TeleportPro
The Intraformant
Titan
Toweyabot
trendiction.de
Turingos
TwengaBot
Twice
Typhoeus
Vagabondo
VB Project
VCI
VidibleScraper
VoidEYE
Voltron
WallpapersHD
WASALive-Bot
WBSearchBot
Webalta
WebCollage
Web Collage
Web Enhancer
WebGo IS
WebImageCollector
WebPix
WebReaper
WebSauger
WebsiteExtractor
WebsiteQuester
Webster
WebStripper
WebWhacker
WebZIP
WeSEE
Whacker
Whatweb
WinHTTrack
WISENutbot
Woobot
Wotbox
WPScan
WWW-Collector-E
WWWOFFLE
Xaldon WebSpider
Xaldon_WebSpider
Xenu
Zade
Zauba
zauba.io
Zermelo
Zitebot