nginx-ultimate-bad-bot-blocker/.dev-tools/test_units/random-bots-for-whitelist-test.list
2019-06-28 21:30:04 +02:00

250 lines
2.6 KiB
Text

360Spider
80legs
Abonti
ADmantX
AIBOT
Aipbot
AllSubmitter
Anarchie
archive.org_bot
backlink-check
BacklinkCrawler
Badass
Bandit
Barkrowler
Battleztar Bazinga
BBBike
BDFetch
Bigfoot
Bitacle
Blackboard
BlackWidow
Blow
Boardreader
Brandprotect
Buddy
BuiltBotTough
Bullseye
BuzzSumo
CATExplorador
CCBot
CheeseBot
CherryPicker
ChinaClaw
Chlooe
Claritybot
Cliqzbot
Collector
com.plumanalytics
CopyRightCheck
Cosmos
Craftbot
crawl.sogou.com
Crescent
CrunchBot
Custo
DatabaseDriverMysqli
DBLBot
demandbase-bot
Digincore
DigitalPebble
Dirbuster
Discobot
DomainAppender
DomainSigmaCrawler
Dragonfly
Drip
DSearch
EasyDL
Ebingbong
EirGrabber
EMail Siphon
EMail Wolf
evc-batch
Evil
Express WebPictures
ExtLinksBot
Extreme Picture Finder
facebookscraper
FHscan
Fimap
Flunky
Genieo
Getintent
GetRight
GetWeb
Gigablast
Gigabot
G-i-g-a-b-o-t
Go-Ahead-Got-It
GrabNet
Grafula
GrapeshotCrawler
GridBot
GT::WWW
HaosouSpider
Havij
HEADMasterSEO
HMView
HTMLparser
HTTP::Lite
HTTrack
HybridBot
IDBot
Image Fetch
InfoNaviRobot
Internet Ninja
internetVista monitor
ips-agent
Iria
IRLbot
Iskanie
JamesBOT
JennyBot
JetCar
Jetty
JikeSpider
JOC Web Spider
Joomla
Jyxobot
Kenjin Spider
Keyword Density
LeechFTP
LeechGet
Lftp
Libwhisker
Lightspeedsystems
Linkdexbot
LinkextractorPro
LinkScan
LinqiaMetadataDownloaderBot
Lipperhey Spider
Lmspider
LNSpiderguy
lwp-request
LWP::Simple
lwp-trivial
Magnet
Mag-Net
magpie-crawler
Mass Downloader
MauiBot
Meanpathbot
mediawords
Microsoft URL Control
Mr.4x3
MSIECrawler
Msrabot
muhstik-scan
Name Intelligence
Navroad
NearSite
Needle
Nessus
netEstate NE Crawler
NetMechanic
Net Vampire
NetZIP
Nmap
NPbot
Offline Navigator
OnCrawl
Openvas
OpenVAS
OutclicksBot
PageGrabber
Panscient
Pavuk
PeoplePal
PHPCrawl
Picsearch
Pixray
PleaseCrawl
POE-Component-Client-HTTP
ProPowerBot
Psbot
Pump
PxBroker
PyCurl
QueryN Metasearch
RankActive
RankFlex
RankingBot
RankurBot
Reaper
RebelMouse
ReGet
Ripper
RocketCrawler
ScanAlert
Scanbot
Scrapy
Semrush
SEOkicks
SEOlyticsCrawler
Seomoz
SeoSiteCheckup
serpstatbot
sexsearcher
SISTRIX
SiteExplorer
SiteLockSpider
SiteSucker
Site Sucker
Snake
Snoopy
Sociscraper
Sogou web spider
Sosospider
SpaceBison
SpankBot
SputnikBot
Sucker
Sucuri
SuperHTTP
SurveyBot
Suzuran
Swiftbot
tAkeOut
TeleportPro
Telesphoreo
TightTwatBot
Titan
Toweyabot
Trendiction
trendiction.de
True_Robot
Turingos
TurnitinBot
UnisterBot
URLy Warning
URLy.Warning
VB Project
VeriCiteCrawler
Virusdie
Web Auto
WebCollage
Web Collage
WebCopier
WEBDAV
Web Enhancer
Web Fuck
WebGo IS
Web Pix
WebSauger
Web Sauger
WebsiteExtractor
Webster
WebSucker
WiseGuys Robot
WISENutbot
Wonderbot
Wprecon
WWW-Collector-E
WWW-Mechanize
WWWOFFLE
x09Mozilla
Xaldon_WebSpider
YoudaoBot
zauba.io
Zermelo