nginx-ultimate-bad-bot-blocker/.dev-tools/test_units/random-bots-for-whitelist-test.list
2020-12-04 11:13:21 +02:00

250 lines
2.6 KiB
Text

360Spider
80legs
Aboundex
Aboundexbot
AfD-Verbotsverfahren
Aipbot
Ankit
Anthill
arquivo.pt
arquivo-web-crawler
Aspiegel
ASPSeek
BacklinkCrawler
BackWeb
BDFetch
BlackWidow
BLEXBot
BlowFish
Boardreader
Bolt
BuiltWith
Bullseye
BunnySlippers
BuzzSumo
CCBot
check1.exe
CheeseBot
CheTeam
ChinaClaw
Chlooe
Cloud mapping
coccocbot-web
CODE87
CopyRightCheck
Copyscape
Cosmos
crawler.feedback
Curious
Custo
CyotekWebCopy
DataCha0s
DBLBot
demandbase-bot
Deusu
Digincore
Dirbuster
Discobot
DittoSpyder
DomainAppender
DomainCrawler
domainsproject.org
Download Wonder
Drip
DSearch
Ebingbong
eCatch
EirGrabber
evc-batch
Evil
Exabot
Express WebPictures
ExtLinksBot
Extreme Picture Finder
Ezooms
facebookscraper
FlashGet
Flunky
Foobot
Freeuploader
FrontPage
FyberSpider
Genieo
GermCrawler
Gigablast
Go-Ahead-Got-It
Grabber
GrabNet
Grafula
GrapeFX
GrapeshotCrawler
GridBot
HaosouSpider
HEADMasterSEO
HTMLparser
IDBot
IDBTE4M
Image Sucker
IndeedBot
Indy Library
InfoNaviRobot
InfoTekies
Intelliseek
internetVista monitor
ips-agent
JennyBot
JikeSpider
JOC Web Spider
Jorgee
Kenjin Spider
Keyword Density
LexiBot
Lftp
LibWeb
LinkpadBot
LinqiaMetadataDownloaderBot
Lipperhey Spider
Litemage_walker
lwp-request
LWP::Simple
lwp-trivial
Magnet
magpie-crawler
Mail.RU_Bot
Majestic12
Majestic-SEO
MarkMonitor
Mass Downloader
Mata Hari
MauiBot
MeanPath Bot
Metauri
MicroMessenger
Microsoft URL Control
Mister PiX
Mojeek
Mozlila
MQQBrowser
Mr.4x3
MSIECrawler
Navroad
NearSite
NetAnts
netEstate NE Crawler
NetLyzer
NetMechanic
NetSpider
Nettrack
Netvibes
NetZIP
NextGenSearchBot
Niki-bot
Nikto
NimbleCrawler
NPbot
Nutch
Octopus
OpenLinkProfiler
OPPO A33
OutfoxBot
PageAnalyzer
PageGrabber
Panscient
PHPCrawl
PictureFinder
Pimonster
Pockey
POE-Component-Client-HTTP
polaris version
ProPowerBot
QueryN Metasearch
RankActive
Recorder
RepoMonkey
s1z.ru
SalesIntelligent
satoristudio.net
Scanbot
Searchestate
SearchmetricsBot
Semrush
SemrushBot
SEOlyticsCrawler
Seomoz
seoscanners
SEOstats
Sitebeam
sitechecker.pro
SiteLockSpider
SmartDownload
Snapbot
Sociscraper
Sogou web spider
Sottopop
SpankBot
Spanner
sp_auditbot
SputnikBot
spyfu
Sqworm
Stripper
Sucuri
SuperBot
SurveyBot
Swiftbot
T8Abot
tAkeOut
TeleportPro
Telesoft
Telesphoreo
TightTwatBot
Toata
True_Robot
Turingos
Turnitin
TurnitinBot
TwengaBot
Twice
Typhoeus
Upflow
URLy Warning
URLy.Warning
Vagabondo
VB Project
Virusdie
Voil
Wallpapers/3.0
WallpapersHD
WASALive-Bot
WebAuto
WebCollage
WebCopier
WEBDAV
Web Fetch
WebFuck
Web Fuck
WebGo IS
WebmasterWorldForumBot
Web Pix
WebReaper
Webshag
WebsiteExtractor
Webster
WebWhacker
WebZIP
WeSEE
Whacker
Who.is Bot
WISENutbot
Wonderbot
WPScan
WWW-Collector-E
WWW::Mechanize
x09Mozilla
Xaldon_WebSpider
YoudaoBot
Zade
Zauba
Zeus
Zitebot
ZmEu