nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-12-31 22:07:05 +00:00

250 lines
2.5 KiB
Text

01h4x.com
360Spider
404checker
404enemy
80legs
ASPSeek
AfD-Verbotsverfahren
AhrefsBot
Aipbot
AllSubmitter
AlphaBot
Anthill
Aspiegel
AwarioSmartBot
BBBike
BDCbot
BDFetch
BLEXBot
BackDoorBot
Bandit
BetaBot
Bitacle
BlowFish
Boardreader
BuiltWith
BunnySlippers
CATExplorador
CCBot
CODE87
CSHttp
Calculon
CazoodleBot
CheeseBot
CherryPicker
Chlooe
Citoid
Claritybot
Cocolyzebot
Collector
Copyscape
CrazyWebCrawler
CrunchBot
Curious
DBLBot
DSearch
Deusu
Digincore
DigitalPebble
Disco
Discobot
DittoSpyder
DnyzBot
DomCopBot
DomainAppender
Dragonfly
Drip
ECCP/1.0
EMail Wolf
EroCrawler
Extractor
FHscan
FemtosearchBot
Firefox/7.0
FyberSpider
G-i-g-a-b-o-t
GetWeb
Go!Zilla
GrapeshotCrawler
GridBot
HTMLparser
Harvest
Humanlinks
IRLbot
IndeedBot
InfoNaviRobot
InterGET
Iria
Iskanie
JOC Web Spider
JamesBOT
Jbrofuzz
JikeSpider
JustView
Jyxobot
Kinza
LWP::Simple
Larbin
Leap
LeechGet
Libwhisker
LinqiaRSSBot
Lmspider
MJ12bot
MQQBrowser
Magnet
Mail.RU_Bot
Majestic SEO
MarkWatch
Mata Hari
MauiBot
Mb2345Browser
Metauri
Microsoft Data Access
Microsoft URL Control
Minefield
Moblie Safari
Mojeek
Mojolicious
Musobot
NPbot
Name Intelligence
Nameprotect
NearSite
Needle
Nessus
NetMechanic
NetZIP
NextGenSearchBot
Nibbler
Niki-bot
Nikto
NimbleCrawler
Ninja
Nutch
Octopus
Openfind
Openvas
OrangeBot
OutfoxBot
Page Analyzer
PageScorer
Pandalytics
Papa Foto
Pavuk
PeoplePal
Picscout
PleaseCrawl
Pockey
Probethenet
PxBroker
PyCurl
Quick-Crawler
RSSingBot
RankActive
RankFlex
RankingBot
Reaper
Recorder
RocketCrawler
SEOkicks
SEOprofiler
SalesIntelligent
ScanAlert
Scanbot
ScoutJet
ScrepyBot
SemanticJuice
SemrushBot
Site Sucker
SiteSnagger
Sitebeam
Siteimprove
SlySearch
Snoopy
Sociscraper
Sogou web spider
Spinn3r
SputnikBot
Sqlworm
Steeler
Sucker
SuperHTTP
Surfbot
Suzuran
Szukacz
T0PHackTeam
Thumbor
TightTwatBot
Titan
Toweyabot
TurnitinBot
Typhoeus
URLy Warning
URLy.Warning
UnisterBot
V-BOT
VB Project
VCI
VelenPublicWebCrawler
VeriCiteCrawler
Voil
Voltron
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
WallpapersHD
WeSEE
Web Collage
Web Fetch
Web Pix
WebCollage
WebCopier
WebFetch
WebFuck
WebLeacher
WebPix
WebSauger
WebStripper
WebSucker
WebWhacker
Webshag
Whacker
Who.is Bot
WinHTTrack
Wotbox
YoudaoBot
Zade
Zauba
Zermelo
ZumBot
ZyBorg
adscanner
archive.org_bot
cognitiveseo
com.plumanalytics
crawler4j
heritrix
internetVista monitor
ips-agent
lwp-request
meanpathbot
mediawords
muhstik-scan
netEstate NE Crawler
pcBrowser
polaris version
satoristudio.net
seobility
seoscanners
serpstatbot
sexsearcher
sitechecker.pro
siteripz
sogouspider
sp_auditbot
spyfu
tAkeOut
trendiction.com