nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-09-02 22:07:26 +00:00

250 lines
2.5 KiB
Text

01h4x.com
360Spider
404enemy
ALittle Client
ASPSeek
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AiHitBot
AllSubmitter
Ankit
Anthill
Apexoo
Aspiegel
Asterias
Attach
AwarioSmartBot
BBBike
BLEXBot
BackDoorBot
Barkrowler
BatchFTP
BlackWidow
Blow
Bolt
BotALot
Buddy
BuiltBotTough
BunnySlippers
CCBot
CODE87
CheeseBot
ChinaClaw
Cliqzbot
CopyRightCheck
Copyscape
Crawling at Home Project
Curious
Custo
DTS Agent
Demon
Devil
Disco
Discoverybot
Dispatch
DittoSpyder
DomCopBot
DomainStatsBot
Dragonfly
Drip
EroCrawler
Extractor
FDM
Firefox/7.0
FrontPage
Fyrebot
G-i-g-a-b-o-t
GT::WWW
Genieo
GermCrawler
GetRight
GetWeb
Getintent
Go-Ahead-Got-It
GoZilla
Gotit
GrabNet
GrapeFX
HTMLparser
Harvest
Havij
IDBTE4M
IlseBot
Image Fetch
Image Sucker
IndeedBot
InterGET
Internet Ninja
IstellaBot
JennyBot
Jorgee
Keyword Density
Kinza
LWP::Simple
Leap
LeechGet
LieBaoFast
Likse
LinkWalker
Linkbot
LinkextractorPro
LinkpadBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Ltx71
MJ12bot
MQQBrowser
MSIECrawler
MTRobot
Mail.RU_Bot
Majestic SEO
MarkWatch
MauiBot
Meanpathbot
MegaIndex.ru
Metauri
MicroMessenger
Mojeek
Mojolicious
Msrabot
Musobot
NPbot
Name Intelligence
Needle
NetLyzer
NetMechanic
NetZIP
Netvibes
Niki-bot
NimbleCrawler
Ninja
Nuclei
Offline Navigator
OnCrawl
OpenVAS
Openfind
OrangeSpider
PECL::HTTP
PageGrabber
Papa Foto
Pavuk
Pi-Monster
Picscout
Picsearch
PictureFinder
Pixray
ProPowerBot
PxBroker
PyCurl
QueryN Metasearch
RankActive
RankingBot
Rankivabot
RealDownload
RebelMouse
Recorder
RedesScrapy
RepoMonkey
Ripper
SEOlyticsCrawler
SEOprofiler
SISTRIX
SMTBot
SalesIntelligent
Searchestate
Semrush
SemrushBot
SeobilityBot
Seomoz
SiteExplorer
SiteLockSpider
SiteSnagger
SlySearch
Sosospider
Sottopop
SpankBot
Spbot
Spinn3r
Steeler
SuperHTTP
Surfbot
Suzuran
Szukacz
Teleport
Telesoft
TheNomad
TightTwatBot
Titan
Tracemyfile
Trendictionbot
True_Robot
Turingos
TurnitinBot
Twice
Typhoeus
UnisterBot
Upflow
VB Project
Vacuum
Vagabondo
VidibleScraper
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
Web Auto
Web Pix
Web Sauger
WebAuto
WebBandit
WebCollage
WebFuck
WebGo IS
WebImageCollector
WebPix
WebStripper
WebSucker
WebZIP
Webalta
WebsiteExtractor
WebsiteQuester
Whack
Whatweb
Widow
WinHTTrack
Wprecon
Xenu
Zeus
Zitebot
ZmEu
ZyBorg
archive.org_bot
backlink-check
cah.io.community
coccocbot-web
cognitiveseo
dataforseo.com
domainsproject.org
eCatch
facebookscraper
ips-agent
lwp-trivial
meanpathbot
muhstik-scan
netEstate NE Crawler
plumanalytics
polaris version
scan.lol
seobility
serpstatbot
sexsearcher
sp_auditbot
sysscan
tAkeOut
trendiction.com
ubermetrics-technologies.com
x22Mozilla
xpymep1.exe
zauba.io