nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-09-30 10:09:30 +00:00

250 lines
2.6 KiB
Text

01h4x.com
360Spider
404checker
80legs
ADmantX
ALittle Client
ASPSeek
Aipbot
Alexibot
Asterias
Attach
AwarioRssBot
AwarioSmartBot
BBBike
BDFetch
BLEXBot
BackStreet
BackWeb
Backlink-Ceck
Bandit
Barkrowler
Bitacle
Blackboard
Brandprotect
Buddy
BuiltWith
Bullseye
BuzzSumo
CATExplorador
CensysInspect
CheTeam
Cogentbot
Copier
Copyscape
Cosmos
CrazyWebCrawler
Crescent
Curious
Custo
DBLBot
DSearch
Deusu
Digincore
DittoSpyder
DomainCrawler
Domains Project
Dotbot
Download Wonder
Dragonfly
Drip
EirGrabber
Evil
Exabot
ExtractorPro
FemtosearchBot
Fimap
Firefox/7.0
Flunky
FrontPage
Fuzz
FyberSpider
GalaxyBot
Genieo
Gigabot
Go!Zilla
GrabNet
Grafula
GrapeshotCrawler
Havij
Heritrix
Hloader
Humanlinks
HybridBot
IDBTE4M
IDBot
IRLbot
Image Fetch
Indy Library
Internet Ninja
InternetSeer
IstellaBot
JamesBOT
Jetty
JikeSpider
Jorgee
Kenjin Spider
Larbin
LeechGet
Lftp
LieBaoFast
Linkbot
Linkdexbot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Litemage_walker
Lmspider
MFC_Tear_Sample
MIIxpc
MJ12bot
MQQBrowser
MTRobot
Mag-Net
Majestic SEO
Majestic-SEO
Majestic12
Masscan
Mb2345Browser
Meanpathbot
MegaIndex.ru
Mister PiX
Mojeek
Mojolicious
Mozlila
Musobot
Nameprotect
Navroad
NetAnts
NetMechanic
NetSpider
Nettrack
Netvibes
NextGenSearchBot
Nibbler
Niki-bot
Nikto
Nuclei
Nutch
Offline Navigator
OpenLinkProfiler
POE-Component-Client-HTTP
PageAnalyzer
PageScorer
Panscient
Pavuk
PeoplePal
ProPowerBot
Probethenet
Psbot
QueryN Metasearch
RankFlex
Rankivabot
RankurBot
Re-re
RealDownload
RebelMouse
Recorder
RocketCrawler
Rogerbot
SBIder
SEOkicks
SEOlyticsCrawler
SEOprofiler
SEOstats
SMTBot
SalesIntelligent
ScoutJet
ScreenerBot
Searchestate
SeobilityBot
Shodan
Siphon
SiteLockSpider
SiteSucker
Siteimprove
SlySearch
Snake
Sociscraper
Sosospider
Sottopop
Spanner
Sucker
Sucuri
Suzuran
Swiftbot
Szukacz
T0PHackTeam
T8Abot
Teleport
Telesoft
Telesphorep
The Intraformant
Toata
Trendiction
Trendictionbot
True_Robot
Twice
Typhoeus
URLy Warning
V-BOT
VB Project
VelenPublicWebCrawler
VeriCiteCrawler
Voltron
WASALive-Bot
WBSearchBot
WEBDAV
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0
Web Enhancer
Web Sauger
Web Sucker
WebCollage
WebFetch
WebFuck
WebGo IS
WebLeacher
WebPix
WebReaper
WebSucker
WebWhacker
WebZIP
Webshag
Widow
Wonderbot
Xaldon WebSpider
Zade
Zitebot
ZmEu
ZoominfoBot
ZumBot
archive.org_bot
clark-crawler
cognitiveseo
crawler.feedback
instabid
internetVista monitor
isitwp.com
lwp-trivial
magpie-crawler
muhstik-scan
oBot
page scorer
pcBrowser
plumanalytics
ripz
s1z.ru
satoristudio.net
scalaj-http
scan.lol
serpstatbot
sitechecker.pro
siteripz
sp_auditbot
tAkeOut
trendiction.de
x09Mozilla
zauba.io