nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-03-10 10:09:25 +00:00

250 lines
2.6 KiB
Text

01h4x.com
360Spider
404enemy
80legs
ADmantX
AIBOT
Abonti
Aboundex
Aboundexbot
AhrefsBot
Alexibot
AllSubmitter
Alligator
AlphaBot
Apexoo
Attach
AwarioSmartBot
BackDoorBot
Backlink-Ceck
BacklinkCrawler
BatchFTP
BetaBot
BlackWidow
Blow
BlowFish
Brandwatch
BunnySlippers
CATExplorador
CCBot
CODE87
CSHttp
Cegbfeieh
CheeseBot
CherryPicker
Citoid
Cocolyzebot
Copier
Craftbot
Crawling at Home Project
Crescent
DTS Agent
DataCha0s
Devil
Disco
Discobot
DomainSigmaCrawler
DomainStatsBot
Domains Project
Dotbot
Dragonfly
ECCP/1.0
EroCrawler
ExtLinksBot
ExtractorPro
Ezooms
Fimap
Foobot
Freeuploader
FrontPage
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo
GetRight
Gigabot
Go-Ahead-Got-It
Gotit
Grabber
Grafula
GrapeshotCrawler
GridBot
HaosouSpider
Harvest
Havij
Heritrix
Humanlinks
HybridBot
IDBTE4M
IDBot
Iblog
Id-search
IlseBot
Image Fetch
Image Sucker
Indy Library
InfoTekies
Internet Ninja
InternetSeer
IstellaBot
JamesBOT
Jetty
Kenjin Spider
Kinza
Leap
LeechGet
LexiBot
LibWeb
Lightspeedsystems
Linkbot
Linkdexbot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
Litemage_walker
MSFrontPage
Majestic12
Masscan
MauiBot
MeanPath Bot
MegaIndex.ru
Microsoft URL Control
Minefield
Mister PiX
Mojeek
MolokaiBot
Msrabot
Navroad
Net Vampire
NetSpider
NetZIP
NextGenSearchBot
Nutch
OrangeBot
OrangeSpider
PHPCrawl
POE-Component-Client-HTTP
PageScorer
Picsearch
Pimonster
Pixray
ProWebWalker
Probethenet
PxBroker
PyCurl
QueryN Metasearch
RankurBot
RebelMouse
Recorder
RepoMonkey
RocketCrawler
SISTRIX
ScoutJet
ScreenerBot
ScrepyBot
Semrush
SentiBot
Site Sucker
SiteSucker
Sitebeam
Sitevigil
SlySearch
Snapbot
Sottopop
SpankBot
SputnikBot
Sqlmap
Sqlworm
Sqworm
Stripper
Sucker
Sucuri
SuperHTTP
SurveyBot
Szukacz
T0PHackTeam
The Intraformant
Thumbor
Toata
Tracemyfile
Turingos
Turnitin
TurnitinBot
TwengaBot
Twice
Typhoeus
URLy Warning
Upflow
Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
Voil
Voltron
WISENutbot
WPScan
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
Wallpapers/3.0
WallpapersHD
Web Auto
Web Fetch
Web Fuck
Web Sauger
WebCollage
WebFetch
WebFuck
WebImageCollector
WebLeacher
WebPix
WebReaper
WebSauger
WebStripper
WebZIP
Webalta
WebmasterWorldForumBot
Webshag
Website Quester
WebsiteExtractor
WebsiteQuester
Webster
Whack
Who.is Bot
Widow
Wonderbot
Woobot
Xaldon WebSpider
Zade
Zermelo
Zitebot
ZoominfoBot
ZumBot
arquivo-web-crawler
com.plumanalytics
crawl.sogou.com
crawler.feedback
crawler4j
domainsproject.org
evc-batch
facebookscraper
instabid
magpie-crawler
meanpathbot
oBot
pcBrowser
ripz
satoristudio.net
scalaj-http
seobility
seocompany.store
seostar
sexsearcher
sitechecker.pro
sysscan
trendiction.com
webgains-bot
webmeup-crawler
webprosbot
zauba.io
zgrab