nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-04-14 22:07:08 +00:00

250 lines
2.6 KiB
Text

404checker
80legs
ASPSeek
AiHitBot
AllSubmitter
Anarchy
Anarchy99
Ankit
Asterias
Attach
AwarioRssBot
BLEXBot
BackDoorBot
BackStreet
BackWeb
BacklinkCrawler
Badass
Bandit
BatchFTP
Battleztar Bazinga
BetaBot
Bigfoot
Blackboard
Blow
Brandwatch
Buck
Buddy
CATExplorador
CSHttp
CheTeam
CheeseBot
ChinaClaw
Citoid
Cliqzbot
Collector
CopyRightCheck
Copyscape
CrazyWebCrawler
Crescent
Curious
CyotekWebCopy
DIIbot
DSearch
DTS Agent
Deusu
Digincore
DigitalPebble
Disco
Discoverybot
DnyzBot
DomCopBot
DomainStatsBot
Dragonfly
EMail Siphon
EMail Wolf
EasyDL
Ecxi
EirGrabber
Express WebPictures
ExtLinksBot
ExtractorPro
EyeNetIE
FDM
FemtosearchBot
Fimap
FlashGet
Foobot
FrontPage
Fyrebot
GT::WWW
GermCrawler
Go!Zilla
Go-Ahead-Got-It
GoZilla
GrabNet
GrapeshotCrawler
GridBot
HMView
HTTP::Lite
Harvest
Havij
Heritrix
Hloader
HonoluluBot
Humanlinks
IDBTE4M
IDBot
Id-search
IlseBot
IndeedBot
InfoNaviRobot
Internet Ninja
Iria
Iskanie
IstellaBot
JikeSpider
Jyxobot
Kozmosbot
Lanshanbot
Larbin
LeechFTP
LeechGet
Lftp
Lightspeedsystems
Likse
LinkWalker
LinkextractorPro
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lmspider
MSFrontPage
Majestic12
MarkWatch
Masscan
Mata Hari
MauiBot
MeanPath Bot
Meanpathbot
Metauri
MicroMessenger
Microsoft URL Control
Mister PiX
Mojolicious
MolokaiBot
Morfeus Fucking Scanner
Navroad
Nessus
NetLyzer
NimbleCrawler
Nmap
Not
Offline Explorer
Offline Navigator
OnCrawl
OpenLinkProfiler
OpenVAS
Openfind
OrangeBot
OutclicksBot
OutfoxBot
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
PageScorer
Papa Foto
Petalbot
Picscout
Piepmatz
Pixray
PleaseCrawl
Pockey
ProPowerBot
PxBroker
Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankFlex
RankingBot
Recorder
RocketCrawler
Rogerbot
SISTRIX
SMTBot
SalesIntelligent
Scrapy
ScreenerBot
SentiBot
Seomoz
Site Sucker
SiteLockSpider
Sitebeam
Siteimprove
Sitevigil
SmartDownload
Snake
Snapbot
Sosospider
Sottopop
Spammen
Spbot
Sqlworm
Sqworm
Steeler
Sucuri
SuperHTTP
Surfbot
SurveyBot
Szukacz
T0PHackTeam
TeleportPro
The Intraformant
Trendiction
Turingos
URLy Warning
UnisterBot
Vagabondo
VeriCiteCrawler
VidibleScraper
Voil
WISENutbot
WWW-Collector-E
Web Auto
Web Collage
Web Enhancer
Web Sauger
Web Sucker
WebCopier
WebFuck
WebImageCollector
WebReaper
WebWhacker
WebsiteQuester
Webster
Whack
Wotbox
Wprecon
Xaldon WebSpider
Xenu
Zitebot
ZmEu
ZyBorg
adscanner
backlink-check
clark-crawler
crawl.sogou.com
dataforseo.com
domainsproject.org
facebookscraper
lwp-trivial
meanpathbot
mediawords
oBot
pcBrowser
ripz
scalaj-http
scan.lol
seobility
seocompany.store
siteripz
trendiction.de
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
webpros.com
xpymep1.exe
zgrab