nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2025-08-13 22:08:38 +00:00

100 lines
1.1 KiB
Text

AdsTxtCrawlerTP
Anarchy
BDFetch
BacklinkCrawler
Bolt
BotALot
Brandwatch
CCBot
CensysInspect
ChinaClaw
Claritybot
Crawling at Home Project
Curious
DSearch
DTS Agent
DataCha0s
Drip
EMail Siphon
EasyDL
Ecxi
EroCrawler
Firefox/7.0
FrontPage
GalaxyBot
GermCrawler
Gotit
HMView
HTTrack
Haansoft
Heritrix
Humanlinks
Iblog
Image Fetch
Image Sucker
IndeedBot
Information Security Team InfraSec Scanner
Iria
JOC Web Spider
Jetty
JustView
Libwhisker
Lightspeedsystems
MSIECrawler
MeanPath Bot
Meanpathbot
Mediatoolkitbot
Mister PiX
NICErsPRO
NetSpider
OnCrawl
Openvas
PECL::HTTP
PHPCrawl
PageAnalyzer
PageGrabber
Pi-Monster
Probethenet
Proximic
RankingBot
RealDownload
RepoMonkey
ScoutJet
SemanticJuice
Semrush
SeoCherryBot
Seomoz
Sosospider
SpankBot
Spider_Bot
Thumbor
Vagabondo
WBSearchBot
WPScan
WWW-Collector-E
WeSEE
Web Sucker
WebCopier
WebEnhancer
WebImageCollector
WebReaper
Webshag
Widow
Xaldon WebSpider
ZyBorg
archive.org_bot
cah.io.community
crawler4j
dataprovider
evc-batch
heritrix
linkfluence
openai
page scorer
pcBrowser
s1z.ru
satoristudio.net
scan.lol
seoscanners
sysscan
voyagerx.com