nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2025-08-06 22:08:29 +00:00

100 lines
1.1 KiB
Text

ADmantX
ALittle Client
ASPSeek
AhrefsBot
AiHitBot
Anarchie
Atomseobot
Attach
AwarioSmartBot
BDFetch
Backlink-Ceck
BacklinkCrawler
Barkrowler
Black Hole
BlackWidow
Blackboard
Blow
Boardreader
Bytespider
ClaudeBot
Copyscape
DataCha0s
DatabaseDriverMysqli
Domains Project
EasyDL
Express WebPictures
Extractor
Foobot
FrontPage
Fyrebot
G-i-g-a-b-o-t
GT::WWW
HEADMasterSEO
IDBTE4M
Image Fetch
IndeedBot
InterGET
JOC Web Spider
JamesBOT
Jbrofuzz
Keyword Density
LWP::Simple
Larbin
LeechGet
LexiBot
Linkbot
Litemage_walker
Masscan
Mb2345Browser
Microsoft URL Control
Navroad
NearSite
Needle
NimbleCrawler
Nutch
Offline Explorer
OutclicksBot
OutfoxBot
Piepmatz
ProWebWalker
PyCurl
RedesScrapy
SEOstats
ScrepyBot
SemrushBot-BA
SeobilityBot
SmartDownload
Sogou web spider
Swiftbot
Turingos
TwengaBot
URLy.Warning
V-BOT
Web Enhancer
WebBandit
WebCollage
WebEnhancer
WebReaper
Webshag
Website Quester
Webster
Wonderbot
Xaldon WebSpider
Zeus
ZoominfoBot
ZumBot
anthropic-ai
crawler.feedback
crawler4j
everyfeed-spider
internetVista monitor
linkfluence
meanpathbot
omgili
openai.com
scalaj-http
seobility
seocompany.store
sitechecker.pro
sysscan