nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2025-07-03 22:08:08 +00:00

100 lines
1 KiB
Text

Alexibot
Anarchy99
Asterias
BlowFish
Boardreader
BotALot
Buck
Bytespider
CCBot
ClaudeBot
Cocolyzebot
Cosmos
Craftbot
Crawling at Home Project
Crescent
DBLBot
DSearch
Dirbuster
EroCrawler
ExtLinksBot
ExtractorPro
GermCrawler
Go-Ahead-Got-It
HTMLparser
HTTP::Lite
Haansoft
HaosouSpider
Intelliseek
Internet Ninja
Iskanie
Jorgee
Lanshanbot
LinkScan
LinkextractorPro
LinkpadBot
Lipperhey
MJ12bot
MQQBrowser
MSFrontPage
Mag-Net
Nessus
NetMechanic
PageScorer
PageThing.com
PeoplePal
Picscout
Picsearch
Pixray
RankActiveLinkBot
RankurBot
Rogerbot
ScanAlert
ScoutJet
Screaming
Semrush
SemrushBot
SemrushBot-SI
SemrushBot-SWA
SenutoBot
Shodan
Snake
Sosospider
Sottopop
Steeler
Suzuran
Telesphoreo
True_Robot
Turnitin
Vacuum
VelenPublicWebCrawler
VidibleScraper
VoidEYE
Voltron
WeSEE
Web Fetch
WebAuto
WebCopier
WebLeacher
Webshag
Website Quester
Who.is Bot
Woobot
Wprecon
YaK
YoudaoBot
Zermelo
ZoominfoBot
awario.com
eCatch
evc-batch
facebookscraper
heritrix
linkdexbot
linkfluence
magpie-crawler
oBot
openai.com
seoscanners
siteripz
sogouspider