nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-10-17 10:07:25 +00:00

100 lines
1.1 KiB
Text

360Spider
404checker
Anarchie
Ankit
Anthill
Attach
BackDoorBot
BatchFTP
Bigfoot
Bitacle
BunnySlippers
ChatGPT-User
ChinaClaw
Crawling at Home Project
CyotekWebCopy
Dragonfly
Ebingbong
EroCrawler
EyeNetIE
FDM
Firefox/7.0
Foobot
HTMLparser
Heritrix
Hloader
IndeedBot
Indy Library
Internet Ninja
JikeSpider
JustView
LieBaoFast
LinkWalker
LinqiaScrapeBot
Lmspider
Majestic SEO
MarkMonitor
MarkWatch
MicroMessenger
Microsoft Data Access
Moblie Safari
Mr.4x3
Navroad
NearSite
Nessus
Net Vampire
NetLyzer
NetSpider
Nmap
Nutch
Offline Explorer
Offline Navigator
OrangeSpider
OutclicksBot
Pavuk
ProWebWalker
Psbot
Quick-Crawler
RankActive
Rankivabot
Re-re
RedesScrapy
RocketCrawler
SalesIntelligent
Scrapy
Searchestate
SeekportBot
SeobilityBot
Seomoz
Siphon
SiteExplorer
Sitebeam
SmartDownload
Sottopop
SpankBot
Spbot
SputnikBot
Surfbot
TheNomad
TinyTestBot
Trendictionbot
Turnitin
WASALive-Bot
Web Sauger
WebImageCollector
WebWhacker
Webster
Widow
anthropic-ai
archive.org_bot
backlink-check
cah.io.community
crawler4j
dataforseobot
demandbase-bot
eCatch
heritrix
s1z.ru
seobility
sitechecker.pro
spyfu