nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-07-04 10:09:20 +00:00

100 lines
1 KiB
Text

360Spider
Acunetix
Aipbot
AwarioBot
BBBike
Bandit
Bitacle
Brandwatch
Calculon
Craftbot
DataCha0s
DnBCrawler-Analytics
Dotbot
EMail Wolf
EasyDL
EyeNetIE
FDM
FyberSpider
Gigabot
Google-Extended
Intelliseek
InterGET
Internet Ninja
Iria
JetCar
Jyxobot
Kinza
Larbin
Libwhisker
LinkextractorPro
LinqiaRSSBot
MIIxpc
MTRobot
Mag-Net
Mass Downloader
MicroMessenger
Microsoft URL Control
NICErsPRO
Navroad
NetAnts
OutclicksBot
POE-Component-Client-HTTP
PageGrabber
Pandalytics
Piepmatz
PxBroker
RSSingBot
Rainbot
RankurBot
Re-re
Rogerbot
SEOprofiler
Scanbot
ScoutJet
Scrapy
SearchmetricsBot
Semrush
SentiBot
Snapbot
Sogou web spider
Sottopop
Stripper
Sucuri
Thumbor
Toata
Toweyabot
Turingos
TurnitinBot
Typhoeus
Voltron
WWW::Mechanize
Web Sucker
WebBandit
WebCollage
WebGo IS
WebPix
WebReaper
WebSucker
Webshag
WebsiteExtractor
Whatweb
WinHTTrack
Wonderbot
Woobot
Xaldon_WebSpider
YoudaoBot
ZoomBot
anthropic-ai
check1.exe
cohere-ai
crawler4j
dataforseobot
domainsproject.org
eCatch
magpie-crawler
s1z.ru
seoscanners
sitechecker.pro
tAkeOut
trendiction.com