nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-07-31 22:04:48 +00:00

100 lines
1 KiB
Text

AIBOT
AhrefsBot
Atomseobot
BDFetch
Blow
Buck
CATExplorador
CazoodleBot
Citoid
ClaudeBot
Copier
Custo
Demon
Discoverybot
ECCP/1.0
Ecxi
Exabot
Extreme Picture Finder
FemtosearchBot
FlashGet
GridBot
HTTP::Lite
Hloader
Jetty
Jyxobot
Keybot Translation-Search-Machine
Lanshanbot
LibWeb
Linkbot
LinkextractorPro
Lipperhey
Lmspider
MFC_Tear_Sample
MSFrontPage
MSIECrawler
Majestic SEO
MarkWatch
Masscan
Mediatoolkitbot
MegaIndex.ru
Microsoft URL Control
Msrabot
Navroad
Nessus
NetLyzer
NetZIP
Nibbler
Niki-bot
Nmap
OnCrawl
OpenLinkProfiler
PageScorer
Panscient
Papa Foto
Pavuk
Psbot
Pump
Rainbot
RankActiveLinkBot
Scrapy
Screaming
ScreenerBot
SeekportBot
SiteSucker
Sitebeam
SmartDownload
Sociscraper
Sogou web spider
Spider_Bot/3.0
SputnikBot
Sucker
SurveyBot
Suzuran
T8Abot
Titan
Toweyabot
Trendiction
TurnitinBot
TwengaBot
UnisterBot
WEBDAV
WISENutbot
WWW-Collector-E
Web Collage
WebBandit
WebFetch
WebLeacher
Wotbox
Xaldon WebSpider
Zade
arquivo-web-crawler
backlink-check
dataforseo.com
openai
page scorer
polaris version
serpstatbot
voyagerx.com
x22Mozilla
xpymep1.exe