nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2025-08-13 10:13:31 +00:00

100 lines
1.1 KiB
Text

Aboundex
Alexibot
Aliyun
AlphaBot
Anarchy
Ankit
Anthill
Apexoo
AwarioRssBot
AwarioSmartBot
BackDoorBot
BacklinksExtendedBot
Bandit
BatchFTP
Bitacle
Buddy
BuiltBotTough
Calculon
Cliqzbot
CopyRightCheck
Curious
Dirbuster
Discoverybot
DnBCrawler-Analytics
DnyzBot
EMail Wolf
Ecxi
Flunky
GT::WWW
GalaxyBot
GeedoProductSearch
Getintent
Go!Zilla
GoZilla
IDBTE4M
Image Sucker
InfoNaviRobot
InterGET
InternetSeer
JOC Web Spider
JustView
LeechFTP
LexiBot
LibWeb
Linkbot
LinkextractorPro
LinqiaMetadataDownloaderBot
Lipperhey
Mail.RU_Bot
Mr.4x3
NICErsPRO
Name Intelligence
Nutch
Offline Explorer
OrangeBot
Page Analyzer
PageAnalyzer
PageScorer
Petalbot
Pi-Monster
PictureFinder
Psbot
Quick-Crawler
RankActiveLinkBot
RankFlex
RebelMouse
RedesScrapy
Ripper
SalesIntelligent
SemrushBot-FT
SemrushBot-OCOB
Seomoz
Shodan
Sitevigil
SocialRankIOBot
Spbot
Swiftbot
Titan
Trendictionbot
Twice
VidibleScraper
Virusdie
WBSearchBot
WWW-Collector-E
WWW-Mechanize
Web Auto
WebEnhancer
WebImageCollector
WebsiteExtractor
ZoomBot
ZumBot
archive.org_bot
crawler4j
domainsproject.org
evc-batch
everyfeed-spider
heritrix
imagesift.com
magpie-crawler
siteripz