nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2023-10-28 10:07:04 +00:00

100 lines
1 KiB
Text

80legs
ALittle Client
Aspiegel
BLEXBot
BatchFTP
Bolt
Buddy
CheTeam
Citoid
Cocolyzebot
Copier
Cosmos
Deusu
Digincore
DigitalPebble
Discobot
Drip
Express WebPictures
ExtractorPro
HTTrack
Indy Library
InternetSeer
JamesBOT
JustView
Keyword Density
Larbin
Leap
LibWeb
Likse
Litemage_walker
Ltx71
Magnet
Majestic SEO
Masscan
Microsoft URL Control
Mister PiX
Mr.4x3
Msrabot
NPbot
Nettrack
Ninja
OutclicksBot
PageScorer
Pi-Monster
PleaseCrawl
Probethenet
PxBroker
PyCurl
Quick-Crawler
RSSingBot
RankFlex
RealDownload
Ripper
SMTBot
Searchestate
SeekportBot
SemanticJuice
SemrushBot
Seomoz
SiteLockSpider
Snake
Snapbot
SocialRankIOBot
Sosospider
Suzuran
Swiftbot
TeleportPro
TheNomad
Thumbor
Toata
Trendiction
VidibleScraper
VoidEYE
WISENutbot
WeSEE
Web Fetch
WebImageCollector
WebZIP
Webster
Xenu
ZumBot
autoemailspider
backlink-check
check1.exe
clark-crawler
com.plumanalytics
crawl.sogou.com
evc-batch
gopher
heritrix
oBot
plumanalytics
s1z.ru
scan.lol
seocompany.store
sitechecker.pro
trendiction.com
webgains-bot
webmeup-crawler
zgrab