nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-11-21 22:05:14 +00:00

100 lines
1 KiB
Text

404enemy
Aboundex
Aboundexbot
Alligator
AwarioBot
BDFetch
BlackWidow
Blow
Bolt
Bytespider
CSHttp
CherryPicker
Chlooe
Cliqzbot
Copier
CopyRightCheck
DIIbot
DataCha0s
Disco
DomainAppender
Exabot
ExtractorPro
Getintent
Go!Zilla
HaosouSpider
IRLbot
Id-search
IndeedBot
Iria
JOC Web Spider
Jbrofuzz
JetCar
JustView
Jyxobot
LexiBot
LinkScan
LinkWalker
Lmspider
MJ12bot
MTRobot
Majestic-SEO
MarkMonitor
Mass Downloader
Mojeek
Needle
NetAnts
NetZIP
Netvibes
PageThing.com
ProPowerBot
PyCurl
Re-re
Reaper
Recorder
RepoMonkey
Ripper
RocketCrawler
Screaming
ScrepyBot
SentiBot
SeoSiteCheckup
SeobilityBot
SiteSnagger
SiteSucker
Siteimprove
Spanner
TheNomad
Turingos
Upflow
Vacuum
VelenPublicWebCrawler
VidibleScraper
WWW-Collector-E
WWW-Mechanize
Web Sucker
WebEnhancer
WebSucker
WebWhacker
Xaldon_WebSpider
Zauba
Zitebot
ZmEu
autoemailspider
check1.exe
clark-crawler
coccocbot
evc-batch
linkdexbot
lwp-trivial
magpie-crawler
openai
page scorer
polaris version
s1z.ru
siteripz
spyfu
trendiction.de
ubermetrics-technologies.com
webgains-bot
zauba.io