nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-07-10 10:07:01 +00:00

100 lines
1 KiB
Text

Aboundexbot
AdsTxtCrawlerTP
Alligator
Apexoo
BLEXBot
Backlink-Ceck
CCBot
CSHttp
CheTeam
Cliqzbot
Copier
Curious
Custo
CyotekWebCopy
Demon
Domains Project
Ebingbong
Ecxi
EroCrawler
Extractor
Fimap
GalaxyBot
Google-Extended
Gotit
Grabber
GridBot
Haansoft
Harvest
Hloader
IRLbot
Iblog
Information Security Team InfraSec Scanner
JOC Web Spider
JennyBot
LWP::Simple
Lanshanbot
Libwhisker
LinkextractorPro
LinqiaMetadataDownloaderBot
Lipperhey
MFC_Tear_Sample
MJ12bot
Magnet
Masscan
Mb2345Browser
Meanpathbot
Metauri
MicroMessenger
Microsoft URL Control
Moblie Safari
NearSite
Nessus
NetLyzer
Nettrack
OpenLinkProfiler
PECL::HTTP
Pixray
Pockey
RankingBot2
ReGet
Rogerbot
SEOkicks-Robot
SEOlyticsCrawler
SISTRIX
ScoutJet
Screaming
SemrushBot
SeobilityBot
Sitebeam
Snake
Snapbot
Sqlworm
Sqworm
Teleport
TightTwatBot
Toata
Turingos
Upflow
V-BOT
VB Project
WWW-Collector-E
Web Collage
WebAuto
WebBandit
WebCopier
WebGo IS
WebWhacker
WebmasterWorldForumBot
Widow
WiseGuys Robot
Wprecon
YoudaoBot
Zade
crawler4j
page scorer
s1z.ru
sexsearcher
sitechecker.pro
sogouspider
xpymep1.exe