nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-05-19 10:09:43 +00:00

250 lines
2.6 KiB
Text

AIBOT
ASPSeek
Aboundex
AhrefsBot
AiHitBot
Aipbot
AlphaBot
Anarchie
Ankit
Aspiegel
Asterias
Attach
BDFetch
BackStreet
Backlink-Ceck
BacklinkCrawler
Bandit
BatchFTP
Battleztar Bazinga
Bigfoot
BlackWidow
Blackboard
Blow
BlowFish
Boardreader
BotALot
Buddy
BunnySlippers
Cegbfeieh
CensysInspect
CheTeam
CherryPicker
ChinaClaw
Chlooe
Citoid
Cocolyzebot
Collector
Copier
CopyRightCheck
Copyscape
Crawling at Home Project
Curious
CyotekWebCopy
DTS Agent
Demon
DigitalPebble
Dirbuster
Disco
Discobot
Discoverybot
DnyzBot
DomCopBot
DomainAppender
DomainSigmaCrawler
Dragonfly
EMail Wolf
EasyDL
Express WebPictures
Extractor
FDM
FHscan
FemtosearchBot
FlashGet
Foobot
Fuzz
Fyrebot
GalaxyBot
GermCrawler
Go-Ahead-Got-It
GrabNet
HEADMasterSEO
HTMLparser
Harvest
Humanlinks
IDBot
IRLbot
IndeedBot
Indy Library
InfoTekies
Intelliseek
IstellaBot
JOC Web Spider
JennyBot
JikeSpider
Jorgee
Kinza
LNSpiderguy
Larbin
Lftp
LibWeb
LieBaoFast
Likse
Linkbot
LinqiaMetadataDownloaderBot
Lmspider
MFC_Tear_Sample
MIDown tool
MSFrontPage
MSIECrawler
Mail.RU_Bot
Majestic12
MarkWatch
Masscan
Meanpathbot
Metauri
Microsoft Data Access
Microsoft URL Control
Moblie Safari
Mojolicious
Mr.4x3
Msrabot
Musobot
Nameprotect
Navroad
NetAnts
NetLyzer
NetSpider
Netvibes
Nibbler
Nikto
Nimbostratus
Ninja
Nutch
OpenVAS
Openfind
OrangeSpider
OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP
PageGrabber
Papa Foto
Picsearch
Piepmatz
PleaseCrawl
Probethenet
Pu_iN
PyCurl
Quick-Crawler
RankActive
Rankivabot
Re-re
Reaper
RocketCrawler
SEOkicks
SEOkicks-Robot
SEOprofiler
SEOstats
SISTRIX
SMTBot
Scanbot
ScoutJet
ScrepyBot
SemrushBot
SentiBot
SeoSiteCheckup
Shodan
SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam
Sitevigil
SlySearch
Snake
Snoopy
Sociscraper
Sogou web spider
Sottopop
Spammen
SpankBot
Steeler
Stripper
SuperBot
SuperHTTP
T0PHackTeam
T8Abot
Telesoft
TheNomad
Thumbor
Toata
Tracemyfile
Trendiction
Trendictionbot
True_Robot
UnisterBot
VB Project
Vacuum
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie
VoidEYE
WEBDAV
WISENutbot
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
Wallpapers
WallpapersHD
Web Enhancer
Web Sauger
WebCopier
WebEnhancer
WebFetch
WebImageCollector
WebLeacher
WebReaper
WebSauger
WebWhacker
WebZIP
Whack
WiseGuys Robot
Wprecon
Xaldon_WebSpider
YoudaoBot
Zade
Zauba
Zitebot
archive.org_bot
arquivo.pt
autoemailspider
backlink-check
coccocbot
com.plumanalytics
crawl.sogou.com
crawler4j
evc-batch
facebookscraper
heritrix
instabid
internetVista monitor
ips-agent
lwp-request
magpie-crawler
oBot
pcBrowser
plumanalytics
ripz
scan.lol
seocompany.store
seoscanners
spyfu
sysscan
tAkeOut
trendiction.com
ubermetrics-technologies.com
voyagerx.com
webgains-bot
webmeup-crawler
x09Mozilla
x22Mozilla