nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-06-09 22:07:02 +00:00

250 lines
2.7 KiB
Text

80legs
ASPSeek
Alexibot
AlphaBot
Anarchy99
AwarioSmartBot
BDCbot
BDFetch
BackDoorBot
BackWeb
Backlink-Ceck
Bandit
Barkrowler
BetaBot
Bitacle
Blackboard
Boardreader
Brandwatch
Buck
Bullseye
BuzzSumo
Calculon
CensysInspect
CheeseBot
Chlooe
Citoid
Cliqzbot
Cloud mapping
Copyscape
CyotekWebCopy
DSearch
Demon
DigitalPebble
Discobot
DittoSpyder
DnBCrawler-Analytics
DnyzBot
DomCopBot
DomainCrawler
DomainStatsBot
Dragonfly
Drip
ECCP/1.0
EMail Siphon
Ebingbong
Ecxi
EirGrabber
EroCrawler
ExtractorPro
EyeNetIE
Ezooms
FDM
FemtosearchBot
FrontPage
FyberSpider
GermCrawler
GetRight
Getintent
Go-Ahead-Got-It
Gotit
GrabNet
Grabber
GrapeFX
HTMLparser
Haansoft
HaosouSpider
Harvest
Heritrix
HybridBot
IDBot
IlseBot
Image Sucker
Iskanie
JOC Web Spider
Jbrofuzz
JennyBot
JetCar
JustView
Jyxobot
Keybot Translation-Search-Machine
Kinza
Kozmosbot
LNSpiderguy
LWP::Simple
Lanshanbot
Leap
LeechGet
LexiBot
Lftp
Libwhisker
Linkbot
LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey
Ltx71
Majestic SEO
MarkWatch
Mass Downloader
Mediatoolkitbot
MegaIndex.ru
MicroMessenger
Minefield
Mister PiX
Moblie Safari
Mojeek
MolokaiBot
Mozlila
Mr.4x3
Msrabot
Musobot
NICErsPRO
Needle
Nessus
NetMechanic
NetSpider
NextGenSearchBot
Niki-bot
NimbleCrawler
Nimbostratus
Nmap
Octopus
Offline Explorer
OpenLinkProfiler
Openfind
Openvas
OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer
PageScorer
Panscient
Papa Foto
Petalbot
Pi-Monster
PictureFinder
Pimonster
PleaseCrawl
Pockey
Probethenet
Psbot
Pu_iN
Pump
RankActiveLinkBot
RankFlex
ReGet
RebelMouse
RepoMonkey
Ripper
RocketCrawler
Rogerbot
SEOkicks
SEOlyticsCrawler
SISTRIX
ScanAlert
Scrapy
Searchestate
SearchmetricsBot
Semrush
SemrushBot
SeoSiteCheckup
SeobilityBot
Seomoz
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
SiteSucker
Siteimprove
SlySearch
Snapbot
SocialRankIOBot
Sociscraper
Sogou web spider
Sosospider
SpaceBison
Spbot
SputnikBot
Sqworm
SuperBot
SuperHTTP
SurveyBot
Suzuran
TeleportPro
Telesoft
TheNomad
Titan
Trendiction
Turnitin
TwengaBot
Twice
Typhoeus
V-BOT
Vacuum
VoidEYE
Voil
WISENutbot
WWW-Mechanize
WWW::Mechanize
Wallpapers
Wallpapers/3.0
Web Collage
Web Pix
Web Sucker
WebCopier
WebEnhancer
WebImageCollector
WebReaper
WebStripper
Webalta
Webshag
Website Quester
Wonderbot
Woobot
Wotbox
Xaldon WebSpider
Zauba
Zeus
Zitebot
ZumBot
ZyBorg
archive.org_bot
arquivo-web-crawler
autoemailspider
backlink-check
clark-crawler
cognitiveseo
crawler4j
eCatch
facebookscraper
ips-agent
lwp-request
lwp-trivial
mediawords
muhstik-scan
netEstate NE Crawler
pcBrowser
plumanalytics
satoristudio.net
seocompany.store
seostar
serpstatbot
sogouspider
tAkeOut
trendiction.de
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
webprosbot
x09Mozilla
zauba.io