nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-05-28 22:07:25 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404enemy
80legs
ADmantX
ALittle Client
Abonti
Aboundex
Aboundexbot
Acunetix
AhrefsBot
AiHitBot
Aipbot
Anarchie
Anarchy
Ankit
Anthill
Apexoo
AwarioSmartBot
BLEXBot
BatchFTP
Bitacle
Black Hole
BlowFish
Bolt
BotALot
Buddy
CATExplorador
CSHttp
CheeseBot
Chlooe
Citoid
Cliqzbot
Cloud mapping
Copier
Copyscape
Craftbot
Crawling at Home Project
CrazyWebCrawler
Custo
DIIbot
DSearch
Demon
Deusu
Digincore
Dirbuster
Discoverybot
Dispatch
DnBCrawler-Analytics
DomCopBot
DomainAppender
DomainCrawler
DomainStatsBot
Dragonfly
Drip
EasyDL
Exabot
ExtLinksBot
Extractor
ExtractorPro
Extreme Picture Finder
EyeNetIE
Ezooms
Flunky
Genieo
GermCrawler
GetRight
GoZilla
HEADMasterSEO
HMView
HTMLparser
HTTP::Lite
Haansoft
HaosouSpider
Id-search
InterGET
Internet Ninja
InternetSeer
Iria
JetCar
Jetty
Joomla
Jorgee
Keyword Density
Kozmosbot
LNSpiderguy
Lanshanbot
Larbin
Leap
LeechFTP
LexiBot
Lightspeedsystems
LinkpadBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Lmspider
MFC_Tear_Sample
MSFrontPage
MSIECrawler
Majestic SEO
MarkWatch
Mass Downloader
Mb2345Browser
MeanPath Bot
MegaIndex.ru
Microsoft URL Control
Minefield
Mister PiX
Mojeek
MolokaiBot
Msrabot
Name Intelligence
Navroad
Needle
NetMechanic
NetZIP
Nettrack
Nikto
NimbleCrawler
Not
Nuclei
Nutch
OnCrawl
OpenLinkProfiler
PHPCrawl
PageAnalyzer
PageThing.com
Pi-Monster
PictureFinder
Pixray
PleaseCrawl
ProWebWalker
Pu_iN
PxBroker
PyCurl
Quick-Crawler
RankActiveLinkBot
RankingBot2
RankurBot
Reaper
RedesScrapy
SBIder
SEOkicks
SISTRIX
SMTBot
Screaming
Searchestate
SearchmetricsBot
SemanticJuice
Semrush
SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz
Shodan
Siphon
Site Sucker
SiteExplorer
SiteLockSpider
SiteSnagger
Siteimprove
Sitevigil
Snoopy
Sociscraper
Sogou web spider
Sottopop
Sqlworm
Stripper
Sucuri
SuperHTTP
Surfbot
SurveyBot
Suzuran
TeleportPro
TightTwatBot
Titan
Toweyabot
Trendiction
True_Robot
Upflow
VB Project
VCI
Vacuum
VidibleScraper
Voil
WBSearchBot
WISENutbot
WPScan
WWWOFFLE
Web Collage
Web Enhancer
Web Fetch
Web Pix
Web Sucker
WebAuto
WebBandit
WebGo IS
WebImageCollector
WebSauger
Webalta
Webshag
Webster
Whacker
WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Xaldon_WebSpider
Xenu
Zermelo
Zeus
ZoominfoBot
ZumBot
ZyBorg
arquivo-web-crawler
autoemailspider
backlink-check
coccocbot
crawler.feedback
crawler4j
demandbase-bot
domainsproject.org
eCatch
evc-batch
facebookscraper
heritrix
ips-agent
lwp-request
lwp-trivial
netEstate NE Crawler
oBot
probe-image-size
satoristudio.net
scalaj-http
seobility
seocompany.store
seoscanners
sexsearcher
sitechecker.pro
spyfu
sysscan
tAkeOut
trendiction.com
ubermetrics-technologies.com
voyagerx.com
webgains-bot
webmeup-crawler
webpros.com
webprosbot