404checker
80legs
AIBOT
ALittle Client
Abonti
Acunetix
AhrefsBot
AllSubmitter
AlphaBot
Anarchie
Anarchy
Apexoo
Asterias
AwarioRssBot
BBBike
BDFetch
BLEXBot
BackStreet
Badass
Bandit
BatchFTP
Bigfoot
Bitacle
Blow
Boardreader
Brandwatch
Buddy
Bullseye
CATExplorador
Calculon
CazoodleBot
CheTeam
CherryPicker
Chlooe
Cliqzbot
Cocolyzebot
Collector
Crawling at Home Project
CrazyWebCrawler
Curious
Custo
CyotekWebCopy
DBLBot
DTS Agent
Devil
Dirbuster
Disco
DittoSpyder
DnBCrawler-Analytics
DomainCrawler
DomainSigmaCrawler
Domains Project
ECCP/1.0
EMail Wolf
EasyDL
Ecxi
EirGrabber
EroCrawler
Extractor
ExtractorPro
FlashGet
FyberSpider
GalaxyBot
GetRight
GetWeb
GoZilla
Gotit
Grafula
GrapeshotCrawler
HMView
HTTP::Lite
Haansoft
Harvest
Havij
Heritrix
HonoluluBot
Humanlinks
HybridBot
Image Fetch
InfoTekies
Iria
JOC Web Spider
Jetty
JikeSpider
JustView
Keybot Translation-Search-Machine
Kozmosbot
Leap
Lftp
Libwhisker
Lightspeedsystems
Likse
LinkextractorPro
LinqiaScrapeBot
Lmspider
MFC_Tear_Sample
MIDown tool
MJ12bot
MTRobot
Mag-Net
Magnet
Mail.RU_Bot
Majestic-SEO
Majestic12
MarkWatch
Masscan
Mata Hari
Mb2345Browser
Mediatoolkitbot
Mojeek
Morfeus Fucking Scanner
Msrabot
NICErsPRO
Name Intelligence
NetAnts
NetMechanic
NetSpider
Netvibes
Nibbler
Niki-bot
NimbleCrawler
Nimbostratus
Nuclei
Nutch
Offline Explorer
Offline Navigator
OnCrawl
OpenVAS
Openfind
Openvas
OrangeBot
OrangeSpider
PECL::HTTP
PHPCrawl
Page Analyzer
PageGrabber
PageThing.com
Petalbot
Picscout
Pixray
PleaseCrawl
Pockey
ProPowerBot
Probethenet
Psbot
Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankingBot
RankingBot2
Recorder
SEOstats
SISTRIX
ScoutJet
Screaming
Seekport
SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
SiteCheckerBotCrawler
SiteExplorer
SiteSnagger
SlySearch
Snake
Snapbot
SocialRankIOBot
Sogou web spider
SpaceBison
SpankBot
Sqlworm
Sqworm
SurveyBot
Suzuran
T0PHackTeam
T8Abot
Telesoft
TheNomad
Titan
TwengaBot
URLy.Warning
VB Project
VCI
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
WASALive-Bot
WWW-Collector-E
WWW::Mechanize
Wallpapers
Wallpapers/3.0
WallpapersHD
Web Fuck
Web Pix
WebCollage
WebCopier
WebFetch
WebFuck
WebGo IS
WebLeacher
WebStripper
WebWhacker
WebZIP
Webshag
Website Quester
WebsiteExtractor
Whacker
Who.is Bot
WinHTTrack
Woobot
Wotbox
Wprecon
Xaldon WebSpider
Xaldon_WebSpider
ZmEu
ZyBorg
arquivo-web-crawler
arquivo.pt
autoemailspider
cah.io.community
com.plumanalytics
crawler.feedback
crawler4j
dataforseo.com
demandbase-bot
domainsproject.org
heritrix
instabid
isitwp.com
lwp-request
lwp-trivial
meanpathbot
muhstik-scan
oBot
page scorer
pcBrowser
scalaj-http
seocompany.store
seoscanners
seostar
sitechecker.pro
siteripz
spyfu
sysscan
voyagerx.com
webprosbot
x09Mozilla
x22Mozilla
zauba.io