404checker
80legs
Abonti
Aboundexbot
Acunetix
ADmantX
AIBOT
Alexibot
AllSubmitter
Anarchie
Anarchy
arquivo-web-crawler
AwarioRssBot
AwarioSmartBot
BackDoorBot
Backlink-Ceck
backlink-check
BacklinkCrawler
BackWeb
Barkrowler
BDCbot
Bitacle
Black Hole
BlowFish
Bolt
Brandwatch
BuiltWith
Calculon
CazoodleBot
CensysInspect
CheeseBot
CherryPicker
ChinaClaw
Chlooe
Cliqzbot
coccocbot-web
Cocolyzebot
com.plumanalytics
Copier
CopyRightCheck
Copyscape
Cosmos
Craftbot
Crescent
DBLBot
Demon
Devil
Digincore
Discoverybot
DittoSpyder
DomainAppender
DomainCrawler
DomainSigmaCrawler
domainsproject.org
DomainStatsBot
DomCopBot
Download Wonder
Dragonfly
DSearch
DTS Agent
EasyDL
eCatch
EMail Siphon
Evil
Exabot
ExtLinksBot
facebookscraper
FemtosearchBot
Fyrebot
GalaxyBot
Genieo
Getintent
Go-Ahead-Got-It
Go!Zilla
Grabber
GrapeFX
GrapeshotCrawler
GT::WWW
HaosouSpider
Harvest
HMView
Iblog
IDBTE4M
Image Fetch
Image Sucker
IndeedBot
Indy Library
InfoTekies
instabid
Internet Ninja
ips-agent
Iria
IRLbot
Iskanie
IstellaBot
JOC Web Spider
Kenjin Spider
Kinza
Larbin
LeechGet
LieBaoFast
Likse
LinkScan
Lipperhey
Lmspider
LNSpiderguy
lwp-request
LWP::Simple
lwp-trivial
Mag-Net
Majestic SEO
Majestic-SEO
MarkWatch
Mata Hari
MauiBot
Metauri
MIDown tool
MJ12bot
Mojeek
Mozlila
MQQBrowser
MSIECrawler
Nameprotect
NearSite
Needle
NetAnts
Netcraft
netEstate NE Crawler
Nettrack
Net Vampire
Nimbostratus
Nuclei
Nutch
Offline Navigator
OnCrawl
Openfind
Openvas
OPPO A33
PageScorer
Pandalytics
Panscient
Papa Foto
PeoplePal
Petalbot
PHPCrawl
Picsearch
PictureFinder
Pixray
PleaseCrawl
plumanalytics
ProWebWalker
Pu_iN
Pump
QueryN Metasearch
Quick-Crawler
RankingBot2
RankurBot
Recorder
RepoMonkey
Re-re
Ripper
ripz
RSSingBot
s1z.ru
satoristudio.net
SBIder
ScanAlert
scan.lol
ScoutJet
Searchestate
SemrushBot
SentiBot
seobility
SeobilityBot
SEOkicks-Robot
seoscanners
serpstatbot
sexsearcher
SISTRIX
SiteCheckerBotCrawler
SiteExplorer
Siteimprove
siteripz
SiteSnagger
SiteSucker
Sitevigil
SmartDownload
SMTBot
sogouspider
Sogou web spider
Sottopop
Spbot
spyfu
Sqlmap
Stripper
SuperHTTP
Suzuran
T0PHackTeam
tAkeOut
TeleportPro
Telesphorep
TheNomad
Titan
Trendiction
Trendictionbot
trendiction.com
True_Robot
TurnitinBot
Typhoeus
Upflow
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie
Voil
WBSearchBot
Webalta
WebAuto
Web Auto
WebBandit
WebCopier
WEBDAV
WebEnhancer
Web Enhancer
WebFuck
WebGo IS
WebLeacher
WebmasterWorldForumBot
webmeup-crawler
WebPix
Web Pix
WebSauger
Web Sauger
WebStripper
WebSucker
WebZIP
Widow
WinHTTrack
WISENutbot
Woobot
WWW-Mechanize
x09Mozilla
Xaldon_WebSpider
Xenu
YoudaoBot
zauba.io
Zeus
zgrab
ZoominfoBot
ZumBot