404checker
404enemy
80legs
Aboundexbot
Acunetix
AIBOT
AiHitBot
Asterias
Attach
autoemailspider
BackDoorBot
BacklinkCrawler
BackWeb
Bandit
Barkrowler
BatchFTP
Battleztar Bazinga
BBBike
BDFetch
Bigfoot
Blackboard
BLEXBot
Boardreader
BotALot
Buddy
Bullseye
BunnySlippers
BuzzSumo
CazoodleBot
CCBot
Cegbfeieh
CheTeam
ChinaClaw
coccocbot-web
Collector
CopyRightCheck
Copyscape
Cosmos
crawl.sogou.com
CrazyWebCrawler
CrunchBot
CSHttp
Curious
DatabaseDriverMysqli
DataCha0s
demandbase-bot
Demon
Devil
Digincore
Discoverybot
Dispatch
DittoSpyder
DnyzBot
DomainCrawler
DomainSigmaCrawler
Dotbot
Download Wonder
Dragonfly
EasyDL
Ebingbong
eCatch
ECCP/1.0
Ecxi
EirGrabber
EMail Siphon
EroCrawler
evc-batch
Express WebPictures
Extractor
Ezooms
Firefox/7.0
FlashGet
Flunky
Foobot
FyberSpider
Genieo
GermCrawler
Getintent
GetRight
GetWeb
Gigabot
Go-Ahead-Got-It
Grabber
GrapeFX
GrapeshotCrawler
GT::WWW
Havij
Hloader
HTMLparser
HTTP::Lite
HTTrack
Humanlinks
IDBot
Image Sucker
InfoNaviRobot
instabid
Intelliseek
internetVista monitor
ips-agent
Iskanie
JamesBOT
JennyBot
Jetty
JOC Web Spider
Jorgee
Kenjin Spider
Keyword Density
Kozmosbot
Lanshanbot
LeechFTP
LeechGet
Lightspeedsystems
Likse
LinkextractorPro
LinkWalker
LinqiaRSSBot
Lipperhey Spider
Litemage_walker
Lmspider
LNSpiderguy
magpie-crawler
Masscan
Mass Downloader
MauiBot
Meanpathbot
MeanPath Bot
MegaIndex.ru
Microsoft Data Access
MIDown tool
MJ12bot
Mojeek
muhstik-scan
Navroad
NearSite
Needle
NetAnts
NetSpider
Net Vampire
Netvibes
NICErsPRO
Nmap
Offline Explorer
Openvas
OpenVAS
OrangeSpider
OutclicksBot
Page Analyzer
PageGrabber
page scorer
Panscient
Papa Foto
Pavuk
Picscout
PictureFinder
Pimonster
PleaseCrawl
Pockey
POE-Component-Client-HTTP
Probethenet
PxBroker
QueryN Metasearch
Quick-Crawler
RankActive
RankFlex
RankingBot2
Rankivabot
RebelMouse
Recorder
RedesScrapy
RepoMonkey
Ripper
RocketCrawler
Rogerbot
s1z.ru
SalesIntelligent
SBIder
Searchestate
SearchmetricsBot
Semrush
SEOkicks-Robot
seoscanners
SeoSiteCheckup
serpstatbot
sexsearcher
Shodan
Sitebeam
SiteSnagger
SiteSucker
Sitevigil
Snapbot
SocialRankIOBot
Sociscraper
Sogou web spider
SpaceBison
SpankBot
Spanner
Sqworm
Stripper
Sucker
Surfbot
sysscan
T0PHackTeam
Telesoft
TheNomad
Titan
Toweyabot
TurnitinBot
TwengaBot
Twice
Vacuum
VB Project
VeriCiteCrawler
Wallpapers/3.0
Webalta
WebAuto
Web Auto
WebBandit
WebCollage
WebCopier
Web Enhancer
WebFetch
Web Fetch
WebGo IS
WebImageCollector
WebLeacher
webmeup-crawler
Web Pix
WebReaper
Web Sauger
WebsiteExtractor
WebsiteQuester
Webster
Web Sucker
Whacker
Whatweb
Widow
WISENutbot
Wprecon
WPScan
WWW::Mechanize
x09Mozilla
Xaldon WebSpider
Xaldon_WebSpider
Xenu
YoudaoBot
Zade
Zeus
Zitebot
ZmEu
ZyBorg