404checker
404enemy
80legs
Abonti
Aboundex
Acunetix
ADmantX
Aipbot
Alligator
Apexoo
ASPSeek
autoemailspider
BackDoorBot
backlink-check
BackStreet
BackWeb
BatchFTP
BDCbot
Bitacle
Blackboard
BlackWidow
BLEXBot
Blow
Boardreader
BuiltWith
BunnySlippers
CATExplorador
CCBot
Cegbfeieh
CheeseBot
ChinaClaw
Chlooe
Cliqzbot
Cloud mapping
coccocbot-web
Collector
com.plumanalytics
Craftbot
crawler.feedback
crawl.sogou.com
CrazyWebCrawler
CrunchBot
Curious
demandbase-bot
Demon
Devil
Digincore
DigitalPebble
DIIbot
Disco
Discoverybot
DnyzBot
DomainSigmaCrawler
Dotbot
DSearch
DTS Agent
Ecxi
EMail Siphon
EMail Wolf
EroCrawler
evc-batch
Exabot
Express WebPictures
ExtLinksBot
Extractor
Firefox/7.0
FyberSpider
Fyrebot
GalaxyBot
Genieo
Getintent
GetRight
GetWeb
G-i-g-a-b-o-t
GoZilla
Go!Zilla
GrapeFX
GridBot
HaosouSpider
Harvest
Heritrix
HMView
HTMLparser
HTTP::Lite
IDBot
Id-search
IlseBot
IndeedBot
Internet Ninja
internetVista monitor
ips-agent
Iria
IRLbot
IstellaBot
Jorgee
Jyxobot
Kenjin Spider
Kozmosbot
Lanshanbot
Larbin
LeechGet
LibWeb
Lightspeedsystems
Likse
Linkdexbot
LinksManager
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Litemage_walker
Ltx71
lwp-request
LWP::Simple
Magnet
MarkMonitor
Masscan
MauiBot
meanpathbot
MeanPath Bot
Mediatoolkitbot
MegaIndex.ru
Metauri
Microsoft Data Access
MIDown tool
MJ12bot
Mojeek
MSIECrawler
muhstik-scan
Musobot
Nameprotect
Nessus
netEstate NE Crawler
Nettrack
Netvibes
Nibbler
Niki-bot
Nmap
Nutch
Offline Explorer
Offline Navigator
OnCrawl
OpenLinkProfiler
OrangeBot
OutclicksBot
PageAnalyzer
Page Analyzer
Pandalytics
Pavuk
PHPCrawl
Picscout
Pi-Monster
Pixray
PleaseCrawl
plumanalytics
Pockey
Psbot
Pump
PyCurl
Quick-Crawler
RankFlex
RankingBot
RealDownload
Recorder
RedesScrapy
Ripper
RocketCrawler
SalesIntelligent
ScanAlert
scan.lol
ScoutJet
Semrush
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
Seomoz
Siphon
SISTRIX
SiteExplorer
SiteSnagger
Site Sucker
SlySearch
SmartDownload
Snake
Snoopy
Sogou web spider
Sosospider
Sottopop
SpaceBison
Spammen
Spbot
SputnikBot
Sqlmap
Sqworm
Steeler
SuperBot
SuperHTTP
SurveyBot
Swiftbot
sysscan
T0PHackTeam
T8Abot
Telesphorep
Thumbor
TightTwatBot
Trendiction
Trendictionbot
Vacuum
Vagabondo
VeriCiteCrawler
Voltron
Wallpapers/3.0
WASALive-Bot
Webalta
WebAuto
Web Auto
WebBandit
WebCollage
Web Collage
WebCopier
WEBDAV
WebEnhancer
Web Enhancer
WebLeacher
webmeup-crawler
WebPix
Website Quester
WebSucker
WeSEE
Whacker
Who.is Bot
Widow
WinHTTrack
WISENutbot
Wonderbot
Wotbox
Wprecon
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
x09Mozilla
x22Mozilla
Xaldon WebSpider
Xenu
YoudaoBot
Zauba
zgrab
Zitebot
ZmEu