360Spider
404checker
404enemy
Abonti
Aboundex
Aboundexbot
Acunetix
ADmantX
AIBOT
AiHitBot
Aipbot
Alligator
Anarchie
Apexoo
ASPSeek
Asterias
Attach
BackDoorBot
BackStreet
Barkrowler
BDCbot
BDFetch
Bigfoot
Bitacle
BlackWidow
Blow
BotALot
Buddy
BuiltBotTough
BunnySlippers
CATExplorador
CheeseBot
CherryPicker
Chlooe
Cloud mapping
coccocbot-web
Collector
Copier
Cosmos
crawler4j
crawl.sogou.com
CSHttp
Curious
DataCha0s
DBLBot
Demon
Deusu
Devil
DigitalPebble
Dirbuster
Disco
Discoverybot
DnyzBot
DomainAppender
DomainCrawler
Dragonfly
DSearch
EasyDL
Ebingbong
ECCP/1.0
EirGrabber
EMail Wolf
EroCrawler
Evil
Exabot
Express WebPictures
ExtLinksBot
Extreme Picture Finder
Fimap
Firefox/7.0
FlashGet
FrontPage
FyberSpider
Fyrebot
GalaxyBot
Getintent
GetRight
Gigabot
Go-Ahead-Got-It
Grabber
GrapeshotCrawler
Haansoft
Havij
HMView
HTTP::Lite
HTTrack
Iblog
IDBot
Id-search
IlseBot
instabid
internetVista monitor
ips-agent
Iria
IRLbot
Iskanie
JamesBOT
Jbrofuzz
JetCar
Jetty
JikeSpider
Joomla
Jorgee
Keyword Density
Kozmosbot
Lanshanbot
Lftp
Linkdexbot
LinkScan
LinkWalker
LinqiaMetadataDownloaderBot
Lipperhey
Litemage_walker
Ltx71
LWP::Simple
lwp-trivial
Mag-Net
magpie-crawler
Mail.RU_Bot
MarkWatch
mediawords
MegaIndex.ru
Metauri
Mister PiX
Mojeek
Mr.4x3
MSFrontPage
Msrabot
MS Web Services Client Protocol
Navroad
Needle
Nessus
Netcraft
netEstate NE Crawler
NetLyzer
NetSpider
Net Vampire
Netvibes
Nikto
NimbleCrawler
Nimbostratus
oBot
OpenVAS
OrangeBot
OutclicksBot
page scorer
Panscient
pcBrowser
PECL::HTTP
PeoplePal
Picscout
Pi-Monster
Pixray
PleaseCrawl
Psbot
Pump
PxBroker
PyCurl
QueryN Metasearch
Quick-Crawler
RankActiveLinkBot
RankingBot
Rankivabot
RankurBot
RealDownload
Reaper
Recorder
RepoMonkey
Ripper
RocketCrawler
Rogerbot
SalesIntelligent
scan.lol
ScoutJet
Screaming
Searchestate
SearchmetricsBot
SemrushBot
SEOkicks-Robot
SEOlyticsCrawler
Seomoz
seoscanners
sexsearcher
SISTRIX
Sitebeam
SMTBot
Snoopy
Sociscraper
Sogou web spider
Sosospider
SpaceBison
Spammen
Spinn3r
SputnikBot
Sqlmap
Sqlworm
Steeler
Sucker
SuperBot
SuperHTTP
Swiftbot
sysscan
T8Abot
Teleport
TeleportPro
Telesphoreo
Telesphorep
TightTwatBot
Toata
Tracemyfile
Turingos
Turnitin
TurnitinBot
TwengaBot
UnisterBot
URLy Warning
URLy.Warning
Vacuum
VCI
VeriCiteCrawler
Virusdie
VoidEYE
Voltron
Wallpapers/3.0
WBSearchBot
WebAuto
WebBandit
Web Collage
WebEnhancer
WebImageCollector
WebmasterWorldForumBot
WebPix
WebReaper
Web Sauger
WebsiteExtractor
WebsiteQuester
Website Quester
WebSucker
WebWhacker
WeSEE
Whacker
Widow
WiseGuys Robot
WWW-Collector-E
WWW-Mechanize
Zade
Zauba
zgrab
ZumBot
ZyBorg