360Spider
404enemy
80legs
Abonti
Aboundexbot
Acunetix
Aipbot
Alexibot
Alligator
Anarchie
ASPSeek
Asterias
Attach
Backlink-Ceck
BackWeb
Barkrowler
Battleztar Bazinga
BDCbot
BetaBot
BlackWidow
BLEXBot
Blow
Bolt
Brandwatch
Buddy
BuiltBotTough
BuiltWith
Bullseye
BuzzSumo
CATExplorador
CCBot
CherryPicker
Claritybot
Cliqzbot
Copyscape
Craftbot
crawler4j
CrunchBot
DataCha0s
DBLBot
demandbase-bot
Deusu
Digincore
DIIbot
Dispatch
DomainCrawler
Download Wonder
DTS Agent
EasyDL
Ebingbong
EMail Siphon
EMail Wolf
EroCrawler
Exabot
ExtractorPro
Extreme Picture Finder
facebookscraper
FDM
Firefox/7.0
Flunky
Foobot
Fyrebot
Genieo
GermCrawler
Getintent
GetRight
Gigablast
Gigabot
Go-Ahead-Got-It
Gotit
GoZilla
Grabber
GrabNet
Grafula
GridBot
GT::WWW
HaosouSpider
HEADMasterSEO
HybridBot
Iblog
Id-search
IlseBot
Indy Library
instabid
Intelliseek
ips-agent
IstellaBot
Jbrofuzz
Jorgee
JustView
Jyxobot
Kozmosbot
Larbin
LeechFTP
LeechGet
LexiBot
Lftp
Libwhisker
Lightspeedsystems
Linkdexbot
LinkextractorPro
LinkpadBot
LinkScan
LinksManager
LinqiaScrapeBot
Lipperhey
Litemage_walker
Lmspider
LNSpiderguy
Ltx71
lwp-request
LWP::Simple
Mag-Net
magpie-crawler
Majestic12
Mata Hari
Meanpathbot
MFC_Tear_Sample
MIIxpc
Mister PiX
MJ12bot
Morfeus Fucking Scanner
Mr.4x3
MSIECrawler
Msrabot
Musobot
Name Intelligence
Navroad
NetLyzer
Net Vampire
Netvibes
NetZIP
Nibbler
NICErsPRO
NimbleCrawler
Nimbostratus
NPbot
Nutch
Offline Navigator
OnCrawl
Openfind
OpenLinkProfiler
OpenVAS
OutclicksBot
PageAnalyzer
Page Analyzer
PageScorer
Panscient
Pavuk
pcBrowser
PeoplePal
Picsearch
PictureFinder
Pimonster
PleaseCrawl
Pockey
POE-Component-Client-HTTP
ProWebWalker
Psbot
PxBroker
PyCurl
QueryN Metasearch
RankActive
RankFlex
RankingBot2
RealDownload
Recorder
RepoMonkey
Ripper
RSSingBot
s1z.ru
SearchmetricsBot
Semrush
SemrushBot
SEOkicks-Robot
SEOlyticsCrawler
Seomoz
seoscanners
SeoSiteCheckup
SEOstats
sexsearcher
Siphon
SiteLockSpider
Site Sucker
Sitevigil
SlySearch
SMTBot
Snake
SocialRankIOBot
Sogou web spider
Spbot
Spinn3r
spyfu
Sqlmap
Stripper
Sucker
Sucuri
Surfbot
Suzuran
sysscan
Szukacz
tAkeOut
Teleport
The Intraformant
TightTwatBot
Titan
Tracemyfile
Trendiction
trendiction.com
True_Robot
Turingos
Turnitin
TurnitinBot
TwengaBot
Upflow
URLy.Warning
Vagabondo
VCI
VeriCiteCrawler
VidibleScraper
Virusdie
Voltron
WallpapersHD
WASALive-Bot
WBSearchBot
Webalta
WebAuto
Web Enhancer
WebFetch
WebFuck
Web Fuck
WebImageCollector
WebmasterWorldForumBot
webmeup-crawler
Webshag
WebsiteExtractor
Website Quester
Webster
WebZIP
Whatweb
WiseGuys Robot
Wprecon
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
Xaldon WebSpider
Xaldon_WebSpider
xpymep1.exe
Zade
Zauba