404checker
404enemy
Abonti
Aboundex
Acunetix
ADmantX
AhrefsBot
AIBOT
AiHitBot
Alexibot
Alligator
AllSubmitter
AlphaBot
arquivo-web-crawler
Attach
AwarioRssBot
AwarioSmartBot
backlink-check
BackStreet
Badass
Bitacle
Blackboard
BlackWidow
BLEXBot
Blow
Brandprotect
BuiltBotTough
BuiltWith
Calculon
CATExplorador
CazoodleBot
CCBot
Cegbfeieh
CheeseBot
CheTeam
Claritybot
Cloud mapping
Collector
CopyRightCheck
crawler.feedback
CrunchBot
DBLBot
Demon
Dirbuster
Disco
DomainCrawler
DomainStatsBot
Dotbot
Ebingbong
eCatch
ECCP/1.0
EirGrabber
EMail Siphon
EMail Wolf
EroCrawler
evc-batch
ExtLinksBot
Extractor
Ezooms
facebookscraper
Fimap
Firefox/7.0
FlashGet
Freeuploader
FyberSpider
Genieo
GermCrawler
GetRight
GetWeb
G-i-g-a-b-o-t
Go-Ahead-Got-It
Gotit
GoZilla
Go!Zilla
GrapeFX
GridBot
GT::WWW
HaosouSpider
Havij
HEADMasterSEO
Heritrix
Hloader
HMView
HTTP::Lite
HTTrack
IDBot
IlseBot
Image Sucker
InfoTekies
instabid
Intelliseek
InterGET
internetVista monitor
ips-agent
IRLbot
IstellaBot
JamesBOT
JennyBot
JOC Web Spider
Joomla
Jorgee
Jyxobot
Kenjin Spider
Keyword Density
Kozmosbot
Larbin
Libwhisker
Linkdexbot
LinkWalker
LinqiaRSSBot
Litemage_walker
Lmspider
Mail.RU_Bot
Majestic12
Mass Downloader
Mata Hari
Mb2345Browser
meanpathbot
Meanpathbot
Metauri
MicroMessenger
Microsoft Data Access
Mister PiX
MJ12bot
Morfeus Fucking Scanner
Mozlila
Mr.4x3
MSFrontPage
Nameprotect
NearSite
Needle
NetAnts
netEstate NE Crawler
NetLyzer
NetSpider
Nettrack
Netvibes
NimbleCrawler
NPbot
oBot
Offline Navigator
Openvas
OrangeBot
PageGrabber
page scorer
Pandalytics
Panscient
Papa Foto
pcBrowser
Picscout
Pixray
POE-Component-Client-HTTP
polaris version
Probethenet
Pump
RankActiveLinkBot
RankingBot
RankingBot2
Rankivabot
Reaper
Recorder
RedesScrapy
ReGet
RepoMonkey
Ripper
RocketCrawler
Rogerbot
RSSingBot
s1z.ru
SBIder
Scanbot
ScoutJet
Scrapy
ScreenerBot
SearchmetricsBot
SEOkicks
SEOlyticsCrawler
Seomoz
SEOprofiler
SEOstats
Shodan
Sitebeam
Site Sucker
SlySearch
SmartDownload
Snake
Snapbot
Snoopy
sogouspider
Sogou web spider
Sosospider
Sottopop
SpankBot
sp_auditbot
Spbot
Spinn3r
spyfu
Sqlmap
Sqlworm
Sqworm
Steeler
SuperBot
SurveyBot
Suzuran
T0PHackTeam
tAkeOut
Telesoft
TheNomad
Thumbor
TightTwatBot
Titan
Toata
Toweyabot
Trendictionbot
trendiction.com
trendiction.de
Turnitin
Twice
UnisterBot
URLy.Warning
VidibleScraper
Voil
WBSearchBot
Webalta
Web Collage
WebCopier
Web Enhancer
WebFetch
WebFuck
WebImageCollector
WebPix
WebReaper
Web Sauger
Webshag
WebsiteExtractor
Website Quester
Webster
Web Sucker
WebZIP
Whatweb
WISENutbot
Woobot
WWW-Mechanize
WWW::Mechanize
x09Mozilla
YoudaoBot
Zauba
Zermelo
Zeus
Zitebot