360Spider
404checker
80legs
Aboundex
AiHitBot
Alexibot
AlphaBot
Anarchy
Anarchy99
Ankit
Anthill
Aspiegel
Asterias
BBBike
BackDoorBot
BackStreet
Badass
BetaBot
Bitacle
Blow
Boardreader
BotALot
Brandprotect
Brandwatch
Buddy
BuiltWith
Bullseye
CODE87
ChinaClaw
Chlooe
Claritybot
Cloud mapping
Cocolyzebot
Collector
CrunchBot
Custo
DBLBot
DIIbot
Demon
Deusu
Disco
DnyzBot
DomCopBot
DomainCrawler
Domains Project
Dragonfly
ECCP/1.0
Ecxi
EirGrabber
EroCrawler
Exabot
ExtLinksBot
Extreme Picture Finder
Fimap
Firefox/7.0
FlashGet
Flunky
Foobot
Genieo
GoZilla
Grafula
GrapeshotCrawler
GridBot
HEADMasterSEO
HMView
HTTrack
HaosouSpider
Harvest
Havij
Hloader
IDBTE4M
Image Fetch
Indy Library
InfoNaviRobot
InterGET
Iskanie
JikeSpider
Jorgee
JustView
Keyword Density
Kozmosbot
LWP::Simple
Lftp
LibWeb
Libwhisker
LieBaoFast
Lightspeedsystems
Likse
LinkScan
LinkWalker
Linkbot
Linkdexbot
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey
Litemage_walker
Lmspider
MFC_Tear_Sample
MJ12bot
MQQBrowser
MTRobot
Majestic SEO
Majestic-SEO
Masscan
Mb2345Browser
Meanpathbot
Metauri
MicroMessenger
Microsoft Data Access
Minefield
Mister PiX
Moblie Safari
Mojeek
Mojolicious
MolokaiBot
Mr.4x3
Msrabot
NearSite
Nessus
NetLyzer
NetMechanic
NetSpider
NetZIP
Nettrack
NextGenSearchBot
NimbleCrawler
Ninja
Nuclei
Nutch
Octopus
OpenLinkProfiler
Openvas
OrangeBot
Page Analyzer
PeoplePal
Petalbot
Pi-Monster
PleaseCrawl
Pockey
ProPowerBot
Pump
Quick-Crawler
RSSingBot
RankFlex
RankingBot2
Rankivabot
ReGet
Reaper
Recorder
RepoMonkey
SEOkicks
SEOprofiler
ScanAlert
Screaming
ScreenerBot
SearchmetricsBot
Seekport
SemanticJuice
SeobilityBot
Site Sucker
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
Sitebeam
Siteimprove
Sitevigil
SmartDownload
Snapbot
Sociscraper
SpaceBison
SputnikBot
Sqlmap
Sqworm
Steeler
SuperBot
SuperHTTP
T0PHackTeam
Teleport
Telesphoreo
Thumbor
TightTwatBot
Titan
Toweyabot
Trendictionbot
True_Robot
UnisterBot
Upflow
V-BOT
VB Project
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie
Voil
Voltron
WBSearchBot
WEBDAV
WWW-Collector-E
WWW::Mechanize
Wallpapers
Wallpapers/3.0
WebCopier
WebFetch
WebGo IS
WebPix
WebSucker
WebZIP
Webshag
Website Quester
WebsiteQuester
Whack
Widow
WiseGuys Robot
Wotbox
Wprecon
Xenu
Zauba
ZmEu
ZyBorg
adscanner
arquivo-web-crawler
autoemailspider
coccocbot-web
cognitiveseo
crawl.sogou.com
crawler.feedback
domainsproject.org
eCatch
evc-batch
instabid
internetVista monitor
isitwp.com
iubenda-radar
lwp-trivial
polaris version
ripz
s1z.ru
scalaj-http
scan.lol
serpstatbot
sexsearcher
siteripz
sogouspider
sp_auditbot
trendiction.com
ubermetrics-technologies.com
webmeup-crawler
webprosbot
x09Mozilla
x22Mozilla
xpymep1.exe