360Spider
80legs
Aboundexbot
ADmantX
AhrefsBot
AIBOT
Alligator
AllSubmitter
Anarchie
Attach
BackDoorBot
Backlink-Ceck
BackStreet
BDFetch
Blow
Brandprotect
Buddy
BuiltBotTough
BunnySlippers
BuzzSumo
Calculon
CATExplorador
CheTeam
ChinaClaw
Claritybot
Cliqzbot
Cogentbot
CopyRightCheck
Copyscape
Craftbot
Crescent
CSHttp
Curious
Custo
DBLBot
Devil
DIIbot
Dirbuster
Discobot
Discoverybot
DomainAppender
DomainCrawler
DomainStatsBot
Dotbot
Dragonfly
ECCP/1.0
Ecxi
EMail Siphon
evc-batch
Exabot
ExtLinksBot
Extractor
ExtractorPro
facebookscraper
FemtosearchBot
Fimap
FlashGet
FrontPage
FyberSpider
GalaxyBot
Getintent
Gigablast
Gigabot
Go-Ahead-Got-It
GoZilla
GrabNet
GrapeshotCrawler
GridBot
Haansoft
HaosouSpider
Harvest
heritrix
Heritrix
HTMLparser
HTTP::Lite
HTTrack
Iblog
IlseBot
IndeedBot
InfoTekies
instabid
Internet Ninja
IRLbot
IstellaBot
JOC Web Spider
Joomla
Jorgee
JustView
Lanshanbot
Lftp
LinkextractorPro
LinkpadBot
LinkScan
LinksManager
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Spider
Lmspider
LNSpiderguy
Ltx71
LWP::Simple
lwp-trivial
Magnet
magpie-crawler
Majestic-SEO
MarkMonitor
MarkWatch
Mass Downloader
Mata Hari
MauiBot
MeanPath Bot
mediawords
Microsoft Data Access
MJ12bot
Morfeus Fucking Scanner
Mr.4x3
MSIECrawler
Msrabot
Musobot
Navroad
Needle
Nessus
NetLyzer
Net Vampire
Netvibes
NICErsPRO
Ninja
oBot
OpenVAS
Page Analyzer
PageScorer
Pandalytics
Papa Foto
Pavuk
pcBrowser
PECL::HTTP
PeoplePal
Pimonster
ProPowerBot
ProWebWalker
Psbot
Pump
PxBroker
PyCurl
Quick-Crawler
RankActive
RankFlex
RankingBot
Rankivabot
RankurBot
RealDownload
Reaper
Recorder
ReGet
Ripper
RocketCrawler
Rogerbot
s1z.ru
SalesIntelligent
SBIder
ScanAlert
ScoutJet
Scrapy
Searchestate
SearchmetricsBot
Semrush
SemrushBot
SEOkicks
Seomoz
SEOprofiler
SeoSiteCheckup
SEOstats
Siteimprove
SiteLockSpider
SiteSnagger
SiteSucker
Site Sucker
SMTBot
Snake
Snoopy
SocialRankIOBot
Sogou web spider
Spammen
Spanner
sp_auditbot
Spbot
Spinn3r
spyfu
Sqworm
Steeler
Sucker
Sucuri
SuperBot
Surfbot
Suzuran
T0PHackTeam
tAkeOut
Teleport
TeleportPro
Telesoft
Telesphoreo
Telesphorep
The Intraformant
TheNomad
trendiction.de
True_Robot
Turingos
Turnitin
TurnitinBot
Upflow
URLy Warning
URLy.Warning
VB Project
VeriCiteCrawler
VoidEYE
Voltron
Wallpapers/3.0
WallpapersHD
WBSearchBot
WebBandit
WebCollage
Web Enhancer
WebFetch
Web Fetch
WebFuck
Web Fuck
WebLeacher
WebPix
WebSauger
Website Quester
Web Sucker
WebWhacker
WeSEE
Whack
Whatweb
Who.is Bot
Widow
WinHTTrack
WiseGuys Robot
Wonderbot
Wprecon
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
x22Mozilla
Zade
zgrab
ZmEu
ZyBorg