360Spider
404enemy
80legs
Aboundex
Aboundexbot
Acunetix
ADmantX
Aipbot
Alligator
AllSubmitter
Apexoo
ASPSeek
Attach
autoemailspider
BacklinkCrawler
BackStreet
BackWeb
Badass
BatchFTP
Battleztar Bazinga
BDFetch
BetaBot
Bigfoot
Blackboard
BotALot
Brandwatch
Buddy
BuiltBotTough
BuiltWith
Bullseye
BunnySlippers
BuzzSumo
CazoodleBot
CCBot
Claritybot
Cloud mapping
cognitiveseo
Collector
Cosmos
Craftbot
crawler.feedback
crawl.sogou.com
Crescent
CrunchBot
Custo
DatabaseDriverMysqli
Deusu
Devil
DigitalPebble
DIIbot
Disco
Discobot
Discoverybot
DnyzBot
Dotbot
Download Wonder
Drip
DTS Agent
Ebingbong
ECCP/1.0
Ecxi
EirGrabber
EMail Wolf
EroCrawler
Evil
Express WebPictures
ExtractorPro
EyeNetIE
Ezooms
FemtosearchBot
Firefox/7.0
Flunky
FyberSpider
GalaxyBot
Genieo
Getintent
G-i-g-a-b-o-t
Go-Ahead-Got-It
Gotit
Go!Zilla
Grafula
GrapeshotCrawler
GridBot
HaosouSpider
Havij
Heritrix
HMView
HTMLparser
HTTP::Lite
Humanlinks
HybridBot
Id-search
IlseBot
Image Fetch
IndeedBot
InfoNaviRobot
InfoTekies
instabid
Iria
IRLbot
Iskanie
IstellaBot
JetCar
Joomla
Jyxobot
Keyword Density
Kozmosbot
Lanshanbot
Larbin
LeechGet
Lftp
Libwhisker
Likse
LinkpadBot
LinksManager
Lipperhey
Litemage_walker
Lmspider
lwp-trivial
Magnet
Mag-Net
magpie-crawler
Majestic SEO
Majestic-SEO
Mata Hari
MauiBot
meanpathbot
MeanPath Bot
MegaIndex.ru
Metauri
Microsoft URL Control
MJ12bot
Mojeek
MSFrontPage
MSIECrawler
Name Intelligence
Nameprotect
Nessus
NetAnts
Netcraft
NetLyzer
NetMechanic
NetSpider
NetZIP
NextGenSearchBot
Nibbler
NICErsPRO
Nimbostratus
NPbot
Nutch
Octopus
Offline Navigator
Openfind
OpenLinkProfiler
OpenVAS
OrangeSpider
OutfoxBot
PageAnalyzer
PageGrabber
PageScorer
Pandalytics
PeoplePal
PictureFinder
Pimonster
Pi-Monster
Pockey
Probethenet
PxBroker
Quick-Crawler
RankingBot2
Rankivabot
RankurBot
RealDownload
Recorder
RedesScrapy
RepoMonkey
Ripper
RocketCrawler
s1z.ru
ScanAlert
Scanbot
scan.lol
Screaming
ScreenerBot
SEOlyticsCrawler
Seomoz
SEOprofiler
SeoSiteCheckup
SEOstats
sexsearcher
Siteimprove
SiteSnagger
SiteSucker
Site Sucker
SmartDownload
Snake
Snoopy
sogouspider
Sogou web spider
Spammen
Spbot
Spinn3r
Sqlworm
Steeler
Stripper
Sucuri
Swiftbot
sysscan
tAkeOut
Teleport
TeleportPro
Telesphoreo
Telesphorep
trendiction.de
Turingos
TwengaBot
Twice
UnisterBot
URLy.Warning
VB Project
VidibleScraper
VoidEYE
Wallpapers/3.0
WallpapersHD
WASALive-Bot
WebEnhancer
Web Enhancer
WebImageCollector
WebPix
WebReaper
Web Sauger
WebStripper
Web Sucker
Whack
Whacker
Whatweb
WinHTTrack
WiseGuys Robot
Wonderbot
WPScan
WWW-Collector-E
WWW::Mechanize
Xaldon WebSpider
Xenu
YoudaoBot
Zauba
zgrab
Zitebot
ZumBot
ZyBorg