360Spider
404checker
404enemy
80legs
Abonti
Acunetix
ADmantX
AiHitBot
Alligator
AllSubmitter
Apexoo
autoemailspider
backlink-check
BacklinkCrawler
BackStreet
Badass
Barkrowler
BBBike
BDCbot
Blow
BlowFish
Boardreader
Brandprotect
Buddy
BuiltBotTough
Bullseye
BunnySlippers
BuzzSumo
CATExplorador
CCBot
Cegbfeieh
CheeseBot
CherryPicker
CheTeam
ChinaClaw
Chlooe
Claritybot
Cliqzbot
Cloud mapping
Collector
com.plumanalytics
Copyscape
Cosmos
crawl.sogou.com
CrazyWebCrawler
Crescent
CSHttp
DBLBot
demandbase-bot
Demon
Devil
DigitalPebble
Disco
Discoverybot
DittoSpyder
DomainStatsBot
Dotbot
EasyDL
eCatch
EroCrawler
evc-batch
ExtLinksBot
ExtractorPro
EyeNetIE
Ezooms
FDM
FHscan
Firefox/7.0
Freeuploader
Fyrebot
GalaxyBot
Gotit
GoZilla
Grafula
GrapeFX
GridBot
GT::WWW
Haansoft
Harvest
Havij
HEADMasterSEO
Hloader
HTTP::Lite
HTTrack
Iblog
Image Sucker
instabid
Intelliseek
Internet Ninja
ips-agent
IRLbot
Iskanie
Jbrofuzz
JennyBot
JetCar
Jetty
JOC Web Spider
Jorgee
Jyxobot
Kozmosbot
LexiBot
Lftp
Libwhisker
Lightspeedsystems
Likse
Linkdexbot
LinkpadBot
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lmspider
LNSpiderguy
Ltx71
lwp-request
Magnet
Mag-Net
Mass Downloader
meanpathbot
Mediatoolkitbot
mediawords
Metauri
Microsoft URL Control
MIDown tool
Mojeek
Morfeus Fucking Scanner
Mr.4x3
MSIECrawler
MS Web Services Client Protocol
Musobot
Nameprotect
Needle
NetAnts
NetMechanic
Nibbler
Offline Explorer
Openfind
OutclicksBot
PageAnalyzer
PageGrabber
page scorer
PageScorer
Pavuk
pcBrowser
PECL::HTTP
PHPCrawl
Picsearch
PleaseCrawl
Pockey
POE-Component-Client-HTTP
ProPowerBot
ProWebWalker
Psbot
Pump
PyCurl
RankActive
RankActiveLinkBot
RankurBot
RealDownload
Reaper
RebelMouse
RedesScrapy
Ripper
RocketCrawler
Rogerbot
SalesIntelligent
SBIder
ScanAlert
Scanbot
scan.lol
Scrapy
Screaming
Searchestate
SearchmetricsBot
Semrush
SemrushBot
SEOlyticsCrawler
SEOprofiler
seoscanners
SeoSiteCheckup
sexsearcher
Shodan
Siphon
Sitebeam
Siteimprove
Snake
Sogou web spider
sp_auditbot
Spbot
Spinn3r
spyfu
Sqlmap
Steeler
Stripper
Sucker
Sucuri
Surfbot
SurveyBot
Szukacz
T0PHackTeam
Teleport
TeleportPro
Telesphoreo
TheNomad
Titan
Toweyabot
Trendiction
trendiction.com
True_Robot
UnisterBot
Upflow
Vacuum
VB Project
Virusdie
Voil
Voltron
Wallpapers/3.0
WBSearchBot
WebAuto
Web Auto
WebBandit
WebCollage
WebCopier
Web Enhancer
Web Fetch
WebFuck
WebmasterWorldForumBot
webmeup-crawler
WebPix
WebSauger
WebsiteQuester
Webster
WebStripper
Web Sucker
WebWhacker
WebZIP
WeSEE
Whacker
WinHTTrack
Wonderbot
Wprecon
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
x09Mozilla
YoudaoBot
Zade
zauba.io
Zermelo
Zeus
Zitebot