360Spider
80legs
Abonti
Aboundexbot
AhrefsBot
Aipbot
Alexibot
Anarchie
Apexoo
Backlink-Ceck
backlink-check
BackWeb
BatchFTP
Battleztar Bazinga
BBBike
BDFetch
BetaBot
Bigfoot
Blackboard
Boardreader
Bolt
BotALot
Brandprotect
Buddy
BuiltWith
Bullseye
Calculon
CCBot
CheeseBot
CherryPicker
ChinaClaw
Cliqzbot
coccocbot-web
cognitiveseo
com.plumanalytics
Copier
crawler4j
crawler.feedback
CrazyWebCrawler
Crescent
DatabaseDriverMysqli
DataCha0s
DBLBot
Devil
DigitalPebble
Discobot
DittoSpyder
DomainSigmaCrawler
DomainStatsBot
DSearch
DTS Agent
eCatch
ECCP/1.0
Ecxi
EMail Siphon
Exabot
Extractor
ExtractorPro
Extreme Picture Finder
EyeNetIE
Ezooms
facebookscraper
Fimap
FlashGet
Freeuploader
FrontPage
FyberSpider
Fyrebot
GalaxyBot
Getintent
GetRight
Gigabot
Gotit
Go!Zilla
Grabber
GrapeFX
GridBot
Harvest
Heritrix
HMView
HTTrack
Humanlinks
HybridBot
Iblog
IDBot
IlseBot
IndeedBot
Indy Library
InfoNaviRobot
InfoTekies
instabid
InterGET
Iria
JamesBOT
Jbrofuzz
JetCar
JikeSpider
JOC Web Spider
JustView
Keyword Density
Kozmosbot
Larbin
LeechGet
LexiBot
Lftp
LibWeb
Libwhisker
Lightspeedsystems
LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Spider
Lmspider
LNSpiderguy
lwp-request
LWP::Simple
magpie-crawler
Mail.RU_Bot
Majestic12
MarkMonitor
MarkWatch
Mata Hari
meanpathbot
MeanPath Bot
Mediatoolkitbot
Metauri
MFC_Tear_Sample
Microsoft Data Access
Mister PiX
MJ12bot
Mr.4x3
muhstik-scan
Name Intelligence
Nameprotect
Nessus
NetSpider
Net Vampire
NICErsPRO
Niki-bot
NimbleCrawler
Ninja
Nmap
NPbot
oBot
Octopus
Openfind
OrangeSpider
OutclicksBot
OutfoxBot
PageAnalyzer
Pandalytics
Papa Foto
PECL::HTTP
PeoplePal
Picsearch
Pi-Monster
PleaseCrawl
plumanalytics
POE-Component-Client-HTTP
Pump
QueryN Metasearch
RankActive
RankActiveLinkBot
RankingBot
RankingBot2
Rankivabot
Recorder
Scanbot
Scrapy
Screaming
SemrushBot
SEOkicks
SEOlyticsCrawler
Seomoz
SEOprofiler
seoscanners
SeoSiteCheckup
Shodan
SISTRIX
SiteLockSpider
SiteSnagger
SiteSucker
SmartDownload
Snake
Sogou web spider
Sosospider
Sottopop
SpankBot
Spanner
sp_auditbot
SputnikBot
spyfu
Sqlmap
Sqlworm
Sqworm
Steeler
Stripper
Sucker
SuperBot
Surfbot
SurveyBot
Suzuran
Swiftbot
sysscan
tAkeOut
Telesphoreo
Telesphorep
The Intraformant
TheNomad
TightTwatBot
Titan
Toweyabot
Tracemyfile
Trendiction
Turingos
Turnitin
Twice
URLy.Warning
Vacuum
Vagabondo
VCI
VidibleScraper
WASALive-Bot
WBSearchBot
Web Auto
Web Collage
WebCopier
Web Enhancer
Web Fetch
Web Fuck
WebImageCollector
WebLeacher
webmeup-crawler
WebPix
Web Sauger
Webster
WebZIP
Whack
Whacker
WiseGuys Robot
Wotbox
WPScan
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
x22Mozilla
YoudaoBot
Zade
ZumBot
ZyBorg