360Spider
80legs
ADmantX
AiHitBot
Aipbot
Alligator
AllSubmitter
AlphaBot
ASPSeek
Asterias
autoemailspider
BackWeb
Badass
Barkrowler
BatchFTP
BDCbot
BetaBot
Bigfoot
Black Hole
Blow
BlowFish
BotALot
Brandwatch
Buddy
BuzzSumo
CazoodleBot
Cegbfeieh
CheeseBot
CheTeam
Claritybot
Cloud mapping
cognitiveseo
Collector
com.plumanalytics
CopyRightCheck
Copyscape
crawler4j
crawl.sogou.com
CrazyWebCrawler
CrunchBot
Curious
DatabaseDriverMysqli
demandbase-bot
Deusu
Devil
Digincore
DIIbot
Disco
Dispatch
DittoSpyder
DnyzBot
DomainCrawler
Download Wonder
DSearch
DTS Agent
Ebingbong
eCatch
ECCP/1.0
EMail Siphon
EMail Wolf
EroCrawler
evc-batch
Evil
Exabot
Express WebPictures
ExtLinksBot
ExtractorPro
FemtosearchBot
Firefox/7.0
Fyrebot
GalaxyBot
Genieo
GetWeb
Gigablast
G-i-g-a-b-o-t
Grabber
GrapeFX
GrapeshotCrawler
GT::WWW
Harvest
HEADMasterSEO
HMView
HTMLparser
HTTP::Lite
Humanlinks
HybridBot
IDBot
Id-search
Image Sucker
IndeedBot
Indy Library
InfoNaviRobot
Internet Ninja
InternetSeer
internetVista monitor
ips-agent
JamesBOT
Jbrofuzz
JennyBot
Jorgee
JustView
Kozmosbot
Lanshanbot
Larbin
Lightspeedsystems
LinkextractorPro
Lipperhey
Litemage_walker
Lmspider
lwp-request
lwp-trivial
Magnet
Majestic12
Majestic SEO
MarkMonitor
Mass Downloader
Mata Hari
MeanPath Bot
MFC_Tear_Sample
Microsoft URL Control
Mojeek
Morfeus Fucking Scanner
MSFrontPage
MSIECrawler
Msrabot
MS Web Services Client Protocol
Musobot
Nameprotect
NearSite
Netcraft
netEstate NE Crawler
NetLyzer
Nettrack
Nikto
NimbleCrawler
Nimbostratus
Ninja
Nmap
Nutch
Offline Explorer
Openfind
OpenLinkProfiler
Openvas
OpenVAS
PageAnalyzer
PeoplePal
PHPCrawl
Picscout
Picsearch
Pimonster
Pi-Monster
plumanalytics
POE-Component-Client-HTTP
Probethenet
PyCurl
QueryN Metasearch
RankingBot
RankurBot
RebelMouse
Ripper
RocketCrawler
SalesIntelligent
Scanbot
scan.lol
ScoutJet
Scrapy
Screaming
ScreenerBot
Searchestate
SEOlyticsCrawler
seoscanners
SeoSiteCheckup
SEOstats
sexsearcher
Siphon
SiteExplorer
SiteLockSpider
SiteSucker
Site Sucker
SlySearch
SmartDownload
SMTBot
Snoopy
SocialRankIOBot
Sociscraper
Sogou web spider
Sosospider
Sottopop
SpankBot
spyfu
Sqlworm
Sqworm
Steeler
Stripper
Sucker
Sucuri
SuperBot
Szukacz
The Intraformant
TheNomad
Thumbor
TightTwatBot
Titan
Toata
Toweyabot
Trendiction
True_Robot
Turnitin
TurnitinBot
UnisterBot
Upflow
Vacuum
Vagabondo
VCI
Virusdie
WallpapersHD
WASALive-Bot
Web Auto
WebBandit
WebCollage
WebCopier
Web Fetch
Web Fuck
WebImageCollector
WebLeacher
webmeup-crawler
Web Pix
WebReaper
WebSauger
Web Sauger
Webster
WebStripper
WebSucker
Web Sucker
WebZIP
WeSEE
Whack
Whacker
Whatweb
Who.is Bot
Wonderbot
Wprecon
WWWOFFLE
Xaldon WebSpider
YoudaoBot
Zade
zauba.io
Zermelo
zgrab
ZyBorg