360Spider
404checker
404enemy
80legs
Abonti
Aboundex
Acunetix
ADmantX
AiHitBot
Alligator
BackDoorBot
backlink-check
BackStreet
Badass
BatchFTP
BDFetch
Blackboard
Black Hole
Blow
BlowFish
Boardreader
Bolt
Brandprotect
Brandwatch
BuiltWith
BuzzSumo
Calculon
CazoodleBot
CheeseBot
CherryPicker
ChinaClaw
Claritybot
Cliqzbot
Cogentbot
Collector
com.plumanalytics
crawler.feedback
crawl.sogou.com
demandbase-bot
Demon
Deusu
DigitalPebble
DIIbot
Dirbuster
Disco
DomainAppender
DomainCrawler
DomainStatsBot
Drip
DSearch
EasyDL
Ebingbong
Ecxi
EirGrabber
EroCrawler
evc-batch
Evil
Exabot
ExtLinksBot
ExtractorPro
EyeNetIE
FDM
FHscan
Firefox/7.0
FyberSpider
Fyrebot
GermCrawler
GetRight
G-i-g-a-b-o-t
Go-Ahead-Got-It
Gotit
Go!Zilla
Grabber
Grafula
HaosouSpider
Harvest
Havij
HEADMasterSEO
HTTP::Lite
Humanlinks
Indy Library
InfoNaviRobot
InfoTekies
Iria
Iskanie
IstellaBot
JamesBOT
JennyBot
Jetty
JikeSpider
Kozmosbot
Lanshanbot
Libwhisker
Likse
LinkextractorPro
LinkScan
LinkWalker
LinqiaMetadataDownloaderBot
lwp-trivial
Magnet
Mag-Net
Mail.RU_Bot
Majestic-SEO
MarkWatch
Mass Downloader
MauiBot
Meanpathbot
Mediatoolkitbot
mediawords
MegaIndex.ru
Microsoft Data Access
MIDown tool
MIIxpc
MSFrontPage
MSIECrawler
Msrabot
Name Intelligence
Navroad
Needle
Nessus
NetLyzer
NetMechanic
Net Vampire
NetZIP
NICErsPRO
NimbleCrawler
Nimbostratus
Ninja
Octopus
OpenVAS
Page Analyzer
Pandalytics
Papa Foto
Pavuk
PECL::HTTP
PHPCrawl
Pi-Monster
Pixray
PleaseCrawl
Pockey
Probethenet
ProWebWalker
Psbot
Pump
PxBroker
PyCurl
QueryN Metasearch
Quick-Crawler
RankingBot
Rankivabot
RealDownload
RebelMouse
RedesScrapy
ScanAlert
Scanbot
ScoutJet
Scrapy
Searchestate
SearchmetricsBot
SEOkicks-Robot
SEOprofiler
SeoSiteCheckup
sexsearcher
Shodan
Siphon
SISTRIX
Sitebeam
SiteExplorer
Siteimprove
Site Sucker
SlySearch
Snake
Sociscraper
Sogou web spider
Sosospider
Sottopop
Spanner
Spbot
Spinn3r
SputnikBot
Sqlworm
Sucuri
SuperBot
Surfbot
Suzuran
Swiftbot
sysscan
T8Abot
Teleport
TeleportPro
Telesphoreo
Telesphorep
TheNomad
Thumbor
TightTwatBot
Titan
Toata
Trendictionbot
trendiction.com
trendiction.de
True_Robot
Turingos
TwengaBot
Twice
Vacuum
VB Project
VeriCiteCrawler
Virusdie
VoidEYE
Voil
Wallpapers/3.0
WallpapersHD
WASALive-Bot
WBSearchBot
Web Auto
WebCollage
WebCopier
Web Enhancer
WebFetch
Web Fetch
WebFuck
Web Fuck
WebLeacher
webmeup-crawler
WebPix
WebReaper
WebSauger
WebsiteExtractor
WebsiteQuester
Website Quester
WebStripper
WebSucker
Web Sucker
WeSEE
Whatweb
WinHTTrack
WISENutbot
Wonderbot
Wotbox
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Xaldon_WebSpider
Xenu
xpymep1.exe
YoudaoBot
zauba.io
zgrab
ZumBot