360Spider
404checker
Abonti
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
AllSubmitter
AlphaBot
arquivo-web-crawler
Aspiegel
AwarioSmartBot
BacklinkCrawler
BackStreet
Bandit
Battleztar Bazinga
BBBike
BDCbot
BetaBot
Blackboard
Blow
Brandprotect
Brandwatch
Buddy
BunnySlippers
Calculon
Cegbfeieh
CheeseBot
Collector
Copier
Cosmos
crawler4j
crawler.feedback
demandbase-bot
Devil
DIIbot
Discobot
Discoverybot
DittoSpyder
DomainSigmaCrawler
domainsproject.org
DomainStatsBot
Dotbot
Download Wonder
DSearch
eCatch
Ecxi
EirGrabber
EMail Siphon
evc-batch
Evil
ExtLinksBot
ExtractorPro
EyeNetIE
FDM
FHscan
FlashGet
Flunky
Foobot
FyberSpider
Genieo
Getintent
G-i-g-a-b-o-t
Go-Ahead-Got-It
GoZilla
GrapeshotCrawler
GridBot
HaosouSpider
Harvest
Havij
Hloader
HTTP::Lite
HTTrack
Humanlinks
HybridBot
Iblog
IDBot
Id-search
Image Sucker
IndeedBot
Indy Library
instabid
InternetSeer
internetVista monitor
ips-agent
Iria
isitwp.com
IstellaBot
JetCar
Jetty
JikeSpider
Joomla
Jorgee
JustView
Jyxobot
Kinza
Kozmosbot
Lanshanbot
Larbin
LeechFTP
LeechGet
LexiBot
Lftp
LibWeb
LieBaoFast
Likse
LinkScan
LinksManager
LinkWalker
Lipperhey Spider
Litemage_walker
LNSpiderguy
Magnet
Mag-Net
Mail.RU_Bot
MarkMonitor
MarkWatch
Masscan
Meanpathbot
Metauri
Microsoft URL Control
MJ12bot
Mojolicious
Mozlila
MQQBrowser
Mr.4x3
MSFrontPage
Msrabot
Name Intelligence
Nameprotect
NearSite
netEstate NE Crawler
NetLyzer
NetSpider
Nettrack
Netvibes
NICErsPRO
NimbleCrawler
Nimbostratus
Ninja
Offline Navigator
Openfind
OpenVAS
OPPO A33
OrangeSpider
OutclicksBot
Page Analyzer
PageScorer
Pandalytics
Pavuk
PeoplePal
PHPCrawl
Picscout
PictureFinder
Pimonster
Pixray
PleaseCrawl
plumanalytics
POE-Component-Client-HTTP
Reaper
RebelMouse
RepoMonkey
Ripper
RocketCrawler
Rogerbot
satoristudio.net
SBIder
Scrapy
ScreenerBot
SearchmetricsBot
SEOkicks
Seomoz
SeoSiteCheckup
serpstatbot
sexsearcher
Sitebeam
SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger
SiteSucker
Site Sucker
Sitevigil
Snake
Snoopy
sogouspider
Sogou web spider
Sottopop
SpaceBison
Spammen
sp_auditbot
Spinn3r
SputnikBot
spyfu
Sqlmap
Sucker
SuperBot
Surfbot
SurveyBot
Suzuran
Szukacz
T0PHackTeam
Teleport
TeleportPro
Telesphoreo
The Intraformant
TheNomad
TightTwatBot
Toata
Tracemyfile
Trendiction
True_Robot
Turnitin
TwengaBot
Twice
UnisterBot
Upflow
URLy Warning
URLy.Warning
Vagabondo
VB Project
VCI
VeriCiteCrawler
VidibleScraper
VoidEYE
Voltron
WallpapersHD
WASALive-Bot
WBSearchBot
Webalta
Web Auto
WebBandit
Web Collage
WebImageCollector
WebLeacher
WebmasterWorldForumBot
webmeup-crawler
WebPix
Webshag
Webster
Web Sucker
WebWhacker
Whatweb
Widow
WinHTTrack
WISENutbot
WPScan
WWW::Mechanize
WWWOFFLE
Xenu
Zade
ZyBorg