360Spider
80legs
Aboundexbot
Acunetix
AfD-Verbotsverfahren
Alexibot
AllSubmitter
AlphaBot
Anarchy99
Anthill
Apexoo
Asterias
AwarioRssBot
BDCbot
BDFetch
BackStreet
BackWeb
Badass
Barkrowler
BatchFTP
Battleztar Bazinga
BetaBot
Bitacle
Black Hole
BlackWidow
Blackboard
Bolt
Brandprotect
Buddy
BuzzSumo
Chlooe
Claritybot
Copier
CopyRightCheck
CrunchBot
Custo
CyotekWebCopy
DTS Agent
Demon
Digincore
DigitalPebble
Discobot
DittoSpyder
DomainAppender
Dotbot
Download Wonder
ECCP/1.0
EasyDL
Ebingbong
EirGrabber
Extractor
Extreme Picture Finder
Firefox/7.0
Flunky
Foobot
FrontPage
Fuzz
Fyrebot
Gigablast
Gigabot
Go!Zilla
Go-Ahead-Got-It
GrabNet
Grabber
GrapeFX
GridBot
HEADMasterSEO
HMView
HTMLparser
HTTrack
Harvest
Heritrix
Humanlinks
IRLbot
Iblog
IlseBot
Indy Library
InfoTekies
InterGET
Iria
JennyBot
Jorgee
Kinza
LWP::Simple
Lanshanbot
LeechGet
Libwhisker
Likse
LinksManager
Lipperhey
MFC_Tear_Sample
MJ12bot
Majestic SEO
MauiBot
MeanPath Bot
Mediatoolkitbot
MegaIndex.ru
Metauri
Mister PiX
Moblie Safari
Mojeek
Mojolicious
Mozlila
Musobot
Navroad
NearSite
Net Vampire
NetAnts
NetMechanic
NetSpider
Netvibes
Nibbler
Niki-bot
Nikto
Nmap
Openfind
Openvas
OrangeBot
OrangeSpider
OutclicksBot
OutfoxBot
PECL::HTTP
PageScorer
Pandalytics
Pavuk
Petalbot
Pixray
PleaseCrawl
ProWebWalker
Probethenet
Pump
PxBroker
Quick-Crawler
RSSingBot
RankActive
RankingBot
RankingBot2
Rankivabot
RankurBot
Re-re
Recorder
Ripper
SBIder
SEOprofiler
SISTRIX
SalesIntelligent
ScanAlert
ScoutJet
Searchestate
SearchmetricsBot
Semrush
SeoSiteCheckup
SiteCheckerBotCrawler
SiteExplorer
SiteSnagger
Sitebeam
Sitevigil
SlySearch
SmartDownload
Snoopy
SocialRankIOBot
Sogou web spider
SpaceBison
Spammen
Spinn3r
SputnikBot
Sqlmap
Sqworm
Stripper
Sucker
Szukacz
Teleport
Telesoft
Telesphoreo
Telesphorep
Thumbor
Toata
Toweyabot
Trendictionbot
True_Robot
TwengaBot
Twice
URLy Warning
URLy.Warning
Upflow
V-BOT
VCI
Vacuum
VidibleScraper
VoidEYE
Voil
Voltron
WASALive-Bot
WEBDAV
WPScan
WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0
WeSEE
Web Fetch
Web Fuck
Web Sauger
Web Sucker
WebBandit
WebFetch
WebFuck
WebPix
WebReaper
WebSauger
WebSucker
WebWhacker
Webshag
Whacker
WinHTTrack
Woobot
YoudaoBot
Zade
Zermelo
Zitebot
archive.org_bot
arquivo-web-crawler
arquivo.pt
autoemailspider
backlink-check
coccocbot-web
crawl.sogou.com
demandbase-bot
domainsproject.org
eCatch
evc-batch
facebookscraper
gopher
instabid
lwp-request
mediawords
netEstate NE Crawler
oBot
pcBrowser
polaris version
satoristudio.net
scalaj-http
scan.lol
seobility
seostar
serpstatbot
sitechecker.pro
webmeup-crawler
x09Mozilla
xpymep1.exe
zgrab