404checker
404enemy
Abonti
Aboundexbot
ADmantX
AfD-Verbotsverfahren
AhrefsBot
Alexibot
Anarchie
archive.org_bot
Asterias
backlink-check
BackStreet
BackWeb
Bandit
BDFetch
BlowFish
Brandwatch
Buddy
BuiltWith
BuzzSumo
Calculon
CATExplorador
CazoodleBot
CCBot
Cegbfeieh
Chlooe
Cliqzbot
Cloud mapping
coccocbot-web
Cogentbot
com.plumanalytics
Copier
Craftbot
crawler4j
crawler.feedback
Crescent
CrunchBot
CSHttp
DatabaseDriverMysqli
Demon
Deusu
Digincore
Dispatch
DittoSpyder
DnyzBot
DomainCrawler
Dragonfly
DSearch
DTS Agent
EasyDL
Ebingbong
eCatch
ECCP/1.0
evc-batch
Express WebPictures
Extractor
FDM
Fimap
Fyrebot
GermCrawler
GetRight
Gigabot
Gotit
Go!Zilla
GrapeFX
GrapeshotCrawler
Haansoft
Harvest
Havij
HEADMasterSEO
Hloader
HTMLparser
HTTrack
HybridBot
Id-search
Image Fetch
Image Sucker
Indy Library
InterGET
internetVista monitor
ips-agent
IstellaBot
JamesBOT
Jbrofuzz
JennyBot
JetCar
JikeSpider
JOC Web Spider
Jyxobot
Kozmosbot
Lanshanbot
Lftp
Libwhisker
LinkextractorPro
LinkScan
LinksManager
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider
Lmspider
lwp-request
Magnet
magpie-crawler
Mail.RU_Bot
Majestic12
Majestic SEO
Majestic-SEO
Masscan
Mass Downloader
Mata Hari
Mediatoolkitbot
mediawords
MegaIndex.ru
Metauri
Microsoft URL Control
MIIxpc
Mister PiX
MSFrontPage
MSIECrawler
Msrabot
Musobot
Name Intelligence
Needle
Nessus
NetAnts
Netcraft
NetLyzer
NetMechanic
Nettrack
Netvibes
NICErsPRO
Niki-bot
Nikto
Ninja
Nutch
oBot
Offline Navigator
OnCrawl
Openfind
Openvas
OrangeBot
OrangeSpider
OutclicksBot
Page Analyzer
PageGrabber
Pandalytics
Panscient
Papa Foto
Pavuk
pcBrowser
PHPCrawl
Picsearch
PictureFinder
plumanalytics
POE-Component-Client-HTTP
ProWebWalker
Pump
PxBroker
PyCurl
QueryN Metasearch
Quick-Crawler
RankActiveLinkBot
RankFlex
RankingBot
Reaper
RedesScrapy
Ripper
RocketCrawler
RSSingBot
SalesIntelligent
Scrapy
ScreenerBot
Searchestate
SemrushBot
SEOkicks-Robot
SEOlyticsCrawler
Seomoz
SEOprofiler
SeoSiteCheckup
SEOstats
sexsearcher
Siphon
Sitebeam
SiteExplorer
SiteLockSpider
SiteSucker
Sitevigil
SmartDownload
Snake
Snapbot
SocialRankIOBot
Sogou web spider
Sottopop
sp_auditbot
SputnikBot
Sqlworm
Stripper
Sucuri
SuperBot
SuperHTTP
SurveyBot
sysscan
tAkeOut
TeleportPro
Telesphoreo
Telesphorep
Toata
trendiction.de
Turingos
Turnitin
Typhoeus
Upflow
Vacuum
VCI
VeriCiteCrawler
VidibleScraper
Voltron
WASALive-Bot
WBSearchBot
Webalta
Web Auto
WebBandit
WebCopier
WEBDAV
WebEnhancer
Web Enhancer
WebImageCollector
WebLeacher
WebPix
Web Pix
WebReaper
Webster
WebSucker
WebZIP
Whack
Widow
WinHTTrack
Wonderbot
Wprecon
WPScan
WWW-Mechanize
YoudaoBot
Zade
Zauba
Zermelo
Zeus
Zitebot
ZumBot