360Spider
404checker
404enemy
ALittle Client
ASPSeek
Abonti
AfD-Verbotsverfahren
AhrefsBot
AllSubmitter
Alligator
AlphaBot
Anarchie
Anarchy99
Ankit
AwarioSmartBot
BDFetch
BacklinkCrawler
Bandit
Battleztar Bazinga
Bitacle
Black Hole
Boardreader
Bolt
BunnySlippers
CATExplorador
CSHttp
CazoodleBot
CensysInspect
CheeseBot
CherryPicker
Cloud mapping
Craftbot
Crawling at Home Project
CrazyWebCrawler
Custo
DBLBot
DIIbot
DatabaseDriverMysqli
Demon
Dirbuster
Disco
Discobot
DnyzBot
DomCopBot
Download Wonder
EMail Siphon
Evil
Express WebPictures
ExtLinksBot
Extreme Picture Finder
FHscan
FemtosearchBot
Firefox/7.0
Flunky
G-i-g-a-b-o-t
Genieo
GetRight
GetWeb
Gigabot
Go!Zilla
Go-Ahead-Got-It
Gotit
GrabNet
Grabber
GrapeshotCrawler
HMView
HTTrack
Harvest
Havij
Hloader
Humanlinks
IDBTE4M
IRLbot
Iblog
IndeedBot
Intelliseek
InterGET
Internet Ninja
Iria
Iskanie
JamesBOT
JetCar
Jetty
Jorgee
Jyxobot
Larbin
Leap
LeechFTP
LeechGet
LexiBot
Lftp
LieBaoFast
Likse
Linkbot
LinkpadBot
LinksManager
Lmspider
Ltx71
MJ12bot
MTRobot
Mag-Net
Magnet
Mail.RU_Bot
Majestic SEO
Masscan
Mb2345Browser
MeanPath Bot
Meanpathbot
Mediatoolkitbot
Metauri
MicroMessenger
Microsoft Data Access
Mister PiX
Moblie Safari
Mojeek
Msrabot
Musobot
Navroad
NearSite
Needle
Nessus
NetAnts
NetLyzer
NetMechanic
Nettrack
NextGenSearchBot
Nikto
Not
Nuclei
Offline Explorer
OpenVAS
Openvas
OutfoxBot
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageGrabber
Pandalytics
Pavuk
PeoplePal
Pockey
Psbot
Pu_iN
Pump
Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankingBot
ReGet
RepoMonkey
Ripper
RocketCrawler
SEOkicks
SEOlyticsCrawler
SEOprofiler
SEOstats
SISTRIX
ScanAlert
ScoutJet
Scrapy
Screaming
ScrepyBot
SemanticJuice
SemrushBot
SentiBot
SeoSiteCheckup
Seomoz
Shodan
SiteLockSpider
SiteSucker
Sitebeam
Snake
Snapbot
Snoopy
Sogou web spider
Sottopop
Spbot
SputnikBot
SuperBot
Surfbot
SurveyBot
Swiftbot
Teleport
The Intraformant
Tracemyfile
Trendiction
Trendictionbot
True_Robot
TurnitinBot
TwengaBot
Twice
URLy Warning
Upflow
V-BOT
VB Project
VCI
Vacuum
Vagabondo
VeriCiteCrawler
VidibleScraper
Virusdie
WBSearchBot
WPScan
WWW-Mechanize
WWWOFFLE
Wallpapers
WeSEE
Web Enhancer
Web Fuck
Web Pix
Web Sauger
WebFetch
WebGo IS
WebImageCollector
WebReaper
WebSauger
Webster
Whack
WinHTTrack
Xaldon WebSpider
Xenu
YoudaoBot
Zade
ZmEu
ZoomBot
ZoominfoBot
adscanner
arquivo-web-crawler
autoemailspider
cah.io.community
coccocbot-web
crawler4j
dataforseo.com
eCatch
gopher
heritrix
instabid
ips-agent
meanpathbot
muhstik-scan
page scorer
pcBrowser
satoristudio.net
sitechecker.pro
tAkeOut
voyagerx.com
webmeup-crawler
webpros.com