404checker
80legs
Acunetix
AhrefsBot
AllSubmitter
Alligator
Ankit
Asterias
Atomseobot
AwarioRssBot
BBBike
BDFetch
BLEXBot
BackDoorBot
BackWeb
BatchFTP
Bitacle
Black Hole
Blackboard
Blow
BotALot
Brandwatch
Buddy
BuzzSumo
CCBot
ChinaClaw
Chlooe
Citoid
Claritybot
Cloud mapping
Cogentbot
Collector
Craftbot
Crawling at Home Project
CrazyWebCrawler
Custo
DBLBot
Demon
Deusu
Devil
DigitalPebble
Dirbuster
Discoverybot
Dispatch
DnBCrawler-Analytics
DnyzBot
DomainSigmaCrawler
DomainStatsBot
Dragonfly
Drip
EMail Wolf
Exabot
Express WebPictures
ExtractorPro
Extreme Picture Finder
Ezooms
FDM
Fimap
G-i-g-a-b-o-t
Genieo
GetRight
Go-Ahead-Got-It
GrabNet
Grabber
GrapeFX
HTTP::Lite
HTTrack
Heritrix
Hloader
HonoluluBot
Humanlinks
HybridBot
IDBot
IRLbot
Iblog
Id-search
IlseBot
Image Fetch
Image Sucker
InfoNaviRobot
Internet Ninja
Iskanie
IstellaBot
Jbrofuzz
JennyBot
Jetty
JustView
Jyxobot
LWP::Simple
Lanshanbot
Larbin
Leap
LexiBot
LibWeb
LieBaoFast
LinkScan
LinkWalker
Linkbot
LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Litemage_walker
Lmspider
MFC_Tear_Sample
MTRobot
MarkWatch
Masscan
Mata Hari
MauiBot
Mb2345Browser
Metauri
Microsoft Data Access
Microsoft URL Control
Minefield
Msrabot
Musobot
NICErsPRO
NetSpider
Netcraft
Netvibes
Niki-bot
Nikto
Octopus
Offline Explorer
OpenLinkProfiler
OpenVAS
Openfind
Openvas
OrangeBot
POE-Component-Client-HTTP
PageAnalyzer
PageScorer
PageThing.com
Papa Foto
Pavuk
PictureFinder
Pixray
PleaseCrawl
Psbot
Pu_iN
PxBroker
PyCurl
RankFlex
RankingBot
RealDownload
Reaper
Recorder
RedesScrapy
RocketCrawler
Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler
SEOstats
SMTBot
ScanAlert
Scrapy
Screaming
ScrepyBot
Semrush
SemrushBot
SeobilityBot
Shodan
SiteExplorer
SiteLockSpider
Siteimprove
Sitevigil
SlySearch
SmartDownload
Snake
SocialRankIOBot
Sogou web spider
Sottopop
SpankBot
SputnikBot
Sqlmap
Sqlworm
Sqworm
Steeler
SurveyBot
Szukacz
Teleport
The Intraformant
TightTwatBot
Titan
Trendiction
Trendictionbot
True_Robot
TwengaBot
Typhoeus
UnisterBot
VB Project
VelenPublicWebCrawler
Virusdie
VoidEYE
Voil
Voltron
WBSearchBot
WWW-Mechanize
Wallpapers/3.0
WeSEE
Web Auto
Web Fuck
WebBandit
WebEnhancer
WebFuck
WebGo IS
WebImageCollector
WebPix
WebReaper
WebSucker
WebZIP
Webshag
Webster
Who.is Bot
Widow
WinHTTrack
WiseGuys Robot
Wotbox
Wprecon
Xaldon WebSpider
Xenu
YoudaoBot
Zade
ZumBot
ZyBorg
archive.org_bot
cah.io.community
clark-crawler
com.plumanalytics
dataforseo.com
demandbase-bot
facebookscraper
heritrix
instabid
ips-agent
isitwp.com
lwp-request
magpie-crawler
mediawords
muhstik-scan
netEstate NE Crawler
s1z.ru
seoscanners
sexsearcher
spyfu
ubermetrics-technologies.com
voyagerx.com
webpros.com
webprosbot