360Spider
404enemy
80legs
ADmantX
ASPSeek
AiHitBot
AllSubmitter
Alligator
Anarchy
Anarchy99
Ankit
Anthill
Apexoo
Aspiegel
Atomseobot
AwarioRssBot
BDCbot
BDFetch
BackDoorBot
BackWeb
Backlink-Ceck
Barkrowler
Battleztar Bazinga
Bigfoot
Bitacle
BlackWidow
Blackboard
Blow
Bolt
Brandprotect
BuiltWith
BuzzSumo
CCBot
Cegbfeieh
CherryPicker
Chlooe
Cliqzbot
Cocolyzebot
Copier
Cosmos
Crawling at Home Project
Curious
Custo
CyotekWebCopy
DIIbot
DSearch
DTS Agent
DatabaseDriverMysqli
Demon
Dirbuster
Discobot
Dispatch
DittoSpyder
DomCopBot
DomainCrawler
DomainStatsBot
ECCP/1.0
EMail Siphon
EasyDL
Ecxi
Evil
ExtractorPro
EyeNetIE
FemtosearchBot
Fimap
Firefox/7.0
Flunky
FrontPage
GalaxyBot
GermCrawler
Getintent
Go-Ahead-Got-It
Grafula
GrapeshotCrawler
HTMLparser
HTTrack
Harvest
Havij
Heritrix
IDBot
IRLbot
Iblog
IlseBot
Image Fetch
IndeedBot
InfoNaviRobot
Intelliseek
InterGET
Internet Ninja
IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz
Jetty
Joomla
Jorgee
JustView
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
LWP::Simple
Lanshanbot
Leap
LeechGet
LieBaoFast
Lightspeedsystems
Likse
LinkWalker
Linkbot
LinksManager
LinqiaMetadataDownloaderBot
Lipperhey
Ltx71
MIDown tool
MJ12bot
MSFrontPage
MSIECrawler
Magnet
Majestic-SEO
MarkMonitor
Masscan
Mata Hari
Meanpathbot
Microsoft Data Access
Microsoft URL Control
Minefield
Morfeus Fucking Scanner
Msrabot
Musobot
NPbot
Needle
Net Vampire
NetLyzer
Nettrack
Nimbostratus
Nuclei
Openvas
OrangeBot
OutclicksBot
Page Analyzer
PageThing.com
PeoplePal
Pi-Monster
Picsearch
Pimonster
PleaseCrawl
Pockey
ProWebWalker
Probethenet
Psbot
PxBroker
Quick-Crawler
RSSingBot
RankingBot
ReGet
Recorder
RocketCrawler
Rogerbot
SEOkicks-Robot
SEOprofiler
SEOstats
ScoutJet
Scrapy
Screaming
SearchmetricsBot
SemanticJuice
Semrush
SeoSiteCheckup
Seomoz
Shodan
Site Sucker
SiteLockSpider
SiteSnagger
SiteSucker
Siteimprove
SlySearch
Sociscraper
Sogou web spider
Sosospider
Sottopop
SpaceBison
SpankBot
Spanner
Spinn3r
SputnikBot
Sqlmap
SuperBot
SuperHTTP
SurveyBot
Swiftbot
Szukacz
TeleportPro
Telesoft
Telesphoreo
TightTwatBot
Titan
Tracemyfile
TurnitinBot
TwengaBot
Twice
URLy Warning
URLy.Warning
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
WBSearchBot
WISENutbot
Web Pix
WebCollage
WebCopier
WebPix
WebSauger
WebStripper
WebWhacker
WebmasterWorldForumBot
Webshag
WebsiteQuester
Webster
Widow
Wonderbot
YoudaoBot
ZmEu
ZoominfoBot
ZumBot
arquivo-web-crawler
arquivo.pt
autoemailspider
backlink-check
cah.io.community
crawl.sogou.com
crawler.feedback
ips-agent
iubenda-radar
lwp-request
lwp-trivial
magpie-crawler
meanpathbot
mediawords
netEstate NE Crawler
ripz
scalaj-http
seoscanners
seostar
sogouspider
spyfu
trendiction.com
trendiction.de
webgains-bot
x22Mozilla
zauba.io