360Spider
404checker
404enemy
Aboundexbot
AhrefsBot
AiHitBot
Alligator
AllSubmitter
AlphaBot
Anarchie
Apexoo
ASPSeek
Backlink-Ceck
backlink-check
BacklinkCrawler
BackWeb
BBBike
BDFetch
BetaBot
Blackboard
Black Hole
BlowFish
Boardreader
Bolt
BotALot
Brandwatch
BunnySlippers
BuzzSumo
CazoodleBot
CCBot
CheeseBot
CherryPicker
ChinaClaw
Chlooe
coccocbot-web
Copier
Copyscape
Cosmos
Craftbot
crawler4j
crawl.sogou.com
CrazyWebCrawler
Crescent
CSHttp
Custo
DatabaseDriverMysqli
DBLBot
demandbase-bot
Deusu
Devil
Digincore
DIIbot
Disco
Discobot
Discoverybot
Dotbot
Drip
DSearch
EasyDL
eCatch
Ecxi
EMail Siphon
EMail Wolf
evc-batch
Exabot
ExtLinksBot
Extractor
EyeNetIE
facebookscraper
FemtosearchBot
FHscan
Fimap
FlashGet
Flunky
Foobot
Freeuploader
FrontPage
FyberSpider
Getintent
Gigablast
Go-Ahead-Got-It
Gotit
GrabNet
HaosouSpider
Harvest
Havij
HMView
HTMLparser
HTTrack
Id-search
Image Fetch
Indy Library
InfoTekies
instabid
Intelliseek
InterGET
Internet Ninja
InternetSeer
JamesBOT
Jbrofuzz
JetCar
Jetty
JustView
Kenjin Spider
Keyword Density
LeechFTP
LexiBot
Lftp
LibWeb
Likse
Linkdexbot
LinqiaRSSBot
Lipperhey
Litemage_walker
lwp-trivial
Magnet
Mag-Net
Majestic12
Majestic SEO
MarkWatch
MauiBot
Mediatoolkitbot
MFC_Tear_Sample
Microsoft URL Control
MIDown tool
Mister PiX
Mojeek
MSFrontPage
MSIECrawler
Msrabot
MS Web Services Client Protocol
Nameprotect
Needle
netEstate NE Crawler
NetLyzer
NetSpider
Nettrack
Net Vampire
Netvibes
Nibbler
Niki-bot
Nikto
OnCrawl
OpenLinkProfiler
OpenVAS
OutfoxBot
PageAnalyzer
Page Analyzer
PictureFinder
Probethenet
ProWebWalker
Pump
Quick-Crawler
RankActiveLinkBot
RankingBot2
Rankivabot
RankurBot
Reaper
RebelMouse
RepoMonkey
RSSingBot
SBIder
Scrapy
ScreenerBot
SearchmetricsBot
Semrush
SemrushBot
SEOlyticsCrawler
Seomoz
SEOprofiler
seoscanners
SeoSiteCheckup
serpstatbot
Shodan
Siphon
SISTRIX
Sitebeam
SiteExplorer
SiteLockSpider
SiteSnagger
Site Sucker
SMTBot
Snapbot
Snoopy
SocialRankIOBot
sogouspider
Sogou web spider
Sosospider
sp_auditbot
SputnikBot
spyfu
Sqlworm
Sqworm
Steeler
Stripper
Sucuri
SuperBot
Surfbot
SurveyBot
sysscan
T0PHackTeam
TeleportPro
Telesphoreo
Telesphorep
The Intraformant
Titan
Toweyabot
Trendiction
Trendictionbot
True_Robot
Turingos
Turnitin
TwengaBot
URLy.Warning
VB Project
VidibleScraper
Voil
Voltron
WASALive-Bot
Webalta
WebAuto
Web Auto
WebCollage
Web Collage
WEBDAV
WebGo IS
WebImageCollector
WebLeacher
WebmasterWorldForumBot
WebReaper
WebSucker
Web Sucker
WebZIP
WeSEE
Whack
Whacker
Who.is Bot
WinHTTrack
WiseGuys Robot
WISENutbot
WPScan
x22Mozilla
Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot
Zade
zauba.io
Zitebot
ZmEu
ZumBot