360Spider
404checker
404enemy
Abonti
ADmantX
AhrefsBot
AiHitBot
Aipbot
Alligator
AllSubmitter
AlphaBot
arquivo-web-crawler
Asterias
Attach
AwarioRssBot
AwarioSmartBot
BackDoorBot
BacklinkCrawler
Barkrowler
Battleztar Bazinga
BDFetch
Bitacle
Black Hole
BLEXBot
BlowFish
Boardreader
BotALot
Brandprotect
BuzzSumo
CATExplorador
Cegbfeieh
CheeseBot
CheTeam
Cogentbot
cognitiveseo
com.plumanalytics
Copyscape
Cosmos
crawler4j
crawler.feedback
CrunchBot
Curious
Custo
DatabaseDriverMysqli
DataCha0s
demandbase-bot
Digincore
DIIbot
Disco
Discobot
Discoverybot
DittoSpyder
DnyzBot
DomainAppender
DomainStatsBot
Download Wonder
DSearch
eCatch
EirGrabber
EroCrawler
evc-batch
Express WebPictures
ExtLinksBot
Extractor
FDM
FemtosearchBot
Fimap
Firefox/7.0
FrontPage
Genieo
GetRight
Go-Ahead-Got-It
GoZilla
Go!Zilla
Grafula
GrapeFX
GridBot
GT::WWW
Havij
HEADMasterSEO
HMView
HTMLparser
HTTrack
Humanlinks
IDBot
IlseBot
IndeedBot
Indy Library
InfoNaviRobot
instabid
Intelliseek
InterGET
internetVista monitor
Iria
IstellaBot
JetCar
Joomla
JustView
Kozmosbot
LeechFTP
LeechGet
Lftp
Likse
LinkextractorPro
LinkScan
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Litemage_walker
lwp-trivial
magpie-crawler
Majestic12
Majestic SEO
Masscan
Meanpathbot
MeanPath Bot
Mediatoolkitbot
MFC_Tear_Sample
Microsoft Data Access
Microsoft URL Control
MIDown tool
MIIxpc
Morfeus Fucking Scanner
Mr.4x3
MSFrontPage
MSIECrawler
muhstik-scan
Musobot
Name Intelligence
Navroad
Needle
Nessus
Netcraft
NetLyzer
NetMechanic
Net Vampire
Netvibes
Nibbler
Nikto
NimbleCrawler
NPbot
oBot
Offline Explorer
Offline Navigator
OnCrawl
OpenVAS
PageAnalyzer
PageGrabber
page scorer
PageScorer
Panscient
pcBrowser
Picscout
Picsearch
Pimonster
Pockey
POE-Component-Client-HTTP
Probethenet
ProPowerBot
ProWebWalker
PyCurl
QueryN Metasearch
Quick-Crawler
RankingBot
Rankivabot
RankurBot
RealDownload
Reaper
Recorder
ReGet
RepoMonkey
Rogerbot
s1z.ru
scan.lol
SEOkicks-Robot
SEOlyticsCrawler
SeoSiteCheckup
sexsearcher
Siteimprove
Sitevigil
SMTBot
Snake
Snapbot
Snoopy
SocialRankIOBot
Sogou web spider
Sosospider
Sottopop
Spammen
SpankBot
Spanner
Sqlmap
Stripper
Sucker
Sucuri
Swiftbot
T0PHackTeam
T8Abot
tAkeOut
TeleportPro
Telesphoreo
TheNomad
TightTwatBot
Titan
Toata
Toweyabot
Tracemyfile
True_Robot
Turnitin
TurnitinBot
VeriCiteCrawler
Virusdie
VoidEYE
Voil
WallpapersHD
WBSearchBot
WebAuto
Web Collage
WebCopier
WEBDAV
WebFetch
WebGo IS
WebImageCollector
WebLeacher
webmeup-crawler
WebPix
WebsiteExtractor
WebsiteQuester
Website Quester
Webster
WebStripper
WeSEE
Whack
Who.is Bot
Widow
WinHTTrack
WISENutbot
Woobot
Wotbox
Wprecon
WWW-Collector-E
WWWOFFLE
x09Mozilla
x22Mozilla
Xaldon_WebSpider
Xenu
YoudaoBot
Zade
Zauba
Zitebot
ZyBorg