404checker
80legs
Aboundexbot
Acunetix
ADmantX
AhrefsBot
AiHitBot
Alligator
AllSubmitter
AlphaBot
Anarchie
Apexoo
ASPSeek
Asterias
AwarioSmartBot
backlink-check
BackStreet
Bandit
BatchFTP
Battleztar Bazinga
BBBike
Bigfoot
Blackboard
Black Hole
BLEXBot
Boardreader
Bolt
Buddy
BuiltWith
Bullseye
BuzzSumo
CheeseBot
Chlooe
Cloud mapping
coccocbot-web
cognitiveseo
Collector
com.plumanalytics
CopyRightCheck
Cosmos
crawler.feedback
crawl.sogou.com
Crescent
CrunchBot
Curious
DataCha0s
DBLBot
Demon
DIIbot
Dispatch
DomainSigmaCrawler
DomainStatsBot
EasyDL
Ebingbong
eCatch
Ecxi
EirGrabber
Express WebPictures
EyeNetIE
FDM
Fimap
Firefox/7.0
FlashGet
FrontPage
GalaxyBot
Genieo
GetRight
Gigabot
G-i-g-a-b-o-t
Gotit
GrapeFX
GrapeshotCrawler
GridBot
GT::WWW
HaosouSpider
Havij
HEADMasterSEO
heritrix
Hloader
HTTP::Lite
Iblog
IDBot
IlseBot
IndeedBot
Indy Library
InfoNaviRobot
InfoTekies
instabid
Intelliseek
InterGET
ips-agent
IRLbot
IstellaBot
JamesBOT
JennyBot
JetCar
Jorgee
JustView
Jyxobot
Kinza
Kozmosbot
Larbin
LeechGet
Lftp
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Litemage_walker
Lmspider
LNSpiderguy
Ltx71
LWP::Simple
Mag-Net
magpie-crawler
Majestic12
Majestic SEO
Mata Hari
MeanPath Bot
mediawords
Mister PiX
Mojeek
Morfeus Fucking Scanner
MQQBrowser
Musobot
NearSite
Needle
Netcraft
NetMechanic
NetSpider
Net Vampire
Nibbler
Niki-bot
Nimbostratus
Nmap
NPbot
Nutch
oBot
OnCrawl
OpenLinkProfiler
Openvas
OpenVAS
OrangeBot
OutclicksBot
PageScorer
Panscient
Pavuk
pcBrowser
PeoplePal
PictureFinder
Pimonster
Pi-Monster
Pixray
plumanalytics
POE-Component-Client-HTTP
polaris version
ProWebWalker
RankActive
RankActiveLinkBot
RankFlex
RankingBot2
Rankivabot
RealDownload
Reaper
RebelMouse
RedesScrapy
Ripper
RSSingBot
SBIder
ScanAlert
Scanbot
scan.lol
ScoutJet
Scrapy
SEOkicks
SEOlyticsCrawler
SEOprofiler
SeoSiteCheckup
Shodan
Siphon
SISTRIX
Sitebeam
Siteimprove
SiteLockSpider
SlySearch
SmartDownload
Snapbot
Sociscraper
sogouspider
Sogou web spider
sp_auditbot
Spbot
Spinn3r
spyfu
Sqlmap
Sqworm
Steeler
Stripper
SurveyBot
Szukacz
T8Abot
Telesoft
Telesphoreo
The Intraformant
TheNomad
Toata
trendiction.de
True_Robot
Turingos
TwengaBot
URLy Warning
URLy.Warning
Vagabondo
VB Project
VoidEYE
Wallpapers/3.0
WallpapersHD
WebAuto
Web Auto
WebBandit
WebCollage
Web Collage
WebCopier
Web Enhancer
Web Fuck
WebGo IS
WebImageCollector
WebmasterWorldForumBot
webmeup-crawler
WebPix
Web Pix
Web Sauger
WebsiteExtractor
Website Quester
WebSucker
Web Sucker
WebWhacker
WebZIP
WeSEE
Whack
WiseGuys Robot
Wotbox
WPScan
WWW-Collector-E
WWWOFFLE
Xaldon WebSpider
Xenu
Zade
Zermelo
Zitebot