404checker
404enemy
Abonti
Aboundexbot
ADmantX
AhrefsBot
AIBOT
AiHitBot
Anarchie
Apexoo
arquivo.pt
Aspiegel
Asterias
autoemailspider
BackDoorBot
Backlink-Ceck
backlink-check
BacklinkCrawler
BackStreet
Badass
Bandit
Barkrowler
BBBike
BDCbot
BDFetch
BetaBot
Blackboard
Blow
Boardreader
Bolt
BotALot
Brandwatch
Buddy
BuiltBotTough
Calculon
CheTeam
Claritybot
Cliqzbot
Cloud mapping
coccocbot-web
cognitiveseo
Collector
Copier
Copyscape
crawler.feedback
CrazyWebCrawler
Crescent
DataCha0s
DBLBot
DigitalPebble
Dirbuster
Disco
Discobot
Discoverybot
Dispatch
DittoSpyder
DnyzBot
DomainStatsBot
Dotbot
Download Wonder
Drip
DSearch
DTS Agent
EasyDL
Ebingbong
Ecxi
EirGrabber
EMail Siphon
Express WebPictures
ExtLinksBot
Extractor
ExtractorPro
Ezooms
facebookscraper
Fimap
Firefox/7.0
Flunky
Freeuploader
FrontPage
FyberSpider
GalaxyBot
Genieo
Getintent
Gigablast
Gigabot
GoZilla
Go!Zilla
GrabNet
GridBot
GT::WWW
Haansoft
Havij
HEADMasterSEO
HTTP::Lite
Iblog
Id-search
IlseBot
IndeedBot
Indy Library
InfoTekies
Intelliseek
Iria
IRLbot
JamesBOT
JennyBot
Jetty
JOC Web Spider
Kenjin Spider
Kinza
Lanshanbot
LexiBot
LibWeb
LieBaoFast
Likse
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Ltx71
Magnet
Mag-Net
Majestic SEO
MarkWatch
Masscan
Mata Hari
MauiBot
MeanPath Bot
MegaIndex.ru
Microsoft URL Control
MIDown tool
MIIxpc
Morfeus Fucking Scanner
MSFrontPage
MSIECrawler
Navroad
NearSite
Nessus
NetAnts
Netcraft
NetLyzer
NetSpider
Nettrack
Net Vampire
Netvibes
NextGenSearchBot
NimbleCrawler
Nimbostratus
Ninja
Offline Navigator
Openvas
OutclicksBot
Papa Foto
pcBrowser
Pimonster
Pockey
polaris version
Psbot
PyCurl
RankActive
RankActiveLinkBot
RankFlex
RebelMouse
Recorder
ReGet
Ripper
s1z.ru
SalesIntelligent
SBIder
ScanAlert
Scanbot
scan.lol
Screaming
SemrushBot
SEOkicks
SEOkicks-Robot
SEOstats
SISTRIX
Site Sucker
Sitevigil
SlySearch
SmartDownload
Snapbot
Snoopy
SocialRankIOBot
sogouspider
Sogou web spider
Spanner
sp_auditbot
Sqlmap
Sqlworm
Stripper
Sucker
Sucuri
SuperHTTP
SurveyBot
Suzuran
Szukacz
tAkeOut
TeleportPro
Telesphoreo
TightTwatBot
Titan
Toweyabot
Trendiction
Trendictionbot
Typhoeus
UnisterBot
Upflow
URLy.Warning
VB Project
VCI
VeriCiteCrawler
VidibleScraper
VoidEYE
Voil
Voltron
WBSearchBot
WebAuto
WebCollage
Web Collage
WEBDAV
Web Fetch
WebFuck
Web Fuck
WebGo IS
WebImageCollector
WebLeacher
WebmasterWorldForumBot
webmeup-crawler
WebReaper
Webshag
Web Sucker
WeSEE
Whatweb
Widow
WiseGuys Robot
WISENutbot
Woobot
WWW-Collector-E
WWWOFFLE
x09Mozilla
x22Mozilla
Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot
zauba.io
Zermelo
Zeus
ZyBorg