404checker
404enemy
AhrefsBot
AllSubmitter
Anarchy99
Apexoo
Backlink-Ceck
BacklinkCrawler
BatchFTP
Battleztar Bazinga
Black Hole
BlackWidow
Buck
BuiltBotTough
Bullseye
BunnySlippers
BuzzSumo
CATExplorador
CCBot
CODE87
CensysInspect
CheTeam
ChinaClaw
Cliqzbot
Cloud mapping
Cocolyzebot
Cogentbot
Copyscape
Cosmos
Crawling at Home Project
CrazyWebCrawler
Curious
Custo
DBLBot
DSearch
DTS Agent
DatabaseDriverMysqli
Deusu
Dirbuster
Discoverybot
Dispatch
DnyzBot
Download Wonder
EMail Siphon
Ebingbong
EirGrabber
Express WebPictures
ExtLinksBot
FDM
Fimap
Firefox/7.0
FrontPage
Fuzz
FyberSpider
Fyrebot
GetWeb
Getintent
Go!Zilla
GoZilla
Grafula
HMView
HTMLparser
Harvest
Hloader
HonoluluBot
Humanlinks
Iblog
Id-search
Image Fetch
Image Sucker
InfoNaviRobot
InfoTekies
Intelliseek
Internet Ninja
Iskanie
JOC Web Spider
JetCar
JikeSpider
Jyxobot
Kenjin Spider
Keyword Density
Kinza
Kozmosbot
LNSpiderguy
Larbin
LeechGet
Libwhisker
LinkWalker
LinqiaMetadataDownloaderBot
Lipperhey Spider
Litemage_walker
Ltx71
MIDown tool
MIIxpc
MSFrontPage
MSIECrawler
Majestic SEO
MauiBot
Mb2345Browser
Meanpathbot
MegaIndex.ru
Microsoft URL Control
Minefield
Mister PiX
Moblie Safari
Mojeek
Mojolicious
Mozlila
Msrabot
NICErsPRO
NPbot
Nameprotect
Navroad
NearSite
Needle
NetAnts
NetSpider
Netcraft
NextGenSearchBot
Nibbler
Niki-bot
Ninja
Nmap
OpenVAS
OrangeBot
OrangeSpider
OutclicksBot
POE-Component-Client-HTTP
Page Analyzer
PageGrabber
PageScorer
Petalbot
Picscout
Pimonster
PleaseCrawl
Pockey
ProWebWalker
Pu_iN
PxBroker
PyCurl
Quick-Crawler
RankActiveLinkBot
RankingBot
Rankivabot
Re-re
Recorder
RocketCrawler
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SemanticJuice
Semrush
SemrushBot
SiteCheckerBotCrawler
Sitebeam
Sitevigil
SmartDownload
Snake
Snapbot
SocialRankIOBot
Sosospider
SpankBot
Spanner
Spbot
Sucker
Sucuri
SuperBot
Surfbot
Szukacz
T8Abot
Telesphorep
TheNomad
Thumbor
Toata
Toweyabot
True_Robot
Turingos
TurnitinBot
TwengaBot
URLy Warning
URLy.Warning
V-BOT
Vacuum
Vagabondo
VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE
WBSearchBot
WEBDAV
WPScan
WWW-Mechanize
WWWOFFLE
Wallpapers/3.0
Web Auto
Web Collage
Web Fetch
Web Fuck
Web Sauger
WebBandit
WebCopier
WebEnhancer
WebFuck
WebGo IS
WebLeacher
WebStripper
Webshag
Webster
Whack
WinHTTrack
Wotbox
Xaldon WebSpider
Zauba
ZoominfoBot
ZumBot
adscanner
backlink-check
clark-crawler
cognitiveseo
crawl.sogou.com
demandbase-bot
domainsproject.org
eCatch
gopher
ips-agent
lwp-trivial
mediawords
muhstik-scan
plumanalytics
polaris version
probe-image-size
scalaj-http
scan.lol
seobility
seoscanners
seostar
sitechecker.pro
sogouspider
sp_auditbot
sysscan
tAkeOut
trendiction.com
trendiction.de
ubermetrics-technologies.com
webgains-bot
webmeup-crawler
webprosbot
x22Mozilla
zauba.io
zgrab