404checker
404enemy
Abonti
Aboundexbot
Acunetix
ADmantX
AhrefsBot
Aipbot
AllSubmitter
AlphaBot
Asterias
Attach
autoemailspider
BackDoorBot
backlink-check
BacklinkCrawler
BackWeb
Bandit
Battleztar Bazinga
Bitacle
Blackboard
Black Hole
BlackWidow
BLEXBot
BotALot
Brandprotect
Buddy
BuiltBotTough
Bullseye
BuzzSumo
CazoodleBot
Cegbfeieh
CherryPicker
Cloud mapping
coccocbot-web
cognitiveseo
Collector
CopyRightCheck
Craftbot
CrazyWebCrawler
CrunchBot
CSHttp
Custo
DatabaseDriverMysqli
DataCha0s
DBLBot
Deusu
Devil
DigitalPebble
DIIbot
Discoverybot
Dispatch
DittoSpyder
DnyzBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Download Wonder
DSearch
DTS Agent
eCatch
EirGrabber
EMail Wolf
Evil
Exabot
Express WebPictures
ExtLinksBot
ExtractorPro
Extreme Picture Finder
Firefox/7.0
Freeuploader
FrontPage
GalaxyBot
GermCrawler
GetRight
Go-Ahead-Got-It
GoZilla
Grabber
Grafula
Harvest
Heritrix
HTMLparser
HTTP::Lite
HTTrack
Humanlinks
Id-search
IlseBot
Image Fetch
Intelliseek
ips-agent
IRLbot
Iskanie
IstellaBot
Jbrofuzz
JetCar
Jetty
Jyxobot
Keyword Density
Kozmosbot
Lftp
Libwhisker
Likse
Linkdexbot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Litemage_walker
Lmspider
lwp-request
lwp-trivial
Mag-Net
Mail.RU_Bot
Majestic12
Majestic SEO
Majestic-SEO
MarkMonitor
Mata Hari
MauiBot
meanpathbot
Mediatoolkitbot
MegaIndex.ru
MFC_Tear_Sample
Microsoft Data Access
Mister PiX
Mr.4x3
MSIECrawler
muhstik-scan
NetAnts
NetSpider
Nettrack
Net Vampire
Netvibes
NetZIP
Nibbler
NICErsPRO
Niki-bot
Nimbostratus
Nmap
Octopus
OpenLinkProfiler
Openvas
OpenVAS
OrangeSpider
OutfoxBot
PageAnalyzer
Page Analyzer
page scorer
PageScorer
Pandalytics
Papa Foto
Pavuk
PECL::HTTP
Picscout
Picsearch
PictureFinder
Pi-Monster
Pixray
PleaseCrawl
Pockey
Probethenet
ProWebWalker
Psbot
PxBroker
PyCurl
QueryN Metasearch
RSSingBot
SalesIntelligent
SBIder
ScanAlert
Scanbot
scan.lol
Scrapy
Screaming
SearchmetricsBot
Semrush
SemrushBot
SEOprofiler
seoscanners
SEOstats
serpstatbot
Sitebeam
SiteLockSpider
SiteSucker
SlySearch
SMTBot
Snapbot
SpankBot
sp_auditbot
spyfu
Sqlworm
Sqworm
Steeler
Sucker
Sucuri
SuperBot
SuperHTTP
Surfbot
Suzuran
T0PHackTeam
Teleport
Telesphoreo
Telesphorep
Thumbor
Titan
Tracemyfile
Trendiction
Trendictionbot
Turnitin
TurnitinBot
Typhoeus
Vacuum
Vagabondo
VCI
VeriCiteCrawler
VoidEYE
Voil
WBSearchBot
Webalta
Web Collage
WebCopier
WEBDAV
WebGo IS
WebLeacher
WebmasterWorldForumBot
webmeup-crawler
WebPix
Web Pix
WebReaper
WebSauger
WebStripper
WebSucker
WebWhacker
WeSEE
Widow
WISENutbot
Wonderbot
Wotbox
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
x22Mozilla
Xaldon WebSpider
Xaldon_WebSpider
Zade
zauba.io
zgrab
Zitebot
ZyBorg