404checker
80legs
AIBOT
ALittle Client
Abonti
Aboundex
Acunetix
AhrefsBot
Alligator
AlphaBot
Anarchy
Anthill
Apexoo
Aspiegel
Attach
AwarioRssBot
BLEXBot
BackStreet
Badass
Barkrowler
Battleztar Bazinga
Bigfoot
Blackboard
Bolt
BotALot
Brandwatch
Buck
Buddy
BuiltWith
Bullseye
BunnySlippers
Calculon
CazoodleBot
Cegbfeieh
ChinaClaw
Chlooe
Claritybot
Cloud mapping
Copier
Copyscape
Craftbot
CrazyWebCrawler
CyotekWebCopy
DSearch
DTS Agent
DataCha0s
Dirbuster
Disco
Dispatch
DnBCrawler-Analytics
DomainSigmaCrawler
Dotbot
ECCP/1.0
EMail Wolf
EasyDL
Ebingbong
EirGrabber
EroCrawler
Express WebPictures
Fimap
Firefox/7.0
Flunky
Freeuploader
FrontPage
Fuzz
FyberSpider
Fyrebot
GT::WWW
GalaxyBot
GetWeb
Getintent
Gigabot
Go!Zilla
Go-Ahead-Got-It
GoZilla
GrabNet
GrapeshotCrawler
HEADMasterSEO
Haansoft
HaosouSpider
Harvest
Humanlinks
IDBot
IRLbot
IlseBot
IndeedBot
InfoNaviRobot
InfoTekies
InternetSeer
Iria
JennyBot
JetCar
JikeSpider
Joomla
Jorgee
Kenjin Spider
Kozmosbot
LeechFTP
LeechGet
LexiBot
Libwhisker
Linkdexbot
LinkextractorPro
LinksManager
LinqiaRSSBot
Lipperhey
Lmspider
Ltx71
MFC_Tear_Sample
MJ12bot
MSIECrawler
MTRobot
Mag-Net
Magnet
Majestic SEO
Majestic12
MarkWatch
Masscan
Mata Hari
MauiBot
MeanPath Bot
MicroMessenger
Microsoft Data Access
Microsoft URL Control
Mister PiX
Mojeek
Mojolicious
Mr.4x3
Msrabot
NICErsPRO
NPbot
Nameprotect
NearSite
Needle
NetLyzer
NetMechanic
NetSpider
NetZIP
NextGenSearchBot
Nikto
NimbleCrawler
Nimbostratus
Openfind
OutclicksBot
POE-Component-Client-HTTP
Page Analyzer
Pi-Monster
Picscout
Picsearch
PictureFinder
Pimonster
Pixray
PleaseCrawl
Pockey
ProPowerBot
Probethenet
Psbot
PxBroker
QueryN Metasearch
RankActive
RankurBot
Re-re
ReGet
RepoMonkey
Ripper
Rogerbot
SEOlyticsCrawler
SEOstats
Scrapy
Screaming
ScreenerBot
ScrepyBot
SemanticJuice
Semrush
Seomoz
SiteLockSpider
Siteimprove
SmartDownload
Snoopy
Sociscraper
Sottopop
Spanner
Spbot
SputnikBot
Sqlmap
SuperBot
Surfbot
SurveyBot
Szukacz
Teleport
The Intraformant
TightTwatBot
Tracemyfile
Trendiction
Trendictionbot
True_Robot
TurnitinBot
TwengaBot
Twice
URLy Warning
URLy.Warning
Upflow
V-BOT
VB Project
VelenPublicWebCrawler
VidibleScraper
Virusdie
WBSearchBot
WEBDAV
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers
WeSEE
Web Auto
Web Sauger
WebImageCollector
WebPix
WebReaper
WebSauger
WebZIP
Website Quester
Widow
WinHTTrack
Wotbox
Xaldon_WebSpider
YoudaoBot
Zade
Zeus
ZmEu
ZoomBot
ZumBot
autoemailspider
backlink-check
com.plumanalytics
dataforseo.com
eCatch
evc-batch
magpie-crawler
muhstik-scan
netEstate NE Crawler
plumanalytics
scan.lol
seocompany.store
spyfu
tAkeOut
ubermetrics-technologies.com
webgains-bot
webmeup-crawler