404checker
80legs
AIBOT
ALittle Client
Aboundex
Aboundexbot
AhrefsBot
Aipbot
AlphaBot
Anthill
Aspiegel
Attach
BackStreet
BackWeb
BacklinkCrawler
Barkrowler
Battleztar Bazinga
Bitacle
Black Hole
Blackboard
Bolt
BotALot
CCBot
CSHttp
Calculon
CensysInspect
CheeseBot
Chlooe
Citoid
Cliqzbot
Cocolyzebot
Cogentbot
CopyRightCheck
Cosmos
Craftbot
Crawling at Home Project
CrazyWebCrawler
CyotekWebCopy
DSearch
DataCha0s
Dirbuster
Dispatch
DittoSpyder
DnBCrawler-Analytics
DnyzBot
DomCopBot
DomainAppender
DomainSigmaCrawler
Download Wonder
ECCP/1.0
EMail Siphon
EasyDL
Ecxi
EirGrabber
Evil
Exabot
ExtLinksBot
Extractor
FDM
FemtosearchBot
Flunky
FrontPage
Fuzz
FyberSpider
GermCrawler
GetRight
GetWeb
Gigabot
Go!Zilla
GridBot
HMView
HaosouSpider
Harvest
Havij
Heritrix
Humanlinks
HybridBot
Id-search
IndeedBot
Indy Library
InterGET
InternetSeer
IstellaBot
JOC Web Spider
JamesBOT
JetCar
Jetty
Jorgee
JustView
Jyxobot
Kenjin Spider
Keyword Density
Kinza
Kozmosbot
Larbin
LeechGet
LexiBot
Libwhisker
LieBaoFast
Likse
LinkScan
LinkWalker
Linkbot
LinkpadBot
LinksManager
LinqiaRSSBot
Lipperhey Spider
Litemage_walker
Lmspider
Ltx71
MJ12bot
MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot
Mass Downloader
Mata Hari
Mediatoolkitbot
MicroMessenger
Microsoft URL Control
Minefield
Mozlila
Mr.4x3
Msrabot
NICErsPRO
Nameprotect
Nessus
Net Vampire
NetMechanic
Niki-bot
NimbleCrawler
Nimbostratus
Nutch
Offline Explorer
OpenLinkProfiler
OpenVAS
Openfind
OrangeSpider
PECL::HTTP
POE-Component-Client-HTTP
PageGrabber
PageScorer
Pandalytics
Panscient
Papa Foto
Picscout
PictureFinder
Pimonster
Pixray
Pockey
PxBroker
RSSingBot
Rankivabot
Re-re
RealDownload
Recorder
RepoMonkey
Ripper
Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler
SEOstats
SISTRIX
Scanbot
Screaming
SearchmetricsBot
SemanticJuice
SemrushBot
SentiBot
SeobilityBot
Siphon
SiteCheckerBotCrawler
SiteLockSpider
Sitebeam
Siteimprove
SmartDownload
SocialRankIOBot
Sogou web spider
Sottopop
SpaceBison
Spanner
SputnikBot
Surfbot
SurveyBot
Teleport
TeleportPro
Telesoft
Telesphoreo
Trendictionbot
Turnitin
Typhoeus
URLy.Warning
UnisterBot
Upflow
V-BOT
VCI
Vacuum
VelenPublicWebCrawler
VidibleScraper
Voltron
WASALive-Bot
WWW::Mechanize
WWWOFFLE
Wallpapers
WeSEE
Web Collage
Web Fetch
Web Fuck
Web Sauger
WebCollage
WebCopier
WebEnhancer
WebGo IS
WebImageCollector
WebPix
WebSauger
WebSucker
WebZIP
Webshag
WebsiteExtractor
Wonderbot
Xenu
YoudaoBot
Zade
Zitebot
ZyBorg
arquivo.pt
backlink-check
check1.exe
crawler.feedback
dataforseo.com
demandbase-bot
domainsproject.org
gopher
heritrix
ips-agent
magpie-crawler
netEstate NE Crawler
probe-image-size
ripz
scan.lol
seocompany.store
seoscanners
sexsearcher
sogouspider
sp_auditbot
webgains-bot
x09Mozilla
x22Mozilla