404checker
Abonti
Aboundex
AIBOT
Aipbot
Alligator
AlphaBot
Anarchy99
Ankit
arquivo-web-crawler
ASPSeek
Asterias
autoemailspider
AwarioRssBot
AwarioSmartBot
BackDoorBot
Backlink-Ceck
BacklinkCrawler
BackStreet
Badass
Bandit
Barkrowler
BatchFTP
BBBike
BDFetch
Bigfoot
Blackboard
BLEXBot
Bolt
BotALot
Brandwatch
Bullseye
CATExplorador
CCBot
Cegbfeieh
CheeseBot
Cliqzbot
coccocbot-web
CODE87
Cogentbot
cognitiveseo
Collector
Copier
crawler4j
crawl.sogou.com
Custo
CyotekWebCopy
DBLBot
Demon
DigitalPebble
DIIbot
Discobot
DittoSpyder
DnyzBot
Domains Project
domainsproject.org
DSearch
DTS Agent
eCatch
Ecxi
EMail Wolf
EroCrawler
Evil
Extractor
EyeNetIE
Fimap
FlashGet
Flunky
Foobot
FrontPage
FyberSpider
Fyrebot
Genieo
GermCrawler
GetWeb
Gigabot
gopher
Go!Zilla
GrabNet
Grafula
GT::WWW
HaosouSpider
Harvest
Havij
HEADMasterSEO
HTTP::Lite
HTTrack
Humanlinks
HybridBot
Iblog
Id-search
Image Fetch
IndeedBot
InfoNaviRobot
Intelliseek
InterGET
InternetSeer
ips-agent
Iria
IRLbot
isitwp.com
JetCar
Jetty
JOC Web Spider
Joomla
Jorgee
Keyword Density
Kinza
Larbin
Lftp
LieBaoFast
Lightspeedsystems
LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Litemage_walker
Ltx71
LWP::Simple
lwp-trivial
magpie-crawler
Mail.RU_Bot
Majestic12
Majestic-SEO
MarkWatch
Masscan
Mass Downloader
Mb2345Browser
Meanpathbot
MeanPath Bot
MegaIndex.ru
Metauri
MFC_Tear_Sample
MicroMessenger
Microsoft URL Control
MIDown tool
MIIxpc
Msrabot
Musobot
Nameprotect
Needle
Nessus
NetAnts
NetLyzer
Nettrack
Nibbler
Niki-bot
Ninja
Nuclei
Octopus
Offline Explorer
OrangeBot
OrangeSpider
OutclicksBot
OutfoxBot
page scorer
PageScorer
Pandalytics
Papa Foto
Picscout
Piepmatz
Pockey
PxBroker
PyCurl
QueryN Metasearch
RankActiveLinkBot
RankFlex
RankingBot
RankingBot2
Rankivabot
RankurBot
RealDownload
RedesScrapy
Ripper
Rogerbot
s1z.ru
SalesIntelligent
ScanAlert
Scrapy
Screaming
ScreenerBot
Semrush
SemrushBot
SEOkicks
SEOkicks-Robot
SEOprofiler
seoscanners
SeoSiteCheckup
SEOstats
sexsearcher
Shodan
SISTRIX
Sitebeam
Site Sucker
SmartDownload
SMTBot
Snoopy
SocialRankIOBot
Sociscraper
SpaceBison
Spanner
sp_auditbot
SputnikBot
Sqlworm
SuperBot
Surfbot
Suzuran
Szukacz
T0PHackTeam
T8Abot
tAkeOut
TeleportPro
Telesphorep
The Intraformant
TheNomad
Toweyabot
Trendiction
Turnitin
Twice
UnisterBot
Virusdie
Voil
WBSearchBot
Webalta
WebAuto
WEBDAV
WebFetch
Web Fetch
WebFuck
WebLeacher
WebmasterWorldForumBot
webmeup-crawler
Web Pix
WebSauger
Webshag
WebsiteQuester
WebStripper
WebSucker
WebZIP
WeSEE
Whatweb
WiseGuys Robot
WISENutbot
Wonderbot
WPScan
WWWOFFLE
xpymep1.exe
Zeus
ZumBot
ZyBorg