360Spider
ADmantX
AhrefsBot
AiHitBot
Aipbot
Alligator
AllSubmitter
AlphaBot
Anarchie
Anthill
Apexoo
archive.org_bot
arquivo-web-crawler
autoemailspider
AwarioRssBot
Backlink-Ceck
backlink-check
BacklinkCrawler
BackWeb
Badass
BBBike
BetaBot
Bigfoot
Black Hole
BlackWidow
BlowFish
Boardreader
Bolt
Buddy
Bullseye
CazoodleBot
CheeseBot
CheTeam
Claritybot
Cloud mapping
Collector
Craftbot
crawler.feedback
crawl.sogou.com
CrazyWebCrawler
Curious
Custo
DataCha0s
Demon
Digincore
Dirbuster
Discoverybot
DomainAppender
Domains Project
domainsproject.org
Dotbot
Drip
DTS Agent
EasyDL
eCatch
ECCP/1.0
EMail Siphon
EMail Wolf
Express WebPictures
ExtLinksBot
Extreme Picture Finder
EyeNetIE
facebookscraper
FHscan
FlashGet
Flunky
Freeuploader
Fyrebot
Genieo
Gigablast
GoZilla
Go!Zilla
GT::WWW
HEADMasterSEO
heritrix
Heritrix
HTMLparser
HTTP::Lite
HTTrack
Humanlinks
IDBTE4M
Image Sucker
Indy Library
InfoNaviRobot
InfoTekies
instabid
InterGET
internetVista monitor
ips-agent
IRLbot
IstellaBot
Jbrofuzz
JennyBot
JetCar
JikeSpider
JustView
Kenjin Spider
Keyword Density
Kinza
Kozmosbot
LexiBot
Lftp
Libwhisker
Lightspeedsystems
LinkextractorPro
LinkpadBot
LinkScan
LinksManager
Litemage_walker
Lmspider
LNSpiderguy
Ltx71
lwp-request
lwp-trivial
magpie-crawler
Mail.RU_Bot
Majestic12
Majestic SEO
MarkMonitor
Masscan
Mb2345Browser
meanpathbot
Mediatoolkitbot
MicroMessenger
Microsoft URL Control
MJ12bot
Morfeus Fucking Scanner
Mozlila
muhstik-scan
Name Intelligence
Nessus
Netcraft
netEstate NE Crawler
NetMechanic
NICErsPRO
Niki-bot
Ninja
Nmap
NPbot
OnCrawl
Openvas
OPPO A33
OutclicksBot
PageAnalyzer
PageScorer
Pandalytics
Pavuk
PECL::HTTP
PHPCrawl
Picscout
Picsearch
Pimonster
Pixray
plumanalytics
POE-Component-Client-HTTP
polaris version
ProWebWalker
QueryN Metasearch
RankingBot
RankingBot2
Rankivabot
RankurBot
RealDownload
Reaper
ReGet
RocketCrawler
RSSingBot
satoristudio.net
SBIder
scan.lol
SemrushBot
SEOkicks
SEOstats
serpstatbot
sexsearcher
Siphon
Sitebeam
SiteCheckerBotCrawler
sitechecker.pro
SiteExplorer
SlySearch
SmartDownload
Snake
Snapbot
Snoopy
SocialRankIOBot
Sociscraper
Sogou web spider
SpaceBison
Spammen
SpankBot
sp_auditbot
Spbot
SputnikBot
Sqlmap
Sqlworm
Sqworm
SuperBot
SuperHTTP
SurveyBot
Suzuran
Szukacz
tAkeOut
Telesoft
Telesphoreo
The Intraformant
Toata
Tracemyfile
Trendiction
Trendictionbot
True_Robot
Turingos
Vacuum
Vagabondo
VB Project
VCI
Voil
Voltron
WASALive-Bot
WBSearchBot
WebBandit
Web Collage
WebCopier
WebFetch
WebFuck
Web Fuck
WebImageCollector
Web Pix
WebsiteQuester
Website Quester
WebZIP
WeSEE
Whack
Whacker
Whatweb
WinHTTrack
Woobot
Wotbox
Wprecon
WPScan
WWW-Collector-E
WWW::Mechanize
x22Mozilla
Xaldon WebSpider
xpymep1.exe
YoudaoBot
Zauba
zgrab
ZmEu
ZyBorg