360Spider
AIBOT
Abonti
Aboundex
Aboundexbot
Acunetix
AhrefsBot
AllSubmitter
AlphaBot
Anarchie
Apexoo
Aspiegel
Atomseobot
AwarioRssBot
AwarioSmartBot
BBBike
BackStreet
BackWeb
BatchFTP
Battleztar Bazinga
BetaBot
Black Hole
Boardreader
Brandwatch
Buddy
BuiltWith
BunnySlippers
CODE87
Calculon
CheeseBot
CherryPicker
Chlooe
Claritybot
Cloud mapping
Cocolyzebot
Cogentbot
Collector
Copyscape
Cosmos
Craftbot
Crawling at Home Project
CrazyWebCrawler
CrunchBot
DatabaseDriverMysqli
Deusu
Disco
Discoverybot
Dispatch
DittoSpyder
DomainCrawler
Download Wonder
Dragonfly
EMail Siphon
EasyDL
Ebingbong
EirGrabber
Evil
Express WebPictures
ExtLinksBot
Extractor
ExtractorPro
Ezooms
FHscan
Fimap
FyberSpider
G-i-g-a-b-o-t
GermCrawler
GetWeb
GoZilla
Gotit
Grafula
GridBot
HEADMasterSEO
HMView
HTTP::Lite
HTTrack
HaosouSpider
Harvest
Heritrix
HonoluluBot
IDBot
Id-search
IndeedBot
Iria
Iskanie
JOC Web Spider
JamesBOT
Jetty
JikeSpider
Keybot Translation-Search-Machine
LNSpiderguy
Lanshanbot
LeechGet
LieBaoFast
LinkScan
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
MIDown tool
MIIxpc
MTRobot
Mail.RU_Bot
Majestic12
Mass Downloader
Mata Hari
MegaIndex.ru
Metauri
Microsoft Data Access
Moblie Safari
Mojolicious
Msrabot
NPbot
NearSite
NetLyzer
NetSpider
NetZIP
Netcraft
Nikto
NimbleCrawler
Not
Nutch
Octopus
OpenVAS
OrangeBot
OrangeSpider
OutfoxBot
PECL::HTTP
PHPCrawl
PageGrabber
Pandalytics
Panscient
Pavuk
Pi-Monster
Picscout
Picsearch
PictureFinder
Pimonster
Probethenet
Psbot
Pump
Quick-Crawler
Re-re
Reaper
RedesScrapy
RepoMonkey
SEOkicks-Robot
SEOprofiler
SEOstats
SISTRIX
Scanbot
Screaming
ScrepyBot
Searchestate
Seekport
Semrush
SentiBot
SeobilityBot
Shodan
SiteExplorer
SiteLockSpider
SiteSnagger
Sitevigil
Snake
Snapbot
Snoopy
Sogou web spider
Spammen
Spanner
SputnikBot
Sqworm
Steeler
SuperBot
SuperHTTP
Surfbot
SurveyBot
Suzuran
T0PHackTeam
Telesoft
Telesphoreo
Turingos
Typhoeus
Upflow
V-BOT
VB Project
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE
Voil
WASALive-Bot
WBSearchBot
WEBDAV
WPScan
WWW-Collector-E
Wallpapers
WeSEE
Web Collage
Web Fuck
Web Pix
WebBandit
WebCollage
WebEnhancer
WebFuck
WebSauger
WebStripper
Webshag
Webster
Whacker
Whatweb
Widow
WiseGuys Robot
Wotbox
Xaldon WebSpider
Zeus
Zitebot
ZmEu
ZoominfoBot
ZyBorg
archive.org_bot
arquivo-web-crawler
arquivo.pt
com.plumanalytics
crawler.feedback
facebookscraper
instabid
internetVista monitor
ips-agent
isitwp.com
linkdexbot
magpie-crawler
mediawords
page scorer
pcBrowser
scalaj-http
scan.lol
seobility
seoscanners
serpstatbot
sitechecker.pro
siteripz
sogouspider
sysscan
tAkeOut
voyagerx.com
webpros.com
x09Mozilla
x22Mozilla
zauba.io
zgrab