404checker
80legs
Abonti
Aboundex
Acunetix
AhrefsBot
AIBOT
AiHitBot
Aipbot
Alligator
AllSubmitter
Anarchie
archive.org_bot
BackStreet
BBBike
BLEXBot
Blow
BlowFish
Bolt
Brandprotect
Buddy
BuiltWith
BunnySlippers
BuzzSumo
CherryPicker
Chlooe
Cliqzbot
coccocbot-web
Cogentbot
com.plumanalytics
CopyRightCheck
Copyscape
Cosmos
crawler4j
crawl.sogou.com
Curious
Custo
Demon
Deusu
Devil
DigitalPebble
DIIbot
Dirbuster
Disco
Discoverybot
Dispatch
DomainAppender
DomainSigmaCrawler
DomainStatsBot
Drip
DTS Agent
EasyDL
Ebingbong
Ecxi
EirGrabber
evc-batch
ExtLinksBot
ExtractorPro
EyeNetIE
Ezooms
facebookscraper
FemtosearchBot
FHscan
FlashGet
Freeuploader
Genieo
Getintent
GetRight
GetWeb
Gigablast
Go-Ahead-Got-It
Go!Zilla
GrabNet
GrapeFX
GT::WWW
Haansoft
Harvest
HEADMasterSEO
Heritrix
HMView
HTMLparser
HTTP::Lite
HTTrack
Humanlinks
IDBot
Image Fetch
Image Sucker
IndeedBot
InfoNaviRobot
InfoTekies
Internet Ninja
internetVista monitor
Jbrofuzz
JennyBot
JikeSpider
Keyword Density
LeechFTP
LeechGet
Likse
Linkdexbot
LinkextractorPro
LinkpadBot
LinkScan
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey
Ltx71
lwp-trivial
Magnet
magpie-crawler
Majestic12
Majestic SEO
MarkWatch
MauiBot
MeanPath Bot
mediawords
Metauri
MSFrontPage
Msrabot
MS Web Services Client Protocol
Name Intelligence
Needle
Nessus
NetSpider
Netvibes
NetZIP
NICErsPRO
Nikto
Ninja
Nmap
NPbot
oBot
Octopus
Offline Navigator
OnCrawl
OpenVAS
OutclicksBot
OutfoxBot
PageAnalyzer
Page Analyzer
PageGrabber
Pandalytics
Panscient
pcBrowser
PeoplePal
PHPCrawl
Picscout
Picsearch
PictureFinder
Pimonster
Pi-Monster
Pixray
PleaseCrawl
plumanalytics
ProPowerBot
ProWebWalker
Psbot
Pump
RankActive
RealDownload
Reaper
RedesScrapy
Rogerbot
s1z.ru
SalesIntelligent
SBIder
ScoutJet
ScreenerBot
Searchestate
SemrushBot
SEOkicks-Robot
SEOprofiler
Shodan
SISTRIX
Sitebeam
Siteimprove
Site Sucker
Sitevigil
SMTBot
Snake
Snapbot
SocialRankIOBot
Sogou web spider
Sosospider
Sottopop
SpaceBison
SpankBot
Spanner
Spinn3r
spyfu
Sqlmap
Sqworm
Sucker
SuperBot
SuperHTTP
Surfbot
Suzuran
sysscan
Szukacz
T8Abot
tAkeOut
TeleportPro
Telesphoreo
Toata
Toweyabot
Tracemyfile
Turingos
Typhoeus
URLy Warning
URLy.Warning
Vacuum
VB Project
VeriCiteCrawler
VidibleScraper
Voltron
Wallpapers/3.0
WebBandit
Web Collage
Web Enhancer
Web Fetch
Web Fuck
WebGo IS
WebLeacher
WebPix
Web Pix
WebSauger
Webshag
Webster
Web Sucker
WebWhacker
WeSEE
Whacker
Whatweb
WinHTTrack
WiseGuys Robot
Woobot
WPScan
WWW-Mechanize
WWWOFFLE
x09Mozilla
Xaldon_WebSpider
Xenu
xpymep1.exe
YoudaoBot
Zade
Zermelo
Zeus
zgrab
ZmEu