404enemy
80legs
Aboundex
Aboundexbot
Acunetix
ADmantX
AhrefsBot
AIBOT
Alligator
Anarchie
Asterias
autoemailspider
Backlink-Ceck
BatchFTP
BDCbot
BDFetch
Bigfoot
Blackboard
Black Hole
BlackWidow
BlowFish
BotALot
Brandprotect
Brandwatch
Buddy
BuiltWith
Bullseye
BunnySlippers
BuzzSumo
CCBot
CheeseBot
ChinaClaw
Chlooe
Cliqzbot
Cloud mapping
coccocbot-web
Collector
com.plumanalytics
Copyscape
Craftbot
crawler4j
crawl.sogou.com
CrunchBot
demandbase-bot
Devil
Digincore
DigitalPebble
Disco
Dispatch
DittoSpyder
DnyzBot
Dragonfly
DSearch
DTS Agent
EasyDL
eCatch
ECCP/1.0
EMail Siphon
EMail Wolf
Evil
Extractor
ExtractorPro
Ezooms
FemtosearchBot
Firefox/7.0
FlashGet
FyberSpider
Fyrebot
GetRight
Gigablast
Go-Ahead-Got-It
Gotit
Go!Zilla
GrabNet
Grafula
GrapeshotCrawler
GridBot
Havij
HTMLparser
HTTrack
Humanlinks
HybridBot
Id-search
Image Fetch
Image Sucker
IndeedBot
Indy Library
instabid
ips-agent
IstellaBot
JamesBOT
JennyBot
Jetty
JikeSpider
JOC Web Spider
Joomla
Jorgee
JustView
Lanshanbot
LeechGet
Linkdexbot
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey
LNSpiderguy
lwp-request
LWP::Simple
lwp-trivial
magpie-crawler
Majestic SEO
Majestic-SEO
MarkMonitor
MarkWatch
meanpathbot
Meanpathbot
mediawords
Metauri
Microsoft Data Access
Microsoft URL Control
MIDown tool
Mister PiX
Mr.4x3
MSIECrawler
Msrabot
MS Web Services Client Protocol
Needle
NetAnts
netEstate NE Crawler
NetLyzer
NetMechanic
NetSpider
Nettrack
NetZIP
Niki-bot
Nikto
NimbleCrawler
Nimbostratus
Ninja
Nutch
oBot
Offline Explorer
Offline Navigator
OpenLinkProfiler
OrangeSpider
OutfoxBot
PageAnalyzer
page scorer
Panscient
Papa Foto
pcBrowser
PHPCrawl
PictureFinder
Pimonster
Pixray
PleaseCrawl
POE-Component-Client-HTTP
ProPowerBot
ProWebWalker
PyCurl
QueryN Metasearch
Quick-Crawler
RankurBot
RebelMouse
RedesScrapy
RepoMonkey
Ripper
Rogerbot
s1z.ru
SalesIntelligent
ScanAlert
scan.lol
Screaming
ScreenerBot
SearchmetricsBot
SEOlyticsCrawler
SEOprofiler
serpstatbot
Shodan
Siteimprove
SmartDownload
Snake
Snapbot
Sociscraper
Sogou web spider
Sottopop
SpaceBison
sp_auditbot
Spbot
Sqlmap
Sqlworm
SuperHTTP
Surfbot
Suzuran
Swiftbot
T8Abot
Teleport
TeleportPro
Telesphoreo
The Intraformant
TheNomad
TightTwatBot
Toata
Tracemyfile
Trendiction
trendiction.com
trendiction.de
True_Robot
TurnitinBot
Twice
Typhoeus
Vagabondo
VB Project
VoidEYE
Voil
Wallpapers/3.0
WallpapersHD
WASALive-Bot
WBSearchBot
WebCollage
WEBDAV
WebFetch
Web Fetch
WebGo IS
WebImageCollector
WebmasterWorldForumBot
WebPix
Web Pix
WebReaper
WebsiteQuester
Website Quester
WebStripper
Web Sucker
WebWhacker
WebZIP
Whacker
Whatweb
Who.is Bot
Widow
WinHTTrack
WISENutbot
WPScan
WWW-Collector-E
WWW::Mechanize
Xaldon WebSpider
Xaldon_WebSpider
Zade
Zauba
Zitebot
ZmEu