404checker
404enemy
Abonti
Aboundexbot
ADmantX
Aipbot
AllSubmitter
ASPSeek
Attach
Backlink-Ceck
backlink-check
BackWeb
Badass
BatchFTP
Battleztar Bazinga
BDCbot
Bigfoot
BlackWidow
BlowFish
Boardreader
Bolt
Brandprotect
BuiltBotTough
BuiltWith
Bullseye
BunnySlippers
CATExplorador
CCBot
Cegbfeieh
CheeseBot
Claritybot
Cliqzbot
Cloud mapping
coccocbot-web
Cogentbot
Collector
com.plumanalytics
Copier
Copyscape
crawler4j
crawler.feedback
Crescent
CSHttp
DatabaseDriverMysqli
DataCha0s
DBLBot
Deusu
Digincore
DIIbot
Dirbuster
Disco
Discoverybot
Dispatch
DomainAppender
DomainCrawler
DomainStatsBot
Dotbot
Download Wonder
DSearch
EasyDL
Ecxi
EMail Siphon
EroCrawler
Evil
Express WebPictures
Extractor
ExtractorPro
Extreme Picture Finder
Ezooms
facebookscraper
FemtosearchBot
Fimap
Firefox/7.0
FlashGet
Flunky
Freeuploader
FrontPage
FyberSpider
GalaxyBot
Genieo
Getintent
GetRight
GetWeb
Gigabot
G-i-g-a-b-o-t
Gotit
GridBot
Haansoft
HaosouSpider
Heritrix
Hloader
HMView
Humanlinks
Id-search
IlseBot
Image Fetch
Indy Library
Intelliseek
InterGET
Internet Ninja
InternetSeer
ips-agent
Iria
IstellaBot
JetCar
JustView
Kenjin Spider
Kozmosbot
Lanshanbot
Larbin
LeechFTP
LibWeb
Libwhisker
Likse
LinkextractorPro
LinkpadBot
LinkScan
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Litemage_walker
LWP::Simple
Magnet
Mag-Net
Majestic SEO
MarkMonitor
Masscan
Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri
MFC_Tear_Sample
Microsoft URL Control
Mister PiX
Mojeek
Musobot
Nameprotect
NearSite
netEstate NE Crawler
NetLyzer
NetSpider
Nibbler
NICErsPRO
NimbleCrawler
Nmap
NPbot
oBot
Offline Explorer
OnCrawl
OpenLinkProfiler
OutclicksBot
PageAnalyzer
PageGrabber
Papa Foto
pcBrowser
PECL::HTTP
PictureFinder
PleaseCrawl
plumanalytics
POE-Component-Client-HTTP
Probethenet
ProPowerBot
ProWebWalker
Psbot
PxBroker
QueryN Metasearch
RankActive
RankActiveLinkBot
RankFlex
RebelMouse
Recorder
ReGet
Rogerbot
SalesIntelligent
SBIder
Scrapy
Screaming
ScreenerBot
Searchestate
SearchmetricsBot
Semrush
SemrushBot
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
seoscanners
SeoSiteCheckup
SEOstats
sexsearcher
Siphon
Sitebeam
Siteimprove
SiteSucker
Site Sucker
Sitevigil
SmartDownload
Sociscraper
Sogou web spider
SpaceBison
Spammen
Spanner
SputnikBot
spyfu
Sqlworm
Sqworm
Sucuri
sysscan
T0PHackTeam
Teleport
Telesphoreo
Telesphorep
The Intraformant
TheNomad
Tracemyfile
Trendiction
Trendictionbot
trendiction.com
Twice
Upflow
Vacuum
Vagabondo
VidibleScraper
Virusdie
WASALive-Bot
WebAuto
WebFuck
WebmasterWorldForumBot
WebPix
Web Pix
WebSauger
Web Sauger
Webster
Web Sucker
WebZIP
Widow
Wonderbot
Wprecon
WPScan
WWW::Mechanize
x22Mozilla
Xaldon_WebSpider
Zermelo
zgrab
Zitebot
ZmEu
ZumBot