404checker
ADmantX
ALittle Client
Aboundexbot
Acunetix
AhrefsBot
Aipbot
AllSubmitter
AlphaBot
Ankit
Anthill
Apexoo
Aspiegel
Asterias
AwarioRssBot
BBBike
BDCbot
BDFetch
Badass
Bandit
Barkrowler
Battleztar Bazinga
Bitacle
BlowFish
Boardreader
Bolt
BotALot
Brandprotect
Buck
Buddy
BuiltBotTough
CCBot
CODE87
CensysInspect
CheeseBot
CherryPicker
Cocolyzebot
Copier
Craftbot
Curious
DSearch
DTS Agent
DataCha0s
DatabaseDriverMysqli
Deusu
DigitalPebble
Dirbuster
Discoverybot
DittoSpyder
DomCopBot
DomainStatsBot
Domains Project
ECCP/1.0
EMail Wolf
EasyDL
Ebingbong
EroCrawler
Extreme Picture Finder
EyeNetIE
Ezooms
FemtosearchBot
Fimap
Firefox/7.0
FlashGet
Flunky
FrontPage
FyberSpider
G-i-g-a-b-o-t
GT::WWW
Gigablast
Go-Ahead-Got-It
Gotit
Grabber
GridBot
HMView
HTMLparser
Haansoft
HaosouSpider
Hloader
IDBot
IRLbot
Id-search
IlseBot
Image Fetch
InfoTekies
Intelliseek
Iria
JOC Web Spider
Jbrofuzz
Jetty
JikeSpider
Jorgee
Jyxobot
Kenjin Spider
Keyword Density
Kinza
Kozmosbot
LNSpiderguy
LWP::Simple
Leap
LeechFTP
LexiBot
Lftp
Libwhisker
Lightspeedsystems
LinkScan
Linkdexbot
LinqiaRSSBot
Litemage_walker
MTRobot
MarkWatch
Mass Downloader
Masscan
MauiBot
Meanpathbot
Mediatoolkitbot
MicroMessenger
Microsoft URL Control
Moblie Safari
Msrabot
Musobot
NearSite
NetAnts
Nettrack
Niki-bot
Ninja
Offline Explorer
OnCrawl
OrangeSpider
OutclicksBot
OutfoxBot
PHPCrawl
POE-Component-Client-HTTP
PageGrabber
PageThing.com
Panscient
Papa Foto
Pi-Monster
Picsearch
Pimonster
Pixray
Pockey
Psbot
Pu_iN
Pump
PyCurl
QueryN Metasearch
Quick-Crawler
RankActive
RankingBot
RepoMonkey
Rogerbot
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SEOstats
SISTRIX
SMTBot
SalesIntelligent
ScanAlert
Scanbot
Scrapy
ScreenerBot
ScrepyBot
SentiBot
SeoSiteCheckup
SiteCheckerBotCrawler
SiteSnagger
Siteimprove
Sitevigil
Snapbot
Sociscraper
SpaceBison
SpankBot
Spanner
Spbot
Sqworm
Sucker
Sucuri
Surfbot
Szukacz
T0PHackTeam
Teleport
Telesoft
Telesphorep
The Intraformant
Thumbor
Toata
Tracemyfile
True_Robot
Turingos
Turnitin
Twice
V-BOT
VB Project
VCI
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie
Voltron
WASALive-Bot
WEBDAV
WPScan
WWW-Collector-E
WWW::Mechanize
Web Sauger
Web Sucker
WebCollage
WebEnhancer
WebGo IS
WebImageCollector
WebLeacher
WebReaper
WebSauger
WebStripper
WebSucker
WebZIP
WebmasterWorldForumBot
Website Quester
Webster
Whack
Whacker
Wprecon
Xaldon_WebSpider
YoudaoBot
Zauba
Zermelo
Zitebot
ZoominfoBot
ZyBorg
archive.org_bot
arquivo.pt
coccocbot-web
cognitiveseo
eCatch
isitwp.com
lwp-request
meanpathbot
page scorer
polaris version
scalaj-http
seobility
seoscanners
seostar
serpstatbot
sexsearcher
sogouspider
voyagerx.com
webmeup-crawler
zgrab