404checker
Aboundexbot
AhrefsBot
AiHitBot
Alligator
AlphaBot
Anarchy99
archive.org_bot
arquivo-web-crawler
Asterias
autoemailspider
AwarioRssBot
AwarioSmartBot
Backlink-Ceck
backlink-check
BacklinkCrawler
Barkrowler
Battleztar Bazinga
BBBike
BDCbot
BDFetch
BlackWidow
BLEXBot
Boardreader
Bolt
Brandwatch
Buddy
BuzzSumo
Calculon
CATExplorador
CazoodleBot
CCBot
CheTeam
Chlooe
Claritybot
Cloud mapping
CODE87
com.plumanalytics
Copier
Copyscape
Cosmos
crawl.sogou.com
CrunchBot
Curious
CyotekWebCopy
DBLBot
demandbase-bot
Devil
Discobot
DittoSpyder
DomainCrawler
DomainSigmaCrawler
Domains Project
Dotbot
DTS Agent
Ebingbong
Ecxi
evc-batch
Evil
EyeNetIE
FHscan
Fimap
Flunky
FrontPage
GalaxyBot
GetRight
Gigablast
Gigabot
Go-Ahead-Got-It
gopher
Gotit
GrapeFX
GrapeshotCrawler
GridBot
Haansoft
HaosouSpider
Hloader
HTTrack
Iblog
IDBot
Id-search
Image Sucker
Indy Library
instabid
Intelliseek
InterGET
InternetSeer
Iria
IRLbot
Iskanie
JennyBot
JetCar
JikeSpider
Kinza
Kozmosbot
LeechGet
Lftp
LibWeb
LieBaoFast
LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Litemage_walker
Lmspider
Ltx71
LWP::Simple
Majestic12
Majestic SEO
Majestic-SEO
MarkMonitor
Mass Downloader
Meanpathbot
mediawords
Metauri
Microsoft Data Access
MJ12bot
Mojolicious
Morfeus Fucking Scanner
Mozlila
Mr.4x3
MTRobot
Nameprotect
Needle
Nessus
NetMechanic
Nettrack
NICErsPRO
Nikto
NimbleCrawler
Ninja
oBot
Offline Explorer
Offline Navigator
OrangeSpider
Page Analyzer
PageScorer
PageThing.com
Pandalytics
Pavuk
pcBrowser
Petalbot
PHPCrawl
Picsearch
PictureFinder
Piepmatz
Pockey
POE-Component-Client-HTTP
polaris version
ProWebWalker
Psbot
PxBroker
QueryN Metasearch
RankFlex
RankingBot
RankingBot2
RankurBot
RebelMouse
Recorder
RedesScrapy
Ripper
Rogerbot
scalaj-http
Scanbot
scan.lol
ScoutJet
Scrapy
Searchestate
SearchmetricsBot
SemrushBot
SEOkicks
SEOkicks-Robot
seoscanners
SEOstats
serpstatbot
SISTRIX
SiteExplorer
SiteSucker
Site Sucker
SmartDownload
Snake
Snapbot
sogouspider
Sosospider
Spanner
Spbot
Spinn3r
spyfu
Steeler
Sucuri
SuperBot
SurveyBot
Swiftbot
sysscan
Szukacz
T0PHackTeam
T8Abot
tAkeOut
Teleport
The Intraformant
TheNomad
Toata
Trendiction
True_Robot
Turingos
TurnitinBot
UnisterBot
URLy.Warning
Vacuum
Vagabondo
VCI
VelenPublicWebCrawler
VoidEYE
Voil
WallpapersHD
Webalta
WebBandit
WebCollage
WebCopier
WEBDAV
WebFetch
Web Fetch
WebmasterWorldForumBot
Web Pix
WebReaper
WebSauger
WebsiteExtractor
WebsiteQuester
Website Quester
WebSucker
Web Sucker
WebZIP
Whacker
Who.is Bot
WISENutbot
Woobot
Wotbox
Wprecon
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Xaldon WebSpider
xpymep1.exe
Zade
zauba.io
Zeus
zgrab
ZmEu
ZyBorg