404checker
404enemy
Abonti
Acunetix
AhrefsBot
Alexibot
Alligator
AlphaBot
archive.org_bot
ASPSeek
backlink-check
Bandit
Barkrowler
Battleztar Bazinga
BBBike
BDFetch
Bitacle
Black Hole
BlackWidow
BotALot
Brandwatch
BuzzSumo
Calculon
CATExplorador
CazoodleBot
CherryPicker
ChinaClaw
Claritybot
Collector
CopyRightCheck
Craftbot
CSHttp
Curious
DatabaseDriverMysqli
DataCha0s
DBLBot
demandbase-bot
Deusu
Devil
DIIbot
Discobot
Dispatch
DittoSpyder
DomainAppender
Drip
Ebingbong
eCatch
EirGrabber
evc-batch
facebookscraper
Fimap
Firefox/7.0
FlashGet
Flunky
Foobot
Freeuploader
FrontPage
FyberSpider
GalaxyBot
GermCrawler
Getintent
GetRight
Gigablast
Gigabot
Go-Ahead-Got-It
Grabber
GrapeFX
Haansoft
HaosouSpider
Harvest
HEADMasterSEO
Hloader
HMView
HTTP::Lite
Iblog
IndeedBot
Indy Library
InfoNaviRobot
InfoTekies
instabid
InterGET
IRLbot
JennyBot
JetCar
JikeSpider
JustView
Kozmosbot
Lanshanbot
Larbin
LeechFTP
LexiBot
Lftp
LibWeb
Libwhisker
Likse
LinkextractorPro
LinkScan
LinqiaScrapeBot
Lipperhey
LNSpiderguy
Ltx71
LWP::Simple
lwp-trivial
Magnet
magpie-crawler
Mail.RU_Bot
Majestic12
Majestic SEO
MarkMonitor
MarkWatch
Mass Downloader
Mata Hari
MauiBot
MeanPath Bot
mediawords
MegaIndex.ru
MFC_Tear_Sample
MIIxpc
Mister PiX
MJ12bot
Morfeus Fucking Scanner
MSFrontPage
MSIECrawler
Musobot
NearSite
Nessus
NetAnts
Netcraft
netEstate NE Crawler
NetSpider
NextGenSearchBot
Nibbler
NICErsPRO
Nikto
Nimbostratus
Ninja
NPbot
oBot
Offline Explorer
Offline Navigator
Openvas
OutclicksBot
PageAnalyzer
Page Analyzer
PageGrabber
pcBrowser
PECL::HTTP
PeoplePal
PictureFinder
Pixray
PleaseCrawl
POE-Component-Client-HTTP
ProWebWalker
Pump
PxBroker
PyCurl
QueryN Metasearch
RankActive
RankFlex
RankurBot
RealDownload
Reaper
Recorder
Ripper
RocketCrawler
Rogerbot
SalesIntelligent
SBIder
ScanAlert
Scanbot
Screaming
SEOkicks
seoscanners
SEOstats
Siphon
SISTRIX
Sitebeam
SiteExplorer
Siteimprove
Site Sucker
Sitevigil
SMTBot
Sogou web spider
Sosospider
Spanner
spyfu
Sqworm
Sucuri
SuperHTTP
Swiftbot
sysscan
Szukacz
T0PHackTeam
TeleportPro
Telesoft
The Intraformant
Titan
Toweyabot
trendiction.com
True_Robot
Turnitin
TurnitinBot
TwengaBot
Twice
Typhoeus
URLy.Warning
Vacuum
Vagabondo
VB Project
VidibleScraper
Voltron
Wallpapers/3.0
WASALive-Bot
Webalta
WebBandit
WebCollage
Web Collage
WEBDAV
Web Fetch
WebFuck
WebImageCollector
WebLeacher
WebmasterWorldForumBot
webmeup-crawler
WebPix
WebSauger
Website Quester
WebStripper
WebWhacker
WebZIP
WeSEE
Whacker
Whatweb
Who.is Bot
Widow
WinHTTrack
WiseGuys Robot
WISENutbot
Wonderbot
WPScan
WWW::Mechanize
WWWOFFLE
x22Mozilla
Xenu
xpymep1.exe
zauba.io
Zeus
zgrab
Zitebot
ZyBorg