360Spider
404checker
404enemy
AIBOT
ALittle Client
Abonti
Aboundexbot
Acunetix
AiHitBot
Anarchie
Aspiegel
Asterias
AwarioRssBot
BBBike
BDCbot
BackStreet
Bandit
BatchFTP
Bigfoot
BlackWidow
BotALot
Brandprotect
BuiltWith
BunnySlippers
CCBot
CSHttp
Calculon
CheTeam
CheeseBot
CherryPicker
Claritybot
Cliqzbot
Cogentbot
CopyRightCheck
Cosmos
Craftbot
Custo
DBLBot
DIIbot
DSearch
DTS Agent
DataCha0s
Demon
Discoverybot
Dispatch
DomCopBot
DomainAppender
Dotbot
Dragonfly
EMail Siphon
EasyDL
Ebingbong
Express WebPictures
ExtLinksBot
FDM
Firefox/7.0
Foobot
Fuzz
FyberSpider
GT::WWW
GalaxyBot
Genieo
GetRight
GetWeb
Gotit
GrabNet
Grabber
GrapeFX
HTMLparser
Haansoft
HaosouSpider
Havij
Heritrix
IDBot
IRLbot
Indy Library
Intelliseek
InternetSeer
Iria
Iskanie
Jbrofuzz
JennyBot
Jetty
JikeSpider
Jyxobot
Kenjin Spider
Kozmosbot
LNSpiderguy
LWP::Simple
Lanshanbot
Larbin
Leap
LexiBot
LibWeb
LinksManager
LinqiaScrapeBot
Litemage_walker
MFC_Tear_Sample
MIIxpc
MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot
Majestic-SEO
Majestic12
Mass Downloader
MeanPath Bot
Mediatoolkitbot
Microsoft Data Access
Microsoft URL Control
Mister PiX
Moblie Safari
Mojeek
Msrabot
Musobot
NICErsPRO
NPbot
Name Intelligence
Nameprotect
Nessus
NetAnts
NetSpider
Netcraft
NextGenSearchBot
Nmap
Offline Navigator
OnCrawl
OpenVAS
Openfind
OrangeBot
PageAnalyzer
PageGrabber
Panscient
Pi-Monster
Picsearch
Pixray
PleaseCrawl
ProWebWalker
Probethenet
Pump
Quick-Crawler
RSSingBot
RankActiveLinkBot
RankFlex
RankingBot
RankurBot
Re-re
Recorder
RepoMonkey
SBIder
SEOprofiler
SISTRIX
ScanAlert
ScreenerBot
SearchmetricsBot
SemrushBot
Seomoz
SiteCheckerBotCrawler
SiteLockSpider
Sitebeam
Sitevigil
SlySearch
Snapbot
Snoopy
SocialRankIOBot
Sociscraper
SpaceBison
SpankBot
Sqlmap
Steeler
Stripper
Sucker
SuperBot
Surfbot
Suzuran
Szukacz
Teleport
Telesoft
Thumbor
TightTwatBot
Tracemyfile
Trendiction
True_Robot
TwengaBot
URLy Warning
URLy.Warning
UnisterBot
Upflow
VCI
Vacuum
VelenPublicWebCrawler
WISENutbot
WallpapersHD
Web Collage
Web Enhancer
Web Fetch
Web Fuck
Web Pix
Web Sauger
Web Sucker
WebAuto
WebCopier
WebFetch
WebFuck
WebGo IS
WebLeacher
WebPix
WebSauger
WebSucker
WebZIP
WebmasterWorldForumBot
Webshag
Website Quester
WebsiteExtractor
Who.is Bot
Widow
Woobot
Wotbox
Xaldon WebSpider
YoudaoBot
Zermelo
Zitebot
ZmEu
clark-crawler
coccocbot
crawl.sogou.com
dataforseo.com
domainsproject.org
facebookscraper
internetVista monitor
linkdexbot
lwp-request
lwp-trivial
mediawords
oBot
page scorer
plumanalytics
scan.lol
seoscanners
seostar
sitechecker.pro
siteripz
sogouspider
ubermetrics-technologies.com
voyagerx.com
webgains-bot
webprosbot
x22Mozilla
xpymep1.exe
zgrab