360Spider
404checker
Aboundex
Acunetix
AhrefsBot
AiHitBot
archive.org_bot
ASPSeek
Asterias
BackDoorBot
BacklinkCrawler
BackWeb
Badass
BBBike
Bigfoot
Bitacle
Blackboard
BlackWidow
Blow
BlowFish
Boardreader
Brandwatch
Buddy
Bullseye
BuzzSumo
CazoodleBot
CCBot
CherryPicker
CheTeam
Chlooe
Cliqzbot
cognitiveseo
com.plumanalytics
Craftbot
crawler.feedback
CrunchBot
Demon
Deusu
Devil
Digincore
DIIbot
Disco
Discoverybot
DomainAppender
DomainCrawler
Dotbot
Dragonfly
DSearch
DTS Agent
EasyDL
Ebingbong
eCatch
ECCP/1.0
Ecxi
EMail Siphon
evc-batch
Exabot
ExtLinksBot
Extreme Picture Finder
EyeNetIE
facebookscraper
FHscan
Fimap
Firefox/7.0
Flunky
Freeuploader
FyberSpider
GalaxyBot
Genieo
GermCrawler
GetRight
Gigablast
Gigabot
Go-Ahead-Got-It
Gotit
GrapeFX
GrapeshotCrawler
GridBot
Haansoft
Harvest
HEADMasterSEO
HTTP::Lite
Id-search
Image Fetch
Image Sucker
InfoNaviRobot
instabid
Internet Ninja
InternetSeer
IRLbot
Iskanie
JamesBOT
JetCar
Joomla
JustView
Jyxobot
Keyword Density
Lanshanbot
LeechFTP
LibWeb
Libwhisker
Lightspeedsystems
Likse
Linkdexbot
LinkextractorPro
LinksManager
LinqiaScrapeBot
Litemage_walker
Ltx71
lwp-request
LWP::Simple
Magnet
Mag-Net
Majestic SEO
MarkMonitor
Mata Hari
Mediatoolkitbot
Microsoft Data Access
MIDown tool
Mister PiX
Morfeus Fucking Scanner
Mr.4x3
MSFrontPage
Msrabot
muhstik-scan
Musobot
Name Intelligence
Nameprotect
Navroad
netEstate NE Crawler
Nibbler
NimbleCrawler
Ninja
Nmap
Nutch
oBot
Offline Navigator
Openfind
OrangeSpider
OutclicksBot
OutfoxBot
PageAnalyzer
Page Analyzer
PageScorer
Panscient
Papa Foto
PECL::HTTP
PeoplePal
Picscout
Picsearch
Pi-Monster
Pixray
POE-Component-Client-HTTP
ProWebWalker
Psbot
PxBroker
QueryN Metasearch
Quick-Crawler
RankActive
RankFlex
RankingBot2
Rankivabot
Reaper
RebelMouse
Ripper
s1z.ru
SBIder
Scanbot
scan.lol
ScoutJet
Scrapy
Screaming
ScreenerBot
SearchmetricsBot
Semrush
SEOlyticsCrawler
Seomoz
Siphon
SISTRIX
Siteimprove
SiteSnagger
Sitevigil
SMTBot
Snoopy
Sociscraper
Sogou web spider
Sosospider
Sottopop
Spbot
Spinn3r
SputnikBot
spyfu
Sqlmap
Sqlworm
Sucuri
Surfbot
Swiftbot
sysscan
Szukacz
T0PHackTeam
T8Abot
tAkeOut
Teleport
TeleportPro
Telesoft
Telesphorep
The Intraformant
TightTwatBot
Titan
Trendiction
trendiction.com
True_Robot
TurnitinBot
Typhoeus
Upflow
URLy.Warning
VB Project
VCI
VeriCiteCrawler
Voil
Wallpapers/3.0
WallpapersHD
WBSearchBot
Webalta
WebAuto
WebBandit
WebEnhancer
WebLeacher
webmeup-crawler
Web Sauger
Webshag
WebsiteQuester
Website Quester
WebWhacker
WebZIP
WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Wprecon
WWW-Collector-E
WWWOFFLE
x09Mozilla
x22Mozilla
Xenu
YoudaoBot
Zauba
Zermelo
zgrab
ZumBot