360Spider
Abonti
Aboundex
ADmantX
AiHitBot
Aipbot
Alexibot
AllSubmitter
AlphaBot
Anarchie
Anarchy
Anarchy99
Apexoo
arquivo-web-crawler
Asterias
autoemailspider
AwarioSmartBot
BackDoorBot
Backlink-Ceck
Bandit
Barkrowler
Battleztar Bazinga
BDFetch
BetaBot
Bitacle
Blackboard
Black Hole
BlackWidow
Blow
Bolt
BuiltBotTough
Bullseye
BuzzSumo
Calculon
CCBot
CensysInspect
check1.exe
Chlooe
Claritybot
Cliqzbot
Copier
Cosmos
CSHttp
demandbase-bot
DIIbot
Discobot
Discoverybot
Dispatch
DnyzBot
Domains Project
DomainStatsBot
DomCopBot
Download Wonder
Dragonfly
DTS Agent
ECCP/1.0
EroCrawler
Exabot
Express WebPictures
ExtractorPro
EyeNetIE
facebookscraper
Fimap
FlashGet
Flunky
Foobot
FrontPage
GalaxyBot
GermCrawler
GetRight
Gigabot
Go!Zilla
GrabNet
GrapeFX
GrapeshotCrawler
GridBot
Haansoft
HaosouSpider
Harvest
HEADMasterSEO
Hloader
HTMLparser
HTTP::Lite
Iblog
IDBTE4M
Id-search
Indy Library
InfoNaviRobot
Intelliseek
Internet Ninja
Iria
isitwp.com
Iskanie
IstellaBot
Jetty
JOC Web Spider
Joomla
Kenjin Spider
Larbin
LexiBot
Libwhisker
Likse
LinkextractorPro
LinksManager
LinqiaRSSBot
LNSpiderguy
LWP::Simple
lwp-trivial
Majestic-SEO
MarkWatch
MeanPath Bot
mediawords
MegaIndex.ru
Microsoft URL Control
MIDown tool
Mister PiX
MJ12bot
Moblie Safari
Mojeek
Morfeus Fucking Scanner
Mozlila
Mr.4x3
MSIECrawler
muhstik-scan
Musobot
Name Intelligence
NearSite
Needle
NetAnts
netEstate NE Crawler
Netvibes
NetZIP
NextGenSearchBot
Nikto
NimbleCrawler
Nimbostratus
Ninja
Octopus
Offline Explorer
Offline Navigator
Openfind
OPPO A33
OutclicksBot
OutfoxBot
PageGrabber
page scorer
PageThing.com
Pandalytics
Papa Foto
PECL::HTTP
Petalbot
PHPCrawl
PleaseCrawl
ProWebWalker
Psbot
Pu_iN
PxBroker
QueryN Metasearch
Quick-Crawler
RankFlex
RankingBot
RankingBot2
Rankivabot
RankurBot
RealDownload
RepoMonkey
RocketCrawler
RSSingBot
SalesIntelligent
scalaj-http
scan.lol
ScoutJet
Scrapy
ScrepyBot
Searchestate
SearchmetricsBot
Semrush
SEOstats
serpstatbot
sexsearcher
SiteLockSpider
siteripz
SiteSucker
Sitevigil
SmartDownload
SMTBot
Snake
Snapbot
Sogou web spider
Sottopop
SpaceBison
Spammen
SpankBot
Spanner
sp_auditbot
SputnikBot
Sqworm
Steeler
Sucker
Sucuri
SuperBot
Surfbot
SurveyBot
sysscan
T0PHackTeam
T8Abot
tAkeOut
TeleportPro
Telesoft
Telesphoreo
TheNomad
TightTwatBot
Turingos
Turnitin
TwengaBot
Twice
URLy Warning
Vacuum
Vagabondo
VCI
VidibleScraper
voyagerx.com
Wallpapers/3.0
WBSearchBot
WebCollage
Web Collage
WebCopier
WEBDAV
WebGo IS
WebLeacher
Webster
WebSucker
Web Sucker
WebZIP
WeSEE
Widow
WiseGuys Robot
Woobot
Wotbox
WWW-Collector-E
WWW-Mechanize
x22Mozilla
Xaldon_WebSpider
YoudaoBot
Zauba
Zeus
zgrab
Zitebot
ZmEu
ZoominfoBot