404checker
80legs
Abonti
Aboundex
Aboundexbot
Acunetix
ADmantX
AIBOT
Aipbot
Alexibot
AllSubmitter
Apexoo
archive.org_bot
ASPSeek
autoemailspider
BacklinkCrawler
BackStreet
Badass
Barkrowler
BBBike
BDCbot
BetaBot
Bigfoot
Bitacle
BlackWidow
Brandprotect
BuiltBotTough
BuiltWith
Bullseye
Calculon
CazoodleBot
CheTeam
Claritybot
coccocbot-web
cognitiveseo
Collector
Copier
Copyscape
crawler4j
crawler.feedback
crawl.sogou.com
CrazyWebCrawler
Crescent
CSHttp
DataCha0s
DBLBot
demandbase-bot
Deusu
Digincore
DigitalPebble
DIIbot
Discobot
Discoverybot
Dispatch
DomainCrawler
DomainSigmaCrawler
Download Wonder
DSearch
Ebingbong
eCatch
Ecxi
EirGrabber
EMail Siphon
EroCrawler
Evil
ExtLinksBot
Extractor
Extreme Picture Finder
Ezooms
FDM
Fimap
FlashGet
Flunky
Freeuploader
FrontPage
Getintent
GetRight
GoZilla
Grabber
Grafula
Haansoft
HaosouSpider
Harvest
Havij
Heritrix
Hloader
HTMLparser
Id-search
IndeedBot
InfoNaviRobot
instabid
InternetSeer
ips-agent
Iria
IRLbot
JamesBOT
Jetty
Jyxobot
Kenjin Spider
Kozmosbot
Larbin
LeechFTP
LeechGet
LexiBot
Lftp
Libwhisker
Lightspeedsystems
Linkdexbot
LinkextractorPro
LinqiaRSSBot
Lmspider
Ltx71
Mail.RU_Bot
Majestic12
Majestic-SEO
MarkWatch
Masscan
Mata Hari
Meanpathbot
MeanPath Bot
Mediatoolkitbot
mediawords
Metauri
MFC_Tear_Sample
Microsoft Data Access
Microsoft URL Control
Mr.4x3
MSFrontPage
Musobot
Needle
Nessus
NetAnts
Netcraft
NetMechanic
Nettrack
Netvibes
NextGenSearchBot
Nibbler
NICErsPRO
Niki-bot
Nikto
NimbleCrawler
Ninja
NPbot
Octopus
OutfoxBot
Page Analyzer
page scorer
PageScorer
Papa Foto
Picsearch
Pi-Monster
Pixray
Pockey
POE-Component-Client-HTTP
Probethenet
ProWebWalker
Psbot
QueryN Metasearch
RankActiveLinkBot
RankFlex
RankurBot
Reaper
Ripper
s1z.ru
SalesIntelligent
ScanAlert
Scrapy
SearchmetricsBot
Semrush
SemrushBot
SEOkicks
SEOlyticsCrawler
SeoSiteCheckup
SEOstats
sexsearcher
Shodan
SISTRIX
Sitebeam
SiteSucker
Sitevigil
SlySearch
SMTBot
Snapbot
sogouspider
Sogou web spider
Sosospider
Sottopop
Spammen
SpankBot
sp_auditbot
Sqlworm
Sqworm
Steeler
Sucker
Sucuri
Surfbot
SurveyBot
sysscan
Szukacz
Teleport
TeleportPro
Toweyabot
Tracemyfile
Trendictionbot
True_Robot
Twice
Typhoeus
URLy Warning
Vagabondo
VB Project
VeriCiteCrawler
VoidEYE
Voltron
WASALive-Bot
WebAuto
Web Auto
WebBandit
WebCollage
Web Collage
WebEnhancer
WebFetch
Web Fuck
WebGo IS
webmeup-crawler
WebReaper
Web Sauger
WebsiteQuester
Webster
WebStripper
WebWhacker
WeSEE
Whatweb
Who.is Bot
Wprecon
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Xaldon_WebSpider
xpymep1.exe
YoudaoBot
Zade
Zauba
zauba.io
Zermelo
Zeus
zgrab
ZumBot