404checker
80legs
Abonti
Aboundex
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AiHitBot
Aipbot
Alligator
AllSubmitter
Apexoo
arquivo.pt
arquivo-web-crawler
Aspiegel
backlink-check
Badass
Bandit
BatchFTP
Battleztar Bazinga
BBBike
BetaBot
Bigfoot
Blackboard
Black Hole
Blow
BlowFish
Boardreader
BunnySlippers
BuzzSumo
CheeseBot
Chlooe
coccocbot-web
cognitiveseo
Copier
CopyRightCheck
Cosmos
Craftbot
crawler4j
crawl.sogou.com
Crescent
CrunchBot
Curious
Custo
demandbase-bot
Devil
Discobot
Dispatch
DnyzBot
DomainAppender
DomainCrawler
domainsproject.org
DomainStatsBot
Dotbot
eCatch
Ecxi
EirGrabber
EMail Wolf
evc-batch
Exabot
Express WebPictures
ExtractorPro
EyeNetIE
facebookscraper
FemtosearchBot
Firefox/7.0
FlashGet
Foobot
Freeuploader
GermCrawler
Getintent
GetRight
Gigablast
Gigabot
G-i-g-a-b-o-t
Go-Ahead-Got-It
GoZilla
Grabber
GrapeshotCrawler
GridBot
Haansoft
HaosouSpider
Havij
heritrix
Heritrix
HTTP::Lite
Humanlinks
Image Fetch
Image Sucker
InfoNaviRobot
InfoTekies
instabid
InterGET
InternetSeer
internetVista monitor
Iria
JamesBOT
JikeSpider
Joomla
JustView
Kinza
Lanshanbot
Larbin
LieBaoFast
Likse
LinkextractorPro
LinkpadBot
LinksManager
LinkWalker
Lipperhey
Litemage_walker
Lmspider
LWP::Simple
lwp-trivial
magpie-crawler
Mail.RU_Bot
Majestic12
Majestic-SEO
MarkMonitor
Masscan
meanpathbot
Mediatoolkitbot
MIDown tool
MIIxpc
Mister PiX
Mojeek
Mojolicious
MQQBrowser
Mr.4x3
MSIECrawler
Name Intelligence
Navroad
NearSite
Nessus
NetAnts
Netcraft
Net Vampire
NetZIP
NICErsPRO
Niki-bot
NimbleCrawler
Nimbostratus
Offline Navigator
OnCrawl
Openvas
OpenVAS
OrangeBot
Page Analyzer
PageGrabber
Panscient
Pavuk
pcBrowser
PECL::HTTP
PHPCrawl
Picscout
Picsearch
Pimonster
Pixray
PleaseCrawl
Probethenet
Psbot
Quick-Crawler
RankActive
Reaper
ReGet
RepoMonkey
RocketCrawler
Rogerbot
RSSingBot
s1z.ru
ScanAlert
Screaming
SearchmetricsBot
Semrush
SEOkicks
SEOkicks-Robot
SeoSiteCheckup
sexsearcher
Siphon
SISTRIX
Sitebeam
SiteExplorer
Siteimprove
SiteLockSpider
Site Sucker
SlySearch
SmartDownload
Snake
SocialRankIOBot
Sociscraper
sogouspider
Sogou web spider
Sosospider
Sottopop
SpankBot
sp_auditbot
Spbot
SputnikBot
Sqlmap
Sqlworm
Steeler
Sucker
Sucuri
SurveyBot
Swiftbot
sysscan
Teleport
Thumbor
Trendictionbot
trendiction.com
Turingos
Twice
UnisterBot
URLy Warning
URLy.Warning
Vacuum
VB Project
VCI
VeriCiteCrawler
WallpapersHD
WebAuto
WebCopier
WebEnhancer
Web Enhancer
WebFetch
Web Fuck
WebGo IS
WebImageCollector
WebmasterWorldForumBot
Web Sauger
Webshag
Web Sucker
WebWhacker
WebZIP
Widow
WiseGuys Robot
WISENutbot
Woobot
Wotbox
Wprecon
WPScan
WWW-Mechanize
WWWOFFLE
x09Mozilla
xpymep1.exe
zauba.io
Zermelo
ZmEu
ZumBot
ZyBorg