404checker
404enemy
Abonti
Aboundex
AhrefsBot
Aipbot
Alexibot
Alligator
AllSubmitter
ASPSeek
Asterias
backlink-check
BacklinkCrawler
BackStreet
Badass
Bandit
BatchFTP
Battleztar Bazinga
BDCbot
BetaBot
BlackWidow
BLEXBot
Blow
BlowFish
Bolt
Brandwatch
Buddy
Bullseye
BunnySlippers
CCBot
Cegbfeieh
CherryPicker
ChinaClaw
Cliqzbot
Cloud mapping
Cogentbot
cognitiveseo
com.plumanalytics
Copier
Copyscape
Craftbot
crawler.feedback
crawl.sogou.com
Crescent
CSHttp
Curious
DBLBot
Demon
Devil
DigitalPebble
Disco
Discobot
DnyzBot
DomainCrawler
DomainStatsBot
Dragonfly
Drip
EMail Wolf
Extreme Picture Finder
FDM
Flunky
Foobot
FyberSpider
Fyrebot
GalaxyBot
GermCrawler
GetWeb
Go-Ahead-Got-It
GoZilla
Grafula
GridBot
HaosouSpider
Havij
HTTP::Lite
Humanlinks
HybridBot
Iblog
Id-search
IlseBot
Image Sucker
InfoNaviRobot
InfoTekies
Intelliseek
Internet Ninja
InternetSeer
internetVista monitor
ips-agent
Iria
Jbrofuzz
JennyBot
Jetty
JikeSpider
Jorgee
Kenjin Spider
Kozmosbot
Larbin
LeechFTP
Likse
LinkextractorPro
LinksManager
LinkWalker
LinqiaMetadataDownloaderBot
Lipperhey
Litemage_walker
Ltx71
lwp-request
LWP::Simple
Magnet
Mag-Net
magpie-crawler
Mail.RU_Bot
MarkMonitor
MarkWatch
Mass Downloader
Mata Hari
MauiBot
Mediatoolkitbot
MFC_Tear_Sample
Microsoft Data Access
Microsoft URL Control
MIDown tool
MIIxpc
MJ12bot
Morfeus Fucking Scanner
Mr.4x3
MSIECrawler
MS Web Services Client Protocol
Name Intelligence
Navroad
NearSite
Netcraft
NetLyzer
NetMechanic
NetSpider
Nettrack
NextGenSearchBot
Nibbler
Niki-bot
Ninja
Nmap
Nutch
oBot
Offline Navigator
Openfind
OpenVAS
PageAnalyzer
PageGrabber
Panscient
Papa Foto
Pavuk
pcBrowser
PECL::HTTP
PHPCrawl
Picsearch
PictureFinder
Pi-Monster
POE-Component-Client-HTTP
ProPowerBot
Psbot
QueryN Metasearch
RankingBot
RealDownload
RepoMonkey
Ripper
RocketCrawler
Rogerbot
s1z.ru
Scrapy
Screaming
ScreenerBot
Searchestate
Semrush
SEOkicks-Robot
SEOlyticsCrawler
Shodan
SISTRIX
Sitebeam
SiteSnagger
Sitevigil
SMTBot
Snake
SocialRankIOBot
Sociscraper
sogouspider
Sogou web spider
Sosospider
SpaceBison
Spammen
SpankBot
sp_auditbot
Sqlmap
Sqlworm
Stripper
SurveyBot
Suzuran
tAkeOut
TeleportPro
Telesoft
Telesphoreo
Telesphorep
Thumbor
Titan
Toata
Toweyabot
Trendiction
Turingos
Turnitin
TurnitinBot
Twice
UnisterBot
URLy Warning
URLy.Warning
Vacuum
Vagabondo
VB Project
VCI
WallpapersHD
Web Auto
WebCollage
Web Collage
WebCopier
Web Fetch
WebReaper
WebSauger
Web Sauger
Webshag
WebsiteQuester
Webster
WebSucker
Web Sucker
WebWhacker
WebZIP
WeSEE
Whacker
WiseGuys Robot
Wonderbot
Woobot
WPScan
WWW-Collector-E
WWW::Mechanize
x22Mozilla
Xaldon WebSpider
xpymep1.exe
YoudaoBot
Zauba
Zermelo
zgrab
Zitebot
ZmEu
ZumBot