404checker
404enemy
AIBOT
Aboundex
Aboundexbot
Acunetix
Aipbot
Anthill
Apexoo
Aspiegel
Asterias
AwarioRssBot
BLEXBot
BackDoorBot
Badass
Barkrowler
Battleztar Bazinga
Bitacle
Blackboard
Bolt
Brandwatch
Buddy
BuiltWith
Bullseye
BuzzSumo
CCBot
CSHttp
Calculon
CheeseBot
Chlooe
Citoid
Claritybot
Cliqzbot
Cocolyzebot
Cogentbot
Copier
Cosmos
Craftbot
Crawling at Home Project
Crescent
CrunchBot
Custo
CyotekWebCopy
DIIbot
DatabaseDriverMysqli
Demon
Devil
Discobot
Dispatch
DnBCrawler-Analytics
DnyzBot
DomainSigmaCrawler
DomainStatsBot
Download Wonder
EMail Siphon
EMail Wolf
EirGrabber
EroCrawler
Evil
ExtLinksBot
FHscan
FemtosearchBot
Fimap
FlashGet
FrontPage
Fuzz
FyberSpider
Genieo
GermCrawler
Getintent
Go!Zilla
Grafula
GrapeFX
HTMLparser
HTTrack
Harvest
Heritrix
HonoluluBot
Humanlinks
HybridBot
Indy Library
Intelliseek
InterGET
Iria
Iskanie
JetCar
Jetty
JikeSpider
Joomla
Jorgee
JustView
Jyxobot
Kenjin Spider
LNSpiderguy
LWP::Simple
Larbin
Leap
LeechGet
LibWeb
LinqiaRSSBot
Lipperhey Spider
Ltx71
MIIxpc
MSFrontPage
MSIECrawler
Magnet
MarkMonitor
Masscan
Mb2345Browser
MeanPath Bot
Microsoft URL Control
Mister PiX
Moblie Safari
MolokaiBot
Msrabot
Musobot
Name Intelligence
Navroad
NearSite
Needle
NetSpider
NextGenSearchBot
NimbleCrawler
Ninja
Nutch
Octopus
Offline Navigator
OpenVAS
Openfind
OrangeBot
OutclicksBot
PHPCrawl
POE-Component-Client-HTTP
PageThing.com
Panscient
Pavuk
PeoplePal
Pi-Monster
Piepmatz
Pimonster
Pixray
ProPowerBot
ProWebWalker
Pu_iN
Pump
QueryN Metasearch
RankActiveLinkBot
RankFlex
Rankivabot
Re-re
ReGet
RealDownload
Reaper
RebelMouse
Recorder
RepoMonkey
Rogerbot
SEOlyticsCrawler
SMTBot
SalesIntelligent
Scrapy
Screaming
ScreenerBot
ScrepyBot
SearchmetricsBot
SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
Shodan
Siphon
Site Sucker
SiteSnagger
SiteSucker
Sitebeam
Sitevigil
SlySearch
SmartDownload
Snoopy
Sociscraper
Sosospider
Spammen
Spinn3r
Sqlmap
Sqlworm
Sqworm
Steeler
Surfbot
Suzuran
Szukacz
TeleportPro
Telesphorep
The Intraformant
TheNomad
TightTwatBot
Toata
Toweyabot
True_Robot
Turnitin
URLy Warning
Vacuum
Vagabondo
VelenPublicWebCrawler
VoidEYE
Voil
Voltron
WWW-Mechanize
Web Fetch
Web Sucker
WebBandit
WebFetch
WebImageCollector
WebLeacher
WebPix
WebReaper
WebSucker
WebmasterWorldForumBot
Webster
Wonderbot
Xaldon WebSpider
Xenu
YoudaoBot
Zade
Zeus
Zitebot
archive.org_bot
cah.io.community
coccocbot-web
crawl.sogou.com
dataforseo.com
demandbase-bot
eCatch
instabid
isitwp.com
iubenda-radar
lwp-request
mediawords
muhstik-scan
oBot
page scorer
pcBrowser
ripz
scan.lol
seoscanners
seostar
siteripz
tAkeOut
webgains-bot
webprosbot
zgrab