404checker
Abonti
ADmantX
AhrefsBot
AIBOT
Alexibot
AlphaBot
Anarchie
Apexoo
archive.org_bot
ASPSeek
Attach
BackDoorBot
Backlink-Ceck
backlink-check
BacklinkCrawler
BackStreet
BackWeb
Bandit
BatchFTP
Battleztar Bazinga
BBBike
BetaBot
BLEXBot
Blow
Brandwatch
Buddy
Bullseye
BunnySlippers
BuzzSumo
Calculon
CCBot
Cegbfeieh
CherryPicker
Chlooe
coccocbot-web
Collector
com.plumanalytics
crawler4j
crawler.feedback
CrazyWebCrawler
Crescent
CrunchBot
CSHttp
Custo
DBLBot
demandbase-bot
Demon
Deusu
Devil
DIIbot
Disco
DomainCrawler
DSearch
EasyDL
eCatch
EirGrabber
evc-batch
Exabot
Express WebPictures
ExtLinksBot
EyeNetIE
Ezooms
facebookscraper
FemtosearchBot
FHscan
Fimap
Foobot
FyberSpider
Fyrebot
Genieo
GoZilla
Go!Zilla
Grabber
GrabNet
GrapeshotCrawler
GridBot
HaosouSpider
Harvest
Havij
HEADMasterSEO
Heritrix
Hloader
HTMLparser
HTTrack
Humanlinks
HybridBot
Image Fetch
IndeedBot
InfoNaviRobot
InfoTekies
instabid
InternetSeer
internetVista monitor
ips-agent
Jbrofuzz
JetCar
Jetty
Jorgee
Jyxobot
Keyword Density
Lanshanbot
Larbin
Lightspeedsystems
Likse
Linkdexbot
LinkextractorPro
LinkpadBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Lmspider
LNSpiderguy
Ltx71
lwp-request
MarkMonitor
MarkWatch
Mass Downloader
Mata Hari
MauiBot
meanpathbot
Meanpathbot
Metauri
Microsoft Data Access
Microsoft URL Control
MIDown tool
MJ12bot
Mojeek
MSFrontPage
MS Web Services Client Protocol
Musobot
Navroad
NearSite
Nessus
Nettrack
Net Vampire
Netvibes
NetZIP
NICErsPRO
NimbleCrawler
Nimbostratus
Ninja
NPbot
Nutch
Offline Explorer
OpenLinkProfiler
OutfoxBot
PageAnalyzer
PageScorer
Panscient
Pavuk
PeoplePal
PHPCrawl
Pixray
PleaseCrawl
plumanalytics
POE-Component-Client-HTTP
ProWebWalker
QueryN Metasearch
RankFlex
Rankivabot
RankurBot
RealDownload
Reaper
RebelMouse
ReGet
Ripper
SalesIntelligent
scan.lol
ScoutJet
Searchestate
SearchmetricsBot
SemrushBot
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SeoSiteCheckup
SiteLockSpider
Sitevigil
SlySearch
Snake
Snapbot
Sociscraper
Sogou web spider
Sottopop
Spanner
sp_auditbot
spyfu
Sqlworm
Sqworm
Steeler
Sucker
Sucuri
SuperBot
Surfbot
SurveyBot
Suzuran
Swiftbot
sysscan
T0PHackTeam
T8Abot
TeleportPro
Telesoft
Telesphorep
Titan
Toweyabot
Trendiction
True_Robot
Typhoeus
Vacuum
Vagabondo
VCI
VeriCiteCrawler
VidibleScraper
VoidEYE
Voltron
WASALive-Bot
WebCollage
WebCopier
WEBDAV
WebEnhancer
Web Fetch
WebFuck
WebImageCollector
WebLeacher
webmeup-crawler
WebPix
WebsiteExtractor
WebsiteQuester
WebSucker
WebWhacker
Whacker
Who.is Bot
Wonderbot
Wotbox
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Xaldon WebSpider
Xenu
YoudaoBot
Zade
zauba.io
Zeus
Zitebot
ZmEu
ZumBot
ZyBorg