404checker
ALittle Client
Abonti
Aboundexbot
AhrefsBot
AiHitBot
Alexibot
AllSubmitter
Anarchie
Anarchy
Apexoo
Attach
AwarioRssBot
BBBike
BDFetch
BLEXBot
BackStreet
BackWeb
BacklinkCrawler
Bandit
Bigfoot
Bitacle
BlackWidow
Blackboard
Blow
BuiltBotTough
CSHttp
Calculon
Cegbfeieh
CheTeam
Claritybot
Cliqzbot
Cloud mapping
Cogentbot
Copier
CopyRightCheck
Copyscape
Cosmos
Craftbot
CrazyWebCrawler
Crescent
Custo
DIIbot
DTS Agent
DataCha0s
DatabaseDriverMysqli
Demon
DigitalPebble
Dirbuster
Discobot
DomainAppender
Dragonfly
ECCP/1.0
Exabot
ExtLinksBot
EyeNetIE
FemtosearchBot
Foobot
GT::WWW
GalaxyBot
Genieo
GetRight
Gigabot
GoZilla
Grabber
GrapeFX
GrapeshotCrawler
HEADMasterSEO
HTTP::Lite
HTTrack
Id-search
Image Fetch
Image Sucker
Indy Library
Intelliseek
Iria
IstellaBot
JamesBOT
Jbrofuzz
Jyxobot
Kinza
LWP::Simple
Lanshanbot
Larbin
LeechFTP
LexiBot
Likse
LinkScan
Linkbot
LinksManager
Lipperhey Spider
Ltx71
MFC_Tear_Sample
MQQBrowser
MSFrontPage
Mag-Net
Mail.RU_Bot
Majestic SEO
Majestic-SEO
Majestic12
MarkMonitor
MarkWatch
Mass Downloader
Masscan
MauiBot
Mediatoolkitbot
Metauri
MicroMessenger
Mister PiX
Moblie Safari
NICErsPRO
NPbot
Navroad
NetAnts
Nettrack
NextGenSearchBot
Not
Nuclei
OrangeSpider
PECL::HTTP
PHPCrawl
Page Analyzer
Panscient
Papa Foto
PeoplePal
Picscout
Picsearch
PictureFinder
PleaseCrawl
Pockey
ProPowerBot
ProWebWalker
Psbot
QueryN Metasearch
RSSingBot
RankActive
RankActiveLinkBot
RankFlex
RankingBot2
Recorder
RedesScrapy
SEOlyticsCrawler
SISTRIX
SMTBot
Scanbot
ScoutJet
Scrapy
Screaming
SearchmetricsBot
SemrushBot
SeobilityBot
Shodan
SiteExplorer
Siteimprove
SlySearch
Sosospider
Spammen
Spanner
Spbot
Sqworm
Sucuri
SuperBot
SurveyBot
Suzuran
Swiftbot
T8Abot
Teleport
TeleportPro
Telesphoreo
Telesphorep
TheNomad
Thumbor
Toweyabot
Tracemyfile
Trendiction
True_Robot
Typhoeus
URLy Warning
URLy.Warning
UnisterBot
V-BOT
VelenPublicWebCrawler
VidibleScraper
VoidEYE
Voil
Voltron
WEBDAV
WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0
WeSEE
Web Enhancer
Web Pix
Web Sauger
Web Sucker
WebAuto
WebBandit
WebCollage
WebEnhancer
WebGo IS
WebPix
WebReaper
WebSauger
WebZIP
WebmasterWorldForumBot
Webshag
Website Quester
Whacker
Who.is Bot
Wotbox
Xaldon_WebSpider
Zauba
Zermelo
Zitebot
ZoominfoBot
ZumBot
ZyBorg
adscanner
arquivo-web-crawler
com.plumanalytics
dataforseo.com
domainsproject.org
eCatch
heritrix
instabid
internetVista monitor
linkdexbot
lwp-trivial
meanpathbot
mediawords
muhstik-scan
oBot
page scorer
pcBrowser
plumanalytics
ripz
s1z.ru
satoristudio.net
scalaj-http
seobility
seoscanners
seostar
serpstatbot
sexsearcher
sogouspider
sp_auditbot
voyagerx.com
webprosbot
x09Mozilla
x22Mozilla