404checker
404enemy
Aboundex
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
Aipbot
Alligator
Anarchy99
Anthill
Apexoo
Asterias
Atomseobot
BBBike
BDCbot
BackDoorBot
BackStreet
Barkrowler
BatchFTP
Battleztar Bazinga
Bigfoot
Bitacle
BlackWidow
Blow
Bolt
BuiltBotTough
BunnySlippers
CATExplorador
CSHttp
Calculon
CazoodleBot
CheTeam
Chlooe
Cloud mapping
Collector
Copier
CopyRightCheck
Craftbot
CrazyWebCrawler
Crescent
CrunchBot
Curious
DBLBot
DIIbot
Demon
DigitalPebble
Dirbuster
Disco
Dispatch
DnyzBot
DomainAppender
Dotbot
Download Wonder
Drip
EroCrawler
Exabot
Express WebPictures
ExtractorPro
Flunky
Freeuploader
Fuzz
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GT::WWW
Genieo
GetRight
GetWeb
Go!Zilla
Go-Ahead-Got-It
GoZilla
GridBot
HTMLparser
HTTP::Lite
Iblog
Image Fetch
Indy Library
InfoTekies
InterGET
Iria
IstellaBot
Jetty
JikeSpider
Joomla
Keyword Density
Kozmosbot
LeechFTP
Lftp
LibWeb
LieBaoFast
Likse
LinkScan
LinkextractorPro
Lipperhey
Lipperhey Spider
MIDown tool
MSFrontPage
MTRobot
Magnet
Mail.RU_Bot
Masscan
MauiBot
MeanPath Bot
Mediatoolkitbot
Metauri
Microsoft URL Control
Mister PiX
Moblie Safari
Mojolicious
Morfeus Fucking Scanner
Mozlila
Mr.4x3
Musobot
NPbot
Navroad
NearSite
Nessus
NetSpider
NetZIP
Netcraft
Netvibes
NextGenSearchBot
Nibbler
Nikto
NimbleCrawler
Nimbostratus
Ninja
Not
Nutch
Octopus
OrangeBot
OutfoxBot
POE-Component-Client-HTTP
Pandalytics
Panscient
Picscout
PictureFinder
Pixray
ProPowerBot
ProWebWalker
Pu_iN
Pump
PyCurl
RankActive
RankurBot
Re-re
ReGet
RocketCrawler
SEOkicks
SEOprofiler
SalesIntelligent
Scanbot
SearchmetricsBot
SemanticJuice
Semrush
SemrushBot
SeoSiteCheckup
Siphon
Site Sucker
SiteCheckerBotCrawler
SiteSnagger
Sitebeam
Siteimprove
Sitevigil
SmartDownload
Snapbot
SocialRankIOBot
Sociscraper
Sosospider
Spbot
Spinn3r
Sqlworm
Sqworm
Steeler
Stripper
Sucuri
SuperBot
SurveyBot
Szukacz
T8Abot
Teleport
Telesoft
Thumbor
TightTwatBot
Titan
Toata
True_Robot
TurnitinBot
Typhoeus
UnisterBot
VeriCiteCrawler
VidibleScraper
VoidEYE
Voil
WASALive-Bot
WISENutbot
WPScan
WWW-Collector-E
WallpapersHD
WeSEE
Web Fuck
Web Sauger
WebCollage
WebCopier
WebFuck
WebGo IS
WebLeacher
WebSauger
WebWhacker
Webalta
WebmasterWorldForumBot
Webshag
Widow
Wonderbot
Xaldon_WebSpider
Xenu
YoudaoBot
Zauba
Zitebot
archive.org_bot
arquivo.pt
autoemailspider
backlink-check
clark-crawler
coccocbot
com.plumanalytics
crawl.sogou.com
crawler.feedback
dataforseo.com
eCatch
evc-batch
gopher
heritrix
internetVista monitor
ips-agent
isitwp.com
iubenda-radar
linkdexbot
mediawords
page scorer
probe-image-size
s1z.ru
seobility
seostar
tAkeOut
ubermetrics-technologies.com
webmeup-crawler
x22Mozilla
zauba.io
zgrab