404checker
80legs
ADmantX
Aboundex
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
Aipbot
Alligator
Anarchie
Anarchy
Ankit
Anthill
Asterias
Attach
BLEXBot
BackWeb
Badass
Bigfoot
Bitacle
BlackWidow
BlowFish
BotALot
Brandprotect
Buck
Buddy
CATExplorador
Calculon
CazoodleBot
Cegbfeieh
CheeseBot
Chlooe
Cloud mapping
Cocolyzebot
Cogentbot
CopyRightCheck
Copyscape
Cosmos
Craftbot
Crescent
CrunchBot
Curious
DatabaseDriverMysqli
Demon
Deusu
Devil
DigitalPebble
Disco
Discoverybot
DomCopBot
Download Wonder
Drip
ECCP/1.0
EasyDL
EroCrawler
Evil
Express WebPictures
Extractor
FDM
FHscan
Firefox/7.0
Flunky
Freeuploader
Fuzz
FyberSpider
Fyrebot
Genieo
GetRight
GetWeb
Go-Ahead-Got-It
GoZilla
Gotit
GrabNet
Grabber
GrapeFX
GridBot
HEADMasterSEO
HaosouSpider
Harvest
IDBTE4M
Id-search
Image Fetch
IndeedBot
Indy Library
InfoNaviRobot
InternetSeer
Iskanie
IstellaBot
JOC Web Spider
JamesBOT
JennyBot
Jetty
Jorgee
Kenjin Spider
Kozmosbot
LNSpiderguy
Lanshanbot
LexiBot
LibWeb
LinkScan
Linkbot
LinksManager
LinqiaMetadataDownloaderBot
Lipperhey
Lipperhey Spider
Litemage_walker
MFC_Tear_Sample
MSFrontPage
Magnet
Majestic12
MarkMonitor
Mass Downloader
MeanPath Bot
Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri
Minefield
NPbot
NearSite
NetSpider
NetZIP
NimbleCrawler
Nimbostratus
Nmap
Nuclei
Nutch
Offline Explorer
Offline Navigator
OnCrawl
OpenLinkProfiler
OpenVAS
PECL::HTTP
POE-Component-Client-HTTP
Page Analyzer
Pavuk
Pi-Monster
Pimonster
Pixray
Pockey
Psbot
PxBroker
Quick-Crawler
RankActiveLinkBot
RankFlex
RankingBot2
ReGet
RebelMouse
Recorder
SEOkicks-Robot
SMTBot
SalesIntelligent
Scanbot
ScoutJet
Scrapy
ScreenerBot
Searchestate
SearchmetricsBot
SemanticJuice
SemrushBot
SiteCheckerBotCrawler
SiteExplorer
Sitevigil
SlySearch
Snake
Snapbot
Sociscraper
SpankBot
Spbot
Spinn3r
Sqworm
Sucuri
Surfbot
Szukacz
T0PHackTeam
TeleportPro
Telesphoreo
TheNomad
Tracemyfile
Trendiction
Turnitin
TurnitinBot
TwengaBot
Twice
Typhoeus
URLy.Warning
Upflow
VB Project
Vacuum
VelenPublicWebCrawler
VidibleScraper
Voltron
WASALive-Bot
WISENutbot
WWW::Mechanize
WWWOFFLE
Web Collage
Web Enhancer
Web Fetch
Web Sauger
Web Sucker
WebEnhancer
WebFetch
WebGo IS
WebPix
WebSauger
Webalta
WebsiteExtractor
WebsiteQuester
Xaldon_WebSpider
YoudaoBot
Zermelo
Zeus
ZmEu
ZumBot
arquivo-web-crawler
autoemailspider
backlink-check
cah.io.community
check1.exe
clark-crawler
crawler4j
domainsproject.org
facebookscraper
internetVista monitor
isitwp.com
lwp-request
lwp-trivial
magpie-crawler
netEstate NE Crawler
oBot
pcBrowser
ripz
s1z.ru
satoristudio.net
scalaj-http
scan.lol
seocompany.store
seostar
sexsearcher
sitechecker.pro
siteripz
sysscan
trendiction.com
trendiction.de
ubermetrics-technologies.com
webgains-bot
x22Mozilla
zauba.io
zgrab