404checker
404enemy
Abonti
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
AIBOT
Alexibot
Anarchie
Apexoo
archive.org_bot
Asterias
Attach
BacklinkCrawler
BDCbot
BDFetch
BetaBot
Bitacle
Black Hole
BlackWidow
Blow
BlowFish
Boardreader
Bolt
Brandwatch
BuiltBotTough
BuiltWith
Bullseye
Calculon
CCBot
Cegbfeieh
CheeseBot
CheTeam
ChinaClaw
Chlooe
Claritybot
Cloud mapping
Cogentbot
Copier
crawler4j
crawler.feedback
CrazyWebCrawler
Crescent
CrunchBot
CSHttp
DatabaseDriverMysqli
DataCha0s
DIIbot
Discoverybot
DittoSpyder
DomainSigmaCrawler
DomainStatsBot
Dragonfly
DSearch
DTS Agent
eCatch
EirGrabber
EMail Wolf
Express WebPictures
ExtLinksBot
Extractor
ExtractorPro
EyeNetIE
facebookscraper
FDM
FHscan
Firefox/7.0
FlashGet
Foobot
Freeuploader
FrontPage
Fyrebot
Genieo
GermCrawler
G-i-g-a-b-o-t
Go-Ahead-Got-It
Go!Zilla
Grafula
GrapeshotCrawler
GT::WWW
HEADMasterSEO
Heritrix
HTTP::Lite
Humanlinks
InterGET
InternetSeer
internetVista monitor
IRLbot
Iskanie
IstellaBot
Jbrofuzz
JetCar
JikeSpider
Jorgee
JustView
Jyxobot
Kenjin Spider
Keyword Density
Kozmosbot
Larbin
LeechGet
Lightspeedsystems
Likse
Linkdexbot
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey
Litemage_walker
magpie-crawler
Mail.RU_Bot
Majestic-SEO
Mata Hari
MauiBot
Meanpathbot
MeanPath Bot
Mediatoolkitbot
MegaIndex.ru
Metauri
Microsoft URL Control
MIIxpc
Mister PiX
MJ12bot
Mojeek
Morfeus Fucking Scanner
Mr.4x3
Msrabot
muhstik-scan
Musobot
NearSite
NetAnts
netEstate NE Crawler
Netvibes
NetZIP
NextGenSearchBot
NICErsPRO
Niki-bot
Nikto
NimbleCrawler
Ninja
Offline Navigator
Openfind
OpenVAS
OrangeSpider
PageAnalyzer
Page Analyzer
PageGrabber
page scorer
Panscient
Papa Foto
Picsearch
Pi-Monster
Pixray
PleaseCrawl
plumanalytics
POE-Component-Client-HTTP
Probethenet
Pump
PxBroker
QueryN Metasearch
RankActive
RankingBot2
Rankivabot
RealDownload
Reaper
Recorder
RedesScrapy
RepoMonkey
RocketCrawler
RSSingBot
s1z.ru
SBIder
ScanAlert
Scanbot
scan.lol
Screaming
Seomoz
SEOstats
serpstatbot
sexsearcher
Shodan
Siphon
SISTRIX
SiteExplorer
SiteLockSpider
SlySearch
SmartDownload
SMTBot
Snapbot
Snoopy
Sosospider
Sottopop
SpaceBison
Spanner
Spinn3r
SputnikBot
spyfu
SuperBot
SurveyBot
Suzuran
T0PHackTeam
tAkeOut
Teleport
Telesphorep
The Intraformant
Thumbor
Toata
Toweyabot
Tracemyfile
Trendiction
trendiction.com
trendiction.de
Turnitin
TwengaBot
Typhoeus
URLy Warning
URLy.Warning
Vacuum
Vagabondo
VCI
VeriCiteCrawler
VidibleScraper
Virusdie
WallpapersHD
WBSearchBot
WEBDAV
Web Enhancer
WebFetch
Web Fuck
WebLeacher
webmeup-crawler
WebPix
Web Pix
WebSauger
Web Sauger
WebSucker
Web Sucker
WebZIP
Whacker
Who.is Bot
WinHTTrack
Wonderbot
Wprecon
WWW-Collector-E
x09Mozilla
Xaldon WebSpider
YoudaoBot
Zade
zgrab
ZmEu