404checker
404enemy
80legs
Aboundex
Aboundexbot
Acunetix
ADmantX
AfD-Verbotsverfahren
Alexibot
Alligator
ASPSeek
Asterias
Backlink-Ceck
BackStreet
BackWeb
BatchFTP
BBBike
BetaBot
Bigfoot
Bitacle
Blackboard
Black Hole
BlackWidow
Blow
Bolt
BotALot
Buddy
Bullseye
BuzzSumo
Calculon
CCBot
Cliqzbot
Cogentbot
Cosmos
crawler4j
crawl.sogou.com
CSHttp
Curious
DatabaseDriverMysqli
Deusu
DIIbot
Disco
Discobot
Discoverybot
Dispatch
DnyzBot
DomainStatsBot
Download Wonder
Drip
EasyDL
Ebingbong
ECCP/1.0
EroCrawler
Evil
Exabot
ExtLinksBot
Extractor
Extreme Picture Finder
FDM
FHscan
Firefox/7.0
FlashGet
Foobot
FrontPage
FyberSpider
Fyrebot
GalaxyBot
Genieo
Gigablast
Go-Ahead-Got-It
GrapeFX
GridBot
GT::WWW
Haansoft
Harvest
HEADMasterSEO
HTMLparser
HTTP::Lite
HTTrack
HybridBot
IDBot
IlseBot
Image Sucker
InternetSeer
internetVista monitor
ips-agent
Iria
IstellaBot
JamesBOT
Jbrofuzz
JetCar
Jetty
JOC Web Spider
Joomla
Jorgee
JustView
Jyxobot
Keyword Density
Lftp
Libwhisker
Lightspeedsystems
Likse
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey
Lipperhey Spider
Litemage_walker
lwp-request
lwp-trivial
Majestic SEO
Masscan
Mass Downloader
Mata Hari
meanpathbot
mediawords
MegaIndex.ru
MFC_Tear_Sample
Microsoft Data Access
MIDown tool
MIIxpc
MJ12bot
Mojeek
Morfeus Fucking Scanner
MSFrontPage
MSIECrawler
Msrabot
Musobot
Name Intelligence
Navroad
Needle
NetAnts
Netcraft
NetSpider
Net Vampire
NextGenSearchBot
NimbleCrawler
Nimbostratus
Ninja
oBot
Octopus
Openfind
OpenLinkProfiler
OutclicksBot
page scorer
PageScorer
Pandalytics
Panscient
PeoplePal
PictureFinder
Pimonster
POE-Component-Client-HTTP
ProPowerBot
Psbot
Quick-Crawler
RealDownload
Reaper
ReGet
Ripper
SBIder
scan.lol
ScoutJet
Scrapy
ScreenerBot
Semrush
SEOkicks
Seomoz
seoscanners
SeoSiteCheckup
SEOstats
serpstatbot
sexsearcher
Siphon
SISTRIX
Siteimprove
SiteLockSpider
SiteSnagger
Sitevigil
SlySearch
Snapbot
Snoopy
sogouspider
Sosospider
SpankBot
Spanner
sp_auditbot
Spbot
spyfu
Steeler
Stripper
Sucker
Sucuri
SuperBot
SuperHTTP
Surfbot
Suzuran
sysscan
Szukacz
T8Abot
tAkeOut
Teleport
Telesoft
Telesphoreo
The Intraformant
Thumbor
TightTwatBot
Toata
Toweyabot
Trendiction
Turnitin
TwengaBot
Typhoeus
Vagabondo
VB Project
VeriCiteCrawler
VidibleScraper
VoidEYE
Wallpapers/3.0
WallpapersHD
WASALive-Bot
WBSearchBot
WebAuto
Web Auto
WebBandit
Web Collage
WebCopier
Web Enhancer
WebFuck
WebGo IS
WebmasterWorldForumBot
webmeup-crawler
WebReaper
WebsiteExtractor
WebStripper
WebSucker
Web Sucker
WebZIP
WeSEE
Whatweb
WinHTTrack
Woobot
Wotbox
WWW-Collector-E
Xaldon WebSpider
Xenu
YoudaoBot
Zade
Zeus
ZmEu