404checker
Abonti
Aboundex
ADmantX
AhrefsBot
AiHitBot
Aipbot
Alligator
AllSubmitter
Apexoo
archive.org_bot
Asterias
Attach
autoemailspider
Backlink-Ceck
Battleztar Bazinga
BBBike
BDCbot
Blackboard
Black Hole
BLEXBot
Blow
BlowFish
Boardreader
Bolt
BotALot
Brandwatch
Buddy
Bullseye
BunnySlippers
CATExplorador
CazoodleBot
Cegbfeieh
CheeseBot
CheTeam
ChinaClaw
Chlooe
Claritybot
Cliqzbot
Cogentbot
cognitiveseo
Copyscape
crawler4j
crawler.feedback
CrunchBot
Custo
DatabaseDriverMysqli
DataCha0s
DBLBot
Demon
Digincore
DIIbot
Dispatch
DittoSpyder
DomainAppender
DomainCrawler
DomainSigmaCrawler
Drip
DSearch
DTS Agent
Ebingbong
eCatch
ECCP/1.0
EMail Siphon
EroCrawler
Evil
Exabot
ExtLinksBot
EyeNetIE
Ezooms
FemtosearchBot
FHscan
Firefox/7.0
FrontPage
GalaxyBot
Genieo
Getintent
Gigabot
G-i-g-a-b-o-t
Go!Zilla
Grabber
GrabNet
Grafula
GridBot
GT::WWW
Haansoft
HEADMasterSEO
Hloader
HTTP::Lite
Humanlinks
Iblog
IDBot
IlseBot
Image Fetch
Image Sucker
IndeedBot
InfoTekies
instabid
Internet Ninja
InternetSeer
ips-agent
Iria
Iskanie
Jetty
JOC Web Spider
Joomla
JustView
Jyxobot
Keyword Density
Lanshanbot
LeechFTP
LibWeb
Linkdexbot
LinkpadBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Spider
Ltx71
lwp-trivial
Magnet
Mag-Net
Majestic12
MarkMonitor
Masscan
meanpathbot
Metauri
Microsoft Data Access
MIIxpc
MJ12bot
Mojeek
Mr.4x3
MSFrontPage
Msrabot
Nameprotect
Navroad
NearSite
Nessus
NetLyzer
NetMechanic
Netvibes
NetZIP
NextGenSearchBot
Nibbler
Nikto
Nimbostratus
Ninja
Nmap
NPbot
oBot
Octopus
Openfind
OpenLinkProfiler
OpenVAS
OrangeSpider
OutclicksBot
PageAnalyzer
Page Analyzer
PageGrabber
page scorer
PageScorer
Pandalytics
PECL::HTTP
Picscout
Picsearch
Pi-Monster
Pixray
PleaseCrawl
Pockey
POE-Component-Client-HTTP
ProWebWalker
Pump
PxBroker
RankActiveLinkBot
RankingBot
RankingBot2
Rankivabot
RankurBot
Reaper
RedesScrapy
RocketCrawler
Rogerbot
SalesIntelligent
SBIder
Scanbot
SEOkicks-Robot
Seomoz
seoscanners
Shodan
Siphon
Siteimprove
SiteLockSpider
SiteSnagger
Sitevigil
SmartDownload
Snake
Snapbot
SocialRankIOBot
sogouspider
Sosospider
SpaceBison
Spammen
Spanner
Spinn3r
Suzuran
Szukacz
T0PHackTeam
tAkeOut
Telesphorep
TheNomad
Toweyabot
Trendiction
Trendictionbot
True_Robot
Turnitin
TwengaBot
Twice
VB Project
VeriCiteCrawler
VidibleScraper
Wallpapers/3.0
WBSearchBot
Webalta
WebCopier
WebEnhancer
WebFetch
WebFuck
WebLeacher
WebmasterWorldForumBot
webmeup-crawler
Web Sauger
Website Quester
WebSucker
Web Sucker
WebZIP
Whack
Whatweb
Who.is Bot
WinHTTrack
WiseGuys Robot
Wotbox
WWW-Mechanize
WWW::Mechanize
x09Mozilla
Xaldon_WebSpider
Xenu
xpymep1.exe
Zauba
Zermelo
zgrab
ZumBot