404checker
404enemy
Aboundexbot
ADmantX
AhrefsBot
Aipbot
Alligator
AllSubmitter
AlphaBot
ASPSeek
Asterias
Attach
AwarioRssBot
BackDoorBot
BacklinkCrawler
Bandit
BatchFTP
Battleztar Bazinga
BBBike
Bitacle
Blackboard
BLEXBot
Blow
Bolt
Brandprotect
Buddy
BuiltBotTough
Calculon
CATExplorador
Cegbfeieh
CherryPicker
coccocbot-web
Cogentbot
Collector
com.plumanalytics
Copier
Copyscape
crawler4j
crawler.feedback
CrazyWebCrawler
CrunchBot
Curious
demandbase-bot
Deusu
Digincore
DigitalPebble
DIIbot
DittoSpyder
DnyzBot
DomainAppender
DomainSigmaCrawler
Dotbot
Download Wonder
DSearch
EirGrabber
EMail Siphon
EroCrawler
ExtractorPro
Extreme Picture Finder
FDM
FemtosearchBot
Fimap
Flunky
Foobot
FrontPage
Fyrebot
GalaxyBot
Genieo
GetRight
Gigablast
Gotit
Go!Zilla
GrapeFX
GridBot
Haansoft
Harvest
Havij
Hloader
HTTrack
HybridBot
Iblog
InfoTekies
InterGET
Internet Ninja
internetVista monitor
ips-agent
Iskanie
IstellaBot
JamesBOT
JennyBot
Jorgee
JustView
Kenjin Spider
Keyword Density
Lanshanbot
Larbin
LexiBot
Lftp
Libwhisker
Lightspeedsystems
Likse
LinkextractorPro
LinkpadBot
LinksManager
LinkWalker
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey
Lipperhey Spider
LNSpiderguy
lwp-trivial
Magnet
Mail.RU_Bot
Majestic12
MarkMonitor
Mass Downloader
Mata Hari
Mediatoolkitbot
MFC_Tear_Sample
Microsoft Data Access
Mister PiX
MJ12bot
Mojeek
Morfeus Fucking Scanner
Mr.4x3
muhstik-scan
Name Intelligence
Navroad
NearSite
Needle
netEstate NE Crawler
Nibbler
NimbleCrawler
Nimbostratus
NPbot
Nutch
Openfind
OpenLinkProfiler
Openvas
OpenVAS
OrangeBot
OrangeSpider
Panscient
Picscout
Picsearch
PictureFinder
Pi-Monster
Pixray
PleaseCrawl
plumanalytics
Probethenet
Psbot
PyCurl
RankActive
RankActiveLinkBot
RankFlex
RankingBot
RankurBot
Reaper
RebelMouse
Ripper
s1z.ru
SBIder
ScanAlert
scan.lol
Scrapy
Screaming
ScreenerBot
Searchestate
SearchmetricsBot
Semrush
SemrushBot
SEOkicks
SEOlyticsCrawler
Seomoz
SEOstats
serpstatbot
sexsearcher
SISTRIX
Sitebeam
Siteimprove
Site Sucker
SlySearch
SmartDownload
SocialRankIOBot
Sociscraper
Sogou web spider
Sosospider
SpaceBison
Spammen
spyfu
Steeler
Stripper
Sucker
SuperHTTP
Surfbot
Suzuran
Swiftbot
T8Abot
TeleportPro
Telesoft
Telesphoreo
The Intraformant
Thumbor
Titan
Tracemyfile
Trendiction
trendiction.de
Turnitin
TurnitinBot
TwengaBot
Twice
Typhoeus
Vacuum
Vagabondo
Virusdie
Voltron
Wallpapers/3.0
Webalta
Web Auto
WebBandit
Web Collage
WebCopier
WEBDAV
WebEnhancer
WebmasterWorldForumBot
webmeup-crawler
WebsiteExtractor
WebStripper
WebSucker
Web Sucker
WeSEE
Whack
Who.is Bot
Widow
Woobot
Wotbox
Wprecon
WPScan
WWW-Collector-E
WWW-Mechanize
x09Mozilla
x22Mozilla
Xaldon_WebSpider
Xenu
xpymep1.exe
Zade
Zauba
zgrab
ZumBot