nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-11-01 22:08:27 +00:00

250 lines
2.6 KiB
Text

01h4x.com
80legs
ALittle Client
ASPSeek
Aboundexbot
Acunetix
AhrefsBot
AiHitBot
Anarchy
Ankit
Anthill
Attach
BBBike
BDFetch
BackWeb
BacklinkCrawler
Badass
Black Hole
Blow
BlowFish
Buck
Buddy
Bullseye
BunnySlippers
Calculon
CazoodleBot
Cegbfeieh
CheTeam
CherryPicker
Chlooe
Cliqzbot
Cloud mapping
Cocolyzebot
Collector
Craftbot
Crawling at Home Project
CrunchBot
Curious
CyotekWebCopy
DIIbot
DataCha0s
Digincore
Dispatch
DittoSpyder
DnBCrawler-Analytics
DomainAppender
DomainCrawler
Download Wonder
EMail Siphon
EMail Wolf
EasyDL
Ecxi
EirGrabber
Express WebPictures
ExtLinksBot
ExtractorPro
Ezooms
FHscan
FemtosearchBot
FlashGet
Flunky
Foobot
Fyrebot
Gigabot
Go!Zilla
GrabNet
Grafula
HEADMasterSEO
HTTrack
Havij
IRLbot
Iblog
Id-search
IlseBot
Image Fetch
Image Sucker
IndeedBot
Iskanie
IstellaBot
JOC Web Spider
JamesBOT
JennyBot
JetCar
Joomla
Jyxobot
Kinza
Larbin
LeechGet
Lftp
Lightspeedsystems
LinkScan
Linkdexbot
LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
MFC_Tear_Sample
MIDown tool
MSIECrawler
Mag-Net
Majestic SEO
Majestic12
MarkWatch
MeanPath Bot
Mediatoolkitbot
Microsoft Data Access
Microsoft URL Control
Mojeek
Mojolicious
Musobot
NPbot
Net Vampire
NetMechanic
Nettrack
Netvibes
NextGenSearchBot
NimbleCrawler
Nimbostratus
Ninja
Octopus
Offline Explorer
OpenVAS
OrangeBot
PageGrabber
Panscient
Papa Foto
Pavuk
Petalbot
PictureFinder
Piepmatz
ProWebWalker
Probethenet
Pu_iN
PxBroker
PyCurl
RSSingBot
RankActive
RankActiveLinkBot
RankingBot
RankingBot2
ReGet
RealDownload
Reaper
Recorder
RedesScrapy
Ripper
RocketCrawler
Rogerbot
SEOprofiler
SEOstats
SISTRIX
ScanAlert
Screaming
ScrepyBot
Searchestate
Semrush
Seomoz
Shodan
Site Sucker
SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger
Snake
Sociscraper
Sogou web spider
Sottopop
SpaceBison
Spanner
Sqlworm
Steeler
Sucker
SuperBot
SurveyBot
Szukacz
Teleport
Telesoft
Telesphoreo
Thumbor
Tracemyfile
Trendictionbot
True_Robot
Turingos
TwengaBot
Twice
URLy Warning
URLy.Warning
VB Project
Vacuum
Vagabondo
VidibleScraper
Voil
WASALive-Bot
WEBDAV
WPScan
WWW-Collector-E
WallpapersHD
WeSEE
Web Auto
Web Enhancer
Web Sauger
WebBandit
WebCollage
WebCopier
WebFuck
WebImageCollector
WebLeacher
WebReaper
WebSauger
WebStripper
Webalta
WebmasterWorldForumBot
Webshag
WebsiteExtractor
Whack
Whatweb
Woobot
Wotbox
Wprecon
Xaldon_WebSpider
Zade
Zauba
Zermelo
Zitebot
ZoomBot
ZyBorg
archive.org_bot
arquivo-web-crawler
backlink-check
check1.exe
clark-crawler
facebookscraper
heritrix
instabid
internetVista monitor
isitwp.com
lwp-trivial
mediawords
pcBrowser
ripz
satoristudio.net
seoscanners
serpstatbot
sexsearcher
sitechecker.pro
siteripz
sp_auditbot
trendiction.de
ubermetrics-technologies.com
voyagerx.com
x09Mozilla
x22Mozilla