nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2023-03-30 22:05:08 +00:00

100 lines
1 KiB
Text

Aboundexbot
AhrefsBot
Anarchie
AwarioSmartBot
BDFetch
BotALot
BuzzSumo
Cegbfeieh
CherryPicker
Cliqzbot
Copier
Copyscape
Crawling at Home Project
Curious
CyotekWebCopy
DTS Agent
Demon
Deusu
Digincore
DigitalPebble
Disco
Discoverybot
DnyzBot
DomainStatsBot
Exabot
ExtLinksBot
FlashGet
GT::WWW
Gigabot
Go-Ahead-Got-It
GoZilla
GridBot
HTTP::Lite
InterGET
JOC Web Spider
Jetty
JikeSpider
LWP::Simple
Leap
LeechFTP
LinkScan
LinkextractorPro
LinqiaMetadataDownloaderBot
MIIxpc
MJ12bot
MTRobot
Mass Downloader
Mata Hari
MegaIndex.ru
Metauri
MicroMessenger
NimbleCrawler
Offline Explorer
OrangeSpider
OutclicksBot
OutfoxBot
Pavuk
PeoplePal
Petalbot
Pixray
PleaseCrawl
PxBroker
RankingBot2
Re-re
ReGet
RebelMouse
SBIder
Scanbot
Seomoz
Siteimprove
Sottopop
SpankBot
Stripper
SuperBot
SuperHTTP
Swiftbot
Szukacz
T0PHackTeam
T8Abot
Telesphorep
Toweyabot
Twice
Upflow
Vacuum
Voltron
WISENutbot
Web Auto
Web Fuck
Web Pix
WebWhacker
Webster
YoudaoBot
ZyBorg
archive.org_bot
arquivo-web-crawler
gopher
mediawords
page scorer
polaris version
s1z.ru