nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-08-30 10:07:14 +00:00

100 lines
1.1 KiB
Text

AIBOT
Abonti
AdsTxtCrawlerTP
Aipbot
Alexibot
Anarchie
BetaBot
Brandprotect
Buddy
BuiltBotTough
Bytespider
Claritybot
Cosmos
Curious
CyotekWebCopy
DataCha0s
Deusu
Devil
Digincore
DigitalPebble
Dispatch
Dragonfly
ExtLinksBot
Extractor
FemtosearchBot
FlashGet
Fyrebot
Grabber
GrapeshotCrawler
HEADMasterSEO
IRLbot
InfoTekies
InternetMeasurement
JikeSpider
Joomla
MTRobot
Masscan
MauiBot
Mb2345Browser
MegaIndex.ru
Microsoft URL Control
Morfeus Fucking Scanner
Mozlila
Needle
NetMechanic
NetSpider
NetZIP
Netcraft
Octopus
OpenLinkProfiler
PageGrabber
Papa Foto
Picscout
Pixray
QueryN Metasearch
Rankivabot
RankurBot
RepoMonkey
RocketCrawler
SEOlyticsCrawler
ScoutJet
Searchestate
SeekportBot
SeoSiteCheckup
SpaceBison
Spammen
SpankBot
Sqworm
Sucker
Sucuri
Telesoft
The Intraformant
Titan
VelenPublicWebCrawler
WEBDAV
WallpapersHD
WebAuto
WebFetch
WebGo IS
WebImageCollector
WebStripper
Website Quester
Wprecon
Xaldon_WebSpider
YaK
YoudaoBot
autoemailspider
backlink-check
check1.exe
coccocbot
crawl.sogou.com
linkdexbot
lwp-request
lwp-trivial
magpie-crawler
page scorer
plumanalytics
sitechecker.pro
siteripz
sysscan