nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-11-28 10:09:01 +00:00

250 lines
2.6 KiB
Text

01h4x.com
360Spider
80legs
AIBOT
ALittle Client
ASPSeek
Abonti
Aboundexbot
Acunetix
AhrefsBot
AiHitBot
Aipbot
Alligator
Anarchy
Anarchy99
Apexoo
AwarioSmartBot
BBBike
BDCbot
BDFetch
BLEXBot
Backlink-Ceck
BetaBot
Bigfoot
BlackWidow
Blackboard
BlowFish
Brandprotect
Brandwatch
Buddy
Bullseye
BunnySlippers
CATExplorador
CazoodleBot
CherryPicker
Chlooe
Citoid
Cocolyzebot
Collector
Cosmos
Craftbot
Crawling at Home Project
CrazyWebCrawler
Crescent
Curious
DBLBot
DTS Agent
Digincore
Disco
Discobot
DittoSpyder
DnyzBot
DomainAppender
DomainCrawler
Dotbot
Dragonfly
Drip
EMail Wolf
EroCrawler
ExtLinksBot
Extractor
EyeNetIE
FemtosearchBot
Firefox/7.0
FlashGet
Flunky
FrontPage
GalaxyBot
GermCrawler
Getintent
Go!Zilla
GrabNet
Grabber
GrapeFX
GrapeshotCrawler
HMView
HTMLparser
HTTP::Lite
HTTrack
Humanlinks
Id-search
Image Fetch
IndeedBot
Indy Library
InfoNaviRobot
Iskanie
Jetty
JikeSpider
Joomla
JustView
Jyxobot
Keyword Density
Kinza
LeechGet
LieBaoFast
Lightspeedsystems
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Spider
Litemage_walker
MJ12bot
MSFrontPage
MTRobot
Magnet
MarkMonitor
Mass Downloader
Masscan
MauiBot
MegaIndex.ru
Metauri
Microsoft Data Access
Microsoft URL Control
Moblie Safari
Mojeek
MolokaiBot
Msrabot
NICErsPRO
Name Intelligence
Navroad
NearSite
Nessus
NetAnts
NetSpider
NetZIP
NimbleCrawler
Ninja
Nmap
Not
Nuclei
Offline Navigator
OnCrawl
OpenLinkProfiler
OpenVAS
OutclicksBot
OutfoxBot
PageScorer
PageThing.com
Papa Foto
Pavuk
Pockey
ProPowerBot
ProWebWalker
Psbot
Pu_iN
Pump
PxBroker
QueryN Metasearch
RSSingBot
RankActive
RankingBot2
ReGet
RedesScrapy
Ripper
RocketCrawler
SEOkicks
SEOlyticsCrawler
SMTBot
SalesIntelligent
ScanAlert
Scanbot
Scrapy
ScreenerBot
Searchestate
SentiBot
Site Sucker
SiteCheckerBotCrawler
SiteLockSpider
Sitebeam
Siteimprove
SlySearch
SmartDownload
Snake
Snapbot
Sociscraper
SpaceBison
SpankBot
Spbot
SputnikBot
SuperHTTP
T0PHackTeam
Teleport
Telesphoreo
The Intraformant
TheNomad
Thumbor
TightTwatBot
Tracemyfile
Trendiction
Trendictionbot
Turnitin
TurnitinBot
TwengaBot
Twice
Typhoeus
URLy Warning
URLy.Warning
Upflow
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
Voil
WASALive-Bot
WBSearchBot
WEBDAV
WWW-Collector-E
WWW-Mechanize
Wallpapers
Web Auto
Web Enhancer
Web Fuck
WebEnhancer
WebGo IS
WebLeacher
WebsiteQuester
Webster
Whatweb
Wotbox
Xaldon WebSpider
Zauba
Zeus
ZoominfoBot
adscanner
autoemailspider
cah.io.community
coccocbot
cognitiveseo
crawler.feedback
crawler4j
evc-batch
facebookscraper
gopher
instabid
ips-agent
iubenda-radar
lwp-trivial
magpie-crawler
ripz
scan.lol
seobility
seocompany.store
seostar
serpstatbot
siteripz
sogouspider
spyfu
trendiction.de
webgains-bot
webpros.com
zauba.io