nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2025-06-14 10:14:31 +00:00

100 lines
1.1 KiB
Text

01h4x.com
360Spider
404enemy
ASPSeek
AdsTxtCrawlerTP
AfD-Verbotsverfahren
AhrefsBot
Apexoo
AwarioRssBot
BDFetch
BackDoorBot
BlackWidow
Brandwatch
Bullseye
Claritybot
Crawling at Home Project
CrazyWebCrawler
Demon
Deusu
Disco
DomainAppender
DomainSigmaCrawler
DomainStatsBot
Download Wonder
Genieo
Go-Ahead-Got-It
GrabNet
GrapeshotCrawler
Hloader
Image Fetch
Indy Library
InternetMeasurement
InternetSeer
Iria
JOC Web Spider
Jbrofuzz
Jetty
Jorgee
LinkextractorPro
LinksManager
Litemage_walker
Ltx71
MIDown tool
Mail.RU_Bot
MicroMessenger
Mister PiX
NextGenSearchBot
Nuclei
Offline Explorer
OpenLinkProfiler
OpenVAS
OrangeSpider
PHPCrawl
Panscient
Pavuk
PxBroker
QueryN Metasearch
RSSingBot
Rankivabot
RankurBot
Recorder
SEOkicks
SEOkicks-Robot
ScrepyBot
SemrushBot-OCOB
SeoCherryBot
SeobilityBot
Seomoz
Shodan
Sitebeam
Sosospider
Spammen
Spanner
SputnikBot
Surfbot
Telesphoreo
The Intraformant
TinyTestBot
Titan
Twice
Typhoeus
UnisterBot
VCI
WBSearchBot
WWW::Mechanize
WeSEE
WebEnhancer
WinHTTrack
Woobot
Xaldon_WebSpider
arquivo-web-crawler
check1.exe
clark-crawler
coccocbot
crawler4j
heritrix
instabid
seobility
trendiction.de
webpros.com