vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3494 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-26 22:06:56 +00:00
parent c55d175c5f
commit 5742b6bb24
47 changed files with 90907 additions and 90910 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3493 #### Version: V4.2022.11.3494
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3493 #### Version: V4.2022.11.3494
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3493 #### Version: V4.2022.11.3494
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11224
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3493 #### Version: V4.2022.11.3494
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11228
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11228
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11228
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11228
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 1; 10.1.1.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 0; 10.1.1.217 0;
127.0.0.1 0; 127.0.0.1 0;

11228
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 1; 10.1.1.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 0; 10.1.1.217 0;
127.0.0.1 0; 127.0.0.1 0;

11228
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 1; 10.1.1.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 0; 10.1.1.217 0;
127.0.0.1 0; 127.0.0.1 0;

11228
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3493 3494

11228
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11228
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.236.131.43 0; 20.119.221.222 0;

11228
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 1; 10.1.1.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 0; 10.1.1.217 0;
127.0.0.1 0; 127.0.0.1 0;

11228
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 1; 10.1.1.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 0; 10.1.1.217 0;
127.0.0.1 0; 127.0.0.1 0;

11228
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 1; 10.1.1.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 0; 10.1.1.217 0;
127.0.0.1 0; 127.0.0.1 0;

11228
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.236.131.43 0; 20.119.221.222 0;

11228
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.236.131.43 0; 20.119.221.222 0;

11228
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 1; 10.1.1.217 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anarchy CSHttp
Cliqzbot Msrabot
CyotekWebCopy Petalbot
RankActiveLinkBot RepoMonkey
Ripper Ripper
Steeler Toweyabot
Webalta WebPix
cognitiveseo Webster
lwp-trivial arquivo.pt
sexsearcher facebookscraper

314
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
80legs 80legs
ADmantX
AIBOT
ALittle Client ALittle Client
ASPSeek
Abonti Abonti
Aboundex Aboundex
Aboundexbot AhrefsBot
Acunetix
AfD-Verbotsverfahren
Alexibot
AllSubmitter AllSubmitter
Alligator
Anarchy
Anarchy99 Anarchy99
Ankit Ankit
Anthill
Apexoo Apexoo
Asterias Asterias
Backlink-Ceck Atomseobot
BBBike
BDFetch
BackDoorBot
BackStreet
BackWeb
Badass Badass
Bandit
BatchFTP
BetaBot BetaBot
Bitacle Bitacle
Black Hole BlackWidow
Blackboard Blow
Bolt Bolt
BotALot Buck
Bullseye BuiltBotTough
CODE87 BuiltWith
CSHttp CATExplorador
Calculon CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CheTeam CheeseBot
Claritybot CherryPicker
Cliqzbot ChinaClaw
Cosmos Chlooe
Craftbot Citoid
Custo Cloud mapping
Cocolyzebot
Cogentbot
Copier
CopyRightCheck
Copyscape
CrazyWebCrawler
Curious
DBLBot DBLBot
DataCha0s
Devil Devil
DittoSpyder Dirbuster
DomainStatsBot Discobot
Dotbot Discoverybot
ECCP/1.0 DnyzBot
EMail Wolf DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
Drip
EasyDL EasyDL
Ecxi EirGrabber
Exabot Exabot
Express WebPictures Ezooms
ExtLinksBot Fimap
Extractor FlashGet
Extreme Picture Finder Foobot
Firefox/7.0 FrontPage
Flunky Fuzz
FyberSpider
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
GermCrawler GermCrawler
GetRight
GetWeb
Go!Zilla
Gotit
GrabNet GrabNet
Grabber
Grafula Grafula
GrapeshotCrawler
GridBot GridBot
HEADMasterSEO
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Haansoft
Havij Havij
Hloader HybridBot
Humanlinks
IDBot IDBot
IndeedBot Iblog
Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies Intelliseek
InterGET
Internet Ninja Internet Ninja
Iria JetCar
Iskanie Jetty
Joomla
JustView JustView
Jyxobot Jyxobot
Keyword Density Kenjin Spider
Kinza Keybot Translation-Search-Machine
LWP::Simple LNSpiderguy
Larbin Larbin
LibWeb Lftp
Likse Libwhisker
LieBaoFast
Lightspeedsystems
LinkWalker LinkWalker
Linkbot
LinkpadBot
LinqiaRSSBot
Lipperhey
Litemage_walker Litemage_walker
Lmspider
Ltx71 Ltx71
Masscan MIIxpc
MQQBrowser
MSFrontPage
Mag-Net
Mail.RU_Bot
Majestic-SEO
Majestic12
Mass Downloader
Mata Hari Mata Hari
MauiBot MauiBot
Mb2345Browser
Meanpathbot Meanpathbot
MegaIndex.ru MegaIndex.ru
MicroMessenger Microsoft Data Access
Microsoft URL Control Mister PiX
Mojeek Morfeus Fucking Scanner
Mojolicious Mozlila
MolokaiBot Msrabot
NICErsPRO Navroad
NPbot Nessus
Net Vampire Net Vampire
NetAnts
NetSpider
Netcraft Netcraft
Netvibes Netvibes
NextGenSearchBot Nikto
Nibbler Nimbostratus
NimbleCrawler
Ninja Ninja
Nmap Nuclei
Not
Offline Navigator Offline Navigator
OpenLinkProfiler OnCrawl
OpenVAS OpenVAS
Openfind
OrangeSpider
OutfoxBot OutfoxBot
PHPCrawl PECL::HTTP
POE-Component-Client-HTTP
PageThing.com
Papa Foto
Pavuk
PeoplePal PeoplePal
Picscout Picscout
Picsearch Picsearch
PictureFinder
Pimonster
Pixray Pixray
ProPowerBot PleaseCrawl
Psbot Pockey
Pump
PxBroker PxBroker
RSSingBot PyCurl
RankActive Quick-Crawler
RankActiveLinkBot
RankingBot RankingBot
Rankivabot Rankivabot
Reaper RankurBot
RedesScrapy RealDownload
RebelMouse
RepoMonkey RepoMonkey
Ripper
RocketCrawler RocketCrawler
Rogerbot SEOkicks
SBIder
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SISTRIX SISTRIX
SMTBot ScanAlert
Scrapy Scrapy
Screaming ScreenerBot
SemrushBot ScrepyBot
Shodan SearchmetricsBot
Site Sucker SemanticJuice
SiteSucker SentiBot
Siteimprove SeoSiteCheckup
SiteCheckerBotCrawler
SiteExplorer
SiteSnagger
SlySearch SlySearch
Snoopy
SocialRankIOBot SocialRankIOBot
Sosospider Sosospider
Spammen Sottopop
Spbot
Spinn3r
SputnikBot
Sqlmap
Steeler Steeler
Stripper Stripper
Sucker
Sucuri Sucuri
SuperBot SuperBot
T0PHackTeam Surfbot
T8Abot SurveyBot
Szukacz
Teleport Teleport
Telesoft Telesoft
Telesphorep Telesphorep
The Intraformant TheNomad
Toweyabot Titan
Tracemyfile Tracemyfile
Trendictionbot Trendictionbot
Turingos Turingos
Twice Turnitin
TwengaBot
Typhoeus Typhoeus
URLy Warning URLy.Warning
UnisterBot
Upflow Upflow
V-BOT
VB Project
VCI
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie VidibleScraper
VoidEYE WWW::Mechanize
Voil WWWOFFLE
Voltron Wallpapers
WASALive-Bot
WPScan
WallpapersHD WallpapersHD
WeSEE
Web Collage
Web Fetch
Web Fuck
Web Sucker Web Sucker
WebEnhancer WebCollage
WebCopier
WebFuck WebFuck
WebGo IS WebImageCollector
WebLeacher
WebPix
WebSauger WebSauger
WebStripper WebWhacker
WebZIP WebmasterWorldForumBot
Webalta Website Quester
Whatweb WebsiteQuester
YoudaoBot Whack
Who.is Bot
WiseGuys Robot
Wotbox
Zade
Zauba Zauba
Zeus
Zitebot Zitebot
ZoominfoBot ZmEu
ZoomBot
ZumBot
ZyBorg ZyBorg
archive.org_bot
arquivo.pt arquivo.pt
autoemailspider cah.io.community
backlink-check
clark-crawler
coccocbot coccocbot
cognitiveseo
crawl.sogou.com
crawler4j crawler4j
dataforseo.com dataforseo.com
demandbase-bot
domainsproject.org domainsproject.org
evc-batch
heritrix heritrix
linkdexbot instabid
lwp-trivial ips-agent
iubenda-radar
lwp-request
magpie-crawler magpie-crawler
meanpathbot meanpathbot
netEstate NE Crawler mediawords
oBot page scorer
pcBrowser polaris version
scan.lol seocompany.store
seoscanners
serpstatbot
sitechecker.pro
siteripz siteripz
sysscan sp_auditbot
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com trendiction.de
webmeup-crawler
webpros.com webpros.com
zgrab webprosbot
x09Mozilla
xpymep1.exe
zauba.io

288
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,103 +1,97 @@
Aboundex 01h4x.com
Aboundexbot 80legs
AhrefsBot ADmantX
Alligator AIBOT
Aspiegel ASPSeek
Attach Acunetix
AlphaBot
Ankit
Anthill
Apexoo
BDCbot BDCbot
BDFetch BackDoorBot
BackWeb BackStreet
Backlink-Ceck BatchFTP
Bandit
Battleztar Bazinga
BetaBot
Bigfoot
Bitacle Bitacle
BlackWidow Black Hole
Blackboard Blackboard
Blow Blow
Boardreader
Bolt Bolt
Brandprotect BotALot
Brandwatch Brandwatch
BuiltBotTough BuiltBotTough
BuiltWith
Bullseye Bullseye
BunnySlippers BunnySlippers
CATExplorador CCBot
CODE87
CSHttp
CheeseBot
CherryPicker
Citoid Citoid
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot
CopyRightCheck
Copyscape Copyscape
Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
CrunchBot
DBLBot
DIIbot DIIbot
DSearch
DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Demon
Deusu Devil
Digincore
Dirbuster
Disco Disco
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics
DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainStatsBot
Download Wonder
Dragonfly
Drip Drip
EMail Siphon ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
Ebingbong Ebingbong
Ecxi EirGrabber
ExtLinksBot Exabot
ExtractorPro ExtractorPro
EyeNetIE Extreme Picture Finder
FHscan
Fimap Fimap
Freeuploader Freeuploader
Fuzz FyberSpider
Fyrebot Fyrebot
Genieo G-i-g-a-b-o-t
GetRight
GetWeb
Gigabot
Go!Zilla Go!Zilla
Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HTMLparser
HTTP::Lite
HTTrack
Haansoft
HaosouSpider
Havij
Hloader Hloader
Humanlinks HonoluluBot
HybridBot
IDBTE4M
Iblog Iblog
Image Fetch Image Fetch
InfoNaviRobot IndeedBot
InfoTekies Indy Library
Intelliseek Intelliseek
InterGET InternetSeer
Iria
IstellaBot IstellaBot
JOC Web Spider
Jbrofuzz Jbrofuzz
JennyBot
JetCar
JikeSpider JikeSpider
Joomla Joomla
Kinza JustView
LNSpiderguy Kenjin Spider
LWP::Simple Keybot Translation-Search-Machine
LeechFTP Keyword Density
Likse Leap
LeechGet
LinkWalker LinkWalker
LinqiaMetadataDownloaderBot LinksManager
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lmspider
MFC_Tear_Sample
MIIxpc MIIxpc
MSFrontPage MSFrontPage
MSIECrawler MSIECrawler
@ -105,146 +99,152 @@ MTRobot
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic-SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mb2345Browser
Masscan Microsoft Data Access
Mata Hari Microsoft URL Control
MegaIndex.ru Mojolicious
Metauri
Mojeek
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Musobot
Nameprotect
Needle Needle
Nessus
Net Vampire
NetAnts NetAnts
NetMechanic
NetSpider NetSpider
NetZIP NetZIP
Netcraft Netcraft
Nettrack Nettrack
Niki-bot NextGenSearchBot
Nimbostratus Nikto
Ninja
Nmap Nmap
Octopus Openvas
Offline Explorer
OpenVAS
OrangeSpider
OutfoxBot
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber
PageThing.com PageThing.com
Pandalytics
Panscient Panscient
Pavuk
Petalbot
Pi-Monster Pi-Monster
Picscout
Pixray
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot
ProWebWalker
Probethenet
Pu_iN Pu_iN
Quick-Crawler PxBroker
RankFlex RankFlex
RankingBot
Rankivabot
RankurBot RankurBot
Re-re Re-re
ReGet
RealDownload
Reaper Reaper
RebelMouse RebelMouse
Recorder RedesScrapy
RepoMonkey
RocketCrawler
Rogerbot Rogerbot
SEOprofiler SBIder
Scanbot SEOkicks
ScrepyBot SEOstats
SearchmetricsBot SMTBot
ScoutJet
ScreenerBot
Searchestate
Seekport Seekport
SemanticJuice
SemrushBot SemrushBot
SeoSiteCheckup
SeobilityBot
Seomoz
Shodan
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider Sitebeam
SiteSucker Siteimprove
Sitevigil Snapbot
Snoopy Snoopy
Sogou web spider
Sosospider
Spanner Spanner
Spbot SputnikBot
Steeler Sqlmap
Sqworm
Stripper
Sucuri Sucuri
SuperBot Surfbot
SurveyBot SurveyBot
Suzuran
Swiftbot Swiftbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
TeleportPro
Telesoft Telesoft
Telesphoreo The Intraformant
Telesphorep Titan
Thumbor Toata
Trendiction Toweyabot
True_Robot True_Robot
Turingos TurnitinBot
Turnitin Upflow
TwengaBot
V-BOT
VB Project VB Project
VCI Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
WWW::Mechanize VidibleScraper
Virusdie
VoidEYE
WISENutbot
WPScan
WWW-Mechanize
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
WeSEE WeSEE
Web Fetch Web Pix
Web Fuck Web Sauger
WebBandit WebAuto
WebCollage WebCollage
WebEnhancer WebGo IS
WebFetch WebLeacher
WebImageCollector WebPix
WebReaper
WebSauger
WebStripper
WebSucker WebSucker
WebWhacker
Webshag Webshag
Website Quester Website Quester
WebsiteExtractor WebsiteExtractor
Whacker Webster
Whatweb
Who.is Bot
Widow
WiseGuys Robot WiseGuys Robot
Wonderbot
Woobot Woobot
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zauba ZumBot
Zeus adscanner
Zitebot
ZmEu
ZoomBot
ZoominfoBot
archive.org_bot archive.org_bot
arquivo-web-crawler autoemailspider
arquivo.pt backlink-check
check1.exe cah.io.community
cognitiveseo coccocbot
crawl.sogou.com crawler4j
domainsproject.org
eCatch eCatch
heritrix facebookscraper
internetVista monitor instabid
ips-agent
isitwp.com isitwp.com
iubenda-radar iubenda-radar
linkdexbot
lwp-request lwp-request
magpie-crawler mediawords
meanpathbot muhstik-scan
polaris version netEstate NE Crawler
probe-image-size oBot
satoristudio.net page scorer
scalaj-http seostar
serpstatbot
siteripz siteripz
tAkeOut
trendiction.com trendiction.com
webgains-bot voyagerx.com
webmeup-crawler webprosbot
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
202ch.com bugof.gq
999webdesign.com cheapbelstaff.online
cjs.com.ru den-noch24.ru
ff30236ddef1465f88547e760973d70a.com freakycheats.com
homeinns.com keywordglobal.co.za
lyngdalhudterapi.no moinozhki.com
nmrk.ru optibuymac.com
podshipniki-ntn.ru pons-presse.com
sextracker.de privhosting.com
sk.golden-praga.ru yoursearch.me

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0912701309f8ce.com
12bet.com 12bet.com
1hwy.com 1stat.ru
2000k.ru 1webmaster.ml
202ch.com 2ads.co.uk
24-ak.ru 2rich4bitches.com
4kplayer.pl 3th.co.in
5000-cotydzien.com 76brighton.co.uk
5u.com 7search.com
72-news.com 7yue.org
8558l.jobs.net 83net.jp
DomainStatsBot 8xv8.com
abcdefh.xyz 98oi.ru
absugars.com 991mostfm.co.id
acmebtn.ml acortarurl.es
activepr.ru adanih.com
adamoads.com adloads.net
adcash.com admanaerofoil.com
adexprts.com ads-cool.pro
adnotbad.com adtology.com
advancedsoftwaresupport.com advokateg.xyz
agapovdg.ru alive-ua.com
ahmedabadwebs.com
aibolita.com
alborzan.com
alekseevec.ru
alf-img.com
all4invest.info all4invest.info
alloysteel.ru alphahoverboards.com
amazingpic.net alpharma.net
am-se.com
an-donut.com
anal-acrobats.hol.es anal-acrobats.hol.es
ananumous.ru anapa-inns.ru
apccargo.com anaseracresar.tk
apibatbrowsecom-a.akamaihd.net android4fun.org
appleid-verification.com apartmentratings.com
approved.su apibetweenlinesn-a.akamaihd.net
arabsexxxtube.com app-ready.xyz
arenanews.com.ua aquarium-pro.ru
ascat.porn asmxsatadriverin.aircus.com
asdfz.pro asrvvv-a.akamaihd.net
asrvrep-a.akamaihd.net aucoinhomes.com
ausmepa.org.au auspolice.com
auto4style.ru autowebmarket.com.ua
autobrennero.it aviav.org
azbukafree.com avon-severozapad.ru
baladur.ru avon-ukraine.com
balla-boo.se azarttoto.com
bambi.ck.ua babyguns.ru
bank.uz beachfix.co
bankiem.pl belinka.com.ua
bdsmgalls.net
belinvestdom.by
best-coupon-offer.com best-coupon-offer.com
bestfortraders.com bestwaystogetpaid.us
betterscooter.com
bibys.com bibys.com
bikini-image.com bif-ru.info
blackplanet.com biglistofwebsites.com
blagovest-med.ru bigpenisguide.com
blockworld.ru bizfly.info
blog.koreadaily.com blogrankers.com
bluejays-jerseys.us boeuklad.com
bluesman.nu bonux.nextview.ru
bmw.afora.ru bookmaker-bet.com
bolitgorlo.net bosefux.esy.es
bristolhotel.com bouda.kvalitne.cz
burger-imperia.com brainboostingsupplements.org
burger-tycoon.com brains2.biz
businesxxl.com brateg.xyz
buyessaynow.biz breastaugmentation.co.za
calstaterealty.us bugof.gq
calvet.altervista.org buyantiviralwp.com
cand.jp buypillsorderonline.com
candy-glam-hp.com buytizanidineonline.blogspot.com
cartujano-pre.de cakemediahq.com.au
cashonads.com candypeople.se
casinopinup-wins.com cartierbraceletsreplica.pw
casinorewards.info cartierjusteunclou.xyz
cb.iphantom.com cb.iphantom.com
cdnanalytics.xyz celejihad.info
cheapmoncler.win checkm8.com
checkhit.com chinajerseyswholesalecoupons.com
chlooe.com cityadspix.com
chuckguilford.com cleanpcnow.com
club-samodelkin.ru clickbanksites.info
coin-hive.com clicksotrk.com
colobit.biz cobaltpro.ru
com-onlinesupport.host coinsspb.com
compliance-nelson.top com-supportcenter.website
compliance-olga.top compliance-alexa.top
compliance-walt.top compliance-elena.top
coolwallpapers4k.info compliance-jessica.top
copypaste.traderzplanet.in compliance-viktor.top
country-chic.ru compucelunlock.net
covadhosting.biz continent-e.tv
credit.co.ua coolgramgoods.com
crest3d.ru coolwallpapers-hd.com
crirussian.ru craftinsta.ru
cryptoswap.biz creditmoney.com.ua
culturevie.info crystalslot.com
cscwtalkto.site
culmimg.pw
cupday.com cupday.com
cutalltheshit.com customcollegeessays.net
cxpromote.com d0t.ru
czeshop.info daneshetabiat.com
decorationspcs.com data-ox.com
den-noch24.ru db.speedup.gdn
dentalpearls.com.au ddpills.com
detskie-konstruktory.ru detskie-konstruktory.ru
dev.citetick.com detsky-nabytek.info
diamond-necklace.info diffbot.com
directivepub.com digital-video-processing.com
discovertreasurenow.com digitalfaq.com
dneprsvet.com.ua distver.ru
documentsite.net diusyawiga.tk
dosug-lux.ru divisioncore.com
djekxa.ru
dominateforex.ml
donvito.unas.cz
dotnetdotcom.org dotnetdotcom.org
drugstoreforyou.com doublepimp.com
dtnlyss.com dptaughtme.com
dverimegapolis.ru drugspowerstore.com
dvr.biz.ua e-poker-2005.com
e-biznes.info eager-nash.188-93-233-196.plesk.page
e90post.com ebooknovel.club
ed-shop01.ru elektrischezi.canalblog.com
ekspresihati.info elektrischezigaretten2.beeplog.com
eleimgo.pw elektronischezigarettekaufen1.beeplog.com
elektrischeziga.livejournal.com elektronischezigarettekaufen2.tumblr.com
elite-sex-finders.com elektrozigarettekaufen2.blogse.nl
elkacentr.ru elitesportsadvisor.com
eloxal.ru emediate.eu
eluxer.net energy-ua.com
empirepoker.com englate.com
englishgamer.com
enthuse.computernetworksonline.com enthuse.computernetworksonline.com
epngo.bz erotik0049.com
estelight.ru esfchat.tk
eugenevaultstorage.com essayservicewriting.org
europages.com.ru estibot.com
eurosamodelki.ru
event-tracking.com
expolicenciaslatam.co
extlabs.io extlabs.io
extrabot.com eywords-monitoring-your-success.com
extremepornos.net failingmarriege.blogspot.com
eyessurgery.ru familienzahnaerzte.com
ezigaretteshop2.mywapblog.com farmingworm.com
faithe.top favornews.com
faptitans.com fealq.com
farm26.ru fearcrow.com
fba-mexico.com fenoyl.batcave.net
fetishinside.com
fifa-coins.online fifa-coins.online
findclan.org filesmonster.porn
findsexguide.com filmci.pro
finejewelryshop.ru fitness-video.net
flauntyoursite.com floating-share-buttons.com
flirt4free.com floridahuntingfishingadventures.com
floridamhca.org focalink.com
folowsite.com food.dtu.dk
footbalive.org footbalive.org
fordsonmajbor.cf footballfarrago.com
forexgb.ru forum-engineering.ru
formulaantiuban.com freefoto.ca
forzeronly.com freegamesplay.online
foto-weinberger.at front.ru
fotosfotos.eu fuel-gas.com
free-social-buttons.xyz fxgallery.com
freewhatsappload.com galeria-zdjec.com
freshmac.space gallery.rennlist.com
futbolkisales.ru gazoblok.net.ua
gearcraft.us ge0ip.net
geileweiber.tk geilehausfrauen.net
generalporn.org genericlowlatencyasiodriverhq.aircus.com
generousdeal-a.akamaihd.net genetworx.com
get-free-social-traffic.com get-free-social-traffic.com
getyourimage.club get-free-traffic-now.com
goldandcard.ru global-ics.co.za
gombita.info goblacked.com
goodwinmetals.co goldadpremium.com
goroda-vsego-mira.ru goo.ne.jp
gpms.org.my gotomontenegro.net
greamimgo.pw gratuitxblcodes.com
greatfind-a.akamaihd.net guod.me
gribkovye-zabolevaniya.com h2monline.com
grizzlysgrill.com hdimagegallery.net
gsou.cf hidemyass.com
gwebtools.com.br holidaypics.org
hidefiles.org home-task.com
hledejvshopech.cz home.myplaycity.com
hotenergy.ru homedecorpicture.us
hotshoppymac.com homegardenlova.com
houseofgaga.ru hop.clickbank.net
houseofrose.com hostcritique.com
hoverboard360.nl houston-vikings.com
howtoclean.club hoverboardforsaledirect.com
hptwaakw.blog.fc2.com idc.com.ua
hscsscotland.com ilmexico.com
hunterboots.online imagefinder.site
hut1.ru imperia31.ru
iblogpress.xyz impotentik.com
ilovevitaly.xyz inbabes.sexushost.com
imagine-ex.co incep.imagine-ex.co
import-sales.com indo-export.ru
impresagaia.it insurple.com
indiasourcemart.in
infokonkurs.ru
intervsem.ru
invest-pamm.ru invest-pamm.ru
investmac.com iscblog.info
islamtoday.co.za iusstf.org
itservicesthatworkforyou.com ivanstroi.ru
japfm.com iwantedmoney.com
javrip.net izismile.com
javxxx18.com jacago.com
jerseyschinabizwholesale.us james13prix.info
jewelryandfiligree.com jennyfire.ru
joingames.org jerseychinabizwholesale.us
juliadiets.com jetsli.de
kakablog.net johnrobertsoninc.com
kambasoft.com kaac.ru
kasino-money.pw kabbalah-red-bracelets.com
katadhin.co kadashihotel.com
keywesthideaways.co kakadu-interior.com.ua
keywordhut.com kazka.ru
keywordteam.net kerch.site
kinomaniatv.pl ketoanhanoi.info
kinotorka.ru keywordblocks.com
knigonosha.net keywordspay.com
komputernaya-pomosh-moscow.ru kinky-fetishes.com
kotaku.com kinoplen.ru
krasnodar.xrus.org kiwe-analytics.com
kutikomi.net klumba55.ru
kvartir-remont.biz kndxbkdx.bloger.index.hr
lanadelreyfans.us koversite.info
larutti.ru krafte.ru
leadn.pl landinez.co
lemon-ade.site laptop-4-less.com
lexaprogeneric.link lechtaczka.net
lightinghomes.net lego4x4.ru
link.ac letsrepair.in
linkredirect.biz linkredirect.biz
linksharingt.com liupis.com
liquimondo.com liveinternet.ru
live-sexcam.tk liverpool.gsofootball.com
live-sexchat.ru ljusihus.se
llastbuy.ru locatellicorretor.com.br
longgreen.info lodki-pvh.dp.ua
longlifelomilomi.info lutherstable.org
lovasszovetseg.hu luxup.ru
ltvperf.com m-google.xyz
luckyshop.net.ua mackeeper-center.club
lyngdalhudterapi.no macotool.com
m5home.ru mainlinehobby.net
mabdoola.blogspot.com make-money-online.com
macdamaged.space manipulyator-peterburg.ru
magicalfind-a.akamaihd.net marblestyle.ru
manyresultshub-a.akamaihd.net medanestesia.ru
marketland.ml medi-fitt.hu
megaapteka.ru mediafresh.online
member-quiz.com medictube.ru
mesbuta.info medkletki.ru
middlerush-a.akamaihd.net medkritika.ru
midst.eu megakino.net
mielec.pl meget.co.za
mir-business-24.ru minecraft-neo.ru
missclub.info minet.club
moesen-ficken.com mini.7zap.com
moomi-daeri.com minyetki.ru
moscow-clining.ru misswell.net
moviemail-online.co.uk mjchamonix.org
mrmoneymustache.com mlvc4zzw.space
muabancantho.info mobilemedia.md
my-big-family.com moetomnenie.com
myhitmp3.club monetizationking.net
mylovelibrabry.com money-every-day.com
mymobilemoneypages.com moneyzzz.ru
myperiod.club morepoweronmac.trade
nalogovyykodeks.ru mototsikl.org
namecrumilchlet.tk mozello.ru
nardulan.com mrvideospornogratis.xxx
narkologiya-voronezh.ru mundoaberrante.com
musezone.su
mycouponizemac.com
mypets.by
mywallpaper.top
mzdish.site
namenectar.com
naughtyconnect.com naughtyconnect.com
needtosellmyhousefast.com netallergy.ru
net-profits.xyz networkad.net
nfljerseyscheapchinabiz.com nevansk.ru
nl.netlog.com newhairstylesformen2014.com
nlfjjunb5.ru nhl17coins.exblog.jp
niroo.info
nisuturnetdgu.tk
novodigs.com
nuit-artisanale.com
nuker.com
nuup.info
o-o-8-o-o.ru
o00.in
odoratus.net odoratus.net
of-ireland.info offer.camp
officedocuments.net onlyforemont.ru
ok.ru ooo-gotovie.ru
okayimage.com openlibrary.org
onlainbesplatno.ru orel-reshka.net
onlinadverts.com origin-my.ru
onlinebay.ru
onlinemeetingnow.com
onlinewritingjobs17.blogspot.ru
open-odyssey.org
oren-cats.ru
ourville.info
outrageousdeal-a.akamaihd.net outrageousdeal-a.akamaihd.net
outshop.ru oz-offers.com
paparazzistudios.com.au partnerads.men
partner-cdn.men pc-virus-d0l92j2.pw
patterntrader-en.com piccshare.com
perfectplanned.com picscout.com
photo.houseofgaga.ru
pic-re.blogspot.com
picsearch.com picsearch.com
picswe.com pinup-casino1.ru
picturesboss.com piratecams.com
pl-vouchers.com pirateday.ru
pl.id-forex.com pladform.ru
platezhka.net platezhka.net
play-movie.pl pochemychka.net
pod-muzyku.club poffet.net
pogosh.com poligon.com
polytopesexempt.com pontiacsolstice.info
popander.mobi porno-raskazy.ru
porndl.org pornosemki.info
porno-chaman.info pornosok.ru
pornofeuer.com pozdravleniya-c.ru
pornolook.net prchecker.info
pornotubexxx.name prefersurvey.net
pornstartits.xblog.in pricheski-video.com
posible.net prlog.ru
positive2b.ru profolan.pl
pravoholding.ru promgirldresses.xyz
printingpeach.com prostitutki-belgoroda.org
privatov-zapisi.ru prostitutki-novgoroda.org
prombudpostach.com.ua purplesphere.in
promoforum.ru pussysaga.com
prostitutki-rostova.org pyramidlitho.webs.com
purchasepillsnorx.com
puteshestvennik.com
quality-traffic.com
qualitymarketzone.com qualitymarketzone.com
quebec-bin.com quickcashlimited.com
quickchange.cc rapevideosmovies.com
rcb101.ru rcb101.ru
redhotfreebies.co.uk regionshop.biz
registratciya-v-moskve.ru remorcicomerciale.ro
registrydomainservices.com renewablewealth.com
relatodelpresente.com.ar resultshub-a.akamaihd.net
remont-mobile-phones.ru rethinkwasteni.info
renecaovilla.online rewardpoll.com
repeatlogo.co.uk rfd-split.hr
respublica-otel.ru rff-cfal.info
rezeptiblud.ru riders.ro
ring4rhino.com ring4rhino.com
rogervivierforsale.com rocis.site
rospromtest.ru runstocks.com
rubbed.us rus-teh.narod.ru
rumamba.com s1z.ru
rusenvironmental.net saecsa.co
russkie-gorki.ru safe-app.net
saddiechoua.com saltspray.ru
saletool.ru santasgift.ml
sandhillsonline.com
sardinie.us
satellite.maps.ilovevitaly.com satellite.maps.ilovevitaly.com
scanner-nelson.top scalerite.co.za
scrapinghub.com scmor.ilxc.cc
seansonline24.pl sdi-pme.com
search.1and1.com seadragonherbery.com
searchimpression.com search.xtconnect.com
searchlock.com secret.xn--oogle-wmc.com
segol.tv seolab.top
seoanalyses.com serialsway.ucoz.ru
seogadget.ru serptehnika.ru
seokicks.de servisural.ru
servingnotice.com
sevenstars7.com
sex-tracker.com sex-tracker.com
sex.hotblog.top sex-tracker.de
sexiporno.net shemale-sex.net
sexy-screen-savers.com shitting.pro
sexytrend.ru shop.xz618.com
shikiso.info simply.net
shodanhq.com sisiynas.ru
shohanb.com snsdeainavi.info
shop.acim.org snworks.com
shopfishing.com.ua socialbutton.xyz
shoppingjequiti.com.br socialmadesimple.com
shopsellcardsdumps.com socialseet.ru
sideeffectsoftizanidine.blogspot.com sodexo.com
sigmund-freud.co.uk sofit-dmd.ru
simple-image.com.ua soft-terminal.ru
sitevalued.com speedup-my.site
skatestick.bid
skincrate.net
slim.sellany.ru
slowmacfaster.trade
smallseotools.com
smstraf.ru
softonicads.com
softxaker.ru
solinf.co
sotkal.lark.ru
sovetogorod.ru
sovinsteel.ru
space2019.top
sportbetfair.com sportbetfair.com
spravka-medosmotr.ru
spylog.com spylog.com
stackthatbucks.com sr-rekneskap.no
stal-rulon.ru sribno.net
stairliftstrue.com
stanthonyscatholicchurch.org stanthonyscatholicchurch.org
staticfs.host
steelmaster.lv
stmassage.ru
stockspmb.info
stop-gepatit.te.ua
stpicks.com
strag-invest.ru strag-invest.ru
strangeduckfilms.com strongsignal-a.akamaihd.net
stroilka.info sumo.com
sugarkun.com superfish.com
svetodiodoff.ru supermesta.ru
sygraem.com support.nopeas.sk
tattooreligion.ru symbaloo.com
the-torrent-tracker.blogspot.com syvertsen-da.no
thegameriders.com tagil.zrus.org
thetattoohut.com tam-gde-more.ru
timeallnews.ru tattoo33.ru
titangel-vietnam.com tech4master.com
titslove.yopoint.in telsis.com
thefarmergame.com
thelottosecrets.com
theplacetoupdating.pw
thepokertimer.com
thepornsex.org
timdreby.com
tizanidine4mg.blogspot.com
tizanidine4mgstreetvalue.blogspot.com
tizanidine4mguses.blogspot.com
tizanidineformigraines.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tizanidinerecreationaluse.blogspot.com tizanidinehclsideeffects.blogspot.com
tootoo.to tizanidineinteractions.blogspot.com
topbestgames.com tizanidinesideeffects.blogspot.com
topkarkas.com tkanorganizma.ru
torrent-to-magnet.com tn811.us
traffic2money.com top10-online-games.com
trafficnetzwerk.de topclickguru.com
tourismvictoria.com
track.deriv.com
tradedeals.biz
trafficjunky.net
trafficmonetize.org
trafficshaper.com trafficshaper.com
trkur.com traxdom.ru
tube8.com turist-strani.ru
tuberkulezanet.ru tversvet.ru
tuckermktg.com uasb.ru
ukrtvir.com.ua uchebavchehii.ru
ultimateclassicrock.com uggbootsoutletsale.us
ugogo.info
unacittaconte.org
unitygame3d.com unitygame3d.com
univerfiles.com uptimebot.net
universals.com.ua ural-buldozer.ru
urldelivery.com urlopener.blogspot.com.au
userequip.com urlopener.com
uzungil.com uroki.net
v720hd.ru usadacha.net
vepad.com usswrite.com
video-woman.com vacances-voyages.info
videochat.world valoresito.com
valueclick.com
veopornogratis.xxx
vesnatehno.ru
video-hollywood.ru
videochaty.ru videochaty.ru
videojam.tv videospornogratisx.net
videonsk.com viktoria-center.ru
vip-file.com vk-mus.ru
vkgaleria.com vkgaleria.com
vkonche.com vpnhowto.info
voloomoney.com vse-pesni.com
vuclip.com vsexkontakte.net
vvpg.ru w3data.co
wait3sec.org waysbetter.cn
wareseeker.com
wdrake.com wdrake.com
webcamdevochka.com we-are-gamers.com
webinstantservice.com webadvance.club
wedding-salon.net webkeyit.com
weprik.ru whatistizanidinehclusedfor.blogspot.com
westvilletowingservices.co.za wieseversa.no
wholesalejerseychinashop.com wildnatureimages.com
wineitudes.wordpress.com wineration.com
womensterritory.ru womama.ru
wordpresscore.com woman-tampon.ru
wordpress-crew.net
wosik-dach.service-for-web.de wosik-dach.service-for-web.de
wurr.voila.net wwwadultcheck.com
www.bookmaker-bets.com xmlinde.com
xblog.in xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xev.ru xn----7sbaphztdjeboffeiof6c.xn--p1ai
xingzi-vision.com xn----9sbebi2bvzr7h.xn--p1ai
xitjw.info xn----dtbndd4ae7eub.top
xlovecam.com xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xmnb.net xn--b1adccaf1bzj.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xn--d1abj0abs9d.in.ua
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xsfetish.org
xn----9sbubg3ambdfl1j.xn--p1ai xxx-treker.ru
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xxxguitars.com
xn--h1aakne2ba.xn--p1ai yatrk.xyz
xxxtubesafari.com yebocasino.com
yachts-cruise.info youdao.com
yottos.com youporn-forum.ga
yougetsignal.com zaidia.xhost.ro
your-bearings.com zakazfutbolki.com
youradulthosting.com zarajbuilders.com
yourserverisdown.com zawyna.ua
zaim-pod-zalog-krasnodar.ru zebradudka.com
zeroredirect6.com zebramart.ru
zetgie.com.pl zigarettenonlinekaufen1.bloog.pl
zigarettenonlinekaufen2.drupalgardens.com zigarettenonlinekaufen2.drupalgardens.com
zlatnajesen.com zirondelli.it
znakomstva-moskva77.ru zixizop.net.ru
znaturaloriginal.com zmoda.hostreo.com
zocaparj.kz zoodrawings.com
zolotoy-lis.ru zygophyceous.womanstars.site
zona-aqua.ru zytpirwai.net
zooggames.com
zvetki.ru
zvuker.net

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
4webmasters.com 214jaluwobapef.cf
777-club.ru 2kata.ru
888.com 2nt.ru
9icmzvn6.website 4replicawatch.net
aaronabel.com a96527gi.beget.tech
aero2.ru aa9bd78f328a6a41279d0fad0a88df1901.com
affordablewebsitesandmobileapps.com acmebtn.ml
aktivator-windows10.blogspot.com admanaerofoil.com
android-systems.ru adsland.men
anticrawler.org adultmeetonline.info
apidigidocketnet-a.akamaihd.net adultnet.in
approved.su adultwebhosting.info
arate.ru aerogo.com
asdfg.pro afora.ru
asseenontvonline.ru ahhjf.com
autobrennero.it all-number.com
autowebmarket.com.ua amospalla.es
azarttoto.com arcteryxsale.online
azbukafree.com as5000.com
bannerbank.ru aucoinhomes.com
battlecarnival.su aussie-prizes.com
bavsac.com b2b-lounge.com
bcmp.org backgroundpictures.net
beremenyashka.com bank.uz
bannerads.de
best-businessman.ru
bestbrainprod.win bestbrainprod.win
bigpenisguide.com besthoro.ru
bimmerpost.com bestssaker.com
binomo.com beta.hotkeys.com
bkgr.se big-boards.info
blagovest-med.ru biographiya.com
blog-hits.com
blog.remote-computer.de
blogporn.in blogporn.in
blogs.rediff.com blogs.rediff.com
boltalko.xyz bmwhighperformers.com
bonky.biz bongacams.com
book-bets.com buyessay3.blogspot.ru
brothers-smaller.ru buyhoverboard.com
buypanicdisorderpill.com cablecar.us
buytizanidineonlinenoprescription.blogspot.com callstevens.com
bzero1jewelry.net calstaterealty.us
cakesplus.com.au canzoni.ru
canrole.com cfacarrosserie74.com
cbcseward.com chatroulette.online
celebsopics.com
cementaresearch.se
chatroulette.si
cheap-pills-norx.com cheap-pills-norx.com
cheapjerseysbizwholesale.us children-learningreading.info
chiblackhawks-jerseys.com christianlouboutinsaleonline.us
circlesl.com chuckguilford.com
cjs.com.ru clash-clans.ru
cleanallspyware.com clevernt.com
cocyq.inwtrade.com compliance-alex.top
comissionka.net compliance-george.top
compliance-alexa.xyz compliance-ivan.top
compliance-willy.top connectionstrenth.com
cool-wedding.net coolgramgoods.com
credit-respect.ru copblock.org
cornerstone-countertops.com
cougfan.info
courtshipgift.com
cryptoswap.biz
customchocolate.business-for-home.com customchocolate.business-for-home.com
cyclobenzaprinevstizanidine.blogspot.com darmebel.com.ua
cyse.tk datadepths.com
dailystrength.org ddsoldes.fr
deplim.com dimaka.info
dignitasdata.se dinosaurus.site
dominateforex.ml div.as
domznaniy.ru dmmspy.com
easytuningshop.ru docsportal.net
easyukraine.com dogoimage.com
eets.net domaineaneblanc.com
efnor-ac.com dosugrostov.site
englate.com downtuptv.gq
erectile.bid drugs-no-rx.info
erotik0049.com drupa.com
everflora.ru education-cz.ru
exmasters.com efkt.jp
exoneration-project.us ekobata.ru
famix.xyz englishgamer.com
fitfloponline.store enskedesquashclub.se
forum.tvmir.org enternet.ee
freza-sverlo.ru euronis-free.com
fxtips.ru exdocsfiles.com
generousdeal-a.akamaihd.net exlarseva.webblog.es
gepezz.info ezofest.sk
get-free-traffic-now.com fastfixing.tech
getdot.ru feeriaclub.ru
gilsonchiro.xyz figuringmoneyout.com
girls-ufa.ru free-traffic.xyz
girlsatgames.ru funnel.co.za
globalsurfari.com geile-lelly.eu
googst2.ru gg.zzyjxs.com
gpirate.com gk170.ru
gratuitbaise.com go2jump.org
guge.io greenshop.su
hamilton.ca gsmlab.pl
hesteel.pl guiadeserraazul.com
hkladys.com guildebzh.info
howopen.ru gwagka.com
hvd-store.com hdwalls.xyz
icaseclub.ru healbio.ru
iideaidekonkatu.info highspeed5.net
iinstalll-fii1leis.jus0wil.pp.ua homeandhealth.ru
infowarcraft.ru hplaserjetpdriver8y.pen.io
internetartfair.com hully.altervista.org
jamiembrown.com hundejo.com
janerikholst.se iblogpress.xyz
javitas.info idealtits.net
jerseychinabizwholesale.com img.wallpaperstock.net
jerseysbizwholesalecheap.com inboxdollars.com
jose.mulinohouse.co infobabki.ru
justdating.online informatiecentro.be
jwcialislrt.com iscblog.info
kakadu-interior.com.ua iskalko.ru
kaz.kz ix20.ru
keywordbasket.com jagg.info
keywords-monitoring-success.com javidol.site
kiinomaniak.pl jofucipiku.tk
kinofak.net jrcigars.com
krasnodar-avtolombards.ru kambasoft.com
kriokomora.info keywordblocks.com
kunstaktien.info kinogolos.ru
linkpulse.com kozhniebolezni.com
livingroomdecor.info laexotic.com
magento-crew.net layola.biz.tc
mararoom.ru linkwithin.com
medi-fitt.hu lottospring.com
medkritika.ru love-baby.cz
meet-flirt-dating.com mahnwachen-helfen.info
megaapteka.ru map028.com
mil-stak.com maximilitary.ru
mmgq.ru mazda-roadsters.com
mobicover.com.ua medicine-4u.org
mobot.site mega-polis.biz.ua
mobplayer.ru metalonly.info
moncleroutletonline.win microsoftportal.net
moneyzzz.ru mindbox.co.za
moomi-daeri.com muznachas-service.ru
morocco-nomad-excursions.com mydeathspace.com
mrvideospornogratis.xxx
mug-na-chas-moscow.ru
my-cash-bot.co
mydirtystuff.com
myfreetutorials.com
mygameplus.ru mygameplus.ru
myhitmp3.club myliveblog.ru
mysexpics.ru mysexpics.ru
namecrumilchlet.tk nac-bearings.ru
namenectar.com narco24.me
networkcheck.xyz netoil.no
noclegonline.info netvouz.com
obiavo.by nfljerseysforsalewholesaler.com
oda.as nikkiewart.ru
omgtnc.com o-dachnik.ru
oneminutesite.it o-o-8-o-o.com
online.ktc45.ru oktube.ru
oops-cinema.ru osnova3.ru
orion-code-access.net parlament.biz
pagesense.com partner-stop.men
paparazzistudios.com.au partnerline.men
partner-print.men pechikamini.ru
pcads.ru phuketscreen.com
pdns.download picanalyzer.data-ox.com
perso.wanadoo.es picquery.com
pestomou.info picscout.com
pharmacyincity.com pierrehardysale.online
pijoto.net piulatte.cz
plastweb.ru pixshark.com
pooleroadmedicalcentre.co.uk pod-muzyku.club
porno-play.net podshipniki-nsk.ru
porno.simple-image.com.ua poiskzakona.ru
pornosee.info pornhub-forum.ga
pornosemki.info potoideas.us
portside.cc promodj.com
princeadvantagesales.com prosmibank.ru
printingpeach.com prpops.com
pro-okis.ru qor360.com
pukaporn.com quelle.ru
pulseonclick.com randki-sex.com
quit-smoking.ga rapevideosmovies.com
qwertty.net
rada.ru
rankia.com
rankingchart.de
rating-casino2021.ru
razvratnoe.org razvratnoe.org
real-time-analytics.com recipedays.com
realitykings.com rfd-split.hr
redirlock.com ringtonepartner.com
registrationdomainsite.com rique.host.sk
reklama1.ru rn-to-bsn.com
remmling.de rockma.se
remont-fridge-tv.ru ryetaw.com
russia-today-video.ru sady-urala.ru
s.lollypopgaming.com sale-japan.com
sa-rewards.co.za savetubevideo.com
selectads.men sbf441.com
seo-prof1.xyz scalerite.co.za
seocheki.net scanner-viktor.top
seoexperimenty.ru sdrescher.net
seolab.top se-welding.ru
sharebutton.org seccioncontrabajo.com
shell-pmr.ru seinterface.com
shoppingjequiti.com.br semxiu.com
sierraapps.com seo18.su
socialmediasuggest.com shinikiev.com.ua
soft-program.com ship-marvel.co.ua
stanthonyscatholicchurch.org silverage.ru
stjamesschool.info snsdeainavi.info
styro.ru soft-terminal.ru
tagil.zrus.org space2019.top
tedy.su sribno.net
teenporn18.net stats-public.grammarly.io
thebitcoincode.com stillmiracle.com
thfox.com sugarlyflex.pw
titangel-vietnam.com supervesti.ru
tizanidine4mg.blogspot.com svarkagid.com
tizanidine4mgprice.blogspot.com tabakur77.com
tizanidine4mgtablets.blogspot.com tattoo33.ru
tizanidineingredients.blogspot.com techart24.com
tmearegion26.com tengohydar.tk
topsiteminecraft.com thebluffs.com
torrentgamer.net thenews-today.info
trackzapper.com thepornsex.org
trafficmp.com tizanidinehcl2mg.blogspot.com
traffpartners.com tizanidinehydrochloride2mg.blogspot.com
tula.mdverey.ru tizanidineinfo.blogspot.com
tzritel.tk tizanidineshowupondrugtest.blogspot.com
ua-company.ru tksn.ru
uptimebot.net tobeyouday.win
us-america.ru topcasinoratings.ru
videtubs.pl torrnada.ru
vitanail.ru truemfilelj.gq
voronezh.xrus.org trustedhealthtips.com
watchmygf.net turbabitload.weebly.com
website-speed-check.site twittruth.com
wholesalejerseychinaoutlet.com uasb.ru
wikes.20fr.com uginekologa.com
woodyguthrie.se uk-zheu20.ru
www.get-free-traffic-now.com upproar.com
x-lime.net varbergsvind.se
xbaboon.com video-hollywood.ru
xblognetwork.com vostoktrade.info
xcc24.pl vsesubwaysurfers.com
xep.info webinstantservice.com
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai welck.octopis.com
xn--80atua3d.xn--p1ai wholesalejerseychinashop.com
xn--h1aakne2ba.xn--p1ai winwotgold.pl
xxxguitars.com wormix-cheats.ru
yourporn.com x-lime.com
youtube-downloader.savetubevideo.com xmladserver.com
zeroredirect12.com xn----7sbifcamovvfggw9d.xn--p1ai
znaniyapolza.ru xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--b1agm2d.net
xxxrus.org
zaloadi.ru
zastroyka.org
zoogdinsney.com

73
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -8,44 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +456,41 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.37 0; 10.1.1.217 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3493 ### Version: V4.2022.11.3494
### Updated: Sat Nov 26 10:03:24 UTC 2022 ### Updated: Sat Nov 26 22:01:08 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3493 ### Version: V4.2022.11.3494
### Updated: Sat Nov 26 10:03:24 UTC 2022 ### Updated: Sat Nov 26 22:01:08 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##