vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 02:59:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2511 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-08 09:42:28 +00:00
parent 08221fe81b
commit a21224b5ad
50 changed files with 1154 additions and 1108 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2510 #### Version: V4.2021.08.2511
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2510 #### Version: V4.2021.08.2511
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -11,9 +11,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2510 #### Version: V4.2021.08.2511
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
_generator_lists/bad-user-agents.list
View file

@ -1,5 +1,3 @@
cah.io.community
Crawling\ at\ Home\ Project
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -90,6 +88,7 @@ CopyRightCheck
Copyscape Copyscape
Cosmos Cosmos
Craftbot Craftbot
Crawling\ at\ Home\ Project
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
CrunchBot CrunchBot
@ -560,6 +559,7 @@ arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider autoemailspider
backlink-check backlink-check
cah.io.community
check1.exe check1.exe
coccocbot-web coccocbot-web
cognitiveseo cognitiveseo

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2510 #### Version: V4.2021.08.2511
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

8
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

8
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2510 ### Version: V3.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

8
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/_robots_input/robots-input.txt
View file

@ -88,6 +88,7 @@ CopyRightCheck
Copyscape Copyscape
Cosmos Cosmos
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
CrunchBot CrunchBot
@ -558,6 +559,7 @@ arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider autoemailspider
backlink-check backlink-check
cah.io.community
check1.exe check1.exe
coccocbot-web coccocbot-web
cognitiveseo cognitiveseo

8
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2510 ### Version: V3.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 1; 10.1.0.47 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 0; 10.1.0.47 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2510 ### Version: V3.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 1; 10.1.0.47 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 0; 10.1.0.47 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2510 ### Version: V3.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 1; 10.1.0.47 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -149,6 +149,7 @@
"~*(?:\b)Copyscape(?:\b|)" 0; "~*(?:\b)Copyscape(?:\b|)" 0;
"~*(?:\b)Cosmos(?:\b|)" 0; "~*(?:\b)Cosmos(?:\b|)" 0;
"~*(?:\b)Craftbot(?:\b|)" 0; "~*(?:\b)Craftbot(?:\b|)" 0;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b|)" 0;
"~*(?:\b)CrazyWebCrawler(?:\b|)" 0; "~*(?:\b)CrazyWebCrawler(?:\b|)" 0;
"~*(?:\b)Crescent(?:\b|)" 0; "~*(?:\b)Crescent(?:\b|)" 0;
"~*(?:\b)CrunchBot(?:\b|)" 0; "~*(?:\b)CrunchBot(?:\b|)" 0;
@ -619,6 +620,7 @@
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
"~*(?:\b)autoemailspider(?:\b|)" 0; "~*(?:\b)autoemailspider(?:\b|)" 0;
"~*(?:\b)backlink-check(?:\b|)" 0; "~*(?:\b)backlink-check(?:\b|)" 0;
"~*(?:\b)cah.io.community(?:\b|)" 0;
"~*(?:\b)check1.exe(?:\b|)" 0; "~*(?:\b)check1.exe(?:\b|)" 0;
"~*(?:\b)coccocbot-web(?:\b|)" 0; "~*(?:\b)coccocbot-web(?:\b|)" 0;
"~*(?:\b)cognitiveseo(?:\b|)" 0; "~*(?:\b)cognitiveseo(?:\b|)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 0; 10.1.0.47 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2510 ### Version: V3.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2510 2511

8
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2510 ### Version: V3.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

8
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.235.86 0; 40.71.167.189 0;

8
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 1; 10.1.0.47 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 0; 10.1.0.47 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 1; 10.1.0.47 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 0; 10.1.0.47 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 1; 10.1.0.47 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -149,6 +149,7 @@
"~*(?:\b)Copyscape(?:\b|)" 0; "~*(?:\b)Copyscape(?:\b|)" 0;
"~*(?:\b)Cosmos(?:\b|)" 0; "~*(?:\b)Cosmos(?:\b|)" 0;
"~*(?:\b)Craftbot(?:\b|)" 0; "~*(?:\b)Craftbot(?:\b|)" 0;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b|)" 0;
"~*(?:\b)CrazyWebCrawler(?:\b|)" 0; "~*(?:\b)CrazyWebCrawler(?:\b|)" 0;
"~*(?:\b)Crescent(?:\b|)" 0; "~*(?:\b)Crescent(?:\b|)" 0;
"~*(?:\b)CrunchBot(?:\b|)" 0; "~*(?:\b)CrunchBot(?:\b|)" 0;
@ -619,6 +620,7 @@
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
"~*(?:\b)autoemailspider(?:\b|)" 0; "~*(?:\b)autoemailspider(?:\b|)" 0;
"~*(?:\b)backlink-check(?:\b|)" 0; "~*(?:\b)backlink-check(?:\b|)" 0;
"~*(?:\b)cah.io.community(?:\b|)" 0;
"~*(?:\b)check1.exe(?:\b|)" 0; "~*(?:\b)check1.exe(?:\b|)" 0;
"~*(?:\b)coccocbot-web(?:\b|)" 0; "~*(?:\b)coccocbot-web(?:\b|)" 0;
"~*(?:\b)cognitiveseo(?:\b|)" 0; "~*(?:\b)cognitiveseo(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 0; 10.1.0.47 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.235.86 0; 40.71.167.189 0;

8
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.235.86 0; 40.71.167.189 0;

8
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:07 UTC 2021 ### Updated: Sun Aug 8 09:36:42 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Copyscape(?:\b)" 3; "~*(?:\b)Copyscape(?:\b)" 3;
"~*(?:\b)Cosmos(?:\b)" 3; "~*(?:\b)Cosmos(?:\b)" 3;
"~*(?:\b)Craftbot(?:\b)" 3; "~*(?:\b)Craftbot(?:\b)" 3;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b)" 3;
"~*(?:\b)CrazyWebCrawler(?:\b)" 3; "~*(?:\b)CrazyWebCrawler(?:\b)" 3;
"~*(?:\b)Crescent(?:\b)" 3; "~*(?:\b)Crescent(?:\b)" 3;
"~*(?:\b)CrunchBot(?:\b)" 3; "~*(?:\b)CrunchBot(?:\b)" 3;
@ -707,6 +708,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;
"~*(?:\b)coccocbot-web(?:\b)" 3; "~*(?:\b)coccocbot-web(?:\b)" 3;
"~*(?:\b)cognitiveseo(?:\b)" 3; "~*(?:\b)cognitiveseo(?:\b)" 3;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 1; 10.1.0.47 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -149,6 +149,7 @@
"~*(?:\b)Copyscape(?:\b|)" 0; "~*(?:\b)Copyscape(?:\b|)" 0;
"~*(?:\b)Cosmos(?:\b|)" 0; "~*(?:\b)Cosmos(?:\b|)" 0;
"~*(?:\b)Craftbot(?:\b|)" 0; "~*(?:\b)Craftbot(?:\b|)" 0;
"~*(?:\b)Crawling\ at\ Home\ Project(?:\b|)" 0;
"~*(?:\b)CrazyWebCrawler(?:\b|)" 0; "~*(?:\b)CrazyWebCrawler(?:\b|)" 0;
"~*(?:\b)Crescent(?:\b|)" 0; "~*(?:\b)Crescent(?:\b|)" 0;
"~*(?:\b)CrunchBot(?:\b|)" 0; "~*(?:\b)CrunchBot(?:\b|)" 0;
@ -619,6 +620,7 @@
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
"~*(?:\b)autoemailspider(?:\b|)" 0; "~*(?:\b)autoemailspider(?:\b|)" 0;
"~*(?:\b)backlink-check(?:\b|)" 0; "~*(?:\b)backlink-check(?:\b|)" 0;
"~*(?:\b)cah.io.community(?:\b|)" 0;
"~*(?:\b)check1.exe(?:\b|)" 0; "~*(?:\b)check1.exe(?:\b|)" 0;
"~*(?:\b)coccocbot-web(?:\b|)" 0; "~*(?:\b)coccocbot-web(?:\b|)" 0;
"~*(?:\b)cognitiveseo(?:\b|)" 0; "~*(?:\b)cognitiveseo(?:\b|)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Joomla Abonti
LibWeb Brandprotect
MarkWatch DatabaseDriverMysqli
Mojolicious FDM
OrangeSpider Offline Explorer
PECL::HTTP OpenVAS
Telesphorep Re-re
Upflow SemrushBot
ZmEu SuperHTTP
eCatch siteripz

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404enemy
360Spider
80legs 80legs
ADmantX ADmantX
AIBOT
ALittle Client
Abonti Abonti
Aboundex Aboundex
AhrefsBot AfD-Verbotsverfahren
AiHitBot Aipbot
Alligator Alexibot
AwarioRssBot AllSubmitter
Anarchie
Aspiegel
Asterias
Attach
AwarioSmartBot
BackStreet BackStreet
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Badass
Bandit
Bigfoot
Black Hole
BlowFish
Boardreader Boardreader
BotALot
Brandprotect
Buddy Buddy
BuiltBotTough Calculon
BuiltWith
BuzzSumo
CATExplorador
CensysInspect
CheTeam CheTeam
CherryPicker
ChinaClaw
Claritybot Claritybot
Cogentbot Cliqzbot
Copyscape Cloud mapping
Cocolyzebot
Cosmos Cosmos
CrunchBot CrazyWebCrawler
DIIbot Crescent
Demon Curious
Deusu DSearch
Devil DataCha0s
DigitalPebble Digincore
Dirbuster Disco
Discobot Discoverybot
Dispatch Dispatch
DomCopBot DomainStatsBot
DomainCrawler Domains Project
Dotbot
Download Wonder Download Wonder
EMail Wolf Drip
Ebingbong ECCP/1.0
EroCrawler EMail Siphon
Exabot Exabot
Express WebPictures ExtLinksBot
ExtractorPro ExtractorPro
Extreme Picture Finder Flunky
Ezooms Fuzz
FHscan Fyrebot
FemtosearchBot G-i-g-a-b-o-t
Fimap
FlashGet
Foobot
GT::WWW GT::WWW
GetRight GetWeb
Getintent Go!Zilla
Gigabot Go-Ahead-Got-It
GoZilla Grabber
GrapeFX GrapeshotCrawler
HTMLparser GridBot
Haansoft
HaosouSpider
Harvest
Havij Havij
HybridBot Heritrix
IDBot IDBot
Image Fetch Image Fetch
InfoNaviRobot Image Sucker
Indy Library
InfoTekies
Intelliseek
InterGET InterGET
InternetSeer InternetSeer
Iria
Iskanie Iskanie
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
Jbrofuzz Jbrofuzz
JetCar JetCar
JustView Jorgee
Jyxobot
Kinza Kinza
Kozmosbot Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Larbin
Leap
LeechFTP LeechFTP
LexiBot LibWeb
LieBaoFast Lightspeedsystems
LinkScan
LinkWalker LinkWalker
Linkdexbot Linkdexbot
LinqiaMetadataDownloaderBot LinkpadBot
LinqiaScrapeBot LinqiaRSSBot
Lmspider Ltx71
MFC_Tear_Sample MIDown tool
MQQBrowser MQQBrowser
MSFrontPage MTRobot
MSIECrawler
Mag-Net Mag-Net
Majestic-SEO Majestic-SEO
Mass Downloader
Masscan Masscan
Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
Mojeek MicroMessenger
Mozlila Moblie Safari
Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Msrabot NPbot
Musobot
Name Intelligence
Nameprotect Nameprotect
Needle Nessus
NetAnts NetMechanic
NetSpider
NetZIP
Netcraft Netcraft
Netvibes Nibbler
Niki-bot Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus Nmap
Nutch
Octopus Octopus
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OnCrawl OpenLinkProfiler
OpenVAS Openfind
OrangeBot OrangeSpider
OutclicksBot OutfoxBot
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber PageGrabber
PageScorer Pandalytics
Pavuk Panscient
Pi-Monster Petalbot
Pixray Picscout
Pockey Picsearch
Pimonster
ProPowerBot ProPowerBot
ProWebWalker
Psbot Psbot
Pump PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot RankActive
RankFlex Rankivabot
RankingBot
RankingBot2
Re-re Re-re
RealDownload RedesScrapy
Reaper
Recorder
RepoMonkey
RocketCrawler
SBIder
SEOkicks SEOkicks
SISTRIX SMTBot
SalesIntelligent ScanAlert
Scanbot Scanbot
ScoutJet
Scrapy Scrapy
ScreenerBot
ScrepyBot
SearchmetricsBot SearchmetricsBot
Semrush
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Shodan
SiteLockSpider SiteLockSpider
SlySearch SiteSnagger
SmartDownload Snake
Snoopy Snapbot
SocialRankIOBot Sociscraper
Sosospider Sogou web spider
Sottopop SpaceBison
Spanner Spanner
Spbot Sucker
Spinn3r
SputnikBot
Sqlmap
Sqlworm
Steeler
Stripper
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
Surfbot
SurveyBot SurveyBot
Suzuran
Szukacz Szukacz
T0PHackTeam T0PHackTeam
T8Abot
TeleportPro
Telesoft Telesoft
Telesphoreo Telesphoreo
Telesphorep
The Intraformant The Intraformant
Thumbor
TightTwatBot TightTwatBot
Toata
Tracemyfile Tracemyfile
Trendiction
Trendictionbot
True_Robot
Turingos Turingos
TwengaBot Turnitin
UnisterBot URLy Warning
URLy.Warning
V-BOT V-BOT
VB Project
VCI VCI
VeriCiteCrawler Vacuum
Virusdie Virusdie
Voltron VoidEYE
WEBDAV
WISENutbot WISENutbot
WWW-Collector-E
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers/3.0
WeSEE
Web Auto
Web Collage
Web Enhancer
Web Fetch Web Fetch
Web Pix Web Sauger
Web Sucker Web Sucker
WebCollage WebImageCollector
WebFuck WebSauger
WebPix WebStripper
WebReaper WebZIP
WebSucker Webalta
WebWhacker WebmasterWorldForumBot
Webshag
Website Quester Website Quester
Whatweb Whack
Who.is Bot Who.is Bot
WiseGuys Robot
Wonderbot
Woobot Woobot
Zermelo Wotbox
Zitebot Xaldon WebSpider
Xenu
Zauba
Zeus
ZmEu ZmEu
ZoomBot
ZumBot ZumBot
ZyBorg
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider
cah.io.community
cognitiveseo cognitiveseo
com.plumanalytics crawl.sogou.com
crawler4j crawler4j
facebookscraper evc-batch
gopher
instabid instabid
lwp-request internetVista monitor
lwp-trivial ips-agent
isitwp.com
magpie-crawler magpie-crawler
meanpathbot page scorer
muhstik-scan pcBrowser
polaris version s1z.ru
probe-image-size
scalaj-http
scan.lol scan.lol
seobility
seoscanners seoscanners
siteripz seostar
sogouspider serpstatbot
spyfu sysscan
tAkeOut trendiction.com
trendiction.de
voyagerx.com
webmeup-crawler webmeup-crawler
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
AIBOT 80legs
ALittle Client
ASPSeek ASPSeek
Abonti Abonti
Acunetix Aboundex
AfD-Verbotsverfahren Anarchy99
AiHitBot Anthill
Anarchie
Aspiegel Aspiegel
Asterias Attach
AwarioSmartBot
BBBike BBBike
BDCbot BackStreet
BDFetch Backlink-Ceck
BackDoorBot
BacklinkCrawler BacklinkCrawler
Barkrowler Bandit
BatchFTP BatchFTP
Bitacle Black Hole
BlackWidow
Blackboard Blackboard
BotALot Blow
BlowFish
Brandprotect Brandprotect
Buddy Buck
BuiltWith
Bullseye
CATExplorador
CCBot CCBot
CODE87
CSHttp CSHttp
Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam CheTeam
CheeseBot CherryPicker
Chlooe
Claritybot Claritybot
Cliqzbot
Cloud mapping Cloud mapping
Cogentbot Cocolyzebot
Copier Collector
CopyRightCheck CopyRightCheck
Cosmos Curious
Craftbot CyotekWebCopy
CrunchBot DIIbot
Custo DTS Agent
DBLBot
DatabaseDriverMysqli
Demon Demon
Devil
Digincore Digincore
DigitalPebble
Disco
Discobot
Discoverybot Discoverybot
DomCopBot DnyzBot
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
ECCP/1.0 Dotbot
EMail Siphon EMail Wolf
EirGrabber Ebingbong
Exabot EroCrawler
Express WebPictures Express WebPictures
ExtLinksBot
Extractor Extractor
Ezooms ExtractorPro
FDM Extreme Picture Finder
FHscan
FemtosearchBot FemtosearchBot
Fimap
Firefox/7.0
FlashGet FlashGet
Foobot Flunky
Freeuploader Freeuploader
FyberSpider FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW
GermCrawler GermCrawler
GetRight Gigabot
GetWeb Go-Ahead-Got-It
Getintent
Gigablast
GoZilla
Gotit Gotit
GrabNet Grafula
Haansoft GrapeFX
GridBot
HTTP::Lite
HaosouSpider HaosouSpider
Harvest
Havij Havij
Hloader HybridBot
IRLbot IDBot
Image Fetch Iblog
IndeedBot IlseBot
Indy Library Image Sucker
InfoNaviRobot InfoTekies
Intelliseek Iskanie
InternetSeer Jbrofuzz
Iria
IstellaBot
JetCar JetCar
Jetty Jetty
JikeSpider JikeSpider
Joomla Joomla
Jorgee Jyxobot
Leap Keyword Density
LeechFTP LNSpiderguy
LexiBot Larbin
Linkbot LeechGet
LinkextractorPro LibWeb
LinkpadBot Libwhisker
LinksManager LinkScan
LinqiaRSSBot LinkWalker
Linkdexbot
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MFC_Tear_Sample
MIIxpc MIIxpc
MQQBrowser MQQBrowser
Mag-Net MSFrontPage
MSIECrawler
Magnet Magnet
Majestic SEO
Majestic12
Mass Downloader Mass Downloader
MeanPath Bot MauiBot
Metauri Mb2345Browser
MicroMessenger Meanpathbot
Microsoft Data Access Microsoft Data Access
Mister PiX Mister PiX
Moblie Safari Moblie Safari
Mojolicious Msrabot
Mr.4x3 NearSite
Musobot Net Vampire
NICErsPRO NetLyzer
Nameprotect
Nessus
NetMechanic
Netcraft Netcraft
Nibbler NextGenSearchBot
Niki-bot
NimbleCrawler
Nimbostratus Nimbostratus
Ninja
Nutch Nutch
Octopus
Offline Explorer Offline Explorer
OrangeBot OrangeBot
OutclicksBot OutclicksBot
PECL::HTTP OutfoxBot
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber PageGrabber
PageThing.com PageThing.com
Pandalytics
Panscient
Pavuk Pavuk
Petalbot Petalbot
Pi-Monster
Picsearch
Piepmatz Piepmatz
Pixray Pimonster
PleaseCrawl
ProPowerBot
Probethenet
Psbot Psbot
Pu_iN Pu_iN
PxBroker Quick-Crawler
RankActive
RankActiveLinkBot RankActiveLinkBot
RankurBot RankFlex
RankingBot2
Rankivabot
ReGet ReGet
RebelMouse RealDownload
Recorder
RedesScrapy RedesScrapy
RocketCrawler RocketCrawler
SEOstats SBIder
SalesIntelligent SEOkicks
SEOprofiler
SISTRIX
ScanAlert
Screaming Screaming
ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot SeobilityBot
SeoSiteCheckup
Shodan
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
SiteSnagger SiteSnagger
Sitebeam
Siteimprove
Sitevigil Sitevigil
SmartDownload SlySearch
Snake
Snapbot Snapbot
Spbot SpankBot
Spinn3r Spinn3r
SputnikBot SputnikBot
Sqlmap Sqlworm
Stripper Sucker
SuperHTTP Sucuri
SuperBot
Surfbot
SurveyBot SurveyBot
Suzuran
Swiftbot Swiftbot
Szukacz T8Abot
T0PHackTeam
Teleport Teleport
Telesphoreo The Intraformant
Thumbor
TightTwatBot TightTwatBot
Toata Toata
True_Robot Toweyabot
Tracemyfile
Trendiction
Turingos Turingos
TurnitinBot TurnitinBot
Twice TwengaBot
URLy Warning Typhoeus
UnisterBot UnisterBot
V-BOT V-BOT
VB Project VCI
Vacuum Vacuum
VelenPublicWebCrawler VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
VoidEYE
Voil
WASALive-Bot
WEBDAV WEBDAV
WISENutbot WWW::Mechanize
WallpapersHD Wallpapers
Web Auto Web Enhancer
Web Collage WebAuto
Web Fuck
Web Pix
Web Sucker
WebBandit WebBandit
WebCollage WebCollage
WebGo IS WebCopier
WebEnhancer
WebFetch
WebLeacher WebLeacher
WebPix WebsiteExtractor
Webalta WebsiteQuester
Website Quester
Whack
Whacker Whacker
Whatweb Whatweb
Who.is Bot Who.is Bot
WinHTTrack Widow
Xaldon WebSpider Wotbox
YoudaoBot Zade
Zermelo Zauba
Zeus ZmEu
ZoomBot arquivo.pt
ZumBot
archive.org_bot
backlink-check backlink-check
cah.io.community
check1.exe check1.exe
cognitiveseo
com.plumanalytics
crawler4j crawler4j
demandbase-bot gopher
facebookscraper
heritrix heritrix
instabid
internetVista monitor
ips-agent
isitwp.com isitwp.com
magpie-crawler muhstik-scan
mediawords page scorer
netEstate NE Crawler
pcBrowser pcBrowser
probe-image-size polaris version
s1z.ru s1z.ru
satoristudio.net scalaj-http
scan.lol
seobility
seoscanners seoscanners
serpstatbot sitechecker.pro
sogouspider sysscan
sp_auditbot voyagerx.com
trendiction.com zgrab
webmeup-crawler
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
energydiet-info.ru cornomase.win
everypony.ru erotik-kostenlos.net
ezofest.sk investzalog.ru
gelezki.com jerseysbizwholesalecheap.com
kiinomaniak.pl medtherapy.ru
megakino.net poiskzakona.ru
officedocuments.net speedup-my.site
rockma.se trumpetedextremes.com
space-worry.ml umekana.ru
vertaform.com videosbox.ru

930
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0lovespells0.blogspot.com
1001watch.com.ua 1001watch.com.ua
100searchengines.com 18ps.ru
101billion.com
1hwy.com 1hwy.com
2020iscoming.info 2daytrendingnews.com
202ch.com 4kepics.com
3w24.com 4pp13.com
4kplayer.pl 5000-cotydzien.com
883zy.com 5i2.net
8lufu.com 70casino.online
9icmzvn6.website 8xv8.com
a.pr-cy.ru aa9bd78f328a6a41279d0fad0a88df1901.com
accessoires-mode.in aaa38852e886ac4af1a3cff9b47cab6272.com
adsfresh.men aaronabel.com
adultnet.in absugars.com
advancedmassagebysara.com addbin.men
advokateg.ru addblueoff.com.ua
admitad.com
ads-cool.pro
ads-seo.men
adultfriendfinder.com
adultmeetonline.info
afmuseum.com afmuseum.com
air-edem.ru ageofclones.com
alert.scansafe.net ahmedabadwebs.com
alienwheels.de akusherok.ru
allesohnegirls.net alekseevec.ru
allsilver925.co.il alert-fdm.xyz
allcredits.su
allnews.md
allpdfmags.net
allstatesugarbowl.org allstatesugarbowl.org
amazingpic.net amateurmatch.com
ananumous.ru
anaseracresar.tk anaseracresar.tk
android-systems.ru angel1777.kz
android-vsem.org
angkortours.vn angkortours.vn
animal-drawings.com angry-fermi-7633.arukascloud.io
aosheng-tech.com ap.senai.br
apartmentratings.com apifasterlightin-a.akamaihd.net
apidigidocketnet-a.akamaihd.net apparel-offer.com
apiportalmorecom-a.akamaihd.net appsecurityr.com
appearance-cool.com arabsexxxtube.com
artavenuegardenstudios.com arclk.net
arvut.org
asiavirtualsolutions.com
asociatia-tipografilor-transilvania.ro asociatia-tipografilor-transilvania.ro
asrv-a.akamoihd.net asscenihotosidea.blogspot.co.za
avon-ukraine.com australia-opening-times.com
ayakino.net autogrep.ru
azartclub.org autotrends.today
babespcs.com autovideobroadcast.com
badavit.com.ua avitocars.ru
baikaleminer.com avto-oligarh.ru
bambi.ck.ua ayanaspa.com
bang-hotties.com backlink4u.net
beremenyashka.com basedecor.ru
best-businessman.ru battlecarnival.su
belsetirehafi.tk
best-deals-products.com
best-mam.ru best-mam.ru
best-seo-solution.com
besthoro.ru besthoro.ru
bestmarriages.com bestserials.com
betterscooter.com bestwaystogetpaid.us
bigcareer.info bestwebsiteawards.com
bio-japan.net betteroffers.review
bioinnovate.co bfz.biz
boo-arts.com bidbuy.co.kr
boole.onlinewebshop.net biglistofwebsites.com
bouda.kvalitne.cz billiard-classic.com.ua
brainboosting.club bioca.org
braintobe.win biocn.dx.am
brakehawk.com bioscorp.ru
buigas.00it.com bizcheapjerseyswholesalechina.com
business-made-fun.com bjanshee.ru
buttons-for-website.com blackle.com
buttons-for-your-website.com blagovest-med.ru
buy-cheap-pills-order-online.com blobar.org
cakesplus.com.au blockworld.ru
callejondelpozo.es blogsfunky672.weebly.com
canrole.com bluesalt.co
bluesman.nu
bonux.nextview.ru
brg8.com
btvn.ru
bukleteg.xyz
buyessayonline19.blogspot.ru
buzz-porno.info
callmd5map.com
canoncdriverq3.pen.io
canvas.gloverid.site canvas.gloverid.site
carapics.com cartierbracelet.xyz
cdncash.org ccjp.eu
cheatcode-lita12.rhcloud.com cenoval.ru
chlooe.com centrdebut.ru
christianlouboutinoutlet.win cheapsergiorossi.online
clean-virus-mac.com chocolateslim-en-espana.com
colobit.biz christianlouboutinreplica.pw
comeondog.info clarithromycin500mg.com
compliance-mark.top clickbanksites.info
contentlook.co clickprotects.com
cool-wedding.net cognitiveseo.com
copenergo.ru collegeessay19.blogspot.ru
coprofam.org com-onlinesupport.host
cosmetic.donna7753191.ru cometorussia.net
cowlmash.com communisave.co.za
craftburg.ru compliance-brian.top
creatives.adbetclickin.pink compliance-julia.top
crynet.cc consorzioilmosaico.org
cscwtalkto.site cougfan.info
customcollegeessays.net counterbot.com
cv.wallhade.co country-chic.ru
darkages.info covid-schutzmasken.de
db.speedup.gdn cpajunkies.com
decorationspcs.com crackguru.tk
deniven.1bb.ru creams.makeforum.eu
depositfiles-porn.ga cupday.com
descargar-musica-gratis.net datract.com
deutschehobbyhuren.net de.zapmeta.com
dev-seo.blog deadlinkchecker.com
dikqlyremy.info delta-line.men
dispo.de demian.kz
djekxa.ru detsky-nabytek.info
dlya-android.org devochki.top
dnepr-avtospar.com.ua dinosaurus.site
dnsrsearch.com disruptingdinnerparties.com
docs4all.com divatraffic.com
docstoc.com doctissimo.fr
dorratex.tn dogclothing.store
downloads-whatsapp.com dogsrun.net
downtuptv.gq domain.webkeyit.com
dstroy.su domznaniy.ru
dumpsmania24.com doska-vsem.ru
dyshagi.ru dotnetdotcom.org
earncash.com.ua doxysexy.com
easycommerce.cf drpornogratisx.xxx
ecommerce-seo.com druzhbany.ru
editmedios.com ebonyporn.site
elektir.ru ecookna.com.ua
elektronischezigarettekaufen1.myblog.de edelstahlschornstein-123.de
eloconcream.blogspot.com efkt.jp
elvenar.com ekspertmed.com
enskedesquashclub.se engines-usa.com
enthuse.computernetworksonline.com eorogo.top
eredijovon.com erotag.com
erotik-kostenlos.net es5.com
este-line.com.ua eshop.md
eu-cookie-law.blogspot.com espaceinventoristes.com
eupornstar.info etur.ru
europeanwatches.ru eu-cookie-law.info
everytalk.tv ewebarticle.info
exportshop.us exoneration-project.us
expresstoplivo.ru expolicenciaslatam.co
extlabs.io
extremez.net
eyessurgery.ru eyessurgery.ru
ezigarettekaufen2.mpbloggar.se
ezigarettenshop1.yolasite.com ezigarettenshop1.yolasite.com
ezigaretteshop2.mywapblog.com f1nder.org
f-online.de fanpagerobot.com
favorcosmetics.com faptitans.com
fetishinside.com fashionavenuegame.com
filesdatabase.net fba-mexico.com
filunika.com.ru feorina.ru
findpik.com ferieboligkbh.dk
fingerprintjs.com fickenbumsen.net
firesub.pl fickluder69.com
forexunion.net filmci.pro
formularz-konkurs.tk finejewelryshop.ru
forodvd.com flauntyoursite.com
foto-telok.net folowsite.com
fotopop.club formulaf1results.blogspot.com
fotosfotos.eu forum.doctissimo.fr
free-share-buttons.blogspot.com foto-weinberger.at
freelotto.com foxtechfpv.com
freemags.cc fres-news.com
frustrated-favorable.gq freshberry.com.ua
frvo.alptandem.ru froggytube.com
fuckbuddybestgilf.info fuck-paid-share-buttons.xyz
gamedayhouse.com fuckingawesome.com
gavuer.ru fungirlsgames.net
gay.adultgalls.com g.starmoe.xyz
gdebestkupit.ru galaxyflowers.ru
gearsadspromo.club gamerscorps.com
generalporn.org ge0ip.com
getlamborghini.ga getprismatic.com
gidonline.one gheus.altervista.org
gilbertbanda.net girls-ufa.ru
glicol.kz glicol.kz
gogps.me globatur.ru
gok-kasten.net goblacked.com
golfresa.lucania.se goldenggames.com
goodhousekeeping.com gombita.info
googlsucks.com gomusix.com
gorodservis.ru gonextmedia.com
gosmeb.ru goodnightjournal.com
gotcher.us googglet.com
gotomontenegro.net grand-chlen.ru
gymi.name gratuitbaise.com
hayate.biz gratuitxblcodes.com
hdhc.ru gsasearchenginerankerhelp.com
headpharmacy.com guidefs.ru
healgastro.com hauleddes.com
health-medical-portal.info hchha.com
hebr.myddns-flir.com hdfreeporno.net
high-speed1.net healbio.ru
hikesearch.net healmytrauma.info
hkdiiohi.skyrock.com hem.passagen.se
homedo.fabpage.com highland-homes.com
homemade.gq homeinns.com
honyaku.yahoofs.jp hoste.octopis.com
hostcritique.com hostingclub.lk
hundejo.com hotblog.top
hydropump.su hotelcrocenzi.sm
ico.re huimin764128.com
humanelydrew.com
hybrid.ru
idealtits.net
ie.57883.net
iflycapetown.co.za
iframe-toloka.com
igtools.club igtools.club
images-free.net iklysha.ml
imgata.com images-graphics-pics.com
iminent.com immobiliaremassaro.com
imk.com.ua intim-uslugi.info
infobanks.ru irunfar.com
inmate-locator.us iscblog.info
it-max.com.ua itis4you.com
itronics.ca iwanttodeliver.com
izhevsk.zrus.org izoll.ru
jagg.info jacago.com
javidol.site jetsli.de
jerseychinabizwholesale.us johannesburgsingles.co.za
kamen-e.ru joy-penguin.com
kazka.ru justkillingti.me
kicknights.gq jwss.cc
kinoduh.ru kaliningrad.zrus.org
kinostorm.net kandidos.com
kiwi237au.tk kinohit1.ru
knogg.net kinosed.net
konteiner24.com krassh.ru
kumuk.info kreativperlen.ch
kutikomi.net kuzinsp.ru
lacloop.info kw21.org
lanasshop.ru labvis.host.sk
landoftracking.com ladsup.com
lankarns.com larutti.ru
lapitec.eu law-check-nine.xyz
larchik.net lazy-z.com
leadwayau.com ldrtrack.com
leftofcentrist.com le-clos-des-alouettes.com
li-er.ru lemon-ade.site
lifehacĸer.com lesbian.xyz
liky.co.ua light.ifmo.ru
lion.bolegapakistan.com likesuccess.com
lion.conboy.us likrot.com
loginduepunti.it linkbolic.com
lost-alpha.ru liquimondo.com
lutherstable.org listiseltemournlan.gq
m4ever.net livefixer.com
machicon-akihabara.info liver-chintai.org
magnetic-bracelets.ru livesex-amateure.info
mainlinehobby.net liveu.infoteka.hu
lmrauction.com
loftdigital.eu
lomb.co
lovelycraftyhome.com
lulea-auktionsverk.se
luxmagazine.cf
m-google.xyz
m1media.net
mackeeper-land-672695126.us-east-1.elb.amazonaws.com
mail.allnews24.in
makedo.ru
makemoneyonline.com
makenahartlin.com makenahartlin.com
malwareremovalcenter.com mall.uk
manifestyourmillion.com marmotstore.online
marketingvici.com maximilitary.ru
maxxtor.eu medicovi.com
mcnamaratech.com
medispainstitute.com.au medispainstitute.com.au
medizinreisen.de member-quiz.com
mensandals.xyz mesbuta.info
migente.com mesto-x.com
mikrobiologies.ru metascephe.com
milblueprint.com mex-annushka.ru
mini.7zap.com midst.eu
mir-business-24.ru militarysale.pro
misswell.net miniature.io
mixx.com mlf.hordo.win
mnogabukaff.net mnogabukaff.net
mobplayer.ru mobplayer.net
monclerjacketsoutlet.win moesen-ficken.com
moneyzzz.ru moneyteam24.com
morlat.altervista.org montazhnic.ru
moviezbonkerssk.cf morocco-nomad-excursions.com
mp3ringtone.info moroccosurfadventures.com
mrbojikobi4.biz mp3s.club
mriyadh.com musezone.ru
mulberryoutletonlineeu.com musicas.baixar-musicas-gratis.com
music7s.me mycouponizemac.com
muz-shoes.ru myfreecams.com
muzaporn.com myprintscreen.com
myblogregistercm.tk na15.ru
mymobilemoneypages.com nagdak.ru
nacap.ru naperehresti.info
naj-filmy24.pl nastroyke.net
napalm51.nut.cc natali-forex.com
nbsproject.ru nbsproject.ru
new-post.tk net-profits.xyz
nicovideo.jp notaria-desalas.com
njkmznnb.ru npoet.ru
nl.netlog.com nrjmobile.fr
nodding-passion.tk nw-servis.ru
nofreezingmac.work oakleyglassesonline.us
noumeda.com oballergiya.ru
novodigs.com oda.as
nudepatch.net offf.info
odesproperty.com okmedia.sk
offer.camp oktube.ru
office-windows.ru
offside2.5v.pl
oivcvx.website
olvanto.ru olvanto.ru
one.net.in online-zaymy.ru
online-hit.info onlinemeetingnow.com
ontargetseo.us onlythegames.com
ooo-gotovie.ru openfrost.net
orgasmatrix.com openstat.com
pateaswing.com outshop.ru
pay2me.pl paceform.com
paydayloanslocal.com panouri-solare-acoperis.com
pcimforum.com paperwritingservice17.blogspot.ru
parajumpersjakkesalgnorge.info
paramountmarble.co.uk
partnersafe.men
pathwhelp.org
pc-virus-d0l92j2.pw
pdns.cz
perfectpracticeweb.com perfectpracticeweb.com
perm.xrus.org pestomou.info
photo-clip.ru petedrummond.com
piccshare.com php-market.ru
pinapchik.com pierrehardysale.online
piratecams.com piluli.info
pizdeishn.com pixgood.com
pizza-imperia.com pl.aasoldes.fr
playlott.com
porno-chaman.info
pornobrazzers.biz
pornodojd.ru
pornofiljmi.com
pornoslive.net
pourvous.info pourvous.info
pozdravleniya-c.ru pr-ten.de
pr0fit-b0x.com
praisong.net praisong.net
predmety.in.ua pravoholding.ru
pricheskaonline.ru preparevideosafesystem4unow.site
privacyassistant.net
privacylocationforloc.com privacylocationforloc.com
privatov-zapisi.ru prizeestates.cricket
privetsochi.ru pro-poly.ru
projectforte.ru proekt-gaz.ru
projefrio.com.br proekt-mos.ru
prombudpostach.com.ua profolan.pl
pron.pro prohoster.info
prostitutki-tyumeni.org pssucai.info
prpops.com pushdata.sendpulse.com
push-ad.com putitin.me
questionmarque.ch pyrodesigns.com.au
quickchange.cc qor360.com
quickloanbank.com ranksonic.com
r-e-f-e-r-e-r.com recipedays.com
rainbowice.ru registry-clean-up.net
razleton.com ric.info
reelheroes.net riralmolamsaca.tk
reeyanaturopathy.com rniaeba.ga
regdefense.com rocketchange.ru
reklama1.ru rollercoin.com
remont-ustanovka-tehniki.ru romhacking.ru
remontvsamare.su royal-investments.net
renecaovillasale.online ruscopybook.com
rfserial.net russintv.fr
riders.ro russkie-gorki.ru
rukino.org rustic-quiver.win
ruscoininvest.company sabaapress.com
rusoft-zone.ru saratov.xrus.org
sabizonline.com scanner-alex.top
sampleletters.net schoolfiles.net
savingsslider-a.akamaihd.net scopich.com
sawin.beth.webd.pl
scanner-andrew.top
scanner-jess.top
school-diplomat.ru
score-ads.men score-ads.men
screentoolkit.com searchtooknow-a.akamaihd.net
sdi-pme.com seasaltwithfood.com
sdsjweb.com seasonvar.ru
search-goo.com sei80.com
searchmywindow-a.akamaihd.net
secret.xn--oogle-wmc.com
security60-e.com
segol.tv
seinterface.com seinterface.com
seo18.su seo-tools-optimizing.com
seocheki.net
seogadget.ru seogadget.ru
seorank.info seojokes.net
serdcenebolit.com seokicks.de
servisural.ru sexuria.net
sex-sex-sex5.com sexystrippe.info
sexobzor.info shisha-swag.de
sexreliz.com shitting.pro
sexs-foto.com shoppingmiracles.co.uk
sextracker.de signoredom.com
sexy.babes.frontend-stack.top signx.info
shiksabd.com similardeals.net
shopsellcardsdumps.com
shtaketniki.kz
sideeffectsoftizanidine.blogspot.com
site-auditor.online site-auditor.online
small-game.com siteaero.com
smichovbike.cz siteimprove.com
soc-econom-problems.ru sixcooler.de
social-fun.ru snegozaderzhatel.ru
social-search.me social-buttons.com
socialmediasuggest.com sockshare.net
softtor.com sofit-dmd.ru
softwaretrend.net soheavyblog.com
sohoindia.net spammen.de
sophang8.com spamnuker.com
soserfis.com steebook.com
soundfrost.org stickers-market.ru
spbchampionat.ru strana-solnca.ru
specialfinanceoffers.com streetfooduncovered.com
speechfoodie.com strfls.com
ssconstruction.co stroilka.info
star61.de
steelmaster.lv
stockquotes.wooeb.com
strag-invest.ru
suggest-keywords.com suggest-keywords.com
supernew.org superlist.biz
supers.com.ua surcentro.com
svetlotorg.ru surflinksmedical.com
takeprofitsystem.com surgut.zrus.org
tampabaywatch.org svarbit.com
taranerymagesswa.blogspot.com symphonyintegratedhealthcare.com
teenbbw.yopoint.in taaaak.com
templates.franklinfire.co taraz.xkaz.org
thecounter.com tattomedia.com
thedownloadfreeonlinegames.blogspot.com tattooreligion.ru
thetardistimes.ovh tds-advert005.info
teastory.co
technika-remont.ru
telegraf.by
telsis.com
tengohydar.tk
terraclicks.com
tetracsaudi.com
texbaza.by
thelottosecrets.com
tiens2010.ru tiens2010.ru
tizanidinecost.blogspot.com timetorelax.biz
tizanidinemusclerelaxant.blogspot.com tizanidineingredients.blogspot.com
tizanidinepills.blogspot.com tizanidinewithdrawalsymptoms.blogspot.com
tizanidinesideeffects.blogspot.com tonerbox.kz
todohr.com torture.ml
tootoo.to trafficfactory.biz
top-karkas.ru trahic.ru
topkarkas.com trkur.com
torrent-to-magnet.com truck-land.life
torrents-tracker.com tuberkulezanet.ru
torrnada.ru twiclub.in
toursmaps.com twitlinks.com
track.deriv.com valuado.com
trainoffend.ml venta-prom.ru
trubywriting.com video--production.com
trucri.me videochat.world
trustedmaccleaner.com videotuber.ru
tryrating.com virus-respirators.com
tvnewsclips.info visa-china.ru
uhod-za-sobakoj.ru
ukr-lit.com
uptime-as.net
upupa.net
urlopener.com
userequip.com
vacuumcleanerguru.com
vashsvet.com
vchulkax.com
veopornogratis.xxx
viagengrarx.com
viberdownload10.com
videochat.mx
videositename.com
videtubs.pl
vigrx-original.ru
vinsit.ru
vipsiterip.org
visitcambridge.org
viven.host.sk
vkmusics.ru
vladhistory.com
vseuznaem.com vseuznaem.com
vzubah.com vykupavto-krasnodar.ru
walpaperlist.com vzlom-na-zakaz.com
wardreapptokone.tk vzlomtw.com
webinstantservice.com w7s.ru
weblibrary.win walkme.com
watchinf.com
web-analytics.date
webadvance.club
webix.me
webjam.com
website-analytics.online website-analytics.online
website-datenbank.de whengirlsgowild.com
website-speed-check.site wonderfulflowers.biz
websiteaccountant.de
webtherapy.ru
wellcome2slovenia.ru
whatzmyip.net
wildnatureimages.com
winx-play.ru
wjgony.com
wnoz.de
woman-orgasm.ru
woodyguthrie.se
wordpresscore.com
worldis.me worldis.me
worst-sites.online wrona.it
writingservices17.blogspot.ru wtsindia.in
www.event-tracking.com xclicks.net
www.wohnkabinen-shop.de xerox-douglas.cf
x-diesel.biz xingzi-vision.com
x5market.ru xitjw.info
xboxster.ru xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xcc24.pl xn----itbeirbjbi7bc6bh2d.xn--p1ai
xgames-04.com xn--80ahvj9e.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai
xn--80abgj3a5acid6ghs.top
xn--b1agm2d.net xn--b1agm2d.net
xxlargepop.com xtraffic.plus
xxxmania.top xxxnatelefon.ru
youdesigner.kz youbloodyripper.com
youporn-forum.ga youradulthosting.com
yourothersite.com yousense.info
yun56.co yun56.co
zaapplesales.blogspot.com
zajm-zalog-krasnodar.ru zajm-zalog-krasnodar.ru
zebradudka.com zaobao.com.sg
zeroredirect8.com zarajbuilders.com
zigarettenonl.canalblog.com zarepta.com
zootoplist.com zed21.net
zootravel.com zeikopay.com
zrus.org zeleznobeton.ru
zs2vm.top zeroredirect5.com
zeroredirect9.com
znakomstvaonlain.ru
zreloeporno.tv
zvetki.ru
zzlgxh.com

472
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 00author.com
3w24.com 03p.info
4pp13.com 18ps.ru
5000-cotydzien.com 24x7-server-support.site
51unlim.ru 4senses.co
5tey7463.weebly.com 5-steps-to-start-business.com
66cpwgln.space aarbur.com
72-news.com acgs.tk
aac500b7a15b2646968f6bd8c6305869d7.com addbin.men
advocatemsk.ru adhome.biz
advokateg.xyz adserver-e7.com
aibolita.com adultfullhd.com
arquapetrarca.info afora.ru
autovideobroadcast.com agadelha.com.br
avkzarabotok.info akuhni.by
azbuka-mo.ru alborzan.com
bashtime.ru alert-fjg.xyz
beauby.ru alice.it
belinka.com.ua alldezservice.kz
best-seo-software.xyz allwidewallpapers.com
bestwebsitesawards.com android-style.com
betune.onlinewebshop.net arcadeplayhouse.com
bif-ru.info arclk.net
bigames.online atelielembrancaqueencanta.com.br
bitcoins-live.ru autorn.ru
bloggedporn.com b00kmarks.com
bookhome.info banki76.ru
brandehk.dk bausparen.bz.it
brendbutik.ru bdsmgalls.net
brokergid.ru beachdriveblog.com
brothers-smaller.ru beauty-clinic.ru
buy-meds24.com bimatoprost-careprost.com.ua
candcstuccoandstone.com bioinnovate.co
capsderfudd.tk biznesluxe.ru
cartechnic.ru brainboostingsupplements.org
cdncash.org brains2.biz
cejewelry.xyz buckspar.gq
cete.ru business-made-fun.com
cfjump.com camdenmemorials.com
cheapcigarettesc.info canyougethighofftizanidine.blogspot.com
checkm8.com cartierjusteunclou.xyz
collegerentals.com check-this-out-now.online
conciergegroup.org chocolateslim-u-srbiji.com
coolgramgoods.com ciceros.co
crirussian.ru classicakuhni.ru
darodar.com compliance-andrew.top
dating-app.ru compliance-checker-7.info
davebestdeals.com compliance-fred.top
diegolopezcastan.com compliance-jessica.top
digital-sale.su contextualyield.com
dignitasdata.se corta.co
dispo.de cowblog.fr
dmmspy.com craftinsta.ru
dms-sw.ru creams.makeforum.eu
dood.live cymbaltaandtizanidine.blogspot.com
doyouknowtheword-flummox.ml cyse.tk
drugstoreforyou.com d2jsp.org
easyukraine.com darkages.info
ecommerce-seo.org deadlinkchecker.com
ege-russian.ru demian.kz
elektrischezigarettekaufen2.cowblog.fr dentalpearls.com.au
elite-sex-finders.com deshevo-nedorogo.ru
esopini.com dikqlyremy.info
everytalk.tv dinosaurus.site
documentsite.net
domination.ml
doreenblog.online
download-of-the-warez.blogspot.com
doxysexy.com
drhomes.biz
drugs-no-rx.info
educontest.net
elektronischezigarettekaufen2.tumblr.com
elektrozigarette1.dreamwidth.org
elmifarhangi.com
espaceinventoristes.com
evaashop.ru
extlinks.com extlinks.com
factorynightclub.com
faithe.top
fashion-mk.net fashion-mk.net
ferrotodo.com faz99.com
forexgb.ru fertilitetsradgivningen.se
foxydeal.com filerockstar298.weebly.com
freenom.link filesclub.net
freshberry.com.ua find1friend.com
g33.org finstroy.net
galeria-zdjec.com fishingwholesale.us
gameprimary.com forminecrafters.ru
gne8.com formulaantiuban.com
gojiberriess.apishops.ru foto-telok.net
google-liar.ru froggytube.com
fullfileaccess.com
fyl.com.ru
gay.adultgalls.com
ghazel.ru
ghernnqr.skyrock.com
gomusix.com
googlsucks.com
greenidesign.co
growmyfunds.ca growmyfunds.ca
gwebtools.com.br gsmlab.pl
helvetia.com.ua hamptonoaks.ca
hentaiheroes.com historichometeam.com
highspeed5.net hotblognetwork.com
horoshieokna.com hyhj.info
hoste.octopis.com ido3.com
hotcore.info igru-xbox.net
hptwaakw.blog.fc2.com ikritikimou.gr
hundejo.com in-tandem.co
hvd-store.com kam-dom.ru
hyiphunter.org keywordhouse.com
i-hobot.ru
imadedinner.net
indianmedicaltourismshop.com
inspiring-desperate.tk
integritylandscapeservices.com
investmac.com
jerseychinabizwholesale.us
jimmychoosale.online
journeydownthescale.info
joy-penguin.com
kamalsinha.com
karpun-iris.ru
karusel-market.ru
kiinomaniak.pl kiinomaniak.pl
klejonka.info kinobaks.com
koptims.tiu.ru la-fa.ru
kredit-blog.ru lacave.ntic.fr
laapp.com landreferat.ru
labvis.host.sk larchik.net
lafriore.ru larose.jb2c.me
languagecode.com lawyersinfo.org
larutti.ru ledis.top
law-check-nine.xyz
lechenie-gemorroya.com
legionalpha.com
lenvred.org
lflash.ru lflash.ru
lion.bolegapakistan.com linkarena.com
loadopia.com lumb.co
longgreen.info lunamedia.co
louisvuittonoutletstore.net
m-google.xyz m-google.xyz
malls.com max-eclat.men
marcogrup.com meddesk.ru
marketingtechniques.info
maxxtor.eu
meble-bogart.info
megaapteka.ru megaapteka.ru
meltwater.com metallosajding.ru
menhealed.net metarip.ru
michaelkorsoutlet.store mex-annushka.ru
misandesign.se michaelkorsoutletstore.net
monthlywinners.com miniads.ca
mriyadh.com mitrasound.ru
mydearest.co mnimmigrantrights.net
myplaycity.com montazhnic.ru
mysex21.com most.gov.iq
nash-krym.info my-aladin.com
nbsproject.ru mydoctorok.ru
newstudio.tv myfreemp3.eu
offer.camp myonigroup.com
oktube.ru netallergy.ru
onlainbesplatno.ru nordstar.pro
northfacestore.online
nowtorrents.com
nudejapan.net
nzfilecloud.weebly.com
oneshotdate.com
online-hit.info
online7777.com
onlineporno.site
onlinewritingjobs17.blogspot.ru onlinewritingjobs17.blogspot.ru
onlyforemont.ru ooomeru.ru
ooo-gotovie.ru oynat.info
ooo-olni.ru pacificair.com
outshop.ru partner-stop.men
ownshop.cf picture-group.com
paceform.com pictures-and-images.com
palocco.it picturesfrom.com
pdns.cz piulatte.cz
perfectpracticeweb.com piuminiita.com
phormchina.com pixelrz.com
pixgood.com podshipniki-ntn.ru
pogruztehnik.ru popugaychiki.com
poisk-zakona.ru porno-transsexuals.ru
pony-business.com pornogad.com
pornofeuer.com pornosee.info
pornoklad.net pornowarp.info
portadd.men pourvous.info
pozvonim.com professionaldieselcare.com
preparevideosafesystem4unow.site profitkode.com
pro-poly.ru proxyradar.com
profitsport.club puteshestvennik.com
projefrio.com.br realting-moscow.ru
prostitutki-almata.org registry-clean-up.net
putitin.me registry-cleaner.net
q-moto.ru reining.lovasszovetseg.hu
rangjued.com
ranksonic.com
razyboard.com
realitykings.com
redirect.trafficreceiver.club
reward-survey.net reward-survey.net
rfd-split.hr
rockingclicks.com rockingclicks.com
rustic-quiver.win rostov.xrus.org
saecsa.co royalads.net
sandhillsonline.com royalcar-ufa.ru
saneyes.com ruclicks.com
sanyuprojects.com santechnik.jimdo.com
savetubevideo.com sax-sex.com
scansafe.net scanner-jack.top
sdelai-prosto.ru scanner-margo.top
seadragonherbery.com scat.porn
seeresultshub-a.akamaihd.net search-goo.com
sendearnings.com securesmrt-dt.com
sexsaoy.com serdcenebolit.com
sierraapps.com serptehnika.ru
skachat-besplatno-obrazcy.ru service.adtech.fr
slujbauborki.ru sideeffectsoftizanidine.blogspot.com
smartshoppymac.com simplesite.com
snsdeainavi.info siteheart.net
siteripz.net
snjack.info
soblaznu.net soblaznu.net
socas.pluto.ro socas.pluto.ro
socialmonkee.com sockshare.net
sodexo.com softwaretrend.net
sohoindia.net sovinsteel.ru
soundfrost.org sstroy44.ru
soviet-portal.do.am stanthonyscatholicchurch.org
spammen.de surgut.zrus.org
spywarebegone.com svetodiodoff.ru
stackthatbucks.com taraz.xkaz.org
start.myplaycity.com templates.franklinfire.co
stopnarco.ru tgtclick.com
stroyhelp-dv.ru theheroes.ru
studentguide.ru thetoiletpaper.com
sukirgenk.dvrlists.com tiens2010.ru
supermodni.com.ua tizanidineoral.blogspot.com
takeprofitsystem.com tkanorganizma.ru
talant-factory.ru touchmods.fr
taqywu51.soup.io
teenforporn.com
terrafootwear.us
thenetinfo.com
timetorelax.biz
tksn.ru
topmira.com
torrents.life
trk-4.net trk-4.net
trucri.me trucri.me
tula.mdverey.ru ufa.xrus.org
twiclub.in uhodzalijami.ru
uhdtv.website unimodemhalfduplefw.pen.io
userequip.com universals.com.ua
validdomain.xyz vandrie-ict.nl
velen.io vbikse.com
viagra.pp.ua vellings.info
viktoria-center.ru viagra-soft.ru
vipms.ru viandpet.com
vita.com.hr viel.su
voloslove.ru vilingstore.net
voprosotvet24.ru vsexkontakte.net
vykupavto-krasnodar.ru weburlopener.com
vzglyadriv.kg wesharepics.site
vzubah.com white-truck.life
w-journal.ru wurr.voila.net
webads.co.nz x-diesel.info
webpromotion.ae x-lime.com
word-vorlagen.net xkaz.org
xdoza.com xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xmronta.com xn----7sbahjd3btneuw1joc.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn--80aanaardaperhcem4a6i.com
xn--80aeahghtf8ac5i.xn--p1ai xn--80ajbshivpvn2i.xn--p1ai
xn--b1agm2d.net xn--e1aggki3c.xn--80adxhks
xn--l1aengat.xn--p1ai xsfetish.org
xxxtubesafari.com xvideospornoru.com
zhacker.net xxxhdvideo.site
znakomstva-moskva77.ru youjizz.com
zaobao.com.sg
zero1.it
zootravel.com
zrus.org

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.228 0; 10.1.0.47 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:43 UTC 2021
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -195,6 +195,8 @@ User-agent: Cosmos
Disallow:/ Disallow:/
User-agent: Craftbot User-agent: Craftbot
Disallow:/ Disallow:/
User-agent: Crawling at Home Project
Disallow:/
User-agent: CrazyWebCrawler User-agent: CrazyWebCrawler
Disallow:/ Disallow:/
User-agent: Crescent User-agent: Crescent
@ -1135,6 +1137,8 @@ User-agent: autoemailspider
Disallow:/ Disallow:/
User-agent: backlink-check User-agent: backlink-check
Disallow:/ Disallow:/
User-agent: cah.io.community
Disallow:/
User-agent: check1.exe User-agent: check1.exe
Disallow:/ Disallow:/
User-agent: coccocbot-web User-agent: coccocbot-web

10
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2510 ### Version: V4.2021.08.2511
### Updated: Sun Aug 8 08:39:08 UTC 2021 ### Updated: Sun Aug 8 09:36:43 UTC 2021
### Bad Bot Count: 618 ### Bad Bot Count: 620
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -195,6 +195,8 @@ User-agent: Cosmos
Disallow:/ Disallow:/
User-agent: Craftbot User-agent: Craftbot
Disallow:/ Disallow:/
User-agent: Crawling at Home Project
Disallow:/
User-agent: CrazyWebCrawler User-agent: CrazyWebCrawler
Disallow:/ Disallow:/
User-agent: Crescent User-agent: Crescent
@ -1135,6 +1137,8 @@ User-agent: autoemailspider
Disallow:/ Disallow:/
User-agent: backlink-check User-agent: backlink-check
Disallow:/ Disallow:/
User-agent: cah.io.community
Disallow:/
User-agent: check1.exe User-agent: check1.exe
Disallow:/ Disallow:/
User-agent: coccocbot-web User-agent: coccocbot-web