vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.09.2570 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-09-04 12:04:50 +00:00
parent 8ae2d19013
commit bf8e557b91
58 changed files with 1148 additions and 1182 deletions
Download patch file Download diff file Expand all files Collapse all files

6
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2569 #### Version: V4.2021.09.2570
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

6
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2569 #### Version: V4.2021.09.2570
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

6
README.md
View file

@ -11,9 +11,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.09.2569 #### Version: V4.2021.09.2570
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

6
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.09.2569 #### Version: V4.2021.09.2570
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_google_analytics_ghost_spam/google-exclude-02.txt
View file

File diff suppressed because one or more lines are too long

1
_google_webmaster_disavow_links/google-disavow.txt
View file

@ -2501,7 +2501,6 @@ domain:gheus.altervista.org
domain:ghostvisitor.com domain:ghostvisitor.com
domain:gidonline.one domain:gidonline.one
domain:gifspics.com domain:gifspics.com
domain:gigablast.com
domain:gigapeta.com domain:gigapeta.com
domain:gigixo.com domain:gigixo.com
domain:gilbertbanda.net domain:gilbertbanda.net

10
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

10
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.09.2569 ### Version: V3.2021.09.2570
### Updated: Sat Sep 4 10:03:38 UTC 2021 ### Updated: Sat Sep 4 11:58:57 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

10
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

1
dev-tools/_robots_input/robots-input.txt
View file

@ -161,7 +161,6 @@ GermCrawler
GetRight GetRight
GetWeb GetWeb
Getintent Getintent
Gigablast
Gigabot Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It

10
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.09.2569 ### Version: V3.2021.09.2570
### Updated: Sat Sep 4 10:03:38 UTC 2021 ### Updated: Sat Sep 4 11:58:57 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 1; 10.1.0.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 0; 10.1.0.87 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.09.2569 ### Version: V3.2021.09.2570
### Updated: Sat Sep 4 10:03:38 UTC 2021 ### Updated: Sat Sep 4 11:58:57 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 1; 10.1.0.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 0; 10.1.0.87 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.09.2569 ### Version: V3.2021.09.2570
### Updated: Sat Sep 4 10:03:38 UTC 2021 ### Updated: Sat Sep 4 11:58:57 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 1; 10.1.0.87 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -222,7 +222,6 @@
"~*(?:\b)GetRight(?:\b|)" 0; "~*(?:\b)GetRight(?:\b|)" 0;
"~*(?:\b)GetWeb(?:\b|)" 0; "~*(?:\b)GetWeb(?:\b|)" 0;
"~*(?:\b)Getintent(?:\b|)" 0; "~*(?:\b)Getintent(?:\b|)" 0;
"~*(?:\b)Gigablast(?:\b|)" 0;
"~*(?:\b)Gigabot(?:\b|)" 0; "~*(?:\b)Gigabot(?:\b|)" 0;
"~*(?:\b)Go!Zilla(?:\b|)" 0; "~*(?:\b)Go!Zilla(?:\b|)" 0;
"~*(?:\b)Go-Ahead-Got-It(?:\b|)" 0; "~*(?:\b)Go-Ahead-Got-It(?:\b|)" 0;

1
dev-tools/beta_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -2548,7 +2548,6 @@
"~*(?:\b)ghostvisitor\.com(?:\b)" 0; "~*(?:\b)ghostvisitor\.com(?:\b)" 0;
"~*(?:\b)gidonline\.one(?:\b)" 0; "~*(?:\b)gidonline\.one(?:\b)" 0;
"~*(?:\b)gifspics\.com(?:\b)" 0; "~*(?:\b)gifspics\.com(?:\b)" 0;
"~*(?:\b)gigablast\.com(?:\b)" 0;
"~*(?:\b)gigapeta\.com(?:\b)" 0; "~*(?:\b)gigapeta\.com(?:\b)" 0;
"~*(?:\b)gigixo\.com(?:\b)" 0; "~*(?:\b)gigixo\.com(?:\b)" 0;
"~*(?:\b)gilbertbanda\.net(?:\b)" 0; "~*(?:\b)gilbertbanda\.net(?:\b)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 0; 10.1.0.87 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.09.2569 ### Version: V3.2021.09.2570
### Updated: Sat Sep 4 10:03:38 UTC 2021 ### Updated: Sat Sep 4 11:58:57 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2569 2570

10
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.09.2569 ### Version: V3.2021.09.2570
### Updated: Sat Sep 4 10:03:38 UTC 2021 ### Updated: Sat Sep 4 11:58:57 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

10
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

1
dev-tools/referrers-regex-format-whitelist-test.txt
View file

@ -2501,7 +2501,6 @@
"~*(?:\b)ghostvisitor\.com(?:\b)" 0; "~*(?:\b)ghostvisitor\.com(?:\b)" 0;
"~*(?:\b)gidonline\.one(?:\b)" 0; "~*(?:\b)gidonline\.one(?:\b)" 0;
"~*(?:\b)gifspics\.com(?:\b)" 0; "~*(?:\b)gifspics\.com(?:\b)" 0;
"~*(?:\b)gigablast\.com(?:\b)" 0;
"~*(?:\b)gigapeta\.com(?:\b)" 0; "~*(?:\b)gigapeta\.com(?:\b)" 0;
"~*(?:\b)gigixo\.com(?:\b)" 0; "~*(?:\b)gigixo\.com(?:\b)" 0;
"~*(?:\b)gilbertbanda\.net(?:\b)" 0; "~*(?:\b)gilbertbanda\.net(?:\b)" 0;

1
dev-tools/referrers-regex-format.txt
View file

@ -2501,7 +2501,6 @@
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.156.75.68 0; 20.94.228.102 0;

10
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 1; 10.1.0.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 0; 10.1.0.87 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 1; 10.1.0.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 0; 10.1.0.87 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 1; 10.1.0.87 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -222,7 +222,6 @@
"~*(?:\b)GetRight(?:\b|)" 0; "~*(?:\b)GetRight(?:\b|)" 0;
"~*(?:\b)GetWeb(?:\b|)" 0; "~*(?:\b)GetWeb(?:\b|)" 0;
"~*(?:\b)Getintent(?:\b|)" 0; "~*(?:\b)Getintent(?:\b|)" 0;
"~*(?:\b)Gigablast(?:\b|)" 0;
"~*(?:\b)Gigabot(?:\b|)" 0; "~*(?:\b)Gigabot(?:\b|)" 0;
"~*(?:\b)Go!Zilla(?:\b|)" 0; "~*(?:\b)Go!Zilla(?:\b|)" 0;
"~*(?:\b)Go-Ahead-Got-It(?:\b|)" 0; "~*(?:\b)Go-Ahead-Got-It(?:\b|)" 0;

1
dev-tools/test1_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -2548,7 +2548,6 @@
"~*(?:\b)ghostvisitor\.com(?:\b)" 0; "~*(?:\b)ghostvisitor\.com(?:\b)" 0;
"~*(?:\b)gidonline\.one(?:\b)" 0; "~*(?:\b)gidonline\.one(?:\b)" 0;
"~*(?:\b)gifspics\.com(?:\b)" 0; "~*(?:\b)gifspics\.com(?:\b)" 0;
"~*(?:\b)gigablast\.com(?:\b)" 0;
"~*(?:\b)gigapeta\.com(?:\b)" 0; "~*(?:\b)gigapeta\.com(?:\b)" 0;
"~*(?:\b)gigixo\.com(?:\b)" 0; "~*(?:\b)gigixo\.com(?:\b)" 0;
"~*(?:\b)gilbertbanda\.net(?:\b)" 0; "~*(?:\b)gilbertbanda\.net(?:\b)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 0; 10.1.0.87 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.156.75.68 0; 20.94.228.102 0;

10
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.156.75.68 0; 20.94.228.102 0;

10
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:37 UTC 2021 ### Updated: Sat Sep 4 11:58:56 UTC 2021
### Bad Referrer Count: 7079 ### Bad Referrer Count: 7078
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -310,7 +310,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)GetRight(?:\b)" 3; "~*(?:\b)GetRight(?:\b)" 3;
"~*(?:\b)GetWeb(?:\b)" 3; "~*(?:\b)GetWeb(?:\b)" 3;
"~*(?:\b)Getintent(?:\b)" 3; "~*(?:\b)Getintent(?:\b)" 3;
"~*(?:\b)Gigablast(?:\b)" 3;
"~*(?:\b)Gigabot(?:\b)" 3; "~*(?:\b)Gigabot(?:\b)" 3;
"~*(?:\b)Go!Zilla(?:\b)" 3; "~*(?:\b)Go!Zilla(?:\b)" 3;
"~*(?:\b)Go-Ahead-Got-It(?:\b)" 3; "~*(?:\b)Go-Ahead-Got-It(?:\b)" 3;
@ -3554,7 +3553,6 @@ map $http_referer $bad_referer {
"~*(?:\b)ghostvisitor\.com(?:\b)" 1; "~*(?:\b)ghostvisitor\.com(?:\b)" 1;
"~*(?:\b)gidonline\.one(?:\b)" 1; "~*(?:\b)gidonline\.one(?:\b)" 1;
"~*(?:\b)gifspics\.com(?:\b)" 1; "~*(?:\b)gifspics\.com(?:\b)" 1;
"~*(?:\b)gigablast\.com(?:\b)" 1;
"~*(?:\b)gigapeta\.com(?:\b)" 1; "~*(?:\b)gigapeta\.com(?:\b)" 1;
"~*(?:\b)gigixo\.com(?:\b)" 1; "~*(?:\b)gigixo\.com(?:\b)" 1;
"~*(?:\b)gilbertbanda\.net(?:\b)" 1; "~*(?:\b)gilbertbanda\.net(?:\b)" 1;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 1; 10.1.0.87 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -222,7 +222,6 @@
"~*(?:\b)GetRight(?:\b|)" 0; "~*(?:\b)GetRight(?:\b|)" 0;
"~*(?:\b)GetWeb(?:\b|)" 0; "~*(?:\b)GetWeb(?:\b|)" 0;
"~*(?:\b)Getintent(?:\b|)" 0; "~*(?:\b)Getintent(?:\b|)" 0;
"~*(?:\b)Gigablast(?:\b|)" 0;
"~*(?:\b)Gigabot(?:\b|)" 0; "~*(?:\b)Gigabot(?:\b|)" 0;
"~*(?:\b)Go!Zilla(?:\b|)" 0; "~*(?:\b)Go!Zilla(?:\b|)" 0;
"~*(?:\b)Go-Ahead-Got-It(?:\b|)" 0; "~*(?:\b)Go-Ahead-Got-It(?:\b|)" 0;

1
dev-tools/test_units/custom-bad-referrers.conf
View file

@ -2548,7 +2548,6 @@
"~*(?:\b)ghostvisitor\.com(?:\b)" 0; "~*(?:\b)ghostvisitor\.com(?:\b)" 0;
"~*(?:\b)gidonline\.one(?:\b)" 0; "~*(?:\b)gidonline\.one(?:\b)" 0;
"~*(?:\b)gifspics\.com(?:\b)" 0; "~*(?:\b)gifspics\.com(?:\b)" 0;
"~*(?:\b)gigablast\.com(?:\b)" 0;
"~*(?:\b)gigapeta\.com(?:\b)" 0; "~*(?:\b)gigapeta\.com(?:\b)" 0;
"~*(?:\b)gigixo\.com(?:\b)" 0; "~*(?:\b)gigixo\.com(?:\b)" 0;
"~*(?:\b)gilbertbanda\.net(?:\b)" 0; "~*(?:\b)gilbertbanda\.net(?:\b)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ADmantX ASPSeek
DnyzBot AhrefsBot
Ezooms Aipbot
Mister PiX JetCar
NetZIP Mb2345Browser
ScoutJet Mozlila
Searchestate RepoMonkey
WiseGuys Robot Rogerbot
meanpathbot VCI
trendiction.com clark-crawler

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,30 +1,23 @@
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404enemy
ADmantX
ALittle Client ALittle Client
Aboundex Abonti
Aboundexbot Aboundexbot
Acunetix
Aipbot Aipbot
AlphaBot Alligator
Anarchie Anarchy99
Anarchy
Ankit Ankit
Anthill Anthill
Apexoo Apexoo
Asterias AwarioRssBot
Attach BDCbot
AwarioSmartBot
BLEXBot
BackWeb BackWeb
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
Barkrowler
BatchFTP
BetaBot BetaBot
Black Hole Bitacle
Blackboard Blow
BlowFish BlowFish
Bolt Bolt
Buck Buck
@ -32,219 +25,226 @@ BuiltBotTough
BuiltWith BuiltWith
Bullseye Bullseye
BuzzSumo BuzzSumo
CATExplorador CCBot
CSHttp CODE87
Calculon
Cegbfeieh
CensysInspect CensysInspect
CheTeam CherryPicker
ChinaClaw ChinaClaw
Cogentbot
Collector Collector
CopyRightCheck DIIbot
Copyscape DSearch
Cosmos
Crawling at Home Project
CrazyWebCrawler
Curious
Custo
DBLBot
DTS Agent DTS Agent
DatabaseDriverMysqli Demon
DigitalPebble Deusu
Devil
Digincore
Dirbuster
Disco
Discobot
Dispatch Dispatch
Download Wonder DomCopBot
DomainAppender
Dotbot
Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Siphon
Ebingbong Ecxi
EroCrawler Express WebPictures
Exabot Extractor
FDM Extreme Picture Finder
Foobot EyeNetIE
Freeuploader Ezooms
FyberSpider Firefox/7.0
FlashGet
Fuzz
Fyrebot Fyrebot
G-i-g-a-b-o-t
GalaxyBot GalaxyBot
GetWeb Genieo
GetRight
Getintent Getintent
Gigablast Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GrabNet
Grabber
Grafula Grafula
HEADMasterSEO GrapeshotCrawler
HMView GridBot
Hloader Haansoft
Humanlinks
IDBTE4M IDBTE4M
IlseBot IDBot
Iblog
Image Fetch
IndeedBot IndeedBot
InfoNaviRobot InfoNaviRobot
InternetSeer
Iria
Iskanie Iskanie
JOC Web Spider
JamesBOT JamesBOT
JennyBot
JikeSpider JikeSpider
Joomla
Jorgee Jorgee
Kenjin Spider Jyxobot
Keyword Density Keyword Density
Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
LeechGet Larbin
LeechFTP
LexiBot
Lftp
LibWeb
Libwhisker Libwhisker
Likse Likse
LinkScan LinkWalker
Linkbot
Linkdexbot
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
Ltx71 Litemage_walker
MSIECrawler MIDown tool
MIIxpc
MJ12bot
MTRobot MTRobot
Mag-Net Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic-SEO
Majestic12 Majestic12
MarkMonitor Mass Downloader
Masscan
Mata Hari Mata Hari
MeanPath Bot MauiBot
MicroMessenger Mb2345Browser
Morfeus Fucking Scanner MegaIndex.ru
Metauri
Mister PiX
Moblie Safari
Mojolicious
Mozlila Mozlila
Mr.4x3 Msrabot
NPbot Name Intelligence
Nessus NearSite
Net Vampire Needle
NetMechanic
NetSpider NetSpider
Nettrack Netcraft
Nibbler
Niki-bot Niki-bot
Nutch Nikto
Octopus NimbleCrawler
Nimbostratus
Ninja
Nmap
Nuclei
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OrangeBot Openvas
OutfoxBot OrangeSpider
PECL::HTTP PECL::HTTP
PHPCrawl Page Analyzer
PageAnalyzer PageGrabber
Pandalytics PageScorer
Panscient
Papa Foto
Pavuk
PeoplePal PeoplePal
Petalbot Petalbot
Pi-Monster
Picscout Picscout
Picsearch
Pimonster
Pixray
PleaseCrawl PleaseCrawl
Probethenet
Psbot Psbot
Pu_iN
PxBroker PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RankActive RSSingBot
RankFlex RankingBot
RankingBot2 RankurBot
Re-re RealDownload
Reaper Reaper
RepoMonkey Recorder
RedesScrapy
Ripper Ripper
SBIder RocketCrawler
SEOkicks-Robot Rogerbot
SEOprofiler SEOprofiler
SEOstats
SMTBot
ScoutJet
Scrapy
Screaming Screaming
SearchmetricsBot
SemrushBot
SentiBot SentiBot
Seomoz Shodan
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteLockSpider
Sitebeam SiteSucker
Sitevigil
SlySearch
SmartDownload SmartDownload
SocialRankIOBot SpaceBison
Sosospider SpankBot
Sottopop Spanner
Spammen Spbot
Spinn3r Spinn3r
SputnikBot SputnikBot
Sqlmap Steeler
Sqlworm
Sucker
Sucuri
SuperBot SuperBot
Swiftbot Surfbot
Szukacz T0PHackTeam
T8Abot Telesoft
Teleport
The Intraformant The Intraformant
TheNomad Titan
Toata Toata
Toweyabot
Trendiction Trendiction
True_Robot True_Robot
Turingos TwengaBot
Turnitin Upflow
TurnitinBot
Typhoeus
URLy Warning
URLy.Warning
V-BOT
VB Project
Vacuum Vacuum
VidibleScraper
VoidEYE VoidEYE
Voil WWW::Mechanize
WWW-Collector-E
Wallpapers/3.0 Wallpapers/3.0
WeSEE
Web Auto
Web Collage
Web Fetch Web Fetch
Web Fuck
Web Pix Web Pix
WebCopier WebCollage
WebEnhancer WebFetch
WebFuck WebImageCollector
WebLeacher WebSauger
WebPix
WebReaper
WebStripper
WebWhacker WebWhacker
Webalta Webalta
Webshag
Website Quester Website Quester
WebsiteExtractor WebsiteExtractor
Whatweb WebsiteQuester
WinHTTrack Webster
WiseGuys Robot Whack
Wonderbot
Woobot
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Zeus Zauba
Zitebot Zitebot
ZoominfoBot ZmEu
ZumBot
archive.org_bot archive.org_bot
autoemailspider arquivo-web-crawler
check1.exe cah.io.community
clark-crawler
coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics crawl.sogou.com
demandbase-bot
evc-batch
gopher gopher
heritrix lwp-request
instabid
ips-agent
isitwp.com
lwp-trivial
magpie-crawler magpie-crawler
mediawords meanpathbot
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
plumanalytics pcBrowser
scalaj-http probe-image-size
seobility satoristudio.net
seostar sogouspider
serpstatbot spyfu
sysscan
tAkeOut tAkeOut
trendiction.de
voyagerx.com
webmeup-crawler webmeup-crawler
webprosbot xpymep1.exe
x22Mozilla zauba.io

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404enemy 404enemy
ADmantX
ALittle Client ALittle Client
Abonti Aboundex
Aboundexbot AhrefsBot
Acunetix
AiHitBot AiHitBot
Aipbot
AllSubmitter AllSubmitter
Alligator
AlphaBot AlphaBot
Anarchie
Anarchy99 Anarchy99
Ankit
Anthill
Apexoo
Aspiegel
Attach
AwarioRssBot
BLEXBot BLEXBot
BackStreet BackDoorBot
BackWeb BackWeb
Badass Badass
Bandit Battleztar Bazinga
Barkrowler Bigfoot
BatchFTP Blackboard
Bitacle Boardreader
BlowFish Bolt
Brandprotect Brandprotect
Buddy Buddy
BuiltBotTough BuiltBotTough
Bullseye BuiltWith
BunnySlippers
CATExplorador CATExplorador
Calculon CCBot
CazoodleBot CSHttp
Cegbfeieh CensysInspect
CherryPicker
ChinaClaw
Chlooe Chlooe
Cliqzbot Claritybot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot
Collector Collector
Copier Cosmos
Craftbot Crawling at Home Project
CrazyWebCrawler
Crescent
CrunchBot CrunchBot
Custo Curious
CyotekWebCopy
DSearch DSearch
DatabaseDriverMysqli DataCha0s
Demon Demon
DigitalPebble
Disco
DittoSpyder
DomainAppender DomainAppender
Dotbot
Download Wonder
Dragonfly Dragonfly
ECCP/1.0 Drip
EMail Siphon
EMail Wolf
EirGrabber EirGrabber
Evil Extreme Picture Finder
Express WebPictures FDM
ExtLinksBot FemtosearchBot
EyeNetIE Freeuploader
FHscan
FlashGet
FyberSpider
Fyrebot Fyrebot
GT::WWW
GalaxyBot GalaxyBot
Genieo
GermCrawler GermCrawler
GetRight
GetWeb GetWeb
Getintent
Gigablast
Gigabot Gigabot
Go-Ahead-Got-It Gotit
GoZilla GrabNet
Grabber Grafula
GrapeFX GrapeFX
HMView
HTTP::Lite HTTP::Lite
HTTrack Heritrix
Haansoft Humanlinks
Hloader IDBTE4M
HybridBot Iblog
IDBot
IRLbot
IlseBot IlseBot
IndeedBot Image Fetch
Image Sucker
InfoNaviRobot
InfoTekies InfoTekies
Internet Ninja
InternetSeer InternetSeer
Iria
Iskanie
JOC Web Spider JOC Web Spider
Jetty Jetty
JikeSpider
Joomla
Jorgee Jorgee
Kenjin Spider
Keyword Density
Kinza Kinza
LNSpiderguy Leap
LWP::Simple LeechFTP
LibWeb Lftp
Linkdexbot Libwhisker
LinkpadBot LieBaoFast
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot Lipperhey Spider
Lmspider Ltx71
MIDown tool MIDown tool
MQQBrowser MQQBrowser
Mag-Net MSIECrawler
Magnet MTRobot
MarkMonitor Mail.RU_Bot
Mata Hari Majestic SEO
Majestic12
MarkWatch
MauiBot MauiBot
Mb2345Browser Mediatoolkitbot
Microsoft URL Control MegaIndex.ru
Moblie Safari Metauri
MicroMessenger
Mozlila
Mr.4x3 Mr.4x3
Msrabot
Musobot Musobot
Name Intelligence NPbot
Navroad
Nessus Nessus
Net Vampire
NetSpider
NetZIP NetZIP
Netcraft
Nettrack
Netvibes Netvibes
Nibbler NextGenSearchBot
Niki-bot
NimbleCrawler
Nuclei Nuclei
Octopus
OpenVAS OpenVAS
OutfoxBot Openfind
POE-Component-Client-HTTP OrangeSpider
PHPCrawl
Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber PeoplePal
PageScorer
Petalbot
Pi-Monster
Picscout
Picsearch Picsearch
Piepmatz PictureFinder
Pimonster
Pockey
ProWebWalker
Probethenet
Psbot
Pu_iN Pu_iN
PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot RankActiveLinkBot
RankurBot RankingBot
Rankivabot
Re-re Re-re
RealDownload ReGet
RebelMouse
RepoMonkey RepoMonkey
RocketCrawler Ripper
SBIder SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler
SEOstats SEOstats
SISTRIX SalesIntelligent
SMTBot Scanbot
ScanAlert ScoutJet
Scrapy SearchmetricsBot
ScreenerBot
ScrepyBot
Semrush Semrush
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz
Shodan Shodan
Siphon
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Siteimprove
Sitevigil Sitevigil
SlySearch
SmartDownload SmartDownload
Snapbot Snoopy
SocialRankIOBot SocialRankIOBot
Sogou web spider
Sosospider Sosospider
Sottopop SpankBot
Spanner Spanner
Spbot Spinn3r
SputnikBot
Sqlworm
Steeler
Sucker Sucker
Sucuri SuperBot
Surfbot Suzuran
T0PHackTeam T8Abot
TeleportPro
The Intraformant The Intraformant
TheNomad Titan
TightTwatBot
Toata
Toweyabot Toweyabot
Trendiction Tracemyfile
Trendictionbot Trendictionbot
True_Robot Turingos
Turnitin TurnitinBot
UnisterBot Twice
URLy.Warning
Upflow Upflow
V-BOT Vagabondo
VB Project Voil
Vacuum WBSearchBot
VeriCiteCrawler
WEBDAV WEBDAV
WISENutbot
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE WeSEE
Web Auto Web Auto
Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Pix Web Pix
Web Sucker Web Sucker
WebSauger WebAuto
WebStripper WebCollage
WebCopier
WebEnhancer
WebFetch
WebPix
WebReaper
WebSucker WebSucker
WebWhacker WebWhacker
WebZIP WebZIP
Webalta
WebmasterWorldForumBot
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Whacker
Whatweb Whatweb
Who.is Bot WinHTTrack
WiseGuys Robot WiseGuys Robot
Woobot Wotbox
Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Zeus
Zade Zitebot
ZmEu
ZoomBot ZoomBot
ZoominfoBot archive.org_bot
ZyBorg arquivo.pt
clark-crawler backlink-check
crawler.feedback cah.io.community
demandbase-bot crawl.sogou.com
facebookscraper crawler4j
instabid ips-agent
lwp-request isitwp.com
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
oBot oBot
pcBrowser polaris version
probe-image-size
ripz ripz
s1z.ru s1z.ru
serpstatbot scan.lol
seobility
seoscanners
sitechecker.pro
siteripz siteripz
sp_auditbot sogouspider
tAkeOut sysscan
trendiction.com webpros.com
x22Mozilla webprosbot
xpymep1.exe x09Mozilla
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adtiger.tk anticrawler.org
biketank.ga guidefs.ru
candlewooddentalcentre.com.au hyipmanager.in
compliance-george.top lizace.com
pdn-4.com nibbler.silktide.com
proxyelite.biz ok-ua.info
sbt-aqua.ru prize44.com
svolze.com trahic.ru
tizanidineandgabapentin.blogspot.com unlimitdocs.net
xn----btbdvdh4aafrfciljm6k.xn--p1ai xpresscare.ru

950
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0c47f8422d3f.com 0c47f8422d3f.com
1-free-share-buttons.com 101raccoon.ru
13tabs.com 1adult.com
14b.info 2nt.ru
1kdailyprofit.me 4ureyesonly.com
404.mn 6128786.com
4kplayer.pl 7ooo.ru
5forex.ru 7xc4n.com
777-club.ru abtasty.com
7wind.ru adclickservice.com
83net.jp adtiger.tk
8si.ru advancedcleaner.com
999webdesign.com advokat-grodno.by
a.frcls.fr affiliate-fr.com
absolute-s.ru agadelha.com.br
absugars.com airlandsea.info
academia-nsk.org akuhni.by
acortarurl.es alessandraleone.com
ad-words.ru alfa-img.com
adelly.bg aliexpresscashback.club
adexprts.com amung.us
adloads.com an-donut.com
adloads.net anabolics.shop
adsref.men angel1777.kz
adtech.us angigreene.com
afslankpillen2017nl.eu animal36.com
agardomains.com animali.deagostinipassion.it
alarmobninsk.ru animalia-life.club
alfabot.xyz animebox.com.ua
allbizne.co.ua apibatbrowsecom-a.akamaihd.net
alldownload.pw arabsexxxtube.com
allforminecraft.ru arcadeplayhouse.com
allmarketsnewdayli.gdn
alltheviews.com
alpharma.net
amazingninja.com
andpolice.com
anti-virus-removal.info
apartmentratings.com
apiadanaknet-a.akamaihd.net
appartement-stumm.at
apxeo.info
arclk.net
arcteryxstore.online arcteryxstore.online
asdfg.pro arenanews.com.ua
asophoto.com arquapetrarca.info
asrv-a.akamoihd.net asscenihotosidea.blogspot.co.za
atmagroup.ru atelielembrancaqueencanta.com.br
automatic-seo.com aversis.net
automobile-spec.com azadnegar.com
autorn.ru azinoofficial777.ru
avtocredit-legko.ru babosas.com
ayakino.net babs.com.ua
b2b-lounge.com babyfactory.fr
bankiem.pl badavit.com.ua
balook.com
banki76.ru
bankofthewext.com bankofthewext.com
baoxaydung.com.vn bannerpower.com
bayareaaeroclub.org bannerspace.com
beauty-things.com bbsoldes.fr
bedandbreakfast.com beget.tech
bestdooz.com bestfortraders.com
bestmobilityscooterstoday.com bet-prognoz.com
betterhdporn.com beyan.host.sk
bigcities.org bgviagrachrx.com
biznesrealnost.ru bioca.org
bizru.info bizfly.info
blackcurranthumidifiers.site bkns.vn
blogos.kz blog.f00kclan.de
bmwhighperformers.com blue-square.biz
bolegapakistan.com boc.kz
bongiornos.info bocoarchives.org
boo-arts.com boeuklad.com
bookhome.info boom-celebs.com
bosefux.esy.es brandov.ru
buyessay3.blogspot.ru bugof.gq
buzzsumo.com bum.com.ru
cacheimages.com bumascloset.com
bus-offer.com
buytizanidineonline.blogspot.com
buzzzg.men
bvlgariwallet.xyz
c-english.ru
cakemediahq.com.au cakemediahq.com.au
callawaygolfstore.online callstevens.com
candlewooddentalcentre.com.au cannibalcheerleader.com
candytiner.com capsderfudd.tk
canzoni.ru carfax.com.ua
cartierjusteunclou.xyz carsdined.org
cartierlove2u.xyz cartierbraceletsreplica.pw
cercacamion.it cayado.snn.gr
cgi2.nintendo.co.jp changde.58.com
chatroulette.world cheapbelstaff.online
cherubinimobili.it chocoslim.pro
citizenclsdriveri7.pen.io cialis-samples.com
classiquebijoux.ru cindymatches.com
cntravelre.com clickcash.com
cocyq.inwtrade.com clickintext.net
compliance-alexa.top clippingphotoindia.com
compliance-don.top compliance-barak.top
compliance-irvin.top compliance-checker-7.info
compliance-willy.top compliance-checker.info
connectingsingles.com continent-e.tv
constantaservice.net coolyarddecorations.com
contentlook.co counciltally.com
coolbar.pro crackzplanet.com
coop-gamers.ru
credit-online.ws credit-online.ws
creditwell.ru customcatchcan.com
cyse.tk customergrowthsystems.com
daptravel.com cwetochki.ru
darkbooks.org cxpromote.com
datadepths.com czat.wp.pl
dealighted.com dalavia.ru
decorazilla.com date-withme.com
deda-moroza-zakaz.ru datsun-do.com
dellalimov.com dawlenie.com
delta-line.men dekoration.us
detki-opt.ru dentalpearls.com.au
dev-seo.blog deplim.com
dktr.ru detsky-nabytek.info
dnepropetrovsk.mistr-x.org dignitasdata.se
dogclothing.store dimfour.com
domik-derevne.ru divci-hry.info
dominateforex.ml dojki-hd.com
dptaughtme.com domain.webkeyit.com
eaplay.ru domaincdn.xyz
egvar.net domaincrawler.com
elmacho.xyz download-of-the-warez.blogspot.com
espaceinventoristes.com downtuptv.gq
essaypro.com draniki.org
event-tracking.com drugs-no-rx.info
evrotekhservis.ru e-poker-2005.com
excaliburfilms.com earn-from-articles.com
exchanges-bet.com el-nation.com
exoneration-project.us eonpal.com
extlinks.com eropornosex.ru
extremal-blog.com erotik0049.com
ezigarettenshop2.postbit.com erotikchat-24.com
fable.in.ua errorfixing.space
faithe.top este-line.com.ua
familyholiday.ml estibot.com
favornews.com eu-cookie-law.blogspot.com
feeriaclub.ru evening-dating-club.info
fellowshipoftheminds.com exchangeit.gq
fertilitetsradgivningen.se exonline.info
fetishinside.com expertnaya-ocenka.ru
financehint.eu extremepornos.net
find1friend.com eyessurgery.ru
floridamhca.org ezigaretteshop.webs.com
folowsite.com f012.de
footbalive.org falllow.gq
formulaf1results.blogspot.com feel-planet.com
forum69.info femdom.twiclub.in
fotopop.club ferieboligkbh.dk
foxjuegos.com fettefrauen.net
fickblock18.com
fil.ru
filesdatabase.net
findsexguide.com
findthe.pet
finteks.ru
fordsonmajbor.cf
forodvd.com
forzeronly.com
foxydeal.com foxydeal.com
franklinfire.co freejabs.com
free-share-buttons.top freemags.cc
freecamdollars.com fun-mobi.pl
freeseedsonline.com gallerily.com
freshdz.com gamerextra.com
freshmac.space gay-site.store
g.starmoe.xyz gaz-voshod.ru
games.kolossale.ru girlgamerdaily.com
gcup.ru glasof.es
gdcentre.ru go2jump.org
ge0ip.net gobongo.info
gearcraft.us
gelezki.com
germetiki.com.ua
getresponse.com
gettpromos.com
ggiaro.com
girlsfuckdick.com
glastecfilms.com.my
goblacked.com
golden-catalog.pro golden-catalog.pro
goldpanningtools.com golfresa.lucania.se
gratuitxblcodes.com grupografico-pilar.com.ar
hccoder.info gsasearchengineranker.top
headpress.ru gta-top.ru
heartofbeijing.blogspot.com hamacapty.com
heatpower.ru hamptonoaks.ca
hanink.biz.ly
hanwei.us
health-medical-portal.info
hebr.myddns-flir.com hebr.myddns-flir.com
help.tpu.ru helicalpile.us
herehloadibs.cf hobbyhuren-datenbank.com
hermesreplica.pw home-task.com
hermesreplica.win home.myplaycity.com
heygidday.biz
histats.com
homeart.space homeart.space
homedo.fabpage.com homedo.fabpage.com
homik.pw
hoporno.com
host-protection.com host-protection.com
hoste.octopis.com hoverboardforsaledirect.com
hotchatdate.com huhn.altervista.org
howtowhitenteethfast.xyz hunthillfarmtrust.org
hreade.com hvd-store.com
hulfingtonpost.com igru-xbox.net
iboss.com iloveitaly.ru
iideaidekonkatu.info ilovevitaly.xyz
inboxdollars.com imediadesk.com
infogame.name imgarit.pw
impisr.ru
innodgfdriverhm.aircus.com
inspiring-desperate.tk inspiring-desperate.tk
iqoption.com intelhdgraphicsgtdrive6w.metroblog.com
it-max.com.ua intermesh.net
ip-guide.com
isotoner.com
itch.io itch.io
itis4you.com iwantmyfreecash.com
javlibrary.cc jamiembrown.com
jennyfire.ru janerikholst.se
jolic2.com jav-fetish.com
jwcialislrt.com jerseychinabizwholesale.com
kalandranis.gr jerseyswholesalechinalimited.com
kareliatobacco.ru jewelryandfiligree.com
kartiniresto.com journeydownthescale.info
katjimej.blog.fc2.com kaidalibor.de
kaz.kz kamalsinha.com
keywesthideaways.co kashubadesign.ru
kfon.eu keywordbasket.com
kidskunst.info keywordblocks.com
kino-key.info keywordsking.com
kinobaks.com khafre.us
kliksaya.com kinomaniatv.pl
klitimg.pw kiwi237au.tk
komputers-best.ru kolotiloff.ru
konkursov.net konkursov.net
kovrenok.ru krasnodar.xrus.org
krasnodar.ru kupit-adenu.ru
kunstaktien.info kvartiry-remont.ucoz.ru
kutikomi.net ldrtrack.com
kvartira-sutochno.com leadn.pl
languagecode.com lerporn.info
lapolis.it letmacworkfaster.site
laptop-4-less.com li-er.ru
larose.jb2c.me likrot.com
laserpen.club liveshoppersmac.com
law-check-nine.xyz loadopia.com
layola.biz.tc lomza.info
le-clos-des-alouettes.com long-beach-air-conditioning.com
legionalpha.com longlifelomilomi.info
lenvred.org lumb.co
levaquin750mg.blogspot.com luxup.ru
lexaprogeneric.link macfix.life
liky.co.ua magicalfind-a.akamaihd.net
linkbuddies.com manipulyator-peterburg.ru
linkbux.ru mapquestz.us
liran-locks.com markjaybeefractal.com
littlesunraiser.com matpre.top
liupis.com mebel-alait.ru
live-sexcam.tk medictube.ru
liver-chintai.org memberty.com
lol-smurfs.com mexicotravelnet.com
louboutinreplica.xyz michaelkorsoutletstore.net
love-baby.cz michellblog.online
lowephotos.info midst.eu
luchshie-topcasino.ru mir-betting.ru
m0r0zk0-krava.ru missis.top
m292.info mixtapetorrent.com
magento-crew.net mixx.com
mainlinehobby.net mobot.site
marcoislandvacations.net moesen-ficken.com
marwer.info moesexy.com
matsdale.com
maximpartnerspr.com
mb140.ru
medtherapy.ru
meduza-consult.ru
mesto-x.com
metalonly.info
mexicosleevegastrectomy.com
migente.com
militarysale.pro
minecraft-neo.ru
mini-modus.ru
minneapoliscopiers.com
mir-business-24.ru
mir-limuzinov.ru
misandesign.se
mobilemedia.md
mobioffertrck.com
mobstarr.com
monetizer.com-01.site monetizer.com-01.site
monitorwebsitespeed.com monsterdivx.tv
monsterdivx.com motherboard.vice.com
moyakuhnia.ru mowser.com
mp3downloadhq.com mrlmedia.net
musezone.ru mrvideospornogratis.xxx
musicspire.online muschisexbilder.com
mybackgroundlandscape.blogspot.com musicas.baixar-musicas-gratis.com
mydirtystuff.com musicpro.monster
mydoctorok.ru mvpicton.co.uk
mymobilemoneypages.com mydearest.co
napalm51.nut.cc myfreecams.com
narkologiya-voronezh.ru mylesosibirsk.ru
national-today-winning-winner.club myprintscreen.com
naturalbreakthroughsresearch.com mysex21.com
naturalshair.site naperehresti.info
news-speaker.com nastroyke.net
nisuturnetdgu.tk navalwiki.info
oddamzadarmo.eu negral.pluto.ro
odywpjtw.bloger.index.hr newhairstylesformen2014.com
ofanda.com nextbackgroundcheck.gq
omsk.xrus.org nextrent-crimea.ru
onemantrip.com nfljerseyscheapbiz.us
onlinebay.ru nfvsz.com
ons-add.men ngps1.ru
oops-cinema.ru nofreezingmac.click
open-odyssey.org nordstar.pro
orion-code-access.net nrjmobile.fr
osoznanie-narkotikam.net o-o-8-o-o.ru
outrageousdeal-a.akamaihd.net obiavo.by
paramountmarble.co.uk obsessionphrases.com
paydayloanslocal.com obuv-kupit.ru
paydayonlinecom.com ohmyrings.com
pdn-4.com okel.co
pdns.download olympescort.com
pensplan4u.com omgtnc.com
phormchina.com oneshotdate.com
pinapchik.com online-hd.pl
pinstake.com ontargetseo.us
pipki.r.acdnpro.com ooomeru.ru
piratecams.com orel-reshka.net
pistonclasico.com osagonline.ru
plastgranar.nu ostrovtaxi.ru
playmsn.com outclicks.net
poems.com.ua p-business.ru
poppen-nw.net par-fallen.ga
pornmania.pl parajumpersjakkesalgnorge.info
pornoforadult.com partners-ship.pro
pornosmola.info penisvergrotendepillennl.ovh
pornozhara.com perosan.com
pozdrawleniya.ru piccshare.com
princeadvantagesales.com polybuild.ru
privatefx-in.ru pons-presse.com
privhosting.com pornhive.org
prize44.com pornoinn.com
prizeestates.cricket pornosexrolik.com
prostitutki-belgoroda.org porto.abuilder.net
quick-offer.com portside.cc
portside.xyz
pozvonim.com
pr0fit-b0x.com
pricheskaonline.ru
printie.com
privatefx.all4invest.info
proekt-gaz.ru
profitkode.com
prostitutki-almata.org
prostitutki-tyumeni.org
puserving.com
pyrodesigns.com.au
quit-smoking.ga quit-smoking.ga
radiogambling.com rainbowice.ru
ragecash.com rapidokbrain.com
randalljhoward.com ratemodels.net
raschtextil.com.ua rating-bestcasino.com
realting-moscow.ru razleton.com
remote-dba.de razorweb-a.akamaihd.net
renewablewealth.com refudiatethissarah.info
respublica-otel.ru registrydomainservices.com
rezeptiblud.ru reining.lovasszovetseg.hu
ringtonepartner.com remont-fridge-tv.ru
sakhboard.ru remont-mobile-phones.ru
sandhillsonline.com reward-survey.net
satoristudio.net richinvestmonitor.com
sayyoethe.blogspot.co.za ricorsogiustizia.org
sbf441.com rogervivierforsale.com
scanner-jack.top roofers.org.uk
score-ads.men running-line.ru
search.1and1.com rutor.vip
search.alot.com rvi.biz
seasonvar.ru rybalka-opt.ru
securesmrt-dt.com saecsa.co
semalt.com santasgift.ml
semaltmedia.com saugatuck.com
seo18.su saveindex.xyz
seofied.com sbetodiodnye-lampy.ru
service-core.ru scansafe.net
sexkontakte-seite.com scenicmissouri.us
sexreliz.com scopich.com
sexs-foto.com scrapy.org
sexy-pings.com searchinterneat-a.akamaihd.net
searchmywindow-a.akamaihd.net
secret.xn--oogle-wmc.com
semp.net
seo-smm.kz
seorank.info
seozoom.it
service.adtech.us
servicecenter.co.ua
serving.adbetclickin.pink
sex-sex-sex5.com
sexyebonyteen.com sexyebonyteen.com
sexystrippe.info shlyahten.ru
sharebutton.net shop.xz618.com
shikiso.info simple-image.com.ua
shohanb.com simul.co
shop.acim.org sitevaluation.org
shopcheermakeup.info sivs.ru
shopsellcardsdumps.com sixcooler.de
shopwme.ru slowmac.tech
sildenafilcitratemed.com sluganarodu.ru
similardeals.net
sinestesia.host.sk
siteimprove.com
skutecznetabletkinaporostwlosow.pl
sky-mine.ru
slutloadlive.com
smailik.org
small-games.biz small-games.biz
sneakyboy.com smart-balancewheel.com
snjack.info smartbalanceworld.com
snworks.com smartpet.ru
sobecjvuwa.com.ru smichovbike.cz
socialbutton.xyz smsactivator.ru
socialseet.ru smstraf.ru
sofit-dmd.ru soft-program.com
softtor.com softxaker.ru
sortthemesitesby.com sogimlecal.tk
sotkal.lark.ru solmarket.by
soundfrost.org spaceshipad.com
spamnuker.com spasswelt.net
sportbetfair.com spin2016.cf
spy-app.info sports-supplements.us
stanthonyscatholicchurch.org spy-sts.com
staynplay.net srecorder.com
stpicks.com ssconstruction.co
strag-invest.ru staff.prairiesouth.ca
strana-krasoty.ru starwars.wikia.com
strangeduckfilms.com steebook.com
stretchmate.net sticken.co
stroyplus.ru stpolice.com
sukirgenk.dvrlists.com studentguide.ru
summerlinhomes411.info students-cheapskate.ml
superstats.com stuff-about-money.com
susanholtphotography.com sugarkun.com
swinginwithme.ru superiends.org
szamponrevita.pl swinger-mobil.net
t-bygg.com taaaak.com
tabakur77.com tandvardshuset.net
takeflyte.com taqplayer.info
tamada69.com tarad.com
tasteidea.com tattomedia.com
taximytishi.ru taximytishi.ru
tecnoteakviareggio.it teastory.co
telsis.com technika-remont.ru
template-kid.com telefonsex-ohne0900.net
thin.me.pn the-usa-games.blogspot.com
tizanidine6mg.blogspot.com thefotosgratis.eu
tizanidinedruginteractions.blogspot.com themeforest.net
tizanidineonline.blogspot.com thenews-today.info
tizanidinestreetprice.blogspot.com timdreby.com
token-lab.org tizanidineandcipro.blogspot.com
tootoo.to tizanidineandgabapentin.blogspot.com
tpu.ru tizanidineformigraines.blogspot.com
trafficshaper.com tizanidineforopiatewithdrawal.blogspot.com
trailer.cinemaflix.website toonfamilies.net
try-rx.com topcasinoratings.ru
twincitiescarservice.com torrent-to-magnet.com
twojebook.pl toys.erolove.in
uhod-za-sobakoj.ru traderzplanet.in
undergroundcityphoto.com trafficnetzwerk.de
uptimechecker.com tuckpointingmasonrysystems.com
vacances-voyages.info tvand.ru
validdomain.xyz twinderbella.com
vancleefreplica.pw tytoona.com
vandrie-ict.nl u17795.netangels.ru
vapsy.com ublaze.ru
uginekologa.com
uroffer.link
us-america.ru
userequip.com
vacuumcleanerguru.com
vashsvet.com vashsvet.com
velen.io vchulkah.net
veles.shop venta-prom.ru
velobikestock.com
venerologiya.com
viberdownload10.com
video-woman.com
videokrik.net
villakohlanta.nu villakohlanta.nu
vip.51.la viteonlusarezzo.it
vipms.ru viveresaniesnelli.it
visitcambridge.org vkak.ru
vod.com.ua vladimir.zrus.org
vseigru.one vodaodessa.com
vtcdns.com voluumtrk.com
vvpg.ru vpnhowto.info
vykup-avto-krasnodar.ru wareseeker.com
wahicbefa31.soup.io website-speed-up.top
watchmyfb.pl websitevaluebot.com
webalan.ru welcomeauto.ru
weclipart.com wemarketing.se
wedding0venues.tk wieseversa.no
wetgames.ru wiosenny-bon-1500.pl
wfb.hatedriveapart.com
whatsappbot.flyland.ru
womama.ru
woman-h.ru
woman-tampon.ru
wonderfulflowers.biz wonderfulflowers.biz
worldoffiles.ru worst-sites.online
wowcasinoonline.ooo wttavern.com
wufak.com www.get-free-traffic-now.com
ww2awards.info x-porno.video
x-diesel.info xbaboon.com
xcc24.pl xgftnlrt.bloger.index.hr
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xmladserver.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn----itbkqkfiq.xn--p1ai xn----7sbahjd3btneuw1joc.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn----9sbubg3ambdfl1j.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--80aodinpgi.xn--p1ai xn--b1agm2d.net
xxxguitars.com xn--e1afanlbnfckd7c3d.xn--p1ai
yebocasino.co.za xtrafficplus.com
youbrainboost.asia xvideosbay.com
youjizz.com xwatt.ru
yourporngay.com xxxdatinglocal.us
yourserverisdown.com xxxmania.top
youtuhe.com yellowfootprints.com
zaim-pod-zalog-krasnodar.ru yougotanewdomain.com
zaimite.ru youradexchange.com
zeroredirect.com yourads.website
zigarettenonlinekaufen1.blox.pl yourmovies.pl
yourothersite.com
youthreaders.com
zapatosenventa.info
zerocash.msk.ru
zeroredirect5.com
zeroredirect8.com
zlatnajesen.com zlatnajesen.com
zona-aqua.ru znaturaloriginal.com
zocaparj.kz
zolotoy-lis.ru
zoodrawings.com zoodrawings.com
zoogdiesney.com zrizvtrnpale.tk
zverokruh-shop.cz zrus.org
zynax.ua zvuker.net
zytpirwai.net

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1bet.com 101flag.ru
1flag.co.za 1flag.co.za
2x2fan.ru 21h2o.com
aa9bd78f328a6a41279d0fad0a88df1901.com 28n2gl3wfyb0.ru
aasoldes.fr acheterviagrafr24.com
abcdefh.xyz activepr.ru
adobereader-free.ru adult-shop.com.ua
adsloads.com advancedcleaner.com
adtech.de advancedsoftwaresupport.com
adzpower.com affordablewebsitesandmobileapps.com
alevinefesleri.com afmuseum.com
allnews.md alltheviews.com
allproblog.com anti-virus-removal.info
aloofly.com apiallgeniusinfo-a.akamaihd.net
amigobulls.com apple.com-webbrowsing-security.science
appartement-stumm.at appleid-verification.com
artefakct.com approved.su
autorn.ru arenda-yeisk.ru
balla-boo.se arius.tech
bangla-chat-uk-paralud.ga asacopaco.tk
asdfg.pro
atdedinotuho.tk
atmovs.com
ausergrubhof.info
autovideobroadcast.com
azte.ch
baltgem.com
bavsac.com
beachtoday.ru
best-games.today
bestdooz.com
bestempresas.es
bestpriceninja.com
bestserials.com
bet-prognoz.com
betonka.pro
betteroffers.review
bin-brokers.com bin-brokers.com
binomo.kz bioscorp.ru
blagovest-med.ru biznesrealnost.ru
blockworld.ru bizzliving.com
bluesalt.co bluerobot.info
bosman.pluto.ro bobinoz.com
brus-vsem.ru boleznikogi.com
bwlx.prepedu.cn bookmark4you.com.biz
candcstuccoandstone.com brainzpod.win
brandedleadgeneration.com
businesn.men
buypillsorderonline.com
c-gainsbourg.com
cakesplus.com.au
canrole.com canrole.com
caveavins.fr cdncash.com
cheapsergiorossi.online cerev.info
checkm8.com chinaelitecheapjerseys.com
checkpagerank.net cleanallspyware.com
chocolateslim-en-espana.com coaching-netz.info
cigarpass.com codysbbq.com
cinemaenergy-hd.ru compliance-alex.xyz
cleanmypc.com compliance-elena.xyz
click2pawn.com compliance-josh.top
cocolyze.com connectionstrenth.com
coinsspb.com conocer-sanabria.com
compliance-brian.top creatives.adbetclickin.pink
cookie-law-enforcement-ff.xyz crirussian.ru
cookielawblog.wordpress.com dawlenie.com
country-chic.ru deanmoore.ie
craftinsta.ru djihispano.com
crd.clan.su domcran.net
creditnation.ru dood.live
customsua.com.ua
cvety24.by
datsun-do.com
dentalpearls.com.au
discountliv.com
domain-submit.info
domashneeruporno.com
drugstoreforyou.com
eas-seo.com
ege-russian.ru
ekosmetyki.net.pl
elektrischeziga.livejournal.com
elektrischezigarette1.onsugar.com elektrischezigarette1.onsugar.com
elektronischezigarettekaufen1.beeplog.com elitepcgames.com
euronis-free.com englate.com
eyessurgery.ru erotik0049.com
ezigarettekaufen2.blox.pl failingmarriege.blogspot.com
ezigarettenshop2.myblog.de familyholiday.ml
filmgo.ru fanrto.com
findclan.org faracontrol.ir
free-floating-buttons.com felizporno.com
free-share-buttons.top filunika.com.ru
freza-sverlo.ru findpik.com
fuckingawesome.com forms-mtm.ru
g7m.pl freefoto.ca
gaming-journal.com funnel.co.za
gemara.com gateway.zscalerone.net
globalscam.ga gaz-voshod.ru
goforexvps.com ge0ip.org
gotorussia.com girlsfuckdick.com
greatzip.com gkvector.ru
guardlink.org glogow.pl
handicapvansarea.com goodwinmetals.co
handsandlegs.ru gourcy.altervista.org
hawaiielectriclight.com gz2.bbsoldes.fr
hit-kino.ru hd-film.pl
hotchatdate.com headpress.ru
hotcore.info home-task.com
hotel-mkad.ru hotenergy.ru
howlongdoestizanidinestayinyoursystem.blogspot.com
howopen.ru
id-forex.com
idegenvezeto.eu
ideoworld.org
iframe-toloka.com iframe-toloka.com
images.gyffu.com investmac.com
indetiske.ya.ru iqbazar.ru
indiasourcemart.in irunfar.com
infoupdate.org isotoner.com
iphantom.com ispaniya-costa-blanca.ru
irkutsk.online-podarki.com istizanidineacontrolledsubstance.blogspot.com
it-max.com.ua istock-mebel.ru
istripper.com
ixora.pro ixora.pro
jav-idol.com jav-idol.com
jobgirl24.ru javcoast.com
jongose.ninja jerseychinabizwholesale.com
jose.mulinohouse.co kazinogames.lv
karting196.ru kidd.reunionwatch.com
keywordsdoctor.com kinogolos.ru
kollekcioner.ru kol-energo.ru
kupiproday.com.ua konkursov.net
lacave.ntic.fr labplus.ru
landoftracking.com larutti.ru
leadn.pl laxdrills.com
lietaer.com ldrtrack.com
littlecity.ch le-clos-des-alouettes.com
loadopia.com life-instyle.com
longgreen.info life.biz.ua
m292.info lifehacĸer.com
macdamaged.tech liver-chintai.org
marketland.ml luckyshop.net.ua
meetingrainstorm.bid makemoneyonline.com
mejoresfotos.eu maximilitary.ru
milblueprint.com mcnamaratech.com
minecraft-rus.org megavolt.net.ua
mini-modus.ru message-warning.net
mirtorrent.net minegam.com
mobsfun.net misterjtbarbers.com
moetomnenie.com moetomnenie.com
moneyzzz.ru mole.pluto.ro
mttwtrack.com mozello.ru
muizre.ru muschisexbilder.com
myblogregistercm.tk mylesosibirsk.ru
myghillie.info mylida.org
nac-bearings.ru
naughtyconnect.com naughtyconnect.com
naval.jislaaik.com nuker.com
netvouz.com o-o-6-o-o.com
networkad.net oconto.ru
nhl17coinsforps3.gratisblog.biz odoratus.net
nicefloor.co.uk ok.ru
nootrino.com olympescort.com
norththeface.store online-x.ru
notebook-pro.ru ons-add.men
office2web.com outrageousdeal-a.akamaihd.net
okmusic.jp paintingplanet.ru
onlinadverts.com paradontozanet.ru
openfrost.com paramountmarble.co.uk
optom-deshevo.ru partner-pop.men
painting-planet.com pc-test.net
picsforkeywordsuggestion.com piratecams.com
pills24h.com pladform.ru
playtap.us porngalleries.top
pod-muzyku.club porno-dojki.net
poffet.net pornonik.com
popads.net pornosee.info
popunder.ru pravoholding.ru
porno.simple-image.com.ua progress-upakovka.ru
pornofeuer.com ratemodels.net
pornoforadult.com rebelmouse.com
pornpost.in reklama1.ru
powc.r.ca.d.sendibm2.com
prizestohandle.club
producm.ru
pronekut.com
pts163.ru
q-moto.ru
rank-checker.online
raschtextil.com.ua
redirect.trafficreceiver.club
reeyanaturopathy.com
reklama-i-rabota.ru
remedyotc.com remedyotc.com
responsive-test.net rep-am.com
rethinkwasteni.info replica-watch.ru
riders.ro responsinator.com
rospromtest.ru rfserial.net
samolet.fr rogervivierforsale.com
saveriopiazza.it ros-ctm.ru
scripted.com roznica.com.ua
seansonline24.pl rukino.org
searchencrypt.com rutor.vip
seasonvar.ru sady-urala.ru
sei80.com sanatorrii.ru
seocdvig.ru scanner-george.top
scanner-jack.top
seo-2-0.com
seolab.top seolab.top
sexy-screen-savers.com seorankinglinks.xyz
shitmovs.com serialsway.ucoz.ru
simon3.ru serptehnika.ru
simplepooltips.com sex-sex-sex5.com
simul.co shoesonlinebuy.cn
skincrate.net slomm.ru
skyway24.ru
smailik.org smailik.org
small-games.biz social-fun.ru
smokewithrabbits.com soda.media
snjack.info softonicads.com
socialsignals24.com solmarket.by
softxaker.ru specialfinanceoffers.com
spl63.fr spymac.net
spylog.com steebook.com
srgwebmail.nl strigkaomsk.ru
stariy-baku.com superstats.com
sticken.co technopellet.gr
stopnarco.ru tedxrj.com
sugarkun.com the-trader.net
superiends.org thefotosgratis.eu
supermama.top thegolfclub.info
svnuppsalaorebro.se tizanidinedosage.blogspot.com
telefonsexi.com toon-families.com
telemetryverification.net toondinsey.com
tomatis.gospartner.com topads.men
totu.info topflownews.com
trafficmania.com trahic.ru
tula.howotorg.ru trustedhealthtips.com
tupper-posuda.ru udsgame.online
twittruth.com vduplo.ru
tytoona.com videotuber.ru
u17795.netangels.ru vikistars.com
uggsale.online vkmusics.ru
unmaroll.ya.ru vseigru.one
upstore.me vsesubwaysurfers.com
uptime-gamma.net wallpapersdesk.info
vabasa.inwtrade.com warningwar.ru
vacances-voyages.info wdrake.com
veerotech.com webcamdevochka.com
vepad.com webmasterhome.cn
videonsk.com webtherapy.ru
vonradio.com westen-v.life
vsexkontakte.net wholesalejerseychinaoutlet.com
vvpg.ru wildcattube.com
warezkeeper.com wingsofrefuge.net
watracker.net worldis.me
weburlopener.com www.get-free-traffic-now.com
whatsappdownload10.com x-rates.ru
wholesalejerseys-cheapest.com xboxster.ru
williamrobsonproperty.com xn--24-glceagatoq7c2a6ioc.xn--p1ai
winx-play.ru
wonderfulflowers.biz
wordpress-crew.net
www.bookmaker-bets.com
x-porno.video
xaijo.com
xaylapdiendanang.com
xerox-douglas.cf
xgftnlrt.bloger.index.hr
xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--lifehacer-1rb.com xn--80aodinpgi.xn--p1ai
youjizz.com xn--e1afanlbnfckd7c3d.xn--p1ai
yourmovies.pl xportvusbdriver8i.snack.ws
yubikk.info xvideospornoru.com
zetmaster.ru y8games-free.com
zigarettenonlinekaufen1.bloog.pl yaaknaa.info
zootoplist.com yadro.ru
yhit.press
youradexchange.com
yousense.info
youtubologia.it
zoosexart.com
zskdla.site
zygophyceous.womanstars.site

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.215 0; 10.1.0.87 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:39 UTC 2021 ### Updated: Sat Sep 4 11:58:58 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -341,8 +341,6 @@ User-agent: GetWeb
Disallow:/ Disallow:/
User-agent: Getintent User-agent: Getintent
Disallow:/ Disallow:/
User-agent: Gigablast
Disallow:/
User-agent: Gigabot User-agent: Gigabot
Disallow:/ Disallow:/
User-agent: Go!Zilla User-agent: Go!Zilla

8
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2569 ### Version: V4.2021.09.2570
### Updated: Sat Sep 4 10:03:39 UTC 2021 ### Updated: Sat Sep 4 11:58:58 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -341,8 +341,6 @@ User-agent: GetWeb
Disallow:/ Disallow:/
User-agent: Getintent User-agent: Getintent
Disallow:/ Disallow:/
User-agent: Gigablast
Disallow:/
User-agent: Gigabot User-agent: Gigabot
Disallow:/ Disallow:/
User-agent: Go!Zilla User-agent: Go!Zilla