vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 12:09:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2932 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-20 10:45:08 +00:00
parent 3b6f7dc231
commit db2781bf42
58 changed files with 1144 additions and 1133 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2931 #### Version: V4.2022.02.2932
#### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2931 #### Version: V4.2022.02.2932
#### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -13,8 +13,8 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2931 #### Version: V4.2022.02.2932
#### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_generator_lists/bad-referrers.list
View file

@ -1,4 +1,3 @@
coccoc.com
000free.us 000free.us
007angels.com 007angels.com
00author.com 00author.com
@ -1359,6 +1358,7 @@ cntravelre.com
co.lumb.co co.lumb.co
coaching-netz.info coaching-netz.info
cobaltpro.ru cobaltpro.ru
coccoc.com
cocolyze.com cocolyze.com
cocyq.inwtrade.com cocyq.inwtrade.com
coderstate.com coderstate.com

4
_google_analytics_ghost_spam/README.md
View file

@ -3,8 +3,8 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2931 #### Version: V4.2022.02.2932
#### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_google_analytics_ghost_spam/google-exclude-01.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-02.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-03.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-04.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-05.txt
View file

File diff suppressed because one or more lines are too long

1
_google_webmaster_disavow_links/google-disavow.txt
View file

@ -1358,6 +1358,7 @@ domain:cntravelre.com
domain:co.lumb.co domain:co.lumb.co
domain:coaching-netz.info domain:coaching-netz.info
domain:cobaltpro.ru domain:cobaltpro.ru
domain:coccoc.com
domain:cocolyze.com domain:cocolyze.com
domain:cocyq.inwtrade.com domain:cocyq.inwtrade.com
domain:coderstate.com domain:coderstate.com

7
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

7
conf.d/globalblacklist-testing-version.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2022.02.2931 ### Version: V3.2022.02.2932
### Updated: Sun Feb 20 10:31:53 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

7
conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

7
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2022.02.2931 ### Version: V3.2022.02.2932
### Updated: Sun Feb 20 10:31:53 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 1; 10.1.0.49 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 0; 10.1.0.49 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2022.02.2931 ### Version: V3.2022.02.2932
### Updated: Sun Feb 20 10:31:53 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 1; 10.1.0.49 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 0; 10.1.0.49 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2022.02.2931 ### Version: V3.2022.02.2932
### Updated: Sun Feb 20 10:31:53 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 1; 10.1.0.49 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/beta_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -1405,6 +1405,7 @@
"~*(?:\b)co\.lumb\.co(?:\b)" 0; "~*(?:\b)co\.lumb\.co(?:\b)" 0;
"~*(?:\b)coaching\-netz\.info(?:\b)" 0; "~*(?:\b)coaching\-netz\.info(?:\b)" 0;
"~*(?:\b)cobaltpro\.ru(?:\b)" 0; "~*(?:\b)cobaltpro\.ru(?:\b)" 0;
"~*(?:\b)coccoc\.com(?:\b)" 0;
"~*(?:\b)cocolyze\.com(?:\b)" 0; "~*(?:\b)cocolyze\.com(?:\b)" 0;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0;
"~*(?:\b)coderstate\.com(?:\b)" 0; "~*(?:\b)coderstate\.com(?:\b)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 0; 10.1.0.49 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2022.02.2931 ### Version: V3.2022.02.2932
### Updated: Sun Feb 20 10:31:53 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2931 2932

7
dev-tools/globalblacklist-testing.template
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2022.02.2931 ### Version: V3.2022.02.2932
### Updated: Sun Feb 20 10:31:53 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

7
dev-tools/globalblacklist.template
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

1
dev-tools/referrers-regex-format-whitelist-test.txt
View file

@ -1358,6 +1358,7 @@
"~*(?:\b)co\.lumb\.co(?:\b)" 0; "~*(?:\b)co\.lumb\.co(?:\b)" 0;
"~*(?:\b)coaching\-netz\.info(?:\b)" 0; "~*(?:\b)coaching\-netz\.info(?:\b)" 0;
"~*(?:\b)cobaltpro\.ru(?:\b)" 0; "~*(?:\b)cobaltpro\.ru(?:\b)" 0;
"~*(?:\b)coccoc\.com(?:\b)" 0;
"~*(?:\b)cocolyze\.com(?:\b)" 0; "~*(?:\b)cocolyze\.com(?:\b)" 0;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0;
"~*(?:\b)coderstate\.com(?:\b)" 0; "~*(?:\b)coderstate\.com(?:\b)" 0;

1
dev-tools/referrers-regex-format.txt
View file

@ -1358,6 +1358,7 @@
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.83.105.253 0; 13.89.96.249 0;

7
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 1; 10.1.0.49 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 0; 10.1.0.49 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 1; 10.1.0.49 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 0; 10.1.0.49 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 1; 10.1.0.49 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -1405,6 +1405,7 @@
"~*(?:\b)co\.lumb\.co(?:\b)" 0; "~*(?:\b)co\.lumb\.co(?:\b)" 0;
"~*(?:\b)coaching\-netz\.info(?:\b)" 0; "~*(?:\b)coaching\-netz\.info(?:\b)" 0;
"~*(?:\b)cobaltpro\.ru(?:\b)" 0; "~*(?:\b)cobaltpro\.ru(?:\b)" 0;
"~*(?:\b)coccoc\.com(?:\b)" 0;
"~*(?:\b)cocolyze\.com(?:\b)" 0; "~*(?:\b)cocolyze\.com(?:\b)" 0;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0;
"~*(?:\b)coderstate\.com(?:\b)" 0; "~*(?:\b)coderstate\.com(?:\b)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 0; 10.1.0.49 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.83.105.253 0; 13.89.96.249 0;

7
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.83.105.253 0; 13.89.96.249 0;

7
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:52 UTC 2022 ### Updated: Sun Feb 20 10:39:11 UTC 2022
### Bad Referrer Count: 7090 ### Bad Referrer Count: 7091
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2425,6 +2425,7 @@ map $http_referer $bad_referer {
"~*(?:\b)co\.lumb\.co(?:\b)" 1; "~*(?:\b)co\.lumb\.co(?:\b)" 1;
"~*(?:\b)coaching\-netz\.info(?:\b)" 1; "~*(?:\b)coaching\-netz\.info(?:\b)" 1;
"~*(?:\b)cobaltpro\.ru(?:\b)" 1; "~*(?:\b)cobaltpro\.ru(?:\b)" 1;
"~*(?:\b)coccoc\.com(?:\b)" 1;
"~*(?:\b)cocolyze\.com(?:\b)" 1; "~*(?:\b)cocolyze\.com(?:\b)" 1;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 1;
"~*(?:\b)coderstate\.com(?:\b)" 1; "~*(?:\b)coderstate\.com(?:\b)" 1;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 1; 10.1.0.49 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/custom-bad-referrers.conf
View file

@ -1405,6 +1405,7 @@
"~*(?:\b)co\.lumb\.co(?:\b)" 0; "~*(?:\b)co\.lumb\.co(?:\b)" 0;
"~*(?:\b)coaching\-netz\.info(?:\b)" 0; "~*(?:\b)coaching\-netz\.info(?:\b)" 0;
"~*(?:\b)cobaltpro\.ru(?:\b)" 0; "~*(?:\b)cobaltpro\.ru(?:\b)" 0;
"~*(?:\b)coccoc\.com(?:\b)" 0;
"~*(?:\b)cocolyze\.com(?:\b)" 0; "~*(?:\b)cocolyze\.com(?:\b)" 0;
"~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0; "~*(?:\b)cocyq\.inwtrade\.com(?:\b)" 0;
"~*(?:\b)coderstate\.com(?:\b)" 0; "~*(?:\b)coderstate\.com(?:\b)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anarchy99 GoZilla
EroCrawler Gotit
Lipperhey LeechFTP
MarkWatch Re-re
Mata Hari Snake
ScanAlert Steeler
SpankBot Swiftbot
evc-batch Trendiction
seoscanners Vacuum
ubermetrics-technologies.com WebCopier

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
ADmantX 360Spider
AIBOT 404enemy
Abonti ALittle Client
Aboundex AfD-Verbotsverfahren
Aboundexbot AiHitBot
AhrefsBot Aipbot
Alexibot Alexibot
Anarchy99 Alligator
AlphaBot
Ankit Ankit
Apexoo Aspiegel
Asterias
Atomseobot
Attach Attach
AwarioSmartBot BackStreet
BDFetch BackWeb
BLEXBot
Backlink-Ceck
Badass
Bandit Bandit
Barkrowler Barkrowler
BatchFTP
Battleztar Bazinga
BetaBot BetaBot
Black Hole Bigfoot
Bitacle
BlackWidow
Blackboard Blackboard
BotALot BlowFish
Boardreader
Bolt
Brandprotect Brandprotect
BuiltWith Buck
Buddy
Bullseye
BunnySlippers BunnySlippers
CATExplorador CATExplorador
CSHttp CODE87
Calculon
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam CheTeam
CherryPicker Citoid
Claritybot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot
Copier Copier
Crawling at Home Project CopyRightCheck
Curious Copyscape
CyotekWebCopy CrazyWebCrawler
DatabaseDriverMysqli Crescent
DBLBot
DIIbot
DataCha0s
Demon Demon
Deusu
Devil
Disco
Discobot Discobot
Discoverybot
Dispatch Dispatch
DomainAppender DnBCrawler-Analytics
DnyzBot
DomainCrawler DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Dotbot ECCP/1.0
Drip EirGrabber
EMail Siphon Exabot
EasyDL ExtLinksBot
Ebingbong
Ecxi
Evil
ExtractorPro ExtractorPro
Ezooms
FDM
Fimap Fimap
Firefox/7.0 Firefox/7.0
Fuzz Foobot
Fyrebot Freeuploader
GetWeb FrontPage
GalaxyBot
Genieo
GermCrawler
Go!Zilla Go!Zilla
GoZilla Go-Ahead-Got-It
HTTrack GrapeFX
Haansoft HEADMasterSEO
Harvest
Havij Havij
Heritrix HonoluluBot
Hloader
Humanlinks Humanlinks
HybridBot IlseBot
Iblog Image Fetch
Id-search Image Sucker
InfoTekies Indy Library
InterGET InterGET
JamesBOT Internet Ninja
Jbrofuzz Jbrofuzz
JennyBot JennyBot
JetCar JetCar
JikeSpider Jetty
Joomla
JustView JustView
Kenjin Spider Kenjin Spider
LWP::Simple Keyword Density
Lanshanbot Kozmosbot
Leap
LeechGet
Lftp Lftp
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems
LinkWalker LinkWalker
Linkbot LinkpadBot
Linkdexbot
LinkextractorPro
LinksManager LinksManager
LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Lmspider MSFrontPage
Ltx71 Mail.RU_Bot
MIIxpc Majestic SEO
MJ12bot MarkWatch
MTRobot Mb2345Browser
Majestic12
Mata Hari
MeanPath Bot
Meanpathbot Meanpathbot
MegaIndex.ru Metauri
Microsoft Data Access Moblie Safari
Minefield
Mojeek Mojeek
Morfeus Fucking Scanner
NICErsPRO
NPbot
Name Intelligence Name Intelligence
Nameprotect Nameprotect
Navroad Needle
Nessus Nessus
Net Vampire
NetAnts NetAnts
NetSpider NetMechanic
NetZIP
Nettrack Nettrack
Netvibes Niki-bot
Nibbler Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja
Not Not
Nutch
Offline Explorer
Offline Navigator Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OpenVAS Openfind
Openvas OrangeBot
OutclicksBot OrangeSpider
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageGrabber PageGrabber
PageScorer PageScorer
PageThing.com Pavuk
Pandalytics PeoplePal
Panscient
Petalbot
Pi-Monster Pi-Monster
Picscout
Picsearch Picsearch
Piepmatz PictureFinder
Pimonster ProWebWalker
Pockey
Psbot
Pu_iN Pu_iN
QueryN Metasearch Pump
RSSingBot RSSingBot
RankFlex RankActiveLinkBot
RankingBot2 Rankivabot
RankurBot ReGet
Reaper RedesScrapy
SISTRIX RepoMonkey
SalesIntelligent Ripper
RocketCrawler
SEOkicks-Robot
SEOprofiler
SMTBot
ScanAlert
Scrapy
Screaming Screaming
Searchestate SearchmetricsBot
Semrush Seekport
SentiBot SemrushBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Shodan
Site Sucker
SiteLockSpider SiteLockSpider
SiteSnagger
Siteimprove Siteimprove
Sitevigil
Snake
Snoopy Snoopy
SocialRankIOBot Sqworm
Sociscraper
Sogou web spider
Sosospider
Spammen
Spinn3r
Sqlmap
Steeler Steeler
Sucker SurveyBot
Surfbot
Suzuran Suzuran
Swiftbot
Szukacz
TeleportPro TeleportPro
Telesoft
Telesphoreo Telesphoreo
The Intraformant Titan
TheNomad
Thumbor
Toata
Toweyabot Toweyabot
Tracemyfile Trendiction
Trendictionbot Trendictionbot
True_Robot
Turnitin
TurnitinBot
Twice
Typhoeus
URLy.Warning
V-BOT V-BOT
VCI VCI
Vagabondo VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
Voil Voil
Voltron
WBSearchBot
WISENutbot WISENutbot
WPScan
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WWW-Mechanize
WeSEE WWWOFFLE
Wallpapers/3.0
WallpapersHD
Web Auto Web Auto
Web Pix
Web Sauger
WebAuto
WebBandit
WebCollage WebCollage
WebGo IS WebGo IS
WebLeacher WebLeacher
WebPix WebSauger
WebStripper WebStripper
WebSucker
Webalta Webalta
Webshag Webshag
Webster Website Quester
WebsiteQuester
Whacker Whacker
Whatweb
Who.is Bot Who.is Bot
WinHTTrack Widow
Woobot Wonderbot
Xaldon WebSpider Wotbox
Xaldon_WebSpider Wprecon
Zade Xenu
Zauba
Zitebot Zitebot
ZmEu ZmEu
ZyBorg ZoomBot
ZumBot
adscanner adscanner
arquivo.pt archive.org_bot
cah.io.community autoemailspider
clark-crawler backlink-check
crawl.sogou.com check1.exe
eCatch coccocbot
facebookscraper com.plumanalytics
evc-batch
heritrix
internetVista monitor internetVista monitor
ips-agent
lwp-request lwp-request
lwp-trivial muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot page scorer
s1z.ru polaris version
scan.lol probe-image-size
seoscanners ripz
seocompany.store
serpstatbot serpstatbot
sitechecker.pro
siteripz
sp_auditbot sp_auditbot
spyfu
sysscan
tAkeOut
voyagerx.com
webpros.com webpros.com
webprosbot webprosbot
x09Mozilla xpymep1.exe
zauba.io zauba.io
zgrab zgrab

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
404enemy 404enemy
ADmantX 80legs
ALittle Client AIBOT
ASPSeek
Aboundexbot
AfD-Verbotsverfahren
AiHitBot AiHitBot
Aipbot Aipbot
AllSubmitter
Alligator Alligator
Anarchie Anarchie
Anarchy Anarchy99
Anthill Ankit
Apexoo Apexoo
Asterias
Atomseobot Atomseobot
AwarioSmartBot BDFetch
BLEXBot
BackDoorBot BackDoorBot
Backlink-Ceck BackStreet
BacklinkCrawler
Bandit Bandit
Black Hole Bigfoot
BlackWidow BlackWidow
Blow
BlowFish BlowFish
BotALot Boardreader
Buck Bolt
Buddy BuiltWith
CODE87 BunnySlippers
CCBot
Calculon Calculon
CazoodleBot CazoodleBot
CensysInspect Cegbfeieh
Claritybot CheTeam
Cliqzbot Citoid
Cloud mapping Cloud mapping
Cogentbot
Collector Collector
CopyRightCheck Cosmos
Copyscape
Craftbot
CrazyWebCrawler
Crescent Crescent
Curious Curious
CyotekWebCopy DBLBot
DataCha0s DataCha0s
Devil
Dirbuster
Discobot
Discoverybot Discoverybot
DittoSpyder DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot Download Wonder
Drip Drip
EMail Wolf EasyDL
EirGrabber Ebingbong
EroCrawler
Evil Evil
Exabot
Extractor Extractor
EyeNetIE EyeNetIE
FHscan FDM
FemtosearchBot
Firefox/7.0
FlashGet FlashGet
Foobot
Freeuploader
Fuzz Fuzz
FyberSpider
Fyrebot Fyrebot
GT::WWW GermCrawler
GalaxyBot GetRight
GetWeb GetWeb
Go-Ahead-Got-It
GoZilla
GrabNet GrabNet
Grabber
Grafula Grafula
GridBot
HEADMasterSEO
HTTP::Lite HTTP::Lite
Heritrix Heritrix
Humanlinks
IDBTE4M
IDBot
Iblog Iblog
Image Sucker
IndeedBot IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies Intelliseek
InternetSeer
Iria
Iskanie Iskanie
JamesBOT Keyword Density
JennyBot Lanshanbot
JetCar
Jetty
JikeSpider
JustView
Jyxobot
Kenjin Spider
Kinza
LNSpiderguy
LWP::Simple
Larbin
LeechFTP LeechFTP
LeechGet
LexiBot LexiBot
Lftp Lftp
LinkWalker LieBaoFast
LinksManager Linkdexbot
LinqiaMetadataDownloaderBot LinkextractorPro
LinqiaScrapeBot LinkpadBot
Lipperhey
Lipperhey Spider Lipperhey Spider
MFC_Tear_Sample Litemage_walker
MIIxpc MIDown tool
MJ12bot MJ12bot
MSIECrawler MSFrontPage
Mag-Net MTRobot
Magnet
Mail.RU_Bot
Majestic SEO
Majestic-SEO Majestic-SEO
MarkWatch MarkMonitor
Mass Downloader
Mata Hari
MauiBot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
MicroMessenger MicroMessenger
Microsoft URL Control Microsoft Data Access
Minefield Mojeek
Mojolicious Mojolicious
Musobot MolokaiBot
NICErsPRO Mr.4x3
Nameprotect Nameprotect
Navroad Navroad
NearSite NearSite
Needle Net Vampire
Nessus NetAnts
Netcraft NetSpider
NetZIP
Nettrack Nettrack
NimbleCrawler Netvibes
NextGenSearchBot
Niki-bot
Nikto
Nimbostratus
Ninja
Nmap
Not Not
Octopus
Offline Explorer Offline Explorer
Offline Navigator OnCrawl
Openfind
OrangeBot
OrangeSpider OrangeSpider
PECL::HTTP OutfoxBot
PHPCrawl
Page Analyzer Page Analyzer
PageGrabber
PageScorer
Pandalytics Pandalytics
Papa Foto PeoplePal
Pavuk Piepmatz
Petalbot Pockey
Pi-Monster ProPowerBot
Picscout Psbot
Pu_iN
Pump Pump
PxBroker RSSingBot
PyCurl RankActive
QueryN Metasearch
RankActiveLinkBot RankActiveLinkBot
RankFlex
RankingBot
RankingBot2
RealDownload RealDownload
RedesScrapy RebelMouse
Recorder
Ripper
RocketCrawler
Rogerbot Rogerbot
SBIder SEOlyticsCrawler
SEOkicks-Robot SEOprofiler
SISTRIX SalesIntelligent
ScanAlert
Scanbot Scanbot
Scrapy Searchestate
SearchmetricsBot SearchmetricsBot
Seekport Seekport
SentiBot SemanticJuice
Semrush
Seomoz Seomoz
SiteCheckerBotCrawler Site Sucker
SiteLockSpider SiteSucker
SiteSnagger Sitebeam
Siteimprove Siteimprove
SmartDownload Sitevigil
Snake
Snapbot Snapbot
Snoopy
Sociscraper
Sottopop
SpaceBison SpaceBison
SpankBot SpankBot
Spanner
Spinn3r Spinn3r
SputnikBot
Stripper
Sucker
SuperBot
SuperHTTP SuperHTTP
SurveyBot SurveyBot
Suzuran
T0PHackTeam T0PHackTeam
T8Abot Teleport
TeleportPro Telesoft
The Intraformant Thumbor
Titan Titan
Tracemyfile
Trendiction Trendiction
Trendictionbot Trendictionbot
Turingos True_Robot
Twice Turnitin
TwengaBot
URLy Warning
UnisterBot UnisterBot
V-BOT V-BOT
VCI
Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE VoidEYE
WASALive-Bot
WEBDAV WEBDAV
WISENutbot
WPScan
WeSEE WeSEE
Web Auto Web Fuck
Web Collage
Web Enhancer
Web Pix Web Pix
Web Sucker
WebAuto WebAuto
WebCollage
WebEnhancer
WebFuck WebFuck
WebGo IS
WebLeacher
WebReaper
WebWhacker WebWhacker
Website Quester WebsiteExtractor
Whack WebsiteQuester
Who.is Bot Webster
Widow Wonderbot
WiseGuys Robot
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu
Zade Zade
Zauba Zeus
ZmEu Zitebot
ZoomBot
ZoominfoBot
ZumBot ZumBot
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
backlink-check
check1.exe check1.exe
clark-crawler crawl.sogou.com
coccocbot crawler.feedback
crawler4j
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
eCatch evc-batch
instabid facebookscraper
gopher
heritrix
internetVista monitor
isitwp.com isitwp.com
lwp-request magpie-crawler
lwp-trivial
muhstik-scan
netEstate NE Crawler
oBot
page scorer page scorer
polaris version
ripz ripz
s1z.ru seobility
satoristudio.net
scan.lol
seocompany.store
seoscanners seoscanners
seostar
serpstatbot serpstatbot
sogouspider sexsearcher
sitechecker.pro
sp_auditbot sp_auditbot
sysscan
tAkeOut
webgains-bot webgains-bot
webmeup-crawler webprosbot
xpymep1.exe x22Mozilla
zauba.io zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
androids-store.com bestpornuha.com
blackplanet.com dynainbox.com
finuse.com martlinker.com
formulaantiuban.com news-readers.ru
isitwp.com nhl17coins.exblog.jp
solution4u.com okuos.com
tfxiq.com saitevpatorii.com
voronezh.xrus.org satellite.maps.ilovevitaly.com
zeroredirect7.com seo-platform.com
zigarettenonlinekaufen2.drupalgardens.com xportvusbdriver8i.snack.ws

952
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 08800.top
01apple.com 1000n1.ru
0daymusic.org 108shot.com
1-99seo.com 2728fb936f0.com
1-free-share-buttons.com 2daytrendingnews.com
1hwy.com 4kplayer.pl
1j7740kd.website 55wmz.ru
1webmaster.ml 57883.net
34.gs 5i2.net
4k-player.pl 76brighton.co.uk
4kepics.com 7fon.ru
4rent.ru 7yue.org
5-steps-to-start-business.com 8si.ru
51unlim.ru aad1f4acb0a373420d9b0c4202d38d94fa.com
70casino.online addbin.men
72-news.com adtech.fr
7minuteworkout.com adtiger.tk
8gold.com adult-shop.com.ua
8jn.dating-app.ru adult3dgames.com
91abcw.com adultgalls.com
adamoads.com advancedmassagebysara.com
adobereader-free.ru advertisingtag.net
adserver-e7.com agreda.pluto.ro
adservme.com agysya.ru
adultfullhd.com albuteroli.com
adultmeetonline.info alevinefesleri.com
adultwebhosting.info algarveglobal.com
afslankpillen2017nl.eu all-streaming-media.com
akuhni.by allfinweb.com
alert-fjg.xyz alphavisions.net
allboard.xobor.de android-systems.ru
alldezservice.kz
allnews.md
alltheviews.com
alphahoverboards.com
alyeskaresort.com
amoi.tn
amospalla.es
amyfoxfitness.com
ananumous.ru
android-vsem.org android-vsem.org
anlimebel.kiev.ua animaltoplist.com
anime.dougasouko.com
anonymizeme.pro anonymizeme.pro
anonymousfox.co anonymousfox.co
anti-virus-removal.info apilinkswiftco-a.akamaihd.net
apartment.ru apioasisspacenet-a.akamaihd.net
apelsinnik.website apxeo.info
apiluckyleapnet-a.akamaihd.net arabgirls.adultgalls.com
apimegabrowsebiz-a.akamaihd.net arabsexxxtube.com
app-ready.xyz arenda-yeisk.ru
appiq.mobi artdeko.info
arendakvartir.kz artpicso.com
arpelsreplica.xyz asophoto.com
arquivo.pt asscenihotosidea.blogspot.co.za
atlant-auto.info auto-moto-elektronika.cz
automatic-seo.com autogrep.ru
automobile-spec.com autotop.com.ua
babyfactory.fr avtoarenda.by
backiee.com azartmsl.com
battle.net back.dog
bensbargains.net bagcionderlab.com
bestcalovebracelet.cn balois.worldbreak.com
bestcurs.org bannerspace.com
bestssaker.com barbourjackets.online
bitcoin-ua.top beachtoday.ru
biteg.xyz best-seo-offer.com
bkgr.se bestwaystogetpaid.us
blog-hits.com bibys.com
blogs.rediff.com bikini-image.com
bodybuilding-shop.biz binomo.kz
boobsimge.com bird1.ru
bookmark4you.com blackcurranthumidifiers.site
bosefux.esy.es bloggerads.net
bugshoot.cn bluerobot.info
bum.com.ru board.f00d.de
buyantiviralwp.com boc.kz
buytizanidineonline.blogspot.com boleznikogi.com
buzzurl.jp bolezniorganov.ru
cacheimages.com bongacams.com
calc-for-credit.ru bouda.kvalitne.cz
callejondelpozo.es bpro1.top
callstevens.com brewdom.ru
canrioloadm.gq brus.city
carabela.com.do buketeg.xyz
carsplate.com butstrap.space
buyscabiescream.com
c1.onedmp.com
calvet.altervista.org
candy-glam-hp.com
canoncdriverq3.pen.io
carapics.com
cardiosport.com.ua
cartierbracelet.xyz cartierbracelet.xyz
cartierreplica.xyz cartierlove2u.xyz
castingbank.ru cartierreplica.win
centrumcoachingu.com centrdebut.ru
cfsrating.sonicwall.com centre-indigo.org.ua
checkpagerank.net cheapjerseysa.com
cheapmarmot.online
cheapmoncler.win
chidporn.com chidporn.com
chinaelitecheapjerseys.com chocolateslim-original.com
chocolateslim-en-espana.com chocolateslim-u-srbiji.com
cialgenisrx.com ci.ua
classiquebijoux.ru cienum.fr
clicksor.com cjs.com.ru
com-secure.download clayvasedesigns.tk
compliance-alexa.xyz click2pawn.com
compliance-brian.top clickbank.net
compliance-irvin.xyz clickhype.com
compliance-jessica.top clicksotrk.com
compliance-julia.top clubfashionista.com
compliance-olga.top compliance-checker.info
compucelunlock.net compliance-elena.xyz
comsysnet.com compliance-george.top
compliance-viktor.top
contentsexpress.com contentsexpress.com
cookingmeat.ru cookie-law-enforcement-aa.xyz
coolwallpapers-hd.com copyrightclaims.org
coolwallpapers4k.info cowlmash.com
coprofam.org cranly.net
cowblog.fr credit-online.ws
customergrowthsystems.com cubs-jerseys.us
cymbaltaandtizanidine.blogspot.com cymbaltaandtizanidine.blogspot.com
czat.wp.pl cypernhuset.se
d2jsp.org d-black.bz
delta-line.men datadepths.com
demian.kz dentalpearls.com.au
dentfidemountpreach.tk dev-seo.blog
deplim.com diffbot.com
descargar-musica-gratis.net dktr.ru
div.as dms-sw.ru
diy-handmade-ideas.com documentbase.net
dneprsvet.com.ua domain-submit.info
doctissimo.fr domaineaneblanc.com
domainanalyzing.xyz domination.ml
domainsatcost.com doreenblog.online
domcran.net dorratex.tn
donvito.unas.cz drugspowerstore.com
dosug-lux.ru
doublepimp.com
doxysexy.com
doyouknowtheword-flummox.ml
drupa.com drupa.com
dstroy.su dverimegapolis.ru
e-collantes.com e90post.com
easytuningshop.ru easyshoppermac.com
electronicadirect.com econom.co
elektrozigaretten1.postbit.com ege-essay.ru
emergencyneeds.org elektrischezi.canalblog.com
eng-lyrics.com eralph.tk
etm-consult.de erotikchat-24.com
everflora.ru euroskat.ru
exchanges-bet.com evaashop.ru
excitacion.info event-tracking.com
extractorandburner.com exmasters.com
extstat.com ez8motelseaworldsandiego.com
f-loaded.de ezigarettekaufen1.hpage.com
facebook-mobile.xyz farm26.ru
facialporntube.com farmingworm.com
fastfixing.tech farmprofi.net
fearcrow.com fashionavenuegame.com
fefo.gdn favornews.com
filesvine.com filmci.pro
findpik.com firstdrugmall.ru
fitness-video.net flex4launch.ru
flooringinstallation-edmonton.com foodgid.net
forex-procto.ru footbalive.org
forminecrafters.ru forumprofi.de
formseo.com free-deals.faith
free-share-buttons.blogspot.com free-today.com
freemaintenancesysforpcandmac.top freefoto.ca
freshberry.com.ua freeuploader.ml
fsakhalin.ru freshdz.com
funcrushgames.com freshmac.space
fx-brokers-review.com froggytube.com
gardene.ru ftns.ru
gaz-voshod.ru g.starmoe.xyz
gazoblok.net.ua gamedayhouse.com
gdcentre.ru gate5.co.za
geilehausfrauen.net gaygalls.net
genetworx.com gayxperience.com
gidonline.one gdzkurokam.ru
glall.ru geckoandfly.com
globalscam.ga gemgrab-a.akamaihd.net
globetrotting-culture.ru gepezz.info
goblacked.com get-your-social-buttons.info
greatdealshop.com getdot.ru
gribokstop.com ghostvisitor.com
growmyfunds.ca gigapeta.com
gsasearchengineranker.top glogow.pl
guigyverpo.cf glopages.ru
gwhwpxbw.bloger.index.hr gloverid.site
hahashka.ru golden-catalog.pro
hamilton.ca goodwriterssales.com
hauleddes.com googlsucks.com
hccoder.info gq-catalog.gq
helicalpile.us graphid.com
herehloadibs.cf gratuitxblcodes.com
hermesbelts.xyz gsbs.com.ua
hermesreplica.win guardlink.com
homedecoguide.info guardlink.org
houseofgaga.ru guruofcasino.com
hoverboard360.at gz2.bbsoldes.fr
hoztorg-opt.ru hacktougroup.ru
hplaserjetpdriver8y.pen.io handicapvantoday.com
hut1.ru hao123.com
ikritikimou.gr hatdc.org
ilovevitaly.ru hatedriveapart.com
imagez.co hdimagegallery.net
imperiafilm.ru hdpixent.com
inboxdollars.com healgastro.com
incest-ru.com hermesbracelets.xyz
infobabki.ru hkdiiohi.skyrock.com
intellektmedia.at hkladys.com
interferencer.ru homegardenlova.com
internetartfair.com honyaku.yahoofs.jp
invitefashion.com horoshieokna.com
iqupdatetmz.win hothor.se
irkutsk.zrus.org houtings.xyz
ispac.org hoverboardforsaledirect.com
istripper.com howlongdoestizanidinestayinyoursystem.blogspot.com
iusstf.org hscsscotland.com
izismile.com hspline.com
izoll.ru hyhj.info
jbl-charge.info iblogpress.xyz
je7.us ie.57883.net
job.icivil.ir ifmo.ru
jobgirl24.ru igithab.com
kamagragelusa.net inbabes.sexushost.com
keki.info internetproviderstucson.com
keywesthideaways.co iplusbit.blogspot.co.za
kino-rf.ru itis4you.com
kinosed.net jerseychinabizwholesale.us
kinozapas.com jmat.cn
kinozapas.org johnnyhaley.top
kol-energo.ru karting196.ru
keenoutlet.online
kinogonew.ru
klikbonus.com
kmd-pto.ru
kndxbkdx.bloger.index.hr
koleso24.com.ua
konpax.com
kosova.de kosova.de
ktotut.net krasivye-devushki.net
kumuk.info labafydjxa.su
la-fa.ru landmania.ru
landoftracking.com laparfumotec.com
lasvegaslockandsafe.com laudit.ru
lazy-z.com lecbter-relationships.ga
lightinghomes.net lechtaczka.net
liquimondo.com ledis.top
listiseltemournlan.gq lego4x4.ru
liky.co.ua
littleberry.ru
littlecity.ch
livejasmin.com
livingcanarias.com livingcanarias.com
locationdesaison.com lmrauction.com
loftdigital.eu lombia.com
lolnex.us lonfon.xyz
luxmagazine.cf long-beach-air-conditioning.com
m.b00kmarks.com lovelycraftyhome.com
magento-crew.net lsex.xyz
maladot.com macdamaged.tech
makemoneyonline.com
makenahartlin.com
manifestation.betteroffers.review manifestation.betteroffers.review
manimpotence.com manve.info
mb140.ru marinetraffic.com
meble-bogart.info masthopehomes.com
mecash.ru maxxtor.eu
med-zdorovie.com.ua mebel-vstroika.ru
medicines-choice.com meetlocalchicks.com
megaapteka.ru mielec.pl
megagrabber.ru mirmedinfo.ru
mexicotravelnet.com mlvc4zzw.space
micasainvest.com mmgq.ru
minharevisao.com mmog-play.ru
minyetki.ru mobicover.com.ua
misandesign.se mobot.site
misslike.ru mobsfun.net
missvietnam.org movpod.in
moesonce.com moyaterapiya.ru
monthlywinners.com mug-na-chas-moscow.ru
morlat.altervista.org musicpro.monster
moviezbonkerssk.cf mxgetcode.com
myanyone.net mybackgroundlandscape.blogspot.com
mybinaryoptionsrobot.com myplaycity.com
mylovelibrabry.com myshopmatemac.com
mysex21.com namenectar.com
mysexpics.ru
mystats.xyz
nalogovyy-kodeks.ru
naperehresti.info naperehresti.info
nardulan.com nardulan.com
narosty.com narkologiya-voronezh.ru
neodownload.webcam neodownload.webcam
nero-us.com netoil.no
netallergy.ru new-post.tk
netanalytics.xyz newfilmsonline.ru
nfljerseysforsalewholesaler.com nfljerseyscheapchinabiz.com
nightvision746.weebly.com nhl09.ru
notasprensa.info njkmznnb.ru
nudepatch.net nonameread45.live
o-o-11-o-o.com nordstar.pro
o-o-8-o-o.com nosecret.com.ua
oakridgemo.com nottyu.xyz
obiavo.net noumeda.com
ochistka-stokov.ru nuker.com
oda.as obiavo.ru
one.net.in oecnhs.info
onetravelguides.com ofanda.com
online-x.ru okonich.com.ua
onlywoman.org okuos.com
ontargetseo.us olgacvetmet.com
ooo-gotovie.ru onblastblog.online
oranga.host.sk onlineporno.site
ourtherapy.ru ooo-olni.ru
oxford-book.com.ua optitrade24.com
oz-offers.com orsonet.ru
parajumpersoutlet.online page2rss.com
patterntrader-en.com paradontozanet.ru
pavlodar.xkaz.org parajumpersjakkesalgnorge.info
pechikamini.ru paramountmarble.co.uk
pb-dv.ru
pcboa.se
pdamods.ru
penisvergrotendepillennl.ovh penisvergrotendepillennl.ovh
perm.xrus.org picsearch.com
phuketscreen.com pinwallpaper.xyz
pic-re.blogspot.com piratecams.com
pic2fly.com pixgood.com
picsforkeywordsuggestion.com plastweb.ru
pipki.r.acdnpro.com poffet.net
pistonclasico.com poquoson.org
piulatte.cz pornoreino.com
plaff-go.ru pornozhara.com
pokemongooo.ml portadd.men
polimga.pw
pooleroadmedicalcentre.co.uk
popmarker.com
popugaychiki.com
pornhub-forum.ga
pornohub.me
pornoted.com
postclass.com
powitania.pl powitania.pl
princeadvantagesales.com praisong.net
producm.ru prchecker.info
presleycollectibles.com
prize44.com
productarium.com
profolan.pl
prohoster.info
prointer.net.ua prointer.net.ua
pron.pro prom23.ru
purplesphere.in promover.org
quelle.ru pronorm.fr
quickloanbank.com prostitutki-kharkova.org
radiodigital.co prpops.com
rankscanner.com pssucai.info
ranksonic.org puteshestvennik.com
rarbg.to qor360.com
ratemodels.net quick-offer.com
rcb101.ru ranksays.com
realitykings.com ranksonic.info
redirectme.net rasteniya-vs-zombi.ru
referencemoi.com refererx.com
refudiatethissarah.info remmling.de
registratciya-v-moskve.ru remont-komputerov-notebook.ru
remontbiz.ru rentehno.ru
renecaovilla.online rezeptiblud.ru
renecaovillasale.online rfd-split.hr
rent2spb.ru robotixix.com
ring4rhino.com roofers.org.uk
roll123.com royal-investments.net
rumamba.com ruf777.com
ruscopybook.com russintv.fr
sack.net saugatuck.com
sale-japan.com sbdl.no
scanner-ivan.top scanner-jessica.top
scanner-julia.top search.1and1.com
scanner-mary.top
scanner-viktor.top
scrapy.org
sdsjweb.com
searchinquire.com searchinquire.com
seasaltwithfood.com sei80.com
seccioncontrabajo.com senger.atspace.co.uk
seoexperimenty.ru seo-prof1.xyz
seomarketings.online seokicks.de
seorussian.ru sex-tracker.com
seozoom.it sexblog.pw
sex-dating.co sexvporno.ru
sex-sex-sex5.com sexytrend.ru
sexphoto.site sfj-ror.no
sexpornotales.net sherlock.se
sexy-pings.com ship-marvel.co.ua
shijian.ac.cn shoesonlinebuy.cn
shitting.pro shop-electron.ru
shlyahten.ru shopfishing.com.ua
shopperifymac.com
shtora66.ru
shymkent.xkaz.org
sibtest.ru
sientalyric.co sientalyric.co
silvercash.com silktide.com
simplepooltips.com site3.free-share-buttons.com
sinel.info sitebeam.net
slkrm.ru smichovbike.cz
slonechka.ru snjack.info
slujbauborki.ru socialmonkee.com
smailik.org soda.media
snip.tw sotkal.lark.ru
socialbutton.xyz souvenir.cc
socialsignals24.com soviet-portal.do.am
sofit-dmd.ru spabali.org
solitaire-game.ru space-worry.ml
souvenirua.com spb.afora.ru
spamnuker.com spravka130.ru
spitfiremusic.com star61.de
spl63.fr staticfs.host
spravka-medosmotr.ru stefanbakosab.se
spy-app.info strana-krasoty.ru
spylog.com streetfire.net
srdrvp.com supernew.org
srecorder.com svetlotorg.ru
sstroy44.ru swinon.site
stair.registrydomainservices.com
stariy-baku.com
steamoff.net
stroimajor.ru
stuffpride.com
sucsesofinspiration.com
sukirgenk.dvrlists.com
superiends.org
surflinksmedical.com
svtrd.com
swagbucks.com
tacbelarus.ru tacbelarus.ru
tastyfoodideas.com tackletarts.co
tds-advert005.info taraz.xkaz.org
tecnoteakviareggio.it td-33.ru
teenbbw.yopoint.in td-l-market.ru
telemetryverification.net tengohydar.tk
tetracsaudi.com thefotosgratis.eu
theendivechronicles.com
thegalerie.eu
thepokertimer.com thepokertimer.com
tiandeural.ru thetattoohut.com
tizanidinehcl2mgsideeffects.blogspot.com tizanidine4mg.blogspot.com
tizanidineingredients.blogspot.com tizanidineandcipro.blogspot.com
tizanidinepain.blogspot.com tizanidineandgabapentin.blogspot.com
tizanidinezanaflex.blogspot.com tizanidinedosageforsleep.blogspot.com
tongkatmadura.info tizanidinehcl4mgtabinfo.blogspot.com
tizanidineorflexeril.blogspot.com
tizanidineshowupondrugtest.blogspot.com
tn811.us
top-karkas.ru top-karkas.ru
topcasinoratings.ru top-l2.com
toursmaps.com tourcroatia.co.uk
trafaret74.ru trudogolik.net
trahvid.com turbabitload.weebly.com
traxdom.ru tversvet.ru
trustaffs.com twitlinks.com
tsan.net twodollarshows.com
tsyndicate.com uginekologa.com
tuckermktg.com uk-zheu20.ru
tvnewsclips.info ukrtvir.com.ua
txxx.com umekana.ru
ublaze.ru umityangin.net
uggbootsoutletsale.us undergroundcityphoto.com
uhod-za-sobakoj.ru unrealcommander.biz
ultimateclassicrock.com uptime-as.net
univerfiles.com uptime-gamma.net
uptimebot.net uptime.com
ural-buldozer.ru usdx.us
usiad.net vadimkravtcov.ru
valuado.com ventelnos.com
varbergsvind.se veselokloun.ru
varikozdok.ru viagengrarx.com
vchulkax.com videochat.ph
venerologiya.com videogamesecrets.com
versaut.xxx-cam.webcam videospornogratisx.net
vgoloveboli.net vltai.com
via-energy-cumpara.com voloslove.ru
via-energy-order.com voluumtrk.com
viagroid.ru vostoktrade.info
video-camer.com vozbujdenie.com
video-hollywood.ru vut.com.ru
viel.su vzlom-na-zakaz.com
vigrx-original.ru w3data.co
vilingstore.net wait3sec.org
vizag.kharkov.ua walkme.com
voluumtracker1.com watchdogs-2.ru
vpnhowto.info we-are-gamers.com
walpaperlist.com webextract.profound.net
wdrake.com webinstantservice.com
web-analytics.date webpromotion.ae
webads.co.nz webscouter.net
webjam.com website-analytics.online
weblibrary.win webtherapy.ru
wedding0venues.tk westsextube.com
where-toget.com
whiteproduct.com
wholinkstome.com
whosonmyserver.com whosonmyserver.com
witherrom55.eklablog.fr williamrobsonproperty.com
wonderfulflowers.biz word-vorlagen.xyz
worldis.me www.ehscloud.cn
wowas31.ucoz.ru www.jbetting.com
woweb.com.ua x-diesel.com
wpsecurity.website xep.info
ww1943.ru xfire.com
wyniki-lista.pl xgftnlrt.bloger.index.hr
wzgyyq.com xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
x-stars.ru xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn----itbeirbjbi7bc6bh2d.xn--p1ai xn----9sbebi2bvzr7h.xn--p1ai
xn--80aikhbrhr.net xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--80ajbshivpvn2i.xn--p1ai xn--90acenikpebbdd4f6d.xn--p1ai
xn--b1addnj3cah.xn--p1ai xn--d1abj0abs9d.in.ua
yhit.press xn--l1aengat.xn--p1ai
ynymnwbm.bloger.index.hr xn--lifehacer-1rb.com
yoluxuryevents.com xxx-cam.webcam
youbloodyripper.com xxxasianporn.net
your-bearings.com xzlive.com
yourserverisdown.com yougetsignal.com
yourtemplatefinder.com
yousense.info yousense.info
za-music.mymobiplanet.com youtubedownload.org
zaimhelp.ru yunque.pluto.ro
zakazfutbolki.com zastenchivosti.net
zawyna.ua zed21.net
zeleznobeton.ru zeikopay.com
zero1.it zeroredirect12.com
zeroredirect10.com zeroredirect2.com
zixizop.net.ru zoogdiesney.com
zooggames.com zscaler.net
zootoplist.com zzbroya.com.ua

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1001watch.com.ua 1-99seo.com
10bet.com 12masterov.com
4rent.ru 1j7740kd.website
4webmasters.com 2nt.ru
9med.net 3w24.com
a14download.com 4ureyesonly.com
abclauncher.com 7minuteworkout.com
adloads.net a.frcls.fr
albamargroup.com acgs.tk
alfa-medosmotr.ru advocatemsk.ru
alienwheels.de aero2.ru
allnews24.in affordablewebsitesandmobileapps.com
alyeskaresort.com aibolita.com
amateurmatch.com allkrim.com
apelsinnik.website allornamenti.com
appearance-cool.com allpdfmags.net
artparquet.ru anatomiy.com
ascat.porn apisecretsaucebi-a.akamaihd.net
asrv-a.akamoihd.net arpelsreplica.xyz
avtolombard-krasnodar.com autoua.top
baersaratov.ru avek.ru
bank.uz ayakino.net
bannerconnect.net batanga.net
bannerspace.com beremenyashka.com
bayareaaeroclub.org bestimagecoollibrary.com
berlininsl.com bestplacetobuyeriacta.jw.lt
bestempresas.es bezsporno.ru
bestwaystogetpaid.us bidbuy.co.kr
biznesluxe.ru bildsuche.ru
blackwitchcraft.ru bimatoprost-careprost.com
bmsco.co bitcoinpile.com
brimstonehillfortress.org blackcurranthumidifiers.site
brus-vsem.ru board.f00d.de
bugshoot.cn boobsimge.com
bupropion-sr-150-mg.us bristolhotel.com
buron.pw buynorxx.com
bus-offer.com camel-beach.com
buy-meds24.com
buyfriend.ru
buyscabiescream.com
calcularpagerank.com
callejondelpozo.es
canrioloadm.gq canrioloadm.gq
casinorewards.info cartierlove2u.com
cgi2.nintendo.co.jp cartujano-pre.de
chastnoeporno.com centrdebut.ru
chatroulette.video chatroulette.si
chidporn.com cheapjerseysfootballshop.com
clean-start.net chuckguilford.com
clippingphotoindia.com ci.ua
compliance-alexa.top cienum.fr
compucelunlock.net club-musics.ru
coslab.club compliance-elena.top
crackguru.tk costablue.xyz
dairyindia.in crest3d.ru
date-withme.com cubs-jerseys.us
dawlenie.com db.speedup.gdn
discovertreasurenow.com delayreferat.ru
divisioncore.com deniven.1bb.ru
drev.biz dev.citetick.com
eaplay.ru djonwatch.ru
easycommerce.cf doctormakarova.ru
elektrischezi.canalblog.com domaincdn.xyz
emergencyneeds.org doxyporno.com
ereko.ru easyshoppermac.com
erotiktreff24.info ecookna.com.ua
essay-writing.work encodable.com
exportshop.us engines-usa.com
ezigarettekaufen2.mpbloggar.se ero-advertising.com
ezigarettenkaufen1.deviantart.com erotik-kostenlos.net
fashion.stellaconstance.co ezigarettekaufen2.blox.pl
filerockstar298.weebly.com ezigarettekaufen2.yolasite.com
financeloan.us ezigarettenshop2.postbit.com
findthe.pet factorynightclub.com
flavors.me fanrto.com
formulaf1results.blogspot.com farm26.ru
forum.doctissimo.fr felizporno.com
forex.osobye.ru
forum.tvmir.org forum.tvmir.org
forum20.smailik.org frequiry.com
freevpn.space freshdz.com
front.ru fullgirl.ru
garciniaxt.us fym.com.ru
gdzkurokam.ru g7m.pl
ge0ip.com gambarkataku.co
geileweiber.tk genericlowlatencyasiodriverhq.aircus.com
getyourimage.club gnuetella.com
golden-praga.ru goodwriterssales.com
googlepositions.com
googlsucks.com
gopro-online.info
gosmeb.ru gosmeb.ru
gowreckdiving.com growboxbl.ru
handicapvansarea.com gtopstats.com
hidefiles.org hccoder.info
hmmm.cz hd-filmy.net
homeart.space hdwalls.xyz
hop.clickbank.net healthcarestore.info
housekuba.org hem.passagen.se
htmlcorner.com hentai-manga.porn
hustoon.over-blog.com hjaoopoa.top
hydropump.su hostnow.men
ibb.com.ua hotchatdate.com
ibmdatamanagement.co ideoworld.org
images-graphics-pics.com ikearugs.xyz
imediadesk.com investmac.com
immigrational.info investsuccess.org
incomekey.net isoveti.ru
istripper.com itbc.kiev.ua
james13prix.info itrevolution.cf
jav-fetish.site itsdp3.com
juliaworld.net iwanttodeliver.com
junglenet-a.akamaihd.net jav-idol.com
jongose.ninja
jyvopys.com jyvopys.com
kam-dom.ru
kamagragelusa.net
karpun-iris.ru karpun-iris.ru
keywordspay.com ketrzyn.pl
kicknights.gq
kit-opt.ru kit-opt.ru
lalalove.ru kol-energo.ru
laptop-4-less.com krasivoe-hd.com
law-check-eight.xyz kursy-ege.ru
law-check-seven.xyz kurwa.win
laxob.com lawyersinfo.org
llastbuy.ru lerporn.info
long-beach-air-conditioning.com linkbux.ru
lostaruban.com lion.snapmap.us
louboutinbooties.xyz lolnex.us
maranbrinfo.com.br lonely-mature.com
mebeldekor.com.ua longadventure.com
mediafresh.online manipulyator-peterburg.ru
missclub.info mansiondelrio.co
mobicover.com.ua manuscript.su
moviemail-online.co.uk marmotstore.online
mxgetcode.com maslenka.kz
national-today-winning-winner.club mediaoffers.click
nbsproject.ru merryhouse.co.uk
nextlnk12.com milblueprint.com
ntic.fr mirzonru.net
nw-servis.ru mompussy.net
offf.info moneytop.ru
one-gear.com morocco-nomad-excursions.com
orsonet.ru moscow-clining.ru
outshop.ru mp3s.club
oyster-green.com musicdaddy.net
panchro.co.uk musicktab.com
peekyou.com musicpro.monster
pekori.to mymobilemoneypages.com
perl.dp.ua mypets.by
physfunc.ru naturtreenspicerx.pw
pierrehardysale.online neks.info
podshipniki-ntn.ru nevansk.ru
poligon.com new-apps.ru
polybuild.ru nyfinance.ml
popander.mobi o-o-8-o-o.ru
popcash.net online247.ml
pornhive.org origin-my.ru
pornogratisdiario.com osnova3.ru
pornoklad.ru ostroike.org
pornosee.info ozas.net
pornosexrolik.com partner-trustworthy.men
pornpost.in photo.houseofgaga.ru
pregnant.guru picphotos.net
privatov-zapisi.ru picturesmania.com
piski.top
pixell.club
pizza-tycoon.com
pobeiranie.pl
poem-paying.gq
porn-w.org
porngalleries.top
pornoinn.com
pp-budpostach.com.ua
pro-poly.ru pro-poly.ru
prokotov.com
promover.org
prostitutki-kharkova.org
prostitutki-tolyatti.org prostitutki-tolyatti.org
pyramidlitho.webs.com quickcashlimited.com
quebec-bin.com rabot.host.sk
ranking2017.ga rankscanner.com
regionshop.biz rasteniya-vs-zombi.ru
registry-clean-up.net rcpmda.ikan1080.xyz
remont-ustanovka-tehniki.ru renewablewealth.com
roma-kukareku.livejournal.com research.ifmo.ru
rome2rio.com ringporno.com
rozalli.com rniaeba.ga
ruclicks.com robotixix.com
rusprostitute.com rockma.se
rutor.group roleforum.ru
rvtv.ru rossmark.ru
sadaholding.com sack.net
sardinie.us saecsa.co
sbf441.com samolet.fr
sbricur.com scanner-fred.top
scat.porn schalke04fc.info
seasaltwithfood.com scoopquest.com
selectads.men senger.atspace.co.uk
seocdvig.ru seokicks.de
sexgalleries.top seonetwizard.com
sexyali.com sexkvartal.com
share-buttons-for-free.com sexpartygirls.net
shemalegalls.blogporn.in sexpornotales.com
shoppytoolmac.com sexsearch.com
shtaketniki.kz shell-pmr.ru
simul.co sibecoprom.ru
snyatie-lomki-v-stacionare.ru simpoed.ufop.br
space4updating.win softwaretrend.net
stanthonyscatholicchurch.org space2019.top
stretchingabuckblog.com spy-app.info
stroydetali.ru standardchartered-forex.com
sudexpert66.ru steebook.com
superstats.com strv.se
the-torrent-tracker.blogspot.com sukirgenk.dvrlists.com
supernew.org
swagbucks.com
swingerseiten.com
tastyfoodideas.com
technika-remont.ru
testbotprocessor44.com
tetracsaudi.com
theallgirlarcade.com theallgirlarcade.com
thegameriders.com time-japan.ru
tizanidinehcl2mgsideeffects.blogspot.com titan-ads.life
tomatis.gospartner.com
torrnada.ru torrnada.ru
trackmedia101.com tracklead.net
truemfilelj.gq tuberkulezanet.ru
trustedhealthtips.com twincitiescarservice.com
tvorozhnaja-zapekanka-recept.ru
twittruth.com
udsgame.online
ukrtextbook.com
ultimateclassicrock.com ultimateclassicrock.com
univerfiles.com validdomain.xyz
valkiria-tk.ru valsalud.com
vapeface.club vegascosmetics.ru
viandpet.com vellings.info
videochat.tv.br viberdownload10.com
villakohlanta.nu vkgaleria.com
voluumtrk.com voprosotvet24.ru
vzubah.com wanker.us
wallpaperaccess.com webenlace.com.ar
wapsite.me webmonetizer.net
waterpurifier.club websitevaluebot.com
webnode.me welcomeauto.ru
wejdz-tu.pl westen-z.life
whatsupinfoley.com wheelchairliftsarea.com
wildcattube.com wildnatureimages.com
wordkeyhelper.com wladimirpayen.com
worst-sites.online wowcasinoonline.ooo
wygraj-teraz.com wrz0iuebwhp5fg.freeddns.com
wzgyyq.com xlovecam.com
xchangetrak.com
xchat26.myfreecams.com
xjrul.com
xlolitka.com
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----ctbigni3aj4h.xn--p1ai xn----ctbigni3aj4h.xn--p1ai
xn--b1ag5cfn.xn--p1ai yhit.press
xn--h1ahbi.com.ua yottos.com
xnxx-n.com yousense.info
xtubeporno.net zawyna.ua
xuki.us zeroredirect12.com
ytmnd.com znakomstva-piter78.ru
zastroyka.org znaturaloriginal.com
zocaparj.kz zojirushi-products.ru
zryydi.com zs2vm.top

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.183 0; 10.1.0.49 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:54 UTC 2022 ### Updated: Sun Feb 20 10:39:12 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2931 ### Version: V4.2022.02.2932
### Updated: Sun Feb 20 10:31:54 UTC 2022 ### Updated: Sun Feb 20 10:39:12 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##