vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2024.09.4779 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2024-09-28 05:51:27 +00:00
parent 1c5f7e36ab
commit b856299b06
38 changed files with 281 additions and 263 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.09.4778 #### Version: V4.2024.09.4779
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [666](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [667](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.09.4778 #### Version: V4.2024.09.4779
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [666](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [667](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -14,9 +14,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2024.09.4778 #### Version: V4.2024.09.4779
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [666](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [667](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_generator_lists/bad-user-agents.list
View file

@ -1,4 +1,3 @@
SeoCherryBot
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -424,6 +423,7 @@ Semrush
SemrushBot SemrushBot
SentiBot SentiBot
SenutoBot SenutoBot
SeoCherryBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz Seomoz

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2024.09.4778 #### Version: V4.2024.09.4779
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [666](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [667](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

7
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

7
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.09.4778 ### Version: V3.2024.09.4779
### Updated: Fri Sep 27 22:01:11 UTC 2024 ### Updated: Sat Sep 28 05:47:45 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -573,6 +573,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

7
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

1
dev-tools/_robots_input/robots-input.txt
View file

@ -423,6 +423,7 @@ Semrush
SemrushBot SemrushBot
SentiBot SentiBot
SenutoBot SenutoBot
SeoCherryBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz Seomoz

2
dev-tools/buildnumber
View file

@ -1 +1 @@
4778 4779

7
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.09.4778 ### Version: V3.2024.09.4779
### Updated: Fri Sep 27 22:01:11 UTC 2024 ### Updated: Sat Sep 28 05:47:45 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -573,6 +573,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

7
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.172.29.188 0; 20.97.191.33 0;

7
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -484,6 +484,7 @@
"~*(?:\b)SemrushBot(?:\b|)" 0; "~*(?:\b)SemrushBot(?:\b|)" 0;
"~*(?:\b)SentiBot(?:\b|)" 0; "~*(?:\b)SentiBot(?:\b|)" 0;
"~*(?:\b)SenutoBot(?:\b|)" 0; "~*(?:\b)SenutoBot(?:\b|)" 0;
"~*(?:\b)SeoCherryBot(?:\b|)" 0;
"~*(?:\b)SeoSiteCheckup(?:\b|)" 0; "~*(?:\b)SeoSiteCheckup(?:\b|)" 0;
"~*(?:\b)SeobilityBot(?:\b|)" 0; "~*(?:\b)SeobilityBot(?:\b|)" 0;
"~*(?:\b)Seomoz(?:\b|)" 0; "~*(?:\b)Seomoz(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.172.29.188 0; 20.97.191.33 0;

7
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.172.29.188 0; 20.97.191.33 0;

7
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:10 UTC 2024 ### Updated: Sat Sep 28 05:47:44 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -576,6 +576,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)SemrushBot(?:\b)" 3; "~*(?:\b)SemrushBot(?:\b)" 3;
"~*(?:\b)SentiBot(?:\b)" 3; "~*(?:\b)SentiBot(?:\b)" 3;
"~*(?:\b)SenutoBot(?:\b)" 3; "~*(?:\b)SenutoBot(?:\b)" 3;
"~*(?:\b)SeoCherryBot(?:\b)" 3;
"~*(?:\b)SeoSiteCheckup(?:\b)" 3; "~*(?:\b)SeoSiteCheckup(?:\b)" 3;
"~*(?:\b)SeobilityBot(?:\b)" 3; "~*(?:\b)SeobilityBot(?:\b)" 3;
"~*(?:\b)Seomoz(?:\b)" 3; "~*(?:\b)Seomoz(?:\b)" 3;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -484,6 +484,7 @@
"~*(?:\b)SemrushBot(?:\b|)" 0; "~*(?:\b)SemrushBot(?:\b|)" 0;
"~*(?:\b)SentiBot(?:\b|)" 0; "~*(?:\b)SentiBot(?:\b|)" 0;
"~*(?:\b)SenutoBot(?:\b|)" 0; "~*(?:\b)SenutoBot(?:\b|)" 0;
"~*(?:\b)SeoCherryBot(?:\b|)" 0;
"~*(?:\b)SeoSiteCheckup(?:\b|)" 0; "~*(?:\b)SeoSiteCheckup(?:\b|)" 0;
"~*(?:\b)SeobilityBot(?:\b|)" 0; "~*(?:\b)SeobilityBot(?:\b|)" 0;
"~*(?:\b)Seomoz(?:\b|)" 0; "~*(?:\b)Seomoz(?:\b|)" 0;

16
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BlowFish BuzzSumo
Firefox/7.0 Firefox/7.0
Heritrix Grafula
LNSpiderguy
Lmspider Lmspider
QueryN Metasearch Musobot
SearchmetricsBot Ripper
Sqlmap Whatweb
WEBDAV Xaldon WebSpider
Zermelo scalaj-http
com.plumanalytics

174
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
01h4x.com ADmantX
Aboundexbot AIBOT
Anarchy Alexibot
Backlink-Ceck Alligator
Battleztar Bazinga Anarchie
Boardreader Attach
AwarioSmartBot
BBBike
BackStreet
Bandit
BetaBot
Bigfoot
Bolt
Brandwatch
CSHttp
CazoodleBot CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect Chlooe
CheTeam ClaudeBot
Copyscape Cogentbot
Cosmos CrunchBot
Curious
DSearch
DTS Agent DTS Agent
DatabaseDriverMysqli Discoverybot
DittoSpyder DnBCrawler-Analytics
Domains Project DomainCrawler
Drip Dotbot
ECCP/1.0 EMail Wolf
FacebookBot EirGrabber
ExtLinksBot
Freeuploader
Fuzz
Genieo
GetWeb GetWeb
Grafula Go-Ahead-Got-It
GrapeFX GoZilla
HMView HybridBot
HaosouSpider IndeedBot
InternetMeasurement InfoNaviRobot
InternetSeer
JOC Web Spider JOC Web Spider
Keyword Density
Kinza
LNSpiderguy LNSpiderguy
LeechFTP LWP::Simple
LinkWalker LinkScan
LinkpadBot Linkbot
LinqiaMetadataDownloaderBot LinqiaRSSBot
LinqiaScrapeBot
Lmspider
Ltx71 Ltx71
MFC_Tear_Sample MSIECrawler
MQQBrowser MegaIndex.ru
Mail.RU_Bot Morfeus Fucking Scanner
Majestic-SEO NetLyzer
MauiBot NetMechanic
MicroMessenger Netvibes
Mojeek Ninja
Musobot Pockey
Name Intelligence RankFlex
Nameprotect Ripper
Needle SEOkicks-Robot
Netcraft ScoutJet
Nuclei SearchmetricsBot
Nutch SemanticJuice
Offline Navigator SentiBot
OpenLinkProfiler SeoCherryBot
Openvas Seomoz
PageAnalyzer SiteSucker
PictureFinder Sitevigil
Probethenet SlySearch
Pu_iN SmartDownload
RankingBot2 Spinn3r
Rankivabot SputnikBot
RankurBot Sqlmap
ReGet Sucker
RealDownload SurveyBot
Rogerbot Telesphorep
SEOlyticsCrawler TightTwatBot
Scanbot Titan
Searchestate Tracemyfile
SemrushBot
Sitebeam
Sqlworm
Stripper
Turnitin
Twice
URLy.Warning
V-BOT
VCI VCI
VeriCiteCrawler
VoidEYE VoidEYE
Voil WEBDAV
WWW-Mechanize Web Enhancer
WWW::Mechanize Web Fetch
WebFetch Web Sauger
WebLeacher
WebReaper WebReaper
WebWhacker
WebZIP
WebmasterWorldForumBot
Webster
Whack Whack
WiseGuys Robot Whacker
Zade Who.is Bot
ZumBot Xaldon_WebSpider
Zermelo
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider
cah.io.community
crawler4j
imagesift.com imagesift.com
omgili meanpathbot
s1z.ru openai
scalaj-http seoscanners
seocompany.store webpros.com
sp_auditbot
voyagerx.com
webmeup-crawler

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
4ur.click aa625d84f1587749c1ab011d6f269f7d64.com
alloysteel.ru boobsimge.com
pornoplen.com buron.pw
sexgalleries.top homegardenlova.com
shopsellcardsdumps.com houseofrose.com
spitfiremusic.com i-midias.net.br
steebook.com infobanks.ru
toy-shop.top medi-fitt.hu
waterpurifier.club mydoctorok.ru
xxxrus.org office2web.com

194
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
101lesbian.xyz 0lovespells0.blogspot.com
6128786.com 1688.com
aa625d84f1587749c1ab011d6f269f7d64.com 1flag.co.za
abcdeg.xyz 300richmond.co.nz
adviceforum.info 777-club.ru
alienwheels.de actices.com
alldezservice.kz activepr.ru
anapa-inns.ru adexprts.com
anonymous-redirect.com apimossnetinfo-a.akamaihd.net
apple.com-cleaner.systems artefakct.com
awency.com azartniy-bonus.com
blackle.com balkanfarma.org
blpmovies.com bankmib.ru
boom-celebs.com beclean-nn.ru
braip.com.br bonzibuddi.com
breget74.ru cheapjerseysa.com
cashkitten-a.akamaihd.net citysecurity.nu
caveavins.fr clickcash.com
citetick.com craftinsta.ru
clash-clans.ru db.speedup.gdn
cognitiveseo.com deadlinkchecker.com
coindirect.io dirtpics.pw
compliance-alexa.top djihispano.com
crd.clan.su dktr.ru
decorazilla.com eduinfosite.com
domain-submit.info filmania-x.ru
domcran.net firesub.pl
easyshoppermac.com flowersbazar.com
eyes-on-you.ga fuckmill.com
fefo.gdn gosmeb.ru
firma-legion.ru home-task.com
forex-procto.ru
gdebestkupit.ru
holidaypics.org
homegardenlova.com
house.sieraddns.com
ideibiznesa2015.ru
imagine-ex.co imagine-ex.co
javxxx18.com incep.imagine-ex.co
jaxcube.info iradiology.ru
jestr.org irunfar.com
keywords-monitoring-your-success.com jennyfire.ru
lampokrat.ws kanimage.com
lovi-moment.com.ua kevblog.top
makenahartlin.com kino2018.club
meddesk.ru lanasshop.ru
mensandals.xyz laptoper.net
mir-betting.ru lightinghomes.net
mirobuvi.com.ua liverpool.gsofootball.com
mystats.xyz lunamedia.co
napalm51.nut.cc macdamaged.space
nash-krym.info metarip.ru
nextconseil.com michellblog.online
niki-mlt.ru monarhs.info
notebook-pro.ru moscow-clining.ru
npoet.ru moviemail-online.co.uk
obsessionphrases.com moz.com
offergroup.info narkologiya-orel.ru
panouri-solare-acoperis.com next-dentists.tk
picturesmania.com obiavo.net
porno-video-chati.ru obiavo.ru
pornoelita.info online.ktc45.ru
promodj.com optikremont.ru
prosmibank.ru park.above.com
qexyfu.bugs3.com pdamods.ru
quickbuck.com perm.xrus.org
pivka.xyz
pornhubforum.tk
pourvous.info
pr0fit-b0x.com
pssucai.info
pulseonclick.com
radiodigital.co
rcpmda.ikan1080.xyz rcpmda.ikan1080.xyz
reining.lovasszovetseg.hu seccioncontrabajo.com
rewardpoll.com
rollercoin.com
rubbed.us
schalke04fc.info
seeingmeerkat.com seeingmeerkat.com
simply.net semaltmedia.com
snts.shell-pmr.ru seoprofiler.com
sotechco.co site-analyzer.com
superstats.com smartbalanceworld.com
symphonyintegratedhealthcare.com snomer1.ru
tako3.com splendorsearch-a.akamaihd.net
taxi-v-eisk.ru staticfs.host
textads.men stpolice.com
unimodemhalfduplefw.pen.io stroymonolit.su
veopornogratis.xxx superinterstitial.com
vgoloveboli.net theguardlan.com
viandpet.com tizanidineforopiatewithdrawal.blogspot.com
vintontech.info tizanidinehydrochloride2mg.blogspot.com
voditeltrezviy.ru tracfone.com
wesharepics.info track-rankings.online
wfb.hatedriveapart.com tracklead.net
whosonmyserver.com trafficinstantly.co
witherrom55.eklablog.fr trudogolik.net
wzgyyq.com uroffer.link
xev.ru validdomain.xyz
xn--e1aggki3c.xn--80adxhks vekzdorov.ru
yunque.pluto.ro villacoloniale.com
zirondelli.it visa-pasport.ru
zooggames.com walpaperlist.com
zrelaya.pw westen-z.life
zverokruh-shop.cz whatzmyip.net
zvetki.ru worst-sites.online
wpsecurity.website
xcombear.ru
xn--90acenikpebbdd4f6d.xn--p1ai

2
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,7 +8,7 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>OCTYPE html> </html><!DOCTYPE html>
<html> <html>
<head> <head>
<title>Welcome to NginX</title> <title>Welcome to NginX</title>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.158 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:12 UTC 2024 ### Updated: Sat Sep 28 05:47:45 UTC 2024
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -865,6 +865,8 @@ User-agent: SentiBot
Disallow:/ Disallow:/
User-agent: SenutoBot User-agent: SenutoBot
Disallow:/ Disallow:/
User-agent: SeoCherryBot
Disallow:/
User-agent: SeoSiteCheckup User-agent: SeoSiteCheckup
Disallow:/ Disallow:/
User-agent: SeobilityBot User-agent: SeobilityBot

8
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.09.4778 ### Version: V4.2024.09.4779
### Updated: Fri Sep 27 22:01:12 UTC 2024 ### Updated: Sat Sep 28 05:47:45 UTC 2024
### Bad Bot Count: 666 ### Bad Bot Count: 667
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -865,6 +865,8 @@ User-agent: SentiBot
Disallow:/ Disallow:/
User-agent: SenutoBot User-agent: SenutoBot
Disallow:/ Disallow:/
User-agent: SeoCherryBot
Disallow:/
User-agent: SeoSiteCheckup User-agent: SeoSiteCheckup
Disallow:/ Disallow:/
User-agent: SeobilityBot User-agent: SeobilityBot