vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2024.07.4629 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2024-07-18 09:44:03 +00:00
parent c904a9bf9e
commit cd79a089d5
38 changed files with 298 additions and 271 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.07.4628 #### Version: V4.2024.07.4629
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [663](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.07.4628 #### Version: V4.2024.07.4629
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [663](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -14,9 +14,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2024.07.4628 #### Version: V4.2024.07.4629
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [663](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

3
_generator_lists/bad-user-agents.list
View file

@ -1,5 +1,3 @@
ClaudeBot
serpstatbot
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -86,6 +84,7 @@ ChinaClaw
Chlooe Chlooe
Citoid Citoid
Claritybot Claritybot
ClaudeBot
Cliqzbot Cliqzbot
Cloud\ mapping Cloud\ mapping
Cocolyzebot Cocolyzebot

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2024.07.4628 #### Version: V4.2024.07.4629
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [663](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

7
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

7
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.07.4628 ### Version: V3.2024.07.4629
### Updated: Wed Jul 17 22:01:04 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 663
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -234,6 +234,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

7
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

1
dev-tools/_robots_input/robots-input.txt
View file

@ -84,6 +84,7 @@ ChinaClaw
Chlooe Chlooe
Citoid Citoid
Claritybot Claritybot
ClaudeBot
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot

2
dev-tools/buildnumber
View file

@ -1 +1 @@
4628 4629

7
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.07.4628 ### Version: V3.2024.07.4629
### Updated: Wed Jul 17 22:01:04 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 663
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -234,6 +234,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

7
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.183.132.82 0; 172.183.131.69 0;

7
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 1; 10.1.1.167 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 0; 10.1.1.167 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 1; 10.1.1.167 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 0; 10.1.1.167 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 1; 10.1.1.167 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -145,6 +145,7 @@
"~*(?:\b)Chlooe(?:\b|)" 0; "~*(?:\b)Chlooe(?:\b|)" 0;
"~*(?:\b)Citoid(?:\b|)" 0; "~*(?:\b)Citoid(?:\b|)" 0;
"~*(?:\b)Claritybot(?:\b|)" 0; "~*(?:\b)Claritybot(?:\b|)" 0;
"~*(?:\b)ClaudeBot(?:\b|)" 0;
"~*(?:\b)Cliqzbot(?:\b|)" 0; "~*(?:\b)Cliqzbot(?:\b|)" 0;
"~*(?:\b)Cloud\ mapping(?:\b|)" 0; "~*(?:\b)Cloud\ mapping(?:\b|)" 0;
"~*(?:\b)Cocolyzebot(?:\b|)" 0; "~*(?:\b)Cocolyzebot(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 0; 10.1.1.167 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.183.132.82 0; 172.183.131.69 0;

7
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.183.132.82 0; 172.183.131.69 0;

7
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:03 UTC 2024 ### Updated: Thu Jul 18 09:40:23 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 662 ### Bad Bot Count: 664
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -237,6 +237,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Chlooe(?:\b)" 3; "~*(?:\b)Chlooe(?:\b)" 3;
"~*(?:\b)Citoid(?:\b)" 3; "~*(?:\b)Citoid(?:\b)" 3;
"~*(?:\b)Claritybot(?:\b)" 3; "~*(?:\b)Claritybot(?:\b)" 3;
"~*(?:\b)ClaudeBot(?:\b)" 3;
"~*(?:\b)Cliqzbot(?:\b)" 3; "~*(?:\b)Cliqzbot(?:\b)" 3;
"~*(?:\b)Cloud\ mapping(?:\b)" 3; "~*(?:\b)Cloud\ mapping(?:\b)" 3;
"~*(?:\b)Cocolyzebot(?:\b)" 3; "~*(?:\b)Cocolyzebot(?:\b)" 3;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 1; 10.1.1.167 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -145,6 +145,7 @@
"~*(?:\b)Chlooe(?:\b|)" 0; "~*(?:\b)Chlooe(?:\b|)" 0;
"~*(?:\b)Citoid(?:\b|)" 0; "~*(?:\b)Citoid(?:\b|)" 0;
"~*(?:\b)Claritybot(?:\b|)" 0; "~*(?:\b)Claritybot(?:\b|)" 0;
"~*(?:\b)ClaudeBot(?:\b|)" 0;
"~*(?:\b)Cliqzbot(?:\b|)" 0; "~*(?:\b)Cliqzbot(?:\b|)" 0;
"~*(?:\b)Cloud\ mapping(?:\b|)" 0; "~*(?:\b)Cloud\ mapping(?:\b|)" 0;
"~*(?:\b)Cocolyzebot(?:\b|)" 0; "~*(?:\b)Cocolyzebot(?:\b|)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AfD-Verbotsverfahren Acunetix
BBBike BatchFTP
Brandprotect CATExplorador
Brandwatch DigitalPebble
Lipperhey Spider G-i-g-a-b-o-t
Name Intelligence Openfind
SalesIntelligent Pavuk
awario.com Thumbor
sitechecker.pro WebAuto
webmeup-crawler zauba.io

186
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
Abonti AIBOT
AdsTxtCrawlerTP Aipbot
AfD-Verbotsverfahren Atomseobot
Anarchie AwarioBot
Anthill BDFetch
Aspiegel BackStreet
BetaBot BatchFTP
Blow Bigfoot
BotALot BlackWidow
Brandprotect
Brandwatch
Buck Buck
BuiltWith ChatGPT-User
BunnySlippers ChinaClaw
Citoid Cocolyzebot
Copier Copier
Cosmos Crescent
CyotekWebCopy DBLBot
Deusu Discobot
Drip DittoSpyder
ECCP/1.0 DnyzBot
EirGrabber DomCopBot
Exabot DomainAppender
ExtLinksBot Download Wonder
FemtosearchBot EMail Siphon
Fimap EMail Wolf
GPTBot Ecxi
Gigabot EroCrawler
Go!Zilla ExtractorPro
GoZilla FacebookBot
GrapeFX FrontPage
HaosouSpider GalaxyBot
HonoluluBot HTTP::Lite
JennyBot Harvest
Jorgee IDBot
Likse Iblog
LinkWalker IlseBot
LinkextractorPro Image Sucker
Lipperhey Indy Library
Litemage_walker Intelliseek
Lmspider LWP::Simple
Ltx71 LieBaoFast
MJ12bot MQQBrowser
Mail.RU_Bot Mag-Net
Majestic12 Majestic12
Mediatoolkitbot Morfeus Fucking Scanner
Moblie Safari NICErsPRO
Mojeek NetSpider
MolokaiBot Nikto
NPbot NimbleCrawler
Nameprotect Nimbostratus
Navroad OrangeSpider
Nessus OutclicksBot
Nettrack PageGrabber
NextGenSearchBot Pandalytics
PECL::HTTP Psbot
PageScorer PyCurl
Pavuk RankFlex
Probethenet RebelMouse
Proximic SEOstats
PxBroker Scanbot
ReGet Scrapy
RocketCrawler Searchestate
SEOprofiler SentiBot
ScoutJet SenutoBot
SeoSiteCheckup SeoSiteCheckup
Site Sucker Sitevigil
SiteLockSpider Snapbot
Sitebeam
SocialRankIOBot
SpaceBison SpaceBison
Sucker Spinn3r
Surfbot Suzuran
Telesoft Szukacz
Telesphoreo TheNomad
Turingos TightTwatBot
V-BOT Toata
WPScan Upflow
Web Collage Vacuum
Web Fetch VeriCiteCrawler
VoidEYE
Voil
WWW-Mechanize
Wallpapers/3.0
WallpapersHD
Web Fuck Web Fuck
Web Sauger Whack
Webshag Widow
Wotbox Zermelo
Xaldon WebSpider Zeus
YaK arquivo-web-crawler
YoudaoBot backlink-check
Zauba check1.exe
ZumBot clark-crawler
adscanner facebookscraper
anthropic-ai page scorer
cognitiveseo polaris version
evc-batch ripz
gopher
isitwp.com
linkfluence
probe-image-size
s1z.ru
seoscanners
siteripz siteripz
sysscan trendiction.de
webprosbot x09Mozilla
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alt-servis.ru 11pikachu.ru
c1.onedmp.com bracketsmackdown.com
centraletermice.us cdn.walkme.com
compliance-julianna.top communisave.co.za
digital-sale.su glass-msk.ru
images-graphics-pics.com gox.com.ua
owathemes.com pwwysydh.com
rebelmouse.com riders.ro
ukrobstep.com robertefuller.com
viddyoze.com royalvegascasino.com

194
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
0n-line.tv 70casino.online
1pamm.ru advancedsoftwaresupport.com
8558l.jobs.net agysya.ru
9med.net allproblog.com
aaronabel.com amazingninja.com
adult-shop.com.ua apartamentwroclaw.eu
afmuseum.com apiappenableinfo-a.akamaihd.net
agahinameh.com apibatbrowsecom-a.akamaihd.net
albuteroli.com app-ready.xyz
allnews.md aruplighting.com
aloofly.com aversis.net
anlimebel.kiev.ua bagcionderlab.com
anonymousfox.co basedecor.ru
apiallgeniusinfo-a.akamaihd.net belinka.com.ua
apple.com-webbrowsing-security.review belstaffstore.online
asociatia-tipografilor-transilvania.ro bestcalovebracelet.cn
avkzarabotok.com bestpornuha.com
avtocredit-legko.ru boazpower.com
blackle.com bongacams.com
board.f00d.de btnativenav.com
boostmyppc.com callmd5map.com
canvas.gloverid.site callstevens.com
chiblackhawks-jerseys.com ccjp.eu
cindymatches.com chatroulette.world
clayvasedesigns.tk ddospanel.com
coolwallpapers4k.info dinkolove.ya.ru
customergrowthsystems.com directrev.com
cymbaltaandtizanidine.blogspot.com edge.sharethis.com
d2jsp.org floridahuntingfishingadventures.com
dimkino.ru fm-upgrade.ru
dmmspy.com fun2cell.net
domain-submit.info gamevalue7.weebly.com
doyouknowtheword-flummox.ml gazoblok.net.ua
easyukraine.com gk170.ru
elektronischezigarettekaufen1.beeplog.com greendream.com.ua
eng-lyrics.com hptwaakw.blog.fc2.com
erotik-kostenlos.net humbmosquina.tk
essay-writing.work ifmo.ru
evogarage.com imadedinner.net
faracontrol.ir inbabes.sexushost.com
farm26.ru jestr.org
foxjuegos.com journalhome.com
galleryawesome.com ketrzyn.pl
hasshe.com kiinomaniak.pl
hd720kino.ru kinozapas.com
hoverboard360.de konkursov.net
i-midias.net.br krasivoe-hd.net
inboxdollars.com linkredirect.biz
innodgfdriverhm.aircus.com lost-alpha.ru
innoslicon.com marketland.ml
integritylandscapeservices.com mosrif.ru
jackpotchances.com myonlinepayday.co
kiwi237au.tk networkcheck.xyz
kuzinsp.ru notaria-desalas.com
linkarena.com nuup.info
macotool.com oneshotdate.com
magicplayer-s.acestream.net online-hit.info
manipulyator-peterburg.ru onlywoman.org
mcar.in.ua orhonit.com
mobstarr.com paleohub.info
muz-shoes.ru pijoto.net
myanyone.net pogosh.com
nailsimg.com polska-poezja.com
naturalshair.site pomoc-drogowa.cba.pl
nbsproject.ru porno-video-chati.ru
nerudlogistik.ru pornozhara.com
perl.dp.ua privhosting.com
perso.wanadoo.es prpops.com
photochki.com ranksonic.info
pics-group.com realtytimes.com
picsfair.com
pirateday.ru
pornokorol.com
pozdravleniya-c.ru
prchecker.info
ruscoininvest.company
samoiedo.it
seasaltwithfood.com seasaltwithfood.com
serviporno.com securityallianceservices.com
simplesite.com semprofile.com
sinel.info service.adtech.fr
sitevalued.com shodanhq.com
smokewithrabbits.com simply.net
sofit-dmd.ru sisiynas.ru
sohoindia.net skyway24.ru
spidtest.org snegozaderzhatel.ru
srdrvp.com social-buttons.com
stair.registrydomainservices.com soft1.ru
sukirgenk.dvrlists.com softonicads.com
svnuppsalaorebro.se tattooreligion.ru
tizanidine4mgtablets.blogspot.com tksn.ru
txxx.com tt-ipd.info
twittrading.com
upupa.net
viagraneggrx.com
virus-schutzmasken.de
vizag.kharkov.ua vizag.kharkov.ua
weekes.biz.tc vse-pesni.com
webalan.ru
website-analyzer.info
wholesalejerseychinaoutlet.com wholesalejerseychinaoutlet.com
wjgony.com winterclassichockeyjerseys.com
xn----9sbubg3ambdfl1j.xn--p1ai wnoz.de
youtubedownload.org writingservices17.blogspot.ru
znaniyapolza.ru yhit.press
zrizvtrnpale.tk zeikopay.com
zeroredirect9.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -878,4 +878,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.32 0; 10.1.1.167 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:05 UTC 2024 ### Updated: Thu Jul 18 09:40:24 UTC 2024
### Bad Bot Count: 662 ### Bad Bot Count: 663
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -187,6 +187,8 @@ User-agent: Citoid
Disallow:/ Disallow:/
User-agent: Claritybot User-agent: Claritybot
Disallow:/ Disallow:/
User-agent: ClaudeBot
Disallow:/
User-agent: Cliqzbot User-agent: Cliqzbot
Disallow:/ Disallow:/
User-agent: Cloud mapping User-agent: Cloud mapping

8
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.07.4628 ### Version: V4.2024.07.4629
### Updated: Wed Jul 17 22:01:05 UTC 2024 ### Updated: Thu Jul 18 09:40:24 UTC 2024
### Bad Bot Count: 662 ### Bad Bot Count: 663
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -187,6 +187,8 @@ User-agent: Citoid
Disallow:/ Disallow:/
User-agent: Claritybot User-agent: Claritybot
Disallow:/ Disallow:/
User-agent: ClaudeBot
Disallow:/
User-agent: Cliqzbot User-agent: Cliqzbot
Disallow:/ Disallow:/
User-agent: Cloud mapping User-agent: Cloud mapping