vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 04:00:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2025.07.5373 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2025-07-31 05:33:35 +00:00
parent 947d4fce60
commit 452d8eb454
38 changed files with 293 additions and 265 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2025.07.5372 #### Version: V4.2025.07.5373
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [678](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [679](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2025.07.5372 #### Version: V4.2025.07.5373
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [678](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [679](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -14,9 +14,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2025.07.5372 #### Version: V4.2025.07.5373
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [678](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [679](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_generator_lists/bad-user-agents.list
View file

@ -1,4 +1,3 @@
dataprovider
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -614,6 +613,7 @@ crawler.feedback
crawler4j crawler4j
dataforseo.com dataforseo.com
dataforseobot dataforseobot
dataprovider
demandbase-bot demandbase-bot
domainsproject.org domainsproject.org
eCatch eCatch

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2025.07.5372 #### Version: V4.2025.07.5373
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [678](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [679](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

7
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

7
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2025.07.5372 ### Version: V3.2025.07.5373
### Updated: Wed Jul 30 22:04:52 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -763,6 +763,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

7
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

1
dev-tools/_robots_input/robots-input.txt
View file

@ -613,6 +613,7 @@ crawler.feedback
crawler4j crawler4j
dataforseo.com dataforseo.com
dataforseobot dataforseobot
dataprovider
demandbase-bot demandbase-bot
domainsproject.org domainsproject.org
eCatch eCatch

2
dev-tools/buildnumber
View file

@ -1 +1 @@
5372 5373

7
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2025.07.5372 ### Version: V3.2025.07.5373
### Updated: Wed Jul 30 22:04:52 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -763,6 +763,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

7
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
74.235.151.0 0; 52.225.73.162 0;

7
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -674,6 +674,7 @@
"~*(?:\b)crawler4j(?:\b|)" 0; "~*(?:\b)crawler4j(?:\b|)" 0;
"~*(?:\b)dataforseo.com(?:\b|)" 0; "~*(?:\b)dataforseo.com(?:\b|)" 0;
"~*(?:\b)dataforseobot(?:\b|)" 0; "~*(?:\b)dataforseobot(?:\b|)" 0;
"~*(?:\b)dataprovider(?:\b|)" 0;
"~*(?:\b)demandbase-bot(?:\b|)" 0; "~*(?:\b)demandbase-bot(?:\b|)" 0;
"~*(?:\b)domainsproject.org(?:\b|)" 0; "~*(?:\b)domainsproject.org(?:\b|)" 0;
"~*(?:\b)eCatch(?:\b|)" 0; "~*(?:\b)eCatch(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
74.235.151.0 0; 52.225.73.162 0;

7
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
74.235.151.0 0; 52.225.73.162 0;

7
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:51 UTC 2025 ### Updated: Thu Jul 31 05:29:54 UTC 2025
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -766,6 +766,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3; "~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)dataprovider(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -674,6 +674,7 @@
"~*(?:\b)crawler4j(?:\b|)" 0; "~*(?:\b)crawler4j(?:\b|)" 0;
"~*(?:\b)dataforseo.com(?:\b|)" 0; "~*(?:\b)dataforseo.com(?:\b|)" 0;
"~*(?:\b)dataforseobot(?:\b|)" 0; "~*(?:\b)dataforseobot(?:\b|)" 0;
"~*(?:\b)dataprovider(?:\b|)" 0;
"~*(?:\b)demandbase-bot(?:\b|)" 0; "~*(?:\b)demandbase-bot(?:\b|)" 0;
"~*(?:\b)domainsproject.org(?:\b|)" 0; "~*(?:\b)domainsproject.org(?:\b|)" 0;
"~*(?:\b)eCatch(?:\b|)" 0; "~*(?:\b)eCatch(?:\b|)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anarchy99 Drip
Atomseobot LexiBot
CODE87 Picscout
InternetSeer Rainbot
Joomla ScanAlert
Mister PiX Steeler
Nimbostratus V-BOT
Petalbot Web Enhancer
SiteCheckerBotCrawler Zade
WWWOFFLE meanpathbot

170
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
ASPSeek Acunetix
AhrefsBot Aliyun
AiHitBot AwarioSmartBot
AliyunSecBot BBBike
AllSubmitter
Anarchie
Ankit
Asterias
Attach
BDCbot BDCbot
BackDoorBot Battleztar Bazinga
BlowFish Black Hole
CCBot Blow
CODE87 ClaudeBot
ChatGPT-User Cogentbot
Claritybot Copier
Cosmos Copyscape
Craftbot DatabaseDriverMysqli
DSearch DomainAppender
Discoverybot Drip
DnBCrawler-Analytics EMail Wolf
Ebingbong Ebingbong
EroCrawler EirGrabber
ExtractorPro Extractor
FemtosearchBot
Firefox/7.0
FrontPage
FyberSpider
GPTBot GPTBot
Genieo GetRight
Getintent GrabNet
GridBot Haansoft
Heritrix HaosouSpider
IRLbot HybridBot
IlseBot IDBot
JennyBot Id-search
Jetty IndeedBot
Intelliseek
IstellaBot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density Lanshanbot
LeechFTP MIIxpc
LeechGet MQQBrowser
LieBaoFast MSIECrawler
LinkScan Mail.RU_Bot
LinkWalker
Lipperhey
Majestic SEO Majestic SEO
Majestic-SEO
MeanPath Bot MeanPath Bot
Mojeek MicroMessenger
NPbot Microsoft URL Control
Name Intelligence Mozlila
Nessus Nameprotect
Net Vampire NetMechanic
NextGenSearchBot Netvibes
Nibbler Niki-bot
OnCrawl Nutch
PECL::HTTP POE-Component-Client-HTTP
Page Analyzer Pandalytics
PageAnalyzer Petalbot
PageGrabber Pimonster
PeoplePal ProPowerBot
Piepmatz Re-re
Pixray Reaper
Pump ScanAlert
Rainbot SearchmetricsBot
RankFlex SemrushBot-BA
Rankivabot Site Sucker
RebelMouse Sitevigil
Ripper SmartDownload
SEOprofiler SocialRankIOBot
ScoutJet
SemrushBot-SI
SentiBot
SiteCheckerBotCrawler
Sogou web spider
Spbot Spbot
SputnikBot Spider_Bot/3.0
Sucuri Spinn3r
Sqworm
Stripper
Teleport Teleport
Telesphorep
TheNomad
TinyTestBot TinyTestBot
Titan Titan
Trendictionbot True_Robot
VeriCiteCrawler TurnitinBot
V-BOT
VB Project
Vagabondo
Voltron Voltron
WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WPScan WWW-Mechanize
Web Fuck WebCopier
WebBandit Widow
WebPix Wprecon
WebZIP
YaK YaK
Zeus Zeus
archive.org_bot ZmEu
arquivo-web-crawler ZoomBot
domainsproject.org eCatch
facebookscraper
heritrix heritrix
imagesift.com linkdexbot
ips-agent lwp-request
isitwp.com lwp-trivial
plumanalytics satoristudio.net
sitechecker.pro scan.lol
webmeup-crawler zauba.io
x22Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
avtolombard-krasnodar.com anti-virus-removal.info
blue-square.biz detsky-nabytek.info
date-withme.com frcls.fr
diesel-parts28.ru gay-file.com
maju.bluesalt.co liquimondo.com
minneapoliscopiers.com mojowhois.com
pic2fly.com mototsikl.org
romhacking.ru outclicks.net
snyatie-lomki-v-stacionare.ru xvideosporn.biz
statistici.ro your-bearings.com

200
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
883zy.com 1stat.ru
acmebtn.ml 2nt.ru
advokateg.ru 5elementov.ru
allknow.info 7fon.ru
apimountainbikei-a.akamaihd.net abcguru.xyz
bannerpower.com agadelha.com.br
berdasovivan.ru aliexpresscashback.club
besttorrentknifta.weebly.com android4fun.org
biteg.xyz asscenihotosidea.blogspot.com
biznesschat.net automatic-seo.com
blog.axant.it avitocars.ru
blpmovies.com bank.uz
brothers-smaller.ru belsetirehafi.tk
bus-offer.com bestcalovebracelet.cn
cascadelink.org bif-ru.info
cementaresearch.se biznesrealnost.ru
compliance-ivan.top brillianty.info
compliance-josh.top capsderfudd.tk
compliance-viktor.top carina-sy.de
cscwtalkto.site cenokos.ru
dawlenie.com chiptuninger.com
divisioncore.com clean-start.net
domcran.net coindirect.io
dorratex.tn counterbot.com
exci.ru customergrowthsystems.com
extremepornos.net detsky-nabytek.info
fanoboi.com dmmspy.com
financeloan.us dvervmoskvu.ru
firesub.pl elektrozigarette2.wordpressy.pl
fm-upgrade.ru epicbrogaming.com
forms-mtm.ru ezigarettenshop2.myblog.de
forsex.info fandlr.com
fsalas.com fitness-video.net
gaz-voshod.ru foto-sisek.porngalleries.top
gov.yanao.ru ghazel.ru
gq-catalog.gq gkvector.ru
hd-filmy.net glall.ru
hmywwogw.bloger.index.hr globetrotting-culture.ru
investingclub.ru homeart.space
iqoption.pro hotloans.ru
javatex.co.id i-service.kz
jerseywholesalebizchina.com iccornacircri.cf
kaac.ru imediadesk.com
karting196.ru infazavr.ru
kiwi237au.tk instakink.com
legionalpha.com intellektmedia.at
lion.snapmap.us ivoiretechnocom.ci
live-cam6.info kabbalah-red-bracelets.com
m3gadownload.pl kambasoft.com
macfix.life keywordteam.net
magda-gadalka.ru krassh.ru
masserect.com languagecode.com
mobifunapp.weebly.com law-check-two.xyz
moscow-region.ru laxdrills.com
myshopmatemac.com liveinternet.ro
notaria-desalas.com map028.com
osnova3.ru marktforschung-stuttgart.com
pewit.pw marwer.info
physfunc.ru medanestesia.ru
pierrehardysale.online mikozstop.com
plugingeorgia.com myblogregistercm.tk
portal-eu.ru pensplan.com
presleycollectibles.com pokerniydom.ru
proprostatit.com porndroids.com
quick-offer.com pornohub.me
referencemoi.com pravoholding.ru
resant.ru reeyanaturopathy.com
rock-cafe.info refererx.com
scat.porn remedyotc.com
sdi-pme.com roznica.com.ua
shakhtar-doneck.ru seoheap.com
sinestesia.host.sk soochi.co
sitevaluation.org sovetogorod.ru
space4updating.win stoki.ru
spywareit.com students-cheapskate.ml
teenforporn.com sugarlyflex.pw
teenfuck.tv surflinksmedical.com
thetattoohut.com theautoprofit.ml
thetoiletpaper.com thebestphotos.eu
tizanidinesideeffects.blogspot.com theguardlan.com
truck-land.life thenetinfo.com
trumpetedextremes.com torrents.life
ua.tc track.deriv.com
uprour.com traffmonster.info
vavilone.com videojam.tv
vbikse.com volume-pills.biz
vids18.site web.cvut.cz
vipms.ru weblo.com
vksex.ru wholinkstome.com
vzlom-na-zakaz.com wma-x.com
waycash.net wordkeyhelper.com
website-audit.com.ua xn----9sbebi2bvzr7h.xn--p1ai
wemedinc.com xnxx-n.com
whatistizanidinehclusedfor.blogspot.com youbrainboost.asia
witclub.info yourothersite.com
wleuaprpxuvr.ga yousense.info
xn--e1aggki3c.xn--80adxhks youtoner.it
xpresscare.ru zaimite.ru
xxlargepop.com zajm-pod-zalog-nedvizhimosti.ru
yourads.website zscaler.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,6 +8,16 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>OCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><!DOCTYPE html> </html><!DOCTYPE html>
<html> <html>
<head> <head>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.111 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:52 UTC 2025 ### Updated: Thu Jul 31 05:29:55 UTC 2025
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -1245,6 +1245,8 @@ User-agent: dataforseo.com
Disallow:/ Disallow:/
User-agent: dataforseobot User-agent: dataforseobot
Disallow:/ Disallow:/
User-agent: dataprovider
Disallow:/
User-agent: demandbase-bot User-agent: demandbase-bot
Disallow:/ Disallow:/
User-agent: domainsproject.org User-agent: domainsproject.org

8
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2025.07.5372 ### Version: V4.2025.07.5373
### Updated: Wed Jul 30 22:04:52 UTC 2025 ### Updated: Thu Jul 31 05:29:55 UTC 2025
### Bad Bot Count: 678 ### Bad Bot Count: 679
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -1245,6 +1245,8 @@ User-agent: dataforseo.com
Disallow:/ Disallow:/
User-agent: dataforseobot User-agent: dataforseobot
Disallow:/ Disallow:/
User-agent: dataprovider
Disallow:/
User-agent: demandbase-bot User-agent: demandbase-bot
Disallow:/ Disallow:/
User-agent: domainsproject.org User-agent: domainsproject.org