vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2024.05.4479 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2024-05-04 08:38:53 +00:00
parent 8ece85d061
commit f11b0cb507
37 changed files with 297 additions and 261 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.05.4478 #### Version: V4.2024.05.4479
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [660](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.05.4478 #### Version: V4.2024.05.4479
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [660](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -14,9 +14,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2024.05.4478 #### Version: V4.2024.05.4479
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [660](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
_generator_lists/bad-user-agents.list
View file

@ -1,5 +1,3 @@
InfraSec\ Scanner
Information\ Security\ Team\ InfraSec\ Scanner
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -211,6 +209,8 @@ IndeedBot
Indy\ Library Indy\ Library
InfoNaviRobot InfoNaviRobot
InfoTekies InfoTekies
Information\ Security\ Team\ InfraSec\ Scanner
InfraSec\ Scanner
Intelliseek Intelliseek
InterGET InterGET
InternetMeasurement InternetMeasurement

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2024.05.4478 #### Version: V4.2024.05.4479
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [660](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [662](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

8
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

8
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.05.4478 ### Version: V3.2024.05.4479
### Updated: Fri May 3 22:01:09 UTC 2024 ### Updated: Sat May 4 08:35:13 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -359,6 +359,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

8
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/_robots_input/robots-input.txt
View file

@ -209,6 +209,8 @@ IndeedBot
Indy Library Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies InfoTekies
Information Security Team InfraSec Scanner
InfraSec Scanner
Intelliseek Intelliseek
InterGET InterGET
InternetMeasurement InternetMeasurement

2
dev-tools/buildnumber
View file

@ -1 +1 @@
4478 4479

8
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.05.4478 ### Version: V3.2024.05.4479
### Updated: Fri May 3 22:01:09 UTC 2024 ### Updated: Sat May 4 08:35:13 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -359,6 +359,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

8
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.91.68.80 0; 172.183.146.91 0;

8
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 1; 10.1.0.57 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 0; 10.1.0.57 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 1; 10.1.0.57 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 0; 10.1.0.57 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 1; 10.1.0.57 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -270,6 +270,8 @@
"~*(?:\b)Indy\ Library(?:\b|)" 0; "~*(?:\b)Indy\ Library(?:\b|)" 0;
"~*(?:\b)InfoNaviRobot(?:\b|)" 0; "~*(?:\b)InfoNaviRobot(?:\b|)" 0;
"~*(?:\b)InfoTekies(?:\b|)" 0; "~*(?:\b)InfoTekies(?:\b|)" 0;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b|)" 0;
"~*(?:\b)InfraSec\ Scanner(?:\b|)" 0;
"~*(?:\b)Intelliseek(?:\b|)" 0; "~*(?:\b)Intelliseek(?:\b|)" 0;
"~*(?:\b)InterGET(?:\b|)" 0; "~*(?:\b)InterGET(?:\b|)" 0;
"~*(?:\b)InternetMeasurement(?:\b|)" 0; "~*(?:\b)InternetMeasurement(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 0; 10.1.0.57 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.91.68.80 0; 172.183.146.91 0;

8
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.91.68.80 0; 172.183.146.91 0;

8
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:08 UTC 2024 ### Updated: Sat May 4 08:35:12 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -362,6 +362,8 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Indy\ Library(?:\b)" 3; "~*(?:\b)Indy\ Library(?:\b)" 3;
"~*(?:\b)InfoNaviRobot(?:\b)" 3; "~*(?:\b)InfoNaviRobot(?:\b)" 3;
"~*(?:\b)InfoTekies(?:\b)" 3; "~*(?:\b)InfoTekies(?:\b)" 3;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)InfraSec\ Scanner(?:\b)" 3;
"~*(?:\b)Intelliseek(?:\b)" 3; "~*(?:\b)Intelliseek(?:\b)" 3;
"~*(?:\b)InterGET(?:\b)" 3; "~*(?:\b)InterGET(?:\b)" 3;
"~*(?:\b)InternetMeasurement(?:\b)" 3; "~*(?:\b)InternetMeasurement(?:\b)" 3;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 1; 10.1.0.57 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -270,6 +270,8 @@
"~*(?:\b)Indy\ Library(?:\b|)" 0; "~*(?:\b)Indy\ Library(?:\b|)" 0;
"~*(?:\b)InfoNaviRobot(?:\b|)" 0; "~*(?:\b)InfoNaviRobot(?:\b|)" 0;
"~*(?:\b)InfoTekies(?:\b|)" 0; "~*(?:\b)InfoTekies(?:\b|)" 0;
"~*(?:\b)Information\ Security\ Team\ InfraSec\ Scanner(?:\b|)" 0;
"~*(?:\b)InfraSec\ Scanner(?:\b|)" 0;
"~*(?:\b)Intelliseek(?:\b|)" 0; "~*(?:\b)Intelliseek(?:\b|)" 0;
"~*(?:\b)InterGET(?:\b|)" 0; "~*(?:\b)InterGET(?:\b|)" 0;
"~*(?:\b)InternetMeasurement(?:\b|)" 0; "~*(?:\b)InternetMeasurement(?:\b|)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AllSubmitter Anarchy
CATExplorador FyberSpider
Fimap HTMLparser
GrabNet Netvibes
Id-search Pockey
OnCrawl Screaming
Web Collage Surfbot
Web Fuck WebFetch
autoemailspider ips-agent
lwp-request zgrab

162
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
AdsTxtCrawlerTP 404enemy
Acunetix
AhrefsBot
Anarchy
Atomseobot
AwarioBot AwarioBot
BLEXBot Brandprotect
Black Hole Buddy
BlowFish BuiltBotTough
Cegbfeieh Bullseye
CheTeam CSHttp
CheeseBot CazoodleBot
Crawling at Home Project CopyRightCheck
Copyscape
CrunchBot CrunchBot
DataCha0s DSearch
Digincore Dirbuster
Discoverybot
Dispatch Dispatch
Ebingbong DomCopBot
ExtractorPro Dotbot
Extreme Picture Finder EasyDL
Extractor
Firefox/7.0
FlashGet FlashGet
Flunky FyberSpider
Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GermCrawler GermCrawler
IDBTE4M Getintent
IRLbot Gotit
Iblog HEADMasterSEO
IndeedBot Haansoft
HybridBot
Image Fetch
Iria Iria
JOC Web Spider Jbrofuzz
Kinza Jorgee
LibWeb LeechFTP
Libwhisker LexiBot
Lightspeedsystems LieBaoFast
Linkbot LinqiaRSSBot
Lipperhey
Lipperhey Spider
MSFrontPage
MSIECrawler
Majestic SEO Majestic SEO
Minefield
Msrabot Msrabot
NICErsPRO
NPbot NPbot
Needle
Net Vampire Net Vampire
NetSpider NetSpider
Nibbler Octopus
NimbleCrawler Offline Explorer
Nmap Page Analyzer
Offline Navigator PageAnalyzer
Openfind Picscout
OrangeBot QueryN Metasearch
PageScorer
Papa Foto
PictureFinder
Psbot
RSSingBot
RankingBot RankingBot
RealDownload RankingBot2
Reaper Re-re
RocketCrawler RebelMouse
SMTBot SEOstats
ScanAlert ScreenerBot
Screaming ScrepyBot
SeekportBot SeekportBot
Seomoz Siteimprove
Site Sucker
SiteSnagger
SlySearch
SmartDownload SmartDownload
SocialRankIOBot Sottopop
Sociscraper Spbot
Telesoft Steeler
Telesphoreo
The Intraformant The Intraformant
Turingos Thumbor
Turnitin Tracemyfile
TurnitinBot URLy Warning
URLy.Warning
V-BOT
VCI VCI
VidibleScraper VeriCiteCrawler
Virusdie WWW-Mechanize
WBSearchBot WebFetch
Web Collage WebGo IS
Web Enhancer WebPix
Web Pix WebSucker
WebStripper
Webshag Webshag
Website Quester Whatweb
Webster ZyBorg
Whack awario.com
Wonderbot crawl.sogou.com
Xenu crawler.feedback
Zauba facebookscraper
Zermelo heritrix
Zitebot
anthropic-ai
arquivo.pt
clark-crawler
crawler4j
imagesift.com imagesift.com
internetVista monitor isitwp.com
mediawords lwp-trivial
sexsearcher netEstate NE Crawler
sogouspider oBot
spyfu pcBrowser
webprosbot plumanalytics
sp_auditbot
webmeup-crawler
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
browsepulse-a.akamaihd.net ekspresihati.info
click2pawn.com getyourimage.club
fix-website-errors.com gsou.cf
freakycheats.com hatdc.org
gotorussia.com linkpulse.com
labelwater.se medicines-choice.com
mafcards.ru orion-v.com
tattomedia.com petrovka-online.com
topflownews.com pornohub.me
vedomstvo.net whengirlsgowild.com

198
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
1qingdao.com 007angels.com
40cg.com 1-free-share-buttons.com
91abcw.com 2rich4bitches.com
ahmedabadwebs.com 4replicawatch.net
allbizne.co.ua 4ureyesonly.com
alldezservice.kz aaronabel.com
amyfoxfitness.com adman.gr
antons-transporte.de akama.com
awaybird.ru alloysteel.ru
babieca.com anmysite.com
balkanfarma.ru anonymous-redirect.com
best-seo-offer.com anticrawler.org
bestprofits-there.com aussie-prizes.com
bestwebsiteawards.com balkanfarma.org
bigcities.org bigpenisguide.com
bio-optomarket.ru binomo.com
bloggedporn.com bryansk.zrus.org
boom-celebs.com bumskontakte.org
casinos4dummies.co.uk buydissertation.net
ccjp.eu cablecar.us
comeondog.info capsderfudd.tk
cpabegins.ru cartierlovebraceletreplica.xyz
cranly.net chlooe.com
dafatiri.com com-onlinesupport.site
dating-time-now.com cookingmeat.ru
deadlinkchecker.com cosmeticswomens-womensports.rhcloud.com
delayreferat.ru culad.com
delta-line.men cv.wallhade.co
draniki.org elektrozigarettekaufen1.devhub.com
e2click.com elitedollars.com
editmedios.com erotische-geschichten-xxl.com
fickkontaktehobbyhuren.com fandlr.com
fighrofacciufreesig.ga fishingwholesale.us
flash4fun.com fortwosmartcar.pw
fotzen-ficken.com gameplexcity.com
free-deals.faith getresponse.com
freelinkbuilding.website.tk globatur.ru
funnymama.com groupmoney.ru
game-top.su hermesbelts.xyz
getmyads24.com highspeed5.net
homedecoguide.info ikritikimou.gr
ictizanidinehcl4mg.blogspot.com informatiecentro.be
ido3.com inome.com.ua
ihc2015.info iskalko.ru
interesnie-faktu.ru istizanidinelikexanax.blogspot.com
janavibekken.no justucalling32211123456789.tk
japan-bearings.ru kazka.ru
journeydownthescale.info kidd.reunionwatch.com
kashubadesign.ru lincolntheatre.com
kinogolos.ru listiseltemournlan.gq
klejonka.info lyrster.com
kletkimehan.ru m292.info
lakomka.com.ua macresource.co.uk
magicdiet.gq macrotek.ru
magnetpress.sk maxxtor.eu
makenahartlin.com milblueprint.com
mediawhirl.net modnie-futbolki.net
medicovi.com moneyviking-a.akamaihd.net
medispainstitute.com.au musicas.baixar-musicas-gratis.com
missis.top mygameplus.ru
mobplayer.net mylida.org
moviemail-online.co.uk mysex21.com
musezone.ru nailsimg.com
mydirtystuff.com onlinadverts.com
narkologiya-penza.ru ourtherapy.ru
nikitsyringedrivelg.pen.io prostitutki-odessa.org
odywpjtw.bloger.index.hr rednise.com
oneclickfiles.com rff-cfal.info
playboyfiles.xblog.in s1z.ru
predmety.in.ua sankt-peterburg.nodup.ru
reyel1985.webnode.fr sbornik-zakonov.ru
sanyuprojects.com sexpornotales.com
sardinie.us sexuria.net
scanner-jess.top similardeals.net
searchimpression.com simple-image.com.ua
senger.atspace.co.uk slomm.ru
seokicks.de
seryeznie-znakomstva.ru
sexobzor.info
sexs-foto.top
shmetall.com.ua
smichovbike.cz smichovbike.cz
spidtest.org stuffpride.com
stackthatbucks.com sundrugstore.com
svnuppsalaorebro.se tds-advert002.info
tabletkinaodchudzanie.com.pl thedownloadfreeonlinegames.blogspot.com
teenforporn.com thewomenlife.com
thesmartsearch.net tizanidinecost.blogspot.com
top-l2.com topcar-krasnodar.ru
top250movies.ru track.deriv.com
tramadolandtizanidine.blogspot.com trafficmp.com
ukrtextbook.com trailer.cinemaflix.website
visa-china.ru ua.tc
wareseeker.com unece.org
wholesalejerseychinashop.com uytmaster.ru
winterclassichockeyjerseys.com verymes.xyz
workle.website via-energy-order.com
xclicks.net vizitki.net
youghbould.wordpress.com vote-up.ru
youtube-downloader.savetubevideo.com vuclip.com
webkeyit.com
websitevaluebot.com
westen-v.life
xaylapdiendanang.com
yoopsie.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.152 0; 10.1.0.57 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:10 UTC 2024 ### Updated: Sat May 4 08:35:14 UTC 2024
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -437,6 +437,10 @@ User-agent: InfoNaviRobot
Disallow:/ Disallow:/
User-agent: InfoTekies User-agent: InfoTekies
Disallow:/ Disallow:/
User-agent: Information Security Team InfraSec Scanner
Disallow:/
User-agent: InfraSec Scanner
Disallow:/
User-agent: Intelliseek User-agent: Intelliseek
Disallow:/ Disallow:/
User-agent: InterGET User-agent: InterGET

10
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.05.4478 ### Version: V4.2024.05.4479
### Updated: Fri May 3 22:01:10 UTC 2024 ### Updated: Sat May 4 08:35:14 UTC 2024
### Bad Bot Count: 660 ### Bad Bot Count: 662
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -437,6 +437,10 @@ User-agent: InfoNaviRobot
Disallow:/ Disallow:/
User-agent: InfoTekies User-agent: InfoTekies
Disallow:/ Disallow:/
User-agent: Information Security Team InfraSec Scanner
Disallow:/
User-agent: InfraSec Scanner
Disallow:/
User-agent: Intelliseek User-agent: Intelliseek
Disallow:/ Disallow:/
User-agent: InterGET User-agent: InterGET