vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.05.3716 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-05-01 04:27:11 +00:00
parent 75bc4349fc
commit 285094c7e4
39 changed files with 389 additions and 372 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.04.3715 #### Version: V4.2023.05.3716
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [639](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.04.3715 #### Version: V4.2023.05.3716
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [639](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -14,9 +14,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.04.3715 #### Version: V4.2023.05.3716
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [639](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
_generator_lists/bad-user-agents.list
View file

@ -1,5 +1,3 @@
Proximic
dataforseobot
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -360,6 +358,7 @@ Pockey
ProPowerBot ProPowerBot
ProWebWalker ProWebWalker
Probethenet Probethenet
Proximic
Psbot Psbot
Pu_iN Pu_iN
Pump Pump
@ -580,6 +579,7 @@ crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
dataforseo.com dataforseo.com
dataforseobot
demandbase-bot demandbase-bot
domainsproject.org domainsproject.org
eCatch eCatch

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.04.3715 #### Version: V4.2023.05.3716
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [639](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

9
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

9
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2023.04.3715 ### Version: V3.2023.05.3716
### Updated: Sun Apr 30 22:01:16 UTC 2023 ### Updated: Mon May 1 04:23:15 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

9
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/_robots_input/robots-input.txt
View file

@ -358,6 +358,7 @@ Pockey
ProPowerBot ProPowerBot
ProWebWalker ProWebWalker
Probethenet Probethenet
Proximic
Psbot Psbot
Pu_iN Pu_iN
Pump Pump
@ -578,6 +579,7 @@ crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
dataforseo.com dataforseo.com
dataforseobot
demandbase-bot demandbase-bot
domainsproject.org domainsproject.org
eCatch eCatch

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3715 3716

9
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2023.04.3715 ### Version: V3.2023.05.3716
### Updated: Sun Apr 30 22:01:16 UTC 2023 ### Updated: Mon May 1 04:23:15 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

9
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.171.49.55 0; 157.55.189.33 0;

9
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.119 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.119 0;
127.0.0.1 0; 127.0.0.1 0;

9
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.119 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.119 0;
127.0.0.1 0; 127.0.0.1 0;

9
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.119 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -419,6 +419,7 @@
"~*(?:\b)ProPowerBot(?:\b|)" 0; "~*(?:\b)ProPowerBot(?:\b|)" 0;
"~*(?:\b)ProWebWalker(?:\b|)" 0; "~*(?:\b)ProWebWalker(?:\b|)" 0;
"~*(?:\b)Probethenet(?:\b|)" 0; "~*(?:\b)Probethenet(?:\b|)" 0;
"~*(?:\b)Proximic(?:\b|)" 0;
"~*(?:\b)Psbot(?:\b|)" 0; "~*(?:\b)Psbot(?:\b|)" 0;
"~*(?:\b)Pu_iN(?:\b|)" 0; "~*(?:\b)Pu_iN(?:\b|)" 0;
"~*(?:\b)Pump(?:\b|)" 0; "~*(?:\b)Pump(?:\b|)" 0;
@ -639,6 +640,7 @@
"~*(?:\b)crawler.feedback(?:\b|)" 0; "~*(?:\b)crawler.feedback(?:\b|)" 0;
"~*(?:\b)crawler4j(?:\b|)" 0; "~*(?:\b)crawler4j(?:\b|)" 0;
"~*(?:\b)dataforseo.com(?:\b|)" 0; "~*(?:\b)dataforseo.com(?:\b|)" 0;
"~*(?:\b)dataforseobot(?:\b|)" 0;
"~*(?:\b)demandbase-bot(?:\b|)" 0; "~*(?:\b)demandbase-bot(?:\b|)" 0;
"~*(?:\b)domainsproject.org(?:\b|)" 0; "~*(?:\b)domainsproject.org(?:\b|)" 0;
"~*(?:\b)eCatch(?:\b|)" 0; "~*(?:\b)eCatch(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.119 0;
127.0.0.1 0; 127.0.0.1 0;

9
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.171.49.55 0; 157.55.189.33 0;

9
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.171.49.55 0; 157.55.189.33 0;

9
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:15 UTC 2023 ### Updated: Mon May 1 04:23:14 UTC 2023
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -507,6 +507,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)ProPowerBot(?:\b)" 3; "~*(?:\b)ProPowerBot(?:\b)" 3;
"~*(?:\b)ProWebWalker(?:\b)" 3; "~*(?:\b)ProWebWalker(?:\b)" 3;
"~*(?:\b)Probethenet(?:\b)" 3; "~*(?:\b)Probethenet(?:\b)" 3;
"~*(?:\b)Proximic(?:\b)" 3;
"~*(?:\b)Psbot(?:\b)" 3; "~*(?:\b)Psbot(?:\b)" 3;
"~*(?:\b)Pu_iN(?:\b)" 3; "~*(?:\b)Pu_iN(?:\b)" 3;
"~*(?:\b)Pump(?:\b)" 3; "~*(?:\b)Pump(?:\b)" 3;
@ -727,6 +728,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)crawler.feedback(?:\b)" 3; "~*(?:\b)crawler.feedback(?:\b)" 3;
"~*(?:\b)crawler4j(?:\b)" 3; "~*(?:\b)crawler4j(?:\b)" 3;
"~*(?:\b)dataforseo.com(?:\b)" 3; "~*(?:\b)dataforseo.com(?:\b)" 3;
"~*(?:\b)dataforseobot(?:\b)" 3;
"~*(?:\b)demandbase-bot(?:\b)" 3; "~*(?:\b)demandbase-bot(?:\b)" 3;
"~*(?:\b)domainsproject.org(?:\b)" 3; "~*(?:\b)domainsproject.org(?:\b)" 3;
"~*(?:\b)eCatch(?:\b)" 3; "~*(?:\b)eCatch(?:\b)" 3;
@ -869,7 +871,6 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Baidu(?:\b)" 2; "~*(?:\b)Baidu(?:\b)" 2;
"~*(?:\b)FlipboardProxy(?:\b)" 2; "~*(?:\b)FlipboardProxy(?:\b)" 2;
"~*(?:\b)MSIE\ 7.0(?:\b)" 2; "~*(?:\b)MSIE\ 7.0(?:\b)" 2;
"~*(?:\b)Proximic(?:\b)" 2;
"~*(?:\b)R6_CommentReader(?:\b)" 2; "~*(?:\b)R6_CommentReader(?:\b)" 2;
"~*(?:\b)R6_FeedFetcher(?:\b)" 2; "~*(?:\b)R6_FeedFetcher(?:\b)" 2;
"~*(?:\b)RED/1(?:\b)" 2; "~*(?:\b)RED/1(?:\b)" 2;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.119 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -419,6 +419,7 @@
"~*(?:\b)ProPowerBot(?:\b|)" 0; "~*(?:\b)ProPowerBot(?:\b|)" 0;
"~*(?:\b)ProWebWalker(?:\b|)" 0; "~*(?:\b)ProWebWalker(?:\b|)" 0;
"~*(?:\b)Probethenet(?:\b|)" 0; "~*(?:\b)Probethenet(?:\b|)" 0;
"~*(?:\b)Proximic(?:\b|)" 0;
"~*(?:\b)Psbot(?:\b|)" 0; "~*(?:\b)Psbot(?:\b|)" 0;
"~*(?:\b)Pu_iN(?:\b|)" 0; "~*(?:\b)Pu_iN(?:\b|)" 0;
"~*(?:\b)Pump(?:\b|)" 0; "~*(?:\b)Pump(?:\b|)" 0;
@ -639,6 +640,7 @@
"~*(?:\b)crawler.feedback(?:\b|)" 0; "~*(?:\b)crawler.feedback(?:\b|)" 0;
"~*(?:\b)crawler4j(?:\b|)" 0; "~*(?:\b)crawler4j(?:\b|)" 0;
"~*(?:\b)dataforseo.com(?:\b|)" 0; "~*(?:\b)dataforseo.com(?:\b|)" 0;
"~*(?:\b)dataforseobot(?:\b|)" 0;
"~*(?:\b)demandbase-bot(?:\b|)" 0; "~*(?:\b)demandbase-bot(?:\b|)" 0;
"~*(?:\b)domainsproject.org(?:\b|)" 0; "~*(?:\b)domainsproject.org(?:\b|)" 0;
"~*(?:\b)eCatch(?:\b|)" 0; "~*(?:\b)eCatch(?:\b|)" 0;

1
dev-tools/test_units/limited-bots-for-test.list
View file

@ -4,7 +4,6 @@ BUbiNG
Baidu Baidu
FlipboardProxy FlipboardProxy
MSIE 7.0 MSIE 7.0
Proximic
R6_CommentReader R6_CommentReader
R6_FeedFetcher R6_FeedFetcher
RED/1 RED/1

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BLEXBot ALittle Client
GridBot Aspiegel
Lftp Bandit
Morfeus Fucking Scanner Devil
NICErsPRO FrontPage
PageAnalyzer Grabber
RankActiveLinkBot Kinza
SearchmetricsBot Octopus
SiteSnagger T8Abot
WebSauger Telesphorep

174
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
AhrefsBot 404enemy
ADmantX
Acunetix
Anarchie
Ankit Ankit
AwarioSmartBot Apexoo
Barkrowler BetaBot
BatchFTP Brandprotect
Black Hole Buddy
Boardreader CazoodleBot
BuiltBotTough ChinaClaw
CODE87 Chlooe
CheTeam Citoid
CherryPicker
Claritybot Claritybot
Craftbot Devil
DatabaseDriverMysqli Discoverybot
Dirbuster
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomainAppender
Dragonfly EirGrabber
Extractor Evil
EyeNetIE ExtractorPro
Freeuploader FHscan
GT::WWW GT::WWW
Genieo GetRight
Go!Zilla GridBot
GoZilla HTMLparser
Gotit HybridBot
Haansoft IlseBot
Hloader Image Fetch
Iskanie InfoNaviRobot
Joomla InfoTekies
LNSpiderguy JamesBOT
LeechFTP JetCar
LinqiaRSSBot Keybot Translation-Search-Machine
Mail.RU_Bot LeechGet
Metauri Litemage_walker
MicroMessenger Majestic-SEO
NPbot MarkMonitor
Name Intelligence Meanpathbot
NetLyzer Mediatoolkitbot
NetZIP Minefield
Netcraft Moblie Safari
NimbleCrawler Mr.4x3
Octopus Nuclei
Offline Explorer Offline Explorer
Offline Navigator
Openfind Openfind
OrangeSpider Openvas
OutclicksBot Panscient
Page Analyzer Pavuk
Pandalytics
Pixray
PleaseCrawl
Psbot
PyCurl PyCurl
RankActiveLinkBot RankActiveLinkBot
RankingBot
RankingBot2
RankurBot
Reaper Reaper
SEOkicks RedesScrapy
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SalesIntelligent
ScoutJet ScoutJet
Scrapy ScreenerBot
Searchestate ScrepyBot
Shodan SemanticJuice
SiteSnagger Semrush
SlySearch SemrushBot
Sosospider SentiBot
SpaceBison SiteLockSpider
Spammen Sociscraper
Spbot Sucker
SputnikBot T8Abot
Suzuran Toweyabot
TeleportPro Turingos
Telesphoreo V-BOT
The Intraformant WISENutbot
Titan WWWOFFLE
True_Robot WeSEE
URLy.Warning Web Enhancer
VelenPublicWebCrawler WebCopier
Virusdie WebReaper
Voil Whatweb
Voltron Wotbox
WBSearchBot Xaldon_WebSpider
WebWhacker YoudaoBot
WebmasterWorldForumBot Zauba
WebsiteExtractor ZoomBot
Who.is Bot ZoominfoBot
Widow archive.org_bot
Xaldon WebSpider backlink-check
demandbase-bot com.plumanalytics
eCatch heritrix
evc-batch instabid
pcBrowser internetVista monitor
scalaj-http isitwp.com
iubenda-radar
lwp-trivial
page scorer
plumanalytics
s1z.ru
sysscan sysscan
tAkeOut
trendiction.com
voyagerx.com voyagerx.com
webmeup-crawler
x09Mozilla x09Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
123any.com groupmoney.ru
all4invest.ru harmonyglen.us
avtointeres.ru letolove.ru
cheapmoncler.win mosrif.ru
checktext.org nootrino.com
ikearugs.xyz picturesify.com
kongoultry.net rome2rio.com
no-fuel.org seorankinglinks.xyz
savetubevideo.com tanieaukcje.com.pl
tagil.zrus.org zarabotat-na-sajte.ru

196
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
101lesbian.xyz 100dollars-seo.com
123locker.com
1millionusd.xyz 1millionusd.xyz
4kplayer.pl 300richmond.co.nz
72-news.com 3dracergames.com
aliexpresscashback.club 4replicawatch.net
alldezservice.kz alta-realestate.com
allkrim.com am15.net
alvtank.se asdfz.pro
ameblo.top ayeartoforget.com
angigreene.com azartmsl.com
applicationg29.com beztuberkuleza.ru
azarttoto.com blavia.00author.com
binomo.com boo-arts.com
bjorkbacken.nu brainsandeggs.blogspot.com
bosefux.esy.es budpost.com.ua
brianhenry.co cartiertrinity.xyz
business-suggested.tk cherrypointplace.ca
casinopinup-wins.com chorus.terakeet.com
cdn.walkme.com christianlouboutinshoes.xyz
christianlouboutinreplica.win clickintext.net
cinemaenergy-hd.ru commoncrawl.org
cntravelre.com compliance-nelson.top
cometorussia.net covadhosting.biz
compliance-irvin.top covetnica.com
conboy.us creativehutindia.com
date-withme.com dailyfinancefix.com
decenttools.com davebestdeals.com
dichvuvesinhhanoi.com deluxedumps.com
dosugrostov.site dividendo.ru
editors.choice6912650.hulfingtonpost.com djekxa.ru
elexies.info eaplay.ru
elitedollars.com elektrischezi.canalblog.com
filmbokep69.com fastfixing.tech
fotzen-ficken.com fix-website-errors.com
glass-msk.ru friendflnder.com
grtyi.com fuckmill.com
gsasearchengineranker.pw gaz-voshod.ru
houtings.xyz goldenggames.com
husky-shop.cz gwebtools.com.br
ikearugs.xyz histock.info
imicrovision.com inet-traffic.com
indo-export.ru infazavr.ru
ivanstroi.ru iplusbit.blogspot.co.za
izismile.com iron-age.info
junketjuice.blogspot.com jbl-charge.info
kirov.zrus.org johannesburgsingles.co.za
kollekcioner.ru kopihijau.info
kovrenok.ru law-check-seven.xyz
lol-smurfs.com lineavabit.it
lomza.info
losangeles-ads.com
makedo.ru makedo.ru
me-ke.com maladot.com
mindeyegames.com medicinacom.ru
mmog-play.ru ngps1.ru
musicas.baixar-musicas-gratis.com nhl17coinsforps3.gratisblog.biz
newstudio.tv openmediasoft.com
nextbackgroundcheck.gq pic2fly.com
northfacestore.online platesauto.com
nvssf.com prointer.net.ua
office-windows.ru proposal-engine.com
oksrv.com prostitutki-belgoroda.org
onlinetvseries.me reklama-i-rabota.ru
optikremont.ru rockingclicks.com
percin.biz.ly rubbed.us
pk-services.ru russintv.fr
pornoted.com sampleletters.net
proekt-mos.ru scanner-don.top
prostitutki-kiev.org scanner-jack.top
proxyelite.biz seorankinglinks.us
quizzitch.net service.adtech.us
redirectingat.com shop.xz618.com
rff-cfal.info sibdevice.ru
runstocks.com site3.free-share-buttons.com
rvi.biz sklad-24.ru
sa-rewards.co.za sovetogorod.ru
sabizonline.com spitfiremusic.com
sexytrend.ru spywareit.com
sharebutton.net stevemonsen.com
snjatie-geroinovoy-lomki.ru superoboi.com.ua
softomix.net support.nopeas.sk
startufa.ru teenfuck.tv
stroyhelp-dv.ru telegraf.by
thecounter.com thebestweightlosspills.ovh
tizanidine4mgstreetvalue.blogspot.com tootoo.to
tizanidineinteractions.blogspot.com topsiteminecraft.com
tizanidinestreetvalue.blogspot.com ultimateclassicrock.com
tracfone.com urengoy.pro
truemfilelj.gq videojam.tv
ugogo.info wholesalejerseychinaoutlet.com
uk-zheu20.ru worldinternetauthority.com
vapeface.club x5market.ru
vedomstvo.net xn--c1acygb.xn--p1ai
vgoloveboli.net xtubeporno.net
video-camer.com zahvat.ru
villacoloniale.com zaim-pod-zalog-krasnodar.ru
wedding0venues.tk zigarettenonlinekaufen2.drupalgardens.com
wetgames.ru zrelaya.pw
wygraj-skiny.win zs2vm.top
xn----itbeirbjbi7bc6bh2d.xn--p1ai
xnxxandxvideos.com
yourads.website

181
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,6 +8,93 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -407,90 +494,6 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -581,14 +584,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.119 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:17 UTC 2023 ### Updated: Mon May 1 04:23:16 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -735,6 +735,8 @@ User-agent: ProWebWalker
Disallow:/ Disallow:/
User-agent: Probethenet User-agent: Probethenet
Disallow:/ Disallow:/
User-agent: Proximic
Disallow:/
User-agent: Psbot User-agent: Psbot
Disallow:/ Disallow:/
User-agent: Pu_iN User-agent: Pu_iN
@ -1175,6 +1177,8 @@ User-agent: crawler4j
Disallow:/ Disallow:/
User-agent: dataforseo.com User-agent: dataforseo.com
Disallow:/ Disallow:/
User-agent: dataforseobot
Disallow:/
User-agent: demandbase-bot User-agent: demandbase-bot
Disallow:/ Disallow:/
User-agent: domainsproject.org User-agent: domainsproject.org

10
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.04.3715 ### Version: V4.2023.05.3716
### Updated: Sun Apr 30 22:01:17 UTC 2023 ### Updated: Mon May 1 04:23:16 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 639
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -735,6 +735,8 @@ User-agent: ProWebWalker
Disallow:/ Disallow:/
User-agent: Probethenet User-agent: Probethenet
Disallow:/ Disallow:/
User-agent: Proximic
Disallow:/
User-agent: Psbot User-agent: Psbot
Disallow:/ Disallow:/
User-agent: Pu_iN User-agent: Pu_iN
@ -1175,6 +1177,8 @@ User-agent: crawler4j
Disallow:/ Disallow:/
User-agent: dataforseo.com User-agent: dataforseo.com
Disallow:/ Disallow:/
User-agent: dataforseobot
Disallow:/
User-agent: demandbase-bot User-agent: demandbase-bot
Disallow:/ Disallow:/
User-agent: domainsproject.org User-agent: domainsproject.org