vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2024.04.4424 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2024-04-10 12:23:49 +00:00
parent 65c7de2332
commit 5d112aa565
37 changed files with 296 additions and 260 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.04.4423 #### Version: V4.2024.04.4424
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [655](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [657](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2024.04.4423 #### Version: V4.2024.04.4424
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [655](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [657](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -14,9 +14,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2024.04.4423 #### Version: V4.2024.04.4424
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [655](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [657](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
_generator_lists/bad-user-agents.list
View file

@ -1,5 +1,3 @@
AwarioBot
awario.com
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -32,6 +30,7 @@ Aspiegel
Asterias Asterias
Atomseobot Atomseobot
Attach Attach
AwarioBot
AwarioRssBot AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BBBike
@ -581,6 +580,7 @@ archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider autoemailspider
awario.com
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe check1.exe

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2024.04.4423 #### Version: V4.2024.04.4424
#### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7104](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [655](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [657](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

8
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

8
conf.d/globalblacklist-testing-version.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.04.4423 ### Version: V3.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:09 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

8
conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/_robots_input/robots-input.txt
View file

@ -30,6 +30,7 @@ Aspiegel
Asterias Asterias
Atomseobot Atomseobot
Attach Attach
AwarioBot
AwarioRssBot AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BBBike
@ -579,6 +580,7 @@ archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider autoemailspider
awario.com
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe check1.exe

2
dev-tools/buildnumber
View file

@ -1 +1 @@
4423 4424

8
dev-tools/globalblacklist-testing.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2024.04.4423 ### Version: V3.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:09 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

8
dev-tools/globalblacklist.template
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.172.7.68 0; 20.55.46.193 0;

8
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 1; 10.1.0.42 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 0; 10.1.0.42 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 1; 10.1.0.42 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 0; 10.1.0.42 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 1; 10.1.0.42 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -91,6 +91,7 @@
"~*(?:\b)Asterias(?:\b|)" 0; "~*(?:\b)Asterias(?:\b|)" 0;
"~*(?:\b)Atomseobot(?:\b|)" 0; "~*(?:\b)Atomseobot(?:\b|)" 0;
"~*(?:\b)Attach(?:\b|)" 0; "~*(?:\b)Attach(?:\b|)" 0;
"~*(?:\b)AwarioBot(?:\b|)" 0;
"~*(?:\b)AwarioRssBot(?:\b|)" 0; "~*(?:\b)AwarioRssBot(?:\b|)" 0;
"~*(?:\b)AwarioSmartBot(?:\b|)" 0; "~*(?:\b)AwarioSmartBot(?:\b|)" 0;
"~*(?:\b)BBBike(?:\b|)" 0; "~*(?:\b)BBBike(?:\b|)" 0;
@ -640,6 +641,7 @@
"~*(?:\b)arquivo-web-crawler(?:\b|)" 0; "~*(?:\b)arquivo-web-crawler(?:\b|)" 0;
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
"~*(?:\b)autoemailspider(?:\b|)" 0; "~*(?:\b)autoemailspider(?:\b|)" 0;
"~*(?:\b)awario.com(?:\b|)" 0;
"~*(?:\b)backlink-check(?:\b|)" 0; "~*(?:\b)backlink-check(?:\b|)" 0;
"~*(?:\b)cah.io.community(?:\b|)" 0; "~*(?:\b)cah.io.community(?:\b|)" 0;
"~*(?:\b)check1.exe(?:\b|)" 0; "~*(?:\b)check1.exe(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 0; 10.1.0.42 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.172.7.68 0; 20.55.46.193 0;

8
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.172.7.68 0; 20.55.46.193 0;

8
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,10 +4,10 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:14 UTC 2024 ### Updated: Wed Apr 10 12:20:08 UTC 2024
### Bad Referrer Count: 7104 ### Bad Referrer Count: 7104
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -179,6 +179,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)Asterias(?:\b)" 3; "~*(?:\b)Asterias(?:\b)" 3;
"~*(?:\b)Atomseobot(?:\b)" 3; "~*(?:\b)Atomseobot(?:\b)" 3;
"~*(?:\b)Attach(?:\b)" 3; "~*(?:\b)Attach(?:\b)" 3;
"~*(?:\b)AwarioBot(?:\b)" 3;
"~*(?:\b)AwarioRssBot(?:\b)" 3; "~*(?:\b)AwarioRssBot(?:\b)" 3;
"~*(?:\b)AwarioSmartBot(?:\b)" 3; "~*(?:\b)AwarioSmartBot(?:\b)" 3;
"~*(?:\b)BBBike(?:\b)" 3; "~*(?:\b)BBBike(?:\b)" 3;
@ -728,6 +729,7 @@ map $http_user_agent $bad_bot {
"~*(?:\b)arquivo-web-crawler(?:\b)" 3; "~*(?:\b)arquivo-web-crawler(?:\b)" 3;
"~*(?:\b)arquivo.pt(?:\b)" 3; "~*(?:\b)arquivo.pt(?:\b)" 3;
"~*(?:\b)autoemailspider(?:\b)" 3; "~*(?:\b)autoemailspider(?:\b)" 3;
"~*(?:\b)awario.com(?:\b)" 3;
"~*(?:\b)backlink-check(?:\b)" 3; "~*(?:\b)backlink-check(?:\b)" 3;
"~*(?:\b)cah.io.community(?:\b)" 3; "~*(?:\b)cah.io.community(?:\b)" 3;
"~*(?:\b)check1.exe(?:\b)" 3; "~*(?:\b)check1.exe(?:\b)" 3;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 1; 10.1.0.42 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -91,6 +91,7 @@
"~*(?:\b)Asterias(?:\b|)" 0; "~*(?:\b)Asterias(?:\b|)" 0;
"~*(?:\b)Atomseobot(?:\b|)" 0; "~*(?:\b)Atomseobot(?:\b|)" 0;
"~*(?:\b)Attach(?:\b|)" 0; "~*(?:\b)Attach(?:\b|)" 0;
"~*(?:\b)AwarioBot(?:\b|)" 0;
"~*(?:\b)AwarioRssBot(?:\b|)" 0; "~*(?:\b)AwarioRssBot(?:\b|)" 0;
"~*(?:\b)AwarioSmartBot(?:\b|)" 0; "~*(?:\b)AwarioSmartBot(?:\b|)" 0;
"~*(?:\b)BBBike(?:\b|)" 0; "~*(?:\b)BBBike(?:\b|)" 0;
@ -640,6 +641,7 @@
"~*(?:\b)arquivo-web-crawler(?:\b|)" 0; "~*(?:\b)arquivo-web-crawler(?:\b|)" 0;
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
"~*(?:\b)autoemailspider(?:\b|)" 0; "~*(?:\b)autoemailspider(?:\b|)" 0;
"~*(?:\b)awario.com(?:\b|)" 0;
"~*(?:\b)backlink-check(?:\b|)" 0; "~*(?:\b)backlink-check(?:\b|)" 0;
"~*(?:\b)cah.io.community(?:\b|)" 0; "~*(?:\b)cah.io.community(?:\b|)" 0;
"~*(?:\b)check1.exe(?:\b|)" 0; "~*(?:\b)check1.exe(?:\b|)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
IDBTE4M BDCbot
NimbleCrawler BLEXBot
ProPowerBot NearSite
Teleport Nmap
Thumbor ProWebWalker
Twice Quick-Crawler
VeriCiteCrawler Sucker
iubenda-radar Suzuran
mediawords TwengaBot
oBot x09Mozilla

164
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
AdsTxtCrawlerTP Acunetix
AfD-Verbotsverfahren AhrefsBot
AwarioRssBot AwarioBot
BackDoorBot BackWeb
Battleztar Bazinga Backlink-Ceck
Blow BlowFish
Bullseye BuiltBotTough
ChatGPT-User Bytespider
Chlooe CheeseBot
Cloud mapping Citoid
Collector Copier
CopyRightCheck
Cosmos Cosmos
CrazyWebCrawler Craftbot
Crescent Crescent
CrunchBot DSearch
DBLBot DatabaseDriverMysqli
Demon
Disco Disco
DomainStatsBot DomainCrawler
EirGrabber DomainSigmaCrawler
EroCrawler Domains Project
Exabot EasyDL
Fimap
Firefox/7.0
Flunky
Fyrebot
GalaxyBot
GetRight
Gigabot
GrabNet
Grabber Grabber
GridBot HTTP::Lite
HTTrack
Harvest Harvest
Image Fetch HonoluluBot
InterGET Id-search
InternetSeer IndeedBot
JOC Web Spider IstellaBot
LeechGet JamesBOT
Likse JennyBot
Jorgee
JustView
Lightspeedsystems
LinksManager LinksManager
Mag-Net Lmspider
Majestic-SEO
Mata Hari
MeanPath Bot
MegaIndex.ru
Minefield Minefield
Nameprotect Mojolicious
Ninja Musobot
Nuclei NearSite
Octopus NetMechanic
OutfoxBot NextGenSearchBot
NimbleCrawler
Nimbostratus
Nutch
OrangeBot
OutclicksBot
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber PageGrabber
Pavuk PageThing.com
Petalbot
Pi-Monster
Proximic Proximic
Psbot Psbot
Quick-Crawler RankFlex
Re-re Rankivabot
RealDownload SemanticJuice
RocketCrawler SenutoBot
SEOstats Seomoz
Siphon SiteSucker
Sogou web spider Sitevigil
Sottopop SmartDownload
Spinn3r Sosospider
Sqworm Szukacz
Sucker T0PHackTeam
Suzuran Thumbor
Swiftbot Titan
Teleport Toata
Turnitin Trendictionbot
Twice Twice
Typhoeus VidibleScraper
Virusdie WPScan
Web Collage Wallpapers/3.0
Web Fuck WallpapersHD
WebSucker WebCollage
WebWhacker WebEnhancer
WebmasterWorldForumBot WebFuck
Webster WebZIP
Woobot Website Quester
Xaldon WebSpider Whack
Xenu Wprecon
Zeus YoudaoBot
ZoominfoBot Zitebot
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider
awario.com
check1.exe
clark-crawler
cohere-ai cohere-ai
crawler.feedback domainsproject.org
heritrix
ips-agent ips-agent
iubenda-radar iubenda-radar
muhstik-scan lwp-trivial
openai.com openai.com
page scorer polaris version
seocompany.store
spyfu spyfu
sysscan
trendiction.de
ubermetrics-technologies.com
webmeup-crawler webmeup-crawler
x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bestcoin.cc 76brighton.co.uk
blockworld.ru cinemaflix.website
domaincrawler.com druzhininevgeniy63.blogspot.com
kazan.zrus.org greendream.com.ua
lottospring.com lion.bolegapakistan.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com luxup.ru
mydirtyhobby.com millionare.com
tsan.net morpicert.pw
xn--e1afanlbnfckd7c3d.xn--p1ai producm.ru
zojirushi-products.ru referencemoi.com

194
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,100 +1,100 @@
1adult.com 3xforum.ro
24-ak.ru 70casino.online
888.com 8jn.dating-app.ru
add-add.men adktrailmap.com
amazon-adsystem.com adpremium.org
animalia-life.club adverster.com
apelsinnik.website allpdfmags.net
arendovalka.xyz amazingpic.net
auspolice.com appearance-cool.com
aversis.net autobrennero.it
avirasecureserver.com automatic-seo.com
avtointeres.ru back.dog
azarttoto.com brainzpod.win
bestdooz.com buyfriend.ru
bonvillan.com buzzonclick.com
bukleteg.xyz bytimedance.ru
cdn.walkme.com chelyabinsk.xrus.org
cenokos.ru cinemaflix.website
clicksotrk.com clickso.com
colobit.biz clipartnew.com
compliance-george.top compliance-margo.top
compliance-josh.top crystalslot.com
compucelunlock.net customerguru.in
de.zapmeta.com daretodonate.co
degerlund.net dbmkfhqk.bloger.index.hr
digital-video-processing.com defenderxtactical.com
dmmspy.com diminishedvalueoforegon.com
domainroam.win domaincrawler.com
dosug-lux.ru downloadmefiranaratb1972.xpg.com.br
dvrlists.com elvenmachine.com
dwomlink.info eshop4u.jp
educatemetv.com ewebarticle.info
essayservicewriting.org failingmarriege.blogspot.com
falcon-images.blogspot.com fatmaelgarny.com
favornews.com feel-planet.com
ge0ip.net fetroshok.ru
girlgamerdaily.com figuringmoneyout.com
hamilton.ca filkhbr.com
herokuapp.com floridamhca.org
houtings.xyz forum-engineering.ru
hoverboard360.de freejabs.com
igrovyeavtomaty777.ru freemags.cc
imgarit.pw funponsel.com
iqoption.com futbolkisales.ru
itbc.kiev.ua fyl.com.ru
jose.mulinohouse.co glicol.kz
kakablog.net hdxnxxtube.mobi
kambasoft.com highstairs-a.akamaihd.net
kevblog.top hptwaakw.blog.fc2.com
keywordhut.com ibb.com.ua
lemon-ade.site iboss.com
magnetpress.sk interior-stickers.ru
matpre.top investzalog.ru
maximpartnerspr.com jolic2.com
katushka.net
kinosed.net
knowsitall.info
koptims.tiu.ru
maranbrinfo.com.br
middlerush-a.akamaihd.net
mobilemedia.md mobilemedia.md
mobilierland.com mompussy.net
mostantikor.ru partnerads.men
myghillie.info peugeot-club.org
mymobilemoneypages.com photochki.com
neobux-bg.info privatov-zapisi.ru
nevansk.ru pufip.com
nhl09.ru pussysaga.com
omsk.xrus.org revistaindustria.com
palocco.it santechnik.jimdo.com
pl-top.pl searchwebknow-a.akamaihd.net
pl-vouchers.com seeingmeerkat.com
pochemychka.net sendearnings.com
potolokelekor.ru seo18.su
predmety.in.ua seoprofiler.com
profitfx.online sexpartygirls.net
pskcijdc.bloger.index.hr sibtest.ru
ragecash.com sideeffectsoftizanidine.blogspot.com
rankia.com siteimprove.com
rusenvironmental.net sloopyjoes.com
sardinie.us socialbuttons.xyz
saveindex.xyz telemetryverification.net
scrapy.org thin.me.pn
secretscook.ru titan-cloud.life
seksotur.ru tizanidinewithdrawal.blogspot.com
shitting.pro toloka.hurtom.com
shopvilleroyboch.com.ua top-deal.com.pl
slavia.info torrentdownloadhub.com
snyatie-lomki-v-stacionare.ru trafficmania.com
softxaker.ru
space4update.pw
ssconstruction.co
svnuppsalaorebro.se
takethatad.com
trahvid.com trahvid.com
urldelivery.com uggsale.online
veerotech.com utrolive.ru
vladimir.zrus.org viel.su
voloomoney.com welcomeauto.ru
whatzmyip.net winterclassichockeyjerseys.com
whiteproduct.com www.ehscloud.cn
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
yourmovies.pl xn--80ab4aa2g.xn--p1ai
zeroredirect6.com xnxx-n.com
zhuravlev.info xxx-treker.ru
zreloeporno.tv

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.120 0; 10.1.0.42 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:15 UTC 2024 ### Updated: Wed Apr 10 12:20:09 UTC 2024
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -79,6 +79,8 @@ User-agent: Atomseobot
Disallow:/ Disallow:/
User-agent: Attach User-agent: Attach
Disallow:/ Disallow:/
User-agent: AwarioBot
Disallow:/
User-agent: AwarioRssBot User-agent: AwarioRssBot
Disallow:/ Disallow:/
User-agent: AwarioSmartBot User-agent: AwarioSmartBot
@ -1177,6 +1179,8 @@ User-agent: arquivo.pt
Disallow:/ Disallow:/
User-agent: autoemailspider User-agent: autoemailspider
Disallow:/ Disallow:/
User-agent: awario.com
Disallow:/
User-agent: backlink-check User-agent: backlink-check
Disallow:/ Disallow:/
User-agent: cah.io.community User-agent: cah.io.community

10
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2024.04.4423 ### Version: V4.2024.04.4424
### Updated: Wed Apr 10 10:03:15 UTC 2024 ### Updated: Wed Apr 10 12:20:09 UTC 2024
### Bad Bot Count: 655 ### Bad Bot Count: 657
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -79,6 +79,8 @@ User-agent: Atomseobot
Disallow:/ Disallow:/
User-agent: Attach User-agent: Attach
Disallow:/ Disallow:/
User-agent: AwarioBot
Disallow:/
User-agent: AwarioRssBot User-agent: AwarioRssBot
Disallow:/ Disallow:/
User-agent: AwarioSmartBot User-agent: AwarioSmartBot
@ -1177,6 +1179,8 @@ User-agent: arquivo.pt
Disallow:/ Disallow:/
User-agent: autoemailspider User-agent: autoemailspider
Disallow:/ Disallow:/
User-agent: awario.com
Disallow:/
User-agent: backlink-check User-agent: backlink-check
Disallow:/ Disallow:/
User-agent: cah.io.community User-agent: cah.io.community