vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2510 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-08 08:44:55 +00:00
parent 31990594e1
commit dd9eb72cea
57 changed files with 1146 additions and 1104 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2509 #### Version: V4.2021.08.2510
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2509 #### Version: V4.2021.08.2510
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -11,8 +11,8 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2509 #### Version: V4.2021.08.2510
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
_generator_lists/bad-referrers.list
View file

@ -1,5 +1,3 @@
commoncrawl.org
cah.io.community
000free.us 000free.us
007angels.com 007angels.com
00author.com 00author.com
@ -1093,6 +1091,7 @@ c1.onedmp.com
cablecar.us cablecar.us
cacheimages.com cacheimages.com
cactussoft.biz cactussoft.biz
cah.io.community
cakemediahq.com.au cakemediahq.com.au
cakesplus.com.au cakesplus.com.au
calc-for-credit.ru calc-for-credit.ru
@ -1381,6 +1380,7 @@ com-supportcenter.website
comeondog.info comeondog.info
cometorussia.net cometorussia.net
comissionka.net comissionka.net
commoncrawl.org
communisave.co.za communisave.co.za
community.allhiphop.com community.allhiphop.com
companies-catalog.com companies-catalog.com

4
_google_analytics_ghost_spam/README.md
View file

@ -3,8 +3,8 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2509 #### Version: V4.2021.08.2510
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_google_analytics_ghost_spam/google-exclude-01.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-02.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-03.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-04.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-05.txt
View file

File diff suppressed because one or more lines are too long

2
_google_webmaster_disavow_links/google-disavow.txt
View file

@ -1091,6 +1091,7 @@ domain:c1.onedmp.com
domain:cablecar.us domain:cablecar.us
domain:cacheimages.com domain:cacheimages.com
domain:cactussoft.biz domain:cactussoft.biz
domain:cah.io.community
domain:cakemediahq.com.au domain:cakemediahq.com.au
domain:cakesplus.com.au domain:cakesplus.com.au
domain:calc-for-credit.ru domain:calc-for-credit.ru
@ -1379,6 +1380,7 @@ domain:com-supportcenter.website
domain:comeondog.info domain:comeondog.info
domain:cometorussia.net domain:cometorussia.net
domain:comissionka.net domain:comissionka.net
domain:commoncrawl.org
domain:communisave.co.za domain:communisave.co.za
domain:community.allhiphop.com domain:community.allhiphop.com
domain:companies-catalog.com domain:companies-catalog.com

8
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

8
conf.d/globalblacklist-testing-version.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2509 ### Version: V3.2021.08.2510
### Updated: Sat Aug 7 22:01:07 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

8
conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

8
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2509 ### Version: V3.2021.08.2510
### Updated: Sat Aug 7 22:01:07 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 1; 10.1.1.228 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 0; 10.1.1.228 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2509 ### Version: V3.2021.08.2510
### Updated: Sat Aug 7 22:01:07 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 1; 10.1.1.228 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 0; 10.1.1.228 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2509 ### Version: V3.2021.08.2510
### Updated: Sat Aug 7 22:01:07 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 1; 10.1.1.228 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -1138,6 +1138,7 @@
"~*(?:\b)cablecar\.us(?:\b)" 0; "~*(?:\b)cablecar\.us(?:\b)" 0;
"~*(?:\b)cacheimages\.com(?:\b)" 0; "~*(?:\b)cacheimages\.com(?:\b)" 0;
"~*(?:\b)cactussoft\.biz(?:\b)" 0; "~*(?:\b)cactussoft\.biz(?:\b)" 0;
"~*(?:\b)cah\.io\.community(?:\b)" 0;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 0; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 0;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 0; "~*(?:\b)cakesplus\.com\.au(?:\b)" 0;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0;
@ -1426,6 +1427,7 @@
"~*(?:\b)comeondog\.info(?:\b)" 0; "~*(?:\b)comeondog\.info(?:\b)" 0;
"~*(?:\b)cometorussia\.net(?:\b)" 0; "~*(?:\b)cometorussia\.net(?:\b)" 0;
"~*(?:\b)comissionka\.net(?:\b)" 0; "~*(?:\b)comissionka\.net(?:\b)" 0;
"~*(?:\b)commoncrawl\.org(?:\b)" 0;
"~*(?:\b)communisave\.co\.za(?:\b)" 0; "~*(?:\b)communisave\.co\.za(?:\b)" 0;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 0; "~*(?:\b)community\.allhiphop\.com(?:\b)" 0;
"~*(?:\b)companies\-catalog\.com(?:\b)" 0; "~*(?:\b)companies\-catalog\.com(?:\b)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 0; 10.1.1.228 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2509 ### Version: V3.2021.08.2510
### Updated: Sat Aug 7 22:01:07 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2509 2510

8
dev-tools/globalblacklist-testing.template
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.08.2509 ### Version: V3.2021.08.2510
### Updated: Sat Aug 7 22:01:07 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

8
dev-tools/globalblacklist.template
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/referrers-regex-format-whitelist-test.txt
View file

@ -1091,6 +1091,7 @@
"~*(?:\b)cablecar\.us(?:\b)" 0; "~*(?:\b)cablecar\.us(?:\b)" 0;
"~*(?:\b)cacheimages\.com(?:\b)" 0; "~*(?:\b)cacheimages\.com(?:\b)" 0;
"~*(?:\b)cactussoft\.biz(?:\b)" 0; "~*(?:\b)cactussoft\.biz(?:\b)" 0;
"~*(?:\b)cah\.io\.community(?:\b)" 0;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 0; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 0;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 0; "~*(?:\b)cakesplus\.com\.au(?:\b)" 0;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0;
@ -1379,6 +1380,7 @@
"~*(?:\b)comeondog\.info(?:\b)" 0; "~*(?:\b)comeondog\.info(?:\b)" 0;
"~*(?:\b)cometorussia\.net(?:\b)" 0; "~*(?:\b)cometorussia\.net(?:\b)" 0;
"~*(?:\b)comissionka\.net(?:\b)" 0; "~*(?:\b)comissionka\.net(?:\b)" 0;
"~*(?:\b)commoncrawl\.org(?:\b)" 0;
"~*(?:\b)communisave\.co\.za(?:\b)" 0; "~*(?:\b)communisave\.co\.za(?:\b)" 0;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 0; "~*(?:\b)community\.allhiphop\.com(?:\b)" 0;
"~*(?:\b)companies\-catalog\.com(?:\b)" 0; "~*(?:\b)companies\-catalog\.com(?:\b)" 0;

2
dev-tools/referrers-regex-format.txt
View file

@ -1091,6 +1091,7 @@
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -1379,6 +1380,7 @@
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.224.104.170 0; 40.121.235.86 0;

8
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 1; 10.1.1.228 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 0; 10.1.1.228 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 1; 10.1.1.228 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 0; 10.1.1.228 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 1; 10.1.1.228 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -1138,6 +1138,7 @@
"~*(?:\b)cablecar\.us(?:\b)" 0; "~*(?:\b)cablecar\.us(?:\b)" 0;
"~*(?:\b)cacheimages\.com(?:\b)" 0; "~*(?:\b)cacheimages\.com(?:\b)" 0;
"~*(?:\b)cactussoft\.biz(?:\b)" 0; "~*(?:\b)cactussoft\.biz(?:\b)" 0;
"~*(?:\b)cah\.io\.community(?:\b)" 0;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 0; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 0;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 0; "~*(?:\b)cakesplus\.com\.au(?:\b)" 0;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0;
@ -1426,6 +1427,7 @@
"~*(?:\b)comeondog\.info(?:\b)" 0; "~*(?:\b)comeondog\.info(?:\b)" 0;
"~*(?:\b)cometorussia\.net(?:\b)" 0; "~*(?:\b)cometorussia\.net(?:\b)" 0;
"~*(?:\b)comissionka\.net(?:\b)" 0; "~*(?:\b)comissionka\.net(?:\b)" 0;
"~*(?:\b)commoncrawl\.org(?:\b)" 0;
"~*(?:\b)communisave\.co\.za(?:\b)" 0; "~*(?:\b)communisave\.co\.za(?:\b)" 0;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 0; "~*(?:\b)community\.allhiphop\.com(?:\b)" 0;
"~*(?:\b)companies\-catalog\.com(?:\b)" 0; "~*(?:\b)companies\-catalog\.com(?:\b)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 0; 10.1.1.228 0;
127.0.0.1 0; 127.0.0.1 0;

8
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.224.104.170 0; 40.121.235.86 0;

8
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.224.104.170 0; 40.121.235.86 0;

8
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:06 UTC 2021 ### Updated: Sun Aug 8 08:39:07 UTC 2021
### Bad Referrer Count: 7077 ### Bad Referrer Count: 7079
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -2138,6 +2138,7 @@ map $http_referer $bad_referer {
"~*(?:\b)cablecar\.us(?:\b)" 1; "~*(?:\b)cablecar\.us(?:\b)" 1;
"~*(?:\b)cacheimages\.com(?:\b)" 1; "~*(?:\b)cacheimages\.com(?:\b)" 1;
"~*(?:\b)cactussoft\.biz(?:\b)" 1; "~*(?:\b)cactussoft\.biz(?:\b)" 1;
"~*(?:\b)cah\.io\.community(?:\b)" 1;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 1; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 1;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 1; "~*(?:\b)cakesplus\.com\.au(?:\b)" 1;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 1;
@ -2426,6 +2427,7 @@ map $http_referer $bad_referer {
"~*(?:\b)comeondog\.info(?:\b)" 1; "~*(?:\b)comeondog\.info(?:\b)" 1;
"~*(?:\b)cometorussia\.net(?:\b)" 1; "~*(?:\b)cometorussia\.net(?:\b)" 1;
"~*(?:\b)comissionka\.net(?:\b)" 1; "~*(?:\b)comissionka\.net(?:\b)" 1;
"~*(?:\b)commoncrawl\.org(?:\b)" 1;
"~*(?:\b)communisave\.co\.za(?:\b)" 1; "~*(?:\b)communisave\.co\.za(?:\b)" 1;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 1; "~*(?:\b)community\.allhiphop\.com(?:\b)" 1;
"~*(?:\b)companies\-catalog\.com(?:\b)" 1; "~*(?:\b)companies\-catalog\.com(?:\b)" 1;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 1; 10.1.1.228 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test_units/custom-bad-referrers.conf
View file

@ -1138,6 +1138,7 @@
"~*(?:\b)cablecar\.us(?:\b)" 0; "~*(?:\b)cablecar\.us(?:\b)" 0;
"~*(?:\b)cacheimages\.com(?:\b)" 0; "~*(?:\b)cacheimages\.com(?:\b)" 0;
"~*(?:\b)cactussoft\.biz(?:\b)" 0; "~*(?:\b)cactussoft\.biz(?:\b)" 0;
"~*(?:\b)cah\.io\.community(?:\b)" 0;
"~*(?:\b)cakemediahq\.com\.au(?:\b)" 0; "~*(?:\b)cakemediahq\.com\.au(?:\b)" 0;
"~*(?:\b)cakesplus\.com\.au(?:\b)" 0; "~*(?:\b)cakesplus\.com\.au(?:\b)" 0;
"~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0; "~*(?:\b)calc\-for\-credit\.ru(?:\b)" 0;
@ -1426,6 +1427,7 @@
"~*(?:\b)comeondog\.info(?:\b)" 0; "~*(?:\b)comeondog\.info(?:\b)" 0;
"~*(?:\b)cometorussia\.net(?:\b)" 0; "~*(?:\b)cometorussia\.net(?:\b)" 0;
"~*(?:\b)comissionka\.net(?:\b)" 0; "~*(?:\b)comissionka\.net(?:\b)" 0;
"~*(?:\b)commoncrawl\.org(?:\b)" 0;
"~*(?:\b)communisave\.co\.za(?:\b)" 0; "~*(?:\b)communisave\.co\.za(?:\b)" 0;
"~*(?:\b)community\.allhiphop\.com(?:\b)" 0; "~*(?:\b)community\.allhiphop\.com(?:\b)" 0;
"~*(?:\b)companies\-catalog\.com(?:\b)" 0; "~*(?:\b)companies\-catalog\.com(?:\b)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AwarioRssBot Joomla
Backlink-Ceck LibWeb
Digincore MarkWatch
Dragonfly Mojolicious
JOC Web Spider OrangeSpider
NetLyzer PECL::HTTP
Scrapy Telesphorep
Shodan Upflow
SiteSucker ZmEu
Turingos eCatch

290
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 01h4x.com
360Spider
80legs 80legs
AIBOT ADmantX
ASPSeek Abonti
Acunetix Aboundex
AhrefsBot
AiHitBot AiHitBot
AllSubmitter
Alligator Alligator
AlphaBot AwarioRssBot
Anarchy99
Ankit
Anthill
AwarioSmartBot
BBBike
BLEXBot
BackStreet BackStreet
BackWeb BackWeb
Badass Backlink-Ceck
Bandit BacklinkCrawler
BatchFTP
Battleztar Bazinga
BetaBot
Bigfoot
Blackboard
Boardreader Boardreader
BotALot BotALot
BunnySlippers Brandprotect
Buddy
BuiltBotTough
BuiltWith
BuzzSumo BuzzSumo
CATExplorador CATExplorador
CCBot CensysInspect
CSHttp CheTeam
Calculon Claritybot
CherryPicker Cogentbot
Chlooe Copyscape
Cliqzbot
Cloud mapping
Collector
Copier
Cosmos Cosmos
Crescent
CrunchBot CrunchBot
Custo DIIbot
DBLBot
DSearch
Demon Demon
Deusu
Devil Devil
Discoverybot DigitalPebble
DittoSpyder Dirbuster
Discobot
Dispatch
DomCopBot
DomainCrawler DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Domains Project
Dotbot Dotbot
EMail Siphon Download Wonder
EMail Wolf
Ebingbong Ebingbong
Ecxi
EroCrawler EroCrawler
Exabot Exabot
ExtLinksBot Express WebPictures
FDM ExtractorPro
Extreme Picture Finder
Ezooms
FHscan FHscan
FemtosearchBot
Fimap Fimap
FlashGet FlashGet
Freeuploader Foobot
Fyrebot GT::WWW
GermCrawler GetRight
GetWeb Getintent
Gigabot
GoZilla GoZilla
Gotit GrapeFX
GrabNet HTMLparser
GrapeshotCrawler Haansoft
GridBot HaosouSpider
HTTP::Lite Harvest
HTTrack
Havij Havij
Hloader HybridBot
IDBTE4M
IDBot IDBot
IlseBot
Image Fetch Image Fetch
IndeedBot InfoNaviRobot
Internet Ninja InterGET
InternetSeer InternetSeer
Iria Iria
Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT Jbrofuzz
JennyBot JetCar
JustView
Jyxobot
Kinza
Kozmosbot Kozmosbot
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
LeechGet LeechFTP
LexiBot LexiBot
LibWeb
LieBaoFast LieBaoFast
Lightspeedsystems
LinkWalker LinkWalker
Linkdexbot Linkdexbot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaScrapeBot
Lipperhey Lmspider
Litemage_walker MFC_Tear_Sample
Ltx71
MIDown tool
MJ12bot
MQQBrowser MQQBrowser
MSFrontPage
MSIECrawler
Mag-Net Mag-Net
MarkMonitor Majestic-SEO
Masscan
Mata Hari Mata Hari
MauiBot Mb2345Browser
MeanPath Bot MeanPath Bot
Metauri
Mojeek Mojeek
Mozlila Mozlila
Mr.4x3 Mr.4x3
Msrabot Msrabot
NICErsPRO Musobot
Name Intelligence Name Intelligence
NearSite Nameprotect
Needle Needle
Nessus
NetAnts NetAnts
NetMechanic NetSpider
NetZIP
Netcraft Netcraft
NextGenSearchBot Netvibes
Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Nmap
Nuclei
Octopus Octopus
Offline Explorer
Offline Navigator Offline Navigator
OpenLinkProfiler OnCrawl
Openfind OpenVAS
Openvas OrangeBot
OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot
PageGrabber PageGrabber
PageScorer PageScorer
PageThing.com Pavuk
Panscient Pi-Monster
Picscout Pixray
Picsearch Pockey
Piepmatz ProPowerBot
PleaseCrawl ProWebWalker
Pu_iN Psbot
Pump Pump
PxBroker QueryN Metasearch
Quick-Crawler RSSingBot
RankFlex
RankingBot RankingBot
Rankivabot RankingBot2
Re-re Re-re
RealDownload RealDownload
Reaper Reaper
RebelMouse
Recorder Recorder
RepoMonkey RepoMonkey
Rogerbot RocketCrawler
SEOkicks-Robot SBIder
SEOlyticsCrawler SEOkicks
SISTRIX
SalesIntelligent SalesIntelligent
Searchestate Scanbot
SemrushBot Scrapy
SearchmetricsBot
SeoSiteCheckup
SeobilityBot SeobilityBot
SiteSnagger Shodan
Sitebeam SiteLockSpider
SlySearch
SmartDownload SmartDownload
Snake
Snoopy Snoopy
Sogou web spider SocialRankIOBot
Sosospider Sosospider
Spammen Sottopop
SpankBot Spanner
Spbot
Spinn3r
SputnikBot SputnikBot
Sqlmap Sqlmap
Sqlworm
Steeler Steeler
Stripper Stripper
Sucker
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
Surfbot Surfbot
SurveyBot
Suzuran
Szukacz Szukacz
T0PHackTeam
T8Abot T8Abot
TeleportPro
Telesoft Telesoft
Telesphoreo Telesphoreo
TheNomad Telesphorep
The Intraformant
Thumbor Thumbor
Toata TightTwatBot
Tracemyfile
Turingos Turingos
Turnitin TwengaBot
Typhoeus
URLy Warning
UnisterBot UnisterBot
Upflow V-BOT
VB Project VB Project
Vagabondo VCI
VelenPublicWebCrawler VeriCiteCrawler
VidibleScraper Virusdie
Voil Voltron
WASALive-Bot
WISENutbot WISENutbot
WWW-Collector-E WWW-Collector-E
Wallpapers/3.0 WWWOFFLE
WallpapersHD Wallpapers
Web Auto Web Fetch
Web Pix
Web Sucker
WebCollage
WebFuck WebFuck
WebAuto WebPix
WebCopier WebReaper
WebSucker WebSucker
WebWhacker WebWhacker
WebmasterWorldForumBot
Webshag
Website Quester Website Quester
WebsiteExtractor
Whack
Whacker
Whatweb Whatweb
Xaldon WebSpider Who.is Bot
Xaldon_WebSpider Woobot
Zade Zermelo
Zitebot Zitebot
ZoominfoBot ZmEu
ZumBot
ZyBorg
archive.org_bot archive.org_bot
coccocbot-web arquivo-web-crawler
crawl.sogou.com arquivo.pt
demandbase-bot cognitiveseo
com.plumanalytics
crawler4j
facebookscraper facebookscraper
gopher gopher
heritrix
instabid instabid
internetVista monitor lwp-request
ips-agent
lwp-trivial lwp-trivial
mediawords magpie-crawler
oBot meanpathbot
pcBrowser muhstik-scan
plumanalytics
polaris version polaris version
probe-image-size probe-image-size
ripz scalaj-http
scan.lol scan.lol
seobility seobility
seostar seoscanners
siteripz siteripz
sp_auditbot sogouspider
spyfu spyfu
tAkeOut
trendiction.de trendiction.de
ubermetrics-technologies.com voyagerx.com
webmeup-crawler webmeup-crawler
x09Mozilla
x22Mozilla x22Mozilla
zgrab xpymep1.exe
zauba.io

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
360Spider
AIBOT AIBOT
ALittle Client ALittle Client
Aboundex ASPSeek
Abonti
Acunetix Acunetix
Alexibot AfD-Verbotsverfahren
AllSubmitter AiHitBot
Anarchy99 Anarchie
Ankit Aspiegel
Anthill Asterias
AwarioRssBot AwarioSmartBot
BBBike
BDCbot BDCbot
BackWeb BDFetch
Backlink-Ceck BackDoorBot
Badass BacklinkCrawler
Bandit
Barkrowler Barkrowler
Black Hole BatchFTP
BlackWidow Bitacle
Bolt Blackboard
BotALot BotALot
Buck Brandprotect
BuzzSumo Buddy
CATExplorador
CCBot CCBot
CSHttp
Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
ChinaClaw CensysInspect
Chlooe CheTeam
Cliqzbot CheeseBot
Claritybot
Cloud mapping
Cogentbot Cogentbot
Collector
Copier Copier
CopyRightCheck CopyRightCheck
Cosmos
Craftbot Craftbot
CrazyWebCrawler CrunchBot
Curious Custo
DBLBot DBLBot
DSearch
DTS Agent
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Devil
Digincore Digincore
DigitalPebble
Dirbuster
Discoverybot Discoverybot
DittoSpyder
DnyzBot
DomCopBot DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot
Drip
ECCP/1.0 ECCP/1.0
Ebingbong EMail Siphon
EroCrawler EirGrabber
Evil Exabot
Extreme Picture Finder Express WebPictures
EyeNetIE Extractor
Ezooms Ezooms
FHscan FDM
Firefox/7.0 FemtosearchBot
FlashGet
Foobot
Freeuploader Freeuploader
FrontPage FyberSpider
Fuzz G-i-g-a-b-o-t
Fyrebot GT::WWW
GermCrawler
GetRight
GetWeb
Getintent Getintent
Gigablast Gigablast
GoZilla
Gotit
GrabNet GrabNet
Grabber Haansoft
GridBot HaosouSpider
HEADMasterSEO
HTTrack
Harvest Harvest
Havij
Hloader Hloader
HybridBot IRLbot
IDBTE4M Image Fetch
IDBot IndeedBot
Image Sucker Indy Library
InfoNaviRobot
Intelliseek Intelliseek
InternetSeer InternetSeer
Iskanie Iria
JOC Web Spider IstellaBot
Jbrofuzz JetCar
Jetty Jetty
JustView JikeSpider
Keyword Density Joomla
LNSpiderguy Jorgee
LWP::Simple
Lanshanbot
Larbin
Leap Leap
LeechGet LeechFTP
LieBaoFast LexiBot
LinkScan Linkbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Lipperhey
Litemage_walker Litemage_walker
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MIIxpc MIIxpc
MJ12bot
MQQBrowser MQQBrowser
Mag-Net Mag-Net
Majestic-SEO Magnet
Majestic SEO
Majestic12 Majestic12
MarkMonitor
Mass Downloader Mass Downloader
Mata Hari
MeanPath Bot MeanPath Bot
Meanpathbot Metauri
Microsoft URL Control MicroMessenger
Mojeek Microsoft Data Access
Mister PiX
Moblie Safari
Mojolicious Mojolicious
Mr.4x3 Mr.4x3
Msrabot
Musobot Musobot
NICErsPRO NICErsPRO
NPbot Nameprotect
Navroad Nessus
Net Vampire
NetMechanic NetMechanic
NetSpider Netcraft
Netvibes Nibbler
Niki-bot
NimbleCrawler NimbleCrawler
OnCrawl Nimbostratus
OpenLinkProfiler Ninja
OpenVAS Nutch
Octopus
Offline Explorer
OrangeBot OrangeBot
OutfoxBot OutclicksBot
PHPCrawl PECL::HTTP
POE-Component-Client-HTTP PageAnalyzer
PageScorer PageGrabber
PageThing.com PageThing.com
Pandalytics
Pavuk Pavuk
Picsearch Petalbot
PictureFinder Piepmatz
ProWebWalker Pixray
Probethenet Psbot
Pu_iN Pu_iN
Pump
PxBroker PxBroker
PyCurl
QueryN Metasearch
RSSingBot
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankurBot
Reaper ReGet
RebelMouse
Recorder
RedesScrapy RedesScrapy
Rogerbot RocketCrawler
SBIder
SEOkicks
SEOstats SEOstats
SalesIntelligent SalesIntelligent
Screaming Screaming
ScrepyBot ScrepyBot
SearchmetricsBot SearchmetricsBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
Siphon Shodan
SiteExplorer Site Sucker
SiteLockSpider SiteCheckerBotCrawler
SiteSnagger SiteSnagger
Sitebeam Sitebeam
Siteimprove Siteimprove
SocialRankIOBot Sitevigil
Sociscraper SmartDownload
Sogou web spider Snapbot
Sosospider Spbot
Sottopop
Spinn3r Spinn3r
SputnikBot
Sqlmap Sqlmap
Steeler Stripper
Sucuri
SuperHTTP SuperHTTP
SurveyBot
Suzuran Suzuran
Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
Teleport Teleport
Telesphoreo Telesphoreo
The Intraformant
Thumbor
TightTwatBot TightTwatBot
Titan Toata
Toweyabot
Trendictionbot
True_Robot True_Robot
Turnitin Turingos
TurnitinBot
Twice
URLy Warning URLy Warning
UnisterBot
V-BOT V-BOT
VB Project VB Project
VCI
Vacuum Vacuum
VeriCiteCrawler VelenPublicWebCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
Wallpapers/3.0 VoidEYE
Voil
WASALive-Bot
WEBDAV
WISENutbot
WallpapersHD WallpapersHD
Web Auto Web Auto
Web Collage Web Collage
Web Sucker
WebAuto
WebFetch
Web Fuck Web Fuck
WebImageCollector Web Pix
Web Sucker
WebBandit
WebCollage
WebGo IS
WebLeacher WebLeacher
WebReaper WebPix
WebSauger
WebWhacker
Webalta Webalta
WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor
Whack Whack
Whacker
Whatweb
Who.is Bot Who.is Bot
WinHTTrack WinHTTrack
WiseGuys Robot Xaldon WebSpider
Wprecon
YoudaoBot YoudaoBot
ZoominfoBot Zermelo
coccocbot-web Zeus
crawl.sogou.com ZoomBot
evc-batch ZumBot
instabid archive.org_bot
internetVista monitor backlink-check
check1.exe
crawler4j
demandbase-bot
facebookscraper
heritrix
isitwp.com isitwp.com
lwp-request
magpie-crawler magpie-crawler
mediawords mediawords
muhstik-scan netEstate NE Crawler
oBot pcBrowser
plumanalytics
probe-image-size probe-image-size
s1z.ru
satoristudio.net satoristudio.net
scalaj-http seoscanners
scan.lol serpstatbot
seobility
seostar
sexsearcher
sogouspider sogouspider
sysscan sp_auditbot
tAkeOut trendiction.com
trendiction.de webmeup-crawler
voyagerx.com xpymep1.exe
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
2pxg8bcf.top energydiet-info.ru
battle.net everypony.ru
bensbargains.net ezofest.sk
cheapwebsitehoster.com gelezki.com
healmytrauma.info kiinomaniak.pl
members.ghanaweb.com megakino.net
nowtorrents.com officedocuments.net
verymes.xyz rockma.se
webalan.ru space-worry.ml
x-stars.ru vertaform.com

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 0lovespells0.blogspot.com
12u.info 1001watch.com.ua
18ps.ru 100searchengines.com
1qingdao.com 101billion.com
2itech.ru 1hwy.com
4webmasters.org 2020iscoming.info
aa9d046aab36af4ff182f097f840430d51.com 202ch.com
above.com 3w24.com
abwa.tk 4kplayer.pl
adman.se 883zy.com
adobereader-free.ru 8lufu.com
adult-shop.com.ua 9icmzvn6.website
advancedcleaner.com a.pr-cy.ru
adventureparkcostarica.com accessoires-mode.in
adsfresh.men
adultnet.in
advancedmassagebysara.com
advokateg.ru
afmuseum.com afmuseum.com
agapovdg.ru air-edem.ru
agroeconom.kz alert.scansafe.net
aktivator-windows10.blogspot.com alienwheels.de
alekseevec.ru allesohnegirls.net
alibestsale.com allsilver925.co.il
all-number.com allstatesugarbowl.org
all-streaming-media.com amazingpic.net
allboard.xobor.de
allwomen.info
aludecor.info
amazon-adsystem.com
ameblo.jp
ananumous.ru ananumous.ru
animal36.com anaseracresar.tk
anonymousfox.co android-systems.ru
anti-virus-removal.info android-vsem.org
apple.com-webbrowsing-security.review angkortours.vn
arabseyes.com animal-drawings.com
arkkivoltti.net aosheng-tech.com
aruplighting.com apartmentratings.com
asscenihotosidea.blogspot.com apidigidocketnet-a.akamaihd.net
aussie-prizes.com apiportalmorecom-a.akamaihd.net
auto-moto-elektronika.cz appearance-cool.com
autoseo-traffic.com artavenuegardenstudios.com
aviav.ru asociatia-tipografilor-transilvania.ro
avtointeres.ru asrv-a.akamoihd.net
b-styles.xyz avon-ukraine.com
battlecarnival.su ayakino.net
bcmp.org azartclub.org
berdasovivan.ru babespcs.com
besttorrentknifta.weebly.com badavit.com.ua
bg6s0.com baikaleminer.com
bif-ru.info bambi.ck.ua
bimatoprost-careprost.com bang-hotties.com
biznesschat.net beremenyashka.com
bloggers.nl best-businessman.ru
blogos.kz best-mam.ru
bmw-ark.ru besthoro.ru
boc.kz bestmarriages.com
bochemit.com.ua betterscooter.com
bocoarchives.org bigcareer.info
bio-japan.net
bioinnovate.co
boo-arts.com
boole.onlinewebshop.net
bouda.kvalitne.cz
brainboosting.club
braintobe.win braintobe.win
butstrap.space brakehawk.com
buy-meds24.com buigas.00it.com
buytizanidineonline.blogspot.com business-made-fun.com
camsex.xxx-cam.webcam buttons-for-website.com
candiceloves.us buttons-for-your-website.com
buy-cheap-pills-order-online.com
cakesplus.com.au
callejondelpozo.es
canrole.com
canvas.gloverid.site
carapics.com carapics.com
cardsharp1.ru cdncash.org
carmuffler.net cheatcode-lita12.rhcloud.com
cartierlovestore.xyz chlooe.com
cdn.walkme.com christianlouboutinoutlet.win
cdncash.net clean-virus-mac.com
celejihad.info colobit.biz
cementaresearch.se comeondog.info
cheapmoncler.pw compliance-mark.top
cherrypointplace.ca contentlook.co
chiptuninger.com cool-wedding.net
chocolateslim-en-france.com copenergo.ru
cialgenisrx.com coprofam.org
ciproandtizanidine.blogspot.com cosmetic.donna7753191.ru
clickbanksites.info cowlmash.com
clickcash.com craftburg.ru
clothesforcash.com creatives.adbetclickin.pink
club-samodelkin.ru
coldfilm.ru
com-supportcenter.website
compliance-irvin.top
compliance-julia.top
compliance-nelson.top
construmac.com.mx
continent-e.tv
copro.pw
coslab.club
country-chic.ru
cowblog.fr
cpajunkies.com
crackguru.tk
crynet.cc crynet.cc
crystalslot.com cscwtalkto.site
custodieva.ru customcollegeessays.net
dangphoto.trade cv.wallhade.co
darkbooks.org darkages.info
date-withme.com db.speedup.gdn
dieswaene.com decorationspcs.com
dieta-personalna.pl deniven.1bb.ru
dipstar.org depositfiles-porn.ga
doktoronline.no descargar-musica-gratis.net
domain.webkeyit.com deutschehobbyhuren.net
dominateforex.ml dev-seo.blog
dosugrostov.site dikqlyremy.info
download-of-the-warez.blogspot.com dispo.de
djekxa.ru
dlya-android.org
dnepr-avtospar.com.ua
dnsrsearch.com
docs4all.com
docstoc.com
dorratex.tn
downloads-whatsapp.com
downtuptv.gq
dstroy.su
dumpsmania24.com dumpsmania24.com
duplicashapp.com dyshagi.ru
dverimegapolis.ru earncash.com.ua
e-avon.ru easycommerce.cf
e-buyeasy.com ecommerce-seo.com
eldiariodeguadalajara.com editmedios.com
electro-prom.com elektir.ru
elektronischezigarettekaufen2.tumblr.com elektronischezigarettekaufen1.myblog.de
elektrozigarette2.webs.com eloconcream.blogspot.com
elektrozigaretten1.tumblr.com
elidelcream.weebly.com
elvenar.com elvenar.com
eng-lyrics.com enskedesquashclub.se
engines-usa.com enthuse.computernetworksonline.com
erectile.bid eredijovon.com
erotische-geschichten-xxl.com erotik-kostenlos.net
ertelecom.ru este-line.com.ua
escort-russian.com eu-cookie-law.blogspot.com
essay-writing.work eupornstar.info
essayservicewriting.org europeanwatches.ru
european-torches.ru everytalk.tv
evehemming.blogspot.com.au exportshop.us
excaliburfilms.com expresstoplivo.ru
exct.net extlabs.io
exlarseva.webblog.es extremez.net
eyessurgery.ru
ezigarettekaufen2.mpbloggar.se
ezigarettenshop1.yolasite.com ezigarettenshop1.yolasite.com
facebook-mobile.xyz ezigaretteshop2.mywapblog.com
factorynightclub.com f-online.de
falcoware.com
fanrto.com
favorcosmetics.com favorcosmetics.com
fba-mexico.com fetishinside.com
fbdownloader.com filesdatabase.net
fearcrow.com filunika.com.ru
feargames.ru
fickenprivat.info
fickluder69.com
fil.ru
find1friend.com
findpik.com findpik.com
fiuxy.com fingerprintjs.com
flagcounter.me firesub.pl
flash4fun.com forexunion.net
floating-share-buttons.com formularz-konkurs.tk
focalink.com forodvd.com
folowsite.com foto-telok.net
forex-indextop20.ru fotopop.club
forum.doctissimo.fr fotosfotos.eu
forum69.info free-share-buttons.blogspot.com
foto-weinberger.at freelotto.com
fototravel.eu freemags.cc
foxweber.com frustrated-favorable.gq
freakycheats.com frvo.alptandem.ru
fuckmill.com fuckbuddybestgilf.info
game-mmorpg.net gamedayhouse.com
gemara.com gavuer.ru
germetiki.com.ua gay.adultgalls.com
getoutofdebtfree.org gdebestkupit.ru
getprismatic.com gearsadspromo.club
generalporn.org
getlamborghini.ga
gidonline.one
gilbertbanda.net
glicol.kz glicol.kz
gne8.com gogps.me
go2album.com gok-kasten.net
goo.ne.jp golfresa.lucania.se
google-liar.ru goodhousekeeping.com
googlsucks.com googlsucks.com
gorodservis.ru
gosmeb.ru gosmeb.ru
gotwebsite1.com gotcher.us
gowreckdiving.com gotomontenegro.net
greenzaim.ru gymi.name
growboxbl.ru
gsasearchengineranker.pw
guardlink.com
hayate.biz hayate.biz
hd720kino.ru hdhc.ru
hdseriale.pl headpharmacy.com
heartofpayne.xyz healgastro.com
heatpower.ru health-medical-portal.info
helicalpile.us hebr.myddns-flir.com
help.tpu.ru high-speed1.net
hentaiheroes.com hikesearch.net
hexpilot.com hkdiiohi.skyrock.com
highland-homes.com
hildinghr.se
hitsbox.info
holidaypics.org
homeandhealth.ru
homedo.fabpage.com homedo.fabpage.com
hornymatches.com homemade.gq
hotel-mkad.ru honyaku.yahoofs.jp
hotelcrocenzi.sm hostcritique.com
hotgirlhdwallpaper.com hundejo.com
hoverboardforsaledirect.com hydropump.su
hully.altervista.org ico.re
hyiphunter.org igtools.club
i-hobot.ru images-free.net
ideibiznesa2015.ru imgata.com
iflycapetown.co.za iminent.com
ilo134uloh.com imk.com.ua
iloveitaly.ru infobanks.ru
imagerydatabase.com inmate-locator.us
imguramx.pw
inboundlinks.win
infogame.name
insomniagamingfestival.com
interesnie-faktu.ru
interfucks.net
investsuccess.org
io9.com
iopeninghours.co.uk
iron-age.info
it-max.com.ua it-max.com.ua
iwantmyfreecash.com itronics.ca
izoll.ru izhevsk.zrus.org
jabimgo.pw jagg.info
jackwolfskinoutlet.online javidol.site
janavibekken.no jerseychinabizwholesale.us
jav-fetish.site
javatex.co.id
javstock.com
je7.us
joessmogtestonly.com
jumptap.com
justprofit.xyz
jwss.cc
kambasoft.com
kamen-e.ru kamen-e.ru
kasino-money.pw kazka.ru
keenoutlet.online kicknights.gq
keki.info kinoduh.ru
keywordhouse.com
keywordteam.net
kino-filmi.com
kinobaks.com
kinomaniatv.pl
kinosed.net
kinostorm.net kinostorm.net
kirov.zrus.org kiwi237au.tk
kiskinhouse.com knogg.net
kleine-titten.biz konteiner24.com
klejonka.info kumuk.info
komputers-best.ru kutikomi.net
koptims.tiu.ru lacloop.info
kotaku.com lanasshop.ru
kursy-ege.ru landoftracking.com
kuzinsp.ru lankarns.com
kwzf.net lapitec.eu
laptoper.net larchik.net
larger.io leadwayau.com
lawyersinfo.org leftofcentrist.com
lazy-z.com li-er.ru
leadn.pl lifehacĸer.com
legalrc.biz liky.co.ua
lennatin.info lion.bolegapakistan.com
lexaprogeneric.link lion.conboy.us
link.ac loginduepunti.it
linkbolic.com lost-alpha.ru
littlesunraiser.com lutherstable.org
louisvuittonoutletstore.net m4ever.net
luckyday.world machicon-akihabara.info
lyrics.home-task.com magnetic-bracelets.ru
mac-shield.com
madot.onlinewebshop.net
magda-gadalka.ru
mainlinehobby.net mainlinehobby.net
malls.com makenahartlin.com
marcowebonyodziezowe.pl malwareremovalcenter.com
marinetraffic.com manifestyourmillion.com
mature.free-websites.com marketingvici.com
me-ke.com maxxtor.eu
mebel-alait.ru mcnamaratech.com
megaapteka.ru medispainstitute.com.au
menetie.ru medizinreisen.de
mensandals.xyz mensandals.xyz
mesto-x.com migente.com
micasainvest.com mikrobiologies.ru
miniads.ca milblueprint.com
mirmedinfo.ru mini.7zap.com
missis.top mir-business-24.ru
misswell.net
mixx.com
mnogabukaff.net mnogabukaff.net
mobifunapp.weebly.com mobplayer.ru
mockupui.com monclerjacketsoutlet.win
mojpreskumanie.com moneyzzz.ru
mompussy.net morlat.altervista.org
monclercheap.xyz moviezbonkerssk.cf
moneyviking-a.akamaihd.net mp3ringtone.info
monsterdivx.com mrbojikobi4.biz
moonci.ru mriyadh.com
mountainstream.ms mulberryoutletonlineeu.com
moz.com music7s.me
muz-tracker.net muz-shoes.ru
mydirtystuff.com muzaporn.com
myfreemp3.eu myblogregistercm.tk
myhealthcare.com mymobilemoneypages.com
naval.jislaaik.com nacap.ru
nextrent-crimea.ru naj-filmy24.pl
napalm51.nut.cc
nbsproject.ru
new-post.tk
nicovideo.jp nicovideo.jp
nonews.co njkmznnb.ru
norththeface.store nl.netlog.com
nodding-passion.tk
nofreezingmac.work
noumeda.com
novodigs.com novodigs.com
novosibirsk.xrus.org nudepatch.net
oecnhs.info odesproperty.com
okonich.com.ua offer.camp
okroshki.ru office-windows.ru
online-zaymy.ru offside2.5v.pl
onlinedomains.ru oivcvx.website
onlyforemont.ru olvanto.ru
ons-add.men one.net.in
ossmalta.com online-hit.info
pammik.ru ontargetseo.us
partner-high.men ooo-gotovie.ru
partnerads.men orgasmatrix.com
pateaswing.com
pay2me.pl
paydayloanslocal.com
pcimforum.com
perfectpracticeweb.com perfectpracticeweb.com
perm-profnastil.ru perm.xrus.org
petitions.whitehouse.gov photo-clip.ru
peugeot-club.org piccshare.com
pewit.pw pinapchik.com
phobia.us piratecams.com
pizda.lol pizdeishn.com
pizza-tycoon.com pizza-imperia.com
poisk-zakona.ru playlott.com
pokemon-go-play.online porno-chaman.info
poppen-nw.net pornobrazzers.biz
porndl.org pornodojd.ru
porno.simple-image.com.ua pornofiljmi.com
pornoelita.info pornoslive.net
pornoklad.ru pourvous.info
pornosmola.info pozdravleniya-c.ru
pornozhara.com
positive2b.ru
powc.r.ca.d.sendibm2.com
pozdrawleniya.com
pr0fit-b0x.com pr0fit-b0x.com
privatefx-in.ru praisong.net
predmety.in.ua
pricheskaonline.ru
privacylocationforloc.com
privatov-zapisi.ru
privetsochi.ru privetsochi.ru
proekt-mos.ru projectforte.ru
promodj.com projefrio.com.br
promoforum.ru prombudpostach.com.ua
prostitutki-novgoroda.org pron.pro
proxyradar.com prostitutki-tyumeni.org
pukaporn.com prpops.com
purchasepillsnorx.com push-ad.com
pussysaga.com questionmarque.ch
pyrodesigns.com.au quickchange.cc
qwesa.ru quickloanbank.com
rankscanner.com r-e-f-e-r-e-r.com
redirlock.com rainbowice.ru
refererx.com razleton.com
relax.ru reelheroes.net
remont-fridge-tv.ru reeyanaturopathy.com
remont-komputerov-notebook.ru regdefense.com
reklama1.ru
remont-ustanovka-tehniki.ru
remontvsamare.su remontvsamare.su
remybutler.fr renecaovillasale.online
resant.ru rfserial.net
research.ifmo.ru riders.ro
ric.info rukino.org
rique.host.sk ruscoininvest.company
roll123.com rusoft-zone.ru
rubbed.us sabizonline.com
runtnc.net sampleletters.net
russkie-gorki.ru
sady-urala.ru
saltspray.ru
sanidumps.com
sarafangel.ru
savingsslider-a.akamaihd.net savingsslider-a.akamaihd.net
sax-sex.com sawin.beth.webd.pl
sayyoethe.blogspot.co.za scanner-andrew.top
scanmyphones.com scanner-jess.top
scanner-ivan.top school-diplomat.ru
scanner-jane.top score-ads.men
search-error.com screentoolkit.com
search.1and1.com sdi-pme.com
search.pch.com sdsjweb.com
search-goo.com
searchmywindow-a.akamaihd.net searchmywindow-a.akamaihd.net
semprofile.com secret.xn--oogle-wmc.com
semxiu.com security60-e.com
senger.atspace.co.uk segol.tv
seorankinglinks.com seinterface.com
seorankinglinks.us seo18.su
seorussian.ru seogadget.ru
sex-znakomstva.online seorank.info
sexsaoy.com serdcenebolit.com
sextracker.be servisural.ru
shivafurnishings.com sex-sex-sex5.com
shopperifymac.com sexobzor.info
signx.info sexreliz.com
silvercash.com sexs-foto.com
sindragosa.comxa.com sextracker.de
sisi-go.ru sexy.babes.frontend-stack.top
site5.com shiksabd.com
sloopyjoes.com shopsellcardsdumps.com
sneakyboy.com shtaketniki.kz
social-buttons.xyz sideeffectsoftizanidine.blogspot.com
soft-terminal.ru site-auditor.online
spamnuker.com small-game.com
speeddream.xyz smichovbike.cz
sportbetfair.com soc-econom-problems.ru
sptslmtrafms.com social-fun.ru
spy-app.info social-search.me
srdrvp.com socialmediasuggest.com
sticken.co softtor.com
stpicks.com softwaretrend.net
strana-solnca.ru sohoindia.net
stretchingabuckblog.com sophang8.com
soserfis.com
soundfrost.org
spbchampionat.ru
specialfinanceoffers.com
speechfoodie.com
ssconstruction.co
star61.de
steelmaster.lv
stockquotes.wooeb.com
strag-invest.ru
suggest-keywords.com suggest-keywords.com
supernew.org supernew.org
support.nopeas.sk supers.com.ua
surfbuyermac.com svetlotorg.ru
surffoundation.nl takeprofitsystem.com
swingerseiten.com tampabaywatch.org
tackletarts.co taranerymagesswa.blogspot.com
taraz.xkaz.org teenbbw.yopoint.in
taxi-v-eisk.ru templates.franklinfire.co
td-33.ru thecounter.com
tds-advert002.info thedownloadfreeonlinegames.blogspot.com
thegioixekhach.com thetardistimes.ovh
titelhelden.eu tiens2010.ru
tizanidine4mgstreetprice.blogspot.com tizanidinecost.blogspot.com
tizanidinecapsules.blogspot.com tizanidinemusclerelaxant.blogspot.com
tizanidineforopiatewithdrawal.blogspot.com tizanidinepills.blogspot.com
toproadrunner5.info tizanidinesideeffects.blogspot.com
topvidos.ru todohr.com
traffic-club.info tootoo.to
trafficgenius.xyz top-karkas.ru
trafficreceiver.club topkarkas.com
traffique.net torrent-to-magnet.com
traxdom.ru torrents-tracker.com
truebeauty.cc torrnada.ru
trustl.life toursmaps.com
tuberkuleznik.ru track.deriv.com
tvorozhnaja-zapekanka-recept.ru trainoffend.ml
txxx.com trubywriting.com
tytoona.com trucri.me
unimodemhalfduplefw.pen.io trustedmaccleaner.com
unmaroll.ya.ru tryrating.com
unrealcommander.com tvnewsclips.info
urll.eu uhod-za-sobakoj.ru
ukr-lit.com
uptime-as.net
upupa.net
urlopener.com urlopener.com
userequip.com userequip.com
vedomstvo.net vacuumcleanerguru.com
vesnatehno.ru vashsvet.com
vgoloveboli.net vchulkax.com
viagraneggrx.com veopornogratis.xxx
viagroid.ru viagengrarx.com
videochat.tv.br viberdownload10.com
videochat.mx
videositename.com
videtubs.pl
vigrx-original.ru
vinsit.ru vinsit.ru
vipsiterip.org
visitcambridge.org
viven.host.sk
vkmusics.ru
vladhistory.com vladhistory.com
vod.com.ua vseuznaem.com
vodaodessa.com vzubah.com
vozbujdenie.com walpaperlist.com
vrotike.ru wardreapptokone.tk
vsexkontakte.net webinstantservice.com
vtc.pw weblibrary.win
vucms.com website-analytics.online
wallpapersdesk.info website-datenbank.de
watchdogs-2.ru
webjam.com
webmonetizer.net
webpromotion.ae
website-speed-check.site website-speed-check.site
whatsupinfoley.com websiteaccountant.de
whipme.yopoint.in webtherapy.ru
wholesalejerseychinashop.com wellcome2slovenia.ru
wieseversa.no whatzmyip.net
wildnatureimages.com
winx-play.ru winx-play.ru
wjgony.com wjgony.com
www.pinnacle-bets.com wnoz.de
x-musics.com woman-orgasm.ru
x-true.info woodyguthrie.se
x69ty.ru wordpresscore.com
xedserver.com worldis.me
xep.info worst-sites.online
xfire.com writingservices17.blogspot.ru
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai www.event-tracking.com
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks www.wohnkabinen-shop.de
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai x-diesel.biz
xn--90acenikpebbdd4f6d.xn--p1ai x5market.ru
xn--d1acah0c.xn--p1ai xboxster.ru
xsion.net xcc24.pl
yoshkarola.zrus.org xgames-04.com
youjizz.com xn----7sbahjd3btneuw1joc.xn--p1ai
xn--80abgj3a5acid6ghs.top
xn--b1agm2d.net
xxlargepop.com
xxxmania.top
youdesigner.kz
youporn-forum.ga youporn-forum.ga
your-bearings.com yourothersite.com
yoursearch.me yun56.co
yugk.net zajm-zalog-krasnodar.ru
zambini.ru
zebradudka.com zebradudka.com
zeroredirect12.com zeroredirect8.com
zirondelli.it zigarettenonl.canalblog.com
znakom.sibtest.ru zootoplist.com
zone-kev717.info zootravel.com
zophim.me zrus.org
zscalertwo.net zs2vm.top

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
2345.com 007angels.com
2daytrendingnews.com 3w24.com
991mostfm.co.id 4pp13.com
a1security.com.ua 5000-cotydzien.com
aab94f698f36684c5a852a2ef272e031bb.com 51unlim.ru
aad1f4acb0a373420d9b0c4202d38d94fa.com 5tey7463.weebly.com
actulite.com 66cpwgln.space
adamoads.com 72-news.com
ads.gold aac500b7a15b2646968f6bd8c6305869d7.com
adtiger.tk advocatemsk.ru
adultmeetonline.info advokateg.xyz
ahmedabadwebs.com aibolita.com
allforminecraft.ru arquapetrarca.info
alta-realestate.com autovideobroadcast.com
appartement-stumm.at avkzarabotok.info
apple.com-cleaner.systems azbuka-mo.ru
aproposde.com bashtime.ru
avek.ru beauby.ru
bazaronline24.ru belinka.com.ua
bestofupload.info best-seo-software.xyz
betterhdporn.com bestwebsitesawards.com
bezlimitko.xyz betune.onlinewebshop.net
blacktwhite.com bif-ru.info
boeuklad.com bigames.online
bristolhotel.com bitcoins-live.ru
bscodecs.com bloggedporn.com
budpost.com.ua bookhome.info
bvlgariwallet.xyz brandehk.dk
cascadelink.org brendbutik.ru
casite-513049.cloudaccess.net brokergid.ru
castingbank.ru brothers-smaller.ru
cdn.walkme.com buy-meds24.com
cdncash.net candcstuccoandstone.com
cementaresearch.se capsderfudd.tk
cartechnic.ru
cdncash.org
cejewelry.xyz
cete.ru cete.ru
cheapmoncler.pw cfjump.com
cherubinimobili.it cheapcigarettesc.info
chocoslim.pro checkm8.com
cleansearch.net collegerentals.com
clmforexeu.com conciergegroup.org
clodo.ru coolgramgoods.com
compliance-jessica.top crirussian.ru
compliance-john.top darodar.com
compliance-olga.top dating-app.ru
conboy.us davebestdeals.com
cornerstone-countertops.com diegolopezcastan.com
countbertwistdisp26.soup.io digital-sale.su
counterbot.com dignitasdata.se
creditmoney.com.ua dispo.de
customcollegeessays.net dmmspy.com
daisye.top dms-sw.ru
danceuniverse.ru dood.live
de.zapmeta.com doyouknowtheword-flummox.ml
dealwifi.com drugstoreforyou.com
derevesendeco.com easyukraine.com
dev.citetick.com ecommerce-seo.org
diffbot.com ege-russian.ru
digital-video-processing.com elektrischezigarettekaufen2.cowblog.fr
discountbarbour.online elite-sex-finders.com
documentsite.net esopini.com
domaincheck.io everytalk.tv
domashneeruporno.com extlinks.com
download-wallpaper.net factorynightclub.com
druzhbany.ru faithe.top
dumpsmania24.com fashion-mk.net
electricwheelchairsarea.com ferrotodo.com
eloconcream.blogspot.com forexgb.ru
embedle.com foxydeal.com
englishdictionaryfree.com freenom.link
englishgamer.com freshberry.com.ua
essayassist.com g33.org
eu-cookie-law.info galeria-zdjec.com
exoclick.com gameprimary.com
ezigarettekaufen.myblog.de gne8.com
fbdownloader.com gojiberriess.apishops.ru
fearcrow.com google-liar.ru
financeloan.us growmyfunds.ca
formulaf1results.blogspot.com gwebtools.com.br
foto-sisek.porngalleries.top helvetia.com.ua
game-top.su hentaiheroes.com
geckoandfly.com highspeed5.net
get-free-social-traffic.com horoshieokna.com
glopages.ru hoste.octopis.com
golfresa.lucania.se hotcore.info
goodwinmetals.co hptwaakw.blog.fc2.com
googlefeud.com hundejo.com
googleseo.com.tr hvd-store.com
greenzaim.ru hyiphunter.org
guildebzh.info i-hobot.ru
hdwallpapers-free.com imadedinner.net
healing-dysplasia.ru indianmedicaltourismshop.com
healmytrauma.info inspiring-desperate.tk
hermesreplica.win integritylandscapeservices.com
hexpilot.com investmac.com
hobbyhuren-datenbank.com
hoverboard360.de
howopen.ru
igadgetsworld.com
ihc2015.info
iinstalll-fii1leis.jus0wil.pp.ua
iminent.com
importchinacoach-teach.com
infoupdate.org
inzn.ru
io9.com
james13prix.info
jerseychinabizwholesale.us jerseychinabizwholesale.us
keywords-monitoring-success.com jimmychoosale.online
kinobaks.com journeydownthescale.info
kladrus.ru joy-penguin.com
klin3952.ru kamalsinha.com
krasivoe-hd.net karpun-iris.ru
kursy-ege.ru karusel-market.ru
kiinomaniak.pl
klejonka.info
koptims.tiu.ru
kredit-blog.ru
laapp.com
labvis.host.sk
lafriore.ru
languagecode.com
larutti.ru
law-check-nine.xyz
lechenie-gemorroya.com lechenie-gemorroya.com
letslowbefast.today legionalpha.com
lida-ru.com lenvred.org
littlesunraiser.com lflash.ru
livingcanarias.com lion.bolegapakistan.com
m.b00kmarks.com loadopia.com
mackeeper-center.club longgreen.info
mall.uk louisvuittonoutletstore.net
maridan.com.ua m-google.xyz
me-ke.com malls.com
meetlocalchicks.com marcogrup.com
menstennisforums.com marketingtechniques.info
mere.host.sk maxxtor.eu
mojpregled.com meble-bogart.info
motherboard.vice.com megaapteka.ru
moviemail-online.co.uk meltwater.com
mtmtv.info menhealed.net
multgo.ru michaelkorsoutlet.store
musicvidz.ru misandesign.se
myshopmatemac.com monthlywinners.com
namecrumilchlet.tk mriyadh.com
narosty.com mydearest.co
new-post.tk myplaycity.com
newhairstylesformen2014.com mysex21.com
newsperuse.com nash-krym.info
nfljerseyscheapbiz.us nbsproject.ru
nhl17coinsforps3.gratisblog.biz newstudio.tv
nicovideo.jp offer.camp
nordvpn.com oktube.ru
nufaq.com onlainbesplatno.ru
obiavo.net onlinewritingjobs17.blogspot.ru
oecnhs.info onlyforemont.ru
onclickpredictiv.com ooo-gotovie.ru
onlinefilmz.net
ooo-olni.ru ooo-olni.ru
osagonline.ru outshop.ru
paleohub.info ownshop.cf
panchro.co.uk paceform.com
pendelprognos.se palocco.it
perfectplanned.com pdns.cz
piccdata.com perfectpracticeweb.com
picsearch.com phormchina.com
pictures-and-images.com pixgood.com
piuminiita.com pogruztehnik.ru
pornodojd.ru poisk-zakona.ru
pravoholding.ru pony-business.com
pshare.biz pornofeuer.com
py100.ru pornoklad.net
quickbuck.com portadd.men
quizzitch.net pozvonim.com
qwarckoine.com preparevideosafesystem4unow.site
realizmobi.com pro-poly.ru
redhotfreebies.co.uk profitsport.club
renecaovilla.online projefrio.com.br
renhacklids.tk prostitutki-almata.org
replicaclub.ru putitin.me
restaurantlescampi.com q-moto.ru
ringtonepartner.com rangjued.com
rosbalt.com.ua ranksonic.com
ruf777.com razyboard.com
russkoe-zdorovie.ru realitykings.com
s1z.ru redirect.trafficreceiver.club
sale-japan.com reward-survey.net
samolet.fr rockingclicks.com
scenarii-1-sentyabrya.uroki.org.ua rustic-quiver.win
sdrescher.net saecsa.co
searchencrypt.com sandhillsonline.com
sell-fb-group-here.com saneyes.com
semaltmedia.com sanyuprojects.com
semp.net savetubevideo.com
seoboxes.com scansafe.net
seoholding.com sdelai-prosto.ru
sexad.net seadragonherbery.com
sexvporno.ru seeresultshub-a.akamaihd.net
shop.acim.org sendearnings.com
sigmund-freud.co.uk sexsaoy.com
simul.co sierraapps.com
sivs.ru skachat-besplatno-obrazcy.ru
smartadserver.com slujbauborki.ru
smsactivator.ru smartshoppymac.com
socialmediasuggest.com snsdeainavi.info
softwaretrend.net soblaznu.net
space-worry.ml socas.pluto.ro
students-cheapskate.ml socialmonkee.com
susanholtphotography.com sodexo.com
swinger-mobil.net sohoindia.net
swinon.site soundfrost.org
swiped.su soviet-portal.do.am
tacbibirfa.tk spammen.de
tanieaukcje.com.pl spywarebegone.com
taqplayer.info stackthatbucks.com
teenporn18.net start.myplaycity.com
thaisamkok.com stopnarco.ru
thetardistimes.ovh stroyhelp-dv.ru
titan-ads.life studentguide.ru
tizanidineinteractions.blogspot.com sukirgenk.dvrlists.com
tizanidinenarcotic.blogspot.com supermodni.com.ua
takeprofitsystem.com
talant-factory.ru
taqywu51.soup.io
teenforporn.com
terrafootwear.us
thenetinfo.com
timetorelax.biz
tksn.ru
topmira.com topmira.com
track112.site torrents.life
trafficjunky.com trk-4.net
traffictrade.life trucri.me
trailer.cinemaflix.website tula.mdverey.ru
trkdf.com twiclub.in
turizmus.us uhdtv.website
twittruth.com userequip.com
typimga.pw
unmaroll.ya.ru
validccseller.com
validdomain.xyz validdomain.xyz
valmetrundan.se velen.io
varikoz24.com viagra.pp.ua
vodaodessa.com viktoria-center.ru
vse-pesni.com vipms.ru
wardreapptokone.tk vita.com.hr
weblo.com voloslove.ru
webscouter.net voprosotvet24.ru
whatistizanidine2mg.blogspot.com vykupavto-krasnodar.ru
winner7777.net vzglyadriv.kg
worldmusicfests.com vzubah.com
www.event-tracking.com w-journal.ru
xchangetrak.com webads.co.nz
xn----7sbahjd3btneuw1joc.xn--p1ai webpromotion.ae
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai word-vorlagen.net
xdoza.com
xmronta.com
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--80aanaardaperhcem4a6i.com xn--80aeahghtf8ac5i.xn--p1ai
xn--d1acah0c.xn--p1ai xn--b1agm2d.net
xpresscare.ru xn--l1aengat.xn--p1ai
xvideosbay.com xxxtubesafari.com
y8games-free.com zhacker.net
ynymnwbm.bloger.index.hr znakomstva-moskva77.ru
youdao.com
yuarra.pluto.ro
zajm-pod-zalog-nedvizhimosti.ru
zastroyka.org
zionstar.net
zoosexart.com
zskdla.site

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.44 0; 10.1.1.228 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:08 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2509 ### Version: V4.2021.08.2510
### Updated: Sat Aug 7 22:01:08 UTC 2021 ### Updated: Sun Aug 8 08:39:08 UTC 2021
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### Version Information ## ### Version Information ##