vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.07.2477 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-07-20 07:26:23 +00:00
parent 2e09bab1d6
commit 93ba15e0c3
55 changed files with 3612 additions and 1099 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.07.2476 #### Version: V4.2021.07.2477
#### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7068](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.07.2476 #### Version: V4.2021.07.2477
#### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7068](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -11,8 +11,8 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.07.2476 #### Version: V4.2021.07.2477
#### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7068](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
_generator_lists/bad-referrers.list
View file

@ -1,4 +1,3 @@
rczhan.com
000free.us 000free.us
007angels.com 007angels.com
00author.com 00author.com
@ -4900,6 +4899,7 @@ razvratnoe.org
razyboard.com razyboard.com
rcb101.ru rcb101.ru
rcpmda.ikan1080.xyz rcpmda.ikan1080.xyz
rczhan.com
real-time-analytics.com real-time-analytics.com
realitykings.com realitykings.com
realizmobi.com realizmobi.com
@ -7066,4 +7066,4 @@ zygophyceous.womanstars.site
zynax.ua zynax.ua
zytpirwai.net zytpirwai.net
zzbroya.com.ua zzbroya.com.ua
zzlgxh.com zzlgxh.com

4
_google_analytics_ghost_spam/README.md
View file

@ -3,8 +3,8 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.07.2476 #### Version: V4.2021.07.2477
#### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7068](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [614](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_google_analytics_ghost_spam/google-exclude-03.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-04.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-05.txt
View file

File diff suppressed because one or more lines are too long

1
_google_webmaster_disavow_links/google-disavow.txt
View file

@ -4899,6 +4899,7 @@ domain:razvratnoe.org
domain:razyboard.com domain:razyboard.com
domain:rcb101.ru domain:rcb101.ru
domain:rcpmda.ikan1080.xyz domain:rcpmda.ikan1080.xyz
domain:rczhan.com
domain:real-time-analytics.com domain:real-time-analytics.com
domain:realitykings.com domain:realitykings.com
domain:realizmobi.com domain:realizmobi.com

7
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

7
conf.d/globalblacklist-testing-version.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.07.2476 ### Version: V3.2021.07.2477
### Updated: Tue Jul 20 07:06:25 UTC 2021 ### Updated: Tue Jul 20 07:20:33 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

7
conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

7
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.07.2476 ### Version: V3.2021.07.2477
### Updated: Tue Jul 20 07:06:25 UTC 2021 ### Updated: Tue Jul 20 07:20:33 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.74 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.74 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.07.2476 ### Version: V3.2021.07.2477
### Updated: Tue Jul 20 07:06:25 UTC 2021 ### Updated: Tue Jul 20 07:20:33 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.74 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.74 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.07.2476 ### Version: V3.2021.07.2477
### Updated: Tue Jul 20 07:06:25 UTC 2021 ### Updated: Tue Jul 20 07:20:33 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.74 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/beta_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -4946,6 +4946,7 @@
"~*(?:\b)razyboard\.com(?:\b)" 0; "~*(?:\b)razyboard\.com(?:\b)" 0;
"~*(?:\b)rcb101\.ru(?:\b)" 0; "~*(?:\b)rcb101\.ru(?:\b)" 0;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0;
"~*(?:\b)rczhan\.com(?:\b)" 0;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 0; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 0;
"~*(?:\b)realitykings\.com(?:\b)" 0; "~*(?:\b)realitykings\.com(?:\b)" 0;
"~*(?:\b)realizmobi\.com(?:\b)" 0; "~*(?:\b)realizmobi\.com(?:\b)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.74 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.07.2476 ### Version: V3.2021.07.2477
### Updated: Tue Jul 20 07:06:25 UTC 2021 ### Updated: Tue Jul 20 07:20:33 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2476 2477

7
dev-tools/globalblacklist-testing.template
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V3.2021.07.2476 ### Version: V3.2021.07.2477
### Updated: Tue Jul 20 07:06:25 UTC 2021 ### Updated: Tue Jul 20 07:20:33 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

7
dev-tools/globalblacklist.template
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

1
dev-tools/referrers-regex-format-whitelist-test.txt
View file

@ -4899,6 +4899,7 @@
"~*(?:\b)razyboard\.com(?:\b)" 0; "~*(?:\b)razyboard\.com(?:\b)" 0;
"~*(?:\b)rcb101\.ru(?:\b)" 0; "~*(?:\b)rcb101\.ru(?:\b)" 0;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0;
"~*(?:\b)rczhan\.com(?:\b)" 0;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 0; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 0;
"~*(?:\b)realitykings\.com(?:\b)" 0; "~*(?:\b)realitykings\.com(?:\b)" 0;
"~*(?:\b)realizmobi\.com(?:\b)" 0; "~*(?:\b)realizmobi\.com(?:\b)" 0;

1
dev-tools/referrers-regex-format.txt
View file

@ -4899,6 +4899,7 @@
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.149.40.186 0; 20.94.253.59 0;

7
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.74 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.74 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.74 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.74 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.74 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -4946,6 +4946,7 @@
"~*(?:\b)razyboard\.com(?:\b)" 0; "~*(?:\b)razyboard\.com(?:\b)" 0;
"~*(?:\b)rcb101\.ru(?:\b)" 0; "~*(?:\b)rcb101\.ru(?:\b)" 0;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0;
"~*(?:\b)rczhan\.com(?:\b)" 0;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 0; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 0;
"~*(?:\b)realitykings\.com(?:\b)" 0; "~*(?:\b)realitykings\.com(?:\b)" 0;
"~*(?:\b)realizmobi\.com(?:\b)" 0; "~*(?:\b)realizmobi\.com(?:\b)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.74 0;
127.0.0.1 0; 127.0.0.1 0;

7
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.149.40.186 0; 20.94.253.59 0;

7
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.149.40.186 0; 20.94.253.59 0;

7
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

@ -4,9 +4,9 @@
### VERSION INFORMATION # ### VERSION INFORMATION #
################################################### ###################################################
### Version: V4.2021.07.2476 ### Version: V4.2021.07.2477
### Updated: Tue Jul 20 07:06:24 UTC 2021 ### Updated: Tue Jul 20 07:20:32 UTC 2021
### Bad Referrer Count: 7067 ### Bad Referrer Count: 7068
### Bad Bot Count: 614 ### Bad Bot Count: 614
################################################### ###################################################
### VERSION INFORMATION ## ### VERSION INFORMATION ##
@ -5942,6 +5942,7 @@ map $http_referer $bad_referer {
"~*(?:\b)razyboard\.com(?:\b)" 1; "~*(?:\b)razyboard\.com(?:\b)" 1;
"~*(?:\b)rcb101\.ru(?:\b)" 1; "~*(?:\b)rcb101\.ru(?:\b)" 1;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 1;
"~*(?:\b)rczhan\.com(?:\b)" 1;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 1; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 1;
"~*(?:\b)realitykings\.com(?:\b)" 1; "~*(?:\b)realitykings\.com(?:\b)" 1;
"~*(?:\b)realizmobi\.com(?:\b)" 1; "~*(?:\b)realizmobi\.com(?:\b)" 1;

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.74 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/custom-bad-referrers.conf
View file

@ -4946,6 +4946,7 @@
"~*(?:\b)razyboard\.com(?:\b)" 0; "~*(?:\b)razyboard\.com(?:\b)" 0;
"~*(?:\b)rcb101\.ru(?:\b)" 0; "~*(?:\b)rcb101\.ru(?:\b)" 0;
"~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0; "~*(?:\b)rcpmda\.ikan1080\.xyz(?:\b)" 0;
"~*(?:\b)rczhan\.com(?:\b)" 0;
"~*(?:\b)real\-time\-analytics\.com(?:\b)" 0; "~*(?:\b)real\-time\-analytics\.com(?:\b)" 0;
"~*(?:\b)realitykings\.com(?:\b)" 0; "~*(?:\b)realitykings\.com(?:\b)" 0;
"~*(?:\b)realizmobi\.com(?:\b)" 0; "~*(?:\b)realizmobi\.com(?:\b)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Cogentbot DomainAppender
Digincore IDBot
DittoSpyder Ltx71
Extreme Picture Finder Majestic12
JOC Web Spider NimbleCrawler
OutfoxBot Panscient
ScanAlert SeoSiteCheckup
UnisterBot Stripper
WebCollage arquivo.pt
Webster tAkeOut

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker 80legs
AIBOT ADmantX
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AllSubmitter
Aipbot
Alexibot
Alligator
AlphaBot
Anarchie Anarchie
Anarchy99 Anarchy99
Ankit
Anthill Anthill
Apexoo Apexoo
Aspiegel
Asterias Asterias
Attach
AwarioRssBot AwarioRssBot
BDCbot BDFetch
BacklinkCrawler BackDoorBot
BackWeb
Badass Badass
Bandit
Barkrowler Barkrowler
BetaBot BatchFTP
Battleztar Bazinga
Bigfoot
Bitacle
BlackWidow BlackWidow
Blackboard
Blow Blow
BotALot Boardreader
Brandprotect
Brandwatch Brandwatch
Bullseye Buck
BuzzSumo Buddy
CCBot BuiltWith
CSHttp CATExplorador
Calculon
CazoodleBot CazoodleBot
CensysInspect CensysInspect
CheeseBot CheeseBot
CherryPicker CherryPicker
ChinaClaw
Claritybot
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Cogentbot
Collector
Copier Copier
CopyRightCheck CopyRightCheck
Crescent Cosmos
DTS Agent Craftbot
Demon CrunchBot
Digincore DSearch
DatabaseDriverMysqli
DigitalPebble
Dirbuster Dirbuster
DnyzBot Discobot
Dispatch
DittoSpyder
DomCopBot
DomainAppender DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
EMail Wolf Dotbot
Ecxi ECCP/1.0
EMail Siphon
EasyDL
Ebingbong
EroCrawler EroCrawler
Evil Evil
Express WebPictures Express WebPictures
ExtLinksBot ExtractorPro
Extractor Extreme Picture Finder
EyeNetIE
FHscan
FlashGet FlashGet
Flunky Flunky
Foobot FrontPage
Freeuploader
Fyrebot Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW Genieo
GermCrawler
GetRight GetRight
Go-Ahead-Got-It Getintent
GoZilla Gigablast
Go!Zilla
Gotit Gotit
GrabNet
Grabber Grabber
Grafula Grafula
GrapeFX HEADMasterSEO
GrapeshotCrawler HTTrack
HMView Harvest
HTTP::Lite
HaosouSpider
Havij Havij
Heritrix Heritrix
IDBTE4M Hloader
IDBot IDBot
IRLbot
Id-search Id-search
IlseBot
Image Fetch
Image Sucker Image Sucker
IndeedBot InfoNaviRobot
Indy Library InterGET
InfoTekies
InternetSeer InternetSeer
Iskanie Iskanie
JikeSpider JennyBot
Joomla JustView
Jorgee
Kenjin Spider Kenjin Spider
Kozmosbot Kozmosbot
LWP::Simple LexiBot
Lanshanbot Libwhisker
LeechFTP
LeechGet
LibWeb
LieBaoFast LieBaoFast
Likse Lightspeedsystems
LinkScan
LinkWalker LinkWalker
Linkbot Linkbot
LinkpadBot Linkdexbot
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MSIECrawler MQQBrowser
MSFrontPage
Mag-Net
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic12 Majestic SEO
Mass Downloader
Masscan Masscan
Mata Hari
MauiBot
Meanpathbot Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mojeek Msrabot
NICErsPRO Musobot
Name Intelligence
Nameprotect Nameprotect
Navroad Navroad
Needle NetAnts
Net Vampire NetMechanic
NetZIP NetZIP
NextGenSearchBot Netcraft
Niki-bot Netvibes
Nikto Nibbler
Ninja Ninja
Offline Navigator Nuclei
OpenLinkProfiler Nutch
Openfind Octopus
OrangeSpider Offline Explorer
POE-Component-Client-HTTP OnCrawl
Page Analyzer OpenVAS
PageAnalyzer Openvas
PageGrabber Panscient
PageThing.com
Papa Foto Papa Foto
ProPowerBot Pavuk
ProWebWalker PeoplePal
PictureFinder
Piepmatz
Pimonster
Pixray
Pockey
Probethenet Probethenet
Psbot
Pu_iN Pu_iN
Pump RankActiveLinkBot
Quick-Crawler Rankivabot
RankurBot RankurBot
Re-re RealDownload
Recorder Reaper
Ripper SEOkicks-Robot
RocketCrawler
SBIder
SEOprofiler
SEOstats SEOstats
ScanAlert ScanAlert
Scanbot Searchestate
Scrapy SentiBot
ScreenerBot
Semrush
SeoSiteCheckup SeoSiteCheckup
Siphon SeobilityBot
SiteExplorer Seomoz
SiteLockSpider
Sitebeam
Sitevigil
SlySearch SlySearch
SmartDownload
Snoopy
Sociscraper Sociscraper
Sosospider
Sottopop
SpaceBison SpaceBison
Spammen SputnikBot
Sqlmap Stripper
Sqworm
Steeler
Sucuri
SuperBot
SuperHTTP
Surfbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
Teleport
TeleportPro
Telesoft
Telesphoreo Telesphoreo
The Intraformant
TightTwatBot
Titan Titan
Toata Toata
Toweyabot
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
Turingos True_Robot
TurnitinBot Turnitin
TwengaBot
Twice Twice
Typhoeus VelenPublicWebCrawler
URLy Warning VeriCiteCrawler
UnisterBot
V-BOT
Virusdie Virusdie
Voil Voil
Voltron
WASALive-Bot WASALive-Bot
WISENutbot
WPScan WPScan
WWW::Mechanize Wallpapers/3.0
Web Auto
Web Collage
Web Enhancer
Web Fuck
Web Pix
Web Sauger Web Sauger
WebBandit WebAuto
WebCollage
WebFetch
WebGo IS
WebImageCollector WebImageCollector
WebStripper
WebSucker WebSucker
WebZIP Webalta
Website Quester WebmasterWorldForumBot
Webshag
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Whatweb Whatweb
Who.is Bot Wonderbot
Widow Woobot
WinHTTrack
WiseGuys Robot
Wotbox Wotbox
Wprecon
YoudaoBot YoudaoBot
Zauba Zade
Zermelo
Zeus
ZmEu
ZoomBot
ZoominfoBot ZoominfoBot
arquivo.pt arquivo.pt
check1.exe backlink-check
coccocbot-web coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics com.plumanalytics
crawler.feedback
crawler4j crawler4j
demandbase-bot demandbase-bot
domainsproject.org
eCatch
facebookscraper
heritrix heritrix
isitwp.com instabid
pcBrowser internetVista monitor
plumanalytics lwp-request
satoristudio.net lwp-trivial
mediawords
oBot
page scorer
probe-image-size
ripz
s1z.ru
scalaj-http scalaj-http
scan.lol
seobility seobility
siteripz serpstatbot
sp_auditbot sexsearcher
spyfu
tAkeOut tAkeOut
trendiction.com trendiction.com
voyagerx.com xpymep1.exe
x22Mozilla
zgrab

284
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker
404enemy 404enemy
ADmantX
AIBOT AIBOT
Aboundexbot ASPSeek
Abonti
Aboundex
AfD-Verbotsverfahren
Aipbot Aipbot
Alligator Alexibot
AlphaBot
Anarchie
Ankit Ankit
Aspiegel BDCbot
Attach BackWeb
AwarioSmartBot
BBBike
BackStreet
BacklinkCrawler
Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bigfoot Bigfoot
Bitacle
Black Hole
BlowFish BlowFish
Bolt
BotALot
Brandprotect Brandprotect
Brandwatch
Buck Buck
Bullseye Buddy
BuzzSumo BuiltBotTough
CATExplorador BuiltWith
CazoodleBot CCBot
Calculon
CheTeam CheTeam
ChinaClaw Claritybot
Cloud mapping Cloud mapping
Crescent Cogentbot
Curious Copier
CyotekWebCopy CopyRightCheck
DSearch Copyscape
DataCha0s DBLBot
DatabaseDriverMysqli DatabaseDriverMysqli
Dirbuster Demon
Digincore
DigitalPebble
Disco Disco
Discobot Discoverybot
Dispatch Dispatch
DnyzBot DnyzBot
DomCopBot DomainCrawler
DomainAppender
DomainStatsBot DomainStatsBot
Domains Project Dragonfly
Dotbot Drip
Download Wonder ECCP/1.0
EasyDL EMail Siphon
Ebingbong EMail Wolf
Ecxi Ecxi
EirGrabber EroCrawler
Exabot ExtLinksBot
Express WebPictures Ezooms
FDM FDM
FHscan FHscan
FemtosearchBot Firefox/7.0
Flunky FlashGet
Freeuploader
FyberSpider FyberSpider
Fyrebot GT::WWW
GetRight GalaxyBot
Go-Ahead-Got-It Getintent
Go!Zilla
Gotit Gotit
GrabNet GrabNet
Grabber
Grafula Grafula
GrapeFX GrapeFX
GridBot GridBot
HEADMasterSEO HEADMasterSEO
HMView
HTTP::Lite HTTP::Lite
HaosouSpider Harvest
Havij
Heritrix Heritrix
Humanlinks Humanlinks
IRLbot IDBTE4M
Id-search Id-search
Image Fetch Image Sucker
Intelliseek InfoNaviRobot
InfoTekies
InterGET InterGET
Internet Ninja Internet Ninja
InternetSeer InternetSeer
Iria
IstellaBot IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
Jetty JetCar
JikeSpider
Joomla Joomla
Jorgee
JustView JustView
LWP::Simple Kozmosbot
Lanshanbot LNSpiderguy
Larbin
LeechGet LeechGet
Libwhisker
Likse Likse
LinkScan LinkextractorPro
Linkbot
Linkdexbot
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider Lipperhey
Lmspider Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MJ12bot
MSFrontPage MQQBrowser
MSIECrawler MSIECrawler
Mag-Net Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
MarkWatch MarkWatch
Mass Downloader
Masscan Masscan
MauiBot MegaIndex.ru
Meanpathbot MicroMessenger
Mister PiX Mister PiX
Mojolicious Mr.4x3
Morfeus Fucking Scanner NPbot
Msrabot
Musobot
Name Intelligence Name Intelligence
Nameprotect Nameprotect
Navroad Navroad
Needle Nessus
Net Vampire Net Vampire
NetAnts
NetLyzer
NetMechanic NetMechanic
Nettrack NetSpider
NextGenSearchBot NimbleCrawler
Ninja
Nuclei Nuclei
Nutch
Octopus Octopus
Offline Explorer Offline Navigator
OpenLinkProfiler
Openfind Openfind
Openvas
OrangeSpider OrangeSpider
PECL::HTTP OutfoxBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer
PageThing.com PageThing.com
Panscient Pandalytics
Pavuk Papa Foto
PeoplePal
Picscout Picscout
Picsearch Pixray
Piepmatz
Pimonster
ProWebWalker ProWebWalker
Probethenet Psbot
Pump Pump
PyCurl PyCurl
Quick-Crawler QueryN Metasearch
RSSingBot
RankActive RankActive
RankingBot2 RankingBot
Re-re RankurBot
ReGet RedesScrapy
RealDownload RepoMonkey
RebelMouse RocketCrawler
Ripper
Rogerbot
SEOlyticsCrawler SEOlyticsCrawler
ScoutJet SearchmetricsBot
Scrapy
ScreenerBot
SemrushBot SemrushBot
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
Seomoz
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSucker SiteSnagger
Sitebeam Sitevigil
Siteimprove
Snake
Snoopy Snoopy
SocialRankIOBot
Sottopop Sottopop
Spammen SpaceBison
Spanner
Spbot Spbot
Spinn3r
Sqlmap Sqlmap
Sqlworm
Sqworm Sqworm
Steeler
Sucuri
SuperHTTP
Surfbot
SurveyBot
Suzuran Suzuran
T0PHackTeam T0PHackTeam
T8Abot
TeleportPro
Telesoft Telesoft
Telesphorep Telesphorep
The Intraformant The Intraformant
TheNomad Thumbor
Titan Titan
Toweyabot
Trendiction Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
Turnitin
TwengaBot
Typhoeus Typhoeus
URLy.Warning
Upflow Upflow
V-BOT VCI
Vagabondo Virusdie
VelenPublicWebCrawler VoidEYE
Voltron Voil
WASALive-Bot WPScan
WEBDAV
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE WeSEE
Wallpapers/3.0
Web Auto
Web Collage Web Collage
Web Enhancer Web Enhancer
Web Fetch
Web Pix
Web Sucker Web Sucker
WebEnhancer WebCopier
WebFuck WebGo IS
WebImageCollector
WebLeacher WebLeacher
WebZIP WebPix
WebmasterWorldForumBot WebWhacker
WebsiteQuester WebsiteQuester
Webster
Whack Whack
WinHTTrack WinHTTrack
WiseGuys Robot Wonderbot
Xaldon WebSpider Wotbox
Xenu Wprecon
Zeus Xaldon_WebSpider
arquivo-web-crawler YoudaoBot
Zade
Zauba
Zermelo
ZoominfoBot
arquivo.pt arquivo.pt
crawl.sogou.com check1.exe
coccocbot-web
crawler.feedback crawler.feedback
crawler4j demandbase-bot
evc-batch eCatch
facebookscraper gopher
heritrix internetVista monitor
instabid ips-agent
isitwp.com
lwp-request lwp-request
magpie-crawler lwp-trivial
mediawords
muhstik-scan
page scorer page scorer
pcBrowser
plumanalytics plumanalytics
polaris version polaris version
ripz ripz
s1z.ru
scalaj-http scalaj-http
seostar scan.lol
sitechecker.pro seobility
seoscanners
siteripz siteripz
sogouspider
sp_auditbot
spyfu spyfu
sysscan trendiction.com
trendiction.de
webmeup-crawler
xpymep1.exe xpymep1.exe
zauba.io zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
dfwu1013.info cookingmeat.ru
empathica.com elektronischezi.livejournal.com
erotik-kostenlos.net evogarage.com
falcoware.com getoutofdebtfree.org
glcomputers.ru instabid.tech
hdwalls.xyz likesdesign.com
mansiondelrio.co mrbojikobi4.biz
maxthon.com ukrobstep.com
sex-tracker.de xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
vancleefreplica.pw zoompegs.com

938
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 00go.com
100dollars-seo.com 202ch.com
12masterov.com 23kw.ru
1314dh.com 24-ak.ru
3rbseyes.com 273-fz.ru
3weekdiet.com 2ads.co.uk
4inn.ru 2pxg8bcf.top
a.frcls.fr 2rich4bitches.com
aa81bf391151884adfa3dd677e41f94be1.com 5tey7463.weebly.com
absolute-s.ru 5ws.dating-app.ru
7minuteworkout.com
7wind.ru
7xc4n.com
8si.ru
aad1f4acb0a373420d9b0c4202d38d94fa.com
abbanreddy.com
above.com
abusora.com
abwa.tk abwa.tk
activepr.ru actulite.com
adbetclickin.pink adman.gr
adsfresh.men ads.gold
adultmeetonline.info adult3dgames.com
advokateg.ru afftrack001.com
agahinameh.com
aghanyna.com
ahrntal.verymes.xyz ahrntal.verymes.xyz
aktivator-windows10.blogspot.com airlandsea.info
all4wap.ru aksonural.ru
allforminecraft.ru alevinefesleri.com
alvtank.se alice.it
amospalla.es allknow.info
anal-acrobats.com alpinism.ru
anatomiy.com ample-awards-today.us
apibrowsesmartne-a.akamaihd.net anabolics.shop
apishops.ru analnoeporno.tv
aproposde.com analytics-ads.xyz
arcadeplayhouse.com ananumous.ru
artdeko.info animenime.ru
artel116.ru anlimebel.kiev.ua
asrvrep-a.akamaihd.net apiitsthirteende-a.akamaihd.net
atofilms.com apptester.tk
aucoinhomes.com arabgirls.adultgalls.com
auspolice.com arendakvartir.kz
avkzarabotok.info arewater.com
avtoarenda.by arkkivoltti.net
artdestshop.eu
asrv-a.akamoihd.net
atovh.local-finders.com
autoblogger24.info
automatic-seo.com
autowebmarket.com.ua
aviva-limoux.com
avtointeres.ru
azartmsl.com azartmsl.com
babespcs.com azartniy-bonus.com
bagsonsale.online azbukafree.com
baikaleminer.com backuperwebcam.weebly.com
baladur.ru bannerads.de
balois.worldbreak.com beauty-b0x.pl
bankcrediti.ru best-mam.ru
bannertesting.com bestempresas.es
bazaronline24.ru bestimagecoollibrary.com
best-businessman.ru bestpriceninja.com
bestcoin.cc besttorrentknifta.weebly.com
bestofupload.info bestwaystogetpaid.us
bestprofits-there.com bestwebsitesawards.com
betshuckclean.com bikini-image.com
betterhdporn.com bin-brokers.com
billiard-classic.com.ua bkgr.se
biocn.dx.am blockety.co
biznesluxe.ru blog.koreadaily.com
biznesrealnost.ru bmw-ark.ru
blackle.com bolegapakistan.com
blog.f00kclan.de bongiornos.info
bloggedporn.com bracketsmackdown.com
blue-square.biz brainzpod.win
bonkers.name break-the-chains.com
bonky.biz bronzeaid-a.akamaihd.net
bosefux.esy.es brothers-smaller.ru
bscodecs.com business-suggested.tk
buqyxa.rincian.info
businesxxl.com
buybest1.biz buybest1.biz
buytizanidineonlinenoprescription.blogspot.com buydissertation.net
buzzonclick.com buzzonclick.com
buzzzg.men bvlgaribracelet.xyz
c1.onedmp.com c-gainsbourg.com
cakemediahq.com.au cactussoft.biz
calvet.altervista.org callawaygolfoutlet.online
carabela.com.do callawaygolfstore.online
cartierbraceletsreplica.pw camdenmemorials.com
cartierlove.xyz candycrushshop.com
cayado.snn.gr candypeople.se
cbox.ws carina-sy.de
ccjp.eu cartujano-pre.de
centrdebut.ru cbozhe.com
cfacarrosserie74.com chastnoeporno.com
chatroulette.online cheapjerseysbizwholesale.us
cheapjerseysfootballshop.com check-host.net
christianlouboutinshoes.xyz chinaelitecheapjerseys.com
cl.s7.exct.net christianlouboutinreplica.pw
clarithromycin500mg.com cleanallspyware.com
clash-clans.ru click2pawn.com
clicksotrk.com clickaider.com
clippingphotoindia.com clipartnew.com
club-lukojl.ru cocolyze.com
compliance-barak.xyz comeondog.info
compliance-fred.top cometorussia.net
compucelunlock.net compliance-alexa.xyz
comsysnet.com compliance-julianna.top
contentsexpress.com connectionstrenth.com
cookielawblog.wordpress.com converse.ddsoldes.fr
coolwallpapers4k.info cornomase.win
costablue.xyz cougfan.info
counter.bloke.com creditwell.ru
covetnica.com customcatchcan.com
cpajunkies.com dairyindia.in
customchocolate.business-for-home.com damasarenai.info
cutt.us daymusam.com
czeshop.info ddpills.com
d0t.ru delfin-aqua.com.ua
dailyrank.net dellalimov.com
davebestdeals.com deluxedumps.com
davidovic.info
deniven.1bb.ru deniven.1bb.ru
dentfidemountpreach.tk dentalpearls.com.au
depositfiles-porn.ga detmebel.su
deshevo-nedorogo.ru dgfitness.co
detsky-nabytek.info distver.ru
dev33.dioniqlabb.se dkvorota.ru
devochki.top docstoc.com
dieswaene.com documentsite.net
directivepub.com dogbreedspicture.net
djonwatch.ru dotnetdotcom.org
djstools.com dpihatinh.gov.vn
doctissimo.fr draniki.org
domini.cat drev.biz
download-of-the-warez.blogspot.com drillsaw.ru
drugspowerstore.com dyt.net
drupa.com eachdayisagift.review
dumuelave.xyz earncash.com.ua
ege-russian.ru editors.choice6912650.hulfingtonpost.com
election.interferencer.ru ee77ee.com
elektrozigarette2.webs.com elektrischezigarette2.onsugar.com
elstal.com.pl elmacho.xyz
empathica.com embedle.com
enternet.ee engines-usa.com
erectile.bid erectile.bid
eropho.net eropornosex.ru
exdocsfiles.com erotiktreff24.info
exoclick.com es5.com
exportshop.us escort-russian.com
extstat.com escortslet.net
f-loaded.de essayservicewriting.org
f-online.de ezigarettekaufen2.mpbloggar.se
f012.de ezigarettenkaufen1.pagina.gr
fakehandbags.xyz falcon-images.blogspot.com
fandlr.com
fashion-stickers.ru
fba-mexico.com fba-mexico.com
fefo.gdn fdzone.org
felizporno.com fertilitetsradgivningen.se
filefilter.weebly.com finansov.info
filkhbr.com findpics.pw
financehint.eu finemanteam.com
findpik.com fix-website-errors.com
finstroy.net
floridamobilebillboards.com floridamobilebillboards.com
flowwwers.com footbalive.org
footballfarrago.com forexgb.ru
free-laptop-reward.com forum-engineering.ru
freshberry.com.ua forum20.smailik.org
freshdz.com foto-telok.net
galeria-zdjec.com foxjuegos.com
gamebackyard.com frameimage.org
franecki.net
free-deals.faith
free-share-buttons.blogspot.com
free411games.com
freejabs.com
freshmac.space
freshsuperbloop.com
futbolkisales.ru
fxtips.ru
galaxy-family.ru
gameplexcity.com gameplexcity.com
gamesprite.me gate5.co.za
gavuer.ru gazobeton-p.com.ua
gelezki.com gdebestkupit.ru
genericviagrasildenafiled.net ggiaro.com
get-your-social-buttons.info glogow.pl
getoutofdebtfree.org
gfaq.ru
global-ics.co.za
gne8.com gne8.com
googlepositions.com gogalleryawesome.com
googlsucks.com gojiberriess.apishops.ru
goroda-vsego-mira.ru golaya.pw
goodwinmetals.co
gratis-sexkontakte.com gratis-sexkontakte.com
greatdealshop.com greatfind-a.akamaihd.net
grtyi.com greatzip.com
gsbs.com.ua gribkovye-zabolevaniya.com
guarrasdelporno.xxx gribokstop.com
hamptonoaks.ca gsou.cf
hao123.com hatedriveapart.com
hasfun.com hdhc.ru
hauleddes.com heroz.fr
haveinc.xyz hesteel.pl
hayate.biz himgaws.pw
hdpixion.com hitsbox.info
heartofpayne.xyz hoste.octopis.com
hjaoopoa.top hotloans.ru
homegardenlova.com houdom.net
homelygarden.com howmuchdoestizanidinecost.blogspot.com
hotel-mkad.ru hoztorg-opt.ru
hoverboardforsaledirect.com hreade.com
howtoclean.club ihtec2019.org
https-legalrc.biz imabase.com
humbmosquina.tk indiakino.net
ibb.com.ua investsuccess.org
ideibiznesa2015.ru investzalog.ru
igru-xbox.net istanbulit.com
ikritikimou.gr jagg.info
incanto.in.ua jestr.org
incep.imagine-ex.co jofucipiku.tk
incolors.club kaac.ru
indetiske.ya.ru kartiniresto.com
infobabki.ru kashubadesign.ru
infogame.name kasino-money.pw
infoupdate.org keywordsuggest.org
intellektmedia.at keywordsuggests.com
ivanstroi.ru kfon.eu
iweblist.info kino-key.info
izhevsk.xrus.org
j-times.ru
jav-idol.com
javatex.co.id
javlibrary.cc
jerseywholesalebizchina.com
jerseywholesaleelitestore.com
jurajskie.info
jwss.cc
karusel-market.ru
kazan.xrus.org
kellyonline.xyz
kichenaid.ru
kinoduh.ru kinoduh.ru
kinofak.net
kinosed.net
kiwi237au.tk kiwi237au.tk
kollesa.ru kleine-titten.biz
konpax.com kmgamex.cf
koronirealestate.gr komputernaya-pomosh-moscow.ru
kostenlos-sexvideos.com krassh.ru
krasnodar.xrus.org krasula.pp.ua
labelwater.se ku6.com
laxob.com kumuk.info
layola.biz.tc kurwa.win
ledis.top ladov.ru
legalrc.biz leadn.pl
lenvred.org lechtaczka.net
lescinq.com lernur.net
letolove.ru lerporn.info
li-er.ru linkwithin.com
lietaer.com liran-locks.com
lineavabit.it littlesexdolls.com
live-sexchat.ru livefixer.com
livingcanarias.com lkbennettstore.online
loadingpages.me localflirtbuddies.com
lomb.co locksmith.jp
lrac.info lovelycraftyhome.com
lsex.xyz lovi-moment.com.ua
ltvperf.com luxup.ru
luckyshop.net.ua mabdoola.blogspot.com
m4ever.net maglid.ru
marinetraffic.com markjaybeefractal.com
master-muznachas.ru
masthopehomes.com
maximpartnerspr.com maximpartnerspr.com
medanestesia.ru medicines-choice.com
meduza-consult.ru mega-bony-2017.pl
member-quiz.com megahdporno.net
mensandals.xyz meltwater.com
merryhouse.co.uk menstennisforums.com
mhi-systems.ru mesbuta.info
minegam.com metarip.ru
minneapoliscopiers.com metascephe.com
missvietnam.org michaelkorsoutlet.store
mlvc4zzw.space michaelkorsoutletstore.net
mojowhois.com mitrasound.ru
montazhnic.ru mmoguider.ru
mnimmigrantrights.net
mobile-appster.ru
mobsfun.net
mockupui.com
moneyzzz.ru
monthlywinners.com monthlywinners.com
moonci.ru morocco-nomad-excursions.com
motherboard.vice.com moscow.xrus.org
moviezbonkerssk.cf mp3downloadhq.com
moz.com
mp3ringtone.info
mrvideospornogratis.xxx mrvideospornogratis.xxx
mundoaberrante.com mtmtv.info
my-floor.in.ua musflashtv.com
mydirtystuff.com mwtpludn.review
mydownloadengine.com my-cash-bot.co
mylida.org myonlinepayday.co
myra.top myplaycity.com
nagdak.ru mywallpaper.top
nalogovyy-kodeks.ru nac-bearings.ru
narco24.me namecrumilchlet.tk
naturalshair.site neodownload.webcam
nbsproject.ru netpics.org
news-speaker.com newstudio.tv
nextlnk12.com nodup.ru
nippon-bearings.ru nootrino.com
no-fuel.org nrv.co.za
nordvpn.com ntic.fr
oakridgemo.com nucia.biz.ly
obiavo.by o-o-6-o-o.com
obiavo.uz oda.as
officedocuments.net ok.ru
offtopic.biz okel.co
ok-ua.info okroshki.ru
okmedia.sk one-gear.com
okout.ru onlinefilmz.net
olympescort.com open-odyssey.org
onclickpredictiv.com origin-my.ru
oren-cats.ru osb.se11.ru
orenburg-gsm.ru osoznanie-narkotikam.net
ostrovtaxi.ru
oz-offers.com
paclitor.com
pammik.ru pammik.ru
pandroid.co panouri-solare-acoperis.com
parajumpersjakkesalgnorge.info paparazzistudios.com.au
partner-high.men pechikamini.ru
patol01.pw perm-profnastil.ru
pc-test.net petrovka-online.com
pcgroup.com.uy pharmacyincity.com
pelfind.me photorepair.ru
peugeot-club.org
photo.houseofgaga.ru
photochki.com
photostudiolightings.com photostudiolightings.com
phpdevops.com phrcialiled.com
piccdata.com physfunc.ru
pillscheap24h.com picturesfrom.com
pinapchik.com picturesify.com
piulatte.cz picturesmania.com
pizza-imperia.com picurams.pw
pk-pomosch.ru pila.pl
pl.id-forex.com pipki.r.acdnpro.com
platezhka.net piski.top
pliks.pl pizda.lol
pochtovyi-index.ru playmsn.com
poisk-zakona.ru pod-muzyku.club
popserve.adscpm.net pogosh.com
popup.matchmaker.com pony-business.com
pornofiljmi.com popander.mobi
pornohd1080.online porndairy.in
pornokajf.com porno-chaman.info
posible.net pornowarp.info
preventheadacheguide.info pornzone.tv
pricheskaonline.ru pretty-mart.com
pro-poly.ru pro-poly.ru
promover.org profitsport.club
promoforum.ru
propranolol40mg.blogspot.com
pssucai.info
pulse33.ru
purplesphere.in
puzo2arbuza.ru
puzzleweb.ru puzzleweb.ru
pwwysydh.com pwwysydh.com
qor360.com pxhdwsm.com
qpypcx.com quebec-bin.com
ranksays.com quick-seeker.com
realresultslist.com quit-smoking.ga
renewablewealth.com rankia.com
rent2spb.ru real-time-analytics.com
ringporno.com realtytimes.com
robertefuller.com recipedays.com
rubbed.us reckonstat.info
rusexy.xyz refererx.com
rutor.group remontgruzovik.ru
s-forum.biz replica-watch.ru
samara.rosfirm.ru ricorsogiustizia.org
santechnik.jimdo.com risparmiocasa.bz.it
sanyuprojects.com rospromtest.ru
sayyoethe.blogspot.co.za royal-betting.net
sbt-aqua.ru runofilms.ru
sbtdesign.co.uk rusoft-zone.ru
scanmyphones.com rustic-quiver.win
scanner-alex.top sadaholding.com
scanner-fred.top saneyes.com
scenicmissouri.us sarm.tk
scrapy.org saveindex.xyz
se.bnt-team.com sbetodiodnye-lampy.ru
search.alot.com scanner-nelson.top
scoopquest.com
scottbywater.com
sdrescher.net
searchencrypt.com
searchinterneat-a.akamaihd.net
sell-fb-group-here.com
seminarykansas904.ml seminarykansas904.ml
seotoolsagency.com seoheap.com
service.adtech.us serptehnika.ru
servingnotice.com servethis.com
serw.clicksor.com sexvporno.ru
sex-foto.pw shop.acim.org
sexblog.pw sibtest.ru
sexkontakte-seite.com
sextracker.de
sexvideo-sex.com
sexy.babes.frontend-stack.top
shareyards.com
shemalegalls.blogporn.in
shijian.ac.cn
shisha-swag.de
shtora66.ru
sibdevice.ru
sildenafilcitratemed.com sildenafilcitratemed.com
silvermature.net sitechecker.pro
simply.net siteexpress.co.il
simul.co siteimprove.com
sirpornogratis.xxx
sisiynas.ru
sitiz.club sitiz.club
skachat-besplatno-obrazcy.ru sklad-24.ru
sky-mine.ru slavkokacunko.de
slavia.info slimcdn.com
snjatie-geroinovoy-lomki.ru slowmacfaster.trade
social-buttons.com slujbauborki.ru
softtor.com slutloadlive.com
softxaker.ru soaksoak.ru
soundfrost.org soft-program.com
space4update.pw sonata-arctica.wz.cz
spyfu.com sonnikforme.ru
sr-rekneskap.no spb-plitka.ru
ssconstruction.co spb.ru
stanthonyscatholicchurch.org spravka130.ru
spywarebegone.com
squidoo.com
starwars.wikia.com
stathat.com stathat.com
stauga.altervista.org staynplay.net
steelmaster.lv stpicks.com
stop-zavisimost.com strangeduckfilms.com
storehouse.ua stroydetali.ru
streetfire.net supernew.org
stroicol.net surintech.ac.th
stromerrealty.com swinginwithme.ru
superlist.biz tacbelarus.ru
taboola.com takethatad.com
taranerymagesswa.blogspot.com tandvardshuset.net
tattooha.com tattomedia.com
thetoiletpaper.com telefonsexkostenlos.tk
titslove.yopoint.in theprofitsmaker.net
tizanidineandcipro.blogspot.com thfox.com
tizanidinehcl4mgisitanarcotic.blogspot.com tiandeural.ru
tizanidinepills.blogspot.com timetorelax.biz
tizanidinerestlesslegsyndrome.blogspot.com tivolibasket.it
tonerbox.kz tizanidinehcl4mgtablet.blogspot.com
toondinsey.com tizanidineingredients.blogspot.com
top1-seo-service.com tizanidinevscyclobenzaprine.blogspot.com
top10registrycleaners.com toloka.hurtom.com
torrentdownloadhub.com tonivedu.it
toys.erolove.in top10-way.com
tpu.ru tour-line.net
trafficjunky.net trackzapper.com
trafficnetzwerk.de trichizobswiv.agddns.net
trion.od.ua trkdf.com
tula.howotorg.ru trumpetedextremes.com
tupper-posuda.ru trustl.life
uchil.net tsstcorpcddvdwshbbdriverfb.aircus.com
ugguk.online ttrraacckkrr.com
ukrtvir.com.ua tube8.com
undergroundcityphoto.com turist-strani.ru
unisexjewelry.org turizm.bz
uptime-delta.net ucsol.ru
urdoot.win uk-zheu20.ru
us-america.ru ukkelberg.no
ussearche.cf ukrobstep.com
vacuumcleanerguru.com ultimateclassicrock.com
vadimkravtcov.ru um-razum.ru
vbikse.com unionmarkt.de
via-energy-order.com unrealcommander.com
via-gra.webstarts.com uptime-gamma.net
uytmaster.ru
vchulkah.net
ventelnos.com
viagraneggrx.com viagraneggrx.com
video-production.com videochat.tv.br
vip-file.com vintontech.info
vip-parfumeria.ru visa-pasport.ru
visionwell.com.cn vladhistory.com
vod.com.ua vladimir.zrus.org
vodaodessa.com volume-pills.biz
voltrknc1.com vpnmouse.com
vote-up.ru w3data.co
vvon.co.uk walkme.com
vysigy.su wallpaperstock.net
w3javascript.com web-analytics.date
wallpapersist.com webcamdevochka.com
webads.co.nz
webcamtalk.net webcamtalk.net
webextract.profound.net wheelchairliftsarea.com
wemarketing.se wholesalejerseychinaoutlet.com
wesharepics.info
westsextube.com
whatzmyip.net
whiteproduct.com
wholesalejerseys-cheapest.com
whos.amung.us whos.amung.us
winner7777.net woman-orgasm.ru
wma-x.com woman-tampon.ru
worldtraveler.world word-vorlagen.xyz
woweb.com.ua writersgroup580.web.fc2.com
wstroika.ru www.arenda-yeisk.ru
www.pinnacle-bets.com www.souvenirua.com
www.wohnkabinen-shop.de x-musics.com
wygraj-teraz.com x5market.ru
x-rates.ru xedserver.com
xfire.com xn----7sbifcamovvfggw9d.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aeahghtf8ac5i.xn--p1ai xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--80atua3d.xn--p1ai xn--90acenikpebbdd4f6d.xn--p1ai
xn--e1aggki3c.xn--80adxhks ynymnwbm.bloger.index.hr
xvideosbay.com yourmovies.pl
xxxasianporn.net yourtemplatefinder.com
yopoint.in zajm-zalog-krasnodar.ru
youandcredit.ru zaobao.com.sg
youporn-forum.ga zeroredirect2.com
yourporngay.com zeroredirect8.com
ytmnd.com zhongwenlink.com
zagadki.in.ua zhuravlev.info
zahvat.ru zigarettenonlinekaufen1.blox.pl
zaimhelp.ru zigarettenonlinekaufen2.drupalgardens.com
zakazfutbolki.com
zazagames.org
zdesoboi.com
zebramart.ru
zeroredirect1.com
zlatnajesen.com
zog.link
zojirushi-products.ru zojirushi-products.ru
zoodrawings.com
zoogdiesney.com
zooggames.com
zrelaya.pw
zreloeporno.tv
zrizvtrnpale.tk
zzbroya.com.ua

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
11235813.webzdarma.cz 2x2fan.ru
14b.info 40cg.com
1bet.com aab94f698f36684c5a852a2ef272e031bb.com
20pascals.nl abcdefh.xyz
3th.co.in absolutelyanalog.com
3xforum.ro admitad.com
4inn.ru amazingninja.com
4k-player.pl amazon-adsystem.com
4replicawatch.net
absolute-s.ru
adman.gr
ads.gold
adventureparkcostarica.com
adverster.com
alot.com
anapa-inns.ru anapa-inns.ru
angkortours.vn animal36.com
animenime.ru animalia-life.club
appearance-cool.com arclk.net
arabgirls.adultgalls.com artavenuegardenstudios.com
arkartex.ru autochoixspinelli.com
asmxsatadriverin.aircus.com aviapanda.ru
asrvvv-a.akamaihd.net avirasecureserver.com
avitocars.ru baikaleminer.com
avto-oligarh.ru bausparen.bz.it
avtovolop.ru belsetirehafi.tk
awency.com bestcoin.cc
ayodhya.co bestserials.com
azartniy-bonus.com bet365.com
azarttoto.com bitporno.sx
azazaporn.com board.f00d.de
azlex.uz bolegapakistan.com
baersaratov.ru book-bets.com
balook.com brainsandeggs.blogspot.com
bannerconnect.net brateg.xyz
bdsmgalls.net bristolhotel.com.ua
best-seo-solution.com buchananshardware.com
bestmobilityscooterstoday.com calc-for-credit.ru
bet-prognoz.com carrauterie.be
bezcmexa.ru cartierlovebraceletreplica.xyz
bolitgorlo.net
bongiornos.info
bosefux.esy.es
brainxs.us
breget74.ru
buytizanidineonlinenoprescription.blogspot.com
cakesplus.com.au
candypeople.se
carsdined.org
cartierlovestore.xyz cartierlovestore.xyz
cartierreplica.pw cartierreplica.pw
cbb1smartlist12.click cfacarrosserie74.com
ccbill.com citizenclsdriveri7.pen.io
centrdebut.ru clean-start.net
ceotrk.com cobaltpro.ru
cheap-trusted-backlinks.com coinsspb.com
cheapestjerseysonwholesale.com compliance-ivan.xyz
cheapjerseysbizwholesale.us compliance-jack.top
christianlouboutinreplica.win computernetworksonline.com
christianlouboutinsaleoutletonline.us cv.wallhade.co
colobit.biz darmebel.com.ua
community.allhiphop.com day-news.info
copenergo.ru dealitright.click
country-chic.ru design-ideas.info
dayibiao.com discovertreasurenow.com
dellalimov.com doctissimo.fr
djstools.com downloader12.ru
dollartree.info downloadmefiranaratb1972.xpg.com.br
donna7753191.ru doyouknowtheword-flummox.ml
e-collantes.com dvr.biz.ua
educatemetv.com eas-seo.com
educontest.net ekosmetyki.net.pl
elektronischezigarettekaufen2.tumblr.com elektrischezigaretten1.blogse.nl
elektrozigarette1.dreamwidth.org elektrischezigaretten2.beeplog.com
elitedollars.com elexies.info
elmifarhangi.com eorogo.top
elvenar.com esfchat.tk
erotik-kostenlos.net etenininrade.ga
errorfixing.space
estibot.com
euroskat.ru
everytalk.tv
evidencecleanergold.com
evrotekhservis.ru evrotekhservis.ru
expolicenciaslatam.co excitacion.info
ezigarettenkaufen1.pagina.gr exlarseva.webblog.es
f012.de ezigarettekaufen2.blox.pl
fandlr.com familienzahnaerzte.com
fdzone.org fastcrawl.com
femmesdenudees.com fetishinside.com
fickenprivat.info filefilter.weebly.com
fickkontakte.org filmci.pro
firstsiteguide.com firstdrugmall.ru
flavors.me franecki.net
forpostlock.ru front.to
forums.toucharcade.com fyl.com.ru
forzeronly.com ge0ip.com
fuel-gas.com ghostvisitor.com
gagrasector.ru gifspics.com
geilehausfrauen.net gigapeta.com
gelezki.com gonextmedia.com
gidonline.one goodnightjournal.com
gigablast.com goosefishpost.bid
gribokstop.com gotomontenegro.net
gwagka.com gotorussia.com
gymi.name greatidea.marketing
harmonyglen.us gta-club.ru
healmytrauma.info guge.io
helpmymacfaster.trade handsandlegs.ru
hostnow.men hanwei.us
hotelcrocenzi.sm heliko.no
houseofgaga.ru hem.passagen.se
hoverboardforsaledirect.com heroz.fr
ikearugs.xyz historichometeam.com
impisr.ru hornymatches.com
inbabes.sexushost.com hothor.se
iplusbit.blogspot.co.za houseofrose.com
isoveti.ru ico.re
jeremyeaton.co ifmo.ru
kareliatobacco.ru igithab.com
keenoutlet.online importchinacoach-teach.com
kerwinandcariza.com instabid.tech
knigonosha.net insurple.com
kosynka-games.ru int.search.tb.ask.com
krassh.ru intermesh.net
kustanay.kz internetartfair.com
lalalove.ru ipostroika.ru
le-clos-des-alouettes.com iqoption-bin.com
lego4x4.ru it-max.com.ua
lescinq.com iyasimasennka.com
lexiangwan.com jacago.com
linerdrilling.com johannesburgsingles.co.za
loadingpages.me kfon.eu
mahnwachen-helfen.info kinobaks.com
manyresultshub-a.akamaihd.net labvis.host.sk
mastershef.club landinez.co
mebel-alait.ru larger.io
meetlocalchicks.com laserpen.club
meget.co.za lecbter-relationships.ga
migente.com lernur.net
moesen-ficken.com light.ifmo.ru
morefastermac.trade littleberry.ru
most.gov.iq liumimgx.pw
mototsikl.org lombardfinder.ru
moviemail-online.co.uk lowephotos.info
movies-in-theaters.net maju.bluesalt.co
mydoctorok.ru makis.nu
mygameplus.com mantramusic.ru
nardulan.com mararoom.ru
niroo.info marsgatan.com
no-fuel.org mediaoffers.click
onlinemegax.com metallosajding.ru
ororodnik.goodbb.ru mir-business-24.ru
morpicert.pw
mrsdalloways.com
mug-na-chas-moscow.ru
naturalbreakthroughsresearch.com
next-dentists.tk
ningessaybe.me
nlfjjunb5.ru
nofreezingmac.work
novgorod.xrus.org
nsatc.net
o-dachnik.ru
okayimage.com
olvanto.ru
omoikiri-japan.ru
onlinemeetingnow.com
onstrapon.purplesphere.in
oranga.host.sk
origin-my.ru
osagonline.ru
ownshop.win ownshop.win
paparazzistudios.com.au ozoz.it
pekori.to pc-services.ru
pflexads.com phormchina.com
photosaga.info
picscout.com picscout.com
piski.top pictures-and-images.com
pistonclasico.com pizza-imperia.com
pix-hd.com pogruztehnik.ru
polybuild.ru polcin.de
popander.mobi pornhub-forum.ga
porngalleries.top pornofiljmi.com
porno-dojki.net pornozhara.com
pornoinn.com predominant-invent.tk
pornosee.info preparevideosafesystem4unow.space
pozdrawleniya.com princevc.com
pssucai.info profitkode.com
puzo2arbuza.ru prostitutki-astana.org
pyrodesigns.com.au puserving.com
qitt.ru quick-offer.com
rankscanner.com quickloanbank.com
raschtextil.com.ua qwarckoine.com
realizmobi.com qwesa.ru
recipedays.ru rainbowice.ru
reklama1.ru razorweb-a.akamaihd.net
rethinkwasteni.info remontvsamare.su
rewardpoll.com ribieiendom.no
ricorsogiustizia.org ringtonepartner.com
roleforum.ru royalcar-ufa.ru
ru-dety.ru saneyes.com
rustic-quiver.win sawin.beth.webd.pl
rvzr-a.akamaihd.net sbricur.com
sa-live.com scanner-marwin.top
samoiedo.it scanner-nelson.top
scenicmissouri.us screentoolkit.com
sdelai-prosto.ru seadragonherbery.com
searchmywindow-a.akamaihd.net sell-fb-group-here.com
seo-tools-optimizing.com semrush.com
seoheap.com serving.adbetclickin.pink
seorankinglinks.com shivafurnishings.com
servicecenter.co.ua shopwme.ru
shoesonlinebuy.xyz sofit-dmd.ru
signx.info steame.ru
sinel.info streetfire.net
siteaero.com stroilka.info
sklad-24.ru sudexpert66.ru
small-game.com supermesta.ru
smokewithrabbits.com survival.betteroffers.review
snegozaderzhatel.ru symphonyintegratedhealthcare.com
soft1.ru tacbibirfa.tk
sotechco.co tackletarts.co
spbchampionat.ru telefonsexkostenlos.tk
ssconstruction.co teletype.in
stairliftstrue.com template-kid.com
startwp.org
statustroll.com
sticken.co
stroydetali.ru
sukirgenk.dvrlists.com
surflinksmedical.com
tacbelarus.ru
tattooha.com
taxi-v-eisk.ru
the-torrent-tracker.blogspot.com the-torrent-tracker.blogspot.com
thegameriders.com thetoiletpaper.com
thegioixekhach.com tizanidinevssoma.blogspot.com
theguardlan.com toondinsey.com
thejournal.ru trustl.life
theplacetoupdating.pw tubeoffline.com
tizanidinedosageforsleep.blogspot.com tuckermktg.com
tizanidineinteractions.blogspot.com turbabitload.weebly.com
tizanidinewithdrawalsymptoms.blogspot.com turbodsp.com
torrentred.games turizm.bz
turizmus.us txxx.com
twitlinks.com userequip.com
uggbootsoutletsale.us ussearche.cf
uginekologa.com videojam.tv
valoresito.com virtuagirl.com
versaut.xxx-cam.webcam visa-china.ru
viel.su vizitki.net
vkonche.com voloomoney.com
watchdogs-2.ru voltrknc1.com
vremya.eu
vvon.co.uk
wait3sec.org
wallpaperstock.net
waycash.net
web-betting.ru
webix.biz webix.biz
wellcome2slovenia.ru webs.com
whatzmyip.net winx-play.ru
womensplay.net womama.ru
wosik-dach.service-for-web.de wygraj-teraz.com
www.wohnkabinen-shop.de xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xerox-douglas.cf xn----7sbahjd3btneuw1joc.xn--p1ai
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn--oogle-wmc.com xn--h1ahbi.com.ua
yaminecraft.ru xtremeeagles.net
yousense.info yebocasino.com
ytmnd.com yourporngay.com
zakazvzloma.com yugo-star.ru
zed21.net zaobao.com.sg
zeleznobeton.ru zapiszto.pl
zeroredirect12.com zatjmuzu.info
zdesoboi.com
zoodrawings.com
zoogdisany.com
zscalerone.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.74 0;
127.0.0.1 0; 127.0.0.1 0;

1246
dev-tools/tmp/robots.txt
View file

File diff suppressed because it is too large Load diff

1246
robots.txt/robots.txt
View file

File diff suppressed because it is too large Load diff