vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2921 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-16 07:30:37 +00:00
parent d07b7bb79c
commit a4f9464b84
56 changed files with 91302 additions and 91281 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2920 #### Version: V4.2022.02.2921
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2920 #### Version: V4.2022.02.2921
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -13,8 +13,8 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2920 #### Version: V4.2022.02.2921
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

11270
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

4
_generator_lists/bad-referrers.list
View file

@ -1,4 +1,3 @@
metafilter.com
000free.us 000free.us
007angels.com 007angels.com
00author.com 00author.com
@ -3771,6 +3770,7 @@ mesbuta.info
message-warning.net message-warning.net
mesto-x.com mesto-x.com
metabar.ru metabar.ru
metafilter.com
metallosajding.ru metallosajding.ru
metalonly.info metalonly.info
metarip.ru metarip.ru
@ -7088,4 +7088,4 @@ zygophyceous.womanstars.site
zynax.ua zynax.ua
zytpirwai.net zytpirwai.net
zzbroya.com.ua zzbroya.com.ua
zzlgxh.com zzlgxh.com

4
_google_analytics_ghost_spam/README.md
View file

@ -3,8 +3,8 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2920 #### Version: V4.2022.02.2921
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7090](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_google_analytics_ghost_spam/google-exclude-03.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-04.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-05.txt
View file

File diff suppressed because one or more lines are too long

1
_google_webmaster_disavow_links/google-disavow.txt
View file

@ -3770,6 +3770,7 @@ domain:mesbuta.info
domain:message-warning.net domain:message-warning.net
domain:mesto-x.com domain:mesto-x.com
domain:metabar.ru domain:metabar.ru
domain:metafilter.com
domain:metallosajding.ru domain:metallosajding.ru
domain:metalonly.info domain:metalonly.info
domain:metarip.ru domain:metarip.ru

11277
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11277
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11277
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11277
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.18 0;
127.0.0.1 0; 127.0.0.1 0;

11277
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.18 0;
127.0.0.1 0; 127.0.0.1 0;

11277
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.18 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/beta_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -3817,6 +3817,7 @@
"~*(?:\b)message\-warning\.net(?:\b)" 0; "~*(?:\b)message\-warning\.net(?:\b)" 0;
"~*(?:\b)mesto\-x\.com(?:\b)" 0; "~*(?:\b)mesto\-x\.com(?:\b)" 0;
"~*(?:\b)metabar\.ru(?:\b)" 0; "~*(?:\b)metabar\.ru(?:\b)" 0;
"~*(?:\b)metafilter\.com(?:\b)" 0;
"~*(?:\b)metallosajding\.ru(?:\b)" 0; "~*(?:\b)metallosajding\.ru(?:\b)" 0;
"~*(?:\b)metalonly\.info(?:\b)" 0; "~*(?:\b)metalonly\.info(?:\b)" 0;
"~*(?:\b)metarip\.ru(?:\b)" 0; "~*(?:\b)metarip\.ru(?:\b)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.18 0;
127.0.0.1 0; 127.0.0.1 0;

11277
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2920 2921

11277
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11277
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

1
dev-tools/referrers-regex-format-whitelist-test.txt
View file

@ -3770,6 +3770,7 @@
"~*(?:\b)message\-warning\.net(?:\b)" 0; "~*(?:\b)message\-warning\.net(?:\b)" 0;
"~*(?:\b)mesto\-x\.com(?:\b)" 0; "~*(?:\b)mesto\-x\.com(?:\b)" 0;
"~*(?:\b)metabar\.ru(?:\b)" 0; "~*(?:\b)metabar\.ru(?:\b)" 0;
"~*(?:\b)metafilter\.com(?:\b)" 0;
"~*(?:\b)metallosajding\.ru(?:\b)" 0; "~*(?:\b)metallosajding\.ru(?:\b)" 0;
"~*(?:\b)metalonly\.info(?:\b)" 0; "~*(?:\b)metalonly\.info(?:\b)" 0;
"~*(?:\b)metarip\.ru(?:\b)" 0; "~*(?:\b)metarip\.ru(?:\b)" 0;

1
dev-tools/referrers-regex-format.txt
View file

@ -3770,6 +3770,7 @@
"~*(?:\b)message\-warning\.net(?:\b)" 1; "~*(?:\b)message\-warning\.net(?:\b)" 1;
"~*(?:\b)mesto\-x\.com(?:\b)" 1; "~*(?:\b)mesto\-x\.com(?:\b)" 1;
"~*(?:\b)metabar\.ru(?:\b)" 1; "~*(?:\b)metabar\.ru(?:\b)" 1;
"~*(?:\b)metafilter\.com(?:\b)" 1;
"~*(?:\b)metallosajding\.ru(?:\b)" 1; "~*(?:\b)metallosajding\.ru(?:\b)" 1;
"~*(?:\b)metalonly\.info(?:\b)" 1; "~*(?:\b)metalonly\.info(?:\b)" 1;
"~*(?:\b)metarip\.ru(?:\b)" 1; "~*(?:\b)metarip\.ru(?:\b)" 1;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.22.48 0; 40.122.225.54 0;

11277
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.18 0;
127.0.0.1 0; 127.0.0.1 0;

11277
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.18 0;
127.0.0.1 0; 127.0.0.1 0;

11277
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.18 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -3817,6 +3817,7 @@
"~*(?:\b)message\-warning\.net(?:\b)" 0; "~*(?:\b)message\-warning\.net(?:\b)" 0;
"~*(?:\b)mesto\-x\.com(?:\b)" 0; "~*(?:\b)mesto\-x\.com(?:\b)" 0;
"~*(?:\b)metabar\.ru(?:\b)" 0; "~*(?:\b)metabar\.ru(?:\b)" 0;
"~*(?:\b)metafilter\.com(?:\b)" 0;
"~*(?:\b)metallosajding\.ru(?:\b)" 0; "~*(?:\b)metallosajding\.ru(?:\b)" 0;
"~*(?:\b)metalonly\.info(?:\b)" 0; "~*(?:\b)metalonly\.info(?:\b)" 0;
"~*(?:\b)metarip\.ru(?:\b)" 0; "~*(?:\b)metarip\.ru(?:\b)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.18 0;
127.0.0.1 0; 127.0.0.1 0;

11277
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.22.48 0; 40.122.225.54 0;

11277
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.22.48 0; 40.122.225.54 0;

11277
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.18 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/custom-bad-referrers.conf
View file

@ -3817,6 +3817,7 @@
"~*(?:\b)message\-warning\.net(?:\b)" 0; "~*(?:\b)message\-warning\.net(?:\b)" 0;
"~*(?:\b)mesto\-x\.com(?:\b)" 0; "~*(?:\b)mesto\-x\.com(?:\b)" 0;
"~*(?:\b)metabar\.ru(?:\b)" 0; "~*(?:\b)metabar\.ru(?:\b)" 0;
"~*(?:\b)metafilter\.com(?:\b)" 0;
"~*(?:\b)metallosajding\.ru(?:\b)" 0; "~*(?:\b)metallosajding\.ru(?:\b)" 0;
"~*(?:\b)metalonly\.info(?:\b)" 0; "~*(?:\b)metalonly\.info(?:\b)" 0;
"~*(?:\b)metarip\.ru(?:\b)" 0; "~*(?:\b)metarip\.ru(?:\b)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AhrefsBot ALittle Client
BlowFish Devil
CensysInspect DomainCrawler
Ebingbong ExtractorPro
QueryN Metasearch Picscout
SemrushBot ScoutJet
SiteSucker UnisterBot
SlySearch Voil
Typhoeus Web Collage
Xaldon_WebSpider clark-crawler

320
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker
404enemy 404enemy
80legs 80legs
ADmantX ALittle Client
AIBOT Aboundex
Aboundexbot Acunetix
Aipbot AfD-Verbotsverfahren
Alexibot
AllSubmitter
AlphaBot AlphaBot
Anarchy
Ankit Ankit
Anthill Apexoo
Atomseobot Asterias
Attach AwarioSmartBot
AwarioRssBot BDFetch
BLEXBot BLEXBot
BackDoorBot BacklinkCrawler
BackStreet BetaBot
BackWeb
Backlink-Ceck
Badass
Bandit
Bigfoot
Bitacle
Black Hole Black Hole
BlackWidow
Blackboard Blackboard
Blow
BlowFish
Brandprotect Brandprotect
Brandwatch Brandwatch
Buck BuzzSumo
CODE87
CSHttp
Calculon Calculon
Cegbfeieh Cegbfeieh
CensysInspect CheeseBot
CheTeam
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Chlooe Claritybot
Collector Cloud mapping
Cocolyzebot
Copier Copier
CopyRightCheck CopyRightCheck
Crawling at Home Project Copyscape
Crescent CrazyWebCrawler
CrunchBot Curious
Custo
CyotekWebCopy CyotekWebCopy
DIIbot DBLBot
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Demon
DigitalPebble Dirbuster
Dispatch Disco
DittoSpyder
DnyzBot DnyzBot
DomainStatsBot DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Download Wonder
Dragonfly
Drip Drip
ECCP/1.0 EMail Siphon
EroCrawler Ebingbong
FHscan Ecxi
FemtosearchBot Extractor
Firefox/7.0 Ezooms
Fimap
FlashGet
Freeuploader
Fuzz Fuzz
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t GalaxyBot
GetWeb GermCrawler
Getintent Gigabot
Go-Ahead-Got-It
GoZilla GoZilla
Gotit Gotit
GridBot
HMView HMView
HTTP::Lite HTMLparser
Haansoft Harvest
HaosouSpider Havij
Hloader Heritrix
Humanlinks Humanlinks
HybridBot
IDBTE4M
IDBot IDBot
IRLbot IRLbot
Iblog Iblog
IlseBot Image Sucker
Image Fetch InfoNaviRobot
Indy Library InfoTekies
Intelliseek
InterGET
Internet Ninja Internet Ninja
Iskanie Iskanie
Jbrofuzz JetCar
Jorgee JikeSpider
JustView
Kenjin Spider Kenjin Spider
Lanshanbot Lanshanbot
Larbin Leap
LeechGet LeechFTP
LexiBot LieBaoFast
LibWeb Lightspeedsystems
Libwhisker Linkbot
LinkWalker LinkextractorPro
LinkpadBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MIDown tool MFC_Tear_Sample
MIIxpc MQQBrowser
MJ12bot
MSIECrawler MSIECrawler
MTRobot MTRobot
Magnet Magnet
Mail.RU_Bot Majestic SEO
Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
Mb2345Browser Masscan
MeanPath Bot Meanpathbot
MegaIndex.ru
Metauri Metauri
MicroMessenger
Microsoft URL Control Microsoft URL Control
Mister PiX Moblie Safari
Morfeus Fucking Scanner Mojeek
Mozlila MolokaiBot
Mr.4x3
Msrabot
Musobot Musobot
Nameprotect Name Intelligence
Navroad Navroad
NearSite NetLyzer
Nessus NetSpider
Net Vampire Netvibes
Netcraft Niki-bot
Nettrack Nikto
NextGenSearchBot NimbleCrawler
Ninja Not
Octopus Nuclei
Offline Navigator Offline Explorer
OnCrawl Openfind
Openvas Openvas
OrangeSpider
OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl PageGrabber
PageAnalyzer PageThing.com
Pandalytics
PeoplePal PeoplePal
Petalbot Petalbot
Picsearch Picscout
PictureFinder
Piepmatz Piepmatz
PleaseCrawl Pockey
ProPowerBot Probethenet
Pu_iN
Pump Pump
PxBroker PxBroker
Quick-Crawler PyCurl
Rankivabot QueryN Metasearch
RSSingBot
RankFlex
RankingBot
RankurBot RankurBot
ReGet Reaper
RealDownload
RebelMouse RebelMouse
SEOkicks RedesScrapy
SEOkicks-Robot RocketCrawler
SEOlyticsCrawler Rogerbot
SEOprofiler SEOstats
SalesIntelligent
ScoutJet
Screaming Screaming
ScrepyBot Seekport
Searchestate
SemanticJuice SemanticJuice
SentiBot SeoSiteCheckup
SeobilityBot SeobilityBot
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteExplorer
SiteLockSpider SiteSnagger
Siteimprove Sitebeam
SmartDownload SmartDownload
Sociscraper SocialRankIOBot
Sottopop Sogou web spider
Spbot Sosospider
SputnikBot SpaceBison
Spammen
Sqlmap Sqlmap
Sqlworm Stripper
Steeler
Sucuri
SuperBot
SuperHTTP SuperHTTP
Surfbot SurveyBot
Szukacz
TeleportPro
Telesoft
Telesphoreo Telesphoreo
The Intraformant
Thumbor Thumbor
Toweyabot Turnitin
Trendictionbot TwengaBot
TurnitinBot URLy.Warning
Typhoeus
UnisterBot UnisterBot
Upflow V-BOT
VB Project
Vacuum
Vagabondo
VidibleScraper
Virusdie
VoidEYE VoidEYE
WBSearchBot Voil
WEBDAV Voltron
WPScan WPScan
WallpapersHD WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
Wallpapers
WeSEE
Web Auto Web Auto
Web Collage Web Collage
Web Fuck Web Fuck
Web Sauger Web Sauger
WebAuto WebBandit
WebCollage WebCollage
WebEnhancer
WebFetch WebFetch
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebReaper WebReaper
WebSauger WebSauger
WebWhacker WebSucker
WebZIP Webalta
WebmasterWorldForumBot
Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whacker
Whatweb
Who.is Bot
Widow
WiseGuys Robot WiseGuys Robot
Xaldon_WebSpider YoudaoBot
Zauba Zermelo
Zeus
ZmEu ZmEu
ZyBorg ZumBot
autoemailspider arquivo-web-crawler
check1.exe arquivo.pt
clark-crawler cah.io.community
dataforseo.com cognitiveseo
evc-batch com.plumanalytics
internetVista monitor crawler.feedback
ips-agent crawler4j
iubenda-radar demandbase-bot
lwp-request eCatch
facebookscraper
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan
netEstate NE Crawler
oBot
page scorer page scorer
pcBrowser pcBrowser
satoristudio.net ripz
scalaj-http
seocompany.store seocompany.store
seoscanners
seostar
sexsearcher sexsearcher
siteripz sitechecker.pro
sogouspider sp_auditbot
spyfu
sysscan sysscan
tAkeOut
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com webpros.com
x22Mozilla
xpymep1.exe

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker 404checker
80legs ASPSeek
ADmantX Aboundex
AIBOT
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren
AiHitBot
Aipbot
Alexibot
AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Anarchy Anarchy
Anarchy99
Anthill Anthill
Apexoo Aspiegel
Atomseobot
AwarioRssBot
BLEXBot
BackStreet BackStreet
BacklinkCrawler Backlink-Ceck
Bandit BetaBot
Barkrowler Bitacle
BatchFTP Black Hole
Battleztar Bazinga Blackboard
BlowFish
Bolt Bolt
BotALot
Brandprotect Brandprotect
Brandwatch Brandwatch
Buddy
BuiltBotTough
BunnySlippers
CCBot CCBot
CSHttp
CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
CheTeam CheTeam
ChinaClaw CherryPicker
Chlooe Chlooe
Claritybot
Cogentbot Cogentbot
CrunchBot Collector
CopyRightCheck
Cosmos
CyotekWebCopy CyotekWebCopy
DSearch DIIbot
DTS Agent DTS Agent
DatabaseDriverMysqli DataCha0s
Deusu Demon
Devil
Digincore Digincore
DigitalPebble
Discoverybot Discoverybot
Dispatch Dispatch
DittoSpyder DnBCrawler-Analytics
DomainAppender DnyzBot
DomCopBot
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Download Wonder Download Wonder
Dragonfly Dragonfly
ECCP/1.0 Drip
EMail Wolf
EasyDL
Ebingbong Ebingbong
Ecxi Ecxi
EroCrawler EroCrawler
Evil Evil
Exabot ExtractorPro
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
EyeNetIE FDM
FemtosearchBot
Fimap
Firefox/7.0 Firefox/7.0
Flunky Freeuploader
Fuzz FrontPage
GrapeFX FyberSpider
GridBot Genieo
Gigabot
Go-Ahead-Got-It
GrabNet
Grafula
HEADMasterSEO
HMView HMView
HTMLparser HTTrack
Haansoft
Harvest Harvest
HonoluluBot IDBot
Image Fetch IRLbot
Id-search
IlseBot
Image Sucker Image Sucker
InterGET IndeedBot
Indy Library
Internet Ninja
Iria Iria
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JennyBot
Jetty
JikeSpider JikeSpider
Joomla Jyxobot
JustView Keyword Density
Kenjin Spider Lanshanbot
Kozmosbot Leap
LeechGet LeechFTP
LexiBot
Lftp
Libwhisker Libwhisker
Likse LinkWalker
LinkScan
Linkbot
Linkdexbot Linkdexbot
LinkextractorPro LinkextractorPro
LinksManager LinkpadBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
MFC_Tear_Sample Ltx71
MIDown tool MIDown tool
MJ12bot MJ12bot
MQQBrowser MSIECrawler
MTRobot MTRobot
Magnet Magnet
Mail.RU_Bot
Majestic12 Majestic12
Mass Downloader
MauiBot
Mb2345Browser
Metauri Metauri
MicroMessenger MicroMessenger
Microsoft Data Access Microsoft URL Control
Minefield Mojeek
Mister PiX Msrabot
Moblie Safari NICErsPRO
Mojolicious
Mr.4x3
Musobot
NPbot
Name Intelligence Name Intelligence
Nessus NetMechanic
NetSpider Nibbler
Niki-bot Niki-bot
NimbleCrawler
Nimbostratus Nimbostratus
Nuclei
Nutch Nutch
Octopus Octopus
Offline Explorer
Offline Navigator
OpenLinkProfiler
OpenVAS
Openfind Openfind
OutclicksBot OrangeBot
OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP Page Analyzer
POE-Component-Client-HTTP
PageGrabber
Pandalytics
Pavuk Pavuk
PeoplePal Petalbot
Picsearch Pi-Monster
PictureFinder Pockey
Piepmatz Probethenet
Pixray Pump
ProPowerBot PxBroker
ProWebWalker
PyCurl PyCurl
Quick-Crawler QueryN Metasearch
RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot ReGet
RankurBot Reaper
Re-re RebelMouse
Recorder SBIder
RepoMonkey SEOkicks-Robot
RocketCrawler SEOlyticsCrawler
SEOprofiler SISTRIX
ScoutJet SMTBot
Scrapy ScanAlert
Searchestate SearchmetricsBot
SemanticJuice
SentiBot SentiBot
SeoSiteCheckup SeobilityBot
Seomoz
Siphon
Site Sucker Site Sucker
Siteimprove SiteCheckerBotCrawler
SiteSnagger
Sitevigil Sitevigil
SlySearch SlySearch
SmartDownload SmartDownload
Sociscraper Snoopy
Sosospider SocialRankIOBot
Sogou web spider
Sottopop Sottopop
SpaceBison SpaceBison
Spanner
Spbot Spbot
Spinn3r Spinn3r
Sqlworm
Steeler Steeler
SuperHTTP Stripper
Surfbot Sucker
SuperBot
Suzuran Suzuran
T0PHackTeam T0PHackTeam
TeleportPro T8Abot
Telesoft Teleport
Telesphoreo The Intraformant
TheNomad TheNomad
Titan TightTwatBot
Trendiction Trendiction
True_Robot
Turingos
Twice
Typhoeus Typhoeus
URLy Warning
VB Project
Vacuum Vacuum
VelenPublicWebCrawler VidibleScraper
VeriCiteCrawler Voil
WBSearchBot WBSearchBot
WEBDAV WISENutbot
WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 WeSEE
WallpapersHD Web Collage
Web Auto Web Fuck
Web Fetch Web Pix
Web Sauger WebCopier
WebBandit WebFetch
WebEnhancer WebImageCollector
WebGo IS WebLeacher
WebPix WebPix
WebWhacker WebSauger
WebStripper
WebZIP WebZIP
Webshag Webshag
Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster
Whack Whack
Whacker Whacker
Whatweb Who.is Bot
Widow Widow
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot Woobot
Wotbox Wprecon
YoudaoBot Xenu
Zauba
Zitebot Zitebot
ZoomBot ZumBot
ZyBorg
adscanner adscanner
arquivo.pt check1.exe
clark-crawler com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
dataforseo.com crawler4j
demandbase-bot facebookscraper
domainsproject.org instabid
heritrix
internetVista monitor internetVista monitor
iubenda-radar
lwp-request
magpie-crawler magpie-crawler
meanpathbot
page scorer
pcBrowser pcBrowser
plumanalytics
polaris version
probe-image-size
ripz ripz
s1z.ru
seocompany.store seocompany.store
seostar
serpstatbot serpstatbot
sitechecker.pro sitechecker.pro
siteripz trendiction.com
sp_auditbot trendiction.de
sysscan
tAkeOut
ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com webpros.com
x09Mozilla
xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
8lufu.com architecturebest.com
dispo.de darkages.info
enthuse.computernetworksonline.com e-c.al
failingmarriege.blogspot.com earn-from-articles.com
highspeed5.net fm-upgrade.ru
konturkrasoty.ru fortwosmartcar.pw
rutor.group luckyday.world
sevendays.com.ua mafa-free.com
truck-addzilla.life rfd-split.hr
xchat26.myfreecams.com works.if.ua

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
08800.top 000free.us
123cha.com
178evakuator178.ru
1flag.co.za 1flag.co.za
1hwy.com 1millionusd.xyz
3th.co.in 20pascals.nl
40cg.com 28n2gl3wfyb0.ru
4ureyesonly.com 2kata.ru
5000-cotydzien.com 300richmond.co.nz
5ws.dating-app.ru 3rbseyes.com
7minuteworkout.com 4istoshop.com
a.frcls.fr 55wmz.ru
8si.ru
a96527gi.beget.tech a96527gi.beget.tech
aa8b68101d388c446389283820863176e7.com absugars.com
adidas.frwebs.fr acmebtn.ml
adssafeprotected.com adclickservice.com
adultfriendfinder.com adktrailmap.com
advancedcleaner.com admitad.com
affrh2015.com adobereader-free.ru
ahmedabadwebs.com ads-seo.men
allfinweb.com adserver-e7.com
allwomen.info adtech.fr
am-se.com adult-shop.com.ua
am15.net advertisingtag.net
advocatemsk.ru
affiliate-fr.com
afmuseum.com
agapovdg.ru
airmaxshoes-2016.com
akita.kz
akvamaster.dp.ua
alarmobninsk.ru
alibestsale.com
allforminecraft.ru
allkrim.com
allmarketsnewdayli.gdn
allwidewallpapers.com
alpharma.net
alpinism.ru
amateurlivechat.org
ameblo.top
amehdaily.com
amoi.tn
analnoeporno.tv analnoeporno.tv
andpolice.com animalrank.com
apiluckyleapnet-a.akamaihd.net animaltoplist.com
apimegabrowsebiz-a.akamaihd.net anime.dougasouko.com
app-ready.xyz antons-transporte.de
apple.com-webbrowsing-security.review apibatbrowsecom-a.akamaihd.net
appsaurus.com apiholdingmypage-a.akamaihd.net
appsecurityr.com apiitsthirteende-a.akamaihd.net
apxeo.info apioasisspacenet-a.akamaihd.net
asrv-a.akamoihd.net apiwebwebgetcom-a.akamaihd.net
astana.xxxkaz.org apparel-offer.com
astrochicks.com appiq.mobi
ausergrubhof.info aquarium-pro.ru
auspolice.com arcadepages.com
autoloans.com arenda-yeisk.ru
avek.ru arvut.org
bamo.xsl.pt asmxsatadriverin.aircus.com
banksrf.ru atleticpharm.org
beauty-b0x.pl auto-zapchasti.org
berrymall.ru autobudpostach.club
bin-brokers.com autodo.info
binaryoptionscops.info autotop.com.ua
avon-novosib.ru
avtovolop.ru
azartmix.com
azbukafree.com
backlink4u.net
balkanfarma.ru
barbourjackets.online
basisches-wasser.net
batanga.net
bausparen.bz.it
beztuberkuleza.ru
billiard-classic.com.ua
bimmerpost.com
bio-market.kz
bitcoinpile.com
blacktwhite.com blacktwhite.com
blockety.co bloggen.be
bluesman.nu bloke.com
boanonihaca.tk bluejays-jerseys.us
brainboosting.club bolegapakistan.com
bravegirlsclub.com bonkers.name
bristolhotel.com bonzybuddy.com
bronzeaid-a.akamaihd.net bosefux.esy.es
braindaily.xyz
brains2.biz
braintobe.win
brakehawk.com
brandbucket.com
browserprotecter.com
btc4u.ru btc4u.ru
buketeg.xyz butstrap.space
buy-loft.ru bzero1jewelry.net
buypanicdisorderpill.com cactussoft.biz
bvlgariwallet.xyz cakesplus.com.au
camdenmemorials.com calcularpagerank.com
candelluxsklep.pl candy-glam-hp.com
capture-room.com carsdined.org
cartierlove2u.xyz cartierbracelet.xyz
cartiertrinity.xyz cashkitten-a.akamaihd.net
cayado.snn.gr catalogs-parts.com
cbb1smartlist12.click chastnoeporno.com
cementaresearch.se cheapbelstaff.online
centraletermice.us cheapsergiorossi.online
centrdebut.ru chlooe.com
cheapmarmot.online chocolateslim-u-srbiji.com
chocolateslim-en-espana.com choosecuisine.com
christianlouboutinsaleoutletonline.us chuckguilford.com
ci.ua cleanallvirus.com
cjmc.info clevernt.com
clarithromycin500mg.com clickhype.com
clipartnew.com clickintext.net
clips.ua.ac.be codq.info
cntravelre.com coinsspb.com
compliance-alex.top compliance-brian.xyz
compliance-alexa.xyz compliance-fred.top
compliance-checker-7.info conboy.us
concealthyself.com concordexoticrentals.com
conciergegroup.org constantaservice.net
construmac.com.mx coslab.club
coolyarddecorations.com
cordstrap.cc
counciltally.com counciltally.com
counter.yadro.ru covid-schutzmasken.de
cranly.net customchocolate.business-for-home.com
creditwell.ru customergrowthsystems.com
cwetochki.ru cyse.tk
dalavia.ru
danceuniverse.ru
dangphoto.trade
daooda.com
darkages.info darkages.info
datadepths.com
datsun-do.com datsun-do.com
dayibiao.com dengi-pod-zalog-nedvizhimosti.ru
ddospanel.com
degerlund.net
deniven.1bb.ru deniven.1bb.ru
dfwu1013.info deshevo-nedorogo.ru
diesel-parts28.ru discountliv.com
digital-video-processing.com divatraffic.com
discountbarbour.online dolohen.com
diusyawiga.tk
dms-sw.ru
dogoimage.com
domain2008.com domain2008.com
doreenblog.online downloads-whatsapp.com
dostavka-v-krym.com dptaughtme.com
dosug-lux.ru drhomes.biz
downvids.net drugspowerstore.com
dpihatinh.gov.vn dtnlyss.com
dprtb.com e-commerce-seo1.com
drugstoreforyou.com earl-brown.info
ecommerce-seo.com edudocs.net
ecookna.com.ua ekspertmed.com
eets.net el-nation.com
elektronischezigarettekaufen1.beeplog.com elektrischezigarette1.blog.pl
elektrozigaretten1.webs.com elektronischezigarettekaufen1.myblog.de
empirepoker.com eluxer.net
en.altezza.travel embedle.com
enginebay.ru enginebay.ru
enhand.se essayassist.com
enskedesquashclub.se europages.com.ru
eonpal.com evrotekhservis.ru
eropho.com executehosting.com
es5.com extener.org
eu-cookie-law.blogspot.com extractorandburner.com
evehemming.blogspot.com.au ezigarettenkaufen1.deviantart.com
evidencecleanergold.com fashion-mk.net
exhibitionplus.eu ferretsoft.com
exmasters.com fettefrauen.net
extlabs.io fickblock18.com
ezigarettekaufen.myblog.de
ezigarettekaufen2.mpbloggar.se
face.hostingx.eu
facecup.top
falllow.gq
fashion.stellaconstance.co
fastcrawl.com
femmesdenudees.com
fickenbumsen.net
fickluder69.com
filerockstar298.weebly.com
filesdatabase.net
financepoints.eu financepoints.eu
firstdrugmall.ru fitness-video.net
firstsiteguide.com fm-upgrade.ru
flooringinstallation-edmonton.com forensicpsychiatry.ru
floridahuntingfishingadventures.com formseo.com
flowersbazar.com forum69.info
fok.nl forumprofi.de
forex21.ru free-deals.faith
forexunion.net free-video-tool.com
fortunejack.com freenom.link
fotoxxxru.com freesitetest.com
free-share-buttons.com frvo.alptandem.ru
freelotto.com gearcraft.us
ftns.ru gearsadspromo.club
fun-mobi.pl gidonline.one
funcrushgames.com gifspics.com
fungamelands.com glall.ru
fz139.ttk.ru global-ics.co.za
galeon.com gobongo.info
gamevalue7.weebly.com golaya.pw
getmyads24.com gombita.info
getyourimage.club gpms.org.my
globalscam.ga gratuitbaise.com
glopages.ru gsasearchengineranker.space
gogalleryawesome.com hacktougroup.ru
goroda-vsego-mira.ru haikuware.com
gq-catalog.gq handicapbathtubarea.com
graphicwe.org hermesbirkinhandbagoutlets.com
graphid.com hobbyhuren-datenbank.com
gratuitxblcodes.com host-tracker.com
greamimgo.pw hostingclub.lk
gribokstop.com houston-vikings.com
gsasearchengineranker.xyz hoverboard360.nl
guildebzh.info hoverboard360.se
havepussy.com hreade.com
hayate.biz htmlcorner.com
hd720kino.ru igithab.com
hdpixion.com imediadesk.com
health-medical-portal.info imguramx.pw
himgaws.pw imitex-plus.ru
homik.pw incest-ru.com
horoshieokna.com investsuccess.org
hotkeys.com invitefashion.com
housediz.com
hoverboard360.at
hoverboardforsaledirect.com
howtoclean.club
hvd-store.com
i-midias.net.br
ico.re
idegenvezeto.eu
ifirestarter.ru
ilikevitaly.com
ilovevitaly.ru
imagefinder.site
inboxdollars.com
incolors.club
indianmedicaltourismshop.com
infodocsportal.com
intimshop-fantasy.ru
investpamm.ru
invivo.hu
ip-guide.com ip-guide.com
iplogger.org iqoption.pro
irunfar.com itservicesthatworkforyou.com
istanbulit.com
istizanidineanarcoticdrug.blogspot.com
istizanidineanopiate.blogspot.com
istock-mebel.ru
jacago.com jacago.com
jackpotchances.com jo24news.com
japan-bearings.ru jrcigars.com
joy-penguin.com justprofit.xyz
jpcycles.com kaac.ru
justucalling32211123456789.tk ketrzyn.pl
kamagragelusa.net kfon.eu
kaz.kz kino-ecran.ru
keyhantercume.com kinohall.ru
keywordbasket.com
keywordsdoctor.com
keywordsuggest.org
kicknights.gq
kino-rating.ru
kinozapas.com
kit-opt.ru kit-opt.ru
komputernaya-pomosh-moscow.ru koptims.tiu.ru
konkursowo-24.pl krasivye-devushki.net
kostenloser-sex.com kunstaktien.info
kraljeva-sutjeska.com lalalove.ru
kredit-blog.ru landmania.ru
kursy-ege.ru lawyersinfo.org
kustanay.kz le-clos-des-alouettes.com
kuzinsp.ru lenvred.org
kw21.org letslowbefast.today
laapp.com lightinghomes.net
lacapilla.info linkbuddies.com
lacloop.info listiseltemournlan.gq
ladov.ru livefixer.com
law-check-eight.xyz livetsomudvekslingstudent.bloggersdelight.dk
law-check-nine.xyz livingcanarias.com
lecbter-relationships.ga locatellicorretor.com.br
lemon-ade.site lonerangergames.com
lezbiyanki.net longgreen.info
live-cam6.info macdamaged.tech
liveshoppersmac.com make-money-online.com
localmatchbook.com makemoneyonline.com
lombia.com mansiondelrio.co
longadventure.com maridan.com.ua
lowephotos.info marmotstore.online
ltvperf.com mediaoffers.click
lutherstable.org mediawhirl.net
macnewtech.com medkletki.ru
macotool.com meetlocalchicks.com
macrotek.ru menetie.ru
mafcards.ru mensandals.xyz
magda-gadalka.ru mesto-x.com
magicdiet.gq mexicotravelnet.com
maju.bluesalt.co militarysale.pro
malls.com mirtorrent.net
marcowebonyodziezowe.pl mobicover.com.ua
matpre.top moetomnenie.com
matsdale.com moncleroutletonline.win
medtherapy.ru
mega-bony-2017.pl
megavolt.net.ua
meget.co.za
menhealed.net
michaelkorsoutlet.store
minecraft-rus.org
minegam.com
mjchamonix.org
mlf.hordo.win
mmog-play.ru
mobile-appster.ru
moneyviking-a.akamaihd.net
morocco-nomad-excursions.com
mosdverka.ru mosdverka.ru
moviemail-online.co.uk mp3s.club
moviezbonkerssk.cf mug-na-chas-moscow.ru
mowser.com muz-baza.net
mozello.ru my-cash-bot.co
mrsdalloways.com myliveblog.ru
mylida.org mymobilemoneypages.com
myyour.eu nagdak.ru
narkologiya-penza.ru najaden.no
narkologiya-voronezh.ru nalogovyykodeks.ru
national-today-winning-winner.club netallergy.ru
naturalbreakthroughsresearch.com networkcheck.xyz
naval.jislaaik.com next-dentists.tk
navalwiki.info nfvsz.com
negral.pluto.ro notfastfood.ru
neodownload.webcam nrjmobile.fr
newhairstylesformen2014.com nudepatch.net
nlfjjunb5.ru o-dachnik.ru
noclegonline.info o.light.d0t.ru
noscrapleftbehind.co obiavo.uz
novatech.vn odoratus.net
novgorod.xrus.org offer.wpsecurity.website
nrv.co.za oklogistic.ru
nuit-artisanale.com okmedia.sk
oda.as okuos.com
office-windows.ru olympescort.com
onclkads.com
oneclickfiles.com
onemactrckr.com
oneminutesite.it oneminutesite.it
online7777.com online-hd.pl
onlinedomains.ru online-zaymy.ru
oren-cats.ru onlinemeetingnow.com
ourtherapy.ru onlineslotmaschine.com
outpersonals.com onlyforemont.ru
ownshop.win ooomeru.ru
openmultipleurl.com
optom-deshevo.ru
orgasmatrix.com
ostroike.org
outshop.ru
oxotl.com oxotl.com
panicatack.com pammik.ru
park.above.com paradontozanet.ru
penisvergrotendepillennl.ovh partner-pop.men
peugeot-club.org pateaswing.com
phormchina.com pc-virus-d0l92j2.pw
photo-clip.ru pcgroup.com.uy
piccdata.com pearlisland.ru
picswe.com pensplan4u.com
pinwallpaper.top pestomou.info
pirateday.ru petsblogroll.com
piter.xrus.org picturesboss.com
pizda.lol picturesify.com
pk-pomosch.ru picurams.pw
pladform.ru pizza-tycoon.com
popup-hgd.xyz play.leadzupc.com
popup-jdh.xyz pochemychka.net
porn555.com poffet.net
porno-raskazy.ru polytopesexempt.com
pornoslive.net popmarker.com
pornosok.ru poptool.net
pornpost.in pornoforadult.com
pozdrawleniya.ru pornotubs.com
pregnant.guru pozdrawleniya.com
preparevideosafesystem4unow.site pozvonim.com
princeadvantagesales.com privat-girl.net
printingpeach.com professionaldieselcare.com
privhosting.com
prizeestates.cricket
pro-tec.kz
produkto.net
profitwithalex.info profitwithalex.info
pronorm.fr promgirldresses.xyz
prosmibank.ru pts163.ru
prostitutki-almata.org r-control.ru
prostitutki-belgoroda.org radiogambling.com
proxyradar.com randalljhoward.com
pst2017.onlinewebshop.net realmonte.net
pyrodesigns.com.au recinziireale.com
qcstrtvt.bloger.index.hr reckonstat.info
qitt.ru registrydomainservices.com
questionmarque.ch remmling.de
qwesa.ru remontbiz.ru
rainbowice.ru remote-dba.de
rangjued.com rome2rio.com
razorweb-a.akamaihd.net rrutw.com
realitykings.com rumamba.com
redirectme.net rutor.group
renecaovilla.online salut-camp.ru
rfid-locker.co salutmontreal.com
rocketchange.ru sawin.beth.webd.pl
roll123.com
royalads.net
ruscoininvest.company
rustic-quiver.win
rvi.biz
rvtv.ru
sale-japan.com
sarm.tk
satellite.maps.ilovevitaly.com
saugatuck.com
sax-sex.com sax-sex.com
scanner-josh.top scanner-irvin.top
scanner-julianna.top scanner-marwin.top
seansonline24.pl scanner-viktor.top
search.pch.com scenicmissouri.us
seinterface.com school-diplomat.ru
selectads.men scopich.com
seo-smm.kz screentoolkit.com
seo-traffic-ranking.info sdsjweb.com
sex-tracker.com search-error.com
sexobzor.info secretscook.ru
sexreliz.com sevendays.com.ua
sexvporno.ru sexpornotales.net
sherlock.se sexy-screen-savers.com
shiksabd.com sfj-ror.no
shop-electron.ru sharpchallenge.com
shopperifymac.com shoplvlv.us
sispe.com.br sigmund-freud.co.uk
sim-service.net
skachat-besplatno-obrazcy.ru
sky-mine.ru
smartshoppymac.com
smstraf.ru
soblaznu.net soblaznu.net
social-s-iii.xyz social-s-iii.xyz
soft-terminal.ru socialmonkee.com
soserfis.com socialsignals24.com
space4updating.win sockshares.tv
sonata-arctica.wz.cz
spbchampionat.ru spbchampionat.ru
spinnerco.ca spinazdrav.ru
stathat.com spyfu.com
steame.ru spywarespy.com
sterva.cc stardevine.com
stevemonsen.com start.myplaycity.com
suchenindeutschland.com starwars.wikia.com
sucsesofinspiration.com stats-public.grammarly.io
superlist.biz stillmiracle.com
sureone.pro stmassage.ru
svarkagid.com stoki.ru
symbaloo.com stop-zavisimost.com
syndicate.fun streha-metalko.si
tabakur77.com stroilka.info
tacbelarus.ru success-seo.com
takeprofitsystem.com sukirgenk.dvrlists.com
tamada69.com superinterstitial.com
taranerymagesswa.blogspot.com superstats.com
telemetryverification.net sweepstakes.rewardit.com
textads.men swingerseiten.com
tgtclick.com t-bygg.com
thaismartloan.com tako3.com
thebitcoincode.com tampabaywatch.org
thedownloadfreeonlinegames.blogspot.com tattomedia.com
thegamerznetwork.com td-33.ru
theheroes.ru teastory.co
thelottosecrets.com telegraf.by
thetattoohut.com theautoprofit.ml
timetorelax.biz thedownloadfromwarez.blogspot.com
tizanidine4mgprice.blogspot.com thefotosgratis.eu
tizanidinedosageforsleep.blogspot.com thejournal.ru
tizanidinedruginteractions.blogspot.com thfox.com
tizanidinemusclerelaxant.blogspot.com thin.me.pn
tizanidinesideeffectsweightloss.blogspot.com tilido.com
tongkatmadura.info titelhelden.eu
top10-online-games.com tizanidineandgabapentin.blogspot.com
tizanidineforopiatewithdrawal.blogspot.com
tjkckpytpnje.com
tomck.com
topmira.com
topsiteminecraft.com topsiteminecraft.com
topsy.com tovaroboom.vast.ru
touchmods.fr track2.shop
tracklead.net trafficshaper.com
tubeoffline.com trafficstars.com
tupper-posuda.ru traffpartners.com
turbodsp.com trainoffend.ml
trichizobswiv.agddns.net
truemfilelj.gq
turist-strani.ru
turkeyreport.tk turkeyreport.tk
twincitiescarservice.com tx41tclega.ru
uni.me tzritel.tk
uptime.com udsgame.online
ustion.ru uhod-za-sobakoj.ru
v720hd.ru ukkelberg.no
valueclick.com uroffer.link
vapomnoncri.tk urzedowski.eu
varikoz24.com vadimkravtcov.ru
vbtracker.net velobikestock.com
vepad.com viagroid.ru
via-energy-acquistare.com video--production.com
viagengrarx.com video-chat.in
videochat.cafe videochat.world
videooko.weebly.com videogamesecrets.com
videos-for-your-business.com vinsit.ru
vipms.ru vipms.ru
virus-schutzmasken.de vkgaleria.com
viveresaniesnelli.it vkontarkte.com
vksaver-all.ru
voluumtracker1.com
voluumtrk.com voluumtrk.com
vpnmouse.com voprosotvet24.ru
vuclip.com vtc.pw
w7s.ru vzlomtw.com
wallpapersdesk.info wahicbefa31.soup.io
watsonrealtycorp.com wallpapers-best.com
wapsite.me
waycash.net waycash.net
wdrake.com waysbetter.cn
websitevaluebot.com webads.co.nz
wejdz-tu.pl webenlace.com.ar
westermarkanjou.se weblibrary.win
whatsupinfoley.com weblo.com
webmonetizer.net
weboptimizes.com
website-analytics.online
websiteexplorer.info
whereverdesperate.gq whereverdesperate.gq
while.cheapwebsitehoster.com wieseversa.no
windowssearch-exp.com wjgony.com
winterclassichockeyjerseys.com woweb.com.ua
www.pinnacle-bets.com writingservices17.blogspot.ru
wyniki-lista.pl wurr.voila.net
x-lime.com xblog.in
x-musics.com xlolitka.com
xerox-douglas.cf xn----7sbahjd3btneuw1joc.xn--p1ai
xmlinde.com
xn----itbeirbjbi7bc6bh2d.xn--p1ai xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn--80aanaardaperhcem4a6i.com xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xvideospornoru.com xn--b1addnj3cah.xn--p1ai
yebocasino.com xn--b1ag5cfn.xn--p1ai
yoursearch.me xnxx-n.com
yourserverisdown.com yellocloud.be
youtube-downloader.savetubevideo.com you-shall-not-pass.is74.ru
zambini.ru youporn-ru.com
zaobao.com.sg youtubedownload.org
zarabotat-na-sajte.ru ypmuseum.ru
zarajbuilders.com yubikk.info
zarepta.com yugo-star.ru
zaidia.xhost.ro
zakazfutbolki.com
zakazvzloma.com
zebradudka.com
zeroredirect1.com
zeroredirect11.com zeroredirect11.com
zeroredirect8.com zetmaster.ru
zmoda.hostreo.com zlatnajesen.com
znakomstva-piter78.ru znakomstva-moskva77.ru
zocaparj.kz zocaparj.kz
zoogdiesney.com zootoplist.com
zytpirwai.net zzbroya.com.ua

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1000n1.ru
123cha.com 123cha.com
1adult.com 1314dh.com
4inn.ru 13tabs.com
51unlim.ru 1hwy.com
5forex.ru 24videos.tv
acunetix-referrer.com 2drittel.de
adktrailmap.com 34.gs
allcredits.su 3dracergames.com
alldezservice.kz 98oi.ru
allpdfmags.net activecampaign.dreamhosters.com
amateurgalls.com adventureparkcostarica.com
ameblo.jp aero2.ru
allwomen.info
amospalla.es amospalla.es
anal-acrobats.hol.es apelsinnik.website
apidigidocketnet-a.akamaihd.net apiallgeniusinfo-a.akamaihd.net
appiq.mobi apisecretsaucebi-a.akamaihd.net
apps-analytics.net artdestshop.eu
azlex.uz arvut.org
badavit.com.ua atlant-auto.info
baikaleminer.com ausergrubhof.info
bang-hotties.com avon-ukraine.com
banksrf.ru azartmix.com
beachfix.co bashtime.ru
benchmarkcommunications.co.uk beclean-nn.ru
bestbuy.ca bestsexyblog.com
bestprofits-there.com biographiya.com
bif-ru.info birzha-truda.eu
blackbabesporn.com blobar.org
blog.koreadaily.com bobinoz.com
bmwhighperformers.com bombla.org
boleznikogi.com brandehk.dk
brainboostingsupplements.org brateg.xyz
braindaily.xyz buckspar.gq
breget74.ru buy-cheap-online.info
brianhenry.co call-of-duty.info
bronzeaid-a.akamaihd.net candytiner.com
buyessay3.blogspot.ru carstrends2015.com
bvlgariring.xyz cartierjusteunclou.xyz
callcafe.info celejihad.info
camsex.xxx-cam.webcam cfjump.com
centrumcoachingu.com chatroulette.online
chee-by.biz cheapjerseysbizwholesale.us
compliance-irvin.top cheapwebsitehoster.com
compliance-josh.top christianlouboutinreplica.win
cookie-law-enforcement-ee.xyz christianlouboutinsaleonline.us
cowlmash.com cindymatches.com
customchocolate.business-for-home.com clayvasedesigns.tk
daydream-studio.ru clickaider.com
dinosaurus.site compliance-andrew.top
domainsatcost.com compliance-jess.top
domoysshop.ru covid-schutzmasken.de
dvd-famille.com cowblog.fr
dvr.biz.ua credit-online.ws
e-collantes.com culad.com
ecxtracking.com danschawbel.com
ekspresihati.info dellalimov.com
elektrozigaretten1.tumblr.com design-ideas.info
diy-handmade-ideas.com
dojki365.online
dolohen.com
domain-tracker.com
domaincheck.io
doska-vsem.ru
dosugrostov.site
downtuptv.gq
drivotracker.com
dyshagi.ru
e2click.com
education-cz.ru
elektrozigaretten2.yn.lt
elitedollars.com elitedollars.com
eluxer.net eorogo.top
enskedesquashclub.se eropho.com
enthuse.computernetworksonline.com estibot.com
erectile.bid eu-cookie-law.blogspot.com
extlinks.com ezigaretteshop2.vefblog.net
extremal-blog.com falopicm.pw
ezb.elvenmachine.com fanoboi.com
facebook-mobile.xyz findsexguide.com
farmingworm.com footballfarrago.com
financetip.eu forensicpsychiatry.ru
findthe.pet forpackningsutveckling.se
forex21.ru forums.d2jsp.org
formseo.com free-share-buttons.top
forodvd.com froggytube.com
forzeronly.com ftns.ru
freecamdollars.com gamedayhouse.com
freenode.info gateway.zscalertwo.net
gallerily.com ge0ip.com
gamerscorps.com getaclueamerica.com
gaypornmovie.net girlspicsa.com
getmiro.com glcomputers.ru
gettpromos.com globatur.ru
gg-arena.ru goatse.ru
ggiaro.com golden-praga.ru
glall.ru goodhousekeeping.com
glavtral.ru gq-catalog.gq
gonextmedia.com gsou.cf
google-liar.ru gtopstats.com
gotomontenegro.net hamptonoaks.ca
greenshop.su hasfun.com
gymi.name highstairs-a.akamaihd.net
healmytrauma.info hotblog.top
hentaiheroes.com hspline.com
hexpilot.com ilo134uloh.com
horoshieokna.com imagecoolpub.com
hully.altervista.org insider.pro
ie.57883.net int.search.mywebsearch.com
img.wallpaperstock.net interior-stickers.ru
impresagaia.it investzalog.ru
incep.imagine-ex.co iusstf.org
infazavr.ru iwantedmoney.com
intellego.info javitas.info
ipornox.xxx javstock.com
iqupdatetmz.win katadhin.co
isotoner.com keywordbasket.com
izhevsk.xrus.org kinky-fetishes.com
jobgirl24.ru kinostorm.net
kaliningrad.zrus.org kletkimehan.ru
karachev-city.ru knogg.net
keyhantercume.com konteiner24.com
keywordblocks.com konturkrasoty.ru
kfon.eu koronirealestate.gr
kinogolos.ru kredit-pod-zalog-krasnodar.ru
kinozapas.org lanasshop.ru
knowyournextmove.com lonely-mature.com
kopihijau.info luckyshop.net.ua
kskjco.club macrotek.ru
lexaprogeneric.link magazin-pics.ru
lineshops.biz maladot.com
liquimondo.com maranbrinfo.com.br
lkbennettoutlet.online marblestyle.ru
longlifelomilomi.info masterseek.com
lutherstable.org mazda-roadsters.com
machicon-ueno.info megahdporno.net
magiadinamibia.blogspot.com megavolt.net.ua
magnetic-bracelets.ru mezaruk.info
magnetpress.sk mister-shop.com
mainhp.net mobile-appster.ru
marsgatan.com moinozhki.com
masserect.com moje-recenze.cz
matsdale.com moomi-daeri.com
mature.free-websites.com multgo.ru
max-eclat.men nfljerseys.online
medanestesia.ru nina.az
meds-online24.com ningessaybe.me
metalonly.info nlfjjunb5.ru
morefastermac.trade nmrk.ru
mosdverka.ru nullrefer.com
moskva.nodup.ru nvssf.com
movies-in-theaters.net okeinfo.online
msk-diplomat.com oksrv.com
music.utrolive.ru online-podarki.com
music7s.me online-x.ru
muznachas-service.ru onlinetvseries.me
mzdish.site openmediasoft.com
nagdak.ru orgasmus-virtual.com
naturalpharm.com.ua osnova3.ru
neodownload.webcam pammik.ru
netfacet.net parfusale.se
networkad.net partner-high.men
nottyu.xyz pensplan4u.com
nuup.info playlott.com
obiavo.su pricheski-video.com
odesproperty.com prize44.com
odoratus.net prizeestates.cricket
ofanda.com prodvigator.ua
oneshotdate.com
ontargetseo.us
ooo-olni.ru
paleohub.info
paparazzistudios.com.au
perm.xrus.org
piski.top
pixgood.com
podshipniki-nsk.ru
pokerniydom.ru
polcin.de
pornmania.pl
porno-dojki.net
pornosemki.info
pozdrawleniya.ru
pravoholding.ru
prointer.net.ua
promgirldresses.xyz
prostitutki-novgoroda.org prostitutki-novgoroda.org
pssucai.info purchasepillsnorx.com
qpypcx.com quangcaons.com
renecaovillasale.online queerspace.com
replicaclub.ru rangoman.date
rn-to-bsn.com realtytimes.com
sankt-peterburg.nodup.ru remorcicomerciale.ro
sawin.beth.webd.pl robertefuller.com
scanmyphones.com robotixix.com
securesmrt-dt.com running-line.ru
seksvideoonlain.com rybalka-opt.ru
sexpartygirls.net samara.rosfirm.ru
sextracker.com sarm.tk
savetubevideo.com
scanhub.ru
searchinquire.com
seo-prof1.xyz
seocdvig.ru
sexphoto.site
sextracker.de
shikiso.info
shiksabd.com
shitting.pro shitting.pro
signoredom.com
site-speed-check.site site-speed-check.site
sk.golden-praga.ru skutecznetabletkinaporostwlosow.pl
smadihome.com snow.nvr163.com
snworks.com space4update.pw
social-s-ggg.xyz spasswelt.net
softomix.net spywarenuker.com
specialfinanceoffers.com stockquotes.wooeb.com
spin2016.cf storehouse.ua
stariy-baku.com stpolice.com
stoki.ru streamin.to
students-cheapskate.ml studentguide.ru
summerlinhomes411.info studworks.org
superfish.com suralink.com
superkanpo.com swinon.site
svarkagid.com swsociety.se
swagbucks.com tako3.com
the-trader.net telefonsexsofort.tk
thefarmergame.com tizanidinedruginteractions.blogspot.com
titan-ads.life tonerbox.kz
tizanidinecost.blogspot.com torrents.life
tizanidinestreetprice.blogspot.com tracksz.co
tizanidineusedfor.blogspot.com tramadolandtizanidine.blogspot.com
tksn.ru ukrtvory.in.ua
tn811.us um-razum.ru
top250movies.ru unacittaconte.org
tubeoffline.com unpredictable.ga
twincitiescarservice.com uytmaster.ru
twitlinks.com vadimkravtcov.ru
twodollarshows.com valmetrundan.se
unrealcommander.biz vepad.com
upproar.com videochat.mx
url-extractor.xyz
usswrite.com
vegan-foods.us
veopornogratis.xxx
video-production.com
videos-for-your-business.com
vintontech.info
viven.host.sk
vtcdns.com vtcdns.com
vucms.com webmonetizer.net
wallpaperaccess.com weboptimizes.com
web.cvut.cz website-speed-checker.site
webcamdevochka.com weekes.biz.tc
weddingdresses.xyz whatistizanidine2mg.blogspot.com
westum.se wholesalejerseyscheapjerseys.us.com
where-toget.com womama.ru
wygraj-skiny.win works.if.ua
wzgyyq.com worldhistory.biz
xev.ru x-porno.video
xmlinde.com xcombear.ru
xmronta.com xn----itbkqkfiq.xn--p1ai
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai xn--90acenikpebbdd4f6d.xn--p1ai
xvideosporn.biz xpresscare.ru
xxxdatinglocal.us xtube.com
youporn-ru.com yaderenergy.ru
yousense.info yellowproxy.net
zarabotat-na-sajte.ru yugk.net
zarabotok--doma.ru zaim-pod-zalog-krasnodar.ru
zerocash.msk.ru zambini.ru
zetgie.com.pl zarabiaj-dzis.pl
zigarettenonlinekaufen2.bloog.pl zhuravlev.info
zigzog.ru zixizop.net.ru
znakomstva-moskva77.ru
zooggames.com
zvetki.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.18 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2920 ### Version: V4.2022.02.2921
### Updated: Tue Feb 15 22:01:15 UTC 2022 ### Updated: Wed Feb 16 07:24:43 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2920 ### Version: V4.2022.02.2921
### Updated: Tue Feb 15 22:01:15 UTC 2022 ### Updated: Wed Feb 16 07:24:43 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##