Version 2.2017.01 - Formatting changes to globalblacklist.conf some sections merged for ease of maintenance between projects

CHANGELOG.md

@@ -1,5 +1,9 @@
 #CHANGELOG - Nginx Bad Bot Blocker
 
+###2017-01-29
+- Changed formatting of globalblacklist.conf, some sections like semalt and miraibot merged into other sections for easier maintenance.
+- Additional notes, testing instructions and commenting added in globalblacklist.conf file
+
 ###2016-12-17
 - Removed "CPython" from bad bots list. This user agent string "python-requests/2.5.3 CPython/2.7.9 Linux/3.16.0-4-amd64" is used by a valid Google Feed Parser called "UniversalFeedParser/5.2.1 +https://code.google.com/p/feedparser/"
 

LICENSE.md

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2017 Mitchell Krog - mitchellkrog@gmail.com
+https://github.com/mitchellkrogza
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

Pull_Requests_Here_Please/badreferers.list

@@ -4,6 +4,7 @@
 00it.com
 0ca29773681c7e82.com
 0n-line.tv
+1-99seo.com
 1-free-share-buttons.com
 1.bp.blogspot.com
 100.kutikomi.net
@@ -348,6 +349,7 @@ azbukafree.com
 azlex.uz
 azte.ch
 b-buyeasy.com
+b0ts.xf0.pw
 b2b-lounge.com
 babespcs.com
 babieca.com
@@ -395,6 +397,9 @@ berlininsl.com
 berrymall.ru
 best-businessman.ru
 best-deals-products.com
+best-seo-offer.com
+best-seo-software.xyz
+best-seo-solution.com
 bestbuy.ca
 bestchoice.cf
 bestcurs.org
@@ -885,6 +890,7 @@ dinkolove.ya.ru
 diplom-nk.com
 dipstar.org
 directrev.com
+disabled.racing
 discountliv.com
 discovertreasure-a.akamaihd.net
 discovertreasurenow.com
@@ -1039,6 +1045,7 @@ envaseslotusama.com
 enviromusic.net
 eonpal.com
 epngo.bz
+eport.queryhost.xyz
 eralph.tk
 erank.eu
 eredijovon.com
@@ -1185,6 +1192,7 @@ fitness-video.net
 fitnesspiks.com
 fiuxy.com
 fiverr.com
+fix-website-errors.com
 fizika.tv
 fjbaxywam.com
 flash4fun.com
@@ -1267,6 +1275,7 @@ free-social-buttons6.xyz
 free-social-buttons7.xyz
 free-stock-illustration.com
 free-traffic.xyz
+free-video-tool.com
 freeasecret.com
 freelotto.com
 freenode.info
@@ -1581,6 +1590,7 @@ i-service.kz
 i-x.wiki
 iamsport.org
 ibb.com.ua
+icmp.online
 ico.re
 icoolgramgoods.com
 ictizanidinehcl4mg.blogspot.com
@@ -1627,6 +1637,7 @@ imperia31.ru
 imperiafilm.ru
 import-sales.com
 impresagaia.it
+imscaredaf.xyz
 inbabes.sexushost.com
 inboundlinks.win
 inboxdollars.com
@@ -1737,6 +1748,8 @@ ketoanhanoi.info
 ketrzyn.pl
 keyword-suggestions.com
 keywordhut.com
+keywords-monitoring-success.com
+keywords-monitoring-your-success.com
 keywordsking.com
 keywordsuggest.org
 keywordteam.net
@@ -2565,6 +2578,7 @@ qitt.ru
 quality-traffic.com
 qualitymarketzone.com
 queen.sim-com.ru
+queryhost.xyz
 quick-offer.com
 quick-seeker.com
 quit-smoking.ga
@@ -2580,6 +2594,7 @@ rangapoker.com
 rank-checker.online
 rankchecker.online
 ranking2017.ga
+rankings-analytics.com
 ranksays.com
 rankscanner.com
 ranksignals.com
@@ -2632,6 +2647,7 @@ replicaclub.ru
 research.ifmo.ru
 resellerclub.com
 responsinator.com
+responsive-test.net
 respublica-otel.ru
 restorator-msk.ru
 resultshub-a.akamaihd.net
@@ -2705,6 +2721,7 @@ samolet.fr
 sampleletters.net
 sanatorrii.ru
 sanjosestartups.com
+santasbigcandycane.cx
 santasgift.ml
 santechnik.jimdo.com
 sanyuprojects.com
@@ -2785,8 +2802,12 @@ seeresultshub-a.akamaihd.net
 segol.tv
 seksotur.ru
 selfipicz.com
+semalt.com
+semaltmedia.com
 semprofile.com
 senger.atspace.co.uk
+seo-2-0.com
+seo-platform.com
 seo-smm.kz
 seo-tools-optimizing.com
 seoanalyses.com
@@ -3055,6 +3076,7 @@ stroymonolit.su
 studiofaca.com
 stuff-about-money.com
 styro.ru
+success-seo.com
 suche.t-online.de
 sugarkun.com
 suggest-keywords.com
@@ -3077,6 +3099,7 @@ svarkagid.com
 svetlotorg.ru
 svetodiodoff.ru
 swagbucks.com
+swinginwithme.ru
 swiped.su
 sygraem.com
 szansadlarolnikow.com.pl
@@ -3217,6 +3240,8 @@ toloka.hurtom.com
 tomatis.gospartner.com
 tomck.com
 tootoo.to
+top1-seo-service.com
+top10-way.com
 top250movies.ru
 topads.men
 topappspro.com
@@ -3381,6 +3406,7 @@ viagra.pp.ua
 viagroid.ru
 viandpet.com
 viberdownload10.com
+video--production.com
 video-chat.cn
 video-chat.in
 video-hollywood.ru
@@ -3392,6 +3418,7 @@ videofrost.com
 videofrost.net
 videokrik.net
 videooko.weebly.com
+videos-for-your-business.com
 videosbox.ru
 videotuber.ru
 viel.su
@@ -3579,6 +3606,7 @@ xblog.in
 xboxster.ru
 xcombear.ru
 xep.info
+xf0.pw
 xfire.com
 xgftnlrt.bloger.index.hr
 xivqt.net.ru

README.md

@@ -1,7 +1,10 @@
 # Nginx Bad Bot Blocker, Referer Blocker, Anti DDOS, Bad IP Blocker and Wordpress Theme Detector Blocker
 ## The Ultimate Bad Bot and Referer Blocker for Nginx Web Servers including anti DDOS system and Wordpress Theme Detector Blocking
 
+### Version 2.2017.01
+
 ### Created by: https://github.com/mitchellkrogza
+### Copyright Mitchell Krog <mitchellkrog@gmail.com>
 
 #### For Nginx Web Server - https://www.nginx.com/
 
@@ -276,13 +279,28 @@ See the Fail2Ban folder for instructions on configuring this great add on for th
 
 ### If this helped you why not [buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=XP2AZ4S5HNAWQ):beer:
 
-##WARRANTY OR LICENSE
+#MIT License
 
-- This is free to use and modify as you wish. 
+##Copyright (c) 2017 Mitchell Krog - mitchellkrog@gmail.com
-- No warranties are express or implied.
+##https://github.com/mitchellkrogza
-- You use this entirely at your own Risk.
+
-- Fork your own copy from this repo and feel free to change it to your needs or contribute to it.
+Permission is hereby granted, free of charge, to any person obtaining a copy
-- If you break it yourself, you fix it yourself.
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
 
 ##### Some other free projects
 

conf.d/globalblacklist.conf

@@ -2,23 +2,36 @@
 ### THE ULTIMATE NGINX BAD BOT BLOCKER
 ### **********************************
 
+### Version 2.2017.01
+
 ### This file implements a checklist / blacklist for good user agents, bad user agents and
 ### bad referrers. It also has whitelisting for your own IP's and known good IP Ranges
 ### and also has rate limiting functionality for bad bots who you only want to rate limit
 ### and not actually block out entirely. It is powerful and also flexible.
 
 ### Created By: https://github.com/mitchellkrogza/
+### Copyright Mitchell Krog - <mitchellkrog@gmail.com>
+
+### Originally Inspired By:
+### - https://github.com/mariusv/nginx-badbot-blocker
+### - https://github.com/oohnoitz/nginx-blacklist
 
 ### Last Updated
-### Sat Jan 28 10:10:46 SAST 2017
+### Sun Jan 29 09:27:09 SAST 2017
 ### End Last Updated
 
 ### Generated in
-### 0.313077449799 seconds
+### 0.713077783585 seconds
 ### End Generated in
 
 ### Tested on: nginx/1.10.0 (Ubuntu 16.04)
 
+### This list was developed and is in use on a live Nginx server running some very busy web sites.
+### It was built from the ground up using real data from daily logs and is updated almost daily.
+### It has been extensively tested for false positives and all additions to the lists of bad user agents,
+### spam referers, rogue IP address, scanners, scrapers and domain hijacking sites are extensively checked
+### before they are added. It is monitored extensively for any false positives.
+
 ### *********
 ### Features:
 ### *********
@@ -101,10 +114,9 @@ map $http_user_agent $bad_bot {
 
 	default		0;
 
-# ***********************
+# ***********************************************
-# Allow Good Bots We Know
+# Allow Good User-Agent Strings We Know and Trust
-# ***********************
+# ***********************************************
-# https://perishablepress.com/list-all-user-agents-top-search-engines/
 
 # START GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
 	"~*adidxbot"		0;
@@ -138,9 +150,10 @@ map $http_user_agent $bad_bot {
 	"~*yahoo"		0;
 # END GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
 
-# ****************************
+# **************************************************
-# Allowed Through
+# User-Agent Strings Allowed Throug but Rate Limited
-# ****************************
+# **************************************************
+# Some people block libwww-perl, it us widely used in many valid (non rogue) agents
 # I allow libwww-perl as I use it for monitoring systems with Munin but it is rate limited
 
 # START ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
@@ -152,9 +165,9 @@ map $http_user_agent $bad_bot {
 	"~*WordPress"		1;
 # END ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
 
-# *******************************************************************
+# **************************************************************
-# Rate Limit / Block Other Bots who get a bit aggressive on bandwidth
+# Rate Limited User-Agents who get a bit aggressive on bandwidth
-# *******************************************************************
+# **************************************************************
 
 # START LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
 	"~*Alexa"		2;
@@ -173,28 +186,18 @@ map $http_user_agent $bad_bot {
 	"~*YandexImages"		2;
 # END LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
 		
-# ****************************
+# *********************************************
-# Known Vulnerability Scanners
+# Bad User-Agent Strings That We Block Outright
-# ****************************
+# *********************************************
-
+# This includes:
-# START KNOWN SCANNERS ### DO NOT EDIT THIS LINE AT ALL ###
+# Known Vulnerability Scanners (now merged into one section)
-	"~*Acunetix"		3;
-	"~*FHscan"		3;
-	"~*masscan"		3;
-	"~*probethenet"		3;
-	"~*scanbot"		3;
-	"~*wprecon"		3;
-# END KNOWN SCANNERS ### DO NOT EDIT THIS LINE AT ALL ###
-	
-# ****************************
-# Known Bad User Agents
-# ****************************
 
 # START BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
 	"~*80legs"		3;
 	"~*360Spider"		3;
 	"~*Aboundex"		3;
 	"~*Abonti"		3;
+	"~*Acunetix"		3;
 	"~*AhrefsBot"		3;
 	"~*AIBOT"		3;
 	"~*aipbot"		3;
@@ -293,6 +296,7 @@ map $http_user_agent $bad_bot {
 	"~*ExtractorPro"		3;
 	"~*EyeNetIE"		3;
 	"~*ezooms"		3;
+	"~*FHscan"		3;
 	"~*fimap"		3;
 	"~*FlashGet"		3;
 	"~*flunky"		3;
@@ -384,6 +388,7 @@ map $http_user_agent $bad_bot {
 	"~*majestic12"		3;
 	"~*MarkMonitor"		3;
 	"~*MarkWatch"		3;
+	"~*masscan"		3;
 	"~*Mass\ Downloader"		3;
 	"~*Mata\ Hari"		3;
 	"~*MegaIndex.ru"		3;
@@ -450,6 +455,7 @@ map $http_user_agent $bad_bot {
 	"~*Pixray"		3;
 	"~*Pockey"		3;
 	"~*POE-Component-Client-HTTP"		3;
+	"~*probethenet"		3;
 	"~*ProPowerBot"		3;
 	"~*ProWebWalker"		3;
 	"~*proximic"		3;
@@ -467,6 +473,7 @@ map $http_user_agent $bad_bot {
 	"~*RepoMonkey"		3;
 	"~*Rogerbot"		3;
 	"~*ScanAlert"		3;
+	"~*scanbot"		3;
 	"~*Scrapy"		3;
 	"~*Screaming\ Frog\ SEO\ Spider"		3;
 	"~*ScreenerBot"		3;
@@ -667,8 +674,7 @@ map $http_referer $bad_words {
 # ************************
 # Bad Referer Single Words
 # ************************
-
+# These are Words and Terms often found tagged onto domains or within url query strings.
-# Found tagged onto domains or in query strings.
 
 # START BAD REFERER WORDS ### DO NOT EDIT THIS LINE AT ALL ###
 	"~*advair"		1;
@@ -757,6 +763,13 @@ map $http_referer $bad_words {
 # Now a list of bad referer urls these domains or any combination of them ie .com .net
 # will be blocked out. Doesn't matter if the protocol is http, https or even ftp
 
+# This section includes:
+# **********************
+# Blocking of SEO company Semalt.com (now merged into this one section)
+# MIRAI Botnet Domains Used for Mass Attacks
+# Other known bad SEO companies and Ad Hijacking Sites  
+# Sites linked to malware, adware and ransomware
+
 # *****************
 # PLEASE TEST !!!!
 # *****************
@@ -799,12 +812,13 @@ map $http_referer $bad_referer {
 	hostnames;
 	default 0;
 
-# **************************************************************************************
+# ************************************
-# GOOD REFERERS - Spared from Checking - Your own web site domain names and server names
+# GOOD REFERERS - Spared from Checking 
-# **************************************************************************************
+# ************************************
 
 # Add your own domain names here to spare them from referer checking (one per line)
-	"~*~*mydomain.com"		0;
+
+  	"~*~*mydomain.com"		0;
 
 # START BAD REFERERS ### DO NOT EDIT THIS LINE AT ALL ###
 	"~*000free.us"		1;
@@ -813,6 +827,7 @@ map $http_referer $bad_referer {
 	"~*00it.com"		1;
 	"~*0ca29773681c7e82.com"		1;
 	"~*0n-line.tv"		1;
+	"~*1-99seo.com"		1;
 	"~*1-free-share-buttons.com"		1;
 	"~*1.bp.blogspot.com"		1;
 	"~*100.kutikomi.net"		1;
@@ -1157,6 +1172,7 @@ map $http_referer $bad_referer {
 	"~*azlex.uz"		1;
 	"~*azte.ch"		1;
 	"~*b-buyeasy.com"		1;
+	"~*b0ts.xf0.pw"		1;
 	"~*b2b-lounge.com"		1;
 	"~*babespcs.com"		1;
 	"~*babieca.com"		1;
@@ -1204,6 +1220,9 @@ map $http_referer $bad_referer {
 	"~*berrymall.ru"		1;
 	"~*best-businessman.ru"		1;
 	"~*best-deals-products.com"		1;
+	"~*best-seo-offer.com"		1;
+	"~*best-seo-software.xyz"		1;
+	"~*best-seo-solution.com"		1;
 	"~*bestbuy.ca"		1;
 	"~*bestchoice.cf"		1;
 	"~*bestcurs.org"		1;
@@ -1694,6 +1713,7 @@ map $http_referer $bad_referer {
 	"~*diplom-nk.com"		1;
 	"~*dipstar.org"		1;
 	"~*directrev.com"		1;
+	"~*disabled.racing"		1;
 	"~*discountliv.com"		1;
 	"~*discovertreasure-a.akamaihd.net"		1;
 	"~*discovertreasurenow.com"		1;
@@ -1848,6 +1868,7 @@ map $http_referer $bad_referer {
 	"~*enviromusic.net"		1;
 	"~*eonpal.com"		1;
 	"~*epngo.bz"		1;
+	"~*eport.queryhost.xyz"		1;
 	"~*eralph.tk"		1;
 	"~*erank.eu"		1;
 	"~*eredijovon.com"		1;
@@ -1994,6 +2015,7 @@ map $http_referer $bad_referer {
 	"~*fitnesspiks.com"		1;
 	"~*fiuxy.com"		1;
 	"~*fiverr.com"		1;
+	"~*fix-website-errors.com"		1;
 	"~*fizika.tv"		1;
 	"~*fjbaxywam.com"		1;
 	"~*flash4fun.com"		1;
@@ -2076,6 +2098,7 @@ map $http_referer $bad_referer {
 	"~*free-social-buttons7.xyz"		1;
 	"~*free-stock-illustration.com"		1;
 	"~*free-traffic.xyz"		1;
+	"~*free-video-tool.com"		1;
 	"~*freeasecret.com"		1;
 	"~*freelotto.com"		1;
 	"~*freenode.info"		1;
@@ -2390,6 +2413,7 @@ map $http_referer $bad_referer {
 	"~*i-x.wiki"		1;
 	"~*iamsport.org"		1;
 	"~*ibb.com.ua"		1;
+	"~*icmp.online"		1;
 	"~*ico.re"		1;
 	"~*icoolgramgoods.com"		1;
 	"~*ictizanidinehcl4mg.blogspot.com"		1;
@@ -2436,6 +2460,7 @@ map $http_referer $bad_referer {
 	"~*imperiafilm.ru"		1;
 	"~*import-sales.com"		1;
 	"~*impresagaia.it"		1;
+	"~*imscaredaf.xyz"		1;
 	"~*inbabes.sexushost.com"		1;
 	"~*inboundlinks.win"		1;
 	"~*inboxdollars.com"		1;
@@ -2546,6 +2571,8 @@ map $http_referer $bad_referer {
 	"~*ketrzyn.pl"		1;
 	"~*keyword-suggestions.com"		1;
 	"~*keywordhut.com"		1;
+	"~*keywords-monitoring-success.com"		1;
+	"~*keywords-monitoring-your-success.com"		1;
 	"~*keywordsking.com"		1;
 	"~*keywordsuggest.org"		1;
 	"~*keywordteam.net"		1;
@@ -3374,6 +3401,7 @@ map $http_referer $bad_referer {
 	"~*quality-traffic.com"		1;
 	"~*qualitymarketzone.com"		1;
 	"~*queen.sim-com.ru"		1;
+	"~*queryhost.xyz"		1;
 	"~*quick-offer.com"		1;
 	"~*quick-seeker.com"		1;
 	"~*quit-smoking.ga"		1;
@@ -3389,6 +3417,7 @@ map $http_referer $bad_referer {
 	"~*rank-checker.online"		1;
 	"~*rankchecker.online"		1;
 	"~*ranking2017.ga"		1;
+	"~*rankings-analytics.com"		1;
 	"~*ranksays.com"		1;
 	"~*rankscanner.com"		1;
 	"~*ranksignals.com"		1;
@@ -3441,6 +3470,7 @@ map $http_referer $bad_referer {
 	"~*research.ifmo.ru"		1;
 	"~*resellerclub.com"		1;
 	"~*responsinator.com"		1;
+	"~*responsive-test.net"		1;
 	"~*respublica-otel.ru"		1;
 	"~*restorator-msk.ru"		1;
 	"~*resultshub-a.akamaihd.net"		1;
@@ -3514,6 +3544,7 @@ map $http_referer $bad_referer {
 	"~*sampleletters.net"		1;
 	"~*sanatorrii.ru"		1;
 	"~*sanjosestartups.com"		1;
+	"~*santasbigcandycane.cx"		1;
 	"~*santasgift.ml"		1;
 	"~*santechnik.jimdo.com"		1;
 	"~*sanyuprojects.com"		1;
@@ -3594,8 +3625,12 @@ map $http_referer $bad_referer {
 	"~*segol.tv"		1;
 	"~*seksotur.ru"		1;
 	"~*selfipicz.com"		1;
+	"~*semalt.com"		1;
+	"~*semaltmedia.com"		1;
 	"~*semprofile.com"		1;
 	"~*senger.atspace.co.uk"		1;
+	"~*seo-2-0.com"		1;
+	"~*seo-platform.com"		1;
 	"~*seo-smm.kz"		1;
 	"~*seo-tools-optimizing.com"		1;
 	"~*seoanalyses.com"		1;
@@ -3864,6 +3899,7 @@ map $http_referer $bad_referer {
 	"~*studiofaca.com"		1;
 	"~*stuff-about-money.com"		1;
 	"~*styro.ru"		1;
+	"~*success-seo.com"		1;
 	"~*suche.t-online.de"		1;
 	"~*sugarkun.com"		1;
 	"~*suggest-keywords.com"		1;
@@ -3886,6 +3922,7 @@ map $http_referer $bad_referer {
 	"~*svetlotorg.ru"		1;
 	"~*svetodiodoff.ru"		1;
 	"~*swagbucks.com"		1;
+	"~*swinginwithme.ru"		1;
 	"~*swiped.su"		1;
 	"~*sygraem.com"		1;
 	"~*szansadlarolnikow.com.pl"		1;
@@ -4026,6 +4063,8 @@ map $http_referer $bad_referer {
 	"~*tomatis.gospartner.com"		1;
 	"~*tomck.com"		1;
 	"~*tootoo.to"		1;
+	"~*top1-seo-service.com"		1;
+	"~*top10-way.com"		1;
 	"~*top250movies.ru"		1;
 	"~*topads.men"		1;
 	"~*topappspro.com"		1;
@@ -4190,6 +4229,7 @@ map $http_referer $bad_referer {
 	"~*viagroid.ru"		1;
 	"~*viandpet.com"		1;
 	"~*viberdownload10.com"		1;
+	"~*video--production.com"		1;
 	"~*video-chat.cn"		1;
 	"~*video-chat.in"		1;
 	"~*video-hollywood.ru"		1;
@@ -4201,6 +4241,7 @@ map $http_referer $bad_referer {
 	"~*videofrost.net"		1;
 	"~*videokrik.net"		1;
 	"~*videooko.weebly.com"		1;
+	"~*videos-for-your-business.com"		1;
 	"~*videosbox.ru"		1;
 	"~*videotuber.ru"		1;
 	"~*viel.su"		1;
@@ -4388,6 +4429,7 @@ map $http_referer $bad_referer {
 	"~*xboxster.ru"		1;
 	"~*xcombear.ru"		1;
 	"~*xep.info"		1;
+	"~*xf0.pw"		1;
 	"~*xfire.com"		1;
 	"~*xgftnlrt.bloger.index.hr"		1;
 	"~*xivqt.net.ru"		1;
@@ -4555,85 +4597,12 @@ map $http_referer $bad_referer {
 	"~*zyzzcentral.ru"		1;
 # END BAD REFERERS ### DO NOT EDIT THIS LINE AT ALL ###
 
-
-# **************************************
-# SEMALT BLOCK - Read following articles
-# **************************************
-
-# https://www.incapsula.com/blog/semalt-botnet-spam.html
-# http://www.kymodo.com.au/what-is-semalt-doing-on-your-website/
-# https://www.ohow.co/what-is-semalt-com-and-semalt-media-referral-analytics/
-# http://www.forthea.com/block-semalt-crawler/
-
-# I actually registered on their site to see what SEO services and reports they
-# offer and within 3 hours I had a phone call from Florida in the US with a sales
-# guy hassling me and telling me how wonderful they were and how they would guarantee
-# me a number one place on Google ... ha ha !!! Goodbye to bad seo companies !!!
-# I have also catered for them buying any other domain like .info .net .org
-
-# I will no doubt be adding to this list as they email me every day now so I am sure
-# I will see more referers of theirs popping up. Domains are cheap !!!
-
-# START SEMALT BLOCK ### DO NOT EDIT THIS LINE AT ALL ###
-	"~*1-99seo.com"		1;
-	"~*best-seo-offer.com"		1;
-	"~*best-seo-software.xyz"		1;
-	"~*best-seo-solution.com"		1;
-	"~*fix-website-errors.com"		1;
-	"~*free-video-tool.com"		1;
-	"~*keywords-monitoring-success.com"		1;
-	"~*keywords-monitoring-your-success.com"		1;
-	"~*rankings-analytics.com"		1;
-	"~*responsive-test.net"		1;
-	"~*semalt.com"		1;
-	"~*semaltmedia.com"		1;
-	"~*seo-2-0.com"		1;
-	"~*seo-platform.com"		1;
-	"~*success-seo.com"		1;
-	"~*top1-seo-service.com"		1;
-	"~*top10-way.com"		1;
-	"~*videos-for-your-business.com"		1;
-	"~*video--production.com"		1;
-# END SEMALT BLOCK ### DO NOT EDIT THIS LINE AT ALL ###
-  
-# *******************
-# MIRAI Botnet Attack
-# ********************
-# New block added for all known domains and referers being used by
-# the MIRAI botnet attack
-# See - http://blog.level3.com/security/grinch-stole-iot/
-  
-# START MIRAI REFERERS ### DO NOT EDIT THIS LINE AT ALL ###
-	"~*b0ts.xf0.pw"		1;
-	"~*disabled.racing"		1;
-	"~*cnc.disabled.racing"		1;
-	"~*dongs.disabled.racing"		1;
-	"~*dongs.icmp.online"		1;
-	"~*eport.queryhost.xyz"		1;
-	"~*gay.disabled.racing"		1;
-	"~*icmp.online"		1;
-	"~*imscaredaf.xyz"		1;
-	"~*kankerc.queryhost.xyz"		1;
-	"~*lol.disabled.racing"		1;
-	"~*meme.icmp.online"		1;
-	"~*network.santasbigcandycane.cx"		1;
-	"~*penis.disabled.racing"		1;
-	"~*queryhost.xyz"		1;
-	"~*report.disabled.racing"		1;
-	"~*report.santasbigcandycane.cx"		1;
-	"~*report.xf0.pw"		1;
-	"~*reports.icmp.online"		1;
-	"~*santasbigcandycane.cx"		1;
-	"~*swinginwithme.ru"		1;
-	"~*xf0.pw"		1;
-# END MIRAI REFERERS ### DO NOT EDIT THIS LINE AT ALL ###
-
 }
 
 
-# *****************************
+# ***********************************************
-# The GOOD the BAD and the UGLY
+# WHITELISTING AND BLACKLISTING IP ADDRESS RANGES
-# *****************************
+# ***********************************************
 
 # Geo directive to deny and also whitelist certain ip addresses
 
@@ -4647,9 +4616,9 @@ geo $validate_client {
 
 	default		0;
 
-# **************************
+# ***********************************
-# Whitelist OWN IP addresses
+# Whitelist all your OWN IP addresses
-# **************************
+# ***********************************
 
 # Whitelist all your own IP addresses from any validate_client checks
 # Add all your IP addresses and ranges below (one per line)
@@ -4722,8 +4691,10 @@ geo $validate_client {
 # music and other materials. I personally believe a lot of block lists who originally recommended
 # blocking Cyveillance have done so to protect their torrent or p2p sites from being scanned.
 # I personally have now unblocked them as image theft is a big problem of mine but if you
-# do want to block Cyveillance you can simply modify the entries in the block from "0" to "1"
+# do want to block Cyveillance you can simply modify the entries in the block below from "0" to "1"
 # Getty Images is one such company who appears to use Cyveillance to help monitor for copyright theft.
+  
+# If you really do want to block them change all the 0's below to 1.
 
 # START CYVEILLANCE BLOCK ### DO NOT EDIT THIS LINE AT ALL ###
 	38.100.19.8/29		0;
@@ -4798,18 +4769,26 @@ geo $validate_client {
 	54.242.239.179		1;
 # END NIBBLER ### DO NOT EDIT THIS LINE AT ALL ###
 
-# *******************
+
-# MIRAI Botnet Attack 
+# ****************************
-# *******************
+# Known Bad IP's and IP Ranges
-# Known IP ranges being used in the Mirai Botnet Attack
+# ****************************
-# See - http://blog.level3.com/security/grinch-stole-iot/
+
-  
+# Add any other IPs or Subnets here that you wish to block
-# START MIRAIBOTNET IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
+# Although any permanent blocks should be done using Fail2Ban and IPTables and not
+# hampering down Nginx with all the checks against perma-banned IP's
+
+# START BAD IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
+	10.17.69.6		1;
+	10.221.36.100		1;
 	104.223.37.150		1;
+	104.5.92.27		1;
+	109.236.83.247		1;
 	137.74.49.205		1;
 	137.74.49.208		1;
 	149.56.151.180		1;
 	149.56.232.146		1;
+	150.70.0.0/16		1;
 	151.80.27.90		1;
 	151.80.99.90		1;
 	151.80.99.91		1;
@@ -4820,7 +4799,14 @@ geo $validate_client {
 	158.69.142.34		1;
 	166.62.80.172		1;
 	173.212.192.219		1;
+	173.234.11.105		1;
+	173.234.153.106		1;
+	173.234.153.30		1;
+	173.234.175.68		1;
+	173.234.31.9		1;
+	173.234.38.25		1;
 	176.126.245.213		1;
+	178.238.234.1		1;
 	185.100.87.238		1;
 	185.115.125.99		1;
 	185.130.225.65		1;
@@ -4837,9 +4823,15 @@ geo $validate_client {
 	185.93.185.11		1;
 	185.93.185.12		1;
 	188.209.52.101		1;
+	190.152.223.27		1;
 	191.96.249.29		1;
 	192.69.89.173		1;
+	195.229.241.174		1;
+	210.212.194.60		1;
+	216.218.147.194		1;
+	23.253.230.158		1;
 	23.89.159.176		1;
+	31.170.160.209		1;
 	45.32.186.11		1;
 	46.249.38.145		1;
 	46.249.38.146		1;
@@ -4852,14 +4844,23 @@ geo $validate_client {
 	46.249.38.154		1;
 	46.249.38.159		1;
 	51.255.172.22		1;
+	54.213.16.154		1;
+	54.213.9.111		1;
 	65.98.91.181		1;
+	69.162.124.237		1;
+	69.64.147.24		1;
 	72.8.183.202		1;
 	77.247.178.191		1;
 	77.247.178.47		1;
 	77.247.181.219		1;
+	78.31.184.0/21		1;
+	78.31.211.0/24		1;
 	80.87.205.10		1;
 	80.87.205.11		1;
+	85.17.230.23		1;
+	85.17.26.68		1;
 	91.185.190.172		1;
+	91.200.12.0/22		1;
 	92.222.66.137		1;
 	93.104.209.11		1;
 	93.158.200.103		1;
@@ -4869,45 +4870,6 @@ geo $validate_client {
 	93.158.200.126		1;
 	93.158.200.66		1;
 	93.158.200.68		1;
-# END MIRAIBOTNET BAD IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
-
-
-# ****************************
-# Other Bad IP's and IP Ranges
-# ****************************
-
-# Add any other IPs or Subnets here that you wish to block
-# Although any permanent blocks should be done using Fail2Ban and IPTables and not
-# hampering down Nginx with all the checks against perma-banned IP's
-
-# START BAD IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
-	10.17.69.6		1;
-	10.221.36.100		1;
-	23.253.230.158		1;
-	31.170.160.209		1;
-	54.213.9.111		1;
-	54.213.16.154		1;
-	69.64.147.24		1;
-	69.162.124.237		1;
-	78.31.184.0/21		1;
-	78.31.211.0/24		1;
-	85.17.26.68		1;
-	85.17.230.23		1;
-	91.200.12.0/22		1;
-	104.5.92.27		1;
-	109.236.83.247		1;
-	173.234.11.105		1;
-	173.234.31.9		1;
-	173.234.38.25		1;
-	173.234.153.30		1;
-	173.234.153.106		1;
-	173.234.175.68		1;
-	178.238.234.1		1;
-	190.152.223.27		1;
-	195.229.241.174		1;
-	210.212.194.60		1;
-	216.218.147.194		1;
-	150.70.0.0/16		1;
 # END BAD IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###  
 }
 

google-disavow.txt

@@ -4,6 +4,7 @@ domain:00go.com
 domain:00it.com
 domain:0ca29773681c7e82.com
 domain:0n-line.tv
+domain:1-99seo.com
 domain:1-free-share-buttons.com
 domain:1.bp.blogspot.com
 domain:100.kutikomi.net
@@ -348,6 +349,7 @@ domain:azbukafree.com
 domain:azlex.uz
 domain:azte.ch
 domain:b-buyeasy.com
+domain:b0ts.xf0.pw
 domain:b2b-lounge.com
 domain:babespcs.com
 domain:babieca.com
@@ -395,6 +397,9 @@ domain:berlininsl.com
 domain:berrymall.ru
 domain:best-businessman.ru
 domain:best-deals-products.com
+domain:best-seo-offer.com
+domain:best-seo-software.xyz
+domain:best-seo-solution.com
 domain:bestbuy.ca
 domain:bestchoice.cf
 domain:bestcurs.org
@@ -885,6 +890,7 @@ domain:dinkolove.ya.ru
 domain:diplom-nk.com
 domain:dipstar.org
 domain:directrev.com
+domain:disabled.racing
 domain:discountliv.com
 domain:discovertreasure-a.akamaihd.net
 domain:discovertreasurenow.com
@@ -1039,6 +1045,7 @@ domain:envaseslotusama.com
 domain:enviromusic.net
 domain:eonpal.com
 domain:epngo.bz
+domain:eport.queryhost.xyz
 domain:eralph.tk
 domain:erank.eu
 domain:eredijovon.com
@@ -1185,6 +1192,7 @@ domain:fitness-video.net
 domain:fitnesspiks.com
 domain:fiuxy.com
 domain:fiverr.com
+domain:fix-website-errors.com
 domain:fizika.tv
 domain:fjbaxywam.com
 domain:flash4fun.com
@@ -1267,6 +1275,7 @@ domain:free-social-buttons6.xyz
 domain:free-social-buttons7.xyz
 domain:free-stock-illustration.com
 domain:free-traffic.xyz
+domain:free-video-tool.com
 domain:freeasecret.com
 domain:freelotto.com
 domain:freenode.info
@@ -1581,6 +1590,7 @@ domain:i-service.kz
 domain:i-x.wiki
 domain:iamsport.org
 domain:ibb.com.ua
+domain:icmp.online
 domain:ico.re
 domain:icoolgramgoods.com
 domain:ictizanidinehcl4mg.blogspot.com
@@ -1627,6 +1637,7 @@ domain:imperia31.ru
 domain:imperiafilm.ru
 domain:import-sales.com
 domain:impresagaia.it
+domain:imscaredaf.xyz
 domain:inbabes.sexushost.com
 domain:inboundlinks.win
 domain:inboxdollars.com
@@ -1737,6 +1748,8 @@ domain:ketoanhanoi.info
 domain:ketrzyn.pl
 domain:keyword-suggestions.com
 domain:keywordhut.com
+domain:keywords-monitoring-success.com
+domain:keywords-monitoring-your-success.com
 domain:keywordsking.com
 domain:keywordsuggest.org
 domain:keywordteam.net
@@ -2565,6 +2578,7 @@ domain:qitt.ru
 domain:quality-traffic.com
 domain:qualitymarketzone.com
 domain:queen.sim-com.ru
+domain:queryhost.xyz
 domain:quick-offer.com
 domain:quick-seeker.com
 domain:quit-smoking.ga
@@ -2580,6 +2594,7 @@ domain:rangapoker.com
 domain:rank-checker.online
 domain:rankchecker.online
 domain:ranking2017.ga
+domain:rankings-analytics.com
 domain:ranksays.com
 domain:rankscanner.com
 domain:ranksignals.com
@@ -2632,6 +2647,7 @@ domain:replicaclub.ru
 domain:research.ifmo.ru
 domain:resellerclub.com
 domain:responsinator.com
+domain:responsive-test.net
 domain:respublica-otel.ru
 domain:restorator-msk.ru
 domain:resultshub-a.akamaihd.net
@@ -2705,6 +2721,7 @@ domain:samolet.fr
 domain:sampleletters.net
 domain:sanatorrii.ru
 domain:sanjosestartups.com
+domain:santasbigcandycane.cx
 domain:santasgift.ml
 domain:santechnik.jimdo.com
 domain:sanyuprojects.com
@@ -2785,8 +2802,12 @@ domain:seeresultshub-a.akamaihd.net
 domain:segol.tv
 domain:seksotur.ru
 domain:selfipicz.com
+domain:semalt.com
+domain:semaltmedia.com
 domain:semprofile.com
 domain:senger.atspace.co.uk
+domain:seo-2-0.com
+domain:seo-platform.com
 domain:seo-smm.kz
 domain:seo-tools-optimizing.com
 domain:seoanalyses.com
@@ -3055,6 +3076,7 @@ domain:stroymonolit.su
 domain:studiofaca.com
 domain:stuff-about-money.com
 domain:styro.ru
+domain:success-seo.com
 domain:suche.t-online.de
 domain:sugarkun.com
 domain:suggest-keywords.com
@@ -3077,6 +3099,7 @@ domain:svarkagid.com
 domain:svetlotorg.ru
 domain:svetodiodoff.ru
 domain:swagbucks.com
+domain:swinginwithme.ru
 domain:swiped.su
 domain:sygraem.com
 domain:szansadlarolnikow.com.pl
@@ -3217,6 +3240,8 @@ domain:toloka.hurtom.com
 domain:tomatis.gospartner.com
 domain:tomck.com
 domain:tootoo.to
+domain:top1-seo-service.com
+domain:top10-way.com
 domain:top250movies.ru
 domain:topads.men
 domain:topappspro.com
@@ -3381,6 +3406,7 @@ domain:viagra.pp.ua
 domain:viagroid.ru
 domain:viandpet.com
 domain:viberdownload10.com
+domain:video--production.com
 domain:video-chat.cn
 domain:video-chat.in
 domain:video-hollywood.ru
@@ -3392,6 +3418,7 @@ domain:videofrost.com
 domain:videofrost.net
 domain:videokrik.net
 domain:videooko.weebly.com
+domain:videos-for-your-business.com
 domain:videosbox.ru
 domain:videotuber.ru
 domain:viel.su
@@ -3579,6 +3606,7 @@ domain:xblog.in
 domain:xboxster.ru
 domain:xcombear.ru
 domain:xep.info
+domain:xf0.pw
 domain:xfire.com
 domain:xgftnlrt.bloger.index.hr
 domain:xivqt.net.ru