From 81d28c6d43f7cee7319b9e4cfae3f7d5d54977b7 Mon Sep 17 00:00:00 2001 From: Mitchell Krog Date: Thu, 20 Apr 2017 15:34:51 +0200 Subject: [PATCH] Version 2.2017.07 - Documentation Updates PLEASE NOTE CRITICAL CHANGES !!! 3 new include files must exist in your /etc/nginx/bots.d/ folder or Nginx will EMERG and fail reloading due to the missing files. PLEASE READ UPDATED CONFIGURATION INSTRUCTIONS --- CHANGELOG.md | 4 +- CONFIGURATION.md | 69 +++++++++++++++---- .../README.md | 2 +- README.md | 67 +++++++++++++++--- install-ngxblocker | 4 ++ 5 files changed, 120 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2b4cb103a..b955f65db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,9 +16,9 @@ - **VERY IMPORTANT** - Without the existence of the new include files blacklist-ips.conf, bad-referrer-words.conf and custom-bad-referrers.conf include files Nginx will FAIL RELOADING with EMERG :exclamation: -- PLEASE READ UPDATED CONFIGURATION INSTRUCTIONS +- PLEASE READ UPDATED CONFIGURATION INSTRUCTIONS :exclamation: :exclamation: :exclamation: -- AUTO UPDATE SCRIPT WILL FAIL NGINX RELOAD WITHOUT THE NEW INCLUDE FILES !!!! +- AUTO UPDATE SCRIPT WILL FAIL NGINX RELOAD WITHOUT THE NEW INCLUDE FILES !!!! :exclamation: :exclamation: :exclamation: - Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker** or simply send a blank email to **apache-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 2eddbd1e2..44abbfde9 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -3,7 +3,7 @@ ##### Created by: https://github.com/mitchellkrogza ##### Copyright Mitchell Krog -### Version 2.2017.05 +### Version 2.2017.07 ## Update Notification System Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** @@ -11,9 +11,9 @@ or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googl Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. # CONFIGURATION OF THE NGINX BAD BOT BLOCKER: -### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY +### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: -**If you miss one step you will get an nginx EMERG error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf or blacklist-user-agents.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.** +**If you miss one step you will get an nginx EMERG :exclamation: error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf, bad-referrer-words.conf, custom-bad-referrers.conf, blacklist-user-agents.conf or blacklist-ips.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.** ## AUTO INSTALLATION INSTRUCTIONS To Make Sure you copy all the correct files you can now use a simple bash setup script for copying the files into the correct nginx folders for you: @@ -97,8 +97,7 @@ Whitelist all your own domain names and IP addresses. **Please note important ch Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script. - -When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses. +When pulling any future updates now your domains and IP whitelists will not be overwritten. ## STEP 4: @@ -108,16 +107,62 @@ Copy the custom User-Agents blacklist file into your /etc/nginx/bots.d folder `cd /etc/nginx/bots.d` -- copy the blacklist-user-agents.conf file into the same folder +- copy the blacklist-user-agents.conf file from the repository `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf` Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block. - +When pulling any future updates now your custom User-Agents blacklist will not be overwritten. ## STEP 5: +**BLACKLIST USING YOUR OWN CUSTOM BAD REFERRERS** + +Copy the custom bad referrers blacklist file into your /etc/nginx/bots.d folder + +`cd /etc/nginx/bots.d` + +- copy the custom-bad-referrers.conf file from the repository + +`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/custom-bad-referrers.conf -O custom-bad-referrers.conf` + +Use nano, vim or any other text editor to edit (if needed) custom-bad-referrers.conf to include your own custom list of bad referrer domains that are not included in the blocker. +When pulling any future updates now your custom referrers list will not be overwritten. + +## STEP 6: + +**BLACKLIST IPS AND IP RANGES USING YOUR OWN CUSTOM LIST** + +Copy the custom IP blacklist file into your /etc/nginx/bots.d folder + +`cd /etc/nginx/bots.d` + +- copy the blacklist-ips.conf file from the repository + +`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-ips.conf -O blacklist-ips.conf` + +Use nano, vim or any other text editor to edit (if needed) blacklist-ips.conf to include your own custom list of IP Addresses and IP Ranges that you wish to block. +When pulling any future updates now your custom IP blacklist will not be overwritten. + +## STEP 7: + +**DOWNLOAD CUSTOM BAD REFERRER WORDS INCLUDE FILE FOR CUSTOMIZED SCANNING OF BAD WORDS** + +Copy the custom bad referrer words include file into your /etc/nginx/bots.d folder + +`cd /etc/nginx/bots.d` + +- copy the bad-referrer-words.conf file from the repository + +`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/bad-referrer-words.conf -O bad-referrer-words.conf` + +Use nano, vim or any other text editor to edit the bad-referrer-words.conf file as you like. +When pulling any future updates now your custom bad referrer words list will not be overwritten. + + +## STEP 8: + **INCLUDE IMPORTANT SETTINGS IN NGINX.CONF** **Also see SAMPLE-nginx.conf file in the root of this repository** @@ -143,14 +188,14 @@ The important settings file above adds the rate limiting functions and hash_buck The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements. -## STEP 6: **VERY IMPORTANT** +## STEP 9: **VERY IMPORTANT** **MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work. - `include /etc/nginx/conf.d/*` -## STEP 7: **VERY IMPORTANT** +## STEP 10: **VERY IMPORTANT** **ADD INCLUDE FILES INTO A VHOST** @@ -164,7 +209,7 @@ These includes MUST be added within a **server {}** block of a vhost otherwise y - `include /etc/nginx/bots.d/ddos.conf;` -## STEP 8: +## STEP 11: **TESTING YOUR NGINX CONFIGURATION** @@ -176,7 +221,7 @@ If you get no errors then you followed my instructions so now you can make the b The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working. -## STEP 9: +## STEP 12: **TESTING** @@ -201,7 +246,7 @@ Should respond with: curl: (52) Empty reply from server The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!! -## STEP 10: +## STEP 13: **UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names. diff --git a/Engintron_for_cPanel_WHM_Configuration_Example/README.md b/Engintron_for_cPanel_WHM_Configuration_Example/README.md index 042d19036..33d77f64f 100644 --- a/Engintron_for_cPanel_WHM_Configuration_Example/README.md +++ b/Engintron_for_cPanel_WHM_Configuration_Example/README.md @@ -23,7 +23,7 @@ - This example contains a modified file in **/utilities/https_vhosts.php** which will do the includes for you on any SSL vhosts you create. -### Version 2.2017.05 +### Version 2.2017.07 ##### Created by: https://github.com/mitchellkrogza ##### Copyright Mitchell Krog diff --git a/README.md b/README.md index 2c40030d3..2a57e796d 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ # Nginx Bad Bot and User-Agent Blocker, Spam Referrer Blocker, Anti DDOS, Bad IP Blocker and Wordpress Theme Detector Blocker ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking -### Version 2.2017.05 +### Version 2.2017.07 ##### Created by: https://github.com/mitchellkrogza ##### Copyright Mitchell Krog @@ -119,7 +119,7 @@ For bot's or spiders that you still want to allow but want to limit their visita # CONFIGURATION INSTRUCTIONS -### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY +### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: **If you miss one step you will get an nginx EMERG error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf or blacklist-user-agents.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.** @@ -205,8 +205,7 @@ Whitelist all your own domain names and IP addresses. **Please note important ch Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script. - -When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses. +When pulling any future updates now your domains and IP whitelists will not be overwritten. ## STEP 4: @@ -216,16 +215,62 @@ Copy the custom User-Agents blacklist file into your /etc/nginx/bots.d folder `cd /etc/nginx/bots.d` -- copy the blacklist-user-agents.conf file into the same folder +- copy the blacklist-user-agents.conf file from the repository `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf` Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block. - +When pulling any future updates now your custom User-Agents blacklist will not be overwritten. ## STEP 5: +**BLACKLIST USING YOUR OWN CUSTOM BAD REFERRERS** + +Copy the custom bad referrers blacklist file into your /etc/nginx/bots.d folder + +`cd /etc/nginx/bots.d` + +- copy the custom-bad-referrers.conf file from the repository + +`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/custom-bad-referrers.conf -O custom-bad-referrers.conf` + +Use nano, vim or any other text editor to edit (if needed) custom-bad-referrers.conf to include your own custom list of bad referrer domains that are not included in the blocker. +When pulling any future updates now your custom referrers list will not be overwritten. + +## STEP 6: + +**BLACKLIST IPS AND IP RANGES USING YOUR OWN CUSTOM LIST** + +Copy the custom IP blacklist file into your /etc/nginx/bots.d folder + +`cd /etc/nginx/bots.d` + +- copy the blacklist-ips.conf file from the repository + +`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-ips.conf -O blacklist-ips.conf` + +Use nano, vim or any other text editor to edit (if needed) blacklist-ips.conf to include your own custom list of IP Addresses and IP Ranges that you wish to block. +When pulling any future updates now your custom IP blacklist will not be overwritten. + +## STEP 7: + +**DOWNLOAD CUSTOM BAD REFERRER WORDS INCLUDE FILE FOR CUSTOMIZED SCANNING OF BAD WORDS** + +Copy the custom bad referrer words include file into your /etc/nginx/bots.d folder + +`cd /etc/nginx/bots.d` + +- copy the bad-referrer-words.conf file from the repository + +`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/bad-referrer-words.conf -O bad-referrer-words.conf` + +Use nano, vim or any other text editor to edit the bad-referrer-words.conf file as you like. +When pulling any future updates now your custom bad referrer words list will not be overwritten. + + +## STEP 8: + **INCLUDE IMPORTANT SETTINGS IN NGINX.CONF** **Also see SAMPLE-nginx.conf file in the root of this repository** @@ -251,14 +296,14 @@ The important settings file above adds the rate limiting functions and hash_buck The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements. -## STEP 6: **VERY IMPORTANT** +## STEP 9: **VERY IMPORTANT** **MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work. - `include /etc/nginx/conf.d/*` -## STEP 7: **VERY IMPORTANT** +## STEP 10: **VERY IMPORTANT** **ADD INCLUDE FILES INTO A VHOST** @@ -272,7 +317,7 @@ These includes MUST be added within a **server {}** block of a vhost otherwise y - `include /etc/nginx/bots.d/ddos.conf;` -## STEP 8: +## STEP 11: **TESTING YOUR NGINX CONFIGURATION** @@ -284,7 +329,7 @@ If you get no errors then you followed my instructions so now you can make the b The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working. -## STEP 9: +## STEP 12: **TESTING** @@ -309,7 +354,7 @@ Should respond with: curl: (52) Empty reply from server The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!! -## STEP 10: +## STEP 13: **UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names. diff --git a/install-ngxblocker b/install-ngxblocker index 716f28dfe..e05ab5ed4 100755 --- a/install-ngxblocker +++ b/install-ngxblocker @@ -2,6 +2,7 @@ # Bash Script for Installing the Nginx Bad Bot Blocker # Copyright - https://github.com/mitchellkrogza # Project Url: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker +# Version 2.2017.07 # PLEASE READ CONFIGURATION INSTRUCTIONS BEFORE USING THIS - THIS IS ONLY A PARTIAL INSTALLER # FOR COPYING THE FILES CORRECTLY TO NGINX INTO THE CORRECT FOLDERS @@ -35,6 +36,9 @@ sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bo sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf -O whitelist-ips.conf sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf -O whitelist-domains.conf sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf +sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-ips.conf -O blacklist-ips.conf +sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/bad-referrer-words.conf -O bad-referrer-words.conf +sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/custom-bad-referrers.conf -O custom-bad-referrers.conf exit 0 # PLEASE READ CONFIGURATION INSTRUCTIONS