vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Version 2.2017.07 - Documentation Updates

Browse source 
PLEASE NOTE CRITICAL CHANGES !!!

3 new include files must exist in your /etc/nginx/bots.d/ folder or Nginx will EMERG and fail reloading due to the missing files.

PLEASE READ UPDATED CONFIGURATION INSTRUCTIONS
This commit is contained in:
Mitchell Krog 2017-04-20 15:34:51 +02:00
parent b5dcef8bbd
commit 81d28c6d43
5 changed files with 120 additions and 26 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
CHANGELOG.md
View file

@ -16,9 +16,9 @@
- **VERY IMPORTANT** - Without the existence of the new include files blacklist-ips.conf, bad-referrer-words.conf and custom-bad-referrers.conf include files Nginx will FAIL RELOADING with EMERG :exclamation: - **VERY IMPORTANT** - Without the existence of the new include files blacklist-ips.conf, bad-referrer-words.conf and custom-bad-referrers.conf include files Nginx will FAIL RELOADING with EMERG :exclamation:
- PLEASE READ UPDATED CONFIGURATION INSTRUCTIONS - PLEASE READ UPDATED CONFIGURATION INSTRUCTIONS :exclamation: :exclamation: :exclamation:
- AUTO UPDATE SCRIPT WILL FAIL NGINX RELOAD WITHOUT THE NEW INCLUDE FILES !!!! - AUTO UPDATE SCRIPT WILL FAIL NGINX RELOAD WITHOUT THE NEW INCLUDE FILES !!!! :exclamation: :exclamation: :exclamation:
- Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker** - Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/apache-ultimate-bad-bot-blocker**
or simply send a blank email to **apache-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. or simply send a blank email to **apache-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe.

69
CONFIGURATION.md
View file

@ -3,7 +3,7 @@
##### Created by: https://github.com/mitchellkrogza ##### Created by: https://github.com/mitchellkrogza
##### Copyright Mitchell Krog <mitchellkrog@gmail.com> ##### Copyright Mitchell Krog <mitchellkrog@gmail.com>
### Version 2.2017.05 ### Version 2.2017.07
## Update Notification System ## Update Notification System
Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker**
@ -11,9 +11,9 @@ or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googl
Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place.
# CONFIGURATION OF THE NGINX BAD BOT BLOCKER: # CONFIGURATION OF THE NGINX BAD BOT BLOCKER:
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
**If you miss one step you will get an nginx EMERG error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf or blacklist-user-agents.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.** **If you miss one step you will get an nginx EMERG :exclamation: error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf, bad-referrer-words.conf, custom-bad-referrers.conf, blacklist-user-agents.conf or blacklist-ips.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.**
## AUTO INSTALLATION INSTRUCTIONS ## AUTO INSTALLATION INSTRUCTIONS
To Make Sure you copy all the correct files you can now use a simple bash setup script for copying the files into the correct nginx folders for you: To Make Sure you copy all the correct files you can now use a simple bash setup script for copying the files into the correct nginx folders for you:
@ -97,8 +97,7 @@ Whitelist all your own domain names and IP addresses. **Please note important ch
Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script. Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script.
When pulling any future updates now your domains and IP whitelists will not be overwritten.
When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses.
## STEP 4: ## STEP 4:
@ -108,16 +107,62 @@ Copy the custom User-Agents blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d` `cd /etc/nginx/bots.d`
- copy the blacklist-user-agents.conf file into the same folder - copy the blacklist-user-agents.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf`
Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block. Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block.
When pulling any future updates now your custom User-Agents blacklist will not be overwritten.
## STEP 5: ## STEP 5:
**BLACKLIST USING YOUR OWN CUSTOM BAD REFERRERS**
Copy the custom bad referrers blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the custom-bad-referrers.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/custom-bad-referrers.conf -O custom-bad-referrers.conf`
Use nano, vim or any other text editor to edit (if needed) custom-bad-referrers.conf to include your own custom list of bad referrer domains that are not included in the blocker.
When pulling any future updates now your custom referrers list will not be overwritten.
## STEP 6:
**BLACKLIST IPS AND IP RANGES USING YOUR OWN CUSTOM LIST**
Copy the custom IP blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the blacklist-ips.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-ips.conf -O blacklist-ips.conf`
Use nano, vim or any other text editor to edit (if needed) blacklist-ips.conf to include your own custom list of IP Addresses and IP Ranges that you wish to block.
When pulling any future updates now your custom IP blacklist will not be overwritten.
## STEP 7:
**DOWNLOAD CUSTOM BAD REFERRER WORDS INCLUDE FILE FOR CUSTOMIZED SCANNING OF BAD WORDS**
Copy the custom bad referrer words include file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the bad-referrer-words.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/bad-referrer-words.conf -O bad-referrer-words.conf`
Use nano, vim or any other text editor to edit the bad-referrer-words.conf file as you like.
When pulling any future updates now your custom bad referrer words list will not be overwritten.
## STEP 8:
**INCLUDE IMPORTANT SETTINGS IN NGINX.CONF** **INCLUDE IMPORTANT SETTINGS IN NGINX.CONF**
**Also see SAMPLE-nginx.conf file in the root of this repository** **Also see SAMPLE-nginx.conf file in the root of this repository**
@ -143,14 +188,14 @@ The important settings file above adds the rate limiting functions and hash_buck
The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements. The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements.
## STEP 6: **VERY IMPORTANT** ## STEP 9: **VERY IMPORTANT**
**MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work. **MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work.
- `include /etc/nginx/conf.d/*` - `include /etc/nginx/conf.d/*`
## STEP 7: **VERY IMPORTANT** ## STEP 10: **VERY IMPORTANT**
**ADD INCLUDE FILES INTO A VHOST** **ADD INCLUDE FILES INTO A VHOST**
@ -164,7 +209,7 @@ These includes MUST be added within a **server {}** block of a vhost otherwise y
- `include /etc/nginx/bots.d/ddos.conf;` - `include /etc/nginx/bots.d/ddos.conf;`
## STEP 8: ## STEP 11:
**TESTING YOUR NGINX CONFIGURATION** **TESTING YOUR NGINX CONFIGURATION**
@ -176,7 +221,7 @@ If you get no errors then you followed my instructions so now you can make the b
The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working. The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working.
## STEP 9: ## STEP 12:
**TESTING** **TESTING**
@ -201,7 +246,7 @@ Should respond with: curl: (52) Empty reply from server
The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!! The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!!
## STEP 10: ## STEP 13:
**UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names. **UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names.

2
Engintron_for_cPanel_WHM_Configuration_Example/README.md
View file

@ -23,7 +23,7 @@
- This example contains a modified file in **/utilities/https_vhosts.php** which will do the includes for you on any SSL vhosts you create. - This example contains a modified file in **/utilities/https_vhosts.php** which will do the includes for you on any SSL vhosts you create.
### Version 2.2017.05 ### Version 2.2017.07
##### Created by: https://github.com/mitchellkrogza ##### Created by: https://github.com/mitchellkrogza
##### Copyright Mitchell Krog <mitchellkrog@gmail.com> ##### Copyright Mitchell Krog <mitchellkrog@gmail.com>

67
README.md
View file

@ -10,7 +10,7 @@
# Nginx Bad Bot and User-Agent Blocker, Spam Referrer Blocker, Anti DDOS, Bad IP Blocker and Wordpress Theme Detector Blocker # Nginx Bad Bot and User-Agent Blocker, Spam Referrer Blocker, Anti DDOS, Bad IP Blocker and Wordpress Theme Detector Blocker
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking
### Version 2.2017.05 ### Version 2.2017.07
##### Created by: https://github.com/mitchellkrogza ##### Created by: https://github.com/mitchellkrogza
##### Copyright Mitchell Krog <mitchellkrog@gmail.com> ##### Copyright Mitchell Krog <mitchellkrog@gmail.com>
@ -119,7 +119,7 @@ For bot's or spiders that you still want to allow but want to limit their visita
# CONFIGURATION INSTRUCTIONS # CONFIGURATION INSTRUCTIONS
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
**If you miss one step you will get an nginx EMERG error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf or blacklist-user-agents.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.** **If you miss one step you will get an nginx EMERG error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf or blacklist-user-agents.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.**
@ -205,8 +205,7 @@ Whitelist all your own domain names and IP addresses. **Please note important ch
Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script. Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script.
When pulling any future updates now your domains and IP whitelists will not be overwritten.
When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses.
## STEP 4: ## STEP 4:
@ -216,16 +215,62 @@ Copy the custom User-Agents blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d` `cd /etc/nginx/bots.d`
- copy the blacklist-user-agents.conf file into the same folder - copy the blacklist-user-agents.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf`
Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block. Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block.
When pulling any future updates now your custom User-Agents blacklist will not be overwritten.
## STEP 5: ## STEP 5:
**BLACKLIST USING YOUR OWN CUSTOM BAD REFERRERS**
Copy the custom bad referrers blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the custom-bad-referrers.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/custom-bad-referrers.conf -O custom-bad-referrers.conf`
Use nano, vim or any other text editor to edit (if needed) custom-bad-referrers.conf to include your own custom list of bad referrer domains that are not included in the blocker.
When pulling any future updates now your custom referrers list will not be overwritten.
## STEP 6:
**BLACKLIST IPS AND IP RANGES USING YOUR OWN CUSTOM LIST**
Copy the custom IP blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the blacklist-ips.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-ips.conf -O blacklist-ips.conf`
Use nano, vim or any other text editor to edit (if needed) blacklist-ips.conf to include your own custom list of IP Addresses and IP Ranges that you wish to block.
When pulling any future updates now your custom IP blacklist will not be overwritten.
## STEP 7:
**DOWNLOAD CUSTOM BAD REFERRER WORDS INCLUDE FILE FOR CUSTOMIZED SCANNING OF BAD WORDS**
Copy the custom bad referrer words include file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the bad-referrer-words.conf file from the repository
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/bad-referrer-words.conf -O bad-referrer-words.conf`
Use nano, vim or any other text editor to edit the bad-referrer-words.conf file as you like.
When pulling any future updates now your custom bad referrer words list will not be overwritten.
## STEP 8:
**INCLUDE IMPORTANT SETTINGS IN NGINX.CONF** **INCLUDE IMPORTANT SETTINGS IN NGINX.CONF**
**Also see SAMPLE-nginx.conf file in the root of this repository** **Also see SAMPLE-nginx.conf file in the root of this repository**
@ -251,14 +296,14 @@ The important settings file above adds the rate limiting functions and hash_buck
The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements. The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements.
## STEP 6: **VERY IMPORTANT** ## STEP 9: **VERY IMPORTANT**
**MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work. **MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work.
- `include /etc/nginx/conf.d/*` - `include /etc/nginx/conf.d/*`
## STEP 7: **VERY IMPORTANT** ## STEP 10: **VERY IMPORTANT**
**ADD INCLUDE FILES INTO A VHOST** **ADD INCLUDE FILES INTO A VHOST**
@ -272,7 +317,7 @@ These includes MUST be added within a **server {}** block of a vhost otherwise y
- `include /etc/nginx/bots.d/ddos.conf;` - `include /etc/nginx/bots.d/ddos.conf;`
## STEP 8: ## STEP 11:
**TESTING YOUR NGINX CONFIGURATION** **TESTING YOUR NGINX CONFIGURATION**
@ -284,7 +329,7 @@ If you get no errors then you followed my instructions so now you can make the b
The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working. The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working.
## STEP 9: ## STEP 12:
**TESTING** **TESTING**
@ -309,7 +354,7 @@ Should respond with: curl: (52) Empty reply from server
The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!! The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!!
## STEP 10: ## STEP 13:
**UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names. **UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names.

4
install-ngxblocker
View file

@ -2,6 +2,7 @@
# Bash Script for Installing the Nginx Bad Bot Blocker # Bash Script for Installing the Nginx Bad Bot Blocker
# Copyright - https://github.com/mitchellkrogza # Copyright - https://github.com/mitchellkrogza
# Project Url: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker # Project Url: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker
# Version 2.2017.07
# PLEASE READ CONFIGURATION INSTRUCTIONS BEFORE USING THIS - THIS IS ONLY A PARTIAL INSTALLER # PLEASE READ CONFIGURATION INSTRUCTIONS BEFORE USING THIS - THIS IS ONLY A PARTIAL INSTALLER
# FOR COPYING THE FILES CORRECTLY TO NGINX INTO THE CORRECT FOLDERS # FOR COPYING THE FILES CORRECTLY TO NGINX INTO THE CORRECT FOLDERS
@ -35,6 +36,9 @@ sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bo
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf -O whitelist-ips.conf sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf -O whitelist-ips.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf -O whitelist-domains.conf sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf -O whitelist-domains.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-ips.conf -O blacklist-ips.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/bad-referrer-words.conf -O bad-referrer-words.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/custom-bad-referrers.conf -O custom-bad-referrers.conf
exit 0 exit 0
# PLEASE READ CONFIGURATION INSTRUCTIONS # PLEASE READ CONFIGURATION INSTRUCTIONS