vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V2.2017.05 - Major Version Update

Browse source 
- PLEASE READ UPDATED CONFIGURATION INSTRUCTIONS
- PLEASE READ CHANGELOG
- New Custom Include File (/etc/nginx/bots.d/blacklist-user-agents.conf ) for Blacklisting your own User-Agents which will not get wiped out with updates
- New include file (/etc/nginx/conf.d/botblocker-nginx-settings.conf) for having the important nging settings automatically included into your nginx.conf for you.
- New Bash Installer Script for Easy Installation and Copying Files Directly from the Repo
- Important to note changes in this Version Update as you will get EMERG errors if you are missing any of the new include files
This commit is contained in:
Mitchell Krog 2017-03-02 12:41:31 +02:00
parent 4a97891513
commit f19f327dcb
12 changed files with 202 additions and 62 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
.travis.yml
View file

@ -44,6 +44,12 @@ deploy:
secure: ${GH_TOKEN} secure: ${GH_TOKEN}
file: file:
- "conf.d/globalblacklist.conf" - "conf.d/globalblacklist.conf"
- "conf.d/botblocker-nginx-settings.conf"
- "bots.d/ddos.conf"
- "bots.d/blockbots.conf"
- "bots.d/whitelist-ips.conf"
- "bots.d/whitelist-domains.conf"
- "bots.d/blacklist-user-agents.conf"
skip_cleanup: true skip_cleanup: true
on: on:
repo: mitchellkrogza/nginx-ultimate-bad-bot-blocker repo: mitchellkrogza/nginx-ultimate-bad-bot-blocker

8
CHANGELOG.md
View file

@ -1,5 +1,13 @@
#CHANGELOG - Nginx Bad Bot Blocker #CHANGELOG - Nginx Bad Bot Blocker
###2017-03-02
- V2.2017.05 - Added New Include Method for Including your own custom list of Bad User-Agents
or Bots that you want to permanently block.
- **You must now have an /etc/nginx/bots.d/blacklist-user-agents.conf file or you will get and Nginx EMERG error**
- Introduced New Bash Installer Script to Ease Installation of this Bad Bot Blocker
- Also introduced new include file (botblocker-nginx-settings.conf) for including the rate limiting zones and server_hash settings for you in your nginx.conf file
**[Please read updated configuration instructions](https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/CONFIGURATION.md)**
###2017-02-16 ###2017-02-16
- V2.2017.04 - Added new include file methods for whitelisting your own domains and IP addresses. - V2.2017.04 - Added new include file methods for whitelisting your own domains and IP addresses.
No more having to remember to include your own IP ranges with any future update. No more having to remember to include your own IP ranges with any future update.

86
CONFIGURATION.md
View file

@ -3,11 +3,21 @@
##### Created by: https://github.com/mitchellkrogza ##### Created by: https://github.com/mitchellkrogza
##### Copyright Mitchell Krog <mitchellkrog@gmail.com> ##### Copyright Mitchell Krog <mitchellkrog@gmail.com>
### Version 2.2017.04 ### Version 2.2017.05
#CONFIGURATION OF THE NGINX BAD BOT BLOCKER: #CONFIGURATION OF THE NGINX BAD BOT BLOCKER:
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY
**If you miss one step you will get an nginx EMERG error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf or blacklist-user-agents.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.**
#### AUTO INSTALLATION INSTRUCTIONS
To Make Sure you copy all the correct files you can now use a simple bash setup script for copying the files into the correct nginx folders for you:
See: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/installnginxblocker.sh
**Please Note:** the bash installer script does not carry out STEP 7 of the manual configuration instructions for you. YOU MUST edit your any vhosts files yourself and manually add the entries in STEP 7 or the blocker will not actually be protecting any sites.
## MANUAL INSTALLATION INSTRUCTIONS
##STEP 1: ##STEP 1:
**COPY THE GLOBALBLACKLIST.CONF FILE FROM THE REPO** **COPY THE GLOBALBLACKLIST.CONF FILE FROM THE REPO**
@ -16,7 +26,7 @@ Copy the contents of **/conf.d/globalblacklist.conf** into your /etc/nginx/conf.
`cd /etc/nginx/conf.d` `cd /etc/nginx/conf.d`
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf -O globalblacklist.conf`
##STEP 2: ##STEP 2:
@ -30,12 +40,12 @@ Copy the contents of **/conf.d/globalblacklist.conf** into your /etc/nginx/conf.
- copy the blockbots.conf file into that folder - copy the blockbots.conf file into that folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blockbots.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blockbots.conf -O blockbots.conf`
- copy the ddos.conf file into the same folder - copy the ddos.conf file into the same folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/ddos.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/ddos.conf -O ddos.conf`
##STEP 3: ##STEP 3:
@ -47,46 +57,66 @@ Whitelist all your own domain names and IP addresses. **Please note important ch
- copy the whitelist-ips.conf file into that folder - copy the whitelist-ips.conf file into that folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf -O whitelist-ips.conf`
- copy the whitelist-domains.conf file into the same folder - copy the whitelist-domains.conf file into the same folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf -O whitelist-domains.conf`
Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script. Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script.
When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses. When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses.
##STEP 4: ##STEP 4:
**BLACKLIST USING YOUR OWN CUSTOM USER-AGENT BLACKLIST**
Copy the custom User-Agents blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the blacklist-user-agents.conf file into the same folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf`
Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block.
##STEP 5:
**INCLUDE IMPORTANT SETTINGS IN NGINX.CONF** **INCLUDE IMPORTANT SETTINGS IN NGINX.CONF**
**Also see SAMPLE-nginx.conf file in the root of this repository**
- From your linux command line type `cd /etc/nginx/conf.d`
- `sudo nano /etc/nginx/nginx.conf` - copy the botblocker-nginx-settings.conf file directly from the repo
#####Add the following settings and rate limiting zones near the top of your nginx.conf file. This is both for the Anti DDOS rate limiting filter and for allowing Nginx to load this very large set of domain names into memory. `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/botblocker-nginx-settings.conf -O botblocker-nginx-settings.conf`
**see SAMPLE-nginx.conf file in the root of this repository**
- `server_names_hash_bucket_size 64;` **What is included in this settings file above for nginx?**
The important settings file above adds the rate limiting functions and hash_bucket settings for nginx for you. Below is what the file contains, you cn add these manually to your nginx.conf file if you so please but the include file above will do it for you ad nginx loads any .conf file in /etc/conf.d (See STEP 6)
- `server_names_hash_max_size 4096;` > server_names_hash_bucket_size 64;
> server_names_hash_max_size 4096;
- `limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;` > limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
> limit_conn_zone $binary_remote_addr zone=addr:50m;
- `limit_conn_zone $binary_remote_addr zone=addr:50m;`
**Make sure** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it.
- `include /etc/nginx/conf.d/*`
**PLEASE NOTE:** The above rate limiting rules are for the DDOS filter, it may seem like high values to you but for wordpress sites with plugins and lots of images, it's not. This will not limit any real visitor to your Wordpress sites but it will immediately rate limit any aggressive bot. Remember that other bots and user agents are rate limited using a different rate limiting rule at the bottom of the globalblacklist.conf file. **PLEASE NOTE:** The above rate limiting rules are for the DDOS filter, it may seem like high values to you but for wordpress sites with plugins and lots of images, it's not. This will not limit any real visitor to your Wordpress sites but it will immediately rate limit any aggressive bot. Remember that other bots and user agents are rate limited using a different rate limiting rule at the bottom of the globalblacklist.conf file.
The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements.
##STEP 5:
##STEP 6: **VERY IMPORTANT**
**MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work.
- `include /etc/nginx/conf.d/*`
##STEP 7: **VERY IMPORTANT**
**ADD INCLUDE FILES INTO A VHOST** **ADD INCLUDE FILES INTO A VHOST**
@ -96,12 +126,11 @@ Open a site config file for Nginx (just one for now) and add the following lines
These includes MUST be added within a **server {}** block of a vhost otherwise you will get EMERG errors from Nginx. These includes MUST be added within a **server {}** block of a vhost otherwise you will get EMERG errors from Nginx.
- `include /etc/nginx/bots.d/blockbots.conf;` - `include /etc/nginx/bots.d/blockbots.conf;`
- `include /etc/nginx/bots.d/ddos.conf;` - `include /etc/nginx/bots.d/ddos.conf;`
##STEP 6: ##STEP 8:
**TESTING YOUR NGINX CONFIGURATION** **TESTING YOUR NGINX CONFIGURATION**
@ -113,7 +142,7 @@ If you get no errors then you followed my instructions so now you can make the b
The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working. The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working.
##STEP 7: ##STEP 9:
**TESTING** **TESTING**
@ -138,7 +167,7 @@ Should respond with: curl: (52) Empty reply from server
The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!! The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!!
##STEP 8: ##STEP 10:
**UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names. **UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names.
@ -160,10 +189,9 @@ See my latest auto updater bash script at:
https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/updatenginxblocker.sh https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/updatenginxblocker.sh
This can now be run as a daily cron to keep you up to date without having to remember to do it yourself.
Relax now and sleep better at night knowing your site is telling all those baddies they are FORBIDDEN !!! Relax now and sleep better at night knowing your site is telling all those baddies they are FORBIDDEN !!!
### PULL REQUESTS: ### PULL REQUESTS:
To contribute your own bad referers please add them into the https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/Pull%20Requests%20Here%20Please/badreferers.list file and then send a Pull Request (PR). To contribute your own bad referers please add them into the https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/Pull%20Requests%20Here%20Please/badreferers.list file and then send a Pull Request (PR).

83
README.md
View file

@ -10,7 +10,7 @@
# Nginx Bad Bot and User-Agent Blocker, Spam Referrer Blocker, Anti DDOS, Bad IP Blocker and Wordpress Theme Detector Blocker # Nginx Bad Bot and User-Agent Blocker, Spam Referrer Blocker, Anti DDOS, Bad IP Blocker and Wordpress Theme Detector Blocker
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking
### Version 2.2017.04 ### Version 2.2017.05
##### Created by: https://github.com/mitchellkrogza ##### Created by: https://github.com/mitchellkrogza
##### Copyright Mitchell Krog <mitchellkrog@gmail.com> ##### Copyright Mitchell Krog <mitchellkrog@gmail.com>
@ -116,6 +116,16 @@ For bot's or spiders that you still want to allow but want to limit their visita
#CONFIGURATION OF THE NGINX BAD BOT BLOCKER: #CONFIGURATION OF THE NGINX BAD BOT BLOCKER:
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY
**If you miss one step you will get an nginx EMERG error. This is normally a result of not downloading either blockbots.conf, ddos.conf, whitelist-ips.conf, whitelist-domains.conf or blacklist-user-agents.conf into your /etc/nginx/bots.d folder. If any of the include files are missing Nginx will EMERG and will not reload.**
#### AUTO INSTALLATION INSTRUCTIONS
To Make Sure you copy all the correct files you can now use a simple bash setup script for copying the files into the correct nginx folders for you:
See: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/installnginxblocker.sh
**Please Note:** the bash installer script does not carry out STEP 7 of the manual configuration instructions for you. YOU MUST edit your any vhosts files yourself and manually add the entries in STEP 7 or the blocker will not actually be protecting any sites.
## MANUAL INSTALLATION INSTRUCTIONS
##STEP 1: ##STEP 1:
**COPY THE GLOBALBLACKLIST.CONF FILE FROM THE REPO** **COPY THE GLOBALBLACKLIST.CONF FILE FROM THE REPO**
@ -124,7 +134,7 @@ Copy the contents of **/conf.d/globalblacklist.conf** into your /etc/nginx/conf.
`cd /etc/nginx/conf.d` `cd /etc/nginx/conf.d`
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf -O globalblacklist.conf`
##STEP 2: ##STEP 2:
@ -138,12 +148,12 @@ Copy the contents of **/conf.d/globalblacklist.conf** into your /etc/nginx/conf.
- copy the blockbots.conf file into that folder - copy the blockbots.conf file into that folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blockbots.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blockbots.conf -O blockbots.conf`
- copy the ddos.conf file into the same folder - copy the ddos.conf file into the same folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/ddos.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/ddos.conf -O ddos.conf`
##STEP 3: ##STEP 3:
@ -155,46 +165,66 @@ Whitelist all your own domain names and IP addresses. **Please note important ch
- copy the whitelist-ips.conf file into that folder - copy the whitelist-ips.conf file into that folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf -O whitelist-ips.conf`
- copy the whitelist-domains.conf file into the same folder - copy the whitelist-domains.conf file into the same folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf` `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf -O whitelist-domains.conf`
Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script. Use nano, vim or any other text editor to edit both whitelist-ips.conf and whitelist-domains.conf to include all your own domain names and IP addresses that you want to specifically whitelist from the blocker script.
When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses. When pulling any future updates now you can simply pull the latest globalblacklist.conf file and it will automatically include your whitelisted domains and IP addresses.
##STEP 4: ##STEP 4:
**BLACKLIST USING YOUR OWN CUSTOM USER-AGENT BLACKLIST**
Copy the custom User-Agents blacklist file into your /etc/nginx/bots.d folder
`cd /etc/nginx/bots.d`
- copy the blacklist-user-agents.conf file into the same folder
`sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf`
Use nano, vim or any other text editor to edit (if needed) blacklist-user-agents.conf to include your own custom list of bad agents that are not included in the blocker like "omgilibot" which some people choose to block.
##STEP 5:
**INCLUDE IMPORTANT SETTINGS IN NGINX.CONF** **INCLUDE IMPORTANT SETTINGS IN NGINX.CONF**
**Also see SAMPLE-nginx.conf file in the root of this repository**
- From your linux command line type `cd /etc/nginx/conf.d`
- `sudo nano /etc/nginx/nginx.conf` - copy the botblocker-nginx-settings.conf file directly from the repo
#####Add the following settings and rate limiting zones near the top of your nginx.conf file. This is both for the Anti DDOS rate limiting filter and for allowing Nginx to load this very large set of domain names into memory. `sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/botblocker-nginx-settings.conf -O botblocker-nginx-settings.conf`
**see SAMPLE-nginx.conf file in the root of this repository**
- `server_names_hash_bucket_size 64;` **What is included in this settings file above for nginx?**
The important settings file above adds the rate limiting functions and hash_bucket settings for nginx for you. Below is what the file contains, you cn add these manually to your nginx.conf file if you so please but the include file above will do it for you ad nginx loads any .conf file in /etc/conf.d (See STEP 6)
- `server_names_hash_max_size 4096;` > server_names_hash_bucket_size 64;
> server_names_hash_max_size 4096;
- `limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;` > limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
> limit_conn_zone $binary_remote_addr zone=addr:50m;
- `limit_conn_zone $binary_remote_addr zone=addr:50m;`
**Make sure** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it.
- `include /etc/nginx/conf.d/*`
**PLEASE NOTE:** The above rate limiting rules are for the DDOS filter, it may seem like high values to you but for wordpress sites with plugins and lots of images, it's not. This will not limit any real visitor to your Wordpress sites but it will immediately rate limit any aggressive bot. Remember that other bots and user agents are rate limited using a different rate limiting rule at the bottom of the globalblacklist.conf file. **PLEASE NOTE:** The above rate limiting rules are for the DDOS filter, it may seem like high values to you but for wordpress sites with plugins and lots of images, it's not. This will not limit any real visitor to your Wordpress sites but it will immediately rate limit any aggressive bot. Remember that other bots and user agents are rate limited using a different rate limiting rule at the bottom of the globalblacklist.conf file.
The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. The server_names_hash settings allows Nginx Server to load this very large list of domain names and IP addresses into memory. You can tweak these settings to your own requirements.
##STEP 5:
##STEP 6: **VERY IMPORTANT**
**MAKE SURE** that your nginx.conf file contains the following include directive. If it's commented out make sure to uncomment it or none of this will work.
- `include /etc/nginx/conf.d/*`
##STEP 7: **VERY IMPORTANT**
**ADD INCLUDE FILES INTO A VHOST** **ADD INCLUDE FILES INTO A VHOST**
@ -208,7 +238,7 @@ These includes MUST be added within a **server {}** block of a vhost otherwise y
- `include /etc/nginx/bots.d/ddos.conf;` - `include /etc/nginx/bots.d/ddos.conf;`
##STEP 6: ##STEP 8:
**TESTING YOUR NGINX CONFIGURATION** **TESTING YOUR NGINX CONFIGURATION**
@ -220,7 +250,7 @@ If you get no errors then you followed my instructions so now you can make the b
The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working. The blocker is now active and working so now you can run some simple tests from another linux machine to make sure it's working.
##STEP 7: ##STEP 9:
**TESTING** **TESTING**
@ -245,7 +275,7 @@ Should respond with: curl: (52) Empty reply from server
The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!! The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!!
##STEP 8: ##STEP 10:
**UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names. **UPDATING THE NGINX BAD BOT BLOCKER** is now easy thanks to the automatic includes for whitelisting your own domain names.
@ -269,6 +299,7 @@ https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/
Relax now and sleep better at night knowing your site is telling all those baddies they are FORBIDDEN !!! Relax now and sleep better at night knowing your site is telling all those baddies they are FORBIDDEN !!!
### PULL REQUESTS: ### PULL REQUESTS:
To contribute your own bad referers please add them into the https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/Pull%20Requests%20Here%20Please/badreferers.list file and then send a Pull Request (PR). To contribute your own bad referers please add them into the https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/Pull%20Requests%20Here%20Please/badreferers.list file and then send a Pull Request (PR).

6
bots.d/blacklist-user-agents.conf Normal file
View file

@ -0,0 +1,6 @@
# Add One Entry Per Line - List all the extra bad User-Agents you want to permanently block
# This is for User-Agents that are not included in the main list of the bot blocker
# This allows you finer control of keeping certain bots blocked and automatic updates will
# Never be able to remove this custom list of yours
"~*mybaduseragentname" 3;

4
conf.d/botblocker-nginx-settings.conf Normal file
View file

@ -0,0 +1,4 @@
server_names_hash_bucket_size 64;
server_names_hash_max_size 4096;
limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
limit_conn_zone $binary_remote_addr zone=addr:50m;

14
conf.d/globalblacklist.conf
View file

@ -2,7 +2,7 @@
### THE ULTIMATE NGINX BAD BOT BLOCKER ### THE ULTIMATE NGINX BAD BOT BLOCKER
### ********************************** ### **********************************
### Version 2.2017.04 ### Version 2.2017.05
### This file implements a checklist / blacklist for good user agents, bad user agents and ### This file implements a checklist / blacklist for good user agents, bad user agents and
### bad referrers. It also has whitelisting for your own IP's and known good IP Ranges ### bad referrers. It also has whitelisting for your own IP's and known good IP Ranges
@ -17,11 +17,11 @@
### - https://github.com/oohnoitz/nginx-blacklist ### - https://github.com/oohnoitz/nginx-blacklist
### Last Updated ### Last Updated
### Thu Mar 2 10:24:06 SAST 2017 ### Thu Mar 2 12:27:06 SAST 2017
### End Last Updated ### End Last Updated
### Generated in ### Generated in
### 0.314051389694 seconds ### 0.217051389694 seconds
### End Generated in ### End Generated in
### Tested on: nginx/1.10.0 (Ubuntu 16.04) ### Tested on: nginx/1.10.0 (Ubuntu 16.04)
@ -202,6 +202,14 @@ map $http_user_agent $bad_bot {
# This includes: # This includes:
# Known Vulnerability Scanners (now merged into one section) # Known Vulnerability Scanners (now merged into one section)
# ***********************************************
# Include your Own Custom List of Bad User Agents
# ***********************************************
# START BLACKLISTED USER-AGENTS / BOTS ### DO NOT EDIT THIS LINE AT ALL ###
include /etc/nginx/bots.d/blacklist-user-agents.conf;
# END BLACKLISTED USER-AGENTS / BOTS ### DO NOT EDIT THIS LINE AT ALL ###
# START BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ### # START BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
"~*360Spider" 3; "~*360Spider" 3;
"~*80legs" 3; "~*80legs" 3;

36
installnginxblocker.sh Executable file
View file

@ -0,0 +1,36 @@
#!/bin/bash
# Bash Script for Installing the Nginx Bad Bot Blocker
# Copyright - https://github.com/mitchellkrogza
# Project Url: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker
# PLEASE READ CONFIGURATION INSTRUCTIONS BEFORE USING THIS - THIS IS ONLY A PARTIAL INSTALLER
# FOR COPYING THE FILES CORRECTLY TO NGINX INTO THE CORRECT FOLDERS
# https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/CONFIGURATION.md
# Use this script only once and thereafter use the Auto Update Bash Script updatenginxblocker.sh
# THIS INSTALL SCRIPT **ONLY** COPIES THE NECESSARY FILES FOR NGINX DIRECT FROM THE REPO
### The installer script does not carry out STEP 6 of the configuration instructions for you.
### You must manually edit any vhost files with the includes in STEP 6 or it will not actually be protecting any sites.
### READ: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/CONFIGURATION.md
# Save this file as /bin/installnginxblocker.sh
# Make it Executable chmod +x /bin/installnginxblocker.sh
# Run it from the command line using sudo /bin/installnginxblocker.sh
# LETS INSTALL NOW
cd /etc/nginx/conf.d
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf -O globalblacklist.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/botblocker-nginx-settings.conf -O botblocker-nginx-settings.conf
sudo mkdir /etc/nginx/bots.d
cd /etc/nginx/bots.d
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blockbots.conf -O blockbots.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/ddos.conf -O ddos.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf -O whitelist-ips.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf -O whitelist-domains.conf
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O blacklist-user-agents.conf
exit 0
# PLEASE READ CONFIGURATION INSTRUCTIONS
# https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/CONFIGURATION.md

6
travisCI/blacklist-user-agents.tpl.conf Normal file
View file

@ -0,0 +1,6 @@
# Add One Entry Per Line - List all the extra bad User-Agents you want to permanently block
# This is for User-Agents that are not included in the main list of the bot blocker
# This allows you finer control of keeping certain bots blocked and automatic updates will
# Never be able to remove this custom list of yours
"~*mybaduseragentname" 3;

4
travisCI/botblocker-nginx-settings.tpl.conf Normal file
View file

@ -0,0 +1,4 @@
server_names_hash_bucket_size 64;
server_names_hash_max_size 4096;
limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
limit_conn_zone $binary_remote_addr zone=addr:50m;

2
travisCI/install-nginx.sh
View file

@ -44,6 +44,8 @@ tpl "$DIR/ddos.tpl.conf" "$DIR/nginx/ddos.conf"
tpl "$DIR/blockbots.tpl.conf" "$DIR/nginx/blockbots.conf" tpl "$DIR/blockbots.tpl.conf" "$DIR/nginx/blockbots.conf"
tpl "$DIR/whitelist-ips.tpl.conf" "$DIR/nginx/bots.d/whitelist-ips.conf" tpl "$DIR/whitelist-ips.tpl.conf" "$DIR/nginx/bots.d/whitelist-ips.conf"
tpl "$DIR/whitelist-domains.tpl.conf" "$DIR/nginx/bots.d/whitelist-domains.conf" tpl "$DIR/whitelist-domains.tpl.conf" "$DIR/nginx/bots.d/whitelist-domains.conf"
tpl "$DIR/blacklist-user-agents.tpl.conf" "$DIR/nginx/bots.d/blacklist-user-agents.conf"
tpl "$DIR/botblocker-nginx-settings.tpl.conf" "$DIR/nginx/botblocker-nginx-settings.conf"
tpl "$DIR/globalblacklist.tpl.conf" "$DIR/nginx/globalblacklist.conf" tpl "$DIR/globalblacklist.tpl.conf" "$DIR/nginx/globalblacklist.conf"
tpl "$DIR/default-site.tpl.conf" "$DIR/nginx/sites-enabled/default-site.conf" tpl "$DIR/default-site.tpl.conf" "$DIR/nginx/sites-enabled/default-site.conf"

9
travisCI/nginx.tpl.conf
View file

@ -24,8 +24,8 @@ http {
tcp_nodelay on; tcp_nodelay on;
keepalive_timeout 65; keepalive_timeout 65;
types_hash_max_size 2048; types_hash_max_size 2048;
server_names_hash_bucket_size 64; #server_names_hash_bucket_size 64;
server_names_hash_max_size 4096; #server_names_hash_max_size 4096;
# Our request limiter zone for wp-login attacks # Our request limiter zone for wp-login attacks
limit_req_zone $binary_remote_addr zone=wp-login:10m rate=1r/s; limit_req_zone $binary_remote_addr zone=wp-login:10m rate=1r/s;
@ -34,9 +34,9 @@ http {
# *************** # ***************
# https://www.nginx.com/blog/mitigating-ddos-attacks-with-nginx-and-nginx-plus/ # https://www.nginx.com/blog/mitigating-ddos-attacks-with-nginx-and-nginx-plus/
# Limiting the Rate of Requests # Limiting the Rate of Requests
limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s; #limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
# Limiting the Number of Connections # Limiting the Number of Connections
limit_conn_zone $binary_remote_addr zone=addr:50m; #limit_conn_zone $binary_remote_addr zone=addr:50m;
include /etc/nginx/mime.types; include /etc/nginx/mime.types;
default_type application/octet-stream; default_type application/octet-stream;
@ -58,6 +58,7 @@ http {
## ##
include {DIR}/nginx/conf.d/*.conf; include {DIR}/nginx/conf.d/*.conf;
include {DIR}/nginx/sites-enabled/*; include {DIR}/nginx/sites-enabled/*;
include {DIR}/nginx/botblocker-nginx-settings.conf;
include {DIR}/nginx/globalblacklist.conf; include {DIR}/nginx/globalblacklist.conf;
upstream php { upstream php {