From 8eca90d4b3ec003ec18470dca39d90bb9aca7c42 Mon Sep 17 00:00:00 2001 From: Mitchell Krog Date: Tue, 11 Apr 2017 14:19:16 +0200 Subject: [PATCH] Blocker Update Notifications Notification System for this repo introduced today. You can subscribe to notifications at: **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** or send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googlegroups.com** Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. --- CHANGELOG.md | 30 +++++++++++++++++------------ CONFIGURATION.md | 5 +++++ Fail2Ban/README.md | 7 ++++++- Pull_Requests_Here_Please/README.md | 7 ++++++- README.md | 5 +++++ robots.txt/README.md | 7 ++++++- 6 files changed, 46 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0454f5b74..787f7f4b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,12 @@ -#CHANGELOG - Nginx Bad Bot Blocker +# CHANGELOG - Nginx Bad Bot Blocker -###2017-03-02 +### 2017-04-11 +- Introduced Repo Email Notification Mailing List +- Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** +or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. +Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. + +### 2017-03-02 - V2.2017.05 - Added New Include Method for Including your own custom list of Bad User-Agents or Bots that you want to permanently block. - **You must now have an (/etc/nginx/bots.d/blacklist-user-agents.conf) file or you will get and Nginx EMERG error** @@ -8,41 +14,41 @@ - Also introduced new include file (/etc/nginx/conf.d/botblocker-nginx-settings.conf) for including the rate limiting zones and server_hash settings for you in your nginx.conf file **[Please read updated configuration instructions](https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/CONFIGURATION.md)** -###2017-02-16 +### 2017-02-16 - V2.2017.04 - Added new include file methods for whitelisting your own domains and IP addresses. No more having to remember to include your own IP ranges with any future update. **[Please read updated configuration instructions](https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/CONFIGURATION.md)** -###2017-02-08 +### 2017-02-08 - V2.2017.03 - Added Travis CI Build Checking and Various Readme File Updates -###2017-02-06 +### 2017-02-06 - V2.2017.02 - Added Whitelisting of Cloudflare IP Ranges -###2017-01-29 +### 2017-01-29 - Changed formatting of globalblacklist.conf, some sections like semalt and miraibot merged into other sections for easier maintenance. - Additional notes, testing instructions and commenting added in globalblacklist.conf file -###2016-12-17 +### 2016-12-17 - Removed "CPython" from bad bots list. This user agent string "python-requests/2.5.3 CPython/2.7.9 Linux/3.16.0-4-amd64" is used by a valid Google Feed Parser called "UniversalFeedParser/5.2.1 +https://code.google.com/p/feedparser/" -###2016-12-14 +### 2016-12-14 - Removed "python-requests" from bad bots list. This user agent string "python-requests/2.5.3 CPython/2.7.9 Linux/3.16.0-4-amd64" is used by a valid Google Feed Parser called "UniversalFeedParser/5.2.1 +https://code.google.com/p/feedparser/" -###2016-12-11 +### 2016-12-11 - Added some repetitively bad IP's - Added extensive blocking for builtwith.com SEO analysis company that scrapes, accumulates and sells SEO web data. Have blocked all their domains and IP's including other domains linked to the owner. Also spent hours digging for IP's linked to this site that were very nicely hidden but through some research I uncovered them. www.builtwith.com DOES obey removal instructions at https://builtwith.com/removals and it is immediate BUT for those who have never heard of builtwith.com or even know their sites are indexed and new sites are being indexed and used by your competitors to outrank you, I have sufficient blocks in place to prevent them ever accessing your Nginx/Apache server. I suggest you FIRST see if your site is indexed, then request removal of each site which requires you to place a simple .html file in your webroot and once that is done, then update to the latest version of the script which will block them indexing new sites from thereon out. I will keep a watch on them and add new IP's as they change. - In progress on growing the BAD IP block list which will be based off repetetive 444 errors from the bad referer domains in the blocker. These bad IP's will be auto added into the blocker and I will generate plain text IP lists and IP tables rules too which can be updated frequently to block them at firewall level and keep your web logs even cleaner. -###2016-12-04 +### 2016-12-04 - Added creation of a Google Disavow File - google-disavow.txt - New Bad Referers Added -###2016-12-03 +### 2016-12-03 - Added Over 205 New Bad Referer Domains - Added google-exclude.txt file for stopping Ghost Spam on your Google Analytics - Readme Updated with Instructions on using google-exclude.txt -###2016-12-02 +### 2016-12-02 - Added Block List for Nibble SEO - Added Block List for Wordpress Theme Detectors diff --git a/CONFIGURATION.md b/CONFIGURATION.md index ce4e0329e..2ad40d5ba 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -5,6 +5,11 @@ ##### Copyright Mitchell Krog ### Version 2.2017.05 +## Update Notification System +Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** +or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. +Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. + # CONFIGURATION OF THE NGINX BAD BOT BLOCKER: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY diff --git a/Fail2Ban/README.md b/Fail2Ban/README.md index 50fc01adc..2fea9f9c7 100644 --- a/Fail2Ban/README.md +++ b/Fail2Ban/README.md @@ -59,4 +59,9 @@ This works great at picking up wp-login attacks. Be sure to add the rate limiting zone to your nginx.conf as per instructions in /filter.d/nginx-limit-req.local -### If this helped you [You can buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=BKF9XT6WHATLG):beer: \ No newline at end of file +### If this helped you [You can buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=BKF9XT6WHATLG):beer: + +## Update Notification System +Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** +or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. +Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. \ No newline at end of file diff --git a/Pull_Requests_Here_Please/README.md b/Pull_Requests_Here_Please/README.md index 314c6a9c2..eac628237 100644 --- a/Pull_Requests_Here_Please/README.md +++ b/Pull_Requests_Here_Please/README.md @@ -7,4 +7,9 @@ as well as the Apache Bad Bot and Spam Referrer Blocker which use this same cent Thank you for contributing to help produce the most extensive bad bot, user-agent and spam referrer blocker. -### If this helped you [You can buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=BKF9XT6WHATLG):beer: \ No newline at end of file +### If this helped you [You can buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=BKF9XT6WHATLG):beer: + +## Update Notification System +Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** +or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. +Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. \ No newline at end of file diff --git a/README.md b/README.md index 7817a082b..25d6bb100 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,11 @@ - Includes the creation of a google-exclude.txt file for creating filters / segments in Google Analytics (see instructions lower down) - Includes the creation of a google-disavow.txt file for use in Google Webmaster Tools (see instructions lower down) +## Update Notification System +Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** +or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. +Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. + ### WHY BLOCK BAD BOTS ? ##### Bad bots are: diff --git a/robots.txt/README.md b/robots.txt/README.md index d93a35f7b..8f1498576 100644 --- a/robots.txt/README.md +++ b/robots.txt/README.md @@ -32,4 +32,9 @@ This will only help stop some bad bots from gaining access to your site this certainly does not provide nearly as much protection as the full Nginx Bad Bot, User-Agent and Spam Referrer Blocker -### If this helped you [You can buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=BKF9XT6WHATLG):beer: \ No newline at end of file +### If this helped you [You can buy me a beer](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=BKF9XT6WHATLG):beer: + +## Update Notification System +Please subscribe your email address to the mailing list at **https://groups.google.com/forum/#!forum/nginx-ultimate-bad-bot-blocker** +or simply send a blank email to **nginx-ultimate-bad-bot-blocker+subscribe@googlegroups.com** to subscribe. +Please make sure you are subscribed to notifications to be notified when the blocker is updated and also to be notified when any important or mission critical changes take place. \ No newline at end of file