mirror of
https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git
synced 2025-09-09 13:55:41 +00:00
V4.2019.06.1645 [ci skip]
This commit is contained in:
Travis
parent
b3601d427e
commit
8ff3b4c1fa
34 changed files with 2737 additions and 2377 deletions
|
|
@ -33,4 +33,4 @@
|
|||
# 111.111.111.111 0;
|
||||
|
||||
|
||||
35.193.7.13 0;
|
||||
35.202.145.110 0;
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@
|
|||
|
||||
### VERSION INFORMATION #
|
||||
###################################################
|
||||
### Version: V4.2019.06.1644
|
||||
### Updated: Thu Jun 27 13:12:21 SAST 2019
|
||||
### Version: V4.2019.06.1645
|
||||
### Updated: Thu Jun 27 13:24:55 SAST 2019
|
||||
### Bad Referrer Count: 6713
|
||||
### Bad Bot Count: 556
|
||||
###################################################
|
||||
|
|
|
|||
|
|
@ -33,4 +33,4 @@
|
|||
# 111.111.111.111 0;
|
||||
|
||||
|
||||
35.193.7.13 0;
|
||||
35.202.145.110 0;
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@
|
|||
|
||||
### VERSION INFORMATION #
|
||||
###################################################
|
||||
### Version: V3.2019.06.1644
|
||||
### Updated: Thu Jun 27 13:12:23 SAST 2019
|
||||
### Version: V3.2019.06.1645
|
||||
### Updated: Thu Jun 27 13:24:56 SAST 2019
|
||||
### Bad Referrer Count: 6713
|
||||
### Bad Bot Count: 556
|
||||
###################################################
|
||||
|
|
|
|||
|
|
@ -1,2 +1,2 @@
|
|||
35.192.85.2 1;
|
||||
104.154.120.187 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
35.192.85.2 0;
|
||||
104.154.120.187 0;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
|
|
@ -1,16 +1,21 @@
|
|||
### **********************************
|
||||
### THE ULTIMATE NGINX BAD BOT BLOCKER
|
||||
### **********************************
|
||||
### -----------------------------------------------------------
|
||||
### THE NGINX ULTIMATE BAD BOT, BAD IP AND BAD REFERRER BLOCKER
|
||||
### -----------------------------------------------------------
|
||||
|
||||
### VERSION INFORMATION #
|
||||
###################################################
|
||||
### Version: V4.2019.06.1643
|
||||
### Updated: Thu Jun 27 12:59:42 SAST 2019
|
||||
### Version: V3.2019.06.1645
|
||||
### Updated: Thu Jun 27 13:24:56 SAST 2019
|
||||
### Bad Referrer Count: 6713
|
||||
### Bad Bot Count: 556
|
||||
###################################################
|
||||
### VERSION INFORMATION ##
|
||||
|
||||
### --------------------------------------------
|
||||
### HELP SUPPORT THIS PROJECT - Send Me a Coffee
|
||||
### https://ko-fi.com/mitchellkrog
|
||||
### --------------------------------------------
|
||||
|
||||
##############################################################################
|
||||
# _ __ _ #
|
||||
# / |/ /__ _(_)__ __ __ #
|
||||
|
|
@ -28,12 +33,16 @@
|
|||
### and also has rate limiting functionality for bad bots who you only want to rate limit
|
||||
### and not actually block out entirely. It is very powerful and also very flexible.
|
||||
|
||||
### --------------------------------------------------------------------------
|
||||
### Created By: https://github.com/mitchellkrogza/
|
||||
### Repo Url: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker
|
||||
### Copyright Mitchell Krog - <mitchellkrog@gmail.com>
|
||||
### Contributors: Stuart Cardall - https://github.com/itoffshore
|
||||
### --------------------------------------------------------------------------
|
||||
|
||||
### --------------------------------------------------------------------------
|
||||
### Tested on: nginx/1.10.3 up to latest Mainstream Version (Ubuntu 16.04)
|
||||
### --------------------------------------------------------------------------
|
||||
|
||||
### This list was developed and is in use on a live Nginx server running some very busy web sites.
|
||||
### It was built from the ground up using real data from daily logs and is updated almost daily.
|
||||
|
|
@ -41,9 +50,9 @@
|
|||
### spam referrers, rogue IP address, scanners, scrapers and domain hijacking sites are extensively checked
|
||||
### before they are added. It is monitored extensively for any false positives.
|
||||
|
||||
### *********
|
||||
### ---------
|
||||
### Features:
|
||||
### *********
|
||||
### ---------
|
||||
### Clear formatting for Ease of Maintenance.
|
||||
### Alphabetically ordered lists for Ease of Maintenance.
|
||||
### Extensive Commenting for Ease of Reference.
|
||||
|
|
@ -54,40 +63,38 @@
|
|||
### IP range blocking / whitelisting.
|
||||
### Rate Limiting Functions.
|
||||
|
||||
### ************
|
||||
### ------------
|
||||
### INSTALLATION
|
||||
### ************
|
||||
### ------------
|
||||
|
||||
### PLEASE use the install, setup and update scripts provided for you to ease your installation.
|
||||
### This Auto Installation procedure is documented in the README.md and AUTO-CONFIGURATION.md files.
|
||||
### Installation, Setup and Update Scripts Contributed by Stuart Cardall - https://github.com/itoffshore
|
||||
### There are also manual configuration instructions provided for those not wishing to do an auto install.
|
||||
|
||||
### ***********************************************
|
||||
### -----------------------------------------------
|
||||
### !!!!! PLEASE READ INLINE NOTES ON TESTING !!!!!
|
||||
### ***********************************************
|
||||
### -----------------------------------------------
|
||||
|
||||
### Note that:
|
||||
### SETTINGS:
|
||||
### ---------------------------------------------
|
||||
### 0 = allowed - no limits
|
||||
### 1 = allowed or rate limited less restrictive
|
||||
### 2 = rate limited more
|
||||
### 3 = block completely
|
||||
### ---------------------------------------------
|
||||
|
||||
### ************************************************************
|
||||
### ------------------------------------------------------------
|
||||
### CONTRIBUTING / PULL REQUESTS / ADDING YOUR OWN BAD REFERRERS
|
||||
### ************************************************************
|
||||
### ------------------------------------------------------------
|
||||
|
||||
### For contributing, corrections or adding bots or referrers to this repo,
|
||||
### Send a Pull Request (PR) on any of the .list files in the _generator_lists folder
|
||||
### All Pull Requests will be checked for accuracy before being merged.
|
||||
|
||||
# *********************************
|
||||
# FIRST BLOCK BY USER-AGENT STRINGS
|
||||
# *********************************
|
||||
|
||||
# ***********************
|
||||
# -----------------------
|
||||
# !!!!! PLEASE TEST !!!!!
|
||||
# ***********************
|
||||
# -----------------------
|
||||
|
||||
# ALWAYS test any User-Agent Strings you add here to make sure you have it right
|
||||
# Use a Chrome Extension called "User-Agent Switcher for Chrome" where you can create your
|
||||
|
|
@ -98,36 +105,46 @@
|
|||
# curl -I http://www.yourdomain.com -A "GoogleBot" ---- GIVES YOU: HTTP/1.1 200 OK (Meaning web page was served to Client)
|
||||
# curl -I http://www.yourdomain.com -A "80legs" ---- GIVES YOU: curl: (52) Empty reply from server (Meaning Nginx gave a 444 Dropped Connection)
|
||||
|
||||
# In this section we allow/disallow specific User Agents / Bots.
|
||||
|
||||
# *********************************************************************
|
||||
# Now we map all good and bad user agents to a variable called $bad_bot
|
||||
# *********************************************************************
|
||||
# =======================
|
||||
# START BLOCKER FUNCTIONS
|
||||
# =======================
|
||||
|
||||
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
# DO NOT EDIT ANYTHING BELOW THIS LINE !!!
|
||||
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
|
||||
# =============================
|
||||
# BEGIN SECTION 1 - USER-AGENTS
|
||||
# =============================
|
||||
|
||||
# ALLOW / BLOCK User Agents / Bots
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Map all GOOD and BAD UA (User Agents) to a variable called $bad_bot
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
map $http_user_agent $bad_bot {
|
||||
|
||||
default 0;
|
||||
|
||||
# ***********************************************
|
||||
# Include your Own Custom List of Bad User Agents
|
||||
# ***********************************************
|
||||
|
||||
# Use the include file below to further customize your own list of additional user-agents you wish to permanently block
|
||||
# This include file allows whitelisting and blacklisting of anything specified below it, meaning you can over-ride anything in this blocker to your liking.
|
||||
|
||||
# ***********************************************************************************
|
||||
# -----------------------------------------------------------------------------------
|
||||
# START CUSTOM BLACKLISTED USER AGENTS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***********************************************************************************
|
||||
# -----------------------------------------------------------------------------------
|
||||
|
||||
# Include your Own Custom List of Bad User Agents
|
||||
# Use the include file below to further customize your own list of additional user-agents you wish to permanently block
|
||||
# This include file allows whitelisting and blacklisting of anything specified below it.
|
||||
# This include file alows you to over-ride any Bad / Good UA (Bot) declared in this blocker to your liking.
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-user-agents.conf;
|
||||
|
||||
# *********************************************************************************
|
||||
# ---------------------------------------------------------------------------------
|
||||
# END CUSTOM BLACKLISTED USER AGENTS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *********************************************************************************
|
||||
# ---------------------------------------------------------------------------------
|
||||
|
||||
# *********************************************
|
||||
# Bad User-Agent Strings That We Block Outright
|
||||
# *********************************************
|
||||
# --------------------------------------------------
|
||||
# BAD UA (User-Agent) Strings That We Block Outright
|
||||
# --------------------------------------------------
|
||||
|
||||
# START BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)360Spider(?:\b|)" 3;
|
||||
|
|
@ -688,9 +705,16 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)ZyBorg(?:\b|)" 3;
|
||||
# END BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ***********************************************
|
||||
# Allow Good User-Agent Strings We Know and Trust
|
||||
# ***********************************************
|
||||
# --------------------------------------------
|
||||
# GOOD UA User-Agent Strings We Know and Trust
|
||||
# --------------------------------------------
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# You can over-ride these in /etc/nginx/bots.d/blacklist-user-agents.conf
|
||||
# by adding the same UA line there and chaning its value of 1
|
||||
# If you think GoogleBot is bad you would simply add them to
|
||||
# blacklist-user-agents.conf with a value of 1
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)adidxbot(?:\b|)" 0;
|
||||
|
|
@ -727,12 +751,13 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)yahoo(?:\b|)" 0;
|
||||
# END GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ***************************************************
|
||||
# User-Agent Strings Allowed Through but Rate Limited
|
||||
# ***************************************************
|
||||
# --------------------------------------------------------
|
||||
# GOOD UA User-Agent Rate Limiting 1 - Disabled by Default
|
||||
# --------------------------------------------------------
|
||||
|
||||
# Some people block libwww-perl, it used widely in many valid (non rogue) agents
|
||||
# I allow libwww-perl as I use it for monitoring systems with Munin but it is rate limited
|
||||
# TO ACTIVATE THIS RATE LIMITING Uncomment these two lines in blockbots.conf
|
||||
#limit_conn bot1_connlimit 100;
|
||||
#limit_req zone=bot1_reqlimitip burst=50;
|
||||
|
||||
# START ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)jetmon(?:\b|)" 1;
|
||||
|
|
@ -743,9 +768,14 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)Wget/1.15(?:\b|)" 1;
|
||||
# END ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# **************************************************************
|
||||
# Rate Limited User-Agents who get a bit aggressive on bandwidth
|
||||
# **************************************************************
|
||||
# -------------------------------------------------------
|
||||
# GOOD UA User-Agent Rate Limiting 2 - Enabled by Default
|
||||
# -------------------------------------------------------
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# You can over-ride these in /etc/nginx/bots.d/blacklist-user-agents.conf
|
||||
# by adding the same UA line there and chaning its value of 1
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)Alexa(?:\b|)" 2;
|
||||
|
|
@ -768,163 +798,186 @@ map $http_user_agent $bad_bot {
|
|||
# END LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
}
|
||||
|
||||
|
||||
# *****************************************
|
||||
# SECOND BLOCK BY REFERRER STRINGS AND URLS
|
||||
# *****************************************
|
||||
# ===========================
|
||||
# END SECTION 1 - USER-AGENTS
|
||||
# ===========================
|
||||
|
||||
# *****************
|
||||
# =======================================
|
||||
# BEGIN SECTION 2 - REFERRERS AND DOMAINS
|
||||
# =======================================
|
||||
|
||||
# ----------------
|
||||
# PLEASE TEST !!!!
|
||||
# *****************
|
||||
|
||||
# ----------------
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# ALWAYS test referrers that you add. This is done manually as follows
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://anything.adcash.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://www.goodwebsite.com/not-adcash --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://www.betterwebsite.com/not/adcash --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://www.google.com --- GIVES YOU: full html output of the web page
|
||||
# curl -I http://www.yourdomain.com -e http://www.microsoft.com --- GIVES YOU: full html output of the web page
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# Because of case-insensitive matching any combination of capitilization in the names will all produce a positive hit
|
||||
# make sure you always test thoroughly and monitor logs. This section below also does NOT check for a preceding www.
|
||||
# and it also does not care if the referrer request was sent with http https or even ftp.
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
# ***********************************************************************
|
||||
# Now we map all bad referrer words below to a variable called $bad_words
|
||||
# ***********************************************************************
|
||||
# ----------------------------------------------------------------
|
||||
# Map all BAD referrer words below to a variable called $bad_words
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
# --------------------------------
|
||||
# START Bad Referrer Word Scanning
|
||||
# --------------------------------
|
||||
|
||||
map $http_referer $bad_words {
|
||||
default 0;
|
||||
|
||||
# **************************
|
||||
# Bad Referrer Word Scanning
|
||||
# **************************
|
||||
|
||||
# -------------------------------------------------------------------------------------------
|
||||
# These are Words and Terms often found tagged onto domains or within url query strings.
|
||||
# Create and Customize Your Own Bad Referrer Words Here using the new Include File Method
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# customized list of bad referrer words are automatically now included for you
|
||||
# Read Comments inside bad-referrer-words.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# BE VERY CAREFUL using the bad-referrer-words.conf file - please read the comment and
|
||||
# BE VERY CAREFUL using this bad-referrer-words.conf file - please read the comments and
|
||||
# examples inside the include file for detailed explanations into how seriously this can
|
||||
# affect your own site from serving assets or other innocent sites from accessing your site
|
||||
# For safety sake the whitelist-domains.conf file is also loaded here before the bad-referrer-words.conf file.
|
||||
# For safety sake the whitelist-domains.conf file is also loaded here before the
|
||||
# bad-referrer-words.conf file is loaded.
|
||||
# -------------------------------------------------------------------------------------------
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# START WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-domains.conf;
|
||||
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
# END WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
# ******************************************************************************
|
||||
# ------------------------------------------------------------------------------
|
||||
# START CUSTOM BAD REFERRER WORDS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ******************************************************************************
|
||||
# ------------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/bad-referrer-words.conf;
|
||||
|
||||
# ****************************************************************************
|
||||
# ----------------------------------------------------------------------------
|
||||
# END CUSTOM BAD REFERRER WORDS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ****************************************************************************
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
# --------------------------------
|
||||
# END Bad Referrer Word Scanning
|
||||
# --------------------------------
|
||||
|
||||
# *************************
|
||||
# Bad Referrer Domain Names
|
||||
# *************************
|
||||
|
||||
# Now a list of bad referrer urls these domains or any combination of them ie .com .net
|
||||
# will be blocked out. Doesn't matter if the protocol is http, https or even ftp
|
||||
# ----------------------------------------
|
||||
# START Good and Bad Referrer Domain Names
|
||||
# ----------------------------------------
|
||||
|
||||
# -------------------------------------------------------------------------------------
|
||||
# Good and Bad referrer urls Doesn't matter if the protocol is http, https or even ftp
|
||||
# -------------------------------------------------------------------------------------
|
||||
|
||||
# ----------------------
|
||||
# This section includes:
|
||||
# **********************
|
||||
# ----------------------
|
||||
# --------------------------------------------------------------------------------
|
||||
# Blocking of SEO company Semalt.com (now merged into this one section)
|
||||
# MIRAI Botnet Domains Used for Mass Attacks
|
||||
# Other known bad SEO companies and Ad Hijacking Sites
|
||||
# Sites linked to malware, adware, clickjacking and ransomware
|
||||
# Domain names and referrers used in referrer spam and seo hijacking
|
||||
# Whitelisting of your own GOOD domains / referrers
|
||||
# Whitelisting of any other GOOD domains / referrers you want explicitly NOT block
|
||||
# --------------------------------------------------------------------------------
|
||||
|
||||
# *****************
|
||||
# ----------------
|
||||
# PLEASE TEST !!!!
|
||||
# *****************
|
||||
|
||||
# ----------------
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# ALWAYS test referrers that you add. This is done manually as follows
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# Because of case-insensitive matching any combination of capitilization will all produce a positive hit - make sure you always test.
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# For Example any of the following variations below of 8gold.com will be detected and blocked
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://NOT-8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://this.is.not8gOlD.net --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e ftp://8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e ftp://www.weare8gold.NET --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e https://subdomain.8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e https://NOT8GolD.org --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# So if you see a bad referrer from wearegoogle.com and you want to block them just add
|
||||
# them as "~*wearegoogle.com" don't ever go and do something like "~*google(-|.)" you will
|
||||
# kill all your SEO in a week.
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# To add your own custom bad referrers use the custom include file
|
||||
# /etc/nginx/bots.d/custom-bad-referrers.conf
|
||||
# Or send a Pull Request to add it to the global blacklist for other users.
|
||||
|
||||
# In the bad referrers section I also include sites that hotlink images without permission.
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
# ***********************************************************************
|
||||
# Now we map all good & bad referrer urls to variable called #bad_referer
|
||||
# ***********************************************************************
|
||||
# --------------------------------------------------------------------
|
||||
# Map all good & bad referrer DOMAINS to a variable called bad_referer
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
map $http_referer $bad_referer {
|
||||
hostnames;
|
||||
default 0;
|
||||
|
||||
# *************************************
|
||||
# GOOD REFERRERS - Spared from Checking
|
||||
# *************************************
|
||||
# --------------------------------------------
|
||||
# GOOD REFERRER DOMAINS - Spared from Checking
|
||||
# --------------------------------------------
|
||||
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# Add all your own web site domain names and server names in this section
|
||||
|
||||
# WHITELIST Your Own Domain Names Here using the new Include File Method
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted domain names are automatically now included for you.
|
||||
# Read Comments inside whitelist-domains.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# ---------------------------------------------------------------------------------------
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# START WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-domains.conf;
|
||||
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
# END WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
# ***********************************
|
||||
# -----------------------------------
|
||||
# CUSTOM BAD REFERRERS - Add your Own
|
||||
# ***********************************
|
||||
# -----------------------------------
|
||||
|
||||
# Add any extra bad referrers in the following include file to have them
|
||||
# permanently included and blocked - avoid duplicates in your custom file
|
||||
# custom-bad-referrers.conf is BOTH a BLACKLIST AND WHITELIST
|
||||
# custom-bad-referrers.conf ALLOWS complete over-riding of anything
|
||||
# If you think google.com is bad you would simply add them to
|
||||
# custom-bad-referrers.conf with a value of 1
|
||||
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
# START CUSTOM BAD REFERRERS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/custom-bad-referrers.conf;
|
||||
|
||||
# ***********************************************************************
|
||||
# -----------------------------------------------------------------------
|
||||
# END CUSTOM BAD REFERRERS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***********************************************************************
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START BAD REFERRERS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)000free\.us(?:\b|)" 1;
|
||||
|
|
@ -7644,137 +7697,49 @@ map $http_referer $bad_referer {
|
|||
# END BAD REFERRERS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
}
|
||||
|
||||
# =====================================
|
||||
# END SECTION 2 - REFERRERS AND DOMAINS
|
||||
# =====================================
|
||||
|
||||
# ***********************************************
|
||||
# WHITELISTING AND BLACKLISTING IP ADDRESS RANGES
|
||||
# ***********************************************
|
||||
# ========================================================================
|
||||
# BEGIN SECTION 3 - WHITELISTING AND BLACKLISTING IP ADDRESSESE AND RANGES
|
||||
# ========================================================================
|
||||
|
||||
# Geo directive to deny and also whitelist certain ip addresses
|
||||
# --------------------------------------------------------------------------------------
|
||||
# Map all GOOD and BAD IP Addresses and Ranges to a variable called geo $validate_client
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
geo $validate_client {
|
||||
|
||||
# ********************
|
||||
# First Our Safety Net
|
||||
# ********************
|
||||
|
||||
# Anything not matching our rules is allowed through with default 0;
|
||||
|
||||
default 0;
|
||||
|
||||
# ***********************************
|
||||
# Whitelist all your OWN IP addresses
|
||||
# ***********************************
|
||||
# ----------------------------------------------
|
||||
# Whitelist all your OWN IP addresses and Ranges
|
||||
# ----------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST all your own IP addresses using the include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside whitelist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# whitelist-ips.conf reigns supreme !!!
|
||||
# Whatever you add to whitelist-ips.conf will be whitelisted FULL STOP
|
||||
# Anything blacklisted above this line will be over-ridden by whitelist-ips.conf
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# **************************************************************************
|
||||
# --------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **************************************************************************
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
# ****************
|
||||
# Google IP Ranges
|
||||
# ****************
|
||||
|
||||
# For Safety Sake all Google's Known IP Ranges are all white listed
|
||||
|
||||
# START GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
108.177.0.0/17 0;
|
||||
172.217.0.0/16 0;
|
||||
173.194.0.0/16 0;
|
||||
2001:4860:4000::/36 0;
|
||||
203.208.60.0/24 0;
|
||||
207.126.144.0/20 0;
|
||||
209.85.128.0/17 0;
|
||||
216.239.32.0/19 0;
|
||||
216.58.192.0/19 0;
|
||||
2404:6800:4000::/36 0;
|
||||
2607:f8b0:4000::/36 0;
|
||||
2800:3f0:4000::/36 0;
|
||||
2a00:1450:4000::/36 0;
|
||||
2c0f:fb50:4000::/36 0;
|
||||
35.192.0.0/12 0;
|
||||
64.18.0.0/20 0;
|
||||
64.233.160.0/19 0;
|
||||
64.68.80.0/21 0;
|
||||
65.52.0.0/14 0;
|
||||
66.102.0.0/20 0;
|
||||
66.249.64.0/19 0;
|
||||
72.14.192.0/18 0;
|
||||
74.125.0.0/16 0;
|
||||
# END GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# **************
|
||||
# Bing IP Ranges
|
||||
# **************
|
||||
|
||||
# For Safety Sake all Bing's Known IP Ranges are all white listed
|
||||
|
||||
# START BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
131.253.21.0/24 0;
|
||||
131.253.22.0/23 0;
|
||||
131.253.24.0/21 0;
|
||||
131.253.24.0/22 0;
|
||||
131.253.32.0/20 0;
|
||||
157.54.0.0/15 0;
|
||||
157.56.0.0/14 0;
|
||||
157.60.0.0/16 0;
|
||||
199.30.16.0/24 0;
|
||||
199.30.27.0/24 0;
|
||||
207.46.0.0/16 0;
|
||||
40.112.0.0/13 0;
|
||||
40.120.0.0/14 0;
|
||||
40.124.0.0/16 0;
|
||||
40.125.0.0/17 0;
|
||||
40.74.0.0/15 0;
|
||||
40.76.0.0/14 0;
|
||||
40.80.0.0/12 0;
|
||||
40.96.0.0/12 0;
|
||||
# END BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ********************
|
||||
# Cloudflare IP Ranges
|
||||
# ********************
|
||||
|
||||
# For Safety Sake all Cloudflare's Known IP Ranges are all white listed
|
||||
|
||||
# START CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
103.21.244.0/22 0;
|
||||
103.22.200.0/22 0;
|
||||
103.31.4.0/22 0;
|
||||
104.16.0.0/12 0;
|
||||
108.162.192.0/18 0;
|
||||
131.0.72.0/22 0;
|
||||
141.101.64.0/18 0;
|
||||
162.158.0.0/15 0;
|
||||
172.64.0.0/13 0;
|
||||
173.245.48.0/20 0;
|
||||
188.114.96.0/20 0;
|
||||
190.93.240.0/20 0;
|
||||
197.234.240.0/22 0;
|
||||
198.41.128.0/17 0;
|
||||
199.27.128.0/21 0;
|
||||
2400:cb00::/32 0;
|
||||
2405:8100::/32 0;
|
||||
2405:b500::/32 0;
|
||||
2606:4700::/32 0;
|
||||
2803:f800::/32 0;
|
||||
2a06:98c0::/29 0;
|
||||
2c0f:f248::/32 0;
|
||||
# END CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# *************************
|
||||
# Wordpress Theme Detectors
|
||||
# *************************
|
||||
# -------------------------------------
|
||||
# BLOCK known Wordpress Theme Detectors
|
||||
# -------------------------------------
|
||||
|
||||
# START WP THEME DETECTORS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
|
|
@ -7818,10 +7783,11 @@ geo $validate_client {
|
|||
89.36.223.188 1; #www.cuteseotools.net/wordpress-theme-detector
|
||||
# END WP THEME DETECTORS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ****************************************
|
||||
# NIBBLER - SEO testing and reporting tool
|
||||
# ****************************************
|
||||
# ----------------------------------------------
|
||||
# BLOCK NIBBLER - SEO testing and reporting tool
|
||||
# ----------------------------------------------
|
||||
# See - http://nibbler.silktide.com/
|
||||
# ----------------------------------------------
|
||||
|
||||
# START NIBBLER ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
52.201.238.175 1;
|
||||
|
|
@ -7834,33 +7800,10 @@ geo $validate_client {
|
|||
# END NIBBLER ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
|
||||
# ****************************
|
||||
# Known Bad IP's and IP Ranges
|
||||
# *************************************************
|
||||
# Blacklist IP addresses and IP Ranges Customizable
|
||||
# *************************************************
|
||||
|
||||
# BLACKLIST all your IP addresses and Ranges using the new include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# Custom Blacklisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside blacklist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
|
||||
# ********************************************************************
|
||||
# START BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ********************************************************************
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-ips.conf;
|
||||
|
||||
# ******************************************************************
|
||||
# END BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ******************************************************************
|
||||
|
||||
# *********************************************************************************************
|
||||
# KNOWN BAD IP ADDRESSES
|
||||
# Source: https://github.com/mitchellkrogza/Suspicious.Snooping.Sniffing.Hacking.IP.Addresses
|
||||
# A Reliable Daily Updated List Included Below of Known Bad IP Addresses, Snoopers and Sniffers
|
||||
# *********************************************************************************************
|
||||
# -----------------------------------------
|
||||
# BLOCK KNOWN BAD IP ADDRESSES
|
||||
# Top known bad IP Adresses from abuseIPDB
|
||||
# -----------------------------------------
|
||||
|
||||
# START KNOWN BAD IP ADDRESSES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
193.32.163.89 1;
|
||||
|
|
@ -17864,33 +17807,176 @@ geo $validate_client {
|
|||
46.101.115.65 1;
|
||||
# END KNOWN BAD IP ADDRESSES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# --------------------------
|
||||
# WHITELIST Google IP Ranges
|
||||
# --------------------------
|
||||
|
||||
# START GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
108.177.0.0/17 0;
|
||||
172.217.0.0/16 0;
|
||||
173.194.0.0/16 0;
|
||||
2001:4860:4000::/36 0;
|
||||
203.208.60.0/24 0;
|
||||
207.126.144.0/20 0;
|
||||
209.85.128.0/17 0;
|
||||
216.239.32.0/19 0;
|
||||
216.58.192.0/19 0;
|
||||
2404:6800:4000::/36 0;
|
||||
2607:f8b0:4000::/36 0;
|
||||
2800:3f0:4000::/36 0;
|
||||
2a00:1450:4000::/36 0;
|
||||
2c0f:fb50:4000::/36 0;
|
||||
35.192.0.0/12 0;
|
||||
64.18.0.0/20 0;
|
||||
64.233.160.0/19 0;
|
||||
64.68.80.0/21 0;
|
||||
65.52.0.0/14 0;
|
||||
66.102.0.0/20 0;
|
||||
66.249.64.0/19 0;
|
||||
72.14.192.0/18 0;
|
||||
74.125.0.0/16 0;
|
||||
# END GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ------------------------
|
||||
# WHITELIST Bing IP Ranges
|
||||
# ------------------------
|
||||
|
||||
# START BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
131.253.21.0/24 0;
|
||||
131.253.22.0/23 0;
|
||||
131.253.24.0/21 0;
|
||||
131.253.24.0/22 0;
|
||||
131.253.32.0/20 0;
|
||||
157.54.0.0/15 0;
|
||||
157.56.0.0/14 0;
|
||||
157.60.0.0/16 0;
|
||||
199.30.16.0/24 0;
|
||||
199.30.27.0/24 0;
|
||||
207.46.0.0/16 0;
|
||||
40.112.0.0/13 0;
|
||||
40.120.0.0/14 0;
|
||||
40.124.0.0/16 0;
|
||||
40.125.0.0/17 0;
|
||||
40.74.0.0/15 0;
|
||||
40.76.0.0/14 0;
|
||||
40.80.0.0/12 0;
|
||||
40.96.0.0/12 0;
|
||||
# END BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ------------------------------
|
||||
# WHITELIST Cloudflare IP Ranges
|
||||
# ------------------------------
|
||||
|
||||
# START CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
103.21.244.0/22 0;
|
||||
103.22.200.0/22 0;
|
||||
103.31.4.0/22 0;
|
||||
104.16.0.0/12 0;
|
||||
108.162.192.0/18 0;
|
||||
131.0.72.0/22 0;
|
||||
141.101.64.0/18 0;
|
||||
162.158.0.0/15 0;
|
||||
172.64.0.0/13 0;
|
||||
173.245.48.0/20 0;
|
||||
188.114.96.0/20 0;
|
||||
190.93.240.0/20 0;
|
||||
197.234.240.0/22 0;
|
||||
198.41.128.0/17 0;
|
||||
199.27.128.0/21 0;
|
||||
2400:cb00::/32 0;
|
||||
2405:8100::/32 0;
|
||||
2405:b500::/32 0;
|
||||
2606:4700::/32 0;
|
||||
2803:f800::/32 0;
|
||||
2a06:98c0::/29 0;
|
||||
2c0f:f248::/32 0;
|
||||
# END CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# -------------------------------------------------
|
||||
# BLACKLIST IP addresses and IP Ranges Customizable
|
||||
# -------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# BLACKLIST all your IP addresses and Ranges using the new include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# Custom Blacklisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside blacklist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# Anything added to blacklist-ips.conf will over-ride anything whitelisted above
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# START BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-ips.conf;
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# END BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
# ----------------------------------------------
|
||||
# Whitelist all your OWN IP addresses and Ranges
|
||||
# ----------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST all your own IP addresses using the include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside whitelist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# whitelist-ips.conf reigns supreme !!!
|
||||
# Whatever you add to whitelist-ips.conf will be whitelisted FULL STOP
|
||||
# Anything blacklisted above this line will be over-ridden by whitelist-ips.conf
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# ------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST your own IPs from the DDOS Filter
|
||||
# Add your own IP addresses and ranges into the custom include file whitelist-ips.conf
|
||||
# to spare them from the rate limiting DDOS filter.
|
||||
# This section includes the same / single whitelist-ips.conf file so you only
|
||||
# need to edit that include file and have it include here for you too.
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
geo $ratelimited {
|
||||
default 1;
|
||||
|
||||
# ***************************************************************************
|
||||
# ---------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES2 ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***************************************************************************
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES2 ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
|
||||
# *****************************************
|
||||
# MAP BAD BOTS TO OUR RATE LIMITER FUNCTION
|
||||
# *****************************************
|
||||
# ======================================================================
|
||||
# END SECTION 3 - WHITELISTING AND BLACKLISTING IP ADDRESSESE AND RANGES
|
||||
# ======================================================================
|
||||
|
||||
|
||||
# ============================================
|
||||
# BEGIN SECTION 4 - ACTIVATE BLOCKER FUNCTIONS
|
||||
# ============================================
|
||||
|
||||
# --------------------------------------------
|
||||
# 1. MAP BAD BOTS TO OUR RATE LIMITER FUNCTION
|
||||
# --------------------------------------------
|
||||
|
||||
map $bad_bot $bot_iplimit {
|
||||
0 "";
|
||||
|
|
@ -17898,30 +17984,31 @@ geo $ratelimited {
|
|||
2 $binary_remote_addr;
|
||||
}
|
||||
|
||||
# ***********************
|
||||
# SET RATE LIMITING ZONES
|
||||
# ***********************
|
||||
# --------------------------
|
||||
# 2. SET RATE LIMITING ZONES
|
||||
# --------------------------
|
||||
|
||||
# BAD BOT RATE LIMITING ZONE
|
||||
# limits for Zone $bad_bot = 1
|
||||
# Nothing Set - you can set a different zone limiter here if you like
|
||||
# We issue a 444 response instead to all bad bots.
|
||||
# Rate limiting will only take effect if on any User-Agents with a value of 2
|
||||
|
||||
# limits for Zone $bad_bot = 2
|
||||
# this rate limiting will only take effect if you change any of the bots and change
|
||||
# their block value from 1 to 2.
|
||||
limit_conn_zone $bot_iplimit zone=bot2_connlimit:16m;
|
||||
limit_req_zone $bot_iplimit zone=bot2_reqlimitip:16m rate=2r/s;
|
||||
limit_req_zone $bot_iplimit zone=bot2_reqlimitip:16m rate=6r/m;
|
||||
|
||||
### *** MAKE SURE TO ADD to your nginx.conf ***
|
||||
### server_names_hash_bucket_size 64;
|
||||
### server_names_hash_max_size 4096;
|
||||
### limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
|
||||
### limit_conn_zone $binary_remote_addr zone=addr:50m;
|
||||
### to allow it to load this large set of domains into memory and to set the rate limiting zones for the DDOS filter.
|
||||
# ==========================================
|
||||
# END SECTION 4 - ACTIVATE BLOCKER FUNCTIONS
|
||||
# ==========================================
|
||||
|
||||
### THE END of the Long and Winding Road
|
||||
# =====================
|
||||
# END BLOCKER FUNCTIONS
|
||||
# =====================
|
||||
|
||||
### Also check out my Ultimate Apache Bad Bot Blocker on Github
|
||||
### https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker
|
||||
### --------------------------------------------
|
||||
### HELP SUPPORT THIS PROJECT - Send Me a Coffee
|
||||
### https://ko-fi.com/mitchellkrog
|
||||
### --------------------------------------------
|
||||
|
||||
### FOR APACHE SERVERS
|
||||
### ---------------------------------------------
|
||||
### Check out the Ultimate Apache Bad Bot Blocker
|
||||
### ---------------------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -1,2 +1,2 @@
|
|||
35.192.85.2 1;
|
||||
104.154.120.187 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
35.192.85.2 0;
|
||||
104.154.120.187 0;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
|
|
@ -1,16 +1,21 @@
|
|||
### **********************************
|
||||
### THE ULTIMATE NGINX BAD BOT BLOCKER
|
||||
### **********************************
|
||||
### -----------------------------------------------------------
|
||||
### THE NGINX ULTIMATE BAD BOT, BAD IP AND BAD REFERRER BLOCKER
|
||||
### -----------------------------------------------------------
|
||||
|
||||
### VERSION INFORMATION #
|
||||
###################################################
|
||||
### Version: V4.2019.06.1643
|
||||
### Updated: Thu Jun 27 12:59:42 SAST 2019
|
||||
### Version: V3.2019.06.1645
|
||||
### Updated: Thu Jun 27 13:24:56 SAST 2019
|
||||
### Bad Referrer Count: 6713
|
||||
### Bad Bot Count: 556
|
||||
###################################################
|
||||
### VERSION INFORMATION ##
|
||||
|
||||
### --------------------------------------------
|
||||
### HELP SUPPORT THIS PROJECT - Send Me a Coffee
|
||||
### https://ko-fi.com/mitchellkrog
|
||||
### --------------------------------------------
|
||||
|
||||
##############################################################################
|
||||
# _ __ _ #
|
||||
# / |/ /__ _(_)__ __ __ #
|
||||
|
|
@ -28,12 +33,16 @@
|
|||
### and also has rate limiting functionality for bad bots who you only want to rate limit
|
||||
### and not actually block out entirely. It is very powerful and also very flexible.
|
||||
|
||||
### --------------------------------------------------------------------------
|
||||
### Created By: https://github.com/mitchellkrogza/
|
||||
### Repo Url: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker
|
||||
### Copyright Mitchell Krog - <mitchellkrog@gmail.com>
|
||||
### Contributors: Stuart Cardall - https://github.com/itoffshore
|
||||
### --------------------------------------------------------------------------
|
||||
|
||||
### --------------------------------------------------------------------------
|
||||
### Tested on: nginx/1.10.3 up to latest Mainstream Version (Ubuntu 16.04)
|
||||
### --------------------------------------------------------------------------
|
||||
|
||||
### This list was developed and is in use on a live Nginx server running some very busy web sites.
|
||||
### It was built from the ground up using real data from daily logs and is updated almost daily.
|
||||
|
|
@ -41,9 +50,9 @@
|
|||
### spam referrers, rogue IP address, scanners, scrapers and domain hijacking sites are extensively checked
|
||||
### before they are added. It is monitored extensively for any false positives.
|
||||
|
||||
### *********
|
||||
### ---------
|
||||
### Features:
|
||||
### *********
|
||||
### ---------
|
||||
### Clear formatting for Ease of Maintenance.
|
||||
### Alphabetically ordered lists for Ease of Maintenance.
|
||||
### Extensive Commenting for Ease of Reference.
|
||||
|
|
@ -54,40 +63,38 @@
|
|||
### IP range blocking / whitelisting.
|
||||
### Rate Limiting Functions.
|
||||
|
||||
### ************
|
||||
### ------------
|
||||
### INSTALLATION
|
||||
### ************
|
||||
### ------------
|
||||
|
||||
### PLEASE use the install, setup and update scripts provided for you to ease your installation.
|
||||
### This Auto Installation procedure is documented in the README.md and AUTO-CONFIGURATION.md files.
|
||||
### Installation, Setup and Update Scripts Contributed by Stuart Cardall - https://github.com/itoffshore
|
||||
### There are also manual configuration instructions provided for those not wishing to do an auto install.
|
||||
|
||||
### ***********************************************
|
||||
### -----------------------------------------------
|
||||
### !!!!! PLEASE READ INLINE NOTES ON TESTING !!!!!
|
||||
### ***********************************************
|
||||
### -----------------------------------------------
|
||||
|
||||
### Note that:
|
||||
### SETTINGS:
|
||||
### ---------------------------------------------
|
||||
### 0 = allowed - no limits
|
||||
### 1 = allowed or rate limited less restrictive
|
||||
### 2 = rate limited more
|
||||
### 3 = block completely
|
||||
### ---------------------------------------------
|
||||
|
||||
### ************************************************************
|
||||
### ------------------------------------------------------------
|
||||
### CONTRIBUTING / PULL REQUESTS / ADDING YOUR OWN BAD REFERRERS
|
||||
### ************************************************************
|
||||
### ------------------------------------------------------------
|
||||
|
||||
### For contributing, corrections or adding bots or referrers to this repo,
|
||||
### Send a Pull Request (PR) on any of the .list files in the _generator_lists folder
|
||||
### All Pull Requests will be checked for accuracy before being merged.
|
||||
|
||||
# *********************************
|
||||
# FIRST BLOCK BY USER-AGENT STRINGS
|
||||
# *********************************
|
||||
|
||||
# ***********************
|
||||
# -----------------------
|
||||
# !!!!! PLEASE TEST !!!!!
|
||||
# ***********************
|
||||
# -----------------------
|
||||
|
||||
# ALWAYS test any User-Agent Strings you add here to make sure you have it right
|
||||
# Use a Chrome Extension called "User-Agent Switcher for Chrome" where you can create your
|
||||
|
|
@ -98,36 +105,46 @@
|
|||
# curl -I http://www.yourdomain.com -A "GoogleBot" ---- GIVES YOU: HTTP/1.1 200 OK (Meaning web page was served to Client)
|
||||
# curl -I http://www.yourdomain.com -A "80legs" ---- GIVES YOU: curl: (52) Empty reply from server (Meaning Nginx gave a 444 Dropped Connection)
|
||||
|
||||
# In this section we allow/disallow specific User Agents / Bots.
|
||||
|
||||
# *********************************************************************
|
||||
# Now we map all good and bad user agents to a variable called $bad_bot
|
||||
# *********************************************************************
|
||||
# =======================
|
||||
# START BLOCKER FUNCTIONS
|
||||
# =======================
|
||||
|
||||
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
# DO NOT EDIT ANYTHING BELOW THIS LINE !!!
|
||||
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
|
||||
# =============================
|
||||
# BEGIN SECTION 1 - USER-AGENTS
|
||||
# =============================
|
||||
|
||||
# ALLOW / BLOCK User Agents / Bots
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Map all GOOD and BAD UA (User Agents) to a variable called $bad_bot
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
map $http_user_agent $bad_bot {
|
||||
|
||||
default 0;
|
||||
|
||||
# ***********************************************
|
||||
# Include your Own Custom List of Bad User Agents
|
||||
# ***********************************************
|
||||
|
||||
# Use the include file below to further customize your own list of additional user-agents you wish to permanently block
|
||||
# This include file allows whitelisting and blacklisting of anything specified below it, meaning you can over-ride anything in this blocker to your liking.
|
||||
|
||||
# ***********************************************************************************
|
||||
# -----------------------------------------------------------------------------------
|
||||
# START CUSTOM BLACKLISTED USER AGENTS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***********************************************************************************
|
||||
# -----------------------------------------------------------------------------------
|
||||
|
||||
# Include your Own Custom List of Bad User Agents
|
||||
# Use the include file below to further customize your own list of additional user-agents you wish to permanently block
|
||||
# This include file allows whitelisting and blacklisting of anything specified below it.
|
||||
# This include file alows you to over-ride any Bad / Good UA (Bot) declared in this blocker to your liking.
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-user-agents.conf;
|
||||
|
||||
# *********************************************************************************
|
||||
# ---------------------------------------------------------------------------------
|
||||
# END CUSTOM BLACKLISTED USER AGENTS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *********************************************************************************
|
||||
# ---------------------------------------------------------------------------------
|
||||
|
||||
# *********************************************
|
||||
# Bad User-Agent Strings That We Block Outright
|
||||
# *********************************************
|
||||
# --------------------------------------------------
|
||||
# BAD UA (User-Agent) Strings That We Block Outright
|
||||
# --------------------------------------------------
|
||||
|
||||
# START BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)360Spider(?:\b|)" 3;
|
||||
|
|
@ -688,9 +705,16 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)ZyBorg(?:\b|)" 3;
|
||||
# END BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ***********************************************
|
||||
# Allow Good User-Agent Strings We Know and Trust
|
||||
# ***********************************************
|
||||
# --------------------------------------------
|
||||
# GOOD UA User-Agent Strings We Know and Trust
|
||||
# --------------------------------------------
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# You can over-ride these in /etc/nginx/bots.d/blacklist-user-agents.conf
|
||||
# by adding the same UA line there and chaning its value of 1
|
||||
# If you think GoogleBot is bad you would simply add them to
|
||||
# blacklist-user-agents.conf with a value of 1
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)adidxbot(?:\b|)" 0;
|
||||
|
|
@ -727,12 +751,13 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)yahoo(?:\b|)" 0;
|
||||
# END GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ***************************************************
|
||||
# User-Agent Strings Allowed Through but Rate Limited
|
||||
# ***************************************************
|
||||
# --------------------------------------------------------
|
||||
# GOOD UA User-Agent Rate Limiting 1 - Disabled by Default
|
||||
# --------------------------------------------------------
|
||||
|
||||
# Some people block libwww-perl, it used widely in many valid (non rogue) agents
|
||||
# I allow libwww-perl as I use it for monitoring systems with Munin but it is rate limited
|
||||
# TO ACTIVATE THIS RATE LIMITING Uncomment these two lines in blockbots.conf
|
||||
#limit_conn bot1_connlimit 100;
|
||||
#limit_req zone=bot1_reqlimitip burst=50;
|
||||
|
||||
# START ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)jetmon(?:\b|)" 1;
|
||||
|
|
@ -743,9 +768,14 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)Wget/1.15(?:\b|)" 1;
|
||||
# END ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# **************************************************************
|
||||
# Rate Limited User-Agents who get a bit aggressive on bandwidth
|
||||
# **************************************************************
|
||||
# -------------------------------------------------------
|
||||
# GOOD UA User-Agent Rate Limiting 2 - Enabled by Default
|
||||
# -------------------------------------------------------
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# You can over-ride these in /etc/nginx/bots.d/blacklist-user-agents.conf
|
||||
# by adding the same UA line there and chaning its value of 1
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)Alexa(?:\b|)" 2;
|
||||
|
|
@ -768,163 +798,186 @@ map $http_user_agent $bad_bot {
|
|||
# END LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
}
|
||||
|
||||
|
||||
# *****************************************
|
||||
# SECOND BLOCK BY REFERRER STRINGS AND URLS
|
||||
# *****************************************
|
||||
# ===========================
|
||||
# END SECTION 1 - USER-AGENTS
|
||||
# ===========================
|
||||
|
||||
# *****************
|
||||
# =======================================
|
||||
# BEGIN SECTION 2 - REFERRERS AND DOMAINS
|
||||
# =======================================
|
||||
|
||||
# ----------------
|
||||
# PLEASE TEST !!!!
|
||||
# *****************
|
||||
|
||||
# ----------------
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# ALWAYS test referrers that you add. This is done manually as follows
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://anything.adcash.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://www.goodwebsite.com/not-adcash --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://www.betterwebsite.com/not/adcash --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://www.google.com --- GIVES YOU: full html output of the web page
|
||||
# curl -I http://www.yourdomain.com -e http://www.microsoft.com --- GIVES YOU: full html output of the web page
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# Because of case-insensitive matching any combination of capitilization in the names will all produce a positive hit
|
||||
# make sure you always test thoroughly and monitor logs. This section below also does NOT check for a preceding www.
|
||||
# and it also does not care if the referrer request was sent with http https or even ftp.
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
# ***********************************************************************
|
||||
# Now we map all bad referrer words below to a variable called $bad_words
|
||||
# ***********************************************************************
|
||||
# ----------------------------------------------------------------
|
||||
# Map all BAD referrer words below to a variable called $bad_words
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
# --------------------------------
|
||||
# START Bad Referrer Word Scanning
|
||||
# --------------------------------
|
||||
|
||||
map $http_referer $bad_words {
|
||||
default 0;
|
||||
|
||||
# **************************
|
||||
# Bad Referrer Word Scanning
|
||||
# **************************
|
||||
|
||||
# -------------------------------------------------------------------------------------------
|
||||
# These are Words and Terms often found tagged onto domains or within url query strings.
|
||||
# Create and Customize Your Own Bad Referrer Words Here using the new Include File Method
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# customized list of bad referrer words are automatically now included for you
|
||||
# Read Comments inside bad-referrer-words.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# BE VERY CAREFUL using the bad-referrer-words.conf file - please read the comment and
|
||||
# BE VERY CAREFUL using this bad-referrer-words.conf file - please read the comments and
|
||||
# examples inside the include file for detailed explanations into how seriously this can
|
||||
# affect your own site from serving assets or other innocent sites from accessing your site
|
||||
# For safety sake the whitelist-domains.conf file is also loaded here before the bad-referrer-words.conf file.
|
||||
# For safety sake the whitelist-domains.conf file is also loaded here before the
|
||||
# bad-referrer-words.conf file is loaded.
|
||||
# -------------------------------------------------------------------------------------------
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# START WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-domains.conf;
|
||||
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
# END WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
# ******************************************************************************
|
||||
# ------------------------------------------------------------------------------
|
||||
# START CUSTOM BAD REFERRER WORDS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ******************************************************************************
|
||||
# ------------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/bad-referrer-words.conf;
|
||||
|
||||
# ****************************************************************************
|
||||
# ----------------------------------------------------------------------------
|
||||
# END CUSTOM BAD REFERRER WORDS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ****************************************************************************
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
# --------------------------------
|
||||
# END Bad Referrer Word Scanning
|
||||
# --------------------------------
|
||||
|
||||
# *************************
|
||||
# Bad Referrer Domain Names
|
||||
# *************************
|
||||
|
||||
# Now a list of bad referrer urls these domains or any combination of them ie .com .net
|
||||
# will be blocked out. Doesn't matter if the protocol is http, https or even ftp
|
||||
# ----------------------------------------
|
||||
# START Good and Bad Referrer Domain Names
|
||||
# ----------------------------------------
|
||||
|
||||
# -------------------------------------------------------------------------------------
|
||||
# Good and Bad referrer urls Doesn't matter if the protocol is http, https or even ftp
|
||||
# -------------------------------------------------------------------------------------
|
||||
|
||||
# ----------------------
|
||||
# This section includes:
|
||||
# **********************
|
||||
# ----------------------
|
||||
# --------------------------------------------------------------------------------
|
||||
# Blocking of SEO company Semalt.com (now merged into this one section)
|
||||
# MIRAI Botnet Domains Used for Mass Attacks
|
||||
# Other known bad SEO companies and Ad Hijacking Sites
|
||||
# Sites linked to malware, adware, clickjacking and ransomware
|
||||
# Domain names and referrers used in referrer spam and seo hijacking
|
||||
# Whitelisting of your own GOOD domains / referrers
|
||||
# Whitelisting of any other GOOD domains / referrers you want explicitly NOT block
|
||||
# --------------------------------------------------------------------------------
|
||||
|
||||
# *****************
|
||||
# ----------------
|
||||
# PLEASE TEST !!!!
|
||||
# *****************
|
||||
|
||||
# ----------------
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# ALWAYS test referrers that you add. This is done manually as follows
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# Because of case-insensitive matching any combination of capitilization will all produce a positive hit - make sure you always test.
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# For Example any of the following variations below of 8gold.com will be detected and blocked
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://NOT-8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://this.is.not8gOlD.net --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e ftp://8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e ftp://www.weare8gold.NET --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e https://subdomain.8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e https://NOT8GolD.org --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# So if you see a bad referrer from wearegoogle.com and you want to block them just add
|
||||
# them as "~*wearegoogle.com" don't ever go and do something like "~*google(-|.)" you will
|
||||
# kill all your SEO in a week.
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# To add your own custom bad referrers use the custom include file
|
||||
# /etc/nginx/bots.d/custom-bad-referrers.conf
|
||||
# Or send a Pull Request to add it to the global blacklist for other users.
|
||||
|
||||
# In the bad referrers section I also include sites that hotlink images without permission.
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
# ***********************************************************************
|
||||
# Now we map all good & bad referrer urls to variable called #bad_referer
|
||||
# ***********************************************************************
|
||||
# --------------------------------------------------------------------
|
||||
# Map all good & bad referrer DOMAINS to a variable called bad_referer
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
map $http_referer $bad_referer {
|
||||
hostnames;
|
||||
default 0;
|
||||
|
||||
# *************************************
|
||||
# GOOD REFERRERS - Spared from Checking
|
||||
# *************************************
|
||||
# --------------------------------------------
|
||||
# GOOD REFERRER DOMAINS - Spared from Checking
|
||||
# --------------------------------------------
|
||||
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# Add all your own web site domain names and server names in this section
|
||||
|
||||
# WHITELIST Your Own Domain Names Here using the new Include File Method
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted domain names are automatically now included for you.
|
||||
# Read Comments inside whitelist-domains.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# ---------------------------------------------------------------------------------------
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# START WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-domains.conf;
|
||||
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
# END WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
# ***********************************
|
||||
# -----------------------------------
|
||||
# CUSTOM BAD REFERRERS - Add your Own
|
||||
# ***********************************
|
||||
# -----------------------------------
|
||||
|
||||
# Add any extra bad referrers in the following include file to have them
|
||||
# permanently included and blocked - avoid duplicates in your custom file
|
||||
# custom-bad-referrers.conf is BOTH a BLACKLIST AND WHITELIST
|
||||
# custom-bad-referrers.conf ALLOWS complete over-riding of anything
|
||||
# If you think google.com is bad you would simply add them to
|
||||
# custom-bad-referrers.conf with a value of 1
|
||||
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
# START CUSTOM BAD REFERRERS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/custom-bad-referrers.conf;
|
||||
|
||||
# ***********************************************************************
|
||||
# -----------------------------------------------------------------------
|
||||
# END CUSTOM BAD REFERRERS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***********************************************************************
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START BAD REFERRERS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)000free\.us(?:\b|)" 1;
|
||||
|
|
@ -7644,137 +7697,49 @@ map $http_referer $bad_referer {
|
|||
# END BAD REFERRERS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
}
|
||||
|
||||
# =====================================
|
||||
# END SECTION 2 - REFERRERS AND DOMAINS
|
||||
# =====================================
|
||||
|
||||
# ***********************************************
|
||||
# WHITELISTING AND BLACKLISTING IP ADDRESS RANGES
|
||||
# ***********************************************
|
||||
# ========================================================================
|
||||
# BEGIN SECTION 3 - WHITELISTING AND BLACKLISTING IP ADDRESSESE AND RANGES
|
||||
# ========================================================================
|
||||
|
||||
# Geo directive to deny and also whitelist certain ip addresses
|
||||
# --------------------------------------------------------------------------------------
|
||||
# Map all GOOD and BAD IP Addresses and Ranges to a variable called geo $validate_client
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
geo $validate_client {
|
||||
|
||||
# ********************
|
||||
# First Our Safety Net
|
||||
# ********************
|
||||
|
||||
# Anything not matching our rules is allowed through with default 0;
|
||||
|
||||
default 0;
|
||||
|
||||
# ***********************************
|
||||
# Whitelist all your OWN IP addresses
|
||||
# ***********************************
|
||||
# ----------------------------------------------
|
||||
# Whitelist all your OWN IP addresses and Ranges
|
||||
# ----------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST all your own IP addresses using the include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside whitelist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# whitelist-ips.conf reigns supreme !!!
|
||||
# Whatever you add to whitelist-ips.conf will be whitelisted FULL STOP
|
||||
# Anything blacklisted above this line will be over-ridden by whitelist-ips.conf
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# **************************************************************************
|
||||
# --------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **************************************************************************
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
# ****************
|
||||
# Google IP Ranges
|
||||
# ****************
|
||||
|
||||
# For Safety Sake all Google's Known IP Ranges are all white listed
|
||||
|
||||
# START GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
108.177.0.0/17 0;
|
||||
172.217.0.0/16 0;
|
||||
173.194.0.0/16 0;
|
||||
2001:4860:4000::/36 0;
|
||||
203.208.60.0/24 0;
|
||||
207.126.144.0/20 0;
|
||||
209.85.128.0/17 0;
|
||||
216.239.32.0/19 0;
|
||||
216.58.192.0/19 0;
|
||||
2404:6800:4000::/36 0;
|
||||
2607:f8b0:4000::/36 0;
|
||||
2800:3f0:4000::/36 0;
|
||||
2a00:1450:4000::/36 0;
|
||||
2c0f:fb50:4000::/36 0;
|
||||
35.192.0.0/12 0;
|
||||
64.18.0.0/20 0;
|
||||
64.233.160.0/19 0;
|
||||
64.68.80.0/21 0;
|
||||
65.52.0.0/14 0;
|
||||
66.102.0.0/20 0;
|
||||
66.249.64.0/19 0;
|
||||
72.14.192.0/18 0;
|
||||
74.125.0.0/16 0;
|
||||
# END GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# **************
|
||||
# Bing IP Ranges
|
||||
# **************
|
||||
|
||||
# For Safety Sake all Bing's Known IP Ranges are all white listed
|
||||
|
||||
# START BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
131.253.21.0/24 0;
|
||||
131.253.22.0/23 0;
|
||||
131.253.24.0/21 0;
|
||||
131.253.24.0/22 0;
|
||||
131.253.32.0/20 0;
|
||||
157.54.0.0/15 0;
|
||||
157.56.0.0/14 0;
|
||||
157.60.0.0/16 0;
|
||||
199.30.16.0/24 0;
|
||||
199.30.27.0/24 0;
|
||||
207.46.0.0/16 0;
|
||||
40.112.0.0/13 0;
|
||||
40.120.0.0/14 0;
|
||||
40.124.0.0/16 0;
|
||||
40.125.0.0/17 0;
|
||||
40.74.0.0/15 0;
|
||||
40.76.0.0/14 0;
|
||||
40.80.0.0/12 0;
|
||||
40.96.0.0/12 0;
|
||||
# END BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ********************
|
||||
# Cloudflare IP Ranges
|
||||
# ********************
|
||||
|
||||
# For Safety Sake all Cloudflare's Known IP Ranges are all white listed
|
||||
|
||||
# START CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
103.21.244.0/22 0;
|
||||
103.22.200.0/22 0;
|
||||
103.31.4.0/22 0;
|
||||
104.16.0.0/12 0;
|
||||
108.162.192.0/18 0;
|
||||
131.0.72.0/22 0;
|
||||
141.101.64.0/18 0;
|
||||
162.158.0.0/15 0;
|
||||
172.64.0.0/13 0;
|
||||
173.245.48.0/20 0;
|
||||
188.114.96.0/20 0;
|
||||
190.93.240.0/20 0;
|
||||
197.234.240.0/22 0;
|
||||
198.41.128.0/17 0;
|
||||
199.27.128.0/21 0;
|
||||
2400:cb00::/32 0;
|
||||
2405:8100::/32 0;
|
||||
2405:b500::/32 0;
|
||||
2606:4700::/32 0;
|
||||
2803:f800::/32 0;
|
||||
2a06:98c0::/29 0;
|
||||
2c0f:f248::/32 0;
|
||||
# END CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# *************************
|
||||
# Wordpress Theme Detectors
|
||||
# *************************
|
||||
# -------------------------------------
|
||||
# BLOCK known Wordpress Theme Detectors
|
||||
# -------------------------------------
|
||||
|
||||
# START WP THEME DETECTORS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
|
|
@ -7818,10 +7783,11 @@ geo $validate_client {
|
|||
89.36.223.188 1; #www.cuteseotools.net/wordpress-theme-detector
|
||||
# END WP THEME DETECTORS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ****************************************
|
||||
# NIBBLER - SEO testing and reporting tool
|
||||
# ****************************************
|
||||
# ----------------------------------------------
|
||||
# BLOCK NIBBLER - SEO testing and reporting tool
|
||||
# ----------------------------------------------
|
||||
# See - http://nibbler.silktide.com/
|
||||
# ----------------------------------------------
|
||||
|
||||
# START NIBBLER ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
52.201.238.175 1;
|
||||
|
|
@ -7834,33 +7800,10 @@ geo $validate_client {
|
|||
# END NIBBLER ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
|
||||
# ****************************
|
||||
# Known Bad IP's and IP Ranges
|
||||
# *************************************************
|
||||
# Blacklist IP addresses and IP Ranges Customizable
|
||||
# *************************************************
|
||||
|
||||
# BLACKLIST all your IP addresses and Ranges using the new include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# Custom Blacklisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside blacklist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
|
||||
# ********************************************************************
|
||||
# START BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ********************************************************************
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-ips.conf;
|
||||
|
||||
# ******************************************************************
|
||||
# END BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ******************************************************************
|
||||
|
||||
# *********************************************************************************************
|
||||
# KNOWN BAD IP ADDRESSES
|
||||
# Source: https://github.com/mitchellkrogza/Suspicious.Snooping.Sniffing.Hacking.IP.Addresses
|
||||
# A Reliable Daily Updated List Included Below of Known Bad IP Addresses, Snoopers and Sniffers
|
||||
# *********************************************************************************************
|
||||
# -----------------------------------------
|
||||
# BLOCK KNOWN BAD IP ADDRESSES
|
||||
# Top known bad IP Adresses from abuseIPDB
|
||||
# -----------------------------------------
|
||||
|
||||
# START KNOWN BAD IP ADDRESSES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
193.32.163.89 1;
|
||||
|
|
@ -17864,33 +17807,176 @@ geo $validate_client {
|
|||
46.101.115.65 1;
|
||||
# END KNOWN BAD IP ADDRESSES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# --------------------------
|
||||
# WHITELIST Google IP Ranges
|
||||
# --------------------------
|
||||
|
||||
# START GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
108.177.0.0/17 0;
|
||||
172.217.0.0/16 0;
|
||||
173.194.0.0/16 0;
|
||||
2001:4860:4000::/36 0;
|
||||
203.208.60.0/24 0;
|
||||
207.126.144.0/20 0;
|
||||
209.85.128.0/17 0;
|
||||
216.239.32.0/19 0;
|
||||
216.58.192.0/19 0;
|
||||
2404:6800:4000::/36 0;
|
||||
2607:f8b0:4000::/36 0;
|
||||
2800:3f0:4000::/36 0;
|
||||
2a00:1450:4000::/36 0;
|
||||
2c0f:fb50:4000::/36 0;
|
||||
35.192.0.0/12 0;
|
||||
64.18.0.0/20 0;
|
||||
64.233.160.0/19 0;
|
||||
64.68.80.0/21 0;
|
||||
65.52.0.0/14 0;
|
||||
66.102.0.0/20 0;
|
||||
66.249.64.0/19 0;
|
||||
72.14.192.0/18 0;
|
||||
74.125.0.0/16 0;
|
||||
# END GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ------------------------
|
||||
# WHITELIST Bing IP Ranges
|
||||
# ------------------------
|
||||
|
||||
# START BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
131.253.21.0/24 0;
|
||||
131.253.22.0/23 0;
|
||||
131.253.24.0/21 0;
|
||||
131.253.24.0/22 0;
|
||||
131.253.32.0/20 0;
|
||||
157.54.0.0/15 0;
|
||||
157.56.0.0/14 0;
|
||||
157.60.0.0/16 0;
|
||||
199.30.16.0/24 0;
|
||||
199.30.27.0/24 0;
|
||||
207.46.0.0/16 0;
|
||||
40.112.0.0/13 0;
|
||||
40.120.0.0/14 0;
|
||||
40.124.0.0/16 0;
|
||||
40.125.0.0/17 0;
|
||||
40.74.0.0/15 0;
|
||||
40.76.0.0/14 0;
|
||||
40.80.0.0/12 0;
|
||||
40.96.0.0/12 0;
|
||||
# END BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ------------------------------
|
||||
# WHITELIST Cloudflare IP Ranges
|
||||
# ------------------------------
|
||||
|
||||
# START CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
103.21.244.0/22 0;
|
||||
103.22.200.0/22 0;
|
||||
103.31.4.0/22 0;
|
||||
104.16.0.0/12 0;
|
||||
108.162.192.0/18 0;
|
||||
131.0.72.0/22 0;
|
||||
141.101.64.0/18 0;
|
||||
162.158.0.0/15 0;
|
||||
172.64.0.0/13 0;
|
||||
173.245.48.0/20 0;
|
||||
188.114.96.0/20 0;
|
||||
190.93.240.0/20 0;
|
||||
197.234.240.0/22 0;
|
||||
198.41.128.0/17 0;
|
||||
199.27.128.0/21 0;
|
||||
2400:cb00::/32 0;
|
||||
2405:8100::/32 0;
|
||||
2405:b500::/32 0;
|
||||
2606:4700::/32 0;
|
||||
2803:f800::/32 0;
|
||||
2a06:98c0::/29 0;
|
||||
2c0f:f248::/32 0;
|
||||
# END CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# -------------------------------------------------
|
||||
# BLACKLIST IP addresses and IP Ranges Customizable
|
||||
# -------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# BLACKLIST all your IP addresses and Ranges using the new include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# Custom Blacklisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside blacklist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# Anything added to blacklist-ips.conf will over-ride anything whitelisted above
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# START BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-ips.conf;
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# END BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
# ----------------------------------------------
|
||||
# Whitelist all your OWN IP addresses and Ranges
|
||||
# ----------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST all your own IP addresses using the include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside whitelist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# whitelist-ips.conf reigns supreme !!!
|
||||
# Whatever you add to whitelist-ips.conf will be whitelisted FULL STOP
|
||||
# Anything blacklisted above this line will be over-ridden by whitelist-ips.conf
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# ------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST your own IPs from the DDOS Filter
|
||||
# Add your own IP addresses and ranges into the custom include file whitelist-ips.conf
|
||||
# to spare them from the rate limiting DDOS filter.
|
||||
# This section includes the same / single whitelist-ips.conf file so you only
|
||||
# need to edit that include file and have it include here for you too.
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
geo $ratelimited {
|
||||
default 1;
|
||||
|
||||
# ***************************************************************************
|
||||
# ---------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES2 ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***************************************************************************
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES2 ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
|
||||
# *****************************************
|
||||
# MAP BAD BOTS TO OUR RATE LIMITER FUNCTION
|
||||
# *****************************************
|
||||
# ======================================================================
|
||||
# END SECTION 3 - WHITELISTING AND BLACKLISTING IP ADDRESSESE AND RANGES
|
||||
# ======================================================================
|
||||
|
||||
|
||||
# ============================================
|
||||
# BEGIN SECTION 4 - ACTIVATE BLOCKER FUNCTIONS
|
||||
# ============================================
|
||||
|
||||
# --------------------------------------------
|
||||
# 1. MAP BAD BOTS TO OUR RATE LIMITER FUNCTION
|
||||
# --------------------------------------------
|
||||
|
||||
map $bad_bot $bot_iplimit {
|
||||
0 "";
|
||||
|
|
@ -17898,30 +17984,31 @@ geo $ratelimited {
|
|||
2 $binary_remote_addr;
|
||||
}
|
||||
|
||||
# ***********************
|
||||
# SET RATE LIMITING ZONES
|
||||
# ***********************
|
||||
# --------------------------
|
||||
# 2. SET RATE LIMITING ZONES
|
||||
# --------------------------
|
||||
|
||||
# BAD BOT RATE LIMITING ZONE
|
||||
# limits for Zone $bad_bot = 1
|
||||
# Nothing Set - you can set a different zone limiter here if you like
|
||||
# We issue a 444 response instead to all bad bots.
|
||||
# Rate limiting will only take effect if on any User-Agents with a value of 2
|
||||
|
||||
# limits for Zone $bad_bot = 2
|
||||
# this rate limiting will only take effect if you change any of the bots and change
|
||||
# their block value from 1 to 2.
|
||||
limit_conn_zone $bot_iplimit zone=bot2_connlimit:16m;
|
||||
limit_req_zone $bot_iplimit zone=bot2_reqlimitip:16m rate=2r/s;
|
||||
limit_req_zone $bot_iplimit zone=bot2_reqlimitip:16m rate=6r/m;
|
||||
|
||||
### *** MAKE SURE TO ADD to your nginx.conf ***
|
||||
### server_names_hash_bucket_size 64;
|
||||
### server_names_hash_max_size 4096;
|
||||
### limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
|
||||
### limit_conn_zone $binary_remote_addr zone=addr:50m;
|
||||
### to allow it to load this large set of domains into memory and to set the rate limiting zones for the DDOS filter.
|
||||
# ==========================================
|
||||
# END SECTION 4 - ACTIVATE BLOCKER FUNCTIONS
|
||||
# ==========================================
|
||||
|
||||
### THE END of the Long and Winding Road
|
||||
# =====================
|
||||
# END BLOCKER FUNCTIONS
|
||||
# =====================
|
||||
|
||||
### Also check out my Ultimate Apache Bad Bot Blocker on Github
|
||||
### https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker
|
||||
### --------------------------------------------
|
||||
### HELP SUPPORT THIS PROJECT - Send Me a Coffee
|
||||
### https://ko-fi.com/mitchellkrog
|
||||
### --------------------------------------------
|
||||
|
||||
### FOR APACHE SERVERS
|
||||
### ---------------------------------------------
|
||||
### Check out the Ultimate Apache Bad Bot Blocker
|
||||
### ---------------------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -1,2 +1,2 @@
|
|||
35.192.85.2 1;
|
||||
104.154.120.187 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
35.192.85.2 0;
|
||||
104.154.120.187 0;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
|
|
@ -1,16 +1,21 @@
|
|||
### **********************************
|
||||
### THE ULTIMATE NGINX BAD BOT BLOCKER
|
||||
### **********************************
|
||||
### -----------------------------------------------------------
|
||||
### THE NGINX ULTIMATE BAD BOT, BAD IP AND BAD REFERRER BLOCKER
|
||||
### -----------------------------------------------------------
|
||||
|
||||
### VERSION INFORMATION #
|
||||
###################################################
|
||||
### Version: V4.2019.06.1643
|
||||
### Updated: Thu Jun 27 12:59:42 SAST 2019
|
||||
### Version: V3.2019.06.1645
|
||||
### Updated: Thu Jun 27 13:24:56 SAST 2019
|
||||
### Bad Referrer Count: 6713
|
||||
### Bad Bot Count: 556
|
||||
###################################################
|
||||
### VERSION INFORMATION ##
|
||||
|
||||
### --------------------------------------------
|
||||
### HELP SUPPORT THIS PROJECT - Send Me a Coffee
|
||||
### https://ko-fi.com/mitchellkrog
|
||||
### --------------------------------------------
|
||||
|
||||
##############################################################################
|
||||
# _ __ _ #
|
||||
# / |/ /__ _(_)__ __ __ #
|
||||
|
|
@ -28,12 +33,16 @@
|
|||
### and also has rate limiting functionality for bad bots who you only want to rate limit
|
||||
### and not actually block out entirely. It is very powerful and also very flexible.
|
||||
|
||||
### --------------------------------------------------------------------------
|
||||
### Created By: https://github.com/mitchellkrogza/
|
||||
### Repo Url: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker
|
||||
### Copyright Mitchell Krog - <mitchellkrog@gmail.com>
|
||||
### Contributors: Stuart Cardall - https://github.com/itoffshore
|
||||
### --------------------------------------------------------------------------
|
||||
|
||||
### --------------------------------------------------------------------------
|
||||
### Tested on: nginx/1.10.3 up to latest Mainstream Version (Ubuntu 16.04)
|
||||
### --------------------------------------------------------------------------
|
||||
|
||||
### This list was developed and is in use on a live Nginx server running some very busy web sites.
|
||||
### It was built from the ground up using real data from daily logs and is updated almost daily.
|
||||
|
|
@ -41,9 +50,9 @@
|
|||
### spam referrers, rogue IP address, scanners, scrapers and domain hijacking sites are extensively checked
|
||||
### before they are added. It is monitored extensively for any false positives.
|
||||
|
||||
### *********
|
||||
### ---------
|
||||
### Features:
|
||||
### *********
|
||||
### ---------
|
||||
### Clear formatting for Ease of Maintenance.
|
||||
### Alphabetically ordered lists for Ease of Maintenance.
|
||||
### Extensive Commenting for Ease of Reference.
|
||||
|
|
@ -54,40 +63,38 @@
|
|||
### IP range blocking / whitelisting.
|
||||
### Rate Limiting Functions.
|
||||
|
||||
### ************
|
||||
### ------------
|
||||
### INSTALLATION
|
||||
### ************
|
||||
### ------------
|
||||
|
||||
### PLEASE use the install, setup and update scripts provided for you to ease your installation.
|
||||
### This Auto Installation procedure is documented in the README.md and AUTO-CONFIGURATION.md files.
|
||||
### Installation, Setup and Update Scripts Contributed by Stuart Cardall - https://github.com/itoffshore
|
||||
### There are also manual configuration instructions provided for those not wishing to do an auto install.
|
||||
|
||||
### ***********************************************
|
||||
### -----------------------------------------------
|
||||
### !!!!! PLEASE READ INLINE NOTES ON TESTING !!!!!
|
||||
### ***********************************************
|
||||
### -----------------------------------------------
|
||||
|
||||
### Note that:
|
||||
### SETTINGS:
|
||||
### ---------------------------------------------
|
||||
### 0 = allowed - no limits
|
||||
### 1 = allowed or rate limited less restrictive
|
||||
### 2 = rate limited more
|
||||
### 3 = block completely
|
||||
### ---------------------------------------------
|
||||
|
||||
### ************************************************************
|
||||
### ------------------------------------------------------------
|
||||
### CONTRIBUTING / PULL REQUESTS / ADDING YOUR OWN BAD REFERRERS
|
||||
### ************************************************************
|
||||
### ------------------------------------------------------------
|
||||
|
||||
### For contributing, corrections or adding bots or referrers to this repo,
|
||||
### Send a Pull Request (PR) on any of the .list files in the _generator_lists folder
|
||||
### All Pull Requests will be checked for accuracy before being merged.
|
||||
|
||||
# *********************************
|
||||
# FIRST BLOCK BY USER-AGENT STRINGS
|
||||
# *********************************
|
||||
|
||||
# ***********************
|
||||
# -----------------------
|
||||
# !!!!! PLEASE TEST !!!!!
|
||||
# ***********************
|
||||
# -----------------------
|
||||
|
||||
# ALWAYS test any User-Agent Strings you add here to make sure you have it right
|
||||
# Use a Chrome Extension called "User-Agent Switcher for Chrome" where you can create your
|
||||
|
|
@ -98,36 +105,46 @@
|
|||
# curl -I http://www.yourdomain.com -A "GoogleBot" ---- GIVES YOU: HTTP/1.1 200 OK (Meaning web page was served to Client)
|
||||
# curl -I http://www.yourdomain.com -A "80legs" ---- GIVES YOU: curl: (52) Empty reply from server (Meaning Nginx gave a 444 Dropped Connection)
|
||||
|
||||
# In this section we allow/disallow specific User Agents / Bots.
|
||||
|
||||
# *********************************************************************
|
||||
# Now we map all good and bad user agents to a variable called $bad_bot
|
||||
# *********************************************************************
|
||||
# =======================
|
||||
# START BLOCKER FUNCTIONS
|
||||
# =======================
|
||||
|
||||
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
# DO NOT EDIT ANYTHING BELOW THIS LINE !!!
|
||||
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
|
||||
# =============================
|
||||
# BEGIN SECTION 1 - USER-AGENTS
|
||||
# =============================
|
||||
|
||||
# ALLOW / BLOCK User Agents / Bots
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Map all GOOD and BAD UA (User Agents) to a variable called $bad_bot
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
map $http_user_agent $bad_bot {
|
||||
|
||||
default 0;
|
||||
|
||||
# ***********************************************
|
||||
# Include your Own Custom List of Bad User Agents
|
||||
# ***********************************************
|
||||
|
||||
# Use the include file below to further customize your own list of additional user-agents you wish to permanently block
|
||||
# This include file allows whitelisting and blacklisting of anything specified below it, meaning you can over-ride anything in this blocker to your liking.
|
||||
|
||||
# ***********************************************************************************
|
||||
# -----------------------------------------------------------------------------------
|
||||
# START CUSTOM BLACKLISTED USER AGENTS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***********************************************************************************
|
||||
# -----------------------------------------------------------------------------------
|
||||
|
||||
# Include your Own Custom List of Bad User Agents
|
||||
# Use the include file below to further customize your own list of additional user-agents you wish to permanently block
|
||||
# This include file allows whitelisting and blacklisting of anything specified below it.
|
||||
# This include file alows you to over-ride any Bad / Good UA (Bot) declared in this blocker to your liking.
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-user-agents.conf;
|
||||
|
||||
# *********************************************************************************
|
||||
# ---------------------------------------------------------------------------------
|
||||
# END CUSTOM BLACKLISTED USER AGENTS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *********************************************************************************
|
||||
# ---------------------------------------------------------------------------------
|
||||
|
||||
# *********************************************
|
||||
# Bad User-Agent Strings That We Block Outright
|
||||
# *********************************************
|
||||
# --------------------------------------------------
|
||||
# BAD UA (User-Agent) Strings That We Block Outright
|
||||
# --------------------------------------------------
|
||||
|
||||
# START BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)360Spider(?:\b|)" 3;
|
||||
|
|
@ -688,9 +705,16 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)ZyBorg(?:\b|)" 3;
|
||||
# END BAD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ***********************************************
|
||||
# Allow Good User-Agent Strings We Know and Trust
|
||||
# ***********************************************
|
||||
# --------------------------------------------
|
||||
# GOOD UA User-Agent Strings We Know and Trust
|
||||
# --------------------------------------------
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# You can over-ride these in /etc/nginx/bots.d/blacklist-user-agents.conf
|
||||
# by adding the same UA line there and chaning its value of 1
|
||||
# If you think GoogleBot is bad you would simply add them to
|
||||
# blacklist-user-agents.conf with a value of 1
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)adidxbot(?:\b|)" 0;
|
||||
|
|
@ -727,12 +751,13 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)yahoo(?:\b|)" 0;
|
||||
# END GOOD BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ***************************************************
|
||||
# User-Agent Strings Allowed Through but Rate Limited
|
||||
# ***************************************************
|
||||
# --------------------------------------------------------
|
||||
# GOOD UA User-Agent Rate Limiting 1 - Disabled by Default
|
||||
# --------------------------------------------------------
|
||||
|
||||
# Some people block libwww-perl, it used widely in many valid (non rogue) agents
|
||||
# I allow libwww-perl as I use it for monitoring systems with Munin but it is rate limited
|
||||
# TO ACTIVATE THIS RATE LIMITING Uncomment these two lines in blockbots.conf
|
||||
#limit_conn bot1_connlimit 100;
|
||||
#limit_req zone=bot1_reqlimitip burst=50;
|
||||
|
||||
# START ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)jetmon(?:\b|)" 1;
|
||||
|
|
@ -743,9 +768,14 @@ map $http_user_agent $bad_bot {
|
|||
"~*(?:\b)Wget/1.15(?:\b|)" 1;
|
||||
# END ALLOWED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# **************************************************************
|
||||
# Rate Limited User-Agents who get a bit aggressive on bandwidth
|
||||
# **************************************************************
|
||||
# -------------------------------------------------------
|
||||
# GOOD UA User-Agent Rate Limiting 2 - Enabled by Default
|
||||
# -------------------------------------------------------
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# You can over-ride these in /etc/nginx/bots.d/blacklist-user-agents.conf
|
||||
# by adding the same UA line there and chaning its value of 1
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)Alexa(?:\b|)" 2;
|
||||
|
|
@ -768,163 +798,186 @@ map $http_user_agent $bad_bot {
|
|||
# END LIMITED BOTS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
}
|
||||
|
||||
|
||||
# *****************************************
|
||||
# SECOND BLOCK BY REFERRER STRINGS AND URLS
|
||||
# *****************************************
|
||||
# ===========================
|
||||
# END SECTION 1 - USER-AGENTS
|
||||
# ===========================
|
||||
|
||||
# *****************
|
||||
# =======================================
|
||||
# BEGIN SECTION 2 - REFERRERS AND DOMAINS
|
||||
# =======================================
|
||||
|
||||
# ----------------
|
||||
# PLEASE TEST !!!!
|
||||
# *****************
|
||||
|
||||
# ----------------
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# ALWAYS test referrers that you add. This is done manually as follows
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://anything.adcash.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://www.goodwebsite.com/not-adcash --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://www.betterwebsite.com/not/adcash --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://www.google.com --- GIVES YOU: full html output of the web page
|
||||
# curl -I http://www.yourdomain.com -e http://www.microsoft.com --- GIVES YOU: full html output of the web page
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
# Because of case-insensitive matching any combination of capitilization in the names will all produce a positive hit
|
||||
# make sure you always test thoroughly and monitor logs. This section below also does NOT check for a preceding www.
|
||||
# and it also does not care if the referrer request was sent with http https or even ftp.
|
||||
# ------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
# ***********************************************************************
|
||||
# Now we map all bad referrer words below to a variable called $bad_words
|
||||
# ***********************************************************************
|
||||
# ----------------------------------------------------------------
|
||||
# Map all BAD referrer words below to a variable called $bad_words
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
# --------------------------------
|
||||
# START Bad Referrer Word Scanning
|
||||
# --------------------------------
|
||||
|
||||
map $http_referer $bad_words {
|
||||
default 0;
|
||||
|
||||
# **************************
|
||||
# Bad Referrer Word Scanning
|
||||
# **************************
|
||||
|
||||
# -------------------------------------------------------------------------------------------
|
||||
# These are Words and Terms often found tagged onto domains or within url query strings.
|
||||
# Create and Customize Your Own Bad Referrer Words Here using the new Include File Method
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# customized list of bad referrer words are automatically now included for you
|
||||
# Read Comments inside bad-referrer-words.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# BE VERY CAREFUL using the bad-referrer-words.conf file - please read the comment and
|
||||
# BE VERY CAREFUL using this bad-referrer-words.conf file - please read the comments and
|
||||
# examples inside the include file for detailed explanations into how seriously this can
|
||||
# affect your own site from serving assets or other innocent sites from accessing your site
|
||||
# For safety sake the whitelist-domains.conf file is also loaded here before the bad-referrer-words.conf file.
|
||||
# For safety sake the whitelist-domains.conf file is also loaded here before the
|
||||
# bad-referrer-words.conf file is loaded.
|
||||
# -------------------------------------------------------------------------------------------
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# START WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-domains.conf;
|
||||
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
# END WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
# ******************************************************************************
|
||||
# ------------------------------------------------------------------------------
|
||||
# START CUSTOM BAD REFERRER WORDS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ******************************************************************************
|
||||
# ------------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/bad-referrer-words.conf;
|
||||
|
||||
# ****************************************************************************
|
||||
# ----------------------------------------------------------------------------
|
||||
# END CUSTOM BAD REFERRER WORDS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ****************************************************************************
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
# --------------------------------
|
||||
# END Bad Referrer Word Scanning
|
||||
# --------------------------------
|
||||
|
||||
# *************************
|
||||
# Bad Referrer Domain Names
|
||||
# *************************
|
||||
|
||||
# Now a list of bad referrer urls these domains or any combination of them ie .com .net
|
||||
# will be blocked out. Doesn't matter if the protocol is http, https or even ftp
|
||||
# ----------------------------------------
|
||||
# START Good and Bad Referrer Domain Names
|
||||
# ----------------------------------------
|
||||
|
||||
# -------------------------------------------------------------------------------------
|
||||
# Good and Bad referrer urls Doesn't matter if the protocol is http, https or even ftp
|
||||
# -------------------------------------------------------------------------------------
|
||||
|
||||
# ----------------------
|
||||
# This section includes:
|
||||
# **********************
|
||||
# ----------------------
|
||||
# --------------------------------------------------------------------------------
|
||||
# Blocking of SEO company Semalt.com (now merged into this one section)
|
||||
# MIRAI Botnet Domains Used for Mass Attacks
|
||||
# Other known bad SEO companies and Ad Hijacking Sites
|
||||
# Sites linked to malware, adware, clickjacking and ransomware
|
||||
# Domain names and referrers used in referrer spam and seo hijacking
|
||||
# Whitelisting of your own GOOD domains / referrers
|
||||
# Whitelisting of any other GOOD domains / referrers you want explicitly NOT block
|
||||
# --------------------------------------------------------------------------------
|
||||
|
||||
# *****************
|
||||
# ----------------
|
||||
# PLEASE TEST !!!!
|
||||
# *****************
|
||||
|
||||
# ----------------
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# ALWAYS test referrers that you add. This is done manually as follows
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# Because of case-insensitive matching any combination of capitilization will all produce a positive hit - make sure you always test.
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# For Example any of the following variations below of 8gold.com will be detected and blocked
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# curl -I http://www.yourdomain.com -e http://NOT-8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e http://this.is.not8gOlD.net --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e ftp://8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e ftp://www.weare8gold.NET --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e https://subdomain.8gold.com --- GIVES YOU: curl: (52) Empty reply from server
|
||||
# curl -I http://www.yourdomain.com -e https://NOT8GolD.org --- GIVES YOU: curl: (52) Empty reply from server
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# So if you see a bad referrer from wearegoogle.com and you want to block them just add
|
||||
# them as "~*wearegoogle.com" don't ever go and do something like "~*google(-|.)" you will
|
||||
# kill all your SEO in a week.
|
||||
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
# To add your own custom bad referrers use the custom include file
|
||||
# /etc/nginx/bots.d/custom-bad-referrers.conf
|
||||
# Or send a Pull Request to add it to the global blacklist for other users.
|
||||
|
||||
# In the bad referrers section I also include sites that hotlink images without permission.
|
||||
# ------------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
# ***********************************************************************
|
||||
# Now we map all good & bad referrer urls to variable called #bad_referer
|
||||
# ***********************************************************************
|
||||
# --------------------------------------------------------------------
|
||||
# Map all good & bad referrer DOMAINS to a variable called bad_referer
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
map $http_referer $bad_referer {
|
||||
hostnames;
|
||||
default 0;
|
||||
|
||||
# *************************************
|
||||
# GOOD REFERRERS - Spared from Checking
|
||||
# *************************************
|
||||
# --------------------------------------------
|
||||
# GOOD REFERRER DOMAINS - Spared from Checking
|
||||
# --------------------------------------------
|
||||
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# Add all your own web site domain names and server names in this section
|
||||
|
||||
# WHITELIST Your Own Domain Names Here using the new Include File Method
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted domain names are automatically now included for you.
|
||||
# Read Comments inside whitelist-domains.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# ---------------------------------------------------------------------------------------
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# START WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-domains.conf;
|
||||
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
# END WHITELISTED DOMAINS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **********************************************************************
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
# ***********************************
|
||||
# -----------------------------------
|
||||
# CUSTOM BAD REFERRERS - Add your Own
|
||||
# ***********************************
|
||||
# -----------------------------------
|
||||
|
||||
# Add any extra bad referrers in the following include file to have them
|
||||
# permanently included and blocked - avoid duplicates in your custom file
|
||||
# custom-bad-referrers.conf is BOTH a BLACKLIST AND WHITELIST
|
||||
# custom-bad-referrers.conf ALLOWS complete over-riding of anything
|
||||
# If you think google.com is bad you would simply add them to
|
||||
# custom-bad-referrers.conf with a value of 1
|
||||
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
# START CUSTOM BAD REFERRERS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/custom-bad-referrers.conf;
|
||||
|
||||
# ***********************************************************************
|
||||
# -----------------------------------------------------------------------
|
||||
# END CUSTOM BAD REFERRERS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***********************************************************************
|
||||
# -----------------------------------------------------------------------
|
||||
|
||||
# START BAD REFERRERS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
"~*(?:\b)000free\.us(?:\b|)" 1;
|
||||
|
|
@ -7644,137 +7697,49 @@ map $http_referer $bad_referer {
|
|||
# END BAD REFERRERS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
}
|
||||
|
||||
# =====================================
|
||||
# END SECTION 2 - REFERRERS AND DOMAINS
|
||||
# =====================================
|
||||
|
||||
# ***********************************************
|
||||
# WHITELISTING AND BLACKLISTING IP ADDRESS RANGES
|
||||
# ***********************************************
|
||||
# ========================================================================
|
||||
# BEGIN SECTION 3 - WHITELISTING AND BLACKLISTING IP ADDRESSESE AND RANGES
|
||||
# ========================================================================
|
||||
|
||||
# Geo directive to deny and also whitelist certain ip addresses
|
||||
# --------------------------------------------------------------------------------------
|
||||
# Map all GOOD and BAD IP Addresses and Ranges to a variable called geo $validate_client
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
geo $validate_client {
|
||||
|
||||
# ********************
|
||||
# First Our Safety Net
|
||||
# ********************
|
||||
|
||||
# Anything not matching our rules is allowed through with default 0;
|
||||
|
||||
default 0;
|
||||
|
||||
# ***********************************
|
||||
# Whitelist all your OWN IP addresses
|
||||
# ***********************************
|
||||
# ----------------------------------------------
|
||||
# Whitelist all your OWN IP addresses and Ranges
|
||||
# ----------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST all your own IP addresses using the include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside whitelist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# whitelist-ips.conf reigns supreme !!!
|
||||
# Whatever you add to whitelist-ips.conf will be whitelisted FULL STOP
|
||||
# Anything blacklisted above this line will be over-ridden by whitelist-ips.conf
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# **************************************************************************
|
||||
# --------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# **************************************************************************
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ************************************************************************
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
# ****************
|
||||
# Google IP Ranges
|
||||
# ****************
|
||||
|
||||
# For Safety Sake all Google's Known IP Ranges are all white listed
|
||||
|
||||
# START GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
108.177.0.0/17 0;
|
||||
172.217.0.0/16 0;
|
||||
173.194.0.0/16 0;
|
||||
2001:4860:4000::/36 0;
|
||||
203.208.60.0/24 0;
|
||||
207.126.144.0/20 0;
|
||||
209.85.128.0/17 0;
|
||||
216.239.32.0/19 0;
|
||||
216.58.192.0/19 0;
|
||||
2404:6800:4000::/36 0;
|
||||
2607:f8b0:4000::/36 0;
|
||||
2800:3f0:4000::/36 0;
|
||||
2a00:1450:4000::/36 0;
|
||||
2c0f:fb50:4000::/36 0;
|
||||
35.192.0.0/12 0;
|
||||
64.18.0.0/20 0;
|
||||
64.233.160.0/19 0;
|
||||
64.68.80.0/21 0;
|
||||
65.52.0.0/14 0;
|
||||
66.102.0.0/20 0;
|
||||
66.249.64.0/19 0;
|
||||
72.14.192.0/18 0;
|
||||
74.125.0.0/16 0;
|
||||
# END GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# **************
|
||||
# Bing IP Ranges
|
||||
# **************
|
||||
|
||||
# For Safety Sake all Bing's Known IP Ranges are all white listed
|
||||
|
||||
# START BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
131.253.21.0/24 0;
|
||||
131.253.22.0/23 0;
|
||||
131.253.24.0/21 0;
|
||||
131.253.24.0/22 0;
|
||||
131.253.32.0/20 0;
|
||||
157.54.0.0/15 0;
|
||||
157.56.0.0/14 0;
|
||||
157.60.0.0/16 0;
|
||||
199.30.16.0/24 0;
|
||||
199.30.27.0/24 0;
|
||||
207.46.0.0/16 0;
|
||||
40.112.0.0/13 0;
|
||||
40.120.0.0/14 0;
|
||||
40.124.0.0/16 0;
|
||||
40.125.0.0/17 0;
|
||||
40.74.0.0/15 0;
|
||||
40.76.0.0/14 0;
|
||||
40.80.0.0/12 0;
|
||||
40.96.0.0/12 0;
|
||||
# END BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ********************
|
||||
# Cloudflare IP Ranges
|
||||
# ********************
|
||||
|
||||
# For Safety Sake all Cloudflare's Known IP Ranges are all white listed
|
||||
|
||||
# START CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
103.21.244.0/22 0;
|
||||
103.22.200.0/22 0;
|
||||
103.31.4.0/22 0;
|
||||
104.16.0.0/12 0;
|
||||
108.162.192.0/18 0;
|
||||
131.0.72.0/22 0;
|
||||
141.101.64.0/18 0;
|
||||
162.158.0.0/15 0;
|
||||
172.64.0.0/13 0;
|
||||
173.245.48.0/20 0;
|
||||
188.114.96.0/20 0;
|
||||
190.93.240.0/20 0;
|
||||
197.234.240.0/22 0;
|
||||
198.41.128.0/17 0;
|
||||
199.27.128.0/21 0;
|
||||
2400:cb00::/32 0;
|
||||
2405:8100::/32 0;
|
||||
2405:b500::/32 0;
|
||||
2606:4700::/32 0;
|
||||
2803:f800::/32 0;
|
||||
2a06:98c0::/29 0;
|
||||
2c0f:f248::/32 0;
|
||||
# END CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# *************************
|
||||
# Wordpress Theme Detectors
|
||||
# *************************
|
||||
# -------------------------------------
|
||||
# BLOCK known Wordpress Theme Detectors
|
||||
# -------------------------------------
|
||||
|
||||
# START WP THEME DETECTORS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
|
|
@ -7818,10 +7783,11 @@ geo $validate_client {
|
|||
89.36.223.188 1; #www.cuteseotools.net/wordpress-theme-detector
|
||||
# END WP THEME DETECTORS ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ****************************************
|
||||
# NIBBLER - SEO testing and reporting tool
|
||||
# ****************************************
|
||||
# ----------------------------------------------
|
||||
# BLOCK NIBBLER - SEO testing and reporting tool
|
||||
# ----------------------------------------------
|
||||
# See - http://nibbler.silktide.com/
|
||||
# ----------------------------------------------
|
||||
|
||||
# START NIBBLER ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
52.201.238.175 1;
|
||||
|
|
@ -7834,33 +7800,10 @@ geo $validate_client {
|
|||
# END NIBBLER ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
|
||||
# ****************************
|
||||
# Known Bad IP's and IP Ranges
|
||||
# *************************************************
|
||||
# Blacklist IP addresses and IP Ranges Customizable
|
||||
# *************************************************
|
||||
|
||||
# BLACKLIST all your IP addresses and Ranges using the new include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# Custom Blacklisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside blacklist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
|
||||
# ********************************************************************
|
||||
# START BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ********************************************************************
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-ips.conf;
|
||||
|
||||
# ******************************************************************
|
||||
# END BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ******************************************************************
|
||||
|
||||
# *********************************************************************************************
|
||||
# KNOWN BAD IP ADDRESSES
|
||||
# Source: https://github.com/mitchellkrogza/Suspicious.Snooping.Sniffing.Hacking.IP.Addresses
|
||||
# A Reliable Daily Updated List Included Below of Known Bad IP Addresses, Snoopers and Sniffers
|
||||
# *********************************************************************************************
|
||||
# -----------------------------------------
|
||||
# BLOCK KNOWN BAD IP ADDRESSES
|
||||
# Top known bad IP Adresses from abuseIPDB
|
||||
# -----------------------------------------
|
||||
|
||||
# START KNOWN BAD IP ADDRESSES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
193.32.163.89 1;
|
||||
|
|
@ -17864,33 +17807,176 @@ geo $validate_client {
|
|||
46.101.115.65 1;
|
||||
# END KNOWN BAD IP ADDRESSES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# --------------------------
|
||||
# WHITELIST Google IP Ranges
|
||||
# --------------------------
|
||||
|
||||
# START GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
108.177.0.0/17 0;
|
||||
172.217.0.0/16 0;
|
||||
173.194.0.0/16 0;
|
||||
2001:4860:4000::/36 0;
|
||||
203.208.60.0/24 0;
|
||||
207.126.144.0/20 0;
|
||||
209.85.128.0/17 0;
|
||||
216.239.32.0/19 0;
|
||||
216.58.192.0/19 0;
|
||||
2404:6800:4000::/36 0;
|
||||
2607:f8b0:4000::/36 0;
|
||||
2800:3f0:4000::/36 0;
|
||||
2a00:1450:4000::/36 0;
|
||||
2c0f:fb50:4000::/36 0;
|
||||
35.192.0.0/12 0;
|
||||
64.18.0.0/20 0;
|
||||
64.233.160.0/19 0;
|
||||
64.68.80.0/21 0;
|
||||
65.52.0.0/14 0;
|
||||
66.102.0.0/20 0;
|
||||
66.249.64.0/19 0;
|
||||
72.14.192.0/18 0;
|
||||
74.125.0.0/16 0;
|
||||
# END GOOGLE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ------------------------
|
||||
# WHITELIST Bing IP Ranges
|
||||
# ------------------------
|
||||
|
||||
# START BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
131.253.21.0/24 0;
|
||||
131.253.22.0/23 0;
|
||||
131.253.24.0/21 0;
|
||||
131.253.24.0/22 0;
|
||||
131.253.32.0/20 0;
|
||||
157.54.0.0/15 0;
|
||||
157.56.0.0/14 0;
|
||||
157.60.0.0/16 0;
|
||||
199.30.16.0/24 0;
|
||||
199.30.27.0/24 0;
|
||||
207.46.0.0/16 0;
|
||||
40.112.0.0/13 0;
|
||||
40.120.0.0/14 0;
|
||||
40.124.0.0/16 0;
|
||||
40.125.0.0/17 0;
|
||||
40.74.0.0/15 0;
|
||||
40.76.0.0/14 0;
|
||||
40.80.0.0/12 0;
|
||||
40.96.0.0/12 0;
|
||||
# END BING IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# ------------------------------
|
||||
# WHITELIST Cloudflare IP Ranges
|
||||
# ------------------------------
|
||||
|
||||
# START CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
103.21.244.0/22 0;
|
||||
103.22.200.0/22 0;
|
||||
103.31.4.0/22 0;
|
||||
104.16.0.0/12 0;
|
||||
108.162.192.0/18 0;
|
||||
131.0.72.0/22 0;
|
||||
141.101.64.0/18 0;
|
||||
162.158.0.0/15 0;
|
||||
172.64.0.0/13 0;
|
||||
173.245.48.0/20 0;
|
||||
188.114.96.0/20 0;
|
||||
190.93.240.0/20 0;
|
||||
197.234.240.0/22 0;
|
||||
198.41.128.0/17 0;
|
||||
199.27.128.0/21 0;
|
||||
2400:cb00::/32 0;
|
||||
2405:8100::/32 0;
|
||||
2405:b500::/32 0;
|
||||
2606:4700::/32 0;
|
||||
2803:f800::/32 0;
|
||||
2a06:98c0::/29 0;
|
||||
2c0f:f248::/32 0;
|
||||
# END CLOUDFLARE IP RANGES ### DO NOT EDIT THIS LINE AT ALL ###
|
||||
|
||||
# -------------------------------------------------
|
||||
# BLACKLIST IP addresses and IP Ranges Customizable
|
||||
# -------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# BLACKLIST all your IP addresses and Ranges using the new include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# Custom Blacklisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside blacklist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# Anything added to blacklist-ips.conf will over-ride anything whitelisted above
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# START BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/blacklist-ips.conf;
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# END BLACKLISTED IPS ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
# ----------------------------------------------
|
||||
# Whitelist all your OWN IP addresses and Ranges
|
||||
# ----------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST all your own IP addresses using the include file below.
|
||||
# New Method Uses the include file below so that when pulling future updates your
|
||||
# whitelisted IP addresses are automatically now included for you.
|
||||
# Read Comments inside whitelist-ips.conf for customization tips.
|
||||
# Updating the main globalblacklist.conf file will not touch your custom include files
|
||||
# whitelist-ips.conf reigns supreme !!!
|
||||
# Whatever you add to whitelist-ips.conf will be whitelisted FULL STOP
|
||||
# Anything blacklisted above this line will be over-ridden by whitelist-ips.conf
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# ------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
|
||||
# --------------------------------------------------------------------------------------
|
||||
# WHITELIST your own IPs from the DDOS Filter
|
||||
# Add your own IP addresses and ranges into the custom include file whitelist-ips.conf
|
||||
# to spare them from the rate limiting DDOS filter.
|
||||
# This section includes the same / single whitelist-ips.conf file so you only
|
||||
# need to edit that include file and have it include here for you too.
|
||||
# --------------------------------------------------------------------------------------
|
||||
|
||||
geo $ratelimited {
|
||||
default 1;
|
||||
|
||||
# ***************************************************************************
|
||||
# ---------------------------------------------------------------------------
|
||||
# START WHITELISTED IP RANGES2 ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# ***************************************************************************
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
include /etc/nginx/bots.d/whitelist-ips.conf;
|
||||
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
# END WHITELISTED IP RANGES2 ### DO NOT EDIT OR REMOVE THIS LINE AT ALL ###
|
||||
# *************************************************************************
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
}
|
||||
|
||||
# *****************************************
|
||||
# MAP BAD BOTS TO OUR RATE LIMITER FUNCTION
|
||||
# *****************************************
|
||||
# ======================================================================
|
||||
# END SECTION 3 - WHITELISTING AND BLACKLISTING IP ADDRESSESE AND RANGES
|
||||
# ======================================================================
|
||||
|
||||
|
||||
# ============================================
|
||||
# BEGIN SECTION 4 - ACTIVATE BLOCKER FUNCTIONS
|
||||
# ============================================
|
||||
|
||||
# --------------------------------------------
|
||||
# 1. MAP BAD BOTS TO OUR RATE LIMITER FUNCTION
|
||||
# --------------------------------------------
|
||||
|
||||
map $bad_bot $bot_iplimit {
|
||||
0 "";
|
||||
|
|
@ -17898,30 +17984,31 @@ geo $ratelimited {
|
|||
2 $binary_remote_addr;
|
||||
}
|
||||
|
||||
# ***********************
|
||||
# SET RATE LIMITING ZONES
|
||||
# ***********************
|
||||
# --------------------------
|
||||
# 2. SET RATE LIMITING ZONES
|
||||
# --------------------------
|
||||
|
||||
# BAD BOT RATE LIMITING ZONE
|
||||
# limits for Zone $bad_bot = 1
|
||||
# Nothing Set - you can set a different zone limiter here if you like
|
||||
# We issue a 444 response instead to all bad bots.
|
||||
# Rate limiting will only take effect if on any User-Agents with a value of 2
|
||||
|
||||
# limits for Zone $bad_bot = 2
|
||||
# this rate limiting will only take effect if you change any of the bots and change
|
||||
# their block value from 1 to 2.
|
||||
limit_conn_zone $bot_iplimit zone=bot2_connlimit:16m;
|
||||
limit_req_zone $bot_iplimit zone=bot2_reqlimitip:16m rate=2r/s;
|
||||
limit_req_zone $bot_iplimit zone=bot2_reqlimitip:16m rate=6r/m;
|
||||
|
||||
### *** MAKE SURE TO ADD to your nginx.conf ***
|
||||
### server_names_hash_bucket_size 64;
|
||||
### server_names_hash_max_size 4096;
|
||||
### limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
|
||||
### limit_conn_zone $binary_remote_addr zone=addr:50m;
|
||||
### to allow it to load this large set of domains into memory and to set the rate limiting zones for the DDOS filter.
|
||||
# ==========================================
|
||||
# END SECTION 4 - ACTIVATE BLOCKER FUNCTIONS
|
||||
# ==========================================
|
||||
|
||||
### THE END of the Long and Winding Road
|
||||
# =====================
|
||||
# END BLOCKER FUNCTIONS
|
||||
# =====================
|
||||
|
||||
### Also check out my Ultimate Apache Bad Bot Blocker on Github
|
||||
### https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker
|
||||
### --------------------------------------------
|
||||
### HELP SUPPORT THIS PROJECT - Send Me a Coffee
|
||||
### https://ko-fi.com/mitchellkrog
|
||||
### --------------------------------------------
|
||||
|
||||
### FOR APACHE SERVERS
|
||||
### ---------------------------------------------
|
||||
### Check out the Ultimate Apache Bad Bot Blocker
|
||||
### ---------------------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@
|
|||
|
||||
### VERSION INFORMATION #
|
||||
###################################################
|
||||
### Version: V3.2019.06.1644
|
||||
### Updated: Thu Jun 27 13:12:23 SAST 2019
|
||||
### Version: V3.2019.06.1645
|
||||
### Updated: Thu Jun 27 13:24:56 SAST 2019
|
||||
### Bad Referrer Count: 6713
|
||||
### Bad Bot Count: 556
|
||||
###################################################
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@
|
|||
|
||||
### VERSION INFORMATION #
|
||||
###################################################
|
||||
### Version: V4.2019.06.1644
|
||||
### Updated: Thu Jun 27 13:12:21 SAST 2019
|
||||
### Version: V4.2019.06.1645
|
||||
### Updated: Thu Jun 27 13:24:55 SAST 2019
|
||||
### Bad Referrer Count: 6713
|
||||
### Bad Bot Count: 556
|
||||
###################################################
|
||||
|
|
|
|||
|
|
@ -1,2 +1,2 @@
|
|||
35.192.85.2 1;
|
||||
104.154.120.187 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
|
|
@ -1,50 +1,50 @@
|
|||
AiHitBot
|
||||
Asterias
|
||||
backlink-check
|
||||
BlowFish
|
||||
CheTeam
|
||||
CrunchBot
|
||||
Ebingbong
|
||||
EyeNetIE
|
||||
GermCrawler
|
||||
Go-Ahead-Got-It
|
||||
Gotit
|
||||
GT::WWW
|
||||
Humanlinks
|
||||
Image Sucker
|
||||
Indy Library
|
||||
JennyBot
|
||||
Larbin
|
||||
Lmspider
|
||||
Mass Downloader
|
||||
netEstate NE Crawler
|
||||
Nettrack
|
||||
NetZIP
|
||||
Nutch
|
||||
OpenVAS
|
||||
OutclicksBot
|
||||
Aboundex
|
||||
Bandit
|
||||
CherryPicker
|
||||
Collector
|
||||
Craftbot
|
||||
DomainAppender
|
||||
EasyDL
|
||||
EMail Siphon
|
||||
Fimap
|
||||
Firefox/7.0
|
||||
GetWeb
|
||||
Gigablast
|
||||
GoZilla
|
||||
Go!Zilla
|
||||
HaosouSpider
|
||||
HTMLparser
|
||||
InfoNaviRobot
|
||||
Jbrofuzz
|
||||
LinkextractorPro
|
||||
LinqiaMetadataDownloaderBot
|
||||
LWP::Simple
|
||||
Masscan
|
||||
MFC_Tear_Sample
|
||||
MIDown tool
|
||||
Ninja
|
||||
PageAnalyzer
|
||||
PictureFinder
|
||||
Pixray
|
||||
scan.lol
|
||||
SEOkicks-Robot
|
||||
SiteSnagger
|
||||
Site Sucker
|
||||
Snapbot
|
||||
Sottopop
|
||||
Surfbot
|
||||
Suzuran
|
||||
Szukacz
|
||||
Psbot
|
||||
RankActiveLinkBot
|
||||
RankFlex
|
||||
RankingBot2
|
||||
RankurBot
|
||||
ScreenerBot
|
||||
Searchestate
|
||||
SemrushBot
|
||||
SISTRIX
|
||||
SpankBot
|
||||
T8Abot
|
||||
Titan
|
||||
Toweyabot
|
||||
trendiction.com
|
||||
Webalta
|
||||
WebAuto
|
||||
WebImageCollector
|
||||
WebmasterWorldForumBot
|
||||
WebWhacker
|
||||
WISENutbot
|
||||
WWW::Mechanize
|
||||
Zeus
|
||||
ZmEu
|
||||
Teleport
|
||||
Tracemyfile
|
||||
VB Project
|
||||
VeriCiteCrawler
|
||||
Wallpapers/3.0
|
||||
WebCopier
|
||||
WEBDAV
|
||||
WebLeacher
|
||||
WebsiteQuester
|
||||
WeSEE
|
||||
Whatweb
|
||||
Xaldon_WebSpider
|
||||
zgrab
|
||||
|
|
|
|||
|
|
@ -1,250 +1,250 @@
|
|||
ADmantX
|
||||
404checker
|
||||
80legs
|
||||
Abonti
|
||||
AhrefsBot
|
||||
AIBOT
|
||||
AiHitBot
|
||||
Aipbot
|
||||
Alexibot
|
||||
AllSubmitter
|
||||
AlphaBot
|
||||
Anarchie
|
||||
Apexoo
|
||||
ASPSeek
|
||||
Asterias
|
||||
archive.org_bot
|
||||
Attach
|
||||
autoemailspider
|
||||
BacklinkCrawler
|
||||
BackStreet
|
||||
BackWeb
|
||||
Badass
|
||||
BBBike
|
||||
Bigfoot
|
||||
Bandit
|
||||
BetaBot
|
||||
Blackboard
|
||||
Blow
|
||||
Boardreader
|
||||
Bolt
|
||||
Black Hole
|
||||
BlackWidow
|
||||
BLEXBot
|
||||
BlowFish
|
||||
BotALot
|
||||
Buddy
|
||||
BuiltBotTough
|
||||
BuiltWith
|
||||
BuzzSumo
|
||||
CATExplorador
|
||||
CazoodleBot
|
||||
CCBot
|
||||
Cegbfeieh
|
||||
CherryPicker
|
||||
CheTeam
|
||||
ChinaClaw
|
||||
CheeseBot
|
||||
Chlooe
|
||||
Claritybot
|
||||
Cogentbot
|
||||
Cliqzbot
|
||||
cognitiveseo
|
||||
Collector
|
||||
com.plumanalytics
|
||||
Copier
|
||||
CopyRightCheck
|
||||
Copyscape
|
||||
Cosmos
|
||||
Craftbot
|
||||
crawl.sogou.com
|
||||
Crescent
|
||||
CrunchBot
|
||||
CSHttp
|
||||
Curious
|
||||
DatabaseDriverMysqli
|
||||
DataCha0s
|
||||
demandbase-bot
|
||||
Deusu
|
||||
Digincore
|
||||
DigitalPebble
|
||||
Dirbuster
|
||||
DIIbot
|
||||
Disco
|
||||
Discoverybot
|
||||
Dispatch
|
||||
DittoSpyder
|
||||
DnyzBot
|
||||
DomainSigmaCrawler
|
||||
Dragonfly
|
||||
Drip
|
||||
DomainStatsBot
|
||||
Dotbot
|
||||
DSearch
|
||||
DTS Agent
|
||||
EirGrabber
|
||||
EMail Siphon
|
||||
EMail Wolf
|
||||
EroCrawler
|
||||
evc-batch
|
||||
Express WebPictures
|
||||
Extreme Picture Finder
|
||||
Extractor
|
||||
ExtractorPro
|
||||
Ezooms
|
||||
FDM
|
||||
FemtosearchBot
|
||||
FHscan
|
||||
Fimap
|
||||
Firefox/7.0
|
||||
Flunky
|
||||
FlashGet
|
||||
Foobot
|
||||
FyberSpider
|
||||
Fyrebot
|
||||
GalaxyBot
|
||||
Genieo
|
||||
Getintent
|
||||
GetRight
|
||||
GetWeb
|
||||
Gigablast
|
||||
Gigabot
|
||||
G-i-g-a-b-o-t
|
||||
Gotit
|
||||
GrabNet
|
||||
GrapeFX
|
||||
Grafula
|
||||
GrapeshotCrawler
|
||||
GridBot
|
||||
Haansoft
|
||||
HEADMasterSEO
|
||||
Heritrix
|
||||
HTMLparser
|
||||
Humanlinks
|
||||
HybridBot
|
||||
Iblog
|
||||
HaosouSpider
|
||||
Harvest
|
||||
Hloader
|
||||
HTTrack
|
||||
Id-search
|
||||
IlseBot
|
||||
Image Fetch
|
||||
Image Sucker
|
||||
IndeedBot
|
||||
Indy Library
|
||||
InfoNaviRobot
|
||||
InfoTekies
|
||||
Intelliseek
|
||||
Iskanie
|
||||
JamesBOT
|
||||
InternetSeer
|
||||
IRLbot
|
||||
Jbrofuzz
|
||||
JikeSpider
|
||||
JOC Web Spider
|
||||
Jorgee
|
||||
JustView
|
||||
Jyxobot
|
||||
Keyword Density
|
||||
Lanshanbot
|
||||
LexiBot
|
||||
Lftp
|
||||
LinkextractorPro
|
||||
LinkpadBot
|
||||
Lightspeedsystems
|
||||
Likse
|
||||
LinksManager
|
||||
LinkWalker
|
||||
LinqiaScrapeBot
|
||||
LinqiaMetadataDownloaderBot
|
||||
LinqiaRSSBot
|
||||
Lipperhey
|
||||
Lipperhey Spider
|
||||
Ltx71
|
||||
lwp-request
|
||||
LWP::Simple
|
||||
Mag-Net
|
||||
Mail.RU_Bot
|
||||
Majestic12
|
||||
Majestic SEO
|
||||
Majestic-SEO
|
||||
MarkWatch
|
||||
Masscan
|
||||
Mass Downloader
|
||||
Mata Hari
|
||||
meanpathbot
|
||||
Mediatoolkitbot
|
||||
MegaIndex.ru
|
||||
Metauri
|
||||
MFC_Tear_Sample
|
||||
Microsoft Data Access
|
||||
Microsoft URL Control
|
||||
MIDown tool
|
||||
Mister PiX
|
||||
MJ12bot
|
||||
Mojeek
|
||||
Morfeus Fucking Scanner
|
||||
Mr.4x3
|
||||
MSFrontPage
|
||||
MSIECrawler
|
||||
muhstik-scan
|
||||
Musobot
|
||||
Nameprotect
|
||||
Navroad
|
||||
Msrabot
|
||||
MS Web Services Client Protocol
|
||||
Needle
|
||||
NetAnts
|
||||
Netcraft
|
||||
Nettrack
|
||||
Nibbler
|
||||
netEstate NE Crawler
|
||||
NetLyzer
|
||||
NetSpider
|
||||
Netvibes
|
||||
NICErsPRO
|
||||
Niki-bot
|
||||
NimbleCrawler
|
||||
Nimbostratus
|
||||
Ninja
|
||||
Octopus
|
||||
Offline Explorer
|
||||
OnCrawl
|
||||
Openvas
|
||||
Nutch
|
||||
oBot
|
||||
Openfind
|
||||
OpenVAS
|
||||
OrangeBot
|
||||
OrangeSpider
|
||||
page scorer
|
||||
OutclicksBot
|
||||
Page Analyzer
|
||||
PageGrabber
|
||||
Panscient
|
||||
Papa Foto
|
||||
Pavuk
|
||||
pcBrowser
|
||||
PeoplePal
|
||||
Picsearch
|
||||
PictureFinder
|
||||
Pi-Monster
|
||||
Pimonster
|
||||
Pixray
|
||||
PleaseCrawl
|
||||
plumanalytics
|
||||
Pockey
|
||||
POE-Component-Client-HTTP
|
||||
Probethenet
|
||||
PxBroker
|
||||
QueryN Metasearch
|
||||
Quick-Crawler
|
||||
RankActiveLinkBot
|
||||
RankActive
|
||||
RankFlex
|
||||
RankingBot
|
||||
RankingBot2
|
||||
Rankivabot
|
||||
RebelMouse
|
||||
RedesScrapy
|
||||
Reaper
|
||||
RepoMonkey
|
||||
RocketCrawler
|
||||
SBIder
|
||||
Ripper
|
||||
SalesIntelligent
|
||||
ScanAlert
|
||||
Scanbot
|
||||
ScoutJet
|
||||
SearchmetricsBot
|
||||
scan.lol
|
||||
Semrush
|
||||
SemrushBot
|
||||
SEOkicks
|
||||
SEOkicks-Robot
|
||||
SEOlyticsCrawler
|
||||
SEOprofiler
|
||||
Seomoz
|
||||
seoscanners
|
||||
SeoSiteCheckup
|
||||
sexsearcher
|
||||
SiteExplorer
|
||||
SEOstats
|
||||
serpstatbot
|
||||
Shodan
|
||||
Siphon
|
||||
SISTRIX
|
||||
Siteimprove
|
||||
SiteSucker
|
||||
SiteLockSpider
|
||||
SiteSnagger
|
||||
Site Sucker
|
||||
Sitevigil
|
||||
SlySearch
|
||||
SMTBot
|
||||
Snake
|
||||
Snoopy
|
||||
SocialRankIOBot
|
||||
sogouspider
|
||||
Sociscraper
|
||||
Sogou web spider
|
||||
Sosospider
|
||||
Sottopop
|
||||
SpaceBison
|
||||
SpankBot
|
||||
Spammen
|
||||
Spanner
|
||||
SputnikBot
|
||||
spyfu
|
||||
Sqlmap
|
||||
Sqworm
|
||||
Steeler
|
||||
Surfbot
|
||||
Sucker
|
||||
Sucuri
|
||||
SurveyBot
|
||||
Suzuran
|
||||
Szukacz
|
||||
T0PHackTeam
|
||||
Teleport
|
||||
tAkeOut
|
||||
Telesphoreo
|
||||
Telesphorep
|
||||
The Intraformant
|
||||
TheNomad
|
||||
Thumbor
|
||||
Trendiction
|
||||
Titan
|
||||
Toata
|
||||
Toweyabot
|
||||
Trendictionbot
|
||||
trendiction.com
|
||||
trendiction.de
|
||||
Turingos
|
||||
TurnitinBot
|
||||
TwengaBot
|
||||
Twice
|
||||
UnisterBot
|
||||
URLy.Warning
|
||||
Typhoeus
|
||||
Upflow
|
||||
Vacuum
|
||||
VB Project
|
||||
Vagabondo
|
||||
VCI
|
||||
VoidEYE
|
||||
Voltron
|
||||
VidibleScraper
|
||||
Virusdie
|
||||
Voil
|
||||
Wallpapers/3.0
|
||||
WallpapersHD
|
||||
WBSearchBot
|
||||
Webalta
|
||||
WebBandit
|
||||
WebCollage
|
||||
Web Collage
|
||||
WebCopier
|
||||
Web Enhancer
|
||||
Web Fuck
|
||||
WebFuck
|
||||
WebImageCollector
|
||||
WebLeacher
|
||||
Web Pix
|
||||
WebPix
|
||||
WebReaper
|
||||
WebSauger
|
||||
Web Sauger
|
||||
Webshag
|
||||
WebsiteQuester
|
||||
Website Quester
|
||||
Webster
|
||||
WebStripper
|
||||
WebZIP
|
||||
WeSEE
|
||||
Whacker
|
||||
Whatweb
|
||||
WinHTTrack
|
||||
WISENutbot
|
||||
Wprecon
|
||||
Whack
|
||||
Wonderbot
|
||||
Woobot
|
||||
WPScan
|
||||
WWW-Collector-E
|
||||
WWW-Mechanize
|
||||
WWW::Mechanize
|
||||
WWWOFFLE
|
||||
x09Mozilla
|
||||
x22Mozilla
|
||||
Xaldon_WebSpider
|
||||
Zade
|
||||
Zauba
|
||||
xpymep1.exe
|
||||
zauba.io
|
||||
Zermelo
|
||||
Zitebot
|
||||
ZmEu
|
||||
|
|
|
|||
|
|
@ -1,250 +1,250 @@
|
|||
360Spider
|
||||
404checker
|
||||
404enemy
|
||||
Aboundex
|
||||
ADmantX
|
||||
AfD-Verbotsverfahren
|
||||
AhrefsBot
|
||||
AIBOT
|
||||
AiHitBot
|
||||
Aipbot
|
||||
Apexoo
|
||||
archive.org_bot
|
||||
Attach
|
||||
Badass
|
||||
Alexibot
|
||||
AlphaBot
|
||||
Anarchie
|
||||
Asterias
|
||||
Backlink-Ceck
|
||||
BacklinkCrawler
|
||||
BackWeb
|
||||
Bandit
|
||||
Barkrowler
|
||||
Battleztar Bazinga
|
||||
BBBike
|
||||
Bitacle
|
||||
Bigfoot
|
||||
Black Hole
|
||||
BlackWidow
|
||||
BotALot
|
||||
Brandprotect
|
||||
BLEXBot
|
||||
Blow
|
||||
BlowFish
|
||||
Bolt
|
||||
Brandwatch
|
||||
Buddy
|
||||
BuiltWith
|
||||
Bullseye
|
||||
CCBot
|
||||
Calculon
|
||||
Cegbfeieh
|
||||
CheTeam
|
||||
CheeseBot
|
||||
CherryPicker
|
||||
ChinaClaw
|
||||
Chlooe
|
||||
Claritybot
|
||||
Cloud mapping
|
||||
coccocbot-web
|
||||
Cogentbot
|
||||
cognitiveseo
|
||||
com.plumanalytics
|
||||
Collector
|
||||
Copier
|
||||
Cosmos
|
||||
CopyRightCheck
|
||||
Copyscape
|
||||
Craftbot
|
||||
crawler4j
|
||||
crawl.sogou.com
|
||||
CrazyWebCrawler
|
||||
Custo
|
||||
crawler.feedback
|
||||
Crescent
|
||||
DatabaseDriverMysqli
|
||||
DataCha0s
|
||||
DBLBot
|
||||
demandbase-bot
|
||||
Demon
|
||||
Deusu
|
||||
Devil
|
||||
DigitalPebble
|
||||
DIIbot
|
||||
Disco
|
||||
Discoverybot
|
||||
Dispatch
|
||||
DnyzBot
|
||||
DomainAppender
|
||||
DomainSigmaCrawler
|
||||
Dotbot
|
||||
Drip
|
||||
EasyDL
|
||||
Ebingbong
|
||||
eCatch
|
||||
ECCP/1.0
|
||||
EirGrabber
|
||||
EMail Siphon
|
||||
EMail Wolf
|
||||
EroCrawler
|
||||
ExtLinksBot
|
||||
Evil
|
||||
Express WebPictures
|
||||
Extractor
|
||||
ExtractorPro
|
||||
Extreme Picture Finder
|
||||
EyeNetIE
|
||||
Ezooms
|
||||
facebookscraper
|
||||
FemtosearchBot
|
||||
FHscan
|
||||
Firefox/7.0
|
||||
FlashGet
|
||||
Foobot
|
||||
FyberSpider
|
||||
Genieo
|
||||
Getintent
|
||||
Freeuploader
|
||||
Fyrebot
|
||||
GalaxyBot
|
||||
GermCrawler
|
||||
GetRight
|
||||
GetWeb
|
||||
Gotit
|
||||
Go!Zilla
|
||||
Grabber
|
||||
GoZilla
|
||||
GrabNet
|
||||
Grafula
|
||||
GrapeshotCrawler
|
||||
GridBot
|
||||
Haansoft
|
||||
HaosouSpider
|
||||
Havij
|
||||
HMView
|
||||
HTMLparser
|
||||
HTTP::Lite
|
||||
HTTrack
|
||||
Humanlinks
|
||||
Iblog
|
||||
IDBot
|
||||
Id-search
|
||||
IlseBot
|
||||
IndeedBot
|
||||
Indy Library
|
||||
InfoTekies
|
||||
instabid
|
||||
InternetSeer
|
||||
ips-agent
|
||||
Intelliseek
|
||||
Internet Ninja
|
||||
internetVista monitor
|
||||
Iria
|
||||
IRLbot
|
||||
Iskanie
|
||||
JamesBOT
|
||||
JennyBot
|
||||
Jbrofuzz
|
||||
JetCar
|
||||
Jetty
|
||||
Joomla
|
||||
Jorgee
|
||||
JustView
|
||||
Jyxobot
|
||||
Kenjin Spider
|
||||
Larbin
|
||||
LeechFTP
|
||||
LexiBot
|
||||
Lightspeedsystems
|
||||
LinkpadBot
|
||||
Keyword Density
|
||||
Kozmosbot
|
||||
Libwhisker
|
||||
Linkdexbot
|
||||
LinkextractorPro
|
||||
LinkScan
|
||||
LinksManager
|
||||
LinqiaMetadataDownloaderBot
|
||||
LinqiaRSSBot
|
||||
LinqiaScrapeBot
|
||||
Lipperhey
|
||||
Lipperhey Spider
|
||||
Litemage_walker
|
||||
Lmspider
|
||||
LNSpiderguy
|
||||
Ltx71
|
||||
lwp-trivial
|
||||
Magnet
|
||||
magpie-crawler
|
||||
Mail.RU_Bot
|
||||
Mag-Net
|
||||
MarkWatch
|
||||
Masscan
|
||||
Mass Downloader
|
||||
Mata Hari
|
||||
MauiBot
|
||||
Meanpathbot
|
||||
MeanPath Bot
|
||||
Mediatoolkitbot
|
||||
mediawords
|
||||
Metauri
|
||||
Microsoft Data Access
|
||||
MIDown tool
|
||||
MIIxpc
|
||||
MJ12bot
|
||||
Mojeek
|
||||
Morfeus Fucking Scanner
|
||||
Mr.4x3
|
||||
muhstik-scan
|
||||
Musobot
|
||||
Name Intelligence
|
||||
Nameprotect
|
||||
Navroad
|
||||
Needle
|
||||
Nessus
|
||||
Netcraft
|
||||
Nettrack
|
||||
Net Vampire
|
||||
Netvibes
|
||||
Nibbler
|
||||
NICErsPRO
|
||||
Niki-bot
|
||||
Nikto
|
||||
Ninja
|
||||
Nutch
|
||||
Openfind
|
||||
OpenLinkProfiler
|
||||
Nmap
|
||||
Octopus
|
||||
Openvas
|
||||
OpenVAS
|
||||
OutclicksBot
|
||||
OrangeBot
|
||||
OrangeSpider
|
||||
OutfoxBot
|
||||
PageGrabber
|
||||
PageScorer
|
||||
Pandalytics
|
||||
Panscient
|
||||
PECL::HTTP
|
||||
PeoplePal
|
||||
PageAnalyzer
|
||||
Page Analyzer
|
||||
Papa Foto
|
||||
pcBrowser
|
||||
PHPCrawl
|
||||
Picscout
|
||||
ProPowerBot
|
||||
Pixray
|
||||
plumanalytics
|
||||
Pockey
|
||||
POE-Component-Client-HTTP
|
||||
ProWebWalker
|
||||
Pump
|
||||
PxBroker
|
||||
QueryN Metasearch
|
||||
PyCurl
|
||||
Quick-Crawler
|
||||
RankActive
|
||||
RankFlex
|
||||
RankingBot2
|
||||
RankurBot
|
||||
RealDownload
|
||||
Reaper
|
||||
Rankivabot
|
||||
RebelMouse
|
||||
Recorder
|
||||
s1z.ru
|
||||
RedesScrapy
|
||||
ReGet
|
||||
SalesIntelligent
|
||||
ScanAlert
|
||||
scan.lol
|
||||
ScreenerBot
|
||||
SearchmetricsBot
|
||||
SBIder
|
||||
Screaming
|
||||
Searchestate
|
||||
SemrushBot
|
||||
SEOkicks-Robot
|
||||
SEOstats
|
||||
SEOkicks
|
||||
SeoSiteCheckup
|
||||
Siphon
|
||||
SISTRIX
|
||||
SiteExplorer
|
||||
Sitebeam
|
||||
Siteimprove
|
||||
SiteLockSpider
|
||||
SiteSnagger
|
||||
SiteSucker
|
||||
Site Sucker
|
||||
SlySearch
|
||||
Snake
|
||||
Snapbot
|
||||
Snoopy
|
||||
SocialRankIOBot
|
||||
Sociscraper
|
||||
sogouspider
|
||||
Sogou web spider
|
||||
Sottopop
|
||||
SpaceBison
|
||||
Spanner
|
||||
Spammen
|
||||
sp_auditbot
|
||||
SputnikBot
|
||||
Sqlmap
|
||||
Sqlworm
|
||||
Sqworm
|
||||
Steeler
|
||||
SuperHTTP
|
||||
Surfbot
|
||||
SurveyBot
|
||||
Stripper
|
||||
Sucker
|
||||
Swiftbot
|
||||
T8Abot
|
||||
tAkeOut
|
||||
Teleport
|
||||
Telesphoreo
|
||||
TeleportPro
|
||||
Telesphorep
|
||||
Thumbor
|
||||
Toweyabot
|
||||
Tracemyfile
|
||||
trendiction.de
|
||||
TurnitinBot
|
||||
TwengaBot
|
||||
Twice
|
||||
The Intraformant
|
||||
Titan
|
||||
Trendiction
|
||||
Trendictionbot
|
||||
UnisterBot
|
||||
Upflow
|
||||
URLy.Warning
|
||||
URLy Warning
|
||||
Vacuum
|
||||
VCI
|
||||
Vagabondo
|
||||
VB Project
|
||||
VeriCiteCrawler
|
||||
Virusdie
|
||||
WBSearchBot
|
||||
Voil
|
||||
Voltron
|
||||
Web Auto
|
||||
WebBandit
|
||||
WebCollage
|
||||
Web Collage
|
||||
WebCopier
|
||||
WEBDAV
|
||||
WebEnhancer
|
||||
WebFetch
|
||||
Web Enhancer
|
||||
Web Fetch
|
||||
WebFuck
|
||||
WebLeacher
|
||||
webmeup-crawler
|
||||
Web Pix
|
||||
WebReaper
|
||||
WebSauger
|
||||
Web Sauger
|
||||
Webshag
|
||||
WebsiteExtractor
|
||||
WebsiteQuester
|
||||
Website Quester
|
||||
WebStripper
|
||||
WebWhacker
|
||||
WebZIP
|
||||
Whatweb
|
||||
WeSEE
|
||||
Whacker
|
||||
Who.is Bot
|
||||
WinHTTrack
|
||||
WISENutbot
|
||||
Woobot
|
||||
Wotbox
|
||||
WWW-Mechanize
|
||||
WPScan
|
||||
WWW::Mechanize
|
||||
x09Mozilla
|
||||
x22Mozilla
|
||||
Xaldon_WebSpider
|
||||
Xenu
|
||||
xpymep1.exe
|
||||
Zauba
|
||||
zauba.io
|
||||
Zermelo
|
||||
YoudaoBot
|
||||
Zade
|
||||
Zitebot
|
||||
ZmEu
|
||||
|
|
|
|||
|
|
@ -1,100 +1,100 @@
|
|||
3th.co.in
|
||||
5i2.net
|
||||
academia-nsk.org
|
||||
adtech.de
|
||||
advancedmassagebysara.com
|
||||
ahmedabadwebs.com
|
||||
akama.com
|
||||
amt-k.ru
|
||||
amyfoxfitness.com
|
||||
anime.dougasouko.com
|
||||
apifasterlightin-a.akamaihd.net
|
||||
ap.senai.br
|
||||
asrvrep-a.akamaihd.net
|
||||
bannerconnect.net
|
||||
bear.gotcher.us
|
||||
bet-prognoz.com
|
||||
blackhatworth.com
|
||||
blavia.00author.com
|
||||
bloglag.com
|
||||
bluesalt.co
|
||||
briomotor.co
|
||||
bugof.gq
|
||||
buntube.net
|
||||
cementaresearch.se
|
||||
chatroulette.life
|
||||
chatseo.com
|
||||
cowblog.fr
|
||||
crynet.cc
|
||||
demenageur.com
|
||||
divci-hry.info
|
||||
domain.webkeyit.com
|
||||
doublepimp.com
|
||||
ee77ee.com
|
||||
elektrozigaretten1.postbit.com
|
||||
eshop.md
|
||||
exchanges-bet.com
|
||||
extlabs.io
|
||||
freshberry.com.ua
|
||||
googlemare.com
|
||||
gotcher.us
|
||||
gpirate.com
|
||||
hermesreplica.win
|
||||
hitsbox.info
|
||||
hol.es
|
||||
igadgetsworld.com
|
||||
immobiliaremassaro.com
|
||||
interfucks.net
|
||||
irkutsk.zrus.org
|
||||
kinohall.ru
|
||||
klikbonus.com
|
||||
likrot.com
|
||||
lottospring.com
|
||||
monarchfind-a.akamaihd.net
|
||||
montredemarque.nl
|
||||
mrbitsandbytes.com
|
||||
mvpicton.co.uk
|
||||
myhealthcare.com
|
||||
myxdate.info
|
||||
nac-bearings.ru
|
||||
new-post.tk
|
||||
nsatc.net
|
||||
offf.info
|
||||
oneclickfiles.com
|
||||
partner-host.men
|
||||
partnerline.men
|
||||
pastaleads.com
|
||||
pawli.eu
|
||||
pochemychka.net
|
||||
pornoblood.com
|
||||
pornosee.info
|
||||
porno-video-chati.ru
|
||||
remont-comp-pomosh.ru
|
||||
ring4rhino.com
|
||||
royalads.net
|
||||
00it.com
|
||||
24x7-server-support.site
|
||||
4replicawatch.net
|
||||
7makemoneyonline.com
|
||||
adult3dgames.com
|
||||
ad-words.ru
|
||||
arvut.org
|
||||
axbocz.net
|
||||
becuo.com
|
||||
bildsuche.ru
|
||||
buyessay3.blogspot.ru
|
||||
buyfriend.ru
|
||||
club-musics.ru
|
||||
collegeessay19.blogspot.ru
|
||||
compliance-donald.xyz
|
||||
custom-product-labels.com
|
||||
downloadeer.net
|
||||
euroskat.ru
|
||||
everypony.ru
|
||||
ezigarettenkaufen2.dreamwidth.org
|
||||
felizporno.com
|
||||
flipper.top
|
||||
flowersbazar.com
|
||||
fullgirl.ru
|
||||
geoads.com
|
||||
gloverid.site
|
||||
gosreg.amchs.ru
|
||||
gotwebsite1.com
|
||||
graphid.com
|
||||
growshop.es
|
||||
gsasearchengineranker.top
|
||||
gsmtlf.ru
|
||||
hao123.com
|
||||
hostnow.men
|
||||
ideibiznesa2015.ru
|
||||
incanto.in.ua
|
||||
infogame.name
|
||||
inspiring-desperate.tk
|
||||
iomoio.net
|
||||
iscblog.info
|
||||
istizanidineanopiate.blogspot.com
|
||||
joingames.org
|
||||
journeydownthescale.info
|
||||
juliadiets.com
|
||||
kadashihotel.com
|
||||
krasnodar.zrus.org
|
||||
lapitec.eu
|
||||
lockerz.com
|
||||
lovi-moment.com.ua
|
||||
mesto-x.com
|
||||
mirtorrent.net
|
||||
mmostrike.ru
|
||||
moonci.ru
|
||||
mtmtv.info
|
||||
mypornfree.ru
|
||||
narutonaruto.ru
|
||||
notasprensa.info
|
||||
o-o-6-o-o.ru
|
||||
pattersonsweb.com
|
||||
pdamods.ru
|
||||
pflexads.com
|
||||
pixelrz.com
|
||||
pony-business.com
|
||||
potoideas.us
|
||||
promodj.com
|
||||
pskcijdc.bloger.index.hr
|
||||
razorweb-a.akamaihd.net
|
||||
richinvestmonitor.com
|
||||
ric.info
|
||||
rimedia.org
|
||||
rockprogblog.com
|
||||
rucrypt.com
|
||||
sanatorrii.ru
|
||||
sbricur.com
|
||||
seojokes.net
|
||||
socialseet.ru
|
||||
soc-proof.su
|
||||
soheavyblog.com
|
||||
sonata-arctica.wz.cz
|
||||
soviet-portal.do.am
|
||||
spb.afora.ru
|
||||
spidtest.org
|
||||
stroiminsk.org
|
||||
tackletarts.co
|
||||
thefarmergame.com
|
||||
torrentgamer.net
|
||||
tracfone.com
|
||||
trichizobswiv.agddns.net
|
||||
umityangin.net
|
||||
urlopener.blogspot.com.au
|
||||
vertaform.com
|
||||
vkmusics.ru
|
||||
webjam.com
|
||||
whiteproduct.com
|
||||
xn----itbeirbjbi7bc6bh2d.xn--p1ai
|
||||
xtube.com
|
||||
youdao.com
|
||||
zeroredirect8.com
|
||||
zverokruh-shop.cz
|
||||
sharebutton.org
|
||||
shemalegalls.blogporn.in
|
||||
siteheart.net
|
||||
sitevalued.com
|
||||
skyway24.ru
|
||||
snow.nvr163.com
|
||||
sptslmtrafms.com
|
||||
sugarkun.com
|
||||
taaaak.com
|
||||
thebestphotos.eu
|
||||
thedownloadfreeonlinegames.blogspot.com
|
||||
topshef.ru
|
||||
turizmus.us
|
||||
uggbootsoutletsale.us
|
||||
unblocksit.es
|
||||
uptimebot.net
|
||||
videochat.ph
|
||||
vzubah.com
|
||||
winterclassichockeyjerseys.com
|
||||
winx-play.ru
|
||||
woman-h.ru
|
||||
xaijo.com
|
||||
yellowstonesafaritours.com
|
||||
zixizop.net.ru
|
||||
zynax.ua
|
||||
zzbroya.com.ua
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -1,250 +1,250 @@
|
|||
2020iscoming.info
|
||||
24videos.tv
|
||||
256bit.by
|
||||
7zap.com
|
||||
adktrailmap.com
|
||||
ads-cool.pro
|
||||
adultactioncam.com
|
||||
advancedsoftwaresupport.com
|
||||
ad-words.ru
|
||||
alekseevec.ru
|
||||
allblogroll.com
|
||||
aosexkontakte.net
|
||||
apartmentbay.ru
|
||||
apibetweenlinesn-a.akamaihd.net
|
||||
atmagroup.ru
|
||||
audiofree.ru
|
||||
auto4style.ru
|
||||
autochoixspinelli.com
|
||||
avtovolop.ru
|
||||
ayakino.net
|
||||
azadnegar.com
|
||||
batanga.net
|
||||
battle.net
|
||||
benchmarkcommunications.co.uk
|
||||
berrymall.ru
|
||||
betonka.pro
|
||||
bif-ru.info
|
||||
bigames.online
|
||||
binaryoptionscops.info
|
||||
bio-market.kz
|
||||
blackwitchcraft.ru
|
||||
bolezniorganov.ru
|
||||
bongacams.com
|
||||
bookmaker-bet.com
|
||||
bosman.pluto.ro
|
||||
1001watch.com.ua
|
||||
1688.com
|
||||
4webmasters.org
|
||||
8xv8.com
|
||||
abiente.ru
|
||||
academiacsmendoza.org
|
||||
acc.eu.org
|
||||
actulite.com
|
||||
adclickthru.net
|
||||
adsloads.com
|
||||
adtech.fr
|
||||
adultfriendfinder.com
|
||||
adultfullhd.com
|
||||
aktivator-windows10.blogspot.com
|
||||
alarmobninsk.ru
|
||||
alessandraleone.com
|
||||
all4invest.info
|
||||
allkrim.com
|
||||
alpinism.ru
|
||||
amazingpic.net
|
||||
ameblo.jp
|
||||
animal-drawings.com
|
||||
apartamentwroclaw.eu
|
||||
api.stathat.com
|
||||
apparel-offer.com
|
||||
autobudpostach.club
|
||||
aviav.ru.com
|
||||
azazu.ru
|
||||
babieca.com
|
||||
babyfactory.fr
|
||||
bdsmgalls.net
|
||||
beclean-nn.ru
|
||||
bedandbreakfast.com
|
||||
best-deals-products.com
|
||||
besttorrentknifta.weebly.com
|
||||
bezlimitko.xyz
|
||||
binomo.com
|
||||
bloggers.nl
|
||||
bobba.dzaba.com
|
||||
brakehawk.com
|
||||
brimstonehillfortress.org
|
||||
bristolhotel.com.ua
|
||||
budilneg.xyz
|
||||
buigas.00it.com
|
||||
buzzonclick.com
|
||||
calcularpagerank.com
|
||||
canacopegdl.com
|
||||
candycrushshop.com
|
||||
chiblackhawks-jerseys.com
|
||||
cialis-samples.com
|
||||
ciproandtizanidine.blogspot.com
|
||||
clickcash.com
|
||||
clodo.ru
|
||||
codysbbq.com
|
||||
compliance-alexa.xyz
|
||||
custom-product-labels.com
|
||||
dailyfinancefix.com
|
||||
dalavia.ru
|
||||
dandiyabeats.in
|
||||
dealwifi.com
|
||||
defenderxtactical.com
|
||||
dekoration.us
|
||||
buketeg.xyz
|
||||
businesn.men
|
||||
buy-cheap-pills-order-online.com
|
||||
candypeople.se
|
||||
cartierlove2u.xyz
|
||||
cartujano-pre.de
|
||||
ccbill.com
|
||||
chatseo.com
|
||||
cheap-pills-norx.com
|
||||
chelnytruck.ru
|
||||
christianlouboutinshoes.xyz
|
||||
colehaanoutlet.store
|
||||
community.allhiphop.com
|
||||
compliance-barak.top
|
||||
compliance-barak.xyz
|
||||
compliance-olga.top
|
||||
comsysnet.com
|
||||
crynet.cc
|
||||
d2jsp.org
|
||||
data1.scopich.com
|
||||
dekorkeramik.ru
|
||||
destinationrealestate.com
|
||||
detmebel.su
|
||||
dlya-android.org
|
||||
dominateforex.ml
|
||||
dptaughtme.com
|
||||
elektrischezigarette1.blog.pl
|
||||
elkacentr.ru
|
||||
elmifarhangi.com
|
||||
elvel.com.ua
|
||||
enternet.ee
|
||||
estelight.ru
|
||||
european-torches.ru
|
||||
dengi-pod-zalog-nedvizhimosti.ru
|
||||
den-noch24.ru
|
||||
diusyawiga.tk
|
||||
dmmspy.com
|
||||
dnsrsearch.com
|
||||
docsportal.net
|
||||
dojki-hd.com
|
||||
domaineaneblanc.com
|
||||
donna7753191.ru
|
||||
dvrlists.com
|
||||
editors.choice6912650.hulfingtonpost.com
|
||||
elektronischezigarettekaufen1.myblog.de
|
||||
encodable.com
|
||||
engines-usa.com
|
||||
enhand.se
|
||||
eralph.tk
|
||||
evogarage.com
|
||||
extstat.com
|
||||
fbdownloader.com
|
||||
fdzone.org
|
||||
fickkontaktehobbyhuren.com
|
||||
fickluder69.com
|
||||
fix-website-errors.com
|
||||
formulaantiuban.com
|
||||
forum-engineering.ru
|
||||
free-today.com
|
||||
freza-sverlo.ru
|
||||
funcrushgames.com
|
||||
gazoblok.net.ua
|
||||
geileweiber.tk
|
||||
gfaq.ru
|
||||
goroda-vsego-mira.ru
|
||||
grtyi.com
|
||||
handsandlegs.ru
|
||||
havepussy.com
|
||||
hornymatches.com
|
||||
hothor.se
|
||||
houseofgaga.ru
|
||||
ideoworld.org
|
||||
igrovyeavtomaty777.ru
|
||||
ilovevitaly.com
|
||||
imagerydatabase.com
|
||||
impotentik.com
|
||||
instakink.com
|
||||
int.search.tb.ask.com
|
||||
it-max.com.ua
|
||||
jamiembrown.com
|
||||
jeremyeaton.co
|
||||
jpcycles.com
|
||||
juliaworld.net
|
||||
kanimage.com
|
||||
karadene.com
|
||||
keywordsdoctor.com
|
||||
kidskunst.info
|
||||
lafriore.ru
|
||||
landinez.co
|
||||
larchik.net
|
||||
laserpen.club
|
||||
leadwayau.com
|
||||
lenvred.org
|
||||
lescinq.com
|
||||
letslowbefast.today
|
||||
lflash.ru
|
||||
lider-zhaluzi.kiev.ua
|
||||
lignofix.ua
|
||||
linkpulse.com
|
||||
linkredirect.biz
|
||||
linksharingt.com
|
||||
littleberry.ru
|
||||
mac-shield.com
|
||||
master-muznachas.ru
|
||||
exdocsfiles.com
|
||||
extremepornos.net
|
||||
ezigarettekaufen2.yolasite.com
|
||||
f07.de
|
||||
filmi-v.online
|
||||
forum.tvmir.org
|
||||
freejabs.com
|
||||
galeon.com
|
||||
gamblingnerd.com
|
||||
game-top.su
|
||||
gamewrath.com
|
||||
gamezblox.com
|
||||
gemara.com
|
||||
generousdeal-a.akamaihd.net
|
||||
getyourimage.club
|
||||
glavprofit.ru
|
||||
gojiberriess.apishops.ru
|
||||
grizzlysgrill.com
|
||||
gsasearchengineranker.pw
|
||||
habermetre.com
|
||||
hidemyass.com
|
||||
homemature.net
|
||||
hoverboard360.es
|
||||
ilmexico.com
|
||||
imgarcade.com
|
||||
inspiring-desperate.tk
|
||||
instasexyblog.com
|
||||
investyb.com
|
||||
invivo.hu
|
||||
istizanidineanarcoticdrug.blogspot.com
|
||||
istizanidineanopiate.blogspot.com
|
||||
ivoiretechnocom.ci
|
||||
iwantmyfreecash.com
|
||||
iyasimasennka.com
|
||||
japfm.com
|
||||
jav-way.site
|
||||
jerseychinabizwholesale.us
|
||||
job.icivil.ir
|
||||
journalhome.com
|
||||
kakablog.net
|
||||
kino-ecran.ru
|
||||
koleso24.com.ua
|
||||
komputernaya-pomosh-moscow.ru
|
||||
krasnodar.zrus.org
|
||||
krassh.ru
|
||||
kupiproday.com.ua
|
||||
kursy-ege.ru
|
||||
law-check-seven.xyz
|
||||
letsrepair.in
|
||||
locksmith.jp
|
||||
madot.onlinewebshop.net
|
||||
massage-info.nl
|
||||
matb3aa.com
|
||||
matchpal-a.akamaihd.net
|
||||
m.b00kmarks.com
|
||||
mbiologi.ru
|
||||
mesto-x.com
|
||||
metarip.ru
|
||||
minegam.com
|
||||
mojpreskumanie.com
|
||||
moneymaster.ru
|
||||
mielec.pl
|
||||
mir-betting.ru
|
||||
mmoguider.ru
|
||||
mnogolok.info
|
||||
moinozhki.com
|
||||
monclerboots.xyz
|
||||
monetizer.com-01.site
|
||||
money-for-placing-articles.com
|
||||
montazhnic.ru
|
||||
moroccosurfadventures.com
|
||||
myfreecams.com
|
||||
mylida.org
|
||||
myonigroup.com
|
||||
myplaycity.com
|
||||
nextlnk12.com
|
||||
nikhilbahl.com
|
||||
november-lax.com
|
||||
oconto.ru
|
||||
olgacvetmet.com
|
||||
onstrapon.purplesphere.in
|
||||
orgasmatrix.com
|
||||
ororodnik.goodbb.ru
|
||||
osb.se11.ru
|
||||
paclitor.com
|
||||
pandarastore.top
|
||||
pandroid.co
|
||||
partner-host.men
|
||||
picsearch.com
|
||||
pinkduck.ga
|
||||
playfortuna-play.ru
|
||||
pochtovyi-index.ru
|
||||
podshipniki-nsk.ru
|
||||
pony-business.com
|
||||
pornoblood.com
|
||||
moviemail-online.co.uk
|
||||
music7s.me
|
||||
mypets.by
|
||||
narkologiya-belgorod.ru
|
||||
narosty.com
|
||||
newstraveller.ru
|
||||
nfljerseysforsalewholesaler.com
|
||||
nfljerseys.online
|
||||
normalegal.ru
|
||||
notfastfood.ru
|
||||
novodigs.com
|
||||
nvssf.com
|
||||
obnal.org
|
||||
onlainbesplatno.ru
|
||||
onlinewritingjobs17.blogspot.ru
|
||||
orion-code-access.net
|
||||
osoznanie-narkotikam.net
|
||||
paleohub.info
|
||||
pensplan4u.com
|
||||
photosaga.info
|
||||
pic2fly.com
|
||||
piccdata.com
|
||||
picquery.com
|
||||
picsforkeywordsuggestion.com
|
||||
pictures-and-images.net
|
||||
pierrehardysale.online
|
||||
pipki.r.acdnpro.com
|
||||
pornobrazzers.biz
|
||||
pornofiljmi.com
|
||||
prchecker.info
|
||||
potolokelekor.ru
|
||||
pozdrawleniya.ru
|
||||
predmety.in.ua
|
||||
primedice.com
|
||||
privacyassistant.net
|
||||
prizesbook.online
|
||||
profitkode.com
|
||||
propranolol40mg.blogspot.com
|
||||
ptr.ruvds.com
|
||||
pukaporn.com
|
||||
rasteniya-vs-zombi.ru
|
||||
realitykings.com
|
||||
remontvsamare.su
|
||||
responsive-test.net
|
||||
rockingclicks.com
|
||||
sad-torg.com.ua
|
||||
scanmyphones.com
|
||||
scanner-margo.top
|
||||
scanner-mary.top
|
||||
searchengineranker.email
|
||||
searchimpression.com
|
||||
searchmywindow-a.akamaihd.net
|
||||
seccioncontrabajo.com
|
||||
security60-e.com
|
||||
seeresultshub-a.akamaihd.net
|
||||
seorank.info
|
||||
serpstat.com
|
||||
sexblog.pw
|
||||
sexkontakteao.info
|
||||
sex-watch.com
|
||||
sfj-ror.no
|
||||
shiksabd.com
|
||||
shoesonlinebuy.cn
|
||||
shoppingmiracles.co.uk
|
||||
sideeffectsoftizanidine.blogspot.com
|
||||
simul.co
|
||||
sisiynas.ru
|
||||
site-auditor.online
|
||||
slowmac.tech
|
||||
smokewithrabbits.com
|
||||
snworks.com
|
||||
socialbookmarksubmission.org
|
||||
softxaker.ru
|
||||
souvenir.cc
|
||||
sticken.co
|
||||
stretchingabuckblog.com
|
||||
strigkaomsk.ru
|
||||
stroicol.net
|
||||
suchenindeutschland.com
|
||||
superstats.com
|
||||
susanholtphotography.com
|
||||
swinger-mobil.net
|
||||
swiped.su
|
||||
push-ad.com
|
||||
pushdata.sendpulse.com
|
||||
qitt.ru
|
||||
qld10000.net
|
||||
qualitymarketzone.com
|
||||
quelle.ru
|
||||
razleton.com
|
||||
refererx.com
|
||||
rennlist.com
|
||||
ritlweb.com
|
||||
roznica.com.ua
|
||||
runtnc.net
|
||||
rvtv.ru
|
||||
sbprabooks.com
|
||||
scanmarine.info
|
||||
scanner-jess.top
|
||||
scanner-willy.top
|
||||
scrapinghub.com
|
||||
scripted.com
|
||||
seeingmeerkat.com
|
||||
segol.tv
|
||||
semaltmedia.com
|
||||
sexs-foto.com
|
||||
sfd-chess.ru
|
||||
shopfishing.com.ua
|
||||
skylta.com
|
||||
smart-balancewheel.com
|
||||
social-s-iii.xyz
|
||||
sonata-arctica.wz.cz
|
||||
soundfrost.org
|
||||
sukirgenk.dvrlists.com
|
||||
sundrugstore.com
|
||||
sweepstakes.rewardit.com
|
||||
syvertsen-da.no
|
||||
tamada69.com
|
||||
tfxiq.com
|
||||
thefarmergame.com
|
||||
thegamerznetwork.com
|
||||
thetardistimes.ovh
|
||||
thexart.club
|
||||
tizanidineduringpregnancy.blogspot.com
|
||||
tizanidineformigraines.blogspot.com
|
||||
tizanidineingredients.blogspot.com
|
||||
tjkckpytpnje.com
|
||||
tmearegion26.com
|
||||
theguardlan.com
|
||||
tmtrck.com
|
||||
tootoo.to
|
||||
topclickguru.com
|
||||
track-rankings.online
|
||||
traffixer.com
|
||||
trailer.cinemaflix.website
|
||||
tx41tclega.ru
|
||||
uhdtv.website
|
||||
ultimatesetnewfreeallsoftupgradesystems.pw
|
||||
uogonline.com
|
||||
uprour.com
|
||||
ussearche.cf
|
||||
v24s.net
|
||||
vapomnoncri.tk
|
||||
videochat.cafe
|
||||
vintontech.info
|
||||
vinylvault.co.uk
|
||||
webenlace.com.ar
|
||||
webshoppermac.com
|
||||
website-analytics.online
|
||||
topshef.ru
|
||||
tourismvictoria.com
|
||||
traffmonster.info
|
||||
tripper.de
|
||||
turkeyreport.tk
|
||||
twu.com.ua
|
||||
uptime.com
|
||||
urzedowski.eu
|
||||
veopornogratis.xxx
|
||||
villakohlanta.nu
|
||||
vkak.ru
|
||||
vriel.batcave.net
|
||||
vsesubwaysurfers.com
|
||||
wait3sec.org
|
||||
watchmyfb.pl
|
||||
waterefficiency.co
|
||||
weprik.ru
|
||||
wetgames.ru
|
||||
wjgony.com
|
||||
w-journal.ru
|
||||
wladimirpayen.com
|
||||
www.888.com
|
||||
wwwadultcheck.com
|
||||
wygraj-skiny.win
|
||||
x-diesel.info
|
||||
xn----itbkqkfiq.xn--p1ai
|
||||
xtrafficplus.com
|
||||
your-bearings.com
|
||||
youtubedownload.org
|
||||
zoogdisany.com
|
||||
zrizvtrnpale.tk
|
||||
word-vorlagen.net
|
||||
worldhistory.biz
|
||||
www.kabbalah-red-bracelets.com
|
||||
xboxster.ru
|
||||
x-mix.info
|
||||
xn----7sbho2agebbhlivy.xn--p1ai
|
||||
xn--e1aggki3c.xn--80adxhks
|
||||
xn--h1ahbi.com.ua
|
||||
yeartwit.com
|
||||
youtoner.it
|
||||
zdesoboi.com
|
||||
zeroredirect5.com
|
||||
zigzog.ru
|
||||
zoogdinsney.com
|
||||
|
|
|
|||
|
|
@ -0,0 +1,99 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Welcome to the Nginx</title>
|
||||
</head>
|
||||
<body>
|
||||
<div align=center>
|
||||
<h1>Welcome to the Nginx Ultimate Bad Bot Blocker</h1>
|
||||
<p>This is merely used for testing !!!</p>
|
||||
</body>
|
||||
</html><html>
|
||||
<head><title>503 Service Temporarily Unavailable</title></head>
|
||||
<body bgcolor="white">
|
||||
<center><h1>503 Service Temporarily Unavailable</h1></center>
|
||||
<hr><center>nginx/1.14.2</center>
|
||||
</body>
|
||||
</html>
|
||||
<html>
|
||||
<head><title>503 Service Temporarily Unavailable</title></head>
|
||||
<body bgcolor="white">
|
||||
<center><h1>503 Service Temporarily Unavailable</h1></center>
|
||||
<hr><center>nginx/1.14.2</center>
|
||||
</body>
|
||||
</html>
|
||||
<html>
|
||||
<head><title>503 Service Temporarily Unavailable</title></head>
|
||||
<body bgcolor="white">
|
||||
<center><h1>503 Service Temporarily Unavailable</h1></center>
|
||||
<hr><center>nginx/1.14.2</center>
|
||||
</body>
|
||||
</html>
|
||||
<html>
|
||||
<head><title>503 Service Temporarily Unavailable</title></head>
|
||||
<body bgcolor="white">
|
||||
<center><h1>503 Service Temporarily Unavailable</h1></center>
|
||||
<hr><center>nginx/1.14.2</center>
|
||||
</body>
|
||||
</html>
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Welcome to the Nginx</title>
|
||||
</head>
|
||||
<body>
|
||||
<div align=center>
|
||||
<h1>Welcome to the Nginx Ultimate Bad Bot Blocker</h1>
|
||||
<p>This is merely used for testing !!!</p>
|
||||
</body>
|
||||
</html><!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Welcome to the Nginx</title>
|
||||
</head>
|
||||
<body>
|
||||
<div align=center>
|
||||
<h1>Welcome to the Nginx Ultimate Bad Bot Blocker</h1>
|
||||
<p>This is merely used for testing !!!</p>
|
||||
</body>
|
||||
</html><!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Welcome to the Nginx</title>
|
||||
</head>
|
||||
<body>
|
||||
<div align=center>
|
||||
<h1>Welcome to the Nginx Ultimate Bad Bot Blocker</h1>
|
||||
<p>This is merely used for testing !!!</p>
|
||||
</body>
|
||||
</html><!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Welcome to the Nginx</title>
|
||||
</head>
|
||||
<body>
|
||||
<div align=center>
|
||||
<h1>Welcome to the Nginx Ultimate Bad Bot Blocker</h1>
|
||||
<p>This is merely used for testing !!!</p>
|
||||
</body>
|
||||
</html><!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Welcome to the Nginx</title>
|
||||
</head>
|
||||
<body>
|
||||
<div align=center>
|
||||
<h1>Welcome to the Nginx Ultimate Bad Bot Blocker</h1>
|
||||
<p>This is merely used for testing !!!</p>
|
||||
</body>
|
||||
</html><!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Welcome to the Nginx</title>
|
||||
</head>
|
||||
<body>
|
||||
<div align=center>
|
||||
<h1>Welcome to the Nginx Ultimate Bad Bot Blocker</h1>
|
||||
<p>This is merely used for testing !!!</p>
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
35.192.85.2 0;
|
||||
104.154.120.187 0;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
127.0.0.1 1;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue