From 07da8a50f8d0866771f677169bbe42967294bfed Mon Sep 17 00:00:00 2001 From: Mitchell Krog UB1 Date: Thu, 12 Jan 2017 16:23:31 +0200 Subject: [PATCH] Added robots.txt for People Unable to Use this Bot Blocker --- robots.txt/README.md | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/robots.txt/README.md b/robots.txt/README.md index 12a853d0f..dfd95025a 100644 --- a/robots.txt/README.md +++ b/robots.txt/README.md @@ -7,21 +7,20 @@ The robots.txt file residing in this directory can be used to ADD onto your exis If your existing robots.txt file looks like this: ->User-agent: * - ->Allow: /wp-admin/admin-ajax.php + User-agent: * + Allow: /wp-admin/admin-ajax.php Then your new file will look like this: ->User-agent: * ->Allow: /wp-admin/admin-ajax.php ->User-agent: Acunetix -> Disallow:/ ->User-agent: FHscan -> Disallow:/ ->User-agent: masscan -> Disallow:/ ->........ rest of file contents + User-agent: * + Allow: /wp-admin/admin-ajax.php + User-agent: Acunetix + Disallow:/ + User-agent: FHscan + Disallow:/ + User-agent: masscan + Disallow:/ + ........ rest of file contents # You use this at your own risk