diff --git a/robots.txt/README.md b/robots.txt/README.md index d0ed87b2e..7e5fa9fb1 100644 --- a/robots.txt/README.md +++ b/robots.txt/README.md @@ -6,11 +6,13 @@ The robots.txt file residing in this directory can be used to ADD onto your exis ## Do NOT make this file the only contents of your robots.txt file !!! If your existing robots.txt file looks like this: -```User-agent: * -Allow: /wp-admin/admin-ajax.php``` + +User-agent: * +Allow: /wp-admin/admin-ajax.php Then your new file will look like this: -```User-agent: * + +User-agent: * Allow: /wp-admin/admin-ajax.php User-agent: Acunetix Disallow:/ @@ -18,7 +20,7 @@ User-agent: FHscan Disallow:/ User-agent: masscan Disallow:/ -........ rest of file contents``` +........ rest of file contents # You use this at your own risk