From f0b90767b1f1c9bc533cc07c372bf17949fe071d Mon Sep 17 00:00:00 2001 From: Mitchell Krog Date: Sun, 11 Jun 2023 08:23:34 +0200 Subject: [PATCH] Add Bytespider Aggressive Crawler - Does not check for robots.txt --- _generator_lists/bad-user-agents.list | 1 + 1 file changed, 1 insertion(+) diff --git a/_generator_lists/bad-user-agents.list b/_generator_lists/bad-user-agents.list index 89fdb2dd4..945d37a44 100755 --- a/_generator_lists/bad-user-agents.list +++ b/_generator_lists/bad-user-agents.list @@ -1,3 +1,4 @@ +Bytespider 01h4x.com 360Spider 404checker