From ec919d888f95def4b36aa9a6ad28f734cca0b7d9 Mon Sep 17 00:00:00 2001 From: Mitchell Krog Date: Fri, 8 Jun 2018 10:28:26 +0200 Subject: [PATCH] Bot (+) Quick-Crawler Referrer (+) scrapinghub.com Closes: #181 --- _generator_lists/bad-referrers.list | 1 + _generator_lists/bad-user-agents.list | 1 + 2 files changed, 2 insertions(+) diff --git a/_generator_lists/bad-referrers.list b/_generator_lists/bad-referrers.list index 8b8fca61b..dc69ffcf5 100755 --- a/_generator_lists/bad-referrers.list +++ b/_generator_lists/bad-referrers.list @@ -1,3 +1,4 @@ +scrapinghub.com 000free.us 007angels.com 00author.com diff --git a/_generator_lists/bad-user-agents.list b/_generator_lists/bad-user-agents.list index 7cbff1cd8..bc4c6f829 100755 --- a/_generator_lists/bad-user-agents.list +++ b/_generator_lists/bad-user-agents.list @@ -1,3 +1,4 @@ +Quick-Crawler 360Spider 404checker 404enemy