From 629429aacc9f688216b72353fbf8015284ad1558 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20R=C5=BCanek?= Date: Sat, 2 Jan 2021 15:46:38 +0100 Subject: [PATCH] Update bad-user-agents.list Adding: * Cocolyze bot: `Mozilla/5.0 (compatible; Cocolyzebot/1.0; https://cocolyze.com/bot)` Actually, whatever it is. I cannot access the given link from any location (tried four different networks!). I always get the Cloudflare lock page. So, sorry, bad crawler as it doesn't even want me to identify it. * ZoomBot/LiknBot - different user agents!, i.e.: `ZoomBot (Linkbot 1.0 http://suite.seozoom.it/bot.html)` No information if they obey robots.txt, because when I go to the given URL I am informed the website is only available for paying customers. So, sorry, no... * Velen - `Mozilla/5.0 (compatible; VelenPublicWebCrawler/1.0; +https://velen.io)` Link takes me to a Google 404 page. Strange. Crawling my website like crazy, not obeying (as far as I can tell) robots.txt. --- _generator_lists/bad-user-agents.list | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/_generator_lists/bad-user-agents.list b/_generator_lists/bad-user-agents.list index 5abfb27a3..581a9e846 100755 --- a/_generator_lists/bad-user-agents.list +++ b/_generator_lists/bad-user-agents.list @@ -80,6 +80,7 @@ Claritybot Cliqzbot Cloud\ mapping coccocbot-web +Cocolyzebot CODE87 Cogentbot cognitiveseo @@ -244,6 +245,7 @@ Libwhisker LieBaoFast Lightspeedsystems Likse +Linkbot Linkdexbot LinkextractorPro LinkpadBot @@ -505,6 +507,7 @@ Vacuum Vagabondo VB\ Project VCI +VelenPublicWebCrawler VeriCiteCrawler VidibleScraper Virusdie @@ -582,6 +585,7 @@ Zeus zgrab Zitebot ZmEu +ZoomBot ZoominfoBot ZumBot ZyBorg