V4.2024.06.4582 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
+80legs
 AIBOT
 Acunetix
-AfD-Verbotsverfahren
-BDFetch
+AhrefsBot
+Anarchy
+Anthill
 BatchFTP
-BlackWidow
-Boardreader
-BuiltBotTough
+Battleztar Bazinga
+BotALot
+Brandprotect
+CATExplorador
+CSHttp
 CazoodleBot
-Chlooe
-Cocolyzebot
-CopyRightCheck
+CheeseBot
 Crawling at Home Project
-Crescent
-CyotekWebCopy
-DSearch
-Disco
-DomCopBot
-DomainCrawler
-EMail Siphon
-Evil
-FacebookBot
+CrazyWebCrawler
+DBLBot
+DIIbot
+DTS Agent
+Demon
+Deusu
+Discoverybot
+Dispatch
+ECCP/1.0
+EyeNetIE
+FHscan
+Flunky
+Freeuploader
+FrontPage
+GT::WWW
+Genieo
 GetWeb
-Gotit
+Go!Zilla
 GrabNet
-Grafula
-GrapeFX
-GrapeshotCrawler
-Harvest
-HonoluluBot
-IDBot
+HaosouSpider
+HybridBot
+IDBTE4M
 Iblog
-InfoTekies
-Internet Ninja
-JustView
-LinksManager
-Lipperhey Spider
-MQQBrowser
-MSIECrawler
-Magnet
-Majestic12
-MegaIndex.ru
-MicroMessenger
-Mister PiX
-NICErsPRO
-NetLyzer
-Nimbostratus
-Openfind
-Openvas
-OutfoxBot
-PHPCrawl
-PageGrabber
-Pavuk
+Indy Library
+InfraSec Scanner
+Intelliseek
+InterGET
+IstellaBot
+Leap
+LinkScan
+LinkextractorPro
+LinqiaRSSBot
+Lipperhey
+MJ12bot
+Masscan
+Meanpathbot
+Mojeek
+Mr.4x3
+Msrabot
+Musobot
+Nameprotect
+Needle
+Octopus
+PageAnalyzer
+PageScorer
+Panscient
 Petalbot
-Picscout
-PictureFinder
-PxBroker
-PyCurl
-RSSingBot
-RankingBot
+Pixray
+Pockey
+ProPowerBot
+Re-re
 ReGet
-RedesScrapy
-SEOstats
+RebelMouse
+SEOlyticsCrawler
+SEOprofiler
 SISTRIX
-SalesIntelligent
-Scanbot
-ScoutJet
-ScreenerBot
-Seomoz
-Sitebeam
+ScrepyBot
+SeekportBot
+SeoSiteCheckup
+SocialRankIOBot
 Sosospider
-TightTwatBot
-Typhoeus
-Vacuum
-VelenPublicWebCrawler
-VeriCiteCrawler
+SpankBot
+Sucuri
+SuperBot
+TeleportPro
+The Intraformant
+Twice
+V-BOT
+VB Project
 Voil
-WWWOFFLE
-WallpapersHD
-Web Sauger
-WebPix
-WebSucker
-Website Quester
-Webster
-Xaldon WebSpider
+WWW-Collector-E
+WebFuck
+Xaldon_WebSpider
 YoudaoBot
-Zermelo
-ZmEu
-anthropic-ai
-arquivo.pt
-clark-crawler
-demandbase-bot
-facebookscraper
-linkfluence
-scalaj-http
-scan.lol
-seoscanners
-sexsearcher
+archive.org_bot
+autoemailspider
+dataforseo.com
+domainsproject.org
+mediawords
+oBot
+seocompany.store
+serpstatbot
+sitechecker.pro
 ubermetrics-technologies.com
-zauba.io
+x22Mozilla