V4.2023.10.4061 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-ADmantX
-ASPSeek
+404enemy
 AhrefsBot
-Ankit
-BBBike
-BatchFTP
-Black Hole
-BotALot
+Aspiegel
+BackStreet
+Badass
+Blow
 Brandprotect
-BuiltBotTough
-Bullseye
-CATExplorador
-CheTeam
-Chlooe
-Citoid
-Copier
+Bytespider
+Calculon
+CazoodleBot
+CrazyWebCrawler
+CrunchBot
+Custo
 Deusu
-Digincore
-DittoSpyder
-DomainStatsBot
-EirGrabber
-FHscan
-Fuzz
+DigitalPebble
+Dirbuster
+DnBCrawler-Analytics
+DnyzBot
+DomainAppender
+EMail Siphon
+EasyDL
+Ezooms
+FDM
+Flunky
+FrontPage
+FyberSpider
+G-i-g-a-b-o-t
+Grabber
 HMView
 HTMLparser
-Hloader
 HybridBot
+IDBTE4M
+IDBot
+Iblog
 Id-search
-Image Sucker
+IlseBot
+Image Fetch
 Intelliseek
-Internet Ninja
-Iskanie
-JetCar
-Jetty
-LNSpiderguy
-LeechGet
-LinkWalker
+Keybot Translation-Search-Machine
+Kinza
+Linkbot
 LinqiaMetadataDownloaderBot
-Lmspider
-Ltx71
-MFC_Tear_Sample
-MIIxpc
-MarkMonitor
-Mb2345Browser
-Meanpathbot
-MicroMessenger
-Microsoft URL Control
-Mojeek
-NPbot
-Nameprotect
-Nessus
-NimbleCrawler
+LinqiaScrapeBot
+MarkWatch
+Masscan
+MeanPath Bot
+Moblie Safari
+Mojolicious
+Morfeus Fucking Scanner
+Msrabot
+NetMechanic
 Nuclei
-Openfind
-OrangeSpider
-OutfoxBot
-Page Analyzer
-Panscient
+PageGrabber
+PageThing.com
+Petalbot
 Picsearch
-Piepmatz
-PleaseCrawl
-Proximic
-Psbot
-RSSingBot
-Ripper
+Pimonster
+PxBroker
+PyCurl
+RankFlex
+RankingBot2
+Rankivabot
+ReGet
+RedesScrapy
+Rogerbot
+SEOlyticsCrawler
+SMTBot
 SalesIntelligent
-SentiBot
-Seomoz
-Siphon
-SmartDownload
-Sociscraper
-Sosospider
-SputnikBot
-Sqworm
-Sucuri
-TinyTestBot
-Titan
-Typhoeus
-Upflow
-VCI
-Wallpapers/3.0
+SearchmetricsBot
+SemanticJuice
+SemrushBot
+SenutoBot
+SeoSiteCheckup
+SiteCheckerBotCrawler
+Spbot
+Sucker
+SuperHTTP
+Swiftbot
+TeleportPro
+Toata
+True_Robot
+VB Project
 WallpapersHD
-Web Sauger
-WebEnhancer
-WebLeacher
-Webalta
-Widow
-ZoominfoBot
-arquivo-web-crawler
-dataforseobot
-domainsproject.org
-heritrix
-instabid
-lwp-request
-pcBrowser
-ripz
+Web Pix
+WebsiteExtractor
+Xaldon WebSpider
+Zade
+Zermelo
+ZoomBot
+ZumBot
+autoemailspider
+cognitiveseo
+com.plumanalytics
+magpie-crawler
+openai.com
+s1z.ru
 satoristudio.net
-seocompany.store
-seostar
-voyagerx.com
+siteripz
+webgains-bot
+webmeup-crawler