V4.2023.03.3633 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
 ADmantX
-AhrefsBot
 AiHitBot
-Aspiegel
-Asterias
+Alligator
+Anarchy
+Anarchy99
+Ankit
+Apexoo
+AwarioSmartBot
 BBBike
 Backlink-Ceck
-BacklinkCrawler
-Bitacle
-BlowFish
-Calculon
-Cliqzbot
-Cloud mapping
-Cogentbot
+Bolt
+Bullseye
+CODE87
+Chlooe
 Copier
+Craftbot
 Crawling at Home Project
-Curious
-Dirbuster
-DnBCrawler-Analytics
+Demon
+Digincore
+Discoverybot
+DnyzBot
+DomainAppender
+DomainSigmaCrawler
 DomainStatsBot
+Dotbot
 EMail Wolf
-Evil
-FDM
-Fuzz
-GT::WWW
-GetWeb
-Getintent
+Ecxi
+Express WebPictures
+EyeNetIE
+FemtosearchBot
+Foobot
+GoZilla
 GrapeFX
-HEADMasterSEO
-HonoluluBot
+Humanlinks
+IDBot
+Iblog
 Image Fetch
-Jbrofuzz
-LWP::Simple
+IndeedBot
+InterGET
+IstellaBot
+JOC Web Spider
+JennyBot
+Lanshanbot
+LeechGet
+Lftp
+Likse
+LinkextractorPro
 LinqiaMetadataDownloaderBot
-LinqiaRSSBot
-MarkWatch
-Mata Hari
-MauiBot
-Mister PiX
-Mojeek
-Musobot
-Nameprotect
-Needle
-NimbleCrawler
-Offline Explorer
-POE-Component-Client-HTTP
-PageAnalyzer
-RSSingBot
-RankActiveLinkBot
-Rankivabot
+MFC_Tear_Sample
+Mail.RU_Bot
+Masscan
+Metauri
+Moblie Safari
+MolokaiBot
+Navroad
+Net Vampire
+Not
+Nuclei
+OnCrawl
+Openfind
+Piepmatz
+Pump
+Re-re
+RealDownload
+RepoMonkey
 SEOkicks
-ScoutJet
-Scrapy
+SEOstats
 SemrushBot
-SeoSiteCheckup
-Seomoz
-Shodan
-SiteCheckerBotCrawler
-SiteExplorer
-Sitebeam
-Sottopop
-SpaceBison
+Site Sucker
+SocialRankIOBot
+Sogou web spider
 SpankBot
-Spbot
-Spinn3r
-SuperHTTP
-Suzuran
-Swiftbot
-Szukacz
-T0PHackTeam
-The Intraformant
-TheNomad
-Trendiction
+Telesphoreo
 True_Robot
-TwengaBot
-VB Project
+Typhoeus
+Vagabondo
+VidibleScraper
+VoidEYE
+WASALive-Bot
 WBSearchBot
-WEBDAV
-WPScan
-WWW-Collector-E
-Web Auto
-Web Fuck
-Web Pix
-WebStripper
-Webalta
-Webster
-Whacker
-Woobot
-Zitebot
-ZyBorg
+WISENutbot
+WebBandit
+WebEnhancer
+WebWhacker
+WebmasterWorldForumBot
+Webshag
+Who.is Bot
+Widow
+WinHTTrack
+Zauba
+ZoominfoBot
 adscanner
-autoemailspider
-cognitiveseo
-iubenda-radar
-polaris version
-probe-image-size
-sexsearcher
-spyfu
-voyagerx.com
-webmeup-crawler
+backlink-check
+crawl.sogou.com
+domainsproject.org
+instabid
+mediawords
+plumanalytics
+serpstatbot
+x09Mozilla