V4.2023.05.3736 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-Aboundexbot
-Aspiegel
-Attach
-BDCbot
-BDFetch
-BacklinkCrawler
-Bandit
-Boardreader
-Bolt
-CCBot
-CSHttp
-CensysInspect
-Copier
-Copyscape
-Curious
-Discobot
-DnyzBot
-DomainAppender
-EyeNetIE
-Fimap
-GetWeb
-Go-Ahead-Got-It
+80legs
+Alligator
+Anarchy99
+Anthill
+Apexoo
+AwarioRssBot
+Battleztar Bazinga
+Bitacle
+Blackboard
+Cegbfeieh
+CheeseBot
+Cocolyzebot
+Custo
+DataCha0s
+Digincore
+DnBCrawler-Analytics
+DomainSigmaCrawler
+EasyDL
+FlashGet
+Fuzz
+GetRight
+GoZilla
 GrapeFX
 HEADMasterSEO
-Humanlinks
-IRLbot
-Iblog
-Internet Ninja
-InternetSeer
-JennyBot
-Jetty
-Kenjin Spider
-LeechGet
-LexiBot
-LinkpadBot
-MFC_Tear_Sample
-Mag-Net
+HTMLparser
+HaosouSpider
+IndeedBot
+Kinza
+LinkScan
+LinkWalker
+LinqiaScrapeBot
+Lipperhey
+MIDown tool
+MJ12bot
 Majestic12
-MarkWatch
+Mb2345Browser
+MeanPath Bot
 Meanpathbot
-NearSite
-Needle
-NetLyzer
+MegaIndex.ru
+MicroMessenger
+Navroad
+NetMechanic
+Nettrack
 Nibbler
-NimbleCrawler
-Nmap
-Octopus
-Offline Explorer
-PHPCrawl
-Papa Foto
+Nimbostratus
+Nuclei
+Nutch
+OpenLinkProfiler
+OutfoxBot
+PECL::HTTP
+PageThing.com
+Panscient
+Pavuk
+Pi-Monster
+PictureFinder
+Pimonster
 PleaseCrawl
-Probethenet
-RankActive
-Re-re
-ReGet
-RocketCrawler
-SEOkicks-Robot
-Screaming
-SenutoBot
-SeobilityBot
-Siphon
-Snake
-Snoopy
-Sosospider
-SpaceBison
-Spammen
-Spbot
-Sqlmap
-Sqworm
-Steeler
-Sucker
-Suzuran
-Telesoft
-The Intraformant
-TwengaBot
-VB Project
-Vacuum
+Pu_iN
+RankFlex
+RedesScrapy
+SEOkicks
+SEOstats
+SeekportBot
+SemanticJuice
+Shodan
+Siteimprove
+SpankBot
+Stripper
+Surfbot
+Upflow
 VelenPublicWebCrawler
-WASALive-Bot
-WBSearchBot
-WWWOFFLE
-Web Fuck
-WebImageCollector
-Xaldon_WebSpider
-cah.io.community
-clark-crawler
-com.plumanalytics
-demandbase-bot
-iubenda-radar
-lwp-request
+VidibleScraper
+VoidEYE
+WallpapersHD
+WeSEE
+Web Sauger
+WebEnhancer
+WebGo IS
+WebLeacher
+WebReaper
+WebsiteExtractor
+WebsiteQuester
+YoudaoBot
+Zauba
+ZoominfoBot
+ZumBot
+coccocbot
+facebookscraper
+gopher
+linkdexbot
 magpie-crawler
-pcBrowser
-polaris version
-s1z.ru
-scan.lol
-siteripz
+mediawords
+netEstate NE Crawler
+plumanalytics
+probe-image-size
+seobility
+seoscanners
 tAkeOut
-webprosbot
-xpymep1.exe
-zgrab
+trendiction.com
+trendiction.de