V4.2023.06.3837 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-ADmantX
-Abonti
+360Spider
+404checker
+AIBOT
 Aboundex
-AhrefsBot
-Anarchy
-Ankit
+AdsTxtCrawlerTP
+Alexibot
+Anarchy99
 Anthill
+Aspiegel
+AwarioRssBot
 BLEXBot
-Boardreader
-Brandprotect
+Badass
+Barkrowler
+BlowFish
 Buck
-Bullseye
-Bytespider
 CCBot
-Cegbfeieh
-Claritybot
-Cocolyzebot
-Cogentbot
-Copyscape
-Discobot
-DomainCrawler
-EMail Siphon
-EroCrawler
-FHscan
-Fimap
-GalaxyBot
-Go!Zilla
-GrapeFX
-Heritrix
+CherryPicker
+Cloud mapping
+DataCha0s
+DnBCrawler-Analytics
+DnyzBot
+DomainSigmaCrawler
+Ecxi
+Express WebPictures
+FlashGet
+GetRight
+Gigabot
+GrapeshotCrawler
+HEADMasterSEO
+HTMLparser
+HTTP::Lite
 HonoluluBot
-Image Sucker
-InternetSeer
-Iria
+Humanlinks
+Iskanie
 IstellaBot
-JennyBot
-JikeSpider
-Jorgee
-Keyword Density
-Lanshanbot
-Likse
-LinqiaMetadataDownloaderBot
-LinqiaScrapeBot
-MFC_Tear_Sample
-MSIECrawler
-Mag-Net
-Mediatoolkitbot
-MicroMessenger
-Mister PiX
+Kenjin Spider
+LWP::Simple
+LexiBot
+Libwhisker
+Lightspeedsystems
+LinkpadBot
+Lipperhey
+MJ12bot
+Mail.RU_Bot
+Majestic-SEO
+Masscan
+MegaIndex.ru
+Metauri
+Microsoft Data Access
+Mojolicious
 Msrabot
-NPbot
-Nameprotect
-NetLyzer
-Netvibes
-NextGenSearchBot
-Nibbler
-NimbleCrawler
-Offline Explorer
+Nessus
+Nimbostratus
+Offline Navigator
+Openfind
 OutclicksBot
-POE-Component-Client-HTTP
-Page Analyzer
-PageThing.com
-Pandalytics
-Pavuk
-Piepmatz
-Pixray
-PyCurl
-RankurBot
-Re-re
-SEOkicks
-SEOstats
-SISTRIX
-ScanAlert
-ScrepyBot
+RankActive
+Rankivabot
+RebelMouse
+Screaming
 Seekport
-Seomoz
-Site Sucker
+Semrush
+Shodan
+SiteSnagger
 Sitevigil
-SputnikBot
-SurveyBot
-Szukacz
+Snake
+Sociscraper
+Swiftbot
+T0PHackTeam
 TeleportPro
+Telesoft
+TightTwatBot
+Toata
+TwengaBot
 Twice
-V-BOT
-WASALive-Bot
-WISENutbot
-WeSEE
-WebEnhancer
-WebStripper
+Vagabondo
+Voltron
+WBSearchBot
+WEBDAV
+WWW-Collector-E
+WWW::Mechanize
+WallpapersHD
+Web Collage
+WebCopier
+WebFetch
+Webalta
 Website Quester
-Webster
-WiseGuys Robot
-Wprecon
-Zeus
-archive.org_bot
+Who.is Bot
+Woobot
 crawler4j
 dataforseo.com
-linkdexbot
-ripz
-satoristudio.net
-zauba.io
+demandbase-bot
+domainsproject.org
+ips-agent
+magpie-crawler
+s1z.ru
+sogouspider
+x09Mozilla
+xpymep1.exe
+zgrab