V4.2024.05.4488 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-404enemy
-AhrefsBot
-Alexibot
-Anarchy99
-Ankit
-AwarioRssBot
+Aboundex
+Aboundexbot
+Acunetix
+Anthill
 BackDoorBot
-Backlink-Ceck
+Barkrowler
 Battleztar Bazinga
-BotALot
+Blackboard
+Blow
+Brandprotect
+Brandwatch
 Buck
-BuiltWith
-Chlooe
-Cliqzbot
-Cogentbot
-CrazyWebCrawler
-DBLBot
-DIIbot
-DatabaseDriverMysqli
-Digincore
-Disco
-DomainAppender
-EMail Wolf
+BuzzSumo
+CATExplorador
+CheTeam
+Curious
+DTS Agent
+DigitalPebble
+Discobot
 Ecxi
-EroCrawler
-Extreme Picture Finder
-Fimap
-FrontPage
-Genieo
-Getintent
-Haansoft
+GPTBot
+GT::WWW
+GrapeshotCrawler
+HEADMasterSEO
+Havij
 Heritrix
-IRLbot
-IlseBot
-InterGET
-Internet Ninja
-Iria
-JikeSpider
-Kenjin Spider
+Image Sucker
+InternetMeasurement
+IstellaBot
+JOC Web Spider
+Jbrofuzz
+Keybot Translation-Search-Machine
+LNSpiderguy
 LeechGet
-LinkScan
+LexiBot
+Linkbot
+Lipperhey Spider
+Lmspider
 Ltx71
-MFC_Tear_Sample
-MJ12bot
-Majestic12
-Mass Downloader
-MauiBot
-MegaIndex.ru
-Morfeus Fucking Scanner
-Musobot
-Nameprotect
-Netcraft
-Netvibes
+MQQBrowser
+MTRobot
+MarkWatch
+Meanpathbot
+Mediatoolkitbot
+Mister PiX
+Mr.4x3
+NetLyzer
 NextGenSearchBot
-Nikto
-Nimbostratus
-OpenLinkProfiler
-OrangeBot
+Nutch
+OnCrawl
 OrangeSpider
-Petalbot
-Picsearch
+PageAnalyzer
+Papa Foto
 PictureFinder
-RSSingBot
-ScreenerBot
-ScrepyBot
+Probethenet
+Pump
+ReGet
+ScoutJet
+Scrapy
 Searchestate
+Seekport
+Semrush
 SemrushBot
+Seomoz
 SiteCheckerBotCrawler
-Sociscraper
-SpaceBison
-Spbot
+Sitevigil
+Sottopop
+Spanner
+Sqworm
+Steeler
 SuperHTTP
-Suzuran
-Toweyabot
-Upflow
+SurveyBot
+T8Abot
+The Intraformant
+TurnitinBot
 VB Project
-VCI
-VelenPublicWebCrawler
-WWW-Collector-E
-Web Collage
-Web Sauger
-WebBandit
-WebImageCollector
-WebWhacker
-Website Quester
-Whatweb
-Who.is Bot
-Wotbox
-Zeus
-ZoominfoBot
-crawl.sogou.com
-demandbase-bot
+Vagabondo
+VoidEYE
+WISENutbot
+WWW-Mechanize
+Wallpapers
+Web Pix
+WebCopier
+WebEnhancer
+WebReaper
+WebSauger
+WebsiteExtractor
+WiseGuys Robot
+Wonderbot
+YoudaoBot
+ZmEu
+crawler.feedback
 domainsproject.org
-muhstik-scan
-omgili
-seobility
-sexsearcher
-sitechecker.pro
-tAkeOut
-webmeup-crawler
+imagesift.com
+netEstate NE Crawler
+page scorer
+serpstatbot
+spyfu
+ubermetrics-technologies.com
+voyagerx.com