V4.2024.01.4262 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-ADmantX
-AIBOT
-Aboundex
-Anarchy
-AwarioSmartBot
-BackStreet
-Bandit
-Blow
-BlowFish
-BuiltBotTough
-BuiltWith
-Cliqzbot
-Cosmos
-CyotekWebCopy
-DTS Agent
-DnyzBot
-DomCopBot
-DomainCrawler
-Dragonfly
-EasyDL
-Express WebPictures
-Fimap
+80legs
+ALittle Client
+Abonti
+Alexibot
+Alligator
+Apexoo
+BackWeb
+BacklinkCrawler
+Barkrowler
+Bitacle
+Black Hole
+Blackboard
+Cegbfeieh
+CopyRightCheck
+Curious
+Demon
+DittoSpyder
+DnBCrawler-Analytics
+Drip
+Ezooms
 Foobot
-FrontPage
-Go-Ahead-Got-It
-HMView
-HybridBot
-Iria
-JOC Web Spider
-JikeSpider
-JustView
-Lanshanbot
-LeechFTP
-LinkextractorPro
-Lipperhey
-Ltx71
+GalaxyBot
+Genieo
+Go!Zilla
+GrabNet
+Id-search
+Iskanie
+JamesBOT
+Jetty
+Jyxobot
+Keybot Translation-Search-Machine
+LNSpiderguy
+LinqiaMetadataDownloaderBot
+MFC_Tear_Sample
 MJ12bot
-Mail.RU_Bot
-MegaIndex.ru
-Mojeek
-MolokaiBot
-NetMechanic
-NetSpider
-Netvibes
-Openvas
-PageAnalyzer
-Pandalytics
-Panscient
-Papa Foto
+MeanPath Bot
+Mediatoolkitbot
+MicroMessenger
+Mozlila
+Musobot
+NICErsPRO
+Netcraft
+Nibbler
+Nimbostratus
+Nmap
+OnCrawl
+OpenVAS
+OrangeBot
+Page Analyzer
+Piepmatz
 Pixray
-Pockey
+PleaseCrawl
+ProPowerBot
+ProWebWalker
+Probethenet
 Pump
-PyCurl
 QueryN Metasearch
+RocketCrawler
 SEOlyticsCrawler
-SalesIntelligent
+ScanAlert
 ScoutJet
-SearchmetricsBot
-Seomoz
-SiteCheckerBotCrawler
-SiteExplorer
-Sitevigil
-SmartDownload
+Scrapy
+SemrushBot
+SiteSucker
+Siteimprove
+Sogou web spider
 Sottopop
-Sqlmap
-Sqlworm
-Sucuri
-Suzuran
-Szukacz
-Tracemyfile
-TwengaBot
-URLy Warning
+SpankBot
+Surfbot
+T0PHackTeam
+TightTwatBot
+Toata
+TurnitinBot
 URLy.Warning
-Upflow
-Vacuum
-Vagabondo
+VCI
+Virusdie
 WASALive-Bot
-WISENutbot
-WebZIP
-Webshag
-Widow
-Wonderbot
+WWW-Collector-E
+WWW::Mechanize
+Web Collage
+WebFuck
+WebImageCollector
+WebReaper
+WebsiteQuester
 Woobot
-arquivo.pt
-autoemailspider
 coccocbot
 com.plumanalytics
-crawl.sogou.com
-dataforseobot
-domainsproject.org
-facebookscraper
-mediawords
-oBot
-openai.com
-pcBrowser
-probe-image-size
-siteripz
+crawler.feedback
+crawler4j
+heritrix
+isitwp.com
+linkdexbot
+lwp-trivial
+ripz
+satoristudio.net
+scalaj-http
+seostar
 trendiction.de
-webgains-bot
-webmeup-crawler
+x09Mozilla
+zgrab