V4.2023.12.4170 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-404checker
+360Spider
+Aboundexbot
+Acunetix
+AiHitBot
 Alexibot
-Aspiegel
-Attach
-BLEXBot
-BacklinkCrawler
-BetaBot
-Blackboard
-BlowFish
-Boardreader
-Bullseye
+Atomseobot
+BDCbot
+Chlooe
 Cogentbot
-Copier
-Custo
-CyotekWebCopy
-DTS Agent
-DnBCrawler-Analytics
-DomCopBot
-EMail Wolf
+Curious
+DataCha0s
+DatabaseDriverMysqli
+Demon
+Discoverybot
+DittoSpyder
+Drip
+EMail Siphon
 Ecxi
-EroCrawler
-Extractor
-FDM
-Fyrebot
-Genieo
-GermCrawler
-GrapeFX
-HMView
-Haansoft
-Havij
-Id-search
-IlseBot
-Intelliseek
-JamesBOT
-Lanshanbot
-Lftp
-LieBaoFast
-LinkScan
-LinqiaScrapeBot
-MIIxpc
+FemtosearchBot
+Fuzz
+GPTBot
+GetRight
+Grabber
+GrapeshotCrawler
+HTTP::Lite
+InfoTekies
+Internet Ninja
+Iria
+JOC Web Spider
+JennyBot
+Jetty
+LNSpiderguy
+Larbin
+LeechFTP
+LibWeb
+Lightspeedsystems
+Lipperhey Spider
+MFC_Tear_Sample
 MSIECrawler
-Mag-Net
-Mass Downloader
-Meanpathbot
-Mediatoolkitbot
-Microsoft URL Control
+Magnet
+Majestic12
 Minefield
-NetAnts
-NimbleCrawler
-Nutch
-Openvas
-Page Analyzer
-PageGrabber
-Pandalytics
-QueryN Metasearch
-Quick-Crawler
+Mojeek
+Mojolicious
+Musobot
+Nessus
+NetLyzer
+NetMechanic
+Niki-bot
+Ninja
+Nuclei
+OpenVAS
+PageThing.com
+Papa Foto
+Pimonster
+Pixray
+RSSingBot
 RankingBot
-SBIder
+RankurBot
+Re-re
+RealDownload
+RocketCrawler
 SMTBot
-ScrepyBot
-Semrush
-SemrushBot
-SenutoBot
-Siteimprove
-Sociscraper
+SemanticJuice
+SentiBot
+Siphon
+Sosospider
+Spammen
+Sucker
 Swiftbot
-Teleport
-Titan
-Vacuum
-VidibleScraper
-VoidEYE
-WeSEE
-WebCopier
-WebImageCollector
-WebLeacher
-Website Quester
-Whack
-Widow
-WiseGuys Robot
-Wonderbot
-Woobot
-Wprecon
+TheNomad
+TightTwatBot
+Tracemyfile
+True_Robot
+TurnitinBot
+Twice
+UnisterBot
+VB Project
+WASALive-Bot
+WWWOFFLE
+Web Collage
+Web Pix
+WebBandit
+WebFetch
+WebsiteExtractor
 Xaldon_WebSpider
-YoudaoBot
-Zade
-Zitebot
-ZyBorg
-arquivo-web-crawler
-crawl.sogou.com
+arquivo.pt
+com.plumanalytics
+crawler.feedback
+dataforseobot
 domainsproject.org
-eCatch
-lwp-request
-magpie-crawler
+evc-batch
+instabid
+lwp-trivial
+netEstate NE Crawler
 oBot
-plumanalytics
-seocompany.store
+openai.com
+seobility
+seoscanners
 serpstatbot
-siteripz
-trendiction.de
-xpymep1.exe