V4.2024.08.4685 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-404enemy
-Abonti
-Aboundexbot
+01h4x.com
+AIBOT
+Acunetix
+AllSubmitter
+AlphaBot
 Ankit
-Apexoo
-Attach
-BBBike
-BackDoorBot
-Bigfoot
-Black Hole
-Buddy
-CCBot
-CSHttp
-CazoodleBot
-ClaudeBot
-Cogentbot
-CopyRightCheck
+BDCbot
+BackStreet
+Badass
+BatchFTP
+Blow
+BlowFish
+Bolt
+Brandprotect
+BunnySlippers
+CODE87
+CensysInspect
+ChatGPT-User
+ChinaClaw
+Chlooe
 Cosmos
-Craftbot
-Crawling at Home Project
-CrazyWebCrawler
-Curious
-DBLBot
-Devil
-DomainAppender
-Download Wonder
-ECCP/1.0
-Fimap
-Freeuploader
-G-i-g-a-b-o-t
-GT::WWW
-Genieo
-GridBot
-HTMLparser
+Digincore
+EMail Siphon
+EMail Wolf
+EasyDL
+Extreme Picture Finder
+Flunky
+Gotit
+HMView
+HTTP::Lite
 HTTrack
-Haansoft
-Id-search
-Indy Library
-InfoTekies
-InfraSec Scanner
-Kozmosbot
-LNSpiderguy
-LibWeb
-Lightspeedsystems
-Linkbot
-LinkextractorPro
-Litemage_walker
-MQQBrowser
-MeanPath Bot
-Meanpathbot
-MicroMessenger
-Navroad
-NetLyzer
+Intelliseek
+InternetMeasurement
+Jorgee
+Leap
+LeechGet
+Libwhisker
+Likse
+LinqiaRSSBot
+Lmspider
+MIDown tool
+MarkMonitor
+Mediatoolkitbot
+Microsoft Data Access
+Nessus
+Netcraft
 Nikto
-Offline Navigator
-OrangeBot
-PECL::HTTP
+Offline Explorer
+OnCrawl
+OpenLinkProfiler
+OrangeSpider
+OutclicksBot
+OutfoxBot
 POE-Component-Client-HTTP
-PageScorer
-Petalbot
+Pandalytics
+Pi-Monster
+PleaseCrawl
+ProPowerBot
+Probethenet
 PyCurl
-RankActive
-ReGet
-RealDownload
-RebelMouse
-Recorder
+Reaper
 RepoMonkey
-SEOkicks
-Searchestate
-SenutoBot
-SiteLockSpider
-Siteimprove
-Sogou web spider
+SISTRIX
+Screaming
+SemrushBot
+Sosospider
+Sottopop
 Spider_Bot/3.0
-Toweyabot
-Trendiction
-Turnitin
-Vacuum
-VeriCiteCrawler
-WEBDAV
+Sucuri
+SuperBot
+SuperHTTP
+T0PHackTeam
+TeleportPro
+Trendictionbot
+Typhoeus
+VB Project
+VidibleScraper
 WWW-Mechanize
-WWW::Mechanize
-WebFetch
-WebGo IS
-WebImageCollector
-WebWhacker
-Webster
-WiseGuys Robot
-Wprecon
-Zeus
-arquivo-web-crawler
-arquivo.pt
+WeSEE
+Web Fetch
+Web Sucker
+Website Quester
+WebsiteQuester
+Wonderbot
+Xaldon WebSpider
+YoudaoBot
+Zauba
+cognitiveseo
+crawl.sogou.com
 dataforseobot
-linkdexbot
-magpie-crawler
-mediawords
-scalaj-http
-seocompany.store
-sexsearcher
-zauba.io
+demandbase-bot
+gopher
+instabid
+netEstate NE Crawler
+page scorer
+pcBrowser
+sitechecker.pro
+siteripz
+webprosbot
+x09Mozilla