V4.2024.02.4325 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
+360Spider
+404checker
+80legs
 Abonti
-AhrefsBot
-Ankit
-Attach
-BLEXBot
-BacklinkCrawler
-Badass
-Black Hole
-Blackboard
-Buddy
+Atomseobot
+Brandwatch
 BuiltWith
-Chlooe
-Cliqzbot
-Copier
-Copyscape
-CrunchBot
-Devil
-DnBCrawler-Analytics
+BuzzSumo
+Bytespider
+CCBot
+CODE87
+CheeseBot
+CherryPicker
+ChinaClaw
+Deusu
 DomainCrawler
-Domains Project
-Download Wonder
+Dragonfly
 EasyDL
-Ecxi
-Evil
-EyeNetIE
-Genieo
-GermCrawler
-GetRight
-GoZilla
+Exabot
+Ezooms
+G-i-g-a-b-o-t
+GPTBot
+GT::WWW
+GalaxyBot
+Getintent
 Gotit
-Grafula
+GrabNet
+GridBot
+HEADMasterSEO
 Haansoft
-HonoluluBot
+Harvest
+Hloader
+IDBTE4M
 Image Fetch
-Intelliseek
-Iria
-JamesBOT
-JikeSpider
-JustView
+JOC Web Spider
+Jyxobot
 LWP::Simple
-LinksManager
+Lanshanbot
+Larbin
+Leap
+LeechGet
+Lftp
+Likse
 LinqiaMetadataDownloaderBot
-Lipperhey
-MIDown tool
-MIIxpc
-MSIECrawler
-Magnet
-Majestic SEO
-Majestic12
+MFC_Tear_Sample
+MSFrontPage
+Mail.RU_Bot
+MarkMonitor
+MarkWatch
+Mister PiX
+Name Intelligence
 NearSite
-NetSpider
-Nikto
-Ninja
+Nimbostratus
 Offline Explorer
-Openvas
-POE-Component-Client-HTTP
-PageGrabber
-Papa Foto
+OnCrawl
+Page Analyzer
 Piepmatz
-ProPowerBot
+PleaseCrawl
 ProWebWalker
-RankActive
+Pu_iN
+Pump
+Quick-Crawler
+RSSingBot
+Re-re
+RealDownload
+RebelMouse
 RedesScrapy
-SEOstats
-SISTRIX
-SearchmetricsBot
-Seekport
-Semrush
-SentiBot
-SiteExplorer
-Sitevigil
-Snoopy
-Sogou web spider
-Spammen
-Spbot
-Sucuri
-Swiftbot
-Telesphorep
-TinyTestBot
-V-BOT
-VCI
-VeriCiteCrawler
+SBIder
+Scanbot
+Siphon
+SmartDownload
+Snake
+Spinn3r
+Sucker
+TheNomad
+Toweyabot
+Twice
+Typhoeus
 VidibleScraper
-Voil
+WEBDAV
 WWW-Collector-E
-WWW-Mechanize
-WWW::Mechanize
-Web Fetch
-Web Fuck
-WebFuck
-WebsiteExtractor
-Who.is Bot
-WiseGuys Robot
-check1.exe
-ips-agent
-isitwp.com
-seocompany.store
-seostar
-sogouspider
-webprosbot
+WWWOFFLE
+WebGo IS
+WebSucker
+WebWhacker
+WebZIP
+Webalta
+WebmasterWorldForumBot
+ZmEu
+ZumBot
+arquivo.pt
+com.plumanalytics
+crawler4j
+domainsproject.org
+evc-batch
+s1z.ru
+scan.lol
+seobility
+sitechecker.pro
+trendiction.com