V4.2023.11.4114 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-404enemy
+AIBOT
+ASPSeek
+Aboundexbot
+AdsTxtCrawlerTP
+Alexibot
+AlphaBot
+Anarchy99
+Asterias
 Atomseobot
-AwarioSmartBot
-Badass
-Barkrowler
-Battleztar Bazinga
-Bigfoot
-Blackboard
-BlowFish
-Bolt
-BotALot
-Bytespider
-CherryPicker
-Claritybot
-Craftbot
+BackDoorBot
+BackWeb
+Bandit
+Bitacle
+Blow
+BuiltWith
+CensysInspect
+Crawling at Home Project
 CrunchBot
-CyotekWebCopy
+Curious
 DBLBot
-DTS Agent
-Demon
-Disco
-Dispatch
-DomCopBot
-Fuzz
-Fyrebot
-GalaxyBot
-HTMLparser
-HaosouSpider
-Iskanie
-LWP::Simple
-Larbin
-LibWeb
-LinkextractorPro
-LinqiaScrapeBot
-MIDown tool
-MQQBrowser
-MSIECrawler
-Microsoft URL Control
+DnyzBot
+Domains Project
+Dotbot
+Drip
+EirGrabber
+Getintent
+Go-Ahead-Got-It
+GoZilla
+Grabber
+Grafula
+GridBot
+IlseBot
+InterGET
+Keybot Translation-Search-Machine
+LNSpiderguy
+Lftp
+LinkpadBot
+MJ12bot
+Magnet
+MarkMonitor
+MarkWatch
+MeanPath Bot
+MegaIndex.ru
 Msrabot
-NearSite
-Netvibes
-OnCrawl
-Page Analyzer
-Panscient
-PeoplePal
-Petalbot
-Piepmatz
+Musobot
+Needle
+Net Vampire
+Nikto
+Nmap
+Offline Explorer
+OutfoxBot
+Papa Foto
+Picsearch
+Pixray
 Pockey
-Probethenet
-RankActive
-Rankivabot
-RankurBot
-Re-re
-ReGet
+RSSingBot
+Rainbot
+RankFlex
+RealDownload
+Reaper
 RebelMouse
-SemrushBot
-Sqworm
-Steeler
-Stripper
-SuperBot
-Thumbor
-Twice
-VB Project
+RepoMonkey
+Rogerbot
+SBIder
+SEOkicks-Robot
+SEOstats
+Semrush
+Snapbot
+Sosospider
+SpaceBison
+Sqlmap
+Surfbot
+Suzuran
+Swiftbot
+TinyTestBot
+Toata
+Trendiction
+Turingos
+URLy.Warning
 VidibleScraper
-WASALive-Bot
 WWW-Mechanize
-Wallpapers
-WeSEE
-WebReaper
-WebSauger
-WebZIP
-Website Quester
-WebsiteQuester
-Whacker
-WiseGuys Robot
-Xaldon WebSpider
-Xaldon_WebSpider
-Xenu
+Web Auto
+Web Collage
+Web Fuck
+WebGo IS
+Widow
+Wonderbot
+Wotbox
 Zermelo
-coccocbot
-cognitiveseo
-eCatch
+ZyBorg
+check1.exe
 evc-batch
-facebookscraper
-lwp-request
-muhstik-scan
-netEstate NE Crawler
-oBot
-page scorer
-plumanalytics
-s1z.ru
-seoscanners
-seostar
-serpstatbot
-sexsearcher
-sitechecker.pro
-sogouspider
-sp_auditbot
-spyfu
-tAkeOut
+internetVista monitor
+isitwp.com
+linkdexbot
+seobility
+webgains-bot
+webmeup-crawler
+zauba.io
+zgrab