V4.2024.03.4373 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
 404checker
-Anarchy99
+80legs
+Abonti
 Ankit
-Atomseobot
-BackWeb
-Barkrowler
-Brandprotect
-Brandwatch
-CSHttp
-Calculon
-CensysInspect
-Cogentbot
+Asterias
+Attach
+BatchFTP
+BuzzSumo
+CATExplorador
+CODE87
+Cegbfeieh
+ChinaClaw
+Cliqzbot
+Copier
+CopyRightCheck
+CrazyWebCrawler
+Custo
+CyotekWebCopy
+DatabaseDriverMysqli
+Deusu
 DnyzBot
-Dragonfly
-EMail Siphon
-EMail Wolf
-Exabot
+Drip
+ECCP/1.0
+Evil
 Express WebPictures
-FDM
-FacebookBot
-Freeuploader
-FrontPage
-Fuzz
-GalaxyBot
+Ezooms
+Foobot
+G-i-g-a-b-o-t
+GetWeb
 GoZilla
-Heritrix
-Id-search
-InternetSeer
-Keyword Density
-LNSpiderguy
-Lanshanbot
-Larbin
-LinkWalker
-Linkbot
+Grafula
+GridBot
+HTTP::Lite
+Indy Library
+InfoTekies
+Internet Ninja
+Jyxobot
+Kinza
+LexiBot
 LinksManager
-Ltx71
-Majestic12
-MegaIndex.ru
-Microsoft Data Access
-Microsoft URL Control
-Mister PiX
-Mojeek
+LinqiaRSSBot
+MFC_Tear_Sample
+MSFrontPage
+Mag-Net
+Magnet
+Mail.RU_Bot
+MarkWatch
+Minefield
+MolokaiBot
 Mr.4x3
-Net Vampire
-NetZIP
-Nibbler
-OrangeSpider
-PageThing.com
-Pixray
+Msrabot
+Musobot
+Navroad
+NetAnts
+NetSpider
+OnCrawl
+OpenLinkProfiler
+OrangeBot
+Pimonster
 Pump
+Quick-Crawler
+RankActive
 RankActiveLinkBot
-RankingBot2
-ReGet
-RealDownload
-Reaper
-RedesScrapy
-SEOlyticsCrawler
-Semrush
-SeoSiteCheckup
-SeobilityBot
-Siphon
-SiteCheckerBotCrawler
+Rogerbot
+SISTRIX
+SearchmetricsBot
+SentiBot
 SiteLockSpider
-SiteSucker
-SocialRankIOBot
-Stripper
-Suzuran
-Telesoft
-Telesphoreo
-The Intraformant
-Toata
-Turingos
-TurnitinBot
-Upflow
-Vagabondo
+SpankBot
+Sqworm
+Sucker
+SuperBot
+SuperHTTP
+Teleport
+Telesphorep
+Twice
+URLy Warning
 VeriCiteCrawler
-VidibleScraper
 WBSearchBot
-WWW::Mechanize
+WEBDAV
 Web Collage
+WebBandit
 WebFuck
+WebPix
+WebSucker
 WebWhacker
 WebZIP
-Webster
+Webshag
+Whacker
 Xaldon_WebSpider
-Zade
-Zermelo
-ZumBot
-anthropic-ai
-check1.exe
-clark-crawler
-coccocbot
-eCatch
-heritrix
-muhstik-scan
-scan.lol
-sexsearcher
-trendiction.com
-trendiction.de
-voyagerx.com
+archive.org_bot
+crawler4j
+gopher
+ips-agent
+iubenda-radar
+lwp-request
+mediawords
+tAkeOut
+webpros.com
+webprosbot