V4.2023.08.3931 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-AhrefsBot
-Alexibot
-AwarioRssBot
-Battleztar Bazinga
-Bigfoot
-BotALot
+404checker
+ADmantX
+AIBOT
+AdsTxtCrawlerTP
+Anarchy99
+Attach
+AwarioSmartBot
+BBBike
+BackDoorBot
+Blackboard
+Brandprotect
 BuiltWith
-Bullseye
-CCBot
-CODE87
-CSHttp
-Citoid
-Cocolyzebot
-Copier
-Copyscape
-Cosmos
-DatabaseDriverMysqli
-Drip
-Ecxi
-ExtLinksBot
-Fyrebot
-GPTBot
+ChinaClaw
+Claritybot
+Craftbot
+CyotekWebCopy
+DBLBot
+DIIbot
+Devil
+DigitalPebble
+DomCopBot
+DomainCrawler
+Express WebPictures
+FemtosearchBot
 Go-Ahead-Got-It
-Gotit
-GrapeFX
-Id-search
-Intelliseek
-Iria
-Iskanie
-JOC Web Spider
+GrabNet
+HTTP::Lite
+Hloader
+IDBTE4M
+IstellaBot
+JamesBOT
 Jbrofuzz
+JennyBot
 Jorgee
-Jyxobot
+Kenjin Spider
 Keyword Density
-Kinza
+LNSpiderguy
+LexiBot
+Lightspeedsystems
 Likse
-Linkbot
-LinkextractorPro
-LinksManager
-LinqiaMetadataDownloaderBot
-Lipperhey Spider
-MIIxpc
-MTRobot
+Ltx71
+MFC_Tear_Sample
 MarkWatch
-MegaIndex.ru
-Mozlila
-Name Intelligence
+MeanPath Bot
+MicroMessenger
+Mister PiX
+Morfeus Fucking Scanner
 Needle
+NetMechanic
+NetSpider
 Nibbler
-NimbleCrawler
-Nutch
-Octopus
+Nikto
+Nmap
+Offline Explorer
+OrangeBot
 OutclicksBot
-PECL::HTTP
-Papa Foto
-PeoplePal
-Pimonster
-Pockey
-ProPowerBot
-RSSingBot
-ReGet
-RedesScrapy
-RocketCrawler
-SEOlyticsCrawler
-SalesIntelligent
-ScanAlert
-ScreenerBot
-SemrushBot
-SentiBot
-Shodan
-SlySearch
-Sottopop
-Sqlworm
-Sucuri
+PleaseCrawl
+Re-re
+RealDownload
+RepoMonkey
+Rogerbot
+SEOprofiler
+SISTRIX
+SMTBot
+SeoSiteCheckup
+Seomoz
+SiteCheckerBotCrawler
+Sitevigil
+Sogou web spider
+Sosospider
+SputnikBot
 SuperBot
-Szukacz
-Telesoft
-TightTwatBot
+Swiftbot
+T8Abot
+Telesphoreo
+The Intraformant
 Titan
-True_Robot
-VelenPublicWebCrawler
-WWW-Collector-E
-WWWOFFLE
-Web Auto
-Web Collage
-WebPix
-WebsiteQuester
-Whatweb
-Zauba
+Toweyabot
+Trendiction
+Turnitin
+URLy.Warning
+Voil
+WEBDAV
+Web Sucker
+WebCopier
+WebFuck
+WebLeacher
+Xaldon_WebSpider
+Xenu
+Zade
 ZyBorg
-archive.org_bot
+arquivo-web-crawler
 crawler.feedback
-domainsproject.org
-openai.com
-plumanalytics
-probe-image-size
-scan.lol
-serpstatbot
-spyfu
-tAkeOut
+dataforseobot
+eCatch
+instabid
+meanpathbot
+openai
+seocompany.store
+siteripz