V4.2024.02.4288 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-ASPSeek
+360Spider
+404checker
+404enemy
+ADmantX
+ALittle Client
 Aboundexbot
-Anarchy99
-BackDoorBot
+Anarchy
+Anthill
+Aspiegel
+AwarioSmartBot
 Backlink-Ceck
-Black Hole
-Bolt
-BuiltWith
-Bytespider
-CSHttp
+Battleztar Bazinga
+Blow
+Brandwatch
+Cegbfeieh
 ChatGPT-User
-CheeseBot
 CherryPicker
-Cliqzbot
-Crawling at Home Project
-Crescent
+Chlooe
+Cloud mapping
+CrazyWebCrawler
+Custo
+Disco
+Dispatch
+DittoSpyder
 DomCopBot
 EMail Siphon
-EMail Wolf
-Ecxi
-GetRight
-HTMLparser
-Humanlinks
-HybridBot
-Id-search
-IlseBot
-InfoNaviRobot
-InfoTekies
-Jorgee
-JustView
-Kinza
-LNSpiderguy
-Lanshanbot
+EasyDL
+EyeNetIE
+FacebookBot
+Fuzz
+Genieo
+GermCrawler
+Gigabot
+Grabber
+Harvest
+Iblog
+InternetSeer
+Iskanie
+JennyBot
+Joomla
+LeechFTP
+LexiBot
+Likse
 LinkpadBot
-Litemage_walker
-MIDown tool
-MIIxpc
-Majestic SEO
-Masscan
+LinksManager
+LinqiaRSSBot
+LinqiaScrapeBot
+Lipperhey
+MQQBrowser
+Mass Downloader
 Mediatoolkitbot
 MegaIndex.ru
-Morfeus Fucking Scanner
-Mr.4x3
-Name Intelligence
-NetLyzer
-NimbleCrawler
-Ninja
-Nmap
-Octopus
-Offline Explorer
-Openfind
-OutfoxBot
-POE-Component-Client-HTTP
-PageThing.com
+Microsoft Data Access
+Mister PiX
+Msrabot
+Navroad
+Nessus
+NetZIP
+Nibbler
+Niki-bot
+PageScorer
 Pandalytics
-Picsearch
-PleaseCrawl
-ProWebWalker
-QueryN Metasearch
-RankActive
-Rankivabot
-RedesScrapy
-SEOkicks-Robot
-SEOprofiler
-SEOstats
-SalesIntelligent
+PictureFinder
+Pixray
+Quick-Crawler
+Rainbot
+RankingBot2
+Reaper
 SemanticJuice
-SiteLockSpider
-SiteSucker
-Snapbot
-SocialRankIOBot
-Surfbot
-T0PHackTeam
-Trendictionbot
-V-BOT
-Voil
-Voltron
-WASALive-Bot
-WISENutbot
+Siphon
+SiteCheckerBotCrawler
+Siteimprove
+SpankBot
+Spanner
+SputnikBot
+Steeler
+SuperHTTP
+Telesphorep
+TightTwatBot
+VeriCiteCrawler
 WWW-Mechanize
-WebImageCollector
+Wallpapers
+Web Fuck
+WebCollage
+WebGo IS
+WebSucker
 WebWhacker
-Whack
+WebZIP
 Whatweb
-YoudaoBot
-Zeus
-ZoominfoBot
-archive.org_bot
-crawl.sogou.com
-crawler.feedback
-crawler4j
-demandbase-bot
-facebookscraper
-ips-agent
-lwp-request
-openai.com
+WinHTTrack
+Xaldon WebSpider
+Xaldon_WebSpider
+clark-crawler
+iubenda-radar
+meanpathbot
+mediawords
 s1z.ru
-sp_auditbot
-webprosbot
-x22Mozilla
+seocompany.store
+trendiction.de
+webmeup-crawler