vrr/multi-scrobbler
2
0
Fork 0
mirror of https://github.com/FoxxMD/multi-scrobbler.git synced 2026-04-28 11:49:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
multi-scrobbler/docsite/docs/configuration/configuration.mdx
FoxxMD 30cdc9f553 docs: Add prometheus usage
2026-03-29 23:50:55 +00:00

750 lines
No EOL
26 KiB
Text
Raw Blame History

---
sidebar_position: 2
title: Overview
toc_max_heading_level: 3
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';
import SchemaLink from "../../src/components/SchemaLink";
import AIOExample from "../../src/components/AIOExample";
import FileExample from "../../src/components/FileExample";
import ScrobbleThreshold from "@site/src/components/snippets/_scrobble-threshold.mdx"
import AIOConfig from '!!raw-loader!../../../config/config.json.example';
import SpotifyConfig from '!!raw-loader!../../../config/spotify.json.example';
:::tip
Check the [**FAQ**](../FAQ.md) if you have any issues after configuration!
:::
## Configuration Types
[**Sources**](/configuration/sources) and [**Clients**](/configuration/clients) are configured using:
* environmental (ENV) variables
* client/source specific json config files
* an all-in-one json config file
**MS will parse configuration from all configuration types.** You can mix and match configurations but it is generally better to stick to one or the other.
<Tabs groupId="configType" queryString>
<TabItem value="env" label="ENV">
MS will parse environmental variables present in the OS/container when it is run. **This method means MS does not require files to run.**
<details>
<summary>Use ENV-based configuration if...</summary>
* You are the only person for whom MS is scrobbling for
* You have a very simple setup for MS such as one scrobble [Client](/configuration/clients) and one [Source](/configuration/sources) IE Plex -> Maloja
</details>
<details>
<summary>Config Example</summary>
For Docker container...
```shell
docker run -e "SPOTIFY_CLIENT_ID=yourId" -e "SPOTIFY_CLIENT_SECRET=yourSecret" ...
```
For Docker Compose
```yaml title="docker-compose.yml"
services:
multi-scrobbler:
image: foxxmd/multi-scrobbler
environment:
- SPOTIFY_CLIENT_ID=yourId
- SPOTIFY_CLIENT_SECRET=yourSecret
- MALOJA_URL=http://domain.tld:42010
- MALOJA_API_KEY=1234
# ...
# ...
```
For a local/node installation export variables before running...
```shell
SPOTIFY_CLIENT_ID=yourId SPOTIFY_CLIENT_SECRET=yourSecret npm run start
```
</details>
</TabItem>
<TabItem value="file" label="File">
MS will parse configuration files located in the directory specified by the `CONFIG_DIR` environmental variable. This variable defaults to:
* Local installation -> `PROJECT_DIR/config`
* Docker -> `/config` (in the container) -- see the [install docs](../installation/installation.mdx#docker) for how to configure this correctly
<details>
<summary>Use File-based configuration if...</summary>
* You have many [Sources](/configuration/sources)
* You have many of each type of **Source** you want to scrobble from IE 2x Plex accounts, 3x Spotify accounts, 1x
Funkwhale...
* You have more than one scrobble **Client** you want to scrobble to IE multiple Maloja servers
* You want only to scrobble to specific **Clients**
* You need to setup more advanced configuration for a Source/Client
* Most Source/Clients only support basic configuration through ENV, all configuration is possible using File/AIO
</details>
:::tip
* There are **example configurations** for all Source/Client types and AIO config located in the [`/config`](https://github.com/FoxxMD/multi-scrobbler/tree/master/config) directory of this project. These can be used as-is by renaming them to `.json`.
* For docker installations these examples are copied to your configuration directory on first-time use.
* There is also a [**kitchensink example**](/configuration/kitchensink) that provides examples of using all sources/clients in a complex configuration.
:::
Each file is named by the **type** of the Client/Source found in below sections. Each file as an **array** of that type of Client/Source.
Example directory structure:
```
/CONFIG_DIR
plex.json
spotify.json
maloja.json
```
<details>
<summary>Config Example</summary>
<CodeBlock title="CONFIG_DIR/spotify.json" language="json5">{SpotifyConfig}</CodeBlock>
</details>
</TabItem>
<TabItem value="aio" label="File AIO">
MS will parse an **all-in-one** configuration file located in the directory specified by the `CONFIG_DIR` environmental variable. This variable defaults to:
* Local installation -> `PROJECT_DIR/config/config.json`
* Docker -> `/config/config.json` (in the container) -- see the [install docs](../installation/installation.mdx#docker) for how to configure this correctly
<details>
<summary>Use AIO-based configuration if...</summary>
* You have many [Sources](/configuration/sources)
* You have many of each type of **Source** you want to scrobble from IE 2x Plex accounts, 3x Spotify accounts, 1x
Funkwhale...
* You have more than one scrobble **Client** you want to scrobble to IE multiple Maloja servers
* You want only to scrobble to specific **Clients**
* You need to setup [monitoring/webhooks](#monitoring)
* You want to setup defaults for all Sources/Clients
</details>
**The AIO config also enables setting default options for sources/clients as well as global options for MS itself.**
:::tip
* An example AIO config files can be found in the project directory at [`/config/config.json.example`](https://github.com/FoxxMD/multi-scrobbler/tree/master/config/config.json.example)
* For docker installations this example is copied to your configuration directory on first-time use.
* There is also a [**kitchensink example**](/configuration/kitchensink) that provides examples of using all sources/clients in a complex configuration.
:::
[**Explore the schema for this configuration, along with an example generator and validator, here**](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fmaster%2Fsrc%2Fbackend%2Fcommon%2Fschema%2Faio.json)
<details>
<summary>Config Example</summary>
<CodeBlock title="CONFIG_DIR/config.json" language="json5">{AIOConfig}</CodeBlock>
</details>
</TabItem>
</Tabs>
## Setup Sources and Clients
See the [Configuration Types](#configuration-types) above for your options for creating Source and Client configurations.
* Reference [**Scrobble Sources**](/configuration/sources) for what Sources are available
* Reference [**Scrobble Clients**](/configuration/clients) for what Clients are available
Each entry for a Source/Client includes a **Configuration** section that describes how to configure it using a [configuration type](#configuration-types).
## Secrets Interpolation
When using [File](./?configType=file#configuration-types) or [AIO](./?configType=aio#configuration-types) Configuration, Multi-Scrobbler can interpolate Environmental Variables into your json files. This can be used, for example, to keep sensitive data (like [Last.fm Client/Secret](/configuration/clients/lastfm#configuration)) out of your configuration files so that they can be committed to git.
Multi-scrobbler will look for patterns in text fields within _all_ of your json files:
* Some part of the text field matches: `[[MY_ENV]]`
* Is replaced by the value of the Environmental Variable named `MY_ENV`
<details>
<summary>Example</summary>
Given this [Last.fm](/configuration/clients/lastfm#configuration) [File](./?configType=file#configuration-types) config:
```json title="lastfm.json"
[
{
"name": "myLastFmClient",
"configureAs": "client",
"data": {
"apiKey": "[[MY_APIKEY]]",
"secret": "[[MY_SECRET]]",
"redirectUri": "http://localhost:9078/lastfm/callback"
}
}
]
```
And these environmental variables, in this scenario set through `environment` in the [docker compose installation](/quickstart#create-docker-compose-file):
```
MY_APIKEY=a89cba1569901a0671d5a9875fed4be1
MY_SECRET=ec42e09d5ae0ee0f0816ca151008412a
```
The resulting json multi-scrobbler would use:
```json title="lastfm.json"
[
{
"name": "myLastFmClient",
"configureAs": "client",
"data": {
"apiKey": "a89cba1569901a0671d5a9875fed4be1",
"secret": "ec42e09d5ae0ee0f0816ca151008412a",
"redirectUri": "http://localhost:9078/lastfm/callback"
}
}
]
```
</details>
<DetailsAdmo type="note" summary="Caveats">
* ENV variable names/interpolation keys are case-insensitive
* Interpolation only works for **string** values within json. This cannot be used for numbers, booleans, objects, etc...
</DetailsAdmo>
<DetailsAdmo type="warning" summary="Missing ENVs">
Multi-scrobbler **will not** throw an error if the environmental value is not found. Instead, it will leave the string as-is and log a warning (`WARN` level) with the names of the missing environmental variable names like so:
```
WARN : [App] [Sources] [spotify Secrets] Matched: None | Unmatched: SPOTIFY_SECRET
```
</DetailsAdmo>
<DetailsAdmo type="warning" summary="ENV Name Collisions">
**Verify that interpolation keys/environmental variable names you will use do not collide with existing ENV names used by multi-scrobbler.** Use the docs search to verify the name you want to use is not already used elsewhere by multi-scrobbler.
</DetailsAdmo>
## Application Options
These options affect multi-scrobbler's behavior and are not specific to any source/client.
### Base URL
Defines the URL that is used to generate default redirect URLs for authentication on [spotify](/configuration/sources/spotify) and [lastfm](/configuration/clients/lastfm) -- as well as some logging hints.
* Default => `http://localhost:9078`
* Set with [ENV](./?configType=env#configuration-types) `BASE_URL` or `baseUrl` [all-in-one configuration](./?configType=aio#configuration-types)
* If protocol is `http` or no protocol is specified MS will try to use port `9078` -- to override this explicitly set the port or use `https`
Useful when running with [docker](../installation/installation.mdx#docker) so that you do not need to specify redirect URLs for each configuration.
<details>
<summary>Example</summary>
EX Lastfm Redirect Url is `BASE_URL:PORT/lastfm/callback` (when no other redirectUri is specified for [lastfm configuration](/configuration/clients/lastfm))
| `BASE_URL` | Redirect URL |
|-------------------------------------|----------------------------------------------------------|
| `192.168.0.101` | `http://192.168.0.101:9078/lastfm/callback` |
| `http://my.domain.local` | `http://my.domain.local:9078/lastfm/callback` |
| `http://192.168.0.101/my/subfolder` | `http://192.168.0.101:9078/my/subfolder/lastfm/callback` |
| `BASE_URL` | Redirect URL |
|-----------------------------------|----------------------------------------------------------|
| `my.domain.local:80` | `http://192.168.0.101:80/lastfm/callback` |
| `my.domain.local:9000` | `http://my.domain.local:9000/lastfm/callback` |
| `192.168.0.101:4000/my/subfolder` | `http://192.168.0.101:4000/my/subfolder/lastfm/callback` |
| `https://192.168.0.101` | `https://192.168.0.101:443/lastfm/callback` |
</details>
### Caching
Multi-scrobbler caches some activities to persist important data across restarts, reduce external API calls, and make some actions faster.
All of the activities below are **always** cached **in-memory** with an optional, configurable [**secondary** store](#secondary-caching-configuration) for persistence.
<Tabs groupId="cachedThings" queryString>
<TabItem value="scrobbles" label="Scrobbles">
**Queued** and **Failed** Scrobbles are cached so that any un-scrobbled data you have is persisted across restarts of multi-scrobbler.
:::tip
By default, this data use a [Secondary](#secondary-caching-configuration) [File](./?cacheType=file#secondary-caching-configuration) store, configured for you automatically.
If you have configured a [persisted volume/bind mount](/installation#storage) for configuration (`/config` is mounted in [docker compose](/quickstart#create-docker-compose-file)) then you are already done. If you are not persisting this directory then you should consider setting up [Valkey Cache](./?cacheType=valkey#secondary-caching-configuration) for this.
:::
##### Configuration
Use any [Secondary Cache](#secondary-caching-configuration), the config examples below show the default values:
<Tabs>
<TabItem value="env" label="ENV">
| Environmental Variable | Required? | Default | Description |
| :--------------------- | --------- | --------- | --------------------- |
| `CACHE_SCROBBLE` | No | `file` | The cache type to use |
| `CACHE_SCROBBLE_CONN` | No | `/config` | |
</TabItem>
<TabItem value="aio" label="AIO">
```json5 title="config.json"
{
"cache": {
"scrobble": {
"provider": "file",
"connection": "/config"
}
},
// ...
}
```
</TabItem>
</Tabs>
</TabItem>
<TabItem value="auth" label="Auth Data">
Authentication sessions/tokens/etc... are cached for quicker requests and for persistence across restarts.
:::tip
By default, this data use a [Secondary](#secondary-caching-configuration) [File](./?cacheType=file#secondary-caching-configuration) store, configured for you automatically.
If you have configured a [persisted volume/bind mount](/installation#storage) for configuration (`/config` is mounted in [docker compose](/quickstart#create-docker-compose-file)) then you are already done. If you are not persisting this directory then you should consider setting up [Valkey Cache](./?cacheType=valkey#secondary-caching-configuration) for this.
:::
##### Configuration
Use any [Secondary Cache](#secondary-caching-configuration), the config examples below show the default values:
<Tabs>
<TabItem value="env" label="ENV">
| Environmental Variable | Required? | Default | Description |
| :--------------------- | --------- | --------- | --------------------- |
| `CACHE_AUTH` | No | `file` | The cache type to use |
| `CACHE_AUTH_CONN` | No | `/config` | |
</TabItem>
<TabItem value="aio" label="AIO">
```json5 title="config.json"
{
"cache": {
"auth": {
"provider": "file",
"connection": "/config"
}
},
// ...
}
```
</TabItem>
</Tabs>
</TabItem>
<TabItem value="metadata" label="Transform API Calls">
API Calls to external (metadata) services used to [Enhance Scrobbles](/configuration/transforms), like calls to [Musicbrainz](/configuration/transforms/musicbrainz), can be cached to avoid duplicate calls and speed up scrobble transformations.
By default, these calls are only cached in memory. If you wish for cached calls to be persisted across restarts then setup [Valkey Cache](./?cacheType=valkey#secondary-caching-configuration).
##### Configuration
Use any [Secondary Cache](#secondary-caching-configuration), the config examples below show the default values:
<Tabs>
<TabItem value="env" label="ENV">
| Environmental Variable | Required? | Default | Description |
| :--------------------- | --------- | ------- | --------------------- |
| `CACHE_METADATA` | No | | The cache type to use |
| `CACHE_METADATA_CONN` | No | | |
</TabItem>
<TabItem value="aio" label="AIO">
```json5 title="config.json"
{
"cache": {
"metadata": {
"provider": "valkey",
"connection": "yourConnectionStringHere"
}
},
// ...
}
```
</TabItem>
</Tabs>
</TabItem>
</Tabs>
#### Secondary Caching Configuration
The type of cache used, and its connection properties, can be configured through **ENV** or **AIO** config.
<Tabs groupId="cacheType" queryString>
<TabItem value="file" label="File">
**File** cache is stored in the `CONFIG_DIR` directory using the pre-defined file name `ms-[cacheName].cache`.
<Tabs groupId="configType" queryString>
<TabItem value="env" label="ENV">
Example
| Environmental Variable | Required? | Default | Description |
| :--------------------- | --------- | --------- | ------------------------------------------------------------ |
| `CACHE_SCROBBLE` | No | `file` | The cache type to use |
| `CACHE_SCROBBLE_CONN` | No | `/config` | The directory, within the container, to store the cache file |
</TabItem>
<TabItem value="aio" label="AIO">
Example
```json5 title="config.json"
{
"cache": {
"scrobble": {
"provider": "file",
"connection": "/config"
}
},
// ...
}
```
</TabItem>
</Tabs>
</TabItem>
<TabItem value="valkey" label="Valkey">
[**Valkey**](https://valkey.io/) is an open-source fork of Redis.
<details>
<summary>Adding Valkey container to multi-scrobbler</summary>
A valkey container can be added to the [multi-scrobbler docker compose stack](/installation?runType=docker-compose#docker):
```yaml title="docker-compose.yml"
services:
multi-scrobbler:
# ...
# adding everything below
// highlight-start
valkey:
image: valkey/valkey
volumes:
- valkeydata:/data
volumes:
valkeydata:
driver: local
// highlight-end
```
Use `redis://valkey:6379` as the connection string in the configurations below.
</details>
The connection string used by multi-scrobbler to connect to your Valkey instance must be in the form:
```
redis://HOST_IP:HOST_PORT
```
<Tabs groupId="configType" queryString>
<TabItem value="env" label="ENV">
Example
| Environmental Variable | Required? | Default | Description |
| :--------------------- | --------- | -------- | ------------------------------------------------------------------- |
| `CACHE_METADATA` | Yes | `valkey` | The cache type to use |
| `CACHE_METADATA_CONN` | Yes | | The host/IP and port to connect to EX: `redis://192.168.0.120:6379` |
</TabItem>
<TabItem value="aio" label="AIO">
Example
```json5 title="config.json"
{
"cache": {
"metadata": {
"provider": "valkey",
"connection": "redis://192.168.0.120:6379"
}
},
// ...
}
```
</TabItem>
</Tabs>
</TabItem>
</Tabs>
### Debug Mode
Turning on Debug Mode will
* override and enable all optional "debugging" options found in configuration
* set log output to DEBUG
Use this as a shortcut for enabling output that can be used for troubleshooting and issue reporting. Be aware that logs will likely be VERY noisy while Debug Mode is on. You should only have this mode on while gathering logs for troubleshooting and then turn it off afterwards.
To set debug mode either add it to [AIO `config.json`](./?configType=aio#configuration-types)
```json5 title="config.json"
{
"debugMode": true,
"sources": [...],
// ...
}
```
or set the [ENV](./?configType=env#configuration-types) `DEBUG_MODE=true`
### Disable Web
If you do not need the dashboard and/or ingress sources, or have security concerns about ingress and cannot control your hosting environment, the web server and API can be disabled.
:::warning
Any **[ingress-based sources](/configuration/sources?sourceComm=ingress#by-communication-method) will be unusable** (Webscrobbler, etc...) if this is disabled.
:::
Disable using either:
* ENV `DISABLE_WEB=true`
* In [All-in-One File](./?configType=aio#configuration-types) use the top-level property `"disableWeb": true`
## Monitoring
### Upstream Services Status
A status page for monitoring the reachability of public services used by some Sources/Clients is available at
**https://status.multi-scrobbler.app**
This monitor runs on a VPS and checks the uptime of *actual* APIs for these services, not just landing pages. On a ~minute interval it checks:
* Listenbrainz API
* Musicbrainz API
* CoverArtArchive API
* Last.fm API
* A few MS-hosted support services
### Webhook Configurations
Webhooks will **push** a notification to your configured servers on these events:
* Source polling started
* Source polling retry
* Source polling stopped on error
* Scrobble client scrobble failure
Webhooks are configured in the AIO [config.json](#configuration-types) file under the `webhook` top-level property. Multiple webhooks may be configured for each webhook type.
<details>
<summary>Example</summary>
```json5 title="config.json"
{
"sources": [
//...
],
"clients": [
//...
],
"webhooks": [
{
"name": "FirstGotifyServer",
"type": "gotify",
"url": "http://192.168.0.100:8070",
"token": "abcd"
},
{
"name": "SecondGotifyServer",
"type": "gotify",
//...
},
{
"name": "NtfyServerOne",
"type": "ntfy",
//...
},
//...
]
}
```
</details>
#### [Gotify](https://gotify.net/)
Refer to the [config schema for GotifyConfig](https://json-schema.app/view/%23/%23%2Fdefinitions%2FGotifyConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fmaster%2Fsrc%2Fbackend%2Fcommon%2Fschema%2Faio.json)
multi-scrobbler optionally supports setting message notification priority via `info` `warn` and `error` mappings.
<details>
<summary>Example</summary>
```json title="config.json"
{
"type": "gotify",
"name": "MyGotifyFriendlyNameForLogs",
"url": "http://192.168.0.100:8070",
"token": "AQZI58fA.rfSZbm",
"priorities": {
"info": 5,
"warn": 7,
"error": 10
}
}
```
</details>
#### [Ntfy](https://ntfy.sh/)
Refer to the [config schema for NtfyConfig](https://json-schema.app/view/%23/%23%2Fdefinitions%2FNtfyConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fmaster%2Fsrc%2Fbackend%2Fcommon%2Fschema%2Faio.json)
multi-scrobbler optionally supports setting message notification priority via `info` `warn` and `error` mappings.
<details>
<summary>Example</summary>
```json title="config.json"
{
"type": "ntfy",
"name": "MyNtfyFriendlyNameForLogs",
"url": "http://192.168.0.100:9991",
"topic": "RvOwKJ1XtIVMXGLR",
"username": "Optional",
"password": "Optional",
"priorities": {
"info": 3,
"warn": 4,
"error": 5
}
}
```
</details>
#### [Apprise](https://github.com/caronc/apprise-api)
Refer to the [config schema for AppriseConfig](https://json-schema.app/view/%23/%23%2Fdefinitions%2FAppriseConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fmaster%2Fsrc%2Fbackend%2Fcommon%2Fschema%2Faio.json)
multi-scrobbler supports [stateless](https://github.com/caronc/apprise-api?tab=readme-ov-file#stateless-solution) and [persistent storage](https://github.com/caronc/apprise-api?tab=readme-ov-file#persistent-storage-solution) endpoints as well as [tags](https://github.com/caronc/apprise-api?tab=readme-ov-file#tagging)/
<details>
<summary>Example</summary>
```json5 title="config.json"
{
"type": "apprise",
"name": "MyAppriseFriendlyNameForLogs",
"host": "http://192.168.0.100:8080",
"urls": ["gotify://192.168.0.101:8070/MyToken"], // stateless endpoints
"keys": ["e90b20526808373353afad7fb98a201198c0c3e0555bea19f182df3388af7b17"], //persistent storage endpoints
"tags": ["my","optional","tags"]
}
```
</details>
### Health Endpoint
An endpoint for monitoring the health of sources/clients is available at GET `http://YourMultiScrobblerDomain/api/health`
* Returns `200 OK` when **everything** is working or `500 Internal Server Error` if **anything** is not
* The plain url (`/api/health`) aggregates status of **all clients/sources** -- so any failing client/source will make status return 500
* Use query params `type` or `name` to restrict client/sources aggregated IE `/api/health?type=spotify` or `/api/health?name=MyMaloja`
* On 500 the response returns a JSON payload with `messages` array that describes any issues
* For any clients/sources that require authentication `/api/health` will return 500 if they are **not authenticated**
* For sources that poll (spotify, yt music, subsonic) `/api/health` will 500 if they are **not polling**
### Prometheus
A [Prometheus](https://prometheus.io/) export endpoint is available at GET `http://YourMultiScrobblerDomain/api/metrics`
It includes metrics for:
* Count of discovered plays per [Source](/configuration/sources)
* Count of Queued/Scrobbled/Deadletter scrobbles per [Client](/configuration/clients)
* Number of issues per Source/Client
* If any of these metrics is > 0 it means your Source/Client is not operating normally
Additionally, general process metrics like cpu and memory usage can be enabled with the env `PROMETHEUS_FULL=true`
<details>
<summary>Example Usage</summary>
Update your [Prometheus Configuration file](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration-file) (usually at `/etc/prometheus.yml`) to add a job to the `scrape_configs`[section](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) like this:
```yaml
scrape_configs:
#
# ...your other jobs are here
#
- job_name: multi-scrobbler
scrape_interval: 30s
metrics_path: '/api/metrics'
static_configs:
- targets: ['myMultiscrobblerIp:9078']
```
</details>
Reference in a new issue View git blame Copy permalink