Commit graph

17 commits

Author SHA1 Message Date
Deluan Quintão
f580d3ea76
feat(plugins): expose the song Matcher as a host service (#5643)
* feat(plugins): add public Track and Artist DTOs for host services

* feat(plugins): add Matcher host-service interface and MatchSong DTO

* feat(plugins): generate Matcher host wrappers, PDK clients, and matcher permission

* feat(plugins): implement Matcher host service and MediaFile-to-Track converter

Also fixes an ndpgen bug where ParseDirectory parsed each host-service file
in isolation, so a service method referencing a struct defined in another
file of the same package (host.Track in track.go) could not be resolved.
ParseDirectory now collects package-wide structs in a first pass, mirroring
ParseCapabilities; PDK clients regenerated cleanly via make gen.

* feat(plugins): register Matcher host service in the manager

* test(plugins): add Matcher host service integration test plugin

* refactor(plugins): simplify matcher converter and parser file collection

- toTrack: use gg.V for nil-able field derefs and slice.Map for genres/
  participants, removing the repeated nil-guard blocks and inner loop
- manager_loader: drop the redundant ds==nil guard (loadEnabledPlugins
  already gates a nil DataStore), matching the other service entries
- ndpgen parser: extract collectGoFiles, shared by ParseDirectory and
  ParseCapabilities instead of duplicating the file-filter loop

* fix(plugins): keep nullable Track numerics as pointers

ReplayGain values, BitDepth, and BPM are nullable in model.MediaFile, and 0
is a valid measured ReplayGain value. Flattening them to value types with
omitempty made a real 0 indistinguishable from absent. Model them as *float64
/*int32 so plugins can tell 'no data' from a measured 0. Regenerated PDK
clients; converter passes the model pointers through (RG) or maps *int->*int32
(BitDepth/BPM).

* refactor(plugins): trim redundant pass labels in ndpgen ParseDirectory

The function doc already explains the two-pass approach; the inline labels
restated it. Reduce to bare waypoints.

* fix(plugins): gate Track.Path on library filesystem permission

MatchSongs copied mf.Path into every result unconditionally, letting a plugin
with only the matcher permission enumerate on-disk file paths by matching known
songs. Gate Path behind library.filesystem, matching the Library host service.
toTrack is now a method carrying the permission flag.

* refactor(plugins): align MatchSong JSON casing and parse Go files once

- MatchSong: artistMBID/albumMBID JSON tags -> artistMbid/albumMbid so the Go
  wire format matches the Rust SDK's camelCase serialization (cross-SDK fix)
- MatchSongs doc reworded to language-neutral 'empty (absent)' so generated
  Rust/Python client docs no longer say Go-specific 'nil'
- ndpgen: parse each package file once (parseGoFiles) and reuse the ASTs across
  both passes in ParseDirectory and ParseCapabilities, instead of re-parsing

* refactor(plugins): use shared types for Matcher host service

Move the Matcher host service onto the shared plugins/types package instead
of the host-local MatchSong and Track structs. MatchSongs now takes
[]types.SongRef and returns []*types.Track, dropping host.MatchSong and moving
host.Track (with its host.Artist dependency collapsed onto types.ArtistRef) into
plugins/types. ArtistRef gains SortName and SubRole so it can back a track's
Participants.

SongRef gains a millisecond-precision DurationMs field that supersedes the now
deprecated seconds-based Duration, with DurationInMs() resolving the effective
value and SetDurationMs() keeping both fields in sync when populating a SongRef
to send to a plugin.

The ndpgen host-wrapper template only ever imported context, json and extism, so
a host service referencing the shared types package produced uncompilable code.
Emit the plugins/types import when the service references shared types directly
(gated on the existing Service.ImportsSharedTypes), matching the client template,
and cover it with GenerateHost tests. This removes the need for host-local
re-export aliases. Regenerated the Go/Rust/Python PDK and capability schemas
accordingly.

* test(plugins): cover SongRef duration and artist conversion

Add unit coverage for the new SongRef behavior: SetDurationMs populating both
DurationMs and the deprecated seconds field, and the SongRef-to-agents.Song
conversion preferring DurationMs over Duration and the Artists list over the
scalar Artist/ArtistMBID.

Extract the inline SongRef-to-agents.Song closure in MatchSongs into a named
toAgentSong function so the conversion can be asserted directly rather than only
through the opaque matcher. The end-to-end wire shape of the moved types is
already validated by the existing MatcherService integration test, so no new
WASM-boundary test is needed.

* fix(plugins): harden and unify SongRef-to-agents.Song duration conversion

Address findings from a code review of the matcher host service:

- DurationInMs now clamps a negative deprecated-seconds value to 0 instead of
  converting it through uint32, which previously wrapped a value like -1s into a
  ~49-day duration that corrupted the matcher's duration-proximity tiebreaker.
- Replace the unused SetDurationMs(uint32) with SetDuration(seconds float32),
  which takes the unit callers actually hold (model.MediaFile.Duration is
  float32 seconds) and centralizes the seconds-to-ms conversion. Wire it into
  mediaFileToSongRef so outbound SongRefs carry both duration fields in sync.
- Make the metadata-agent path use DurationInMs() so every consumer of the
  shared SongRef honors the DurationMs-over-Duration precedence contract; a
  plugin sending only DurationMs no longer loses its duration on that path.
- Collapse the matcher's duplicate toAgentSong/agentArtists helpers into the
  existing songRefToAgentSong converter, so there is a single SongRef-to-Song
  mapping. Tests narrowed to the duration cases, with artist precedence still
  covered in metadata_agent_test.go.

* feat(plugins): allow Matcher host service to scope a match to a user

Add an options struct to the Matcher host service so a plugin can run a match as
a specific user. When MatchOptions.Username is set, the match is run in that
user's context: their favourites and ratings inform the matcher's tiebreaker, and
the returned tracks carry that user's per-user annotations (Starred, StarredAt,
Rating, PlayCount, PlayDate, added to types.Track). An empty username preserves
the previous unscoped behaviour.

Cross-user access is gated by the same allowedUsers/allUsers permission the Users
and SubsonicAPI host services use: an unknown username, or one the plugin is not
permitted to act as, returns an error. User-library access applies automatically
once the user is in context (applyLibraryFilter). Independently, results are now
restricted to the libraries the plugin itself may access via the precomputed
libraryAccess set, dropping any matched track outside that set (the input index
stays unmatched) — this applies even without a username and even for an
admin-scoped user.

core/matcher is unchanged: it already loads and uses annotations and applies
user-library filtering from context, so the feature works by deriving the request
context and post-filtering by plugin library access in the host adapter. The new
opts parameter and the Track annotation fields are propagated to all PDK clients
(Go/Rust/Python) by make gen.

* fix(plugins): correct Matcher library scope and unify user-access checks

Address findings from a code review of the user-scoped Matcher host service:

- The plugin-library post-filter previously dropped every match for a plugin that
  holds only the matcher permission, because library config is tied to the Library
  permission and a matcher-only plugin has none (empty allowedLibraries,
  AllLibraries=false). Gate the filter on whether the plugin actually declared the
  Library permission: matcher-only plugins are no longer library-restricted, while
  plugins that opt into a library scope are enforced as before. The per-user
  library filter (applyLibraryFilter) still applies whenever a non-admin user is
  scoped.

- resolveUser collapsed every FindByUsername error (including transient DB
  failures) into a misleading "not found". Extract a shared userAccess type
  (alongside libraryAccess) whose resolve() distinguishes model.ErrNotFound from a
  real backend error and authorizes the user against the allowed set. The Matcher
  service now uses it, and host_subsonicapi shares the same userAccess type for its
  permission check (preserving its existing error messages), removing a third
  divergent copy of the resolve-and-authorize logic.

- Document in the matcher tests that the mock MediaFileRepo returns annotations
  unconditionally, so the unit tests cover the adapter's scoped-flag gating and
  access checks but not the SQL per-user join. Add tests for the library-permission
  gating and for surfacing a backend error instead of masking it as not-found.

* fix(plugins): require a library scope for Matcher, fail closed

Reverse the permissive default introduced when fixing the library post-filter: a
Matcher plugin now must be granted a library scope (all libraries, or at least one
specific library) and MatchSongs rejects the request with "no libraries
configured" when it has none, instead of either silently matching nothing or
defaulting to every library.

This mirrors how the SubsonicAPI host service requires a user scope
(checkPermissions errors with "no users configured" when none is set): the check
is a runtime guard via libraryAccess.configured(), needs no manifest changes, and
keeps the failure loud rather than silent. The per-match library post-filter then
always applies, and the restrictLibraries flag added in the previous commit is
removed.

* fix(plugins): require library permission for matcher; guard nil user

Close the gap where a plugin declaring only the matcher permission loaded
successfully but failed every MatchSongs call with "no libraries configured",
with no way for an admin to grant a library scope (the library-config UI is gated
on the library permission). Add a cross-field manifest rule, mirroring the
existing "subsonicapi requires users" rule, so the matcher permission requires
the library permission to be declared. A matcher plugin therefore also surfaces
the library-config panel and is subject to the existing load/enable-time library
configuration gate, making the fail-closed library check reachable and fixable
rather than a silent dead end. The test plugin manifest now declares the library
permission accordingly.

Also restore a defensive nil-user guard in userAccess.resolve: if a DataStore's
FindByUsername ever returns (nil, nil) instead of model.ErrNotFound, return a
clean "not found" error rather than dereferencing a nil *model.User.

* feat(plugins): expose track AverageRating in Matcher results

Add AverageRating to the Matcher's Track DTO. Unlike the per-user annotations
(Starred, Rating, PlayCount, ...), AverageRating is an aggregate stored on the
track itself and is loaded regardless of the request user, so it is populated
unconditionally rather than gated on a scoped username. Propagated to the PDK
types by make gen.

Signed-off-by: Deluan <deluan@navidrome.org>

* style(plugins): trim verbose comments in matcher host service

Condense the over-long explanatory comments added across the matcher host
service to one-liners that state the why, and simplify the ptrInt32/unixPtr
helpers to Go 1.26's new(value). No behavior change.

* refactor(plugins): pass userAccess into newSubsonicAPIService

Move newUserAccess construction to the loader call site so the SubsonicAPI service
constructor takes a userAccess value directly, matching newMatcherService. Pure
refactor: the service already stored a userAccess internally, so behavior and error
messages are unchanged.

* fix(plugins): regenerate PDK and drop omitempty from AverageRating

Re-run make gen so the generated PDK doc comments match the source comment
trimmed in an earlier commit (the source was simplified but the PDK was not
regenerated, leaving the committed files stale — a 'generated files up to date'
hazard).

Also drop omitempty from Track.AverageRating: it is always set (0 when unrated),
so it should be present in the payload like the other always-set fields
(BirthTime/CreatedAt/UpdatedAt), not dropped at zero. Tag change propagated to the
PDK by the same regeneration.

* fix(plugins): reject user-scoped match before lookup when plugin has no user scope

A matcher plugin requires the library permission but not the users permission, so
a matcher-only plugin always has an empty user scope (allUsers=false, no allowed
users). MatchSongs still ran FindByUsername for any opts.Username before checking
authorization and returned distinguishable errors ('user X not found' vs 'not
allowed to act as user X'), letting such a plugin enumerate account names from the
error text.

Guard userAccess.resolve to reject with a single fixed error before the lookup when
the plugin has no user scope, mirroring how the SubsonicAPI service short-circuits
with 'no users configured'. The unscoped match path (no username) is unaffected, so
matcher-only plugins still match normally.

* fix(plugins): run unscoped matcher as admin, not the inherited request user

A matcher host call can arrive on a context that already carries a request user
(e.g. a plugin capability invoked while serving that user's request — extism
propagates the call context into host functions). With no opts.Username, MatchSongs
passed that context straight through, so the media-file repository applied the
caller's library filter and per-user annotation ranking to an explicitly unscoped
match.

Set the user context explicitly: a username scopes to that user (overriding any
inherited one), and an unscoped match runs under adminContext so only the plugin's
own library scope constrains results. Adds tests using a context-capturing
DataStore to assert the user the matcher resolves in both cases.

* chore(plugins): drop the generated Python matcher PDK

The Python plugin PDK is no longer supported (ndpgen generates only Go and Rust
clients), so remove the stale generated nd_host_matcher.py rather than leave a
client that drifts from the host interface.

* docs(plugins): deprecate SongRef.Artist/ArtistMBID in favor of Artists

Mark the scalar single-artist fields deprecated; Artists (the ArtistRef list) is
the preferred way to supply artist data and already takes precedence for matching.
Propagated to the PDK and capability schemas by make gen.

* refactor(plugins): flatten Track.Participants and add Role to ArtistRef

Change Track.Participants from map[role][]ArtistRef to a flat []ArtistRef, and give
ArtistRef a Role field (the participation category: artist/composer/performer/...)
alongside SubRole (a specialization within a role, e.g. the instrument for a
performer). In the flat list each entry now self-describes its role rather than
relying on a map key, matching how SongRef.Artists is already a flat list; the
converter tags each entry with its role and emits them in a stable role order.
Propagated to the PDK and capability schemas by make gen.

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-07-01 11:19:15 -04:00
Deluan Quintão
0e5b9e3263
feat(plugins): share plugin DTOs via a types package (#5655)
* refactor(plugins): remove Python PDK generation from ndpgen

* feat(plugins): parse Go type aliases distinctly in ndpgen

* feat(plugins): resolve shared-type aliases against a registry in ndpgen

* fix(plugins): resolve host-service shared aliases package-wide

Mirror the capability approach in ParseDirectoryWithShared: do a first
pass over all package files to build a package-wide alias map, then pass
it into parseServiceFile so that a shared-type alias declared in a sibling
file is visible when resolving types in the service interface file.

Add a focused test that writes the alias in one file and the hostservice
in another, confirming RED before the fix and GREEN after. Also
strengthens the existing Task 3 test with an ArtistRef.Target assertion.

* feat(plugins): add ndpgen -shared-types mode for the Go types package

* feat(plugins): generate the nd-pdk-types Rust crate from -shared-types

* feat(plugins): inject types import and emit deprecated aliases in Go output

* feat(plugins): emit deprecated Rust aliases to the shared types crate

* feat(plugins): inline shared-type shapes into XTP schemas

* feat(plugins): add nd-pdk-types crate and wire dependents

* feat(plugins): move shared capability types to plugins/types with deprecated aliases

* fix(plugins): point Rust deprecated-alias note at the replacement type

* fix(plugins): include shared aliases in KnownStructs so Rust fields keep their type

Capability.KnownStructs() and Service.KnownStructs() previously only
registered names from .Structs. After the shared-types migration, types
like ArtistRef/TrackInfo/SongRef live in .SharedAliases instead, so
ToRustTypeWithStructs could not find them and fell back to serde_json::Value
for every struct field referencing a shared type.

Add the shared-alias names to the knownStructs map in both methods.
Regenerate the Rust capability files; track/song/artist fields now render
as their named types (TrackInfo, SongRef, ArtistRef, etc.).

Add a regression test that verifies a struct field whose type is only in
SharedAliases renders as the named type and not serde_json::Value.

* docs(plugins): remove stale Python references from ndpgen and plugins READMEs

ndpgen no longer has a -python flag; remove it from the usage synopsis,
flags table, and defaults note in ndpgen/README.md. Delete the "Python
Client Library" section that described its output.

plugins/README.md referenced plugins/pdk/python/host/ (deleted) as the
source for Python host-service stubs. Remove that paragraph; Python plugins
still work via the XTP-schema / extism-py path (see examples/*-py).

* refactor(plugins): dedupe ndpgen helpers and tidy shared-type codegen

* docs(plugins): restore Python as a supported XTP schema target

The ndpgen-generated Python PDK was removed, but the XTP YAML schemas are
language-neutral and the XTP CLI still generates Python bindings from them
(as the extism-py examples demonstrate). Only the ndpgen Python output was
dropped, not Python support itself.

* test(plugins): use the shared types package in test plugins

The test fixtures referenced the now-deprecated capability aliases
(sonicsimilarity.SongRef, metadata.ArtistRef/SongRef). Point them at the
canonical types package so our own fixtures don't depend on symbols slated
for removal.

* refactor(plugins): use the shared types package in host adapters

Replace deprecated capabilities.TrackInfo, capabilities.ArtistRef, and
capabilities.SongRef aliases with the canonical types.TrackInfo,
types.ArtistRef, and types.SongRef from plugins/types.

* fix(plugins): reference shared types by canonical path in generated Rust

Previously the generator emitted `pub field: SongRef` (the local deprecated
alias) for struct fields whose type came from SharedAliases. Refactored
ToRustTypeWithStructs into a private toRustType that accepts a shared map,
and added ToRustTypeWithShared which resolves shared-alias names to their
canonical nd_pdk_types::X path before falling through to the knownStructs
check. Both rustCapabilityFuncMap and rustFuncMap now build the shared map
from SharedAliases and use it for fieldRustType, so the generated capability
files reference nd_pdk_types::SongRef / nd_pdk_types::TrackInfo directly.
The deprecated pub type aliases remain in place as the external back-compat
surface. Deprecation warning count from cargo build drops to 0.

* fix(examples): implement missing Scrobbler.playback_report in Rust examples

The Scrobbler trait gained a playback_report method but the two Rust example
plugins (webhook-rs and discord-rich-presence-rs) were not updated, causing
E0046 compile errors. Added the missing fn playback_report to both: webhook-rs
logs and returns Ok(()) mirroring its now_playing handler; discord-rich-presence-rs
is a no-op since Discord presence does not need playback reports. make all-rust
now exits 0.

* refactor(plugins): point Rust deprecation notes at the nd_pdk::types umbrella path

Plugin authors depend on the nd-pdk umbrella crate, which re-exports
nd_pdk_types as 'types', so the migration target they should type is
nd_pdk::types::X. The alias target stays nd_pdk_types::X (the real path
inside nd-pdk-capabilities).

* fix(plugins): error when a shared-type alias can't be resolved against the registry

* refactor(plugins): parse each Go source file once in ndpgen

* fix(plugins): correct ndpgen review nits (flag name, unused dep, docs)

* refactor(plugins): drop the now-unused path param from parseServiceFile

* refactor(plugins): use shared types directly, rename TrackInfo to Track

Capability interfaces now reference the shared `types` package by qualified
name (types.Track, types.SongRef, types.ArtistRef) instead of the package-local
deprecated aliases, and the shared TrackInfo type is renamed to Track to match
its role as the plugin-facing projection of a library media file.

The deprecated bare aliases (scrobbler.TrackInfo, metadata.ArtistRef,
sonicsimilarity.SongRef, etc.) are kept as re-exports so existing plugins keep
compiling, with a deprecation warning steering them to the canonical types.

To support this, ndpgen now resolves qualified types.X references: it collects
them during type discovery, maps each used canonical type back to its declared
deprecated alias for re-export, emits nd_pdk_types::X paths in Rust, and names
the XTP schema components by their canonical type. Regenerated the Go and Rust
PDK and the XTP schemas, and added generator tests covering the qualified-ref
path. Also adds clarifying doc comments to the shared types.

* refactor(plugins): extract shared types selector into a named const

Replace the "types." string literal that detects and strips the shared types
package selector with a single sharedTypesPrefix constant across the ndpgen
generator (parser, types, generator, xtp_schema), giving the package one source
of truth for the selector.

Also restore the single reused scratch map (cleared each iteration) in the
resolveSharedAliases BFS instead of allocating a fresh map per shared-struct
field, matching the prior implementation.

Pure cleanup from a /simplify pass: regeneration produces byte-for-byte
identical Go, Rust, and XTP output.

* refactor(plugins): keep TrackInfo in the capability package for now

Move the track type back out of the shared plugins/types package: it is again
defined inline as TrackInfo in plugins/capabilities/scrobbler.go and referenced
directly by the scrobbler and lyrics capabilities, reverting the rename to
types.Track. The host helper is renamed back to mediaFileToTrackInfo and now
returns capabilities.TrackInfo. SongRef and ArtistRef stay in the shared types
package; TrackInfo keeps using types.ArtistRef for its artist lists.

This type is expected to be reshaped in upcoming work, so leaving it in the
capability package avoids churning the shared types twice. Regenerated the Go
and Rust PDK and the XTP schemas accordingly.

* fix(plugins): emit the Go types import for direct shared-type refs

ndpgen's Capability/Service.ImportsSharedTypes only reported a shared-types
dependency when a deprecated re-export alias (type X = types.X) was declared. A
struct field referencing the canonical form directly (e.g. types.SongRef) with
no such alias produced an empty SharedAliases slice, so the Go templates skipped
the import while still emitting fields/signatures using types.SongRef — leaving
generated PDK code for new shared DTOs uncompilable unless an otherwise
unnecessary alias was added.

ImportsSharedTypes now also returns true when any struct field references the
types. package by qualified name, via a new structsReferenceSharedTypes helper
that reuses collectReferencedTypes (so []types.X and map[...]types.X are covered
too).

* fix(plugins): preserve base64 encoding for shared byte fields in Rust

The Rust shared-types crate template rendered a []byte field as a plain Vec<u8>
without the base64_bytes serde override used by the capability/client templates.
Go's encoding/json serializes []byte as a base64 string, so a Rust plugin using
nd_pdk::types would have serialized an array of numbers instead of the wire
format the Go/server side expects.

GenerateSharedTypesRust now registers the base64_bytes partial and passes a
HasByteFields flag (new anyFieldIsByteSlice helper); types.rs.tmpl emits the
base64_bytes module and a #[serde(with = "base64_bytes")] attribute on []byte
fields, mirroring the capability template.

* fix(plugins): include directly-referenced shared types in XTP schemas

buildSchemas registered shared types into the schema components only by iterating
cap.SharedAliases, which records deprecated re-export aliases. A capability that
referenced a shared DTO solely as types.Foo (no declared alias) therefore never
got Foo into the component set, so the self-contained XTP schema rendered the
field as a generic object (or emitted a dangling $ref), breaking the direct
shared-type use case enabled by -shared.

resolveSharedAliases now also returns the resolved shapes of every used shared
type (alias or not); these are carried on the new Capability.SharedTypes field
and registered by buildSchemas alongside SharedAliases. Validated end-to-end with
the xtp CLI: a direct types.Foo reference now produces a proper component plus a
$ref, so xtp generates a typed struct instead of an untyped serde_json::Map.

* fix(plugins): resolve renamed shared aliases to canonical schema refs

When a deprecated alias renames its canonical type (e.g. type TrackInfo =
types.Track) and a capability field is typed with the alias name (TrackInfo),
buildProperty emitted a $ref to #/components/schemas/TrackInfo. Components are
keyed by the canonical name (Track), so no TrackInfo component was emitted,
leaving a dangling reference that crashes the xtp code generator.

buildSchemas now builds an alias->canonical map; buildProperty (and the slice
item path) resolves $ref targets through it, and a used alias name marks its
canonical component used so it is emitted. Validated with the xtp CLI: the
renamed-alias schema previously crashed xtp and now generates cleanly.

* fix(plugins): detect shared types used directly in method signatures

ImportsSharedTypes only inspected struct fields, so a capability method using a
shared type directly in its signature (e.g. types.SongRef as input/output rather
than inside a local struct) was not detected. The generated Go templates still
rendered the provider/export signatures with types.SongRef, so the capability
package omitted the types import and failed to compile; the same gap applied to
service params/returns.

ImportsSharedTypes now also scans capability method input/output types and
service method params/returns, via a typeReferencesSharedTypes helper that reuses
collectReferencedTypes (covering pointer/slice/map wrappers).

* fix(plugins): add base64 dependency to the shared Rust types crate

When a shared DTO has a []byte field, ndpgen emits the base64_bytes serde helper
and use base64::... imports into nd-pdk-types/src/lib.rs, but the crate manifest
declared only serde. In that case make gen produced a crate that failed to
compile with 'unresolved module base64'.

Add base64 = "0.22" (matching nd-pdk-capabilities) so the generated shared types
crate compiles whenever a []byte field is present. Verified by generating a
shared crate with a []byte field and confirming cargo check fails before and
passes after.

* fix(plugins): translate shared method types in generated Rust

A capability method using a shared DTO directly as input/output (e.g.
types.SongRef) was passed through rustOutputType unchanged, so the Rust template
emitted invalid trait and extism_pdk::Json<$crate::pkg::types.SongRef> signatures
that do not compile.

Method input/output types now resolve through the shared registry: trait
signatures use rustTraitType (shared -> nd_pdk_types::X, locals stay bare) and the
export macros use rustMethodType (fully qualified: shared -> nd_pdk_types::X,
primitives -> Rust, locals -> $crate::<pkg>::X). Verified end-to-end by compiling
a generated capability that takes types.SongRef directly against the real
nd-pdk-types crate.

* fix(plugins): canonicalize XTP export refs for renamed shared aliases

buildSchemas canonicalized alias-to-canonical references for struct-field $ref
targets, but buildExport built export input/output $refs straight from
fieldBaseType. A capability whose export used a renamed deprecated alias
directly (e.g. type TrackInfo = types.Track with NowPlaying(TrackInfo)) emitted
$ref: #/components/schemas/TrackInfo, while the component is emitted under the
canonical name Track — a dangling export reference.

Lift the alias-to-canonical map into GenerateSchema (buildAliasToCanonical) and
apply it to export refs via canonicalRefName, the same resolution already used
for field properties.

* fix(plugins): route shared macro types through $crate for plugin builds

When a capability method used a shared type directly, the generated export macro
named the type as nd_pdk_types::SongRef. The macro expands in the downstream
plugin crate, which depends on the umbrella nd-pdk crate and not on nd-pdk-types
directly, so that path is unresolvable there and the plugin fails to build.

rustMethodType (macro-facing) now emits $crate::types::X, and the generated
nd-pdk-capabilities lib.rs re-exports nd_pdk_types as types so $crate resolves
it. Trait signatures keep nd_pdk_types::X since they live in nd-pdk-capabilities,
which has the direct dependency. Verified end-to-end: a plugin crate depending
only on the umbrella that uses a capability with a direct types.X method now
compiles via the macro.

* fix(plugins): add nd-pdk-types dependency to the Rust host crate

When a host service uses a shared type, ndpgen emits nd_pdk_types::X into the
generated nd-pdk-host client wrappers, but the host crate's manifest did not
depend on nd-pdk-types, so the crate failed to compile with 'unresolved module
nd_pdk_types'. Host client wrappers are plain functions resolved in the host
crate's own context (not macros expanded downstream), so a direct dependency is
the right fix.

Add nd-pdk-types = { path = "../nd-pdk-types" } to nd-pdk-host, mirroring
nd-pdk-capabilities. Found while auditing all Rust paths against the realistic
crate topology after the capability-side $crate fix; verified by generating a
host service with a shared-type return and confirming cargo check fails before
and passes after.

* fix(plugins): resolve shared aliases in Rust host signatures

The Rust host client rendered method params and returns through
RustTypeWithStructs, which only consults KnownStructs. A host service using a
shared alias in a signature (e.g. type Track = types.Track plus
MatchSongs(...) ([]Track, error)) therefore emitted a bare Vec<Track>, but the
client template emits no Track alias or import, so the generated nd-pdk-host
crate did not compile. Only struct fields went through the shared map.

rustType/rustParamType now use the shared map too (RustTypeWithShared /
RustParamTypeWithShared), so an aliased param/return resolves to its canonical
nd_pdk_types::X path, matching field handling. Verified by generating a host
service returning a shared alias and confirming cargo check fails before and
passes after.
2026-06-29 21:20:33 -04:00
Deluan Quintão
aa5aa731dc
refactor(lyrics): single ParseLyrics entry point + all-format plugin lyrics (#5632)
* refactor(lyrics): read sidecar files via library storage FS

Routes fromExternalFile reads through storage.For(mf.LibraryPath) instead
of os.Open on AbsolutePath, fixing sidecar reads for non-local backends.
UTF-16 LE/BE and BOM handling preserved via ioutils.UTF8Reader.

* refactor(lyrics): address review feedback on sidecar FS read

- Move blank local-storage import from sources.go into lyrics_suite_test.go
  (the test suite already imports the local package for RegisterExtractor,
  so local's init() runs; production binaries get the scheme via normal wiring)
- Fix misleading comment: model.ParseLyrics → model.ParseLyricsFile
- Replace what-comment with why-comment in BeforeSuite explaining the
  log.Fatal guard that requires the no-op extractor registration

* test(lyrics): add subsonic e2e baseline for getLyrics endpoints

Establishes a behavioral baseline for getLyricsBySongId (v2 structured)
and getLyrics (legacy) before the lyrics parser refactor. Covers embedded
formats (LRC synced, plain text, TTML) and sidecar formats (LRC, SRT,
YAML), all isolated under a Lyrics/ fixture folder so the new fixtures
do not perturb existing test behavior beyond fixture counts.

Sidecar files are injected as raw &fstest.MapFile{Data: []byte(...)}
entries; the scanner skips non-audio extensions (.lrc, .srt, .yaml) so
they are invisible to scanning but reachable via the fake FS at request
time through fromExternalFile/storage.For.

Update album/artist/song counts in the album-list, multi-library, and
search3 empty-query tests to reflect the six new tracks (1 new artist,
1 new album, 6 new songs).

* test(lyrics): strengthen e2e lyrics baseline (lang assertions, rename helper)

Rename the local helper `main` to `firstLyric` to avoid collision with the
reserved-feeling built-in name. Add `Lang` assertions to both embedded and
sidecar DescribeTable entries, locking the current observed values: "xxx"
(ISO 639-2 "no language specified") for all embedded and LRC/SRT sidecars,
and "eng" for the YAML sidecar (which explicitly sets `language: eng`).

* feat(lyrics): detect Lyricsfile YAML in content-sniffing

* feat(plugins): content-sniff plugin lyrics for all formats

Replace model.ToLyrics (LRC/plain only) with model.ParseEmbedded so plugin
responses are content-sniffed for TTML, SRT, YAML, LRC, and plain text.
ParseEmbedded returns a LyricList, so the loop now flattens multiple tracks
per response entry.

The test-lyrics WASM plugin gains a "ttml" format mode (configured via
pdk.GetConfig) that returns a minimal TTML document; rebuilt with the
standard Go wasip1 toolchain (GOOS=wasip1 GOARCH=wasm). A new Ginkgo test
asserts Synced==true and the exact cue value, which the old plain-text path
could not produce.

GetLyrics doc comment updated to reflect content-sniffing; a later task will
retarget it to ParseLyrics once that function is introduced.

* test(plugins): validate plugin lyrics auto-detect across all formats

The test-lyrics WASM plugin now supports per-format modes via the
"format" config key: ttml, srt, yaml, lrc, and plain, in addition to
the existing default plain-text response. The plugin is rebuilt with the
standard Go wasip1 compiler.

lyrics_adapter_test.go gains a DescribeTable covering all five formats,
asserting both Synced (the discriminator that proves correct format
detection) and the exact line value. This validates the full
auto-detect chain (TTML → SRT → YAML/Lyricsfile → LRC → plain) end-to-end
through the real plugin → adapter → parser flow.

* refactor(lyrics): consolidate parsers into model.ParseLyrics

* refactor(lyrics): retarget legacy callers to model.ParseLyrics

Pin suffix to ".lrc" to preserve byte-identical output for stored
plain/LRC text that was previously handled by the now-removed ToLyrics.

* test(lyrics): fix lyrics tests after parser consolidation

- Rewrite the YAML-fallback test to assert the correct design: a
  non-Lyricsfile .yaml sidecar returns as plain text and shadows
  lower-priority sources (rather than falling through to .lrc).
- Add LibraryPath + relative Path split to the three subsonic tests
  that read sidecar files via storage.For(), so they resolve against
  the correct fixtures directory.
- Register a no-op extractor in api_suite_test.go BeforeSuite so
  newLocalStorage does not fatal when storage.For is called during
  sidecar-lyrics tests.

* test(lyrics): add per-format ParseLyrics benchmarks

Baseline measurements (count=2 runs) on M2:

BenchmarkParseLyrics_LRC-8           	    5725	    178796 ns/op	  49.78 MB/s	  427877 B/op	     523 allocs/op
BenchmarkParseLyrics_Plain-8         	    5425	    230854 ns/op	  32.44 MB/s	  102508 B/op	      16 allocs/op
BenchmarkParseLyrics_EnhancedLRC-8   	    1942	    605893 ns/op	  17.66 MB/s	  860678 B/op	    4256 allocs/op
BenchmarkParseLyrics_SRT-8           	    3249	    373991 ns/op	  25.91 MB/s	 1113575 B/op	    4407 allocs/op
BenchmarkParseLyrics_TTML-8          	    1483	    813027 ns/op	  13.86 MB/s	 2198052 B/op	    8665 allocs/op
BenchmarkParseLyrics_YAML-8          	    1700	    678250 ns/op	  13.01 MB/s	 1235096 B/op	    8288 allocs/op
BenchmarkParseLyrics_SniffTTML-8     	    1525	    776482 ns/op	  14.51 MB/s	 2225448 B/op	    8681 allocs/op
BenchmarkParseLyrics_SniffSRT-8      	    2528	    451210 ns/op	  21.48 MB/s	 1157000 B/op	    4422 allocs/op
BenchmarkParseLyrics_SniffYAML-8     	    1333	    827152 ns/op	  10.67 MB/s	 1337195 B/op	    8718 allocs/op
BenchmarkParseLyrics_SniffLRC-8      	    2820	    413038 ns/op	  21.55 MB/s	  588934 B/op	    1812 allocs/op
BenchmarkParseLyrics_SniffPlain-8    	    2968	    409091 ns/op	  18.31 MB/s	  254470 B/op	    1491 allocs/op

Content-sniff path overhead: 1.5–15% depending on format.

* test(lyrics): use real public-domain fixtures for parser benchmarks

Replace synthetic benchmark payloads with 'Auld Lang Syne' (Robert Burns,
1788, public domain) rendered into every supported format (LRC, plain,
enhanced LRC, SRT, TTML, Lyricsfile YAML) so the numbers reflect realistic
content. Same song across formats makes per-format cost comparable.

Baseline (Apple M-series, -benchmem, real fixtures):
  LRC          ~28 us/op   42 KB   147 allocs
  Plain        ~23 us/op   18 KB    22 allocs
  EnhancedLRC  ~37 us/op   51 KB   374 allocs
  SRT          ~52 us/op  139 KB   581 allocs
  TTML        ~119 us/op  276 KB  1227 allocs
  YAML        ~142 us/op  193 KB  1732 allocs
  Sniff(LRC)   ~47 us/op   57 KB   237 allocs
  Sniff(TTML) ~122 us/op  282 KB  1250 allocs
  Sniff(YAML) ~186 us/op  218 KB  1847 allocs

Fixtures in tests/fixtures/lyrics/.

* fix(lyrics): preserve [] (not null) for empty lyrics in backfill migration

ParseLyrics returns nil for zero-line input (whitespace-only stored
lyrics). json.Marshal(nil LyricList) produces null, violating the DB
invariant that media_file.lyrics uses [] for empty lyrics, never null.
Initialize to model.LyricList{} when ParseLyrics returns nil so the
marshalled result is always [].

* refactor(lyrics): unify parser dispatch and centralize empty-list invariant

Apply thermo-nuclear review findings (behavior-preserving):

- Replace the suffix switch + three single-use closure adapters
  (parseTTMLKnown/parseSRTKnown + inline YAML closure) with a
  bySuffix map of a single lyricParser(lang, contents) signature.
  Normalize parseTTMLWithDefaultLang/parseSRTWithLanguage to that
  (lang, contents) order so no adapter glue is needed.
- Collapse the parallel sniffLyrics engine into one parseFirstMatch
  primitive shared by both the suffix and content-sniff paths
  (sniffOrder candidate list). TTML stays gated via parseTTMLIfDocument
  in sniff mode to avoid running the XML decoder on plain/LRC text.
- Add LyricList.MarshalJSON so empty/nil always serializes to [] (the
  lyrics column invariant), in one canonical place. Delete the
  migration's nil-guard, which the marshaler now subsumes.

Behavior verified unchanged: full suite + race + e2e green.

* refactor(lyrics): single registry drives both suffix dispatch and sniff order

Collapse the bySuffix map and sniffOrder slice into one ordered registry:
slice order is the content-sniff probe order, each row's suffixes drive
sidecar dispatch, and per-row bySuffix/byContent parsers preserve the
gated-TTML-when-sniffing distinction. One source of truth, no duplicated
parser references.

* refactor(lyrics): self-skipping parsers collapse the format table to one column

Move the TTML <tt>-document gate into parseTTMLWithDefaultLang itself (after
the encoding fixup, so UTF-16-declared docs are still recognized): non-TTML
content returns (nil, nil) to skip; a malformed <tt> document still errors.
SRT and Lyricsfile YAML already self-skip. With every structured parser
self-skipping, the format table drops to one {suffixes, parse} column named
lyricFormats — no bySuffix/byContent split, no separate sniff-only TTML gate.
Both the suffix and content-sniff paths share the same parser per format.

* refactor(lyrics): strip BOM once at ParseLyrics entry for all paths

Previously only the content-sniff path stripped the BOM; the suffix path
relied on its callers (fromExternalFile via UTF8Reader) having already
stripped it. That implicit contract was fragile — a caller passing raw
BOM-prefixed bytes with a suffix would reach the parsers with the BOM intact
(SanitizeText does not strip it). Strip once at entry so every path and
parser sees clean bytes regardless of caller. No-op for already-stripped
input.

* refactor(lyrics): trim verbose comments to essential why

* refactor(lyrics): move LRC parser to its own lyrics_lrc.go

Extract parseLRC, the enhanced-LRC helpers (parseEnhancedLine, adjustGroup,
stripEnhancedMarkers, shiftELRCCues), parseTime, and the LRC regexes from
lyrics.go into lyrics_lrc.go, with the parseLRC tests in lyrics_lrc_test.go.
This makes the layout symmetric — one file per format (lrc/srt/ttml/yaml) —
and leaves lyrics.go holding only shared types and cue normalization. All
moved symbols were already LRC-private; no behavior change.

* refactor(lyrics): collapse ParseLyrics suffix/sniff branches into one loop

Both modes differ only in which formats to try, so select candidates in a
single loop (all formats when sniffing, the suffix's own otherwise) and run
them through parseFirstMatch once. Drops the projected-slice make+index and
the ContainsFunc closure; unmatched suffixes yield no candidates and fall to
the plain-text floor, as before.

* refactor(lyrics): apply simplify-review cleanups

- stripBOM: bytes.TrimPrefix instead of []byte<->string round-trip (no alloc)
- ParseLyrics: pre-size the candidates slice
- move isTTMLDocument to lyrics_ttml.go beside its only caller (the dispatch
  layer should hold no per-format knowledge)

* refactor(lyrics): simplify test descriptions for structured lyrics

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(lyrics): fold parseLyricsfile into lyricParser signature and rename file

- parseLyricsfile now matches the lyricParser signature directly (reads via
  bytes.NewReader), removing the parseLyricsfileBytes adapter and the
  string(contents) copy; the lyricFormats table references it directly.
- StructuredLyrics drops the vestigial LyricList{} init (json.Unmarshal
  overwrites; MarshalJSON owns the empty->[] invariant).
- Rename lyricsfile.go -> lyrics_lyricsfile.go (and its test) to match the
  lyrics_<format>.go convention used by lrc/srt/ttml.

* refactor(lyrics): move test-only parseTTML/parseSRT wrappers to test files

These zero-arg wrappers (defaulting lang to "xxx") had no production callers
after the consolidation — only the format tests used them. Move each beside
its tests so the production files carry no test-only code.

* build: exclude generated *_gen.go files from linting

The plugin host *_gen.go files (ndpgen output) were tripping the whitespace
linter despite carrying a generated marker. Exclude them by path so make lint
and the pre-push hook pass on untouched generated code.

* perf(lyrics): drop []byte/string round-trips in parsers

Apply code-review feedback to remove avoidable allocations in the lyrics
parsers. isTTMLDocument now takes []byte directly, so parseTTMLWithDefaultLang
no longer copies its buffer into a string before the TTML probe. parseSRTBlock
splits its block with strings.Split instead of converting to []byte and back
per line. ParseLyrics hoists strings.ToLower(suffix) out of the format loop.

No behavior change; the dropped len(scanner)==0 SRT guard was dead (strings.Split
never returns an empty slice, and the existing len(lines)==0 check still covers
empty input).

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(lyrics): colocate and unexport cue-normalization helpers

Move the cue-normalization machinery out of lyrics.go into a dedicated
lyrics_normalize.go (with lyrics_normalize_test.go), leaving lyrics.go to hold
just the shared lyric types and their methods. lyrics.go was mixing the domain
type/contract definitions with format-agnostic post-processing.

Unexport normalizeLyrics, normalizeCueLines, and normalizeLineTiming: they have
no callers outside the model package, so they should not be part of its public
API. NormalizeCueEnds stays exported because the Subsonic enhanced-lyrics
serializer (server/subsonic/lyrics.go) resolves cue ends per agent group while
building the response; that is the only legitimate cross-package caller.

Also includes a small no-op robustness tweak in parseLRC: len(times) == 0
instead of times == nil (equivalent here, more idiomatic).

No behavior change.

* test(lyrics): add direct coverage for NormalizeCueEnds

NormalizeCueEnds is exported and carries the most intricate logic in the
normalization cluster (fill-from-next, fill-from-fallback, both clamps, and the
all-or-none clear), but was only exercised transitively. Add a focused spec
covering each branch plus the empty-input and no-mutation guarantees, bringing
the function to 100% coverage.

* test(lyrics): cover legacy getLyrics across formats and sources

Expand the legacy getLyrics e2e coverage from a single embedded-plain case to a
table over all six fixtures: embedded LRC/plain/TTML and sidecar LRC/SRT/YAML.

Each case asserts the v1 plain-text fallback contract — the structured lyric is
flattened to LRC-style plain text with no timing markup leaking through (no LRC
brackets, SRT arrows, or XML tags), regardless of the source format or whether
it is embedded or a sidecar file. This pins the behavior that synced TTML/SRT/
YAML formats degrade gracefully to plain text on the legacy endpoint.

* test(lyrics): cover songLyrics v1 vs v2 with word-level fixtures

Correct and expand the e2e lyrics coverage to match the OpenSubsonic songLyrics
extension contract:

- v1 (getLyricsBySongId, no enhanced): line-level lyrics with no cueLine, kind,
  or agents — even for word-level formats (ELRC, Lyricsfile YAML).
- v2 (getLyricsBySongId?enhanced=true): word-level cueLine surfaces for ELRC and
  YAML sources; kind="main" is set; a line-level source (SRT) still yields no
  cueLine even when enhanced.
- legacy getLyrics (artist/title): the original Subsonic endpoint, flattening any
  format to plain text. A prior commit mislabeled this as the "v1 contract";
  getLyrics predates OpenSubsonic and is unrelated to the extension versions.

Drive these with the public-domain tests/fixtures/lyrics files (the same set the
parser benchmarks use) so the e2e content stays in sync and actually carries the
word-level timing needed to distinguish v1 from v2. The embedded "synced LRC"
fixture is upgraded to ELRC (word-level); track counts are unchanged, so the
rest of the suite is unaffected.

* test(lyrics): parameterize v2 enhanced coverage across all formats

Convert the v2 (enhanced) e2e block from three ad-hoc cases into a DescribeTable
covering all six formats, matching the v1 and legacy tables. Each entry declares
whether the source carries word-level timing: ELRC, TTML, and Lyricsfile YAML
surface a cueLine; LRC, SRT, and plain text do not. All six get kind="main".

Add word-level <span> timing to the first line of the auld-lang-syne.ttml
fixture so TTML exercises the word-level cueLine path (the parser already
supports <span begin/end>, but the fixture was line-level only). The first line
now yields the same five word cues as the ELRC and YAML fixtures, keeping the
table assertions uniform across formats.

* fix(lyrics): honor caller language when Lyricsfile YAML omits it

parseLyricsfile discarded the caller's language argument, so a Lyricsfile YAML
parsed from an embedded tag or plugin response with no metadata.language was
labeled "xxx" even when ParseLyrics was given a language. The SRT and TTML
parsers already use the caller language as their default; fall back to it here
too, preferring the document's own metadata.language when present.

Also reword a misleading TTML comment: isTTMLDocument still runs an XML decode
(it stops at the first element), so the skip avoids the full TTML parse, not the
XML decoder entirely.

* refactor(lyrics): consolidate lyrics parsing functions names

Signed-off-by: Deluan <deluan@navidrome.org>

* test(lyrics): drop test-only parse wrappers after parser rename

Commit 48c0173e8 renamed the production parsers to parseTTML/parseSRT, which
collided with the same-named test-only wrappers and broke the model test build
(parseTTML/parseSRT redeclared). Remove the wrappers and call the production
parsers directly with the placeholder language at each test site.

* test(lyrics): complete the truncated enhanced-LRC fixture

The auld-lang-syne.elrc fixture stopped after the first two stanzas (8 lyric
lines) while every other format fixture carries the full 24-line song. Extend it
to all 24 lines with per-word timing so it is a faithful enhanced-LRC sample and
the EnhancedLRC parser benchmark runs on a workload comparable to the others.
The first line's word timings are unchanged, so the e2e cueLine assertions still
hold.

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-06-19 18:25:35 -04:00
Deluan Quintão
ae0e0c89d9
feat(plugins): add PlaybackReport to scrobbler capability (#5452)
Some checks are pending
Pipeline: Test, Lint, Build / Get version info (push) Waiting to run
Pipeline: Test, Lint, Build / Lint Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test Go code (Windows) (push) Waiting to run
Pipeline: Test, Lint, Build / Test JS code (push) Waiting to run
Pipeline: Test, Lint, Build / Lint i18n files (push) Waiting to run
Pipeline: Test, Lint, Build / Check Docker configuration (push) Waiting to run
Pipeline: Test, Lint, Build / Build (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-1 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Package/Release (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-2 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-3 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-4 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-5 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Upload Linux PKG (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-6 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-7 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-8 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-9 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-10 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to GHCR (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to Docker Hub (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Cleanup digest artifacts (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build Windows installers (push) Blocked by required conditions
* feat(plugins): add PlaybackReport to Scrobbler interface and all implementations

* feat(plugins): add PlaybackReport worker and dispatch in PlayTracker

* feat(plugins): add PlaybackReportRequest to plugin scrobbler capability

* chore(plugins): regenerate PDK files with PlaybackReport

* feat(plugins): add PlaybackReport to test scrobbler plugin

* feat(plugins): add PlaybackReport to plugin scrobbler adapter

* refactor(plugins): fix double DB fetch in StateStopped and batch getActiveScrobblers

- Hoist mf from scrobble branch so PlaybackReport reuses it instead of
  fetching again from DB
- Call getActiveScrobblers once per drain batch instead of per-entry

* chore(plugins): include generated scrobbler schema with PlaybackReport

* fix(plugins): skip PlaybackReport for plugins that don't export it

Plugins detected as scrobblers only need to export one scrobbler
function. Older plugins that don't export nd_scrobbler_playback_report
would cause noisy error logs on every reportPlayback call. Now
errFunctionNotFound and errNotImplemented are treated as no-ops.

* refactor: rename NowPlayingInfo to PlaybackReport

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename stopNowPlayingWorker to stopBackgroundWorkers

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: move NowPlaying and PlaybackReport logic to separate worker files

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scrobbler): rename NowPlayingInfo to PlaybackSession and add expired state

Rename NowPlayingInfo struct to PlaybackSession to better reflect its role
as a complete playback session representation. Add UserId field to make
sessions self-contained, removing redundant userId parameters from
PlaybackReport interface method and internal dispatch functions. Introduce
StateExpired internal state that fires when a session cache entry expires
without an explicit stop, ensuring plugins always receive a terminal event
regardless of client behavior.

* fix(scrobbler): update playback state description to include 'expired'

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scrobbler): resolve data race in OnExpiration callback

Capture conf.Server.EnableNowPlaying at construction time instead of
reading it from the background ttlcache eviction goroutine. The previous
code raced with test config cleanup that writes to the same field
concurrently.

* fix(scrobbler): return error when media file lookup fails in StateStopped

Simplify the MediaFile population logic in the stopped case to return an
error if the track cannot be found. A stop report with an empty MediaFile
is useless to plugins, and returning the error allows clients to retry
or alert the user when auto-scrobble is enabled.

* refactor(scrobbler): use session data directly in PlaybackReport adapter

Use info.Username from PlaybackSession instead of extracting it from
context in the plugin adapter, since the session is now self-contained.
Add debug/trace logging for session expiration and enqueue the expired
report with a user-enriched context so downstream handlers can identify
the user.

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-05-02 16:14:53 -04:00
Deluan Quintão
259c1a9484
feat(subsonic): add sonicSimilarity extension as plugin capability (#5419)
* feat(plugins): add sonicSimilarity capability types

Defines the SonicSimilarity plugin capability interface with
GetSonicSimilarTracks and FindSonicPath methods, along with
their request/response types.

* feat(sonic): add core sonic similarity service

Implements the Sonic service with HasProvider, GetSonicSimilarTracks,
and FindSonicPath, delegating to the PluginLoader and using the
Matcher for index-preserving library resolution.

* test(sonic): add sonic service unit tests

Covers HasProvider, GetSonicSimilarTracks, and FindSonicPath with
mock plugin loader and provider, verifying error propagation and
successful match resolution via the library matcher.

* feat(matcher): add MatchSongsToLibraryMap for index-preserving matching

Adds a new method alongside MatchSongsToLibrary that returns a
map[int]MediaFile keyed by input song index rather than a flat slice,
enabling callers to correlate similarity scores back to the original
position in the results.

* fix(sonic): check provider availability before MediaFile lookup

Avoids unnecessary DB call when no plugin is available, and ensures
the correct error path is tested.

* feat(plugins): add sonic similarity adapter

Adds SonicSimilarityAdapter implementing sonic.Provider, bridging the
plugin system to the core sonic service via Extism plugin functions.
Reuses existing songRefsToAgentSongs helper for SongRef conversion.

* feat(plugins): add LoadSonicSimilarity to plugin manager

Adds Manager.LoadSonicSimilarity method following the pattern of
LoadLyricsProvider, enabling the core sonic service to load a
SonicSimilarityAdapter from a named plugin.

* feat(subsonic): add sonicMatch response type

Add SonicMatch struct with Entry and Similarity fields, and a SonicMatches slice to the Subsonic response struct. These types support the OpenSubsonic sonicSimilarity extension for returning similarity-scored track results.

* feat(subsonic): add getSonicSimilarTracks and findSonicPath handlers

Add two new Subsonic API handlers for the sonicSimilarity OpenSubsonic extension: GetSonicSimilarTracks returns similarity-scored tracks similar to a given song, and FindSonicPath returns a path of tracks connecting two songs. Both handlers delegate to the sonic core service and map results to SonicMatch response types.

* feat(subsonic): advertise sonicSimilarity extension when plugin available

Update GetOpenSubsonicExtensions to conditionally include the sonicSimilarity extension only when a sonic similarity plugin provider is available. The nil guard ensures backward compatibility with tests that pass nil for the sonic field. Also update the existing test to pass the new nil parameter.

* feat(subsonic): wire sonic similarity service into router

Add the sonic.Sonic service to the Router struct and New() constructor, register the getSonicSimilarTracks and findSonicPath routes, and wire sonic.New and its PluginLoader binding into the Wire dependency injection graph. Update all existing test call sites to pass the new nil parameter. Regenerate wire_gen.go.

* fix(e2e): add sonic parameter to subsonic.New call in e2e tests

* test(subsonic): add sonicSimilarity extension advertisement tests

Restructures the GetOpenSubsonicExtensions test into two contexts: one verifying the baseline 5 extensions are returned when no sonic similarity plugin is configured, and one verifying that the sonicSimilarity extension is advertised (making 6 total) when a plugin loader reports an available provider. Adds a mockSonicPluginLoader to satisfy the sonic.PluginLoader interface without requiring a real plugin.

* feat(subsonic): add nil guard and e2e tests for sonic similarity endpoints

Handlers return ErrorDataNotFound when no sonic service is configured,
preventing nil panics. E2e tests verify both endpoints return proper
error responses when no plugin is available.

* fix(subsonic): return HTTP 404 when no sonic similarity plugin available

Endpoints are always registered but return 404 when no provider is
available, rather than a subsonic error code 70.

* refactor: clean up sonic similarity code after review

Extract shared helpers to reduce duplication across the sonic similarity
implementation: loadAllMatches in matcher consolidates the 4-phase
matching pipeline, songRefToAgentSong eliminates per-iteration slice
allocation in the adapter, sonicMatchResponse deduplicates response
building in handlers, and a package-level constant replaces raw
capability name strings in core/sonic.

* fix empty response shapes

Signed-off-by: Deluan <deluan@navidrome.org>

* test(plugins): add testdata plugin and e2e tests for sonic similarity

Add a test-sonic-similarity WASM plugin that implements both
GetSonicSimilarTracks and FindSonicPath via the generated sonicsimilarity
PDK. The plugin returns deterministic test data with decreasing similarity
scores and supports error injection via config. Adapter tests verify the
full round-trip through the WASM plugin including error handling. Also
includes regenerated PDK code from make gen.

* docs: update README to include new capabilities and usage examples for plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* test(e2e): enhance sonic similarity tests with additional scenarios and mock provider

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: address PR review feedback for sonic similarity

Fix incorrect field names in README documentation ({from, to} →
{startSong, endSong}) and remove unnecessary XML serialization test
from e2e suite since OpenSubsonic endpoints only use JSON.

* refactor: rename Matcher methods for conciseness

Rename MatchSongsToLibrary to MatchSongs and MatchSongsToLibraryMap to
MatchSongsIndexed. The Matcher receiver already establishes the "to
library" context, making that suffix redundant, and "Indexed" better
describes the intent (preserving input ordering) than "Map" which
describes the data structure.

* refactor: standardize variable naming for media files in sonic path methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: simplify plugin loading by introducing adapter constructors

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-04-27 17:50:09 -04:00
Deluan Quintão
f03ca44a8e
feat(plugins): add lyrics provider plugin capability (#5126)
Some checks failed
Pipeline: Test, Lint, Build / Get version info (push) Has been cancelled
Pipeline: Test, Lint, Build / Lint Go code (push) Has been cancelled
Pipeline: Test, Lint, Build / Test Go code (push) Has been cancelled
Pipeline: Test, Lint, Build / Test JS code (push) Has been cancelled
Pipeline: Test, Lint, Build / Lint i18n files (push) Has been cancelled
Pipeline: Test, Lint, Build / Check Docker configuration (push) Has been cancelled
Pipeline: Test, Lint, Build / Build (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-1 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-2 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-3 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-4 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-5 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-6 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-7 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-8 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-9 (push) Has been cancelled
Pipeline: Test, Lint, Build / Build-10 (push) Has been cancelled
Pipeline: Test, Lint, Build / Push to GHCR (push) Has been cancelled
Pipeline: Test, Lint, Build / Push to Docker Hub (push) Has been cancelled
Pipeline: Test, Lint, Build / Cleanup digest artifacts (push) Has been cancelled
Pipeline: Test, Lint, Build / Build Windows installers (push) Has been cancelled
Pipeline: Test, Lint, Build / Package/Release (push) Has been cancelled
Pipeline: Test, Lint, Build / Upload Linux PKG (push) Has been cancelled
* feat(plugins): add lyrics provider plugin capability

Refactor the lyrics system from a static function to an interface-based
service that supports WASM plugin providers. Plugins listed in the
LyricsPriority config (alongside "embedded" and file extensions) are
now resolved through the plugin system.

Includes capability definition, Go/Rust PDK, adapter, Wire integration,
and tests for plugin fallback behavior.

* test(plugins): add lyrics capability integration test with test plugin

* fix(plugins): default lyrics language to 'xxx' when plugin omits it

Per the OpenSubsonic spec, the server must return 'und' or 'xxx' when
the lyrics language is unknown. The lyrics plugin adapter was passing
an empty string through when a plugin didn't provide a language value.
This defaults the language to 'xxx', consistent with all other callers
of model.ToLyrics() in the codebase.

* refactor(plugins): rename lyrics import to improve clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(lyrics): update TrackInfo description for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(lyrics): enhance lyrics plugin handling and case sensitivity

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update payload type to string with byte format for task data

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-03-03 15:48:39 -05:00
Deluan Quintão
668869b6c7
feat(plugins): add TaskQueue host service for persistent background task queues (#5116)
* feat(plugins): define TaskQueue host service interface

Add the TaskQueueService interface with CreateQueue, Enqueue,
GetTaskStatus, and CancelTask methods plus QueueConfig struct.

* feat(plugins): define TaskWorker capability for task execution callbacks

* feat(plugins): add taskqueue permission to manifest schema

Add TaskQueuePermission with maxConcurrency option.

* feat(plugins): implement TaskQueue service with SQLite persistence and workers

Per-plugin SQLite database with queues and tasks tables. Worker goroutines
dequeue tasks and invoke nd_task_execute callback. Exponential backoff
retries, rate limiting via delayMs, automatic cleanup of terminal tasks.

* feat(plugins): require TaskWorker capability for taskqueue permission

* feat(plugins): register TaskQueue host service in manager

* feat(plugins): add test-taskqueue plugin for integration testing

* feat(plugins): add integration tests for TaskQueue host service

* docs: document TaskQueue module for persistent task queues

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): harden TaskQueue host service with validation and safety improvements

Add input validation (queue name length, payload size limits), extract
status string constants to eliminate raw SQL literals, make CreateQueue
idempotent via upsert for crash recovery, fix RetentionMs default check
for negative values, cap exponential backoff at 1 hour to prevent
overflow, and replace manual mutex-based delay enforcement with
rate.Limiter from golang.org/x/time/rate for correct concurrent worker
serialization.

* refactor(plugins): remove capability check for TaskWorker in TaskQueue host service

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): use context-aware database execution in TaskQueue host service

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): streamline task queue configuration and error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): increase maxConcurrency for task queue and handle budget exhaustion

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): simplify goroutine management in task queue service

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): update TaskWorker interface to return status messages and refactor task queue service

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add ClearQueue function to remove pending tasks from a specified queue

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): use migrateDB for task queue schema and fix constant name collision

Replaced the raw db.Exec call in createTaskQueueSchema with migrateDB,
matching the pattern used by createKVStoreSchema. This enables version-tracked
schema migrations via SQLite's PRAGMA user_version, allowing future schema
changes to be appended incrementally. Also renamed cleanupInterval to
taskCleanupInterval to resolve a redeclaration conflict with host_kvstore.go.

* regenerate PDKs

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-03-03 13:48:49 -05:00
Deluan
6fd044fb09 feat(plugins): change websockets Data field type to []byte for binary support
Signed-off-by: Deluan <deluan@navidrome.org>
2026-03-02 16:38:00 -05:00
Deluan Quintão
30df004d4d
test(plugins): speed up integration tests (~45% improvement) (#5137)
* test(plugins): speed up integration tests with shared wazero cache

Reduce plugin test suite runtime from ~22s to ~12s by:

- Creating a shared wazero compilation cache directory in TestPlugins()
  and setting conf.Server.CacheFolder globally so all test Manager
  instances reuse compiled WASM binaries from disk cache
- Moving 6 createTestManager* calls from inside It blocks to BeforeAll
  blocks in scrobbler_adapter_test.go and manager_call_test.go
- Replacing time.Sleep(2s) in KVStore TTL test with Eventually polling
- Reducing WebSocket callback sleeps from 100ms to 10ms

Signed-off-by: Deluan <deluan@navidrome.org>

* test(plugins): enhance websocket tests by storing server messages for verification

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-03-02 16:18:30 -05:00
Deluan Quintão
2471bb9cf6
feat(plugins): add TTL support, batch operations, and hardening to kvstore (#5127)
Some checks are pending
Pipeline: Test, Lint, Build / Test JS code (push) Waiting to run
Pipeline: Test, Lint, Build / Get version info (push) Waiting to run
Pipeline: Test, Lint, Build / Lint Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Lint i18n files (push) Waiting to run
Pipeline: Test, Lint, Build / Check Docker configuration (push) Waiting to run
Pipeline: Test, Lint, Build / Build (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-1 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-2 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-3 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-4 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-5 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Upload Linux PKG (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to GHCR (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Cleanup digest artifacts (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build Windows installers (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Package/Release (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-6 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-7 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-8 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-9 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-10 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to Docker Hub (push) Blocked by required conditions
* feat(plugins): add expires_at column to kvstore schema

* feat(plugins): filter expired keys in kvstore Get, Has, List

* feat(plugins): add periodic cleanup of expired kvstore keys

* feat(plugins): add SetWithTTL, DeleteByPrefix, and GetMany to kvstore

Add three new methods to the KVStore host service:

- SetWithTTL: store key-value pairs with automatic expiration
- DeleteByPrefix: remove all keys matching a prefix in one operation
- GetMany: retrieve multiple values in a single call

All methods include comprehensive unit tests covering edge cases,
expiration behavior, size tracking, and LIKE-special characters.

* feat(plugins): regenerate code and update test plugin for new kvstore methods

Regenerate host function wrappers and PDK bindings for Go, Python,
and Rust. Update the test-kvstore plugin to exercise SetWithTTL,
DeleteByPrefix, and GetMany.

* feat(plugins): add integration tests for new kvstore methods

Add WASM integration tests for SetWithTTL, DeleteByPrefix, and GetMany
operations through the plugin boundary, verifying end-to-end behavior
including TTL expiration, prefix deletion, and batch retrieval.

* fix(plugins): address lint issues in kvstore implementation

Handle tx.Rollback error return and suppress gosec false positive
for parameterized SQL query construction in GetMany.

* fix(plugins): Set clears expires_at when overwriting a TTL'd key

Previously, calling Set() on a key that was stored with SetWithTTL()
would leave the expires_at value intact, causing the key to silently
expire even though Set implies permanent storage.

Also excludes expired keys from currentSize calculation at startup.

* refactor(plugins): simplify kvstore by removing in-memory size cache

Replaced the in-memory currentSize cache (atomic.Int64), periodic cleanup
timer, and mutex with direct database queries for storage accounting.
This eliminates race conditions and cache drift issues at negligible
performance cost for plugin-sized datasets. Also unified Set and
SetWithTTL into a shared setValue method, simplified DeleteByPrefix to
use RowsAffected instead of a transaction, and added an index on
expires_at for efficient expiration filtering.

* feat(plugins): add generic SQLite migration helper and refactor kvstore schema

Add a reusable migrateDB helper that tracks schema versions via SQLite's
PRAGMA user_version and applies pending migrations transactionally. Replace
the ad-hoc createKVStoreSchema function in kvstore with a declarative
migrations slice, making it easy to add future schema changes. Remove the
now-redundant schema migration test since migrateDB has its own test suite
and every kvstore test exercises the migrations implicitly.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): harden kvstore with explicit NULL handling, prefix validation, and cleanup timeout

- Use sql.NullString for expires_at to explicitly send NULL instead of
  relying on datetime('now', '') returning NULL by accident
- Reject empty prefix in DeleteByPrefix to prevent accidental data wipe
- Add 5s timeout context to cleanupExpired on Close
- Replace time.Sleep in unit tests with pre-expired timestamps

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): use batch processing in GetMany

Process keys in chunks of 200 using slice.CollectChunks to avoid
hitting SQLite's SQLITE_MAX_VARIABLE_NUMBER limit with large key sets.

* feat(plugins): add periodic cleanup goroutine for expired kvstore keys

Use the manager's context to control a background goroutine that purges
expired keys every hour, stopping naturally on shutdown when the context
is cancelled.

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-28 23:12:17 -05:00
Deluan Quintão
e8863ed147
feat(plugins): add SubsonicAPI CallRaw, with support for raw=true binary response for host functions (#4982)
Some checks are pending
Pipeline: Test, Lint, Build / Get version info (push) Waiting to run
Pipeline: Test, Lint, Build / Lint Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test JS code (push) Waiting to run
Pipeline: Test, Lint, Build / Lint i18n files (push) Waiting to run
Pipeline: Test, Lint, Build / Check Docker configuration (push) Waiting to run
Pipeline: Test, Lint, Build / Build (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-1 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-2 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-3 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-4 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-5 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-6 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-7 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-8 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-9 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-10 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to GHCR (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to Docker Hub (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Cleanup digest artifacts (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build Windows installers (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Package/Release (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Upload Linux PKG (push) Blocked by required conditions
* feat: implement raw binary framing for host function responses

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add CallRaw method for Subsonic API to handle binary responses

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add tests for raw=true methods and binary framing generation

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: improve error message for malformed raw responses to indicate incomplete header

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: add wasm_import_module attribute for raw methods and improve content-type handling

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-04 15:48:08 -05:00
Deluan Quintão
1afcf7775b
feat: add ISRC matching for similar songs (#4946)
Some checks are pending
Pipeline: Test, Lint, Build / Push to GHCR (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to Docker Hub (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Cleanup digest artifacts (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build Windows installers (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Package/Release (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Get version info (push) Waiting to run
Pipeline: Test, Lint, Build / Lint Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test JS code (push) Waiting to run
Pipeline: Test, Lint, Build / Lint i18n files (push) Waiting to run
Pipeline: Test, Lint, Build / Check Docker configuration (push) Waiting to run
Pipeline: Test, Lint, Build / Build (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-1 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Upload Linux PKG (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-2 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-3 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-4 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-5 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-6 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-7 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-8 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-9 (push) Blocked by required conditions
* feat: add ISRC support to similar songs matching and plugin interface

Add ISRC (International Standard Recording Code) as a high-priority
identifier in the provider matching algorithm, alongside MBID. The
matching pipeline now uses four strategies in priority order:
ID > MBID > ISRC > Title+Artist fuzzy match.

- Add ISRC field to agents.Song struct
- Add ISRC field to plugin capability SongRef (Go, Rust PDKs)
- Add loadTracksByISRC using json_tree query on tags column
- Integrate ISRC into matchSongsToLibrary, selectBestMatchingSongs,
  and buildTitleQueries

https://claude.ai/code/session_01Dd4mTq1VQZag4RNjCVusiF

* chore: regenerate plugin schema after ISRC addition

Run `make gen` to update the generated YAML schema for the
metadata agent capability with the new ISRC field on SongRef.

https://claude.ai/code/session_01Dd4mTq1VQZag4RNjCVusiF

* feat(mediafile): add GetAllByTags method to MediaFileRepository for tag-based retrieval

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(provider): speed up track matching by incorporating prior matches in ISRC and MBID lookups

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Claude <noreply@anthropic.com>
2026-01-27 14:54:29 -05:00
Deluan Quintão
fda35dd8ce
feat(plugins): add similar songs retrieval functions and improve duration consistency (#4933)
Some checks are pending
Pipeline: Test, Lint, Build / Get version info (push) Waiting to run
Pipeline: Test, Lint, Build / Lint Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test JS code (push) Waiting to run
Pipeline: Test, Lint, Build / Lint i18n files (push) Waiting to run
Pipeline: Test, Lint, Build / Check Docker configuration (push) Waiting to run
Pipeline: Test, Lint, Build / Build (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-1 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-2 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-3 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-4 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-5 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-6 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-7 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-8 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-9 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to GHCR (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to Docker Hub (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Cleanup digest artifacts (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build Windows installers (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Package/Release (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Upload Linux PKG (push) Blocked by required conditions
* feat: add duration filtering for similar songs matching

Signed-off-by: Deluan <deluan@navidrome.org>

* test: refactor expectations for similar songs in provider matching tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add functions to retrieve similar songs by track, album, and artist

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): support uint32 in ndpgen

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update duration field to use seconds as float instead of milliseconds as uint32

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: add helper functions for Rust's skip_serializing_if with numeric types

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(provider): enhance track matching logic to fallback to title match when duration-filtered tracks fail

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-01-26 18:28:41 -05:00
Deluan Quintão
f1e75c40dc
feat(plugins): add JSONForms-based plugin configuration UI (#4911)
Some checks are pending
Pipeline: Test, Lint, Build / Get version info (push) Waiting to run
Pipeline: Test, Lint, Build / Lint Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test Go code (push) Waiting to run
Pipeline: Test, Lint, Build / Test JS code (push) Waiting to run
Pipeline: Test, Lint, Build / Lint i18n files (push) Waiting to run
Pipeline: Test, Lint, Build / Check Docker configuration (push) Waiting to run
Pipeline: Test, Lint, Build / Build (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-1 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-2 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-3 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-4 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-5 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-6 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-7 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-8 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build-9 (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to GHCR (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Push to Docker Hub (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Cleanup digest artifacts (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Build Windows installers (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Package/Release (push) Blocked by required conditions
Pipeline: Test, Lint, Build / Upload Linux PKG (push) Blocked by required conditions
* feat(plugins): add JSONForms schema for plugin configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance error handling by formatting validation errors with field names

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enforce required fields in config validation and improve error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* format JS code

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add config schema validation and enhance manifest structure

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: refactor plugin config parsing and add unit tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add config validation error message in Portuguese

* feat: enhance AlwaysExpandedArrayLayout with description support and improve array control testing

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update Discord Rust plugin configuration to use JSONForm for user tokens and enhance schema validation

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: resolve React Hooks linting issues in plugin UI components

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* format code

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: migrate schema validation to use santhosh-tekuri/jsonschema and improve error formatting

Signed-off-by: Deluan <deluan@navidrome.org>

* address PR comments

Signed-off-by: Deluan <deluan@navidrome.org>

* fix flaky test

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance array layout and configuration handling with AJV defaults

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement custom tester to exclude enum arrays from AlwaysExpandedArrayLayout

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add error boundary for schema rendering and improve error messages

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: refine non-enum array control logic by utilizing JSONForms schema resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add error styling to ToggleEnabledSwitch for disabled state

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: adjust label positioning and styling in SchemaConfigEditor for improved layout

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement outlined input controls renderers to replace custom fragile CSS

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: remove margin from last form control inside array items for better spacing

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance AJV error handling to transform required errors for field-level validation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: set default value for User Tokens in manifest.json to improve user experience

Signed-off-by: Deluan <deluan@navidrome.org>

* format

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add margin to outlined input controls for improved spacing

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: remove redundant margin rule for last form control in array items

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: adjust font size of label elements in SchemaConfigEditor for improved readability

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-19 20:51:00 -05:00
Deluan Quintão
03a45753e9
feat(plugins): New Plugin System with multi-language PDK support (#4833)
* chore(plugins): remove the old plugins system implementation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement new plugin system with using Extism

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add capability detection for plugins based on exported functions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add auto-reload functionality for plugins with file watcher support

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add auto-reload functionality for plugins with file watcher support

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): standardize variable names and remove superfluous wrapper functions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): improve error handling and logging in plugin manager

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): implement plugin function call helper and refactor MetadataAgent methods

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): race condition in plugin manager

* tests(plugins): change BeforeEach to BeforeAll in MetadataAgent tests

Signed-off-by: Deluan <deluan@navidrome.org>

* tests(plugins): optimize tests

Signed-off-by: Deluan <deluan@navidrome.org>

* tests(plugins): more optimizations

Signed-off-by: Deluan <deluan@navidrome.org>

* test(plugins): ignore goroutine leaks from notify library in tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add Wikimedia plugin for Navidrome to fetch artist metadata

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): enhance plugin logging and set User-Agent header

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement scrobbler plugin with authorization and scrobbling capabilities

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): integrate logs

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): clean up manifest struct and improve plugin loading logic

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add metadata agent and scrobbler schemas for bootstrapping plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(hostgen): add hostgen tool for generating Extism host function wrappers

- Implemented hostgen tool to generate wrappers from annotated Go interfaces.
- Added command-line flags for input/output directories and package name.
- Introduced parsing and code generation logic for host services.
- Created test data for various service interfaces and expected generated code.
- Added documentation for host services and annotations for code generation.
- Implemented SubsonicAPI service with corresponding generated code.

* feat(subsonicapi): update Call method to return JSON string response

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement SubsonicAPI host function integration with permissions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(generator): error-only methods in response handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): generate client wrappers for host functions

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(generator): remove error handling for response.Error in client templates

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scheduler): add Scheduler service interface with host function wrappers for scheduling tasks

* feat(plugins): add WASI build constraints to client wrapper templates, to avoid lint errors

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scheduler): implement Scheduler service with one-time and recurring scheduling capabilities

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(manifest): remove unused ConfigPermission from permissions schema

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scheduler): add scheduler callback schema and implementation for plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scheduler): streamline scheduling logic and remove unused callback tracking

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scheduler): add Close method for resource cleanup on plugin unload

Signed-off-by: Deluan <deluan@navidrome.org>

* docs(scheduler): clarify SchedulerCallback requirement for scheduling functions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: update wasm build rule to include all Go files in the directory

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: rewrite the wikimedia plugin using the XTP CLI

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scheduler): replace uuid with id.NewRandom for schedule ID generation

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: capabilities registration

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add scheduler service isolation test for plugin instances

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: update plugin manager initialization and encapsulate logic

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add WebSocket service definitions for plugin communication

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement WebSocket service for plugin integration and connection management

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Crypto Ticker example plugin for real-time cryptocurrency price updates via Coinbase WebSocket API

Also add the lifecycle capability

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: use context.Background() in invokeCallback for scheduled tasks

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename plugin.create() to plugin.instance()

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename pluginInstance to plugin for consistency across the codebase

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: simplify schedule cloning in Close method and enhance plugin cleanup error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement Artwork service for generating artwork URLs in Navidrome plugins - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: moved public URL builders to avoid import cycles

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Cache service for in-memory TTL-based caching in plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Discord Rich Presence example plugin for Navidrome integration

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: host function wrappers to use structured request and response types

- Updated the host function signatures in `nd_host_artwork.go`, `nd_host_scheduler.go`, `nd_host_subsonicapi.go`, and `nd_host_websocket.go` to accept a single parameter for JSON requests.
- Introduced structured request and response types for various cache operations in `nd_host_cache.go`.
- Modified cache functions to marshal requests to JSON and unmarshal responses, improving error handling and code clarity.
- Removed redundant memory allocation for string parameters in favor of JSON marshaling.
- Enhanced error handling in WebSocket and cache operations to return structured error responses.

* refactor: error handling in various plugins to convert response.Error to Go errors

- Updated error handling in `nd_host_scheduler.go`, `nd_host_websocket.go`, `nd_host_artwork.go`, `nd_host_cache.go`, and `nd_host_subsonicapi.go` to convert string errors from responses into Go errors.
- Removed redundant error checks in test data plugins for cleaner code.
- Ensured consistent error handling across all plugins to improve reliability and maintainability.

* refactor: rename fake plugins to test plugins for clarity in integration tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add help target to Makefile for plugin usage instructions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Cover Art Archive plugin as an example of Python plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update Makefile and README to clarify Go plugin usage

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: include plugin capabilities in loading log message

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add trace logging for plugin availability and error handling in agents

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Now Playing Logger plugin to showcase calling host functions from Python plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: generate Python client wrappers for various host services

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add generated host function wrappers for Scheduler and SubsonicAPI services

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update Python plugin documentation and usage instructions for host function wrappers

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Webhook Scrobbler plugin in Rust to send HTTP notifications on scrobble events

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enable parallel loading of plugins during startup

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update README to include WebSocket callback schema in plugin documentation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: extend plugin watcher with improved logging and debounce duration adjustment

Signed-off-by: Deluan <deluan@navidrome.org>

* add trace message for plugin recompiles

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement plugin cache purging functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* test: move purgeCacheBySize unit tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins UI): add plugin repository and database support

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins UI): add plugin management routes and middleware

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins UI): implement plugin synchronization with database for add, update, and remove actions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins UI): add PluginList and PluginShow components with plugin management functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): optimize plugin change detection

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins UI): improve PluginList structure

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins UI): enhance PluginShow with author, website, and permissions display

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins UI): refactor to use MUI and RA components

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins UI): add error handling for plugin enable/disable actions

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): inject PluginManager into native API

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update GetManager to accept DataStore parameter

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add subsonicRouter to Manager and refactor host service registration

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): enhance debug logging for plugin actions and recompile logic

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): break manager.go into smaller, focused files

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): streamline error handling and improve plugin retrieval logic

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update newWebSocketService to use WebSocketPermission for allowed hosts

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): introduce ToggleEnabledSwitch for managing plugin enable/disable state

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update READMEs

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): add Library service for metadata access and filesystem integration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add Library Inspector plugin for periodic library inspection and file size logging

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update README to reflect JSON configuration format for plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(build): update target to wasm32-wasip1 for improved WASI support

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement configuration management UI with key-value pairs support

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): adjust grid layout in InfoRow component for improved responsiveness

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): rename ErrorIndicator to EnabledOrErrorField and enhance error handling logic

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(i18n): add Portuguese translations for plugin management and notifications

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add support for .ndp plugin packages and update build process

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update README for .ndp plugin packaging and installation instructions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement KVStore service for persistent key-value storage

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: enhance README with Extism plugin development resources and recommendations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): integrate event broker into plugin manager

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): update config handling in PluginShow to track last record state

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add Rust host function library and example implementation of Discord Rich Presence plugin in Rust

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): generate Rust lib.rs file to expose host function wrappers

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update JSON field names to camelCase for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: reduce cyclomatic complexity by refactoring main function

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): enhance Rust code generation with typed struct support and improved type handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add Go client library with host function wrappers and documentation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): generate Go client stubs for non-WASM platforms

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): update client template file names for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add initial implementation of the Navidrome Plugin Development Kit code generator - Pahse 1

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 (2)

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 3

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 4

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 5

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): consistent naming/types across PDK

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): streamline plugin function signatures and error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update scrobbler interface to return errors directly instead of response structs

Signed-off-by: Deluan <deluan@navidrome.org>

* test: make all test plugins use the PDK

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): reorganize and sort type definitions for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update error handling for methods to return errors directly

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update function signatures to return values directly instead of response structs

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update request/response types to use private naming conventions

Signed-off-by: Deluan <deluan@navidrome.org>

* build: mark .wasm files as intermediate for cleanup after building .ndp

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): consolidate PDK module path and update Go version to 1.25

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement Rust PDK

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): reorganize Rust output structure to follow standard conventions

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update Discord Rich Presence and Library Inspector plugins to use nd-pdk for service calls and implement lifecycle management

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update macro names for websocket and metadata registration to improve clarity and consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename scheduler callback methods for consistency and clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update export wrappers to use `//go:wasmexport` for WebAssembly compatibility

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update plugin registration docs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): generate host wrappers

Signed-off-by: Deluan <deluan@navidrome.org>

* test(plugins): conditionally run goleak checks based on CI environment

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update README to reflect changes in plugin import paths

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update plugin instance creation to accept context for cancellation support

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update return types in metadata interfaces to use pointers

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): enhance type handling for Rust and XTP output in capability generation

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update IsAuthorized method to return boolean instead of response object

Signed-off-by: Deluan <deluan@navidrome.org>

* test(plugins): add unit tests for rustOutputType and isPrimitiveRustType functions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement XTP JSONSchema validation for generated schemas

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update response types in testMetadataAgent methods to use pointers

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update Go and Rust plugin developer sections for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: correct example link for library inspector in README

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: clarify artwork URL generation capabilities in service descriptions

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: update README to include Rust PDK crate information for plugin developers

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle URL parsing errors and use atomic upsert in plugin repository

Added proper error handling for url.Parse calls in PublicURL and AbsoluteURL
functions. When parsing fails, PublicURL now falls back to AbsoluteURL, and
AbsoluteURL logs the error and returns an empty string, preventing malformed
URLs from being generated.

Replaced the non-atomic UPDATE-then-INSERT pattern in plugin repository Put
method with a single atomic INSERT ... ON CONFLICT statement. This eliminates
potential race conditions and improves consistency with the upsert pattern
already used in host_kvstore.go.

* feat: implement mock service instances for non-WASM builds using testify/mock

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: Discord RPC struct to encapsulate WebSocket logic

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add support for experimental WebAssembly threads

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add PDK abstraction layer with mock support for non-WASM builds

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add unit tests for Discord plugin and RPC functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: update return types in minimalPlugin and wikimediaPlugin methods to use pointers

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: context cancellation and implement WebSocket callback timeout for improved error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: conditionally include error handling in generated client code templates

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement ConfigService for plugin configuration management

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance plugin manager to support metrics recording

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: make MockPDK private

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: update interface types to use 'any' in plugin repository methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename List method to Keys for clarity in configuration management

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add ndpgen plugin tests in the pipeline and update Makefile

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add users permission management to plugin system

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: streamline users integration tests and enhance plugin user management

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove UserID from scrobbler request structure

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add integration tests for UsersService enable gate behavior

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement user permissions for SubsonicAPI and scrobbler plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: show proper error in the UI when enabling a plugin fails

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add library permission management to plugin system

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add user permission for processing scrobbles in Discord Rich Presence plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: implement dynamic loading for buffered scrobbler plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add GetAdmins method to retrieve admin users from the plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update Portuguese translations for user and library permissions

Signed-off-by: Deluan <deluan@navidrome.org>

* reorder migrations

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: remove unnecessary bulkActionButtons prop from PluginList component

* feat: add manual plugin rescan functionality and corresponding UI action

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement user/library and plugin management integration with cleanup on deletion

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: replace core mock services with test-specific implementations to avoid import cycles

* feat: add ID fields to Artist and Song structs and enhance track loading logic by prioritizing ID matches

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update plugin permissions from allowedHosts to requiredHosts for better clarity and consistency

* feat: refactor plugin host permissions to use RequiredHosts directly for improved clarity

* fix: don't record metrics for plugin calls that aren't implemented at all

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: enhance connection management with improved error handling and cleanup logic

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: introduce ArtistRef struct for better artist representation and update track metadata handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update user configuration handling to use user key prefix for improved clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance ConfigCard input fields with multiline support and vertical resizing

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: rust plugin compilation error

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement IsOptionPattern method for better return type handling in Rust PDK generation

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-01-14 19:22:48 -05:00
Deluan Quintão
66eaac2762
fix(plugins): add metrics on callbacks and improve plugin method calling (#4304)
* refactor: implement OnSchedulerCallback method in wasmSchedulerCallback

Added the OnSchedulerCallback method to the wasmSchedulerCallback struct, enabling it to handle scheduler callback events. This method constructs a SchedulerCallbackRequest and invokes the corresponding plugin method, facilitating better integration with the scheduling system. The changes improve the plugin's ability to respond to scheduled events, enhancing overall functionality.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update executeCallback method to use callMethod

Modified the executeCallback method to accept an additional parameter,
methodName, which specifies the callback method to be executed. This change
ensures that the correct method is called for each WebSocket event,
improving the accuracy of callback execution for plugins.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): capture OnInit metrics

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): improve logging for metrics in callMethod

Updated the logging statement in the callMethod function to include the
elapsed time as a separate key in the log output. This change enhances
the clarity of the logged metrics, making it easier to analyze the
performance of plugin requests and troubleshoot any issues that may arise.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): enhance logging for schedule callback execution

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(server): streamline scrobbler stopping logic

Refactored the logic for stopping scrobbler instances when they are removed.
The new implementation introduces a `stoppableScrobbler` interface to
simplify the type assertion process, allowing for a more concise and
readable code structure. This change ensures that any scrobbler
implementing the `Stop` method is properly stopped before removal,
improving the overall reliability of the plugin management system.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): improve plugin lifecycle management and error handling

Enhanced the plugin lifecycle management by implementing error handling in the OnInit method. The changes include the addition of specific error conditions that can be returned during plugin initialization, allowing for better management of plugin states. Additionally, the unregisterPlugin method was updated to ensure proper cleanup of plugins that fail to initialize, improving overall stability and reliability of the plugin system.

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): remove unused LoadAllPlugins and related methods

Eliminated the LoadAllPlugins, LoadAllMediaAgents, and LoadAllScrobblers
methods from the manager implementation as they were not utilized in the codebase.
This cleanup reduces complexity and improves maintainability by removing
redundant code, allowing for a more streamlined plugin management process.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update logging configuration for plugins

Configured logging for multiple plugins to remove timestamps and source file/line information, while adding specific prefixes for better identification.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): clear initialization state when unregistering a plugin

Added functionality to clear the initialization state of a plugin in the
lifecycle manager when it is unregistered. This change ensures that the
lifecycle state is accurately maintained, preventing potential issues with
plugins that may be re-registered after being unregistered. The new method
`clearInitialized` was implemented to handle this state management.

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add unit tests for convertError function, rename to checkErr

Added comprehensive unit tests for the convertError function to ensure
correct behavior across various scenarios, including handling nil responses,
typed nils, and responses implementing errorResponse. These tests validate
that the function returns the expected results without panicking and
correctly wraps original errors when necessary.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): update plugin base implementation and method calls

Refactored the plugin base implementation by renaming `wasmBasePlugin` to `baseCapability` across multiple files. Updated method calls in the `wasmMediaAgent`, `wasmSchedulerCallback`, and `wasmScrobblerPlugin` to align with the new base structure. These changes improve code clarity and maintainability by standardizing the plugin architecture, ensuring consistent usage of the base capabilities across different plugin types.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(discord): handle failed connections and improve heartbeat checks

Added a new method to clean up failed connections, which cancels the heartbeat schedule, closes the WebSocket connection, and removes cache entries. Enhanced the heartbeat check to log failures and trigger the cleanup process on the first failure. These changes ensure better management of user connections and improve the overall reliability of the RPC system.

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-07-05 09:03:49 -03:00
Deluan Quintão
f1fc2cd9b9
feat(plugins): experimental support for plugins (#3998)
* feat(plugins): add minimal test agent plugin with API definitions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add plugin manager with auto-registration and unique agent names

Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator.

* test: add Ginkgo suite and test for plugin manager auto-registration

Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support.

* test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement persistent compilation cache for WASM agent plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement instance pooling for wasmAgent to improve resource management

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): enhance logging for wasmAgent and plugin manager operations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement HttpService for handling HTTP requests in WASM plugins

Also add a sample Wikimedia plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): standardize error handling in wasmAgent and MinimalAgent

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: clean up wikimedia plugin code

Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy.

* refactor: use unified SPARQLResult struct and parser for SPARQL responses

Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability.

* feat(plugins): improve error handling in HTTP request processing

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: background plugin compilation, logging, and race safety

Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass.

* perf: implement true lazy loading for agents

Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass.

* fix: ensure wasm plugin instances are closed via runtime.AddCleanup

Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup.

* refactor: break down long functions in plugin manager and agent

Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go.

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename wasmAgent to wasmArtistAgent

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugin): rename MinimalAgent for artist metadata service

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): enhance plugin pool creation with custom runtime and precompilation support

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): implement generic plugin pool and agent factory for improved service handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): reorganize plugin management

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): improve function signatures for clarity and consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement background precompilation for plugins and agent factory creation

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): include instanceID in logging for better traceability

Signed-off-by: Deluan <deluan@navidrome.org>

* test(plugins): add tests for plugin pre-compilation and agent factory synchronization

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add minimal album test agent plugin for AlbumMetadataService

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): rename fake artist and album test agent plugins for metadata services

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(makefile): add Makefile for building plugin WASM binaries

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages

Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized.

* feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove wasm module pooling

it was causing issues with the GC and the Close methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename metadata service files to adapter naming convention

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: unify album and artist method calls by introducing callMethod function

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: unify album and artist method calls by introducing callMethod function

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle nil values in data redaction process

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: add timeout for plugin compilation to prevent indefinite blocking

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement ScrobblerService plugin with authorization and scrobbling capabilities

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: simplify generalization

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance plugin management by improving scanning and loading mechanisms

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: update plugin creation functions to return specific interfaces for better type safety

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement MediaMetadataService with combined artist and album methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: improve MediaMetadataService plugin implementation and testing structure

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: add tests for Adapter Media Agent and improve plugin documentation

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance agent management with plugin loading and caching

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: update agent discovery logic to include only local agent when no config is specified

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: correct file extension filter in goimports command

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: use defer to unlock the mutex

Signed-off-by: Deluan <deluan@navidrome.org>

* chore: move Cover Art Archive AlbumMetadataService plugins to an example folder

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle errors when creating media metadata and scrobbler service plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: increase compilation timeout to one minute

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add configurable plugin compilation timeout

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement plugin scrobbler support in PlayTracker

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add context management and Stop method to buffered scrobbler

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add username field to scrobbler requests and update logging

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: data race in test

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename http proto files to host and update references

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove unused plugin registration methods from manager

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: extend plugin manifests and implement plugin management commands

Signed-off-by: Deluan <deluan@navidrome.org>

* Update utils/files.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip")

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* feat: add plugin dev workflow support

Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: lint errors

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(README): update documentation to include TimerCallbackService and its functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add InitService with OnInit method and initialization tracking - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add tests for InitService and plugin initialization tracking

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): expand documentation on plugin system implementation and architecture

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: panic

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): redirect plugins' stderr to logs

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add safe accessor methods for TimerService

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add plugin-specific configuration support in InitRequest and documentation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add TimerCallbackService plugin adapter and integration

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename services for consistency and clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add mutex for configuration access and clone plugin config

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(tests): remove configtest dependency to prevent data races in integration tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): correctly pass error to done function in wasmBasePlugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename service types to capabilities for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename InitService to LifecycleManagement for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): fix instance ID logging in wasmBasePlugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): make timers be isolated per plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): make timers be isolated per plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add config service for plugin-specific configuration management

Signed-off-by: Deluan <deluan@navidrome.org>

* Update plugins/manager.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update plugins/manager.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat(crontab): implement crontab service for scheduling and canceling jobs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(singleton): fix deadlock issue when a constructor calls GetSingleton again

Signed-off-by: Deluan <deluan@navidrome.org> (+1 squashed commit)
Squashed commits:
[325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scheduler): consolidate job scheduling logic into a single helper function

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugin): rename GetInstance method to Instantiate for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add WebSocket service for handling connections and messages

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(websocket): enhance connection management and callback handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manager): only create one adapter instance for each adapter/capability pair

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: flaky test

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugin): refactor WebSocket service integration and improve error logging

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugin): add SchedulerCallback support and improve reconnection logic

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: test panic

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: add crypto-ticker plugin example to README

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(websocket): add error field to response messages for better error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(cache): implement CacheService with string, int, float, and byte operations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(cache): simplify cache service implementation and remove unnecessary synchronization

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(tests): add build step for test plugins in the test suite

Signed-off-by: Deluan <deluan@navidrome.org>

* wip

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(rpc): enhance activity image processing and improve error handling in Discord integration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(discord): enhance activity state with artist list and add large text asset

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(artwork): implement ArtworkService for retrieving artwork URLs

Signed-off-by: Deluan <deluan@navidrome.org>

* Add playback position to scrobble NowPlaying (#4089)

* test(playtracker): cover playback position

* address review comment

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* fix merge

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove unnecessary check for empty slice in Map function

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: update reflex.conf to include .wasm file extension

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): normalize attribute strings and add edge case tests for PID calculation

Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458

Signed-off-by: Deluan <deluan@navidrome.org>

* test(ui): fix warnings (#4187)

* fix(ui): address test warnings

* ignore lint error in test

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(server): optimize top songs lookup (#4189)

* optimize top songs lookup

* Optimize title matching queries

* refactor: simplify top songs matching

* improve error handling and logging in track loading functions

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add cases for fallback to title matching and combined MBID/title matching

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): playlist details overflow in spotify-based themes (#4184)

* test: ensure playlist details width

* fix(test): simplify expectation for minWidth in NDPlaylistDetails

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(test): test all themes

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): update TagLib to version 2.1 (#4185)

* chore: update cross-taglib

* fix(taglib): add logging for TagLib version

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* test: verify agents fallback (#4191)

* build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version.

This is to reduce the image size, as we don't really need the latest.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(runtime): implement pooled WASM runtime and module for better instance management

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(discord-plugin): adjust timer delay calculation for track completion

Signed-off-by: Deluan <deluan@navidrome.org>

* resolve PR comments

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement cache cleanup by size functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(manager): return error from getCompilationCache and handle it in ScanPlugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix possible rce condition

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(docs): update README to include Cache and Artwork services

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manager): add permissions support for host services in custom runtime - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manifest): add permissions field to plugin manifests - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* test(permissions): implement permission validation and testing for plugins - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add unauthorized_plugin to test permission enforcement - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(docs): add Plugin Permission System section to README - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manifest): add detailed reasons for permissions in plugin manifests - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(permissions): implement granular HTTP permissions for plugins - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: unexport all plugins package private symbols

Signed-off-by: Deluan <deluan@navidrome.org>

* update docs

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename plugin_lifecycle_manager

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: add discord-rich-presence plugin example to README

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add support for PATCH, HEAD, and OPTIONS HTTP methods

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: use folder names as unique identifiers for plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: read config just once, to avoid data race in tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename pluginName to pluginID for consistency across services

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: use symlink name instead of folder name for plugin registration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update plugin output format to include ID and enhance README with symlink usage

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement shared plugin discovery function to streamline plugin scanning and error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: show plugin permissions in `plugin info`

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement typed permissions for plugins to enhance permission handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: refactor plugin permissions to use typed schema and improve validation - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: remove unused JSON schema validation for plugin manifests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: remove unused fields from PluginPackage struct in package.go

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update file permissions in tests and remove unused permission parsing function

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: refactor test plugin creation to use typed permissions and remove legacy helper

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add website field to plugin manifests and update test cases

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: permission schema to use basePermission structure for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance host service management by adding permission checks for each service

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: reorganize code files

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: simplify custom runtime creation by removing compilation cache parameter

Signed-off-by: Deluan <deluan@navidrome.org>

* doc: add WebSocketService and update ConfigService for plugin-specific configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement WASM loading optimization to enhance plugin instance creation speed

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename custom runtime functions and update related tests for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance plugin structure with compilation handling and error reporting

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: improve logging and context tracing in runtime and wasm base plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance runtime management with scoped runtime and caching improvements

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement EnsureCompiled method for improved plugin compilation handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement cached module management with TTL for improved performance

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: replace map with sync.Map

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance image processing with fallback mechanism for improved error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: review test plugins readme

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: set default timeout for HTTP client to 10 seconds

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance wasm instance pool with concurrency limits and timeout settings

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(discordrp): implement caching for processed image URLs with configurable TTL

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-06-22 20:45:38 -04:00