navidrome/utils/str
Deluan Quintão 427d4b9bce
fix(search): artists with atomic non-ASCII names unfindable after FTS5 migration (#5703)
* fix(scanner): update artist search_normalized when rescanning

The FTS5 migration back-fills artist.search_normalized with a SQL
punctuation-strip approximation, relying on the next scan to compute the
precise value in Go (normalizeForFTS transliterates atomic letters like
Ø/æ/ß that FTS5's remove_diacritics cannot fold). But the scanner
persisted artists with an explicit column list that omitted
search_normalized, so not even a full scan ever repaired it: an artist
migrated from a pre-FTS database (e.g. "GØGGS") stayed unfindable by
any ASCII search, while their albums and songs, which are saved with all
columns, were fixed by a full scan. Add search_normalized to the column
list so a full scan re-indexes the artist via the artist_fts trigger.

* refactor(persistence): move normalizeForFTS to utils/str

Export it as str.NormalizeForFTS so the upcoming migration can reuse the
exact index-time normalization. Migrations cannot import the persistence
package (persistence -> db -> db/migrations would be an import cycle).

* fix(persistence): backfill artist search_normalized via migration

Recompute artist.search_normalized with the precise Go normalization for
databases migrated from pre-FTS5 versions, where the SQL back-fill could
not transliterate atomic letters (Ø/æ/ß) and the scanner never rewrote
the column. Only changed rows are updated, so the artist_fts update
trigger re-indexes exactly the affected artists, making artists like
GØGGS or MØ findable again without requiring a full scan.

* refactor(persistence): share FTS punctuation-strip regex via utils/str

Index-time normalization (NormalizeForFTS) and query-time processing
(buildFTS5Query/ftsQueryDegraded) must produce matching tokens, so keep
the punctuation-strip pattern in a single exported symbol instead of two
identical private copies that could drift. Also document that derived
columns computed in dbArtist.PostMapArgs must be listed in the scanner's
artist Put, which is how search_normalized went stale in the first place.

* chore(migrations): announce artist search backfill in the log

Match the FTS5 migration's notice() pattern so startup isn't silent
while the backfill runs on large libraries.

* docs: tighten comments added in this branch

* docs: describe FTSPunctStrip by what it matches, not one replacement
2026-07-02 12:53:10 -04:00
..
normalize_fts.go fix(search): artists with atomic non-ASCII names unfindable after FTS5 migration (#5703) 2026-07-02 12:53:10 -04:00
normalize_fts_test.go fix(search): artists with atomic non-ASCII names unfindable after FTS5 migration (#5703) 2026-07-02 12:53:10 -04:00
sanitize_strings.go feat(cli): add pls export/import subcommands for bulk playlist management (#5412) 2026-04-25 20:54:02 -04:00
sanitize_strings_test.go fix: split html sanitization from plaintext handling (#5403) 2026-04-23 17:53:28 -04:00
str.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
str_suite_test.go Refactor string utilities into its own package str 2024-06-05 22:09:27 -04:00
str_test.go refactor: extract TruncateRunes function for safe string truncation with suffix 2025-11-06 14:27:38 -05:00