navidrome/persistence
Deluan Quintão 81a17f6bbb
fix(search): normalization for non-NFKD Unicode letters (ø, æ, œ, ß) (#5413)
* fix(search): transliterate non-ASCII letters symmetrically in FTS5 path

Songs and artists with letters like ø, æ, œ, ß were unsearchable. The
query path in server/subsonic/searching.go transliterates with
sanitize.Accents (Øystein → Oystein), but the FTS5 tokenizer's
remove_diacritics 2 only strips NFKD-decomposable marks — atomic
letters with built-in strokes/ligatures survive tokenization, so the
query side and index side disagreed.

Apply sanitize.Accents on both sides:

- normalizeForFTS now also emits an ASCII-transliterated form for each
  word, so search_normalized contains the variant the query produces.
- buildFTS5Query transliterates the unquoted portion of the input so
  every caller (Subsonic, REST fullTextFilter) gets the same handling.
  Quoted phrases stay as typed, preserving phrase matches against the
  original title/artist columns.

Existing libraries pick up the fix as records are re-scanned; users
can trigger a manual full rescan to refresh older entries.

* fix(search): cache transliteration and add ß/quoted-phrase test coverage

Address review feedback: call sanitize.Accents once per word and reuse
the result for both the punct-stripped and accent-only paths. Add missing
test entries for ß→ss transliteration and quoted Unicode phrase preservation.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-25 20:27:38 -04:00
..
e2e feat(smartplaylists): relax playlist visibility in inPlaylist/notInPlaylist rules (#5411) 2026-04-25 14:59:06 -04:00
album_repository.go fix(subsonic): always emit required created field on AlbumID3 (#5340) 2026-04-10 19:29:20 -04:00
album_repository_test.go fix(subsonic): always emit required created field on AlbumID3 (#5340) 2026-04-10 19:29:20 -04:00
artist_repository.go fix(server): clean up uploaded artist images during GC 2026-03-17 19:47:09 -04:00
artist_repository_test.go fix(server): clean up uploaded artist images during GC 2026-03-17 19:47:09 -04:00
collation_test.go fix(ui): make playlist name sorting case-insensitive (#4845) 2026-01-05 19:05:11 -05:00
criteria_sql.go feat(smartplaylists): relax playlist visibility in inPlaylist/notInPlaylist rules (#5411) 2026-04-25 14:59:06 -04:00
criteria_sql_test.go feat(smartplaylists): relax playlist visibility in inPlaylist/notInPlaylist rules (#5411) 2026-04-25 14:59:06 -04:00
export_test.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
folder_repository.go fix(scanner): add nil guards to cursor wrapping (#5139) 2026-03-03 07:58:14 -05:00
folder_repository_test.go ci: run Go tests on Windows (#5380) 2026-04-19 13:16:47 -04:00
genre_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
genre_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
helpers.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
helpers_test.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
library_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
library_repository_test.go ci: run Go tests on Windows (#5380) 2026-04-19 13:16:47 -04:00
mediafile_repository.go feat(subsonic): implement OpenSubsonic Transcoding extension (#4990) 2026-03-08 23:57:49 -04:00
mediafile_repository_test.go fix(tests): update media file paths in tests to be relative 2026-04-19 20:07:23 -04:00
persistence.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
persistence_suite_test.go fix(tests): update media file paths in tests to be relative 2026-04-19 20:07:23 -04:00
persistence_test.go fix(server): play queue should not return empty entries for deleted tracks 2024-09-20 11:22:37 -04:00
player_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
player_repository_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
playlist_repository.go feat(smartplaylists): relax playlist visibility in inPlaylist/notInPlaylist rules (#5411) 2026-04-25 14:59:06 -04:00
playlist_repository_test.go fix(tests): update media file paths in tests to be relative 2026-04-19 20:07:23 -04:00
playlist_track_repository.go refactor: move playlist business logic from repositories to service layer (#5027) 2026-02-21 19:57:13 -05:00
playqueue_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
playqueue_repository_test.go fix(server): ensure single record per user by reusing existing playqueue ID 2025-06-11 17:26:13 -04:00
plugin_cleanup.go feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
plugin_cleanup_test.go feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
plugin_repository.go fix(plugins): clear plugin errors on startup to allow retrying 2026-03-02 08:56:56 -05:00
plugin_repository_test.go fix(plugins): clear plugin errors on startup to allow retrying 2026-03-02 08:56:56 -05:00
property_repository.go Replace beego/orm with dbx (#2693) 2023-12-09 13:52:17 -05:00
property_repository_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
radio_repository.go feat(ui): add cover art support for internet radio stations (#5229) 2026-03-18 18:57:33 -04:00
radio_repository_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
scrobble_buffer_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
scrobble_buffer_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
scrobble_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
scrobble_repository_test.go feat(server): track scrobble/linstens history (#4770) 2025-12-06 11:07:18 -05:00
share_repository.go fix(share): add ownership checks to Delete and Update (#5189) 2026-03-15 00:12:58 -04:00
share_repository_test.go fix(share): add ownership checks to Delete and Update (#5189) 2026-03-15 00:12:58 -04:00
sql_annotations.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
sql_annotations_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
sql_base_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
sql_base_repository_test.go fix(server): headless library access improvements (#4362) 2025-07-20 15:58:21 -04:00
sql_bookmarks.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
sql_bookmarks_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
sql_participations.go fix(scanner): remove stale role associations when artist role changes. Fix #4242 2025-12-16 06:38:50 -05:00
sql_restful.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_restful_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search_fts.go fix(search): normalization for non-NFKD Unicode letters (ø, æ, œ, ß) (#5413) 2026-04-25 20:27:38 -04:00
sql_search_fts_test.go fix(search): normalization for non-NFKD Unicode letters (ø, æ, œ, ß) (#5413) 2026-04-25 20:27:38 -04:00
sql_search_like.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search_like_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_tags.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
tag_library_filtering_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
tag_repository.go feat(scanner): improve error messages for cleanup operations in annotations, bookmarks, and tags 2025-11-20 09:27:42 -05:00
tag_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
transcoding_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
transcoding_repository_test.go fix(transcoding): restrict transcoding operations to admin users (#4096) 2025-05-21 22:19:23 -04:00
user_props_repository.go Replace beego/orm with dbx (#2693) 2023-12-09 13:52:17 -05:00
user_repository.go feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
user_repository_test.go fix: qualify user id filter to avoid ambiguous column (#4511) 2025-11-06 14:54:01 -05:00