readest/.gitmodules
Huang Xin 58d4661b74
feat(ios): CarPlay support and native TTS playout with Now Playing integration (#5085)
* feat(ios): add CarPlay support for TTS

Surface active TTS sessions on Apple CarPlay, mirroring the Android Auto
integration. A CarPlaySceneDelegate presents a CPListTemplate root
showing the currently reading book (or a placeholder when idle) and
pushes the system Now Playing template. Transport and metadata ride the
existing MPNowPlayingInfoCenter / MPRemoteCommandCenter path, so no
CarPlay button is routed by hand. A new iOS-only update_carplay_state
command lets the media bridge report session start and stop, and TTS
autoplay deep links are consumed even when the target book is already
mounted, for example when the app cold restores straight into the
reader.

Registering the UIApplicationSceneManifest crashed the app at launch on
real devices: tao's configuration_for_connecting_scene_session returned
a raw borrow of a Retained UISceneConfiguration it immediately dropped,
so UIKit retained freed memory. Add the tao fork as the packages/tao
submodule (github.com/readest/tao, branch readest-0.35: tao 0.35.3 plus
only the callback returning an autoreleased strong reference) and patch
it in the workspace. Bump @tauri-apps/api to 2.11.1 and @tauri-apps/cli
to 2.11.4 for the tao 0.35/wry 0.55 stack, and advance the tauri fork
(branch readest-2.11) with a guard in the generated unlisten script
against already-removed listeners.

The tauri 2.11 stack needs two CI accommodations. First, the
build_tauri_app job overrides setup-rust-toolchain's default RUSTFLAGS
of -D warnings: cargo builds path-patched dependencies without
cap-lints, so upstream warnings in the tao fork (deprecated glib
channel calls in the Linux backend) would fail the app build; warnings
in our own crate stay enforced by the rust_lint job. Second, tauri 2.11
enforces ACL for IPC from remote origins even without an AppManifest
(upstream security fix #15266), and the webdriver harness serves the
vitest tester page from its own port, which is a remote origin, so
every custom app command was denied and 56 tauri tests failed.
Production is unaffected since real app pages are local origins.
Declare the app commands in the ACL app manifest in build.rs (generated
allow permissions committed under src-tauri/permissions/autogenerated),
grant them in capabilities/default.json for local windows, and add a
webdriver-remote capability for localhost remote origins that is pinned
out of shipped builds via app.security.capabilities in tauri.conf.json;
scripts/test-tauri.sh opts into it with a --config override. Verified
with pnpm test:tauri: 112 passed, 1 skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(android): handle audio focus properly for background TTS

Request focus with a modern AudioFocusRequest (USAGE_MEDIA,
CONTENT_TYPE_SPEECH) and pause instead of ducking, matching how spoken
audio should yield to navigation prompts. Resume after transient loss
only when the loss is what paused us, forward focus transitions to the
web side as media-session play/pause events, and pause on
ACTION_AUDIO_BECOMING_NOISY so unplugging headphones does not blast the
speaker.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(ios): native audio playout for Edge TTS with Now Playing support

WebAudio in WKWebView renders in WebKit's GPU process under an audio
session the app cannot own, which made every system media behavior
unreliable on device: no lock-screen card, the card vanished on pause,
AirPods resumed another app, and the mute switch silenced playback.

Play the Edge TTS utterances with an in-process AVPlayer instead. The
native-tts plugin gains a deliberately dumb playout engine
(playout_enqueue, playout_control, playout_position) and the new
NativeAudioPlayer mirrors the WebAudioPlayer session interface, so
EdgeTTSClient just picks the player per platform. Word highlighting
reads the player's media clock, which AVPlayer reports in item time and
is therefore unaffected by the playback rate; rate changes stay pitch
preserving via the timeDomain pitch algorithm and inter-sentence gaps
run on a native timer immune to background JS throttling.

With the audio in the app process the plugin claims a non-mixable
.playback session with .spokenAudio mode, so navigation prompts pause
instead of duck. Handle interruption began and ended with shouldResume,
pause on route change to the speaker, and bind an MPNowPlayingSession
to the playout player so the paused state is reported explicitly and
the Now Playing slot survives a pause. The silent keep-alive player
pauses in lockstep, KVC playback state writes are simulator only where
they are required and harmful on device, and the mixWithOthers
use_background_audio calls are removed. On the web side the composite
iOS media session mirrors metadata and position into
navigator.mediaSession, remote commands map to directional play and
pause plus a toggle, and the silent unblocker element is skipped on iOS
Tauri since a playing HTMLMediaElement makes WebKit register its own
competing now-playing client.

Volume-key page flip previously defeated all of this: its handler
unconditionally reconfigured the shared session with .mixWithOthers,
and a mixable session is ineligible for the Now Playing slot, so
pausing TTS dismissed the card and handed AirPod controls to another
app. The two plugins now coordinate through in-process notifications:
native-tts announces claim and release of the audio session, and the
volume key handler leaves the session untouched while TTS owns it, so
volume keys keep flipping pages during a pause without losing the card,
and re-owns the session after TTS teardown regardless of ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 10:53:31 +02:00

30 lines
1.3 KiB
Text

[submodule "packages/foliate-js"]
path = packages/foliate-js
url = https://github.com/readest/foliate-js.git
[submodule "packages/tauri"]
path = packages/tauri
url = https://github.com/readest/tauri.git
[submodule "packages/tauri-plugins"]
path = packages/tauri-plugins
url = https://github.com/readest/tauri-plugins-workspace.git
[submodule "packages/simplecc-wasm"]
path = packages/simplecc-wasm
url = https://github.com/readest/simplecc-wasm.git
[submodule "apps/readest-app/src-tauri/plugins/tauri-plugin-turso"]
path = apps/readest-app/src-tauri/plugins/tauri-plugin-turso
url = https://github.com/readest/tauri-plugin-turso.git
[submodule "apps/readest-app/.claude/skills/gstack"]
path = apps/readest-app/.claude/skills/gstack
url = https://github.com/garrytan/gstack.git
[submodule "packages/qcms"]
path = packages/qcms
url = https://github.com/mozilla/pdf.js.qcms.git
[submodule "packages/js-mdict"]
path = packages/js-mdict
url = https://github.com/readest/js-mdict.git
[submodule "apps/readest-app/src-tauri/plugins/tauri-plugin-webview-upgrade"]
path = apps/readest-app/src-tauri/plugins/tauri-plugin-webview-upgrade
url = https://github.com/readest/tauri-plugin-webview-upgrade.git
[submodule "packages/tao"]
path = packages/tao
url = https://github.com/readest/tao.git