mirror of
https://github.com/Darkatse/TauriTavern.git
synced 2026-10-19 07:35:14 +00:00
Fix (Android): Correctly fix the Android media contract to support video background extensions.
This commit is contained in:
parent
fa709ebde8
commit
c9d9ef964e
6 changed files with 342 additions and 51 deletions
|
|
@ -349,3 +349,28 @@ https://v2.tauri.app/develop/resources/#android
|
|||
- fullscreen 业务逻辑必须继续留在自维护文件(`MainActivity.kt` / `AndroidWebFullscreenController.kt`),不要把状态机堆回 generated 文件;
|
||||
- 不做前端 fullscreen polyfill 或静默降级,失败直接暴露,便于定位真实链路问题;
|
||||
- 未来升级 Tauri / Wry 时,只需要对比 upstream 的 `RustWebChromeClient.kt` 与本地替代版本的差异。
|
||||
|
||||
---
|
||||
|
||||
## 9. Android WebView 视频背景 Range 语义差异(SillyTavern-VideoBackgrounds)
|
||||
|
||||
现象:
|
||||
|
||||
- Android 端 `<video src="/backgrounds/*.mp4">` 无法进入播放,页面表现为“只有一个大播放按钮”。
|
||||
- DevTools 常见表现为:
|
||||
- 早期出现 `416 Range Not Satisfiable` 或某个 `Range: bytes=...-` 请求被快速 canceled;
|
||||
- `<video>` 长时间停留在 `readyState=HAVE_NOTHING`,无法触发 `loadedmetadata`。
|
||||
|
||||
根因(运行时语义差异):
|
||||
|
||||
- Android WebView 在 `shouldInterceptRequest` 的资源拦截链路中,会对“拦截返回的响应流”再次应用请求 Range 语义。
|
||||
- 若宿主已经按 Range 做了 seek/slice(返回已经截取过的 bytes),WebView 的二次 Range 会把非 0 起点范围再次应用到截取后的流上,导致不可满足(历史上表现为 `416` / canceled)。
|
||||
|
||||
当前已落地 workaround(仅背景视频):
|
||||
|
||||
- 实现:`src-tauri/src/presentation/web_resources/user_data_endpoint.rs`
|
||||
- 策略:对 Android + `/backgrounds/*` + `video/*` + `Range start != 0`:
|
||||
- 返回 `206` + 正确 `Content-Range/Content-Length`
|
||||
- body 提供完整文件 bytes,让 WebView 自己在流上执行 Range(skip)
|
||||
|
||||
更多细节与全平台媒体契约见:`docs/CurrentState/MediaAssetContract.md`。
|
||||
|
|
|
|||
99
docs/CurrentState/MediaAssetContract.md
Normal file
99
docs/CurrentState/MediaAssetContract.md
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
# 媒体资源契约(Media Assets)现状
|
||||
|
||||
本文档记录当前**已经落地**的“浏览器原生媒体加载契约”:`<video>` / `<audio>` 在桌面与移动端对用户静态资源端点(尤其是 `/backgrounds/*`)的请求方式,以及宿主目前承诺的响应语义。
|
||||
|
||||
> 实现位置:`src-tauri/src/presentation/web_resources/user_data_endpoint.rs`
|
||||
|
||||
---
|
||||
|
||||
## 1. 范围与结论
|
||||
|
||||
目标(Public Contract):
|
||||
|
||||
- 上游 SillyTavern 与第三方扩展可以把 `/backgrounds/*` 等路径当作“普通 HTTP 资源端点”使用(子资源加载 + Range)。
|
||||
- 媒体文件(`video/*` / `audio/*`)必须满足浏览器媒体管线的最小网络契约:**支持 `Range`(单范围)并返回 `206 + Content-Range`**。
|
||||
|
||||
涉及端点(由 `user_data_endpoint.rs` 提供):
|
||||
|
||||
- `/backgrounds/*`(图片背景 + 视频背景)
|
||||
- `/assets/*`、`/user/files/*`(可能承载音视频/下载内容)
|
||||
- 以及同一实现覆盖的其它用户静态资源:`/characters/*`、`/User Avatars/*`、`/user/images/*`
|
||||
|
||||
---
|
||||
|
||||
## 2. 端点基础语义(全平台)
|
||||
|
||||
这些端点必须能被浏览器原生子资源加载(`<img src>` / `<video src>` / `CSS url()`),且 dev/prod 语义一致:
|
||||
|
||||
- 仅接受 `GET` / `HEAD` / `OPTIONS`,其他方法返回 `405`
|
||||
- 未命中返回真实 `404`(不回退到 `index.html`)
|
||||
- `Content-Type` 必须与文件类型匹配(基于扩展名推断)
|
||||
- `Cache-Control: no-store`
|
||||
- `Accept-Ranges: bytes`
|
||||
|
||||
---
|
||||
|
||||
## 3. Range 契约(单范围)
|
||||
|
||||
支持的 `Range` 形态(仅单范围):
|
||||
|
||||
- `Range: bytes=<start>-<end>`
|
||||
- `Range: bytes=<start>-`
|
||||
- `Range: bytes=-<suffixLen>`
|
||||
|
||||
响应语义:
|
||||
|
||||
- 满足范围:返回 `206 Partial Content`
|
||||
- `Content-Range: bytes <start>-<end>/<total>`
|
||||
- `Content-Length: <rangeLen>`
|
||||
- 非法/不满足:返回 `416 Range Not Satisfiable`
|
||||
- `Content-Range: bytes */<total>`
|
||||
|
||||
显式不支持:
|
||||
|
||||
- multi-range(例如 `bytes=0-1,2-3`)会按“非法 Range”处理并返回 `416`。
|
||||
|
||||
---
|
||||
|
||||
## 4. Android WebView 差异与当前 workaround(视频背景)
|
||||
|
||||
现象(历史问题):
|
||||
|
||||
- Android WebView 对 `video/mp4` 的请求序列通常包含多个 Range(例如 `bytes=0-`、`bytes=131072-`、尾部 Range 等)。
|
||||
- 在 Tauri mobile 的资源拦截链路中(`shouldInterceptRequest`),Android WebView 会对**拦截返回的响应流**再次应用请求的 Range 语义。
|
||||
- 若宿主已经按 Range 做了 seek/slice,再遇到非 0 起点 Range,会出现“再次 seek 导致不可满足”的情况,表现为 `416` 或请求被快速取消,最终 `<video>` 卡在 `HAVE_NOTHING`,无法进入 `loadedmetadata`。
|
||||
|
||||
当前已落地 workaround(仅 Android + 背景视频 + 非 0 Range 起点):
|
||||
|
||||
- 匹配条件:
|
||||
- 平台:Android(WebView 拦截链路)
|
||||
- 路径:`/backgrounds/*`
|
||||
- MIME:`video/*`
|
||||
- Range:`start != 0`
|
||||
- 响应策略:
|
||||
- 返回 **`206` + 正确的 `Content-Range/Content-Length`(对齐请求的范围)**
|
||||
- 但 body 提供**完整文件 bytes**,让 WebView 自己在流上执行 Range(skip)并喂给媒体管线
|
||||
|
||||
这保持了“对媒体管线可观察的 HTTP 契约”,并避开 Android WebView 的二次 Range 应用问题。
|
||||
|
||||
已知代价:
|
||||
|
||||
- 对这些请求会读取完整文件到内存;背景视频过大时可能带来额外内存/IO 压力。
|
||||
|
||||
---
|
||||
|
||||
## 5. 回归与诊断要点
|
||||
|
||||
最小回归探针(桌面/移动均适用):
|
||||
|
||||
- `fetch('/backgrounds/<file>.mp4', { headers: { Range: 'bytes=0-1' } })` 应返回 `206` 且包含 `Content-Range`
|
||||
|
||||
Android 端额外关注:
|
||||
|
||||
- 对 `Range: bytes=131072-` 等非 0 起点 Range,应依然返回 `206` 且包含 `Content-Range`
|
||||
- 若 `<video>` 长时间停留在 `readyState=HAVE_NOTHING`,优先排查 Range 契约是否被破坏(`206`、`Content-Range`、以及是否出现快速 canceled)
|
||||
|
||||
当需要确认媒体编码兼容性(解码层问题):
|
||||
|
||||
- 使用 `ffprobe` 检查视频编码/音频声道布局(Android 上 AAC 5.1 可能存在兼容风险)。
|
||||
|
||||
|
|
@ -44,3 +44,6 @@
|
|||
|
||||
7. `docs/CurrentState/BootstrapOptimization.md`
|
||||
- bootstrap / 启动链路中与冷启动内存基线相关的优化现状(如 tokenCache 避免 whole-load)
|
||||
|
||||
8. `docs/CurrentState/MediaAssetContract.md`
|
||||
- `<video>/<audio>` 依赖的全平台媒体资源契约现状(`Range`/`Content-Range`/Android WebView workaround)
|
||||
|
|
|
|||
|
|
@ -84,6 +84,7 @@
|
|||
2. 通过 `src-tauri/src/infrastructure/third_party_paths.rs` 解析并校验路径
|
||||
3. 通过 `src-tauri/src/infrastructure/third_party_assets.rs` 定位文件并推断 MIME
|
||||
4. 返回真实 bytes、正确 `Content-Type`、`Cache-Control: no-store`
|
||||
- 对用户静态资源端点(如 `/backgrounds/*`)若请求携带 `Range`,支持单范围并返回 `206 + Content-Range`(见 `docs/CurrentState/MediaAssetContract.md`)
|
||||
5. 未命中时返回真正 `404`,不回退到 `index.html`
|
||||
|
||||
开发态本地 Web 入口:
|
||||
|
|
|
|||
|
|
@ -212,6 +212,7 @@
|
|||
- 仅接受 `GET` / `HEAD` / `OPTIONS`
|
||||
- 未命中返回真实 `404`(不回退 `index.html`)
|
||||
- `Content-Type` 正确,`Cache-Control: no-store`
|
||||
- 媒体文件(`video/*` / `audio/*`)必须支持 `Range`(单范围)并返回 `206 + Content-Range`(见 `docs/CurrentState/MediaAssetContract.md`)
|
||||
|
||||
禁止事项(为了保持契约稳定):
|
||||
|
||||
|
|
@ -279,6 +280,7 @@ header 名也可从 `window.__TAURITAVERN__?.traceHeader` 获取(用于避免
|
|||
- `/thumbnail?type=bg|avatar|persona&file=...` 能返回图片 bytes(无 `blob:` 魔法);不存在返回真实 `404`
|
||||
- `/characters/*`、`/User Avatars/*`、`/backgrounds/*`、`/assets/*`、`/user/images/*`、`/user/files/*` 作为子资源可直接加载
|
||||
- `/scripts/extensions/third-party/*` 的 ESM/CSS/图片/字体均可加载,未命中返回 `404`
|
||||
- 媒体 Range 契约:`/backgrounds/<file>.mp4` 的 `Range: bytes=0-1` 返回 `206` 且包含 `Content-Range`
|
||||
|
||||
任何涉及第 3/4 节契约的改动,都必须至少跑通以上 smoke tests。
|
||||
|
||||
|
|
|
|||
|
|
@ -16,23 +16,51 @@ use crate::presentation::web_resources::response_helpers::{
|
|||
|
||||
const USER_DATA_ALLOWED_METHODS: &str = "GET, HEAD, OPTIONS";
|
||||
|
||||
#[derive(Clone, Copy)]
|
||||
struct UserDataAssetRequestPolicy {
|
||||
android_webview_reapplies_range_semantics: bool,
|
||||
}
|
||||
|
||||
impl UserDataAssetRequestPolicy {
|
||||
const fn for_current_platform() -> Self {
|
||||
Self {
|
||||
android_webview_reapplies_range_semantics: cfg!(target_os = "android"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn handle_user_data_asset_web_request(
|
||||
user_dirs: &DefaultUserWebDirs,
|
||||
request: &tauri::http::Request<Vec<u8>>,
|
||||
response: &mut tauri::http::Response<Cow<'static, [u8]>>,
|
||||
) {
|
||||
handle_user_data_asset_web_request_with_policy(
|
||||
user_dirs,
|
||||
request,
|
||||
response,
|
||||
UserDataAssetRequestPolicy::for_current_platform(),
|
||||
);
|
||||
}
|
||||
|
||||
fn handle_user_data_asset_web_request_with_policy(
|
||||
user_dirs: &DefaultUserWebDirs,
|
||||
request: &tauri::http::Request<Vec<u8>>,
|
||||
response: &mut tauri::http::Response<Cow<'static, [u8]>>,
|
||||
policy: UserDataAssetRequestPolicy,
|
||||
) {
|
||||
let request_path = request.uri().path();
|
||||
if !is_user_data_asset_route(request_path) {
|
||||
return;
|
||||
}
|
||||
|
||||
handle_user_data_asset_route_request(user_dirs, request, response);
|
||||
handle_user_data_asset_route_request_with_policy(user_dirs, request, response, policy);
|
||||
}
|
||||
|
||||
fn handle_user_data_asset_route_request(
|
||||
fn handle_user_data_asset_route_request_with_policy(
|
||||
user_dirs: &DefaultUserWebDirs,
|
||||
request: &tauri::http::Request<Vec<u8>>,
|
||||
response: &mut tauri::http::Response<Cow<'static, [u8]>>,
|
||||
policy: UserDataAssetRequestPolicy,
|
||||
) {
|
||||
use tauri::http::Method;
|
||||
|
||||
|
|
@ -111,6 +139,18 @@ fn handle_user_data_asset_route_request(
|
|||
return;
|
||||
}
|
||||
|
||||
// Android WebView re-applies request range semantics to intercepted responses.
|
||||
// If we serve already-sliced bytes, non-zero ranges can become unsatisfiable and yield 416.
|
||||
//
|
||||
// However, the media pipeline still expects a 206 + Content-Range when it requests a range.
|
||||
// Workaround: return correct range headers but provide the full file bytes so WebView can
|
||||
// apply the range itself (skip `range.start` bytes in the returned stream).
|
||||
//
|
||||
// See docs/CurrentState/MediaAssetContract.md.
|
||||
let is_android_background_video = policy.android_webview_reapplies_range_semantics
|
||||
&& parsed.kind == UserDataAssetKind::Background
|
||||
&& mime_type.starts_with("video/");
|
||||
|
||||
if let Some(range_header) = request.headers().get(RANGE) {
|
||||
let header_value = match range_header.to_str() {
|
||||
Ok(value) => value,
|
||||
|
|
@ -159,71 +199,110 @@ fn handle_user_data_asset_route_request(
|
|||
}
|
||||
};
|
||||
|
||||
let range_len = match usize::try_from(range.len()) {
|
||||
Ok(value) => value,
|
||||
Err(_) => {
|
||||
if is_android_background_video && range.start != 0 {
|
||||
match std::fs::read(&asset_path) {
|
||||
Ok(bytes) => {
|
||||
respond_bytes(response, StatusCode::PARTIAL_CONTENT, bytes, &mime_type);
|
||||
response.headers_mut().insert(
|
||||
CONTENT_RANGE,
|
||||
HeaderValue::from_str(&format!(
|
||||
"bytes {}-{}/{}",
|
||||
range.start,
|
||||
range.end,
|
||||
metadata.len()
|
||||
))
|
||||
.expect("Invalid Content-Range"),
|
||||
);
|
||||
response.headers_mut().insert(
|
||||
CONTENT_LENGTH,
|
||||
HeaderValue::from_str(&range.len().to_string())
|
||||
.expect("Invalid Content-Length"),
|
||||
);
|
||||
tracing::debug!(
|
||||
"User data asset Android video range workaround hit: {}",
|
||||
parsed.relative_path_display
|
||||
);
|
||||
}
|
||||
Err(error) => {
|
||||
let status = match error.kind() {
|
||||
std::io::ErrorKind::NotFound => StatusCode::NOT_FOUND,
|
||||
_ => StatusCode::INTERNAL_SERVER_ERROR,
|
||||
};
|
||||
respond_plain_text(
|
||||
response,
|
||||
status,
|
||||
&format!("Failed to read user data asset: {}", error),
|
||||
);
|
||||
}
|
||||
}
|
||||
return;
|
||||
} else {
|
||||
let range_len = match usize::try_from(range.len()) {
|
||||
Ok(value) => value,
|
||||
Err(_) => {
|
||||
respond_plain_text(
|
||||
response,
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
"Range is too large to serve",
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let mut file = match std::fs::File::open(&asset_path) {
|
||||
Ok(value) => value,
|
||||
Err(error) => {
|
||||
respond_plain_text(
|
||||
response,
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
&format!("Failed to open user data asset: {}", error),
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
if let Err(error) = file.seek(std::io::SeekFrom::Start(range.start)) {
|
||||
respond_plain_text(
|
||||
response,
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
"Range is too large to serve",
|
||||
&format!("Failed to seek user data asset: {}", error),
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let mut file = match std::fs::File::open(&asset_path) {
|
||||
Ok(value) => value,
|
||||
Err(error) => {
|
||||
let mut bytes = vec![0u8; range_len];
|
||||
if let Err(error) = file.read_exact(&mut bytes) {
|
||||
respond_plain_text(
|
||||
response,
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
&format!("Failed to open user data asset: {}", error),
|
||||
&format!("Failed to read user data asset range: {}", error),
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
if let Err(error) = file.seek(std::io::SeekFrom::Start(range.start)) {
|
||||
respond_plain_text(
|
||||
response,
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
&format!("Failed to seek user data asset: {}", error),
|
||||
respond_bytes(response, StatusCode::PARTIAL_CONTENT, bytes, &mime_type);
|
||||
response.headers_mut().insert(
|
||||
CONTENT_RANGE,
|
||||
HeaderValue::from_str(&format!(
|
||||
"bytes {}-{}/{}",
|
||||
range.start,
|
||||
range.end,
|
||||
metadata.len()
|
||||
))
|
||||
.expect("Invalid Content-Range"),
|
||||
);
|
||||
response.headers_mut().insert(
|
||||
CONTENT_LENGTH,
|
||||
HeaderValue::from_str(&range.len().to_string()).expect("Invalid Content-Length"),
|
||||
);
|
||||
|
||||
tracing::debug!(
|
||||
"User data asset range hit: {:?}/{}",
|
||||
parsed.kind,
|
||||
parsed.relative_path_display
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let mut bytes = vec![0u8; range_len];
|
||||
if let Err(error) = file.read_exact(&mut bytes) {
|
||||
respond_plain_text(
|
||||
response,
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
&format!("Failed to read user data asset range: {}", error),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
respond_bytes(response, StatusCode::PARTIAL_CONTENT, bytes, &mime_type);
|
||||
response.headers_mut().insert(
|
||||
CONTENT_RANGE,
|
||||
HeaderValue::from_str(&format!(
|
||||
"bytes {}-{}/{}",
|
||||
range.start,
|
||||
range.end,
|
||||
metadata.len()
|
||||
))
|
||||
.expect("Invalid Content-Range"),
|
||||
);
|
||||
response.headers_mut().insert(
|
||||
CONTENT_LENGTH,
|
||||
HeaderValue::from_str(&range.len().to_string()).expect("Invalid Content-Length"),
|
||||
);
|
||||
|
||||
tracing::debug!(
|
||||
"User data asset range hit: {:?}/{}",
|
||||
parsed.kind,
|
||||
parsed.relative_path_display
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
match std::fs::read(&asset_path) {
|
||||
|
|
@ -257,7 +336,7 @@ fn handle_user_data_asset_route_request(
|
|||
mod tests {
|
||||
use super::*;
|
||||
use std::path::PathBuf;
|
||||
use tauri::http::header::CONTENT_RANGE;
|
||||
use tauri::http::header::{CONTENT_LENGTH, CONTENT_RANGE};
|
||||
|
||||
fn dirs(root: &PathBuf) -> DefaultUserWebDirs {
|
||||
DefaultUserWebDirs {
|
||||
|
|
@ -410,4 +489,86 @@ mod tests {
|
|||
Some("bytes */4")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serves_background_video_assets_with_single_range_on_non_android() {
|
||||
let temp = TempDirGuard::new("user-data-endpoint-background-video-range-non-android");
|
||||
std::fs::create_dir_all(temp.path.join("backgrounds")).expect("create backgrounds dir");
|
||||
std::fs::write(temp.path.join("backgrounds").join("a.mp4"), b"abcd").expect("write asset");
|
||||
|
||||
let request = tauri::http::Request::builder()
|
||||
.method("GET")
|
||||
.uri("/backgrounds/a.mp4")
|
||||
.header("range", "bytes=1-2")
|
||||
.body(Vec::new())
|
||||
.expect("request");
|
||||
let mut response = tauri::http::Response::new(Cow::Owned(Vec::new()));
|
||||
|
||||
handle_user_data_asset_web_request_with_policy(
|
||||
&dirs(&temp.path),
|
||||
&request,
|
||||
&mut response,
|
||||
UserDataAssetRequestPolicy {
|
||||
android_webview_reapplies_range_semantics: false,
|
||||
},
|
||||
);
|
||||
|
||||
assert_eq!(response.status(), StatusCode::PARTIAL_CONTENT);
|
||||
assert_eq!(response.body().as_ref(), b"bc");
|
||||
assert_eq!(
|
||||
response
|
||||
.headers()
|
||||
.get(CONTENT_RANGE)
|
||||
.map(|value| value.to_str().unwrap_or("")),
|
||||
Some("bytes 1-2/4")
|
||||
);
|
||||
assert_eq!(
|
||||
response
|
||||
.headers()
|
||||
.get(CONTENT_LENGTH)
|
||||
.map(|value| value.to_str().unwrap_or("")),
|
||||
Some("2")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serves_background_video_assets_with_android_range_workaround() {
|
||||
let temp = TempDirGuard::new("user-data-endpoint-background-video-range-android");
|
||||
std::fs::create_dir_all(temp.path.join("backgrounds")).expect("create backgrounds dir");
|
||||
std::fs::write(temp.path.join("backgrounds").join("a.mp4"), b"abcd").expect("write asset");
|
||||
|
||||
let request = tauri::http::Request::builder()
|
||||
.method("GET")
|
||||
.uri("/backgrounds/a.mp4")
|
||||
.header("range", "bytes=1-2")
|
||||
.body(Vec::new())
|
||||
.expect("request");
|
||||
let mut response = tauri::http::Response::new(Cow::Owned(Vec::new()));
|
||||
|
||||
handle_user_data_asset_web_request_with_policy(
|
||||
&dirs(&temp.path),
|
||||
&request,
|
||||
&mut response,
|
||||
UserDataAssetRequestPolicy {
|
||||
android_webview_reapplies_range_semantics: true,
|
||||
},
|
||||
);
|
||||
|
||||
assert_eq!(response.status(), StatusCode::PARTIAL_CONTENT);
|
||||
assert_eq!(response.body().as_ref(), b"abcd");
|
||||
assert_eq!(
|
||||
response
|
||||
.headers()
|
||||
.get(CONTENT_RANGE)
|
||||
.map(|value| value.to_str().unwrap_or("")),
|
||||
Some("bytes 1-2/4")
|
||||
);
|
||||
assert_eq!(
|
||||
response
|
||||
.headers()
|
||||
.get(CONTENT_LENGTH)
|
||||
.map(|value| value.to_str().unwrap_or("")),
|
||||
Some("2")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue