vrr/readest
2
0
Fork 0
mirror of https://github.com/readest/readest.git synced 2026-07-09 16:00:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

feat(reader): Word Wise inline vocabulary hints (#4589)
Some checks are pending
CodeQL Advanced / Analyze (actions) (push) Waiting to run
Details
CodeQL Advanced / Analyze (javascript-typescript) (push) Waiting to run
Details
CodeQL Advanced / Analyze (rust) (push) Waiting to run
Details
Publish Docker image / build (linux/amd64, ubuntu-latest) (push) Waiting to run
Details
Publish Docker image / build (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Details
Publish Docker image / merge (push) Blocked by required conditions
Details
PR checks / rust_lint (push) Waiting to run
Details
PR checks / build_web_app (push) Waiting to run
Details
PR checks / test_web_app (1) (push) Waiting to run
Details
PR checks / test_web_app (2) (push) Waiting to run
Details
PR checks / test_extensions (push) Waiting to run
Details
PR checks / build_tauri_app (push) Waiting to run
Details
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Details
Deploy to vercel on merge / build_and_deploy (push) Waiting to run
Details

Browse source 
* feat(reader): Word Wise — inline native-language vocabulary hints

Kindle-style Word Wise: a short native-language gloss renders above difficult words
as you read (always-on ruby), gated by a CEFR vocabulary-level slider (A1–C2);
tapping a glossed word opens the existing dictionary.

- Pipeline: CEFR→frequency-rank difficulty, inflection-aware gloss index, pure
  offset-aware planner (EN regex + jieba for CJK).
- Rendering: <ruby cfi-skip>…<rt cfi-inert> injected per occurrence — CFI-transparent
  (verified), so highlights/bookmarks/progress are unaffected; kept out of TTS word
  offsets and find-in-book.
- Delivery: gloss packs are version-controlled in data/wordwise/, mirrored to R2, and
  downloaded on demand into local storage (sha-verified, single-flight) when enabled.
- Settings: a Word Wise sub-page under Settings → Language (enable, level, hint
  language, per-pack download/manage, auto-download toggle).
- Build tooling: scripts/build-wordwise-data.mjs (ECDICT / CC-CEDICT+HSK / WikDict +
  FrequencyWords, with lemmatization) and scripts/sync-wordwise-r2.mjs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* data(wordwise): bundled gloss packs + manifest + attribution

13 frequency-trimmed gloss packs (en↔中文 + es/fr/de/pt/it/ru↔en, ~19 MB) generated
by build-wordwise-data.mjs from ECDICT (MIT), CC-CEDICT + HSK, and WikDict +
FrequencyWords (CC-BY-SA). Source of truth, mirrored to the CDN via `pnpm wordwise:sync`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Huang Xin 2026-06-15 13:56:00 +08:00 • committed by GitHub
parent 51fede1a0d
commit 4908245042
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
90 changed files with 6573 additions and 125 deletions
Download patch file Download diff file Expand all files Collapse all files

51
apps/readest-app/data/wordwise/ATTRIBUTION.md Normal file
View file

@ -0,0 +1,51 @@
# Word Wise data attributions
> **Note:** The committed `en-zh.json` (top 30,000 by COCA frequency) and
> `zh-en.json` (top 12,000, HSK-ranked) are frequency-trimmed derivatives generated
> from the datasets below via `scripts/build-wordwise-data.mjs`. The full source
> corpora (ECDICT ~66 MB, CC-CEDICT) are not committed — regenerate with the commands
> below. Glosses are cleaned (POS tags, `[…]` annotations, and CC-CEDICT `CL:` clauses
> stripped) and capped to the first 1–2 short senses.
The **en↔中文** packs use dedicated dictionaries (highest quality there); **all other
pairs** (es/fr/de/pt/it/ru ↔ en) use the lightweight WikDict + FrequencyWords stack.
## Sources
- **ECDICT** (English→中文 glosses, frequency ranks, inflections) — MIT License.
https://github.com/skywind3000/ECDICT
- **CC-CEDICT** (中文→English glosses) — CC-BY-SA 4.0.
https://cc-cedict.org/
- **HSK vocabulary levels** (中文 difficulty ranking) — open dataset.
https://github.com/drkameleon/complete-hsk-vocabulary
- **WikDict** (bilingual glosses for es/fr/de/pt/it/ru ↔ en) — CC-BY-SA 3.0,
derived from DBnary / Wiktionary. https://www.wikdict.com/
- **FrequencyWords** (difficulty ranking; word-frequency lists from OpenSubtitles/OPUS)
— CC-BY-SA 4.0. https://github.com/hermitdave/FrequencyWords
- **lemmatization-lists** (form→lemma mappings used to lemmatize non-English source
words, e.g. Spanish `corriendo`→`correr`) — compiled from Wiktionary + open lemma
data. https://github.com/michmech/lemmatization-lists (English form→lemma comes from
ECDICT's `exchange` field instead.)
The committed `*.json` packs are frequency-trimmed derivatives generated by
`scripts/build-wordwise-data.mjs`. Redistribution complies with the above licenses; the
CC-BY-SA packs (everything except `en-zh.json`) carry their license + attribution in each
pack's `meta`. Per-pack sizes: 0.3–2.6 MB.
## Regenerating the assets (maintainer step)
```bash
# English → 中文 (ECDICT csv) and 中文 → English (CC-CEDICT + HSK)
node scripts/build-wordwise-data.mjs en-zh /path/to/ecdict.csv 30000
node scripts/build-wordwise-data.mjs zh-en /path/to/cedict.txt /path/to/hsk.json 12000
# Any other pair (one side English) from WikDict + FrequencyWords. Needs the `sqlite3`
# CLI. Download <pair>.sqlite3 from https://download.wikdict.com/dictionaries/sqlite/2/
# and <src>_50k.txt from FrequencyWords/content/2018/<src>/.
node scripts/build-wordwise-data.mjs build-wikdict es en es_50k.txt es-en.sqlite3 20000
node scripts/build-wordwise-data.mjs build-wikdict en es en_50k.txt en-es.sqlite3 20000
# (repeat for fr/de/pt/it/ru ↔ en)
# For maximum coverage, the kaikki Wiktionary dump can be used instead of WikDict:
node scripts/build-wordwise-data.mjs build es en es_50k.txt /path/to/es-extract.jsonl 20000
```

87
apps/readest-app/data/wordwise/README.md Normal file
View file

@ -0,0 +1,87 @@
# Updating the Word Wise gloss packs
**Model:** the committed `data/wordwise/*.json` + `manifest.json` are the source of
truth. They are **not** bundled into the app — `pnpm wordwise:sync` mirrors them to the
`cdn.readest.com` R2 bucket, and the app downloads each pack on demand and **re-downloads
it automatically whenever its `sha256` changes in the manifest**. So updating data is:
regenerate → sync → commit. No app release required.
See `ATTRIBUTION.md` for the data sources + licenses.
## Prerequisites
- `sqlite3` CLI (for the WikDict pairs).
- `wrangler` logged in to Cloudflare + the R2 bucket name (for sync).
- A scratch dir for source corpora (e.g. `/tmp/ww-data`).
## 1. Fetch source corpora
```bash
mkdir -p /tmp/ww-data && cd /tmp/ww-data
# en→中文: ECDICT (MIT) — ~66 MB
curl -sL -o ecdict.csv https://raw.githubusercontent.com/skywind3000/ECDICT/master/ecdict.csv
# 中文→en: CC-CEDICT (CC-BY-SA) + HSK levels (drkameleon)
curl -sL -o cedict.txt.gz https://www.mdbg.net/chinese/export/cedict/cedict_1_0_ts_utf-8_mdbg.txt.gz && gunzip -f cedict.txt.gz
for n in $(seq 1 9); do curl -sL -o hsk-$n.json https://raw.githubusercontent.com/drkameleon/complete-hsk-vocabulary/main/wordlists/exclusive/new/$n.json; done
node -e "const fs=require('fs');let o=[];for(let n=1;n<=9;n++){try{for(const it of JSON.parse(fs.readFileSync('hsk-'+n+'.json','utf8')))if(it.simplified)o.push({simplified:it.simplified,level:n});}catch(e){}}fs.writeFileSync('hsk.json',JSON.stringify(o))"
# Other pairs: WikDict SQLite (CC-BY-SA-3.0) + FrequencyWords (CC-BY-SA-4.0)
for c in en es fr de pt it ru; do curl -sL -o ${c}_50k.txt https://raw.githubusercontent.com/hermitdave/FrequencyWords/master/content/2018/$c/${c}_50k.txt; done
for p in es-en fr-en de-en pt-en it-en ru-en en-es en-fr en-de en-pt en-ru; do curl -sL -o $p.sqlite3 https://download.wikdict.com/dictionaries/sqlite/2/$p.sqlite3; done
# Source-language lemmatization lists (michmech) — used to lemmatize X→en source words
for c in es fr de pt it ru; do curl -sL -o lemmatization-$c.txt https://raw.githubusercontent.com/michmech/lemmatization-lists/master/lemmatization-$c.txt; done
```
## 2. Generate packs (run from `apps/readest-app`)
> **Order matters:** build `en-zh` **first** — the en→X WikDict builds reuse its English
> inflection table to lemmatize (`kept`→`keep`).
```bash
cd apps/readest-app
# Flagship pairs (dedicated dictionaries — higher quality)
node scripts/build-wordwise-data.mjs en-zh /tmp/ww-data/ecdict.csv 30000
node scripts/build-wordwise-data.mjs zh-en /tmp/ww-data/cedict.txt /tmp/ww-data/hsk.json 12000
# X→en (foreign source): pass the source-language lemmatization list (6th arg) so
# inflected source words ("corriendo" -> "correr") resolve to their lemma's gloss.
for src in es fr de pt it ru; do
node scripts/build-wordwise-data.mjs build-wikdict "$src" en "/tmp/ww-data/${src}_50k.txt" "/tmp/ww-data/$src-en.sqlite3" 20000 "/tmp/ww-data/lemmatization-$src.txt"
done
# en→X (English source): lemmatized automatically via en-zh.json (build it first)
for tgt in es fr de pt ru; do
node scripts/build-wordwise-data.mjs build-wikdict en "$tgt" /tmp/ww-data/en_50k.txt "/tmp/ww-data/en-$tgt.sqlite3" 20000
done
```
- Each build writes `data/wordwise/<pair>.json` **and** regenerates `manifest.json`
(sha256 + bytes + entry count). Rebuild only the manifest with `pnpm wordwise:manifest`.
- The last CLI arg is `topN` (default 30000 for en-zh, 20000 otherwise).
- **Add a new pair** (e.g. en→ja): fetch `en-ja.sqlite3` + `en_50k.txt`, run
`build-wikdict en ja …` — it joins the manifest automatically. (ja/ko/th as a *source*
language still need a word segmenter — deferred.)
> Max-coverage alternative to WikDict (heavier): the kaikki Wiktionary dump via the
> `build <src> <tgt> <freq.txt> <wiktionary.jsonl>` mode — see `ATTRIBUTION.md`.
## 3. Sync to R2
```bash
WORDWISE_R2_BUCKET=<cdn-bucket> pnpm wordwise:sync
```
Uploads every pack (immutable cache) + `manifest.json` (5-min cache), manifest last.
## 4. Commit
```bash
git add data/wordwise && git commit -m "chore(wordwise): refresh gloss packs"
```
## 5. How clients update
On next load the app fetches `manifest.json` (≤5-min CDN cache), compares each pack's
`sha256` to the locally cached copy, and re-downloads any that changed. Nothing else to do.
## Tuning knobs
| What | Where |
| --- | --- |
| `topN` per pack | the build CLI's last arg |
| commonest-N words skipped (`skipTop`) + per-chapter render cap (`DEFAULT_CAP`) | `src/services/wordwise/{planner,…}` and `buildPack` in the build script |
| difficulty cutoffs per slider level | `src/services/wordwise/difficulty.ts` |
| gloss cleaning (POS/`[…]`/`CL:` strip, 24-char cap) | `shortGloss` in `scripts/build-wordwise-data.mjs` |

1
apps/readest-app/data/wordwise/de-en.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/en-de.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/en-es.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/en-fr.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/en-pt.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/en-ru.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/en-zh.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/es-en.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/fr-en.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/it-en.json Normal file
View file

File diff suppressed because one or more lines are too long

122
apps/readest-app/data/wordwise/manifest.json Normal file
View file

@ -0,0 +1,122 @@
{
"schemaVersion": 1,
"packs": [
{
"pair": "de-en",
"source": "de",
"target": "en",
"file": "de-en.json",
"bytes": 1938011,
"sha256": "697c7072d7808abba08e554f47efd92d5ac5555e6d4edf14bbf8a7d65697fff1",
"entries": 13806
},
{
"pair": "en-de",
"source": "en",
"target": "de",
"file": "en-de.json",
"bytes": 962032,
"sha256": "e7475f241743006278a3d771b08570f4bc9a49faa32007837e7a74461ee188e9",
"entries": 14487
},
{
"pair": "en-es",
"source": "en",
"target": "es",
"file": "en-es.json",
"bytes": 946854,
"sha256": "9b91c73c693bdade1458eb234eaa606a6490a6ec851deb229556fa4cbac21081",
"entries": 14322
},
{
"pair": "en-fr",
"source": "en",
"target": "fr",
"file": "en-fr.json",
"bytes": 933430,
"sha256": "ff7dbd0a8b6a063db4ee69bcf595b718cd8f23542860a6f8377b43e0c093abdc",
"entries": 14653
},
{
"pair": "en-pt",
"source": "en",
"target": "pt",
"file": "en-pt.json",
"bytes": 924391,
"sha256": "776ce8b30aca99856927ddc767c432d987c79efe0024729418ada9f1c261bce8",
"entries": 14209
},
{
"pair": "en-ru",
"source": "en",
"target": "ru",
"file": "en-ru.json",
"bytes": 1197025,
"sha256": "1d8e7c044d2412c6608dce7844dd0a90f40f041c95692879a9649284a55e6d6a",
"entries": 14491
},
{
"pair": "en-zh",
"source": "en",
"target": "zh",
"file": "en-zh.json",
"bytes": 2343610,
"sha256": "693a94940e07e64c332366a9033dd3f3bd86ce18b6d439fdb7074383668bb5c6",
"entries": 26729
},
{
"pair": "es-en",
"source": "es",
"target": "en",
"file": "es-en.json",
"bytes": 1764204,
"sha256": "5b4765271c693f7fe3f1f4b39beb2e2a14dc77c48ee86aac7e19fa8f6010ff64",
"entries": 10949
},
{
"pair": "fr-en",
"source": "fr",
"target": "en",
"file": "fr-en.json",
"bytes": 1898800,
"sha256": "6568130bd48cb6a18e18d90770dbb16ed286f0527bbeea42c4c88b97f71ea78b",
"entries": 12361
},
{
"pair": "it-en",
"source": "it",
"target": "en",
"file": "it-en.json",
"bytes": 1979183,
"sha256": "4536f8663ea9976b4e1e10b340989ed1d7c5a77cc5715eb705c54f9b92744f55",
"entries": 11657
},
{
"pair": "pt-en",
"source": "pt",
"target": "en",
"file": "pt-en.json",
"bytes": 2510246,
"sha256": "0be519b683c2c46c9c3f79a81c79d74e3e17e35c3ffdacf5fa8d9ec5c32a404e",
"entries": 11808
},
{
"pair": "ru-en",
"source": "ru",
"target": "en",
"file": "ru-en.json",
"bytes": 2200460,
"sha256": "edd67795bc4a30fa296ca929d8f2b6e052c024874300136c1b2b419456ee5eb8",
"entries": 8780
},
{
"pair": "zh-en",
"source": "zh",
"target": "en",
"file": "zh-en.json",
"bytes": 602685,
"sha256": "01804d83f5342ac877c1a7682affda7d663511dde44990bf883242522fef92fd",
"entries": 12000
}
]
}

1
apps/readest-app/data/wordwise/pt-en.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/ru-en.json Normal file
View file

File diff suppressed because one or more lines are too long

1
apps/readest-app/data/wordwise/zh-en.json Normal file
View file

File diff suppressed because one or more lines are too long

1606
apps/readest-app/docs/superpowers/plans/2026-06-14-word-wise.md Normal file
View file

File diff suppressed because it is too large Load diff

392
apps/readest-app/docs/superpowers/specs/2026-06-14-word-wise-design.md Normal file
View file

@ -0,0 +1,392 @@
# Word Wise — inline vocabulary hints for Readest
**Date:** 2026-06-14
**Status:** Design approved (pending spec review) → next is `writing-plans`
**Scope owner:** reader / dictionaries
---
## 1. Summary
Build the single core function of **Kindle Word Wise** into Readest: as the user reads,
a small **native-language gloss appears above difficult words**, inline in the text, with
no interaction required. A **difficulty slider** controls how many words get a hint (fewer =
only the rarest words; more = include easier words). **Tapping a glossed word opens
Readest's existing dictionary popup** for the full entry.
This is modeled on Kindle Word Wise. We build **only the inline-hint function** — explicitly
**not** SRS review decks, audio narration, or whole-book difficulty ("book fit") scoring.
### Locked product decisions
| Decision | Choice |
| --- | --- |
| Hint content | **Native-language gloss** (a short translation above the hard word) |
| Difficulty control | **Frequency slider** (Kindle-style: fewer ↔ more hints) |
| Book languages (v1) | **English + Chinese** source text (Japanese deferred — no JP analyzer) |
| Gloss data source | **Bundled offline dataset** (frequency-trimmed), lazy-loaded |
| Display | **Always-on inline** (DOM `<ruby>`), tap opens the existing dictionary popup |
| Occurrences | **Every occurrence** on the page, capped per section |
| Inert marker | **Reuse `cfi-inert`** as the single content-inert marker across CFI / search / TTS |
---
## 2. User-facing behavior
- A **Word Wise** settings panel (reader settings) with: an **Enable** toggle and a
**difficulty slider** (5 levels: *fewer hints* … *more hints*).
- When enabled and the book's text language has a bundled gloss index:
- Difficult words (rarer than the slider's threshold) render with a small muted gloss
above them, e.g. `cryptic` → `晦涩的` (EN→中文), `斟酌` → `to consider` (中文→EN).
- The gloss sits in native ruby position; the line spacing grows to fit, exactly like
Kindle. No layout overlap.
- Tapping a glossed word opens the existing dictionary popup for that word.
- When disabled, or for an unsupported book language, **no glosses render** and there is
zero effect on layout, CFI, TTS, search, or selection.
- Settings persist **per-book and globally**, using the existing `saveViewSettings` path.
---
## 3. Architecture
New, isolated units. Each has one purpose, a small interface, and is independently testable.
```
src/services/wordwise/
types.ts # WordGloss, GlossEntry, GlossOccurrence, SupportedSourceLang
glossIndex.ts # lazy-load + cache per-source-language index; lemmatize(); lookup()
difficulty.ts # pure: sliderLevel -> rank cutoff; isDifficult(rank, cutoff)
planner.ts # pure: (sectionText, sourceLang, cutoff, index) -> GlossOccurrence[]
index.ts # thin service facade used by the reader
src/app/reader/utils/
wordwiseRuby.ts # inject/unwrap <ruby cfi-skip>…<rt cfi-inert>…</rt></ruby> into a section doc
src/components/settings/
WordWisePanel.tsx # toggle + slider (mirrors TTSPanel)
scripts/
build-wordwise-data.mjs # offline data-prep: source datasets -> trimmed assets + attributions
public/wordwise/
en-zh.json # trimmed gloss index, English source -> Chinese gloss
zh-en.json # trimmed gloss index, Chinese source -> English gloss
ATTRIBUTION.md # ECDICT (MIT), CC-CEDICT (CC-BY-SA), HSK list attributions
```
### Responsibilities / boundaries
- **`planner.ts`** is the heart and is **pure** (no DOM, no async): given the plain text of a
section, the source language, the difficulty cutoff, and a gloss index, it returns a list of
`GlossOccurrence { start, end, word, gloss }` (character offsets into the section text).
It owns tokenization-dispatch (English vs CJK), inflection resolution, threshold filtering,
and the per-section occurrence cap. **All the interesting logic lives here and is unit-tested
without a browser.**
- **`glossIndex.ts`** owns data: lazy `load(sourceLang)` fetches `public/wordwise/<pair>.json`
once, builds an in-memory `Map<string, GlossEntry>` plus an inflection reverse-map
(`running → run`), and exposes `lemmatize(word)` and `lookup(lemma)`.
- **`wordwiseRuby.ts`** is the only DOM-mutating unit: given a live section `Document` and a
`GlossOccurrence[]`, it maps offsets → DOM Ranges (reusing the section's text walk) and wraps
each occurrence in `<ruby class="ww-gloss" cfi-skip>word<rt cfi-inert>gloss</rt></ruby>`. It
also `unwrap(doc)` — removes every `.ww-gloss` and `normalize()`s — so toggling/recomputing
is clean and idempotent.
- **`FoliateViewer`** orchestrates: on section `stabilized`, builds the section text, calls the
planner, then the injector, per loaded content doc; on settings change, unwraps + recomputes
or unwraps to disable. Tap handling routes clicks inside `.ww-gloss` to the dictionary popup.
---
## 4. Data: datasets, prep, format, sizes, licensing
The full source datasets are too large to ship. A **build-time prep script** produces compact,
frequency-trimmed indices that are **lazy-loaded only when the feature is enabled** (not in the
initial app bundle).
### English → 中文 (flagship): ECDICT
- **License:** MIT (redistributable with notice).
- **Full size:** ~760k entries, ~160–200 MB CSV — **not shipped raw**.
- **Fields used:** `word`, `translation` (Chinese), `frq` (COCA rank) / `bnc` (BNC rank),
`exchange` (inflections, `/`-delimited), `tag` (exam tags, optional future use).
- **Difficulty metric:** `frq` (COCA, modern usage) primary, `bnc` fallback. Lower rank = more
common = easier.
- **Trim:** top ~20k–50k headwords by `frq`, keep `{ word, frq, gloss }` where `gloss` is the
first 1–2 short Chinese senses (the `translation` field truncated to a hint-length string).
- **Inflection map:** parse `exchange` into a reverse map `{ form: lemma }` so `"running"` resolves
to `"run"` without storing every inflected form.
- **Estimated shipped size:** top-20k ≈ **1–3 MB** JSON, **< ~1 MB gzipped**.
### 中文 → English: CC-CEDICT + HSK
- **License:** CC-CEDICT is **CC-BY-SA** (ship attribution + the license; the derived asset is a
share-alike work — acceptable). HSK level lists are open (GitHub).
- **Gloss:** CC-CEDICT entry, first short English sense.
- **Difficulty metric:** HSK level (1–9) as the primary tier; word frequency (SUBTLEX-CH / Jun Da)
as a secondary ordering within/above HSK.
- **Tokenization:** existing **jieba** (`cutZh`) segments Chinese into words for lookup.
- **Trim:** top ~10k entries by frequency + full HSK list. **Estimated shipped size:** ~300 KB
uncompressed, ~100 KB gzipped.
### Japanese — **deferred** (documented gap)
No Japanese morphological analyzer is bundled (jieba covers Chinese only); JMdict + JLPT exist but
kanji-only segmentation is insufficient (particles collide). Out of v1; revisit with mecab-wasm or
a kanji-only fallback later.
### Other native languages — **deferred**
Offline bilingual data for non-中文/EN pairs is sparse. The gloss layer is intentionally pluggable
so a **cached live-translation fallback** (reusing `src/services/translators`) can be added later
for arbitrary target languages without reworking the planner or renderer.
### Asset format
`public/wordwise/<src>-<tgt>.json`:
```jsonc
{
"meta": { "source": "en", "target": "zh", "metric": "frq", "version": 1, "count": 20000 },
"entries": { "run": { "r": 312, "g": "跑；经营" }, /* ... */ }, // r = rank, g = gloss
"inflections": { "running": "run", "ran": "run", "runs": "run" }
}
```
- `glossIndex.load()` fetches once, hydrates `entries` into a `Map`, keeps `inflections` as a `Map`.
- Memory: ~20k short-string entries ≈ a few MB RAM; only when the feature is on.
---
## 5. Difficulty model & slider
- `WordWiseConfig.wordWiseLevel: number` — 5 steps, default 3. Slider label: *fewer hints ↔ more hints*.
- `difficulty.ts` maps `level → rankCutoff` **per source language**, e.g. (EN, by `frq`):
| Level | Cutoff (gloss words with rank ≥ cutoff) | Effect |
| --- | --- | --- |
| 1 | ≥ 20000 | only the rarest words |
| 2 | ≥ 12000 | |
| 3 | ≥ 7000 | balanced (default) |
| 4 | ≥ 4000 | |
| 5 | ≥ 2000 | many hints |
For 中文, the slider maps to an **HSK-level threshold** (e.g. level 1 = only HSK 7–9 / off-list;
level 5 = HSK 4+). Exact cutoffs are tuned with sample books during implementation and documented
in `difficulty.ts`.
- `isDifficult(rank, cutoff) = rank >= cutoff`. Words absent from the index (rank unknown) are
treated as difficult **only** if a gloss exists (otherwise skipped — nothing to show).
---
## 6. Rendering — DOM `<ruby>` with `cfi-inert` (verified safe)
Each difficult-word occurrence is wrapped, after section load:
```html
<ruby class="ww-gloss" cfi-skip>word<rt cfi-inert>gloss</rt></ruby>
```
### Why this is CFI-safe (verified against `packages/foliate-js/epubcfi.js`)
- `cfi-skip` on `<ruby>` → `rawChildNodes` (epubcfi.js:199–203) **splices the wrapper out**,
hoisting the word's text node up into the paragraph.
- `cfi-inert` on `<rt>` → `rawChildNodes` **removes the gloss subtree** entirely.
- `indexChildNodes` (epubcfi.js:222–235) **re-merges the now-adjacent text nodes** (`"The "` +
hoisted `"quick"` + `" fox"`) into a single chunk.
- Both directions sum across the chunk: `nodeToParts` (278–294) hoists past skip wrappers and
re-merges; `partsToNode` (262–269) resolves an offset into the multi-text-node chunk.
- **Result:** CFIs are byte-identical to the unwrapped baseline. Saved highlights, bookmarks, and
progress are unaffected, and a CFI saved with Word Wise on resolves correctly with it off
(and vice-versa). **No `epubcfi.js` change is required.**
### Layout / CSS
- Native `<ruby>` grows line-height and positions the gloss above the base word — no overlay math,
no manual line-height bump, reflows natively on resize/page-turn.
- Extend `getRubyStyles()` (or add `getWordWiseStyles()`) in `src/utils/style.ts` with a `.ww-gloss`
rule: small `rt` font (~0.5em), muted color, `user-select: none` (already present for `rt`, so the
gloss is excluded from copy). E-ink: muted color must remain legible — verify under
`[data-eink='true']`.
- Must not double-wrap: the planner skips any token already inside a book's own `<ruby>`
(furigana/pinyin) to avoid nesting.
### Injection lifecycle
- **Inject** in `FoliateViewer`'s `stabilizedHandler` for each loaded content doc (mirrors the
existing warichu relayout there). Multiview preloads adjacent sections — inject per content doc.
- **Recompute** on `wordWiseEnabled` / `wordWiseLevel` change: `unwrap(doc)` then re-inject.
- **Disable / book close:** `unwrap(doc)` (replace each `.ww-gloss` with its base text node and
`normalize()`).
- Idempotent: re-running inject first unwraps, so there is never nested or stale ruby.
### Tap-to-dictionary
- Glossed words are real `.ww-gloss` elements → natural tap targets. The existing iframe click
handler detects a target inside `.ww-gloss`, reads the base word, and opens the dictionary popup
(`setShowDictionaryPopup(true)` with the word), reusing the full existing dictionary stack.
---
## 7. Keeping the gloss invisible to TTS / search / annotation matching
The gloss text must not be read aloud, matched by find-in-book, or fold into TTS word offsets.
CFI already ignores `cfi-inert`. The remaining text walkers are all reachable **without patching
foliate-js**:
1. **Shared reject filter — `src/utils/node.ts` `createRejectFilter`.**
This helper builds the `nodeFilter` passed to foliate TTS (`TTSController.ts` → `view.initTTS`)
**and** the search `acceptNode` (`SearchBar.tsx` → `view.search`). It already rejects
`script`/`style` by default.
**Change:** reject any element with the `cfi-inert` attribute (and its subtree) **by default** —
one line, covers spoken text and find-in-book together. (`cfi-inert` is an injected, non-content
marker; no real book content uses it, so a global default is safe and also correctly excludes
foliate's own injected a11y skip-links.) Other text walkers with their own inline `acceptNode`
(`globalAnnotations.ts`, `proofread.ts`, `simplecc.ts`) run at load **before** glosses are injected
at `stabilized`, so they don't see the gloss; if any later proves to need it, it adopts the same
`closest('[cfi-inert]')` check.
2. **Edge-TTS word highlighting — `src/services/tts/wordHighlight.ts` + `TTSController.ts`.**
`prepareSpeakWords` (TTSController.ts:666) matches Edge boundary `words` against
`range.toString()`, and `getTextSubRange` (wordHighlight.ts:64–71) walks `SHOW_TEXT` with **no
filter**. Because the spoken `words` already exclude the gloss (via the filtered `nodeFilter`),
both the matching text and the sub-range walk must also exclude it or offsets drift.
**Change:** add a shared predicate `isInsideInert(node) = !!node.parentElement?.closest('[cfi-inert]')`;
skip such text nodes in `getTextSubRange`, and replace `range.toString()` with a small
`rangeTextExcludingInert(range)` that concatenates the same filtered text nodes. This keeps `text`,
the walk, and `words` mutually consistent.
3. **(optional) `packages/foliate-js/overlayer.js:88`** highlight split-range `acceptNode` — reject
`cfi-inert` so a highlight drawn across a glossed word doesn't extend over the gloss box. Cosmetic;
include only if it shows in testing.
**Net foliate-js change: none required for correctness** (optional overlayer cosmetic only). All
isolation is readest-side.
---
## 8. Settings & persistence wiring
- **`src/types/book.ts`** — add `WordWiseConfig { wordWiseEnabled: boolean; wordWiseLevel: number }`
and include it in the `ViewSettings` union.
- **`src/services/constants.ts`** — `DEFAULT_WORD_WISE_CONFIG = { wordWiseEnabled: false, wordWiseLevel: 3 }`
and spread into the default view settings object.
- **Store** — no change: `getViewSettings`/`setViewSettings` (`readerStore.ts:331–362`) and the
global+per-book merge already handle arbitrary `ViewSettings` keys; `saveViewSettings`
(`src/helpers/settings.ts`) persists and applies.
- **`WordWisePanel.tsx`** — mirrors `TTSPanel`: `BoxedList` + `SettingsSwitchRow` (enable) +
slider (`NumberInput` 1–5, or the existing slider primitive) for level. E-ink-correct primitives.
- **`SettingsDialog.tsx`** — register a `WordWise` tab (icon + label) and render `WordWisePanel`.
- **`FoliateViewer.tsx`** — add `wordWiseEnabled` + `wordWiseLevel` to the deps of the
settings-application effect; recompute/unwrap accordingly.
- **i18n** — key-as-content `_()` for all UI strings; scanner extracts them.
---
## 9. Data flow
```
settings (enabled, level) ──┐
book text language ─────────┤
▼
section 'stabilized' ─▶ build section text ─▶ planner(text, srcLang, cutoff, index)
│ (tokenize EN/CJK, lemmatize,
│ lookup, threshold, cap)
▼
GlossOccurrence[] ─▶ wordwiseRuby.inject(doc)
│ (offsets→Ranges→<ruby>)
▼
glosses visible
tap on .ww-gloss ─────────────────────────────────────────▶ dictionary popup(word)
settings change / disable ────────────────────────────────▶ wordwiseRuby.unwrap(doc) [+ re-inject]
```
---
## 10. Performance & limits
- Index is in-memory after one lazy fetch; lookups are O(1) sync.
- One tokenize+lookup pass per section, only for **loaded** content docs.
- **Per-section occurrence cap** (e.g. 200) to bound DOM growth; `log()`/`console.warn` when capped
(no silent truncation).
- Glossing every occurrence (Kindle-faithful); the cap is the only limiter.
- jieba init is async and already used elsewhere; planner treats "jieba not ready" as "no CJK glosses
yet" and the `stabilized` re-run picks them up once ready.
---
## 11. Error handling / edge cases
- Index fetch fails → feature no-ops, single `console.warn`; reader unaffected.
- Unsupported book language (no index for source) → no glosses; the panel may show a one-line note.
- Native-language mismatch (e.g. a non-Chinese user reading English): v1 only has EN→中文, so glosses
show Chinese; this is acceptable for v1's audience and the panel notes the active pair. (Generalized
via the deferred live-translation fallback.)
- Book already uses `<ruby>` → skip those tokens (no nesting).
- Pre-paginated / fixed-layout books → skip (no reflow room); gloss only in reflowable EPUB/text.
- Vertical writing mode → v1 may disable Word Wise (ruby placement in vertical text is an edge case);
decided during implementation, documented if disabled.
---
## 12. Testing (test-first)
Write failing tests first, then implement.
**Unit (vitest, no browser):**
- `planner.test.ts` — offsets correctness; threshold filtering by cutoff; inflection mapping
(`running→run`); English whitespace tokenization; CJK path with a mocked `cutZh`; skip-inside-ruby;
per-section cap behavior.
- `difficulty.test.ts` — `level→cutoff` mapping per language; `isDifficult` boundaries.
- `glossIndex.test.ts` — `lemmatize`/`lookup`/miss against a fixture index; lazy-load caches once.
- `epubcfi-ruby-inline.test.ts` — **the key correctness guarantee.** Mirror
`epubcfi-inert.test.ts`/`epubcfi-skip.test.ts` but for **mid-text inline** ruby with text on both
sides: `<p>The <ruby cfi-skip>quick<rt cfi-inert>x</rt></ruby> fox</p>` must produce CFIs identical
to `<p>The quick fox</p>` in both `fromRange` and `toRange`. (Existing tests cover block wrappers and
prepended skip-links only.)
- TTS gloss alignment — extend `tts-word-highlight.test.ts`: with a glossed sentence, the
`cfi-inert`-skipping `getTextSubRange` + `rangeTextExcludingInert` keep word ranges aligned with the
gloss-free boundary `words`.
- `createRejectFilter` — rejects `[cfi-inert]` subtrees (search/TTS reuse).
**Browser/integration (Playwright):**
- glosses render above difficult words when enabled; toggle off removes them and restores layout;
tap on a glossed word opens the dictionary popup; an annotation made with Word Wise on still anchors
correctly with it off (CFI regression guard).
**Verification gates (from `.agents/rules/verification.md`):** `pnpm test`, `pnpm lint`. Rust/Lua
gates not applicable (no `src-tauri`/koplugin changes expected).
---
## 13. File change list
**New**
- `src/services/wordwise/{types,glossIndex,difficulty,planner,index}.ts`
- `src/app/reader/utils/wordwiseRuby.ts`
- `src/components/settings/WordWisePanel.tsx`
- `scripts/build-wordwise-data.mjs`
- `public/wordwise/{en-zh.json,zh-en.json,ATTRIBUTION.md}`
- tests listed in §12
**Edited**
- `src/types/book.ts` — `WordWiseConfig` + `ViewSettings` union
- `src/services/constants.ts` — `DEFAULT_WORD_WISE_CONFIG` + default merge
- `src/components/settings/SettingsDialog.tsx` — register panel
- `src/app/reader/components/FoliateViewer.tsx` — inject/unwrap lifecycle + settings deps + tap routing
- `src/utils/style.ts` — `.ww-gloss` ruby styling
- `src/utils/node.ts` — `createRejectFilter` rejects `[cfi-inert]` by default
- `src/services/tts/wordHighlight.ts` — skip `[cfi-inert]`; `rangeTextExcludingInert`
- `src/services/tts/TTSController.ts` — use the inert-excluding text for word matching
- (optional) `packages/foliate-js/overlayer.js` — reject `cfi-inert` in highlight split
---
## 14. Phasing
1. **Data prep + index** — `build-wordwise-data.mjs`, ship `en-zh.json`; `glossIndex` + `difficulty`
+ `planner` with full unit tests (no UI yet).
2. **Rendering** — `wordwiseRuby` inject/unwrap; CFI inline-ruby test; `createRejectFilter` +
`wordHighlight` inert handling with TTS test; `.ww-gloss` CSS.
3. **Wiring** — `WordWiseConfig` + defaults + `WordWisePanel` + `SettingsDialog`; `FoliateViewer`
lifecycle + tap-to-dictionary; browser integration tests.
4. **CJK** — `zh-en.json` (CC-CEDICT + HSK), jieba path in planner, 中文 difficulty mapping.
**Deferred (out of v1):** Japanese; non-中文/EN native targets (live-translation fallback); SRS/audio/
book-fit.
---
## 15. Open questions / risks
- **Inline-ruby CFI transparency** is verified by reading the code but only block-wrapper cases are
currently tested upstream — the new `epubcfi-ruby-inline.test.ts` is the gate that must pass before
relying on it. *(Highest-priority risk; fully mitigated by the test.)*
- **Gloss truncation quality** (ECDICT `translation` → hint-length string) needs tuning against real
books to stay short but useful.
- **Difficulty cutoffs** per level are first-cut; tune with sample EPUBs during phase 1.
- **Default-rejecting `cfi-inert` in `createRejectFilter`** is a behavior change to a shared helper;
confirm no current caller relies on walking `cfi-inert` skip-links (they shouldn't — those are
injected non-content).

248
apps/readest-app/docs/superpowers/specs/2026-06-14-wordwise-r2-delivery.md Normal file
View file

@ -0,0 +1,248 @@
# Word Wise — runtime gloss-pack delivery (R2) + Language-panel placement
**Date:** 2026-06-14 · **Status:** Design (pending review) · Branch `feat/word-wise`
**Extends:** `2026-06-14-word-wise-design.md` (the gloss/render/CFI pipeline is unchanged).
---
## 1. Goal
Stop bundling gloss packs into the app. Keep them version-controlled in-repo as the source of
truth, mirror them to `cdn.readest.com` (R2), and **download on demand** into durable local
storage the first time a (book-language → hint-language) pair is needed. Make the pipeline
**multi-pair** so tiers 0–2 ship as data drops now and tier 3 later, and move the Word Wise UI
into **Settings → Language** with an explicit **hint-language** selector.
This PR delivers the **infrastructure + generalization**, routed end-to-end through the existing
EN↔中文 packs. Adding each further pair (es→en, en→es, fr→en, …) is then just "generate a pack +
manifest entry + sync" — no code change.
### Locked decisions
| | |
| --- | --- |
| Delivery | `https://cdn.readest.com/wordwise/…` (direct cross-origin fetch) |
| Download trigger | Auto-download with progress; best-effort prompt on metered connection; a global "auto-download" toggle |
| Hint language | Explicit selector (default = app UI language), limited to targets available for the book's source language |
| UI placement | Settings → Language → **Word Wise** `NavigationRow` → sub-page |
| Local storage | Durable `'Data'` base (not evictable `'Cache'`); managed in the Word Wise sub-page |
---
## 2. Repo layout — source of truth, never bundled
Move packs out of `public/wordwise/` (which is bundled into both the web deploy and the Tauri
installer) into a committed, **non-bundled** directory referenced only by build/sync scripts:
```
apps/readest-app/data/wordwise/ # committed; NOT under public/ or src/
manifest.json # the index of available packs
en-zh.v<hash>.json # content-hashed pack files (GlossIndexData shape)
zh-en.v<hash>.json
ATTRIBUTION.md
```
`data/wordwise/` is consumed by `build-wordwise-data.mjs` (writes packs + manifest) and
`sync-wordwise-r2.mjs` (uploads to R2). **No `*.json` under `public/wordwise/` ships** — delete it.
(`data/` is already used for committed non-bundled content like `src/data/demo`, but keep these at
the app root `data/`, outside `src/`, so bundlers never import them.)
### `manifest.json`
```jsonc
{
"schemaVersion": 1,
"packs": [
{ "pair": "en-zh", "source": "en", "target": "zh",
"file": "en-zh.v8f3a1.json", "bytes": 2622655, "sha256": "8f3a1…", "entries": 30000 }
// … one per available (source→target) pair
]
}
```
- `file` is content-hash-versioned → immutable URLs, trivial cache-busting.
- The app downloads a pack iff its local copy's recorded sha256 ≠ the manifest's.
---
## 3. Sync to R2
`scripts/sync-wordwise-r2.mjs` uploads `data/wordwise/*` (packs + `manifest.json`) to the
`cdn.readest.com`-backed bucket under `/wordwise/`. Run manually and in CI on release.
- Tooling: `wrangler r2 object put` (or the S3-compatible API). Bucket name + creds via env
(`WORDWISE_R2_BUCKET`, Cloudflare token) — **not** the app's `wrangler.toml` (that bucket is the
Next inc-cache; the CDN bucket is separate). Pack files use `cache-control: public, max-age=31536000, immutable`; `manifest.json` uses a short max-age (e.g. 300s) so new packs surface quickly.
- A `pnpm wordwise:sync` script wraps it.
---
## 4. Runtime: manifest → download → cache → load
New `src/services/wordwise/glossPacks.ts` (replaces the bundled-`fetch` path in `glossIndex.ts`):
```ts
export const WORDWISE_CDN_BASE = 'https://cdn.readest.com/wordwise';
export interface WordWisePack {
pair: string; source: string; target: string;
file: string; bytes: number; sha256: string; entries: number;
}
export interface WordWiseManifest { schemaVersion: number; packs: WordWisePack[]; }
// Session-cached; also persisted to 'Data' (manifest.json) for offline reuse.
fetchManifest(appService, opts?): Promise<WordWiseManifest | null>
// pick the pack for (source → hint) or null if none published.
resolvePack(manifest, source, hint): WordWisePack | null
// Ensure the pack file is in local 'Data'. Returns its stored path, or null.
// - if exists locally with matching sha → reuse (no network)
// - else download WORDWISE_CDN_BASE/<file> with onProgress, verify sha256, write to 'Data'
// - single-flight per pair (no concurrent double download)
ensurePack(appService, pack, onProgress?): Promise<string | null>
// High-level: manifest → resolvePack → ensurePack → readFile('Data') → GlossIndex.fromData
loadGlossIndex(appService, source, hint, onProgress?): Promise<GlossIndex | null>
```
- **Download mechanism:** reuse `downloadFile`/`webDownload`/`tauriDownload` from `src/libs/storage.ts`
with `onProgress: ProgressHandler` (`{progress,total,transferSpeed}` from `utils/transfer.ts`).
Store via `appService.writeFile(file, 'Data', arrayBuffer)`; check `appService.exists(file,'Data')`;
delete via `appService.removeFile`. Web maps `'Data'`→IndexedDB (`webAppService.resolvePath`), so a
~3 MB JSON persists across sessions; Tauri writes to the app data dir.
- **Integrity:** verify sha256 of the downloaded bytes against the manifest before committing the
write (discard + warn on mismatch).
- **Offline:** if a pack is already local, everything works offline. If not local and offline →
`null` (no glosses) + a one-time toast ("Word Wise data unavailable offline").
- **Cache invalidation:** store a tiny sidecar (`<pair>.meta.json` or a row) recording the sha of the
local pack; re-download when the manifest sha differs.
### `glossIndex.ts` / `wordwiseSection.ts` changes
- `getGlossIndex(lang, baseUrl)` → removed; callers use `loadGlossIndex(appService, source, hint, onProgress)`.
- `refreshSectionGlosses(doc, viewSettings, bookLang)` gains access to `appService` and the hint
language (`viewSettings.wordWiseHintLang`), resolves `source = toWordWiseSource(bookLang)` and
`hint = viewSettings.wordWiseHintLang || appUILang`, calls `loadGlossIndex(...)`. The generation
guard, jieba gate, planner call, and DOM apply are unchanged.
- The cache in `glossIndex` becomes keyed by `pair` (source-hint), not just source.
---
## 5. Generalizing beyond en/zh (so tiers 1–3 are data-only)
- **Source languages:** `WordWiseSourceLang` broadens from `'en'|'zh'` to a general ISO-639-1 string.
A source is *usable* iff (a) the manifest has a pack for (source→hint) **and** (b) we can tokenize
it: **Latin/space-delimited** (en, es, fr, de, pt, it, …) via the existing regex tokenizer, or
**Chinese** via jieba. Japanese/Korean are **blocked** until a segmenter ships (tier 3) — gate
with a `canTokenize(source)` helper; unsupported source → no glosses (+ note).
- **Difficulty cutoffs (`difficulty.ts`):** generalize `getRankCutoff(lang, level)` to use a shared
**frequency-scale** table for all frequency-ranked sources (en + other Latin langs all use a
frequency rank), and the existing **HSK-scale** table only for `zh`. (Also fixes the current
zh/cutoff mismatch found while testing: align the zh cutoffs to the HSK-derived ranks the build
script emits, `level×3000`.)
- **Planner:** already routes `zh → jieba`, everything else → Latin tokenizer — no change beyond the
broadened type.
---
## 6. Hint-language selector + Language-panel placement
- **Move the UI into `LangPanel.tsx`** (translation already lives there). Remove the dedicated
`WordWise` tab: delete it from `SettingsDialog` `tabConfig` + render switch, drop `'WordWise'` from
`SettingsPanelType`, remove the `panelIcons` entry in `commandRegistry.ts`, and remove the
`PiTranslate` line from the `command-registry-extended.test.ts` mock. (Net deletion.)
- Add a **`NavigationRow` "Word Wise"** in `LangPanel` opening a sub-page that hosts the existing
`WordWisePanel` content, expanded with the hint-language selector + downloads. (Sub-page over
inline because the panel is already dense and Word Wise now has enable + slider + hint-lang +
per-pack download/manage.)
- **Hint-language selector:** a `SettingsSelect` mirroring the "Translate To" row — options from
`getLangOptions(TRANSLATED_LANGS)` filtered to targets the manifest offers for the current book's
source language; value persisted as `viewSettings.wordWiseHintLang` via `saveViewSettings`.
Default when unset = app UI language (`i18n.language` / `getLocale()`), falling back to
`translateTargetLang`.
- **Download/manage UI:** under the selector, show the resolved pack with its size and state —
"Download (2.6 MB)" / radial-progress while downloading (mirror `PublicationView`) / "Downloaded ·
Delete". `removeFile` on delete. Success/failure via `eventDispatcher.dispatch('toast', …)`.
- **Auto-download:** new global `settings.globalReadSettings.wordWiseAutoDownload` (default `true`).
When enabling Word Wise / opening a book whose pack isn't local: if auto-download is on and the
connection isn't detectably metered, fetch silently with progress; otherwise surface the
"Download (size)" affordance. Metered detection is **best-effort** via the Network Information API
(`navigator.connection?.type === 'cellular'` / `saveData`) where available — absent on iOS/Tauri,
so it simply falls through to auto.
### `WordWiseConfig` additions (`types/book.ts` + `constants.ts`)
```ts
export interface WordWiseConfig {
wordWiseEnabled: boolean;
wordWiseLevel: number; // 1..5 (existing)
wordWiseHintLang: string; // '' = auto (app UI language)
}
// DEFAULT: { wordWiseEnabled:false, wordWiseLevel:3, wordWiseHintLang:'' }
```
(`wordWiseAutoDownload` is a global read-setting, not per-book.)
---
## 7. Data flow
```
enable WW / open book
→ source = toWordWiseSource(book.primaryLanguage); hint = viewSettings.wordWiseHintLang || appLang
→ loadGlossIndex(appService, source, hint, onProgress):
manifest (session/'Data' cache) → resolvePack(source,hint)
→ exists in 'Data' & sha matches → read → GlossIndex
→ else (auto && !metered) download→verify→write('Data')→read→GlossIndex
→ else expose "Download (size)" in the panel
→ planGlosses → applyGlosses (unchanged)
```
---
## 8. Testing (test-first)
- `glossPacks.test.ts` — `resolvePack` selection; `ensurePack` (a) reuses a present matching-sha
local file without network, (b) downloads+verifies+writes when absent, (c) rejects on sha
mismatch, (d) single-flights concurrent calls — using a fake `appService` (in-memory exists/read/
write) and a stubbed downloader.
- `difficulty.test.ts` — extend: generalized frequency table applies to a non-en Latin source;
zh cutoffs aligned to the build script's `level×3000` scale.
- Manifest round-trip: `build-wordwise-data.mjs` emits a `manifest.json` whose sha256/bytes match the
written pack (unit test on the manifest-writing helper with a synthetic pack).
- Existing planner/ruby/CFI/TTS tests unchanged and green.
- Browser: enabling Word Wise with a stubbed CDN serves a pack → glosses render; delete removes the
local file and glosses stop.
- Full `pnpm test` + `pnpm lint` green.
---
## 9. File change list
**New:** `src/services/wordwise/glossPacks.ts` (+ test); `scripts/sync-wordwise-r2.mjs`;
`apps/readest-app/data/wordwise/{manifest.json, *.json, ATTRIBUTION.md}`; a `WordWiseSubPage`/panel
under `src/components/settings/`.
**Modified:** `src/services/wordwise/{glossIndex.ts → folded into glossPacks, types.ts, difficulty.ts}`;
`src/app/reader/utils/wordwiseSection.ts`; `src/app/reader/components/FoliateViewer.tsx` (pass
`appService` + hint lang; route progress to a toast/indicator); `src/types/book.ts` +
`src/services/constants.ts` (`wordWiseHintLang`, `WORDWISE_CDN_BASE`, default + `wordWiseAutoDownload`);
`src/components/settings/LangPanel.tsx` (NavigationRow + sub-page mount + hint-lang selector +
download/manage); `src/components/settings/SettingsDialog.tsx` + `src/services/commandRegistry.ts` +
`command-registry-extended.test.ts` (remove the dedicated tab); `scripts/build-wordwise-data.mjs`
(emit `data/wordwise/` + `manifest.json` with sha/bytes/hashed filenames); delete `public/wordwise/*`.
---
## 10. Phasing
1. **Manifest + build/sync**: build script writes `data/wordwise/` + `manifest.json` (hash/bytes);
`sync-wordwise-r2.mjs`; delete `public/wordwise/`.
2. **Runtime download/cache** (`glossPacks.ts`) + refactor `glossIndex`/`wordwiseSection`/FoliateViewer
to load via `appService` from `'Data'`, downloading on demand; tests.
3. **Generalization**: broaden source langs + `canTokenize` gate + generalized difficulty cutoffs.
4. **UI move**: LangPanel NavigationRow + Word Wise sub-page + hint-language selector + download/manage
+ auto-download toggle; remove the dedicated tab.
## 11. Risks / notes
- **Metered detection is unreliable** cross-platform → "prompt on cellular" is best-effort; the
always-visible size + the global auto-download toggle are the real guardrails.
- **CDN base is hardcoded** (no env precedent) → add a single `WORDWISE_CDN_BASE` constant.
- **`'Data'` durability**: packs survive cache clears (good for offline); managed only via the Word
Wise sub-page (not the generic Manage Cache), so deletion is explicit.
- **Out of scope:** generating tier-1/2/3 *data* (es/fr/de/… and ja/ko segmenters) — this PR ships the
infra + en↔zh through it; further pairs are data drops + a manifest entry.

2
apps/readest-app/package.json
View file

@ -80,6 +80,8 @@
"patch-build-webpack": "if [ \"$(uname)\" = \"Darwin\" ]; then sed -i '' 's/next build\"/next build --webpack\"/' package.json; else sed -i 's/next build\"/next build --webpack\"/' package.json; fi",
"restore-build-original": "if [ \"$(uname)\" = \"Darwin\" ]; then sed -i '' 's/next build --webpack\"/next build\"/' package.json; else sed -i 's/next build --webpack\"/next build\"/' package.json; fi",
"update-metadata": "bash ./scripts/sync-release-notes.sh release-notes.json ../../data/metainfo/appdata.xml",
"wordwise:manifest": "node scripts/build-wordwise-data.mjs manifest",
"wordwise:sync": "node scripts/sync-wordwise-r2.mjs",
"check:optional-chaining": "count=$(grep -rno '\\?\\.[a-zA-Z_$]' .next/static/chunks/* out/_next/static/chunks/* | wc -l); if [ \"$count\" -gt 0 ]; then echo '❌ Optional chaining found in output!'; exit 1; else echo '✅ No optional chaining found.'; fi",
"check:translations": "count=$(grep -rno '__STRING_NOT_TRANSLATED__' public/locales/* | wc -l); if [ \"$count\" -gt 0 ]; then echo '❌ Untranslated strings found!'; exit 1; else echo '✅ All strings translated.'; fi",
"check:lookbehind-regex": "count=$(grep -rnoE '\\(\\?<[!=]' .next/static/chunks/* out/_next/static/chunks/* | wc -l); if [ \"$count\" -gt 0 ]; then echo '❌ Lookbehind regex found in output!'; exit 1; else echo '✅ No lookbehind regex found.'; fi",

21
apps/readest-app/public/locales/ar/translation.json
View file

@ -1862,5 +1862,24 @@
"Play audio": "تشغيل الصوت",
"Resume audio": "استئناف متابعة الصوت",
"Following audio": "يتابع الصوت",
" · estimated": " · تقديري"
" · estimated": " · تقديري",
"Nightly Builds (Unstable)": "إصدارات ليلية (غير مستقرة)",
"Early daily builds; may be unstable": "إصدارات يومية مبكرة؛ قد تكون غير مستقرة",
"Downloading Word Wise data…": "جارٍ تنزيل بيانات Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "إظهار تلميح قصير بلغتك الأم فوق الكلمات الصعبة.",
"Word Wise data downloaded": "تم تنزيل بيانات Word Wise",
"Failed to download Word Wise data": "فشل تنزيل بيانات Word Wise",
"Word Wise data removed": "تمت إزالة بيانات Word Wise",
"Data pack": "حزمة البيانات",
"Open a book to manage its data pack.": "افتح كتابًا لإدارة حزمة بياناته.",
"No data available for this language pair yet.": "لا تتوفر بيانات لهذا الزوج اللغوي بعد.",
"Download {{size}}": "تنزيل {{size}}",
"Enable Word Wise": "تفعيل Word Wise",
"Vocabulary level": "مستوى المفردات",
"Words above your level get a hint": "تحصل الكلمات الأعلى من مستواك على تلميح",
"Hint Language": "لغة التلميح",
"Data": "البيانات",
"Download data packs automatically when needed.": "تنزيل حزم البيانات تلقائيًا عند الحاجة.",
"Failed to check for updates": "فشل التحقق من التحديثات"
}

21
apps/readest-app/public/locales/bn/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "অডিও চালান",
"Resume audio": "আবার অনুসরণ করুন",
"Following audio": "অডিও অনুসরণ করছে",
" · estimated": " · আনুমানিক"
" · estimated": " · আনুমানিক",
"Nightly Builds (Unstable)": "নাইটলি বিল্ড (অস্থিতিশীল)",
"Early daily builds; may be unstable": "প্রাথমিক দৈনিক বিল্ড; অস্থিতিশীল হতে পারে",
"Downloading Word Wise data…": "Word Wise ডেটা ডাউনলোড হচ্ছে…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "কঠিন শব্দের উপরে মাতৃভাষায় একটি সংক্ষিপ্ত ইঙ্গিত দেখান।",
"Word Wise data downloaded": "Word Wise ডেটা ডাউনলোড হয়েছে",
"Failed to download Word Wise data": "Word Wise ডেটা ডাউনলোড করতে ব্যর্থ",
"Word Wise data removed": "Word Wise ডেটা সরানো হয়েছে",
"Data pack": "ডেটা প্যাক",
"Open a book to manage its data pack.": "এর ডেটা প্যাক পরিচালনা করতে একটি বই খুলুন।",
"No data available for this language pair yet.": "এই ভাষা জোড়ার জন্য এখনও কোনো ডেটা উপলব্ধ নেই।",
"Download {{size}}": "{{size}} ডাউনলোড করুন",
"Enable Word Wise": "Word Wise চালু করুন",
"Vocabulary level": "শব্দভান্ডার স্তর",
"Words above your level get a hint": "আপনার স্তরের উপরের শব্দগুলির জন্য একটি ইঙ্গিত দেখানো হয়",
"Hint Language": "ইঙ্গিতের ভাষা",
"Data": "ডেটা",
"Download data packs automatically when needed.": "প্রয়োজনে স্বয়ংক্রিয়ভাবে ডেটা প্যাক ডাউনলোড করুন।",
"Failed to check for updates": "আপডেট পরীক্ষা করতে ব্যর্থ"
}

21
apps/readest-app/public/locales/bo/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "སྒྲ་གདངས་གཏོང་བ།",
"Resume audio": "རྗེས་འདེད་མུ་མཐུད།",
"Following audio": "སྒྲ་གདངས་རྗེས་འདེད་བྱེད་བཞིན་པ།",
" · estimated": " · ཚོད་དཔག"
" · estimated": " · ཚོད་དཔག",
"Nightly Builds (Unstable)": "ཉིན་ལྟར་བཟོ་སྐྲུན། (བརྟན་པོ་མིན)",
"Early daily builds; may be unstable": "ཉིན་ལྟར་བཟོ་སྐྲུན་སྔ་མོ། བརྟན་པོ་མེད་སྲིད།",
"Downloading Word Wise data…": "Word Wise གཞི་གྲངས་ཕབ་ལེན་བྱེད་བཞིན་པ།…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "ཚིག་དཀའ་མོའི་སྟེང་དུ་རང་སྐད་ནང་མཚོན་བྱེད་ཐུང་ངུ་ཞིག་སྟོན།",
"Word Wise data downloaded": "Word Wise གཞི་གྲངས་ཕབ་ལེན་ཟིན།",
"Failed to download Word Wise data": "Word Wise གཞི་གྲངས་ཕབ་ལེན་མ་ཐུབ།",
"Word Wise data removed": "Word Wise གཞི་གྲངས་བསུབས་ཟིན།",
"Data pack": "གཞི་གྲངས་ཐུམ་སྒྲིལ།",
"Open a book to manage its data pack.": "དེའི་གཞི་གྲངས་ཐུམ་སྒྲིལ་དོ་དམ་བྱེད་པར་དཔེ་ཆ་ཞིག་ཁ་ཕྱེ།",
"No data available for this language pair yet.": "སྐད་ཡིག་ཟུང་འདིའི་ཆེད་ད་དུང་གཞི་གྲངས་མེད།",
"Download {{size}}": "{{size}} ཕབ་ལེན།",
"Enable Word Wise": "Word Wise སྤྱོད་རུང་བཟོ།",
"Vocabulary level": "ཚིག་མཛོད་རིམ་པ།",
"Words above your level get a hint": "ཁྱེད་ཀྱི་རིམ་པ་ལས་མཐོ་བའི་ཚིག་ལ་མཚོན་བྱེད་ཐོབ།",
"Hint Language": "མཚོན་བྱེད་སྐད་ཡིག",
"Data": "གཞི་གྲངས།",
"Download data packs automatically when needed.": "དགོས་མཁོ་བྱུང་སྐབས་གཞི་གྲངས་ཐུམ་སྒྲིལ་རང་འགུལ་གྱིས་ཕབ་ལེན་བྱེད།",
"Failed to check for updates": "གསར་སྒྱུར་ཞིབ་བཤེར་མ་ཐུབ།"
}

21
apps/readest-app/public/locales/de/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "Audio abspielen",
"Resume audio": "Wieder folgen",
"Following audio": "Folgt dem Audio",
" · estimated": " · geschätzt"
" · estimated": " · geschätzt",
"Nightly Builds (Unstable)": "Nightly-Builds (instabil)",
"Early daily builds; may be unstable": "Frühe tägliche Builds; können instabil sein",
"Downloading Word Wise data…": "Word-Wise-Daten werden heruntergeladen…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Zeigt einen kurzen Hinweis in der Muttersprache über schwierigen Wörtern an.",
"Word Wise data downloaded": "Word-Wise-Daten heruntergeladen",
"Failed to download Word Wise data": "Word-Wise-Daten konnten nicht heruntergeladen werden",
"Word Wise data removed": "Word-Wise-Daten entfernt",
"Data pack": "Datenpaket",
"Open a book to manage its data pack.": "Öffne ein Buch, um sein Datenpaket zu verwalten.",
"No data available for this language pair yet.": "Für dieses Sprachpaar sind noch keine Daten verfügbar.",
"Download {{size}}": "{{size}} herunterladen",
"Enable Word Wise": "Word Wise aktivieren",
"Vocabulary level": "Vokabelniveau",
"Words above your level get a hint": "Wörter über deinem Niveau erhalten einen Hinweis",
"Hint Language": "Hinweissprache",
"Data": "Daten",
"Download data packs automatically when needed.": "Datenpakete bei Bedarf automatisch herunterladen.",
"Failed to check for updates": "Suche nach Updates fehlgeschlagen"
}

21
apps/readest-app/public/locales/el/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "Αναπαραγωγή ήχου",
"Resume audio": "Συνέχιση ήχου",
"Following audio": "Ακολουθεί τον ήχο",
" · estimated": " · κατ' εκτίμηση"
" · estimated": " · κατ' εκτίμηση",
"Nightly Builds (Unstable)": "Νυχτερινές εκδόσεις (ασταθείς)",
"Early daily builds; may be unstable": "Πρώιμες ημερήσιες εκδόσεις· ενδέχεται να είναι ασταθείς",
"Downloading Word Wise data…": "Λήψη δεδομένων Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Εμφάνιση σύντομης υπόδειξης στη μητρική σας γλώσσα πάνω από δύσκολες λέξεις.",
"Word Wise data downloaded": "Τα δεδομένα Word Wise λήφθηκαν",
"Failed to download Word Wise data": "Αποτυχία λήψης δεδομένων Word Wise",
"Word Wise data removed": "Τα δεδομένα Word Wise αφαιρέθηκαν",
"Data pack": "Πακέτο δεδομένων",
"Open a book to manage its data pack.": "Ανοίξτε ένα βιβλίο για να διαχειριστείτε το πακέτο δεδομένων του.",
"No data available for this language pair yet.": "Δεν υπάρχουν ακόμη διαθέσιμα δεδομένα για αυτό το ζεύγος γλωσσών.",
"Download {{size}}": "Λήψη {{size}}",
"Enable Word Wise": "Ενεργοποίηση Word Wise",
"Vocabulary level": "Επίπεδο λεξιλογίου",
"Words above your level get a hint": "Οι λέξεις πάνω από το επίπεδό σας λαμβάνουν υπόδειξη",
"Hint Language": "Γλώσσα υπόδειξης",
"Data": "Δεδομένα",
"Download data packs automatically when needed.": "Αυτόματη λήψη πακέτων δεδομένων όταν χρειάζεται.",
"Failed to check for updates": "Αποτυχία ελέγχου για ενημερώσεις"
}

21
apps/readest-app/public/locales/es/translation.json
View file

@ -1763,5 +1763,24 @@
"Play audio": "Reproducir audio",
"Resume audio": "Reanudar audio",
"Following audio": "Siguiendo el audio",
" · estimated": " · estimado"
" · estimated": " · estimado",
"Nightly Builds (Unstable)": "Versiones nightly (inestables)",
"Early daily builds; may be unstable": "Versiones diarias anticipadas; pueden ser inestables",
"Downloading Word Wise data…": "Descargando datos de Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Muestra una breve ayuda en tu idioma encima de las palabras difíciles.",
"Word Wise data downloaded": "Datos de Word Wise descargados",
"Failed to download Word Wise data": "No se pudieron descargar los datos de Word Wise",
"Word Wise data removed": "Datos de Word Wise eliminados",
"Data pack": "Paquete de datos",
"Open a book to manage its data pack.": "Abre un libro para gestionar su paquete de datos.",
"No data available for this language pair yet.": "Aún no hay datos disponibles para este par de idiomas.",
"Download {{size}}": "Descargar {{size}}",
"Enable Word Wise": "Activar Word Wise",
"Vocabulary level": "Nivel de vocabulario",
"Words above your level get a hint": "Las palabras por encima de tu nivel reciben una ayuda",
"Hint Language": "Idioma de las ayudas",
"Data": "Datos",
"Download data packs automatically when needed.": "Descargar los paquetes de datos automáticamente cuando sea necesario.",
"Failed to check for updates": "No se pudo buscar actualizaciones"
}

21
apps/readest-app/public/locales/fa/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "پخش صدا",
"Resume audio": "ادامهٔ دنبال‌کردن صدا",
"Following audio": "در حال دنبال‌کردن صدا",
" · estimated": " · تخمینی"
" · estimated": " · تخمینی",
"Nightly Builds (Unstable)": "نسخه‌های شبانه (ناپایدار)",
"Early daily builds; may be unstable": "نسخه‌های روزانهٔ اولیه؛ ممکن است ناپایدار باشند",
"Downloading Word Wise data…": "در حال دانلود داده‌های Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "نمایش راهنمایی کوتاه به زبان مادری بالای واژه‌های دشوار.",
"Word Wise data downloaded": "داده‌های Word Wise دانلود شد",
"Failed to download Word Wise data": "دانلود داده‌های Word Wise ناموفق بود",
"Word Wise data removed": "داده‌های Word Wise حذف شد",
"Data pack": "بستهٔ داده",
"Open a book to manage its data pack.": "برای مدیریت بستهٔ دادهٔ آن، کتابی را باز کنید.",
"No data available for this language pair yet.": "هنوز داده‌ای برای این جفت‌زبان موجود نیست.",
"Download {{size}}": "دانلود {{size}}",
"Enable Word Wise": "فعال‌سازی Word Wise",
"Vocabulary level": "سطح واژگان",
"Words above your level get a hint": "واژه‌های بالاتر از سطح شما راهنمایی می‌گیرند",
"Hint Language": "زبان راهنمایی",
"Data": "داده",
"Download data packs automatically when needed.": "دانلود خودکار بسته‌های داده در صورت نیاز.",
"Failed to check for updates": "بررسی به‌روزرسانی‌ها ناموفق بود"
}

21
apps/readest-app/public/locales/fr/translation.json
View file

@ -1763,5 +1763,24 @@
"Play audio": "Lire l'audio",
"Resume audio": "Reprendre l'audio",
"Following audio": "Suit l'audio",
" · estimated": " · estimé"
" · estimated": " · estimé",
"Nightly Builds (Unstable)": "Versions nightly (instables)",
"Early daily builds; may be unstable": "Versions quotidiennes anticipées ; peuvent être instables",
"Downloading Word Wise data…": "Téléchargement des données Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Affiche une courte explication dans votre langue au-dessus des mots difficiles.",
"Word Wise data downloaded": "Données Word Wise téléchargées",
"Failed to download Word Wise data": "Échec du téléchargement des données Word Wise",
"Word Wise data removed": "Données Word Wise supprimées",
"Data pack": "Pack de données",
"Open a book to manage its data pack.": "Ouvrez un livre pour gérer son pack de données.",
"No data available for this language pair yet.": "Aucune donnée disponible pour cette paire de langues pour le moment.",
"Download {{size}}": "Télécharger {{size}}",
"Enable Word Wise": "Activer Word Wise",
"Vocabulary level": "Niveau de vocabulaire",
"Words above your level get a hint": "Les mots au-dessus de votre niveau reçoivent une explication",
"Hint Language": "Langue des explications",
"Data": "Données",
"Download data packs automatically when needed.": "Télécharger automatiquement les packs de données si nécessaire.",
"Failed to check for updates": "Échec de la recherche de mises à jour"
}

21
apps/readest-app/public/locales/he/translation.json
View file

@ -1763,5 +1763,24 @@
"Play audio": "השמעת שמע",
"Resume audio": "חידוש מעקב",
"Following audio": "עוקב אחר השמע",
" · estimated": " · משוער"
" · estimated": " · משוער",
"Nightly Builds (Unstable)": "גרסאות לילה (לא יציבות)",
"Early daily builds; may be unstable": "גרסאות יומיות מוקדמות; עשויות להיות לא יציבות",
"Downloading Word Wise data…": "מוריד נתוני Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "הצגת רמז קצר בשפת האם מעל מילים קשות.",
"Word Wise data downloaded": "נתוני Word Wise הורדו",
"Failed to download Word Wise data": "הורדת נתוני Word Wise נכשלה",
"Word Wise data removed": "נתוני Word Wise הוסרו",
"Data pack": "חבילת נתונים",
"Open a book to manage its data pack.": "פתח ספר כדי לנהל את חבילת הנתונים שלו.",
"No data available for this language pair yet.": "אין עדיין נתונים זמינים עבור צמד שפות זה.",
"Download {{size}}": "הורדה {{size}}",
"Enable Word Wise": "הפעלת Word Wise",
"Vocabulary level": "רמת אוצר מילים",
"Words above your level get a hint": "מילים מעל הרמה שלך מקבלות רמז",
"Hint Language": "שפת הרמז",
"Data": "נתונים",
"Download data packs automatically when needed.": "הורדת חבילות נתונים אוטומטית בעת הצורך.",
"Failed to check for updates": "בדיקת העדכונים נכשלה"
}

21
apps/readest-app/public/locales/hi/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "ऑडियो चलाएँ",
"Resume audio": "ऑडियो फिर से अनुसरण करें",
"Following audio": "ऑडियो का अनुसरण कर रहा है",
" · estimated": " · अनुमानित"
" · estimated": " · अनुमानित",
"Nightly Builds (Unstable)": "नाइटली बिल्ड (अस्थिर)",
"Early daily builds; may be unstable": "शुरुआती दैनिक बिल्ड; अस्थिर हो सकते हैं",
"Downloading Word Wise data…": "Word Wise डेटा डाउनलोड हो रहा है…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "कठिन शब्दों के ऊपर मातृभाषा में एक संक्षिप्त संकेत दिखाएँ।",
"Word Wise data downloaded": "Word Wise डेटा डाउनलोड हो गया",
"Failed to download Word Wise data": "Word Wise डेटा डाउनलोड करने में विफल",
"Word Wise data removed": "Word Wise डेटा हटा दिया गया",
"Data pack": "डेटा पैक",
"Open a book to manage its data pack.": "इसके डेटा पैक को प्रबंधित करने के लिए कोई पुस्तक खोलें।",
"No data available for this language pair yet.": "इस भाषा युग्म के लिए अभी कोई डेटा उपलब्ध नहीं है।",
"Download {{size}}": "{{size}} डाउनलोड करें",
"Enable Word Wise": "Word Wise सक्षम करें",
"Vocabulary level": "शब्दावली स्तर",
"Words above your level get a hint": "आपके स्तर से ऊपर के शब्दों के लिए संकेत मिलता है",
"Hint Language": "संकेत भाषा",
"Data": "डेटा",
"Download data packs automatically when needed.": "ज़रूरत पड़ने पर डेटा पैक स्वतः डाउनलोड करें।",
"Failed to check for updates": "अपडेट की जाँच करने में विफल"
}

21
apps/readest-app/public/locales/hu/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "Hang lejátszása",
"Resume audio": "Követés folytatása",
"Following audio": "Hangot követi",
" · estimated": " · becsült"
" · estimated": " · becsült",
"Nightly Builds (Unstable)": "Éjszakai buildek (instabil)",
"Early daily builds; may be unstable": "Korai napi buildek; instabilak lehetnek",
"Downloading Word Wise data…": "Word Wise-adatok letöltése…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Rövid, anyanyelvi tipp megjelenítése a nehéz szavak felett.",
"Word Wise data downloaded": "Word Wise-adatok letöltve",
"Failed to download Word Wise data": "A Word Wise-adatok letöltése sikertelen",
"Word Wise data removed": "Word Wise-adatok eltávolítva",
"Data pack": "Adatcsomag",
"Open a book to manage its data pack.": "Nyiss meg egy könyvet az adatcsomagjának kezeléséhez.",
"No data available for this language pair yet.": "Ehhez a nyelvpárhoz még nincsenek adatok.",
"Download {{size}}": "Letöltés ({{size}})",
"Enable Word Wise": "Word Wise bekapcsolása",
"Vocabulary level": "Szókincsszint",
"Words above your level get a hint": "A szintednél nehezebb szavakhoz tipp jár",
"Hint Language": "Tippek nyelve",
"Data": "Adatok",
"Download data packs automatically when needed.": "Adatcsomagok automatikus letöltése, amikor szükséges.",
"Failed to check for updates": "A frissítések keresése sikertelen"
}

21
apps/readest-app/public/locales/id/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "Putar audio",
"Resume audio": "Lanjutkan mengikuti",
"Following audio": "Mengikuti audio",
" · estimated": " · perkiraan"
" · estimated": " · perkiraan",
"Nightly Builds (Unstable)": "Build Harian (Tidak Stabil)",
"Early daily builds; may be unstable": "Build harian awal; mungkin tidak stabil",
"Downloading Word Wise data…": "Mengunduh data Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Tampilkan petunjuk singkat dalam bahasa ibu di atas kata-kata sulit.",
"Word Wise data downloaded": "Data Word Wise telah diunduh",
"Failed to download Word Wise data": "Gagal mengunduh data Word Wise",
"Word Wise data removed": "Data Word Wise dihapus",
"Data pack": "Paket data",
"Open a book to manage its data pack.": "Buka buku untuk mengelola paket datanya.",
"No data available for this language pair yet.": "Belum ada data tersedia untuk pasangan bahasa ini.",
"Download {{size}}": "Unduh {{size}}",
"Enable Word Wise": "Aktifkan Word Wise",
"Vocabulary level": "Tingkat kosakata",
"Words above your level get a hint": "Kata di atas tingkat Anda akan diberi petunjuk",
"Hint Language": "Bahasa Petunjuk",
"Data": "Data",
"Download data packs automatically when needed.": "Unduh paket data secara otomatis saat diperlukan.",
"Failed to check for updates": "Gagal memeriksa pembaruan"
}

21
apps/readest-app/public/locales/it/translation.json
View file

@ -1763,5 +1763,24 @@
"Play audio": "Riproduci audio",
"Resume audio": "Riprendi audio",
"Following audio": "Segue l'audio",
" · estimated": " · stimato"
" · estimated": " · stimato",
"Nightly Builds (Unstable)": "Versioni nightly (instabili)",
"Early daily builds; may be unstable": "Versioni giornaliere anticipate; potrebbero essere instabili",
"Downloading Word Wise data…": "Download dei dati Word Wise in corso…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Mostra un breve suggerimento nella tua lingua sopra le parole difficili.",
"Word Wise data downloaded": "Dati Word Wise scaricati",
"Failed to download Word Wise data": "Impossibile scaricare i dati Word Wise",
"Word Wise data removed": "Dati Word Wise rimossi",
"Data pack": "Pacchetto dati",
"Open a book to manage its data pack.": "Apri un libro per gestire il suo pacchetto dati.",
"No data available for this language pair yet.": "Nessun dato ancora disponibile per questa coppia di lingue.",
"Download {{size}}": "Scarica {{size}}",
"Enable Word Wise": "Attiva Word Wise",
"Vocabulary level": "Livello di vocabolario",
"Words above your level get a hint": "Le parole sopra il tuo livello ricevono un suggerimento",
"Hint Language": "Lingua dei suggerimenti",
"Data": "Dati",
"Download data packs automatically when needed.": "Scarica automaticamente i pacchetti dati quando necessario.",
"Failed to check for updates": "Impossibile verificare la presenza di aggiornamenti"
}

21
apps/readest-app/public/locales/ja/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "音声を再生",
"Resume audio": "音声に再追従",
"Following audio": "音声に追従中",
" · estimated": " · 推定"
" · estimated": " · 推定",
"Nightly Builds (Unstable)": "ナイトリービルド（不安定版）",
"Early daily builds; may be unstable": "毎日配信の先行ビルド。不安定な場合があります",
"Downloading Word Wise data…": "Word Wise データをダウンロード中…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "難しい単語の上に母国語の短いヒントを表示します。",
"Word Wise data downloaded": "Word Wise データをダウンロードしました",
"Failed to download Word Wise data": "Word Wise データのダウンロードに失敗しました",
"Word Wise data removed": "Word Wise データを削除しました",
"Data pack": "データパック",
"Open a book to manage its data pack.": "データパックを管理するには本を開いてください。",
"No data available for this language pair yet.": "この言語ペアに対応するデータはまだありません。",
"Download {{size}}": "ダウンロード {{size}}",
"Enable Word Wise": "Word Wise を有効にする",
"Vocabulary level": "語彙レベル",
"Words above your level get a hint": "レベルを超える単語にヒントを表示します",
"Hint Language": "ヒント言語",
"Data": "データ",
"Download data packs automatically when needed.": "必要に応じてデータパックを自動的にダウンロードします。",
"Failed to check for updates": "更新の確認に失敗しました"
}

21
apps/readest-app/public/locales/ko/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "오디오 재생",
"Resume audio": "오디오 다시 따라가기",
"Following audio": "오디오 따라가는 중",
" · estimated": " · 추정"
" · estimated": " · 추정",
"Nightly Builds (Unstable)": "나이틀리 빌드(불안정)",
"Early daily builds; may be unstable": "매일 배포되는 미리보기 빌드로, 불안정할 수 있습니다",
"Downloading Word Wise data…": "Word Wise 데이터 다운로드 중…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "어려운 단어 위에 모국어로 된 짧은 힌트를 표시합니다.",
"Word Wise data downloaded": "Word Wise 데이터를 다운로드했습니다",
"Failed to download Word Wise data": "Word Wise 데이터 다운로드에 실패했습니다",
"Word Wise data removed": "Word Wise 데이터를 삭제했습니다",
"Data pack": "데이터 팩",
"Open a book to manage its data pack.": "데이터 팩을 관리하려면 책을 여세요.",
"No data available for this language pair yet.": "이 언어 쌍에 사용할 수 있는 데이터가 아직 없습니다.",
"Download {{size}}": "다운로드 {{size}}",
"Enable Word Wise": "Word Wise 사용",
"Vocabulary level": "어휘 수준",
"Words above your level get a hint": "수준을 넘는 단어에 힌트를 표시합니다",
"Hint Language": "힌트 언어",
"Data": "데이터",
"Download data packs automatically when needed.": "필요할 때 데이터 팩을 자동으로 다운로드합니다.",
"Failed to check for updates": "업데이트 확인에 실패했습니다"
}

21
apps/readest-app/public/locales/ms/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "Main audio",
"Resume audio": "Sambung ikut audio",
"Following audio": "Mengikut audio",
" · estimated": " · anggaran"
" · estimated": " · anggaran",
"Nightly Builds (Unstable)": "Binaan Harian (Tidak Stabil)",
"Early daily builds; may be unstable": "Binaan harian awal; mungkin tidak stabil",
"Downloading Word Wise data…": "Memuat turun data Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Tunjukkan petunjuk ringkas dalam bahasa ibunda di atas perkataan sukar.",
"Word Wise data downloaded": "Data Word Wise telah dimuat turun",
"Failed to download Word Wise data": "Gagal memuat turun data Word Wise",
"Word Wise data removed": "Data Word Wise dialih keluar",
"Data pack": "Pakej data",
"Open a book to manage its data pack.": "Buka buku untuk mengurus pakej datanya.",
"No data available for this language pair yet.": "Belum ada data tersedia untuk pasangan bahasa ini.",
"Download {{size}}": "Muat turun {{size}}",
"Enable Word Wise": "Dayakan Word Wise",
"Vocabulary level": "Tahap perbendaharaan kata",
"Words above your level get a hint": "Perkataan melebihi tahap anda akan diberi petunjuk",
"Hint Language": "Bahasa Petunjuk",
"Data": "Data",
"Download data packs automatically when needed.": "Muat turun pakej data secara automatik apabila diperlukan.",
"Failed to check for updates": "Gagal menyemak kemas kini"
}

21
apps/readest-app/public/locales/nl/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "Audio afspelen",
"Resume audio": "Audio hervatten",
"Following audio": "Volgt audio",
" · estimated": " · geschat"
" · estimated": " · geschat",
"Nightly Builds (Unstable)": "Nightly-builds (onstabiel)",
"Early daily builds; may be unstable": "Vroege dagelijkse builds; kunnen onstabiel zijn",
"Downloading Word Wise data…": "Word Wise-gegevens downloaden…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Toon een korte hint in je moedertaal boven moeilijke woorden.",
"Word Wise data downloaded": "Word Wise-gegevens gedownload",
"Failed to download Word Wise data": "Downloaden van Word Wise-gegevens mislukt",
"Word Wise data removed": "Word Wise-gegevens verwijderd",
"Data pack": "Gegevenspakket",
"Open a book to manage its data pack.": "Open een boek om het gegevenspakket te beheren.",
"No data available for this language pair yet.": "Nog geen gegevens beschikbaar voor deze taalcombinatie.",
"Download {{size}}": "Download {{size}}",
"Enable Word Wise": "Word Wise inschakelen",
"Vocabulary level": "Woordenschatniveau",
"Words above your level get a hint": "Woorden boven je niveau krijgen een hint",
"Hint Language": "Hinttaal",
"Data": "Gegevens",
"Download data packs automatically when needed.": "Download gegevenspakketten automatisch wanneer nodig.",
"Failed to check for updates": "Controleren op updates mislukt"
}

21
apps/readest-app/public/locales/pl/translation.json
View file

@ -1796,5 +1796,24 @@
"Play audio": "Odtwórz dźwięk",
"Resume audio": "Wznów podążanie",
"Following audio": "Podąża za dźwiękiem",
" · estimated": " · szacowane"
" · estimated": " · szacowane",
"Nightly Builds (Unstable)": "Wersje nocne (niestabilne)",
"Early daily builds; may be unstable": "Wczesne codzienne wersje; mogą być niestabilne",
"Downloading Word Wise data…": "Pobieranie danych Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Pokazuj krótką podpowiedź w języku ojczystym nad trudnymi słowami.",
"Word Wise data downloaded": "Pobrano dane Word Wise",
"Failed to download Word Wise data": "Nie udało się pobrać danych Word Wise",
"Word Wise data removed": "Usunięto dane Word Wise",
"Data pack": "Pakiet danych",
"Open a book to manage its data pack.": "Otwórz książkę, aby zarządzać jej pakietem danych.",
"No data available for this language pair yet.": "Brak danych dla tej pary języków.",
"Download {{size}}": "Pobierz {{size}}",
"Enable Word Wise": "Włącz Word Wise",
"Vocabulary level": "Poziom słownictwa",
"Words above your level get a hint": "Słowa powyżej Twojego poziomu otrzymują podpowiedź",
"Hint Language": "Język podpowiedzi",
"Data": "Dane",
"Download data packs automatically when needed.": "Pobieraj pakiety danych automatycznie w razie potrzeby.",
"Failed to check for updates": "Nie udało się sprawdzić aktualizacji"
}

21
apps/readest-app/public/locales/pt-BR/translation.json
View file

@ -1763,5 +1763,24 @@
"Play audio": "Reproduzir áudio",
"Resume audio": "Retomar áudio",
"Following audio": "Seguindo o áudio",
" · estimated": " · estimado"
" · estimated": " · estimado",
"Nightly Builds (Unstable)": "Versões nightly (instáveis)",
"Early daily builds; may be unstable": "Versões diárias antecipadas; podem ser instáveis",
"Downloading Word Wise data…": "Baixando dados do Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Mostra uma breve dica no seu idioma acima das palavras difíceis.",
"Word Wise data downloaded": "Dados do Word Wise baixados",
"Failed to download Word Wise data": "Falha ao baixar os dados do Word Wise",
"Word Wise data removed": "Dados do Word Wise removidos",
"Data pack": "Pacote de dados",
"Open a book to manage its data pack.": "Abra um livro para gerenciar seu pacote de dados.",
"No data available for this language pair yet.": "Ainda não há dados disponíveis para este par de idiomas.",
"Download {{size}}": "Baixar {{size}}",
"Enable Word Wise": "Ativar o Word Wise",
"Vocabulary level": "Nível de vocabulário",
"Words above your level get a hint": "As palavras acima do seu nível recebem uma dica",
"Hint Language": "Idioma das dicas",
"Data": "Dados",
"Download data packs automatically when needed.": "Baixar pacotes de dados automaticamente quando necessário.",
"Failed to check for updates": "Falha ao verificar atualizações"
}

21
apps/readest-app/public/locales/pt/translation.json
View file

@ -1763,5 +1763,24 @@
"Play audio": "Reproduzir áudio",
"Resume audio": "Retomar áudio",
"Following audio": "A seguir o áudio",
" · estimated": " · estimado"
" · estimated": " · estimado",
"Nightly Builds (Unstable)": "Versões nightly (instáveis)",
"Early daily builds; may be unstable": "Versões diárias antecipadas; podem ser instáveis",
"Downloading Word Wise data…": "A transferir dados do Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Mostra uma breve dica na sua língua por cima das palavras difíceis.",
"Word Wise data downloaded": "Dados do Word Wise transferidos",
"Failed to download Word Wise data": "Falha ao transferir os dados do Word Wise",
"Word Wise data removed": "Dados do Word Wise removidos",
"Data pack": "Pacote de dados",
"Open a book to manage its data pack.": "Abra um livro para gerir o respetivo pacote de dados.",
"No data available for this language pair yet.": "Ainda não há dados disponíveis para este par de idiomas.",
"Download {{size}}": "Transferir {{size}}",
"Enable Word Wise": "Ativar o Word Wise",
"Vocabulary level": "Nível de vocabulário",
"Words above your level get a hint": "As palavras acima do seu nível recebem uma dica",
"Hint Language": "Idioma das dicas",
"Data": "Dados",
"Download data packs automatically when needed.": "Transferir pacotes de dados automaticamente quando necessário.",
"Failed to check for updates": "Falha ao procurar atualizações"
}

21
apps/readest-app/public/locales/ro/translation.json
View file

@ -1763,5 +1763,24 @@
"Play audio": "Redă audio",
"Resume audio": "Reia urmărirea",
"Following audio": "Urmărește audio",
" · estimated": " · estimat"
" · estimated": " · estimat",
"Nightly Builds (Unstable)": "Versiuni nightly (instabile)",
"Early daily builds; may be unstable": "Versiuni zilnice timpurii; pot fi instabile",
"Downloading Word Wise data…": "Se descarcă datele Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Afișează un scurt indiciu în limba maternă deasupra cuvintelor dificile.",
"Word Wise data downloaded": "Datele Word Wise au fost descărcate",
"Failed to download Word Wise data": "Descărcarea datelor Word Wise a eșuat",
"Word Wise data removed": "Datele Word Wise au fost eliminate",
"Data pack": "Pachet de date",
"Open a book to manage its data pack.": "Deschide o carte pentru a gestiona pachetul ei de date.",
"No data available for this language pair yet.": "Încă nu există date pentru această pereche de limbi.",
"Download {{size}}": "Descarcă {{size}}",
"Enable Word Wise": "Activează Word Wise",
"Vocabulary level": "Nivel de vocabular",
"Words above your level get a hint": "Cuvintele peste nivelul tău primesc un indiciu",
"Hint Language": "Limba indiciilor",
"Data": "Date",
"Download data packs automatically when needed.": "Descarcă automat pachetele de date când este necesar.",
"Failed to check for updates": "Verificarea actualizărilor a eșuat"
}

21
apps/readest-app/public/locales/ru/translation.json
View file

@ -1796,5 +1796,24 @@
"Play audio": "Воспроизвести аудио",
"Resume audio": "Возобновить слежение",
"Following audio": "Следует за аудио",
" · estimated": " · примерно"
" · estimated": " · примерно",
"Nightly Builds (Unstable)": "Ночные сборки (нестабильные)",
"Early daily builds; may be unstable": "Ранние ежедневные сборки; могут быть нестабильными",
"Downloading Word Wise data…": "Загрузка данных Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Показывать краткую подсказку на родном языке над сложными словами.",
"Word Wise data downloaded": "Данные Word Wise загружены",
"Failed to download Word Wise data": "Не удалось загрузить данные Word Wise",
"Word Wise data removed": "Данные Word Wise удалены",
"Data pack": "Пакет данных",
"Open a book to manage its data pack.": "Откройте книгу, чтобы управлять её пакетом данных.",
"No data available for this language pair yet.": "Для этой языковой пары пока нет данных.",
"Download {{size}}": "Скачать {{size}}",
"Enable Word Wise": "Включить Word Wise",
"Vocabulary level": "Уровень словарного запаса",
"Words above your level get a hint": "Слова выше вашего уровня получают подсказку",
"Hint Language": "Язык подсказок",
"Data": "Данные",
"Download data packs automatically when needed.": "Автоматически загружать пакеты данных при необходимости.",
"Failed to check for updates": "Не удалось проверить обновления"
}

21
apps/readest-app/public/locales/si/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "ශ්‍රව්‍යය වාදනය කරන්න",
"Resume audio": "නැවත අනුගමනය කරන්න",
"Following audio": "ශ්‍රව්‍යය අනුගමනය කරයි",
" · estimated": " · ඇස්තමේන්තුගත"
" · estimated": " · ඇස්තමේන්තුගත",
"Nightly Builds (Unstable)": "රාත්‍රික නිකුතු (අස්ථිර)",
"Early daily builds; may be unstable": "මුල් දෛනික නිකුතු; අස්ථිර විය හැක",
"Downloading Word Wise data…": "Word Wise දත්ත බාගත වෙමින්…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "අසීරු වචන ඉහළින් මව්බසින් කෙටි ඉඟියක් පෙන්වන්න.",
"Word Wise data downloaded": "Word Wise දත්ත බාගත කරන ලදී",
"Failed to download Word Wise data": "Word Wise දත්ත බාගත කිරීමට අසමත් විය",
"Word Wise data removed": "Word Wise දත්ත ඉවත් කරන ලදී",
"Data pack": "දත්ත පැකේජය",
"Open a book to manage its data pack.": "එහි දත්ත පැකේජය කළමනාකරණය කිරීමට පොතක් විවෘත කරන්න.",
"No data available for this language pair yet.": "මෙම භාෂා යුගලය සඳහා තවම දත්ත නොමැත.",
"Download {{size}}": "{{size}} බාගන්න",
"Enable Word Wise": "Word Wise සක්‍රීය කරන්න",
"Vocabulary level": "වචන මාලා මට්ටම",
"Words above your level get a hint": "ඔබේ මට්ටමට වඩා ඉහළ වචන සඳහා ඉඟියක් ලැබේ",
"Hint Language": "ඉඟි භාෂාව",
"Data": "දත්ත",
"Download data packs automatically when needed.": "අවශ්‍ය විට දත්ත පැකේජ ස්වයංක්‍රීයව බාගන්න.",
"Failed to check for updates": "යාවත්කාලීන සඳහා පරීක්ෂා කිරීමට අසමත් විය"
}

21
apps/readest-app/public/locales/sl/translation.json
View file

@ -1796,5 +1796,24 @@
"Play audio": "Predvajaj zvok",
"Resume audio": "Nadaljuj sledenje",
"Following audio": "Sledi zvoku",
" · estimated": " · ocenjeno"
" · estimated": " · ocenjeno",
"Nightly Builds (Unstable)": "Nočne različice (nestabilne)",
"Early daily builds; may be unstable": "Zgodnje dnevne različice; lahko so nestabilne",
"Downloading Word Wise data…": "Prenašanje podatkov Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Nad težkimi besedami prikaži kratek namig v maternem jeziku.",
"Word Wise data downloaded": "Podatki Word Wise so preneseni",
"Failed to download Word Wise data": "Podatkov Word Wise ni bilo mogoče prenesti",
"Word Wise data removed": "Podatki Word Wise so odstranjeni",
"Data pack": "Paket podatkov",
"Open a book to manage its data pack.": "Odprite knjigo za upravljanje njenega paketa podatkov.",
"No data available for this language pair yet.": "Za ta jezikovni par še ni podatkov.",
"Download {{size}}": "Prenesi {{size}}",
"Enable Word Wise": "Omogoči Word Wise",
"Vocabulary level": "Raven besedišča",
"Words above your level get a hint": "Besede nad vašo ravnjo dobijo namig",
"Hint Language": "Jezik namigov",
"Data": "Podatki",
"Download data packs automatically when needed.": "Samodejno prenesi pakete podatkov, ko je to potrebno.",
"Failed to check for updates": "Preverjanje posodobitev ni uspelo"
}

21
apps/readest-app/public/locales/sv/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "Spela upp ljud",
"Resume audio": "Återuppta ljud",
"Following audio": "Följer ljudet",
" · estimated": " · uppskattat"
" · estimated": " · uppskattat",
"Nightly Builds (Unstable)": "Nattliga versioner (instabila)",
"Early daily builds; may be unstable": "Tidiga dagliga versioner; kan vara instabila",
"Downloading Word Wise data…": "Laddar ner Word Wise-data…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Visa en kort ledtråd på ditt modersmål ovanför svåra ord.",
"Word Wise data downloaded": "Word Wise-data nedladdad",
"Failed to download Word Wise data": "Det gick inte att ladda ner Word Wise-data",
"Word Wise data removed": "Word Wise-data borttagen",
"Data pack": "Datapaket",
"Open a book to manage its data pack.": "Öppna en bok för att hantera dess datapaket.",
"No data available for this language pair yet.": "Inga data tillgängliga för det här språkparet ännu.",
"Download {{size}}": "Ladda ner {{size}}",
"Enable Word Wise": "Aktivera Word Wise",
"Vocabulary level": "Ordförrådsnivå",
"Words above your level get a hint": "Ord över din nivå får en ledtråd",
"Hint Language": "Språk för ledtrådar",
"Data": "Data",
"Download data packs automatically when needed.": "Ladda ner datapaket automatiskt vid behov.",
"Failed to check for updates": "Det gick inte att söka efter uppdateringar"
}

21
apps/readest-app/public/locales/ta/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "ஆடியோவை இயக்கு",
"Resume audio": "மீண்டும் பின்தொடர்",
"Following audio": "ஆடியோவைப் பின்தொடர்கிறது",
" · estimated": " · தோராயமாக"
" · estimated": " · தோராயமாக",
"Nightly Builds (Unstable)": "நைட்லி பதிப்புகள் (நிலையற்றவை)",
"Early daily builds; may be unstable": "ஆரம்ப தினசரி பதிப்புகள்; நிலையற்றதாக இருக்கலாம்",
"Downloading Word Wise data…": "Word Wise தரவு பதிவிறக்கப்படுகிறது…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "கடினமான சொற்களுக்கு மேல் தாய்மொழியில் ஒரு சிறிய குறிப்பைக் காட்டு.",
"Word Wise data downloaded": "Word Wise தரவு பதிவிறக்கப்பட்டது",
"Failed to download Word Wise data": "Word Wise தரவைப் பதிவிறக்க முடியவில்லை",
"Word Wise data removed": "Word Wise தரவு அகற்றப்பட்டது",
"Data pack": "தரவுத் தொகுப்பு",
"Open a book to manage its data pack.": "அதன் தரவுத் தொகுப்பை நிர்வகிக்க ஒரு புத்தகத்தைத் திற.",
"No data available for this language pair yet.": "இந்த மொழி இணைக்கு இன்னும் தரவு எதுவும் கிடைக்கவில்லை.",
"Download {{size}}": "{{size}} பதிவிறக்கு",
"Enable Word Wise": "Word Wise-ஐ இயக்கு",
"Vocabulary level": "சொல்வளம் நிலை",
"Words above your level get a hint": "உங்கள் நிலைக்கு மேலான சொற்களுக்கு ஒரு குறிப்பு கிடைக்கும்",
"Hint Language": "குறிப்பு மொழி",
"Data": "தரவு",
"Download data packs automatically when needed.": "தேவைப்படும்போது தரவுத் தொகுப்புகளைத் தானாகப் பதிவிறக்கு.",
"Failed to check for updates": "புதுப்பிப்புகளைச் சரிபார்க்க முடியவில்லை"
}

21
apps/readest-app/public/locales/th/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "เล่นเสียง",
"Resume audio": "ติดตามเสียงต่อ",
"Following audio": "กำลังติดตามเสียง",
" · estimated": " · โดยประมาณ"
" · estimated": " · โดยประมาณ",
"Nightly Builds (Unstable)": "รุ่นทดสอบรายวัน (ไม่เสถียร)",
"Early daily builds; may be unstable": "รุ่นทดสอบรายวันช่วงแรก อาจไม่เสถียร",
"Downloading Word Wise data…": "กำลังดาวน์โหลดข้อมูล Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "แสดงคำใบ้สั้น ๆ ในภาษาแม่เหนือคำที่ยาก",
"Word Wise data downloaded": "ดาวน์โหลดข้อมูล Word Wise แล้ว",
"Failed to download Word Wise data": "ดาวน์โหลดข้อมูล Word Wise ไม่สำเร็จ",
"Word Wise data removed": "ลบข้อมูล Word Wise แล้ว",
"Data pack": "ชุดข้อมูล",
"Open a book to manage its data pack.": "เปิดหนังสือเพื่อจัดการชุดข้อมูลของหนังสือ",
"No data available for this language pair yet.": "ยังไม่มีข้อมูลสำหรับคู่ภาษานี้",
"Download {{size}}": "ดาวน์โหลด {{size}}",
"Enable Word Wise": "เปิดใช้งาน Word Wise",
"Vocabulary level": "ระดับคำศัพท์",
"Words above your level get a hint": "คำที่อยู่เหนือระดับของคุณจะมีคำใบ้",
"Hint Language": "ภาษาของคำใบ้",
"Data": "ข้อมูล",
"Download data packs automatically when needed.": "ดาวน์โหลดชุดข้อมูลโดยอัตโนมัติเมื่อจำเป็น",
"Failed to check for updates": "ตรวจหาอัปเดตไม่สำเร็จ"
}

21
apps/readest-app/public/locales/tr/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "Sesi oynat",
"Resume audio": "Sesi takibe devam et",
"Following audio": "Sesi takip ediyor",
" · estimated": " · tahmini"
" · estimated": " · tahmini",
"Nightly Builds (Unstable)": "Gecelik Sürümler (Kararsız)",
"Early daily builds; may be unstable": "Erken günlük sürümler; kararsız olabilir",
"Downloading Word Wise data…": "Word Wise verileri indiriliyor…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Zor sözcüklerin üzerinde ana dilde kısa bir ipucu göster.",
"Word Wise data downloaded": "Word Wise verileri indirildi",
"Failed to download Word Wise data": "Word Wise verileri indirilemedi",
"Word Wise data removed": "Word Wise verileri kaldırıldı",
"Data pack": "Veri paketi",
"Open a book to manage its data pack.": "Veri paketini yönetmek için bir kitap açın.",
"No data available for this language pair yet.": "Bu dil çifti için henüz veri yok.",
"Download {{size}}": "İndir {{size}}",
"Enable Word Wise": "Word Wise'ı etkinleştir",
"Vocabulary level": "Kelime düzeyi",
"Words above your level get a hint": "Düzeyinizin üzerindeki sözcükler bir ipucu alır",
"Hint Language": "İpucu Dili",
"Data": "Veri",
"Download data packs automatically when needed.": "Gerektiğinde veri paketlerini otomatik olarak indir.",
"Failed to check for updates": "Güncellemeler denetlenemedi"
}

21
apps/readest-app/public/locales/uk/translation.json
View file

@ -1796,5 +1796,24 @@
"Play audio": "Відтворити аудіо",
"Resume audio": "Відновити стеження",
"Following audio": "Стежить за аудіо",
" · estimated": " · приблизно"
" · estimated": " · приблизно",
"Nightly Builds (Unstable)": "Нічні збірки (нестабільні)",
"Early daily builds; may be unstable": "Ранні щоденні збірки; можуть бути нестабільними",
"Downloading Word Wise data…": "Завантаження даних Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Показувати коротку підказку рідною мовою над складними словами.",
"Word Wise data downloaded": "Дані Word Wise завантажено",
"Failed to download Word Wise data": "Не вдалося завантажити дані Word Wise",
"Word Wise data removed": "Дані Word Wise видалено",
"Data pack": "Пакет даних",
"Open a book to manage its data pack.": "Відкрийте книгу, щоб керувати її пакетом даних.",
"No data available for this language pair yet.": "Для цієї мовної пари ще немає даних.",
"Download {{size}}": "Завантажити {{size}}",
"Enable Word Wise": "Увімкнути Word Wise",
"Vocabulary level": "Рівень словникового запасу",
"Words above your level get a hint": "Слова, вищі за ваш рівень, отримують підказку",
"Hint Language": "Мова підказок",
"Data": "Дані",
"Download data packs automatically when needed.": "Автоматично завантажувати пакети даних за потреби.",
"Failed to check for updates": "Не вдалося перевірити оновлення"
}

21
apps/readest-app/public/locales/uz/translation.json
View file

@ -1730,5 +1730,24 @@
"Play audio": "Audioni ijro etish",
"Resume audio": "Ergashishni davom ettirish",
"Following audio": "Audioga ergashmoqda",
" · estimated": " · taxminiy"
" · estimated": " · taxminiy",
"Nightly Builds (Unstable)": "Tungi yig'malar (Beqaror)",
"Early daily builds; may be unstable": "Erta kunlik yig'malar; beqaror bo'lishi mumkin",
"Downloading Word Wise data…": "Word Wise ma'lumotlari yuklab olinmoqda…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Qiyin so'zlar ustida ona tilida qisqa maslahat ko'rsatish.",
"Word Wise data downloaded": "Word Wise ma'lumotlari yuklab olindi",
"Failed to download Word Wise data": "Word Wise ma'lumotlarini yuklab olib bo'lmadi",
"Word Wise data removed": "Word Wise ma'lumotlari o'chirildi",
"Data pack": "Ma'lumotlar to'plami",
"Open a book to manage its data pack.": "Ma'lumotlar to'plamini boshqarish uchun kitob oching.",
"No data available for this language pair yet.": "Bu til juftligi uchun hali ma'lumot yo'q.",
"Download {{size}}": "Yuklab olish {{size}}",
"Enable Word Wise": "Word Wise'ni yoqish",
"Vocabulary level": "Lug'at darajasi",
"Words above your level get a hint": "Darajangizdan yuqori so'zlar uchun maslahat beriladi",
"Hint Language": "Maslahat tili",
"Data": "Ma'lumotlar",
"Download data packs automatically when needed.": "Kerak bo'lganda ma'lumotlar to'plamlarini avtomatik yuklab olish.",
"Failed to check for updates": "Yangilanishlarni tekshirib bo'lmadi"
}

21
apps/readest-app/public/locales/vi/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "Phát âm thanh",
"Resume audio": "Tiếp tục theo âm thanh",
"Following audio": "Đang theo âm thanh",
" · estimated": " · ước tính"
" · estimated": " · ước tính",
"Nightly Builds (Unstable)": "Bản dựng hằng đêm (Không ổn định)",
"Early daily builds; may be unstable": "Bản dựng hằng ngày sớm; có thể không ổn định",
"Downloading Word Wise data…": "Đang tải dữ liệu Word Wise…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "Hiển thị gợi ý ngắn bằng tiếng mẹ đẻ phía trên các từ khó.",
"Word Wise data downloaded": "Đã tải xong dữ liệu Word Wise",
"Failed to download Word Wise data": "Không tải được dữ liệu Word Wise",
"Word Wise data removed": "Đã xóa dữ liệu Word Wise",
"Data pack": "Gói dữ liệu",
"Open a book to manage its data pack.": "Mở một cuốn sách để quản lý gói dữ liệu của nó.",
"No data available for this language pair yet.": "Chưa có dữ liệu cho cặp ngôn ngữ này.",
"Download {{size}}": "Tải xuống {{size}}",
"Enable Word Wise": "Bật Word Wise",
"Vocabulary level": "Trình độ từ vựng",
"Words above your level get a hint": "Các từ vượt trình độ của bạn sẽ có gợi ý",
"Hint Language": "Ngôn ngữ gợi ý",
"Data": "Dữ liệu",
"Download data packs automatically when needed.": "Tự động tải gói dữ liệu khi cần.",
"Failed to check for updates": "Không kiểm tra được bản cập nhật"
}

21
apps/readest-app/public/locales/zh-CN/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "播放音频",
"Resume audio": "恢复跟随",
"Following audio": "正在跟随音频",
" · estimated": " · 估算"
" · estimated": " · 估算",
"Nightly Builds (Unstable)": "每夜构建版（不稳定）",
"Early daily builds; may be unstable": "每日抢先构建，可能不稳定",
"Downloading Word Wise data…": "正在下载 Word Wise 数据…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "在生词上方显示简短的母语提示。",
"Word Wise data downloaded": "Word Wise 数据已下载",
"Failed to download Word Wise data": "下载 Word Wise 数据失败",
"Word Wise data removed": "Word Wise 数据已删除",
"Data pack": "数据包",
"Open a book to manage its data pack.": "打开一本书以管理其数据包。",
"No data available for this language pair yet.": "暂无适用于此语言对的数据。",
"Download {{size}}": "下载 {{size}}",
"Enable Word Wise": "启用 Word Wise",
"Vocabulary level": "词汇水平",
"Words above your level get a hint": "超出你水平的词会显示提示",
"Hint Language": "提示语言",
"Data": "数据",
"Download data packs automatically when needed.": "需要时自动下载数据包。",
"Failed to check for updates": "检查更新失败"
}

21
apps/readest-app/public/locales/zh-TW/translation.json
View file

@ -1697,5 +1697,24 @@
"Play audio": "播放音訊",
"Resume audio": "恢復跟隨",
"Following audio": "正在跟隨音訊",
" · estimated": " · 估算"
" · estimated": " · 估算",
"Nightly Builds (Unstable)": "每夜建置版（不穩定）",
"Early daily builds; may be unstable": "每日搶先建置，可能不穩定",
"Downloading Word Wise data…": "正在下載 Word Wise 資料…",
"Word Wise": "Word Wise",
"Show a short native-language hint above difficult words.": "在生難字上方顯示簡短的母語提示。",
"Word Wise data downloaded": "Word Wise 資料已下載",
"Failed to download Word Wise data": "下載 Word Wise 資料失敗",
"Word Wise data removed": "Word Wise 資料已移除",
"Data pack": "資料包",
"Open a book to manage its data pack.": "開啟一本書以管理其資料包。",
"No data available for this language pair yet.": "尚無適用於此語言配對的資料。",
"Download {{size}}": "下載 {{size}}",
"Enable Word Wise": "啟用 Word Wise",
"Vocabulary level": "詞彙程度",
"Words above your level get a hint": "超出你程度的字會顯示提示",
"Hint Language": "提示語言",
"Data": "資料",
"Download data packs automatically when needed.": "需要時自動下載資料包。",
"Failed to check for updates": "檢查更新失敗"
}

613
apps/readest-app/scripts/build-wordwise-data.mjs Normal file
View file

@ -0,0 +1,613 @@
// Build trimmed Word Wise gloss indices from open datasets.
//
// node scripts/build-wordwise-data.mjs en-zh path/to/ecdict.csv [topN]
// node scripts/build-wordwise-data.mjs zh-en path/to/cedict.txt path/to/hsk.json [topN]
// node scripts/build-wordwise-data.mjs build <src> <tgt> <freq.txt> <gloss.jsonl> [topN]
//
// The generalized `build` mode assembles a pack for any (src→tgt) pair where one
// side is English, from two open datasets:
// - FrequencyWords (CC-BY-SA-4.0): `word count` per line, descending → rank.
// - kaikki Wiktionary extract (CC-BY-SA-4.0): JSONL, used for the gloss map.
// tgt === 'en' → foreign headword → English glosses (extractXToEn).
// src === 'en' → English headword → target-language words (extractEnToX).
//
// Outputs data/wordwise/<pair>.json in the GlossIndexData shape:
// { meta, entries: { word: { r, g } }, inflections: { form: lemma } }
// plus data/wordwise/manifest.json indexing the available packs.
//
// ECDICT (MIT): columns word,phonetic,definition,translation,pos,collins,
// oxford,tag,bnc,frq,exchange,detail,audio. We keep word, frq (rank),
// a short translation (gloss), and parse `exchange` into an inflection map.
// CC-CEDICT (CC-BY-SA): lines `trad simp [pinyin] /sense/sense/`. HSK json
// gives difficulty; higher HSK level => higher (rarer) rank.
import {
readFileSync,
writeFileSync,
mkdirSync,
readdirSync,
createReadStream,
existsSync,
} from 'node:fs';
import { createHash } from 'node:crypto';
import { resolve } from 'node:path';
import { createInterface } from 'node:readline';
import { execFileSync } from 'node:child_process';
const OUT_DIR = resolve('data/wordwise');
const TOP_DEFAULT = 30000;
// Keep a hint short + clean: drop bracket annotations ([医], [网络], [ge4]),
// leading part-of-speech tags (ECDICT "a." / "vt." / "n."), and CC-CEDICT
// classifier clauses (CL:...); then keep the first 1–2 senses.
export function shortGloss(s) {
return s
.split(/[;；/]/)
.map((x) =>
x
.replace(/\[[^\]]*\]/g, '') // [医] [网络] [ge4] etc.
.replace(/^\s*(?:[a-zA-Z]{1,5}\.\s*)+/, '') // leading POS: "a. " "vt. "
.replace(/\bCL:[^;；/]*/g, '') // CC-CEDICT classifier clause
.replace(/\s+/g, ' ')
.trim(),
)
.filter(Boolean)
.slice(0, 2)
.join('；')
.slice(0, 24);
}
// Minimal CSV line parser (ECDICT quotes fields containing commas/newlines).
export function parseCsvLine(line) {
if (line == null) return null;
const out = [];
let cur = '',
inQ = false;
for (let i = 0; i < line.length; i++) {
const ch = line[i];
if (inQ) {
if (ch === '"' && line[i + 1] === '"') {
cur += '"';
i++;
} else if (ch === '"') inQ = false;
else cur += ch;
} else if (ch === '"') inQ = true;
else if (ch === ',') {
out.push(cur);
cur = '';
} else cur += ch;
}
out.push(cur);
return out;
}
// ECDICT exchange tags: 0 lemma, 1 lemma-type, p past, d done, i ing, 3 3rd,
// r comparative, t superlative, s plural. Collect inflected forms (not the lemma).
export function parseExchange(exchange, word) {
const forms = new Set();
for (const part of exchange.split('/')) {
const [tag, val] = part.split(':');
if (!val) continue;
if (['p', 'd', 'i', '3', 'r', 't', 's'].includes(tag)) forms.add(val);
}
forms.delete(word);
return [...forms];
}
// Build the EN→中文 index from ECDICT CSV *text* (so it's unit-testable).
export function buildEnZh(csvText, topN) {
const rows = csvText.split('\n');
const header = parseCsvLine(rows[0]) ?? [];
const col = (name) => header.indexOf(name);
const iWord = col('word'),
iTr = col('translation'),
iFrq = col('frq'),
iEx = col('exchange');
const entries = {};
const inflections = {};
const parsed = [];
for (let i = 1; i < rows.length; i++) {
const c = parseCsvLine(rows[i]);
if (!c || !c[iWord]) continue;
const frq = parseInt(c[iFrq] || '0', 10) || Number.MAX_SAFE_INTEGER;
const g = shortGloss((c[iTr] || '').replace(/\\n/g, '；'));
if (!g) continue;
parsed.push({ word: c[iWord], frq, g, exchange: c[iEx] || '' });
}
parsed.sort((a, b) => a.frq - b.frq);
for (const e of parsed.slice(0, topN)) {
entries[e.word.toLowerCase()] = { r: e.frq, g: e.g };
// exchange: "p:past/i:ing/3:thirdperson/..." — map every form back to the lemma.
// `parsed` is sorted most-common-first, so the FIRST lemma to claim a surface
// form wins: ambiguous forms resolve to the most frequent lemma (e.g. "does"
// -> "do", not "doe"; "putting" -> "put", not "putt").
for (const form of parseExchange(e.exchange, e.word)) {
const key = form.toLowerCase();
if (!inflections[key]) inflections[key] = e.word.toLowerCase();
}
}
// Drop standalone inflected-form entries (e.g. "kept", "children") when their
// lemma is also present: at lookup the surface form resolves to the lemma via
// `inflections`, so it inherits the lemma's difficulty rank + real gloss instead
// of being judged on its own + showing a cross-reference ("keep的过去式…").
for (const [form, lemma] of Object.entries(inflections)) {
if (entries[form] && entries[lemma]) delete entries[form];
}
return {
meta: {
source: 'en',
target: 'zh',
metric: 'frq',
version: 1,
count: Object.keys(entries).length,
},
entries,
inflections,
};
}
// Parse one CC-CEDICT line: `傳統 传统 [chuan2 tong3] /tradition/traditional/`.
// Returns { simp, senses: [...] } or null for comments / malformed lines.
export function parseCedictLine(line) {
if (!line || line.startsWith('#')) return null;
const space = line.indexOf(' ');
if (space === -1) return null;
const rest = line.slice(space + 1);
const space2 = rest.indexOf(' ');
if (space2 === -1) return null;
const simp = rest.slice(0, space2);
if (!simp) return null;
const firstSlash = line.indexOf('/');
const lastSlash = line.lastIndexOf('/');
if (firstSlash === -1 || lastSlash <= firstSlash) return null;
const senses = line
.slice(firstSlash + 1, lastSlash)
.split('/')
.map((x) => x.trim())
.filter(Boolean);
if (!senses.length) return null;
return { simp, senses };
}
// HSK json -> Map(word -> level). Tolerates either { "传统": 4 } or
// [{ "hanzi": "传统", "level": 4 }] shapes.
function buildHskLevels(hskJson) {
const levels = new Map();
if (Array.isArray(hskJson)) {
for (const item of hskJson) {
if (!item) continue;
const word = item.hanzi ?? item.simplified ?? item.word;
const level = Number(item.level ?? item.HSK ?? item.hsk);
if (word && Number.isFinite(level)) levels.set(word, level);
}
} else if (hskJson && typeof hskJson === 'object') {
for (const [word, level] of Object.entries(hskJson)) {
const n = Number(level);
if (Number.isFinite(n)) levels.set(word, n);
}
}
return levels;
}
// Build the 中文→EN index from CC-CEDICT *text* + an HSK json object.
// Rank is derived from HSK level (higher level => rarer => higher rank); words
// absent from HSK fall back to a constant "advanced" rank. No inflections.
export function buildZhEn(cedictText, hskJson, topN) {
const levels = buildHskLevels(hskJson);
const rankForLevel = (level) => {
if (!Number.isFinite(level) || level <= 0) return 20000;
return Math.min(level, 9) * 3000;
};
const entries = {};
const seen = new Set();
const parsed = [];
for (const line of cedictText.split('\n')) {
const row = parseCedictLine(line.trim());
if (!row) continue;
// First simplified headword wins (CC-CEDICT lists variants on later lines).
if (seen.has(row.simp)) continue;
seen.add(row.simp);
const g = shortGloss(row.senses.join('/'));
if (!g) continue;
const level = levels.get(row.simp);
parsed.push({ word: row.simp, rank: rankForLevel(level), g });
}
// Lower rank (more common) first so topN keeps the most useful entries.
parsed.sort((a, b) => a.rank - b.rank);
for (const e of parsed.slice(0, topN)) {
entries[e.word] = { r: e.rank, g: e.g };
}
return {
meta: {
source: 'zh',
target: 'en',
metric: 'hsk',
version: 1,
count: Object.keys(entries).length,
},
entries,
inflections: {},
};
}
// ---------------------------------------------------------------------------
// Generalized (frequency + Wiktionary) pack generation for any (src→tgt) pair.
// ---------------------------------------------------------------------------
// FrequencyWords text → [{ word, rank }]. Each non-blank line is `word count`;
// we keep the token before the first space (lowercased + trimmed); rank is the
// running 1-based index of kept lines (= difficulty rank, most common first).
export function parseFrequencyWords(text) {
const out = [];
let rank = 0;
for (const line of text.split('\n')) {
const trimmed = line.trim();
if (!trimmed) continue;
const word = trimmed.split(' ')[0].toLowerCase().trim();
if (!word) continue;
out.push({ word, rank: ++rank });
}
return out;
}
// Merge a foreign-headword entry's English glosses into the accumulator map.
// `glossMap` is Map(headword -> string[]); senses recur per POS so we merge
// and dedupe, capping at 4 glosses per headword. Returns nothing (mutates map).
function mergeXToEnEntry(obj, sourceCode, glossMap) {
if (!obj || !obj.word) return;
if (obj.lang_code && obj.lang_code !== sourceCode) return;
const senses = Array.isArray(obj.senses) ? obj.senses : [];
const glosses = [];
for (const sense of senses) {
const g = sense?.glosses?.[0];
if (typeof g === 'string' && g.trim()) glosses.push(g.trim());
if (glosses.length >= 4) break;
}
if (!glosses.length) return;
const key = String(obj.word).toLowerCase().trim();
if (!key) return;
const existing = glossMap.get(key) ?? [];
for (const g of glosses) {
if (existing.length >= 4) break;
if (!existing.includes(g)) existing.push(g);
}
glossMap.set(key, existing);
}
// X→en gloss map from in-memory JSONL text (used by tests). headword → glosses.
export function extractXToEn(jsonlText, sourceCode) {
const glossMap = new Map();
for (const line of jsonlText.split('\n')) {
if (!line.trim()) continue;
let obj;
try {
obj = JSON.parse(line);
} catch {
continue;
}
mergeXToEnEntry(obj, sourceCode, glossMap);
}
return glossMap;
}
// Merge an English-headword entry's target-language translations into the map.
// Gathers sense-level translations then top-level ones; keeps t.code === target
// with a t.word; value = `${word} (${roman})` when a roman field is present.
function mergeEnToXEntry(obj, targetCode, glossMap) {
if (!obj || !obj.word) return;
if (obj.lang_code !== 'en') return;
const collected = [];
const consider = (t) => {
if (!t || t.code !== targetCode || !t.word) return;
const word = String(t.word).trim();
if (!word) return;
const value = t.roman ? `${word} (${String(t.roman).trim()})` : word;
if (!collected.includes(value)) collected.push(value);
};
const senses = Array.isArray(obj.senses) ? obj.senses : [];
for (const sense of senses) {
for (const t of sense?.translations ?? []) consider(t);
}
for (const t of obj.translations ?? []) consider(t);
if (!collected.length) return;
const key = String(obj.word).toLowerCase().trim();
if (!key) return;
const existing = glossMap.get(key) ?? [];
for (const v of collected) {
if (existing.length >= 4) break;
if (!existing.includes(v)) existing.push(v);
}
glossMap.set(key, existing);
}
// en→X gloss map from in-memory JSONL text (used by tests). headword → words.
export function extractEnToX(jsonlText, targetCode) {
const glossMap = new Map();
for (const line of jsonlText.split('\n')) {
if (!line.trim()) continue;
let obj;
try {
obj = JSON.parse(line);
} catch {
continue;
}
mergeEnToXEntry(obj, targetCode, glossMap);
}
return glossMap;
}
// WikDict (DBnary/Wiktionary, CC-BY-SA-3.0): rows from the `simple_translation`
// table (written_rep, trans_list). trans_list is ` | `-joined, best-first.
// Returns Map(headword-lowercased -> string[] senses), merged + deduped, cap 6.
export function extractWikDict(rows) {
const glossMap = new Map();
for (const row of rows ?? []) {
if (!row || !row.written_rep || !row.trans_list) continue;
const key = String(row.written_rep).toLowerCase().trim();
if (!key) continue;
const senses = String(row.trans_list)
.split('|')
.map((s) => s.trim())
.filter(Boolean);
if (!senses.length) continue;
const existing = glossMap.get(key) ?? [];
for (const s of senses) {
if (existing.length >= 6) break;
if (!existing.includes(s)) existing.push(s);
}
glossMap.set(key, existing);
}
return glossMap;
}
// Parse a michmech-style lemmatization list (`lemma<TAB>form` per line, BOM-led)
// into a Map(form-lowercased -> lemma-lowercased). Used to lemmatize a non-English
// SOURCE language so an inflected word (e.g. Spanish "perros") gets glossed via its
// lemma ("perro"). First-wins on the rare ambiguous form.
export function parseLemmatizationList(text) {
const map = new Map();
for (const line of text.replace(/^/, '').split('\n')) {
const tab = line.indexOf('\t');
if (tab === -1) continue;
const lemma = line.slice(0, tab).trim().toLowerCase();
const form = line.slice(tab + 1).trim().toLowerCase();
if (!lemma || !form || form === lemma) continue;
if (!map.has(form)) map.set(form, lemma);
}
return map;
}
// Read a pack file's `inflections` map (form -> lemma) as a Map. Used to lemmatize
// English-source WikDict packs by reusing the en-zh pack's full inflection table.
export function inflectionMapFromPack(jsonText) {
try {
const infl = JSON.parse(jsonText).inflections || {};
return new Map(Object.entries(infl));
} catch {
return new Map();
}
}
// Stream a (possibly ~1 GB) JSONL file line-by-line, applying `perLine(obj)` to
// each parsed object. Shared by the streaming extractors so the CLI never holds
// the whole file in memory; parse errors are skipped silently.
async function streamJsonl(path, perLine) {
const rl = createInterface({
input: createReadStream(path, { encoding: 'utf8' }),
crlfDelay: Infinity,
});
for await (const line of rl) {
if (!line.trim()) continue;
let obj;
try {
obj = JSON.parse(line);
} catch {
continue;
}
perLine(obj);
}
}
// Streaming X→en (file path) — same per-line logic as extractXToEn.
export async function extractXToEnStream(path, sourceCode) {
const glossMap = new Map();
await streamJsonl(path, (obj) => mergeXToEnEntry(obj, sourceCode, glossMap));
return glossMap;
}
// Streaming en→X (file path) — same per-line logic as extractEnToX.
export async function extractEnToXStream(path, targetCode) {
const glossMap = new Map();
await streamJsonl(path, (obj) => mergeEnToXEntry(obj, targetCode, glossMap));
return glossMap;
}
// Assemble a pack in the GlossIndexData shape from a frequency list + gloss map.
// Walks freqList in order, skips the easiest `skipTop`, and emits up to `topN`
// entries; a surface word missing from glossMap falls back to its lemma (via
// lemmaMap). Inflections map each form to a lemma that is itself an entry.
export function buildPack({ freqList, glossMap, meta, topN = 30000, skipTop = 1000, lemmaMap = null }) {
const entries = {};
let count = 0;
for (let i = skipTop; i < freqList.length; i++) {
if (count >= topN) break;
const { word, rank } = freqList[i];
let senses = glossMap.get(word);
if (!senses && lemmaMap) {
const lemma = lemmaMap.get(word);
if (lemma) senses = glossMap.get(lemma);
}
if (!senses || !senses.length) continue;
const g = shortGloss(senses.join('；'));
if (!g) continue;
entries[word] = { r: rank, g };
count++;
}
// Inflected surface forms resolve to their lemma's entry via `inflections`, so
// drop any standalone inflected-form entry whose lemma is present (it then
// inherits the lemma's difficulty rank + gloss instead of being glossed itself).
const inflections = {};
if (lemmaMap) {
for (const [form, lemma] of lemmaMap) {
if (!entries[lemma]) continue;
delete entries[form];
inflections[form] = lemma;
}
}
return {
meta: { ...meta, metric: 'frequency', version: 1, count: Object.keys(entries).length },
entries,
inflections,
};
}
// Hex SHA-256 of a UTF-8 string (used for pack integrity in the manifest).
export function sha256Hex(text) {
return createHash('sha256').update(text, 'utf8').digest('hex');
}
// Build a manifest pack entry from a pack file's name + its raw JSON text.
export function packEntry(file, jsonText) {
const data = JSON.parse(jsonText);
const source = data.meta?.source,
target = data.meta?.target;
if (!source || !target) return null; // not a pack file
return {
pair: `${source}-${target}`,
source,
target,
file,
bytes: Buffer.byteLength(jsonText, 'utf8'),
sha256: sha256Hex(jsonText),
entries: Object.keys(data.entries || {}).length,
};
}
// Assemble the manifest object from pack entries (drops nulls, sorts by pair).
export function buildManifest(entries) {
const packs = entries.filter(Boolean).sort((a, b) => a.pair.localeCompare(b.pair));
return { schemaVersion: 1, packs };
}
// (Re)write manifest.json by scanning OUT_DIR for pack json files.
function writeManifest() {
const files = readdirSync(OUT_DIR).filter((f) => f.endsWith('.json') && f !== 'manifest.json');
const entries = files.map((f) => packEntry(f, readFileSync(resolve(OUT_DIR, f), 'utf8')));
const manifest = buildManifest(entries);
writeFileSync(resolve(OUT_DIR, 'manifest.json'), JSON.stringify(manifest, null, 2) + '\n');
return manifest;
}
// CLI entry point — skipped when imported by tests.
async function main() {
const [pair, ...rest] = process.argv.slice(2);
mkdirSync(OUT_DIR, { recursive: true });
if (pair === 'build') {
const [src, tgt, freqPath, glossPath, topN] = rest;
if (!src || !tgt || !freqPath || !glossPath)
throw new Error(
'usage: build-wordwise-data.mjs build <src> <tgt> <freq.txt> <gloss.jsonl> [topN]',
);
const freqList = parseFrequencyWords(readFileSync(freqPath, 'utf8'));
let glossMap;
if (tgt === 'en') {
glossMap = await extractXToEnStream(glossPath, src);
} else if (src === 'en') {
glossMap = await extractEnToXStream(glossPath, tgt);
} else {
throw new Error("build: one side must be 'en'");
}
const meta = {
source: src,
target: tgt,
license: 'CC-BY-SA-4.0',
attribution:
'Glosses: Wiktionary (CC-BY-SA-4.0) via kaikki.org. Frequency: hermitdave/FrequencyWords from OpenSubtitles/OPUS (CC-BY-SA-4.0).',
};
const data = buildPack({ freqList, glossMap, meta, topN: Number(topN) || TOP_DEFAULT });
const file = `${src}-${tgt}.json`;
writeFileSync(resolve(OUT_DIR, file), JSON.stringify(data));
console.log(`${file}: ${data.meta.count} entries`);
writeManifest();
} else if (pair === 'build-wikdict') {
const [src, tgt, freqPath, dbPath, topN, lemmaFile] = rest;
if (!src || !tgt || !freqPath || !dbPath)
throw new Error(
'usage: build-wordwise-data.mjs build-wikdict <src> <tgt> <freq.txt> <wikdict.sqlite3> [topN] [lemma.txt]',
);
let rows;
try {
const json = execFileSync(
'sqlite3',
['-json', dbPath, 'SELECT written_rep, trans_list FROM simple_translation'],
{ encoding: 'utf8', maxBuffer: 1 << 30 },
);
rows = json.trim() ? JSON.parse(json) : [];
} catch (err) {
throw new Error(
`build-wikdict: failed to read ${dbPath} via the 'sqlite3' CLI (is sqlite3 installed?). ${err.message}`,
);
}
const freqList = parseFrequencyWords(readFileSync(freqPath, 'utf8'));
const glossMap = extractWikDict(rows);
// Lemmatize the SOURCE language so inflected words are glossed via their lemma.
// English source reuses the en-zh pack's inflection table ("kept"->"keep"); a
// non-English source uses an optional michmech lemmatization list ("perros"->
// "perro"). No-op if neither is available.
let lemmaMap = null;
const enZhPath = resolve(OUT_DIR, 'en-zh.json');
if (src === 'en' && existsSync(enZhPath)) {
lemmaMap = inflectionMapFromPack(readFileSync(enZhPath, 'utf8'));
} else if (tgt === 'en' && lemmaFile && existsSync(lemmaFile)) {
lemmaMap = parseLemmatizationList(readFileSync(lemmaFile, 'utf8'));
}
const meta = {
source: src,
target: tgt,
license: 'CC-BY-SA-3.0',
attribution:
'Glosses: WikDict (CC-BY-SA-3.0), derived from DBnary/Wiktionary. Frequency: hermitdave/FrequencyWords from OpenSubtitles/OPUS (CC-BY-SA-4.0).',
};
const data = buildPack({ freqList, glossMap, meta, topN: Number(topN) || 20000, lemmaMap });
const file = `${src}-${tgt}.json`;
writeFileSync(resolve(OUT_DIR, file), JSON.stringify(data));
console.log(`${file}: ${data.meta.count} entries`);
writeManifest();
} else if (pair === 'en-zh') {
const [csv, topN] = rest;
if (!csv) throw new Error('usage: build-wordwise-data.mjs en-zh <ecdict.csv> [topN]');
const data = buildEnZh(readFileSync(csv, 'utf8'), Number(topN) || TOP_DEFAULT);
writeFileSync(resolve(OUT_DIR, 'en-zh.json'), JSON.stringify(data));
console.log(
`en-zh.json: ${data.meta.count} entries, ${Object.keys(data.inflections).length} inflections`,
);
writeManifest();
} else if (pair === 'zh-en') {
const [cedict, hsk, topN] = rest;
if (!cedict || !hsk)
throw new Error('usage: build-wordwise-data.mjs zh-en <cedict.txt> <hsk.json> [topN]');
const data = buildZhEn(
readFileSync(cedict, 'utf8'),
JSON.parse(readFileSync(hsk, 'utf8')),
Number(topN) || TOP_DEFAULT,
);
writeFileSync(resolve(OUT_DIR, 'zh-en.json'), JSON.stringify(data));
console.log(`zh-en.json: ${data.meta.count} entries`);
writeManifest();
} else if (pair === 'manifest') {
const m = writeManifest();
console.log('manifest.json:', m.packs.length, 'packs');
} else {
throw new Error(
'usage: build-wordwise-data.mjs <en-zh|zh-en|build|build-wikdict|manifest> <sources...> [topN]',
);
}
}
// Only run the CLI when executed directly (`node build-wordwise-data.mjs ...`),
// not when imported by the unit tests.
if (process.argv[1] && import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => {
console.error(err);
process.exit(1);
});
}

114
apps/readest-app/scripts/sync-wordwise-r2.mjs Normal file
View file

@ -0,0 +1,114 @@
// Upload the committed Word Wise gloss packs + manifest to the cdn.readest.com
// R2 bucket under /wordwise/. Maintainer/CI tool — run after regenerating data.
//
// WORDWISE_R2_BUCKET=<bucket> node scripts/sync-wordwise-r2.mjs
//
// Pack files get a one-year immutable cache (the app cache-busts via a ?v=<sha8>
// query); manifest.json gets a short max-age so new packs surface quickly. Packs
// upload first, manifest LAST, so the CDN manifest never points at a missing pack.
//
// Robustness: each `wrangler r2 object put` is a separate process, spawned with NO
// stdin (so it can't block on a prompt) and telemetry off. Some wrangler versions
// don't exit promptly after the upload when behind an HTTP proxy (the socket stays
// open), which would wedge a synchronous loop on the very first file. So we watch
// the output and, once "Upload complete" is printed, give wrangler a moment to exit
// and otherwise kill it and move on. A per-file timeout is the backstop; one failed
// file is logged and the batch continues, returning a non-zero exit if any failed.
import { readdirSync } from 'node:fs';
import { spawn } from 'node:child_process';
import { resolve } from 'node:path';
const SRC_DIR = resolve('data/wordwise');
const PACK_CACHE = 'public, max-age=31536000, immutable';
const MANIFEST_CACHE = 'public, max-age=300';
const SUCCESS_RE = /Upload complete/i;
const POST_SUCCESS_GRACE_MS = 2_000; // wait this long for a clean exit after success
const PER_FILE_TIMEOUT_MS = 120_000; // hard backstop per file
const uploadOne = (bucket, file) =>
new Promise((resolveP) => {
const cacheControl = file === 'manifest.json' ? MANIFEST_CACHE : PACK_CACHE;
const key = `${bucket}/wordwise/${file}`;
console.log(`Uploading ${file} -> ${key}`);
const child = spawn(
'wrangler',
[
'r2', 'object', 'put', key,
'--file', resolve(SRC_DIR, file),
'--remote',
'--content-type', 'application/json',
'--cache-control', cacheControl,
],
{ stdio: ['ignore', 'pipe', 'pipe'], env: { ...process.env, WRANGLER_SEND_METRICS: 'false' } },
);
let out = '';
let sawSuccess = false;
let settled = false;
let graceTimer = null;
const finish = (success) => {
if (settled) return;
settled = true;
clearTimeout(hardTimer);
clearTimeout(graceTimer);
if (!child.killed) child.kill('SIGKILL');
resolveP(success);
};
const onData = (d) => {
const s = d.toString();
process.stdout.write(s);
out += s;
if (!sawSuccess && SUCCESS_RE.test(out)) {
sawSuccess = true;
// Upload landed. Let wrangler exit on its own; kill it if it hangs (proxy).
graceTimer = setTimeout(() => finish(true), POST_SUCCESS_GRACE_MS);
}
};
child.stdout.on('data', onData);
child.stderr.on('data', onData);
const hardTimer = setTimeout(() => {
console.error(` ✗ ${file}: timed out after ${PER_FILE_TIMEOUT_MS / 1000}s`);
finish(sawSuccess);
}, PER_FILE_TIMEOUT_MS);
child.on('exit', (code) => {
const success = code === 0 || sawSuccess;
if (!success) console.error(` ✗ ${file}: wrangler exited ${code}`);
finish(success);
});
child.on('error', (e) => {
console.error(` ✗ ${file}: ${e.message}`);
finish(false);
});
});
async function main() {
const bucket = process.env.WORDWISE_R2_BUCKET;
if (!bucket) {
throw new Error('WORDWISE_R2_BUCKET env var is required (the cdn.readest.com R2 bucket name)');
}
const all = readdirSync(SRC_DIR).filter((f) => f.endsWith('.json'));
if (!all.includes('manifest.json')) {
throw new Error('manifest.json missing — run `pnpm wordwise:manifest` first');
}
const ordered = [...all.filter((f) => f !== 'manifest.json').sort(), 'manifest.json'];
let ok = 0;
const failed = [];
for (const file of ordered) {
// Sequential: keep output readable and avoid hammering the proxy.
// eslint-disable-next-line no-await-in-loop
if (await uploadOne(bucket, file)) ok += 1;
else failed.push(file);
}
console.log(`\nSynced ${ok}/${ordered.length} files to ${bucket}/wordwise/`);
if (failed.length) {
console.error(`Failed: ${failed.join(', ')}`);
process.exit(1);
}
}
main().catch((err) => {
console.error(err instanceof Error ? err.message : err);
process.exit(1);
});

55
apps/readest-app/src/__tests__/app/reader/wordwise-ruby.browser.test.ts Normal file
View file

@ -0,0 +1,55 @@
import { describe, it, expect, afterEach } from 'vitest';
import {
buildSectionTextModel,
applyGlosses,
clearGlosses,
findGlossWord,
} from '@/app/reader/utils/wordwiseRuby';
afterEach(() => {
document.body.innerHTML = '';
});
describe('wordwiseRuby', () => {
it('builds a text model whose string matches visible text and locates offsets', () => {
document.body.innerHTML = `<p id="p">The quick fox</p>`;
const model = buildSectionTextModel(document);
expect(model.text).toBe('The quick fox');
const startLoc = model.locate(4);
const endLoc = model.locate(9);
const r = document.createRange();
r.setStart(startLoc.node, startLoc.offset);
r.setEnd(endLoc.node, endLoc.offset);
expect(r.toString()).toBe('quick');
});
it('skips text inside existing ruby and rt', () => {
document.body.innerHTML = `<p>a<ruby>本<rt>もと</rt></ruby>b</p>`;
const model = buildSectionTextModel(document);
expect(model.text).toBe('ab');
});
it('wraps occurrences as cfi-inert ruby and clears them', () => {
document.body.innerHTML = `<p id="p">The quick fox</p>`;
const model = buildSectionTextModel(document);
applyGlosses(document, model, [{ start: 4, end: 9, word: 'quick', gloss: '敏捷' }]);
const ruby = document.querySelector('ruby.ww-gloss')!;
expect(ruby.getAttribute('cfi-skip')).toBe('');
expect(ruby.querySelector('rt')!.getAttribute('cfi-inert')).toBe('');
expect(ruby.querySelector('rt')!.textContent).toBe('敏捷');
expect(document.getElementById('p')!.textContent).toBe('The quick敏捷 fox');
clearGlosses(document);
expect(document.querySelector('ruby.ww-gloss')).toBeNull();
expect(document.getElementById('p')!.textContent).toBe('The quick fox');
});
it('finds the base word for a tap inside a gloss', () => {
document.body.innerHTML = `<p>The quick fox</p>`;
const model = buildSectionTextModel(document);
applyGlosses(document, model, [{ start: 4, end: 9, word: 'quick', gloss: '敏捷' }]);
const rt = document.querySelector('rt')!;
expect(findGlossWord(rt as unknown as HTMLElement)).toBe('quick');
});
});

29
apps/readest-app/src/__tests__/components/rsvp-overlay-context.test.tsx
View file

@ -480,35 +480,6 @@ describe('RSVPOverlay — dictionary lookup (#4475)', () => {
});
});
describe('RSVPOverlay — audio toggle layout (#3235 mobile follow-up)', () => {
afterEach(() => cleanup());
const wordsState = () =>
buildState({ words: [{ text: 'a', orpIndex: 0, pauseMultiplier: 1 }], currentIndex: 0 });
test('the audio toggle and settings flank the transport in flow, not an overlapping absolute cluster', () => {
const { container } = renderOverlay(wordsState());
const audioBtn = container.querySelector('[aria-label="Play audio"]') as HTMLElement;
const settingsBtn = container.querySelector('[aria-label="Settings"]') as HTMLElement;
const playBtn = container.querySelector('[aria-label="Play"]') as HTMLElement;
expect(audioBtn).not.toBeNull();
expect(settingsBtn).not.toBeNull();
expect(playBtn).not.toBeNull();
// The audio toggle and settings gear must be siblings of the play button in
// the same flex row so the cluster never overlaps the centered transport on
// a narrow (mobile) viewport.
expect(audioBtn.parentElement).toBe(playBtn.parentElement);
expect(settingsBtn.parentElement).toBe(playBtn.parentElement);
// No control may live inside an absolutely-positioned wrapper (the prior
// `absolute end-0` cluster that overlapped the transport on mobile).
expect(audioBtn.closest('.absolute')).toBeNull();
expect(settingsBtn.closest('.absolute')).toBeNull();
expect((playBtn.parentElement as HTMLElement).className).not.toContain('absolute');
});
});
describe('RSVPOverlay — start delay setting (#4478)', () => {
afterEach(() => {
cleanup();

28
apps/readest-app/src/__tests__/document/wordwise.browser.test.ts Normal file
View file

@ -0,0 +1,28 @@
import { describe, it, expect, afterEach } from 'vitest';
import { buildSectionTextModel, applyGlosses, clearGlosses } from '@/app/reader/utils/wordwiseRuby';
afterEach(() => {
document.body.innerHTML = '';
});
describe('Word Wise rendering (browser)', () => {
it('renders a gloss above a word and grows line height, then clears cleanly', () => {
document.body.innerHTML = `<div id="root" style="font-size:16px;line-height:1.2"><p>A cryptic note appears.</p></div>`;
const root = document.getElementById('root')!;
const before = root.getBoundingClientRect().height;
const model = buildSectionTextModel(document);
const start = model.text.indexOf('cryptic');
applyGlosses(document, model, [
{ start, end: start + 'cryptic'.length, word: 'cryptic', gloss: '晦涩的' },
]);
const rt = document.querySelector('ruby.ww-gloss > rt')!;
expect(rt.textContent).toBe('晦涩的');
expect(root.getBoundingClientRect().height).toBeGreaterThan(before);
clearGlosses(document);
expect(document.querySelector('ruby.ww-gloss')).toBeNull();
expect(root.textContent).toBe('A cryptic note appears.');
});
});

9
apps/readest-app/src/__tests__/fixtures/wordwise/en-zh.fixture.json Normal file
View file

@ -0,0 +1,9 @@
{
"meta": { "source": "en", "target": "zh", "metric": "frq", "version": 1, "count": 3 },
"entries": {
"run": { "r": 312, "g": "跑；经营" },
"cryptic": { "r": 18000, "g": "晦涩的" },
"the": { "r": 1, "g": "这" }
},
"inflections": { "running": "run", "ran": "run", "runs": "run" }
}

9
apps/readest-app/src/__tests__/fixtures/wordwise/zh-en.fixture.json Normal file
View file

@ -0,0 +1,9 @@
{
"meta": { "source": "zh", "target": "en", "metric": "hsk", "version": 1, "count": 3 },
"entries": {
"斟酌": { "r": 12000, "g": "to consider" },
"经济": { "r": 6000, "g": "economy" },
"的": { "r": 3000, "g": "(particle)" }
},
"inflections": {}
}

474
apps/readest-app/src/__tests__/scripts/build-wordwise-data.test.ts Normal file
View file

@ -0,0 +1,474 @@
import { describe, it, expect } from 'vitest';
// Import the real exported helpers from the .mjs build script (vitest/vite can
// import ESM .mjs directly), so the test exercises the actual logic.
import {
parseCsvLine,
parseExchange,
shortGloss,
sha256Hex,
packEntry,
buildManifest,
buildEnZh as buildEnZhUntyped,
buildZhEn as buildZhEnUntyped,
parseFrequencyWords as parseFrequencyWordsUntyped,
extractXToEn as extractXToEnUntyped,
extractEnToX as extractEnToXUntyped,
extractWikDict as extractWikDictUntyped,
inflectionMapFromPack as inflectionMapFromPackUntyped,
parseLemmatizationList as parseLemmatizationListUntyped,
buildPack as buildPackUntyped,
} from '../../../scripts/build-wordwise-data.mjs';
import type { GlossIndexData } from '@/services/wordwise/types';
// The .mjs script has no type annotations; pin the builders' returns to the
// real GlossIndexData shape so the assertions are type-checked.
const buildEnZh = buildEnZhUntyped as (csvText: string, topN: number) => GlossIndexData;
const buildZhEn = buildZhEnUntyped as (
cedictText: string,
hskJson: unknown,
topN: number,
) => GlossIndexData;
interface FreqEntry {
word: string;
rank: number;
}
const parseFrequencyWords = parseFrequencyWordsUntyped as (text: string) => FreqEntry[];
const extractXToEn = extractXToEnUntyped as (
jsonlText: string,
sourceCode: string,
) => Map<string, string[]>;
const extractEnToX = extractEnToXUntyped as (
jsonlText: string,
targetCode: string,
) => Map<string, string[]>;
const extractWikDict = extractWikDictUntyped as (
rows: { written_rep: string; trans_list: string }[],
) => Map<string, string[]>;
const inflectionMapFromPack = inflectionMapFromPackUntyped as (
jsonText: string,
) => Map<string, string>;
const parseLemmatizationList = parseLemmatizationListUntyped as (
text: string,
) => Map<string, string>;
const buildPack = buildPackUntyped as (args: {
freqList: FreqEntry[];
glossMap: Map<string, string[]>;
meta: Record<string, string>;
topN?: number;
skipTop?: number;
lemmaMap?: Map<string, string> | null;
}) => GlossIndexData;
describe('parseCsvLine', () => {
it('keeps a quoted field containing a comma intact', () => {
const cols = parseCsvLine('run,/rʌn/,"to run, operate",312');
expect(cols).toEqual(['run', '/rʌn/', 'to run, operate', '312']);
});
it('unescapes doubled quotes inside a quoted field', () => {
const cols = parseCsvLine('word,"a ""quoted"" word",1');
expect(cols).toEqual(['word', 'a "quoted" word', '1']);
});
});
describe('parseExchange', () => {
it('returns inflected forms and excludes the lemma', () => {
const forms = parseExchange('p:ran/i:running/3:runs/0:run', 'run');
expect(forms.sort()).toEqual(['ran', 'running', 'runs']);
expect(forms).not.toContain('run');
});
it('ignores tags outside the inflection set', () => {
// 0 = lemma, 1 = lemma-type: not inflected forms.
const forms = parseExchange('0:run/1:i', 'ran');
expect(forms).toEqual([]);
});
});
describe('shortGloss', () => {
it('truncates to the first 1-2 senses joined by ；and ≤24 chars', () => {
const g = shortGloss('to run; to operate; to manage; foo');
expect(g).toBe('to run；to operate');
expect(g.length).toBeLessThanOrEqual(24);
});
it('caps overly long senses at 24 chars', () => {
const g = shortGloss('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');
expect(g.length).toBe(24);
});
it('strips leading POS tags, bracket annotations, and CL: clauses', () => {
expect(shortGloss('a. 神秘的')).toBe('神秘的');
expect(shortGloss('vt. 做；vi. 看')).toBe('做；看');
expect(shortGloss('[网络] 隐；晦涩的')).toBe('隐；晦涩的');
expect(shortGloss('government/CL:個|个[ge4]')).toBe('government');
});
});
describe('buildEnZh', () => {
const header =
'word,phonetic,definition,translation,pos,collins,oxford,tag,bnc,frq,exchange,detail,audio';
// ECDICT separates senses in `translation` with a literal "\n" (backslash-n),
// which buildEnZh rewrites to ；before shortGloss runs.
const csv = [
header,
'Run,/rʌn/,def,跑；经营,v,3,,,100,312,p:ran/i:running/3:runs/0:run,,',
String.raw`cryptic,/ˈkrɪptɪk/,def,"晦涩的\n神秘的",adj,2,,,500,18000,,,`,
].join('\n');
it('produces a GlossIndexData shape with lowercased headwords from the CSV text', () => {
const data = buildEnZh(csv, 100);
expect(data.meta).toEqual({
source: 'en',
target: 'zh',
metric: 'frq',
version: 1,
count: 2,
});
// headword lowercased, r from frq, g from translation.
expect(data.entries['run']).toEqual({ r: 312, g: '跑；经营' });
expect(data.entries['cryptic']).toEqual({ r: 18000, g: '晦涩的；神秘的' });
// exchange forms map back to the lemma; lemma itself is not an inflection.
expect(data.inflections['ran']).toBe('run');
expect(data.inflections['running']).toBe('run');
expect(data.inflections['runs']).toBe('run');
expect(data.inflections['run']).toBeUndefined();
});
it('honours topN by frequency rank (keeps the most common)', () => {
const data = buildEnZh(csv, 1);
expect(Object.keys(data.entries)).toEqual(['run']);
expect(data.meta.count).toBe(1);
});
it('resolves an ambiguous inflected form to the most frequent lemma (first-wins)', () => {
// Both "do" (common) and "doe" (rare) claim the form "does"; the more frequent
// lemma must win so "does" -> "do", not "doe".
const ambiguous = [
header,
'do,/duː/,def,做,v,5,,,5,30,3:does/p:did,,',
'doe,/dəʊ/,def,母鹿,n,1,,,9000,15000,s:does,,',
].join('\n');
const data = buildEnZh(ambiguous, 100);
expect(data.inflections['does']).toBe('do');
});
it('drops an inflected-form entry when its lemma is present (lemmatize)', () => {
const lemmaInfl = [
header,
'keep,/kiːp/,def,保持；保留,v,5,,,50,120,p:kept/d:kept/i:keeping/3:keeps,,',
'kept,/kept/,def,keep的过去式和过去分词,v,,,,800,2500,,,',
].join('\n');
const data = buildEnZh(lemmaInfl, 100);
// "kept" is not a standalone entry; it resolves to "keep" via inflections.
expect(data.entries['kept']).toBeUndefined();
expect(data.entries['keep']).toEqual({ r: 120, g: '保持；保留' });
expect(data.inflections['kept']).toBe('keep');
expect(data.meta.count).toBe(1);
});
});
describe('buildZhEn', () => {
const cedict = [
'# CC-CEDICT comment line',
'傳統 传统 [chuan2 tong3] /tradition/traditional/',
'斟酌 斟酌 [zhen1 zhuo2] /to consider/to weigh/to deliberate/',
'傳統 传统 [chuan2 tong3] /variant entry/', // duplicate simplified: ignored
].join('\n');
it('parses simplified headwords with first English sense via shortGloss, no inflections', () => {
const data = buildZhEn(cedict, { 传统: 4 }, 100);
expect(data.meta.source).toBe('zh');
expect(data.meta.target).toBe('en');
expect(data.inflections).toEqual({});
// simplified headword, first 1-2 senses joined by ；
expect(data.entries['传统']).toEqual({ r: 12000, g: 'tradition；traditional' });
expect(data.entries['斟酌']).toEqual({ r: 20000, g: 'to consider；to weigh' });
// duplicate simplified line did not add a second entry.
expect(data.meta.count).toBe(2);
});
it('derives a higher (rarer) rank for a higher HSK level', () => {
const hsk2 = buildZhEn(cedict, { 传统: 2 }, 100).entries['传统']!.r;
const hsk6 = buildZhEn(cedict, { 传统: 6 }, 100).entries['传统']!.r;
expect(hsk6).toBeGreaterThan(hsk2);
});
});
describe('sha256Hex', () => {
it('matches the known SHA-256 hex of "abc"', () => {
expect(sha256Hex('abc')).toBe(
'ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad',
);
});
});
describe('packEntry', () => {
it('builds a manifest entry from a pack file name + its raw JSON text', () => {
const jsonText = JSON.stringify({
meta: { source: 'en', target: 'zh' },
entries: { run: { r: 1, g: '跑' } },
});
expect(packEntry('en-zh.json', jsonText)).toEqual({
pair: 'en-zh',
source: 'en',
target: 'zh',
file: 'en-zh.json',
bytes: Buffer.byteLength(jsonText, 'utf8'),
sha256: sha256Hex(jsonText),
entries: 1,
});
});
it('returns null for a JSON file without pack meta', () => {
expect(packEntry('x.json', JSON.stringify({ entries: {} }))).toBeNull();
});
});
describe('buildManifest', () => {
it('returns a schemaVersion-1 manifest with packs sorted by pair', () => {
const entryA = {
pair: 'en-zh',
source: 'en',
target: 'zh',
file: 'en-zh.json',
bytes: 10,
sha256: 'a',
entries: 1,
};
const entryB = {
pair: 'zh-en',
source: 'zh',
target: 'en',
file: 'zh-en.json',
bytes: 20,
sha256: 'b',
entries: 2,
};
expect(buildManifest([entryB, entryA])).toEqual({
schemaVersion: 1,
packs: [entryA, entryB],
});
});
});
describe('parseFrequencyWords', () => {
it('takes the token before the first space, 1-based rank, skipping blanks', () => {
expect(parseFrequencyWords('the 100\nde 90\n\nla 80')).toEqual([
{ word: 'the', rank: 1 },
{ word: 'de', rank: 2 },
{ word: 'la', rank: 3 },
]);
});
it('lowercases and trims the word and skips malformed lines', () => {
expect(parseFrequencyWords(' El 5\n\n \nLOS 3')).toEqual([
{ word: 'el', rank: 1 },
{ word: 'los', rank: 2 },
]);
});
});
describe('extractXToEn', () => {
const jsonl = [
JSON.stringify({
word: 'perro',
lang_code: 'es',
senses: [{ glosses: ['dog'] }, { glosses: ['scoundrel'] }],
}),
JSON.stringify({ word: 'hund', lang_code: 'de', senses: [{ glosses: ['dog'] }] }),
].join('\n');
it('maps the foreign headword to its english glosses and excludes wrong-lang lines', () => {
const map = extractXToEn(jsonl, 'es');
expect(map.get('perro')).toEqual(['dog', 'scoundrel']);
expect(map.has('hund')).toBe(false);
});
it('merges senses across recurring headwords and skips parse errors', () => {
const text = [
'not json at all',
JSON.stringify({
word: 'casa',
lang_code: 'es',
pos: 'noun',
senses: [{ glosses: ['house'] }],
}),
JSON.stringify({
word: 'Casa',
lang_code: 'es',
pos: 'verb',
senses: [{ glosses: ['to marry'] }],
}),
].join('\n');
expect(extractXToEn(text, 'es').get('casa')).toEqual(['house', 'to marry']);
});
});
describe('extractEnToX', () => {
const enEntry = JSON.stringify({
word: 'dog',
lang_code: 'en',
senses: [
{
glosses: ['animal'],
translations: [
{ code: 'es', word: 'perro' },
{ code: 'fr', word: 'chien' },
],
},
],
translations: [{ code: 'es', word: 'perro' }],
});
it('collects target translations deduped across sense + top-level', () => {
expect(extractEnToX(enEntry, 'es').get('dog')).toEqual(['perro']);
expect(extractEnToX(enEntry, 'fr').get('dog')).toEqual(['chien']);
});
it('appends a roman field in parens when present', () => {
const ru = JSON.stringify({
word: 'dog',
lang_code: 'en',
translations: [{ code: 'ru', word: 'собака', roman: 'sobaka' }],
});
expect(extractEnToX(ru, 'ru').get('dog')).toEqual(['собака (sobaka)']);
});
});
describe('buildPack', () => {
const meta = {
source: 'es',
target: 'en',
license: 'CC-BY-SA-4.0',
attribution: 'Glosses: Wiktionary. Frequency: FrequencyWords.',
};
it('skips the easiest skipTop words, honours topN, drops words lacking a gloss', () => {
const freqList: FreqEntry[] = [
{ word: 'la', rank: 1 }, // within skipTop:1 -> skipped
{ word: 'perro', rank: 2 },
{ word: 'gato', rank: 3 },
{ word: 'xyzzy', rank: 4 }, // no gloss -> dropped
{ word: 'casa', rank: 5 },
];
const glossMap = new Map<string, string[]>([
['la', ['the']],
['perro', ['dog']],
['gato', ['cat']],
['casa', ['house']],
]);
const data = buildPack({ freqList, glossMap, meta, topN: 2, skipTop: 1 });
expect(Object.keys(data.entries)).toEqual(['perro', 'gato']);
expect(data.entries['perro']).toEqual({ r: 2, g: 'dog' });
expect(typeof data.entries['perro']!.g).toBe('string');
expect(data.entries['gato']!.r).toBe(3);
expect(data.inflections).toEqual({});
expect(data.meta).toEqual({
source: 'es',
target: 'en',
license: 'CC-BY-SA-4.0',
attribution: 'Glosses: Wiktionary. Frequency: FrequencyWords.',
metric: 'frequency',
version: 1,
count: 2,
});
});
it('resolves a surface form via lemmaMap and emits inflections only for present lemmas', () => {
const freqList: FreqEntry[] = [
{ word: 'perros', rank: 1 },
{ word: 'gatos', rank: 2 },
];
const glossMap = new Map<string, string[]>([['perro', ['dog']]]);
const lemmaMap = new Map<string, string>([
['perros', 'perro'],
['gatos', 'gato'], // lemma 'gato' has no entry
]);
const data = buildPack({ freqList, glossMap, meta, topN: 30000, skipTop: 0, lemmaMap });
// perros resolves via lemma 'perro'
expect(data.entries['perros']).toEqual({ r: 1, g: 'dog' });
// gato has no gloss so gatos is dropped, and no inflection is emitted for it
expect(data.entries['gatos']).toBeUndefined();
// inflection emitted only because entries['perro'] would need to exist...
// entries here keyed on surface form 'perros', lemma 'perro' is not an entry key,
// so no inflection is emitted.
expect(data.inflections).toEqual({});
});
it('drops the inflected form (mapping it to its lemma) when the lemma is an entry', () => {
const freqList: FreqEntry[] = [
{ word: 'perro', rank: 1 },
{ word: 'perros', rank: 2 },
];
const glossMap = new Map<string, string[]>([['perro', ['dog']]]);
const lemmaMap = new Map<string, string>([['perros', 'perro']]);
const data = buildPack({ freqList, glossMap, meta, topN: 30000, skipTop: 0, lemmaMap });
expect(data.entries['perro']).toEqual({ r: 1, g: 'dog' });
// The inflected form is NOT a standalone entry; it resolves to its lemma.
expect(data.entries['perros']).toBeUndefined();
expect(data.inflections['perros']).toBe('perro');
});
});
describe('extractWikDict', () => {
it('splits trans_list on |, lowercases the key, and keeps senses', () => {
const m = extractWikDict([
{ written_rep: 'Rápido', trans_list: 'quick | fast | rapid' },
{ written_rep: 'casa', trans_list: 'house | home' },
]);
expect(m.get('rápido')).toEqual(['quick', 'fast', 'rapid']);
expect(m.get('casa')).toEqual(['house', 'home']);
});
it('merges + dedupes recurring headwords, capped at 6', () => {
const m = extractWikDict([
{ written_rep: 'banco', trans_list: 'bank | bench' },
{ written_rep: 'banco', trans_list: 'bench | shoal | seat | stand | rail | desk' },
]);
const senses = m.get('banco')!;
expect(senses.slice(0, 3)).toEqual(['bank', 'bench', 'shoal']);
expect(senses.length).toBe(6);
expect(new Set(senses).size).toBe(6); // deduped
});
it('skips rows with empty trans_list or written_rep', () => {
const m = extractWikDict([
{ written_rep: '', trans_list: 'x' },
{ written_rep: 'y', trans_list: '' },
]);
expect(m.size).toBe(0);
});
});
describe('inflectionMapFromPack', () => {
it('reads a pack JSON inflections map into a Map', () => {
const m = inflectionMapFromPack(
JSON.stringify({ entries: {}, inflections: { kept: 'keep', children: 'child' } }),
);
expect(m.get('kept')).toBe('keep');
expect(m.get('children')).toBe('child');
expect(m.size).toBe(2);
});
it('returns an empty Map on missing inflections or bad JSON', () => {
expect(inflectionMapFromPack(JSON.stringify({ entries: {} })).size).toBe(0);
expect(inflectionMapFromPack('not json').size).toBe(0);
});
});
describe('parseLemmatizationList', () => {
it('maps form -> lemma (col2 -> col1), strips a BOM, skips self-maps', () => {
const m = parseLemmatizationList('perro\tperros\nperro\tperra\ntener\ttuvo\ncasa\tcasa');
expect(m.get('perros')).toBe('perro');
expect(m.get('perra')).toBe('perro');
expect(m.get('tuvo')).toBe('tener');
expect(m.has('casa')).toBe(false); // form === lemma skipped
});
it('first-wins on an ambiguous form', () => {
const m = parseLemmatizationList('a\tx\nb\tx');
expect(m.get('x')).toBe('a');
});
});

32
apps/readest-app/src/__tests__/services/tts-word-highlight.test.ts
View file

@ -1,8 +1,9 @@
import { describe, test, expect } from 'vitest';
import { describe, test, it, expect } from 'vitest';
import {
computeWordOffsets,
findBoundaryIndexAtTime,
getTextSubRange,
rangeTextExcludingInert,
} from '@/services/tts/wordHighlight';
describe('computeWordOffsets', () => {
@ -149,3 +150,32 @@ describe('getTextSubRange', () => {
expect(getTextSubRange(base, -1, 2)).toBeNull();
});
});
const makeBaseRangeFrom = (html: string): Range => {
document.body.innerHTML = html;
const root = document.body.firstElementChild!;
const walker = document.createTreeWalker(root, NodeFilter.SHOW_TEXT);
const nodes: Text[] = [];
for (let n = walker.nextNode(); n; n = walker.nextNode()) nodes.push(n as Text);
const range = document.createRange();
range.setStart(nodes[0]!, 0);
range.setEnd(nodes.at(-1)!, nodes.at(-1)!.length);
return range;
};
describe('gloss-aware word highlighting', () => {
it('rangeTextExcludingInert drops cfi-inert gloss text', () => {
const base = makeBaseRangeFrom(
`<p>The <ruby cfi-skip="">quick<rt cfi-inert="">敏捷</rt></ruby> fox</p>`,
);
expect(rangeTextExcludingInert(base)).toBe('The quick fox');
});
it('getTextSubRange ignores the gloss when slicing a word', () => {
const base = makeBaseRangeFrom(
`<p>The <ruby cfi-skip="">quick<rt cfi-inert="">敏捷</rt></ruby> fox</p>`,
);
const sub = getTextSubRange(base, 10, 13)!;
expect(sub.toString()).toBe('fox');
});
});

60
apps/readest-app/src/__tests__/services/wordwise/difficulty.test.ts Normal file
View file

@ -0,0 +1,60 @@
import { describe, it, expect } from 'vitest';
import {
WORD_WISE_MIN_LEVEL,
WORD_WISE_MAX_LEVEL,
CEFR_LEVELS,
getRankCutoff,
cefrLabel,
isDifficult,
canTokenizeSource,
} from '@/services/wordwise/difficulty';
describe('difficulty', () => {
it('exposes a 1..6 (A1..C2) level range', () => {
expect(WORD_WISE_MIN_LEVEL).toBe(1);
expect(WORD_WISE_MAX_LEVEL).toBe(6);
expect(CEFR_LEVELS).toEqual(['A1', 'A2', 'B1', 'B2', 'C1', 'C2']);
});
it('labels each level with its CEFR band', () => {
expect(cefrLabel(1)).toBe('A1');
expect(cefrLabel(3)).toBe('B1');
expect(cefrLabel(6)).toBe('C2');
});
it('RAISES the EN cutoff as the level rises (A1 = most hints, C2 = fewest)', () => {
expect(getRankCutoff('en', 1)).toBeLessThan(getRankCutoff('en', 6));
expect(getRankCutoff('en', 1)).toBe(1000); // A1
expect(getRankCutoff('en', 3)).toBe(4000); // B1
});
it('uses the shared frequency table for other Latin sources', () => {
expect(getRankCutoff('es', 3)).toBe(4000);
expect(getRankCutoff('fr', 1)).toBe(getRankCutoff('en', 1));
});
it('uses the HSK scale for zh, aligned to the build script ranks', () => {
expect(getRankCutoff('zh', 1)).toBe(6000); // A1
expect(getRankCutoff('zh', 6)).toBe(24000); // C2
expect(getRankCutoff('zh', 1)).toBeLessThan(getRankCutoff('zh', 6));
});
it('clamps out-of-range levels', () => {
expect(getRankCutoff('en', 0)).toBe(getRankCutoff('en', 1));
expect(getRankCutoff('en', 99)).toBe(getRankCutoff('en', 6));
});
it('treats a word as difficult when its rank is at or beyond the cutoff', () => {
expect(isDifficult(8000, 7000)).toBe(true);
expect(isDifficult(7000, 7000)).toBe(true);
expect(isDifficult(6999, 7000)).toBe(false);
});
it('reports tokenizable sources (Latin + zh) and blocks segmenter-less CJK', () => {
expect(canTokenizeSource('en')).toBe(true);
expect(canTokenizeSource('zh')).toBe(true);
expect(canTokenizeSource('es')).toBe(true);
expect(canTokenizeSource('ja')).toBe(false);
expect(canTokenizeSource('ko')).toBe(false);
});
});

22
apps/readest-app/src/__tests__/services/wordwise/glossIndex.test.ts Normal file
View file

@ -0,0 +1,22 @@
import { describe, it, expect } from 'vitest';
import { GlossIndex } from '@/services/wordwise/glossIndex';
import fixture from '../../fixtures/wordwise/en-zh.fixture.json';
import type { GlossIndexData } from '@/services/wordwise/types';
const index = GlossIndex.fromData(fixture as GlossIndexData);
describe('GlossIndex', () => {
it('looks up an exact headword', () => {
expect(index.lookup('cryptic')).toEqual({ rank: 18000, gloss: '晦涩的' });
});
it('is case-insensitive', () => {
expect(index.lookup('The')).toEqual({ rank: 1, gloss: '这' });
});
it('resolves inflected forms to the lemma', () => {
expect(index.lookup('running')).toEqual({ rank: 312, gloss: '跑；经营' });
expect(index.lookup('RAN')).toEqual({ rank: 312, gloss: '跑；经营' });
});
it('returns null for unknown words', () => {
expect(index.lookup('zzzq')).toBeNull();
});
});

473
apps/readest-app/src/__tests__/services/wordwise/glossPacks.test.ts Normal file
View file

@ -0,0 +1,473 @@
import { describe, it, expect, beforeEach, vi } from 'vitest';
import type { AppService, BaseDir } from '@/types/system';
import type { GlossIndexData } from '@/services/wordwise/types';
import type {
BytesDownloader,
WordWiseManifest,
WordWisePack,
} from '@/services/wordwise/glossPacks';
// Fresh module instance per test so the in-session memos (manifestPromise,
// indexCache, ensureFlights, storeDirReady) start clean — via vi.resetModules()
// instead of a production-side reset helper.
const importGlossPacks = () => import('@/services/wordwise/glossPacks');
// ---- in-memory fake AppService (only the file IO glossPacks touches) ----
// Mirrors the REAL web appService: writeFile stores content verbatim and a 'text'
// read returns it UN-decoded (an ArrayBuffer stays an ArrayBuffer). This is what
// broke loadGlossIndex when the pack was written as bytes — keep the fake faithful
// so the suite guards that class of bug instead of hiding it.
const createFakeAppService = (): {
appService: AppService;
store: Map<string, string | ArrayBuffer>;
} => {
const store = new Map<string, string | ArrayBuffer>();
const key = (path: string, base: BaseDir) => `${base}:${path}`;
const encoder = new TextEncoder();
const appService = {
appPlatform: 'web',
async exists(path: string, base: BaseDir) {
return store.has(key(path, base));
},
async readFile(path: string, base: BaseDir, mode: 'text' | 'binary') {
if (!store.has(key(path, base))) throw new Error(`ENOENT: ${key(path, base)}`);
const content = store.get(key(path, base))!;
// 'text' returns the stored value verbatim (no decode, like web); 'binary'
// coerces a stored string to an ArrayBuffer.
if (mode === 'text') return content;
return typeof content === 'string'
? (encoder.encode(content).buffer as ArrayBuffer)
: content;
},
async writeFile(path: string, base: BaseDir, content: string | ArrayBuffer | File) {
store.set(key(path, base), content as string | ArrayBuffer);
},
async deleteFile(path: string, base: BaseDir) {
store.delete(key(path, base));
},
async createDir() {
/* no-op: flat in-memory store */
},
} as unknown as AppService;
return { appService, store };
};
// sha256 hex of bytes, matching glossPacks' own hashing.
const sha256Hex = async (buf: ArrayBuffer): Promise<string> =>
[...new Uint8Array(await crypto.subtle.digest('SHA-256', buf))]
.map((b) => b.toString(16).padStart(2, '0'))
.join('');
const packData: GlossIndexData = {
meta: { source: 'en', target: 'zh', metric: 'frq', version: 1, count: 1 },
entries: { cryptic: { r: 18000, g: '晦涩的' } },
inflections: {},
};
const packBytes = (): ArrayBuffer =>
new TextEncoder().encode(JSON.stringify(packData)).buffer as ArrayBuffer;
const makePack = (overrides: Partial<WordWisePack>, sha: string): WordWisePack => ({
pair: 'en-zh',
source: 'en',
target: 'zh',
file: 'en-zh.json',
bytes: packBytes().byteLength,
sha256: sha,
entries: 1,
...overrides,
});
describe('glossPacks', () => {
beforeEach(() => {
vi.resetModules();
});
describe('downloadViaTempFile', () => {
// Fake AppService where the temp file lives at an ABSOLUTE path. The bug
// was: download wrote to a CWD-relative path while the read resolved against
// 'Data' → write dst !== read dst → read threw. This fake keys its store by
// the absolute path and only resolves base 'None' against it, so the test
// passes only when the download dst and the read path are the same absolute
// path (the fix). The old relative-path code reads at a different key and
// throws, failing the test.
const createAbsAppService = (): { appService: AppService; map: Map<string, ArrayBuffer> } => {
const map = new Map<string, ArrayBuffer>();
const appService = {
appPlatform: 'tauri',
// 'Data' relative path -> absolute path under a fake Data root.
async resolveFilePath(path: string, base: BaseDir) {
expect(base).toBe('Data');
return `/abs/Data/${path}`;
},
async readFile(path: string, base: BaseDir, mode: 'text' | 'binary') {
expect(base).toBe('None'); // absolute path addressed with base 'None'
const buf = map.get(path);
if (!buf) throw new Error(`ENOENT: ${path}`);
return mode === 'binary' ? buf : new TextDecoder().decode(buf);
},
async deleteFile(path: string, base: BaseDir) {
expect(base).toBe('None');
map.delete(path);
},
async createDir() {
/* stubbed */
},
async exists() {
return false;
},
} as unknown as AppService;
return { appService, map };
};
it('downloads to an absolute temp path, reads back the SAME path, then deletes it', async () => {
const { downloadViaTempFile } = await importGlossPacks();
const { appService, map } = createAbsAppService();
const bytes = packBytes();
// Fake Rust downloader: write the bytes into the map at the exact `dst`
// it was handed (the absolute path). Read must use that same path.
const downloadFileFn = vi.fn(async ({ dst }: { dst: string }) => {
expect(dst.startsWith('/abs/Data/wordwise/.dl-')).toBe(true);
map.set(dst, bytes);
return new Headers();
});
const result = await downloadViaTempFile(appService, downloadFileFn, 'https://x/pack.json');
// Returns the downloaded bytes → proves write dst === read dst.
expect(new TextDecoder().decode(result)).toBe(new TextDecoder().decode(bytes));
expect(downloadFileFn).toHaveBeenCalledTimes(1);
// Temp file deleted afterward.
expect(map.size).toBe(0);
});
});
describe('resolvePack', () => {
it('picks the pack matching (source, target)', async () => {
const { resolvePack } = await importGlossPacks();
const manifest: WordWiseManifest = {
schemaVersion: 1,
packs: [
makePack({ pair: 'en-zh', source: 'en', target: 'zh' }, 'a'),
makePack({ pair: 'zh-en', source: 'zh', target: 'en', file: 'zh-en.json' }, 'b'),
],
};
expect(resolvePack(manifest, 'en', 'zh')?.pair).toBe('en-zh');
expect(resolvePack(manifest, 'zh', 'en')?.pair).toBe('zh-en');
});
it('returns null when no pack matches', async () => {
const { resolvePack } = await importGlossPacks();
const manifest: WordWiseManifest = {
schemaVersion: 1,
packs: [makePack({ source: 'en', target: 'zh' }, 'a')],
};
expect(resolvePack(manifest, 'en', 'fr')).toBeNull();
expect(resolvePack(null, 'en', 'zh')).toBeNull();
});
});
describe('ensurePack', () => {
it('downloads when absent, verifies sha, writes file + sidecar, returns the path', async () => {
const { ensurePack } = await importGlossPacks();
const { appService, store } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
const download: BytesDownloader = vi.fn(async () => packBytes());
const path = await ensurePack(appService, pack, { download });
expect(path).toBe('wordwise/en-zh.json');
expect(download).toHaveBeenCalledTimes(1);
expect(await appService.exists('wordwise/en-zh.json', 'Data')).toBe(true);
expect(await appService.readFile('wordwise/en-zh.json.sha', 'Data', 'text')).toBe(sha);
expect(store.size).toBe(2);
});
it('reuses the local file (no download) when file + matching sidecar exist', async () => {
const { ensurePack } = await importGlossPacks();
const { appService } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
// Seed local file + sidecar.
await appService.writeFile('wordwise/en-zh.json', 'Data', packBytes());
await appService.writeFile('wordwise/en-zh.json.sha', 'Data', sha);
const download: BytesDownloader = vi.fn(async () => packBytes());
const path = await ensurePack(appService, pack, { download });
expect(path).toBe('wordwise/en-zh.json');
expect(download).not.toHaveBeenCalled();
});
it('returns null and does not persist when downloaded bytes sha != pack.sha256', async () => {
const { ensurePack } = await importGlossPacks();
const { appService, store } = createFakeAppService();
const pack = makePack({}, 'deadbeef'); // wrong expected sha
const download: BytesDownloader = vi.fn(async () => packBytes());
const path = await ensurePack(appService, pack, { download });
expect(path).toBeNull();
expect(await appService.exists('wordwise/en-zh.json', 'Data')).toBe(false);
expect(store.size).toBe(0);
});
it('single-flights concurrent calls for the same pair (download once)', async () => {
const { ensurePack } = await importGlossPacks();
const { appService } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
const download: BytesDownloader = vi.fn(async () => packBytes());
const [a, b] = await Promise.all([
ensurePack(appService, pack, { download }),
ensurePack(appService, pack, { download }),
]);
expect(a).toBe('wordwise/en-zh.json');
expect(b).toBe('wordwise/en-zh.json');
expect(download).toHaveBeenCalledTimes(1);
});
});
describe('loadGlossIndex', () => {
it('manifest -> resolve -> ensure -> read -> GlossIndex that looks up a word', async () => {
const { loadGlossIndex } = await importGlossPacks();
const { appService } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
const manifest: WordWiseManifest = { schemaVersion: 1, packs: [pack] };
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
const download: BytesDownloader = vi.fn(async (url: string) =>
url.includes('manifest.json') ? manifestBytes : packBytes(),
);
const index = await loadGlossIndex(appService, 'en', 'zh', { download });
expect(index).not.toBeNull();
expect(index!.lookup('cryptic')).toEqual({ rank: 18000, gloss: '晦涩的' });
});
it('loads a pack cached as a raw ArrayBuffer (pre-fix web verbatim storage)', async () => {
const { loadGlossIndex } = await importGlossPacks();
const { appService } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
// Seed the cache the OLD way: pack stored as a raw ArrayBuffer (+ sidecar).
// A 'text' read returns the ArrayBuffer verbatim, so loadGlossIndex must
// decode it rather than JSON.parse("[object ArrayBuffer]").
await appService.writeFile('wordwise/en-zh.json', 'Data', packBytes());
await appService.writeFile('wordwise/en-zh.json.sha', 'Data', sha);
const manifest: WordWiseManifest = { schemaVersion: 1, packs: [pack] };
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
// Only the manifest is fetched; the pack is already cached.
const download: BytesDownloader = vi.fn(async () => manifestBytes);
const index = await loadGlossIndex(appService, 'en', 'zh', {
download,
allowDownload: false,
});
expect(index).not.toBeNull();
expect(index!.lookup('cryptic')).toEqual({ rank: 18000, gloss: '晦涩的' });
});
it('returns null when the manifest has no pack for the pair', async () => {
const { loadGlossIndex } = await importGlossPacks();
const { appService } = createFakeAppService();
const manifest: WordWiseManifest = { schemaVersion: 1, packs: [] };
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
const download: BytesDownloader = vi.fn(async () => manifestBytes);
const index = await loadGlossIndex(appService, 'en', 'fr', { download });
expect(index).toBeNull();
});
});
describe('allowDownload', () => {
it('ensurePack with allowDownload:false returns null and never downloads when uncached', async () => {
const { ensurePack } = await importGlossPacks();
const { appService, store } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
const download: BytesDownloader = vi.fn(async () => packBytes());
const path = await ensurePack(appService, pack, { download, allowDownload: false });
expect(path).toBeNull();
expect(download).not.toHaveBeenCalled();
expect(store.size).toBe(0);
});
it('ensurePack with allowDownload:false returns the cached path without downloading', async () => {
const { ensurePack } = await importGlossPacks();
const { appService } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
await appService.writeFile('wordwise/en-zh.json', 'Data', packBytes());
await appService.writeFile('wordwise/en-zh.json.sha', 'Data', sha);
const download: BytesDownloader = vi.fn(async () => packBytes());
const path = await ensurePack(appService, pack, { download, allowDownload: false });
expect(path).toBe('wordwise/en-zh.json');
expect(download).not.toHaveBeenCalled();
});
it('loadGlossIndex with allowDownload:false never downloads the pack when uncached', async () => {
const { loadGlossIndex } = await importGlossPacks();
const { appService } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
const manifest: WordWiseManifest = { schemaVersion: 1, packs: [pack] };
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
// Manifest still downloads; the *pack* must not.
const download: BytesDownloader = vi.fn(async (url: string) => {
if (url.includes('manifest.json')) return manifestBytes;
throw new Error('pack download must not happen with allowDownload:false');
});
const index = await loadGlossIndex(appService, 'en', 'zh', {
download,
allowDownload: false,
});
expect(index).toBeNull();
});
});
describe('getPackStatus', () => {
const buildManifest = (pack: WordWisePack): WordWiseManifest => ({
schemaVersion: 1,
packs: [pack],
});
it('reports downloaded=false before ensurePack, then true after', async () => {
const { getPackStatus, ensurePack } = await importGlossPacks();
const { appService } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
const manifest = buildManifest(pack);
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
const download: BytesDownloader = vi.fn(async (url: string) =>
url.includes('manifest.json') ? manifestBytes : packBytes(),
);
const before = await getPackStatus(appService, 'en', 'zh', { download });
expect(before).not.toBeNull();
expect(before!.pack.pair).toBe('en-zh');
expect(before!.downloaded).toBe(false);
await ensurePack(appService, pack, { download });
const after = await getPackStatus(appService, 'en', 'zh', { download });
expect(after!.downloaded).toBe(true);
});
it('returns null when the manifest has no pack for the pair', async () => {
const { getPackStatus } = await importGlossPacks();
const { appService } = createFakeAppService();
const manifest: WordWiseManifest = { schemaVersion: 1, packs: [] };
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
const download: BytesDownloader = vi.fn(async () => manifestBytes);
const status = await getPackStatus(appService, 'en', 'fr', { download });
expect(status).toBeNull();
});
});
describe('deletePack', () => {
it('removes the pack file + sidecar so getPackStatus flips back to false', async () => {
const { getPackStatus, ensurePack, deletePack } = await importGlossPacks();
const { appService, store } = createFakeAppService();
const sha = await sha256Hex(packBytes());
const pack = makePack({}, sha);
const manifest: WordWiseManifest = { schemaVersion: 1, packs: [pack] };
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
const download: BytesDownloader = vi.fn(async (url: string) =>
url.includes('manifest.json') ? manifestBytes : packBytes(),
);
await ensurePack(appService, pack, { download });
expect((await getPackStatus(appService, 'en', 'zh', { download }))!.downloaded).toBe(true);
await deletePack(appService, pack);
expect(await appService.exists('wordwise/en-zh.json', 'Data')).toBe(false);
expect(await appService.exists('wordwise/en-zh.json.sha', 'Data')).toBe(false);
// Only the persisted manifest remains.
expect(store.has('Data:wordwise/manifest.json')).toBe(true);
expect((await getPackStatus(appService, 'en', 'zh', { download }))!.downloaded).toBe(false);
});
it('ignores a missing file', async () => {
const { deletePack } = await importGlossPacks();
const { appService } = createFakeAppService();
const pack = makePack({}, 'abc');
await expect(deletePack(appService, pack)).resolves.toBeUndefined();
});
});
describe('listAvailableTargets', () => {
it('returns the target codes the manifest offers for a source', async () => {
const { listAvailableTargets } = await importGlossPacks();
const manifest: WordWiseManifest = {
schemaVersion: 1,
packs: [
makePack({ pair: 'en-zh', source: 'en', target: 'zh' }, 'a'),
makePack({ pair: 'en-fr', source: 'en', target: 'fr', file: 'en-fr.json' }, 'b'),
makePack({ pair: 'zh-en', source: 'zh', target: 'en', file: 'zh-en.json' }, 'c'),
],
};
expect(listAvailableTargets(manifest, 'en').sort()).toEqual(['fr', 'zh']);
expect(listAvailableTargets(manifest, 'zh')).toEqual(['en']);
expect(listAvailableTargets(manifest, 'de')).toEqual([]);
expect(listAvailableTargets(null, 'en')).toEqual([]);
});
});
describe('fetchManifest', () => {
it('downloads, persists to Data, then serves the persisted copy when offline', async () => {
const { fetchManifest } = await importGlossPacks();
const { appService } = createFakeAppService();
const manifest: WordWiseManifest = {
schemaVersion: 1,
packs: [makePack({}, 'abc')],
};
const manifestBytes = new TextEncoder().encode(JSON.stringify(manifest))
.buffer as ArrayBuffer;
const download: BytesDownloader = vi.fn(async () => manifestBytes);
const first = await fetchManifest(appService, { download });
expect(first?.packs[0]?.pair).toBe('en-zh');
expect(await appService.exists('wordwise/manifest.json', 'Data')).toBe(true);
// force: true bypasses the in-session memo, so we re-attempt the (now
// offline) download and fall back to the persisted copy.
const offlineDownload: BytesDownloader = vi.fn(async () => {
throw new Error('offline');
});
const second = await fetchManifest(appService, { download: offlineDownload, force: true });
expect(second?.packs[0]?.pair).toBe('en-zh');
});
it('returns null when network fails and no persisted copy exists', async () => {
const { fetchManifest } = await importGlossPacks();
const { appService } = createFakeAppService();
const download: BytesDownloader = vi.fn(async () => {
throw new Error('offline');
});
const result = await fetchManifest(appService, { download });
expect(result).toBeNull();
});
});
});

85
apps/readest-app/src/__tests__/services/wordwise/planner.test.ts Normal file
View file

@ -0,0 +1,85 @@
import { describe, it, expect } from 'vitest';
import { planGlosses } from '@/services/wordwise/planner';
import { GlossIndex } from '@/services/wordwise/glossIndex';
import { getRankCutoff } from '@/services/wordwise/difficulty';
import type { GlossSource, GlossEntry, GlossIndexData } from '@/services/wordwise/types';
import zhEnFixture from '../../fixtures/wordwise/zh-en.fixture.json';
const source: GlossSource = {
lookup(word) {
const table: Record<string, GlossEntry> = {
cryptic: { rank: 18000, gloss: '晦涩的' },
running: { rank: 312, gloss: '跑' },
the: { rank: 1, gloss: '这' },
斟酌: { rank: 9000, gloss: 'to consider' },
};
return table[word.toLowerCase()] ?? table[word] ?? null;
},
};
describe('planGlosses (English)', () => {
it('glosses words at/above the cutoff with correct offsets', () => {
const text = 'A cryptic note.';
const occ = planGlosses(text, source, { sourceLang: 'en', rankCutoff: 7000 });
expect(occ).toEqual([{ start: 2, end: 9, word: 'cryptic', gloss: '晦涩的' }]);
expect(text.slice(2, 9)).toBe('cryptic');
});
it('skips words below the cutoff', () => {
const occ = planGlosses('the cryptic', source, { sourceLang: 'en', rankCutoff: 7000 });
expect(occ.map((o) => o.word)).toEqual(['cryptic']);
});
it('respects the per-call occurrence cap', () => {
const occ = planGlosses('cryptic cryptic cryptic', source, {
sourceLang: 'en',
rankCutoff: 7000,
maxOccurrences: 2,
});
expect(occ).toHaveLength(2);
});
});
describe('planGlosses (Chinese)', () => {
it('uses the injected segmenter and locates segments by cursor', () => {
const text = '请你斟酌一下';
const cutZh = (t: string) => ['请', '你', '斟酌', '一下'].filter((w) => t.includes(w));
const occ = planGlosses(text, source, {
sourceLang: 'zh',
rankCutoff: 7000,
cutZh,
});
expect(occ).toEqual([{ start: 2, end: 4, word: '斟酌', gloss: 'to consider' }]);
expect(text.slice(2, 4)).toBe('斟酌');
});
});
describe('planGlosses against a zh-en fixture', () => {
// Decoupled from the shipping data/wordwise/zh-en.json: the committed pack is
// (re)generated from real corpora, so the test owns its ranks via a fixture.
const data = zhEnFixture as GlossIndexData;
const index = GlossIndex.fromData(data);
// A real headword from the committed starter set + its rank.
const word = '斟酌';
const entry = data.entries[word]!;
// Stub segmenter that segments the starter word out of the sentence.
const cutZh = (t: string) => ['请', '你', word, '一下'].filter((w) => t.includes(w));
const text = `请你${word}一下`;
it('glosses the fixture word at a beginner level (A1 => most hints, low cutoff)', () => {
const occ = planGlosses(text, index, {
sourceLang: 'zh',
rankCutoff: getRankCutoff('zh', 1), // A1
cutZh,
});
expect(occ.map((o) => o.word)).toContain(word);
expect(occ.find((o) => o.word === word)?.gloss).toBe(entry.g);
});
it('does NOT gloss it at an advanced level (C2 => fewest hints, high cutoff)', () => {
const cutoff = getRankCutoff('zh', 6); // C2
// Guard: the fixture word must sit below the C2 cutoff for this to be meaningful.
expect(entry.r).toBeLessThan(cutoff);
const occ = planGlosses(text, index, { sourceLang: 'zh', rankCutoff: cutoff, cutZh });
expect(occ.map((o) => o.word)).not.toContain(word);
});
});

61
apps/readest-app/src/__tests__/utils/epubcfi-ruby-inline.test.ts Normal file
View file

@ -0,0 +1,61 @@
import { describe, it, expect } from 'vitest';
import * as CFI from 'foliate-js/epubcfi.js';
const XHTML = (str: string) => new DOMParser().parseFromString(str, 'application/xhtml+xml');
const rangeInParagraph = (doc: Document, pId: string, start: number, end: number): Range => {
const p = doc.getElementById(pId)!;
const walker = doc.createTreeWalker(p, NodeFilter.SHOW_TEXT);
const node = walker.nextNode() as Text;
const range = doc.createRange();
range.setStart(node, start);
range.setEnd(node, end);
return range;
};
const wrap = (html: string) =>
XHTML(`<html xmlns="http://www.w3.org/1999/xhtml"><body id="b">${html}</body></html>`);
describe('epubcfi mid-text inline ruby transparency', () => {
it('produces the same CFI for a glossed word as for plain text', () => {
const plain = wrap(`<p id="p">The quick fox</p>`);
const plainCfi = CFI.fromRange(rangeInParagraph(plain, 'p', 4, 9));
const ruby = wrap(
`<p id="p">The <ruby cfi-skip="">quick<rt cfi-inert="">x</rt></ruby> fox</p>`,
);
const p = ruby.getElementById('p')!;
const rb = p.querySelector('ruby')!.firstChild as Text; // "quick"
const r = ruby.createRange();
r.setStart(rb, 0);
r.setEnd(rb, 5);
expect(CFI.fromRange(r)).toBe(plainCfi);
});
it('round-trips a saved CFI through the glossed DOM to the right text', () => {
const plain = wrap(`<p id="p">The quick fox</p>`);
const cfi = CFI.fromRange(rangeInParagraph(plain, 'p', 4, 9)); // "quick"
const ruby = wrap(
`<p id="p">The <ruby cfi-skip="">quick<rt cfi-inert="">x</rt></ruby> fox</p>`,
);
const resolved = CFI.toRange(ruby, CFI.parse(cfi));
expect(resolved.toString()).toBe('quick');
});
it('keeps offsets stable for text AFTER the gloss', () => {
const plain = wrap(`<p id="p">The quick fox</p>`);
const plainCfi = CFI.fromRange(rangeInParagraph(plain, 'p', 10, 13));
const ruby = wrap(
`<p id="p">The <ruby cfi-skip="">quick<rt cfi-inert="">x</rt></ruby> fox</p>`,
);
const p = ruby.getElementById('p')!;
const after = p.querySelector('ruby')!.nextSibling as Text; // " fox"
const r = ruby.createRange();
r.setStart(after, 1);
r.setEnd(after, 4);
expect(CFI.fromRange(r)).toBe(plainCfi);
});
});

57
apps/readest-app/src/app/reader/components/FoliateViewer.tsx
View file

@ -41,6 +41,7 @@ import {
import { applyScrollableStyle, applyTableTouchScroll } from '@/utils/scrollable';
import { mountAdditionalFonts, mountCustomFont } from '@/styles/fonts';
import { layoutWarichu, relayoutWarichu } from '@/utils/warichu';
import { refreshSectionGlosses } from '@/app/reader/utils/wordwiseSection';
import { getBookDirFromLanguage, getBookDirFromWritingMode } from '@/utils/book';
import { getIndexFromCfi } from '@/utils/cfi';
import { useUICSS } from '@/hooks/useUICSS';
@ -70,6 +71,8 @@ import { getViewInsets } from '@/utils/insets';
import { handleA11yNavigation } from '@/utils/a11y';
import { isCJKLang } from '@/utils/lang';
import { getLocale } from '@/utils/misc';
import { isMetered } from '@/utils/network';
import { eventDispatcher } from '@/utils/event';
import { isFontType } from '@/utils/font';
import { ParagraphControl } from './paragraph';
import Spinner from '@/components/Spinner';
@ -405,10 +408,44 @@ const FoliateViewer: React.FC<{
}
};
// Build the Word Wise refresh context: gate silent auto-download on the global
// toggle AND a best-effort metered-connection check, and show a single
// "Downloading…" toast on the first progress tick (the per-percent progress
// lives in the Word Wise settings panel). `wordWiseToastShownRef` de-dupes the
// toast across the multiple section docs a refresh pass touches.
const wordWiseToastShownRef = useRef(false);
const buildWordWiseCtx = (bookLang?: string | null) => {
// Read the live setting (not the first-render `settings` snapshot closed over
// by the empty-deps `stabilizedHandler`) so toggling Auto-download mid-session
// takes effect on the next section refresh.
const liveSettings = useSettingsStore.getState().settings;
const allowDownload =
(liveSettings.globalReadSettings.wordWiseAutoDownload ?? true) && !isMetered();
return {
appService: appService!,
bookLang,
appLang: getLocale().split('-')[0] || 'en',
allowDownload,
onProgress: () => {
if (wordWiseToastShownRef.current) return;
wordWiseToastShownRef.current = true;
eventDispatcher.dispatch('toast', {
type: 'info',
message: _('Downloading Word Wise data…'),
});
},
};
};
const stabilizedHandler = useCallback(() => {
setLoading(false);
// Layout/relayout warichu after paginator has set column-width via columnize()
const contents = viewRef.current?.renderer?.getContents?.() || [];
const vs = getViewSettings(bookKey);
const bookLang = getBookData(bookKey)?.book?.primaryLanguage;
// Fixed-layout (pre-paginated) books have no reflow room; injecting ruby
// would overflow their fixed boxes, so skip Word Wise glosses there.
const isFixedLayout = bookDoc.rendition?.layout === 'pre-paginated';
for (const { doc } of contents) {
if (doc) {
const hasPending = doc.querySelectorAll('.warichu-pending').length > 0;
@ -418,8 +455,12 @@ const FoliateViewer: React.FC<{
} else if (hasExisting) {
relayoutWarichu(doc);
}
if (vs && appService && !isFixedLayout) {
void refreshSectionGlosses(doc, vs, buildWordWiseCtx(bookLang));
}
}
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
const docRelocateHandler = (event: Event) => {
@ -781,6 +822,22 @@ const FoliateViewer: React.FC<{
viewSettings?.hideScrollbar,
]);
useEffect(() => {
const contents = viewRef.current?.renderer?.getContents?.() || [];
const vs = getViewSettings(bookKey);
if (!vs || !appService) return;
const bookLang = getBookData(bookKey)?.book?.primaryLanguage;
const isFixedLayout = bookDoc.rendition?.layout === 'pre-paginated';
if (isFixedLayout) return;
// A settings change is the moment a fresh download may start; let the
// one-time "Downloading…" toast fire again for it.
wordWiseToastShownRef.current = false;
for (const { doc } of contents) {
if (doc) void refreshSectionGlosses(doc, vs, buildWordWiseCtx(bookLang));
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [viewSettings?.wordWiseEnabled, viewSettings?.wordWiseLevel, viewSettings?.wordWiseHintLang]);
useEffect(() => {
const mountCustomFonts = async () => {
await loadCustomFonts(envConfig);

55
apps/readest-app/src/app/reader/components/annotator/Annotator.tsx
View file

@ -174,6 +174,10 @@ const Annotator: React.FC<{ bookKey: string; contentInsets: Insets }> = ({
// (Android long-press selects text via selectionchange before touchend). The
// pending action runs on touchend so popups don't open under an active touch.
const deferredQuickActionRef = useRef(createDeferredActionState());
// Set when a Word Wise gloss tap synthesizes a selection so the
// selection-change effect opens the dictionary popup instead of the
// annotation toolbar. Cleared as soon as it's consumed.
const pendingWordWiseDictRef = useRef(false);
const showingPopup =
showAnnotPopup || showDictionaryPopup || showDeepLPopup || showProofreadPopup;
@ -556,6 +560,47 @@ const Annotator: React.FC<{ bookKey: string; contentInsets: Insets }> = ({
useFoliateEvents(view, { onLoad, onCreateOverlay, onDrawAnnotation, onShowAnnotation });
// Word Wise: open the dictionary popup for a tapped glossed word. The tap is
// detected in the iframe click handler (iframeEventHandlers.ts), which sends
// the gloss <ruby> element here. We synthesize a selection over the base word
// (excluding the <rt> hint) so the existing dictionary popup positions itself.
useEffect(() => {
const handleWordWiseDictionary = (event: CustomEvent) => {
const { element, word } = event.detail as { element: Element | null; word: string };
if (event.detail?.bookKey !== bookKey || !element || !word) return;
// Read the view fresh: this handler is registered once (deps [bookKey]) and
// the closed-over `view` may still be null from when the effect first ran.
const view = getView(bookKey);
const doc = element.ownerDocument;
const content = view?.renderer?.getContents().find((c) => c.doc === doc);
if (!content || content.index == null) return;
const index = content.index;
const rt = element.querySelector('rt');
const range = doc.createRange();
try {
range.selectNodeContents(element);
if (rt) range.setEndBefore(rt);
} catch {
return;
}
const text = range.toString().trim() || word;
pendingWordWiseDictRef.current = true;
setSelection({
key: bookKey,
text,
range,
index,
cfi: view?.getCFI(index, range),
page: index + 1,
});
};
eventDispatcher.on('wordwise-dictionary', handleWordWiseDictionary);
return () => {
eventDispatcher.off('wordwise-dictionary', handleWordWiseDictionary);
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [bookKey]);
useEffect(() => {
handleShowPopup(showingPopup);
// eslint-disable-next-line react-hooks/exhaustive-deps
@ -772,6 +817,11 @@ const Annotator: React.FC<{ bookKey: string; contentInsets: Insets }> = ({
useEffect(() => {
setHighlightOptionsVisible(!!(selection && selection.annotated));
if (selection && selection.text.trim().length > 0) {
// Read-and-reset the Word Wise dictionary flag up front so it can never
// stick to a later selection if an early return below fires (e.g. a gloss
// tap whose synthesized range yields an off-frame/zero position).
const wantWordWiseDict = pendingWordWiseDictRef.current;
pendingWordWiseDictRef.current = false;
const gridFrame = document.querySelector(`#gridcell-${bookKey}`);
if (!gridFrame) return;
const rect = gridFrame.getBoundingClientRect();
@ -816,7 +866,10 @@ const Annotator: React.FC<{ bookKey: string; contentInsets: Insets }> = ({
setTrianglePosition(triangPos);
const { enableAnnotationQuickActions, annotationQuickAction } = viewSettings;
if (enableAnnotationQuickActions && annotationQuickAction && isTextSelected.current) {
if (wantWordWiseDict) {
setShowAnnotPopup(false);
setShowDictionaryPopup(true);
} else if (enableAnnotationQuickActions && annotationQuickAction && isTextSelected.current) {
handleQuickAction();
} else {
handleShowAnnotPopup();

104
apps/readest-app/src/app/reader/components/rsvp/RSVPOverlay.tsx
View file

@ -1070,41 +1070,11 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
</div>
</div>
{/* Playback controls — a single full-width flex row on mobile so the
audio toggle (far left) and settings gear (far right) flank the
centered transport in normal flow instead of overlapping it from an
absolute corner; a centered cluster on md+ where there is room. */}
<div className='flex items-center justify-between md:justify-center md:gap-2'>
{/* Audio (TTS) read-along toggle — starts TTS from the displayed word,
or stops it when engaged (decision 5, #3235). Far-left peer of the
transport so the centered play button stays centered and nothing
overlaps on mobile. Active state uses a filled glyph + eink-bordered
surface so it reads in e-ink without relying on color. */}
<button
aria-label={ttsActive ? _('Pause audio') : _('Play audio')}
className={clsx(
'touch-target flex h-8 w-8 shrink-0 cursor-pointer items-center justify-center rounded-full border-none transition-colors active:scale-95 md:h-9 md:w-9',
ttsActive
? 'eink-bordered bg-[color-mix(in_srgb,var(--rsvp-accent)_18%,transparent)]'
: 'bg-transparent hover:bg-gray-500/20',
)}
onClick={() => onToggleTtsAudio?.()}
title={ttsActive ? _('Pause audio') : _('Play audio')}
>
{ttsActive ? (
<IoVolumeHigh
className='h-4 w-4 md:h-5 md:w-5'
style={{ color: accentColor }}
aria-hidden='true'
/>
) : (
<IoVolumeMediumOutline className='h-4 w-4 md:h-5 md:w-5' aria-hidden='true' />
)}
</button>
{/* Playback controls */}
<div className='relative flex items-center justify-center gap-1 md:gap-2'>
<button
aria-label={_('Skip back 15 words')}
className='flex shrink-0 cursor-pointer items-center gap-0.5 rounded-full border-none bg-transparent px-1.5 py-1.5 transition-colors hover:bg-gray-500/20 active:scale-95 md:px-2'
className='flex cursor-pointer items-center gap-0.5 rounded-full border-none bg-transparent px-2 py-1.5 transition-colors hover:bg-gray-500/20 active:scale-95'
onClick={() => controller.skipBackward(15)}
title={_('Back 15 words (Shift+Left)')}
>
@ -1114,7 +1084,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
<button
aria-label={_('Decrease speed')}
className='flex h-8 w-8 shrink-0 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95 md:h-9 md:w-9'
className='flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95'
onClick={() => controller.decreaseSpeed()}
title={_('Slower (Left/Down)')}
>
@ -1123,7 +1093,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
<button
aria-label={_('Previous word')}
className='flex h-8 w-8 shrink-0 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95 md:h-9 md:w-9'
className='flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95'
onClick={() => controller.prevWord()}
title={_('Previous word (,)')}
>
@ -1133,7 +1103,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
<button
aria-label={transportPlaying ? _('Pause') : _('Play')}
className={clsx(
'flex h-14 w-14 shrink-0 cursor-pointer items-center justify-center rounded-full border-none bg-gray-500/15 transition-colors hover:bg-gray-500/25 active:scale-95 md:h-16 md:w-16',
'flex h-14 w-14 cursor-pointer items-center justify-center rounded-full border-none bg-gray-500/15 transition-colors hover:bg-gray-500/25 active:scale-95 md:h-16 md:w-16',
transportPlaying ? '' : 'ps-1',
)}
onClick={() => transportToggleRef.current()}
@ -1148,7 +1118,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
<button
aria-label={_('Next word')}
className='flex h-8 w-8 shrink-0 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95 md:h-9 md:w-9'
className='flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95'
onClick={() => controller.nextWord()}
title={_('Next word (.)')}
>
@ -1157,7 +1127,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
<button
aria-label={_('Increase speed')}
className='flex h-8 w-8 shrink-0 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95 md:h-9 md:w-9'
className='flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95'
onClick={() => controller.increaseSpeed()}
title={_('Faster (Right/Up)')}
>
@ -1166,7 +1136,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
<button
aria-label={_('Skip forward 15 words')}
className='flex shrink-0 cursor-pointer items-center gap-0.5 rounded-full border-none bg-transparent px-1.5 py-1.5 transition-colors hover:bg-gray-500/20 active:scale-95 md:px-2'
className='flex cursor-pointer items-center gap-0.5 rounded-full border-none bg-transparent px-2 py-1.5 transition-colors hover:bg-gray-500/20 active:scale-95'
onClick={() => controller.skipForward(15)}
title={_('Forward 15 words (Shift+Right)')}
>
@ -1174,20 +1144,48 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
<span className='text-xs font-semibold opacity-80'>15</span>
</button>
{/* Settings — far-right peer mirroring the audio toggle on the left,
so both flank the centered transport in normal flow (decision 5,
#3235) without an absolute cluster overlapping it on mobile. */}
<button
aria-label={_('Settings')}
className={clsx(
'flex h-8 w-8 shrink-0 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95 md:h-9 md:w-9',
showSettings && 'bg-gray-500/15',
)}
onClick={() => setShowSettings((prev) => !prev)}
title={_('Settings')}
>
<IoSettingsSharp className='h-4 w-4 md:h-5 md:w-5' />
</button>
{/* Trailing cluster: audio (TTS) toggle + divider + settings gear.
The audio toggle starts TTS from the displayed word (or stops it
when engaged) — never a second play triangle (decision 5). Active
state uses a filled glyph + eink-bordered surface so it reads in
e-ink without relying on color. */}
<div className='absolute end-0 flex items-center gap-1'>
<button
aria-label={ttsActive ? _('Pause audio') : _('Play audio')}
className={clsx(
'touch-target flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none transition-colors active:scale-95',
ttsActive
? 'eink-bordered bg-[color-mix(in_srgb,var(--rsvp-accent)_18%,transparent)]'
: 'bg-transparent hover:bg-gray-500/20',
)}
onClick={() => onToggleTtsAudio?.()}
title={ttsActive ? _('Pause audio') : _('Play audio')}
>
{ttsActive ? (
<IoVolumeHigh
className='h-4 w-4 md:h-5 md:w-5'
style={{ color: accentColor }}
aria-hidden='true'
/>
) : (
<IoVolumeMediumOutline className='h-4 w-4 md:h-5 md:w-5' aria-hidden='true' />
)}
</button>
<span className='h-5 w-px bg-gray-500/30' aria-hidden='true' />
<button
aria-label={_('Settings')}
className={clsx(
'flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95',
showSettings && 'bg-gray-500/15',
)}
onClick={() => setShowSettings((prev) => !prev)}
title={_('Settings')}
>
<IoSettingsSharp className='h-4 w-4 md:h-5 md:w-5' />
</button>
</div>
</div>
{/* Settings row (collapsible) */}

2
apps/readest-app/src/app/reader/components/sidebar/SearchBar.tsx
View file

@ -247,6 +247,8 @@ const SearchBar: React.FC<SearchBarProps> = ({ isVisible, bookKey, onHideSearchB
query: term,
acceptNode: createRejectFilter({
tags: primaryLang.startsWith('ja') ? ['rt'] : [],
// Word Wise gloss text (<rt cfi-inert>) is injected, non-book content.
attributes: ['cfi-inert'],
}),
results: cachedResults,
});

10
apps/readest-app/src/app/reader/utils/iframeEventHandlers.ts
View file

@ -1,5 +1,6 @@
import { DOUBLE_CLICK_INTERVAL_THRESHOLD_MS, LONG_HOLD_THRESHOLD } from '@/services/constants';
import { eventDispatcher } from '@/utils/event';
import { findGlossWord } from '@/app/reader/utils/wordwiseRuby';
let lastClickTime = 0;
let longHoldTimeout: ReturnType<typeof setTimeout> | null = null;
@ -224,6 +225,15 @@ export const handleClick = (
return;
}
// Word Wise: tapping a glossed word looks it up in the dictionary. Checked
// after the drag/long-hold guards so only a clean single tap triggers it.
const glossWord = findGlossWord(element);
if (glossWord) {
const ruby = element?.closest('ruby.ww-gloss') ?? null;
eventDispatcher.dispatch('wordwise-dictionary', { bookKey, element: ruby, word: glossWord });
return;
}
window.postMessage(
{
type: 'iframe-single-click',

128
apps/readest-app/src/app/reader/utils/wordwiseRuby.ts Normal file
View file

@ -0,0 +1,128 @@
import type { GlossOccurrence } from '@/services/wordwise/types';
const GLOSS_CLASS = 'ww-gloss';
interface Segment {
node: Text;
/** Offset of this node's text within the concatenated model string. */
start: number;
}
export interface SectionTextModel {
text: string;
locate(offset: number): { node: Text; offset: number };
}
const isEligible = (node: Text): boolean => {
let p: Node | null = node.parentNode;
while (p) {
if (p.nodeType === Node.ELEMENT_NODE) {
const tag = (p as Element).tagName.toLowerCase();
if (tag === 'rt' || tag === 'rp' || tag === 'ruby' || tag === 'script' || tag === 'style') {
return false;
}
}
p = p.parentNode;
}
return true;
};
/** Concatenate eligible text nodes and remember each node's slice offset. */
export const buildSectionTextModel = (doc: Document): SectionTextModel => {
const root = doc.body ?? doc.documentElement;
const segments: Segment[] = [];
let text = '';
if (root) {
const walker = doc.createTreeWalker(root, NodeFilter.SHOW_TEXT, {
acceptNode: (n) =>
isEligible(n as Text) ? NodeFilter.FILTER_ACCEPT : NodeFilter.FILTER_REJECT,
});
for (let n = walker.nextNode(); n; n = walker.nextNode()) {
const t = n as Text;
const data = t.data;
if (!data) continue;
segments.push({ node: t, start: text.length });
text += data;
}
}
const locate = (offset: number) => {
let lo = 0;
let hi = segments.length - 1;
let seg = segments[0]!;
while (lo <= hi) {
const mid = (lo + hi) >> 1;
const s = segments[mid]!;
if (offset < s.start) hi = mid - 1;
else {
seg = s;
lo = mid + 1;
}
}
return { node: seg.node, offset: offset - seg.start };
};
return { text, locate };
};
const occurrenceRange = (
doc: Document,
model: SectionTextModel,
occ: GlossOccurrence,
): Range | null => {
const s = model.locate(occ.start);
const e = model.locate(occ.end);
if (s.node !== e.node) return null;
const range = doc.createRange();
range.setStart(s.node, s.offset);
range.setEnd(e.node, e.offset);
return range;
};
/** Wrap each occurrence as <ruby class=ww-gloss cfi-skip>word<rt cfi-inert>gloss</rt></ruby>. */
export const applyGlosses = (
doc: Document,
model: SectionTextModel,
occurrences: GlossOccurrence[],
): void => {
const sorted = [...occurrences].sort((a, b) => b.start - a.start);
for (const occ of sorted) {
const range = occurrenceRange(doc, model, occ);
if (!range) continue;
const ruby = doc.createElement('ruby');
ruby.className = GLOSS_CLASS;
ruby.setAttribute('cfi-skip', '');
const rt = doc.createElement('rt');
rt.setAttribute('cfi-inert', '');
rt.textContent = occ.gloss;
try {
const word = range.extractContents();
ruby.appendChild(word);
ruby.appendChild(rt);
range.insertNode(ruby);
} catch {
// Range became invalid (concurrent mutation); skip this one.
}
}
};
/** Unwrap every injected gloss, restoring the original text. */
export const clearGlosses = (doc: Document): void => {
const rubies = doc.querySelectorAll(`ruby.${GLOSS_CLASS}`);
rubies.forEach((ruby) => {
ruby.querySelectorAll('rt').forEach((rt) => rt.remove());
const parent = ruby.parentNode;
if (!parent) return;
while (ruby.firstChild) parent.insertBefore(ruby.firstChild, ruby);
parent.removeChild(ruby);
});
(doc.body ?? doc.documentElement)?.normalize();
};
/** Given a tap target, return the base word if it is inside a gloss, else null. */
export const findGlossWord = (target: HTMLElement | null): string | null => {
const ruby = target?.closest(`ruby.${GLOSS_CLASS}`);
if (!ruby) return null;
const clone = ruby.cloneNode(true) as Element;
clone.querySelectorAll('rt').forEach((rt) => rt.remove());
const word = clone.textContent?.trim() ?? '';
return word || null;
};

77
apps/readest-app/src/app/reader/utils/wordwiseSection.ts Normal file
View file

@ -0,0 +1,77 @@
import type { ViewSettings } from '@/types/book';
import type { AppService } from '@/types/system';
import type { ProgressHandler } from '@/utils/transfer';
import { canTokenizeSource, getRankCutoff } from '@/services/wordwise/difficulty';
import { loadGlossIndex } from '@/services/wordwise/glossPacks';
import { planGlosses } from '@/services/wordwise/planner';
import { buildSectionTextModel, applyGlosses, clearGlosses } from '@/app/reader/utils/wordwiseRuby';
import { cutZh, isJiebaReady, initJieba } from '@/utils/jieba';
/** Normalize a book language tag to its 2-letter base source code, or null. */
export const toWordWiseSource = (lang?: string | null): string | null => {
if (!lang) return null;
const base = lang.toLowerCase().split('-')[0];
return base || null;
};
interface RefreshContext {
appService: AppService;
bookLang?: string | null;
/** App UI language base code, used as the hint when none is selected. */
appLang: string;
/**
* Whether the reader may silently download an uncached pack. Threaded to
* loadGlossIndex → ensurePack; when false an uncached pack yields no glosses
* (the user downloads it explicitly from the Word Wise sub-page).
*/
allowDownload?: boolean;
onProgress?: ProgressHandler;
}
// Per-document generation counter. Dragging the difficulty slider fires the
// settings effect repeatedly, producing overlapping refresh calls. A later call
// supersedes earlier ones: each call stamps its generation, and any call whose
// stamp is stale after the `await` bails before touching the DOM, so the latest
// refresh always builds and applies against a clean, un-glossed document.
const refreshGen = new WeakMap<Document, number>();
/** Re-render glosses for one section doc. Clears first, then injects if enabled. */
export const refreshSectionGlosses = async (
doc: Document,
viewSettings: ViewSettings,
ctx: RefreshContext,
): Promise<void> => {
// This runs fire-and-forget (`void refreshSectionGlosses(...)`), and its
// post-await synchronous DOM work (buildSectionTextModel / planGlosses /
// applyGlosses → range.insertNode) can throw. Contain everything so a failure
// never escapes as an unhandledrejection.
try {
const myGen = (refreshGen.get(doc) ?? 0) + 1;
refreshGen.set(doc, myGen);
clearGlosses(doc);
if (!viewSettings.wordWiseEnabled) return;
const source = toWordWiseSource(ctx.bookLang);
if (!source || !canTokenizeSource(source)) return;
const hint = (viewSettings.wordWiseHintLang || ctx.appLang).toLowerCase().split('-')[0] || '';
if (!hint || hint === source) return; // no self-gloss
const index = await loadGlossIndex(ctx.appService, source, hint, {
onProgress: ctx.onProgress,
allowDownload: ctx.allowDownload,
});
if (refreshGen.get(doc) !== myGen) return; // a newer refresh superseded us
if (!index) return;
if (source === 'zh' && !isJiebaReady()) {
void initJieba();
return;
}
const model = buildSectionTextModel(doc);
const occ = planGlosses(model.text, index, {
sourceLang: source,
rankCutoff: getRankCutoff(source, viewSettings.wordWiseLevel),
cutZh: source === 'zh' ? cutZh : undefined,
});
if (occ.length) applyGlosses(doc, model, occ);
} catch (err) {
console.warn('[wordwise] refresh failed', err);
}
};

32
apps/readest-app/src/components/settings/LangPanel.tsx
View file

@ -26,6 +26,8 @@ import {
SettingsSwitchRow,
} from './primitives';
import CustomDictionaries from './CustomDictionaries';
import WordWisePanel from './WordWisePanel';
import { PiTranslate } from 'react-icons/pi';
const LangPanel: React.FC<SettingsPanelPanelProp> = ({ bookKey, onRegisterReset }) => {
const _ = useTranslation();
@ -50,15 +52,20 @@ const LangPanel: React.FC<SettingsPanelPanelProp> = ({ bookKey, onRegisterReset
viewSettings.convertChineseVariant,
);
const [showCustomDictionaries, setShowCustomDictionaries] = useState(false);
const [showWordWise, setShowWordWise] = useState(false);
// Android Back / Esc: when the Manage Dictionaries sub-page is open,
// intercept and step back to the language list instead of letting
// <Dialog>'s listener close the whole Settings dialog. See the matching
// comment in FontPanel.tsx for the LIFO-dispatch reasoning.
// Android Back / Esc: when a sub-page is open, intercept and step back to the
// language list instead of letting <Dialog>'s listener close the whole
// Settings dialog. See the matching comment in FontPanel.tsx for the
// LIFO-dispatch reasoning.
useKeyDownActions({
enabled: showCustomDictionaries,
onCancel: () => setShowCustomDictionaries(false),
});
useKeyDownActions({
enabled: showWordWise,
onCancel: () => setShowWordWise(false),
});
// Deep-link: callers (e.g. the dictionary popup's manage icon) can set
// activeSettingsItemId to `'settings.language.dictionaries.manage'` to
@ -281,6 +288,10 @@ const LangPanel: React.FC<SettingsPanelPanelProp> = ({ bookKey, onRegisterReset
);
}
if (showWordWise) {
return <WordWisePanel bookKey={bookKey} onBack={() => setShowWordWise(false)} />;
}
return (
<div className={clsx('my-4 w-full space-y-6')}>
<BoxedList title={_('Language')} data-setting-id='settings.language.interfaceLanguage'>
@ -306,6 +317,19 @@ const LangPanel: React.FC<SettingsPanelPanelProp> = ({ bookKey, onRegisterReset
/>
</BoxedList>
<BoxedList
title={_('Word Wise')}
data-setting-id='settings.language.wordwise'
cardClassName='overflow-hidden'
>
<NavigationRow
icon={PiTranslate}
title={_('Word Wise')}
status={_('Show a short native-language hint above difficult words.')}
onClick={() => setShowWordWise(true)}
/>
</BoxedList>
<BoxedList title={_('Translation')} data-setting-id='settings.language.translationEnabled'>
<SettingsSwitchRow
label={_('Enable Translation')}

340
apps/readest-app/src/components/settings/WordWisePanel.tsx Normal file
View file

@ -0,0 +1,340 @@
import clsx from 'clsx';
import React, { useEffect, useState } from 'react';
import { useEnv } from '@/context/EnvContext';
import { useReaderStore } from '@/store/readerStore';
import { useBookDataStore } from '@/store/bookDataStore';
import { useSettingsStore } from '@/store/settingsStore';
import { useTranslation } from '@/hooks/useTranslation';
import { saveViewSettings } from '@/helpers/settings';
import { getLocale } from '@/utils/misc';
import { formatBytes } from '@/utils/book';
import { eventDispatcher } from '@/utils/event';
import { TRANSLATED_LANGS } from '@/services/constants';
import {
WORD_WISE_MIN_LEVEL,
WORD_WISE_MAX_LEVEL,
cefrLabel,
} from '@/services/wordwise/difficulty';
import { toWordWiseSource } from '@/app/reader/utils/wordwiseSection';
import {
deletePack,
ensurePack,
fetchManifest,
getPackStatus,
listAvailableTargets,
type WordWiseManifest,
type WordWisePack,
} from '@/services/wordwise/glossPacks';
import SubPageHeader from './SubPageHeader';
import { BoxedList, SettingsRow, SettingsSelect, SettingsSwitchRow } from './primitives';
interface WordWisePanelProps {
bookKey: string;
onBack: () => void;
}
const baseCode = (lang?: string | null): string => (lang || '').toLowerCase().split('-')[0] || '';
const WordWisePanel: React.FC<WordWisePanelProps> = ({ bookKey, onBack }) => {
const _ = useTranslation();
const { envConfig, appService } = useEnv();
const { getViewSettings, setViewSettings } = useReaderStore();
const { getBookData } = useBookDataStore();
const { settings, setSettings, saveSettings } = useSettingsStore();
const viewSettings = getViewSettings(bookKey) || settings.globalViewSettings;
const bookData = getBookData(bookKey);
const appLang = baseCode(getLocale());
const bookSource = toWordWiseSource(bookData?.book?.primaryLanguage);
const [wordWiseEnabled, setWordWiseEnabled] = useState(viewSettings.wordWiseEnabled ?? false);
const [wordWiseLevel, setWordWiseLevel] = useState(viewSettings.wordWiseLevel ?? 3);
const [hintLang, setHintLang] = useState(viewSettings.wordWiseHintLang || appLang);
const [autoDownload, setAutoDownload] = useState(
settings.globalReadSettings.wordWiseAutoDownload ?? true,
);
const [manifest, setManifest] = useState<WordWiseManifest | null>(null);
const [packStatus, setPackStatus] = useState<{ pack: WordWisePack; downloaded: boolean } | null>(
null,
);
const [resolving, setResolving] = useState(false);
const [downloading, setDownloading] = useState(false);
const [progress, setProgress] = useState<number | null>(null);
// Fetch the manifest once on mount to filter the hint-language selector and
// resolve the data-pack row. If it fails, the selector falls back to the full
// TRANSLATED_LANGS list (see availableTargets below).
useEffect(() => {
if (!appService) return;
let cancelled = false;
void fetchManifest(appService).then((m) => {
if (!cancelled) setManifest(m);
});
return () => {
cancelled = true;
};
}, [appService]);
useEffect(() => {
if (wordWiseEnabled === viewSettings.wordWiseEnabled) return;
saveViewSettings(envConfig, bookKey, 'wordWiseEnabled', wordWiseEnabled, false, false);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [wordWiseEnabled]);
useEffect(() => {
if (wordWiseLevel === viewSettings.wordWiseLevel) return;
saveViewSettings(envConfig, bookKey, 'wordWiseLevel', wordWiseLevel, false, false);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [wordWiseLevel]);
// Re-resolve the data-pack row whenever the (source → hint) pair changes.
useEffect(() => {
if (!appService || !bookSource) {
setPackStatus(null);
return;
}
const hint = baseCode(hintLang) || appLang;
if (!hint || hint === bookSource) {
setPackStatus(null);
return;
}
let cancelled = false;
setResolving(true);
void getPackStatus(appService, bookSource, hint)
.then((status) => {
if (!cancelled) setPackStatus(status);
})
.finally(() => {
if (!cancelled) setResolving(false);
});
return () => {
cancelled = true;
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [appService, bookSource, hintLang, appLang, manifest]);
// The manifest's `target` codes are 2-letter base codes (en, zh, fr…), while
// TRANSLATED_LANGS keys can be regional (zh-CN, pt-BR). Match by base code so
// e.g. an en→zh pack surfaces the "简体中文"/"正體中文" options.
const availableTargets = manifest && bookSource ? listAvailableTargets(manifest, bookSource) : [];
const getHintLangOptions = () => {
const entries = Object.entries(TRANSLATED_LANGS).map(([value, label]) => ({ value, label }));
// When the manifest is available, restrict to the targets it offers for the
// book's source language (matching the resolvable packs). Without a manifest
// we show the full list so the selector stays usable offline.
const filtered =
availableTargets.length > 0
? entries.filter((o) => availableTargets.includes(baseCode(o.value)))
: entries;
filtered.sort((a, b) => a.label.localeCompare(b.label));
return [{ value: '', label: _('Auto') }, ...filtered];
};
const hintLangOptions = getHintLangOptions();
// Map the stored hint (possibly '' = auto, or a base code like 'zh') to the
// option that should appear selected. '' shows Auto; a base code resolves to
// the first option whose base code matches (e.g. 'zh' → 'zh-CN').
const selectedHintValue = (() => {
const stored = viewSettings.wordWiseHintLang;
if (!stored) return '';
if (hintLangOptions.some((o) => o.value === stored)) return stored;
const byBase = hintLangOptions.find((o) => baseCode(o.value) === baseCode(stored));
return byBase?.value ?? '';
})();
const handleSelectHintLang = (event: React.ChangeEvent<HTMLSelectElement>) => {
const option = event.target.value;
setHintLang(option || appLang);
saveViewSettings(envConfig, bookKey, 'wordWiseHintLang', option, false, false);
viewSettings.wordWiseHintLang = option;
setViewSettings(bookKey, { ...viewSettings });
};
const handleToggleAutoDownload = () => {
const next = !autoDownload;
setAutoDownload(next);
settings.globalReadSettings.wordWiseAutoDownload = next;
setSettings(settings);
saveSettings(envConfig, settings);
};
const handleDownload = async () => {
if (!appService || !packStatus || downloading) return;
setDownloading(true);
setProgress(0);
try {
const path = await ensurePack(appService, packStatus.pack, {
allowDownload: true,
onProgress: ({ progress: done, total }) => {
if (total > 0) setProgress(Math.floor((done / total) * 100));
},
});
if (path) {
setPackStatus({ ...packStatus, downloaded: true });
eventDispatcher.dispatch('toast', {
type: 'success',
message: _('Word Wise data downloaded'),
});
} else {
eventDispatcher.dispatch('toast', {
type: 'error',
message: _('Failed to download Word Wise data'),
});
}
} catch {
eventDispatcher.dispatch('toast', {
type: 'error',
message: _('Failed to download Word Wise data'),
});
} finally {
setDownloading(false);
setProgress(null);
}
};
const handleDelete = async () => {
if (!appService || !packStatus) return;
await deletePack(appService, packStatus.pack);
setPackStatus({ ...packStatus, downloaded: false });
eventDispatcher.dispatch('toast', { type: 'info', message: _('Word Wise data removed') });
};
const renderDataPackRow = () => {
if (!bookSource) {
return (
<SettingsRow label={_('Data pack')}>
<span className='text-base-content/60 settings-content text-end'>
{_('Open a book to manage its data pack.')}
</span>
</SettingsRow>
);
}
if (resolving) {
return (
<SettingsRow label={_('Data pack')}>
<span className='loading loading-spinner loading-sm' />
</SettingsRow>
);
}
if (!packStatus) {
return (
<SettingsRow label={_('Data pack')}>
<span className='text-base-content/60 settings-content text-end'>
{_('No data available for this language pair yet.')}
</span>
</SettingsRow>
);
}
const size = formatBytes(packStatus.pack.bytes);
if (packStatus.downloaded) {
return (
<SettingsRow label={_('Data pack')} description={`${_('Downloaded')} · ${size}`}>
<button
type='button'
onClick={handleDelete}
className='btn btn-ghost btn-sm eink-bordered shrink-0'
>
{_('Delete')}
</button>
</SettingsRow>
);
}
return (
<SettingsRow label={_('Data pack')}>
<div className='flex items-center gap-2'>
{downloading && progress !== null && progress > 0 && (
<div
className='radial-progress flex items-center justify-center'
style={
{
'--value': progress,
'--size': '2.25rem',
fontSize: '0.6rem',
lineHeight: '0.8rem',
} as React.CSSProperties
}
aria-valuenow={progress}
role='progressbar'
>
{progress}%
</div>
)}
<button
type='button'
onClick={handleDownload}
disabled={downloading}
className='btn btn-primary btn-sm shrink-0'
>
{_('Download {{size}}', { size })}
</button>
</div>
</SettingsRow>
);
};
return (
<div className={clsx('my-4 w-full space-y-6')}>
<SubPageHeader
parentLabel={_('Language')}
currentLabel={_('Word Wise')}
description={_('Show a short native-language hint above difficult words.')}
onBack={onBack}
/>
<BoxedList title={_('Word Wise')} data-setting-id='settings.wordwise.main'>
<SettingsSwitchRow
label={_('Enable Word Wise')}
checked={wordWiseEnabled}
onChange={() => setWordWiseEnabled(!wordWiseEnabled)}
data-setting-id='settings.wordwise.enabled'
/>
<SettingsRow
label={_('Vocabulary level')}
description={_('Words above your level get a hint')}
disabled={!wordWiseEnabled}
>
<div className='flex items-center gap-2'>
<input
type='range'
className='range range-sm eink-bordered'
min={WORD_WISE_MIN_LEVEL}
max={WORD_WISE_MAX_LEVEL}
step={1}
value={wordWiseLevel}
disabled={!wordWiseEnabled}
aria-label={_('Vocabulary level')}
onChange={(e) => setWordWiseLevel(Number(e.target.value))}
data-setting-id='settings.wordwise.level'
/>
<span className='text-base-content/70 w-6 text-end text-sm tabular-nums'>
{cefrLabel(wordWiseLevel)}
</span>
</div>
</SettingsRow>
<SettingsRow label={_('Hint Language')}>
<SettingsSelect
value={selectedHintValue}
onChange={handleSelectHintLang}
ariaLabel={_('Hint Language')}
options={hintLangOptions}
/>
</SettingsRow>
</BoxedList>
<BoxedList title={_('Data')}>
{renderDataPackRow()}
<SettingsSwitchRow
label={_('Auto-download')}
description={_('Download data packs automatically when needed.')}
checked={autoDownload}
onChange={handleToggleAutoDownload}
/>
</BoxedList>
</div>
);
};
export default WordWisePanel;

8
apps/readest-app/src/services/constants.ts
View file

@ -15,6 +15,7 @@ import {
ViewConfig,
ViewSettings,
ViewSettingsConfig,
WordWiseConfig,
} from '@/types/book';
import {
HardcoverSettings,
@ -197,6 +198,7 @@ export const DEFAULT_READSETTINGS: ReadSettings = {
autohideCursor: true,
translationProvider: 'deepl',
translateTargetLang: 'EN',
wordWiseAutoDownload: true,
customThemes: [],
highlightStyle: 'highlight',
@ -402,6 +404,12 @@ export const DEFAULT_ANNOTATOR_CONFIG: AnnotatorConfig = {
noteExportConfig: DEFAULT_NOTE_EXPORT_CONFIG,
};
export const DEFAULT_WORD_WISE_CONFIG: WordWiseConfig = {
wordWiseEnabled: false,
wordWiseLevel: 3,
wordWiseHintLang: '',
};
export const DEFAULT_SCREEN_CONFIG: ScreenConfig = {
screenOrientation: 'auto',
};

2
apps/readest-app/src/services/settingsService.ts
View file

@ -20,6 +20,7 @@ import {
SETTINGS_FILENAME,
DEFAULT_MOBILE_SYSTEM_SETTINGS,
DEFAULT_ANNOTATOR_CONFIG,
DEFAULT_WORD_WISE_CONFIG,
DEFAULT_EINK_VIEW_SETTINGS,
DEFAULT_VIEW_SETTINGS_CONFIG,
} from './constants';
@ -44,6 +45,7 @@ export function getDefaultViewSettings(ctx: Context): ViewSettings {
...DEFAULT_TTS_CONFIG,
...DEFAULT_SCREEN_CONFIG,
...DEFAULT_ANNOTATOR_CONFIG,
...DEFAULT_WORD_WISE_CONFIG,
...DEFAULT_VIEW_SETTINGS_CONFIG,
...(ctx.isMobile ? DEFAULT_MOBILE_VIEW_SETTINGS : {}),
...(ctx.isEink ? DEFAULT_EINK_VIEW_SETTINGS : {}),

9
apps/readest-app/src/services/tts/TTSController.ts
View file

@ -10,7 +10,12 @@ import { EdgeTTSClient } from './EdgeTTSClient';
import { TTSUtils } from './TTSUtils';
import { TTSClient } from './TTSClient';
import { isValidLang } from '@/utils/lang';
import { computeWordOffsets, getTextSubRange, TTSWordOffset } from './wordHighlight';
import {
computeWordOffsets,
getTextSubRange,
rangeTextExcludingInert,
TTSWordOffset,
} from './wordHighlight';
type TTSState =
| 'stopped'
@ -684,7 +689,7 @@ export class TTSController extends EventTarget {
const range = this.view.tts?.getLastRange();
if (!range) return;
this.#speakWordBaseRange = range;
this.#speakWordOffsets = computeWordOffsets(range.toString(), words);
this.#speakWordOffsets = computeWordOffsets(rangeTextExcludingInert(range), words);
this.#speakWordRanges = [];
if (words.length === 0) {
// No word boundaries for this chunk: the sentence highlight was

40
apps/readest-app/src/services/tts/wordHighlight.ts
View file

@ -12,6 +12,42 @@ export interface TTSWordOffset {
// Edge TTS word-boundary offsets are in 100-nanosecond ticks.
const TICKS_PER_SECOND = 10_000_000;
// Gloss markup (<rt cfi-inert>) and any cfi-inert subtree is injected, non-book
// content — invisible to CFI and to spoken text (the TTS node filter rejects
// <rt>). Word-offset matching must ignore it too, or boundary words (gloss-free)
// won't align with the walked text.
const isInertText = (node: Node): boolean => {
let p: Node | null = node.parentNode;
while (p) {
if (p.nodeType === Node.ELEMENT_NODE) {
const el = p as Element;
const tag = el.tagName.toLowerCase();
if (tag === 'rt' || tag === 'rp' || el.hasAttribute('cfi-inert')) return true;
}
p = p.parentNode;
}
return false;
};
/** range.toString() minus any inert (gloss) text — the matching baseline. */
export const rangeTextExcludingInert = (base: Range): string => {
const root = base.commonAncestorContainer;
const doc = root.ownerDocument ?? (root as Document);
if (root.nodeType === Node.TEXT_NODE) {
return isInertText(root) ? '' : (root as Text).data;
}
const walker = doc.createTreeWalker(root, NodeFilter.SHOW_TEXT);
let out = '';
for (let node = walker.nextNode(); node; node = walker.nextNode()) {
if (!base.intersectsNode(node) || isInertText(node)) continue;
const text = node as Text;
const from = node === base.startContainer ? base.startOffset : 0;
const to = node === base.endContainer ? base.endOffset : text.data.length;
if (to > from) out += text.data.slice(from, to);
}
return out;
};
export const computeWordOffsets = (text: string, words: string[]): (TTSWordOffset | null)[] => {
const offsets: (TTSWordOffset | null)[] = [];
let cursor = 0;
@ -67,7 +103,9 @@ export const getTextSubRange = (base: Range, start: number, end: number): Range
return;
}
const walker = doc.createTreeWalker(root, NodeFilter.SHOW_TEXT);
for (let node = walker.nextNode(); node; node = walker.nextNode()) yield node;
for (let node = walker.nextNode(); node; node = walker.nextNode()) {
if (!isInertText(node)) yield node;
}
};
let pos = 0;
let startNode: Text | null = null;

44
apps/readest-app/src/services/wordwise/difficulty.ts Normal file
View file

@ -0,0 +1,44 @@
import type { WordWiseSourceLang } from './types';
// CEFR proficiency levels. The slider picks the reader's level; a word is glossed
// when it's ABOVE that level (rarer than the vocabulary a learner at that level
// typically knows). This maps corpus-FREQUENCY bands to CEFR as an APPROXIMATION —
// true per-word CEFR data is English-only and wouldn't generalize to the zh/es/fr/…
// packs; for zh it's an analogy over the HSK rank scale.
export const CEFR_LEVELS = ['A1', 'A2', 'B1', 'B2', 'C1', 'C2'] as const;
export type CefrLevel = (typeof CEFR_LEVELS)[number];
export const WORD_WISE_MIN_LEVEL = 1; // A1
export const WORD_WISE_MAX_LEVEL = CEFR_LEVELS.length; // 6 (C2)
// Level (1=A1 … 6=C2) -> rank cutoff = the vocabulary a learner at that level
// knows. A word is glossed when its rank >= cutoff, so a LOWER level (A1) => LOWER
// cutoff => MORE hints (a beginner needs help with almost everything); a HIGHER
// level (C2) => only the rarest words get a hint. index 0 = level 1 (A1).
// Frequency-rank scale: en + every Latin/space-delimited source (es, fr, de, …).
// Bands ≈ the vocabulary size a learner at each CEFR level commands.
const FREQUENCY: readonly number[] = [1000, 2000, 4000, 8000, 14000, 24000];
// HSK scale for zh (build script ranks by HSK level×3000, non-HSK 30000). A1
// glosses above HSK1; C2 only the rarest. An analogy, not real CEFR.
const HSK: readonly number[] = [6000, 9000, 12000, 15000, 18000, 24000];
const clampLevel = (level: number): number =>
Math.min(WORD_WISE_MAX_LEVEL, Math.max(WORD_WISE_MIN_LEVEL, Math.round(level)));
export const getRankCutoff = (lang: WordWiseSourceLang, level: number): number =>
(lang === 'zh' ? HSK : FREQUENCY)[clampLevel(level) - 1]!;
/** CEFR label ('A1'…'C2') for a 1..6 slider level. */
export const cefrLabel = (level: number): CefrLevel => CEFR_LEVELS[clampLevel(level) - 1]!;
export const isDifficult = (rank: number, cutoff: number): boolean => rank >= cutoff;
// Sources we can tokenize: zh (via jieba) + Latin/space-delimited languages
// (tokenized by the planner's regex). CJK languages lacking a segmenter
// (ja/ko) and Thai need a tier-3 segmenter and are blocked until then.
const UNSUPPORTED = new Set(['ja', 'ko', 'th']);
export const canTokenizeSource = (source: string): boolean =>
!UNSUPPORTED.has(source.toLowerCase().split('-')[0]!);

34
apps/readest-app/src/services/wordwise/glossIndex.ts Normal file
View file

@ -0,0 +1,34 @@
import type { GlossEntry, GlossIndexData, GlossSource } from './types';
/** In-memory, synchronous gloss lookup built from a downloaded gloss pack. */
export class GlossIndex implements GlossSource {
#entries: Map<string, GlossEntry>;
#inflections: Map<string, string>;
private constructor(entries: Map<string, GlossEntry>, inflections: Map<string, string>) {
this.#entries = entries;
this.#inflections = inflections;
}
static fromData(data: GlossIndexData): GlossIndex {
const entries = new Map<string, GlossEntry>();
for (const [word, { r, g }] of Object.entries(data.entries)) {
entries.set(word.toLowerCase(), { rank: r, gloss: g });
}
const inflections = new Map<string, string>();
for (const [form, lemma] of Object.entries(data.inflections)) {
inflections.set(form.toLowerCase(), lemma.toLowerCase());
}
return new GlossIndex(entries, inflections);
}
lookup(word: string): GlossEntry | null {
const w = word.trim().toLowerCase();
if (!w) return null;
const direct = this.#entries.get(w);
if (direct) return direct;
const lemma = this.#inflections.get(w);
if (lemma) return this.#entries.get(lemma) ?? null;
return null;
}
}

311
apps/readest-app/src/services/wordwise/glossPacks.ts Normal file
View file

@ -0,0 +1,311 @@
import { isWebAppPlatform } from '@/services/environment';
import { downloadFile } from '@/libs/storage';
import type { AppService } from '@/types/system';
import type { ProgressHandler } from '@/utils/transfer';
import { webDownload } from '@/utils/transfer';
import { GlossIndex } from './glossIndex';
import type { GlossIndexData } from './types';
export const WORDWISE_CDN_BASE = 'https://cdn.readest.com/wordwise';
const STORE_DIR = 'wordwise'; // relative dir under BaseDir 'Data'
const MANIFEST_FILE = 'manifest.json';
export interface WordWisePack {
pair: string;
source: string;
target: string;
file: string;
bytes: number;
sha256: string;
entries: number;
}
export interface WordWiseManifest {
schemaVersion: number;
packs: WordWisePack[];
}
/** Injectable byte-getter so the loader stays cross-platform AND unit-testable. */
export type BytesDownloader = (url: string, onProgress?: ProgressHandler) => Promise<ArrayBuffer>;
const storePath = (file: string) => `${STORE_DIR}/${file}`;
const sidecarPath = (file: string) => `${STORE_DIR}/${file}.sha`;
const sha256OfBytes = async (buf: ArrayBuffer): Promise<string> =>
[...new Uint8Array(await crypto.subtle.digest('SHA-256', buf))]
.map((b) => b.toString(16).padStart(2, '0'))
.join('');
// Idempotent + cheap: a recursive create is a no-op when the dir already
// exists, so we re-run it each time rather than caching a module-level latch
// (which would go stale if the user switches the custom data-root mid-session).
const ensureStoreDir = async (appService: AppService): Promise<void> => {
try {
await appService.createDir(STORE_DIR, 'Data', true);
} catch {
/* dir already exists (some platforms throw instead of no-op) */
}
};
// Type of the (injectable) Rust-backed downloader, matching libs/storage's
// `downloadFile`. Injected so the temp-file path can be unit-tested.
type DownloadFileFn = (params: {
appService: AppService;
dst: string;
cfp: string;
url: string;
onProgress?: ProgressHandler;
singleThreaded?: boolean;
}) => Promise<unknown>;
/**
* Download `url` through the Rust downloader into an absolute temp path under
* 'Data', read the bytes back, then delete the temp file. Both the download
* destination and the read/delete use the SAME absolute path (resolved via
* `resolveFilePath(rel, 'Data')` and addressed with base 'None'), mirroring
* the OPDS auto-download idiom in `services/opds/autoDownload.ts`. Routing
* through Rust (rather than a webview fetch) avoids cross-origin/CORS concerns
* on the webview — CSP itself is fine, since tauri.conf's connect-src
* whitelists https://*.readest.com.
*/
export const downloadViaTempFile = async (
appService: AppService,
downloadFileFn: DownloadFileFn,
url: string,
onProgress?: ProgressHandler,
): Promise<ArrayBuffer> => {
await ensureStoreDir(appService);
const ids = new Uint32Array(1);
crypto.getRandomValues(ids);
const tmpRel = `${STORE_DIR}/.dl-${ids[0]!.toString(36)}.tmp`;
const dst = await appService.resolveFilePath(tmpRel, 'Data'); // ABSOLUTE path under Data
await downloadFileFn({ appService, dst, cfp: dst, url, onProgress, singleThreaded: true });
try {
return (await appService.readFile(dst, 'None', 'binary')) as ArrayBuffer;
} finally {
try {
await appService.deleteFile(dst, 'None');
} catch {
/* best-effort temp cleanup */
}
}
};
// Default cross-platform downloader. On web a plain fetch to the CDN works and
// gives streaming progress; on Tauri we route through the Rust download path to
// avoid cross-origin/CORS concerns on the webview (CSP is fine — tauri.conf's
// connect-src whitelists https://*.readest.com), writing to a temp file and
// reading the bytes back for hashing.
export const defaultDownloader = async (
appService: AppService,
url: string,
onProgress?: ProgressHandler,
): Promise<ArrayBuffer> => {
if (isWebAppPlatform()) {
const { blob } = await webDownload(url, onProgress);
return await blob.arrayBuffer();
}
return downloadViaTempFile(appService, downloadFile, url, onProgress);
};
const getDownloader =
(appService: AppService, override?: BytesDownloader): BytesDownloader =>
(url, onProgress) =>
override ? override(url, onProgress) : defaultDownloader(appService, url, onProgress);
export const resolvePack = (
manifest: WordWiseManifest | null,
source: string,
hint: string,
): WordWisePack | null =>
manifest?.packs.find((p) => p.source === source && p.target === hint) ?? null;
/** The `target` codes the manifest offers for `source` (for the hint selector). */
export const listAvailableTargets = (manifest: WordWiseManifest | null, source: string): string[] =>
manifest?.packs.filter((p) => p.source === source).map((p) => p.target) ?? [];
// In-session memoized manifest (one network attempt per session unless forced).
let manifestPromise: Promise<WordWiseManifest | null> | null = null;
// Module-level lazy cache: one resolved GlossIndex per pair per session.
const indexCache = new Map<string, Promise<GlossIndex | null>>();
const readPersistedManifest = async (appService: AppService): Promise<WordWiseManifest | null> => {
try {
if (!(await appService.exists(storePath(MANIFEST_FILE), 'Data'))) return null;
const text = (await appService.readFile(storePath(MANIFEST_FILE), 'Data', 'text')) as string;
return JSON.parse(text) as WordWiseManifest;
} catch {
return null;
}
};
export const fetchManifest = async (
appService: AppService,
opts?: { download?: BytesDownloader; force?: boolean },
): Promise<WordWiseManifest | null> => {
if (manifestPromise && !opts?.force) return manifestPromise;
const download = getDownloader(appService, opts?.download);
manifestPromise = (async () => {
try {
const bytes = await download(`${WORDWISE_CDN_BASE}/${MANIFEST_FILE}`);
const text = new TextDecoder().decode(bytes);
const manifest = JSON.parse(text) as WordWiseManifest;
await ensureStoreDir(appService);
await appService.writeFile(storePath(MANIFEST_FILE), 'Data', text);
return manifest;
} catch (err) {
console.warn('[wordwise] manifest fetch failed; trying persisted copy', err);
return readPersistedManifest(appService);
}
})();
return manifestPromise;
};
// Single-flight per pair: collapse concurrent ensurePack calls for the same file.
const ensureFlights = new Map<string, Promise<string | null>>();
const ensurePackUncached = async (
appService: AppService,
pack: WordWisePack,
opts?: { onProgress?: ProgressHandler; download?: BytesDownloader; allowDownload?: boolean },
): Promise<string | null> => {
const dst = storePath(pack.file);
const sidecar = sidecarPath(pack.file);
// Reuse a present local file whose recorded sha matches the manifest's.
if ((await appService.exists(dst, 'Data')) && (await appService.exists(sidecar, 'Data'))) {
const recorded = (await appService.readFile(sidecar, 'Data', 'text')) as string;
if (recorded === pack.sha256) return dst;
}
// Reader path with auto-download off: never hit the network for an uncached
// pack. The settings panel passes allowDownload:true for explicit downloads.
if (opts?.allowDownload === false) return null;
const download = getDownloader(appService, opts?.download);
const url = `${WORDWISE_CDN_BASE}/${pack.file}?v=${pack.sha256.slice(0, 8)}`;
let bytes: ArrayBuffer;
try {
bytes = await download(url, opts?.onProgress);
} catch (err) {
console.warn('[wordwise] pack download failed', pack.pair, err);
return null;
}
const actual = await sha256OfBytes(bytes);
if (actual !== pack.sha256) {
console.warn('[wordwise] pack sha mismatch; discarding', pack.pair, {
actual,
expected: pack.sha256,
});
return null;
}
await ensureStoreDir(appService);
// Store as decoded text (like the manifest) so readFile(..., 'text') returns a
// string on every platform. The web appService keeps an ArrayBuffer verbatim and
// does NOT decode it for a 'text' read, which made JSON.parse choke on
// "[object ArrayBuffer]". Re-encoding valid UTF-8 JSON is byte-identical.
await appService.writeFile(dst, 'Data', new TextDecoder().decode(bytes));
await appService.writeFile(sidecar, 'Data', pack.sha256);
return dst;
};
export const ensurePack = async (
appService: AppService,
pack: WordWisePack,
opts?: { onProgress?: ProgressHandler; download?: BytesDownloader; allowDownload?: boolean },
): Promise<string | null> => {
const existing = ensureFlights.get(pack.pair);
if (existing) return existing;
const flight = ensurePackUncached(appService, pack, opts).finally(() => {
ensureFlights.delete(pack.pair);
});
ensureFlights.set(pack.pair, flight);
return flight;
};
/**
* Resolve the pack for (source → hint) and report whether it's already in
* local 'Data' (file + matching-sha sidecar present). Returns null when the
* manifest has no pack for the pair. For the settings sub-page's size/state UI.
*/
export const getPackStatus = async (
appService: AppService,
source: string,
hint: string,
opts?: { download?: BytesDownloader },
): Promise<{ pack: WordWisePack; downloaded: boolean } | null> => {
const manifest = await fetchManifest(appService, { download: opts?.download });
const pack = resolvePack(manifest, source, hint);
if (!pack) return null;
const dst = storePath(pack.file);
const sidecar = sidecarPath(pack.file);
let downloaded = false;
try {
if ((await appService.exists(dst, 'Data')) && (await appService.exists(sidecar, 'Data'))) {
const recorded = (await appService.readFile(sidecar, 'Data', 'text')) as string;
downloaded = recorded === pack.sha256;
}
} catch {
downloaded = false;
}
return { pack, downloaded };
};
/**
* Delete a downloaded pack's file + its `.sha` sidecar from 'Data' (ignoring
* not-found), and evict the in-session GlossIndex memo so a later re-enable
* reloads from a fresh download.
*/
export const deletePack = async (appService: AppService, pack: WordWisePack): Promise<void> => {
for (const path of [storePath(pack.file), sidecarPath(pack.file)]) {
try {
await appService.deleteFile(path, 'Data');
} catch {
/* best-effort: ignore not-found */
}
}
indexCache.delete(`${pack.source}-${pack.target}`);
};
export const loadGlossIndex = async (
appService: AppService,
source: string,
hint: string,
opts?: { onProgress?: ProgressHandler; download?: BytesDownloader; allowDownload?: boolean },
): Promise<GlossIndex | null> => {
const key = `${source}-${hint}`;
const existing = indexCache.get(key);
if (existing) return existing;
const promise = (async () => {
try {
const manifest = await fetchManifest(appService, { download: opts?.download });
const pack = resolvePack(manifest, source, hint);
if (!pack) return null;
const path = await ensurePack(appService, pack, {
onProgress: opts?.onProgress,
download: opts?.download,
allowDownload: opts?.allowDownload,
});
if (!path) return null;
// Tolerate a non-decoding 'text' read (web stores ArrayBuffer verbatim) so an
// already-cached pack from before the write-as-text fix still loads.
const raw = await appService.readFile(path, 'Data', 'text');
const text = typeof raw === 'string' ? raw : new TextDecoder().decode(raw);
return GlossIndex.fromData(JSON.parse(text) as GlossIndexData);
} catch (err) {
console.warn('[wordwise] loadGlossIndex failed', key, err);
return null;
}
})();
indexCache.set(key, promise);
// Don't memoize a failed/skipped resolve: with auto-download off an uncached
// pack yields null, and we want a later refresh (auto-download re-enabled, or
// a just-finished manual download) to retry instead of serving the cached null.
promise.then((index) => {
if (!index && indexCache.get(key) === promise) indexCache.delete(key);
});
return promise;
};

77
apps/readest-app/src/services/wordwise/planner.ts Normal file
View file

@ -0,0 +1,77 @@
import type { GlossOccurrence, GlossSource, WordWiseSourceLang } from './types';
import { isDifficult } from './difficulty';
export interface PlanOptions {
sourceLang: WordWiseSourceLang;
/** A word is glossed when its rank >= rankCutoff. */
rankCutoff: number;
/** Hard cap on occurrences per call (default 2000). Logged when hit. */
maxOccurrences?: number;
/** Chinese segmenter; injected for tests. Required for sourceLang 'zh'. */
cutZh?: (text: string) => string[];
}
// A foliate "section" is usually a whole chapter, so this is a per-chapter bound:
// high enough to fully gloss a normal chapter, low enough to protect against a
// pathological single-section book (e.g. a whole novel in one HTML file).
const DEFAULT_CAP = 2000;
interface Token {
word: string;
start: number;
end: number;
}
// Latin/English: Unicode letters with internal apostrophes/hyphens, offsets kept.
const tokenizeLatin = (text: string): Token[] => {
const re = /[\p{L}][\p{L}\p{M}’'-]*/gu;
const tokens: Token[] = [];
let m: RegExpExecArray | null;
while ((m = re.exec(text))) {
tokens.push({ word: m[0], start: m.index, end: m.index + m[0].length });
}
return tokens;
};
// Chinese: jieba segments cover the text in order; walk a cursor to recover
// offsets. Segments that aren't found at/after the cursor (whitespace it
// dropped, etc.) are skipped without stalling.
const tokenizeChinese = (text: string, cutZh: (t: string) => string[]): Token[] => {
const tokens: Token[] = [];
let cursor = 0;
for (const seg of cutZh(text)) {
if (!seg) continue;
const at = text.indexOf(seg, cursor);
if (at === -1) continue;
tokens.push({ word: seg, start: at, end: at + seg.length });
cursor = at + seg.length;
}
return tokens;
};
export const planGlosses = (
text: string,
source: GlossSource,
opts: PlanOptions,
): GlossOccurrence[] => {
if (!text) return [];
const cap = opts.maxOccurrences ?? DEFAULT_CAP;
const tokens =
opts.sourceLang === 'zh'
? opts.cutZh
? tokenizeChinese(text, opts.cutZh)
: []
: tokenizeLatin(text);
const occurrences: GlossOccurrence[] = [];
for (const t of tokens) {
const entry = source.lookup(t.word);
if (!entry || !isDifficult(entry.rank, opts.rankCutoff)) continue;
occurrences.push({ start: t.start, end: t.end, word: t.word, gloss: entry.gloss });
if (occurrences.length >= cap) {
console.warn(`[wordwise] occurrence cap (${cap}) hit; some hints omitted`);
break;
}
}
return occurrences;
};

40
apps/readest-app/src/services/wordwise/types.ts Normal file
View file

@ -0,0 +1,40 @@
// Word Wise: inline native-language hints above difficult words.
// See docs/superpowers/specs/2026-06-14-word-wise-design.md
/** Book source language as an ISO-639-1 base code (en, zh, es, …). */
export type WordWiseSourceLang = string;
/** A difficulty rank + native-language gloss for one headword. */
export interface GlossEntry {
/** Frequency rank; lower = more common. Number.MAX_SAFE_INTEGER if unknown. */
rank: number;
/** Short native-language hint shown above the word. */
gloss: string;
}
/** Anything the planner can ask "is this word difficult, and what's its gloss?". */
export interface GlossSource {
/** Resolve a surface word (handling case + inflection) to its entry, or null. */
lookup(word: string): GlossEntry | null;
}
/** One word to gloss, located by character offsets into the section's plain text. */
export interface GlossOccurrence {
/** Inclusive start offset into the section text model string. */
start: number;
/** Exclusive end offset. */
end: number;
/** Surface form as it appears in the text. */
word: string;
/** Native-language gloss to render in <rt>. */
gloss: string;
}
/** On-disk shape of a downloaded gloss pack (data/wordwise/<pair>.json, served from R2). */
export interface GlossIndexData {
meta: { source: string; target: string; metric: string; version: number; count: number };
/** headword -> { r: rank, g: gloss }. Compact keys to shrink the asset. */
entries: Record<string, { r: number; g: string }>;
/** inflected form -> lemma, e.g. { running: "run" }. */
inflections: Record<string, string>;
}

9
apps/readest-app/src/types/book.ts
View file

@ -332,6 +332,14 @@ export interface AnnotatorConfig {
noteExportConfig: NoteExportConfig;
}
export interface WordWiseConfig {
wordWiseEnabled: boolean;
/** Difficulty slider, 1 (fewest hints) .. 5 (most hints). */
wordWiseLevel: number;
/** Hint (target) language; '' = auto (app UI language). */
wordWiseHintLang: string;
}
export interface ScreenConfig {
screenOrientation: 'auto' | 'portrait' | 'landscape';
}
@ -372,6 +380,7 @@ export interface ViewSettings
ScreenConfig,
ProofreadRulesConfig,
AnnotatorConfig,
WordWiseConfig,
ViewSettingsConfig {}
export interface BookProgress {

6
apps/readest-app/src/types/settings.ts
View file

@ -54,6 +54,12 @@ export interface ReadSettings {
autohideCursor: boolean;
translationProvider: string;
translateTargetLang: string;
/**
* Global Word Wise toggle: auto-download a gloss pack on demand when the
* pair isn't cached locally. When off, the reader never fetches packs
* silently; users download them explicitly from the Word Wise sub-page.
*/
wordWiseAutoDownload: boolean;
highlightStyle: HighlightStyle;
highlightStyles: Record<HighlightStyle, HighlightColor>;

22
apps/readest-app/src/utils/network.ts
View file

@ -1,3 +1,25 @@
/**
* Network Information API surface we care about. It's non-standard and absent
* on iOS Safari / Tauri webviews, so everything is optional and `isMetered`
* falls through to `false` (treat as unmetered) when it can't tell.
*/
type NavWithConnection = Navigator & {
connection?: { type?: string; saveData?: boolean };
};
/**
* Best-effort metered-connection detection. Returns `true` only when the
* Network Information API positively reports a cellular connection or the
* user's data-saver preference; returns `false` when the API is unavailable or
* inconclusive. Used to gate silent Word Wise pack auto-downloads.
*/
export const isMetered = (): boolean => {
if (typeof navigator === 'undefined') return false;
const connection = (navigator as NavWithConnection).connection;
if (!connection) return false;
return connection.type === 'cellular' || connection.saveData === true;
};
export const isLanAddress = (url: string) => {
try {
const urlObj = new URL(url);

10
apps/readest-app/src/utils/style.ts
View file

@ -760,6 +760,16 @@ const getRubyStyles = () => `
rp {
display: none !important;
}
ruby.ww-gloss {
cursor: help;
}
ruby.ww-gloss > rt {
font-size: 0.5em;
line-height: 1.1;
opacity: 0.7;
font-weight: normal;
text-align: center;
}
`;
export interface ThemeCode {

3
biome.json
View file

@ -10,7 +10,8 @@
"!**/.claude/**",
"!**/gen/**",
"!**/autogenerated/**",
"!**/schemas/**"
"!**/schemas/**",
"!**/data/wordwise/**"
]
},
"formatter": {