mirror of
https://github.com/Trollobot/Telemax.git
synced 2026-10-19 10:23:24 +00:00
1.3.0-beta.1: большие файлы по ссылкам (общая служба файлов на хост, порты 80/443 только при живых ссылках, IP-сертификат), пульт «Файлы», мини-приложения ботов ссылками
This commit is contained in:
Folist
parent
fc2c054eb6
commit
f8202464a1
17 changed files with 1645 additions and 9 deletions
|
|
@ -2,6 +2,14 @@
|
|||
|
||||
Пользовательские описания релизов. Именно ЭТОТ файл (не заголовки коммитов) показывается в уведомлении об обновлении: раздел на версию, пункты — строки, начинающиеся с «- ». Заполняется ПЕРЕД тегированием релиза — scripts/release.sh откажется публиковать версию без её раздела здесь. Коммиты остаются техническими и пользователям не показываются.
|
||||
|
||||
## 1.3.0
|
||||
|
||||
- Большие файлы теперь пересылаются по ссылке. Из Telegram в MAX файл больше 20 МБ (столько бот может скачать из Telegram) загружается через страницу на вашем сервере и уходит в MAX обычным файлом. Из MAX в Telegram файл больше 50 МБ (столько бот может отправить) сохраняется на сервере, а в тему приходит кнопка «Скачать». Файлы больше 4 ГБ (лимит MAX) уходят собеседнику ссылкой
|
||||
- Пульт → «📁 Файлы»: какие файлы лежат на сервере, новая ссылка или удаление. Файл хранится 3 дня от последней ссылки, потом удаляется сам. Перед каждой передачей мост проверяет свободное место и всегда оставляет 1,5 ГБ запаса для обновлений
|
||||
- Для ссылок на сервере открываются порты 80 и 443 — только пока есть хотя бы одна действующая ссылка. Ссылок нет — служба файлов остановлена, порты закрыты. HTTPS настраивается сам, домен не нужен. Отключить совсем: FILES=off в .env
|
||||
- Кнопки мини-приложений ботов MAX стали ссылками: нажатие открывает мини-приложение в MAX
|
||||
- Если файл не прошёл по лимиту Telegram, мост объясняет, в чём дело, а не показывает «file is too big»
|
||||
|
||||
## 1.2.1 — 2026-10-05
|
||||
|
||||
- Поиск по нику снова находит ботов MAX: ник можно писать с «@» или без, а можно вставить ссылку max.ru/ник. Найденный бот приходит карточкой с кнопкой «Начать чат», дальше с ним можно переписываться и нажимать его кнопки
|
||||
|
|
|
|||
29
README.md
29
README.md
|
|
@ -7,6 +7,8 @@
|
|||
## Возможности
|
||||
|
||||
- Текстовые сообщения, фото, файлы, голосовые, видео и видео-кружки (обе стороны)
|
||||
- Большие файлы — по ссылке: из Telegram в MAX больше 20 МБ, из MAX в Telegram больше 50 МБ, в MAX больше 4 ГБ (см. [«Большие файлы»](#большие-файлы))
|
||||
- Кнопки ботов MAX — в Telegram: ссылки, копирование, нажатия; мини-приложения открываются ссылкой в MAX
|
||||
- Стикеры — статичные и анимированные (обе стороны)
|
||||
- Геолокация и контакты (обе стороны)
|
||||
- Опросы: создание и голосование в обе стороны, актуальный счёт по команде `/poll`
|
||||
|
|
@ -106,6 +108,31 @@ curl -fsSL https://raw.githubusercontent.com/Trollobot/Telemax/main/install.sh |
|
|||
|
||||
После успешной авторизации сессия сохраняется зашифрованной в `./data` — повторной авторизации при перезапуске контейнера не требуется. Повторный вход тем же путём (`/login`) нужен только после `/kill`, смены номера или если MAX отклонил сессию (бот сам напишет об этом в группу).
|
||||
|
||||
## Большие файлы
|
||||
|
||||
У ботов Telegram жёсткие лимиты: скачать из Telegram бот может файл не больше **20 МБ**, отправить в Telegram — не больше **50 МБ**. MAX принимает файлы до 4 ГБ. Всё, что не пролезает, мост пересылает по ссылке:
|
||||
|
||||
- **Из Telegram в MAX, больше 20 МБ.** Бот отвечает в теме кнопкой «📤 Загрузить файл»: по ссылке открывается страница на вашем сервере, файл загружается из браузера, и мост отправляет его в MAX обычным файлом. Файл больше 4 ГБ (лимит MAX) уходит собеседнику ссылкой на скачивание — она ведёт на ваш сервер.
|
||||
- **Из MAX в Telegram, больше 50 МБ.** Мост сохраняет файл на сервере и присылает в тему кнопку «⬇️ Скачать».
|
||||
- **Пульт → «📁 Файлы».** Список файлов на сервере: новая ссылка (продлевает хранение) или удаление. Файл живёт 3 дня от последней выданной ссылки, потом удаляется сам.
|
||||
|
||||
Как это устроено:
|
||||
|
||||
- Ссылки обслуживает одна общая служба на сервер (`files-service/`: Caddy + маленькое приложение на Node) — для всех мостов сразу. Её запускает автообновление, и **только пока есть действующие ссылки**: тогда открыты порты 80 и 443. Ссылок нет — служба остановлена, порты закрыты. Поэтому первая ссылка может готовиться до пары минут.
|
||||
- HTTPS — без домена: сертификат Let's Encrypt выпускается на IP-адрес сервера и продлевается сам. Если есть свой домен, укажите его: `FILES_DOMAIN=files.example.ru` в `.env` первого моста.
|
||||
- Перед каждой передачей проверяется свободное место, и мост всегда оставляет запас 1,5 ГБ — столько нужно для обновления.
|
||||
- По ссылке открывается страница с кнопкой, а не сам файл: предпросмотр ссылок в мессенджерах её не «съедает». Ссылки случайные, без списков файлов, с ограниченным сроком.
|
||||
- Нужны свободные порты 80 и 443. Если их занимает другой веб-сервер, бот так и скажет, а файлы больше лимитов пересылаться не будут.
|
||||
|
||||
Настройки в `.env` (необязательные):
|
||||
|
||||
```bash
|
||||
FILES=off # выключить пересылку больших файлов совсем
|
||||
FILES_TTL_HOURS=72 # сколько живёт файл после выдачи ссылки
|
||||
FILES_RESERVE_MB=1536 # сколько места всегда оставлять свободным
|
||||
FILES_DOMAIN= # свой домен вместо IP-адреса
|
||||
```
|
||||
|
||||
## Безопасность
|
||||
|
||||
Бот отвечает только внутри той Telegram-группы, что указана в `TARGET_TELEGRAM_GROUP` — любое сообщение или нажатие кнопки из другого чата (личка боту, добавление в чужую группу и т.п.) игнорируется.
|
||||
|
|
@ -114,7 +141,7 @@ curl -fsSL https://raw.githubusercontent.com/Trollobot/Telemax/main/install.sh |
|
|||
|
||||
Учтите: любой участник Telegram-группы видит в ней **все** зеркалированные MAX-чаты (по теме на чат), а не какой-то один. Доступ к самой группе — чувствительная вещь, раздавайте его осознанно.
|
||||
|
||||
Мост не открывает ни одного входящего порта: MAX — исходящее TCP-соединение, Telegram — long-polling. Управление — только из Telegram-группы (админами) и по SSH через `./setup.sh`. Единственная открытая дверь на сервере — SSH; `install.sh` предлагает поставить Fail2ban, который банит подбор пароля (5 промахов за 10 минут → бан на 10 минут; IP вашей сессии установки — в белом списке).
|
||||
Мост не открывает входящих портов: MAX — исходящее TCP-соединение, Telegram — long-polling. Единственное исключение — пересылка больших файлов (см. [«Большие файлы»](#большие-файлы)): пока на сервере есть хотя бы одна действующая ссылка на файл, открыты порты 80 и 443, а когда ссылок нет — служба останавливается и порты закрываются. Управление — только из Telegram-группы (админами) и по SSH через `./setup.sh`. Единственная открытая дверь на сервере — SSH; `install.sh` предлагает поставить Fail2ban, который банит подбор пароля (5 промахов за 10 минут → бан на 10 минут; IP вашей сессии установки — в белом списке).
|
||||
|
||||
Сертификаты российского государственного УЦ (нужны только потому, что через них выпущен TLS-сертификат самого MAX) доверяются **только** соединениям с серверами MAX — запросы к Telegram, GitHub и всему остальному проверяются исключительно по стандартному набору корневых сертификатов Mozilla (подробнее — `certs/README.md`).
|
||||
|
||||
|
|
|
|||
204
files-service/reconcile.sh
Normal file
204
files-service/reconcile.sh
Normal file
|
|
@ -0,0 +1,204 @@
|
|||
#!/usr/bin/env bash
|
||||
# Starts or stops the shared Telemax files service (Caddy + files-service/server.mjs) — ONE per
|
||||
# host for every bridge in /etc/telemax/instances. Called every minute by update-watcher.sh, i.e.
|
||||
# by the root update dispatcher, so installs get it with a normal update (no setup.sh re-run).
|
||||
#
|
||||
# Ports 80/443 are open ONLY while some bridge has a live big-file link: each bridge keeps
|
||||
# `data/files/active` while it has one (src/bridge/fileShare.ts). No marker anywhere → the service
|
||||
# is stopped. The result goes back to every bridge as `data/files/service.json`.
|
||||
#
|
||||
# TLS without a domain: Let's Encrypt issues short-lived (~6-day) certificates for a bare IP
|
||||
# (generally available since 2026-01; Caddy needs the `shortlived` profile and `default_sni`,
|
||||
# because browsers send no SNI for an IP). FILES_DOMAIN in the first bridge's .env uses a domain
|
||||
# instead. Caddy keeps its certificate in /var/lib/telemax-files/caddy-data between runs.
|
||||
set -uo pipefail
|
||||
|
||||
REG=/etc/telemax/instances
|
||||
STATE_DIR=/var/lib/telemax-files
|
||||
PROJECT=telemax-files
|
||||
CADDY_IMAGE='caddy:2.11.7@sha256:f2a1290d0463aad60660d4ec134943f183ee2a5f6c3eb7bf32dd984f2f020772'
|
||||
SELF="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
|
||||
[ -f "$REG" ] || exit 0
|
||||
[ "$(id -u)" = "0" ] || exit 0
|
||||
command -v docker >/dev/null 2>&1 || exit 0
|
||||
|
||||
# One owner: the first registered install that ships this script. Every bridge's watcher calls us
|
||||
# each minute; without a single owner two installs on different versions would flip-flop the config.
|
||||
LEADER=""
|
||||
while IFS= read -r d; do
|
||||
[ -n "$d" ] && [ -f "$d/files-service/reconcile.sh" ] && { LEADER="$d"; break; }
|
||||
done < "$REG"
|
||||
[ "$LEADER" = "$SELF" ] || exit 0
|
||||
|
||||
mkdir -p "$STATE_DIR"
|
||||
log() { printf '%s %s\n' "$(date -u +%FT%TZ)" "$*" >> "$STATE_DIR/reconcile.log"; tail -n 200 "$STATE_DIR/reconcile.log" > "$STATE_DIR/reconcile.log.tmp" 2>/dev/null && mv "$STATE_DIR/reconcile.log.tmp" "$STATE_DIR/reconcile.log"; }
|
||||
env_value() { grep -E "^$2=" "$1/.env" 2>/dev/null | tail -1 | cut -d= -f2- | tr -d "\"' \r"; }
|
||||
|
||||
INSTANCES=()
|
||||
WANTED=0
|
||||
while IFS= read -r d; do
|
||||
[ -n "$d" ] && [ -d "$d/data/files" ] || continue
|
||||
case "$(env_value "$d" FILES | tr '[:upper:]' '[:lower:]')" in 0|false|off|no) continue ;; esac
|
||||
INSTANCES+=("$d")
|
||||
[ -f "$d/data/files/active" ] && WANTED=1
|
||||
done < "$REG"
|
||||
|
||||
write_state() { # state url detail
|
||||
local now detail
|
||||
now=$(date +%s)
|
||||
detail=$(printf '%s' "$3" | tr -d '"\\\n\r' | cut -c1-200)
|
||||
for d in "${INSTANCES[@]}"; do
|
||||
printf '{"state":"%s","url":"%s","at":%s,"detail":"%s"}\n' "$1" "$2" "$now" "$detail" > "$d/data/files/service.json.tmp" \
|
||||
&& chmod 644 "$d/data/files/service.json.tmp" && mv -f "$d/data/files/service.json.tmp" "$d/data/files/service.json"
|
||||
done
|
||||
}
|
||||
|
||||
running() { docker ps -q --filter "label=com.docker.compose.project=$PROJECT" | grep -q .; }
|
||||
compose() { docker compose -p "$PROJECT" -f "$STATE_DIR/compose.yml" "$@"; }
|
||||
|
||||
if [ "$WANTED" = 0 ]; then
|
||||
if running; then
|
||||
compose down >/dev/null 2>&1 && log "no live links — service stopped, ports 80/443 closed"
|
||||
fi
|
||||
rm -f "$STATE_DIR/starting-since"
|
||||
write_state down "" ""
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --- where the links point ---------------------------------------------------------------------
|
||||
is_public_ipv4() {
|
||||
[[ "$1" =~ ^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})$ ]] || return 1
|
||||
local a=${BASH_REMATCH[1]} b=${BASH_REMATCH[2]}
|
||||
[ "$a" = 10 ] || [ "$a" = 127 ] || [ "$a" = 0 ] && return 1
|
||||
[ "$a" = 192 ] && [ "$b" = 168 ] && return 1
|
||||
[ "$a" = 172 ] && [ "$b" -ge 16 ] && [ "$b" -le 31 ] && return 1
|
||||
[ "$a" = 100 ] && [ "$b" -ge 64 ] && [ "$b" -le 127 ] && return 1
|
||||
[ "$a" = 169 ] && [ "$b" = 254 ] && return 1
|
||||
return 0
|
||||
}
|
||||
DOMAIN=$(env_value "$LEADER" FILES_DOMAIN)
|
||||
IS_IP=0
|
||||
if [ -n "$DOMAIN" ]; then
|
||||
HOST="$DOMAIN"
|
||||
else
|
||||
IS_IP=1
|
||||
HOST=""
|
||||
# Cached for a day: the lookup is the only outside request this script makes.
|
||||
if [ -f "$STATE_DIR/public-ip" ] && [ -n "$(find "$STATE_DIR/public-ip" -mmin -1440 2>/dev/null)" ]; then
|
||||
HOST=$(cat "$STATE_DIR/public-ip")
|
||||
fi
|
||||
if ! is_public_ipv4 "$HOST"; then
|
||||
HOST=$(ip -4 route get 1.1.1.1 2>/dev/null | awk '{for (i=1;i<NF;i++) if ($i=="src") print $(i+1)}' | head -1)
|
||||
is_public_ipv4 "$HOST" || HOST=$(curl -4 -fsS --max-time 6 https://api.ipify.org 2>/dev/null || true)
|
||||
is_public_ipv4 "$HOST" || HOST=$(curl -4 -fsS --max-time 6 https://ifconfig.me/ip 2>/dev/null || true)
|
||||
is_public_ipv4 "$HOST" && printf '%s\n' "$HOST" > "$STATE_DIR/public-ip"
|
||||
fi
|
||||
if ! is_public_ipv4 "$HOST"; then
|
||||
log "no public IPv4 found"
|
||||
write_state no-ip "" ""
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- ports: someone else's web server must not be fought with ---------------------------------
|
||||
if ! running && ss -ltnH '( sport = :80 or sport = :443 )' 2>/dev/null | grep -q .; then
|
||||
log "ports 80/443 are taken by another service"
|
||||
write_state ports-busy "" ""
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --- config -------------------------------------------------------------------------------------
|
||||
NODE_IMAGE=$(sed -n 's/^FROM \(node:[^ ]*\).*/\1/p' "$LEADER/Dockerfile" | head -1)
|
||||
[ -n "$NODE_IMAGE" ] || NODE_IMAGE=node:22-slim
|
||||
TZ_VALUE=$(env_value "$LEADER" TZ)
|
||||
|
||||
{
|
||||
echo "{"
|
||||
echo " admin off"
|
||||
if [ "$IS_IP" = 1 ]; then
|
||||
echo " default_sni $HOST"
|
||||
echo " cert_issuer acme {"
|
||||
echo " profile shortlived"
|
||||
echo " }"
|
||||
fi
|
||||
echo "}"
|
||||
echo "https://$HOST {"
|
||||
echo " reverse_proxy app:8080"
|
||||
echo "}"
|
||||
} > "$STATE_DIR/Caddyfile.new"
|
||||
|
||||
{
|
||||
echo "name: $PROJECT"
|
||||
echo "services:"
|
||||
echo " caddy:"
|
||||
echo " image: $CADDY_IMAGE"
|
||||
echo " restart: unless-stopped"
|
||||
echo " ports: [\"80:80\", \"443:443\"]"
|
||||
echo " volumes:"
|
||||
echo " - $STATE_DIR/Caddyfile:/etc/caddy/Caddyfile:ro"
|
||||
echo " - $STATE_DIR/caddy-data:/data"
|
||||
echo " - $STATE_DIR/caddy-config:/config"
|
||||
echo " depends_on: [app]"
|
||||
echo " logging: {driver: json-file, options: {max-size: \"5m\", max-file: \"2\"}}"
|
||||
echo " app:"
|
||||
echo " image: $NODE_IMAGE"
|
||||
echo " restart: unless-stopped"
|
||||
echo " user: \"1000:1000\""
|
||||
echo " command: [\"node\", \"/svc/server.mjs\"]"
|
||||
echo " environment:"
|
||||
echo " TZ: \"${TZ_VALUE:-Europe/Moscow}\""
|
||||
echo " volumes:"
|
||||
echo " - $LEADER/files-service:/svc:ro"
|
||||
i=0
|
||||
for d in "${INSTANCES[@]}"; do
|
||||
i=$((i + 1))
|
||||
echo " - $d/data/files:/srv/$i"
|
||||
done
|
||||
echo " logging: {driver: json-file, options: {max-size: \"5m\", max-file: \"2\"}}"
|
||||
} > "$STATE_DIR/compose.yml.new"
|
||||
|
||||
CADDY_CHANGED=0
|
||||
cmp -s "$STATE_DIR/Caddyfile.new" "$STATE_DIR/Caddyfile" 2>/dev/null || CADDY_CHANGED=1
|
||||
mv -f "$STATE_DIR/Caddyfile.new" "$STATE_DIR/Caddyfile"
|
||||
mv -f "$STATE_DIR/compose.yml.new" "$STATE_DIR/compose.yml"
|
||||
|
||||
WAS_RUNNING=0
|
||||
running && WAS_RUNNING=1
|
||||
if ! compose up -d --remove-orphans >"$STATE_DIR/compose-up.log" 2>&1; then
|
||||
log "compose up failed: $(tail -n 3 "$STATE_DIR/compose-up.log" | tr '\n' ' ')"
|
||||
write_state error "" "$(grep -iE 'error|bind|address already' "$STATE_DIR/compose-up.log" | tail -1)"
|
||||
exit 0
|
||||
fi
|
||||
[ "$WAS_RUNNING" = 0 ] && log "live links — service started for https://$HOST, ports 80/443 open"
|
||||
[ "$WAS_RUNNING" = 1 ] && [ "$CADDY_CHANGED" = 1 ] && compose restart caddy >/dev/null 2>&1
|
||||
# server.mjs is mounted, not baked in: after an update brings a new one, restart the app to load it.
|
||||
APP_SUM=$(sha256sum "$LEADER/files-service/server.mjs" 2>/dev/null | cut -c1-64)
|
||||
if [ "$WAS_RUNNING" = 1 ] && [ -n "$APP_SUM" ] && [ "$APP_SUM" != "$(cat "$STATE_DIR/app.sum" 2>/dev/null)" ]; then
|
||||
compose restart app >/dev/null 2>&1 && log "files service app restarted with the updated server.mjs"
|
||||
fi
|
||||
[ -n "$APP_SUM" ] && printf '%s\n' "$APP_SUM" > "$STATE_DIR/app.sum"
|
||||
|
||||
# --- healthy? -----------------------------------------------------------------------------------
|
||||
# A fresh start needs a few seconds (and a certificate on the very first run): wait a little here
|
||||
# rather than make the waiting bridge sit through another dispatcher minute.
|
||||
healthy() { curl -fsS --max-time 8 "https://$HOST/health" >/dev/null 2>&1; }
|
||||
if [ "$WAS_RUNNING" = 0 ]; then
|
||||
for _ in 1 2 3 4 5 6 7 8; do healthy && break; sleep 4; done
|
||||
fi
|
||||
if healthy; then
|
||||
rm -f "$STATE_DIR/starting-since"
|
||||
write_state up "https://$HOST" ""
|
||||
exit 0
|
||||
fi
|
||||
[ -f "$STATE_DIR/starting-since" ] || date +%s > "$STATE_DIR/starting-since"
|
||||
SINCE=$(cat "$STATE_DIR/starting-since" 2>/dev/null || date +%s)
|
||||
if [ $(( $(date +%s) - SINCE )) -gt 240 ]; then
|
||||
WHY=$(docker logs --tail 50 "$(docker ps -q --filter "label=com.docker.compose.project=$PROJECT" --filter "label=com.docker.compose.service=caddy" | head -1)" 2>&1 \
|
||||
| grep -oE '"(error|msg)":"[^"]*"' | grep -iE 'error|fail|challenge|refused|timeout' | tail -1)
|
||||
log "not healthy after 4 min: $WHY"
|
||||
write_state error "" "HTTPS не отвечает: ${WHY:-нет подробностей}"
|
||||
else
|
||||
write_state starting "" ""
|
||||
fi
|
||||
exit 0
|
||||
263
files-service/server.mjs
Normal file
263
files-service/server.mjs
Normal file
|
|
@ -0,0 +1,263 @@
|
|||
// Telemax files service — serves the bridges' big-file links (see src/bridge/fileShare.ts).
|
||||
//
|
||||
// Runs behind Caddy (TLS) in the shared `telemax-files` compose project that
|
||||
// files-service/reconcile.sh starts on the host only while some bridge has a live link. Plain
|
||||
// Node built-ins, no build step and no dependencies: the container is the stock node image with
|
||||
// this directory mounted read-only. Each bridge's `data/files` directory is mounted at
|
||||
// /srv/<n>; a link token is looked up in every one of them.
|
||||
//
|
||||
// GET /health → ok
|
||||
// GET /f/<token> → a page: «Скачать» for a download, a file picker for an upload
|
||||
// GET /f/<token>/file → the file itself (Range supported — a broken download resumes)
|
||||
// PUT /f/<token>/upload → the uploaded body, streamed to disk (X-File-Name: URI-encoded)
|
||||
//
|
||||
// A page is not the file: link previews (MAX and Telegram fetch every URL sent into a chat) only
|
||||
// ever see the page, so they can't burn an upload link or start a multi-gigabyte download.
|
||||
import http from 'node:http';
|
||||
import path from 'node:path';
|
||||
import { createReadStream, createWriteStream } from 'node:fs';
|
||||
import { mkdir, open, readFile, readdir, rename, rm, stat, statfs, writeFile } from 'node:fs/promises';
|
||||
import { pipeline } from 'node:stream/promises';
|
||||
import { Transform } from 'node:stream';
|
||||
|
||||
const ROOT = process.env.FILES_ROOT || '/srv';
|
||||
const PORT = Number(process.env.PORT || 8080);
|
||||
const TZ = process.env.TZ || 'Europe/Moscow';
|
||||
const TOKEN_RE = /^[A-Za-z0-9_-]{20,64}$/;
|
||||
|
||||
/** @returns {Promise<{root: string, token: string, body: any} | null>} */
|
||||
async function findToken(token) {
|
||||
if (!TOKEN_RE.test(token)) return null;
|
||||
let roots = [];
|
||||
try {
|
||||
roots = (await readdir(ROOT)).map((d) => path.join(ROOT, d));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
for (const root of roots) {
|
||||
try {
|
||||
const body = JSON.parse(await readFile(path.join(root, 'tokens', `${token}.json`), 'utf8'));
|
||||
if (typeof body.expiresAt !== 'number' || body.expiresAt <= Date.now()) return null;
|
||||
return { root, token, body };
|
||||
} catch {
|
||||
/* not this bridge */
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Strict inside-the-root resolution: a token's path can't point anywhere else. */
|
||||
function inside(root, rel) {
|
||||
const full = path.resolve(root, rel);
|
||||
return full.startsWith(path.resolve(root) + path.sep) ? full : null;
|
||||
}
|
||||
|
||||
function safeName(raw) {
|
||||
let name = String(raw ?? '').replace(/[\u0000-\u001f\u007f/\\:*?"<>|]/g, '_').trim().replace(/^\.+/, '').trim();
|
||||
if (!name) name = 'file';
|
||||
return name.length > 180 ? name.slice(0, 180) : name;
|
||||
}
|
||||
|
||||
function formatBytes(b) {
|
||||
if (b >= 1024 ** 3) return `${(b / 1024 ** 3).toFixed(1)} ГБ`;
|
||||
if (b >= 1024 ** 2) return `${Math.round(b / 1024 ** 2)} МБ`;
|
||||
return `${Math.max(1, Math.round(b / 1024))} КБ`;
|
||||
}
|
||||
|
||||
function formatDate(ms) {
|
||||
return new Intl.DateTimeFormat('ru-RU', { day: 'numeric', month: 'long', hour: '2-digit', minute: '2-digit', timeZone: TZ }).format(new Date(ms));
|
||||
}
|
||||
|
||||
const esc = (s) => String(s).replace(/[&<>"']/g, (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c]);
|
||||
|
||||
const SECURITY_HEADERS = {
|
||||
'X-Robots-Tag': 'noindex, nofollow',
|
||||
'Referrer-Policy': 'no-referrer',
|
||||
'X-Content-Type-Options': 'nosniff',
|
||||
'Cache-Control': 'no-store',
|
||||
};
|
||||
|
||||
function page(res, status, title, body) {
|
||||
res.writeHead(status, {
|
||||
...SECURITY_HEADERS,
|
||||
'Content-Type': 'text/html; charset=utf-8',
|
||||
'Content-Security-Policy': "default-src 'none'; style-src 'unsafe-inline'; script-src 'unsafe-inline'; connect-src 'self'; form-action 'none'; frame-ancestors 'none'",
|
||||
});
|
||||
res.end(`<!doctype html><html lang="ru"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<meta name="robots" content="noindex,nofollow"><title>${esc(title)}</title><style>
|
||||
:root{color-scheme:light dark;--bg:#f4f5f7;--card:#fff;--fg:#1c1d21;--mut:#6b6f7a;--acc:#2a7de1;--bar:#e3e6eb}
|
||||
@media(prefers-color-scheme:dark){:root{--bg:#15171b;--card:#1f2228;--fg:#e8eaee;--mut:#9aa0ab;--bar:#30343c}}
|
||||
body{margin:0;background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,Segoe UI,Roboto,sans-serif;display:flex;min-height:100vh;align-items:center;justify-content:center}
|
||||
.c{background:var(--card);border-radius:16px;padding:28px;max-width:440px;width:calc(100% - 32px);box-sizing:border-box;box-shadow:0 2px 14px rgba(0,0,0,.08)}
|
||||
h1{font-size:20px;margin:0 0 6px}.m{color:var(--mut);font-size:14px;margin:0 0 18px;word-break:break-word}
|
||||
.b{display:block;width:100%;box-sizing:border-box;text-align:center;background:var(--acc);color:#fff;border:0;border-radius:10px;padding:13px;font-size:16px;text-decoration:none;cursor:pointer}
|
||||
.b[disabled]{opacity:.5;cursor:default}input[type=file]{width:100%;margin:0 0 14px}
|
||||
.p{height:8px;background:var(--bar);border-radius:4px;overflow:hidden;margin:14px 0 6px;display:none}.p i{display:block;height:100%;width:0;background:var(--acc)}
|
||||
.s{font-size:14px;color:var(--mut);min-height:21px}.f{margin-top:18px;font-size:12px;color:var(--mut)}
|
||||
</style></head><body><div class="c">${body}<div class="f">Telemax · ссылка действует ограниченное время, потом файл удаляется</div></div></body></html>`);
|
||||
}
|
||||
|
||||
function notFound(res) {
|
||||
page(res, 404, 'Ссылка недействительна', '<h1>Ссылка недействительна</h1><p class="m">Ссылка устарела, уже использована или набрана с ошибкой. Попросите прислать новую.</p>');
|
||||
}
|
||||
|
||||
async function freeBytes(root) {
|
||||
const s = await statfs(root);
|
||||
return Number(s.bavail) * Number(s.bsize);
|
||||
}
|
||||
|
||||
async function showLink(res, t) {
|
||||
const { body, token } = t;
|
||||
if (body.kind === 'download') {
|
||||
page(
|
||||
res,
|
||||
200,
|
||||
body.name,
|
||||
`<h1>${esc(body.name)}</h1><p class="m">${esc(formatBytes(body.size))} · ссылка действует до ${esc(formatDate(body.expiresAt))}</p>
|
||||
<a class="b" href="/f/${token}/file">Скачать</a>`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (body.kind === 'upload') {
|
||||
const done = await stat(path.join(t.root, 'incoming', token, 'done.json')).then(() => true, () => false);
|
||||
if (done) {
|
||||
page(res, 410, 'Файл уже получен', '<h1>Файл уже получен</h1><p class="m">Мост отправит его в MAX. Эту страницу можно закрыть.</p>');
|
||||
return;
|
||||
}
|
||||
const maxBytes = Math.max(0, (await freeBytes(t.root)) - (body.reserveBytes ?? 0));
|
||||
page(
|
||||
res,
|
||||
200,
|
||||
'Загрузка файла',
|
||||
`<h1>Загрузите файл</h1><p class="m">Ждём «${esc(body.name)}» (${esc(formatBytes(body.expectedSize ?? 0))}). Мост отправит его в MAX.<br>Можно выбрать и другой файл — до ${esc(formatBytes(maxBytes))}. Ссылка действует до ${esc(formatDate(body.expiresAt))}.</p>
|
||||
<input type="file" id="f"><button class="b" id="go" disabled>Отправить</button>
|
||||
<div class="p" id="p"><i id="bar"></i></div><div class="s" id="s"></div>
|
||||
<script>
|
||||
const f=document.getElementById('f'),go=document.getElementById('go'),s=document.getElementById('s'),p=document.getElementById('p'),bar=document.getElementById('bar');
|
||||
const MAX=${maxBytes};
|
||||
const fmt=b=>b>=1073741824?(b/1073741824).toFixed(1)+' ГБ':b>=1048576?Math.round(b/1048576)+' МБ':Math.max(1,Math.round(b/1024))+' КБ';
|
||||
f.onchange=()=>{const x=f.files[0];s.textContent='';go.disabled=!x;if(x&&x.size>MAX){s.textContent='Файл '+fmt(x.size)+' не поместится: на сервере свободно '+fmt(MAX)+'.';go.disabled=true}};
|
||||
go.onclick=()=>{const x=f.files[0];if(!x)return;go.disabled=true;f.disabled=true;p.style.display='block';
|
||||
const r=new XMLHttpRequest();r.open('PUT',location.pathname.replace(/\\/$/,'')+'/upload');r.setRequestHeader('X-File-Name',encodeURIComponent(x.name));
|
||||
r.upload.onprogress=e=>{if(e.lengthComputable){bar.style.width=(e.loaded/e.total*100).toFixed(1)+'%';s.textContent=fmt(e.loaded)+' из '+fmt(e.total)}};
|
||||
r.onload=()=>{if(r.status===200){bar.style.width='100%';s.textContent='Готово! Мост отправит файл в MAX — страницу можно закрыть.'}else{s.textContent='Не получилось: '+(r.responseText||r.status);go.disabled=false;f.disabled=false}};
|
||||
r.onerror=()=>{s.textContent='Связь прервалась — попробуйте ещё раз.';go.disabled=false;f.disabled=false};r.send(x)};
|
||||
</script>`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
notFound(res);
|
||||
}
|
||||
|
||||
async function sendFile(req, res, t) {
|
||||
const { root, body } = t;
|
||||
if (body.kind !== 'download') return notFound(res);
|
||||
const full = inside(root, body.path);
|
||||
if (!full) return notFound(res);
|
||||
let size;
|
||||
try {
|
||||
size = (await stat(full)).size;
|
||||
} catch {
|
||||
return notFound(res);
|
||||
}
|
||||
const name = safeName(body.name);
|
||||
const headers = {
|
||||
...SECURITY_HEADERS,
|
||||
'Content-Type': 'application/octet-stream',
|
||||
'Content-Disposition': `attachment; filename="${name.replace(/[^\x20-\x7e]/g, '_')}"; filename*=UTF-8''${encodeURIComponent(name)}`,
|
||||
'Accept-Ranges': 'bytes',
|
||||
};
|
||||
const m = /^bytes=(\d*)-(\d*)$/.exec(req.headers.range ?? '');
|
||||
if (m && (m[1] || m[2])) {
|
||||
let start = m[1] ? Number(m[1]) : size - Number(m[2]);
|
||||
let end = m[1] && m[2] ? Number(m[2]) : size - 1;
|
||||
if (start < 0) start = 0;
|
||||
if (end >= size) end = size - 1;
|
||||
if (start > end || start >= size) {
|
||||
res.writeHead(416, { ...headers, 'Content-Range': `bytes */${size}` });
|
||||
return res.end();
|
||||
}
|
||||
res.writeHead(206, { ...headers, 'Content-Range': `bytes ${start}-${end}/${size}`, 'Content-Length': String(end - start + 1) });
|
||||
if (req.method === 'HEAD') return res.end();
|
||||
return pipeline(createReadStream(full, { start, end }), res).catch(() => {});
|
||||
}
|
||||
res.writeHead(200, { ...headers, 'Content-Length': String(size) });
|
||||
if (req.method === 'HEAD') return res.end();
|
||||
return pipeline(createReadStream(full), res).catch(() => {});
|
||||
}
|
||||
|
||||
function reply(res, status, text) {
|
||||
res.writeHead(status, { ...SECURITY_HEADERS, 'Content-Type': 'text/plain; charset=utf-8' });
|
||||
res.end(text);
|
||||
}
|
||||
|
||||
async function receiveUpload(req, res, t) {
|
||||
const { root, token, body } = t;
|
||||
if (body.kind !== 'upload') return reply(res, 404, 'Ссылка недействительна');
|
||||
const length = Number(req.headers['content-length']);
|
||||
if (!Number.isFinite(length) || length <= 0) return reply(res, 411, 'Не указан размер файла');
|
||||
const room = (await freeBytes(root)) - (body.reserveBytes ?? 0);
|
||||
if (length > room) return reply(res, 507, `Не хватает места на сервере: свободно ${formatBytes(Math.max(0, room))}`);
|
||||
|
||||
const dir = path.join(root, 'incoming', token);
|
||||
await mkdir(dir, { recursive: true });
|
||||
if (await stat(path.join(dir, 'done.json')).then(() => true, () => false)) return reply(res, 410, 'Файл по этой ссылке уже получен');
|
||||
// One upload at a time per link: the lock file is created exclusively and removed on failure.
|
||||
let lock;
|
||||
try {
|
||||
lock = await open(path.join(dir, '.lock'), 'wx');
|
||||
} catch {
|
||||
return reply(res, 409, 'По этой ссылке уже идёт загрузка');
|
||||
}
|
||||
let rawName = 'file';
|
||||
try {
|
||||
rawName = decodeURIComponent(String(req.headers['x-file-name'] ?? 'file'));
|
||||
} catch {
|
||||
/* keep the default */
|
||||
}
|
||||
const name = safeName(rawName);
|
||||
const part = path.join(dir, `${name}.part`);
|
||||
let received = 0;
|
||||
const guard = new Transform({
|
||||
transform(chunk, _enc, cb) {
|
||||
received += chunk.length;
|
||||
cb(received > length ? new Error('body longer than Content-Length') : null, chunk);
|
||||
},
|
||||
});
|
||||
try {
|
||||
await pipeline(req, guard, createWriteStream(part));
|
||||
if (received !== length) throw new Error(`got ${received} of ${length} bytes`);
|
||||
await rename(part, path.join(dir, name));
|
||||
await writeFile(path.join(dir, 'done.json'), JSON.stringify({ name, file: name, size: received, at: Date.now() }));
|
||||
reply(res, 200, 'ok');
|
||||
} catch (err) {
|
||||
await rm(part, { force: true });
|
||||
if (!res.headersSent) reply(res, 400, `Загрузка прервалась: ${err.message}`);
|
||||
} finally {
|
||||
await lock.close().catch(() => {});
|
||||
await rm(path.join(dir, '.lock'), { force: true });
|
||||
}
|
||||
}
|
||||
|
||||
const server = http.createServer(async (req, res) => {
|
||||
try {
|
||||
const url = new URL(req.url ?? '/', 'http://x');
|
||||
if (url.pathname === '/health') return reply(res, 200, 'ok');
|
||||
const m = /^\/f\/([A-Za-z0-9_-]+)(\/file|\/upload)?\/?$/.exec(url.pathname);
|
||||
if (!m) return notFound(res);
|
||||
const t = await findToken(m[1]);
|
||||
if (!t) return m[2] === '/upload' ? reply(res, 404, 'Ссылка недействительна или устарела') : notFound(res);
|
||||
if (!m[2] && (req.method === 'GET' || req.method === 'HEAD')) return await showLink(res, t);
|
||||
if (m[2] === '/file' && (req.method === 'GET' || req.method === 'HEAD')) return await sendFile(req, res, t);
|
||||
if (m[2] === '/upload' && (req.method === 'PUT' || req.method === 'POST')) return await receiveUpload(req, res, t);
|
||||
reply(res, 405, 'Method not allowed');
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
if (!res.headersSent) reply(res, 500, 'Внутренняя ошибка');
|
||||
else res.destroy();
|
||||
}
|
||||
});
|
||||
// Gigabyte uploads outlast Node's default 5-minute request timeout.
|
||||
server.requestTimeout = 0;
|
||||
server.listen(PORT, () => console.log(`telemax files service on :${PORT}, roots under ${ROOT}`));
|
||||
for (const sig of ['SIGTERM', 'SIGINT']) process.on(sig, () => server.close(() => process.exit(0)));
|
||||
4
package-lock.json
generated
4
package-lock.json
generated
|
|
@ -1,12 +1,12 @@
|
|||
{
|
||||
"name": "telemax",
|
||||
"version": "1.2.1",
|
||||
"version": "1.3.0-beta.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "telemax",
|
||||
"version": "1.2.1",
|
||||
"version": "1.3.0-beta.1",
|
||||
"license": "AGPL-3.0-only",
|
||||
"dependencies": {
|
||||
"dotenv": "^17.2.3",
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
"name": "telemax",
|
||||
"private": true,
|
||||
"license": "AGPL-3.0-only",
|
||||
"version": "1.2.1",
|
||||
"version": "1.3.0-beta.1",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "tsx watch src/server/app.ts",
|
||||
|
|
|
|||
|
|
@ -189,6 +189,17 @@ async function withDownloadFallback<T>(
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The CDN URL of a MAX FILE attach (FILE_DOWNLOAD with the forward fallback) — for the big-file
|
||||
* path, which streams it to disk itself. Null when MAX refuses for good; TransientDownloadError
|
||||
* when the socket was down.
|
||||
*/
|
||||
export async function getMaxFileUrl(att: MaxAttachment, ctx: DownloadContext): Promise<string | null> {
|
||||
if (att._type !== 'FILE' || att.fileId == null) return null;
|
||||
const fileId = att.fileId;
|
||||
return withDownloadFallback(ctx, `FILE_DOWNLOAD fileId ${String(fileId)}`, (chatId, messageId) => ctx.max.getFileDownloadUrl(chatId, messageId, fileId));
|
||||
}
|
||||
|
||||
/**
|
||||
* Downloads a MAX attachment for re-upload to Telegram. Null means "no file to send" — the caller
|
||||
* posts describeAttachment()'s placeholder. A transient failure (see TransientDownloadError) is
|
||||
|
|
@ -368,6 +379,21 @@ export function pressableButton(key: string): PressableButton | undefined {
|
|||
return pressable.get(key);
|
||||
}
|
||||
|
||||
/**
|
||||
* The public deep link of a MAX mini-app button: `https://max.ru/<bot>?startapp=<payload>` (MAX
|
||||
* docs, dev.max.ru/docs/webapps/introduction). Live shape 2026-10-06: {type:'OPEN_APP', text,
|
||||
* webApp:'maxnotifications_bot', contactId, payload}. MAX drops a payload over 512 chars or with
|
||||
* anything but [A-Za-z0-9_-], so such a payload is left out (the app opens on its start screen).
|
||||
* The payload is per-account — fine, the button only ever reaches the account it was sent to.
|
||||
*/
|
||||
export function miniAppLink(b: Record<string, unknown> | null | undefined): string | undefined {
|
||||
const app = typeof b?.webApp === 'string' ? b.webApp : '';
|
||||
if (!/^[A-Za-z0-9_]{1,64}$/.test(app)) return undefined;
|
||||
const payload = typeof b?.payload === 'string' ? b.payload : '';
|
||||
const start = /^[A-Za-z0-9_-]{1,512}$/.test(payload) ? `=${payload}` : '';
|
||||
return `https://max.ru/${app}?startapp${start}`;
|
||||
}
|
||||
|
||||
/** callback_data of a MAX button with no Telegram counterpart (yet): pressing it only shows a hint. */
|
||||
export const KEYBOARD_NA = 'tlmx_kb_na';
|
||||
|
||||
|
|
@ -403,6 +429,9 @@ export function maxKeyboardToTelegram(att: unknown, chatId?: unknown): InlineKey
|
|||
} else if (chatId != null && b?.type === 'MESSAGE') {
|
||||
// Pressing it in MAX sends the button's text to the bot as the user's own message.
|
||||
buttons.push({ text, callback_data: registerButton({ chatId, callbackId: '', text, sendsText: true }) });
|
||||
} else if (b?.type === 'OPEN_APP' && miniAppLink(b)) {
|
||||
// A mini-app can't run inside Telegram, but its public deep link opens it in MAX.
|
||||
buttons.push({ text, url: miniAppLink(b)! });
|
||||
} else if (b?.type === 'REQUEST_GEO_LOCATION') {
|
||||
buttons.push({ text, callback_data: KEYBOARD_GEO });
|
||||
} else if (b?.type === 'REQUEST_CONTACT') {
|
||||
|
|
|
|||
198
src/bridge/bigFiles.ts
Normal file
198
src/bridge/bigFiles.ts
Normal file
|
|
@ -0,0 +1,198 @@
|
|||
/**
|
||||
* The chat side of big files (storage and links: fileShare.ts). Every message here says WHY a file
|
||||
* goes by link — to the user a link instead of a file otherwise looks like a bug.
|
||||
*/
|
||||
import { Markup, type Telegraf } from 'telegraf';
|
||||
import type { MaxClient } from '../max/client.js';
|
||||
import { maxFetchBig } from '../max/ca.js';
|
||||
import { createLogger } from '../logger.js';
|
||||
import { getMaxFileUrl, describeAttachment, type DownloadContext, type MaxAttachment } from './attachments.js';
|
||||
import { FileShare, MAX_FILE_LIMIT, nodeReadable, type UploadTicket } from './fileShare.js';
|
||||
import { formatBytes } from './status.js';
|
||||
import { uploadFileFromDiskToMax } from './upload.js';
|
||||
import { isTransientHttpStatus, isTransientNetworkError, TransientDownloadError } from './transient.js';
|
||||
|
||||
const logger = createLogger('big-files');
|
||||
|
||||
const WHY_TG_DOWNLOAD = 'Telegram не отдаёт ботам файлы больше 20 МБ';
|
||||
const WHY_TG_UPLOAD = 'Telegram не принимает от ботов файлы больше 50 МБ';
|
||||
|
||||
export function formatExpiry(ms: number): string {
|
||||
return new Intl.DateTimeFormat('ru-RU', { day: 'numeric', month: 'long', hour: '2-digit', minute: '2-digit', timeZone: process.env.TZ || 'Europe/Moscow' }).format(new Date(ms));
|
||||
}
|
||||
|
||||
type SentMessage = { message_id: number };
|
||||
type SendOpts = { message_thread_id: number; reply_parameters?: { message_id: number; allow_sending_without_reply: boolean } };
|
||||
|
||||
export interface BigFilesDeps {
|
||||
bot: Telegraf;
|
||||
max: MaxClient;
|
||||
targetGroupId: string;
|
||||
fileShare: FileShare;
|
||||
/** Records a message the bridge itself sent to MAX (echo suppression + the TG↔MAX link). */
|
||||
onSentToMax: (ticket: UploadTicket, sent: { cid: number; messageId: unknown; time: unknown }) => void;
|
||||
}
|
||||
|
||||
export interface BigFiles {
|
||||
/** Telegram → MAX: a file the bot can't download — answer with an upload link. */
|
||||
offerUpload(p: { maxChatId: string; topicId: number; telegramMessageId: number; caption?: string; name: string; size: number }): Promise<void>;
|
||||
/** MAX → Telegram: a FILE over the bot upload limit — save it and post a download link. */
|
||||
relayFromMax(att: MaxAttachment, ctx: DownloadContext, opts: SendOpts): Promise<SentMessage>;
|
||||
}
|
||||
|
||||
export function createBigFiles({ bot, max, targetGroupId, fileShare, onSentToMax }: BigFilesDeps): BigFiles {
|
||||
const send = (text: string, opts: SendOpts, markup?: ReturnType<typeof Markup.inlineKeyboard>) =>
|
||||
bot.telegram.sendMessage(targetGroupId, text, { ...opts, ...(markup ? { reply_markup: markup.reply_markup } : {}) });
|
||||
const edit = async (messageId: number | undefined, topicId: number, text: string, markup?: ReturnType<typeof Markup.inlineKeyboard>) => {
|
||||
if (messageId != null) {
|
||||
try {
|
||||
await bot.telegram.editMessageText(targetGroupId, messageId, undefined, text, markup ? { reply_markup: markup.reply_markup } : {});
|
||||
return;
|
||||
} catch (err) {
|
||||
if (/not modified/i.test((err as Error).message ?? '')) return;
|
||||
logger.error('Editing a big-file notice failed — posting a new one', err);
|
||||
}
|
||||
}
|
||||
await send(text, { message_thread_id: topicId }, markup).catch((e) => logger.error('Posting a big-file notice failed', e));
|
||||
};
|
||||
const reserveText = formatBytes(fileShare.opts.reserveBytes);
|
||||
|
||||
// --- Telegram → MAX -------------------------------------------------------------------------
|
||||
|
||||
async function offerUpload(p: Parameters<BigFiles['offerUpload']>[0]): Promise<void> {
|
||||
const label = `«${p.name}» (${formatBytes(p.size)})`;
|
||||
const opts: SendOpts = { message_thread_id: p.topicId, reply_parameters: { message_id: p.telegramMessageId, allow_sending_without_reply: true } };
|
||||
if (!fileShare.enabled) {
|
||||
await send(`⚠️ Файл ${label} не отправлен в MAX: ${WHY_TG_DOWNLOAD}. Отправьте его из самого MAX или поделите на части.`, opts);
|
||||
return;
|
||||
}
|
||||
const room = await fileShare.roomFor(p.size);
|
||||
if (!room.fits) {
|
||||
await send(
|
||||
`⚠️ Файл ${label} не отправлен в MAX: ${WHY_TG_DOWNLOAD}, а принять его по ссылке не выйдет — на сервере свободно ${formatBytes(room.freeBytes)}, нужно ${formatBytes(p.size)} и ещё ${reserveText} запаса для обновлений.`,
|
||||
opts,
|
||||
);
|
||||
return;
|
||||
}
|
||||
const prompt = await send(`⏳ Файл ${label} больше 20 МБ — ${WHY_TG_DOWNLOAD}. Готовлю ссылку, чтобы загрузить его в обход Telegram…`, opts);
|
||||
const ticket = await fileShare.createUploadTicket({
|
||||
maxChatId: p.maxChatId,
|
||||
topicId: p.topicId,
|
||||
telegramMessageId: p.telegramMessageId,
|
||||
caption: p.caption,
|
||||
name: p.name,
|
||||
expectedSize: p.size,
|
||||
promptMessageId: prompt.message_id,
|
||||
});
|
||||
logger.info(`TG -> MAX: ${p.name} is ${p.size} bytes, over the bot download limit — offering an upload link`);
|
||||
const svc = await fileShare.ensureService();
|
||||
if (!svc.ok) {
|
||||
await fileShare.dropTicket(ticket.token);
|
||||
await edit(prompt.message_id, p.topicId, `⚠️ Файл ${label} не отправлен в MAX: ${WHY_TG_DOWNLOAD}, а ссылку для загрузки подготовить не удалось — ${svc.reason}.`);
|
||||
return;
|
||||
}
|
||||
await edit(
|
||||
prompt.message_id,
|
||||
p.topicId,
|
||||
`📤 Файл ${label} больше 20 МБ — ${WHY_TG_DOWNLOAD}.\nЗагрузите его по ссылке, и мост отправит его в MAX. Ссылка действует до ${formatExpiry(ticket.expiresAt)}.`,
|
||||
Markup.inlineKeyboard([Markup.button.url('📤 Загрузить файл', `${svc.url}/f/${ticket.token}`)]),
|
||||
);
|
||||
}
|
||||
|
||||
fileShare.onUpload(async (ticket, filePath, size, name) => {
|
||||
const label = `«${name}» (${formatBytes(size)})`;
|
||||
if (size <= MAX_FILE_LIMIT) {
|
||||
await edit(ticket.promptMessageId, ticket.topicId, `⏳ Файл ${label} получен — отправляю в MAX…`);
|
||||
try {
|
||||
const attach = await uploadFileFromDiskToMax(max, filePath, size, name);
|
||||
const sent = await max.sendMessage(ticket.maxChatId, ticket.caption ?? null, [attach]);
|
||||
onSentToMax(ticket, sent);
|
||||
await fileShare.dropTicket(ticket.token);
|
||||
await edit(ticket.promptMessageId, ticket.topicId, `✅ Файл ${label} отправлен в MAX.`);
|
||||
logger.info(`TG -> MAX: uploaded ${name} (${size} bytes) sent to chat ${ticket.maxChatId}`);
|
||||
} catch (err) {
|
||||
logger.error(`Sending the uploaded ${name} to MAX failed — keeping it as a link`, err);
|
||||
const file = await fileShare.adoptUpload(ticket, filePath, size, name);
|
||||
await edit(
|
||||
ticket.promptMessageId,
|
||||
ticket.topicId,
|
||||
`⚠️ Файл ${label} получен, но в MAX не ушёл: ${(err as Error).message}.\nОн сохранён до ${formatExpiry(file.expiresAt)} — ссылку на него можно взять в пульте → 📁 Файлы и отправить вручную.`,
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
// Over MAX's own limit: the recipient gets a download link instead of the file.
|
||||
const file = await fileShare.adoptUpload(ticket, filePath, size, name);
|
||||
const svc = await fileShare.ensureService(30_000);
|
||||
if (!svc.ok) {
|
||||
await edit(ticket.promptMessageId, ticket.topicId, `⚠️ Файл ${label} получен, но ссылку для MAX подготовить не удалось — ${svc.reason}. Файл сохранён до ${formatExpiry(file.expiresAt)}, ссылка — в пульте → 📁 Файлы.`);
|
||||
return;
|
||||
}
|
||||
const url = `${svc.url}/f/${file.token}`;
|
||||
const text = `📎 ${name} (${formatBytes(size)}) — файл больше 4 ГБ, MAX такие не принимает.\nСкачать до ${formatExpiry(file.expiresAt)}: ${url}${ticket.caption ? `\n\n${ticket.caption}` : ''}`;
|
||||
try {
|
||||
const sent = await max.sendMessage(ticket.maxChatId, text, []);
|
||||
onSentToMax(ticket, sent);
|
||||
await edit(
|
||||
ticket.promptMessageId,
|
||||
ticket.topicId,
|
||||
`✅ Файл ${label} больше 4 ГБ — MAX такие не принимает, поэтому собеседнику ушла ссылка на скачивание (до ${formatExpiry(file.expiresAt)}). Ссылка ведёт на ваш сервер. Управлять файлом — в пульте → 📁 Файлы.`,
|
||||
);
|
||||
} catch (err) {
|
||||
logger.error(`Sending the link for ${name} to MAX failed`, err);
|
||||
await edit(ticket.promptMessageId, ticket.topicId, `⚠️ Ссылку на файл ${label} отправить в MAX не удалось: ${(err as Error).message}. Файл сохранён — ссылка в пульте → 📁 Файлы.`);
|
||||
}
|
||||
});
|
||||
|
||||
// --- MAX → Telegram -------------------------------------------------------------------------
|
||||
|
||||
async function relayFromMax(att: MaxAttachment, ctx: DownloadContext, opts: SendOpts): Promise<SentMessage> {
|
||||
const name = att.name ?? 'file';
|
||||
const size = Number(att.size ?? 0);
|
||||
const label = `«${name}» (${formatBytes(size)})`;
|
||||
if (!fileShare.enabled) return send(`${describeAttachment(att)} ${formatBytes(size)} — ${WHY_TG_UPLOAD}. Откройте его в MAX.`, opts);
|
||||
const room = await fileShare.roomFor(size);
|
||||
if (!room.fits) {
|
||||
return send(
|
||||
`📎 Файл ${label} — ${WHY_TG_UPLOAD}, а сохранить его для скачивания не выйдет: на сервере свободно ${formatBytes(room.freeBytes)}, нужно ещё ${reserveText} запаса для обновлений. Откройте его в MAX.`,
|
||||
opts,
|
||||
);
|
||||
}
|
||||
const url = await getMaxFileUrl(att, ctx);
|
||||
if (!url) return send(describeAttachment(att), opts);
|
||||
let file;
|
||||
try {
|
||||
const res = await maxFetchBig(url);
|
||||
if (!res.ok || !res.body) {
|
||||
if (isTransientHttpStatus(res.status)) throw new TransientDownloadError(`MAX CDN answered ${res.status}`);
|
||||
throw new Error(`MAX отдал ${res.status}`);
|
||||
}
|
||||
file = await fileShare.storeStream(nodeReadable(res.body), name, size);
|
||||
logger.info(`MAX -> TG: ${name} (${file.size} bytes) saved for a download link`);
|
||||
} catch (err) {
|
||||
const transient = err instanceof TransientDownloadError || isTransientNetworkError(err);
|
||||
if (transient && ctx.throwOnTransient) throw err instanceof TransientDownloadError ? err : new TransientDownloadError(`big file download: ${(err as Error).message}`, err);
|
||||
logger.error(`Saving the big MAX file ${name} failed`, err);
|
||||
return send(`📎 Файл ${label} — ${WHY_TG_UPLOAD}, а сохранить его для скачивания не удалось (${(err as Error).message}). Откройте его в MAX.`, opts);
|
||||
}
|
||||
const msg = await send(`📎 Файл ${label} — ${WHY_TG_UPLOAD}. Готовлю ссылку для скачивания…`, opts);
|
||||
// Not awaited: the service may take a minute or two to come up, and this runs inside the
|
||||
// chat's queue — the next message must not wait for it.
|
||||
void (async () => {
|
||||
const svc = await fileShare.ensureService();
|
||||
if (!svc.ok) {
|
||||
await edit(msg.message_id, opts.message_thread_id, `📎 Файл ${label} — ${WHY_TG_UPLOAD}. Ссылку подготовить не удалось: ${svc.reason}.\nФайл сохранён до ${formatExpiry(file.expiresAt)} — новую ссылку можно получить в пульте → 📁 Файлы.`);
|
||||
return;
|
||||
}
|
||||
await edit(
|
||||
msg.message_id,
|
||||
opts.message_thread_id,
|
||||
`📎 Файл ${label} — ${WHY_TG_UPLOAD}, поэтому он лежит на вашем сервере. Скачайте по ссылке до ${formatExpiry(file.expiresAt)} — потом файл удалится.`,
|
||||
Markup.inlineKeyboard([Markup.button.url('⬇️ Скачать', `${svc.url}/f/${file.token}`)]),
|
||||
);
|
||||
})().catch((err) => logger.error('Posting the download link failed', err));
|
||||
return msg;
|
||||
}
|
||||
|
||||
return { offerUpload, relayFromMax };
|
||||
}
|
||||
452
src/bridge/fileShare.ts
Normal file
452
src/bridge/fileShare.ts
Normal file
|
|
@ -0,0 +1,452 @@
|
|||
/**
|
||||
* Big files through links — what the Bot API can't carry. A bot may DOWNLOAD from Telegram only
|
||||
* ≤20 MB (getFile) and SEND only ≤50 MB, while MAX takes up to 4 GB. So:
|
||||
* - Telegram → MAX over 20 MB: the topic gets a one-time UPLOAD link; the file arrives through a
|
||||
* browser, then goes to MAX as a normal file (≤4 GB) or as a download link (bigger);
|
||||
* - MAX → Telegram over 50 MB: the bridge saves the file and the topic gets a DOWNLOAD link.
|
||||
*
|
||||
* The links are served by ONE shared service per host (files-service/: Caddy + a small Node app),
|
||||
* which the host's update dispatcher starts only while some bridge has a live link and stops when
|
||||
* none is left — so ports 80/443 are open only then. The bridge (non-root, no Docker access) talks
|
||||
* to it through files on the shared `.data/files` volume:
|
||||
* active — exists while this bridge has a live link (the host reads it every minute);
|
||||
* service.json — written by the host: {state, url, at, detail};
|
||||
* tokens/<t>.json — what a link may do (download a stored file / receive one upload);
|
||||
* items/<id>/… — stored files, deleted when their link expires;
|
||||
* incoming/<t>/… — uploads written by the service (`done.json` once complete).
|
||||
*/
|
||||
import path from 'node:path';
|
||||
import { createWriteStream } from 'node:fs';
|
||||
import { mkdir, readFile, readdir, rename, rm, stat, statfs, unlink, writeFile } from 'node:fs/promises';
|
||||
import { randomBytes } from 'node:crypto';
|
||||
import { pipeline } from 'node:stream/promises';
|
||||
import { Readable, Transform } from 'node:stream';
|
||||
import { createLogger } from '../logger.js';
|
||||
import { parseEnvBool } from '../env.js';
|
||||
|
||||
const logger = createLogger('files');
|
||||
|
||||
/** Bot API getFile refuses bigger files with "400: Bad Request: file is too big". */
|
||||
export const TELEGRAM_BOT_DOWNLOAD_LIMIT = 20 * 1024 * 1024;
|
||||
/** MAX's own limit for one file (the user's number, 2026-10-09); bigger goes to MAX as a link. */
|
||||
export const MAX_FILE_LIMIT = 4_000_000_000;
|
||||
|
||||
export interface StoredFile {
|
||||
id: string;
|
||||
name: string;
|
||||
size: number;
|
||||
/** max2tg: saved from MAX for the topic; tg2max: uploaded for MAX (over MAX's limit). */
|
||||
direction: 'max2tg' | 'tg2max';
|
||||
token: string;
|
||||
createdAt: number;
|
||||
expiresAt: number;
|
||||
}
|
||||
|
||||
export interface UploadTicket {
|
||||
token: string;
|
||||
maxChatId: string;
|
||||
topicId: number;
|
||||
/** The Telegram message with the too-big file — the MAX copy gets linked to it. */
|
||||
telegramMessageId: number;
|
||||
/** The bot's «загрузите по ссылке» message, edited as the upload progresses. */
|
||||
promptMessageId?: number;
|
||||
caption?: string;
|
||||
name: string;
|
||||
expectedSize: number;
|
||||
createdAt: number;
|
||||
expiresAt: number;
|
||||
}
|
||||
|
||||
interface Index {
|
||||
files: StoredFile[];
|
||||
tickets: UploadTicket[];
|
||||
}
|
||||
|
||||
export type ServiceResult = { ok: true; url: string } | { ok: false; reason: string };
|
||||
|
||||
/** What the host writes into service.json. */
|
||||
interface ServiceState {
|
||||
state?: string;
|
||||
url?: string;
|
||||
/** Unix seconds. */
|
||||
at?: number;
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
export interface FileShareOptions {
|
||||
enabled: boolean;
|
||||
ttlMs: number;
|
||||
/** Kept free on top of any file: an update needs ~1.5 GB, a full disk would block it. */
|
||||
reserveBytes: number;
|
||||
}
|
||||
|
||||
export function fileShareOptionsFromEnv(env: NodeJS.ProcessEnv = process.env): FileShareOptions {
|
||||
const ttlHours = Number(env.FILES_TTL_HOURS);
|
||||
const reserveMb = Number(env.FILES_RESERVE_MB);
|
||||
return {
|
||||
enabled: parseEnvBool(env.FILES) !== false,
|
||||
ttlMs: (Number.isFinite(ttlHours) && ttlHours > 0 ? ttlHours : 72) * 3600_000,
|
||||
reserveBytes: (Number.isFinite(reserveMb) && reserveMb >= 0 ? reserveMb : 1536) * 1024 * 1024,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A file name safe for a path segment and a Content-Disposition: no separators, no control
|
||||
* characters, no leading dots, at most 180 UTF-16 units (extension kept). Pure + exported for tests.
|
||||
*/
|
||||
export function safeFileName(raw: string | undefined): string {
|
||||
let name = (raw ?? '').replace(/[\u0000-\u001f\u007f/\\:*?"<>|]/g, '_').trim();
|
||||
name = name.replace(/^\.+/, '').trim();
|
||||
if (!name) return 'file';
|
||||
if (name.length > 180) {
|
||||
const dot = name.lastIndexOf('.');
|
||||
const ext = dot > 0 && name.length - dot <= 16 ? name.slice(dot) : '';
|
||||
name = name.slice(0, 180 - ext.length) + ext;
|
||||
}
|
||||
return name;
|
||||
}
|
||||
|
||||
const newToken = (): string => randomBytes(18).toString('base64url');
|
||||
const sleep = (ms: number) => new Promise<void>((r) => setTimeout(r, ms));
|
||||
|
||||
/** How long service.json stays believable: the host rewrites it every minute while it matters. */
|
||||
const SERVICE_STATE_FRESH_MS = 150_000;
|
||||
const SERVICE_WAIT_MS = 150_000;
|
||||
const HEARTBEAT_STALE_MS = 5 * 60_000;
|
||||
|
||||
export class FileShare {
|
||||
readonly dir: string;
|
||||
private readonly indexPath: string;
|
||||
private index: Index = { files: [], tickets: [] };
|
||||
private chain: Promise<unknown> = Promise.resolve();
|
||||
private uploadHandler: ((t: UploadTicket, filePath: string, size: number, name: string) => Promise<void>) | null = null;
|
||||
private busyTokens = new Set<string>();
|
||||
|
||||
constructor(
|
||||
dataDir: string,
|
||||
readonly opts: FileShareOptions,
|
||||
) {
|
||||
this.dir = path.join(dataDir, 'files');
|
||||
this.indexPath = path.join(this.dir, 'index.json');
|
||||
}
|
||||
|
||||
get enabled(): boolean {
|
||||
return this.opts.enabled;
|
||||
}
|
||||
|
||||
async init(): Promise<void> {
|
||||
await mkdir(path.join(this.dir, 'items'), { recursive: true });
|
||||
await mkdir(path.join(this.dir, 'tokens'), { recursive: true });
|
||||
await mkdir(path.join(this.dir, 'incoming'), { recursive: true });
|
||||
try {
|
||||
const parsed = JSON.parse(await readFile(this.indexPath, 'utf8')) as Partial<Index>;
|
||||
this.index = { files: Array.isArray(parsed.files) ? parsed.files : [], tickets: Array.isArray(parsed.tickets) ? parsed.tickets : [] };
|
||||
} catch {
|
||||
this.index = { files: [], tickets: [] };
|
||||
}
|
||||
await this.sweep();
|
||||
}
|
||||
|
||||
/** Serialized: index writes and the token files they imply never interleave. */
|
||||
private locked<T>(fn: () => Promise<T>): Promise<T> {
|
||||
const run = this.chain.then(fn, fn);
|
||||
this.chain = run.catch(() => {});
|
||||
return run;
|
||||
}
|
||||
|
||||
private async save(): Promise<void> {
|
||||
const tmp = `${this.indexPath}.tmp`;
|
||||
await writeFile(tmp, JSON.stringify(this.index, null, 2), 'utf8');
|
||||
await rename(tmp, this.indexPath);
|
||||
await this.syncActiveMarker();
|
||||
}
|
||||
|
||||
/** `active` exists exactly while a link is alive — the host opens ports 80/443 only then. */
|
||||
private async syncActiveMarker(): Promise<void> {
|
||||
const marker = path.join(this.dir, 'active');
|
||||
if (this.index.files.length + this.index.tickets.length > 0) {
|
||||
await writeFile(marker, new Date().toISOString(), 'utf8');
|
||||
} else {
|
||||
await unlink(marker).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
private async writeToken(token: string, body: Record<string, unknown>): Promise<void> {
|
||||
const file = path.join(this.dir, 'tokens', `${token}.json`);
|
||||
await writeFile(`${file}.tmp`, JSON.stringify({ v: 1, ...body }), 'utf8');
|
||||
await rename(`${file}.tmp`, file);
|
||||
}
|
||||
|
||||
private async dropToken(token: string): Promise<void> {
|
||||
await unlink(path.join(this.dir, 'tokens', `${token}.json`)).catch(() => {});
|
||||
}
|
||||
|
||||
list(): StoredFile[] {
|
||||
return [...this.index.files].sort((a, b) => b.createdAt - a.createdAt);
|
||||
}
|
||||
|
||||
get(id: string): StoredFile | undefined {
|
||||
return this.index.files.find((f) => f.id === id);
|
||||
}
|
||||
|
||||
pendingUploads(): number {
|
||||
return this.index.tickets.length;
|
||||
}
|
||||
|
||||
/** Free bytes left for a file of `size` after the reserve; negative when it won't fit. */
|
||||
async roomFor(size: number): Promise<{ fits: boolean; freeBytes: number }> {
|
||||
const s = await statfs(this.dir);
|
||||
const freeBytes = Number(s.bavail) * Number(s.bsize);
|
||||
return { fits: freeBytes - this.opts.reserveBytes >= size, freeBytes };
|
||||
}
|
||||
|
||||
// --- Telegram → MAX ---------------------------------------------------------------------
|
||||
|
||||
async createUploadTicket(t: Omit<UploadTicket, 'token' | 'createdAt' | 'expiresAt'>): Promise<UploadTicket> {
|
||||
return this.locked(async () => {
|
||||
const now = Date.now();
|
||||
const ticket: UploadTicket = { ...t, token: newToken(), createdAt: now, expiresAt: now + this.opts.ttlMs };
|
||||
await this.writeToken(ticket.token, {
|
||||
kind: 'upload',
|
||||
name: ticket.name,
|
||||
expectedSize: ticket.expectedSize,
|
||||
expiresAt: ticket.expiresAt,
|
||||
reserveBytes: this.opts.reserveBytes,
|
||||
});
|
||||
this.index.tickets.push(ticket);
|
||||
await this.save();
|
||||
return ticket;
|
||||
});
|
||||
}
|
||||
|
||||
async setPromptMessage(token: string, messageId: number): Promise<void> {
|
||||
await this.locked(async () => {
|
||||
const t = this.index.tickets.find((x) => x.token === token);
|
||||
if (!t) return;
|
||||
t.promptMessageId = messageId;
|
||||
await this.save();
|
||||
});
|
||||
}
|
||||
|
||||
async dropTicket(token: string): Promise<void> {
|
||||
await this.locked(async () => {
|
||||
this.index.tickets = this.index.tickets.filter((t) => t.token !== token);
|
||||
await this.dropToken(token);
|
||||
await rm(path.join(this.dir, 'incoming', token), { recursive: true, force: true });
|
||||
await this.save();
|
||||
});
|
||||
}
|
||||
|
||||
/** Moves a finished upload into storage (a file over MAX's limit lives on as a download link). */
|
||||
async adoptUpload(ticket: UploadTicket, filePath: string, size: number, name: string): Promise<StoredFile> {
|
||||
const file = await this.locked(async () => {
|
||||
const id = newToken();
|
||||
await mkdir(path.join(this.dir, 'items', id), { recursive: true });
|
||||
const target = path.join(this.dir, 'items', id, safeFileName(name));
|
||||
await rename(filePath, target);
|
||||
return this.addFile(id, safeFileName(name), size, 'tg2max');
|
||||
});
|
||||
await this.dropTicket(ticket.token);
|
||||
return file;
|
||||
}
|
||||
|
||||
/** Called with each finished upload; the bridge sends it to MAX. */
|
||||
onUpload(handler: (t: UploadTicket, filePath: string, size: number, name: string) => Promise<void>): void {
|
||||
this.uploadHandler = handler;
|
||||
}
|
||||
|
||||
// --- MAX → Telegram ---------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Streams `body` into storage and opens a download link for it. Throws when the stream fails or
|
||||
* runs past `size` + the free space (the partial file is removed either way).
|
||||
*/
|
||||
async storeStream(body: Readable, name: string, size: number): Promise<StoredFile> {
|
||||
const id = newToken();
|
||||
const safe = safeFileName(name);
|
||||
const dir = path.join(this.dir, 'items', id);
|
||||
await mkdir(dir, { recursive: true });
|
||||
const part = path.join(dir, `${safe}.part`);
|
||||
let written = 0;
|
||||
const limit = Math.max(size, 0) + 1024 * 1024; // the declared size, a little slack
|
||||
const guard = new Transform({
|
||||
transform(chunk: Buffer, _enc, cb) {
|
||||
written += chunk.length;
|
||||
if (written > limit) cb(new Error(`file grew past its declared ${size} bytes`));
|
||||
else cb(null, chunk);
|
||||
},
|
||||
});
|
||||
try {
|
||||
await pipeline(body, guard, createWriteStream(part));
|
||||
await rename(part, path.join(dir, safe));
|
||||
} catch (err) {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
throw err;
|
||||
}
|
||||
return this.locked(() => this.addFile(id, safe, written, 'max2tg'));
|
||||
}
|
||||
|
||||
private async addFile(id: string, name: string, size: number, direction: StoredFile['direction']): Promise<StoredFile> {
|
||||
const now = Date.now();
|
||||
const file: StoredFile = { id, name, size, direction, token: newToken(), createdAt: now, expiresAt: now + this.opts.ttlMs };
|
||||
await this.writeToken(file.token, { kind: 'download', name, size, path: `items/${id}/${name}`, expiresAt: file.expiresAt });
|
||||
this.index.files.push(file);
|
||||
await this.save();
|
||||
return file;
|
||||
}
|
||||
|
||||
// --- the panel ----------------------------------------------------------------------------
|
||||
|
||||
/** A fresh link (the old one stops working) and a fresh TTL. */
|
||||
async renew(id: string): Promise<StoredFile | undefined> {
|
||||
return this.locked(async () => {
|
||||
const file = this.index.files.find((f) => f.id === id);
|
||||
if (!file) return undefined;
|
||||
await this.dropToken(file.token);
|
||||
file.token = newToken();
|
||||
file.expiresAt = Date.now() + this.opts.ttlMs;
|
||||
await this.writeToken(file.token, { kind: 'download', name: file.name, size: file.size, path: `items/${id}/${file.name}`, expiresAt: file.expiresAt });
|
||||
await this.save();
|
||||
return file;
|
||||
});
|
||||
}
|
||||
|
||||
async remove(id: string): Promise<boolean> {
|
||||
return this.locked(async () => {
|
||||
const file = this.index.files.find((f) => f.id === id);
|
||||
if (!file) return false;
|
||||
this.index.files = this.index.files.filter((f) => f.id !== id);
|
||||
await this.dropToken(file.token);
|
||||
await rm(path.join(this.dir, 'items', id), { recursive: true, force: true });
|
||||
await this.save();
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
// --- the host service -----------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Waits (≤2.5 min: the host checks once a minute, then Caddy may need a certificate) until the
|
||||
* shared files service answers for this bridge. Call AFTER creating the link — its `active`
|
||||
* marker is what makes the host start the service.
|
||||
*/
|
||||
async ensureService(waitMs = SERVICE_WAIT_MS): Promise<ServiceResult> {
|
||||
if (!this.opts.enabled) return { ok: false, reason: 'пересылка больших файлов отключена в настройках (FILES=off)' };
|
||||
await this.syncActiveMarker();
|
||||
const deadline = Date.now() + waitMs;
|
||||
let heartbeatChecked = false;
|
||||
for (;;) {
|
||||
const st = await this.readServiceState();
|
||||
const fresh = st?.at != null && Date.now() - st.at * 1000 < SERVICE_STATE_FRESH_MS;
|
||||
if (fresh && st?.state === 'up' && st.url) return { ok: true, url: st.url.replace(/\/+$/, '') };
|
||||
if (fresh && st?.state && st.state !== 'up' && st.state !== 'starting' && st.state !== 'down') {
|
||||
return { ok: false, reason: describeServiceProblem(st) };
|
||||
}
|
||||
if (!heartbeatChecked) {
|
||||
heartbeatChecked = true;
|
||||
if (!(await this.watcherAlive())) {
|
||||
return {
|
||||
ok: false,
|
||||
reason: 'службу файлов запускает автообновление, а оно на этом сервере не работает (setup.sh запускали не от root?)',
|
||||
};
|
||||
}
|
||||
}
|
||||
if (Date.now() >= deadline) return { ok: false, reason: 'служба файлов не запустилась за 2,5 минуты — подробности в /var/lib/telemax-files на сервере' };
|
||||
await sleep(3000);
|
||||
}
|
||||
}
|
||||
|
||||
private async readServiceState(): Promise<ServiceState | null> {
|
||||
try {
|
||||
return JSON.parse(await readFile(path.join(this.dir, 'service.json'), 'utf8')) as ServiceState;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private async watcherAlive(): Promise<boolean> {
|
||||
try {
|
||||
const s = await stat(path.join(this.dir, '..', 'watcher-heartbeat'));
|
||||
return Date.now() - s.mtimeMs < HEARTBEAT_STALE_MS;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// --- housekeeping ---------------------------------------------------------------------------
|
||||
|
||||
/** Drops expired links with their files, and anything on disk the index no longer knows. */
|
||||
async sweep(): Promise<void> {
|
||||
await this.locked(async () => {
|
||||
const now = Date.now();
|
||||
const expiredFiles = this.index.files.filter((f) => f.expiresAt <= now);
|
||||
const expiredTickets = this.index.tickets.filter((t) => t.expiresAt <= now && !this.busyTokens.has(t.token));
|
||||
this.index.files = this.index.files.filter((f) => f.expiresAt > now);
|
||||
this.index.tickets = this.index.tickets.filter((t) => !expiredTickets.includes(t));
|
||||
for (const f of expiredFiles) logger.info(`File «${f.name}» expired — deleted`);
|
||||
const liveTokens = new Set([...this.index.files.map((f) => f.token), ...this.index.tickets.map((t) => t.token)]);
|
||||
const liveItems = new Set(this.index.files.map((f) => f.id));
|
||||
for (const name of await readdir(path.join(this.dir, 'tokens')).catch(() => [] as string[])) {
|
||||
if (!liveTokens.has(name.replace(/\.json(\.tmp)?$/, ''))) await unlink(path.join(this.dir, 'tokens', name)).catch(() => {});
|
||||
}
|
||||
for (const name of await readdir(path.join(this.dir, 'items')).catch(() => [] as string[])) {
|
||||
if (!liveItems.has(name)) await rm(path.join(this.dir, 'items', name), { recursive: true, force: true });
|
||||
}
|
||||
for (const name of await readdir(path.join(this.dir, 'incoming')).catch(() => [] as string[])) {
|
||||
if (!liveTokens.has(name)) await rm(path.join(this.dir, 'incoming', name), { recursive: true, force: true });
|
||||
}
|
||||
await this.save();
|
||||
});
|
||||
}
|
||||
|
||||
/** Hands every finished upload to the handler, one at a time. */
|
||||
async pollUploads(): Promise<void> {
|
||||
if (!this.uploadHandler) return;
|
||||
for (const ticket of [...this.index.tickets]) {
|
||||
if (this.busyTokens.has(ticket.token)) continue;
|
||||
const dir = path.join(this.dir, 'incoming', ticket.token);
|
||||
let done: { name?: string; size?: number; file?: string } | null;
|
||||
try {
|
||||
done = JSON.parse(await readFile(path.join(dir, 'done.json'), 'utf8')) as typeof done;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const name = safeFileName(done?.name ?? ticket.name);
|
||||
const filePath = path.join(dir, done?.file ?? name);
|
||||
this.busyTokens.add(ticket.token);
|
||||
try {
|
||||
const size = (await stat(filePath)).size;
|
||||
logger.info(`Upload for «${name}» (${size} bytes) arrived — sending it on`);
|
||||
await this.uploadHandler(ticket, filePath, size, name);
|
||||
} catch (err) {
|
||||
logger.error(`Processing the upload «${name}» failed`, err);
|
||||
} finally {
|
||||
this.busyTokens.delete(ticket.token);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Upload polling every 5 s, sweeping every 10 min. Timers are unref'd. */
|
||||
startBackground(): void {
|
||||
const poll = setInterval(() => void this.pollUploads().catch((err) => logger.error('pollUploads failed', err)), 5000);
|
||||
poll.unref();
|
||||
const sweep = setInterval(() => void this.sweep().catch((err) => logger.error('sweep failed', err)), 10 * 60_000);
|
||||
sweep.unref();
|
||||
}
|
||||
}
|
||||
|
||||
function describeServiceProblem(st: ServiceState): string {
|
||||
switch (st.state) {
|
||||
case 'ports-busy':
|
||||
return 'порты 80/443 на сервере заняты другим сервисом';
|
||||
case 'no-ip':
|
||||
return 'не удалось определить внешний IPv4-адрес сервера (можно указать свой домен: FILES_DOMAIN в .env)';
|
||||
default:
|
||||
return `служба файлов не запустилась${st.detail ? `: ${st.detail}` : ''}`;
|
||||
}
|
||||
}
|
||||
|
||||
/** Readable from a WHATWG stream (undici's Response.body). */
|
||||
export function nodeReadable(body: ReadableStream<Uint8Array>): Readable {
|
||||
return Readable.fromWeb(body as Parameters<typeof Readable.fromWeb>[0]);
|
||||
}
|
||||
|
|
@ -3,9 +3,11 @@ import { readFile, writeFile, mkdir } from 'node:fs/promises';
|
|||
import { Markup, type Telegraf, type Context } from 'telegraf';
|
||||
import type { MaxClient, MaxContactInfo } from '../max/client.js';
|
||||
import { getAppVersion } from './version.js';
|
||||
import { maskPhone } from './status.js';
|
||||
import { formatBytes, maskPhone } from './status.js';
|
||||
import { createLogger } from '../logger.js';
|
||||
import { truncateUtf16 } from './text.js';
|
||||
import { formatExpiry } from './bigFiles.js';
|
||||
import type { FileShare, StoredFile } from './fileShare.js';
|
||||
|
||||
const logger = createLogger('panel');
|
||||
|
||||
|
|
@ -38,6 +40,8 @@ export interface ControlPanelDeps {
|
|||
getStatus?: (pausedLabel: string | null) => Promise<string>;
|
||||
/** The MAX pause (server/maxSession.ts owns it): a deliberate disconnect with no auto-reconnect until resumed or the timer fires. */
|
||||
pause: PauseControl;
|
||||
/** Big files kept for download links — the «📁 Файлы» section. */
|
||||
files?: FileShare;
|
||||
}
|
||||
|
||||
export interface PauseControl {
|
||||
|
|
@ -135,6 +139,56 @@ function rootView(phone: string, pausedUntil: number | null): View {
|
|||
markup: Markup.inlineKeyboard([
|
||||
[Markup.button.callback('👤 Найти контакт', 'tlmx_panel:contacts'), Markup.button.callback('🚫 Чаты', 'tlmx_panel:chats')],
|
||||
[Markup.button.callback('🔐 Вход в MAX', 'tlmx_panel:web'), Markup.button.callback('⚙️ Система', 'tlmx_panel:system')],
|
||||
[Markup.button.callback('📁 Файлы', 'tlmx_panel:files')],
|
||||
]).reply_markup,
|
||||
};
|
||||
}
|
||||
|
||||
/** 72 h → «3 дня», 12 h → «12 ч». */
|
||||
function ttlText(ms: number): string {
|
||||
const hours = Math.round(ms / 3600_000);
|
||||
if (hours % 24 !== 0) return `${hours} ч`;
|
||||
const d = hours / 24;
|
||||
const word = d % 10 === 1 && d % 100 !== 11 ? 'день' : [2, 3, 4].includes(d % 10) && ![12, 13, 14].includes(d % 100) ? 'дня' : 'дней';
|
||||
return `${d} ${word}`;
|
||||
}
|
||||
|
||||
function filesView(files: FileShare): View {
|
||||
const back = [Markup.button.callback('◀️ Назад', 'tlmx_panel:root')];
|
||||
if (!files.enabled) {
|
||||
return { text: '📁 Пересылка больших файлов отключена: FILES=off в .env.', markup: Markup.inlineKeyboard([back]).reply_markup };
|
||||
}
|
||||
const list = files.list();
|
||||
const pending = files.pendingUploads();
|
||||
const pendingLine = pending ? `\n\n⏳ Ждут загрузки по ссылке: ${pending}.` : '';
|
||||
const ttl = ttlText(files.opts.ttlMs);
|
||||
if (list.length === 0) {
|
||||
return {
|
||||
text: `📁 Файлов нет.\n\nСюда попадают большие файлы, которые мост пересылает по ссылке: из MAX — больше 50 МБ, в MAX — больше 4 ГБ. Файл живёт ${ttl} от последней выданной ссылки, потом удаляется.${pendingLine}`,
|
||||
markup: Markup.inlineKeyboard([back]).reply_markup,
|
||||
};
|
||||
}
|
||||
const total = list.reduce((s, f) => s + f.size, 0);
|
||||
const lines = list.map((f, i) => `${i + 1}. ${f.direction === 'max2tg' ? '⬇️' : '⬆️'} ${f.name} — ${formatBytes(f.size)}, до ${formatExpiry(f.expiresAt)}`);
|
||||
return {
|
||||
text: `📁 Файлы на сервере: ${list.length}, всего ${formatBytes(total)}.\n⬇️ — из MAX, ⬆️ — для MAX. Ссылка живёт ${ttl}, потом файл удаляется.\n\n${lines.join('\n')}${pendingLine}`,
|
||||
markup: Markup.inlineKeyboard([
|
||||
...list.slice(0, 20).map((f, i) => [Markup.button.callback(`${i + 1}. ${truncateUtf16(f.name, 48)}`, `tlmx_panel:file:${f.id}`)]),
|
||||
back,
|
||||
]).reply_markup,
|
||||
};
|
||||
}
|
||||
|
||||
function fileCardView(f: StoredFile, files: FileShare, link?: string): View {
|
||||
const where = f.direction === 'max2tg' ? 'пришёл из MAX' : 'загружен для MAX';
|
||||
const linkLine = link ? `\n\n🔗 ${link}\nСсылку можно переслать — по ней откроется страница со скачиванием.` : '';
|
||||
return {
|
||||
text: `📄 ${f.name}\n${formatBytes(f.size)} · ${where}\nХранится до ${formatExpiry(f.expiresAt)}, потом удалится.${linkLine}`,
|
||||
markup: Markup.inlineKeyboard([
|
||||
...(link ? [[Markup.button.url('⬇️ Открыть ссылку', link)]] : []),
|
||||
[Markup.button.callback(`🔗 Новая ссылка (+${ttlText(files.opts.ttlMs)})`, `tlmx_panel:frenew:${f.id}`)],
|
||||
[Markup.button.callback('🗑 Удалить', `tlmx_panel:fdel:${f.id}`)],
|
||||
[Markup.button.callback('◀️ К файлам', 'tlmx_panel:files')],
|
||||
]).reply_markup,
|
||||
};
|
||||
}
|
||||
|
|
@ -197,7 +251,7 @@ function pauseView(): View {
|
|||
}
|
||||
|
||||
export function wireControlPanel(deps: ControlPanelDeps): void {
|
||||
const { bot, targetGroupId, max, getActivePhone, triggerFullResync, leaves, startDialog, resolveContactName, getStatus, pause } = deps;
|
||||
const { bot, targetGroupId, max, getActivePhone, triggerFullResync, leaves, startDialog, resolveContactName, getStatus, pause, files } = deps;
|
||||
const root = (): View => rootView(getActivePhone(), pause.until());
|
||||
const system = (): View => systemView(pause.until() != null);
|
||||
|
||||
|
|
@ -274,6 +328,44 @@ export function wireControlPanel(deps: ControlPanelDeps): void {
|
|||
await ctx.answerCbQuery();
|
||||
await edit(ctx, pauseView());
|
||||
});
|
||||
// --- Big files («📁 Файлы»): list, a fresh link (+TTL), delete ------------------------------
|
||||
bot.action('tlmx_panel:files', async (ctx) => {
|
||||
await ctx.answerCbQuery();
|
||||
if (files) await edit(ctx, filesView(files));
|
||||
});
|
||||
bot.action(/^tlmx_panel:file:(.+)$/, async (ctx) => {
|
||||
await ctx.answerCbQuery();
|
||||
const f = files?.get(ctx.match[1] ?? '');
|
||||
if (!files || !f) {
|
||||
if (files) await edit(ctx, filesView(files));
|
||||
return;
|
||||
}
|
||||
await edit(ctx, fileCardView(f, files));
|
||||
});
|
||||
bot.action(/^tlmx_panel:frenew:(.+)$/, async (ctx) => {
|
||||
const id = ctx.match[1] ?? '';
|
||||
// Answered first: the service may take a minute or two to start, far past the callback's life.
|
||||
await ctx.answerCbQuery('Готовлю ссылку…').catch(() => {});
|
||||
const f = files ? await files.renew(id) : undefined;
|
||||
if (!files || !f) {
|
||||
if (files) await edit(ctx, filesView(files));
|
||||
return;
|
||||
}
|
||||
await ctx.editMessageText(`⏳ Готовлю ссылку на «${f.name}»… Если служба файлов не запущена, это займёт до пары минут.`).catch(() => {});
|
||||
const svc = await files.ensureService();
|
||||
if (!svc.ok) {
|
||||
const card = fileCardView(f, files);
|
||||
await ctx.editMessageText(`${card.text}\n\n⚠️ Ссылку подготовить не удалось: ${svc.reason}.`, { reply_markup: card.markup }).catch(() => {});
|
||||
return;
|
||||
}
|
||||
await edit(ctx, fileCardView(f, files, `${svc.url}/f/${f.token}`));
|
||||
});
|
||||
bot.action(/^tlmx_panel:fdel:(.+)$/, async (ctx) => {
|
||||
const removed = files ? await files.remove(ctx.match[1] ?? '') : false;
|
||||
await ctx.answerCbQuery(removed ? 'Файл удалён' : 'Файла уже нет').catch(() => {});
|
||||
if (files) await edit(ctx, filesView(files));
|
||||
});
|
||||
|
||||
bot.action('tlmx_panel:dismiss', async (ctx) => {
|
||||
await ctx.answerCbQuery();
|
||||
await ctx.deleteMessage().catch(() => {});
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import {
|
|||
downloadMaxAttachment,
|
||||
describeAttachment,
|
||||
telegramSendKind,
|
||||
TELEGRAM_UPLOAD_LIMIT_BYTES,
|
||||
takeKeyboard,
|
||||
KEYBOARD_NA,
|
||||
KEYBOARD_PRESS,
|
||||
|
|
@ -23,6 +24,8 @@ import {
|
|||
} from './attachments.js';
|
||||
import { splitTelegramText, truncateCodePoints, truncateUtf16, MAX_TEXT_LIMIT, TELEGRAM_CAPTION_LIMIT, TELEGRAM_TEXT_LIMIT } from './text.js';
|
||||
import { uploadTelegramAttachmentToMax } from './upload.js';
|
||||
import { FileShare, fileShareOptionsFromEnv, TELEGRAM_BOT_DOWNLOAD_LIMIT } from './fileShare.js';
|
||||
import { createBigFiles } from './bigFiles.js';
|
||||
import { canRenderAnimatedStickers } from './lottie.js';
|
||||
import { reportBridgeError } from './errorReporter.js';
|
||||
import { wireControlPanel, type PauseControl } from './panel.js';
|
||||
|
|
@ -595,6 +598,9 @@ export function isRenderableAttach(att: MaxAttachment): boolean {
|
|||
type SentMessage = { message_id: number };
|
||||
type AttachmentSendOpts = { message_thread_id: number; reply_parameters?: { message_id: number; allow_sending_without_reply: boolean } };
|
||||
|
||||
/** Set by wireBridge: sends a MAX file over Telegram's bot upload limit as a download link. */
|
||||
let bigFileRelay: ((att: MaxAttachment, ctx: DownloadContext, opts: AttachmentSendOpts) => Promise<SentMessage>) | null = null;
|
||||
|
||||
/**
|
||||
* Sends a MAX message's attachments into a topic, one Telegram message each, and returns EVERY
|
||||
* sent message_id in order — the caller links them all (buildLinkIds). Each Telegram call has its
|
||||
|
|
@ -674,6 +680,10 @@ async function sendOneAttachment(
|
|||
? withFloodRetry(() => bot.telegram.sendContact(groupId, `+${String(phone)}`, displayName, opts))
|
||||
: withFloodRetry(() => bot.telegram.sendMessage(groupId, `👤 Контакт: ${displayName}`, opts));
|
||||
}
|
||||
// A file the bot can't upload to Telegram goes through a download link (bigFiles.ts).
|
||||
if (bigFileRelay && att._type === 'FILE' && Number(att.size ?? 0) > TELEGRAM_UPLOAD_LIMIT_BYTES) {
|
||||
return bigFileRelay(att, downloadCtx, opts);
|
||||
}
|
||||
const downloaded = await downloadMaxAttachment(att, downloadCtx);
|
||||
// Bot API upload limits (50 MB, a photo 10 MB) — see telegramSendKind.
|
||||
const kind = downloaded ? telegramSendKind(downloaded.kind, downloaded.buffer.byteLength) : null;
|
||||
|
|
@ -1548,6 +1558,25 @@ export function wireBridge({
|
|||
chatSync.runInChat(chatId, () => chatSync.advanceCursor(chatId, serverTime, 'outgoing send')).catch(() => undefined);
|
||||
}
|
||||
|
||||
// Big files through links (fileShare.ts / bigFiles.ts): Telegram's bot limits are 20 MB down and
|
||||
// 50 MB up, MAX takes 4 GB.
|
||||
const fileShare = new FileShare(path.join(process.cwd(), '.data'), fileShareOptionsFromEnv());
|
||||
void fileShare
|
||||
.init()
|
||||
.then(() => fileShare.startBackground())
|
||||
.catch((err) => logger.error('Big-file storage failed to start', err));
|
||||
const bigFiles = createBigFiles({
|
||||
bot,
|
||||
max,
|
||||
targetGroupId,
|
||||
fileShare,
|
||||
onSentToMax: (ticket, sent) => {
|
||||
rememberOutgoingSend(ticket.maxChatId, sent.cid, sent.time);
|
||||
messageLinks.add({ maxChatId: ticket.maxChatId, maxMessageId: sent.messageId, telegramMessageId: ticket.telegramMessageId, telegramTopicId: ticket.topicId, outgoing: true });
|
||||
},
|
||||
});
|
||||
bigFileRelay = bigFiles.relayFromMax;
|
||||
|
||||
// `${chatId}:${messageId}` -> what we last relayed, so the sticky lastReactedMessageId/lastReaction
|
||||
// fields on CHAT_UPDATE (repeated across unrelated pushes) don't re-trigger the same Telegram call.
|
||||
// Doubles as the poll list for reaction removal; keeps the original messageId (BigInt) for MSG_GET_REACTIONS.
|
||||
|
|
@ -2781,6 +2810,7 @@ export function wireBridge({
|
|||
triggerFullResync,
|
||||
startDialog,
|
||||
pause,
|
||||
files: fileShare,
|
||||
// Warm-cache name lookup (buildRoster already fetched the profiles); undefined on a miss lets
|
||||
// the panel fall back to CONTACT_INFO.
|
||||
resolveContactName: (uid) => {
|
||||
|
|
@ -3514,6 +3544,22 @@ export function wireBridge({
|
|||
return;
|
||||
}
|
||||
|
||||
// Over the Bot API download limit getFile answers «file is too big» — offer an upload link
|
||||
// instead (bigFiles.ts). Fire-and-forget: the link can take a minute or two to get ready, and
|
||||
// this handler must not hold up the next update meanwhile.
|
||||
const fileFields = ctx.message as unknown as Record<string, { file_size?: number; file_name?: string } | undefined>;
|
||||
const tooBigKey = ['document', 'video', 'audio', 'animation', 'video_note', 'voice'].find(
|
||||
(k) => (fileFields[k]?.file_size ?? 0) > TELEGRAM_BOT_DOWNLOAD_LIMIT,
|
||||
);
|
||||
if (tooBigKey) {
|
||||
const f = fileFields[tooBigKey]!;
|
||||
const name = f.file_name ?? (tooBigKey === 'video_note' ? 'video_note.mp4' : tooBigKey === 'voice' ? 'voice.ogg' : 'file');
|
||||
void bigFiles
|
||||
.offerUpload({ maxChatId: mapping.maxChatId, topicId, telegramMessageId: ctx.message.message_id, caption: caption || undefined, name, size: f.file_size! })
|
||||
.catch((err) => logger.error('Offering an upload link failed', err));
|
||||
return;
|
||||
}
|
||||
|
||||
let attach: Record<string, unknown> | null = null;
|
||||
const largestPhoto = photo?.[photo.length - 1];
|
||||
if (largestPhoto) {
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
*/
|
||||
import { gzipSync } from 'node:zlib';
|
||||
import https from 'node:https';
|
||||
import { createReadStream } from 'node:fs';
|
||||
import type { Agent } from 'node:http';
|
||||
import type { Telegraf } from 'telegraf';
|
||||
// undici's own FormData, not the global one: maxFetch runs on undici's fetch, and
|
||||
|
|
@ -13,7 +14,7 @@ import type { Telegraf } from 'telegraf';
|
|||
// undici-types copy behind the global — same class at runtime either way.
|
||||
import { FormData } from 'undici';
|
||||
import type { MaxClient } from '../max/client.js';
|
||||
import { maxFetch } from '../max/ca.js';
|
||||
import { maxFetch, maxFetchBig } from '../max/ca.js';
|
||||
import { getTelegramProxyAgent } from '../telegram/proxy.js';
|
||||
import { renderTgsToWebm } from './lottie.js';
|
||||
|
||||
|
|
@ -116,6 +117,34 @@ async function uploadFileToMax(max: MaxClient, buffer: Buffer, filename: string)
|
|||
return { _type: 'FILE', fileId: slot.fileId };
|
||||
}
|
||||
|
||||
/**
|
||||
* uploadFileToMax for a file on disk, streamed — a multi-gigabyte upload (fileShare.ts) must not
|
||||
* pass through memory. Same slot, headers and Content-Range shape as the buffered version.
|
||||
*/
|
||||
export async function uploadFileFromDiskToMax(max: MaxClient, filePath: string, size: number, filename: string): Promise<{ _type: 'FILE'; fileId: unknown }> {
|
||||
const slot = await max.requestFileUpload();
|
||||
const res = await maxFetchBig(slot.url, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/octet-stream',
|
||||
'Content-Range': `bytes 0-${size - 1}/${size}`,
|
||||
'Content-Length': String(size),
|
||||
'Content-Disposition': contentDispositionFor(filename),
|
||||
},
|
||||
body: createReadStream(filePath),
|
||||
duplex: 'half',
|
||||
});
|
||||
if (!res.ok) {
|
||||
const bodyText = await res.text().catch(() => '<unreadable>');
|
||||
throw new Error(`File upload POST failed: ${res.status} ${res.statusText} — ${bodyText}`);
|
||||
}
|
||||
await res.body?.cancel().catch(() => {});
|
||||
// The buffered path's 3 s, plus time for MAX to digest a big file before MSG_SEND may use it
|
||||
// (a guess: +1 s per 100 MB, capped at a minute).
|
||||
await new Promise((resolve) => setTimeout(resolve, 3000 + Math.min(60_000, Math.floor(size / (100 * 1024 * 1024)) * 1000)));
|
||||
return { _type: 'FILE', fileId: slot.fileId };
|
||||
}
|
||||
|
||||
/**
|
||||
* Voice notes go through the SAME upload opcode as video (0x52, `type: 2`), not
|
||||
* FILE_UPLOAD — confirmed live by the user 2026-08-13 after two earlier guesses
|
||||
|
|
|
|||
|
|
@ -58,3 +58,12 @@ const maxDispatcher = new Agent({ connect: { ca: MAX_TLS_CA } });
|
|||
export function maxFetch(url: string, init: RequestInit = {}): Promise<Response> {
|
||||
return undiciFetch(url, { ...init, dispatcher: maxDispatcher });
|
||||
}
|
||||
|
||||
// Gigabyte transfers (fileShare.ts): the defaults (5 min to the response headers) are fine for a
|
||||
// photo but not for MAX digesting a 4 GB upload; bodyTimeout stays an IDLE limit between chunks.
|
||||
const maxBigDispatcher = new Agent({ connect: { ca: MAX_TLS_CA }, headersTimeout: 30 * 60_000, bodyTimeout: 5 * 60_000 });
|
||||
|
||||
/** maxFetch for streamed multi-gigabyte up/downloads — same trust set, longer timeouts. */
|
||||
export function maxFetchBig(url: string, init: RequestInit = {}): Promise<Response> {
|
||||
return undiciFetch(url, { ...init, dispatcher: maxBigDispatcher });
|
||||
}
|
||||
|
|
|
|||
171
tests/fileShare.test.ts
Normal file
171
tests/fileShare.test.ts
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { Readable } from 'node:stream';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { mkdir, mkdtemp, readFile, readdir, rm, utimes, writeFile } from 'node:fs/promises';
|
||||
import { FileShare, fileShareOptionsFromEnv, safeFileName } from '../src/bridge/fileShare.js';
|
||||
|
||||
let dataDir: string;
|
||||
const opts = { enabled: true, ttlMs: 72 * 3600_000, reserveBytes: 0 };
|
||||
|
||||
beforeEach(async () => {
|
||||
dataDir = await mkdtemp(path.join(os.tmpdir(), 'tlmx-files-'));
|
||||
});
|
||||
afterEach(async () => {
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
async function fresh(o = opts): Promise<FileShare> {
|
||||
const share = new FileShare(dataDir, o);
|
||||
await share.init();
|
||||
return share;
|
||||
}
|
||||
const token = (share: FileShare, t: string) => path.join(share.dir, 'tokens', `${t}.json`);
|
||||
|
||||
describe('safeFileName', () => {
|
||||
it('keeps a normal name and neutralizes separators, control chars and leading dots', () => {
|
||||
expect(safeFileName('Документы.zip')).toBe('Документы.zip');
|
||||
expect(safeFileName('../../etc/passwd')).toBe('_.._etc_passwd');
|
||||
expect(safeFileName('a\u0000b\nc:d?.txt')).toBe('a_b_c_d_.txt');
|
||||
expect(safeFileName('...')).toBe('file');
|
||||
expect(safeFileName(undefined)).toBe('file');
|
||||
});
|
||||
|
||||
it('cuts a long name but keeps its extension', () => {
|
||||
const n = safeFileName(`${'я'.repeat(300)}.tar.gz`);
|
||||
expect(n.length).toBe(180);
|
||||
expect(n.endsWith('.gz')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('fileShareOptionsFromEnv', () => {
|
||||
it('is on by default with 3 days and a 1.5 GB reserve', () => {
|
||||
expect(fileShareOptionsFromEnv({})).toEqual({ enabled: true, ttlMs: 72 * 3600_000, reserveBytes: 1536 * 1024 * 1024 });
|
||||
expect(fileShareOptionsFromEnv({ FILES: 'off', FILES_TTL_HOURS: '24', FILES_RESERVE_MB: '0' })).toEqual({ enabled: false, ttlMs: 24 * 3600_000, reserveBytes: 0 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('FileShare', () => {
|
||||
it('a stored file gets a download token, the active marker, and survives a restart', async () => {
|
||||
const share = await fresh();
|
||||
const f = await share.storeStream(Readable.from([Buffer.from('hello')]), 'a.txt', 5);
|
||||
expect(await readFile(path.join(share.dir, 'items', f.id, 'a.txt'), 'utf8')).toBe('hello');
|
||||
const t = JSON.parse(await readFile(token(share, f.token), 'utf8'));
|
||||
expect(t).toMatchObject({ v: 1, kind: 'download', name: 'a.txt', size: 5, path: `items/${f.id}/a.txt` });
|
||||
expect(existsSync(path.join(share.dir, 'active'))).toBe(true);
|
||||
const again = await fresh();
|
||||
expect(again.list().map((x) => x.id)).toEqual([f.id]);
|
||||
});
|
||||
|
||||
it('renew issues a new token and a new TTL; remove deletes the file and closes the marker', async () => {
|
||||
const share = await fresh();
|
||||
const f = await share.storeStream(Readable.from([Buffer.from('x')]), 'x.bin', 1);
|
||||
const oldToken = f.token;
|
||||
const renewed = await share.renew(f.id);
|
||||
expect(renewed?.token).not.toBe(oldToken);
|
||||
expect(existsSync(token(share, oldToken))).toBe(false);
|
||||
expect(existsSync(token(share, renewed!.token))).toBe(true);
|
||||
expect(await share.remove(f.id)).toBe(true);
|
||||
expect(existsSync(path.join(share.dir, 'items', f.id))).toBe(false);
|
||||
expect(existsSync(path.join(share.dir, 'active'))).toBe(false);
|
||||
});
|
||||
|
||||
it('a stream that runs past its declared size is refused and leaves nothing behind', async () => {
|
||||
const share = await fresh();
|
||||
const big = Buffer.alloc(3 * 1024 * 1024, 1);
|
||||
await expect(share.storeStream(Readable.from([big]), 'lie.bin', 10)).rejects.toThrow(/declared/);
|
||||
expect(await readdir(path.join(share.dir, 'items'))).toEqual([]);
|
||||
});
|
||||
|
||||
it('sweep deletes expired files with their tokens', async () => {
|
||||
const share = await fresh({ ...opts, ttlMs: 1 });
|
||||
const f = await share.storeStream(Readable.from([Buffer.from('x')]), 'old.bin', 1);
|
||||
await new Promise((r) => setTimeout(r, 5));
|
||||
await share.sweep();
|
||||
expect(share.list()).toEqual([]);
|
||||
expect(existsSync(token(share, f.token))).toBe(false);
|
||||
expect(existsSync(path.join(share.dir, 'items', f.id))).toBe(false);
|
||||
});
|
||||
|
||||
it('an upload ticket hands the finished upload to the handler once', async () => {
|
||||
const share = await fresh();
|
||||
const ticket = await share.createUploadTicket({ maxChatId: '5', topicId: 7, telegramMessageId: 9, name: 'big.zip', expectedSize: 3 });
|
||||
expect(JSON.parse(await readFile(token(share, ticket.token), 'utf8'))).toMatchObject({ kind: 'upload', name: 'big.zip', expectedSize: 3 });
|
||||
const seen: Array<[string, number, string]> = [];
|
||||
share.onUpload(async (t, filePath, size, name) => {
|
||||
seen.push([t.token, size, name]);
|
||||
expect(await readFile(filePath, 'utf8')).toBe('abc');
|
||||
await share.dropTicket(t.token);
|
||||
});
|
||||
await share.pollUploads(); // nothing uploaded yet
|
||||
const dir = path.join(share.dir, 'incoming', ticket.token);
|
||||
await mkdir(dir, { recursive: true });
|
||||
await writeFile(path.join(dir, 'other.zip'), 'abc');
|
||||
await writeFile(path.join(dir, 'done.json'), JSON.stringify({ name: 'other.zip', file: 'other.zip', size: 3 }));
|
||||
await share.pollUploads();
|
||||
await share.pollUploads();
|
||||
expect(seen).toEqual([[ticket.token, 3, 'other.zip']]);
|
||||
expect(share.pendingUploads()).toBe(0);
|
||||
expect(existsSync(dir)).toBe(false);
|
||||
});
|
||||
|
||||
it('adoptUpload keeps a too-big upload as a stored file with a link', async () => {
|
||||
const share = await fresh();
|
||||
const ticket = await share.createUploadTicket({ maxChatId: '5', topicId: 7, telegramMessageId: 9, name: 'huge.iso', expectedSize: 2 });
|
||||
const dir = path.join(share.dir, 'incoming', ticket.token);
|
||||
await mkdir(dir, { recursive: true });
|
||||
await writeFile(path.join(dir, 'huge.iso'), 'zz');
|
||||
const f = await share.adoptUpload(ticket, path.join(dir, 'huge.iso'), 2, 'huge.iso');
|
||||
expect(f.direction).toBe('tg2max');
|
||||
expect(await readFile(path.join(share.dir, 'items', f.id, 'huge.iso'), 'utf8')).toBe('zz');
|
||||
expect(share.pendingUploads()).toBe(0);
|
||||
});
|
||||
|
||||
it('roomFor counts the reserve', async () => {
|
||||
const share = await fresh({ ...opts, reserveBytes: Number.MAX_SAFE_INTEGER });
|
||||
expect((await share.roomFor(1)).fits).toBe(false);
|
||||
expect((await (await fresh()).roomFor(1)).fits).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('FileShare.ensureService', () => {
|
||||
const heartbeat = async (ageMs = 0) => {
|
||||
const p = path.join(dataDir, 'watcher-heartbeat');
|
||||
await writeFile(p, 'x');
|
||||
const t = (Date.now() - ageMs) / 1000;
|
||||
await utimes(p, t, t);
|
||||
};
|
||||
const service = async (share: FileShare, state: string, url = '', ageSec = 0) =>
|
||||
writeFile(path.join(share.dir, 'service.json'), JSON.stringify({ state, url, at: Math.floor(Date.now() / 1000) - ageSec, detail: '' }));
|
||||
|
||||
it('returns the url once the host reports the service up', async () => {
|
||||
const share = await fresh();
|
||||
await heartbeat();
|
||||
await service(share, 'up', 'https://1.2.3.4/');
|
||||
expect(await share.ensureService(1000)).toEqual({ ok: true, url: 'https://1.2.3.4' });
|
||||
});
|
||||
|
||||
it('explains busy ports right away', async () => {
|
||||
const share = await fresh();
|
||||
await heartbeat();
|
||||
await service(share, 'ports-busy');
|
||||
const r = await share.ensureService(1000);
|
||||
expect(r.ok).toBe(false);
|
||||
expect(!r.ok && r.reason).toMatch(/80\/443/);
|
||||
});
|
||||
|
||||
it('says so when no host dispatcher serves this install', async () => {
|
||||
const share = await fresh();
|
||||
const r = await share.ensureService(1000);
|
||||
expect(!r.ok && r.reason).toMatch(/автообновление/);
|
||||
});
|
||||
|
||||
it('ignores a stale "up" and is off when disabled', async () => {
|
||||
const share = await fresh();
|
||||
await heartbeat();
|
||||
await service(share, 'up', 'https://1.2.3.4', 600);
|
||||
expect((await share.ensureService(10)).ok).toBe(false);
|
||||
expect((await (await fresh({ ...opts, enabled: false })).ensureService(10)).ok).toBe(false);
|
||||
});
|
||||
});
|
||||
87
tests/filesService.test.ts
Normal file
87
tests/filesService.test.ts
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawn, type ChildProcess } from 'node:child_process';
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||
import { existsSync } from 'node:fs';
|
||||
|
||||
// files-service/server.mjs as the host runs it: its own process, bridges' file dirs under a root.
|
||||
let root: string;
|
||||
let proc: ChildProcess;
|
||||
let base: string;
|
||||
const bridge = () => path.join(root, '1');
|
||||
|
||||
async function tokenFile(name: string, body: Record<string, unknown>) {
|
||||
await writeFile(path.join(bridge(), 'tokens', `${name}.json`), JSON.stringify({ v: 1, ...body }));
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
root = await mkdtemp(path.join(os.tmpdir(), 'tlmx-srv-'));
|
||||
await mkdir(path.join(bridge(), 'tokens'), { recursive: true });
|
||||
await mkdir(path.join(bridge(), 'items', 'id1'), { recursive: true });
|
||||
await writeFile(path.join(bridge(), 'items', 'id1', 'отчёт.txt'), '0123456789');
|
||||
const later = Date.now() + 3600_000;
|
||||
await tokenFile('dddddddddddddddddddddd', { kind: 'download', name: 'отчёт.txt', size: 10, path: 'items/id1/отчёт.txt', expiresAt: later });
|
||||
await tokenFile('eeeeeeeeeeeeeeeeeeeeee', { kind: 'download', name: 'x', size: 1, path: 'items/id1/отчёт.txt', expiresAt: Date.now() - 1 });
|
||||
await tokenFile('tttttttttttttttttttttt', { kind: 'download', name: 'x', size: 1, path: '../../../etc/hosts', expiresAt: later });
|
||||
await tokenFile('uuuuuuuuuuuuuuuuuuuuuu', { kind: 'upload', name: 'big.zip', expectedSize: 5, expiresAt: later, reserveBytes: 0 });
|
||||
await tokenFile('rrrrrrrrrrrrrrrrrrrrrr', { kind: 'upload', name: 'big.zip', expectedSize: 5, expiresAt: later, reserveBytes: Number.MAX_SAFE_INTEGER });
|
||||
const port = 18000 + Math.floor(Math.random() * 2000);
|
||||
base = `http://127.0.0.1:${port}`;
|
||||
proc = spawn(process.execPath, [path.join('files-service', 'server.mjs')], { env: { ...process.env, FILES_ROOT: root, PORT: String(port) }, stdio: ['ignore', 'pipe', 'pipe'] });
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const t = setTimeout(() => reject(new Error('server did not start')), 10_000);
|
||||
proc.stdout!.on('data', (d) => String(d).includes('files service') && (clearTimeout(t), resolve()));
|
||||
});
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
proc?.kill();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('files service', () => {
|
||||
it('answers health and hides everything without a token', async () => {
|
||||
expect(await (await fetch(`${base}/health`)).text()).toBe('ok');
|
||||
expect((await fetch(`${base}/`)).status).toBe(404);
|
||||
expect((await fetch(`${base}/f/short`)).status).toBe(404);
|
||||
expect((await fetch(`${base}/f/eeeeeeeeeeeeeeeeeeeeee`)).status).toBe(404); // expired
|
||||
});
|
||||
|
||||
it('shows a page first, then the file with its name and Range support', async () => {
|
||||
const pageRes = await fetch(`${base}/f/dddddddddddddddddddddd`);
|
||||
expect(pageRes.status).toBe(200);
|
||||
expect(pageRes.headers.get('x-robots-tag')).toMatch(/noindex/);
|
||||
expect(await pageRes.text()).toContain('/f/dddddddddddddddddddddd/file');
|
||||
const file = await fetch(`${base}/f/dddddddddddddddddddddd/file`);
|
||||
expect(await file.text()).toBe('0123456789');
|
||||
expect(file.headers.get('content-disposition')).toContain(`filename*=UTF-8''${encodeURIComponent('отчёт.txt')}`);
|
||||
const part = await fetch(`${base}/f/dddddddddddddddddddddd/file`, { headers: { Range: 'bytes=4-6' } });
|
||||
expect(part.status).toBe(206);
|
||||
expect(await part.text()).toBe('456');
|
||||
expect(part.headers.get('content-range')).toBe('bytes 4-6/10');
|
||||
});
|
||||
|
||||
it('never serves a path outside the bridge directory', async () => {
|
||||
expect((await fetch(`${base}/f/tttttttttttttttttttttt/file`)).status).toBe(404);
|
||||
});
|
||||
|
||||
it('takes one upload per link and refuses a second', async () => {
|
||||
expect(await (await fetch(`${base}/f/uuuuuuuuuuuuuuuuuuuuuu`)).text()).toContain('type="file"');
|
||||
const put = (body: string) =>
|
||||
fetch(`${base}/f/uuuuuuuuuuuuuuuuuuuuuu/upload`, { method: 'PUT', body, headers: { 'X-File-Name': encodeURIComponent('../архив.zip') } });
|
||||
const first = await put('hello');
|
||||
expect(first.status).toBe(200);
|
||||
const dir = path.join(bridge(), 'incoming', 'uuuuuuuuuuuuuuuuuuuuuu');
|
||||
const done = JSON.parse(await readFile(path.join(dir, 'done.json'), 'utf8'));
|
||||
expect(done).toMatchObject({ name: '_архив.zip', size: 5 });
|
||||
expect(await readFile(path.join(dir, done.file), 'utf8')).toBe('hello');
|
||||
expect((await put('again')).status).toBe(410);
|
||||
expect(existsSync(path.join(dir, '.lock'))).toBe(false);
|
||||
});
|
||||
|
||||
it('refuses an upload that would eat the reserve', async () => {
|
||||
const r = await fetch(`${base}/f/rrrrrrrrrrrrrrrrrrrrrr/upload`, { method: 'PUT', body: 'hello' });
|
||||
expect(r.status).toBe(507);
|
||||
});
|
||||
});
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { Telegraf } from 'telegraf';
|
||||
import { KEYBOARD_CONTACT, KEYBOARD_GEO, KEYBOARD_NA, KEYBOARD_PRESS, maxKeyboardToTelegram, pressableButton, takeKeyboard } from '../src/bridge/attachments.js';
|
||||
import { KEYBOARD_CONTACT, KEYBOARD_GEO, KEYBOARD_NA, KEYBOARD_PRESS, maxKeyboardToTelegram, miniAppLink, pressableButton, takeKeyboard } from '../src/bridge/attachments.js';
|
||||
import { isMarkupRefusal, isRenderableAttach, sendAttachments, sendTextPieces } from '../src/bridge/sync.js';
|
||||
|
||||
const kb = (buttons: unknown) => ({ _type: 'INLINE_KEYBOARD', keyboard: { buttons }, callbackId: 'x' });
|
||||
|
|
@ -29,7 +29,7 @@ describe('maxKeyboardToTelegram', () => {
|
|||
[{ type: 'LINK', text: 'Сайт', url: 'tg://resolve?domain=x' }],
|
||||
]);
|
||||
expect(maxKeyboardToTelegram(attach)).toEqual({
|
||||
inline_keyboard: [[na('Да'), na('Нет')], [na('Открыть')], [{ text: 'Контакт', callback_data: KEYBOARD_CONTACT }, { text: 'Гео', callback_data: KEYBOARD_GEO }, na('?'), na('без типа')], [{ text: 'Сайт', url: 'tg://resolve?domain=x' }]],
|
||||
inline_keyboard: [[na('Да'), na('Нет')], [{ text: 'Открыть', url: 'https://max.ru/app?startapp' }], [{ text: 'Контакт', callback_data: KEYBOARD_CONTACT }, { text: 'Гео', callback_data: KEYBOARD_GEO }, na('?'), na('без типа')], [{ text: 'Сайт', url: 'tg://resolve?domain=x' }]],
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -154,3 +154,18 @@ describe('CALLBACK buttons — presses relayed to MAX', () => {
|
|||
expect(pressableButton('no-such-key')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('miniAppLink', () => {
|
||||
it('builds the public deep link of the live «Настройки» sample (2026-10-06)', () => {
|
||||
const b = { type: 'OPEN_APP', text: 'Настройки', webApp: 'maxnotifications_bot', contactId: 18948480, payload: 'v2bdo2qespQHrvVOL0TsKEKUXUk2RsTqvbcEFsHcdRp63ILky2my5' };
|
||||
expect(miniAppLink(b)).toBe('https://max.ru/maxnotifications_bot?startapp=v2bdo2qespQHrvVOL0TsKEKUXUk2RsTqvbcEFsHcdRp63ILky2my5');
|
||||
});
|
||||
|
||||
it('drops a payload MAX would strip anyway, and refuses a bad app name', () => {
|
||||
expect(miniAppLink({ webApp: 'shop_bot', payload: 'a b' })).toBe('https://max.ru/shop_bot?startapp');
|
||||
expect(miniAppLink({ webApp: 'shop_bot', payload: 'x'.repeat(513) })).toBe('https://max.ru/shop_bot?startapp');
|
||||
expect(miniAppLink({ webApp: 'shop_bot' })).toBe('https://max.ru/shop_bot?startapp');
|
||||
expect(miniAppLink({ webApp: '../evil' })).toBeUndefined();
|
||||
expect(miniAppLink({ payload: 'x' })).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -16,6 +16,12 @@ cd "$(dirname "$0")"
|
|||
mkdir -p data
|
||||
date -u +%Y-%m-%dT%H:%M:%SZ > data/watcher-heartbeat 2>/dev/null || true
|
||||
|
||||
# The shared big-file service (files-service/): started only while some bridge has a live link,
|
||||
# stopped when none is left. Never allowed to break or delay-fail the update check below.
|
||||
if [ -f files-service/reconcile.sh ]; then
|
||||
timeout 120 bash files-service/reconcile.sh >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
MARKER="data/update-requested"
|
||||
[ -f "$MARKER" ] || exit 0
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue