mirror of
https://github.com/Runnin4ik/dpi-detector.git
synced 2026-10-19 17:33:18 +00:00
feat(classify,cli): name TLS-stack failures, fix the site-family redirect rule, trace test 6
The detail vocabulary had one hole: a rustls failure that is not a peer alert
reached the table as its own English sentence (`connection error | cannot decrypt
peer's message`) — the only `Detail` whose code is prose, so `--json` carried
that sentence and no test could pin it. `Detail::StackFailure(StackKind)` names
the six rustls failures a middlebox leaves behind (decrypt error, invalid /
inappropriate / inappropriate-handshake message, peer misbehaved, no application
protocol) through `classify/stack.rs`, the table that mirrors `alert.rs`. The
status stays `Unknown` on purpose: a decrypt error fits an injected record and a
buggy server alike, and the classifier still refuses to read "our stack lacks
this feature" as a peer failure.
Redirect details became values as well (`Redirect { host }`, `UpgradeHttps
{ status }`), which retires the last two `Other` sites that built display prose
inside the core, and the same-site rule is symmetric now: `m.holod.media` →
`holod.media` was a red REDIR while `www.holod.media` → `holod.media` was OK, so
a subdomain naming its main domain read as a redirection away from the site.
Test 6 gained `--trace [<PATH>]`: one line per attempt — UTC, round, shape, host,
attempt, status, detail code, milliseconds — to stderr or to a file. The file
form is what works under the TUI, and it is where a "whole column is red"
question gets answered. `BurstObserver` grew a defaulted `host_probed`, so the
live line and the trace are two observers of one run and neither knows about the
other; an observer that wants neither implements nothing.
README covers all three: the flag, the symmetric redirect rule, the grouped
detail column, and the HTTP/2 rows the fingerprint profiles no longer deviate on.
This commit is contained in:
Kirill Minovsky
parent
d1f97abb51
commit
2f2c76f088
16 changed files with 591 additions and 79 deletions
20
README.md
20
README.md
|
|
@ -109,6 +109,7 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install
|
|||
--burst-profiles <LIST> Fingerprint для теста 6: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [по умолчанию: all]
|
||||
--burst-tls <VERSION> Версия TLS для теста 6: 1.2|1.3 [по умолчанию: 1.3]
|
||||
--burst-alpn <PROTOCOL> ALPN для теста 6: h2 (предлагает h2 с откатом на http/1.1)|http/1.1 (только http/1.1) [по умолчанию: h2]
|
||||
--trace [<PATH>] Тест 6: трасса по строке на попытку (раунд, отпечаток, домен, статус, деталь, мс); без пути — в stderr, с путём — в указанный файл [по умолчанию: выключено]
|
||||
-h, --help Показать справку
|
||||
-V, --version Показать версию
|
||||
```
|
||||
|
|
@ -118,7 +119,7 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install
|
|||
### Локализация (`crates/dpi-core/src/i18n`)
|
||||
|
||||
* `Messages` - по одному полю на строку, четыре блока (`En`, `Ru`, `Zh`, `Fa`); шаблоны используют `{}`.
|
||||
* `legend_sections*()` - таблицы `--legend`; `details.rs` - текст для деталей `DET_*` (в русском блоке строка выводится как есть).
|
||||
* `legend_sections*()` - таблицы `--legend`; `details.rs` - текст для каждой детали `Detail` (в русском блоке строка выводится как есть).
|
||||
* Язык Farsi рендерится **Finglish** (латиница, без персидской графики и диакритики).
|
||||
* Статусные бейджи и имена протоколов (`OK`, `BLOCKED`, `TLS RST`, `SNI`, `ClientHello`) остаются латиницей в любом языке — так цифры, IP-адреса и строки логов читаются одинаково во всех локалях.
|
||||
|
||||
|
|
@ -136,14 +137,23 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install
|
|||
|
||||
При выборе теста `6` открывается свой экран настроек: число одновременных рукопожатий, таймаут одного рукопожатия **в секундах**, **версия TLS** (`1.2`/`1.3`), **протокол HTTP в ALPN** (`h2` — как у браузеров, с откатом на `http/1.1`, или `http/1.1` — только он), домен (в поле показан хост, который проверял прошлый прогон; первый набранный символ заменяет его, а пустое поле — весь список, как в тесте `2`; вставленный URL приводится к имени хоста тем же очистителем, что и список из конфига) и **Fingerprint** (по умолчанию все). Поле домена подсвечивается: серый фон — строка выбрана, синий — в поле идёт ввод. `Q`/`Esc` отменяет тест.
|
||||
|
||||
На каждый домен и отпечаток выполняется N одновременных попыток — отдельные TCP-соединения, свежие сессии без resumption. Попытка не заканчивается рукопожатием: сразу после него тем же профилем (и тем же `User-Agent`/заголовками) уходит `GET /` по тому протоколу, который выбрал ALPN, и ответ классифицируется так же, как в тесте 2. Рукопожатие ограничено таймаутом из настроек, ответ ждёт `read_timeout` из `config.yml`; попытка считается удачной, только если вернулся ответ — обрыв на запросе, редирект на чужой хост, `HTTP BLOCK` или таймаут чтения видны как своя строка, а не как «ответили». Две оси дают 4 комбинации, и они измеряют разные вещи: `1.3` против `1.2` — реакцию на версию протокола, `h2` против `http/1.1` — реакцию на ALPN (некоторые триггеры срабатывают именно на h2-хелло; JA4 при этом меняет поле ALPN, `h2` → `h1`).
|
||||
На каждый домен и отпечаток выполняется N одновременных попыток — отдельные TCP-соединения, свежие сессии без resumption. Попытка не заканчивается рукопожатием: сразу после него тем же профилем (и тем же `User-Agent`/заголовками) уходит `GET /` по тому протоколу, который выбрал ALPN, и ответ классифицируется так же, как в тесте 2. Рукопожатие ограничено таймаутом из настроек, ответ ждёт `read_timeout` из `config.yml`; попытка считается удачной, только если вернулся ответ — обрыв на запросе, редирект на чужой хост, `HTTP BLOCK` или таймаут чтения видны как своя строка, а не как «ответили». Редирект внутри своего сайта вердикта не меняет: поддомен и основной домен — это одна и та же площадка, поэтому `holod.media` → `www.holod.media` и `m.holod.media` → `holod.media` читаются как `OK`, а на чужой домен (`www.instagram.com` → `www.facebook.com`) — как `REDIR` с именем хоста в детали. Две оси дают 4 комбинации, и они измеряют разные вещи: `1.3` против `1.2` — реакцию на версию протокола, `h2` против `http/1.1` — реакцию на ALPN (некоторые триггеры срабатывают именно на h2-хелло; JA4 при этом меняет поле ALPN, `h2` → `h1`).
|
||||
|
||||
Ось `1.3` шлёт **браузерный** оффер — 1.2 и 1.3 вместе, как Chrome, Safari и Firefox, — потому что тест спрашивает, блокируется ли *форма браузера*, а хелло, пришпиленный к 1.3, не шлёт ни один браузер. Ответ требуется именно 1.3: если пир выбрал 1.2, строка получает `NO TLS 1.3`. Ось `1.2` — намеренно синтетический клиент, предлагающий только 1.2: хелло с обоими протоколами никогда не получает ответ 1.2, и иначе проверить выживание 1.2-рукопожатия нельзя. Тот же компромисс, что у двух колонок теста 2. В таблице `M/N` по каждому отпечатку (зелёный — все попытки прошли целиком (рукопожатие и запрос), жёлтый — часть, красный — никто), в колонке «Детали» — самый частый вердикт отказа (`TLS RST`, `TLS DROP`, `SYN DROP`, …). Тест идёт последним: серия рукопожатий может «сжечь» цель на время, и измерения после неё недействительны.
|
||||
Ось `1.3` шлёт **браузерный** оффер — 1.2 и 1.3 вместе, как Chrome, Safari и Firefox, — потому что тест спрашивает, блокируется ли *форма браузера*, а хелло, пришпиленный к 1.3, не шлёт ни один браузер. Ответ требуется именно 1.3: если пир выбрал 1.2, строка получает `NO TLS 1.3`. Ось `1.2` — намеренно синтетический клиент, предлагающий только 1.2: хелло с обоими протоколами никогда не получает ответ 1.2, и иначе проверить выживание 1.2-рукопожатия нельзя. Тот же компромисс, что у двух колонок теста 2. В таблице `M/N` по каждому отпечатку (зелёный — все попытки прошли целиком (рукопожатие и запрос), жёлтый — часть, красный — никто), в колонке «Детали» — **все** вердикты строки, сгруппированные по статусу: частые первыми, при равенстве — в порядке появления (`TLS ALERT ×8 · TLS RST ×4`), так что двенадцать потерянных рукопожатий не схлопываются в одну «победившую» ошибку. Группировка идёт по статусу, а не по детали: разные алерты — одна группа `TLS ALERT`, а конкретная деталь каждой попытки остаётся в `--json` и в трассе. Тест идёт последним: серия рукопожатий может «сжечь» цель на время, и измерения после неё недействительны.
|
||||
|
||||
Во время прогона печатается одна живая строка `Тестируем: CHROME 107 3/4 12/35 · 00:04` — какая форма сейчас в эфире, который это раунд из скольких, сколько хостов раунда готово и сколько идёт время. Отдельной шапки с настройками и блок «Итог» тест не выводит: отпечаток с версией и так виден в колонках таблицы, а пустой итог не рисуется.
|
||||
|
||||
Из неинтерактивного запуска настройки задаются флагами `--burst`, `--burst-timeout`, `--burst-profiles`, `--burst-tls`, `--burst-alpn`; в `--json` результат лежит в ключе `results.fingerprint_burst` (вместе с `tls` и `alpn` этого прогона).
|
||||
|
||||
Флаг `--trace [<PATH>]` пишет по строке на каждую попытку — время UTC, раунд, форма, домен, номер попытки, статус, код детали и миллисекунды:
|
||||
|
||||
```text
|
||||
13:19:30Z r1/1 chrome www.google.com #1 tls_alert alert_unexpected_message 719 ms
|
||||
13:19:30Z r1/1 chrome standby-rezka.tv #1 tls_alert alert_illegal_parameter 1031 ms
|
||||
```
|
||||
|
||||
Без пути трасса идёт в stderr, с путём — в указанный файл, что и делает её пригодной для интерактивного экрана: `--trace C:\temp\trace.log` плюс `Get-Content -Wait C:\temp\trace.log` во втором окне, пока TUI перерисовывает кадр. Строки не переводятся — это машинный журнал: видно, какая именно из N попыток, с какой задержкой и по какой причине не прошла (тишина до нашего таймаута, сброс за десятки миллисекунд или алерт от пира), и `grep` по форме или домену отвечает на вопрос «почему эта колонка красная».
|
||||
|
||||
### Профили отпечатка (`--fingerprint`)
|
||||
|
||||
Профиль задаёт три слоя сразу, и все три взяты из одного и того же набора `curl-impersonate v2.2.2`, поэтому клиент, выглядящий как `curl_chrome107` в одном слое, выглядит так же и в остальных:
|
||||
|
|
@ -152,11 +162,11 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install
|
|||
| --- | --- | --- |
|
||||
| TLS | список шифров и их порядок, группы, `signature_algorithms`, ALPN, порядок расширений, GREASE, ALPS, padding до 512, сжатие сертификата, а у Safari ещё и фолбэки `0x0302`/`0x0301` в `supported_versions`; тесты пинуют JA3 и JA4 против бандла | ECH (синтез ломает рукопожатие с Google и Cloudflare, поэтому у Firefox форма на одно расширение и на 357 байт короче оригинала); там, где версия пришпилена ради изоляции (две колонки теста 2 и ось `1.2` теста 6), в `supported_versions` ровно одна версия — JA3/JA4 этого не видят, читающий тело middlebox видит |
|
||||
| HTTP | `User-Agent` и набор заголовков клиента в его порядке (`sec-ch-ua*`, `accept`, `sec-fetch-*`, `accept-language`; у Firefox ещё `priority` и `te`) | `accept-encoding` всегда `identity`: тесты 2–4 считают байты до обрыва, а согласованное сжатие сделало бы эти числа зависящими от сжимаемости ответа |
|
||||
| HTTP/2 | значения `SETTINGS` и то, какие из них отправляются, плюс оконный `WINDOW_UPDATE` | порядок псевдозаголовков (`m,s,a,p` — порядок hyper; Chrome шлёт `m,a,s,p`, Firefox `m,p,a,s`, Safari `m,s,p,a`) и приоритет на `HEADERS` (h2 0.4 не умеет приоритеты) |
|
||||
| HTTP/2 | значения и **состав** `SETTINGS`, **порядок** `SETTINGS` (у Safari `4:4194304` идёт перед `3:100`), оконный `WINDOW_UPDATE`, порядок псевдозаголовков (`m,a,s,p` — Chrome, `m,p,a,s` — Firefox, `m,s,p,a` — Safari) и приоритет на `HEADERS` с весом и зависимостью (Chrome 256, Firefox 42, Safari 255) | — |
|
||||
|
||||
Свой `user_agent` в `config.yml` перебивает профиль: если поле отличается от встроенного значения по умолчанию, на провод уходит именно оно. `rustls` — контрольная форма: она никого не изображает и оставляет заголовки и преамбулу HTTP/2 такими, какими их отправляли все прошлые измерения.
|
||||
|
||||
Проверка на `tls.peet.ws`: JA3/JA4, `User-Agent`, набор и порядок заголовков, `SETTINGS` и `WINDOW_UPDATE` совпадают с `curl_chrome107` в точности; `peetprint` (он сворачивает в один хэш ещё и порядок псевдозаголовков с приоритетом) — нет.
|
||||
Проверка на `tls.peet.ws`: JA3 и его хэш, JA4/JA4_r, `peetprint` с хэшем, `User-Agent`, набор и порядок заголовков, `SETTINGS` (состав, значения и порядок), `WINDOW_UPDATE`, порядок псевдозаголовков, приоритет на `HEADERS` и akamai-отпечаток совпадают с бандлами в точности — у `curl_chrome107`, `curl_firefox133` и `curl_safari155` соответственно. Расхождения, которые остаются во всех трёх профилях, — ECH у Firefox и `accept-encoding: identity` (см. таблицу выше).
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
use std::io;
|
||||
use super::alert;
|
||||
use super::detail::{AlertKind, Detail};
|
||||
use super::stack;
|
||||
use super::types::{ConnectionStage, DpiStatus};
|
||||
|
||||
fn short_detail(msg: &str) -> String {
|
||||
|
|
@ -16,6 +17,39 @@ fn is_tls_stage(stage: ConnectionStage) -> bool {
|
|||
)
|
||||
}
|
||||
|
||||
/// Messages about a protocol element our own TLS stack does not implement are
|
||||
/// not evidence about the network. rustls words these with "certificate" (`got
|
||||
/// CompressedCertificate when expecting Certificate`,
|
||||
/// `UnknownCertificateExtension`), so without this they read as a certificate
|
||||
/// interception — and, since rustls reports them the same way it reports a peer
|
||||
/// that broke the protocol, they would also take a stack failure's name. Real
|
||||
/// interference still lands in the alert, record and EOF branches.
|
||||
fn is_unimplemented_element(msg_lower: &str) -> bool {
|
||||
[
|
||||
"unsupportedextension",
|
||||
"unsupported extension",
|
||||
"unknowncertificateextension",
|
||||
"unknown certificate extension",
|
||||
"compressedcertificate",
|
||||
"compressed certificate",
|
||||
]
|
||||
.iter()
|
||||
.any(|m| msg_lower.contains(m))
|
||||
}
|
||||
|
||||
/// The detail for a message nothing else classified: a named TLS-stack failure
|
||||
/// carries the rustls name it was classified by, anything else keeps its own
|
||||
/// words.
|
||||
fn unclassified_detail(msg_lower: &str, raw: &str) -> Detail {
|
||||
if is_unimplemented_element(msg_lower) {
|
||||
return Detail::Other(short_detail(raw));
|
||||
}
|
||||
match stack::from_message(msg_lower) {
|
||||
Some(kind) => stack::detail_for(kind),
|
||||
None => Detail::Other(short_detail(raw)),
|
||||
}
|
||||
}
|
||||
|
||||
/// The verdict for a message that reports a peer alert: the description it names
|
||||
/// decides the detail.
|
||||
///
|
||||
|
|
@ -79,22 +113,10 @@ pub fn classify_ssl_error(
|
|||
return (DpiStatus::TlsAlert, Detail::FakeTlsAlert);
|
||||
}
|
||||
// A message about a protocol element our own TLS stack does not implement is
|
||||
// not evidence about the network. rustls words these with "certificate"
|
||||
// (`got CompressedCertificate when expecting Certificate`,
|
||||
// `UnknownCertificateExtension`), so without this they are reported as an
|
||||
// interception. Real interference still lands in the branches above: a
|
||||
// censor's spoofed ServerHello arrives as a version/record/alert error.
|
||||
if [
|
||||
"unsupportedextension",
|
||||
"unsupported extension",
|
||||
"unknowncertificateextension",
|
||||
"unknown certificate extension",
|
||||
"compressedcertificate",
|
||||
"compressed certificate",
|
||||
]
|
||||
.iter()
|
||||
.any(|m| msg.contains(m))
|
||||
{
|
||||
// not evidence about the network (see `is_unimplemented_element`). Real
|
||||
// interference still lands in the branches above: a censor's spoofed
|
||||
// ServerHello arrives as a version/record/alert error.
|
||||
if is_unimplemented_element(&msg) {
|
||||
return (DpiStatus::Unknown, Detail::Other(short_detail(err_msg)));
|
||||
}
|
||||
|
||||
|
|
@ -141,7 +163,7 @@ pub fn classify_ssl_error(
|
|||
return (DpiStatus::NoTls13, Detail::NoTls13);
|
||||
}
|
||||
|
||||
(DpiStatus::Unknown, Detail::Other(short_detail(err_msg)))
|
||||
(DpiStatus::Unknown, unclassified_detail(&msg, err_msg))
|
||||
}
|
||||
|
||||
fn dns_failure_text(msg: &str) -> bool {
|
||||
|
|
@ -271,7 +293,7 @@ pub fn classify_connect_error_full(
|
|||
return (DpiStatus::OsErr, Detail::Other(format!("OS errno {}", code)));
|
||||
}
|
||||
|
||||
(DpiStatus::Unknown, Detail::Other(short_detail(err_msg)))
|
||||
(DpiStatus::Unknown, unclassified_detail(&full, err_msg))
|
||||
}
|
||||
|
||||
/// Legacy io::Error-based entry point (stage unknown → tcp_connect).
|
||||
|
|
@ -331,27 +353,31 @@ pub fn classify_tls_error(
|
|||
|
||||
// A reset/EOF after ClientHello with zero bytes back is a DPI SNI RST,
|
||||
// even when the OS masks it as a generic error.
|
||||
if status == DpiStatus::Unknown
|
||||
&& (stage == ConnectionStage::TlsClientHelloSent || (bytes_sent > 0 && bytes_recv == 0))
|
||||
{
|
||||
if status == DpiStatus::Unknown {
|
||||
let lower = err_msg.to_ascii_lowercase();
|
||||
if lower.contains("reset")
|
||||
|| lower.contains("eof")
|
||||
|| lower.contains("closed")
|
||||
|| lower.contains("abort")
|
||||
|| lower.contains("broken pipe")
|
||||
|| lower.contains("10054")
|
||||
|| lower.contains(" 104")
|
||||
if (stage == ConnectionStage::TlsClientHelloSent || (bytes_sent > 0 && bytes_recv == 0))
|
||||
&& (lower.contains("reset")
|
||||
|| lower.contains("eof")
|
||||
|| lower.contains("closed")
|
||||
|| lower.contains("abort")
|
||||
|| lower.contains("broken pipe")
|
||||
|| lower.contains("10054")
|
||||
|| lower.contains(" 104"))
|
||||
{
|
||||
return (
|
||||
DpiStatus::TlsRst,
|
||||
Detail::TlsRstHello,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if status == DpiStatus::Unknown {
|
||||
return (DpiStatus::Unknown, Detail::Other(format!("TLS error: {}", short_detail(err_msg))));
|
||||
// The stack's own wording survives as a detail; anything else keeps its
|
||||
// own words under the TLS label.
|
||||
return match stack::from_message(&lower) {
|
||||
Some(kind) if !is_unimplemented_element(&lower) => {
|
||||
(DpiStatus::Unknown, stack::detail_for(kind))
|
||||
}
|
||||
_ => (DpiStatus::Unknown, Detail::Other(format!("TLS error: {}", short_detail(err_msg)))),
|
||||
};
|
||||
}
|
||||
(status, detail)
|
||||
}
|
||||
|
|
@ -370,7 +396,7 @@ pub fn classify_read_error(
|
|||
let (status, detail) =
|
||||
classify_connect_error_full(err_msg, raw_os_error, kind, bytes_read, "reading_data");
|
||||
if status == DpiStatus::Unknown {
|
||||
return (DpiStatus::Unknown, Detail::Other(short_detail(err_msg)));
|
||||
return (DpiStatus::Unknown, unclassified_detail(&err_msg.to_ascii_lowercase(), err_msg));
|
||||
}
|
||||
(status, detail)
|
||||
}
|
||||
|
|
@ -505,11 +531,48 @@ mod tests {
|
|||
for msg in [
|
||||
"received unexpected handshake message: got CompressedCertificate when expecting Certificate or CertificateRequest",
|
||||
"received corrupt message of type UnknownCertificateExtension",
|
||||
"peer misbehaved: UnsolicitedServerHelloExtension",
|
||||
] {
|
||||
let (s, d) = classify_ssl_error(msg, 0, ConnectionStage::TlsClientHelloSent);
|
||||
assert_eq!(s, DpiStatus::Unknown, "{msg} -> {}", d.code());
|
||||
// Our stack lacking a feature is not a peer failure: the message
|
||||
// keeps its own words instead of taking a stack failure's name.
|
||||
assert!(matches!(d, Detail::Other(_)), "{msg} -> {}", d.code());
|
||||
}
|
||||
|
||||
// A peer that broke the protocol is a stack failure the reader can name,
|
||||
// but still not evidence of interception: the status stays Unknown.
|
||||
let (s, d) = classify_ssl_error(
|
||||
"peer misbehaved: UnsolicitedServerHelloExtension",
|
||||
0,
|
||||
ConnectionStage::TlsClientHelloSent,
|
||||
);
|
||||
assert_eq!((s, d.code().as_ref()), (DpiStatus::Unknown, "tls_peer_misbehaved"));
|
||||
}
|
||||
|
||||
/// A record that will not decrypt is the signature of an injected or
|
||||
/// rewritten record, and it used to reach the table as the rustls sentence
|
||||
/// in English. It now carries the stack failure's name; the status stays
|
||||
/// Unknown because a buggy server produces the same message.
|
||||
#[test]
|
||||
fn test_a_stack_failure_is_named_not_quoted() {
|
||||
let observed = "connection error | cannot decrypt peer's message";
|
||||
let (s, d) = classify_read_error(observed, None, None, 0);
|
||||
assert_eq!((s, d.code().as_ref()), (DpiStatus::Unknown, "tls_decrypt_error"));
|
||||
|
||||
// The same wording through the TLS path keeps the same detail.
|
||||
let (s, d) = classify_tls_error(
|
||||
ConnectionStage::TlsClientHelloSent,
|
||||
100,
|
||||
0,
|
||||
observed,
|
||||
false,
|
||||
);
|
||||
assert_eq!((s, d.code().as_ref()), (DpiStatus::Unknown, "tls_decrypt_error"));
|
||||
|
||||
// A message naming nothing known keeps its own words.
|
||||
let (s, d) = classify_read_error("connection error | oops", None, None, 0);
|
||||
assert_eq!(s, DpiStatus::Unknown);
|
||||
assert_eq!(d.code().as_ref(), "connection error | oops");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -49,6 +49,10 @@ pub enum Detail {
|
|||
/// A peer alert that named its description: the code is
|
||||
/// `alert_<description>` (`alert_illegal_parameter`).
|
||||
Alert(AlertKind),
|
||||
/// A TLS-stack failure that is not an alert from the peer: our own TLS layer
|
||||
/// refused the bytes it received — `cannot decrypt peer's message`, `peer
|
||||
/// misbehaved: …`. The code is `tls_<kind>`.
|
||||
StackFailure(StackKind),
|
||||
|
||||
// ─── Certificates ───
|
||||
NoRootCa,
|
||||
|
|
@ -107,6 +111,12 @@ pub enum Detail {
|
|||
IspBlockpage { arrow: bool, ip: String },
|
||||
/// The resolver answered with a loopback/private address (`-> <ip>`).
|
||||
LocalIp { ip: String },
|
||||
/// The peer redirected to `host`: on the same site family the verdict stays
|
||||
/// OK, a foreign one is a red `REDIR`.
|
||||
Redirect { host: String },
|
||||
/// A hop to `https` on the same host: `-> https`, or `301 -> https` when the
|
||||
/// status that carried it is known.
|
||||
UpgradeHttps { status: Option<u16> },
|
||||
/// An HTTP status line.
|
||||
HttpStatus(u16),
|
||||
/// How long a successful check took.
|
||||
|
|
@ -211,6 +221,54 @@ impl AlertKind {
|
|||
}
|
||||
}
|
||||
|
||||
/// One failure of our own TLS stack over bytes the peer sent, named as rustls
|
||||
/// names it.
|
||||
///
|
||||
/// These are the messages a middlebox leaves behind when it rewrites or injects
|
||||
/// records: a record that does not decrypt, a message that arrived out of order,
|
||||
/// a peer that broke the protocol. rustls words each one differently and the
|
||||
/// difference is the diagnosis, so they are values, not free text — the free
|
||||
/// text would be English prose inside a Russian table, and it changes with the
|
||||
/// rustls version, which no test can pin.
|
||||
///
|
||||
/// The names are rustls's own [`Error`] variants, snake_case (see
|
||||
/// `vendor/rustls/src/error.rs`). They are protocol tokens, so they stay Latin
|
||||
/// in every language (Rule 4) and `--json` carries them inside the detail code
|
||||
/// after `tls_`.
|
||||
///
|
||||
/// [`Error`]: https://docs.rs/rustls/latest/rustls/enum.Error.html
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum StackKind {
|
||||
/// `cannot decrypt peer's message`: a record we could not open.
|
||||
DecryptError,
|
||||
/// `received corrupt message of type …`.
|
||||
InvalidMessage,
|
||||
/// `received unexpected message: got … when expecting …`.
|
||||
InappropriateMessage,
|
||||
/// `received unexpected handshake message: got … when expecting …`.
|
||||
InappropriateHandshakeMessage,
|
||||
/// `peer misbehaved: …`.
|
||||
PeerMisbehaved,
|
||||
/// `peer doesn't support any known protocol`: no ALPN in common.
|
||||
NoApplicationProtocol,
|
||||
}
|
||||
|
||||
impl StackKind {
|
||||
/// The rustls name, snake_case: the token a [`Detail::StackFailure`]'s code
|
||||
/// carries after `tls_`.
|
||||
pub fn code(self) -> &'static str {
|
||||
use StackKind::*;
|
||||
match self {
|
||||
DecryptError => "decrypt_error",
|
||||
InvalidMessage => "invalid_message",
|
||||
InappropriateMessage => "inappropriate_message",
|
||||
InappropriateHandshakeMessage => "inappropriate_handshake_message",
|
||||
PeerMisbehaved => "peer_misbehaved",
|
||||
NoApplicationProtocol => "no_application_protocol",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Detail {
|
||||
/// The machine token. `--json` carries exactly this; it never changes with
|
||||
/// `--lang`.
|
||||
|
|
@ -231,6 +289,7 @@ impl Detail {
|
|||
ProtocolVersionAlert => Cow::Borrowed("protocol_version_alert"),
|
||||
FakeTlsAlert => Cow::Borrowed("fake_tls_alert"),
|
||||
Alert(kind) => Cow::Owned(format!("alert_{}", kind.code())),
|
||||
StackFailure(kind) => Cow::Owned(format!("tls_{}", kind.code())),
|
||||
NoRootCa => Cow::Borrowed("no_root_certificates"),
|
||||
CertExpired => Cow::Borrowed("cert_expired"),
|
||||
SelfSigned => Cow::Borrowed("self_signed_cert"),
|
||||
|
|
@ -266,6 +325,8 @@ impl Detail {
|
|||
TimeoutStage { stage } => Cow::Owned(format!("timeout_{}", stage)),
|
||||
IspBlockpage { .. } => Cow::Borrowed("isp_blockpage"),
|
||||
LocalIp { .. } => Cow::Borrowed("local_ip"),
|
||||
Redirect { .. } => Cow::Borrowed("redirect_to_host"),
|
||||
UpgradeHttps { .. } => Cow::Borrowed("upgrade_https"),
|
||||
HttpStatus(code) => Cow::Owned(format!("http_{}", code)),
|
||||
Elapsed(secs) => Cow::Owned(format!("elapsed_{}ms", (secs * 1000.0).round() as u64)),
|
||||
Other(text) => Cow::Owned(text.clone()),
|
||||
|
|
@ -339,6 +400,11 @@ mod tests {
|
|||
Detail::Elapsed(0.3),
|
||||
Detail::Alert(AlertKind::IllegalParameter),
|
||||
Detail::Alert(AlertKind::BadCertificateStatusResponse),
|
||||
Detail::StackFailure(StackKind::DecryptError),
|
||||
Detail::StackFailure(StackKind::InappropriateHandshakeMessage),
|
||||
Detail::Redirect { host: "example.com".into() },
|
||||
Detail::UpgradeHttps { status: Some(301) },
|
||||
Detail::UpgradeHttps { status: None },
|
||||
];
|
||||
for d in &all {
|
||||
let code = d.code();
|
||||
|
|
@ -348,6 +414,13 @@ mod tests {
|
|||
);
|
||||
}
|
||||
assert_eq!(Detail::HttpStatus(403).code(), "http_403");
|
||||
assert_eq!(Detail::StackFailure(StackKind::DecryptError).code(), "tls_decrypt_error");
|
||||
assert_eq!(
|
||||
Detail::StackFailure(StackKind::InappropriateHandshakeMessage).code(),
|
||||
"tls_inappropriate_handshake_message"
|
||||
);
|
||||
assert_eq!(Detail::Redirect { host: "example.com".into() }.code(), "redirect_to_host");
|
||||
assert_eq!(Detail::UpgradeHttps { status: Some(301) }.code(), "upgrade_https");
|
||||
assert_eq!(Detail::Elapsed(0.3).code(), "elapsed_300ms");
|
||||
assert_eq!(Detail::None.code(), "");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,11 @@
|
|||
pub mod alert;
|
||||
pub mod classifier;
|
||||
pub mod stack;
|
||||
pub mod stream;
|
||||
pub mod detail;
|
||||
pub mod types;
|
||||
|
||||
pub use classifier::{classify_connect_error, classify_connect_error_full, classify_read_error, classify_ssl_error, classify_tls_error};
|
||||
pub use stream::{DpiProbeStream, DpiProbeTracker};
|
||||
pub use detail::{AlertKind, Detail};
|
||||
pub use detail::{AlertKind, Detail, StackKind};
|
||||
pub use types::{ConnectionStage, DpiStatus, ProbeMetrics};
|
||||
|
|
|
|||
111
crates/dpi-core/src/classify/stack.rs
Normal file
111
crates/dpi-core/src/classify/stack.rs
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
//! The TLS-stack table: an error message to the rustls variant it names.
|
||||
//!
|
||||
//! Where `alert.rs` reads what the *peer* said, this reads what our own stack
|
||||
//! said about the peer's bytes. The two never overlap: an alert arrives as
|
||||
//! `received fatal alert: …`, a stack failure as rustls's own wording, and both
|
||||
//! tables match on the same lowercased, separator-stripped message.
|
||||
//!
|
||||
//! Matching happens on the words with their separators removed, so
|
||||
//! `peer misbehaved: …`, `PeerMisbehaved` and `peermisbehaved` reach the same
|
||||
//! value. The needles are the `Display` texts of `rustls::Error` as the vendored
|
||||
//! rustls spells them (`vendor/rustls/src/error.rs`); a rustls upgrade that
|
||||
//! rewords one is a change to this table, and the tests below pin the wording so
|
||||
//! that upgrade fails loudly instead of silently unclassifying the case.
|
||||
//!
|
||||
//! A message the table does not know is left to the caller, which keeps its own
|
||||
//! words (`Detail::Other`) — the long tail of OS and library text stays as it
|
||||
//! came.
|
||||
//!
|
||||
//! Ordering matters at the call sites: the classifier's "our stack does not
|
||||
//! implement this" early-out runs first, so a server using certificate
|
||||
//! compression is not read as a peer that misbehaved.
|
||||
|
||||
use super::detail::{Detail, StackKind};
|
||||
|
||||
use StackKind::*;
|
||||
|
||||
/// `(needle without separators, kind)`, matched in order against a lowercased,
|
||||
/// separator-stripped message.
|
||||
///
|
||||
/// The longer wording comes first where one needle is a prefix of another
|
||||
/// (`receivedunexpectedhandshakemessage` before `receivedunexpectedmessage`).
|
||||
const TABLE: &[(&str, StackKind)] = &[
|
||||
("receivedunexpectedhandshakemessage", InappropriateHandshakeMessage),
|
||||
("receivedunexpectedmessage", InappropriateMessage),
|
||||
("receivedcorruptmessageoftype", InvalidMessage),
|
||||
("cannotdecryptpeer'smessage", DecryptError),
|
||||
("peermisbehaved", PeerMisbehaved),
|
||||
("doesn'tsupportanyknownprotocol", NoApplicationProtocol),
|
||||
];
|
||||
|
||||
/// The rustls failure a message names, or `None` when it names none.
|
||||
///
|
||||
/// `msg_lower` is the message already lowercased by the caller — the classifier
|
||||
/// keeps one lowercased copy of every error it classifies.
|
||||
pub fn from_message(msg_lower: &str) -> Option<StackKind> {
|
||||
let flat: String = msg_lower
|
||||
.chars()
|
||||
.filter(|c| !c.is_whitespace() && *c != '_' && *c != '-')
|
||||
.collect();
|
||||
TABLE.iter().find(|(needle, _)| flat.contains(needle)).map(|(_, kind)| *kind)
|
||||
}
|
||||
|
||||
/// The detail a message reports as.
|
||||
pub fn detail_for(kind: StackKind) -> Detail {
|
||||
Detail::StackFailure(kind)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The wording rustls puts on the wire, as the probes see it: the stack's
|
||||
/// own text, appended after hyper's `connection error` (see
|
||||
/// `probe::http::hyper_err_info`).
|
||||
#[test]
|
||||
fn the_rustls_spelling_names_the_failure() {
|
||||
for (msg, kind) in [
|
||||
("connection error | cannot decrypt peer's message", DecryptError),
|
||||
("cannot decrypt peer's message", DecryptError),
|
||||
("received corrupt message of type ChangeCipherSpec", InvalidMessage),
|
||||
(
|
||||
"received unexpected message: got Alert when expecting Handshake",
|
||||
InappropriateMessage,
|
||||
),
|
||||
(
|
||||
"received unexpected handshake message: got ServerHello when expecting ClientHello",
|
||||
InappropriateHandshakeMessage,
|
||||
),
|
||||
("peer misbehaved: UnsolicitedServerHelloExtension", PeerMisbehaved),
|
||||
("peer doesn't support any known protocol", NoApplicationProtocol),
|
||||
] {
|
||||
let lower = msg.to_ascii_lowercase();
|
||||
assert_eq!(from_message(&lower), Some(kind), "{msg}");
|
||||
}
|
||||
}
|
||||
|
||||
/// An alert is not a stack failure: its wording belongs to `alert.rs`, and
|
||||
/// the two tables must not claim the same message.
|
||||
#[test]
|
||||
fn an_alert_wording_is_not_a_stack_failure() {
|
||||
for msg in [
|
||||
"connection error | received fatal alert: UnexpectedMessage",
|
||||
"received fatal alert: IllegalParameter",
|
||||
"peer sent no certificates",
|
||||
"invalid peer certificate: Expired",
|
||||
] {
|
||||
assert_eq!(from_message(&msg.to_ascii_lowercase()), None, "{msg}");
|
||||
}
|
||||
}
|
||||
|
||||
/// A stack failure is still not interception: the status stays unclassified
|
||||
/// while the detail says what rustls objected to.
|
||||
#[test]
|
||||
fn the_detail_is_the_kind() {
|
||||
assert_eq!(detail_for(DecryptError).code(), "tls_decrypt_error");
|
||||
assert_eq!(
|
||||
detail_for(InappropriateHandshakeMessage).code(),
|
||||
"tls_inappropriate_handshake_message"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -327,6 +327,20 @@ pub trait BurstObserver {
|
|||
|
||||
/// One host finished its round, answered or not.
|
||||
fn host_finished(&self) {}
|
||||
|
||||
/// One host's round landed, with every attempt it produced.
|
||||
///
|
||||
/// The live line counts hosts and ignores this; a trace prints the attempts
|
||||
/// the moment they exist, which is what turns "SAFARI 0/4" into a status and
|
||||
/// a millisecond count per attempt. A caller that wants neither implements
|
||||
/// nothing.
|
||||
fn host_probed(
|
||||
&self,
|
||||
_fingerprint: TlsFingerprint,
|
||||
_domain: &str,
|
||||
_report: &BurstProfileReport,
|
||||
) {
|
||||
}
|
||||
}
|
||||
|
||||
impl BurstObserver for () {}
|
||||
|
|
@ -381,6 +395,7 @@ pub async fn burst_targets(
|
|||
while let Some(joined) = rounds.join_next().await {
|
||||
observer.host_finished();
|
||||
if let Ok((index, report)) = joined {
|
||||
observer.host_probed(fingerprint, &targets[index].domain, &report);
|
||||
reports[index].profiles.push(report);
|
||||
}
|
||||
}
|
||||
|
|
@ -875,6 +890,7 @@ mod tests {
|
|||
struct Rounds {
|
||||
started: std::sync::Mutex<Vec<(TlsFingerprint, usize, usize, usize)>>,
|
||||
finished: std::sync::atomic::AtomicUsize,
|
||||
probed: std::sync::Mutex<Vec<String>>,
|
||||
}
|
||||
impl BurstObserver for Rounds {
|
||||
fn round_started(&self, fp: TlsFingerprint, index: usize, total: usize, hosts: usize) {
|
||||
|
|
@ -883,6 +899,12 @@ mod tests {
|
|||
fn host_finished(&self) {
|
||||
self.finished.fetch_add(1, Ordering::SeqCst);
|
||||
}
|
||||
fn host_probed(&self, fp: TlsFingerprint, domain: &str, report: &BurstProfileReport) {
|
||||
self.probed
|
||||
.lock()
|
||||
.expect("lock")
|
||||
.push(format!("{} {domain} x{}", fp.code(), report.attempts.len()));
|
||||
}
|
||||
}
|
||||
let rounds = Rounds::default();
|
||||
let reports = burst_targets(&AppConfig::default(), &targets, &plan, 4, &rounds).await;
|
||||
|
|
@ -904,6 +926,19 @@ mod tests {
|
|||
]
|
||||
);
|
||||
assert_eq!(rounds.finished.load(Ordering::SeqCst), 4, "one tick per host per round");
|
||||
// The trace reads this: every host's round arrives once per shape, with
|
||||
// the attempts it produced — silent hosts are not silently dropped.
|
||||
let mut probed = rounds.probed.lock().expect("lock").clone();
|
||||
probed.sort();
|
||||
assert_eq!(
|
||||
probed,
|
||||
vec![
|
||||
"chrome fast.example x2",
|
||||
"chrome slow.example x2",
|
||||
"rustls fast.example x2",
|
||||
"rustls slow.example x2",
|
||||
]
|
||||
);
|
||||
assert_eq!(reports.len(), 2);
|
||||
for report in &reports {
|
||||
let shapes: Vec<TlsFingerprint> = report.profiles.iter().map(|p| p.fingerprint).collect();
|
||||
|
|
|
|||
|
|
@ -762,21 +762,50 @@ mod tests {
|
|||
fn test_classify_redirect_same_https() {
|
||||
let (s, d) = classify_redirect("example.com", "https://example.com", 301, "https://example.com/", false);
|
||||
assert_eq!(s, DpiStatus::Ok);
|
||||
assert_eq!(d, Detail::Other("→ https".to_string()));
|
||||
assert_eq!(d, Detail::UpgradeHttps { status: None });
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_classify_redirect_foreign() {
|
||||
let (s, d) = classify_redirect("example.com", "https://example.com", 302, "https://evil.com/block", false);
|
||||
assert_eq!(s, DpiStatus::RedirSuspect);
|
||||
assert!(d.code().contains("evil.com"));
|
||||
assert_eq!(d, Detail::Redirect { host: "evil.com".to_string() });
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_classify_redirect_http_phase() {
|
||||
let (s, d) = classify_redirect("example.com", "http://example.com", 301, "https://example.com/", true);
|
||||
assert_eq!(s, DpiStatus::Ok);
|
||||
assert_eq!(d, Detail::Other("301 → https".to_string()));
|
||||
assert_eq!(d, Detail::UpgradeHttps { status: Some(301) });
|
||||
}
|
||||
|
||||
/// The rule the table reads by: a redirect that stays inside one site family
|
||||
/// is a normal `OK`, whatever the direction — a subdomain spelling out the
|
||||
/// main domain (`www.holod.media` → `holod.media`) and the main domain
|
||||
/// naming a subdomain (`holod.media` → `cdn.holod.media`) are both the site
|
||||
/// itself. Only a name outside the family is a red `REDIR`.
|
||||
#[test]
|
||||
fn test_a_subdomain_redirect_inside_the_same_site_is_ok() {
|
||||
let cases: &[(&str, &str, DpiStatus, &str)] = &[
|
||||
// (requested domain, Location, expected status, expected redirect host when foreign)
|
||||
("www.holod.media", "https://holod.media/x", DpiStatus::Ok, ""),
|
||||
("holod.media", "https://www.holod.media/x", DpiStatus::Ok, ""),
|
||||
("holod.media", "https://cdn.holod.media/x", DpiStatus::Ok, ""),
|
||||
("m.holod.media", "https://holod.media/x", DpiStatus::Ok, ""),
|
||||
("www.holod.media", "http://holod.media/x", DpiStatus::Ok, ""),
|
||||
// A different registrable domain is not the same site, however it
|
||||
// reads: this is the instagram → facebook case.
|
||||
("www.instagram.com", "https://www.facebook.com/x", DpiStatus::RedirSuspect, "www.facebook.com"),
|
||||
("holod.media", "https://not-holod.media/x", DpiStatus::RedirSuspect, "not-holod.media"),
|
||||
("holod.media", "https://holod.media.evil.com/x", DpiStatus::RedirSuspect, "holod.media.evil.com"),
|
||||
];
|
||||
for (domain, location, want, foreign) in cases {
|
||||
let (s, d) = classify_redirect(domain, &format!("https://{domain}"), 301, location, false);
|
||||
assert_eq!(s, *want, "{domain} + {location} -> {}", d.code());
|
||||
if *want == DpiStatus::RedirSuspect {
|
||||
assert_eq!(d, Detail::Redirect { host: (*foreign).to_string() }, "{domain} + {location}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Same-host-or-subdomain redirects read as a plain `OK`, a redirect to
|
||||
|
|
@ -787,31 +816,30 @@ mod tests {
|
|||
#[test]
|
||||
fn test_classify_redirect_matrix() {
|
||||
// (base, status, location, expected status, expected detail)
|
||||
// (base, status, location, expected status, expected detail code)
|
||||
let cases: &[(&str, u16, &str, DpiStatus, &str)] = &[
|
||||
("https://holod.media", 301, "https://holod.media/", DpiStatus::Ok, "→ https"),
|
||||
("https://holod.media", 301, "/en/", DpiStatus::Ok, "→ https"),
|
||||
("https://holod.media", 301, "index.html", DpiStatus::Ok, "→ https"),
|
||||
("https://holod.media", 301, "?q=1", DpiStatus::Ok, "→ https"),
|
||||
("https://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, "→ https"),
|
||||
("https://holod.media", 302, "http://holod.media/x", DpiStatus::Ok, "→ holod.media"),
|
||||
("http://holod.media", 301, "https://holod.media/", DpiStatus::Ok, "301 → https"),
|
||||
("http://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, "301 → https"),
|
||||
("http://holod.media", 301, "http://holod.media/x", DpiStatus::Ok, "http_301"),
|
||||
("http://holod.media", 301, "/en/", DpiStatus::Ok, "http_301"),
|
||||
("http://holod.media", 301, "index.html", DpiStatus::Ok, "http_301"),
|
||||
("http://holod.media", 301, "?q=1", DpiStatus::Ok, "http_301"),
|
||||
("https://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, "→ evil.com"),
|
||||
("http://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, "→ evil.com"),
|
||||
("https://holod.media", 302, "https://evil.com/block", DpiStatus::RedirSuspect, "→ evil.com"),
|
||||
("https://holod.media", 302, "https://not-holod.media/", DpiStatus::RedirSuspect, "→ not-holod.media"),
|
||||
("https://holod.media", 302, "https://holod.media.evil.com/", DpiStatus::RedirSuspect, "→ holod.media.evil.com"),
|
||||
let cases: &[(&str, u16, &str, DpiStatus, Detail)] = &[
|
||||
("https://holod.media", 301, "https://holod.media/", DpiStatus::Ok, Detail::UpgradeHttps { status: None }),
|
||||
("https://holod.media", 301, "/en/", DpiStatus::Ok, Detail::UpgradeHttps { status: None }),
|
||||
("https://holod.media", 301, "index.html", DpiStatus::Ok, Detail::UpgradeHttps { status: None }),
|
||||
("https://holod.media", 301, "?q=1", DpiStatus::Ok, Detail::UpgradeHttps { status: None }),
|
||||
("https://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, Detail::UpgradeHttps { status: None }),
|
||||
("https://holod.media", 302, "http://holod.media/x", DpiStatus::Ok, Detail::Redirect { host: "holod.media".into() }),
|
||||
("http://holod.media", 301, "https://holod.media/", DpiStatus::Ok, Detail::UpgradeHttps { status: Some(301) }),
|
||||
("http://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, Detail::UpgradeHttps { status: Some(301) }),
|
||||
("http://holod.media", 301, "http://holod.media/x", DpiStatus::Ok, Detail::HttpStatus(301)),
|
||||
("http://holod.media", 301, "/en/", DpiStatus::Ok, Detail::HttpStatus(301)),
|
||||
("http://holod.media", 301, "index.html", DpiStatus::Ok, Detail::HttpStatus(301)),
|
||||
("http://holod.media", 301, "?q=1", DpiStatus::Ok, Detail::HttpStatus(301)),
|
||||
("https://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, Detail::Redirect { host: "evil.com".into() }),
|
||||
("http://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, Detail::Redirect { host: "evil.com".into() }),
|
||||
("https://holod.media", 302, "https://evil.com/block", DpiStatus::RedirSuspect, Detail::Redirect { host: "evil.com".into() }),
|
||||
("https://holod.media", 302, "https://not-holod.media/", DpiStatus::RedirSuspect, Detail::Redirect { host: "not-holod.media".into() }),
|
||||
("https://holod.media", 302, "https://holod.media.evil.com/", DpiStatus::RedirSuspect, Detail::Redirect { host: "holod.media.evil.com".into() }),
|
||||
];
|
||||
for (base, status, location, want, detail) in cases {
|
||||
let http_phase = base.starts_with("http:");
|
||||
let (s, d) = classify_redirect("holod.media", base, *status, location, http_phase);
|
||||
assert_eq!(s, *want, "{base} + {location} -> {}", d.code());
|
||||
assert_eq!(d.code(), *detail, "{base} + {location}");
|
||||
assert_eq!(&d, detail, "{base} + {location}");
|
||||
// A legitimate redirect counts as success, a foreign one does not.
|
||||
assert_eq!(s.is_ok_status(), *want != DpiStatus::RedirSuspect, "{base} + {location}");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -269,8 +269,18 @@ fn resolve_location(base: &str, location: &str) -> String {
|
|||
}
|
||||
}
|
||||
|
||||
/// Classifies a redirect by host: same host or subdomain → OK (in the HTTP phase a
|
||||
/// same-host hop to https reads `301 → https`), a foreign host → REDIR with its short name.
|
||||
/// Classifies a redirect by host: the same site family → OK (in the HTTP phase a
|
||||
/// same-host hop to https reads `301 → https`, elsewhere `→ https`), a foreign
|
||||
/// host → REDIR with the host it named.
|
||||
///
|
||||
/// "Same family" means the two hosts are the same name, or one is a subdomain of
|
||||
/// the other at a label boundary, with a leading `www.` ignored on both sides.
|
||||
/// The comparison is symmetric because both directions are how a site spells
|
||||
/// itself: `holod.media` → `www.holod.media` and `www.holod.media` →
|
||||
/// `holod.media`, and equally `m.holod.media` → `holod.media`. Everything else
|
||||
/// counts as foreign, so a redirect from `www.instagram.com` to
|
||||
/// `www.facebook.com` stays a red REDIR — and so does `a.example.com` →
|
||||
/// `b.example.com`, which is two different sites under one parent.
|
||||
pub(crate) fn classify_redirect(
|
||||
domain: &str,
|
||||
base_url: &str,
|
||||
|
|
@ -285,28 +295,30 @@ pub(crate) fn classify_redirect(
|
|||
let scheme_https = resolved.to_ascii_lowercase().starts_with("https");
|
||||
let norm_loc = strip_www(&loc_host).to_string();
|
||||
let norm_dom = strip_www(&domain.to_ascii_lowercase()).to_string();
|
||||
let same_host = norm_loc == norm_dom || norm_loc.ends_with(&format!(".{}", norm_dom));
|
||||
let same_host = norm_loc == norm_dom
|
||||
|| norm_loc.ends_with(&format!(".{}", norm_dom))
|
||||
|| norm_dom.ends_with(&format!(".{}", norm_loc));
|
||||
let short_host: String = loc_host.chars().take(30).collect();
|
||||
|
||||
if http_phase {
|
||||
if same_host && scheme_https {
|
||||
return (DpiStatus::Ok, Detail::Other(format!("{} → https", status)));
|
||||
return (DpiStatus::Ok, Detail::UpgradeHttps { status: Some(status) });
|
||||
}
|
||||
if same_host {
|
||||
// Same domain (or a subdomain) is a normal redirect: OK, not a badge
|
||||
// of its own. Only a foreign domain is flagged, as a red REDIR.
|
||||
return (DpiStatus::Ok, Detail::HttpStatus(status));
|
||||
}
|
||||
return (DpiStatus::RedirSuspect, Detail::Other(format!("→ {}", short_host)));
|
||||
return (DpiStatus::RedirSuspect, Detail::Redirect { host: short_host });
|
||||
}
|
||||
|
||||
if same_host && scheme_https {
|
||||
return (DpiStatus::Ok, Detail::Other("→ https".to_string()));
|
||||
return (DpiStatus::Ok, Detail::UpgradeHttps { status: None });
|
||||
}
|
||||
if same_host {
|
||||
return (DpiStatus::Ok, Detail::Other(format!("→ {}", short_host)));
|
||||
return (DpiStatus::Ok, Detail::Redirect { host: short_host });
|
||||
}
|
||||
(DpiStatus::RedirSuspect, Detail::Other(format!("→ {}", short_host)))
|
||||
(DpiStatus::RedirSuspect, Detail::Redirect { host: short_host })
|
||||
}
|
||||
|
||||
pub(crate) fn inner_hyper(
|
||||
|
|
|
|||
|
|
@ -32,6 +32,9 @@ pub struct CliArgs {
|
|||
pub burst_tls: Option<String>,
|
||||
/// Test 6: ALPN to offer, `h2` or `http/1.1`.
|
||||
pub burst_alpn: Option<String>,
|
||||
/// Test 6: per-attempt trace. `Some("")` is stderr, `Some(path)` that file,
|
||||
/// `None` means no trace.
|
||||
pub trace: Option<String>,
|
||||
}
|
||||
|
||||
/// Builds the CLI definition with `msg`'s language: `about`, per-argument help
|
||||
|
|
@ -179,6 +182,17 @@ pub fn command(msg: &Messages) -> Command {
|
|||
.help_heading(msg.cli_options_heading)
|
||||
.help(msg.cli_burst_alpn),
|
||||
)
|
||||
.arg(
|
||||
// An optional value: no path means stderr, a path means that file.
|
||||
// The empty string is what clap stores for "flag given, no value".
|
||||
Arg::new("trace")
|
||||
.long("trace")
|
||||
.value_name("PATH")
|
||||
.num_args(0..=1)
|
||||
.default_missing_value("")
|
||||
.help_heading(msg.cli_options_heading)
|
||||
.help(msg.cli_trace),
|
||||
)
|
||||
.arg(
|
||||
Arg::new("help")
|
||||
.short('h')
|
||||
|
|
@ -221,5 +235,6 @@ pub fn parse_cli(lang: Language) -> CliArgs {
|
|||
burst_profiles: m.get_one::<String>("burst-profiles").cloned(),
|
||||
burst_tls: m.get_one::<String>("burst-tls").cloned(),
|
||||
burst_alpn: m.get_one::<String>("burst-alpn").cloned(),
|
||||
trace: m.get_one::<String>("trace").cloned(),
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -65,6 +65,16 @@ pub fn detail_text(detail: &Detail, lang: Language) -> String {
|
|||
&format!("Alert-e TLS ({name})"),
|
||||
)
|
||||
}
|
||||
Detail::StackFailure(kind) => {
|
||||
let name = kind.code();
|
||||
t4(
|
||||
lang,
|
||||
&format!("TLS-стек ({name})"),
|
||||
&format!("TLS stack ({name})"),
|
||||
&format!("TLS 协议栈（{name}）"),
|
||||
&format!("Khatay-e TLS ({name})"),
|
||||
)
|
||||
}
|
||||
Detail::UnknownConnectionFailure => t4(lang, "Unknown connection failure", "Unknown connection failure", "未知连接错误", "Khata-ye nashenakhte dar ettesal"),
|
||||
Detail::DomainNotFound => t4(lang, "Домен не найден", "Domain not found", "域名未找到", "Domain peyda nashod"),
|
||||
Detail::DnsTimeoutUnavailable => t4(lang, "DNS таймаут/недоступен", "DNS timeout/unavailable", "DNS 超时/不可用", "DNS mohlat ya dar dastras nist"),
|
||||
|
|
@ -91,6 +101,11 @@ pub fn detail_text(detail: &Detail, lang: Language) -> String {
|
|||
Detail::IspBlockpage { arrow: false, ip } => format!("{} {}", msg.detail_isp_stub, ip),
|
||||
Detail::LocalIp { ip } => format!("{} -> {}", msg.detail_local_ip, ip),
|
||||
Detail::HttpStatus(code) => format!("HTTP {}", code),
|
||||
// The target of a redirect is a host and a scheme, not prose: the arrow
|
||||
// is the same in every language (and maps to `->` on an ASCII console).
|
||||
Detail::Redirect { host } => format!("→ {host}"),
|
||||
Detail::UpgradeHttps { status: Some(code) } => format!("{code} → https"),
|
||||
Detail::UpgradeHttps { status: None } => "→ https".to_string(),
|
||||
Detail::Elapsed(secs) => format!("{:.1}s", secs),
|
||||
// Free text from the OS or the TLS stack: appended as it came.
|
||||
Detail::Other(text) => text.clone(),
|
||||
|
|
@ -101,7 +116,7 @@ pub fn detail_text(detail: &Detail, lang: Language) -> String {
|
|||
mod tests {
|
||||
use super::*;
|
||||
use crate::i18n::Language::*;
|
||||
use dpi_core::classify::detail::AlertKind;
|
||||
use dpi_core::classify::detail::{AlertKind, StackKind};
|
||||
|
||||
#[test]
|
||||
fn composed_details_keep_their_tokens() {
|
||||
|
|
@ -119,6 +134,22 @@ mod tests {
|
|||
assert_eq!(detail_text(&stub, En), "ISP blockpage -> 1.1.1.1");
|
||||
assert_eq!(detail_text(&Detail::LocalIp { ip: "10.0.0.1".into() }, Zh), "本地 IP -> 10.0.0.1");
|
||||
assert_eq!(detail_text(&Detail::HttpStatus(451), Zh), "HTTP 451");
|
||||
assert_eq!(
|
||||
detail_text(&Detail::StackFailure(StackKind::DecryptError), Ru),
|
||||
"TLS-стек (decrypt_error)"
|
||||
);
|
||||
assert_eq!(
|
||||
detail_text(&Detail::StackFailure(StackKind::PeerMisbehaved), En),
|
||||
"TLS stack (peer_misbehaved)"
|
||||
);
|
||||
// A redirect reads as its target in every language, and an https hop
|
||||
// names the status that carried it when one is known.
|
||||
let foreign = Detail::Redirect { host: "www.facebook.com".into() };
|
||||
for lang in [En, Ru, Zh, Fa] {
|
||||
assert_eq!(detail_text(&foreign, lang), "→ www.facebook.com");
|
||||
}
|
||||
assert_eq!(detail_text(&Detail::UpgradeHttps { status: Some(301) }, Ru), "301 → https");
|
||||
assert_eq!(detail_text(&Detail::UpgradeHttps { status: None }, Ru), "→ https");
|
||||
assert_eq!(detail_text(&Detail::Elapsed(0.3), Ru), "0.3s");
|
||||
assert_eq!(detail_text(&Detail::None, Ru), "");
|
||||
assert_eq!(detail_text(&Detail::Other("→ https".into()), En), "→ https");
|
||||
|
|
|
|||
|
|
@ -219,6 +219,8 @@ pub(crate) fn messages() -> Messages {
|
|||
cli_burst_profiles: "Test 6 fingerprints: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [default: all]",
|
||||
cli_burst_tls: "Test 6 TLS version: 1.2|1.3 [default: 1.3]",
|
||||
cli_burst_alpn: "Test 6 ALPN: h2 (offers h2 with the http/1.1 fallback)|http/1.1 (offers http/1.1 only) [default: h2]",
|
||||
cli_trace: "Test 6 trace: one line per attempt (round, shape, host, status, detail, ms); without a path to stderr, with one to that file [default: off]",
|
||||
trace_open_failed: "Could not open the trace file {}",
|
||||
cli_domains: "Path to custom domain list file",
|
||||
cli_tcp16: "Path to custom TCP16 target file",
|
||||
cli_ascii: "ASCII-only output for legacy consoles (no Unicode glyphs or borders)",
|
||||
|
|
|
|||
|
|
@ -212,6 +212,8 @@ pub(crate) fn messages() -> Messages {
|
|||
cli_burst_profiles: "Fingerprint haye test 6: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [pishfarz: all]",
|
||||
cli_burst_tls: "Version-e TLS baraye test 6: 1.2|1.3 [pishfarz: 1.3]",
|
||||
cli_burst_alpn: "ALPN baraye test 6: h2 (h2 ba bazgasht be http/1.1)|http/1.1 (faghat http/1.1) [pishfarz: h2]",
|
||||
cli_trace: "Trace baraye test 6: yek khat baraye har talash (dore, fingerPrint, domain, vaziyat, joz'yat, ms); bedun-e masir be stderr, ba masir be an file [pishfarz: khamush]",
|
||||
trace_open_failed: "File-e trace baz nashod {}",
|
||||
cli_domains: "Masir-e file-e list-e domain ha",
|
||||
cli_tcp16: "Masir-e file-e target haye TCP16",
|
||||
cli_ascii: "Khorooj-e faghat ASCII baraye console haye ghadimi (bedun-e glyph ya border-e Unicode)",
|
||||
|
|
|
|||
|
|
@ -232,6 +232,9 @@ pub struct Messages {
|
|||
pub cli_burst_profiles: &'static str,
|
||||
pub cli_burst_tls: &'static str,
|
||||
pub cli_burst_alpn: &'static str,
|
||||
pub cli_trace: &'static str,
|
||||
/// Shown when the `--trace` path cannot be opened; `{}` is `<path>: <error>`.
|
||||
pub trace_open_failed: &'static str,
|
||||
pub cli_domains: &'static str,
|
||||
pub cli_tcp16: &'static str,
|
||||
pub cli_ascii: &'static str,
|
||||
|
|
|
|||
|
|
@ -219,6 +219,8 @@ pub(crate) fn messages() -> Messages {
|
|||
cli_burst_profiles: "Fingerprint для теста 6: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [по умолчанию: all]",
|
||||
cli_burst_tls: "Версия TLS для теста 6: 1.2|1.3 [по умолчанию: 1.3]",
|
||||
cli_burst_alpn: "ALPN для теста 6: h2 (предлагает h2 с откатом на http/1.1)|http/1.1 (только http/1.1) [по умолчанию: h2]",
|
||||
cli_trace: "Трассировка теста 6: строка на каждую попытку (раунд, отпечаток, домен, статус, деталь, мс); без пути — в stderr, с путём — в указанный файл [по умолчанию: выключено]",
|
||||
trace_open_failed: "Не удалось открыть файл трассировки {}",
|
||||
cli_domains: "Путь к файлу со списком доменов",
|
||||
cli_tcp16: "Путь к файлу целей TCP16",
|
||||
cli_ascii: "Только ASCII для старых консолей (без Unicode-глифов и рамок)",
|
||||
|
|
|
|||
|
|
@ -219,6 +219,8 @@ pub(crate) fn messages() -> Messages {
|
|||
cli_burst_profiles: "测试 6 的 Fingerprint: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [默认: all]",
|
||||
cli_burst_tls: "测试 6 的 TLS 版本: 1.2|1.3 [默认: 1.3]",
|
||||
cli_burst_alpn: "测试 6 的 ALPN: h2（提供 h2 并回退 http/1.1）|http/1.1（仅 http/1.1）[默认: h2]",
|
||||
cli_trace: "测试 6 追踪: 每次尝试一行（轮次、指纹、域名、状态、详情、毫秒）；无路径写入 stderr，有路径写入该文件 [默认: 关闭]",
|
||||
trace_open_failed: "无法打开追踪文件 {}",
|
||||
cli_domains: "自定义域名列表文件路径",
|
||||
cli_tcp16: "自定义 TCP16 目标文件路径",
|
||||
cli_ascii: "面向旧终端的纯 ASCII 输出（无 Unicode 符号或边框）",
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
//! share one 1700-line file.
|
||||
|
||||
use std::collections::HashSet;
|
||||
use std::io::IsTerminal;
|
||||
use std::io::{IsTerminal, Write};
|
||||
use std::net::{IpAddr, SocketAddr};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
|
@ -21,8 +21,8 @@ use crate::i18n::{legend_text, Language, Messages};
|
|||
use dpi_core::net::fingerprint::TlsFingerprint;
|
||||
use dpi_core::net::netinfo::{detect_bypass_tools, fetch_public_ips, get_system_dns, is_tun_name};
|
||||
use dpi_core::probe::burst::{
|
||||
burst_targets, BurstAlpn, BurstObserver, BurstSettings, BurstTarget, BurstTlsVersion,
|
||||
BURST_DEFAULT_ATTEMPTS, BURST_DEFAULT_TIMEOUT_SECS,
|
||||
burst_targets, BurstAlpn, BurstObserver, BurstProfileReport, BurstSettings, BurstTarget,
|
||||
BurstTlsVersion, BURST_DEFAULT_ATTEMPTS, BURST_DEFAULT_TIMEOUT_SECS,
|
||||
};
|
||||
use dpi_core::probe::cymru::{fetch_ip_cymru, IpCymruInfo};
|
||||
use dpi_core::probe::domains::{
|
||||
|
|
@ -266,6 +266,128 @@ impl BurstObserver for BurstLine {
|
|||
}
|
||||
}
|
||||
|
||||
/// `--trace`: one line per attempt of test 6, written the moment the attempt
|
||||
/// exists.
|
||||
///
|
||||
/// The line is machine-readable on purpose — a UTC stamp, the round, the shape,
|
||||
/// the host, the attempt, the status token, the detail code and the milliseconds
|
||||
/// — so a round that fell over can be read back afterwards (`grep safari
|
||||
/// trace.log`) or watched live in a second window (`Get-Content -Wait`) while
|
||||
/// the screen redraws. It carries no translated text, so it reads the same in
|
||||
/// every language, and it never touches stdout: the frame and the table stay
|
||||
/// where they were.
|
||||
struct TraceObserver {
|
||||
out: std::sync::Mutex<Box<dyn Write + Send>>,
|
||||
round: std::sync::Mutex<(usize, usize)>,
|
||||
}
|
||||
|
||||
impl TraceObserver {
|
||||
/// `None` when no trace was asked for, or when the file it named cannot be
|
||||
/// opened — in which case the run continues untraced and says why.
|
||||
fn new(target: Option<&str>, msg: &Messages) -> Option<Self> {
|
||||
let target = target?;
|
||||
let out: Box<dyn Write + Send> = if target.is_empty() {
|
||||
Box::new(std::io::stderr())
|
||||
} else {
|
||||
match std::fs::File::create(target) {
|
||||
Ok(file) => Box::new(file),
|
||||
Err(err) => {
|
||||
eprintln!(
|
||||
"{}",
|
||||
msg.trace_open_failed
|
||||
.replacen("{}", &format!("{target}: {err}"), 1)
|
||||
);
|
||||
return None;
|
||||
}
|
||||
}
|
||||
};
|
||||
Some(Self { out: std::sync::Mutex::new(out), round: std::sync::Mutex::new((0, 0)) })
|
||||
}
|
||||
|
||||
fn line(&self, text: &str) {
|
||||
if let Ok(mut out) = self.out.lock() {
|
||||
// A failed write is not worth stopping a diagnostic run for.
|
||||
let _ = writeln!(out, "{text}");
|
||||
let _ = out.flush();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl BurstObserver for TraceObserver {
|
||||
fn round_started(&self, fingerprint: TlsFingerprint, index: usize, total: usize, hosts: usize) {
|
||||
if let Ok(mut round) = self.round.lock() {
|
||||
*round = (index, total);
|
||||
}
|
||||
self.line(&format!(
|
||||
"{} round {}/{} {} ({}) hosts={hosts}",
|
||||
utc_clock(),
|
||||
index + 1,
|
||||
total,
|
||||
fingerprint.display_label(),
|
||||
fingerprint.code()
|
||||
));
|
||||
}
|
||||
|
||||
fn host_probed(&self, fingerprint: TlsFingerprint, domain: &str, report: &BurstProfileReport) {
|
||||
let (index, total) = self.round.lock().map(|r| *r).unwrap_or((0, 0));
|
||||
for (attempt, result) in report.attempts.iter().enumerate() {
|
||||
self.line(&format!(
|
||||
"{} r{}/{} {:<8} {:<28} #{:<2} {:<12} {:<40} {:>5} ms",
|
||||
utc_clock(),
|
||||
index + 1,
|
||||
total,
|
||||
fingerprint.code(),
|
||||
domain,
|
||||
attempt + 1,
|
||||
result.status.as_str(),
|
||||
result.detail.code(),
|
||||
result.ms,
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// UTC, because a timezone database is not worth carrying for a trace: the lines
|
||||
/// only have to line up with each other and with the run.
|
||||
fn utc_clock() -> String {
|
||||
let secs = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs();
|
||||
format!("{:02}:{:02}:{:02}Z", (secs / 3600) % 24, (secs / 60) % 60, secs % 60)
|
||||
}
|
||||
|
||||
/// The observers one run may have: the live line when the terminal can take one,
|
||||
/// and the trace when `--trace` asked for it. Either may be absent, so neither
|
||||
/// mode has to know about the other.
|
||||
struct Observers<'a> {
|
||||
line: Option<&'a BurstLine>,
|
||||
trace: Option<&'a TraceObserver>,
|
||||
}
|
||||
|
||||
impl BurstObserver for Observers<'_> {
|
||||
fn round_started(&self, fingerprint: TlsFingerprint, index: usize, total: usize, hosts: usize) {
|
||||
if let Some(line) = self.line {
|
||||
line.round_started(fingerprint, index, total, hosts);
|
||||
}
|
||||
if let Some(trace) = self.trace {
|
||||
trace.round_started(fingerprint, index, total, hosts);
|
||||
}
|
||||
}
|
||||
|
||||
fn host_finished(&self) {
|
||||
if let Some(line) = self.line {
|
||||
line.host_finished();
|
||||
}
|
||||
}
|
||||
|
||||
fn host_probed(&self, fingerprint: TlsFingerprint, domain: &str, report: &BurstProfileReport) {
|
||||
if let Some(trace) = self.trace {
|
||||
trace.host_probed(fingerprint, domain, report);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(crate) async fn run_test_suite(
|
||||
tests_str: &str,
|
||||
|
|
@ -734,12 +856,12 @@ pub(crate) async fn run_test_suite(
|
|||
// live line that names the shape currently on the wire and how far the
|
||||
// run has got, and nothing else is printed until the table.
|
||||
let line = BurstLine { live: Arc::clone(&live), msg: *msg };
|
||||
let observer: &dyn BurstObserver = if args.json || !std::io::stderr().is_terminal() {
|
||||
&()
|
||||
} else {
|
||||
&line
|
||||
let trace = TraceObserver::new(args.trace.as_deref(), msg);
|
||||
let observers = Observers {
|
||||
line: (!args.json && std::io::stderr().is_terminal()).then_some(&line),
|
||||
trace: trace.as_ref(),
|
||||
};
|
||||
let reports = burst_targets(cfg, &targets, settings, concurrency, observer).await;
|
||||
let reports = burst_targets(cfg, &targets, settings, concurrency, &observers).await;
|
||||
if !args.json && std::io::stderr().is_terminal() {
|
||||
live.finish();
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue