diff --git a/README.md b/README.md index 497bbd1..e59e3a6 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,7 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install --burst-profiles Fingerprint для теста 6: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [по умолчанию: all] --burst-tls Версия TLS для теста 6: 1.2|1.3 [по умолчанию: 1.3] --burst-alpn ALPN для теста 6: h2 (предлагает h2 с откатом на http/1.1)|http/1.1 (только http/1.1) [по умолчанию: h2] + --trace [] Тест 6: трасса по строке на попытку (раунд, отпечаток, домен, статус, деталь, мс); без пути — в stderr, с путём — в указанный файл [по умолчанию: выключено] -h, --help Показать справку -V, --version Показать версию ``` @@ -118,7 +119,7 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install ### Локализация (`crates/dpi-core/src/i18n`) * `Messages` - по одному полю на строку, четыре блока (`En`, `Ru`, `Zh`, `Fa`); шаблоны используют `{}`. -* `legend_sections*()` - таблицы `--legend`; `details.rs` - текст для деталей `DET_*` (в русском блоке строка выводится как есть). +* `legend_sections*()` - таблицы `--legend`; `details.rs` - текст для каждой детали `Detail` (в русском блоке строка выводится как есть). * Язык Farsi рендерится **Finglish** (латиница, без персидской графики и диакритики). * Статусные бейджи и имена протоколов (`OK`, `BLOCKED`, `TLS RST`, `SNI`, `ClientHello`) остаются латиницей в любом языке — так цифры, IP-адреса и строки логов читаются одинаково во всех локалях. @@ -136,14 +137,23 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install При выборе теста `6` открывается свой экран настроек: число одновременных рукопожатий, таймаут одного рукопожатия **в секундах**, **версия TLS** (`1.2`/`1.3`), **протокол HTTP в ALPN** (`h2` — как у браузеров, с откатом на `http/1.1`, или `http/1.1` — только он), домен (в поле показан хост, который проверял прошлый прогон; первый набранный символ заменяет его, а пустое поле — весь список, как в тесте `2`; вставленный URL приводится к имени хоста тем же очистителем, что и список из конфига) и **Fingerprint** (по умолчанию все). Поле домена подсвечивается: серый фон — строка выбрана, синий — в поле идёт ввод. `Q`/`Esc` отменяет тест. -На каждый домен и отпечаток выполняется N одновременных попыток — отдельные TCP-соединения, свежие сессии без resumption. Попытка не заканчивается рукопожатием: сразу после него тем же профилем (и тем же `User-Agent`/заголовками) уходит `GET /` по тому протоколу, который выбрал ALPN, и ответ классифицируется так же, как в тесте 2. Рукопожатие ограничено таймаутом из настроек, ответ ждёт `read_timeout` из `config.yml`; попытка считается удачной, только если вернулся ответ — обрыв на запросе, редирект на чужой хост, `HTTP BLOCK` или таймаут чтения видны как своя строка, а не как «ответили». Две оси дают 4 комбинации, и они измеряют разные вещи: `1.3` против `1.2` — реакцию на версию протокола, `h2` против `http/1.1` — реакцию на ALPN (некоторые триггеры срабатывают именно на h2-хелло; JA4 при этом меняет поле ALPN, `h2` → `h1`). +На каждый домен и отпечаток выполняется N одновременных попыток — отдельные TCP-соединения, свежие сессии без resumption. Попытка не заканчивается рукопожатием: сразу после него тем же профилем (и тем же `User-Agent`/заголовками) уходит `GET /` по тому протоколу, который выбрал ALPN, и ответ классифицируется так же, как в тесте 2. Рукопожатие ограничено таймаутом из настроек, ответ ждёт `read_timeout` из `config.yml`; попытка считается удачной, только если вернулся ответ — обрыв на запросе, редирект на чужой хост, `HTTP BLOCK` или таймаут чтения видны как своя строка, а не как «ответили». Редирект внутри своего сайта вердикта не меняет: поддомен и основной домен — это одна и та же площадка, поэтому `holod.media` → `www.holod.media` и `m.holod.media` → `holod.media` читаются как `OK`, а на чужой домен (`www.instagram.com` → `www.facebook.com`) — как `REDIR` с именем хоста в детали. Две оси дают 4 комбинации, и они измеряют разные вещи: `1.3` против `1.2` — реакцию на версию протокола, `h2` против `http/1.1` — реакцию на ALPN (некоторые триггеры срабатывают именно на h2-хелло; JA4 при этом меняет поле ALPN, `h2` → `h1`). -Ось `1.3` шлёт **браузерный** оффер — 1.2 и 1.3 вместе, как Chrome, Safari и Firefox, — потому что тест спрашивает, блокируется ли *форма браузера*, а хелло, пришпиленный к 1.3, не шлёт ни один браузер. Ответ требуется именно 1.3: если пир выбрал 1.2, строка получает `NO TLS 1.3`. Ось `1.2` — намеренно синтетический клиент, предлагающий только 1.2: хелло с обоими протоколами никогда не получает ответ 1.2, и иначе проверить выживание 1.2-рукопожатия нельзя. Тот же компромисс, что у двух колонок теста 2. В таблице `M/N` по каждому отпечатку (зелёный — все попытки прошли целиком (рукопожатие и запрос), жёлтый — часть, красный — никто), в колонке «Детали» — самый частый вердикт отказа (`TLS RST`, `TLS DROP`, `SYN DROP`, …). Тест идёт последним: серия рукопожатий может «сжечь» цель на время, и измерения после неё недействительны. +Ось `1.3` шлёт **браузерный** оффер — 1.2 и 1.3 вместе, как Chrome, Safari и Firefox, — потому что тест спрашивает, блокируется ли *форма браузера*, а хелло, пришпиленный к 1.3, не шлёт ни один браузер. Ответ требуется именно 1.3: если пир выбрал 1.2, строка получает `NO TLS 1.3`. Ось `1.2` — намеренно синтетический клиент, предлагающий только 1.2: хелло с обоими протоколами никогда не получает ответ 1.2, и иначе проверить выживание 1.2-рукопожатия нельзя. Тот же компромисс, что у двух колонок теста 2. В таблице `M/N` по каждому отпечатку (зелёный — все попытки прошли целиком (рукопожатие и запрос), жёлтый — часть, красный — никто), в колонке «Детали» — **все** вердикты строки, сгруппированные по статусу: частые первыми, при равенстве — в порядке появления (`TLS ALERT ×8 · TLS RST ×4`), так что двенадцать потерянных рукопожатий не схлопываются в одну «победившую» ошибку. Группировка идёт по статусу, а не по детали: разные алерты — одна группа `TLS ALERT`, а конкретная деталь каждой попытки остаётся в `--json` и в трассе. Тест идёт последним: серия рукопожатий может «сжечь» цель на время, и измерения после неё недействительны. Во время прогона печатается одна живая строка `Тестируем: CHROME 107 3/4 12/35 · 00:04` — какая форма сейчас в эфире, который это раунд из скольких, сколько хостов раунда готово и сколько идёт время. Отдельной шапки с настройками и блок «Итог» тест не выводит: отпечаток с версией и так виден в колонках таблицы, а пустой итог не рисуется. Из неинтерактивного запуска настройки задаются флагами `--burst`, `--burst-timeout`, `--burst-profiles`, `--burst-tls`, `--burst-alpn`; в `--json` результат лежит в ключе `results.fingerprint_burst` (вместе с `tls` и `alpn` этого прогона). +Флаг `--trace []` пишет по строке на каждую попытку — время UTC, раунд, форма, домен, номер попытки, статус, код детали и миллисекунды: + +```text +13:19:30Z r1/1 chrome www.google.com #1 tls_alert alert_unexpected_message 719 ms +13:19:30Z r1/1 chrome standby-rezka.tv #1 tls_alert alert_illegal_parameter 1031 ms +``` + +Без пути трасса идёт в stderr, с путём — в указанный файл, что и делает её пригодной для интерактивного экрана: `--trace C:\temp\trace.log` плюс `Get-Content -Wait C:\temp\trace.log` во втором окне, пока TUI перерисовывает кадр. Строки не переводятся — это машинный журнал: видно, какая именно из N попыток, с какой задержкой и по какой причине не прошла (тишина до нашего таймаута, сброс за десятки миллисекунд или алерт от пира), и `grep` по форме или домену отвечает на вопрос «почему эта колонка красная». + ### Профили отпечатка (`--fingerprint`) Профиль задаёт три слоя сразу, и все три взяты из одного и того же набора `curl-impersonate v2.2.2`, поэтому клиент, выглядящий как `curl_chrome107` в одном слое, выглядит так же и в остальных: @@ -152,11 +162,11 @@ curl -fsSL https://raw.githubusercontent.com/Runnin4ik/dpi-detector/rust/install | --- | --- | --- | | TLS | список шифров и их порядок, группы, `signature_algorithms`, ALPN, порядок расширений, GREASE, ALPS, padding до 512, сжатие сертификата, а у Safari ещё и фолбэки `0x0302`/`0x0301` в `supported_versions`; тесты пинуют JA3 и JA4 против бандла | ECH (синтез ломает рукопожатие с Google и Cloudflare, поэтому у Firefox форма на одно расширение и на 357 байт короче оригинала); там, где версия пришпилена ради изоляции (две колонки теста 2 и ось `1.2` теста 6), в `supported_versions` ровно одна версия — JA3/JA4 этого не видят, читающий тело middlebox видит | | HTTP | `User-Agent` и набор заголовков клиента в его порядке (`sec-ch-ua*`, `accept`, `sec-fetch-*`, `accept-language`; у Firefox ещё `priority` и `te`) | `accept-encoding` всегда `identity`: тесты 2–4 считают байты до обрыва, а согласованное сжатие сделало бы эти числа зависящими от сжимаемости ответа | -| HTTP/2 | значения `SETTINGS` и то, какие из них отправляются, плюс оконный `WINDOW_UPDATE` | порядок псевдозаголовков (`m,s,a,p` — порядок hyper; Chrome шлёт `m,a,s,p`, Firefox `m,p,a,s`, Safari `m,s,p,a`) и приоритет на `HEADERS` (h2 0.4 не умеет приоритеты) | +| HTTP/2 | значения и **состав** `SETTINGS`, **порядок** `SETTINGS` (у Safari `4:4194304` идёт перед `3:100`), оконный `WINDOW_UPDATE`, порядок псевдозаголовков (`m,a,s,p` — Chrome, `m,p,a,s` — Firefox, `m,s,p,a` — Safari) и приоритет на `HEADERS` с весом и зависимостью (Chrome 256, Firefox 42, Safari 255) | — | Свой `user_agent` в `config.yml` перебивает профиль: если поле отличается от встроенного значения по умолчанию, на провод уходит именно оно. `rustls` — контрольная форма: она никого не изображает и оставляет заголовки и преамбулу HTTP/2 такими, какими их отправляли все прошлые измерения. -Проверка на `tls.peet.ws`: JA3/JA4, `User-Agent`, набор и порядок заголовков, `SETTINGS` и `WINDOW_UPDATE` совпадают с `curl_chrome107` в точности; `peetprint` (он сворачивает в один хэш ещё и порядок псевдозаголовков с приоритетом) — нет. +Проверка на `tls.peet.ws`: JA3 и его хэш, JA4/JA4_r, `peetprint` с хэшем, `User-Agent`, набор и порядок заголовков, `SETTINGS` (состав, значения и порядок), `WINDOW_UPDATE`, порядок псевдозаголовков, приоритет на `HEADERS` и akamai-отпечаток совпадают с бандлами в точности — у `curl_chrome107`, `curl_firefox133` и `curl_safari155` соответственно. Расхождения, которые остаются во всех трёх профилях, — ECH у Firefox и `accept-encoding: identity` (см. таблицу выше). --- diff --git a/crates/dpi-core/src/classify/classifier.rs b/crates/dpi-core/src/classify/classifier.rs index d3bf89d..75931eb 100644 --- a/crates/dpi-core/src/classify/classifier.rs +++ b/crates/dpi-core/src/classify/classifier.rs @@ -1,6 +1,7 @@ use std::io; use super::alert; use super::detail::{AlertKind, Detail}; +use super::stack; use super::types::{ConnectionStage, DpiStatus}; fn short_detail(msg: &str) -> String { @@ -16,6 +17,39 @@ fn is_tls_stage(stage: ConnectionStage) -> bool { ) } +/// Messages about a protocol element our own TLS stack does not implement are +/// not evidence about the network. rustls words these with "certificate" (`got +/// CompressedCertificate when expecting Certificate`, +/// `UnknownCertificateExtension`), so without this they read as a certificate +/// interception — and, since rustls reports them the same way it reports a peer +/// that broke the protocol, they would also take a stack failure's name. Real +/// interference still lands in the alert, record and EOF branches. +fn is_unimplemented_element(msg_lower: &str) -> bool { + [ + "unsupportedextension", + "unsupported extension", + "unknowncertificateextension", + "unknown certificate extension", + "compressedcertificate", + "compressed certificate", + ] + .iter() + .any(|m| msg_lower.contains(m)) +} + +/// The detail for a message nothing else classified: a named TLS-stack failure +/// carries the rustls name it was classified by, anything else keeps its own +/// words. +fn unclassified_detail(msg_lower: &str, raw: &str) -> Detail { + if is_unimplemented_element(msg_lower) { + return Detail::Other(short_detail(raw)); + } + match stack::from_message(msg_lower) { + Some(kind) => stack::detail_for(kind), + None => Detail::Other(short_detail(raw)), + } +} + /// The verdict for a message that reports a peer alert: the description it names /// decides the detail. /// @@ -79,22 +113,10 @@ pub fn classify_ssl_error( return (DpiStatus::TlsAlert, Detail::FakeTlsAlert); } // A message about a protocol element our own TLS stack does not implement is - // not evidence about the network. rustls words these with "certificate" - // (`got CompressedCertificate when expecting Certificate`, - // `UnknownCertificateExtension`), so without this they are reported as an - // interception. Real interference still lands in the branches above: a - // censor's spoofed ServerHello arrives as a version/record/alert error. - if [ - "unsupportedextension", - "unsupported extension", - "unknowncertificateextension", - "unknown certificate extension", - "compressedcertificate", - "compressed certificate", - ] - .iter() - .any(|m| msg.contains(m)) - { + // not evidence about the network (see `is_unimplemented_element`). Real + // interference still lands in the branches above: a censor's spoofed + // ServerHello arrives as a version/record/alert error. + if is_unimplemented_element(&msg) { return (DpiStatus::Unknown, Detail::Other(short_detail(err_msg))); } @@ -141,7 +163,7 @@ pub fn classify_ssl_error( return (DpiStatus::NoTls13, Detail::NoTls13); } - (DpiStatus::Unknown, Detail::Other(short_detail(err_msg))) + (DpiStatus::Unknown, unclassified_detail(&msg, err_msg)) } fn dns_failure_text(msg: &str) -> bool { @@ -271,7 +293,7 @@ pub fn classify_connect_error_full( return (DpiStatus::OsErr, Detail::Other(format!("OS errno {}", code))); } - (DpiStatus::Unknown, Detail::Other(short_detail(err_msg))) + (DpiStatus::Unknown, unclassified_detail(&full, err_msg)) } /// Legacy io::Error-based entry point (stage unknown → tcp_connect). @@ -331,27 +353,31 @@ pub fn classify_tls_error( // A reset/EOF after ClientHello with zero bytes back is a DPI SNI RST, // even when the OS masks it as a generic error. - if status == DpiStatus::Unknown - && (stage == ConnectionStage::TlsClientHelloSent || (bytes_sent > 0 && bytes_recv == 0)) - { + if status == DpiStatus::Unknown { let lower = err_msg.to_ascii_lowercase(); - if lower.contains("reset") - || lower.contains("eof") - || lower.contains("closed") - || lower.contains("abort") - || lower.contains("broken pipe") - || lower.contains("10054") - || lower.contains(" 104") + if (stage == ConnectionStage::TlsClientHelloSent || (bytes_sent > 0 && bytes_recv == 0)) + && (lower.contains("reset") + || lower.contains("eof") + || lower.contains("closed") + || lower.contains("abort") + || lower.contains("broken pipe") + || lower.contains("10054") + || lower.contains(" 104")) { return ( DpiStatus::TlsRst, Detail::TlsRstHello, ); } - } - if status == DpiStatus::Unknown { - return (DpiStatus::Unknown, Detail::Other(format!("TLS error: {}", short_detail(err_msg)))); + // The stack's own wording survives as a detail; anything else keeps its + // own words under the TLS label. + return match stack::from_message(&lower) { + Some(kind) if !is_unimplemented_element(&lower) => { + (DpiStatus::Unknown, stack::detail_for(kind)) + } + _ => (DpiStatus::Unknown, Detail::Other(format!("TLS error: {}", short_detail(err_msg)))), + }; } (status, detail) } @@ -370,7 +396,7 @@ pub fn classify_read_error( let (status, detail) = classify_connect_error_full(err_msg, raw_os_error, kind, bytes_read, "reading_data"); if status == DpiStatus::Unknown { - return (DpiStatus::Unknown, Detail::Other(short_detail(err_msg))); + return (DpiStatus::Unknown, unclassified_detail(&err_msg.to_ascii_lowercase(), err_msg)); } (status, detail) } @@ -505,11 +531,48 @@ mod tests { for msg in [ "received unexpected handshake message: got CompressedCertificate when expecting Certificate or CertificateRequest", "received corrupt message of type UnknownCertificateExtension", - "peer misbehaved: UnsolicitedServerHelloExtension", ] { let (s, d) = classify_ssl_error(msg, 0, ConnectionStage::TlsClientHelloSent); assert_eq!(s, DpiStatus::Unknown, "{msg} -> {}", d.code()); + // Our stack lacking a feature is not a peer failure: the message + // keeps its own words instead of taking a stack failure's name. + assert!(matches!(d, Detail::Other(_)), "{msg} -> {}", d.code()); } + + // A peer that broke the protocol is a stack failure the reader can name, + // but still not evidence of interception: the status stays Unknown. + let (s, d) = classify_ssl_error( + "peer misbehaved: UnsolicitedServerHelloExtension", + 0, + ConnectionStage::TlsClientHelloSent, + ); + assert_eq!((s, d.code().as_ref()), (DpiStatus::Unknown, "tls_peer_misbehaved")); + } + + /// A record that will not decrypt is the signature of an injected or + /// rewritten record, and it used to reach the table as the rustls sentence + /// in English. It now carries the stack failure's name; the status stays + /// Unknown because a buggy server produces the same message. + #[test] + fn test_a_stack_failure_is_named_not_quoted() { + let observed = "connection error | cannot decrypt peer's message"; + let (s, d) = classify_read_error(observed, None, None, 0); + assert_eq!((s, d.code().as_ref()), (DpiStatus::Unknown, "tls_decrypt_error")); + + // The same wording through the TLS path keeps the same detail. + let (s, d) = classify_tls_error( + ConnectionStage::TlsClientHelloSent, + 100, + 0, + observed, + false, + ); + assert_eq!((s, d.code().as_ref()), (DpiStatus::Unknown, "tls_decrypt_error")); + + // A message naming nothing known keeps its own words. + let (s, d) = classify_read_error("connection error | oops", None, None, 0); + assert_eq!(s, DpiStatus::Unknown); + assert_eq!(d.code().as_ref(), "connection error | oops"); } #[test] diff --git a/crates/dpi-core/src/classify/detail.rs b/crates/dpi-core/src/classify/detail.rs index cc4dc79..a817c29 100644 --- a/crates/dpi-core/src/classify/detail.rs +++ b/crates/dpi-core/src/classify/detail.rs @@ -49,6 +49,10 @@ pub enum Detail { /// A peer alert that named its description: the code is /// `alert_` (`alert_illegal_parameter`). Alert(AlertKind), + /// A TLS-stack failure that is not an alert from the peer: our own TLS layer + /// refused the bytes it received — `cannot decrypt peer's message`, `peer + /// misbehaved: …`. The code is `tls_`. + StackFailure(StackKind), // ─── Certificates ─── NoRootCa, @@ -107,6 +111,12 @@ pub enum Detail { IspBlockpage { arrow: bool, ip: String }, /// The resolver answered with a loopback/private address (`-> `). LocalIp { ip: String }, + /// The peer redirected to `host`: on the same site family the verdict stays + /// OK, a foreign one is a red `REDIR`. + Redirect { host: String }, + /// A hop to `https` on the same host: `-> https`, or `301 -> https` when the + /// status that carried it is known. + UpgradeHttps { status: Option }, /// An HTTP status line. HttpStatus(u16), /// How long a successful check took. @@ -211,6 +221,54 @@ impl AlertKind { } } +/// One failure of our own TLS stack over bytes the peer sent, named as rustls +/// names it. +/// +/// These are the messages a middlebox leaves behind when it rewrites or injects +/// records: a record that does not decrypt, a message that arrived out of order, +/// a peer that broke the protocol. rustls words each one differently and the +/// difference is the diagnosis, so they are values, not free text — the free +/// text would be English prose inside a Russian table, and it changes with the +/// rustls version, which no test can pin. +/// +/// The names are rustls's own [`Error`] variants, snake_case (see +/// `vendor/rustls/src/error.rs`). They are protocol tokens, so they stay Latin +/// in every language (Rule 4) and `--json` carries them inside the detail code +/// after `tls_`. +/// +/// [`Error`]: https://docs.rs/rustls/latest/rustls/enum.Error.html +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum StackKind { + /// `cannot decrypt peer's message`: a record we could not open. + DecryptError, + /// `received corrupt message of type …`. + InvalidMessage, + /// `received unexpected message: got … when expecting …`. + InappropriateMessage, + /// `received unexpected handshake message: got … when expecting …`. + InappropriateHandshakeMessage, + /// `peer misbehaved: …`. + PeerMisbehaved, + /// `peer doesn't support any known protocol`: no ALPN in common. + NoApplicationProtocol, +} + +impl StackKind { + /// The rustls name, snake_case: the token a [`Detail::StackFailure`]'s code + /// carries after `tls_`. + pub fn code(self) -> &'static str { + use StackKind::*; + match self { + DecryptError => "decrypt_error", + InvalidMessage => "invalid_message", + InappropriateMessage => "inappropriate_message", + InappropriateHandshakeMessage => "inappropriate_handshake_message", + PeerMisbehaved => "peer_misbehaved", + NoApplicationProtocol => "no_application_protocol", + } + } +} + impl Detail { /// The machine token. `--json` carries exactly this; it never changes with /// `--lang`. @@ -231,6 +289,7 @@ impl Detail { ProtocolVersionAlert => Cow::Borrowed("protocol_version_alert"), FakeTlsAlert => Cow::Borrowed("fake_tls_alert"), Alert(kind) => Cow::Owned(format!("alert_{}", kind.code())), + StackFailure(kind) => Cow::Owned(format!("tls_{}", kind.code())), NoRootCa => Cow::Borrowed("no_root_certificates"), CertExpired => Cow::Borrowed("cert_expired"), SelfSigned => Cow::Borrowed("self_signed_cert"), @@ -266,6 +325,8 @@ impl Detail { TimeoutStage { stage } => Cow::Owned(format!("timeout_{}", stage)), IspBlockpage { .. } => Cow::Borrowed("isp_blockpage"), LocalIp { .. } => Cow::Borrowed("local_ip"), + Redirect { .. } => Cow::Borrowed("redirect_to_host"), + UpgradeHttps { .. } => Cow::Borrowed("upgrade_https"), HttpStatus(code) => Cow::Owned(format!("http_{}", code)), Elapsed(secs) => Cow::Owned(format!("elapsed_{}ms", (secs * 1000.0).round() as u64)), Other(text) => Cow::Owned(text.clone()), @@ -339,6 +400,11 @@ mod tests { Detail::Elapsed(0.3), Detail::Alert(AlertKind::IllegalParameter), Detail::Alert(AlertKind::BadCertificateStatusResponse), + Detail::StackFailure(StackKind::DecryptError), + Detail::StackFailure(StackKind::InappropriateHandshakeMessage), + Detail::Redirect { host: "example.com".into() }, + Detail::UpgradeHttps { status: Some(301) }, + Detail::UpgradeHttps { status: None }, ]; for d in &all { let code = d.code(); @@ -348,6 +414,13 @@ mod tests { ); } assert_eq!(Detail::HttpStatus(403).code(), "http_403"); + assert_eq!(Detail::StackFailure(StackKind::DecryptError).code(), "tls_decrypt_error"); + assert_eq!( + Detail::StackFailure(StackKind::InappropriateHandshakeMessage).code(), + "tls_inappropriate_handshake_message" + ); + assert_eq!(Detail::Redirect { host: "example.com".into() }.code(), "redirect_to_host"); + assert_eq!(Detail::UpgradeHttps { status: Some(301) }.code(), "upgrade_https"); assert_eq!(Detail::Elapsed(0.3).code(), "elapsed_300ms"); assert_eq!(Detail::None.code(), ""); } diff --git a/crates/dpi-core/src/classify/mod.rs b/crates/dpi-core/src/classify/mod.rs index 0280197..f03fc13 100644 --- a/crates/dpi-core/src/classify/mod.rs +++ b/crates/dpi-core/src/classify/mod.rs @@ -1,10 +1,11 @@ pub mod alert; pub mod classifier; +pub mod stack; pub mod stream; pub mod detail; pub mod types; pub use classifier::{classify_connect_error, classify_connect_error_full, classify_read_error, classify_ssl_error, classify_tls_error}; pub use stream::{DpiProbeStream, DpiProbeTracker}; -pub use detail::{AlertKind, Detail}; +pub use detail::{AlertKind, Detail, StackKind}; pub use types::{ConnectionStage, DpiStatus, ProbeMetrics}; diff --git a/crates/dpi-core/src/classify/stack.rs b/crates/dpi-core/src/classify/stack.rs new file mode 100644 index 0000000..3e64ddd --- /dev/null +++ b/crates/dpi-core/src/classify/stack.rs @@ -0,0 +1,111 @@ +//! The TLS-stack table: an error message to the rustls variant it names. +//! +//! Where `alert.rs` reads what the *peer* said, this reads what our own stack +//! said about the peer's bytes. The two never overlap: an alert arrives as +//! `received fatal alert: …`, a stack failure as rustls's own wording, and both +//! tables match on the same lowercased, separator-stripped message. +//! +//! Matching happens on the words with their separators removed, so +//! `peer misbehaved: …`, `PeerMisbehaved` and `peermisbehaved` reach the same +//! value. The needles are the `Display` texts of `rustls::Error` as the vendored +//! rustls spells them (`vendor/rustls/src/error.rs`); a rustls upgrade that +//! rewords one is a change to this table, and the tests below pin the wording so +//! that upgrade fails loudly instead of silently unclassifying the case. +//! +//! A message the table does not know is left to the caller, which keeps its own +//! words (`Detail::Other`) — the long tail of OS and library text stays as it +//! came. +//! +//! Ordering matters at the call sites: the classifier's "our stack does not +//! implement this" early-out runs first, so a server using certificate +//! compression is not read as a peer that misbehaved. + +use super::detail::{Detail, StackKind}; + +use StackKind::*; + +/// `(needle without separators, kind)`, matched in order against a lowercased, +/// separator-stripped message. +/// +/// The longer wording comes first where one needle is a prefix of another +/// (`receivedunexpectedhandshakemessage` before `receivedunexpectedmessage`). +const TABLE: &[(&str, StackKind)] = &[ + ("receivedunexpectedhandshakemessage", InappropriateHandshakeMessage), + ("receivedunexpectedmessage", InappropriateMessage), + ("receivedcorruptmessageoftype", InvalidMessage), + ("cannotdecryptpeer'smessage", DecryptError), + ("peermisbehaved", PeerMisbehaved), + ("doesn'tsupportanyknownprotocol", NoApplicationProtocol), +]; + +/// The rustls failure a message names, or `None` when it names none. +/// +/// `msg_lower` is the message already lowercased by the caller — the classifier +/// keeps one lowercased copy of every error it classifies. +pub fn from_message(msg_lower: &str) -> Option { + let flat: String = msg_lower + .chars() + .filter(|c| !c.is_whitespace() && *c != '_' && *c != '-') + .collect(); + TABLE.iter().find(|(needle, _)| flat.contains(needle)).map(|(_, kind)| *kind) +} + +/// The detail a message reports as. +pub fn detail_for(kind: StackKind) -> Detail { + Detail::StackFailure(kind) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The wording rustls puts on the wire, as the probes see it: the stack's + /// own text, appended after hyper's `connection error` (see + /// `probe::http::hyper_err_info`). + #[test] + fn the_rustls_spelling_names_the_failure() { + for (msg, kind) in [ + ("connection error | cannot decrypt peer's message", DecryptError), + ("cannot decrypt peer's message", DecryptError), + ("received corrupt message of type ChangeCipherSpec", InvalidMessage), + ( + "received unexpected message: got Alert when expecting Handshake", + InappropriateMessage, + ), + ( + "received unexpected handshake message: got ServerHello when expecting ClientHello", + InappropriateHandshakeMessage, + ), + ("peer misbehaved: UnsolicitedServerHelloExtension", PeerMisbehaved), + ("peer doesn't support any known protocol", NoApplicationProtocol), + ] { + let lower = msg.to_ascii_lowercase(); + assert_eq!(from_message(&lower), Some(kind), "{msg}"); + } + } + + /// An alert is not a stack failure: its wording belongs to `alert.rs`, and + /// the two tables must not claim the same message. + #[test] + fn an_alert_wording_is_not_a_stack_failure() { + for msg in [ + "connection error | received fatal alert: UnexpectedMessage", + "received fatal alert: IllegalParameter", + "peer sent no certificates", + "invalid peer certificate: Expired", + ] { + assert_eq!(from_message(&msg.to_ascii_lowercase()), None, "{msg}"); + } + } + + /// A stack failure is still not interception: the status stays unclassified + /// while the detail says what rustls objected to. + #[test] + fn the_detail_is_the_kind() { + assert_eq!(detail_for(DecryptError).code(), "tls_decrypt_error"); + assert_eq!( + detail_for(InappropriateHandshakeMessage).code(), + "tls_inappropriate_handshake_message" + ); + } +} diff --git a/crates/dpi-core/src/probe/burst.rs b/crates/dpi-core/src/probe/burst.rs index 2d2812b..7d7ac3f 100644 --- a/crates/dpi-core/src/probe/burst.rs +++ b/crates/dpi-core/src/probe/burst.rs @@ -327,6 +327,20 @@ pub trait BurstObserver { /// One host finished its round, answered or not. fn host_finished(&self) {} + + /// One host's round landed, with every attempt it produced. + /// + /// The live line counts hosts and ignores this; a trace prints the attempts + /// the moment they exist, which is what turns "SAFARI 0/4" into a status and + /// a millisecond count per attempt. A caller that wants neither implements + /// nothing. + fn host_probed( + &self, + _fingerprint: TlsFingerprint, + _domain: &str, + _report: &BurstProfileReport, + ) { + } } impl BurstObserver for () {} @@ -381,6 +395,7 @@ pub async fn burst_targets( while let Some(joined) = rounds.join_next().await { observer.host_finished(); if let Ok((index, report)) = joined { + observer.host_probed(fingerprint, &targets[index].domain, &report); reports[index].profiles.push(report); } } @@ -875,6 +890,7 @@ mod tests { struct Rounds { started: std::sync::Mutex>, finished: std::sync::atomic::AtomicUsize, + probed: std::sync::Mutex>, } impl BurstObserver for Rounds { fn round_started(&self, fp: TlsFingerprint, index: usize, total: usize, hosts: usize) { @@ -883,6 +899,12 @@ mod tests { fn host_finished(&self) { self.finished.fetch_add(1, Ordering::SeqCst); } + fn host_probed(&self, fp: TlsFingerprint, domain: &str, report: &BurstProfileReport) { + self.probed + .lock() + .expect("lock") + .push(format!("{} {domain} x{}", fp.code(), report.attempts.len())); + } } let rounds = Rounds::default(); let reports = burst_targets(&AppConfig::default(), &targets, &plan, 4, &rounds).await; @@ -904,6 +926,19 @@ mod tests { ] ); assert_eq!(rounds.finished.load(Ordering::SeqCst), 4, "one tick per host per round"); + // The trace reads this: every host's round arrives once per shape, with + // the attempts it produced — silent hosts are not silently dropped. + let mut probed = rounds.probed.lock().expect("lock").clone(); + probed.sort(); + assert_eq!( + probed, + vec![ + "chrome fast.example x2", + "chrome slow.example x2", + "rustls fast.example x2", + "rustls slow.example x2", + ] + ); assert_eq!(reports.len(), 2); for report in &reports { let shapes: Vec = report.profiles.iter().map(|p| p.fingerprint).collect(); diff --git a/crates/dpi-core/src/probe/domains.rs b/crates/dpi-core/src/probe/domains.rs index 8e198e7..b341d78 100644 --- a/crates/dpi-core/src/probe/domains.rs +++ b/crates/dpi-core/src/probe/domains.rs @@ -762,21 +762,50 @@ mod tests { fn test_classify_redirect_same_https() { let (s, d) = classify_redirect("example.com", "https://example.com", 301, "https://example.com/", false); assert_eq!(s, DpiStatus::Ok); - assert_eq!(d, Detail::Other("→ https".to_string())); + assert_eq!(d, Detail::UpgradeHttps { status: None }); } #[test] fn test_classify_redirect_foreign() { let (s, d) = classify_redirect("example.com", "https://example.com", 302, "https://evil.com/block", false); assert_eq!(s, DpiStatus::RedirSuspect); - assert!(d.code().contains("evil.com")); + assert_eq!(d, Detail::Redirect { host: "evil.com".to_string() }); } #[test] fn test_classify_redirect_http_phase() { let (s, d) = classify_redirect("example.com", "http://example.com", 301, "https://example.com/", true); assert_eq!(s, DpiStatus::Ok); - assert_eq!(d, Detail::Other("301 → https".to_string())); + assert_eq!(d, Detail::UpgradeHttps { status: Some(301) }); + } + + /// The rule the table reads by: a redirect that stays inside one site family + /// is a normal `OK`, whatever the direction — a subdomain spelling out the + /// main domain (`www.holod.media` → `holod.media`) and the main domain + /// naming a subdomain (`holod.media` → `cdn.holod.media`) are both the site + /// itself. Only a name outside the family is a red `REDIR`. + #[test] + fn test_a_subdomain_redirect_inside_the_same_site_is_ok() { + let cases: &[(&str, &str, DpiStatus, &str)] = &[ + // (requested domain, Location, expected status, expected redirect host when foreign) + ("www.holod.media", "https://holod.media/x", DpiStatus::Ok, ""), + ("holod.media", "https://www.holod.media/x", DpiStatus::Ok, ""), + ("holod.media", "https://cdn.holod.media/x", DpiStatus::Ok, ""), + ("m.holod.media", "https://holod.media/x", DpiStatus::Ok, ""), + ("www.holod.media", "http://holod.media/x", DpiStatus::Ok, ""), + // A different registrable domain is not the same site, however it + // reads: this is the instagram → facebook case. + ("www.instagram.com", "https://www.facebook.com/x", DpiStatus::RedirSuspect, "www.facebook.com"), + ("holod.media", "https://not-holod.media/x", DpiStatus::RedirSuspect, "not-holod.media"), + ("holod.media", "https://holod.media.evil.com/x", DpiStatus::RedirSuspect, "holod.media.evil.com"), + ]; + for (domain, location, want, foreign) in cases { + let (s, d) = classify_redirect(domain, &format!("https://{domain}"), 301, location, false); + assert_eq!(s, *want, "{domain} + {location} -> {}", d.code()); + if *want == DpiStatus::RedirSuspect { + assert_eq!(d, Detail::Redirect { host: (*foreign).to_string() }, "{domain} + {location}"); + } + } } /// Same-host-or-subdomain redirects read as a plain `OK`, a redirect to @@ -787,31 +816,30 @@ mod tests { #[test] fn test_classify_redirect_matrix() { // (base, status, location, expected status, expected detail) - // (base, status, location, expected status, expected detail code) - let cases: &[(&str, u16, &str, DpiStatus, &str)] = &[ - ("https://holod.media", 301, "https://holod.media/", DpiStatus::Ok, "→ https"), - ("https://holod.media", 301, "/en/", DpiStatus::Ok, "→ https"), - ("https://holod.media", 301, "index.html", DpiStatus::Ok, "→ https"), - ("https://holod.media", 301, "?q=1", DpiStatus::Ok, "→ https"), - ("https://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, "→ https"), - ("https://holod.media", 302, "http://holod.media/x", DpiStatus::Ok, "→ holod.media"), - ("http://holod.media", 301, "https://holod.media/", DpiStatus::Ok, "301 → https"), - ("http://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, "301 → https"), - ("http://holod.media", 301, "http://holod.media/x", DpiStatus::Ok, "http_301"), - ("http://holod.media", 301, "/en/", DpiStatus::Ok, "http_301"), - ("http://holod.media", 301, "index.html", DpiStatus::Ok, "http_301"), - ("http://holod.media", 301, "?q=1", DpiStatus::Ok, "http_301"), - ("https://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, "→ evil.com"), - ("http://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, "→ evil.com"), - ("https://holod.media", 302, "https://evil.com/block", DpiStatus::RedirSuspect, "→ evil.com"), - ("https://holod.media", 302, "https://not-holod.media/", DpiStatus::RedirSuspect, "→ not-holod.media"), - ("https://holod.media", 302, "https://holod.media.evil.com/", DpiStatus::RedirSuspect, "→ holod.media.evil.com"), + let cases: &[(&str, u16, &str, DpiStatus, Detail)] = &[ + ("https://holod.media", 301, "https://holod.media/", DpiStatus::Ok, Detail::UpgradeHttps { status: None }), + ("https://holod.media", 301, "/en/", DpiStatus::Ok, Detail::UpgradeHttps { status: None }), + ("https://holod.media", 301, "index.html", DpiStatus::Ok, Detail::UpgradeHttps { status: None }), + ("https://holod.media", 301, "?q=1", DpiStatus::Ok, Detail::UpgradeHttps { status: None }), + ("https://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, Detail::UpgradeHttps { status: None }), + ("https://holod.media", 302, "http://holod.media/x", DpiStatus::Ok, Detail::Redirect { host: "holod.media".into() }), + ("http://holod.media", 301, "https://holod.media/", DpiStatus::Ok, Detail::UpgradeHttps { status: Some(301) }), + ("http://holod.media", 301, "https://sub.holod.media/x", DpiStatus::Ok, Detail::UpgradeHttps { status: Some(301) }), + ("http://holod.media", 301, "http://holod.media/x", DpiStatus::Ok, Detail::HttpStatus(301)), + ("http://holod.media", 301, "/en/", DpiStatus::Ok, Detail::HttpStatus(301)), + ("http://holod.media", 301, "index.html", DpiStatus::Ok, Detail::HttpStatus(301)), + ("http://holod.media", 301, "?q=1", DpiStatus::Ok, Detail::HttpStatus(301)), + ("https://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, Detail::Redirect { host: "evil.com".into() }), + ("http://holod.media", 302, "//evil.com/block", DpiStatus::RedirSuspect, Detail::Redirect { host: "evil.com".into() }), + ("https://holod.media", 302, "https://evil.com/block", DpiStatus::RedirSuspect, Detail::Redirect { host: "evil.com".into() }), + ("https://holod.media", 302, "https://not-holod.media/", DpiStatus::RedirSuspect, Detail::Redirect { host: "not-holod.media".into() }), + ("https://holod.media", 302, "https://holod.media.evil.com/", DpiStatus::RedirSuspect, Detail::Redirect { host: "holod.media.evil.com".into() }), ]; for (base, status, location, want, detail) in cases { let http_phase = base.starts_with("http:"); let (s, d) = classify_redirect("holod.media", base, *status, location, http_phase); assert_eq!(s, *want, "{base} + {location} -> {}", d.code()); - assert_eq!(d.code(), *detail, "{base} + {location}"); + assert_eq!(&d, detail, "{base} + {location}"); // A legitimate redirect counts as success, a foreign one does not. assert_eq!(s.is_ok_status(), *want != DpiStatus::RedirSuspect, "{base} + {location}"); } diff --git a/crates/dpi-core/src/probe/http.rs b/crates/dpi-core/src/probe/http.rs index c0a05fd..d421389 100644 --- a/crates/dpi-core/src/probe/http.rs +++ b/crates/dpi-core/src/probe/http.rs @@ -269,8 +269,18 @@ fn resolve_location(base: &str, location: &str) -> String { } } -/// Classifies a redirect by host: same host or subdomain → OK (in the HTTP phase a -/// same-host hop to https reads `301 → https`), a foreign host → REDIR with its short name. +/// Classifies a redirect by host: the same site family → OK (in the HTTP phase a +/// same-host hop to https reads `301 → https`, elsewhere `→ https`), a foreign +/// host → REDIR with the host it named. +/// +/// "Same family" means the two hosts are the same name, or one is a subdomain of +/// the other at a label boundary, with a leading `www.` ignored on both sides. +/// The comparison is symmetric because both directions are how a site spells +/// itself: `holod.media` → `www.holod.media` and `www.holod.media` → +/// `holod.media`, and equally `m.holod.media` → `holod.media`. Everything else +/// counts as foreign, so a redirect from `www.instagram.com` to +/// `www.facebook.com` stays a red REDIR — and so does `a.example.com` → +/// `b.example.com`, which is two different sites under one parent. pub(crate) fn classify_redirect( domain: &str, base_url: &str, @@ -285,28 +295,30 @@ pub(crate) fn classify_redirect( let scheme_https = resolved.to_ascii_lowercase().starts_with("https"); let norm_loc = strip_www(&loc_host).to_string(); let norm_dom = strip_www(&domain.to_ascii_lowercase()).to_string(); - let same_host = norm_loc == norm_dom || norm_loc.ends_with(&format!(".{}", norm_dom)); + let same_host = norm_loc == norm_dom + || norm_loc.ends_with(&format!(".{}", norm_dom)) + || norm_dom.ends_with(&format!(".{}", norm_loc)); let short_host: String = loc_host.chars().take(30).collect(); if http_phase { if same_host && scheme_https { - return (DpiStatus::Ok, Detail::Other(format!("{} → https", status))); + return (DpiStatus::Ok, Detail::UpgradeHttps { status: Some(status) }); } if same_host { // Same domain (or a subdomain) is a normal redirect: OK, not a badge // of its own. Only a foreign domain is flagged, as a red REDIR. return (DpiStatus::Ok, Detail::HttpStatus(status)); } - return (DpiStatus::RedirSuspect, Detail::Other(format!("→ {}", short_host))); + return (DpiStatus::RedirSuspect, Detail::Redirect { host: short_host }); } if same_host && scheme_https { - return (DpiStatus::Ok, Detail::Other("→ https".to_string())); + return (DpiStatus::Ok, Detail::UpgradeHttps { status: None }); } if same_host { - return (DpiStatus::Ok, Detail::Other(format!("→ {}", short_host))); + return (DpiStatus::Ok, Detail::Redirect { host: short_host }); } - (DpiStatus::RedirSuspect, Detail::Other(format!("→ {}", short_host))) + (DpiStatus::RedirSuspect, Detail::Redirect { host: short_host }) } pub(crate) fn inner_hyper( diff --git a/crates/dpi-detector/src/args.rs b/crates/dpi-detector/src/args.rs index ff20f80..fa69676 100644 --- a/crates/dpi-detector/src/args.rs +++ b/crates/dpi-detector/src/args.rs @@ -32,6 +32,9 @@ pub struct CliArgs { pub burst_tls: Option, /// Test 6: ALPN to offer, `h2` or `http/1.1`. pub burst_alpn: Option, + /// Test 6: per-attempt trace. `Some("")` is stderr, `Some(path)` that file, + /// `None` means no trace. + pub trace: Option, } /// Builds the CLI definition with `msg`'s language: `about`, per-argument help @@ -179,6 +182,17 @@ pub fn command(msg: &Messages) -> Command { .help_heading(msg.cli_options_heading) .help(msg.cli_burst_alpn), ) + .arg( + // An optional value: no path means stderr, a path means that file. + // The empty string is what clap stores for "flag given, no value". + Arg::new("trace") + .long("trace") + .value_name("PATH") + .num_args(0..=1) + .default_missing_value("") + .help_heading(msg.cli_options_heading) + .help(msg.cli_trace), + ) .arg( Arg::new("help") .short('h') @@ -221,5 +235,6 @@ pub fn parse_cli(lang: Language) -> CliArgs { burst_profiles: m.get_one::("burst-profiles").cloned(), burst_tls: m.get_one::("burst-tls").cloned(), burst_alpn: m.get_one::("burst-alpn").cloned(), + trace: m.get_one::("trace").cloned(), } } diff --git a/crates/dpi-detector/src/i18n/details.rs b/crates/dpi-detector/src/i18n/details.rs index db55127..68c5c40 100644 --- a/crates/dpi-detector/src/i18n/details.rs +++ b/crates/dpi-detector/src/i18n/details.rs @@ -65,6 +65,16 @@ pub fn detail_text(detail: &Detail, lang: Language) -> String { &format!("Alert-e TLS ({name})"), ) } + Detail::StackFailure(kind) => { + let name = kind.code(); + t4( + lang, + &format!("TLS-стек ({name})"), + &format!("TLS stack ({name})"), + &format!("TLS 协议栈（{name}）"), + &format!("Khatay-e TLS ({name})"), + ) + } Detail::UnknownConnectionFailure => t4(lang, "Unknown connection failure", "Unknown connection failure", "未知连接错误", "Khata-ye nashenakhte dar ettesal"), Detail::DomainNotFound => t4(lang, "Домен не найден", "Domain not found", "域名未找到", "Domain peyda nashod"), Detail::DnsTimeoutUnavailable => t4(lang, "DNS таймаут/недоступен", "DNS timeout/unavailable", "DNS 超时/不可用", "DNS mohlat ya dar dastras nist"), @@ -91,6 +101,11 @@ pub fn detail_text(detail: &Detail, lang: Language) -> String { Detail::IspBlockpage { arrow: false, ip } => format!("{} {}", msg.detail_isp_stub, ip), Detail::LocalIp { ip } => format!("{} -> {}", msg.detail_local_ip, ip), Detail::HttpStatus(code) => format!("HTTP {}", code), + // The target of a redirect is a host and a scheme, not prose: the arrow + // is the same in every language (and maps to `->` on an ASCII console). + Detail::Redirect { host } => format!("→ {host}"), + Detail::UpgradeHttps { status: Some(code) } => format!("{code} → https"), + Detail::UpgradeHttps { status: None } => "→ https".to_string(), Detail::Elapsed(secs) => format!("{:.1}s", secs), // Free text from the OS or the TLS stack: appended as it came. Detail::Other(text) => text.clone(), @@ -101,7 +116,7 @@ pub fn detail_text(detail: &Detail, lang: Language) -> String { mod tests { use super::*; use crate::i18n::Language::*; - use dpi_core::classify::detail::AlertKind; + use dpi_core::classify::detail::{AlertKind, StackKind}; #[test] fn composed_details_keep_their_tokens() { @@ -119,6 +134,22 @@ mod tests { assert_eq!(detail_text(&stub, En), "ISP blockpage -> 1.1.1.1"); assert_eq!(detail_text(&Detail::LocalIp { ip: "10.0.0.1".into() }, Zh), "本地 IP -> 10.0.0.1"); assert_eq!(detail_text(&Detail::HttpStatus(451), Zh), "HTTP 451"); + assert_eq!( + detail_text(&Detail::StackFailure(StackKind::DecryptError), Ru), + "TLS-стек (decrypt_error)" + ); + assert_eq!( + detail_text(&Detail::StackFailure(StackKind::PeerMisbehaved), En), + "TLS stack (peer_misbehaved)" + ); + // A redirect reads as its target in every language, and an https hop + // names the status that carried it when one is known. + let foreign = Detail::Redirect { host: "www.facebook.com".into() }; + for lang in [En, Ru, Zh, Fa] { + assert_eq!(detail_text(&foreign, lang), "→ www.facebook.com"); + } + assert_eq!(detail_text(&Detail::UpgradeHttps { status: Some(301) }, Ru), "301 → https"); + assert_eq!(detail_text(&Detail::UpgradeHttps { status: None }, Ru), "→ https"); assert_eq!(detail_text(&Detail::Elapsed(0.3), Ru), "0.3s"); assert_eq!(detail_text(&Detail::None, Ru), ""); assert_eq!(detail_text(&Detail::Other("→ https".into()), En), "→ https"); diff --git a/crates/dpi-detector/src/i18n/en.rs b/crates/dpi-detector/src/i18n/en.rs index a49496f..3b44cdc 100644 --- a/crates/dpi-detector/src/i18n/en.rs +++ b/crates/dpi-detector/src/i18n/en.rs @@ -219,6 +219,8 @@ pub(crate) fn messages() -> Messages { cli_burst_profiles: "Test 6 fingerprints: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [default: all]", cli_burst_tls: "Test 6 TLS version: 1.2|1.3 [default: 1.3]", cli_burst_alpn: "Test 6 ALPN: h2 (offers h2 with the http/1.1 fallback)|http/1.1 (offers http/1.1 only) [default: h2]", + cli_trace: "Test 6 trace: one line per attempt (round, shape, host, status, detail, ms); without a path to stderr, with one to that file [default: off]", + trace_open_failed: "Could not open the trace file {}", cli_domains: "Path to custom domain list file", cli_tcp16: "Path to custom TCP16 target file", cli_ascii: "ASCII-only output for legacy consoles (no Unicode glyphs or borders)", diff --git a/crates/dpi-detector/src/i18n/fa.rs b/crates/dpi-detector/src/i18n/fa.rs index 62f6c86..7b120b7 100644 --- a/crates/dpi-detector/src/i18n/fa.rs +++ b/crates/dpi-detector/src/i18n/fa.rs @@ -212,6 +212,8 @@ pub(crate) fn messages() -> Messages { cli_burst_profiles: "Fingerprint haye test 6: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [pishfarz: all]", cli_burst_tls: "Version-e TLS baraye test 6: 1.2|1.3 [pishfarz: 1.3]", cli_burst_alpn: "ALPN baraye test 6: h2 (h2 ba bazgasht be http/1.1)|http/1.1 (faghat http/1.1) [pishfarz: h2]", + cli_trace: "Trace baraye test 6: yek khat baraye har talash (dore, fingerPrint, domain, vaziyat, joz'yat, ms); bedun-e masir be stderr, ba masir be an file [pishfarz: khamush]", + trace_open_failed: "File-e trace baz nashod {}", cli_domains: "Masir-e file-e list-e domain ha", cli_tcp16: "Masir-e file-e target haye TCP16", cli_ascii: "Khorooj-e faghat ASCII baraye console haye ghadimi (bedun-e glyph ya border-e Unicode)", diff --git a/crates/dpi-detector/src/i18n/messages.rs b/crates/dpi-detector/src/i18n/messages.rs index 2a59b57..23b0122 100644 --- a/crates/dpi-detector/src/i18n/messages.rs +++ b/crates/dpi-detector/src/i18n/messages.rs @@ -232,6 +232,9 @@ pub struct Messages { pub cli_burst_profiles: &'static str, pub cli_burst_tls: &'static str, pub cli_burst_alpn: &'static str, + pub cli_trace: &'static str, + /// Shown when the `--trace` path cannot be opened; `{}` is `: `. + pub trace_open_failed: &'static str, pub cli_domains: &'static str, pub cli_tcp16: &'static str, pub cli_ascii: &'static str, diff --git a/crates/dpi-detector/src/i18n/ru.rs b/crates/dpi-detector/src/i18n/ru.rs index b0b88c8..a4fbdca 100644 --- a/crates/dpi-detector/src/i18n/ru.rs +++ b/crates/dpi-detector/src/i18n/ru.rs @@ -219,6 +219,8 @@ pub(crate) fn messages() -> Messages { cli_burst_profiles: "Fingerprint для теста 6: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [по умолчанию: all]", cli_burst_tls: "Версия TLS для теста 6: 1.2|1.3 [по умолчанию: 1.3]", cli_burst_alpn: "ALPN для теста 6: h2 (предлагает h2 с откатом на http/1.1)|http/1.1 (только http/1.1) [по умолчанию: h2]", + cli_trace: "Трассировка теста 6: строка на каждую попытку (раунд, отпечаток, домен, статус, деталь, мс); без пути — в stderr, с путём — в указанный файл [по умолчанию: выключено]", + trace_open_failed: "Не удалось открыть файл трассировки {}", cli_domains: "Путь к файлу со списком доменов", cli_tcp16: "Путь к файлу целей TCP16", cli_ascii: "Только ASCII для старых консолей (без Unicode-глифов и рамок)", diff --git a/crates/dpi-detector/src/i18n/zh.rs b/crates/dpi-detector/src/i18n/zh.rs index 051d3e2..6fd1e0c 100644 --- a/crates/dpi-detector/src/i18n/zh.rs +++ b/crates/dpi-detector/src/i18n/zh.rs @@ -219,6 +219,8 @@ pub(crate) fn messages() -> Messages { cli_burst_profiles: "测试 6 的 Fingerprint: all|rustls,custom(firefox133),chrome(chrome107),safari(safari155) [默认: all]", cli_burst_tls: "测试 6 的 TLS 版本: 1.2|1.3 [默认: 1.3]", cli_burst_alpn: "测试 6 的 ALPN: h2（提供 h2 并回退 http/1.1）|http/1.1（仅 http/1.1）[默认: h2]", + cli_trace: "测试 6 追踪: 每次尝试一行（轮次、指纹、域名、状态、详情、毫秒）；无路径写入 stderr，有路径写入该文件 [默认: 关闭]", + trace_open_failed: "无法打开追踪文件 {}", cli_domains: "自定义域名列表文件路径", cli_tcp16: "自定义 TCP16 目标文件路径", cli_ascii: "面向旧终端的纯 ASCII 输出（无 Unicode 符号或边框）", diff --git a/crates/dpi-detector/src/runner.rs b/crates/dpi-detector/src/runner.rs index 1fa78fa..ae9bb40 100644 --- a/crates/dpi-detector/src/runner.rs +++ b/crates/dpi-detector/src/runner.rs @@ -4,7 +4,7 @@ //! share one 1700-line file. use std::collections::HashSet; -use std::io::IsTerminal; +use std::io::{IsTerminal, Write}; use std::net::{IpAddr, SocketAddr}; use std::sync::Arc; use std::time::{Duration, Instant}; @@ -21,8 +21,8 @@ use crate::i18n::{legend_text, Language, Messages}; use dpi_core::net::fingerprint::TlsFingerprint; use dpi_core::net::netinfo::{detect_bypass_tools, fetch_public_ips, get_system_dns, is_tun_name}; use dpi_core::probe::burst::{ - burst_targets, BurstAlpn, BurstObserver, BurstSettings, BurstTarget, BurstTlsVersion, - BURST_DEFAULT_ATTEMPTS, BURST_DEFAULT_TIMEOUT_SECS, + burst_targets, BurstAlpn, BurstObserver, BurstProfileReport, BurstSettings, BurstTarget, + BurstTlsVersion, BURST_DEFAULT_ATTEMPTS, BURST_DEFAULT_TIMEOUT_SECS, }; use dpi_core::probe::cymru::{fetch_ip_cymru, IpCymruInfo}; use dpi_core::probe::domains::{ @@ -266,6 +266,128 @@ impl BurstObserver for BurstLine { } } +/// `--trace`: one line per attempt of test 6, written the moment the attempt +/// exists. +/// +/// The line is machine-readable on purpose — a UTC stamp, the round, the shape, +/// the host, the attempt, the status token, the detail code and the milliseconds +/// — so a round that fell over can be read back afterwards (`grep safari +/// trace.log`) or watched live in a second window (`Get-Content -Wait`) while +/// the screen redraws. It carries no translated text, so it reads the same in +/// every language, and it never touches stdout: the frame and the table stay +/// where they were. +struct TraceObserver { + out: std::sync::Mutex>, + round: std::sync::Mutex<(usize, usize)>, +} + +impl TraceObserver { + /// `None` when no trace was asked for, or when the file it named cannot be + /// opened — in which case the run continues untraced and says why. + fn new(target: Option<&str>, msg: &Messages) -> Option { + let target = target?; + let out: Box = if target.is_empty() { + Box::new(std::io::stderr()) + } else { + match std::fs::File::create(target) { + Ok(file) => Box::new(file), + Err(err) => { + eprintln!( + "{}", + msg.trace_open_failed + .replacen("{}", &format!("{target}: {err}"), 1) + ); + return None; + } + } + }; + Some(Self { out: std::sync::Mutex::new(out), round: std::sync::Mutex::new((0, 0)) }) + } + + fn line(&self, text: &str) { + if let Ok(mut out) = self.out.lock() { + // A failed write is not worth stopping a diagnostic run for. + let _ = writeln!(out, "{text}"); + let _ = out.flush(); + } + } +} + +impl BurstObserver for TraceObserver { + fn round_started(&self, fingerprint: TlsFingerprint, index: usize, total: usize, hosts: usize) { + if let Ok(mut round) = self.round.lock() { + *round = (index, total); + } + self.line(&format!( + "{} round {}/{} {} ({}) hosts={hosts}", + utc_clock(), + index + 1, + total, + fingerprint.display_label(), + fingerprint.code() + )); + } + + fn host_probed(&self, fingerprint: TlsFingerprint, domain: &str, report: &BurstProfileReport) { + let (index, total) = self.round.lock().map(|r| *r).unwrap_or((0, 0)); + for (attempt, result) in report.attempts.iter().enumerate() { + self.line(&format!( + "{} r{}/{} {:<8} {:<28} #{:<2} {:<12} {:<40} {:>5} ms", + utc_clock(), + index + 1, + total, + fingerprint.code(), + domain, + attempt + 1, + result.status.as_str(), + result.detail.code(), + result.ms, + )); + } + } +} + +/// UTC, because a timezone database is not worth carrying for a trace: the lines +/// only have to line up with each other and with the run. +fn utc_clock() -> String { + let secs = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap_or_default() + .as_secs(); + format!("{:02}:{:02}:{:02}Z", (secs / 3600) % 24, (secs / 60) % 60, secs % 60) +} + +/// The observers one run may have: the live line when the terminal can take one, +/// and the trace when `--trace` asked for it. Either may be absent, so neither +/// mode has to know about the other. +struct Observers<'a> { + line: Option<&'a BurstLine>, + trace: Option<&'a TraceObserver>, +} + +impl BurstObserver for Observers<'_> { + fn round_started(&self, fingerprint: TlsFingerprint, index: usize, total: usize, hosts: usize) { + if let Some(line) = self.line { + line.round_started(fingerprint, index, total, hosts); + } + if let Some(trace) = self.trace { + trace.round_started(fingerprint, index, total, hosts); + } + } + + fn host_finished(&self) { + if let Some(line) = self.line { + line.host_finished(); + } + } + + fn host_probed(&self, fingerprint: TlsFingerprint, domain: &str, report: &BurstProfileReport) { + if let Some(trace) = self.trace { + trace.host_probed(fingerprint, domain, report); + } + } +} + #[allow(clippy::too_many_arguments)] pub(crate) async fn run_test_suite( tests_str: &str, @@ -734,12 +856,12 @@ pub(crate) async fn run_test_suite( // live line that names the shape currently on the wire and how far the // run has got, and nothing else is printed until the table. let line = BurstLine { live: Arc::clone(&live), msg: *msg }; - let observer: &dyn BurstObserver = if args.json || !std::io::stderr().is_terminal() { - &() - } else { - &line + let trace = TraceObserver::new(args.trace.as_deref(), msg); + let observers = Observers { + line: (!args.json && std::io::stderr().is_terminal()).then_some(&line), + trace: trace.as_ref(), }; - let reports = burst_targets(cfg, &targets, settings, concurrency, observer).await; + let reports = burst_targets(cfg, &targets, settings, concurrency, &observers).await; if !args.json && std::io::stderr().is_terminal() { live.finish(); }