The round now presents one shape; a build that still resums four of its five
attempts must not be handed out as the same version as the one that does not.
A round reuses one `RustlsConnector` across its attempts, and the session-ticket
store lives in the `ClientConfig` that connector holds. As soon as the first
handshake completes and the server's ticket lands in that store, the later
attempts resume the session — and a resumed hello is not the shape the round
claims to measure: it grows a `pre_shared_key`, changes its extension set and its
length.
Measured with a listener that answers real handshakes and sends tickets
(`-d <name>.nip.io` + a local stand on port 443), a round of five:
* shared connector: attempt 1 = 517 bytes, 18 extensions, no PSK;
attempts 2-5 = 788 bytes, 19 extensions, `pre_shared_key` present.
* one connector per attempt: all five = 517 bytes, 18 extensions, no PSK.
The bundle cannot expose this: one curl process makes one connection, so its
client never resumes inside a round — which is why every byte comparison against
it passed while the round was quietly firing two different shapes. The recipe
(and the trap) is now step 7 of the verification note in
`examples/tls_fingerprint.rs`.
`create_tls_config` already documents the invariant this restores: a private
resumption store per connection is what keeps one probe's ticket out of the next
probe's ClientHello.
Sequential attempts answer the count question but not the rate one. The trigger
test 6 reproduces watches a *frequency* — several connection attempts inside a
window of a few hundred milliseconds — and a round that waits for each attempt to
finish before starting the next reaches that rate only on a fast link: the gap
between starts is the previous attempt's duration. Measured on the eight shipped
hosts, that gap runs 90–400 ms, so three consecutive starts span 220–750 ms —
half of them outside a 250 ms window before anything is even blocked.
Firing everything at one instant is the opposite error: every attempt is
established before the block lands, the whole round passes, and the block only
shows up in the next run.
The round now launches attempt k at k × 20 ms and lets all of them overlap. Five
attempts span 80 ms, inside any window a trigger would plausibly use, while the
block still has room to land *between* two starts — the attempt it lands on is
the verdict the report prints, in launch order. The round also costs about one
timeout again instead of N (the worst case was ~40 s at the defaults, now ~8.1).
`install.sh`, `install.ps1` and the README's release badge each carry the tag
they fetch, and the two scripts are what the release itself publishes next to
the binaries: an installer that names the previous tag downloads the previous
build, so the version moves in all four places at once (Cargo.toml, both
scripts, the badge).
Found by asking where else `v5.0.0-alpha.12` appeared rather than where the
version field was.
Test 6's default count moves from four to five. The value is now one above the
commonest throttle — a link that cuts the fourth connection — because at four the
run ends exactly where the answer starts: the fourth attempt is the one that
would have shown the block, and with a default of four there is nothing left to
ask after it.
Everything user-visible follows the number: the `--burst` help line in all four
languages and both README mentions (the flag's default, and the worst-case round
time, now ~40 s at the usual 8 s handshake timeout).
Two changes to test 6, both about what its number means.
The N attempts of one shape against one host used to be released together
through a barrier. That answers the wrong question: an ISP that starts dropping
after the third connection answers all N of a simultaneous round, so the block
the round itself caused is invisible until the *next* run, where the domain has
"suddenly" stopped answering. One at a time, the k-th attempt is asked after the
first k−1, and `M/N` reads as "this host answered M connections in a row and then
stopped" — the count of answers before the first refusal. Nothing else moves:
hosts still run concurrently (`-c`), and shapes are still fired one at a time
over the whole target list.
The cost is honest and documented: a round is N sequential attempts, so a host
that swallows packets silently spends N × `--burst-timeout` (plus the requests).
Test 6 also stops borrowing test 2's host list. The two tests ask opposite
questions of a domain — test 2 wants sites that are censored, test 6 wants sites
that are not, because a host that is already blocked fails every attempt for a
reason test 6 does not measure. Its list is now `burst-domains.txt` (embedded in
the binary): realmeteo.ru, ezgame.su, scpslgame.ru, kyles.ru, amserg.com,
smartape.ru, reg.ru, info.paymaster.ru. `-d` and `--domains` still pick the hosts
for a run; the configured `domains_file` stays test 2's, so a domains.txt next to
the binary can no longer put the censored-sites list under test 6.
`custom` was the name of a hand-built shape; the profile has been pinned to
`curl_firefox133` for a while and every table already labels it FIREFOX 133,
while the machine token the CLI, the config and `--json` accept still said
`custom`. It is `firefox` now, and the old spelling is rejected with the same
warning an unknown value gets rather than aliased — this build is pre-1.0 and a
silent alias would keep two names alive for one shape.
Test 6's TLS axis had two values with a misleading name: `1.3` sent the
*browser's own offer*, which is 1.3 **and** 1.2, and only required the answer to
be 1.3. Three values now say what goes on the wire:
* `1.3+1.2` (default) — the browser's offer, answering 1.2 is `NO TLS1.3`;
* `1.3` — a hello offering 1.3 alone, which is test 2's first column;
* `1.2` — a hello offering 1.2 alone, which is test 2's second column.
The 1.3-only axis is new and answers a question the other two cannot: whether a
middlebox treats a one-version offer differently from the browser's. Both pinned
axes are synthetic on purpose, and the screen cycles all three.
`live` reached `tls.peet.ws` with a hand-written HTTP/1.1 request over an
h2-negotiated connection: enough for the TLS hashes, but the service then has no
HTTP/2 frames to report and the akamai fingerprint — SETTINGS payload and order,
WINDOW_UPDATE, PRIORITY, pseudo-header order — stayed unmeasured. The report also
arrived as raw bytes behind a status line, so the four printed fields were found
by grepping the compressed-and-chunked body.
`peet <profile>` now runs the *probe's own* path: `HttpSender::handshake` with the
profile, so the preface, the request shape and the ordered header list are the
ones the tests put on the wire, and the report is parsed as JSON (status line
stripped, chunking unwrapped if present) and printed in full: the four TLS
hashes, the akamai fingerprint and hash, and every request header in order.
`liveany` covers the other configuration — the browser's own offer, which test
6's TLS 1.3 axis and tests 3/4 send — so an unpinned probe can be compared with a
bundle script run without version flags.
Measured against the bundle for `realmeteo.ru`, all three profiles agree on the
akamai fingerprint and on every header but one: ours sends
`accept-encoding: identity` where the bundles send `gzip, deflate, br` (`+ zstd`
for Firefox), which is the tool's deliberate choice — tests 2-4 count the bytes a
connection carries.
The byte comparison in the same run found two Firefox deviations that no hash
covers, now documented on `firefox_like`: the cert-compression list lacks zstd,
and the hello carries two key shares against the bundle's three.
The harness had the "run this after a rebase" story but not the method. Two ways
to compare, and what each one cannot see:
* Captured bytes. Point the bundle and a probe at a local listener, keep the SNI
identical, mask GREASE and per-connection randomness, then compare cipher order,
extension order with lengths, and every body. Take every expected value from the
*bundle's* capture, never from our dump — a constant copied from our own output
turns the test into a lock-in, which is how the Safari profile kept a duplicated
`rsa_pss_rsae_sha384` and the closing GREASE went without BoringSSL's byte.
* An echo service. Point *both* clients at it and compare like for like, offer the
same protocol (the ALPN value is JA4's first field), read each hash for what it
covers, and remember it reports the path's version of the connection. Never
conclude "identical" from it alone: the two bugs above were missed in the very
runs that called the Safari profile identical.
The pinned bundles send their last GREASE extension with a one-byte body
(`2300 01 00`) and their opening one empty; we sent both empty. The hello is
padded to 512 either way, so the missing byte only made our padding one byte
longer — one byte on the wire, invisible to JA3 and JA4, which is why no test
caught it.
It was caught by comparing test 6's Safari hello (TLS 1.3 + h2) against a
parameterless `curl_safari155` for `realmeteo.ru`, byte for byte: same 20 ciphers
in the same order, the same 14 extensions in the same order, every body equal —
the padding was the one entry that differed, and the GREASE markers explain it.
With the byte in place those two hellos are identical outside randomness: GREASE
values, the client random, the session id and the key share's public key.
`PATCH.diff` is regenerated for the new `hello_profile.rs` body and re-validated:
`patch -p1 --dry-run` against pristine 0.23.43 passes, and applying it reproduces
all seven touched files byte for byte — six against the blobs in git, the new one
against the working tree.
`curl_safari155.bat` names `rsa_pss_rsae_sha384` twice in its
`--signature-hashes` list, and BoringSSL de-duplicates it: the pinned bundle's
ClientHello carries 10 schemes. The profile was transcribed from the script
instead of from the wire, so it sent 11 — one extra entry in the extension JA4
hashes and JA3 does not, which is why every JA3 test passed while the pinned
JA4s were ours rather than the bundle's.
A byte comparison against the pinned bundle caught it: same cipher list, same
extension order, same 512-byte hello, but 24 bytes against 22 in the
signature-algorithms extension. With the duplicate gone, test 6's hello (TLS 1.3
+ h2, Safari profile) for `realmeteo.ru` matches the parameterless
`curl_safari155` outside randomness only: the same 20 ciphers in the same order,
the same 14 extensions in the same order, every body equal except the key
share's public key and the padding filler — and there ours runs one byte longer
because BoringSSL puts a zero byte in its closing GREASE extension where we send
it empty, which the padding absorbs.
The Safari JA4 constants move with the extension hash; the JA3s do not, since
JA3 never reads the signature list.
A column that pins TLS 1.3 rewrote `supported_versions` and nothing else, so
the hello advertised one version while still carrying the other's extensions
and cipher suites: Chrome 107 and Safari 15.5 kept `extended_master_secret`,
`renegotiation_info`, `ec_point_formats` and `session_ticket` plus the whole
TLS 1.2 cipher list, and a 1.2 hello kept `supported_versions`, ALPS and the
padding a hello that short has no use for.
That shape exists nowhere. Read off the `curl-impersonate v2.2.2` bundles pinned
the same way (`--tlsv1.3 --tls-max 1.3`, `--tls-max 1.2`) through a local
ClientHello sniffer, the bundles drop exactly those, and our pinned 1.3 hellos
are now byte-identical to theirs — same cipher list, same extension list in the
same order, same 512-byte size. `standby-rezka.tv` answers the hybrid with
`alert_illegal_parameter`; it answers the bundle's 200.
`fingerprint::pinned_drop` holds the per-profile table (Firefox keeps
`extended_master_secret` and `renegotiation_info` at 1.3, so it is not one rule
for all three), and `apply_profile` applies it through `suppress_extensions` —
an extension merely missing from `extension_order` is still emitted, only later,
and one the profile supplies verbatim would come from `raw_extensions` — plus a
cipher-suite filter. The 1.2 builder loses the padding extension too, matching
the bundles' 198/210-byte 1.2 hellos against our 185/199.
Two gaps stay documented: `compress_certificate` cannot be offered in a 1.2
hello (rustls offers it only when 1.3 is offered as well), so those constants
stop one extension short of the bundles, and Firefox's 1.3 hello still omits
ECH.
JA4's version field now falls back to the ClientHello's legacy version when a
hello carries no `supported_versions`: a 1.2-only browser hello is not a TLS 1.0
client, which is what the field said before. The harness gains `dump13`/`dump12`
so the pinned shapes can be dumped without a network.
`m.youtube.com` → `www.youtube.com` is one site spelling itself, but the rule
only compared the two hosts directly — equal, or one a subdomain of the other —
so two siblings under one parent read as a hop away from the site.
The site is now also taken as the host minus its first label, so
`m.youtube.com` and `www.youtube.com` agree on `youtube.com`. A two-label host
has no site in that sense (its parent is a TLD), which keeps
`youtube.com` → `example.com` foreign: reading `com` as a site would make every
`.com` host one site, and the test pins exactly that case.
The trade-off the label arithmetic accepts is documented on `site_of`: parents
whose labels belong to different owners (`user1.github.io`, `user2.github.io`)
are read as one site, which is the price of not carrying a suffix list.
README states the rule in the same three shapes, and gains a note on reading a
collapsed column: attempts that die at exactly our timeout for every shape at
once are the path, not the fingerprint.
The README had grown into working notes: the module tree, the i18n section that
named `Messages` and `legend_sections*()`, the fingerprint-profile table with its
JA3/JA4 pinning, the `tls.peet.ws` verification note, the rationale for test 6's
TLS axes, and the repository's own test and lint commands. A reader who wants to
install and run the tool does not need any of it.
What stays is what a user acts on: the one-line installers, the binary table, the
router notes with `DPI_UPX`, the CLI block and the test list, then test 6 with its
settings screen, the `M/N` and detail columns and the new `--trace` flag, the
profiles at the level of "which browser this imitates", the build commands and a
feedback footer.
The detail vocabulary had one hole: a rustls failure that is not a peer alert
reached the table as its own English sentence (`connection error | cannot decrypt
peer's message`) — the only `Detail` whose code is prose, so `--json` carried
that sentence and no test could pin it. `Detail::StackFailure(StackKind)` names
the six rustls failures a middlebox leaves behind (decrypt error, invalid /
inappropriate / inappropriate-handshake message, peer misbehaved, no application
protocol) through `classify/stack.rs`, the table that mirrors `alert.rs`. The
status stays `Unknown` on purpose: a decrypt error fits an injected record and a
buggy server alike, and the classifier still refuses to read "our stack lacks
this feature" as a peer failure.
Redirect details became values as well (`Redirect { host }`, `UpgradeHttps
{ status }`), which retires the last two `Other` sites that built display prose
inside the core, and the same-site rule is symmetric now: `m.holod.media` →
`holod.media` was a red REDIR while `www.holod.media` → `holod.media` was OK, so
a subdomain naming its main domain read as a redirection away from the site.
Test 6 gained `--trace [<PATH>]`: one line per attempt — UTC, round, shape, host,
attempt, status, detail code, milliseconds — to stderr or to a file. The file
form is what works under the TUI, and it is where a "whole column is red"
question gets answered. `BurstObserver` grew a defaulted `host_probed`, so the
live line and the trace are two observers of one run and neither knows about the
other; an observer that wants neither implements nothing.
README covers all three: the flag, the symmetric redirect rule, the grouped
detail column, and the HTTP/2 rows the fingerprint profiles no longer deviate on.
Test 6's detail column asked a host for its dominant failure, so a row whose 12
lost handshakes came back as two different errors was labelled with one of them —
`TLS ALERT ×8` standing for eight alerts and four resets. A reader cannot tell a
single middlebox from two, which is the whole point of the column.
`BurstReport::failure_counts` and `BurstProfileReport::failure_counts` now return
every failure grouped by status: commonest first, ties in the order the statuses
were first seen (a stable sort, so the same input renders the same row twice).
`dominant_failure` went with its last caller, and its two tests became the
grouping tests — one of them the reported case: twelve losses, two groups.
The view joins the groups with ` · `, the separator the live progress line
already uses, and keeps colouring the cell by the leading group. Grouping is by
status and not by `Detail`: a row is read as badges (rule 4), so two alerts of
different kinds are one `TLS ALERT` group, and the detail behind each attempt
stays in the `--json` payload, which is unchanged.
`·` also joined `asc_with`'s substitution table: it is not in CP866, and the live
line has been printing it into ASCII consoles unchanged.
h2 walks the settings it sends ascending by id, and the last profile gap was
that Safari does not: `curl_safari155` sends `4:4194304` before `3:100`, RFC 9113
§6.5 makes the order insignificant, and `tls.peet.ws` folds it into the akamai
fingerprint — so Safari's preface stayed distinguishable on the last point where
its shape was ours rather than the browser's.
`Settings` gained the order it encodes in: `set_order` names the ids to put
first, everything else follows ascending, so a caller can reorder the list but
never drop a setting from it, and an empty order bytes the frame exactly as
before. `client::Builder::settings_order` exposes it, hyper forwards it through
its new `settings_order` (see vendor/hyper/README-PATCH.md), and `H2Fingerprint`
pins it per profile: empty for Chrome and Firefox, `[4, 3]` for Safari.
All three browser profiles now send the same SETTINGS frames as the bundles they
imitate, in the same order — verified through the server's own report
(`tls.peet.ws/api/all`): every akamai fingerprint matches
`curl_chrome107`/`curl_firefox133`/`curl_safari155` exactly. What remains is the
Firefox ECH extension (no HPKE provider) and the deliberate
`accept-encoding: identity`.
hyper's HTTP/2 client builds the connection preface unconditionally: its
`new_builder` always calls h2's `max_header_list_size` and `enable_push(false)`,
so every hyper client advertises `SETTINGS_MAX_HEADER_LIST_SIZE` and
`SETTINGS_ENABLE_PUSH = 0`. The bundles do not agree on either — Firefox sends
the push setting but no header-list size, Safari neither — and `tls.peet.ws`
includes the settings in its akamai fingerprint, so Firefox and Safari profiles
stayed distinguishable on the first frame they sent.
No hyper API could say "leave this setting out": `max_header_list_size` takes a
plain `u32` and `enable_push` has no setter at all. `vendor/hyper` therefore
carries a 73-line patch (two files) making both options, with defaults that
reproduce upstream's wire exactly, so callers that touch neither setter — the
DNS/DoH paths, the rustls baseline — are unaffected. h2 needed no change: its
`Settings` fields are already `Option`s, and omitting `MAX_HEADER_LIST_SIZE`
leaves the receive-side cap at h2's own bounded default.
`H2Fingerprint` now pins both per profile and `HttpSender` passes them through.
Verified against the bundles' own reports (`tls.peet.ws/api/all`): Firefox and
Safari now send the same settings as `curl_firefox133` and `curl_safari155`;
Chrome was already identical and stays so.
An h2 client's request is part of its fingerprint and no ClientHello hash sees
it: JA3 and JA4 stop at TLS. Measured against the curl-impersonate v2.2.2 bundles
(decrypted with each bundle's SSLKEYLOGFILE), an HTTP/2 request to the same host
looks like this:
bundle HEADERS flags weight exclusive pseudo-headers
curl_chrome107 0x25 256 yes m,a,s,p
curl_firefox133 0x25 42 no m,p,a,s
curl_safari155 0x25 255 no m,s,p,a
h2 (unpatched) 0x05 - - m,s,a,p
The wrappers name two of the orders outright (`--http2-pseudo-headers-order`);
Chrome's is the bundle's default. h2 hard-codes its own order and 0.4 has no API
that puts a priority on a request's HEADERS frame, so vendor/h2 carries the same
kind of patch vendor/rustls does: `RequestShape` travels in the request's
extensions (the route h2 already uses for `ext::Protocol`) and the encoder writes
the five priority bytes through the `stream_dep` field the decoder already
parsed. hyper needs no patch — it forwards the extensions untouched.
`H2Fingerprint` now pins the shape beside the SETTINGS each profile already had,
and `HttpSender` attaches it to every request on the connection, so a run that
fires several profiles never mixes them. A request that carries no shape goes out
exactly as before: the rustls baseline, the DNS/DoH clients and anything else
resolving through the patched h2 are unaffected.
Verified against the bundles through the server's own view (`tls.peet.ws`):
JA3, JA4, peetprint, the akamai fingerprint and the request headers of our Chrome
profile are now identical to curl_chrome107's.
A peer that refuses a handshake says why in the alert's description, and the
report threw that away: every received alert became `fake_tls_alert`, whether the
server had refused the SNI, the version, the cipher list or a hello it could not
parse. The two alerts this network produces — Google's `unexpected_message` after
an h2 request and a front's `illegal_parameter` on the ClientHello — were
indistinguishable in `--json`.
`classify::alert` now carries all 36 IANA descriptions and the spellings the
stacks use for them: rustls prints its Rust name (`received fatal alert:
UnexpectedMessage`), OpenSSL the spaced or snake_case one (`sslv3 alert bad
certificate`, `tlsv1 alert unknown_ca`), some wrap it as
`alert(handshake_failure)`. Matching happens on the description with its
separators removed, so all of them reach the same value.
`Detail::Alert(AlertKind)` reports it as `alert_<iana name>`, and the display text
keeps the description Latin in every language (Rule 4). Three descriptions keep
the code the vocabulary already had for them (`handshake_failure`,
`unrecognised_name`, `protocol_version`), an alert that names nothing keeps
`fake_tls_alert`, and four generic codes that the named ones replace
(`handshake_alert`, `sni_alert`, `version_alert`, `tls_alert`) are gone — none of
them was documented. Both alert sites, `classify_ssl_error` and
`classify_connect_error_full`, now take the same verdict, which also makes a
`protocol_version` alert a block on the connect-error path as it already was on
the ssl one.
CI has been red on every push: `cargo test --workspace` on Linux could not
compile `net/sysinfo/os.rs`'s tests (`split_list` is `#[cfg(target_os =
"windows")]`) and clippy with `-D warnings` tripped over an unused `use super::*`
in `tui/backend.rs`, whose only test is `#[cfg(windows)]`. Both modules now carry
the same gate as the code they exercise, and
cargo clippy --target x86_64-unknown-linux-musl --workspace --all-targets -- -D warnings
— what the ubuntu job runs — is clean.
ClientHello profiles now carry all three layers taken from the same
curl-impersonate v2.2.2 wrapper — the TLS offer (Safari's 1.1/1.0 fallbacks
included), the UA and headers in the client's own order, and the HTTP/2 preface —
and test 6's attempts finish with the request instead of the handshake, on the
browser's own offer for the TLS 1.3 axis.
`curl_safari155` advertises `GREASE, 0x0304, 0x0303, 0x0302, 0x0301`; ours stopped
at 1.2, so Safari's hello was a shape no Safari sends, and `peetprint` — which
hashes the version list and the padding that JA4 does not — separated us from the
bundle even though JA3 and JA4 matched.
The vendored rustls patch gains `ClientHelloProfile::legacy_versions`, written into
`supported_versions` behind the configured ones, and the Safari profile lists TLS
1.1 and 1.0 there. Nothing about negotiation changes: rustls still speaks only
what the config offers, so a peer that selects 1.1 ends the handshake in
`PeerIncompatible::ServerDoesNotSupportTls12Or13` — which the classifier already
reads as `NO TLS1.3`, not as a block. Pinned runs (test 2's two columns, test 6's
TLS 1.2 axis) drop the fallbacks on the copy they build, because isolating one
version is the whole point of those runs.
Measured against `tls.peet.ws`: the Safari profile is now identical to
`curl_safari155` on JA3, JA4 *and* peetprint (`b2bafdc6…`, was `c8705f31…`), and
on the wire test 6 sends `GREASE, 0x0304, 0x0303, 0x0302, 0x0301`, 512 bytes.
`PATCH.diff` is regenerated from the crates.io source (741 lines, 7 files) and
validated with `patch -p1 --dry-run` against pristine 0.23.43; the copy it
replaces had missed the GREASE hooks added later, so the file now matches the
tree it claims to describe.
Test 6 reported how many handshakes were answered and stopped there, so a shape
that was answered and then cut, redirected or blocked the moment the request went
out read exactly like one that was fine — the behaviour the test exists to catch
was invisible.
The attempt now carries on: after the handshake the same profile sends `GET /`
over whatever protocol ALPN negotiated, and the response is judged by the code
test 2 has always used. That phase moves out of `domains.rs` into
`probe::http::check_http`, so both tests send the same request and mean the same
status when they report one — no second convention, no duplicated classification.
Consequences: a row counts as answered only when the handshake *and* the request
came back (`BurstProfileReport::answered`), and a shape that answers 1.2 to the
TLS 1.3 axis still reports `no_tls13` ahead of the request's own verdict, because
that explains why the run is not measuring 1.3 at all. Budgets stay where the
settings screen says they are: the configured timeout bounds the handshake, the
request waits `read_timeout` from `config.yml` (CLI help updated in all four
languages).
Measured against live hosts: `tls.peet.ws` and `example.com` are `ok,ok`;
`bit.ly`, whose handshake succeeds and whose `/` redirects to a foreign host, is
now `redir` with detail `→ bitly.com` — a verdict a handshake-only attempt could
not produce.
Test 6 asks whether a *browser shape* gets blocked, so the hello it puts on the
wire has to be the one a browser sends. The axis was pinned to TLS 1.3, and a
pinned config writes `supported_versions` from itself: the hello carried
`[GREASE, 0x0304]` where Chrome, Safari and Firefox carry
`[GREASE, 0x0304, 0x0303]`, and the padding that fills the hello to 512 bytes came
out two bytes short of the bundle's.
The 1.3 axis now builds the unpinned profile — the same offer tests 3 and 4 send —
so its hello is the browser's. Captured against `tls.peet.ws`: JA3
`cd08e31494f9531f560d64c695473da9`, JA4 `t13d1516h2_8daaf6152771_e5627efa2ab1`,
`supported_versions = GREASE, 0x0304, 0x0303`, all identical to `curl_chrome107`.
A peer that answers it with 1.2 now reports `no_tls13` instead of a plain success:
the handshake completed, but not with what the axis asks about.
The 1.2 axis stays pinned on purpose — a client offering both versions is never
answered with 1.2, so only a 1.2-only hello can ask whether such a handshake
survives. That is the same trade test 2's two columns make, and both stay as they
are.
Five doc comments in the core still called it test 7 — the number the legend
used to have. The UI, the CLI help and `main.rs` all say test 6 (the legend is
test 7), so the comments now say it too.
A `--fingerprint chrome` probe sent a `chrome107` ClientHello behind a
`Chrome/133` User-Agent, two headers (`user-agent`, `accept-encoding: identity`)
and hyper's default HTTP/2 preface: `SETTINGS` of 2 MiB / 16 KiB with no
HEADER_TABLE_SIZE and no MAX_CONCURRENT_STREAMS, a 5 MiB connection window, and
the `m,s,a,p` pseudo-header order. Measured against `tls.peet.ws` the TLS hashes
matched `curl_chrome107` while every header and the whole HTTP/2 fingerprint did
not.
The profiles now carry all three layers from the same `curl-impersonate v2.2.2`
wrapper the ClientHello is pinned to: the UA and header set in the impersonated
client's own order, and the `SETTINGS` values, which of them are sent, and the
connection window that becomes the `WINDOW_UPDATE` increment. `accept-encoding`
stays `identity` — tests 2–4 count the bytes a connection carries before it is
cut, and a negotiated Content-Encoding would tie those numbers to how well a
response happens to compress — and a `user_agent` set in `config.yml` still wins
over the profile's.
peet.ws now reports identical JA3/JA4, UA, header list and order, `SETTINGS` and
`WINDOW_UPDATE`. What still differs is the pseudo-header order and the priority
on the request `HEADERS` frame, neither of which h2 0.4 exposes; the profile
docs and the README say so, together with the version-pinned `supported_versions`
deviation.
The tolerating config layer stays: it accepts hand-written config.yml files that the
strict serde schema would reject (numeric strings, key case, nesting, unknown keys as
warnings) and it is covered by tests, including parity with config.yml. Removing it
would save ~300 lines and stop accepting configurations that work today. Decision
recorded in docs/REFACTORING.md, which now reads as a finished plan rather than an
open one.
`docs/REFACTORING.md` now lists every phase with its commit and the deviations that
were taken deliberately (P4/P4.5/P7 sharing a commit, P4.5 not shrinking the file,
the deleted blockpage list, the `Messages` diet, the eleven panics that remain with
their invariants written down). `README.md` and `AGENTS.md` describe the split
workspace: engine (protocols, probes, `Detail`), binary (i18n, `tui`, `views`, JSON
schema, CLI). Rule 6 moves the single source of interface text to the binary, and
rule 5 describes `detail` as the `Detail` code rather than a `DET_*` string.
`selection_flags` returned a nine-element tuple; swapping two flags at a call site
compiled silently, and `let (_, _, _, _, _, _, _, _, only_legend) = …` at three sites
said nothing about what it read. `TestSelection::parse` returns a struct with named
fields, the two runner call sites bind it once, and the tests now compare whole
structs instead of destructuring positionally.
`i18n` is presentation: which words a table header uses, how a badge is explained,
what the menu says. It lived in the engine crate, so the engine carried four language
tables and depended on them for nothing. It is now `crates/dpi-detector/src/i18n/`
(one file per language plus `messages.rs`, `mod.rs`, `details.rs`), and the core keeps
only what the wire needs: `PhaseId`, `ProgressBlock`, and `Detail` with its `code()`.
Three things the move needed: the binary depends on `winreg` now (the Windows display
language is an interface concern), `classify::detail::kb_display` is public so a
renderer can format a KB offset, and the doc comments in `config.rs` and `detail.rs`
say where the wording lives instead of linking into another crate.
Dead strings went with it: `Messages` had 213 fields, of which seven were read by
nothing (`banner_subtitle`, `stage`, `bytes`, `duration`, `bypass_tools`, `gateway`,
`detail_write_timeout`) and five only by the coverage test. That test is gone as well
— four struct literals already fail to compile when a field is missing, which is the
whole of what it checked, for a dozen fields by hand.
Checked live: `--legend --lang en` and `--lang fa` (Latin-only: no glyph outside the
box/arrow set).
`render.rs` (3013 lines) and `menu.rs` (1532) held everything from the Win32 console
FFI to the test tables. They are now a facade plus:
* `tui/backend.rs` — terminal modes and raw escape plumbing (`ascii_mode`,
`plain_mode`, `has_vt`, `clean_output`, `strip_ansi`, `output_str`, `frame_home`,
the Win32 FFI);
* `tui/widgets.rs` — boxes, colouring and width math, including `frame_repaint`:
the width math and the repaint path stay in one module, and `frame_repaint` is
still the only place a frame is drawn;
* `tui/progress.rs` — the live line, its spinner and the refresh timer;
* `tui/input.rs` — keyboard-layout normalisation (`nav_key`, `latin_key`);
* `tui/screens/{main,burst,legend,post_run}.rs` — the menu screens;
* `views/*.rs` — one file per test's report (banner, netinfo, dns, domains, tcp,
whitelist, telegram, burst, summary).
The internal markup layer is gone: the 16 `[green]…[/]`-style call sites write SGR
sequences directly (same escape for the same colour as the converter produced) and
`markup_to_ansi` no longer exists. The `"0.0.0.0".parse().unwrap()` comparison in
the DNS view became `Ipv4Addr::UNSPECIFIED`, so the binary has no `unwrap()` or
`expect()` outside tests.
Checked live in a pty: main menu, legend screen, and a full test-6 run with its
burst table.
Three phases landed together: they share `net/http_client.rs` and the new
`TlsProfile`, and neither half compiles without the other.
netinfo: `net/netinfo.rs` (988 lines) is now an 11-line facade over
`net/http_client.rs` (the HTTPS GET primitives), `net/public_ip.rs` and
`net/sysinfo/{os,bypass}.rs` (registry, adapters, route, resolver files). The Cymru
lookup moved to `probe/cymru.rs`, and that is what breaks the `net → dns` cycle:
`crates/dpi-core/src/net/` contains zero `crate::dns` references. The facade keeps
every name the binary imports, so no caller needed a new path.
TLS profiles: nine `create_*` factories and seven connector constructors collapse
into `TlsProfile { fingerprint, version, alpn, verify }` with
`create_tls_config(&TlsProfile)` and `From<TlsProfile> for RustlsConnector`. The
wire shape is untouched — `grease_version_leads_supported_versions`,
`bundle_versions_match_their_ja3` and `bundle_versions_match_their_ja4` pin it. The
file grew by 23 lines: the factory surface shrank, while the two crypto providers,
the insecure verifier and the pinning rationale stayed put.
Panics: `Semaphore::acquire().await.unwrap()` in twelve places became
`probe::permit(sem)`, which states the invariant once (nothing in this crate closes
a gate); `AppConfig::defaults()` replaces three
`serde_yaml::from_str("{}").unwrap()` in the config load path; one `resolved.unwrap()`
in the domain flow became a `let ... else`; `net/tls.rs` went from four `expect`s to
one, on `with_protocol_versions`, with the reason written next to it.
`i18n/mod.rs` was 1711 lines: the `Messages` struct, a 280-line literal per
language, four legend tables and the dispatch helpers. Now:
* `messages.rs` — the `Messages` struct and its language-neutral impls
(`phase_text`, `menu_test_label`, `config_warning`);
* `en.rs`, `ru.rs`, `zh.rs`, `fa.rs` — one language each: its `messages()` literal,
its legend table (the English one keeps the name `legend_sections`), and the
Farsi Latin-only test beside the block it guards;
* `mod.rs` (271 lines) — `Language`, `get_messages`, `legend_text`,
`fingerprint_label`, `fmt_speed`, `fmt_size`, the format helpers and the tests
that are not tied to one language.
No string, field name, field order or field count changed. Adding a language is now
one new file plus its `Language` variant.
The repository only had the release matrix, which runs on a tag. Every push now
runs build + test + clippy with warnings denied on the stable host toolchain (the
crates are pure Rust, so no cross toolchain is needed). No formatting check: the
tree is CRLF and rustfmt would rewrite nearly every file.
`runner.rs` built the document out of 14 `json!` literals scattered over 500 lines,
so the schema existed only as the union of them. `src/json.rs` now holds one struct
per test: `serde` writes the keys, `results` fields are `Option` and absent when a
test did not run, and the values come from the core's own `Serialize` impls —
`DpiStatus` as its snake_case token, `Detail` as its code. `TcpRow` serializes
directly instead of being copied into a literal.
`run_test_suite` no longer returns the map: its only caller discarded it (`let _ =
stats`). Verified against the reference dump: `-t 0`, `-t 2 -d example.com` and
`-t 6` produce the same key set as before, with the detail values now tokens.
Classification compared Russian prose: `is_detected` ran `detail.contains(" at ")`,
whitelist ban detection re-checked the same marker, `build_domain_row` tested
`eq_ignore_ascii_case("timeout")`, and `i18n` re-derived the parts by splitting the
string on `" at "` and stripping `"KB"`. `Detail` replaces the `DET_*` constants with
one enum, so:
* classification matches variants (`matches!(detail, Detail::AtKb { .. })`);
* `code()` is the machine token `--json` carries — snake_case, never translated:
`tls_drop_handshake`, `read_timeout_at_24kb`, `timeout_reading_data`, `http_403`,
`elapsed_300ms`;
* `i18n::detail_text` is an exhaustive `match`, so a new variant does not compile
until all four languages have their wording. Russian is spelled out in that table
now instead of travelling as the canonical string, which also folded the two
spellings of "TCP RST on ClientHello" into one variant;
* `ALL_DET_DETAILS` and the runtime coverage test are gone: exhaustiveness is the
guard. `classify/constants.rs` disappears with them.
The domain row's detail cell is typed too: `build_domain_row` returns
`Vec<(Option<&'static str>, Detail)>` (the protocol tags stay Latin) and the table
renders it, instead of the core building a `"T1.3:..."` string that the renderer had
to parse back. Free text from the OS or the TLS stack keeps its words
(`Detail::Other`), and `Messages::detail_timeout_word` went away — the timeout word
is a variant now.
`dns/` is the wire layer: build a query, carry it over UDP/DoH/DoT/SOCKS5, parse the
answer. `dns::availability` is test 1 — it judges resolvers, knows about poisoning
and hijacking — and it was the only part of `dns/` reaching up into `probe` (for
`fake_ip_type`) and into `classify` (for the verdicts). It becomes `probe::dns_avail`;
the binary's nine import sites follow, and the module graph loses both edges.
A library crate cannot see a dead `pub` item, so the list came from an audit and
every removal was checked by the compiler:
* `dns/mod.rs` re-exported 30 names; only `query_doh_txt`, `resolve_host` and
`parse_socks_proxy` are used through the flat path — callers spell out
`dns::availability::…`, `dns::udp::…`. The others are gone.
* `probe::connector`: `new_insecure`, `new_verifying` and the argument-less
`new_insecure_tls13`/`_tls12` had no caller (only the `_with` variants are used,
by `domains.rs` and `burst.rs`), and `DpiTlsConnector`/`RustlsConnector` are named
only inside `dpi-core` — both demoted to `pub(crate)`.
* `DpiProbeStream::{into_inner, get_ref, get_mut}`, `QTYPE_{NS,SOA,PTR,MX}` and
`Language::{code, name, is_rtl}`: no caller anywhere.
* `profile`: `default_resolvers`, `default_doh` and `blockpage_signatures` were read
only by a test asserting they are non-empty, and a test that only checks a list is
non-empty pins nothing. `default_domains` stays — it is what the CLI uses when no
`-d` is given. With `probe/tls.rs` gone, no path looked at the blockpage markers
either; they live in git history until something reads them.
* `net::cert_compression::covers` guards the ClientHello profiles from a test and is
called by nothing on a live connection: `#[cfg(test)]`.
* `create_insecure_dpi_tls_config_tls13/_tls12` stay `pub`: `examples/tls_fingerprint.rs`
is a separate crate and consumes them.
* `#![warn(unreachable_pub)]` in the crate root so the surface cannot creep back.
`net::version` is an application-level service: it talks to the GitHub Releases
API, renders a localized badge, and it was the only core module that used `i18n`
at all. It becomes `dpi-detector::update`; `net/mod.rs` loses the module and the
two consumers import it as `crate::update`.
While moving: `ReleaseInfo::tag` was read by nothing but the tests (`version` is
derived from it), and `version_badge` was a Russian-language wrapper around
`version_badge_lang` that only its own test called. Both gone; the banner badge
of the rebuilt TUI was checked by hand.
`probe::tls` (196 lines) had no caller — test 2 grew its own L7 path — and the
blockpage signatures it checked are already kept per region in
`RegionProfile::blockpage_signatures`, so the module only duplicated knowledge and
kept nine names alive in the public surface. Deleted with its two re-exports.
The remaining raw dials (DoH, DoT, SOCKS5, the netinfo GET and the three Telegram
paths) now go through one `net::tcp::set_no_delay`, so the rule that `net/tcp.rs`
states — Nagle off is a requirement of the exchange, not a tuning knob — holds for
every dial instead of four of ten.
Every phase carries the evidence that motivated it (the three import cycles, the
detail strings the classifier parses, the hand-rolled JSON, the dead public
surface), its cost and the check that closes it — plus the proposals that were
measured and rejected, so they are not raised again. Kept out of README: it is a
working document, not user-facing documentation.
Roughly 150 comment blocks across both crates, two lines in config.yml and one in
docs/OPTIMIZATIONS.md pointed at a prototype that is not part of this repository:
a "mirrors `_foo`" note is unverifiable for anyone reading the tree. Each one now
states what the Rust code does — thresholds, stage tokens, timeouts and registry
keys kept verbatim — and nothing but comments changed, apart from two names that
were themselves the reference: the summary markup converter (`markup_to_ansi`) and
one test (`netinfo_panel_matches_expected_rows`).
The project is 5.0.0-alpha and still moving, so the docs now describe the shape
the code produces today and say how to change it (one commit: code + tests +
README), instead of declaring it immutable. Also drops the "source language"
phrasing around the DET_* details and the claim that a status token is frozen,
and refreshes the CLI block, the measured binary sizes and the workspace layout
so the README matches the binary it ships with.
The menu and the settings screen each carried their own copy of the panel
assembly — border glyphs, the title pad and the per-row pad to the right edge —
while `render::panel_with` already does exactly that (and the netinfo and legend
panels were drawn with it). Both screens now hand their body rows to
`panel_to_string`, so the padding rule that `frame_repaint` and the CJK-console
notes describe lives in one place. The visible output is unchanged: the only
byte difference is a redundant border-colour re-arm before the right corner.
`get_test_options` was a second literal copy of the digit-to-test table that
`Messages::menu_test_label` already holds; it is now derived from it, so adding
or renaming a test touches one table.
Test 2 pins one version per phase, and rustls writes `supported_versions` from
the config, so a browser profile advertises `[GREASE, 0x0304]` where the Chrome
107 it imitates sends `[GREASE, 0x0304, 0x0303]` -- `[0x0304]` against
`[0x0304, 0x0303]` for Firefox, which greases nothing.
JA3 and JA4 cannot see it (they hash extension codes and the maximum version),
so the pinned bundle hashes still match, but a middlebox reading the body can.
It is kept: one hello per stack is what makes the TLS 1.3 column a TLS 1.3
verdict, since a hello offering both would let the server choose. Written next
to the builders that pin, and asserted in the test that pins the lists, so it
cannot drift unnoticed either way.
Second pass over the review's remaining claims.
`net::tcp::dial_tcp` replaces the open-coded
`timeout(.., TcpStream::connect(..)).await` + `set_nodelay` at the three
match-style dial sites (domains twice, tcp16, burst), and it says *why* the
socket is unwrapped: every probe writes one small message and then waits for the
answer, so Nagle would hold the write back. The two `?`-style dials (DoH, DoT)
stay inline on purpose: each maps the two failures to its own `DnsError` at the
call site, and routing them through the helper made that mapping longer, not
shorter.
The three target cascades in `main` (CLI -> configured file -> embedded ->
default) become `load_domains`, `load_tcp16_targets` and
`load_whitelist_sni_list`, one per list. A single generic helper over three
closures was rejected: the three differ in their fallbacks and their types, and
the generic form hides that behind a type parameter.
`get_system_dns` and `detect_bypass_tools` shell out on Windows (`route print`,
`tasklist`, 5 s caps) and read the registry. They ran inline in the run's async
task, so on the single-threaded runtime a slow helper stalled the worker — and
the badge poll with it. Both now run on the blocking pool, in parallel.
Verified: 107 + 34 tests, clippy --all-targets 0 warnings, live `-t 0`, `-t 2`
and `-t 6` runs emitting valid JSON, and a pty session whose Cyrillic keys
navigate and quit. Release-local binary: 5,725,184 -> 5,720,576 bytes.
A review pass (Gemini) claimed a list of duplications, a keyboard-layout bug in
the burst-settings screen and a diverged timeout classification. Verified each
against the source; this is the part that held up.
Bugs:
* Burst settings answered only Latin w/s/a/d/q. The menus answer the physical
key on six layouts by listing each layout's characters, and this screen
simply had no such list -- so a user in a Cyrillic, Persian, Bopomofo or
Hebrew layout could not move the cursor there. Key handling now resolves once
(`nav_key`: IME width, then the Latin key at the same physical position) and
every screen uses it; the per-layout character lists are gone. The domain box
is exempt on purpose: inside it a letter is text.
* `classify_connect_error_full` classified timeouts twice. The message-text
branch knew the send/read stages, the OS-code branch did not, so a
WSAETIMEDOUT (10060) while reading was reported as a bare `Timeout` instead of
`ReadTimeout` -- on Windows, whose error text carries no "timeout". Both
branches now return `timeout_at_stage(stage)`.
Duplication, all verified present:
* DNS A/AAAA extraction: four copies -> `DnsResponse::collect_ips`.
* hyper io::Error unwrapping: `hyper_err_info` (domains) and `hyper_info`
(tcp16) -> one `probe::http::hyper_err_info`.
* ASN display: `asn_display_of` (core) and a copy in the run loop ->
`Tcp16Target::display_asn`.
* config list parsing: file and embedded variants of both the domain and the
SNI lists -> `parse_domains` / `parse_whitelist_sni`.
* box glyphs: three copies -> `render::box_chars`.
* winreg REG_SZ/REG_MULTI_SZ reads: four copies -> `netinfo::reg_strings`.
Cost: `create_verifying_tls_config` and the DoH variant rebuilt the WebPKI root
store and its verifier index on every call -- once per DoT connection and per
DoH request, ~70 times in a DNS run. Both are immutable and `Arc`-shared, so
they are built once.
Structure: main.rs 1778 -> 640 lines. `runner.rs` takes the run orchestration
(`run_test_suite` and the burst plan), `menu.rs` takes the key handling and the
post-run/legend screens, `terminal.rs` takes the Win32 console probe -- the only
`unsafe` in the crate, now alone in its file. The single-threaded runtime keeps
its `current_thread` flavour, with the reasoning (docs/OPTIMIZATIONS.md §2.5,
~50-120 KB, 1-2 core routers, auto_gate) written down at the attribute, since
the review read it as an oversight.
Verified: 107 + 34 tests, clippy --all-targets 0 warnings, a live `-t 3 --json`
run, and a pty run of the TUI -- Cyrillic keys move the menu and the burst
screen (they did not before), a full 35-domain x 4-profile test 6 run printed its
table, and the domain box still inserts Latin text without transliterating.
rustls's `brotli`/`zlib` features implement both halves of RFC 8879 and publish
each as a `&'static dyn` constant, so `default_cert_compressors()` kept brotli's
encoder linked in a client that only ever decompresses what the server sent:
241 KiB of encoder code and 384 KiB of encoder tables (`logs_16`,
`kStaticDictionaryBuckets`, `kStaticDictionaryHash`) in the release build, plus
zlib-rs's deflate half and rustls's own compression glue.
The decompressors are now ours (`net::cert_compression`), over the decoder-only
crates, and the rustls features are off. The advertised extension 27 and its
algorithm list are the profile's business and are untouched; what the profile no
longer does is carry code it cannot reach.
x86_64: 4,908,032 -> 4,008,960 (-878 KiB, -18.3%)
mipsel: 5,591,792 -> 4,625,044 (-944 KiB, -17.3%, stripped)
The brotli dictionary (122,784 B) stays: the decoder needs it. Verified live on
cloudflare.com, which answers the Chrome and Firefox profiles with a
CompressedCertificate (brotli, 1821 -> 2571 B) — decoded and handshaked by the
new implementation.
