feat: implement pending Hello cache to handle split ClientHello segme… (#281)

This commit is contained in:
Daniel Lavrushin 2026-07-30 21:55:12 +02:00 committed by GitHub
parent 52f9d8e1f4
commit 2e87634fd3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
37 changed files with 2503 additions and 155 deletions

View file

@ -1,7 +1,19 @@
# B4 - Bye Bye Big Bro
## [1.74.0] - 2026-07-2x
## [1.74.0] - 2026-07-30
- FIXED: **The watchdog reported a site as healed while it was still unreachable** - a healing run crowned a strategy after one successful fetch, and against a filter that blocks intermittently one of the twenty strategies it tries lands in a gap by chance. That result went straight into the set and the domain was marked healthy, with nothing checking whether ordinary traffic to the site worked afterwards. A healed strategy has to reproduce the fetch several times and survive a re-check through the live engine, or the configuration is put back.
- FIXED: **A healing run could quietly replace a hand-tuned set** - it overwrote the whole tcp, udp, fragmentation and faking section while logging only the fragmentation strategy name, so a switch between two combo variants read as "combo -> combo" and every other tuned value disappeared without a trace.
- FIXED: **A fetch that returned nothing counted as a working strategy during discovery** - a response without a declared length was accepted as complete even when zero bytes of the page arrived.
- FIXED: **The fake packet sent before a blocked greeting did not resemble the greeting it shadowed** - it carried whatever length the fake payload happened to be, in one common case more than twice the real greeting, leaving a filter that reassembles the connection an untouched copy of the real bytes to read. Per-set `fake_len_mode: "match"` sizes it to the greeting.
- FIXED: **The fake's TTL was ignored unless the faking strategy was set to "ttl"** - a set could carry a TTL value while its fake still went out at the full system TTL and reached the far end. Per-set `apply_ttl` applies it alongside any faking strategy.
- ADDED: **The TCP MD5 option on the fake greeting itself** - `md5_on_fake` makes the far end discard the fake while a filter in between still reads it. The option only ever went on a separate fake connection opening, and only alongside a fragmentation strategy.
- CHANGED: **The fake and the real packet no longer share an IP identification number** - one field gave away that both came from the same source.
- ADDED: **Fake-only discovery strategies** - three built around a low-TTL fake with no fragmentation, plus MD5-signed variants of the TTL sweep. None of the sixteen strategies discovery started from used a TTL fake, so against a filter that reassembles TCP well enough to see through every split, discovery had nothing that could work.
- CHANGED: **The TLS split position can follow the domain name** - `middle_sni` on a set using the `tls` fragmentation strategy places the split inside the domain name instead of a fixed offset from the record header.
- ADDED: **The STUN fake payload ships with b4** - `tls_stun.bin` is built in and picked as "Preset: STUN" under `Fake Payload Type`. It had to be uploaded as a capture file on every device, and a set shared with someone else arrived pointing at a `.bin` they did not have.
- ADDED: **Import of a shared set said nothing when the set referenced a payload file the device does not have** - the set pointed at a `.bin` capture that only existed on the machine it came from, the fake packets silently fell back to a built-in payload, and the set behaved differently than for the person who shared it. The import screen names the missing files and links to the Payloads settings.
- FIXED: **b4 would not start on an OpenWRT router whose kernel has no packet-queue module** - startup ended with a raw firewall error that pointed nowhere near the missing kernel module. [#275](https://github.com/DanielLavrushin/b4/issues/275)
- FIXED: **The installer reported the router as ready when it could not run b4 at all** - the check passed if any one of three queue modules was present, so a router carrying the wrong one installed cleanly and failed at first start.
- FIXED: **The diagnostics report showed no queue number, worker count or queue mode** - it read them from config keys b4 has never written, so those lines were skipped on every router.
@ -11,6 +23,10 @@
- FIXED: **Telegram on Android could sit at "Connecting" when a set routed it through the built-in Telegram bridge** - the bridge closed connections that stayed silent for five seconds, though Telegram opens them before it has anything to send. [#277](https://github.com/DanielLavrushin/b4/issues/277)
- FIXED: **The "already exists in other sets" warning missed overlaps that do change routing** - it compared entries character for character while traffic is matched by domain suffix, so `example.com` in another set drew no warning for `www.example.com`. [#273](https://github.com/DanielLavrushin/b4/issues/273)
- FIXED: **The DC Relay socat helper generated commands that pointed at the wrong Telegram servers** - it built them from Telegram's published proxy list, whose addresses drop the connection right after the handshake.
- FIXED: **The first request to a domain after an idle spell could hang for twenty seconds or time out, and YouTube on Android could sit for minutes before its interface appeared** - when a client sent a TLS ClientHello too large for one packet, the domain name could land in the second one; b4 read each packet on its own and never joined them, so the set did not match and the connection went out with no bypass strategy at all. Chrome reorders the fields inside that greeting on every connection, which is why the same domain failed on one attempt and loaded on the next. [#279](https://github.com/DanielLavrushin/b4/issues/279) [#280](https://github.com/DanielLavrushin/b4/issues/280)
- FIXED: **The first connection to an address b4 had just resolved for a client went out without the set that owned the domain** - nothing linked a DNS answer, or a domain seen in a rejected QUIC attempt, to the connection that followed it, so until some connection produced a readable domain name the traffic was treated as unknown. That gap was the one place where a domain b4 had already seen went unrecognised, and on Android YouTube it could leave the app retrying for minutes. The link is remembered per client and destination and is discarded when two domains on one address belong to different sets. [#279](https://github.com/DanielLavrushin/b4/issues/279)
- FIXED: **One device could decide which set another device's traffic went through** - the domain learned for a shared CDN address was stored once for the whole network and overwritten by whichever device connected last, so on an address serving both YouTube previews and video, one phone's choice was applied to everyone else. Evidence a device produced itself, from its own DNS answers or QUIC attempts, is preferred over the shared record. [#279](https://github.com/DanielLavrushin/b4/issues/279)
- FIXED: **A plain connection opening handshake was taken apart and re-sent for no reason** - any set with fragmentation or fake-SNI turned on pulled every packet without content out of the kernel and pushed it back through a raw socket, including the opening SYN, although none of those techniques act on a packet that carries no data.
- FIXED: **A set built from a large geosite category took several times more memory than its domains** - reading a few categories out of a 51 MB file needed around 90 MB, and the matcher and a switched-off SOCKS5 server each kept a spare copy of every domain.
## [1.73.0] - 2026-07-05

View file

@ -1,7 +1,19 @@
# B4 - Bye Bye Big Bro
## [1.74.0] - 2026-07-2x
## [1.74.0] - 2026-07-30
- ИСПРАВЛЕНО: **Вотчдог сообщал, что сайт вылечен, пока тот оставался недоступен** - лечение объявляло стратегию рабочей после одной удачной загрузки, а при фильтрации, которая срабатывает не всегда, одна из двух десятков перебираемых стратегий случайно попадает в паузу. Такой результат сразу уходил в сет, домен помечался здоровым, и ничто не проверяло, работает ли после этого обычный трафик к сайту. Вылеченная стратегия обязана повторить загрузку несколько раз и пройти перепроверку через рабочий движок, иначе конфигурация возвращается назад.
- ИСПРАВЛЕНО: **Лечение могло незаметно заменить вручную настроенный сет** - оно переписывало целиком разделы tcp, udp, фрагментации и подделки, а в лог выводило только название стратегии фрагментации, поэтому переход между двумя вариантами combo читался как «combo -> combo», и все остальные настроенные значения исчезали бесследно.
- ИСПРАВЛЕНО: **Загрузка, не вернувшая ничего, засчитывалась при дискавери как рабочая стратегия** - ответ без объявленной длины принимался за полный, даже если не пришло ни одного байта страницы.
- ИСПРАВЛЕНО: **Поддельный пакет перед заблокированным приветствием не походил на то приветствие, которое прикрывал** - он нёс ту длину, какая оказалась у поддельной нагрузки, в одном частом случае более чем вдвое превышая настоящее приветствие, из-за чего фильтру, собирающему соединение, оставалась нетронутая копия настоящих байт. Параметр сета `fake_len_mode: "match"` подгоняет подделку под приветствие.
- ИСПРАВЛЕНО: **TTL подделки игнорировался, если стратегия подделки не была равна «ttl»** - сет мог нести значение TTL, а его подделка всё равно уходила с полным системным TTL и доходила до другого конца. Параметр сета `apply_ttl` применяет его вместе с любой стратегией подделки.
- ДОБАВЛЕНО: **Опция TCP MD5 на самом поддельном приветствии** - `md5_on_fake` заставляет другой конец отбросить подделку, пока фильтр по пути всё равно её читает. Раньше опция ставилась только на отдельное поддельное открытие соединения и только вместе со стратегией фрагментации.
- ИЗМЕНЕНО: **Подделка и настоящий пакет больше не уходят с одинаковым идентификатором IP** - одно поле выдавало их общее происхождение.
- ДОБАВЛЕНО: **Стратегии дискавери из одной подделки** - три штуки, построенные на подделке с низким TTL и без фрагментации, плюс варианты с подписью MD5 для перебора TTL. Ни одна из шестнадцати стратегий, с которых начинало дискавери, не использовала подделку по TTL, поэтому против фильтра, который собирает TCP достаточно хорошо, чтобы видеть сквозь любое разбиение, у дискавери не оставалось ничего работоспособного.
- ИЗМЕНЕНО: **Позиция разреза TLS может следовать за именем домена** - `middle_sni` у сета со стратегией фрагментации `tls` ставит разрез внутри имени домена, а не на фиксированном смещении от заголовка записи.
- ДОБАВЛЕНО: **Фейковый payload STUN входит в состав b4** - `tls_stun.bin` встроен и выбирается как «Пресет: STUN» в типах фейкового payload. Раньше его приходилось загружать как capture-файл на каждом устройстве, а сет, переданный другому человеку, приходил со ссылкой на `.bin`, которого у того нет.
- ДОБАВЛЕНО: **При импорте чужого сета ничего не сообщалось о том, что сет ссылается на файл payload, которого нет на устройстве** - сет указывал на `.bin`, существовавший только на машине автора, фейковые пакеты молча собирались из встроенного payload, и сет вёл себя не так, как у того, кто им поделился. Экран импорта называет отсутствующие файлы и ведёт в настройки Payloads.
- ИСПРАВЛЕНО: **b4 не запускался на роутере OpenWRT, в ядре которого нет модуля очереди пакетов** - запуск обрывался сырой ошибкой фаервола, по которой отсутствующий модуль ядра было не найти. [#275](https://github.com/DanielLavrushin/b4/issues/275)
- ИСПРАВЛЕНО: **Установщик сообщал, что роутер готов, хотя b4 на нём вообще не мог работать** - проверка считалась пройденной при наличии любого из трёх модулей очереди, поэтому роутер с неподходящим модулем устанавливался без замечаний и падал при первом запуске.
- ИСПРАВЛЕНО: **Отчёт диагностики не показывал номер очереди, число рабочих потоков и режим очереди** - эти значения читались из ключей конфигурации, которых b4 никогда не записывал, поэтому строки пропускались на любом роутере.
@ -9,6 +21,10 @@
- ИСПРАВЛЕНО: **Ядро без счётчиков пакетов соединения вообще не давало b4 запуститься** - правило очереди отклонялось целиком и запуск падал из-за оптимизации, без которой можно было обойтись.
- ИЗМЕНЕНО: **«Сведения о системе» показывали не те модули ядра на роутерах с одним nftables** - раздел отмечал как отсутствующие модули эпохи iptables и ничего не говорил о модулях nftables, от которых b4 там зависит.
- ИСПРАВЛЕНО: **Telegram на Android мог висеть в состоянии «Подключение», когда сет направлял его во встроенный мост Telegram** - мост закрывал соединения, молчавшие пять секунд, хотя Telegram открывает их заранее, ещё не имея данных для отправки. [#277](https://github.com/DanielLavrushin/b4/issues/277)
- ИСПРАВЛЕНО: **Первый запрос к домену после простоя мог висеть двадцать секунд или завершаться тайм-аутом, а YouTube на Android мог несколько минут не показывать интерфейс** - когда клиент отправлял TLS ClientHello, не поместившийся в один пакет, имя домена могло оказаться во втором; b4 разбирал каждый пакет отдельно и никогда их не соединял, поэтому сет не совпадал и соединение уходило вообще без стратегии обхода. Chrome меняет порядок полей внутри этого приветствия на каждом соединении, поэтому один и тот же домен то не открывался, то загружался со следующей попытки. [#279](https://github.com/DanielLavrushin/b4/issues/279) [#280](https://github.com/DanielLavrushin/b4/issues/280)
- ИСПРАВЛЕНО: **Первое соединение к адресу, который b4 только что разрешил для клиента, уходило без сета, которому принадлежал домен** - ни DNS-ответ, ни домен из отклонённой попытки QUIC ничем не связывались с последующим соединением, поэтому до первого соединения с читаемым именем домена трафик считался неизвестным. Это было единственное место, где уже известный b4 домен оставался неопознанным, а на Android YouTube приложение могло из-за этого несколько минут повторять попытки. Связь запоминается отдельно для каждой пары клиент-адрес и отбрасывается, если два домена на одном адресе принадлежат разным сетам. [#279](https://github.com/DanielLavrushin/b4/issues/279)
- ИСПРАВЛЕНО: **Одно устройство могло решать, через какой сет пойдёт трафик другого устройства** - домен, выученный для общего CDN-адреса, хранился в единственном экземпляре на всю сеть и перезаписывался тем устройством, которое подключилось последним, поэтому на адресе, отдающем и превью YouTube, и видео, выбор одного телефона применялся ко всем остальным. Доказательства, которые устройство получило само - из своих DNS-ответов или попыток QUIC - имеют приоритет над общей записью. [#279](https://github.com/DanielLavrushin/b4/issues/279)
- ИСПРАВЛЕНО: **Обычное начало соединения без необходимости разбиралось и отправлялось заново** - любой сет с фрагментацией или fake SNI забирал из ядра каждый пакет без содержимого и отправлял его обратно через raw-сокет, включая первый SYN, хотя ни одна из этих техник не работает с пакетом без данных.
- ИСПРАВЛЕНО: **Предупреждение «Уже существует в других сетах» пропускало пересечения, которые влияют на маршрутизацию** - записи сравнивались посимвольно, тогда как трафик сопоставляется по суффиксу домена, поэтому `example.com` в другом сете не давал предупреждения для `www.example.com`. [#273](https://github.com/DanielLavrushin/b4/issues/273)
- ИСПРАВЛЕНО: **Помощник socat для DC Relay выдавал команды, указывающие на неправильные серверы Telegram** - он строил их из опубликованного списка прокси Telegram, адреса которого обрывают соединение сразу после handshake.
- ИСПРАВЛЕНО: **Сет, собранный из большой категории geosite, занимал в несколько раз больше памяти, чем его домены** - чтение нескольких категорий из файла в 51 МБ требовало около 90 МБ, а матчер и выключенный сервер SOCKS5 держали каждый свою лишнюю копию всех доменов.

View file

@ -63,7 +63,13 @@
"tcp.syn_fake": "Sends a fake SYN packet BEFORE the real SYN that opens the connection. The fake SYN looks like a normal connection attempt to DPI but is discarded by the real server (low TTL or other tricks similar to fake-SNI). Goal: prime DPI's per-flow state at connection setup, before any TLS payload is exchanged. Pairs with syn_fake_len (the fake's payload size, 0 = no payload) and syn_ttl. Useful when DPI decides whether to inspect a flow based on what it sees in the early SYN exchange.",
"faking.tcp_md5": "Adds the TCP MD5 signature option (RFC 2385) to fake packets b4 emits. Real servers without an MD5 key configured silently discard MD5-signed packets — the option is rare outside BGP. Some DPI engines accept the MD5-signed fakes into their per-flow state because the option looks 'authentic', then discount subsequent unsigned packets (the real ClientHello) as inconsistent. Has no effect outside the fake-emission paths (fake-SNI, syn_fake) — does not modify the real outgoing data. Combine with syn_fake or fake-SNI for the full effect.",
"faking.tcp_md5": "Sends an extra fake SYN carrying the TCP MD5 signature option (RFC 2385) before the real handshake. Real servers without an MD5 key configured silently discard MD5-signed packets — the option is rare outside BGP. Some DPI engines accept the MD5-signed fake into their per-flow state because the option looks 'authentic', then discount subsequent unsigned packets as inconsistent. This option affects the connection-opening fake only; to put MD5 on the fake ClientHello itself use faking.md5_on_fake. Does not modify the real outgoing data.",
"faking.md5_on_fake": "Puts the TCP MD5 signature option (RFC 2385) on the fake ClientHello itself, rather than on a separate fake SYN. The server discards the signed fake because it has no MD5 key configured, while DPI in the middle still reads it and takes its contents into the flow's state. This is the option byedpi exposes as -S, and it is usually paired with a low fake TTL and faking.fake_len_mode 'match' so the fake both dies in transit and covers exactly the bytes the real ClientHello occupies. Requires faking.sni.",
"faking.apply_ttl": "Applies faking.ttl to the fake packet regardless of which faking strategy is selected. Without it the TTL is only written when faking.strategy is 'ttl', so a set could carry a TTL value while its fake still left at the full system TTL and reached the far end. Turn it on to combine a low-TTL fake (which expires before the server) with a sequence or timestamp trick in the same set. The value is clamped below the real packet's TTL and never drops under 1.",
"faking.fake_len_mode": "Controls how long the fake packet is. Default ('') emits the fake payload at whatever size it happens to be, which can be several times the size of the real ClientHello. 'match' sizes the fake to the real TLS payload, padding by repeating the payload or truncating it, and rewrites the TLS record length so the fake still parses as one complete record. A fake that does not cover the same byte range as the real data leaves a DPI that reassembles the connection an untouched copy of the real bytes to read, which is why byedpi always sizes its fake to the split it performs.",
"tcp.desync.mode": "Sends a fake control packet (RST/FIN/ACK or combinations) crafted to make DPI think the connection is being torn down or has changed state, while the real connection continues. Modes: 'off' disables. 'rst' sends a fake RST. 'fin' sends a fake FIN. 'ack' sends a fake ACK. 'combo' sends RST+FIN+ACK. 'full' sends the strongest combination. Goal: trick DPI into dropping its per-flow inspection state for this connection; the real server is unaffected because the fakes are crafted to be discarded (low TTL/bad sequence/etc.). desync.ttl and desync.count tune the fakes' TTL and how many copies are sent. Cost: extra packets per matched connection.",

BIN
src/config/bin/stun.bin Normal file

Binary file not shown.

View file

@ -263,13 +263,15 @@ var DefaultConfig = Config{
ReferenceDNS: []string{"9.9.9.9", "1.1.1.1", "8.8.8.8", "9.9.1.1", "8.8.4.4"},
ValidationTries: 1,
Watchdog: WatchdogConfig{
Enabled: false,
Domains: []string{},
IntervalSec: 300,
FailureInterval: 60,
Cooldown: 900,
TimeoutSec: 15,
MaxRetries: 3,
Enabled: false,
Domains: []string{},
IntervalSec: 300,
FailureInterval: 60,
Cooldown: 900,
TimeoutSec: 15,
MaxRetries: 3,
HealValidationTries: 3,
VerifyTries: 2,
},
},
API: ApiConfig{

View file

@ -768,6 +768,8 @@ func (c *Config) LoadCapturePayloads() {
set.Faking.PayloadData = FakeSNI1
case FakePayloadDefault2:
set.Faking.PayloadData = FakeSNI2
case FakePayloadSTUN:
set.Faking.PayloadData = FakeSTUN
case FakePayloadCustom:
set.Faking.PayloadData = []byte(set.Faking.CustomPayload)
case FakePayloadCapture:

View file

@ -12,6 +12,9 @@ var FakeQUIC1 []byte
//go:embed bin/quic2.bin
var FakeQUIC2 []byte
//go:embed bin/stun.bin
var FakeSTUN []byte
const (
FakePayloadPreset1 = "@preset:quic1"
FakePayloadPreset2 = "@preset:quic2"

View file

@ -64,6 +64,7 @@ const (
FakePayloadZero // All-zero payload (0x00000000)
FakePayloadInverted // Bitwise-inverted original TLS payload
FakePayloadDomain
FakePayloadSTUN
)
type ApiConfig struct {
@ -221,6 +222,10 @@ type FakingConfig struct {
SNIMutation SNIMutationConfig `json:"sni_mutation"`
TCPMD5 bool `json:"tcp_md5"` // Enable TCP MD5 option insertion
ApplyTTL bool `json:"apply_ttl"` // Apply TTL to the fake regardless of Strategy
MD5OnFake bool `json:"md5_on_fake"` // Put the TCP MD5 option on the fake ClientHello itself
FakeLenMode string `json:"fake_len_mode"` // "" = keep the payload's own length, "match" = size the fake to the real TLS payload
}
type SNIMutationConfig struct {
@ -341,13 +346,15 @@ type DiscoveryConfig struct {
}
type WatchdogConfig struct {
Enabled bool `json:"enabled"`
Domains []string `json:"domains"`
IntervalSec int `json:"interval_sec"`
FailureInterval int `json:"failure_interval"`
Cooldown int `json:"cooldown_sec"`
TimeoutSec int `json:"timeout_sec"`
MaxRetries int `json:"max_retries"`
Enabled bool `json:"enabled"`
Domains []string `json:"domains"`
IntervalSec int `json:"interval_sec"`
FailureInterval int `json:"failure_interval"`
Cooldown int `json:"cooldown_sec"`
TimeoutSec int `json:"timeout_sec"`
MaxRetries int `json:"max_retries"`
HealValidationTries int `json:"heal_validation_tries"`
VerifyTries int `json:"verify_tries"`
}
type Logging struct {

View file

@ -29,6 +29,8 @@ const (
FailureUnknown FailureMode = "unknown"
validationRetryDelay = 100 * time.Millisecond
minSuccessBytes = 1024
)
func NewDiscoverySuite(inputs []string, pool *nfq.Pool, skipDNS bool, skipCache bool, payloadFiles []string, validationTries int, tlsVersion string, ipVersion string, flowMark uint) *DiscoverySuite {
@ -1334,6 +1336,12 @@ evaluate:
}
}
if bytesRead < minSuccessBytes {
result.Status = CheckStatusFailed
result.Error = fmt.Sprintf("insufficient data: %d bytes", bytesRead)
return result
}
result.Status = CheckStatusComplete
return result
}

View file

@ -519,6 +519,90 @@ func GetPhase1Presets() []ConfigPreset {
},
},
},
// 17. Fake only, no fragmentation. Some DPIs reassemble TCP well enough
// that every split is pointless while a single low-TTL fake still lands.
{
Name: "fake-ttl6-md5",
Description: "Low-TTL fake sized to the ClientHello and signed with TCP MD5, no fragmentation",
Family: FamilyFakeSNI,
Phase: PhaseBaseline,
Priority: 17,
Config: config.SetConfig{
TCP: config.TCPConfig{
ConnBytesLimit: 19,
},
UDP: udp,
Fragmentation: config.FragmentationConfig{
Strategy: config.ConfigNone,
},
Faking: config.FakingConfig{
SNI: true,
TTL: 6,
ApplyTTL: true,
Strategy: "ttl",
SNISeqLength: 1,
SNIType: config.FakePayloadDefault1,
FakeLenMode: "match",
MD5OnFake: true,
},
},
},
// 18. Same shape without MD5, for paths where the option is stripped.
{
Name: "fake-ttl8",
Description: "Low-TTL fake sized to the ClientHello, no fragmentation, no MD5",
Family: FamilyFakeSNI,
Phase: PhaseBaseline,
Priority: 18,
Config: config.SetConfig{
TCP: config.TCPConfig{
ConnBytesLimit: 19,
},
UDP: udp,
Fragmentation: config.FragmentationConfig{
Strategy: config.ConfigNone,
},
Faking: config.FakingConfig{
SNI: true,
TTL: 8,
ApplyTTL: true,
Strategy: "ttl",
SNISeqLength: 1,
SNIType: config.FakePayloadDefault1,
FakeLenMode: "match",
},
},
},
// 19. Low-TTL fake kept alongside combo fragmentation.
{
Name: "fake-ttl6-combo",
Description: "Low-TTL matched fake combined with combo fragmentation",
Family: FamilyFakeSNI,
Phase: PhaseBaseline,
Priority: 19,
Config: config.SetConfig{
TCP: config.TCPConfig{
ConnBytesLimit: 19,
Seg2Delay: 20,
Seg2DelayMax: 50,
},
UDP: udp,
Fragmentation: combo,
Faking: config.FakingConfig{
SNI: true,
TTL: 6,
ApplyTTL: true,
Strategy: "ttl",
SNISeqLength: 1,
SNIType: config.FakePayloadDefault1,
FakeLenMode: "match",
MD5OnFake: true,
},
},
},
}
}
@ -1016,9 +1100,28 @@ func GetPhase2Presets(family StrategyFamily) []ConfigPreset {
Config: withFaking(base, config.FakingConfig{
SNI: true,
TTL: ttl,
ApplyTTL: true,
Strategy: "ttl",
SNISeqLength: 1,
SNIType: config.FakePayloadDefault1,
FakeLenMode: "match",
}),
})
presets = append(presets, ConfigPreset{
Name: formatName("fake-ttl%d-md5", ttl),
Family: FamilyFakeSNI,
Phase: PhaseOptimize,
Priority: int(ttl),
Config: withFaking(base, config.FakingConfig{
SNI: true,
TTL: ttl,
ApplyTTL: true,
Strategy: "ttl",
SNISeqLength: 1,
SNIType: config.FakePayloadDefault1,
FakeLenMode: "match",
MD5OnFake: true,
}),
})
}

View file

@ -1,16 +1,19 @@
import { useState, useEffect, useMemo } from "react";
import { Button, Stack, Typography } from "@mui/material";
import { Link } from "react-router";
import {
ImportExportIcon,
CopyIcon,
DownloadIcon,
CheckCircleIcon,
WarningIcon,
} from "@b4.icons";
import { B4Alert, B4Section, B4TextField } from "@b4.elements";
import { useCaptures } from "@b4.capture";
import { useSnackbar } from "@context/SnackbarProvider";
import { useTranslation, Trans } from "react-i18next";
import { B4SetConfig } from "@models/config";
import { B4SetConfig, FakingPayloadType } from "@models/config";
import { createDefaultSet } from "@models/defaults";
import { copyText } from "@utils";
@ -161,6 +164,25 @@ function buildExportJson(config: B4SetConfig): Record<string, unknown> {
return result;
}
function payloadBaseName(path: string): string {
return path.split(/[\\/]/).pop() ?? path;
}
function collectPayloadRefs(cfg: B4SetConfig): string[] {
const refs: string[] = [];
if (
cfg.faking?.sni_type === FakingPayloadType.CAPTURE &&
cfg.faking.payload_file
) {
refs.push(cfg.faking.payload_file);
}
const udpFile = cfg.udp?.fake_payload_file ?? "";
if (cfg.udp?.mode === "fake" && udpFile && !udpFile.startsWith("@")) {
refs.push(udpFile);
}
return [...new Set(refs)];
}
interface ImportExportSettingsProps {
config: B4SetConfig;
onImport: (importedConfig: B4SetConfig) => void;
@ -173,7 +195,10 @@ export const ImportExportSettings = ({
const { t } = useTranslation();
const [jsonValue, setJsonValue] = useState("");
const [importSuccess, setImportSuccess] = useState(false);
const [importedPayloadRefs, setImportedPayloadRefs] = useState<string[]>([]);
const [capturesReady, setCapturesReady] = useState(false);
const { showSuccess, showError } = useSnackbar();
const { captures, loadCaptures } = useCaptures();
const hasSourceDevices = useMemo(
() => (config.targets.source_devices ?? []).length > 0,
[config.targets.source_devices],
@ -183,6 +208,18 @@ export const ImportExportSettings = ({
setJsonValue(JSON.stringify(buildExportJson(config)));
}, [config]);
useEffect(() => {
void loadCaptures().then(() => setCapturesReady(true));
}, [loadCaptures]);
const missingPayloads = useMemo(() => {
if (!capturesReady || importedPayloadRefs.length === 0) return [];
const available = new Set(captures.map((c) => payloadBaseName(c.filepath)));
return importedPayloadRefs.filter(
(ref) => !available.has(payloadBaseName(ref)),
);
}, [capturesReady, captures, importedPayloadRefs]);
function migrateSetConfig(set: Record<string, unknown>): B4SetConfig {
const tcp = set.tcp as Record<string, unknown> | undefined;
@ -277,6 +314,8 @@ export const ImportExportSettings = ({
parsed.id = config.id;
onImport(parsed);
setImportedPayloadRefs(collectPayloadRefs(parsed));
void loadCaptures();
setImportSuccess(true);
return true;
} catch {
@ -316,6 +355,18 @@ export const ImportExportSettings = ({
{t("sets.importExport.infoAlert")}
</B4Alert>
)}
{missingPayloads.length > 0 && (
<B4Alert severity="warning" icon={<WarningIcon />} sx={{ mb: 2 }}>
<Trans
i18nKey="sets.importExport.missingPayloads"
count={missingPayloads.length}
values={{ files: missingPayloads.join(", ") }}
/>{" "}
<Link to="/settings/payloads">
{t("sets.importExport.missingPayloadsLink")}
</Link>
</B4Alert>
)}
<Stack spacing={2}>
<B4TextField
label={t("sets.importExport.jsonLabel")}
@ -324,6 +375,7 @@ export const ImportExportSettings = ({
onChange={(e) => {
setJsonValue(e.target.value);
setImportSuccess(false);
setImportedPayloadRefs([]);
}}
onPaste={handlePaste}
multiline

View file

@ -56,6 +56,7 @@ export const TcpFaking = ({ config, onChange }: TcpFakingProps) => {
{ value: 0, label: t("sets.faking.fakeSni.payloadRandom") },
{ value: 2, label: t("sets.faking.fakeSni.payloadGoogle") },
{ value: 3, label: t("sets.faking.fakeSni.payloadDuckDuckGo") },
{ value: 8, label: t("sets.faking.fakeSni.payloadStun") },
{ value: 4, label: t("sets.faking.fakeSni.payloadFile") },
{ value: 5, label: t("sets.faking.fakeSni.payloadZeros") },
{ value: 6, label: t("sets.faking.fakeSni.payloadInverted") },
@ -178,6 +179,11 @@ export const TcpFaking = ({ config, onChange }: TcpFakingProps) => {
.filter(Boolean)
.join(" + ") || "Disabled";
// The TTL slider only reaches the wire when the fake strategy is TTL, or when
// apply_ttl carries it alongside another strategy.
const isFakeTtlActive =
config.faking.strategy === "ttl" || !!config.faking.apply_ttl;
const desyncStatus = isDesyncEnabled
? desyncModeOptions.find((o) => o.value === config.tcp.desync.mode)
?.label || "Enabled"
@ -253,7 +259,7 @@ export const TcpFaking = ({ config, onChange }: TcpFakingProps) => {
defaultExpanded
>
<Grid container spacing={2}>
<Grid size={{ xs: 12 }}>
<Grid size={{ xs: 12, md: 6 }}>
<B4Switch
label={t("sets.faking.fakeSni.enable")}
checked={config.faking.sni}
@ -263,17 +269,22 @@ export const TcpFaking = ({ config, onChange }: TcpFakingProps) => {
/>
</Grid>
<Grid size={{ xs: 12, md: 6 }}>
<B4Select
label={t("sets.faking.fakeSni.strategy")}
value={config.faking.strategy}
options={FAKE_STRATEGIES}
onChange={(e) => onChange("faking.strategy", e.target.value)}
helperText={t("sets.faking.fakeSni.strategyHelper")}
<B4Slider
label={t("sets.faking.fakeSni.packetCount")}
value={config.faking.sni_seq_length}
onChange={(value: number) =>
onChange("faking.sni_seq_length", value)
}
min={1}
max={20}
step={1}
helperText={t("sets.faking.fakeSni.packetCountHelper")}
disabled={!config.faking.sni}
aiTopic="faking.strategy"
aiContext={{ available: FAKE_STRATEGIES.map((s) => s.value) }}
/>
</Grid>
<B4FormHeader label={t("sets.faking.fakeSni.payloadSection")} />
<Grid size={{ xs: 12, md: 6 }}>
<Stack>
<B4Select
@ -322,6 +333,18 @@ export const TcpFaking = ({ config, onChange }: TcpFakingProps) => {
)}
</Stack>
</Grid>
<Grid size={{ xs: 12, md: 6 }}>
<B4Switch
label={t("sets.faking.fakeSni.matchLength")}
description={t("sets.faking.fakeSni.matchLengthDesc")}
checked={config.faking.fake_len_mode === "match"}
onChange={(checked) =>
onChange("faking.fake_len_mode", checked ? "match" : "")
}
disabled={!config.faking.sni}
aiTopic="faking.fake_len_mode"
/>
</Grid>
{config.faking.sni_type === FakingPayloadType.CAPTURE && (
<Grid container size={{ xs: 12 }}>
{captures.length > 0 && (
@ -363,56 +386,6 @@ export const TcpFaking = ({ config, onChange }: TcpFakingProps) => {
</Grid>
</Grid>
)}
<Grid size={{ xs: 12, md: 4 }}>
<B4Slider
label={t("sets.faking.fakeSni.ttl")}
value={config.faking.ttl}
onChange={(value: number) => onChange("faking.ttl", value)}
min={1}
max={64}
step={1}
helperText={t("sets.faking.fakeSni.ttlHelper")}
disabled={!config.faking.sni}
/>
</Grid>
{(config.faking.strategy === "pastseq" ||
config.faking.strategy === "randseq") && (
<Grid size={{ xs: 12, md: 4 }}>
<B4NumberField
label={t("sets.faking.fakeSni.seqOffset")}
value={config.faking.seq_offset}
onChange={(n) => onChange("faking.seq_offset", n)}
helperText={t("sets.faking.fakeSni.seqOffsetHelper")}
disabled={!config.faking.sni}
/>
</Grid>
)}
{config.faking.strategy === "timestamp" && (
<Grid size={{ xs: 12, md: 4 }}>
<B4NumberField
label={t("sets.faking.fakeSni.timestampDecrease")}
value={config.faking.timestamp_decrease || 600000}
onChange={(n) => onChange("faking.timestamp_decrease", n)}
min={0}
helperText={t("sets.faking.fakeSni.timestampDecreaseHelper")}
disabled={!config.faking.sni}
/>
</Grid>
)}
<Grid size={{ xs: 12, md: 4 }}>
<B4Slider
label={t("sets.faking.fakeSni.packetCount")}
value={config.faking.sni_seq_length}
onChange={(value: number) =>
onChange("faking.sni_seq_length", value)
}
min={1}
max={20}
step={1}
helperText={t("sets.faking.fakeSni.packetCountHelper")}
disabled={!config.faking.sni}
/>
</Grid>
{/* TLS Mod Options - only show when payload has TLS structure */}
{config.faking.sni_type !== FakingPayloadType.RANDOM && (
<Grid size={{ xs: 12 }}>
@ -456,6 +429,78 @@ export const TcpFaking = ({ config, onChange }: TcpFakingProps) => {
</Stack>
</Grid>
)}
<B4FormHeader label={t("sets.faking.fakeSni.rejectionSection")} />
<Grid size={{ xs: 12, md: 6 }}>
<B4Select
label={t("sets.faking.fakeSni.strategy")}
value={config.faking.strategy}
options={FAKE_STRATEGIES}
onChange={(e) => onChange("faking.strategy", e.target.value)}
helperText={t("sets.faking.fakeSni.strategyHelper")}
disabled={!config.faking.sni}
aiTopic="faking.strategy"
aiContext={{ available: FAKE_STRATEGIES.map((s) => s.value) }}
/>
</Grid>
{(config.faking.strategy === "pastseq" ||
config.faking.strategy === "randseq") && (
<Grid size={{ xs: 12, md: 6 }}>
<B4NumberField
label={t("sets.faking.fakeSni.seqOffset")}
value={config.faking.seq_offset}
onChange={(n) => onChange("faking.seq_offset", n)}
helperText={t("sets.faking.fakeSni.seqOffsetHelper")}
disabled={!config.faking.sni}
/>
</Grid>
)}
{config.faking.strategy === "timestamp" && (
<Grid size={{ xs: 12, md: 6 }}>
<B4NumberField
label={t("sets.faking.fakeSni.timestampDecrease")}
value={config.faking.timestamp_decrease || 600000}
onChange={(n) => onChange("faking.timestamp_decrease", n)}
min={0}
helperText={t("sets.faking.fakeSni.timestampDecreaseHelper")}
disabled={!config.faking.sni}
/>
</Grid>
)}
<Grid size={{ xs: 12, md: 4 }}>
<B4Slider
label={t("sets.faking.fakeSni.ttl")}
value={config.faking.ttl}
onChange={(value: number) => onChange("faking.ttl", value)}
min={1}
max={64}
step={1}
helperText={t("sets.faking.fakeSni.ttlHelper")}
disabled={!config.faking.sni || !isFakeTtlActive}
/>
</Grid>
<Grid size={{ xs: 12, md: 4 }}>
<B4Switch
label={t("sets.faking.fakeSni.applyTtl")}
description={t("sets.faking.fakeSni.applyTtlDesc")}
checked={config.faking.apply_ttl || false}
onChange={(checked) => onChange("faking.apply_ttl", checked)}
disabled={!config.faking.sni || config.faking.strategy === "ttl"}
aiTopic="faking.apply_ttl"
/>
</Grid>
<Grid size={{ xs: 12, md: 4 }}>
<B4Switch
label={t("sets.faking.fakeSni.md5OnFake")}
description={t("sets.faking.fakeSni.md5OnFakeDesc")}
checked={config.faking.md5_on_fake || false}
onChange={(checked) => onChange("faking.md5_on_fake", checked)}
disabled={!config.faking.sni}
aiTopic="faking.md5_on_fake"
/>
</Grid>
</Grid>
</B4Accordion>

View file

@ -1085,6 +1085,8 @@
"title": "Fake SNI Packets",
"enable": "Enable Fake SNI",
"enableDesc": "Send fake SNI packets before real ClientHello",
"payloadSection": "Fake payload",
"rejectionSection": "How the server rejects it",
"strategy": "Fake Strategy",
"strategyHelper": "How to make fake packets unprocessable by server",
"strategyTtl": "TTL",
@ -1098,6 +1100,7 @@
"payloadRandom": "Random",
"payloadGoogle": "Preset: Google (classic)",
"payloadDuckDuckGo": "Preset: DuckDuckGo",
"payloadStun": "Preset: STUN",
"payloadFile": "My own Payload File",
"payloadZeros": "All Zeros",
"payloadInverted": "Inverted Original",
@ -1115,6 +1118,12 @@
"navigateSettings": "Navigate to Settings to generate or upload TLS ClientHello payloads.",
"ttl": "Fake TTL",
"ttlHelper": "TTL for fake packets (should expire before server)",
"applyTtl": "Apply Fake TTL",
"applyTtlDesc": "Send the fake at the TTL above whatever the fake strategy is. Without this the TTL only applies when the strategy is TTL",
"matchLength": "Match Fake Size",
"matchLengthDesc": "Size the fake to the real ClientHello instead of using the payload's own length",
"md5OnFake": "MD5 Signature on Fake",
"md5OnFakeDesc": "Put the TCP MD5 option on the fake ClientHello so the server discards it while DPI still reads it",
"seqOffset": "Sequence Offset",
"seqOffsetHelper": "TCP sequence number offset for pastseq strategy",
"timestampDecrease": "Timestamp Decrease",
@ -1421,7 +1430,10 @@
"copiedToClipboard": "Copied to clipboard",
"copyFailed": "Failed to copy to clipboard",
"invalidFields": "Invalid set configuration: missing required fields",
"invalidJson": "Invalid JSON format"
"invalidJson": "Invalid JSON format",
"missingPayloads_one": "This set points to a payload file that does not exist on this device: {{files}}. Fake packets fall back to a built-in payload, so the set behaves differently than on the machine it was exported from. Ask the author for the .bin file and upload it under the same name.",
"missingPayloads_other": "This set points to payload files that do not exist on this device: {{files}}. Fake packets fall back to a built-in payload, so the set behaves differently than on the machine it was exported from. Ask the author for the .bin files and upload them under the same names.",
"missingPayloadsLink": "Open Payloads settings"
},
"editor": {
"namePlaceholder": "Set name...",

View file

@ -1082,6 +1082,8 @@
"title": "Фейковые SNI-пакеты",
"enable": "Включить фейковый SNI",
"enableDesc": "Отправлять фейковые SNI-пакеты перед реальным ClientHello",
"payloadSection": "Содержимое фейка",
"rejectionSection": "Как сервер его отбрасывает",
"strategy": "Стратегия фейка",
"strategyHelper": "Как сделать фейковые пакеты необрабатываемыми сервером",
"strategyTtl": "TTL",
@ -1095,6 +1097,7 @@
"payloadRandom": "Случайное",
"payloadGoogle": "Пресет: Google (классический)",
"payloadDuckDuckGo": "Пресет: DuckDuckGo",
"payloadStun": "Пресет: STUN",
"payloadFile": "Свой файл payload",
"payloadZeros": "Все нули",
"payloadInverted": "Инвертированный оригинал",
@ -1112,6 +1115,12 @@
"navigateSettings": "Перейдите в Настройки для генерации или загрузки TLS ClientHello payloads.",
"ttl": "TTL фейка",
"ttlHelper": "TTL для фейковых пакетов (должен истечь до сервера)",
"applyTtl": "Применять TTL фейка",
"applyTtlDesc": "Отправлять фейк с указанным выше TTL при любой стратегии фейка. Без этого TTL действует только при стратегии TTL",
"matchLength": "Подгонять размер фейка",
"matchLengthDesc": "Делать фейк того же размера, что и настоящий ClientHello, вместо собственной длины payload",
"md5OnFake": "MD5 Signature на фейке",
"md5OnFakeDesc": "Ставить опцию TCP MD5 на сам фейковый ClientHello: сервер его отбросит, а DPI всё равно прочитает",
"seqOffset": "Смещение Sequence",
"seqOffsetHelper": "Смещение TCP sequence number для стратегии pastseq",
"timestampDecrease": "Уменьшение Timestamp",
@ -1418,7 +1427,11 @@
"copiedToClipboard": "Скопировано в буфер обмена",
"copyFailed": "Не удалось скопировать в буфер обмена",
"invalidFields": "Недопустимая конфигурация сета: отсутствуют обязательные поля",
"invalidJson": "Неверный формат JSON"
"invalidJson": "Неверный формат JSON",
"missingPayloads_one": "Сет ссылается на файл payload, которого нет на этом устройстве: {{files}}. Фейковые пакеты будут собраны из встроенного payload, поэтому сет работает не так, как на машине, откуда его экспортировали. Запросите .bin у автора и загрузите его под тем же именем.",
"missingPayloads_few": "Сет ссылается на файлы payload, которых нет на этом устройстве: {{files}}. Фейковые пакеты будут собраны из встроенного payload, поэтому сет работает не так, как на машине, откуда его экспортировали. Запросите .bin у автора и загрузите их под теми же именами.",
"missingPayloads_many": "Сет ссылается на файлы payload, которых нет на этом устройстве: {{files}}. Фейковые пакеты будут собраны из встроенного payload, поэтому сет работает не так, как на машине, откуда его экспортировали. Запросите .bin у автора и загрузите их под теми же именами.",
"missingPayloadsLink": "Открыть настройки Payloads"
},
"editor": {
"namePlaceholder": "Название сета...",

View file

@ -15,6 +15,7 @@ export enum FakingPayloadType {
ZERO = 5,
INVERTED = 6,
DOMAIN = 7,
STUN = 8,
}
export type MutationMode =
@ -47,7 +48,12 @@ export interface FakingConfig {
tls_mod: string[];
tcp_md5: boolean;
timestamp_decrease: number;
apply_ttl: boolean;
md5_on_fake: boolean;
fake_len_mode: FakeLenMode;
}
export type FakeLenMode = "" | "match";
export type FragmentationStrategy =
| "tcp"
| "ip"

View file

@ -14,6 +14,7 @@ func Options(extra ...goleak.Option) []goleak.Option {
goleak.IgnoreTopFunction("github.com/daniellavrushin/b4/quic.cleanupStaleEntries"),
goleak.IgnoreTopFunction("github.com/daniellavrushin/b4/log.startFlusherLocked.func1"),
goleak.IgnoreTopFunction("github.com/daniellavrushin/b4/metrics.(*MetricsCollector).updateLoop"),
goleak.IgnoreTopFunction("github.com/daniellavrushin/b4/capture.(*Manager).cleanupExpiredProbes"),
}
return append(base, extra...)
}

205
src/nfq/clienthello.go Normal file
View file

@ -0,0 +1,205 @@
package nfq
import (
"sync"
"sync/atomic"
"time"
)
const (
maxPendingHelloEntries = 2048
maxPendingHelloBytes = 1 << 20
maxPendingHelloRecord = 4096
pendingHelloTTL = 2 * time.Second
)
type pendingHello struct {
startSeq uint32
endSeq uint32
data []byte
storedAt time.Time
}
type pendingHelloCache struct {
mu sync.Mutex
flows map[string]*pendingHello
bytes int
live atomic.Int64
}
func newPendingHelloCache() *pendingHelloCache {
return &pendingHelloCache{flows: make(map[string]*pendingHello)}
}
func truncatedClientHello(payload []byte) bool {
if len(payload) < 6 || payload[0] != TLSHandshakeType || payload[5] != TLSClientHello {
return false
}
recLen := int(payload[3])<<8 | int(payload[4])
if recLen <= 0 {
return false
}
return 5+recLen > len(payload)
}
func (c *pendingHelloCache) Feed(connKey string, seq uint32, payload []byte) ([]byte, int, bool) {
if c == nil || len(payload) == 0 {
return nil, 0, false
}
if c.live.Load() == 0 && !truncatedClientHello(payload) {
return nil, 0, false
}
c.mu.Lock()
defer c.mu.Unlock()
defer c.syncLiveLocked()
entry := c.flows[connKey]
if entry != nil && time.Since(entry.storedAt) > pendingHelloTTL {
c.dropLocked(connKey)
entry = nil
}
if entry == nil {
c.storeLocked(connKey, seq, payload)
return nil, 0, false
}
joined, prefix, related := joinSegment(entry, payload, seq)
if !related {
c.dropLocked(connKey)
c.storeLocked(connKey, seq, payload)
return nil, 0, false
}
if joined == nil {
return nil, 0, false
}
startSeq := entry.startSeq
c.dropLocked(connKey)
if len(joined) <= maxPendingHelloRecord {
c.storeLocked(connKey, startSeq, joined)
}
return joined, prefix, true
}
func joinSegment(entry *pendingHello, payload []byte, seq uint32) ([]byte, int, bool) {
gap := int32(seq - entry.endSeq)
if gap > 0 {
return nil, 0, false
}
if int32(seq-entry.startSeq) < 0 {
return nil, 0, false
}
overlap := int(-gap)
if overlap >= len(payload) {
return nil, 0, true
}
fresh := payload[overlap:]
joined := make([]byte, 0, len(entry.data)+len(fresh))
joined = append(joined, entry.data...)
joined = append(joined, fresh...)
return joined, len(entry.data), true
}
func (c *pendingHelloCache) Drop(connKey string) {
if c == nil {
return
}
c.mu.Lock()
defer c.mu.Unlock()
c.dropLocked(connKey)
c.syncLiveLocked()
}
func (c *pendingHelloCache) syncLiveLocked() {
c.live.Store(int64(len(c.flows)))
}
func (c *pendingHelloCache) dropLocked(connKey string) {
entry, ok := c.flows[connKey]
if !ok {
return
}
c.bytes -= len(entry.data)
delete(c.flows, connKey)
}
func (c *pendingHelloCache) storeLocked(connKey string, seq uint32, payload []byte) {
if len(payload) > maxPendingHelloRecord || !truncatedClientHello(payload) {
return
}
c.evictLocked(len(payload))
if len(c.flows) >= maxPendingHelloEntries || c.bytes+len(payload) > maxPendingHelloBytes {
return
}
data := make([]byte, len(payload))
copy(data, payload)
c.flows[connKey] = &pendingHello{
startSeq: seq,
endSeq: seq + uint32(len(payload)),
data: data,
storedAt: time.Now(),
}
c.bytes += len(data)
}
func (c *pendingHelloCache) evictLocked(incoming int) {
if len(c.flows) < maxPendingHelloEntries && c.bytes+incoming <= maxPendingHelloBytes {
return
}
now := time.Now()
for k, v := range c.flows {
if now.Sub(v.storedAt) > pendingHelloTTL {
c.bytes -= len(v.data)
delete(c.flows, k)
}
}
for len(c.flows) >= maxPendingHelloEntries || c.bytes+incoming > maxPendingHelloBytes {
var oldestKey string
var oldestAt time.Time
for k, v := range c.flows {
if oldestAt.IsZero() || v.storedAt.Before(oldestAt) {
oldestKey = k
oldestAt = v.storedAt
}
}
if oldestKey == "" {
return
}
c.dropLocked(oldestKey)
}
}
func (c *pendingHelloCache) Cleanup() {
if c == nil {
return
}
c.mu.Lock()
defer c.mu.Unlock()
now := time.Now()
for k, v := range c.flows {
if now.Sub(v.storedAt) > pendingHelloTTL {
c.bytes -= len(v.data)
delete(c.flows, k)
}
}
c.syncLiveLocked()
}
func (c *pendingHelloCache) Len() int {
if c == nil {
return 0
}
c.mu.Lock()
defer c.mu.Unlock()
return len(c.flows)
}

567
src/nfq/clienthello_test.go Normal file
View file

@ -0,0 +1,567 @@
package nfq
import (
"bytes"
"encoding/binary"
"strings"
"testing"
"time"
"github.com/daniellavrushin/b4/config"
"github.com/daniellavrushin/b4/engine"
"github.com/daniellavrushin/b4/sni"
)
func newPassiveSet() config.SetConfig {
set := config.NewSetConfig()
set.Id = "yt-images"
set.Name = "YT images"
set.Enabled = true
set.Targets.DomainOnly = true
set.Targets.DomainsToMatch = []string{"ytimg.com"}
set.TCP.RSTProtection.Enabled = true
set.Fragmentation.Strategy = config.ConfigNone
set.Fragmentation.StrategyPool = nil
set.Faking.SNI = false
set.Faking.SNIMutation.Mode = config.ConfigOff
set.TCP.Desync.Mode = config.ConfigOff
set.TCP.Win.Mode = config.ConfigOff
set.TCP.DropSACK = false
return set
}
func newTestWorker(t *testing.T, cfg *config.Config) *Worker {
t.Helper()
cfg.ConfigPath = t.TempDir() + "/b4.json"
w := NewWorkerWithQueue(cfg, 0)
w.matcher.Store(buildMatcher(cfg))
w.ipToMac.Store(make(map[string]string))
state := newRuntimeState()
w.tlsCache = state.tlsCache
w.connTracker = state.connState
w.destState = state.destState
w.pendingHello = state.pendingHello
w.hostHints = state.hostHints
return w
}
func buildExtension(extType uint16, data []byte) []byte {
out := make([]byte, 4+len(data))
binary.BigEndian.PutUint16(out[0:2], extType)
binary.BigEndian.PutUint16(out[2:4], uint16(len(data)))
copy(out[4:], data)
return out
}
func buildSNIExtension(host string) []byte {
data := make([]byte, 5+len(host))
binary.BigEndian.PutUint16(data[0:2], uint16(3+len(host)))
data[2] = 0
binary.BigEndian.PutUint16(data[3:5], uint16(len(host)))
copy(data[5:], host)
return buildExtension(0, data)
}
func buildClientHello(host string, padBefore int, padFiller byte) []byte {
pad := bytes.Repeat([]byte{padFiller}, padBefore)
var exts []byte
exts = append(exts, buildExtension(21, pad)...)
exts = append(exts, buildSNIExtension(host)...)
exts = append(exts, buildExtension(43, []byte{0x02, 0x03, 0x04})...)
body := make([]byte, 0, 64+len(exts))
body = append(body, 0x03, 0x03)
body = append(body, bytes.Repeat([]byte{0x11}, 32)...)
body = append(body, 32)
body = append(body, bytes.Repeat([]byte{0x22}, 32)...)
body = append(body, 0x00, 0x02, 0x13, 0x01)
body = append(body, 0x01, 0x00)
extLen := make([]byte, 2)
binary.BigEndian.PutUint16(extLen, uint16(len(exts)))
body = append(body, extLen...)
body = append(body, exts...)
handshake := make([]byte, 4+len(body))
handshake[0] = 0x01
handshake[1] = byte(len(body) >> 16)
handshake[2] = byte(len(body) >> 8)
handshake[3] = byte(len(body))
copy(handshake[4:], body)
record := make([]byte, 5+len(handshake))
record[0] = 0x16
record[1] = 0x03
record[2] = 0x01
binary.BigEndian.PutUint16(record[3:5], uint16(len(handshake)))
copy(record[5:], handshake)
return record
}
func TestTruncatedClientHello(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
if truncatedClientHello(full) {
t.Fatal("complete ClientHello must not be reported as truncated")
}
if !truncatedClientHello(full[:1396]) {
t.Fatal("first segment of a split ClientHello must be reported as truncated")
}
if truncatedClientHello(full[1396:]) {
t.Fatal("continuation segment must not be reported as truncated")
}
if truncatedClientHello([]byte{0x16, 0x03, 0x01, 0x00, 0x10, 0x02}) {
t.Fatal("non-ClientHello handshake record must not be buffered")
}
if truncatedClientHello([]byte{0x17, 0x03, 0x03, 0x00, 0x10, 0x01}) {
t.Fatal("application data record must not be buffered")
}
}
func TestSplitClientHelloIsNotClassifiableAlone(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
if len(full) <= 1396 {
t.Fatalf("fixture too small to split: %d bytes", len(full))
}
if host, _, _ := sni.ParseTLSClientHelloSNI(full); host != "i.ytimg.com" {
t.Fatalf("complete ClientHello: want SNI i.ytimg.com, got %q", host)
}
if host, _, _ := sni.ParseTLSClientHelloSNI(full[:1396]); host != "" {
t.Fatalf("first segment should not yield an SNI, got %q", host)
}
}
func TestPendingHelloCacheJoinsSplitClientHello(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
seg1, seg2 := full[:1396], full[1396:]
const baseSeq = uint32(5000)
c := newPendingHelloCache()
if _, _, ok := c.Feed("flow", baseSeq, seg1); ok {
t.Fatal("first segment must not report a join")
}
if c.Len() != 1 {
t.Fatalf("first segment should be buffered, entries=%d", c.Len())
}
joined, prefix, ok := c.Feed("flow", baseSeq+uint32(len(seg1)), seg2)
if !ok {
t.Fatal("continuation segment should join the buffered prefix")
}
if prefix != len(seg1) {
t.Fatalf("prefix length: want %d, got %d", len(seg1), prefix)
}
if !bytes.Equal(joined, full) {
t.Fatalf("joined payload differs from original (%d vs %d bytes)", len(joined), len(full))
}
host, tlsVersion, _ := sni.ParseTLSClientHelloSNI(joined)
if host != "i.ytimg.com" {
t.Fatalf("recovered SNI: want i.ytimg.com, got %q", host)
}
if tlsVersion != 0x0304 {
t.Fatalf("recovered TLS version: want 0x0304, got 0x%04x", tlsVersion)
}
c.Drop("flow")
if c.Len() != 0 {
t.Fatalf("Drop should release the flow, entries=%d", c.Len())
}
}
func TestPendingHelloCacheThreeSegments(t *testing.T) {
full := buildClientHello("youtubei.googleapis.com", 2900, 0xCD)
if len(full) <= 2800 {
t.Fatalf("fixture too small: %d bytes", len(full))
}
const baseSeq = uint32(77)
c := newPendingHelloCache()
c.Feed("flow", baseSeq, full[:1400])
if _, _, ok := c.Feed("flow", baseSeq+1400, full[1400:2800]); !ok {
t.Fatal("second segment should join")
}
joined, _, ok := c.Feed("flow", baseSeq+2800, full[2800:])
if !ok {
t.Fatal("third segment should join")
}
if !bytes.Equal(joined, full) {
t.Fatalf("three-way join mismatch (%d vs %d bytes)", len(joined), len(full))
}
if host, _, _ := sni.ParseTLSClientHelloSNI(joined); host != "youtubei.googleapis.com" {
t.Fatalf("recovered SNI: got %q", host)
}
}
func TestPendingHelloCacheRetransmit(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
seg1 := full[:1396]
const baseSeq = uint32(9000)
c := newPendingHelloCache()
c.Feed("flow", baseSeq, seg1)
if _, _, ok := c.Feed("flow", baseSeq, seg1); ok {
t.Fatal("retransmitted first segment must not produce a join")
}
if c.Len() != 1 {
t.Fatalf("retransmit must keep the buffered prefix, entries=%d", c.Len())
}
joined, _, ok := c.Feed("flow", baseSeq+uint32(len(seg1)), full[1396:])
if !ok || !bytes.Equal(joined, full) {
t.Fatal("prefix should still join after a retransmit")
}
}
func TestPendingHelloCacheOverlap(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
const baseSeq = uint32(400)
const overlap = 200
c := newPendingHelloCache()
c.Feed("flow", baseSeq, full[:1396])
joined, prefix, ok := c.Feed("flow", baseSeq+1396-overlap, full[1396-overlap:])
if !ok {
t.Fatal("overlapping segment should join")
}
if prefix != 1396 {
t.Fatalf("prefix length: want 1396, got %d", prefix)
}
if !bytes.Equal(joined, full) {
t.Fatalf("overlap trimming produced %d bytes, want %d", len(joined), len(full))
}
}
func TestPendingHelloCacheGapDropsPrefix(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
const baseSeq = uint32(1)
c := newPendingHelloCache()
c.Feed("flow", baseSeq, full[:1396])
if _, _, ok := c.Feed("flow", baseSeq+2000, full[1396:]); ok {
t.Fatal("out-of-order segment must not join")
}
if c.Len() != 0 {
t.Fatalf("gap should discard the stale prefix, entries=%d", c.Len())
}
}
func TestPendingHelloCacheExpiry(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
const baseSeq = uint32(1)
c := newPendingHelloCache()
c.Feed("flow", baseSeq, full[:1396])
c.mu.Lock()
c.flows["flow"].storedAt = time.Now().Add(-2 * pendingHelloTTL)
c.mu.Unlock()
if _, _, ok := c.Feed("flow", baseSeq+1396, full[1396:]); ok {
t.Fatal("expired prefix must not join")
}
c.Cleanup()
if c.Len() != 0 {
t.Fatalf("Cleanup should drop expired entries, entries=%d", c.Len())
}
}
func TestPendingHelloCacheRejectsOversizedRecord(t *testing.T) {
oversized := buildClientHello("i.ytimg.com", maxPendingHelloRecord*2, 0xAB)
c := newPendingHelloCache()
c.Feed("flow", 1, oversized[:maxPendingHelloRecord+1])
if c.Len() != 0 {
t.Fatalf("payload above the record cap must not be buffered, entries=%d", c.Len())
}
}
func TestPendingHelloCacheBoundsEntries(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
seg1 := full[:1396]
c := newPendingHelloCache()
for i := 0; i < maxPendingHelloEntries+64; i++ {
c.Feed(string(rune(i))+"-flow", uint32(i), seg1)
}
if c.Len() > maxPendingHelloEntries {
t.Fatalf("entry cap exceeded: %d", c.Len())
}
if c.bytes > maxPendingHelloBytes {
t.Fatalf("byte cap exceeded: %d", c.bytes)
}
}
func TestPendingHelloCacheNilReceiver(t *testing.T) {
var c *pendingHelloCache
if _, _, ok := c.Feed("flow", 1, []byte{0x16}); ok {
t.Fatal("nil cache must not report a join")
}
c.Drop("flow")
c.Cleanup()
if c.Len() != 0 {
t.Fatal("nil cache must report zero entries")
}
}
func TestPendingHelloCacheSkipsBulkPayloadWhenEmpty(t *testing.T) {
c := newPendingHelloCache()
bulk := bytes.Repeat([]byte{0x17, 0x03, 0x03}, 500)
if _, _, ok := c.Feed("flow", 1, bulk); ok {
t.Fatal("bulk data must not report a join")
}
if c.Len() != 0 {
t.Fatalf("bulk data must not be buffered, entries=%d", c.Len())
}
if c.live.Load() != 0 {
t.Fatalf("live counter should stay at zero, got %d", c.live.Load())
}
}
func TestPendingHelloCacheLiveCounterTracksEntries(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
seg1 := full[:1396]
c := newPendingHelloCache()
c.Feed("flow", 1, seg1)
if c.live.Load() != 1 {
t.Fatalf("live counter after store: want 1, got %d", c.live.Load())
}
bulk := bytes.Repeat([]byte{0x17, 0x03, 0x03}, 500)
if _, _, ok := c.Feed("other", 1, bulk); ok {
t.Fatal("an unrelated bulk payload must not join")
}
c.Feed("flow", 1+uint32(len(seg1)), full[1396:])
c.Drop("flow")
if c.live.Load() != 0 {
t.Fatalf("live counter after drop: want 0, got %d", c.live.Load())
}
c.Feed("flow", 1, seg1)
c.mu.Lock()
c.flows["flow"].storedAt = time.Now().Add(-2 * pendingHelloTTL)
c.mu.Unlock()
c.Cleanup()
if c.live.Load() != 0 {
t.Fatalf("live counter after cleanup: want 0, got %d", c.live.Load())
}
}
func BenchmarkPendingHelloFeedBulkPayload(b *testing.B) {
c := newPendingHelloCache()
bulk := bytes.Repeat([]byte{0x17, 0x03, 0x03}, 500)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
c.Feed("10.0.0.1:12345->1.2.3.4:443", uint32(i), bulk)
}
}
func BenchmarkPendingHelloFeedBulkPayloadParallel(b *testing.B) {
c := newPendingHelloCache()
bulk := bytes.Repeat([]byte{0x17, 0x03, 0x03}, 500)
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for i := 0; pb.Next(); i++ {
c.Feed("10.0.0.1:12345->1.2.3.4:443", uint32(i), bulk)
}
})
}
func BenchmarkPendingHelloFeedBulkPayloadWhileBuffering(b *testing.B) {
c := newPendingHelloCache()
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
c.Feed("some-other-flow", 1, full[:1396])
bulk := bytes.Repeat([]byte{0x17, 0x03, 0x03}, 500)
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for i := 0; pb.Next(); i++ {
c.Feed("10.0.0.1:12345->1.2.3.4:443", uint32(i), bulk)
}
})
}
func TestLocateSNIInContinuationSegment(t *testing.T) {
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
seg2 := full[1396:]
if _, _, ok := locateSNIInRecord(seg2); ok {
t.Fatal("structural parse must not succeed on a mid-record segment")
}
start, end, ok := locateSNI(seg2)
if !ok {
t.Fatal("locateSNI should find the hostname in the continuation segment")
}
if got := string(seg2[start:end]); got != "i.ytimg.com" {
t.Fatalf("located hostname: want i.ytimg.com, got %q", got)
}
}
func TestLocateSNIStillPrefersStructuralParse(t *testing.T) {
full := buildClientHello("i.ytimg.com", 16, 0xAB)
start, end, ok := locateSNI(full)
if !ok {
t.Fatal("locateSNI should parse a complete ClientHello")
}
if got := string(full[start:end]); got != "i.ytimg.com" {
t.Fatalf("located hostname: want i.ytimg.com, got %q", got)
}
}
func TestHandlerRecoversSplitClientHelloClassification(t *testing.T) {
set := newPassiveSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
const baseSeq = uint32(1000)
const cut = 1396
const connKey = "10.0.0.1:12345->1.2.3.4:443"
if v := w.ProcessPacket(makeV4TCPPacket(full[:cut], baseSeq)); v != engine.VerdictAccept {
t.Fatalf("first segment: want accept, got %v", v)
}
if host, _, found := w.tlsCache.Lookup(connKey); found {
t.Fatalf("first segment must not yield a cached host, got %q", host)
}
if bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443); bound != nil {
t.Fatalf("first segment must not bind a set, got %q", bound.Name)
}
if w.pendingHello.Len() != 1 {
t.Fatalf("first segment should be buffered, entries=%d", w.pendingHello.Len())
}
w.ProcessPacket(makeV4TCPPacket(full[cut:], baseSeq+cut))
host, tlsVersion, found := w.tlsCache.Lookup(connKey)
if !found || host != "i.ytimg.com" {
t.Fatalf("continuation segment should recover the SNI, got host=%q found=%v", host, found)
}
if tlsVersion != 0x0304 {
t.Fatalf("recovered TLS version: want 0x0304, got 0x%04x", tlsVersion)
}
bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443)
if bound == nil {
t.Fatal("continuation segment should bind the matched set to the flow")
}
if bound.Id != "yt-images" {
t.Fatalf("bound set: want yt-images, got %q", bound.Id)
}
}
func TestHandlerRecoversWhenSplitFallsInsideHostname(t *testing.T) {
set := newPassiveSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
full := buildClientHello("i.ytimg.com", 1400, 0xAB)
nameAt := bytes.Index(full, []byte("i.ytimg.com"))
if nameAt < 0 {
t.Fatal("fixture does not contain the hostname")
}
cut := nameAt + 4
const baseSeq = uint32(1000)
w.ProcessPacket(makeV4TCPPacket(full[:cut], baseSeq))
w.ProcessPacket(makeV4TCPPacket(full[cut:], baseSeq+uint32(cut)))
host, _, found := w.tlsCache.Lookup("10.0.0.1:12345->1.2.3.4:443")
if !found || host != "i.ytimg.com" {
t.Fatalf("a hostname straddling the segment boundary should still be recovered, got %q found=%v", host, found)
}
if bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443); bound == nil {
t.Fatal("straddled hostname should still bind the matched set")
}
if _, _, ok := locateSNI(full[cut:]); ok {
t.Fatal("a partial hostname must not be reported as a located SNI")
}
}
func TestHandlerClassifiesUnsplitClientHelloOnFirstPacket(t *testing.T) {
set := newPassiveSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
full := buildClientHello("i.ytimg.com", 16, 0xAB)
w.ProcessPacket(makeV4TCPPacket(full, 1000))
if bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443); bound == nil {
t.Fatal("a complete ClientHello must classify on the first packet")
}
if w.pendingHello.Len() != 0 {
t.Fatalf("a classified flow must not be buffered, entries=%d", w.pendingHello.Len())
}
}
func buildBareSNIExtension(host string) []byte {
ext := make([]byte, 9+len(host))
binary.BigEndian.PutUint16(ext[0:2], 0)
binary.BigEndian.PutUint16(ext[2:4], uint16(5+len(host)))
binary.BigEndian.PutUint16(ext[4:6], uint16(3+len(host)))
ext[6] = 0
binary.BigEndian.PutUint16(ext[7:9], uint16(len(host)))
copy(ext[9:], host)
return ext
}
func TestScanSNIExtensionEnforcesMaxSNILength(t *testing.T) {
longest := strings.Repeat("a", MaxSNILength-4) + ".com"
if len(longest) != MaxSNILength {
t.Fatalf("fixture length: want %d, got %d", MaxSNILength, len(longest))
}
payload := append([]byte{0xAB, 0xCD}, buildBareSNIExtension(longest)...)
start, end, ok := scanSNIExtension(payload)
if !ok {
t.Fatal("a hostname at the maximum length should still be located")
}
if string(payload[start:end]) != longest {
t.Fatal("located hostname does not match the fixture")
}
oversized := strings.Repeat("a", MaxSNILength-3) + ".com"
payload = append([]byte{0xAB, 0xCD}, buildBareSNIExtension(oversized)...)
if _, _, ok := scanSNIExtension(payload); ok {
t.Fatalf("a hostname longer than %d bytes is not a legal SNI and must be rejected", MaxSNILength)
}
}
func TestScanSNIExtensionRejectsNonTLS(t *testing.T) {
if _, _, ok := scanSNIExtension(bytes.Repeat([]byte{0x00}, 2048)); ok {
t.Fatal("all-zero padding must not yield a hostname")
}
if _, _, ok := scanSNIExtension(bytes.Repeat([]byte{0xFF}, 2048)); ok {
t.Fatal("all-ones payload must not yield a hostname")
}
appData := append([]byte{0x17, 0x03, 0x03, 0x05, 0x00}, buildSNIExtension("i.ytimg.com")...)
if _, _, ok := scanSNIExtension(appData); ok {
t.Fatal("application data records must not be scanned")
}
}

View file

@ -9,6 +9,7 @@ import (
"github.com/daniellavrushin/b4/config"
"github.com/daniellavrushin/b4/log"
"github.com/daniellavrushin/b4/sni"
"github.com/daniellavrushin/b4/sock"
)
@ -160,6 +161,51 @@ func uniqueSorted(splits []int, maxVal int) []int {
}
func locateSNI(payload []byte) (start, end int, ok bool) {
if s, e, found := locateSNIInRecord(payload); found {
return s, e, true
}
return scanSNIExtension(payload)
}
func scanSNIExtension(payload []byte) (start, end int, ok bool) {
if len(payload) < 10 {
return 0, 0, false
}
switch payload[0] {
case 20, 21, 23, 24:
return 0, 0, false
}
for i := 0; i+9 <= len(payload); i++ {
if payload[i] != 0x00 || payload[i+1] != 0x00 {
continue
}
extLen := int(binary.BigEndian.Uint16(payload[i+2 : i+4]))
listLen := int(binary.BigEndian.Uint16(payload[i+4 : i+6]))
if listLen < 4 || extLen != listLen+2 {
continue
}
if payload[i+6] != 0x00 {
continue
}
nameLen := int(binary.BigEndian.Uint16(payload[i+7 : i+9]))
if nameLen != listLen-3 || nameLen > MaxSNILength {
continue
}
s := i + 9
e := s + nameLen
if e > len(payload) {
continue
}
if !sni.IsValidSNI(payload[s:e]) {
continue
}
return s, e, true
}
return 0, 0, false
}
func locateSNIInRecord(payload []byte) (start, end int, ok bool) {
if len(payload) < 5 || payload[0] != TLSHandshakeType {
return 0, 0, false
}

View file

@ -370,13 +370,17 @@ func (t *destStateTracker) RecordRSTKill(host string, threshold int, window time
}
type runtimeState struct {
tlsCache *tlsInfoCache
connState *connStateTracker
destState *destStateTracker
tlsCache *tlsInfoCache
connState *connStateTracker
destState *destStateTracker
pendingHello *pendingHelloCache
hostHints *hostHintCache
}
func newRuntimeState() *runtimeState {
return &runtimeState{
pendingHello: newPendingHelloCache(),
hostHints: newHostHintCache(),
tlsCache: &tlsInfoCache{
conns: make(map[string]*tlsInfo),
},

View file

@ -217,6 +217,7 @@ func (w *Worker) processDnsPacket(vc *verdictCtx, ipVersion byte, sport uint16,
clientMac := w.getMacByIp(clientIP.String())
if matched, set := w.getMatcher().MatchSNIWithSource(domain, clientMac); matched && set.Enabled {
ips := dns.ParseResponseIPs(payload)
w.storeHostHints(clientIP, set, domain, ips)
if set.Routing.Enabled && !set.Targets.DomainOnly && len(ips) > 0 {
cfg := w.getConfig()
if RoutingHandleDNSFunc != nil && !cfg.Queue.IsDiscovery {
@ -232,8 +233,9 @@ func (w *Worker) processDnsPacket(vc *verdictCtx, ipVersion byte, sport uint16,
); hit && !routed {
if ips := dns.ParseResponseIPs(payload); len(ips) > 0 {
cfg := w.getConfig()
if set := cfg.GetSetById(setID); set != nil && !set.Targets.DomainOnly {
if RoutingHandleDNSFunc != nil && !cfg.Queue.IsDiscovery {
if set := cfg.GetSetById(setID); set != nil {
w.storeHostHints(clientIP, set, domain, ips)
if !set.Targets.DomainOnly && RoutingHandleDNSFunc != nil && !cfg.Queue.IsDiscovery {
RoutingHandleDNSFunc(cfg, set, ips)
}
}
@ -272,8 +274,10 @@ func (w *Worker) resolveDNSRedirect(ipVersion byte, set *config.SetConfig, cfg *
w.sendDNSResponseToClient(ipVersion, originalDst, clientIP, clientPort, resp)
if set.Routing.Enabled && !set.Targets.DomainOnly && !cfg.Queue.IsDiscovery && RoutingHandleDNSFunc != nil {
if ips := dns.ParseResponseIPs(resp); len(ips) > 0 {
if ips := dns.ParseResponseIPs(resp); len(ips) > 0 {
domain, _ := dns.ParseQueryDomain(query)
w.storeHostHints(clientIP, set, strings.ToLower(domain), ips)
if set.Routing.Enabled && !set.Targets.DomainOnly && !cfg.Queue.IsDiscovery && RoutingHandleDNSFunc != nil {
RoutingHandleDNSFunc(cfg, set, ips)
}
}

View file

@ -120,6 +120,14 @@ func needsTCPSynInjection(set *config.SetConfig) bool {
return set.TCP.SynFake || (hasActiveStrategy && set.Faking.TCPMD5)
}
func needsPayloadlessInjection(set *config.SetConfig) bool {
if set == nil {
return false
}
return set.TCP.DropSACK
}
func (w *Worker) parseIPHeaders(raw []byte) (*pktInfo, bool) {
v := raw[0] >> 4
if v != IPv4 && v != IPv6 {
@ -217,13 +225,28 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
set = nil
}
matchedHint := false
hintHost := ""
if !matched && cfg.IsTCPPort(dport) {
if hintSet, hinted := w.lookupHostHint(cfg, pkt.srcStr, pkt.dstStr, pkt.srcMac); hintSet != nil {
if hintSet.MatchesTCPDPort(dport) {
matched = true
set = hintSet
matchedHint = true
hintHost = hinted
}
}
}
matchedLearned := false
if mLearned, learnedSet, _ := matcher.MatchLearnedIPWithSource(pkt.dst, pkt.srcMac); mLearned {
if learnedSet.MatchesTCPDPort(dport) {
matched = true
set = learnedSet
st = learnedSet
matchedLearned = true
if !matchedHint {
if mLearned, learnedSet, _ := matcher.MatchLearnedIPWithSource(pkt.dst, pkt.srcMac); mLearned {
if learnedSet.MatchesTCPDPort(dport) {
matched = true
set = learnedSet
st = learnedSet
matchedLearned = true
}
}
}
@ -296,7 +319,7 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
if set.TCP.SynFake {
w.sendFakeSyn(set, pkt.raw, pkt.ihl, datOff)
}
if set.Fragmentation.Strategy != config.ConfigNone && set.Faking.TCPMD5 {
if set.Faking.TCPMD5 {
w.sendFakeSynWithMD5(set, pkt.raw, pkt.ihl, pkt.dst)
}
_ = w.sock.SendIPv4(pkt.raw, pkt.dst)
@ -304,7 +327,7 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
if set.TCP.SynFake {
w.sendFakeSynV6(set, pkt.raw, pkt.ihl, datOff)
}
if set.Fragmentation.Strategy != config.ConfigNone && set.Faking.TCPMD5 {
if set.Faking.TCPMD5 {
w.sendFakeSynWithMD5V6(set, pkt.raw, pkt.dst)
}
_ = w.sock.SendIPv6(pkt.raw, pkt.dst)
@ -326,6 +349,7 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
matchedSNI := false
ipTarget := ""
sniTarget := ""
classifyReason := ""
if !matchedIP && matched && set != nil {
ipTarget = set.Name
@ -340,6 +364,21 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
connKey := fmt.Sprintf(connKeyFormat, pkt.srcStr, sport, pkt.dstStr, dport)
host, tlsVersion, _ = sni.ParseTLSClientHelloSNI(payload)
if host == "" {
seq := binary.BigEndian.Uint32(tcp[4:8])
if joined, prefix, ok := w.pendingHello.Feed(connKey, seq, payload); ok {
if joinedHost, joinedTLS, _ := sni.ParseTLSClientHelloSNI(joined); joinedHost != "" {
host = joinedHost
tlsVersion = joinedTLS
classifyReason = "split-hello"
w.pendingHello.Drop(connKey)
log.Tracef("recovered SNI %q for %s:%d from split ClientHello (%d buffered + %d bytes)",
host, pkt.dstStr, dport, prefix, len(payload))
}
}
}
isClientHello = host != ""
if host != "" && tlsVersion != 0 {
@ -373,6 +412,14 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
set = nil
}
}
if matchedHint && !matchedSNI && isClientHello && host != "" {
log.Tracef("host hint for %s dropped: %s carries a clear SNI that matches no set", pkt.dstStr, host)
matched = false
set = nil
matchedHint = false
hintHost = ""
}
}
if host == "" || tlsVersion == 0 {
@ -387,6 +434,13 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
}
}
if host == "" && hintHost != "" {
host = hintHost
}
if matchedHint && !matchedSNI && classifyReason == "" {
classifyReason = "dns-hint"
}
if matchedSNI {
sniTarget = set.Name
} else if matchedIP {
@ -435,7 +489,7 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
}
if !cfg.Queue.IsDiscovery {
log.LogConnection("TCP", sniTarget, host, pkt.srcStr, sport, ipTarget, pkt.dstStr, dport, pkt.srcMac, config.TLSVersionString(tlsVersion), "")
log.LogConnection("TCP", sniTarget, host, pkt.srcStr, sport, ipTarget, pkt.dstStr, dport, pkt.srcMac, config.TLSVersionString(tlsVersion), classifyReason)
}
{
@ -542,6 +596,10 @@ func (w *Worker) handleTCPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
return vc.accept()
}
if len(payload) == 0 && !needsPayloadlessInjection(set) {
return vc.accept()
}
packetCopy := make([]byte, len(pkt.raw))
copy(packetCopy, pkt.raw)
@ -656,6 +714,7 @@ func (w *Worker) handleUDPPacket(vc *verdictCtx, pkt *pktInfo, cfg *config.Confi
sniTarget = sniSet.Name
matcher.LearnIPToDomain(pkt.dst, host, sniSet)
registerLearnedRoute(cfg, sniSet, pkt.dst)
w.storeHostHint(pkt.srcStr, pkt.dstStr, sniSet, host, "quic")
}
}
}

236
src/nfq/hosthint.go Normal file
View file

@ -0,0 +1,236 @@
package nfq
import (
"net"
"sync"
"time"
"github.com/daniellavrushin/b4/config"
"github.com/daniellavrushin/b4/log"
"github.com/daniellavrushin/b4/sni"
)
const (
maxHostHintEntries = 4096
maxHostHintCandidates = 4
hostHintTTL = 120 * time.Second
)
type hostHintCandidate struct {
setId string
host string
expires time.Time
}
type hostHintEntry struct {
candidates []hostHintCandidate
}
type hostHintKey struct {
client string
dest string
}
type hostHintCache struct {
mu sync.RWMutex
keys map[hostHintKey]*hostHintEntry
}
func newHostHintCache() *hostHintCache {
return &hostHintCache{keys: make(map[hostHintKey]*hostHintEntry)}
}
func (c *hostHintCache) Store(clientIP, destIP, setId, host string) {
if c == nil || clientIP == "" || destIP == "" || setId == "" {
return
}
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
key := hostHintKey{client: clientIP, dest: destIP}
entry := c.keys[key]
if entry == nil {
c.evictLocked(now)
if len(c.keys) >= maxHostHintEntries {
return
}
entry = &hostHintEntry{}
c.keys[key] = entry
}
for i := range entry.candidates {
if entry.candidates[i].setId == setId && entry.candidates[i].host == host {
entry.candidates[i].expires = now.Add(hostHintTTL)
return
}
}
entry.candidates = append(pruneCandidates(entry.candidates, now), hostHintCandidate{
setId: setId,
host: host,
expires: now.Add(hostHintTTL),
})
if len(entry.candidates) > maxHostHintCandidates {
entry.candidates = entry.candidates[len(entry.candidates)-maxHostHintCandidates:]
}
}
func (c *hostHintCache) Lookup(clientIP, destIP string) (string, string, bool) {
if c == nil {
return "", "", false
}
now := time.Now()
c.mu.RLock()
defer c.mu.RUnlock()
entry := c.keys[hostHintKey{client: clientIP, dest: destIP}]
if entry == nil {
return "", "", false
}
setId := ""
host := ""
for _, candidate := range entry.candidates {
if !now.Before(candidate.expires) {
continue
}
if setId == "" {
setId = candidate.setId
host = candidate.host
continue
}
if candidate.setId != setId {
log.Tracef("host hint for %s -> %s is ambiguous between sets %s and %s, ignoring",
clientIP, destIP, setId, candidate.setId)
return "", "", false
}
}
if setId == "" {
return "", "", false
}
return setId, host, true
}
func pruneCandidates(candidates []hostHintCandidate, now time.Time) []hostHintCandidate {
live := candidates[:0]
for _, candidate := range candidates {
if now.Before(candidate.expires) {
live = append(live, candidate)
}
}
return live
}
func (c *hostHintCache) evictLocked(now time.Time) {
if len(c.keys) < maxHostHintEntries {
return
}
for key, entry := range c.keys {
entry.candidates = pruneCandidates(entry.candidates, now)
if len(entry.candidates) == 0 {
delete(c.keys, key)
}
}
for len(c.keys) >= maxHostHintEntries {
var oldestKey hostHintKey
var oldestAt time.Time
found := false
for key, entry := range c.keys {
at := entry.candidates[0].expires
for _, candidate := range entry.candidates[1:] {
if candidate.expires.Before(at) {
at = candidate.expires
}
}
if !found || at.Before(oldestAt) {
oldestKey = key
oldestAt = at
found = true
}
}
if !found {
return
}
delete(c.keys, oldestKey)
}
}
func (c *hostHintCache) Cleanup() {
if c == nil {
return
}
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
for key, entry := range c.keys {
entry.candidates = pruneCandidates(entry.candidates, now)
if len(entry.candidates) == 0 {
delete(c.keys, key)
}
}
}
func (w *Worker) storeHostHint(clientIP, destIP string, set *config.SetConfig, host, source string) {
if w == nil || set == nil || set.Id == "" || host == "" || clientIP == "" || destIP == "" {
return
}
w.hostHints.Store(clientIP, destIP, set.Id, host)
log.Tracef("host hint from %s: %s -> %s is %s (set: %s)", source, clientIP, destIP, host, set.Name)
}
func (w *Worker) storeHostHints(clientIP net.IP, set *config.SetConfig, host string, ips []net.IP) {
if w == nil || clientIP == nil || len(ips) == 0 {
return
}
client := clientIP.String()
for _, ip := range ips {
if ip == nil {
continue
}
w.storeHostHint(client, ip.String(), set, host, "dns")
}
}
func (w *Worker) lookupHostHint(cfg *config.Config, clientIP, destIP, srcMac string) (*config.SetConfig, string) {
if w == nil || cfg == nil {
return nil, ""
}
setId, host, ok := w.hostHints.Lookup(clientIP, destIP)
if !ok {
return nil, ""
}
set := cfg.GetSetById(setId)
if set == nil || !set.Enabled {
return nil, ""
}
if set.Targets.DomainOnly {
log.Tracef("host hint for %s -> %s names domain-only set %s, not applied", clientIP, destIP, set.Name)
return nil, ""
}
if !sni.SetMatchesSource(set, srcMac) {
return nil, ""
}
return set, host
}
func (c *hostHintCache) Len() int {
if c == nil {
return 0
}
c.mu.RLock()
defer c.mu.RUnlock()
return len(c.keys)
}

475
src/nfq/hosthint_test.go Normal file
View file

@ -0,0 +1,475 @@
package nfq
import (
"encoding/binary"
"fmt"
"net"
"strings"
"sync"
"testing"
"time"
"github.com/daniellavrushin/b4/config"
"github.com/daniellavrushin/b4/engine"
)
func encodeDNSName(name string) []byte {
var out []byte
for _, label := range strings.Split(name, ".") {
out = append(out, byte(len(label)))
out = append(out, label...)
}
return append(out, 0)
}
func buildDNSResponse(txid uint16, domain string, ips []net.IP) []byte {
msg := make([]byte, 12)
binary.BigEndian.PutUint16(msg[0:2], txid)
binary.BigEndian.PutUint16(msg[2:4], 0x8180)
binary.BigEndian.PutUint16(msg[4:6], 1)
binary.BigEndian.PutUint16(msg[6:8], uint16(len(ips)))
msg = append(msg, encodeDNSName(domain)...)
msg = append(msg, 0x00, 0x01, 0x00, 0x01)
for _, ip := range ips {
msg = append(msg, 0xC0, 0x0C)
msg = append(msg, 0x00, 0x01, 0x00, 0x01)
msg = append(msg, 0x00, 0x00, 0x00, 0x3C)
msg = append(msg, 0x00, 0x04)
msg = append(msg, ip.To4()...)
}
return msg
}
func makeV4UDPPacket(payload []byte, srcIP, dstIP net.IP, sport, dport uint16) []byte {
const ipHL = 20
pkt := make([]byte, ipHL+8+len(payload))
pkt[0] = 0x45
binary.BigEndian.PutUint16(pkt[2:4], uint16(len(pkt)))
pkt[8] = 64
pkt[9] = 17
copy(pkt[12:16], srcIP.To4())
copy(pkt[16:20], dstIP.To4())
binary.BigEndian.PutUint16(pkt[ipHL:ipHL+2], sport)
binary.BigEndian.PutUint16(pkt[ipHL+2:ipHL+4], dport)
binary.BigEndian.PutUint16(pkt[ipHL+4:ipHL+6], uint16(8+len(payload)))
copy(pkt[ipHL+8:], payload)
return pkt
}
func makeV4TCPPacketFlags(payload []byte, seq uint32, flags byte) []byte {
pkt := makeV4TCPPacket(payload, seq)
pkt[20+13] = flags
return pkt
}
func newHintSet() config.SetConfig {
set := newPassiveSet()
set.Targets.DomainOnly = false
return set
}
func TestHostHintStoreAndLookup(t *testing.T) {
c := newHostHintCache()
c.Store("10.0.0.1", "1.2.3.4", "video", "rr1.googlevideo.com")
setId, host, ok := c.Lookup("10.0.0.1", "1.2.3.4")
if !ok {
t.Fatal("stored hint should be found")
}
if setId != "video" || host != "rr1.googlevideo.com" {
t.Fatalf("hint: got setId=%q host=%q", setId, host)
}
}
func TestHostHintIsSourceScoped(t *testing.T) {
c := newHostHintCache()
c.Store("10.0.0.1", "1.2.3.4", "video", "rr1.googlevideo.com")
if _, _, ok := c.Lookup("10.0.0.2", "1.2.3.4"); ok {
t.Fatal("another client must not see the first client's hint")
}
if _, _, ok := c.Lookup("10.0.0.1", "5.6.7.8"); ok {
t.Fatal("another destination must not reuse the hint")
}
}
func TestHostHintSameSetSeveralHosts(t *testing.T) {
c := newHostHintCache()
c.Store("10.0.0.1", "1.2.3.4", "yt", "i.ytimg.com")
c.Store("10.0.0.1", "1.2.3.4", "yt", "s.ytimg.com")
setId, _, ok := c.Lookup("10.0.0.1", "1.2.3.4")
if !ok || setId != "yt" {
t.Fatalf("hostnames agreeing on one set should resolve, got setId=%q ok=%v", setId, ok)
}
}
func TestHostHintAmbiguousBetweenSets(t *testing.T) {
c := newHostHintCache()
c.Store("10.0.0.1", "1.2.3.4", "images", "i.ytimg.com")
c.Store("10.0.0.1", "1.2.3.4", "video", "rr1.googlevideo.com")
if setId, _, ok := c.Lookup("10.0.0.1", "1.2.3.4"); ok {
t.Fatalf("hostnames pointing at different sets must not resolve, got %q", setId)
}
}
func TestHostHintRepeatedStoreIsNotAmbiguous(t *testing.T) {
c := newHostHintCache()
for i := 0; i < 10; i++ {
c.Store("10.0.0.1", "1.2.3.4", "yt", "i.ytimg.com")
}
if _, _, ok := c.Lookup("10.0.0.1", "1.2.3.4"); !ok {
t.Fatal("repeating the same evidence must stay resolvable")
}
}
func TestHostHintExpiry(t *testing.T) {
c := newHostHintCache()
c.Store("10.0.0.1", "1.2.3.4", "yt", "i.ytimg.com")
c.mu.Lock()
entry := c.keys[hostHintKey{client: "10.0.0.1", dest: "1.2.3.4"}]
entry.candidates[0].expires = time.Now().Add(-time.Second)
c.mu.Unlock()
if _, _, ok := c.Lookup("10.0.0.1", "1.2.3.4"); ok {
t.Fatal("expired hint must not resolve")
}
if c.Len() != 1 {
t.Fatalf("Lookup must not mutate the cache, entries=%d", c.Len())
}
c.Cleanup()
if c.Len() != 0 {
t.Fatalf("Cleanup should drop the expired key, entries=%d", c.Len())
}
}
func TestHostHintCleanup(t *testing.T) {
c := newHostHintCache()
c.Store("10.0.0.1", "1.2.3.4", "yt", "i.ytimg.com")
c.Store("10.0.0.2", "1.2.3.4", "yt", "i.ytimg.com")
c.mu.Lock()
for _, entry := range c.keys {
entry.candidates[0].expires = time.Now().Add(-time.Second)
}
c.mu.Unlock()
c.Cleanup()
if c.Len() != 0 {
t.Fatalf("Cleanup should drop expired keys, entries=%d", c.Len())
}
}
func TestHostHintCandidateCap(t *testing.T) {
c := newHostHintCache()
for i := 0; i < maxHostHintCandidates*3; i++ {
c.Store("10.0.0.1", "1.2.3.4", "yt", fmt.Sprintf("host%d.ytimg.com", i))
}
c.mu.Lock()
got := len(c.keys[hostHintKey{client: "10.0.0.1", dest: "1.2.3.4"}].candidates)
c.mu.Unlock()
if got > maxHostHintCandidates {
t.Fatalf("candidate cap exceeded: %d", got)
}
}
func TestHostHintEntryCap(t *testing.T) {
c := newHostHintCache()
for i := 0; i < maxHostHintEntries+128; i++ {
c.Store(fmt.Sprintf("10.%d.%d.%d", i>>16&0xff, i>>8&0xff, i&0xff), "1.2.3.4", "yt", "i.ytimg.com")
}
if c.Len() > maxHostHintEntries {
t.Fatalf("entry cap exceeded: %d", c.Len())
}
}
func TestHostHintNilReceiver(t *testing.T) {
var c *hostHintCache
c.Store("10.0.0.1", "1.2.3.4", "yt", "i.ytimg.com")
if _, _, ok := c.Lookup("10.0.0.1", "1.2.3.4"); ok {
t.Fatal("nil cache must not resolve")
}
c.Cleanup()
if c.Len() != 0 {
t.Fatal("nil cache must report zero entries")
}
}
func TestHostHintConcurrentReadersAndWriters(t *testing.T) {
c := newHostHintCache()
c.Store("10.0.0.1", "1.2.3.4", "yt", "i.ytimg.com")
var wg sync.WaitGroup
for w := 0; w < 4; w++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
for i := 0; i < 2000; i++ {
c.Store(fmt.Sprintf("10.0.%d.%d", id, i&0xff), "1.2.3.4", "yt", "i.ytimg.com")
}
}(w)
}
for r := 0; r < 4; r++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 2000; i++ {
c.Lookup("10.0.0.1", "1.2.3.4")
c.Lookup("10.0.0.1", "9.9.9.9")
}
}()
}
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 200; i++ {
c.Cleanup()
}
}()
wg.Wait()
if _, _, ok := c.Lookup("10.0.0.1", "1.2.3.4"); !ok {
t.Fatal("the original hint should have survived concurrent access")
}
}
func BenchmarkHostHintLookupMiss(b *testing.B) {
c := newHostHintCache()
c.Store("10.0.0.9", "9.9.9.9", "yt", "i.ytimg.com")
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
c.Lookup("10.0.0.1", "1.2.3.4")
}
}
func BenchmarkHostHintLookupMissIPv6(b *testing.B) {
c := newHostHintCache()
c.Store("2001:4860:4860::8888", "2606:4700:4700::1111", "yt", "i.ytimg.com")
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
c.Lookup("2001:db8:85a3::8a2e:370:7334", "2606:4700:4700::1001")
}
}
func BenchmarkHostHintLookupMissParallel(b *testing.B) {
c := newHostHintCache()
c.Store("10.0.0.9", "9.9.9.9", "yt", "i.ytimg.com")
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
c.Lookup("10.0.0.1", "1.2.3.4")
}
})
}
func TestLookupHostHintRefusesDomainOnlySet(t *testing.T) {
set := newHintSet()
set.Targets.DomainOnly = true
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
w.hostHints.Store("10.0.0.1", "1.2.3.4", set.Id, "i.ytimg.com")
if got, _ := w.lookupHostHint(&cfg, "10.0.0.1", "1.2.3.4", ""); got != nil {
t.Fatalf("a domain-only set must not be selected from a DNS hint, got %q", got.Name)
}
}
func TestLookupHostHintRefusesDisabledSet(t *testing.T) {
set := newHintSet()
set.Enabled = false
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
w.hostHints.Store("10.0.0.1", "1.2.3.4", set.Id, "i.ytimg.com")
if got, _ := w.lookupHostHint(&cfg, "10.0.0.1", "1.2.3.4", ""); got != nil {
t.Fatalf("a disabled set must not be selected, got %q", got.Name)
}
}
func TestLookupHostHintRefusesStaleSetId(t *testing.T) {
set := newHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
w.hostHints.Store("10.0.0.1", "1.2.3.4", "set-removed-by-a-config-reload", "i.ytimg.com")
if got, _ := w.lookupHostHint(&cfg, "10.0.0.1", "1.2.3.4", ""); got != nil {
t.Fatalf("a hint naming a set that no longer exists must not resolve, got %q", got.Name)
}
}
func TestLookupHostHintChecksSourceDevices(t *testing.T) {
set := newHintSet()
set.Targets.SourceDevices = []string{"AA:BB:CC:DD:EE:FF"}
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
w.hostHints.Store("10.0.0.1", "1.2.3.4", set.Id, "i.ytimg.com")
if got, _ := w.lookupHostHint(&cfg, "10.0.0.1", "1.2.3.4", "11:22:33:44:55:66"); got != nil {
t.Fatalf("a set restricted to other devices must not be selected, got %q", got.Name)
}
if got, _ := w.lookupHostHint(&cfg, "10.0.0.1", "1.2.3.4", "aa:bb:cc:dd:ee:ff"); got == nil {
t.Fatal("the permitted device should be selected")
}
}
func TestDNSResponseFeedsHostHint(t *testing.T) {
set := newHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
resp := buildDNSResponse(0x1234, "i.ytimg.com", []net.IP{net.ParseIP("1.2.3.4")})
pkt := makeV4UDPPacket(resp, net.ParseIP("8.8.8.8"), net.ParseIP("10.0.0.1"), 53, 5353)
w.ProcessPacket(pkt)
setId, host, ok := w.hostHints.Lookup("10.0.0.1", "1.2.3.4")
if !ok {
t.Fatal("a DNS answer for a matched domain should leave a hint for the client")
}
if setId != set.Id || host != "i.ytimg.com" {
t.Fatalf("hint: got setId=%q host=%q", setId, host)
}
}
func TestFirstFlowClassifiesFromDNSHint(t *testing.T) {
set := newHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
resp := buildDNSResponse(0x1234, "i.ytimg.com", []net.IP{net.ParseIP("1.2.3.4")})
w.ProcessPacket(makeV4UDPPacket(resp, net.ParseIP("8.8.8.8"), net.ParseIP("10.0.0.1"), 53, 5353))
w.ProcessPacket(makeV4TCPPacket([]byte("GET / HTTP/1.1\r\n\r\n"), 1000))
bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443)
if bound == nil {
t.Fatal("the first flow to a resolved address should inherit the set from the DNS hint")
}
if bound.Id != set.Id {
t.Fatalf("bound set: want %q, got %q", set.Id, bound.Id)
}
}
func TestClearSNIForAnotherDomainCancelsHint(t *testing.T) {
set := newHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
resp := buildDNSResponse(0x1234, "i.ytimg.com", []net.IP{net.ParseIP("1.2.3.4")})
w.ProcessPacket(makeV4UDPPacket(resp, net.ParseIP("8.8.8.8"), net.ParseIP("10.0.0.1"), 53, 5353))
unrelated := buildClientHello("maps.example.org", 16, 0xAB)
w.ProcessPacket(makeV4TCPPacket(unrelated, 1000))
if bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443); bound != nil {
t.Fatalf("a clear SNI matching no set must cancel the hint, got %q", bound.Name)
}
}
func TestClearSNIForHintedDomainKeepsSet(t *testing.T) {
set := newHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
resp := buildDNSResponse(0x1234, "i.ytimg.com", []net.IP{net.ParseIP("1.2.3.4")})
w.ProcessPacket(makeV4UDPPacket(resp, net.ParseIP("8.8.8.8"), net.ParseIP("10.0.0.1"), 53, 5353))
hello := buildClientHello("i.ytimg.com", 16, 0xAB)
w.ProcessPacket(makeV4TCPPacket(hello, 1000))
bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443)
if bound == nil || bound.Id != set.Id {
t.Fatal("a clear SNI matching the set must keep it bound")
}
}
func TestFirstFlowUnclassifiedWithoutHint(t *testing.T) {
set := newHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
w.ProcessPacket(makeV4TCPPacket([]byte("GET / HTTP/1.1\r\n\r\n"), 1000))
if bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443); bound != nil {
t.Fatalf("without DNS evidence the flow must stay unclassified, got %q", bound.Name)
}
}
func TestNeedsPayloadlessInjection(t *testing.T) {
set := config.NewSetConfig()
set.TCP.DropSACK = false
if needsPayloadlessInjection(&set) {
t.Fatal("a set without SACK stripping has no work to do on a payload-less packet")
}
set.TCP.DropSACK = true
if !needsPayloadlessInjection(&set) {
t.Fatal("SACK stripping must still process payload-less packets")
}
if needsPayloadlessInjection(nil) {
t.Fatal("nil set must not request injection")
}
}
func TestCleanSynIsAcceptedWithoutSynTechnique(t *testing.T) {
set := config.NewSetConfig()
set.Id = "yt-video"
set.Name = "YT video"
set.Enabled = true
set.Targets.IpsToMatch = []string{"1.2.3.4"}
set.TCP.SynFake = false
set.Faking.TCPMD5 = false
set.TCP.DropSACK = false
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
if needsTCPSynInjection(&set) {
t.Fatal("fixture should not request an explicit SYN technique")
}
if !needsTCPInjection(&set) {
t.Fatal("fixture should request payload injection so the clean-SYN path is reached")
}
syn := makeV4TCPPacketFlags(nil, 1000, 0x02)
if v := w.ProcessPacket(syn); v != engine.VerdictAccept {
t.Fatalf("a clean SYN must pass through untouched, got %v", v)
}
fin := makeV4TCPPacketFlags(nil, 2000, 0x11)
if v := w.ProcessPacket(fin); v != engine.VerdictAccept {
t.Fatalf("a payload-less FIN must pass through untouched, got %v", v)
}
}

View file

@ -429,6 +429,12 @@ func (w *Worker) sendFakeSNISequence(cfg *config.SetConfig, original []byte, dst
}
fake := sock.BuildFakeSNIPacketV4(original, cfg)
if fake == nil {
return
}
if fk.MD5OnFake {
fake = sock.AddTCPMD5Option(fake, false)
}
ipHdrLen := int((fake[0] & 0x0F) * 4)
tcpHdrLen := int((fake[ipHdrLen+12] >> 4) * 4)

View file

@ -337,6 +337,9 @@ func (w *Worker) sendFakeSNISequencev6(cfg *config.SetConfig, original []byte, d
if fake == nil {
return
}
if faking.MD5OnFake {
fake = sock.AddTCPMD5Option(fake, true)
}
ipv6HdrLen := 40

View file

@ -73,6 +73,8 @@ func NewPool(cfg *config.Config) *Pool {
w.tlsCache = state.tlsCache
w.connTracker = state.connState
w.destState = state.destState
w.pendingHello = state.pendingHello
w.hostHints = state.hostHints
ws = append(ws, w)
}
@ -105,7 +107,9 @@ func NewPool(cfg *config.Config) *Pool {
pool.state.connState.Cleanup()
pool.state.tlsCache.Cleanup()
pool.state.destState.Cleanup(300 * time.Second)
pool.state.hostHints.Cleanup()
case <-escalationTicker.C:
pool.state.pendingHello.Cleanup()
metrics.GetMetricsCollector().UpdateEscalations(pool.GetEscalations())
case <-pool.stopCleanup:
return

245
src/nfq/quichint_test.go Normal file
View file

@ -0,0 +1,245 @@
package nfq
import (
"crypto/aes"
"crypto/cipher"
"crypto/hmac"
"crypto/sha256"
"encoding/binary"
"io"
"net"
"testing"
"github.com/daniellavrushin/b4/config"
"github.com/daniellavrushin/b4/sni"
"golang.org/x/crypto/hkdf"
)
var quicSaltV1 = []byte{
0x38, 0x76, 0x2c, 0xf7, 0xf5, 0x59, 0x34, 0xb3, 0x4d, 0x17,
0x9a, 0xe6, 0xa4, 0xc8, 0x0c, 0xad, 0xcc, 0xbb, 0x7f, 0x0a,
}
func quicExpandLabel(t *testing.T, secret []byte, label string, outLen int) []byte {
t.Helper()
full := "tls13 " + label
info := make([]byte, 2+1+len(full)+1)
info[0] = byte(outLen >> 8)
info[1] = byte(outLen)
info[2] = byte(len(full))
copy(info[3:], full)
out := make([]byte, outLen)
if _, err := io.ReadFull(hkdf.Expand(sha256.New, secret, info), out); err != nil {
t.Fatalf("hkdf expand %q: %v", label, err)
}
return out
}
func quicInitialKeys(t *testing.T, dcid []byte) (cipher.AEAD, cipher.Block, []byte) {
t.Helper()
extract := hmac.New(sha256.New, quicSaltV1)
_, _ = extract.Write(dcid)
clientSecret := quicExpandLabel(t, extract.Sum(nil), "client in", 32)
block, err := aes.NewCipher(quicExpandLabel(t, clientSecret, "quic key", 16))
if err != nil {
t.Fatalf("aes key: %v", err)
}
aead, err := cipher.NewGCM(block)
if err != nil {
t.Fatalf("gcm: %v", err)
}
hp, err := aes.NewCipher(quicExpandLabel(t, clientSecret, "quic hp", 16))
if err != nil {
t.Fatalf("aes hp: %v", err)
}
return aead, hp, quicExpandLabel(t, clientSecret, "quic iv", 12)
}
func buildClientHelloHandshake(host string) []byte {
record := buildClientHello(host, 16, 0xAB)
return record[5:]
}
func encodeQUICVarint(v uint64) []byte {
switch {
case v < 64:
return []byte{byte(v)}
case v < 16384:
out := make([]byte, 2)
binary.BigEndian.PutUint16(out, uint16(v)|0x4000)
return out
default:
out := make([]byte, 4)
binary.BigEndian.PutUint32(out, uint32(v)|0x80000000)
return out
}
}
func buildQUICInitialWithSNI(t *testing.T, dcid []byte, host string) []byte {
t.Helper()
aead, hp, iv := quicInitialKeys(t, dcid)
hello := buildClientHelloHandshake(host)
frame := []byte{0x06}
frame = append(frame, encodeQUICVarint(0)...)
frame = append(frame, encodeQUICVarint(uint64(len(hello)))...)
frame = append(frame, hello...)
header := []byte{0xC0}
header = append(header, 0x00, 0x00, 0x00, 0x01)
header = append(header, byte(len(dcid)))
header = append(header, dcid...)
header = append(header, 0x00)
header = append(header, 0x00)
header = append(header, encodeQUICVarint(uint64(1+len(frame)+aead.Overhead()))...)
pnOff := len(header)
aad := append(append([]byte{}, header...), 0x00)
nonce := append([]byte{}, iv...)
ciphertext := aead.Seal(nil, nonce, frame, aad)
packet := append(append([]byte{}, aad...), ciphertext...)
if pnOff+4+16 > len(packet) {
t.Fatalf("packet too short for header protection sample: %d", len(packet))
}
var mask [16]byte
hp.Encrypt(mask[:], packet[pnOff+4:pnOff+4+16])
packet[0] ^= mask[0] & 0x0f
packet[pnOff] ^= mask[1]
return packet
}
func TestQUICInitialFixtureIsParseable(t *testing.T) {
pkt := buildQUICInitialWithSNI(t, []byte{1, 2, 3, 4, 5, 6, 7, 8}, "rr1.googlevideo.com")
host, ok := sni.ParseQUICClientHelloSNI(pkt)
if !ok {
t.Fatal("fixture must be a decryptable QUIC Initial carrying a ClientHello")
}
if host != "rr1.googlevideo.com" {
t.Fatalf("fixture SNI: want rr1.googlevideo.com, got %q", host)
}
}
func newQUICHintSet() config.SetConfig {
set := config.NewSetConfig()
set.Id = "yt-video"
set.Name = "YT video"
set.Enabled = true
set.Targets.DomainsToMatch = []string{"googlevideo.com"}
set.TCP.RSTProtection.Enabled = true
set.Fragmentation.Strategy = config.ConfigNone
set.Fragmentation.StrategyPool = nil
set.Faking.SNI = false
set.Faking.SNIMutation.Mode = config.ConfigOff
set.TCP.Desync.Mode = config.ConfigOff
set.TCP.Win.Mode = config.ConfigOff
set.TCP.DropSACK = false
set.UDP.Mode = config.ConfigNone
return set
}
func TestQUICSNILeavesSourceScopedHint(t *testing.T) {
set := newQUICHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
initial := buildQUICInitialWithSNI(t, []byte{9, 9, 9, 9, 1, 1, 1, 1}, "rr1.googlevideo.com")
pkt := makeV4UDPPacket(initial, net.ParseIP("10.0.0.1"), net.ParseIP("1.2.3.4"), 51000, 443)
w.ProcessPacket(pkt)
setId, host, ok := w.hostHints.Lookup("10.0.0.1", "1.2.3.4")
if !ok {
t.Fatal("a QUIC ClientHello with a clear SNI should leave a hint for the client")
}
if setId != set.Id || host != "rr1.googlevideo.com" {
t.Fatalf("hint: got setId=%q host=%q", setId, host)
}
if _, _, other := w.hostHints.Lookup("10.0.0.2", "1.2.3.4"); other {
t.Fatal("the QUIC hint must not leak to another client")
}
}
func TestSourceScopedHintBeatsGlobalLearnedIP(t *testing.T) {
images := newHintSet()
images.Id = "yt-images"
images.Name = "YT images"
images.Targets.DomainsToMatch = []string{"ytimg.com"}
video := newQUICHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&images, &video}
w := newTestWorker(t, &cfg)
initial := buildQUICInitialWithSNI(t, []byte{3, 3, 3, 3, 4, 4, 4, 4}, "rr1.googlevideo.com")
w.ProcessPacket(makeV4UDPPacket(initial, net.ParseIP("10.0.0.2"), net.ParseIP("1.2.3.4"), 51000, 443))
if _, learned, _ := w.getMatcher().MatchLearnedIPWithSource(net.ParseIP("1.2.3.4"), ""); learned == nil || learned.Id != video.Id {
t.Fatal("the other client's QUIC attempt should have populated the global learned-IP cache")
}
resp := buildDNSResponse(0x1234, "i.ytimg.com", []net.IP{net.ParseIP("1.2.3.4")})
w.ProcessPacket(makeV4UDPPacket(resp, net.ParseIP("8.8.8.8"), net.ParseIP("10.0.0.1"), 53, 5353))
w.ProcessPacket(makeV4TCPPacket([]byte("GET / HTTP/1.1\r\n\r\n"), 1000))
bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443)
if bound == nil {
t.Fatal("the flow should have been classified")
}
if bound.Id != images.Id {
t.Fatalf("a client's own domain evidence must win over another client's learned IP: want %q, got %q", images.Id, bound.Id)
}
}
func TestLearnedIPStillAppliesWithoutHint(t *testing.T) {
video := newQUICHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&video}
w := newTestWorker(t, &cfg)
initial := buildQUICInitialWithSNI(t, []byte{5, 5, 5, 5, 6, 6, 6, 6}, "rr1.googlevideo.com")
w.ProcessPacket(makeV4UDPPacket(initial, net.ParseIP("10.0.0.2"), net.ParseIP("1.2.3.4"), 51000, 443))
unrelated := buildClientHello("maps.example.org", 16, 0xAB)
w.ProcessPacket(makeV4TCPPacket(unrelated, 1000))
bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443)
if bound == nil || bound.Id != video.Id {
t.Fatal("with no hint for this client the learned-IP match must still apply")
}
}
func TestQUICToTCPHandoff(t *testing.T) {
set := newQUICHintSet()
cfg := config.NewConfig()
cfg.Sets = []*config.SetConfig{&set}
w := newTestWorker(t, &cfg)
initial := buildQUICInitialWithSNI(t, []byte{7, 7, 7, 7, 2, 2, 2, 2}, "rr1.googlevideo.com")
w.ProcessPacket(makeV4UDPPacket(initial, net.ParseIP("10.0.0.1"), net.ParseIP("1.2.3.4"), 51000, 443))
w.ProcessPacket(makeV4TCPPacket([]byte("GET / HTTP/1.1\r\n\r\n"), 1000))
bound := w.connTracker.GetSetForIncoming("10.0.0.1", 12345, "1.2.3.4", 443)
if bound == nil {
t.Fatal("the TCP flow following a rejected QUIC attempt should inherit the set")
}
if bound.Id != set.Id {
t.Fatalf("bound set: want %q, got %q", set.Id, bound.Id)
}
}

View file

@ -8,6 +8,45 @@ import (
"github.com/daniellavrushin/b4/sock"
)
// resolveTLSSplit picks the byte offset inside the TCP payload to split at.
// TLSRecordPosition is measured from the end of the 5-byte record header, which
// is what makes position 1 land right after it. When MiddleSNI is set the split
// is placed inside the SNI itself, the equivalent of byedpi's "+s" offset flag.
func resolveTLSSplit(cfg *config.SetConfig, payload []byte, payloadLen int) int {
split := 0
if cfg.Fragmentation.MiddleSNI {
if s, e, ok := locateSNI(payload); ok && e > s {
split = s + (e-s)/2
}
}
if split <= 0 {
pos := config.ResolveRange(cfg.Fragmentation.TLSRecordPosition, cfg.Fragmentation.TLSRecordPositionMax)
if pos <= 0 {
pos = 1
}
split = 5 + pos
}
if split >= payloadLen {
split = payloadLen / 2
}
if split < 6 {
split = 6
}
if split >= payloadLen {
split = payloadLen - 1
}
return split
}
// sendTLSFragments splits the TCP segment at an offset measured from the end of
// the TLS record header, or at the SNI when MiddleSNI is set. This is a TCP-level
// split at a record-relative position, not a rewrite into two TLS records: b4
// operates on a live kernel flow, and inserting a second 5-byte record header
// would shift every following sequence number in the connection.
func (w *Worker) sendTLSFragments(cfg *config.SetConfig, packet []byte, dst net.IP) {
ipHdrLen := int((packet[0] & 0x0F) * 4)
tcpHdrLen := int((packet[ipHdrLen+12] >> 4) * 4)
@ -20,19 +59,7 @@ func (w *Worker) sendTLSFragments(cfg *config.SetConfig, packet []byte, dst net.
return
}
splitPos := config.ResolveRange(cfg.Fragmentation.TLSRecordPosition, cfg.Fragmentation.TLSRecordPositionMax)
if splitPos <= 0 {
splitPos = 1
}
absoluteSplit := 5 + splitPos
if absoluteSplit >= payloadLen {
absoluteSplit = payloadLen / 2
}
if absoluteSplit < 6 {
absoluteSplit = 6
}
absoluteSplit := resolveTLSSplit(cfg, payload, payloadLen)
seg1Len := payloadStart + absoluteSplit
seg1 := make([]byte, seg1Len)
@ -72,19 +99,7 @@ func (w *Worker) sendTLSFragmentsV6(cfg *config.SetConfig, packet []byte, dst ne
return
}
splitPos := config.ResolveRange(cfg.Fragmentation.TLSRecordPosition, cfg.Fragmentation.TLSRecordPositionMax)
if splitPos <= 0 {
splitPos = 1
}
absoluteSplit := 5 + splitPos
if absoluteSplit >= payloadLen {
absoluteSplit = payloadLen / 2
}
if absoluteSplit < 6 {
absoluteSplit = 6
}
absoluteSplit := resolveTLSSplit(cfg, payload, payloadLen)
seg1Len := payloadStart + absoluteSplit
seg1 := make([]byte, seg1Len)

View file

@ -51,5 +51,7 @@ type Worker struct {
tlsCache *tlsInfoCache
connTracker *connStateTracker
destState *destStateTracker
pendingHello *pendingHelloCache
hostHints *hostHintCache
srcResolver *tunSrcResolver
}

View file

@ -582,6 +582,13 @@ func setMatchesSource(set *config.SetConfig, srcMAC string) bool {
return exclude
}
func SetMatchesSource(set *config.SetConfig, srcMAC string) bool {
if set == nil {
return false
}
return setMatchesSource(set, srcMAC)
}
func (s *SuffixSet) MatchSNIWithSource(host string, srcMAC string) (bool, *config.SetConfig) {
return s.MatchSNIWithSourceTLS(host, srcMAC, 0, 0)
}

View file

@ -125,6 +125,10 @@ func ParseTLSClientHelloSNI(b []byte) (string, uint16, bool) {
return "", 0, false
}
func IsValidSNI(name []byte) bool {
return validateSNI(string(name))
}
func ParseTLSClientHelloBodySNI(ch []byte) (string, bool) {
sni, _, _, _ := parseTLSClientHelloMeta(ch)
if sni == "" {

View file

@ -36,31 +36,27 @@ func BuildFakeSNIPacketV4(original []byte, cfg *config.SetConfig) []byte {
fakePayload = ApplyTLSMod(fakePayload, originalTLS, flags)
}
fakePayload = MatchPayloadLength(fakePayload, originalTLS, cfg.Faking.FakeLenMode)
fakeLen := ipHdrLen + tcpHdrLen + len(fakePayload)
fake := make([]byte, fakeLen)
copy(fake[:ipHdrLen+tcpHdrLen], original[:ipHdrLen+tcpHdrLen])
copy(fake[ipHdrLen+tcpHdrLen:], fakePayload)
binary.BigEndian.PutUint16(fake[2:4], uint16(fakeLen))
setDistinctIPID(fake, original)
off := cfg.Faking.SeqOffset
if off <= 0 {
off = 10000
}
if cfg.Faking.ApplyTTL || cfg.Faking.Strategy == "ttl" {
fake[8] = resolveFakeTTL(cfg.Faking.TTL, original[8])
}
switch cfg.Faking.Strategy {
case "ttl":
ttl := cfg.Faking.TTL
if ttl == 0 {
ttl = 5
}
if origTTL := original[8]; ttl >= origTTL && origTTL > 1 {
ttl = origTTL - 1
}
if ttl < 1 {
ttl = 1
}
fake[8] = ttl
case "pastseq":
off := uint32(cfg.Faking.SeqOffset)
if off == 0 {

View file

@ -37,6 +37,8 @@ func BuildFakeSNIPacketV6(original []byte, cfg *config.SetConfig) []byte {
fakePayload = ApplyTLSMod(fakePayload, originalTLS, flags)
}
fakePayload = MatchPayloadLength(fakePayload, originalTLS, cfg.Faking.FakeLenMode)
fakeLen := ipv6HdrLen + tcpHdrLen + len(fakePayload)
fake := make([]byte, fakeLen)
copy(fake[:ipv6HdrLen+tcpHdrLen], original[:ipv6HdrLen+tcpHdrLen])
@ -51,19 +53,12 @@ func BuildFakeSNIPacketV6(original []byte, cfg *config.SetConfig) []byte {
off = 10000
}
if cfg.Faking.ApplyTTL || cfg.Faking.Strategy == "ttl" {
fake[7] = resolveFakeTTL(cfg.Faking.TTL, original[7])
}
switch cfg.Faking.Strategy {
case "ttl":
ttl := cfg.Faking.TTL
if ttl == 0 {
ttl = 5
}
if origHL := original[7]; ttl >= origHL && origHL > 1 {
ttl = origHL - 1
}
if ttl < 1 {
ttl = 1
}
fake[7] = ttl
case "pastseq":
off := uint32(cfg.Faking.SeqOffset)
if off == 0 {

View file

@ -13,6 +13,62 @@ func cloneBytes(src []byte) []byte {
return dst
}
func MatchPayloadLength(fakePayload, originalTLS []byte, mode string) []byte {
if mode != "match" || len(originalTLS) == 0 || len(fakePayload) == 0 {
return fakePayload
}
target := len(originalTLS)
if len(fakePayload) == target {
return fakePayload
}
out := make([]byte, target)
for i := 0; i < target; i++ {
out[i] = fakePayload[i%len(fakePayload)]
}
fixTLSRecordLength(out)
return out
}
func fixTLSRecordLength(payload []byte) {
if len(payload) < 5 || payload[0] != 0x16 {
return
}
payload[3] = byte((len(payload) - 5) >> 8)
payload[4] = byte(len(payload) - 5)
}
func resolveFakeTTL(configured, originalTTL uint8) uint8 {
ttl := configured
if ttl == 0 {
ttl = 5
}
if ttl >= originalTTL && originalTTL > 1 {
ttl = originalTTL - 1
}
if ttl < 1 {
ttl = 1
}
return ttl
}
func setDistinctIPID(fake, original []byte) {
if len(fake) < 6 || len(original) < 6 {
return
}
var r [2]byte
if _, err := rand.Read(r[:]); err != nil {
log.Warnf("crypto/rand read failed: %v", err)
return
}
id := uint16(r[0])<<8 | uint16(r[1])
if id == uint16(original[4])<<8|uint16(original[5]) {
id++
}
fake[4] = byte(id >> 8)
fake[5] = byte(id)
}
func GetPayload(faking *config.FakingConfig) []byte {
switch faking.SNIType {
case config.FakePayloadRandom:
@ -34,6 +90,8 @@ func GetPayload(faking *config.FakingConfig) []byte {
switch faking.SNIType {
case config.FakePayloadDefault2:
return cloneBytes(config.FakeSNI2)
case config.FakePayloadSTUN:
return cloneBytes(config.FakeSTUN)
case config.FakePayloadCustom:
return []byte(faking.CustomPayload)
}

View file

@ -107,7 +107,8 @@ func applyGroup(cfg *config.Config, group []domainWithSet) {
}
if existingSet != nil {
oldStrategy := existingSet.Fragmentation.Strategy
changes := describeSetChanges(existingSet, refSet)
existingSet.TCP = refSet.TCP
existingSet.UDP = refSet.UDP
existingSet.Fragmentation = refSet.Fragmentation
@ -120,8 +121,13 @@ func applyGroup(cfg *config.Config, group []domainWithSet) {
}
}
log.Infof("[WATCHDOG] %s: applied to set %q (strategy: %s -> %s)",
strings.Join(groupDomains, ", "), existingSet.Name, oldStrategy, refSet.Fragmentation.Strategy)
if len(changes) == 0 {
log.Infof("[WATCHDOG] %s: set %q already matched the discovered strategy, left unchanged",
strings.Join(groupDomains, ", "), existingSet.Name)
} else {
log.Infof("[WATCHDOG] %s: overwrote tcp/udp/fragmentation/faking of set %q (%s)",
strings.Join(groupDomains, ", "), existingSet.Name, strings.Join(changes, ", "))
}
} else {
newSet := config.NewSetConfig()
newSet.Id = uuid.New().String()
@ -139,6 +145,37 @@ func applyGroup(cfg *config.Config, group []domainWithSet) {
}
}
// describeSetChanges lists the fields the discovered strategy will overwrite, so
// the log says what a heal actually did to a hand-tuned set rather than just
// naming the fragmentation strategy.
func describeSetChanges(old, next *config.SetConfig) []string {
var changes []string
add := func(field string, from, to any) {
if fmt.Sprint(from) == fmt.Sprint(to) {
return
}
changes = append(changes, fmt.Sprintf("%s %v -> %v", field, from, to))
}
add("fragmentation.strategy", old.Fragmentation.Strategy, next.Fragmentation.Strategy)
add("fragmentation.sni_position", old.Fragmentation.SNIPosition, next.Fragmentation.SNIPosition)
add("fragmentation.tlsrec_pos", old.Fragmentation.TLSRecordPosition, next.Fragmentation.TLSRecordPosition)
add("fragmentation.combo.shuffle_mode", old.Fragmentation.Combo.ShuffleMode, next.Fragmentation.Combo.ShuffleMode)
add("fragmentation.combo.first_delay_ms", old.Fragmentation.Combo.FirstDelayMs, next.Fragmentation.Combo.FirstDelayMs)
add("faking.strategy", old.Faking.Strategy, next.Faking.Strategy)
add("faking.ttl", old.Faking.TTL, next.Faking.TTL)
add("faking.sni_type", old.Faking.SNIType, next.Faking.SNIType)
add("faking.sni_seq_length", old.Faking.SNISeqLength, next.Faking.SNISeqLength)
add("faking.tcp_md5", old.Faking.TCPMD5, next.Faking.TCPMD5)
add("faking.tls_mod", strings.Join(old.Faking.TLSMod, "+"), strings.Join(next.Faking.TLSMod, "+"))
add("tcp.seg2delay", old.TCP.Seg2Delay, next.TCP.Seg2Delay)
add("tcp.conn_bytes_limit", old.TCP.ConnBytesLimit, next.TCP.ConnBytesLimit)
add("tcp.desync.mode", old.TCP.Desync.Mode, next.TCP.Desync.Mode)
return changes
}
func normalizeDomain(s string) string {
return strings.ToLower(strings.TrimSpace(s))
}

View file

@ -11,6 +11,8 @@ import (
"github.com/daniellavrushin/b4/netprobe"
)
const verifyRetryDelay = 3 * time.Second
type Watchdog struct {
cfgPtr *atomic.Pointer[config.Config]
discoveryRT *discovery.Runtime
@ -227,9 +229,14 @@ func (w *Watchdog) healBatch(domains []string) {
log.Infof("[WATCHDOG] starting discovery for %d domains: %v", len(domains), domains)
tries := wdCfg.HealValidationTries
if tries < 1 {
tries = 1
}
suite, err := w.discoveryRT.StartSuite(cfg, domains, discovery.StartSuiteOptions{
SkipDNS: true,
ValidationTries: 1,
ValidationTries: tries,
})
if err != nil {
log.Warnf("[WATCHDOG] failed to start discovery: %v", err)
@ -282,8 +289,8 @@ func (w *Watchdog) healBatch(domains []string) {
if cs.Status == discovery.CheckStatusComplete || cs.Status == discovery.CheckStatusFailed || cs.Status == discovery.CheckStatusCanceled {
break
}
if cs.SuccessfulChecks >= len(domains) {
log.Infof("[WATCHDOG] working strategies found for all domains, canceling discovery early")
if cs.SuccessfulChecks >= len(domains) && tries > 1 {
log.Infof("[WATCHDOG] strategies found for all domains and confirmed over %d tries, canceling discovery early", tries)
discovery.CancelCheckSuite(suite.Id)
time.Sleep(1 * time.Second)
break
@ -307,9 +314,29 @@ func (w *Watchdog) healBatch(domains []string) {
return
}
rollbackCfg := w.cfgPtr.Load().Clone()
freshCfg := w.cfgPtr.Load().Clone()
applyErrors := applyBatchResults(freshCfg, domains, cs, w.saveFunc)
applied := make([]string, 0, len(domains))
for _, domain := range domains {
if err, failed := applyErrors[domain]; failed && err != nil {
continue
}
applied = append(applied, domain)
}
verified := w.verifyApplied(applied, wdCfg)
rollback := len(applied) > 0 && len(verified) == 0
if rollback {
if err := w.saveFunc(rollbackCfg); err != nil {
log.Warnf("[WATCHDOG] failed to roll back config after failed verification: %v", err)
} else {
log.Warnf("[WATCHDOG] verification failed for all healed domains, rolled config back")
}
}
w.mu.Lock()
defer w.mu.Unlock()
@ -327,18 +354,79 @@ func (w *Watchdog) healBatch(domains []string) {
}
dr := cs.DomainDiscoveryResults[ExtractDomain(domain)]
if dr != nil && dr.BestSuccess {
log.Infof("[WATCHDOG] %s: healed (%s, %.0f KB/s)", domain, dr.BestPreset, dr.BestSpeed/1024)
if res, ok := verified[domain]; ok {
if dr != nil && dr.BestSuccess {
log.Infof("[WATCHDOG] %s: healed with %s, verified at %.0f KB/s", domain, dr.BestPreset, res.Speed/1024)
} else {
log.Infof("[WATCHDOG] %s: healed, verified at %.0f KB/s", domain, res.Speed/1024)
}
st.Status = StatusHealthy
st.ConsecutiveFailures = 0
st.Interval = wdCfg.IntervalSec
st.LastHeal = time.Now()
st.LastError = ""
st.CooldownUntil = time.Now().Add(time.Duration(wdCfg.Cooldown) * time.Second)
continue
}
st.Status = StatusHealthy
preset := "unknown"
if dr != nil && dr.BestPreset != "" {
preset = dr.BestPreset
}
log.Warnf("[WATCHDOG] %s: discovery reported %s working but it did not survive verification, not healed, cooldown %ds",
domain, preset, wdCfg.Cooldown)
st.Status = StatusDegraded
st.ConsecutiveFailures = 0
st.Interval = wdCfg.IntervalSec
st.LastHeal = time.Now()
st.LastError = ""
st.LastError = "applied strategy failed post-apply verification"
st.CooldownUntil = time.Now().Add(time.Duration(wdCfg.Cooldown) * time.Second)
}
}
// verifyApplied re-checks each domain through the live engine after the healed
// config has been applied. Discovery runs on its own queues with its own probe
// client, so a preset succeeding there is not evidence that normal traffic works.
func (w *Watchdog) verifyApplied(domains []string, wdCfg config.WatchdogConfig) map[string]CheckResult {
verified := make(map[string]CheckResult, len(domains))
if len(domains) == 0 {
return verified
}
tries := wdCfg.VerifyTries
if tries < 1 {
tries = 1
}
cfg := w.cfgPtr.Load()
mark := cfg.Queue.Mark
timeout := time.Duration(wdCfg.TimeoutSec) * time.Second
pending := append([]string(nil), domains...)
for i := 0; i < tries && len(pending) > 0; i++ {
if i > 0 {
select {
case <-w.stop:
return verified
case <-time.After(verifyRetryDelay):
}
}
results := checkAllConcurrently(pending, mark, timeout)
var stillFailing []string
for _, domain := range pending {
res := results[domain]
if res.OK {
verified[domain] = res
continue
}
log.Warnf("[WATCHDOG] %s: post-apply verification try %d/%d failed (%s)", domain, i+1, tries, res.Error)
stillFailing = append(stillFailing, domain)
}
pending = stillFailing
}
return verified
}
func (w *Watchdog) syncDomainStates(wdCfg config.WatchdogConfig) {
active := make(map[string]bool, len(wdCfg.Domains))
for _, d := range wdCfg.Domains {