nDPI/tests/cfgs/default/result/steam.pcapng.out
Adrian Pekar 5f312c0cd6
Fix JA4 fingerprinting (#2915)
* Fix JA4 ALPN fingerprint to use first and last characters

According to the JA4 specification (line 2139), the ALPN field should
contain the first and last characters of the first ALPN extension value.

Currently, nDPI uses the first and second characters (alpn[0] and alpn[1]),
which produces incorrect fingerprints that don't match other JA4
implementations like Wireshark.

For example, with ALPN 'http/1.1':
- Current (incorrect): 'ht' (first + second char)
- Fixed (correct):     'h1' (first + last char)

This change ensures nDPI's JA4 implementation conforms to the official
specification and maintains interoperability with other JA4 tools.

Fixes: Incorrect JA4 ALPN fingerprint generation

* Fix JA4 ALPN implementation to correctly parse first ALPN protocol

The previous fix attempted to use strlen(ja->client.alpn)-1 but this was
insufficient because nDPI modifies the ALPN string by:
1. Adding null terminators that truncate the last character
2. Converting semicolons to dashes, affecting multi-protocol ALPNs

This complete fix:
- Adds alpn_original_last field to store the true last character
- Captures the last character of the FIRST ALPN protocol only (before ;/,)
- Preserves the original character before nDPI's string modifications

Now correctly implements JA4 spec: first + last characters of first ALPN protocol
Examples:
- ALPN 'h2;http/1.1' -> 'h2' (not 'h.' or 'h1')
- ALPN 'http/1.1' -> 'h1' (not 'ht' or 'h.')

Fixes: #2914

* Fix JA4 SNI detection to properly handle missing SNI extensions

Previously, nDPI incorrectly set JA4 SNI flag to 'd' (domain present) for
flows without any SNI extension. This was because the logic only checked
for NDPI_NUMERIC_IP_HOST risk (set when SNI contains IP) but didn't
distinguish between missing SNI and domain SNI.

Now properly detects:
- No SNI extension → 'i' flag
- SNI with IP address → 'i' flag
- SNI with domain → 'd' flag

This matches the JA4 specification.
2025-07-10 14:03:27 +02:00

44 lines
6.2 KiB
Text

DPI Packets (TCP): 28 (5.60 pkts/flow)
DPI Packets (UDP): 2 (1.00 pkts/flow)
Confidence DPI : 7 (flows)
Num dissector calls: 107 (15.29 diss/flow)
LRU cache ookla: 0/0/0 (insert/search/found)
LRU cache bittorrent: 0/0/0 (insert/search/found)
LRU cache stun: 0/0/0 (insert/search/found)
LRU cache tls_cert: 0/0/0 (insert/search/found)
LRU cache mining: 0/0/0 (insert/search/found)
LRU cache msteams: 0/0/0 (insert/search/found)
LRU cache fpc_dns: 0/5/0 (insert/search/found)
Automa host: 5/5 (search/found)
Automa domain: 5/0 (search/found)
Automa tls cert: 0/0 (search/found)
Automa risk mask: 0/0 (search/found)
Automa common alpns: 7/7 (search/found)
Patricia risk mask: 0/0 (search/found)
Patricia risk mask IPv6: 0/0 (search/found)
Patricia risk: 0/0 (search/found)
Patricia risk IPv6: 0/0 (search/found)
Patricia protocols: 11/3 (search/found)
Patricia protocols IPv6: 0/0 (search/found)
Steam 36 9897 5
SteamDatagramRelay 2 284 1
Dota2 10 2545 1
Fun 48 12726 7
Game 48 12726 7
JA Host Stats:
IP Address # JA4C
1 192.168.88.231 3
2 162.254.198.46 1
1 TCP 192.168.88.231:57749 <-> 23.52.29.119:443 [proto: 91.74/TLS.Steam][IP: 0/Unknown][Encrypted][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 6][cat: Game/8][5 pkts/1378 bytes <-> 5 pkts/1865 bytes][Goodput ratio: 75/82][< 1 sec][Hostname/SNI: api.steampowered.com][(Advertised) ALPNs: h2;http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][bytes ratio: -0.150 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 8/0 18/18 24/39 7/17][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 276/373 589/1494 253/562][TCP Fingerprint: 2_64_65535_d227986fac6c/Unknown][TLSv1.3][JA4: t13d3213h2_85d9c3ed342d_e802cdec6a7f][JA3S: 15af977ce25de452b96affa2addb1036][Firefox][Cipher: TLS_AES_256_GCM_SHA384][PLAIN TEXT (67viP.eampowered.com)][Plen Bins: 0,0,0,25,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,25,0,0,0]
2 TCP 192.168.88.231:54243 <-> 188.114.98.224:443 [proto: 91.386/TLS.Dota2][IP: 220/Cloudflare][Encrypted][Confidence: DPI][FPC: 220/Cloudflare, Confidence: IP address][DPI packets: 6][cat: Game/8][5 pkts/1378 bytes <-> 5 pkts/1167 bytes][Goodput ratio: 75/71][< 1 sec][Hostname/SNI: www.dota2.com][(Advertised) ALPNs: http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][bytes ratio: 0.083 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 13/47 43/68 98/94 32/17][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 276/233 589/796 253/284][TCP Fingerprint: 2_64_65535_d227986fac6c/Unknown][TLSv1.3][JA4: t13d3213h1_85d9c3ed342d_e802cdec6a7f][JA3S: 15af977ce25de452b96affa2addb1036][Firefox][Cipher: TLS_AES_256_GCM_SHA384][Plen Bins: 0,0,0,25,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
3 TCP 192.168.88.231:42070 <-> 95.100.141.15:443 [proto: 91.74/TLS.Steam][IP: 0/Unknown][Encrypted][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 6][cat: Game/8][3 pkts/723 bytes <-> 3 pkts/1634 bytes][Goodput ratio: 71/87][< 1 sec][Hostname/SNI: store.steampowered.com][(Advertised) ALPNs: h2;http/1.1][TLS Supported Versions: GREASE;TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][bytes ratio: -0.387 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/1 12/12 24/24 12/12][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 241/545 583/1494 242/671][TCP Fingerprint: 2_64_32120_2e3cee914fc1/Linux][TLSv1.3][JA4: t13d1515h2_8daaf6152771_de4a06bb82e3][JA3S: 15af977ce25de452b96affa2addb1036][Chrome][Cipher: TLS_AES_256_GCM_SHA384][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0]
4 TCP 162.254.198.46:27038 <-> 192.168.88.231:50983 [proto: 91.74/TLS.Steam][IP: 74/Steam][Encrypted][Confidence: DPI][FPC: 74/Steam, Confidence: IP address][DPI packets: 4][cat: Game/8][2 pkts/1560 bytes <-> 2 pkts/625 bytes][Goodput ratio: 92/83][< 1 sec][Hostname/SNI: ext3-sto1.steamserver.net][(Advertised) ALPNs: h2;http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][Risk Info: Expected on port 443][TLSv1.3][JA4: t13d3213h2_85d9c3ed342d_e802cdec6a7f][JA3S: 15af977ce25de452b96affa2addb1036][Firefox][Cipher: TLS_AES_256_GCM_SHA384][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0]
5 TCP 192.168.88.231:59739 <-> 2.20.254.25:80 [proto: 7.74/HTTP.Steam][IP: 0/Unknown][ClearText][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 6][cat: Game/8][6 pkts/629 bytes <-> 4 pkts/559 bytes][Goodput ratio: 36/51][< 1 sec][Hostname/SNI: test.steampowered.com][bytes ratio: 0.059 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 6002/2 30001/3 12000/2][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 105/140 291/353 83/123][URL: test.steampowered.com/204][StatusCode: 204][Content-Type: text/plain][Server: AkamaiNetStorage][User-Agent: Valve/Steam HTTP Client 1.0][TCP Fingerprint: 2_64_65535_d227986fac6c/Unknown][PLAIN TEXT (GET /204 HTTP/1.1)][Plen Bins: 0,0,0,0,0,0,0,50,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
6 UDP 192.168.88.231:27036 -> 192.168.88.255:27036 [proto: 74/Steam][IP: 0/Unknown][Encrypted][Confidence: DPI][FPC: 74/Steam, Confidence: DPI][DPI packets: 1][cat: Game/8][6 pkts/924 bytes -> 0 pkts/0 bytes][Goodput ratio: 73/0][22.02 sec][bytes ratio: 1.000 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 4403/0 15014/0 5613/0][Pkt Len c2s/s2c min/avg/max/stddev: 82/0 154/0 226/0 72/0][PLAIN TEXT (localhost)][Plen Bins: 0,50,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
7 UDP 192.168.88.231:46604 -> 155.133.252.86:27045 [proto: 235/SteamDatagramRelay][IP: 74/Steam][ClearText][Confidence: DPI][FPC: 235/SteamDatagramRelay, Confidence: DPI][DPI packets: 1][cat: Game/8][2 pkts/284 bytes -> 0 pkts/0 bytes][Goodput ratio: 70/0][< 1 sec][Risk: ** Unidirectional Traffic **][Risk Score: 10][Risk Info: No server to client traffic][PLAIN TEXT (sdping)][Plen Bins: 0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]