Add the concept of protocols stack: more than 2 protocols per flow (#2913)

The idea is to remove the limitation of only two protocols ("master" and
"app") in the flow classifcation.
This is quite handy expecially for STUN flows and, in general, for any
flows where there is some kind of transitionf from a cleartext protocol
to TLS: HTTP_PROXY -> TLS/Youtube; SMTP -> SMTPS (via STARTTLS msg).

In the vast majority of the cases, the protocol stack is simply
Master/Application.

Examples of real stacks (from the unit tests)  different from the standard
"master/app":
* "STUN.WhatsAppCall.SRTP": a WA call
* "STUN.DTLS.GoogleCall": a Meet call
* "Telegram.STUN.DTLS.TelegramVoip": a Telegram call
* "SMTP.SMTPS.Google": a SMTP connection to Google server started in
  cleartext and updated to TLS
* "HTTP.Google.ntop": a HTTP connection to a Google domain (match via
  "Host" header) and to a ntop server (match via "Server" header)

The logic to create the stack is still a bit coarse: we have a decade of
code try to push everything in only ywo protocols... Therefore, the
content of the stack is still **highly experimental** and might change
in the next future; do you have any suggestions?

It is quite likely that the legacy fields "master_protocol" and
"app_protocol" will be there for a long time.

Add some helper to use the stack:
```
ndpi_stack_get_upper_proto();
ndpi_stack_get_lower_proto();
bool ndpi_stack_contains(struct ndpi_proto_stack *s, u_int16_t proto_id);
bool ndpi_stack_is_tls_like(struct ndpi_proto_stack *s);
bool ndpi_stack_is_http_like(struct ndpi_proto_stack *s);

```

Be sure new stack logic is compatible with legacy code:
```
assert(ndpi_stack_get_upper_proto(&flow->detected_protocol.protocol_stack) ==
       ndpi_get_upper_proto(flow->detected_protocol));
assert(ndpi_stack_get_lower_proto(&flow->detected_protocol.protocol_stack) ==
       ndpi_get_lower_proto(flow->detected_protocol));
```
This commit is contained in:
Ivan Nardi 2025-08-01 10:05:50 +02:00 committed by GitHub
parent 79f0cbd32a
commit 8dd2220116
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
689 changed files with 13984 additions and 13680 deletions

View file

@ -26,4 +26,4 @@ Unsafe 18 1700 1
Download 18 1700 1
1 TCP [2a00:d40:1:3:192:12:193:11]:44724 <-> [2a00:800:1010::1]:21 [proto: 1/FTP_CONTROL][IP: 0/Unknown][ClearText][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 8][cat: Download/7][Breed: Unsafe][10 pkts/892 bytes <-> 8 pkts/808 bytes][Goodput ratio: 3/14][7.24 sec][User: hello][Pwd: ][Auth Failed][bytes ratio: 0.049 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 896/1442 5304/5318 1757/2052][Pkt Len c2s/s2c min/avg/max/stddev: 86/86 89/101 98/126 4/15][Risk: ** Unsafe Protocol **** Clear-Text Credentials **][Risk Score: 110][Risk Info: Found FTP username (hello)][TCP Fingerprint: 2_64_28800_83b2f9a5576c/Unknown][PLAIN TEXT (vsFTPd 3.0.3)][Plen Bins: 71,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
1 TCP [2a00:d40:1:3:192:12:193:11]:44724 <-> [2a00:800:1010::1]:21 [proto: 1/FTP_CONTROL][Stack: FTP_CONTROL][IP: 0/Unknown][ClearText][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 8][cat: Download/7][Breed: Unsafe][10 pkts/892 bytes <-> 8 pkts/808 bytes][Goodput ratio: 3/14][7.24 sec][User: hello][Pwd: ][Auth Failed][bytes ratio: 0.049 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 896/1442 5304/5318 1757/2052][Pkt Len c2s/s2c min/avg/max/stddev: 86/86 89/101 98/126 4/15][Risk: ** Unsafe Protocol **** Clear-Text Credentials **][Risk Score: 110][Risk Info: Found FTP username (hello)][TCP Fingerprint: 2_64_28800_83b2f9a5576c/Unknown][PLAIN TEXT (vsFTPd 3.0.3)][Plen Bins: 71,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]