Add the concept of protocols stack: more than 2 protocols per flow (#2913)

The idea is to remove the limitation of only two protocols ("master" and
"app") in the flow classifcation.
This is quite handy expecially for STUN flows and, in general, for any
flows where there is some kind of transitionf from a cleartext protocol
to TLS: HTTP_PROXY -> TLS/Youtube; SMTP -> SMTPS (via STARTTLS msg).

In the vast majority of the cases, the protocol stack is simply
Master/Application.

Examples of real stacks (from the unit tests)  different from the standard
"master/app":
* "STUN.WhatsAppCall.SRTP": a WA call
* "STUN.DTLS.GoogleCall": a Meet call
* "Telegram.STUN.DTLS.TelegramVoip": a Telegram call
* "SMTP.SMTPS.Google": a SMTP connection to Google server started in
  cleartext and updated to TLS
* "HTTP.Google.ntop": a HTTP connection to a Google domain (match via
  "Host" header) and to a ntop server (match via "Server" header)

The logic to create the stack is still a bit coarse: we have a decade of
code try to push everything in only ywo protocols... Therefore, the
content of the stack is still **highly experimental** and might change
in the next future; do you have any suggestions?

It is quite likely that the legacy fields "master_protocol" and
"app_protocol" will be there for a long time.

Add some helper to use the stack:
```
ndpi_stack_get_upper_proto();
ndpi_stack_get_lower_proto();
bool ndpi_stack_contains(struct ndpi_proto_stack *s, u_int16_t proto_id);
bool ndpi_stack_is_tls_like(struct ndpi_proto_stack *s);
bool ndpi_stack_is_http_like(struct ndpi_proto_stack *s);

```

Be sure new stack logic is compatible with legacy code:
```
assert(ndpi_stack_get_upper_proto(&flow->detected_protocol.protocol_stack) ==
       ndpi_get_upper_proto(flow->detected_protocol));
assert(ndpi_stack_get_lower_proto(&flow->detected_protocol.protocol_stack) ==
       ndpi_get_lower_proto(flow->detected_protocol));
```
This commit is contained in:
Ivan Nardi 2025-08-01 10:05:50 +02:00 committed by GitHub
parent 79f0cbd32a
commit 8dd2220116
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
689 changed files with 13984 additions and 13680 deletions

View file

@ -26,4 +26,4 @@ Acceptable 19 3076 1
Database 19 3076 1
1 TCP 10.220.20.67:58574 <-> 10.220.20.67:60102 [proto: 19/PostgreSQL][IP: 0/Unknown][ClearText][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 6][cat: Database/11][Breed: Acceptable][10 pkts/1252 bytes <-> 9 pkts/1824 bytes][Goodput ratio: 64/78][0.01 sec][bytes ratio: -0.186 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 1/1 2/2 1/1][Pkt Len c2s/s2c min/avg/max/stddev: 44/44 125/203 372/1360 119/410][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][Risk Info: Expected on port 5432][TCP Fingerprint: 2_128_65535_7eab44e4c8df/Unknown][Plen Bins: 25,0,12,25,0,0,0,0,0,12,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,12,0,0,0,0,0,0]
1 TCP 10.220.20.67:58574 <-> 10.220.20.67:60102 [proto: 19/PostgreSQL][Stack: PostgreSQL][IP: 0/Unknown][ClearText][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 6][cat: Database/11][Breed: Acceptable][10 pkts/1252 bytes <-> 9 pkts/1824 bytes][Goodput ratio: 64/78][0.01 sec][bytes ratio: -0.186 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 1/1 2/2 1/1][Pkt Len c2s/s2c min/avg/max/stddev: 44/44 125/203 372/1360 119/410][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][Risk Info: Expected on port 5432][TCP Fingerprint: 2_128_65535_7eab44e4c8df/Unknown][Plen Bins: 25,0,12,25,0,0,0,0,0,12,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,12,0,0,0,0,0,0]