vrr/nDPI
2
0
Fork 0
mirror of https://github.com/vel21ripn/nDPI.git synced 2026-05-01 16:30:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Add the concept of protocols stack: more than 2 protocols per flow (#2913)

Browse source 
The idea is to remove the limitation of only two protocols ("master" and
"app") in the flow classifcation.
This is quite handy expecially for STUN flows and, in general, for any
flows where there is some kind of transitionf from a cleartext protocol
to TLS: HTTP_PROXY -> TLS/Youtube; SMTP -> SMTPS (via STARTTLS msg).

In the vast majority of the cases, the protocol stack is simply
Master/Application.

Examples of real stacks (from the unit tests)  different from the standard
"master/app":
* "STUN.WhatsAppCall.SRTP": a WA call
* "STUN.DTLS.GoogleCall": a Meet call
* "Telegram.STUN.DTLS.TelegramVoip": a Telegram call
* "SMTP.SMTPS.Google": a SMTP connection to Google server started in
  cleartext and updated to TLS
* "HTTP.Google.ntop": a HTTP connection to a Google domain (match via
  "Host" header) and to a ntop server (match via "Server" header)

The logic to create the stack is still a bit coarse: we have a decade of
code try to push everything in only ywo protocols... Therefore, the
content of the stack is still **highly experimental** and might change
in the next future; do you have any suggestions?

It is quite likely that the legacy fields "master_protocol" and
"app_protocol" will be there for a long time.

Add some helper to use the stack:
```
ndpi_stack_get_upper_proto();
ndpi_stack_get_lower_proto();
bool ndpi_stack_contains(struct ndpi_proto_stack *s, u_int16_t proto_id);
bool ndpi_stack_is_tls_like(struct ndpi_proto_stack *s);
bool ndpi_stack_is_http_like(struct ndpi_proto_stack *s);

```

Be sure new stack logic is compatible with legacy code:
```
assert(ndpi_stack_get_upper_proto(&flow->detected_protocol.protocol_stack) ==
       ndpi_get_upper_proto(flow->detected_protocol));
assert(ndpi_stack_get_lower_proto(&flow->detected_protocol.protocol_stack) ==
       ndpi_get_lower_proto(flow->detected_protocol));
```
This commit is contained in:
Ivan Nardi 2025-08-01 10:05:50 +02:00 • committed by GitHub
parent 79f0cbd32a
commit 8dd2220116
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
689 changed files with 13984 additions and 13680 deletions
Download patch file Download diff file Expand all files Collapse all files

8
tests/cfgs/default/result/mpeg-dash.pcap.out
View file

@ -26,7 +26,7 @@ Fun 13 4669 4
Media 13 4669 4
1 TCP 10.84.1.81:60926 <-> 166.248.152.10:80 [proto: 7.291/HTTP.MpegDash][IP: 0/Unknown][ClearText][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 4][cat: Media/1][Breed: Fun][2 pkts/456 bytes <-> 2 pkts/1520 bytes][Goodput ratio: 72/92][0.30 sec][Hostname/SNI: gdl.news-cdn.site][URL: gdl.news-cdn.site/as/bigo-ad-creatives/3s3/2lOTA7.mp4][StatusCode: 200][Content-Type: video/mp4][Server: openresty][User-Agent: Mozilla/5.0 (Linux; Android 11; SM-A715F Build/RP1A.200720.012; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/89.0.4389.105 Mobile Safari/537.36][TCP Fingerprint: 2_64_65535_d876f498b09e/Android][PLAIN TEXT (GET /as/bigo)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0]
2 TCP 54.161.101.85:80 <-> 192.168.2.105:59144 [proto: 7.291/HTTP.MpegDash][IP: 265/AmazonAWS][ClearText][Confidence: DPI][FPC: 7.291/HTTP.MpegDash, Confidence: DPI][DPI packets: 4][cat: Media/1][Breed: Fun][2 pkts/1649 bytes <-> 2 pkts/323 bytes][Goodput ratio: 92/59][0.01 sec][Hostname/SNI: livesim.dashif.org][URL: livesim.dashif.org/livesim/sts_1652783809/sid_40c11e12/chunkdur_1/ato_7/testpic4_8s/V2400/206598098.m4s][User-Agent: VLC/3.0.16 LibVLC/3.0.16][PLAIN TEXT (OHTTP/1.1 200 OK)][Plen Bins: 0,0,33,0,0,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,0,0]
3 TCP 192.168.2.105:59142 <-> 54.161.101.85:80 [proto: 7.291/HTTP.MpegDash][IP: 265/AmazonAWS][ClearText][Confidence: DPI][FPC: 265/AmazonAWS, Confidence: IP address][DPI packets: 4][cat: Media/1][Breed: Fun][3 pkts/390 bytes <-> 1 pkts/74 bytes][Goodput ratio: 47/0][0.10 sec][Hostname/SNI: livesim.dashif.org][URL: livesim.dashif.org/livesim/sts_1652783809/sid_40c11e12/chunkdur_1/ato_7/testpic4_8s/A48/init.mp4][User-Agent: VLC/3.0.16 LibVLC/3.0.16][TCP Fingerprint: 2_64_64240_2e3cee914fc1/Linux][PLAIN TEXT (IGET /livesim/sts)][Plen Bins: 0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
4 TCP 192.168.2.105:59146 -> 54.161.101.85:80 [proto: 7.291/HTTP.MpegDash][IP: 265/AmazonAWS][ClearText][Confidence: DPI][FPC: 7.291/HTTP.MpegDash, Confidence: DPI][DPI packets: 1][cat: Media/1][Breed: Fun][1 pkts/257 bytes -> 0 pkts/0 bytes][Goodput ratio: 74/0][< 1 sec][Hostname/SNI: livesim.dashif.org][URL: livesim.dashif.org/livesim/sts_1652783809/sid_40c11e12/chunkdur_1/ato_7/testpic4_8s/V2400/206598099.m4s][User-Agent: VLC/3.0.16 LibVLC/3.0.16][Risk: ** Unidirectional Traffic **][Risk Score: 10][Risk Info: No server to client traffic][PLAIN TEXT (GET /livesim/sts)][Plen Bins: 0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
1 TCP 10.84.1.81:60926 <-> 166.248.152.10:80 [proto: 7.291/HTTP.MpegDash][Stack: HTTP.MpegDash][IP: 0/Unknown][ClearText][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 4][cat: Media/1][Breed: Fun][2 pkts/456 bytes <-> 2 pkts/1520 bytes][Goodput ratio: 72/92][0.30 sec][Hostname/SNI: gdl.news-cdn.site][URL: gdl.news-cdn.site/as/bigo-ad-creatives/3s3/2lOTA7.mp4][StatusCode: 200][Content-Type: video/mp4][Server: openresty][User-Agent: Mozilla/5.0 (Linux; Android 11; SM-A715F Build/RP1A.200720.012; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/89.0.4389.105 Mobile Safari/537.36][TCP Fingerprint: 2_64_65535_d876f498b09e/Android][PLAIN TEXT (GET /as/bigo)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0]
2 TCP 54.161.101.85:80 <-> 192.168.2.105:59144 [proto: 7.291/HTTP.MpegDash][Stack: HTTP.MpegDash][IP: 265/AmazonAWS][ClearText][Confidence: DPI][FPC: 7.291/HTTP.MpegDash, Confidence: DPI][DPI packets: 4][cat: Media/1][Breed: Fun][2 pkts/1649 bytes <-> 2 pkts/323 bytes][Goodput ratio: 92/59][0.01 sec][Hostname/SNI: livesim.dashif.org][URL: livesim.dashif.org/livesim/sts_1652783809/sid_40c11e12/chunkdur_1/ato_7/testpic4_8s/V2400/206598098.m4s][User-Agent: VLC/3.0.16 LibVLC/3.0.16][PLAIN TEXT (OHTTP/1.1 200 OK)][Plen Bins: 0,0,33,0,0,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,0,0]
3 TCP 192.168.2.105:59142 <-> 54.161.101.85:80 [proto: 7.291/HTTP.MpegDash][Stack: HTTP.MpegDash][IP: 265/AmazonAWS][ClearText][Confidence: DPI][FPC: 265/AmazonAWS, Confidence: IP address][DPI packets: 4][cat: Media/1][Breed: Fun][3 pkts/390 bytes <-> 1 pkts/74 bytes][Goodput ratio: 47/0][0.10 sec][Hostname/SNI: livesim.dashif.org][URL: livesim.dashif.org/livesim/sts_1652783809/sid_40c11e12/chunkdur_1/ato_7/testpic4_8s/A48/init.mp4][User-Agent: VLC/3.0.16 LibVLC/3.0.16][TCP Fingerprint: 2_64_64240_2e3cee914fc1/Linux][PLAIN TEXT (IGET /livesim/sts)][Plen Bins: 0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
4 TCP 192.168.2.105:59146 -> 54.161.101.85:80 [proto: 7.291/HTTP.MpegDash][Stack: HTTP.MpegDash][IP: 265/AmazonAWS][ClearText][Confidence: DPI][FPC: 7.291/HTTP.MpegDash, Confidence: DPI][DPI packets: 1][cat: Media/1][Breed: Fun][1 pkts/257 bytes -> 0 pkts/0 bytes][Goodput ratio: 74/0][< 1 sec][Hostname/SNI: livesim.dashif.org][URL: livesim.dashif.org/livesim/sts_1652783809/sid_40c11e12/chunkdur_1/ato_7/testpic4_8s/V2400/206598099.m4s][User-Agent: VLC/3.0.16 LibVLC/3.0.16][Risk: ** Unidirectional Traffic **][Risk Score: 10][Risk Info: No server to client traffic][PLAIN TEXT (GET /livesim/sts)][Plen Bins: 0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]