Remove classification "by-ip" from protocol stack (#1743)

Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687
This commit is contained in:
Ivan Nardi 2022-09-20 22:24:47 +02:00 committed by GitHub
parent 174cd739db
commit a7c2734b38
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
379 changed files with 9022 additions and 9061 deletions

View file

@ -21,5 +21,5 @@ Patricia protocols: 8/0 (search/found)
PostgreSQL 39 4709 2
1 TCP 127.0.0.1:45930 <-> 127.0.0.1:5432 [proto: 19/PostgreSQL][ClearText][Confidence: DPI][cat: Database/11][12 pkts/1366 bytes <-> 12 pkts/1664 bytes][Goodput ratio: 41/52][15.40 sec][bytes ratio: -0.098 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 1002/1011 8826/8907 2767/2792][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 114/139 327/348 73/104][PLAIN TEXT (database)][Plen Bins: 8,41,0,16,0,8,0,0,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
2 TCP 127.0.0.1:45931 <-> 127.0.0.1:5432 [proto: 19/PostgreSQL][ClearText][Confidence: DPI][cat: Database/11][7 pkts/705 bytes <-> 8 pkts/974 bytes][Goodput ratio: 33/45][0.12 sec][bytes ratio: -0.160 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 9/14 45/40 18/16][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 101/122 222/251 52/72][PLAIN TEXT (database)][Plen Bins: 14,28,14,0,14,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
1 TCP 127.0.0.1:45930 <-> 127.0.0.1:5432 [proto: 19/PostgreSQL][IP: 0/Unknown][ClearText][Confidence: DPI][cat: Database/11][12 pkts/1366 bytes <-> 12 pkts/1664 bytes][Goodput ratio: 41/52][15.40 sec][bytes ratio: -0.098 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 1002/1011 8826/8907 2767/2792][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 114/139 327/348 73/104][PLAIN TEXT (database)][Plen Bins: 8,41,0,16,0,8,0,0,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
2 TCP 127.0.0.1:45931 <-> 127.0.0.1:5432 [proto: 19/PostgreSQL][IP: 0/Unknown][ClearText][Confidence: DPI][cat: Database/11][7 pkts/705 bytes <-> 8 pkts/974 bytes][Goodput ratio: 33/45][0.12 sec][bytes ratio: -0.160 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 9/14 45/40 18/16][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 101/122 222/251 52/72][PLAIN TEXT (database)][Plen Bins: 14,28,14,0,14,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]