Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is
extracted from the CH, the latter from the SH.
Add some entries to the known ALPN list.
Fix printing of "TLS Supported Versions" field.
0 as size value disable the cache.
The diffs in unit tests are due to the fact that some lookups are
performed before the first insert: before this change these lookups
weren't counted because the cache was not yet initialized, now they are.
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.
There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN dissectors.
In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.
Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.
See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.
Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.
Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.
Close#1687
Add detection over TCP and fix detection over IPv6.
Rename some variables since Stun dissector is no more "udp-centric".
Stun dissector should always classified the flow as `STUN` or
`STUN/Something`.
Don't touch `flow->guessed_host_protocol_id` field, which should be
always be related to "ip-classification" only.
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.
Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.
Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.
In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).
Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.
Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).
Add a partial list of some ADS/Tracking stuff.
Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.
Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.
Fix 3 LGTM warnings.
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.
Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
Error detected with valgrind.
==13127== Conditional jump or move depends on uninitialised value(s)
==13127== at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127== by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127== by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127== by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127== by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)
See also 8c3674e9
Error detected with valgrind.
==125883== Conditional jump or move depends on uninitialised value(s)
==125883== at 0x438F57: processClientServerHello (tls.c:1421)
==125883== by 0x43B35A: processTLSBlock (tls.c:712)
==125883== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125883== by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125883== by 0x42E920: ndpi_detection_process_packet (ndpi_main.c:5301)
==125916== Conditional jump or move depends on uninitialised value(s)
==125916== at 0x438D7D: processClientServerHello (tls.c:1379)
==125916== by 0x43B35A: processTLSBlock (tls.c:712)
==125916== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125916== by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125932== Conditional jump or move depends on uninitialised value(s)
==125932== at 0x438C1D: processClientServerHello (tls.c:1298)
==125932== by 0x43B35A: processTLSBlock (tls.c:712)
==125932== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125932== by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125950== Conditional jump or move depends on uninitialised value(s)
==125950== at 0x438D4F: processClientServerHello (tls.c:1371)
==125950== by 0x43B35A: processTLSBlock (tls.c:712)
==125950== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125950== by 0x42C079: check_ndpi_detection_func (ndpi_main.c:4443)
