Implementation borrowed from the
https://github.com/ARMmbed/mbedtls.git project (v3.1.0)
Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz):
gcrypt-gnu Test md 2897 ms enc 2777 ms dec 942 ms
gcrypt-int Test md 3668 ms enc 1312 ms dec 2836 ms
gcrypt-int-noaesni Test md 3652 ms enc 1916 ms dec 4458 ms
gcrypt-gnu-nonopt Test md 3763 ms enc 4978 ms dec 3999 ms
gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration
--disable-padlock-support --disable-aesni-support \
--disable-shaext-support --disable-pclmul-support \
--disable-sse41-support --disable-drng-support \
--disable-avx-support --disable-avx2-support \
--disable-neon-support --disable-arm-crypto-support \
--disable-ppc-crypto-support
--disable-amd64-as-feature-detection
* Add few scripts to easily update some IPs lists
Some IPs lists should be updated frequently: try to easy the process.
The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure
addresses): one specific .c.inc file and one script for each protocol.
Add the possibility to don't load a specific list.
Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK,
to identify Hotmail/Outlook/Exchange flows.
TODO: ipv6
Remove the 9 addresses associated to BitTorrent: they have been added in
e2f21116 but it is not clear why all the traffic to/from these ips
should be classified as BitTorrent.
* Added quotes
* Added quotes
Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
Memory allocation or ndpi_tsearch might fail, so the two values should be
incremented only when insertion actually happened.
Co-authored-by: Andrey Izrailev <Andrey.Izrailev@oktetlabs.ru>
* Checking for port 5353/5355 is not enough.
* Added additional multicast address and header checks.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280
```
==263603==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9
#1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7
#2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7
#3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13
#4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2
#5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5
#6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6
```
Remove the last uses of `struct ndpi_id_struct`.
That code is not really used and it has not been updated for a very long
time: see #1279 for details.
Correlation among flows is achieved via LRU caches.
This change allows to further reduce memory consumption (see also
91bb77a8).
At nDPI 4.0 (more precisly, at a6b10cf, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
nDPI Memory (once): 221.15 KB
Flow Memory (per flow): 2.94 KB
```
Now:
```
nDPI Memory statistics:
nDPI Memory (once): 235.27 KB
Flow Memory (per flow): 688 B <--------
```
i.e. memory usage per flow has been reduced by 77%.
Close#1279
We should have two protocols in classification results only when the
"master" protocol allows some sub-protocols.
Classifications like `AmazonAWS`, `TLS/AmazonAWS`, `DNS/AmazonAWS` are
fine. However classifications like `NTP/Apple`, `BitTorrent/Azure`,
`DNScrypt.AmazonAWS` or `NestLogSink.Google` are misleading.
For example, `ndpiReader`shows `BitTorrent/Azure` flows under `Azure`
statistics; that seems to be wrong or, at least, very misleading.
This is quite important since we have lots of addresses from CDN
operators.
The only drawback of this solution is that right now ICMP traffic is
classified simply as `ICMP`; if we are really interested in ICMP stuff
we can restore the old behaviour later.
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.
Fix some RX false positives.
Fix some "Uncommon TLS ALPN"-risk false positives.
Fix "confidence" value for some Zoom/Torrent classifications.
Minor fix in Lua script for Wireshark extcap.
Update .gitignore file.
Let GitHub correctly detect the language type of *.inc files.
Zattoo example has been provided by @subhajit-cdot in #1148.
* Added u32 pads to `union ip_tuple` so btree search should now work as expected.
The bug caused new flow's when the remote answers, resulting in two Flows per direction. Fail.
* Fixed a race condition during shutdown phase.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
