Autodetecting the needed buffer size is quite complex (especially with
float/double values) so it is mandatory to properly check for
`ndpi_snprintf` truncation.
These issues have been undetected so far probably because the default
buffer is big enough for all common cases.
Add an example of usage of `ndpi_deserialize_clone_all()` (taken from
`ntopng`)
The goal of this fuzzer is to test init and deinit of the library, with
different configurations. In details:
* random memory allocation failures, even during init phase
* random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()`
* random LRU caches sizes
* random bitmask of enabled protocols
* random parameters of `ndpi_set_detection_preferences()`
* random initialization of opportunistic TLS
* random load/don't load of configuration files
This new fuzzer is a C++ file, because it uses `FuzzedDataProvider`
class (see
https://github.com/google/fuzzing/blob/master/docs/split-inputs.md).
Note that the (existing) fuzzers need to be linked with C++ compiler
anyway, so this new fuzzer doesn't add any new requirements.
This commit add (optional) support for Link-Time-Optimization and Gold
linker.
This is the first, mandatory step needed to make nDPI compliant with
"introspector" sanitizer requirements in OSS-Fuzz: see
https://github.com/google/oss-fuzz/issues/8939
Gold linker is not supported by Windows and by macOS, so this feature is
disabled by default. It has been enable in CI in two linux targets
("latest" gcc and clang).
Fix some warnings triggered by LTO.
The changes in `src/lib/ndpi_serializer.c` seams reasonable.
However, the change in `tests/unit/unit.c` is due to the following
warning, which seems to be a false positive.
```
unit.c: In function ‘serializerUnitTest’:
ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here
67 | ndpi_serializer serializer, deserializer;
| ^
ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here
67 | ndpi_serializer serializer, deserializer;
```
Since this warning is triggered only with an old version of gcc and
`tests/unit/unit.c` is used only during the tests, the easiest fix has
been applied.
Some (unknown to me) combinations of OS and compiler trigger the
following warnings at linker time (with sanitizer and gold linker)
```
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm'
[..]
```
I have not found any references to this kind of message, with the only
exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975
which seems to suggest that these messages can be safely ignored.
In any case, the compilation results are sound.
Fix `clean` target in the Makefile in the `example` directory.
In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error
(as always, when the gold linker is involved...).
It's come out that the culprit was the `tempnam` function: the code has
been changed to use `tmpfile` instead. No sure why... :(
Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a`
anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds
as workaround for the "missing dependencies errors" described in
https://github.com/google/oss-fuzz/issues/8939
QUIC packets are encrypted/obfuscated; that means that we need to
decrypt them before parsing the real (TLS) message.
Fuzzing is not effective here, since a random buffer is hardly a valid
encrypted QUIC packet.
Add a new fuzzer, testing *decrypted* QUIC packets.
Add a basic corpus.
Fix a few bugs already found by this fuzzer.
ubuntu-18.04 is deprecated (ubuntu-latest points to 20.04).
macos-latest points to macos-11, so it makes sense to test macos-12,
too.
About the compilers, the general idea it to test the oldest and the
newest versions easily available: switch to gcc-11 and clang-14.
See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
* CI fixes
* some build systems do not like that (e.g. OpenWrt)
* fixed some rrdtool related build warnings/errors
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* use -ltcmalloc_and_profiler and try to get rid of LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
GCC analyzer won't complain about possible use-after-free (false positive).
* tests/do.sh prints word diff's only once and not the same over and over again
* sync unit tests
Signed-off-by: lns <matzeton@googlemail.com>
This fixes some build/test issues resulting when using tarballs.
* nDPI uses autotools (especially autoconf) in a wrong way, see #1163
Signed-off-by: lns <matzeton@googlemail.com>
* Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
Fixed errors for bigendian platforms in ndpiReader.
All address and port comparisons and hash calculations are done with
endian in mind.
The get_ndpi_flow_info() function searched for an existing flow for the
forward and reverse direction of the packet.
The ndpi_workflow_node_cmp() function looked for a flow regardless of
the packet's direction. This is what led to an error in determining the
direction of transmission of the packet.
Fixed error in "synscan" test: the number of packets in the forward and
reverse direction is incorrectly defined (verified via tcpdump).
Fixed bug with icmp protocol checksum check for big endian platforms.
* Testing more code in CI environments.
* Added strict option checking for `./configure' in CI environments.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* As there is now a builtin, lightweight libgcrypt
there is no need to disable tls-clho decryption.
* It is still possible to use a host libgcrypt
with `--with-local-libgcrypt'.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Disable unit tests on CI for big-endian target. We know we have multiple
issues on big-endian architectures (see #1312) and so the unit tests
always fail there. Ignore this error for the time being and let the CI
pass if we don't have other issues.
Remove an unused automa definition
* Added ARM build and unit test run for SonarCloud-CI.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed Mingw64 build.
* adapted to SonarCloud-CI workflow
* removed broken and incomplete Windows example (tested on VS2017/VS2019)
* removed unnecessary include (e.g. pthread.h for the library which does not make use of it)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
