Five fuzzers (fuzz_process_packet, fuzz_ndpi_reader, fuzz_ndpi_reader_alloc_fail,
fuzz_ndpi_reader_payload_analyzer, and fuzz_tls_certificate) were not rebuilding
when libndpi.a changed because their explicit DEPENDENCIES declarations only
included dictionary files.
In Automake, when prog_DEPENDENCIES is explicitly set, it overrides the automatic
dependency generation from LDADD. This caused these fuzzers to miss the library
dependency that the other 55 fuzzers correctly inherited.
This commit adds $(top_builddir)/src/lib/libndpi.a to the DEPENDENCIES for all
5 affected fuzzers, ensuring they rebuild whenever the library changes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Commit e49e93cc17 broke coverage
instrumentation for fuzzing targets due to two issues with how
AM_LDFLAGS and target-specific CFLAGS/CXXFLAGS interact.
Problems:
---------
1. Missing AM_LDFLAGS in link command:
The commit changed LIB_FUZZING_ENGINE from being added to LDFLAGS
to being added to AM_LDFLAGS (line 26):
Before: `LDFLAGS += $(LIB_FUZZING_ENGINE)`
After: `AM_LDFLAGS += $(LIB_FUZZING_ENGINE)`
However, FUZZ_LINK_COMMAND (line 34) was not updated to include
`$(AM_LDFLAGS)`, so `-fsanitize=fuzzer` was missing from link commands.
2. Target-specific CFLAGS/CXXFLAGS override AM_CFLAGS/AM_CXXFLAGS:
When automake sees target-specific CFLAGS (like fuzz_ndpi_reader_CFLAGS),
it COMPLETELY REPLACES AM_CFLAGS instead of adding to it. Even empty
assignments like `fuzz_process_packet_CFLAGS =` mean "use nothing"
rather than "use AM_CFLAGS". This means `-fsanitize=fuzzer` from
AM_CFLAGS was not being used during compilation.
Example:
`AM_CFLAGS = @NDPI_CFLAGS@ -fsanitize=fuzzer`
`fuzz_ndpi_reader_CFLAGS = -I$(top_srcdir)/example/`
Result: Only `-I$(top_srcdir)/example/` is used, AM_CFLAGS is ignored!
Without `-fsanitize=fuzzer` during both compilation and linking:
- No coverage instrumentation is generated
- LibFuzzer cannot collect coverage information
- Fuzzer warns: "WARNING: no interesting inputs were found so far.
Is the code instrumented for coverage?"
Solutions:
----------
1. Add `$(AM_LDFLAGS)` to FUZZ_LINK_COMMAND (line 34) before
`$(LDFLAGS)`
This ensures LIB_FUZZING_ENGINE is included during linking.
2. For targets with non-empty CFLAGS/CXXFLAGS, prefix with `$(AM_CFLAGS)/$(AM_CXXFLAGS)`:
Changed: `fuzz_*_CFLAGS = -DFOO`
To: `fuzz_*_CFLAGS = $(AM_CFLAGS) -DFOO`
3. For targets with empty CFLAGS/CXXFLAGS, remove the assignments entirely:
Removed: `fuzz_*_CFLAGS =`
This allows automake to automatically use AM_CFLAGS/AM_CXXFLAGS.
The flag ordering (package flags before user flags) is maintained.
Testing:
--------
Before fix:
$ ./fuzz_ndpi_reader -runs=10
INFO: Seed: 437565050
WARNING: no interesting inputs were found so far. Is the code instrumented for coverage?
After fix:
$ ./fuzz_ndpi_reader -runs=10
INFO: Loaded 1 modules (4802 inline 8-bit counters)
INFO: Loaded 1 PC tables (4802 PCs)
#2 INITED cov: 4 ft: 5 corp: 1/1b exec/s: 0 rss: 81Mb
#10 DONE cov: 4 ft: 5 corp: 1/1b lim: 4 exec/s: 0 rss: 81Mb
$ ./fuzz_process_packet -runs=10
INFO: Loaded 1 modules (25 inline 8-bit counters)
INFO: Loaded 1 PC tables (25 PCs)
#2 INITED cov: 2 ft: 2 corp: 1/1b exec/s: 0 rss: 65Mb
#10 DONE cov: 2 ft: 2 corp: 1/1b lim: 4 exec/s: 0 rss: 65Mb
Verified with:
CC=clang CXX=clang++ ./configure --enable-fuzztargets --with-sanitizer
make -j4
./fuzz/fuzz_ndpi_reader -runs=10
./fuzz/fuzz_process_packet -runs=10
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace relative path references (../) with $(top_srcdir) in
fuzz/Makefile.am to properly support out-of-tree builds (VPATH builds).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
Fix improper handling of CFLAGS and LDFLAGS throughout the build system.
Also remove hardcoded debug flags that prevented production builds
without symbols.
Problems:
---------
1. CFLAGS/LDFLAGS handling:
The build system was using `CFLAGS +=` and `LDFLAGS +=` to append
package-specific flags, which modifies the user's environment variables
instead of keeping package and user flags separate. This caused:
- User-specified optimization levels being overridden by package defaults
- Inability to properly override flags at configure or make time
- Problems with cross-compilation and embedded toolchains
2. Hardcoded -g flags:
Debug symbols (-g) were hardcoded in several Makefiles, forcing debug
symbols in all builds including production. This caused:
- Larger binary sizes (library and tools)
- No way to build without debug symbols
- Conflicts with user's debug level preferences (-g1, -g2, -g3)
- Redundancy with configure options (--enable-debug-build)
Solutions:
----------
1. Implement proper CFLAGS/LDFLAGS separation using AM_CFLAGS/AM_LDFLAGS:
- Added `CFLAGS = @CFLAGS@` to preserve configure-time flags
- Added `LDFLAGS = @LDFLAGS@` to preserve configure-time flags
- Changed `CFLAGS +=` to `AM_CFLAGS =` and `AM_CFLAGS +=`
- Changed `LDFLAGS +=` to `AM_LDFLAGS =` and `AM_LDFLAGS +=`
- Updated compilation rules: $(CC) $(AM_CFLAGS) $(CFLAGS) ...
- Updated linking rules: $(CC) ... $(AM_LDFLAGS) $(LDFLAGS) ...
2. Remove all hardcoded -g flags from Makefiles:
- Debug symbols now controlled via configure (--enable-debug-build)
or user CFLAGS (e.g., CFLAGS="-g3")
Flag ordering ensures:
- Package flags come first (e.g., -O2, -fPIC)
- User flags come after and can override (e.g., -O3)
- Last flag wins for conflicting options
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
This commit implements comprehensive improvements to the nDPI build system
to enhance portability, enable parallel testing, and ensure reliable
out-of-tree (VPATH) builds across all platforms.
Changes:
1. Optimize library linking order (configure.ac, all Makefiles)
- Reorder ADDITIONAL_LIBS to follow proper dependency hierarchy
- Move low-level libraries (libm) to end of link line
- Ensures compatibility with --as-needed linker flag
- Improves LTO and static linking support
2. Fix VPATH build dependencies (all Makefiles)
- Add explicit dependencies on generated headers (ndpi_config.h, ndpi_define.h)
- Prevents race conditions in parallel builds (make -j)
- Ensures headers exist before compilation starts
3. Replace mkdir -p with portable $(MKDIR_P) macro
4. Enable parallel test execution (configure.ac)
- Add 'parallel-tests' option to AM_INIT_AUTOMAKE
- Allows test suites to run concurrently during 'make check'
5. Add defensive .NOTPARALLEL directive (Makefile.am)
- Prevents race conditions if 'make -j clean distclean' is run
6. Fix clean target completeness (src/lib/Makefile.in)
- Remove all .so symlinks (libndpi.so, libndpi.so.N)
- Add cleanup for Windows DLL files (*.dll)
- Explicitly remove versioned shared libraries
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Ivan Nardi <nardi.ivan@gmail.com>
- Fixed some typos and inconsistent option names in error messages
- Improved Git Detection in configure script
- Added informative warnings when optional dependencies are missing
- Improve error handling on autogen.sh script
- Simplify fuzzing Makefile, creating common FUZZ_LINK_COMMAND template for all fuzz targets
- Added *_DEPENDENCIES declarations for fuzz targets with dictionaries
- Implemented incremental corpus building to avoid unnecessary rebuilds
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
Initial work to support out-of-tree builds
```
./autogen.sh
mkdir build
cd build
../configure
make
make check
```
IMPORTANT: `autogen.sh` doesn't call `configure` automatically anymore!!
You have to do: `./autogen.sh && ./configure --$OPTIONS`.
A little bit annoying but the pattern `autogen && configure && make` is
very common on Linux.
Known issues:
* `make doc` doesn't work in out-of-tree builds, yet
* Windows/MinGW/DPDK (out-of-tree) builds have not been tested, so it is unlikely they work
See: #2992
See: f2bccee04
This is clearly a workaround for a introspector bug/limittaions. It
seems that we need separate files for every fuzzers to get per-fuzzer
coverage stats
The idea: one c file for each fuzzer.
If it works, we can extend the same logic to every `fuzz_ndpoi_reader*`
fuzzers, otherwise we can revert that in a few days...
Change the API to enable/disable protocols: you can set that via the
standard `ndpi_set_config()` function, as every configuration
parameters. By default, all protocols are enabled.
Split the (local) context initialization into two phases:
* `ndpi_init_detection_module()`: generic part. It does not depend on the
configuration and on the protocols being enabled or not. It also
calculates the real number of internal protocols
* `ndpi_finalize_initialization()`: apply the configuration. All the
initialization stuff that depend on protocols being enabled or not
must be put here
This is the last step to have the protocols number fully calculated at
runtime
Remove a (now) useless fuzzer.
Important API changes:
* remove `NDPI_LAST_IMPLEMENTED_PROTOCOL` define
* remove `ndpi_get_num_internal_protocols()`. To get the number of
configured protocols (internal and custom) you must use
`ndpi_get_num_protocols()` after having called `ndpi_finalize_initialization()`
It might be usefull to be able to match traffic against a list of
suspicious JA4C fingerprints
Use the same code/logic/infrastructure used for JA3C (note that we are
going to remove JA3C...)
See: #2551
Fix detection of WebDAV and Gnutella (over HTTP)
Fix detection of z3950
Add two fuzzers to test `ndpi_memmem()` and `ndpi_strnstr()`
Remove some dead code:
* RTP: the same exact check is performed at the very beginning of the
function
* MQTT: use a better helper to exclude the protocol
* Colletd: `ndpi_hostname_sni_set()` never fails
Update pl7m code (fix a Use-of-uninitialized-value error)
Pl7m is a custom mutator (used for structure aware fuzzing) for network
traffic packet captures (i.e. pcap files).
The output of the mutator is always a valid pcap file, containing the
same flows/sessions of the input file. That's it: the mutator only
changes the packet payload after the TCP/UDP header, keeping all the
original L2/L3 information (IP addresses and L4 ports).
See: https://github.com/IvanNardi/pl7m
* Integrated RoaringBitmap v3
* Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse
* Fixes to ndpi_bitmap for new roaring library
* Fixes for bitmap serialization
* Fixed format
* Warning fix
* Conversion fix
* Warning fix
* Added check for roaring v3 support
* Updated file name
* Updated path
* Uses clang-9 (instead of clang-7) for builds
* Fixed fuzz_ds_bitmap64_fuse
* Fixes nDPI printf handling
* Disabled printf
* Yet another printf fix
* Cleaup
* Fx for compiling on older platforms
* Fixes for old compilers
* Initialization changes
* Added compiler check
* Fixes for old compilers
* Inline function is not static inline
* Added missing include
Some changes in the parameters names.
Add a fuzzer to fuzz the configuration file format.
Add the infrastructure to configuratin callbacks.
Add an helper to map LRU cache indexes to names.
This is the first step into providing (more) configuration options in nDPI.
The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.
You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format
This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.
The complete list of configuration options is provided in
`doc/configuration_parameters.md`.
As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
1) Public API/headers in `src/include/` [as it has always been]
2) Private API/headers in `src/lib/`
Try to keep the "ndpi_" prefix only for the public functions
Try to have a faster classification, on first packet; use standard extra
dissection data path for sub-classification, metadata extraction and
monitoring.
STUN caches:
* use the proper confidence value
* lookup into the caches only once per flow, after having found a proper
STUN classification
Add identification of Telegram VoIP calls.
