Commit graph

5634 commits

Author SHA1 Message Date
Luca Deri
37ca034697 (C) update 2026-01-01 10:31:40 +01:00
Luca Deri
8aaff75ede
Fixes protocol inconsistencies (#3069)
* Added code to address inconsistencies
* Added check for discarding dig errors
* Temporarily disabled whatsapp script: an update is required. See https://developers.facebook.com/docs/whatsapp/on-premises/sunset
2025-12-31 20:54:35 +01:00
Luca Deri
bfbb6eb2df Added detection of plugins not enabled 2025-12-30 18:47:40 +01:00
Luca
7d00f37528 Removed unncessary serialization 2025-12-29 18:48:33 +01:00
Luca Deri
612c1d2264 tls_blocks in JSON are now symbolic 2025-12-27 21:04:59 +01:00
Luca Deri
8b7e588e42
Enhanced TLS Blocks Computation (#3068) 2025-12-27 20:43:59 +01:00
Luca Deri
e49fa91627 Added tls_blocks serialization in JSON/csv
Use --cfg "tls,blocks_analysis,1" with ndpiReader
2025-12-26 21:06:19 +01:00
Luca Deri
5a0df66a45 Exported bins in JSON/csv 2025-12-26 19:53:03 +01:00
Luca Deri
459c6ac9b7 Added check for unset value that prevents a crash as the dns dissector manipulates protos.dns without setting the protocol 2025-12-24 09:12:33 +01:00
Luca Deri
159c05f032
Added ability to export SSH key exchanges (disabled by default).
It's possible to enable it using "--cfg=ssh,metadata.ssh_data,1" in ndpiReader. When enabled
the negotiated SSH key exchange method is returned.
2025-12-20 20:19:17 +01:00
Luca Deri
45e3213d7f
Reworked TLS dissector to export further metadata (#3066)
* Reworked TLS dissector to give users the ability to )optionnaly)
collect and export TLS metadata via "--cfg=tls,metadata.ja_data,1"
2025-12-20 17:33:09 +01:00
Luca Deri
712f4bc72a
Reverted Win change (#3064) 2025-12-18 06:53:20 +01:00
Luca Deri
56d7a268ed Win fixes 2025-12-15 12:26:34 +01:00
Toni
246462592e
Add additional msgpack protocol validations (Fix #3060, false-positives) (#3061)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-11 14:18:00 +01:00
Ivan Nardi
ce04aea085 condif: fix "only_classification" configuration 2025-12-09 15:15:11 +01:00
Ivan Nardi
87440c59bb
fuzz: extend fuzzing coverage and fix loading of TCP fingerprints from file (#3059) 2025-12-09 14:03:46 +01:00
Luca Deri
0a97483835 Defined type for extra packet function 2025-12-08 21:24:52 +01:00
Ivan Nardi
85a8ed8a10 Skip all-local and clean-local targets when building with --with-only-libndpi
The all-local and clean-local targets copy configuration files from example/,
lists/, and tests/cfgs/ directories for out-of-tree builds. These files are only
needed for example programs and tests, not for library-only builds.

When configured with --with-only-libndpi, EXTRA_TARGETS is empty, so we can skip
these setup/cleanup operations by checking if EXTRA_TARGETS is non-empty.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-08 17:51:32 +01:00
Ivan Nardi
99b0668765 Fix 2025-12-08 17:51:32 +01:00
Ivan Nardi
f3b6cb95b4 Fix tarball 2025-12-08 17:51:32 +01:00
Ivan Nardi
39f3cd9558 test: rework main script
The issue about `config.txt` files is that they contains paths:
* to configuration files, which are in the source tree
* to the dynamic plugins, which are in the build tree

Solution:
* copy all configuration files into the build tree
* all those paths are about the build tree
* tests run from the build tree, no from the source tree anymore
2025-12-08 17:51:32 +01:00
Toni Uhlig
285496d0b9 Add (generic) MsgPack protocol dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-08 17:50:20 +01:00
Ivan Nardi
155484a140 CI: test --disable-plugin-support and add a job on FreeBSD 2025-12-08 16:55:40 +01:00
Toni
19ab035178
Force disabling plugin system --disable-plugin-support (#3057)
* third party software might not need plugins or does not want it
 * no forced linkage against `dl`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-08 14:36:08 +01:00
Toni
5ec473be4d
Substitution pattern ${string/pattern/replace} does not work with FreeBSD 14/15 (#3055)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-08 14:33:43 +01:00
Ivan Nardi
11be5b31c0
Fix fuzzing instrumentation broken by e49e93cc (#3054)
Commit e49e93cc17 broke coverage
instrumentation for fuzzing targets due to two issues with how
AM_LDFLAGS and target-specific CFLAGS/CXXFLAGS interact.

Problems:
---------
1. Missing AM_LDFLAGS in link command:
   The commit changed LIB_FUZZING_ENGINE from being added to LDFLAGS
   to being added to AM_LDFLAGS (line 26):
     Before: `LDFLAGS += $(LIB_FUZZING_ENGINE)`
     After:  `AM_LDFLAGS += $(LIB_FUZZING_ENGINE)`

   However, FUZZ_LINK_COMMAND (line 34) was not updated to include
   `$(AM_LDFLAGS)`, so `-fsanitize=fuzzer` was missing from link commands.

2. Target-specific CFLAGS/CXXFLAGS override AM_CFLAGS/AM_CXXFLAGS:
   When automake sees target-specific CFLAGS (like fuzz_ndpi_reader_CFLAGS),
   it COMPLETELY REPLACES AM_CFLAGS instead of adding to it. Even empty
   assignments like `fuzz_process_packet_CFLAGS =` mean "use nothing"
   rather than "use AM_CFLAGS". This means `-fsanitize=fuzzer` from
   AM_CFLAGS was not being used during compilation.

   Example:
     `AM_CFLAGS = @NDPI_CFLAGS@ -fsanitize=fuzzer`
     `fuzz_ndpi_reader_CFLAGS = -I$(top_srcdir)/example/`

   Result: Only `-I$(top_srcdir)/example/` is used, AM_CFLAGS is ignored!

Without `-fsanitize=fuzzer` during both compilation and linking:
- No coverage instrumentation is generated
- LibFuzzer cannot collect coverage information
- Fuzzer warns: "WARNING: no interesting inputs were found so far.
  Is the code instrumented for coverage?"

Solutions:
----------
1. Add `$(AM_LDFLAGS)` to FUZZ_LINK_COMMAND (line 34) before
   `$(LDFLAGS)`
   This ensures LIB_FUZZING_ENGINE is included during linking.

2. For targets with non-empty CFLAGS/CXXFLAGS, prefix with `$(AM_CFLAGS)/$(AM_CXXFLAGS)`:
   Changed: `fuzz_*_CFLAGS = -DFOO`
   To:      `fuzz_*_CFLAGS = $(AM_CFLAGS) -DFOO`

3. For targets with empty CFLAGS/CXXFLAGS, remove the assignments entirely:
   Removed: `fuzz_*_CFLAGS =`

   This allows automake to automatically use AM_CFLAGS/AM_CXXFLAGS.

The flag ordering (package flags before user flags) is maintained.

Testing:
--------
Before fix:
  $ ./fuzz_ndpi_reader -runs=10
  INFO: Seed: 437565050
  WARNING: no interesting inputs were found so far. Is the code instrumented for coverage?

After fix:
  $ ./fuzz_ndpi_reader -runs=10
  INFO: Loaded 1 modules   (4802 inline 8-bit counters)
  INFO: Loaded 1 PC tables (4802 PCs)
  #2  INITED cov: 4 ft: 5 corp: 1/1b exec/s: 0 rss: 81Mb
  #10 DONE   cov: 4 ft: 5 corp: 1/1b lim: 4 exec/s: 0 rss: 81Mb

  $ ./fuzz_process_packet -runs=10
  INFO: Loaded 1 modules   (25 inline 8-bit counters)
  INFO: Loaded 1 PC tables (25 PCs)
  #2  INITED cov: 2 ft: 2 corp: 1/1b exec/s: 0 rss: 65Mb
  #10 DONE   cov: 2 ft: 2 corp: 1/1b lim: 4 exec/s: 0 rss: 65Mb

Verified with:
  CC=clang CXX=clang++ ./configure --enable-fuzztargets --with-sanitizer
  make -j4
  ./fuzz/fuzz_ndpi_reader -runs=10
  ./fuzz/fuzz_process_packet -runs=10

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 22:31:59 +01:00
Toni
aa3241e17b
Add (generic) JSON protocol dissector. (#2492)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-06 20:15:19 +01:00
Luca Deri
11cc612e3a Added ability to redefine existing dissectors inside plugins 2025-12-05 15:47:21 +01:00
Ivan Nardi
73d2c8657d
CI: update jobs (#3050) 2025-12-04 20:15:28 +01:00
Ivan Nardi
8b167cc25b
Build system: use proper autoconf variable for absolute paths (#3049) 2025-12-04 19:07:17 +01:00
Luca Deri
3f2f1f8ce4
Added ability to define protocol dissectors in shared libraries (#3047)
* Added ability to define protocol dissectors in shred libraries and load them at runtime

---------

Co-authored-by: Ivan Nardi <nardi.ivan@gmail.com>
2025-12-04 15:26:15 +01:00
Ivan Nardi
5cae544a40
s7comm: small fixes and extend tests (#3046) 2025-11-30 15:52:22 +01:00
Ivan Nardi
b7ef41e091 Revert "fuzz: temporary hack for introspector builds (#3041)"
This reverts commit 16916fa8bb.
2025-11-30 14:49:14 +01:00
Luca Deri
b6f0d08086 Added testing pcap files for EthernetIP 2025-11-29 11:38:55 +01:00
Ivan Nardi
b762509177
S7Comm: follow-up to complete monitoring feature (#3045) 2025-11-28 18:11:24 +01:00
Alfredo Cardigliano
9e980968d6 Extend s7comm dissector with metadata extractions (count requests by type) 2025-11-28 17:15:12 +01:00
Ivan Nardi
d76007054c
Build system: Fix --disable-shared and --disable-static flags being ignored (#3044)
The configure flags --disable-shared and --disable-static were properly
recognized by libtool but ignored by nDPI's custom src/lib/Makefile.in,
which always built both static and shared libraries regardless of the
flags specified.

This commit fixes the issue by:

1. Exporting enable_shared and enable_static variables from configure.ac
   via AC_SUBST so they're available in Makefiles

2. Adding configure-time error checks:
   - Prevent both --disable-shared and --disable-static simultaneously
   - Require static library for --enable-fuzztargets (fuzz targets need
     static linking for proper instrumentation)

3. Modifying src/lib/Makefile.in to conditionally build libraries

4. Updating all build targets to support dynamic linking when static
   library is disabled.
   These targets now:
   - Use static library when available (preferred, default behavior)
   - Fall back to dynamic linking with -lndpi when --disable-static

5. Adding configuration summary output showing which libraries will be
   built (enabled/disabled status for both shared and static)

fuzz: disable creation of (unused) shared library

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-25 15:32:29 +01:00
Ivan Nardi
31060d1645
Build system: Remove libndpiReader.a intermediate archive (#3043)
Simplify the example/Makefile.in build process by removing the
libndpiReader.a static archive target. Instead, compile and link
all common object files directly into the ndpiReader executable.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-25 12:54:53 +01:00
Ivan Nardi
44ed6f8e91
Build system: Fix out-of-tree builds for fuzz targets (#3042)
Replace relative path references (../) with $(top_srcdir) in
fuzz/Makefile.am to properly support out-of-tree builds (VPATH builds).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-24 21:10:36 +01:00
Ivan Nardi
16916fa8bb
fuzz: temporary hack for introspector builds (#3041) 2025-11-24 20:25:10 +01:00
Ivan Nardi
b93c2a8dbc
Fix some warnings when compiling with -pedantic (#3040) 2025-11-24 19:47:11 +01:00
Ivan Nardi
b2357c29c3
Build system: Standardize and improve clean/distclean targets (#3039)
This commit improves the nDPI build system by standardizing cleanup
targets, improving portability, and ensuring complete removal of
generated files during `make distclean`.

Changes:

1. Standardize clean targets (replace /bin/rm with portable $(RM))

2. Add distclean-local targets for complete cleanup

3. Add missing clean/distclean targets

4. Remove obsolete commented-out curl detection code

5. fuzz/Makefile.am: Fix out-of-tree build compatibility by replacing hardcoded
   relative paths (../example/fuzz_*.o) with proper $(top_builddir) variables.
   Add distclean-local target.
2025-11-24 18:42:03 +01:00
Alfredo Cardigliano
3c279d33b7 Fix escape of special chars in ndpi_json_string_escape 2025-11-24 18:24:34 +01:00
Luca Deri
2a1bc86bca RPM fix 2025-11-23 11:04:15 +01:00
Ivan Nardi
19ee4f6c33
Build system: minor fixes about flag compilation and example dependencies (#3038)
- always use `-Wextra` compilation flag; it was already used in CI
- always compile `ndpiSimpleIntegration` when building examples
- don't mess with optimization flags: `CFLAGS` default value is "-g -O2"
  and the user can change it

Try to test -O1,2,3,s flags in CI.

Fix some warnings.
2025-11-21 15:51:29 +01:00
Ivan Nardi
4096d758bb
Build system: Add configuration summary at end of ./configure (#3037)
Display a comprehensive configuration summary after ./configure completes,
showing:
- Package information (version, API version, git date)
- Installation paths
- Build configuration (compilers, debug mode, sanitizers, coverage, LTO)
- Core features (PCRE2, nBPF, libgcrypt, global context, TLS sigs, CRoaring)
- Optional dependencies (MaxMindDB, JSON-C, RRDtool, libnuma, gperftools)
- Build targets (library-only mode, examples, unit tests, fuzz targets, DPDK)
- Compiler flags (CFLAGS, NDPI_CFLAGS, LDFLAGS, NDPI_LDFLAGS, libs)

The summary is built dynamically as a single string variable and output with
one AC_MSG_NOTICE call, resulting in clean output without "configure:" prefix
on every line.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 22:11:39 +01:00
Ivan Nardi
af1d80605c
Build system: Fix incomplete .inc file dependency tracking (#3036)
Fix missing dependencies for .inc files included by ndpi_main.c and
ndpi_fingerprint.c.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 19:41:20 +01:00
Ivan Nardi
59d9135751
Build system: Add proper cross-compilation support (#3035)
This commit improves cross-compilation support by replacing runtime
uname calls with autotools host detection. Changes include:

- Add AC_CANONICAL_HOST to detect build/host/target systems
- Replace uname-based OS detection with $host_os checks
- Use AC_CHECK_TOOL for AR and RANLIB (cross-compilation aware)
- Set MACHINE from $host_cpu instead of uname -m
- Remove ARM-specific libnuma exclusion (let configure detect)
- Export OS_TYPE to Makefiles for consistent platform checks
- Stop overriding CC in Makefiles (respects configure settings)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 15:06:41 +01:00
Ivan Nardi
e49e93cc17
Build system: Respect user CFLAGS and LDFLAGS, remove hardcoded -g (#3034)
Fix improper handling of CFLAGS and LDFLAGS throughout the build system.
Also remove hardcoded debug flags that prevented production builds
without symbols.

Problems:
---------
1. CFLAGS/LDFLAGS handling:
   The build system was using `CFLAGS +=` and `LDFLAGS +=` to append
   package-specific flags, which modifies the user's environment variables
   instead of keeping package and user flags separate. This caused:
   - User-specified optimization levels being overridden by package defaults
   - Inability to properly override flags at configure or make time
   - Problems with cross-compilation and embedded toolchains

2. Hardcoded -g flags:
   Debug symbols (-g) were hardcoded in several Makefiles, forcing debug
   symbols in all builds including production. This caused:
   - Larger binary sizes (library and tools)
   - No way to build without debug symbols
   - Conflicts with user's debug level preferences (-g1, -g2, -g3)
   - Redundancy with configure options (--enable-debug-build)

Solutions:
----------
1. Implement proper CFLAGS/LDFLAGS separation using AM_CFLAGS/AM_LDFLAGS:
   - Added `CFLAGS = @CFLAGS@` to preserve configure-time flags
   - Added `LDFLAGS = @LDFLAGS@` to preserve configure-time flags
   - Changed `CFLAGS +=` to `AM_CFLAGS =` and `AM_CFLAGS +=`
   - Changed `LDFLAGS +=` to `AM_LDFLAGS =` and `AM_LDFLAGS +=`
   - Updated compilation rules: $(CC) $(AM_CFLAGS) $(CFLAGS) ...
   - Updated linking rules: $(CC) ... $(AM_LDFLAGS) $(LDFLAGS) ...

2. Remove all hardcoded -g flags from Makefiles:
   - Debug symbols now controlled via configure (--enable-debug-build)
     or user CFLAGS (e.g., CFLAGS="-g3")

Flag ordering ensures:
- Package flags come first (e.g., -O2, -fPIC)
- User flags come after and can override (e.g., -O3)
- Last flag wins for conflicting options

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 12:43:11 +01:00
Ivan Nardi
1bbafbd5d2 Build system: Apply warning flags consistently to performance tests
Ensure that performance test tools are compiled with project warning flags
(NDPI_CFLAGS) for consistent code quality standards across the codebase.

Previously, tests/performance/Makefile.in compiled tools (gcrypt-int,
gcrypt-gnu, substringsearch, strnstr, geo, patriciasearch) with only
user-provided CFLAGS, missing the project's warning flags (-W, -Wall,
-Wno-address-of-packed-member).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-18 21:08:53 +01:00