Commit graph

1619 commits

Author SHA1 Message Date
Ivan Nardi
d6cbd624d0
TLS: fix JA4 when there are no ciphers or extensions (#3084) 2026-01-13 19:33:23 +01:00
Ivan Nardi
6828c1ef30
TLS: fix JA4 when there are more than 99 ciphers or extensions (#3083) 2026-01-13 19:31:00 +01:00
Ivan Nardi
411af3e639
Fix HTTP hostname normalization with IPv6 literal address (#3081)
Close #3065
2026-01-13 18:43:02 +01:00
Luca Deri
901e317422
Added --cfg "tls,max_num_blocks_to_analyze,X" for dynamically setting TLS blocks number (#3073)
* Added --cfg "tls,max_num_blocks_to_analyze,X" where if X > 0 TLS blocks are analyzed

Example --cfg "tls,max_num_blocks_to_analyze,8"

* TLS blocks now include a time-delta (msec) with respect to the previous TLS block.
The format is @<msec delta>. Example:

 "tls_blocks": [
         "22:1=232@191",
         "22:2=-122@5,20=-1@5,21=-23@5,21=-905@5,21=-281@5",
         "21=-53@0",
         "20=1@3,21=53@3",
         "21=-218@119,21=-218@119",
 ]
2026-01-08 23:36:13 +01:00
Luca Deri
6eb2256ce6 Added JA4 testing pcap 2026-01-02 15:36:24 +01:00
Luca Deri
1ffc582e76 Case fix 2026-01-01 11:16:30 +01:00
Luca Deri
37ca034697 (C) update 2026-01-01 10:31:40 +01:00
Toni
246462592e
Add additional msgpack protocol validations (Fix #3060, false-positives) (#3061)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-11 14:18:00 +01:00
Ivan Nardi
99b0668765 Fix 2025-12-08 17:51:32 +01:00
Ivan Nardi
f3b6cb95b4 Fix tarball 2025-12-08 17:51:32 +01:00
Ivan Nardi
39f3cd9558 test: rework main script
The issue about `config.txt` files is that they contains paths:
* to configuration files, which are in the source tree
* to the dynamic plugins, which are in the build tree

Solution:
* copy all configuration files into the build tree
* all those paths are about the build tree
* tests run from the build tree, no from the source tree anymore
2025-12-08 17:51:32 +01:00
Toni Uhlig
285496d0b9 Add (generic) MsgPack protocol dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-08 17:50:20 +01:00
Toni
aa3241e17b
Add (generic) JSON protocol dissector. (#2492)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-12-06 20:15:19 +01:00
Ivan Nardi
8b167cc25b
Build system: use proper autoconf variable for absolute paths (#3049) 2025-12-04 19:07:17 +01:00
Luca Deri
3f2f1f8ce4
Added ability to define protocol dissectors in shared libraries (#3047)
* Added ability to define protocol dissectors in shred libraries and load them at runtime

---------

Co-authored-by: Ivan Nardi <nardi.ivan@gmail.com>
2025-12-04 15:26:15 +01:00
Ivan Nardi
5cae544a40
s7comm: small fixes and extend tests (#3046) 2025-11-30 15:52:22 +01:00
Ivan Nardi
b7ef41e091 Revert "fuzz: temporary hack for introspector builds (#3041)"
This reverts commit 16916fa8bb.
2025-11-30 14:49:14 +01:00
Luca Deri
b6f0d08086 Added testing pcap files for EthernetIP 2025-11-29 11:38:55 +01:00
Ivan Nardi
b762509177
S7Comm: follow-up to complete monitoring feature (#3045) 2025-11-28 18:11:24 +01:00
Ivan Nardi
d76007054c
Build system: Fix --disable-shared and --disable-static flags being ignored (#3044)
The configure flags --disable-shared and --disable-static were properly
recognized by libtool but ignored by nDPI's custom src/lib/Makefile.in,
which always built both static and shared libraries regardless of the
flags specified.

This commit fixes the issue by:

1. Exporting enable_shared and enable_static variables from configure.ac
   via AC_SUBST so they're available in Makefiles

2. Adding configure-time error checks:
   - Prevent both --disable-shared and --disable-static simultaneously
   - Require static library for --enable-fuzztargets (fuzz targets need
     static linking for proper instrumentation)

3. Modifying src/lib/Makefile.in to conditionally build libraries

4. Updating all build targets to support dynamic linking when static
   library is disabled.
   These targets now:
   - Use static library when available (preferred, default behavior)
   - Fall back to dynamic linking with -lndpi when --disable-static

5. Adding configuration summary output showing which libraries will be
   built (enabled/disabled status for both shared and static)

fuzz: disable creation of (unused) shared library

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-25 15:32:29 +01:00
Ivan Nardi
16916fa8bb
fuzz: temporary hack for introspector builds (#3041) 2025-11-24 20:25:10 +01:00
Ivan Nardi
b93c2a8dbc
Fix some warnings when compiling with -pedantic (#3040) 2025-11-24 19:47:11 +01:00
Ivan Nardi
b2357c29c3
Build system: Standardize and improve clean/distclean targets (#3039)
This commit improves the nDPI build system by standardizing cleanup
targets, improving portability, and ensuring complete removal of
generated files during `make distclean`.

Changes:

1. Standardize clean targets (replace /bin/rm with portable $(RM))

2. Add distclean-local targets for complete cleanup

3. Add missing clean/distclean targets

4. Remove obsolete commented-out curl detection code

5. fuzz/Makefile.am: Fix out-of-tree build compatibility by replacing hardcoded
   relative paths (../example/fuzz_*.o) with proper $(top_builddir) variables.
   Add distclean-local target.
2025-11-24 18:42:03 +01:00
Alfredo Cardigliano
3c279d33b7 Fix escape of special chars in ndpi_json_string_escape 2025-11-24 18:24:34 +01:00
Ivan Nardi
e49e93cc17
Build system: Respect user CFLAGS and LDFLAGS, remove hardcoded -g (#3034)
Fix improper handling of CFLAGS and LDFLAGS throughout the build system.
Also remove hardcoded debug flags that prevented production builds
without symbols.

Problems:
---------
1. CFLAGS/LDFLAGS handling:
   The build system was using `CFLAGS +=` and `LDFLAGS +=` to append
   package-specific flags, which modifies the user's environment variables
   instead of keeping package and user flags separate. This caused:
   - User-specified optimization levels being overridden by package defaults
   - Inability to properly override flags at configure or make time
   - Problems with cross-compilation and embedded toolchains

2. Hardcoded -g flags:
   Debug symbols (-g) were hardcoded in several Makefiles, forcing debug
   symbols in all builds including production. This caused:
   - Larger binary sizes (library and tools)
   - No way to build without debug symbols
   - Conflicts with user's debug level preferences (-g1, -g2, -g3)
   - Redundancy with configure options (--enable-debug-build)

Solutions:
----------
1. Implement proper CFLAGS/LDFLAGS separation using AM_CFLAGS/AM_LDFLAGS:
   - Added `CFLAGS = @CFLAGS@` to preserve configure-time flags
   - Added `LDFLAGS = @LDFLAGS@` to preserve configure-time flags
   - Changed `CFLAGS +=` to `AM_CFLAGS =` and `AM_CFLAGS +=`
   - Changed `LDFLAGS +=` to `AM_LDFLAGS =` and `AM_LDFLAGS +=`
   - Updated compilation rules: $(CC) $(AM_CFLAGS) $(CFLAGS) ...
   - Updated linking rules: $(CC) ... $(AM_LDFLAGS) $(LDFLAGS) ...

2. Remove all hardcoded -g flags from Makefiles:
   - Debug symbols now controlled via configure (--enable-debug-build)
     or user CFLAGS (e.g., CFLAGS="-g3")

Flag ordering ensures:
- Package flags come first (e.g., -O2, -fPIC)
- User flags come after and can override (e.g., -O3)
- Last flag wins for conflicting options

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 12:43:11 +01:00
Ivan Nardi
1bbafbd5d2 Build system: Apply warning flags consistently to performance tests
Ensure that performance test tools are compiled with project warning flags
(NDPI_CFLAGS) for consistent code quality standards across the codebase.

Previously, tests/performance/Makefile.in compiled tools (gcrypt-int,
gcrypt-gnu, substringsearch, strnstr, geo, patriciasearch) with only
user-provided CFLAGS, missing the project's warning flags (-W, -Wall,
-Wno-address-of-packed-member).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-18 21:08:53 +01:00
Ivan Nardi
9587059598 Build system: Improve portability, parallelization, and VPATH builds
This commit implements comprehensive improvements to the nDPI build system
to enhance portability, enable parallel testing, and ensure reliable
out-of-tree (VPATH) builds across all platforms.

Changes:

1. Optimize library linking order (configure.ac, all Makefiles)
   - Reorder ADDITIONAL_LIBS to follow proper dependency hierarchy
   - Move low-level libraries (libm) to end of link line
   - Ensures compatibility with --as-needed linker flag
   - Improves LTO and static linking support

2. Fix VPATH build dependencies (all Makefiles)
   - Add explicit dependencies on generated headers (ndpi_config.h, ndpi_define.h)
   - Prevents race conditions in parallel builds (make -j)
   - Ensures headers exist before compilation starts

3. Replace mkdir -p with portable $(MKDIR_P) macro

4. Enable parallel test execution (configure.ac)
   - Add 'parallel-tests' option to AM_INIT_AUTOMAKE
   - Allows test suites to run concurrently during 'make check'

5. Add defensive .NOTPARALLEL directive (Makefile.am)
   - Prevents race conditions if 'make -j clean distclean' is run

6. Fix clean target completeness (src/lib/Makefile.in)
   - Remove all .so symlinks (libndpi.so, libndpi.so.N)
   - Add cleanup for Windows DLL files (*.dll)
   - Explicitly remove versioned shared libraries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Ivan Nardi <nardi.ivan@gmail.com>
2025-11-18 21:08:53 +01:00
Ivan Nardi
e58f23dc75 tests: extend utests 2025-11-18 13:32:14 +01:00
Ivan Nardi
bf120e2aad
tests: improve parallelization (#3027)
Allow multiple configuration to run in parallel.
Allow fast failure
2025-11-18 10:31:50 +01:00
Ivan Nardi
173f718f5a
fuzz: another attempt to fix introspector builds (#3025) 2025-11-12 19:57:14 +01:00
Ivan Nardi
14dc8eb1bd utests: small fix 2025-11-12 19:13:18 +01:00
Luca Deri
bb10ecc380 RDP: Added check to detect probing attempts 2025-11-06 22:45:19 +01:00
Ivan Nardi
c37937a211
fuzz: improve fuzzing coverage (#3020)
We should pay attention to tell ndpiReader configuration files and
libnDPI configuration files!! Better solution?

Be sure that configuration files are located where they are expected.
In oss-fuzz enviroment we can't make any assumptions about the current
working directory of your fuzz target.
2025-11-04 21:04:29 +01:00
Ivan Nardi
bb4c6b0a3a
Update every lists (#3017) 2025-11-03 13:00:27 +01:00
Ivan Nardi
6ab338928c
Add support for out-of-tree builds (#2993)
Initial work to support out-of-tree builds
```
./autogen.sh
mkdir build
cd build
../configure
make
make check
```
IMPORTANT: `autogen.sh` doesn't call `configure` automatically anymore!!

You have to do: `./autogen.sh && ./configure --$OPTIONS`.
A little bit annoying but the pattern `autogen && configure && make` is
very common on Linux.

Known issues:
* `make doc` doesn't work in out-of-tree builds, yet
* Windows/MinGW/DPDK (out-of-tree) builds have not been tested, so it is unlikely they work

See: #2992
2025-11-03 11:58:59 +01:00
Ivan Nardi
71033e0370
Extend http-url custom rules: support for category and breed (#3014) 2025-10-24 19:17:48 +02:00
Ivan Nardi
1b566135d7
Fix flow risks with custom rules (#3010) 2025-10-23 19:51:12 +02:00
Ivan Nardi
1fdb6df2b1
Fix FPC confidence with custom rules (#3008) 2025-10-23 12:29:39 +02:00
Ivan Nardi
01836e0071
Proper handling of internal/external ids in FPC; fix FPC with custom rules (#3007) 2025-10-22 21:28:12 +02:00
Ivan Nardi
faca0a6565 ndpiReader: improve statistics 2025-10-22 20:34:29 +02:00
Ivan Nardi
9a925abd28 Proper handling of internal/external ids in ndpi_detection_giveup() 2025-10-22 20:14:43 +02:00
Luca Deri
4ce936bd83
Reworked custom rule protocol classification (#3005) 2025-10-22 17:30:21 +02:00
Luca Deri
ef159add87 Updated test results 2025-10-22 11:39:50 +02:00
Luca Deri
5abe185e2c Added support for urlXXXX@proto in protos.txt
Fixed varisous protocol mapping in custom protocols definition
2025-10-22 09:00:58 +02:00
Ivan Nardi
b9c847a176 config: fix "only_classification" configuration 2025-10-21 20:19:56 +02:00
Ivan Nardi
00c0eb947b
Fix the hash statistics of public suffix lists (#3003)
In the flow risk information always report the original domain name.
Extend the unit tests
2025-10-21 17:34:25 +02:00
Ivan Nardi
f3ec1cca05
ndpi fingerprint: avoid calculating it for flows without TCP and TLS handshakes (#3002) 2025-10-20 20:39:03 +02:00
Ivan Nardi
9c27c2df3a
Allow to overwrite domain matching via custom rules (#2999)
This is basically the revert of 0db12b1390 and 43d9caac00.
Add some tests about this feature
2025-10-20 15:28:16 +02:00
Ivan Nardi
6eb63d9cf9
tests: fixed protocol ids for all custom rules (#3000)
To ease PR/Commit comparisons
2025-10-20 14:59:15 +02:00
Ivan Nardi
e7bba509fb
Follow-up of d69446893 (#2998)
Update the documentation.
We can't return public id on `ndpi_guess_host_protocol_id()` because we
use that value internally:
```
src/lib/ndpi_main.c:  flow->guessed_protocol_id_by_ip = ndpi_guess_host_protocol_id(ndpi_str, flow);
```
2025-10-19 12:38:51 +02:00