Commit graph

28 commits

Author SHA1 Message Date
Ivan Nardi
b7ef41e091 Revert "fuzz: temporary hack for introspector builds (#3041)"
This reverts commit 16916fa8bb.
2025-11-30 14:49:14 +01:00
Ivan Nardi
d76007054c
Build system: Fix --disable-shared and --disable-static flags being ignored (#3044)
The configure flags --disable-shared and --disable-static were properly
recognized by libtool but ignored by nDPI's custom src/lib/Makefile.in,
which always built both static and shared libraries regardless of the
flags specified.

This commit fixes the issue by:

1. Exporting enable_shared and enable_static variables from configure.ac
   via AC_SUBST so they're available in Makefiles

2. Adding configure-time error checks:
   - Prevent both --disable-shared and --disable-static simultaneously
   - Require static library for --enable-fuzztargets (fuzz targets need
     static linking for proper instrumentation)

3. Modifying src/lib/Makefile.in to conditionally build libraries

4. Updating all build targets to support dynamic linking when static
   library is disabled.
   These targets now:
   - Use static library when available (preferred, default behavior)
   - Fall back to dynamic linking with -lndpi when --disable-static

5. Adding configuration summary output showing which libraries will be
   built (enabled/disabled status for both shared and static)

fuzz: disable creation of (unused) shared library

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-25 15:32:29 +01:00
Ivan Nardi
16916fa8bb
fuzz: temporary hack for introspector builds (#3041) 2025-11-24 20:25:10 +01:00
Ivan Nardi
e49e93cc17
Build system: Respect user CFLAGS and LDFLAGS, remove hardcoded -g (#3034)
Fix improper handling of CFLAGS and LDFLAGS throughout the build system.
Also remove hardcoded debug flags that prevented production builds
without symbols.

Problems:
---------
1. CFLAGS/LDFLAGS handling:
   The build system was using `CFLAGS +=` and `LDFLAGS +=` to append
   package-specific flags, which modifies the user's environment variables
   instead of keeping package and user flags separate. This caused:
   - User-specified optimization levels being overridden by package defaults
   - Inability to properly override flags at configure or make time
   - Problems with cross-compilation and embedded toolchains

2. Hardcoded -g flags:
   Debug symbols (-g) were hardcoded in several Makefiles, forcing debug
   symbols in all builds including production. This caused:
   - Larger binary sizes (library and tools)
   - No way to build without debug symbols
   - Conflicts with user's debug level preferences (-g1, -g2, -g3)
   - Redundancy with configure options (--enable-debug-build)

Solutions:
----------
1. Implement proper CFLAGS/LDFLAGS separation using AM_CFLAGS/AM_LDFLAGS:
   - Added `CFLAGS = @CFLAGS@` to preserve configure-time flags
   - Added `LDFLAGS = @LDFLAGS@` to preserve configure-time flags
   - Changed `CFLAGS +=` to `AM_CFLAGS =` and `AM_CFLAGS +=`
   - Changed `LDFLAGS +=` to `AM_LDFLAGS =` and `AM_LDFLAGS +=`
   - Updated compilation rules: $(CC) $(AM_CFLAGS) $(CFLAGS) ...
   - Updated linking rules: $(CC) ... $(AM_LDFLAGS) $(LDFLAGS) ...

2. Remove all hardcoded -g flags from Makefiles:
   - Debug symbols now controlled via configure (--enable-debug-build)
     or user CFLAGS (e.g., CFLAGS="-g3")

Flag ordering ensures:
- Package flags come first (e.g., -O2, -fPIC)
- User flags come after and can override (e.g., -O3)
- Last flag wins for conflicting options

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 12:43:11 +01:00
Ivan Nardi
173f718f5a
fuzz: another attempt to fix introspector builds (#3025) 2025-11-12 19:57:14 +01:00
Ivan Nardi
c37937a211
fuzz: improve fuzzing coverage (#3020)
We should pay attention to tell ndpiReader configuration files and
libnDPI configuration files!! Better solution?

Be sure that configuration files are located where they are expected.
In oss-fuzz enviroment we can't make any assumptions about the current
working directory of your fuzz target.
2025-11-04 21:04:29 +01:00
Ivan Nardi
6ab338928c
Add support for out-of-tree builds (#2993)
Initial work to support out-of-tree builds
```
./autogen.sh
mkdir build
cd build
../configure
make
make check
```
IMPORTANT: `autogen.sh` doesn't call `configure` automatically anymore!!

You have to do: `./autogen.sh && ./configure --$OPTIONS`.
A little bit annoying but the pattern `autogen && configure && make` is
very common on Linux.

Known issues:
* `make doc` doesn't work in out-of-tree builds, yet
* Windows/MinGW/DPDK (out-of-tree) builds have not been tested, so it is unlikely they work

See: #2992
2025-11-03 11:58:59 +01:00
Ivan Nardi
a07d55005d
fuzz: try to improve fuzzing coverage (#2981) 2025-10-06 20:44:31 +02:00
Ivan Nardi
a2b9705fb9 An attempt to fix oss-fuzz introspector builds 2025-09-03 17:44:24 +02:00
Ivan Nardi
0a0b3f901b fuzz: without --with-only-libndpi we don't need to explicitly link to libpcap 2025-07-24 21:22:03 +02:00
Adam Korczynski
04cd23074f build tests in oss-fuzz
Signed-off-by: Adam Korczynski <adam@adalogics.com>
2025-07-24 21:22:03 +02:00
Ivan Nardi
6cbc8d1471
fuzz: fuzz loading of external protocols lists (#2897) 2025-06-22 20:43:16 +02:00
Ivan Nardi
f4691c518a
fuzz: extend coverage (#2786) 2025-03-31 17:54:14 +02:00
Ivan Nardi
63a3547f99
Add (kind of) support for loading a list of JA4C malicious fingerprints (#2678)
It might be usefull to be able to match traffic against a list of
suspicious JA4C fingerprints

Use the same code/logic/infrastructure used for JA3C (note that we are
going to remove JA3C...)

See: #2551
2025-01-14 12:05:03 +01:00
Petr
c35a5ca087
shell: reformatted, fixed inspections, typos (#2506)
Reformatted shell scripts according to [ShellCheck](https://github.com/koalaman/shellcheck/).

I. Most common changes:
1. https://github.com/koalaman/shellcheck/wiki/SC2086
	`$var` → `"$var"`
	Note: this isn't always necessary and I've been careful not to substitute where it wasn't necessary in meaning.
2. https://github.com/koalaman/shellcheck/wiki/SC2006
	`` `command` `` → `$(command)`
3. https://github.com/koalaman/shellcheck/wiki/SC2004
	`$(( $a + $b ))` → `$(( a + b ))`
4. https://github.com/koalaman/shellcheck/wiki/SC2164
	`cd "$dir"` → `cd "$dir" || exit 1`
5. https://github.com/koalaman/shellcheck/wiki/SC2166
	`[ check1 -o check2 ]` → `[ check1 ] || [ check2 ]`
6. https://github.com/koalaman/shellcheck/wiki/SC2002
	`cat "${file}" | wc -c` → `< "${file}" wc -c`
	Note: this looks a bit uglier but works faster.

II. Some special changes:
1. In file `utils/common.sh`:
	https://github.com/koalaman/shellcheck/wiki/SC2112
	This script is interpreted by `sh`, not by `bash`, but uses the keyword `function`.
	So I replaced `#!/usr/bin/env sh` to `#!/usr/bin/env bash`.
2. After that I thought of replacing all shebangs to `#!/usr/bin/env bash` for consistency and cross-platform compatibility, especially since most of the files already use bash.
3. But in cases when it was `#!/bin/sh -e` or `#!/bin/bash -eu` another problem appears:
	https://github.com/koalaman/shellcheck/wiki/SC2096
	So I decided to make all shebangs look uniform:
	```
	#!/usr/bin/env bash
	set -e (or set -eu) (if needed)
	```
4. In file `tests/ossfuzz.sh`:
	https://github.com/koalaman/shellcheck/wiki/SC2162
	`read i` → `read -r i`
	Note: I think that there is no need in special treatment for backslashes, but I could be wrong.
5. In file `tests/do.sh.in`:
	https://github.com/koalaman/shellcheck/wiki/SC2035
	`ls *.*cap*` → `ls -- *.*cap*`
6. In file `utils/verify_dist_tarball.sh`:
	https://github.com/koalaman/shellcheck/wiki/SC2268
	`[ "x${TARBALL}" = x ]` → `[ -z "${TARBALL}" ]`
7. In file `utils/check_symbols.sh`:
	https://github.com/koalaman/shellcheck/wiki/SC2221
	`'[ndpi_utils.o]'|'[ndpi_memory.o]'|'[roaring.o]')` → `'[ndpi_utils.o]'|'[ndpi_memory.o]')`
8. In file `autogen.sh`:
	https://github.com/koalaman/shellcheck/wiki/SC2145
	`echo "./configure $@"` → `echo "./configure $*"`
	https://github.com/koalaman/shellcheck/wiki/SC2068
	`./configure $@` → `./configure "$@"`

III. `LIST6_MERGED` and `LIST_MERGED6`
	There were typos with this variables in files `utils/aws_ip_addresses_download.sh`, `utils/aws_ip_addresses_download.sh` and `utils/microsoft_ip_addresses_download.sh` where variable `LIST6_MERGED` was defined, but `LIST_MERGED6` was removed by `rm`.
	I changed all `LIST_MERGED6` to `LIST6_MERGED`.

Not all changes are absolutely necessary, but some may save you from future bugs.
2024-07-18 17:32:49 +02:00
Ivan Nardi
41050af5b5
fuzz: fix build of oss-introspector (#2452) 2024-05-22 19:41:59 +02:00
Ivan Nardi
7c6910d9e5
Fix/improve fuzzing (#2426) 2024-05-08 11:46:02 +02:00
Ivan Nardi
d577508727
fuzz: extend fuzzing coverage (#2281) 2024-01-24 21:16:58 +01:00
Nardi Ivan
d72a760ac3 New API for library configuration
This is the first step into providing (more) configuration options in nDPI.

The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.

You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format

This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.

The complete list of configuration options is provided in
`doc/configuration_parameters.md`.

As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
2024-01-18 10:21:24 +01:00
Toni
6dcecd73d3
Added malicious sites from the polish cert. (#2121)
* added handling of parsing errors

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-02 09:04:04 +01:00
Nardi Ivan
1366d94156 fuzzing: extend fuzzing coverage
Try fuzzing some functions which write to file/file descriptor; to avoid
slowing the fuzzer, close its stdout
2023-10-09 15:41:46 +02:00
Nardi Ivan
70814002a9 fuzz: extend fuzzing coverage 2023-09-16 11:26:11 +02:00
Ivan Nardi
8e6500a0da
fuzz: extend fuzzing coverage (#2083) 2023-09-10 21:30:26 +02:00
Ivan Nardi
cc4461f424
fuzz: extend coverage (#2073) 2023-08-20 15:18:19 +02:00
Toni
859d9ea3c3
Fixed heap-overflow if compiled with --enable-tls-sigs. (#2038)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 18:52:06 +02:00
Nardi Ivan
96340ccea2 oss-fuzz: sync build script with upstream
File copied from https://github.com/google/oss-fuzz/blob/master/projects/ndpi/build.sh

The general idea is to keep the build script in our repository and use
it from oss-fuzz builder: updating it from our side is easier and faster
then passing via an oss-fuzz PR.

The original idea is from @utoni in 3068306b60.

Once this change has been merged, we can update the code in oss-fuzz.
2023-06-12 15:17:56 +02:00
Toni Uhlig
6e8f268873
ossfuzz.sh: do not use wildcards for fuzzer e.g. fuzz/fuzz*
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-05 20:18:16 +02:00
Toni Uhlig
3068306b60
Copy&Paste ./tests/ossfuzz.sh from https://github.com/google/oss-fuzz/pull/4041
* Changing the OSS-Fuzz script from our side is much more easier then
   opening a PR to google/oss-fuzz every time we have to change a single line.
 * https://github.com/google/oss-fuzz/pull/4041 will be updated once this PR is merged

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-29 20:50:09 +02:00