Commit graph

1409 commits

Author SHA1 Message Date
Vitaly Lavrov
aa5e82288a Merge commit 'c54152a2f3' into flow_info-4 2026-10-07 15:54:02 +03:00
Vitaly Lavrov
d145b2e532 Merge commit 'a5012e6ebc' into flow_info-4 2026-10-07 10:21:58 +03:00
Vitaly Lavrov
094c9014a4 Merge commit 'eb8024ea92' into flow_info-4 2026-10-07 10:18:05 +03:00
Vitaly Lavrov
7d5fc92248 Merge commit 'a6e250b724' into flow_info-4 2026-10-07 10:03:49 +03:00
Vitaly Lavrov
633df5a1f5 Merge commit 'feab46858f' into flow_info-4 2026-10-05 14:16:35 +03:00
Vitaly Lavrov
cab81ce615 Merge commit '7d1edd831c' into flow_info-4 2026-10-05 14:07:49 +03:00
Vitaly Lavrov
ecfab97ceb Merge commit 'a105d9328a' into flow_info-4 2026-10-02 15:37:40 +03:00
Vitaly Lavrov
5fa42a1942 Merge commit 'dedf61e9cc' into flow_info-4 2026-10-02 15:32:03 +03:00
Luca Deri
a22b7a4316 Updated nDPI TCP fingerprint format
Released nDPI TCP fingerprint specification
2026-09-29 10:32:21 +02:00
Luca Deri
b79beacb2b Fixed JA5 implementation 2026-09-19 23:58:22 +02:00
Luca Deri
f9daaf170d
Dynamic host_server_name and add JA5 implementation(#3257) 2026-09-14 10:34:33 +02:00
Thomas
a6e250b724
Fix ndpi_network_ptree6_match using the v4 tree maxbits (#3255)
Add a unit test for network_ptree
2026-09-11 08:49:32 +02:00
Luca Deri
61337dad12
Reworked code to split flow definition in core and metadata 2026-09-05 15:48:02 +02:00
Ivan Nardi
f290b81018
Build: compiler-driven header dependency tracking; fix --with-ndpi-pro (#3239)
The hand-written Makefiles compiled objects with a rule of the form
"%.o: %.c $(HEADERS)" where $(HEADERS) was a hand-maintained list. That list
missed transitive headers, third_party/include/*.h, the generated *.c.inc
files, and the ndpi-pro/nDPI-custom protocol snippets that tls.c #includes, so
editing any of those did not trigger a rebuild.

Replace it with automatic dependency tracking: compile with -MMD -MP so the
compiler writes an exact .d fragment per object, and read those back with
-include $(wildcard *.d). -MP emits phony targets for each header so a removed
header does not break the build. The explicit $(HEADERS) list is dropped
(kept only the generated-header ordering for the first compile); .d files are
removed by "make clean" and ignored via *.d in .gitignore.

Applied to src/lib, src/lib/plugins, example, tests/unit and tests/dga; the
automake-generated Makefiles already do this.

Also fix a typo that made --with-ndpi-pro a no-op: the AS_IF tested
${with_npri_pro} instead of ${with_ndpi_pro}, so CUSTOM_NDPI /
-DCUSTOM_NDPI_PROTOCOLS was never set.
2026-08-29 14:54:11 +02:00
Raja Muhammad Awais
720dbe3481
ndpiReader: exclude TCP retransmissions from goodput (#3234)
Close: #1853

Co-authored-by: RajaMuhammadAwais <1.19938988e+08+RajaMuhammadAwais@users.noreply.github.com>
2026-08-28 18:14:28 +02:00
Luca Deri
326b64cc53 Code cleanup 2026-08-25 19:42:26 +02:00
Luca Deri
d7c97ba097 Improved dual-license wording 2026-08-25 16:58:08 +02:00
Luca
a105d9328a Fix for monitoring mode 2026-08-24 16:32:13 +02:00
Ivan Nardi
1f2606902c
License: non-commercial vs commercial applications (#3220)
* WIP

* Fix compilation of fuzzing

* Fix fuzzing compilation

* Cleaned up old quic contributed code (#3222)

Recode commits 713cdd0, 26df140

* QUIC: refactoring

* WIP

* WIP

* Revert "QUIC: refactoring"

This reverts commit 50afd07f28.

* QUIC: refactoring original Wireshark code (#3224)

* Added ability to specify in protocol definition, when a (partial) protocol
can change. This can be queries using a new APi call
named ndpi_can_partial_proto_classification_change()

* TLS: minor changes and remove same stale comments

* Fix out-of-bounds read of TLS certificate dates (#3225)

notBefore and notAfter are u_int32_t, but ndpi_tls2json() passed their
address to ndpi_gmtime_r() as a time_t *. Wherever time_t is 8 bytes this
reads 4 bytes past the field, picking up the adjacent member, so the value
handed to gmtime_r() is notBefore | (notAfter << 32) on little-endian.

gmtime_r() then fails on that out-of-range value and returns NULL, leaving
before/after NULL, so the "notbefore" and "notafter" fields are silently
dropped from the JSON output. On architectures that require aligned access
the 8-byte load traps instead of merely being wrong.

Convert to a real time_t instead of casting the address. The temporaries
are named not_before/not_after because notBefore/notAfter are already
char[32] buffers in the same function.

* WIP

* fuzz: fix compilation

---------

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
Co-authored-by: Luca Deri <deri@ntop.org>
Co-authored-by: shalseth <33956011+shalseth@users.noreply.github.com>
2026-08-22 17:57:28 +02:00
Vitaly Lavrov
c79d883936 Merge branch 'github.dev' into flow_info-4
Some checks failed
CIFuzz / Fuzzing (address) (push) Has been cancelled
CIFuzz / Fuzzing (memory) (push) Has been cancelled
CIFuzz / Fuzzing (undefined) (push) Has been cancelled
2026-07-07 12:14:31 +03:00
Ivan Nardi
79178b88c8
CI: compilation checks on more BSD flavors and add support for ubuntu 26.04 (#3188) 2026-06-22 17:20:16 +02:00
Vitaly Lavrov
a71c75f0be Merge commit 'cf193d33c1' into flow_info-4 2026-05-07 12:00:40 +03:00
Vitaly Lavrov
c8f7d5afc1 Merge commit '914084cba8' into flow_info-4 2026-05-07 11:32:10 +03:00
Vitaly Lavrov
6580a390c4 Merge commit '315a70591a' into flow_info-4 2026-05-07 11:17:51 +03:00
Vitaly Lavrov
e5d3178c64 Merge commit '68c7159a72' into flow_info-4 2026-05-07 10:17:25 +03:00
Vitaly Lavrov
47f7d09550 Merge commit 'c4c18440cc' into flow_info-4 2026-05-06 14:58:30 +03:00
Ivan Nardi
211d26151d
CI: run internal unit tests only once (#3177)
Internal tests take a non-negligible duration: run them only once in CI.
This way, CI is faster.

To manually run them, use `ndpiReader --run-tests`; note that this way
the application runs ONLY these tests.
2026-05-02 21:39:04 +02:00
Luca Deri
92dd5cc768 Improved IPSec dissection: now we store the IKE request and response 2026-04-21 23:09:46 +02:00
Fábio Depin
14620854e3
ndpiReader: install custom memory allocators before any nDPI API usage (#3164)
ndpiReader was setting custom memory allocation hooks only inside
test_lib(), which is executed after several code paths that already
use nDPI APIs (e.g., parseOptions, help/extcap paths, host checks,
DoH initialization, etc.).

This could lead to inconsistent allocator usage where memory is
allocated using the default libc allocator (when hooks are unset)
and later freed using custom allocators, resulting in undefined
behavior with non-trivial allocators.

This partially addresses #1280.
2026-04-21 10:20:33 +02:00
parasol-aser
315a70591a
fuzz: harden ndpi_hex_decode and add targeted parser harnesses (#3163)
Closes #3159

Co-authored-by: jeffhuang <jeff@docker.xydrsucermoubd24xgo33yhsgd.bx.internal.cloudapp.net>
2026-04-20 16:21:30 +02:00
Ivan Nardi
853d5931dc
Rework internal unit tests (#3149)
Put all the internal unit tests in a separate file.
The tests suite run (via `ndpiReader`):
* if the option `--run-tests` is used,
* always if the library is compiled with `-DNDPI_EXTENDED_SANITY_CHECKS`
  (useful in CI)

Extend tests using part of #3146.

We keep `tests/unit/unit.c` for serialization tests, like now, because
that is the only code that depends on the external library libjson-c
2026-04-16 13:32:07 +02:00
Luca Deri
ae586c875c Extended IPSEC dissection and added metadata extraction 2026-04-09 15:08:48 +02:00
Luca Deri
1b2c282f47
Fixes Some Windows Tests (#3150)
Fix extraction of IRC username and fix test configuration

Avoid check on double values which depends on the arch: see similar code
2026-03-31 12:15:48 +02:00
Ivan Nardi
9c6d2be8c3
Add configuration for the new nDPI server fingerprint (#3147) 2026-03-27 19:52:04 +01:00
Luca Deri
c4c18440cc
Implemented nDPI server fingerprint (#3139) 2026-03-24 00:09:11 +01:00
Luca Deri
5ce0a0cd62
Added new API calls for implementing anomaly detection (#3137)
void* ndpi_alloc_iforest(const double *data, int n_samples, int n_features);
void ndpi_free_iforest(void *forest);
double ndpi_iforest_score_single(void *_forest, const double *sample);

ndpi_anomaly_model* ndpi_alloc_anomaly_model(u_int16_t n_features);
void ndpi_free_anomaly_model(ndpi_anomaly_model *m);
bool ndpi_train_anomaly_model(ndpi_anomaly_model *m, u_int32_t *training_data);
bool ndpi_compute_anomaly_score(ndpi_anomaly_model *m, u_int32_t *testing_data);
2026-03-22 16:53:17 +01:00
Vitaly Lavrov
7c61d68bd9 Merge commit '6258bda34d' into flow_info-4
Fix after merge
2026-03-20 14:45:11 +03:00
Vitaly Lavrov
60bf93c7bc Merge commit 'e3801f4d14' into flow_info-4 2026-03-20 11:52:21 +03:00
Vitaly Lavrov
cee87ef833 Merge commit '2870b8c457' into flow_info-4
Test for elf.pcap failed.
2026-03-20 11:32:03 +03:00
Vitaly Lavrov
b93c5af596 Merge commit '035cb4ae66' into flow_info-4 2026-03-20 11:09:35 +03:00
Vitaly Lavrov
4928347756 Merge commit '15bac7a892' into flow_info-4 2026-03-20 11:06:51 +03:00
Ivan Nardi
8a3aa62f5f
USDT probes: add two new probes for detecting IP fragments (#3134) 2026-03-14 21:46:09 +01:00
Ivan Nardi
31c3610ea7 ndpiReader: better memory statistics: malloc vs realloc 2026-03-04 18:27:58 +01:00
Luca Deri
896ded6106 Fixed JA4 not matching QUIC flows
Minor TLS matching API change (simplified code)
2026-02-26 18:35:34 +01:00
Luca Deri
11807612e2
Extended ja4 protocol rule with the ability to specify tls blocks (#3125) 2026-02-20 23:15:02 +01:00
Ivan Nardi
035cb4ae66 example: update perf.sh script 2026-02-20 15:59:32 +01:00
Ivan Nardi
1189526f4b
ndpiReader: add perf support for "runtime" code (#3124)
In most scenarios, we are not interested in profiling the
initialization/de-initialization phases
2026-02-20 12:27:49 +01:00
Ivan Nardi
afe8cf28ad
custom categories: more unit tests and minor changes to the API (#3123) 2026-02-17 15:14:02 +01:00
Ivan Nardi
299a588f51
ndpiReader: fix a masan error (#3118) 2026-02-17 10:29:48 +01:00
Vitaly Lavrov
fd203e25ee Merge commit '766ec15036' into flow_info-4 2026-02-09 10:59:44 +03:00