Commit graph

211 commits

Author SHA1 Message Date
Vitaly Lavrov
517e9bad48 Merge commit '81ca06b0c7' into flow_info-5 2025-08-19 10:24:44 +03:00
Vitaly Lavrov
26ad40ee1a Merge commit '978ca1ba1a' into flow_info-5
New API to enable/disable protocols. Removed `NDPI_LAST_IMPLEMENTED_PROTOCOL` (#2894)
2025-08-11 20:36:39 +03:00
Vitaly Lavrov
bf4a30935c Merge commit '6cbc8d1471' into flow_info-5 2025-08-11 20:23:50 +03:00
Vitaly Lavrov
d3ebb45589 Merge commit 'cbd7136b34' into flow_info-5
Remove `NDPI_PROTOCOL_BITMASK`; add a new generic bitmask data structure (#2871)
2025-07-29 15:46:04 +03:00
Vitaly Lavrov
fd4b5103b3 Merge commit 'ed21057710' into flow_info-5 2025-07-29 15:11:41 +03:00
Vitaly Lavrov
54387218a1 Merge commit '70a72f1638' into flow_info-5
Major changes to initialization logic.
Don't use.
2025-07-28 12:29:49 +03:00
Fábio Depin
4eff2cdb99
Refactor: make src_name/dst_name dynamically allocated to reduce RAM usage (#2908)
- Changed ndpi_flow_info: replaced fixed-size char arrays (always INET6_ADDRSTRLEN) for src_name and dst_name with char* pointers.
- Now IPv4 flows use only INET_ADDRSTRLEN when needed, instead of always reserving IPv6 size.
2025-07-02 07:41:55 +02:00
Fábio Depin
c2526cffc1
Fix stats memory reuse and cleanup across duration loops in ndpiReader (#2903) (#2904)
Refactored stats allocation and reset logic to avoid segmentation faults
when running ndpiReader in live_capture mode with the -m (duration) option.

- Introduced ndpi_stats_init(), ndpi_stats_reset(), and ndpi_stats_free()
  to encapsulate lifecycle management of stats.
- Applied these functions in ndpiReader.c and reader_util.{c,h}.
- Prevented multiple allocations and ensured safe reuse of cumulative_stats
  and per-thread stats structures between capture iterations.

Fixes: https://github.com/ntop/nDPI/issues/2903
2025-06-24 09:48:34 +02:00
Ivan Nardi
978ca1ba1a
New API to enable/disable protocols. Removed NDPI_LAST_IMPLEMENTED_PROTOCOL (#2894)
Change the API to enable/disable protocols: you can set that via the
standard `ndpi_set_config()` function, as every configuration
parameters. By default, all protocols are enabled.

Split the (local) context initialization into two phases:
* `ndpi_init_detection_module()`: generic part. It does not depend on the
configuration and on the protocols being enabled or not. It also
calculates the real number of internal protocols
* `ndpi_finalize_initialization()`: apply the configuration. All the
initialization stuff that depend on protocols being enabled or not
must be put here

This is the last step to have the protocols number fully calculated at
runtime

Remove a (now) useless fuzzer.

Important API changes:
* remove `NDPI_LAST_IMPLEMENTED_PROTOCOL` define
* remove `ndpi_get_num_internal_protocols()`. To get the number of
configured protocols (internal and custom) you must use
`ndpi_get_num_protocols()` after having called `ndpi_finalize_initialization()`
2025-06-23 11:24:18 +02:00
Ivan Nardi
aa6dcad15e
ndpiReader: print categories summary (#2895) 2025-06-21 12:41:00 +02:00
Vitaly Lavrov
3395e8c654 Merge commit '8350cc68d4' into flow_info-4 2025-06-09 10:55:32 +03:00
Ivan Nardi
cbd7136b34
Remove NDPI_PROTOCOL_BITMASK; add a new generic bitmask data structure (#2871)
The main difference is that the memory is allocated at runtime

Typical usercase:
```
struct ndpi_bitmask b;

ndpi_bitmask_alloc(&b, ndpi_get_num_internal_protocols());

ndpi_bitmask_set(&b, $BIT);
ndpi_bitmask_is_set(&b, $BIT);
[...]

ndpi_bitmask_dealloc(&b);

```

See #2136
2025-06-09 09:00:17 +02:00
Ivan Nardi
ed21057710
First step into a dynamic number of protocols (#2857)
We want to get rid of the defines `NDPI_MAX_SUPPORTED_PROTOCOLS` and
`NDPI_MAX_NUM_CUSTOM_PROTOCOLS`.

You can use:
```
ndpi_get_num_protocols()
```

See #2136

Removed some unused functions from public API
2025-06-03 10:22:15 +02:00
Ivan Nardi
70a72f1638
New API to enable/disable protocols; remove ndpi_set_protocol_detection_bitmask2() (#2853)
The main goal is not to have the bitmask depending on the total number
of protocols anymore: `NDPI_INTERNAL_PROTOCOL_BITMASK` depends only on
internal protocols, i.e. on `NDPI_MAX_INTERNAL_PROTOCOLS`, i.e.
custom-defined protocols are not counted.
See #2136

Keep the old data structure `NDPI_PROTOCOL_BITMASK` with the old
semantic.

Since we need to change the API (and all the application code...)
anyway, simplify the API: by default all the protocols are enabled.
If you need otherwise, please use `ndpi_init_detection_module_ext()`
instead of `ndpi_init_detection_module()` (you can find an example in
the `ndpiReader` code).

To update the application code you likely only need to remove these 3
lines from your code:
```
- NDPI_PROTOCOL_BITMASK all;
- NDPI_BITMASK_SET_ALL(all);
- ndpi_set_protocol_detection_bitmask2(ndpi_str, &all);
```

Removed an unused field and struct definition.
2025-06-03 09:45:46 +02:00
Ivan Nardi
8350cc68d4
BFCP: fix check on payload length and extract metadata (#2854)
We should be able to identified this protocol on the first packet,
without keeping any state

Close #2745
2025-05-26 15:08:53 +02:00
Vitaly Lavrov
aa1cc29df6 Merge commit '9e38b55a9a' into flow_info-4 2025-04-29 13:21:49 +03:00
Ivan Nardi
78ca74cf45
TLS: avoid exporting TLS heuristic fingerprint as metadata (#2783) 2025-03-27 14:54:49 +01:00
Vitaly Lavrov
0a979fcd8a Merge commit '8bada1fc02' into flow_info-4 2025-03-26 10:02:40 +03:00
Ivan Nardi
70728b3425
FastCGI: use specific metadata, not the HTTP ones (#2774)
We are going to use HTTP metadata only for real HTTP traffic; FastCGI
should be the only protocol using them improperly
2025-03-19 16:22:53 +01:00
Vitaly Lavrov
d774dcfbe5 Merge commit 'cbfc20d' into flow_info-4 2025-03-03 16:17:12 +03:00
Vitaly Lavrov
f43a9d707e Merge commit '642cf57' into flow_info-4
Fix override host name.
2025-02-28 14:31:57 +03:00
Vitaly Lavrov
1f48a01b3b Merge commit 'af011e3' into flow_info-4 2025-02-28 11:52:19 +03:00
Vitaly Lavrov
548ce2e0e5 Merge commit '63a3547' into flow_info-4 2025-02-28 11:45:54 +03:00
Luca Deri
30c3613f2f Improved RTP dissection with EVS and other mobile voice codecs 2025-02-20 22:59:23 +01:00
Luca Deri
3bef87666d Exported RTP payload in packet metadata
Added ndpi_rtp_payload_type2str() API call
2025-02-19 11:34:19 +01:00
Ivan Nardi
baca06bfd2
ndpiReader: print more DNS information (#2717) 2025-02-11 18:16:55 +01:00
Luca Deri
1577955fca Added ndpi_find_protocol_qoe() API call
Updated (C)
2025-02-10 21:21:51 +01:00
Ivan Nardi
9a5533d796
ndpiReader: add some global statistics about FPC (#2680)
Enabled via `--dump-fpc-stats` option
2025-01-17 11:47:37 +01:00
Ivan Nardi
af011e338e
TLS: remove JA3C (#2679)
Last step of removing JA3C fingerprint

Remove some duplicate tests: testing with ja4c/ja3s disabled is already
performed by `disable_metadata_and_flowrisks` configuration.

Close:#2551
2025-01-14 15:02:20 +01:00
Ivan Nardi
c3d19be26f
ndpiReader: update JA statistics (#2646)
Show JA4C and JA3S information (instead of JA3C and JA3S)
See #2551 for context
2025-01-06 15:09:25 +01:00
Ivan Nardi
2e20f670dd
QUIC: extract "max idle timeout" parameter (#2649)
Even if it is only the proposed value by the client (and not the
negotiated one), it might be use as hint for timeout by the (external)
flows manager
2025-01-06 13:45:12 +01:00
Ivan Nardi
cae9fb9989
TLS: remove ESNI support (#2648)
ESNI has been superseded by ECH for years, now.
See: https://blog.cloudflare.com/encrypted-client-hello/
Set the existing flow risk if we still found this extension.
2025-01-06 11:04:50 +01:00
Vitaly Lavrov
06c34dead8 Merge commit '4d17dc635c' into flow_info-4 2025-01-04 17:21:40 +03:00
Vitaly Lavrov
3ce5dcdd08 Merge commit '214a2e5356' into flow_info-4 2024-12-14 12:46:31 +03:00
Vitaly Lavrov
80a3a797e7 Merge commit 'd7d942586a' into flow_info-4 2024-12-14 12:44:13 +03:00
Ivan Nardi
803410542e
STUN/RTP: improve metadata extraction (#2641) 2024-12-11 15:28:00 +01:00
Luca Deri
b1d1f8cfe0 STUN counter changes 2024-11-29 17:13:43 +01:00
Luca Deri
6b6b5c7c4e Enhanced STUN stats 2024-11-28 23:19:25 +01:00
Ivan Nardi
cff8bd1bb2
Update flow->flow_multimedia_types to a bitmask (#2625)
In the same flow, we can have multiple multimedia types
2024-11-25 10:12:48 +01:00
Ivan Nardi
1bda2bf414 SIP: extract some basic metadata 2024-11-12 13:34:25 +01:00
Vitaly Lavrov
3c82662a19 Merge commit '35ef56cc24' into flow_info-4 2024-11-11 11:41:49 +03:00
Vitaly Lavrov
0c7f311065 Merge commit '0fb30c857d' into flow_info-4 2024-11-11 11:34:44 +03:00
Vitaly Lavrov
8deaf8f88d Merge commit '6b6dad4fdb' into flow_info-4 2024-11-11 11:27:00 +03:00
Vitaly Lavrov
89e835393c Merge commit '521d0ca7a0' into flow_info-4 2024-11-11 11:22:25 +03:00
Vitaly Lavrov
eadcb47807 Merge commit 'faaa5c5799' into flow_info-4 2024-11-11 11:12:28 +03:00
Luca Deri
412ca8700f Added HTTP credentials extraction 2024-10-31 21:20:46 +01:00
Ivan Nardi
9da99075aa
TLS: export heuristic fingerprint as metadata (#2609) 2024-10-28 23:36:51 +01:00
Ivan Nardi
51556113a3
ndpiReader: add some statistics about monitoring (#2602) 2024-10-19 19:44:00 +02:00
Luca Deri
6b6dad4fdb Implemented nDPI TCP fingerprint 2024-10-15 21:21:05 +02:00
Ivan Nardi
521d0ca7a0
Add monitoring capability (#2588)
Allow nDPI to process the entire flows and not only the first N packets.
Usefull when the application is interested in some metadata spanning the
entire life of the session.

As initial step, only STUN flows can be put in monitoring.

See `doc/monitoring.md` for further details.

This feature is disabled by default.

Close #2583
2024-10-14 18:05:35 +02:00