Commit graph

748 commits

Author SHA1 Message Date
Luca Deri
9a71f141b4 Renamed custom protocol labels 2025-06-17 23:01:36 +02:00
Ivan Nardi
28ae2e14d8
Check ndpi_finalize_initialization() return value (#2884) 2025-06-14 11:31:23 +02:00
Ivan Nardi
6da6991320
Rework sanity checks and remove some functions from API (#2882) 2025-06-12 16:07:56 +02:00
Ivan Nardi
bcfa3f5477 Rename ndpi_bitmask_dealloc into ndpi_bitmask_free 2025-06-09 09:30:30 +02:00
Ivan Nardi
cbd7136b34
Remove NDPI_PROTOCOL_BITMASK; add a new generic bitmask data structure (#2871)
The main difference is that the memory is allocated at runtime

Typical usercase:
```
struct ndpi_bitmask b;

ndpi_bitmask_alloc(&b, ndpi_get_num_internal_protocols());

ndpi_bitmask_set(&b, $BIT);
ndpi_bitmask_is_set(&b, $BIT);
[...]

ndpi_bitmask_dealloc(&b);

```

See #2136
2025-06-09 09:00:17 +02:00
Luca Deri
2a77c58ebe Improved HTTP risk report
PCRE2 is now enabled (if present) by default as necessary to report some HTTP risks
2025-06-08 07:33:19 +02:00
Ivan Nardi
a8ad57bab2 Sync unit tests results 2025-06-05 14:37:53 +02:00
Ivan Nardi
5e54531282
Remove ndpi_set_proto_defaults() from the API (#2863)
Add an explicit field to indicate if the protocol is custom or internal
2025-06-03 17:43:28 +02:00
Ivan Nardi
ed21057710
First step into a dynamic number of protocols (#2857)
We want to get rid of the defines `NDPI_MAX_SUPPORTED_PROTOCOLS` and
`NDPI_MAX_NUM_CUSTOM_PROTOCOLS`.

You can use:
```
ndpi_get_num_protocols()
```

See #2136

Removed some unused functions from public API
2025-06-03 10:22:15 +02:00
Ivan Nardi
70a72f1638
New API to enable/disable protocols; remove ndpi_set_protocol_detection_bitmask2() (#2853)
The main goal is not to have the bitmask depending on the total number
of protocols anymore: `NDPI_INTERNAL_PROTOCOL_BITMASK` depends only on
internal protocols, i.e. on `NDPI_MAX_INTERNAL_PROTOCOLS`, i.e.
custom-defined protocols are not counted.
See #2136

Keep the old data structure `NDPI_PROTOCOL_BITMASK` with the old
semantic.

Since we need to change the API (and all the application code...)
anyway, simplify the API: by default all the protocols are enabled.
If you need otherwise, please use `ndpi_init_detection_module_ext()`
instead of `ndpi_init_detection_module()` (you can find an example in
the `ndpiReader` code).

To update the application code you likely only need to remove these 3
lines from your code:
```
- NDPI_PROTOCOL_BITMASK all;
- NDPI_BITMASK_SET_ALL(all);
- ndpi_set_protocol_detection_bitmask2(ndpi_str, &all);
```

Removed an unused field and struct definition.
2025-06-03 09:45:46 +02:00
Luca Deri
b4ce01d507 Moved variable across files 2025-05-29 10:51:02 +02:00
Luca Deri
bebacc853d Changed variable name 2025-05-29 09:18:37 +02:00
Luca Deri
2e679ba864 Added ndpi_load_protocols_dir() API call for loading IP-based protocol detection
Added --protocols-list-dir <dir> to ndpiReader for loading IP_based protocol
2025-05-28 22:13:11 +02:00
Ivan Nardi
8350cc68d4
BFCP: fix check on payload length and extract metadata (#2854)
We should be able to identified this protocol on the first packet,
without keeping any state

Close #2745
2025-05-26 15:08:53 +02:00
Vladimir Gavrilov
06cb933073
Add ndpi_memcasecmp, refactor mail protocol dissectors (#2849) 2025-05-24 16:18:58 +02:00
Luca Deri
c590dc4955 Added new APi calls
- ndpi_is_master_only_protocol()
- ndpi_normalize_protocol()

These two APi calls are used to normalize mater/app nDPI protocols
2025-05-20 23:14:59 +02:00
Ivan Nardi
9a6d0b3629
Fix unit test compilation on Window (#2802)
```
 ndpiReader.c:6415:28: error: initializer-string for array of 'unsigned char' truncates NUL terminator but destination lacks 'nonstring' attribute (65 chars into 64 available) [-Werror=unterminated-string-initialization]
 6415 |   u_char enc_dec_key[64] = "9dedb817e5a8805c1de62eb8982665b9a2b4715174c34d23b9a46ffafacfb2a7" /* SHA256("nDPI") */;
```
2025-04-28 11:59:50 +02:00
Ivan Nardi
78ca74cf45
TLS: avoid exporting TLS heuristic fingerprint as metadata (#2783) 2025-03-27 14:54:49 +01:00
Miguel Torres
25492cd4c7
Fix typo in -P help for ndpiReader (#2775) 2025-03-24 10:50:39 +01:00
Ivan Nardi
70728b3425
FastCGI: use specific metadata, not the HTTP ones (#2774)
We are going to use HTTP metadata only for real HTTP traffic; FastCGI
should be the only protocol using them improperly
2025-03-19 16:22:53 +01:00
Luca Deri
30c3613f2f Improved RTP dissection with EVS and other mobile voice codecs 2025-02-20 22:59:23 +01:00
Luca Deri
3bef87666d Exported RTP payload in packet metadata
Added ndpi_rtp_payload_type2str() API call
2025-02-19 11:34:19 +01:00
Luca Deri
e052e5b6b7 Further domain checks 2025-02-19 10:32:30 +01:00
Ivan Nardi
baca06bfd2
ndpiReader: print more DNS information (#2717) 2025-02-11 18:16:55 +01:00
Ivan Nardi
1dccaf37b0
DNS: fix check for DGA domain (#2716)
If we have a (potential) valid sub-classification, we shoudn't check for
DGA, even if the subclassification itself is disabled!
2025-02-11 15:48:53 +01:00
Luca Deri
1577955fca Added ndpi_find_protocol_qoe() API call
Updated (C)
2025-02-10 21:21:51 +01:00
Ivan Nardi
9a5533d796
ndpiReader: add some global statistics about FPC (#2680)
Enabled via `--dump-fpc-stats` option
2025-01-17 11:47:37 +01:00
Ivan Nardi
63a3547f99
Add (kind of) support for loading a list of JA4C malicious fingerprints (#2678)
It might be usefull to be able to match traffic against a list of
suspicious JA4C fingerprints

Use the same code/logic/infrastructure used for JA3C (note that we are
going to remove JA3C...)

See: #2551
2025-01-14 12:05:03 +01:00
Luca Deri
6b686be9d3
Fix code scanning alert no. 7: Multiplication result converted to larger type (#2677)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-01-13 20:13:46 +01:00
Ivan Nardi
72fd940301
Remove JA3C output from ndpiReader (#2667)
Removing JA3C is an big task. Let's start with a simple change having an
huge impact on unit tests: remove printing of JA3C information from
ndpiReader.

This way, when we will delete the actual code, the unit tests diffs
should be a lot simpler to look at.

Note that the information if the client/server cipher is weak or
obsolete is still available via flow risk

See: #2551
2025-01-12 13:24:27 +01:00
Ivan Nardi
c3d19be26f
ndpiReader: update JA statistics (#2646)
Show JA4C and JA3S information (instead of JA3C and JA3S)
See #2551 for context
2025-01-06 15:09:25 +01:00
Ivan Nardi
2e20f670dd
QUIC: extract "max idle timeout" parameter (#2649)
Even if it is only the proposed value by the client (and not the
negotiated one), it might be use as hint for timeout by the (external)
flows manager
2025-01-06 13:45:12 +01:00
Ivan Nardi
cae9fb9989
TLS: remove ESNI support (#2648)
ESNI has been superseded by ECH for years, now.
See: https://blog.cloudflare.com/encrypted-client-hello/
Set the existing flow risk if we still found this extension.
2025-01-06 11:04:50 +01:00
Ivan Nardi
803410542e
STUN/RTP: improve metadata extraction (#2641) 2024-12-11 15:28:00 +01:00
Ivan Nardi
4974f806f9
Add a configuration file to ndpiReader (#2629)
Example:
./example/ndpiReader --conf=./example/calls.conf -i ./tests/pcap/signal_videocall.pcapng -v2

Close #2608
2024-11-27 08:24:31 +01:00
Ivan Nardi
cff8bd1bb2
Update flow->flow_multimedia_types to a bitmask (#2625)
In the same flow, we can have multiple multimedia types
2024-11-25 10:12:48 +01:00
Alfredo Cardigliano
2dbf6b3cf8 Remove unused variable 2024-11-18 16:55:50 +01:00
Ivan Nardi
d7d942586a
ndpiReader: fix statistic about total number of flows (#2622)
When capturing live traffic, accounting and export of expired flows is
 wrong (see #2617).
Let's try to fix some statistics, at least
2024-11-18 13:01:53 +01:00
Ivan Nardi
1bda2bf414 SIP: extract some basic metadata 2024-11-12 13:34:25 +01:00
Toni
ccbbcdf283
Unify ndpi debug logging to always use a u16 protocol id (#2613)
* fixes SonarCloud complaint

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-11-11 15:17:33 +01:00
Luca Deri
35ef56cc24 Crash fix when -f is specified with a non-existing pcap file (-i) 2024-11-04 22:14:09 +01:00
Luca Deri
183175fc6b Exported is_ndpi_proto definition 2024-11-04 22:02:13 +01:00
Luca Deri
412ca8700f Added HTTP credentials extraction 2024-10-31 21:20:46 +01:00
Ivan Nardi
9da99075aa
TLS: export heuristic fingerprint as metadata (#2609) 2024-10-28 23:36:51 +01:00
Ivan Nardi
a55d8d95bc
ndpiReader: fix command line options used by wireshark (#2605) 2024-10-21 18:26:28 +02:00
Ivan Nardi
51556113a3
ndpiReader: add some statistics about monitoring (#2602) 2024-10-19 19:44:00 +02:00
Luca Deri
afc4d9e34d Added -L <domain suffix> for loading domain suffixes
Exported domainanme in JSON file (-K JSON)
2024-10-15 22:25:48 +02:00
Luca Deri
6b6dad4fdb Implemented nDPI TCP fingerprint 2024-10-15 21:21:05 +02:00
Ivan Nardi
521d0ca7a0
Add monitoring capability (#2588)
Allow nDPI to process the entire flows and not only the first N packets.
Usefull when the application is interested in some metadata spanning the
entire life of the session.

As initial step, only STUN flows can be put in monitoring.

See `doc/monitoring.md` for further details.

This feature is disabled by default.

Close #2583
2024-10-14 18:05:35 +02:00
Luca Deri
faaa5c5799 Added support for printing JA4r when enabled 2024-10-11 19:24:06 +02:00