Commit graph

1008 commits

Author SHA1 Message Date
Ivan Nardi
d4fb7b0aa1
fuzz: extend fuzzing coverage (#2696) 2025-01-23 15:23:01 +01:00
Ivan Nardi
9a5533d796
ndpiReader: add some global statistics about FPC (#2680)
Enabled via `--dump-fpc-stats` option
2025-01-17 11:47:37 +01:00
Ivan Nardi
af011e338e
TLS: remove JA3C (#2679)
Last step of removing JA3C fingerprint

Remove some duplicate tests: testing with ja4c/ja3s disabled is already
performed by `disable_metadata_and_flowrisks` configuration.

Close:#2551
2025-01-14 15:02:20 +01:00
Ivan Nardi
63a3547f99
Add (kind of) support for loading a list of JA4C malicious fingerprints (#2678)
It might be usefull to be able to match traffic against a list of
suspicious JA4C fingerprints

Use the same code/logic/infrastructure used for JA3C (note that we are
going to remove JA3C...)

See: #2551
2025-01-14 12:05:03 +01:00
Luca Deri
6b686be9d3
Fix code scanning alert no. 7: Multiplication result converted to larger type (#2677)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-01-13 20:13:46 +01:00
Luca Deri
1a1fa63dda Fixes https://github.com/ntop/nDPI/issues/2672 2025-01-13 20:06:31 +01:00
Ivan Nardi
72fd940301
Remove JA3C output from ndpiReader (#2667)
Removing JA3C is an big task. Let's start with a simple change having an
huge impact on unit tests: remove printing of JA3C information from
ndpiReader.

This way, when we will delete the actual code, the unit tests diffs
should be a lot simpler to look at.

Note that the information if the client/server cipher is weak or
obsolete is still available via flow risk

See: #2551
2025-01-12 13:24:27 +01:00
Ivan Nardi
c3d19be26f
ndpiReader: update JA statistics (#2646)
Show JA4C and JA3S information (instead of JA3C and JA3S)
See #2551 for context
2025-01-06 15:09:25 +01:00
Ivan Nardi
2e20f670dd
QUIC: extract "max idle timeout" parameter (#2649)
Even if it is only the proposed value by the client (and not the
negotiated one), it might be use as hint for timeout by the (external)
flows manager
2025-01-06 13:45:12 +01:00
Ivan Nardi
cae9fb9989
TLS: remove ESNI support (#2648)
ESNI has been superseded by ECH for years, now.
See: https://blog.cloudflare.com/encrypted-client-hello/
Set the existing flow risk if we still found this extension.
2025-01-06 11:04:50 +01:00
Ivan Nardi
803410542e
STUN/RTP: improve metadata extraction (#2641) 2024-12-11 15:28:00 +01:00
Ivan Nardi
a156d69ea4
STUN: fix monitoring (#2639) 2024-12-06 20:19:28 +01:00
Luca Deri
b1d1f8cfe0 STUN counter changes 2024-11-29 17:13:43 +01:00
Luca Deri
6b6b5c7c4e Enhanced STUN stats 2024-11-28 23:19:25 +01:00
Ivan Nardi
4974f806f9
Add a configuration file to ndpiReader (#2629)
Example:
./example/ndpiReader --conf=./example/calls.conf -i ./tests/pcap/signal_videocall.pcapng -v2

Close #2608
2024-11-27 08:24:31 +01:00
Ivan Nardi
cff8bd1bb2
Update flow->flow_multimedia_types to a bitmask (#2625)
In the same flow, we can have multiple multimedia types
2024-11-25 10:12:48 +01:00
Alfredo Cardigliano
2dbf6b3cf8 Remove unused variable 2024-11-18 16:55:50 +01:00
Ivan Nardi
d7d942586a
ndpiReader: fix statistic about total number of flows (#2622)
When capturing live traffic, accounting and export of expired flows is
 wrong (see #2617).
Let's try to fix some statistics, at least
2024-11-18 13:01:53 +01:00
Ivan Nardi
1bda2bf414 SIP: extract some basic metadata 2024-11-12 13:34:25 +01:00
Toni
ccbbcdf283
Unify ndpi debug logging to always use a u16 protocol id (#2613)
* fixes SonarCloud complaint

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-11-11 15:17:33 +01:00
Luca Deri
35ef56cc24 Crash fix when -f is specified with a non-existing pcap file (-i) 2024-11-04 22:14:09 +01:00
Luca Deri
183175fc6b Exported is_ndpi_proto definition 2024-11-04 22:02:13 +01:00
Luca Deri
412ca8700f Added HTTP credentials extraction 2024-10-31 21:20:46 +01:00
Ivan Nardi
9da99075aa
TLS: export heuristic fingerprint as metadata (#2609) 2024-10-28 23:36:51 +01:00
Ivan Nardi
a55d8d95bc
ndpiReader: fix command line options used by wireshark (#2605) 2024-10-21 18:26:28 +02:00
Ivan Nardi
9021e08901
ndpiReader: explicitly remove non ipv4/6 packets (#2601) 2024-10-19 21:44:32 +02:00
Ivan Nardi
51556113a3
ndpiReader: add some statistics about monitoring (#2602) 2024-10-19 19:44:00 +02:00
Luca Deri
0cc84e4fdd Improved TCP fingepring calculation
Adde basidc OS detection based on TCP fingerprint
2024-10-18 23:47:34 +02:00
Luca Deri
afc4d9e34d Added -L <domain suffix> for loading domain suffixes
Exported domainanme in JSON file (-K JSON)
2024-10-15 22:25:48 +02:00
Luca Deri
6b6dad4fdb Implemented nDPI TCP fingerprint 2024-10-15 21:21:05 +02:00
Ivan Nardi
521d0ca7a0
Add monitoring capability (#2588)
Allow nDPI to process the entire flows and not only the first N packets.
Usefull when the application is interested in some metadata spanning the
entire life of the session.

As initial step, only STUN flows can be put in monitoring.

See `doc/monitoring.md` for further details.

This feature is disabled by default.

Close #2583
2024-10-14 18:05:35 +02:00
Luca Deri
faaa5c5799 Added support for printing JA4r when enabled 2024-10-11 19:24:06 +02:00
Ivan Nardi
3e74c95d19
Fix unit tests on CI on GitHub Actions (#2587)
On CI, tests run in parallel, because of `NDPI_FORCE_PARALLEL_UTESTS`
define
2024-10-11 13:08:48 +02:00
Luca Deri
2e5edd2cc9 Added -N option for dumping/restoring the DNS cache (when enabled)
Example ndpiReader -i en0 --cfg=dpi.address_cache_size,32768 -N /tmp/a
2024-10-10 18:10:03 +02:00
Luca Deri
b9348e9d6e Added new API calls for serializing/restoring the DNS cache
-  bool ndpi_address_cache_dump(struct ndpi_address_cache *cache, char *path, u_int32_t epoch_now);
-  u_int32_t ndpi_address_cache_restore(struct ndpi_address_cache *cache, char *path, u_int32_t epoch_now);
2024-10-10 16:51:45 +02:00
Luca Deri
89316ab4c0 Test unit fix 2024-10-08 08:58:24 +02:00
Luca Deri
55fa92490a Implemented (disabled by default) DNS host cache. You can set the cache size as follows:
ndpiReader --cfg=dpi.address_cache_size,1000 -i <pcap>.pcap

In the above example the cache has up to 1000 entries.

In jcase ndpiReader exports data in JSON, the cache hostname (if found) is exported in the field server_hostname
2024-10-07 20:08:53 +02:00
Luca
45323e3bf8 Exports DNS A/AAAA responses (up to 4 addresses)
Changed the default to IPv4 (used to be IPv6) in case of DNS error response
2024-10-02 15:55:35 +02:00
Ivan Nardi
8972b74fd0
Fix builds on Windows (#2580)
Quick fix with latest Windows image on GitHub CI, where we got:
```
 ndpiReader.c:2860:38: error: '%s' directive output may be truncated writing up to 64 bytes into a region of size 63 [-Werror=format-truncation=]
 2860 |     snprintf(srcip, sizeof(srcip), "[%s]", flow->src_name);
      |                                      ^~
ndpiReader.c:2860:5: note: 'snprintf' output between 3 and 67 bytes into a destination of size 64
 2860 |     snprintf(srcip, sizeof(srcip), "[%s]", flow->src_name);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ndpiReader.c:2861:38: error: '%s' directive output may be truncated writing up to 64 bytes into a region of size 63 [-Werror=format-truncation=]
 2861 |     snprintf(dstip, sizeof(dstip), "[%s]", flow->dst_name);
      |                                      ^~
ndpiReader.c:2861:5: note: 'snprintf' output between 3 and 67 bytes into a destination of size 64
 2861 |     snprintf(dstip, sizeof(dstip), "[%s]", flow->dst_name);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
2024-10-01 15:51:16 +02:00
Ivan Nardi
bfab402e05
wireshark: extcap: allow configuration of OpenVPN/TLS heuristics via GUI (#2576) 2024-09-30 16:55:22 +02:00
Ivan Nardi
e2ed23a72a
Let the library returning the packet direction calculated internally (#2572)
wireshark, lua: add basic analysis of possible obfuscated flows
2024-09-27 18:51:47 +02:00
Ivan Nardi
ddd08f913c
Add some heuristics to detect encrypted/obfuscated/proxied TLS flows (#2553)
Based on the paper: "Fingerprinting Obfuscated Proxy Traffic with
Encapsulated TLS Handshakes".
See: https://www.usenix.org/conference/usenixsecurity24/presentation/xue-fingerprinting

Basic idea:
* the packets/bytes distribution of a TLS handshake is quite unique
* this fingerprint is still detectable if the handshake is
encrypted/proxied/obfuscated

All heuristics are disabled by default.
2024-09-24 14:20:31 +02:00
Luca Deri
9a8dc640be buffer lenghtt is now returned by ndpi_quick_encrypt() and ndpi_quick_deecrypt() 2024-09-24 12:51:55 +02:00
Luca Deri
9f3b7cfd65
Added ndpi_quick_encrypt() ndpi_quick_decrypt() APi calls (#2568)
* Added ndpi_quick_encrypt() ndpi_quick_decrypt(0 APi calls based on AES

* Added aes.c
2024-09-24 09:40:21 +02:00
Luca Deri
191694f797 Implemented ndpi_strrstr()
Fixed bug in ndpi_get_host_domain
2024-09-19 13:18:26 +02:00
Luca
1d1edfc1c7 domain lists are not loaded when -E is used 2024-09-17 19:04:01 +02:00
Ivan Nardi
9d07cf2811
fuzz: try to be a little bit faster (#2559)
Some fuzzers don't really need a real and complete local context.
Try to avoid setting it up, creating a simpler fake version with only the
features really needed.
That is a kind of experiment: if it works, we can extend the same logic
to other fuzzers
2024-09-17 11:46:55 +02:00
Ivan Nardi
1197cbdf58
wireshark: extcap: rework trailer header (#2557) 2024-09-16 22:19:20 +02:00
Luca
6de91c7895 Reworked fingerprint export now in JSON 2024-09-16 09:21:17 +02:00
Luca Deri
b77d3e3ab6 Enhanced DHCP fingerprint
Exported it with -E
2024-09-15 20:21:32 +02:00