Commit graph

192 commits

Author SHA1 Message Date
Ivan Nardi
d738b60cac
DNS: evaluate all flow risks even if sub-classification is disabled (#2714) 2025-02-11 13:16:03 +01:00
Ivan Nardi
dd4807f8ee
bittorrent: add configuration for "hash" metadata (#2706)
Fix confidence value for same TCP flows
2025-01-31 17:42:47 +01:00
Ivan Nardi
cf8f761b93
HTTP: add configuration for some metadata (#2704)
Extend file configuration for just subclassification.
2025-01-31 16:26:53 +01:00
Ivan Nardi
ecf0f8ace3
Create a specific configuration for classification only (#2689)
In some scenarios, you might not be interested in flow metadata or
flow-risks at all, but you might want only flow (sub-)classification.
Examples: you only want to forward the traffic according to the
classification or you are only interested in some protocol statistics.

Create a new configuration file (for `ndpiReader`, but you can trivially
adapt it for the library itself) allowing exactly that. You can use it
via: `ndpiReader --conf=example/only_classification.conf ...`

Note that this way, the nDPI overhead is lower because it might need
less packets per flow:
* TLS: nDPI processes only the CH (in most cases) and not also the SH
  and certificates
* DNS: only the request is processed (instead of both request and
  response)

We might extend the same "shortcut-logic" (stop processing the flow
immediately when there is a final sub-classification) for others
protocols.

Add the configuration options to enable/disable the extraction of some
TLS metadata.
2025-01-31 15:10:30 +01:00
Ivan Nardi
5ec0e1da84
CI: fix documentation job (#2698) 2025-01-24 17:55:47 +01:00
Ivan Nardi
f3532f0bad
Unify "Skype" and "Teams" ids (#2687)
* Rename `NDPI_PROTOCOL_SKYPE_TEAMS_CALL` ->
  `NDPI_PROTOCOL_MSTEAMS_CALL`

* Rename ip list from "Skype/Teams" to "Teams"
2025-01-20 18:06:56 +01:00
Ivan Nardi
949bff7741
Minor follow-up for DigitalOcean support (#2682) 2025-01-17 20:38:35 +01:00
Ivan Nardi
af011e338e
TLS: remove JA3C (#2679)
Last step of removing JA3C fingerprint

Remove some duplicate tests: testing with ja4c/ja3s disabled is already
performed by `disable_metadata_and_flowrisks` configuration.

Close:#2551
2025-01-14 15:02:20 +01:00
Luca Deri
69a4f8120a Temporarely reverts d351907af8 2025-01-13 20:38:37 +01:00
Vladimir Gavrilov
674428d824
Add Vivox support (#2668) 2025-01-11 19:37:31 +01:00
Ivan Nardi
d351907af8
Improve documentation (#2661)
Integrate .md files into official documentation
See: https://stackoverflow.com/questions/46278683/include-my-markdown-readme-into-sphinx/68005314#68005314
2025-01-10 19:22:47 +01:00
Ivan Nardi
bf830b4236
Add the ability to enable/disable every specific flow risks (#2653) 2025-01-06 16:53:29 +01:00
Evgeny Shtanov
74792e49c8
Add support Yandex Alice (#2633)
Co-authored-by: Evgeny Shtanov <evg.shtanov@gmail.comm>
Co-authored-by: Ivan Nardi <nardi.ivan@gmail.com>
2024-11-29 14:13:36 +01:00
Ivan Nardi
7330f65939 Add support for Paramount+ streaming service 2024-11-25 14:01:55 +01:00
Ivan Nardi
59ee1fe115
Add support for some Chinese shopping platforms (Temu, Shein and Taobao) (#2615)
Extend content match list
2024-11-12 20:11:07 +01:00
Ivan Nardi
1bda2bf414 SIP: extract some basic metadata 2024-11-12 13:34:25 +01:00
Vladimir Gavrilov
137d87fd87
Add Naver protocol support (#2610) 2024-11-01 14:56:25 +01:00
Vladimir Gavrilov
dc125dc2a8
Add Paltalk protocol support (#2606) 2024-10-28 16:57:05 +01:00
Ivan Nardi
819291b7e4
Add configuration of TCP fingerprint computation (#2598)
Extend configuration of raw format of JA4C fingerprint
2024-10-18 16:58:06 +02:00
Ivan Nardi
521d0ca7a0
Add monitoring capability (#2588)
Allow nDPI to process the entire flows and not only the first N packets.
Usefull when the application is interested in some metadata spanning the
entire life of the session.

As initial step, only STUN flows can be put in monitoring.

See `doc/monitoring.md` for further details.

This feature is disabled by default.

Close #2583
2024-10-14 18:05:35 +02:00
Vladimir Gavrilov
6cb1631132
Add DingTalk protocol support (#2581) 2024-10-07 15:45:51 +02:00
Ivan Nardi
623b7e236f
TLS: detect abnormal padding usage (#2579)
Padding is usually some hundreds byte long. Longer padding might be used
as obfuscation technique to force unusual CH fragmentation
2024-10-01 17:15:03 +02:00
Liam Wilson
cdda369e92
Add enable/disable guessing using client IP/port (#2569)
Add configurable options for whether to include client port or client IP
in the flow's protocol guesses. This defaults to include both client
port/IP if the protocol is not guessed with the server IP/port.

This is intended for when flow direction detection is enabled, so we
know that sport = client port, dport = server port.
2024-09-27 09:23:22 +02:00
Ivan Nardi
ddd08f913c
Add some heuristics to detect encrypted/obfuscated/proxied TLS flows (#2553)
Based on the paper: "Fingerprinting Obfuscated Proxy Traffic with
Encapsulated TLS Handshakes".
See: https://www.usenix.org/conference/usenixsecurity24/presentation/xue-fingerprinting

Basic idea:
* the packets/bytes distribution of a TLS handshake is quite unique
* this fingerprint is still detectable if the handshake is
encrypted/proxied/obfuscated

All heuristics are disabled by default.
2024-09-24 14:20:31 +02:00
Liam Wilson
80971e4a17
Allow IP guess before port in ndpi_detection_giveup (#2562)
Add dpi.guess_ip_before_port which when enabled uses classification
by-ip before classification by-port.
2024-09-20 10:25:41 +02:00
Ivan Nardi
0ddbda1f82
Add an heuristic to detect encrypted/obfuscated OpenVPN flows (#2547)
Based on the paper: "OpenVPN is Open to VPN Fingerprinting"
See: https://www.usenix.org/conference/usenixsecurity22/presentation/xue-diwen

Basic idea:
* the distribution of the first byte of the messages (i.e. the distribution
of the op-codes) is quite unique
* this fingerprint might be still detectable even if the OpenVPN packets are
somehow fully encrypted/obfuscated

The heuristic is disabled by default.
2024-09-16 18:38:26 +02:00
Ivan Nardi
bc9472277d
RTMP: improve detection (#2549) 2024-09-10 16:33:06 +02:00
Nardi Ivan
2964c23ca1 Add detection of Windscribe VPN 2024-09-05 16:36:50 +02:00
Nardi Ivan
c99646e4af Add detection of CactusVPN 2024-09-05 16:36:50 +02:00
Nardi Ivan
5b0374c28b Add detection of SurfShark VPN 2024-09-05 16:36:50 +02:00
Nardi Ivan
85ebda434d OpenVPN, Wireguard: improve sub-classification
Allow sub-classification of OpenVPN/Wireguard flows using their server IP.
That is useful to detect the specific VPN application/app used.
At the moment, the supported protocols are: Mullvad, NordVPN, ProtonVPN.

This feature is configurable.
2024-09-05 16:36:50 +02:00
Nardi Ivan
f350379e95 Add detection of NordVPN 2024-09-05 16:36:50 +02:00
Vladimir Gavrilov
81eaa3bd52
Add Lustre protocol detection support (#2544) 2024-09-04 10:22:04 +02:00
Ivan Nardi
338eedd05b
HTTP, QUIC, TLS: allow to disable sub-classification (#2533) 2024-09-03 12:35:45 +02:00
Alfredo Cardigliano
2d040247a7 Update doc 2024-09-02 18:11:02 +02:00
Vladimir Gavrilov
64a5dc3cb3
Add TRDP protocol support (#2528)
The Train Real Time Data Protocol (TRDP) is a UDP/TCP-based communication protocol designed for IP networks in trains, enabling data exchange between devices such as door controls and air conditioning systems. It is standardized by the IEC under IEC 61375-2-3 and is not related to the Remote Desktop Protocol (RDP).
2024-08-25 13:31:39 +02:00
wssxsxxsx
8894ebc76f
Add Automatic Tank Gauge protocol (#2527)
See also #2523

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-08-23 22:35:08 +02:00
Vladimir Gavrilov
a10c48c80a
Add CNP/IP protocol support (#2521)
ISO/IEC 14908-4 defines how to tunnel Control Network Protocol (CNP) over IP networks. It encapsulates protocols like EIA-709, EIA-600, and CNP, making it a versatile solution for building automation and control systems.
2024-08-22 15:26:32 +02:00
Vladimir Gavrilov
b15337a32b
Add OpenWire support (#2513) 2024-07-22 19:20:44 +02:00
Ivan Nardi
65e31b0ea3
FPC: small improvements (#2512)
Add printing of fpc_dns statistics and add a general cconfiguration option.
Rework the code to be more generic and ready to handle other logics.
2024-07-22 17:42:23 +02:00
mmanoj
67f5cdafc0
FPC: add DNS correlation (#2497)
Use DNS information to get a better First Packet Classification.

See: #2322

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-07-22 11:34:07 +02:00
Petr
2a3f4dc8b4
Performed some grammar and typo fixes (#2511) 2024-07-19 11:22:35 +02:00
Vladimir Gavrilov
6a77a891a8
Add Nano (XNO) protocol support (#2508) 2024-07-18 16:18:12 +02:00
Vladimir Gavrilov
c3fff52646
Add HLS support (#2502) 2024-07-16 12:01:28 +02:00
Toni
8fd649ab1e
Add Ripe Atlas probe protocol. (#2473)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-06-17 11:00:38 +02:00
Nardi Ivan
526cf6f291 Zoom: remove "stun_zoom" LRU cache
Since 070a0908b we are able to detect P2P calls directly from the packet
content, without any correlation among flows
2024-06-17 10:19:55 +02:00
Mark Jeffery
f796c94375
Added protocol - JRMI - Java Remote Method Invocation (#2470) 2024-06-15 10:52:28 +02:00
Ivan Nardi
b90d39c4ac
RTP/STUN: look for STUN packets after RTP/RTCP classification (#2465)
After a flow has been classified as RTP or RTCP, nDPI might analyse more
packets to look for STUN/DTLS packets, i.e. to try to tell if this flow
is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with
STUN/DTLS.
Useful for proper (sub)classification when the beginning of the flows
are not captured or if there are lost packets in the the captured traffic.

Disabled by default
2024-06-07 13:12:04 +02:00
Ivan Nardi
070a0908b3
Zoom: faster detection of P2P flows (#2467) 2024-06-07 09:50:41 +02:00
Toni
80171dbcf3
Add ZUG consensus protocol dissector. (#2458)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-28 20:29:48 +02:00