Commit graph

129 commits

Author SHA1 Message Date
Ivan Nardi
91fd1bccd2
Rework the old MapleStory code to identify traffic from generic Nexon games (#2773)
Remove `NDPI_PROTOCOL_MAPLESTORY` and add a generic
`NDPI_PROTOCOL_NEXON`
2025-03-19 17:58:42 +01:00
Toni
6a591b67aa
Add GearUP Booster protocol dissector (heuristic based). (#2765)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-03-07 20:05:44 +01:00
Ivan Nardi
f568313363
Add configuration parameter to enable/disable export of flow risk info (#2761)
For the most common protocols, avoid creating the string message if we
are not going to use it
2025-03-05 16:14:03 +01:00
Ivan Nardi
c040faef08 ICMP: move all the logic to the proper dissector file
There are no reasons to keep entropy calculation and sanity checks code
on the "guessing" algorithm.
BTW, this change also fix the entropy calculation for non TCP/UDP/ICMP
flows
2025-02-28 19:11:59 +01:00
Ivan Nardi
678d284a25
TOR: fix ip lists (#2748)
One list is from ingress nodes (used for protocol classification) and
the second one is from exit nodes (used for flow risk check)
2025-02-27 10:55:54 +01:00
Toni
5858e1debf
Add LagoFast protocol dissector. (#2743)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-02-23 13:13:38 +01:00
Luca Deri
30c3613f2f Improved RTP dissection with EVS and other mobile voice codecs 2025-02-20 22:59:23 +01:00
Ivan Nardi
5f8545d97a
SSDP: add configuration for disabling metadata extraction (#2736) 2025-02-17 15:16:37 +01:00
Ivan Kapranov
6d2e0ce930
added metadata fields for M-NOTIFY (#2733) 2025-02-17 13:38:24 +01:00
Ivan Kapranov
ccb15db9b3
Implement SSDP Metadata export (#2729)
Close #2524
2025-02-16 17:04:16 +01:00
Ivan Nardi
7dc5890c0f
DNS: rework adding entries to the FPC-DNS cache (#2730)
Try to populate the FPC-DNS cache using directly the info from the current
packet, and not from the metadata saved in `struct ndpi_flow_struct`. This
will be important when adding monitoring support
2025-02-16 13:33:08 +01:00
Luca Deri
1577955fca Added ndpi_find_protocol_qoe() API call
Updated (C)
2025-02-10 21:21:51 +01:00
Ivan Nardi
dd4807f8ee
bittorrent: add configuration for "hash" metadata (#2706)
Fix confidence value for same TCP flows
2025-01-31 17:42:47 +01:00
Ivan Nardi
cf8f761b93
HTTP: add configuration for some metadata (#2704)
Extend file configuration for just subclassification.
2025-01-31 16:26:53 +01:00
Ivan Nardi
ecf0f8ace3
Create a specific configuration for classification only (#2689)
In some scenarios, you might not be interested in flow metadata or
flow-risks at all, but you might want only flow (sub-)classification.
Examples: you only want to forward the traffic according to the
classification or you are only interested in some protocol statistics.

Create a new configuration file (for `ndpiReader`, but you can trivially
adapt it for the library itself) allowing exactly that. You can use it
via: `ndpiReader --conf=example/only_classification.conf ...`

Note that this way, the nDPI overhead is lower because it might need
less packets per flow:
* TLS: nDPI processes only the CH (in most cases) and not also the SH
  and certificates
* DNS: only the request is processed (instead of both request and
  response)

We might extend the same "shortcut-logic" (stop processing the flow
immediately when there is a final sub-classification) for others
protocols.

Add the configuration options to enable/disable the extraction of some
TLS metadata.
2025-01-31 15:10:30 +01:00
Ivan Nardi
26824be51c Minor extension for custom nDPI 2025-01-23 15:22:16 +01:00
Luca Deri
1bda2a2650 Renamed ips_match to ndpi_ips_match 2025-01-17 19:20:45 +01:00
Ivan Nardi
252be78acc
STUN: improve detection of Telegram calls (#2671) 2025-01-14 17:33:34 +01:00
Ivan Nardi
af011e338e
TLS: remove JA3C (#2679)
Last step of removing JA3C fingerprint

Remove some duplicate tests: testing with ja4c/ja3s disabled is already
performed by `disable_metadata_and_flowrisks` configuration.

Close:#2551
2025-01-14 15:02:20 +01:00
Ivan Nardi
63a3547f99
Add (kind of) support for loading a list of JA4C malicious fingerprints (#2678)
It might be usefull to be able to match traffic against a list of
suspicious JA4C fingerprints

Use the same code/logic/infrastructure used for JA3C (note that we are
going to remove JA3C...)

See: #2551
2025-01-14 12:05:03 +01:00
Toni
9a0a3bb8e7
Improved WebSocket-over-HTTP detection (#2664)
* detect `chisel` SSH-over-HTTP-WebSocket
 * use `strncasecmp()` for `LINE_*` matching macros

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-01-11 11:23:42 +01:00
Ivan Nardi
4756904222
QUIC: remove extraction of user-agent (#2650)
In very old (G)QUIC versions by Google, the user agent was available on
plain text. That is not true anymore, since about end of 2021.
See: f282c934f4
2025-01-07 19:58:43 +01:00
Ivan Nardi
bf830b4236
Add the ability to enable/disable every specific flow risks (#2653) 2025-01-06 16:53:29 +01:00
Ivan Nardi
2e20f670dd
QUIC: extract "max idle timeout" parameter (#2649)
Even if it is only the proposed value by the client (and not the
negotiated one), it might be use as hint for timeout by the (external)
flows manager
2025-01-06 13:45:12 +01:00
Luca Deri
170849f156 Removed old USE_LEGACY_AHO_CORASICK code 2024-11-26 18:43:22 +01:00
Ivan Nardi
cff8bd1bb2
Update flow->flow_multimedia_types to a bitmask (#2625)
In the same flow, we can have multiple multimedia types
2024-11-25 10:12:48 +01:00
Ivan Nardi
c5bd9d8bff
RTP, STUN: improve detection of multimedia flow type (#2620)
Let's see if we are able to tell audio from video calls only looking at
RTP Payload Type field...
2024-11-19 16:38:14 +01:00
Luca Deri
de8c326cd2 Added ndpi_intoav6()
Implemented Mikrotik JSON serialization
2024-11-17 10:07:29 +01:00
Luca
4fd12278b1 Added DICOM support
Testing pcaps courtesy of https://github.com/virtalabs/tapirx.git
2024-11-15 18:45:51 +01:00
Luca Deri
3ce8d0e508
Implemented Mikrotik discovery protocol dissection and metadata extraction (#2618) 2024-11-14 23:34:31 +01:00
Ivan Nardi
1bda2bf414 SIP: extract some basic metadata 2024-11-12 13:34:25 +01:00
Vladimir Gavrilov
dc125dc2a8
Add Paltalk protocol support (#2606) 2024-10-28 16:57:05 +01:00
Ivan Nardi
819291b7e4
Add configuration of TCP fingerprint computation (#2598)
Extend configuration of raw format of JA4C fingerprint
2024-10-18 16:58:06 +02:00
Ivan Nardi
521d0ca7a0
Add monitoring capability (#2588)
Allow nDPI to process the entire flows and not only the first N packets.
Usefull when the application is interested in some metadata spanning the
entire life of the session.

As initial step, only STUN flows can be put in monitoring.

See `doc/monitoring.md` for further details.

This feature is disabled by default.

Close #2583
2024-10-14 18:05:35 +02:00
Luca Deri
ec5efe5cf2 Added sonos dissector 2024-10-13 18:50:34 +02:00
Luca Deri
55fa92490a Implemented (disabled by default) DNS host cache. You can set the cache size as follows:
ndpiReader --cfg=dpi.address_cache_size,1000 -i <pcap>.pcap

In the above example the cache has up to 1000 entries.

In jcase ndpiReader exports data in JSON, the cache hostname (if found) is exported in the field server_hostname
2024-10-07 20:08:53 +02:00
Vladimir Gavrilov
6cb1631132
Add DingTalk protocol support (#2581) 2024-10-07 15:45:51 +02:00
Ivan Nardi
e2ed23a72a
Let the library returning the packet direction calculated internally (#2572)
wireshark, lua: add basic analysis of possible obfuscated flows
2024-09-27 18:51:47 +02:00
Liam Wilson
cdda369e92
Add enable/disable guessing using client IP/port (#2569)
Add configurable options for whether to include client port or client IP
in the flow's protocol guesses. This defaults to include both client
port/IP if the protocol is not guessed with the server IP/port.

This is intended for when flow direction detection is enabled, so we
know that sport = client port, dport = server port.
2024-09-27 09:23:22 +02:00
Ivan Nardi
ddd08f913c
Add some heuristics to detect encrypted/obfuscated/proxied TLS flows (#2553)
Based on the paper: "Fingerprinting Obfuscated Proxy Traffic with
Encapsulated TLS Handshakes".
See: https://www.usenix.org/conference/usenixsecurity24/presentation/xue-fingerprinting

Basic idea:
* the packets/bytes distribution of a TLS handshake is quite unique
* this fingerprint is still detectable if the handshake is
encrypted/proxied/obfuscated

All heuristics are disabled by default.
2024-09-24 14:20:31 +02:00
Liam Wilson
80971e4a17
Allow IP guess before port in ndpi_detection_giveup (#2562)
Add dpi.guess_ip_before_port which when enabled uses classification
by-ip before classification by-port.
2024-09-20 10:25:41 +02:00
Ivan Nardi
0ddbda1f82
Add an heuristic to detect encrypted/obfuscated OpenVPN flows (#2547)
Based on the paper: "OpenVPN is Open to VPN Fingerprinting"
See: https://www.usenix.org/conference/usenixsecurity22/presentation/xue-diwen

Basic idea:
* the distribution of the first byte of the messages (i.e. the distribution
of the op-codes) is quite unique
* this fingerprint might be still detectable even if the OpenVPN packets are
somehow fully encrypted/obfuscated

The heuristic is disabled by default.
2024-09-16 18:38:26 +02:00
Luca Deri
42ded07139 Implemented JA4 raw (ja4_r) fingerprint
Example:
./example/ndpiReader -i tests/pcap/safari.pcap --cfg=tls,metadata.ja4r_fingerprint,1
2024-09-05 19:47:01 +02:00
Nardi Ivan
85ebda434d OpenVPN, Wireguard: improve sub-classification
Allow sub-classification of OpenVPN/Wireguard flows using their server IP.
That is useful to detect the specific VPN application/app used.
At the moment, the supported protocols are: Mullvad, NordVPN, ProtonVPN.

This feature is configurable.
2024-09-05 16:36:50 +02:00
Vladimir Gavrilov
81eaa3bd52
Add Lustre protocol detection support (#2544) 2024-09-04 10:22:04 +02:00
Ivan Nardi
338eedd05b
HTTP, QUIC, TLS: allow to disable sub-classification (#2533) 2024-09-03 12:35:45 +02:00
Vladimir Gavrilov
64a5dc3cb3
Add TRDP protocol support (#2528)
The Train Real Time Data Protocol (TRDP) is a UDP/TCP-based communication protocol designed for IP networks in trains, enabling data exchange between devices such as door controls and air conditioning systems. It is standardized by the IEC under IEC 61375-2-3 and is not related to the Remote Desktop Protocol (RDP).
2024-08-25 13:31:39 +02:00
wssxsxxsx
8894ebc76f
Add Automatic Tank Gauge protocol (#2527)
See also #2523

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-08-23 22:35:08 +02:00
Vladimir Gavrilov
a10c48c80a
Add CNP/IP protocol support (#2521)
ISO/IEC 14908-4 defines how to tunnel Control Network Protocol (CNP) over IP networks. It encapsulates protocols like EIA-709, EIA-600, and CNP, making it a versatile solution for building automation and control systems.
2024-08-22 15:26:32 +02:00
Vladimir Gavrilov
b15337a32b
Add OpenWire support (#2513) 2024-07-22 19:20:44 +02:00