Commit graph

56 commits

Author SHA1 Message Date
Luca Deri
14b076a58b Improved TCP fingerprint 2024-10-20 22:25:55 +02:00
Luca Deri
0cc84e4fdd Improved TCP fingepring calculation
Adde basidc OS detection based on TCP fingerprint
2024-10-18 23:47:34 +02:00
Luca Deri
0ef0752c80
Increased struct ndpi_flow_struct size (#2596)
Build fix
2024-10-18 07:17:03 +02:00
Ivan Nardi
2d7085a23e
STUN: if the same metadata is found multiple times, keep the first value (#2591) 2024-10-15 15:12:37 +02:00
Ivan Nardi
85501c9aaa
FPC: add DPI information (#2514)
If the flow is classified (via DPI) after the first packet, we should
use this information as FPC
2024-07-23 08:50:27 +02:00
Ivan Nardi
65e31b0ea3
FPC: small improvements (#2512)
Add printing of fpc_dns statistics and add a general cconfiguration option.
Rework the code to be more generic and ready to handle other logics.
2024-07-22 17:42:23 +02:00
Ivan Nardi
843e487270
Add infrastructure for explicit support of Fist Packet Classification (#2488)
Let's start with some basic helpers and with FPC based on flow addresses.

See: #2322
2024-07-03 18:02:07 +02:00
Nardi Ivan
526cf6f291 Zoom: remove "stun_zoom" LRU cache
Since 070a0908b we are able to detect P2P calls directly from the packet
content, without any correlation among flows
2024-06-17 10:19:55 +02:00
Maatuq
6127e04900
support rtp/rtcp over tcp (#2422) (#2457)
Support rtp/rtcp over tcp as per rfc4571.

Signed-off-by: mmaatuq <mahmoudmatook.mm@gmail.com>
2024-05-28 22:01:08 +02:00
Ivan Nardi
fd02baa13a
DTLS: fix JA4 fingerprint (#2446) 2024-05-21 18:13:25 +02:00
Toni
18e03a26ca
Add extra entropy checks and more precise(?) analysis. (#2383)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-09 15:24:11 +02:00
Ivan Nardi
95fe21015d
Remove "zoom" cache (#2420)
This cache was added in b6b4967aa, when there was no real Zoom support.
With 63f349319, a proper identification of multimedia stream has been
added, making this cache quite useless: any improvements on Zoom
classification should be properly done in Zoom dissector.

Tested for some months with a few 10Gbits links of residential traffic: the
cache pretty much never returned a valid hit.
2024-05-06 12:51:45 +02:00
Ivan Nardi
e31ef00715
TLS: avoid setting NDPI_TLS_SELFSIGNED_CERTIFICATE for webrtc traffic (#2417)
See RFC8122: it is quite likely that STUN/DTLS/SRTP flows use
self-signed certificates

Follow-up of b287d6ec8
2024-05-06 10:20:07 +02:00
Ivan Nardi
8edb2f133c
STUN: add support for ipv6 in some metadata (#2389) 2024-04-13 14:12:20 +02:00
Ivan Nardi
0535e54484
STUN: fix boundary checks on attribute list parsing (#2387)
Restore all unit tests.
Add some configuration knobs.
Fix the endianess.
2024-04-12 22:55:51 +02:00
Luca Deri
a8507c3308 STUN:
- Fixed issue with XOR-MAPPED-ADDRESS decoding
- Implemented MAPPED_ADDRESS (IPv4 only)
2024-04-12 15:49:42 +02:00
Ivan Nardi
1b3ef7d7b2
STUN: improve extraction of Mapped-Address metadata (#2370)
Enable parsing of Mapped-Address attribute for all STUN flows: that
means that STUN classification might require more packets.

Add a configuration knob to enable/disable this feature.

Note that we can have (any) STUN metadata also for flows *not*
classified as STUN (because of DTLS).

Add support for ipv6.

Restore the correct extra dissection logic for Telegram flows.
2024-04-08 10:24:51 +02:00
Nardi Ivan
f758f3605a STUN: dissection of TURN Channel Data 2024-03-20 09:39:15 +01:00
Ivan Nardi
56ce228a8b
Add a specific protocol id for audio/video calls made using Google apps (#2341)
Same logic already used for Signal/Whatsapp/Line/Facebook/...
2024-03-07 13:48:19 +01:00
Nardi Ivan
b287d6ec85 TLS: avoid setting some flow risks for webrtc traffic
Is quite rare to have a SNI or an ALPN on Client Hello of STUN/DTLS/SRTP
traffic
2024-02-26 10:20:05 +01:00
Ivan Nardi
c95e8c184e
STUN: fix category when sub-classification is set in "extra-dissection" data path (#2320) 2024-02-24 09:44:24 +01:00
Ivan Nardi
c83698c957
STUN: fix flow risks when DTLS packets are found (#2266)
When switching to (D)TLS dissector from the STUN one, we need to clear
any flow risks set from the latter (because we don't have anymore
`NDPI_PROTOCOL_STUN` in the classification results)
2024-01-19 08:57:39 +01:00
Ivan Nardi
40797521af
ndpiReader: add breed stats on output used for CI (#2236) 2024-01-05 13:02:39 +01:00
Vladimir Gavrilov
0f4d9f5054
Remove Google Hangouts/Duo stuff (#2233)
* Remove Google Hangouts/Duo support

* Update protocols.rst
2024-01-02 14:01:33 +01:00
Luca Deri
8285fffdae Implements JA4 Support (#2191) 2023-12-22 20:40:42 +01:00
Ivan Nardi
241c42ad7e
ndpiReader: fix guessed_flow_protocols statistic (#2203)
Increment the counter only if the flow has been guessed
2023-12-12 19:44:03 +01:00
Ivan Nardi
6f046df0dc
STUN: fix detection of DTLS (#2187)
Fix a memory leak
```
==97697==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01)
    #1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46
    #4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5
    #5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11
    #6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5
    #7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3
    #8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4
    #9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9
    #10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5
    #11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22
    #12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31
    #13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10
    #14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7
[...]
SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564
2023-11-30 09:09:40 +01:00
Ivan Nardi
b539b0d090
fuzz: improve coverage and remove dead code (#2135)
We are not able to remove custom rules: remove the empty stubs (which
originate from the original OpenDPI code).

`ndpi_guess_protocol_id()` is only called on the first packet of the
flow, so the bitmask `flow->excluded_protocol_bitmask` is always empty,
since we didn't call any dissectors yet.

Move another hash function to the dedicated source file.
2023-11-07 17:46:29 +01:00
Ivan Nardi
42d24f8799
STUN: major code rework (#2116)
Try to have a faster classification, on first packet; use standard extra
dissection data path for sub-classification, metadata extraction and
monitoring.

STUN caches:
* use the proper confidence value
* lookup into the caches only once per flow, after having found a proper
STUN classification

Add identification of Telegram VoIP calls.
2023-10-30 10:28:19 +01:00
Ivan Nardi
32b50f5aa4
IPv6: add support for IPv6 risk exceptions (#2122) 2023-10-29 12:14:20 +01:00
Ivan Nardi
e8e4b9e8ff
IPv6: add support for IPv6 risk tree (#2118)
Fix the script to download crawler addressess
2023-10-27 13:58:15 +02:00
Ivan Nardi
8b07be4b9f
Jabber: remove support for UDP (#2115)
Jabber/XMPP is only over TCP (even the name `ndpi_search_jabber_tcp`
suggests that...).

Bug introduced in 5266c726f
2023-10-26 20:16:27 +02:00
Ivan Nardi
611c3b66f0
ipv6: add support for ipv6 addresses lists (#2113) 2023-10-26 20:15:44 +02:00
Maatuq
4a8e7105b2
add ethereum protocol dissector. (#2111)
as explained here for bitcoin https://www.ntop.org/guides/nDPI/protocols.html#ndpi-protocol-bitcoin
the same is applicable for ethereum.
ethereum detection was removed from mining protocol and is now handled separately.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-10-25 12:44:33 +02:00
Toni
e70333de87
Added generic Google Protobuf dissector. (#2109)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-24 12:18:31 +02:00
Toni Uhlig
a443bba0dd Add CAN over Ethernet dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-23 13:45:56 +02:00
Toni Uhlig
f69909d49b Add Remote Management Control Protocol (RMCP).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-19 19:50:57 +02:00
Toni
a98d7ff433
Added HAProxy protocol. (#2088)
* fixed tests/do.sh.in failure print

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-02 18:10:47 +02:00
Ivan Nardi
6925890383
Add support for (un-encrypted) HTTP/2 (#2087)
Plaintext HTTP/2 is quite rare on the general "internet" but it is
used in some private networks (example: 5G core network)
2023-09-18 14:06:09 +02:00
Toni
e4d3d619bc
Add Service Location Protocol dissector. (#2036)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:50:46 +02:00
Luca Deri
fea09e825b Fixes risk mask exception handling while improving the overall performance 2023-07-14 19:52:34 +02:00
Ivan Nardi
2c7fb91794
Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
Regardless of the name, the removed trace doesn't contain meaningful
Hangout traffic.

Remove last piece of sub-classifiction based only on ip addresses.
2023-06-27 10:33:28 +02:00
Ivan Nardi
31a9da238c
STUN: add dissection of DTLS handshake (#2018) 2023-06-26 12:07:26 +02:00
Ivan Nardi
7e64d9f66d
RTP: rework code (#2021)
Try avoiding false positives: look for 3 RTP packets before classifing
the flow as such.

Add a generic function `is_rtp_or_rtcp()` to identify RTP/RTCP packets also
in other dissectors (see 3608ab01b commit message for an example)
2023-06-23 08:12:40 +02:00
Toni
1678888284
Add Apache Thrift protocol dissector. (#2007)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-22 13:07:32 +02:00
Nardi Ivan
570c75d601 STUN: fix detection over TCP
TCP framing is optional
2023-06-21 10:38:44 +02:00
Ivan Nardi
3608ab01b6
STUN: keep monitoring/processing STUN flows (#2012)
Look for RTP packets in the STUN sessions.
TODO: tell RTP from RTCP
2023-06-21 09:16:20 +02:00
Maatuq
e17fa1259a
Add bitcoing protocol dissector. (#1992)
* Add bitcoing protocol dissector.

* remove bitcoin protcol detection from mining.c
* add a new bitcoin deissector.
* add a new category: Cryptocurrency.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* Remove useless checks and add missing windows and docs file.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update affected tests.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* add a brief version.

Add notes on the difference between normal bitcoin protocol and the
mining protocol.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update enable_payload_stat test after dev rebasing.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

---------

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-05-31 07:31:01 +02:00
Ivan Nardi
b11e6a453b
Add support for Epic Games and GeForceNow/Nvidia (#1990) 2023-05-27 12:13:54 +02:00
Ivan Nardi
31d23aee56
All protocols should be excluded sooner or later (#1969)
For a lot of protocols, reduce the number of packets after which the
protocols dissector gives up.
The values are quite arbitary, tring to not impact on classification
2023-05-10 12:50:24 +02:00