Commit graph

116 commits

Author SHA1 Message Date
Vladimir Gavrilov
dc125dc2a8
Add Paltalk protocol support (#2606) 2024-10-28 16:57:05 +01:00
Ivan Nardi
ad9c5744f7 Fix Windows build 2024-10-13 19:38:23 +02:00
Ivan Nardi
57776d026d Fix compilation on Windows 2024-10-13 19:22:18 +02:00
Luca Deri
ec5efe5cf2 Added sonos dissector 2024-10-13 18:50:34 +02:00
Vladimir Gavrilov
6cb1631132
Add DingTalk protocol support (#2581) 2024-10-07 15:45:51 +02:00
Luca Deri
7629b94a26 Moved ndpi_lru in a separate file 2024-10-04 14:51:08 +02:00
Luca Deri
9f3b7cfd65
Added ndpi_quick_encrypt() ndpi_quick_decrypt() APi calls (#2568)
* Added ndpi_quick_encrypt() ndpi_quick_decrypt(0 APi calls based on AES

* Added aes.c
2024-09-24 09:40:21 +02:00
Luca Deri
7fdc4b2472
Implemented algorithms for K-Nearest Neighbor Search (KNN) (#2554)
* Extended API with functions for vector similarity based on KD-trees https://en.wikipedia.org/wiki/K-d_tree

ndpi_kd_tree* ndpi_kd_create(u_int num_dimensions);
void ndpi_kd_free(ndpi_kd_tree *tree);
void ndpi_kd_clear(ndpi_kd_tree *tree);
bool ndpi_kd_insert(ndpi_kd_tree *tree, const double *data_vector, void *user_data);
ndpi_kd_tree_result *ndpi_kd_nearest(ndpi_kd_tree *tree, const double *data_vector);
u_int32_t ndpi_kd_num_results(ndpi_kd_tree_result *res);
bool ndpi_kd_result_end(ndpi_kd_tree_result *res);
double* ndpi_kd_result_get_item(ndpi_kd_tree_result *res, double **user_data);
bool ndpi_kd_result_next(ndpi_kd_tree_result *res);
void ndpi_kd_result_free(ndpi_kd_tree_result *res);
double ndpi_kd_distance(double *a1, double *b2, u_int num_dimensions);
2024-09-10 16:22:06 +02:00
Vladimir Gavrilov
81eaa3bd52
Add Lustre protocol detection support (#2544) 2024-09-04 10:22:04 +02:00
Vladimir Gavrilov
64a5dc3cb3
Add TRDP protocol support (#2528)
The Train Real Time Data Protocol (TRDP) is a UDP/TCP-based communication protocol designed for IP networks in trains, enabling data exchange between devices such as door controls and air conditioning systems. It is standardized by the IEC under IEC 61375-2-3 and is not related to the Remote Desktop Protocol (RDP).
2024-08-25 13:31:39 +02:00
wssxsxxsx
8894ebc76f
Add Automatic Tank Gauge protocol (#2527)
See also #2523

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-08-23 22:35:08 +02:00
Vladimir Gavrilov
a10c48c80a
Add CNP/IP protocol support (#2521)
ISO/IEC 14908-4 defines how to tunnel Control Network Protocol (CNP) over IP networks. It encapsulates protocols like EIA-709, EIA-600, and CNP, making it a versatile solution for building automation and control systems.
2024-08-22 15:26:32 +02:00
Vladimir Gavrilov
b15337a32b
Add OpenWire support (#2513) 2024-07-22 19:20:44 +02:00
Vladimir Gavrilov
6a77a891a8
Add Nano (XNO) protocol support (#2508) 2024-07-18 16:18:12 +02:00
Ivan Nardi
456f0fd427
Improve detection of Cloudflare WARP traffic (#2491)
See: #2484
2024-07-04 08:59:04 +02:00
Toni
8fd649ab1e
Add Ripe Atlas probe protocol. (#2473)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-06-17 11:00:38 +02:00
Mark Jeffery
f796c94375
Added protocol - JRMI - Java Remote Method Invocation (#2470) 2024-06-15 10:52:28 +02:00
Toni
80171dbcf3
Add ZUG consensus protocol dissector. (#2458)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-28 20:29:48 +02:00
Vladimir Gavrilov
3d1da00d8d
Add Call of Duty Mobile support (#2438) 2024-05-15 12:46:02 +02:00
Vladimir Gavrilov
2d33431948
Add Ethernet Global Data support (#2437) 2024-05-11 16:49:16 +02:00
Luca Deri
6195dbad76 Renamed radius source file to avoid name clashes on Windows 2024-05-07 09:05:15 +02:00
Ivan Nardi
266af02752
Merge RTP and RTCP logic (#2416)
Avoid code duplication between these two protocols.

We remove support for RTCP over TCP; it is quite rare to find this kind
of traffic and, more important, we have never had support for RTP
over TCP: we should try to add both detecion as follow-up.

Fix a message log in the LINE code
2024-05-06 10:19:46 +02:00
0x41CEA55
66036a14f4
Remove PPStream protocol and add iQIYI (#2403)
P2P video player PPStream was discontinued shortly after the purchase of PPS.tv by Baidu (iQIYI) on 2013 (see https://www.techinasia.com/report-baidu-acquires-video-rival-pps)
So we remove the old `NDPI_PROTOCOL_PPSTREAM` logic and add `NDPI_PROTOCOL_IQIYI` id to handle all the iQIYI traffic, which is basically video streaming traffic.

A video hosting service, called PPS.tv, is still offered by the same company: for the time being we classified both services with the same protocol id.
2024-04-23 18:01:36 +02:00
0x41CEA55
fd388845d5
Add BFCP protocol support (#2401) 2024-04-23 15:35:19 +02:00
0x41CEA55
905120588b
Remove obsolete protocols: tuenty, tvuplayer and kontiki (#2398) 2024-04-19 21:35:32 +02:00
0x41CEA55
e75d7a620e
Add KNXnet/IP protocol support (#2397)
* Add KNXnet/IP protocol support

* Improve KNXnet/IP over TCP detection
2024-04-19 12:54:00 +02:00
Luca Deri
ad117bfaab
Domain Classification Improvements (#2396)
* Added
size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize);
size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize);

used to compress short strings such as domain names. This code is based on
https://github.com/Ed-von-Schleck/shoco

* Major code rewrite for ndpi_hash and ndpi_domain_classify

* Improvements to make sure custom categories are loaded and enabled

* Fixed string encoding

* Extended SalesForce/Cloudflare domains list
2024-04-18 23:21:40 +02:00
Vladimir Gavrilov
c63697205b
Add Label Distribution Protocol support (#2385)
* Add Label Distribution Protocol support

* Fix typo

* Update unit test results
2024-04-12 17:44:36 +02:00
Vladimir Gavrilov
9ff4bece33
Add The Elder Scrolls Online support (#2376)
* Add The Elder Scrolls Online support

* Use ndpi_memmem instead of memmem from libc

* Add protocol description

* Change selection bitmask to V4_V6

* Update protocols.rst
2024-04-10 18:04:02 +02:00
Vladimir Gavrilov
5b32c98a21
Add LoL: Wild Rift detection (#2356) 2024-03-26 08:11:14 +01:00
Luca Deri
51f5fc7140
Added support for roaring bitmap v3 (#2355)
* Integrated RoaringBitmap v3

* Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse

* Fixes to ndpi_bitmap for new roaring library

* Fixes for bitmap serialization

* Fixed format

* Warning fix

* Conversion fix

* Warning fix

* Added check for roaring v3 support

* Updated file name

* Updated path

* Uses clang-9 (instead of clang-7) for builds

* Fixed fuzz_ds_bitmap64_fuse

* Fixes nDPI printf handling

* Disabled printf

* Yet another printf fix

* Cleaup

* Fx for compiling on older platforms

* Fixes for old compilers

* Initialization changes

* Added compiler check

* Fixes for old compilers

* Inline function is not static inline

* Added missing include
2024-03-25 08:15:19 +01:00
Vladimir Gavrilov
e6474d835f
Add FLUTE protocol dissector (#2351)
* Add FLUTE protocol dissector

* Add flute.c to MSVC project
2024-03-19 09:11:04 +01:00
Vladimir Gavrilov
8fad77991d
Add PFCP protocol dissector (#2342) 2024-03-13 20:18:43 +01:00
Vladimir Gavrilov
e2949048e0
Add Path of Exile protocol dissector (#2337)
* Add Path of Exile protocol dissector

* Update protocols.rst
2024-03-06 19:59:09 +01:00
Vladimir Gavrilov
e7f1946ae7
Add NetEase Games detection support (#2335) 2024-03-05 11:00:52 +01:00
Vladimir Gavrilov
58fdc9fafb
Add Naraka Bladepoint detection support (#2334) 2024-03-04 08:30:54 +01:00
Vladimir Gavrilov
f2e3c7fb90
Add BFD protocol dissector (#2332) 2024-02-29 08:19:00 +01:00
Vladimir Gavrilov
66b6e2b3f2
Add DLEP protocol dissector (#2326) 2024-02-20 16:05:41 +01:00
Vladimir Gavrilov
e93bcfd619
Add ANSI C12.22 protocol dissector (#2317)
* Add ANSI C12.22 protocol dissector

* Add UDP sample
2024-02-15 09:36:06 +01:00
Ivan Nardi
ae36648c6c
Skype: remove old detection logic (#1954)
Skype has been using standard protocols (STUN/ICE or TLS) for a long,
long time, now. Long gone are the days of Skype as a distribuited
protocol.

See: #2166
2024-02-12 18:39:24 +01:00
Vladimir Gavrilov
f5cec001f3
Add detection of Gaijin Entertainment games (#2311)
* Add detection of Gaijin Entertainment games

* Short NDPI_PROTOCOL_GAIJINENTERTAINMENT to NDPI_PROTOCOL_GAIJIN

* Add default UDP port for Gaijin Entertainment games

* Remove NDPI_PROTOCOL_CROSSOUT protocol id
2024-02-09 11:07:48 +01:00
Vladimir Gavrilov
6207be43fa
Add TencentGames protocol dissector (#2306) 2024-02-08 08:10:35 +01:00
Vladimir Gavrilov
4c9446379f
Add Gearman protocol dissector (#2297) 2024-02-01 19:46:57 +01:00
Ivan Nardi
400cd516b5
Allow multiple struct ndpi_detection_module_struct to share some state (#2271)
Add the concept of "global context".

Right now every instance of `struct ndpi_detection_module_struct` (we
will call it "local context" in this description) is completely
independent from each other. This provide optimal performances in
multithreaded environment, where we pin each local context to a thread,
and each thread to a specific CPU core: we don't have any data shared
across the cores.

Each local context has, internally, also some information correlating
**different** flows; something like:
```
if flow1 (PeerA <-> Peer B) is PROTOCOL_X; then
  flow2 (PeerC <-> PeerD) will be PROTOCOL_Y
```
To get optimal classification results, both flow1 and flow2 must be
processed by the same local context. This is not an issue at all in the far
most common scenario where there is only one local context, but it might
be impractical in some more complex scenarios.

Create the concept of "global context": multiple local contexts can use
the same global context and share some data (structures) using it.
This way the data correlating multiple flows can be read/write from
different local contexts.
This is an optional feature, disabled by default.

Obviously data structures shared in a global context must be thread safe.
This PR updates the code of the LRU implementation to be, optionally,
thread safe.

Right now, only the LRU caches can be shared; the other main structures
(trees and automas) are basically read-only: there is little sense in
sharing them. Furthermore, these structures don't have any information
correlating multiple flows.

Every LRU cache can be shared, independently from the others, via
`ndpi_set_config(ndpi_struct, NULL, "lru.$CACHE_NAME.scope", "1")`.

It's up to the user to find the right trade-off between performances
(i.e. without shared data) and classification results (i.e. with some
shared data among the local contexts), depending on the specific traffic
patterns and on the algorithms used to balance the flows across the
threads/cores/local contexts.

Add some basic examples of library initialization in
`doc/library_initialization.md`.

This code needs libpthread as external dependency. It shouldn't be a big
issue; however a configure flag has been added to disable global context
support. A new CI job has been added to test it.

TODO: we should need to find a proper way to add some tests on
multithreaded enviroment... not an easy task...

*** API changes ***

If you are not interested in this feature, simply add a NULL parameter to
any `ndpi_init_detection_module()` calls.
2024-02-01 15:33:11 +01:00
Luca Deri
47f72443fa Implemented CIP I/O (UDP version of the CIP protocol), Common Industrial protocol 2024-01-29 13:28:41 +01:00
Vladimir Gavrilov
c807d84054
Fix RESP detection (#2289)
* Rename redis_net.c to resp.c

* Fix RESP detection
2024-01-27 21:19:34 +01:00
Toni
bcca89b78c
Add Raft protocol dissector. (#2286)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-25 20:26:18 +01:00
Vladimir Gavrilov
4e712e3ab5
Add Radmin protocol dissector (#2283)
* Add Radmin protocol dissector

* Update test results
2024-01-25 08:10:29 +01:00
Vladimir Gavrilov
f04b4450a1
Add STOMP protocol dissector (#2280) 2024-01-23 21:08:13 +01:00
Toni
7d24e1258d
Add Yojimbo (netcode) protocol dissector (#2277)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-21 17:47:42 +01:00