vrr/nDPI
2
0
Fork 0
mirror of https://github.com/vel21ripn/nDPI.git synced 2026-05-01 08:29:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
5656 commits 9 branches 12 tags 263 MiB
Commit graph

5656 commits

This branch
This branch
All branches
Author SHA1 Message Date
Luca Deri
e9751cec26
Added TLS Block Analysis (#3016) 
* Enabled TLS block analysis via --cfg=tls,blocks_analysis,1

* Added comment and optimization

* Updated output format

* Code cleanup
 2025-10-27 10:21:26 +01:00
Ivan Nardi
71033e0370
Extend http-url custom rules: support for category and breed (#3014) 2025-10-24 19:17:48 +02:00
Ivan Nardi
20892cf4fc
Extend values saved in hash data structure to u_int64_t (#3013) 
Move from `u_int32_t` to `u_int64_t`.
We want to be able to save protocol + category + breed in the same
entry.
 2025-10-24 17:58:08 +02:00
Ivan Nardi
4f18701b99
Rework internal functions (#3011) 
Better separation between:
* internal code, which should use `struct ndpi_flow_struct` data and
  only internal protocol ids
* public API should use `ndpi_protocol` and only public protocol ids

`ndpi_protocol` is no more only a "protocol"... should we rename it as
"ndpi_classification_results` or something similar?
 2025-10-24 11:54:06 +02:00
Toni
d9f7871692
Fix broken header install (#3012) 
* header files are expected to reside in prefix/includedir/ndpi/
   instead of prefix/includedir/

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2025-10-24 11:00:48 +02:00
Ivan Nardi
1b566135d7
Fix flow risks with custom rules (#3010) 2025-10-23 19:51:12 +02:00
Ivan Nardi
95aae105f9
fuzz: keep only real/interesting corpora (#3009) 2025-10-23 14:18:11 +02:00
Ivan Nardi
1fdb6df2b1
Fix FPC confidence with custom rules (#3008) 2025-10-23 12:29:39 +02:00
Ivan Nardi
01836e0071
Proper handling of internal/external ids in FPC; fix FPC with custom rules (#3007) 2025-10-22 21:28:12 +02:00
Ivan Nardi
faca0a6565 ndpiReader: improve statistics 2025-10-22 20:34:29 +02:00
Ivan Nardi
dae135151e Rework parsing of protocol parameters from custom rules 
Note that you can specify custom id mappings for internal protocols, yet
 2025-10-22 20:14:43 +02:00
Ivan Nardi
9a925abd28 Proper handling of internal/external ids in ndpi_detection_giveup() 2025-10-22 20:14:43 +02:00
Luca Deri
4ce936bd83
Reworked custom rule protocol classification (#3005) 2025-10-22 17:30:21 +02:00
Luca Deri
ef159add87 Updated test results 2025-10-22 11:39:50 +02:00
Luca
ce33286ba6 Added protocol mapping check 2025-10-22 11:36:46 +02:00
Luca Deri
5abe185e2c Added support for urlXXXX@proto in protos.txt 
Fixed varisous protocol mapping in custom protocols definition
 2025-10-22 09:00:58 +02:00
Ivan Nardi
b9c847a176 config: fix "only_classification" configuration 2025-10-21 20:19:56 +02:00
Ivan Nardi
00c0eb947b
Fix the hash statistics of public suffix lists (#3003) 
In the flow risk information always report the original domain name.
Extend the unit tests
 2025-10-21 17:34:25 +02:00
Luca Deri
eca94a4f8b Added wildcasd mapping support in categories 2025-10-21 16:58:40 +02:00
Luca Deri
79b74115d2 Fixes invalid initialization that caused the two commands below to return different results 
./example/ndpiReader -t -i ./tests/pcap/bets.pcapng -L ./lists/public_suffix_list.dat -G ./lists/
 ./example/ndpiReader -t -i ./tests/pcap/bets.pcapng -G ./lists/
 2025-10-21 15:10:28 +02:00
Ivan Nardi
f3ec1cca05
ndpi fingerprint: avoid calculating it for flows without TCP and TLS handshakes (#3002) 2025-10-20 20:39:03 +02:00
Ivan Nardi
9c27c2df3a
Allow to overwrite domain matching via custom rules (#2999) 
This is basically the revert of 0db12b1390 and 43d9caac00.
Add some tests about this feature
 2025-10-20 15:28:16 +02:00
Toni
bc6dc56d8f
Fix CI RPM build (switch to Alma Linux 8). Fix #2997 (#3001) 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2025-10-20 15:25:29 +02:00
Ivan Nardi
6eb63d9cf9
tests: fixed protocol ids for all custom rules (#3000) 
To ease PR/Commit comparisons
 2025-10-20 14:59:15 +02:00
Ivan Nardi
e7bba509fb
Follow-up of d69446893 (#2998) 
Update the documentation.
We can't return public id on `ndpi_guess_host_protocol_id()` because we
use that value internally:
```
src/lib/ndpi_main.c:  flow->guessed_protocol_id_by_ip = ndpi_guess_host_protocol_id(ndpi_str, flow);
```
 2025-10-19 12:38:51 +02:00
Luca Deri
6001c9f34c Akamai IP address download 2025-10-18 12:25:16 +02:00
Luca Deri
3a05aacf93 Updated spec file 2025-10-18 10:02:39 +02:00
Luca Deri
c28c0a5edc Updated test results 2025-10-18 00:43:18 +02:00
Luca Deri
50458af909 Updated test 2025-10-18 00:43:09 +02:00
Luca Deri
735e0df40c Updated test 2025-10-18 00:22:14 +02:00
Luca Deri
f80aa7845d Updated results 
Signed-off-by: Luca Deri <deri@ntop.org>
 2025-10-18 00:03:54 +02:00
Luca Deri
c734fe4142 Akamai IP addresses 2025-10-17 23:52:18 +02:00
Luca Deri
d69446893d Added NDPI_MISMATCHING_PROTOCOL_WITH_IP flow risk 
Fixed host protocol matching
Added NDPI_PROTOCOL_AKAMAI protocol
 2025-10-17 23:48:44 +02:00
Ivan Nardi
9d22805954
Add statistics about hash data structures (#2995) 2025-10-17 20:39:15 +02:00
Ivan Nardi
cc799c1872
fuzz: fix makefile (#2996) 2025-10-17 19:38:07 +02:00
Luca Deri
cb9e63fc8c Improved Android classification 2025-10-16 15:44:06 +02:00
Ivan Nardi
523fe3ebc4
doc: improve public API header documentation (#2985) 
This commit significantly improves the documentation quality in ndpi_api.h,
the main public API header file for nDPI.

Changes include:

1. Fixed 11 typos:
   - "fucntion" → "function"
   - "ckeck" → "check"
   - "guesses" → "guessed"
   - "searhing" → "searching"
   - "@paw" → "@par" (incorrect Doxygen tag)
   - "addeed" → "added"
   - "readeable" → "readable" (function name)
   - "creaign" → "creating"
   - "lenght" → "length" (3 occurrences)
   - "hosti tself" → "host itself"

2. Added comprehensive documentation for memory management functions:
   - ndpi_malloc(), ndpi_calloc(), ndpi_realloc()
   - ndpi_strdup(), ndpi_strndup()
   - ndpi_free()
   - ndpi_flow_malloc(), ndpi_flow_free()
   - ndpi_get_tot_allocated_memory()

   These critical functions were previously undocumented, which could
   confuse users about custom allocator support and memory tracking.

3. Documented high-priority utility functions:
   - ndpi_match_string_value() - automaton string matching
   - ndpi_strip_leading_trailing_spaces() - string trimming
   - ndpi_handle_risk_exceptions() - risk exception handling
   - set_ndpi_malloc(), set_ndpi_free() - custom allocator setup
   - set_ndpi_flow_malloc(), set_ndpi_flow_free() - flow allocator setup
   - set_ndpi_debug_function() - custom debug logging

4. Added detailed documentation for Community ID hash functions:
   - ndpi_flowv4_flow_hash() - IPv4 flow hashing
   - ndpi_flowv6_flow_hash() - IPv6 flow hashing
   - Added reference to Community ID specification
   - Clarified parameter byte ordering and buffer requirements

All documentation follows Doxygen format with @param and @return tags.
Build and tests verified: all tests pass (3/3).

Stats: +173 lines of documentation, -19 lines (typo fixes)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-14 21:07:16 +02:00
Ivan Nardi
8d68dc66dc
doc: Complete and enhance protocols documentation (#2984) 
This commit significantly improves the protocols.rst documentation by:

1. Adding 41 missing protocol entries (100% coverage achieved):
   - Popular services: Discord, Slack, Facebook, Twitter, YouTube,
     Dropbox, Gmail, Google Maps, LinkedIn, SoundCloud
   - Cloud & storage: Google Drive, Microsoft 365, OneDrive,
     Cloudflare, Windows Update
   - Developer tools: Git, Pastebin
   - IoT & Industrial: MQTT, SOMEIP, CIP, HART-IP
   - Financial: FIX protocol, DRDA
   - VPN & Security: PPTP, Teredo, Hotspot Shield, OpenDNS
   - Gaming: PlayStation, IMO, App/Play Store
   - Specialized: RX, EdgeCast, Tesla Services, and more

2. Enhancing existing protocol descriptions:
   - MS_RPCH: Added Exchange/Outlook context
   - RESP: Clarified Redis protocol details
   - S7COMM: Expanded industrial automation context
   - Oracle: Added enterprise RDBMS information
   - RakNet: Explained game networking middleware
   - GTP-U: Added mobile network context (3G/4G/5G)
   - HSRP: Explained Cisco redundancy protocol
   - PGM: Detailed reliable multicast capabilities
   - And several others with improved technical descriptions

3. Adding comprehensive reference links:
   - Official websites and documentation
   - RFC specifications for standardized protocols
   - API documentation for developer services
   - Technical specifications from standards bodies
     (OASIS, AUTOSAR, 3GPP, ODVA, etc.)

The documentation now provides 100% coverage of all 466 protocols
defined in ndpi_protocol_ids.h, with 98% having reference links.
Each entry now includes better technical context to help users
understand protocol purpose and usage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-14 15:52:28 +02:00
Luca Deri
9daac6d20d Added incldue files in packages 2025-10-14 15:06:02 +02:00
Ivan Nardi
9aeb80f902
Fix library installation path duplication (issue #1971) (#2986) 
* Fix library installation path duplication (issue #1971)

Fix a bug where libraries were being installed to incorrect
paths due to improper concatenation of PREFIX and libdir variables.

The libdir variable already contains the full installation path
(e.g., /usr/lib or /opt/custom/lib) : concatenating $(PREFIX)$(libdir)
caused path duplication.

Add proper prefix and exec_prefix variable definitions for autoconf
compatibility and maintain backwards compatibility by keeping PREFIX
as an alias.

Full credits to @utoni and @OldManYellsAtCloud

Fixes: #1971
Related: #1823
 2025-10-14 08:54:46 +02:00
Ivan Nardi
a5fdcb3c62
configure: improve roaring version detection (#2989) 
* configure: improve roaring version detection

Replace GCC version heuristic with proper C11 atomics feature detection.

Previously, the configure script used GCC version >= 7 as a proxy to
determine whether to use roaring v4 or fall back to the old version.
This approach had several limitations:
- Only worked reliably with GCC
- Didn't verify actual C11 support
- Could fail with other compilers (Clang, ICC, etc.)

Roaring v4 requires C11 atomics (stdatomic.h, _Atomic, etc.) as per
roaring.h:547. This commit implements a proper feature test using
AC_COMPILE_IFELSE that checks:
- C11 standard support (__STDC_VERSION__ >= 201112L)
- C11 atomics not disabled (__STDC_NO_ATOMICS__)
- Working <stdatomic.h> header
- Functional atomic operations (atomic_fetch_add_explicit, etc.)

Benefits:
- Works correctly with any C11-compliant compiler
- Tests actual requirements instead of compiler version
- More robust across different platforms

The --enable-old-croaring flag continues to work as before, allowing
users to force the old roaring version when needed.

On CI, we can now autodetect roaring version even with mingw compiler.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Ivan Nardi <nardi.ivan@gmail.com>

* Fix compilation with mingw compiler

Fix the warning:
```
 third_party/src/roaring.c: In function ‘roaring64_bitmap_remove_bulk’:
third_party/src/roaring.c:24508:61: error: ‘leaf’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
24508 | static inline uint64_t get_index(leaf_t leaf) { return leaf >> 8; }
      |                                                        ~~~~~^~~~
third_party/src/roaring.c:25166:20: note: ‘leaf’ was declared here
25166 |             leaf_t leaf;
      |                    ^~~~
cc1: all warnings being treated as errors
```

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2025-10-13 21:50:14 +02:00
Ivan Nardi
b99d942d89
fuzz: simplify Makefile (#2991) 
Add proper `clean` target
 2025-10-13 21:49:09 +02:00
Ivan Nardi
42776e19f6 doc: fix table format 
Close #2990
 2025-10-13 17:01:43 +02:00
Ivan Nardi
a9cc75d634
ndpiReader: fix memory accounting (#2988) 
We don't know how much memory we are currently using: we only know the
amount of total memory allocated. Use proper label to report this
information in a correct way
 2025-10-12 18:12:01 +02:00
Ivan Nardi
730d8ee584
configure: avoid compiling rrdtool if --with-only-libndpi is set (#2987) 
Update .gitignore
 2025-10-12 18:11:46 +02:00
Ivan Nardi
dc5214b764
We are not interested into entropy for encrypted flows (#2983) 
Update `only_classification.conf` configuration
 2025-10-09 14:35:26 +02:00
Alfredo Cardigliano
7b8b1eb7f7 Check ndpi_init_deserializer_buf params 2025-10-08 15:04:36 +02:00
Toni
c67d8b63fa
Improved Telnet detection. Fixes #2936 (#2982) 
* get rid of telnet stage's

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2025-10-07 16:01:17 +02:00
Ivan Nardi
a07d55005d
fuzz: try to improve fuzzing coverage (#2981) 2025-10-06 20:44:31 +02:00
Ivan Nardi
d06291d125
Add detection of ESPN traffic (#2980) 2025-10-05 21:03:53 +02:00