Commit graph

176 commits

Author SHA1 Message Date
Adrian Pekar
68dfdb6120
Upgrade nDPI from 4.7.0 to 5.0 with Windows libpcap fix (#235)
* Upgrade nDPI from 4.7.0 to 5.0 with Windows build fix (#230)


- Upgrade nDPI submodule from 4.7.0 to 5.0
- Fix all 9 nDPI 5.0 API breaking changes:
  * Protocol structure: .app_protocol -> .proto.app_protocol
  * Protocol structure: .master_protocol -> .proto.master_protocol
  * ndpi_protocol2name() signature change
  * ndpi_detection_giveup() signature change (removed enable_guess/guessed params)
  * ndpi_extra_dissection_possible() replaced with state check
  * ndpi_init_detection_module() now expects NULL
  * Protocol bitmask removed (all protocols enabled by default)
  * TCP/UDP struct size checks removed
  * Fingerprint extraction: ja3_client -> ja4_client

- Maintain backward compatibility by keeping field names:
  * client_fingerprint: Now JA4 (TLS), HASSH client (SSH), DHCP fingerprint
  * server_fingerprint: Now JA3S (TLS), HASSH server (SSH)

- Update build system for nDPI 5.0:
  * Add --with-only-libndpi flag to build.sh
  * Make CFFI marker extraction optional in engine_build.py

- Update tests for nDPI 5.0:
  * Update fingerprint assertions (JA3 MD5 -> JA4 format)
  * Regenerate all 228 baseline test results
  * Remove invalid test file (memcached.cap)

- All 29 tests passing

* Fix memcpy buffer overreads and enable DNS subclassification

- Fix buffer overread in fingerprint memcpy calls by using source size
  instead of destination size:
  - HASSH client/server: 33 bytes (was reading 48)
  - JA4 client: 37 bytes (was reading 48)
  - JA3 server: 33 bytes (was reading 48)
  - DHCP: 48 bytes (consistent with others)

- Enable DNS subclassification (disabled by default in nDPI 5.0)
  to restore detection of DNS.Apple, DNS.Google, etc.

- Regenerate test baselines with fixes applied

* Replace outdated Steam test pcaps with nDPI 5.0 version

- Remove steam.pcap and steam_datagram_relay_ping.pcapng (obsolete)
- Add steam.pcapng from nDPI 5.0 test suite
- nDPI 5.0 reworked Steam detection (ntop/nDPI#2264)

* Fix Windows build: add missing configure step for nDPI 5.0

The Windows build script was missing the ./configure --with-only-libndpi
step before make, causing nDPI headers to not be properly installed.
This resulted in the CFFI preprocessing marker //CFFI.NDPI_PACKED_STRUCTURES
being absent from ndpi_cdefinitions_packed.h, crashing engine_build.py.

Aligns build_windows.sh with build.sh which already had this step.

* Fix Windows build: install libpcap-dev for nDPI 5.0 configure

nDPI 5.0 configure requires libpcap when --with-only-libndpi is used
on MinGW (see ntop/nDPI#3114). Install mingw-w64-x86_64-libpcap via
pacman to satisfy this dependency.
2026-02-15 20:30:16 +01:00
Zied Aouini
3cb5f6cac8
Revert "Upgrade nDPI from 4.7.0 to 5.0 with Windows build fix (#230)" (#234)
This reverts commit b1b2ee87e2.
2026-02-14 21:34:30 +01:00
Adrian Pekar
b1b2ee87e2
Upgrade nDPI from 4.7.0 to 5.0 with Windows build fix (#230)
* Upgrade nDPI from 4.7.0 to 5.0 with backward compatibility

- Upgrade nDPI submodule from 4.7.0 to 5.0
- Fix all 9 nDPI 5.0 API breaking changes:
  * Protocol structure: .app_protocol -> .proto.app_protocol
  * Protocol structure: .master_protocol -> .proto.master_protocol
  * ndpi_protocol2name() signature change
  * ndpi_detection_giveup() signature change (removed enable_guess/guessed params)
  * ndpi_extra_dissection_possible() replaced with state check
  * ndpi_init_detection_module() now expects NULL
  * Protocol bitmask removed (all protocols enabled by default)
  * TCP/UDP struct size checks removed
  * Fingerprint extraction: ja3_client -> ja4_client

- Maintain backward compatibility by keeping field names:
  * client_fingerprint: Now JA4 (TLS), HASSH client (SSH), DHCP fingerprint
  * server_fingerprint: Now JA3S (TLS), HASSH server (SSH)

- Update build system for nDPI 5.0:
  * Add --with-only-libndpi flag to build.sh
  * Make CFFI marker extraction optional in engine_build.py

- Update tests for nDPI 5.0:
  * Update fingerprint assertions (JA3 MD5 -> JA4 format)
  * Regenerate all 228 baseline test results
  * Remove invalid test file (memcached.cap)

- All 29 tests passing

* Fix memcpy buffer overreads and enable DNS subclassification

- Fix buffer overread in fingerprint memcpy calls by using source size
  instead of destination size:
  - HASSH client/server: 33 bytes (was reading 48)
  - JA4 client: 37 bytes (was reading 48)
  - JA3 server: 33 bytes (was reading 48)
  - DHCP: 48 bytes (consistent with others)

- Enable DNS subclassification (disabled by default in nDPI 5.0)
  to restore detection of DNS.Apple, DNS.Google, etc.

- Regenerate test baselines with fixes applied

* Replace outdated Steam test pcaps with nDPI 5.0 version

- Remove steam.pcap and steam_datagram_relay_ping.pcapng (obsolete)
- Add steam.pcapng from nDPI 5.0 test suite
- nDPI 5.0 reworked Steam detection (ntop/nDPI#2264)

* Fix Windows build: add missing configure step for nDPI 5.0

The Windows build script was missing the ./configure --with-only-libndpi
step before make, causing nDPI headers to not be properly installed.
This resulted in the CFFI preprocessing marker //CFFI.NDPI_PACKED_STRUCTURES
being absent from ndpi_cdefinitions_packed.h, crashing engine_build.py.

Aligns build_windows.sh with build.sh which already had this step.
2026-02-14 17:48:00 +01:00
Zied Aouini
8a13708698
Revert "Upgrade nDPI from 4.7.0 to 5.0 (#220)" (#228)
This reverts commit 22cd213550.
2026-02-13 22:16:13 +01:00
Adrian Pekar
22cd213550
Upgrade nDPI from 4.7.0 to 5.0 (#220)
* Upgrade nDPI from 4.7.0 to 5.0 with backward compatibility

- Upgrade nDPI submodule from 4.7.0 to 5.0
- Fix all 9 nDPI 5.0 API breaking changes:
  * Protocol structure: .app_protocol -> .proto.app_protocol
  * Protocol structure: .master_protocol -> .proto.master_protocol
  * ndpi_protocol2name() signature change
  * ndpi_detection_giveup() signature change (removed enable_guess/guessed params)
  * ndpi_extra_dissection_possible() replaced with state check
  * ndpi_init_detection_module() now expects NULL
  * Protocol bitmask removed (all protocols enabled by default)
  * TCP/UDP struct size checks removed
  * Fingerprint extraction: ja3_client -> ja4_client

- Maintain backward compatibility by keeping field names:
  * client_fingerprint: Now JA4 (TLS), HASSH client (SSH), DHCP fingerprint
  * server_fingerprint: Now JA3S (TLS), HASSH server (SSH)

- Update build system for nDPI 5.0:
  * Add --with-only-libndpi flag to build.sh
  * Make CFFI marker extraction optional in engine_build.py

- Update tests for nDPI 5.0:
  * Update fingerprint assertions (JA3 MD5 -> JA4 format)
  * Regenerate all 228 baseline test results
  * Remove invalid test file (memcached.cap)

- All 29 tests passing

* Fix memcpy buffer overreads and enable DNS subclassification

- Fix buffer overread in fingerprint memcpy calls by using source size
  instead of destination size:
  - HASSH client/server: 33 bytes (was reading 48)
  - JA4 client: 37 bytes (was reading 48)
  - JA3 server: 33 bytes (was reading 48)
  - DHCP: 48 bytes (consistent with others)

- Enable DNS subclassification (disabled by default in nDPI 5.0)
  to restore detection of DNS.Apple, DNS.Google, etc.

- Regenerate test baselines with fixes applied

* Replace outdated Steam test pcaps with nDPI 5.0 version

- Remove steam.pcap and steam_datagram_relay_ping.pcapng (obsolete)
- Add steam.pcapng from nDPI 5.0 test suite
- nDPI 5.0 reworked Steam detection (ntop/nDPI#2264)
2026-02-13 22:03:42 +01:00
Zied Aouini
bb1831584d
fix missing import 2025-11-16 17:24:14 +01:00
Zied Aouini
5481132981
black formatting 2024-05-10 18:46:03 +02:00
Adrian Pekar
6c520bcdeb Extended splt_analysis parameter to allow up to 65535 values 2023-06-28 10:46:47 +02:00
Evan.Lai
777945a60a fix: socket_buffer_size exponent operator 2023-03-02 21:13:10 +08:00
Zied Aouini
50a41d708a Minor fixes. 2023-03-01 13:58:46 +01:00
Zied Aouini
d132d72ede Minor documentation improvements. 2022-10-29 23:28:43 +02:00
Zied Aouini
2f5e1a882b Drop browser extension experimental implementation. 2022-10-28 00:24:21 +02:00
Zied Aouini
57f6afd321 Update nDPI. 2022-10-25 18:12:50 +02:00
Zied Aouini
12136c5816 Implement max_nflows parameter. 2022-10-25 17:55:14 +02:00
Zied Aouini
2e8d685640 Update nDPI. 2022-09-27 17:06:48 +02:00
Zied Aouini
fc94e134d9 Fix test cases. 2022-07-28 13:57:38 +02:00
Zied Aouini
6c5eb7b584 Add debug trace. 2022-07-28 13:09:04 +02:00
Zied Aouini
bd1ec912fb Fix consecutive pcap files reading (https://github.com/nfstream/nfstream/pull/131) 2022-07-28 12:59:46 +02:00
Zied Aouini
cf8145bd36 Fix consecutive pcap files reading (https://github.com/nfstream/nfstream/pull/131) 2022-07-28 12:50:21 +02:00
Zied Aouini
db7518823e Sync nDPI. 2022-07-22 18:31:58 +02:00
aouinizied
051441e81b Minor fixes. 2022-04-27 12:10:47 +02:00
aouinizied
d4753df318 Fix Windows CI. 2022-04-16 05:55:07 +02:00
aouinizied
3952d2e2e0 Fix Windows CI. 2022-04-16 05:19:47 +02:00
aouinizied
2ed89ad932 Fix QUIC detection on Windows. 2022-04-15 17:44:02 +02:00
aouinizied
863f6bb23a CI fixes. 2022-04-14 12:19:34 +02:00
aouinizied
73bc0869d4 CI fixes. 2022-04-13 22:36:07 +02:00
aouinizied
0f765160ad Migrate tests to pytest. 2022-04-13 19:29:00 +02:00
aouinizied
67e6b017e5 Migrate tests to pytest. 2022-04-13 19:07:20 +02:00
aouinizied
a79357bd68 Improve build for Windows. 2022-04-07 01:50:28 +02:00
aouinizied
78a499f11e FiX arm64 CI. 2022-03-08 21:13:19 +01:00
aouinizied
ff51419f6e Drop application_is_guessed flow feature. 2022-03-08 20:08:06 +01:00
aouinizied
801fe26e5d Fix arm64 CI. 2022-03-08 19:46:04 +01:00
aouinizied
7495d94428 Fix windows CI. 2022-03-07 18:21:50 +01:00
aouinizied
7dc6fa8b31 Add test traces. 2022-03-07 16:19:30 +01:00
aouinizied
fe229e9130 Add test debug traces. 2022-03-07 15:52:15 +01:00
aouinizied
b07918f581 nDPI sync. 2022-03-07 15:20:12 +01:00
aouinizied
7fe6a644d4 nDPI sync. 2022-03-07 14:52:01 +01:00
Zied Aouini
037ab5ded5 Update Copyright. 2022-01-07 15:52:06 +01:00
Zied Aouini
50a2fe172f Minor fixes. 2021-12-09 16:42:09 +01:00
Zied Aouini
f65c6bce43 Introduce Windows Support. 2021-12-03 17:46:31 +01:00
Zied Aouini
6a1fd3307a Improve flow key computation function. 2021-12-03 17:08:07 +01:00
Zied Aouini
fedffc0033 [CI] Switch back to macos-10.15. 2021-12-03 15:53:50 +01:00
Zied Aouini
070115c4fd [WIP] Windows support. 2021-11-30 11:36:09 +01:00
Zied Aouini
e2a90bafc3 [WIP] Windows support. 2021-11-29 16:21:56 +01:00
Zied Aouini
dba1b1f9e4 [WIP] Windows support. 2021-11-29 14:30:39 +01:00
aouinizied
4984cc9352 [WIP] Windows support. 2021-11-29 10:23:07 +01:00
Zied Aouini
0ee3ffdf1a Introduce system visibility feature. 2021-11-04 18:59:07 +01:00
Zied Aouini
b6cdca351a Dissector sync (https://github.com/nfstream/nfstream/issues/77). 2021-10-07 10:44:00 +02:00
aouinizied
7d7e4deb56 Sync nDPI (ipv6 + reduce memory). 2021-09-21 15:40:01 +02:00
aouinizied
f0e080562a Fix truncated test files. 2021-09-01 18:54:01 +02:00