* Upgrade nDPI from 4.7.0 to 5.0 with Windows build fix (#230)
- Upgrade nDPI submodule from 4.7.0 to 5.0
- Fix all 9 nDPI 5.0 API breaking changes:
* Protocol structure: .app_protocol -> .proto.app_protocol
* Protocol structure: .master_protocol -> .proto.master_protocol
* ndpi_protocol2name() signature change
* ndpi_detection_giveup() signature change (removed enable_guess/guessed params)
* ndpi_extra_dissection_possible() replaced with state check
* ndpi_init_detection_module() now expects NULL
* Protocol bitmask removed (all protocols enabled by default)
* TCP/UDP struct size checks removed
* Fingerprint extraction: ja3_client -> ja4_client
- Maintain backward compatibility by keeping field names:
* client_fingerprint: Now JA4 (TLS), HASSH client (SSH), DHCP fingerprint
* server_fingerprint: Now JA3S (TLS), HASSH server (SSH)
- Update build system for nDPI 5.0:
* Add --with-only-libndpi flag to build.sh
* Make CFFI marker extraction optional in engine_build.py
- Update tests for nDPI 5.0:
* Update fingerprint assertions (JA3 MD5 -> JA4 format)
* Regenerate all 228 baseline test results
* Remove invalid test file (memcached.cap)
- All 29 tests passing
* Fix memcpy buffer overreads and enable DNS subclassification
- Fix buffer overread in fingerprint memcpy calls by using source size
instead of destination size:
- HASSH client/server: 33 bytes (was reading 48)
- JA4 client: 37 bytes (was reading 48)
- JA3 server: 33 bytes (was reading 48)
- DHCP: 48 bytes (consistent with others)
- Enable DNS subclassification (disabled by default in nDPI 5.0)
to restore detection of DNS.Apple, DNS.Google, etc.
- Regenerate test baselines with fixes applied
* Replace outdated Steam test pcaps with nDPI 5.0 version
- Remove steam.pcap and steam_datagram_relay_ping.pcapng (obsolete)
- Add steam.pcapng from nDPI 5.0 test suite
- nDPI 5.0 reworked Steam detection (ntop/nDPI#2264)
* Fix Windows build: add missing configure step for nDPI 5.0
The Windows build script was missing the ./configure --with-only-libndpi
step before make, causing nDPI headers to not be properly installed.
This resulted in the CFFI preprocessing marker //CFFI.NDPI_PACKED_STRUCTURES
being absent from ndpi_cdefinitions_packed.h, crashing engine_build.py.
Aligns build_windows.sh with build.sh which already had this step.
* Fix Windows build: install libpcap-dev for nDPI 5.0 configure
nDPI 5.0 configure requires libpcap when --with-only-libndpi is used
on MinGW (see ntop/nDPI#3114). Install mingw-w64-x86_64-libpcap via
pacman to satisfy this dependency.
* Upgrade nDPI from 4.7.0 to 5.0 with backward compatibility
- Upgrade nDPI submodule from 4.7.0 to 5.0
- Fix all 9 nDPI 5.0 API breaking changes:
* Protocol structure: .app_protocol -> .proto.app_protocol
* Protocol structure: .master_protocol -> .proto.master_protocol
* ndpi_protocol2name() signature change
* ndpi_detection_giveup() signature change (removed enable_guess/guessed params)
* ndpi_extra_dissection_possible() replaced with state check
* ndpi_init_detection_module() now expects NULL
* Protocol bitmask removed (all protocols enabled by default)
* TCP/UDP struct size checks removed
* Fingerprint extraction: ja3_client -> ja4_client
- Maintain backward compatibility by keeping field names:
* client_fingerprint: Now JA4 (TLS), HASSH client (SSH), DHCP fingerprint
* server_fingerprint: Now JA3S (TLS), HASSH server (SSH)
- Update build system for nDPI 5.0:
* Add --with-only-libndpi flag to build.sh
* Make CFFI marker extraction optional in engine_build.py
- Update tests for nDPI 5.0:
* Update fingerprint assertions (JA3 MD5 -> JA4 format)
* Regenerate all 228 baseline test results
* Remove invalid test file (memcached.cap)
- All 29 tests passing
* Fix memcpy buffer overreads and enable DNS subclassification
- Fix buffer overread in fingerprint memcpy calls by using source size
instead of destination size:
- HASSH client/server: 33 bytes (was reading 48)
- JA4 client: 37 bytes (was reading 48)
- JA3 server: 33 bytes (was reading 48)
- DHCP: 48 bytes (consistent with others)
- Enable DNS subclassification (disabled by default in nDPI 5.0)
to restore detection of DNS.Apple, DNS.Google, etc.
- Regenerate test baselines with fixes applied
* Replace outdated Steam test pcaps with nDPI 5.0 version
- Remove steam.pcap and steam_datagram_relay_ping.pcapng (obsolete)
- Add steam.pcapng from nDPI 5.0 test suite
- nDPI 5.0 reworked Steam detection (ntop/nDPI#2264)
* Fix Windows build: add missing configure step for nDPI 5.0
The Windows build script was missing the ./configure --with-only-libndpi
step before make, causing nDPI headers to not be properly installed.
This resulted in the CFFI preprocessing marker //CFFI.NDPI_PACKED_STRUCTURES
being absent from ndpi_cdefinitions_packed.h, crashing engine_build.py.
Aligns build_windows.sh with build.sh which already had this step.
pandas 3.0.0 (released Jan 21, 2026) segfaults on import when running
under PyPy. Cap pandas to <3 for PyPy installations while allowing
any version for CPython using environment markers.
- Replace manual string concatenation with csv.writer() for RFC 4180 compliance
- Update open_file() to use text mode with proper encoding and newline handling
- Remove manual quoting from csv_converter() as csv.writer() handles this properly
This addresses potential data corruption with special characters (commas, quotes,
newlines) in network flow data by using Python's standard CSV module instead
of manual string formatting.
Make IPv6 ip_size calculation consistent with IPv4 for unified accounting modes.
IPv6 ip6_un1_plen field excludes the IPv6 header (per RFC), while IPv4 tot_len
includes the IPv4 header. This caused inconsistent accounting behavior between
IPv4 and IPv6 flows.
Updated packet_get_ipv6_info() to include IPv6 header size in total length
calculation for consistent accounting semantics:
- Before: iph.tot_len = iph6->ip6_hdr.ip6_un1_plen (payload only)
- After: iph.tot_len = htons(sizeof(struct ndpi_ipv6hdr) + ntohs(iph6->ip6_hdr.ip6_un1_plen))
This ensures unified accounting behavior across IPv4 and IPv6:
- Mode 0: Raw packet (all headers + data)
- Mode 1: IP packet (IP header + transport + data)
- Mode 2: Transport segment (transport header + data)
- Mode 3: Application payload (data only)
Previously IPv6 Mode 1 and Mode 2 were identical, now they differ by 40 bytes.
