* fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors
* introduced portable version of gmtime_r aka ndpi_gmtime_r
* do as most as possible of the serialization work in ndpi_utils.c
* use flow2json in ndpiReader
Signed-off-by: lns <matzeton@googlemail.com>
The big change in TLS code is to allow "master" protocols other than
TLS/DTLS, like SMTPS, POPS and IMAPS.
This change will allow, in a future, a proper and complete TLS dissection
for all these protocols with "STARTTLS"-like messages.
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.
Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.
Add an option to `ndpiReader` to dump them; disabled by default to avoid
too much fuss with the unit tests.
Treat HTTP-Proxy and HTTP-Connect flows like the HTTP ones:
print/serialize all the attributes and allow parsing of replies.
The line about "1kxun" has been removed to avoid regressions in 1KXUN
classification in `tests/pcap/1kxun.pcap`. I haven't fully understod
what was happening but the comment at the beginning of `static
ndpi_category_match category_match[]` says that we can't have overlaps
between `host_match` and `category_match` lists and that is no longer true
since 938e89ca.
Bottom line: removing this line seems the right thing to do, anyway.
In a lot of places in ndPI we use *packet* source/dest info
(address/port/direction) when we are interested in *flow* client/server
info, instead.
Add basic logic to autodetect this kind of information.
nDPI doesn't perform any "flow management" itself but this task is
delegated to the external application. It is then likely that the
application might provide more reliable hints about flow
client/server direction and about the TCP handshake presence: in that case,
these information might be (optionally) passed to the library, disabling
the internal "autodetect" logic.
These new fields have been used in some LRU caches and in the "guessing"
algorithm.
It is quite likely that some other code needs to be updated.
* CI fixes
* some build systems do not like that (e.g. OpenWrt)
* fixed some rrdtool related build warnings/errors
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* fixed possible memory leak caused by an invalid call to `node_proto_guess_walker()` during serialization
* execute serialization code while running regression tests
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add some statistics to ndpiReader
The purpose of this version of ndpiReader is too adding some other statistics printed by ndpiReader. In this simple version the domain names(in the flows) that are collected are:
flow-> ssh_tls.server_info
flow-> host_server_name
and are placed in a UT_hash_table, ordering them by number of occurrences.
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update example/ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
* Update ndpiReader.c
* Update ndpiReader.c
* Update ndpiReader.c
* Update ndpiReader.c
Co-authored-by: Toni <matzeton@googlemail.com>
We can access `flow->protos` union only after checking the protocol.
Checking `flow->detected_protocol.master_protocol` is redundant because
we already check it in `is_ndpi_proto`
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29739==ERROR: AddressSanitizer: SEGV on unknown address 0x000000353820 (pc 0x7f9b64dd2717 bp 0x7fff161a52f0 sp 0x7fff161a4aa8 T0)
==29739==The signal is caused by a READ memory access.
#0 0x7f9b64dd2717 /build/glibc-SzIz7B/glibc-2.31/string/../sysdeps/x86_64/multiarch/strlen-avx2.S:96
#1 0x555c65e597d8 in __interceptor_strlen (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x6407d8) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)
#2 0x555c65fd85fa in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:269:13
#3 0x555c65f3e8c6 in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1188:36
#4 0x555c65f52cab in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1567:2
#5 0x555c65f4b632 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110:10
#6 0x555c65f04d29 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7
#7 0x555c65f054bb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
#8 0x7f9b64c6e082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
#9 0x555c65e4253d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x62953d) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)
```
Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=48020
This fixes some build/test issues resulting when using tarballs.
* nDPI uses autotools (especially autoconf) in a wrong way, see #1163
Signed-off-by: lns <matzeton@googlemail.com>
* It is outdated (from 2018) and will most likely not work anymore
due to changed protocol files and outdated API usage in the example integration
* Removal is the only option besides fixing those issues and
provide a CI integration for future automatic checks
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixes#1528
* Serialization Interface should also fuzzed
* libjson-c may only be used in the unit test to verify the internal serialization interface
* Serialization Interface supports tlv(broken), csv and json
* Unit test does work again and requires libjson-c
Signed-off-by: lns <matzeton@googlemail.com>
* Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
- partial revert of:
commit 51cfdfb0d8
Author: Luca Deri <deri@ntop.org>
Date: Sun Nov 17 17:51:45 2019 +0100
Removed unused JSON-C code
- Json option is changed from 'j' to 'k' as it's used in the new codebase.
- use HAVE_LIBJSON_C instead of HAVE_JSON_C.
- tabs vs spaces clean ups.
Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
Conflicts:
example/ndpiReader.c
