Commit graph

21 commits

Author SHA1 Message Date
Ivan Nardi
db9f6ec1b4
Add basic profiling of memory allocations on data-path (#1789)
The goal is to have an idea of the memory allocation sizes performed in
the **library data-path**, i.e. excluding init/deinit phases and all
the allocations made by the application itself.
In other words, how much memory is needed per-flow, by nDPI, other than
`struct ndpi_flow_struct`?

It works only on single-thread configurations.

It is not enabled by default (in the unit tests) since different
canfiguration options (example: `--enable-pcre`) lead to diffferent
results.

See: #1781
2022-10-28 20:41:37 +02:00
Nardi Ivan
cca585053e Fix compilation and sync utests results 2022-10-04 22:17:05 +02:00
Toni
b3e722e5a8
Improved nDPI JSON serialization. (#1689)
* fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors
 * introduced portable version of gmtime_r aka ndpi_gmtime_r
 * do as most as possible of the serialization work in ndpi_utils.c
 * use flow2json in ndpiReader

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 17:54:44 +02:00
Ivan Nardi
1d492c9ed8
Fix some debug messages (#1583)
Increase max number of flows handled during fuzzing
2022-06-05 14:45:21 +02:00
Toni
87f93ea4fd
Replaced ndpiReader's libjson-c support with libnDPI's internal serialization interface. (#1535)
* Fixes #1528
 * Serialization Interface should also fuzzed
 * libjson-c may only be used in the unit test to verify the internal serialization interface
 * Serialization Interface supports tlv(broken), csv and json
 * Unit test does work again and requires libjson-c

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:26:09 +02:00
Ivan Nardi
fbb9700086
fuzz: purge old sessions (#1451)
At every fuzz iteration (i.e for every trace file):
* keep the same ndpi context (`ndpi_init_detection_module` is very
slow);
* reset the flow table, otherwise it grows indefinitely.

This change should fix the "out-of-memory" errors reported by oss-fuzz.
2022-02-21 20:32:50 +01:00
Ivan Nardi
75a53ad064
fuzz: make fuzz_ndpi_reader faster (#1446)
Initialize ndpi_workflow_init context only once.

On a quite old notebook, before:
```
$ ./fuzz/fuzz_ndpi_reader -max_total_time="${MAX_TOTAL_TIME:-360}" -print_pcs=1 -workers="${FUZZY_WORKERS:-0}" -jobs="${FUZZY_JOBS:-0}" ./tests/pcap/
[...]
Done 3256 runs in 361 second(s)
```
after:
```
Done 5032771 runs in 361 second(s)   <----------- ~1400X
```
oss-fuzz will be happy!
2022-02-20 10:19:22 +01:00
Ivan Nardi
b080a1c136
Fix two use-of-uninitialized-value errors (#1398)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40269
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=41432

Fix fuzz compilation (follow-up of f5545a80)
2022-01-12 20:24:57 +01:00
Ivan Nardi
8fdffbf3a1
Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Vitaly Lavrov
2fcf641e87
Fixed uninitialized memory use. (#1198)
The pointer "header" must be initialized before first call pcap_next_ex().
2021-06-07 10:08:39 +02:00
Toni
8c28613eb2
Check datalink during fuzzing to prevent console / logfile spam. See #1175 for more information. (#1177)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-09 15:09:43 +02:00
Luca Deri
87ad2b58dc Compilation fix 2021-04-26 10:50:19 +02:00
Luca Deri
b757cf606b Fixed mispelled API call 2021-02-22 23:59:08 +01:00
Hosein Ghahremanzadeh
bb33d57971
Fix a memory leak on fuzzing code (#1036)
After allocation of pcap_buffer it is necessary to free it
2020-10-21 15:05:11 +02:00
Luca Deri
ea10b8e757 Added memory checks 2020-08-02 13:00:31 +02:00
Toni Uhlig
20fed83e0f
Removed csv_fp as external symbol. Instead passing csv_fp through as argument.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-08 23:21:35 +02:00
Toni Uhlig
c17a21359b
Fixed CodeInspector issues.
* Added compiler search list for AC_PROG_CC, AM_PROG_CC_C_O, AC_PROG_CXX, AC_PROG_CC_STDC
   for Mac OS X only
   The list rendered the AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer]) useless as it did use
   clang for AX_CHECK_COMPILE_FLAG but gcc during the compile process. Seems broken somehow.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-05 20:41:02 +02:00
Toni Uhlig
4a6c525db8
Improved fuzz_ndpi_reader which supports now SMP/MT w/o race-coniditions.
./tests/do.sh can supports SMP/MT via environment variables.
Removed -fno-sanitize=shift as well, was fixed by 317d3ffd.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-29 19:07:59 +02:00
Luca Deri
d86d632fe6 Added memory boundary check in HTTP dissector 2020-01-24 21:04:27 +01:00
Philippe Antoine
4f230943cf Debug compilation flag for fuzz targets
And adding include file to fuzz_ndpi_reader
2020-01-24 09:10:52 +01:00
Philippe Antoine
10738a0ca3 New fuzz target : fuzz_ndpi_reader 2020-01-14 08:20:15 +01:00