Commit graph

159 commits

Author SHA1 Message Date
Toni
6ad0d6666c
Implemented function to retrieve flow information. #1253 (#1254)
* fixed [h]euristic typo

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-23 10:37:20 +02:00
Luca Deri
b01b60a2b5 Implementation of flow risk eception (work in progress) 2021-07-22 01:35:57 +02:00
Ivan Nardi
cccf794265
ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri
43a8576efb Reworked human readeable string search in flows
Removed fragment manager code
2021-05-17 20:55:06 +02:00
Toni
8c28613eb2
Check datalink during fuzzing to prevent console / logfile spam. See #1175 for more information. (#1177)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-09 15:09:43 +02:00
Luca
ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Toni
da3e6bd61b
Check for common ALPNs and set a flow risk if not known. (#1175)
* Increased risk bitmask to 64bit (instead of 32bit).
 * Removed annoying "Unknown datalink" error message for fuzzers.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-27 07:22:04 +02:00
Luca Deri
4a09707e48 Added flow risk to wireshark dissection 2021-04-26 10:17:29 +02:00
Ivan Nardi
9ca62ed7ac
Fix detunneling of GTP-U traffic (#1168)
Fuzzing #1161 exposed some (completely unrelated) issues on GTP-U
detunneling code.
(see https://github.com/ntop/nDPI/actions/runs/719882047)
2021-04-18 21:37:51 +02:00
Luca Deri
fcbc16da00 Fixed invalid guess stats 2021-03-30 17:49:48 +02:00
Ivan Nardi
a6029d250d
ndpiReader: print an error msg if we found an unsupported datalink type (#1157) 2021-03-23 11:47:29 +01:00
Luca Deri
565a7bfce3 Reworked extendal dependency across testing tools 2021-03-14 20:48:21 +01:00
Luca Deri
f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Ivan Nardi
c50a8d4808
Add support for Snapchat voip calls (#1147)
* Add support for Snapchat voip calls

Snapchat multiplexes some of its audio/video real time traffic with QUIC
sessions. The peculiarity of these sessions is that they are Q046 and
don't have any SNI.

* Fix tests with libgcrypt disabled
2021-03-06 05:48:36 +01:00
Toni
1e12c90c66
Fixed memory leaks caused by conditional free'ing for some TLS connec… (#1132)
* Fixed memory leaks caused by conditional free'ing for some TLS connections.

 * Members of tls_quic struct should also free'd if the detected master protocol is IMAPS / POPS / SMTPS / etc.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent reader_util.c from exit()'ing if maximum flow count reached.
This confuses the fuzzer.

 * Improved fuzz/Makefile.am to use LDADD for ../example/libndpiReader.a instead of LDFLAGS.
   That way, fuzz_ndpi_reader re-links to ../example/libndpiReader.a if something changed there.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-10 15:24:11 +01:00
Luca Deri
60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri
4b181be58e Improved debug message 2021-02-03 11:49:14 +01:00
Luca Deri
a31bd5ac3c Cleaned up tls/quic datatypes 2021-01-21 19:17:33 +01:00
Luca Deri
15295ef4c5 Reworked TLS fingerprint calcolation
Modified TLS memory free
2021-01-21 19:06:05 +01:00
Luca Deri
68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri
eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Luca Deri
b7376cc690 Restored QUIC stats 2020-12-30 12:12:33 +01:00
Luca Deri
9c1827a77b Fixed output when tLS (nad not QUIC) is used 2020-12-28 09:19:39 +01:00
Luca Deri
a89642ad04 Fixes bug introduced by https://github.com/ntop/nDPI/pull/1085 2020-12-12 12:32:44 +01:00
Zied Aouini
5bd5461f96
Fix minimum packet length condition (#1087) 2020-12-12 11:12:59 +01:00
Toni
74a77e7b3d
Added --ignore-vlanid / -I to exclude VLAN ids for flow hash calculation. #1073 (#1085)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:51 +01:00
Toni
af02ffb60f
Support raw IPv4 / IPv6 pcap packet processing. (#1053)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:18:05 +01:00
Luca Deri
017e395ed1 Cosmetic changes 2020-11-03 16:46:30 +01:00
Luca Deri
48d640583a Moved global in reader_util.c 2020-10-27 08:40:00 +01:00
Luca Deri
948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri
9873972acb Various improvemement when using ndpi_pref_enable_tls_block_dissection:
application data TLS blocks are now ignored when exchanged before
- the end of certificate negotiation (up to TLS 1.2)
- change cipher
2020-10-24 19:22:56 +02:00
Ivan Nardi
6027a7c799
Fix parsing of DLT_PPP datalink type (#1042) 2020-10-21 22:27:42 +02:00
Zied Aouini
43c1f6a3fd
CAPWAP tunnel decoding fix (#1038)
* Fix CAPWAP processing.

* Update result.
2020-10-21 15:07:20 +02:00
aouinizied
d5d2a7e3f3 Fix CAPWAP handling. 2020-10-13 19:13:07 +02:00
Luca
05d93790e4 Added ndpi_quick_16_byte_hash
Warning fix
2020-10-05 08:26:24 +02:00
Luca Deri
32e4922c5a Warning fix 2020-10-03 17:14:40 +02:00
Luca Deri
d81bc1add6 Reworked MDNS dissector that is not based on the DNS dissector 2020-09-17 23:24:02 +02:00
Nardi Ivan
a1014e8895 http: create a common function to parse User Agent field
Prepare the code to handle UA information from flows other than HTTP
2020-09-08 10:34:05 +02:00
Adrian Zgorzałek
8f74d5733d OpenBSD: Introduce pkt_timeval to deal with (bpf_)_timeval
Some BSD APIs called in example/ return `struct bpf_timeval`, where nDPI
APIs expect `struct timeval`. These two structs, besides having
a different name, share the exact same set of fields.
2020-08-09 14:30:12 +01:00
Luca Deri
a828ac0191 Tiny changes for TLS block lenght dissection 2020-07-29 22:36:27 +02:00
Luca Deri
802254327a wrapper cleanup 2020-07-13 10:53:25 +02:00
Toni Uhlig
96c193cf79
Fixed heap overflow caused by missing lengthcheck in 802.11 LLC header parsing.
* triggered by fuzz traces from wireshark

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-11 13:10:03 +02:00
Toni Uhlig
20fed83e0f
Removed csv_fp as external symbol. Instead passing csv_fp through as argument.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-08 23:21:35 +02:00
Luca Deri
1c60c22893 Added ndpi_cluster_bins() for clustering bins and ancillary functions for bins manipulation 2020-07-07 15:10:51 +02:00
Luca Deri
db707e0829
Merge pull request #932 from IvanNardi/log
Log
2020-07-07 14:43:32 +02:00
Toni Uhlig
de25ba7d0b
Fixed heap overflow caused by missing lengthcheck in reader uutil.
* triggered by fuzz traces from wireshark

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-06 23:30:40 +02:00
Nardi Ivan
b24f5c4c0a Fix memory leak about purged/expired flows
Create an helper to avoid similar errors in the future
Fixes: 1a62f4c7
2020-06-28 12:05:12 +02:00
Nardi Ivan
ece5d3e199 Fix (harmless) memory leaks when DPDK is enabled 2020-06-28 12:05:12 +02:00
Nardi Ivan
56d87186f7 Fix compilation with --enable-debug-messages flag
NDPI_LOG* macros dereference ndpi_detection_module_struct object which is
private to ndpi library (via NDPI_LIB_COMPILATION define). So we can't use
them outside the library itself, i.e. in ndpiReader code
Therefore, in files in example/, convert all (rare) uses of NDPI_LOG* macros
to a new very simple macro, private to ndpiReader program. If necessary,
such macro may be improved.

According to a comment in ndpi_define.h, each dissector must define its own
NDPI_CURRENT_PROTO macro before including ndpi_api.h file
2020-06-26 12:04:02 +02:00
Luca Deri
1a62f4c799 Added ndpi_bin_XXX API
Added packet lenght distribution bins
2020-06-22 01:02:54 +02:00