* Add bitcoing protocol dissector.
* remove bitcoin protcol detection from mining.c
* add a new bitcoin deissector.
* add a new category: Cryptocurrency.
Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
* Remove useless checks and add missing windows and docs file.
Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
* update affected tests.
Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
* add a brief version.
Add notes on the difference between normal bitcoin protocol and the
mining protocol.
Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
* update enable_payload_stat test after dev rebasing.
Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
---------
Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
Return the "classification-by-ip" as protocol results only if no other
results are available.
In particular, never return something like
"protocol_by_port/protocol_by_ip" (i.e. `NTP/Apple`,
BitTorrent/GoogleCloud`, `Zoom/AWS`) because this kind of classification
is quite confusing, if not plainly wrong.
Notes:
* the information about "classification-by-ip" is always available, so
no information is lost with this change;
* in the unit tests, the previous classifications with confidence
`NDPI_CONFIDENCE_DPI_PARTIAL` were wrong, as noted in #1957
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".
Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
The goal if to correlate the right request-response pair, exporting
metadata from only one transaction (for example, the right url & return
state pair)
As a nice side effect, the code should be much cleaner, but that is a
matter of taste.
Two differences respect to the previous code:
* as it happens in the CI, if in the flow there are only one response
(before) and one request (after), only the metadata of the response are
saved/exported
* for performance reasons, we don't call `ndpi_parse_packet_line_info()`
anymore for ALL packets triggering the HTTP dissector, but only for the
packets that we already know belong to an HTTP flow. This is the reason
for the changes in RTSP/SOAP/... code
When we reconcile a TLS session to Teams, we need to keep TLs as master.
This way:
* we keep exporting all the TLS metadata
* we avoid some memory leaks (of these metadata themeselves)
For a lot of protocols, reduce the number of packets after which the
protocols dissector gives up.
The values are quite arbitary, tring to not impact on classification
This piece of code has multiple problems:
* nDPI is able to detect some TCP protocols even with mid-flows (i.e.
without the initial packets of the session); TLS is the most
significative example
* since e6b332aa4a it is perfectly valid
to not pass the TCP Handshake packets to nDPI
* in any case, we shouldn't call `ndpi_detection_giveup()`. That
function is usually called by the application and we end up calling it
twice in some cases.
The simple solution is to completely remove that code: process these
kinds of flows like everyone else.
Note that the application can always avoid to pass to nDPI any TCP flows
without the initial handshake; the flow managemnt is always up to the
application.
Looking at the CI results, some rare flows are now processed significantly
longer. As a follow-up we could look into that.
In some networks, there are some anomalous TCP flows where the smallest
ACK packets have some kind of zero padding.
It looks like the IP and TCP headers in those frames wrongly consider the
0x00 Ethernet padding bytes as part of the TCP payload.
While this kind of packets is perfectly valid per-se, in some conditions
they might be treated by the TCP reassembler logic as (partial) overlaps,
deceiving the classification engine.
Add an heuristic to detect these packets and to ignore them, allowing
correct detection/classification.
This heuristic is configurable. Default value:
* in the library, it is disabled
* in `ndpiReader` and in the fuzzers, it is enabled (to ease testing)
Credit to @vel21ripn for the initial patch.
Close#1946
Extend internal unit tests to handle multiple configurations.
As some examples, add tests about:
* disabling some protocols
* disabling Ookla aggressiveness
Every configurations data is stored in a dedicated directory under
`tests\cfgs`
