Avoid collisions between user-ids and internal-ids protocols in the
`example/protos.txt` file.
Add a new value for the classification confidence:
`NDPI_CONFIDENCE_CUSTOM_RULE`
With `./example/ndpiReader -p example/protos.txt -H` we now see also the
custom protocols and their internal/external ids:
```
nDPI supported protocols:
Id Userd-id Protocol Layer_4 Nw_Proto Breed Category
0 0 Unknown TCP X Unrated Unspecified
...
387 387 Mumble UDP X Fun VoIP
388 388 iSCSI TCP Acceptable Unspecified
389 389 Kibana TCP Acceptable Unspecified
390 390 TestProto TCP Acceptable Unspecified
391 391 HomeRouter TCP Acceptable Unspecified
392 392 CustomProtocol TCP Acceptable Unspecified
393 393 AmazonPrime TCP Acceptable Unspecified
394 394 CustomProtocolA TCP Acceptable Unspecified
395 395 CustomProtocolB TCP Acceptable Unspecified
396 800 CustomProtocolC TCP Acceptable Unspecified
397 1024 CustomProtocolD TCP Acceptable Unspecified
398 2048 CustomProtocolE TCP Acceptable Unspecified
399 2049 CustomProtocolF TCP Acceptable Unspecified
400 2050 CustomProtocolG TCP Acceptable Unspecified
401 65535 CustomProtocolH TCP Acceptable Unspecified
```
We likely need to take a better look in general at the iteration between
internal and external protocols ids...
This PR fixes the issue observed in
https://github.com/ntop/nDPI/pull/2274#discussion_r1460674874 and in
https://github.com/ntop/nDPI/pull/2275.
Some changes in the parameters names.
Add a fuzzer to fuzz the configuration file format.
Add the infrastructure to configuratin callbacks.
Add an helper to map LRU cache indexes to names.
This is the first step into providing (more) configuration options in nDPI.
The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.
You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format
This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.
The complete list of configuration options is provided in
`doc/configuration_parameters.md`.
As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
Allow to disable this check.
I don't know how much sense these limits have in the application
(especially with those default values...) since we have always had a
hard limit on the library itself (`max_packets_to_process` set to 32).
The only value might be that they provide different limits for TCP and
UDP traffic.
Keep them for the time being...
- ndpi_load_domain_suffixes()
- ndpi_get_host_domain_suffix()
whose goal is to find the domain name of a hostname. Example:
www.bbc.co.uk -> co.uk
mail.apple.com -> com
Remove some unreached/duplicated code.
Add error checking for `atoi()` calls.
About `isdigit()` and similar functions. The warning reported is:
```
Negative Character Value help
isdigit() is invoked here with an argument of signed type char, but only
has defined behavior for int arguments that are either representable
as unsigned char or equal to the value of macro EOF(-1).
Casting the argument to unsigned char will avoid the undefined behavior.
In a number of libc implementations, isdigit() is implemented using lookup
tables (arrays): passing in a negative value can result in a read underrun.
```
Switching to our macros fix that.
Add a check to `check_symbols.sh` to avoid using the original functions
from libc.
The main difference with the original function is that we allow to
specify the base.
Credit for the original idea and the first implementation to @0xA50C1A1
Try using latest gcc and clang versions.
We still care about RHEL7: since handling a RHEL7 runner on GitHub is
quite complex, let try to use a similar version of gcc, at least
Keep looking for RTP packets but remove the monitoring concept.
We will re-introduce a more general concept of "flow in monitoring
state" later.
The function was disabled by default.
Some configuration knobs will be provided when/if #2190 is merged.
* logging is instead redirected to `ndpi_debug_printf`
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
approach for handling Internet domain names.
For switching back to Aho-Corasick it is necessary to edit
ndpi-typedefs.h and uncomment the line
// #define USE_LEGACY_AHO_CORASICK
[1] With Aho-Corasick
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once): 37.34 KB
Flow Memory (per flow): 960 B
Actual Memory: 33.09 MB
Peak Memory: 33.09 MB
[2] With the new algorithm
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once): 37.31 KB
Flow Memory (per flow): 960 B
Actual Memory: 7.42 MB
Peak Memory: 7.42 MB
In essence from ~33 MB to ~7 MB
This new algorithm will enable larger lists to be loaded (e.g. top 1M domans
https://s3-us-west-1.amazonaws.com/umbrella-static/index.html)
In ./lists there are file names that are named as <category>_<string>.list
With -G ndpiReader can load all of them at startup
