Removed bigram_automata, impossible_bigram_automata, trigram_automata.
The ahocorasick structure is replaced with a bitmap.
The bitmap size for ndpi_en_bigram is 176 bytes.
The bitmap size for ndpi_en_trigram is 2201 bytes.
On the test machine, the test execution time was reduced from 27.3 seconds to 24.7 (9%).
The common actions required to call the ac_automata_search() function
have been moved to the ndpi_match_string_common function. This made it
possible to simplify the ndpi_match_string, ndpi_match_string_protocol_id,
ndpi_match_string_value, ndpi_match_custom_category, ndpi_match_string_subprotocol,
ndpi_match_bigram, ndpi_match_trigram functions.
Using u_int16_t type for protocol identifiers when working with the
ahocorasick library (changes src/include/ndpi_api.h.in and src/include/ndpi_typedefs.h).
Reworked "finalization" of all AC_AUTOMATA_t structures.
Changing the order of fields in the ndpi_call_function_struct structure
reduces the size of the ndpi_detection_module_struct structure by 10 kB (for x86_64).
The new version is about 25% faster with -O2 and 45% faster with -O3.
No recursion is used (smaller stack size required).
Uses less memory (by valgrind info)
bigram:
- original 1796 allocs, 247864 bytes allocated
- new 1232 allocs, 158880 bytes allocated
host_match:
- original 18038 allocs, 3004576 bytes allocated
- new 6861 allocs, 396624 bytes allocated
The function ac_automata_search() is thread safe.
Optional case-insensitive comparison.
Matching at the beginning and at the end of the string is supported.
One code file and one header file.
```
In file included from protocols/fasttrack.c:29:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
1504 | const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
| ^~~~~
In file included from protocols/amazon_video.c:28:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
1504 | const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
| ^~~~~
...
ndpi_utils.c: In function ‘ndpi_risk2severity’:
ndpi_utils.c:1834:1: warning: control reaches end of non-void function [-Wreturn-type]
1834 | }
| ^
```
```
==69562==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6100009000fb at pc 0x7f41882003a7 bp 0x7f4183cfbfc0 sp 0x7f4183cfb768
READ of size 32 at 0x6100009000fb thread T1
#0 0x7f41882003a6 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
#1 0x560b2d7462a1 in processClientServerHello protocols/tls.c:1647
#2 0x560b2d73be6a in processTLSBlock protocols/tls.c:712
#3 0x560b2d73e61f in ndpi_search_tls_udp protocols/tls.c:968
```
Error detected with valgrind.
==13127== Conditional jump or move depends on uninitialised value(s)
==13127== at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127== by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127== by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127== by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127== by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)
See also 8c3674e9
