vrr/nDPI
2
0
Fork 0
mirror of https://github.com/vel21ripn/nDPI.git synced 2026-04-29 23:49:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Domain Classification Improvements (#2396)

Browse source 
* Added
size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize);
size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize);

used to compress short strings such as domain names. This code is based on
https://github.com/Ed-von-Schleck/shoco

* Major code rewrite for ndpi_hash and ndpi_domain_classify

* Improvements to make sure custom categories are loaded and enabled

* Fixed string encoding

* Extended SalesForce/Cloudflare domains list
This commit is contained in:
Luca Deri 2024-04-18 23:21:40 +02:00 • committed by GitHub
parent 108b8331d5
commit ad117bfaab
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
65 changed files with 1194 additions and 612 deletions
Download patch file Download diff file Expand all files Collapse all files

12
tests/cfgs/enable_payload_stat/result/1kxun.pcap.out
View file

@ -214,9 +214,9 @@ JA3 Host Stats:
128 UDP [fe80::9bd:81dd:2fdc:5750]:61548 -> [ff02::1:3]:5355 [flowId: 51][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/190 bytes -> 0 pkts/0 bytes][Goodput ratio: 35/0][0.41 sec][Hostname/SNI: caesar-thinkpad][PLAIN TEXT (caesar)][Plen Bins: 0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
129 UDP [fe80::9bd:81dd:2fdc:5750]:64568 -> [ff02::1:3]:5355 [flowId: 102][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/190 bytes -> 0 pkts/0 bytes][Goodput ratio: 35/0][0.41 sec][Hostname/SNI: caesar-thinkpad][PLAIN TEXT (caesar)][Plen Bins: 0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
130 UDP 192.168.5.45:137 -> 192.168.255.255:137 [flowId: 68][proto: 10/NetBIOS][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: System/18][2 pkts/184 bytes -> 0 pkts/0 bytes][Goodput ratio: 54/0][1.02 sec][Hostname/SNI: nasfile][PLAIN TEXT ( EOEBFDEGEJEMEFCACACACACACACACA)][Plen Bins: 0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
131 UDP [fe80::e98f:bae2:19f7:6b0f]:51451 -> [ff02::1:3]:5355 [flowId: 96][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/184 bytes -> 0 pkts/0 bytes][Goodput ratio: 32/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
132 UDP [fe80::e98f:bae2:19f7:6b0f]:54888 -> [ff02::1:3]:5355 [flowId: 32][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/184 bytes -> 0 pkts/0 bytes][Goodput ratio: 32/0][0.10 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
133 UDP [fe80::e98f:bae2:19f7:6b0f]:58779 -> [ff02::1:3]:5355 [flowId: 18][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/184 bytes -> 0 pkts/0 bytes][Goodput ratio: 32/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
131 UDP [fe80::e98f:bae2:19f7:6b0f]:51451 -> [ff02::1:3]:5355 [flowId: 96][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/184 bytes -> 0 pkts/0 bytes][Goodput ratio: 32/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Risk Info: Invalid chars detected in domain name][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
132 UDP [fe80::e98f:bae2:19f7:6b0f]:54888 -> [ff02::1:3]:5355 [flowId: 32][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/184 bytes -> 0 pkts/0 bytes][Goodput ratio: 32/0][0.10 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Risk Info: Invalid chars detected in domain name][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
133 UDP [fe80::e98f:bae2:19f7:6b0f]:58779 -> [ff02::1:3]:5355 [flowId: 18][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/184 bytes -> 0 pkts/0 bytes][Goodput ratio: 32/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Risk Info: Invalid chars detected in domain name][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
134 UDP [fe80::e034:7be:d8f9:6197]:49766 -> [ff02::1:3]:5355 [flowId: 124][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/182 bytes -> 0 pkts/0 bytes][Goodput ratio: 32/0][0.41 sec][Hostname/SNI: charming-pc][PLAIN TEXT (charming)][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
135 UDP [fe80::5d92:62a8:ebde:1319]:49735 -> [ff02::1:3]:5355 [flowId: 89][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/178 bytes -> 0 pkts/0 bytes][Goodput ratio: 30/0][0.10 sec][Hostname/SNI: wangs-ltw][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
136 UDP [fe80::5d92:62a8:ebde:1319]:58468 -> [ff02::1:3]:5355 [flowId: 127][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/178 bytes -> 0 pkts/0 bytes][Goodput ratio: 30/0][0.10 sec][Hostname/SNI: wangs-ltw][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
@ -230,9 +230,9 @@ JA3 Host Stats:
144 UDP [fe80::edf5:240a:c8c0:8312]:61603 -> [ff02::1:3]:5355 [flowId: 9][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/172 bytes -> 0 pkts/0 bytes][Goodput ratio: 28/0][0.41 sec][Hostname/SNI: ro_x1c][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
145 UDP 192.168.5.49:61548 -> 224.0.0.252:5355 [flowId: 52][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/150 bytes -> 0 pkts/0 bytes][Goodput ratio: 44/0][0.41 sec][Hostname/SNI: caesar-thinkpad][PLAIN TEXT (caesar)][Plen Bins: 0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
146 UDP 192.168.5.49:64568 -> 224.0.0.252:5355 [flowId: 103][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/150 bytes -> 0 pkts/0 bytes][Goodput ratio: 44/0][0.41 sec][Hostname/SNI: caesar-thinkpad][PLAIN TEXT (caesar)][Plen Bins: 0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
147 UDP 192.168.3.95:51451 -> 224.0.0.252:5355 [flowId: 97][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/144 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
148 UDP 192.168.3.95:54888 -> 224.0.0.252:5355 [flowId: 33][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/144 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.10 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
149 UDP 192.168.3.95:58779 -> 224.0.0.252:5355 [flowId: 19][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/144 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
147 UDP 192.168.3.95:51451 -> 224.0.0.252:5355 [flowId: 97][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/144 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Risk Info: Invalid chars detected in domain name][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
148 UDP 192.168.3.95:54888 -> 224.0.0.252:5355 [flowId: 33][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/144 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.10 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Risk Info: Invalid chars detected in domain name][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
149 UDP 192.168.3.95:58779 -> 224.0.0.252:5355 [flowId: 19][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/144 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.41 sec][Hostname/SNI: ????????????][Risk: ** Non-Printable/Invalid Chars Detected **][Risk Score: 100][Risk Info: Invalid chars detected in domain name][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
150 UDP 192.168.5.50:57143 -> 224.0.0.252:5355 [flowId: 123][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/142 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.41 sec][Hostname/SNI: charming-pc][PLAIN TEXT (charming)][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
151 UDP 192.168.5.50:62756 -> 224.0.0.252:5355 [flowId: 81][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/142 bytes -> 0 pkts/0 bytes][Goodput ratio: 41/0][0.41 sec][Hostname/SNI: charming-pc][PLAIN TEXT (charming)][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
152 UDP 192.168.5.48:59797 -> 224.0.0.252:5355 [flowId: 77][proto: 154/LLMNR][IP: 0/Unknown][ClearText][Confidence: DPI][DPI packets: 1][cat: Network/14][2 pkts/140 bytes -> 0 pkts/0 bytes][Goodput ratio: 40/0][0.41 sec][Hostname/SNI: kasper-mac][PLAIN TEXT (Kasper)][Plen Bins: 100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]