Commit graph

895 commits

Author SHA1 Message Date
emanuele-f
898811ff6f Fix MDNS/Netbios names mixed with host labels
Addresses #3677
2020-05-08 12:12:52 +02:00
Luca Deri
698265c4fb Added TLS issuerDN, subjectDN 2020-05-07 19:06:13 +02:00
Simone Mainardi
ebbe0155e3 Several ZMQ traffic and throughput calc fixes
Addresses #3863
2020-05-07 17:28:44 +02:00
Luca Deri
0f9cec48c0 Compilation fixes due to nDPI API update 2020-05-07 00:16:17 +02:00
Simone Mainardi
96f443dee8 Compilation fix 2020-05-06 10:02:12 +02:00
emanuele-f
8773bfcca3 Fix heap overflow in Flow::dissectHTTP
Fixes #3885
2020-05-05 14:37:48 +02:00
Simone Mainardi
6082024c13 Updates flow throughput immediately for ZMQ flows
Possibly addresses #3863
2020-04-30 19:20:09 +02:00
Alfredo Cardigliano
823af75000 Propagate and print in flow details AS src/dst/prev/next from collected flows 2020-04-30 10:09:26 +00:00
Simone Mainardi
f472196830 Protects local/remote stats updates from possibly NULL hosts (views) 2020-04-28 16:15:39 +02:00
Simone Mainardi
28480748aa Implements runtime flows dump toggle 2020-04-22 18:12:49 +02:00
Luca Deri
8ce11c8a48 Added flow.getnDPIMatchPacket() API call. See
https://github.com/ntop/ntopng/blob/dev/doc/src/api/lua_c/flow_user_scripts/flow.lua
for more info

Fixes #3797
2020-04-19 21:16:34 +02:00
Luca Deri
f99e184e90 Split LuaEngine.cpp apart (further improvements are necessary) 2020-04-19 12:29:13 +02:00
Simone Mainardi
6ea50fb004 Implements the Elasticsearch alerts endpoint
Implements #3105

Initial work to add elastisearch alerts

Initial implementation of ES alerts

Implements version check

Implements flow alerts format

Changes Elasticsearch index name

Comments the Elasticsearch alert endpoint
2020-04-18 16:35:14 +02:00
Alfredo Cardigliano
a254a3ad6b Support for Enterprise L license 2020-04-02 11:02:56 +00:00
Simone Mainardi
43284e22f8 Fixes crash and removes unused PortContactStats
Class was not used and didn't handle concurrent accesses to strings,
yielding crashes such as

Thread 14 "stats_update.lu" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffb3fff700 (LWP 25268)]
0x00005555556b9183 in IpAddress::intoa (this=0x28c, buf=0x7fffb3ffe030 "", bufLen=64, bitmask=255 '\377') at src/IpAddress.cpp:352
352     src/IpAddress.cpp: No such file or directory.
(gdb)
(gdb) bt
    at src/GenericHash.cpp:222
    #9  0x00005555556c0413 in NetworkInterface::walker (this=0x555556159e70, begin_slot=0x7fffb3ffe454, walk_all=true, wtype=walker_flows, walker=0x5555556c6b79 <host_flow_update_stats(GenericHashEntry*, void*, bool*)>,
        user_data=0x7fffb3ffe458) at src/NetworkInterface.cpp:795
	#10 0x00005555556c6f7c in NetworkInterface::periodicStatsUpdate (this=0x555556159e70, vm=0x7fffac0a5578) at src/NetworkInterface.cpp:2589
	#11 0x0000555555671d62 in ntop_periodic_stats_update (vm=0x7fffac0a5578) at src/LuaEngine.cpp:6304
	#12 0x000055555576c2f6 in luaD_precall ()
	#13 0x0000555555777acd in luaV_execute ()
	#14 0x000055555576c5cf in luaD_call ()
	#15 0x000055555576c621 in luaD_callnoyield ()
	#16 0x000055555576ba42 in luaD_rawrunprotected ()
	#17 0x000055555576c91b in luaD_pcall ()
	#18 0x0000555555769cd4 in lua_pcallk ()
	#19 0x0000555555681e8a in LuaEngine::run_loaded_script (this=0x7fffac01fe10) at src/LuaEngine.cpp:12188
	#20 0x000055555563eca2 in ThreadedActivity::runScript (this=0x555562aee4e0, now=1585309410, script_path=0x7fff88002620 "/usr/share/ntopng/scripts/callbacks/interface/stats_update.lua", iface=0x555556159e70, deadline=1585309420)
	    at src/ThreadedActivity.cpp:418
	    #21 0x00005555556ba04d in ThreadPool::run (this=0x5555624525a0) at src/ThreadPool.cpp:103
	    #22 0x00005555556b9c79 in doRun (ptr=0x5555624525a0) at src/ThreadPool.cpp:31
	    #23 0x00007ffff65aa6db in start_thread (arg=0x7fffb3fff700) at pthread_create.c:463
	    #24 0x00007ffff3ea388f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
2020-03-27 13:36:51 +01:00
Simone Mainardi
60bc75bb57 Fixes multiple flow Lua calls to protocolDetected 2020-03-24 22:41:56 +01:00
emanuele-f
3052e1370d Fix invalid characters in MDNS name
Fixes #3614
2020-03-24 10:42:56 +01:00
Simone Mainardi
992ca90023 Fixes ndpi_netbios_name_interpret call after nDPI API change 2020-03-12 22:22:46 +01:00
Simone Mainardi
4dcef7ac7d Fix nEdge not blocking DNS Flows 2020-03-12 19:58:12 +01:00
Simone Mainardi
5bcd301a7d Allows nDPI dissection to continue on DNS flows
Fixes #3106
2020-03-12 19:09:48 +01:00
Simone Mainardi
88693f5f6a Adds flow tcp seq issues (ooo, retx, lost) over ZMQ 2020-03-04 15:01:24 +01:00
Alfredo Cardigliano
ce1e730203 -F nindex;debug to add all nindex info to the json 2020-03-02 10:59:08 +01:00
emanuele-f
23f08f31ed Compilation fix 2020-02-26 16:11:56 +01:00
Alfredo Cardigliano
c3d3984285 Add more info to flow serialize (json) 2020-02-26 13:23:07 +01:00
Alfredo Cardigliano
dbe07bbfcd Score computation fix (external alerts) 2020-02-24 12:27:48 +01:00
emanuele-f
a428bb665f Add work completion indicator to ht_stats/stats/5min dump 2020-02-21 17:43:07 +01:00
Simone Mainardi
5b70db90ad Handles deadlines for flow user scripts 2020-02-19 10:46:44 +01:00
emanuele-f
7a582753bf Fix domain-based custom protocols match on ZMQ interfaces 2020-02-17 18:50:51 +01:00
emanuele-f
15898e8dad Flow dissection code cleanup.
The Flow API is now composed of the following methods:
  - Flow::setDetectedProtocol to manually set a protocol on the flow
    and terminate the dissection
  - Flow::processPacket to run the nDPI dissection on the raw packets
  - Flow::endProtocolDissection to terminate or give up the dissection

Also fixes the flow sampling rate preference, which was never used.
2020-02-17 17:36:06 +01:00
Luca Deri
8786c9fb30 Added support for ALPN and Client Supported TLS Protocols reported by nDPI 2020-02-17 16:29:24 +01:00
Simone Mainardi
330379cb89 Fixes active alerted flows less than idle alerted flows 2020-02-14 17:49:40 +01:00
Simone Mainardi
6b8d926729 Fixes flows from pcap files not dumped to database
Fixes #3412
2020-02-14 11:44:56 +01:00
emanuele-f
107c9e50f2 Add shadow dns query to prevent use after free 2020-02-14 11:01:05 +01:00
emanuele-f
3635c7e36c 54a4ef8b30 2020-02-13 14:39:03 +01:00
Simone Mainardi
f7d79e50f9 Implements anti stall when dumping flows (MySQL, nIndex, ES, ...)
Implements #3411
2020-02-13 11:36:48 +01:00
Luca Deri
aea228e376 Compilation fixes 2020-02-08 10:16:56 +00:00
Simone Mainardi
bbeb491534 Fixes certain DNS flows with multiple queries not reset 2020-02-07 20:29:44 +01:00
emanuele-f
7105cb8a65 Add score and user script config link into the flow details additional status 2020-02-07 20:02:19 +01:00
emanuele-f
e3d3d3992f Replace an existing flow alert if a more critical problem is found
Also add the flow score into the database
2020-02-07 19:20:57 +01:00
emanuele-f
0a0a3c4537 Rework flow status accounting 2020-02-07 19:17:07 +01:00
Simone Mainardi
38ee7f8439 Fixes for idle transitions not performed
Fixes #3394

Commit also affects #3106 as now DNS flows are adjusted
periodically when they contain more than one query
2020-02-07 19:13:45 +01:00
emanuele-f
5882926b0b Host score increment fix for PCAP dump interfaces 2020-02-07 11:32:22 +01:00
Luca Deri
f1d8fb00f4 Handled partial protocol detection 2020-02-06 22:01:32 +01:00
Luca
30f52179d9 Fixes #3106 2020-02-06 19:21:05 +01:00
Simone Mainardi
05750cf3ef Fixes and cleanup 2020-02-05 18:07:54 +01:00
emanuele-f
8d51dae0e5 Implement Misbehaving vs Alerted flows chart 2020-01-27 15:37:36 +01:00
Luca Deri
e55854aaff Initial work on host bins 2020-01-27 11:55:00 +01:00
Simone Mainardi
ca2322531c Fixes 3WH computation with ECE and CWR flags
Fixes #3255
2020-01-20 11:53:41 +01:00
emanuele-f
566b9ece0b Score changes
- Move score from status definition to user scripts
- Separate flow score counter from the peers score
- Create a new HostScore class to hold the score data
2020-01-16 18:11:14 +01:00
emanuele-f
01d5d83f21 Account host score also on volatile flows (e.g. scans)
Such flows may go idle too early and miss the minute.lua iteration
2020-01-16 14:08:49 +01:00