Commit graph

78 commits

Author SHA1 Message Date
Simone Mainardi
efe6bcf691 Fixes notifications.lua running for too long
Fixes #4523
2020-10-05 13:14:55 +02:00
Simone Mainardi
72e4382333 Reworks purging of idle entries without Lua periodic scripts 2020-10-01 16:08:21 +02:00
Simone Mainardi
0db456c0cf Reworks flow user scripts execution in C++ 2020-09-30 18:41:36 +02:00
Simone Mainardi
8392e126f8 Simplifies and reduces the number of threads for purging idle entries 2020-09-29 18:51:44 +02:00
Simone Mainardi
5133af720f Reworks view interfaces with queues; rework idling and flow hooks exec 2020-09-29 18:30:10 +02:00
Simone Mainardi
33bd6bb1c9 Fixes callbacks last duration
Fixes #4436
2020-09-16 18:10:51 +02:00
Simone Mainardi
5e7f2cfff4 Implements thread for notifications handling 2020-09-09 12:16:51 +02:00
Simone Mainardi
99366d6ee3 Optimizes execution of user scripts (avoids multiple hosts HT walks) 2020-07-29 14:45:52 +02:00
Simone Mainardi
1703488796 Fixes system CPU load 2020-07-29 11:42:39 +02:00
Simone Mainardi
8418d932ec Avoids spawning num_interfaces + threads to update stats 2020-07-27 15:50:11 +02:00
emanuele-f
32c61136ed Add one thread in periodic activities for the system interface
This avoids that the system interface and the non-system interfaces
interfere each other.
2020-03-30 14:44:54 +02:00
Luca Deri
49c80c4615 Win changes 2020-03-20 09:50:36 +01:00
emanuele-f
1c7dabf247 Pass current periodic activity time from C
This prevents possible inconsistencies between the C and lua time
2020-03-05 17:09:40 +01:00
Luca
31466a9832 Reworked periodic activities thread pool 2020-03-04 19:37:59 +01:00
Luca Deri
317e30a230 Reworked timeseries.lua behaviour to run more often with no deadline whatsoever 2020-03-04 12:37:08 +01:00
Simone Mainardi
1e82eb073b Adds and enforces periodic activities max duration
Implements #3477
2020-03-02 19:09:54 +01:00
Luca Deri
517184c2aa Reworked thread pool assignment 2020-02-26 08:28:10 +01:00
Simone Mainardi
55cdb6c5ea Reworks slow rrd calc for periodic activites 2020-02-25 13:30:10 +01:00
Simone Mainardi
1cd375d756 Implements per-activity rrd writes duration 2020-02-19 15:34:29 +01:00
Simone Mainardi
7f449f8e64 Implements deadline-related ntopng methods 2020-02-18 15:47:28 +01:00
Simone Mainardi
6542516d27 System interface threaded activity fixes 2020-02-13 15:38:02 +01:00
emanuele-f
e3d3d3992f Replace an existing flow alert if a more critical problem is found
Also add the flow score into the database
2020-02-07 19:20:57 +01:00
Luca Deri
57e6a93065 Improved scheduling algorithm to guarantee minimum service time when number of entries allow
Updated (C)
2020-01-06 23:36:36 +01:00
emanuele-f
c74fa6e908 Implement reusable lua engine
The same Lua vm is now reused. After 5 minutes the vm is destroyed and
reloaded to prevent memory to grow too much. inotify is used to detected
changes in the filesystem (partially implemented, must implement a recursive
watch)

This optimizes periodic scripts calls by a 10x factor.
2019-12-24 18:35:58 +01:00
Simone Mainardi
893350653b Fixes incomplete data with pcap files
Fixes #3128
2019-12-17 12:35:06 +01:00
Simone Mainardi
ff09e5b61f Allows flow lua scripts to be run on pcap dump interfaces 2019-11-21 12:53:25 +01:00
Simone Mainardi
bfbe661b8f Prevents low-pri stats update to preemt hash table updates 2019-11-19 12:29:52 +01:00
emanuele-f
df0556cb87 Optimize flow alerts generation
This provides a ~10x speedup by performing the JSON serialization work in C
(and thus avoiding Lua->C overhead). This also implements two in-memory alerts
queues (one for sqlite and one for the notifications) in order to reduce Redis load.
Alerts queue are now global instead of per-interface as there is only 1 dequeing thread.
2019-11-14 11:54:13 +01:00
emanuele-f
d44dad6f31 Fix shutdown crash in PeriodicActivities
A use-after-free occurred since ThreadedActivity where destroyed while their task was possibly
still running into the ThreadPool thread.
2019-11-06 11:46:22 +01:00
emanuele-f
94643f1fe1 Fix system script concurrent run
Multiple instances of the housekeeping and timeseries scripts were launched
since commit 84bf948f0f .

Fixes #3035
2019-11-04 12:42:26 +01:00
Luca Deri
84bf948f0f Reworked thread handling policy 2019-11-01 12:15:08 +01:00
Luca Deri
aee1b3defb Split threaded activities in high and standard priority tasks 2019-11-01 11:03:56 +01:00
Simone Mainardi
e8277e46ff Reworks view interfaces implementation 2019-10-30 11:47:56 +01:00
Simone Mainardi
dc748b1b6d Implements stats for periodic scripts duration 2019-10-28 17:10:03 +01:00
Simone Mainardi
f98dd234e8 Decouples periodic stats updates using a thread pool 2019-10-28 12:21:57 +01:00
Simone Mainardi
68246efd01 Decouples periodic hash table updates using a thread pool 2019-10-27 17:40:07 +01:00
Simone Mainardi
9863cfd0ba Reworks InfluxDB export to better handle retries 2019-06-24 17:48:53 +02:00
Luca Deri
bedff2c8e2 Updated (C) 2019-01-07 19:28:49 +01:00
Luca Deri
8545c78cfc Compilation fixes 2018-12-20 11:03:12 +01:00
Luca Deri
25039127e4 Win changes 2018-12-20 10:44:03 +01:00
Simone Mainardi
d34dd054fb Fixes ignored housekeeping activity num threads 2018-10-16 09:33:58 +02:00
Luca
2e8e39851a Improved termination tracings 2018-09-12 08:43:16 +02:00
emanuele-f
a594e648c4 Generalize high resolution timeseries
In InfluxDB, 5 minutes timeseries are now written with a minimum of 1 minute resolution
2018-09-03 14:43:52 +02:00
emanuele-f
0004e08191 Add experimental support for 5sec granularity for host timeseries
To enable it (only on *InfluxDB* right now):

- uncomment HOST_NUM_TIMESERIES_POINTS in ntop_defines.h
- run `redis-cli set ntopng.prefs.30_sec_dump 1` and restart ntopng
2018-08-23 18:10:46 +02:00
Luca Deri
ec2053f1a8 Minor code cleanup 2018-05-30 22:54:13 +02:00
Luca Deri
f1b50009f6 Added PERIODIC_DEBUG for debugging purposes 2018-05-29 08:11:22 +02:00
Luca Deri
f0c936c368 Updated (C) to 2018 2018-03-22 21:59:42 +01:00
emanuele-f
99b67b16ad Remove old nedge define and move to new one 2018-02-09 13:01:08 +01:00
Luca Deri
c9ecd931d9 Code cleanup 2017-12-16 08:23:56 +01:00
emanuele-f
c750d5c3aa Increase pinger script interval 2017-12-15 14:25:59 +01:00