2381 commits

Author	SHA1	Message	Date
emanuele-f	c19bda5425	Move internal alerts queue from redis to C	2019-11-14 12:07:29 +01:00
emanuele-f	df0556cb87	Optimize flow alerts generation ... This provides a ~10x speedup by performing the JSON serialization work in C (and thus avoiding Lua->C overhead). This also implements two in-memory alerts queues (one for sqlite and one for the notifications) in order to reduce Redis load. Alerts queue are now global instead of per-interface as there is only 1 dequeing thread.	2019-11-14 11:54:13 +01:00
emanuele-f	99bcd4c701	Fix multiple issues in username validation ... Fixes #3054	2019-11-13 12:08:35 +01:00
Simone Mainardi	47169b82be	Reworks counting of skipped flow lua calls	2019-11-11 18:22:37 +01:00
emanuele-f	739c87a52d	Initial flow user scripts optimization	2019-11-11 18:04:48 +01:00
emanuele-f	a57049f3dc	Fix invalid cli2srv.packets field	2019-11-11 15:51:32 +01:00
emanuele-f	d9fd30b9e6	Reduce redis load due to queue rating algorithmn	2019-11-11 10:49:08 +01:00
emanuele-f	4d45ffff08	Add missed calls stats	2019-11-08 17:13:24 +01:00
Luca	5f72aa0c2e	nDPI cleanup	2019-11-08 11:38:03 +00:00
emanuele-f	23e427e361	Reduce llen number of calls for alerts queue rating	2019-11-08 11:51:49 +01:00
emanuele-f	af1dc8a05e	Limit alerts insertions if the queues are full	2019-11-08 11:26:49 +01:00
emanuele-f	2708450e96	Add normalizeSeriesPoints	2019-11-07 17:15:16 +01:00
Alfredo Cardigliano	d54e34b102	Reworked flow fields parsing. Fixed SIP/RTP details (Fix #3043)	2019-11-07 12:51:13 +01:00
Luca	1fb32a0795	Unified nDPI struct across interfaces	2019-11-06 23:02:35 +00:00
emanuele-f	58d88d161e	Implement redis num calls charts	2019-11-06 18:12:17 +01:00
Alfredo Cardigliano	cb2f37529e	Reworked storeFlowAlert (using Lua table instead of many params now)	2019-11-06 10:02:15 +01:00
emanuele-f	8b6c3cb80f	Fix slack alert endpoint error	2019-11-05 16:39:28 +01:00
emanuele-f	4d322c3c09	Add missing max devices check	2019-11-05 16:00:26 +01:00
Alfredo Cardigliano	acdab024da	Enqueueing flow alerts to be stored/notified from Lua, removed DB select to notiy alert (using the alert object directly)	2019-11-05 15:53:10 +01:00
emanuele-f	17a22ef986	nil fix	2019-11-05 15:35:56 +01:00
emanuele-f	54140b6f1a	Migrate alertable disabled alerts preference to lua ... This avoids performing multiple hget on redis, thus increasing performance	2019-11-05 14:59:38 +01:00
emanuele-f	a96a90b65f	Optimize disabled alerts bitmap redis access	2019-11-05 12:29:08 +01:00
Alfredo Cardigliano	280213e148	Enqueueing alerts as soon as they are detected, instead of using 2-steps (#3038)	2019-11-05 11:04:44 +01:00
Simone Mainardi	c4ad235bf8	Fixes empty callbacks lists with alerts disabled ... Fixes #3037	2019-11-04 18:38:20 +01:00
emanuele-f	675adcf90e	Migrate hosts disabled flow status preference to lua ... By loading all the preferences at once in flow.lua setup() the redis load is reduced and design is simpler.	2019-11-04 16:50:26 +01:00
emanuele-f	191e530f6f	Clean category lists error status on startup and remove debug prints	2019-11-04 13:07:24 +01:00
Luca Deri	d61470c653	Workaround for #3033 but a proper fix is necessary	2019-11-02 20:22:35 +01:00
emanuele-f	9b5be79b1d	Remove redundant script key ... The script file name is used as the key	2019-10-31 18:24:58 +01:00
emanuele-f	cfc1f0fe57	Fix ht:lua_calls chart ... Fixes #3021	2019-10-31 18:06:02 +01:00
emanuele-f	33949d31fd	Rework benchmark stats and charts	2019-10-31 17:48:53 +01:00
Alfredo Cardigliano	d10dfc7762	Deferred flow alerts insertion through a redis queue. Removed alerts DB select when printing flow details.	2019-10-31 12:53:23 +01:00
emanuele-f	e978a83151	Improve user scripts charts ... Fixes #3020	2019-10-31 13:30:24 +01:00
Simone Mainardi	64b3892909	Counts user scripts execution time using clock ticks	2019-10-30 17:52:59 +01:00
Simone Mainardi	18a1d5ea8d	Minor fix	2019-10-30 16:43:36 +01:00
Simone Mainardi	e10dfb1b24	Fixes for the CZ translation	2019-10-30 16:41:19 +01:00
emanuele-f	34c196d99e	User scripts stats improvements	2019-10-30 10:53:54 +01:00
emanuele-f	29e5b10e6f	Fix network interface alias not used in alerts configuration	2019-10-29 16:47:22 +01:00
emanuele-f	63660b59f4	Add stacked top user scripts chart	2019-10-29 16:36:00 +01:00
emanuele-f	13a7b8ba67	Add total row in flow user scripts benchmarks	2019-10-29 15:38:22 +01:00
emanuele-f	1de959ed88	Missing reload fix	2019-10-29 12:07:27 +01:00
emanuele-f	378a1e2cd8	Optimize nDPI categories reload to avoid packet drops	2019-10-29 12:07:27 +01:00
Simone Mainardi	cce73c2f44	Implements lua interpreters telemetry	2019-10-28 23:47:03 +01:00
Simone Mainardi	dc748b1b6d	Implements stats for periodic scripts duration	2019-10-28 17:10:03 +01:00
Alfredo Cardigliano	fa57c701bb	Displaying file id to retrieve files stored by Suricata	2019-10-28 12:24:15 +01:00
emanuele-f	943012e8c3	Avoid un-necessary hourly reload of category lists when nothing changed	2019-10-28 12:22:57 +01:00
emanuele-f	930fdc9bd1	Fix flows script table	2019-10-25 18:41:37 +02:00
emanuele-f	e238605f86	Fix latest run label	2019-10-25 17:51:29 +02:00
Alfredo Cardigliano	c7120e23f3	Hide periodic user scripts in pcap dump mode (unless --original-speed is specified)	2019-10-25 16:45:23 +02:00
emanuele-f	8a2d3dafff	Fix interface paused icon	2019-10-25 16:44:20 +02:00
Alfredo Cardigliano	96ef3c05e7	New alert when the number of idle entries in the hash tabel exceeds a configured threshold (implements #3005)	2019-10-25 14:29:37 +02:00