diff --git a/doc/src/advanced_features/device_protocols.rst b/doc/src/advanced_features/device_protocols.rst index 47494fda45..38e04c712c 100644 --- a/doc/src/advanced_features/device_protocols.rst +++ b/doc/src/advanced_features/device_protocols.rst @@ -7,7 +7,7 @@ of the devices connected to a network. This includes printers, IoT and phone dev For each device type, ntopng provides a configurable set of policies to determine which applications are acceptable for the specific device type. When a non-acceptable application is detected, ntopng generates an alert. ntopng provides some built-in policies which should suit most environments. -The `Device Application Not Allowed` flow user script be enabled in order to generate alerts. +The `Device Application Not Allowed` flow check be enabled in order to generate alerts. Policies can be reviewed and customized in the `Device Applications` page. diff --git a/doc/src/alerts/available_recipients.rst b/doc/src/alerts/available_recipients.rst index 700c1dad94..de7b21967d 100644 --- a/doc/src/alerts/available_recipients.rst +++ b/doc/src/alerts/available_recipients.rst @@ -121,6 +121,7 @@ Webhook TODO +.. _ElasticsearchAlerts: Elasticsearch ------------- diff --git a/doc/src/alerts/evaluating_alerts.rst b/doc/src/alerts/evaluating_alerts.rst index 66a3194078..7ec182a683 100644 --- a/doc/src/alerts/evaluating_alerts.rst +++ b/doc/src/alerts/evaluating_alerts.rst @@ -1,32 +1,32 @@ Evaluating Alerts ================= -User Scripts +Checks ------------ -ntopng alerts are evaluated with :ref:`WebUIUserScripts`. User Scripts are executed for hosts, interfaces, SNMP devices, and other network elements, and are configurable from the settings +ntopng alerts are evaluated with :ref:`WebUIUserScripts`. Checks are executed for hosts, interfaces, SNMP devices, and other network elements, and are configurable from the settings -.. figure:: ../img/alerts_user_scripts_management.png +.. figure:: ../img/alerts_checks_management.png :align: center - :alt: User Scripts Configuration + :alt: Checks Configuration -Although only a **Default** configuration is shown in the figure above, each User Script can have multiple configurations. A configuration contains values for thresholds and other User Uscript-specific parameters. +Although only a **Default** configuration is shown in the figure above, each Check can have multiple configurations. A configuration contains values for thresholds and other User Uscript-specific parameters. -In the example below, the **Default** configuration for Host User Scripts is configured to trigger a flow flood victim alert when the number of new flows per second generated exceeds 256. +In the example below, the **Default** configuration for Host Checks is configured to trigger a flow flood victim alert when the number of new flows per second generated exceeds 256. .. figure:: ../img/alerts_default_host_configuration.png :align: center - :alt: Default Configuration for Host User Scripts + :alt: Default Configuration for Host Checks -Additional configurations can be created to specify different thresholds (e.g., a value different than 256 for the new flows generated per second), or to toggle on/off each individual User Script. +Additional configurations can be created to specify different thresholds (e.g., a value different than 256 for the new flows generated per second), or to toggle on/off each individual Check. Pools ----- -User Script Configurations are applied to pools. Pools are used to group together network elements. :ref:`BasicConceptsHostPools` group together multiple hosts. Similarly, Interface pools group together multiple interfaces, and so on. The same configuration can be applied to multiple pools, but a pool has one and only one configuration applied. Pools are managed from the system interface. +Check Configurations are applied to pools. Pools are used to group together network elements. :ref:`BasicConceptsHostPools` group together multiple hosts. Similarly, Interface pools group together multiple interfaces, and so on. The same configuration can be applied to multiple pools, but a pool has one and only one configuration applied. Pools are managed from the system interface. .. figure:: ../img/alerts_pools_management.png :align: center @@ -34,7 +34,7 @@ User Script Configurations are applied to pools. Pools are used to group togethe The configuration applied to each pool, shown in the table above, can be changed using the *Edit* button under *Actions*. -User Scripts run against all pool members will use the configuration specified for that pool. For example, an Interface pool **LAN pool** with interface :code:`eno1` can be created and associated with a User Scripts configuration **LAN config** as shown below +Checks run against all pool members will use the configuration specified for that pool. For example, an Interface pool **LAN pool** with interface :code:`eno1` can be created and associated with a Checks configuration **LAN config** as shown below .. figure:: ../img/alerts_example_lan_pool.png @@ -45,6 +45,6 @@ Configuration **LAN config** was previously created from the Users Scripts Confi .. figure:: ../img/alerts_example_lan_pool_config.png :align: center - :alt: User Script Configurations list with **LAN config** + :alt: Check Configurations list with **LAN config** -Given the association shown above, Interface User Scripts will be run against :code:`eno1` using the configuration **LAN pool**. All the other interfaces monitored have not been associated to a pool and so will have the **Default** configuration applied. \ No newline at end of file +Given the association shown above, Interface Checks will be run against :code:`eno1` using the configuration **LAN pool**. All the other interfaces monitored have not been associated to a pool and so will have the **Default** configuration applied. diff --git a/doc/src/alerts/index.rst b/doc/src/alerts/index.rst index a93a5f220e..1dbd756fbe 100644 --- a/doc/src/alerts/index.rst +++ b/doc/src/alerts/index.rst @@ -3,7 +3,7 @@ Alerts ntopng alerts are: -- Evaluated with User Scripts for pools of hosts, interfaces, SNMP devices, and other network elements +- Evaluated with Checks for pools of hosts, interfaces, SNMP devices, and other network elements - Delivered to recipients using type- or severity-based criteria .. toctree:: diff --git a/doc/src/api/lua_c/flow_user_scripts/flow.lua b/doc/src/api/lua_c/flow_checks/flow.lua similarity index 100% rename from doc/src/api/lua_c/flow_user_scripts/flow.lua rename to doc/src/api/lua_c/flow_checks/flow.lua diff --git a/doc/src/api/lua_c/flow_checks/index.rst b/doc/src/api/lua_c/flow_checks/index.rst new file mode 100644 index 0000000000..0690d6da39 --- /dev/null +++ b/doc/src/api/lua_c/flow_checks/index.rst @@ -0,0 +1,8 @@ +Flow Checks +================= + +The Flows Checks API is only available from the `Flow Checks Hooks`_. + +.. doxygenfile:: flow.lua.cpp + +.. _`Flow Checks Hooks`: ../../../plugins/check_hooks.html#flow-user-script-hooks diff --git a/doc/src/api/lua_c/flow_user_scripts/index.rst b/doc/src/api/lua_c/flow_user_scripts/index.rst deleted file mode 100644 index 5ec73e75c3..0000000000 --- a/doc/src/api/lua_c/flow_user_scripts/index.rst +++ /dev/null @@ -1,8 +0,0 @@ -Flow User Scripts -================= - -The Flows User Scripts API is only available from the `Flow User Scripts Hooks`_. - -.. doxygenfile:: flow.lua.cpp - -.. _`Flow User Scripts Hooks`: ../../../plugins/user_script_hooks.html#flow-user-script-hooks diff --git a/doc/src/api/lua_c/host_user_scripts/host.lua b/doc/src/api/lua_c/host_checks/host.lua similarity index 100% rename from doc/src/api/lua_c/host_user_scripts/host.lua rename to doc/src/api/lua_c/host_checks/host.lua diff --git a/doc/src/api/lua_c/host_checks/index.rst b/doc/src/api/lua_c/host_checks/index.rst new file mode 100644 index 0000000000..6b9bd2e000 --- /dev/null +++ b/doc/src/api/lua_c/host_checks/index.rst @@ -0,0 +1,8 @@ +Host Checks +================= + +The Host Checks API is only available from the Host `Checks Hooks`_. + +.. doxygenfile:: host.lua.cpp + +.. _`Checks Hooks`: ../../../plugins/check_hooks.html diff --git a/doc/src/api/lua_c/host_user_scripts/index.rst b/doc/src/api/lua_c/host_user_scripts/index.rst deleted file mode 100644 index 53b7a3d60e..0000000000 --- a/doc/src/api/lua_c/host_user_scripts/index.rst +++ /dev/null @@ -1,8 +0,0 @@ -Host User Scripts -================= - -The Host User Scripts API is only available from the Host `User Scripts Hooks`_. - -.. doxygenfile:: host.lua.cpp - -.. _`User Scripts Hooks`: ../../../plugins/user_script_hooks.html diff --git a/doc/src/api/lua_c/index.rst b/doc/src/api/lua_c/index.rst index 5769172be6..4513917698 100644 --- a/doc/src/api/lua_c/index.rst +++ b/doc/src/api/lua_c/index.rst @@ -5,7 +5,7 @@ The ntopng core provides a Lua C API to interact with it. The API is internally by periodic scripts and GUI scripts to extract information or to apply configuration changes. -Custom user scripts can use the API, for example, to provide new data visualizations +Custom checks can use the API, for example, to provide new data visualizations or extract the data to send it to an external program. Since some API functions can return very complex objects, which will not be covered here, @@ -28,7 +28,7 @@ so it's available through the interface object API via `interface.getFlowsInfo() ntop/index interface/index - flow_user_scripts/index - interface_user_scripts/index - host_user_scripts/index - network_user_scripts/index + flow_checks/index + interface_checks/index + host_checks/index + network_checks/index diff --git a/doc/src/api/lua_c/interface_checks/index.rst b/doc/src/api/lua_c/interface_checks/index.rst new file mode 100644 index 0000000000..10d3bb45bf --- /dev/null +++ b/doc/src/api/lua_c/interface_checks/index.rst @@ -0,0 +1,7 @@ +Interface Checks +====================== + +`Checks Hooks`_ can use the full network interface API `documented here`_. + +.. _`Checks Hooks`: ../../../plugins/check_hooks.html +.. _`documented here`: ../interface/index.html diff --git a/doc/src/api/lua_c/interface_user_scripts/index.rst b/doc/src/api/lua_c/interface_user_scripts/index.rst deleted file mode 100644 index 24a3632c1f..0000000000 --- a/doc/src/api/lua_c/interface_user_scripts/index.rst +++ /dev/null @@ -1,7 +0,0 @@ -Interface User Scripts -====================== - -`User Scripts Hooks`_ can use the full network interface API `documented here`_. - -.. _`User Scripts Hooks`: ../../../plugins/user_script_hooks.html -.. _`documented here`: ../interface/index.html diff --git a/doc/src/api/lua_c/network_checks/index.rst b/doc/src/api/lua_c/network_checks/index.rst new file mode 100644 index 0000000000..7b0b5a3472 --- /dev/null +++ b/doc/src/api/lua_c/network_checks/index.rst @@ -0,0 +1,8 @@ +Network Checks +==================== + +The Network Checks API is only available from the Network `Checks Hooks`_. + +.. doxygenfile:: network.lua.cpp + +.. _`Checks Hooks`: ../../../plugins/check_hooks.html diff --git a/doc/src/api/lua_c/network_user_scripts/network.lua b/doc/src/api/lua_c/network_checks/network.lua similarity index 100% rename from doc/src/api/lua_c/network_user_scripts/network.lua rename to doc/src/api/lua_c/network_checks/network.lua diff --git a/doc/src/api/lua_c/network_user_scripts/index.rst b/doc/src/api/lua_c/network_user_scripts/index.rst deleted file mode 100644 index ed8d8b823f..0000000000 --- a/doc/src/api/lua_c/network_user_scripts/index.rst +++ /dev/null @@ -1,8 +0,0 @@ -Network User Scripts -==================== - -The Network User Scripts API is only available from the Network `User Scripts Hooks`_. - -.. doxygenfile:: network.lua.cpp - -.. _`User Scripts Hooks`: ../../../plugins/user_script_hooks.html diff --git a/doc/src/api/rest/examples_v1.rst b/doc/src/api/rest/examples_v1.rst index 5b739c9ad2..47606990f0 100644 --- a/doc/src/api/rest/examples_v1.rst +++ b/doc/src/api/rest/examples_v1.rst @@ -1033,7 +1033,7 @@ Response: "severity": "error", "count": 1, "entity_val": "", - "msg": "TLS Certificate Expired [24/08/2019 18:04:13 - 22/11/2019 18:04:13] [Flow: Info 192.168.1.93:61650 [ 28:37:37:00:6D:C8 ] 192.168.1.176:443 [ 00:80:8F:9A:AE:BD ]] " + "msg": "TLS Certificate Expired [24/08/2019 18:04:13 - 22/11/2019 18:04:13] [Flow: Info 192.168.1.93:61650 [ 28:37:37:00:6D:C8 ] 192.168.1.176:443 [ 00:80:8F:9A:AE:BD ]] " }, { "entity": "flow", @@ -1043,7 +1043,7 @@ Response: "severity": "error", "count": 1, "entity_val": "", - "msg": "TLS Certificate Mismatch [Client Requested: cdn.gigya.com] [Server Names: a248.e.akamai.net,*.akamaized-staging.net,*.akamaized.net,*.akamaihd-staging.net,*.akamaihd.net] [Flow: Info 192.168.1.93:61632 [ 28:37:37:00:6D:C8 ] 184.51.127.56:443 [ 10:13:31:F1:39:76 ]] " + "msg": "TLS Certificate Mismatch [Client Requested: cdn.gigya.com] [Server Names: a248.e.akamai.net,*.akamaized-staging.net,*.akamaized.net,*.akamaihd-staging.net,*.akamaihd.net] [Flow: Info 192.168.1.93:61632 [ 28:37:37:00:6D:C8 ] 184.51.127.56:443 [ 10:13:31:F1:39:76 ]] " } ], "rc": 0, diff --git a/doc/src/api/rest/rest-api-v1.json b/doc/src/api/rest/rest-api-v1.json index 4310bc1a1b..cf5f97236e 100644 --- a/doc/src/api/rest/rest-api-v1.json +++ b/doc/src/api/rest/rest-api-v1.json @@ -2828,7 +2828,7 @@ "All" ], "summary": "Export all configurations as a JSON file", - "description": "Export all configurations including Pools, Active Monitoring, Notifications (Endpoints and Recipients), User Scripts, SNMP devices", + "description": "Export all configurations including Pools, Active Monitoring, Notifications (Endpoints and Recipients), Checks, SNMP devices", "operationId": "export_all_config", "produces": [ "application/json" @@ -2916,7 +2916,7 @@ "Pools" ], "summary": "Export the pools configuration as a JSON file", - "description": "Export the configuration of all pools and dependencies (notifications, SNMP, active monitoring, user scripts", + "description": "Export the configuration of all pools and dependencies (notifications, SNMP, active monitoring, checks", "operationId": "export_pool_config", "produces": [ "application/json" @@ -2944,8 +2944,8 @@ "tags": [ "" ], - "summary": "Export the User Scripts configuration as a JSON file", - "description": "Export the configuration for all user scripts", + "summary": "Export the Checks configuration as a JSON file", + "description": "Export the configuration for all checks", "operationId": "export_scripts_config", "produces": [ "application/json" @@ -3003,7 +3003,7 @@ "All" ], "summary": "Import all configurations providing a JSON file", - "description": "Import all configurations including Pools, Active Monitoring, Notifications (Endpoints and Recipients), User Scripts, SNMP devices", + "description": "Import all configurations including Pools, Active Monitoring, Notifications (Endpoints and Recipients), Checks, SNMP devices", "operationId": "import_all_config", "produces": [ "application/json" @@ -3099,7 +3099,7 @@ "Pools" ], "summary": "Import the pools configuration providing a JSON file", - "description": "Import the configuration of all pools and dependencies (notifications, SNMP, active monitoring, user scripts", + "description": "Import the configuration of all pools and dependencies (notifications, SNMP, active monitoring, checks", "operationId": "import_pool_config", "produces": [ "application/json" @@ -3130,8 +3130,8 @@ "tags": [ "" ], - "summary": "Import the User Scripts configuration providing a JSON file", - "description": "Import the configuration for all user scripts", + "summary": "Import the Checks configuration providing a JSON file", + "description": "Import the configuration for all checks", "operationId": "import_scripts_config", "produces": [ "application/json" diff --git a/doc/src/basic_concepts/alerts.rst b/doc/src/basic_concepts/alerts.rst index 94429c4fd8..77f21c22db 100644 --- a/doc/src/basic_concepts/alerts.rst +++ b/doc/src/basic_concepts/alerts.rst @@ -3,7 +3,7 @@ Alerts ====== -ntopng generates alerts to report the occurrence of events and user-configurable thresholds. :ref:`WebUIUserScripts` are responsible for the generation of alerts. Enabling and disabling a user scripts enables or disables the corresponding alerts. +ntopng generates alerts to report the occurrence of events and user-configurable thresholds. :ref:`WebUIUserScripts` are responsible for the generation of alerts. Enabling and disabling a checks enables or disables the corresponding alerts. Alerts include, but are not limited to: diff --git a/doc/src/basic_concepts/plugins.rst b/doc/src/basic_concepts/plugins.rst index bb1d3c57a6..17c69de9ff 100644 --- a/doc/src/basic_concepts/plugins.rst +++ b/doc/src/basic_concepts/plugins.rst @@ -1,4 +1,4 @@ -Plugins and User Scripts +Plugins and Checks ======================== End users can extend the ntopng functionalities by creating plugins which @@ -6,7 +6,7 @@ allow them, for example, to trigger custom alerts or provide new data visualizat in ntopng. Plugins can contain different resources, for example `localization files`_, `custom pages`_ -and `alert definitions`_. The `User Scripts`_ are one particular resource which +and `alert definitions`_. The `Checks`_ are one particular resource which allow the user to implement a custom logic in response to an event (such events are called Hooks in ntopng). @@ -16,4 +16,4 @@ For an extensive discussion on how to develop new plugins check out the `Plugins .. _`localization files`: ../plugins/localization.html .. _`custom pages`: ../plugins/custom_pages.html .. _`alert definitions`: ../plugins/alert_definitions.html -.. _`User Scripts`: ../plugins/user_scripts.html +.. _`Checks`: ../plugins/checks.html diff --git a/doc/src/developing_alerts/developing_alerts.rst b/doc/src/developing_alerts/developing_alerts.rst index 45e996d738..a2db5ba600 100644 --- a/doc/src/developing_alerts/developing_alerts.rst +++ b/doc/src/developing_alerts/developing_alerts.rst @@ -1,92 +1,92 @@ -Callbacks +Checks ========= -Alerts are created inside callbacks. Callbacks are chunks of code executed by ntopng. Callbacks are implemented as C++ classes with a predefined interface. +Alerts are created inside checks. Checks are chunks of code executed by ntopng. Checks are implemented as C++ classes with a predefined interface. -Callback interfaces are declared in classes: +Check interfaces are declared in classes: -- :code:`include/FlowCallback.h` for flows -- :code:`include/HostCallback.h` for hosts +- :code:`include/FlowCheck.h` for flows +- :code:`include/HostCheck.h` for hosts -Those classes must be used as base classes when implementing callbacks: +Those classes must be used as base classes when implementing checks: - - Every host callback implemented must inherit from :code:`HostCallback` - - Every flow callback implemented must inherit from :code:`FlowCallback` + - Every host check implemented must inherit from :code:`HostCheck` + - Every flow check implemented must inherit from :code:`FlowCheck` Classes are implemented with two files, namely a :code:`.h` file with the class declaration, and a :code:`.cpp` file with the class definition: - - Host callback declarations (:code:`.h` files) are under :code:`include/host_callbacks`. Host callback definitions (:code:`.cpp`) files are under :code:`src/host_callbacks`. - - Flow callback declarations (:code:`.h` files) are under :code:`include/flow_callbacks`. Flow callback definitions (:code:`.cpp`) files are under :code:`src/host_callbacks`. + - Host check declarations (:code:`.h` files) are under :code:`include/host_checks`. Host check definitions (:code:`.cpp`) files are under :code:`src/host_checks`. + - Flow check declarations (:code:`.h` files) are under :code:`include/flow_checks`. Flow check definitions (:code:`.cpp`) files are under :code:`src/host_checks`. -Callback Execution +Check Execution ------------------ -Callbacks execution for hosts consists in ntopng calling: +Checks execution for hosts consists in ntopng calling: -- :code:`HostCallback::periodicUpdate` approximately every 60 seconds +- :code:`HostCheck::periodicUpdate` approximately every 60 seconds -Every host callback, when subclassing :code:`HostCallback`, must override :code:`periodicUpdate` to implement the desired callback behavior. +Every host check, when subclassing :code:`HostCheck`, must override :code:`periodicUpdate` to implement the desired check behavior. -Callbacks execution for flows consists in ntopng calling for every flow: +Checks execution for flows consists in ntopng calling for every flow: -- :code:`FlowCallback::protocolDetected` as soon as the Layer-7 is detected -- :code:`FlowCallback::periodicUpdate` approximately every 300 seconds only for flows with a minimum duration of 300 seconds -- :code:`FlowCallback::flowEnd` as soon as the flow ends, i.e., when a TCP session is closed or when an UDP flow timeouts +- :code:`FlowCheck::protocolDetected` as soon as the Layer-7 is detected +- :code:`FlowCheck::periodicUpdate` approximately every 300 seconds only for flows with a minimum duration of 300 seconds +- :code:`FlowCheck::flowEnd` as soon as the flow ends, i.e., when a TCP session is closed or when an UDP flow timeouts -Every flow callback, when subclassing :code:`FlowCallback`, must override one or more of the methods above to implement the desired callback behavior. +Every flow check, when subclassing :code:`FlowCheck`, must override one or more of the methods above to implement the desired check behavior. -Callback Configuration +Check Configuration ---------------------- -Callbacks are configured from the ntopng Web UI. Configuration involves the ability to: +Checks are configured from the ntopng Web UI. Configuration involves the ability to: -- Turn any callback on or off -- Set configuration parameters selectively for every callback +- Turn any check on or off +- Set configuration parameters selectively for every check -A callback that is turned off is not executed. Configuration parameters can be used to set a threshold used by the callback to decide if it is time to create an alert. Similarly, configuration parameters can be used to indicate a list of IP addresses to exclude when executing callbacks. +A check that is turned off is not executed. Configuration parameters can be used to set a threshold used by the check to decide if it is time to create an alert. Similarly, configuration parameters can be used to indicate a list of IP addresses to exclude when executing checks. -ntopng, to populate the callback configuration UI and to properly store the configured callback parameters that will be passed to the C++ callback class instances, needs to know along with other information: +ntopng, to populate the check configuration UI and to properly store the configured check parameters that will be passed to the C++ check class instances, needs to know along with other information: -- Strings (optionally localized) for callback names and descriptions +- Strings (optionally localized) for check names and descriptions - Type and format of the configuration parameters -- Default parameters, e.g, whether the callback is on or off by default +- Default parameters, e.g, whether the check is on or off by default ntopng reads this information from small Lua files located in: -- :code:`scripts/lua/modules/callback_definitions/flow/` for flow callbacks -- :code:`scripts/lua/modules/callback_definitions/host` for host callbacks +- :code:`scripts/lua/modules/check_definitions/flow/` for flow checks +- :code:`scripts/lua/modules/check_definitions/host` for host checks -These files, documented here (add ref) are mandatory and must be present for a callback to be properly executed. +These files, documented here (add ref) are mandatory and must be present for a check to be properly executed. -ntopng use names to link callback configuration with its C++ class instance. A common :code:`` must be used as: +ntopng use names to link check configuration with its C++ class instance. A common :code:`` must be used as: -- The name of the Lua file under :code:`scripts/lua/modules/callback_definitions`, e.g., :code:`.lua` +- The name of the Lua file under :code:`scripts/lua/modules/check_definitions`, e.g., :code:`.lua` - The string returned by method :code:`getName` in the C++ class file, e.g., :code:`std::string getName() const { return(std::string("")); }`. Example ------- -The following figure shows the interplay between the various components of a flow callback. :code:`BlacklistedFlow` is used for reference. Full-screen is recommended to properly visualize the figure. +The following figure shows the interplay between the various components of a flow check. :code:`BlacklistedFlow` is used for reference. Full-screen is recommended to properly visualize the figure. -.. figure:: ../img/developing_alerts_callback_structure.png +.. figure:: ../img/developing_alerts_check_structure.png :align: center - :alt: BlacklistedFlow Flow Callback + :alt: BlacklistedFlow Flow Check - BlacklistedFlow Flow Callback + BlacklistedFlow Flow Check -File :code:`BlacklistedFlow.h` *(1)* contains the declaration of class `BlacklistedFlow`, a subclass of :code:`FlowCallback`. The class is defined in :code:`BlacklistedFlow.cpp` *(2)* that contains class methods implementation. +File :code:`BlacklistedFlow.h` *(1)* contains the declaration of class `BlacklistedFlow`, a subclass of :code:`FlowCheck`. The class is defined in :code:`BlacklistedFlow.cpp` *(2)* that contains class methods implementation. -To have :code:`BlacklistedFlow` compiled, an :code:`#include` directive must be added in file :code:`include/flow_callbacks_includes.h` *(3)*. The directive must contain the path to the class declaration file :code:`BlacklistedFlow.h`. +To have :code:`BlacklistedFlow` compiled, an :code:`#include` directive must be added in file :code:`include/flow_checks_includes.h` *(3)*. The directive must contain the path to the class declaration file :code:`BlacklistedFlow.h`. -To have the callback loaded and executed at runtime, :code:`BlacklistedFlow` must be instantiated and added to the ntopng callbacks in file :code:`src/FlowCallbacksLoader.cpp` *(4)*. +To have the check loaded and executed at runtime, :code:`BlacklistedFlow` must be instantiated and added to the ntopng checks in file :code:`src/FlowChecksLoader.cpp` *(4)*. Method :code:`protocolDetected` is overridden and implemented in :code:`BlacklistedFlow.cpp` *(5)* so that ntopng will call it for every flow as soon as the Layer-7 application protocol is detected. -Callback configuration UI is populated according to the contents of :code:`scripts/lua/modules/callback_definitions/flow/blacklisted.lua` *(6)*. ntopng is able to link the callback configuration with its C++ class thanks to the name :code:`blacklisted` as highlighted with the arrow starting at *(6)*. Indeed, to have the C++ and the Lua properly linked, the same name is used for: +Check configuration UI is populated according to the contents of :code:`scripts/lua/modules/check_definitions/flow/blacklisted.lua` *(6)*. ntopng is able to link the check configuration with its C++ class thanks to the name :code:`blacklisted` as highlighted with the arrow starting at *(6)*. Indeed, to have the C++ and the Lua properly linked, the same name is used for: - The name of the Lua file - The string returned by method :code:`getName` in the C++ class file @@ -95,7 +95,7 @@ Callback configuration UI is populated according to the contents of :code:`scrip Alerts ====== -Callbacks create alerts as part of their implementation. A callback, during its execution, can detect a certain condition (e.g., an anomaly) for which it decides to create an alert. When the callback decides to create an alert, it informs ntopng by passing a reference to the alert. +Checks create alerts as part of their implementation. A check, during its execution, can detect a certain condition (e.g., an anomaly) for which it decides to create an alert. When the check decides to create an alert, it informs ntopng by passing a reference to the alert. Alerts are implemented with C++ classes. Alert interfaces are declared in classes: @@ -167,17 +167,17 @@ These files are mandatory and must be present for an alert to be properly create Creating Flow Alerts -------------------- -Alert classes are instantiated inside :code:`buildAlert`, a method that must be implemented by each flow callback. This method is called by ntopng to create the alert, when it has been told to do so from a flow callback. +Alert classes are instantiated inside :code:`buildAlert`, a method that must be implemented by each flow check. This method is called by ntopng to create the alert, when it has been told to do so from a flow check. -Callbacks use :code:`triggerAlertAsync` to tell ntopng to create an alert. Indeed, The actual alert creation is triggered from the flow callback with the call :code:`f->triggerAlertAsync`. This call tells ntopng to create an alert identified with :code:`BlacklistedFlowAlert::getClassType()` on the flow instance pointed by :code:`f`. +Checks use :code:`triggerAlertAsync` to tell ntopng to create an alert. Indeed, The actual alert creation is triggered from the flow check with the call :code:`f->triggerAlertAsync`. This call tells ntopng to create an alert identified with :code:`BlacklistedFlowAlert::getClassType()` on the flow instance pointed by :code:`f`. Creating Host Alerts -------------------- -Alert classes are instantiated inside host callbacks. +Alert classes are instantiated inside host checks. -Callbacks use :code:`triggerAlert` to tell ntopng to create an alert. Indeed, The actual alert creation is triggered from the host callback with the call :code:`h->triggerAlert` that wants a pointer to the host alert instance as parameter. This call tells ntopng to create an alert on the host instance pointed by :code:`h`. +Checks use :code:`triggerAlert` to tell ntopng to create an alert. Indeed, The actual alert creation is triggered from the host check with the call :code:`h->triggerAlert` that wants a pointer to the host alert instance as parameter. This call tells ntopng to create an alert on the host instance pointed by :code:`h`. Example @@ -196,9 +196,9 @@ File :code:`BlacklistedFlowAlert.h` *(1)* contains the declaration of class `Bla To have :code:`BlacklistedFlowAlert` compiled, an :code:`#include` directive must be added in file :code:`include/flow_alerts_includes.h` *(3)*. The directive must contain the path to the class declaration file :code:`BlacklistedFlowAlert.h`. -Class :code:`BlacklistedFlowAlert` is instantiated inside :code:`buildAlert` *(4)*, a method of flow callback :code:`BlacklistedFlow`. Indeed, as seen in the previous section, alerts are created from callbacks. This method is called by ntopng to create the alert, when it has been told to do so from a callback. +Class :code:`BlacklistedFlowAlert` is instantiated inside :code:`buildAlert` *(4)*, a method of flow check :code:`BlacklistedFlow`. Indeed, as seen in the previous section, alerts are created from checks. This method is called by ntopng to create the alert, when it has been told to do so from a check. -The actual alert creation is triggered from the flow callback with the call :code:`f->triggerAlertAsync` *(5)*. This call tells ntopng to create an alert identified with :code:`BlacklistedFlowAlert::getClassType()` on the flow instance pointed by :code:`f`. +The actual alert creation is triggered from the flow check with the call :code:`f->triggerAlertAsync` *(5)*. This call tells ntopng to create an alert identified with :code:`BlacklistedFlowAlert::getClassType()` on the flow instance pointed by :code:`f`. Method :code:`getClassType()` returns an alert key *(6)* that is enumerated inside file :code:`ntop_typedefs.h`, as part of the :code:`FlowAlertTypeEnum` enumeration - follow the arrow starting at *(6)*. The same key is also enumerated in :code:`flow_alert_keys.lua` *(7)*, with the same enumeration name and number. @@ -213,16 +213,16 @@ Flows To create a flow alert, say :code:`BadFlowAlert`, check the following items: -- Implement a flow callback :code:`BadFlow` that inherits from :code:`FlowCallback` +- Implement a flow check :code:`BadFlow` that inherits from :code:`FlowCheck` - - Place the class declaration file :code:`BadFlow.h` inside :code:`include/flow_callbacks/BadFlow.h` - - Place the class definition file :code:`BadFlow.cpp` inside :code:`src/flow_callbacks/BadFlow.cpp` - - Add an :code:`#include "flow_callbacks/BadFlow.h"` directive in :code:`include/flow_callbacks_includes.h` - - Add a :code:`new BadFlow()` constructor in :code:`src/FlowCallbacksLoader.cpp` + - Place the class declaration file :code:`BadFlow.h` inside :code:`include/flow_checks/BadFlow.h` + - Place the class definition file :code:`BadFlow.cpp` inside :code:`src/flow_checks/BadFlow.cpp` + - Add an :code:`#include "flow_checks/BadFlow.h"` directive in :code:`include/flow_checks_includes.h` + - Add a :code:`new BadFlow()` constructor in :code:`src/FlowChecksLoader.cpp` -- Implement a Lua file :code:`bad_flow.lua` for the callback configuration +- Implement a Lua file :code:`bad_flow.lua` for the check configuration - - Place :code:`bad_flow.lua` inside :code:`scripts/lua/modules/callback_definitions/flow/` + - Place :code:`bad_flow.lua` inside :code:`scripts/lua/modules/check_definitions/flow/` - Edit method :code:`getName` in :code:`BadFlow.h` to return string :code:`bad_flow` - Implement a flow alert :code:`BadFlowAlert` that inherits from :code:`FlowAlert` @@ -243,16 +243,16 @@ Hosts To create an host alert, say :code:`BadHostAlert`, check the following items: -- Implement an host callback :code:`BadHost` that inherits from :code:`HostCallback` +- Implement an host check :code:`BadHost` that inherits from :code:`HostCheck` - - Place the class declaration file :code:`BadHost.h` inside :code:`include/host_callbacks/BadHost.h` - - Place the class definition file :code:`BadHost.cpp` inside :code:`src/host_callbacks/BadHost.cpp` - - Add an :code:`#include "host_callbacks/BadHost.h"` directive in :code:`include/host_callbacks_includes.h` - - Add a :code:`new BadHost()` constructor in :code:`src/HostCallbacksLoader.cpp` + - Place the class declaration file :code:`BadHost.h` inside :code:`include/host_checks/BadHost.h` + - Place the class definition file :code:`BadHost.cpp` inside :code:`src/host_checks/BadHost.cpp` + - Add an :code:`#include "host_checks/BadHost.h"` directive in :code:`include/host_checks_includes.h` + - Add a :code:`new BadHost()` constructor in :code:`src/HostChecksLoader.cpp` -- Implement a Lua file :code:`bad_host.lua` for the callback configuration +- Implement a Lua file :code:`bad_host.lua` for the check configuration - - Place :code:`bad_host.lua` inside :code:`scripts/lua/modules/callback_definitions/host/` + - Place :code:`bad_host.lua` inside :code:`scripts/lua/modules/check_definitions/host/` - Edit method :code:`getName` in :code:`BadHost.h` to return string :code:`bad_host` - Implement an host alert :code:`BadHostAlert` that inherits from :code:`HostAlert` diff --git a/doc/src/developing_alerts/index.rst b/doc/src/developing_alerts/index.rst index 013e5decf5..0c6c9e9759 100644 --- a/doc/src/developing_alerts/index.rst +++ b/doc/src/developing_alerts/index.rst @@ -3,7 +3,7 @@ Developing Alerts ntopng has the ability to create alerts for flows, hosts, and other network elements. Alerts for flows and hosts are created inside the C++ core of ntopng for performance. This section describes how to create alerts for hosts and flows. Alerts for other network elements are created by means of plugins (:ref:`Plugin Structure`). -Alerts are created inside callbacks. This section starts with a description of callbacks, and then moves to the alerts. The interplay between alerts and callbacks is presented, along with examples with the aim of giving a comprehensive overview of all the components at play. The section ends with handy checklists that can be used as reference when developing alerts. +Alerts are created inside checks. This section starts with a description of checks, and then moves to the alerts. The interplay between alerts and checks is presented, along with examples with the aim of giving a comprehensive overview of all the components at play. The section ends with handy checklists that can be used as reference when developing alerts. .. toctree:: :maxdepth: 2 diff --git a/doc/src/faq.rst b/doc/src/faq.rst index b8fad7588a..77611ec099 100644 --- a/doc/src/faq.rst +++ b/doc/src/faq.rst @@ -193,7 +193,7 @@ In order to reset the ntopng preferences the following commands can be used: On windows instead of restarting ntopnng with the systemctl tool you need to use the Windows Services Manager. How Can I Troubleshoot SNMP ? -============================ +============================= If SNMP polling is not working as expected it may be necessary to inspect the low-level SNMP messages exchanged by ntopng and the SNMP agents. diff --git a/doc/src/img/alerts_user_scripts_management.png b/doc/src/img/alerts_checks_management.png similarity index 100% rename from doc/src/img/alerts_user_scripts_management.png rename to doc/src/img/alerts_checks_management.png diff --git a/doc/src/img/user_scripts_gui_items_list.png b/doc/src/img/checks_gui_items_list.png similarity index 100% rename from doc/src/img/user_scripts_gui_items_list.png rename to doc/src/img/checks_gui_items_list.png diff --git a/doc/src/img/user_scripts_gui_threshold_cross.png b/doc/src/img/checks_gui_threshold_cross.png similarity index 100% rename from doc/src/img/user_scripts_gui_threshold_cross.png rename to doc/src/img/checks_gui_threshold_cross.png diff --git a/doc/src/img/developing_alerts_alert_structure.png b/doc/src/img/developing_alerts_alert_structure.png index f79ba889a6..8c6205a045 100644 Binary files a/doc/src/img/developing_alerts_alert_structure.png and b/doc/src/img/developing_alerts_alert_structure.png differ diff --git a/doc/src/img/developing_alerts_callback_structure.png b/doc/src/img/developing_alerts_callback_structure.png deleted file mode 100644 index 15a49cf674..0000000000 Binary files a/doc/src/img/developing_alerts_callback_structure.png and /dev/null differ diff --git a/doc/src/img/developing_alerts_check_structure.png b/doc/src/img/developing_alerts_check_structure.png new file mode 100644 index 0000000000..185e79bd00 Binary files /dev/null and b/doc/src/img/developing_alerts_check_structure.png differ diff --git a/doc/src/img/internals_user_scripts.png b/doc/src/img/internals_checks.png similarity index 100% rename from doc/src/img/internals_user_scripts.png rename to doc/src/img/internals_checks.png diff --git a/doc/src/img/web_gui_checks.png b/doc/src/img/web_gui_checks.png new file mode 100644 index 0000000000..952b88e8d4 Binary files /dev/null and b/doc/src/img/web_gui_checks.png differ diff --git a/doc/src/img/web_gui_user_scripts_configurations.png b/doc/src/img/web_gui_checks_configurations.png similarity index 100% rename from doc/src/img/web_gui_user_scripts_configurations.png rename to doc/src/img/web_gui_checks_configurations.png diff --git a/doc/src/img/web_gui_user_scripts.png b/doc/src/img/web_gui_user_scripts.png deleted file mode 100644 index feb0c75500..0000000000 Binary files a/doc/src/img/web_gui_user_scripts.png and /dev/null differ diff --git a/doc/src/img/web_gui_user_scripts_configurations_edit.png b/doc/src/img/web_gui_user_scripts_configurations_edit.png deleted file mode 100644 index a077df8d2a..0000000000 Binary files a/doc/src/img/web_gui_user_scripts_configurations_edit.png and /dev/null differ diff --git a/doc/src/img/web_gui_user_scripts_configurations_edit_traffic.png b/doc/src/img/web_gui_user_scripts_configurations_edit_traffic.png deleted file mode 100644 index 392aa3f976..0000000000 Binary files a/doc/src/img/web_gui_user_scripts_configurations_edit_traffic.png and /dev/null differ diff --git a/doc/src/plugins/alert_definitions.rst b/doc/src/plugins/alert_definitions.rst index f1d3d5ea5d..e160ce24db 100644 --- a/doc/src/plugins/alert_definitions.rst +++ b/doc/src/plugins/alert_definitions.rst @@ -69,7 +69,7 @@ Alert description :code:`i18n_description` can be either a string with the alert String ~~~~~~ -When the alert description is string, it is localized as described in :ref:`Plugin Localization`. An :code:`alert_json` table is passed as the parameters table for the localization. Keys and values of :code:`alert_json` can be used to add parameters to the localization string. Refer to :ref:`Triggering Alerts` to see how to create and :code:`alert_json`. +When the alert description is string, it is localized as described in :ref:`Plugin Localization`. An :code:`alert_json` table is passed as the parameters table for the localization. Keys and values of :code:`alert_json` can be used to add parameters to the localization string. Function ~~~~~~~~ @@ -80,8 +80,6 @@ When the alert description is a function, it gets called with three parameters: - :code:`alert`: A Lua table containing the details of the alert. - :code:`alert_json`: A Lua table that can be used to add parameters to the localization string. -Refer to :ref:`Triggering Alerts` for additional details on these three parameters. - The function is expected to return a string which is possibly localized. It is up to the plugin to call the :code:`i18n()` localization function to do the actual localization. ntopng will not perform any localization on the returned value of the function. Examples diff --git a/doc/src/plugins/user_script_gui.rst b/doc/src/plugins/check_gui.rst similarity index 86% rename from doc/src/plugins/user_script_gui.rst rename to doc/src/plugins/check_gui.rst index 1ac7882031..bf5c3088df 100644 --- a/doc/src/plugins/user_script_gui.rst +++ b/doc/src/plugins/check_gui.rst @@ -1,12 +1,12 @@ .. _Web GUI: -User Scripts GUI +Checks GUI ################ -User Scripts can define their own configuration and allow the final user +Checks can define their own configuration and allow the final user to modify it by providing a specialized GUI. -The GUI is defined into the User Script via the :code:`gui` attribute: +The GUI is defined into the Check via the :code:`gui` attribute: .. code:: lua @@ -35,7 +35,7 @@ The most important GUI attributes are: description of the element - :code:`input_builder`: defines the template which will be used to render the GUI. If it's not defined, the user will only be able to - enable or disable the user scripts without any additional configuration. + enable or disable the checks without any additional configuration. The :code:`default_value` can be used to define a default for the configuration. Its format depends on the `input_builder` used, see below for more information. @@ -53,7 +53,7 @@ available input builders are: - :code:`flow_mud`: template specialized for the `flow MUD plugin`_ Currently the user is limited to the above input builders but in the future it will be -possible to define new input builders. The input builders backend code can be found in `user_scripts_templates.lua`_ +possible to define new input builders. The input builders backend code can be found in `checks_templates.lua`_ whereas the frontend code can be found in `scripts-list-utils.js`_. Here is a description of the most useful input builders. @@ -65,12 +65,12 @@ The Threshold Cross input builder allows the user to configure a threshold for a metric (e.g. the DNS traffic) for one or more granularities. Each granularity can be separately enabled. -.. figure:: ../img/user_scripts_gui_threshold_cross.png +.. figure:: ../img/checks_gui_threshold_cross.png :align: center :alt: Threshold Cross GUI :scale: 60 -When the `threshold_cross` input builder is used, the user script can specify some additional parameters into +When the `threshold_cross` input builder is used, the check can specify some additional parameters into the :code:`gui` section: - :code:`field_max`: max value for the threshold field @@ -79,7 +79,7 @@ the :code:`gui` section: - :code:`field_operator`: can be used to fix a specific threshold operator: :code:`gt` for the ">" operator, :code:`lt` for the "<" operator - :code:`i18n_field_unit`: localization string to be displayed after the threshold - field (e.g. "Bytes" in the example above). `user_scripts.field_units` provides some commonly used values. + field (e.g. "Bytes" in the example above). `checks.field_units` provides some commonly used values. Here is an example showing the usage of the `threshold_cross` input builder: @@ -93,7 +93,7 @@ Here is an example showing the usage of the `threshold_cross` input builder: i18n_description = "entity_thresholds.syn_victim_description", input_builder = "threshold_cross", - i18n_field_unit = user_scripts.field_units.syn_sec, -- Syn/Sec + i18n_field_unit = checks.field_units.syn_sec, -- Syn/Sec field_max = 65535, -- Max 65535 Syn/Sec field_min = 1, -- Min 1 Syn/Sec field_operator = "gt"; -- Only ">" will be shown @@ -109,7 +109,7 @@ Here is an example showing the usage of the `threshold_cross` input builder: .. -The current script configuration is available into `params.user_script_config`: +The current script configuration is available into `params.check_config`: - :code:`operator`: contains :code:`gt` for ">" or :code:`lt` for "<" - :code:`threshold`: contains the numeric value for the threshold @@ -123,7 +123,7 @@ Items List This input builder allows the user to insert a list of items. The items must be separated by a comma. -.. figure:: ../img/user_scripts_gui_items_list.png +.. figure:: ../img/checks_gui_items_list.png :align: center :alt: Items List GUI :scale: 60 @@ -178,5 +178,5 @@ As shown into the example, the configured list of countries can be found in :code:`conf.items`. .. _`scripts-list-utils.js`: https://github.com/ntop/ntopng/blob/dev/httpdocs/js/config_callbacks/scripts-list-utils.js -.. _`user_scripts_templates.lua`: https://github.com/ntop/ntopng/blob/dev/scripts/lua/modules/user_scripts_templates.lua +.. _`checks_templates.lua`: https://github.com/ntop/ntopng/blob/dev/scripts/lua/modules/checks_templates.lua .. _`flow MUD plugin`: https://github.com/ntop/ntopng/tree/dev/scripts/plugins/mud diff --git a/doc/src/plugins/check_hooks.rst b/doc/src/plugins/check_hooks.rst new file mode 100644 index 0000000000..290565f4a2 --- /dev/null +++ b/doc/src/plugins/check_hooks.rst @@ -0,0 +1,96 @@ +.. _Check Hooks: + +Check Hooks +================= + +ntopng uses hooks to know when to execute a check. Hooks are string keys of the plugin :code:`hooks` table and have a check function assigned. Hooks are associated to: + +- Predefined events for flows +- Intervals of time for any other network element such as an host, or a network + +:ref:`Flow Check Hooks` and :ref:`Other Check Hooks` are discussed below. + +.. _Flow Check Hooks: + +Flow Check Hooks +---------------------- + +Available hooks for flow checks are the following: + +- :code:`protocolDetected`: Called after the Layer-7 application protocol has been detected. +- :code:`statusChanged`: Called when the internal status of the flow has changed since the previous invocation. +- :code:`periodicUpdate`: Called every few minutes on long-lived flows. +- :code:`flowEnd`: Called when the flow is considered finished. +- :code:`all`: A special hook which will cause the associated check to be called for all the available hooks. + +Flow Check Hooks Parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +ntopng calls flow checks with two parameters: + +- :code:`now`: An integer indicating the current epoch +- :code:`script_config`: A table containing the check configuration submitted by the user from the :ref:`Web GUI`. Table can be empty if the script doesn not require user-submitted configuration. + +Flow Check Hook Example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A check which needs to be called every time a flow goes idle, will implement a check function and assign it to hook :code:`flowEnd`. + +.. code:: lua + + hooks = { + flowEnd = function (now, script_config) + --[[ Check function body --]] + end + } + + +.. _Other Check Hooks: + +Other Check Hooks +----------------------- + +Available hooks for non-flow checks are the following: + +- :code:`min`: Called every minute. +- :code:`5mins`: Called every 5 minutes. +- :code:`hour`: Called every hour. +- :code:`day`: Called every day (at midnight localtime). +- :code:`all`: A special hook name which will cause the associated check to be called for all the available hooks. + +Other Check Hooks Parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +ntopng calls every check hook function with a :code:`params` Lua table as argument. The script hook function is expected to have this structure: + +.. code:: lua + + function my_check(params) + -- ... + end + +The :code:`params` contains the following keys: + +- :code:`granularity`: one of :code:`aperiodic`, :code:`min`, :code:`5mins`, :code:`hour`, :code:`day`. +- :code:`alert_entity`: A table carrying information on the current entity which can be used to generate alerts. +- :code:`entity_info`: A string identifying the current entity. +- :code:`cur_alerts`: Currently engaged alert for the entity. +- :code:`check_config`: The check configuration submitted by the user from the :ref:`Web GUI`. Table can be empty if the script doesn not require user-submitted configuration. +- :code:`check`: The name of the check which is being called. +- :code:`when`: An integer indicating the current epoch. +- :code:`ifid`: The interface id of the current interface. +- :code:`ts_enabled`: True when the timeseries generation is enabled for the current timeseries. + +It is ntopng which takes care of calling the hook check function with table :code:`params` opportunely populated. + + +Other Check Hooks Example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A check which needs to be called every minute will implement a check function and assign it to hook :code:`min` + +.. code:: lua + + hooks = {min = function (params) --[[ Check function body --]] end } + + diff --git a/doc/src/plugins/user_scripts.rst b/doc/src/plugins/checks.rst similarity index 65% rename from doc/src/plugins/user_scripts.rst rename to doc/src/plugins/checks.rst index 8b0f21baa5..294dabb9b8 100644 --- a/doc/src/plugins/user_scripts.rst +++ b/doc/src/plugins/checks.rst @@ -1,21 +1,21 @@ -.. _User Scripts: +.. _Checks: -User Scripts +Checks ============ -User scripts are the core of plugins. They actually allow certain +Checks are the core of plugins. They actually allow certain actions to be performed periodically, or when a certain event occurs. The logic of a plugin is contained in its users -scripts. A plugin contains zero to many user scripts. +scripts. A plugin contains zero to many checks. Structure --------- -The structure of a user script is the following: +The structure of a check is the following: .. code:: lua - local user_scripts = require("user_scripts") + local checks = require("checks") -- ################################################################# @@ -37,10 +37,10 @@ The structure of a user script is the following: return(script) -Every user script must return a Lua table with the following keys: +Every check must return a Lua table with the following keys: - - :code:`hooks`: a Lua table with hook names as key and callbacks as values. :ref:`User Script Hooks` are events or points in time. ntopng uses hooks to know when to call a user script. A user script defining a hook will get the hook callaback called by ntopng. User scripts must register to at least one hook. See :ref:`User Script Hooks`. - - :code:`gui`: a Lua table specifying user script name, description and configuration. Data is used by ntopng to show the user script configurable from the :ref:`Web GUI`. + - :code:`hooks`: a Lua table with hook names as key and checks as values. :ref:`Check Hooks` are events or points in time. ntopng uses hooks to know when to call a check. A check defining a hook will get the hook callaback called by ntopng. Checks must register to at least one hook. See :ref:`Check Hooks`. + - :code:`gui`: a Lua table specifying check name, description and configuration. Data is used by ntopng to show the check configurable from the :ref:`Web GUI`. - :code:`packet_interface_only` (optional): only execute the script on packet interfaces, excluding ZMQ interfaces. - :code:`nedge_only` (optional): if true, the script is only executed in nEdge. - :code:`nedge_exclude` (optional): if true, the script is not executed in nEdge. @@ -50,35 +50,35 @@ Every user script must return a Lua table with the following keys: Furthermore, a script may define the following extra functions, which are only called once per script: - - :code:`setup()`: called once per user script. If it returns :code:`false` then the script is considered + - :code:`setup()`: called once per check. If it returns :code:`false` then the script is considered disabled and its hooks are not be called. - :code:`teardown()`: called after the script operation is complete (e.g. after all the hosts have been iterated and hooks called). -.. _Flow User Scripts: +.. _Flow Checks: -Flow User Scripts +Flow Checks ----------------- -Flow user scripts are executed on each network flow directly from the C++ with flow callbacks. The user script have access to flow information such as L4 and L7 protocols, peers involved in the communication, and other things. -This information can be retrieved via the `Flow User Scripts API`_. +Flow checks are executed on each network flow directly from the C++ with flow checks. The check have access to flow information such as L4 and L7 protocols, peers involved in the communication, and other things. +This information can be retrieved via the `Flow Checks API`_. -Refer to :ref:`Flow User Script Hooks` for available hooks. +Refer to :ref:`Flow Check Hooks` for available hooks. -.. _`Flow User Scripts API`: ../api/lua_c/flow_user_scripts/index.html +.. _`Flow Checks API`: ../api/lua_c/flow_checks/index.html ntopng supports users scripts for the following traffic elements: - - :code:`interface`: a network interface of ntopng. Check out the `Interface User Scripts API`_. - - :code:`network`: a local network of ntopng. Check out the `Network User Scripts API`_. + - :code:`interface`: a network interface of ntopng. Check out the `Interface Checks API`_. + - :code:`network`: a local network of ntopng. Check out the `Network Checks API`_. - :code:`system`: the system on top of which is running ntopng - :code:`SNMP interfaces`: interfaces of monitored SNMP devices -Refer to :ref:`Other User Script Hooks` for available hooks. +Refer to :ref:`Other Check Hooks` for available hooks. -.. _`Interface User Scripts API`: ../api/lua_c/interface_user_scripts/index.html -.. _`Network User Scripts API`: ../api/lua_c/network_user_scripts/index.html +.. _`Interface Checks API`: ../api/lua_c/interface_checks/index.html +.. _`Network Checks API`: ../api/lua_c/network_checks/index.html -Syslog User Scripts +Syslog Checks ------------------- Syslog scripts are used to handle syslog events and ingest data, @@ -89,12 +89,12 @@ Scripts Location ~~~~~~~~~~~~~~~~ Syslog scripts are located under -:code:`/usr/share/ntopng/scripts/callbacks/syslog` and should use the +:code:`/usr/share/ntopng/scripts/callbacks/system/syslog` and should use the source name (e.g. application name) with the :code:`.lua` extension as file name. In fact messages demultiplexing is implemented by using the source name for matching the script name. For example, log messages coming from :code:`suricata` will be delivered to the -:code:`/usr/share/ntopng/scripts/callbacks/syslog/suricata.lua` +:code:`/usr/share/ntopng/scripts/checks/syslog/suricata.lua` script. Script API diff --git a/doc/src/plugins/distributing_plugins.rst b/doc/src/plugins/distributing_plugins.rst index 8d7414662c..e787692d97 100644 --- a/doc/src/plugins/distributing_plugins.rst +++ b/doc/src/plugins/distributing_plugins.rst @@ -87,14 +87,14 @@ becomes | |-- manifest.lua | |-- status_definitions | | `-- status_blacklisted.lua - | `-- user_scripts + | `-- checks | `-- flow | `-- blacklisted.lua `-- flow_flood |-- alert_definitions | `-- alert_flows_flood.lua |-- manifest.lua - `-- user_scripts + `-- checks |-- host | |-- flow_flood_attacker.lua | `-- flow_flood_victim.lua diff --git a/doc/src/plugins/examples.rst b/doc/src/plugins/examples.rst index 16cebe1954..39fbc9312f 100644 --- a/doc/src/plugins/examples.rst +++ b/doc/src/plugins/examples.rst @@ -27,7 +27,7 @@ The complete structure of the plugin is as follows: blacklisted |-- manifest.lua - |-- user_scripts + |-- checks `-- flow `-- blacklisted.lua |-- alert_definitions @@ -56,17 +56,16 @@ sub-directories and a :code:`manifest.lua` (see :ref:`Manifest`) file containing } Sub-directories -:code:`alert_definitions` and :code:`status_definitions` contain Lua scripts necessary to define alerts and flow statuses. Specifically, :ref:`Alert Definitions` define the alerts the plugin is going to trigger, whereas :ref:`Flow Definitions` define flow statues the plugin is going to set. In this specific plugin, :code:`alert_flow_blacklisted.lua` tells ntopng the plugin is willing to create an alert for blacklisted flows. Similarly, :code:`status_blacklisted.lua` tells ntopng the plugin is going to set a blacklisted status for certain flows. Those two directories, as said by their names, contain just definitions of alerts and flow status, -the actual logic which sets the status and trigger the alert resides in directory :code:`user_scripts`. +the actual logic which sets the status and trigger the alert resides in directory :code:`checks`. As this plugin requires flows to carry on its task, directory -:code:`user_scripts` (see :ref:`User Scripts`) with the logic must contain a sub-directory +:code:`checks` (see :ref:`Checks`) with the logic must contain a sub-directory :code:`flow`, which, in turn, contains file :code:`blacklisted.lua`. ntopng knows it has to execute :code:`blacklisted.lua` against each flow it sees because @@ -90,8 +89,8 @@ contents of :code:`blacklisted.lua`: hooks = {}, gui = { - i18n_title = "flow_callbacks_config.blacklisted", - i18n_description = "flow_callbacks_config.blacklisted_description", + i18n_title = "flow_checks_config.blacklisted", + i18n_description = "flow_checks_config.blacklisted_description", } } @@ -125,7 +124,7 @@ The first thing to observe, is that :code:`blacklisted.lua` contains a single :code:`function` with a predefined name :code:`script.hooks.protocolDetected`. This name tells ntopng to execute the plugin for every flow, as soon as the flow has -its :code:`protocolDetected`, which is one of the several :ref:`User Script Hooks` +its :code:`protocolDetected`, which is one of the several :ref:`Check Hooks` a plugin can attach to. The body of the function has access to a :code:`flow` Lua table, with @@ -188,7 +187,7 @@ The complete structure of the plugin is as follows: |-- manifest.lua |-- alert_definitions | `-- alert_flows_flood.lua - `-- user_scripts + `-- checks |-- host | |-- flow_flood_attacker.lua | `-- flow_flood_victim.lua @@ -214,13 +213,13 @@ representative for the plugin. The :code:`manifest.lua` (see :ref:`Manifest`) sc } This plugin doesn't work on flows, so no :code:`flow` directory is -present under :code:`user_scripts` and no :code:`status_definitions` +present under :code:`checks` and no :code:`status_definitions` is necessary as it has been seen for the `Blacklisted Flows`_. However, as this plugin generates alerts, :code:`alert_flows_flood.lua` is needed under :code:`alert_definitions` to tell ntopng about this. -The logic stays under :code:`user_scripts` (see :ref:`User Scripts`) which +The logic stays under :code:`checks` (see :ref:`Checks`) which has two sub-directories: :code:`host` and :code:`network`, each one containing Lua files with the logic necessary to trigger the alert. ntopng will execute scripts under the :code:`host` directory on @@ -238,7 +237,7 @@ scripts executed on hosts (the other Lua script are similar): local alerts_api = require("alerts_api") local alert_consts = require("alert_consts") - local user_scripts = require("user_scripts") + local checks = require("checks") local script = { default_enabled = true, @@ -257,7 +256,7 @@ scripts executed on hosts (the other Lua script are similar): gui = { i18n_title = "entity_thresholds.flow_attacker_title", i18n_description = "entity_thresholds.flow_attacker_description", - i18n_field_unit = user_scripts.field_units.flow_sec, + i18n_field_unit = checks.field_units.flow_sec, input_builder = "threshold_cross", field_max = 65535, field_min = 1, @@ -280,7 +279,7 @@ scripts executed on hosts (the other Lua script are similar): return script The first thing to observe is that the script has only one function -with a predefined name :code:`script.hooks.min` which is part of the :ref:`User Script Hooks` table. This name tells +with a predefined name :code:`script.hooks.min` which is part of the :ref:`Check Hooks` table. This name tells ntopng to call this function on every host, *every minute*. The body of the function is fairly straightforward. It access a Lua table :code:`host`, with several methods available to be called. This Lua @@ -315,16 +314,16 @@ script is to generate alerts. An empty :code:`hooks` table is then specified. This table is used by ntopng to determine when a certain -user script needs do be called. Remember the function +check needs do be called. Remember the function :code:`script.hooks.min`? That actually adds the entry :code:`min` to the :code:`hooks` table so this plugin will be executed every minute! Finally, there is a :code:`gui` table to give ntopng instructions on -how to render the configuration page of this user script. Basically, a +how to render the configuration page of this check. Basically, a title, description and unit of measure are indicated, along with an input builder and upper and lower bounds for the input. Input builders, as it will be seen in the next section, are used by ntopng -to render the configuration of the user script. +to render the configuration of the check. Log Network Traffic ------------------- @@ -335,7 +334,7 @@ This example shows how to log the traffic of a `local network`_. network_monitor/ |-- manifest.lua - `-- user_scripts + `-- checks `-- network `-- traffic_log.lua @@ -345,12 +344,12 @@ so it won't be discussed again. The core logic is contained into the .. code:: lua - local user_scripts = require("user_scripts") + local checks = require("checks") require("lua_utils") local script = { -- This is a network related script - category = user_scripts.script_categories.network, + category = checks.script_categories.network, -- This module is enabled by default default_enabled = true, @@ -378,7 +377,7 @@ so it won't be discussed again. The core logic is contained into the -- ################################################################# - return(user_scripts) + return(checks) The `script.hooks.min` hook is called by ntopng every minute for every local network. It prints into the console the local network CIDR along @@ -391,7 +390,7 @@ parameter. The most relevant fields are: - :code:`alert_entity`: the alert entity, can be passed to the alerts API to trigger alerts - :code:`entity_info`: information about the network, see below for details -- :code:`user_script_config`: the current configuration of this user script +- :code:`check_config`: the current configuration of this check The current network status is available into the `info.entity_info` field. Here are reported the most important fields: @@ -455,7 +454,7 @@ The complete structure of the plugin is as follows: |-- manifest.lua |-- alert_definitions | `-- alert_snmp_topology_changed.lua - `-- user_scripts + `-- checks `-- snmp_device `-- lldp_topology_changed.lua @@ -463,12 +462,12 @@ This plugin uses the `LLDP `. +The check keeps track of the old arcs by storing them into the Redis key `ntopng.cache.snmp_topology_arcs_monitor.`. By comparing the old registered arcs with the new ones it can determine if an arc was removed or added. diff --git a/doc/src/plugins/index.rst b/doc/src/plugins/index.rst index 08a67d21d4..7bd5abd446 100644 --- a/doc/src/plugins/index.rst +++ b/doc/src/plugins/index.rst @@ -9,14 +9,12 @@ Developing Plugins manifest alert_endpoints alert_definitions - flow_status_definitions - user_scripts - user_script_hooks - user_script_gui + checks + check_hooks + check_gui localization custom_pages timeseries_schemas examples - tutorial distributing_plugins diff --git a/doc/src/plugins/localization.rst b/doc/src/plugins/localization.rst index b2a7b442d3..f38d24667a 100644 --- a/doc/src/plugins/localization.rst +++ b/doc/src/plugins/localization.rst @@ -38,7 +38,7 @@ Plugins can put their own localized strings into localization files under the the country code as the file name, e.g. `it.lua` contains localization strings in the Italian language. ntopng currently supports the following languages: English, Czech, German, Italian, Japanese, Portuguese. Providing the English -localization file is mandatory for the plugin as it is the fallback +localization file is mandatory for the plugin as it is the fheck used when a particular string is missing for a specific language. Here is an example of a `en.lua` localization file for a plugin which @@ -85,7 +85,7 @@ A localization table is a Lua table containing keys and translated strings as va A string :code:`s` is localized as follows: 1. :code:`s` is looked up into the localization table of the language set for the user. If a key :code:`s` exists in the localization table, the localized string is taken as the value of key :code:`s` and the localization ends. If key :code:`s` does not exists: -2. :code:`s` is looked up into the English fallback localization table. If a key :code:`s` exists in the English fallback localization table, the localized string is taken as the value of key :code:`s` and the localization ends. If key :code:`s` does not exists: +2. :code:`s` is looked up into the English fheck localization table. If a key :code:`s` exists in the English fheck localization table, the localized string is taken as the value of key :code:`s` and the localization ends. If key :code:`s` does not exists: 3. The string :code:`s` is taken verbatim. If the language set for the user is English, only the English localization table is used. @@ -116,7 +116,7 @@ Consider Prefix :code:`i18n_` tells ntopng :code:`alerts_dashboard.blacklisted_flow` needs to be localized. Assuming german is set as language for the current user: 1. ntopng looks up key :code:`alerts_dashboard` in the German localization table. If the key is found and the value is a table, ntopng looks up key :code:`blacklisted_flow` in the table found as value. If key :code:`blacklisted_flow` is found, then it's value is taken as the localized string and the localization ends. If any of the two keys does not exists: -2. Step 1. is repeated on the English fallback localization table. If no localized string is found: +2. Step 1. is repeated on the English fheck localization table. If no localized string is found: 3. :code:`alerts_dashboard.blacklisted_flow` is taken verbatim. Consider now the entry diff --git a/doc/src/plugins/structure.rst b/doc/src/plugins/structure.rst index 447c84fe45..abc2126694 100644 --- a/doc/src/plugins/structure.rst +++ b/doc/src/plugins/structure.rst @@ -18,7 +18,7 @@ The most complete example of plugin structure is the following | `-- alert_example.lua |-- status_definitions | `-- status_example.lua - |-- user_scripts + |-- checks | |-- interface | | `-- example.lua | |-- network @@ -41,14 +41,14 @@ representative for the plugin. Sub-directories contain: - :code:`locales` (optional): files for the localization of strings used within the plugin, such as the description of a generated alert. When this directory is omitted, strings found in the plugin will be taken verbatim. See :ref:`Plugin Localization`. - :code:`alert_endpoints` (optional): files to create alert endpoints. An alert endpoint is called by ntopng every time an alert is generated. Alert endpoints enable an alert to be post-processed or delivered downstream to an external alert collector. This directory can be omitted when the plugin does not create alert endpoints. See :ref:`Alert Endpoints`. -- :code:`user_scripts`: files with the logic necessary to +- :code:`checks`: files with the logic necessary to perform custom actions. This directory contains additional sub-directories, namely, :code:`interface`, :code:`network`, :code:`snmp_device`, and :code:`system`. ntopng guarantees files found under the :code:`interface` directory are be executed for every interface; files found under the :code:`network` directory will be executed for every local network; and so on. Sub-directories can be missing or empty, depending - on whether the plugins wants to perform certain actions or not. See :ref:`User Scripts`. + on whether the plugins wants to perform certain actions or not. See :ref:`Checks`. - :code:`ts_schemas`: contains timeseries schemas definitions. See :ref:`Timeseries Schemas`. - :code:`web_gui`: file to create custom ntopng pages and link them in the main ntopng menu. See :ref:`Custom Pages`. diff --git a/doc/src/plugins/timeseries_schemas.rst b/doc/src/plugins/timeseries_schemas.rst index e4174e63d4..9ae520f5c8 100644 --- a/doc/src/plugins/timeseries_schemas.rst +++ b/doc/src/plugins/timeseries_schemas.rst @@ -3,7 +3,7 @@ Timeseries Schemas ================== -User scripts invoked via the `user scripts hooks`_ can use the `Timeseries +Checks invoked via the `checks hooks`_ can use the `Timeseries API`_ to write their own timeseries data and then visualize it in `Custom Pages`_. A full example of a plugin specialized in collecting Redis metrics is the `Redis monitor plugin`_ . @@ -47,7 +47,7 @@ identified by the `ifid` tag and contains one gauge metric named `num_hosts`. Schemas Usage ------------- -A user script could then use the above schema as follows: +A check could then use the above schema as follows: .. code:: lua @@ -73,8 +73,8 @@ A user script could then use the above schema as follows: It's important to check that `params.ts_enabled` flag in order to call `ts_utils.append` only if the timeseries are enabled for the given entity currently processed. -.. _`user scripts hooks`: user_script_hooks.html#user-script-hooks +.. _`checks hooks`: check_hooks.html#user-script-hooks .. _`Timeseries API`: ../api/timeseries/index.html .. _`Custom Pages`: custom_pages.html .. _`Redis monitor plugin`: https://github.com/ntop/ntopng/tree/dev/scripts/plugins/redis_monitor -.. _`the relevant page`: user_script_hooks.html#other-user-script-hooks +.. _`the relevant page`: check_hooks.html#other-user-script-hooks diff --git a/doc/src/plugins/tutorial.rst b/doc/src/plugins/tutorial.rst deleted file mode 100644 index e4ba2e26f0..0000000000 --- a/doc/src/plugins/tutorial.rst +++ /dev/null @@ -1,180 +0,0 @@ -.. _Plugin Tutorial: - -Tutorial -======== - -In this tutorial a plugin which detects executable files :code:`.exe` requested over HTTP is created. This tutorial is step-by-step: - -1. A basic plugin is created to print URLs containing :code:`.exe` to the command line is created -2. Created plugin is extended to create alerts and set a flow status - -To create this plugin, `ntopng sources `_ are used. However, this is not a requirement. Plugins can be created also for packaged versions of ntopng. The directory which will contain plugin files is created under the ntopng plugins directory as - -.. code:: bash - - # Launch this command from the root directory of the ntopng sources tree - mkdir -p ./scripts/plugins/exes_download/ - - -As this plugin detectes executable files :code:`.exe`, it must be run agains every HTTP flow. To run the plugin against every HTTP flow, a flow user script (see :ref:`Flow User Scripts`) must be placed under :code:`user_scripts/flows`: - -.. code:: bash - - mkdir -p ./scripts/plugins/exes_download/user_scripts/flow/ - touch ./scripts/plugins/exes_download/user_scripts/flow/exes_download.lua - -The file :code:`exes_download.lua` can then be edited as: - -.. code:: lua - - local user_scripts = require("user_scripts") - - local script = { - -- Script category - category = user_scripts.script_categories.security, - - -- This module is disabled by default - default_enabled = true, - - -- See below - hooks = {}, - - -- Allow user script configuration from the GUI - gui = { - i18n_title = "EXEs Download", - i18n_description = "Detects .exe downloaded via HTTP", - } - } - - -- ################################################################# - - -- Defines an hook which is executed every time a protocol of a flow is detected - function script.hooks.protocolDetected(now) - local http_info = flow.getHTTPInfo() - - -- if the flow is HTTP and it contains a last_url... - if http_info and http_info["protos.http.last_url"] then - -- if an .exe is found in the URL... - if http_info["protos.http.last_url"]:match("%.exe") then - -- Prepare a text line to be printed to the console - local line = string.format("last_url: %s [%s]\n", http_info["protos.http.last_url"], shortFlowLabel(flow.getInfo())) - -- Actually print the line to the consol - io.write(line) - end - end - end - - -- ################################################################# - - return script - -The first line - -.. code:: lua - - local user_scripts = require("user_scripts") - -Is necessary to specify a flow category in the :code:`script` table which must be returned at the end of the script. Indeed, the first key of this table is :code:`category` and has a value of :code:`user_scripts.script_categories.security`. Other categories are available in :code:`user_scripts.script_categories`. Failing to set a category would cause ntopng to choose a default category. The :code:`script` table then contains a boolean :code:`default_enabled = true` to make the user script enabled by default. This means ntopng will execute it and it will appear under the enabled flow user scripts in the web GUI. A table :code:`hooks = {}` is specified as well and is populated with :code:`function script.hooks.protocolDetected`. Finally a table :code:`gui` indicates a title and a description which will be shown under the flow user scripts of the ntopng web GUI. - -The function :code:`function script.hooks.protocolDetected` gets executed every time the Layer-7 application protocol of a flow is detected (see :ref:`Flow User Script Hooks`). This function accesses the API with :code:`flow.getHTTPInfo()` to get flow HTTP data. If the flow is not HTTP, this table will be :code:`nil`. If not :code:`nil`, the :code:`protos.http.last_url` of the flow is read and a :code:`:match` regexp is used to search for the string :code:`.exe` in the URL. If found, a simple line is prepared and printed to the console with :code:`io.write`. At this point, the plugin is functional. Restart ntopng and try to fetch a URL with a :code:`.exe`: this will cause ntopng to print flow details and URL to the console. - -To extend this plugin to generate alerts and flow statuses, two additional directories need to be created, for :ref:`Flow Definitions` and :ref:`Alert Definitions`, respectively: - -.. code:: bash - - mkdir -p ./scripts/plugins/exes_download/alert_definitions/ - mkdir -p ./scripts/plugins/exes_download/status_definitions/ - -Then, an alert definition and a flow status definition are created with two files: - -.. code:: bash - - touch scripts/plugins/exes_download/status_definitions/status_exe_download.lua - touch scripts/plugins/exes_download/alert_definitions/alert_exe_download.lua - -Set the alert definition file :code:`alert_exe_download.lua` contents as: - -.. code:: lua - - local alert_keys = require "alert_keys" - - -- ####################################################### - - -- @brief Prepare an alert table used to generate the alert - -- @param alert_severity A severity as defined in `alert_consts.alert_severities` - -- @param tls_info A Lua table with HTTP info generated calling `flow.getHTTPInfo()` - -- @return A table with the alert built - local function createExeDownload(alert_severity, http_info) - local built = { - alert_severity = alert_severity, - alert_type_params = http_info -- This info will go into the alert JSON - } - - return built - end - - -- ####################################################### - - return { - alert_key = alert_keys.user.alert_user_01, - -- equivalent - -- alert_key = {0, alert_keys.user.alert_user_01}, - -- custom pens - -- alert_key = {312 -- PEN -- , 513 --alert id --]]}, - i18n_title = "EXE download", - icon = "fas fa-exclamation", - creator = createExeDownload, - } - -The file contains the alert title and an icon which will be used by ntopng to print the alerts. As this is a user-developed plugin, and no other user-developed plugin is using it, key :code:`alert_keys.user.alert_user_01` is chosen as :code:`alert_key`. A :code:`createExeDownload` is implemented as well to add the detected HTTP information straight into the alert JSON. - -Set the status definition file :code:`status_exe_download.lua` as: - -.. code:: lua - - local alert_consts = require("alert_consts") - local status_keys = require "flow_keys" - - return { - status_key = status_keys.user.status_user_01, - alert_severity = alert_consts.alert_severities.error, - alert_type = alert_consts.alert_types.alert_exe_download, - i18n_title = "EXE download", - i18n_description = "Flow has downloaded an executable file", - } - -The file contains a status title and a description which will be used by ntopng when showing the flow status. It also contains :code:`alert_severity` and :code:`alert_type` which tell ntopng the status is going to cause an alert of type :code:`alert_exe_download` to be triggered. As this is a user-developed plugin, and no other user-developed plugin is using it, key :code:`status_keys.user.status_user_01` is chosen as :code:`status_key`. - -The final thing which is required to set the flow status and trigger the alert is to add an extra require to the user script - -.. code:: lua - - local flow_consts = require("flow_consts") - -And modify :code:`function script.hooks.protocolDetected(now)` as follow: - -.. code:: lua - - -- Defines an hook which is executed every time a procotol of a flow is detected - function script.hooks.protocolDetected(now) - local http_info = flow.getHTTPInfo() - - -- if the flow is HTTP and it contains a last_url... - if http_info and http_info["protos.http.last_url"] then - -- if an .exe is found in the URL... - if http_info["protos.http.last_url"]:match("%.exe") then - flow.triggerStatus( - flow_consts.status_types.status_exe_download.create( - flow_consts.status_types.status_exe_download.alert_severity, - http_info - ), - 100 --[[ flow_score --]], - 100 --[[ cli_score ]], - 10 --[[ srv_score]]) - end - end - end - -Basically, a new function :code:`flow.triggerStatus` is added. This function wants the result of a call to :code:`create` as first parameter. Function :code:`create` takes a severity and an :code:`http_info` as first and second parameters, respectively. These two parameters are be passed to function :code:`createExeDownload` created in the alert definition file above. Then :code:`flow.triggerStatus` takes thress scores which are added to the flow, client and server scores, respectively. - -Now the plugin is fully functional and ready to set flow statuses and trigger alerts when it detects and :code:`.exe` file. English strings can be localized as described in :ref:`Plugin Localization`. diff --git a/doc/src/plugins/user_script_hooks.rst b/doc/src/plugins/user_script_hooks.rst deleted file mode 100644 index 8a7dc866bf..0000000000 --- a/doc/src/plugins/user_script_hooks.rst +++ /dev/null @@ -1,96 +0,0 @@ -.. _User Script Hooks: - -User Script Hooks -================= - -ntopng uses hooks to know when to execute a user script. Hooks are string keys of the plugin :code:`hooks` table and have a callback function assigned. Hooks are associated to: - -- Predefined events for flows -- Intervals of time for any other network element such as an host, or a network - -:ref:`Flow User Script Hooks` and :ref:`Other User Script Hooks` are discussed below. - -.. _Flow User Script Hooks: - -Flow User Script Hooks ----------------------- - -Available hooks for flow user scripts are the following: - -- :code:`protocolDetected`: Called after the Layer-7 application protocol has been detected. -- :code:`statusChanged`: Called when the internal status of the flow has changed since the previous invocation. -- :code:`periodicUpdate`: Called every few minutes on long-lived flows. -- :code:`flowEnd`: Called when the flow is considered finished. -- :code:`all`: A special hook which will cause the associated callback to be called for all the available hooks. - -Flow User Script Hooks Parameters -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -ntopng calls flow user scripts with two parameters: - -- :code:`now`: An integer indicating the current epoch -- :code:`script_config`: A table containing the user script configuration submitted by the user from the :ref:`Web GUI`. Table can be empty if the script doesn not require user-submitted configuration. - -Flow User Script Hook Example -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A user script which needs to be called every time a flow goes idle, will implement a callback function and assign it to hook :code:`flowEnd`. - -.. code:: lua - - hooks = { - flowEnd = function (now, script_config) - --[[ Callback function body --]] - end - } - - -.. _Other User Script Hooks: - -Other User Script Hooks ------------------------ - -Available hooks for non-flow user scripts are the following: - -- :code:`min`: Called every minute. -- :code:`5mins`: Called every 5 minutes. -- :code:`hour`: Called every hour. -- :code:`day`: Called every day (at midnight localtime). -- :code:`all`: A special hook name which will cause the associated callback to be called for all the available hooks. - -Other User Script Hooks Parameters -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -ntopng calls every user script hook function with a :code:`params` Lua table as argument. The script hook function is expected to have this structure: - -.. code:: lua - - function my_callback(params) - -- ... - end - -The :code:`params` contains the following keys: - -- :code:`granularity`: one of :code:`aperiodic`, :code:`min`, :code:`5mins`, :code:`hour`, :code:`day`. -- :code:`alert_entity`: A table carrying information on the current entity which can be used to generate alerts. -- :code:`entity_info`: A string identifying the current entity. -- :code:`cur_alerts`: Currently engaged alert for the entity. -- :code:`user_script_config`: The user script configuration submitted by the user from the :ref:`Web GUI`. Table can be empty if the script doesn not require user-submitted configuration. -- :code:`user_script`: The name of the user script which is being called. -- :code:`when`: An integer indicating the current epoch. -- :code:`ifid`: The interface id of the current interface. -- :code:`ts_enabled`: True when the timeseries generation is enabled for the current timeseries. - -It is ntopng which takes care of calling the hook callback function with table :code:`params` opportunely populated. - - -Other User Script Hooks Example -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A user script which needs to be called every minute will implement a callback function and assign it to hook :code:`min` - -.. code:: lua - - hooks = {min = function (params) --[[ Callback function body --]] end } - - diff --git a/doc/src/self_monitoring/index.rst b/doc/src/self_monitoring/index.rst index d36b4ba3a7..e2650b5dab 100644 --- a/doc/src/self_monitoring/index.rst +++ b/doc/src/self_monitoring/index.rst @@ -10,4 +10,4 @@ ntopng implements checks for its own status and health as well as for services u internals redis_monitor - influxdb_monitor \ No newline at end of file + influxdb_monitor diff --git a/doc/src/self_monitoring/internals.rst b/doc/src/self_monitoring/internals.rst index f8f455e95a..ca005f69b2 100644 --- a/doc/src/self_monitoring/internals.rst +++ b/doc/src/self_monitoring/internals.rst @@ -7,7 +7,7 @@ Internals expose status and health of certain ntopng components. Internal compon - `Hash Tables` - `Periodic Activities` -- `User Scripts` +- `Checks` Information exposed is accessible from the `wrench` icon of any interface - including the :ref:`BasicConceptSystemInterface` - and is discussed detail below for each component. @@ -66,10 +66,10 @@ Information shown is useful to troubleshoot the following issues: Periodic Activities ------------------- -Periodic activities are Lua scripts executed by ntopng at regular intervals of time. Lua scripts are found in the `callbacks `_. They can be run, in parallel, for each interface or for the system. Periodic activities include, but are not limited to: +Periodic activities are Lua scripts executed by ntopng at regular intervals of time. Lua scripts are found in the `checks `_. They can be run, in parallel, for each interface or for the system. Periodic activities include, but are not limited to: - `Timeseries generation`. -- `Execution of User Scripts` as described in :ref:`WebUIUserScripts`. +- `Execution of Checks` as described in :ref:`WebUIUserScripts`. - `Handling idle and active hash table entries` as described in :ref:`InternalsHashTables`. Multiple threads are available for the execution of periodic activities. A thread executes one periodic activity at time. Multiple periodic activities are executed sequentially by the same thread. Multiple parallel threads execute multiple periodic activities simultaneously. @@ -92,7 +92,7 @@ Failing to meet one or more of the conditions above can cause ntopng to malfunct - All threads are busy so no one can execute a periodic activity ad the right `frequency`. - A bug is causing a periodic activity to take more than its `max duration` to complete. -- A user activity is executing too many :ref:`User Scripts` that make it slow. +- A user activity is executing too many :ref:`Checks` that make it slow. - Too many alerts are being generated and the export cannot keep up with the generation. @@ -106,7 +106,7 @@ Aim of the `Periodic Activities` internals table is to monitor the execution of Information shown in the table columns is: -- `Periodic Activity`: The name of the periodic activity. Name equals the file name of the periodic activity Lua script which can be either found under the `system `_ periodic activities, `interface `_ periodic activities, or both. +- `Periodic Activity`: The name of the periodic activity. Name equals the file name of the periodic activity Lua script which can be either found under the `system `_ periodic activities, `interface `_ periodic activities, or both. - `Frequency`: How often a periodic activity has to be executed. - `Max Duration`: How long the periodic activity execution can take. - `Chart`: A link to the historical charts of the periodic activity. @@ -145,29 +145,29 @@ Periodic activities with issues also have their alerts. Alerts are engaged when Internals: Periodic Activities Alerts -Periodic activities are in charge of running plugin user scripts. +Periodic activities are in charge of running plugin checks. -User Scripts +Checks ------------ -:ref:`User Scripts` are part of ntopng plugins. They are executed periodically or when a certain event occurs. +:ref:`Checks` are part of ntopng plugins. They are executed periodically or when a certain event occurs. -Aim of the `User Scripts` internals table is to monitor the execution of scripts. +Aim of the `Checks` internals table is to monitor the execution of scripts. -.. figure:: ../img/internals_user_scripts.png +.. figure:: ../img/internals_checks.png :align: center - :alt: Internals: User Scripts + :alt: Internals: Checks - Internals: User Scripts + Internals: Checks Information shown in the table columns is: -- `User Script`: The name of the user script which is executed. -- `Target`: The target of the user script, either an `host`, a `flow` or one of the other :ref:`Other User Scripts` targets. -- `Hook`: One of the :ref:`User Script Hooks`. -- `Last Num Calls`: The number of times the user script has been called the last time a periodic activity has executed it. -- `Last Duration`: The total duration of the user script, computed as the sum of the duration of any of its `Last Num Calls`. +- `Check`: The name of the check which is executed. +- `Target`: The target of the check, either an `host`, a `flow` or one of the other targets. +- `Hook`: One of the :ref:`Check Hooks`. +- `Last Num Calls`: The number of times the check has been called the last time a periodic activity has executed it. +- `Last Duration`: The total duration of the check, computed as the sum of the duration of any of its `Last Num Calls`. Information shown is useful to troubleshoot the following issues: -- `Troubleshoot` periodic activities with :ref:`Degraded Performance`: A periodic activity may be slow because it is executing too many user scripts. Combining data from the `Periodic Activities` internals table with this table can highlight this condition. +- `Troubleshoot` periodic activities with :ref:`Degraded Performance`: A periodic activity may be slow because it is executing too many checks. Combining data from the `Periodic Activities` internals table with this table can highlight this condition. diff --git a/doc/src/third_party_integrations/pfsense.rst b/doc/src/third_party_integrations/pfsense.rst index 1ec170612e..e23b97ca57 100644 --- a/doc/src/third_party_integrations/pfsense.rst +++ b/doc/src/third_party_integrations/pfsense.rst @@ -1,4 +1,4 @@ -.. _OPNsenseIntegration: +.. _pfSenseIntegration: pfsense ######## diff --git a/doc/src/use_cases/large_networks.rst b/doc/src/use_cases/large_networks.rst index 9f910b933c..9de996ca9c 100644 --- a/doc/src/use_cases/large_networks.rst +++ b/doc/src/use_cases/large_networks.rst @@ -1,5 +1,3 @@ -.. _UseCaseMikrotik: - Monitoring Large Networks ######################### diff --git a/doc/src/use_cases/mirror_tap_monitoring.rst b/doc/src/use_cases/mirror_tap_monitoring.rst index c33544a71a..aa0f46fb20 100644 --- a/doc/src/use_cases/mirror_tap_monitoring.rst +++ b/doc/src/use_cases/mirror_tap_monitoring.rst @@ -7,4 +7,4 @@ To monitor data from a mirror/SPAN port or from a tap, refer to `Monitoring a Po .. note:: - `PF_RING Zero Copy `_ licenses may be required when the traffic is above 1Gbps. In this case, see :ref:`OperatingNtopngOnLargeNetworks` and blog post `Best Practices for Efficiently Running ntopng `_. \ No newline at end of file + `PF_RING Zero Copy `_ licenses may be required when the traffic is above 1Gbps. In this case, see :ref:`OperatingNtopngOnLargeNetworks` and blog post `Best Practices for Efficiently Running ntopng `_. diff --git a/doc/src/use_cases/multiple_locations_monitoring.rst b/doc/src/use_cases/multiple_locations_monitoring.rst index b27c29353b..d3ab4c54be 100644 --- a/doc/src/use_cases/multiple_locations_monitoring.rst +++ b/doc/src/use_cases/multiple_locations_monitoring.rst @@ -7,4 +7,4 @@ To monitor data from multiple locations, even geographically distributed, refer .. note:: - An nProbe Standard or nProbe Professional license is required for each location. \ No newline at end of file + An nProbe Standard or nProbe Professional license is required for each location. diff --git a/doc/src/use_cases/netflow_sflow_monitoring.rst b/doc/src/use_cases/netflow_sflow_monitoring.rst index 130922fbaa..dfb995c53b 100644 --- a/doc/src/use_cases/netflow_sflow_monitoring.rst +++ b/doc/src/use_cases/netflow_sflow_monitoring.rst @@ -7,4 +7,4 @@ To monitor data from Netflow/sFlow-capable devices, refer to :ref:`UsingNtopngWi .. note:: - An nProbe Standard or nProbe Professional license is required. \ No newline at end of file + An nProbe Standard or nProbe Professional license is required. diff --git a/doc/src/use_cases/processes_monitoring.rst b/doc/src/use_cases/processes_monitoring.rst index 6d9a48eeca..c31efb213b 100644 --- a/doc/src/use_cases/processes_monitoring.rst +++ b/doc/src/use_cases/processes_monitoring.rst @@ -10,4 +10,4 @@ On Linux, ntopng has the ability to provide visibility into the processes which .. note:: - An nProbe Agent license is required. \ No newline at end of file + An nProbe Agent license is required. diff --git a/doc/src/use_cases/snmp_monitoring.rst b/doc/src/use_cases/snmp_monitoring.rst index 385451bd96..1f374008e9 100644 --- a/doc/src/use_cases/snmp_monitoring.rst +++ b/doc/src/use_cases/snmp_monitoring.rst @@ -7,4 +7,4 @@ ntopng has the ability to combine traffic data with SNMP data. This means it per .. note:: - An ntopng Enterprise M license or above is required. \ No newline at end of file + An ntopng Enterprise M license or above is required. diff --git a/doc/src/using_with_nprobe.rst b/doc/src/using_with_nprobe.rst index 82129df9c1..e84c68849a 100644 --- a/doc/src/using_with_nprobe.rst +++ b/doc/src/using_with_nprobe.rst @@ -7,4 +7,4 @@ Using ntopng with nProbe ######################## -See :ref:`UsingNtopngWithNprobe`. \ No newline at end of file +See :ref:`UsingNtopngWithNprobe`. diff --git a/doc/src/using_with_nprobe_agent.rst b/doc/src/using_with_nprobe_agent.rst index 5235150610..b6d60b481b 100644 --- a/doc/src/using_with_nprobe_agent.rst +++ b/doc/src/using_with_nprobe_agent.rst @@ -7,4 +7,4 @@ Using ntopng with nProbe Agent ############################## -See :ref:`UsingNtopngWithNprobeAgent`. \ No newline at end of file +See :ref:`UsingNtopngWithNprobeAgent`. diff --git a/doc/src/web_gui/checks.rst b/doc/src/web_gui/checks.rst new file mode 100644 index 0000000000..b420a9f33e --- /dev/null +++ b/doc/src/web_gui/checks.rst @@ -0,0 +1,60 @@ +.. _WebUIUserScripts: + +Checks +############ + +Checks process network elements and perform actions such as triggering alerts. Creating checks is discussed in detail in :ref:`Checks`. This section describes how users scripts can be enabled and disabled, configured, and applied to network elements such as hosts and flows. + +Checks can be configured from any interface, from the menu `Settings->Checks`. + +.. figure:: ../img/web_gui_checks.png + :align: center + :alt: Checks Settings + + Checks Settings + +Configuring a check means: + +- Choosing the network elements the script is going to be executed for. +- Enabling/disabling it. +- Setting its configuration values. + +`Configurations` are used to accomplish this. + +Configurations +-------------- + +Checks are configured by means of `Configurations`. A `Default` configuration is always present. The `Default` configuration is used to configure: + +- SNMP devices checks. +- Global system-wide checks. + +The default configuration is also applied to: + +- Any host of every interface. +- Any flow of every interface. +- Any interface. +- Any local network defined. + +Additional configurations can be created to limit the scope of their application. This means any additional configurations can be applied to: + +- Any subset of hosts of every interface. +- Any flow of a subset of the active interfaces. +- A subset of the active interfaces. +- A subset of the defined local networks. + +The scope of application is shown in column `Applied To`. The Default configuration is `Applied to` any network element which is not included in any other configuration. + +Editing Configurations +~~~~~~~~~~~~~~~~~~~~~~ + +Checks configurations can be edited for any of the configurable network elements. The edit page shows three tabs, namely `All`, `Enabled`, and `Disabled` to give immediate access to the whole list of checks, or to limit the view to only those that are enabled or disabled. + +A check can be enabled or disabled by clicking the corresponding button under the `Actions` column. Certain checks can also have their values configured. In this case, an extra button `Edit` becomes available under the `Actions` column. + +Resetting, Importing and Exporting Configurations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +All existing configurations can be imported and exported in bulk, to ease backup, restore, and replication across multiple machines. Buttons to perform this task are available below the configuration list. Button `Export Configuration` allows to download a JSON file, whereas button `Import Configuration` accepts a previously downloaded JSON file to load the configurations. `Reset Configuration` cleans up any existing configuration on the current system. + + diff --git a/doc/src/web_gui/import_export.rst b/doc/src/web_gui/import_export.rst index 848c971296..ddcc592e02 100644 --- a/doc/src/web_gui/import_export.rst +++ b/doc/src/web_gui/import_export.rst @@ -9,7 +9,7 @@ Through the web GUI it is possible to export selected configurations, including: - SNMP configuration - Active Monitoring configuration -- User Scripts configuration +- Checks configuration - Alerts Endpoints and Recipients - All Pools (this will also include all the previous items, as this depends on them) diff --git a/doc/src/web_gui/index.rst b/doc/src/web_gui/index.rst index 7aaafb66e9..57cb371c9f 100644 --- a/doc/src/web_gui/index.rst +++ b/doc/src/web_gui/index.rst @@ -61,7 +61,7 @@ Each individual menu bar entry will be discussed below. historical report flows - user_scripts + checks categories hosts host_details diff --git a/doc/src/web_gui/user_scripts.rst b/doc/src/web_gui/user_scripts.rst deleted file mode 100644 index 978e4a341d..0000000000 --- a/doc/src/web_gui/user_scripts.rst +++ /dev/null @@ -1,101 +0,0 @@ -.. _WebUIUserScripts: - -User Scripts -############ - -User scripts process network elements and perform actions such as triggering alerts. Creating user scripts is discussed in detail in :ref:`User Scripts`. This section describes how users scripts can be enabled and disabled, configured, and applied to network elements such as hosts and flows. - -User scripts can be configured from any interface, from the menu `Settings->User Scripts`. - -.. figure:: ../img/web_gui_user_scripts.png - :align: center - :alt: User Scripts Settings - - User Scripts Settings - -Configuring a user script means: - -- Choosing the network elements the script is going to be executed for. -- Enabling/disabling it. -- Setting its configuration values. - -`Configurations` are used to accomplish this. - -Configurations --------------- - -User scripts are configured by means of `Configurations`. A `Default` configuration is always present. The `Default` configuration is used to configure: - -- SNMP devices user scripts. -- Global system-wide user scripts. - -The default configuration is also applied to: - -- Any host of every interface. -- Any flow of every interface. -- Any interface. -- Any local network defined. - -Additional configurations can be created to limit the scope of their application. This means any additional configurations can be applied to: - -- Any subset of hosts of every interface. -- Any flow of a subset of the active interfaces. -- A subset of the active interfaces. -- A subset of the defined local networks. - -The scope of application is shown in column `Applied To`. The Default configuration is `Applied to` any network element which is not included in any other configuration. - -Creating New Configurations -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -New configurations can be created by `cloning` the default or any other existing configuration. To clone a configuration select the `clone` button in the `Actions` column. The cloned configuration has user scripts configurations which are equal to the configuration it has been cloned from. - -A newly cloned configuration shows a message `Warning: this config is not applied to any specific target!` in the `Applied To` column. Clicking on the `edit` button in the `Actions` column allows to specify the scope of application: - -- Under `Hosts`, a comma separated list of IPv4 and IPv6 CIDR prefixes can be specified to choose to which hosts the configuration is going to be applied. Hosts matching these prefixes will have this configuration applied. The longest-match principle holds. When an host matches multiple configurations, the one with the longest-match is chosen for application to the host. Prefixes are applied to hosts in any interface. -- Under `Interfaces` and `flows`, a multi-selection allows to choose to which interfaces the configuration is going to be applied. -- Under `Local Networks`, a multi-selections allows to choose to which local networks the configuration is going to be applied. - -An example of three configurations is shown in the image: - -.. figure:: ../img/web_gui_user_scripts_configurations.png - :align: center - :alt: User Scripts Configurations - - User Scripts Configurations - -In the image there are two configurations `Guests` and `IT Department`, in addition to the `Default` configuration. `Guests` is applied to all hosts matching `10.10.0.0/16`, whereas `IT Department` applies to all hosts matching `10.20.0.0/16`. Hosts not matching any of the two prefixes will have the `Default` configuration applied. - -As user scripts can be controlled on a per-configuration basis: - -- `Guests` can have certain user scripts enabled. The same user scripts can be disabled for `IT Department`. -- `Guests` can have certain user scripts configuration values (such as thresholds) which are different from those configured for `IT Department`. - -User scripts configuration values can be controlled using button `Edit` from the `Actions` column. - -Editing Configurations -~~~~~~~~~~~~~~~~~~~~~~ - -User scripts configurations can be edited for any of the configurable network elements. The edit page shows three tabs, namely `All`, `Enabled`, and `Disabled` to give immediate access to the whole list of user scripts, or to limit the view to only those that are enabled or disabled. - -.. figure:: ../img/web_gui_user_scripts_configurations_edit.png - :align: center - :alt: User Scripts Configuration Edit - - User Scripts Configuration Edit - -A user script can be enabled or disabled by clicking the corresponding button under the `Actions` column. Certain user scripts can also have their values configured. In this case, an extra button `Edit` becomes available under the `Actions` column. For example, the traffic alert user script can be edited as follows: - - -.. figure:: ../img/web_gui_user_scripts_configurations_edit_traffic.png - :align: center - :alt: User Scripts Configuration Edit Traffic Alert - - User Scripts Configuration Edit Traffic Alert - -Resetting, Importing and Exporting Configurations -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -All existing configurations can be imported and exported in bulk, to ease backup, restore, and replication across multiple machines. Buttons to perform this task are available below the configuration list. Button `Export Configuration` allows to download a JSON file, whereas button `Import Configuration` accepts a previously downloaded JSON file to load the configurations. `Reset Configuration` cleans up any existing configuration on the current system. - - diff --git a/httpdocs/js/config_callbacks/config-list-utils.js b/httpdocs/js/config_callbacks/config-list-utils.js index 8163fa13b9..f0a50aa852 100644 --- a/httpdocs/js/config_callbacks/config-list-utils.js +++ b/httpdocs/js/config_callbacks/config-list-utils.js @@ -114,7 +114,7 @@ $(function() { } }, ajax: { - url: `${http_prefix}/lua/get_scripts_configsets.lua?check_subdir=${subdir}`, + url: `${http_prefix}/lua/get_checks_configsets.lua?check_subdir=${subdir}`, type: 'GET', dataSrc: '' }, diff --git a/httpdocs/misc/ntopng-utils-manage-updates.in b/httpdocs/misc/ntopng-utils-manage-updates.in index 2af2111b7a..0596f3e75c 100644 --- a/httpdocs/misc/ntopng-utils-manage-updates.in +++ b/httpdocs/misc/ntopng-utils-manage-updates.in @@ -223,11 +223,14 @@ function run_upgrade() { ${APT_GET} update # Install - ${APT_GET} upgrade --assume-yes --fix-broken --allow-unauthenticated --with-new-pkgs ${PRODUCT} + #${APT_GET} upgrade --assume-yes --fix-broken --allow-unauthenticated --with-new-pkgs ${PRODUCT} + # Note: using install instead of upgrade to avoid blocking the installation due to 'kept back' packages + ${APT_GET} install --assume-yes --fix-broken --allow-unauthenticated ${PRODUCT} RESULT=$? # Check if installation is successful (we do not trust $?) - NTOPNG_VERSION=$(${APT_GET} --just-print upgrade 2>&1 | grep "Inst ${PRODUCT} " | cut -d'(' -f2 | cut -d' ' -f1) + #NTOPNG_VERSION=$(${APT_GET} --just-print upgrade 2>&1 | grep "Inst ${PRODUCT} " | cut -d'(' -f2 | cut -d' ' -f1) + NTOPNG_VERSION=$(${APT_GET} --just-print install ${PRODUCT} 2>&1 | grep "Inst ${PRODUCT} " | cut -d'(' -f2 | cut -d' ' -f1) if [ -z "${NTOPNG_VERSION}" ]; then RESULT=0 fi @@ -277,12 +280,14 @@ function check_updates() { ${APT_GET} update -o Dir::Etc::sourcelist="sources.list.d/${APT_SOURCE}" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0" # Check for broken packages - ${APT_GET} --just-print upgrade >/dev/null 2>&1 + #${APT_GET} --just-print upgrade >/dev/null 2>&1 + ${APT_GET} --just-print install ${PRODUCT} >/dev/null 2>&1 if [ ! $? -eq 0 ]; then # Something went wrong, trying to fix it ${APT_GET} --assume-yes --fix-broken install - ${APT_GET} --just-print upgrade >/dev/null 2>&1 + #${APT_GET} --just-print upgrade >/dev/null 2>&1 + ${APT_GET} --just-print install ${PRODUCT} >/dev/null 2>&1 fi if [ ! $? -eq 0 ]; then @@ -290,7 +295,9 @@ function check_updates() { ${REDIS_CLI} ${REDIS_CLI_OPT} SET ${REDIS_UPDATE_FAILURE_KEY} "update-failure" >/dev/null 2>&1 else # Check update and get version - NTOPNG_VERSION=$(${APT_GET} --just-print --assume-yes --with-new-pkgs upgrade 2>&1 | grep "Inst ${PRODUCT} " | cut -d'(' -f2 | cut -d' ' -f1) + #NTOPNG_VERSION=$(${APT_GET} --just-print --assume-yes --with-new-pkgs upgrade 2>&1 | grep "Inst ${PRODUCT} " | cut -d'(' -f2 | cut -d' ' -f1) + # Note: using install instead of upgrade to handle 'kept back' packages + NTOPNG_VERSION=$(${APT_GET} --just-print --assume-yes install ${PRODUCT} 2>&1 | grep "Inst ${PRODUCT} " | cut -d'(' -f2 | cut -d' ' -f1) RESULT=0 fi else diff --git a/include/Flow.h b/include/Flow.h index da3a1f9244..1c3de889e5 100644 --- a/include/Flow.h +++ b/include/Flow.h @@ -79,7 +79,7 @@ class Flow : public GenericHashEntry { #ifdef NTOPNG_PRO bool ingress2egress_direction; u_int8_t routing_table_id; - bool lateral_movement, create_or_delete; + bool lateral_movement; #ifndef HAVE_NEDGE FlowProfile *trafficProfile; #else @@ -346,9 +346,7 @@ class Flow : public GenericHashEntry { #ifdef NTOPNG_PRO inline bool isLateralMovement() const { return(lateral_movement); } - inline bool isCreateOrDelete() const { return(create_or_delete); } inline void setLateralMovement(bool change) { lateral_movement = change; } - inline void setCreateOrDelete(bool change) { create_or_delete = change; } #endif inline bool isCliDeviceAllowedProtocol() const { diff --git a/include/NetworkInterface.h b/include/NetworkInterface.h index 95917d1b03..08a08dae31 100644 --- a/include/NetworkInterface.h +++ b/include/NetworkInterface.h @@ -636,8 +636,10 @@ class NetworkInterface : public NetworkInterfaceAlertableEntity { inline float getThroughputPps() { return pkts_thpt.getThpt(); }; #if defined(NTOPNG_PRO) && !defined(HAVE_NEDGE) inline ServiceMap* getServiceMap() { return(sMap); }; + inline bool isServiceMapEnabled() { return(sMap ? true : false); }; inline void flushServiceMap() { if(sMap) sMap->flush(); }; inline PeriodicityMap* getPeriodicityMap() { return(pMap); }; + inline bool isPeriodicityMapEnabled() { return(pMap ? true : false); }; inline void flushPeriodicityMap() { if(pMap) pMap->flush(); }; void updateFlowPeriodicity(Flow *f); void updateServiceMap(Flow *f); diff --git a/include/flow_alerts/LateralMovementAlert.h b/include/flow_alerts/LateralMovementAlert.h index f1eb4361f5..bdbd381725 100644 --- a/include/flow_alerts/LateralMovementAlert.h +++ b/include/flow_alerts/LateralMovementAlert.h @@ -25,9 +25,6 @@ #include "ntop_includes.h" class LateralMovementAlert : public FlowAlert { - private: - ndpi_serializer* getAlertJSON(ndpi_serializer* serializer); - public: static FlowAlertType getClassType() { return { flow_alert_lateral_movement, alert_category_network }; } static u_int8_t getDefaultScore() { return SCORE_LEVEL_NOTICE; }; diff --git a/scripts/locales/en.lua b/scripts/locales/en.lua index f0ab2d57e0..842d919319 100644 --- a/scripts/locales/en.lua +++ b/scripts/locales/en.lua @@ -26,6 +26,8 @@ local lang = { ["as_number"] = "AS number", ["asn"] = "ASN", ["attacker"] = "Is Attacker", + ["asn_id"] = "ASN %{id}", + ["network_id"] = "Network %{id}", ["autolayout_description"] = "Performing auto-layout could change the network's current layout drastically.", ["availability"] = "Availability", ["available"] = "Available", @@ -507,6 +509,11 @@ local lang = { ["score_as_server"] = "Score Behaviour as Server", }, }, + ["alert_behaviors"] = { + ["score_behavior"] = "Score", + ["traffic_tx_behavior"] = "Traffic TX", + ["traffic_rx_behavior"] = "Traffic RX", + }, ["alert_entities"] = { ["am_host"] = "Active Monitoring", ["flow"] = "Flow", @@ -535,7 +542,7 @@ local lang = { ["device_has_disconnected"] = "The device %{device} has disconnected from the network.", ["flow_flood_attacker"] = "%{entity}%{host_category} is a flow flooder [%{value} > %{threshold} client flows/sec for ~3 sec]", ["flow_flood_victim"] = "%{entity}%{host_category} is under flow flood attack [%{value} > %{threshold} server flows/sec for ~3 sec]", - ["flow_number_anomaly"] = "Detected %{role} flow number anomaly for host %{host}", + ["flow_number_anomaly"] = "%{role} Flows: %{value}[Expected Range: %{lower_bound} ... %{upper_bound}]", ["host_alert_dangerous_host"] = "Score limit exceeded by %{entity}. Dangerous host [Score: %{score}][Duration: %{duration} mins]", ["host_alert_dns_server_contacts"] = "Too many DNS servers contacted by %{entity} [%{value} > %{threshold} servers]", ["host_alert_dns_traffic"] = "DNS traffic exceeded by %{entity} [%{value} %{op} %{threshold}]", @@ -582,7 +589,7 @@ local lang = { ["proto_time_quotas"] = "%{proto} time", ["remote_to_local_insecure_proto"] = "Remote to Local Insecure Protocol [Category: %{ndpi_category}][Breed: %{ndpi_breed}]", ["remote_to_remote"] = "Remote %{entity}%{host_category} contacted a remote host", - ["score_number_anomaly"] = "Detected score anomaly as %{role} [Score: %{score}][Lower Bound: %{lower_bound}][Upper Bound: %{upper_bound}][Cybersecurity Score: %{security}% | Network Score: %{network}%]", + ["score_number_anomaly"] = "%{role} Score: %{score}[Expected Range: %{lower_bound} ... %{upper_bound}]", ["shell_script_executed"] = "Shell script '%{script_exec_comm}' executed in response of an alert: '%{alert_type}'", ["slow_periodic_activity"] = "Periodic activity \"%{script}\" running for too long [more than %{max_duration}] or executed too late (blocked in queue).", ["slow_purge"] = "Hash table idle entries purging on %{iface} is too slow. This could lead to high memory utilization, data accuracy loss and missing alerts. [%{idle}%% > %{max_idle}%%]", @@ -618,6 +625,8 @@ local lang = { ["alert_severity"] = "Severity", ["alert_snmp_device_reset_description"] = "SNMP device %{device} has been restarted", ["alert_snmp_device_reset_title"] = "SNMP Device Restart", + ["alert_unexpected_behavior_description"] = "Trigger an alert when an unexpected behavior is identified from the %{type}", + ["alert_unexpected_behavior_title"] = "Unexpected %{type} Behavior", ["alert_snmp_interface_mac_changed_description"] = "MAC %{mac} seen on SNMP device %{device} interface %{port} previously seen on SNMP device %{prev_device} interface %{prev_port}", ["alert_snmp_interface_mac_changed_title"] = "SNMP Interface MAC Changed", ["alert_type"] = "Type", @@ -700,6 +709,7 @@ local lang = { ["last_hour"] = "Last Hour", ["last_minute"] = "Last Minute", ["lateral_movement"] = "Lateral Movement", + ["lateral_movement_descr"] = "Service Map change detected %{info}. Click the bell below to jump to it %{href}", ["list_download_failed"] = "List Download Failed", ["list_download_succeeded"] = "List Download Succeeded", ["local_only"] = "Local Hosts Only", @@ -828,6 +838,7 @@ local lang = { ["total_alerts"] = "Total Alerts", ["trailing_msg"] = "Time Window", ["trailing_msg_compact"] = "Window", + ["unexpected_behavior_anomaly_description"] = "%{entity} %{href} [%{type_of_behaviour}: %{value} | Expected Range: %{lower_bound} ... %{upper_bound}]", ["unexpected_host_behavior_description"] = "%{host} [%{type_of_behaviour}][Value: %{value}][Expected Range: %{lower_bound} ... %{upper_bound}]", ["unexpected_host_behaviour_act_flows_title"] = "Unexpected Host Active Flows Behaviour", ["unexpected_host_behaviour_contacts_title"] = "Unexpected Host Contacts Behaviour", diff --git a/scripts/lua/admin/manage_configurations.lua b/scripts/lua/admin/manage_configurations.lua index b004fa48db..00243680e5 100644 --- a/scripts/lua/admin/manage_configurations.lua +++ b/scripts/lua/admin/manage_configurations.lua @@ -22,14 +22,14 @@ if not ntop.isnEdge() then all = {key = "all", label = i18n("manage_configurations.everything", {product = ntop.getInfo()["product"]}), order = 0 }, snmp = {key = "snmp", label = i18n("manage_configurations.snmp"), order = 1}, active_monitoring = {key = "active_monitoring", label = i18n("manage_configurations.active_monitoring"), order = 2}, - scripts = {key = "scripts", label = i18n("manage_configurations.checks"), order = 3}, + checks = {key = "checks", label = i18n("manage_configurations.checks"), order = 3}, notifications = {key = "notifications", label = i18n("manage_configurations.notifications"), order = 4}, pool = {key = "pool", label = i18n("manage_configurations.pool_endpoint_recipients"), order = 5}, } else configuration_items = { all = {key = "all", label = i18n("manage_configurations.everything", {product = ntop.getInfo()["product"]}), order = 0 }, - scripts = {key = "scripts", label = i18n("manage_configurations.checks"), order = 1}, + checks = {key = "checks", label = i18n("manage_configurations.checks"), order = 1}, } end diff --git a/scripts/lua/flow_details.lua b/scripts/lua/flow_details.lua index 6c4294b602..19ae7667a2 100644 --- a/scripts/lua/flow_details.lua +++ b/scripts/lua/flow_details.lua @@ -1320,25 +1320,23 @@ else for _, score_alerts in pairsByKeys(alerts_by_score, rev) do for _, score_alert in pairsByField(score_alerts, "message", asc) do if first then - print(""..i18n("flow_details.flow_issues").."") + print(""..i18n("flow_details.flow_issues")..""..i18n("description")..""..i18n("actions").."") first = false end - if score_alert.is_predominant then - print(status_icon.." ") - end + print(string.format('')) - print(score_alert.message) + print(string.format('%s %s', score_alert.message, score_alert.is_predominant and status_icon or '')) if score_alert.alert_id then - print(string.format(' ', score_alert.alert_id, score_alert.alert_label)) + print(string.format('', score_alert.alert_id, score_alert.alert_label)) + else -- These are unhandled alerts, e.g., flow risks for which a check doesn't exist + print(string.format('')) end - print("
") + print('') end end - - print("\n") end -- ###################################### @@ -1631,8 +1629,8 @@ print [[ const $type = $(`${alert.alert_label}`); $(`#alerts_filter_dialog .alert_label`).text($type.text().trim()); - const cliLabel = "]] print(flowinfo2hostname(flow,"cli")) print[["; - const srvLabel = "]] print(flowinfo2hostname(flow,"srv")) print[["; + const cliLabel = "]] local n = flowinfo2hostname(flow,"cli"); if n ~= flow["cli.ip"] then print(string.format("%s (%s)", n, flow["cli.ip"])) else print(n) end print[["; + const srvLabel = "]] local n = flowinfo2hostname(flow,"srv"); if n ~= flow["srv.ip"] then print(string.format("%s (%s)", n, flow["srv.ip"])) else print(n) end print[["; $(`#cli_addr`).text(cliLabel); $(`#cli_radio`).val("]] print(flow["cli.ip"]) print[["); diff --git a/scripts/lua/get_scripts_configsets.lua b/scripts/lua/get_checks_configsets.lua similarity index 100% rename from scripts/lua/get_scripts_configsets.lua rename to scripts/lua/get_checks_configsets.lua diff --git a/scripts/lua/modules/alert_definitions/flow/alert_behavior_anomaly.lua b/scripts/lua/modules/alert_definitions/flow/alert_behavior_anomaly.lua new file mode 100644 index 0000000000..c7273d372e --- /dev/null +++ b/scripts/lua/modules/alert_definitions/flow/alert_behavior_anomaly.lua @@ -0,0 +1,65 @@ +-- +-- (C) 2019-21 - ntop.org +-- + +-- ############################################## + +local other_alert_keys = require "other_alert_keys" +local classes = require "classes" +local alert = require "alert" + +-- ############################################## + +local alert_behavior_anomaly = classes.class(alert) + +local i18n_title = i18n("alerts_dashboard.alert_unexpected_behavior_title", {type = ""}) + +-- ############################################## + +alert_behavior_anomaly.meta = { + alert_key = other_alert_keys.alert_behavior_anomaly, + i18n_title = i18n_title, + icon = "fas fa-fw fa-exclamation", +} + +-- ############################################## + +-- @brief Prepare an alert table used to generate the alert +-- @param value The value got from the measurement +-- @param lower_bound The lower bound of the measurement +-- @param upper_bound The upper bound of the measurement +-- @return A table with the alert built +function alert_behavior_anomaly:init(entity, type_of_behaviour, value, upper_bound, lower_bound) + -- Call the parent constructor + self.super:init() + + self.alert_type_params = { + entity = entity, + type_of_behaviour = type_of_behaviour, + value = value, + upper_bound = upper_bound, + lower_bound = lower_bound, + } +end + +-- ####################################################### + +-- @brief Format an alert into a human-readable string +-- @param ifid The integer interface id of the generated alert +-- @param alert The alert description table, including alert data such as the generating entity, timestamp, granularity, type +-- @param alert_type_params Table `alert_type_params` as built in the `:init` method +-- @return A human-readable string +function alert_behavior_anomaly.format(ifid, alert, alert_type_params) + return(i18n("alerts_dashboard.unexpected_behavior_anomaly_description", + { + entity = alert_type_params.entity, + type_of_behaviour = alert_type_params.type_of_behaviour, + value = alert_type_params.value, + lower_bound = alert_type_params.lower_bound or 0, + upper_bound = alert_type_params.upper_bound or 0, + })) +end + +-- ####################################################### + +return alert_behavior_anomaly \ No newline at end of file diff --git a/scripts/lua/modules/alert_definitions/flow/alert_lateral_movement.lua b/scripts/lua/modules/alert_definitions/flow/alert_lateral_movement.lua index 0949ea7e7b..6dfe92fee1 100644 --- a/scripts/lua/modules/alert_definitions/flow/alert_lateral_movement.lua +++ b/scripts/lua/modules/alert_definitions/flow/alert_lateral_movement.lua @@ -40,30 +40,22 @@ end -- @param alert_type_params Table `alert_type_params` as built in the `:init` method -- @return A human-readable string function alert_lateral_movement.format(ifid, alert, alert_type_params) - local vlan_id = tonumber(alert.vlan_id) or 0 - local client = {host = alert.cli_ip, vlan = vlan_id} - local server = {host = alert.srv_ip, vlan = vlan_id} + -- Extracting info field local info = "" + local href = "" if alert.json then info = json.decode(alert["json"]) - if info["info"] then - info = info["info"] + if not isEmptyString(info["info"]) then + info = "[" .. info["info"] .. "]" else info = "" end end - local rsp = hostinfo2detailshref(client, nil, hostinfo2label(client)).. - " " .. - hostinfo2detailshref(server, nil, hostinfo2label(server)) + href = '' - rsp = rsp .. " ["..interface.getnDPIProtoName(alert.l7_proto).."]" - if not isEmptyString(info) then - rsp = rsp .. "[" .. info .. "]" - end - - return(rsp) + return(i18n("alerts_dashboard.lateral_movement_descr", { info = info, href = href })) end -- ####################################################### diff --git a/scripts/lua/modules/alert_definitions/host/host_alert_flow_anomaly.lua b/scripts/lua/modules/alert_definitions/host/host_alert_flow_anomaly.lua index e61057892c..fc071bd9e4 100644 --- a/scripts/lua/modules/alert_definitions/host/host_alert_flow_anomaly.lua +++ b/scripts/lua/modules/alert_definitions/host/host_alert_flow_anomaly.lua @@ -46,28 +46,24 @@ end -- @param alert_type_params Table `alert_type_params` as built in the `:init` method -- @return A human-readable string function host_alert_flow_anomaly.format(ifid, alert, alert_type_params) - local alert_consts = require("alert_consts") - local json = json.decode(alert.json) - local is_both = alert_type_params["is_both"] - local is_client_alert = alert_type_params["is_client_alert"] - local role - local host = alert_consts.formatHostAlert(ifid, alert["ip"], alert["vlan_id"]) + local is_both = alert_type_params["is_both"] + local is_client_alert = alert_type_params["is_client_alert"] + local role - if(is_both) then - role = "client and server" - elseif(is_client_alert) then - role = "client" - else - role = "server" - end + if(is_both) then + role = i18n("client_and_server") + elseif(is_client_alert) then + role = i18n("client") + else + role = i18n("server") + end - return i18n("alert_messages.flow_number_anomaly", { - role = role, - host = host, - value = alert_type_params["value"], - lower_bound = alert_type_params["lower_bound"], - upper_bound = alert_type_params["upper_bound"], - }) + return i18n("alert_messages.flow_number_anomaly", { + role = role, + value = alert_type_params["value"], + lower_bound = alert_type_params["lower_bound"], + upper_bound = alert_type_params["upper_bound"], + }) end -- ####################################################### diff --git a/scripts/lua/modules/alert_definitions/host/host_alert_score_anomaly.lua b/scripts/lua/modules/alert_definitions/host/host_alert_score_anomaly.lua index 24a2464945..7c6caee2ba 100644 --- a/scripts/lua/modules/alert_definitions/host/host_alert_score_anomaly.lua +++ b/scripts/lua/modules/alert_definitions/host/host_alert_score_anomaly.lua @@ -81,18 +81,13 @@ function host_alert_score_anomaly.format(ifid, alert, alert_type_params) local is_both = alert_type_params["is_both"] local role local host = alert_consts.formatHostAlert(ifid, alert["ip"], alert["vlan_id"]) - local sec_cat = 0 - local net_cat = 0 if(is_both) then role = i18n("client_and_server") - net_cat, sec_cat = get_problematic_category(alert_type_params, true) elseif(is_client_alert) then role = i18n("client") - net_cat, sec_cat = get_problematic_category(alert_type_params, nil, "client") else role = i18n("server") - net_cat, sec_cat = get_problematic_category(alert_type_params, nil, "server") end return i18n("alert_messages.score_number_anomaly", { @@ -101,8 +96,6 @@ function host_alert_score_anomaly.format(ifid, alert, alert_type_params) score = alert_type_params["value"], lower_bound = alert_type_params["lower_bound"], upper_bound = alert_type_params["upper_bound"], - network = net_cat, - security = sec_cat, }) end diff --git a/scripts/lua/modules/alert_exclusions.lua b/scripts/lua/modules/alert_exclusions.lua index 1c2de59bf8..71dcbca403 100644 --- a/scripts/lua/modules/alert_exclusions.lua +++ b/scripts/lua/modules/alert_exclusions.lua @@ -316,6 +316,22 @@ end -- ############################################## +-- @brief Import a previously `export`ed exclusions configuration +function alert_exclusions.import(exclusions) + _set_configured_alert_exclusions(exclusions) +end + +-- ############################################## + +-- @brief Exports the current configuration +function alert_exclusions.export() + local exclusions = _get_configured_alert_exclusions() + + return exclusions +end + +-- ############################################## + -- @brief Delete all alert_exclusions function alert_exclusions.cleanup() local locked = _lock() diff --git a/scripts/lua/modules/alert_utils.lua b/scripts/lua/modules/alert_utils.lua index 25097a77f7..03a73626fb 100644 --- a/scripts/lua/modules/alert_utils.lua +++ b/scripts/lua/modules/alert_utils.lua @@ -777,4 +777,28 @@ function alert_utils.notify_ntopng_stop() return(notify_ntopng_status(false)) end +function alert_utils.formatBehaviorAlert(params, anomalies, stats, id, subtype) + -- Cycle throught the behavior stats + for anomaly_type, anomaly in pairs(anomalies) do + local alert = alert_consts.alert_types.alert_behavior_anomaly.new( + i18n(subtype .. "_id", {id = id}), + i18n("alert_behaviors." .. anomaly_type), + stats[anomaly_type]["value"], + stats[anomaly_type]["lower_bound"], + stats[anomaly_type]["upper_bound"] + ) + + alert:set_score_warning() + alert:set_granularity(params.granularity) + alert:set_subtype(subtype .. "_" .. id) + + -- Trigger an alert if an anomaly is found + if anomaly == true then + alert:trigger(params.alert_entity, nil, params.cur_alerts) + else + alert:release(params.alert_entity, nil, params.cur_alerts) + end + end +end + return alert_utils diff --git a/scripts/lua/modules/import_export/scripts_import_export.lua b/scripts/lua/modules/import_export/checks_import_export.lua similarity index 73% rename from scripts/lua/modules/import_export/scripts_import_export.lua rename to scripts/lua/modules/import_export/checks_import_export.lua index 6ebfca16a2..250c0fe08d 100644 --- a/scripts/lua/modules/import_export/scripts_import_export.lua +++ b/scripts/lua/modules/import_export/checks_import_export.lua @@ -8,28 +8,29 @@ require "lua_utils" local import_export = require "import_export" local json = require "dkjson" local checks = require "checks" +local alert_exclusions = require "alert_exclusions" local rest_utils = require "rest_utils" -- ############################################## -local scripts_import_export = {} +local checks_import_export = {} -- ############################################## -function scripts_import_export:create(args) +function checks_import_export:create(args) -- Instance of the base class - local _scripts_import_export = import_export:create() + local _checks_import_export = import_export:create() -- Subclass using the base class instance self.key = "scripts" -- self is passed as argument so it will be set as base class metatable -- and this will actually make it possible to override functions - local _scripts_import_export_instance = _scripts_import_export:create(self) + local _checks_import_export_instance = _checks_import_export:create(self) -- Compute -- Return the instance - return _scripts_import_export_instance + return _checks_import_export_instance end -- ############################################## @@ -37,7 +38,7 @@ end -- @brief Import configuration -- @param conf The configuration to be imported -- @return A table with a key "success" set to true is returned on success. A key "err" is set in case of failure, with one of the errors defined in rest_utils.consts.err. -function scripts_import_export:import(conf) +function checks_import_export:import(conf) local res = {} if table.empty(conf) then @@ -45,7 +46,7 @@ function scripts_import_export:import(conf) return res end - local config_set = conf["0"] + local config_set = conf[tostring(checks.DEFAULT_CONFIGSET_ID)] if config_set == nil then res.err = rest_utils.consts.err.bad_content @@ -62,6 +63,12 @@ function scripts_import_export:import(conf) end if not res.err then + + -- Import exclusions (if present) + if conf["alert_exclusions"] then + alert_exclusions.import(conf["alert_exclusions"]) + end + res.success = true end @@ -72,10 +79,11 @@ end -- @brief Export configuration -- @return The current configuration -function scripts_import_export:export() +function checks_import_export:export() local conf = {} conf[checks.DEFAULT_CONFIGSET_ID] = checks.getConfigset() + conf["alert_exclusions"] = alert_exclusions.export() return conf end @@ -83,10 +91,10 @@ end -- ############################################## -- @brief Reset configuration -function scripts_import_export:reset() +function checks_import_export:reset() checks.resetConfigset() end -- ############################################## -return scripts_import_export +return checks_import_export diff --git a/scripts/lua/rest/v1/export/scripts/config.lua b/scripts/lua/rest/v1/export/checks/config.lua similarity index 87% rename from scripts/lua/rest/v1/export/scripts/config.lua rename to scripts/lua/rest/v1/export/checks/config.lua index 0cf3ae2bc3..83e8ff1e22 100644 --- a/scripts/lua/rest/v1/export/scripts/config.lua +++ b/scripts/lua/rest/v1/export/checks/config.lua @@ -7,7 +7,7 @@ package.path = dirs.installdir .. "/scripts/lua/modules/import_export/?.lua;" .. require "lua_utils" -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local json = require "dkjson" local rest_utils = require "rest_utils" local import_export_rest_utils = require "import_export_rest_utils" @@ -27,6 +27,6 @@ if not haveAdminPrivileges() then end local instances = {} -instances["scripts"] = scripts_import_export:create() +instances["scripts"] = checks_import_export:create() import_export_rest_utils.export(instances, not isEmptyString(download)) diff --git a/scripts/lua/rest/v1/export/pool/config.lua b/scripts/lua/rest/v1/export/pool/config.lua index 0a010b3ae1..3524333106 100644 --- a/scripts/lua/rest/v1/export/pool/config.lua +++ b/scripts/lua/rest/v1/export/pool/config.lua @@ -11,7 +11,7 @@ local snmp_import_export = require "snmp_import_export" local plugins_utils = require("plugins_utils") local am_import_export = plugins_utils.loadModule("active_monitoring", "am_import_export") local notifications_import_export = require "notifications_import_export" -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local pool_import_export = require "pool_import_export" local json = require "dkjson" local rest_utils = require "rest_utils" @@ -36,7 +36,7 @@ local instances = {} instances["snmp"] = snmp_import_export:create() instances["active_monitoring"] = am_import_export:create() instances["notifications"] = notifications_import_export:create() -instances["scripts"] = scripts_import_export:create() +instances["scripts"] = checks_import_export:create() instances["pool"] = pool_import_export:create() import_export_rest_utils.export(instances, not isEmptyString(download)) diff --git a/scripts/lua/rest/v1/get/scripts/.dontdelete b/scripts/lua/rest/v1/get/checks/.dontdelete similarity index 100% rename from scripts/lua/rest/v1/get/scripts/.dontdelete rename to scripts/lua/rest/v1/get/checks/.dontdelete diff --git a/scripts/lua/rest/v1/get/scripts/config.lua b/scripts/lua/rest/v1/get/checks/config.lua similarity index 86% rename from scripts/lua/rest/v1/get/scripts/config.lua rename to scripts/lua/rest/v1/get/checks/config.lua index bb6e4f3c70..96fe83ef09 100644 --- a/scripts/lua/rest/v1/get/scripts/config.lua +++ b/scripts/lua/rest/v1/get/checks/config.lua @@ -9,7 +9,7 @@ require "lua_utils" local info = ntop.getInfo() -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local json = require ("dkjson") local page_utils = require("page_utils") local format_utils = require("format_utils") @@ -30,8 +30,8 @@ if not haveAdminPrivileges() then return end -local scripts_import_export = scripts_import_export:create() -local res = scripts_import_export:export() +local checks_import_export = checks_import_export:create() +local res = checks_import_export:export() if isEmptyString(download) then rest_utils.answer(rest_utils.consts.success.ok, res) diff --git a/scripts/lua/rest/v1/import/scripts/config.lua b/scripts/lua/rest/v1/import/checks/config.lua similarity index 90% rename from scripts/lua/rest/v1/import/scripts/config.lua rename to scripts/lua/rest/v1/import/checks/config.lua index 500e69a20b..d6b4da23bd 100644 --- a/scripts/lua/rest/v1/import/scripts/config.lua +++ b/scripts/lua/rest/v1/import/checks/config.lua @@ -7,7 +7,7 @@ package.path = dirs.installdir .. "/scripts/lua/modules/import_export/?.lua;" .. require "lua_utils" -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local json = require "dkjson" local rest_utils = require "rest_utils" local import_export_rest_utils = require "import_export_rest_utils" @@ -39,7 +39,7 @@ end local items = {} -local scripts_ie = scripts_import_export:create() +local scripts_ie = checks_import_export:create() items[#items+1] = { name = "scripts", conf = modules["scripts"], diff --git a/scripts/lua/rest/v1/import/pool/config.lua b/scripts/lua/rest/v1/import/pool/config.lua index 8a7c53bdf2..d1100d3a05 100644 --- a/scripts/lua/rest/v1/import/pool/config.lua +++ b/scripts/lua/rest/v1/import/pool/config.lua @@ -11,7 +11,7 @@ local snmp_import_export = require "snmp_import_export" local plugins_utils = require("plugins_utils") local am_import_export = plugins_utils.loadModule("active_monitoring", "am_import_export") local notifications_import_export = require "notifications_import_export" -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local pool_import_export = require "pool_import_export" local json = require "dkjson" local rest_utils = require "rest_utils" @@ -75,7 +75,7 @@ items[#items+1] = { instance = notifications_ie } -local scripts_ie = scripts_import_export:create() +local scripts_ie = checks_import_export:create() items[#items+1] = { name = "scripts", conf = modules["scripts"], diff --git a/scripts/lua/rest/v1/reset/scripts/config.lua b/scripts/lua/rest/v1/reset/checks/config.lua similarity index 87% rename from scripts/lua/rest/v1/reset/scripts/config.lua rename to scripts/lua/rest/v1/reset/checks/config.lua index 6fb3c61f0e..99afa9c2ce 100644 --- a/scripts/lua/rest/v1/reset/scripts/config.lua +++ b/scripts/lua/rest/v1/reset/checks/config.lua @@ -7,7 +7,7 @@ package.path = dirs.installdir .. "/scripts/lua/modules/import_export/?.lua;" .. require "lua_utils" -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local json = require "dkjson" local rest_utils = require "rest_utils" local import_export_rest_utils = require "import_export_rest_utils" @@ -26,6 +26,6 @@ if not auth.has_capability(auth.capabilities.checks) then end local instances = {} -instances["scripts"] = scripts_import_export:create() +instances["scripts"] = checks_import_export:create() import_export_rest_utils.reset(instances) diff --git a/scripts/lua/rest/v1/reset/pool/config.lua b/scripts/lua/rest/v1/reset/pool/config.lua index 17a0cdddf3..c8ef7572ba 100644 --- a/scripts/lua/rest/v1/reset/pool/config.lua +++ b/scripts/lua/rest/v1/reset/pool/config.lua @@ -11,7 +11,7 @@ local snmp_import_export = require "snmp_import_export" local plugins_utils = require("plugins_utils") local am_import_export = plugins_utils.loadModule("active_monitoring", "am_import_export") local notifications_import_export = require "notifications_import_export" -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local pool_import_export = require "pool_import_export" local json = require "dkjson" local rest_utils = require "rest_utils" diff --git a/scripts/lua/rest/v1/set/scripts/.dontdelete b/scripts/lua/rest/v1/set/checks/.dontdelete similarity index 100% rename from scripts/lua/rest/v1/set/scripts/.dontdelete rename to scripts/lua/rest/v1/set/checks/.dontdelete diff --git a/scripts/lua/rest/v1/set/scripts/config.lua b/scripts/lua/rest/v1/set/checks/config.lua similarity index 87% rename from scripts/lua/rest/v1/set/scripts/config.lua rename to scripts/lua/rest/v1/set/checks/config.lua index 60c8c1d651..ba648bf762 100644 --- a/scripts/lua/rest/v1/set/scripts/config.lua +++ b/scripts/lua/rest/v1/set/checks/config.lua @@ -9,7 +9,7 @@ require "lua_utils" local info = ntop.getInfo() -local scripts_import_export = require "scripts_import_export" +local checks_import_export = require "checks_import_export" local json = require ("dkjson") local page_utils = require("page_utils") local format_utils = require("format_utils") @@ -37,8 +37,8 @@ end local data = json.decode(_POST["JSON"]) -local scripts_import_export = scripts_import_export:create() -local res = scripts_import_export:import(data) +local checks_import_export = checks_import_export:create() +local res = checks_import_export:import(data) if res.err then rest_utils.answer(res.err) diff --git a/scripts/lua/vlan_details.lua b/scripts/lua/vlan_details.lua index c27305d1ba..9d8675451f 100644 --- a/scripts/lua/vlan_details.lua +++ b/scripts/lua/vlan_details.lua @@ -34,7 +34,7 @@ if vlan_id == nil or tonumber(vlan_id) == nil or tonumber(vlan_id) == 0 then return end -if(not areVlanTimeseriesEnabled(ifId)) then +if(not areVlanTimeseriesEnabled(ifId)) and (page ~= "config") then print("
" .. i18n("vlan_details.no_available_stats_for_vlan_message",{vlan_id=vlan_id, product=info["product"]}).."
") dofile(dirs.installdir .. "/scripts/lua/inc/footer.lua") return @@ -94,7 +94,7 @@ else return end print[[ -
+ ]] @@ -120,7 +120,7 @@ else

]] diff --git a/src/Flow.cpp b/src/Flow.cpp index 91c8aba7d3..95eee199b8 100644 --- a/src/Flow.cpp +++ b/src/Flow.cpp @@ -132,7 +132,7 @@ Flow::Flow(NetworkInterface *_iface, memset(&custom_app, 0, sizeof(custom_app)); #ifdef NTOPNG_PRO - lateral_movement = create_or_delete = false; + lateral_movement = false; HostPools *hp = iface->getHostPools(); routing_table_id = DEFAULT_ROUTING_TABLE_ID; diff --git a/src/ParserInterface.cpp b/src/ParserInterface.cpp index ea74e24f82..c4de7546ff 100755 --- a/src/ParserInterface.cpp +++ b/src/ParserInterface.cpp @@ -289,11 +289,16 @@ bool ParserInterface::processFlow(ParsedFlow *zflow) { /* There's a breadown between client and server TCP flags */ if(zflow->tcp.client_tcp_flags) flow->updateTcpFlags(&now_tv, zflow->tcp.client_tcp_flags, src2dst_direction); + if(zflow->tcp.server_tcp_flags) flow->updateTcpFlags(&now_tv, zflow->tcp.server_tcp_flags, !src2dst_direction); - } else if(zflow->tcp.tcp_flags) { - /* TCP flags are cumulated */ - flow->updateTcpFlags(&now_tv, zflow->tcp.tcp_flags, src2dst_direction); + + if(zflow->tcp.tcp_flags + && (zflow->tcp.client_tcp_flags == 0) + && (zflow->tcp.server_tcp_flags == 0)) { + /* TCP flags are cumulative and set only if client/server flags are zero */ + flow->updateTcpFlags(&now_tv, zflow->tcp.tcp_flags, src2dst_direction); + } } flow->updateTcpSeqIssues(zflow); diff --git a/src/ZMQParserInterface.cpp b/src/ZMQParserInterface.cpp index a22860a4a7..96444822c7 100755 --- a/src/ZMQParserInterface.cpp +++ b/src/ZMQParserInterface.cpp @@ -638,10 +638,12 @@ bool ZMQParserInterface::parsePENNtopField(ParsedFlow * const flow, u_int32_t fi case CLIENT_TCP_FLAGS: flow->tcp.client_tcp_flags = value->int_num; + flow->tcp.tcp_flags |= flow->tcp.client_tcp_flags; break; case SERVER_TCP_FLAGS: flow->tcp.server_tcp_flags = value->int_num; + flow->tcp.tcp_flags |= flow->tcp.server_tcp_flags; break; case APPL_LATENCY_MS: diff --git a/src/flow_alerts/LateralMovementAlert.cpp b/src/flow_alerts/LateralMovementAlert.cpp deleted file mode 100644 index 158255acd0..0000000000 --- a/src/flow_alerts/LateralMovementAlert.cpp +++ /dev/null @@ -1,38 +0,0 @@ -/* - * - * (C) 2013-21 - ntop.org - * - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write to the Free Software Foundation, - * Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. - * - */ - -#include "flow_checks_includes.h" - -/* ***************************************************** */ - -ndpi_serializer *LateralMovementAlert::getAlertJSON(ndpi_serializer* serializer) { - Flow *f = getFlow(); - Host *cli = f->get_cli_host(), *srv = f->get_srv_host(); - char buf[128]; - - if(serializer == NULL) - return NULL; - - ndpi_serialize_string_boolean(serializer, "create_or_delete", f->isCreateOrDelete()); - - return serializer; -} - diff --git a/tests/rest/result/alert_flow_invalid_dns_query_01.out b/tests/rest/result/alert_flow_invalid_dns_query_01.out index cbd18f8997..0b669f126f 100644 --- a/tests/rest/result/alert_flow_invalid_dns_query_01.out +++ b/tests/rest/result/alert_flow_invalid_dns_query_01.out @@ -1,3 +1,3 @@ {"success":true} {"rc":0,"rc_str":"OK","rc_str_hr":"Success","rsp":{"success":true}} -{"rc":0,"rc_str":"OK","rc_str_hr":"Success","recordsFiltered":1,"recordsTotal":1,"rsp":{"records":[{"additional_alerts":"Invalid DNS query [Score: 10]","alert_id":{"label":" Suspicious DGA Domain","value":"47"},"alert_name":" Suspicious DGA Domain","cli_role":{"label":"Is Attacker","tag_label":"Has Attacker","value":"attacker"},"count":1,"description":"Suspicious DGA Domain [asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.sfadsfadfasdfasdfadsf.adsfadfasdfasdfadsfadsf.adfasdfasdfadsf.adsfadfasdfasdfadsfaasdf.google.it]","duration":0,"family":"flow","flow":{"active_url":"/lua/flow_details.lua?flow_key=3383892960&flow_hash_id=0","cli_ip":{"label":"192.168.2.222","reference":"","value":"192.168.2.222"},"cli_port":"59610","historical_url":"","srv_ip":{"label":"9.9.9.9","reference":"","value":"9.9.9.9"},"srv_port":"53"},"l7_proto":{"label":"UDP:DNS","value":"5"},"msg":{"configset_ref":" ","description":"Suspicious DGA Domain [asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.sfadsfadfasdfasdfadsf.adsfadfasdfasdfadsfadsf.adfasdfasdfadsf.adsfadfasdfasdfadsfaasdf.google.it]","name":"Suspicious DGA Domain","value":47},"proto":{"label":"UDP","value":"17"},"row_id":"1","score":{"color":"#ff3231","label":"110","value":110},"script_key":"ndpi_suspicious_dga_domain","severity":{"color":"#ff3231","label":" ","value":5},"tstamp":{"highlight":"#ff3231","label":"17:27:09","value":1623857229},"vlan_id":"0"}]}} +{"rc":0,"rc_str":"OK","rc_str_hr":"Success","recordsFiltered":1,"recordsTotal":1,"rsp":{"records":[{"additional_alerts":"","alert_id":{"label":" Invalid DNS query","value":"7"},"alert_name":" Invalid DNS query","cli_role":{"label":"Is Attacker","tag_label":"Has Attacker","value":"attacker"},"count":1,"description":"Invalid DNS query","duration":0,"family":"flow","flow":{"active_url":"/lua/flow_details.lua?flow_key=3383892960&flow_hash_id=0","cli_ip":{"label":"192.168.2.222","reference":"","value":"192.168.2.222"},"cli_port":"59610","historical_url":"","srv_ip":{"label":"9.9.9.9","reference":"","value":"9.9.9.9"},"srv_port":"53"},"l7_proto":{"label":"UDP:DNS","value":"5"},"msg":{"configset_ref":" ","description":"","name":"Invalid DNS query","value":7},"proto":{"label":"UDP","value":"17"},"row_id":"1","score":{"color":"#5cd65c","label":"10","value":10},"script_key":"invalid_dns_query","severity":{"color":"#5cd65c","label":" ","value":3},"tstamp":{"highlight":"#5cd65c","label":"13:06:37","value":1623927997},"vlan_id":"0"}]}} diff --git a/tests/rest/result/alert_flow_invalid_dns_query_02.out b/tests/rest/result/alert_flow_invalid_dns_query_02.out index e1e82d1417..39dd59e56a 100644 --- a/tests/rest/result/alert_flow_invalid_dns_query_02.out +++ b/tests/rest/result/alert_flow_invalid_dns_query_02.out @@ -1,2 +1,2 @@ {"success":true} -{"rc":0,"rc_str":"OK","rc_str_hr":"Success","recordsFiltered":1,"recordsTotal":1,"rsp":{"records":[{"additional_alerts":"Unexpected DNS server found [Score: 100], Invalid DNS query [Score: 10], Remote to Remote [Score: 10]","alert_id":{"label":" Suspicious DGA Domain","value":"47"},"alert_name":" Suspicious DGA Domain","cli_role":{"label":"Is Attacker","tag_label":"Has Attacker","value":"attacker"},"count":1,"description":"Suspicious DGA Domain [asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.sfadsfadfasdfasdfadsf.adsfadfasdfasdfadsfadsf.adfasdfasdfadsf.adsfadfasdfasdfadsfaasdf.google.it]","duration":0,"family":"flow","flow":{"active_url":"/lua/flow_details.lua?flow_key=3383892960&flow_hash_id=0","cli_ip":{"label":"192.168.2.222","reference":"","value":"192.168.2.222"},"cli_port":"59610","historical_url":"","srv_ip":{"label":"9.9.9.9","reference":"","value":"9.9.9.9"},"srv_port":"53"},"l7_proto":{"label":"UDP:DNS","value":"5"},"msg":{"configset_ref":" ","description":"Suspicious DGA Domain [asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.asdfadsfadsfadfasdfasdfad.sfadsfadfasdfasdfadsf.adsfadfasdfasdfadsfadsf.adfasdfasdfadsf.adsfadfasdfasdfadsfaasdf.google.it]","name":"Suspicious DGA Domain","value":47},"proto":{"label":"UDP","value":"17"},"row_id":"1","score":{"color":"#ff3231","label":"220","value":220},"script_key":"ndpi_suspicious_dga_domain","severity":{"color":"#ff3231","label":" ","value":5},"tstamp":{"highlight":"#ff3231","label":"17:27:30","value":1623857250},"vlan_id":"0"}]}} +{"rc":0,"rc_str":"OK","rc_str_hr":"Success","recordsFiltered":1,"recordsTotal":1,"rsp":{"records":[{"additional_alerts":"Invalid DNS query [Score: 10], Remote to Remote [Score: 10]","alert_id":{"label":" Unexpected DNS server found","value":"33"},"alert_name":" Unexpected DNS server found","cli_role":{"label":"Is Attacker","tag_label":"Has Attacker","value":"attacker"},"count":1,"description":"Unexpected DNS server found: 9.9.9.9","duration":0,"family":"flow","flow":{"active_url":"/lua/flow_details.lua?flow_key=3383892960&flow_hash_id=0","cli_ip":{"label":"192.168.2.222","reference":"","value":"192.168.2.222"},"cli_port":"59610","historical_url":"","srv_ip":{"label":"9.9.9.9","reference":"","value":"9.9.9.9"},"srv_port":"53"},"l7_proto":{"label":"UDP:DNS","value":"5"},"msg":{"configset_ref":" ","description":"Unexpected DNS server found: 9.9.9.9","name":"Unexpected DNS server found","value":33},"proto":{"label":"UDP","value":"17"},"row_id":"1","score":{"color":"#ff3231","label":"120","value":120},"script_key":"unexpected_dns","severity":{"color":"#ff3231","label":" ","value":5},"tstamp":{"highlight":"#ff3231","label":"13:06:59","value":1623928019},"vlan_id":"0"}]}} diff --git a/tests/rest/result/get_alert_type_consts_01.out b/tests/rest/result/get_alert_type_consts_01.out index 797d12856c..f68972daee 100644 --- a/tests/rest/result/get_alert_type_consts_01.out +++ b/tests/rest/result/get_alert_type_consts_01.out @@ -1 +1 @@ -{"rc":0,"rc_str":"OK","rc_str_hr":"Success","rsp":[{"key":4166,"type":"alert_excessive_traffic"},{"key":9,"type":"host_alert_dns_traffic"},{"key":8,"type":"host_alert_p2p_traffic"},{"key":7,"type":"alert_dns_invalid_query"},{"key":13,"type":"host_alert_host_log"},{"key":51,"type":"alert_ndpi_tls_suspicious_esni_usage"},{"key":6,"type":"alert_dns_data_exfiltration"},{"key":37,"type":"alert_iec_invalid_transition"},{"key":4145,"type":"alert_list_download_succeeded"},{"key":57,"type":"alert_remote_access"},{"key":20,"type":"alert_tcp_connection_refused"},{"key":32,"type":"alert_unexpected_dhcp_server"},{"key":12,"type":"host_alert_remote_connection"},{"key":43,"type":"alert_ndpi_http_numeric_ip_host"},{"key":4116,"type":"alert_nfq_flushed"},{"key":4142,"type":"alert_attack_mitigation_via_snmp"},{"key":49,"type":"alert_ndpi_ssh_obsolete"},{"key":1,"type":"alert_flow_blacklisted"},{"key":54,"type":"alert_ndpi_tls_missing_sni"},{"key":55,"type":"alert_iec_unexpected_type_id"},{"key":12,"type":"alert_flow_low_goodput"},{"key":16,"type":"alert_remote_to_remote"},{"key":4114,"type":"alert_misconfigured_app"},{"key":25,"type":"alert_tls_unsafe_ciphers"},{"key":38,"type":"alert_remote_to_local_insecure_proto"},{"key":4164,"type":"alert_broadcast_domain_too_large"},{"key":4111,"type":"alert_login_failed"},{"key":50,"type":"alert_ndpi_smb_insecure_version"},{"key":42,"type":"alert_ndpi_http_suspicious_user_agent"},{"key":31,"type":"alert_flow_risk"},{"key":14,"type":"alert_internals"},{"key":4147,"type":"alert_unexpected_new_device"},{"key":4125,"type":"alert_process_notification"},{"key":26,"type":"alert_udp_unidirectional"},{"key":53,"type":"alert_ndpi_dns_suspicious_traffic"},{"key":4149,"type":"alert_periodicity_update"},{"key":4126,"type":"alert_quota_exceeded"},{"key":11,"type":"alert_longlived"},{"key":40,"type":"alert_ndpi_url_possible_sql_injection"},{"key":4110,"type":"alert_list_download_failed"},{"key":4121,"type":"alert_port_errors"},{"key":4097,"type":"alert_device_connection"},{"key":2,"type":"host_alert_dns_server_contacts"},{"key":4144,"type":"alert_lateral_movement"},{"key":13,"type":"alert_malicious_signature"},{"key":4128,"type":"alert_slow_periodic_activity"},{"key":2,"type":"alert_blacklisted_country"},{"key":8,"type":"alert_elephant_flow"},{"key":33,"type":"alert_unexpected_dns_server"},{"key":3,"type":"alert_flow_blocked"},{"key":4130,"type":"alert_snmp_device_reset"},{"key":14,"type":"host_alert_dangerous_host"},{"key":22,"type":"alert_tls_certificate_expired"},{"key":4165,"type":"alert_ngi_trust_event"},{"key":4137,"type":"alert_too_many_drops"},{"key":11,"type":"host_alert_score_anomaly"},{"key":23,"type":"alert_tls_certificate_mismatch"},{"key":5,"type":"host_alert_tcp_syn_scan"},{"key":4112,"type":"alert_mac_ip_association_change"},{"key":46,"type":"alert_ndpi_tls_not_carrying_https"},{"key":4123,"type":"alert_port_mac_changed"},{"key":4119,"type":"alert_am_threshold_cross"},{"key":4153,"type":"alert_flow_flood_victim"},{"key":39,"type":"alert_ndpi_url_possible_xss"},{"key":24,"type":"alert_tls_old_protocol_version"},{"key":4157,"type":"alert_tcp_syn_scan_victim"},{"key":4159,"type":"alert_contacts_anomaly"},{"key":4124,"type":"alert_port_status_change"},{"key":4103,"type":"alert_host_pool_connection"},{"key":52,"type":"alert_ndpi_unsafe_protocol"},{"key":4155,"type":"alert_tcp_syn_flood_victim"},{"key":19,"type":"alert_connection_issues"},{"key":4122,"type":"alert_port_load_threshold_exceeded"},{"key":4148,"type":"alert_shell_script_executed"},{"key":4106,"type":"alert_influxdb_error"},{"key":41,"type":"alert_ndpi_url_possible_rce_injection"},{"key":15,"type":"alert_potentially_dangerous_protocol"},{"key":4131,"type":"alert_snmp_topology_changed"},{"key":29,"type":"alert_suspicious_file_transfer"},{"key":4104,"type":"alert_host_pool_disconnection"},{"key":4146,"type":"alert_no_if_activity"},{"key":4098,"type":"alert_device_disconnection"},{"key":4136,"type":"alert_threshold_cross"},{"key":4120,"type":"alert_port_duplexstatus_change"},{"key":4139,"type":"alert_user_activity"},{"key":35,"type":"alert_unexpected_ntp_server"},{"key":5,"type":"alert_device_protocol_not_allowed"},{"key":4118,"type":"alert_periodic_activity_not_executed"},{"key":4129,"type":"alert_slow_purge"},{"key":6,"type":"host_alert_tcp_syn_flood"},{"key":1,"type":"host_alert_smtp_server_contacts"},{"key":56,"type":"alert_flow_tcp_no_data_exchanged"},{"key":10,"type":"host_alert_flow_anomaly"},{"key":4,"type":"host_alert_flow_flood"},{"key":36,"type":"alert_zero_tcp_window"},{"key":10,"type":"alert_external"},{"key":4102,"type":"alert_ghost_network"},{"key":44,"type":"alert_ndpi_http_suspicious_url"},{"key":4107,"type":"alert_influxdb_export_failure"},{"key":27,"type":"alert_web_mining"},{"key":34,"type":"alert_unexpected_smtp_server"},{"key":48,"type":"alert_ndpi_malformed_packet"},{"key":4109,"type":"alert_ip_outsite_dhcp_range"},{"key":4151,"type":"alert_fail2ban_executed"},{"key":4099,"type":"alert_dropped_alerts"},{"key":3,"type":"host_alert_ntp_server_contacts"},{"key":28,"type":"alert_tls_certificate_selfsigned"},{"key":47,"type":"alert_ndpi_suspicious_dga_domain"},{"key":4,"type":"alert_data_exfiltration"},{"key":45,"type":"alert_ndpi_http_suspicious_header"},{"key":30,"type":"alert_known_proto_on_non_std_port"}]} +{"rc":0,"rc_str":"OK","rc_str_hr":"Success","rsp":[{"key":12,"type":"alert_flow_low_goodput"},{"key":13,"type":"alert_malicious_signature"},{"key":4145,"type":"alert_list_download_succeeded"},{"key":9,"type":"host_alert_dns_traffic"},{"key":5,"type":"host_alert_tcp_syn_scan"},{"key":27,"type":"alert_web_mining"},{"key":14,"type":"host_alert_dangerous_host"},{"key":2,"type":"host_alert_dns_server_contacts"},{"key":53,"type":"alert_ndpi_dns_suspicious_traffic"},{"key":2,"type":"alert_blacklisted_country"},{"key":24,"type":"alert_tls_old_protocol_version"},{"key":41,"type":"alert_ndpi_url_possible_rce_injection"},{"key":34,"type":"alert_unexpected_smtp_server"},{"key":8,"type":"host_alert_p2p_traffic"},{"key":4121,"type":"alert_port_errors"},{"key":29,"type":"alert_suspicious_file_transfer"},{"key":4147,"type":"alert_unexpected_new_device"},{"key":4123,"type":"alert_port_mac_changed"},{"key":14,"type":"alert_internals"},{"key":19,"type":"alert_connection_issues"},{"key":22,"type":"alert_tls_certificate_expired"},{"key":4126,"type":"alert_quota_exceeded"},{"key":4137,"type":"alert_too_many_drops"},{"key":4112,"type":"alert_mac_ip_association_change"},{"key":47,"type":"alert_ndpi_suspicious_dga_domain"},{"key":4110,"type":"alert_list_download_failed"},{"key":4098,"type":"alert_device_disconnection"},{"key":20,"type":"alert_tcp_connection_refused"},{"key":42,"type":"alert_ndpi_http_suspicious_user_agent"},{"key":3,"type":"host_alert_ntp_server_contacts"},{"key":35,"type":"alert_unexpected_ntp_server"},{"key":11,"type":"host_alert_score_anomaly"},{"key":58,"type":"alert_lateral_movement"},{"key":15,"type":"alert_potentially_dangerous_protocol"},{"key":4129,"type":"alert_slow_purge"},{"key":4120,"type":"alert_port_duplexstatus_change"},{"key":50,"type":"alert_ndpi_smb_insecure_version"},{"key":4099,"type":"alert_dropped_alerts"},{"key":4165,"type":"alert_ngi_trust_event"},{"key":37,"type":"alert_iec_invalid_transition"},{"key":38,"type":"alert_remote_to_local_insecure_proto"},{"key":4151,"type":"alert_fail2ban_executed"},{"key":4153,"type":"alert_flow_flood_victim"},{"key":10,"type":"alert_external"},{"key":30,"type":"alert_known_proto_on_non_std_port"},{"key":54,"type":"alert_ndpi_tls_missing_sni"},{"key":4157,"type":"alert_tcp_syn_scan_victim"},{"key":57,"type":"alert_remote_access"},{"key":45,"type":"alert_ndpi_http_suspicious_header"},{"key":4106,"type":"alert_influxdb_error"},{"key":4131,"type":"alert_snmp_topology_changed"},{"key":56,"type":"alert_flow_tcp_no_data_exchanged"},{"key":33,"type":"alert_unexpected_dns_server"},{"key":26,"type":"alert_udp_unidirectional"},{"key":4114,"type":"alert_misconfigured_app"},{"key":4164,"type":"alert_broadcast_domain_too_large"},{"key":4111,"type":"alert_login_failed"},{"key":4107,"type":"alert_influxdb_export_failure"},{"key":4166,"type":"alert_excessive_traffic"},{"key":4128,"type":"alert_slow_periodic_activity"},{"key":4,"type":"alert_data_exfiltration"},{"key":3,"type":"alert_flow_blocked"},{"key":51,"type":"alert_ndpi_tls_suspicious_esni_usage"},{"key":4136,"type":"alert_threshold_cross"},{"key":4159,"type":"alert_contacts_anomaly"},{"key":4097,"type":"alert_device_connection"},{"key":4155,"type":"alert_tcp_syn_flood_victim"},{"key":39,"type":"alert_ndpi_url_possible_xss"},{"key":4130,"type":"alert_snmp_device_reset"},{"key":4146,"type":"alert_no_if_activity"},{"key":28,"type":"alert_tls_certificate_selfsigned"},{"key":4103,"type":"alert_host_pool_connection"},{"key":52,"type":"alert_ndpi_unsafe_protocol"},{"key":1,"type":"alert_flow_blacklisted"},{"key":4104,"type":"alert_host_pool_disconnection"},{"key":4124,"type":"alert_port_status_change"},{"key":4139,"type":"alert_user_activity"},{"key":4122,"type":"alert_port_load_threshold_exceeded"},{"key":4118,"type":"alert_periodic_activity_not_executed"},{"key":11,"type":"alert_longlived"},{"key":8,"type":"alert_elephant_flow"},{"key":23,"type":"alert_tls_certificate_mismatch"},{"key":49,"type":"alert_ndpi_ssh_obsolete"},{"key":4148,"type":"alert_shell_script_executed"},{"key":4149,"type":"alert_periodicity_update"},{"key":5,"type":"alert_device_protocol_not_allowed"},{"key":4116,"type":"alert_nfq_flushed"},{"key":4142,"type":"alert_attack_mitigation_via_snmp"},{"key":4125,"type":"alert_process_notification"},{"key":4109,"type":"alert_ip_outsite_dhcp_range"},{"key":4119,"type":"alert_am_threshold_cross"},{"key":4102,"type":"alert_ghost_network"},{"key":1,"type":"host_alert_smtp_server_contacts"},{"key":4,"type":"host_alert_flow_flood"},{"key":12,"type":"host_alert_remote_connection"},{"key":10,"type":"host_alert_flow_anomaly"},{"key":13,"type":"host_alert_host_log"},{"key":7,"type":"alert_dns_invalid_query"},{"key":46,"type":"alert_ndpi_tls_not_carrying_https"},{"key":16,"type":"alert_remote_to_remote"},{"key":31,"type":"alert_flow_risk"},{"key":44,"type":"alert_ndpi_http_suspicious_url"},{"key":4167,"type":"alert_behavior_anomaly"},{"key":48,"type":"alert_ndpi_malformed_packet"},{"key":6,"type":"host_alert_tcp_syn_flood"},{"key":55,"type":"alert_iec_unexpected_type_id"},{"key":6,"type":"alert_dns_data_exfiltration"},{"key":25,"type":"alert_tls_unsafe_ciphers"},{"key":40,"type":"alert_ndpi_url_possible_sql_injection"},{"key":36,"type":"alert_zero_tcp_window"},{"key":32,"type":"alert_unexpected_dhcp_server"},{"key":43,"type":"alert_ndpi_http_numeric_ip_host"}]} diff --git a/tests/rest/result/get_scripts_config_01.out b/tests/rest/result/get_scripts_config_01.out index 813b45fa15..c9036231d2 100644 --- a/tests/rest/result/get_scripts_config_01.out +++ b/tests/rest/result/get_scripts_config_01.out @@ -1 +1 @@ -{"rc":0,"rc_str":"OK","rc_str_hr":"Success","rsp":{"0":{"config":{"flow":{"blacklisted":{"all":{"enabled":true,"script_conf":[]}},"country_check":{"all":{"enabled":false,"script_conf":{"items":[]}}},"data_exfiltration":{"all":{"enabled":true,"script_conf":[]}},"device_protocol_not_allowed":{"all":{"enabled":true,"script_conf":[]}},"dns_data_exfiltration":{"all":{"enabled":true,"script_conf":[]}},"elephant_flows":{"all":{"enabled":true,"script_conf":{"l2r_bytes_value":1073741824,"r2l_bytes_value":1073741824}}},"external_alert_check":{"all":{"enabled":true,"script_conf":[]}},"iec_invalid_transition":{"all":{"enabled":true,"script_conf":[]}},"iec_unexpected_type_id":{"all":{"enabled":true,"script_conf":{"items":[9,13,36,45,46,48,30,103,100,37]}}},"invalid_dns_query":{"all":{"enabled":true,"script_conf":{"items":["sophosxl.net"]}}},"known_proto_on_non_std_port":{"all":{"enabled":true,"script_conf":[]}},"long_lived":{"all":{"enabled":true,"script_conf":{"min_duration":43200}}},"low_goodput":{"all":{"enabled":true,"script_conf":[]}},"ndpi_dns_suspicious_traffic":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_numeric_ip_host":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_suspicious_header":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_suspicious_url":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_suspicious_user_agent":{"all":{"enabled":true,"script_conf":[]}},"ndpi_malformed_packet":{"all":{"enabled":true,"script_conf":[]}},"ndpi_smb_insecure_version":{"all":{"enabled":true,"script_conf":[]}},"ndpi_ssh_obsolete":{"all":{"enabled":true,"script_conf":[]}},"ndpi_suspicious_dga_domain":{"all":{"enabled":true,"script_conf":[]}},"ndpi_tls_missing_sni":{"all":{"enabled":true,"script_conf":[]}},"ndpi_tls_not_carrying_https":{"all":{"enabled":true,"script_conf":[]}},"ndpi_tls_suspicious_esni_usage":{"all":{"enabled":true,"script_conf":[]}},"ndpi_unsafe_protocol":{"all":{"enabled":true,"script_conf":[]}},"ndpi_url_possible_rce_injection":{"all":{"enabled":true,"script_conf":[]}},"ndpi_url_possible_sql_injection":{"all":{"enabled":true,"script_conf":[]}},"ndpi_url_possible_xss":{"all":{"enabled":true,"script_conf":[]}},"nedge_blocked":{"all":{"enabled":true,"script_conf":[]}},"not_purged":{"all":{"enabled":true,"script_conf":[]}},"potentially_dangerous":{"all":{"enabled":true,"script_conf":[]}},"remote_access":{"all":{"enabled":true,"script_conf":[]}},"remote_to_local_insecure_proto":{"all":{"enabled":true,"script_conf":[]}},"remote_to_remote":{"all":{"enabled":false,"script_conf":[]}},"suspicious_file_transfer":{"all":{"enabled":true,"script_conf":[]}},"suspicious_tcp_probing":{"all":{"enabled":true,"script_conf":[]}},"suspicious_tcp_syn_probing":{"all":{"enabled":true,"script_conf":[]}},"tcp_connection_refused":{"all":{"enabled":true,"script_conf":[]}},"tcp_issues_generic":{"all":{"enabled":true,"script_conf":[]}},"tcp_no_data_exchanged":{"all":{"enabled":true,"script_conf":[]}},"tls_certificate_expired":{"all":{"enabled":true,"script_conf":[]}},"tls_certificate_mismatch":{"all":{"enabled":true,"script_conf":[]}},"tls_certificate_selfsigned":{"all":{"enabled":true,"script_conf":[]}},"tls_malicious_signature":{"all":{"enabled":true,"script_conf":[]}},"tls_old_protocol_version":{"all":{"enabled":true,"script_conf":[]}},"tls_unsafe_ciphers":{"all":{"enabled":true,"script_conf":[]}},"udp_unidirectional":{"all":{"enabled":true,"script_conf":[]}},"unexpected_dhcp":{"all":{"enabled":false,"script_conf":{"items":[]}}},"unexpected_dns":{"all":{"enabled":false,"script_conf":{"items":[]}}},"unexpected_ntp":{"all":{"enabled":false,"script_conf":{"items":[]}}},"unexpected_smtp":{"all":{"enabled":false,"script_conf":{"items":[]}}},"web_mining":{"all":{"enabled":true,"script_conf":{"items":[]}}},"zero_tcp_window":{"all":{"enabled":true,"script_conf":[]}}},"host":{"dangerous_host":{"min":{"enabled":true,"script_conf":{"operator":"gt","threshold":"1000"}}},"dns":{"min":{"enabled":false,"script_conf":{"operator":"gt"}}},"dns_contacts":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":5}}},"flow_flood":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":256}}},"flows_anomaly":{"min":{"enabled":true,"script_conf":[]}},"ntp_contacts":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":5}}},"p2p":{"min":{"enabled":false,"script_conf":{"operator":"gt"}}},"remote_connection":{"min":{"enabled":false,"script_conf":[]}},"score_anomaly":{"min":{"enabled":true,"script_conf":[]}},"smtp_contacts":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":5}}},"syn_flood":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":256}}},"syn_scan":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":256}}}},"interface":{"dropped_alerts":{"min":{"enabled":true,"script_conf":[]}},"excessive_traffic":{"day":{"enabled":false,"script_conf":[]}},"ghost_networks":{"min":{"enabled":true,"script_conf":[]}},"lateral_movement":{"min":{"enabled":true,"script_conf":[]}},"no_if_activity":{"min":{"enabled":true,"script_conf":[]}},"periodic_activity_not_executed":{"min":{"enabled":true,"script_conf":[]}},"periodicity_update":{"min":{"enabled":true,"script_conf":[]}},"pool_quota_exceeded":{"min":{"enabled":false,"script_conf":[]}},"slow_periodic_activity":{"min":{"enabled":true,"script_conf":[]}},"slow_purge":{"min":{"enabled":true,"script_conf":{"operator":"gt","threshold":50}}},"too_many_drops":{"min":{"enabled":true,"script_conf":{"operator":"gt","threshold":5}}},"unexpected_new_device":{"min":{"enabled":false,"script_conf":{"items":[]}}}},"network":{"egress":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":1073741824}}},"flow_flood_victim":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":32768}}},"ingress":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":1073741824}}},"inner":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":1073741824}}},"syn_flood_victim":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":32768}}},"syn_scan_victim":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":32768}}}},"snmp_device":{"lldp_topology_changed":{"all":{"enabled":false,"script_conf":[]}},"snmp_device_interface_discards_errors":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_interface_duplex_status_change":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_interface_load":{"all":{"enabled":true,"script_conf":{"operator":"gt","threshold":99}}},"snmp_device_interface_mac_changed":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_interface_status_change":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_reset":{"all":{"enabled":true,"script_conf":[]}}},"syslog":{"fortinet":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"host_log":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"ngi_trust":{"all":{"enabled":true,"script_conf":[]}},"openvpn":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"opnsense":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"sonicwall":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"sophos":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"suricata":{"all":{"enabled":true,"script_conf":[]}}},"system":{"active_monitoring":{"5mins":{"enabled":true,"script_conf":[]},"hour":{"enabled":true,"script_conf":[]},"min":{"enabled":true,"script_conf":[]}},"alerts_ts":{"min":{"enabled":true,"script_conf":[]}},"disk_monitor":{"hour":{"enabled":true,"script_conf":[]}},"dropped_alerts":{"min":{"enabled":true,"script_conf":[]}},"influxdb_monitor":{"5mins":{"enabled":true,"script_conf":[]},"min":{"enabled":true,"script_conf":[]}},"memory_ts":{"min":{"enabled":true,"script_conf":[]}},"periodic_activity_not_executed":{"min":{"enabled":true,"script_conf":[]}},"redis_monitor":{"min":{"enabled":true,"script_conf":[]}},"slow_periodic_activity":{"min":{"enabled":true,"script_conf":[]}}}},"filters":{"flow":{"elephant_flows":[{"l7_cat":"DataTransfer"},{"l7_cat":"Cloud"},{"l7_cat":"SoftwareUpdate"},{"l7_cat":"Streaming"},{"l7_cat":"VPN"}],"long_lived":[{"l7_cat":"Database"}],"low_goodput":[{"l7_proto":8},{"l7_proto":26},{"l7_proto":39},{"l7_proto":48},{"l7_proto":65},{"l7_proto":77},{"l7_proto":92},{"l7_proto":142},{"l7_proto":185},{"l7_proto":193},{"l7_proto":197}]}}}}} +{"rc":0,"rc_str":"OK","rc_str_hr":"Success","rsp":{"0":{"config":{"flow":{"blacklisted":{"all":{"enabled":true,"script_conf":[]}},"country_check":{"all":{"enabled":false,"script_conf":{"items":[]}}},"data_exfiltration":{"all":{"enabled":true,"script_conf":[]}},"device_protocol_not_allowed":{"all":{"enabled":true,"script_conf":[]}},"dns_data_exfiltration":{"all":{"enabled":true,"script_conf":[]}},"elephant_flows":{"all":{"enabled":true,"script_conf":{"l2r_bytes_value":1073741824,"r2l_bytes_value":1073741824}}},"external_alert_check":{"all":{"enabled":true,"script_conf":[]}},"iec_invalid_transition":{"all":{"enabled":true,"script_conf":[]}},"iec_unexpected_type_id":{"all":{"enabled":true,"script_conf":{"items":[9,13,36,45,46,48,30,103,100,37]}}},"invalid_dns_query":{"all":{"enabled":true,"script_conf":{"items":["sophosxl.net"]}}},"known_proto_on_non_std_port":{"all":{"enabled":true,"script_conf":[]}},"lateral_movement":{"all":{"enabled":true,"script_conf":[]}},"long_lived":{"all":{"enabled":true,"script_conf":{"min_duration":43200}}},"low_goodput":{"all":{"enabled":true,"script_conf":[]}},"ndpi_dns_suspicious_traffic":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_numeric_ip_host":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_suspicious_header":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_suspicious_url":{"all":{"enabled":true,"script_conf":[]}},"ndpi_http_suspicious_user_agent":{"all":{"enabled":true,"script_conf":[]}},"ndpi_malformed_packet":{"all":{"enabled":true,"script_conf":[]}},"ndpi_smb_insecure_version":{"all":{"enabled":true,"script_conf":[]}},"ndpi_ssh_obsolete":{"all":{"enabled":true,"script_conf":[]}},"ndpi_suspicious_dga_domain":{"all":{"enabled":true,"script_conf":[]}},"ndpi_tls_missing_sni":{"all":{"enabled":true,"script_conf":[]}},"ndpi_tls_not_carrying_https":{"all":{"enabled":true,"script_conf":[]}},"ndpi_tls_suspicious_esni_usage":{"all":{"enabled":true,"script_conf":[]}},"ndpi_unsafe_protocol":{"all":{"enabled":true,"script_conf":[]}},"ndpi_url_possible_rce_injection":{"all":{"enabled":true,"script_conf":[]}},"ndpi_url_possible_sql_injection":{"all":{"enabled":true,"script_conf":[]}},"ndpi_url_possible_xss":{"all":{"enabled":true,"script_conf":[]}},"nedge_blocked":{"all":{"enabled":true,"script_conf":[]}},"not_purged":{"all":{"enabled":true,"script_conf":[]}},"potentially_dangerous":{"all":{"enabled":true,"script_conf":[]}},"remote_access":{"all":{"enabled":true,"script_conf":[]}},"remote_to_local_insecure_proto":{"all":{"enabled":true,"script_conf":[]}},"remote_to_remote":{"all":{"enabled":false,"script_conf":[]}},"suspicious_file_transfer":{"all":{"enabled":true,"script_conf":[]}},"suspicious_tcp_probing":{"all":{"enabled":true,"script_conf":[]}},"suspicious_tcp_syn_probing":{"all":{"enabled":true,"script_conf":[]}},"tcp_connection_refused":{"all":{"enabled":true,"script_conf":[]}},"tcp_issues_generic":{"all":{"enabled":true,"script_conf":[]}},"tcp_no_data_exchanged":{"all":{"enabled":true,"script_conf":[]}},"tls_certificate_expired":{"all":{"enabled":true,"script_conf":[]}},"tls_certificate_mismatch":{"all":{"enabled":true,"script_conf":[]}},"tls_certificate_selfsigned":{"all":{"enabled":true,"script_conf":[]}},"tls_malicious_signature":{"all":{"enabled":true,"script_conf":[]}},"tls_old_protocol_version":{"all":{"enabled":true,"script_conf":[]}},"tls_unsafe_ciphers":{"all":{"enabled":true,"script_conf":[]}},"udp_unidirectional":{"all":{"enabled":true,"script_conf":[]}},"unexpected_dhcp":{"all":{"enabled":false,"script_conf":{"items":[]}}},"unexpected_dns":{"all":{"enabled":false,"script_conf":{"items":[]}}},"unexpected_ntp":{"all":{"enabled":false,"script_conf":{"items":[]}}},"unexpected_smtp":{"all":{"enabled":false,"script_conf":{"items":[]}}},"web_mining":{"all":{"enabled":true,"script_conf":{"items":[]}}},"zero_tcp_window":{"all":{"enabled":true,"script_conf":[]}}},"host":{"dangerous_host":{"min":{"enabled":true,"script_conf":{"operator":"gt","threshold":"1000"}}},"dns":{"min":{"enabled":false,"script_conf":{"operator":"gt"}}},"dns_contacts":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":5}}},"flow_flood":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":256}}},"flows_anomaly":{"min":{"enabled":true,"script_conf":[]}},"ntp_contacts":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":5}}},"p2p":{"min":{"enabled":false,"script_conf":{"operator":"gt"}}},"remote_connection":{"min":{"enabled":false,"script_conf":[]}},"score_anomaly":{"min":{"enabled":true,"script_conf":[]}},"smtp_contacts":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":5}}},"syn_flood":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":256}}},"syn_scan":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":256}}}},"interface":{"dropped_alerts":{"min":{"enabled":true,"script_conf":[]}},"excessive_traffic":{"day":{"enabled":false,"script_conf":[]}},"ghost_networks":{"min":{"enabled":true,"script_conf":[]}},"no_if_activity":{"min":{"enabled":true,"script_conf":[]}},"periodic_activity_not_executed":{"min":{"enabled":true,"script_conf":[]}},"periodicity_update":{"min":{"enabled":true,"script_conf":[]}},"pool_quota_exceeded":{"min":{"enabled":false,"script_conf":[]}},"slow_periodic_activity":{"min":{"enabled":true,"script_conf":[]}},"slow_purge":{"min":{"enabled":true,"script_conf":{"operator":"gt","threshold":50}}},"too_many_drops":{"min":{"enabled":true,"script_conf":{"operator":"gt","threshold":5}}},"unexpected_new_device":{"min":{"enabled":false,"script_conf":{"items":[]}}}},"network":{"egress":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":1073741824}}},"flow_flood_victim":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":32768}}},"ingress":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":1073741824}}},"inner":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":1073741824}}},"syn_flood_victim":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":32768}}},"syn_scan_victim":{"min":{"enabled":false,"script_conf":{"operator":"gt","threshold":32768}}}},"snmp_device":{"lldp_topology_changed":{"all":{"enabled":false,"script_conf":[]}},"snmp_device_interface_discards_errors":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_interface_duplex_status_change":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_interface_load":{"all":{"enabled":true,"script_conf":{"operator":"gt","threshold":99}}},"snmp_device_interface_mac_changed":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_interface_status_change":{"all":{"enabled":true,"script_conf":[]}},"snmp_device_reset":{"all":{"enabled":true,"script_conf":[]}}},"syslog":{"fortinet":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"host_log":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"ngi_trust":{"all":{"enabled":true,"script_conf":[]}},"openvpn":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"opnsense":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"sonicwall":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"sophos":{"all":{"enabled":true,"script_conf":{"operator":"lt","threshold":5}}},"suricata":{"all":{"enabled":true,"script_conf":[]}}},"system":{"active_monitoring":{"5mins":{"enabled":true,"script_conf":[]},"hour":{"enabled":true,"script_conf":[]},"min":{"enabled":true,"script_conf":[]}},"alerts_ts":{"min":{"enabled":true,"script_conf":[]}},"disk_monitor":{"hour":{"enabled":true,"script_conf":[]}},"dropped_alerts":{"min":{"enabled":true,"script_conf":[]}},"influxdb_monitor":{"5mins":{"enabled":true,"script_conf":[]},"min":{"enabled":true,"script_conf":[]}},"memory_ts":{"min":{"enabled":true,"script_conf":[]}},"periodic_activity_not_executed":{"min":{"enabled":true,"script_conf":[]}},"redis_monitor":{"min":{"enabled":true,"script_conf":[]}},"slow_periodic_activity":{"min":{"enabled":true,"script_conf":[]}}}},"filters":{"flow":{"elephant_flows":[{"l7_cat":"DataTransfer"},{"l7_cat":"Cloud"},{"l7_cat":"SoftwareUpdate"},{"l7_cat":"Streaming"},{"l7_cat":"VPN"}],"long_lived":[{"l7_cat":"Database"}],"low_goodput":[{"l7_proto":8},{"l7_proto":26},{"l7_proto":39},{"l7_proto":48},{"l7_proto":65},{"l7_proto":77},{"l7_proto":92},{"l7_proto":142},{"l7_proto":185},{"l7_proto":193},{"l7_proto":197}]}}}}}