mirror of
https://github.com/ntop/ntopng.git
synced 2026-04-28 15:09:33 +00:00
100 lines
3.2 KiB
Text
100 lines
3.2 KiB
Text
Introduction
|
|
------------
|
|
You can instruct ntopng to save flow information in Elasticsearch so that you can
|
|
create your own reports using an "ENK" stack (Elasticsearch-ntopng-Kibana)
|
|
|
|
|
|
How To Enable It
|
|
----------------
|
|
In order to enable this feature you need to start ntopng with the -F.
|
|
|
|
Example:
|
|
$ ntopng -F "es;ntopng;ntopng-%Y.%m.%d;http://localhost:9200/_bulk;"
|
|
|
|
Syntax:
|
|
es;<index type>;<index name>;<ES Bulk API>;{optional username};{optional pass};
|
|
|
|
Note: the <idx name> accepts the strftime() format.
|
|
|
|
ntopng will create new indexes automatically so no action is required from your side.
|
|
ntopng will NOT create the mappings automatically - a template should be created.
|
|
|
|
Example:
|
|
$ curl -XPUT localhost:9200/_template/ntopng -d '
|
|
{
|
|
"template" : "ntopng-*",
|
|
"settings" : {
|
|
"index.refresh_interval" : "5s"
|
|
},
|
|
"mappings" : {
|
|
"_default_" : {
|
|
"_all" : {"enabled" : true, "omit_norms" : true},
|
|
"dynamic_templates" : [ {
|
|
"string_fields" : {
|
|
"match" : "*",
|
|
"match_mapping_type" : "string",
|
|
"mapping" : {
|
|
"type" : "string", "index" : "analyzed", "omit_norms" : true,
|
|
"fields" : {
|
|
"raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
|
|
}
|
|
}
|
|
}
|
|
}, {
|
|
"geo_fields" : {
|
|
"match" : "*_IP_LOCATION",
|
|
"mapping": {
|
|
"type": "geo_point"
|
|
}
|
|
}
|
|
}, {
|
|
"ip_fields" : {
|
|
"match" : "IPV4_*",
|
|
"match_mapping_type" : "string",
|
|
"mapping": {
|
|
"type": "ip"
|
|
}
|
|
}
|
|
} ],
|
|
"properties" : {
|
|
"@version": { "type": "string", "index": "not_analyzed" }
|
|
}
|
|
}
|
|
}
|
|
}'
|
|
|
|
|
|
Using It
|
|
--------
|
|
Enabling this option will cause ntopng dump flow information in Elasticsearch
|
|
and allows the user/integrator to recreate statistics and monitoring logic using
|
|
Kibana or other ES Clients in parallel ntopng's own interface.
|
|
|
|
Once started, ntopng will push to ES flows that are expired or periodically send
|
|
(every 5 mins) partial flows for long lasting flows.
|
|
|
|
ES Information is currently not used back by ntopng, integration with historical
|
|
views is planned in upcoming versions.
|
|
|
|
|
|
Data Retention
|
|
--------------
|
|
Flows from ntopng typically use a date-stamped index (ie ntopng-%Y.%m.%d ).
|
|
You can use Elasticsearch Curator to delete indexes over the retention period.
|
|
|
|
Example:
|
|
$ curator --host localhost delete indices --older-than 30 /
|
|
--time-unit days --timestring '%Y.%m.%d'
|
|
|
|
Indexes can also be manually removed during testing:
|
|
$ curl -XDELETE 'http://localhost:9200/ntopng-2015.09.27/'
|
|
|
|
To remove all Indexes generated by ntopng (according to standard formats):
|
|
$ curl -XDELETE 'http://localhost:9200/ntopng-*/'
|
|
|
|
Future Work
|
|
-----------
|
|
In the future we plan:
|
|
|
|
- adding automated mapping and rotation of ES indexes
|
|
- adding ElasticSearch support to historical view of ntopng
|