mirror of
https://github.com/ntop/ntopng.git
synced 2026-07-29 17:50:59 +00:00
597 lines
32 KiB
HTML
597 lines
32 KiB
HTML
<?xml version="1.0" ?>
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<title>rrdcreate</title>
|
|
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
|
|
<link rev="made" href="mailto:root@localhost" />
|
|
</head>
|
|
|
|
<body style="background-color: white">
|
|
|
|
|
|
<!-- INDEX BEGIN -->
|
|
<div name="index">
|
|
<p><a name="__index__"></a></p>
|
|
<!--
|
|
|
|
<ul>
|
|
|
|
<li><a href="#name">NAME</a></li>
|
|
<li><a href="#synopsis">SYNOPSIS</a></li>
|
|
<li><a href="#description">DESCRIPTION</a></li>
|
|
<ul>
|
|
|
|
<li><a href="#filename"><em>filename</em></a></li>
|
|
<li><a href="#__start__b_start_time__default__now___10s_"><strong>-start</strong>|<strong>-b</strong> <em>start time</em> (default: now - 10s)</a></li>
|
|
<li><a href="#__step__s_step__default__300_seconds_"><strong>-step</strong>|<strong>-s</strong> <em>step</em> (default: 300 seconds)</a></li>
|
|
<li><a href="#__no_overwrite"><strong>-no-overwrite</strong></a></li>
|
|
<li><a href="#ds_ds_name_dst_dst_arguments"><strong>DS:</strong><em>ds-name</em><strong>:</strong><em>DST</em><strong>:</strong><em>dst arguments</em></a></li>
|
|
<li><a href="#rra_cf_cf_arguments"><strong>RRA:</strong><em>CF</em><strong>:</strong><em>cf arguments</em></a></li>
|
|
</ul>
|
|
|
|
<li><a href="#aberrant_behavior_detection_with_holt_winters_forecasting">Aberrant Behavior Detection with Holt-Winters Forecasting</a></li>
|
|
<li><a href="#the_heartbeat_and_the_step">The HEARTBEAT and the STEP</a></li>
|
|
<li><a href="#how_to_measure">HOW TO MEASURE</a></li>
|
|
<li><a href="#example">EXAMPLE</a></li>
|
|
<li><a href="#example_2">EXAMPLE 2</a></li>
|
|
<li><a href="#example_3">EXAMPLE 3</a></li>
|
|
<li><a href="#author">AUTHOR</a></li>
|
|
</ul>
|
|
|
|
-->
|
|
|
|
|
|
</div>
|
|
<!-- INDEX END -->
|
|
|
|
<p>
|
|
</p>
|
|
<h1><a name="name">NAME</a></h1>
|
|
<p>rrdcreate - Set up a new Round Robin Database</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="synopsis">SYNOPSIS</a></h1>
|
|
<p><strong>rrdtool</strong> <strong>create</strong> <em>filename</em>
|
|
[<strong>--start</strong>|<strong>-b</strong> <em>start time</em>]
|
|
[<strong>--step</strong>|<strong>-s</strong> <em>step</em>]
|
|
[<strong>--no-overwrite</strong>]
|
|
[<strong>DS:</strong><em>ds-name</em><strong>:</strong><em>DST</em><strong>:</strong><em>dst arguments</em>]
|
|
[<strong>RRA:</strong><em>CF</em><strong>:</strong><em>cf arguments</em>]</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="description">DESCRIPTION</a></h1>
|
|
<p>The create function of RRDtool lets you set up new Round Robin
|
|
Database (<strong>RRD</strong>) files. The file is created at its final, full size
|
|
and filled with <em>*UNKNOWN*</em> data.</p>
|
|
<p>
|
|
</p>
|
|
<h2><a name="filename"><em>filename</em></a></h2>
|
|
<p>The name of the <strong>RRD</strong> you want to create. <strong>RRD</strong> files should end
|
|
with the extension <em class="file">.rrd</em>. However, <strong>RRDtool</strong> will accept any
|
|
filename.</p>
|
|
<p>
|
|
</p>
|
|
<h2><a name="__start__b_start_time__default__now___10s_"><strong>--start</strong>|<strong>-b</strong> <em>start time</em> (default: now - 10s)</a></h2>
|
|
<p>Specifies the time in seconds since 1970-01-01 UTC when the first
|
|
value should be added to the <strong>RRD</strong>. <strong>RRDtool</strong> will not accept
|
|
any data timed before or at the time specified.</p>
|
|
<p>See also AT-STYLE TIME SPECIFICATION section in the
|
|
<em>rrdfetch</em> documentation for other ways to specify time.</p>
|
|
<p>
|
|
</p>
|
|
<h2><a name="__step__s_step__default__300_seconds_"><strong>--step</strong>|<strong>-s</strong> <em>step</em> (default: 300 seconds)</a></h2>
|
|
<p>Specifies the base interval in seconds with which data will be fed
|
|
into the <strong>RRD</strong>.</p>
|
|
<p>
|
|
</p>
|
|
<h2><a name="__no_overwrite"><strong>--no-overwrite</strong></a></h2>
|
|
<p>Do not clobber an existing file of the same name.</p>
|
|
<p>
|
|
</p>
|
|
<h2><a name="ds_ds_name_dst_dst_arguments"><strong>DS:</strong><em>ds-name</em><strong>:</strong><em>DST</em><strong>:</strong><em>dst arguments</em></a></h2>
|
|
<p>A single <strong>RRD</strong> can accept input from several data sources (<strong>DS</strong>),
|
|
for example incoming and outgoing traffic on a specific communication
|
|
line. With the <strong>DS</strong> configuration option you must define some basic
|
|
properties of each data source you want to store in the <strong>RRD</strong>.</p>
|
|
<p><em>ds-name</em> is the name you will use to reference this particular data
|
|
source from an <strong>RRD</strong>. A <em>ds-name</em> must be 1 to 19 characters long in
|
|
the characters [a-zA-Z0-9_].</p>
|
|
<p><em>DST</em> defines the Data Source Type. The remaining arguments of a
|
|
data source entry depend on the data source type. For GAUGE, COUNTER,
|
|
DERIVE, and ABSOLUTE the format for a data source entry is:</p>
|
|
<p><strong>DS:</strong><em>ds-name</em><strong>:</strong><em>GAUGE | COUNTER | DERIVE | ABSOLUTE</em><strong>:</strong><em>heartbeat</em><strong>:</strong><em>min</em><strong>:</strong><em>max</em></p>
|
|
<p>For COMPUTE data sources, the format is:</p>
|
|
<p><strong>DS:</strong><em>ds-name</em><strong>:</strong><em>COMPUTE</em><strong>:</strong><em>rpn-expression</em></p>
|
|
<p>In order to decide which data source type to use, review the
|
|
definitions that follow. Also consult the section on "HOW TO MEASURE"
|
|
for further insight.</p>
|
|
<dl>
|
|
<dt><strong><a name="gauge" class="item"><strong>GAUGE</strong></a></strong></dt>
|
|
|
|
<dd>
|
|
<p>is for things like temperatures or number of people in a room or the
|
|
value of a RedHat share.</p>
|
|
</dd>
|
|
<dt><strong><a name="counter" class="item"><strong>COUNTER</strong></a></strong></dt>
|
|
|
|
<dd>
|
|
<p>is for continuous incrementing counters like the ifInOctets counter in
|
|
a router. The <strong>COUNTER</strong> data source assumes that the counter never
|
|
decreases, except when a counter overflows. The update function takes
|
|
the overflow into account. The counter is stored as a per-second
|
|
rate. When the counter overflows, RRDtool checks if the overflow
|
|
happened at the 32bit or 64bit border and acts accordingly by adding
|
|
an appropriate value to the result.</p>
|
|
</dd>
|
|
<dt><strong><a name="derive" class="item"><strong>DERIVE</strong></a></strong></dt>
|
|
|
|
<dd>
|
|
<p>will store the derivative of the line going from the last to the
|
|
current value of the data source. This can be useful for gauges, for
|
|
example, to measure the rate of people entering or leaving a
|
|
room. Internally, derive works exactly like COUNTER but without
|
|
overflow checks. So if your counter does not reset at 32 or 64 bit you
|
|
might want to use DERIVE and combine it with a MIN value of 0.</p>
|
|
<p><strong>NOTE on COUNTER vs DERIVE</strong></p>
|
|
<p>by Don Baarda <<a href="mailto:don.baarda@baesystems.com">don.baarda@baesystems.com</a>></p>
|
|
<p>If you cannot tolerate ever mistaking the occasional counter reset for a
|
|
legitimate counter wrap, and would prefer "Unknowns" for all legitimate
|
|
counter wraps and resets, always use DERIVE with min=0. Otherwise, using
|
|
COUNTER with a suitable max will return correct values for all legitimate
|
|
counter wraps, mark some counter resets as "Unknown", but can mistake some
|
|
counter resets for a legitimate counter wrap.</p>
|
|
<p>For a 5 minute step and 32-bit counter, the probability of mistaking a
|
|
counter reset for a legitimate wrap is arguably about 0.8% per 1Mbps of
|
|
maximum bandwidth. Note that this equates to 80% for 100Mbps interfaces, so
|
|
for high bandwidth interfaces and a 32bit counter, DERIVE with min=0 is
|
|
probably preferable. If you are using a 64bit counter, just about any max
|
|
setting will eliminate the possibility of mistaking a reset for a counter
|
|
wrap.</p>
|
|
</dd>
|
|
<dt><strong><a name="absolute" class="item"><strong>ABSOLUTE</strong></a></strong></dt>
|
|
|
|
<dd>
|
|
<p>is for counters which get reset upon reading. This is used for fast counters
|
|
which tend to overflow. So instead of reading them normally you reset them
|
|
after every read to make sure you have a maximum time available before the
|
|
next overflow. Another usage is for things you count like number of messages
|
|
since the last update.</p>
|
|
</dd>
|
|
<dt><strong><a name="compute" class="item"><strong>COMPUTE</strong></a></strong></dt>
|
|
|
|
<dd>
|
|
<p>is for storing the result of a formula applied to other data sources
|
|
in the <strong>RRD</strong>. This data source is not supplied a value on update, but
|
|
rather its Primary Data Points (PDPs) are computed from the PDPs of
|
|
the data sources according to the rpn-expression that defines the
|
|
formula. Consolidation functions are then applied normally to the PDPs
|
|
of the COMPUTE data source (that is the rpn-expression is only applied
|
|
to generate PDPs). In database software, such data sets are referred
|
|
to as "virtual" or "computed" columns.</p>
|
|
</dd>
|
|
</dl>
|
|
<p><em>heartbeat</em> defines the maximum number of seconds that may pass
|
|
between two updates of this data source before the value of the
|
|
data source is assumed to be <em>*UNKNOWN*</em>.</p>
|
|
<p><em>min</em> and <em>max</em> define the expected range values for data supplied by a
|
|
data source. If <em>min</em> and/or <em>max</em> are specified any value outside the defined range
|
|
will be regarded as <em>*UNKNOWN*</em>. If you do not know or care about min and
|
|
max, set them to U for unknown. Note that min and max always refer to the
|
|
processed values of the DS. For a traffic-<strong>COUNTER</strong> type DS this would be
|
|
the maximum and minimum data-rate expected from the device.</p>
|
|
<p><em>If information on minimal/maximal expected values is available,
|
|
always set the min and/or max properties. This will help RRDtool in
|
|
doing a simple sanity check on the data supplied when running update.</em></p>
|
|
<p><em>rpn-expression</em> defines the formula used to compute the PDPs of a
|
|
COMPUTE data source from other data sources in the same <RRD>. It is
|
|
similar to defining a <strong>CDEF</strong> argument for the graph command. Please
|
|
refer to that manual page for a list and description of RPN operations
|
|
supported. For COMPUTE data sources, the following RPN operations are
|
|
not supported: COUNT, PREV, TIME, and LTIME. In addition, in defining
|
|
the RPN expression, the COMPUTE data source may only refer to the
|
|
names of data source listed previously in the create command. This is
|
|
similar to the restriction that <strong>CDEF</strong>s must refer only to <strong>DEF</strong>s
|
|
and <strong>CDEF</strong>s previously defined in the same graph command.</p>
|
|
<p>
|
|
</p>
|
|
<h2><a name="rra_cf_cf_arguments"><strong>RRA:</strong><em>CF</em><strong>:</strong><em>cf arguments</em></a></h2>
|
|
<p>The purpose of an <strong>RRD</strong> is to store data in the round robin archives
|
|
(<strong>RRA</strong>). An archive consists of a number of data values or statistics for
|
|
each of the defined data-sources (<strong>DS</strong>) and is defined with an <strong>RRA</strong> line.</p>
|
|
<p>When data is entered into an <strong>RRD</strong>, it is first fit into time slots
|
|
of the length defined with the <strong>-s</strong> option, thus becoming a <em>primary
|
|
data point</em>.</p>
|
|
<p>The data is also processed with the consolidation function (<em>CF</em>) of
|
|
the archive. There are several consolidation functions that
|
|
consolidate primary data points via an aggregate function: <strong>AVERAGE</strong>,
|
|
<strong>MIN</strong>, <strong>MAX</strong>, <strong>LAST</strong>.</p>
|
|
<dl>
|
|
<dt><strong><a name="average" class="item">AVERAGE</a></strong></dt>
|
|
|
|
<dd>
|
|
<p>the average of the data points is stored.</p>
|
|
</dd>
|
|
<dt><strong><a name="min" class="item">MIN</a></strong></dt>
|
|
|
|
<dd>
|
|
<p>the smallest of the data points is stored.</p>
|
|
</dd>
|
|
<dt><strong><a name="max" class="item">MAX</a></strong></dt>
|
|
|
|
<dd>
|
|
<p>the largest of the data points is stored.</p>
|
|
</dd>
|
|
<dt><strong><a name="last" class="item">LAST</a></strong></dt>
|
|
|
|
<dd>
|
|
<p>the last data points is used.</p>
|
|
</dd>
|
|
</dl>
|
|
<p>Note that data aggregation inevitably leads to loss of precision and
|
|
information. The trick is to pick the aggregate function such that the
|
|
<em>interesting</em> properties of your data is kept across the aggregation
|
|
process.</p>
|
|
<p>The format of <strong>RRA</strong> line for these
|
|
consolidation functions is:</p>
|
|
<p><strong>RRA:</strong><em>AVERAGE | MIN | MAX | LAST</em><strong>:</strong><em>xff</em><strong>:</strong><em>steps</em><strong>:</strong><em>rows</em></p>
|
|
<p><em>xff</em> The xfiles factor defines what part of a consolidation interval may
|
|
be made up from <em>*UNKNOWN*</em> data while the consolidated value is still
|
|
regarded as known. It is given as the ratio of allowed <em>*UNKNOWN*</em> PDPs
|
|
to the number of PDPs in the interval. Thus, it ranges from 0 to 1 (exclusive).</p>
|
|
<p><em>steps</em> defines how many of these <em>primary data points</em> are used to build
|
|
a <em>consolidated data point</em> which then goes into the archive.</p>
|
|
<p><em>rows</em> defines how many generations of data values are kept in an <strong>RRA</strong>.
|
|
Obviously, this has to be greater than zero.</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="aberrant_behavior_detection_with_holt_winters_forecasting">Aberrant Behavior Detection with Holt-Winters Forecasting</a></h1>
|
|
<p>In addition to the aggregate functions, there are a set of specialized
|
|
functions that enable <strong>RRDtool</strong> to provide data smoothing (via the
|
|
Holt-Winters forecasting algorithm), confidence bands, and the
|
|
flagging aberrant behavior in the data source time series:</p>
|
|
<ul>
|
|
<li>
|
|
<p><strong>RRA:</strong><em>HWPREDICT</em><strong>:</strong><em>rows</em><strong>:</strong><em>alpha</em><strong>:</strong><em>beta</em><strong>:</strong><em>seasonal period</em>[<strong>:</strong><em>rra-num</em>]</p>
|
|
</li>
|
|
<li>
|
|
<p><strong>RRA:</strong><em>MHWPREDICT</em><strong>:</strong><em>rows</em><strong>:</strong><em>alpha</em><strong>:</strong><em>beta</em><strong>:</strong><em>seasonal period</em>[<strong>:</strong><em>rra-num</em>]</p>
|
|
</li>
|
|
<li>
|
|
<p><strong>RRA:</strong><em>SEASONAL</em><strong>:</strong><em>seasonal period</em><strong>:</strong><em>gamma</em><strong>:</strong><em>rra-num</em>[<strong>:smoothing-window=</strong><em>fraction</em>]</p>
|
|
</li>
|
|
<li>
|
|
<p><strong>RRA:</strong><em>DEVSEASONAL</em><strong>:</strong><em>seasonal period</em><strong>:</strong><em>gamma</em><strong>:</strong><em>rra-num</em>[<strong>:smoothing-window=</strong><em>fraction</em>]</p>
|
|
</li>
|
|
<li>
|
|
<p><strong>RRA:</strong><em>DEVPREDICT</em><strong>:</strong><em>rows</em><strong>:</strong><em>rra-num</em></p>
|
|
</li>
|
|
<li>
|
|
<p><strong>RRA:</strong><em>FAILURES</em><strong>:</strong><em>rows</em><strong>:</strong><em>threshold</em><strong>:</strong><em>window length</em><strong>:</strong><em>rra-num</em></p>
|
|
</li>
|
|
</ul>
|
|
<p>These <strong>RRAs</strong> differ from the true consolidation functions in several ways.
|
|
First, each of the <strong>RRA</strong>s is updated once for every primary data point.
|
|
Second, these <strong>RRAs</strong> are interdependent. To generate real-time confidence
|
|
bounds, a matched set of SEASONAL, DEVSEASONAL, DEVPREDICT, and either
|
|
HWPREDICT or MHWPREDICT must exist. Generating smoothed values of the primary
|
|
data points requires a SEASONAL <strong>RRA</strong> and either an HWPREDICT or MHWPREDICT
|
|
<strong>RRA</strong>. Aberrant behavior detection requires FAILURES, DEVSEASONAL, SEASONAL,
|
|
and either HWPREDICT or MHWPREDICT.</p>
|
|
<p>The predicted, or smoothed, values are stored in the HWPREDICT or MHWPREDICT
|
|
<strong>RRA</strong>. HWPREDICT and MHWPREDICT are actually two variations on the
|
|
Holt-Winters method. They are interchangeable. Both attempt to decompose data
|
|
into three components: a baseline, a trend, and a seasonal coefficient.
|
|
HWPREDICT adds its seasonal coefficient to the baseline to form a prediction, whereas
|
|
MHWPREDICT multiplies its seasonal coefficient by the baseline to form a
|
|
prediction. The difference is noticeable when the baseline changes
|
|
significantly in the course of a season; HWPREDICT will predict the seasonality
|
|
to stay constant as the baseline changes, but MHWPREDICT will predict the
|
|
seasonality to grow or shrink in proportion to the baseline. The proper choice
|
|
of method depends on the thing being modeled. For simplicity, the rest of this
|
|
discussion will refer to HWPREDICT, but MHWPREDICT may be substituted in its
|
|
place.</p>
|
|
<p>The predicted deviations are stored in DEVPREDICT (think a standard deviation
|
|
which can be scaled to yield a confidence band). The FAILURES <strong>RRA</strong> stores
|
|
binary indicators. A 1 marks the indexed observation as failure; that is, the
|
|
number of confidence bounds violations in the preceding window of observations
|
|
met or exceeded a specified threshold. An example of using these <strong>RRAs</strong> to graph
|
|
confidence bounds and failures appears in <a href="././rrdgraph.html">the rrdgraph manpage</a>.</p>
|
|
<p>The SEASONAL and DEVSEASONAL <strong>RRAs</strong> store the seasonal coefficients for the
|
|
Holt-Winters forecasting algorithm and the seasonal deviations, respectively.
|
|
There is one entry per observation time point in the seasonal cycle. For
|
|
example, if primary data points are generated every five minutes and the
|
|
seasonal cycle is 1 day, both SEASONAL and DEVSEASONAL will have 288 rows.</p>
|
|
<p>In order to simplify the creation for the novice user, in addition to
|
|
supporting explicit creation of the HWPREDICT, SEASONAL, DEVPREDICT,
|
|
DEVSEASONAL, and FAILURES <strong>RRAs</strong>, the <strong>RRDtool</strong> create command supports
|
|
implicit creation of the other four when HWPREDICT is specified alone and
|
|
the final argument <em>rra-num</em> is omitted.</p>
|
|
<p><em>rows</em> specifies the length of the <strong>RRA</strong> prior to wrap around. Remember
|
|
that there is a one-to-one correspondence between primary data points and
|
|
entries in these RRAs. For the HWPREDICT CF, <em>rows</em> should be larger than
|
|
the <em>seasonal period</em>. If the DEVPREDICT <strong>RRA</strong> is implicitly created, the
|
|
default number of rows is the same as the HWPREDICT <em>rows</em> argument. If the
|
|
FAILURES <strong>RRA</strong> is implicitly created, <em>rows</em> will be set to the <em>seasonal
|
|
period</em> argument of the HWPREDICT <strong>RRA</strong>. Of course, the <strong>RRDtool</strong>
|
|
<em>resize</em> command is available if these defaults are not sufficient and the
|
|
creator wishes to avoid explicit creations of the other specialized function
|
|
<strong>RRAs</strong>.</p>
|
|
<p><em>seasonal period</em> specifies the number of primary data points in a seasonal
|
|
cycle. If SEASONAL and DEVSEASONAL are implicitly created, this argument for
|
|
those <strong>RRAs</strong> is set automatically to the value specified by HWPREDICT. If
|
|
they are explicitly created, the creator should verify that all three
|
|
<em>seasonal period</em> arguments agree.</p>
|
|
<p><em>alpha</em> is the adaption parameter of the intercept (or baseline)
|
|
coefficient in the Holt-Winters forecasting algorithm. See <a href="././rrdtool.html">the rrdtool manpage</a> for a
|
|
description of this algorithm. <em>alpha</em> must lie between 0 and 1. A value
|
|
closer to 1 means that more recent observations carry greater weight in
|
|
predicting the baseline component of the forecast. A value closer to 0 means
|
|
that past history carries greater weight in predicting the baseline
|
|
component.</p>
|
|
<p><em>beta</em> is the adaption parameter of the slope (or linear trend) coefficient
|
|
in the Holt-Winters forecasting algorithm. <em>beta</em> must lie between 0 and 1
|
|
and plays the same role as <em>alpha</em> with respect to the predicted linear
|
|
trend.</p>
|
|
<p><em>gamma</em> is the adaption parameter of the seasonal coefficients in the
|
|
Holt-Winters forecasting algorithm (HWPREDICT) or the adaption parameter in
|
|
the exponential smoothing update of the seasonal deviations. It must lie
|
|
between 0 and 1. If the SEASONAL and DEVSEASONAL <strong>RRAs</strong> are created
|
|
implicitly, they will both have the same value for <em>gamma</em>: the value
|
|
specified for the HWPREDICT <em>alpha</em> argument. Note that because there is
|
|
one seasonal coefficient (or deviation) for each time point during the
|
|
seasonal cycle, the adaptation rate is much slower than the baseline. Each
|
|
seasonal coefficient is only updated (or adapts) when the observed value
|
|
occurs at the offset in the seasonal cycle corresponding to that
|
|
coefficient.</p>
|
|
<p>If SEASONAL and DEVSEASONAL <strong>RRAs</strong> are created explicitly, <em>gamma</em> need not
|
|
be the same for both. Note that <em>gamma</em> can also be changed via the
|
|
<strong>RRDtool</strong> <em>tune</em> command.</p>
|
|
<p><em>smoothing-window</em> specifies the fraction of a season that should be
|
|
averaged around each point. By default, the value of <em>smoothing-window</em> is
|
|
0.05, which means each value in SEASONAL and DEVSEASONAL will be occasionally
|
|
replaced by averaging it with its (<em>seasonal period</em>*0.05) nearest neighbors.
|
|
Setting <em>smoothing-window</em> to zero will disable the running-average smoother
|
|
altogether.</p>
|
|
<p><em>rra-num</em> provides the links between related <strong>RRAs</strong>. If HWPREDICT is
|
|
specified alone and the other <strong>RRAs</strong> are created implicitly, then
|
|
there is no need to worry about this argument. If <strong>RRAs</strong> are created
|
|
explicitly, then carefully pay attention to this argument. For each
|
|
<strong>RRA</strong> which includes this argument, there is a dependency between
|
|
that <strong>RRA</strong> and another <strong>RRA</strong>. The <em>rra-num</em> argument is the 1-based
|
|
index in the order of <strong>RRA</strong> creation (that is, the order they appear
|
|
in the <em>create</em> command). The dependent <strong>RRA</strong> for each <strong>RRA</strong>
|
|
requiring the <em>rra-num</em> argument is listed here:</p>
|
|
<ul>
|
|
<li>
|
|
<p>HWPREDICT <em>rra-num</em> is the index of the SEASONAL <strong>RRA</strong>.</p>
|
|
</li>
|
|
<li>
|
|
<p>SEASONAL <em>rra-num</em> is the index of the HWPREDICT <strong>RRA</strong>.</p>
|
|
</li>
|
|
<li>
|
|
<p>DEVPREDICT <em>rra-num</em> is the index of the DEVSEASONAL <strong>RRA</strong>.</p>
|
|
</li>
|
|
<li>
|
|
<p>DEVSEASONAL <em>rra-num</em> is the index of the HWPREDICT <strong>RRA</strong>.</p>
|
|
</li>
|
|
<li>
|
|
<p>FAILURES <em>rra-num</em> is the index of the DEVSEASONAL <strong>RRA</strong>.</p>
|
|
</li>
|
|
</ul>
|
|
<p><em>threshold</em> is the minimum number of violations (observed values outside
|
|
the confidence bounds) within a window that constitutes a failure. If the
|
|
FAILURES <strong>RRA</strong> is implicitly created, the default value is 7.</p>
|
|
<p><em>window length</em> is the number of time points in the window. Specify an
|
|
integer greater than or equal to the threshold and less than or equal to 28.
|
|
The time interval this window represents depends on the interval between
|
|
primary data points. If the FAILURES <strong>RRA</strong> is implicitly created, the
|
|
default value is 9.</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="the_heartbeat_and_the_step">The HEARTBEAT and the STEP</a></h1>
|
|
<p>Here is an explanation by Don Baarda on the inner workings of RRDtool.
|
|
It may help you to sort out why all this *UNKNOWN* data is popping
|
|
up in your databases:</p>
|
|
<p>RRDtool gets fed samples/updates at arbitrary times. From these it builds Primary
|
|
Data Points (PDPs) on every "step" interval. The PDPs are
|
|
then accumulated into the RRAs.</p>
|
|
<p>The "heartbeat" defines the maximum acceptable interval between
|
|
samples/updates. If the interval between samples is less than "heartbeat",
|
|
then an average rate is calculated and applied for that interval. If
|
|
the interval between samples is longer than "heartbeat", then that
|
|
entire interval is considered "unknown". Note that there are other
|
|
things that can make a sample interval "unknown", such as the rate
|
|
exceeding limits, or a sample that was explicitly marked as unknown.</p>
|
|
<p>The known rates during a PDP's "step" interval are used to calculate
|
|
an average rate for that PDP. If the total "unknown" time accounts for
|
|
more than <strong>half</strong> the "step", the entire PDP is marked
|
|
as "unknown". This means that a mixture of known and "unknown" sample
|
|
times in a single PDP "step" may or may not add up to enough "known"
|
|
time to warrant a known PDP.</p>
|
|
<p>The "heartbeat" can be short (unusual) or long (typical) relative to
|
|
the "step" interval between PDPs. A short "heartbeat" means you
|
|
require multiple samples per PDP, and if you don't get them mark the
|
|
PDP unknown. A long heartbeat can span multiple "steps", which means
|
|
it is acceptable to have multiple PDPs calculated from a single
|
|
sample. An extreme example of this might be a "step" of 5 minutes and a
|
|
"heartbeat" of one day, in which case a single sample every day will
|
|
result in all the PDPs for that entire day period being set to the
|
|
same average rate. <em>-- Don Baarda <<a href="mailto:don.baarda@baesystems.com">don.baarda@baesystems.com</a>></em></p>
|
|
<pre>
|
|
time|
|
|
axis|
|
|
begin__|00|
|
|
|01|
|
|
u|02|----* sample1, restart "hb"-timer
|
|
u|03| /
|
|
u|04| /
|
|
u|05| /
|
|
u|06|/ "hbt" expired
|
|
u|07|
|
|
|08|----* sample2, restart "hb"
|
|
|09| /
|
|
|10| /
|
|
u|11|----* sample3, restart "hb"
|
|
u|12| /
|
|
u|13| /
|
|
step1_u|14| /
|
|
u|15|/ "swt" expired
|
|
u|16|
|
|
|17|----* sample4, restart "hb", create "pdp" for step1 =
|
|
|18| / = unknown due to 10 "u" labled secs > 0.5 * step
|
|
|19| /
|
|
|20| /
|
|
|21|----* sample5, restart "hb"
|
|
|22| /
|
|
|23| /
|
|
|24|----* sample6, restart "hb"
|
|
|25| /
|
|
|26| /
|
|
|27|----* sample7, restart "hb"
|
|
step2__|28| /
|
|
|22| /
|
|
|23|----* sample8, restart "hb", create "pdp" for step1, create "cdp"
|
|
|24| /
|
|
|25| /</pre>
|
|
<p>graphics by <em><a href="mailto:vladimir.lavrov@desy.de">vladimir.lavrov@desy.de</a></em>.</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="how_to_measure">HOW TO MEASURE</a></h1>
|
|
<p>Here are a few hints on how to measure:</p>
|
|
<dl>
|
|
<dt><strong><a name="temperature" class="item">Temperature</a></strong></dt>
|
|
|
|
<dd>
|
|
<p>Usually you have some type of meter you can read to get the temperature.
|
|
The temperature is not really connected with a time. The only connection is
|
|
that the temperature reading happened at a certain time. You can use the
|
|
<strong>GAUGE</strong> data source type for this. RRDtool will then record your reading
|
|
together with the time.</p>
|
|
</dd>
|
|
<dt><strong><a name="mail_messages" class="item">Mail Messages</a></strong></dt>
|
|
|
|
<dd>
|
|
<p>Assume you have a method to count the number of messages transported by
|
|
your mail server in a certain amount of time, giving you data like '5
|
|
messages in the last 65 seconds'. If you look at the count of 5 like an
|
|
<strong>ABSOLUTE</strong> data type you can simply update the RRD with the number 5 and the
|
|
end time of your monitoring period. RRDtool will then record the number of
|
|
messages per second. If at some later stage you want to know the number of
|
|
messages transported in a day, you can get the average messages per second
|
|
from RRDtool for the day in question and multiply this number with the
|
|
number of seconds in a day. Because all math is run with Doubles, the
|
|
precision should be acceptable.</p>
|
|
</dd>
|
|
<dt><strong><a name="it_s_always_a_rate" class="item">It's always a Rate</a></strong></dt>
|
|
|
|
<dd>
|
|
<p>RRDtool stores rates in amount/second for COUNTER, DERIVE and ABSOLUTE
|
|
data. When you plot the data, you will get on the y axis
|
|
amount/second which you might be tempted to convert to an absolute
|
|
amount by multiplying by the delta-time between the points. RRDtool
|
|
plots continuous data, and as such is not appropriate for plotting
|
|
absolute amounts as for example "total bytes" sent and received in a
|
|
router. What you probably want is plot rates that you can scale to
|
|
bytes/hour, for example, or plot absolute amounts with another tool
|
|
that draws bar-plots, where the delta-time is clear on the plot for
|
|
each point (such that when you read the graph you see for example GB
|
|
on the y axis, days on the x axis and one bar for each day).</p>
|
|
</dd>
|
|
</dl>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="example">EXAMPLE</a></h1>
|
|
<pre>
|
|
rrdtool create temperature.rrd --step 300 \
|
|
DS:temp:GAUGE:600:-273:5000 \
|
|
RRA:AVERAGE:0.5:1:1200 \
|
|
RRA:MIN:0.5:12:2400 \
|
|
RRA:MAX:0.5:12:2400 \
|
|
RRA:AVERAGE:0.5:12:2400</pre>
|
|
<p>This sets up an <strong>RRD</strong> called <em class="file">temperature.rrd</em> which accepts one
|
|
temperature value every 300 seconds. If no new data is supplied for
|
|
more than 600 seconds, the temperature becomes <em>*UNKNOWN*</em>. The
|
|
minimum acceptable value is -273 and the maximum is 5'000.</p>
|
|
<p>A few archive areas are also defined. The first stores the
|
|
temperatures supplied for 100 hours (1'200 * 300 seconds = 100
|
|
hours). The second RRA stores the minimum temperature recorded over
|
|
every hour (12 * 300 seconds = 1 hour), for 100 days (2'400 hours). The
|
|
third and the fourth RRA's do the same for the maximum and
|
|
average temperature, respectively.</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="example_2">EXAMPLE 2</a></h1>
|
|
<pre>
|
|
rrdtool create monitor.rrd --step 300 \
|
|
DS:ifOutOctets:COUNTER:1800:0:4294967295 \
|
|
RRA:AVERAGE:0.5:1:2016 \
|
|
RRA:HWPREDICT:1440:0.1:0.0035:288</pre>
|
|
<p>This example is a monitor of a router interface. The first <strong>RRA</strong> tracks the
|
|
traffic flow in octets; the second <strong>RRA</strong> generates the specialized
|
|
functions <strong>RRAs</strong> for aberrant behavior detection. Note that the <em>rra-num</em>
|
|
argument of HWPREDICT is missing, so the other <strong>RRAs</strong> will implicitly be
|
|
created with default parameter values. In this example, the forecasting
|
|
algorithm baseline adapts quickly; in fact the most recent one hour of
|
|
observations (each at 5 minute intervals) accounts for 75% of the baseline
|
|
prediction. The linear trend forecast adapts much more slowly. Observations
|
|
made during the last day (at 288 observations per day) account for only
|
|
65% of the predicted linear trend. Note: these computations rely on an
|
|
exponential smoothing formula described in the LISA 2000 paper.</p>
|
|
<p>The seasonal cycle is one day (288 data points at 300 second intervals), and
|
|
the seasonal adaption parameter will be set to 0.1. The RRD file will store 5
|
|
days (1'440 data points) of forecasts and deviation predictions before wrap
|
|
around. The file will store 1 day (a seasonal cycle) of 0-1 indicators in
|
|
the FAILURES <strong>RRA</strong>.</p>
|
|
<p>The same RRD file and <strong>RRAs</strong> are created with the following command,
|
|
which explicitly creates all specialized function <strong>RRAs</strong>.</p>
|
|
<pre>
|
|
rrdtool create monitor.rrd --step 300 \
|
|
DS:ifOutOctets:COUNTER:1800:0:4294967295 \
|
|
RRA:AVERAGE:0.5:1:2016 \
|
|
RRA:HWPREDICT:1440:0.1:0.0035:288:3 \
|
|
RRA:SEASONAL:288:0.1:2 \
|
|
RRA:DEVPREDICT:1440:5 \
|
|
RRA:DEVSEASONAL:288:0.1:2 \
|
|
RRA:FAILURES:288:7:9:5</pre>
|
|
<p>Of course, explicit creation need not replicate implicit create, a
|
|
number of arguments could be changed.</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="example_3">EXAMPLE 3</a></h1>
|
|
<pre>
|
|
rrdtool create proxy.rrd --step 300 \
|
|
DS:Total:DERIVE:1800:0:U \
|
|
DS:Duration:DERIVE:1800:0:U \
|
|
DS:AvgReqDur:COMPUTE:Duration,Requests,0,EQ,1,Requests,IF,/ \
|
|
RRA:AVERAGE:0.5:1:2016</pre>
|
|
<p>This example is monitoring the average request duration during each 300 sec
|
|
interval for requests processed by a web proxy during the interval.
|
|
In this case, the proxy exposes two counters, the number of requests
|
|
processed since boot and the total cumulative duration of all processed
|
|
requests. Clearly these counters both have some rollover point, but using the
|
|
DERIVE data source also handles the reset that occurs when the web proxy is
|
|
stopped and restarted.</p>
|
|
<p>In the <strong>RRD</strong>, the first data source stores the requests per second rate
|
|
during the interval. The second data source stores the total duration of all
|
|
requests processed during the interval divided by 300. The COMPUTE data source
|
|
divides each PDP of the AccumDuration by the corresponding PDP of
|
|
TotalRequests and stores the average request duration. The remainder of the
|
|
RPN expression handles the divide by zero case.</p>
|
|
<p>
|
|
</p>
|
|
<hr />
|
|
<h1><a name="author">AUTHOR</a></h1>
|
|
<p>Tobias Oetiker <<a href="mailto:tobi@oetiker.ch">tobi@oetiker.ch</a>></p>
|
|
|
|
</body>
|
|
|
|
</html>
|