Describe flow metering.

docs/source/introduction.rst

@@ -1,13 +1,13 @@
 Introduction
 ============
 
-.. image:: asset/simplified_architecture.png
+.. image:: asset/arch.png
   :scale: 100%
   :align: center
 
 
 A step by step walk through each process involved when performing flow monitoring is
-developed in the this section. Our aim is to provide you with a reminder about how
+developed in this section. Our aim is to provide you with a reminder about how
 things works in theory. Consequently, an easier understanding of nfstream features
 and implementation is possible.
 
@@ -45,4 +45,42 @@ network data.
 having specific properties from those not having them. A packet is selected if
 some specific fields are equal or in the range of given values. Another technique is
 a hash based filtering, applying a hash function on a portion of the packet,
-the result is compared to a value or a range of values.
+the result is compared to a value or a range of values.
+
+Flow Metering
+-------------
+It includes packets aggregation into flows and flow entry expiration management.
+Second, the metering process associates a packet to a flow entry using a defined key.
+Third, it performs the aggregation of packets into flow entry based on a set of metrics.
+Then, a flow entry is cached until it is considered as terminated (entry expiration).
+Finally, optional steps such as flow sampling and filtering may be performed.
+
+**Flow Cache:** Flow cache consist of table in which the metering process stores
+information regarding active flows in the network. A flow key (typically IP source
+and destination addresses, source and destination ports, protocol and the VLAN
+identifier) determines whether a packet is matching an existing flow entry in the cache
+or not. In the first case, flow’s counters are updated. In the latter one, a new entry
+is created. Non-key fields are utilized to collect flow metrics (e.g. packets/bytes
+count, etc.). If IP addresses are part of flows key, and that traffic between two
+pairs generates flows on both directions. We define a flow as bidirectional when we consider that pair and it reverse
+belongs to same entry.The cache’s size depends on exporter device memory capacity
+and should be configured based on criteria such as key/non-key fields, maximum number
+of flows expected and expiration policy.
+
+**Entry expiration:** Cache entries are maintained in the cache table until they are
+considered as terminated. Termination of a flow is triggered by an expiration event.
+The metering process should consider an entry as expired based on:
+
+* Natural expiration: observed TCP packet belonging to a flow with FIN/RST flag.
+* Emergency expiration: flush a certain number of entries to free some space when the cache become full.
+* Active timeout: a flow entry expires after being considered active during a certain period (range from 120 seconds to 30 minutes). Counters are reset while start/end timestamp are updated.
+* Idle timeout: a flow entry expires if no packets belonging to it are observed during a specific period (range from 15 seconds to 5 minutes).
+* Resource constraints: special heuristics such as dynamic timeouts configuration at runtime.
+* Cache flush: flush of all the entries due to unexpected situations.
+
+It is possible to configure our metering process based on expiration policy to
+reduce the amount of records exported.
+
+**Flow Sampling and Filtering:** Flow sampling and filtering processes are quite like packet sampling and filtering
+process explained above. The major differences are the processed unit; while packet sampling and filtering process
+packets, flow sampling and filtering process flow records coming from the metering process