Output of flowgrind

After starting a test, flowgrind first prints a preamble with the date of the test, the number of flows as well as the reporting interval length and the throughput measurement unit. Last but not least the preamble contains the version number of flowgrind which is important to reproduce and understand the results of the test should the capabilities of flowgrind change in future versions.

While the test runs the controller collects the interval reports from the daemons participating the test and prints the report lines. It is not visible in this example output, but flowgrind does not make any guarantees on the order the reports from different flows or endpoints are printed. Only the reports from the same flow and endpoint are shown ordered. The reason is that gathering and displaying the results is done asynchronously. This needs to be taken into consideration when parsing the output.

During the test, a header with the column names and units is displayed periodically. The columns are resized dynamically to make best use of the available space. This can be seen in the sample output, at the second time the header gets displayed the width of the cwnd column changes. If desired, groups of columns can be disabled from showing by a configuration option. The columns seen in the figure show all the metrics measured by flowgrind:

#: The endpoint, either S for sender or R for receiver.
ID: The flow identifier number.
begin/end: The boundaries of the report interval in seconds.
through: The transmitting throughput of the flow endpoint during this report interval, can be display either in Mbps (default) or Mb/s. The slightly lower figures for incoming traffic on receiver side compared to outgoing traffic on sender side in the final report line originate in the fact that the data which is in flight during the connection termination has been successfully sent, but not received.
transac: The Network Transactions per second.
requ/resp: The number of successful written request and response blocks in this report interval. This column is not shown by default because it is redundant to the transac column.
IAT/RTT: The application layer IAT and RTT as described in \autoref{minisec:RelatedWorks:PerformanceMetrics:Delay. For both delays the minimum and maximum encountered values in that interval are displayed in addition to the arithmetic mean. If no corresponding block arrives during the report interval, inf is displayed.

The following variables are extracted through the TCP_INFO socket options at the end of every reporting interval and represent the internal state of the Linux TCP stack; accordingly these are only available in Linux. Also the meaning can differ in newer versions of Linux.

cwnd: Size of TCP congestion window in number of segments.
ssth: The slow start threshold of the sender in number of segments. Arbitrary numbers are optionally replaced by their symbolic name to enhanced readability. Linux for example initializes the SSTHRESH with INT_MAX ($2147483647$).
uack/sack: Number of unacknowledged and selectively acknowledged segments.
lost: Number of segments assumed lost at the end of the report interval.
retr/tret: Number of all retransmissions (retr) and timeout based retransmissions (tret).
fack: Number of segments between SND.UNA and the highest selectively acknowledged sequence number.
reor: Segment reordering metric. The Linux kernel can detect and cope with reordering without loss of performance if the distance a segment gets displaced does not exceed the reordering metric.
bkof: Number of consecutive exponential backoffs in the current recovery phase.
rtt/rttvar: TCP round-trip time estimation and its variance given in ms.
rto: The retransmission timeout given in ms.
ca state: Internal state of the congestion control state machine as implemented in the Linux kernel. Can be one of open, disorder, cwr, recovery or loss:

Open is the normal state. It indicates that there are no issues with the connection.
Disorder is similar to Open but is entered upon receiving duplicate ACKs or selective acknowledgments as special attention might be needed in the near future.
CWR is entered when the size of the congestion window got lowered due to receiving an ICMP Source Quench message or a notification from ECN.
Recovery indicates that the congestion window got lowered and a segment is fast-retransmitted.
Loss is entered if the RTO expires. Again the size of the congestion window got lowered in this state.

smss: The sender maximum segment size, the size of the largest segment the sender can transmit.
pmtu: The Path MTU, the largest possible MTU along the path between sender and receiver.

After the test finishes, a final report line header is displayed for each flow endpoint. Flowgrind tries to guess the used network type from the value of the MSS and MTU, in this case it correctly identified the used network as Ethernet. The final report line also contains the size of the send buffer and the size of the receive buffer, the advertised initial window. For comparison the size of the data blocks is displayed as well for each direction. Afterwards the initial delay and the flow duration are shown. Following these scheduling options some information about the used socket options may be shown, though no special options were configured for this simple test. Afterwards the average throughput is shown and the number of data blocks sent and received is printed.

Following the final report headers the flow metrics for the entire test duration are displayed. In the final report the columns containing information returned by the TCP_INFO socket option show the values measured just at the time the final report was taken, right before the socket gets destroyed.