7.1. Storage Utility Plugins¶
7.1.1. CSV¶
This is a type of output storage plugin that stores the data provided to it in Comma-Separated Value (CSV) format. It is meant to be used with Converter to perform conversion of a recorded database into CSV output files. Typically it helps solve use cases like offline analysis or incident investigation. For more information about the configuration tags, see Section 5.2.9.
By default, the plug-in generates a separate csv
file for each recorded Topic.
The content and format of each file is as follows:
Row 1: Topic Entry |
|
||||
Row 2: Type Header |
|
|
|
… |
|
Row 3: Data values |
Reception timestamp of the sample (in nanoseconds) |
Value for member1 |
Value for member2 |
… |
Value for memberN |
The filename matches the topic name, with a .csv
extension. All
the files are placed in a directory that can be specified in the plug-in
configuration.
Note
If the topic name contains characters that cannot appear in the file name
because they are reserved by the underlying operating system, these characters
will be replaced by the token #
.
7.1.1.1. Mapping a data sample into columns¶
7.1.1.1.1. General case¶
As shown in the table above, a data sample is represented in a single row comprised of multiple columns. Each cell holds only a value for a final or leaf member in a complex data type. That is, for a member whose type is a Simple type (integer, char, short), enumeration, or String.
Each data value in a cell corresponds to a member whose name is in the type header. Given a complex data type, the name for a member is constructed as follows:
.<parent_member1>.<parent_member2>...<parent_memberN>.<final_member>
where <parent_member>
is the name of the parent complex member that
contains the subsequent member.
If either <parent_member>
or <final_member>
is a Collection type
(array or sequence), the member name is suffixed with [<index>]
, and a column
for each possible element in the Collection is created.
For example, consider the following type described in IDL:
struct NestedStruct {
long m_long;
};
struct TopLevelStruct {
String m_string;
long m_array[2];
NestedStruct m_complex;
NestedStruct m_complex_array[2];
};
The resulting type header row will look like this:
.m_string |
.m_array[0] |
.m_array[1] |
.m_complex.a_long |
.m_complex_array[0].a_long |
.m_complex_array[1].a_long |
Note
Considerations about collection types: Due to the column consistency
required by the CSV format, mapping a collection type requires us
to generate as many columns as the number of elements that can be present
in the collection, even if for a given data sample only a few of them
are present (such as for a Sequence
type). If the recorded type has
collections with large sizes, the generated file may hit the column limit
of some CSV processors.
7.1.1.1.2. Sequences¶
Mapping a Sequence
type is based on the mapping of a Collection
type explained above, plus an additional column to indicate the length
of the sequence. That is:
.<seq_member.length>.<seq_member[0]>...<seq_member[N-1]>
where the column <seq_member.length>
indicates how many elements are set
in the sequence (the number of columns with non-null values) and N
is the
maximum length of the sequence.
For example, consider the following type described in IDL:
struct StructType {
sequence<long, 4> m_seq;
};
The resulting type header row will look like this:
.m_seq.length |
.m_seq[0] |
.m_seq[1] |
.m_seq[2] |
.m_seq[3] |
7.1.1.1.3. Unions¶
The mapping of a Union
type is similar to a Struct
type except that
a discriminator column with name disc
is placed before all the members.
For example, consider the following type described in IDL:
union UnionType switch (long) {
case 0:
long case1;
case 1:
StructType case2;
default:
long case_default;
};
struct StructType {
UnionType m_union;
};
The resulting type header row will look like this:
.m_union.disc |
.m_union.case1 |
.m_union.case2 |
.m_union.default |
7.1.1.1.4. Data Values¶
For a given data sample, the value for each member is placed under the
corresponding column represented as a String
, which applies to all primitive
types. For primitive types, the output text will correspond to the standard
conventions for the type (e.g., decimal point for floating point numbers,
single quotes around a character, etc.). By default, enumerations are printed
with their corresponding text label. This behavior for enums is configurable.
There may not be values for every column in a sample. This may occur for the following situations:
A Sequence member that does not contain all the possible elements.
A Union member, which can only set a member at a time.
An optional member, which may or may not be set.
By default, the value of an empty member is represented as nil
.
For example, consider the following type described in IDL:
struct StructType {
sequence<long, 2> m_sequence;
@optional String m_optional;
};
And two samples with the following values (represented in JSON):
{
"m_sequence": [1, 2],
"m_optional": "hello"
}
{
"m_sequence": [1],
"m_optional":
}
The resulting type header row and the two data value rows will look like this:
.m_sequence.length |
.m_sequence[0] |
.m_sequence[1] |
.m_optional |
2 |
1 |
2 |
hello |
1 |
1 |
nil |
nil |