7.1. Storage Utility Plugins

7.1.1. CSV

This is a type of output storage plugin that stores the data provided to it in Comma-Separated Value (CSV) format. It is meant to be used with Converter to perform conversion of a recorded database into CSV output files. Typically it helps solve use cases like offline analysis or incident investigation. For more information about the configuration tags, see Section 5.2.9.

By default, the plug-in generates a separate csv file for each recorded Topic. The content and format of each file is as follows:

Table 7.1 CSV file format

Row 1: Topic Entry

Topic name: <Name>

Row 2: Type Header

timestamp

member1

member2

memberN

Row 3: Data values

Reception timestamp of the sample (in nanoseconds)

Value for member1

Value for member2

Value for memberN

The filename matches the topic name, with a .csv extension. All the files are placed in a directory that can be specified in the plug-in configuration.

Note

If the topic name contains characters that cannot appear in the file name because they are reserved by the underlying operating system, these characters will be replaced by the token #.

7.1.1.1. Mapping a data sample into columns

7.1.1.1.1. General case

As shown in the table above, a data sample is represented in a single row comprised of multiple columns. Each cell holds only a value for a final or leaf member in a complex data type. That is, for a member whose type is a Simple type (integer, char, short), enumeration, or String.

Each data value in a cell corresponds to a member whose name is in the type header. Given a complex data type, the name for a member is constructed as follows:

.<parent_member1>.<parent_member2>...<parent_memberN>.<final_member>

where <parent_member> is the name of the parent complex member that contains the subsequent member.

If either <parent_member> or <final_member> is a Collection type (array or sequence), the member name is suffixed with [<index>], and a column for each possible element in the Collection is created.

For example, consider the following type described in IDL:

struct NestedStruct {
    long m_long;
};

struct TopLevelStruct {
    String m_string;
    long m_array[2];
    NestedStruct m_complex;
    NestedStruct m_complex_array[2];
};

The resulting type header row will look like this:

.m_string

.m_array[0]

.m_array[1]

.m_complex.a_long

.m_complex_array[0].a_long

.m_complex_array[1].a_long

Note

Considerations about collection types: Due to the column consistency required by the CSV format, mapping a collection type requires us to generate as many columns as the number of elements that can be present in the collection, even if for a given data sample only a few of them are present (such as for a Sequence type). If the recorded type has collections with large sizes, the generated file may hit the column limit of some CSV processors.

7.1.1.1.2. Sequences

Mapping a Sequence type is based on the mapping of a Collection type explained above, plus an additional column to indicate the length of the sequence. That is:

.<seq_member.length>.<seq_member[0]>...<seq_member[N-1]>

where the column <seq_member.length> indicates how many elements are set in the sequence (the number of columns with non-null values) and N is the maximum length of the sequence.

For example, consider the following type described in IDL:

struct StructType {
    sequence<long, 4> m_seq;
};

The resulting type header row will look like this:

.m_seq.length

.m_seq[0]

.m_seq[1]

.m_seq[2]

.m_seq[3]

7.1.1.1.3. Unions

The mapping of a Union type is similar to a Struct type except that a discriminator column with name disc is placed before all the members.

For example, consider the following type described in IDL:

union UnionType switch (long) {
    case 0:
    long case1;

    case 1:
    StructType case2;

    default:
    long case_default;
};

struct StructType {
    UnionType m_union;
};

The resulting type header row will look like this:

.m_union.disc

.m_union.case1

.m_union.case2

.m_union.default

7.1.1.1.4. Data Values

For a given data sample, the value for each member is placed under the corresponding column represented as a String, which applies to all primitive types. For primitive types, the output text will correspond to the standard conventions for the type (e.g., decimal point for floating point numbers, single quotes around a character, etc.). By default, enumerations are printed with their corresponding text label. This behavior for enums is configurable.

There may not be values for every column in a sample. This may occur for the following situations:

  • A Sequence member that does not contain all the possible elements.

  • A Union member, which can only set a member at a time.

  • An optional member, which may or may not be set.

By default, the value of an empty member is represented as nil.

For example, consider the following type described in IDL:

struct StructType {
    sequence<long, 2> m_sequence;
    @optional String m_optional;
};

And two samples with the following values (represented in JSON):

{
    "m_sequence": [1, 2],
    "m_optional": "hello"
}

{
    "m_sequence": [1],
    "m_optional":
}

The resulting type header row and the two data value rows will look like this:

.m_sequence.length

.m_sequence[0]

.m_sequence[1]

.m_optional

2

1

2

hello

1

1

nil

nil