Inter-operability issues btw RTI and OpenDDS due to "align to 8 bytes" issue

Hello Gautam,

OK. I see. I had already created a response before I saw your note from the OpenDDS forum. It is good to know they added an option to be compatible with RTI DDS... I include below the explanation I prepared so it can serve as future reference...

This alignment issue arises because the specification for the "boundary" for alignment of the serialized data has some room for ambiguity. I believe RTI's interpretation is correct as I will explain below but I can see how it can be mis-interpreted since we also made the same error up until version 4.2e. 

But if you jump to the end you will see there is an option to rtiddsgen (option -use42eAlignment) that might allow you to be compatible with what OpenDDS is doing. More details follow. 

Background

As you may know the CDR representation dictates that types should be serialized at byte addresses that are "aligned" to certain boundaries that depend on the data type. For example 32-but integers should be serialized at byte-positions that are aligned to 4 (i.e. multiples of 4). Whereas 64-bit integers and doubles should be serialized aligned to 8-byte boundaries. While serializing to a byte stream if the "serialization" needs to serialize something but the curent position is not aligned it needs to add padding  (i.e. skip those byte addresses) until it reaches a byte address that is a multiple of the alignment.

So say you are serializing something on a existing CDR stream that has currently filled up to byte 11.  If you were serializing a long you can do it starting from byte 12 (because 12 is a multiple of 4 which is the alignment for long). However if you are serializing a double, then you need to insert 4 bytes of "padding" so skip to byte 16 before serializing the double. This is because 12 is not multiple of 8 (the alignment of double) and the next multiple is 16.

The above is something that is well specified and all vendors follow.

The potential ambiguity

The question is where to start counting the offset "CDR" stream. Where is the beginning of the DFR stream? Is ut the beginning of he IP packet, or the UDP?, or the RTPS message, or the DATA submessage?

The DDS-RTPS specification does say something about it. But it is a big ambiguous. In section 9.2.2.12 SerializedPayload it draws the following picture:

SerializedPayload 
0...2...........8...............16.............24...............32 
+---------------+---------------+---------------+---------------+ 
|                                                               | 
~                     octet serializedPayload[]                 ~ 
|                                                               | 
+---------------+---------------+---------------+---------------+

And then it says:

Note that when using CDR, primitive types must be aligned to their length. For example, a long must start on a 4-byte boundary. The boundaries are counted from the start of the CDR stream.

Then in section 12.2.1.2 OMG CDR the RTPS specification says:

In addition to the encapsulation schema identifier, the OMG CDR encapsulation specifies a 16-bit options field followed by the data encoded using CDR. The options field is left for future extensions. This version of the specification should not interpret it when it reads it.

And then draws the following picture (note that there is also a corresponding figure for Big Endian (CDR_BE) which I omitted here.

0...2...........8...............16..............24..............32 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|            CDR_LE             |     ushort options            | 
+---------------+---------------+---------------+---------------+ 
|                                                               | 
~             Serialized Data (CDR Little Endian)               ~ 
|                                                               | 
+---------------+---------------+---------------+---------------+

From these two descriptions RTI came to the conclusion that the CDR stream should be considered to be starting after the completion of the 4-byte header containing the CDR_LE + options. That is the CDR stream matches exactly the section above where it says "Serialized Data".

Application to a concrete IDL-defined type

In the example you mention of the simple type with a double:

struct MyType {
    double value;
};  

Then if apply the above rules the double would be serialized immediately following the 4-bytes: CDR_LE (0x0001) + options (0x0000). This is because the CDR stream logically starts here which means that would be offset 0 which is aligned to an 8-byte boundary. This is what you see in the RTI packet capture that you sent. The "CDR_LE + options header" end at wireshark offset 0x65. So we consider the CDR stream to begin at 0x66 and this becomes byte "0" of the CDR stream so the double starts being serialized immediately (at wireshark address 0x66).

It seems like OpenDDS is not doing this. Instead it is inserting 4 bytes of padding. The  "CDR_LE + options header" end at wireshark offset 0x65 and they start serializing the double at 0x6A. Perhaps instead of considering the CDR stream to beging after the   "CDR_LE + options header"  they consider it to begin just before the header (i.e. at wireshark address 0x62) In this case when they get to the double they would be at an offset of 4 relative to the beginning of the CDR stream and thus insert a 4-byte padding).

We also used to do it "wrong"

RTI DDS version 4.2e and earlier used to also do this "wrong" i.e. interpret the CDR stream to start before the "CDR_LE + options header".  When we realized of this we changed so versions 4.2f onwards work as I described above.

Being compatible with the other interpretation

Since DDS versions 4.2e and earlier used the  "wrong" interpretation the correction introduced in 4.2f broke compatibility with 42e and earlier. In order to help people transition rtiddsgen has an option to generate serialization code with the "legacy 4.5e alignment". This is controlled passing to rtiddsgen  the command-line option  -use42eAlignment.

So it ay be that if you used the option  -use42eAlignment when running rtiddsgen the serialization code would be compatible with that OpenDDS expects.

Gerardo