Inter-operability issues btw RTI and OpenDDS due to "align to 8 bytes" issue

4 posts / 0 new
Last post
Offline
Last seen: 8 years 11 months ago
Joined: 11/15/2015
Posts: 5
Inter-operability issues btw RTI and OpenDDS due to "align to 8 bytes" issue

I have been interested in some inter-operability issues that I have seen between RTI and OpenDDS. I am posting this here and will also cross post/email to OpenDDS community. By no means do I claim that interoperability problems exist due to RTI's fault (or anyone's fault.) (What I see could very well be due to some programming oversight on my part.)

Summary:  When a "double" value is the first thing in a CDR stream (that is, first after "encapsulation kind" and "encapsulation options"), RTI does not pad the stream to have the double align to an 8 byte offset, but OpenDDS does. This leads to interoperability issues.  I did 4 tests:

1. RTI_to_RTI  (work fine)
2. RTI_to_OpenDDS (incorrect values for "double" are received)
3. OpenDDS_to_OpenDDS (works fine)
4. OpenDDS_to_RTI (incorrect values for "double" are received)

I have attached a file with output of the subscriber showing erroneous values. I have also attached two wireshark images showing the possible issues. (Of course this could very much due to some error on my part, I appreciate any feedback.)


Please see .txt file and two png files attached. I can upload the 4 PCAP files as well as sources if desired.


Gautam

 

 

Organization:
Offline
Last seen: 8 years 11 months ago
Joined: 11/15/2015
Posts: 5

I should have stated that tests were with OpenDDS 3.5.1 and RTI 5.2. I realize there is an newer version of OpenDDS, and I will test w/ that, but I do not believe this alighnment behavior has been recently changed in newer OpenDDS releases, but I may have missed it in reading the release notes.

Offline
Last seen: 8 years 11 months ago
Joined: 11/15/2015
Posts: 5

Apparently I am late to this party.  On OpenDDS list I received the following response (which I duplicate here from completeness):

This was previously discussed on the list so please see those older 
posts for more details.  Look for the subject line "Rti 
interoperability" in April 2015.  Mailing list archives are available at 
sourceforge.

I believe OpenDDS (of any version) is meeting the specification here.  
In OpenDDS 3.7, we added an option to handle serialization the "RTI 
way," as documented in the release notes (NEWS file):
- Support command line option -DCPSRTISerialization when integrating 
with RTI
   DDS using RTPS, to account for a different serialization algorithm.

Thanks,
Adam Mitz
Principal Software Engineer
Object Computing, Inc.

 

Gerardo Pardo's picture
Offline
Last seen: 4 months 1 week ago
Joined: 06/02/2010
Posts: 602

Hello Gautam,

OK. I see. I had already created a response before I saw your note from the OpenDDS forum. It is good to know they added an option to be compatible with RTI DDS... I include below the explanation I prepared so it can serve as future reference...

This alignment issue arises because the specification for the "boundary" for alignment of the serialized data has some room for ambiguity. I believe RTI's interpretation is correct as I will explain below but I can see how it can be mis-interpreted since we also made the same error up until version 4.2e. 

But if you jump to the end you will see there is an option to rtiddsgen (option -use42eAlignment) that might allow you to be compatible with what OpenDDS is doing. More details follow. 

Background

As you may know the CDR representation dictates that types should be serialized at byte addresses that are "aligned" to certain boundaries that depend on the data type. For example 32-but integers should be serialized at byte-positions that are aligned to 4 (i.e. multiples of 4). Whereas 64-bit integers and doubles should be serialized aligned to 8-byte boundaries. While serializing to a byte stream if the "serialization" needs to serialize something but the curent position is not aligned it needs to add padding  (i.e. skip those byte addresses) until it reaches a byte address that is a multiple of the alignment.

So say you are serializing something on a existing CDR stream that has currently filled up to byte 11.  If you were serializing a long you can do it starting from byte 12 (because 12 is a multiple of 4 which is the alignment for long). However if you are serializing a double, then you need to insert 4 bytes of "padding" so skip to byte 16 before serializing the double. This is because 12 is not multiple of 8 (the alignment of double) and the next multiple is 16.

The above is something that is well specified and all vendors follow.

The potential ambiguity

The question is where to start counting the offset "CDR" stream. Where is the beginning of the DFR stream? Is ut the beginning of he IP packet, or the UDP?, or the RTPS message, or the DATA submessage?

The DDS-RTPS specification does say something about it. But it is a big ambiguous. In section 9.2.2.12 SerializedPayload it draws the following picture:

SerializedPayload 
0...2...........8...............16.............24...............32 
+---------------+---------------+---------------+---------------+ 
|                                                               | 
~                     octet serializedPayload[]                 ~ 
|                                                               | 
+---------------+---------------+---------------+---------------+

And then it says:

Note that when using CDR, primitive types must be aligned to their length. For example, a long must start on a 4-byte boundary. The boundaries are counted from the start of the CDR stream.

Then in section 12.2.1.2 OMG CDR the RTPS specification says:

In addition to the encapsulation schema identifier, the OMG CDR encapsulation specifies a 16-bit options field followed by the data encoded using CDR. The options field is left for future extensions. This version of the specification should not interpret it when it reads it.

And then draws the following picture (note that there is also a corresponding figure for Big Endian (CDR_BE) which I omitted here.

0...2...........8...............16..............24..............32 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|            CDR_LE             |     ushort options            | 
+---------------+---------------+---------------+---------------+ 
|                                                               | 
~             Serialized Data (CDR Little Endian)               ~ 
|                                                               | 
+---------------+---------------+---------------+---------------+

From these two descriptions RTI came to the conclusion that the CDR stream should be considered to be starting after the completion of the 4-byte header containing the CDR_LE + options. That is the CDR stream matches exactly the section above where it says "Serialized Data".

Application to a concrete IDL-defined type

In the example you mention of the simple type with a double:

struct MyType {
    double value;
};  
 

Then if apply the above rules the double would be serialized immediately following the 4-bytes: CDR_LE (0x0001) + options (0x0000). This is because the CDR stream logically starts here which means that would be offset 0 which is aligned to an 8-byte boundary. This is what you see in the RTI packet capture that you sent. The "CDR_LE + options header" end at wireshark offset 0x65. So we consider the CDR stream to begin at 0x66 and this becomes byte "0" of the CDR stream so the double starts being serialized immediately (at wireshark address 0x66).

It seems like OpenDDS is not doing this. Instead it is inserting 4 bytes of padding. The  "CDR_LE + options header" end at wireshark offset 0x65 and they start serializing the double at 0x6A. Perhaps instead of considering the CDR stream to beging after the   "CDR_LE + options header"  they consider it to begin just before the header (i.e. at wireshark address 0x62) In this case when they get to the double they would be at an offset of 4 relative to the beginning of the CDR stream and thus insert a 4-byte padding).

We also used to do it "wrong"

RTI DDS version 4.2e and earlier used to also do this "wrong" i.e. interpret the CDR stream to start before the "CDR_LE + options header".  When we realized of this we changed so versions 4.2f onwards work as I described above.

Being compatible with the other interpretation

Since DDS versions 4.2e and earlier used the  "wrong" interpretation the correction introduced in 4.2f broke compatibility with 42e and earlier. In order to help people transition rtiddsgen has an option to generate serialization code with the "legacy 4.5e alignment". This is controlled passing to rtiddsgen  the command-line option  -use42eAlignment.

So it ay be that if you used the option  -use42eAlignment when running rtiddsgen the serialization code would be compatible with that OpenDDS expects.

Gerardo