Inherited dds topic structs

Hello udayk,

I actually went over the posts in the forums recently and saved some interesting posts, including this one.

To sum up how, to my understanding, this post is relevant to your question:

While you can set a reader to support type A and have various writers writing different types that extend A, and while data written by those writers will reach the reader, the reader will only be able to extract data that exists in type A.

That is, you will be able to receive the general shape properties sent by the "square" writer, by the "rectangle" writer, etc, but you will not be able to get the extra rectangle data.

In such situations I recommend one of the following alternatives:

1. Make the shape properties part of a shared struct that will be a field within triangle/rectangle/square

2. Use unions (similar to the first option)

3. If possible, use an enum to denote shape kind per sample and allow all "shape" readers and writers to send all shape types (this will, of course, introduce difficulties in creating shape type specific logic)

There may be other alternatives I haven't thought of but hopefully these will do for your use case.

Good luck!

Hi udayk,

You can alternatively use Extensible types for your scenario. For instance, if you defined a ShapeType and a ShapeTypeExtended:

enum ShapeFillKind {
    SOLID_FILL,
    TRANSPARENT_FILL,
    HORIZONTAL_HATCH_FILL,
    VERTICAL_HATCH_FILL
};

struct ShapeType {
    string<128> color; //@key
    long x;
    long y;
    long shapesize;
};//@Extensibility EXTENSIBLE_EXTENSIBILITY

struct ShapeTypeExtended : ShapeType {
    ShapeFillKind fillKind;
    float angle;
};//@Extensibility EXTENSIBLE_EXTENSIBILITY

Your subscriber application could create a Square topic using ShapeType, which would allow it to receive both other Squares defined as ShapeType and Squares that contain the extra fields in ShapeTypeExtended.

For more information on how to use Extensible Types check out the following section in the User's Manual. Note that this example is taken from RTI Shapes Demo, which enables you to publish Shapes using ShapeTypeExtended or ShapeType.

Best,
Fernando.

Hey Fernando,

I don't know so for sure but given what udayk wrote in his original post I have a hard time believing your solution works for him:

1. The subscriber, subscribing on ShapeType, will not get the added information that is sent by a publisher that publishes ShapeTypeExtended

2. Even with a solution that reverses the case (say, the subscriber subscribes to ShapeTypeExtended and is therefore able to receive all the data published by ShapeType publishers AND ShapeTypeExtended publishers) he will still not be able to get the polymorphic behavior he seems to crave (as is suggested by "I want the listener/receiver to listen to "one" topic (say, Shape) but the sender to send any of the inherited topics")

Perhaps I'm assuming too much regarding udayks' needs, but I believe that isn't the case.

Roy.

I feared as much.

If you gave an explanation about your "problem" and current architecture perhaps we could suggest alternative architectures that don't require c++ like polymorphism.

Roy.

Hey udayk,

I'll tell you why I think the situation you are describing is not as problematic as you first portrayed it but let me ask some questions:

1. Does the subscriber NEED the extra data that some formats can supply? If so, how does it handle images in formats where this data does not exist.

2. Does the subscriber WANT the extra data that some formats can supply? If so, why not have the subscriber subscribe to an additional topic, where "additional data relevant to a certain format" can be sent if an image is of that format?

3. Does the (supposedly polymorphic) subscriber not care about the additional fields? If so, you can simply have it listening on type Image and have the publishers publishing various extensions of Image (so that only fields of Image are received by this subscriber and additional fields relevant to each extension can be received by other subscribers interested in that specific type)

Of course, if the subscriber wants the additional fields you could also use a large type with optional fields but that architecture tends to be quite ugly.

.

I hope this helped a bit.

Hello Uday,

If I understood correctly, you can accomplish what you want with XTYPES as long as the subscriber is aware of the whole set of types that can be published. If that is not the case then you would need to define an extensibility mechanism inside the type itself. I will cover that later.

KickR's description of the XTYPES behavior on his first post is exactly right:

While you can set a reader to support type A and have various writers writing different types that extend A, and while data written by those writers will reach the reader, the reader will only be able to extract data that exists in type A.

The reason is this: Unlike the situation with C++/Java/C# polymorphism, the DataReader only has compiled-in knowledge for Type A. So even if it receives data of other types A1, A2 compatible (or derived) from A, it can only treat that type as an "A" type as it is the only code that it ever compiled. However, it is true that would be possible to get the derived types but doing that would require using a dynamic data API to read the data (as there is no compile-time code of anything other than A) and also would require using dynamic deserialization that looks at the type serialized by the sender. This is possible, but when we created XTYPES we considered that this was going to end up being quite complex and heavy and, therefore, settled for rules that allowed data to be de-serialized looking only at the (single) type the reader knows (i.e. not  looking into each message to dispatch based on the type the writer sent).

Now, assuming the DataReader does know what all the possible types are, then you can use the following approach to effectively getting the "polymorphic" behavior:

Define your type hierarchy using MUTABLE extensibility, and define the added members as OPTIONAL members. For example:

struct BaseImage {
   long              image_type;   //@ID 0
   string            format;       //@ID 1
   string            description;  //@ID 2
   sequence          image_data;   //@ID 3
 
}; //@Extensibility MUTABLE_EXTENSIBILITY

// Derived types are careful to define their members names and ID values 
// to not collide with each other
struct ImageKind1 : BaseImage {
   // Extra stuff for ImageKind1
   long    image1_extra1;  //@ID 101
                           //@Optional
   string  image1_extra2;  //@ID 102
                           //@Optional
   //  ...
}; //@Extensibility MUTABLE_EXTENSIBILITY
 
struct ImageKind2 : BaseImage {   
   // Extra stuff for ImageKind2 
   long  image2_extra1;  //@ID 201 
                         //@Optional
   long  image2_extra2;  //@ID 202  
                         //@Optional
   //  ...
}; //@Extensibility MUTABLE_EXTENSIBILITY


// We need a receiver "consolidated" or type that combines all member IDs
// This can be done using inheritance or copying the members 
// as lonf as member ID and names were chosen to be different on each derived type 
// it is possible to build a ConsolidatedImage  
struct ConsolidatedImage : BaseImage  {   
  // Extra stuff for ImageKind1
   long    image1_extra1;  //@ID 101
                           //@Optional
   string  image1_extra2;  //@ID 102
                           //@Optional
   
   // Extra stuff for ImageKind2 
   long  image2_extra1;  //@ID 201 
                         //@Optional
   long  image2_extra2;  //@ID 202  
                         //@Optional
}; //@Extensibility MUTABLE_EXTENSIBILITY
 

Note that since structs only support single inheritance each time you need to  "inherit" from more than one type you need to "copy" the elements inside instead. This is what I had to do for the ConsolidatedImage which logically would want to also "inherit" from ImageKInd1 and ImageKind2.

Now the receiver has all the code needed to recognize any kind of image. It can recognize the actual image either using the image_type member, or alternatively by checking the values of the added members (image1_extra1, etc.). Since these were marked //@Optional the receiver will set those members to "NULL" pointers if they were not part of the type sent.

The above approach works as long as you can compile the reader with the knowledge of all the types. If that is not the case you would have to use a different approach, such as defining something inside the type that you can use for effectively extending it, without actually changing the type itself. For example:

For example:

  
struct NameValuePair {
    string name;
    string value;
};
typedef sequence NameValuePairSequence;

struct NameBinaryValuePair {
    string name;
    sequence value;
}; 
typedef sequence NameBinaryValuePairSequence;

struct ExtensibleImage {
   long              image_type;   
   string            format;       
   string            description;  
   sequence   image_data;   
   
   NameValuePairSequence       string_nvps;
   NameBinaryValuePairSequence binary_nvps;
}; 
 

This approach allows you to put extra info into the ExtensibleImage using the string_nvps and binary_nvps fields. But if you want to put structured data then have to do the encoding/decoding yourself...

-Gerardo

Hey Gerardo,

Thank you for verifying what I said and also for a detailed explanation of reasonable solutions.

Obviously both of them have their own disadvantages but then this is a tough case for DDS to handle.

Your 2nd solution actually relates to a thought I had about using JSON over DDS to gain much more flexibility (obviously at the price of performance, among other things) without losing the discovery and multicast abilities of DDS.

Food for thought, I suppose.

Roy.

Hi,

Yes adding a JSON string inside a string to get extensibility will give you similar capabilities. But if you used JSON as the actual way to encode all your data (i.e. you told DDS it was a string but fill-in a JSON-formatted string) then, performance aside, there would be other side-effects:

You would lose type-compatibility checking. 

You would not be able to leverage content-filtered Topics, unless you define custom filters that are able to understand the JSON format

The tools (DataVisualizationAdminConsole, recording/replay, etc.) would not be able to understand the content beyond it being a string so you would loose some capabilities.

So  I think a mixed approach where you define some types fully in the IDL and perhaps have extra 'string' fields to fill with data encoded in other formats jile JSON or name-value pairs is a good compromise...

Gerardo

Hey,

Firstly, I wrote "among other things" :)

Of course I realize there are other prices to pay when using DDS to send JSON.

By "Food for thought" I meant that I need to consider those prices against the benefits my system could gain from such an approach (mainly type-flexibility)

Roy.

Hey udayk,

You said "Name value pair approach doesn't work for me as it is not just additional images but it could be additional fields which can be of boolean/int/short/double/string etc.." but you should note that Gerardo did say that if you want to use other kinds of data (apart from Strings and Bytes) you will have to also implement application level encoding / decoding (as is common for people using tcp/udp sockets for communication over binary streams or people using websockets/http for communication over strings).

About your questions regarding optional and inherited:

Optional (for mutuable extensibility) means that a field may be sent empty (and thus, they can also be expected to sometimes arrive as "empty").

That allows the reader (which can get just the basic fields which are not optional, can get the basic fields + the optional fields of of image kind 1, or can ge the basic fields + the optional fields of image kind2) to have a built in "empty" value (instead of having a "default" value which may cause some problems in detecting that a field was not sent by the writer).

Inheritance doesn't contradict with the above, but allows for a better understood structure model.

Regarding your last question:

No.

That is, if you defined your structs like the 2nd option you listed:

You are able to send the common stuff from EXTRA1 and from EXTRA2 to the TOPIC_STRUCT but you cannot create a reader which can receive all of the data from both extended types.

To complete this option you need to definte a consolidating type which inherits topic_struct and also defines type1 fields and type2 fields (or a type which inherits type1 and also defines type2 fields).

In all of the above options you must make sure that the ordinals are a. consistent between types (field x in type extra1 should have the same ordinal as field x in a consolidated type) and do not contradict (field x of type 1 should not have the same ordinal as field y of type 2).

Good luck,

Roy.

This: https://community.rti.com/rti-doc/500/ndds.5.0.0/doc/pdf/RTI_CoreLibrariesAndUtilities_GettingStarted_ExtensibleTypesAddendum.pdf (and specifically section 2.3.1) should help you.

Note that it also directs you at the XTypes documentation which may come in handy.

Roy.

Hello udayk,

By setting the type consistency to be DDS_ALLOW_TYPE_COERCION (assuming you've verified that the qos that you believe that you are using really is being used) you've allowed for writers and readers using these different types to communicate.

However, as specified in the link I gave you in my last post, there are more requirements to be met for that to work.

Primarily, as I understand, the reader and writer need to use TypeObjects during discovery for a match to be made.

I am not certain if mutable extensibility is fully supported in rti dds 5.0.0 but it would seem unlikely that even a simple case like this wouldn't work.

I would recommend trying ImageDim and Image1 communication (in both ways) to see how big your problem may be.

Good luck,

Roy.