Sending byte array using DDS_Dynamic Data

12 posts / 0 new
Last post
Offline
Last seen: 7 years 4 months ago
Joined: 03/03/2015
Posts: 19
Sending byte array using DDS_Dynamic Data

Hello,

I'm using the DDS_DynamicData feature to send data of multiple types using RTI DDS. I managed to define my dynamic data type support for simple types (primitives). However, I have more complex data types which I want to send using the DynamicData feature, like arrays, strings, structs, etc. 

I want to make the application as generic as possible, so I don't really want to create a type support for each data type, as I want to send the complex types as "byte array" or something similar. 

I found 2 possible solutions, but I'm not sure if either of them would work.

The dynamic data type support should look something like this:

struct MyType{

int dataType;

uint32 dataSize;

unsigned char* dataValue;

}

I solved the issue for the "dataType" and "dataSize" fields, but I don't seem to find a good solution for the "dataValue" field for my structure. The structure is pretty similar to the "DDS_Octets" type, by I have to use a DDS DynamicDataWriter and a DDS DynamicDataReader to write and read my data.

1. Is there a way I can set a "DDS_Octets" type on a DDS_DynamicDataTypeSupport

or

2. Is there a DDS TypeCode that can be used for the "unsigned char*" type?

 

Thank you!

Best Regards,

Andrei

 

sara's picture
Offline
Last seen: 1 year 11 months ago
Joined: 01/16/2013
Posts: 128

Hi Andrei,

In the LabVIEW toolkit we do something similar. We need to map generic LabVIEW types into DDS types. For Unsigned Char we used DDS Octets. And the way to extract that typecode is calling the followign function:

DDS_TypeCode *tc = NULL;

tc = DDS_TypeCodeFactory_get_primitive_tc(factory,
DDS_TK_OCTET);

At least in C. And then you can create a sequence or an array of those elements. You could even create an unbounded sequence if you don't know the maximum size of the dataValue that you are going to obtain.

I hope this helps! Thanks,
Sara

 

 

Offline
Last seen: 7 years 4 months ago
Joined: 03/03/2015
Posts: 19

Hey Sara,

Thank you for your answer. I actually started looking into using sequences with DDS_Octets.

How are you handling variable size arrays in the LabVIEW toolkit? Are you using sequences and provide them with a very large number as the length? I know that for sequences, only the actual size is sent over the wire, but they need a max value as well. I don't really know what the maximum size for arrays in LabVIEW is but I'm trying to do something similar with what you described above, as I have multiple data types and I have to map them to DDS_TypeCodes.

 

Thank you again for your support,

Best Regards,

Andrei

sara's picture
Offline
Last seen: 1 year 11 months ago
Joined: 01/16/2013
Posts: 128

Hi Andrei,

LabVIEW arrays can have variable size, so we detect the size of the array in LabVIEW at creation time and then create the DDS type accordingly. Then at run-time LabVIEW arrays could have different sizes and sequences will still work, but always to the maximum lenght established during creation. Or we make them unbounded if the user selects that option.

Do your origin type have a size defined at some point? Which technologies are you trying to bridge with DDS? 

Thanks,
Sara

Offline
Last seen: 7 years 4 months ago
Joined: 03/03/2015
Posts: 19

Hey Sara

I'm just prototyping for the moment. I'm calling DDS directly from a C++ prototyping project. My variable size types are allocated dynamically at runtime (Ex: Arrays)

When I first create the DDS_TypeCode, I have a size for the origin type (Let's assume for an Array<Int32>). I may configure the sequence to use that size, but I may get larger arrays during the program run. So what I did was defining a sequence of type <Int32> with a very large bound (about 1.500.000.000). Of course, this implies that I guess(and hope) that nobody will send an array with a size larger than that. 

You mentioned that you are making the sequences unbounded if the user selects the option. Don't you have to specify a max lenght for sequences when defining the Type Code? 

Thank you :),

Andrei

sara's picture
Offline
Last seen: 1 year 11 months ago
Joined: 01/16/2013
Posts: 128

Hi Andrei,

Actually, you just need to set the bound to the maximum integer (RTI_INT32_MAX) and that works. However, there are some issues with resources you may run into. For instance, I would recommend you setup the following QoS settings:

<datawriter_qos> 
    <property>
        <value>
            <element>
                <name>
                dds.data_writer.history.memory_manager.fast_pool.pool_buffer_max_size
                </name>
                <value>4096</value>
            </element>
        </value>
    </property>
</datawriter_qos>

<datareader_qos>
    <property>
        <value>
            <element>
                <name>
                dds.data_reader.history.memory_manager.fast_pool.pool_buffer_max_size
                </name>
                <value>4096</value>
            </element>
        </value>
    </property>
</datareader_qos>

In addition, when managing DynamicData, there are other settings to configure. I am working on a KB article with that additional information and on verifying this information, but here you have a summary of the ideas I would try:

struct DDS_DynamicDataTypeProperty_t prop = DDS_DynamicDataProperty_t_INITIALIZER;
prop.serialization.trim_to_size = DDS_BOOLEAN_TRUE;
prop.serialization.min_size_serialized =  DDS_LENGTH_UNLIMITED; //I need to verify this 
prop.serialization.max_size_serialized = DDS_LENGTH_UNLIMITED;

Please give it a try and let me know how it goes.

Thanks,
Sara

Offline
Last seen: 7 years 4 months ago
Joined: 03/03/2015
Posts: 19

Hey Sara,

I've set the max sequence size to RTI_INT32_MAX and set the QoS configuration that you've mentioned.

I don't know where to initialize and use the DDS_DynamicDataTypeProperty_t that you've mentioned, but communication was done in a fair maner without the property configuration.

Still, I got into an issue when I called the "delete_datareader" function on the DDSDomainParticipant during cleanup, as I wanted to delete the DDS_DataReader. When I called that function I got a null reference exception (I'm still investigating it and will try to come up with a callstack). When I modified the maximum sequence length and set a smaller value(ex: 10000000), the "delete_datareader" function executed successfully. May that come up because I didn't use the "DDS_DynamicDataTypeProperty_t" configuration that I've mentioned above?

 

Thank you,

Andrei

Offline
Last seen: 1 week 6 days ago
Joined: 01/13/2016
Posts: 58

sorry to bother you, i want to dynamic create topic idl like this:
struct SimData {
long nRow;
long nCol;
char Data[16];
};
i don't know how to create member "char Data[16]", its an array of primitive char type. i use the codes in the fowllowing:
struct DDS_UnsignedLongSeq dimensions;
DDS_TypeCode* sz = DDS_TypeCodeFactory_create_array_tc(factory, &dimensions, DDS_TypeCodeFactory_get_primitive_tc(factory, DDS_TK_CHAR), &ex);
if (ex != DDS_NO_EXCEPTION_CODE)
{
return NULL;
}

DDS_TypeCode_add_member(portTc, "Data", DDS_MEMBER_ID_INVALID, sz, DDS_TYPECODE_NONKEY_REQUIRED_MEMBER, &ex);
if (ex != DDS_NO_EXCEPTION_CODE)
{
return NULL;
}
i know the variable dimensions is not setting correctlly. can you help me?

Offline
Last seen: 7 years 4 months ago
Joined: 03/03/2015
Posts: 19

I think you can solve your problem is you use sequences instead of arrays. If you create it as an array, you have to always set 16 elements on it. If you use sequences (DDS_TypeCodeFactory_create_sequence_tc), you can set a maximum of 16 elements on the sequence, but if you set less than 16 elements on the sequence, only the actual size will get sent over the wire.

Offline
Last seen: 1 week 6 days ago
Joined: 01/13/2016
Posts: 58

I got the way to solve it, there are functions for creating it, like this

DDS_UnsignedLongSeq_initialize

DDS_UnsignedLongSeq_ensure_length

DDS_UnsignedLongSeq_get_reference

DDS_UnsignedLongSeq_finalize

sara's picture
Offline
Last seen: 1 year 11 months ago
Joined: 01/16/2013
Posts: 128

Hi Andrei,

Sorry I forgot to mention that.  You have to add that when creating the TypeSupport for your DynamicData:

typeSupport = DDS_DynamicDataTypeSupport_new(typeCode, &prop);

Thanks,

Sara

Offline
Last seen: 7 years 2 months ago
Joined: 08/30/2017
Posts: 1

Hi all,

I have the following IDL definition:

file.idl:
---
module idlModule {
struct idlDataTopic {
int dataType;
int dataSize;
string<4096> dataValue;
};
}
---
So I use the autogenerated code by rtdddsgen to compound the data to publish.

And dataValue should accept several different C structs. Now it is a string but I guess it is not the correct aproach.

Can I use your solution about DynamicData only for dataValue and still use at the same time the code autogenerated by rtdddsgen for the rest of topic?

Best regards,

Luis.