We have a need to use DynamicData with a custom transport using the modern C++ API.
I was able to generate and run the custom FILE transport example for C++11 but that uses an IDL defined data type, CustomTransport.
I have a HelloWorld publisher and subscriber that use DynamicData that I was hoping to get working with the custom FILE transport instead of the built-ins transports.
I was hoping I might be able to simply use the custom_transportPlugin.cxx and hpp files generated by the custom transport example and change "CustomTransport" to "dds::core::xtypes::DynamicData".
In doing so, one of the problems I ran into is that CustomTransportPlugin_on_endpoint_attached() registers a callback for creating data samples.
The signature for that callback doesn't take any input parameters.
typedef void* (*PRESTypePluginDefaultEndpointDataCreateSampleFunction) (void);
That works fine for an IDL generated type but a DynamicType is required for the construction of DynamicData data sample.
The above function signature doesn't support passing in a type parameter.
Is there an example of using DynamicData with a custom transport using the modern C++ API?
DynamicData with a custom transport
Mon, 08/16/2021 - 10:29
#1
DynamicData with a custom transport
Hi Paul,
I'm not quite sure I understand what you're trying to do. The File transport example...I'm assuming that you're referring to this:
https://community.rti.com/examples/creating-custom-transport
is an example of how to create a custom Transport plugin for Connext DDS to use. The plugin API is C-based...so I'm not sure what you mean by creating a custom transport example for C++11.
If you mean you created a user application in C++11 that uses the FILE custom transport plugin that you compiled in C...that makes sense.
However, there is NO connection/dependency on the FILE custom transport plugin on the datatype that you use in you C++11 application. That transport plugin should work with ANY datatype. A Transport plugin is only responsible for sending/receiving buffers of bytes to/from a transport technology...in the example, that's a file in a filesystem. It doesn't know about or need to know about the datatype. The byte buffers being sent through a transport are not just user data, but ALL messages that need to be exchanged between participants in DDS, including metadata (discovery), ack/nack/HB for reliable connections, etc...all using the DDS-RTPS interoperability protocol.
I am guessing that your fundamental problem is how to use the Dynamic Data API in C++11. In which case, you should start with this example:
https://community.rti.com/static/documentation/connext-dds/6.1.0/doc/api/connext_dds/api_cpp2/group__DDSXTypesExampleModule.html
When using DynamicData, you do NOT need to use anything from an IDL file. There is no generated code that needs to be compiled and linked when using DynamicData.
Hi Howard,
Thanks for getting back to me. We're just getting started with ConnextDDS.
We'll be using the Modern C++ API with DynamicData and a custom transport.
So I'm looking for an example publisher/subscriber app that uses the Modern C++ API with DynamicData and a custom transport.
I started by getting a HelloWorld publisher and subscriber working that uses DynamicData and the Modern C++ API.
That's working with the built-in transports, no problem. In fact, I followed the example code you cited for those.
Next, I took a look at the custom transport example.
You're correct, the custom transport API is in C; so plugins written for it have to be in C.
The app that the FILE custom transport example generates by default is also in C. I got that working.
Because the generated example app was in C, I wanted to see if I could get a C++ app generated for the FILE custom transport example.
That was so I could see how you would integrate a custom transport with an app that uses the Modern C++ API.
I got the FILE custom transport example working for a generated C++ app.
I changed the LANG param in the connextdds_add_example() call in CMakeLists.txt to C++11 and it generated a C++ app instead of a C app.
I assume that the librti_custom_transports.so that it generated is in C but it generated CXX files for the app level stuff.
That all built and ran.
Next, I wanted to try integrating the FILE custom transport with my HelloWorld app that uses DynamicData.
I saw that the generated C++ app for the FILE custom transport generated a custom_transportPlugin.cxx file;
so I thought I might need something similar to integrate the FILE custom transport with my DynamicData HelloWorld C++ app.
However, the generated custom_transportPlugin.cxx file references the generated CustomTransport class which came from the IDL.
That where I got stuck because the custom transport example uses an IDL not dynamic data.
Hope this explains thing better.
Paul
OK, I think I understand...but what you need to do is way simpler than what you've done
Once you build the custom transport plugin, i.e., once you have the shared library that is the compiled transport plugin, e.g., librti_custom_transports.so, you just need to modify the QoS in the XML file (as shown in the example) for ANY application for DDS to use that transport.
There is NO code changes needed to an application to use the transport...just configuration changes that can all be done via the XML file.
So, with your working C++11/dynamic data application, just modify the QoS XML file that it's using with the contents of the example XML file included in the FILE custom transport example, to get your C++11/dynamic data application to use the transport.
My HelloWorld apps use XML AppCreate to instantiate things as that's another need of ours.
So the participants, pubilshers, subscribers, topics, etc all get instantiated from XML via a QosProvider (create_participant_from_config).
Once again, that's all working with the built-in transports.
Prior to posting on the forum, I had already tried incorporating the QOS profile from the FILE custom transport example into the XML for my HelloWorld apps.
Like you say, I thought that should be all I need to use the custom transport with my apps.
My XML intially just had "DefaultProfile" as it's QOS profile.
<qos_library name="qosLibrary">
<qos_profile name="DefaultProfile">
</qos_profile>
</qos_library>
I replaced that with the QOS profile from the example:
<qos_library name="CustomTransport_Library">
<qos_profile name="CustomTransport_Profile" is_default_qos="true">
... much more taken verbatim from the example
</qos_profile>
</qos_library>
And I changed the all the data readers, topics, etc in my XML to use CustomTransport_Library::CustomTransport_Profile instead of qosLibrary::DefaultProfile
However, it still uses the shmem transport instead of the FILE transport even with the built-in transports "shut-off" in the QOS profile:
<participant_qos>
<transport_builtin>
<mask>MASK_NONE</mask>
</transport_builtin>
That's when I decided to look at generating the C++ app for the example to see how things were done in that.
I thought I might be able to figure out what I'm missing from that.
I seem to be missing a piece to get this working, I just don't know what that is.
I would guess that the problem is in your XML application definition file. The <participant_qos> that sets the transport_builtin.mask will certainly disable all builtin transports, including shared memory, if it's being used to create the participant.
Are you sure that the participant is being created with the QoS Profile that defines the <participant_qos>?
Also, how are you determining that the shared memory transport is being used...it's usually difficult to determine that.
Hi Howard,
Good question about the participant.
I got my original XML config from one of the examples though I don't remember which one.
It had entries for the topic_qos and the datareader_qos but not for the particpant_qos nor the datawriter_qos.
I added entries for the particpant_qos and the datawriter_qos to point to the FILE custom transport and now my apps are using the custom transport.
I now see logging in my apps from the FILE custom transport but my publisher (and sometimes my subscriber) fails with the following error and my subscriber never gets any data:
NDDS_Transport_FILE_destroy_sendresource_srEA: Send resource (0x563f1aee8ab8) for port 454644080 destroyed: port= 454644080, address= 0.0.0.0
free(): invalid pointer
Aborted (core dumped)
As far as how I knew it was using the shmem transport, my subscriber was still receiving data but no files were being created under /tmp/dds/FileTransport.
I also saw log entries to that mentioned shmem when I turned on logging.
Paul
OK, so, the QoSes of Participants is pretty independent than that of DataWriters and DataReaders. DataWriter and DataReader QoSes don't affect the Participant.
In your example, you don't have to configure the DataWriter or DataReader QoSes to get the Participant to use the custom transport plugin.
As for the FILE transport plugin seg faulting, it's possible that the example code has some bug in the destruction logic...however, transport plugins are only going to be deleted when the application exits.
Did the FILE transport work in the example that you generated from IDL? If it didn't work there, then it probably won't work in your own XML application creation example.
In any case, the FILE transport is only an example of a custom transport plugin. It not very practical to use a file on a filesystem as a transport mechanism to talk between processes. I doubt it was fully tested to work on all operating systems for all file systems...maybe there's a missing flush() command or something.
You can try to debug it and get it to work correctly for your system, but if you're trying to write your own transport plugin, then I would only use it as an example to understand what each plugin interface function is supposed to do and then implement your own transport plugin with that knowledge. You're not gonna get alot of useful information debugging the FILE transport plugin I don't think.
Finally, RTI offers consulting through the Professional Services organization to help customer customize Connext DDS, such as writing custom transport plugins. You can discuss this with Paul or Katie (your RTI account team) if you're interested.
Hi Howard,
Thanks for your help on this.
To close out and answer your question, the FILE custom transport plug-in worked with RTI's example that uses code generated from an IDL.
As it worked in RTI's example, I was starting by trying to get it working with my HelloWorld apps before diving into writing my own plug-in.
Thanks again,
Paul