Transfering large data over DDS?

3 posts / 0 new
Last post
Offline
Joined: 08/25/2015
Posts: 31
Transfering large data over DDS?

Hi,

is it feasible to transfer large chunks of data through dds? I need to send images, pdf, ... from a client to a server and vice versa. Or should I use something other (e.g. websockets?).

What QoS settings are needed for such a task?

Thx,

Christian

Fernando Garcia's picture
Offline
Joined: 05/18/2011
Posts: 173

Hi Christian,

To send large data through DDS reliably you will need to use an Asynchronous Publisher, which sends the user data in a external middleware thread.

There is an example on GitHub that shows how to configure Asynchronous Publication, but probably the easiest option is to use one of the built-in QoS profiles designed for that purpose. You could use:

  • Generic.StrictReliable.LargeDatawhich enables asynchronous publication with the default flow controller (i.e,. no flow control).
  • Generic.StrictReliable.LargeData.FastFlow, which enables strict reliable communication for large data with a 100 MB/sec (838 Mb/sec) flow controller.
  • Generic.StrictReliable.LargeData.MediumFlow, which enables strictly reliable communication for large data with a 25 MB/sec (209 Mb/sec) flow controller.
  • Generic.StrictReliable.LargeData.SlowFlow, which enables strictly reliable communication for large data with a 6.25 MB/sec (52 Mb/sec) flow controller.

Let me know if these solutions work for you. Otherwise, we can go over your example for a more specific configuration.

Thanks,
Fernando.

Offline
Joined: 08/25/2015
Posts: 31

Hi,

thx for the information. We've the problem that rti crashes / throws an exception when we try to transport large data. Looks like there is a bug deep inside the serialization because it only crashes under certain conditions. I've attached a testcase which shows the problem (tested with i86Win32VS2015 and x64Linux3gcc4.8.2). The qos must be an async qos like BuiltinQosLibExp::Generic.StrictReliable.LargeData

The idl must be exact the given idl. Removing one of the parameters does not trigger the problem. Also the size of the content must not be 1024*1024. Therefore it looks like some kind of alignment problem to me.

Here the error:

 RTICdrStream_serializeParameterLength:!precondition: me == ((void *)0) || lengthPosition == ((void *)0) || state == ((void *)0)
PRESWriterHistoryDriver_initializeSample:!serialize
WriterHistoryMemoryPlugin_addEntryToSessions:!initialize sample
WriterHistoryMemoryPlugin_getEntry:!add virtual sample to sessions
WriterHistoryMemoryPlugin_addSample:!get entry
PRESWriterHistoryDriver_addWrite:!add_sample
PRESPsWriter_writeInternal:!collator addWrite

I'll file a bug report for this.

 

/edit: forgot to add that we're using 5.2.3-6

File Attachments: