4.12. Sending Large Data

Connext DDS Micro supports transmission and reception of data types that exceed the maximum message size of a transport. This section describes the behavior and the configuration options.

This section includes:

4.12.1. Overview

Connext DDS Micro supports transmission and reception of data samples that exceed the maximum message size of a transport. For example, UDP supports a maximum user payload of 65507 bytes. In order to send samples larger than 65507 bytes, Connext DDS Micro must split the sample into multiple UDP payloads.

When a sample larger than the transport size is sent, Connext DDS Micro splits the sample into fragmants and starts sending fragments based based on a flow-control algorithm. A bandwidth allocation parameters on the DataWriter and the scheduling rate determined how frequent and how much data can be sent each period.

When a sample is received in multiple fragments, the receiver reassmbles the each fragment into a complete serialized sample. The serialized data is then de-serialized and made available to the user as regular data.

When working with large data, it is important to keep the following in mind:

  • Fragmentation is always enabled.
  • Fragmentation is per DataWriter.
  • Flow-control is per DataWriter. It is important to keep this in mind since in |rti_core_pro| the flow-controller works accross all *DataWriters in the same publisher.
  • Fragmentation is on a per sample basis. That is, two samples of the same type may lead to fragmentation of one sample, but not the other. The application is never exposed to fragments.
  • It is the DataWriters that determines the fragmentation size, and different DataWriters can use different fragmentation sizes for the same type.
  • All fragments must be received before a sample can be re-constructed. When using best-effort, if a fragment is lost, the entire sample is lost. When using reliability, a fragment that is not received may be resent. If a fragment is no longer available, the entire sample is dropped.
  • If one of the DDS write() APIs are called too fast when writing large samples, Connext DDS Micro may run out of resources. This is because sample may take a long time to send, and resources are not freed until the complete sample has been sent.

It is important to distinguish between the following concepts:

  • Fragmentation by Connext DDS Micro
  • Fragmentation by an underlying transport, e.g. IP fragmentation when UDP datagrams exceeds about 1488 bytes.
  • The maximum transmit message size of the sender. This is the maximum size of any payload going over the transport.
  • The maximum transport transmit buffer size of the sender. This is the maximum number of bytes that can be stored by the transport.
  • The maximum receive message size of a receiver. This is the maximum size of a single payload on a transport.
  • The maximum receive buffer size of a receiver. This is the maximum number of bytes which can be received.

4.12.2. Configuration of Large Data

For a general overview of writing large data, please refer to these sections in the RTI Connext DDS Core Libraries User’s Manual:

  • the ASYNCHRONOUS_PUBLISHER QoSPolicy section (available here if you have Internet access)
  • the FlowControllers section (available here if you have Internet access)

NOTE: Connext DDS Micro only supports the default FlowController.

Asynchronous publishing is handled by a separate thread that runs at a fixed rate. The rate and properties of this thread can be adjusted in the OSAPI_SystemProperty and the following fields before DomainParticipantFactory_get_instance() is called.

struct OSAPI_SystemProperty sys_property = OSAPI_SystemProperty_INITIALIZER;
DDS_DomainParticipantFactory *factory = NULL;

if (!OSAPI_System_get_property(&sys_property))
{
    /* error */
}

sys_property.task_scheduler.thread.stack_size = ....
sys_property.task_scheduler.thread.options = ....
sys_property.task_scheduler.thread.priority = ....
sys_property.task_scheduler.rate = rate in nanosec;

if (!OSAPI_System_set_property(&sys_property))
{
    /* error */
}

factory = DDS_DomainParticipantFactory_get_instance();

....

4.12.3. Limitations

The following are known limitations and issues with Large Data support:

  • It is not possible to disable fragmentation support.
  • The scheduler thread accuracy is based on the operating system.