5.15. Zero Copy Transfer¶

Zero Copy transfer enables RTI Connext Micro to transmit data samples without copying them internally, similar to Zero Copy Transfer Over Shared Memory in Connext Professional. This offers several benefits, including higher throughput of user data, reduced latency between DDS endpoints (compared to other transports that send serialized data, such as UDP), and decoupling sample size from latency. This is particularly useful in applications with large sample sizes, such as image or lidar point cloud data.

At a high level, Zero Copy transfer works by a DataWriter and DataReader accessing the same shared memory; see Figure 5.5 below. A Zero Copy-enabled DataWriter creates a structure in a shared memory region and allocates samples from its pool to shared memory. When samples are published by the DataWriter, matching DataReaders are notified via a transport that new samples are available in shared memory. The DataReader then accesses the samples in shared memory using the standard DDS read/take APIs. Note that the DataReader and DataWriter must be co-located—that is, within the same operating system instance.

Figure 5.5 Zero Copy Transfer in Connext Micro¶

There are two methods for performing Zero Copy transfer in Connext Micro, which we refer to as Zero Copy v1 and v2 in this documentation. The main difference between these two methods is the transport used to notify DataReaders that new samples are available:

Zero Copy v1: Shared Memory Transport (SHMEM)
Zero Copy v2: Zero Copy v2 Transport

When choosing between the two versions, consider primarily whether you plan to interoperate Connext Micro with Connext Cert or Connext Professional. Zero Copy v1 is compatible with Connext Professional, while Zero Copy v2 is compatible with Connext Cert. Other functional differences between v1 and v2 are described in Compatibility.

5.15.1. Compatibility¶

Zero Copy v1 and v2 are API-compatible, meaning the API is identical whether you are using the Shared Memory transport or Zero Copy v2 transport. However, there are some important functional differences between the two versions:

Interoperability: Zero Copy v1 is compatible with the Zero Copy Transfer Over Shared Memory feature in Connext Professional, but not with Connext Cert. Zero Copy v2 is not compatible with Connext Professional, but is compatible with the Zero Copy transfer feature in Connext Cert.

Warning

Zero Copy transfer is not supported on all versions of Connext Cert. Consult the documentation in your Connext Cert installation for more information.
Sample synchronization: Zero Copy v2 synchronizes samples between the DataWriter and the DataReader. This ensures that a DataReader cannot access a sample while it is being modified by the DataWriter. As a result, the DataReader does not need to call the API FooDataReader_is_data_consistent to verify whether the sample has been changed, because the API always returns TRUE. In contrast, Zero Copy v1 does not provide any consistency checks, making it necessary for you to call FooDataReader_is_data_consistent.

Note

In Zero Copy v2, a slow DataReader may still miss samples if the DataWriter overwrites them. These missed samples will not be delivered to the reader. This is similar to Zero Copy v1, where samples can be missed, but v1 offers no guarantees of consistency between DataWriter and DataReader.
Volatile DataReaders: When using Zero Copy v2, a volatile DataReader receives all the historical samples available in the DataWriter’s queue. Zero Copy v1 uses the depth that is set for historical samples.
Sample acknowledgement: Zero Copy v2 does not support sample acknowledgments. The transport is inherently reliable, meaning that the DataWriter’s samples are immediately available to the DataReader. However, samples may be removed from the DataWriter’s cache when the queue reaches its history.depth, even if the reader has not accessed them.
KEEP_ALL support: Zero Copy v2 does not support the KEEP_ALL policy. Zero Copy v1 supports KEEP_ALL.
Version priority: If both Zero Copy v1 and v2 are enabled, Connext Micro will prioritize Zero Copy v1 over v2.

The following table summarizes the key differences between the two versions:

Table 5.9 Zero Copy Compatibility¶
Feature	Zero Copy v1	Zero Copy v2
Compatibility with Connext Professional	✓	✗
Compatibility with Connext Cert	✗	✓ 1
Sample synchronization	✗	✓
Sample acknowledgement	✓	✗
Support for `KEEP_ALL`	✓	✗
Transfer of discovery data	✓ 2	✗

1: Only if Zero Copy v2 is supported on the corresponding version of Connext Cert.
2: Discovery data is sent over the Shared Memory Transport (SHMEM), but it is not zero copied over.

5.15.2. Overview¶

Zero Copy samples reside in a shared memory region accessible from multiple processes. When creating a FooDataWriter that supports Zero Copy transfer of user samples, a sample must be created with a new non-DDS API (FooDataWriter_get_loan()). This will return a pointer A* to a sample Foo that lies inside a shared memory segment. A reference to this sample will be sent to a receiving FooDataReader across the shared memory. This FooDataReader will attach to a shared memory segment, and a pointer B* to sample Foo will be presented to the user. Because the two processes share different memory spaces, A* and B* will be different but they will point to the same place in RAM.

This feature requires using new RTI DDS Extension APIs:

5.15.3. Getting started¶

To enable Zero Copy transfer (either v1 or v2), follow these steps:

Annotate your type with the @transfer_mode(SHMEM_REF) annotation.

Currently, variable-length types (strings and sequences) are not supported for types using this transfer mode when a type is annotated with the PLAIN language binding (e.g., @language_binding(PLAIN) in IDL).
```
@transfer_mode(SHMEM_REF)
struct HelloWorld
{
    long id;
    char raw_image_data[1024 * 1024]; // 1 MB
};
```
Register the Shared Memory Transport (SHMEM) OR the Zero Copy v2 Transport. References will be sent across the chosen transport.

Note

If both transports are registered, Connext Micro will prioritize the Shared Memory transport (SHMEM) over the Zero Copy v2 transport as described in Compatibility.
Warning

If neither transport is registered AND your type is annotated with @transfer_mode(SHMEM_REF) (which can occur when using your annotated type with a version of Connext Micro that doesn’t support Zero Copy), two things will happen:
1. Connext Micro will not create a shared memory region for data samples, and;
2. Calls to FooDataWriter_get_loan() will fail with PRECONDITION_NOT_MET.
However, the DataWriter will still be created and can be used to send samples without using Zero Copy transfer.
Create a FooDataWriter for the above type.
Get a loan on a sample using FooDataWriter_get_loan().
Write a sample using FooDataWriter_write().

For more information, see the example HelloWorld_zero_copy, or generate an example for a type annotated with @transfer_mode(SHMEM_REF):

rtiddsgen -example -micro -language C HelloWorld.idl

5.15.3.1. Writing samples¶

The following code illustrates how to write samples annotated with @transfer_mode(SHMEM_REF):

for (int i = 0; i < 10; i++)
{
    Foo* sample;
    DDS_ReturnCode_t dds_rc;
    /* NEW API
       IMPORTANT - call get_loan each time when writing a NEW sample
     */
    dds_rc = FooDataWriter_get_loan(hw_datawriter, &sample);

    if (dds_rc != DDS_RETCODE_OK)
    {
        printf("Failed to get a loan\n");
        return -1;
    }

    /* After this function returns with DDS_RETCODE_OK,
     * the middleware owns the sample
     */
    dds_rc = FooDataWriter_write(hw_datawriter, sample, &DDS_HANDLE_NIL);
}

5.15.3.2. Reading samples¶

The following code illustrates how to read samples annotated with @transfer_mode(SHMEM_REF):

DDS_ReturnCode_t dds_rc;
dds_rc = FooDataReader_take(...)

/* process sample here */
/* NEW API
   IMPORTANT - is_data_consistent will always return true when ZC v2 is being used
 */

dds_rc = FooDataReader_is_data_consistent(hw_reader,
                                           &is_data_consistent,
                                           sample,sample_info);

if (dds_rc == DDS_RETCODE_OK)
{
     if (is_data_consistent)
     {
         /* Sample is consistent. Processing of sample is valid */
     }
     else
     {
        /* Sample is NOT consistent. Any processing of the sample should
         * be discarded and considered invalid.
         */
     }
}

5.15.4. Synchronizing samples¶

Zero Copy v1 and v2 handle sample synchronization differently. The following sections explain these differences in detail.

5.15.4.1. Zero Copy v1 synchronization¶

In Zero Copy v1, no synchronization exists between the sender (DataWriter) and the receiver (DataReader) for Zero Copy samples. This means that a sample’s content can be invalidated before the receiver has a chance to read it.

For example, consider creating a best-effort DataWriter with max_samples = 1. When the DataWriter is initialized, the middleware pre-allocates a pool of max_samples + 1 (2) samples in a shared memory region. These samples are loaned to the DataWriter when calling FooDataWriter_get_loan().

The following code illustrates this:

DDS_ReturnCode_t ddsrc;
Foo* sample;

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
sample->value = 10000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
* As this is a best-effort writer, the middleware immediately makes
* this sample available for reuse by another FooDataWriter_get_loan(...) call.
*/

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 2 */
sample->value = 20000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
* Again, the sample is made available immediately for reuse.
*/

/*
* At this point, the sample may have been received by the DataReader
* but not yet presented to the user.
*/

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
/*
* sample->value will now contain 10000 because the sample is reused
* from a pool with only 2 buffers.
*
* Additionally, references to both sample 1 and sample 2 might already
* have been received by the middleware on the *DataReader* side and stored
* in its internal cache. However, these samples may not yet have been delivered
* to the application. If sample->value is modified to 999 at this point,
* a subsequent call to *read()* or *take()* from the Subscriber will return 999,
* not the expected 10000. This happens because both the Publisher and Subscriber
* share the same memory region.
*
* Use `FooDataReader_is_data_consistent` to verify sample consistency and avoid
* this issue.
*
* Note: A sample becomes invalidated right after `FooDataWriter_get_loan(dw, &sample)`
* is completed. If the sample address has already been written to and has not yet
* been read by the receiver, the previously written data is invalidated.
*/

ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);

5.15.4.2. Zero Copy v2 synchronization¶

Zero Copy v2 provides synchronization between the DataWriter and DataReader. Since the queue size is limited, samples are reused once the queue is full. However, if a DataWriter modifies a sample before the DataReader has accessed it, that sample will not be presented to the user. Additionally, samples currently being read by the DataReader are locked, preventing the DataWriter from accessing them.

Consider the following example with max_samples = 1 (internally, the middleware will allocate 2 samples):

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
sample->value = 10000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 2 */
sample->value = 20000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);

/* Both samples are now available to the user, but the Reader may not have accessed them yet. */

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
sample->value = 30000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);

If the DataReader takes all the samples:

FooDataReader_take(reader,
                  &sample_seq, &info_seq,
                  DDS_LENGTH_UNLIMITED,
                  DDS_ANY_SAMPLE_STATE,
                  DDS_ANY_VIEW_STATE,
                  DDS_ANY_INSTANCE_STATE);

It will only receive two samples with values of 20000 and 30000, respectively.

If both the samples are currently being accessed by the user by calling FooDataReader_read() or FooDataReader_take(), they will be locked. Any attempt to call FooDataWriter_get_loan() on the DataWriter will result in an OUT_OF_RESOURCES error.

5.15.5. Caveats¶

After you call FooDataWriter_write(), the middleware takes ownership of the sample. It is no longer safe to make any changes to the sample that was written. If, for whatever reason, you call FooDataWriter_get_loan() but never write the sample, you must call FooDataWriter_discard_loan() to return the sample back to the FooDataWriter. Otherwise, subsequently calling FooDataWriter_get_loan() may fail, because the FooDataWriter has no samples to loan.
The current maximum supported sample size is a little under the maximum value of a signed 32-bit integer. For that reason, do not use any samples greater than 2000000000 bytes.