4.13. Zero Copy Transfer Over Shared Memory

4.13.1. Introduction

This section is organized as follows:

4.13.2. Overview

Zero Copy transfer over shared memory allows large samples to be transmitted with a minimum number of copies. These samples reside in a shared memory region accessible from multiple processes. When creating a FooDataWriter that supports Zero Copy Transfer of user samples, a sample must be created with a new non-DDS API (FooDataWriter_get_loan(…)). This will return a pointer A* to a sample Foo that lies inside a shared memory segment. A reference to this sample will be sent to a receiving FooDataReader across the shared memory. This FooDataReader will attach to a shared memory segment and a pointer B* to sample Foo will be presented to the user. Because the two processes shared different memory spaces, A* and B* will be different but they will point to the same place in RAM.

This feature requires the usage of new RTI DDS Extension APIs:

  • FooDataWriter_get_loan()
  • FooDataWriter_discard_loan()
  • FooDataReader_is_data_consistent()

For detailed information, see the C API Reference and C++ API Reference.

4.13.3. Getting Started

To enable Zero Copy transfer over shared memory, follow these steps:

  1. Annotate your type with the @transfer_mode(SHMEM_REF) annotation. Currently, variable-length types (strings and sequences) are not supported for types using this transfer mode when the language binding is INBAND.

    @transfer_mode(SHMEM_REF)
struct HelloWorld {
    long id;
    char raw_image_data[1024 * 1024]; // 1 MB
};

  2. Register the Shared Memory Transport (see Registering the SHMEM Transport). References will be sent across the shared memory transport.

  3. Create a FooDataWriter for the above type.

  4. Get a loan on a sample using FooDataWriter_get_loan().

  5. Write a sample using FooDataWriter_write().

For more information, see the example HelloWorld_zero_copy, or generate an example for a type annotated with @transfer_mode(SHMEM_REF):

rtiddsgen -example -micro -language C HelloWorld.idl

4.13.3.1. Writer Side

Best practice for writing samples annotated with @transfer_mode(SHMEM_REF):

for (int i = 0; i < 10; i++)
{
    Foo* sample;
    DDS_ReturnCode_t dds_rc;
    /* NEW API
       IMPORTANT - call get_loan each time when writing a NEW sample
     */
    dds_rc = FooDataWriter_get_loan(hw_datawriter, &sample);

    if (dds_rc != DDS_RETCODE_OK)
    {
        printf("Failed to get a loan\n");
        return -1;
    }

    /* After this function returns with DDS_RETCODE_OK,
     * the middleware owns the sample
     */
    dds_rc = FooDataWriter_write(hw_datawriter, sample, &DDS_HANDLE_NIL);
}

4.13.3.2. Reader Side

DDS_ReturnCode_t dds_rc;
dds_rc = FooDataReader_take(...)

/* process sample here */

dds_rc = FooDataReader_is_data_consistent(hw_reader,
                                           &is_data_consistent,
                                           sample,sample_info);

if (dds_rc != DDS_RETCODE_OK)
{
    /* Error occurred when performing the consistency check */
}

if (is_data_consistent)
{
    /* Sample is consistent. Processing of sample is valid */
}
else
{
    /* Sample is NOT consistent. Any processing of the sample should
     * be discarded and considered invalid.
     */
}

4.13.4. Synchronization of Zero Copy Samples

There is NO synchronization of a zero copy sample between a sender (DataWriter) and receiver (DataReader) application. It is possible for a sample’s content to be invalidated before the receiver application actually has had a chance to read it.

To illustrate this scenario, consider creating the case of creating a Best-effort DataWriter with max_samples of X=1. When the DataWriter is created the middleware will pre-allocate a pool of X+1 (2) samples residing in a shared memory region. This pool will be used to loan samples when calling FooDataWriter_get_loan(…) ,

DDS_ReturnCode_t ddsrc;
Foo* sample;

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
sample->value = 10000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
 * Because the datawriter is using best effort, the middleware immediately
 * makes this sample available to be returned by another FooDataWriter_get_loan(...
 */

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 2 */
sample->value = 20000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
 * Because the datawriter is using best effort, the middleware immediately
 * makes this sample available to be returned by another FooDataWriter_get_loan(...
 */

/*
 * At this point, it is possible the sample has been received by the receiving application
 * but has not been presented yet to the user.
 */

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
/* sample->value will contain the integer 10000 because we are re-using samples
 * from a list that contains only 2 buffers.
 *
 * Also, at this point in time a referemce to sample 1 and 2 may have already been received
 * by the middleware on the DataReader side and are lying inside a DataReader's internal cache.
 * However, the sample may not have been received by the
 * application. If at this point the sample's value (sample->value) was changed to 999,
 * the sample returned from the Subscribers
 * read(...) or take(...) would contain unexpected values (999 instead of 10000). This is because
 * both the Publisher and the Subscriber process have mapped into their virtual
 * address space the same shared memory region where the sample lies.
 *
 * Use **FooDataReader_is_data_consistent** to verify the consistency, to prevent this
 * scenario.
 *
 * Note, a sample is actually invalidated right after the completion
 * of FooDataWriter_get_loan(dw, &sample). If the address of the newly created sample has been
 * previously written and its contents has not been read by the receiver application,
 * then the previously written sample has been invalidated.
 */

ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);

4.13.5. Caveats

  • After you call FooDataWriter_write(), the middleware takes ownership of the sample. It is no longer safe to make any changes to the sample that was written. If, for whatever reason, you call FooDataWriter_get_loan() but never write the sample, you must call FooDataWriter_discard_loan() to return the sample back to FooDataWriter. Otherwise, subsequent FooDataWriter_get_loan may fail, because the FooDataWriter has no samples to loan.
  • The current maximum supported sample size is a little under the maximum value of a signed 32-bit integer. For that reason, do not use any samples greater than 2000000000 bytes.

4.13.6. Further Information