5.14. Zero Copy Transfer Over Shared Memory

This section is organized as follows:

5.14.1. Overview

Zero Copy transfer over shared memory allows large samples to be transmitted with a minimum number of copies. These samples reside in a shared memory region accessible from multiple processes. When creating a FooDataWriter that supports Zero Copy Transfer of user samples, a sample must be created with a new non-DDS API (FooDataWriter_get_loan(…)). This will return a pointer A* to a sample Foo that lies inside a shared memory segment. A reference to this sample will be sent to a receiving FooDataReader across the shared memory. This FooDataReader will attach to a shared memory segment and a pointer B* to sample Foo will be presented to the user. Because the two processes shared different memory spaces, A* and B* will be different but they will point to the same place in RAM.

This feature requires the usage of new RTI DDS Extension APIs:

  • FooDataWriter_get_loan()

  • FooDataWriter_discard_loan()

  • FooDataReader_is_data_consistent()

For detailed information, see the C API Reference and C++ API Reference.

5.14.2. Getting Started

To enable Zero Copy transfer over shared memory, follow these steps:

  1. Annotate your type with the @transfer_mode(SHMEM_REF) annotation.

    Currently, variable-length types (strings and sequences) are not supported for types using this transfer mode when a type is annotated with the PLAIN language binding (e.g., @language_binding(PLAIN) in IDL).

    @transfer_mode(SHMEM_REF)
struct HelloWorld
{
    long id;
    char raw_image_data[1024 * 1024]; // 1 MB
};

  2. Register the Shared Memory Transport (see Registering the SHMEM Transport). References will be sent across the shared memory transport.

  3. Create a FooDataWriter for the above type.

  4. Get a loan on a sample using FooDataWriter_get_loan().

  5. Write a sample using FooDataWriter_write().

For more information, see the example HelloWorld_zero_copy, or generate an example for a type annotated with @transfer_mode(SHMEM_REF):

rtiddsgen -example -micro -language C HelloWorld.idl

5.14.2.1. Writer Side

Best practice for writing samples annotated with @transfer_mode(SHMEM_REF):

for (int i = 0; i < 10; i++)
{
    Foo* sample;
    DDS_ReturnCode_t dds_rc;
    /* NEW API
       IMPORTANT - call get_loan each time when writing a NEW sample
     */
    dds_rc = FooDataWriter_get_loan(hw_datawriter, &sample);

    if (dds_rc != DDS_RETCODE_OK)
    {
        printf("Failed to get a loan\n");
        return -1;
    }

    /* After this function returns with DDS_RETCODE_OK,
     * the middleware owns the sample
     */
    dds_rc = FooDataWriter_write(hw_datawriter, sample, &DDS_HANDLE_NIL);
}

5.14.2.2. Reader Side

DDS_ReturnCode_t dds_rc;
dds_rc = FooDataReader_take(...)

/* process sample here */

dds_rc = FooDataReader_is_data_consistent(hw_reader,
                                           &is_data_consistent,
                                           sample,sample_info);

if (dds_rc == DDS_RETCODE_OK)
{
     if (is_data_consistent)
     {
         /* Sample is consistent. Processing of sample is valid */
     }
     else
     {
        /* Sample is NOT consistent. Any processing of the sample should
         * be discarded and considered invalid.
         */
     }
}

5.14.3. Synchronization of Zero Copy Samples

There is NO synchronization of a zero copy sample between a sender (DataWriter) and receiver (DataReader) application. It is possible for a sample’s content to be invalidated before the receiver application actually has had a chance to read it.

To illustrate this scenario, consider creating the case of creating a Best-effort DataWriter with max_samples of X=1. When the DataWriter is created the middleware will pre-allocate a pool of X+1 (2) samples residing in a shared memory region. This pool will be used to loan samples when calling FooDataWriter_get_loan(…) ,

DDS_ReturnCode_t ddsrc;
Foo* sample;

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
sample->value = 10000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
 * Because the datawriter is using best effort, the middleware immediately
 * makes this sample available to be returned by another FooDataWriter_get_loan(...
 */

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 2 */
sample->value = 20000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
 * Because the datawriter is using best effort, the middleware immediately
 * makes this sample available to be returned by another FooDataWriter_get_loan(...
 */

/*
 * At this point, it is possible the sample has been received by the receiving application
 * but has not been presented yet to the user.
 */

ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
/* sample->value will contain the integer 10000 because we are re-using samples
 * from a list that contains only 2 buffers.
 *
 * Also, at this point in time a referemce to sample 1 and 2 may have already been received
 * by the middleware on the DataReader side and are lying inside a DataReader's internal cache.
 * However, the sample may not have been received by the
 * application. If at this point the sample's value (sample->value) was changed to 999,
 * the sample returned from the Subscribers
 * read(...) or take(...) would contain unexpected values (999 instead of 10000). This is because
 * both the Publisher and the Subscriber process have mapped into their virtual
 * address space the same shared memory region where the sample lies.
 *
 * Use **FooDataReader_is_data_consistent** to verify the consistency, to prevent this
 * scenario.
 *
 * Note, a sample is actually invalidated right after the completion
 * of FooDataWriter_get_loan(dw, &sample). If the address of the newly created sample has been
 * previously written and its contents has not been read by the receiver application,
 * then the previously written sample has been invalidated.
 */

ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);

5.14.4. Caveats

  • After you call FooDataWriter_write(), the middleware takes ownership of the sample. It is no longer safe to make any changes to the sample that was written. If, for whatever reason, you call FooDataWriter_get_loan() but never write the sample, you must call FooDataWriter_discard_loan() to return the sample back to FooDataWriter. Otherwise, subsequent FooDataWriter_get_loan may fail, because the FooDataWriter has no samples to loan.

  • The current maximum supported sample size is a little under the maximum value of a signed 32-bit integer. For that reason, do not use any samples greater than 2000000000 bytes.

5.14.5. Further Information

For more information, see the section on Zero Copy Transfer Over Shared Memory in the RTI Connext DDS Core Libraries User’s Manual (available here if you have Internet access).