4.13. Zero Copy Transfer Over Shared Memory¶
This section is organized as follows:
4.13.1. Overview¶
Zero Copy transfer over shared memory allows large samples to be transmitted with a minimum number of copies. These samples reside in a shared memory region accessible from multiple processes. When creating a FooDataWriter that supports Zero Copy Transfer of user samples, a sample must be created with a new non-DDS API (FooDataWriter_get_loan(…)). This will return a pointer A* to a sample Foo that lies inside a shared memory segment. A reference to this sample will be sent to a receiving FooDataReader across the shared memory. This FooDataReader will attach to a shared memory segment and a pointer B* to sample Foo will be presented to the user. Because the two processes shared different memory spaces, A* and B* will be different but they will point to the same place in RAM.
This feature requires the usage of new RTI DDS Extension APIs:
- FooDataWriter_get_loan()
- FooDataWriter_discard_loan()
- FooDataReader_is_data_consistent()
For detailed information, see the C API Reference and C++ API Reference.
4.13.2. Getting Started¶
To enable Zero Copy transfer over shared memory, follow these steps:
Annotate your type with the @transfer_mode(SHMEM_REF) annotation. Currently, variable-length types (strings and sequences) are not supported for types using this transfer mode when the language binding is INBAND.
@transfer_mode(SHMEM_REF) struct HelloWorld { long id; char raw_image_data[1024 * 1024]; // 1 MB };
Register the Shared Memory Transport (see Registering the SHMEM Transport). References will be sent across the shared memory transport.
Create a FooDataWriter for the above type.
Get a loan on a sample using FooDataWriter_get_loan().
Write a sample using FooDataWriter_write().
For more information, see the example HelloWorld_zero_copy, or generate an example for a type annotated with @transfer_mode(SHMEM_REF):
rtiddsgen -example -micro -language C HelloWorld.idl
4.13.2.1. Writer Side¶
Best practice for writing samples annotated with @transfer_mode(SHMEM_REF):
for (int i = 0; i < 10; i++)
{
Foo* sample;
DDS_ReturnCode_t dds_rc;
/* NEW API
IMPORTANT - call get_loan each time when writing a NEW sample
*/
dds_rc = FooDataWriter_get_loan(hw_datawriter, &sample);
if (dds_rc != DDS_RETCODE_OK)
{
printf("Failed to get a loan\n");
return -1;
}
/* After this function returns with DDS_RETCODE_OK,
* the middleware owns the sample
*/
dds_rc = FooDataWriter_write(hw_datawriter, sample, &DDS_HANDLE_NIL);
}
4.13.2.2. Reader Side¶
DDS_ReturnCode_t dds_rc;
dds_rc = FooDataReader_take(...)
/* process sample here */
dds_rc = FooDataReader_is_data_consistent(hw_reader,
&is_data_consistent,
sample,sample_info);
if (dds_rc == DDS_RETCODE_OK)
{
if (is_data_consistent)
{
/* Sample is consistent. Processing of sample is valid */
}
else
{
/* Sample is NOT consistent. Any processing of the sample should
* be discarded and considered invalid.
*/
}
}
4.13.3. Synchronization of Zero Copy Samples¶
There is NO synchronization of a zero copy sample between a sender (DataWriter) and receiver (DataReader) application. It is possible for a sample’s content to be invalidated before the receiver application actually has had a chance to read it.
To illustrate this scenario, consider creating the case of creating a Best-effort DataWriter with max_samples of X=1. When the DataWriter is created the middleware will pre-allocate a pool of X+1 (2) samples residing in a shared memory region. This pool will be used to loan samples when calling FooDataWriter_get_loan(…) ,
DDS_ReturnCode_t ddsrc;
Foo* sample;
ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
sample->value = 10000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
* Because the datawriter is using best effort, the middleware immediately
* makes this sample available to be returned by another FooDataWriter_get_loan(...
*/
ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 2 */
sample->value = 20000;
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
/*
* Because the datawriter is using best effort, the middleware immediately
* makes this sample available to be returned by another FooDataWriter_get_loan(...
*/
/*
* At this point, it is possible the sample has been received by the receiving application
* but has not been presented yet to the user.
*/
ddsrc = FooDataWriter_get_loan(dw, &sample); /* returns pointer to sample 1 */
/* sample->value will contain the integer 10000 because we are re-using samples
* from a list that contains only 2 buffers.
*
* Also, at this point in time a referemce to sample 1 and 2 may have already been received
* by the middleware on the DataReader side and are lying inside a DataReader's internal cache.
* However, the sample may not have been received by the
* application. If at this point the sample's value (sample->value) was changed to 999,
* the sample returned from the Subscribers
* read(...) or take(...) would contain unexpected values (999 instead of 10000). This is because
* both the Publisher and the Subscriber process have mapped into their virtual
* address space the same shared memory region where the sample lies.
*
* Use **FooDataReader_is_data_consistent** to verify the consistency, to prevent this
* scenario.
*
* Note, a sample is actually invalidated right after the completion
* of FooDataWriter_get_loan(dw, &sample). If the address of the newly created sample has been
* previously written and its contents has not been read by the receiver application,
* then the previously written sample has been invalidated.
*/
ddsrc = FooDataWriter_write(datawriter, sample, &DDS_HANDLE_NIL);
4.13.4. Caveats¶
- After you call FooDataWriter_write(), the middleware takes ownership of the sample. It is no longer safe to make any changes to the sample that was written. If, for whatever reason, you call FooDataWriter_get_loan() but never write the sample, you must call FooDataWriter_discard_loan() to return the sample back to FooDataWriter. Otherwise, subsequent FooDataWriter_get_loan may fail, because the FooDataWriter has no samples to loan.
- The current maximum supported sample size is a little under the maximum value of a signed 32-bit integer. For that reason, do not use any samples greater than 2000000000 bytes.