RTI transport  Version 1.0af [build 01]
NDDS Transport-Plugin API

Documentation for implementing a Transport Plugin using NDDS's abstract pluggable-transport interface. More...

Modules

 Methods
 Methods that every Transport-Plugin implementation must provide.
 
 Resources
 Handle to information needed to send or receive messagees by a Transport Plugin.
 
 Message
 Support for a message received from a Transport Plugin.
 
 Listener
 The API used by a Transport Plugin to notify the NDDS core of dynamic changes in the Transport Plugin.
 

Detailed Description

Documentation for implementing a Transport Plugin using NDDS's abstract pluggable-transport interface.

The Methods abstracts the actual mechanism that NDDS uses to send and receive messages. It is the API between the NDDS core and the "transport" layer.

Overview

The goal of this module is to provide a general API that can be mapped into any desired "physical transport" such as IP sockets, reflective memory, serial, wireless or other transports.

The API has been designed for high-performance and allows NDDS to take advantage of scatter-gather, zero-copy and other high-performance facilities offered by a particular transport.

A transport is modeled as a facility that provides the means to describe the address of the receiver of a message, send messages and receive messages. The transport treats the messages themselves as an opaque packet of bytes of a given length. For NDDS, the messages passed to the transport layer are actually RTPS messages, with each message formated following the Real-Time Publish Subscribe on-the-wire protocol.

A Transport Plugin is created by providing a set of functions that define how to create, destroy, and use resources to send and receive messages (conceptually, SendResources and RecvResources). Exactly what these "resources" are will depend on the implementation of the Transport Plugin itself. For example, for IP socket communications, one of the resources will be a handle, nominally a socket descriptor, to the socket used to send or receive data.

Together, these functions are grouped into the Methods. Code that implements the Methods will herein be referred to as a "Transport Plugin".

Since NDDS is an implementation of the DDS standard, there are some explanations of the Transport-Plugin design that will tie into concepts introduced by DDS. Thus, this documentation assumes some level of familiarity with terminology of the DDS standard.

Design

The following paragraphs explain the design of the Methods, as well as some of the rationale behind the design. First note, there are multiple entities involved with the API of a Transport Plugin.

The first is the implementor of a Transport Plugin. This person is responsible for creating code that meets the API specification as well as properly interacts with whatever transport hardware/software that it uses for communications.

Another is NDDS itself. The NDDS core code will establish contact and exchange data with other NDDS applications using a Transport Plugin via its API.

Finally, there is the creator of an application that uses NDDS who also wants to use a particular Transport Plugin. This person, also known as an "end user", will need to call the API provided by the Transport Plugin to instantiate and configure the plugin for use by NDDS in their application.

Some portions of the Transport-Plugin API will only be used by the NDDS core, and others, only by the end user.

Basic Elements

Transport Properties

A Transport-Plugin implementation needs to provide values for a well-defined set of general, transport-independent properties so that the NDDS core code understands some limits of the plugin, for example, maximum message size. These properties are defined by the NDDS_Transport_Property_t structure.

These properties are immutable. Once a Transport Plugin has been registered with the NDDS core, the value of these properties cannot change while the plugin exists.

It is important that the properties on the sending side are compatible with those on the receiving side of communicating applications using different instances of same Transport-Plugin class. For example, if one side is configured to send messages larger than can be received by the other side, then communications may fail through the plugin.

Of course, this does not preclude the Transport-Plugin implementor from creating plugins that somehow verify that they have compatible properties when connections are established. However, the NDDS core itself will only use the values of the properties that were set when the plugin instance is initially registered.

Addressing in Transport Plugins

The destination of a message is represented by an address and a port number. The address is represented using a 16-byte number in the IPv6 notation, see Address. A transport will use this address to determine how to send the message to a remote machine. In addition, the port number may be used by some transports to further identify to which process or thread on the remote machine the message is delivered.

The address will be the address of an interface of a Transport Plugin, see Interface. An instance of a Transport Plugin that has multiple interfaces can receive messages that have different destination addresses.

A specific Transport Plugin implementation may not need nor be able to address all 2^128 possible values in a 16-byte address space. It will be up to each implementation of a Transport Plugin to determine how many addresses it needs from the 16-byte address space, and how to map those addresses to whatever addressing scheme that is native to the transport itself.

Examples
A Transport Plugin based on IPv4 will never need more than 4 bytes for its destination addresses. In addition, for IPv4, the mapping of the 4 bytes of an IPv4 address to the required 16-byte addressing scheme of the NDDS Transport-Plugin API is trivial.
Transport Plugins that are based on dedicated point-to-point links, such as a serial port, may not need any addresses at all (recall, they are only needed to allow the transport to determine to which destination interface it sends a message). Given a message to send, a Serial-based plugin can only send it to one destination, the one on the other end of the serial port. Again, the mapping to a 16-byte address is trivial.
Some transports may not use numbers in their native addressing schemes, for example, Star Fabric by Stargen. Now it will be up to the implementor of an NDDS plugin for the transport to create a unique mapping from the native addressing to a 16-byte numeric value and vice versa.

Thus, all transports must map their own internal representation of "destination addresses" to a numeric 16-byte representation. The Transport Plugin should only use as many bits of the 16-byte address space as required by the transport. The remaining bits will be used by NDDS to as a way to uniquely identify different transports that have overlapping address spaces. See NDDS_Transport_Property_t::address_bit_count.

For example, an IPv4-UDP Transport Plugin has a 4-byte address space (0:0:0:0:0:0:xxxx:xxxx), so its address_bit_count is 96. A Transport Plugin that can use up to 256 serial ports simultaneously only needs a 1-byte address space (0:0:0:0:0:0:0:00xx, address_bit_count is 120). However, if an application uses both plugins, then the address 0:0:0:0:0:0:0001 is ambiguous because it is a valid destination address in both plugins.

Thus, the end user must be able to configure NDDS to use both Transport Plugins such that the addresses used by NDDS are unique and non-overlapping across all transports. So the concept of a network address is introduced.

The network address is the the portion of a 16-byte address that is not used by a Transport Plugin. This can vary from 0 bits to 128 bits. The address_bit_count of the structure NDDS_Transport_Property_t is used to inform the NDDS core how many bits of the address are significant to the transport. Whatever is left, 128 - address_bit_count, can be used as the network address.

Just as long there is at least 1 bit available for the network address, NDDS can use that one bit to distinguish a single instance of that transport from other transports.

For example, for the UDP and Serial Transport Plugins previously described, the network address will be 12 bytes long for the UDP transport, and 15 bytes long for the Serial transport.

The yy portion of the address is the network address, the xx portion is used by the transport itself.

Now the end user can configure the Transport Plugins such that NDDS can uniquely determine which transport is used to send a message based its full 16-byte address. See for this configuration API.

For example, one can set the network address of the IPv4-UDP transport to 1000:0:0:0:0:0:xxxx:xxxx and the network address of the Serial transport to 2000:0:0:0:0:0:0:00xx. Then, messages with destination addresses that start with a 1 will be sent through the IPv4-UDP transport, and ones that start with a 2 will be sent through the serial transport.

Should there be more than just these two instances of Transport Plugins installed in the application, then the non-used portion of the network address would serve to create unique, non-overlapping address ranges for all plugins.

Note:
All instances of the same Transport Plugin in all of the different application connected via the same physical transport, that is, talking via the transport on the same "network", must be configured with the same network address.

See the configuration API for more information about registering a Transport Plugin with a network address.

No Need for Reliability at the Transport Level

The actual transport used by the Transport Plugin does not need to be reliable. Messages sent via a transport may be dropped or otherwise lost. NDDS implements a reliable protocol on top of the Methods to support the reliable messaging capabilities needed by the DDS API.

However, it is up to the Transport Plugin to ensure that the content of messages that are delivered to the NDDS core have not been altered in any way. So messages must be delivered without corruption and in their entirety. If a transport detects but cannot correct for corrupted or truncated messages, then it should drop those messages and rely on the NDDS core to resend messages as needed.

Multiple Transports

As implied in earlier discussions, an application may configure NDDS to use multiple transports simultaneously. By default, NDDS will automatically instantiate, configure and register built-in transports. These transports enable inter-process communications using system-shared memory (see NDDS_Transport_Shmem_Plugin), and unicast and multicast communications across IP networks using IPv4-UDP sockets (see NDDS_Transport_UDPv4_Plugin). The builtin Transport Plugins conform to the same API as any plugin that may be created by an NDDS user.

The NDDS user may instantiate, configure and register custom Transport Plugins and use them combined with or instead of the builtin transports as desired. The user may wish to implement their own version of the NDDS builtin Transport Plugins to customize the behavior of those transports in ways not supported by the builtin plugins. See NDDSTransportUserModule for more information.

Connectionless or Connection-Oriented

The Transport-Plugin API does not assume a connection-oriented or connectionless transports. One should be able to implement the API using either a connection-oriented, for example, TCP, or connectionless, for example, UDP, transport.

Network Interfaces

A Transport Plugin may "manage" more than a single network interface in a single instance of the plugin. What is a network interface? While it may translate to different physical devices for a particular type of transport, a network interface can be defined as a unique address to which other interfaces on the same network can use to send messages (see Interface).

For example, for IP-based transports, there maybe many different interfaces, both physical (actual NIC cards) and virtual (e.g., VPN tunnels), installed on a computer. The implementor of a Transport Plugin that uses IP-based transport may chose to design the plugin so that a single instance can use (send and receive) over all of the IP interfaces on the computer simultaneously. Or a less sophistocated design may limit an instance to only attach to a single interface. Or a more sophistocated design may allow the user to select which network interfaces an instance is allowed to use (as is the case with the NDDS_Transport_UDPv4_Plugin).

NDDS will call a function provided by the plugin, get_receive_interfaces_cEA(), to get a list of interfaces that a plugin represents (uses). In the following discussions on sending and receiving using the Transport Plugin, it will be important to note when the action is expected to affect all interfaces that a plugin manages or only a subset (Sending with Multiple Interfaces and Receiving with Multiple Interfaces.)

The Sending Side

This section describes the interface to a Transport Plugin for sending messages.

Gather Send

When NDDS sends a message through a Transport Plugin, it will provide the Transport Plugin with an array of buffers. It is the responsibility of the Transport Plugin to concatenate this array of buffers into a single message that is delivered. The concatenation of the all of the buffers in the array is the actual message that NDDS wants to be received on the other side of the transport.

The receiving side of the Transport Plugin should only present a full message to NDDS and not individually present each buffer that makes up the message. So while on the sending side the Transport Plugin can send the array of buffers a single buffer at a time, on the receiving side it should wait until all of the buffers have been received before a full message is delivered to NDDS.

This approach presents a processing-time-efficient interface between the NDDS core and the Transport Plugin. It avoids the need for an additional copy by NDDS to coalesce the sub-parts of a message into one consecutive buffer for sending. This approach also allows the Transport Plugin to take advantage of physical transports that also provide a "gather-send" API such as IP sockets.

In the case that the underlying transport mechanism can only send single buffer, the implementation of the Transport Plugin will either need to copy the buffers in the buffer array into one consecutive buffer before sending or put some logic in place to detect when all buffers that have been sent individually have been received.

Only Synchronous Sends

The "send"-call of the Transport-Plugin API should be synchronous. That is, when the "send"-call returns, the NDDS core can assume that the message has been sent by the plugin and that the buffer array passed to the "send"-call are free to be reused.

Note that performance tests on several real-time operating systems showed no significant performance improvements from using asynchronous, zero-copy approaches on the sending side. In general, taking advantage of an asynchronous send is much more complicated for the design of the NDDS core and is currently not supported by NDDS.

The SendResource Concept

Before NDDS can send a message to a destination, NDDS must determine which Transport Plugin is capable of sending to the destination (IPv6-Address + Port combination). It does so by "pre-registering" a destination with a Transport Plugin. In other words, the NDDS core will call a function in the Transport Plugin (create_sendresource_srEA()) to create whatever resources the transport needs to send to a destination before NDDS tries to send a message to the destination.

The Transport Plugin should determine whether or not it is capable of sending to the destination (address/port). If it can, then the plugin should create or otherwise initialize whatever underlying transport mechanisms are required to do so and return a handle to those results as a "SendResource". NDDS will return the same handle to the Transport Plugin when it later tries to send a message to that destination.

For certain types of transports, the plugin may not need to do anything different to send to different destinations. In that case, the plugin may be able to share the same SendResource amoung multiple destinations. Thus, whenever NDDS find a new destination for a Transport Plugin, it will iterate through the existing SendResources of a Transport Plugin and call share_sendresource_srEA() to try to share one before creating a new SendResource by calling create_sendresource_srEA().

An example of when NDDS finds a new destination for a Transport Plugin is when NDDS detects a new DDS DataReader has been created in a remote application for a local DDS DataWriter.

When NDDS determines that it does not need to send to a particular destination over a particular Transport Plugin, for example, if a DataReader is destroyed in the remote application, then NDDS will inform the Transport Plugin that the destination is no longer used by either calling unshare_sendresource_srEA() or destroy_sendresource_srEA() with the SendResource linked to the destination. Exactly which function is called is determined by whether or not the SendResource has been shared amoung multiple destinations.

What destroying or unsharing a SendResource means for a particular Transport-Plugin implementation is entirely up to the implementation. It could be anything from closing a socket and releasing related resources to doing nothing at all.

To give an idea of how often NDDS will try to share or create a SendResource for a particular instance of a Transport Plugin, consider a simple NDDS application with a single participant (the following discussion assumes familiarity with DDS concepts and some knowledge of how NDDS application discovery works and is configured).

Then for all peers in the peer-locator list, , that have an address that can be serviced by the transport (must have the same network address as the Transport Plugin, see Addressing in Transport Plugins), NDDS will try to share/create a SendResource (because it will try send messages to that peer).

In addition, for any peer that it discovers (are actually alive and have exchanged messages with), NDDS will receive a list of interfaces/ports from the peer to which the application should send RTPS metatraffic packets. So for any of the remote peer interfaces that can be reached by a Transport Plugin, NDDS will try to share/create a SendResource for the plugin.

Then after discovery is complete, as NDDS discovers remote DataReaders that match local DataWriters, it will receive a list of interface/port combinations that should be used to send data to the remote DataReader. Again, for any of the interfaces that can be serviced by the Transport Plugin, NDDS will try to share/create a SendResource for the plugin.

So in the simplest application with a DataWriter that has a matching DataReader in another application, assuming both applications only have one Transport Plugin that manage a single interface, NDDS will try to share/create 2 SendResources; one for metatraffic, one for user-data traffic. The actual number of SendResources created may be only 1 if the Transport Plugin can share the same SendResource for metatraffic as well as user-data traffic.

NDDS will ask the Transport Plugin to share/create more SendResource as it discovers more peer applications, as more matching DataWriter/DataReader pairs exist, or if the end user configures applications to use multicast addresses or specify ports for DataReaders to receive messages on other than the default port.

NDDS will only ask a Transport Plugin to share/create new SendResources for new unique address/port combinations, so that the Transport Plugin implementation will never be asked to share or create a SendResource for an address/port for which it has already shared or created a SendResource.

Sending with Multiple Interfaces

When NDDS calls send(), it will be trying to send a message to a specific destination (address/port). For Transport Plugins that manage more than one interface (see Network Interfaces), the question of which interface should be used may arise. Unless the destination is a multicast message, then normally, the plugin should chose only a single interface through which it will send the message.

For example, a UDP packet sent by an IP plugin would normally be routed by the IP stack of the operating system so that it only gets sent out of a single interface in a multi-NIC system. Usually, a routing table determines which interface is used for which destination addresses.

So, while each Transport Plugin implementation may have a different way of handling multiple interfaces (details of which may be configurable by the end user), usually messages with a unicast destination are sent through single interface. However, an implementation may chose to send a unicast message redundantly through multiple interfaces. Also, since messages with a multicast address are supposed to be received by all remote interfaces that listen to the multicast address, a Transport Plugin should send multicast messages over all interfaces that support multicast.

While the NDDS core is designed to accomodate the processing of duplicate messages correctly (which would happen if the same message was received over multiple interfaces or even from different Transport Plugins), the implementor should be aware that extra resources (CPU) will be consumed in processing duplicate messages. Also depending on timing, NDDS may unnecessarily generate (and process) extra metatraffic as a result of duplicate messages.

Sending on Connection-Oriented Transports before the Receiver is Initialized

There is a chicken and egg problem when building a publish-subscribe networking mechanism, which is inherently connectionless, on top of connection-oriented transports. For one application to publish data to another, it needs to know that the other application has subscribed to its data. But first, the knowledge of that subscription must be sent by the subscription application to the publication application. Which application should connect to the other first? The publication application or the subscription application? Wait, aren't NDDS applications allowed to be both a publisher and a subscriber?

For connectionless transports, this is not much of a problem. One can send information without first establishing a connection. So the create_sendresource_srEA() and send() calls can succeed even if there are no other applications ready to receive the messages. So, there is no need to establish a connection before information is sent. If the receiving application is up, then it will receive the data. If it is not up or ready, then the data is lost, and it will be up to the sending application to resend the information.

In a connection-oriented transport, one usually cannot send before a connection is established. In the Transport-Plugin API, for a connection-oriented transport, a connection to a particular destination would be something that is usually establish in the create_sendresource_srEA() call. However, because applications are asynchronously started, the destination application may not be up or ready yet to complete a connection when create_sendresource_srEA() is called in the first application.

So when create_sendresource_srEA() returns, it is possible that a connection has not been fully initialized (connection not established) between the sender and the receiver applications. The create_sendresource_srEA() call of a connection-oriented Transport Plugin must not return a failure code if this is the case. create_sendresource_srEA() should only return failure if the plugin determines that there no possible way to send to the given destination using the plugin, not just because it could not establish a connection at that time.

If the connection could not be established by create_sendresource_srEA(), it should still create and return a SendResource to NDDS. The SendResource should contain some sort of indication that it is not fully initialized, that is, the connection to the receiving application has not been established.

Then, when NDDS needs to send a message to a destination using the SendResource, it will pass the partially-initialized SendResource in the send() command of the Transport Plugin. For connection-oriented transports, the send() call must check to see if a connection related to the SendResource is fully established. If not, the send() call itself should try to establish the connection before sending. It could be the case that the connection to the destination still cannot be made (the destination application is still not ready to accept connections). In that case send() should return 0 to indicate that no message was actually sent.

For all intents and purposes, NDDS will treat this case as if a message was sent by the plugin by somehow got dropped. NDDS will call the send() function of the Transport Plugin to send the message again. So, the Transport Plugin will have a periodic opportunity to make a connection to the destination application. Assuming that the receiving application finally does start, then the plugin will eventually establish a connection and send a message when send() is invoked by NDDS.

In this way, NDDS can use connection-oriented transports while being independent of which side of the transport is initialized first.

The Receiving Side

This section describes the interface to the Transport Plugin for receiving messages.

The RecvResource Concept

The complement to the SendResource for sending is the RecvResource for receiving. A RecvResource must be linked (associated) to a particular port (and possibly multicast address) to receive messages sent to the application via the transport. Like SendResources, a Transport Plugin may be able to share a RecvResource for multiple ports. For new ports for which NDDS will want to receive messages, NDDS will iteratively call the share_recvresource_rrEA() to share an existing RecvResource, or finally, call create_recvresource_rrEA() if none could be shared by the Transport Plugin.

Note that only a port is needed to define a RecvResource since the destination address of the message received by the transport will be the address of one of the interfaces managed by the Transport Plugin (see Receiving with Multiple Interfaces). A port is simply a way for NDDS to configure a transport to further route a message after it has reached the plugin. So, all messages sent to the same port should be received through the same RecvResource. What specific port numbers are used and why are determined by layers of NDDS higher than the Transport Plugin.
NDDS will create separate threads to receive on different RecvResources. Thus, calls to receive_rEA() on different RecvResources will be made by separate threads.

To give an idea of how often NDDS will try to share or create a RecvResource for a particular instance of a Transport Plugin, consider a simple NDDS application with a single participant.

When the participant is created, NDDS will try to first share, and then create if none can be shared, a RecvResource for unicast metatraffic for each instance of a Transport Plugin. If configured using , NDDS will also try to share/create RecvResources for multicast metatraffic.

Then if the application has DataReaders, NDDS will share/create a RecvResource to receive messages on a default port. By default, only one will be created for all DataReaders. However, the user can configure a DataReader to receive messages on ports other than the default using . The user can also configure a DataReader to receive messages on a multicast address using . When those DataReaders are created, NDDS will try to share/create RecvResources for the different ports and/or multicast addresses.

It is up to the Transport-Plugin implementation to determine whether a different RecvResource needs to be created for a different port/multicast address or if an existing RecvResource can be shared. Again, note that for each RecvResource created, NDDS will use a different thread to receive messages from the RecvResource.

If an application has DataWriters, NDDS will also create a default RecvResource to receive messages associated with reliable DataWriter. This RecvResource is used to receive the metatraffic such as ACKs from corresponding DataReaders in support of reliable messaging. By default, a single RecvResource is created for all DataWriters. However, like DataReaders, the user can configure a DataWriter to receive its metatraffic on different ports using . For those DataWriters, when they are created, NDDS will try to share/create a RecvResource for the specified ports.

In the simplest application with a DataWriter and a DataReader with default configurations and multicast is not being used, then NDDS will try to share/create 3 RecvResources (one for the discovery metatraffic, one for the DataWriter metatraffic, and one for the DataReader). NDDS will try to share/create more RecvResources if DataReaders and DataWriters are created that are configured to receive messages on anything but the default unicast port and/or using multicast.

What should a Transport Plugin do when asked to create a RecvResource?

In essence, the Transport Plugin must create all resources it needs and otherwise initialize the underlying transport hardware and software to begin receiving and storing (queuing) messages sent by another instance of the Transport Plugin. A handle to Transport-Plugin-specific resources for collecting those messages is passed back to NDDS as the RecvResource. When the upper layers of NDDS is ready to retrieve received messages from the RecvResource, NDDS will call receive_rEA().

Typically, the RecvResource represents a FIFO queue of incoming messages that are sent to ports associated with the RecvResource. How the queue is sized is up to the implementation. This can be a property of the Transport Plugin set at creation time or even dynamically adjusted for more complex implementations.

Receiving with Multiple Interfaces

If a transport manages multiple interfaces, then when NDDS creates a RecvResource, it expects the Transport Plugin to prepare to receive messages addressed to the given port on all of the interfaces.

For example, if a Serial Transport Plugin is configured to manage to multiple serial ports and thus has multiple interfaces, then when a RecvResource is created for the Serial Transport Plugin, the plugin should do what it needs to do to start receiving messages from all of the serial ports.

Blocking Receives

The NDDS core will call receive_rEA() to collect the messages stored by a RecvResource. Note that the RecvResource may store messages received for different destinations if the RecvResource was shared.

When receive_rEA() is called, it should return a single message from the RecvResource's queue. If there are no messages waiting in the queue, the receive_rEA() must block the caller thread until a message is received by the transport.

Should messages arrive for a RecvResource faster than the NDDS core calls receive_rEA() to collect the messages, then the Transport Plugin may run out of memory (space in the queue) to store the incoming messages. In this case, the Transport Plugin is allowed to quietly drop the messages. NDDS itself will resend messages if it determines that it needs to (for example, to support reliable messaging).

A Transport Plugin must implement the unblock_receive_rrEA() function as a way to unblock the thread calling the receive_rEA() function even if no messages are received. Primarily used when shutting down, another thread in the NDDS core will call the unblock_receive_rrEA() function to wake up the receive thread.

The receive_rEA() function should be aware that it was woken up by unblock_receive_rrEA() and return indicating that no message was received (0 bytes).

Note that if unblock_receive_rrEA() is called and there is no thread currently blocked in receive_rEA(), then the next call to receive_rEA() must return indicating that no message was received–even if there are received messages in the queue. This means that somehow the Transport Plugin must remember that unblock_receive_rrEA() was called but no thread were woken up by the call. When the receive thread finally calls receive_rEA(), the earlier call to unblock_receive_rrEA() will cause receive_rEA() to return immediately indicating no message was received (even if there are messages in the receive queue).
This "memory" (which can be implemented using a counting semaphore if a semaphore is used to block the receive thread) is needed to avoid race conditions when a thread trying to shutdown NDDS calls unblock_receive_rrEA(), but the receive thread is busy processing a received message and not actually blocked in receive_rEA().

Reception into a Contiguous Buffer

In contrast to the gather-send design of the sending side of the Transport Plugin, the receiving side must always store a received message into one contiguous memory area (buffer).

A Transport Plugin implementation may send a message however it needs to, for example, as a single packet, multiple packets, or a bit stream with headers and deliminators. The receive side should collect the bytes received until a message is received in its entirety and stored in a contiguous memory location. Only then should the message be stored in a queue for later retrieval via the receive_rEA() call.

Should the Transport Plugin determine that a part of the message is missing or has otherwise been corrupted, the entire message should be dropped. Incomplete or partial messages should never be passed to NDDS.

The Optional "Loaned-Buffer" Mechanism

In the receive_rEA() call, the Transport-Plugin implementation may return a received message back to NDDS by copying the message to an NDDS buffer that was passed in as an argument, buffer_in. Optionally, the Transport Plugin may ignore buffer_in, and instead, pass the received message back by returning a pointer to an "internal" buffer where the message is (already) stored.

This method is referred to as "loaning a buffer" to the NDDS core. When the NDDS core is done processing the message in the loaned-buffer, it will call a function provided by the Transport Plugin, plugin->return_loaned_buffer_rEA(), to return the loaned buffer back to the plugin. A loaned buffer will always be returned before NDDS calls receive_rEA() for the same RecvResource again.

The main reason for using a "loaned-buffer" mechanism is to support an efficient zero-copy receive implementation through the Transport Plugin to the NDDS core.

Some physical transports already have a zero-copy interface at the device-driver level. For example, in certain operating systems like VxWorks, the IP stack can itself loan a single buffer that contains the received message. In this case, the Transport Plugin should be able to pass this buffer directly to the NDDS core without the need for intermediate copies. This is the "zero-copy" optimization.

Transports can be divided into 3 classes.

The NDDS core itself will handle these three cases in a fairly transparent manner. Unless the NDDS_TRANSPORT_PROPERTY_BIT_BUFFER_ALWAYS_LOANED bit is set for a particular Transport Plugin, receive_rEA() will always be called with a buffer large enough to hold the largest message that the Transport Plugin is configured to receive.

If the Transport Plugin implementation installs a NDDS_Transport_PluginImpl::return_loaned_buffer_rEA function, then if the transport has loaned a buffer to NDDS in the receive_rEA() function, NDDS will call return_loaned_buffer_rEA() when it is finished processing the message.

If a message was not loaned, and return_loaned_buffer_rEA() should not be called, then the Transport Plugin must set the value of NDDS_Transport_Message_t::loaned_buffer_param to -1. NDDS will interpret any other value as an indication that the buffer in the message was loaned and call return_loaned_buffer_rEA() with that message.

Exclusive Areas (_xxEA)

You have have noted that the names of API functions that have to be implemented by a Transport Plugin have a cryptic postfix, either _rEA, _srEA, _rrEA or _cEA. In fact, the only one that does not is the send() function. These postfixes are used to indicate the level of multithreading safety that NDDS guarantees when it calls the function so that the implementation of these functions by a plugin may be simplified.

The _xxEA postfix is used to indicate how the functions are partitioned into different multithread safety groups, or exclusive areas.

Since the NDDS core is multithreaded, it is possible that multiple threads will be calling the API of an instance of a Transport Plugin simultaneously. However, the design of NDDS is such that this behavior is well specified. Functions have been group together into "Exclusive Areas (EAs)" so that NDDS guarantees that functions within the same EA will be called in a single threaded manner (usually with respect to some piece of data).

Since NDDS itself already has semaphores and mutexes to protect against multithreaded interactions within an EA, Transport-Plugin implementors can take advantage of that design and only create semaphores or mutexes when there may be transport-specific multithreaded interactions between functions in different EAs.

First all EAs apply to an instance of a Transport Plugin. No guarantees are given between different instances of the same Transport Plugin class. The EAs that have been defined are


RTI transport Version 1.0af [build 01] Copyright © Tue Feb 4 2014 Real-Time Innovations, Inc