32.3 Overview of the Reliable Protocol

An important advantage of Connext is that it can offer the reliability and other QoS guarantees mandated by DDS on top of a very wide variety of transports, including packet-based transports, unreliable networks, multicast-capable transports, bursty or high-latency transports, etc. Connext is also capable of maintaining liveliness and application-level QoS even in the presence of sporadic connectivity loss at the transport level, an important benefit in mobile networks. Connext accomplishes this by implementing a reliable protocol that sequences and acknowledges application-level messages and monitors the liveliness of the link. This is called the Real-Time Publish-Subscribe (RTPS) protocol; it is an open, international standard.1 For a link to the RTPS specification, see the RTI website, www.rti.com.

In order to work in this wide range of environments, the reliable protocol defined by RTPS is highly configurable with a set of parameters that let the application fine-tune its behavior to trade-off latency, responsiveness, liveliness, throughput, and resource utilization. This section describes the most important features to the extent needed to understand how the configuration parameters affect its operation.

The most important features of the RTPS protocol are:

• Support for both push and pull operating modes
• Support for both positive and negative acknowledgments
• Support for high data-rate DataWriters
• Support for multicast DataReaders
• Support for high-latency environments

In order to support these features, RTPS uses several types of messages: Data messages (DATA), acknowledgments (ACKNACKs), and heartbeats (HBs).

• DATA messages contain snapshots of the value of data-objects and associate the snapshot with a sequence number that Connext uses to identify them within the DataWriter’s history. These snapshots are stored in the history as a direct result of the application calling write() on the DataWriter. Incremental sequence numbers are automatically assigned by the DataWriter each time write() is called. In Figure 32.1: Basic RTPS Reliable Protocol through 32.4 Using QosPolicies to Tune the Reliable Protocol, these messages are represented using the notation DATA(<value>, <sequenceNum>). For example, DATA(A,1) represents a message that communicates the value ‘A’ and associates the sequence number ‘1’ with this message. A DATA message is used for both keyed and non-keyed data types.
• HB messages announce to the DataReader that it should have received all snapshots up to the one tagged with a range of sequence numbers and can also request the DataReader to send an acknowledgement back. For example, HB(1-3) indicates to the DataReader that it should have received snapshots tagged with sequence numbers 1, 2, and 3 and asks the DataReader to confirm this.
• ACKNACK messages communicate to the DataWriter that particular snapshots have been successfully stored in the DataReader’s history. ACKNACKs also tell the DataWriter which snapshots are missing on the DataReader side. The ACKNACK message includes a set of sequence numbers represented as a bit map. The sequence numbers indicate which ones the DataReader is missing. (The bit map contains the base sequence number that has not been received, followed by the number of bits in bit map and the optional bit map. The maximum size of the bit map is 256.) All numbers up to (not including) those in the set are considered positively acknowledged. They are represented in Figure 32.1: Basic RTPS Reliable Protocol through Figure 32.7: Use of heartbeat_period as ACKNACK(<first-missing>) or ACKNACK(<first-missing>-<last-missing>). For example, ACKNACK(4) indicates that the snapshots with sequence numbers 1, 2, and 3 have been successfully stored in the DataReader history, and that 4 has not been received.

It is important to note that Connext can bundle multiple of the above messages within a single network packet. This ‘submessage bundling’ provides for higher performance communications.

It is also worth noting that because HB and ACKNACK messages communicate the state of reliable communication between individual writer and reader pairs, Connext requires at least one unicast destination so that these messages can be sent to the correct destinations, as opposed to being broadcast over a multicast destination. Connext does support enabling sending periodic heartbeats to a multicast destination using the enable_multicast_periodic_heartbeat in the 44.10 WIRE_PROTOCOL QosPolicy (DDS Extension).

Figure 32.1: Basic RTPS Reliable Protocol

Figure 32.1: Basic RTPS Reliable Protocol illustrates the basic behavior of the protocol when an application calls the write() operation on a DataWriter that is associated with a DataReader. As mentioned, the RTPS protocol can bundle multiple submessages into a single network packet. In Figure 32.1: Basic RTPS Reliable Protocol this feature is used to piggyback a HB message to the DATA message. Note that before the message is sent, the data is given a sequence number (1 in this case) which is stored in the DataWriter’s send queue. As soon as the message is received by the DataReader, it places it into the DataReader’s receive queue. From the sequence number the DataReader can tell that it has not missed any messages and therefore it can make the data available immediately to the user (and call the DataReaderListener). This is indicated by the “ü” symbol. The reception of the HB(1) causes the DataReader to check that it has indeed received all updates up to and including the one with sequenceNumber=1. Since this is true, it replies with an ACKNACK(2) to positively acknowledge all messages up to (but not including) sequence number 2. The DataWriter notes that the update has been acknowledged, so it no longer needs to be retained in its send queue. This is indicated by the “ü” symbol.

Figure 32.2: RTPS Reliable Protocol in the Presence of Message Loss

Figure 32.2: RTPS Reliable Protocol in the Presence of Message Loss illustrates the behavior of the protocol in the presence of lost messages. Assume that the message containing DATA(A,1) is dropped by the network. When the DataReader receives the next message (DATA(B,2); HB(1-2)) the DataReader will notice that the data associated with sequence number 1 was never received. It realizes this because the heartbeat HB(1-2) tells the DataReader that it should have received all messages up to and including the one with sequence number 2. This realization has two consequences:

• The data associated with sequence number 2 (B) is tagged with ‘X’ to indicate that it is not deliverable to the application (that is, it should not be made available to the application, because the application needs to receive the data associated with DDS sample 1 (A) first).
• An ACKNACK(1) is sent to the DataWriter to request that the data tagged with sequence number 1 be resent.

Reception of the ACKNACK(1) causes the DataWriter to resend DATA(A,1). Once the DataReader receives it, it can ‘commit’ both A and B such that the application can now access both (indicated by the “ü”) and call the DataReaderListener. From there on, the protocol proceeds as before for the next data message (C) and so forth.

A subtle but important feature of the RTPS protocol is that ACKNACK messages are only sent as a direct response to HB messages. This allows the DataWriter to better control the overhead of these ‘administrative’ messages. For example, if the DataWriter knows that it is about to send a chain of DATA messages, it can bundle them all and include a single HB at the end, which minimizes ACKNACK traffic.