Contents, Part 3

iii

iv

v

vi

vii

Chapter 10 Reliable Communications

Connext uses best-effort delivery by default. The other type of delivery that Connext supports is called reliable. This chapter provides instructions on how to set up and use reliable communication.

This chapter includes the following sections:

❏Sending Data Reliably (Section 10.1)

❏Overview of the Reliable Protocol (Section 10.2)

❏Using QosPolicies to Tune the Reliable Protocol (Section 10.3)

10.1Sending Data Reliably

The DCPS reliability model recognizes that the optimal balance between time-determinism and data-delivery reliability varies widely among applications and can vary among different publications within the same application. For example, individual samples of signal data can often be dropped because their value disappears when the next sample is sent. However, each sample of command data must be received and it must be received in the order sent.

The QosPolicies provide a way to customize the determinism/reliability trade-off on a per Topic basis, or even on a per DataWriter/DataReader basis.

There are two delivery models:

❏Best-effort delivery mode “I’m not concerned about missed or unordered samples.”

❏Reliable delivery model “Make sure all samples get there, in order.”

10.1.1Best-effort Delivery Model

By default, Connext uses the best-effort delivery model: there is no effort spent ensuring in-order delivery or resending lost samples. Best-effort DataReaders ignore lost samples in favor of the latest sample. Your application is only notified if it does not receive a new sample within a certain time period (set in the DEADLINE QosPolicy (Section 6.5.5)).

The best-effort delivery model is best for time-critical information that is sent continuously. For instance, consider a DataWriter for the value of a sensor device (such as a the pressure inside a tank), and assume the DataWriter sends samples continuously. In this situation, a DataReader for this Topic is only interested in having the latest pressure reading available—older samples are obsolete.

10-1

10.1.2Reliable Delivery Model

Reliable delivery means the samples are guaranteed to arrive, in the order published.

The DataWriter maintains a send queue with space to hold the last X number of samples sent. Similarly, a DataReader maintains a receive queue with space for consecutive X expected samples.

The send and receive queues are used to temporarily cache samples until Connext is sure the samples have been delivered and are not needed anymore. Connext removes samples from a publication’s send queue after the sample has been acknowledged by all reliable subscriptions. When positive acknowledgements are disabled (see DATA_WRITER_PROTOCOL QosPolicy (DDS Extension) (Section 6.5.3) and DATA_READER_PROTOCOL QosPolicy (DDS Extension) (Section 7.6.1)), samples are removed from the send queue after the corresponding keep- duration has elapsed (see Table 6.36, “DDS_RtpsReliableWriterProtocol_t,” on page 6-81).

If an out-of-order sample arrives, Connext speculatively caches it in the DataReader’s receive queue (provided there is space in the queue). Only consecutive samples are passed on to the

DataReader.

DataWriters can be set up to wait for available queue space when sending samples. This will cause the sending thread to block until there is space in the send queue. (Or, you can decide to sacrifice sending samples reliably so that the sending rate is not compromised.) If the DataWriter is set up to ignore the full queue and sends anyway, then older cached samples will be pushed out of the queue before all DataReaders have received them. In this case, the DataReader (or its Subscriber) is notified of the missing samples through its Listener and/or Conditions.

Connext automatically sends acknowledgments (ACKNACKs) as necessary to maintain reliable communications. The DataWriter may choose to block for a specified duration to wait for these acknowledgments (see Waiting for Acknowledgments in a DataWriter (Section 6.3.11)).

Connext establishes a virtual reliable channel between the matching DataWriter and all DataReaders. This mechanism isolates DataReaders from each other, allows the application to control memory usage, and provides mechanisms for the DataWriter to balance reliability and determinism. Moreover, the use of send and receive queues allows Connext to be implemented efficiently without introducing unnecessary delays in the stream.

Note that a successful return code (DDS_RETCODE_OK) from write() does not necessarily mean that all DataReaders have received the data. It only means that the sample has been added to the DataWriter’s queue. To see if all DataReaders have received the data, look at the RELIABLE_WRITER_CACHE_CHANGED Status (DDS Extension) (Section 6.3.6.7) to see if any samples are unacknowledged.

Suppose DataWriter A reliably publishes a Topic to which DataReaders B and C reliably subscribe. B has space in its queue, but C does not. Will DataWriter A be notified? Will DataReader C receive any error messages or callbacks? The exact behavior depends on the QoS settings:

❏If HISTORY_KEEP_ALL is specified for C, C will reject samples that cannot be put into the queue and request A to resend missing samples. The Listener is notified with the on_sample_rejected() callback (see SAMPLE_REJECTED Status (Section 7.3.7.8)). If A has a queue large enough, or A is no longer writing new samples, A won’t notice unless it checks the RELIABLE_WRITER_CACHE_CHANGED Status (DDS Extension) (Section 6.3.6.7).

❏If HISTORY_KEEP_LAST is specified for C, C will drop old samples and accept new ones. The Listener is notified with the on_sample_lost() callback (see SAMPLE_LOST Status (Section 7.3.7.7)). To A, it is as if all samples have been received by C (that is, they have all been acknowledged).

10-2

10.2Overview of the Reliable Protocol

An important advantage of Connext is that it can offer the reliability and other QoS guarantees mandated by DDS on top of a very wide variety of transports, including packet-based transports, unreliable networks, multicast-capable transports, bursty or high-latency transports, etc. Connext is also capable of maintaining liveliness and application-level QoS even in the presence of sporadic connectivity loss at the transport level, an important benefit in mobile networks. Connext accomplishes this by implementing a reliable protocol that sequences and acknowledges application-level messages and monitors the liveliness of the link. This is called the Real-Time Publish-Subscribe (RTPS) protocol; it is an open, international standard.1

In order to work in this wide range of environments, the reliable protocol defined by RTPS is highly configurable with a set of parameters that let the application fine-tune its behavior to trade-off latency, responsiveness, liveliness, throughput, and resource utilization. This section describes the most important features to the extent needed to understand how the configuration parameters affect its operation.

The most important features of the RTPS protocol are:

❏Support for both push and pull operating modes

❏Support for both positive and negative acknowledgments

❏Support for high data-rate DataWriters

❏Support for multicast DataReaders

❏Support for high-latency environments

In order to support these features, RTPS uses several types of messages: Data messages (DATA), acknowledgments (ACKNACKs), and heartbeats (HBs).

❏DATA messages contain snapshots of the value of data-objects and associate the snapshot with a sequence number that Connext uses to identify them within the DataWriter’s history. These snapshots are stored in the history as a direct result of the application calling write() on the DataWriter. Incremental sequence numbers are automatically assigned by the DataWriter each time write() is called. In Figure 10.1 through Figure 10.7, these messages are represented using the notation DATA(<value>, <sequenceNum>). For example, DATA(A,1) represents a message that communicates the value ‘A’ and associates the sequence number ‘1’ with this message. A DATA is used for both keyed and non-keyed data types.

❏HB messages announce to the DataReader that it should have received all snapshots up to the one tagged with a range of sequence numbers and can also request the DataReader to send an acknowledgement back. For example, HB(1-3) indicates to the DataReader that it should have received snapshots tagged with sequence numbers 1, 2, and 3 and asks the DataReader to confirm this.

❏ACKNACK messages communicate to the DataWriter that particular snapshots have been successfully stored in the DataReader’s history. ACKNACKs also tell the DataWriter which snapshots are missing on the DataReader side. The ACKNACK message includes a set of sequence numbers represented as a bit map. The sequence numbers indicate which ones the DataReader is missing. (The bit map contains the base sequence number that has not been received, followed by the number of bits in bit map and the optional bit map.

The maximum size of the bit map is 256.) All numbers up to (not including) those in the set are considered positively acknowledged. They are represented in Figure 10.1 through Figure 10.7 as ACKNACK(<first-missing>) or ACKNACK(<first-missing>-<last-

1. For a link to the RTPS specification, see the RTI website, www.rti.com.

10-3

missing>). For example, ACKNACK(4) indicates that the snapshots with sequence numbers 1, 2, and 3 have been successfully stored in the DataReader history, and that 4 has not been received.

It is important to note that Connext can bundle multiple of the above messages within a single network packet. This ‘submessage bundling’ provides for higher performance communications.

Figure 10.1 Basic RTPS Reliable Protocol

Assigned sequence number

History of send data values

Whether or not the sample has been delivered to the reader history

Figure 10.1 illustrates the basic behavior of the protocol when an application calls the write() operation on a DataWriter that is associated with a DataReader. As mentioned, the RTPS protocol can bundle multiple submessages into a single network packet. In Figure 10.1 this feature is used to piggyback a HB message to the DATA message. Note that before the message is sent, the data is given a sequence number (1 in this case) which is stored in the DataWriter’s send queue. As soon as the message is received by the DataReader, it places it into the DataReader’s receive queue. From the sequence number the DataReader can tell that it has not missed any messages and therefore it can make the data available immediately to the user (and call the DataReaderListener). This is indicated by the “✔” symbol. The reception of the HB(1) causes the DataReader to check that it has indeed received all updates up to and including the one with sequenceNumber=1. Since this is true, it replies with an ACKNACK(2) to positively acknowledge all messages up to (but not including) sequence number 2. The DataWriter notes that the update has been acknowledged, so it no longer needs to be retained in its send queue. This is indicated by the “✔” symbol.

Figure 10.2 illustrates the behavior of the protocol in the presence of lost messages. Assume that the message containing DATA(A,1) is dropped by the network. When the DataReader receives

10-4

Figure 10.2 RTPS Reliable Protocol in the Presence of Message Loss

the next message (DATA(B,2); HB(1-2)) the DataReader will notice that the data associated with sequence number 1 was never received. It realizes this because the heartbeat HB(1-2) tells the DataReader that it should have received all messages up to and including the one with sequence number 2. This realization has two consequences:

1.The data associated with sequence number 2 (B) is tagged with ‘X’ to indicate that it is not deliverable to the application (that is, it should not be made available to the application, because the application needs to receive the data associated with sample 1

(A) first).

2.An ACKNACK(1) is sent to the DataWriter to request that the data tagged with sequence number 1 be resent.

Reception of the ACKNACK(1) causes the DataWriter to resend DATA(A,1). Once the DataReader receives it, it can ‘commit’ both A and B such that the application can now access both (indicated

10-5

by the “✔”) and call the DataReaderListener. From there on, the protocol proceeds as before for the next data message (C) and so forth.

A subtle but important feature of the RTPS protocol is that ACKNACK messages are only sent as a direct response to HB messages. This allows the DataWriter to better control the overhead of these ‘administrative’ messages. For example, if the DataWriter knows that it is about to send a chain of DATA messages, it can bundle them all and include a single HB at the end, which minimizes ACKNACK traffic.

10.3Using QosPolicies to Tune the Reliable Protocol

Reliability is controlled by the QosPolicies in Table 10.1. To enable reliable delivery, read the following sections to learn how to change the QoS for the DataWriter and DataReader:

❏Enabling Reliability (Section 10.3.1)

❏Tuning Queue Sizes and Other Resource Limits (Section 10.3.2)

❏Controlling Heartbeats and Retries with DataWriterProtocol QosPolicy (Section 10.3.4)

❏Avoiding Message Storms with DataReaderProtocol QosPolicy (Section 10.3.5)

❏Resending Samples to Late-Joiners with the Durability QosPolicy (Section 10.3.6)

Then see this section to explore example use cases:

❏ Use Cases (Section 10.3.7)

Table 10.1 QosPolicies for Reliable Communications

10-6

Table 10.1 QosPolicies for Reliable Communications

a.DW = DataWriter, DR = DataReader

10.3.1Enabling Reliability

You must modify the RELIABILITY QosPolicy (Section 6.5.19) of the DataWriter and each of its reliable DataReaders. Set the kind field to DDS_RELIABLE_RELIABILITY_QOS:

❏ DataWriter

writer_qos.reliability.kind = DDS_RELIABLE_RELIABILITY_QOS;

❏ DataReader

reader_qos.reliability.kind = DDS_RELIABLE_RELIABILITY_QOS;

10.3.1.1Blocking until the Send Queue Has Space Available

The max_blocking_time property in the RELIABILITY QosPolicy (Section 6.5.19) indicates how long a DataWriter can be blocked during a write().

If max_blocking_time is non-zero and the reliability send queue is full, the write is blocked (the sample is not sent). If max_blocking_time has passed and the sample is still not sent, write() returns DDS_RETCODE_TIMEOUT and the sample is not sent.

If the number of unacknowledged samples in the reliability send queue drops below max_samples (set in the RESOURCE_LIMITS QosPolicy (Section 6.5.20)) before max_blocking_time, the sample is sent and write() returns DDS_RETCODE_OK.

If max_blocking_time is zero and the reliability send queue is full, write() returns DDS_RETCODE_TIMEOUT and the sample is not sent.

10.3.2Tuning Queue Sizes and Other Resource Limits

Set the HISTORY QosPolicy (Section 6.5.10) appropriately to accommodate however many samples should be saved in the DataWriter’s send queue or the DataReader’s receive queue. The defaults may suit your needs; if so, you do not have to modify this QosPolicy.

Set the DDS_RtpsReliableWriterProtocol_t in the DATA_WRITER_PROTOCOL QosPolicy (DDS Extension) (Section 6.5.3) appropriately to accommodate the number of unacknowledged samples that can be in-flight at a time from a DataWriter.

For more information, see the following sections:

❏Understanding the Send Queue and Setting its Size (Section 10.3.2.1)

❏Understanding the Receive Queue and Setting Its Size (Section 10.3.2.2)

Note: The HistoryQosPolicy’s depth must be less than or equal to the ResourceLimitsQosPolicy’s max_samples_per_instance; max_samples_per_instance must be less than or equal to the ResourceLimitsQosPolicy’s max_samples (see RESOURCE_LIMITS QosPolicy (Section 6.5.20)), and max_samples_per_remote_writer (see

10-7

DATA_READER_RESOURCE_LIMITS QosPolicy (DDS Extension) (Section 7.6.2)) must be less than or equal to max_samples.

❏depth <= max_samples_per_instance <= max_samples

❏max_samples_per_remote_writer <= max_samples

Examples:

❏ DataWriter

writer_qos.resource_limits.initial_instances = 10; writer_qos.resource_limits.initial_samples = 200; writer_qos.resource_limits.max_instances = 100; writer_qos.resource_limits.max_samples = 2000; writer_qos.resource_limits.max_samples_per_instance = 20; writer_qos.history.depth = 20;

❏ DataReader

reader_qos.resource_limits.initial_instances = 10; reader_qos.resource_limits.initial_samples = 200; reader_qos.resource_limits.max_instances = 100; reader_qos.resource_limits.max_samples = 2000; reader_qos.resource_limits.max_samples_per_instance = 20; reader_qos.history.depth = 20; reader_qos.reader_resource_limits.max_samples_per_remote_writer = 20;

10.3.2.1Understanding the Send Queue and Setting its Size

A DataWriter’s send queue is used to store each sample it writes. A sample will be removed from the send queue after it has been acknowledged (through an ACKNACK) by all the reliable DataReaders. A DataReader can request that the DataWriter resend a missing sample (through an ACKNACK). If that sample is still available in the send queue, it will be resent. To elicit timely ACKNACKs, the DataWriter will regularly send heartbeats to its reliable DataReaders.

A DataWriter’s send queue size is determined by its RESOURCE_LIMITS QosPolicy (Section 6.5.20), specifically the max_samples field. The appropriate value depends on application parameters such as how fast the publication calls write().

A DataWriter has a "send window" that is the maximum number of unacknowledged samples allowed in the send queue at a time. The send window enables configuration of the number of samples queued for reliability to be done independently from the number of samples queued for history. This is of great benefit when the size of the history queue is much different than the size of the reliability queue. For example, you may want to resend a large history to late-joining DataReaders, so the send queue size is large. However, you do not want performance to suffer due to a large send queue; this can happen when the send rate is greater than the read rate, and the DataWriter has to resend many samples from its large historical send queue. If the send queue size was both the historical and reliability queue size, then both these goals could not be met. Now, with the send window, having a large history with good live reliability performance is possible.

The send window is determined by the DataWriterProtocolQosPolicy, specifically the fields min_send_window_size and max_send_window_size within the rtps_reliable_writer field of type DDS_RtpsReliableWriterProtocol_t. Other fields control a dynamic send window, where the send window size changes in response to network congestion to maximize the effective send rate. Like for max_samples, the appropriate values depend on application parameters.

Strict reliability: If a DataWriter does not receive ACKNACKs from one or more reliable DataReaders, it is possible for the reliability send queue—either its finite send window, or max_samples if its send window is infinite—to fill up. If you want to achieve strict reliability, the kind field in the HISTORY QosPolicy (Section 6.5.10) for both the DataReader and DataWriter must be set to KEEP_ALL, positive acknowledgments must be enabled for both the DataReader and DataWriter, and your publishing application should wait until space is available in the

10-8

reliability queue before writing any more samples. Connext provides two mechanisms to do this:

❏Allow the write() operation to block until there is space in the reliability queue again to store the sample. The maximum time this call blocks is determined by the max_blocking_time field in the RELIABILITY QosPolicy (Section 6.5.19) (also discussed in Section 10.3.1.1).

❏Use the DataWriter’s Listener to be notified when the reliability queue fills up or empties again.

When the HISTORY QosPolicy (Section 6.5.10) on the DataWriter is set to KEEP_LAST, strict reliability is not guaranteed. When there are depth number of samples in the queue (set in the HISTORY QosPolicy (Section 6.5.10), see Section 10.3.3) the oldest sample will be dropped from the queue when a new sample is written. Note that in such a reliable mode, when the send window is larger than max_samples, the DataWriter will never block, but strict reliability is no longer guaranteed.

If there is a request for the purged sample from any DataReaders, the DataWriter will send a heartbeat that no longer contains the sequence number of the dropped sample (it will not be able to send the sample).

Alternatively, a DataWriter with KEEP_LAST may block on write() when its send window is smaller than its send queue. The DataWriter will block when its send window is full. Only after the blocking time has elapsed, the DataWriter will purge a sample, and then strict reliability is no longer guaranteed.

The send queue size is set in the max_samples field of the RESOURCE_LIMITS QosPolicy (Section 6.5.20). The appropriate size for the send queue depends on application parameters (such as the send rate), channel parameters (such as end-to-end delay and probability of packet loss), and quality of service requirements (such as maximum acceptable probability of sample loss).

The DataReader’s receive queue size should generally be larger than the DataWriter’s send queue size. Receive queue size is discussed in Section 10.3.2.2.

A good rule of thumb, based on a simple model that assumes individual packet drops are not correlated and time-independent, is that the size of the reliability send queue, N, is as shown in Figure 10.3.

Figure 10.3 Calculating Minimum Send Queue Size for a Desired Level of Reliability

NRTlog ( 1 – Q)

=2 -------------------------

log ( p)

Simple formula for determining the minimum size of the send queue required for strict reliability.

In the above equation, R is the rate of sending samples, T is the round-trip transmission time, p is the probability of a packet loss in a round trip, and Q is the required probability that a sample is eventually successfully delivered. Of course, network-transport dropouts must also be taken into account and may influence or dominate this calculation.

Table 10.2 gives the required size of the send queue for several common scenarios.

Table 10.2 Required Size of the Send Queue for Different Network Parameters

10-9

Table 10.2 Required Size of the Send Queue for Different Network Parameters

a."Q" is the desired level of reliability measured as the probability that any data update will eventually be delivered successfully. In other words, percentage of samples that will be successfully delivered.

b."p" is the probability that any single packet gets lost in the network.

c."T" is the round-trip transport delay in the network

d."R" is the rate at which the publisher is sending updates.

e."N" is the minimum required size of the send queue to accomplish the desired level of reliability "Q".

f.The typical round-trip delay for a dedicated 100 Mbit/second ethernet is about 0.001 seconds.

Note: Packet loss on a network frequently happens in bursts, and the packet loss events are correlated. This means that the probability of a packet being lost is much higher if the previous packet was lost because it indicates a congested network or busy receiver. For this situation, it may be better to use a queue size that can accommodate the longest period of network congestion, as illustrated in Figure 10.4.

Figure 10.4 Calculating Minimum Send Queue Size for Networks with Dropouts

N = RD( Q)

Send queue size as a function of send rate "R" and maximum dropout time D.

In the above equation R is the rate of sending samples, D(Q) is a time such that Q percent of the dropouts are of equal or lesser length, and Q is the required probability that a sample is eventually successfully delivered. The problem with the above formula is that it is hard to determine the value of D(Q) for different values of Q.

For example, if we want to ensure that 99.9% of the samples are eventually delivered successfully, and we know that the 99.9% of the network dropouts are shorter than 0.1 seconds, then we would use N = 0.1*R. So for a rate of 100Hz, we would use a send queue of N = 10; for a rate of 2000Hz, we would use N = 200.

10.3.2.2Understanding the Receive Queue and Setting Its Size

Samples are stored in the DataReader’s receive queue, which is accessible to the user’s application.

A sample is removed from the receive queue after it has been accessed by take(), as described in Accessing Data Samples with Read or Take (Section 7.4.3). Note that read() does not remove samples from the queue.

A DataReader's receive queue size is limited by its RESOURCE_LIMITS QosPolicy (Section 6.5.20), specifically the max_samples field. The storage of out-of-order samples for each DataWriter is also allocated from the DataReader’s receive queue; this sample resource is shared among all reliable DataWriters. That is, max_samples includes both ordered and out-of-order samples.

A DataReader can maintain reliable communications with multiple DataWriters (e.g., in the case of the OWNERSHIP_STRENGTH QosPolicy (Section 6.5.16) setting of SHARED). The maximum number of out-of-order samples from any one DataWriter that can occupy in the receive queue is set in the max_samples_per_remote_writer field of the DATA_READER_RESOURCE_LIMITS QosPolicy (DDS Extension) (Section 7.6.2); this value can be used to prevent a single DataWriter from using all the space in the receive queue. max_samples_per_remote_writer must be set to be <= max_samples.

10-10

The DataReader will cache samples that arrive out of order while waiting for missing samples to be resent. (Up to 256 samples can be resent; this limitation is imposed by the wire protocol.) If there is no room, the DataReader has to reject out-of-order samples and request them again later after the missing samples have arrived.

The appropriate size of the receive queue depends on application parameters, such as the DataWriter’s sending rate and the probability of a dropped sample. However, the receive queue size should generally be larger than the send queue size. Send queue size is discussed in Section 10.3.2.1.

Figure 10.5 and Figure 10.6 compare two hypothetical DataReaders, both interacting with the same DataWriter. The queue on the left represents an ordering cache, allocated from receive queue—samples are held here if they arrive out of order. The DataReader in Figure 10.5 on page 10-11 has a sufficiently large receive queue (max_samples) for the given send rate of the DataWriter and other operational parameters. In both cases, we assume that all samples are taken from the DataReader in the Listener callback. (See Accessing Data Samples with Read or Take (Section 7.4.3) for information on take() and related operations.)

In Figure 10.6 on page 10-12, max_samples is too small to cache out-of-order samples for the same operational parameters. In both cases, the DataReaders eventually receive all the samples in order. However, the DataReader with the larger max_samples will get the samples earlier and with fewer transactions. In particular, sample “4” is never resent for the DataReader with the larger queue size.

Figure 10.5 Effect of Receive-Queue Size on Performance: Large Queue Size

10-11

Figure 10.6 Effect of Receive Queue Size on Performance: Small Queue Size

DataWriter DataReader

10-12

10.3.3Controlling Queue Depth with the History QosPolicy

If you want to achieve strict reliability, set the kind field in the HISTORY QosPolicy (Section 6.5.10) for both the DataReader and DataWriter to KEEP_ALL; in this case, the depth does not matter.

Or, for non-strict reliability, you can leave the kind set to KEEP_LAST (the default). This will provide non-strict reliability; some samples may not be delivered if the resource limit is reached.

The depth field in the HISTORY QosPolicy (Section 6.5.10) controls how many samples Connext will attempt to keep on the DataWriter’s send queue or the DataReader’s receive queue. For reliable communications, depth should be >= 1. The depth can be set to 1, but cannot be more than the max_samples_per_instance in RESOURCE_LIMITS QosPolicy (Section 6.5.20).

Example:

❏ DataWriter

writer_qos.history.depth = <number of samples to keep in send queue>;

❏ DataReader

reader_qos.history.depth = <number of samples to keep in receive queue>;

10.3.4Controlling Heartbeats and Retries with DataWriterProtocol QosPolicy

In the Connext reliability model, the DataWriter sends data samples and heartbeats to reliable DataReaders. A DataReader responds to a heartbeat by sending an ACKNACK, which tells the DataWriter what the DataReader has received so far.

In addition, the DataReader can request missing samples (by sending an ACKNACK) and the DataWriter will respond by resending the missing samples. This section describes some advanced timing parameters that control the behavior of this mechanism. Many applications do not need to change these settings. These parameters are contained in the

DATA_WRITER_PROTOCOL QosPolicy (DDS Extension) (Section 6.5.3).

The protocol described in Overview of the Reliable Protocol (Section 10.2) uses very simple rules such as piggybacking HB messages to each DATA message and responding immediately to ACKNACKs with the requested repair messages. While correct, this protocol would not be capable of accommodating optimum performance in more advanced use cases.

This section describes some of the parameters configurable by means of the rtps_reliable_writer structure in the DATA_WRITER_PROTOCOL QosPolicy (DDS Extension) (Section 6.5.3) and how they affect the behavior of the RTPS protocol.

10.3.4.1How Often Heartbeats are Resent (heartbeat_period)

If a DataReader does not acknowledge a sample that has been sent, the DataWriter resends the heartbeat. These heartbeats are resent at the rate set in the DATA_WRITER_PROTOCOL QosPolicy (DDS Extension) (Section 6.5.3), specifically its heartbeat_period field.

For example, a heartbeat_period of 3 seconds means that if a DataReader does not receive the latest sample (for example, it gets dropped by the network), it might take up to 3 seconds before the DataReader realizes it is missing data. The application can lower this value when it is important that recovery from packet loss is very fast.

The basic approach of sending HB messages as a piggyback to DATA messages has the advantage of minimizing network traffic. However, there is a situation where this approach, by itself, may result in large latencies. Suppose there is a DataWriter that writes bursts of data, separated by relatively long periods of silence. Furthermore assume that the last message in one of the bursts is lost by the network. This is the case shown for message DATA(B, 2) in Figure 10.7. If HBs were only sent piggybacked to DATA messages, the DataReader would not realize it missed the ‘B’ DATA message with sequence number ‘2’ until the DataWriter wrote the next message. This may be a long time if data is written sporadically. To avoid this situation,

10-13

Connext can be configured so that HBs are sent periodically as long as there are samples that have not been acknowledged even if no data is being sent. The period at which these HBs are sent is configurable by setting the rtps_reliable_writer.heartbeat_period field in the DATA_WRITER_PROTOCOL QosPolicy (DDS Extension) (Section 6.5.3).

Note that a small value for the heartbeat_period will result in a small worst-case latency if the last message in a burst is lost. This comes at the expense of the higher overhead introduced by more frequent HB messages.

Also note that the heartbeat_period should not be less than the rtps_reliable_reader.heartbeat_suppression_duration in the DATA_READER_PROTOCOL QosPolicy (DDS Extension) (Section 7.6.1); otherwise those HBs will be lost.

Figure 10.7 Use of heartbeat_period

write(A)

1 A X

cache (A, 1)

DATA (A,1)

write(B)