47.2 BATCH QosPolicy (DDS Extension)

This QosPolicy can be used to decrease the amount of communication overhead associated with the transmission and (in the case of reliable communication) acknowledgment of small DDS samples, in order to increase throughput.

It specifies and configures the mechanism that allows Connext to collect multiple user data DDS samples to be sent in a single network packet, to take advantage of the efficiency of sending larger packets and thus increase effective throughput.

This QosPolicy can be used to increase effective throughput dramatically for small data DDS samples. Throughput for small DDS samples (size < 2048 bytes) is typically limited by CPU capacity and not by network bandwidth. Batching many smaller DDS samples to be sent in a single large packet will increase network utilization and thus throughput in terms of DDS samples per second.

It contains the members listed in Table 47.5 DDS_BatchQosPolicy.

Table 47.5 DDS_BatchQosPolicy

Type	Field Name	Description
DDS_Boolean	enable	Enables/disables batching.
DDS_Long	max_data_bytes	Sets the maximum cumulative length of all serialized DDS samples in a batch. Before or when this limit is reached, the batch is automatically flushed. The size does not include the meta-data associated with the batch DDS samples.
DDS_Long	max_samples	Sets the maximum number of DDS samples in a batch. When this limit is reached, the batch is automatically flushed.
struct DDS_Duration_t	max_flush_delay	Sets the maximum flush delay. When this duration is reached, the batch is automatically flushed. The delay is measured from the time the first DDS sample in the batch is written by the application.
struct DDS_Duration_t	source_timestamp_ resolution	Sets the batch source timestamp resolution. The value of this field determines how the source timestamp is associated with the DDS samples in a batch. A DDS sample written with timestamp 't' inherits the source timestamp 't2' associated with the previous DDS sample, unless ('t' - 't2') is greater than source_timestamp_resolution. If source_timestamp_resolution is DURATION_INFINITE, every DDS sample in the batch will share the source timestamp associated with the first DDS sample. If source_timestamp_resolution is zero, every DDS sample in the batch will contain its own source timestamp corresponding to the moment when the DDS sample was written. The performance of the batching process is better when source_timestamp_resolution is set to DURATION_INFINITE.
DDS_Boolean	thread_safe_write	Determines whether or not the write operation is thread-safe. If TRUE, multiple threads can call write on the DataWriter concurrently. A setting of FALSE can be used to increase batching throughput for batches with many small DDS samples.

If batching is enabled (not the default), DDS samples are not immediately sent when they are written. Instead, they get collected into a "batch." A batch always contains whole number of DDS samples—a DDS sample will never be fragmented into multiple batches.

A batch is sent on the network ("flushed") when one of the following things happens:

• User-configurable flushing conditions
  • A batch size limit (max_data_bytes) is reached.
  • A number of DDS samples are in the batch (max_samples).
  • A time-limit (max_flush_delay) is reached, as measured from the time the first DDS sample in the batch is written by the application.
  • The application explicitly calls a DataWriter's flush() operation.
• Non-user configurable flushing conditions:
  • A coherent set starts or ends.
  • The number of DDS samples in the batch is equal to max_samples in RESOURCE_LIMITS for unkeyed topics or max_samples_per_instance in RESOURCE_LIMITS for keyed topics.

Additional batching configuration takes place in the Publisher’s 46.1 ASYNCHRONOUS_PUBLISHER QosPolicy (DDS Extension).

The flush() operation is described in 31.9 Flushing Batches of DDS Data Samples.

47.2.1 Synchronous and Asynchronous Flushing

Usually, a batch is flushed synchronously:

• When a batch reaches its application-defined size limit (max_data_bytes or max_samples) because the application called write(), the batch is flushed immediately in the context of the writing thread.
• When an application manually flushes a batch, the batch is flushed immediately in the context of the calling thread.
• When the first DDS sample in a coherent set is written, the batch in progress (without including the DDS sample in the coherent set) is immediately flushed in the context of the writing thread.
• When a coherent set ends, the batch in progress is immediately flushed in the context of the calling thread.
• When the number of DDS samples in a batch is equal to max_samples in RESOURCE_LIMITS for unkeyed topics or max_samples_per_instance in RESOURCE_LIMITS for keyed topics, the batch is flushed immediately in the context of the writing thread.

However, some behavior is asynchronous:

• To flush batches based on a time limit (max_flush_delay), enable asynchronous batch flushing in the 46.1 ASYNCHRONOUS_PUBLISHER QosPolicy (DDS Extension) of the DataWriter's Publisher. This will cause the Publisher to create an additional thread that will be used to flush batches of that Publisher's DataWriters. This behavior is analogous to the way asynchronous publishing works.
• You may also use batching alongside asynchronous publication with 34.4 FlowControllers (DDS Extension). These features are independent of one another. Flushing a batch on an asynchronous DataWriter makes it available for sending to the DataWriter's FlowController. From the point of view of the FlowController, a batch is treated like one large DDS sample.

47.2.2 Batching vs. Coalescing

Even when batching is disabled, Connext will sometimes coalesce multiple DDS samples into a single network datagram. For example, DDS samples buffered by a FlowController or sent in response to a negative acknowledgement (NACK) may be coalesced. This behavior is distinct from DDS sample batching.

DDS samples that are sent individually (not part of a batch) are always treated as separate DDS samples by Connext. Each DDS sample is accompanied by a complete RTPS header on the network (although DDS samples may share UDP and IP headers) and (in the case of reliable communication) a unique physical sequence number that must be positively or negatively acknowledged.

In contrast, batched DDS samples share an RTPS header and an entire batch is acknowledged —positively or negatively—as a unit, potentially reducing the amount of meta-traffic on the network and the amount of processing per individual DDS sample.

Batching can also improve latency relative to simply coalescing. Consider two use cases:

1. A DataWriter is configured to write asynchronously with a FlowController. Even if the FlowController's rules would allow it to publish a new DDS sample immediately, the send will always happen in the context of the asynchronous publishing thread. This context switch can add latency to the send path.
2. A DataWriter is configured to write synchronously but with batching turned on. When the batch is full, it will be sent on the wire immediately, eliminating a thread context switch from the send path.

47.2.3 Batching and ContentFilteredTopics

When batching is enabled, content filtering is always done on the reader side.

47.2.4 Turbo Mode: Automatically Adjusting the Number of Bytes in a Batch—Experimental Feature

Turbo Mode is an experimental feature that uses an intelligent algorithm that automatically adjusts the number of bytes in a batch at run time according to current system conditions, such as write speed (or write frequency) and DDS sample size. This intelligence is what gives it the ability to increase throughput at high message rates and avoid negatively impacting message latency at low message rates.

To enable Turbo mode, set the DataWriter's property dds.data_writer.enable_turbo_mode to true. Turbo mode is not enabled by default.

Note: If you explicitly enable batching by setting enable to TRUE in BatchQosPolicy, the value of the turbo mode property is ignored and turbo mode is not used.

47.2.5 Performance Considerations

The purpose of batching is to increase throughput when writing small DDS samples at a high rate. In such cases, throughput can be increased several-fold, approaching much more closely the physical limitations of the underlying network transport.

However, collecting DDS samples into a batch implies that they are not sent on the network immediately when the application writes them; this can potentially increase latency. However, if the application sends data faster than the network can support, an increased proportion of the network's available bandwidth will be spent on acknowledgements and DDS sample resends. In this case, reducing that overhead by turning on batching could decrease latency while increasing throughput.

As a general rule, to improve batching throughput:

• Set thread_safe_write to FALSE when the batch contains a big number of small DDS samples. If you do not use a thread-safe write configuration, asynchronous batch flushing must be disabled.
• Set source_timestamp_resolution to DURATION_INFINITE. Note that you set this value, every DDS sample in the batch will share the same source timestamp.

Batching affects how often piggyback heartbeats are sent; see heartbeats_per_max_samples in Table 47.14 DDS_RtpsReliableWriterProtocol_t.

47.2.6 Maximum Transport Datagram Size

Batches cannot be fragmented. As a result, the maximum batch size (max_data_bytes) must be set no larger than the maximum transport datagram size. For example, a UDP datagram is limited to 64 KB, so any batches sent over UDP must be less than or equal to that size.

47.2.7 Bandwidth Considerations

A minimum overhead of 8-bytes is added to each sample in a batch; however, the overhead may be bigger in some cases. For example:

• When you add a source timestamp per sample instead of per batch, there will be 8 more bytes for the source timestamp. You can control this behavior with writer_qos.batch.source_timestamp_resolution.
• By default, for keyed topics, Connext adds the key hash for the instance, adding an extra overhead of 20 bytes. If you don’t want to add the key hash and instead get it from the serialized data on the DataReader side, set writer_qos.protocol.disable_inline_keyhash to true.
• Disposed/unregistered samples also need an additional 8-byte overhead to mark the status as disposed or unregistered.
• There are other scenarios in which overhead may increase—for example, when using collaborative DataWriters or group order access.

To summarize:

• For a data sample for a keyed topic, by default, the overhead will be 32-bytes (8 (minimum) + 20 (for the key hash) + 4 (for the sentinel)). You can reduce this to 8 bytes by not sending the key hash (in which case, the sentinel goes away, too).
• For disposed/unregistered samples for a keyed topic, by default, the overhead will be 40-bytes (8 (minimum) + 20 (for the key hash) + 8 (for the status information) + 4 (for the sentinel)). You can reduce this to 20 bytes by not sending the key hash (the sentinel remains for the status information).
• For an unkeyed topic, the overhead is typically 8 bytes.

47.2.8 Properties

This QosPolicy cannot be modified after the DataWriter is enabled.

Since it is only for DataWriters, there are no compatibility restrictions for how it is set on the publishing and subscribing sides.

All batching configuration occurs on the publishing side. A subscribing application does not configure anything specific to receive batched DDS samples, and in many cases, it will be oblivious to whether the DDS samples it processes were received individually or as part of a batch.

Consistency rules:

• max_samples must be consistent with max_data_bytes: they cannot both be set to LENGTH_UNLIMITED.
• If max_flush_delay is not DURATION_INFINITE, disable_asynchronous_batch in the 46.1 ASYNCHRONOUS_PUBLISHER QosPolicy (DDS Extension) must be FALSE.
• If thread_safe_write is FALSE, source_timestamp_resolution must be DURATION_INFINITE.

47.2.9 Related QosPolicies

To flush batches based on a time limit, enable batching in the 46.1 ASYNCHRONOUS_PUBLISHER QosPolicy (DDS Extension) of the DataWriter's Publisher.

Be careful when configuring a DataWriter's 47.14 LIFESPAN QoS Policy with a duration shorter than the batch flush period (max_flush_delay). If the batch does not fill up before the flush period elapses, by default the short duration will cause the DDS samples to be dropped without being sent. (You can, however, change this default behavior. See the last paragraph in this section.)

Do not configure the DataReader’s or DataWriter’s 47.12 HISTORY QosPolicy to be shallower than the DataWriter's maximum batch size (max_samples). When the HISTORY QosPolicy is shallower on the DataWriter, by default some DDS samples may not be sent. (You can, however, change this default behavior. See the last paragraph in this section.) When the HISTORY QosPolicy is shallower on the DataReader, DDS samples may be lost before being provided to the application.

The initial and maximum numbers of batches that a DataWriter will manage is set in the 47.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension).

The maximum number of DDS samples that a DataWriter can store is determined by the value max_samples in the 47.22 RESOURCE_LIMITS QosPolicy and max_batches in the 47.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension). The limit that is reached first is applied.

The amount of resources required for batching depends on the configuration of the 47.22 RESOURCE_LIMITS QosPolicy and the 47.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension). See 47.2.11 System Resource Considerations.

By default, samples marked as removed in a batch are dropped. Examples of removed samples in a batch are samples that were replaced due to KEEP_LAST_HISTORY_QOS on the DataWriter (see 47.12 HISTORY QosPolicy) or samples that outlived the DataWriter's 47.14 LIFESPAN QoS Policy duration. You can keep track of the number of these dropped samples via writer_removed_batch_sample_dropped_sample_count in the 40.7.2 DATA_READER_CACHE_STATUS. You can also choose not to drop these samples at all by setting the property dds.data_reader.accept_writer_removed_batch_samples to TRUE (by default it is set to FALSE); you can set this property via the 47.19 PROPERTY QosPolicy (DDS Extension) .

47.2.10 Applicable DDS Entities

• 31. DataWriters

47.2.11 System Resource Considerations

• Batching requires additional resources to store the meta-data associated with the DDS samples in the batch.
  • For unkeyed topics, the meta-data will be at least 8 bytes, with a maximum of 20 bytes.
  • For keyed topics, the meta-data will be at least 8 bytes, with a maximum of 52 bytes.
• Other resource considerations are described in 47.2.9 Related QosPolicies .