Threading model when using Asynchronous Publishing

Using Connext DDS Professional, it is possible to configure your application such that the sending of a sample in calls like FooDataWriter_write is done by a separate thread (by default, the sending occurs in the thread calling to FooDataWriter_write). This is accomplished using the Asynchronous Publisher Qos and Publish Mode QoS. Asynchronous Publishing requires you to select a Flow Controller that will be used to write the data. You can either select one of the predefined FlowControllers which are provided by Connext DDS Professional, or create your own. This article explains the threading model of the asynchronous publishing feature.

FlowControllers provide a mechanism to throttle the amount of data being sent by an application. A FlowController is associated with a DomainParticipant. DataWriters can share FlowControllers (resulting in data throttling across all of the DataWriters sharing the same FlowController). A DomainParticipant can create multiple FlowControllers.

The number of threads in a Connext DDS Professional application can vary greatly depending on its configuration (this Knowledge Base article describes how to identify the threads in your application). A common configuration is to have the following 9 threads:

• 5 Receive threads (more information here)

• 1 Database thread

• 1 Event thread

• 1 Interface tracking thread

• 1 User thread

When asynchronous publishing is enabled, a new thread is created -- the asynchronous write thread. By default (i.e., when asynchronous publishing is not used) a sample is sent by the user thread calling write in a DataWriter. When a FlowController is used, the write call adds the sample to the DataWriter queue, but the sending on the network is performed by the asynchronous write thread. It is important to note that the asynchronous write thread is per Publisher. This means that if you have multiple DataWriters using asynchronous publishing, but all are created by the same Publisher, there will only be a single asynchronous write thread. Similarly, if you have multiple DataWriters, each belonging to a separate Publisher, they will each have their own asynchronous write thread.

Given all of the above, there are 3 common configurations of asynchronous publication, each providing a different level of concurrency and fulfilling a different use case:

1) Multiple DataWriters, all created by the same Publisher and using the same FlowController

• This is the simplest of the three configurations. It requires fewer system resources than the other two options. It also has the most sources of contention, since all of the DataWriters share the same asynchronous write thread.

2) Multiple DataWriters, each created by a separate Publisher but all using the same FlowController

•  This configuration is clearly more concurrent than having a single Publisher (and therefore a single asynchronous write thread). It is important to note that the number of cores available on your machine should be taken into account when using this approach. If there are many Publishers using asynchronous publishing, each one will create its own asynchronous write thread and more threads mean more context switches.

3) Multiple DataWriters, each created by a separate Publisher and each using its own FlowController

• This configuration is the most concurrent because there is no contention accessing the FlowController. It is important to note that given we now have N FlowControllers, the properties of those FlowControllers will need to be modified accordingly in order to maintain the same data throttling. For example, suppose that we want to limit the amount of data written to be 1MiB/second. A configuration with only a single FlowController might be to set max_tokens to 1024, bytes_per_token to 1024 and period to 1 second. If N was 2 (i.e., there were 2 FlowControllers), we would need to halve one of these parameters to maintain the same level of data throttling.