Scenario: Slow Consumer

Unless special measures are taken, the presence of slow consumers can impact the overall behavior of the system. If a DataReader is not keeping up with the DDS samples being sent by the DataWriter, it will apply back-pressure to the DataWriter to slow the rate at which the DataWriter can write DDS samples. With delegated reliability (see Scenario: Delegated Reliability ), the original DataWriter can offload the processing of the ACK/NACK messages generated by the DataReaders to a PRSTDataWriter. However, the original DataWriter still has a reliable channel with the PRSTDataReader that can slow it down.

By default, Persistence Service uses the Connext DDS receive thread to read DDS samples from the PRStDataReaders, write the DDS samples to the PRSTDataWriters history, and send ACKs to the original DataWriter. With this configuration, a PRSTDataReader does not ACK DDS samples to the original DataWriter until they are written into the corresponding PRSTDataWriter’s history. Since multiple DataReaders may be accessing the PRSTDataWriter history at the same time that the persistence service is trying to write new DDS samples, the PRSTDataWriter history becomes a contention point that can indirectly slow down the original DataWriter (see Slow-Consumer Scenario with Delegated Reliability).

Figure 4 Slow-Consumer Scenario with Delegated Reliability

To remove this contention point and decouple the slow consumer from the original DataWriter, Persistence Service supports a mode where DDS samples can be buffered prior to being added to the PRSTDataWriter’s queue (see Slow Consumer Scenario with Delegated Reliability and DDS Sample Log).

Figure 5 Slow Consumer Scenario with Delegated Reliability and DDS Sample Log

If the PRSTDataWriter slows down due to the presence of slow consumers, the buffer will hold DDS samples such that the original DataWriter and the rest of the system are not impacted. This buffer is called the Persistence Service sample log. The persistence service creates a separate DDS sample log per PRSTDataWriter in the group. In addition to the DDS sample log, the persistence service creates a thread (write thread) whose main function is to read DDS samples from the log and write them to the associated PRSTDataWriter. There is one thread per PRSTDataWriter.

Persistence Service currently does not allow multiple DDS sample logs to share the same write thread.

Persistence Service can be configured to enable DDS sample logging per persistence group using the <sample_logging> XML tag to specify the log’s configuration parameters—see Table 1 ..

Table 1 Sample Logging Tags

Tags within <sample_logging>	Description	Number of Tags Allowed
<enable>	A DDS_Boolean (see Table 1 ) that indicates whether or not DDS sample logging is enabled in the container persistence group. Default: 0	0 or 1
<log_file_ size>	Specifies the maximum size of a DDS sample log file in Mbytes. When a log file becomes full, Persistence Service creates a new log file. Default: 60 MB	0 or 1
<log_flush_ period>	The period (in milliseconds) at which Persistence Service removes DDS sample log files whose full content have been written into the PRSTDataWriter by the DDS sample log write thread. Default: 10000 milliseconds	0 or 1
<log_read_ batch>	Determines how many DDS samples should be read and processed at once by the DDS sample log write thread. Default: 100 DDS samples	0 or 1
<log_bookmark_ period>	DDS samples in the DDS sample log are identified by two attributes: The file ID The row ID (position within the file) The read bookmark indicates the most recently processed DDS sample. This tag indicates how often (in milliseconds) the read bookmark is persisted into disk. Default: 1000 milliseconds	0 or 1

Enabling DDS sample logging in a persistence group is expensive. For every PRSTDataWriter, Persistence Service will create a write thread and an event thread that will be in charge of flushing the log files and storing the read bookmark. Therefore, DDS sample logging should be enabled only for the persistence groups where it is needed based on the potential presence of slow consumers and/or the expected data rate in the persistence group. Small data rates will likely not require a DDS sample log.

© 2015 RTI