Chapter 61 Topic Queries

TopicQueries allow a DataReader to query the sample cache of its matching DataWriters. You can create a TopicQuery with the DataReader's create_topic_query() API. When a DataReader creates a TopicQuery, DDS will propagate TopicQueries to other DomainParticipants and their DataWriters. When a DataWriter matching with the DataReader that created the TopicQuery receives it, it will send the cached samples that pass the TopicQuery's filter.

Only samples that fall within the writer_depth (in the 47.9 DURABILITY QosPolicy) for an instance are evaluated against the TopicQuery filter. While the DataWriter is waiting for acknowledgements from one or more DataReaders, there may temporarily be more than writer_depth samples per instance in the DataWriter's queue if the 47.12 HISTORY QosPolicy depth is set to a higher value than writer_depth. Those additional samples past writer_depth are not eligible to be sent in response to the TopicQuery.

To enable and configure TopicQueries, use the following QoS policies:

The delivery of TopicQuery samples occurs in a separate RTPS channel to allow DataReaders to receive TopicQuery samples and live samples in parallel. This is a key difference with respect to the 47.9 DURABILITY QosPolicy, with which new non-volatile DataReaders first receive all historical data before they start receiving live data.

Late-joining DataWriters will also discover existing TopicQueries. To delete a TopicQuery you must use the DataReader's delete_topic_query().

After deleting a TopicQuery, new DataWriters will not discover it and existing DataWriters currently publishing cached samples may stop before delivering all of them.

By default, a TopicQuery queries the samples that were in the DataWriter's queue at the time the DataWriter received the TopicQuery. These TopicQueries are snapshot TopicQueries. However, a TopicQuery can be created in “continuous” mode; in this case, a DataWriter will continue delivering samples that pass a continuous TopicQuery filter until the DataReader application explicitly deletes it. These TopicQueries are continuous TopicQueries.

The samples received in response to a TopicQuery are stored in the associated DataReader's cache. Any of the read/take operations can retrieve TopicQuery samples. The field DDS_SampleInfo::topic_query_guid associates each sample with its TopicQuery. If the read sample is not in response to a TopicQuery, this field will be DDS_GUID_UNKNOWN.

You can choose to read or take only TopicQuery samples, only live samples, or both. To support this, ReadConditions and QueryConditions provide the DataReader's create_querycondition_w_params() and create_readcondition_w_params() APIs.

Each TopicQuery is identified by a GUID that can be accessed using the TopicQuery's get_guid() method.

61.1 Snapshot TopicQueries

A snapshot TopicQuery queries the samples that are in a DataWriter's queue when the DataWriter receives the TopicQuery.

When a DataWriter receives a snapshot TopicQuery, it registers the last sequence number available in the DataWriter queue (known as the cut-off SN). Then, the DataWriter applies the TopicQuery expression to all the samples up to the cut-off SN. If a sample with a SN smaller than or equal to the cut-off SN passes the filter expression, it will be included in the TopicQuery response.

When a DataWriter responds to a snapshot TopicQuery, it doesn't create any copies of the samples that are part of the response. As a result, a sample that passes the TopicQuery expression and should have been included in the response may be removed from the DataWriter queue before it reaches the DataReader that issued the TopicQuery. This can result in incomplete TopicQueries, which may or may not be necessary for your application to detect.

For instance, your application may issue a TopicQuery on a keyed topic using a filter on key-only fields and expect to receive the latest value for an instance that passes the filter. However, if the latest value for the instance is replaced with a newer value before the sample is delivered to the DataReader, it is important to detect this situation and re-issue the TopicQuery.

On the other hand, some applications may not care about which samples are returned in response to a TopicQuery, and are only interested in a snapshot of the DataWriter's queue. In such cases, the concept of incomplete TopicQueries can be ignored.

The application can detect incomplete TopicQueries by checking the DDS_INCOMPLETE_SNAPSHOT_TOPIC_QUERY bit in the DDS_SampleInfo::flag field of the last sample received for a snapshot TopicQuery.

To reduce the probability of receiving incomplete responses, DataWriters configured with a DDS_KEEP_LAST_HISTORY_QOS kind in the 47.12 HISTORY QosPolicy recalculate the cut-off SN for a snapshot TopicQuery if a sample is replaced in the DataWriter queue due to KEEP_LAST replacement before the complete TopicQuery response is published.

Snapshot TopicQueries are not automatically deleted once the DataWriters that respond to the TopicQuery have finished delivering the samples included in the response. It is your responsibility to delete the TopicQuery when it is no longer needed. To detect the end of a snapshot TopicQuery response, check for the DDS_INTERMEDIATE_TOPIC_QUERY_SAMPLE flag in the DDS_SampleInfo::flag field of the samples included in the response. For more information on how to use this flag, see 61.2 Reading TopicQuery Samples.

61.2 Reading TopicQuery Samples

A TopicQuery response is the set of samples published by a DataWriter in response to a TopicQuery. Because multiple DataWriters may respond to a TopicQuery, a DataReader may receive multiple responses for a single TopicQuery.

Responses can be proxied by Routing Service DataWriters by setting <topic_query_proxy>/<mode> to PROPAGATION. These DataWriters forward a TopicQuery response, but they are not considered to be responding to the TopicQuery.

Samples that a DataReader receives in response to a TopicQuery have a value for DDS_SampleInfo::topic_query_guid different than DDS_GUID_UNKNOWN.

In addition, the virtual GUID of the DataWriter(s) that responded to a TopicQuery is set in DDS_SampleInfo::source_guid.

Intermediate samples, not the final sample, have the flag DDS_INTERMEDIATE_TOPIC_QUERY_SAMPLE set in DDS_SampleInfo::flag for a given response to a snapshot TopicQuery. See 41.6 The SampleInfo Structure for more information.

If two DataWriters respond to a snapshot TopicQuery, you should expect two final samples, one per unique DDS_SampleInfo::source_guid responding to the TopicQuery, assuming that at least one sample passes the TopicQuery filter for each DataWriter.

The DDS_SampleInfo::flag field of the last sample received for a snapshot TopicQuery may have the DDS_INCOMPLETE_SNAPSHOT_TOPIC_QUERY bit set if the TopicQuery response is incomplete, which may or may not be necessary for your application to detect. See 61.1 Snapshot TopicQueries for more information.

61.3 Debugging Topic Queries

There are a number of ways in which to gain more insight into what is happening in an application that is creating Topic Queries.

61.3.1 The Built-in ServiceRequest DataReader

TopicQueries are communicated to publishing applications through a built-in ServiceRequest channel. The ServiceRequest channel is designed to be generic so that it can be used for many different purposes, one of which is TopicQueries.

When a DataReader creates a TopicQuery, a ServiceRequest message is sent containing the TopicQuery information. Just as there are built-in DataReaders for ParticipantBuiltinTopicData, SubscriptionBuiltinTopicData, and PublicationBuiltinTopicData, there is a fourth built-in DataReader for ServiceRequests. This built-in DataReader can be retrieved using the built-in Subscriber and its lookup_datareader(). The topic name is DDS_SERVICE_REQUEST_TOPIC_NAME. Installing a listener with the DataReaderListener's on_data_available callback() implemented will allow a publishing application to be notified whenever a TopicQuery has been received from a subscribing application.

The service_id of a ServiceRequest corresponding to a TopicQuery will be DDS_TOPIC_QUERY_SERVICE_REQUEST_ID and the instance_id will be equal to the GUID of the TopicQuery.

The request_body is a sequence of bytes containing more information about the TopicQuery. This information can be retrieved using the DDS_TopicQueryHelper_topic_query_data_from_service_request() function. The resulting TopicQueryData contains the TopicQuerySelection that the TopicQuery was created with, the GUID of the original DataReader that created the TopicQuery, and the topic name of that DataReader.

When TopicQueries are propagated through one or more instances of Routing Service, the last DataReader that issued the TopicQuery will be a Routing Service DataReader. The DDS_TopicQueryData::original_related_reader_guid, however, will be that of the first DataReader to have created the TopicQuery.

If you are seeing traffic from the ServiceRequest endpoints during system startup but are not using any of the features (such as TopicQueries) that rely on the ServiceRequest channel, you can disable the channel using the enabled_builtin_channels field in the 44.3 DISCOVERY_CONFIG QosPolicy (DDS Extension).

61.3.2 The on_service_request_accepted() DataWriter Listener Callback

It is possible that a ServiceRequest for a TopicQuery is received but is not immediately dispatched to a DataWriter. This can happen, for example, if a DataWriter was not matching with a DataReader at the time that the TopicQuery was received by the publishing application. The DDS_DataWriterListener's on_service_request_accepted() callback notifies a DataWriter when a ServiceRequest has been dispatched to that DataWriter. The DDS_ServiceRequestAcceptedStatus provides information about how many ServiceRequests have been accepted by the DataWriter since the last time that the status was read. The status also includes the DDS_ServiceRequestAcceptedStatus::last_request_handle, which is the InstanceHandle of the last ServiceRequest that was accepted. This instance handle can be used to read samples per instance from the built-in ServiceRequest DataReader and correlate which ServiceRequests have been dispatched to which DataWriters.

61.4 System Resource Considerations

61.4.1 Publishing Application

On the publishing side, the resource allocation associated with TopicQueries can be controlled using remote_topic_query_allocation (in the44.4 DOMAIN_PARTICIPANT_RESOURCE_LIMITS QosPolicy (DDS Extension) at the DomainParticipant level.

At the DataWriter level, you can control how many TopicQueries can be served in parallel by the DataWriter by setting the resource limit max_active_topic_queries in the 47.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension)).

61.4.2 Subscribing Application

On the DataReader side, each TopicQuery will get its own resources. These resources will not interfere with the resource limits associated with live data samples or other TopicQueries. For example, if max_samples (see 47.22 RESOURCE_LIMITS QosPolicy) is set to 10 and the DataReader creates one TopicQuery, then the DataReader will be able to store 10 samples for that TopicQuery and 10 samples for live data.

The maximum number of active TopicQueries that can be associated with a DataReader is configured using the resource limit max_topic_queries (see 48.2 DATA_READER_RESOURCE_LIMITS QosPolicy (DDS Extension)).