Working with data filters.
Working with data filters.
Introduction <br>
RTI Connext supports filtering data either during the exchange from DDS_DataWriter to DDS_DataReader, or after the data has been stored at the DDS_DataReader.
Filtering during the exchange process is performed by a DDS_ContentFilteredTopic, which is created by the DDS_DataReader as a way of specifying a subset of the data samples that it wishes to receive.
Filtering samples that have already been received by the DDS_DataReader is performed by creating a DDS_QueryCondition, which can then used to check for matching samples, be alerted when matching samples arrive, or retrieve matching samples through use of the FooDataReader_read_w_condition or FooDataReader_take_w_condition functions. (Conditions may also be used with the APIs FooDataReader_read_next_instance_w_condition and FooDataReader_take_next_instance_w_condition.)
Filtering may be performed on any topic, either keyed or un-keyed, except the Built-in Topics. Filtering may be perfomed on any field, subset of fields, or combination of fields, subject only to the limitations of the filter syntax, and some restrictions against filtering some sparse value types of the Dynamic Data API.
RTI Connext contains built in support for filtering using SQL syntax, described in the Queries and Filters Syntax module.
Overview of ContentFilteredTopic
Each DDS_ContentFilteredTopic is created based on an existing DDS_Topic. The DDS_Topic specifies the field_names and field_types of the data contained within the topic. The DDS_ContentFilteredTopic, by means of its filter_expression
and expression_parameters
, further specifies the values of the data which the DDS_DataReader wishes to receive.
Custom filters may also be constructed and utilized as described in the Creating Custom Content Filters module.
Once the DDS_ContentFilteredTopic has been created, a DDS_DataReader can be created using the filtered topic. The filter's characteristics are exchanged between the DDS_DataReader and any matching DDS_DataWriter during the discovery process.
If the DDS_DataWriter allows (by DDS_DataWriterResourceLimitsQosPolicy::max_remote_reader_filters), and the DDS_DataReader 's DDS_TransportMulticastQosPolicy is empty, then the DDS_DataWriter will filter out samples that don't meet the filtering criteria.
If disallowed by the DDS_DataWriter, or the DDS_DataReader has set the DDS_TransportMulticastQosPolicy, then the DDS_DataWriter sends all samples to the DDS_DataReader, and the DDS_DataReader discards any samples that do not meet the filtering criteria.
The expression_parameters
can be modified using DDS_ContentFilteredTopic_set_expression_parameters. Any changes made to the filtering criteria by means of DDS_ContentFilteredTopic_set_expression_parameters, will be conveyed to any connected DDS_DataWriter. New samples will be subject to the modified filtering criteria, but samples that have already been accepted or rejected are unaffected. However, if the DDS_DataReader connects to a DDS_DataWriter that re-sends its data, the re-sent samples will be subjected to the new filtering criteria.
Overview of QueryCondition
DDS_QueryCondition combine aspects of the content filtering capabilities of DDS_ContentFilteredTopic with state filtering capabilities of DDS_ReadCondition to create a reconfigurable means of filtering or searching data in the DDS_DataReader queue.
DDS_QueryCondition may be created on a disabled DDS_DataReader, or after the DDS_DataReader has been enabled. If the DDS_DataReader is enabled, and has already recevied and stored samples in its queue, then all data samples in the are filtered against the DDS_QueryCondition filter criteria at the time that the DDS_QueryCondition is created. (Note that an exclusive lock is held on the DDS_DataReader sample queue for the duration of the DDS_QueryCondition creation).
Once created, incoming samples are filtered against all DDS_QueryCondition filter criteria at the time of their arrival and storage into the DDS_DataReader queue.
The number of DDS_QueryCondition filters that an individual DDS_DataReader may create is set by DDS_DataReaderResourceLimitsQosPolicy::max_query_condition_filters, to an upper maximum of 32.
Filtering with ContentFilteredTopic
- Set up subscriber
- Set up a topic
- Create a ContentFilteredTopic, of user data type
Foo:
const char* cft_param_list[] = {"1", "100"};
DDS_StringSeq_initialize(&cft_parameters);
DDS_StringSeq_set_maximum(&cft_parameters, 2);
DDS_StringSeq_from_array(&cft_parameters, param_list, 2);
"ContentFilteredTopic",
Foo_topic,
"value > %0 AND value < %1",
&cft_parameters);
if (cft == NULL) {
printf("create_contentfilteredtopic error\n");
subscriber_shutdown(participant);
return -1;
}
DDS_ContentFilteredTopic * DDS_DomainParticipant_create_contentfilteredtopic(DDS_DomainParticipant *self, const char *name, DDS_Topic *related_topic, const char *filter_expression, const struct DDS_StringSeq *expression_parameters)
Creates a DDS_ContentFilteredTopic, that can be used to do content-based subscriptions.
struct DDS_ContentFilteredTopicWrapperI DDS_ContentFilteredTopic
<<interface>> Specialization of DDS_TopicDescription that allows for content-based subscriptions.
Definition: topic.ifc:713
Instantiates FooSeq < char* > with value type semantics.
Definition: infrastructure.ifc:407
- Create a FooReader using the ContentFilteredTopic:
&reader_qos,
&reader_listener,
if (reader == NULL) {
printf("create_datareader error\n");
subscriber_shutdown(participant);
return -1;
}
if (Foo_reader == NULL) {
printf("DataReader narrow error\n");
subscriber_shutdown(participant);
return -1;
}
FooDataReader * FooDataReader_narrow(DDS_DataReader *reader)
Narrow the given DDS_DataReader pointer to a FooDataReader pointer.
struct DDS_DataReaderImpl DDS_DataReader
<<interface>> Allows the application to: (1) declare the data it wishes to receive (i....
Definition: subscription.ifc:230
#define DDS_STATUS_MASK_ALL
All bits are set.
Definition: infrastructure.ifc:1431
DDS_DataReader * DDS_Subscriber_create_datareader(DDS_Subscriber *self, DDS_TopicDescription *topic, const struct DDS_DataReaderQos *qos, const struct DDS_DataReaderListener *listener, DDS_StatusMask mask)
Creates a DDS_DataReader that will be attached and belong to the DDS_Subscriber.
DDS_TopicDescription * DDS_ContentFilteredTopic_as_topicdescription(DDS_ContentFilteredTopic *contentFilteredTopic)
Access a DDS_ContentFilteredTopic's supertype instance.
<<interface>> <<generic>> User data type-specific data reader.
Definition: data.ifc:795
Once setup, reading samples with a DDS_ContentFilteredTopic is exactly the same as normal reads or takes, as decribed in DataReader Use Cases.
- Changing filter crieria using set_expression_parameters:
¶meters);
printf("set_expression_parameters error\n");
subscriber_shutdown(participant);
return -1;
}
@ DDS_RETCODE_OK
Successful return.
Definition: infrastructure.ifc:1354
void DDS_String_free(char *str)
Delete a string.
char * DDS_String_dup(const char *str)
Clone a string. Creates a new string that duplicates the value of string.
DDS_ReturnCode_t DDS_ContentFilteredTopic_set_expression_parameters(DDS_ContentFilteredTopic *self, const struct DDS_StringSeq *parameters)
Set the expression_parameters.
Filtering with Query Conditions
- Given a data reader of type Foo
- Creating a QueryCondition
const char *qc_param_list[] = {"1","100"};
DDS_StringSeq_initialize(&qc_parameters);
DDS_StringSeq_set_maximum(&qc_parameters, 2);
DDS_StringSeq_from_array(&qc_parameters, qc_param_list, 2);
"value > %0 AND value < %1",
&qc_parameters);
if (queryCondition == NULL) {
printf("create_query_condition error\n");
subscriber_shutdown(participant);
return -1;
}
@ DDS_ALIVE_INSTANCE_STATE
Instance is currently in existence.
Definition: subscription.ifc:1115
struct DDS_QueryConditionImpl DDS_QueryCondition
<<interface>> These are specialised DDS_ReadCondition objects that allow the application to also spec...
Definition: subscription.ifc:1498
DDS_QueryCondition * DDS_DataReader_create_querycondition(DDS_DataReader *self, DDS_SampleStateMask sample_states, DDS_ViewStateMask view_states, DDS_InstanceStateMask instance_states, const char *query_expression, const struct DDS_StringSeq *query_parameters)
Creates a DDS_QueryCondition.
@ DDS_NOT_READ_SAMPLE_STATE
Sample has not been read.
Definition: subscription.ifc:1050
const DDS_ViewStateMask DDS_ANY_VIEW_STATE
Any view state DDS_NEW_VIEW_STATE | DDS_NOT_NEW_VIEW_STATE.
- Reading matching samples with a DDS_QueryCondition
&data_seq, &info_seq,
queryCondition);
printf("no matching data\n");
printf("read_w_condition error %d\n", retcode);
subscriber_shutdown(participant);
return -1;
} else {
if (DDS_SampleInfoSeq_get_reference(&info_seq, i)->valid_data) {
}
&data_seq, &info_seq);
printf("return loan error %d\n", retcode);
subscriber_shutdown(participant);
return -1;
}
}
DDS_ReturnCode_t FooDataReader_read_w_condition(FooDataReader *self, struct FooSeq *received_data, struct DDS_SampleInfoSeq *info_seq, DDS_Long max_samples, DDS_ReadCondition *condition)
Accesses via FooDataReader_read the samples that match the criteria specified in the DDS_ReadConditio...
DDS_ReturnCode_t FooDataReader_return_loan(FooDataReader *self, FooSeq *received_data, struct DDS_SampleInfoSeq *info_seq)
Indicates to the DDS_DataReader that the application is done accessing the collection of received_dat...
const DDS_Long DDS_LENGTH_UNLIMITED
A special value indicating an unlimited quantity.
@ DDS_RETCODE_NO_DATA
Indicates a transient situation where the operation did not return any data but there is no inherent ...
Definition: infrastructure.ifc:1398
#define DDS_SEQUENCE_INITIALIZER
An initializer for new sequence instances.
Definition: sequence.ifc:566
DDS_Long FooSeq_get_length(const struct FooSeq *self)
Get the logical length of this sequence.
Declares IDL sequence < DDS_SampleInfo > .
Definition: subscription.ifc:1348
<<interface>> <<generic>> A type-safe, ordered collection of elements. The type of these elements is ...
Definition: sequence.ifc:580
- DDS_QueryCondition_set_query_parameters is used similarly to DDS_ContentFilteredTopic_set_expression_parameters, and the same coding techniques can be used.
- Any DDS_QueryCondition that have been created must be deleted before the DDS_DataReader can be deleted. This can be done using DDS_DataReader_delete_contained_entities or manually as in:
queryCondition);
DDS_ReturnCode_t DDS_DataReader_delete_readcondition(DDS_DataReader *self, DDS_ReadCondition *condition)
Deletes a DDS_ReadCondition or DDS_QueryCondition attached to the DDS_DataReader.
Filtering Performance
Although RTI Connext supports filtering on any field or combination of fields using the SQL syntax of the built-in filter, filters for keyed topics that filter solely on the contents of key fields have the potential for much higher performance. This is because for key-only filters, the DDS_DataReader caches the results of the filter (pass or not pass) for each instance. When another sample of the same instance is seen at the DDS_DataReader, the filter results are retrieved from the cache, dispensing with the need to call the filter function.
This optimization applies to all filtering using the built-in SQL filter, performed by the DDS_DataReader, for either DDS_ContentFilteredTopic or DDS_QueryCondition. This does not apply to filtering perfomed for DDS_ContentFilteredTopic by the DDS_DataWriter.