Working with data filters.
Working with data filters.
Introduction <br>
RTI Connext supports filtering data either during the exchange from DDSDataWriter to DDSDataReader, or after the data has been stored at the DDSDataReader.
Filtering during the exchange process is performed by a DDSContentFilteredTopic, which is created by the DDSDataReader as a way of specifying a subset of the data samples that it wishes to receive.
Filtering samples that have already been received by the DDSDataReader is performed by creating a DDSQueryCondition, which can then used to check for matching samples, be alerted when matching samples arrive, or retrieve matching samples through use of the FooDataReader::read_w_condition or FooDataReader::take_w_condition functions. (Conditions may also be used with the APIs FooDataReader::read_next_instance_w_condition and FooDataReader::take_next_instance_w_condition.)
Filtering may be performed on any topic, either keyed or un-keyed, except the Built-in Topics. Filtering may be perfomed on any field, subset of fields, or combination of fields, subject only to the limitations of the filter syntax, and some restrictions against filtering some sparse value types of the Dynamic Data API.
RTI Connext contains built in support for filtering using SQL syntax, described in the Queries and Filters Syntax module.
Overview of ContentFilteredTopic
Each DDSContentFilteredTopic is created based on an existing DDSTopic. The DDSTopic specifies the field_names and field_types of the data contained within the topic. The DDSContentFilteredTopic, by means of its filter_expression
and expression_parameters
, further specifies the values of the data which the DDSDataReader wishes to receive.
Custom filters may also be constructed and utilized as described in the Creating Custom Content Filters module.
Once the DDSContentFilteredTopic has been created, a DDSDataReader can be created using the filtered topic. The filter's characteristics are exchanged between the DDSDataReader and any matching DDSDataWriter during the discovery process.
If the DDSDataWriter allows (by DDS_DataWriterResourceLimitsQosPolicy::max_remote_reader_filters), and the DDSDataReader 's DDS_TransportMulticastQosPolicy is empty, then the DDSDataWriter will filter out samples that don't meet the filtering criteria.
If disallowed by the DDSDataWriter, or the DDSDataReader has set the DDS_TransportMulticastQosPolicy, then the DDSDataWriter sends all samples to the DDSDataReader, and the DDSDataReader discards any samples that do not meet the filtering criteria.
The expression_parameters
can be modified using DDSContentFilteredTopic::set_expression_parameters. Any changes made to the filtering criteria by means of DDSContentFilteredTopic::set_expression_parameters, will be conveyed to any connected DDSDataWriter. New samples will be subject to the modified filtering criteria, but samples that have already been accepted or rejected are unaffected. However, if the DDSDataReader connects to a DDSDataWriter that re-sends its data, the re-sent samples will be subjected to the new filtering criteria.
Overview of QueryCondition
DDSQueryCondition combine aspects of the content filtering capabilities of DDSContentFilteredTopic with state filtering capabilities of DDSReadCondition to create a reconfigurable means of filtering or searching data in the DDSDataReader queue.
DDSQueryCondition may be created on a disabled DDSDataReader, or after the DDSDataReader has been enabled. If the DDSDataReader is enabled, and has already recevied and stored samples in its queue, then all data samples in the are filtered against the DDSQueryCondition filter criteria at the time that the DDSQueryCondition is created. (Note that an exclusive lock is held on the DDSDataReader sample queue for the duration of the DDSQueryCondition creation).
Once created, incoming samples are filtered against all DDSQueryCondition filter criteria at the time of their arrival and storage into the DDSDataReader queue.
The number of DDSQueryCondition filters that an individual DDSDataReader may create is set by DDS_DataReaderResourceLimitsQosPolicy::max_query_condition_filters, to an upper maximum of 32.
Filtering with ContentFilteredTopic
- Set up subscriber
- Set up a topic
- Create a ContentFilteredTopic, of user data type
Foo:
const char* cft_param_list[] = {"1", "100"};
cft_parameters.from_array(cft_param_list, 2);
cft = participant->create_contentfilteredtopic("ContentFilteredTopic",
Foo_topic,
"value > %0 AND value < %1",
cft_parameters);
if (cft == NULL) {
printf("create_contentfilteredtopic error\n");
subscriber_shutdown(participant);
return -1;
}
<<interface>> Specialization of DDSTopicDescription that allows for content-based subscriptions.
Definition: topic.ifcxx:128
Instantiates FooSeq < char* > with value type semantics.
Definition: infrastructure.ifc:407
- Create a FooReader using the ContentFilteredTopic:
reader = subscriber->create_datareader(cft,
datareader_qos,
reader_listener,
if (reader == NULL) {
printf("create_datareader error\n");
subscriber_shutdown(participant);
return -1;
}
if (Foo_reader == NULL) {
printf("DataReader narrow error\n");
subscriber_shutdown(participant);
return -1;
}
<<interface>> Allows the application to: (1) declare the data it wishes to receive (i....
Definition: subscription.ifcxx:565
#define DDS_STATUS_MASK_ALL
All bits are set.
Definition: infrastructure.ifc:1431
<<interface>> <<generic>> User data type-specific data reader.
Definition: data.ifc:795
static FooDataReader * narrow(DDSDataReader *reader)
Narrow the given DDSDataReader pointer to a FooDataReader pointer.
Once setup, reading samples with a DDSContentFilteredTopic is exactly the same as normal reads or takes, as decribed in DataReader Use Cases.
- Changing filter crieria using set_expression_parameters:
printf("set_expression_parameters error\n");
subscriber_shutdown(participant);
return -1;
}
virtual DDS_ReturnCode_t get_expression_parameters(DDS_StringSeq ¶meters)=0
Get the expression_parameters.
virtual DDS_ReturnCode_t set_expression_parameters(const DDS_StringSeq ¶meters)=0
Set the expression_parameters.
@ DDS_RETCODE_OK
Successful return.
Definition: infrastructure.ifc:1354
void DDS_String_free(char *str)
Delete a string.
char * DDS_String_dup(const char *str)
Clone a string. Creates a new string that duplicates the value of string.
Filtering with Query Conditions
- Given a data reader of type Foo
- Creating a QueryCondition
const char *qc_param_list[] = {"0","100"};
qc_parameters.from_array(qc_param_list,2);
"value > %0 AND value < %1",
qc_parameters);
if (queryCondition == NULL) {
printf("create_query_condition error\n");
goto error_exit;
}
virtual DDSQueryCondition * create_querycondition(DDS_SampleStateMask sample_states, DDS_ViewStateMask view_states, DDS_InstanceStateMask instance_states, const char *query_expression, const struct DDS_StringSeq &query_parameters)
Creates a DDSQueryCondition.
Definition: subscription.ifcxx:596
<<interface>> These are specialised DDSReadCondition objects that allow the application to also speci...
Definition: subscription.ifcxx:288
@ DDS_ALIVE_INSTANCE_STATE
Instance is currently in existence.
Definition: subscription.ifc:1115
@ DDS_NOT_READ_SAMPLE_STATE
Sample has not been read.
Definition: subscription.ifc:1050
const DDS_ViewStateMask DDS_ANY_VIEW_STATE
Any view state DDS_NEW_VIEW_STATE | DDS_NOT_NEW_VIEW_STATE.
- Reading matching samples with a DDSQueryCondition
queryCondition);
printf("no matching data\n");
printf("read_w_condition error %d\n", retcode);
subscriber_shutdown(participant);
return -1;
} else {
for (i = 0; i < data_seq.
length(); ++i) {
if (info_seq[i].valid_data) {
}
printf("return loan error %d\n", retcode);
subscriber_shutdown(participant);
return -1;
}
}
const DDS_Long DDS_LENGTH_UNLIMITED
A special value indicating an unlimited quantity.
@ DDS_RETCODE_NO_DATA
Indicates a transient situation where the operation did not return any data but there is no inherent ...
Definition: infrastructure.ifc:1398
Declares IDL sequence < DDS_SampleInfo > .
Definition: subscription.ifc:1348
virtual DDS_ReturnCode_t return_loan(FooSeq &received_data, DDS_SampleInfoSeq &info_seq)=0
Indicates to the DDSDataReader that the application is done accessing the collection of received_data...
virtual DDS_ReturnCode_t read_w_condition(FooSeq &received_data, DDS_SampleInfoSeq &info_seq, DDS_Long max_samples, DDSReadCondition *condition)=0
Accesses via FooDataReader::read the samples that match the criteria specified in the DDSReadConditio...
<<interface>> <<generic>> A type-safe, ordered collection of elements. The type of these elements is ...
Definition: sequence.ifc:580
DDS_Long length() const
Get the logical length of this sequence.
- DDSQueryCondition::set_query_parameters is used similarly to DDSContentFilteredTopic::set_expression_parameters, and the same coding techniques can be used.
- Any DDSQueryCondition that have been created must be deleted before the DDSDataReader can be deleted. This can be done using DDSDataReader::delete_contained_entities or manually as in:
virtual DDS_ReturnCode_t delete_readcondition(DDSReadCondition *condition)
Deletes a DDSReadCondition or DDSQueryCondition attached to the DDSDataReader.
Definition: subscription.ifcxx:646
Filtering Performance
Although RTI Connext supports filtering on any field or combination of fields using the SQL syntax of the built-in filter, filters for keyed topics that filter solely on the contents of key fields have the potential for much higher performance. This is because for key-only filters, the DDSDataReader caches the results of the filter (pass or not pass) for each instance. When another sample of the same instance is seen at the DDSDataReader, the filter results are retrieved from the cache, dispensing with the need to call the filter function.
This optimization applies to all filtering using the built-in SQL filter, performed by the DDSDataReader, for either DDSContentFilteredTopic or DDSQueryCondition. This does not apply to filtering perfomed for DDSContentFilteredTopic by the DDSDataWriter.