Filter Use Cases

Working with data filters. More...

Working with data filters.

Introduction

RTI Connext supports filtering data either during the exchange from com.rti.dds.publication.DataWriter to com.rti.dds.subscription.DataReader, or after the data has been stored at the com.rti.dds.subscription.DataReader.

Filtering during the exchange process is performed by a com.rti.dds.topic.ContentFilteredTopic, which is created by the com.rti.dds.subscription.DataReader as a way of specifying a subset of the data samples that it wishes to receive.

Filtering samples that have already been received by the com.rti.dds.subscription.DataReader is performed by creating a com.rti.dds.subscription.QueryCondition, which can then used to check for matching samples, be alerted when matching samples arrive, or retrieve matching samples through use of the com.rti.ndds.example.FooDataReader.read_w_condition or com.rti.ndds.example.FooDataReader.take_w_condition functions. (Conditions may also be used with the APIs com.rti.ndds.example.FooDataReader.read_next_instance_w_condition and com.rti.ndds.example.FooDataReader.take_next_instance_w_condition.)

Filtering may be performed on any topic, either keyed or un-keyed, except the Built-in Topics. Filtering may be perfomed on any field, subset of fields, or combination of fields, subject only to the limitations of the filter syntax, and some restrictions against filtering some sparse value types of the Dynamic Data API.

RTI Connext contains built in support for filtering using SQL syntax, described in the Queries and Filters Syntax module.

Overview of ContentFilteredTopic

Each com.rti.dds.topic.ContentFilteredTopic is created based on an existing com.rti.dds.topic.Topic. The com.rti.dds.topic.Topic specifies the field_names and field_types of the data contained within the topic. The com.rti.dds.topic.ContentFilteredTopic, by means of its filter_expression and expression_parameters, futher specifies the values of the data which the com.rti.dds.subscription.DataReader wishes to receive.

Custom filters may also be constructed and utilized as described in the Creating Custom Content Filters module.

Once the com.rti.dds.topic.ContentFilteredTopic has been created, a com.rti.dds.subscription.DataReader can be created using the filtered topic. The filter's characteristics are exchanged between the com.rti.dds.subscription.DataReader and any matching com.rti.dds.publication.DataWriter during the discovery processs.

If the com.rti.dds.publication.DataWriter allows (by com.rti.dds.infrastructure.DataWriterResourceLimitsQosPolicy.max_remote_reader_filters) and the number of filtered com.rti.dds.subscription.DataReader is less than or equal to 32, and the com.rti.dds.subscription.DataReader 's com.rti.dds.infrastructure.TransportMulticastQosPolicy is empty, then the com.rti.dds.publication.DataWriter will performing filtering and send to the com.rti.dds.subscription.DataReader only those samples that meet the filtering criteria.

If disallowed by the com.rti.dds.publication.DataWriter, or if more than 32 com.rti.dds.subscription.DataReader require filtering, or the com.rti.dds.subscription.DataReader has set the com.rti.dds.infrastructure.TransportMulticastQosPolicy, then the com.rti.dds.publication.DataWriter sends all samples to the com.rti.dds.subscription.DataReader, and the com.rti.dds.subscription.DataReader discards any samples that do not meet the filtering criteria.

Although the filter_expression cannot be changed once the com.rti.dds.topic.ContentFilteredTopic has been created, the expression_parameters can be modified using com.rti.dds.topic.ContentFilteredTopic.set_expression_parameters. Any changes made to the filtering criteria by means of com.rti.dds.topic.ContentFilteredTopic.set_expression_parameters, will be conveyed to any connected com.rti.dds.publication.DataWriter. New samples will be subject to the modified filtering criteria, but samples that have already been accepted or rejected are unaffected. However, if the com.rti.dds.subscription.DataReader connects to a com.rti.dds.publication.DataWriter that re-sends its data, the re-sent samples will be subjected to the new filtering criteria.

Overview of QueryCondition

com.rti.dds.subscription.QueryCondition combine aspects of the content filtering capabilities of com.rti.dds.topic.ContentFilteredTopic with state filtering capabilities of com.rti.dds.subscription.ReadCondition to create a reconfigurable means of filtering or searching data in the com.rti.dds.subscription.DataReader queue.

com.rti.dds.subscription.QueryCondition may be created on a disabled com.rti.dds.subscription.DataReader, or after the com.rti.dds.subscription.DataReader has been enabled. If the com.rti.dds.subscription.DataReader is enabled, and has already recevied and stored samples in its queue, then all data samples in the are filtered against the com.rti.dds.subscription.QueryCondition filter criteria at the time that the com.rti.dds.subscription.QueryCondition is created. (Note that an exclusive lock is held on the com.rti.dds.subscription.DataReader sample queue for the duration of the com.rti.dds.subscription.QueryCondition creation).

Once created, incoming samples are filtered against all com.rti.dds.subscription.QueryCondition filter criteria at the time of their arrival and storage into the com.rti.dds.subscription.DataReader queue.

The number of com.rti.dds.subscription.QueryCondition filters that an individual com.rti.dds.subscription.DataReader may create is set by com.rti.dds.infrastructure.DataReaderResourceLimitsQosPolicy.max_query_condition_filters, to an upper maximum of 32.

Filtering with ContentFilteredTopic

• Set up subscriber

• Set up a topic

• Create a ContentFilteredTopic, of user data type Foo:

  String cft_param_list[] = {"1", "100"};
  StringSeq cft_parameters = new StringSeq(java.util.Arrays.asList(cft_param_list));
  
  ContentFilteredTopic cft = participant.create_contentfilteredtopic(
                                  "ContentFilteredTopic",
                                  topic,
                                  "value > %0 AND value < %1",
                                  cft_parameters);
  if (cft == null) {
      System.err.println("create_contentfilteredtopic error\n");
      return;
  }                     

• Create a FooReader using the ContentFilteredTopic:

  FooDataReader reader = (FooDataReader)
                             subscriber.create_datareader(
                                 cft,
                                 datareader_qos, // or Subscriber.DATAREADER_QOS_DEFAULT //
                                 listener, // or null //
                                 StatusKind.STATUS_MASK_ALL);
  
  if (reader == null) {
      System.err.println("create_datareader error\n");
      return;
  }                         

  Once setup, reading samples with a com.rti.dds.topic.ContentFilteredTopic is exactly the same as normal reads or takes, as decribed in DataReader Use Cases.

• Changing filter crieria using set_expression_parameters:

  cft_parameters.set(0, "5");
  cft_parameters.set(1, "9");
  cft.set_expression_parameters(cft_parameters);

Filtering with Query Conditions

• Given a data reader of type Foo

  FooDataReader reader = ...;

• Creating a QueryCondition

  QueryCondition queryCondition = null;
  String qc_param_list[] = {"1", "100"};
  
  StringSeq qc_parameters = new StringSeq(java.util.Arrays.asList(cft_param_list));
  queryCondition = reader.create_querycondition(SampleStateKind.NOT_READ_SAMPLE_STATE,
                                                ViewStateKind.ANY_VIEW_STATE,
                                                InstanceStateKind.ANY_INSTANCE_STATE,
                                                "value > %0 AND value < %1",
                                                qc_parameters);
  if (queryCondition == null) {
      System.err.println("create_querycondition error\n");
      return;
  }

• Reading matching samples with a com.rti.dds.subscription.QueryCondition

  FooSeq _dataSeq = new FooSeq();
  SampleInfoSeq _infoSeq = new SampleInfoSeq();
  
  try {
      reader.read_w_condition(_dataSeq, _infoSeq,
                              ResourceLimitsQosPolicy.LENGTH_UNLIMITED,
                              queryCondition);
      for(int i = 0; i < _dataSeq.size(); ++i) {
          SampleInfo info = (SampleInfo)_infoSeq.get(i);
          if (info.valid_data) {
              // --- Process data here --- //
          }
      }
  } catch (RETCODE_NO_DATA noData) {
      // No data to process
  } finally {
      reader.return_loan(_dataSeq, _infoSeq);
  }

• com.rti.dds.subscription.QueryCondition.set_query_parameters is used similarly to com.rti.dds.topic.ContentFilteredTopic.set_expression_parameters, and the same coding techniques can be used.

• Any com.rti.dds.subscription.QueryCondition that have been created must be deleted before the com.rti.dds.subscription.DataReader can be deleted. This can be done using com.rti.dds.subscription.DataReader.delete_contained_entities or manually as in:

  retcode = reader.delete_readcondition(queryCondition);

Filtering Performance

Although RTI Connext supports filtering on any field or combination of fields using the SQL syntax of the built-in filter, filters for keyed topics that filter solely on the contents of key fields have the potential for much higher performance. This is because for key field only filters, the com.rti.dds.subscription.DataReader caches the results of the filter (pass or not pass) for each instance. When another sample of the same instance is seen at the com.rti.dds.subscription.DataReader, the filter results are retrieved from cache, dispensing with the need to call the filter function.

This optimization applies to all filtering using the built-in SQL filter, performed by the com.rti.dds.subscription.DataReader, for either com.rti.dds.topic.ContentFilteredTopic or com.rti.dds.subscription.QueryCondition. This does not apply to filtering perfomed for com.rti.dds.topic.ContentFilteredTopic by the com.rti.dds.publication.DataWriter.