Custom Content Filters

You are here:

Custom Content Filters

By default, a ContentFilteredTopic will use a SQL-like content filter, DDS_SQLFILTER_NAME (see SQL Filter Expression Notation), which implements a superset of the content filter. There is another builtin filter, DDS_STRINGMATCHFILTER_NAME (see STRINGMATCH Filter Expression Notation). Both of these are automatically registered.

If you want to use a different filter, you must register it first, then create the ContentFilteredTopic using create_contentfilteredtopic_with_filter() (see Creating ContentFilteredTopics).

One reason to use a custom filter is that the default filter can only filter based on relational operations between topic members, not on a computation involving topic members. For example, if you want to filter based on the sum of the members, you must create your own filter.

Notes:

The API for using a custom content filter is subject to change in a future release.
Custom content filters are not supported when using the .NET APIs

Filtering on the Writer Side with Custom Filters

There are two approaches for performing writer-side filtering. The first approach is to evaluate each written DDS sample against filters of all the readers that have content filter specified and identify the readers whose filter passes the DDS sample.

The second approach is to evaluate the written DDS sample once for the writer and then rely on the filter implementation to provide a set of readers whose filter passes the DDS sample. This approach allows the filter implementation to cache the result of filtering, if possible. For example, consider a scenario where the data is described by the struct shown below, where 10<x<20:

struct MyData {
	int x;
	int y;
};

If the filter expression is based only on the x field, the filter implementation can maintain a hash map for all the different values of x and cache the filtering results in the hash map. Then any future evaluations will only be O(1), because it only requires a lookup in the hash map.

But if in the same example, a reader has a content filter that is based on both x and y, or just y, the filter implementation cannot cache the result—because the filter was only maintaining a hash map for x. In this case, the filter implementation can inform Connext DDS that it will not be caching the result for those DataReaders. The filter can use DDS_ExpressionProperty to indicate to the middleware whether or not it will cache the results for DataReader. DDS_ExpressionProperty describes DDS_ExpressionProperty.

DDS_ExpressionProperty
Type	Field Name	Description
DDS_Boolean	key_only_filter	Indicates if the filter expression is based only on key fields. In this case, Connext DDS itself can cache the filtering results.
DDS_Boolean	writer_side_filter_optimization	Indicates if the filter implementation can cache the filtering result for the expression provided. If this is true then Connext DDS will do no caching or explicit filter evaluation for the associated DataReader. It will instead rely on the filter implementation to provide appropriate results.

Registering a Custom Filter

To use a custom filter, it must be registered in the following places:

Register the custom filter in any subscribing application in which the filter is used to create a ContentFilteredTopic and corresponding DataReader.
In each publishing application, you only need to register the custom filter if you want to perform writer-side filtering. A DataWriter created with an associated filter will use that filter if it discovers a matched DataReader that uses the same filter.

For example, suppose Application A on the subscription side creates a Topic named X and a ContentFilteredTopic named filteredX (and a corresponding DataReader), using a previously registered content filter, myFilter. With only that, you will have filtering on the subscription side. If you also want to perform filtering in any application that publishes Topic X, then you also need to register the same definition of the ContentFilter myFilter in that application.

To register a new filter, use the DomainParticipant’s register_contentfilter() operation1This operation is an extension to the DDS standard.:

DDS_ReturnCode_t register_contentfilter(
	const char * filter_name,
	const DDSContentFilter * contentfilter)

filter_name

The name of the filter. The name must be unique within the DomainParticipant. The filter_name cannot have a length of 0. The same filtering functions and handle can be registered under different names.

content_filter

This class specifies the functions that will be used to process the filter.

You must derive from the DDSContentFilter base class and implement the virtual compile, evaluate, and finalize functions described below.

Optionally, you can derive from the DDSWriterContentFilter base class instead, to implement additional filtering operations that will be used by the DataWriter. When performing writer-side filtering, these operations allow a DDS sample to be evaluated once for the DataWriter, instead of evaluating the DDS sample for every DataReader that is matched with the DataWriter. An instance of the derived class is then used as an argument when calling register_contentfilter().

compile

The function that will be used to compile a filter expression and parameters. Connext DDS will call this function when a ContentFilteredTopic is created and when the filter parameters are changed. This parameter cannot be NULL. See Compile Function. This is a member of DDSContentFilter and DDSWriterContentFilter.

evaluate

The function that will be called by Connext DDS each time a DDS sample is received. Its purpose is to evaluate the DDS sample based on the filter. This parameter cannot be NULL. See Evaluate Function. This is a member of DDSContentFilter and DDSWriterContentFilter.

finalize

The function that will be called by Connext DDS when an instance of the custom content filter is no longer needed. This parameter may be NULL. See Finalize Function. This is a member of DDSContentFilter and DDSWriterContentFilter.

writer_attach

The function that will be used to create some state required to perform filtering on the writer side using the operations provided in DDSWriterContentFilter. Connext DDS will call this function for every DataWriter; it will be called only the first time the DataWriter matches a DataReader using the specified filter. This function will not be called for any subsequent DataReaders that match the DataWriter and are using the same filter. See Writer Attach Function. This is a member of DDSWriterContentFilter.

writer_detach

The function that will be used to delete any state created using the writer_attach function. Connext DDS will call this function when the DataWriter is deleted. See Writer Detach Function. This is a member of DDSWriterContentFilter.

writer_compile

The function that will be used by the DataWriter to compile filter expression and parameters provided by the reader. Connext DDS will call this function when the DataWriter discovers a DataReader with a ContentFilteredTopic or when a DataWriter is notified of a change in DataReader’s filter parameter. This function will receive as an input a DDS_Cookie_t which uniquely identifies the DataReader for which the function was invoked. See Writer Compile Function. This is a member of DDSWriterContentFilter.

writer_evaluate

The function that will be called by Connext DDS every time a DataWriter writes a new DDS sample. Its purpose is to evaluate the DDS sample for all the readers for which the DataWriter is performing writer-side filtering and return the list of DDS_Cookie_t associated with the DataReaders whose filter pass the DDS sample. See Writer Evaluate Function.

writer_return_loan

The function that will be called by Connext DDS to return the loan on a sequence of DDS_Cookie_t provided by the writer_evaluate function. See Writer Return Loan Function. This is a member of DDSWriterContentFilter.

writer_finalize

The function that will be called by Connext DDS to notify the filter implementation that the DataWriter is no longer matching with a DataReader for which it was previously performing writer-side filtering. This will allow the filter to purge any state it was maintaining for the DataReader. See Writer Finalize Function. This is a member of DDSWriterContentFilter.

Unregistering a Custom Filter

To unregister a filter, use the DomainParticipant’s unregister_contentfilter() operation2This operation is an extension to the DDS standard., which is useful if you want to reuse a particular filter name. (Note: You do not have to unregister the filter before deleting the parent DomainParticipant. If you do not need to reuse the filter name to register another filter, there is no reason to unregister the filter.)

DDS_ReturnCode_t unregister_contentfilter(const char * filter_name)

filter_name

The name of the previously registered filter. The name must be unique within the DomainParticipant. The filter_name cannot have a length of 0.

If you attempt to unregister a filter that is still being used by a ContentFilteredTopic, unregister_contentfilter() will return PRECONDITION_NOT_MET.

If there are still existing discovered DataReaders with the same filter_name and the filter's compile function has previously been called on the discovered DataReaders, the filter’s finalize function will be called on those discovered DataReaders before the content filter is unregistered. This means filtering will be performed on the application that is creating the DataReader.

Retrieving a ContentFilter

If you know the name of a ContentFilter, you can get a pointer to its structure. If the ContentFilter has not already been registered, this operation will return NULL.

DDS_ContentFilter *lookup_contentfilter (const char * filter_name)

Compile Function

The compile function specified in the ContentFilter will be used to compile a filter expression and parameters. Please note that the term ‘compile’ is intentionally defined very broadly. It is entirely up to you, as the user, to decide what this function should do. The only requirement is that the error_code parameter passed to the compile function must return OK on successful execution. For example:

DDS_ReturnCode_t sample_compile_function(
    void **  new_compile_data, const char * expression,
    const DDS_StringSeq & parameters, 
    const DDS_TypeCode * type_code,
    const char *  type_class_name,
    void *  old_compile_data) 
{
  *new_compile_data = (void*)DDS_String_dup(parameters[0]);
  return DDS_RETCODE_OK;
}

Where:

new_compile_data	A user-specified opaque pointer of this instance of the content filter. This value is passed to the evaluate and finalize functions
expression	An ASCIIZ string with the filter expression the ContentFilteredTopic was created with. Note that the memory used by the parameter pointer is owned by Connext DDS. If you want to manipulate this string, you must make a copy of it first. Do not free the memory for this string.
parameters	A string sequence of expression parameters used to create the ContentFilteredTopic. The string sequence is equal (but not identical) to the string sequence passed to create_contentfilteredtopic() (see expression_parameters in Creating ContentFilteredTopics). The sequence passed to the compile function is owned by Connext DDS and must not be referred to outside the compile function.
type_code	A pointer to the type code of the related Topic. A type code is a description of the topic members, such as their type (long, octet, etc.), but does not contain any information with respect to the memory layout of the structures. The type code can be used to write filters that can be used with any type. See Using Generated Types without Connext DDS (Standalone) . [Note: If you are using the Java API, this parameter will always be NULL.]
type_class_name	Fully qualified class name of the related Topic.
old_compile_data	The new_compile_data value from a previous call to this instance of a content filter. If compile is called more than once for an instance of a ContentFilteredTopic (such as if the expression parameters are changed), then the new_compile_data value returned by the previous invocation is passed in the old_compile_data parameter (which can be NULL). If this is a new instance of the filter, NULL is passed. This parameter is useful for freeing or reusing previously allocated resources.

Evaluate Function

The evaluate function specified in the ContentFilter will be called each time a DDS sample is received. This function’s purpose is to determine if a DDS sample should be filtered out (not put in the receive queue).

For example:

DDS_Boolean sample_evaluate_function(
      void* compile_data,
      const void* sample,
      struct DDS_FilterSampleInfo * meta_data) {
  char *parameter = (char*)compile_data;
  DDS_Long x;
  Foo *foo_sample = (Foo*)sample;
  sscanf(parameter,"%d",&x);
  return (foo_sample->x > x ? DDS_BOOLEAN_FALSE : DDS_BOOLEAN_TRUE);
}

The function may use the following parameters:

compile_data	The last return value from the compile function for this instance of the content filter. Can be NULL.
sample	A pointer to a C structure with the data to filter. Note that the evaluate function always receives deserialized data.
meta_data	A pointer to the meta data associated with the DDS sample.

Note: Currently the meta_data field only supports related_sample_identity (described in DDS_WriteParams_t).

Finalize Function

The finalize function specified in the ContentFilter will be called when an instance of the custom content filter is no longer needed. When this function is called, it is safe to free all resources used by this particular instance of the custom content filter.

For example:

void sample_finalize_function ( void* compile_data) {
    /* free parameter string from compile function */
    DDS_String_free((char *)compile_data);
}

The finalize function may use the following optional parameters:

system_key	See Compile Function.
handle	This is the opaque returned by the last call to the compile function.

Writer Attach Function

The writer_attach function specified in the WriterContentFilter will be used to create some state that can be used by the filter to perform writer-side filtering more efficiently. It is entirely up to you, as the implementer of the filter, to decide if the filter requires this state.

The function has the following parameter:

writer_filter_data

A user-specified opaque pointer to some state created on the writer side that will help perform writer-side filtering efficiently.

Writer Detach Function

The writer_detach function specified in the WriterContentFilter will be used to free up any state that was created using the writer_attach function.

The function has the following parameter:

writer_filter_data

A pointer to the state created using the writer_attach function.

Writer Compile Function

The writer_compile function specified in the WriterContentFilter will be used by a DataWriter to compile a filter expression and parameters associated with a DataReader for which the DataWriter is performing filtering. The function will receive as input a DDS_Cookie_t that uniquely identifies the DataReader for which the function was invoked.

The function has the following parameters:

writer_filter_data	A pointer to the state created using the writer_attach function.
prop	A pointer to DDS_ExpressionProperty. This is an output parameter. It allows you to indicate to Connext DDS if a filter expression can be optimized (as described in Filtering on the Writer Side with Custom Filters).
expression	An ASCIIZ string with the filter expression the ContentFilteredTopic was created with. Note that the memory used by the parameter pointer is owned by Connext DDS. If you want to manipulate this string, you must make a copy of it first. Do not free the memory for this string.
parameters	A string sequence of expression parameters used to create the ContentFilteredTopic. The string sequence is equal (but not identical) to the string sequence passed to create_contentfilteredtopic() (see expression_parameters in Creating ContentFilteredTopics). The sequence passed to the compile function is owned by Connext DDS and must not be referred to outside the writer_compile function.
type_code	A pointer to the type code of the related Topic. A type code is a description of the topic members, such as their type (long, octet, etc.), but does not contain any information with respect to the memory layout of the structures. The type code can be used to write filters that can be used with any type. See Using Generated Types without Connext DDS (Standalone) . [Note: If you are using the Java API, this parameter will always be NULL.]
type_class_name	The fully qualified class name of the related Topic.
cookie	A DDS_Cookie_t to uniquely identify the DataReader for which the writer_compile function was called.

Writer Evaluate Function

The writer_evaluate function specified in the WriterContentFilter will be used by a DataWriter to retrieve the list of DataReaders whose filter passed the DDS sample. The writer_evaluate function returns a sequence of cookies which identifies the set of DataReaders whose filter passes the DDS sample.

The function has the following parameters:

writer_filter_data	A pointer to the state created using the writer_attach function.
sample	A pointer to the data to be filtered. Note that the writer_evaluate function always receives deserialized data.
meta_data	A pointer to the meta-data associated with the DDS sample.

Note: Currently the meta_data field only supports related_sample_identity (described in DDS_WriteParams_t).

Writer Return Loan Function

Connext DDS uses the writer_return_loan function specified in the WriterContentFilter to indicate to the filter implementation that it has finished using the sequence of cookies returned by the filter’s writer_evaluate function. Your filter implementation should not free the memory associated with the cookie sequence before the writer_return_loan function is called.

The function has the following parameters:

writer_filter_data	A pointer to the state created using the writer_attach function.
cookies	The sequence of cookies for which the writer_return_loan function was called.

Writer Finalize Function

The writer_finalize function specified in the WriterContentFilter will be called when the DataWriter no longer matches with a DataReader that was created with ContentFilteredTopic. This will allow the filter implementation to delete any state it was maintaining for the DataReader.

The function has the following parameters:

writer_filter_data	A pointer to the state created using the writer_attach function.
cookie	A DDS_Cookie_t to uniquely identify the DataReader for which the writer_finalize was called.