Filter MATCH <string pattern> syntax

5 posts / 0 new
Last post
Offline
Last seen: 2 years 9 months ago
Joined: 01/27/2014
Posts: 22
Filter MATCH <string pattern> syntax

Hello,

I have two questions.

1 - What is the exact syntax of the <string pattern> used in MATCH filter expressions used in multichannel datawriters ? Though there is examples, it is not very clear in the documentations.

2 - Is it possible to use custom content filter for multichannel datawriters filtering configuration ?

 

Thank you.

Boris.

Organization:
Howard's picture
Offline
Last seen: 19 hours 36 min ago
Joined: 11/29/2012
Posts: 618
Offline
Last seen: 2 years 9 months ago
Joined: 01/27/2014
Posts: 22

Thanks for your answers.

I did a lot of testing with multichannel datawriters, CFT and custom CFT and I got two other questions...

Is it allowed to set a custom CFT on the reader side, with multichannel datawriters using a STRINGMATCH filter (I assumed yes, but I have some doubts) ?

If yes, is the custom CFT evaluated on the writer side (it seems so, based on my observations) and why (I thought multichannel writer only evaluate its multichannel filter) ?

I ask this last question because I got a DDS error log with this setup:

REDAInlineList_assertNodeToBackEA:!precondition: "(node->inlineList != ((void *)0) && node->inlineList != list)"

Because of this error, I finally fell back on using a STRINGMATCH filter on the reader side in place of my custom CFT.

Hope my questions are worthwhile !

Boris.

 

Howard's picture
Offline
Last seen: 19 hours 36 min ago
Joined: 11/29/2012
Posts: 618

So it would be interesting to know your use case.  For what reason do you think that the Multichannel DataWriter feature is useful in your system...?

Fundamentally, the Multichannel DataWriter feature is designed to allow users to make tradeoffs in a system where there are many subscribers to a topic that all want a different subset of the data based on data value.  However, any single piece of data may need to go to multiple subscribers.  Just that there are different "sets" of subscribers (datareaders) who want any particular value of data.

So, on one extreme, using unicast, the each DataReader could use a content-filter (executed by the DataWriter) and only receive the subset of data that it wants.  How this would be executed is when the user apps sends a data sample, the DataWriter would test the sample value with the content-filter associated with each DataReader.  For any filter that it passes, it will send a copy of the data directly to that DataReader.  This has 2 drawbacks,

a) the publishing host is loaded with executing N filters for N DataReaders for each data sample being sent

b) the network may need to transfer M copies of data assuming that M out of N DataReaders want a particular value of data

At the other extreme is to configure all of the DataReaders to receive the data on the same multicast address.  Then either use a content-filter (now applied by DDS in each subscribing app) or application-level logic to disgard unwanted data values.  This offers 2 advantages...the publishing host isn't loaded with any filter execution and data samples are only sent once on the network.  But at the disadvantage of ALL of the data samples being sent will be received by every subscribing host and having the filter logic applied there.

Now, if there is a pattern to the subset of values that different applications want to subscribe to data...or most applications only subscribe to a small subset of data values, then the Multichannel DataWriter feature offers something in between the two extremes.

You setup multiple channels, with each channel associated with a specific multicast address and carrying only a subset of data as passed by the content-filter associated with the channel.  So, for example, configure 26 channels, each channel will carry the data that has the "name" field that begins with a different letter of the alphabet, "A*", "B*", etc. 

Now, for a DataReader, you set a content-filter that describes the subset data values that it wants to receive.  So, if the reader only wants "Apple" and "Orange", it needs to receive data on 2 multicast addresses.  The subscribing host will receive all data beginning with "A" and all data beginning with "O", but that's hopefully much less than getting all of the data A-Z.  And DDS will apply the DataReader's content-filter locally so that only values of "Apple" and "Orange" makes it to program logic.

For this to work...and actually provide the performance advantages offered by the feature, DDS needs to figure out which DataWriter channels (multicast addresses) that a DataReader needs to subscribe to.

Fundamentally, it needs to calculate the intersection of 2 filters.  For each channel, DDS needs to know if the content-filter of the channel has a non-empty intersection with the content-filter of the DataReader.   Which is kind of impossible to do (at least not without testing every possible data value) for custom-content-filters that may be doing mathematical operations (or anything that can be put into code) in the filter logic. 

So, fundamentally, the only filters supported by Multichannel DataWriters are SQL and STRINGMATCH....AND the filter used by the DataWriter and DataReader have to be the SAME, either both SQL or both STRINGMATCH...AND the filter expressions used by a Multichannel DataWriter and by a subscribing DataReader MUST have common filter fields (structure members that are being used in the filter), or else DDS won't be able to determine which channels should be used by the DataReader...and will configure the DataReader in that case to listen on all channels...which will also happen if the DataReader uses a custom-content-filter.

Which is the long reason why using custom-content-filters to create DataReaders for Multichannel DataWriters don't make sense....even if the middleware doesn't complain about it (which it should...but it actually can't do it when you create the DataReader because DDS doesn't know that a content-filtered DataReader is being matched with a Multichannel DataWriter), the result is that all of the data is going to be sent to that DataReader, which you probably didn't want to happen.

Offline
Last seen: 2 years 9 months ago
Joined: 01/27/2014
Posts: 22

Thanks a lot for this detailed explanation, whith deep technical justifications. I like that !