I'm using recording service with RTI DDS Connext 6.1.2 to capture data of type KeyedString on the same topic name ('MOVEMENT_LINE')
The key is a normal string, and the value is an xml content.
Bellow example of data where some messages are used as responses and others as requests:
[
{
"key": "2b44272e-694c-4666-a3f3-0ad125812042",
"value": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<movementLineResponse creationTime=\"2024-04-26T11:46:31.315+02:00\" messageId=\"2b44272e-694c-4666-a3f3-0ad125812042\" xmlns=\"http://www.dummy.com/ABC/mvtResponse/1\">...</movementLineResponse>..."
},
{
"key": "8b30d34f-818c-4e3f-bc15-0621ea51fcd5",
"value": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?><movementLine version=\"6\" creationTime=\"2024-04-26T11:47:08.528+02:00\" messageId=\"8b30d34f-818c-4e3f-bc15-0621ea51fcd5\" xmlns=\"http://www.dummy.com/ABC/mvt/1\">...</movementLine>..."
},
{
"key": "RESPONSE",
"value": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<movementLineResponse creationTime=\"2024-04-26T11:46:31.315+02:00\" messageId=\"2b44272e-694c-4666-a3f3-0ad125812042\" xmlns=\"http://www.dummy.com/ABC/mvtResponse/1\">...</movementLineResponse>..."
},
{
"key": "REQUEST",
"value": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?><movementLine version=\"6\" creationTime=\"2024-04-26T11:47:08.528+02:00\" messageId=\"8b30d34f-818c-4e3f-bc15-0621ea51fcd5\" xmlns=\"http://www.dummy.com/ABC/mvt/1\">...</movementLine>..."
}
]
The meaningful difference between the data is only available in the value so the only way to capture the request is by filtering on the data example the xmlns of the request is http://www.dummy.com/ABC/mvt/1 or the request does not contain movementLineResponse tag."
My issue is I never succeeded to filter on request data. Here after the config filter, I'm using:
I wanted to test it on filtering the key that match 'REQUEST', I tested it with builtin.sql and stringMatch kinds and with LIKE keyword or sign equal (=) and never succeeded:
<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="file:/opt/rti_connext_dds/resource/schema/rti_recording_service.xsd"><types>
<include file="./builtinTypes.xml" />
</types>...
<session name="DefaultSession">
<topic_group name="Recorder_part1" participant_ref="part1">
<allow_topic_name_filter>MOVEMENT_LINE</allow_topic_name_filter>
<deny_topic_name_filter>rti/*</deny_topic_name_filter>
<datareader_qos base_name="QosLibrary::StrictReliable"/>
<content_filter name="MVT_LINE_filter" kind="builtin.stringMatch">
<expression>key = 'REQUEST'</expression>
</content_filter>
</topic_group></session>
</recording_service></dds>
Also, I tested on value field and it never worked for me.
The type definition of keyedString is :
<?xml version="1.0" encoding="UTF-8"?>
<types xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="file:/opt/rti_connext_dds/resource/app/app_support/rtiddsgen/schema/rti_dds_topic_types.xsd">
<module name="DDS">
<struct name="KeyedString">
<member name="key" type="string" key="true" stringMaxLength="1024" />
<member name="value" type="string" stringMaxLength="2147483647" />
</struct>
</module>
</types>
For filtering I used the following syntax:
<content_filter name="MVT_LINE_filter" kind="builtin.stringMatch">
<expression>key = %0</expression>
<expression_parameters>
<element>REQUEST</element>
</expression_parameters>
</content_filter>
I don't know if my config is correct or not because on all circumstances the application starts without issue and the problem is that I can't see in rtiadminconsole the reader of the recording service so I can investigate further the content filter.
Can you provide any tips or help on this topic?
So, you're running to a number of issues including what I think may be a bug in RTI Recording Service.
1) there may be (to be confirmed) a bug preventing the use of content-filters with <topic_group>. Instead, you will need to use <topic> and apply content-filters on a per-topic basis
So, this doesn't work (even though it's supposed to)
but this DOES work
Hello Howard,
Your recomendation helped me and I can make it work and filter my data.
The problem I have right know is when the field contains the special character '/', the filtering never work.
Example of data:
key: "/t REQUEST /n" or " REQUEST /"
The filter expression I used is: <expression>key MATCH '*REQUEST*' </expression>.
I tested it with stringMatch and sql filter kind and it gives the same results.
So when the field contains this special caracter, the filter always evalued to false and the data was rejected.
Hi Saber,
Although we have created a dedicated support case for this with you, I track this here too for completness.
The behavior with the '/' is due to the FNM_PATHNAME flag we use for fnmatch. With it, the '/' won't match the '*' wildcard expression.
That means that for your filter expression you need to know the number of '/' that are before and after the substring you want to match. For example for your keys: "/t REQUEST /n" or " REQUEST /" there is not a single filter expression that match both. You will require two: key MATCH '*/*REQUEST*/*' or key MATCH '*REQUEST*/*'
I have created a request for improvement for not using the FNM_PATHNAME flag in Content Filters. Keep the ID for future references: CORE-14725.