Keep track of 'subscribed instances' in publishers

2 posts / 0 new
Last post
Offline
Last seen: 6 years 12 months ago
Joined: 10/21/2013
Posts: 1
Keep track of 'subscribed instances' in publishers

Hello,

I'm evaluating DDS for a distributed system with 100 publishers, each of one publish around 10000 variables.

The critical point is that the calculation of that variables in the publisher it's a very resource intensive process and on average only 2% of the variables are required by subscribers at a given time.

Right now in my system (no DSS) the subscribers notify the publishers which variables they are going to need sending subscribe/unsubscribe messages with the variable ids so the publishers can keep track of which variables needs to keep calculating.

I can keep this approach whith DDS usign some kind of 'subscribe' 'unsubscribe' topics but I wonder if I can achive this with the built in features of DDS.

With my little knowledge of DDS in 'theory' I can do it using one topic per variable but it's not seems a practical way

Is there any way two keep the publisher infomed about the Instances (Topic+Key) requested by the subscribers?

Thanks,

Jose Luis

 

Gerardo Pardo's picture
Offline
Last seen: 1 month 2 weeks ago
Joined: 06/02/2010
Posts: 594

Hi,

One Topic per variable is definitely not practical bacause it requires too many DataWriters and DataReaders and it would not scale well.  It would be much better to use a single Topic  (assuming all variables have the same associated data-type) and map each variable to a different Topic-Key.  

Mapping each Variable to a Topic-Key would result on each Variable being published as a separate message (DDS Sample). The other option would be to define the Topic data-type as a sequence of Variables and be able to combine multiple Variables into a single DDS Sample.

One Variable per DDS Sample seems like a cleaner approach, assuming you can afford the extra overhead of writing all these separate Samples (2% of 10000 is about 200) rather combining all into a single Sample , containing the 200 variables inside... Of course combining multiple variables into a Sample which would result on subscribers getting variables they do not need...

In the rest of this post I assume one Variable per DDS Sample.

One way for a subscriber to communicate to the publisher the list of Variable IDs (Topic Keys) it is interested in is to piggiback that information to the DDS Discovery data. DDS already sends data about each DataReader to the DataWriter using the DDS Builtin Topics (see also the Connext DDS User's Manual, section 16.2 Built-in DataReaders). This data is accessible to the publisher reading the SubscriptionsBuiltinTopicData.   This Topic contains the QoS of the DataReader and well as extra information the user can put there, such as the USER_DATA Qos Policy, or the Property Qos Policy.   Basically all you need to do is modify the DataReader Qos to set the USER_DATA or the PROPERTY Qos policies to contain the list or variables (keys) you are interested in and that information will automatically be sent to the publisher, which will receive it in the SubscriptionsBuiltinTopicDataDataReader

I am thinking that there is even a better way than sending the list of keys in the USERT_DATA or the Property Qos.  You could send it as a ContentFilteredTopic which has the additinal benefit of having DDS use it to deliver the Variables only to the intended recipients.

A cool way to do this would be to use the STRINGMATCH Filter.  To do this you would define the Key attribute in your variable that identifies the variable as a string. Let's assume your Variable data-type is:

struct Variable {
    string<8> variable_id;
    float value;
};

Then create a STRINGMATCH ComtentFilteredTopic with the call: DDSDomainParticipant::create_contentfilteredtopic_with_filter specifying the constant DDS_STRINGMATCHFILTER_NAME as the name of the filter and the expression "variable_id MATCHES %0" as the filter_expression.

You can manipulate the filter expression paramaters adding and removing variables of interest calling the operations:

Then you can use the operations ContentFilteredTopic::append_from_expression_parameter and ContentFilteredTopic::remove_from_expression_parameter. As you do this, the modified filter is propagated by DDS Discovery and the publisher can access it by reading the content_filter_topic attribute in the SubscriptionsBuiltinTopicData. Once the publisher sees it, it can parse the expression_parameters and learn what Variables are of interest to that DataReader to determine with Variables to calculate and publish.

As a sibe benefit the DataReader is using that same expression in its ContentFilter to define the data-samples it wants to receive so it will only receive the variables it has expressed interest on.

Disclaimer: I have not tried this approach so I am not sure if I have missed something that could make it not advisable of it impractical... But it would seem like a natiral way to leverage the features already in DDS.

Gerardo