6. ContentFilteredTopics¶
Prerequisites |
|
Time to complete |
45 minutes |
Concepts covered in this module |
Content Filtering |
In the Publish/Subscribe communication pattern, DataWriters send data to DataReaders with the same Topic. But sometimes a DataReader may be interested in only a subset of the data that is being sent. Some examples are:
A DataReader only cares about the temperature when it goes outside of a certain bound.
A DataReader only cares about log messages with levels WARNING or ERROR.
A DataReader only cares about the ChocolateLotState if the DataReader belongs to the next station that processes the chocolate lot.
Note that in all of these examples, there might be some DataReaders that want to receive a subset of data, and other DataReaders that might want to receive all data. For example, in our chocolate factory, an individual station only cares about a chocolate lot if it is the next station to process the lot, but the monitoring application wants to receive every update about every chocolate lot.
Connext DDS offers a mechanism to allow DataReaders to filter data so that they only receive the data they are interested in. Applications can do this by creating ContentFilteredTopics instead of normal Topics. Since different DataReaders may want to receive a different subset of the data, ContentFilteredTopics are specified on the DataReaders. No changes are required on DataWriters to use content-filters: they continue to use a normal Topic, and they can communicate both with DataReaders that use a normal Topic and DataReaders that use a ContentFilteredTopic.
Although ContentFilteredTopics are specified on DataReaders, in most cases the DataWriter does the filtering, in which case no network bandwidth is used for those filtered samples. (The DataWriter finds out during discovery that a DataReader has a ContentFilteredTopic.)
Definition
A ContentFilteredTopic is a Topic with filtering properties. It makes it possible to subscribe to Topics and at the same time specify that you are only interested in a subset of the Topic’s data.
A ContentFilteredTopic consists of a Topic, a filter expression, and optional parameters. The filter expression and parameters can be changed at runtime. You can think of the filter expression as being like the “where” clause in SQL.
![Content Filter](_images/cft.png)
Figure 6.1 A ContentFilteredTopic that only allows samples with degrees > 32.¶
6.1. The Complete Chocolate Factory¶
ContentFilteredTopics are the final piece we need to put together the last part of the Chocolate Factory: the ingredient stations.
If you recall, the process the chocolate lot takes through the chocolate factory looks like this:
![Ingredient Stations Using Content Filtering](_images/stations.png)
Figure 6.2 DataReaders in our system will use content filtering to filter out any ChocolateLotState that doesn’t have “next_station” equal to that DataReader’s station.¶
The Monitoring/Control Application starts each lot by writing a sample
to the ChocolateLotState Topic, with next_station
set to one of the
ingredient stations. Each ingredient station processes the lot by
adding an ingredient, and then updates the next_station
. The last
station is the tempering machine, which processes the lot by tempering
it, and then calls dispose()
on the lot instance to say that the lot has
completed.
![Next Station](_images/cocoa_station.png)
Figure 6.3 A chocolate lot is waiting, processing, or completed by an
ingredient station. When a lot is completed, the next_station
field
indicates the next station to process the chocolate lot.¶
![Last Station](_images/temper_station.png)
Figure 6.4 A chocolate lot is waiting, processing, or disposed by the tempering application.¶
The Monitoring/Control Application wants to monitor every state of the
chocolate lots as they are processed by the stations. However, an
individual station only wants to know about a lot if the next_station
field indicates it is the station that will process the lot next.
In the previous exercises, we had code in the ProcessLot
function in
TemperingApplication.cs
to check whether the tempering station was the
next station:
private void ProcessLot(
DataReader<ChocolateLotState> lotStateReader,
DataWriter<ChocolateLotState> lotStateWriter)
{
using var samples = lotStateReader.Take();
foreach (var sample in samples.ValidData())
{
if (sample.next_station == StationKind.TEMPERING_CONTROLLER)
{
...
}
}
}
Instead of doing that, we can use a ContentFilteredTopic to filter out
all the lot updates where the tempering controller is not the
next_station
. This cleans up the logic when the tempering application
receives data, because it no longer needs to check whether a chocolate
lot update is intended for it. This can also save network bandwidth because
DataWriters perform the filtering. When the application uses a ContentFilteredTopic,
Connext DDS can filter out the data more efficiently than the application, and
may not even send updates to an application that is not interested.
6.2. Hands-On 1: Update the ChocolateLotState DataReader with a ContentFilteredTopic¶
We are going to update the DataReader in the Tempering Application to use a
ContentFilteredTopic instead of checking whether the data is important
to this application every time a DataWriter updates the ChocolateLotState.
Remember that the Tempering Application only cares about a lot if the
next_station
field indicates that the tempering machine is the next
station.
Find the examples as described in Clone Repository.
Generate the C# code for
chocolate_factory.idl
as explained in Compile the Applications
Create a ContentFilteredTopic that filters out ChocolateLotState data unless the
next_station
field in the data refers to this application.Open
TemperingApplication.cs
and look for the comment:// Exercise #1.1: Create a Content-Filtered Topic that filters out // chocolate lot state unless the next_station = TEMPERING_CONTROLLER
Add the following code after the comment, so it looks like the following:
// Exercise #1.1: Create a Content-Filtered Topic that filters out // chocolate lot state unless the next_station = TEMPERING_CONTROLLER ContentFilteredTopic<ChocolateLotState> filteredLotStateTopic = participant.CreateContentFilteredTopic( name: "FilteredLot", relatedTopic: lotStateTopic, filter: new Filter( expression: "next_station = %0", parameters: new string[] { "'TEMPERING_CONTROLLER'" }));
This code creates a ContentFilteredTopic, using the ChocolateLotState Topic, that will filter out all chocolate lot state data that the Tempering Application does not care about.
Use the ContentFilteredTopic instead of a plain Topic.
In
TemperingApplication.cs
, look for the comment:// Exercise #1.2: Change the DataReader's Topic to use a // Content-Filtered Topic
Replace these lines:
DataReader<ChocolateLotState> lotStateReader = subscriber.CreateDataReader( lotStateTopic, ...);
So the code looks like this (the highlighted line has changed):
// Exercise #1.2: Change the DataReader's Topic to use a // Content-Filtered Topic DataReader<ChocolateLotState> lotStateReader = subscriber.CreateDataReader( filteredLotStateTopic, ...);
Now the DataReader is using the ContentFilteredTopic. You don’t need to make any changes on the DataWriter side.
Remove the code from the
ProcessLot
that checks thatnext_station
is the Tempering Application.In
TemperingApplication.cs
, look for the comment:// Exercise #1.3: Remove the check that the Tempering Application is // the next_station. This will now be filtered automatically.
Delete the following
if
condition:if (sample.next_station == StationKind.TEMPERING_CONTROLLER)
So the code looks like this:
foreach (var sample in samples.ValidData()) { // Exercise #1.3: Remove the check that the Tempering Application is // the next_station. This will now be filtered automatically. Console.WriteLine("Processing lot " + sample.lot_id); ... Console.WriteLine("Lot completed"); }
Now Connext DDS automatically filters out ChocolateLotState data before your application processes it.
6.3. Hands-On 2: Review the Temperature DataReader’s ContentFilteredTopic¶
We have already made a similar change as you made in Hands-On 1 to the Monitoring/Control Application’s Temperature DataReader: we added content filtering to it. That application now filters out data unless the temperature is outside of the expected range. (And it prints an error when the temperature is above or below that range.)
In
MonitoringCtrlApplication.cs
, review the following code:// A Topic has a name and a datatype. Create a Topic with type // ChocolateLotState. Topic name is a constant defined in the IDL file. Topic<ChocolateLotState> lotStateTopic = participant.CreateTopic<ChocolateLotState>("ChocolateLotState"); // Add a Topic for Temperature to this application Topic<Temperature> temperatureTopic = participant.CreateTopic<Temperature>("ChocolateTemperature"); ContentFilteredTopic<Temperature> filteredTemperatureTopic = participant.CreateContentFilteredTopic( name: "FilteredTemperature", relatedTopic: temperatureTopic, filter: new Filter( expression: "degrees > %0 or degrees < %1", parameters: new string[] { "32", "30" }));
Notice that we have added a ContentFilteredTopic that will filter out temperature data unless it’s outside of the expected range. This uses a slightly more complex filter expression than the one you added to the Tempering Application in Hands-On 1: Update the ChocolateLotState DataReader with a ContentFilteredTopic. More information about the possible filter expressions can be found in SQL Filter Expression Notation, in the RTI Connext DDS Core Libraries User’s Manual.
You can see later in the code that the Temperature DataReader has been updated to use the ContentFilteredTopic:
DataReader<Temperature> temperatureReader = subscriber.CreateDataReader(filteredTemperatureTopic, readerQos);
The logic in the
MonitorTemperature
function no longer checks whether the temperature is out of range, because temperature data is only received when it’s out of range. Now the function looks like this:foreach (var data in samples.ValidData()) { // Receive updates from tempering station about chocolate temperature. // Only an error if below 30 or over 32 degrees Fahrenheit. Console.WriteLine("Temperature high: " + data); }
Build the applications.
$ dotnet build
Use the script to start all the applications at once:
$ ./start_all.sh
$ ./start_all.sh
> start_all.bat
The script runs four copies of the Ingredient Application, each one specifying a different type of ingredient it is providing. The script also runs the Tempering Application and the Monitoring/Control Application. You can watch each lot get processed by the ingredient applications, and then the tempering machine.
If you do not want to run the script, you can start up each of the applications by creating a new terminal for each application, and starting each application in a different shell as follows:
$ dotnet run -p IngredientApplication -- --station-kind COCOA_BUTTER_CONTROLLER $ dotnet run -p IngredientApplication -- --station-kind SUGAR_CONTROLLER $ dotnet run -p IngredientApplication -- --station-kind MILK_CONTROLLER $ dotnet run -p IngredientApplication -- --station-kind VANILLA_CONTROLLER $ dotnet run -p TemperingApplication $ dotnet run -p MonitoringCtrlApplication
6.4. Hands-On 3: Review the Larger System in Admin Console¶
We now have six applications that are publishing and subscribing to data, which is not a very large distributed system. This can start to look overwhelming, so we’re going to open Admin Console to get an overview of this system. This hands-on exercise does not relate directly to content filtering, but shows how multiple applications start to look in Admin Console.
Make sure all of your applications are still running.
Open Admin Console.
Open Admin Console from RTI Launcher.
Choose the Administration view:
(The Administration view might be selected for you already.)
In the DDS Logical View, click on the Domain in your system:
Explore the chocolate factory system. Even though it looks complex with multiple applications running, you can navigate by clicking on the Topic name or the application. This makes it much easier to visualize the larger system.
Figure 6.5 Each Ingredient Application shows one DataReader and one DataWriter communicating on the ChocolateLotState Topic¶