I am following the example "vehicle tracking" provided by RTI, in particular note that I am using an evaluation license for RTI 7.3.0. The scenario I am creating is the following:
- A Docker container inside a bridge subnetwork A is going to run the "RadarGenerator" executable with flag "--no-multicast".
- A Docker container inside that same bridge subnetwork A is going to run the "TrackGui" executable with flag "--no-multicast".
Of course, both entities are attached to the same domain. I wanted to implement the TRANSPORT_PRIORITY QoS by setting the priority for the messages written by "RadarGenerator" on the topic "AirTrack" to "100". To do so I read the documentation and asked the RTI Connext Chatbot for guidance, based on its suggestions I did the following:
- I added the following to the
MulticastNotAvailabletag in the filebase_profile_no_multicast.xmlof on both containers.
<transport_builtin>
<udpv4>
<transport_priority_mask>0xFF</transport_priority_mask>
</udpv4>
</transport_builtin>
<!-- Only in the XML file of the container running RadarGenerator -->
<datawriter_qos>
<transport_priority>
<value>100</value>
</transport_priority>
</datawriter_qos>
<!-- Only in the XML file of the container running TrackGui -->
<datareader_qos>
<transport_priority>
<value>100</value>
</transport_priority>
</datareader_qos>
2. I also modified the radar_profiles_no_multicast.xml to include the value of transport priority set to 100 for both the "kinds" of datawriters.
From my understanding this should be enough to set the TRANSPORT_PRIORITY QoS, I want to obtain a confirmation of the QoS policy actually being in place but when I open the admin console and select either the datareader or the datawriter associated with the AirTrack topic I do not see "transport_priority" or anything similar listed among the QoS in the right-side panel titled "DDS QoS". However, I also do not get any error, how can I have the certainty that the QoS is properly in place?
My questions are:
- Is there something I am missing in the configuration of the policy? If yes, what else should be done? If no, could it be that Docker's bridge interferes with the UDPv4 communication among the DDS entities running in separate containers?
- If I wanted to extend the policy to a scenario in which a node running the routing service is present, do I need to do something more "specific" or can I just set the appropriate tags in the RTI_ROUTING_SERVICE.xml file to regulate the QoS of the datawriters and datareaders created by the routing service entity?
Admin Console only shows the QoS values that are exchanged during discovery. Transport Priority is NOT a QoS whose value is sent at discovery.
Fundamentally, the only way to confirm that the Transport Priority QoS Policy has had any effect is to use wireshark or tcpdump on the interface in which the data is being sent, and check the DSCP bits in the IP header (using Wireshark) to see if they have been changed to the expected value.
For RoutingService, yes, you can configure the QoS of the DataWriters and DataReaders created by RoutingService for the routes that it creates in the configuration file of RoutingService (per the documentation).
Thank you, that was very clear and I verified as expected that the DSCP bits were set using Wireshark, meaning that I had implemented correctly the QoS.
I am computing some performance statistics for the messages exchanged on the topic AirTrack using Admin Console's Sample Log and a Python script to compute the latency and jitter based on the source and received timestamps.
Am I expected to notice any difference in terms of performance when I compare a scenario in which RadarGenerator uses the priority set to 0x8B (Expedited Forwarding) to the one where RadarGenerator uses the default 0x00?
From the data collected so far I did not notice any consitent trend, besides the latency being a bit more than 50% of the times slightly reduced (by 0.05 ms or so). I am just spawning a TrackGui and a RadarGenerator and then I subscribe to the AirTrack topic to generate the Sample Log.
I also set up RoutingService to have twosessions:
By expanding the view of the RoutingService process in the Admin Console I was able of seeing the "Latency" computed automatically for each route, and I noticed that the route from domain 0 to domain 1 consistently had a latency slightly higher than the route from domain 1 to domain 0, which uses the priority expedited forwarding. The difference is about 0.01 ms or 0.02 ms, but it is consistent. Is there a chance that this is an effect of the priority being in place (nothing else changed among the two route), or is it most likely a meaningless difference?
So, what the network does to "treat"...route...packets with different DSCP values differently is outside the control of Connext DDS.
Generally, you actually have to enable switches/routers/any network device that may be in the network between the sending and receiving machines through which packets must past to actually pay attention to and treat packets with different DSCP values differently.
I think (am not an expert on this) that most network elements, out-of-the-box, aren't enabled to
I asked chatgpt "are most network switches and routers enabled by default to look at DSCP bits and route higher priority packets differently, or do you have to change a default configuration to do so?"
It responded:
Also, if you're testing between 2 processes on the same host...I doubt you're doing to see much difference...even if you're able to somehow enable any sort of virtual switch/router to act on DSCP.
DSCP is targeted to help a real network differentiate between the priorities of different data streams passing through the network, usually data coming from different hosts.
Even in that type of system, unless there is a decision to make...should I forward this packet first or that packet...which implies both packets have been received and buffered and both packets need to go out the same port...which doesn't happen much when the data is coming from a single process....that host is connected to a switch with a single ethernet line (or wifi) and packets are already sent one after the other...the switch will forward the packets from that host in order received since it's usually fast enough to send the packets as they come.
The DSCP makes a difference when a switch (network element) receiving data on multiple port simultaneously and actually has an instance where there are multiple packets availiable at the same time that must be routed...and now it can use the DSCP bits to decide which one to route first.
Thank you for your very detailed response! Now I understand perfectly the nature of this QoS.
I wanted to implement a priority mechanism among data writers in an alternative way, which led me to consider High-Priority First (HPF) Flow Controllers. I managed to implement the HPF FC correctly in a scenario where only one domain is present (always using the vehicle tracking example as a starting point). I am now trying to implement it inside my container running the routing service, which means that I must modify the participant QoS related the routing service I am interested in and the datawriter QoS within the RTI_ROUTING_SERVICE.xml file. I have used the RTI chatbot to guide me and validate my configuration, which seems to get validated by it, but when I actually run the routing service an error is generated.
In particular, I added the following content to the <participant_qos> tag that I define within the <administration> tag of the <routing_service name="defaultBothWays">:
<element>
<name>dds.flow_controller.token_bucket.priority_flow.scheduling_policy</name>
<value>DDS_HPF_FLOW_CONTROLLER_SCHED_POLICY</value>
</element>
<element>
<name>dds.flow_controller.token_bucket.priority_flow.token_bucket.max_tokens</name>
<value>128</value>
</element>
<element>
<name>dds.flow_controller.token_bucket.priority_flow.token_bucket.tokens_added_per_period</name>
<value>128</value>
</element>
<element>
<name>dds.flow_controller.token_bucket.priority_flow.token_bucket.bytes_per_token</name>
<value>8192</value>
</element>
<element>
<name>dds.flow_controller.token_bucket.priority_flow.token_bucket.period.sec</name>
<value>0</value>
</element>
<element>
<name>dds.flow_controller.token_bucket.priority_flow.token_bucket.period.nanosec</name>
<value>10000000</value>
</element>
Adding this does not cause any error when running the routing service. Then I updated my datawriter qos for the Session 2 going from domain 0 to domain 1 to include the following:
<datawriter_qos>
<durability>
<kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind>
</durability>
<!-- [!!!] The next tag is the problematic one -->
<publish_mode>
<kind>ASYNCHRONOUS_PUBLISH_MODE_QOS</kind>
<flow_controller_name>dds.flow_controller.token_bucket.priority_flow</flow_controller_name>
<priority>20</priority>
</publish_mode>
<reliability>
<kind>RELIABLE_RELIABILITY_QOS</kind>
<max_blocking_time>
<sec>5</sec>
<nanosec>0</nanosec>
</max_blocking_time>
</reliability>
<history>
<kind>KEEP_ALL_HISTORY_QOS</kind>
</history>
<resource_limits>
<initial_samples>1</initial_samples>
</resource_limits>
<protocol>
<rtps_reliable_writer>
<max_send_window_size>10</max_send_window_size>
<min_send_window_size>10</min_send_window_size>
<heartbeats_per_max_samples>10</heartbeats_per_max_samples>
<heartbeat_period>
<sec>0</sec>
<nanosec>200000000</nanosec>
</heartbeat_period>
<fast_heartbeat_period>
<sec>0</sec>
<nanosec>20000000</nanosec>
</fast_heartbeat_period>
<late_joiner_heartbeat_period>
<sec>0</sec>
<nanosec>20000000</nanosec>
</late_joiner_heartbeat_period>
<max_nack_response_delay>
<sec>0</sec>
<nanosec>0</nanosec>
</max_nack_response_delay>
<high_watermark>25</high_watermark>
<low_watermark>10</low_watermark>
<max_heartbeat_retries>500</max_heartbeat_retries>
</rtps_reliable_writer>
</protocol>
<property>
<value>
<element>
<name>dds.data_writer.history.memory_manager.fast_pool.pool_buffer_max_size</name>
<value>32768</value>
</element>
</value>
</property>
</datawriter_qos>
The problematic tag is in particular the <publish_mode> one, my whole configuration does not raise any error if this tag is not present, but if it is it generates a series of errors in the logs, which seem to stem from the fact that the name of the flow controller seems to not be recognized, even if the name used in the publish mode tag "dds.flow_controller.token_bucket.priority_flow" is the same name used to "declare" the flow controller in the participant qos.
I attach to this post the following files:
I wonder what I am missing and why my configuration is not accepted, any guidance on this issue would be greatly appreciated. I have also used tools like RTI Connext Chatbot and Claude Sonnet 3.7 but I was unable to solve this issue. As an initial reference for my configuration I used the following example: https://github.com/rticommunity/rticonnextdds-examples/blob/master/examples/connext_dds/high_priority_first_flow_controller/c%2B%2B98/USER_QOS_PROFILES.xml.
My idea is using the HPF Flow Controller to assign a certain priority value to the datawriters from domain 1, so modifying the datawriter qos in session 2,which should be higher than the priority assiged to datawriters from domain 0.
Well, a few things.
1) you define a QoS Profile that defines the flow controller, but you don't use it. Instead, the flow controller definition is redefined (copy/paste) into the <participant_qos> of the <administration> participant created by the "defaultBothWays" Routing Service configuration.
I suggest that if you define a QoS Profile, that using it instead of copy/pasting it, is a better practice.
2) you are defining the flow controller for participant used by the <administration> participant...this is not the participant used by your routes.
Your routes use the participant defined by:
<domain_route name="TwoWayDomainRoute">
<!-- Defines which domains should be joined by the RS -->
<participant name="1">
<domain_id>0</domain_id>
</participant>
<participant name="2">
<domain_id>1</domain_id>
</participant>
And those participants are the ones that are used to create the DataWriters/DataReaders of the route and thus should be configured to define the flow controller...