(This article expands on the community post: How do I restrict RTI Connext to use only a subset of interfaces?)

On a machine with multiple NICs, users can restrict RTI Connext® traffic to a subset of the available network interfaces (NICs). RTI’s Connext DDS 7.3 introduced the capability to prioritize the subset of interfaces. Connext will use the highest-ranked interface for incoming data transmissions. If that interface becomes unavailable, Connext will dynamically switch to the next-highest ranked interface.

Background

The link prioritization feature is an extension to the Quality of Service (QoS) policy allow_interfaces_list. Defining the allow_interfaces_list has multiple implications:

    1. Acts as a ‘whitelist’ that specifies which interfaces can be used to receive traffic.
    2. Specifies the set of interfaces that will be used for outgoing multicast traffic.
    3. Specifies the set of interfaces to be included as locators in discovery packets (data(p)).

There is a similar QoS policy, which is deny_interfaces_list. If both lists are configured, the allow list is applied before the deny list. Therefore, if an interface is defined in both lists by mistake, the interface will be excluded.

Additionally, it is important to note that defining the allow_interfaces_list does not impact outgoing unicast traffic. The interface used for outgoing unicast traffic is determined by the operating system routing table.

Locators are sent during discovery to remote Domain Participants to indicate how to contact the local Domain Participant (for example - IP address, shared memory address, etc.). When using the UDP transport, a locator is generated per interface/IP address. If either the allow/deny interface list is configured, the generated locators are restricted to those in the list.

It’s also important to mention the initial peers list. The initial peers list defines the locators to communicate with. Therefore, the initial peers list should be taken into consideration when configuring the allow_interfaces_list to ensure local and remote Domain Participants are able to communicate.

How Link Priority Works

Interface ranking – The interfaces defined in the allow_interfaces_list are given a rank. The rank is determined by the order in the list. The first interface is given the highest rank, the second interface is given the next-highest rank, and so on.

Dynamic switching – If the highest ranked interface becomes unavailable, Connext will automatically switch to the next-highest ranked interface. For example, if the interface ranked 1 becomes unavailable, Connext will switch to interface ranked 2. If an interface ranked 1 becomes available again later, Connext will switch back.

Multiple interfaces with the same rank – Multiple interfaces can have the same rank if a wildcard is used in the allow_interfaces_list (see Figure 1). If this situation occurs, the priority is determined by the operating system. When the operating system is queried for the interfaces, the order in which the interfaces are returned will dictate the interface rankings. This determination can vary depending on the operating system being used.

Benefits of Link Priority

In a system with redundant interfaces, link priority ensures high availability in case of the primary link failing. In real-time systems, link priority is crucial to help avoid data loss.

Link priority enables optimized network usage in multiple ways:

    1. Reduction of redundant network traffic. The discovery process is automatic and happens behind the scenes. The process requires constant communication to maintain existing connections and initiate new connections. This process can be chatty. Therefore, leveraging link priority can reduce redundant network traffic by only communicating on specified interfaces. Link prioritization is an easy mechanism for performing basic tuning of the discovery process. The result is removing potential latencies introduced by redundant traffic.

    2. Increased data reliability. By configuring link priority, Connext will use the most reliable interface for communication. As a result, Connext can help users manage reliability of communications in situations where bandwidth is constrained or variable.

Use Cases

In a system with redundant paths, typically used for high availability, link priority can help reduce:

    1. Redundant traffic. Avoids sending duplicate traffic by reducing the number of interfaces for Connext to use simultaneously.
    2. Cost. A deployed system could rely on both cellular and satellite communication. This helps reduce costs by prioritizing cellular communication, when available.

How to enable link priority

Link priority can be enabled either in Quality of Service XML configuration or in code. A new QoS parameter was introduced in Connext 7.3 called max_interface_count. Link prioritization is not enabled by default. In order to enable link prioritization, the max_interface_count parameter must be set, and the value must be less than the total number of interfaces on the system. 

<participant_qos>
  <transport_builtin>
    <udpv4>
      <max_interface_count>1</max_interface_count>
      <allow_interfaces_list>
        <element>10.1.15.1</element>
        <element>10.1.15.2</element>
        <element>10.2.15.*</element>
      </allow_interfaces_list>
    </udpv4>
  </transport_builtin>
</participant_qos>

Figure 1 - XML Configuration Example

Figure 1 highlights enabling link priority to use only one interface by configuring max_interface_count to one. So only one interface will be active and that is the first one listed - 10.1.15.1. If the 10.1.15.1 interface becomes unavailable, the system will dynamically switch to the 10.1.15.2 interface. If both the 10.1.15.1 and 10.1.15.2 interfaces become unavailable, the order in which the system provides the interfaces will determine which 10.2.15.* interface is selected. In this example, IP addresses are used in the allow_interfaces_list. Interface names can also be used. Using interface names can be helpful if the interface does not have a static IP address.

Figure 2 - C++11 Example

Figure 2 highlights enabling link priority to leverage two interfaces simultaneously, specifically the interfaces with addresses 192.168.1.1 and 192.168.1.2. It’s important to note that the assumption in this example is that there are more than two interfaces on the system. Otherwise, setting max_interface_count to 2 would not enable link priority functionality.

Example use cases with configuration 

Figure 3 - Wired/Wireless Interfaces Use Case

Figure 3 shows a system with wired and wireless interfaces. When the computer is connected to a docking station, the wired interface is prioritized. Once the computer is disconnected from the docking station and no longer has access to the wired connection, Connext will dynamically switch to wireless. The link priority configuration for this use case would be configured on the computer with both wired and wireless interfaces available.

Figure 4 - Example Configuration for Computer with Wired/Wireless Interfaces

Figure 5 - High Availability Use Case

Figure 5 shows a system with two interfaces connected to different routers to ensure high availability in case of failure. In this scenario, link prioritization is configured on each system. If the higher priority router becomes unavailable, it will be unavailable for both systems. Therefore, the switch to the secondary interface will occur simultaneously for both systems. With this configuration, a secondary benefit is the reduction in redundant traffic. This reduction occurs because both systems are limiting the available interfaces to be in use at any one time.

Figure 6 - Example Configuration for High Availability

Figure 7 - Redundant Traffic Reduction / Redundant Path Use Case

Figure 7 shows a system with two interfaces connected to the same router. In this scenario, without link prioritization, data transmission would occur over both interfaces, therefore producing redundant network traffic. Like the high availability use case (Figure 6), link prioritization configuration is configured for both systems.

Figure 8 - Example Configuration for Reducing Redundant Traffic

Figure 9 – Cost Use Case

Figure 9 shows a vehicle doing agricultural work near the ground station. In this scenario, the vehicle has two interfaces: wireless and cellular. When the vehicle is within range of the ground station which controls/monitors the vehicle, it will leverage wireless. As the vehicle moves out of range of the ground station, it will switch to cellular to allow communication to continue. The vehicle prioritizes wireless communication when available to avoid the added cost of using cellular.

For this example, link prioritization configuration can be applied to both the vehicle and ground station. Since the vehicle is in motion, the ground station will not be aware of the interface change. Therefore, the ground station will be configured to use two interfaces simultaneously. Figure 11 provides an example configuration for the ground station. Though only two interfaces are discussed in our scenario, the ground station could have many additional interfaces. The additional interfaces are relevant because if they are not present, the two interfaces are used by default and link prioritization does not need to be configured.

Figure 10 – Example Configuration for Vehicle

Figure 11 - Example Configuration for Ground Station

Figure 12 – Reliable Channel Use Case

Figure 12 shows a military vehicle driving in hostile territory. Real-time information needs to be available for any incoming threats. As the vehicle is moving, cellular could become unavailable, therefore causing a switch to satellite communication. The vehicle will prioritize cellular communication, as it is more reliable than satellite communication. Link priority configuration will be applied to the vehicle (see Figure 10).

Conclusion

The RTI Connext link prioritization feature offers additional ways to leverage the commonly used allow_interfaces_list QoS property. It is an easy mechanism to modify discovery, in order to help reduce redundant traffic and potentially reduce cost. These are just some of the applicable examples, though there are many additional use cases. Overall, this feature is a smart and expedient way to help systems meet requirements.