52.13 Troubleshooting Real-Time WAN Transport

52.13.1 Communication Stops Working after Application Transitions to Different Network

Possible Root Cause:

If you are using Cloud Discovery Service, it is possible that before the network transition, all the applications were behind cone NATs. When the transition occurs, the application that is changing to a new network connects to a symmetric NAT.

This configuration is not currently supported. Symmetric NATs are only supported in scenarios like the one described in Figure 52.7: Peer-to-Peer between a Participant behind Any Kind of NAT and an External Participant.

To validate if you are behind a symmetric NAT in the new network, you can run the application natat as described in 52.3.3 Identifying the NAT Type.

Solution:

When you do not know in advance whether the networks in which applications run are behind cone NATs or symmetric NATs, you may want to do a relay deployment as indicated in 52.5.2 Relayed Edge-to-Edge Deployment Scenario.

52.13.2 Communication not Established after Changing Cloud Discovery Service <receiver_port>

For example, when changing the Cloud Discovery Service configuration from:

<dds>

    <cloud_discovery_service name="CDS">

        <transport>

            <element>

                <alias>builtin.udpv4_wan</alias>

                <receive_port>2345</receive_port>

                <property>

                    <element>

                        <name>dds.transport.UDPv4_WAN.builtin.public_address</name>

                        <value>50.10.23.45</value>

                    </element>

                </property>

            </element>

        </transport>

    </cloud_discovery_service>

</dds>

To:

<dds>

    <cloud_discovery_service name="CDS">

        <transport>

            <element>

                <alias>builtin.udpv4_wan</alias>

                <receive_port>6001</receive_port>

                <property>

                    <element>

                        <name>dds.transport.UDPv4_WAN.builtin.public_address</name>

                        <value>50.10.23.45</value>

                    </element>

                </property>

            </element>

        </transport>

    </cloud_discovery_service>

</dds>

Possible Root Cause:

If Cloud Discovery Service (CDS) is running behind a NAT-enabled router, it is possible that you have not created a static NAT binding in the router for the new public address: 50.10.23.45:6001.

Solution:

Create a new static NAT binding to support the port change. The NAT binding must create this mapping:

<CDS private address X>:<port number Y> → <new public address Z>:<port number Y>

<CDS private address>:6001 → 50.10.23.45:6001

Note that the private host CDS port and public port must be the same (6001 is the port number for both in the example above). To make them different, use the transport property dds.transport.UDPv4_WAN.builtin.comm_ports. See 52.8.1 Setting Real-Time WAN Transport Properties.

52.13.3 Communication not Established even though Transport Settings are Set Correctly

Possible Root Cause:

There may be an IP fragmentation problem. For WAN communications, it is not a good idea to rely on IP fragmentation. IP fragmentation causes significant issues in UDP, where there is no support for an MTU (maximum transmission unit) discovery protocol as there is in TCP. These are some of the problems associated with IP fragmentation:

• To successfully reassemble a packet, all fragments must be delivered. No fragment can become corrupt or get lost in-flight. If a fragment is lost, the whole packet will be lost.
• Before reassembly, a host must hold partial, fragment datagrams in memory. This opens an opportunity for memory exhaustion attacks.
• Subsequent fragments lack the higher-layer header. The TCP or UDP header is only present in the first fragment, making it impossible for firewalls to filter fragment datagrams based on criteria like source or destination ports.

When testing over some cellular networks, in some cases you may not be able to send samples larger than the IP MTU, such as images, without losing a large percentage of the frames.

Solution:

Disable IP fragmentation by letting Connext do fragmentation at the RTPS level. For details, see section 52.8.3 Disabling IP Fragmentation for Real-Time WAN Transport.

52.13.4 Slow Discovery using Cloud Discovery Service

Possible Root Cause:

If you are using Cloud Discovery Service, it is possible that the Participant Announcements sent by Cloud Discovery Service to other DomainParticipants are lost when discovering a new DomainParticipant or when there are changes in the configuration of a DomainParticipant. This may delay the discovery process in lossy network environments.

Solution:

You can increase the number of times that a Participant Announcement from a new DomainParticipant is forwarded by Cloud Discovery Service to other DomainParticipants. The default is 5 times.

You can also adjust the frequency at which the Participant Announcements are forwarded to the other DomainParticipants. The default is 1 second.

For example:

<dds>

    <cloud_discovery_service name="CDS">

        <transport>

            <element>

                <alias>builtin.udpv4_wan</alias>

                <receive_port>2345</receive_port>

                <property>

                    <element>

                        <name>dds.transport.UDPv4_WAN.builtin.public_address</name>

                        <value>60.10.23.45</value>

                    </element>

                </property>

            </element>

        </transport>

        <forwarder>

            <event>

                <!-- Set to a high value of 10 for lossy networks -->

                <new_or_change_participant_announcements>

                    10

                </new_or_change_participant_announcements>

                <min_new_or_change_participant_announcements_period>

                    <sec>1</sec>

                    <nanosec>0</nanosec>

                </min_new_or_change_participant_announcements_period>

                <!-- Increase the spacing between resend event samples -->

                <max_new_or_change_participant_announcements_period>

                    <sec>2</sec>

                    <nanosec>0</nanosec>

                </max_new_or_change_participant_announcements_period>

            </event>

        </forwarder>

    </cloud_discovery_service>

</dds>

For additional information, see "Forwarder" section, in XML Tags for Configuring RTI Cloud Discovery Service, in the RTI Cloud Discovery Service documentation.