Configuring Discovery and Transports

Objectives of this user guide:

• Background and references for understanding DDS discovery.
• Understanding dynamic participant, dynamic endpoint (DPDE) discovery.
• Understanding dynamic participant, static endpoint (DPSE) discovery.
• Configuring the list of peers to which participant discovery messages are sent.
• Configuring the enabled transports used per DDS entity for discovery and user traffic.
• Configuring transport properties, especially for the builtin UDP transport
• Understanding when unicast and multicast locators are used.

Introduction to DDS Discovery

An RTI Connext DDS Micro application will create one or more DomainParticipants, each containing DataWriters and/or DataReaders. Each DataWriter and DataReader will, respectively, publish and subscribe to a particular Topic. A DataWriter must send its data only to DataReaders with which it matches: these are the DataReaders that share the DataWriter's Topic and have consistent QoS configuration. Likewise, a DataReader must receive data only from DataWriters it matches. How does RTI Connext DDS Micro know which DataWriters match with which DataReaders?

Discovery is the behind-the-scenes way by which RTI Connext DDS Micro entities (DomainParticipants, DataWriters, and DataReaders) on different nodes find out about and match with each other. For every DomainParticipant, the discovery process builds a complete picture of all the entities contained in the DomainParticipant and in others, both local or remote. Discovery involves DomainParticipants exchanging messages containing QoS information about themselves and the DataWriters and DataReaders they contain.The result of the discovery process is the determination of which DataWriters can communicate with which DataReaders.

Discovery happens in two phases. The first phase, known as Simple Participant Discovery, involves DomainParticipants discovering other DomainParticipants. When two DomainParticipants have discovered one another, they move on to the second phase, known as Simple Endpoint Discovery. During this phase, they exchange information about the DataWriters and DataReaders they contain and they establish matches between their endpoints.

Detailed information on the discovery process and its message protocol can be found:

• From RTI Documentation, Chapter 14 (Discovery) of RTI Connext DDS's Core Libraries and Utilities User's Manual.

• From OMG, Section 9.6 (Mapping of the RTPS Protocol) of the Real-time Publish-Subscribe Wire Protocol DDS Interoperability Wire Protocol Specification, Version 2.1.

Transports in RTI Connext DDS Micro

RTI Connext DDS Micro is designed with a transport layer that provides a common interface that every transport must implement. Each transport is registered within RTI Connext DDS Micro with a unique name. An RTI Connext DDS Micro application chooses which transports to register and enable for its DDS entities to use for communication between local endpoints and with remote endpoints.

Upon creating a DomainParticipant, RTI Connext DDS Micro will query the platform for the available network interfaces that support the enabled transports. RTI Connext DDS Micro supports a maximum of four unicast and/or multicast transports for discovery traffic, and likewise a maximum of four unicast and/or multicast transports for user traffic. If there are more than these maximum number of available network interfaces, RTI Connext DDS Micro will choose the first ones returned by the system.

Choosing Between DomainParticipant and Endpoint Transports

DDS_DomainParticipantQos::transports sets the default enabled transports for a DomainParticipants. DataWriters and DataReaders belonging to the DomainParticipant use DDS_DomainParticipantQos::transports as their enabled transports, unless they themselves have had their DDS_DataWriterQos::transport or DDS_DataReaderQos::transport configured, whereupon those endpoint specific transports have precedence and are used.

Choosing Between Unicast and Multicast Locators

DDS entities can have a mix of unicast and multicast locators to which they will send data. DDS_DomainParticipantQos::transports and DDS_DataReaderQos::transport can contain unicast or multicast, while DDS_DataWriterQos::transport can contain only unicast.

When interoperating with RTI Connext DDS, RTI Connext DDS Micro must specify at least one unicast transport for each DataWriter and DataReader, either from DDS_DomainParticipantQos::transports or the endpoint DDS_DataReaderQos::transport and DDS_DataWriterQos::transport, as it expects to use the unicast transport's RTPS port mapping to determine automatic participant IDs if needed. This also affects RTI Connext DDS Micro itself, where participant IDs must be set manually if only multicast transports are enabled.

Also when interoperating with RTI Connext DDS, only one multicast transport can be specified per DataReader of RTI Connext DDS Micro.

Builtin Transports: UDP and Intra

RTI Connext DDS Micro provides two builtin transports: UDPv4, named "_udp", and an intra-participant transport, named "_intra". The intra-participant transport can be used instead of UDP loopback for faster communication between DataWriters and DataReaders belonging to the same DomainParticipant.

Transport Descriptors

A transport descriptor string is used to configure transports, with a prefix for the transport name, and the remaining being a transport specific address. For example, "_udp://127.0.0.1" is UDP loopback, while "_udp://" itself represents the addresses of all active interfaces of the system that support UDPv4.

UDP Configuration

Configuring the builtin UDPv4 transport is a common step for RTI Connext DDS Micro applications. The UDP_InterfaceFactoryProperty of the UDP transport is an input upon registering the transport. Because "_udp" is a builtin transport, it is already registered within the RTI Connext DDS Micro registry of transports when the DomainParticipantFactory is first accessed. Configuring _udp thus requires:

• Unregistering _udp from the RTI Connext DDS Micro registry.
• Setting the UDP property.
• Re-registering _udp with the updated UDP property.

These steps are done within the example code snippet below, adapted from the HelloWorld_dpde example. udp_property.allow_interface is a commonly configured field, and this example sets it to two available interfaces, lo and eth0:

 /* Configure UDP transport's allowed interfaces.  
    These are the network interfaces that can be used for sending and 
    receiving DDS traffic.  
    Here, we allow the loopback interface and eth0
  */

 /* The builtin UDP transport with the well-known name of "_udp" must 
     be unregistered and re-registered with a new UDP property
  */
 if (!RT_Registry_unregister(registry, "_udp", NULL, NULL))
 {
     printf("failed to unregister udp\n");
     goto done;
 }

 struct UDP_InterfaceFactoryProperty udp_property = 
     UDP_INTERFACE_FACTORY_PROPERTY_DEFAULT;   

 /* For additional allowed interface(s), increase maximum and length, and
    set interface below:
 */
 REDA_StringSeq_set_maximum(&udp_property.allow_interface, 2);
 REDA_StringSeq_set_length(&udp_property.allow_interface, 2);

 /* loopback interface */
 *REDA_StringSeq_get_reference(&udp_property.allow_interface, 0) = 
     REDA_String_dup("lo");

 /* external interface */
 *REDA_StringSeq_get_reference(&udp_property.allow_interface, 1) = 
     REDA_String_dup("eth0");

 if (!RT_Registry_register(registry, "_udp",
                      UDP_InterfaceFactory_get_interface(),
                     (struct RT_ComponentFactoryProperty*)&udp_property, 
                     NULL))
 {
    printf("failed to register udp\n");
    goto done;
} 

Participant Discovery in RTI Connext DDS Micro

Simple Participant Discovery is the process by which each DomainParticipant sends a message about itself to a configurable list of peer locators, to allow other DomainParticipants, receiving on one of the specified peer locators, will discover the existence of the sending DomainParticipant.

A participant discovery announcement contains identifying information about a DomainParticipant, including its GUID, its name and the unicast and/or multicast locators that other DDS entities can use to send data and to reach it. These locators are picked from DomainParticipantQos, using the transports enabled for discovery and user traffic (see Enabling Transports).

In RTI Connext DDS Micro, DomainParticipantQos contains a DiscoveryQos field for configuring the transports that are enabled for discovery (DDS_DiscoveryQosPolicy::enabled_transports) and the initial list of peer locators (DDS_DiscoveryQosPolicy::initial_peers).

Enabling Transports

A transport in RTI Connext DDS Micro is a component implementing a common transport interface. Every transport is registered with RTI Connext DDS Micro with a name. RTI Connext DDS Micro provides and enables by default two transports: UDPv4 ("_udp") and intra-participant ("_intra"). If a DomainParticipant contains a DataWriter and a matching DataReader, they will communicate using the intra-participant transport if it is enabled.

A few QoS have enabled_transports as a field: DDS_DomainParticipantQos::transports, DDS_DomainParticipantQos::discovery, and DDS_DomainParticipantQos::user_traffic. DDS_DomainParticipantQos::transports sets the transports that are enabled and useable for all entities belonging to the same DomainParticipant. For exxample, this means if DDS_DomainParticipantQos::transports only has the "_intra" intra-participant transport enabled, DDS_DomainParticipantQos::user_traffic cannot have locators using transport "_udp".

The enabled_transports of DDS_DomainParticipantQos::discovery sets the transports used for discovery traffic: data sent and received between builtin discovery DataWriters and DataReaders for completing discovery. The enabled_transports of DDS_DomainParticipantQos::user_traffic sets the transports used for user traffic: data sent and received between DataWriters and DataReaders for user defined topics.

Default Enabled Transports

enabled_transports is empty by default, and when empty, each QoS policy automatically adds a different set of default transports:

• DDS_DomainParticipantQos::transports: "_udp", "_intra"
• DDS_DomainParticipantQos::user_traffic: "_udp://", "_intra://", "_udp://127.0.0.1"
• DDS_DomainParticipantQos::discovery:"_udp://", "_udp://239.255.0.1", "_udp://127.0.0.1"

For the builtin UDP transport, the default max message size is 8KB. This can be increased up to the UDP max message size limit (64KB) by setting max_message_size in UDP_InterfaceFactoryProperty, and it may need to be increased to best interoperate with the larger default max message sizes of RTI Connext DDS. Note, RTI Connext DDS Micro does not support RTPS fragementation for messages larger than max_message_size. UDP_InterfaceFactoryProperty also can set the maximum sizes of the socket send and receive buffers (max_send_buffer_size and max_receive_buffer_size).

Enabled Transports Example

As an example of setting enabled_transports, if an application has only one DomainParticipant, then using only the "_intra" transport will achieve faster performance, so enabled_transports for discovery and user traffic can be set like so:

/* Discovery enabled_transports */
DDS_StringSeq_set_maximum(&dp_qos.discovery.enabled_transports, 1);
DDS_StringSeq_set_length(&dp_qos.discovery.enabled_transports, 1);

*DDS_StringSeq_get_reference(&dp_qos.discovery.enabled_transports, 0) = 
     DDS_String_dup("_intra");

/* User Traffic enabled_transports */
DDS_StringSeq_set_maximum(&dp_qos.user_traffic.enabled_transports, 1);
DDS_StringSeq_set_length(&dp_qos.user_traffic.enabled_transports, 1);

*DDS_StringSeq_get_reference(&dp_qos.user_traffic.enabled_transports, 0) = 
     DDS_String_dup("_intra");

Configuring Participant Discovery Peers

An RTI Connext DDS Micro DomainParticipant must be able to send participant discovery announcement messages for other DomainParticipants to discover itself, and it must receive announcements from other DomainParticipants to discover them.

To do so, each DomainParticipant will send its discovery announcements to a set of locators known as its peer list, where a peer is the transport locator of one or more potential other DomainParticipants to discover.

peer_desc_string

A peer descriptor string of the initial_peers string sequence conveys the interface and address of the locator to which to send, as well as the indices of participants to which to send. For example:

DDS_StringSeq_set_maximum(&dp_qos.discovery.initial_peers, 3);
DDS_StringSeq_set_length(&dp_qos.discovery.initial_peers, 3);

*DDS_StringSeq_get_reference(&dp_qos.discovery.initial_peers, 0) = 
     DDS_String_dup("4@_udp://239.255.0.1");
    
*DDS_StringSeq_get_reference(&dp_qos.discovery.initial_peers, 1) = 
     DDS_String_dup("[1,3,4]@_udp://10.10.30.101");

*DDS_StringSeq_get_reference(&dp_qos.discovery.initial_peers, 2) = 
     DDS_String_dup("[2]@_udp://10.10.30.102");

The peer descriptor format is

    [index@][interface://]address

Remember that every DomainParticipant has a participant index that is unique within a DDS domain. The participant index (also referred to as the participant ID), together with the DDS domain ID, is used to calculate the network port on which DataReaders of that participant will receive messages. Thus, by specifying the participant index, or a range of indices, for a peer locator, that locator becomes a port to which messages will be sent only if addressed to the entities of a particular DomainParticipant. Specifying indices restricts the number of participant announcements sent to a locator where other DomainParticipants exist and, thus, should be considered to minimize network bandwidth usage.

In the above example, the first peer, "4@_udp://239.255.0.1," has the default UDPv4 multicast peer locator. It also has the prefix of "4". In peer descriptor syntax, a lone integer represents a range of participant indices from zero to that integer, exclusive, meaning participant discovery messages will be sent to 239.255.0.1, with ports corresponding to participant indices 0 through 3.

The second peer, "[1,3,4]@_udp://10.10.30.101," has a unicast address. It also has indices in brackets. These represent individual participant indices. Thus, participant discovery messages with will be sent to participants with indices 1, 3, and 4.

Lastly, the third peer, "[2]@_udp://10.10.30.102," is a unicast locator to a single participant with index 2.

Initial Peers and Adding Peers

DDS_DiscoveryQosPolicy::initial_peers is the list of peers a DomainParticipant sends its participant announcement messages, when it is enabled, as part of the discovery process.

DDS_DiscoveryQosPolicy::initial_peers is an empty sequence by default, so while DDS_DiscoveryQosPolicy::enabled_transports by default includes the DDS default loopback and multicast (239.255.0.1) addresses, initial_peers must be configured to include them.

Peers can also be added to the list, before and after a DomainParticipant has been enabled, by using DDS_DomainParticipant_add_peer.

The DomainParticipant will start sending participant announcement messages to the new peer as soon as it is enabled.

Endpoint Discovery in RTI Connext DDS Micro

When a DomainParticipant receives a participant discovery message from another DomainParticipant, it will engage in the process of exchanging information of user created DataWriter and DataReader endpoints.

RTI Connext DDS Micro provides two ways of determinig endpoint information of other DomainParticipants: Dynamic Endpoint Discovery, and Static Endpoint Discovery.

Dynamic Endpoint Discovery

Dynamic endpoint discovery uses builtin discovery DataWriters and DataReader to exchange messages about user created DataWriter and DataReaders. A DomainParticipant using dynamic participant, dynamic endpoint (DPDE) discovery will have a pair of builtin DataWriters for sending messages about its own user created DataWriters and DataReaders, and a pair of builtin DataReaders for receiving messages from other DomainParticipants about their user created DataWriters and DataReaders.

Given a DomainParticipant with a user DataWriter, receiving an endpoint discovery message for a user DataReader allows the DomainParticipant to get the type, topic, and QoS of the DataReader that determine whether the DataReader is a match. When a matching DataReader is discovered, the DataWriter will include that DataReader and its locators as destinations for its subsequent writes.

Static Endpoint Discovery

Static endpoint discovery uses function calls to statically assert information about remote endpoints belonging to remote DomainParticipants. An application with a DomainParticipant using dynamic participant, static endpoint (DPSE) discovery has control over which endpoints belonging to particular remote DomainParticipants are discoverable.

Whereas dynamic endpoint-discovery can establish matches for all endpoint-discovery messages it receives, static endpoint-discovery establishes matches only for the endpoint that have been asserted programmatically.

With DPSE, a user needs to know a priori the configuration of the entities that will need to be discovered by its application. The user must know the names of all DomainParticipants within the DDS domain and the exact QoS of the remote DataWriters and DataReaders.

The APIs for remote entity assertion:

• DPSE_RemoteParticipant_assert
• DPSE_RemotePublication_assert
• DPSE_RemoteSubscription_assert

Remote Participant Assertion

Given a local DomainParticipant, static discovery requires first the names of remote DomainParticipants to be asserted, in order for endpoints on them to match. This is done by calling DPSE_RemoteParticipant_assert with the name of a remote DomainParticipant. The name must match the name contained in the participant discovery announcement produced by that DomainParticipant. This has to be done reciprocally between two DomainParticipants so that they may discover one another.

For example, a DomainParticipant has entity name "participant_1", while another DomainParticipant has name "participant_2." participant_1 should call DPSE_RemoteParticipant_assert("participant_2") in order to discover participant_2. Similarly, participant_2 must also assert participant_1 for discovery between the two to succeed.

/* participant_1 is asserting (remote) participant_2 */
retcode = DPSE_RemoteParticipant_assert(participant_1,
                                        "participant_2");
if (retcode != DDS_RETCODE_OK) {
    printf("participant_1 failed to assert participant_2\n");
    goto done;
}

Remote Publication and Subscription Assertion

Next, a DomainParticipant needs to assert the remote endpoints it wants to match that belong to an already asserted remote DomainParticipant. The endpoint assertion function is used, specifying an argument which contains all the QoS and configuration of the remote endpoint. Where DPDE gets remote endpoint QoS information from received endpoint-discovery messages, in DPSE, the remote endpoint's QoS must be configured locally. With remote endpoints asserted, the DomainParticipant then waits until it receives a participant discovery announcement from an asserted remote DomainParticipant. Once received that, all endpoints that have been asserted for that remote DomainParticipant are considered discovered and ready to be matched with local endpoints.

Assume participant_1 contains a DataWriter, and participant_2 has a DataReader, both communicating on topic HelloWorld. participant_1 needs to assert the DataReader in participant_2 as a remote subscription. The remote subscription data passed to the operation must match exactly the QoS actually used by the remote DataReader:

/* Set participant_2's reader's QoS in remote subscription data  */
rem_subscription_data.key.value[DDS_BUILTIN_TOPIC_KEY_OBJECT_ID] = 200;
rem_subscription_data.topic_name = DDS_String_dup("Example HelloWorld");
rem_subscription_data.type_name = DDS_String_dup("HelloWorld");
rem_subscription_data.reliability.kind = DDS_RELIABLE_RELIABILITY_QOS;

/* Assert reader as a remote subscription belonging to (remote) participant_2 */
retcode = DPSE_RemoteSubscription_assert(participant_1,
                                         "participant_2",
                                         &rem_subscription_data,
                                         HelloWorld_get_key_kind(HelloWorldTypePlugin_get(), NULL)))
{
    printf("failed to assert remote subscription\n");
    goto done;
}

Reciprocally, participant_2 must assert participant_1's DataWriter as a remote publication, also specifying matching QoS parameters:

/* Set participant_1's writer's QoS in remote publication data  */
rem_publication_data.key.value[DDS_BUILTIN_TOPIC_KEY_OBJECT_ID] = 100;
rem_publication_data.key.value.topic_name = DDS_String_dup("Example HelloWorld");
rem_publication_data.key.value.type_name = DDS_String_dup("HelloWorld");
rem_publication_data.key.value.reliability.kind = DDS_RELIABLE_RELIABILITY_QOS;

/* Assert writer as a remote publication belonging to (remote) participant_1 */
retcode = DPSE_RemotePublication_assert(participant_2,
                                        "participant_1",
                                        &rem_publication_data,
                                        HelloWorld_get_key_kind(HelloWorldTypePlugin_get(), NULL)))
{
    printf("failed to assert remote publication\n");
    goto done;
}

When participant_1 receives a participant discovery message from participant_2, it is aware of participant_2, based on its previous assertion, and it knows participant_2 has a matching DataReader, also based on the previous assertion of the remote endpoint. It therefore establishes a match between its DataWriter and participant_2's DataReader. Likewise, participant_2 will match participant_1's DataWriter with its local DataRead, upon receiving one of participant_1's participant discovery messages.

Note, with DPSE, there is no runtime check of QoS consistency between DataWriters and DataReaders, because no endpoint discovery messages are exchanged. This makes it extremely important that users of DPSE ensure that the QoS set for a local DataWriter and DataReader is the same QoS being used by another DomainParticipant to assert it as a remote DataWriter or DataReader.