3. Core Concepts
This section aims to provide a deeper understanding of what Cloud Discovery Service is made of, to give you the required insight to configure and use it effectively.
You will learn about:
Domain lists: Sets the context for Cloud Discovery Service in DDS domains.
Transport: Shows how the transports are modeled in Cloud Discovery Service, which is the basis for communication with DomainParticipants.
Forwarder: Explains the active logic of Cloud Discovery Service that is in charge of processing the incoming participant announcements, building the system discovery state, and providing the information to all DomainParticipants.
Database: Describes the collection of discovered DomainParticipant information that Cloud Discovery Service maintains to represent the current state of the system.
3.1. Domain Lists
A single Cloud Discovery Service can forward participant announcements in multiple DDS domains. Yet Cloud Discovery Service preserves the isolation of DDS domain IDs such that a DomainParticipant only discovers other DomainParticipants with the same DDS domain ID.
Figure 3.1 Cloud Discovery Service works with Multiple Domains
By providing a domain list, you can control on which domains Cloud Discovery Service operates. A domain list is a representation of domains by their IDs. You can learn more about configuring domain lists in Domain List.
Support for multiple domain IDs can simplify system deployments, allowing a few Cloud Discovery Service instances to provide discovery services for all possible expected domain IDs, even if all these domain IDs are not known in advance.
3.1.2. Domain Participant Partitions
Domain Participant Partitions provide a way to create different communication planes within a (domain ID, domain Tag). DomainParticipants can join and leave these communication planes at any time while they are running. Partitioning at the DomainParticipant level can be particularly useful in large, WAN, distributed systems (with thousands of DomainParticipants) in which not all DomainParticipants need to know about each other at any given time. Partitioning at the DomainParticipant level helps reduce network, CPU, and memory utilization, because DomainParticipants without matching partitions will not exchange information about their DataWriters and DataReaders.
To summarize, the characteristics of Domain Participant Partitions are:
Mutable: A DomainParticipant can change its partition values at runtime. This allows flexibility within a running system.
Multiple Values: A DomainParticipant can contain multiple values including wildcard characters for its partitions. This allows a DomainParticipant to be a member of multiple partitions simultaneously.
Efficient: They are more efficient than Endpoint Partitions, since traffic is restricted only until the Participant Discovery level. No Endpoint Discovery takes place if partition values for two DomainParticipants don’t match.
Cloud Discovery Service supports systems that utilize Domain Participant Partitions. It forwards announcements to DomainParticipants with matching tuple of (domainID, domainTag) and matching Domain Participant Partitions. In this way, Cloud Discovery Service increases bandwidth reduction even further, because an announcement from a DomainParticipant is only forwarded to DomainParticipants with a matching partition. (Cloud Discovery Service accomplishes this targeted forwarding by maintaining an internal state to help with the filtering related to partitions.)
Figure 3.4 shows the Domain Participant Partitions’ effect when Cloud Discovery Service drives the discovery process.
Figure 3.4 Cloud Discovery Service with Domain Participant Partitions
Cloud Discovery Service detects the Domain Participant Partitions of each DomainParticipant and maintains a mapping for its matching DomainParticipants. That way it can forward the participant announcements only to those DomainParticipants that pass the matching criteria highlighted above. When a DomainParticipant changes its Domain Participant Partitions value, Cloud Discovery Service updates the mapping for the new value before forwarding the incoming participant announcement.
You can specify the Domain Participant Partitions for a DomainParticipant at creation time via
DomainParticipantQos in its Partition QoS Policy. For XML, you need to set
the <partition>
tag, whose value contains the list of partitions associated
with the DomainParticipant. You can also change the value for Domain Participant Partitions at runtime
while the DomainParticipant is running. For more information, see
Isolating DomainParticipants and Endpoints from Each Other
in RTI Connext Core Libraries User’s Manual.
Note
All DomainParticipants are members of at least one concrete partition. If a DomainParticipant does not specify any partition or if it only specifies wildcard partitions, the DomainParticipant is considered a member of the empty partition. The absence of a partition tag is treated like a special value (empty partition). DomainParticipants that do not specify a partition tag will communicate only with each other.
3.1.2.1. Specifying Domain Participant Partitions in XML
The following XML snippet shows how to specify the partition values within
<domain_participant_qos>
.
<domain_participant_qos>
<partition>
<name>
<element>US</element>
<element>A*</element>
</name>
</partition>
</domain_participant_qos>
3.1.2.2. Changing Domain Participant Partitions in code
The following code snippet shows how to change the Domain Participant Partitions value at runtime via code.
DDS_DomainParticipant *participant = NULL;
struct DDS_DomainParticipantQos participantQos =
DDS_DomainParticipantQos_INITIALIZER;
/* QoS configuration happens here */
/* Create the participant */
participant = DDS_DomainParticipantFactory_create_participant(
DDS_TheParticipantFactory,
domainId,
&participantQos,
NULL,
DDS_STATUS_MASK_NONE);
if (participant == NULL) {
/* Handle error */
}
/* Participant is now up and running */
/* Get the QoS */
if (DDS_DomainParticipant_get_qos(participant, &participantQos))
!= DDS_RETCODE_OK) {
/* Handle error */
}
/* Set the sequence length and values */
if (!DDS_StringSeq_ensure_length(
&participantQos.partition.name,
3,
3)) {
/* Handle error */
}
*DDS_StringSeq_get_reference(&participantQos.partition.name, 0) =
DDS_String_dup("US");
if (*DDS_StringSeq_get_reference(&participantQos.partition.name, 0)
== NULL) {
/* Handle error */
}
*DDS_StringSeq_get_reference(&participantQos.partition.name, 1) =
DDS_String_dup("Africa");
if (*DDS_StringSeq_get_reference(&participantQos.partition.name, 1)
== NULL) {
/* Handle error */
}
*DDS_StringSeq_get_reference(&participantQos.partition.name, 2) =
DDS_String_dup("Asia");
if (*DDS_StringSeq_get_reference(&participantQos.partition.name, 2)
== NULL) {
/* Handle error */
}
/* Assign the new QoS */
if (DDS_DomainParticipant_set_qos(participant, &participantQos))
!= DDS_RETCODE_OK) {
/* Handle error */
}
using namespace dds::domain;
using namespace dds::domain::qos;
using namespace dds::core::policy;
using namespace dds::core;
DomainParticipantQos qos = DomainParticipant::default_participant_qos();
// QoS configuration happens here
// Create the participant
DomainParticipant participant(domain_id(), qos);
// Participant is now up and running
// Create a new partition value
Partition newPartitionValues(StringSeq({ "US", "Africa", "Asia" }));
// Update the QoS
qos << newPartitionValues;
// Assign the new QoS
participant << qos;
3.2. Transport
Cloud Discovery Service allows you to select which transports to use to send and receive discovery traffic. The selection of transports defines the possible locations where the service can be reached. DomainParticipants require this information to communicate with the service.
In a Connext application, DomainParticipants automatically configure the underlying transport ports based on the domain ID and the network capabilities of the host machine where the application runs. In Cloud Discovery Service, this configuration is manual and explicit.
Note
Cloud Discovery Service only works with unicast transports. In this manual, any reference to the transport will always imply unicast.
In particular, Cloud Discovery Service allows you to choose:
Transport class instance: A transport class is a concrete realization of a networking transport (e.g., UDP, TCP, etc.). You can instantiate this class and uniquely identify it with a transport alias. In addition, for each instantiation you can configure properties specific to the implementation (e.g., network interfaces, receive buffer sizes, etc.).
Transport Address: A transport address represents an interface for a specific transport class (for example, an IP address for the UDP or TCP transport).
Receive Port: Identifies where the service listens for incoming data. For more information about ports, see About Ports.
Each transport instance-address tuple constitutes a Transport Locator, which is a unique data-reception endpoint. Cloud Discovery Service creates send resources to put discovery traffic on the wire. Connext uses ephemeral ports for outbound data, hence a send resource is specified by a transport instance only. Cloud Discovery Service creates a send resource for each transport instance specified.
Cloud Discovery Service relies on the same pluggable transport framework that is available for Connext applications to create and access transport resources. This implies you can use not only RTI Connext builtin transports, but also your own transport implementations. For more information, see transport plugins in RTI Connext Core Libraries User’s Manual.
DomainParticipants can communicate with Cloud Discovery Service as long as they are configured with the proper transport settings. Then they can reach a specific service instance in two ways:
By addressing the service instance through an RTPS peer descriptor which allows you to identify a DDS service in a generic way by its locator. DomainParticipants can include these in their lists of initial peers.
By addressing the service instance through a peer participant descriptor such that the resulting destination port, computed from the domain ID of the DomainParticipant and well-known ports configuration, matches the listening port of the service.
Cloud Discovery Service determines how to reach DomainParticipants based on the locator information they propagate as part of their participant announcements. Cloud Discovery Service sends discovery traffic to a DomainParticipant by sending data to each of its locators.
3.2.1. RTPS Peer Descriptor
A peer descriptor is a string representation of a set of locators for DDS DomainParticipants. It provides a compact way to indicate a list of locators where a DomainParticipant can find other DomainParticipants. This is known as a participant peer descriptor, or simply a peer descriptor.
RTI Connext applications use the peer descriptor to bootstrap the participant discovery process with other DomainParticipants. Refer to discovery peer configuration in the RTI Connext Core Libraries User’s Manual.
The RTPS peer descriptor is another kind of peer descriptor that allows addressing a service with which you communicate through the RTPS protocol, and that does not necessarily imply the existence of a DomainParticipant. Cloud Discovery Service is an example of such a service.
Note
The RTPS peer descriptor format is supported only by applications using RTI DDS Connext versions 5.3.0 and higher.
The RTPS peer descriptor format is shown below:
Figure 3.5 RTPS Peer Descriptor Format
Table 3.1 describes all the elements in the RTPS peer descriptor.
Element |
Description |
Required |
Default |
|
Keyword to indicate the RTPS descriptor kind. |
Yes |
|
|
Separator. |
Only when |
|
|
Specifies a transport and an address. See locator format. |
No |
udpv4://localhost |
|
Separator. |
Only when |
|
|
RTPS Peer receive port. See About Ports. |
No |
7400 |
3.2.1.1. Example: RTPS Peer Descriptors
|
|
|
|
|
|
|
|
3.2.1.2. Example: Transport Setup and Resulting RTPS Descriptor
Assume Cloud Discovery Service is configured with the following transport
settings and running on host with address CDS-Host-IP-Address
:
<transport>
<element>
<alias>udpv4</alias>
<receive_port>57410</receive_port>
</element>
</transport>
DomainParticipants can send discovery traffic to this service by adding the following peer to their initial peer lists:
rtps@udpv4://<CDS-Host-IP-Address>:57410
3.2.2. About Ports
Discovery traffic through RTPS relies on well-known ports to establish communication between peers. These ports are a logical concept that allow multiplexing communications at the middleware layer.
It is up to the underlying transport implementation to decide how to map these logical port numbers into the physical transport address scheme. For instance, the RTI builtin UDP transport directly maps the logical port number into the physical UDP port.
The port mapping is especially important in Cloud environments where UDP and TCP dominate the transport layer. The presence of these protocols often requires knowing details about the communication ports to properly set up application services.
In a DomainParticipant, unless explicitly configured, ports are determined automatically based on the domain ID and participant ID. Also, the port mapping is fully controlled by the underlying transport. In Cloud Discovery Service, the ports must be configured explicitly. Moreover, for the direct known RTI transport implementations that rely on UDP and TCP, Cloud Discovery Service maps the (logical) receive port directly to the transport physical port.
See RTPS Ports Used for Communication to learn more about RTPS ports.
3.3. Forwarder
The forwarder is the Cloud Discovery Service component where all the discovery logic resides. Its responsibility is to build/maintain the discovery state and forward participant announcements to the peer DomainParticipants so that they can discover each other.
Figure 3.6 shows a representation of the forwarder element of Cloud Discovery Service.
Figure 3.6 Participant Announcement Forwarder
The forwarder is composed of four main blocks:
Receiver: The element in charge of retrieving incoming participant announcements. This element interfaces with the receive resources, which provide the participant announcements received from all the peer DomainParticipants.
Announcement Queues: The received participant announcements are placed in queues so they can be forwarded to their proper destinations. Participant announcements can be categorized in three classes:
New: Announcements from DomainParticipants that are considered new from the service perspective.
Change/Update: Announcements from DomainParticipants that the service has already seen but that contain content changes relative to the previous announcement from the same DomainParticipant. For example, QoS or Locator changes or Locator resolutions for the DomainParticipant.
Refresh/Repeat: Announcements from DomainParticipants that the service has already seen and that do not contain any content changes from the previously received announcement.
This classification allows the forwarder to process the different categories differently, allowing you to prioritize and regulate the bandwidth used by each traffic class.
Flow Controller: The entity that removes announcements from the queue and forwards them to the proper destinations. The flow controller regulates the output announcement traffic. The flow controller uses the send resources to direct the announcements to the proper DomainParticipants.
Event Manager: This element is in charge of handling events like UDPv4 WAN locator resolution and announcement resend. Each of these events is described below:
UDPv4 WAN Locator Resolution: This event happens when the Real-Time WAN Transport is used as a transport in a WAN scenario. In such an environment due to the presence of a NAT, locators in the incoming participant announcement are resolved asynchronously to their public IP addresses. Once the locator is resolved by Cloud Discovery Service, it creates a job of the Change type. For more details on this scenario refer to section NAT Traversal.
Announcement Resend: This event is triggered when Cloud Discovery Service has been configured to perform resends. Resends are performed when a New or Change announcement is received. It is also performed after a UDPv4 WAN Locator Resolution event occurs, which is considered as a Change. Resend artifically generates jobs of the announcement kind Repeat that send out multiple copies of an incoming or resolved participant announcement.
Resends are particularly useful in networks with high probability of packet loss that hampers discovery speed. This is especially true when using the Real-Time WAN Transport for connectivity over the internet. For more details on configuring this mechanism refer to section Forwarder.
3.3.1. Flow Controller
Cloud Discovery Service incorporates a configurable flow controller, which allows shaping the generated output traffic as a result of forwarding participant announcements. The parameters that configure a flow controller are:
Output capacity: The maximum amount of announcements forwarded in a period of time. It is measured in jobs-per-time units (e.g., announcements per second). Each received announcement represents a job to forward the announcement to all discovered DomainParticipants. This parameter allows setting an upper bound to the output traffic to avoid network congestion.
Maximum job burst: The maximum amount of consecutive jobs that can be forwarded in a period of time. This parameter allows setting an upper limit to the output traffic peak.
Flush period: The period at which the flow controller attempts to forward pending announcements.
Note
Because forwarding an announcement requires sending each announcement to multiple discovered participants, the actual output bandwidth depends on the number of participants.
3.3.1.1. Operation Mode
The flow controller generates job tokens at the rate specified by the output capacity. Forwarding an announcement takes exactly one job token. The forwarder queues announcements upon reception and attempts to forward them as soon as tokens are available.
If tokens are not available, the announcement jobs will remain in the queue in a pending state. The forwarder wakes up at every flush period to check for available tokens to forward the pending announcements.
If tokens are generated faster than jobs are received, the flow controller accumulates the tokens for future jobs. The flow controller will accumulate no more than the maximum job burst tokens.
3.4. Database
Cloud Discovery Service uses an internal database to keep information about remote entities. This is the information that represents the discovery state of the system.
This state is maintained upon reception of discovery information. Remote entries in the database are added or removed based on the received information. The database will release the resources for each removed entry as needed.
Cloud Discovery Service relies on a dedicated thread, which periodically cleans up any removed state from the database. This model enhances concurrency while maintaining thread safety.
This element is equivalent to the Database Thread of a DomainParticipant.