.. include:: vars.rst .. _section-routingData: ******************************************** Routing Data: Connecting and Scaling Systems ******************************************** This chapter is devoted to present the most elemental function of |RS|: routing data across multiple DDS domains. Routing data refers to the process of propagating the |Topic| user data from domain to another, allowing systems to interconnect and scale. :numref:`FigureRouterBasicInoutBox` shows the most basic view of the |RS| model. You can think of it as a black box composed of multiple *Input* |DRs| and *Output* |DWs|, each associated with a specific |TOPIC|. Data flows from the input |DRs| to the output |DWs|. The input |DRs| receive data from the *publication side*, whereas the output |DWs| send data to the *subscription side*. .. figure:: static/RouterBasicInoutBox.svg :figwidth: 90 % :name: FigureRouterBasicInoutBox :align: center Basic model of |RS| The |RS| engine takes the data from an input |DR| and passes it along to a specific output |DW|, as if there was a link connecting input and output. This activity is known as the *forwarding process*. |RS| allows configuring this forwarding process. The following sections will guide you through all the |RS| entities involved in the forwarding process and how they are configured. .. note:: All the following sections assume you are already familiar with basic DDS concepts. Additionally you should be familiar with the *RTI* |SHAPESDEMO| tool. Refer to :ref:`section-Tutorials` if you need more information. .. _section-routingData-SingleTopic: Routing a |TOPIC| between two different domains =============================================== The most basic use case of |RS| is about forwarding the data for a specific |TOPIC| from one domain to another. This process is known as *routing a* |TOPIC|. :numref:`FigureRouterBasicTwoTopics` illustrates this concept. .. figure:: static/RouterBasicTwoTopics.svg :figwidth: 90 % :name: FigureRouterBasicTwoTopics :align: center Basic |TOPIC| routing among domains for a |TOPIC| with name Squares The samples for the |TOPIC| named ``Square`` in domain 0 are forwarded to the same |TOPIC| but in domain 1. You will first run :ref:`section-Tutorials-ExampleSingleTopic` in your machine to see the functionality in action. Then we will break down all the parts related to |RS|. Let's review step-by-step each element that appears in the |RS| XML configuration, understanding its purpose and what each of its entities is modeling. Define the service configuration element ---------------------------------------- The first step is to define the top-level element for the |RS| configuration: .. code-block:: xml ... This element defines a configuration profile for |RS|. It must appear within the tag ````–the root tag for all the elements related to |CONNEXT|–. The configuration shall contain a ``name`` attribute that uniquely identifies the service, and determines the *service configuration name*. You can define multiple service configurations in one XML file, and **select one to instantiate a** |RS| by providing the configurations name with the ``-cfgName`` option (or ``ServiceProperty::cfg_name`` member when using the Service API). As we'll see further below, the ``name`` attribute is an important concept since it establishes the *configuration name* of a |RS| entity. This name can be used from other elements in the configuration to refer to a specific entity. .. seealso:: :ref:`section-Usage` How to run |RS| using the shipped executable or embeddeding it into your application with the Service API. :ref:`section-config-routing-service` Reference for the XML configuration of the service element. Specify which domains to join ----------------------------- Within the top-level |RS| configuration we need to specify which *domains* |RS| will be joining. The specification of the domains occurs within the |DOMAINROUTE|, which represents a *mapping* between multiple DDS domains through a collection of |DPs|. In our example, we are joining domains 0 and 1 and we relay on the default participant QoS settings, so the XML looks as follows: .. code-block:: xml 0 1 ... You can specify as many |DPs| as needed. An important aspect to pay attention is the **configuration name assigned to each participant**. This name is what uniquely identifies a domain and is referenced later by |INPUTs| and |OUTPUTs| to indicate the |DP| from which the |DR| and |DW| are created, respectively. .. note:: The value specified with ``>`` in the XML participant configuration can be offset with the ``-domainIdBase`` command-line option. The participant will be created with domain ID = ```` + ``-domainIdBase``. In addition, the ``name`` attribute of the participant configuration is used to form the name assigned to the actual |DP| by setting the :link_connext_users_man_s:`EntityName QoS `. .. seealso:: :numref:`TableParticipantTag` in :ref:`section-Config-DomainRoute` How |RS| constructs the name assigned to the |DP|. :numref:`FigureRouterDomainRouteResource` shows the |DOMAINROUTE| resource model, denoting the association with the service and participant entities. .. figure:: static/RouterDomainRouteResource.svg :figwidth: 90 % :name: FigureRouterDomainRouteResource :align: center |DOMAINROUTE| resource model Define a processing context --------------------------- One of the main aspects that contributes to the high performance of |RS| is the ability to parallelize the processing of the data streams. You can create *threading contexts* to execute of all the activities related to the processing of the data streams. A threading context involves **one or more threads**–a thread pool–, and is specified by the |SESSION| entity. In our example we define a single |SESSION| to take care of processing the data for the single |TOPIC| that is forwarded: .. code-block:: xml ... The |SESSION| must appear inside the |DOMAINROUTE| and you can specify as many |SESSIONs| as you want. In our configuration we rely on the default values, which define a single-threaded context. You could specify a thread pool if, for example, you wanted to parallelize the forwarding of multiple |TOPICs|. :numref:`FigureRouterSessionResource` shows the |SESSION| resource model. .. figure:: static/RouterSessionResource.svg :figwidth: 90 % :name: FigureRouterSessionResource :align: center |SESSION| resource model .. seealso:: Session configuration in :ref:`section-config-session` Reference for the XML configuration of the |SESSION| element. Define the data flow -------------------- The last step consists of defining the flow of *data streams*. For the |TOPIC| *routing* use case, we need to indicate that the **data from a** *Topic* **in the publication side shall be routed to the same topic in the subscription side**. The |TR| is the entity that allows you to define these data flows for the forwarded data. A |TR| is a data processing unit composed of the DDS |INPUTs| and |OUTPUTS| that receive and send the data, respectively. Hence a |TR| effectively represents the establishment of a *route* that data streams follow. Data from the publication side is forwarded to the subscription side. In our example we just define a |TR| with a single |INPUT|–containing a |DR|– and a single |OUTPUT|–containing |DW|–. .. code-block:: xml Square ShapeType Square ShapeType Notice how the |INPUT| and |OUTPUT| are *attached* to a concrete |DP| using the the ``participant`` attribute. The value of this attribute is the name of one the participant configurations defined in the parent |DOMAINROUTE|. This is how you indicate to which domain the |INPUT| and |OUTPUT| are connected to–or from which |DP| the |DR| and |DW| are created, respectively–. In our example, the |INPUT| is attached to the participant configuration with name ``domain0`` for domain 0, whereas the |OUTPUT| is attached to ``domain1`` for domain 1. Additionally, for each |INPUT| and |OUTPUT| we need to specify at least two elements: - **Name of their associated** *Topic*. This indicates the name of the topic for which the |DR| and |DW| are created. In this example, both entities are created from the ``Square`` topic. - **Registered name of the type** associated with the |TOPIC|. This is the name used to identify the type of the user-data samples that are read and written by the input |DR| and output |DW|. |RS| needs to obtain the information prior to create the |DR| and |DW|. There are two provide the type information: - Manually by defining the type in XML - Through discovery from any of the |DP| within the parent |DOMAINROUTE|. This is the mechanism our example relies to get the type and in this case the type is identified by the registered name ``ShapeType`` (you can find the the definition of this type in ``[NDDSHOME]/resource/idl/ShapeType.idl``) You can learn more about type registration and how to configure it in :ref:`section-Config-StreamPort-Types`. For this case of routing a |TOPIC|, both the **input and output topic its associated type are the same**. This is often the situation when you want to simply route data across domains for system integration and scalability. Nevertheless, |RS| is flexible to allow using different topics and types. In that case you will need to *plug* custom code to perform the routing. :ref:`section-controllingData` addresses this use case. :numref:`FigureRouterRouteResource` shows the |TR| resource model. .. figure:: static/RouterRouteResource.svg :figwidth: 90 % :name: FigureRouterRouteResource :align: center |TR| resource model .. seealso:: |TR| configuration in :ref:`section-Config-Route` Reference for the XML configuration of the |TR| element. .. _section-routingData-TopicGroup: Routing a group of |TOPICs| =========================== In section :ref:`section-routingData-SingleTopic` we learned how to route a specific |TOPIC|. We showed how to create a dedicated |TR| to forward the data for a concrete |TOPIC|. You can replicate this process for each |TOPIC| you want to route. However, this process may become repetitive and in some cases avoidable. When such is the case, you can use the |ATR| to **automate the routing for a group of** |TOPICs|. An |ATR| allows you to specify a set of *potential* |TRs| that |RS| will create on-demand upon dynamic *discovery* of the |TOPIC| to be routed. :numref:`FigureRouterBasicAutoRouteConcept` shows the concept of the |ATR|. An |ATR| specifies a *regular expression* that is applied upon the discovery of any new |TOPIC|. The |ATR| creates a new |TR| for each newly discovered |TOPIC| whose name matches with the |ATR|'s expression. .. figure:: static/RouterBasicAutoRouteConcept.svg :figwidth: 90 % :name: FigureRouterBasicAutoRouteConcept :align: center |ATR| concept An |ATR| allows defining a set of potential |TRs| that have a single |INPUT| and a single |OUTPUT|, both tied to their corresponding domain. A regular expression can be specified separately for publication and subscription |TOPICs|. Hence, when the |ATR| matches either with a publication or subscription |TOPIC|, it will create a |TR| to route the matched |TOPIC|. Let's first run :ref:`section-Tutorials-ExampleAllTopics` to see this functionality. This example shows how to configure a |RS| to route *all* the |TOPICs| from domain 0 to domain 1 using an |ATR|. To accomplish that, we have defined the |ATR| as follows: .. code-block:: xml true * * rti/* * * rti/* The configuration of the |ATR| is such that matches the name and registered type name of every |TOPIC| on either ``domain1`` or ``domain2``, *except* the |TOPICs| whose name starts with ``rti/``. An |ATR| allows you to specify two sets of regular expressions for both the input and output of the potential |TRs|: - ``allow_topic_name_filter`` and ``allow_registered_type_name_filter`` specify the set of |TOPIC| names and types that are accepted for the dynamic creation of |TRs|. If **both expressions evaluate as true**, a new |TR| will be created, unless one of the deny filter evaluates as true. - ``deny_topic_name_filter`` and ``deny_registered_type_name_filter`` specify the set of |TOPIC| names and types for which the creation of |TRs| is denied. If **any of the expressions evaluate as true**, the creation of the |TR| will be rejected. These expressions are evaluated after the allow filters, and only if these evaluated as true. The configuration for the input and output of the |ATR| can contain a |DR| and |DW| QoS respectively. You can leverage the concept of **QoS topic filters** to use a different QoS profile based on the name of the matched |TOPIC| (See :ref:`section-routingData-CustomQos-topicFilters`). You can also observe from the example that the |ATR| is defined under a |SESSION|. This means that all the created |TRs| will run under that context. :numref:`FigureRouterAutoRouteResource` shows the |ATR| resource model. .. figure:: static/RouterAutoRouteResource.svg :figwidth: 90 % :name: FigureRouterAutoRouteResource :align: center |ATR| resource model .. seealso:: |TR| configuration in :ref:`section-Config-AutoRoute` Reference for the XML configuration of the |ATR| element. .. _section-routingData-CustomQos: Using custom QoS Profiles ========================= In the previous sections, we showed scenarios in which all the DDS entities of |RS| are created with default QoS. That is, all the QoS policies are set with the initial default values as specified in the |CONNEXT| documentation (see :link_qos_cheatsheet:`QoS Reference Guide <>`). For the majority of the cases though, you may want to specify your custom QoS values for the DDS entities of |RS|. You can easily do that in XML by defining your QoS Profiles and **inherit from them** when specifying the configuration of QoS for each DDS entity. Let's take a look to each step individually. Defining a QoS Library ---------------------- You can define XML QoS profiles for |RS| the same way you can do it for a regular |CONNEXT| application. You can define QoS libraries containing profiles directly under the ```` root element. For example: .. code-block:: xml ... ... ... ... ... As we will see shortly in the next step, within the |RS| configuration you can reference these profiles in order to configure the corresponding underlying DDS entities. You can define as many QoS libraries as you want, each with multiple profiles. Additionally, the definition of QoS libraries can appear either in the same file that contains the |RS| configuration or in a separate one. For information on how to configure QoS in XML, see :link_configuring_qos_xml:`Configuring QoS with XML in the RTI Connext User's Manual<>`. .. seealso:: Loading XML configurations in :ref:`section-Common-Config` How lo load XML configurations in |RS|. Specifying QoS for DDS entities ------------------------------- You can configure the QoS for each DDS entity that |RS| creates. To accomplish this, each |RS| entity that creates an underlying DDS entity provides a corresponding tag to specify its QoS. For example, to configure the QoS for the |DPs| of a |DOMAINROUTE|, you can specify a ```` tag as follows: .. code-block:: xml ... ... ... The QoS tag can have a ``base_name`` attribute to inherit from any available QoS profile, including :link_connext_builtin_qos:`builtin QoS profiles <>`. Additionally, inline values for QoS policies can be specified in order to override default values or set by the base profile. :numref:`TableCustomQosEntityTags` shows the a list of |RS| entities and the DDS entities they create, along with the tags that configure them. .. list-table:: Configuration of the |RS|'s underlying DDS entities. :name: TableCustomQosEntityTags :widths: 20 20 60 :header-rows: 1 :class: longtable * - |RS| Entity - DDS Entity - QoS tag * - |DOMAINROUTE| - |DP| - ```` Example: .. code-block:: xml * - |SESSION| - |PUB| - ```` Example: .. code-block:: xml * - - |SUB| - ```` Example: .. code-block:: xml * - |TR|'s |INPUT| or |ATR|'s |INPUT| - |DR| - ```` Example: .. code-block:: xml * - |TR|'s |OUTPUT| or |ATR|'s |OUTPUT| - |DW| - ```` Example: .. code-block:: xml .. _section-routingData-CustomQos-topicFilters: Applying topic filters to DDS |INPUTs| and |OUTPUTs| ---------------------------------------------------- You can leverage the concept of :link_connext_users_man_s:`topic filters ` to select a QoS for a DDS |INPUT|'s |DR| and |OUTPUT|'s |DW|. You simply need to define a QoS profile containing top-level QoS with a topic filter each, and then inherit from this profile when you specify the QoS for the input |DR| and output |DW|. |RS| will select the appropriate QoS when it creates the |DR| and |DW| based on the name of their associated |TOPIC|. For example, consider a system where there are three types of |TOPIC| categories: user data, monitoring, and administration. Each category has different QoS requirements. You could define a QoS Profile that contains three different |DR| QoS configurations, one for each category: .. code-block:: xml ... ... ... ... Then you can define an |ATR| to route all the |TOPICs| in the system by simply indicating that the input |DR| shall be created using with the QoS obtained from our profile: .. code-block:: xml When the |ATR| creates a |TR| for a matching publication or subscription |TOPIC|, the QoS for the |TR|'s input and output is resolved by matching the topic filter against the |TOPIC| name. The topic filter is applied at the time the |ATR| matches with a publication or subscription |TOPIC|, so the right topic name can be used to match against the topic filter. The selected QoS will be used to create the input |DR| and output |DW| of the generated |TR|. .. _section-routingData-TraversingWan: Traversing Wide Area Networks ============================= In the previous sections we learned to how to route |TOPICs| between domains, understanding the steps required to join the domains, and defining the |TRs| or |ATRs| to route the data. In this section, we will focus on routing data between domains separated geographically. Many systems today have the need to communicate over Wide Area Networks (WAN). This may be the case to connect systems separated geographically. More importantly, it may be the case to provide system connectivity to and within the *cloud*. Access to data centers is often common when there's a requirement for data analytics. You can use |RS| to provide WAN connectivity between sub-systems composed of multiple applications communicating over a Local Area Network (LAN). This architecture allows you to scale the global system efficiently creating multiple databus layers dispersed over the WAN. :numref:`FigureRouterWanTraversal` shows this use case. .. figure:: static/RouterWanTraversal.svg :figwidth: 90% :name: FigureRouterWanTraversal :align: center WAN traversal with |RS|. |RS| can act as an entiry/exit *gateway* to provide connectivity to a WAN or cloud-based data center. The applications running in a LAN only need to know how to reach their gateway |RS|. **Only the gateway services need to know to contact each other**, and they shall be **publicly accessible through the WAN**. This model simplifies the network configuration under presence of NATs/Firewalls, since they just need to be configured to forward the traffic only between the gateway |RS|. You can benefit from this architecture by configuring |RS| to use a *WAN-enabled Transport* to provide communication outside of the private LAN or shared memory network. :numref:`FigureRouterWanConnectivity` illustrates this setup. .. figure:: static/RouterWanConnectivity.svg :figwidth: 60% :name: FigureRouterWanConnectivity :align: center |RS| as WAN/Cloud gateway We will demonstrate how this is possible through the :ref:`section-Tutorials-TcpExample`. This example will help you understand how |RS| can route |TOPICs| between two geographically separated DDS domains comprised of a set of |CONNEXT| applications connected in a LAN. The example scenario is shown in :numref:`FigureRouterExampleWanTCP`. .. figure:: static/RouterExampleWanTCP.svg :figwidth: 90 % :name: FigureRouterExampleWanTCP :align: center Example using the TCP transport to traverse WAN First run the example to see the communication flowing between the *RoutingServices*. You can run all the steps in the same machine for a quicker setup. Let's go through the steps to configure the gateway |RS|. .. note:: For better understanding of this section, we recommend you get familiar with the :link_connext_users_man_s:`RTI TCP Transport `. Define a QoS profile that configures the RTI TCP transport ---------------------------------------------------------- The configuration of the transport is done through the Property QoS for the |DP|. It requires specifying a set of properties to load the transport library (if it's an external transport plugin) and specific values to configure its behavior. To avoid repeating the same configuration for each participant in |RS|, we define a base profile with all the common properties: .. code-block:: xml MASK_NONE dds.transport.load_plugins dds.transport.TCPv4.tcp1 dds.transport.TCPv4.tcp1.library nddstransporttcp dds.transport.TCPv4.tcp1.create_function NDDS_Transport_TCPv4_create dds.transport.TCPv4.tcp1.parent.classid NDDS_TRANSPORT_CLASSID_TCPV4_WAN dds.transport.TCPv4.tcp1.public_address $(PUBLIC_ADDRESS) dds.transport.TCPv4.tcp1.server_bind_port $(BIND_PORT) dds.transport.TCPv4.tcp1.disable_nagle 1 $(REMOTE_RS_PEER) In addition to the transport configuration, the profile also sets the value for the *initial peers* required for the |DP| of the |RS| **to reach the peer remote gateway**. For the definition of this profile, we're leveraging the *XML configuration variables* to reduce even more code duplication. Namely, we define the following variables that are set accordingly when running each |RS|: - ``PUBLIC_ADDRESS``: the public IP address and public port where the |RS| is reachable. - ``BIND_PORT``: the host port that the TCP connection of the |RS| is bound to. This value is important to create a port forwarding rule between public port and host port in the NAT configuration. - ``REMOTE_RS_PEER``: shall contain the discovery peer of the remote |RS| to communicate over the WAN. In this example, the remote peer is the public address and public port of the |RS| gateway for the remote site. This value is used as the *initial peers* of the |DP| that provides WAN connectivity. See :link_discovery_peers_config:`discovery peer configuration <>` for details on setting discovery peers. .. seealso:: :link_transport_plugins:`Transport Plugins <>` Documentation for the Connext Transport Plugin conncept. :link_tcpv4_properties:`RTI TCP Transport properties <>` Available configuration properties for the RTI TCP Transport. Specify the domains to join and which transport to use ------------------------------------------------------ This is the key step that makes possible to forward data from a DDS application in a LAN to the WAN. The main idea is to define two different |DPs| to provide access to the different networks. :numref:`FigureRouterExampleWanTcpDomainRoute` shows the entity model of the |DOMAINROUTE| with its two |DPs|, each using a different underlying transport to communicate with different networks. .. figure:: static/RouterExampleWanTcpDomainRoute.svg :figwidth: 90 % :name: FigureRouterExampleWanTcpDomainRoute :align: center Configuration of the |DOMAINROUTE| to forward data over the WAN The ``DomainLAN`` |DP| is configured to join domain 0 and use the default UDPv4 LAN and shared memory transports to communicate with the applications on the site A LAN. Alternatively, The ``DomainWAN`` |DP| is configured to join domain 1 and use the *RTI TCP Transport* to communicate over the WAN. ``DomainWAN`` is the *gateway* |DP| that **communicates with the remote** |RS| **gateway at a different location**. The definition of these participants appear in a |DOMAINROUTE| as follows: .. code-block:: xml 0 1 You can observe how the ``DomainWAN`` participant is configured with a QoS that inherits from the ``QosLib::TcpWanProfile``, which configures the RTI TCP transport, in addition to other discovery settings. The QoS for this participant provides two additional transport properties to configure the TCP server public address and bind port. Specify the |TOPICs| to be routed --------------------------------- In this example we want to route all the topics between the LAN domains, and we want the communication to be bidirectional. We'll do this by defining two |ATRs| to forward any |TOPIC| for a different communication direction each. We'll place both under a single |SESSION| configured with default settings: .. code-block:: xml rti/* rti/* rti/* rti/* |ATR| ``FromLANtoWAN`` is configured to forward any |TOPIC| coming from the LAN domain to the WAN domain. ``FromWANtoLAN`` |ATR| is configured to forward any |TOPIC| coming from the WAN domain–which connects to the remote LAN Domain–to the local LAN domain. .. figure:: static/RouterExampleWanTcpAutoRoute.svg :figwidth: 90% :name: FigureRouterExampleWanTcpAutoRoute :align: center Definition of |ATR| to forward topics bidirectionally :numref:`FigureRouterExampleWanTcpAutoRoute` illustrates the definition of the |ATRs| to forward all topics between the LAN and WAN domains. Each |ATR| is configured with both input and output filters to match any |TOPIC|. The difference between the |ATRs| is simply the domain assigned to the |INPUT| and |OUTPUT|–the |DP| from which the input |DR| and output |DW| will be created–. Key Terms ========= .. glossary:: Forwarding Process The action of routing data from input to output. Entity Configuration Name Name assigned to uniquely identify an entity. Specified by the attribute ``name``. Publication Side Side of the communication from where |RS| inputs receive data. Subscription Side Side of the communication t0 where |RS| outputs write data. Resource model A model to represent |RS| entities viewed as resources and their relationships. DomainRoute A collection of |DPs|. Session The threading context where the forwarding process takes place. TopicRoute Processing unit for data streams. Composed of multiple |INPUTs| and |OUTPUTs|. AutoTopicRoute Factory of |TRs| based on topic name regular expression matching. Input Entity that reads data from a specific domain. For DDS domains, it contains an underlying |DR|. Output Entity that that writes to a specific domain.For DDS domains, it contains an underlying |DW|. Transport Internal component of a |DP| that provides connectivity to a concrete network technology. Discovery Peer A DDS address that identifies a remote application.