2.1.2. Discovery Performance
This section provides the results of testing Simple Discovery in RTI Connext 7.3 in different scenarios, where we modify the number of participants (DomainParticipants) and endpoints (DataWriters and DataReaders).
These numbers should only be used as a first rough approximation, since the results are highly dependent on the hardware, software architecture, QoS in use, data types, and network infrastructure of the system.
2.1.2.1. Time to Complete Discovery (SPDP and SPDP2)
The following test measures the time it takes to complete participant discovery as well as endpoint discovery.
To simplify the configuration of the test environment, we assume the following:
Every application in the system runs a single DomainParticipant.
There is only one endpoint per DomainParticipant. Every DomainParticipant creates either a DataReader or a DataWriter.
There is only one Topic in the system.
50% of the endpoints are DataReaders, 50% are DataWriters.
We perform the tests when the entities use Unicast to discover each other and when they use Multicast.
All applications are equally distributed across all the available machines for the test.
Note
This scenario is likely not the optimal design solution for a real-life architecture. The purpose of the test is to demonstrate how powerful the Connext discovery protocol is even in a flat configuration.
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
In this particular multicast scenario, SPDP2’s network usage is higher than SPDP because repairs for the participant configuration packets are sent over unicast and these packets are sent from each participant to each remote participant, whereas a single SPDP multicast Participant announcement (DATA(P)) can be received via multicast by all participants.
The RECEIVED network load is still higher for SPDP because an SPDP participant may receive many multicast DATA(P)s over multicast and these are larger than the bootstrap messages.
Sent Bytes
Received Bytes
Sent Packets
Received Packets
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
In this particular multicast scenario, SPDP2’s network usage is higher than SPDP because repairs for the participant configuration packets are sent over unicast and these packets are sent from each participant to each remote participant, whereas a single SPDP multicast Participant announcement (DATA(P)) can be received via multicast by all participants.
Packet Sent
Packet Received
Packet Receive Errors
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
However, in this particular unicast scenario it is worth noticing that the new SPDP2 protocol can get to substancially better results for large numbers of participants. This is because the SPDP2 protocol’s bootstrap messages and liveliness messages are smaller than the participant announcement messages sent by SPDP. SPDP2 will only need to send the liveliness messages after the initial bootstrap message while SPDP will need to send the whole participant announcement messages (larger in size), this process has to be done by all the participant to every other discovered participant.
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
However, in this particular unicast scenario it is worth noticing that the new SPDP2 protocol can get to substancially better results for large numbers of participants. This is because the SPDP2 protocol’s bootstrap messages and liveliness messages are smaller than the participant announcement messages sent by SPDP. SPDP2 will only need to send the liveliness messages after the initial bootstrap message while SPDP will need to send the whole participant announcement messages (larger in size), this process has to be done by all the participant to every other discovered participant.
Sent Bytes
Received Bytes
Sent Packets
Received Packets
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
However, in this particular unicast scenario it is worth noticing that the new SPDP2 protocol can get to substancially better results for large numbers of participants. This is because the SPDP2 protocol’s bootstrap messages and liveliness messages are smaller than the participant announcement messages sent by SPDP. SPDP2 will only need to send the liveliness messages after the initial bootstrap message while SPDP will need to send the whole participant announcement messages (larger in size), this process has to be done by all the participant to every other discovered participant.
Packet Sent
Packet Received
Packet Receive Errors
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
The parameters for testing this scenario are:
Number of hosts: 12
Participants in the system: <variable we increase>
Topics in the system: 1
Readers per topic: half of the participants
Writers per topic: half of the participants
QoS profiles used: DynamicProfile
Software information
RTI developed a testing framework specifically designed for discovery benchmarking. This framework was used to perform the tests detailed in this section. This framework is capable of distributing and executing the different DDS entities across the different machines available in RTI’s Performance and Discovery Lab. It also gathers information about discovery time as well as network usage and memory usage.
The Middleware version used to perform these tests is:
RTI Connext DDS 7.3.0 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)
Hardware information
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
QoS Used
1<?xml version="1.0"?>
2<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
3 xsi:noNamespaceSchemaLocation="https://community.rti.com/schema/7.2.0/rti_dds_qos_profiles.xsd">
4
5 <qos_library name="QosLibrary">
6
7 <qos_profile name="FactoryDefault" is_default_participant_factory_profile="true">
8 <participant_factory_qos>
9 <entity_factory>
10 <autoenable_created_entities>$(autoenable_created_entities)</autoenable_created_entities>
11 </entity_factory>
12 <!-- <logging>
13 <verbosity>WARNING</verbosity>
14 </logging> -->
15 </participant_factory_qos>
16 <participant_qos>
17 <discovery_config>
18 <builtin_discovery_plugins>$(qos_discovery_protocol)</builtin_discovery_plugins>
19 </discovery_config>
20 </participant_qos>
21 </qos_profile>
22
23 <qos_profile name="DynamicProfile_DefaultQoS" base_name="QosLibrary::FactoryDefault">
24 <participant_qos>
25 <!-- This does not affect performance, but it is needed for > 1000 participants -->
26 <wire_protocol>
27 <rtps_well_known_ports>
28 <domain_id_gain>500</domain_id_gain>
29 </rtps_well_known_ports>
30 </wire_protocol>
31 <!-- To make the test a bit more fair -->
32 <transport_builtin>
33 <mask>UDPv4</mask>
34 </transport_builtin>
35 <property>
36 <value>
37 <element>
38 <name>dds.transport.UDPv4.builtin.parent.allow_interfaces_list</name>
39 <value>$(interface_name)</value>
40 </element>
41 <element>
42 <name>dds.transport.UDPv4.builtin.parent.max_interface_count</name>
43 <value>2</value>
44 </element>
45 <element>
46 <name>dds.participant.property_validation_action</name>
47 <value>1</value>
48 </element>
49 <element>
50 <name>dds.transport.UDPv4.builtin.gather_detailed_statistics</name>
51 <value>$(gather_detailed_statistics)</value>
52 </element>
53 </value>
54 </property>
55 </participant_qos>
56 </qos_profile>
57
58 <qos_profile name="DynamicProfile" base_name="QosLibrary::DynamicProfile_DefaultQoS" is_default_qos="true">
59 <participant_qos>
60 <resource_limits>
61 <type_object_max_serialized_length>0</type_object_max_serialized_length>
62 <type_code_max_serialized_length>0</type_code_max_serialized_length>
63 <remote_writer_hash_buckets>625</remote_writer_hash_buckets>
64 <remote_reader_hash_buckets>625</remote_reader_hash_buckets>
65 <remote_participant_hash_buckets>625</remote_participant_hash_buckets>
66 <matching_reader_writer_pair_hash_buckets>625</matching_reader_writer_pair_hash_buckets>
67 <matching_writer_reader_pair_hash_buckets>625</matching_writer_reader_pair_hash_buckets>
68 </resource_limits>
69 <discovery_config>
70 <initial_participant_announcements>5</initial_participant_announcements>
71 <participant_liveliness_lease_duration>
72 <sec>500</sec>
73 <nanosec>0</nanosec>
74 </participant_liveliness_lease_duration>
75 <participant_liveliness_assert_period>
76 <sec>5</sec>
77 <nanosec>0</nanosec>
78 </participant_liveliness_assert_period>
79 <remote_participant_purge_kind>LIVELINESS_BASED_REMOTE_PARTICIPANT_PURGE</remote_participant_purge_kind>
80 <max_liveliness_loss_detection_period>
81 <sec>10</sec>
82 <nanosec>0</nanosec>
83 </max_liveliness_loss_detection_period>
84 </discovery_config>
85 <property>
86 <value>
87 <element>
88 <name>dds.transport.UDPv4.builtin.recv_socket_buffer_size</name>
89 <value>5048576</value>
90 </element>
91 </value>
92 </property>
93 </participant_qos>
94 </qos_profile>
95
96 <qos_profile name="Unicast_10Gbps_lab_snippet">
97 <participant_qos>
98 <discovery>
99 <initial_peers>
100 <element>$(initial-peers)</element>
101 </initial_peers>
102 <multicast_receive_addresses></multicast_receive_addresses>
103 </discovery>
104 </participant_qos>
105 </qos_profile>
106
107 <qos_profile name="Unicast_1Gbps_lab_snippet">
108 <participant_qos>
109 <discovery>
110 <initial_peers>
111 <element>$(initial-peers)</element>
112 </initial_peers>
113 <multicast_receive_addresses></multicast_receive_addresses>
114 </discovery>
115 </participant_qos>
116 </qos_profile>
117
118 <qos_profile name="DynamicProfile_unicast_cds" base_name="QosLibrary::DynamicProfile">
119 <participant_qos>
120 <transport_builtin>
121 <mask>UDPv4</mask>
122 </transport_builtin>
123 <discovery>
124 <initial_peers>
125 <element>rtps@udpv4://$(CDS_IP):7400</element>
126 </initial_peers>
127 <multicast_receive_addresses></multicast_receive_addresses>
128 </discovery>
129 </participant_qos>
130 </qos_profile>
131
132 <qos_profile name="StaticProfile" base_name="QosLibrary::DynamicProfile">
133 <participant_qos>
134 <discovery_config>
135 <builtin_discovery_plugins>SPDP</builtin_discovery_plugins>
136 </discovery_config>
137 <property>
138 <value>
139 <element>
140 <name>dds.discovery.endpoint.lbediscovery.library</name>
141 <value>rtilbedisc</value>
142 </element>
143 <element>
144 <name>dds.discovery.endpoint.lbediscovery.create_function</name>
145 <value>DDS_LBEDiscoveryPlugin_create</value>
146 </element>
147 <element>
148 <name>dds.discovery.endpoint.load_plugins</name>
149 <value>dds.discovery.endpoint.lbediscovery</value>
150 </element>
151 </value>
152 </property>
153 </participant_qos>
154 </qos_profile>
155
156 <!-- Security -->
157 <qos_profile name="DynamicProfileSecurity" base_name="QosLibrary::DynamicProfile">
158 <participant_qos>
159 <property>
160 <value>
161 <element>
162 <name>com.rti.serv.load_plugin</name>
163 <value>com.rti.serv.secure</value>
164 </element>
165 <element>
166 <name>com.rti.serv.secure.library</name>
167 <value>nddssecurity</value>
168 </element>
169 <element>
170 <name>com.rti.serv.secure.create_function</name>
171 <value>RTI_Security_PluginSuite_create</value>
172 </element>
173 <element>
174 <name>com.rti.serv.secure.authentication.ca_file</name>
175 <value>resources/secure/certAuthority/$(discovery_security_algo)/ca/$(discovery_security_algo)RootCaCert.pem</value>
176 </element>
177 <element>
178 <name>com.rti.serv.secure.authentication.private_key_file</name>
179 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Key.pem</value>
180 </element>
181 <element>
182 <name>com.rti.serv.secure.authentication.certificate_file</name>
183 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Cert.pem</value>
184 </element>
185 <element>
186 <name>com.rti.serv.secure.access_control.permissions_authority_file</name>
187 <value>resources/secure/certAuthority/$(discovery_permissions_authority_file_algo)/ca/$(discovery_permissions_authority_file_algo)RootCaCert.pem</value>
188 </element>
189 <element>
190 <name>com.rti.serv.secure.access_control.governance_file</name>
191 <value>resources/secure/certAuthority/$(discovery_security_algo)/governances/signed_governance_$(security_governance).xml</value>
192 </element>
193 <element>
194 <name>com.rti.serv.secure.access_control.permissions_file</name>
195 <value>resources/secure/certAuthority/$(discovery_security_algo)/signed_myPermissions.xml</value>
196 </element>
197 <element>
198 <name>com.rti.serv.secure.authentication.key_establishment_algorithm</name>
199 <value>auto</value>
200 </element>
201 <element>
202 <name>dds.participant.trust_plugins.authentication_timeout.sec</name>
203 <value>$(discovery_security_authentication_timeout)</value>
204 </element>
205 <element>
206 <name>dds.participant.trust_plugins.authentication_request_delay.sec</name>
207 <value>$(discovery_security_authentication_request_delay)</value>
208 </element>
209 <element>
210 <name>dds.participant.trust_plugins.authentication_request_timeout.sec</name>
211 <value>$(discovery_security_authentication_request_timeout)</value>
212 </element>
213 <element>
214 <name>com.rti.serv.secure.authentication.enable_custom_algorithms</name>
215 <value>true</value>
216 </element>
217 </value>
218 </property>
219 </participant_qos>
220 </qos_profile>
221
222 <!-- Security Unicast 10Gbps-->
223 <qos_profile name="DynamicProfileSecurity_unicast" base_name="QosLibrary::DynamicProfileSecurity">
224 <base_name>
225 <element>QosLibrary::Unicast_10Gbps_lab_snippet</element>
226 </base_name>
227 </qos_profile>
228
229 <!-- Security Unicast 10Gbps-->
230 <qos_profile name="DynamicProfileSecurity_unicast_enp1s0f0" base_name="QosLibrary::DynamicProfileSecurity_unicast">
231 </qos_profile>
232
233 <!-- Security Unicast 1Gbps -->
234 <qos_profile name="DynamicProfileSecurity_unicast_eno1" base_name="QosLibrary::DynamicProfileSecurity">
235 <base_name>
236 <element>QosLibrary::Unicast_1Gbps_lab_snippet</element>
237 </base_name>
238 </qos_profile>
239
240 <!-- Security HMAC ONLY -->
241 <qos_profile name="DynamicProfileSecurity_HMAC" base_name="QosLibrary::DynamicProfile">
242 <participant_qos>
243 <property>
244 <value>
245 <element>
246 <name>com.rti.serv.load_plugin</name>
247 <value>com.rti.serv.secure</value>
248 </element>
249 <element>
250 <name>com.rti.serv.secure.library</name>
251 <value>nddssecurity</value>
252 </element>
253 <element>
254 <name>com.rti.serv.secure.create_function</name>
255 <value>RTI_Security_PluginSuite_create</value>
256 </element>
257 <element>
258 <name>com.rti.serv.secure.hmac_only.enabled</name>
259 <value>1</value>
260 </element>
261 <element>
262 <name>com.rti.serv.secure.hmac_only.cryptography.key</name>
263 <value>str:SecretKey</value>
264 </element>
265 </value>
266 </property>
267 </participant_qos>
268 </qos_profile>
269
270 <!-- Security + PSK-->
271 <qos_profile name="DynamicProfileSecurity_PSK" base_name="QosLibrary::DynamicProfileSecurity">
272 <participant_qos>
273 <discovery_config>
274 <default_domain_announcement_period>
275 <sec>DURATION_INFINITE_SEC</sec>
276 <nanosec>DURATION_INFINITE_NSEC</nanosec>
277 </default_domain_announcement_period>
278 </discovery_config>
279 <property>
280 <value>
281 <element>
282 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
283 <!-- <value>str:1:SecretKey</value> -->
284 <value>data:,1:SecretKey</value>
285 </element>
286 </value>
287 </property>
288 </participant_qos>
289 </qos_profile>
290
291 <!-- LW Security + PSK -->
292 <qos_profile name="DynamicProfileLWS_PSK" base_name="QosLibrary::DynamicProfile">
293 <participant_qos>
294 <discovery_config>
295 <default_domain_announcement_period>
296 <sec>DURATION_INFINITE_SEC</sec>
297 <nanosec>DURATION_INFINITE_NSEC</nanosec>
298 </default_domain_announcement_period>
299 </discovery_config>
300 <property>
301 <value>
302 <element>
303 <name>com.rti.serv.load_plugin</name>
304 <value>com.rti.serv.secure</value>
305 </element>
306 <element>
307 <name>com.rti.serv.secure.library</name>
308 <value>nddslightweightsecurity</value>
309 </element>
310 <element>
311 <name>com.rti.serv.secure.create_function</name>
312 <value>RTI_SecurityLightweight_PluginSuite_create</value>
313 </element>
314 <element>
315 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
316 <!-- <value>str:1:SecretKey</value> -->
317 <value>data:,1:SecretKey</value>
318 </element>
319 <element>
320 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key_algorithm</name>
321 <value>$(lws_psk_algorithm)</value>
322 </element>
323 </value>
324 </property>
325 </participant_qos>
326 </qos_profile>
327
328 </qos_library>
329</dds>
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
In this particular multicast scenario, SPDP2’s network usage is higher than SPDP because repairs for the participant configuration packets are sent over unicast and these packets are sent from each participant to each remote participant, whereas a single SPDP multicast Participant announcement (DATA(P)) can be received via multicast by all participants.
Sent Bytes
Received Bytes
Sent Packets
Received Packets
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
In this particular multicast scenario, SPDP2’s network usage is higher than SPDP because repairs for the participant configuration packets are sent over unicast and these packets are sent from each participant to each remote participant, whereas a single SPDP multicast Participant announcement (DATA(P)) can be received via multicast by all participants.
Packet Sent
Packet Received
Packet Receive Errors
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
However, in this particular unicast scenario it is worth noticing that the new SPDP2 protocol can get to substancially better results for large numbers of participants. This is because the SPDP2 protocol’s bootstrap messages and liveliness messages are smaller than the participant announcement messages sent by SPDP. SPDP2 will only need to send the liveliness messages after the initial bootstrap message while SPDP will need to send the whole participant announcement messages (larger in size), this process has to be done by all the participant to every other discovered participant.
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
However, in this particular unicast scenario it is worth noticing that the new SPDP2 protocol can get to substancially better results for large numbers of participants. This is because the SPDP2 protocol’s bootstrap messages and liveliness messages are smaller than the participant announcement messages sent by SPDP. SPDP2 will only need to send the liveliness messages after the initial bootstrap message while SPDP will need to send the whole participant announcement messages (larger in size), this process has to be done by all the participant to every other discovered participant.
Sent Bytes
Received Bytes
Sent Packets
Received Packets
As mentioned in the note above, this is a flat configuration: A single LAN with one single endpoint per participant, one participant per application and a single topic across the system. Even though this configuration can be useful for benchmarking purposes, it is not recommended for real-life applications.
However, in this particular unicast scenario it is worth noticing that the new SPDP2 protocol can get to substancially better results for large numbers of participants. This is because the SPDP2 protocol’s bootstrap messages and liveliness messages are smaller than the participant announcement messages sent by SPDP. SPDP2 will only need to send the liveliness messages after the initial bootstrap message while SPDP will need to send the whole participant announcement messages (larger in size), this process has to be done by all the participant to every other discovered participant.
Packet Sent
Packet Received
Packet Receive Errors
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
The parameters for testing this scenario are:
Number of hosts: 12
Participants in the system: <variable we increase>
Topics in the system: 1
Readers per topic: half of the participants
Writers per topic: half of the participants
QoS profiles used: DynamicProfile
Software information
RTI developed a testing framework specifically designed for discovery benchmarking. This framework was used to perform the tests detailed in this section. This framework is capable of distributing and executing the different DDS entities across the different machines available in RTI’s Performance and Discovery Lab. It also gathers information about discovery time as well as network usage and memory usage.
The Middleware version used to perform these tests is:
RTI Connext DDS 7.3.0 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)
Hardware information
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
QoS Used
1<?xml version="1.0"?>
2<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
3 xsi:noNamespaceSchemaLocation="https://community.rti.com/schema/7.2.0/rti_dds_qos_profiles.xsd">
4
5 <qos_library name="QosLibrary">
6
7 <qos_profile name="FactoryDefault" is_default_participant_factory_profile="true">
8 <participant_factory_qos>
9 <entity_factory>
10 <autoenable_created_entities>$(autoenable_created_entities)</autoenable_created_entities>
11 </entity_factory>
12 <!-- <logging>
13 <verbosity>WARNING</verbosity>
14 </logging> -->
15 </participant_factory_qos>
16 <participant_qos>
17 <discovery_config>
18 <builtin_discovery_plugins>$(qos_discovery_protocol)</builtin_discovery_plugins>
19 </discovery_config>
20 </participant_qos>
21 </qos_profile>
22
23 <qos_profile name="DynamicProfile_DefaultQoS" base_name="QosLibrary::FactoryDefault">
24 <participant_qos>
25 <!-- This does not affect performance, but it is needed for > 1000 participants -->
26 <wire_protocol>
27 <rtps_well_known_ports>
28 <domain_id_gain>500</domain_id_gain>
29 </rtps_well_known_ports>
30 </wire_protocol>
31 <!-- To make the test a bit more fair -->
32 <transport_builtin>
33 <mask>UDPv4</mask>
34 </transport_builtin>
35 <property>
36 <value>
37 <element>
38 <name>dds.transport.UDPv4.builtin.parent.allow_interfaces_list</name>
39 <value>$(interface_name)</value>
40 </element>
41 <element>
42 <name>dds.transport.UDPv4.builtin.parent.max_interface_count</name>
43 <value>2</value>
44 </element>
45 <element>
46 <name>dds.participant.property_validation_action</name>
47 <value>1</value>
48 </element>
49 <element>
50 <name>dds.transport.UDPv4.builtin.gather_detailed_statistics</name>
51 <value>$(gather_detailed_statistics)</value>
52 </element>
53 </value>
54 </property>
55 </participant_qos>
56 </qos_profile>
57
58 <qos_profile name="DynamicProfile" base_name="QosLibrary::DynamicProfile_DefaultQoS" is_default_qos="true">
59 <participant_qos>
60 <resource_limits>
61 <type_object_max_serialized_length>0</type_object_max_serialized_length>
62 <type_code_max_serialized_length>0</type_code_max_serialized_length>
63 <remote_writer_hash_buckets>625</remote_writer_hash_buckets>
64 <remote_reader_hash_buckets>625</remote_reader_hash_buckets>
65 <remote_participant_hash_buckets>625</remote_participant_hash_buckets>
66 <matching_reader_writer_pair_hash_buckets>625</matching_reader_writer_pair_hash_buckets>
67 <matching_writer_reader_pair_hash_buckets>625</matching_writer_reader_pair_hash_buckets>
68 </resource_limits>
69 <discovery_config>
70 <initial_participant_announcements>5</initial_participant_announcements>
71 <participant_liveliness_lease_duration>
72 <sec>500</sec>
73 <nanosec>0</nanosec>
74 </participant_liveliness_lease_duration>
75 <participant_liveliness_assert_period>
76 <sec>5</sec>
77 <nanosec>0</nanosec>
78 </participant_liveliness_assert_period>
79 <remote_participant_purge_kind>LIVELINESS_BASED_REMOTE_PARTICIPANT_PURGE</remote_participant_purge_kind>
80 <max_liveliness_loss_detection_period>
81 <sec>10</sec>
82 <nanosec>0</nanosec>
83 </max_liveliness_loss_detection_period>
84 </discovery_config>
85 <property>
86 <value>
87 <element>
88 <name>dds.transport.UDPv4.builtin.recv_socket_buffer_size</name>
89 <value>5048576</value>
90 </element>
91 </value>
92 </property>
93 </participant_qos>
94 </qos_profile>
95
96 <qos_profile name="Unicast_10Gbps_lab_snippet">
97 <participant_qos>
98 <discovery>
99 <initial_peers>
100 <element>$(initial-peers)</element>
101 </initial_peers>
102 <multicast_receive_addresses></multicast_receive_addresses>
103 </discovery>
104 </participant_qos>
105 </qos_profile>
106
107 <qos_profile name="Unicast_1Gbps_lab_snippet">
108 <participant_qos>
109 <discovery>
110 <initial_peers>
111 <element>$(initial-peers)</element>
112 </initial_peers>
113 <multicast_receive_addresses></multicast_receive_addresses>
114 </discovery>
115 </participant_qos>
116 </qos_profile>
117
118 <qos_profile name="DynamicProfile_unicast_cds" base_name="QosLibrary::DynamicProfile">
119 <participant_qos>
120 <transport_builtin>
121 <mask>UDPv4</mask>
122 </transport_builtin>
123 <discovery>
124 <initial_peers>
125 <element>rtps@udpv4://$(CDS_IP):7400</element>
126 </initial_peers>
127 <multicast_receive_addresses></multicast_receive_addresses>
128 </discovery>
129 </participant_qos>
130 </qos_profile>
131
132 <qos_profile name="StaticProfile" base_name="QosLibrary::DynamicProfile">
133 <participant_qos>
134 <discovery_config>
135 <builtin_discovery_plugins>SPDP</builtin_discovery_plugins>
136 </discovery_config>
137 <property>
138 <value>
139 <element>
140 <name>dds.discovery.endpoint.lbediscovery.library</name>
141 <value>rtilbedisc</value>
142 </element>
143 <element>
144 <name>dds.discovery.endpoint.lbediscovery.create_function</name>
145 <value>DDS_LBEDiscoveryPlugin_create</value>
146 </element>
147 <element>
148 <name>dds.discovery.endpoint.load_plugins</name>
149 <value>dds.discovery.endpoint.lbediscovery</value>
150 </element>
151 </value>
152 </property>
153 </participant_qos>
154 </qos_profile>
155
156 <!-- Security -->
157 <qos_profile name="DynamicProfileSecurity" base_name="QosLibrary::DynamicProfile">
158 <participant_qos>
159 <property>
160 <value>
161 <element>
162 <name>com.rti.serv.load_plugin</name>
163 <value>com.rti.serv.secure</value>
164 </element>
165 <element>
166 <name>com.rti.serv.secure.library</name>
167 <value>nddssecurity</value>
168 </element>
169 <element>
170 <name>com.rti.serv.secure.create_function</name>
171 <value>RTI_Security_PluginSuite_create</value>
172 </element>
173 <element>
174 <name>com.rti.serv.secure.authentication.ca_file</name>
175 <value>resources/secure/certAuthority/$(discovery_security_algo)/ca/$(discovery_security_algo)RootCaCert.pem</value>
176 </element>
177 <element>
178 <name>com.rti.serv.secure.authentication.private_key_file</name>
179 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Key.pem</value>
180 </element>
181 <element>
182 <name>com.rti.serv.secure.authentication.certificate_file</name>
183 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Cert.pem</value>
184 </element>
185 <element>
186 <name>com.rti.serv.secure.access_control.permissions_authority_file</name>
187 <value>resources/secure/certAuthority/$(discovery_permissions_authority_file_algo)/ca/$(discovery_permissions_authority_file_algo)RootCaCert.pem</value>
188 </element>
189 <element>
190 <name>com.rti.serv.secure.access_control.governance_file</name>
191 <value>resources/secure/certAuthority/$(discovery_security_algo)/governances/signed_governance_$(security_governance).xml</value>
192 </element>
193 <element>
194 <name>com.rti.serv.secure.access_control.permissions_file</name>
195 <value>resources/secure/certAuthority/$(discovery_security_algo)/signed_myPermissions.xml</value>
196 </element>
197 <element>
198 <name>com.rti.serv.secure.authentication.key_establishment_algorithm</name>
199 <value>auto</value>
200 </element>
201 <element>
202 <name>dds.participant.trust_plugins.authentication_timeout.sec</name>
203 <value>$(discovery_security_authentication_timeout)</value>
204 </element>
205 <element>
206 <name>dds.participant.trust_plugins.authentication_request_delay.sec</name>
207 <value>$(discovery_security_authentication_request_delay)</value>
208 </element>
209 <element>
210 <name>dds.participant.trust_plugins.authentication_request_timeout.sec</name>
211 <value>$(discovery_security_authentication_request_timeout)</value>
212 </element>
213 <element>
214 <name>com.rti.serv.secure.authentication.enable_custom_algorithms</name>
215 <value>true</value>
216 </element>
217 </value>
218 </property>
219 </participant_qos>
220 </qos_profile>
221
222 <!-- Security Unicast 10Gbps-->
223 <qos_profile name="DynamicProfileSecurity_unicast" base_name="QosLibrary::DynamicProfileSecurity">
224 <base_name>
225 <element>QosLibrary::Unicast_10Gbps_lab_snippet</element>
226 </base_name>
227 </qos_profile>
228
229 <!-- Security Unicast 10Gbps-->
230 <qos_profile name="DynamicProfileSecurity_unicast_enp1s0f0" base_name="QosLibrary::DynamicProfileSecurity_unicast">
231 </qos_profile>
232
233 <!-- Security Unicast 1Gbps -->
234 <qos_profile name="DynamicProfileSecurity_unicast_eno1" base_name="QosLibrary::DynamicProfileSecurity">
235 <base_name>
236 <element>QosLibrary::Unicast_1Gbps_lab_snippet</element>
237 </base_name>
238 </qos_profile>
239
240 <!-- Security HMAC ONLY -->
241 <qos_profile name="DynamicProfileSecurity_HMAC" base_name="QosLibrary::DynamicProfile">
242 <participant_qos>
243 <property>
244 <value>
245 <element>
246 <name>com.rti.serv.load_plugin</name>
247 <value>com.rti.serv.secure</value>
248 </element>
249 <element>
250 <name>com.rti.serv.secure.library</name>
251 <value>nddssecurity</value>
252 </element>
253 <element>
254 <name>com.rti.serv.secure.create_function</name>
255 <value>RTI_Security_PluginSuite_create</value>
256 </element>
257 <element>
258 <name>com.rti.serv.secure.hmac_only.enabled</name>
259 <value>1</value>
260 </element>
261 <element>
262 <name>com.rti.serv.secure.hmac_only.cryptography.key</name>
263 <value>str:SecretKey</value>
264 </element>
265 </value>
266 </property>
267 </participant_qos>
268 </qos_profile>
269
270 <!-- Security + PSK-->
271 <qos_profile name="DynamicProfileSecurity_PSK" base_name="QosLibrary::DynamicProfileSecurity">
272 <participant_qos>
273 <discovery_config>
274 <default_domain_announcement_period>
275 <sec>DURATION_INFINITE_SEC</sec>
276 <nanosec>DURATION_INFINITE_NSEC</nanosec>
277 </default_domain_announcement_period>
278 </discovery_config>
279 <property>
280 <value>
281 <element>
282 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
283 <!-- <value>str:1:SecretKey</value> -->
284 <value>data:,1:SecretKey</value>
285 </element>
286 </value>
287 </property>
288 </participant_qos>
289 </qos_profile>
290
291 <!-- LW Security + PSK -->
292 <qos_profile name="DynamicProfileLWS_PSK" base_name="QosLibrary::DynamicProfile">
293 <participant_qos>
294 <discovery_config>
295 <default_domain_announcement_period>
296 <sec>DURATION_INFINITE_SEC</sec>
297 <nanosec>DURATION_INFINITE_NSEC</nanosec>
298 </default_domain_announcement_period>
299 </discovery_config>
300 <property>
301 <value>
302 <element>
303 <name>com.rti.serv.load_plugin</name>
304 <value>com.rti.serv.secure</value>
305 </element>
306 <element>
307 <name>com.rti.serv.secure.library</name>
308 <value>nddslightweightsecurity</value>
309 </element>
310 <element>
311 <name>com.rti.serv.secure.create_function</name>
312 <value>RTI_SecurityLightweight_PluginSuite_create</value>
313 </element>
314 <element>
315 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
316 <!-- <value>str:1:SecretKey</value> -->
317 <value>data:,1:SecretKey</value>
318 </element>
319 <element>
320 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key_algorithm</name>
321 <value>$(lws_psk_algorithm)</value>
322 </element>
323 </value>
324 </property>
325 </participant_qos>
326 </qos_profile>
327
328 </qos_library>
329</dds>
2.1.2.2. Limited Bandwidth Plugin (LBED)
The idea and methodology for this test are similar to the one explained above, except now we use the RTI Limited Bandwidth Plugins to see how much we can decrease bandwidth utilization and speed up the discovery process.
This test requires additional configuration files. For more information, see RTI Limited Bandwidth Plugin User’s Manual.
Note
This scenario is likely not the optimal design solution for a real-life architecture. The purpose of the test is to demonstrate how powerful the Connext discovery protocol is in a flat configuration.
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Sent
Received
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
The parameters for testing this scenario are:
Number of hosts: 12
Participants in the system: <variable we increase>
Topics in the system: 1
Readers per topic: half of the participants
Writers per topic: half of the participants
QoS profiles used: StaticProfile and DynamicProfile
Software information
RTI developed a testing framework specifically designed for discovery benchmarking. This framework was used to perform the tests detailed in this section. This framework is capable of distributing and executing the different DDS entities across the different machines available in RTI’s Performance and Discovery Lab. It also gathers information about discovery time as well as network usage and memory usage.
The Middleware version used to perform these tests is:
RTI Connext DDS 7.3.0 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)
Hardware information
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
QoS Used
1<?xml version="1.0"?>
2<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
3 xsi:noNamespaceSchemaLocation="https://community.rti.com/schema/7.2.0/rti_dds_qos_profiles.xsd">
4
5 <qos_library name="QosLibrary">
6
7 <qos_profile name="FactoryDefault" is_default_participant_factory_profile="true">
8 <participant_factory_qos>
9 <entity_factory>
10 <autoenable_created_entities>$(autoenable_created_entities)</autoenable_created_entities>
11 </entity_factory>
12 <!-- <logging>
13 <verbosity>WARNING</verbosity>
14 </logging> -->
15 </participant_factory_qos>
16 <participant_qos>
17 <discovery_config>
18 <builtin_discovery_plugins>$(qos_discovery_protocol)</builtin_discovery_plugins>
19 </discovery_config>
20 </participant_qos>
21 </qos_profile>
22
23 <qos_profile name="DynamicProfile_DefaultQoS" base_name="QosLibrary::FactoryDefault">
24 <participant_qos>
25 <!-- This does not affect performance, but it is needed for > 1000 participants -->
26 <wire_protocol>
27 <rtps_well_known_ports>
28 <domain_id_gain>500</domain_id_gain>
29 </rtps_well_known_ports>
30 </wire_protocol>
31 <!-- To make the test a bit more fair -->
32 <transport_builtin>
33 <mask>UDPv4</mask>
34 </transport_builtin>
35 <property>
36 <value>
37 <element>
38 <name>dds.transport.UDPv4.builtin.parent.allow_interfaces_list</name>
39 <value>$(interface_name)</value>
40 </element>
41 <element>
42 <name>dds.transport.UDPv4.builtin.parent.max_interface_count</name>
43 <value>2</value>
44 </element>
45 <element>
46 <name>dds.participant.property_validation_action</name>
47 <value>1</value>
48 </element>
49 <element>
50 <name>dds.transport.UDPv4.builtin.gather_detailed_statistics</name>
51 <value>$(gather_detailed_statistics)</value>
52 </element>
53 </value>
54 </property>
55 </participant_qos>
56 </qos_profile>
57
58 <qos_profile name="DynamicProfile" base_name="QosLibrary::DynamicProfile_DefaultQoS" is_default_qos="true">
59 <participant_qos>
60 <resource_limits>
61 <type_object_max_serialized_length>0</type_object_max_serialized_length>
62 <type_code_max_serialized_length>0</type_code_max_serialized_length>
63 <remote_writer_hash_buckets>625</remote_writer_hash_buckets>
64 <remote_reader_hash_buckets>625</remote_reader_hash_buckets>
65 <remote_participant_hash_buckets>625</remote_participant_hash_buckets>
66 <matching_reader_writer_pair_hash_buckets>625</matching_reader_writer_pair_hash_buckets>
67 <matching_writer_reader_pair_hash_buckets>625</matching_writer_reader_pair_hash_buckets>
68 </resource_limits>
69 <discovery_config>
70 <initial_participant_announcements>5</initial_participant_announcements>
71 <participant_liveliness_lease_duration>
72 <sec>500</sec>
73 <nanosec>0</nanosec>
74 </participant_liveliness_lease_duration>
75 <participant_liveliness_assert_period>
76 <sec>5</sec>
77 <nanosec>0</nanosec>
78 </participant_liveliness_assert_period>
79 <remote_participant_purge_kind>LIVELINESS_BASED_REMOTE_PARTICIPANT_PURGE</remote_participant_purge_kind>
80 <max_liveliness_loss_detection_period>
81 <sec>10</sec>
82 <nanosec>0</nanosec>
83 </max_liveliness_loss_detection_period>
84 </discovery_config>
85 <property>
86 <value>
87 <element>
88 <name>dds.transport.UDPv4.builtin.recv_socket_buffer_size</name>
89 <value>5048576</value>
90 </element>
91 </value>
92 </property>
93 </participant_qos>
94 </qos_profile>
95
96 <qos_profile name="Unicast_10Gbps_lab_snippet">
97 <participant_qos>
98 <discovery>
99 <initial_peers>
100 <element>$(initial-peers)</element>
101 </initial_peers>
102 <multicast_receive_addresses></multicast_receive_addresses>
103 </discovery>
104 </participant_qos>
105 </qos_profile>
106
107 <qos_profile name="Unicast_1Gbps_lab_snippet">
108 <participant_qos>
109 <discovery>
110 <initial_peers>
111 <element>$(initial-peers)</element>
112 </initial_peers>
113 <multicast_receive_addresses></multicast_receive_addresses>
114 </discovery>
115 </participant_qos>
116 </qos_profile>
117
118 <qos_profile name="DynamicProfile_unicast_cds" base_name="QosLibrary::DynamicProfile">
119 <participant_qos>
120 <transport_builtin>
121 <mask>UDPv4</mask>
122 </transport_builtin>
123 <discovery>
124 <initial_peers>
125 <element>rtps@udpv4://$(CDS_IP):7400</element>
126 </initial_peers>
127 <multicast_receive_addresses></multicast_receive_addresses>
128 </discovery>
129 </participant_qos>
130 </qos_profile>
131
132 <qos_profile name="StaticProfile" base_name="QosLibrary::DynamicProfile">
133 <participant_qos>
134 <discovery_config>
135 <builtin_discovery_plugins>SPDP</builtin_discovery_plugins>
136 </discovery_config>
137 <property>
138 <value>
139 <element>
140 <name>dds.discovery.endpoint.lbediscovery.library</name>
141 <value>rtilbedisc</value>
142 </element>
143 <element>
144 <name>dds.discovery.endpoint.lbediscovery.create_function</name>
145 <value>DDS_LBEDiscoveryPlugin_create</value>
146 </element>
147 <element>
148 <name>dds.discovery.endpoint.load_plugins</name>
149 <value>dds.discovery.endpoint.lbediscovery</value>
150 </element>
151 </value>
152 </property>
153 </participant_qos>
154 </qos_profile>
155
156 <!-- Security -->
157 <qos_profile name="DynamicProfileSecurity" base_name="QosLibrary::DynamicProfile">
158 <participant_qos>
159 <property>
160 <value>
161 <element>
162 <name>com.rti.serv.load_plugin</name>
163 <value>com.rti.serv.secure</value>
164 </element>
165 <element>
166 <name>com.rti.serv.secure.library</name>
167 <value>nddssecurity</value>
168 </element>
169 <element>
170 <name>com.rti.serv.secure.create_function</name>
171 <value>RTI_Security_PluginSuite_create</value>
172 </element>
173 <element>
174 <name>com.rti.serv.secure.authentication.ca_file</name>
175 <value>resources/secure/certAuthority/$(discovery_security_algo)/ca/$(discovery_security_algo)RootCaCert.pem</value>
176 </element>
177 <element>
178 <name>com.rti.serv.secure.authentication.private_key_file</name>
179 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Key.pem</value>
180 </element>
181 <element>
182 <name>com.rti.serv.secure.authentication.certificate_file</name>
183 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Cert.pem</value>
184 </element>
185 <element>
186 <name>com.rti.serv.secure.access_control.permissions_authority_file</name>
187 <value>resources/secure/certAuthority/$(discovery_permissions_authority_file_algo)/ca/$(discovery_permissions_authority_file_algo)RootCaCert.pem</value>
188 </element>
189 <element>
190 <name>com.rti.serv.secure.access_control.governance_file</name>
191 <value>resources/secure/certAuthority/$(discovery_security_algo)/governances/signed_governance_$(security_governance).xml</value>
192 </element>
193 <element>
194 <name>com.rti.serv.secure.access_control.permissions_file</name>
195 <value>resources/secure/certAuthority/$(discovery_security_algo)/signed_myPermissions.xml</value>
196 </element>
197 <element>
198 <name>com.rti.serv.secure.authentication.key_establishment_algorithm</name>
199 <value>auto</value>
200 </element>
201 <element>
202 <name>dds.participant.trust_plugins.authentication_timeout.sec</name>
203 <value>$(discovery_security_authentication_timeout)</value>
204 </element>
205 <element>
206 <name>dds.participant.trust_plugins.authentication_request_delay.sec</name>
207 <value>$(discovery_security_authentication_request_delay)</value>
208 </element>
209 <element>
210 <name>dds.participant.trust_plugins.authentication_request_timeout.sec</name>
211 <value>$(discovery_security_authentication_request_timeout)</value>
212 </element>
213 <element>
214 <name>com.rti.serv.secure.authentication.enable_custom_algorithms</name>
215 <value>true</value>
216 </element>
217 </value>
218 </property>
219 </participant_qos>
220 </qos_profile>
221
222 <!-- Security Unicast 10Gbps-->
223 <qos_profile name="DynamicProfileSecurity_unicast" base_name="QosLibrary::DynamicProfileSecurity">
224 <base_name>
225 <element>QosLibrary::Unicast_10Gbps_lab_snippet</element>
226 </base_name>
227 </qos_profile>
228
229 <!-- Security Unicast 10Gbps-->
230 <qos_profile name="DynamicProfileSecurity_unicast_enp1s0f0" base_name="QosLibrary::DynamicProfileSecurity_unicast">
231 </qos_profile>
232
233 <!-- Security Unicast 1Gbps -->
234 <qos_profile name="DynamicProfileSecurity_unicast_eno1" base_name="QosLibrary::DynamicProfileSecurity">
235 <base_name>
236 <element>QosLibrary::Unicast_1Gbps_lab_snippet</element>
237 </base_name>
238 </qos_profile>
239
240 <!-- Security HMAC ONLY -->
241 <qos_profile name="DynamicProfileSecurity_HMAC" base_name="QosLibrary::DynamicProfile">
242 <participant_qos>
243 <property>
244 <value>
245 <element>
246 <name>com.rti.serv.load_plugin</name>
247 <value>com.rti.serv.secure</value>
248 </element>
249 <element>
250 <name>com.rti.serv.secure.library</name>
251 <value>nddssecurity</value>
252 </element>
253 <element>
254 <name>com.rti.serv.secure.create_function</name>
255 <value>RTI_Security_PluginSuite_create</value>
256 </element>
257 <element>
258 <name>com.rti.serv.secure.hmac_only.enabled</name>
259 <value>1</value>
260 </element>
261 <element>
262 <name>com.rti.serv.secure.hmac_only.cryptography.key</name>
263 <value>str:SecretKey</value>
264 </element>
265 </value>
266 </property>
267 </participant_qos>
268 </qos_profile>
269
270 <!-- Security + PSK-->
271 <qos_profile name="DynamicProfileSecurity_PSK" base_name="QosLibrary::DynamicProfileSecurity">
272 <participant_qos>
273 <discovery_config>
274 <default_domain_announcement_period>
275 <sec>DURATION_INFINITE_SEC</sec>
276 <nanosec>DURATION_INFINITE_NSEC</nanosec>
277 </default_domain_announcement_period>
278 </discovery_config>
279 <property>
280 <value>
281 <element>
282 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
283 <!-- <value>str:1:SecretKey</value> -->
284 <value>data:,1:SecretKey</value>
285 </element>
286 </value>
287 </property>
288 </participant_qos>
289 </qos_profile>
290
291 <!-- LW Security + PSK -->
292 <qos_profile name="DynamicProfileLWS_PSK" base_name="QosLibrary::DynamicProfile">
293 <participant_qos>
294 <discovery_config>
295 <default_domain_announcement_period>
296 <sec>DURATION_INFINITE_SEC</sec>
297 <nanosec>DURATION_INFINITE_NSEC</nanosec>
298 </default_domain_announcement_period>
299 </discovery_config>
300 <property>
301 <value>
302 <element>
303 <name>com.rti.serv.load_plugin</name>
304 <value>com.rti.serv.secure</value>
305 </element>
306 <element>
307 <name>com.rti.serv.secure.library</name>
308 <value>nddslightweightsecurity</value>
309 </element>
310 <element>
311 <name>com.rti.serv.secure.create_function</name>
312 <value>RTI_SecurityLightweight_PluginSuite_create</value>
313 </element>
314 <element>
315 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
316 <!-- <value>str:1:SecretKey</value> -->
317 <value>data:,1:SecretKey</value>
318 </element>
319 <element>
320 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key_algorithm</name>
321 <value>$(lws_psk_algorithm)</value>
322 </element>
323 </value>
324 </property>
325 </participant_qos>
326 </qos_profile>
327
328 </qos_library>
329</dds>
2.1.2.3. Participant Partitions
In this scenario, we test a new feature introduced in RTI Connext 7.0.0: DomainParticipant partitions. For more information, see PARTITION QosPolicy, in the RTI Connext Core Libraries User’s Manual.
Note
This scenario is likely not the optimal design solution for a real-life architecture. The purpose of the test is to demonstrate how powerful the Connext discovery protocol is in a flat configuration.
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Sent
Received
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Sent
Received
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Sent
Received
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Sent
Received
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
The parameters for testing this scenario are:
Number of hosts: 12
Participants in the system: <variable we increase>
Topics in the system: 1
Readers per topic: half of the participants
Writers per topic: half of the participants
QoS profiles used: DynamicProfile_unicast, DynamicProfile
Software information
RTI developed a testing framework specifically designed for discovery benchmarking. This framework was used to perform the tests detailed in this section. This framework is capable of distributing and executing the different DDS entities across the different machines available in RTI’s Performance and Discovery Lab. It also gathers information about discovery time as well as network usage and memory usage.
The Middleware version used to perform these tests is:
RTI Connext DDS 7.3.0 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)
Hardware information
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
QoS Used
1<?xml version="1.0"?>
2<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
3 xsi:noNamespaceSchemaLocation="https://community.rti.com/schema/7.2.0/rti_dds_qos_profiles.xsd">
4
5 <qos_library name="QosLibrary">
6
7 <qos_profile name="FactoryDefault" is_default_participant_factory_profile="true">
8 <participant_factory_qos>
9 <entity_factory>
10 <autoenable_created_entities>$(autoenable_created_entities)</autoenable_created_entities>
11 </entity_factory>
12 <!-- <logging>
13 <verbosity>WARNING</verbosity>
14 </logging> -->
15 </participant_factory_qos>
16 <participant_qos>
17 <discovery_config>
18 <builtin_discovery_plugins>$(qos_discovery_protocol)</builtin_discovery_plugins>
19 </discovery_config>
20 </participant_qos>
21 </qos_profile>
22
23 <qos_profile name="DynamicProfile_DefaultQoS" base_name="QosLibrary::FactoryDefault">
24 <participant_qos>
25 <!-- This does not affect performance, but it is needed for > 1000 participants -->
26 <wire_protocol>
27 <rtps_well_known_ports>
28 <domain_id_gain>500</domain_id_gain>
29 </rtps_well_known_ports>
30 </wire_protocol>
31 <!-- To make the test a bit more fair -->
32 <transport_builtin>
33 <mask>UDPv4</mask>
34 </transport_builtin>
35 <property>
36 <value>
37 <element>
38 <name>dds.transport.UDPv4.builtin.parent.allow_interfaces_list</name>
39 <value>$(interface_name)</value>
40 </element>
41 <element>
42 <name>dds.transport.UDPv4.builtin.parent.max_interface_count</name>
43 <value>2</value>
44 </element>
45 <element>
46 <name>dds.participant.property_validation_action</name>
47 <value>1</value>
48 </element>
49 <element>
50 <name>dds.transport.UDPv4.builtin.gather_detailed_statistics</name>
51 <value>$(gather_detailed_statistics)</value>
52 </element>
53 </value>
54 </property>
55 </participant_qos>
56 </qos_profile>
57
58 <qos_profile name="DynamicProfile" base_name="QosLibrary::DynamicProfile_DefaultQoS" is_default_qos="true">
59 <participant_qos>
60 <resource_limits>
61 <type_object_max_serialized_length>0</type_object_max_serialized_length>
62 <type_code_max_serialized_length>0</type_code_max_serialized_length>
63 <remote_writer_hash_buckets>625</remote_writer_hash_buckets>
64 <remote_reader_hash_buckets>625</remote_reader_hash_buckets>
65 <remote_participant_hash_buckets>625</remote_participant_hash_buckets>
66 <matching_reader_writer_pair_hash_buckets>625</matching_reader_writer_pair_hash_buckets>
67 <matching_writer_reader_pair_hash_buckets>625</matching_writer_reader_pair_hash_buckets>
68 </resource_limits>
69 <discovery_config>
70 <initial_participant_announcements>5</initial_participant_announcements>
71 <participant_liveliness_lease_duration>
72 <sec>500</sec>
73 <nanosec>0</nanosec>
74 </participant_liveliness_lease_duration>
75 <participant_liveliness_assert_period>
76 <sec>5</sec>
77 <nanosec>0</nanosec>
78 </participant_liveliness_assert_period>
79 <remote_participant_purge_kind>LIVELINESS_BASED_REMOTE_PARTICIPANT_PURGE</remote_participant_purge_kind>
80 <max_liveliness_loss_detection_period>
81 <sec>10</sec>
82 <nanosec>0</nanosec>
83 </max_liveliness_loss_detection_period>
84 </discovery_config>
85 <property>
86 <value>
87 <element>
88 <name>dds.transport.UDPv4.builtin.recv_socket_buffer_size</name>
89 <value>5048576</value>
90 </element>
91 </value>
92 </property>
93 </participant_qos>
94 </qos_profile>
95
96 <qos_profile name="Unicast_10Gbps_lab_snippet">
97 <participant_qos>
98 <discovery>
99 <initial_peers>
100 <element>$(initial-peers)</element>
101 </initial_peers>
102 <multicast_receive_addresses></multicast_receive_addresses>
103 </discovery>
104 </participant_qos>
105 </qos_profile>
106
107 <qos_profile name="Unicast_1Gbps_lab_snippet">
108 <participant_qos>
109 <discovery>
110 <initial_peers>
111 <element>$(initial-peers)</element>
112 </initial_peers>
113 <multicast_receive_addresses></multicast_receive_addresses>
114 </discovery>
115 </participant_qos>
116 </qos_profile>
117
118 <qos_profile name="DynamicProfile_unicast_cds" base_name="QosLibrary::DynamicProfile">
119 <participant_qos>
120 <transport_builtin>
121 <mask>UDPv4</mask>
122 </transport_builtin>
123 <discovery>
124 <initial_peers>
125 <element>rtps@udpv4://$(CDS_IP):7400</element>
126 </initial_peers>
127 <multicast_receive_addresses></multicast_receive_addresses>
128 </discovery>
129 </participant_qos>
130 </qos_profile>
131
132 <qos_profile name="StaticProfile" base_name="QosLibrary::DynamicProfile">
133 <participant_qos>
134 <discovery_config>
135 <builtin_discovery_plugins>SPDP</builtin_discovery_plugins>
136 </discovery_config>
137 <property>
138 <value>
139 <element>
140 <name>dds.discovery.endpoint.lbediscovery.library</name>
141 <value>rtilbedisc</value>
142 </element>
143 <element>
144 <name>dds.discovery.endpoint.lbediscovery.create_function</name>
145 <value>DDS_LBEDiscoveryPlugin_create</value>
146 </element>
147 <element>
148 <name>dds.discovery.endpoint.load_plugins</name>
149 <value>dds.discovery.endpoint.lbediscovery</value>
150 </element>
151 </value>
152 </property>
153 </participant_qos>
154 </qos_profile>
155
156 <!-- Security -->
157 <qos_profile name="DynamicProfileSecurity" base_name="QosLibrary::DynamicProfile">
158 <participant_qos>
159 <property>
160 <value>
161 <element>
162 <name>com.rti.serv.load_plugin</name>
163 <value>com.rti.serv.secure</value>
164 </element>
165 <element>
166 <name>com.rti.serv.secure.library</name>
167 <value>nddssecurity</value>
168 </element>
169 <element>
170 <name>com.rti.serv.secure.create_function</name>
171 <value>RTI_Security_PluginSuite_create</value>
172 </element>
173 <element>
174 <name>com.rti.serv.secure.authentication.ca_file</name>
175 <value>resources/secure/certAuthority/$(discovery_security_algo)/ca/$(discovery_security_algo)RootCaCert.pem</value>
176 </element>
177 <element>
178 <name>com.rti.serv.secure.authentication.private_key_file</name>
179 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Key.pem</value>
180 </element>
181 <element>
182 <name>com.rti.serv.secure.authentication.certificate_file</name>
183 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Cert.pem</value>
184 </element>
185 <element>
186 <name>com.rti.serv.secure.access_control.permissions_authority_file</name>
187 <value>resources/secure/certAuthority/$(discovery_permissions_authority_file_algo)/ca/$(discovery_permissions_authority_file_algo)RootCaCert.pem</value>
188 </element>
189 <element>
190 <name>com.rti.serv.secure.access_control.governance_file</name>
191 <value>resources/secure/certAuthority/$(discovery_security_algo)/governances/signed_governance_$(security_governance).xml</value>
192 </element>
193 <element>
194 <name>com.rti.serv.secure.access_control.permissions_file</name>
195 <value>resources/secure/certAuthority/$(discovery_security_algo)/signed_myPermissions.xml</value>
196 </element>
197 <element>
198 <name>com.rti.serv.secure.authentication.key_establishment_algorithm</name>
199 <value>auto</value>
200 </element>
201 <element>
202 <name>dds.participant.trust_plugins.authentication_timeout.sec</name>
203 <value>$(discovery_security_authentication_timeout)</value>
204 </element>
205 <element>
206 <name>dds.participant.trust_plugins.authentication_request_delay.sec</name>
207 <value>$(discovery_security_authentication_request_delay)</value>
208 </element>
209 <element>
210 <name>dds.participant.trust_plugins.authentication_request_timeout.sec</name>
211 <value>$(discovery_security_authentication_request_timeout)</value>
212 </element>
213 <element>
214 <name>com.rti.serv.secure.authentication.enable_custom_algorithms</name>
215 <value>true</value>
216 </element>
217 </value>
218 </property>
219 </participant_qos>
220 </qos_profile>
221
222 <!-- Security Unicast 10Gbps-->
223 <qos_profile name="DynamicProfileSecurity_unicast" base_name="QosLibrary::DynamicProfileSecurity">
224 <base_name>
225 <element>QosLibrary::Unicast_10Gbps_lab_snippet</element>
226 </base_name>
227 </qos_profile>
228
229 <!-- Security Unicast 10Gbps-->
230 <qos_profile name="DynamicProfileSecurity_unicast_enp1s0f0" base_name="QosLibrary::DynamicProfileSecurity_unicast">
231 </qos_profile>
232
233 <!-- Security Unicast 1Gbps -->
234 <qos_profile name="DynamicProfileSecurity_unicast_eno1" base_name="QosLibrary::DynamicProfileSecurity">
235 <base_name>
236 <element>QosLibrary::Unicast_1Gbps_lab_snippet</element>
237 </base_name>
238 </qos_profile>
239
240 <!-- Security HMAC ONLY -->
241 <qos_profile name="DynamicProfileSecurity_HMAC" base_name="QosLibrary::DynamicProfile">
242 <participant_qos>
243 <property>
244 <value>
245 <element>
246 <name>com.rti.serv.load_plugin</name>
247 <value>com.rti.serv.secure</value>
248 </element>
249 <element>
250 <name>com.rti.serv.secure.library</name>
251 <value>nddssecurity</value>
252 </element>
253 <element>
254 <name>com.rti.serv.secure.create_function</name>
255 <value>RTI_Security_PluginSuite_create</value>
256 </element>
257 <element>
258 <name>com.rti.serv.secure.hmac_only.enabled</name>
259 <value>1</value>
260 </element>
261 <element>
262 <name>com.rti.serv.secure.hmac_only.cryptography.key</name>
263 <value>str:SecretKey</value>
264 </element>
265 </value>
266 </property>
267 </participant_qos>
268 </qos_profile>
269
270 <!-- Security + PSK-->
271 <qos_profile name="DynamicProfileSecurity_PSK" base_name="QosLibrary::DynamicProfileSecurity">
272 <participant_qos>
273 <discovery_config>
274 <default_domain_announcement_period>
275 <sec>DURATION_INFINITE_SEC</sec>
276 <nanosec>DURATION_INFINITE_NSEC</nanosec>
277 </default_domain_announcement_period>
278 </discovery_config>
279 <property>
280 <value>
281 <element>
282 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
283 <!-- <value>str:1:SecretKey</value> -->
284 <value>data:,1:SecretKey</value>
285 </element>
286 </value>
287 </property>
288 </participant_qos>
289 </qos_profile>
290
291 <!-- LW Security + PSK -->
292 <qos_profile name="DynamicProfileLWS_PSK" base_name="QosLibrary::DynamicProfile">
293 <participant_qos>
294 <discovery_config>
295 <default_domain_announcement_period>
296 <sec>DURATION_INFINITE_SEC</sec>
297 <nanosec>DURATION_INFINITE_NSEC</nanosec>
298 </default_domain_announcement_period>
299 </discovery_config>
300 <property>
301 <value>
302 <element>
303 <name>com.rti.serv.load_plugin</name>
304 <value>com.rti.serv.secure</value>
305 </element>
306 <element>
307 <name>com.rti.serv.secure.library</name>
308 <value>nddslightweightsecurity</value>
309 </element>
310 <element>
311 <name>com.rti.serv.secure.create_function</name>
312 <value>RTI_SecurityLightweight_PluginSuite_create</value>
313 </element>
314 <element>
315 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
316 <!-- <value>str:1:SecretKey</value> -->
317 <value>data:,1:SecretKey</value>
318 </element>
319 <element>
320 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key_algorithm</name>
321 <value>$(lws_psk_algorithm)</value>
322 </element>
323 </value>
324 </property>
325 </participant_qos>
326 </qos_profile>
327
328 </qos_library>
329</dds>
2.1.2.4. Limiting Interfaces
The idea and methodology for this test are similar to the one explained above. In this case we aim to explore how the network usage (throughput in and out) and the discovery time changes when we announce the participants over more than one interface versus when we limit that announcement to a single interface.
Note
This scenario is likely not the optimal design solution for a real-life architecture. The purpose of the test is to demonstrate how powerful the Connext discovery protocol is in a flat configuration.
Endpoint Discovery
The following graph displays the time it takes to complete endpoint discovery, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum times that the participants took to complete endpoint discovery. (Maximums and minimums are the dashed lines; medians are the solid lines).
The following graphs display the amount of bytes sent and received until the discovery process completes, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
Sent
Received
The following graphs display the amount of memory required by the application after completing the discovery process, per number of participants. There is one endpoint for each participant; across all participants, half the endpoints are DataWriters and half are DataReaders. For each scenario, we graph three values: the maximum, median, and minimum bytes reported by the participants until the completion of endpoint discovery (Maximums and minimums are the dashed lines; medians are the solid lines).
The parameters for testing this scenario are:
Number of hosts: 12
Participants in the system: <variable we increase>
Topics in the system: 1
Readers per topic: half of the participants
Writers per topic: half of the participants
QoS profiles used: DynamicProfile_unicast, DynamicProfile
Software information
RTI developed a testing framework specifically designed for discovery benchmarking. This framework was used to perform the tests detailed in this section. This framework is capable of distributing and executing the different DDS entities across the different machines available in RTI’s Performance and Discovery Lab. It also gathers information about discovery time as well as network usage and memory usage.
The Middleware version used to perform these tests is:
RTI Connext DDS 7.3.0 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)
Hardware information
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
QoS Used
1<?xml version="1.0"?>
2<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
3 xsi:noNamespaceSchemaLocation="https://community.rti.com/schema/7.2.0/rti_dds_qos_profiles.xsd">
4
5 <qos_library name="QosLibrary">
6
7 <qos_profile name="FactoryDefault" is_default_participant_factory_profile="true">
8 <participant_factory_qos>
9 <entity_factory>
10 <autoenable_created_entities>$(autoenable_created_entities)</autoenable_created_entities>
11 </entity_factory>
12 <!-- <logging>
13 <verbosity>WARNING</verbosity>
14 </logging> -->
15 </participant_factory_qos>
16 <participant_qos>
17 <discovery_config>
18 <builtin_discovery_plugins>$(qos_discovery_protocol)</builtin_discovery_plugins>
19 </discovery_config>
20 </participant_qos>
21 </qos_profile>
22
23 <qos_profile name="DynamicProfile_DefaultQoS" base_name="QosLibrary::FactoryDefault">
24 <participant_qos>
25 <!-- This does not affect performance, but it is needed for > 1000 participants -->
26 <wire_protocol>
27 <rtps_well_known_ports>
28 <domain_id_gain>500</domain_id_gain>
29 </rtps_well_known_ports>
30 </wire_protocol>
31 <!-- To make the test a bit more fair -->
32 <transport_builtin>
33 <mask>UDPv4</mask>
34 </transport_builtin>
35 <property>
36 <value>
37 <element>
38 <name>dds.transport.UDPv4.builtin.parent.allow_interfaces_list</name>
39 <value>$(interface_name)</value>
40 </element>
41 <element>
42 <name>dds.transport.UDPv4.builtin.parent.max_interface_count</name>
43 <value>2</value>
44 </element>
45 <element>
46 <name>dds.participant.property_validation_action</name>
47 <value>1</value>
48 </element>
49 <element>
50 <name>dds.transport.UDPv4.builtin.gather_detailed_statistics</name>
51 <value>$(gather_detailed_statistics)</value>
52 </element>
53 </value>
54 </property>
55 </participant_qos>
56 </qos_profile>
57
58 <qos_profile name="DynamicProfile" base_name="QosLibrary::DynamicProfile_DefaultQoS" is_default_qos="true">
59 <participant_qos>
60 <resource_limits>
61 <type_object_max_serialized_length>0</type_object_max_serialized_length>
62 <type_code_max_serialized_length>0</type_code_max_serialized_length>
63 <remote_writer_hash_buckets>625</remote_writer_hash_buckets>
64 <remote_reader_hash_buckets>625</remote_reader_hash_buckets>
65 <remote_participant_hash_buckets>625</remote_participant_hash_buckets>
66 <matching_reader_writer_pair_hash_buckets>625</matching_reader_writer_pair_hash_buckets>
67 <matching_writer_reader_pair_hash_buckets>625</matching_writer_reader_pair_hash_buckets>
68 </resource_limits>
69 <discovery_config>
70 <initial_participant_announcements>5</initial_participant_announcements>
71 <participant_liveliness_lease_duration>
72 <sec>500</sec>
73 <nanosec>0</nanosec>
74 </participant_liveliness_lease_duration>
75 <participant_liveliness_assert_period>
76 <sec>5</sec>
77 <nanosec>0</nanosec>
78 </participant_liveliness_assert_period>
79 <remote_participant_purge_kind>LIVELINESS_BASED_REMOTE_PARTICIPANT_PURGE</remote_participant_purge_kind>
80 <max_liveliness_loss_detection_period>
81 <sec>10</sec>
82 <nanosec>0</nanosec>
83 </max_liveliness_loss_detection_period>
84 </discovery_config>
85 <property>
86 <value>
87 <element>
88 <name>dds.transport.UDPv4.builtin.recv_socket_buffer_size</name>
89 <value>5048576</value>
90 </element>
91 </value>
92 </property>
93 </participant_qos>
94 </qos_profile>
95
96 <qos_profile name="Unicast_10Gbps_lab_snippet">
97 <participant_qos>
98 <discovery>
99 <initial_peers>
100 <element>$(initial-peers)</element>
101 </initial_peers>
102 <multicast_receive_addresses></multicast_receive_addresses>
103 </discovery>
104 </participant_qos>
105 </qos_profile>
106
107 <qos_profile name="Unicast_1Gbps_lab_snippet">
108 <participant_qos>
109 <discovery>
110 <initial_peers>
111 <element>$(initial-peers)</element>
112 </initial_peers>
113 <multicast_receive_addresses></multicast_receive_addresses>
114 </discovery>
115 </participant_qos>
116 </qos_profile>
117
118 <qos_profile name="DynamicProfile_unicast_cds" base_name="QosLibrary::DynamicProfile">
119 <participant_qos>
120 <transport_builtin>
121 <mask>UDPv4</mask>
122 </transport_builtin>
123 <discovery>
124 <initial_peers>
125 <element>rtps@udpv4://$(CDS_IP):7400</element>
126 </initial_peers>
127 <multicast_receive_addresses></multicast_receive_addresses>
128 </discovery>
129 </participant_qos>
130 </qos_profile>
131
132 <qos_profile name="StaticProfile" base_name="QosLibrary::DynamicProfile">
133 <participant_qos>
134 <discovery_config>
135 <builtin_discovery_plugins>SPDP</builtin_discovery_plugins>
136 </discovery_config>
137 <property>
138 <value>
139 <element>
140 <name>dds.discovery.endpoint.lbediscovery.library</name>
141 <value>rtilbedisc</value>
142 </element>
143 <element>
144 <name>dds.discovery.endpoint.lbediscovery.create_function</name>
145 <value>DDS_LBEDiscoveryPlugin_create</value>
146 </element>
147 <element>
148 <name>dds.discovery.endpoint.load_plugins</name>
149 <value>dds.discovery.endpoint.lbediscovery</value>
150 </element>
151 </value>
152 </property>
153 </participant_qos>
154 </qos_profile>
155
156 <!-- Security -->
157 <qos_profile name="DynamicProfileSecurity" base_name="QosLibrary::DynamicProfile">
158 <participant_qos>
159 <property>
160 <value>
161 <element>
162 <name>com.rti.serv.load_plugin</name>
163 <value>com.rti.serv.secure</value>
164 </element>
165 <element>
166 <name>com.rti.serv.secure.library</name>
167 <value>nddssecurity</value>
168 </element>
169 <element>
170 <name>com.rti.serv.secure.create_function</name>
171 <value>RTI_Security_PluginSuite_create</value>
172 </element>
173 <element>
174 <name>com.rti.serv.secure.authentication.ca_file</name>
175 <value>resources/secure/certAuthority/$(discovery_security_algo)/ca/$(discovery_security_algo)RootCaCert.pem</value>
176 </element>
177 <element>
178 <name>com.rti.serv.secure.authentication.private_key_file</name>
179 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Key.pem</value>
180 </element>
181 <element>
182 <name>com.rti.serv.secure.authentication.certificate_file</name>
183 <value>resources/secure/certAuthority/$(discovery_security_algo)/identities/$(discovery_security_algo)Peer01Cert.pem</value>
184 </element>
185 <element>
186 <name>com.rti.serv.secure.access_control.permissions_authority_file</name>
187 <value>resources/secure/certAuthority/$(discovery_permissions_authority_file_algo)/ca/$(discovery_permissions_authority_file_algo)RootCaCert.pem</value>
188 </element>
189 <element>
190 <name>com.rti.serv.secure.access_control.governance_file</name>
191 <value>resources/secure/certAuthority/$(discovery_security_algo)/governances/signed_governance_$(security_governance).xml</value>
192 </element>
193 <element>
194 <name>com.rti.serv.secure.access_control.permissions_file</name>
195 <value>resources/secure/certAuthority/$(discovery_security_algo)/signed_myPermissions.xml</value>
196 </element>
197 <element>
198 <name>com.rti.serv.secure.authentication.key_establishment_algorithm</name>
199 <value>auto</value>
200 </element>
201 <element>
202 <name>dds.participant.trust_plugins.authentication_timeout.sec</name>
203 <value>$(discovery_security_authentication_timeout)</value>
204 </element>
205 <element>
206 <name>dds.participant.trust_plugins.authentication_request_delay.sec</name>
207 <value>$(discovery_security_authentication_request_delay)</value>
208 </element>
209 <element>
210 <name>dds.participant.trust_plugins.authentication_request_timeout.sec</name>
211 <value>$(discovery_security_authentication_request_timeout)</value>
212 </element>
213 <element>
214 <name>com.rti.serv.secure.authentication.enable_custom_algorithms</name>
215 <value>true</value>
216 </element>
217 </value>
218 </property>
219 </participant_qos>
220 </qos_profile>
221
222 <!-- Security Unicast 10Gbps-->
223 <qos_profile name="DynamicProfileSecurity_unicast" base_name="QosLibrary::DynamicProfileSecurity">
224 <base_name>
225 <element>QosLibrary::Unicast_10Gbps_lab_snippet</element>
226 </base_name>
227 </qos_profile>
228
229 <!-- Security Unicast 10Gbps-->
230 <qos_profile name="DynamicProfileSecurity_unicast_enp1s0f0" base_name="QosLibrary::DynamicProfileSecurity_unicast">
231 </qos_profile>
232
233 <!-- Security Unicast 1Gbps -->
234 <qos_profile name="DynamicProfileSecurity_unicast_eno1" base_name="QosLibrary::DynamicProfileSecurity">
235 <base_name>
236 <element>QosLibrary::Unicast_1Gbps_lab_snippet</element>
237 </base_name>
238 </qos_profile>
239
240 <!-- Security HMAC ONLY -->
241 <qos_profile name="DynamicProfileSecurity_HMAC" base_name="QosLibrary::DynamicProfile">
242 <participant_qos>
243 <property>
244 <value>
245 <element>
246 <name>com.rti.serv.load_plugin</name>
247 <value>com.rti.serv.secure</value>
248 </element>
249 <element>
250 <name>com.rti.serv.secure.library</name>
251 <value>nddssecurity</value>
252 </element>
253 <element>
254 <name>com.rti.serv.secure.create_function</name>
255 <value>RTI_Security_PluginSuite_create</value>
256 </element>
257 <element>
258 <name>com.rti.serv.secure.hmac_only.enabled</name>
259 <value>1</value>
260 </element>
261 <element>
262 <name>com.rti.serv.secure.hmac_only.cryptography.key</name>
263 <value>str:SecretKey</value>
264 </element>
265 </value>
266 </property>
267 </participant_qos>
268 </qos_profile>
269
270 <!-- Security + PSK-->
271 <qos_profile name="DynamicProfileSecurity_PSK" base_name="QosLibrary::DynamicProfileSecurity">
272 <participant_qos>
273 <discovery_config>
274 <default_domain_announcement_period>
275 <sec>DURATION_INFINITE_SEC</sec>
276 <nanosec>DURATION_INFINITE_NSEC</nanosec>
277 </default_domain_announcement_period>
278 </discovery_config>
279 <property>
280 <value>
281 <element>
282 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
283 <!-- <value>str:1:SecretKey</value> -->
284 <value>data:,1:SecretKey</value>
285 </element>
286 </value>
287 </property>
288 </participant_qos>
289 </qos_profile>
290
291 <!-- LW Security + PSK -->
292 <qos_profile name="DynamicProfileLWS_PSK" base_name="QosLibrary::DynamicProfile">
293 <participant_qos>
294 <discovery_config>
295 <default_domain_announcement_period>
296 <sec>DURATION_INFINITE_SEC</sec>
297 <nanosec>DURATION_INFINITE_NSEC</nanosec>
298 </default_domain_announcement_period>
299 </discovery_config>
300 <property>
301 <value>
302 <element>
303 <name>com.rti.serv.load_plugin</name>
304 <value>com.rti.serv.secure</value>
305 </element>
306 <element>
307 <name>com.rti.serv.secure.library</name>
308 <value>nddslightweightsecurity</value>
309 </element>
310 <element>
311 <name>com.rti.serv.secure.create_function</name>
312 <value>RTI_SecurityLightweight_PluginSuite_create</value>
313 </element>
314 <element>
315 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key</name>
316 <!-- <value>str:1:SecretKey</value> -->
317 <value>data:,1:SecretKey</value>
318 </element>
319 <element>
320 <name>com.rti.serv.secure.cryptography.rtps_protection_preshared_key_algorithm</name>
321 <value>$(lws_psk_algorithm)</value>
322 </element>
323 </value>
324 </property>
325 </participant_qos>
326 </qos_profile>
327
328 </qos_library>
329</dds>