6.1.1.1. Real-Time WAN
Real-Time WAN vs UDPv4 in LAN
In this initial test we aim to show that the performance obtained with the Real-Time WAN Transport is similar to the one obtained by using the UDPv4 transport. In other words, Real-Time WAN Transport barely adds overheads when used in LAN.
Unkeyed, UDPv4_WAN 10Gbps Network, C++98
The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.
Note
We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.
Note
By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.
The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
TCP/TLS vs Real-Time WAN and Security
The following tests compare the differences in performance between the use of TCP/TLS and Real-Time WAN Transport (UDPv4 over WAN) with and without Security in a more realistic scenario: the communications between a machine in our lab (HQ) and an Amazon Web Services (AWS) virtual machine.
Real-Time WAN Transport is a transport that enables secure, scalable, and high-performance communication over wide area networks (WANs), including public networks. It can be used to extend Connext capabilities to WAN environments. Real-Time WAN Transport uses UDPv4 as the underlying IP transport-layer protocol to better anticipate and adapt to the challenges of diverse network conditions, device mobility, and the dynamic nature of WAN system architectures. For more information, see RTI Real-Time WAN Transport, in the RTI Connext Core Libraries User’s Manual.
The following tests compare TCP vs Real-Time WAN Transport (RWT). They also compare TLS, which implements encryption over TCP, with an equivalent level of security using Security Plugins over the Real-Time WAN Transport.
Data Tables
TCP
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
56513 |
14895.8 |
33342 |
67423 |
66165 |
66662 |
66886 |
67423 |
67423 |
1.46 |
64 |
58089 |
14076.4 |
33344 |
67196 |
66373 |
66632 |
66696 |
67196 |
67196 |
1.53 |
128 |
52247 |
15623.7 |
33317 |
68177 |
64415 |
66504 |
67374 |
68177 |
68177 |
1.43 |
256 |
45870 |
12688.9 |
33333 |
66482 |
37122 |
63221 |
64828 |
66482 |
66482 |
1.52 |
512 |
44657 |
8328.1 |
33396 |
58792 |
43262 |
55690 |
58237 |
58792 |
58792 |
1.62 |
1024 |
45735 |
9063.0 |
33375 |
58479 |
42885 |
57472 |
58323 |
58479 |
58479 |
1.52 |
2048 |
45714 |
6598.6 |
33318 |
55801 |
47170 |
53022 |
54134 |
55801 |
55801 |
1.66 |
4096 |
45500 |
12585.6 |
33315 |
63342 |
37913 |
62437 |
63234 |
63342 |
63342 |
1.57 |
8192 |
45531 |
7616.5 |
33659 |
56629 |
45414 |
54773 |
56410 |
56629 |
56629 |
1.56 |
16384 |
45135 |
8924.8 |
33634 |
58757 |
41972 |
57825 |
58459 |
58757 |
58757 |
1.62 |
32768 |
45282 |
8389.4 |
33732 |
100317 |
44533 |
54253 |
56350 |
100317 |
100317 |
1.76 |
63000 |
45459 |
10948.8 |
34140 |
167265 |
46374 |
52047 |
53323 |
167265 |
167265 |
1.89 |
100000 |
47832 |
16429.1 |
34480 |
200825 |
38074 |
63125 |
63873 |
200825 |
200825 |
2.26 |
200000 |
70078 |
16839.4 |
68067 |
267995 |
68263 |
68583 |
100636 |
267995 |
267995 |
2.26 |
300000 |
129572 |
27437.3 |
101993 |
334909 |
135258 |
136129 |
334909 |
334909 |
334909 |
1.61 |
500000 |
182037 |
36188.5 |
169316 |
429641 |
170131 |
202979 |
429641 |
429641 |
429641 |
1.57 |
1048576 |
441878 |
36148.1 |
406666 |
604516 |
438370 |
441713 |
604516 |
604516 |
604516 |
1.34 |
TLS
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
56513 |
14895.8 |
33342 |
67423 |
66165 |
66662 |
66886 |
67423 |
67423 |
1.46 |
64 |
58089 |
14076.4 |
33344 |
67196 |
66373 |
66632 |
66696 |
67196 |
67196 |
1.53 |
128 |
52247 |
15623.7 |
33317 |
68177 |
64415 |
66504 |
67374 |
68177 |
68177 |
1.43 |
256 |
45870 |
12688.9 |
33333 |
66482 |
37122 |
63221 |
64828 |
66482 |
66482 |
1.52 |
512 |
44657 |
8328.1 |
33396 |
58792 |
43262 |
55690 |
58237 |
58792 |
58792 |
1.62 |
1024 |
45735 |
9063.0 |
33375 |
58479 |
42885 |
57472 |
58323 |
58479 |
58479 |
1.52 |
2048 |
45714 |
6598.6 |
33318 |
55801 |
47170 |
53022 |
54134 |
55801 |
55801 |
1.66 |
4096 |
45500 |
12585.6 |
33315 |
63342 |
37913 |
62437 |
63234 |
63342 |
63342 |
1.57 |
8192 |
45531 |
7616.5 |
33659 |
56629 |
45414 |
54773 |
56410 |
56629 |
56629 |
1.56 |
16384 |
45135 |
8924.8 |
33634 |
58757 |
41972 |
57825 |
58459 |
58757 |
58757 |
1.62 |
32768 |
45282 |
8389.4 |
33732 |
100317 |
44533 |
54253 |
56350 |
100317 |
100317 |
1.76 |
63000 |
45459 |
10948.8 |
34140 |
167265 |
46374 |
52047 |
53323 |
167265 |
167265 |
1.89 |
100000 |
47832 |
16429.1 |
34480 |
200825 |
38074 |
63125 |
63873 |
200825 |
200825 |
2.26 |
200000 |
70078 |
16839.4 |
68067 |
267995 |
68263 |
68583 |
100636 |
267995 |
267995 |
2.26 |
300000 |
129572 |
27437.3 |
101993 |
334909 |
135258 |
136129 |
334909 |
334909 |
334909 |
1.61 |
500000 |
182037 |
36188.5 |
169316 |
429641 |
170131 |
202979 |
429641 |
429641 |
429641 |
1.57 |
1048576 |
441878 |
36148.1 |
406666 |
604516 |
438370 |
441713 |
604516 |
604516 |
604516 |
1.34 |
RWT (No Security)
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
33120 |
151.6 |
33041 |
34835 |
33088 |
33141 |
33775 |
34835 |
34835 |
1.46 |
64 |
33100 |
78.4 |
33038 |
33755 |
33083 |
33126 |
33463 |
33755 |
33755 |
1.46 |
128 |
33098 |
49.9 |
33029 |
33441 |
33088 |
33119 |
33377 |
33441 |
33441 |
1.46 |
256 |
33133 |
102.4 |
33039 |
33971 |
33105 |
33186 |
33738 |
33971 |
33971 |
1.54 |
512 |
33146 |
96.5 |
33064 |
33979 |
33125 |
33171 |
33730 |
33979 |
33979 |
1.51 |
1024 |
33203 |
128.2 |
33130 |
35021 |
33174 |
33271 |
33619 |
35021 |
35021 |
1.55 |
2048 |
33370 |
2202.3 |
33165 |
71419 |
33224 |
33283 |
33881 |
71419 |
71419 |
1.46 |
4096 |
33369 |
176.3 |
33216 |
35475 |
33312 |
33520 |
34076 |
35475 |
35475 |
1.42 |
8192 |
33542 |
245.1 |
33371 |
35081 |
33455 |
33728 |
34710 |
35081 |
35081 |
1.55 |
16384 |
33840 |
133.8 |
33664 |
35177 |
33822 |
33933 |
34520 |
35177 |
35177 |
2.01 |
32768 |
34561 |
641.9 |
34291 |
42429 |
34477 |
34672 |
38765 |
42429 |
42429 |
2.32 |
63000 |
43309 |
14773.6 |
34792 |
74219 |
35463 |
70384 |
73205 |
74219 |
74219 |
2.64 |
100000 |
38143 |
233.0 |
37885 |
39789 |
38096 |
38407 |
38932 |
39789 |
39789 |
2.19 |
200000 |
41480 |
334.6 |
41147 |
43421 |
41359 |
41847 |
43006 |
43421 |
43421 |
2.69 |
300000 |
44838 |
410.9 |
44466 |
47410 |
44675 |
45296 |
46821 |
47410 |
47410 |
2.91 |
500000 |
52639 |
2783.1 |
51122 |
86690 |
52120 |
54400 |
58473 |
86690 |
86690 |
3.55 |
1048576 |
153956 |
12409.0 |
104219 |
177577 |
153248 |
171977 |
177577 |
177577 |
177577 |
3.16 |
RWT (RTPS Sign + Encrypt)
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
33123 |
132.8 |
33055 |
34731 |
33097 |
33158 |
33654 |
34731 |
34731 |
1.78 |
64 |
33154 |
205.7 |
33018 |
35093 |
33104 |
33208 |
34105 |
35093 |
35093 |
1.81 |
128 |
33140 |
150.9 |
33051 |
34837 |
33103 |
33186 |
33819 |
34837 |
34837 |
1.74 |
256 |
33153 |
87.2 |
33049 |
33742 |
33118 |
33266 |
33541 |
33742 |
33742 |
1.78 |
512 |
33163 |
107.7 |
33060 |
34415 |
33137 |
33218 |
33586 |
34415 |
34415 |
1.83 |
1024 |
33227 |
216.0 |
33106 |
35591 |
33175 |
33264 |
34272 |
35591 |
35591 |
1.73 |
2048 |
33327 |
2139.2 |
33128 |
70276 |
33180 |
33239 |
34013 |
70276 |
70276 |
1.74 |
4096 |
33283 |
143.4 |
33187 |
34889 |
33249 |
33357 |
33868 |
34889 |
34889 |
1.83 |
8192 |
33448 |
155.0 |
33309 |
34622 |
33398 |
33586 |
34382 |
34622 |
34622 |
1.82 |
16384 |
34116 |
145.9 |
33950 |
35284 |
34083 |
34252 |
34922 |
35284 |
35284 |
1.92 |
32768 |
35096 |
128.7 |
34928 |
36186 |
35066 |
35206 |
35796 |
36186 |
36186 |
2.05 |
63000 |
36838 |
231.5 |
36342 |
39799 |
36810 |
36982 |
37188 |
39799 |
39799 |
6.01 |
100000 |
38076 |
249.2 |
37827 |
40478 |
38007 |
38282 |
39070 |
40478 |
40478 |
3.33 |
200000 |
41518 |
275.8 |
41191 |
42922 |
41425 |
41865 |
42670 |
42922 |
42922 |
4.05 |
300000 |
44895 |
472.6 |
44590 |
48221 |
44769 |
45230 |
47457 |
48221 |
48221 |
4.96 |
500000 |
52658 |
1262.9 |
51200 |
57150 |
52351 |
54487 |
56292 |
57150 |
57150 |
6.30 |
1048576 |
151806 |
12836.5 |
104333 |
176735 |
152939 |
167952 |
176735 |
176735 |
176735 |
6.05 |
Data Tables
TCP
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
945664 |
47103 |
12.1 |
0 |
0.00 |
0.23 |
64 |
526720 |
26216 |
13.4 |
0 |
0.00 |
0.19 |
128 |
278656 |
13922 |
14.3 |
0 |
0.00 |
0.17 |
256 |
144000 |
7186 |
14.7 |
0 |
0.00 |
0.17 |
512 |
72992 |
3632 |
14.9 |
0 |
0.00 |
0.16 |
1024 |
36664 |
1825 |
15.0 |
0 |
0.00 |
0.17 |
2048 |
18424 |
920 |
15.1 |
0 |
0.00 |
0.17 |
4096 |
9274 |
461 |
15.1 |
0 |
0.00 |
0.18 |
8192 |
4637 |
231 |
15.2 |
0 |
0.00 |
0.17 |
16384 |
2323 |
115 |
15.2 |
0 |
0.00 |
0.17 |
32768 |
1157 |
58 |
15.2 |
0 |
0.00 |
0.16 |
63000 |
599 |
30 |
15.2 |
0 |
0.00 |
0.17 |
100000 |
423 |
19 |
15.3 |
0 |
0.00 |
0.24 |
200000 |
191 |
9 |
15.3 |
0 |
0.00 |
0.18 |
300000 |
150 |
6 |
14.7 |
0 |
0.00 |
0.16 |
500000 |
5 |
0.8 |
0 |
0.00 |
0.00 |
|
1048576 |
3 |
0.6 |
0 |
0.00 |
0.01 |
TLS
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
938496 |
46746 |
12.0 |
0 |
0.00 |
0.24 |
64 |
524544 |
26181 |
13.4 |
0 |
0.00 |
0.20 |
128 |
277952 |
13826 |
14.2 |
0 |
0.00 |
0.19 |
256 |
142880 |
7121 |
14.6 |
0 |
0.00 |
0.17 |
512 |
72336 |
3606 |
14.8 |
0 |
0.00 |
0.18 |
1024 |
36392 |
1809 |
14.8 |
0 |
0.00 |
0.17 |
2048 |
18208 |
908 |
14.9 |
0 |
0.00 |
0.17 |
4096 |
9164 |
456 |
15.0 |
0 |
0.00 |
0.19 |
8192 |
4589 |
228 |
15.0 |
0 |
0.00 |
0.16 |
16384 |
2297 |
114 |
15.0 |
0 |
0.00 |
0.16 |
32768 |
1154 |
57 |
15.2 |
0 |
0.00 |
0.16 |
63000 |
595 |
29 |
15.1 |
0 |
0.00 |
0.16 |
100000 |
419 |
19 |
15.2 |
0 |
0.00 |
0.17 |
200000 |
199 |
9 |
15.1 |
0 |
0.00 |
0.16 |
300000 |
140 |
6 |
14.6 |
0 |
0.00 |
0.16 |
500000 |
72 |
2 |
8.2 |
0 |
0.00 |
0.15 |
1048576 |
52 |
5.9 |
0 |
0.00 |
0.12 |
RWT (No Security)
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
3711233 |
185519 |
47.5 |
0 |
0.00 |
1.76 |
64 |
1829121 |
91425 |
46.8 |
0 |
0.00 |
1.64 |
128 |
925185 |
46249 |
47.4 |
0 |
0.00 |
1.63 |
256 |
462625 |
23123 |
47.4 |
0 |
0.00 |
1.60 |
512 |
230833 |
11539 |
47.3 |
0 |
0.00 |
1.57 |
1024 |
115201 |
5759 |
47.2 |
0 |
0.00 |
1.57 |
2048 |
57709 |
2885 |
47.3 |
0 |
0.00 |
1.55 |
4096 |
28945 |
1447 |
47.4 |
0 |
0.00 |
1.58 |
8192 |
14353 |
717 |
47.0 |
0 |
0.00 |
1.55 |
16384 |
11989 |
599 |
78.6 |
0 |
0.00 |
1.12 |
32768 |
9431 |
471 |
123.6 |
0 |
0.00 |
0.97 |
63000 |
5433 |
270 |
136.3 |
0 |
0.00 |
0.56 |
100000 |
3224 |
160 |
128.0 |
0 |
0.00 |
0.53 |
200000 |
1747 |
85 |
136.9 |
0 |
0.00 |
0.49 |
300000 |
1069 |
51 |
124.5 |
0 |
0.00 |
0.41 |
400000 |
862 |
42 |
135.9 |
0 |
0.00 |
0.47 |
500000 |
553 |
26 |
106.6 |
0 |
0.00 |
0.49 |
1048576 |
301 |
14 |
125.3 |
0 |
0.00 |
0.43 |
RWT (RTPS Sign + Encrypt)
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
3642113 |
182062 |
46.6 |
0 |
0.00 |
1.91 |
64 |
1835137 |
91655 |
46.9 |
0 |
0.00 |
1.85 |
128 |
923904 |
46184 |
47.3 |
0 |
0.00 |
1.83 |
256 |
461601 |
23076 |
47.3 |
0 |
0.00 |
1.81 |
512 |
230849 |
11540 |
47.3 |
0 |
0.00 |
1.79 |
1024 |
115057 |
5751 |
47.1 |
0 |
0.00 |
1.78 |
2048 |
57868 |
2893 |
47.4 |
0 |
0.00 |
1.77 |
4096 |
28815 |
1440 |
47.2 |
0 |
0.00 |
1.76 |
8192 |
14377 |
718 |
47.1 |
0 |
0.00 |
1.75 |
16384 |
12052 |
602 |
79.0 |
0 |
0.00 |
1.27 |
32768 |
9161 |
458 |
120.1 |
0 |
0.00 |
0.98 |
63000 |
5561 |
278 |
140.1 |
0 |
0.00 |
0.74 |
100000 |
2709 |
133 |
106.7 |
0 |
0.00 |
0.57 |
200000 |
1605 |
79 |
127.2 |
0 |
0.00 |
0.63 |
300000 |
803 |
39 |
95.8 |
0 |
0.00 |
0.44 |
400000 |
771 |
38 |
122.8 |
0 |
0.00 |
0.62 |
500000 |
668 |
33 |
132.4 |
0 |
0.00 |
0.54 |
1048576 |
281 |
13 |
117.1 |
0 |
0.00 |
0.52 |
Test Hardware
The following hardware was used to perform these tests:
Local Node
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Location: United States of America, California
AWS Machine
Kind: Ubuntu Server 20.04 LTS (HVM), SSD Volume Type
Processor: Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz (1 core)
RAM: 1GB
Location: United States of America, East Coast
Data Tables
TCP
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
64448 |
7665.2 |
33269 |
67448 |
66595 |
66786 |
67020 |
67448 |
67448 |
0.02 |
64 |
66843 |
3117.6 |
33448 |
79962 |
66557 |
69506 |
70607 |
79962 |
79962 |
0.02 |
128 |
65453 |
5848.4 |
33301 |
68486 |
66542 |
66735 |
68411 |
68486 |
68486 |
0.02 |
256 |
63945 |
9399.1 |
33299 |
88690 |
66563 |
66988 |
71025 |
88690 |
88690 |
0.02 |
512 |
44617 |
12167.9 |
33373 |
63713 |
36631 |
62510 |
63526 |
63713 |
63713 |
0.02 |
1024 |
44643 |
7676.6 |
33310 |
56535 |
44417 |
54158 |
55528 |
56535 |
56535 |
0.02 |
2048 |
44718 |
11271.2 |
33304 |
63071 |
38575 |
60785 |
62485 |
63071 |
63071 |
0.03 |
4096 |
44276 |
10361.0 |
33345 |
61413 |
39054 |
59267 |
60720 |
61413 |
61413 |
0.03 |
8192 |
44696 |
7115.5 |
33608 |
53354 |
46443 |
52217 |
53085 |
53354 |
53354 |
0.03 |
16384 |
51365 |
15648.9 |
33547 |
68028 |
63872 |
66548 |
66841 |
68028 |
68028 |
0.03 |
32768 |
45050 |
7730.7 |
33824 |
102497 |
48539 |
50714 |
53184 |
102497 |
102497 |
0.06 |
63000 |
44949 |
12019.9 |
34116 |
170815 |
43199 |
54762 |
58257 |
170815 |
170815 |
0.09 |
100000 |
67620 |
11644.8 |
39267 |
203470 |
66665 |
66901 |
93761 |
203470 |
203470 |
0.07 |
200000 |
70004 |
14607.8 |
68239 |
240789 |
68445 |
68751 |
101881 |
240789 |
240789 |
0.10 |
300000 |
137864 |
16370.6 |
134474 |
276582 |
135821 |
137226 |
276582 |
276582 |
276582 |
0.09 |
500000 |
174815 |
31399.7 |
169215 |
409727 |
169790 |
172659 |
409727 |
409727 |
409727 |
0.11 |
1048576 |
415427 |
43991.3 |
402717 |
626209 |
403679 |
435947 |
626209 |
626209 |
626209 |
0.10 |
TLS
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
60281 |
12473.1 |
33376 |
66777 |
66226 |
66354 |
66622 |
66777 |
66777 |
0.03 |
64 |
57168 |
14596.1 |
33475 |
66798 |
66247 |
66675 |
66790 |
66798 |
66798 |
0.02 |
128 |
52150 |
11615.4 |
33813 |
70781 |
48195 |
69599 |
70663 |
70781 |
70781 |
0.02 |
256 |
44157 |
12079.9 |
33241 |
63556 |
36360 |
62131 |
63459 |
63556 |
63556 |
0.03 |
512 |
44674 |
8229.5 |
33315 |
57619 |
43416 |
55078 |
57562 |
57619 |
57619 |
0.03 |
1024 |
44492 |
9342.8 |
33298 |
58389 |
41162 |
57193 |
58181 |
58389 |
58389 |
0.03 |
2048 |
44911 |
6960.1 |
33358 |
53630 |
46972 |
52877 |
53468 |
53630 |
53630 |
0.03 |
4096 |
44472 |
12728.1 |
33402 |
64275 |
36351 |
62622 |
63939 |
64275 |
64275 |
0.03 |
8192 |
43924 |
8014.1 |
33481 |
55878 |
43950 |
54285 |
55624 |
55878 |
55878 |
0.03 |
16384 |
44963 |
9119.5 |
33639 |
67167 |
41891 |
58083 |
58493 |
67167 |
67167 |
0.04 |
32768 |
45451 |
8418.3 |
33849 |
100466 |
44791 |
54590 |
55901 |
100466 |
100466 |
0.07 |
63000 |
44669 |
9356.4 |
34087 |
133867 |
45875 |
52048 |
53352 |
133867 |
133867 |
0.09 |
100000 |
49863 |
12433.4 |
36788 |
135595 |
41394 |
61997 |
63419 |
135595 |
135595 |
0.09 |
200000 |
70439 |
14033.9 |
68760 |
234097 |
69002 |
69338 |
100551 |
234097 |
234097 |
0.11 |
300000 |
127088 |
24672.3 |
101738 |
301154 |
134572 |
135164 |
301154 |
301154 |
301154 |
0.10 |
500000 |
181016 |
32465.4 |
169561 |
402557 |
170388 |
202892 |
402557 |
402557 |
402557 |
0.10 |
1048576 |
428978 |
46682.1 |
403150 |
639097 |
435818 |
437529 |
639097 |
639097 |
639097 |
0.11 |
RWT (No Security)
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
33120 |
151.6 |
33041 |
34835 |
33088 |
33141 |
33775 |
34835 |
34835 |
1.46 |
64 |
33100 |
78.4 |
33038 |
33755 |
33083 |
33126 |
33463 |
33755 |
33755 |
1.46 |
128 |
33098 |
49.9 |
33029 |
33441 |
33088 |
33119 |
33377 |
33441 |
33441 |
1.46 |
256 |
33133 |
102.4 |
33039 |
33971 |
33105 |
33186 |
33738 |
33971 |
33971 |
1.54 |
512 |
33146 |
96.5 |
33064 |
33979 |
33125 |
33171 |
33730 |
33979 |
33979 |
1.51 |
1024 |
33203 |
128.2 |
33130 |
35021 |
33174 |
33271 |
33619 |
35021 |
35021 |
1.55 |
2048 |
33370 |
2202.3 |
33165 |
71419 |
33224 |
33283 |
33881 |
71419 |
71419 |
1.46 |
4096 |
33369 |
176.3 |
33216 |
35475 |
33312 |
33520 |
34076 |
35475 |
35475 |
1.42 |
8192 |
33542 |
245.1 |
33371 |
35081 |
33455 |
33728 |
34710 |
35081 |
35081 |
1.55 |
16384 |
33953 |
45.5 |
33876 |
34169 |
33947 |
34012 |
34094 |
34169 |
34169 |
0.18 |
32768 |
34465 |
111.5 |
34343 |
35651 |
34443 |
34533 |
35023 |
35651 |
35651 |
0.18 |
63000 |
35442 |
133.0 |
35274 |
36829 |
35416 |
35557 |
35758 |
36829 |
36829 |
0.18 |
100000 |
35863 |
134.9 |
35610 |
36687 |
35836 |
36026 |
36241 |
36687 |
36687 |
0.20 |
200000 |
37850 |
4758.5 |
36811 |
78056 |
37153 |
37576 |
75152 |
78056 |
78056 |
0.22 |
300000 |
38936 |
4654.8 |
37778 |
77752 |
38215 |
38967 |
76090 |
77752 |
77752 |
0.25 |
500000 |
41465 |
3783.6 |
39829 |
82164 |
40746 |
43122 |
45101 |
82164 |
82164 |
0.29 |
1048576 |
87688 |
8442.9 |
81632 |
128161 |
85634 |
89649 |
123899 |
128161 |
128161 |
0.23 |
RWT (RTPS Sign + Encrypt)
Sample Size (Bytes) |
Avg (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
CPU (%) |
---|---|---|---|---|---|---|---|---|---|---|
32 |
33241 |
178.4 |
33174 |
35438 |
33213 |
33249 |
33767 |
35438 |
35438 |
0.22 |
64 |
33217 |
29.6 |
33178 |
33549 |
33212 |
33245 |
33315 |
33549 |
33549 |
0.22 |
128 |
33256 |
209.6 |
33181 |
35460 |
33220 |
33262 |
34058 |
35460 |
35460 |
0.23 |
256 |
33234 |
44.7 |
33185 |
33644 |
33227 |
33251 |
33507 |
33644 |
33644 |
0.22 |
512 |
33263 |
55.5 |
33214 |
33793 |
33254 |
33284 |
33554 |
33793 |
33793 |
0.23 |
1024 |
33299 |
24.8 |
33258 |
33392 |
33294 |
33333 |
33382 |
33392 |
33392 |
0.22 |
2048 |
33430 |
51.0 |
33372 |
33875 |
33420 |
33470 |
33722 |
33875 |
33875 |
0.23 |
4096 |
33496 |
62.4 |
33433 |
34410 |
33488 |
33533 |
33617 |
34410 |
34410 |
0.22 |
8192 |
33636 |
45.3 |
33560 |
33917 |
33629 |
33681 |
33828 |
33917 |
33917 |
0.23 |
16384 |
33944 |
52.0 |
33837 |
34235 |
33933 |
34007 |
34166 |
34235 |
34235 |
0.23 |
32768 |
34471 |
93.2 |
34347 |
35393 |
34458 |
34533 |
34807 |
35393 |
35393 |
0.23 |
63000 |
35458 |
101.8 |
35285 |
35844 |
35431 |
35606 |
35827 |
35844 |
35844 |
0.23 |
100000 |
35817 |
157.2 |
35630 |
36588 |
35762 |
36060 |
36391 |
36588 |
36588 |
0.26 |
200000 |
37562 |
3369.7 |
36877 |
76535 |
37190 |
37572 |
39519 |
76535 |
76535 |
0.29 |
300000 |
38532 |
3435.1 |
37718 |
77416 |
38115 |
38825 |
40647 |
77416 |
77416 |
0.31 |
500000 |
40778 |
886.6 |
39838 |
44315 |
40492 |
42005 |
43834 |
44315 |
44315 |
0.37 |
1048576 |
86786 |
6416.3 |
82230 |
129337 |
85741 |
88412 |
124847 |
129337 |
129337 |
0.31 |
Data Tables
TCP
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
1606400 |
80206 |
20.5 |
0 |
0.00 |
5.95 |
64 |
871297 |
43550 |
22.3 |
0 |
0.00 |
5.10 |
128 |
432449 |
21610 |
22.1 |
0 |
0.00 |
3.97 |
256 |
214592 |
10696 |
21.9 |
0 |
0.00 |
3.07 |
512 |
109873 |
5481 |
22.5 |
0 |
0.00 |
3.33 |
1024 |
56160 |
2802 |
23.0 |
0 |
0.00 |
3.17 |
2048 |
27924 |
1394 |
22.9 |
0 |
0.00 |
2.86 |
4096 |
14075 |
702 |
23.0 |
0 |
0.00 |
3.02 |
8192 |
6971 |
348 |
22.8 |
0 |
0.00 |
2.79 |
16384 |
3029 |
151 |
19.9 |
0 |
0.00 |
2.59 |
32768 |
2195 |
109 |
28.8 |
0 |
0.00 |
3.06 |
63000 |
1176 |
58 |
29.7 |
0 |
0.00 |
1.52 |
100000 |
821 |
39 |
31.4 |
0 |
0.00 |
1.64 |
200000 |
379 |
18 |
28.9 |
0 |
0.00 |
3.01 |
300000 |
199 |
7 |
18.8 |
0 |
0.00 |
2.04 |
500000 |
46 |
1 |
5.4 |
0 |
0.00 |
0.44 |
1048576 |
51 |
1 |
8.4 |
0 |
0.00 |
0.44 |
TLS
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
1347840 |
67381 |
17.2 |
0 |
0.00 |
7.27 |
64 |
849792 |
42385 |
21.7 |
0 |
0.00 |
5.57 |
128 |
405952 |
20266 |
20.8 |
0 |
0.00 |
5.57 |
256 |
216896 |
10813 |
22.1 |
0 |
0.00 |
4.06 |
512 |
105856 |
5283 |
21.6 |
0 |
0.00 |
4.02 |
1024 |
54857 |
2742 |
22.5 |
0 |
0.00 |
4.07 |
2048 |
27429 |
1369 |
22.4 |
0 |
0.00 |
3.73 |
4096 |
13626 |
680 |
22.3 |
0 |
0.00 |
3.63 |
8192 |
7015 |
350 |
23.0 |
0 |
0.00 |
3.51 |
16384 |
4088 |
203 |
26.7 |
0 |
0.00 |
2.48 |
32768 |
2205 |
110 |
29.0 |
0 |
0.00 |
2.13 |
63000 |
1149 |
57 |
29.0 |
0 |
0.00 |
2.12 |
100000 |
808 |
38 |
31.2 |
0 |
0.00 |
2.34 |
200000 |
432 |
19 |
31.0 |
0 |
0.00 |
2.25 |
300000 |
294 |
12 |
30.4 |
0 |
0.00 |
2.07 |
500000 |
101 |
4 |
18.8 |
0 |
0.00 |
1.67 |
1048576 |
38 |
1 |
9.9 |
0 |
0.00 |
0.72 |
RWT (No Security)
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
1798913 |
89370 |
22.9 |
0 |
0.00 |
9.37 |
64 |
1673473 |
83559 |
42.8 |
0 |
0.00 |
9.62 |
128 |
888961 |
44442 |
45.5 |
0 |
0.00 |
7.64 |
256 |
456897 |
22803 |
46.7 |
0 |
0.00 |
6.32 |
512 |
230513 |
11500 |
47.1 |
0 |
0.00 |
6.40 |
1024 |
116025 |
5791 |
47.4 |
0 |
0.00 |
5.96 |
2048 |
58189 |
2909 |
47.7 |
0 |
0.00 |
5.64 |
4096 |
28423 |
1417 |
46.4 |
0 |
0.00 |
5.56 |
8192 |
13941 |
696 |
45.7 |
0 |
0.00 |
5.15 |
16384 |
7924 |
394 |
51.7 |
0 |
0.00 |
3.68 |
32768 |
2591 |
128 |
33.7 |
0 |
0.00 |
2.69 |
63000 |
1398 |
67 |
33.9 |
0 |
0.00 |
2.45 |
100000 |
684 |
31 |
25.4 |
0 |
0.00 |
2.42 |
200000 |
581 |
27 |
44.2 |
0 |
0.00 |
2.57 |
300000 |
470 |
21 |
50.6 |
0 |
0.00 |
2.70 |
400000 |
305 |
14 |
45.2 |
0 |
0.00 |
2.59 |
500000 |
324 |
13 |
55.5 |
0 |
0.00 |
2.80 |
1048576 |
170 |
5 |
48.4 |
0 |
0.00 |
2.64 |
RWT (RTPS Sign + Encrypt)
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
CPU (%) |
---|---|---|---|---|---|---|
32 |
1660673 |
82608 |
21.1 |
0 |
0.00 |
10.59 |
64 |
1665537 |
83083 |
42.5 |
0 |
0.00 |
11.82 |
128 |
880257 |
43917 |
45.0 |
0 |
0.00 |
9.77 |
256 |
450753 |
22529 |
46.1 |
0 |
0.00 |
8.63 |
512 |
230353 |
11509 |
47.1 |
0 |
0.00 |
8.53 |
1024 |
116137 |
5798 |
47.5 |
0 |
0.00 |
7.88 |
2048 |
56509 |
2824 |
46.3 |
0 |
0.00 |
7.88 |
4096 |
28437 |
1421 |
46.6 |
0 |
0.00 |
7.52 |
8192 |
14071 |
701 |
46.0 |
0 |
0.00 |
7.18 |
16384 |
8061 |
401 |
52.6 |
0 |
0.00 |
5.44 |
32768 |
2578 |
124 |
32.7 |
0 |
0.00 |
3.97 |
63000 |
1450 |
70 |
35.5 |
0 |
0.00 |
3.69 |
100000 |
660 |
29 |
23.6 |
0 |
0.00 |
3.81 |
200000 |
588 |
28 |
46.0 |
0 |
0.00 |
3.97 |
300000 |
480 |
21 |
52.6 |
0 |
0.00 |
4.28 |
400000 |
374 |
16 |
53.4 |
0 |
0.00 |
4.32 |
500000 |
327 |
14 |
56.9 |
0 |
0.00 |
4.41 |
1048576 |
172 |
5 |
48.9 |
0 |
0.00 |
4.10 |
Test Hardware
The following hardware was used to perform these tests:
Local Node
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Location: United States of America, California
AWS Machine
Kind: Ubuntu Server 20.04 LTS (HVM), SSD Volume Type
Processor: Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz (1 core)
RAM: 1GB
Location: United States of America, East Coast
Message Size Max vs Throughput
In this test we are going to compare the behavior of RWT when setting the message_size_max at the DDS level to the maximum value allowed by the transport (64KB) and to a smaller value (1500B). We will compare the throughput obtained in both cases.
Unkeyed, UDPv4_WAN 10Gbps Network, C++98
The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.
Note
We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.
Note
By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.
The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.
Note
By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.
The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)