3.1.1.1. Real-Time WAN

Real-Time WAN vs UDPv4 in LAN

In this initial test we aim to show that the performance obtained with the Real-Time WAN Transport is similar to the one obtained by using the UDPv4 transport. In other words, Real-Time WAN Transport barely adds overheads when used in LAN.

Unkeyed, UDPv4_WAN 10Gbps Network, C++98

Linux x64

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)

TCP/TLS vs Real-Time WAN and Security

The following tests compare the differences in performance between the use of TCP/TLS and Real-Time WAN Transport (UDPv4 over WAN) with and without Security in a more realistic scenario: the communications between a machine in our lab (HQ) and an Amazon Web Services (AWS) virtual machine.

Real-Time WAN Transport is a transport that enables secure, scalable, and high-performance communication over wide area networks (WANs), including public networks. It can be used to extend Connext capabilities to WAN environments. Real-Time WAN Transport uses UDPv4 as the underlying IP transport-layer protocol to better anticipate and adapt to the challenges of diverse network conditions, device mobility, and the dynamic nature of WAN system architectures. For more information, see RTI Real-Time WAN Transport, in the RTI Connext Core Libraries User’s Manual.

The following tests compare TCP vs Real-Time WAN Transport (RWT). They also compare TLS, which implements encryption over TCP, with an equivalent level of security using Security Plugins over the Real-Time WAN Transport.

AWS to HQ (Downstream)HQ to AWS (Upstream)

Minimum LatencyMaximum Throughput (Batching)

Data Tables

• TCP

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)	CPU (%)
32	56513	14895.8	33342	67423	66165	66662	66886	67423	67423	1.46
64	58089	14076.4	33344	67196	66373	66632	66696	67196	67196	1.53
128	52247	15623.7	33317	68177	64415	66504	67374	68177	68177	1.43
256	45870	12688.9	33333	66482	37122	63221	64828	66482	66482	1.52
512	44657	8328.1	33396	58792	43262	55690	58237	58792	58792	1.62
1024	45735	9063.0	33375	58479	42885	57472	58323	58479	58479	1.52
2048	45714	6598.6	33318	55801	47170	53022	54134	55801	55801	1.66
4096	45500	12585.6	33315	63342	37913	62437	63234	63342	63342	1.57
8192	45531	7616.5	33659	56629	45414	54773	56410	56629	56629	1.56
16384	45135	8924.8	33634	58757	41972	57825	58459	58757	58757	1.62
32768	45282	8389.4	33732	100317	44533	54253	56350	100317	100317	1.76
63000	45459	10948.8	34140	167265	46374	52047	53323	167265	167265	1.89
100000	47832	16429.1	34480	200825	38074	63125	63873	200825	200825	2.26
200000	70078	16839.4	68067	267995	68263	68583	100636	267995	267995	2.26
300000	129572	27437.3	101993	334909	135258	136129	334909	334909	334909	1.61
500000	182037	36188.5	169316	429641	170131	202979	429641	429641	429641	1.57
1048576	441878	36148.1	406666	604516	438370	441713	604516	604516	604516	1.34

• TLS

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)	CPU (%)
32	56513	14895.8	33342	67423	66165	66662	66886	67423	67423	1.46
64	58089	14076.4	33344	67196	66373	66632	66696	67196	67196	1.53
128	52247	15623.7	33317	68177	64415	66504	67374	68177	68177	1.43
256	45870	12688.9	33333	66482	37122	63221	64828	66482	66482	1.52
512	44657	8328.1	33396	58792	43262	55690	58237	58792	58792	1.62
1024	45735	9063.0	33375	58479	42885	57472	58323	58479	58479	1.52
2048	45714	6598.6	33318	55801	47170	53022	54134	55801	55801	1.66
4096	45500	12585.6	33315	63342	37913	62437	63234	63342	63342	1.57
8192	45531	7616.5	33659	56629	45414	54773	56410	56629	56629	1.56
16384	45135	8924.8	33634	58757	41972	57825	58459	58757	58757	1.62
32768	45282	8389.4	33732	100317	44533	54253	56350	100317	100317	1.76
63000	45459	10948.8	34140	167265	46374	52047	53323	167265	167265	1.89
100000	47832	16429.1	34480	200825	38074	63125	63873	200825	200825	2.26
200000	70078	16839.4	68067	267995	68263	68583	100636	267995	267995	2.26
300000	129572	27437.3	101993	334909	135258	136129	334909	334909	334909	1.61
500000	182037	36188.5	169316	429641	170131	202979	429641	429641	429641	1.57
1048576	441878	36148.1	406666	604516	438370	441713	604516	604516	604516	1.34

• RWT (No Security)

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)	CPU (%)
32	33120	151.6	33041	34835	33088	33141	33775	34835	34835	1.46
64	33100	78.4	33038	33755	33083	33126	33463	33755	33755	1.46
128	33098	49.9	33029	33441	33088	33119	33377	33441	33441	1.46
256	33133	102.4	33039	33971	33105	33186	33738	33971	33971	1.54
512	33146	96.5	33064	33979	33125	33171	33730	33979	33979	1.51
1024	33203	128.2	33130	35021	33174	33271	33619	35021	35021	1.55
2048	33370	2202.3	33165	71419	33224	33283	33881	71419	71419	1.46
4096	33369	176.3	33216	35475	33312	33520	34076	35475	35475	1.42
8192	33542	245.1	33371	35081	33455	33728	34710	35081	35081	1.55
16384	33840	133.8	33664	35177	33822	33933	34520	35177	35177	2.01
32768	34561	641.9	34291	42429	34477	34672	38765	42429	42429	2.32
63000	43309	14773.6	34792	74219	35463	70384	73205	74219	74219	2.64
100000	38143	233.0	37885	39789	38096	38407	38932	39789	39789	2.19
200000	41480	334.6	41147	43421	41359	41847	43006	43421	43421	2.69
300000	44838	410.9	44466	47410	44675	45296	46821	47410	47410	2.91
500000	52639	2783.1	51122	86690	52120	54400	58473	86690	86690	3.55
1048576	153956	12409.0	104219	177577	153248	171977	177577	177577	177577	3.16

• RWT (RTPS Sign + Encrypt)

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)	CPU (%)
32	33123	132.8	33055	34731	33097	33158	33654	34731	34731	1.78
64	33154	205.7	33018	35093	33104	33208	34105	35093	35093	1.81
128	33140	150.9	33051	34837	33103	33186	33819	34837	34837	1.74
256	33153	87.2	33049	33742	33118	33266	33541	33742	33742	1.78
512	33163	107.7	33060	34415	33137	33218	33586	34415	34415	1.83
1024	33227	216.0	33106	35591	33175	33264	34272	35591	35591	1.73
2048	33327	2139.2	33128	70276	33180	33239	34013	70276	70276	1.74
4096	33283	143.4	33187	34889	33249	33357	33868	34889	34889	1.83
8192	33448	155.0	33309	34622	33398	33586	34382	34622	34622	1.82
16384	34116	145.9	33950	35284	34083	34252	34922	35284	35284	1.92
32768	35096	128.7	34928	36186	35066	35206	35796	36186	36186	2.05
63000	36838	231.5	36342	39799	36810	36982	37188	39799	39799	6.01
100000	38076	249.2	37827	40478	38007	38282	39070	40478	40478	3.33
200000	41518	275.8	41191	42922	41425	41865	42670	42922	42922	4.05
300000	44895	472.6	44590	48221	44769	45230	47457	48221	48221	4.96
500000	52658	1262.9	51200	57150	52351	54487	56292	57150	57150	6.30
1048576	151806	12836.5	104333	176735	152939	167952	176735	176735	176735	6.05