UDPv4¶

The following tests have been performed by executing an RTI Perftest C++98 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to UDPv4.

Find information about the hardware, network, and command-line parameters after each of the tests.

Unkeyed, UDPv4 1Gbps Network, C++98¶

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux® nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	35	1.0	24	80	35	36	37	46	80
64	36	1.5	25	244	36	37	37	48	244
128	37	1.1	26	80	37	38	39	48	80
256	37	2.5	27	80	37	40	41	48	80
512	44	1.4	33	166	44	45	45	56	166
1024	53	1.1	43	107	53	54	55	93	107
8192	249	37.3	163	320	249	255	257	265	268
63000	666	2.7	634	755	666	668	674	727	755

Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	37	3.7	24	119	35	46	47	53	119
64	38	3.3	25	91	37	46	47	53	91
128	39	2.8	28	90	38	46	47	56	90
256	41	2.3	30	105	40	46	48	58	105
512	45	2.3	34	81	44	50	51	62	81
1024	58	12.1	43	102	54	92	93	100	102
8192	246	27.4	115	292	259	268	273	281	292
63000	668	8.6	636	783	668	671	696	748	783
100000	1056	9.5	984	1121	1057	1063	1086	1096	1121
500000	4351	32.9	4294	4766	4348	4396	4461	4766	4766
1048576	9002	19.2	8971	9615	9002	9012	9037	9615	9615
1548576	13220	32.8	13168	14191	13218	13221	13285	14191	14191
4194304	35773	131.1	35644	37978	35750	35828	35921	37978	37978
10485760	90116	475.3	89999	95697	90044	90154	92487	95697	95697

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.1.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML\
        -latencyTest"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.1.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_sub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.

The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.

Detailed Statistics

This table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps	Lost Samples	Lost Samples (%)
32	88247808	2938942	752.4	1474048	1.64
64	49448064	1647828	843.7	14613	0.03
128	26017344	867042	887.9	295936	1.12
256	13509344	450258	922.1	2400	0.02
512	6853776	228420	935.6	1200	0.02
1024	3451776	115050	942.5	600	0.02
8192	435682	14521	951.7	74	0.02
63000	56820	1893	954.4	408	0.71

Reliable

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps
32	89384193	2978096	762.4
64	49421253	1647005	843.3
128	26248769	874832	895.8
256	13502209	450014	921.6
512	6849713	228294	935.1
1024	3450033	114987	942.0
8192	435541	14516	951.3
63000	55625	2019	957.3
100000	35784	1192	954.0
500000	7187	239	957.2
1048576	3428	113	954.0
1548576	2316	76	948.8
4194304	869	28	945.0
10485760	345	11	952.3

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.1.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/thr_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.1.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
touch $my_file
export my_file=$output_folder/thr_udpv4_sub_keyed.csv
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Raspberry Pi 4 nodes using a 1Gbps network. The numbers are for best-effort as well as strict reliable delivery scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	102	6.6	97	480	101	103	120	267	480
64	104	6.9	98	644	103	105	124	278	644
128	104	6.5	98	440	103	105	122	268	440
256	108	6.1	102	394	107	109	128	261	394
512	114	9.4	106	392	111	131	139	263	392
1024	126	9.9	118	444	123	144	155	294	444
8192	231	13.0	221	684	228	237	267	537	684
63000	983	41.9	958	2561	976	1003	1037	2262	2561

Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	115	9.9	105	439	111	121	155	317	439
64	117	10.6	106	423	113	123	162	322	423
128	117	11.0	107	417	113	124	164	331	417
256	121	10.5	111	458	118	128	166	347	458
512	127	11.2	116	755	123	133	175	348	755
1024	137	10.5	127	468	134	145	183	344	468
8192	252	18.9	235	719	247	267	329	594	719
63000	1033	70.8	990	2768	1014	1074	1270	2513	2768
100000	1758	91.3	1672	5006	1758	1809	1882	5006	5006
500000	14618	2158.6	9734	48598	15178	15221	20257	48598	48598
1048576	46370	28227.7	18665	147401	33078	34078	134117	147401	147401
1548576	64804	21468.4	27804	182680	68675	76205	154373	182680	182680
4194304	163292	41851.4	79366	310288	155529	239825	310288	310288	310288
10485760	314345	18251.7	286705	378544	318726	333794	378544	378544	378544

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.1.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML\
        -latencyTest"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.1.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_sub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Raspberry Pi 4 Model B

Processor: Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
RAM: 4GB LPDDR4-3200 SDRAM
NIC 1: Gigabit Ethernet
OS: Raspbian (x32)

.. Note::
    These machines are using the most popular OS in a Raspberry, which is
    **Raspbian** (Devian for Raspberry). This however is not the ideal choice
    in terms of performance, as it is a 32 bits OS in a 64 bits CPU.

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the maximum throughput between a Publisher and a Subscriber running in two Raspberry Pi 4 nodes using a 1Gbps network. The numbers are for best-effort as well as strict reliable delivery scenarios.

Note

By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.

The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.

Detailed Statistics

This table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps	Lost Samples	Lost Samples (%)
32	10674175	355947	91.1	128000	1.18
64	8968320	298707	152.9	96000	1.06
128	7052307	234898	240.5	1280	0.02
256	4922752	163961	335.8	174816	3.43
512	3080672	102635	420.4	175568	5.39
1024	1624488	54124	443.4	264448	14.00
8192	277823	9255	606.6	19066	6.42
63000	55064	1834	924.6	1988	3.48

Reliable

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps
32	9428814	314067	80.4
64	8282987	275888	141.3
128	6484032	215984	221.2
256	4556640	151789	310.9
512	2713664	90393	370.3
1024	1510688	50319	412.2
8192	212952	7093	464.9
63000	41837	1393	702.4
100000	28289	941	753.4
500000	6205	206	826.5
1048576	2490	83	696.4
1548576	1621	54	669.0
4194304	573	19	642.3
10485760	214	7	600.0

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=169.254.61.198
export nic=169.254.2.242
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -peer $peer \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time"

mkdir -p $output_folder

echo ">> RAW"
export my_file=$output_folder/thr_udpv4_pub_raw.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -raw $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML"

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;


echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/thr_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=169.254.61.198
export peer=169.254.2.242
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -peer $peer \
        -noPrint \
        -noOutputHeaders"

mkdir -p $output_folder

echo ">> RAW"
export my_file=$output_folder/thr_udpv4_sub_raw.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -raw $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="\
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
touch $my_file
export my_file=$output_folder/thr_udpv4_sub_keyed.csv
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Raspberry Pi 4 Model B

Processor: Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
RAM: 4GB LPDDR4-3200 SDRAM
NIC 1: Gigabit Ethernet
OS: Raspbian (x32)

.. Note::
    These machines are using the most popular OS in a Raspberry, which is
    **Raspbian** (Devian for Raspberry). This however is not the ideal choice
    in terms of performance, as it is a 32 bits OS in a 64 bits CPU.

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

Unkeyed, UDPv4 10Gbps Network, C++98¶

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	20	0.8	19	319	20	21	22	29	319
64	20	0.7	20	59	20	21	22	29	59
128	21	0.6	20	85	20	21	22	28	85
256	22	0.6	21	86	21	22	23	30	86
512	22	0.7	21	63	22	23	24	31	63
1024	23	0.6	23	69	23	24	25	32	69
8192	43	0.8	42	187	43	44	45	54	187
63000	108	0.9	106	152	108	109	112	124	152

Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	22	0.8	20	80	21	22	25	32	80
64	22	0.7	21	80	21	22	25	33	80
128	22	0.7	21	66	22	22	25	34	66
256	23	0.8	21	67	22	23	26	33	67
512	23	0.7	22	60	23	24	26	34	60
1024	25	0.7	24	68	25	25	28	39	68
8192	45	0.9	43	107	45	45	48	59	107
63000	109	1.1	107	153	109	110	114	129	153
100000	151	3.9	147	337	150	153	166	185	337
500000	577	19.5	506	830	575	580	679	706	830
1048576	1072	98.9	996	8720	1008	1150	1230	1655	8720
1548576	1670	29.4	1447	2409	1668	1705	1717	2409	2409
4194304	4109	46.6	4065	6411	4114	4124	4137	6411	6411
10485760	11098	170.7	11068	16902	11091	11105	11122	16902	16902

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.0.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML\
        -latencyTest"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.0.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_sub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.

The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.

Detailed Statistics

This table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps	Lost Samples	Lost Samples (%)
32	99848960	5033936	1288.7	151040	0.15
64	99973760	4668791	2390.4	26240	0.03
128	100000000	4111291	4210.0	0	0.00
256	100000000	3340956	6842.3	0	0.00
512	69812480	2335490	9575.2	0	0.00
1024	35708504	1190222	9750.3	0	0.00
8192	4508508	150280	9848.8	0	0.00
63000	590273	19675	9916.5	2	0.00

Reliable

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps
32	100000000	4776439	1222.8
64	100000000	4333897	2219.0
128	100000000	3470061	3553.3
256	82417984	2744986	5621.7
512	56613633	1885618	7723.5
1024	35265288	1174554	9621.9
8192	4506938	150221	9844.9
63000	590289	19674	9916.0
100000	371720	12388	9911.1
500000	68000	2265	9060.3
1048576	29076	968	8124.9
1548576	22044	734	9098.6
4194304	7793	259	8702.6
10485760	2911	96	8129.6

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.0.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/thr_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.0.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
touch $my_file
export my_file=$output_folder/thr_udpv4_sub_keyed.csv
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

Keyed, UDPv4 1Gbps Network, C++98¶

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	37	0.9	27	162	37	38	39	48	162
64	38	1.1	27	226	38	39	40	53	226
128	39	1.1	29	229	39	40	42	54	229
256	41	1.3	30	237	41	42	44	54	237
512	45	1.5	35	232	45	46	48	61	232
1024	54	1.5	45	178	55	56	57	97	178
8192	259	6.2	141	454	260	261	266	279	454
63000	671	2.7	650	859	671	672	678	725	859

Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	39	2.8	28	224	38	45	48	57	224
64	39	2.7	31	260	39	44	48	58	260
128	40	2.3	30	161	40	46	47	58	161
256	42	1.7	32	228	42	44	48	59	228
512	48	2.1	38	231	47	50	54	65	231
1024	62	12.4	47	177	56	94	95	107	177
8192	257	16.0	123	358	260	268	273	282	358
63000	673	7.6	633	779	672	675	704	750	779

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.1.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML\
        -latencyTest"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.1.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_sub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.

The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.

Detailed Statistics

This table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps	Lost Samples	Lost Samples (%)
32	37718272	1256087	321.6	363776	0.96
64	36701413	1222108	625.7	35712	0.10
128	22089792	735586	753.2	341696	1.52
256	12365280	411959	843.7	2176	0.02
512	6494320	216417	886.4	86144	1.31
1024	3377296	112565	922.1	976	0.03
8192	434530	14481	949.1	75	0.02
63000	53821	1793	904.1	3428	5.99

Reliable

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps
32	33419008	1112833	284.9
64	33398528	1112212	569.5
128	22349760	744452	762.3
256	12355776	411753	843.3
512	6562305	218708	895.8
1024	3375553	112503	921.6
8192	434331	14476	948.7
63000	54772	1825	919.9

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.1.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/thr_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.1.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
touch $my_file
export my_file=$output_folder/thr_udpv4_sub_keyed.csv
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Raspberry Pi 4 nodes using a 1Gbps network. The numbers are for best-effort as well as strict reliable delivery scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	125	8.6	115	876	124	131	150	328	876
64	124	8.7	113	919	123	130	151	315	919
128	126	8.7	116	881	125	132	151	316	881
256	129	8.8	119	885	127	135	155	332	885
512	134	10.3	123	887	132	141	167	325	887
1024	148	12.5	137	530	144	167	188	333	530
8192	254	15.7	240	763	251	264	293	580	763
63000	1022	60.0	999	5890	1018	1033	1070	2709	5890

Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	140	13.2	125	950	136	151	193	376	950
64	143	14.0	127	898	139	154	201	395	898
128	143	14.9	128	2033	139	154	200	400	2033
256	147	13.3	132	555	144	158	205	390	555
512	158	13.9	142	563	155	170	212	436	563
1024	164	14.0	148	567	161	177	221	413	567
8192	284	22.1	262	806	278	304	369	665	806
63000	1079	70.6	1033	2916	1060	1131	1282	2762	2916

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.1.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML\
        -latencyTest"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.1.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_sub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Raspberry Pi 4 Model B

Processor: Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
RAM: 4GB LPDDR4-3200 SDRAM
NIC 1: Gigabit Ethernet
OS: Raspbian (x32)

.. Note::
    These machines are using the most popular OS in a Raspberry, which is
    **Raspbian** (Devian for Raspberry). This however is not the ideal choice
    in terms of performance, as it is a 32 bits OS in a 64 bits CPU.

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the maximum throughput between a Publisher and a Subscriber running in two Raspberry Pi 4 nodes using a 1Gbps network. The numbers are for best-effort as well as strict reliable delivery scenarios.

Note

By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.

The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.

Detailed Statistics

This table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps	Lost Samples	Lost Samples (%)
32	3661056	122098	31.3	23552	0.64
64	3423835	114061	58.4	16896	0.49
128	3172816	105678	108.2	14400	0.45
256	2672256	88997	182.3	4992	0.19
512	2088144	69568	285.0	14832	0.71
1024	1333009	44403	363.8	88216	6.21
8192	259592	8650	566.9	23382	8.26
63000	45210	1506	759.3	11708	20.57

Reliable

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps
32	3070721	102094	26.1
64	3088000	102885	52.7
128	2805312	93442	95.7
256	2258624	75241	154.1
512	1753536	58405	239.2
1024	1096040	36509	299.1
8192	231675	7719	505.9
63000	43893	1462	736.9

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=169.254.61.198
export nic=169.254.2.242
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -peer $peer \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time"

mkdir -p $output_folder

echo ">> RAW"
export my_file=$output_folder/thr_udpv4_pub_raw.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -raw $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML"

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;


echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/thr_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=169.254.61.198
export peer=169.254.2.242
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -peer $peer \
        -noPrint \
        -noOutputHeaders"

mkdir -p $output_folder

echo ">> RAW"
export my_file=$output_folder/thr_udpv4_sub_raw.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -raw $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="\
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
touch $my_file
export my_file=$output_folder/thr_udpv4_sub_keyed.csv
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="\
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Raspberry Pi 4 Model B

Processor: Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
RAM: 4GB LPDDR4-3200 SDRAM
NIC 1: Gigabit Ethernet
OS: Raspbian (x32)

.. Note::
    These machines are using the most popular OS in a Raspberry, which is
    **Raspbian** (Devian for Raspberry). This however is not the ideal choice
    in terms of performance, as it is a 32 bits OS in a 64 bits CPU.

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

Keyed, UDPv4 10Gbps Network, C++98¶

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	23	0.9	22	207	23	23	25	33	207
64	23	0.8	22	210	23	23	25	33	210
128	23	0.8	22	206	23	24	25	33	206
256	23	0.9	22	207	23	24	26	33	207
512	25	0.9	24	216	25	25	27	35	216
1024	26	0.9	25	223	26	26	28	36	223
8192	46	0.9	45	238	46	47	48	61	238
63000	110	1.3	108	304	110	111	114	132	304

Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	24	0.9	23	207	24	25	28	35	207
64	24	0.9	23	208	24	25	28	35	208
128	25	0.9	23	208	24	25	28	36	208
256	26	0.9	24	208	26	26	29	36	208
512	26	0.9	24	210	26	26	29	37	210
1024	27	0.9	26	209	27	28	31	38	209
8192	47	0.9	46	230	47	48	51	58	230
63000	112	1.8	110	295	112	114	120	133	295

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.0.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML\
        -latencyTest"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.0.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_sub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.

The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.

Detailed Statistics

This table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Best Effort

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps	Lost Samples	Lost Samples (%)
32	38440677	1280405	327.8	346624	0.89
64	38068864	1268282	649.4	247040	0.64
128	36341190	1210761	1239.8	168064	0.46
256	33867552	1128371	2310.9	42976	0.13
512	29695392	989227	4051.9	38384	0.13
1024	23570072	785139	6431.9	29440	0.12
8192	4495574	149847	9820.4	22	0.00
63000	600052	19768	9922.8	1	0.00

Reliable

Sample Size (Bytes)	Total Samples	Avg Samples/s	Avg Mbps
32	33848832	1127146	288.5
64	33671104	1121257	574.1
128	32957042	1097777	1124.1
256	29737376	990989	2029.5
512	25606912	853313	3495.2
1024	19447560	647895	5307.6
8192	4496569	149788	9816.5
63000	590050	19668	9912.8

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.0.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_pub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_pub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/thr_udpv4_pub_keyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $pub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_thr_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.0.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/thr_udpv4_sub_unkeyed_rel.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/thr_udpv4_sub_keyed_be.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
touch $my_file
export my_file=$output_folder/thr_udpv4_sub_keyed.csv
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9