1.2.2. Modern C++ API

The following tests have been performed by executing an RTI Perftest C++11 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to UDPv4.

Find information about the hardware, network, and command-line parameters after each of the tests.

1.2.2.1. Unkeyed, UDPv4 10Gbps Network, C++11

Linux x64

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	16	0.9	15	62	16	17	18	46	62
64	16	1.0	15	68	16	17	18	46	68
128	16	1.0	16	55	16	17	18	46	55
256	16	1.0	16	58	16	17	18	46	58
512	19	1.5	16	59	19	21	22	48	59
1024	25	3.7	18	66	26	28	33	55	66
2048	42	151.1	19	100084	41	62	65	79	100084
4096	33	133.8	24	100090	30	43	59	67	100090
8192	38	192.6	33	100100	36	64	70	86	100100
16384	90	23.4	46	153	80	121	126	140	153
32768	174	2.4	90	227	174	176	177	203	227
63000	186	2.7	103	222	186	188	190	216	222

• Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	17	1.2	16	64	17	18	22	47	64
64	17	1.2	16	76	17	18	22	47	76
128	17	1.2	16	57	17	18	24	47	57
256	18	1.4	17	62	18	20	22	49	62
512	20	2.5	18	62	19	22	29	49	62
1024	25	9.6	19	5057	26	28	43	57	5057
2048	29	21.8	21	5014	30	32	42	67	5014
4096	31	16.6	26	4463	30	34	49	67	4463
8192	42	49.9	35	5032	37	57	66	3390	5032
16384	92	26.7	48	4086	82	124	130	145	4086
32768	158	17.9	75	205	163	176	179	202	205
63000	167	15.8	111	219	164	188	191	216	219
100000	202	19.7	158	2991	200	224	228	254	2991
500000	585	31.8	507	907	585	624	666	735	907
1048576	1208	111.3	1020	2163	1205	1316	1754	2163	2163
1548576	1586	78.3	1467	2484	1628	1667	1689	2484	2484
4194304	4224	204.9	4034	6639	4091	4519	4608	6639	6639
10485760	10314	215.6	10266	16674	10303	10336	10355	16674	16674

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++11. The exact script used can be found here:

echo EXECUTABLE IS $1
export executable=$1

export folder_base="$(dirname "${executable}")"/../../..

echo OUTPUT PATH IS $2
export output_folder=$2
export pub_sub="pub"
export lat_thr="lat"
export num_reps="1 2 3 4"
export dataLens="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export domain="2"

if [[ -z "$3" ]]; then
    echo "You need a third argument with publisher or subscriber"
    exit -1
else
    if [[ "$3" == "publisher" ]]; then
        echo "Publisher"
        export pub_sub="pub"
        
    elif [[ "$3" == "subscriber" ]]; then
        echo "Subscriber"
        export pub_sub="sub"
    else
        echo "It must be either publisher or subscriber"
        exit -1
    fi
fi

if [[ -z "$4" ]]; then
    echo "You need a forth argument with lat or thr"
    exit -1
else
    if [[ "$4" == "thr" ]]; then
        echo "Throughput test"
        export ${lat_thr}_thr="thr"
        
    elif [[ "$4" == "lat" ]]; then
        echo "Latency test"
        export ${lat_thr}_thr="lat"
    else
        echo "It must be either lat or thr"
        exit -1
    fi
fi

if [[ -z "$5" ]]; then
    echo "Using default nics"
    export nic1=172.16.0.1
    export nic2=172.16.0.2
else
    echo "Using custom nic: $5"
    export nic1=$5
    export nic2=$5
fi

sudo /set_${lat_thr}_mode.sh
sleep 10

export exec_time=20

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic1 \
        -noPrint \
        -exec $exec_time"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic2 \
        -noPrint"

if [[ "$pub_sub" == "pub" ]]; then
    echo "Publisher side"
    export commands_string=${pub_string}
else
    echo "Subscriber side"
    export commands_string=${sub_string}
fi


cd $folder_base
mkdir -p $output_folder

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in 32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000 100000 500000 1048576 1548576 4194304 10485760; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

if [[ ${lat_thr} == "thr" ]]; then

    if [[ "$pub_sub" == "pub" ]]; then
        echo "Publisher side"
        export commands_string="${commands_string} -batchSize 0"
    fi

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

fi

Test Software

The following software was used to perform these tests:

RTI Connext DDS 7.0.0 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch

1.2.2.2. Keyed, UDPv4 10Gbps Network, C++11

Linux x64

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	19	1.2	17	160	18	19	21	49	160
64	18	1.1	17	171	18	19	21	48	171
128	18	1.1	17	161	18	19	21	48	161
256	20	1.2	18	162	19	20	22	50	162
512	19	1.1	18	159	19	20	22	49	159
1024	25	2.5	20	177	26	28	29	56	177
2048	29	3.0	23	173	30	32	33	59	173
4096	30	1.2	27	172	30	31	32	59	172
8192	45	7.4	37	190	41	58	62	80	190
16384	94	25.5	49	240	84	128	134	154	240
32768	177	2.2	94	315	177	178	180	214	315
63000	189	2.8	106	329	189	190	192	219	329

• Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	21	1.2	19	161	20	21	24	51	161
64	20	1.2	19	161	20	21	25	50	161
128	21	1.3	19	160	20	21	27	50	160
256	21	1.9	19	162	20	22	31	50	162
512	22	2.1	20	161	21	23	30	52	161
1024	26	7.7	21	2819	26	28	43	57	2819
2048	29	7.5	23	2566	30	32	47	66	2566
4096	31	10.1	28	4321	30	32	48	66	4321
8192	45	9.3	39	960	41	55	90	119	960
16384	95	25.4	50	267	85	129	136	151	267
32768	158	18.1	80	225	158	178	181	205	225
63000	182	14.3	115	265	190	192	195	221	265

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++11. The exact script used can be found here:

echo EXECUTABLE IS $1
export executable=$1

export folder_base="$(dirname "${executable}")"/../../..

echo OUTPUT PATH IS $2
export output_folder=$2
export pub_sub="pub"
export lat_thr="lat"
export num_reps="1 2 3 4"
export dataLens="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export domain="2"

if [[ -z "$3" ]]; then
    echo "You need a third argument with publisher or subscriber"
    exit -1
else
    if [[ "$3" == "publisher" ]]; then
        echo "Publisher"
        export pub_sub="pub"
        
    elif [[ "$3" == "subscriber" ]]; then
        echo "Subscriber"
        export pub_sub="sub"
    else
        echo "It must be either publisher or subscriber"
        exit -1
    fi
fi

if [[ -z "$4" ]]; then
    echo "You need a forth argument with lat or thr"
    exit -1
else
    if [[ "$4" == "thr" ]]; then
        echo "Throughput test"
        export ${lat_thr}_thr="thr"
        
    elif [[ "$4" == "lat" ]]; then
        echo "Latency test"
        export ${lat_thr}_thr="lat"
    else
        echo "It must be either lat or thr"
        exit -1
    fi
fi

if [[ -z "$5" ]]; then
    echo "Using default nics"
    export nic1=172.16.0.1
    export nic2=172.16.0.2
else
    echo "Using custom nic: $5"
    export nic1=$5
    export nic2=$5
fi

sudo /set_${lat_thr}_mode.sh
sleep 10

export exec_time=20

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic1 \
        -noPrint \
        -exec $exec_time"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic2 \
        -noPrint"

if [[ "$pub_sub" == "pub" ]]; then
    echo "Publisher side"
    export commands_string=${pub_string}
else
    echo "Subscriber side"
    export commands_string=${sub_string}
fi


cd $folder_base
mkdir -p $output_folder

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in 32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000 100000 500000 1048576 1548576 4194304 10485760; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

if [[ ${lat_thr} == "thr" ]]; then

    if [[ "$pub_sub" == "pub" ]]; then
        echo "Publisher side"
        export commands_string="${commands_string} -batchSize 0"
    fi

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

fi

Test Software

The following software was used to perform these tests:

RTI Connext DDS 7.0.0 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch