UDPv4

The following tests have been performed by executing an RTI Perftest C++98 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to UDPv4.

Find information about the hardware, network, and command-line parameters after each of the tests.

Unkeyed, UDPv4 1Gbps Network, C++98

Linux x64Linux ARM v8

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux® nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	52	1.6	48	158	52	55	56	82	158
64	55	1.8	50	118	55	57	58	84	118
128	56	1.6	51	119	56	58	60	85	119
256	61	1.5	56	158	61	63	64	90	158
512	67	1.4	62	125	67	68	70	96	125
1024	80	1.4	75	136	80	81	83	108	136
2048	108	1.2	96	158	108	109	110	136	158
4096	160	1.3	154	212	160	161	163	188	212
8192	265	1.5	258	318	265	267	268	295	318
16384	341	1.5	336	397	341	343	344	371	397
32768	474	1.4	467	536	474	475	476	503	536
63000	735	1.6	728	802	735	737	738	766	802

• Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	55	2.2	47	118	56	57	61	84	118
64	57	1.6	48	105	57	59	62	86	105
128	58	1.8	50	121	58	60	63	88	121
256	64	1.4	57	123	63	65	67	92	123
512	69	1.5	64	129	68	70	73	98	129
1024	82	1.5	76	138	82	84	86	111	138
2048	109	1.4	104	161	109	111	114	138	161
4096	162	1.3	156	211	162	163	166	191	211
8192	267	1.5	261	320	267	268	271	295	320
16384	343	1.5	337	398	343	345	346	374	398
32768	475	1.5	468	535	475	477	478	505	535
63000	737	1.7	730	804	737	738	741	772	804
100000	1044	3.9	1037	1140	1043	1050	1055	1140	1140
500000	4314	10.6	4306	4616	4312	4316	4360	4616	4616
1048576	8798	33.6	8774	9348	8795	8806	8912	9348	9348
1548576	12887	30.7	12865	13655	12885	12887	12906	13655	13655
4194304	34644	117.7	34616	36420	34633	34641	34672	36420	36420
10485760	86671	417.8	86603	90632	86616	86627	88726	90632	90632

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2
export pub_sub="pub"
export lat_thr="lat"
export num_reps="1 2"
export dataLens="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export domain="2"

if [[ -z "$3" ]]; then
    echo "You need a third argument with publisher or subscriber"
    exit -1
else
    if [[ "$3" == "publisher" ]]; then
        echo "Publisher"
        export pub_sub="pub"

    elif [[ "$3" == "subscriber" ]]; then
        echo "Subscriber"
        export pub_sub="sub"
    else
        echo "It must be either publisher or subscriber"
        exit -1
    fi
fi

if [[ -z "$4" ]]; then
    echo "You need a forth argument with lat or thr"
    exit -1
else
    if [[ "$4" == "thr" ]]; then
        echo "Throughput test"
        export ${lat_thr}_thr="thr"

    elif [[ "$4" == "lat" ]]; then
        echo "Latency test"
        export ${lat_thr}_thr="lat"
    else
        echo "It must be either lat or thr"
        exit -1
    fi
fi

if [[ -z "$5" ]]; then
    echo "Using default nics"
    export nic1=eno1
    export nic2=eno1
else
    echo "Using custom nic: $5"
    export nic1=eno1
    export nic2=eno1
fi

sudo /set_${lat_thr}_mode.sh
sleep 10

export exec_time=20

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic1 \
        -noPrint \
        -exec $exec_time \
        -noXML"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic2 \
        -noPrint \
        -noXML"

if [[ "$pub_sub" == "pub" ]]; then
    echo "Publisher side"
    export commands_string=${pub_string}
else
    echo "Subscriber side"
    export commands_string=${sub_string}
fi

mkdir -p $output_folder

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in 32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000 100000 500000 1048576 1548576 4194304 10485760; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

if [[ ${lat_thr} == "thr" ]]; then

    if [[ "$pub_sub" == "pub" ]]; then
        echo "Publisher side"
        export commands_string="${commands_string} -batchSize 0"
    fi

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

fi

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch

Unkeyed, UDPv4 10Gbps Network, C++98

Linux x64

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	15	0.8	14	65	15	15	16	44	65
64	15	0.8	14	65	15	15	17	44	65
128	15	0.9	14	67	15	16	17	45	67
256	15	1.0	15	65	15	16	17	45	65
512	19	1.8	15	71	19	21	22	47	71
1024	25	5.0	16	74	26	32	33	56	74
2048	29	36.2	20	4905	30	33	41	2106	4905
4096	32	132.7	23	100058	30	40	58	67	100058
8192	42	28.3	35	4931	36	57	65	85	4931
16384	85	23.2	45	213	84	121	135	150	213
32768	173	2.5	65	224	173	175	176	202	224
63000	185	2.9	101	223	185	187	189	216	223

• Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	17	1.1	16	70	17	18	21	47	70
64	17	1.1	16	67	17	17	21	46	67
128	17	1.2	16	73	17	17	23	46	73
256	18	2.0	16	68	17	19	28	47	68
512	20	2.7	17	69	19	21	29	49	69
1024	25	4.9	18	72	26	28	38	57	72
2048	29	36.2	20	4905	30	33	41	2106	4905
4096	31	23.8	25	4965	30	35	48	69	4965
8192	42	28.3	35	4931	36	57	65	85	4931
16384	88	25.9	47	166	87	126	140	158	166
32768	174	6.6	74	290	175	176	180	204	290
63000	162	14.3	105	220	160	186	190	214	220
100000	203	12.0	161	279	208	213	223	263	279
500000	654	22.9	558	802	661	669	699	774	802
1048576	1138	59.7	1034	1630	1180	1196	1211	1630	1630
1548576	1672	40.3	1485	2228	1660	1757	1768	2228	2228
4194304	4043	42.0	3996	5768	4045	4057	4080	5768	5768
10485760	10337	149.0	10293	14495	10329	10346	10367	14495	14495

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

echo EXECUTABLE IS $1
export executable=$1

export folder_base="$(dirname "${executable}")"/../../..

echo OUTPUT PATH IS $2
export output_folder=$2
export pub_sub="pub"
export lat_thr="lat"
export num_reps="1 2 3 4"
export dataLens="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export domain="2"

if [[ -z "$3" ]]; then
    echo "You need a third argument with publisher or subscriber"
    exit -1
else
    if [[ "$3" == "publisher" ]]; then
        echo "Publisher"
        export pub_sub="pub"
        
    elif [[ "$3" == "subscriber" ]]; then
        echo "Subscriber"
        export pub_sub="sub"
    else
        echo "It must be either publisher or subscriber"
        exit -1
    fi
fi

if [[ -z "$4" ]]; then
    echo "You need a forth argument with lat or thr"
    exit -1
else
    if [[ "$4" == "thr" ]]; then
        echo "Throughput test"
        export ${lat_thr}_thr="thr"
        
    elif [[ "$4" == "lat" ]]; then
        echo "Latency test"
        export ${lat_thr}_thr="lat"
    else
        echo "It must be either lat or thr"
        exit -1
    fi
fi

if [[ -z "$5" ]]; then
    echo "Using default nics"
    export nic1=172.16.0.1
    export nic2=172.16.0.2
else
    echo "Using custom nic: $5"
    export nic1=$5
    export nic2=$5
fi

sudo /set_${lat_thr}_mode.sh
sleep 10

export exec_time=20

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic1 \
        -noPrint \
        -exec $exec_time"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic2 \
        -noPrint"

if [[ "$pub_sub" == "pub" ]]; then
    echo "Publisher side"
    export commands_string=${pub_string}
else
    echo "Subscriber side"
    export commands_string=${sub_string}
fi


cd $folder_base
mkdir -p $output_folder

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in 32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000 100000 500000 1048576 1548576 4194304 10485760; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

if [[ ${lat_thr} == "thr" ]]; then

    if [[ "$pub_sub" == "pub" ]]; then
        echo "Publisher side"
        export commands_string="${commands_string} -batchSize 0"
    fi

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

fi

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch

Keyed, UDPv4 1Gbps Network, C++98

Linux x64Linux ARM v8

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	58	1.7	53	207	58	60	62	88	207
64	59	1.6	54	209	59	61	62	89	209
128	61	1.7	55	208	61	62	64	90	208
256	65	1.3	60	213	65	66	68	94	213
512	71	1.5	64	216	70	72	73	99	216
1024	84	1.4	78	236	84	85	87	114	236
2048	111	1.4	105	255	111	112	114	142	255
4096	164	1.5	158	311	163	165	167	194	311
8192	269	1.7	262	414	269	270	273	299	414
16384	345	1.8	338	490	345	347	348	382	490
32768	478	1.9	471	624	478	479	481	519	624
63000	739	2.1	731	883	739	740	742	810	883

• Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	61	1.7	54	206	60	62	66	89	206
64	61	1.8	55	208	61	63	66	91	208
128	63	1.6	57	212	62	64	68	92	212
256	67	1.5	61	217	67	68	71	96	217
512	73	1.6	67	221	72	74	78	102	221
1024	86	1.7	80	234	85	87	91	115	234
2048	113	1.5	107	188	113	115	118	142	188
4096	166	1.5	158	236	165	167	170	195	236
8192	271	1.6	265	326	271	272	275	303	326
16384	347	1.6	341	405	347	348	350	376	405
32768	479	1.5	472	541	479	481	482	507	541
63000	740	1.9	734	813	740	742	744	772	813

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2
export pub_sub="pub"
export lat_thr="lat"
export num_reps="1 2"
export dataLens="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export domain="2"

if [[ -z "$3" ]]; then
    echo "You need a third argument with publisher or subscriber"
    exit -1
else
    if [[ "$3" == "publisher" ]]; then
        echo "Publisher"
        export pub_sub="pub"

    elif [[ "$3" == "subscriber" ]]; then
        echo "Subscriber"
        export pub_sub="sub"
    else
        echo "It must be either publisher or subscriber"
        exit -1
    fi
fi

if [[ -z "$4" ]]; then
    echo "You need a forth argument with lat or thr"
    exit -1
else
    if [[ "$4" == "thr" ]]; then
        echo "Throughput test"
        export ${lat_thr}_thr="thr"

    elif [[ "$4" == "lat" ]]; then
        echo "Latency test"
        export ${lat_thr}_thr="lat"
    else
        echo "It must be either lat or thr"
        exit -1
    fi
fi

if [[ -z "$5" ]]; then
    echo "Using default nics"
    export nic1=eno1
    export nic2=eno1
else
    echo "Using custom nic: $5"
    export nic1=eno1
    export nic2=eno1
fi

sudo /set_${lat_thr}_mode.sh
sleep 10

export exec_time=20

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic1 \
        -noPrint \
        -exec $exec_time \
        -noXML"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic2 \
        -noPrint \
        -noXML"

if [[ "$pub_sub" == "pub" ]]; then
    echo "Publisher side"
    export commands_string=${pub_string}
else
    echo "Subscriber side"
    export commands_string=${sub_string}
fi

mkdir -p $output_folder

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in 32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000 100000 500000 1048576 1548576 4194304 10485760; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

if [[ ${lat_thr} == "thr" ]]; then

    if [[ "$pub_sub" == "pub" ]]; then
        echo "Publisher side"
        export commands_string="${commands_string} -batchSize 0"
    fi

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

fi

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch

Keyed, UDPv4 10Gbps Network, C++98

Linux x64

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	18	1.1	16	167	17	18	20	47	167
64	18	1.1	16	205	17	18	20	47	205
128	18	1.1	17	167	18	18	20	47	167
256	18	1.1	17	170	18	19	21	48	170
512	19	1.2	18	171	19	20	22	49	171
1024	25	3.0	19	178	26	28	29	54	178
2048	29	9.5	22	3315	30	32	44	68	3315
4096	31	10.6	27	3970	30	33	52	67	3970
8192	44	8.1	38	186	40	56	74	101	186
16384	90	25.9	48	237	89	129	144	158	237
32768	177	3.0	71	322	177	178	180	207	322
63000	189	2.8	106	262	189	190	193	220	262

• Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	19	1.1	18	170	19	20	23	49	170
64	19	1.2	18	167	19	20	24	49	167
128	20	1.3	18	168	19	20	25	49	168
256	20	1.7	18	167	20	21	28	50	167
512	21	5.6	19	3359	20	22	33	50	3359
1024	26	9.7	20	4551	26	28	51	59	4551
2048	29	9.5	22	3315	30	32	44	68	3315
4096	31	10.6	27	3970	30	33	52	67	3970
8192	44	8.1	38	186	40	56	74	101	186
16384	92	26.8	49	238	91	131	148	167	238
32768	174	9.5	77	249	177	179	183	207	249
63000	190	2.9	112	267	190	192	196	223	267

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

echo EXECUTABLE IS $1
export executable=$1

export folder_base="$(dirname "${executable}")"/../../..

echo OUTPUT PATH IS $2
export output_folder=$2
export pub_sub="pub"
export lat_thr="lat"
export num_reps="1 2 3 4"
export dataLens="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export domain="2"

if [[ -z "$3" ]]; then
    echo "You need a third argument with publisher or subscriber"
    exit -1
else
    if [[ "$3" == "publisher" ]]; then
        echo "Publisher"
        export pub_sub="pub"
        
    elif [[ "$3" == "subscriber" ]]; then
        echo "Subscriber"
        export pub_sub="sub"
    else
        echo "It must be either publisher or subscriber"
        exit -1
    fi
fi

if [[ -z "$4" ]]; then
    echo "You need a forth argument with lat or thr"
    exit -1
else
    if [[ "$4" == "thr" ]]; then
        echo "Throughput test"
        export ${lat_thr}_thr="thr"
        
    elif [[ "$4" == "lat" ]]; then
        echo "Latency test"
        export ${lat_thr}_thr="lat"
    else
        echo "It must be either lat or thr"
        exit -1
    fi
fi

if [[ -z "$5" ]]; then
    echo "Using default nics"
    export nic1=172.16.0.1
    export nic2=172.16.0.2
else
    echo "Using custom nic: $5"
    export nic1=$5
    export nic2=$5
fi

sudo /set_${lat_thr}_mode.sh
sleep 10

export exec_time=20

export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic1 \
        -noPrint \
        -exec $exec_time"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic2 \
        -noPrint"

if [[ "$pub_sub" == "pub" ]]; then
    echo "Publisher side"
    export commands_string=${pub_string}
else
    echo "Subscriber side"
    export commands_string=${sub_string}
fi


cd $folder_base
mkdir -p $output_folder

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in 32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000 100000 500000 1048576 1548576 4194304 10485760; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

if [[ ${lat_thr} == "thr" ]]; then

    if [[ "$pub_sub" == "pub" ]]; then
        echo "Publisher side"
        export commands_string="${commands_string} -batchSize 0"
    fi

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

    export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel_noBatch.csv
    touch $my_file
    export extra_args=""
    for index in ${num_reps}; do
        for DATALEN in ${dataLens}; do
            export command="taskset -c 0 \
            $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
            echo $command ---- $index
            $command >> $my_file;
            if [[ "$domain" == "1" ]]; then
                export domain="2"
            else
                export domain="1"
            fi
            export extra_args=" -noOutputHeaders "
        done
    done

fi

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch