Real-Time WAN Transport (UDPv4_WAN)

The following tests have been performed by executing an RTI Perftest C++98 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to Real-Time WAN Transport (UDPv4_WAN).

Note

Even though we are using the Real-Time WAN Transport, this testing has been performed on a LAN to be able to compare the results with the built-in UDPv4 transport. The differences are negligible as shown in the graphs below. Find more information and tests in the Real-Time WAN vs UDPv4 in LAN section.

Find information about the hardware, network, and command-line parameters after each of the tests.

Unkeyed, UDPv4_WAN 10Gbps Network, C++98

Linux x64

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Best Effort

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	16	1.0	14	54	16	16	17	44	54
64	17	1.0	16	52	17	17	18	45	52
128	16	1.0	14	124	16	16	17	44	124
256	17	0.9	16	53	17	18	19	46	53
512	19	1.1	16	49	19	20	21	46	49
1024	26	1.0	18	55	26	27	27	50	55
2048	50	1435.6	19	100056	30	31	36	100054	100056
4096	37	817.2	25	100061	30	31	42	63	100061
8192	40	8.2	34	87	35	55	61	75	87
16384	84	21.2	45	160	83	118	133	144	160
32768	172	2.1	66	203	172	173	175	202	203
63000	185	2.7	101	215	185	186	188	214	215

• Reliable

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	16	0.9	15	65	16	17	19	46	65
64	18	1.1	17	52	18	19	22	47	52
128	17	1.2	16	130	17	18	23	46	130
256	19	1.7	17	54	19	21	26	48	54
512	20	2.5	17	56	19	25	28	48	56
1024	25	3.2	19	70	26	27	35	54	70
2048	29	32.5	20	4990	30	31	36	62	4990
4096	31	16.8	26	4800	30	31	43	62	4800
8192	40	7.1	35	107	36	53	61	77	107
16384	86	26.2	46	170	85	124	137	153	170
32768	173	5.2	73	205	174	175	178	203	205
63000	186	2.6	110	220	186	187	192	216	220
100000	208	7.4	169	283	210	212	218	244	283
500000	653	23.4	556	814	660	664	714	776	814
1048576	1135	116.4	1012	8481	1156	1201	1266	8481	8481
1548576	1648	121.3	1468	7921	1641	1682	1775	7921	7921
4194304	4003	103.2	3946	7232	3999	4012	4031	7232	7232
10485760	10556	680.7	10447	21491	10488	10534	11516	21491	21491

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

echo EXECUTABLE IS $1
export executable=$1

export folder_base="$(dirname "${executable}")"/../../..

echo OUTPUT PATH IS $2
export output_folder=$2
export pub_sub="pub"
export lat_thr="lat"
export num_reps="1 2"
export dataLens="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export domain="2"

if [[ -z "$3" ]]; then
    echo "You need a third argument with publisher or subscriber"
    exit -1
else
    if [[ "$3" == "publisher" ]]; then
        echo "Publisher"
        export pub_sub="pub"
        
    elif [[ "$3" == "subscriber" ]]; then
        echo "Subscriber"
        export pub_sub="sub"
    else
        echo "It must be either publisher or subscriber"
        exit -1
    fi
fi

if [[ -z "$4" ]]; then
    echo "You need a forth argument with lat or thr"
    exit -1
else
    if [[ "$4" == "thr" ]]; then
        echo "Throughput test"
        export ${lat_thr}_thr="thr"
        
    elif [[ "$4" == "lat" ]]; then
        echo "Latency test"
        export ${lat_thr}_thr="lat"
    else
        echo "It must be either lat or thr"
        exit -1
    fi
fi

if [[ -z "$5" ]]; then
    echo "Using default nics"
    export nic1=10.2.78.20
    export nic2=10.2.78.21
else
    echo "Using custom nic NOT IMPLEMENTED YET."
    export nic1=10.2.78.20
    export nic2=10.2.78.21
fi


sudo /set_${lat_thr}_mode.sh
sleep 10

export exec_time=20

export pub_string="-pub \
        -transport UDPv4_WAN \
        -transportPublicAddress $nic1:7400 \
        -nic $nic1 \
        -noPrint \
        -exec $exec_time"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        -transport UDPv4_WAN \
        -peer 0@udpv4_wan://$nic1:7400 \
        -nic $nic2 \
        -noPrint"

if [[ "$pub_sub" == "pub" ]]; then
    echo "Publisher side"
    export commands_string=${pub_string}
else
    echo "Subscriber side"
    export commands_string=${sub_string}
fi


cd $folder_base
mkdir -p $output_folder

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in 32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000 100000 500000 1048576 1548576 4194304 10485760; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_rel.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

export my_file=$output_folder/${lat_thr}_${pub_sub}_keyed_be.csv
touch $my_file
export extra_args=""
for index in ${num_reps}; do
    for DATALEN in ${dataLens}; do
        export command="taskset -c 0 \
        $executable -domain $domain -datalen $DATALEN -keyed -instances 100000 -best $commands_string $extra_args"
        echo $command ---- $index
        $command >> $my_file;
        if [[ "$domain" == "1" ]]; then
            export domain="2"
        else
            export domain="1"
        fi
        export extra_args=" -noOutputHeaders "
    done
done

# if [[ ${lat_thr} == "thr" ]]; then

#     if [[ "$pub_sub" == "pub" ]]; then
#         echo "Publisher side"
#         export commands_string="${commands_string} -batchSize 0"
#     fi

#     export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_be_noBatch.csv
#     touch $my_file
#     export extra_args=""
#     for index in ${num_reps}; do
#         for DATALEN in ${dataLens}; do
#             export command="taskset -c 0 \
#             $executable -domain $domain -datalen $DATALEN -best $commands_string $extra_args"
#             echo $command ---- $index
#             $command >> $my_file;
#             if [[ "$domain" == "1" ]]; then
#                 export domain="2"
#             else
#                 export domain="1"
#             fi
#             export extra_args=" -noOutputHeaders "
#         done
#     done

#     export my_file=$output_folder/${lat_thr}_${pub_sub}_unkeyed_rel_noBatch.csv
#     touch $my_file
#     export extra_args=""
#     for index in ${num_reps}; do
#         for DATALEN in ${dataLens}; do
#             export command="taskset -c 0 \
#             $executable -domain $domain -datalen $DATALEN $commands_string $extra_args"
#             echo $command ---- $index
#             $command >> $my_file;
#             if [[ "$domain" == "1" ]]; then
#                 export domain="2"
#             else
#                 export domain="1"
#             fi
#             export extra_args=" -noOutputHeaders "
#         done
#     done

# fi

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch