TCP

The following tests have been performed by executing RTI Perftest C++98 Publisher and a Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to TCP.

Find the information about the hardware, network and command-line parameters after each of the tests.

Unkeyed, TCP 10Gbps, C++98

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

21

18.0

17

151

18

18

124

129

151

64

24

23.3

17

153

18

18

124

132

153

128

19

10.5

17

137

18

19

64

126

137

256

19

5.9

18

145

19

19

23

125

145

512

26

145.9

17

100054

22

45

69

137

100054

1024

25

136.8

18

100057

23

33

65

139

100057

2048

25

162.9

21

100056

22

27

58

132

100056

4096

28

14.8

25

8125

27

28

47

125

8125

8192

38

149.1

34

100042

37

40

75

140

100042

16384

89

36.3

42

196

67

127

167

176

196

32768

78

28.1

59

203

61

124

129

182

203

63000

96

17.0

90

241

91

100

193

229

241

  • Reliable

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

21

12.0

18

149

19

21

124

132

149

64

22

15.6

19

156

19

21

124

132

156

128

21

8.3

19

153

20

21

36

130

153

256

22

14.9

19

150

20

21

124

132

150

512

25

20.2

19

5057

21

35

124

132

5057

1024

23

13.1

20

4855

22

23

52

129

4855

2048

28

15.1

22

167

24

43

125

130

167

4096

29

3.6

27

132

28

29

41

125

132

8192

45

110.6

36

5050

39

53

125

4649

5050

16384

103

40.3

44

199

108

167

169

186

199

32768

79

27.1

61

192

62

124

130

180

192

63000

98

16.1

91

240

92

116

189

229

240

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13
 14# We will use some colors to improve visibility of errors and info messages.
 15RED='\033[0;31m'
 16GREEN='\033[0;32m'
 17YELLOW='\033[0;33m'
 18BLUE='\033[0;34m'
 19LIGHTBLUE='\033[0;36m'
 20NC='\033[0m'
 21INFO_TAG="${GREEN}[INFO]:${NC}"
 22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 23ERROR_TAG="${RED}[ERROR]:${NC}"
 24
 25################################################################################
 26
 27function disable_colors() {
 28    export RED=""
 29    export GREEN=""
 30    export YELLOW=""
 31    export NC=""
 32    export BLUE=""
 33    export LIGHTBLUE=""
 34    export INFO_TAG="${GREEN}[INFO]:${NC}"
 35    export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 36    export ERROR_TAG="${RED}[ERROR]:${NC}"
 37}
 38
 39function change_domain() {
 40    if [[ "$domain" == "1" ]]; then
 41        export domain="2"
 42    else
 43        export domain="1"
 44    fi
 45}
 46
 47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 48function execute_test() {
 49
 50    local keyed_unkeyed=$1
 51    local rel_be=$2
 52    local datasizes_test=$3
 53    local other_args=$4
 54    local name_suffix=$5
 55
 56    local commands_string_test=$commands_string
 57    local tag=""
 58
 59    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 60        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 61        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 62    else
 63        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 64    fi
 65
 66    if [[ "${rel_be}" == "be" ]]; then
 67        commands_string_test="${commands_string_test} -bestEffort"
 68        tag="${tag}${YELLOW}BE${NC}]"
 69    else
 70        tag="${tag}${RED}REL${NC}]"
 71    fi
 72
 73    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 74
 75    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 76
 77    if [[ "$role" == "pub" ]]; then
 78        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 79    fi
 80
 81    if [[ "$NO_TASKSET" == "" ]]; then
 82        if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
 83            export pre_command_string="taskset -c $core"
 84        fi
 85    fi
 86
 87    if [[ "$LANGUAGE" == "python" ]]; then
 88        export pre_command_string="python3 "
 89    fi
 90
 91    if [[ "$DOCKER" == "1" ]]; then
 92        export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
 93        executable=""
 94    fi
 95
 96    # Get the aprox time this will take:
 97    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
 98    total_time=$((total_tests * exec_time))
 99
100    touch $output_file
101    local no_headers=""
102    local current_test=0
103    for index in $(seq 1 ${num_reps}); do
104        for DATALEN in ${datasizes_test}; do
105            current_test=$((current_test + 1))
106            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107            if [[ "$role" == "pub" ]]; then
108                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109                echo -e ${BLUE}$command${NC}
110            else
111                echo -e ${LIGHTBLUE}$command${NC}
112            fi
113            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114                sleep 3
115            fi
116            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117                sleep 5
118            fi
119            if [[ "${get_netstat_info}" == "1" ]]; then
120                echo -e "${INFO_TAG} Getting netstat info before"
121                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122            fi
123            eval $command >> $output_file;
124            if [[ "${get_netstat_info}" == "1" ]]; then
125                echo -e "${INFO_TAG} Getting netstat info after"
126                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128                python3 $script_location/../../../tools/diff_netstat_output.py \
129                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131                    -d $DATALEN $no_header_netstat \
132                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134                no_header_netstat=" -nh"
135            fi
136            no_headers=" -noOutputHeaders"
137
138            change_domain
139        done
140    done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147    case $1 in
148        --executable)
149            executable=$2
150            shift
151            ;;
152        --docker)
153            DOCKER="1"
154            ;;
155        --output-folder)
156            output_folder=$2
157            shift
158            ;;
159        --sub-folder)
160            sub_folder=$2
161            shift
162            ;;
163        --role)
164            export role=$2
165            shift
166            ;;
167        --core)
168            export core=$2
169            shift
170            ;;
171        --test-kind)
172            export lat_thr=$2
173            shift
174            ;;
175        --interface1)
176            export interface=$2
177            shift
178            ;;
179        --interface2)
180            export interface2=$2
181            shift
182            ;;
183        --ip1)
184            export ip1=$2
185            shift
186            ;;
187        --ip2)
188            export ip2=$2
189            shift
190            ;;
191        --repetitions)
192            export num_reps=$2
193            shift
194            ;;
195        --domain)
196            export domain=$2
197            shift
198            ;;
199        --execution-time)
200            export exec_time=$2
201            shift
202            ;;
203        --transport)
204            export transport=$2
205            shift
206            ;;
207        --datalen)
208            export datalen_input=$2
209            shift
210            ;;
211        --file-suffix)
212            export file_suffix=$2
213            shift
214            ;;
215        --executable-suffix)
216            export executable_suffix=$2
217            shift
218            ;;
219        --extra-arguments)
220            export extra_arguments=$2
221            shift
222            ;;
223        --extra-arguments-pub)
224            export extra_arguments_pub=$2
225            shift
226            ;;
227        --extra-arguments-sub)
228            export extra_arguments_sub=$2
229            shift
230            ;;
231        --skip-no-batching)
232            export skip_no_batching="1"
233            ;;
234        --skip-be)
235            export skip_be_tests="1"
236            ;;
237        --skip-rel)
238            export skip_rel_tests="1"
239            ;;
240        --skip-keyed)
241            export skip_keyed_data="1"
242            ;;
243        --skip-large-data)
244            export skip_large_data="1"
245            ;;
246        --large-data)
247            export large_data="1"
248            ;;
249        --keyed)
250            export skip_unkeyed="1"
251            ;;
252        --unkeyed)
253            export skip_keyed_data="1"
254            ;;
255        --no-batching | --skip-batching)
256            export no_batching_only="1"
257            ;;
258        --reliable)
259            export skip_be_tests="1"
260            ;;
261        --best-effort)
262            export skip_rel_tests="1"
263            ;;
264        --security-gov)
265            export security_only="$2"
266            shift
267            ;;
268        --micro)
269            export micro="1"
270            ;;
271        --cert)
272            export cert="1"
273            ;;
274        --raw | --raw-transport)
275            export raw="1"
276            ;;
277        --tss)
278            export tss="1"
279            ;;
280        --no-colors)
281            export NO_COLORS="1"
282            ;;
283        --language)
284            export LANGUAGE=$2
285            shift
286            ;;
287        --loss-rate)
288            export loss_rate=$2
289            shift
290            ;;
291        --get-netstat-info | --netstat)
292            export get_netstat_info="1"
293            ;;
294        --no-taskset)
295            export NO_TASKSET="1"
296            ;;
297        --reduced-data-sizes-set)
298            export REDUCED_DATA_SIZES_SET="1"
299            ;;
300        *)
301            echo -e "unknown parameter \"$1\""
302            exit 255
303            ;;
304    esac
305    shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309    disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
315    export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318    export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322    export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326    export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335    export skip_keyed_data="1"
336    export skip_large_data="1"
337    export skip_be_tests="1"
338    export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342    export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344    export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349    export datasizes=${datalen_input}
350    export datasizes_extended=${datalen_input}
351    if [[ "${no_batching_only}" != "1" ]]; then
352        export skip_large_data="1"
353    fi
354else 
355    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357        export datasizes="32 128 512 2048 8192 32768 63000"
358        export datasizes_extended="${datasizes} 102400 1048576 10485760"
359    fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364    exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368    echo -e "${ERROR_TAG} It must be either lat or thr"
369    exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373    echo "Using default nics"
374    export nic_publisher=${ip_machine_1}
375    export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377    export nic_publisher="enp1s0f0,eno1"
378    export nic_subscriber="enp1s0f0,eno1"
379    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382    export nic_publisher=$interface
383    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385    if [[ "${interface2}" == "" ]]; then
386        export nic_subscriber=$interface
387    else
388        export nic_subscriber=$interface2
389    fi
390    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392    if [[ "${ip1}" != "" ]]; then
393        export ip_publisher=$ip1
394        echo "Using ip_publisher: ${ip_publisher}"
395    fi
396
397    if [[ "${ip2}" != "" ]]; then
398        export ip_subscriber=$ip2
399        echo "Using ip_subscriber: ${ip_subscriber}"
400    fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408    export transport_string_pub="$transport_string -nic $nic_publisher"
409    export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411    if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414    fi
415
416elif [[ "$transport" == "TCP" ]]; then
417    export transport_string_pub="$transport_string \
418        -nic $nic_publisher \
419        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420    export transport_string_sub="$transport_string \
421        -nic $nic_subscriber \
422        -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424    export transport_string_pub="$transport_string \
425        -nic $nic_publisher \
426        -peer tlsv4_lan://${ip_subscriber}:7400"
427    export transport_string_sub="$transport_string \
428        -nic $nic_subscriber \
429        -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431    export transport_string_pub="$transport_string \
432        -nic $nic_publisher \
433        -transportPublicAddress $ip_publisher:7400"
434    export transport_string_sub="$transport_string \
435        -nic $nic_subscriber \
436        -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438    export transport_string_pub="$transport_string"
439    export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445        ${transport_string_pub} \
446        -noPrintIntervals \
447        -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450    export pub_string="$pub_string \
451        -latencyTest"
452fi
453
454export sub_string="-sub \
455        ${transport_string_sub} \
456        -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459    echo -e "$INFO_TAG Publisher side running"
460    export commands_string=${pub_string}
461    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463    echo -e "$INFO_TAG Subscriber side running"
464    export commands_string=${sub_string}
465    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490    # UNKEYED
491    if [[ "${skip_unkeyed}" == "" ]]; then
492
493        # RELIABLE
494        if [[ "${skip_rel_tests}" == "" ]]; then
495            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496        fi
497
498        # BEST EFFORT
499        if [[ "${skip_be_tests}" == "" ]]; then
500            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501        fi
502    fi
503
504    # KEYED
505    if [[ "${skip_keyed_data}" == "" ]]; then
506
507        # RELIABLE
508        if [[ "${skip_rel_tests}" == "" ]]; then
509            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510        fi
511
512        # BEST EFFORT
513        if [[ "${skip_be_tests}" == "" ]]; then
514            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515        fi
516    fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521    no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527    if [[ "$role" == "pub" ]]; then
528        export commands_string="${commands_string} -batchSize 0"
529    fi
530
531    # UNKEYED
532    if [[ "${skip_unkeyed}" == "" ]]; then
533
534        # RELIABLE
535        if [[ "${skip_rel_tests}" == "" ]]; then
536            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537        fi
538
539        # BEST EFFORT
540        if [[ "${skip_be_tests}" == "" ]]; then
541            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542        fi
543    fi
544
545    # KEYED
546    if [[ "${skip_keyed_data}" == "" ]]; then
547
548        # RELIABLE
549        if [[ "${skip_rel_tests}" == "" ]]; then
550            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551        fi
552
553        # BEST EFFORT
554        if [[ "${skip_be_tests}" == "" ]]; then
555            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556        fi
557    fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562    echo -e "${INFO_TAG} Disabling loss rate"
563    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi

1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport TCP \
7    --skip-no-batching --skip-large-data \
8    --extra-arguments-pub "-initialBurst 100"

Test Software

The following software was used to perform these tests:

RTI Connext DDS 6.1.1 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)

Keyed, TCP 10Gbps, C++98

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

23

14.8

19

170

21

22

124

129

170

64

24

18.4

20

201

21

22

124

131

201

128

25

18.5

20

267

21

23

124

131

267

256

39

38.3

20

198

22

124

125

134

198

512

29

131.0

21

100058

26

36

69

141

100058

1024

25

44.0

21

20616

23

25

56

126

20616

2048

31

138.8

23

100063

27

40

65

142

100063

4096

30

19.7

28

8418

30

31

40

125

8418

8192

41

148.3

37

100077

39

43

81

139

100077

16384

108

38.3

46

242

110

170

172

195

242

32768

81

27.0

62

264

65

124

131

183

264

63000

99

15.7

93

240

94

104

190

229

240

  • Reliable

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

28

23.0

21

217

22

25

125

133

217

64

28

23.4

21

180

22

25

125

133

180

128

31

26.6

21

172

23

26

125

135

172

256

31

27.5

21

175

23

27

125

135

175

512

25

12.8

21

5017

23

26

55

130

5017

1024

27

12.0

22

3394

25

29

66

131

3394

2048

30

24.3

24

4536

27

33

124

132

4536

4096

32

12.3

29

4538

31

33

39

125

4538

8192

44

4.6

39

200

42

50

61

125

200

16384

109

40.0

47

209

112

171

173

192

209

32768

83

26.2

63

268

66

124

131

183

268

63000

101

15.1

94

241

96

114

188

230

241

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13
 14# We will use some colors to improve visibility of errors and info messages.
 15RED='\033[0;31m'
 16GREEN='\033[0;32m'
 17YELLOW='\033[0;33m'
 18BLUE='\033[0;34m'
 19LIGHTBLUE='\033[0;36m'
 20NC='\033[0m'
 21INFO_TAG="${GREEN}[INFO]:${NC}"
 22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 23ERROR_TAG="${RED}[ERROR]:${NC}"
 24
 25################################################################################
 26
 27function disable_colors() {
 28    export RED=""
 29    export GREEN=""
 30    export YELLOW=""
 31    export NC=""
 32    export BLUE=""
 33    export LIGHTBLUE=""
 34    export INFO_TAG="${GREEN}[INFO]:${NC}"
 35    export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 36    export ERROR_TAG="${RED}[ERROR]:${NC}"
 37}
 38
 39function change_domain() {
 40    if [[ "$domain" == "1" ]]; then
 41        export domain="2"
 42    else
 43        export domain="1"
 44    fi
 45}
 46
 47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 48function execute_test() {
 49
 50    local keyed_unkeyed=$1
 51    local rel_be=$2
 52    local datasizes_test=$3
 53    local other_args=$4
 54    local name_suffix=$5
 55
 56    local commands_string_test=$commands_string
 57    local tag=""
 58
 59    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 60        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 61        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 62    else
 63        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 64    fi
 65
 66    if [[ "${rel_be}" == "be" ]]; then
 67        commands_string_test="${commands_string_test} -bestEffort"
 68        tag="${tag}${YELLOW}BE${NC}]"
 69    else
 70        tag="${tag}${RED}REL${NC}]"
 71    fi
 72
 73    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 74
 75    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 76
 77    if [[ "$role" == "pub" ]]; then
 78        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 79    fi
 80
 81    if [[ "$NO_TASKSET" == "" ]]; then
 82        if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
 83            export pre_command_string="taskset -c $core"
 84        fi
 85    fi
 86
 87    if [[ "$LANGUAGE" == "python" ]]; then
 88        export pre_command_string="python3 "
 89    fi
 90
 91    if [[ "$DOCKER" == "1" ]]; then
 92        export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
 93        executable=""
 94    fi
 95
 96    # Get the aprox time this will take:
 97    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
 98    total_time=$((total_tests * exec_time))
 99
100    touch $output_file
101    local no_headers=""
102    local current_test=0
103    for index in $(seq 1 ${num_reps}); do
104        for DATALEN in ${datasizes_test}; do
105            current_test=$((current_test + 1))
106            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107            if [[ "$role" == "pub" ]]; then
108                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109                echo -e ${BLUE}$command${NC}
110            else
111                echo -e ${LIGHTBLUE}$command${NC}
112            fi
113            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114                sleep 3
115            fi
116            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117                sleep 5
118            fi
119            if [[ "${get_netstat_info}" == "1" ]]; then
120                echo -e "${INFO_TAG} Getting netstat info before"
121                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122            fi
123            eval $command >> $output_file;
124            if [[ "${get_netstat_info}" == "1" ]]; then
125                echo -e "${INFO_TAG} Getting netstat info after"
126                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128                python3 $script_location/../../../tools/diff_netstat_output.py \
129                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131                    -d $DATALEN $no_header_netstat \
132                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134                no_header_netstat=" -nh"
135            fi
136            no_headers=" -noOutputHeaders"
137
138            change_domain
139        done
140    done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147    case $1 in
148        --executable)
149            executable=$2
150            shift
151            ;;
152        --docker)
153            DOCKER="1"
154            ;;
155        --output-folder)
156            output_folder=$2
157            shift
158            ;;
159        --sub-folder)
160            sub_folder=$2
161            shift
162            ;;
163        --role)
164            export role=$2
165            shift
166            ;;
167        --core)
168            export core=$2
169            shift
170            ;;
171        --test-kind)
172            export lat_thr=$2
173            shift
174            ;;
175        --interface1)
176            export interface=$2
177            shift
178            ;;
179        --interface2)
180            export interface2=$2
181            shift
182            ;;
183        --ip1)
184            export ip1=$2
185            shift
186            ;;
187        --ip2)
188            export ip2=$2
189            shift
190            ;;
191        --repetitions)
192            export num_reps=$2
193            shift
194            ;;
195        --domain)
196            export domain=$2
197            shift
198            ;;
199        --execution-time)
200            export exec_time=$2
201            shift
202            ;;
203        --transport)
204            export transport=$2
205            shift
206            ;;
207        --datalen)
208            export datalen_input=$2
209            shift
210            ;;
211        --file-suffix)
212            export file_suffix=$2
213            shift
214            ;;
215        --executable-suffix)
216            export executable_suffix=$2
217            shift
218            ;;
219        --extra-arguments)
220            export extra_arguments=$2
221            shift
222            ;;
223        --extra-arguments-pub)
224            export extra_arguments_pub=$2
225            shift
226            ;;
227        --extra-arguments-sub)
228            export extra_arguments_sub=$2
229            shift
230            ;;
231        --skip-no-batching)
232            export skip_no_batching="1"
233            ;;
234        --skip-be)
235            export skip_be_tests="1"
236            ;;
237        --skip-rel)
238            export skip_rel_tests="1"
239            ;;
240        --skip-keyed)
241            export skip_keyed_data="1"
242            ;;
243        --skip-large-data)
244            export skip_large_data="1"
245            ;;
246        --large-data)
247            export large_data="1"
248            ;;
249        --keyed)
250            export skip_unkeyed="1"
251            ;;
252        --unkeyed)
253            export skip_keyed_data="1"
254            ;;
255        --no-batching | --skip-batching)
256            export no_batching_only="1"
257            ;;
258        --reliable)
259            export skip_be_tests="1"
260            ;;
261        --best-effort)
262            export skip_rel_tests="1"
263            ;;
264        --security-gov)
265            export security_only="$2"
266            shift
267            ;;
268        --micro)
269            export micro="1"
270            ;;
271        --cert)
272            export cert="1"
273            ;;
274        --raw | --raw-transport)
275            export raw="1"
276            ;;
277        --tss)
278            export tss="1"
279            ;;
280        --no-colors)
281            export NO_COLORS="1"
282            ;;
283        --language)
284            export LANGUAGE=$2
285            shift
286            ;;
287        --loss-rate)
288            export loss_rate=$2
289            shift
290            ;;
291        --get-netstat-info | --netstat)
292            export get_netstat_info="1"
293            ;;
294        --no-taskset)
295            export NO_TASKSET="1"
296            ;;
297        --reduced-data-sizes-set)
298            export REDUCED_DATA_SIZES_SET="1"
299            ;;
300        *)
301            echo -e "unknown parameter \"$1\""
302            exit 255
303            ;;
304    esac
305    shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309    disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
315    export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318    export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322    export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326    export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335    export skip_keyed_data="1"
336    export skip_large_data="1"
337    export skip_be_tests="1"
338    export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342    export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344    export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349    export datasizes=${datalen_input}
350    export datasizes_extended=${datalen_input}
351    if [[ "${no_batching_only}" != "1" ]]; then
352        export skip_large_data="1"
353    fi
354else 
355    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357        export datasizes="32 128 512 2048 8192 32768 63000"
358        export datasizes_extended="${datasizes} 102400 1048576 10485760"
359    fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364    exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368    echo -e "${ERROR_TAG} It must be either lat or thr"
369    exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373    echo "Using default nics"
374    export nic_publisher=${ip_machine_1}
375    export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377    export nic_publisher="enp1s0f0,eno1"
378    export nic_subscriber="enp1s0f0,eno1"
379    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382    export nic_publisher=$interface
383    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385    if [[ "${interface2}" == "" ]]; then
386        export nic_subscriber=$interface
387    else
388        export nic_subscriber=$interface2
389    fi
390    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392    if [[ "${ip1}" != "" ]]; then
393        export ip_publisher=$ip1
394        echo "Using ip_publisher: ${ip_publisher}"
395    fi
396
397    if [[ "${ip2}" != "" ]]; then
398        export ip_subscriber=$ip2
399        echo "Using ip_subscriber: ${ip_subscriber}"
400    fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408    export transport_string_pub="$transport_string -nic $nic_publisher"
409    export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411    if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414    fi
415
416elif [[ "$transport" == "TCP" ]]; then
417    export transport_string_pub="$transport_string \
418        -nic $nic_publisher \
419        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420    export transport_string_sub="$transport_string \
421        -nic $nic_subscriber \
422        -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424    export transport_string_pub="$transport_string \
425        -nic $nic_publisher \
426        -peer tlsv4_lan://${ip_subscriber}:7400"
427    export transport_string_sub="$transport_string \
428        -nic $nic_subscriber \
429        -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431    export transport_string_pub="$transport_string \
432        -nic $nic_publisher \
433        -transportPublicAddress $ip_publisher:7400"
434    export transport_string_sub="$transport_string \
435        -nic $nic_subscriber \
436        -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438    export transport_string_pub="$transport_string"
439    export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445        ${transport_string_pub} \
446        -noPrintIntervals \
447        -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450    export pub_string="$pub_string \
451        -latencyTest"
452fi
453
454export sub_string="-sub \
455        ${transport_string_sub} \
456        -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459    echo -e "$INFO_TAG Publisher side running"
460    export commands_string=${pub_string}
461    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463    echo -e "$INFO_TAG Subscriber side running"
464    export commands_string=${sub_string}
465    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490    # UNKEYED
491    if [[ "${skip_unkeyed}" == "" ]]; then
492
493        # RELIABLE
494        if [[ "${skip_rel_tests}" == "" ]]; then
495            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496        fi
497
498        # BEST EFFORT
499        if [[ "${skip_be_tests}" == "" ]]; then
500            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501        fi
502    fi
503
504    # KEYED
505    if [[ "${skip_keyed_data}" == "" ]]; then
506
507        # RELIABLE
508        if [[ "${skip_rel_tests}" == "" ]]; then
509            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510        fi
511
512        # BEST EFFORT
513        if [[ "${skip_be_tests}" == "" ]]; then
514            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515        fi
516    fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521    no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527    if [[ "$role" == "pub" ]]; then
528        export commands_string="${commands_string} -batchSize 0"
529    fi
530
531    # UNKEYED
532    if [[ "${skip_unkeyed}" == "" ]]; then
533
534        # RELIABLE
535        if [[ "${skip_rel_tests}" == "" ]]; then
536            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537        fi
538
539        # BEST EFFORT
540        if [[ "${skip_be_tests}" == "" ]]; then
541            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542        fi
543    fi
544
545    # KEYED
546    if [[ "${skip_keyed_data}" == "" ]]; then
547
548        # RELIABLE
549        if [[ "${skip_rel_tests}" == "" ]]; then
550            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551        fi
552
553        # BEST EFFORT
554        if [[ "${skip_be_tests}" == "" ]]; then
555            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556        fi
557    fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562    echo -e "${INFO_TAG} Disabling loss rate"
563    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi

1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport TCP \
7    --skip-no-batching --skip-large-data \
8    --extra-arguments-pub "-initialBurst 100"

Test Software

The following software was used to perform these tests:

RTI Connext DDS 6.1.1 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)