UDPv4

The following tests have been performed by executing an RTI Perftest C++98 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to UDPv4.

Find information about the hardware, network, and command-line parameters after each of the tests.

Unkeyed, UDPv4 1Gbps Network, C++98

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux® nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

52

1.6

48

158

52

55

56

82

158

64

55

1.8

50

118

55

57

58

84

118

128

56

1.6

51

119

56

58

60

85

119

256

61

1.5

56

158

61

63

64

90

158

512

67

1.4

62

125

67

68

70

96

125

1024

80

1.4

75

136

80

81

83

108

136

2048

108

1.2

96

158

108

109

110

136

158

4096

160

1.3

154

212

160

161

163

188

212

8192

265

1.5

258

318

265

267

268

295

318

16384

341

1.5

336

397

341

343

344

371

397

32768

474

1.4

467

536

474

475

476

503

536

63000

735

1.6

728

802

735

737

738

766

802

  • Reliable

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

55

2.2

47

118

56

57

61

84

118

64

57

1.6

48

105

57

59

62

86

105

128

58

1.8

50

121

58

60

63

88

121

256

64

1.4

57

123

63

65

67

92

123

512

69

1.5

64

129

68

70

73

98

129

1024

82

1.5

76

138

82

84

86

111

138

2048

109

1.4

104

161

109

111

114

138

161

4096

162

1.3

156

211

162

163

166

191

211

8192

267

1.5

261

320

267

268

271

295

320

16384

343

1.5

337

398

343

345

346

374

398

32768

475

1.5

468

535

475

477

478

505

535

63000

737

1.7

730

804

737

738

741

772

804

100000

1044

3.9

1037

1140

1043

1050

1055

1140

1140

500000

4314

10.6

4306

4616

4312

4316

4360

4616

4616

1048576

8798

33.6

8774

9348

8795

8806

8912

9348

9348

1548576

12887

30.7

12865

13655

12885

12887

12906

13655

13655

4194304

34644

117.7

34616

36420

34633

34641

34672

36420

36420

10485760

86671

417.8

86603

90632

86616

86627

88726

90632

90632

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13
 14# We will use some colors to improve visibility of errors and info messages.
 15RED='\033[0;31m'
 16GREEN='\033[0;32m'
 17YELLOW='\033[0;33m'
 18BLUE='\033[0;34m'
 19LIGHTBLUE='\033[0;36m'
 20NC='\033[0m'
 21INFO_TAG="${GREEN}[INFO]:${NC}"
 22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 23ERROR_TAG="${RED}[ERROR]:${NC}"
 24
 25################################################################################
 26
 27function disable_colors() {
 28    export RED=""
 29    export GREEN=""
 30    export YELLOW=""
 31    export NC=""
 32    export BLUE=""
 33    export LIGHTBLUE=""
 34    export INFO_TAG="${GREEN}[INFO]:${NC}"
 35    export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 36    export ERROR_TAG="${RED}[ERROR]:${NC}"
 37}
 38
 39function change_domain() {
 40    if [[ "$domain" == "1" ]]; then
 41        export domain="2"
 42    else
 43        export domain="1"
 44    fi
 45}
 46
 47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 48function execute_test() {
 49
 50    local keyed_unkeyed=$1
 51    local rel_be=$2
 52    local datasizes_test=$3
 53    local other_args=$4
 54    local name_suffix=$5
 55
 56    local commands_string_test=$commands_string
 57    local tag=""
 58
 59    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 60        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 61        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 62    else
 63        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 64    fi
 65
 66    if [[ "${rel_be}" == "be" ]]; then
 67        commands_string_test="${commands_string_test} -bestEffort"
 68        tag="${tag}${YELLOW}BE${NC}]"
 69    else
 70        tag="${tag}${RED}REL${NC}]"
 71    fi
 72
 73    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 74
 75    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 76
 77    if [[ "$role" == "pub" ]]; then
 78        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 79    fi
 80
 81    if [[ "$NO_TASKSET" == "" ]]; then
 82        if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
 83            export pre_command_string="taskset -c $core"
 84        fi
 85    fi
 86
 87    if [[ "$LANGUAGE" == "python" ]]; then
 88        export pre_command_string="python3 "
 89    fi
 90
 91    if [[ "$DOCKER" == "1" ]]; then
 92        export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
 93        executable=""
 94    fi
 95
 96    # Get the aprox time this will take:
 97    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
 98    total_time=$((total_tests * exec_time))
 99
100    touch $output_file
101    local no_headers=""
102    local current_test=0
103    for index in $(seq 1 ${num_reps}); do
104        for DATALEN in ${datasizes_test}; do
105            current_test=$((current_test + 1))
106            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107            if [[ "$role" == "pub" ]]; then
108                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109                echo -e ${BLUE}$command${NC}
110            else
111                echo -e ${LIGHTBLUE}$command${NC}
112            fi
113            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114                sleep 3
115            fi
116            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117                sleep 5
118            fi
119            if [[ "${get_netstat_info}" == "1" ]]; then
120                echo -e "${INFO_TAG} Getting netstat info before"
121                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122            fi
123            eval $command >> $output_file;
124            if [[ "${get_netstat_info}" == "1" ]]; then
125                echo -e "${INFO_TAG} Getting netstat info after"
126                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128                python3 $script_location/../../../tools/diff_netstat_output.py \
129                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131                    -d $DATALEN $no_header_netstat \
132                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134                no_header_netstat=" -nh"
135            fi
136            no_headers=" -noOutputHeaders"
137
138            change_domain
139        done
140    done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147    case $1 in
148        --executable)
149            executable=$2
150            shift
151            ;;
152        --docker)
153            DOCKER="1"
154            ;;
155        --output-folder)
156            output_folder=$2
157            shift
158            ;;
159        --sub-folder)
160            sub_folder=$2
161            shift
162            ;;
163        --role)
164            export role=$2
165            shift
166            ;;
167        --core)
168            export core=$2
169            shift
170            ;;
171        --test-kind)
172            export lat_thr=$2
173            shift
174            ;;
175        --interface1)
176            export interface=$2
177            shift
178            ;;
179        --interface2)
180            export interface2=$2
181            shift
182            ;;
183        --ip1)
184            export ip1=$2
185            shift
186            ;;
187        --ip2)
188            export ip2=$2
189            shift
190            ;;
191        --repetitions)
192            export num_reps=$2
193            shift
194            ;;
195        --domain)
196            export domain=$2
197            shift
198            ;;
199        --execution-time)
200            export exec_time=$2
201            shift
202            ;;
203        --transport)
204            export transport=$2
205            shift
206            ;;
207        --datalen)
208            export datalen_input=$2
209            shift
210            ;;
211        --file-suffix)
212            export file_suffix=$2
213            shift
214            ;;
215        --executable-suffix)
216            export executable_suffix=$2
217            shift
218            ;;
219        --extra-arguments)
220            export extra_arguments=$2
221            shift
222            ;;
223        --extra-arguments-pub)
224            export extra_arguments_pub=$2
225            shift
226            ;;
227        --extra-arguments-sub)
228            export extra_arguments_sub=$2
229            shift
230            ;;
231        --skip-no-batching)
232            export skip_no_batching="1"
233            ;;
234        --skip-be)
235            export skip_be_tests="1"
236            ;;
237        --skip-rel)
238            export skip_rel_tests="1"
239            ;;
240        --skip-keyed)
241            export skip_keyed_data="1"
242            ;;
243        --skip-large-data)
244            export skip_large_data="1"
245            ;;
246        --large-data)
247            export large_data="1"
248            ;;
249        --keyed)
250            export skip_unkeyed="1"
251            ;;
252        --unkeyed)
253            export skip_keyed_data="1"
254            ;;
255        --no-batching | --skip-batching)
256            export no_batching_only="1"
257            ;;
258        --reliable)
259            export skip_be_tests="1"
260            ;;
261        --best-effort)
262            export skip_rel_tests="1"
263            ;;
264        --security-gov)
265            export security_only="$2"
266            shift
267            ;;
268        --micro)
269            export micro="1"
270            ;;
271        --cert)
272            export cert="1"
273            ;;
274        --raw | --raw-transport)
275            export raw="1"
276            ;;
277        --tss)
278            export tss="1"
279            ;;
280        --no-colors)
281            export NO_COLORS="1"
282            ;;
283        --language)
284            export LANGUAGE=$2
285            shift
286            ;;
287        --loss-rate)
288            export loss_rate=$2
289            shift
290            ;;
291        --get-netstat-info | --netstat)
292            export get_netstat_info="1"
293            ;;
294        --no-taskset)
295            export NO_TASKSET="1"
296            ;;
297        --reduced-data-sizes-set)
298            export REDUCED_DATA_SIZES_SET="1"
299            ;;
300        *)
301            echo -e "unknown parameter \"$1\""
302            exit 255
303            ;;
304    esac
305    shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309    disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
315    export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318    export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322    export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326    export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335    export skip_keyed_data="1"
336    export skip_large_data="1"
337    export skip_be_tests="1"
338    export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342    export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344    export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349    export datasizes=${datalen_input}
350    export datasizes_extended=${datalen_input}
351    if [[ "${no_batching_only}" != "1" ]]; then
352        export skip_large_data="1"
353    fi
354else 
355    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357        export datasizes="32 128 512 2048 8192 32768 63000"
358        export datasizes_extended="${datasizes} 102400 1048576 10485760"
359    fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364    exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368    echo -e "${ERROR_TAG} It must be either lat or thr"
369    exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373    echo "Using default nics"
374    export nic_publisher=${ip_machine_1}
375    export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377    export nic_publisher="enp1s0f0,eno1"
378    export nic_subscriber="enp1s0f0,eno1"
379    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382    export nic_publisher=$interface
383    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385    if [[ "${interface2}" == "" ]]; then
386        export nic_subscriber=$interface
387    else
388        export nic_subscriber=$interface2
389    fi
390    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392    if [[ "${ip1}" != "" ]]; then
393        export ip_publisher=$ip1
394        echo "Using ip_publisher: ${ip_publisher}"
395    fi
396
397    if [[ "${ip2}" != "" ]]; then
398        export ip_subscriber=$ip2
399        echo "Using ip_subscriber: ${ip_subscriber}"
400    fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408    export transport_string_pub="$transport_string -nic $nic_publisher"
409    export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411    if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414    fi
415
416elif [[ "$transport" == "TCP" ]]; then
417    export transport_string_pub="$transport_string \
418        -nic $nic_publisher \
419        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420    export transport_string_sub="$transport_string \
421        -nic $nic_subscriber \
422        -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424    export transport_string_pub="$transport_string \
425        -nic $nic_publisher \
426        -peer tlsv4_lan://${ip_subscriber}:7400"
427    export transport_string_sub="$transport_string \
428        -nic $nic_subscriber \
429        -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431    export transport_string_pub="$transport_string \
432        -nic $nic_publisher \
433        -transportPublicAddress $ip_publisher:7400"
434    export transport_string_sub="$transport_string \
435        -nic $nic_subscriber \
436        -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438    export transport_string_pub="$transport_string"
439    export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445        ${transport_string_pub} \
446        -noPrintIntervals \
447        -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450    export pub_string="$pub_string \
451        -latencyTest"
452fi
453
454export sub_string="-sub \
455        ${transport_string_sub} \
456        -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459    echo -e "$INFO_TAG Publisher side running"
460    export commands_string=${pub_string}
461    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463    echo -e "$INFO_TAG Subscriber side running"
464    export commands_string=${sub_string}
465    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490    # UNKEYED
491    if [[ "${skip_unkeyed}" == "" ]]; then
492
493        # RELIABLE
494        if [[ "${skip_rel_tests}" == "" ]]; then
495            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496        fi
497
498        # BEST EFFORT
499        if [[ "${skip_be_tests}" == "" ]]; then
500            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501        fi
502    fi
503
504    # KEYED
505    if [[ "${skip_keyed_data}" == "" ]]; then
506
507        # RELIABLE
508        if [[ "${skip_rel_tests}" == "" ]]; then
509            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510        fi
511
512        # BEST EFFORT
513        if [[ "${skip_be_tests}" == "" ]]; then
514            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515        fi
516    fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521    no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527    if [[ "$role" == "pub" ]]; then
528        export commands_string="${commands_string} -batchSize 0"
529    fi
530
531    # UNKEYED
532    if [[ "${skip_unkeyed}" == "" ]]; then
533
534        # RELIABLE
535        if [[ "${skip_rel_tests}" == "" ]]; then
536            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537        fi
538
539        # BEST EFFORT
540        if [[ "${skip_be_tests}" == "" ]]; then
541            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542        fi
543    fi
544
545    # KEYED
546    if [[ "${skip_keyed_data}" == "" ]]; then
547
548        # RELIABLE
549        if [[ "${skip_rel_tests}" == "" ]]; then
550            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551        fi
552
553        # BEST EFFORT
554        if [[ "${skip_be_tests}" == "" ]]; then
555            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556        fi
557    fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562    echo -e "${INFO_TAG} Disabling loss rate"
563    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi

1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport UDPv4

Test Software

The following software was used to perform these tests:

RTI Connext DDS 6.1.1 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)

Unkeyed, UDPv4 10Gbps Network, C++98

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Ave (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

15

132.0

14

100050

15

15

16

44

100050

64

15

0.9

14

64

15

16

17

44

64

128

15

1.0

15

65

15

16

17

45

65

256

15

1.0

14

73

15

15

16

44

73

512

19

1.9

15

69

19

21

22

47

69

1024

25

3.3

17

64

26

28

29

53

64

2048

32

194.2

18

100076

30

55

69

80

100076

4096

32

131.3

23

100058

30

34

54

69

100058

8192

40

8.7

33

89

34

55

62

77

89

16384

83

22.5

44

173

82

118

131

143

173

32768

172

2.2

65

216

172

173

174

201

216

63000

183

2.7

99

213

183

185

186

212

213

  • Reliable

Sample Size (Bytes)

Ave (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

16

1.0

15

96

16

17

20

46

96

64

17

1.1

16

68

16

17

21

46

68

128

17

1.3

16

67

16

17

23

46

67

256

17

2.1

16

64

17

19

28

46

64

512

20

3.0

17

70

19

21

32

49

70

1024

25

4.8

18

85

26

29

39

57

85

2048

29

31.9

20

4917

30

33

44

68

4917

4096

31

18.1

24

4235

30

36

48

68

4235

8192

41

26.6

34

5016

36

57

65

82

5016

16384

86

25.3

45

161

85

123

136

146

161

32768

171

8.0

74

203

173

174

176

202

203

63000

169

15.0

104

223

167

186

188

215

223

100000

205

9.2

154

254

208

210

216

241

254

500000

619

32.6

504

2920

619

651

697

1011

2920

1048576

1180

43.9

1069

1683

1204

1207

1224

1683

1683

1548576

1704

20.5

1541

2331

1701

1730

1752

2331

2331

4194304

4052

47.9

4010

6083

4047

4073

4099

6083

6083

10485760

10185

166.8

10152

15009

10177

10188

10211

15009

15009

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13
 14# We will use some colors to improve visibility of errors and info messages.
 15RED='\033[0;31m'
 16GREEN='\033[0;32m'
 17YELLOW='\033[0;33m'
 18BLUE='\033[0;34m'
 19LIGHTBLUE='\033[0;36m'
 20NC='\033[0m'
 21INFO_TAG="${GREEN}[INFO]:${NC}"
 22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 23ERROR_TAG="${RED}[ERROR]:${NC}"
 24
 25################################################################################
 26
 27function disable_colors() {
 28    export RED=""
 29    export GREEN=""
 30    export YELLOW=""
 31    export NC=""
 32    export BLUE=""
 33    export LIGHTBLUE=""
 34    export INFO_TAG="${GREEN}[INFO]:${NC}"
 35    export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 36    export ERROR_TAG="${RED}[ERROR]:${NC}"
 37}
 38
 39function change_domain() {
 40    if [[ "$domain" == "1" ]]; then
 41        export domain="2"
 42    else
 43        export domain="1"
 44    fi
 45}
 46
 47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 48function execute_test() {
 49
 50    local keyed_unkeyed=$1
 51    local rel_be=$2
 52    local datasizes_test=$3
 53    local other_args=$4
 54    local name_suffix=$5
 55
 56    local commands_string_test=$commands_string
 57    local tag=""
 58
 59    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 60        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 61        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 62    else
 63        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 64    fi
 65
 66    if [[ "${rel_be}" == "be" ]]; then
 67        commands_string_test="${commands_string_test} -bestEffort"
 68        tag="${tag}${YELLOW}BE${NC}]"
 69    else
 70        tag="${tag}${RED}REL${NC}]"
 71    fi
 72
 73    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 74
 75    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 76
 77    if [[ "$role" == "pub" ]]; then
 78        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 79    fi
 80
 81    if [[ "$NO_TASKSET" == "" ]]; then
 82        if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
 83            export pre_command_string="taskset -c $core"
 84        fi
 85    fi
 86
 87    if [[ "$LANGUAGE" == "python" ]]; then
 88        export pre_command_string="python3 "
 89    fi
 90
 91    if [[ "$DOCKER" == "1" ]]; then
 92        export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
 93        executable=""
 94    fi
 95
 96    # Get the aprox time this will take:
 97    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
 98    total_time=$((total_tests * exec_time))
 99
100    touch $output_file
101    local no_headers=""
102    local current_test=0
103    for index in $(seq 1 ${num_reps}); do
104        for DATALEN in ${datasizes_test}; do
105            current_test=$((current_test + 1))
106            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107            if [[ "$role" == "pub" ]]; then
108                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109                echo -e ${BLUE}$command${NC}
110            else
111                echo -e ${LIGHTBLUE}$command${NC}
112            fi
113            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114                sleep 3
115            fi
116            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117                sleep 5
118            fi
119            if [[ "${get_netstat_info}" == "1" ]]; then
120                echo -e "${INFO_TAG} Getting netstat info before"
121                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122            fi
123            eval $command >> $output_file;
124            if [[ "${get_netstat_info}" == "1" ]]; then
125                echo -e "${INFO_TAG} Getting netstat info after"
126                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128                python3 $script_location/../../../tools/diff_netstat_output.py \
129                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131                    -d $DATALEN $no_header_netstat \
132                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134                no_header_netstat=" -nh"
135            fi
136            no_headers=" -noOutputHeaders"
137
138            change_domain
139        done
140    done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147    case $1 in
148        --executable)
149            executable=$2
150            shift
151            ;;
152        --docker)
153            DOCKER="1"
154            ;;
155        --output-folder)
156            output_folder=$2
157            shift
158            ;;
159        --sub-folder)
160            sub_folder=$2
161            shift
162            ;;
163        --role)
164            export role=$2
165            shift
166            ;;
167        --core)
168            export core=$2
169            shift
170            ;;
171        --test-kind)
172            export lat_thr=$2
173            shift
174            ;;
175        --interface1)
176            export interface=$2
177            shift
178            ;;
179        --interface2)
180            export interface2=$2
181            shift
182            ;;
183        --ip1)
184            export ip1=$2
185            shift
186            ;;
187        --ip2)
188            export ip2=$2
189            shift
190            ;;
191        --repetitions)
192            export num_reps=$2
193            shift
194            ;;
195        --domain)
196            export domain=$2
197            shift
198            ;;
199        --execution-time)
200            export exec_time=$2
201            shift
202            ;;
203        --transport)
204            export transport=$2
205            shift
206            ;;
207        --datalen)
208            export datalen_input=$2
209            shift
210            ;;
211        --file-suffix)
212            export file_suffix=$2
213            shift
214            ;;
215        --executable-suffix)
216            export executable_suffix=$2
217            shift
218            ;;
219        --extra-arguments)
220            export extra_arguments=$2
221            shift
222            ;;
223        --extra-arguments-pub)
224            export extra_arguments_pub=$2
225            shift
226            ;;
227        --extra-arguments-sub)
228            export extra_arguments_sub=$2
229            shift
230            ;;
231        --skip-no-batching)
232            export skip_no_batching="1"
233            ;;
234        --skip-be)
235            export skip_be_tests="1"
236            ;;
237        --skip-rel)
238            export skip_rel_tests="1"
239            ;;
240        --skip-keyed)
241            export skip_keyed_data="1"
242            ;;
243        --skip-large-data)
244            export skip_large_data="1"
245            ;;
246        --large-data)
247            export large_data="1"
248            ;;
249        --keyed)
250            export skip_unkeyed="1"
251            ;;
252        --unkeyed)
253            export skip_keyed_data="1"
254            ;;
255        --no-batching | --skip-batching)
256            export no_batching_only="1"
257            ;;
258        --reliable)
259            export skip_be_tests="1"
260            ;;
261        --best-effort)
262            export skip_rel_tests="1"
263            ;;
264        --security-gov)
265            export security_only="$2"
266            shift
267            ;;
268        --micro)
269            export micro="1"
270            ;;
271        --cert)
272            export cert="1"
273            ;;
274        --raw | --raw-transport)
275            export raw="1"
276            ;;
277        --tss)
278            export tss="1"
279            ;;
280        --no-colors)
281            export NO_COLORS="1"
282            ;;
283        --language)
284            export LANGUAGE=$2
285            shift
286            ;;
287        --loss-rate)
288            export loss_rate=$2
289            shift
290            ;;
291        --get-netstat-info | --netstat)
292            export get_netstat_info="1"
293            ;;
294        --no-taskset)
295            export NO_TASKSET="1"
296            ;;
297        --reduced-data-sizes-set)
298            export REDUCED_DATA_SIZES_SET="1"
299            ;;
300        *)
301            echo -e "unknown parameter \"$1\""
302            exit 255
303            ;;
304    esac
305    shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309    disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
315    export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318    export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322    export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326    export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335    export skip_keyed_data="1"
336    export skip_large_data="1"
337    export skip_be_tests="1"
338    export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342    export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344    export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349    export datasizes=${datalen_input}
350    export datasizes_extended=${datalen_input}
351    if [[ "${no_batching_only}" != "1" ]]; then
352        export skip_large_data="1"
353    fi
354else 
355    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357        export datasizes="32 128 512 2048 8192 32768 63000"
358        export datasizes_extended="${datasizes} 102400 1048576 10485760"
359    fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364    exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368    echo -e "${ERROR_TAG} It must be either lat or thr"
369    exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373    echo "Using default nics"
374    export nic_publisher=${ip_machine_1}
375    export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377    export nic_publisher="enp1s0f0,eno1"
378    export nic_subscriber="enp1s0f0,eno1"
379    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382    export nic_publisher=$interface
383    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385    if [[ "${interface2}" == "" ]]; then
386        export nic_subscriber=$interface
387    else
388        export nic_subscriber=$interface2
389    fi
390    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392    if [[ "${ip1}" != "" ]]; then
393        export ip_publisher=$ip1
394        echo "Using ip_publisher: ${ip_publisher}"
395    fi
396
397    if [[ "${ip2}" != "" ]]; then
398        export ip_subscriber=$ip2
399        echo "Using ip_subscriber: ${ip_subscriber}"
400    fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408    export transport_string_pub="$transport_string -nic $nic_publisher"
409    export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411    if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414    fi
415
416elif [[ "$transport" == "TCP" ]]; then
417    export transport_string_pub="$transport_string \
418        -nic $nic_publisher \
419        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420    export transport_string_sub="$transport_string \
421        -nic $nic_subscriber \
422        -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424    export transport_string_pub="$transport_string \
425        -nic $nic_publisher \
426        -peer tlsv4_lan://${ip_subscriber}:7400"
427    export transport_string_sub="$transport_string \
428        -nic $nic_subscriber \
429        -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431    export transport_string_pub="$transport_string \
432        -nic $nic_publisher \
433        -transportPublicAddress $ip_publisher:7400"
434    export transport_string_sub="$transport_string \
435        -nic $nic_subscriber \
436        -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438    export transport_string_pub="$transport_string"
439    export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445        ${transport_string_pub} \
446        -noPrintIntervals \
447        -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450    export pub_string="$pub_string \
451        -latencyTest"
452fi
453
454export sub_string="-sub \
455        ${transport_string_sub} \
456        -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459    echo -e "$INFO_TAG Publisher side running"
460    export commands_string=${pub_string}
461    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463    echo -e "$INFO_TAG Subscriber side running"
464    export commands_string=${sub_string}
465    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490    # UNKEYED
491    if [[ "${skip_unkeyed}" == "" ]]; then
492
493        # RELIABLE
494        if [[ "${skip_rel_tests}" == "" ]]; then
495            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496        fi
497
498        # BEST EFFORT
499        if [[ "${skip_be_tests}" == "" ]]; then
500            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501        fi
502    fi
503
504    # KEYED
505    if [[ "${skip_keyed_data}" == "" ]]; then
506
507        # RELIABLE
508        if [[ "${skip_rel_tests}" == "" ]]; then
509            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510        fi
511
512        # BEST EFFORT
513        if [[ "${skip_be_tests}" == "" ]]; then
514            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515        fi
516    fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521    no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527    if [[ "$role" == "pub" ]]; then
528        export commands_string="${commands_string} -batchSize 0"
529    fi
530
531    # UNKEYED
532    if [[ "${skip_unkeyed}" == "" ]]; then
533
534        # RELIABLE
535        if [[ "${skip_rel_tests}" == "" ]]; then
536            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537        fi
538
539        # BEST EFFORT
540        if [[ "${skip_be_tests}" == "" ]]; then
541            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542        fi
543    fi
544
545    # KEYED
546    if [[ "${skip_keyed_data}" == "" ]]; then
547
548        # RELIABLE
549        if [[ "${skip_rel_tests}" == "" ]]; then
550            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551        fi
552
553        # BEST EFFORT
554        if [[ "${skip_be_tests}" == "" ]]; then
555            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556        fi
557    fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562    echo -e "${INFO_TAG} Disabling loss rate"
563    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi

1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport UDPv4

Test Software

The following software was used to perform these tests:

RTI Connext DDS 6.1.2 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)

Keyed, UDPv4 1Gbps Network, C++98

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 1Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

58

1.7

53

207

58

60

62

88

207

64

59

1.6

54

209

59

61

62

89

209

128

61

1.7

55

208

61

62

64

90

208

256

65

1.3

60

213

65

66

68

94

213

512

71

1.5

64

216

70

72

73

99

216

1024

84

1.4

78

236

84

85

87

114

236

2048

111

1.4

105

255

111

112

114

142

255

4096

164

1.5

158

311

163

165

167

194

311

8192

269

1.7

262

414

269

270

273

299

414

16384

345

1.8

338

490

345

347

348

382

490

32768

478

1.9

471

624

478

479

481

519

624

63000

739

2.1

731

883

739

740

742

810

883

  • Reliable

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

61

1.7

54

206

60

62

66

89

206

64

61

1.8

55

208

61

63

66

91

208

128

63

1.6

57

212

62

64

68

92

212

256

67

1.5

61

217

67

68

71

96

217

512

73

1.6

67

221

72

74

78

102

221

1024

86

1.7

80

234

85

87

91

115

234

2048

113

1.5

107

188

113

115

118

142

188

4096

166

1.5

158

236

165

167

170

195

236

8192

271

1.6

265

326

271

272

275

303

326

16384

347

1.6

341

405

347

348

350

376

405

32768

479

1.5

472

541

479

481

482

507

541

63000

740

1.9

734

813

740

742

744

772

813

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13
 14# We will use some colors to improve visibility of errors and info messages.
 15RED='\033[0;31m'
 16GREEN='\033[0;32m'
 17YELLOW='\033[0;33m'
 18BLUE='\033[0;34m'
 19LIGHTBLUE='\033[0;36m'
 20NC='\033[0m'
 21INFO_TAG="${GREEN}[INFO]:${NC}"
 22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 23ERROR_TAG="${RED}[ERROR]:${NC}"
 24
 25################################################################################
 26
 27function disable_colors() {
 28    export RED=""
 29    export GREEN=""
 30    export YELLOW=""
 31    export NC=""
 32    export BLUE=""
 33    export LIGHTBLUE=""
 34    export INFO_TAG="${GREEN}[INFO]:${NC}"
 35    export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 36    export ERROR_TAG="${RED}[ERROR]:${NC}"
 37}
 38
 39function change_domain() {
 40    if [[ "$domain" == "1" ]]; then
 41        export domain="2"
 42    else
 43        export domain="1"
 44    fi
 45}
 46
 47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 48function execute_test() {
 49
 50    local keyed_unkeyed=$1
 51    local rel_be=$2
 52    local datasizes_test=$3
 53    local other_args=$4
 54    local name_suffix=$5
 55
 56    local commands_string_test=$commands_string
 57    local tag=""
 58
 59    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 60        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 61        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 62    else
 63        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 64    fi
 65
 66    if [[ "${rel_be}" == "be" ]]; then
 67        commands_string_test="${commands_string_test} -bestEffort"
 68        tag="${tag}${YELLOW}BE${NC}]"
 69    else
 70        tag="${tag}${RED}REL${NC}]"
 71    fi
 72
 73    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 74
 75    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 76
 77    if [[ "$role" == "pub" ]]; then
 78        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 79    fi
 80
 81    if [[ "$NO_TASKSET" == "" ]]; then
 82        if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
 83            export pre_command_string="taskset -c $core"
 84        fi
 85    fi
 86
 87    if [[ "$LANGUAGE" == "python" ]]; then
 88        export pre_command_string="python3 "
 89    fi
 90
 91    if [[ "$DOCKER" == "1" ]]; then
 92        export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
 93        executable=""
 94    fi
 95
 96    # Get the aprox time this will take:
 97    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
 98    total_time=$((total_tests * exec_time))
 99
100    touch $output_file
101    local no_headers=""
102    local current_test=0
103    for index in $(seq 1 ${num_reps}); do
104        for DATALEN in ${datasizes_test}; do
105            current_test=$((current_test + 1))
106            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107            if [[ "$role" == "pub" ]]; then
108                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109                echo -e ${BLUE}$command${NC}
110            else
111                echo -e ${LIGHTBLUE}$command${NC}
112            fi
113            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114                sleep 3
115            fi
116            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117                sleep 5
118            fi
119            if [[ "${get_netstat_info}" == "1" ]]; then
120                echo -e "${INFO_TAG} Getting netstat info before"
121                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122            fi
123            eval $command >> $output_file;
124            if [[ "${get_netstat_info}" == "1" ]]; then
125                echo -e "${INFO_TAG} Getting netstat info after"
126                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128                python3 $script_location/../../../tools/diff_netstat_output.py \
129                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131                    -d $DATALEN $no_header_netstat \
132                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134                no_header_netstat=" -nh"
135            fi
136            no_headers=" -noOutputHeaders"
137
138            change_domain
139        done
140    done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147    case $1 in
148        --executable)
149            executable=$2
150            shift
151            ;;
152        --docker)
153            DOCKER="1"
154            ;;
155        --output-folder)
156            output_folder=$2
157            shift
158            ;;
159        --sub-folder)
160            sub_folder=$2
161            shift
162            ;;
163        --role)
164            export role=$2
165            shift
166            ;;
167        --core)
168            export core=$2
169            shift
170            ;;
171        --test-kind)
172            export lat_thr=$2
173            shift
174            ;;
175        --interface1)
176            export interface=$2
177            shift
178            ;;
179        --interface2)
180            export interface2=$2
181            shift
182            ;;
183        --ip1)
184            export ip1=$2
185            shift
186            ;;
187        --ip2)
188            export ip2=$2
189            shift
190            ;;
191        --repetitions)
192            export num_reps=$2
193            shift
194            ;;
195        --domain)
196            export domain=$2
197            shift
198            ;;
199        --execution-time)
200            export exec_time=$2
201            shift
202            ;;
203        --transport)
204            export transport=$2
205            shift
206            ;;
207        --datalen)
208            export datalen_input=$2
209            shift
210            ;;
211        --file-suffix)
212            export file_suffix=$2
213            shift
214            ;;
215        --executable-suffix)
216            export executable_suffix=$2
217            shift
218            ;;
219        --extra-arguments)
220            export extra_arguments=$2
221            shift
222            ;;
223        --extra-arguments-pub)
224            export extra_arguments_pub=$2
225            shift
226            ;;
227        --extra-arguments-sub)
228            export extra_arguments_sub=$2
229            shift
230            ;;
231        --skip-no-batching)
232            export skip_no_batching="1"
233            ;;
234        --skip-be)
235            export skip_be_tests="1"
236            ;;
237        --skip-rel)
238            export skip_rel_tests="1"
239            ;;
240        --skip-keyed)
241            export skip_keyed_data="1"
242            ;;
243        --skip-large-data)
244            export skip_large_data="1"
245            ;;
246        --large-data)
247            export large_data="1"
248            ;;
249        --keyed)
250            export skip_unkeyed="1"
251            ;;
252        --unkeyed)
253            export skip_keyed_data="1"
254            ;;
255        --no-batching | --skip-batching)
256            export no_batching_only="1"
257            ;;
258        --reliable)
259            export skip_be_tests="1"
260            ;;
261        --best-effort)
262            export skip_rel_tests="1"
263            ;;
264        --security-gov)
265            export security_only="$2"
266            shift
267            ;;
268        --micro)
269            export micro="1"
270            ;;
271        --cert)
272            export cert="1"
273            ;;
274        --raw | --raw-transport)
275            export raw="1"
276            ;;
277        --tss)
278            export tss="1"
279            ;;
280        --no-colors)
281            export NO_COLORS="1"
282            ;;
283        --language)
284            export LANGUAGE=$2
285            shift
286            ;;
287        --loss-rate)
288            export loss_rate=$2
289            shift
290            ;;
291        --get-netstat-info | --netstat)
292            export get_netstat_info="1"
293            ;;
294        --no-taskset)
295            export NO_TASKSET="1"
296            ;;
297        --reduced-data-sizes-set)
298            export REDUCED_DATA_SIZES_SET="1"
299            ;;
300        *)
301            echo -e "unknown parameter \"$1\""
302            exit 255
303            ;;
304    esac
305    shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309    disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
315    export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318    export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322    export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326    export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335    export skip_keyed_data="1"
336    export skip_large_data="1"
337    export skip_be_tests="1"
338    export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342    export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344    export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349    export datasizes=${datalen_input}
350    export datasizes_extended=${datalen_input}
351    if [[ "${no_batching_only}" != "1" ]]; then
352        export skip_large_data="1"
353    fi
354else 
355    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357        export datasizes="32 128 512 2048 8192 32768 63000"
358        export datasizes_extended="${datasizes} 102400 1048576 10485760"
359    fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364    exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368    echo -e "${ERROR_TAG} It must be either lat or thr"
369    exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373    echo "Using default nics"
374    export nic_publisher=${ip_machine_1}
375    export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377    export nic_publisher="enp1s0f0,eno1"
378    export nic_subscriber="enp1s0f0,eno1"
379    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382    export nic_publisher=$interface
383    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385    if [[ "${interface2}" == "" ]]; then
386        export nic_subscriber=$interface
387    else
388        export nic_subscriber=$interface2
389    fi
390    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392    if [[ "${ip1}" != "" ]]; then
393        export ip_publisher=$ip1
394        echo "Using ip_publisher: ${ip_publisher}"
395    fi
396
397    if [[ "${ip2}" != "" ]]; then
398        export ip_subscriber=$ip2
399        echo "Using ip_subscriber: ${ip_subscriber}"
400    fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408    export transport_string_pub="$transport_string -nic $nic_publisher"
409    export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411    if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414    fi
415
416elif [[ "$transport" == "TCP" ]]; then
417    export transport_string_pub="$transport_string \
418        -nic $nic_publisher \
419        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420    export transport_string_sub="$transport_string \
421        -nic $nic_subscriber \
422        -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424    export transport_string_pub="$transport_string \
425        -nic $nic_publisher \
426        -peer tlsv4_lan://${ip_subscriber}:7400"
427    export transport_string_sub="$transport_string \
428        -nic $nic_subscriber \
429        -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431    export transport_string_pub="$transport_string \
432        -nic $nic_publisher \
433        -transportPublicAddress $ip_publisher:7400"
434    export transport_string_sub="$transport_string \
435        -nic $nic_subscriber \
436        -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438    export transport_string_pub="$transport_string"
439    export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445        ${transport_string_pub} \
446        -noPrintIntervals \
447        -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450    export pub_string="$pub_string \
451        -latencyTest"
452fi
453
454export sub_string="-sub \
455        ${transport_string_sub} \
456        -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459    echo -e "$INFO_TAG Publisher side running"
460    export commands_string=${pub_string}
461    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463    echo -e "$INFO_TAG Subscriber side running"
464    export commands_string=${sub_string}
465    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490    # UNKEYED
491    if [[ "${skip_unkeyed}" == "" ]]; then
492
493        # RELIABLE
494        if [[ "${skip_rel_tests}" == "" ]]; then
495            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496        fi
497
498        # BEST EFFORT
499        if [[ "${skip_be_tests}" == "" ]]; then
500            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501        fi
502    fi
503
504    # KEYED
505    if [[ "${skip_keyed_data}" == "" ]]; then
506
507        # RELIABLE
508        if [[ "${skip_rel_tests}" == "" ]]; then
509            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510        fi
511
512        # BEST EFFORT
513        if [[ "${skip_be_tests}" == "" ]]; then
514            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515        fi
516    fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521    no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527    if [[ "$role" == "pub" ]]; then
528        export commands_string="${commands_string} -batchSize 0"
529    fi
530
531    # UNKEYED
532    if [[ "${skip_unkeyed}" == "" ]]; then
533
534        # RELIABLE
535        if [[ "${skip_rel_tests}" == "" ]]; then
536            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537        fi
538
539        # BEST EFFORT
540        if [[ "${skip_be_tests}" == "" ]]; then
541            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542        fi
543    fi
544
545    # KEYED
546    if [[ "${skip_keyed_data}" == "" ]]; then
547
548        # RELIABLE
549        if [[ "${skip_rel_tests}" == "" ]]; then
550            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551        fi
552
553        # BEST EFFORT
554        if [[ "${skip_be_tests}" == "" ]]; then
555            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556        fi
557    fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562    echo -e "${INFO_TAG} Disabling loss rate"
563    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi

1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport UDPv4

Test Software

The following software was used to perform these tests:

RTI Connext DDS 6.1.1 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)

Keyed, UDPv4 10Gbps Network, C++98

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Ave (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

17

1.0

17

167

17

18

20

47

167

64

17

1.2

17

170

17

18

21

46

170

128

18

1.1

17

167

18

19

21

48

167

256

18

1.1

17

165

18

19

21

48

165

512

19

1.2

17

175

19

20

22

49

175

1024

25

3.5

19

175

26

28

29

56

175

2048

32

171.7

21

100070

30

61

64

79

100070

4096

33

158.2

26

100062

30

44

58

70

100062

8192

46

214.6

35

100070

41

60

64

85

100070

16384

93

25.2

48

276

82

126

131

151

276

32768

176

2.5

70

321

176

177

179

206

321

63000

188

2.8

105

331

188

189

191

217

331

  • Reliable

Sample Size (Bytes)

Ave (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

19

1.2

18

166

19

20

23

49

166

64

20

1.3

18

166

19

20

24

50

166

128

19

1.3

18

167

19

20

25

49

167

256

20

5.0

18

3169

19

21

27

50

3169

512

21

2.3

19

171

20

22

29

51

171

1024

26

8.5

20

3437

26

28

46

58

3437

2048

29

14.5

22

4215

30

32

43

66

4215

4096

31

18.9

27

5027

30

33

51

66

5027

8192

44

48.3

37

4854

39

55

73

3352

4854

16384

93

24.9

48

270

86

126

132

152

270

32768

162

15.8

78

244

167

177

180

205

244

63000

189

2.7

108

260

188

190

194

219

260

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13
 14# We will use some colors to improve visibility of errors and info messages.
 15RED='\033[0;31m'
 16GREEN='\033[0;32m'
 17YELLOW='\033[0;33m'
 18BLUE='\033[0;34m'
 19LIGHTBLUE='\033[0;36m'
 20NC='\033[0m'
 21INFO_TAG="${GREEN}[INFO]:${NC}"
 22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 23ERROR_TAG="${RED}[ERROR]:${NC}"
 24
 25################################################################################
 26
 27function disable_colors() {
 28    export RED=""
 29    export GREEN=""
 30    export YELLOW=""
 31    export NC=""
 32    export BLUE=""
 33    export LIGHTBLUE=""
 34    export INFO_TAG="${GREEN}[INFO]:${NC}"
 35    export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
 36    export ERROR_TAG="${RED}[ERROR]:${NC}"
 37}
 38
 39function change_domain() {
 40    if [[ "$domain" == "1" ]]; then
 41        export domain="2"
 42    else
 43        export domain="1"
 44    fi
 45}
 46
 47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 48function execute_test() {
 49
 50    local keyed_unkeyed=$1
 51    local rel_be=$2
 52    local datasizes_test=$3
 53    local other_args=$4
 54    local name_suffix=$5
 55
 56    local commands_string_test=$commands_string
 57    local tag=""
 58
 59    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 60        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 61        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 62    else
 63        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 64    fi
 65
 66    if [[ "${rel_be}" == "be" ]]; then
 67        commands_string_test="${commands_string_test} -bestEffort"
 68        tag="${tag}${YELLOW}BE${NC}]"
 69    else
 70        tag="${tag}${RED}REL${NC}]"
 71    fi
 72
 73    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 74
 75    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 76
 77    if [[ "$role" == "pub" ]]; then
 78        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 79    fi
 80
 81    if [[ "$NO_TASKSET" == "" ]]; then
 82        if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
 83            export pre_command_string="taskset -c $core"
 84        fi
 85    fi
 86
 87    if [[ "$LANGUAGE" == "python" ]]; then
 88        export pre_command_string="python3 "
 89    fi
 90
 91    if [[ "$DOCKER" == "1" ]]; then
 92        export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
 93        executable=""
 94    fi
 95
 96    # Get the aprox time this will take:
 97    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
 98    total_time=$((total_tests * exec_time))
 99
100    touch $output_file
101    local no_headers=""
102    local current_test=0
103    for index in $(seq 1 ${num_reps}); do
104        for DATALEN in ${datasizes_test}; do
105            current_test=$((current_test + 1))
106            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107            if [[ "$role" == "pub" ]]; then
108                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109                echo -e ${BLUE}$command${NC}
110            else
111                echo -e ${LIGHTBLUE}$command${NC}
112            fi
113            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114                sleep 3
115            fi
116            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117                sleep 5
118            fi
119            if [[ "${get_netstat_info}" == "1" ]]; then
120                echo -e "${INFO_TAG} Getting netstat info before"
121                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122            fi
123            eval $command >> $output_file;
124            if [[ "${get_netstat_info}" == "1" ]]; then
125                echo -e "${INFO_TAG} Getting netstat info after"
126                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128                python3 $script_location/../../../tools/diff_netstat_output.py \
129                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131                    -d $DATALEN $no_header_netstat \
132                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134                no_header_netstat=" -nh"
135            fi
136            no_headers=" -noOutputHeaders"
137
138            change_domain
139        done
140    done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147    case $1 in
148        --executable)
149            executable=$2
150            shift
151            ;;
152        --docker)
153            DOCKER="1"
154            ;;
155        --output-folder)
156            output_folder=$2
157            shift
158            ;;
159        --sub-folder)
160            sub_folder=$2
161            shift
162            ;;
163        --role)
164            export role=$2
165            shift
166            ;;
167        --core)
168            export core=$2
169            shift
170            ;;
171        --test-kind)
172            export lat_thr=$2
173            shift
174            ;;
175        --interface1)
176            export interface=$2
177            shift
178            ;;
179        --interface2)
180            export interface2=$2
181            shift
182            ;;
183        --ip1)
184            export ip1=$2
185            shift
186            ;;
187        --ip2)
188            export ip2=$2
189            shift
190            ;;
191        --repetitions)
192            export num_reps=$2
193            shift
194            ;;
195        --domain)
196            export domain=$2
197            shift
198            ;;
199        --execution-time)
200            export exec_time=$2
201            shift
202            ;;
203        --transport)
204            export transport=$2
205            shift
206            ;;
207        --datalen)
208            export datalen_input=$2
209            shift
210            ;;
211        --file-suffix)
212            export file_suffix=$2
213            shift
214            ;;
215        --executable-suffix)
216            export executable_suffix=$2
217            shift
218            ;;
219        --extra-arguments)
220            export extra_arguments=$2
221            shift
222            ;;
223        --extra-arguments-pub)
224            export extra_arguments_pub=$2
225            shift
226            ;;
227        --extra-arguments-sub)
228            export extra_arguments_sub=$2
229            shift
230            ;;
231        --skip-no-batching)
232            export skip_no_batching="1"
233            ;;
234        --skip-be)
235            export skip_be_tests="1"
236            ;;
237        --skip-rel)
238            export skip_rel_tests="1"
239            ;;
240        --skip-keyed)
241            export skip_keyed_data="1"
242            ;;
243        --skip-large-data)
244            export skip_large_data="1"
245            ;;
246        --large-data)
247            export large_data="1"
248            ;;
249        --keyed)
250            export skip_unkeyed="1"
251            ;;
252        --unkeyed)
253            export skip_keyed_data="1"
254            ;;
255        --no-batching | --skip-batching)
256            export no_batching_only="1"
257            ;;
258        --reliable)
259            export skip_be_tests="1"
260            ;;
261        --best-effort)
262            export skip_rel_tests="1"
263            ;;
264        --security-gov)
265            export security_only="$2"
266            shift
267            ;;
268        --micro)
269            export micro="1"
270            ;;
271        --cert)
272            export cert="1"
273            ;;
274        --raw | --raw-transport)
275            export raw="1"
276            ;;
277        --tss)
278            export tss="1"
279            ;;
280        --no-colors)
281            export NO_COLORS="1"
282            ;;
283        --language)
284            export LANGUAGE=$2
285            shift
286            ;;
287        --loss-rate)
288            export loss_rate=$2
289            shift
290            ;;
291        --get-netstat-info | --netstat)
292            export get_netstat_info="1"
293            ;;
294        --no-taskset)
295            export NO_TASKSET="1"
296            ;;
297        --reduced-data-sizes-set)
298            export REDUCED_DATA_SIZES_SET="1"
299            ;;
300        *)
301            echo -e "unknown parameter \"$1\""
302            exit 255
303            ;;
304    esac
305    shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309    disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
315    export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318    export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322    export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326    export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335    export skip_keyed_data="1"
336    export skip_large_data="1"
337    export skip_be_tests="1"
338    export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342    export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344    export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349    export datasizes=${datalen_input}
350    export datasizes_extended=${datalen_input}
351    if [[ "${no_batching_only}" != "1" ]]; then
352        export skip_large_data="1"
353    fi
354else 
355    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357        export datasizes="32 128 512 2048 8192 32768 63000"
358        export datasizes_extended="${datasizes} 102400 1048576 10485760"
359    fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364    exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368    echo -e "${ERROR_TAG} It must be either lat or thr"
369    exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373    echo "Using default nics"
374    export nic_publisher=${ip_machine_1}
375    export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377    export nic_publisher="enp1s0f0,eno1"
378    export nic_subscriber="enp1s0f0,eno1"
379    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382    export nic_publisher=$interface
383    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385    if [[ "${interface2}" == "" ]]; then
386        export nic_subscriber=$interface
387    else
388        export nic_subscriber=$interface2
389    fi
390    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392    if [[ "${ip1}" != "" ]]; then
393        export ip_publisher=$ip1
394        echo "Using ip_publisher: ${ip_publisher}"
395    fi
396
397    if [[ "${ip2}" != "" ]]; then
398        export ip_subscriber=$ip2
399        echo "Using ip_subscriber: ${ip_subscriber}"
400    fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408    export transport_string_pub="$transport_string -nic $nic_publisher"
409    export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411    if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414    fi
415
416elif [[ "$transport" == "TCP" ]]; then
417    export transport_string_pub="$transport_string \
418        -nic $nic_publisher \
419        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420    export transport_string_sub="$transport_string \
421        -nic $nic_subscriber \
422        -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424    export transport_string_pub="$transport_string \
425        -nic $nic_publisher \
426        -peer tlsv4_lan://${ip_subscriber}:7400"
427    export transport_string_sub="$transport_string \
428        -nic $nic_subscriber \
429        -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431    export transport_string_pub="$transport_string \
432        -nic $nic_publisher \
433        -transportPublicAddress $ip_publisher:7400"
434    export transport_string_sub="$transport_string \
435        -nic $nic_subscriber \
436        -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438    export transport_string_pub="$transport_string"
439    export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445        ${transport_string_pub} \
446        -noPrintIntervals \
447        -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450    export pub_string="$pub_string \
451        -latencyTest"
452fi
453
454export sub_string="-sub \
455        ${transport_string_sub} \
456        -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459    echo -e "$INFO_TAG Publisher side running"
460    export commands_string=${pub_string}
461    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463    echo -e "$INFO_TAG Subscriber side running"
464    export commands_string=${sub_string}
465    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490    # UNKEYED
491    if [[ "${skip_unkeyed}" == "" ]]; then
492
493        # RELIABLE
494        if [[ "${skip_rel_tests}" == "" ]]; then
495            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496        fi
497
498        # BEST EFFORT
499        if [[ "${skip_be_tests}" == "" ]]; then
500            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501        fi
502    fi
503
504    # KEYED
505    if [[ "${skip_keyed_data}" == "" ]]; then
506
507        # RELIABLE
508        if [[ "${skip_rel_tests}" == "" ]]; then
509            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510        fi
511
512        # BEST EFFORT
513        if [[ "${skip_be_tests}" == "" ]]; then
514            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515        fi
516    fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521    no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527    if [[ "$role" == "pub" ]]; then
528        export commands_string="${commands_string} -batchSize 0"
529    fi
530
531    # UNKEYED
532    if [[ "${skip_unkeyed}" == "" ]]; then
533
534        # RELIABLE
535        if [[ "${skip_rel_tests}" == "" ]]; then
536            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537        fi
538
539        # BEST EFFORT
540        if [[ "${skip_be_tests}" == "" ]]; then
541            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542        fi
543    fi
544
545    # KEYED
546    if [[ "${skip_keyed_data}" == "" ]]; then
547
548        # RELIABLE
549        if [[ "${skip_rel_tests}" == "" ]]; then
550            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551        fi
552
553        # BEST EFFORT
554        if [[ "${skip_be_tests}" == "" ]]; then
555            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556        fi
557    fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562    echo -e "${INFO_TAG} Disabling loss rate"
563    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi

1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport UDPv4

Test Software

The following software was used to perform these tests:

RTI Connext DDS 6.1.2 Host and Target Libraries for x64 Linux (x64Linux4gcc7.3.0)

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)