2.2.4. C# API

The following tests have been performed by executing an RTI Perftest C# Publisher and Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to UDPv4.

Find information about the hardware, network, and command-line parameters after each of the tests.

2.2.4.1. Unkeyed, UDPv4 10Gbps Network, C#

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by a simplified version of RTI Perftest for the Modern C# implementation. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

16

3.7

15

636

16

17

18

25

636

64

16

3.9

16

572

16

17

18

27

572

128

16

5.2

16

582

17

17

18

25

582

256

17

3.3

16

615

17

18

20

27

615

512

19

4.4

17

483

19

21

22

29

483

1024

27

369.2

18

100064

26

31

51

67

100064

2048

34

138.5

20

100061

31

50

65

73

100061

4096

46

226.0

25

100098

46

58

63

78

100098

8192

44

10.2

36

510

39

59

62

70

510

16384

91

26.3

47

552

82

123

129

153

552

32768

176

8.6

70

643

176

178

181

196

643

63000

189

8.4

108

1177

189

190

194

203

1177

  • Reliable

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

18

4.1

17

645

18

19

20

33

645

64

18

1.4

17

571

18

18

20

46

571

128

18

5.8

17

639

18

19

38

41

639

256

20

7.5

18

870

19

30

38

45

870

512

20

7.4

18

484

20

22

41

49

484

1024

25

7.8

20

766

26

29

54

58

766

2048

29

9.5

21

954

30

33

46

420

954

4096

31

11.7

26

962

31

36

61

625

962

8192

45

14.4

37

723

40

61

80

124

723

16384

90

27.5

48

552

89

129

145

158

552

32768

139

17.7

75

601

133

167

180

190

601

63000

191

14.4

116

1048

191

193

198

210

1048


Perftest Scripts

To produce these tests, we executed RTI Perftest for C#. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13export run_batching_tests="1"
 14export run_no_batching_tests="1"
 15
 16# We will use some colors to improve visibility of errors and info messages.
 17RED='\033[0;31m'
 18GREEN='\033[0;32m'
 19YELLOW='\033[0;33m'
 20BLUE='\033[0;34m'
 21LIGHTBLUE='\033[0;36m'
 22NC='\033[0m'
 23INFO_TAG="${GREEN}[INFO][$role]:${NC}"
 24WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
 25ERROR_TAG="${RED}[ERROR][$role]:${NC}"
 26
 27################################################################################
 28
 29function disable_colors() {
 30    export RED=""
 31    export GREEN=""
 32    export YELLOW=""
 33    export NC=""
 34    export BLUE=""
 35    export LIGHTBLUE=""
 36    export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
 37    export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
 38    export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
 39}
 40
 41function change_domain() {
 42    if [[ "$domain" == "1" ]]; then
 43        export domain="2"
 44    else
 45        export domain="1"
 46    fi
 47}
 48
 49# Function to process and append lines to the output file
 50function append_to_output_file() {
 51    local line=${1?line required}; readonly line
 52    local append=$(echo "${2?append required}" | tr -s ' '); readonly append
 53    local file=${3?file required}; readonly file
 54    if [[ "${line: -1}" == $'\n' ]]; then
 55        line="${line::-1}"
 56    fi
 57    echo "${line}${append}" >> "$file"
 58}
 59
 60# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 61function execute_test() {
 62
 63    local keyed_unkeyed=$1
 64    local rel_be=$2
 65    local datasizes_test=$3
 66    local other_args=$4
 67    local name_suffix=$5
 68
 69    local commands_string_test=$commands_string
 70    local tag=""
 71
 72    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 73        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 74        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 75    else
 76        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 77    fi
 78
 79    if [[ "${rel_be}" == "be" ]]; then
 80        commands_string_test="${commands_string_test} -bestEffort"
 81        tag="${tag}${YELLOW}BE${NC}]"
 82    else
 83        tag="${tag}${RED}REL${NC}]"
 84    fi
 85
 86    # If batch_size is set, we will add it to the command line of the publisher side.
 87    if [[ "$batch_size" != "" && "$role" == "pub" ]]; then
 88        commands_string_test="${commands_string_test} -batchSize $batch_size"
 89    fi
 90
 91    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 92
 93    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 94
 95    if [[ "$role" == "pub" ]]; then
 96        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 97    fi
 98
 99    if [[ "$thread_cpu_affinity" != "" ]]; then
100        commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
101    fi
102
103    local pre_command_string=""
104
105    if [[ "$thread_priorities" != "" ]]; then
106        echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
107        # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
108        export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
109        pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
110        commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
111    fi
112
113    if [[ "$pin_memory" != "" ]]; then
114        commands_string_test="${commands_string_test} -pinMemory"
115    fi
116
117    if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
118        pre_command_string="$pre_command_string taskset -c $core"
119    fi
120
121    if [[ "$LANGUAGE" == "python" ]]; then
122        pre_command_string="$pre_command_string python3 "
123    fi
124
125    if [[ "$DOCKER" == "1" ]]; then
126        pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
127        executable=""
128    fi
129
130    # Get the aprox time this will take:
131    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
132    total_time=$((total_tests * exec_time))
133
134    touch $output_file
135    local no_headers=""
136    local current_test=0
137    for index in $(seq 1 ${num_reps}); do
138        for DATALEN in ${datasizes_test}; do
139            current_test=$((current_test + 1))
140
141            if [[ ! -s $output_file ]]; then
142                echo -e "${INFO_TAG} Output file is empty, filling the header."
143                no_headers=""
144            else
145                echo -e "${INFO_TAG} Output file is not empty."
146                no_headers=" -noOutputHeaders"
147            fi
148
149            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
150            if [[ "$role" == "pub" ]]; then
151                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
152                echo -e ${BLUE}$command${NC}
153            else
154                echo -e ${LIGHTBLUE}$command${NC}
155            fi
156
157            # In certain cases we need to wait a bit before running the test, this is
158            # because the previous test might not be finished on the other side yet, or because the
159            # discovery mechanism does not work like in connext DDS.
160            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
161                sleep 3
162            fi
163            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
164                sleep 5
165            fi
166
167            # Gather netstat info before running the test
168            if [[ "${get_netstat_info}" == "1" ]]; then
169                echo -e "${INFO_TAG} Getting netstat info before"
170                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
171            fi
172
173            # Execute the command and capture the output
174            if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
175                echo -e "${INFO_TAG} Using C++98, using -outputFile option"
176                touch ${output_file}_tmp_line
177                chmod 777 ${output_file}_tmp_line
178                eval $command -outputFile ${output_file}_tmp_line
179            else
180                eval $command > ${output_file}_tmp_line
181            fi
182
183            number_of_lines=$(wc -l < ${output_file}_tmp_line)
184            echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
185
186            # Check if this is the first test for this output file.
187            local is_first_test=0
188            if [[ ! -s $output_file ]]; then
189                is_first_test=1
190            fi
191            local expected_lines=$((is_first_test+1))
192
193            if [[ $number_of_lines -ne ${expected_lines} ]]; then
194                echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
195                echo -e "${RED} The content is: \n\"\"\""
196                cat ${output_file}_tmp_line
197                echo -e "\"\"\""
198                echo -e "Not adding it to the output file. ${NC}"
199            else
200                if [[ $number_of_lines -gt 1 ]]; then
201                    header_line=$(head -n 1 "${output_file}_tmp_line")
202                    echo -e "${INFO_TAG} Header line is: $header_line"
203                fi
204                result_line=$(tail -n 1 "${output_file}_tmp_line")
205                echo -e "${INFO_TAG} Result line is: $result_line"
206
207                # If this is the first test, we need to add the header line to the output file
208                if [[ $is_first_test -eq 1 ]]; then
209                    append_to_output_file "$header_line" ", command-line" $output_file
210                fi
211
212                command=$(echo $command | sed 's/,/:comma:/g')
213                # Remove the `"` characters from the command line
214                command=$(echo $command | sed 's/\"//g')
215
216                # Append the result line to the output file
217                append_to_output_file "$result_line" ", $command" $output_file
218            fi
219
220            # Always remove the temporary file
221            rm -rf ${output_file}_tmp_line
222
223            # Gather netstat info after running the test
224            if [[ "${get_netstat_info}" == "1" ]]; then
225                echo -e "${INFO_TAG} Getting netstat info after"
226                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
227                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
228                python3 $script_location/../../../tools/diff_netstat_output.py \
229                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
230                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
231                    -d $DATALEN $no_header_netstat \
232                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
233                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
234                no_header_netstat=" -nh"
235            fi
236
237            change_domain
238        done
239    done
240}
241
242################################################################################
243# PARSE COMMAND LINE OPTIONS:
244
245while [ "$1" != "" ]; do
246    case $1 in
247        --executable)
248            executable=$2
249            shift
250            ;;
251        --docker)
252            DOCKER="1"
253            ;;
254        --output-folder)
255            output_folder=$2
256            shift
257            ;;
258        --role)
259            export role=$2
260            shift
261            ;;
262        --core)
263            export core=$2
264            shift
265            ;;
266        --assign-core-list | --thread-cpu-affinity)
267            export thread_cpu_affinity=$2
268            shift
269            ;;
270        --assign-thread-priorities | --thread-priorities)
271            export thread_priorities=$2
272            shift
273            ;;
274        --pin-memory)
275            export pin_memory="1"
276            ;;
277        --test-kind)
278            export lat_thr=$2
279            shift
280            ;;
281        --interface1)
282            export interface=$2
283            shift
284            ;;
285        --interface2)
286            export interface2=$2
287            shift
288            ;;
289        --ip1)
290            export ip1=$2
291            shift
292            ;;
293        --ip2)
294            export ip2=$2
295            shift
296            ;;
297        --repetitions)
298            export num_reps=$2
299            shift
300            ;;
301        --domain)
302            export domain=$2
303            shift
304            ;;
305        --execution-time)
306            export exec_time=$2
307            shift
308            ;;
309        --transport)
310            export transport=$2
311            shift
312            ;;
313        --datalen)
314            export datalen_input=$2
315            shift
316            ;;
317        --file-suffix)
318            export file_suffix=$2
319            shift
320            ;;
321        --folder-suffix)
322            export folder_suffix=$2
323            shift
324            ;;
325        --sub-folder)
326            sub_folder=$2
327            shift
328            ;;
329        --executable-suffix)
330            export executable_suffix=$2
331            shift
332            ;;
333        --extra-arguments)
334            export extra_arguments=$2
335            shift
336            ;;
337        --extra-arguments-pub)
338            export extra_arguments_pub=$2
339            shift
340            ;;
341        --extra-arguments-sub)
342            export extra_arguments_sub=$2
343            shift
344            ;;
345        --skip-be)
346            export skip_be_tests="1"
347            ;;
348        --skip-rel)
349            export skip_rel_tests="1"
350            ;;
351        --skip-keyed)
352            export skip_keyed_data="1"
353            ;;
354        --skip-large-data)
355            export skip_large_data="1"
356            ;;
357        --large-data)
358            export large_data="1"
359            ;;
360        --keyed)
361            export skip_unkeyed="1"
362            ;;
363        --unkeyed)
364            export skip_keyed_data="1"
365            ;;
366        --no-batching | --skip-batching)
367            export run_batching_tests="0"
368            export run_no_batching_tests="1"
369            ;;
370        --skip-no-batching)
371            export run_batching_tests="1"
372            export run_no_batching_tests="0"
373            ;;
374        --batch-size)
375            export batch_size=$2
376            shift
377            ;;
378        --reliable)
379            export skip_be_tests="1"
380            ;;
381        --best-effort)
382            export skip_rel_tests="1"
383            ;;
384        --security-gov)
385            export security_only="$2"
386            shift
387            ;;
388        --micro)
389            export micro="1"
390            ;;
391        --cert)
392            export cert="1"
393            ;;
394        --raw | --raw-transport)
395            export raw="1"
396            ;;
397        --tss)
398            export tss="1"
399            ;;
400        --no-colors)
401            export NO_COLORS="1"
402            ;;
403        --language)
404            export LANGUAGE=$2
405            shift
406            ;;
407        --loss-rate)
408            export loss_rate=$2
409            shift
410            ;;
411        --get-netstat-info | --netstat)
412            export get_netstat_info="1"
413            ;;
414        --reduced-data-sizes-set)
415            export REDUCED_DATA_SIZES_SET="1"
416            ;;
417        --dont-change-tuned-settings)
418            export dont_change_tuned_settings="1"
419            ;;
420        --no-taskset)
421            export no_taskset=1
422            ;;
423        *)
424            echo -e "unknown parameter \"$1\""
425            exit 255
426            ;;
427    esac
428    shift
429done
430
431if [[ "$NO_COLORS" == "1" ]]; then
432    disable_colors
433fi
434
435export folder_base="$(dirname "${executable}")"/../../..
436
437if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
438    export folder_base="$(dirname "${executable}")"/../..
439fi
440if [[ $tss == "1" ]]; then
441    export folder_base="$(dirname "${executable}")"/../../../../..
442fi
443
444if [[ "${executable_suffix}" != "" ]]; then
445    export executable="${executable}${executable_suffix}"
446fi
447
448if [[ -n "${folder_suffix}" ]]; then
449    export output_folder="${output_folder}${folder_suffix}"
450fi
451
452if [[ -n "${sub_folder}" ]]; then
453    export output_folder="${output_folder}/${sub_folder}"
454fi
455
456echo -e "${INFO_TAG} Perftest executable is: $executable"
457echo -e "${INFO_TAG} Output folder is: $output_folder"
458
459################################################################################
460
461if [[ "$LANGUAGE" == "python" ]]; then
462    export skip_keyed_data="1"
463    export skip_large_data="1"
464    export skip_be_tests="1"
465    export run_no_batching_tests="0"
466fi
467
468if [[ "${skip_large_data}" == "1" ]]; then
469    export datasizes_extended=${datasizes}
470elif [[ "${large_data}" == "1" ]]; then
471    export datasizes=${datasizes_extended}
472fi
473
474if [[ "${datalen_input}" != "" ]]; then
475    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
476    export datasizes=${datalen_input}
477    export datasizes_extended=${datalen_input}
478    if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
479        export skip_large_data="1"
480    fi
481else 
482    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
483        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
484        export datasizes="32 128 512 2048 8192 32768 63000"
485        export datasizes_extended="${datasizes} 102400 1048576 10485760"
486    fi
487fi
488
489if [[ "$role" != "pub" && "$role" != "sub" ]]; then
490    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
491    exit 255
492fi
493
494if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
495    echo -e "${ERROR_TAG} It must be either lat or thr"
496    exit 255
497fi
498
499if [[ "${interface}" == "" ]]; then
500    echo "Using default nics"
501    export nic_publisher=${ip_machine_1}
502    export nic_subscriber=${ip_machine_2}
503elif [[ "${interface}" == "both" ]]; then
504    export nic_publisher="enp1s0f0,eno1"
505    export nic_subscriber="enp1s0f0,eno1"
506    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
507    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
508else
509    export nic_publisher=$interface
510    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
511
512    if [[ "${interface2}" == "" ]]; then
513        export nic_subscriber=$interface
514    else
515        export nic_subscriber=$interface2
516    fi
517    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
518
519    if [[ "${ip1}" != "" ]]; then
520        export ip_publisher=$ip1
521        echo "Using ip_publisher: ${ip_publisher}"
522    fi
523
524    if [[ "${ip2}" != "" ]]; then
525        export ip_subscriber=$ip2
526        echo "Using ip_subscriber: ${ip_subscriber}"
527    fi
528
529fi
530
531export transport_string="-transport $transport"
532
533if [[ "$transport" == "UDPv4" ]]; then
534
535    export transport_string_pub="$transport_string -nic $nic_publisher"
536    export transport_string_sub="$transport_string -nic $nic_subscriber"
537
538    if [[ "$raw" == "1" ]]; then
539        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
540        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
541    fi
542
543    if [[ "$micro" == "1" || "$cert" == "1" ]]; then
544        export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
545        export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
546    fi
547
548elif [[ "$transport" == "TCP" ]]; then
549    export transport_string_pub="$transport_string \
550        -nic $nic_publisher \
551        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
552    export transport_string_sub="$transport_string \
553        -nic $nic_subscriber \
554        -peer 0@tcpv4_lan://${ip_publisher}:7400"
555elif [[ "$transport" == "TLS" ]]; then
556    export transport_string_pub="$transport_string \
557        -nic $nic_publisher \
558        -peer tlsv4_lan://${ip_subscriber}:7400"
559    export transport_string_sub="$transport_string \
560        -nic $nic_subscriber \
561        -peer tlsv4_lan://${ip_publisher}:7400"
562elif [[ "$transport" == "UDPv4_WAN" ]]; then
563    export transport_string_pub="$transport_string \
564        -nic $nic_publisher \
565        -transportPublicAddress $ip_publisher:7400"
566    export transport_string_sub="$transport_string \
567        -nic $nic_subscriber \
568        -peer 0@udpv4_wan://${ip_publisher}:7400"
569else
570    export transport_string_pub="$transport_string"
571    export transport_string_sub="$transport_string"
572fi
573
574################################################################################
575
576export pub_string="-pub \
577        ${transport_string_pub} \
578        -noPrintIntervals \
579        -executionTime $exec_time"
580
581if [[ ${lat_thr} == "lat" ]]; then
582    export pub_string="$pub_string \
583        -latencyTest"
584fi
585
586export sub_string="-sub \
587        ${transport_string_sub} \
588        -noPrintIntervals"
589
590if [[ "$role" == "pub" ]]; then
591    echo -e "$INFO_TAG Publisher side running"
592    export commands_string=${pub_string}
593    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
594else
595    echo -e "$INFO_TAG Subscriber side running"
596    export commands_string=${sub_string}
597    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
598fi
599
600###############################################################################
601
602if [[ "$dont_change_tuned_settings" != "1" ]]; then
603    echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
604    sudo /set_${lat_thr}_mode.sh
605    sleep 5
606fi
607
608echo -e "${INFO_TAG} Disabling any loss rate"
609sudo tc qdisc add dev $nic_publisher root netem loss 0%
610sudo tc qdisc del dev $nic_publisher root netem loss 0%
611
612if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
613    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
614    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
615fi
616
617cd $folder_base
618echo -e "${INFO_TAG} Folder Base is: $PWD"
619mkdir -p $output_folder
620
621if [[ "${batch_size}" != "" ]]; then
622    if [[ "${lat_thr}" == "thr" ]]; then
623        if [[ "${batch_size}" -eq 0 ]]; then
624            echo -e "${INFO_TAG} Batch size is set to 0"
625            export run_batching_tests="0"
626            export run_no_batching_tests="1"
627        else
628            echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
629            export run_batching_tests="1"
630            export run_no_batching_tests="0"
631        fi
632    else
633        echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
634        unset batch_size
635    fi
636fi
637
638# Tests that may use batching (when doing throughput tests). Also Latency tests.
639if [[ ${run_batching_tests} == "1" ]]; then
640
641    # UNKEYED
642    if [[ "${skip_unkeyed}" == "" ]]; then
643
644        # RELIABLE
645        if [[ "${skip_rel_tests}" == "" ]]; then
646            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
647        fi
648
649        # BEST EFFORT
650        if [[ "${skip_be_tests}" == "" ]]; then
651            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
652        fi
653    fi
654
655    # KEYED
656    if [[ "${skip_keyed_data}" == "" ]]; then
657
658        # RELIABLE
659        if [[ "${skip_rel_tests}" == "" ]]; then
660            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
661        fi
662
663        # BEST EFFORT
664        if [[ "${skip_be_tests}" == "" ]]; then
665            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
666        fi
667    fi
668
669fi
670
671# Tests that will not use batching
672if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
673
674    if [[ "$role" == "pub" ]]; then
675        export commands_string="${commands_string} -batchSize 0"
676    fi
677
678    # UNKEYED
679    if [[ "${skip_unkeyed}" == "" ]]; then
680
681        # RELIABLE
682        if [[ "${skip_rel_tests}" == "" ]]; then
683            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
684        fi
685
686        # BEST EFFORT
687        if [[ "${skip_be_tests}" == "" ]]; then
688            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
689        fi
690    fi
691
692    # KEYED
693    if [[ "${skip_keyed_data}" == "" ]]; then
694
695        # RELIABLE
696        if [[ "${skip_rel_tests}" == "" ]]; then
697            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
698        fi
699
700        # BEST EFFORT
701        if [[ "${skip_be_tests}" == "" ]]; then
702            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
703        fi
704    fi
705
706fi
707
708if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
709    echo -e "${INFO_TAG} Disabling loss rate"
710    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
711fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport UDPv4

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)

2.2.4.2. Keyed, UDPv4 10Gbps Network, C#

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers are for best-effort as well as strict reliable reliability scenarios.

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by a simplified version of RTI Perftest for the Modern C# implementation. These numbers are the exact output with no further processing.

  • Best Effort

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

3185

2189613.7

18

1514046223

19

19

20

28

1514046223

64

19

3.3

18

642

19

20

21

28

642

128

19

4.4

18

650

19

20

21

29

650

256

19

2.2

18

585

19

19

21

28

585

512

19

6.2

19

584

20

20

21

31

584

1024

25

6.5

20

312

26

28

32

48

312

2048

46

391.6

22

100070

43

62

68

90

100070

4096

44

354.1

27

100098

41

57

63

86

100098

8192

46

8.1

38

331

41

59

62

70

331

16384

90

31.3

49

419

84

125

138

154

419

32768

178

10.2

83

555

178

180

181

203

555

63000

191

11.2

108

1047

191

193

196

202

1047

  • Reliable

Sample Size (Bytes)

Avg (μs)

Std (μs)

Min (μs)

Max (μs)

50% (μs)

90% (μs)

99% (μs)

99.99% (μs)

99.9999% (μs)

32

20

4.6

19

774

20

21

22

49

774

64

20

3.6

19

586

20

21

22

37

586

128

21

3.2

20

776

21

22

23

41

776

256

22

8.1

20

750

21

32

39

48

750

512

24

5.1

21

680

22

32

41

48

680

1024

26

5.5

21

147

26

28

51

61

147

2048

29

10.2

24

927

30

33

58

81

927

4096

31

11.9

29

902

30

33

64

549

902

8192

46

12.9

40

290

42

60

84

119

290

16384

94

29.3

50

361

87

129

136

144

361

32768

176

14.4

77

654

180

182

187

193

654

63000

158

16.2

110

1196

159

170

185

196

1196


Perftest Scripts

To produce these tests, we executed RTI Perftest for C#. The exact script used can be found here:

  1#!/bin/bash
  2filename=$0
  3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
  4
  5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
  6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
  7
  8export domain="2"
  9export exec_time=20
 10export num_reps=1
 11export instance_number=100000
 12export core=0
 13export run_batching_tests="1"
 14export run_no_batching_tests="1"
 15
 16# We will use some colors to improve visibility of errors and info messages.
 17RED='\033[0;31m'
 18GREEN='\033[0;32m'
 19YELLOW='\033[0;33m'
 20BLUE='\033[0;34m'
 21LIGHTBLUE='\033[0;36m'
 22NC='\033[0m'
 23INFO_TAG="${GREEN}[INFO][$role]:${NC}"
 24WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
 25ERROR_TAG="${RED}[ERROR][$role]:${NC}"
 26
 27################################################################################
 28
 29function disable_colors() {
 30    export RED=""
 31    export GREEN=""
 32    export YELLOW=""
 33    export NC=""
 34    export BLUE=""
 35    export LIGHTBLUE=""
 36    export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
 37    export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
 38    export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
 39}
 40
 41function change_domain() {
 42    if [[ "$domain" == "1" ]]; then
 43        export domain="2"
 44    else
 45        export domain="1"
 46    fi
 47}
 48
 49# Function to process and append lines to the output file
 50function append_to_output_file() {
 51    local line=${1?line required}; readonly line
 52    local append=$(echo "${2?append required}" | tr -s ' '); readonly append
 53    local file=${3?file required}; readonly file
 54    if [[ "${line: -1}" == $'\n' ]]; then
 55        line="${line::-1}"
 56    fi
 57    echo "${line}${append}" >> "$file"
 58}
 59
 60# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
 61function execute_test() {
 62
 63    local keyed_unkeyed=$1
 64    local rel_be=$2
 65    local datasizes_test=$3
 66    local other_args=$4
 67    local name_suffix=$5
 68
 69    local commands_string_test=$commands_string
 70    local tag=""
 71
 72    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
 73        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
 74        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
 75    else
 76        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
 77    fi
 78
 79    if [[ "${rel_be}" == "be" ]]; then
 80        commands_string_test="${commands_string_test} -bestEffort"
 81        tag="${tag}${YELLOW}BE${NC}]"
 82    else
 83        tag="${tag}${RED}REL${NC}]"
 84    fi
 85
 86    # If batch_size is set, we will add it to the command line of the publisher side.
 87    if [[ "$batch_size" != "" && "$role" == "pub" ]]; then
 88        commands_string_test="${commands_string_test} -batchSize $batch_size"
 89    fi
 90
 91    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
 92
 93    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
 94
 95    if [[ "$role" == "pub" ]]; then
 96        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
 97    fi
 98
 99    if [[ "$thread_cpu_affinity" != "" ]]; then
100        commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
101    fi
102
103    local pre_command_string=""
104
105    if [[ "$thread_priorities" != "" ]]; then
106        echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
107        # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
108        export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
109        pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
110        commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
111    fi
112
113    if [[ "$pin_memory" != "" ]]; then
114        commands_string_test="${commands_string_test} -pinMemory"
115    fi
116
117    if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
118        pre_command_string="$pre_command_string taskset -c $core"
119    fi
120
121    if [[ "$LANGUAGE" == "python" ]]; then
122        pre_command_string="$pre_command_string python3 "
123    fi
124
125    if [[ "$DOCKER" == "1" ]]; then
126        pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
127        executable=""
128    fi
129
130    # Get the aprox time this will take:
131    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
132    total_time=$((total_tests * exec_time))
133
134    touch $output_file
135    local no_headers=""
136    local current_test=0
137    for index in $(seq 1 ${num_reps}); do
138        for DATALEN in ${datasizes_test}; do
139            current_test=$((current_test + 1))
140
141            if [[ ! -s $output_file ]]; then
142                echo -e "${INFO_TAG} Output file is empty, filling the header."
143                no_headers=""
144            else
145                echo -e "${INFO_TAG} Output file is not empty."
146                no_headers=" -noOutputHeaders"
147            fi
148
149            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
150            if [[ "$role" == "pub" ]]; then
151                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
152                echo -e ${BLUE}$command${NC}
153            else
154                echo -e ${LIGHTBLUE}$command${NC}
155            fi
156
157            # In certain cases we need to wait a bit before running the test, this is
158            # because the previous test might not be finished on the other side yet, or because the
159            # discovery mechanism does not work like in connext DDS.
160            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
161                sleep 3
162            fi
163            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
164                sleep 5
165            fi
166
167            # Gather netstat info before running the test
168            if [[ "${get_netstat_info}" == "1" ]]; then
169                echo -e "${INFO_TAG} Getting netstat info before"
170                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
171            fi
172
173            # Execute the command and capture the output
174            if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
175                echo -e "${INFO_TAG} Using C++98, using -outputFile option"
176                touch ${output_file}_tmp_line
177                chmod 777 ${output_file}_tmp_line
178                eval $command -outputFile ${output_file}_tmp_line
179            else
180                eval $command > ${output_file}_tmp_line
181            fi
182
183            number_of_lines=$(wc -l < ${output_file}_tmp_line)
184            echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
185
186            # Check if this is the first test for this output file.
187            local is_first_test=0
188            if [[ ! -s $output_file ]]; then
189                is_first_test=1
190            fi
191            local expected_lines=$((is_first_test+1))
192
193            if [[ $number_of_lines -ne ${expected_lines} ]]; then
194                echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
195                echo -e "${RED} The content is: \n\"\"\""
196                cat ${output_file}_tmp_line
197                echo -e "\"\"\""
198                echo -e "Not adding it to the output file. ${NC}"
199            else
200                if [[ $number_of_lines -gt 1 ]]; then
201                    header_line=$(head -n 1 "${output_file}_tmp_line")
202                    echo -e "${INFO_TAG} Header line is: $header_line"
203                fi
204                result_line=$(tail -n 1 "${output_file}_tmp_line")
205                echo -e "${INFO_TAG} Result line is: $result_line"
206
207                # If this is the first test, we need to add the header line to the output file
208                if [[ $is_first_test -eq 1 ]]; then
209                    append_to_output_file "$header_line" ", command-line" $output_file
210                fi
211
212                command=$(echo $command | sed 's/,/:comma:/g')
213                # Remove the `"` characters from the command line
214                command=$(echo $command | sed 's/\"//g')
215
216                # Append the result line to the output file
217                append_to_output_file "$result_line" ", $command" $output_file
218            fi
219
220            # Always remove the temporary file
221            rm -rf ${output_file}_tmp_line
222
223            # Gather netstat info after running the test
224            if [[ "${get_netstat_info}" == "1" ]]; then
225                echo -e "${INFO_TAG} Getting netstat info after"
226                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
227                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
228                python3 $script_location/../../../tools/diff_netstat_output.py \
229                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
230                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
231                    -d $DATALEN $no_header_netstat \
232                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
233                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
234                no_header_netstat=" -nh"
235            fi
236
237            change_domain
238        done
239    done
240}
241
242################################################################################
243# PARSE COMMAND LINE OPTIONS:
244
245while [ "$1" != "" ]; do
246    case $1 in
247        --executable)
248            executable=$2
249            shift
250            ;;
251        --docker)
252            DOCKER="1"
253            ;;
254        --output-folder)
255            output_folder=$2
256            shift
257            ;;
258        --role)
259            export role=$2
260            shift
261            ;;
262        --core)
263            export core=$2
264            shift
265            ;;
266        --assign-core-list | --thread-cpu-affinity)
267            export thread_cpu_affinity=$2
268            shift
269            ;;
270        --assign-thread-priorities | --thread-priorities)
271            export thread_priorities=$2
272            shift
273            ;;
274        --pin-memory)
275            export pin_memory="1"
276            ;;
277        --test-kind)
278            export lat_thr=$2
279            shift
280            ;;
281        --interface1)
282            export interface=$2
283            shift
284            ;;
285        --interface2)
286            export interface2=$2
287            shift
288            ;;
289        --ip1)
290            export ip1=$2
291            shift
292            ;;
293        --ip2)
294            export ip2=$2
295            shift
296            ;;
297        --repetitions)
298            export num_reps=$2
299            shift
300            ;;
301        --domain)
302            export domain=$2
303            shift
304            ;;
305        --execution-time)
306            export exec_time=$2
307            shift
308            ;;
309        --transport)
310            export transport=$2
311            shift
312            ;;
313        --datalen)
314            export datalen_input=$2
315            shift
316            ;;
317        --file-suffix)
318            export file_suffix=$2
319            shift
320            ;;
321        --folder-suffix)
322            export folder_suffix=$2
323            shift
324            ;;
325        --sub-folder)
326            sub_folder=$2
327            shift
328            ;;
329        --executable-suffix)
330            export executable_suffix=$2
331            shift
332            ;;
333        --extra-arguments)
334            export extra_arguments=$2
335            shift
336            ;;
337        --extra-arguments-pub)
338            export extra_arguments_pub=$2
339            shift
340            ;;
341        --extra-arguments-sub)
342            export extra_arguments_sub=$2
343            shift
344            ;;
345        --skip-be)
346            export skip_be_tests="1"
347            ;;
348        --skip-rel)
349            export skip_rel_tests="1"
350            ;;
351        --skip-keyed)
352            export skip_keyed_data="1"
353            ;;
354        --skip-large-data)
355            export skip_large_data="1"
356            ;;
357        --large-data)
358            export large_data="1"
359            ;;
360        --keyed)
361            export skip_unkeyed="1"
362            ;;
363        --unkeyed)
364            export skip_keyed_data="1"
365            ;;
366        --no-batching | --skip-batching)
367            export run_batching_tests="0"
368            export run_no_batching_tests="1"
369            ;;
370        --skip-no-batching)
371            export run_batching_tests="1"
372            export run_no_batching_tests="0"
373            ;;
374        --batch-size)
375            export batch_size=$2
376            shift
377            ;;
378        --reliable)
379            export skip_be_tests="1"
380            ;;
381        --best-effort)
382            export skip_rel_tests="1"
383            ;;
384        --security-gov)
385            export security_only="$2"
386            shift
387            ;;
388        --micro)
389            export micro="1"
390            ;;
391        --cert)
392            export cert="1"
393            ;;
394        --raw | --raw-transport)
395            export raw="1"
396            ;;
397        --tss)
398            export tss="1"
399            ;;
400        --no-colors)
401            export NO_COLORS="1"
402            ;;
403        --language)
404            export LANGUAGE=$2
405            shift
406            ;;
407        --loss-rate)
408            export loss_rate=$2
409            shift
410            ;;
411        --get-netstat-info | --netstat)
412            export get_netstat_info="1"
413            ;;
414        --reduced-data-sizes-set)
415            export REDUCED_DATA_SIZES_SET="1"
416            ;;
417        --dont-change-tuned-settings)
418            export dont_change_tuned_settings="1"
419            ;;
420        --no-taskset)
421            export no_taskset=1
422            ;;
423        *)
424            echo -e "unknown parameter \"$1\""
425            exit 255
426            ;;
427    esac
428    shift
429done
430
431if [[ "$NO_COLORS" == "1" ]]; then
432    disable_colors
433fi
434
435export folder_base="$(dirname "${executable}")"/../../..
436
437if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
438    export folder_base="$(dirname "${executable}")"/../..
439fi
440if [[ $tss == "1" ]]; then
441    export folder_base="$(dirname "${executable}")"/../../../../..
442fi
443
444if [[ "${executable_suffix}" != "" ]]; then
445    export executable="${executable}${executable_suffix}"
446fi
447
448if [[ -n "${folder_suffix}" ]]; then
449    export output_folder="${output_folder}${folder_suffix}"
450fi
451
452if [[ -n "${sub_folder}" ]]; then
453    export output_folder="${output_folder}/${sub_folder}"
454fi
455
456echo -e "${INFO_TAG} Perftest executable is: $executable"
457echo -e "${INFO_TAG} Output folder is: $output_folder"
458
459################################################################################
460
461if [[ "$LANGUAGE" == "python" ]]; then
462    export skip_keyed_data="1"
463    export skip_large_data="1"
464    export skip_be_tests="1"
465    export run_no_batching_tests="0"
466fi
467
468if [[ "${skip_large_data}" == "1" ]]; then
469    export datasizes_extended=${datasizes}
470elif [[ "${large_data}" == "1" ]]; then
471    export datasizes=${datasizes_extended}
472fi
473
474if [[ "${datalen_input}" != "" ]]; then
475    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
476    export datasizes=${datalen_input}
477    export datasizes_extended=${datalen_input}
478    if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
479        export skip_large_data="1"
480    fi
481else 
482    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
483        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
484        export datasizes="32 128 512 2048 8192 32768 63000"
485        export datasizes_extended="${datasizes} 102400 1048576 10485760"
486    fi
487fi
488
489if [[ "$role" != "pub" && "$role" != "sub" ]]; then
490    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
491    exit 255
492fi
493
494if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
495    echo -e "${ERROR_TAG} It must be either lat or thr"
496    exit 255
497fi
498
499if [[ "${interface}" == "" ]]; then
500    echo "Using default nics"
501    export nic_publisher=${ip_machine_1}
502    export nic_subscriber=${ip_machine_2}
503elif [[ "${interface}" == "both" ]]; then
504    export nic_publisher="enp1s0f0,eno1"
505    export nic_subscriber="enp1s0f0,eno1"
506    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
507    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
508else
509    export nic_publisher=$interface
510    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
511
512    if [[ "${interface2}" == "" ]]; then
513        export nic_subscriber=$interface
514    else
515        export nic_subscriber=$interface2
516    fi
517    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
518
519    if [[ "${ip1}" != "" ]]; then
520        export ip_publisher=$ip1
521        echo "Using ip_publisher: ${ip_publisher}"
522    fi
523
524    if [[ "${ip2}" != "" ]]; then
525        export ip_subscriber=$ip2
526        echo "Using ip_subscriber: ${ip_subscriber}"
527    fi
528
529fi
530
531export transport_string="-transport $transport"
532
533if [[ "$transport" == "UDPv4" ]]; then
534
535    export transport_string_pub="$transport_string -nic $nic_publisher"
536    export transport_string_sub="$transport_string -nic $nic_subscriber"
537
538    if [[ "$raw" == "1" ]]; then
539        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
540        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
541    fi
542
543    if [[ "$micro" == "1" || "$cert" == "1" ]]; then
544        export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
545        export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
546    fi
547
548elif [[ "$transport" == "TCP" ]]; then
549    export transport_string_pub="$transport_string \
550        -nic $nic_publisher \
551        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
552    export transport_string_sub="$transport_string \
553        -nic $nic_subscriber \
554        -peer 0@tcpv4_lan://${ip_publisher}:7400"
555elif [[ "$transport" == "TLS" ]]; then
556    export transport_string_pub="$transport_string \
557        -nic $nic_publisher \
558        -peer tlsv4_lan://${ip_subscriber}:7400"
559    export transport_string_sub="$transport_string \
560        -nic $nic_subscriber \
561        -peer tlsv4_lan://${ip_publisher}:7400"
562elif [[ "$transport" == "UDPv4_WAN" ]]; then
563    export transport_string_pub="$transport_string \
564        -nic $nic_publisher \
565        -transportPublicAddress $ip_publisher:7400"
566    export transport_string_sub="$transport_string \
567        -nic $nic_subscriber \
568        -peer 0@udpv4_wan://${ip_publisher}:7400"
569else
570    export transport_string_pub="$transport_string"
571    export transport_string_sub="$transport_string"
572fi
573
574################################################################################
575
576export pub_string="-pub \
577        ${transport_string_pub} \
578        -noPrintIntervals \
579        -executionTime $exec_time"
580
581if [[ ${lat_thr} == "lat" ]]; then
582    export pub_string="$pub_string \
583        -latencyTest"
584fi
585
586export sub_string="-sub \
587        ${transport_string_sub} \
588        -noPrintIntervals"
589
590if [[ "$role" == "pub" ]]; then
591    echo -e "$INFO_TAG Publisher side running"
592    export commands_string=${pub_string}
593    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
594else
595    echo -e "$INFO_TAG Subscriber side running"
596    export commands_string=${sub_string}
597    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
598fi
599
600###############################################################################
601
602if [[ "$dont_change_tuned_settings" != "1" ]]; then
603    echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
604    sudo /set_${lat_thr}_mode.sh
605    sleep 5
606fi
607
608echo -e "${INFO_TAG} Disabling any loss rate"
609sudo tc qdisc add dev $nic_publisher root netem loss 0%
610sudo tc qdisc del dev $nic_publisher root netem loss 0%
611
612if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
613    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
614    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
615fi
616
617cd $folder_base
618echo -e "${INFO_TAG} Folder Base is: $PWD"
619mkdir -p $output_folder
620
621if [[ "${batch_size}" != "" ]]; then
622    if [[ "${lat_thr}" == "thr" ]]; then
623        if [[ "${batch_size}" -eq 0 ]]; then
624            echo -e "${INFO_TAG} Batch size is set to 0"
625            export run_batching_tests="0"
626            export run_no_batching_tests="1"
627        else
628            echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
629            export run_batching_tests="1"
630            export run_no_batching_tests="0"
631        fi
632    else
633        echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
634        unset batch_size
635    fi
636fi
637
638# Tests that may use batching (when doing throughput tests). Also Latency tests.
639if [[ ${run_batching_tests} == "1" ]]; then
640
641    # UNKEYED
642    if [[ "${skip_unkeyed}" == "" ]]; then
643
644        # RELIABLE
645        if [[ "${skip_rel_tests}" == "" ]]; then
646            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
647        fi
648
649        # BEST EFFORT
650        if [[ "${skip_be_tests}" == "" ]]; then
651            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
652        fi
653    fi
654
655    # KEYED
656    if [[ "${skip_keyed_data}" == "" ]]; then
657
658        # RELIABLE
659        if [[ "${skip_rel_tests}" == "" ]]; then
660            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
661        fi
662
663        # BEST EFFORT
664        if [[ "${skip_be_tests}" == "" ]]; then
665            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
666        fi
667    fi
668
669fi
670
671# Tests that will not use batching
672if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
673
674    if [[ "$role" == "pub" ]]; then
675        export commands_string="${commands_string} -batchSize 0"
676    fi
677
678    # UNKEYED
679    if [[ "${skip_unkeyed}" == "" ]]; then
680
681        # RELIABLE
682        if [[ "${skip_rel_tests}" == "" ]]; then
683            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
684        fi
685
686        # BEST EFFORT
687        if [[ "${skip_be_tests}" == "" ]]; then
688            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
689        fi
690    fi
691
692    # KEYED
693    if [[ "${skip_keyed_data}" == "" ]]; then
694
695        # RELIABLE
696        if [[ "${skip_rel_tests}" == "" ]]; then
697            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
698        fi
699
700        # BEST EFFORT
701        if [[ "${skip_be_tests}" == "" ]]; then
702            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
703        fi
704    fi
705
706fi
707
708if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
709    echo -e "${INFO_TAG} Disabling loss rate"
710    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
711fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5echo -e "[Calling base_script/script.sh]"
6"${script_location}/../base_script/script.sh" $@ --transport UDPv4

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)