5.2.1.1. LightWeight Security PSK vs HMAC Only vs Full Security PSK
In RTI Connext 7.1 we introduced Pre-Shared Key Protection (PSK)` to secure the RTPS communication.
This PSK can be leveraged in two ways: As a part of RTI Security Plugins, protecting bootstrapping
communications before authentication is successfully concluded or as a dedicated library named RTI
Lightweight Security Plugins, where PSK` is the only option and protects the entirety of the communication.
PSK can be configured to use various cryptographic algorithms: AES128 or AES256 in either GMAC
(for integrity) or GCM (for both integrity and confidentiality) modes.
In RTI Connext 7.1 we deprecated HMAC-only mode which is scheduled to be superseded in the next release with Lightweight Security.
It supports non-configurable HMAC-SHA256 which only protects data integrity without its confidentiality.
Charts below compare performance of different Lightweight Security’s AES256 GCM and GMAC algorithms
with HMAC-only and a non-secure scenario. In all tests PSK performs better than HMAC-only.
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set, we will add it to the command line of the publisher side.
89 if [[ "$batch_size" != "" && "$role" == "pub" ]]; then
90 commands_string_test="${commands_string_test} -batchSize $batch_size"
91 fi
92
93 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
94
95 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
96
97 if [[ "$role" == "pub" ]]; then
98 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
99 fi
100
101 if [[ "$thread_cpu_affinity" != "" ]]; then
102 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
103 fi
104
105 local pre_command_string=""
106
107 if [[ "$thread_priorities" != "" ]]; then
108 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
109 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
110 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
111 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
112 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
113 fi
114
115 if [[ "$pin_memory" != "" ]]; then
116 commands_string_test="${commands_string_test} -pinMemory"
117 fi
118
119 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
120 pre_command_string="$pre_command_string taskset -c $core"
121 fi
122
123 if [[ "$LANGUAGE" == "python" ]]; then
124 pre_command_string="$pre_command_string python3 "
125 fi
126
127 if [[ "$DOCKER" == "1" ]]; then
128 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
129 executable=""
130 fi
131
132 # Get the aprox time this will take:
133 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
134 total_time=$((total_tests * exec_time))
135
136 touch $output_file
137 local no_headers=""
138 local current_test=0
139 for index in $(seq 1 ${num_reps}); do
140 for DATALEN in ${datasizes_test}; do
141 current_test=$((current_test + 1))
142
143 if [[ ! -s $output_file ]]; then
144 echo -e "${INFO_TAG} Output file is empty, filling the header."
145 no_headers=""
146 else
147 echo -e "${INFO_TAG} Output file is not empty."
148 no_headers=" -noOutputHeaders"
149 fi
150
151 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
152 if [[ "$role" == "pub" ]]; then
153 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
154 echo -e ${BLUE}$command${NC}
155 else
156 echo -e ${LIGHTBLUE}$command${NC}
157 fi
158
159 # In certain cases we need to wait a bit before running the test, this is
160 # because the previous test might not be finished on the other side yet, or because the
161 # discovery mechanism does not work like in connext DDS.
162 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
163 sleep 3
164 fi
165 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
166 sleep 5
167 fi
168
169 # Gather netstat info before running the test
170 if [[ "${get_netstat_info}" == "1" ]]; then
171 echo -e "${INFO_TAG} Getting netstat info before"
172 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
173 fi
174
175 # Execute the command and capture the output
176 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
177 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
178 touch ${output_file}_tmp_line
179 chmod 777 ${output_file}_tmp_line
180 eval $command -outputFile ${output_file}_tmp_line
181 else
182 eval $command > ${output_file}_tmp_line
183 fi
184
185 number_of_lines=$(wc -l < ${output_file}_tmp_line)
186 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
187
188 # Check if this is the first test for this output file.
189 local is_first_test=0
190 if [[ ! -s $output_file ]]; then
191 is_first_test=1
192 fi
193 local expected_lines=$((is_first_test+1))
194
195 if [[ $number_of_lines -ne ${expected_lines} ]]; then
196 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
197 echo -e "${RED} The content is: \n\"\"\""
198 cat ${output_file}_tmp_line
199 echo -e "\"\"\""
200 echo -e "Not adding it to the output file. ${NC}"
201 else
202 if [[ $number_of_lines -gt 1 ]]; then
203 header_line=$(head -n 1 "${output_file}_tmp_line")
204 echo -e "${INFO_TAG} Header line is: $header_line"
205 fi
206 result_line=$(tail -n 1 "${output_file}_tmp_line")
207 echo -e "${INFO_TAG} Result line is: $result_line"
208
209 # If this is the first test, we need to add the header line to the output file
210 if [[ $is_first_test -eq 1 ]]; then
211 append_to_output_file "$header_line" ", command-line" $output_file
212 fi
213
214 command=$(echo $command | sed 's/,/:comma:/g')
215 # Remove the `"` characters from the command line
216 command=$(echo $command | sed 's/\"//g')
217
218 # Append the result line to the output file
219 append_to_output_file "$result_line" ", $command" $output_file
220 fi
221
222 # Always remove the temporary file
223 rm -rf ${output_file}_tmp_line
224
225 # Gather netstat info after running the test
226 if [[ "${get_netstat_info}" == "1" ]]; then
227 echo -e "${INFO_TAG} Getting netstat info after"
228 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
229 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
230 python3 $script_location/../../../tools/diff_netstat_output.py \
231 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
232 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
233 -d $DATALEN $no_header_netstat \
234 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
235 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
236 no_header_netstat=" -nh"
237 fi
238
239 change_domain
240 done
241 done
242}
243
244################################################################################
245# PARSE COMMAND LINE OPTIONS:
246
247while [ "$1" != "" ]; do
248 case $1 in
249 --executable)
250 executable=$2
251 shift
252 ;;
253 --docker)
254 DOCKER="1"
255 ;;
256 --output-folder)
257 output_folder=$2
258 shift
259 ;;
260 --role)
261 export role=$2
262 shift
263 ;;
264 --core)
265 export core=$2
266 shift
267 ;;
268 --assign-core-list | --thread-cpu-affinity)
269 export thread_cpu_affinity=$2
270 shift
271 ;;
272 --assign-thread-priorities | --thread-priorities)
273 export thread_priorities=$2
274 shift
275 ;;
276 --pin-memory)
277 export pin_memory="1"
278 ;;
279 --test-kind)
280 export lat_thr=$2
281 shift
282 ;;
283 --interface1)
284 export interface=$2
285 shift
286 ;;
287 --interface2)
288 export interface2=$2
289 shift
290 ;;
291 --ip1)
292 export ip1=$2
293 shift
294 ;;
295 --ip2)
296 export ip2=$2
297 shift
298 ;;
299 --repetitions)
300 export num_reps=$2
301 shift
302 ;;
303 --domain)
304 export domain=$2
305 shift
306 ;;
307 --execution-time)
308 export exec_time=$2
309 shift
310 ;;
311 --transport)
312 export transport=$2
313 shift
314 ;;
315 --datalen)
316 export datalen_input=$2
317 shift
318 ;;
319 --file-suffix)
320 export file_suffix=$2
321 shift
322 ;;
323 --folder-suffix)
324 export folder_suffix=$2
325 shift
326 ;;
327 --sub-folder)
328 sub_folder=$2
329 shift
330 ;;
331 --executable-suffix)
332 export executable_suffix=$2
333 shift
334 ;;
335 --extra-arguments)
336 export extra_arguments=$2
337 shift
338 ;;
339 --extra-arguments-pub)
340 export extra_arguments_pub=$2
341 shift
342 ;;
343 --extra-arguments-sub)
344 export extra_arguments_sub=$2
345 shift
346 ;;
347 --skip-be)
348 export skip_be_tests="1"
349 ;;
350 --skip-rel)
351 export skip_rel_tests="1"
352 ;;
353 --skip-keyed)
354 export skip_keyed_data="1"
355 ;;
356 --skip-large-data)
357 export skip_large_data="1"
358 ;;
359 --large-data)
360 export large_data="1"
361 ;;
362 --keyed)
363 export skip_unkeyed="1"
364 ;;
365 --unkeyed)
366 export skip_keyed_data="1"
367 ;;
368 --no-batching | --skip-batching)
369 export run_batching_tests="0"
370 export run_no_batching_tests="1"
371 ;;
372 --skip-no-batching)
373 export run_batching_tests="1"
374 export run_no_batching_tests="0"
375 ;;
376 --batch-size)
377 export batch_size=$2
378 shift
379 ;;
380 --reliable)
381 export skip_be_tests="1"
382 ;;
383 --best-effort)
384 export skip_rel_tests="1"
385 ;;
386 --security-gov)
387 export security_only="$2"
388 shift
389 ;;
390 --micro)
391 export micro="1"
392 ;;
393 --cert)
394 export cert="1"
395 ;;
396 --raw | --raw-transport)
397 export raw="1"
398 ;;
399 --tss)
400 export tss="1"
401 ;;
402 --no-colors)
403 export NO_COLORS="1"
404 ;;
405 --language)
406 export LANGUAGE=$2
407 shift
408 ;;
409 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
410 export nic_loss_rate=$2
411 export configure_network="1"
412 shift
413 ;;
414 --delay | --nic-delay-ms | --nic-delay)
415 export nic_delay_ms=$2
416 export configure_network="1"
417 shift
418 ;;
419 --get-netstat-info | --netstat)
420 export get_netstat_info="1"
421 ;;
422 --reduced-data-sizes-set)
423 export REDUCED_DATA_SIZES_SET="1"
424 ;;
425 --dont-change-tuned-settings)
426 export dont_change_tuned_settings="1"
427 ;;
428 --no-taskset)
429 export no_taskset=1
430 ;;
431 --asynchronous | --async)
432 export ASYNC="1"
433 ;;
434 *)
435 echo -e "unknown parameter \"$1\""
436 exit 255
437 ;;
438 esac
439 shift
440done
441
442if [[ "$NO_COLORS" == "1" ]]; then
443 disable_colors
444fi
445
446export folder_base="$(dirname "${executable}")"/../../..
447
448if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
449 export folder_base="$(dirname "${executable}")"/../..
450fi
451if [[ $tss == "1" ]]; then
452 export folder_base="$(dirname "${executable}")"/../../../../..
453fi
454
455if [[ "${executable_suffix}" != "" ]]; then
456 export executable="${executable}${executable_suffix}"
457fi
458
459if [[ -n "${folder_suffix}" ]]; then
460 export output_folder="${output_folder}${folder_suffix}"
461fi
462
463if [[ -n "${sub_folder}" ]]; then
464 export output_folder="${output_folder}/${sub_folder}"
465fi
466
467if [[ "${ASYNC}" == "1" ]]; then
468 export output_folder="${output_folder}/async"
469fi
470
471echo -e "${INFO_TAG} Perftest executable is: $executable"
472echo -e "${INFO_TAG} Output folder is: $output_folder"
473
474################################################################################
475
476if [[ "$LANGUAGE" == "python" ]]; then
477 export skip_keyed_data="1"
478 export skip_large_data="1"
479 export skip_be_tests="1"
480 export run_no_batching_tests="0"
481fi
482
483if [[ "${skip_large_data}" == "1" ]]; then
484 export datasizes_extended=${datasizes}
485elif [[ "${large_data}" == "1" ]]; then
486 export datasizes=${datasizes_extended}
487fi
488
489if [[ "${datalen_input}" != "" ]]; then
490 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
491 export datasizes=${datalen_input}
492 export datasizes_extended=${datalen_input}
493 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
494 export skip_large_data="1"
495 fi
496else
497 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
498 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
499 export datasizes="32 128 512 2048 8192 32768 63000"
500 export datasizes_extended="${datasizes} 102400 1048576 10485760"
501 fi
502fi
503
504if [[ "$role" != "pub" && "$role" != "sub" ]]; then
505 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
506 exit 255
507fi
508
509if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
510 echo -e "${ERROR_TAG} It must be either lat or thr"
511 exit 255
512fi
513
514if [[ "${interface}" == "" ]]; then
515 echo "Using default nics"
516 export nic_publisher=${ip_machine_1}
517 export nic_subscriber=${ip_machine_2}
518elif [[ "${interface}" == "both" ]]; then
519 export nic_publisher="enp1s0f0,eno1"
520 export nic_subscriber="enp1s0f0,eno1"
521 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
522 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
523else
524 export nic_publisher=$interface
525 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
526
527 if [[ "${interface2}" == "" ]]; then
528 export nic_subscriber=$interface
529 else
530 export nic_subscriber=$interface2
531 fi
532 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
533
534 if [[ "${ip1}" != "" ]]; then
535 export ip_publisher=$ip1
536 echo "Using ip_publisher: ${ip_publisher}"
537 fi
538
539 if [[ "${ip2}" != "" ]]; then
540 export ip_subscriber=$ip2
541 echo "Using ip_subscriber: ${ip_subscriber}"
542 fi
543
544fi
545
546if [[ "$transport" != "" ]]; then
547 export transport_string="-transport $transport"
548
549 if [[ "$transport" == "UDPv4" ]]; then
550
551 export transport_string_pub="$transport_string -nic $nic_publisher"
552 export transport_string_sub="$transport_string -nic $nic_subscriber"
553
554 if [[ "$raw" == "1" ]]; then
555 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
556 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
557 fi
558
559 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
560 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
561 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
562 fi
563
564 elif [[ "$transport" == "TCP" ]]; then
565 export transport_string_pub="$transport_string \
566 -nic $nic_publisher \
567 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
568 export transport_string_sub="$transport_string \
569 -nic $nic_subscriber \
570 -peer 0@tcpv4_lan://${ip_publisher}:7400"
571 elif [[ "$transport" == "TLS" ]]; then
572 export transport_string_pub="$transport_string \
573 -nic $nic_publisher \
574 -peer tlsv4_lan://${ip_subscriber}:7400"
575 export transport_string_sub="$transport_string \
576 -nic $nic_subscriber \
577 -peer tlsv4_lan://${ip_publisher}:7400"
578 elif [[ "$transport" == "UDPv4_WAN" ]]; then
579 export transport_string_pub="$transport_string \
580 -nic $nic_publisher \
581 -transportPublicAddress $ip_publisher:7400"
582 export transport_string_sub="$transport_string \
583 -nic $nic_subscriber \
584 -peer 0@udpv4_wan://${ip_publisher}:7400"
585 else
586 export transport_string_pub="$transport_string"
587 export transport_string_sub="$transport_string"
588 fi
589fi
590
591################################################################################
592
593export pub_string="-pub \
594 ${transport_string_pub} \
595 -noPrintIntervals \
596 -executionTime $exec_time"
597
598if [[ ${lat_thr} == "lat" ]]; then
599 export pub_string="$pub_string \
600 -latencyTest"
601fi
602
603if [[ "$role" == "pub" ]]; then
604
605 if [[ "$configure_network" == "1" ]]; then
606 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
607 export pub_string="$pub_string -initialBurst 1"
608 fi
609
610 if [[ "$ASYNC" == "1" ]]; then
611 export pub_string="${pub_string} -asynchronous"
612 fi
613fi
614
615export sub_string="-sub \
616 ${transport_string_sub} \
617 -noPrintIntervals"
618
619if [[ "$role" == "pub" ]]; then
620 echo -e "$INFO_TAG Publisher side running"
621 export commands_string=${pub_string}
622 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
623else
624 echo -e "$INFO_TAG Subscriber side running"
625 export commands_string=${sub_string}
626 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
627fi
628
629###############################################################################
630
631if [[ "$dont_change_tuned_settings" != "1" ]]; then
632 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
633 sudo /set_${lat_thr}_mode.sh
634 sleep 5
635fi
636
637if [[ "$role" == "pub" ]]; then
638 export interface_to_configure=$nic_publisher
639else
640 export interface_to_configure=$nic_subscriber
641fi
642
643echo -e "${INFO_TAG} Resetting network interface to default state"
644# Try to delete any existing netem rules (ignore errors if none exist)
645sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
646# Reset to default (pfifo_fast)
647sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
648
649if [[ "${configure_network}" == "1" ]]; then
650 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
651 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
652fi
653
654cd $folder_base
655echo -e "${INFO_TAG} Folder Base is: $PWD"
656mkdir -p $output_folder
657
658if [[ "${batch_size}" != "" ]]; then
659 if [[ "${lat_thr}" == "thr" ]]; then
660 if [[ "${batch_size}" -eq 0 ]]; then
661 echo -e "${INFO_TAG} Batch size is set to 0"
662 export run_batching_tests="0"
663 export run_no_batching_tests="1"
664 else
665 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
666 export run_batching_tests="1"
667 export run_no_batching_tests="0"
668 fi
669 else
670 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
671 unset batch_size
672 fi
673fi
674
675# Tests that may use batching (when doing throughput tests). Also Latency tests.
676if [[ ${run_batching_tests} == "1" ]]; then
677
678 # UNKEYED
679 if [[ "${skip_unkeyed}" == "" ]]; then
680
681 # RELIABLE
682 if [[ "${skip_rel_tests}" == "" ]]; then
683 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
684 fi
685
686 # BEST EFFORT
687 if [[ "${skip_be_tests}" == "" ]]; then
688 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
689 fi
690 fi
691
692 # KEYED
693 if [[ "${skip_keyed_data}" == "" ]]; then
694
695 # RELIABLE
696 if [[ "${skip_rel_tests}" == "" ]]; then
697 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
698 fi
699
700 # BEST EFFORT
701 if [[ "${skip_be_tests}" == "" ]]; then
702 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
703 fi
704 fi
705
706fi
707
708# Tests that will not use batching
709if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
710
711 if [[ "$role" == "pub" ]]; then
712 export commands_string="${commands_string} -batchSize 0"
713 fi
714
715 # UNKEYED
716 if [[ "${skip_unkeyed}" == "" ]]; then
717
718 # RELIABLE
719 if [[ "${skip_rel_tests}" == "" ]]; then
720 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
721 fi
722
723 # BEST EFFORT
724 if [[ "${skip_be_tests}" == "" ]]; then
725 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
726 fi
727 fi
728
729 # KEYED
730 if [[ "${skip_keyed_data}" == "" ]]; then
731
732 # RELIABLE
733 if [[ "${skip_rel_tests}" == "" ]]; then
734 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
735 fi
736
737 # BEST EFFORT
738 if [[ "${skip_be_tests}" == "" ]]; then
739 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
740 fi
741 fi
742
743fi
744
745# Replace the cleanup section at the end:
746if [[ "${configure_network}" == "1" ]]; then
747 echo -e "${INFO_TAG} Resetting network interface to default state"
748 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
749 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
750fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
10
11# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
12# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
13# --extra-arguments "-securePSK str:1:SecretKey"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Not using security libraries
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set, we will add it to the command line of the publisher side.
89 if [[ "$batch_size" != "" && "$role" == "pub" ]]; then
90 commands_string_test="${commands_string_test} -batchSize $batch_size"
91 fi
92
93 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
94
95 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
96
97 if [[ "$role" == "pub" ]]; then
98 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
99 fi
100
101 if [[ "$thread_cpu_affinity" != "" ]]; then
102 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
103 fi
104
105 local pre_command_string=""
106
107 if [[ "$thread_priorities" != "" ]]; then
108 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
109 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
110 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
111 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
112 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
113 fi
114
115 if [[ "$pin_memory" != "" ]]; then
116 commands_string_test="${commands_string_test} -pinMemory"
117 fi
118
119 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
120 pre_command_string="$pre_command_string taskset -c $core"
121 fi
122
123 if [[ "$LANGUAGE" == "python" ]]; then
124 pre_command_string="$pre_command_string python3 "
125 fi
126
127 if [[ "$DOCKER" == "1" ]]; then
128 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
129 executable=""
130 fi
131
132 # Get the aprox time this will take:
133 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
134 total_time=$((total_tests * exec_time))
135
136 touch $output_file
137 local no_headers=""
138 local current_test=0
139 for index in $(seq 1 ${num_reps}); do
140 for DATALEN in ${datasizes_test}; do
141 current_test=$((current_test + 1))
142
143 if [[ ! -s $output_file ]]; then
144 echo -e "${INFO_TAG} Output file is empty, filling the header."
145 no_headers=""
146 else
147 echo -e "${INFO_TAG} Output file is not empty."
148 no_headers=" -noOutputHeaders"
149 fi
150
151 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
152 if [[ "$role" == "pub" ]]; then
153 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
154 echo -e ${BLUE}$command${NC}
155 else
156 echo -e ${LIGHTBLUE}$command${NC}
157 fi
158
159 # In certain cases we need to wait a bit before running the test, this is
160 # because the previous test might not be finished on the other side yet, or because the
161 # discovery mechanism does not work like in connext DDS.
162 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
163 sleep 3
164 fi
165 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
166 sleep 5
167 fi
168
169 # Gather netstat info before running the test
170 if [[ "${get_netstat_info}" == "1" ]]; then
171 echo -e "${INFO_TAG} Getting netstat info before"
172 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
173 fi
174
175 # Execute the command and capture the output
176 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
177 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
178 touch ${output_file}_tmp_line
179 chmod 777 ${output_file}_tmp_line
180 eval $command -outputFile ${output_file}_tmp_line
181 else
182 eval $command > ${output_file}_tmp_line
183 fi
184
185 number_of_lines=$(wc -l < ${output_file}_tmp_line)
186 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
187
188 # Check if this is the first test for this output file.
189 local is_first_test=0
190 if [[ ! -s $output_file ]]; then
191 is_first_test=1
192 fi
193 local expected_lines=$((is_first_test+1))
194
195 if [[ $number_of_lines -ne ${expected_lines} ]]; then
196 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
197 echo -e "${RED} The content is: \n\"\"\""
198 cat ${output_file}_tmp_line
199 echo -e "\"\"\""
200 echo -e "Not adding it to the output file. ${NC}"
201 else
202 if [[ $number_of_lines -gt 1 ]]; then
203 header_line=$(head -n 1 "${output_file}_tmp_line")
204 echo -e "${INFO_TAG} Header line is: $header_line"
205 fi
206 result_line=$(tail -n 1 "${output_file}_tmp_line")
207 echo -e "${INFO_TAG} Result line is: $result_line"
208
209 # If this is the first test, we need to add the header line to the output file
210 if [[ $is_first_test -eq 1 ]]; then
211 append_to_output_file "$header_line" ", command-line" $output_file
212 fi
213
214 command=$(echo $command | sed 's/,/:comma:/g')
215 # Remove the `"` characters from the command line
216 command=$(echo $command | sed 's/\"//g')
217
218 # Append the result line to the output file
219 append_to_output_file "$result_line" ", $command" $output_file
220 fi
221
222 # Always remove the temporary file
223 rm -rf ${output_file}_tmp_line
224
225 # Gather netstat info after running the test
226 if [[ "${get_netstat_info}" == "1" ]]; then
227 echo -e "${INFO_TAG} Getting netstat info after"
228 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
229 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
230 python3 $script_location/../../../tools/diff_netstat_output.py \
231 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
232 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
233 -d $DATALEN $no_header_netstat \
234 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
235 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
236 no_header_netstat=" -nh"
237 fi
238
239 change_domain
240 done
241 done
242}
243
244################################################################################
245# PARSE COMMAND LINE OPTIONS:
246
247while [ "$1" != "" ]; do
248 case $1 in
249 --executable)
250 executable=$2
251 shift
252 ;;
253 --docker)
254 DOCKER="1"
255 ;;
256 --output-folder)
257 output_folder=$2
258 shift
259 ;;
260 --role)
261 export role=$2
262 shift
263 ;;
264 --core)
265 export core=$2
266 shift
267 ;;
268 --assign-core-list | --thread-cpu-affinity)
269 export thread_cpu_affinity=$2
270 shift
271 ;;
272 --assign-thread-priorities | --thread-priorities)
273 export thread_priorities=$2
274 shift
275 ;;
276 --pin-memory)
277 export pin_memory="1"
278 ;;
279 --test-kind)
280 export lat_thr=$2
281 shift
282 ;;
283 --interface1)
284 export interface=$2
285 shift
286 ;;
287 --interface2)
288 export interface2=$2
289 shift
290 ;;
291 --ip1)
292 export ip1=$2
293 shift
294 ;;
295 --ip2)
296 export ip2=$2
297 shift
298 ;;
299 --repetitions)
300 export num_reps=$2
301 shift
302 ;;
303 --domain)
304 export domain=$2
305 shift
306 ;;
307 --execution-time)
308 export exec_time=$2
309 shift
310 ;;
311 --transport)
312 export transport=$2
313 shift
314 ;;
315 --datalen)
316 export datalen_input=$2
317 shift
318 ;;
319 --file-suffix)
320 export file_suffix=$2
321 shift
322 ;;
323 --folder-suffix)
324 export folder_suffix=$2
325 shift
326 ;;
327 --sub-folder)
328 sub_folder=$2
329 shift
330 ;;
331 --executable-suffix)
332 export executable_suffix=$2
333 shift
334 ;;
335 --extra-arguments)
336 export extra_arguments=$2
337 shift
338 ;;
339 --extra-arguments-pub)
340 export extra_arguments_pub=$2
341 shift
342 ;;
343 --extra-arguments-sub)
344 export extra_arguments_sub=$2
345 shift
346 ;;
347 --skip-be)
348 export skip_be_tests="1"
349 ;;
350 --skip-rel)
351 export skip_rel_tests="1"
352 ;;
353 --skip-keyed)
354 export skip_keyed_data="1"
355 ;;
356 --skip-large-data)
357 export skip_large_data="1"
358 ;;
359 --large-data)
360 export large_data="1"
361 ;;
362 --keyed)
363 export skip_unkeyed="1"
364 ;;
365 --unkeyed)
366 export skip_keyed_data="1"
367 ;;
368 --no-batching | --skip-batching)
369 export run_batching_tests="0"
370 export run_no_batching_tests="1"
371 ;;
372 --skip-no-batching)
373 export run_batching_tests="1"
374 export run_no_batching_tests="0"
375 ;;
376 --batch-size)
377 export batch_size=$2
378 shift
379 ;;
380 --reliable)
381 export skip_be_tests="1"
382 ;;
383 --best-effort)
384 export skip_rel_tests="1"
385 ;;
386 --security-gov)
387 export security_only="$2"
388 shift
389 ;;
390 --micro)
391 export micro="1"
392 ;;
393 --cert)
394 export cert="1"
395 ;;
396 --raw | --raw-transport)
397 export raw="1"
398 ;;
399 --tss)
400 export tss="1"
401 ;;
402 --no-colors)
403 export NO_COLORS="1"
404 ;;
405 --language)
406 export LANGUAGE=$2
407 shift
408 ;;
409 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
410 export nic_loss_rate=$2
411 export configure_network="1"
412 shift
413 ;;
414 --delay | --nic-delay-ms | --nic-delay)
415 export nic_delay_ms=$2
416 export configure_network="1"
417 shift
418 ;;
419 --get-netstat-info | --netstat)
420 export get_netstat_info="1"
421 ;;
422 --reduced-data-sizes-set)
423 export REDUCED_DATA_SIZES_SET="1"
424 ;;
425 --dont-change-tuned-settings)
426 export dont_change_tuned_settings="1"
427 ;;
428 --no-taskset)
429 export no_taskset=1
430 ;;
431 --asynchronous | --async)
432 export ASYNC="1"
433 ;;
434 *)
435 echo -e "unknown parameter \"$1\""
436 exit 255
437 ;;
438 esac
439 shift
440done
441
442if [[ "$NO_COLORS" == "1" ]]; then
443 disable_colors
444fi
445
446export folder_base="$(dirname "${executable}")"/../../..
447
448if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
449 export folder_base="$(dirname "${executable}")"/../..
450fi
451if [[ $tss == "1" ]]; then
452 export folder_base="$(dirname "${executable}")"/../../../../..
453fi
454
455if [[ "${executable_suffix}" != "" ]]; then
456 export executable="${executable}${executable_suffix}"
457fi
458
459if [[ -n "${folder_suffix}" ]]; then
460 export output_folder="${output_folder}${folder_suffix}"
461fi
462
463if [[ -n "${sub_folder}" ]]; then
464 export output_folder="${output_folder}/${sub_folder}"
465fi
466
467if [[ "${ASYNC}" == "1" ]]; then
468 export output_folder="${output_folder}/async"
469fi
470
471echo -e "${INFO_TAG} Perftest executable is: $executable"
472echo -e "${INFO_TAG} Output folder is: $output_folder"
473
474################################################################################
475
476if [[ "$LANGUAGE" == "python" ]]; then
477 export skip_keyed_data="1"
478 export skip_large_data="1"
479 export skip_be_tests="1"
480 export run_no_batching_tests="0"
481fi
482
483if [[ "${skip_large_data}" == "1" ]]; then
484 export datasizes_extended=${datasizes}
485elif [[ "${large_data}" == "1" ]]; then
486 export datasizes=${datasizes_extended}
487fi
488
489if [[ "${datalen_input}" != "" ]]; then
490 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
491 export datasizes=${datalen_input}
492 export datasizes_extended=${datalen_input}
493 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
494 export skip_large_data="1"
495 fi
496else
497 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
498 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
499 export datasizes="32 128 512 2048 8192 32768 63000"
500 export datasizes_extended="${datasizes} 102400 1048576 10485760"
501 fi
502fi
503
504if [[ "$role" != "pub" && "$role" != "sub" ]]; then
505 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
506 exit 255
507fi
508
509if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
510 echo -e "${ERROR_TAG} It must be either lat or thr"
511 exit 255
512fi
513
514if [[ "${interface}" == "" ]]; then
515 echo "Using default nics"
516 export nic_publisher=${ip_machine_1}
517 export nic_subscriber=${ip_machine_2}
518elif [[ "${interface}" == "both" ]]; then
519 export nic_publisher="enp1s0f0,eno1"
520 export nic_subscriber="enp1s0f0,eno1"
521 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
522 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
523else
524 export nic_publisher=$interface
525 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
526
527 if [[ "${interface2}" == "" ]]; then
528 export nic_subscriber=$interface
529 else
530 export nic_subscriber=$interface2
531 fi
532 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
533
534 if [[ "${ip1}" != "" ]]; then
535 export ip_publisher=$ip1
536 echo "Using ip_publisher: ${ip_publisher}"
537 fi
538
539 if [[ "${ip2}" != "" ]]; then
540 export ip_subscriber=$ip2
541 echo "Using ip_subscriber: ${ip_subscriber}"
542 fi
543
544fi
545
546if [[ "$transport" != "" ]]; then
547 export transport_string="-transport $transport"
548
549 if [[ "$transport" == "UDPv4" ]]; then
550
551 export transport_string_pub="$transport_string -nic $nic_publisher"
552 export transport_string_sub="$transport_string -nic $nic_subscriber"
553
554 if [[ "$raw" == "1" ]]; then
555 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
556 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
557 fi
558
559 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
560 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
561 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
562 fi
563
564 elif [[ "$transport" == "TCP" ]]; then
565 export transport_string_pub="$transport_string \
566 -nic $nic_publisher \
567 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
568 export transport_string_sub="$transport_string \
569 -nic $nic_subscriber \
570 -peer 0@tcpv4_lan://${ip_publisher}:7400"
571 elif [[ "$transport" == "TLS" ]]; then
572 export transport_string_pub="$transport_string \
573 -nic $nic_publisher \
574 -peer tlsv4_lan://${ip_subscriber}:7400"
575 export transport_string_sub="$transport_string \
576 -nic $nic_subscriber \
577 -peer tlsv4_lan://${ip_publisher}:7400"
578 elif [[ "$transport" == "UDPv4_WAN" ]]; then
579 export transport_string_pub="$transport_string \
580 -nic $nic_publisher \
581 -transportPublicAddress $ip_publisher:7400"
582 export transport_string_sub="$transport_string \
583 -nic $nic_subscriber \
584 -peer 0@udpv4_wan://${ip_publisher}:7400"
585 else
586 export transport_string_pub="$transport_string"
587 export transport_string_sub="$transport_string"
588 fi
589fi
590
591################################################################################
592
593export pub_string="-pub \
594 ${transport_string_pub} \
595 -noPrintIntervals \
596 -executionTime $exec_time"
597
598if [[ ${lat_thr} == "lat" ]]; then
599 export pub_string="$pub_string \
600 -latencyTest"
601fi
602
603if [[ "$role" == "pub" ]]; then
604
605 if [[ "$configure_network" == "1" ]]; then
606 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
607 export pub_string="$pub_string -initialBurst 1"
608 fi
609
610 if [[ "$ASYNC" == "1" ]]; then
611 export pub_string="${pub_string} -asynchronous"
612 fi
613fi
614
615export sub_string="-sub \
616 ${transport_string_sub} \
617 -noPrintIntervals"
618
619if [[ "$role" == "pub" ]]; then
620 echo -e "$INFO_TAG Publisher side running"
621 export commands_string=${pub_string}
622 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
623else
624 echo -e "$INFO_TAG Subscriber side running"
625 export commands_string=${sub_string}
626 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
627fi
628
629###############################################################################
630
631if [[ "$dont_change_tuned_settings" != "1" ]]; then
632 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
633 sudo /set_${lat_thr}_mode.sh
634 sleep 5
635fi
636
637if [[ "$role" == "pub" ]]; then
638 export interface_to_configure=$nic_publisher
639else
640 export interface_to_configure=$nic_subscriber
641fi
642
643echo -e "${INFO_TAG} Resetting network interface to default state"
644# Try to delete any existing netem rules (ignore errors if none exist)
645sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
646# Reset to default (pfifo_fast)
647sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
648
649if [[ "${configure_network}" == "1" ]]; then
650 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
651 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
652fi
653
654cd $folder_base
655echo -e "${INFO_TAG} Folder Base is: $PWD"
656mkdir -p $output_folder
657
658if [[ "${batch_size}" != "" ]]; then
659 if [[ "${lat_thr}" == "thr" ]]; then
660 if [[ "${batch_size}" -eq 0 ]]; then
661 echo -e "${INFO_TAG} Batch size is set to 0"
662 export run_batching_tests="0"
663 export run_no_batching_tests="1"
664 else
665 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
666 export run_batching_tests="1"
667 export run_no_batching_tests="0"
668 fi
669 else
670 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
671 unset batch_size
672 fi
673fi
674
675# Tests that may use batching (when doing throughput tests). Also Latency tests.
676if [[ ${run_batching_tests} == "1" ]]; then
677
678 # UNKEYED
679 if [[ "${skip_unkeyed}" == "" ]]; then
680
681 # RELIABLE
682 if [[ "${skip_rel_tests}" == "" ]]; then
683 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
684 fi
685
686 # BEST EFFORT
687 if [[ "${skip_be_tests}" == "" ]]; then
688 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
689 fi
690 fi
691
692 # KEYED
693 if [[ "${skip_keyed_data}" == "" ]]; then
694
695 # RELIABLE
696 if [[ "${skip_rel_tests}" == "" ]]; then
697 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
698 fi
699
700 # BEST EFFORT
701 if [[ "${skip_be_tests}" == "" ]]; then
702 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
703 fi
704 fi
705
706fi
707
708# Tests that will not use batching
709if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
710
711 if [[ "$role" == "pub" ]]; then
712 export commands_string="${commands_string} -batchSize 0"
713 fi
714
715 # UNKEYED
716 if [[ "${skip_unkeyed}" == "" ]]; then
717
718 # RELIABLE
719 if [[ "${skip_rel_tests}" == "" ]]; then
720 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
721 fi
722
723 # BEST EFFORT
724 if [[ "${skip_be_tests}" == "" ]]; then
725 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
726 fi
727 fi
728
729 # KEYED
730 if [[ "${skip_keyed_data}" == "" ]]; then
731
732 # RELIABLE
733 if [[ "${skip_rel_tests}" == "" ]]; then
734 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
735 fi
736
737 # BEST EFFORT
738 if [[ "${skip_be_tests}" == "" ]]; then
739 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
740 fi
741 fi
742
743fi
744
745# Replace the cleanup section at the end:
746if [[ "${configure_network}" == "1" ]]; then
747 echo -e "${INFO_TAG} Resetting network interface to default state"
748 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
749 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
750fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
10
11# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
12# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
13# --extra-arguments "-securePSK str:1:SecretKey"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set, we will add it to the command line of the publisher side.
89 if [[ "$batch_size" != "" && "$role" == "pub" ]]; then
90 commands_string_test="${commands_string_test} -batchSize $batch_size"
91 fi
92
93 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
94
95 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
96
97 if [[ "$role" == "pub" ]]; then
98 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
99 fi
100
101 if [[ "$thread_cpu_affinity" != "" ]]; then
102 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
103 fi
104
105 local pre_command_string=""
106
107 if [[ "$thread_priorities" != "" ]]; then
108 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
109 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
110 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
111 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
112 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
113 fi
114
115 if [[ "$pin_memory" != "" ]]; then
116 commands_string_test="${commands_string_test} -pinMemory"
117 fi
118
119 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
120 pre_command_string="$pre_command_string taskset -c $core"
121 fi
122
123 if [[ "$LANGUAGE" == "python" ]]; then
124 pre_command_string="$pre_command_string python3 "
125 fi
126
127 if [[ "$DOCKER" == "1" ]]; then
128 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
129 executable=""
130 fi
131
132 # Get the aprox time this will take:
133 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
134 total_time=$((total_tests * exec_time))
135
136 touch $output_file
137 local no_headers=""
138 local current_test=0
139 for index in $(seq 1 ${num_reps}); do
140 for DATALEN in ${datasizes_test}; do
141 current_test=$((current_test + 1))
142
143 if [[ ! -s $output_file ]]; then
144 echo -e "${INFO_TAG} Output file is empty, filling the header."
145 no_headers=""
146 else
147 echo -e "${INFO_TAG} Output file is not empty."
148 no_headers=" -noOutputHeaders"
149 fi
150
151 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
152 if [[ "$role" == "pub" ]]; then
153 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
154 echo -e ${BLUE}$command${NC}
155 else
156 echo -e ${LIGHTBLUE}$command${NC}
157 fi
158
159 # In certain cases we need to wait a bit before running the test, this is
160 # because the previous test might not be finished on the other side yet, or because the
161 # discovery mechanism does not work like in connext DDS.
162 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
163 sleep 3
164 fi
165 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
166 sleep 5
167 fi
168
169 # Gather netstat info before running the test
170 if [[ "${get_netstat_info}" == "1" ]]; then
171 echo -e "${INFO_TAG} Getting netstat info before"
172 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
173 fi
174
175 # Execute the command and capture the output
176 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
177 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
178 touch ${output_file}_tmp_line
179 chmod 777 ${output_file}_tmp_line
180 eval $command -outputFile ${output_file}_tmp_line
181 else
182 eval $command > ${output_file}_tmp_line
183 fi
184
185 number_of_lines=$(wc -l < ${output_file}_tmp_line)
186 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
187
188 # Check if this is the first test for this output file.
189 local is_first_test=0
190 if [[ ! -s $output_file ]]; then
191 is_first_test=1
192 fi
193 local expected_lines=$((is_first_test+1))
194
195 if [[ $number_of_lines -ne ${expected_lines} ]]; then
196 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
197 echo -e "${RED} The content is: \n\"\"\""
198 cat ${output_file}_tmp_line
199 echo -e "\"\"\""
200 echo -e "Not adding it to the output file. ${NC}"
201 else
202 if [[ $number_of_lines -gt 1 ]]; then
203 header_line=$(head -n 1 "${output_file}_tmp_line")
204 echo -e "${INFO_TAG} Header line is: $header_line"
205 fi
206 result_line=$(tail -n 1 "${output_file}_tmp_line")
207 echo -e "${INFO_TAG} Result line is: $result_line"
208
209 # If this is the first test, we need to add the header line to the output file
210 if [[ $is_first_test -eq 1 ]]; then
211 append_to_output_file "$header_line" ", command-line" $output_file
212 fi
213
214 command=$(echo $command | sed 's/,/:comma:/g')
215 # Remove the `"` characters from the command line
216 command=$(echo $command | sed 's/\"//g')
217
218 # Append the result line to the output file
219 append_to_output_file "$result_line" ", $command" $output_file
220 fi
221
222 # Always remove the temporary file
223 rm -rf ${output_file}_tmp_line
224
225 # Gather netstat info after running the test
226 if [[ "${get_netstat_info}" == "1" ]]; then
227 echo -e "${INFO_TAG} Getting netstat info after"
228 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
229 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
230 python3 $script_location/../../../tools/diff_netstat_output.py \
231 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
232 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
233 -d $DATALEN $no_header_netstat \
234 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
235 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
236 no_header_netstat=" -nh"
237 fi
238
239 change_domain
240 done
241 done
242}
243
244################################################################################
245# PARSE COMMAND LINE OPTIONS:
246
247while [ "$1" != "" ]; do
248 case $1 in
249 --executable)
250 executable=$2
251 shift
252 ;;
253 --docker)
254 DOCKER="1"
255 ;;
256 --output-folder)
257 output_folder=$2
258 shift
259 ;;
260 --role)
261 export role=$2
262 shift
263 ;;
264 --core)
265 export core=$2
266 shift
267 ;;
268 --assign-core-list | --thread-cpu-affinity)
269 export thread_cpu_affinity=$2
270 shift
271 ;;
272 --assign-thread-priorities | --thread-priorities)
273 export thread_priorities=$2
274 shift
275 ;;
276 --pin-memory)
277 export pin_memory="1"
278 ;;
279 --test-kind)
280 export lat_thr=$2
281 shift
282 ;;
283 --interface1)
284 export interface=$2
285 shift
286 ;;
287 --interface2)
288 export interface2=$2
289 shift
290 ;;
291 --ip1)
292 export ip1=$2
293 shift
294 ;;
295 --ip2)
296 export ip2=$2
297 shift
298 ;;
299 --repetitions)
300 export num_reps=$2
301 shift
302 ;;
303 --domain)
304 export domain=$2
305 shift
306 ;;
307 --execution-time)
308 export exec_time=$2
309 shift
310 ;;
311 --transport)
312 export transport=$2
313 shift
314 ;;
315 --datalen)
316 export datalen_input=$2
317 shift
318 ;;
319 --file-suffix)
320 export file_suffix=$2
321 shift
322 ;;
323 --folder-suffix)
324 export folder_suffix=$2
325 shift
326 ;;
327 --sub-folder)
328 sub_folder=$2
329 shift
330 ;;
331 --executable-suffix)
332 export executable_suffix=$2
333 shift
334 ;;
335 --extra-arguments)
336 export extra_arguments=$2
337 shift
338 ;;
339 --extra-arguments-pub)
340 export extra_arguments_pub=$2
341 shift
342 ;;
343 --extra-arguments-sub)
344 export extra_arguments_sub=$2
345 shift
346 ;;
347 --skip-be)
348 export skip_be_tests="1"
349 ;;
350 --skip-rel)
351 export skip_rel_tests="1"
352 ;;
353 --skip-keyed)
354 export skip_keyed_data="1"
355 ;;
356 --skip-large-data)
357 export skip_large_data="1"
358 ;;
359 --large-data)
360 export large_data="1"
361 ;;
362 --keyed)
363 export skip_unkeyed="1"
364 ;;
365 --unkeyed)
366 export skip_keyed_data="1"
367 ;;
368 --no-batching | --skip-batching)
369 export run_batching_tests="0"
370 export run_no_batching_tests="1"
371 ;;
372 --skip-no-batching)
373 export run_batching_tests="1"
374 export run_no_batching_tests="0"
375 ;;
376 --batch-size)
377 export batch_size=$2
378 shift
379 ;;
380 --reliable)
381 export skip_be_tests="1"
382 ;;
383 --best-effort)
384 export skip_rel_tests="1"
385 ;;
386 --security-gov)
387 export security_only="$2"
388 shift
389 ;;
390 --micro)
391 export micro="1"
392 ;;
393 --cert)
394 export cert="1"
395 ;;
396 --raw | --raw-transport)
397 export raw="1"
398 ;;
399 --tss)
400 export tss="1"
401 ;;
402 --no-colors)
403 export NO_COLORS="1"
404 ;;
405 --language)
406 export LANGUAGE=$2
407 shift
408 ;;
409 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
410 export nic_loss_rate=$2
411 export configure_network="1"
412 shift
413 ;;
414 --delay | --nic-delay-ms | --nic-delay)
415 export nic_delay_ms=$2
416 export configure_network="1"
417 shift
418 ;;
419 --get-netstat-info | --netstat)
420 export get_netstat_info="1"
421 ;;
422 --reduced-data-sizes-set)
423 export REDUCED_DATA_SIZES_SET="1"
424 ;;
425 --dont-change-tuned-settings)
426 export dont_change_tuned_settings="1"
427 ;;
428 --no-taskset)
429 export no_taskset=1
430 ;;
431 --asynchronous | --async)
432 export ASYNC="1"
433 ;;
434 *)
435 echo -e "unknown parameter \"$1\""
436 exit 255
437 ;;
438 esac
439 shift
440done
441
442if [[ "$NO_COLORS" == "1" ]]; then
443 disable_colors
444fi
445
446export folder_base="$(dirname "${executable}")"/../../..
447
448if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
449 export folder_base="$(dirname "${executable}")"/../..
450fi
451if [[ $tss == "1" ]]; then
452 export folder_base="$(dirname "${executable}")"/../../../../..
453fi
454
455if [[ "${executable_suffix}" != "" ]]; then
456 export executable="${executable}${executable_suffix}"
457fi
458
459if [[ -n "${folder_suffix}" ]]; then
460 export output_folder="${output_folder}${folder_suffix}"
461fi
462
463if [[ -n "${sub_folder}" ]]; then
464 export output_folder="${output_folder}/${sub_folder}"
465fi
466
467if [[ "${ASYNC}" == "1" ]]; then
468 export output_folder="${output_folder}/async"
469fi
470
471echo -e "${INFO_TAG} Perftest executable is: $executable"
472echo -e "${INFO_TAG} Output folder is: $output_folder"
473
474################################################################################
475
476if [[ "$LANGUAGE" == "python" ]]; then
477 export skip_keyed_data="1"
478 export skip_large_data="1"
479 export skip_be_tests="1"
480 export run_no_batching_tests="0"
481fi
482
483if [[ "${skip_large_data}" == "1" ]]; then
484 export datasizes_extended=${datasizes}
485elif [[ "${large_data}" == "1" ]]; then
486 export datasizes=${datasizes_extended}
487fi
488
489if [[ "${datalen_input}" != "" ]]; then
490 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
491 export datasizes=${datalen_input}
492 export datasizes_extended=${datalen_input}
493 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
494 export skip_large_data="1"
495 fi
496else
497 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
498 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
499 export datasizes="32 128 512 2048 8192 32768 63000"
500 export datasizes_extended="${datasizes} 102400 1048576 10485760"
501 fi
502fi
503
504if [[ "$role" != "pub" && "$role" != "sub" ]]; then
505 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
506 exit 255
507fi
508
509if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
510 echo -e "${ERROR_TAG} It must be either lat or thr"
511 exit 255
512fi
513
514if [[ "${interface}" == "" ]]; then
515 echo "Using default nics"
516 export nic_publisher=${ip_machine_1}
517 export nic_subscriber=${ip_machine_2}
518elif [[ "${interface}" == "both" ]]; then
519 export nic_publisher="enp1s0f0,eno1"
520 export nic_subscriber="enp1s0f0,eno1"
521 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
522 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
523else
524 export nic_publisher=$interface
525 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
526
527 if [[ "${interface2}" == "" ]]; then
528 export nic_subscriber=$interface
529 else
530 export nic_subscriber=$interface2
531 fi
532 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
533
534 if [[ "${ip1}" != "" ]]; then
535 export ip_publisher=$ip1
536 echo "Using ip_publisher: ${ip_publisher}"
537 fi
538
539 if [[ "${ip2}" != "" ]]; then
540 export ip_subscriber=$ip2
541 echo "Using ip_subscriber: ${ip_subscriber}"
542 fi
543
544fi
545
546if [[ "$transport" != "" ]]; then
547 export transport_string="-transport $transport"
548
549 if [[ "$transport" == "UDPv4" ]]; then
550
551 export transport_string_pub="$transport_string -nic $nic_publisher"
552 export transport_string_sub="$transport_string -nic $nic_subscriber"
553
554 if [[ "$raw" == "1" ]]; then
555 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
556 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
557 fi
558
559 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
560 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
561 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
562 fi
563
564 elif [[ "$transport" == "TCP" ]]; then
565 export transport_string_pub="$transport_string \
566 -nic $nic_publisher \
567 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
568 export transport_string_sub="$transport_string \
569 -nic $nic_subscriber \
570 -peer 0@tcpv4_lan://${ip_publisher}:7400"
571 elif [[ "$transport" == "TLS" ]]; then
572 export transport_string_pub="$transport_string \
573 -nic $nic_publisher \
574 -peer tlsv4_lan://${ip_subscriber}:7400"
575 export transport_string_sub="$transport_string \
576 -nic $nic_subscriber \
577 -peer tlsv4_lan://${ip_publisher}:7400"
578 elif [[ "$transport" == "UDPv4_WAN" ]]; then
579 export transport_string_pub="$transport_string \
580 -nic $nic_publisher \
581 -transportPublicAddress $ip_publisher:7400"
582 export transport_string_sub="$transport_string \
583 -nic $nic_subscriber \
584 -peer 0@udpv4_wan://${ip_publisher}:7400"
585 else
586 export transport_string_pub="$transport_string"
587 export transport_string_sub="$transport_string"
588 fi
589fi
590
591################################################################################
592
593export pub_string="-pub \
594 ${transport_string_pub} \
595 -noPrintIntervals \
596 -executionTime $exec_time"
597
598if [[ ${lat_thr} == "lat" ]]; then
599 export pub_string="$pub_string \
600 -latencyTest"
601fi
602
603if [[ "$role" == "pub" ]]; then
604
605 if [[ "$configure_network" == "1" ]]; then
606 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
607 export pub_string="$pub_string -initialBurst 1"
608 fi
609
610 if [[ "$ASYNC" == "1" ]]; then
611 export pub_string="${pub_string} -asynchronous"
612 fi
613fi
614
615export sub_string="-sub \
616 ${transport_string_sub} \
617 -noPrintIntervals"
618
619if [[ "$role" == "pub" ]]; then
620 echo -e "$INFO_TAG Publisher side running"
621 export commands_string=${pub_string}
622 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
623else
624 echo -e "$INFO_TAG Subscriber side running"
625 export commands_string=${sub_string}
626 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
627fi
628
629###############################################################################
630
631if [[ "$dont_change_tuned_settings" != "1" ]]; then
632 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
633 sudo /set_${lat_thr}_mode.sh
634 sleep 5
635fi
636
637if [[ "$role" == "pub" ]]; then
638 export interface_to_configure=$nic_publisher
639else
640 export interface_to_configure=$nic_subscriber
641fi
642
643echo -e "${INFO_TAG} Resetting network interface to default state"
644# Try to delete any existing netem rules (ignore errors if none exist)
645sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
646# Reset to default (pfifo_fast)
647sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
648
649if [[ "${configure_network}" == "1" ]]; then
650 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
651 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
652fi
653
654cd $folder_base
655echo -e "${INFO_TAG} Folder Base is: $PWD"
656mkdir -p $output_folder
657
658if [[ "${batch_size}" != "" ]]; then
659 if [[ "${lat_thr}" == "thr" ]]; then
660 if [[ "${batch_size}" -eq 0 ]]; then
661 echo -e "${INFO_TAG} Batch size is set to 0"
662 export run_batching_tests="0"
663 export run_no_batching_tests="1"
664 else
665 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
666 export run_batching_tests="1"
667 export run_no_batching_tests="0"
668 fi
669 else
670 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
671 unset batch_size
672 fi
673fi
674
675# Tests that may use batching (when doing throughput tests). Also Latency tests.
676if [[ ${run_batching_tests} == "1" ]]; then
677
678 # UNKEYED
679 if [[ "${skip_unkeyed}" == "" ]]; then
680
681 # RELIABLE
682 if [[ "${skip_rel_tests}" == "" ]]; then
683 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
684 fi
685
686 # BEST EFFORT
687 if [[ "${skip_be_tests}" == "" ]]; then
688 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
689 fi
690 fi
691
692 # KEYED
693 if [[ "${skip_keyed_data}" == "" ]]; then
694
695 # RELIABLE
696 if [[ "${skip_rel_tests}" == "" ]]; then
697 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
698 fi
699
700 # BEST EFFORT
701 if [[ "${skip_be_tests}" == "" ]]; then
702 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
703 fi
704 fi
705
706fi
707
708# Tests that will not use batching
709if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
710
711 if [[ "$role" == "pub" ]]; then
712 export commands_string="${commands_string} -batchSize 0"
713 fi
714
715 # UNKEYED
716 if [[ "${skip_unkeyed}" == "" ]]; then
717
718 # RELIABLE
719 if [[ "${skip_rel_tests}" == "" ]]; then
720 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
721 fi
722
723 # BEST EFFORT
724 if [[ "${skip_be_tests}" == "" ]]; then
725 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
726 fi
727 fi
728
729 # KEYED
730 if [[ "${skip_keyed_data}" == "" ]]; then
731
732 # RELIABLE
733 if [[ "${skip_rel_tests}" == "" ]]; then
734 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
735 fi
736
737 # BEST EFFORT
738 if [[ "${skip_be_tests}" == "" ]]; then
739 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
740 fi
741 fi
742
743fi
744
745# Replace the cleanup section at the end:
746if [[ "${configure_network}" == "1" ]]; then
747 echo -e "${INFO_TAG} Resetting network interface to default state"
748 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
749 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
750fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GCM" \
10 --executable-suffix "_lws" \
11 --file-suffix "_AES256_GCM"
12
13"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
14 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
15 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES128+GCM" \
16 --executable-suffix "_lws" \
17 --file-suffix "_AES128_GCM"
18
19# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
20# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
21# --extra-arguments "-securePSK str:1:SecretKey -securePSKAlgorithm AES256+GCM" \
22# --executable-suffix "_lws" \
23# --file-suffix "_AES256_GCM"
24
25"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
26 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
27 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GMAC" \
28 --executable-suffix "_lws" \
29 --file-suffix "_AES256_GMAC"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Not using security libraries
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set, we will add it to the command line of the publisher side.
89 if [[ "$batch_size" != "" && "$role" == "pub" ]]; then
90 commands_string_test="${commands_string_test} -batchSize $batch_size"
91 fi
92
93 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
94
95 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
96
97 if [[ "$role" == "pub" ]]; then
98 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
99 fi
100
101 if [[ "$thread_cpu_affinity" != "" ]]; then
102 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
103 fi
104
105 local pre_command_string=""
106
107 if [[ "$thread_priorities" != "" ]]; then
108 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
109 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
110 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
111 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
112 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
113 fi
114
115 if [[ "$pin_memory" != "" ]]; then
116 commands_string_test="${commands_string_test} -pinMemory"
117 fi
118
119 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
120 pre_command_string="$pre_command_string taskset -c $core"
121 fi
122
123 if [[ "$LANGUAGE" == "python" ]]; then
124 pre_command_string="$pre_command_string python3 "
125 fi
126
127 if [[ "$DOCKER" == "1" ]]; then
128 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
129 executable=""
130 fi
131
132 # Get the aprox time this will take:
133 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
134 total_time=$((total_tests * exec_time))
135
136 touch $output_file
137 local no_headers=""
138 local current_test=0
139 for index in $(seq 1 ${num_reps}); do
140 for DATALEN in ${datasizes_test}; do
141 current_test=$((current_test + 1))
142
143 if [[ ! -s $output_file ]]; then
144 echo -e "${INFO_TAG} Output file is empty, filling the header."
145 no_headers=""
146 else
147 echo -e "${INFO_TAG} Output file is not empty."
148 no_headers=" -noOutputHeaders"
149 fi
150
151 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
152 if [[ "$role" == "pub" ]]; then
153 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
154 echo -e ${BLUE}$command${NC}
155 else
156 echo -e ${LIGHTBLUE}$command${NC}
157 fi
158
159 # In certain cases we need to wait a bit before running the test, this is
160 # because the previous test might not be finished on the other side yet, or because the
161 # discovery mechanism does not work like in connext DDS.
162 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
163 sleep 3
164 fi
165 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
166 sleep 5
167 fi
168
169 # Gather netstat info before running the test
170 if [[ "${get_netstat_info}" == "1" ]]; then
171 echo -e "${INFO_TAG} Getting netstat info before"
172 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
173 fi
174
175 # Execute the command and capture the output
176 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
177 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
178 touch ${output_file}_tmp_line
179 chmod 777 ${output_file}_tmp_line
180 eval $command -outputFile ${output_file}_tmp_line
181 else
182 eval $command > ${output_file}_tmp_line
183 fi
184
185 number_of_lines=$(wc -l < ${output_file}_tmp_line)
186 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
187
188 # Check if this is the first test for this output file.
189 local is_first_test=0
190 if [[ ! -s $output_file ]]; then
191 is_first_test=1
192 fi
193 local expected_lines=$((is_first_test+1))
194
195 if [[ $number_of_lines -ne ${expected_lines} ]]; then
196 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
197 echo -e "${RED} The content is: \n\"\"\""
198 cat ${output_file}_tmp_line
199 echo -e "\"\"\""
200 echo -e "Not adding it to the output file. ${NC}"
201 else
202 if [[ $number_of_lines -gt 1 ]]; then
203 header_line=$(head -n 1 "${output_file}_tmp_line")
204 echo -e "${INFO_TAG} Header line is: $header_line"
205 fi
206 result_line=$(tail -n 1 "${output_file}_tmp_line")
207 echo -e "${INFO_TAG} Result line is: $result_line"
208
209 # If this is the first test, we need to add the header line to the output file
210 if [[ $is_first_test -eq 1 ]]; then
211 append_to_output_file "$header_line" ", command-line" $output_file
212 fi
213
214 command=$(echo $command | sed 's/,/:comma:/g')
215 # Remove the `"` characters from the command line
216 command=$(echo $command | sed 's/\"//g')
217
218 # Append the result line to the output file
219 append_to_output_file "$result_line" ", $command" $output_file
220 fi
221
222 # Always remove the temporary file
223 rm -rf ${output_file}_tmp_line
224
225 # Gather netstat info after running the test
226 if [[ "${get_netstat_info}" == "1" ]]; then
227 echo -e "${INFO_TAG} Getting netstat info after"
228 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
229 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
230 python3 $script_location/../../../tools/diff_netstat_output.py \
231 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
232 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
233 -d $DATALEN $no_header_netstat \
234 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
235 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
236 no_header_netstat=" -nh"
237 fi
238
239 change_domain
240 done
241 done
242}
243
244################################################################################
245# PARSE COMMAND LINE OPTIONS:
246
247while [ "$1" != "" ]; do
248 case $1 in
249 --executable)
250 executable=$2
251 shift
252 ;;
253 --docker)
254 DOCKER="1"
255 ;;
256 --output-folder)
257 output_folder=$2
258 shift
259 ;;
260 --role)
261 export role=$2
262 shift
263 ;;
264 --core)
265 export core=$2
266 shift
267 ;;
268 --assign-core-list | --thread-cpu-affinity)
269 export thread_cpu_affinity=$2
270 shift
271 ;;
272 --assign-thread-priorities | --thread-priorities)
273 export thread_priorities=$2
274 shift
275 ;;
276 --pin-memory)
277 export pin_memory="1"
278 ;;
279 --test-kind)
280 export lat_thr=$2
281 shift
282 ;;
283 --interface1)
284 export interface=$2
285 shift
286 ;;
287 --interface2)
288 export interface2=$2
289 shift
290 ;;
291 --ip1)
292 export ip1=$2
293 shift
294 ;;
295 --ip2)
296 export ip2=$2
297 shift
298 ;;
299 --repetitions)
300 export num_reps=$2
301 shift
302 ;;
303 --domain)
304 export domain=$2
305 shift
306 ;;
307 --execution-time)
308 export exec_time=$2
309 shift
310 ;;
311 --transport)
312 export transport=$2
313 shift
314 ;;
315 --datalen)
316 export datalen_input=$2
317 shift
318 ;;
319 --file-suffix)
320 export file_suffix=$2
321 shift
322 ;;
323 --folder-suffix)
324 export folder_suffix=$2
325 shift
326 ;;
327 --sub-folder)
328 sub_folder=$2
329 shift
330 ;;
331 --executable-suffix)
332 export executable_suffix=$2
333 shift
334 ;;
335 --extra-arguments)
336 export extra_arguments=$2
337 shift
338 ;;
339 --extra-arguments-pub)
340 export extra_arguments_pub=$2
341 shift
342 ;;
343 --extra-arguments-sub)
344 export extra_arguments_sub=$2
345 shift
346 ;;
347 --skip-be)
348 export skip_be_tests="1"
349 ;;
350 --skip-rel)
351 export skip_rel_tests="1"
352 ;;
353 --skip-keyed)
354 export skip_keyed_data="1"
355 ;;
356 --skip-large-data)
357 export skip_large_data="1"
358 ;;
359 --large-data)
360 export large_data="1"
361 ;;
362 --keyed)
363 export skip_unkeyed="1"
364 ;;
365 --unkeyed)
366 export skip_keyed_data="1"
367 ;;
368 --no-batching | --skip-batching)
369 export run_batching_tests="0"
370 export run_no_batching_tests="1"
371 ;;
372 --skip-no-batching)
373 export run_batching_tests="1"
374 export run_no_batching_tests="0"
375 ;;
376 --batch-size)
377 export batch_size=$2
378 shift
379 ;;
380 --reliable)
381 export skip_be_tests="1"
382 ;;
383 --best-effort)
384 export skip_rel_tests="1"
385 ;;
386 --security-gov)
387 export security_only="$2"
388 shift
389 ;;
390 --micro)
391 export micro="1"
392 ;;
393 --cert)
394 export cert="1"
395 ;;
396 --raw | --raw-transport)
397 export raw="1"
398 ;;
399 --tss)
400 export tss="1"
401 ;;
402 --no-colors)
403 export NO_COLORS="1"
404 ;;
405 --language)
406 export LANGUAGE=$2
407 shift
408 ;;
409 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
410 export nic_loss_rate=$2
411 export configure_network="1"
412 shift
413 ;;
414 --delay | --nic-delay-ms | --nic-delay)
415 export nic_delay_ms=$2
416 export configure_network="1"
417 shift
418 ;;
419 --get-netstat-info | --netstat)
420 export get_netstat_info="1"
421 ;;
422 --reduced-data-sizes-set)
423 export REDUCED_DATA_SIZES_SET="1"
424 ;;
425 --dont-change-tuned-settings)
426 export dont_change_tuned_settings="1"
427 ;;
428 --no-taskset)
429 export no_taskset=1
430 ;;
431 --asynchronous | --async)
432 export ASYNC="1"
433 ;;
434 *)
435 echo -e "unknown parameter \"$1\""
436 exit 255
437 ;;
438 esac
439 shift
440done
441
442if [[ "$NO_COLORS" == "1" ]]; then
443 disable_colors
444fi
445
446export folder_base="$(dirname "${executable}")"/../../..
447
448if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
449 export folder_base="$(dirname "${executable}")"/../..
450fi
451if [[ $tss == "1" ]]; then
452 export folder_base="$(dirname "${executable}")"/../../../../..
453fi
454
455if [[ "${executable_suffix}" != "" ]]; then
456 export executable="${executable}${executable_suffix}"
457fi
458
459if [[ -n "${folder_suffix}" ]]; then
460 export output_folder="${output_folder}${folder_suffix}"
461fi
462
463if [[ -n "${sub_folder}" ]]; then
464 export output_folder="${output_folder}/${sub_folder}"
465fi
466
467if [[ "${ASYNC}" == "1" ]]; then
468 export output_folder="${output_folder}/async"
469fi
470
471echo -e "${INFO_TAG} Perftest executable is: $executable"
472echo -e "${INFO_TAG} Output folder is: $output_folder"
473
474################################################################################
475
476if [[ "$LANGUAGE" == "python" ]]; then
477 export skip_keyed_data="1"
478 export skip_large_data="1"
479 export skip_be_tests="1"
480 export run_no_batching_tests="0"
481fi
482
483if [[ "${skip_large_data}" == "1" ]]; then
484 export datasizes_extended=${datasizes}
485elif [[ "${large_data}" == "1" ]]; then
486 export datasizes=${datasizes_extended}
487fi
488
489if [[ "${datalen_input}" != "" ]]; then
490 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
491 export datasizes=${datalen_input}
492 export datasizes_extended=${datalen_input}
493 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
494 export skip_large_data="1"
495 fi
496else
497 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
498 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
499 export datasizes="32 128 512 2048 8192 32768 63000"
500 export datasizes_extended="${datasizes} 102400 1048576 10485760"
501 fi
502fi
503
504if [[ "$role" != "pub" && "$role" != "sub" ]]; then
505 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
506 exit 255
507fi
508
509if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
510 echo -e "${ERROR_TAG} It must be either lat or thr"
511 exit 255
512fi
513
514if [[ "${interface}" == "" ]]; then
515 echo "Using default nics"
516 export nic_publisher=${ip_machine_1}
517 export nic_subscriber=${ip_machine_2}
518elif [[ "${interface}" == "both" ]]; then
519 export nic_publisher="enp1s0f0,eno1"
520 export nic_subscriber="enp1s0f0,eno1"
521 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
522 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
523else
524 export nic_publisher=$interface
525 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
526
527 if [[ "${interface2}" == "" ]]; then
528 export nic_subscriber=$interface
529 else
530 export nic_subscriber=$interface2
531 fi
532 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
533
534 if [[ "${ip1}" != "" ]]; then
535 export ip_publisher=$ip1
536 echo "Using ip_publisher: ${ip_publisher}"
537 fi
538
539 if [[ "${ip2}" != "" ]]; then
540 export ip_subscriber=$ip2
541 echo "Using ip_subscriber: ${ip_subscriber}"
542 fi
543
544fi
545
546if [[ "$transport" != "" ]]; then
547 export transport_string="-transport $transport"
548
549 if [[ "$transport" == "UDPv4" ]]; then
550
551 export transport_string_pub="$transport_string -nic $nic_publisher"
552 export transport_string_sub="$transport_string -nic $nic_subscriber"
553
554 if [[ "$raw" == "1" ]]; then
555 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
556 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
557 fi
558
559 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
560 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
561 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
562 fi
563
564 elif [[ "$transport" == "TCP" ]]; then
565 export transport_string_pub="$transport_string \
566 -nic $nic_publisher \
567 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
568 export transport_string_sub="$transport_string \
569 -nic $nic_subscriber \
570 -peer 0@tcpv4_lan://${ip_publisher}:7400"
571 elif [[ "$transport" == "TLS" ]]; then
572 export transport_string_pub="$transport_string \
573 -nic $nic_publisher \
574 -peer tlsv4_lan://${ip_subscriber}:7400"
575 export transport_string_sub="$transport_string \
576 -nic $nic_subscriber \
577 -peer tlsv4_lan://${ip_publisher}:7400"
578 elif [[ "$transport" == "UDPv4_WAN" ]]; then
579 export transport_string_pub="$transport_string \
580 -nic $nic_publisher \
581 -transportPublicAddress $ip_publisher:7400"
582 export transport_string_sub="$transport_string \
583 -nic $nic_subscriber \
584 -peer 0@udpv4_wan://${ip_publisher}:7400"
585 else
586 export transport_string_pub="$transport_string"
587 export transport_string_sub="$transport_string"
588 fi
589fi
590
591################################################################################
592
593export pub_string="-pub \
594 ${transport_string_pub} \
595 -noPrintIntervals \
596 -executionTime $exec_time"
597
598if [[ ${lat_thr} == "lat" ]]; then
599 export pub_string="$pub_string \
600 -latencyTest"
601fi
602
603if [[ "$role" == "pub" ]]; then
604
605 if [[ "$configure_network" == "1" ]]; then
606 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
607 export pub_string="$pub_string -initialBurst 1"
608 fi
609
610 if [[ "$ASYNC" == "1" ]]; then
611 export pub_string="${pub_string} -asynchronous"
612 fi
613fi
614
615export sub_string="-sub \
616 ${transport_string_sub} \
617 -noPrintIntervals"
618
619if [[ "$role" == "pub" ]]; then
620 echo -e "$INFO_TAG Publisher side running"
621 export commands_string=${pub_string}
622 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
623else
624 echo -e "$INFO_TAG Subscriber side running"
625 export commands_string=${sub_string}
626 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
627fi
628
629###############################################################################
630
631if [[ "$dont_change_tuned_settings" != "1" ]]; then
632 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
633 sudo /set_${lat_thr}_mode.sh
634 sleep 5
635fi
636
637if [[ "$role" == "pub" ]]; then
638 export interface_to_configure=$nic_publisher
639else
640 export interface_to_configure=$nic_subscriber
641fi
642
643echo -e "${INFO_TAG} Resetting network interface to default state"
644# Try to delete any existing netem rules (ignore errors if none exist)
645sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
646# Reset to default (pfifo_fast)
647sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
648
649if [[ "${configure_network}" == "1" ]]; then
650 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
651 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
652fi
653
654cd $folder_base
655echo -e "${INFO_TAG} Folder Base is: $PWD"
656mkdir -p $output_folder
657
658if [[ "${batch_size}" != "" ]]; then
659 if [[ "${lat_thr}" == "thr" ]]; then
660 if [[ "${batch_size}" -eq 0 ]]; then
661 echo -e "${INFO_TAG} Batch size is set to 0"
662 export run_batching_tests="0"
663 export run_no_batching_tests="1"
664 else
665 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
666 export run_batching_tests="1"
667 export run_no_batching_tests="0"
668 fi
669 else
670 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
671 unset batch_size
672 fi
673fi
674
675# Tests that may use batching (when doing throughput tests). Also Latency tests.
676if [[ ${run_batching_tests} == "1" ]]; then
677
678 # UNKEYED
679 if [[ "${skip_unkeyed}" == "" ]]; then
680
681 # RELIABLE
682 if [[ "${skip_rel_tests}" == "" ]]; then
683 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
684 fi
685
686 # BEST EFFORT
687 if [[ "${skip_be_tests}" == "" ]]; then
688 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
689 fi
690 fi
691
692 # KEYED
693 if [[ "${skip_keyed_data}" == "" ]]; then
694
695 # RELIABLE
696 if [[ "${skip_rel_tests}" == "" ]]; then
697 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
698 fi
699
700 # BEST EFFORT
701 if [[ "${skip_be_tests}" == "" ]]; then
702 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
703 fi
704 fi
705
706fi
707
708# Tests that will not use batching
709if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
710
711 if [[ "$role" == "pub" ]]; then
712 export commands_string="${commands_string} -batchSize 0"
713 fi
714
715 # UNKEYED
716 if [[ "${skip_unkeyed}" == "" ]]; then
717
718 # RELIABLE
719 if [[ "${skip_rel_tests}" == "" ]]; then
720 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
721 fi
722
723 # BEST EFFORT
724 if [[ "${skip_be_tests}" == "" ]]; then
725 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
726 fi
727 fi
728
729 # KEYED
730 if [[ "${skip_keyed_data}" == "" ]]; then
731
732 # RELIABLE
733 if [[ "${skip_rel_tests}" == "" ]]; then
734 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
735 fi
736
737 # BEST EFFORT
738 if [[ "${skip_be_tests}" == "" ]]; then
739 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
740 fi
741 fi
742
743fi
744
745# Replace the cleanup section at the end:
746if [[ "${configure_network}" == "1" ]]; then
747 echo -e "${INFO_TAG} Resetting network interface to default state"
748 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
749 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
750fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GCM" \
10 --executable-suffix "_lws" \
11 --file-suffix "_AES256_GCM"
12
13"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
14 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
15 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES128+GCM" \
16 --executable-suffix "_lws" \
17 --file-suffix "_AES128_GCM"
18
19# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
20# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
21# --extra-arguments "-securePSK str:1:SecretKey -securePSKAlgorithm AES256+GCM" \
22# --executable-suffix "_lws" \
23# --file-suffix "_AES256_GCM"
24
25"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
26 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
27 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GMAC" \
28 --executable-suffix "_lws" \
29 --file-suffix "_AES256_GMAC"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)