3.2.1.1. LightWeight Security PSK vs HMAC Only vs Full Security PSK
In RTI Connext 7.1 we introduced Pre-Shared Key Protection (PSK)` to secure the RTPS communication.
This PSK can be leveraged in two ways: As a part of RTI Security Plugins, protecting bootstrapping
communications before authentication is successfully concluded or as a dedicated library named RTI
Lightweight Security Plugins, where PSK` is the only option and protects the entirety of the communication.
PSK can be configured to use various cryptographic algorithms: AES128 or AES256 in either GMAC
(for integrity) or GCM (for both integrity and confidentiality) modes.
In RTI Connext 7.1 we deprecated HMAC-only mode which is scheduled to be superseded in the next release with Lightweight Security.
It supports non-configurable HMAC-SHA256 which only protects data integrity without its confidentiality.
Charts below compare performance of different Lightweight Security’s AES256 GCM and GMAC algorithms
with HMAC-only and a non-secure scenario. In all tests PSK performs better than HMAC-only.
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set to a non-zero value, we will add it to the command line of the publisher side.
89 # When batch_size is 0, it is already handled by the no-batching block which appends -batchSize 0 to commands_string.
90 if [[ "$batch_size" != "" && "$batch_size" != "0" && "$role" == "pub" ]]; then
91 commands_string_test="${commands_string_test} -batchSize $batch_size"
92 fi
93
94 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
95
96 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
97
98 if [[ "$role" == "pub" ]]; then
99 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
100 fi
101
102 if [[ "$thread_cpu_affinity" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
103 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
104 fi
105
106 local pre_command_string=""
107
108 if [[ "$thread_priorities" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
109 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
110 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
111 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
112 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
113 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
114 fi
115
116 if [[ "$pin_memory" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
117 commands_string_test="${commands_string_test} -pinMemory"
118 fi
119
120 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
121 pre_command_string="$pre_command_string taskset -c $core"
122 fi
123
124 if [[ "$LANGUAGE" == "python" ]]; then
125 pre_command_string="$pre_command_string python3 "
126 fi
127
128 if [[ "$DOCKER" == "1" ]]; then
129 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
130 executable=""
131 fi
132
133 # Get the aprox time this will take:
134 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
135 total_time=$((total_tests * exec_time))
136
137 touch $output_file
138 local no_headers=""
139 local current_test=0
140 for index in $(seq 1 ${num_reps}); do
141 for DATALEN in ${datasizes_test}; do
142 current_test=$((current_test + 1))
143
144 if [[ ! -s $output_file ]]; then
145 echo -e "${INFO_TAG} Output file is empty, filling the header."
146 no_headers=""
147 else
148 echo -e "${INFO_TAG} Output file is not empty."
149 no_headers=" -noOutputHeaders"
150 fi
151
152 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
153 if [[ "$role" == "pub" ]]; then
154 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
155 echo -e ${BLUE}$command${NC}
156 else
157 echo -e ${LIGHTBLUE}$command${NC}
158 fi
159
160 # In certain cases we need to wait a bit before running the test, this is
161 # because the previous test might not be finished on the other side yet, or because the
162 # discovery mechanism does not work like in connext DDS.
163 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
164 sleep 3
165 fi
166 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
167 sleep 5
168 fi
169
170 # Gather netstat info before running the test
171 if [[ "${get_netstat_info}" == "1" ]]; then
172 echo -e "${INFO_TAG} Getting netstat info before"
173 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
174 fi
175
176 # Execute the command and capture the output
177 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
178 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
179 touch ${output_file}_tmp_line
180 chmod 777 ${output_file}_tmp_line
181 eval $command -outputFile ${output_file}_tmp_line
182 else
183 eval $command > ${output_file}_tmp_line
184 fi
185
186 number_of_lines=$(wc -l < ${output_file}_tmp_line)
187 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
188
189 # Check if this is the first test for this output file.
190 local is_first_test=0
191 if [[ ! -s $output_file ]]; then
192 is_first_test=1
193 fi
194 local expected_lines=$((is_first_test+1))
195
196 if [[ $number_of_lines -ne ${expected_lines} ]]; then
197 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
198 echo -e "${RED} The content is: \n\"\"\""
199 cat ${output_file}_tmp_line
200 echo -e "\"\"\""
201 echo -e "Not adding it to the output file. ${NC}"
202 else
203 if [[ $number_of_lines -gt 1 ]]; then
204 header_line=$(head -n 1 "${output_file}_tmp_line")
205 echo -e "${INFO_TAG} Header line is: $header_line"
206 fi
207 result_line=$(tail -n 1 "${output_file}_tmp_line")
208 echo -e "${INFO_TAG} Result line is: $result_line"
209
210 # If this is the first test, we need to add the header line to the output file
211 if [[ $is_first_test -eq 1 ]]; then
212 append_to_output_file "$header_line" ", command-line" $output_file
213 fi
214
215 command=$(echo $command | sed 's/,/:comma:/g')
216 # Remove the `"` characters from the command line
217 command=$(echo $command | sed 's/\"//g')
218
219 # Append the result line to the output file
220 append_to_output_file "$result_line" ", $command" $output_file
221 fi
222
223 # Always remove the temporary file
224 rm -rf ${output_file}_tmp_line
225
226 # Gather netstat info after running the test
227 if [[ "${get_netstat_info}" == "1" ]]; then
228 echo -e "${INFO_TAG} Getting netstat info after"
229 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
230 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
231 python3 $script_location/../../../tools/diff_netstat_output.py \
232 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
233 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
234 -d $DATALEN $no_header_netstat \
235 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
236 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
237 no_header_netstat=" -nh"
238 fi
239
240 change_domain
241 done
242 done
243}
244
245################################################################################
246# PARSE COMMAND LINE OPTIONS:
247
248while [ "$1" != "" ]; do
249 case $1 in
250 --executable)
251 executable=$2
252 shift
253 ;;
254 --docker)
255 DOCKER="1"
256 ;;
257 --output-folder)
258 output_folder=$2
259 shift
260 ;;
261 --role)
262 export role=$2
263 shift
264 ;;
265 --core)
266 export core=$2
267 shift
268 ;;
269 --assign-core-list | --thread-cpu-affinity)
270 export thread_cpu_affinity=$2
271 shift
272 ;;
273 --assign-thread-priorities | --thread-priorities)
274 export thread_priorities=$2
275 shift
276 ;;
277 --pin-memory)
278 export pin_memory="1"
279 ;;
280 --test-kind)
281 export lat_thr=$2
282 shift
283 ;;
284 --interface1)
285 export interface=$2
286 shift
287 ;;
288 --interface2)
289 export interface2=$2
290 shift
291 ;;
292 --ip1)
293 export ip1=$2
294 shift
295 ;;
296 --ip2)
297 export ip2=$2
298 shift
299 ;;
300 --repetitions)
301 export num_reps=$2
302 shift
303 ;;
304 --domain)
305 export domain=$2
306 shift
307 ;;
308 --execution-time)
309 export exec_time=$2
310 shift
311 ;;
312 --transport)
313 export transport=$2
314 shift
315 ;;
316 --datalen)
317 export datalen_input=$2
318 shift
319 ;;
320 --file-suffix)
321 export file_suffix=$2
322 shift
323 ;;
324 --folder-suffix)
325 export folder_suffix=$2
326 shift
327 ;;
328 --sub-folder)
329 sub_folder=$2
330 shift
331 ;;
332 --executable-suffix)
333 export executable_suffix=$2
334 shift
335 ;;
336 --extra-arguments)
337 export extra_arguments=$2
338 shift
339 ;;
340 --extra-arguments-pub)
341 export extra_arguments_pub=$2
342 shift
343 ;;
344 --extra-arguments-sub)
345 export extra_arguments_sub=$2
346 shift
347 ;;
348 --skip-be)
349 export skip_be_tests="1"
350 ;;
351 --skip-rel)
352 export skip_rel_tests="1"
353 ;;
354 --skip-keyed)
355 export skip_keyed_data="1"
356 ;;
357 --skip-large-data)
358 export skip_large_data="1"
359 ;;
360 --large-data)
361 export large_data="1"
362 ;;
363 --keyed)
364 export skip_unkeyed="1"
365 ;;
366 --unkeyed)
367 export skip_keyed_data="1"
368 ;;
369 --no-batching | --skip-batching)
370 export run_batching_tests="0"
371 export run_no_batching_tests="1"
372 ;;
373 --skip-no-batching)
374 export run_batching_tests="1"
375 export run_no_batching_tests="0"
376 ;;
377 --batch-size)
378 export batch_size=$2
379 shift
380 ;;
381 --reliable)
382 export skip_be_tests="1"
383 ;;
384 --best-effort)
385 export skip_rel_tests="1"
386 ;;
387 --security-gov)
388 export security_only="$2"
389 shift
390 ;;
391 --micro)
392 export micro="1"
393 ;;
394 --cert)
395 export cert="1"
396 ;;
397 --raw | --raw-transport)
398 export raw="1"
399 ;;
400 --tss)
401 export tss="1"
402 ;;
403 --no-colors)
404 export NO_COLORS="1"
405 ;;
406 --language)
407 export LANGUAGE=$2
408 shift
409 ;;
410 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
411 export nic_loss_rate=$2
412 export configure_network="1"
413 shift
414 ;;
415 --delay | --nic-delay-ms | --nic-delay)
416 export nic_delay_ms=$2
417 export configure_network="1"
418 shift
419 ;;
420 --get-netstat-info | --netstat)
421 export get_netstat_info="1"
422 ;;
423 --reduced-data-sizes-set)
424 export REDUCED_DATA_SIZES_SET="1"
425 ;;
426 --dont-change-tuned-settings)
427 export dont_change_tuned_settings="1"
428 ;;
429 --no-taskset)
430 export no_taskset=1
431 ;;
432 --asynchronous | --async)
433 export ASYNC="1"
434 ;;
435 *)
436 echo -e "unknown parameter \"$1\""
437 exit 255
438 ;;
439 esac
440 shift
441done
442
443if [[ "$NO_COLORS" == "1" ]]; then
444 disable_colors
445fi
446
447export folder_base="$(dirname "${executable}")"/../../..
448
449if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
450 export folder_base="$(dirname "${executable}")"/../..
451fi
452if [[ $tss == "1" ]]; then
453 export folder_base="$(dirname "${executable}")"/../../../../..
454fi
455
456if [[ "${executable_suffix}" != "" ]]; then
457 export executable="${executable}${executable_suffix}"
458fi
459
460if [[ -n "${folder_suffix}" ]]; then
461 export output_folder="${output_folder}${folder_suffix}"
462fi
463
464if [[ -n "${sub_folder}" ]]; then
465 export output_folder="${output_folder}/${sub_folder}"
466fi
467
468if [[ "${ASYNC}" == "1" ]]; then
469 export output_folder="${output_folder}/async"
470fi
471
472echo -e "${INFO_TAG} Perftest executable is: $executable"
473echo -e "${INFO_TAG} Output folder is: $output_folder"
474
475################################################################################
476
477if [[ "$LANGUAGE" == "python" ]]; then
478 export skip_keyed_data="1"
479 export skip_large_data="1"
480 export skip_be_tests="1"
481 export run_no_batching_tests="0"
482fi
483
484if [[ "$LANGUAGE" == "java" || "$LANGUAGE" == "cs" ]]; then
485 # Java and C# perftest do not support these execution options.
486 unset thread_cpu_affinity
487 unset thread_priorities
488 unset pin_memory
489 export no_taskset=1
490fi
491
492if [[ "${skip_large_data}" == "1" ]]; then
493 export datasizes_extended=${datasizes}
494elif [[ "${large_data}" == "1" ]]; then
495 export datasizes=${datasizes_extended}
496fi
497
498if [[ "${datalen_input}" != "" ]]; then
499 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
500 export datasizes=${datalen_input}
501 export datasizes_extended=${datalen_input}
502 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
503 export skip_large_data="1"
504 fi
505else
506 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
507 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
508 export datasizes="32 128 512 2048 8192 32768 63000"
509 export datasizes_extended="${datasizes} 102400 1048576 10485760"
510 fi
511fi
512
513if [[ "$role" != "pub" && "$role" != "sub" ]]; then
514 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
515 exit 255
516fi
517
518if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
519 echo -e "${ERROR_TAG} It must be either lat or thr"
520 exit 255
521fi
522
523if [[ "${interface}" == "" ]]; then
524 echo "Using default nics"
525 export nic_publisher=${ip_machine_1}
526 export nic_subscriber=${ip_machine_2}
527elif [[ "${interface}" == "both" ]]; then
528 export nic_publisher="enp1s0f0,eno1"
529 export nic_subscriber="enp1s0f0,eno1"
530 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
531 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
532else
533 export nic_publisher=$interface
534 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
535
536 if [[ "${interface2}" == "" ]]; then
537 export nic_subscriber=$interface
538 else
539 export nic_subscriber=$interface2
540 fi
541 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
542
543 if [[ "${ip1}" != "" ]]; then
544 export ip_publisher=$ip1
545 echo "Using ip_publisher: ${ip_publisher}"
546 fi
547
548 if [[ "${ip2}" != "" ]]; then
549 export ip_subscriber=$ip2
550 echo "Using ip_subscriber: ${ip_subscriber}"
551 fi
552
553fi
554
555if [[ "$transport" != "" ]]; then
556 export transport_string="-transport $transport"
557
558 if [[ "$transport" == "UDPv4" ]]; then
559
560 export transport_string_pub="$transport_string -nic $nic_publisher"
561 export transport_string_sub="$transport_string -nic $nic_subscriber"
562
563 if [[ "$raw" == "1" ]]; then
564 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
565 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
566 fi
567
568 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
569 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
570 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
571 fi
572
573 elif [[ "$transport" == "TCP" ]]; then
574 export transport_string_pub="$transport_string \
575 -nic $nic_publisher \
576 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
577 export transport_string_sub="$transport_string \
578 -nic $nic_subscriber \
579 -peer 0@tcpv4_lan://${ip_publisher}:7400"
580 elif [[ "$transport" == "TLS" ]]; then
581 export transport_string_pub="$transport_string \
582 -nic $nic_publisher \
583 -peer tlsv4_lan://${ip_subscriber}:7400"
584 export transport_string_sub="$transport_string \
585 -nic $nic_subscriber \
586 -peer tlsv4_lan://${ip_publisher}:7400"
587 elif [[ "$transport" == "UDPv4_WAN" ]]; then
588 export transport_string_pub="$transport_string \
589 -nic $nic_publisher \
590 -transportPublicAddress $ip_publisher:7400"
591 export transport_string_sub="$transport_string \
592 -nic $nic_subscriber \
593 -peer 0@udpv4_wan://${ip_publisher}:7400"
594 else
595 export transport_string_pub="$transport_string"
596 export transport_string_sub="$transport_string"
597 fi
598fi
599
600################################################################################
601
602export pub_string="-pub \
603 ${transport_string_pub} \
604 -noPrintIntervals \
605 -executionTime $exec_time"
606
607if [[ ${lat_thr} == "lat" ]]; then
608 export pub_string="$pub_string \
609 -latencyTest"
610fi
611
612if [[ "$role" == "pub" ]]; then
613
614 if [[ "$configure_network" == "1" ]]; then
615 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
616 export pub_string="$pub_string -initialBurst 1"
617 fi
618
619 if [[ "$ASYNC" == "1" ]]; then
620 export pub_string="${pub_string} -asynchronous"
621 fi
622fi
623
624export sub_string="-sub \
625 ${transport_string_sub} \
626 -noPrintIntervals"
627
628if [[ "$role" == "pub" ]]; then
629 echo -e "$INFO_TAG Publisher side running"
630 export commands_string=${pub_string}
631 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
632else
633 echo -e "$INFO_TAG Subscriber side running"
634 export commands_string=${sub_string}
635 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
636fi
637
638###############################################################################
639
640if [[ "$dont_change_tuned_settings" != "1" ]]; then
641 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
642 sudo /set_${lat_thr}_mode.sh
643 sleep 5
644fi
645
646if [[ "$role" == "pub" ]]; then
647 export interface_to_configure=$nic_publisher
648else
649 export interface_to_configure=$nic_subscriber
650fi
651
652echo -e "${INFO_TAG} Resetting network interface to default state"
653# Try to delete any existing netem rules (ignore errors if none exist)
654sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
655# Reset to default (pfifo_fast)
656sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
657
658if [[ "${configure_network}" == "1" ]]; then
659 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
660 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
661fi
662
663cd $folder_base
664echo -e "${INFO_TAG} Folder Base is: $PWD"
665mkdir -p $output_folder
666
667if [[ "${batch_size}" != "" ]]; then
668 if [[ "${lat_thr}" == "thr" ]]; then
669 if [[ "${batch_size}" -eq 0 ]]; then
670 echo -e "${INFO_TAG} Batch size is set to 0"
671 export run_batching_tests="0"
672 export run_no_batching_tests="1"
673 else
674 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
675 export run_batching_tests="1"
676 export run_no_batching_tests="0"
677 fi
678 else
679 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
680 unset batch_size
681 fi
682fi
683
684# Tests that may use batching (when doing throughput tests). Also Latency tests.
685if [[ ${run_batching_tests} == "1" ]]; then
686
687 # UNKEYED
688 if [[ "${skip_unkeyed}" == "" ]]; then
689
690 # RELIABLE
691 if [[ "${skip_rel_tests}" == "" ]]; then
692 # For the time being, we will skip the throughput reliable tests for CERT.
693 # More info in PERF-1175
694 if ! [[ "${cert}" == "1" && "${lat_thr}" == "thr" && "${transport}" == "UDPv4" ]]; then
695 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
696 else
697 echo -e "${INFO_TAG} Skipping reliable throughput tests for CERT. See PERF-1175 for more information."
698 fi
699 fi
700
701 # BEST EFFORT
702 if [[ "${skip_be_tests}" == "" ]]; then
703 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
704 fi
705 fi
706
707 # KEYED
708 if [[ "${skip_keyed_data}" == "" ]]; then
709
710 # RELIABLE
711 if [[ "${skip_rel_tests}" == "" ]]; then
712 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
713 fi
714
715 # BEST EFFORT
716 if [[ "${skip_be_tests}" == "" ]]; then
717 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
718 fi
719 fi
720
721fi
722
723# Tests that will not use batching
724if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
725
726 if [[ "$role" == "pub" ]]; then
727 export commands_string="${commands_string} -batchSize 0"
728 fi
729
730 # UNKEYED
731 if [[ "${skip_unkeyed}" == "" ]]; then
732
733 # RELIABLE
734 if [[ "${skip_rel_tests}" == "" ]]; then
735 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
736 fi
737
738 # BEST EFFORT
739 if [[ "${skip_be_tests}" == "" ]]; then
740 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
741 fi
742 fi
743
744 # KEYED
745 if [[ "${skip_keyed_data}" == "" ]]; then
746
747 # RELIABLE
748 if [[ "${skip_rel_tests}" == "" ]]; then
749 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
750 fi
751
752 # BEST EFFORT
753 if [[ "${skip_be_tests}" == "" ]]; then
754 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
755 fi
756 fi
757
758fi
759
760# Replace the cleanup section at the end:
761if [[ "${configure_network}" == "1" ]]; then
762 echo -e "${INFO_TAG} Resetting network interface to default state"
763 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
764 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
765fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
10
11# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
12# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
13# --extra-arguments "-securePSK str:1:SecretKey"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Not using security libraries
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set to a non-zero value, we will add it to the command line of the publisher side.
89 # When batch_size is 0, it is already handled by the no-batching block which appends -batchSize 0 to commands_string.
90 if [[ "$batch_size" != "" && "$batch_size" != "0" && "$role" == "pub" ]]; then
91 commands_string_test="${commands_string_test} -batchSize $batch_size"
92 fi
93
94 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
95
96 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
97
98 if [[ "$role" == "pub" ]]; then
99 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
100 fi
101
102 if [[ "$thread_cpu_affinity" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
103 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
104 fi
105
106 local pre_command_string=""
107
108 if [[ "$thread_priorities" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
109 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
110 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
111 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
112 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
113 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
114 fi
115
116 if [[ "$pin_memory" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
117 commands_string_test="${commands_string_test} -pinMemory"
118 fi
119
120 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
121 pre_command_string="$pre_command_string taskset -c $core"
122 fi
123
124 if [[ "$LANGUAGE" == "python" ]]; then
125 pre_command_string="$pre_command_string python3 "
126 fi
127
128 if [[ "$DOCKER" == "1" ]]; then
129 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
130 executable=""
131 fi
132
133 # Get the aprox time this will take:
134 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
135 total_time=$((total_tests * exec_time))
136
137 touch $output_file
138 local no_headers=""
139 local current_test=0
140 for index in $(seq 1 ${num_reps}); do
141 for DATALEN in ${datasizes_test}; do
142 current_test=$((current_test + 1))
143
144 if [[ ! -s $output_file ]]; then
145 echo -e "${INFO_TAG} Output file is empty, filling the header."
146 no_headers=""
147 else
148 echo -e "${INFO_TAG} Output file is not empty."
149 no_headers=" -noOutputHeaders"
150 fi
151
152 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
153 if [[ "$role" == "pub" ]]; then
154 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
155 echo -e ${BLUE}$command${NC}
156 else
157 echo -e ${LIGHTBLUE}$command${NC}
158 fi
159
160 # In certain cases we need to wait a bit before running the test, this is
161 # because the previous test might not be finished on the other side yet, or because the
162 # discovery mechanism does not work like in connext DDS.
163 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
164 sleep 3
165 fi
166 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
167 sleep 5
168 fi
169
170 # Gather netstat info before running the test
171 if [[ "${get_netstat_info}" == "1" ]]; then
172 echo -e "${INFO_TAG} Getting netstat info before"
173 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
174 fi
175
176 # Execute the command and capture the output
177 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
178 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
179 touch ${output_file}_tmp_line
180 chmod 777 ${output_file}_tmp_line
181 eval $command -outputFile ${output_file}_tmp_line
182 else
183 eval $command > ${output_file}_tmp_line
184 fi
185
186 number_of_lines=$(wc -l < ${output_file}_tmp_line)
187 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
188
189 # Check if this is the first test for this output file.
190 local is_first_test=0
191 if [[ ! -s $output_file ]]; then
192 is_first_test=1
193 fi
194 local expected_lines=$((is_first_test+1))
195
196 if [[ $number_of_lines -ne ${expected_lines} ]]; then
197 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
198 echo -e "${RED} The content is: \n\"\"\""
199 cat ${output_file}_tmp_line
200 echo -e "\"\"\""
201 echo -e "Not adding it to the output file. ${NC}"
202 else
203 if [[ $number_of_lines -gt 1 ]]; then
204 header_line=$(head -n 1 "${output_file}_tmp_line")
205 echo -e "${INFO_TAG} Header line is: $header_line"
206 fi
207 result_line=$(tail -n 1 "${output_file}_tmp_line")
208 echo -e "${INFO_TAG} Result line is: $result_line"
209
210 # If this is the first test, we need to add the header line to the output file
211 if [[ $is_first_test -eq 1 ]]; then
212 append_to_output_file "$header_line" ", command-line" $output_file
213 fi
214
215 command=$(echo $command | sed 's/,/:comma:/g')
216 # Remove the `"` characters from the command line
217 command=$(echo $command | sed 's/\"//g')
218
219 # Append the result line to the output file
220 append_to_output_file "$result_line" ", $command" $output_file
221 fi
222
223 # Always remove the temporary file
224 rm -rf ${output_file}_tmp_line
225
226 # Gather netstat info after running the test
227 if [[ "${get_netstat_info}" == "1" ]]; then
228 echo -e "${INFO_TAG} Getting netstat info after"
229 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
230 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
231 python3 $script_location/../../../tools/diff_netstat_output.py \
232 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
233 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
234 -d $DATALEN $no_header_netstat \
235 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
236 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
237 no_header_netstat=" -nh"
238 fi
239
240 change_domain
241 done
242 done
243}
244
245################################################################################
246# PARSE COMMAND LINE OPTIONS:
247
248while [ "$1" != "" ]; do
249 case $1 in
250 --executable)
251 executable=$2
252 shift
253 ;;
254 --docker)
255 DOCKER="1"
256 ;;
257 --output-folder)
258 output_folder=$2
259 shift
260 ;;
261 --role)
262 export role=$2
263 shift
264 ;;
265 --core)
266 export core=$2
267 shift
268 ;;
269 --assign-core-list | --thread-cpu-affinity)
270 export thread_cpu_affinity=$2
271 shift
272 ;;
273 --assign-thread-priorities | --thread-priorities)
274 export thread_priorities=$2
275 shift
276 ;;
277 --pin-memory)
278 export pin_memory="1"
279 ;;
280 --test-kind)
281 export lat_thr=$2
282 shift
283 ;;
284 --interface1)
285 export interface=$2
286 shift
287 ;;
288 --interface2)
289 export interface2=$2
290 shift
291 ;;
292 --ip1)
293 export ip1=$2
294 shift
295 ;;
296 --ip2)
297 export ip2=$2
298 shift
299 ;;
300 --repetitions)
301 export num_reps=$2
302 shift
303 ;;
304 --domain)
305 export domain=$2
306 shift
307 ;;
308 --execution-time)
309 export exec_time=$2
310 shift
311 ;;
312 --transport)
313 export transport=$2
314 shift
315 ;;
316 --datalen)
317 export datalen_input=$2
318 shift
319 ;;
320 --file-suffix)
321 export file_suffix=$2
322 shift
323 ;;
324 --folder-suffix)
325 export folder_suffix=$2
326 shift
327 ;;
328 --sub-folder)
329 sub_folder=$2
330 shift
331 ;;
332 --executable-suffix)
333 export executable_suffix=$2
334 shift
335 ;;
336 --extra-arguments)
337 export extra_arguments=$2
338 shift
339 ;;
340 --extra-arguments-pub)
341 export extra_arguments_pub=$2
342 shift
343 ;;
344 --extra-arguments-sub)
345 export extra_arguments_sub=$2
346 shift
347 ;;
348 --skip-be)
349 export skip_be_tests="1"
350 ;;
351 --skip-rel)
352 export skip_rel_tests="1"
353 ;;
354 --skip-keyed)
355 export skip_keyed_data="1"
356 ;;
357 --skip-large-data)
358 export skip_large_data="1"
359 ;;
360 --large-data)
361 export large_data="1"
362 ;;
363 --keyed)
364 export skip_unkeyed="1"
365 ;;
366 --unkeyed)
367 export skip_keyed_data="1"
368 ;;
369 --no-batching | --skip-batching)
370 export run_batching_tests="0"
371 export run_no_batching_tests="1"
372 ;;
373 --skip-no-batching)
374 export run_batching_tests="1"
375 export run_no_batching_tests="0"
376 ;;
377 --batch-size)
378 export batch_size=$2
379 shift
380 ;;
381 --reliable)
382 export skip_be_tests="1"
383 ;;
384 --best-effort)
385 export skip_rel_tests="1"
386 ;;
387 --security-gov)
388 export security_only="$2"
389 shift
390 ;;
391 --micro)
392 export micro="1"
393 ;;
394 --cert)
395 export cert="1"
396 ;;
397 --raw | --raw-transport)
398 export raw="1"
399 ;;
400 --tss)
401 export tss="1"
402 ;;
403 --no-colors)
404 export NO_COLORS="1"
405 ;;
406 --language)
407 export LANGUAGE=$2
408 shift
409 ;;
410 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
411 export nic_loss_rate=$2
412 export configure_network="1"
413 shift
414 ;;
415 --delay | --nic-delay-ms | --nic-delay)
416 export nic_delay_ms=$2
417 export configure_network="1"
418 shift
419 ;;
420 --get-netstat-info | --netstat)
421 export get_netstat_info="1"
422 ;;
423 --reduced-data-sizes-set)
424 export REDUCED_DATA_SIZES_SET="1"
425 ;;
426 --dont-change-tuned-settings)
427 export dont_change_tuned_settings="1"
428 ;;
429 --no-taskset)
430 export no_taskset=1
431 ;;
432 --asynchronous | --async)
433 export ASYNC="1"
434 ;;
435 *)
436 echo -e "unknown parameter \"$1\""
437 exit 255
438 ;;
439 esac
440 shift
441done
442
443if [[ "$NO_COLORS" == "1" ]]; then
444 disable_colors
445fi
446
447export folder_base="$(dirname "${executable}")"/../../..
448
449if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
450 export folder_base="$(dirname "${executable}")"/../..
451fi
452if [[ $tss == "1" ]]; then
453 export folder_base="$(dirname "${executable}")"/../../../../..
454fi
455
456if [[ "${executable_suffix}" != "" ]]; then
457 export executable="${executable}${executable_suffix}"
458fi
459
460if [[ -n "${folder_suffix}" ]]; then
461 export output_folder="${output_folder}${folder_suffix}"
462fi
463
464if [[ -n "${sub_folder}" ]]; then
465 export output_folder="${output_folder}/${sub_folder}"
466fi
467
468if [[ "${ASYNC}" == "1" ]]; then
469 export output_folder="${output_folder}/async"
470fi
471
472echo -e "${INFO_TAG} Perftest executable is: $executable"
473echo -e "${INFO_TAG} Output folder is: $output_folder"
474
475################################################################################
476
477if [[ "$LANGUAGE" == "python" ]]; then
478 export skip_keyed_data="1"
479 export skip_large_data="1"
480 export skip_be_tests="1"
481 export run_no_batching_tests="0"
482fi
483
484if [[ "$LANGUAGE" == "java" || "$LANGUAGE" == "cs" ]]; then
485 # Java and C# perftest do not support these execution options.
486 unset thread_cpu_affinity
487 unset thread_priorities
488 unset pin_memory
489 export no_taskset=1
490fi
491
492if [[ "${skip_large_data}" == "1" ]]; then
493 export datasizes_extended=${datasizes}
494elif [[ "${large_data}" == "1" ]]; then
495 export datasizes=${datasizes_extended}
496fi
497
498if [[ "${datalen_input}" != "" ]]; then
499 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
500 export datasizes=${datalen_input}
501 export datasizes_extended=${datalen_input}
502 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
503 export skip_large_data="1"
504 fi
505else
506 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
507 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
508 export datasizes="32 128 512 2048 8192 32768 63000"
509 export datasizes_extended="${datasizes} 102400 1048576 10485760"
510 fi
511fi
512
513if [[ "$role" != "pub" && "$role" != "sub" ]]; then
514 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
515 exit 255
516fi
517
518if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
519 echo -e "${ERROR_TAG} It must be either lat or thr"
520 exit 255
521fi
522
523if [[ "${interface}" == "" ]]; then
524 echo "Using default nics"
525 export nic_publisher=${ip_machine_1}
526 export nic_subscriber=${ip_machine_2}
527elif [[ "${interface}" == "both" ]]; then
528 export nic_publisher="enp1s0f0,eno1"
529 export nic_subscriber="enp1s0f0,eno1"
530 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
531 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
532else
533 export nic_publisher=$interface
534 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
535
536 if [[ "${interface2}" == "" ]]; then
537 export nic_subscriber=$interface
538 else
539 export nic_subscriber=$interface2
540 fi
541 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
542
543 if [[ "${ip1}" != "" ]]; then
544 export ip_publisher=$ip1
545 echo "Using ip_publisher: ${ip_publisher}"
546 fi
547
548 if [[ "${ip2}" != "" ]]; then
549 export ip_subscriber=$ip2
550 echo "Using ip_subscriber: ${ip_subscriber}"
551 fi
552
553fi
554
555if [[ "$transport" != "" ]]; then
556 export transport_string="-transport $transport"
557
558 if [[ "$transport" == "UDPv4" ]]; then
559
560 export transport_string_pub="$transport_string -nic $nic_publisher"
561 export transport_string_sub="$transport_string -nic $nic_subscriber"
562
563 if [[ "$raw" == "1" ]]; then
564 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
565 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
566 fi
567
568 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
569 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
570 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
571 fi
572
573 elif [[ "$transport" == "TCP" ]]; then
574 export transport_string_pub="$transport_string \
575 -nic $nic_publisher \
576 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
577 export transport_string_sub="$transport_string \
578 -nic $nic_subscriber \
579 -peer 0@tcpv4_lan://${ip_publisher}:7400"
580 elif [[ "$transport" == "TLS" ]]; then
581 export transport_string_pub="$transport_string \
582 -nic $nic_publisher \
583 -peer tlsv4_lan://${ip_subscriber}:7400"
584 export transport_string_sub="$transport_string \
585 -nic $nic_subscriber \
586 -peer tlsv4_lan://${ip_publisher}:7400"
587 elif [[ "$transport" == "UDPv4_WAN" ]]; then
588 export transport_string_pub="$transport_string \
589 -nic $nic_publisher \
590 -transportPublicAddress $ip_publisher:7400"
591 export transport_string_sub="$transport_string \
592 -nic $nic_subscriber \
593 -peer 0@udpv4_wan://${ip_publisher}:7400"
594 else
595 export transport_string_pub="$transport_string"
596 export transport_string_sub="$transport_string"
597 fi
598fi
599
600################################################################################
601
602export pub_string="-pub \
603 ${transport_string_pub} \
604 -noPrintIntervals \
605 -executionTime $exec_time"
606
607if [[ ${lat_thr} == "lat" ]]; then
608 export pub_string="$pub_string \
609 -latencyTest"
610fi
611
612if [[ "$role" == "pub" ]]; then
613
614 if [[ "$configure_network" == "1" ]]; then
615 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
616 export pub_string="$pub_string -initialBurst 1"
617 fi
618
619 if [[ "$ASYNC" == "1" ]]; then
620 export pub_string="${pub_string} -asynchronous"
621 fi
622fi
623
624export sub_string="-sub \
625 ${transport_string_sub} \
626 -noPrintIntervals"
627
628if [[ "$role" == "pub" ]]; then
629 echo -e "$INFO_TAG Publisher side running"
630 export commands_string=${pub_string}
631 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
632else
633 echo -e "$INFO_TAG Subscriber side running"
634 export commands_string=${sub_string}
635 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
636fi
637
638###############################################################################
639
640if [[ "$dont_change_tuned_settings" != "1" ]]; then
641 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
642 sudo /set_${lat_thr}_mode.sh
643 sleep 5
644fi
645
646if [[ "$role" == "pub" ]]; then
647 export interface_to_configure=$nic_publisher
648else
649 export interface_to_configure=$nic_subscriber
650fi
651
652echo -e "${INFO_TAG} Resetting network interface to default state"
653# Try to delete any existing netem rules (ignore errors if none exist)
654sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
655# Reset to default (pfifo_fast)
656sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
657
658if [[ "${configure_network}" == "1" ]]; then
659 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
660 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
661fi
662
663cd $folder_base
664echo -e "${INFO_TAG} Folder Base is: $PWD"
665mkdir -p $output_folder
666
667if [[ "${batch_size}" != "" ]]; then
668 if [[ "${lat_thr}" == "thr" ]]; then
669 if [[ "${batch_size}" -eq 0 ]]; then
670 echo -e "${INFO_TAG} Batch size is set to 0"
671 export run_batching_tests="0"
672 export run_no_batching_tests="1"
673 else
674 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
675 export run_batching_tests="1"
676 export run_no_batching_tests="0"
677 fi
678 else
679 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
680 unset batch_size
681 fi
682fi
683
684# Tests that may use batching (when doing throughput tests). Also Latency tests.
685if [[ ${run_batching_tests} == "1" ]]; then
686
687 # UNKEYED
688 if [[ "${skip_unkeyed}" == "" ]]; then
689
690 # RELIABLE
691 if [[ "${skip_rel_tests}" == "" ]]; then
692 # For the time being, we will skip the throughput reliable tests for CERT.
693 # More info in PERF-1175
694 if ! [[ "${cert}" == "1" && "${lat_thr}" == "thr" && "${transport}" == "UDPv4" ]]; then
695 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
696 else
697 echo -e "${INFO_TAG} Skipping reliable throughput tests for CERT. See PERF-1175 for more information."
698 fi
699 fi
700
701 # BEST EFFORT
702 if [[ "${skip_be_tests}" == "" ]]; then
703 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
704 fi
705 fi
706
707 # KEYED
708 if [[ "${skip_keyed_data}" == "" ]]; then
709
710 # RELIABLE
711 if [[ "${skip_rel_tests}" == "" ]]; then
712 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
713 fi
714
715 # BEST EFFORT
716 if [[ "${skip_be_tests}" == "" ]]; then
717 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
718 fi
719 fi
720
721fi
722
723# Tests that will not use batching
724if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
725
726 if [[ "$role" == "pub" ]]; then
727 export commands_string="${commands_string} -batchSize 0"
728 fi
729
730 # UNKEYED
731 if [[ "${skip_unkeyed}" == "" ]]; then
732
733 # RELIABLE
734 if [[ "${skip_rel_tests}" == "" ]]; then
735 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
736 fi
737
738 # BEST EFFORT
739 if [[ "${skip_be_tests}" == "" ]]; then
740 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
741 fi
742 fi
743
744 # KEYED
745 if [[ "${skip_keyed_data}" == "" ]]; then
746
747 # RELIABLE
748 if [[ "${skip_rel_tests}" == "" ]]; then
749 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
750 fi
751
752 # BEST EFFORT
753 if [[ "${skip_be_tests}" == "" ]]; then
754 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
755 fi
756 fi
757
758fi
759
760# Replace the cleanup section at the end:
761if [[ "${configure_network}" == "1" ]]; then
762 echo -e "${INFO_TAG} Resetting network interface to default state"
763 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
764 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
765fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
10
11# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
12# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
13# --extra-arguments "-securePSK str:1:SecretKey"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set to a non-zero value, we will add it to the command line of the publisher side.
89 # When batch_size is 0, it is already handled by the no-batching block which appends -batchSize 0 to commands_string.
90 if [[ "$batch_size" != "" && "$batch_size" != "0" && "$role" == "pub" ]]; then
91 commands_string_test="${commands_string_test} -batchSize $batch_size"
92 fi
93
94 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
95
96 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
97
98 if [[ "$role" == "pub" ]]; then
99 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
100 fi
101
102 if [[ "$thread_cpu_affinity" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
103 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
104 fi
105
106 local pre_command_string=""
107
108 if [[ "$thread_priorities" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
109 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
110 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
111 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
112 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
113 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
114 fi
115
116 if [[ "$pin_memory" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
117 commands_string_test="${commands_string_test} -pinMemory"
118 fi
119
120 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
121 pre_command_string="$pre_command_string taskset -c $core"
122 fi
123
124 if [[ "$LANGUAGE" == "python" ]]; then
125 pre_command_string="$pre_command_string python3 "
126 fi
127
128 if [[ "$DOCKER" == "1" ]]; then
129 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
130 executable=""
131 fi
132
133 # Get the aprox time this will take:
134 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
135 total_time=$((total_tests * exec_time))
136
137 touch $output_file
138 local no_headers=""
139 local current_test=0
140 for index in $(seq 1 ${num_reps}); do
141 for DATALEN in ${datasizes_test}; do
142 current_test=$((current_test + 1))
143
144 if [[ ! -s $output_file ]]; then
145 echo -e "${INFO_TAG} Output file is empty, filling the header."
146 no_headers=""
147 else
148 echo -e "${INFO_TAG} Output file is not empty."
149 no_headers=" -noOutputHeaders"
150 fi
151
152 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
153 if [[ "$role" == "pub" ]]; then
154 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
155 echo -e ${BLUE}$command${NC}
156 else
157 echo -e ${LIGHTBLUE}$command${NC}
158 fi
159
160 # In certain cases we need to wait a bit before running the test, this is
161 # because the previous test might not be finished on the other side yet, or because the
162 # discovery mechanism does not work like in connext DDS.
163 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
164 sleep 3
165 fi
166 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
167 sleep 5
168 fi
169
170 # Gather netstat info before running the test
171 if [[ "${get_netstat_info}" == "1" ]]; then
172 echo -e "${INFO_TAG} Getting netstat info before"
173 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
174 fi
175
176 # Execute the command and capture the output
177 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
178 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
179 touch ${output_file}_tmp_line
180 chmod 777 ${output_file}_tmp_line
181 eval $command -outputFile ${output_file}_tmp_line
182 else
183 eval $command > ${output_file}_tmp_line
184 fi
185
186 number_of_lines=$(wc -l < ${output_file}_tmp_line)
187 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
188
189 # Check if this is the first test for this output file.
190 local is_first_test=0
191 if [[ ! -s $output_file ]]; then
192 is_first_test=1
193 fi
194 local expected_lines=$((is_first_test+1))
195
196 if [[ $number_of_lines -ne ${expected_lines} ]]; then
197 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
198 echo -e "${RED} The content is: \n\"\"\""
199 cat ${output_file}_tmp_line
200 echo -e "\"\"\""
201 echo -e "Not adding it to the output file. ${NC}"
202 else
203 if [[ $number_of_lines -gt 1 ]]; then
204 header_line=$(head -n 1 "${output_file}_tmp_line")
205 echo -e "${INFO_TAG} Header line is: $header_line"
206 fi
207 result_line=$(tail -n 1 "${output_file}_tmp_line")
208 echo -e "${INFO_TAG} Result line is: $result_line"
209
210 # If this is the first test, we need to add the header line to the output file
211 if [[ $is_first_test -eq 1 ]]; then
212 append_to_output_file "$header_line" ", command-line" $output_file
213 fi
214
215 command=$(echo $command | sed 's/,/:comma:/g')
216 # Remove the `"` characters from the command line
217 command=$(echo $command | sed 's/\"//g')
218
219 # Append the result line to the output file
220 append_to_output_file "$result_line" ", $command" $output_file
221 fi
222
223 # Always remove the temporary file
224 rm -rf ${output_file}_tmp_line
225
226 # Gather netstat info after running the test
227 if [[ "${get_netstat_info}" == "1" ]]; then
228 echo -e "${INFO_TAG} Getting netstat info after"
229 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
230 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
231 python3 $script_location/../../../tools/diff_netstat_output.py \
232 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
233 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
234 -d $DATALEN $no_header_netstat \
235 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
236 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
237 no_header_netstat=" -nh"
238 fi
239
240 change_domain
241 done
242 done
243}
244
245################################################################################
246# PARSE COMMAND LINE OPTIONS:
247
248while [ "$1" != "" ]; do
249 case $1 in
250 --executable)
251 executable=$2
252 shift
253 ;;
254 --docker)
255 DOCKER="1"
256 ;;
257 --output-folder)
258 output_folder=$2
259 shift
260 ;;
261 --role)
262 export role=$2
263 shift
264 ;;
265 --core)
266 export core=$2
267 shift
268 ;;
269 --assign-core-list | --thread-cpu-affinity)
270 export thread_cpu_affinity=$2
271 shift
272 ;;
273 --assign-thread-priorities | --thread-priorities)
274 export thread_priorities=$2
275 shift
276 ;;
277 --pin-memory)
278 export pin_memory="1"
279 ;;
280 --test-kind)
281 export lat_thr=$2
282 shift
283 ;;
284 --interface1)
285 export interface=$2
286 shift
287 ;;
288 --interface2)
289 export interface2=$2
290 shift
291 ;;
292 --ip1)
293 export ip1=$2
294 shift
295 ;;
296 --ip2)
297 export ip2=$2
298 shift
299 ;;
300 --repetitions)
301 export num_reps=$2
302 shift
303 ;;
304 --domain)
305 export domain=$2
306 shift
307 ;;
308 --execution-time)
309 export exec_time=$2
310 shift
311 ;;
312 --transport)
313 export transport=$2
314 shift
315 ;;
316 --datalen)
317 export datalen_input=$2
318 shift
319 ;;
320 --file-suffix)
321 export file_suffix=$2
322 shift
323 ;;
324 --folder-suffix)
325 export folder_suffix=$2
326 shift
327 ;;
328 --sub-folder)
329 sub_folder=$2
330 shift
331 ;;
332 --executable-suffix)
333 export executable_suffix=$2
334 shift
335 ;;
336 --extra-arguments)
337 export extra_arguments=$2
338 shift
339 ;;
340 --extra-arguments-pub)
341 export extra_arguments_pub=$2
342 shift
343 ;;
344 --extra-arguments-sub)
345 export extra_arguments_sub=$2
346 shift
347 ;;
348 --skip-be)
349 export skip_be_tests="1"
350 ;;
351 --skip-rel)
352 export skip_rel_tests="1"
353 ;;
354 --skip-keyed)
355 export skip_keyed_data="1"
356 ;;
357 --skip-large-data)
358 export skip_large_data="1"
359 ;;
360 --large-data)
361 export large_data="1"
362 ;;
363 --keyed)
364 export skip_unkeyed="1"
365 ;;
366 --unkeyed)
367 export skip_keyed_data="1"
368 ;;
369 --no-batching | --skip-batching)
370 export run_batching_tests="0"
371 export run_no_batching_tests="1"
372 ;;
373 --skip-no-batching)
374 export run_batching_tests="1"
375 export run_no_batching_tests="0"
376 ;;
377 --batch-size)
378 export batch_size=$2
379 shift
380 ;;
381 --reliable)
382 export skip_be_tests="1"
383 ;;
384 --best-effort)
385 export skip_rel_tests="1"
386 ;;
387 --security-gov)
388 export security_only="$2"
389 shift
390 ;;
391 --micro)
392 export micro="1"
393 ;;
394 --cert)
395 export cert="1"
396 ;;
397 --raw | --raw-transport)
398 export raw="1"
399 ;;
400 --tss)
401 export tss="1"
402 ;;
403 --no-colors)
404 export NO_COLORS="1"
405 ;;
406 --language)
407 export LANGUAGE=$2
408 shift
409 ;;
410 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
411 export nic_loss_rate=$2
412 export configure_network="1"
413 shift
414 ;;
415 --delay | --nic-delay-ms | --nic-delay)
416 export nic_delay_ms=$2
417 export configure_network="1"
418 shift
419 ;;
420 --get-netstat-info | --netstat)
421 export get_netstat_info="1"
422 ;;
423 --reduced-data-sizes-set)
424 export REDUCED_DATA_SIZES_SET="1"
425 ;;
426 --dont-change-tuned-settings)
427 export dont_change_tuned_settings="1"
428 ;;
429 --no-taskset)
430 export no_taskset=1
431 ;;
432 --asynchronous | --async)
433 export ASYNC="1"
434 ;;
435 *)
436 echo -e "unknown parameter \"$1\""
437 exit 255
438 ;;
439 esac
440 shift
441done
442
443if [[ "$NO_COLORS" == "1" ]]; then
444 disable_colors
445fi
446
447export folder_base="$(dirname "${executable}")"/../../..
448
449if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
450 export folder_base="$(dirname "${executable}")"/../..
451fi
452if [[ $tss == "1" ]]; then
453 export folder_base="$(dirname "${executable}")"/../../../../..
454fi
455
456if [[ "${executable_suffix}" != "" ]]; then
457 export executable="${executable}${executable_suffix}"
458fi
459
460if [[ -n "${folder_suffix}" ]]; then
461 export output_folder="${output_folder}${folder_suffix}"
462fi
463
464if [[ -n "${sub_folder}" ]]; then
465 export output_folder="${output_folder}/${sub_folder}"
466fi
467
468if [[ "${ASYNC}" == "1" ]]; then
469 export output_folder="${output_folder}/async"
470fi
471
472echo -e "${INFO_TAG} Perftest executable is: $executable"
473echo -e "${INFO_TAG} Output folder is: $output_folder"
474
475################################################################################
476
477if [[ "$LANGUAGE" == "python" ]]; then
478 export skip_keyed_data="1"
479 export skip_large_data="1"
480 export skip_be_tests="1"
481 export run_no_batching_tests="0"
482fi
483
484if [[ "$LANGUAGE" == "java" || "$LANGUAGE" == "cs" ]]; then
485 # Java and C# perftest do not support these execution options.
486 unset thread_cpu_affinity
487 unset thread_priorities
488 unset pin_memory
489 export no_taskset=1
490fi
491
492if [[ "${skip_large_data}" == "1" ]]; then
493 export datasizes_extended=${datasizes}
494elif [[ "${large_data}" == "1" ]]; then
495 export datasizes=${datasizes_extended}
496fi
497
498if [[ "${datalen_input}" != "" ]]; then
499 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
500 export datasizes=${datalen_input}
501 export datasizes_extended=${datalen_input}
502 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
503 export skip_large_data="1"
504 fi
505else
506 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
507 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
508 export datasizes="32 128 512 2048 8192 32768 63000"
509 export datasizes_extended="${datasizes} 102400 1048576 10485760"
510 fi
511fi
512
513if [[ "$role" != "pub" && "$role" != "sub" ]]; then
514 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
515 exit 255
516fi
517
518if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
519 echo -e "${ERROR_TAG} It must be either lat or thr"
520 exit 255
521fi
522
523if [[ "${interface}" == "" ]]; then
524 echo "Using default nics"
525 export nic_publisher=${ip_machine_1}
526 export nic_subscriber=${ip_machine_2}
527elif [[ "${interface}" == "both" ]]; then
528 export nic_publisher="enp1s0f0,eno1"
529 export nic_subscriber="enp1s0f0,eno1"
530 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
531 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
532else
533 export nic_publisher=$interface
534 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
535
536 if [[ "${interface2}" == "" ]]; then
537 export nic_subscriber=$interface
538 else
539 export nic_subscriber=$interface2
540 fi
541 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
542
543 if [[ "${ip1}" != "" ]]; then
544 export ip_publisher=$ip1
545 echo "Using ip_publisher: ${ip_publisher}"
546 fi
547
548 if [[ "${ip2}" != "" ]]; then
549 export ip_subscriber=$ip2
550 echo "Using ip_subscriber: ${ip_subscriber}"
551 fi
552
553fi
554
555if [[ "$transport" != "" ]]; then
556 export transport_string="-transport $transport"
557
558 if [[ "$transport" == "UDPv4" ]]; then
559
560 export transport_string_pub="$transport_string -nic $nic_publisher"
561 export transport_string_sub="$transport_string -nic $nic_subscriber"
562
563 if [[ "$raw" == "1" ]]; then
564 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
565 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
566 fi
567
568 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
569 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
570 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
571 fi
572
573 elif [[ "$transport" == "TCP" ]]; then
574 export transport_string_pub="$transport_string \
575 -nic $nic_publisher \
576 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
577 export transport_string_sub="$transport_string \
578 -nic $nic_subscriber \
579 -peer 0@tcpv4_lan://${ip_publisher}:7400"
580 elif [[ "$transport" == "TLS" ]]; then
581 export transport_string_pub="$transport_string \
582 -nic $nic_publisher \
583 -peer tlsv4_lan://${ip_subscriber}:7400"
584 export transport_string_sub="$transport_string \
585 -nic $nic_subscriber \
586 -peer tlsv4_lan://${ip_publisher}:7400"
587 elif [[ "$transport" == "UDPv4_WAN" ]]; then
588 export transport_string_pub="$transport_string \
589 -nic $nic_publisher \
590 -transportPublicAddress $ip_publisher:7400"
591 export transport_string_sub="$transport_string \
592 -nic $nic_subscriber \
593 -peer 0@udpv4_wan://${ip_publisher}:7400"
594 else
595 export transport_string_pub="$transport_string"
596 export transport_string_sub="$transport_string"
597 fi
598fi
599
600################################################################################
601
602export pub_string="-pub \
603 ${transport_string_pub} \
604 -noPrintIntervals \
605 -executionTime $exec_time"
606
607if [[ ${lat_thr} == "lat" ]]; then
608 export pub_string="$pub_string \
609 -latencyTest"
610fi
611
612if [[ "$role" == "pub" ]]; then
613
614 if [[ "$configure_network" == "1" ]]; then
615 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
616 export pub_string="$pub_string -initialBurst 1"
617 fi
618
619 if [[ "$ASYNC" == "1" ]]; then
620 export pub_string="${pub_string} -asynchronous"
621 fi
622fi
623
624export sub_string="-sub \
625 ${transport_string_sub} \
626 -noPrintIntervals"
627
628if [[ "$role" == "pub" ]]; then
629 echo -e "$INFO_TAG Publisher side running"
630 export commands_string=${pub_string}
631 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
632else
633 echo -e "$INFO_TAG Subscriber side running"
634 export commands_string=${sub_string}
635 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
636fi
637
638###############################################################################
639
640if [[ "$dont_change_tuned_settings" != "1" ]]; then
641 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
642 sudo /set_${lat_thr}_mode.sh
643 sleep 5
644fi
645
646if [[ "$role" == "pub" ]]; then
647 export interface_to_configure=$nic_publisher
648else
649 export interface_to_configure=$nic_subscriber
650fi
651
652echo -e "${INFO_TAG} Resetting network interface to default state"
653# Try to delete any existing netem rules (ignore errors if none exist)
654sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
655# Reset to default (pfifo_fast)
656sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
657
658if [[ "${configure_network}" == "1" ]]; then
659 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
660 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
661fi
662
663cd $folder_base
664echo -e "${INFO_TAG} Folder Base is: $PWD"
665mkdir -p $output_folder
666
667if [[ "${batch_size}" != "" ]]; then
668 if [[ "${lat_thr}" == "thr" ]]; then
669 if [[ "${batch_size}" -eq 0 ]]; then
670 echo -e "${INFO_TAG} Batch size is set to 0"
671 export run_batching_tests="0"
672 export run_no_batching_tests="1"
673 else
674 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
675 export run_batching_tests="1"
676 export run_no_batching_tests="0"
677 fi
678 else
679 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
680 unset batch_size
681 fi
682fi
683
684# Tests that may use batching (when doing throughput tests). Also Latency tests.
685if [[ ${run_batching_tests} == "1" ]]; then
686
687 # UNKEYED
688 if [[ "${skip_unkeyed}" == "" ]]; then
689
690 # RELIABLE
691 if [[ "${skip_rel_tests}" == "" ]]; then
692 # For the time being, we will skip the throughput reliable tests for CERT.
693 # More info in PERF-1175
694 if ! [[ "${cert}" == "1" && "${lat_thr}" == "thr" && "${transport}" == "UDPv4" ]]; then
695 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
696 else
697 echo -e "${INFO_TAG} Skipping reliable throughput tests for CERT. See PERF-1175 for more information."
698 fi
699 fi
700
701 # BEST EFFORT
702 if [[ "${skip_be_tests}" == "" ]]; then
703 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
704 fi
705 fi
706
707 # KEYED
708 if [[ "${skip_keyed_data}" == "" ]]; then
709
710 # RELIABLE
711 if [[ "${skip_rel_tests}" == "" ]]; then
712 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
713 fi
714
715 # BEST EFFORT
716 if [[ "${skip_be_tests}" == "" ]]; then
717 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
718 fi
719 fi
720
721fi
722
723# Tests that will not use batching
724if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
725
726 if [[ "$role" == "pub" ]]; then
727 export commands_string="${commands_string} -batchSize 0"
728 fi
729
730 # UNKEYED
731 if [[ "${skip_unkeyed}" == "" ]]; then
732
733 # RELIABLE
734 if [[ "${skip_rel_tests}" == "" ]]; then
735 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
736 fi
737
738 # BEST EFFORT
739 if [[ "${skip_be_tests}" == "" ]]; then
740 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
741 fi
742 fi
743
744 # KEYED
745 if [[ "${skip_keyed_data}" == "" ]]; then
746
747 # RELIABLE
748 if [[ "${skip_rel_tests}" == "" ]]; then
749 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
750 fi
751
752 # BEST EFFORT
753 if [[ "${skip_be_tests}" == "" ]]; then
754 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
755 fi
756 fi
757
758fi
759
760# Replace the cleanup section at the end:
761if [[ "${configure_network}" == "1" ]]; then
762 echo -e "${INFO_TAG} Resetting network interface to default state"
763 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
764 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
765fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GCM" \
10 --executable-suffix "_lws" \
11 --file-suffix "_AES256_GCM"
12
13"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
14 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
15 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES128+GCM" \
16 --executable-suffix "_lws" \
17 --file-suffix "_AES128_GCM"
18
19# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
20# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
21# --extra-arguments "-securePSK str:1:SecretKey -securePSKAlgorithm AES256+GCM" \
22# --executable-suffix "_lws" \
23# --file-suffix "_AES256_GCM"
24
25"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
26 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
27 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GMAC" \
28 --executable-suffix "_lws" \
29 --file-suffix "_AES256_GMAC"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Not using security libraries
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The scripts used to execute the tests can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13export run_batching_tests="1"
14export run_no_batching_tests="1"
15export nic_loss_rate=0
16export nic_delay_ms=0
17
18# We will use some colors to improve visibility of errors and info messages.
19RED='\033[0;31m'
20GREEN='\033[0;32m'
21YELLOW='\033[0;33m'
22BLUE='\033[0;34m'
23LIGHTBLUE='\033[0;36m'
24NC='\033[0m'
25INFO_TAG="${GREEN}[INFO][$role]:${NC}"
26WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
27ERROR_TAG="${RED}[ERROR][$role]:${NC}"
28
29################################################################################
30
31function disable_colors() {
32 export RED=""
33 export GREEN=""
34 export YELLOW=""
35 export NC=""
36 export BLUE=""
37 export LIGHTBLUE=""
38 export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
39 export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
40 export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
41}
42
43function change_domain() {
44 if [[ "$domain" == "1" ]]; then
45 export domain="2"
46 else
47 export domain="1"
48 fi
49}
50
51# Function to process and append lines to the output file
52function append_to_output_file() {
53 local line=${1?line required}; readonly line
54 local append=$(echo "${2?append required}" | tr -s ' '); readonly append
55 local file=${3?file required}; readonly file
56 if [[ "${line: -1}" == $'\n' ]]; then
57 line="${line::-1}"
58 fi
59 echo "${line}${append}" >> "$file"
60}
61
62# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
63function execute_test() {
64
65 local keyed_unkeyed=$1
66 local rel_be=$2
67 local datasizes_test=$3
68 local other_args=$4
69 local name_suffix=$5
70
71 local commands_string_test=$commands_string
72 local tag=""
73
74 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
75 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
76 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
77 else
78 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
79 fi
80
81 if [[ "${rel_be}" == "be" ]]; then
82 commands_string_test="${commands_string_test} -bestEffort"
83 tag="${tag}${YELLOW}BE${NC}]"
84 else
85 tag="${tag}${RED}REL${NC}]"
86 fi
87
88 # If batch_size is set to a non-zero value, we will add it to the command line of the publisher side.
89 # When batch_size is 0, it is already handled by the no-batching block which appends -batchSize 0 to commands_string.
90 if [[ "$batch_size" != "" && "$batch_size" != "0" && "$role" == "pub" ]]; then
91 commands_string_test="${commands_string_test} -batchSize $batch_size"
92 fi
93
94 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
95
96 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
97
98 if [[ "$role" == "pub" ]]; then
99 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
100 fi
101
102 if [[ "$thread_cpu_affinity" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
103 commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
104 fi
105
106 local pre_command_string=""
107
108 if [[ "$thread_priorities" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
109 echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
110 # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
111 export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
112 pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
113 commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
114 fi
115
116 if [[ "$pin_memory" != "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
117 commands_string_test="${commands_string_test} -pinMemory"
118 fi
119
120 if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
121 pre_command_string="$pre_command_string taskset -c $core"
122 fi
123
124 if [[ "$LANGUAGE" == "python" ]]; then
125 pre_command_string="$pre_command_string python3 "
126 fi
127
128 if [[ "$DOCKER" == "1" ]]; then
129 pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
130 executable=""
131 fi
132
133 # Get the aprox time this will take:
134 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
135 total_time=$((total_tests * exec_time))
136
137 touch $output_file
138 local no_headers=""
139 local current_test=0
140 for index in $(seq 1 ${num_reps}); do
141 for DATALEN in ${datasizes_test}; do
142 current_test=$((current_test + 1))
143
144 if [[ ! -s $output_file ]]; then
145 echo -e "${INFO_TAG} Output file is empty, filling the header."
146 no_headers=""
147 else
148 echo -e "${INFO_TAG} Output file is not empty."
149 no_headers=" -noOutputHeaders"
150 fi
151
152 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
153 if [[ "$role" == "pub" ]]; then
154 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
155 echo -e ${BLUE}$command${NC}
156 else
157 echo -e ${LIGHTBLUE}$command${NC}
158 fi
159
160 # In certain cases we need to wait a bit before running the test, this is
161 # because the previous test might not be finished on the other side yet, or because the
162 # discovery mechanism does not work like in connext DDS.
163 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
164 sleep 3
165 fi
166 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
167 sleep 5
168 fi
169
170 # Gather netstat info before running the test
171 if [[ "${get_netstat_info}" == "1" ]]; then
172 echo -e "${INFO_TAG} Getting netstat info before"
173 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
174 fi
175
176 # Execute the command and capture the output
177 if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
178 echo -e "${INFO_TAG} Using C++98, using -outputFile option"
179 touch ${output_file}_tmp_line
180 chmod 777 ${output_file}_tmp_line
181 eval $command -outputFile ${output_file}_tmp_line
182 else
183 eval $command > ${output_file}_tmp_line
184 fi
185
186 number_of_lines=$(wc -l < ${output_file}_tmp_line)
187 echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"
188
189 # Check if this is the first test for this output file.
190 local is_first_test=0
191 if [[ ! -s $output_file ]]; then
192 is_first_test=1
193 fi
194 local expected_lines=$((is_first_test+1))
195
196 if [[ $number_of_lines -ne ${expected_lines} ]]; then
197 echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
198 echo -e "${RED} The content is: \n\"\"\""
199 cat ${output_file}_tmp_line
200 echo -e "\"\"\""
201 echo -e "Not adding it to the output file. ${NC}"
202 else
203 if [[ $number_of_lines -gt 1 ]]; then
204 header_line=$(head -n 1 "${output_file}_tmp_line")
205 echo -e "${INFO_TAG} Header line is: $header_line"
206 fi
207 result_line=$(tail -n 1 "${output_file}_tmp_line")
208 echo -e "${INFO_TAG} Result line is: $result_line"
209
210 # If this is the first test, we need to add the header line to the output file
211 if [[ $is_first_test -eq 1 ]]; then
212 append_to_output_file "$header_line" ", command-line" $output_file
213 fi
214
215 command=$(echo $command | sed 's/,/:comma:/g')
216 # Remove the `"` characters from the command line
217 command=$(echo $command | sed 's/\"//g')
218
219 # Append the result line to the output file
220 append_to_output_file "$result_line" ", $command" $output_file
221 fi
222
223 # Always remove the temporary file
224 rm -rf ${output_file}_tmp_line
225
226 # Gather netstat info after running the test
227 if [[ "${get_netstat_info}" == "1" ]]; then
228 echo -e "${INFO_TAG} Getting netstat info after"
229 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
230 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
231 python3 $script_location/../../../tools/diff_netstat_output.py \
232 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
233 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
234 -d $DATALEN $no_header_netstat \
235 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
236 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
237 no_header_netstat=" -nh"
238 fi
239
240 change_domain
241 done
242 done
243}
244
245################################################################################
246# PARSE COMMAND LINE OPTIONS:
247
248while [ "$1" != "" ]; do
249 case $1 in
250 --executable)
251 executable=$2
252 shift
253 ;;
254 --docker)
255 DOCKER="1"
256 ;;
257 --output-folder)
258 output_folder=$2
259 shift
260 ;;
261 --role)
262 export role=$2
263 shift
264 ;;
265 --core)
266 export core=$2
267 shift
268 ;;
269 --assign-core-list | --thread-cpu-affinity)
270 export thread_cpu_affinity=$2
271 shift
272 ;;
273 --assign-thread-priorities | --thread-priorities)
274 export thread_priorities=$2
275 shift
276 ;;
277 --pin-memory)
278 export pin_memory="1"
279 ;;
280 --test-kind)
281 export lat_thr=$2
282 shift
283 ;;
284 --interface1)
285 export interface=$2
286 shift
287 ;;
288 --interface2)
289 export interface2=$2
290 shift
291 ;;
292 --ip1)
293 export ip1=$2
294 shift
295 ;;
296 --ip2)
297 export ip2=$2
298 shift
299 ;;
300 --repetitions)
301 export num_reps=$2
302 shift
303 ;;
304 --domain)
305 export domain=$2
306 shift
307 ;;
308 --execution-time)
309 export exec_time=$2
310 shift
311 ;;
312 --transport)
313 export transport=$2
314 shift
315 ;;
316 --datalen)
317 export datalen_input=$2
318 shift
319 ;;
320 --file-suffix)
321 export file_suffix=$2
322 shift
323 ;;
324 --folder-suffix)
325 export folder_suffix=$2
326 shift
327 ;;
328 --sub-folder)
329 sub_folder=$2
330 shift
331 ;;
332 --executable-suffix)
333 export executable_suffix=$2
334 shift
335 ;;
336 --extra-arguments)
337 export extra_arguments=$2
338 shift
339 ;;
340 --extra-arguments-pub)
341 export extra_arguments_pub=$2
342 shift
343 ;;
344 --extra-arguments-sub)
345 export extra_arguments_sub=$2
346 shift
347 ;;
348 --skip-be)
349 export skip_be_tests="1"
350 ;;
351 --skip-rel)
352 export skip_rel_tests="1"
353 ;;
354 --skip-keyed)
355 export skip_keyed_data="1"
356 ;;
357 --skip-large-data)
358 export skip_large_data="1"
359 ;;
360 --large-data)
361 export large_data="1"
362 ;;
363 --keyed)
364 export skip_unkeyed="1"
365 ;;
366 --unkeyed)
367 export skip_keyed_data="1"
368 ;;
369 --no-batching | --skip-batching)
370 export run_batching_tests="0"
371 export run_no_batching_tests="1"
372 ;;
373 --skip-no-batching)
374 export run_batching_tests="1"
375 export run_no_batching_tests="0"
376 ;;
377 --batch-size)
378 export batch_size=$2
379 shift
380 ;;
381 --reliable)
382 export skip_be_tests="1"
383 ;;
384 --best-effort)
385 export skip_rel_tests="1"
386 ;;
387 --security-gov)
388 export security_only="$2"
389 shift
390 ;;
391 --micro)
392 export micro="1"
393 ;;
394 --cert)
395 export cert="1"
396 ;;
397 --raw | --raw-transport)
398 export raw="1"
399 ;;
400 --tss)
401 export tss="1"
402 ;;
403 --no-colors)
404 export NO_COLORS="1"
405 ;;
406 --language)
407 export LANGUAGE=$2
408 shift
409 ;;
410 --loss-rate | --nic-loss-rate | --nic-loss-rate-percent)
411 export nic_loss_rate=$2
412 export configure_network="1"
413 shift
414 ;;
415 --delay | --nic-delay-ms | --nic-delay)
416 export nic_delay_ms=$2
417 export configure_network="1"
418 shift
419 ;;
420 --get-netstat-info | --netstat)
421 export get_netstat_info="1"
422 ;;
423 --reduced-data-sizes-set)
424 export REDUCED_DATA_SIZES_SET="1"
425 ;;
426 --dont-change-tuned-settings)
427 export dont_change_tuned_settings="1"
428 ;;
429 --no-taskset)
430 export no_taskset=1
431 ;;
432 --asynchronous | --async)
433 export ASYNC="1"
434 ;;
435 *)
436 echo -e "unknown parameter \"$1\""
437 exit 255
438 ;;
439 esac
440 shift
441done
442
443if [[ "$NO_COLORS" == "1" ]]; then
444 disable_colors
445fi
446
447export folder_base="$(dirname "${executable}")"/../../..
448
449if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
450 export folder_base="$(dirname "${executable}")"/../..
451fi
452if [[ $tss == "1" ]]; then
453 export folder_base="$(dirname "${executable}")"/../../../../..
454fi
455
456if [[ "${executable_suffix}" != "" ]]; then
457 export executable="${executable}${executable_suffix}"
458fi
459
460if [[ -n "${folder_suffix}" ]]; then
461 export output_folder="${output_folder}${folder_suffix}"
462fi
463
464if [[ -n "${sub_folder}" ]]; then
465 export output_folder="${output_folder}/${sub_folder}"
466fi
467
468if [[ "${ASYNC}" == "1" ]]; then
469 export output_folder="${output_folder}/async"
470fi
471
472echo -e "${INFO_TAG} Perftest executable is: $executable"
473echo -e "${INFO_TAG} Output folder is: $output_folder"
474
475################################################################################
476
477if [[ "$LANGUAGE" == "python" ]]; then
478 export skip_keyed_data="1"
479 export skip_large_data="1"
480 export skip_be_tests="1"
481 export run_no_batching_tests="0"
482fi
483
484if [[ "$LANGUAGE" == "java" || "$LANGUAGE" == "cs" ]]; then
485 # Java and C# perftest do not support these execution options.
486 unset thread_cpu_affinity
487 unset thread_priorities
488 unset pin_memory
489 export no_taskset=1
490fi
491
492if [[ "${skip_large_data}" == "1" ]]; then
493 export datasizes_extended=${datasizes}
494elif [[ "${large_data}" == "1" ]]; then
495 export datasizes=${datasizes_extended}
496fi
497
498if [[ "${datalen_input}" != "" ]]; then
499 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
500 export datasizes=${datalen_input}
501 export datasizes_extended=${datalen_input}
502 if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
503 export skip_large_data="1"
504 fi
505else
506 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
507 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
508 export datasizes="32 128 512 2048 8192 32768 63000"
509 export datasizes_extended="${datasizes} 102400 1048576 10485760"
510 fi
511fi
512
513if [[ "$role" != "pub" && "$role" != "sub" ]]; then
514 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
515 exit 255
516fi
517
518if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
519 echo -e "${ERROR_TAG} It must be either lat or thr"
520 exit 255
521fi
522
523if [[ "${interface}" == "" ]]; then
524 echo "Using default nics"
525 export nic_publisher=${ip_machine_1}
526 export nic_subscriber=${ip_machine_2}
527elif [[ "${interface}" == "both" ]]; then
528 export nic_publisher="enp1s0f0,eno1"
529 export nic_subscriber="enp1s0f0,eno1"
530 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
531 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
532else
533 export nic_publisher=$interface
534 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
535
536 if [[ "${interface2}" == "" ]]; then
537 export nic_subscriber=$interface
538 else
539 export nic_subscriber=$interface2
540 fi
541 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
542
543 if [[ "${ip1}" != "" ]]; then
544 export ip_publisher=$ip1
545 echo "Using ip_publisher: ${ip_publisher}"
546 fi
547
548 if [[ "${ip2}" != "" ]]; then
549 export ip_subscriber=$ip2
550 echo "Using ip_subscriber: ${ip_subscriber}"
551 fi
552
553fi
554
555if [[ "$transport" != "" ]]; then
556 export transport_string="-transport $transport"
557
558 if [[ "$transport" == "UDPv4" ]]; then
559
560 export transport_string_pub="$transport_string -nic $nic_publisher"
561 export transport_string_sub="$transport_string -nic $nic_subscriber"
562
563 if [[ "$raw" == "1" ]]; then
564 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
565 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
566 fi
567
568 if [[ "$micro" == "1" || "$cert" == "1" ]]; then
569 export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
570 export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
571 fi
572
573 elif [[ "$transport" == "TCP" ]]; then
574 export transport_string_pub="$transport_string \
575 -nic $nic_publisher \
576 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
577 export transport_string_sub="$transport_string \
578 -nic $nic_subscriber \
579 -peer 0@tcpv4_lan://${ip_publisher}:7400"
580 elif [[ "$transport" == "TLS" ]]; then
581 export transport_string_pub="$transport_string \
582 -nic $nic_publisher \
583 -peer tlsv4_lan://${ip_subscriber}:7400"
584 export transport_string_sub="$transport_string \
585 -nic $nic_subscriber \
586 -peer tlsv4_lan://${ip_publisher}:7400"
587 elif [[ "$transport" == "UDPv4_WAN" ]]; then
588 export transport_string_pub="$transport_string \
589 -nic $nic_publisher \
590 -transportPublicAddress $ip_publisher:7400"
591 export transport_string_sub="$transport_string \
592 -nic $nic_subscriber \
593 -peer 0@udpv4_wan://${ip_publisher}:7400"
594 else
595 export transport_string_pub="$transport_string"
596 export transport_string_sub="$transport_string"
597 fi
598fi
599
600################################################################################
601
602export pub_string="-pub \
603 ${transport_string_pub} \
604 -noPrintIntervals \
605 -executionTime $exec_time"
606
607if [[ ${lat_thr} == "lat" ]]; then
608 export pub_string="$pub_string \
609 -latencyTest"
610fi
611
612if [[ "$role" == "pub" ]]; then
613
614 if [[ "$configure_network" == "1" ]]; then
615 echo -e "${INFO_TAG} Adding -initialBurst 1 to publisher side"
616 export pub_string="$pub_string -initialBurst 1"
617 fi
618
619 if [[ "$ASYNC" == "1" ]]; then
620 export pub_string="${pub_string} -asynchronous"
621 fi
622fi
623
624export sub_string="-sub \
625 ${transport_string_sub} \
626 -noPrintIntervals"
627
628if [[ "$role" == "pub" ]]; then
629 echo -e "$INFO_TAG Publisher side running"
630 export commands_string=${pub_string}
631 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
632else
633 echo -e "$INFO_TAG Subscriber side running"
634 export commands_string=${sub_string}
635 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
636fi
637
638###############################################################################
639
640if [[ "$dont_change_tuned_settings" != "1" ]]; then
641 echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
642 sudo /set_${lat_thr}_mode.sh
643 sleep 5
644fi
645
646if [[ "$role" == "pub" ]]; then
647 export interface_to_configure=$nic_publisher
648else
649 export interface_to_configure=$nic_subscriber
650fi
651
652echo -e "${INFO_TAG} Resetting network interface to default state"
653# Try to delete any existing netem rules (ignore errors if none exist)
654sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
655# Reset to default (pfifo_fast)
656sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
657
658if [[ "${configure_network}" == "1" ]]; then
659 echo -e "${INFO_TAG} Setting loss rate to ${nic_loss_rate}% and delay to ${nic_delay_ms}ms"
660 sudo tc qdisc add dev $interface_to_configure root netem loss ${nic_loss_rate}% delay ${nic_delay_ms}ms limit 10000000
661fi
662
663cd $folder_base
664echo -e "${INFO_TAG} Folder Base is: $PWD"
665mkdir -p $output_folder
666
667if [[ "${batch_size}" != "" ]]; then
668 if [[ "${lat_thr}" == "thr" ]]; then
669 if [[ "${batch_size}" -eq 0 ]]; then
670 echo -e "${INFO_TAG} Batch size is set to 0"
671 export run_batching_tests="0"
672 export run_no_batching_tests="1"
673 else
674 echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
675 export run_batching_tests="1"
676 export run_no_batching_tests="0"
677 fi
678 else
679 echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
680 unset batch_size
681 fi
682fi
683
684# Tests that may use batching (when doing throughput tests). Also Latency tests.
685if [[ ${run_batching_tests} == "1" ]]; then
686
687 # UNKEYED
688 if [[ "${skip_unkeyed}" == "" ]]; then
689
690 # RELIABLE
691 if [[ "${skip_rel_tests}" == "" ]]; then
692 # For the time being, we will skip the throughput reliable tests for CERT.
693 # More info in PERF-1175
694 if ! [[ "${cert}" == "1" && "${lat_thr}" == "thr" && "${transport}" == "UDPv4" ]]; then
695 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
696 else
697 echo -e "${INFO_TAG} Skipping reliable throughput tests for CERT. See PERF-1175 for more information."
698 fi
699 fi
700
701 # BEST EFFORT
702 if [[ "${skip_be_tests}" == "" ]]; then
703 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
704 fi
705 fi
706
707 # KEYED
708 if [[ "${skip_keyed_data}" == "" ]]; then
709
710 # RELIABLE
711 if [[ "${skip_rel_tests}" == "" ]]; then
712 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
713 fi
714
715 # BEST EFFORT
716 if [[ "${skip_be_tests}" == "" ]]; then
717 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
718 fi
719 fi
720
721fi
722
723# Tests that will not use batching
724if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then
725
726 if [[ "$role" == "pub" ]]; then
727 export commands_string="${commands_string} -batchSize 0"
728 fi
729
730 # UNKEYED
731 if [[ "${skip_unkeyed}" == "" ]]; then
732
733 # RELIABLE
734 if [[ "${skip_rel_tests}" == "" ]]; then
735 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
736 fi
737
738 # BEST EFFORT
739 if [[ "${skip_be_tests}" == "" ]]; then
740 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
741 fi
742 fi
743
744 # KEYED
745 if [[ "${skip_keyed_data}" == "" ]]; then
746
747 # RELIABLE
748 if [[ "${skip_rel_tests}" == "" ]]; then
749 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
750 fi
751
752 # BEST EFFORT
753 if [[ "${skip_be_tests}" == "" ]]; then
754 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
755 fi
756 fi
757
758fi
759
760# Replace the cleanup section at the end:
761if [[ "${configure_network}" == "1" ]]; then
762 echo -e "${INFO_TAG} Resetting network interface to default state"
763 sudo tc qdisc del dev $interface_to_configure root netem 2>/dev/null || true
764 sudo tc qdisc del dev $interface_to_configure root 2>/dev/null || true
765fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GCM" \
10 --executable-suffix "_lws" \
11 --file-suffix "_AES256_GCM"
12
13"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
14 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
15 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES128+GCM" \
16 --executable-suffix "_lws" \
17 --file-suffix "_AES128_GCM"
18
19# "${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
20# --skip-no-batching --skip-be --skip-keyed --skip-large-data \
21# --extra-arguments "-securePSK str:1:SecretKey -securePSKAlgorithm AES256+GCM" \
22# --executable-suffix "_lws" \
23# --file-suffix "_AES256_GCM"
24
25"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
26 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
27 --extra-arguments "-securePSK data:,1:SecretKey -securePSKAlgorithm AES256+GMAC" \
28 --executable-suffix "_lws" \
29 --file-suffix "_AES256_GMAC"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4_WAN \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-secureRtpsHmacOnly str:JaviTheBest"
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7"${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
8 --skip-no-batching --skip-be --skip-keyed --skip-large-data \
9 --extra-arguments "-securePSK data:,1:SecretKey -secureGovernanceFile resource/secure/signed_PerftestGovernance_RtpsNoneDiscoveryNonePresharedProtectionSign.xml"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)