6.1.1.2. Compression
The user-data compression feature allows a DataWriter to compress the samples that it sends to matching DataReaders. Only the user payload is compressed, not the RTPS protocol headers. This compression feature can be configured using the compression_settings in the DATA_REPRESENTATION QoS Policy. For more information, see the “Data Compression” section in DATA_REPRESENTATION QosPolicy, in the RTI Connext Core Libraries User’s Manual.
The following tests have been performed by executing an RTI Perftest C++98 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. Compression has been enabled with different combinations of settings, the communication has been restricted to a single interface, and the transport has been set to UDPv4.
For latency and throughput tests, the compression threshold is set to the default value (8192 bytes), so compression is disabled for sizes lower than this value. We assume that the performance for small sample sizes usually does not improve in terms of latency or throughput, but it still can be beneficial for reducing bandwidth utilization.
The data sent and used on these tests to measure the performance of the compression feature is text data corresponding to the logs generated by Connext in a real customer scenario. We parse the data and crop it into samples of different sizes. The gathered results show a lower performance than is possible, but a more realistic one. Keep in mind that the type of data sent has a huge impact on the compression algorithm behavior.
The network used for this test is the 1Gbps network where this feature is able to boost the throughput beyond the hardware limitation of 1Gbps.
Find information about the hardware, network, and command-line parameters after each of the tests.
Compression LZ4, Unkeyed, UDPv4, C++98
The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes connected locally in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: LZ4
writer_level: 5
writer_threshold: 8192 (default)
Detailed Statistics
The following table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
|---|---|---|---|---|---|---|---|---|---|
512 |
71 |
1.3 |
64 |
131 |
71 |
72 |
74 |
99 |
131 |
1024 |
83 |
2.2 |
67 |
140 |
84 |
85 |
87 |
112 |
140 |
2048 |
104 |
6.4 |
70 |
165 |
105 |
111 |
113 |
135 |
165 |
4096 |
130 |
15.0 |
71 |
258 |
132 |
148 |
157 |
173 |
258 |
8192 |
164 |
28.6 |
72 |
248 |
169 |
198 |
216 |
238 |
248 |
16384 |
215 |
48.3 |
79 |
314 |
227 |
269 |
299 |
310 |
314 |
32768 |
278 |
60.7 |
93 |
384 |
315 |
325 |
334 |
375 |
384 |
63000 |
340 |
61.0 |
119 |
475 |
363 |
391 |
414 |
426 |
475 |
Best Effort
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
|---|---|---|---|---|---|---|---|---|---|
512 |
69 |
1.4 |
62 |
173 |
68 |
70 |
72 |
98 |
173 |
1024 |
82 |
2.1 |
67 |
138 |
82 |
83 |
84 |
111 |
138 |
2048 |
102 |
6.3 |
67 |
160 |
103 |
109 |
111 |
134 |
160 |
4096 |
128 |
14.9 |
70 |
212 |
130 |
146 |
155 |
171 |
212 |
8192 |
162 |
28.6 |
70 |
239 |
168 |
196 |
214 |
236 |
239 |
16384 |
213 |
48.3 |
79 |
310 |
225 |
267 |
297 |
307 |
310 |
32768 |
276 |
60.7 |
90 |
384 |
313 |
323 |
332 |
371 |
384 |
63000 |
338 |
61.0 |
116 |
483 |
361 |
389 |
412 |
425 |
483 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25################################################################################
26
27function disable_colors() {
28 export RED=""
29 export GREEN=""
30 export YELLOW=""
31 export NC=""
32 export BLUE=""
33 export LIGHTBLUE=""
34 export INFO_TAG="${GREEN}[INFO]:${NC}"
35 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
36 export ERROR_TAG="${RED}[ERROR]:${NC}"
37}
38
39function change_domain() {
40 if [[ "$domain" == "1" ]]; then
41 export domain="2"
42 else
43 export domain="1"
44 fi
45}
46
47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
48function execute_test() {
49
50 local keyed_unkeyed=$1
51 local rel_be=$2
52 local datasizes_test=$3
53 local other_args=$4
54 local name_suffix=$5
55
56 local commands_string_test=$commands_string
57 local tag=""
58
59 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
60 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
61 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
62 else
63 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
64 fi
65
66 if [[ "${rel_be}" == "be" ]]; then
67 commands_string_test="${commands_string_test} -bestEffort"
68 tag="${tag}${YELLOW}BE${NC}]"
69 else
70 tag="${tag}${RED}REL${NC}]"
71 fi
72
73 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
74
75 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
76
77 if [[ "$role" == "pub" ]]; then
78 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
79 fi
80
81 if [[ "$NO_TASKSET" == "" ]]; then
82 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
83 export pre_command_string="taskset -c $core"
84 fi
85 fi
86
87 if [[ "$LANGUAGE" == "python" ]]; then
88 export pre_command_string="python3 "
89 fi
90
91 if [[ "$DOCKER" == "1" ]]; then
92 export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
93 executable=""
94 fi
95
96 # Get the aprox time this will take:
97 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
98 total_time=$((total_tests * exec_time))
99
100 touch $output_file
101 local no_headers=""
102 local current_test=0
103 for index in $(seq 1 ${num_reps}); do
104 for DATALEN in ${datasizes_test}; do
105 current_test=$((current_test + 1))
106 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107 if [[ "$role" == "pub" ]]; then
108 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109 echo -e ${BLUE}$command${NC}
110 else
111 echo -e ${LIGHTBLUE}$command${NC}
112 fi
113 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114 sleep 3
115 fi
116 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117 sleep 5
118 fi
119 if [[ "${get_netstat_info}" == "1" ]]; then
120 echo -e "${INFO_TAG} Getting netstat info before"
121 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122 fi
123 eval $command >> $output_file;
124 if [[ "${get_netstat_info}" == "1" ]]; then
125 echo -e "${INFO_TAG} Getting netstat info after"
126 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128 python3 $script_location/../../../tools/diff_netstat_output.py \
129 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131 -d $DATALEN $no_header_netstat \
132 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134 no_header_netstat=" -nh"
135 fi
136 no_headers=" -noOutputHeaders"
137
138 change_domain
139 done
140 done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147 case $1 in
148 --executable)
149 executable=$2
150 shift
151 ;;
152 --docker)
153 DOCKER="1"
154 ;;
155 --output-folder)
156 output_folder=$2
157 shift
158 ;;
159 --sub-folder)
160 sub_folder=$2
161 shift
162 ;;
163 --role)
164 export role=$2
165 shift
166 ;;
167 --core)
168 export core=$2
169 shift
170 ;;
171 --test-kind)
172 export lat_thr=$2
173 shift
174 ;;
175 --interface1)
176 export interface=$2
177 shift
178 ;;
179 --interface2)
180 export interface2=$2
181 shift
182 ;;
183 --ip1)
184 export ip1=$2
185 shift
186 ;;
187 --ip2)
188 export ip2=$2
189 shift
190 ;;
191 --repetitions)
192 export num_reps=$2
193 shift
194 ;;
195 --domain)
196 export domain=$2
197 shift
198 ;;
199 --execution-time)
200 export exec_time=$2
201 shift
202 ;;
203 --transport)
204 export transport=$2
205 shift
206 ;;
207 --datalen)
208 export datalen_input=$2
209 shift
210 ;;
211 --file-suffix)
212 export file_suffix=$2
213 shift
214 ;;
215 --executable-suffix)
216 export executable_suffix=$2
217 shift
218 ;;
219 --extra-arguments)
220 export extra_arguments=$2
221 shift
222 ;;
223 --extra-arguments-pub)
224 export extra_arguments_pub=$2
225 shift
226 ;;
227 --extra-arguments-sub)
228 export extra_arguments_sub=$2
229 shift
230 ;;
231 --skip-no-batching)
232 export skip_no_batching="1"
233 ;;
234 --skip-be)
235 export skip_be_tests="1"
236 ;;
237 --skip-rel)
238 export skip_rel_tests="1"
239 ;;
240 --skip-keyed)
241 export skip_keyed_data="1"
242 ;;
243 --skip-large-data)
244 export skip_large_data="1"
245 ;;
246 --large-data)
247 export large_data="1"
248 ;;
249 --keyed)
250 export skip_unkeyed="1"
251 ;;
252 --unkeyed)
253 export skip_keyed_data="1"
254 ;;
255 --no-batching | --skip-batching)
256 export no_batching_only="1"
257 ;;
258 --reliable)
259 export skip_be_tests="1"
260 ;;
261 --best-effort)
262 export skip_rel_tests="1"
263 ;;
264 --security-gov)
265 export security_only="$2"
266 shift
267 ;;
268 --micro)
269 export micro="1"
270 ;;
271 --cert)
272 export cert="1"
273 ;;
274 --raw | --raw-transport)
275 export raw="1"
276 ;;
277 --tss)
278 export tss="1"
279 ;;
280 --no-colors)
281 export NO_COLORS="1"
282 ;;
283 --language)
284 export LANGUAGE=$2
285 shift
286 ;;
287 --loss-rate)
288 export loss_rate=$2
289 shift
290 ;;
291 --get-netstat-info | --netstat)
292 export get_netstat_info="1"
293 ;;
294 --no-taskset)
295 export NO_TASKSET="1"
296 ;;
297 --reduced-data-sizes-set)
298 export REDUCED_DATA_SIZES_SET="1"
299 ;;
300 *)
301 echo -e "unknown parameter \"$1\""
302 exit 255
303 ;;
304 esac
305 shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309 disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
315 export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318 export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322 export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326 export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335 export skip_keyed_data="1"
336 export skip_large_data="1"
337 export skip_be_tests="1"
338 export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342 export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344 export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349 export datasizes=${datalen_input}
350 export datasizes_extended=${datalen_input}
351 if [[ "${no_batching_only}" != "1" ]]; then
352 export skip_large_data="1"
353 fi
354else
355 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357 export datasizes="32 128 512 2048 8192 32768 63000"
358 export datasizes_extended="${datasizes} 102400 1048576 10485760"
359 fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364 exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368 echo -e "${ERROR_TAG} It must be either lat or thr"
369 exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373 echo "Using default nics"
374 export nic_publisher=${ip_machine_1}
375 export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377 export nic_publisher="enp1s0f0,eno1"
378 export nic_subscriber="enp1s0f0,eno1"
379 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382 export nic_publisher=$interface
383 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385 if [[ "${interface2}" == "" ]]; then
386 export nic_subscriber=$interface
387 else
388 export nic_subscriber=$interface2
389 fi
390 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392 if [[ "${ip1}" != "" ]]; then
393 export ip_publisher=$ip1
394 echo "Using ip_publisher: ${ip_publisher}"
395 fi
396
397 if [[ "${ip2}" != "" ]]; then
398 export ip_subscriber=$ip2
399 echo "Using ip_subscriber: ${ip_subscriber}"
400 fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408 export transport_string_pub="$transport_string -nic $nic_publisher"
409 export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411 if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414 fi
415
416elif [[ "$transport" == "TCP" ]]; then
417 export transport_string_pub="$transport_string \
418 -nic $nic_publisher \
419 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420 export transport_string_sub="$transport_string \
421 -nic $nic_subscriber \
422 -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424 export transport_string_pub="$transport_string \
425 -nic $nic_publisher \
426 -peer tlsv4_lan://${ip_subscriber}:7400"
427 export transport_string_sub="$transport_string \
428 -nic $nic_subscriber \
429 -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431 export transport_string_pub="$transport_string \
432 -nic $nic_publisher \
433 -transportPublicAddress $ip_publisher:7400"
434 export transport_string_sub="$transport_string \
435 -nic $nic_subscriber \
436 -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438 export transport_string_pub="$transport_string"
439 export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445 ${transport_string_pub} \
446 -noPrintIntervals \
447 -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450 export pub_string="$pub_string \
451 -latencyTest"
452fi
453
454export sub_string="-sub \
455 ${transport_string_sub} \
456 -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459 echo -e "$INFO_TAG Publisher side running"
460 export commands_string=${pub_string}
461 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463 echo -e "$INFO_TAG Subscriber side running"
464 export commands_string=${sub_string}
465 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490 # UNKEYED
491 if [[ "${skip_unkeyed}" == "" ]]; then
492
493 # RELIABLE
494 if [[ "${skip_rel_tests}" == "" ]]; then
495 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496 fi
497
498 # BEST EFFORT
499 if [[ "${skip_be_tests}" == "" ]]; then
500 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501 fi
502 fi
503
504 # KEYED
505 if [[ "${skip_keyed_data}" == "" ]]; then
506
507 # RELIABLE
508 if [[ "${skip_rel_tests}" == "" ]]; then
509 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510 fi
511
512 # BEST EFFORT
513 if [[ "${skip_be_tests}" == "" ]]; then
514 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515 fi
516 fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521 no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527 if [[ "$role" == "pub" ]]; then
528 export commands_string="${commands_string} -batchSize 0"
529 fi
530
531 # UNKEYED
532 if [[ "${skip_unkeyed}" == "" ]]; then
533
534 # RELIABLE
535 if [[ "${skip_rel_tests}" == "" ]]; then
536 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537 fi
538
539 # BEST EFFORT
540 if [[ "${skip_be_tests}" == "" ]]; then
541 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542 fi
543 fi
544
545 # KEYED
546 if [[ "${skip_keyed_data}" == "" ]]; then
547
548 # RELIABLE
549 if [[ "${skip_rel_tests}" == "" ]]; then
550 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551 fi
552
553 # BEST EFFORT
554 if [[ "${skip_be_tests}" == "" ]]; then
555 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556 fi
557 fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562 echo -e "${INFO_TAG} Disabling loss rate"
563 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: LZ4
writer_level: 5
writer_threshold: 8192 (default)
Note
By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.
LZ4 is not compatible with batching. Therefore, we are comparing batched regular data with un-batched compressed data. This explains some of the differences in throughput for sizes lower than 8192 bytes.
The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.
Detailed Statistics
The following table contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
|---|---|---|---|---|---|
512 |
3916138 |
195803 |
802.0 |
0 |
0.00 |
1024 |
2253601 |
112677 |
923.1 |
0 |
0.00 |
2048 |
1356171 |
67806 |
1110.9 |
0 |
0.00 |
4096 |
890076 |
44501 |
1458.2 |
0 |
0.00 |
8192 |
620691 |
31031 |
2033.7 |
0 |
0.00 |
16384 |
431211 |
21557 |
2825.6 |
0 |
0.00 |
32768 |
173157 |
8656 |
2269.2 |
0 |
0.00 |
63000 |
109984 |
5497 |
2770.7 |
0 |
0.00 |
Best Effort
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
|---|---|---|---|---|---|
512 |
3942782 |
196893 |
806.5 |
0 |
0.00 |
1024 |
2261667 |
113006 |
925.7 |
0 |
0.00 |
2048 |
1359233 |
67926 |
1112.9 |
0 |
0.00 |
4096 |
891577 |
44551 |
1459.9 |
0 |
0.00 |
8192 |
621361 |
31054 |
2035.2 |
0 |
0.00 |
16384 |
243325 |
12156 |
1593.4 |
188474 |
43.65 |
32768 |
128879 |
6441 |
1688.6 |
162784 |
55.81 |
63000 |
105272 |
5260 |
2651.5 |
91597 |
46.53 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25################################################################################
26
27function disable_colors() {
28 export RED=""
29 export GREEN=""
30 export YELLOW=""
31 export NC=""
32 export BLUE=""
33 export LIGHTBLUE=""
34 export INFO_TAG="${GREEN}[INFO]:${NC}"
35 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
36 export ERROR_TAG="${RED}[ERROR]:${NC}"
37}
38
39function change_domain() {
40 if [[ "$domain" == "1" ]]; then
41 export domain="2"
42 else
43 export domain="1"
44 fi
45}
46
47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
48function execute_test() {
49
50 local keyed_unkeyed=$1
51 local rel_be=$2
52 local datasizes_test=$3
53 local other_args=$4
54 local name_suffix=$5
55
56 local commands_string_test=$commands_string
57 local tag=""
58
59 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
60 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
61 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
62 else
63 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
64 fi
65
66 if [[ "${rel_be}" == "be" ]]; then
67 commands_string_test="${commands_string_test} -bestEffort"
68 tag="${tag}${YELLOW}BE${NC}]"
69 else
70 tag="${tag}${RED}REL${NC}]"
71 fi
72
73 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
74
75 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
76
77 if [[ "$role" == "pub" ]]; then
78 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
79 fi
80
81 if [[ "$NO_TASKSET" == "" ]]; then
82 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
83 export pre_command_string="taskset -c $core"
84 fi
85 fi
86
87 if [[ "$LANGUAGE" == "python" ]]; then
88 export pre_command_string="python3 "
89 fi
90
91 if [[ "$DOCKER" == "1" ]]; then
92 export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
93 executable=""
94 fi
95
96 # Get the aprox time this will take:
97 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
98 total_time=$((total_tests * exec_time))
99
100 touch $output_file
101 local no_headers=""
102 local current_test=0
103 for index in $(seq 1 ${num_reps}); do
104 for DATALEN in ${datasizes_test}; do
105 current_test=$((current_test + 1))
106 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107 if [[ "$role" == "pub" ]]; then
108 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109 echo -e ${BLUE}$command${NC}
110 else
111 echo -e ${LIGHTBLUE}$command${NC}
112 fi
113 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114 sleep 3
115 fi
116 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117 sleep 5
118 fi
119 if [[ "${get_netstat_info}" == "1" ]]; then
120 echo -e "${INFO_TAG} Getting netstat info before"
121 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122 fi
123 eval $command >> $output_file;
124 if [[ "${get_netstat_info}" == "1" ]]; then
125 echo -e "${INFO_TAG} Getting netstat info after"
126 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128 python3 $script_location/../../../tools/diff_netstat_output.py \
129 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131 -d $DATALEN $no_header_netstat \
132 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134 no_header_netstat=" -nh"
135 fi
136 no_headers=" -noOutputHeaders"
137
138 change_domain
139 done
140 done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147 case $1 in
148 --executable)
149 executable=$2
150 shift
151 ;;
152 --docker)
153 DOCKER="1"
154 ;;
155 --output-folder)
156 output_folder=$2
157 shift
158 ;;
159 --sub-folder)
160 sub_folder=$2
161 shift
162 ;;
163 --role)
164 export role=$2
165 shift
166 ;;
167 --core)
168 export core=$2
169 shift
170 ;;
171 --test-kind)
172 export lat_thr=$2
173 shift
174 ;;
175 --interface1)
176 export interface=$2
177 shift
178 ;;
179 --interface2)
180 export interface2=$2
181 shift
182 ;;
183 --ip1)
184 export ip1=$2
185 shift
186 ;;
187 --ip2)
188 export ip2=$2
189 shift
190 ;;
191 --repetitions)
192 export num_reps=$2
193 shift
194 ;;
195 --domain)
196 export domain=$2
197 shift
198 ;;
199 --execution-time)
200 export exec_time=$2
201 shift
202 ;;
203 --transport)
204 export transport=$2
205 shift
206 ;;
207 --datalen)
208 export datalen_input=$2
209 shift
210 ;;
211 --file-suffix)
212 export file_suffix=$2
213 shift
214 ;;
215 --executable-suffix)
216 export executable_suffix=$2
217 shift
218 ;;
219 --extra-arguments)
220 export extra_arguments=$2
221 shift
222 ;;
223 --extra-arguments-pub)
224 export extra_arguments_pub=$2
225 shift
226 ;;
227 --extra-arguments-sub)
228 export extra_arguments_sub=$2
229 shift
230 ;;
231 --skip-no-batching)
232 export skip_no_batching="1"
233 ;;
234 --skip-be)
235 export skip_be_tests="1"
236 ;;
237 --skip-rel)
238 export skip_rel_tests="1"
239 ;;
240 --skip-keyed)
241 export skip_keyed_data="1"
242 ;;
243 --skip-large-data)
244 export skip_large_data="1"
245 ;;
246 --large-data)
247 export large_data="1"
248 ;;
249 --keyed)
250 export skip_unkeyed="1"
251 ;;
252 --unkeyed)
253 export skip_keyed_data="1"
254 ;;
255 --no-batching | --skip-batching)
256 export no_batching_only="1"
257 ;;
258 --reliable)
259 export skip_be_tests="1"
260 ;;
261 --best-effort)
262 export skip_rel_tests="1"
263 ;;
264 --security-gov)
265 export security_only="$2"
266 shift
267 ;;
268 --micro)
269 export micro="1"
270 ;;
271 --cert)
272 export cert="1"
273 ;;
274 --raw | --raw-transport)
275 export raw="1"
276 ;;
277 --tss)
278 export tss="1"
279 ;;
280 --no-colors)
281 export NO_COLORS="1"
282 ;;
283 --language)
284 export LANGUAGE=$2
285 shift
286 ;;
287 --loss-rate)
288 export loss_rate=$2
289 shift
290 ;;
291 --get-netstat-info | --netstat)
292 export get_netstat_info="1"
293 ;;
294 --no-taskset)
295 export NO_TASKSET="1"
296 ;;
297 --reduced-data-sizes-set)
298 export REDUCED_DATA_SIZES_SET="1"
299 ;;
300 *)
301 echo -e "unknown parameter \"$1\""
302 exit 255
303 ;;
304 esac
305 shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309 disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
315 export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318 export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322 export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326 export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335 export skip_keyed_data="1"
336 export skip_large_data="1"
337 export skip_be_tests="1"
338 export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342 export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344 export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349 export datasizes=${datalen_input}
350 export datasizes_extended=${datalen_input}
351 if [[ "${no_batching_only}" != "1" ]]; then
352 export skip_large_data="1"
353 fi
354else
355 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357 export datasizes="32 128 512 2048 8192 32768 63000"
358 export datasizes_extended="${datasizes} 102400 1048576 10485760"
359 fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364 exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368 echo -e "${ERROR_TAG} It must be either lat or thr"
369 exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373 echo "Using default nics"
374 export nic_publisher=${ip_machine_1}
375 export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377 export nic_publisher="enp1s0f0,eno1"
378 export nic_subscriber="enp1s0f0,eno1"
379 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382 export nic_publisher=$interface
383 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385 if [[ "${interface2}" == "" ]]; then
386 export nic_subscriber=$interface
387 else
388 export nic_subscriber=$interface2
389 fi
390 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392 if [[ "${ip1}" != "" ]]; then
393 export ip_publisher=$ip1
394 echo "Using ip_publisher: ${ip_publisher}"
395 fi
396
397 if [[ "${ip2}" != "" ]]; then
398 export ip_subscriber=$ip2
399 echo "Using ip_subscriber: ${ip_subscriber}"
400 fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408 export transport_string_pub="$transport_string -nic $nic_publisher"
409 export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411 if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414 fi
415
416elif [[ "$transport" == "TCP" ]]; then
417 export transport_string_pub="$transport_string \
418 -nic $nic_publisher \
419 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420 export transport_string_sub="$transport_string \
421 -nic $nic_subscriber \
422 -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424 export transport_string_pub="$transport_string \
425 -nic $nic_publisher \
426 -peer tlsv4_lan://${ip_subscriber}:7400"
427 export transport_string_sub="$transport_string \
428 -nic $nic_subscriber \
429 -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431 export transport_string_pub="$transport_string \
432 -nic $nic_publisher \
433 -transportPublicAddress $ip_publisher:7400"
434 export transport_string_sub="$transport_string \
435 -nic $nic_subscriber \
436 -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438 export transport_string_pub="$transport_string"
439 export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445 ${transport_string_pub} \
446 -noPrintIntervals \
447 -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450 export pub_string="$pub_string \
451 -latencyTest"
452fi
453
454export sub_string="-sub \
455 ${transport_string_sub} \
456 -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459 echo -e "$INFO_TAG Publisher side running"
460 export commands_string=${pub_string}
461 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463 echo -e "$INFO_TAG Subscriber side running"
464 export commands_string=${sub_string}
465 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490 # UNKEYED
491 if [[ "${skip_unkeyed}" == "" ]]; then
492
493 # RELIABLE
494 if [[ "${skip_rel_tests}" == "" ]]; then
495 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496 fi
497
498 # BEST EFFORT
499 if [[ "${skip_be_tests}" == "" ]]; then
500 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501 fi
502 fi
503
504 # KEYED
505 if [[ "${skip_keyed_data}" == "" ]]; then
506
507 # RELIABLE
508 if [[ "${skip_rel_tests}" == "" ]]; then
509 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510 fi
511
512 # BEST EFFORT
513 if [[ "${skip_be_tests}" == "" ]]; then
514 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515 fi
516 fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521 no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527 if [[ "$role" == "pub" ]]; then
528 export commands_string="${commands_string} -batchSize 0"
529 fi
530
531 # UNKEYED
532 if [[ "${skip_unkeyed}" == "" ]]; then
533
534 # RELIABLE
535 if [[ "${skip_rel_tests}" == "" ]]; then
536 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537 fi
538
539 # BEST EFFORT
540 if [[ "${skip_be_tests}" == "" ]]; then
541 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542 fi
543 fi
544
545 # KEYED
546 if [[ "${skip_keyed_data}" == "" ]]; then
547
548 # RELIABLE
549 if [[ "${skip_rel_tests}" == "" ]]; then
550 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551 fi
552
553 # BEST EFFORT
554 if [[ "${skip_be_tests}" == "" ]]; then
555 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556 fi
557 fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562 echo -e "${INFO_TAG} Disabling loss rate"
563 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Compression ZLIB, Unkeyed, UDPv4, C++98
The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes connected locally in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: ZLIB
writer_level: 5
writer_threshold: 8192 (default)
Detailed Statistics
The following table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
|---|---|---|---|---|---|---|---|---|---|
512 |
122 |
3.1 |
98 |
185 |
122 |
125 |
130 |
154 |
185 |
1024 |
131 |
6.0 |
100 |
267 |
130 |
138 |
143 |
168 |
267 |
2048 |
145 |
9.2 |
107 |
205 |
144 |
156 |
166 |
186 |
205 |
4096 |
170 |
15.6 |
120 |
291 |
171 |
189 |
206 |
235 |
291 |
8192 |
211 |
26.6 |
140 |
307 |
215 |
243 |
266 |
299 |
307 |
16384 |
288 |
43.6 |
189 |
460 |
300 |
337 |
372 |
443 |
460 |
32768 |
432 |
70.9 |
262 |
692 |
453 |
506 |
546 |
660 |
692 |
63000 |
685 |
112.3 |
395 |
977 |
727 |
794 |
848 |
953 |
977 |
Best Effort
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
|---|---|---|---|---|---|---|---|---|---|
512 |
119 |
2.9 |
95 |
183 |
120 |
122 |
126 |
152 |
183 |
1024 |
129 |
5.9 |
98 |
198 |
128 |
136 |
141 |
165 |
198 |
2048 |
143 |
9.2 |
105 |
239 |
142 |
154 |
163 |
186 |
239 |
4096 |
168 |
15.3 |
118 |
268 |
168 |
187 |
201 |
222 |
268 |
8192 |
209 |
26.3 |
138 |
296 |
213 |
240 |
260 |
294 |
296 |
16384 |
286 |
43.5 |
186 |
442 |
298 |
334 |
369 |
439 |
442 |
32768 |
431 |
71.7 |
261 |
668 |
453 |
506 |
543 |
667 |
668 |
63000 |
682 |
112.1 |
394 |
950 |
724 |
790 |
843 |
948 |
950 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25################################################################################
26
27function disable_colors() {
28 export RED=""
29 export GREEN=""
30 export YELLOW=""
31 export NC=""
32 export BLUE=""
33 export LIGHTBLUE=""
34 export INFO_TAG="${GREEN}[INFO]:${NC}"
35 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
36 export ERROR_TAG="${RED}[ERROR]:${NC}"
37}
38
39function change_domain() {
40 if [[ "$domain" == "1" ]]; then
41 export domain="2"
42 else
43 export domain="1"
44 fi
45}
46
47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
48function execute_test() {
49
50 local keyed_unkeyed=$1
51 local rel_be=$2
52 local datasizes_test=$3
53 local other_args=$4
54 local name_suffix=$5
55
56 local commands_string_test=$commands_string
57 local tag=""
58
59 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
60 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
61 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
62 else
63 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
64 fi
65
66 if [[ "${rel_be}" == "be" ]]; then
67 commands_string_test="${commands_string_test} -bestEffort"
68 tag="${tag}${YELLOW}BE${NC}]"
69 else
70 tag="${tag}${RED}REL${NC}]"
71 fi
72
73 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
74
75 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
76
77 if [[ "$role" == "pub" ]]; then
78 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
79 fi
80
81 if [[ "$NO_TASKSET" == "" ]]; then
82 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
83 export pre_command_string="taskset -c $core"
84 fi
85 fi
86
87 if [[ "$LANGUAGE" == "python" ]]; then
88 export pre_command_string="python3 "
89 fi
90
91 if [[ "$DOCKER" == "1" ]]; then
92 export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
93 executable=""
94 fi
95
96 # Get the aprox time this will take:
97 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
98 total_time=$((total_tests * exec_time))
99
100 touch $output_file
101 local no_headers=""
102 local current_test=0
103 for index in $(seq 1 ${num_reps}); do
104 for DATALEN in ${datasizes_test}; do
105 current_test=$((current_test + 1))
106 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107 if [[ "$role" == "pub" ]]; then
108 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109 echo -e ${BLUE}$command${NC}
110 else
111 echo -e ${LIGHTBLUE}$command${NC}
112 fi
113 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114 sleep 3
115 fi
116 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117 sleep 5
118 fi
119 if [[ "${get_netstat_info}" == "1" ]]; then
120 echo -e "${INFO_TAG} Getting netstat info before"
121 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122 fi
123 eval $command >> $output_file;
124 if [[ "${get_netstat_info}" == "1" ]]; then
125 echo -e "${INFO_TAG} Getting netstat info after"
126 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128 python3 $script_location/../../../tools/diff_netstat_output.py \
129 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131 -d $DATALEN $no_header_netstat \
132 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134 no_header_netstat=" -nh"
135 fi
136 no_headers=" -noOutputHeaders"
137
138 change_domain
139 done
140 done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147 case $1 in
148 --executable)
149 executable=$2
150 shift
151 ;;
152 --docker)
153 DOCKER="1"
154 ;;
155 --output-folder)
156 output_folder=$2
157 shift
158 ;;
159 --sub-folder)
160 sub_folder=$2
161 shift
162 ;;
163 --role)
164 export role=$2
165 shift
166 ;;
167 --core)
168 export core=$2
169 shift
170 ;;
171 --test-kind)
172 export lat_thr=$2
173 shift
174 ;;
175 --interface1)
176 export interface=$2
177 shift
178 ;;
179 --interface2)
180 export interface2=$2
181 shift
182 ;;
183 --ip1)
184 export ip1=$2
185 shift
186 ;;
187 --ip2)
188 export ip2=$2
189 shift
190 ;;
191 --repetitions)
192 export num_reps=$2
193 shift
194 ;;
195 --domain)
196 export domain=$2
197 shift
198 ;;
199 --execution-time)
200 export exec_time=$2
201 shift
202 ;;
203 --transport)
204 export transport=$2
205 shift
206 ;;
207 --datalen)
208 export datalen_input=$2
209 shift
210 ;;
211 --file-suffix)
212 export file_suffix=$2
213 shift
214 ;;
215 --executable-suffix)
216 export executable_suffix=$2
217 shift
218 ;;
219 --extra-arguments)
220 export extra_arguments=$2
221 shift
222 ;;
223 --extra-arguments-pub)
224 export extra_arguments_pub=$2
225 shift
226 ;;
227 --extra-arguments-sub)
228 export extra_arguments_sub=$2
229 shift
230 ;;
231 --skip-no-batching)
232 export skip_no_batching="1"
233 ;;
234 --skip-be)
235 export skip_be_tests="1"
236 ;;
237 --skip-rel)
238 export skip_rel_tests="1"
239 ;;
240 --skip-keyed)
241 export skip_keyed_data="1"
242 ;;
243 --skip-large-data)
244 export skip_large_data="1"
245 ;;
246 --large-data)
247 export large_data="1"
248 ;;
249 --keyed)
250 export skip_unkeyed="1"
251 ;;
252 --unkeyed)
253 export skip_keyed_data="1"
254 ;;
255 --no-batching | --skip-batching)
256 export no_batching_only="1"
257 ;;
258 --reliable)
259 export skip_be_tests="1"
260 ;;
261 --best-effort)
262 export skip_rel_tests="1"
263 ;;
264 --security-gov)
265 export security_only="$2"
266 shift
267 ;;
268 --micro)
269 export micro="1"
270 ;;
271 --cert)
272 export cert="1"
273 ;;
274 --raw | --raw-transport)
275 export raw="1"
276 ;;
277 --tss)
278 export tss="1"
279 ;;
280 --no-colors)
281 export NO_COLORS="1"
282 ;;
283 --language)
284 export LANGUAGE=$2
285 shift
286 ;;
287 --loss-rate)
288 export loss_rate=$2
289 shift
290 ;;
291 --get-netstat-info | --netstat)
292 export get_netstat_info="1"
293 ;;
294 --no-taskset)
295 export NO_TASKSET="1"
296 ;;
297 --reduced-data-sizes-set)
298 export REDUCED_DATA_SIZES_SET="1"
299 ;;
300 *)
301 echo -e "unknown parameter \"$1\""
302 exit 255
303 ;;
304 esac
305 shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309 disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
315 export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318 export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322 export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326 export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335 export skip_keyed_data="1"
336 export skip_large_data="1"
337 export skip_be_tests="1"
338 export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342 export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344 export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349 export datasizes=${datalen_input}
350 export datasizes_extended=${datalen_input}
351 if [[ "${no_batching_only}" != "1" ]]; then
352 export skip_large_data="1"
353 fi
354else
355 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357 export datasizes="32 128 512 2048 8192 32768 63000"
358 export datasizes_extended="${datasizes} 102400 1048576 10485760"
359 fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364 exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368 echo -e "${ERROR_TAG} It must be either lat or thr"
369 exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373 echo "Using default nics"
374 export nic_publisher=${ip_machine_1}
375 export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377 export nic_publisher="enp1s0f0,eno1"
378 export nic_subscriber="enp1s0f0,eno1"
379 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382 export nic_publisher=$interface
383 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385 if [[ "${interface2}" == "" ]]; then
386 export nic_subscriber=$interface
387 else
388 export nic_subscriber=$interface2
389 fi
390 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392 if [[ "${ip1}" != "" ]]; then
393 export ip_publisher=$ip1
394 echo "Using ip_publisher: ${ip_publisher}"
395 fi
396
397 if [[ "${ip2}" != "" ]]; then
398 export ip_subscriber=$ip2
399 echo "Using ip_subscriber: ${ip_subscriber}"
400 fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408 export transport_string_pub="$transport_string -nic $nic_publisher"
409 export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411 if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414 fi
415
416elif [[ "$transport" == "TCP" ]]; then
417 export transport_string_pub="$transport_string \
418 -nic $nic_publisher \
419 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420 export transport_string_sub="$transport_string \
421 -nic $nic_subscriber \
422 -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424 export transport_string_pub="$transport_string \
425 -nic $nic_publisher \
426 -peer tlsv4_lan://${ip_subscriber}:7400"
427 export transport_string_sub="$transport_string \
428 -nic $nic_subscriber \
429 -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431 export transport_string_pub="$transport_string \
432 -nic $nic_publisher \
433 -transportPublicAddress $ip_publisher:7400"
434 export transport_string_sub="$transport_string \
435 -nic $nic_subscriber \
436 -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438 export transport_string_pub="$transport_string"
439 export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445 ${transport_string_pub} \
446 -noPrintIntervals \
447 -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450 export pub_string="$pub_string \
451 -latencyTest"
452fi
453
454export sub_string="-sub \
455 ${transport_string_sub} \
456 -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459 echo -e "$INFO_TAG Publisher side running"
460 export commands_string=${pub_string}
461 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463 echo -e "$INFO_TAG Subscriber side running"
464 export commands_string=${sub_string}
465 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490 # UNKEYED
491 if [[ "${skip_unkeyed}" == "" ]]; then
492
493 # RELIABLE
494 if [[ "${skip_rel_tests}" == "" ]]; then
495 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496 fi
497
498 # BEST EFFORT
499 if [[ "${skip_be_tests}" == "" ]]; then
500 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501 fi
502 fi
503
504 # KEYED
505 if [[ "${skip_keyed_data}" == "" ]]; then
506
507 # RELIABLE
508 if [[ "${skip_rel_tests}" == "" ]]; then
509 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510 fi
511
512 # BEST EFFORT
513 if [[ "${skip_be_tests}" == "" ]]; then
514 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515 fi
516 fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521 no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527 if [[ "$role" == "pub" ]]; then
528 export commands_string="${commands_string} -batchSize 0"
529 fi
530
531 # UNKEYED
532 if [[ "${skip_unkeyed}" == "" ]]; then
533
534 # RELIABLE
535 if [[ "${skip_rel_tests}" == "" ]]; then
536 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537 fi
538
539 # BEST EFFORT
540 if [[ "${skip_be_tests}" == "" ]]; then
541 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542 fi
543 fi
544
545 # KEYED
546 if [[ "${skip_keyed_data}" == "" ]]; then
547
548 # RELIABLE
549 if [[ "${skip_rel_tests}" == "" ]]; then
550 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551 fi
552
553 # BEST EFFORT
554 if [[ "${skip_be_tests}" == "" ]]; then
555 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556 fi
557 fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562 echo -e "${INFO_TAG} Disabling loss rate"
563 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: ZLIB
writer_level: 5
writer_threshold: 8192 (default)
Note
By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext Core Libraries User’s Manual for more information on batching.
The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.
Detailed Statistics
The following table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
|---|---|---|---|---|---|
512 |
2132452 |
106498 |
436.2 |
0 |
0.00 |
1024 |
1377836 |
68815 |
563.7 |
0 |
0.00 |
2048 |
810267 |
40468 |
663.0 |
0 |
0.00 |
4096 |
449900 |
22470 |
736.3 |
0 |
0.00 |
8192 |
224468 |
11213 |
734.9 |
0 |
0.00 |
16384 |
154590 |
7723 |
1012.3 |
0 |
0.00 |
32768 |
98350 |
4914 |
1288.2 |
0 |
0.00 |
63000 |
59601 |
2977 |
1500.9 |
0 |
0.00 |
Best Effort
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
|---|---|---|---|---|---|
512 |
2130701 |
106437 |
436.0 |
0 |
0.00 |
1024 |
1380689 |
68972 |
565.0 |
0 |
0.00 |
2048 |
815582 |
40722 |
667.2 |
0 |
0.00 |
4096 |
451489 |
22555 |
739.1 |
0 |
0.00 |
8192 |
230871 |
11531 |
755.7 |
0 |
0.00 |
16384 |
158906 |
7937 |
1040.3 |
0 |
0.00 |
32768 |
100040 |
4997 |
1309.9 |
0 |
0.00 |
63000 |
60147 |
3003 |
1513.5 |
0 |
0.00 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25################################################################################
26
27function disable_colors() {
28 export RED=""
29 export GREEN=""
30 export YELLOW=""
31 export NC=""
32 export BLUE=""
33 export LIGHTBLUE=""
34 export INFO_TAG="${GREEN}[INFO]:${NC}"
35 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
36 export ERROR_TAG="${RED}[ERROR]:${NC}"
37}
38
39function change_domain() {
40 if [[ "$domain" == "1" ]]; then
41 export domain="2"
42 else
43 export domain="1"
44 fi
45}
46
47# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
48function execute_test() {
49
50 local keyed_unkeyed=$1
51 local rel_be=$2
52 local datasizes_test=$3
53 local other_args=$4
54 local name_suffix=$5
55
56 local commands_string_test=$commands_string
57 local tag=""
58
59 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
60 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
61 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
62 else
63 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
64 fi
65
66 if [[ "${rel_be}" == "be" ]]; then
67 commands_string_test="${commands_string_test} -bestEffort"
68 tag="${tag}${YELLOW}BE${NC}]"
69 else
70 tag="${tag}${RED}REL${NC}]"
71 fi
72
73 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
74
75 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
76
77 if [[ "$role" == "pub" ]]; then
78 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
79 fi
80
81 if [[ "$NO_TASKSET" == "" ]]; then
82 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
83 export pre_command_string="taskset -c $core"
84 fi
85 fi
86
87 if [[ "$LANGUAGE" == "python" ]]; then
88 export pre_command_string="python3 "
89 fi
90
91 if [[ "$DOCKER" == "1" ]]; then
92 export pre_command_string="taskset -c $core docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
93 executable=""
94 fi
95
96 # Get the aprox time this will take:
97 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
98 total_time=$((total_tests * exec_time))
99
100 touch $output_file
101 local no_headers=""
102 local current_test=0
103 for index in $(seq 1 ${num_reps}); do
104 for DATALEN in ${datasizes_test}; do
105 current_test=$((current_test + 1))
106 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
107 if [[ "$role" == "pub" ]]; then
108 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
109 echo -e ${BLUE}$command${NC}
110 else
111 echo -e ${LIGHTBLUE}$command${NC}
112 fi
113 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
114 sleep 3
115 fi
116 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
117 sleep 5
118 fi
119 if [[ "${get_netstat_info}" == "1" ]]; then
120 echo -e "${INFO_TAG} Getting netstat info before"
121 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
122 fi
123 eval $command >> $output_file;
124 if [[ "${get_netstat_info}" == "1" ]]; then
125 echo -e "${INFO_TAG} Getting netstat info after"
126 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
127 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
128 python3 $script_location/../../../tools/diff_netstat_output.py \
129 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
130 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
131 -d $DATALEN $no_header_netstat \
132 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
133 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
134 no_header_netstat=" -nh"
135 fi
136 no_headers=" -noOutputHeaders"
137
138 change_domain
139 done
140 done
141}
142
143################################################################################
144# PARSE COMMAND LINE OPTIONS:
145
146while [ "$1" != "" ]; do
147 case $1 in
148 --executable)
149 executable=$2
150 shift
151 ;;
152 --docker)
153 DOCKER="1"
154 ;;
155 --output-folder)
156 output_folder=$2
157 shift
158 ;;
159 --sub-folder)
160 sub_folder=$2
161 shift
162 ;;
163 --role)
164 export role=$2
165 shift
166 ;;
167 --core)
168 export core=$2
169 shift
170 ;;
171 --test-kind)
172 export lat_thr=$2
173 shift
174 ;;
175 --interface1)
176 export interface=$2
177 shift
178 ;;
179 --interface2)
180 export interface2=$2
181 shift
182 ;;
183 --ip1)
184 export ip1=$2
185 shift
186 ;;
187 --ip2)
188 export ip2=$2
189 shift
190 ;;
191 --repetitions)
192 export num_reps=$2
193 shift
194 ;;
195 --domain)
196 export domain=$2
197 shift
198 ;;
199 --execution-time)
200 export exec_time=$2
201 shift
202 ;;
203 --transport)
204 export transport=$2
205 shift
206 ;;
207 --datalen)
208 export datalen_input=$2
209 shift
210 ;;
211 --file-suffix)
212 export file_suffix=$2
213 shift
214 ;;
215 --executable-suffix)
216 export executable_suffix=$2
217 shift
218 ;;
219 --extra-arguments)
220 export extra_arguments=$2
221 shift
222 ;;
223 --extra-arguments-pub)
224 export extra_arguments_pub=$2
225 shift
226 ;;
227 --extra-arguments-sub)
228 export extra_arguments_sub=$2
229 shift
230 ;;
231 --skip-no-batching)
232 export skip_no_batching="1"
233 ;;
234 --skip-be)
235 export skip_be_tests="1"
236 ;;
237 --skip-rel)
238 export skip_rel_tests="1"
239 ;;
240 --skip-keyed)
241 export skip_keyed_data="1"
242 ;;
243 --skip-large-data)
244 export skip_large_data="1"
245 ;;
246 --large-data)
247 export large_data="1"
248 ;;
249 --keyed)
250 export skip_unkeyed="1"
251 ;;
252 --unkeyed)
253 export skip_keyed_data="1"
254 ;;
255 --no-batching | --skip-batching)
256 export no_batching_only="1"
257 ;;
258 --reliable)
259 export skip_be_tests="1"
260 ;;
261 --best-effort)
262 export skip_rel_tests="1"
263 ;;
264 --security-gov)
265 export security_only="$2"
266 shift
267 ;;
268 --micro)
269 export micro="1"
270 ;;
271 --cert)
272 export cert="1"
273 ;;
274 --raw | --raw-transport)
275 export raw="1"
276 ;;
277 --tss)
278 export tss="1"
279 ;;
280 --no-colors)
281 export NO_COLORS="1"
282 ;;
283 --language)
284 export LANGUAGE=$2
285 shift
286 ;;
287 --loss-rate)
288 export loss_rate=$2
289 shift
290 ;;
291 --get-netstat-info | --netstat)
292 export get_netstat_info="1"
293 ;;
294 --no-taskset)
295 export NO_TASKSET="1"
296 ;;
297 --reduced-data-sizes-set)
298 export REDUCED_DATA_SIZES_SET="1"
299 ;;
300 *)
301 echo -e "unknown parameter \"$1\""
302 exit 255
303 ;;
304 esac
305 shift
306done
307
308if [[ "$NO_COLORS" == "1" ]]; then
309 disable_colors
310fi
311
312export folder_base="$(dirname "${executable}")"/../../..
313
314if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
315 export folder_base="$(dirname "${executable}")"/../..
316fi
317if [[ $tss == "1" ]]; then
318 export folder_base="$(dirname "${executable}")"/../../../../..
319fi
320
321if [[ "${executable_suffix}" != "" ]]; then
322 export executable="${executable}${executable_suffix}"
323fi
324
325if [[ "${sub_folder}" != "" ]]; then
326 export output_folder="${output_folder}/${sub_folder}"
327fi
328
329echo -e "${INFO_TAG} Perftest executable is: $executable"
330echo -e "${INFO_TAG} Output folder is: $output_folder"
331
332################################################################################
333
334if [[ "$LANGUAGE" == "python" ]]; then
335 export skip_keyed_data="1"
336 export skip_large_data="1"
337 export skip_be_tests="1"
338 export skip_no_batching="1"
339fi
340
341if [[ "${skip_large_data}" == "1" ]]; then
342 export datasizes_extended=${datasizes}
343elif [[ "${large_data}" == "1" ]]; then
344 export datasizes=${datasizes_extended}
345fi
346
347if [[ "${datalen_input}" != "" ]]; then
348 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
349 export datasizes=${datalen_input}
350 export datasizes_extended=${datalen_input}
351 if [[ "${no_batching_only}" != "1" ]]; then
352 export skip_large_data="1"
353 fi
354else
355 if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
356 echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
357 export datasizes="32 128 512 2048 8192 32768 63000"
358 export datasizes_extended="${datasizes} 102400 1048576 10485760"
359 fi
360fi
361
362if [[ "$role" != "pub" && "$role" != "sub" ]]; then
363 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
364 exit 255
365fi
366
367if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
368 echo -e "${ERROR_TAG} It must be either lat or thr"
369 exit 255
370fi
371
372if [[ "${interface}" == "" ]]; then
373 echo "Using default nics"
374 export nic_publisher=${ip_machine_1}
375 export nic_subscriber=${ip_machine_2}
376elif [[ "${interface}" == "both" ]]; then
377 export nic_publisher="enp1s0f0,eno1"
378 export nic_subscriber="enp1s0f0,eno1"
379 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
380 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
381else
382 export nic_publisher=$interface
383 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
384
385 if [[ "${interface2}" == "" ]]; then
386 export nic_subscriber=$interface
387 else
388 export nic_subscriber=$interface2
389 fi
390 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
391
392 if [[ "${ip1}" != "" ]]; then
393 export ip_publisher=$ip1
394 echo "Using ip_publisher: ${ip_publisher}"
395 fi
396
397 if [[ "${ip2}" != "" ]]; then
398 export ip_subscriber=$ip2
399 echo "Using ip_subscriber: ${ip_subscriber}"
400 fi
401
402fi
403
404export transport_string="-transport $transport"
405
406if [[ "$transport" == "UDPv4" ]]; then
407
408 export transport_string_pub="$transport_string -nic $nic_publisher"
409 export transport_string_sub="$transport_string -nic $nic_subscriber"
410
411 if [[ "$micro" == "1" || "$raw" == "1" || "$cert" == "1" ]]; then
412 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
413 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
414 fi
415
416elif [[ "$transport" == "TCP" ]]; then
417 export transport_string_pub="$transport_string \
418 -nic $nic_publisher \
419 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
420 export transport_string_sub="$transport_string \
421 -nic $nic_subscriber \
422 -peer 0@tcpv4_lan://${ip_publisher}:7400"
423elif [[ "$transport" == "TLS" ]]; then
424 export transport_string_pub="$transport_string \
425 -nic $nic_publisher \
426 -peer tlsv4_lan://${ip_subscriber}:7400"
427 export transport_string_sub="$transport_string \
428 -nic $nic_subscriber \
429 -peer tlsv4_lan://${ip_publisher}:7400"
430elif [[ "$transport" == "UDPv4_WAN" ]]; then
431 export transport_string_pub="$transport_string \
432 -nic $nic_publisher \
433 -transportPublicAddress $ip_publisher:7400"
434 export transport_string_sub="$transport_string \
435 -nic $nic_subscriber \
436 -peer 0@udpv4_wan://${ip_publisher}:7400"
437else
438 export transport_string_pub="$transport_string"
439 export transport_string_sub="$transport_string"
440fi
441
442################################################################################
443
444export pub_string="-pub \
445 ${transport_string_pub} \
446 -noPrintIntervals \
447 -executionTime $exec_time"
448
449if [[ ${lat_thr} == "lat" ]]; then
450 export pub_string="$pub_string \
451 -latencyTest"
452fi
453
454export sub_string="-sub \
455 ${transport_string_sub} \
456 -noPrintIntervals"
457
458if [[ "$role" == "pub" ]]; then
459 echo -e "$INFO_TAG Publisher side running"
460 export commands_string=${pub_string}
461 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
462else
463 echo -e "$INFO_TAG Subscriber side running"
464 export commands_string=${sub_string}
465 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
466fi
467
468###############################################################################
469
470echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
471sudo /set_${lat_thr}_mode.sh
472sleep 5
473
474echo -e "${INFO_TAG} Disabling any loss rate"
475sudo tc qdisc add dev $nic_publisher root netem loss 0%
476sudo tc qdisc del dev $nic_publisher root netem loss 0%
477
478if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
479 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
480 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
481fi
482
483cd $folder_base
484echo -e "${INFO_TAG} Folder Base is: $PWD"
485mkdir -p $output_folder
486
487# Tests that may use batching (when doing throughput tests)
488if [[ ${no_batching_only} != "1" ]]; then
489
490 # UNKEYED
491 if [[ "${skip_unkeyed}" == "" ]]; then
492
493 # RELIABLE
494 if [[ "${skip_rel_tests}" == "" ]]; then
495 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
496 fi
497
498 # BEST EFFORT
499 if [[ "${skip_be_tests}" == "" ]]; then
500 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
501 fi
502 fi
503
504 # KEYED
505 if [[ "${skip_keyed_data}" == "" ]]; then
506
507 # RELIABLE
508 if [[ "${skip_rel_tests}" == "" ]]; then
509 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
510 fi
511
512 # BEST EFFORT
513 if [[ "${skip_be_tests}" == "" ]]; then
514 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
515 fi
516 fi
517
518fi
519
520if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
521 no_batching_tests="1"
522fi
523
524# Tests that will not use batching
525if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
526
527 if [[ "$role" == "pub" ]]; then
528 export commands_string="${commands_string} -batchSize 0"
529 fi
530
531 # UNKEYED
532 if [[ "${skip_unkeyed}" == "" ]]; then
533
534 # RELIABLE
535 if [[ "${skip_rel_tests}" == "" ]]; then
536 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
537 fi
538
539 # BEST EFFORT
540 if [[ "${skip_be_tests}" == "" ]]; then
541 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
542 fi
543 fi
544
545 # KEYED
546 if [[ "${skip_keyed_data}" == "" ]]; then
547
548 # RELIABLE
549 if [[ "${skip_rel_tests}" == "" ]]; then
550 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
551 fi
552
553 # BEST EFFORT
554 if [[ "${skip_be_tests}" == "" ]]; then
555 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
556 fi
557 fi
558
559fi
560
561if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
562 echo -e "${INFO_TAG} Disabling loss rate"
563 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
564fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Compression Bandwidth Savings
The following table contains the bandwidth savings gain by enabling compression with ZLIB, which is one of algorithms with a higher compression ratio and the only one that can be used in combination with batching.
The compression settings used for these tests are as follows:
compression_ids: ZLIB
writer_compression_level: 10 (best compression possible, default setting)
writer_compression_threshold: 0 (try to compress all samples)
The data used for this test is the same as used in the previous latency and throughput tests. In this test, each sample has its own size.
This test uses the compression example available in the github example repository: https://github.com/rticommunity/rticonnextdds-examples/tree/develop/examples/connext_dds/compression.
Batching |
NONE Compression (Mb) |
ZLIB Compression(Mb) |
Diff Saved Mb |
Diff Saved % |
|---|---|---|---|---|
Disabled |
70.59 |
53.74 |
16.85 |
23.87 |
2 samples |
69.40 |
37.08 |
32.32 |
46.57 |
5 samples |
69.26 |
22.08 |
47.18 |
68.12 |
10 samples |
69.25 |
17.02 |
52.23 |
75.42 |
These results show up to a 75% reduction in network bandwidth utilization by using ZLIB in combination with batching, in a real use-case scenario.