3.1.1.2. Compression
The user-data compression feature allows a DataWriter to compress the samples that it sends to matching DataReaders. Only the user payload is compressed, not the RTPS protocol headers. This compression feature can be configured using the compression_settings in the DATA_REPRESENTATION QoS Policy. For more information, see the “Data Compression” section in DATA_REPRESENTATION QosPolicy, in the RTI Connext Core Libraries User’s Manual.
The following tests have been performed by executing an RTI Perftest C++98 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. Compression has been enabled with different combinations of settings, the communication has been restricted to a single interface, and the transport has been set to UDPv4.
For latency and throughput tests, the compression threshold is set to the default value (8192 bytes), so compression is disabled for sizes lower than this value. We assume that the performance for small sample sizes usually does not improve in terms of latency or throughput, but it still can be beneficial for reducing bandwidth utilization.
The data sent and used on these tests to measure the performance of the compression feature is text data corresponding to the logs generated by Connext in a real customer scenario. We parse the data and crop it into samples of different sizes. The gathered results show a lower performance than is possible, but a more realistic one. Keep in mind that the type of data sent has a huge impact on the compression algorithm behavior.
The network used for this test is the 1Gbps network where this feature is able to boost the throughput beyond the hardware limitation of 1Gbps.
Find information about the hardware, network, and command-line parameters after each of the tests.
Compression LZ4, Unkeyed, UDPv4, C++98
The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes connected locally in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: LZ4
writer_level: 5
writer_threshold: 8192 (default)
Detailed Statistics
The following table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
---|---|---|---|---|---|---|---|---|---|
512 |
71 |
1.3 |
64 |
131 |
71 |
72 |
74 |
99 |
131 |
1024 |
83 |
2.2 |
67 |
140 |
84 |
85 |
87 |
112 |
140 |
2048 |
104 |
6.4 |
70 |
165 |
105 |
111 |
113 |
135 |
165 |
4096 |
130 |
15.0 |
71 |
258 |
132 |
148 |
157 |
173 |
258 |
8192 |
164 |
28.6 |
72 |
248 |
169 |
198 |
216 |
238 |
248 |
16384 |
215 |
48.3 |
79 |
314 |
227 |
269 |
299 |
310 |
314 |
32768 |
278 |
60.7 |
93 |
384 |
315 |
325 |
334 |
375 |
384 |
63000 |
340 |
61.0 |
119 |
475 |
363 |
391 |
414 |
426 |
475 |
Best Effort
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
---|---|---|---|---|---|---|---|---|---|
512 |
69 |
1.4 |
62 |
173 |
68 |
70 |
72 |
98 |
173 |
1024 |
82 |
2.1 |
67 |
138 |
82 |
83 |
84 |
111 |
138 |
2048 |
102 |
6.3 |
67 |
160 |
103 |
109 |
111 |
134 |
160 |
4096 |
128 |
14.9 |
70 |
212 |
130 |
146 |
155 |
171 |
212 |
8192 |
162 |
28.6 |
70 |
239 |
168 |
196 |
214 |
236 |
239 |
16384 |
213 |
48.3 |
79 |
310 |
225 |
267 |
297 |
307 |
310 |
32768 |
276 |
60.7 |
90 |
384 |
313 |
323 |
332 |
371 |
384 |
63000 |
338 |
61.0 |
116 |
483 |
361 |
389 |
412 |
425 |
483 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25export ip_machine_1="10.2.78.20"
26export ip_machine_2="10.2.78.21"
27export if10Gbps="enp1s0f0"
28export if1Gbps="eno1"
29
30################################################################################
31
32function disable_colors() {
33 export RED=""
34 export GREEN=""
35 export YELLOW=""
36 export NC=""
37 export BLUE=""
38 export LIGHTBLUE=""
39 export INFO_TAG="${GREEN}[INFO]:${NC}"
40 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
41 export ERROR_TAG="${RED}[ERROR]:${NC}"
42}
43
44function change_domain() {
45 if [[ "$domain" == "1" ]]; then
46 export domain="2"
47 else
48 export domain="1"
49 fi
50}
51
52# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
53function execute_test() {
54
55 local keyed_unkeyed=$1
56 local rel_be=$2
57 local datasizes_test=$3
58 local other_args=$4
59 local name_suffix=$5
60
61 local commands_string_test=$commands_string
62 local tag=""
63
64 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
65 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
66 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
67 else
68 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
69 fi
70
71 if [[ "${rel_be}" == "be" ]]; then
72 commands_string_test="${commands_string_test} -bestEffort"
73 tag="${tag}${YELLOW}BE${NC}]"
74 else
75 tag="${tag}${RED}REL${NC}]"
76 fi
77
78 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
79
80 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
81
82 if [[ "$role" == "pub" ]]; then
83 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be. ${NC}"
84 fi
85
86 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
87 export pre_command_string="taskset -c $core"
88 fi
89
90 if [[ "$LANGUAGE" == "python" ]]; then
91 export pre_command_string="python3 "
92 fi
93
94 # Get the aprox time this will take:
95 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
96 total_time=$((total_tests * exec_time))
97
98 touch $output_file
99 local no_headers=""
100 local current_test=0
101 for index in $(seq 1 ${num_reps}); do
102 for DATALEN in ${datasizes_test}; do
103 current_test=$((current_test + 1))
104 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
105 if [[ "$role" == "pub" ]]; then
106 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
107 echo -e ${BLUE}$command${NC}
108 else
109 echo -e ${LIGHTBLUE}$command${NC}
110 fi
111 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
112 sleep 3
113 fi
114 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
115 sleep 5
116 fi
117 if [[ "${get_netstat_info}" == "1" ]]; then
118 echo -e "${INFO_TAG} Getting netstat info before"
119 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
120 fi
121 eval $command >> $output_file;
122 if [[ "${get_netstat_info}" == "1" ]]; then
123 echo -e "${INFO_TAG} Getting netstat info after"
124 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
125 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
126 python3 $script_location/../../../tools/diff_netstat_output.py \
127 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
128 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
129 -d $DATALEN $no_header_netstat \
130 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
131 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
132 no_header_netstat=" -nh"
133 fi
134 no_headers=" -noOutputHeaders"
135
136 change_domain
137 done
138 done
139}
140
141################################################################################
142# PARSE COMMAND LINE OPTIONS:
143
144while [ "$1" != "" ]; do
145 case $1 in
146 --executable)
147 executable=$2
148 shift
149 ;;
150 --output-folder)
151 output_folder=$2
152 shift
153 ;;
154 --sub-folder)
155 sub_folder=$2
156 shift
157 ;;
158 --role)
159 export role=$2
160 shift
161 ;;
162 --core)
163 export core=$2
164 shift
165 ;;
166 --test-kind)
167 export lat_thr=$2
168 shift
169 ;;
170 --interface1)
171 export interface=$2
172 shift
173 ;;
174 --interface2)
175 export interface2=$2
176 shift
177 ;;
178 --ip1)
179 export ip1=$2
180 shift
181 ;;
182 --ip2)
183 export ip2=$2
184 shift
185 ;;
186 --repetitions)
187 export num_reps=$2
188 shift
189 ;;
190 --domain)
191 export domain=$2
192 shift
193 ;;
194 --execution-time)
195 export exec_time=$2
196 shift
197 ;;
198 --transport)
199 export transport=$2
200 shift
201 ;;
202 --datalen)
203 export datalen_input=$2
204 shift
205 ;;
206 --file-suffix)
207 export file_suffix=$2
208 shift
209 ;;
210 --executable-suffix)
211 export executable_suffix=$2
212 shift
213 ;;
214 --extra-arguments)
215 export extra_arguments=$2
216 shift
217 ;;
218 --extra-arguments-pub)
219 export extra_arguments_pub=$2
220 shift
221 ;;
222 --extra-arguments-sub)
223 export extra_arguments_sub=$2
224 shift
225 ;;
226 --skip-no-batching)
227 export skip_no_batching="1"
228 ;;
229 --skip-be)
230 export skip_be_tests="1"
231 ;;
232 --skip-rel)
233 export skip_rel_tests="1"
234 ;;
235 --skip-keyed)
236 export skip_keyed_data="1"
237 ;;
238 --skip-large-data)
239 export skip_large_data="1"
240 ;;
241 --large-data)
242 export large_data="1"
243 ;;
244 --keyed)
245 export skip_unkeyed="1"
246 ;;
247 --unkeyed)
248 export skip_keyed_data="1"
249 ;;
250 --no-batching)
251 export no_batching_only="1"
252 ;;
253 --reliable)
254 export skip_be_tests="1"
255 ;;
256 --best-effort)
257 export skip_rel_tests="1"
258 ;;
259 --security)
260 export security_only="$2"
261 shift
262 ;;
263 --micro)
264 export micro="1"
265 ;;
266 --raw | --raw-transport)
267 export raw="1"
268 ;;
269 --tss)
270 export tss="1"
271 ;;
272 --no-colors)
273 export NO_COLORS="1"
274 ;;
275 --language)
276 export LANGUAGE=$2
277 shift
278 ;;
279 --loss-rate)
280 export loss_rate=$2
281 shift
282 ;;
283 --get-netstat-info)
284 export get_netstat_info="1"
285 ;;
286 *)
287 echo -e "unknown parameter \"$1\""
288 exit 255
289 ;;
290 esac
291 shift
292done
293
294if [[ "$NO_COLORS" == "1" ]]; then
295 disable_colors
296fi
297
298export folder_base="$(dirname "${executable}")"/../../..
299
300if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
301 export folder_base="$(dirname "${executable}")"/../..
302fi
303if [[ $tss == "1" ]]; then
304 export folder_base="$(dirname "${executable}")"/../../../../..
305fi
306
307if [[ "${executable_suffix}" != "" ]]; then
308 export executable="${executable}${executable_suffix}"
309fi
310
311if [[ "${sub_folder}" != "" ]]; then
312 export output_folder="${output_folder}/${sub_folder}"
313fi
314
315echo -e "${INFO_TAG} Perftest executable is: $executable"
316echo -e "${INFO_TAG} Output folder is: $output_folder"
317
318################################################################################
319
320if [[ "$LANGUAGE" == "python" ]]; then
321 export skip_keyed_data="1"
322 export skip_large_data="1"
323 export skip_be_tests="1"
324 export skip_no_batching="1"
325fi
326
327if [[ "${skip_large_data}" == "1" ]]; then
328 export datasizes_extended=${datasizes}
329elif [[ "${large_data}" == "1" ]]; then
330 export datasizes=${datasizes_extended}
331fi
332
333if [[ "${datalen_input}" != "" ]]; then
334 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
335 export datasizes=${datalen_input}
336 export datasizes_extended=${datalen_input}
337 if [[ "${no_batching_only}" != "1" ]]; then
338 export skip_large_data="1"
339 fi
340fi
341
342if [[ "$role" != "pub" && "$role" != "sub" ]]; then
343 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
344 exit 255
345fi
346
347if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
348 echo -e "${ERROR_TAG} It must be either lat or thr"
349 exit 255
350fi
351
352if [[ "${interface}" == "" ]]; then
353 echo "Using default nics"
354 export nic_publisher=${ip_machine_1}
355 export nic_subscriber=${ip_machine_2}
356else
357 export nic_publisher=$interface
358 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
359
360 if [[ "${interface2}" == "" ]]; then
361 export nic_subscriber=$interface
362 else
363 export nic_subscriber=$interface2
364 fi
365 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
366
367 if [[ "${ip1}" != "" ]]; then
368 export ip_publisher=$ip1
369 echo "Using ip_publisher: ${ip_publisher}"
370 fi
371
372 if [[ "${ip2}" != "" ]]; then
373 export ip_subscriber=$ip2
374 echo "Using ip_subscriber: ${ip_subscriber}"
375 fi
376
377fi
378
379export transport_string="-transport $transport"
380
381if [[ "$transport" == "UDPv4" ]]; then
382
383 export transport_string_pub="$transport_string -nic $nic_publisher"
384 export transport_string_sub="$transport_string -nic $nic_subscriber"
385
386 if [[ "$micro" == "1" || "$raw" == "1" ]]; then
387 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
388 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
389 fi
390
391elif [[ "$transport" == "TCP" ]]; then
392 export transport_string_pub="$transport_string \
393 -nic $nic_publisher \
394 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
395 export transport_string_sub="$transport_string \
396 -nic $nic_subscriber \
397 -peer 0@tcpv4_lan://${ip_publisher}:7400"
398elif [[ "$transport" == "TLS" ]]; then
399 export transport_string_pub="$transport_string \
400 -nic $nic_publisher \
401 -peer tlsv4_lan://${ip_subscriber}:7400"
402 export transport_string_sub="$transport_string \
403 -nic $nic_subscriber \
404 -peer tlsv4_lan://${ip_publisher}:7400"
405elif [[ "$transport" == "UDPv4_WAN" ]]; then
406 export transport_string_pub="$transport_string \
407 -nic $nic_publisher \
408 -transportPublicAddress $ip_publisher:7400"
409 export transport_string_sub="$transport_string \
410 -nic $nic_subscriber \
411 -peer 0@udpv4_wan://${ip_publisher}:7400"
412else
413 export transport_string_pub="$transport_string"
414 export transport_string_sub="$transport_string"
415fi
416
417################################################################################
418
419export pub_string="-pub \
420 ${transport_string_pub} \
421 -noPrintIntervals \
422 -executionTime $exec_time"
423
424if [[ ${lat_thr} == "lat" ]]; then
425 export pub_string="$pub_string \
426 -latencyTest"
427fi
428
429export sub_string="-sub \
430 ${transport_string_sub} \
431 -noPrintIntervals"
432
433if [[ "$role" == "pub" ]]; then
434 echo -e "$INFO_TAG Publisher side running"
435 export commands_string=${pub_string}
436 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
437else
438 echo -e "$INFO_TAG Subscriber side running"
439 export commands_string=${sub_string}
440 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
441fi
442
443###############################################################################
444
445echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
446sudo /set_${lat_thr}_mode.sh
447sleep 5
448
449echo -e "${INFO_TAG} Disabling any loss rate"
450sudo tc qdisc add dev $nic_publisher root netem loss 0%
451sudo tc qdisc del dev $nic_publisher root netem loss 0%
452
453if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
454 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
455 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
456fi
457
458cd $folder_base
459echo -e "${INFO_TAG} Folder Base is: $PWD"
460mkdir -p $output_folder
461
462# Tests that may use batching (when doing throughput tests)
463if [[ ${no_batching_only} != "1" ]]; then
464
465 # UNKEYED
466 if [[ "${skip_unkeyed}" == "" ]]; then
467
468 # RELIABLE
469 if [[ "${skip_rel_tests}" == "" ]]; then
470 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
471 fi
472
473 # BEST EFFORT
474 if [[ "${skip_be_tests}" == "" ]]; then
475 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
476 fi
477 fi
478
479 # KEYED
480 if [[ "${skip_keyed_data}" == "" ]]; then
481
482 # RELIABLE
483 if [[ "${skip_rel_tests}" == "" ]]; then
484 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
485 fi
486
487 # BEST EFFORT
488 if [[ "${skip_be_tests}" == "" ]]; then
489 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
490 fi
491 fi
492
493fi
494
495if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
496 no_batching_tests="1"
497fi
498
499# Tests that will not use batching
500if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
501
502 if [[ "$role" == "pub" ]]; then
503 export commands_string="${commands_string} -batchSize 0"
504 fi
505
506 # UNKEYED
507 if [[ "${skip_unkeyed}" == "" ]]; then
508
509 # RELIABLE
510 if [[ "${skip_rel_tests}" == "" ]]; then
511 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
512 fi
513
514 # BEST EFFORT
515 if [[ "${skip_be_tests}" == "" ]]; then
516 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
517 fi
518 fi
519fi
520
521if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
522 echo -e "${INFO_TAG} Disabling loss rate"
523 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
524fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: LZ4
writer_level: 5
writer_threshold: 8192 (default)
Note
By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.
LZ4 is not compatible with batching. Therefore, we are comparing batched regular data with un-batched compressed data. This explains some of the differences in throughput for sizes lower than 8192 bytes.
The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.
Detailed Statistics
The following table contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
---|---|---|---|---|---|
512 |
3916138 |
195803 |
802.0 |
0 |
0.00 |
1024 |
2253601 |
112677 |
923.1 |
0 |
0.00 |
2048 |
1356171 |
67806 |
1110.9 |
0 |
0.00 |
4096 |
890076 |
44501 |
1458.2 |
0 |
0.00 |
8192 |
620691 |
31031 |
2033.7 |
0 |
0.00 |
16384 |
431211 |
21557 |
2825.6 |
0 |
0.00 |
32768 |
173157 |
8656 |
2269.2 |
0 |
0.00 |
63000 |
109984 |
5497 |
2770.7 |
0 |
0.00 |
Best Effort
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
---|---|---|---|---|---|
512 |
3942782 |
196893 |
806.5 |
0 |
0.00 |
1024 |
2261667 |
113006 |
925.7 |
0 |
0.00 |
2048 |
1359233 |
67926 |
1112.9 |
0 |
0.00 |
4096 |
891577 |
44551 |
1459.9 |
0 |
0.00 |
8192 |
621361 |
31054 |
2035.2 |
0 |
0.00 |
16384 |
243325 |
12156 |
1593.4 |
188474 |
43.65 |
32768 |
128879 |
6441 |
1688.6 |
162784 |
55.81 |
63000 |
105272 |
5260 |
2651.5 |
91597 |
46.53 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25export ip_machine_1="10.2.78.20"
26export ip_machine_2="10.2.78.21"
27export if10Gbps="enp1s0f0"
28export if1Gbps="eno1"
29
30################################################################################
31
32function disable_colors() {
33 export RED=""
34 export GREEN=""
35 export YELLOW=""
36 export NC=""
37 export BLUE=""
38 export LIGHTBLUE=""
39 export INFO_TAG="${GREEN}[INFO]:${NC}"
40 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
41 export ERROR_TAG="${RED}[ERROR]:${NC}"
42}
43
44function change_domain() {
45 if [[ "$domain" == "1" ]]; then
46 export domain="2"
47 else
48 export domain="1"
49 fi
50}
51
52# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
53function execute_test() {
54
55 local keyed_unkeyed=$1
56 local rel_be=$2
57 local datasizes_test=$3
58 local other_args=$4
59 local name_suffix=$5
60
61 local commands_string_test=$commands_string
62 local tag=""
63
64 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
65 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
66 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
67 else
68 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
69 fi
70
71 if [[ "${rel_be}" == "be" ]]; then
72 commands_string_test="${commands_string_test} -bestEffort"
73 tag="${tag}${YELLOW}BE${NC}]"
74 else
75 tag="${tag}${RED}REL${NC}]"
76 fi
77
78 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
79
80 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
81
82 if [[ "$role" == "pub" ]]; then
83 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be. ${NC}"
84 fi
85
86 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
87 export pre_command_string="taskset -c $core"
88 fi
89
90 if [[ "$LANGUAGE" == "python" ]]; then
91 export pre_command_string="python3 "
92 fi
93
94 # Get the aprox time this will take:
95 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
96 total_time=$((total_tests * exec_time))
97
98 touch $output_file
99 local no_headers=""
100 local current_test=0
101 for index in $(seq 1 ${num_reps}); do
102 for DATALEN in ${datasizes_test}; do
103 current_test=$((current_test + 1))
104 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
105 if [[ "$role" == "pub" ]]; then
106 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
107 echo -e ${BLUE}$command${NC}
108 else
109 echo -e ${LIGHTBLUE}$command${NC}
110 fi
111 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
112 sleep 3
113 fi
114 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
115 sleep 5
116 fi
117 if [[ "${get_netstat_info}" == "1" ]]; then
118 echo -e "${INFO_TAG} Getting netstat info before"
119 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
120 fi
121 eval $command >> $output_file;
122 if [[ "${get_netstat_info}" == "1" ]]; then
123 echo -e "${INFO_TAG} Getting netstat info after"
124 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
125 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
126 python3 $script_location/../../../tools/diff_netstat_output.py \
127 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
128 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
129 -d $DATALEN $no_header_netstat \
130 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
131 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
132 no_header_netstat=" -nh"
133 fi
134 no_headers=" -noOutputHeaders"
135
136 change_domain
137 done
138 done
139}
140
141################################################################################
142# PARSE COMMAND LINE OPTIONS:
143
144while [ "$1" != "" ]; do
145 case $1 in
146 --executable)
147 executable=$2
148 shift
149 ;;
150 --output-folder)
151 output_folder=$2
152 shift
153 ;;
154 --sub-folder)
155 sub_folder=$2
156 shift
157 ;;
158 --role)
159 export role=$2
160 shift
161 ;;
162 --core)
163 export core=$2
164 shift
165 ;;
166 --test-kind)
167 export lat_thr=$2
168 shift
169 ;;
170 --interface1)
171 export interface=$2
172 shift
173 ;;
174 --interface2)
175 export interface2=$2
176 shift
177 ;;
178 --ip1)
179 export ip1=$2
180 shift
181 ;;
182 --ip2)
183 export ip2=$2
184 shift
185 ;;
186 --repetitions)
187 export num_reps=$2
188 shift
189 ;;
190 --domain)
191 export domain=$2
192 shift
193 ;;
194 --execution-time)
195 export exec_time=$2
196 shift
197 ;;
198 --transport)
199 export transport=$2
200 shift
201 ;;
202 --datalen)
203 export datalen_input=$2
204 shift
205 ;;
206 --file-suffix)
207 export file_suffix=$2
208 shift
209 ;;
210 --executable-suffix)
211 export executable_suffix=$2
212 shift
213 ;;
214 --extra-arguments)
215 export extra_arguments=$2
216 shift
217 ;;
218 --extra-arguments-pub)
219 export extra_arguments_pub=$2
220 shift
221 ;;
222 --extra-arguments-sub)
223 export extra_arguments_sub=$2
224 shift
225 ;;
226 --skip-no-batching)
227 export skip_no_batching="1"
228 ;;
229 --skip-be)
230 export skip_be_tests="1"
231 ;;
232 --skip-rel)
233 export skip_rel_tests="1"
234 ;;
235 --skip-keyed)
236 export skip_keyed_data="1"
237 ;;
238 --skip-large-data)
239 export skip_large_data="1"
240 ;;
241 --large-data)
242 export large_data="1"
243 ;;
244 --keyed)
245 export skip_unkeyed="1"
246 ;;
247 --unkeyed)
248 export skip_keyed_data="1"
249 ;;
250 --no-batching)
251 export no_batching_only="1"
252 ;;
253 --reliable)
254 export skip_be_tests="1"
255 ;;
256 --best-effort)
257 export skip_rel_tests="1"
258 ;;
259 --security)
260 export security_only="$2"
261 shift
262 ;;
263 --micro)
264 export micro="1"
265 ;;
266 --raw | --raw-transport)
267 export raw="1"
268 ;;
269 --tss)
270 export tss="1"
271 ;;
272 --no-colors)
273 export NO_COLORS="1"
274 ;;
275 --language)
276 export LANGUAGE=$2
277 shift
278 ;;
279 --loss-rate)
280 export loss_rate=$2
281 shift
282 ;;
283 --get-netstat-info)
284 export get_netstat_info="1"
285 ;;
286 *)
287 echo -e "unknown parameter \"$1\""
288 exit 255
289 ;;
290 esac
291 shift
292done
293
294if [[ "$NO_COLORS" == "1" ]]; then
295 disable_colors
296fi
297
298export folder_base="$(dirname "${executable}")"/../../..
299
300if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
301 export folder_base="$(dirname "${executable}")"/../..
302fi
303if [[ $tss == "1" ]]; then
304 export folder_base="$(dirname "${executable}")"/../../../../..
305fi
306
307if [[ "${executable_suffix}" != "" ]]; then
308 export executable="${executable}${executable_suffix}"
309fi
310
311if [[ "${sub_folder}" != "" ]]; then
312 export output_folder="${output_folder}/${sub_folder}"
313fi
314
315echo -e "${INFO_TAG} Perftest executable is: $executable"
316echo -e "${INFO_TAG} Output folder is: $output_folder"
317
318################################################################################
319
320if [[ "$LANGUAGE" == "python" ]]; then
321 export skip_keyed_data="1"
322 export skip_large_data="1"
323 export skip_be_tests="1"
324 export skip_no_batching="1"
325fi
326
327if [[ "${skip_large_data}" == "1" ]]; then
328 export datasizes_extended=${datasizes}
329elif [[ "${large_data}" == "1" ]]; then
330 export datasizes=${datasizes_extended}
331fi
332
333if [[ "${datalen_input}" != "" ]]; then
334 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
335 export datasizes=${datalen_input}
336 export datasizes_extended=${datalen_input}
337 if [[ "${no_batching_only}" != "1" ]]; then
338 export skip_large_data="1"
339 fi
340fi
341
342if [[ "$role" != "pub" && "$role" != "sub" ]]; then
343 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
344 exit 255
345fi
346
347if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
348 echo -e "${ERROR_TAG} It must be either lat or thr"
349 exit 255
350fi
351
352if [[ "${interface}" == "" ]]; then
353 echo "Using default nics"
354 export nic_publisher=${ip_machine_1}
355 export nic_subscriber=${ip_machine_2}
356else
357 export nic_publisher=$interface
358 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
359
360 if [[ "${interface2}" == "" ]]; then
361 export nic_subscriber=$interface
362 else
363 export nic_subscriber=$interface2
364 fi
365 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
366
367 if [[ "${ip1}" != "" ]]; then
368 export ip_publisher=$ip1
369 echo "Using ip_publisher: ${ip_publisher}"
370 fi
371
372 if [[ "${ip2}" != "" ]]; then
373 export ip_subscriber=$ip2
374 echo "Using ip_subscriber: ${ip_subscriber}"
375 fi
376
377fi
378
379export transport_string="-transport $transport"
380
381if [[ "$transport" == "UDPv4" ]]; then
382
383 export transport_string_pub="$transport_string -nic $nic_publisher"
384 export transport_string_sub="$transport_string -nic $nic_subscriber"
385
386 if [[ "$micro" == "1" || "$raw" == "1" ]]; then
387 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
388 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
389 fi
390
391elif [[ "$transport" == "TCP" ]]; then
392 export transport_string_pub="$transport_string \
393 -nic $nic_publisher \
394 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
395 export transport_string_sub="$transport_string \
396 -nic $nic_subscriber \
397 -peer 0@tcpv4_lan://${ip_publisher}:7400"
398elif [[ "$transport" == "TLS" ]]; then
399 export transport_string_pub="$transport_string \
400 -nic $nic_publisher \
401 -peer tlsv4_lan://${ip_subscriber}:7400"
402 export transport_string_sub="$transport_string \
403 -nic $nic_subscriber \
404 -peer tlsv4_lan://${ip_publisher}:7400"
405elif [[ "$transport" == "UDPv4_WAN" ]]; then
406 export transport_string_pub="$transport_string \
407 -nic $nic_publisher \
408 -transportPublicAddress $ip_publisher:7400"
409 export transport_string_sub="$transport_string \
410 -nic $nic_subscriber \
411 -peer 0@udpv4_wan://${ip_publisher}:7400"
412else
413 export transport_string_pub="$transport_string"
414 export transport_string_sub="$transport_string"
415fi
416
417################################################################################
418
419export pub_string="-pub \
420 ${transport_string_pub} \
421 -noPrintIntervals \
422 -executionTime $exec_time"
423
424if [[ ${lat_thr} == "lat" ]]; then
425 export pub_string="$pub_string \
426 -latencyTest"
427fi
428
429export sub_string="-sub \
430 ${transport_string_sub} \
431 -noPrintIntervals"
432
433if [[ "$role" == "pub" ]]; then
434 echo -e "$INFO_TAG Publisher side running"
435 export commands_string=${pub_string}
436 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
437else
438 echo -e "$INFO_TAG Subscriber side running"
439 export commands_string=${sub_string}
440 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
441fi
442
443###############################################################################
444
445echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
446sudo /set_${lat_thr}_mode.sh
447sleep 5
448
449echo -e "${INFO_TAG} Disabling any loss rate"
450sudo tc qdisc add dev $nic_publisher root netem loss 0%
451sudo tc qdisc del dev $nic_publisher root netem loss 0%
452
453if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
454 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
455 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
456fi
457
458cd $folder_base
459echo -e "${INFO_TAG} Folder Base is: $PWD"
460mkdir -p $output_folder
461
462# Tests that may use batching (when doing throughput tests)
463if [[ ${no_batching_only} != "1" ]]; then
464
465 # UNKEYED
466 if [[ "${skip_unkeyed}" == "" ]]; then
467
468 # RELIABLE
469 if [[ "${skip_rel_tests}" == "" ]]; then
470 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
471 fi
472
473 # BEST EFFORT
474 if [[ "${skip_be_tests}" == "" ]]; then
475 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
476 fi
477 fi
478
479 # KEYED
480 if [[ "${skip_keyed_data}" == "" ]]; then
481
482 # RELIABLE
483 if [[ "${skip_rel_tests}" == "" ]]; then
484 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
485 fi
486
487 # BEST EFFORT
488 if [[ "${skip_be_tests}" == "" ]]; then
489 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
490 fi
491 fi
492
493fi
494
495if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
496 no_batching_tests="1"
497fi
498
499# Tests that will not use batching
500if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
501
502 if [[ "$role" == "pub" ]]; then
503 export commands_string="${commands_string} -batchSize 0"
504 fi
505
506 # UNKEYED
507 if [[ "${skip_unkeyed}" == "" ]]; then
508
509 # RELIABLE
510 if [[ "${skip_rel_tests}" == "" ]]; then
511 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
512 fi
513
514 # BEST EFFORT
515 if [[ "${skip_be_tests}" == "" ]]; then
516 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
517 fi
518 fi
519fi
520
521if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
522 echo -e "${INFO_TAG} Disabling loss rate"
523 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
524fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Compression ZLIB, Unkeyed, UDPv4, C++98
The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes connected locally in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: ZLIB
writer_level: 5
writer_threshold: 8192 (default)
Detailed Statistics
The following table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
---|---|---|---|---|---|---|---|---|---|
512 |
122 |
3.1 |
98 |
185 |
122 |
125 |
130 |
154 |
185 |
1024 |
131 |
6.0 |
100 |
267 |
130 |
138 |
143 |
168 |
267 |
2048 |
145 |
9.2 |
107 |
205 |
144 |
156 |
166 |
186 |
205 |
4096 |
170 |
15.6 |
120 |
291 |
171 |
189 |
206 |
235 |
291 |
8192 |
211 |
26.6 |
140 |
307 |
215 |
243 |
266 |
299 |
307 |
16384 |
288 |
43.6 |
189 |
460 |
300 |
337 |
372 |
443 |
460 |
32768 |
432 |
70.9 |
262 |
692 |
453 |
506 |
546 |
660 |
692 |
63000 |
685 |
112.3 |
395 |
977 |
727 |
794 |
848 |
953 |
977 |
Best Effort
Sample Size (Bytes) |
Ave (μs) |
Std (μs) |
Min (μs) |
Max (μs) |
50% (μs) |
90% (μs) |
99% (μs) |
99.99% (μs) |
99.9999% (μs) |
---|---|---|---|---|---|---|---|---|---|
512 |
119 |
2.9 |
95 |
183 |
120 |
122 |
126 |
152 |
183 |
1024 |
129 |
5.9 |
98 |
198 |
128 |
136 |
141 |
165 |
198 |
2048 |
143 |
9.2 |
105 |
239 |
142 |
154 |
163 |
186 |
239 |
4096 |
168 |
15.3 |
118 |
268 |
168 |
187 |
201 |
222 |
268 |
8192 |
209 |
26.3 |
138 |
296 |
213 |
240 |
260 |
294 |
296 |
16384 |
286 |
43.5 |
186 |
442 |
298 |
334 |
369 |
439 |
442 |
32768 |
431 |
71.7 |
261 |
668 |
453 |
506 |
543 |
667 |
668 |
63000 |
682 |
112.1 |
394 |
950 |
724 |
790 |
843 |
948 |
950 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25export ip_machine_1="10.2.78.20"
26export ip_machine_2="10.2.78.21"
27export if10Gbps="enp1s0f0"
28export if1Gbps="eno1"
29
30################################################################################
31
32function disable_colors() {
33 export RED=""
34 export GREEN=""
35 export YELLOW=""
36 export NC=""
37 export BLUE=""
38 export LIGHTBLUE=""
39 export INFO_TAG="${GREEN}[INFO]:${NC}"
40 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
41 export ERROR_TAG="${RED}[ERROR]:${NC}"
42}
43
44function change_domain() {
45 if [[ "$domain" == "1" ]]; then
46 export domain="2"
47 else
48 export domain="1"
49 fi
50}
51
52# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
53function execute_test() {
54
55 local keyed_unkeyed=$1
56 local rel_be=$2
57 local datasizes_test=$3
58 local other_args=$4
59 local name_suffix=$5
60
61 local commands_string_test=$commands_string
62 local tag=""
63
64 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
65 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
66 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
67 else
68 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
69 fi
70
71 if [[ "${rel_be}" == "be" ]]; then
72 commands_string_test="${commands_string_test} -bestEffort"
73 tag="${tag}${YELLOW}BE${NC}]"
74 else
75 tag="${tag}${RED}REL${NC}]"
76 fi
77
78 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
79
80 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
81
82 if [[ "$role" == "pub" ]]; then
83 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be. ${NC}"
84 fi
85
86 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
87 export pre_command_string="taskset -c $core"
88 fi
89
90 if [[ "$LANGUAGE" == "python" ]]; then
91 export pre_command_string="python3 "
92 fi
93
94 # Get the aprox time this will take:
95 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
96 total_time=$((total_tests * exec_time))
97
98 touch $output_file
99 local no_headers=""
100 local current_test=0
101 for index in $(seq 1 ${num_reps}); do
102 for DATALEN in ${datasizes_test}; do
103 current_test=$((current_test + 1))
104 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
105 if [[ "$role" == "pub" ]]; then
106 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
107 echo -e ${BLUE}$command${NC}
108 else
109 echo -e ${LIGHTBLUE}$command${NC}
110 fi
111 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
112 sleep 3
113 fi
114 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
115 sleep 5
116 fi
117 if [[ "${get_netstat_info}" == "1" ]]; then
118 echo -e "${INFO_TAG} Getting netstat info before"
119 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
120 fi
121 eval $command >> $output_file;
122 if [[ "${get_netstat_info}" == "1" ]]; then
123 echo -e "${INFO_TAG} Getting netstat info after"
124 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
125 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
126 python3 $script_location/../../../tools/diff_netstat_output.py \
127 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
128 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
129 -d $DATALEN $no_header_netstat \
130 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
131 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
132 no_header_netstat=" -nh"
133 fi
134 no_headers=" -noOutputHeaders"
135
136 change_domain
137 done
138 done
139}
140
141################################################################################
142# PARSE COMMAND LINE OPTIONS:
143
144while [ "$1" != "" ]; do
145 case $1 in
146 --executable)
147 executable=$2
148 shift
149 ;;
150 --output-folder)
151 output_folder=$2
152 shift
153 ;;
154 --sub-folder)
155 sub_folder=$2
156 shift
157 ;;
158 --role)
159 export role=$2
160 shift
161 ;;
162 --core)
163 export core=$2
164 shift
165 ;;
166 --test-kind)
167 export lat_thr=$2
168 shift
169 ;;
170 --interface1)
171 export interface=$2
172 shift
173 ;;
174 --interface2)
175 export interface2=$2
176 shift
177 ;;
178 --ip1)
179 export ip1=$2
180 shift
181 ;;
182 --ip2)
183 export ip2=$2
184 shift
185 ;;
186 --repetitions)
187 export num_reps=$2
188 shift
189 ;;
190 --domain)
191 export domain=$2
192 shift
193 ;;
194 --execution-time)
195 export exec_time=$2
196 shift
197 ;;
198 --transport)
199 export transport=$2
200 shift
201 ;;
202 --datalen)
203 export datalen_input=$2
204 shift
205 ;;
206 --file-suffix)
207 export file_suffix=$2
208 shift
209 ;;
210 --executable-suffix)
211 export executable_suffix=$2
212 shift
213 ;;
214 --extra-arguments)
215 export extra_arguments=$2
216 shift
217 ;;
218 --extra-arguments-pub)
219 export extra_arguments_pub=$2
220 shift
221 ;;
222 --extra-arguments-sub)
223 export extra_arguments_sub=$2
224 shift
225 ;;
226 --skip-no-batching)
227 export skip_no_batching="1"
228 ;;
229 --skip-be)
230 export skip_be_tests="1"
231 ;;
232 --skip-rel)
233 export skip_rel_tests="1"
234 ;;
235 --skip-keyed)
236 export skip_keyed_data="1"
237 ;;
238 --skip-large-data)
239 export skip_large_data="1"
240 ;;
241 --large-data)
242 export large_data="1"
243 ;;
244 --keyed)
245 export skip_unkeyed="1"
246 ;;
247 --unkeyed)
248 export skip_keyed_data="1"
249 ;;
250 --no-batching)
251 export no_batching_only="1"
252 ;;
253 --reliable)
254 export skip_be_tests="1"
255 ;;
256 --best-effort)
257 export skip_rel_tests="1"
258 ;;
259 --security)
260 export security_only="$2"
261 shift
262 ;;
263 --micro)
264 export micro="1"
265 ;;
266 --raw | --raw-transport)
267 export raw="1"
268 ;;
269 --tss)
270 export tss="1"
271 ;;
272 --no-colors)
273 export NO_COLORS="1"
274 ;;
275 --language)
276 export LANGUAGE=$2
277 shift
278 ;;
279 --loss-rate)
280 export loss_rate=$2
281 shift
282 ;;
283 --get-netstat-info)
284 export get_netstat_info="1"
285 ;;
286 *)
287 echo -e "unknown parameter \"$1\""
288 exit 255
289 ;;
290 esac
291 shift
292done
293
294if [[ "$NO_COLORS" == "1" ]]; then
295 disable_colors
296fi
297
298export folder_base="$(dirname "${executable}")"/../../..
299
300if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
301 export folder_base="$(dirname "${executable}")"/../..
302fi
303if [[ $tss == "1" ]]; then
304 export folder_base="$(dirname "${executable}")"/../../../../..
305fi
306
307if [[ "${executable_suffix}" != "" ]]; then
308 export executable="${executable}${executable_suffix}"
309fi
310
311if [[ "${sub_folder}" != "" ]]; then
312 export output_folder="${output_folder}/${sub_folder}"
313fi
314
315echo -e "${INFO_TAG} Perftest executable is: $executable"
316echo -e "${INFO_TAG} Output folder is: $output_folder"
317
318################################################################################
319
320if [[ "$LANGUAGE" == "python" ]]; then
321 export skip_keyed_data="1"
322 export skip_large_data="1"
323 export skip_be_tests="1"
324 export skip_no_batching="1"
325fi
326
327if [[ "${skip_large_data}" == "1" ]]; then
328 export datasizes_extended=${datasizes}
329elif [[ "${large_data}" == "1" ]]; then
330 export datasizes=${datasizes_extended}
331fi
332
333if [[ "${datalen_input}" != "" ]]; then
334 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
335 export datasizes=${datalen_input}
336 export datasizes_extended=${datalen_input}
337 if [[ "${no_batching_only}" != "1" ]]; then
338 export skip_large_data="1"
339 fi
340fi
341
342if [[ "$role" != "pub" && "$role" != "sub" ]]; then
343 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
344 exit 255
345fi
346
347if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
348 echo -e "${ERROR_TAG} It must be either lat or thr"
349 exit 255
350fi
351
352if [[ "${interface}" == "" ]]; then
353 echo "Using default nics"
354 export nic_publisher=${ip_machine_1}
355 export nic_subscriber=${ip_machine_2}
356else
357 export nic_publisher=$interface
358 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
359
360 if [[ "${interface2}" == "" ]]; then
361 export nic_subscriber=$interface
362 else
363 export nic_subscriber=$interface2
364 fi
365 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
366
367 if [[ "${ip1}" != "" ]]; then
368 export ip_publisher=$ip1
369 echo "Using ip_publisher: ${ip_publisher}"
370 fi
371
372 if [[ "${ip2}" != "" ]]; then
373 export ip_subscriber=$ip2
374 echo "Using ip_subscriber: ${ip_subscriber}"
375 fi
376
377fi
378
379export transport_string="-transport $transport"
380
381if [[ "$transport" == "UDPv4" ]]; then
382
383 export transport_string_pub="$transport_string -nic $nic_publisher"
384 export transport_string_sub="$transport_string -nic $nic_subscriber"
385
386 if [[ "$micro" == "1" || "$raw" == "1" ]]; then
387 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
388 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
389 fi
390
391elif [[ "$transport" == "TCP" ]]; then
392 export transport_string_pub="$transport_string \
393 -nic $nic_publisher \
394 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
395 export transport_string_sub="$transport_string \
396 -nic $nic_subscriber \
397 -peer 0@tcpv4_lan://${ip_publisher}:7400"
398elif [[ "$transport" == "TLS" ]]; then
399 export transport_string_pub="$transport_string \
400 -nic $nic_publisher \
401 -peer tlsv4_lan://${ip_subscriber}:7400"
402 export transport_string_sub="$transport_string \
403 -nic $nic_subscriber \
404 -peer tlsv4_lan://${ip_publisher}:7400"
405elif [[ "$transport" == "UDPv4_WAN" ]]; then
406 export transport_string_pub="$transport_string \
407 -nic $nic_publisher \
408 -transportPublicAddress $ip_publisher:7400"
409 export transport_string_sub="$transport_string \
410 -nic $nic_subscriber \
411 -peer 0@udpv4_wan://${ip_publisher}:7400"
412else
413 export transport_string_pub="$transport_string"
414 export transport_string_sub="$transport_string"
415fi
416
417################################################################################
418
419export pub_string="-pub \
420 ${transport_string_pub} \
421 -noPrintIntervals \
422 -executionTime $exec_time"
423
424if [[ ${lat_thr} == "lat" ]]; then
425 export pub_string="$pub_string \
426 -latencyTest"
427fi
428
429export sub_string="-sub \
430 ${transport_string_sub} \
431 -noPrintIntervals"
432
433if [[ "$role" == "pub" ]]; then
434 echo -e "$INFO_TAG Publisher side running"
435 export commands_string=${pub_string}
436 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
437else
438 echo -e "$INFO_TAG Subscriber side running"
439 export commands_string=${sub_string}
440 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
441fi
442
443###############################################################################
444
445echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
446sudo /set_${lat_thr}_mode.sh
447sleep 5
448
449echo -e "${INFO_TAG} Disabling any loss rate"
450sudo tc qdisc add dev $nic_publisher root netem loss 0%
451sudo tc qdisc del dev $nic_publisher root netem loss 0%
452
453if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
454 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
455 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
456fi
457
458cd $folder_base
459echo -e "${INFO_TAG} Folder Base is: $PWD"
460mkdir -p $output_folder
461
462# Tests that may use batching (when doing throughput tests)
463if [[ ${no_batching_only} != "1" ]]; then
464
465 # UNKEYED
466 if [[ "${skip_unkeyed}" == "" ]]; then
467
468 # RELIABLE
469 if [[ "${skip_rel_tests}" == "" ]]; then
470 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
471 fi
472
473 # BEST EFFORT
474 if [[ "${skip_be_tests}" == "" ]]; then
475 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
476 fi
477 fi
478
479 # KEYED
480 if [[ "${skip_keyed_data}" == "" ]]; then
481
482 # RELIABLE
483 if [[ "${skip_rel_tests}" == "" ]]; then
484 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
485 fi
486
487 # BEST EFFORT
488 if [[ "${skip_be_tests}" == "" ]]; then
489 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
490 fi
491 fi
492
493fi
494
495if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
496 no_batching_tests="1"
497fi
498
499# Tests that will not use batching
500if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
501
502 if [[ "$role" == "pub" ]]; then
503 export commands_string="${commands_string} -batchSize 0"
504 fi
505
506 # UNKEYED
507 if [[ "${skip_unkeyed}" == "" ]]; then
508
509 # RELIABLE
510 if [[ "${skip_rel_tests}" == "" ]]; then
511 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
512 fi
513
514 # BEST EFFORT
515 if [[ "${skip_be_tests}" == "" ]]; then
516 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
517 fi
518 fi
519fi
520
521if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
522 echo -e "${INFO_TAG} Disabling loss rate"
523 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
524fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 1Gbps network.
The compression settings used for these tests are as follows:
compression_ids: ZLIB
writer_level: 5
writer_threshold: 8192 (default)
Note
By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext Core Libraries User’s Manual for more information on batching.
The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.
Detailed Statistics
The following table contains the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.
Reliable
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
---|---|---|---|---|---|
512 |
2132452 |
106498 |
436.2 |
0 |
0.00 |
1024 |
1377836 |
68815 |
563.7 |
0 |
0.00 |
2048 |
810267 |
40468 |
663.0 |
0 |
0.00 |
4096 |
449900 |
22470 |
736.3 |
0 |
0.00 |
8192 |
224468 |
11213 |
734.9 |
0 |
0.00 |
16384 |
154590 |
7723 |
1012.3 |
0 |
0.00 |
32768 |
98350 |
4914 |
1288.2 |
0 |
0.00 |
63000 |
59601 |
2977 |
1500.9 |
0 |
0.00 |
Best Effort
Sample Size (Bytes) |
Total Samples |
Avg Samples/s |
Avg Mbps |
Lost Samples |
Lost Samples (%) |
---|---|---|---|---|---|
512 |
2130701 |
106437 |
436.0 |
0 |
0.00 |
1024 |
1380689 |
68972 |
565.0 |
0 |
0.00 |
2048 |
815582 |
40722 |
667.2 |
0 |
0.00 |
4096 |
451489 |
22555 |
739.1 |
0 |
0.00 |
8192 |
230871 |
11531 |
755.7 |
0 |
0.00 |
16384 |
158906 |
7937 |
1040.3 |
0 |
0.00 |
32768 |
100040 |
4997 |
1309.9 |
0 |
0.00 |
63000 |
60147 |
3003 |
1513.5 |
0 |
0.00 |
Perftest Scripts
To produce these tests, we executed RTI Perftest for C++98. The exact script used can be found here:
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
6export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"
7
8export domain="2"
9export exec_time=20
10export num_reps=1
11export instance_number=100000
12export core=0
13
14# We will use some colors to improve visibility of errors and info messages.
15RED='\033[0;31m'
16GREEN='\033[0;32m'
17YELLOW='\033[0;33m'
18BLUE='\033[0;34m'
19LIGHTBLUE='\033[0;36m'
20NC='\033[0m'
21INFO_TAG="${GREEN}[INFO]:${NC}"
22WARNING_TAG="${YELLOW}[WARNING]:${NC}"
23ERROR_TAG="${RED}[ERROR]:${NC}"
24
25export ip_machine_1="10.2.78.20"
26export ip_machine_2="10.2.78.21"
27export if10Gbps="enp1s0f0"
28export if1Gbps="eno1"
29
30################################################################################
31
32function disable_colors() {
33 export RED=""
34 export GREEN=""
35 export YELLOW=""
36 export NC=""
37 export BLUE=""
38 export LIGHTBLUE=""
39 export INFO_TAG="${GREEN}[INFO]:${NC}"
40 export WARNING_TAG="${YELLOW}[WARNING]:${NC}"
41 export ERROR_TAG="${RED}[ERROR]:${NC}"
42}
43
44function change_domain() {
45 if [[ "$domain" == "1" ]]; then
46 export domain="2"
47 else
48 export domain="1"
49 fi
50}
51
52# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
53function execute_test() {
54
55 local keyed_unkeyed=$1
56 local rel_be=$2
57 local datasizes_test=$3
58 local other_args=$4
59 local name_suffix=$5
60
61 local commands_string_test=$commands_string
62 local tag=""
63
64 if [[ "${keyed_unkeyed}" == "keyed" ]]; then
65 commands_string_test="${commands_string_test} -keyed -instances $instance_number"
66 tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
67 else
68 tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
69 fi
70
71 if [[ "${rel_be}" == "be" ]]; then
72 commands_string_test="${commands_string_test} -bestEffort"
73 tag="${tag}${YELLOW}BE${NC}]"
74 else
75 tag="${tag}${RED}REL${NC}]"
76 fi
77
78 tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"
79
80 local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv
81
82 if [[ "$role" == "pub" ]]; then
83 echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be. ${NC}"
84 fi
85
86 if [[ "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
87 export pre_command_string="taskset -c $core"
88 fi
89
90 if [[ "$LANGUAGE" == "python" ]]; then
91 export pre_command_string="python3 "
92 fi
93
94 # Get the aprox time this will take:
95 total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
96 total_time=$((total_tests * exec_time))
97
98 touch $output_file
99 local no_headers=""
100 local current_test=0
101 for index in $(seq 1 ${num_reps}); do
102 for DATALEN in ${datasizes_test}; do
103 current_test=$((current_test + 1))
104 export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
105 if [[ "$role" == "pub" ]]; then
106 echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
107 echo -e ${BLUE}$command${NC}
108 else
109 echo -e ${LIGHTBLUE}$command${NC}
110 fi
111 if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
112 sleep 3
113 fi
114 if [[ "$raw" == "1" && "$role" == "sub" ]]; then
115 sleep 5
116 fi
117 if [[ "${get_netstat_info}" == "1" ]]; then
118 echo -e "${INFO_TAG} Getting netstat info before"
119 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
120 fi
121 eval $command >> $output_file;
122 if [[ "${get_netstat_info}" == "1" ]]; then
123 echo -e "${INFO_TAG} Getting netstat info after"
124 netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
125 touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
126 python3 $script_location/../../../tools/diff_netstat_output.py \
127 -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
128 -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
129 -d $DATALEN $no_header_netstat \
130 -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
131 rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
132 no_header_netstat=" -nh"
133 fi
134 no_headers=" -noOutputHeaders"
135
136 change_domain
137 done
138 done
139}
140
141################################################################################
142# PARSE COMMAND LINE OPTIONS:
143
144while [ "$1" != "" ]; do
145 case $1 in
146 --executable)
147 executable=$2
148 shift
149 ;;
150 --output-folder)
151 output_folder=$2
152 shift
153 ;;
154 --sub-folder)
155 sub_folder=$2
156 shift
157 ;;
158 --role)
159 export role=$2
160 shift
161 ;;
162 --core)
163 export core=$2
164 shift
165 ;;
166 --test-kind)
167 export lat_thr=$2
168 shift
169 ;;
170 --interface1)
171 export interface=$2
172 shift
173 ;;
174 --interface2)
175 export interface2=$2
176 shift
177 ;;
178 --ip1)
179 export ip1=$2
180 shift
181 ;;
182 --ip2)
183 export ip2=$2
184 shift
185 ;;
186 --repetitions)
187 export num_reps=$2
188 shift
189 ;;
190 --domain)
191 export domain=$2
192 shift
193 ;;
194 --execution-time)
195 export exec_time=$2
196 shift
197 ;;
198 --transport)
199 export transport=$2
200 shift
201 ;;
202 --datalen)
203 export datalen_input=$2
204 shift
205 ;;
206 --file-suffix)
207 export file_suffix=$2
208 shift
209 ;;
210 --executable-suffix)
211 export executable_suffix=$2
212 shift
213 ;;
214 --extra-arguments)
215 export extra_arguments=$2
216 shift
217 ;;
218 --extra-arguments-pub)
219 export extra_arguments_pub=$2
220 shift
221 ;;
222 --extra-arguments-sub)
223 export extra_arguments_sub=$2
224 shift
225 ;;
226 --skip-no-batching)
227 export skip_no_batching="1"
228 ;;
229 --skip-be)
230 export skip_be_tests="1"
231 ;;
232 --skip-rel)
233 export skip_rel_tests="1"
234 ;;
235 --skip-keyed)
236 export skip_keyed_data="1"
237 ;;
238 --skip-large-data)
239 export skip_large_data="1"
240 ;;
241 --large-data)
242 export large_data="1"
243 ;;
244 --keyed)
245 export skip_unkeyed="1"
246 ;;
247 --unkeyed)
248 export skip_keyed_data="1"
249 ;;
250 --no-batching)
251 export no_batching_only="1"
252 ;;
253 --reliable)
254 export skip_be_tests="1"
255 ;;
256 --best-effort)
257 export skip_rel_tests="1"
258 ;;
259 --security)
260 export security_only="$2"
261 shift
262 ;;
263 --micro)
264 export micro="1"
265 ;;
266 --raw | --raw-transport)
267 export raw="1"
268 ;;
269 --tss)
270 export tss="1"
271 ;;
272 --no-colors)
273 export NO_COLORS="1"
274 ;;
275 --language)
276 export LANGUAGE=$2
277 shift
278 ;;
279 --loss-rate)
280 export loss_rate=$2
281 shift
282 ;;
283 --get-netstat-info)
284 export get_netstat_info="1"
285 ;;
286 *)
287 echo -e "unknown parameter \"$1\""
288 exit 255
289 ;;
290 esac
291 shift
292done
293
294if [[ "$NO_COLORS" == "1" ]]; then
295 disable_colors
296fi
297
298export folder_base="$(dirname "${executable}")"/../../..
299
300if [[ $LANGUAGE == "java" || "$LANGUAGE" == "cs" ]]; then
301 export folder_base="$(dirname "${executable}")"/../..
302fi
303if [[ $tss == "1" ]]; then
304 export folder_base="$(dirname "${executable}")"/../../../../..
305fi
306
307if [[ "${executable_suffix}" != "" ]]; then
308 export executable="${executable}${executable_suffix}"
309fi
310
311if [[ "${sub_folder}" != "" ]]; then
312 export output_folder="${output_folder}/${sub_folder}"
313fi
314
315echo -e "${INFO_TAG} Perftest executable is: $executable"
316echo -e "${INFO_TAG} Output folder is: $output_folder"
317
318################################################################################
319
320if [[ "$LANGUAGE" == "python" ]]; then
321 export skip_keyed_data="1"
322 export skip_large_data="1"
323 export skip_be_tests="1"
324 export skip_no_batching="1"
325fi
326
327if [[ "${skip_large_data}" == "1" ]]; then
328 export datasizes_extended=${datasizes}
329elif [[ "${large_data}" == "1" ]]; then
330 export datasizes=${datasizes_extended}
331fi
332
333if [[ "${datalen_input}" != "" ]]; then
334 echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
335 export datasizes=${datalen_input}
336 export datasizes_extended=${datalen_input}
337 if [[ "${no_batching_only}" != "1" ]]; then
338 export skip_large_data="1"
339 fi
340fi
341
342if [[ "$role" != "pub" && "$role" != "sub" ]]; then
343 echo -e "${ERROR_TAG} It must be either publisher or subscriber"
344 exit 255
345fi
346
347if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
348 echo -e "${ERROR_TAG} It must be either lat or thr"
349 exit 255
350fi
351
352if [[ "${interface}" == "" ]]; then
353 echo "Using default nics"
354 export nic_publisher=${ip_machine_1}
355 export nic_subscriber=${ip_machine_2}
356else
357 export nic_publisher=$interface
358 echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
359
360 if [[ "${interface2}" == "" ]]; then
361 export nic_subscriber=$interface
362 else
363 export nic_subscriber=$interface2
364 fi
365 echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
366
367 if [[ "${ip1}" != "" ]]; then
368 export ip_publisher=$ip1
369 echo "Using ip_publisher: ${ip_publisher}"
370 fi
371
372 if [[ "${ip2}" != "" ]]; then
373 export ip_subscriber=$ip2
374 echo "Using ip_subscriber: ${ip_subscriber}"
375 fi
376
377fi
378
379export transport_string="-transport $transport"
380
381if [[ "$transport" == "UDPv4" ]]; then
382
383 export transport_string_pub="$transport_string -nic $nic_publisher"
384 export transport_string_sub="$transport_string -nic $nic_subscriber"
385
386 if [[ "$micro" == "1" || "$raw" == "1" ]]; then
387 export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
388 export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
389 fi
390
391elif [[ "$transport" == "TCP" ]]; then
392 export transport_string_pub="$transport_string \
393 -nic $nic_publisher \
394 -peer 0@tcpv4_lan://${ip_subscriber}:7400"
395 export transport_string_sub="$transport_string \
396 -nic $nic_subscriber \
397 -peer 0@tcpv4_lan://${ip_publisher}:7400"
398elif [[ "$transport" == "TLS" ]]; then
399 export transport_string_pub="$transport_string \
400 -nic $nic_publisher \
401 -peer tlsv4_lan://${ip_subscriber}:7400"
402 export transport_string_sub="$transport_string \
403 -nic $nic_subscriber \
404 -peer tlsv4_lan://${ip_publisher}:7400"
405elif [[ "$transport" == "UDPv4_WAN" ]]; then
406 export transport_string_pub="$transport_string \
407 -nic $nic_publisher \
408 -transportPublicAddress $ip_publisher:7400"
409 export transport_string_sub="$transport_string \
410 -nic $nic_subscriber \
411 -peer 0@udpv4_wan://${ip_publisher}:7400"
412else
413 export transport_string_pub="$transport_string"
414 export transport_string_sub="$transport_string"
415fi
416
417################################################################################
418
419export pub_string="-pub \
420 ${transport_string_pub} \
421 -noPrintIntervals \
422 -executionTime $exec_time"
423
424if [[ ${lat_thr} == "lat" ]]; then
425 export pub_string="$pub_string \
426 -latencyTest"
427fi
428
429export sub_string="-sub \
430 ${transport_string_sub} \
431 -noPrintIntervals"
432
433if [[ "$role" == "pub" ]]; then
434 echo -e "$INFO_TAG Publisher side running"
435 export commands_string=${pub_string}
436 export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
437else
438 echo -e "$INFO_TAG Subscriber side running"
439 export commands_string=${sub_string}
440 export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
441fi
442
443###############################################################################
444
445echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
446sudo /set_${lat_thr}_mode.sh
447sleep 5
448
449echo -e "${INFO_TAG} Disabling any loss rate"
450sudo tc qdisc add dev $nic_publisher root netem loss 0%
451sudo tc qdisc del dev $nic_publisher root netem loss 0%
452
453if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
454 echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
455 sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
456fi
457
458cd $folder_base
459echo -e "${INFO_TAG} Folder Base is: $PWD"
460mkdir -p $output_folder
461
462# Tests that may use batching (when doing throughput tests)
463if [[ ${no_batching_only} != "1" ]]; then
464
465 # UNKEYED
466 if [[ "${skip_unkeyed}" == "" ]]; then
467
468 # RELIABLE
469 if [[ "${skip_rel_tests}" == "" ]]; then
470 execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
471 fi
472
473 # BEST EFFORT
474 if [[ "${skip_be_tests}" == "" ]]; then
475 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
476 fi
477 fi
478
479 # KEYED
480 if [[ "${skip_keyed_data}" == "" ]]; then
481
482 # RELIABLE
483 if [[ "${skip_rel_tests}" == "" ]]; then
484 execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
485 fi
486
487 # BEST EFFORT
488 if [[ "${skip_be_tests}" == "" ]]; then
489 execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
490 fi
491 fi
492
493fi
494
495if [[ "${skip_no_batching}" == "" || "${no_batching_only}" == "1" ]]; then
496 no_batching_tests="1"
497fi
498
499# Tests that will not use batching
500if [[ "${lat_thr}" == "thr" && "${no_batching_tests}" == "1" ]]; then
501
502 if [[ "$role" == "pub" ]]; then
503 export commands_string="${commands_string} -batchSize 0"
504 fi
505
506 # UNKEYED
507 if [[ "${skip_unkeyed}" == "" ]]; then
508
509 # RELIABLE
510 if [[ "${skip_rel_tests}" == "" ]]; then
511 execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
512 fi
513
514 # BEST EFFORT
515 if [[ "${skip_be_tests}" == "" ]]; then
516 execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
517 fi
518 fi
519fi
520
521if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
522 echo -e "${INFO_TAG} Disabling loss rate"
523 sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
524fi
1#!/bin/bash
2filename=$0
3script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)
4
5export input_params=$@
6
7export datalen_input=""
8while [ "$1" != "" ]; do
9 case $1 in
10 --executable)
11 executable=$2
12 shift
13 ;;
14 --test-kind)
15 export lat_thr=$2
16 shift
17 ;;
18 --datalen)
19 export datalen_input=$2
20 shift
21 ;;
22 *)
23 ;;
24 esac
25 shift
26done
27
28
29if [[ ${datalen_input} == "" ]]; then
30 export datalen_input="512 1024 2048 4096 8192 16384 32768 63000"
31fi
32
33# If the test is thr, then force no-batching. We need to handle this use-case.
34
35
36${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
37 --skip-keyed --skip-no-batching \
38 --file-suffix "_zlib${suffix}" \
39 --datalen "${datalen_input}" \
40 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
41 --extra-arguments "-compressionId ZLIB \
42 -compressionThreshold 0 \
43 -compressionLevel 10"
44
45
46
47if [[ "$lat_thr" == "thr" ]]; then
48 export thr_command="-batchSize 0"
49 export suffix="_noBatch"
50fi
51
52${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
53 --skip-no-batching --skip-keyed --skip-no-batching \
54 --file-suffix "_lz4"\
55 --datalen "${datalen_input}" \
56 --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages $thr_command" \
57 --extra-arguments "-compressionId LZ4 \
58 -compressionThreshold 0 \
59 -compressionLevel 5"
60
61# export folder_base="$(dirname "${executable}")"/../../..
62# export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure
63
64# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
65# --skip-no-batching --skip-keyed --skip-be \
66# --datalen "${datalen_input}" \
67# --file-suffix "_no_compression_security_data_encrypt"\
68# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
69# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml"
70
71# ${script_location}/../base_script/script.sh $input_params --transport UDPv4 \
72# --skip-no-batching --skip-keyed --skip-be \
73# --datalen "${datalen_input}" \
74# --file-suffix "_zlib_security_data_encrypt"\
75# --extra-arguments-pub "-loadDataFromFile /home/perfuser/rtiperftest/payload_test/messages" \
76# --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_EncryptData.xml \
77# -compressionId ZLIB \
78# -compressionThreshold 0 \
79# -compressionLevel 5"
Test Hardware
The following hardware was used to perform these tests:
Linux Nodes
Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0
Switch
Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)
Compression Bandwidth Savings
The following table contains the bandwidth savings gain by enabling compression with ZLIB, which is one of algorithms with a higher compression ratio and the only one that can be used in combination with batching.
The compression settings used for these tests are as follows:
compression_ids: ZLIB
writer_compression_level: 10 (best compression possible, default setting)
writer_compression_threshold: 0 (try to compress all samples)
The data used for this test is the same as used in the previous latency and throughput tests. In this test, each sample has its own size.
This test uses the compression example available in the github example repository: https://github.com/rticommunity/rticonnextdds-examples/tree/develop/examples/connext_dds/compression.
Batching |
NONE Compression (Mb) |
ZLIB Compression(Mb) |
Diff Saved Mb |
Diff Saved % |
---|---|---|---|---|
Disabled |
70.59 |
53.74 |
16.85 |
23.87 |
2 samples |
69.40 |
37.08 |
32.32 |
46.57 |
5 samples |
69.26 |
22.08 |
47.18 |
68.12 |
10 samples |
69.25 |
17.02 |
52.23 |
75.42 |
These results show up to a 75% reduction in network bandwidth utilization by using ZLIB in combination with batching, in a real use-case scenario.