1.5.1.1. Different Security Governance Configuration Levels

The following one-to-one tests have been performed by executing an RTI Perftest C++98 Publisher and Subscriber between two nodes, connected to a switch via Ethernet. The communication has been restricted to a single interface and the transport has been set to UDPv4.

These tests are equivalent to the ones performed in the Core Libraries UDPv4 section (Unkeyed, UDPv4 10Gbps Network, C++98), but additionally enabling different Security Profiles (using different Governance configurations). The “Not using security libraries” values in the graphs below are equivalent to the Core Libraries numbers, so you can compare Connext performance with and without the Security Plugins.

Find information about the hardware, network, and command-line parameters after each of the tests.

Linux x64 (10Gbps Network)Linux ARM v8 (1Gbps Network)

Minimum LatencyMaximum Throughput

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes in a 10Gbps network. The numbers have been taken using strict reliable reliability for all the different Security Profiles (described below).

Note

We use the median (50th percentile) instead of the average in order to get a more stable measurement that does not account for spurious outliers. We also calculate the average value and other percentile values, which can be seen in the Detailed Statistics section below.

Detailed Statistics

The following tables contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

• Not using security libraries

Sample Size (Bytes)	Ave (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	18	1.0	17	56	17	18	21	47	56
64	17	1.1	16	73	17	18	21	46	73
128	17	1.3	16	61	17	18	24	47	61
256	19	1.8	17	59	18	20	27	48	59
512	20	7.9	17	5057	19	22	31	50	5057
1024	25	4.8	18	65	26	28	41	57	65
2048	29	27.3	20	4871	30	33	41	66	4871
4096	31	18.9	25	4868	30	36	51	71	4868
8192	43	67.9	35	5031	37	56	68	4380	5031
16384	87	26.5	46	169	86	125	138	157	169
32768	141	20.6	70	267	125	172	183	202	267
63000	186	2.6	107	275	186	187	192	215	275

• No protection

Sample Size (Bytes)	Ave (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	18	1.0	17	59	17	18	21	47	59
64	17	1.1	16	62	17	18	21	47	62
128	17	1.2	16	64	17	18	23	46	64
256	19	1.6	17	70	18	20	26	48	70
512	20	2.4	18	84	19	22	28	49	84
1024	25	8.2	18	3296	26	28	41	57	3296
2048	29	30.1	21	4899	30	33	41	71	4899
4096	31	22.4	25	4965	30	36	54	70	4965
8192	43	67.4	35	4871	37	56	68	4364	4871
16384	87	27.0	46	168	86	125	138	154	168
32768	137	19.7	74	238	125	168	182	198	238
63000	162	13.7	107	215	159	186	189	208	215

• RTPS Sign

Sample Size (Bytes)	Ave (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	24	1.2	23	65	24	25	28	54	65
64	24	1.2	23	67	24	24	28	54	67
128	24	1.2	23	75	24	25	28	54	75
256	25	1.5	24	72	25	26	33	56	72
512	26	1.7	24	61	25	27	34	55	61
1024	27	16.0	25	4638	26	29	40	58	4638
2048	31	6.6	28	3141	30	33	46	65	3141
4096	35	8.3	33	4041	34	38	49	67	4041
8192	48	6.1	44	158	46	56	76	110	158
16384	103	29.4	57	197	96	143	170	183	197
32768	167	23.2	84	259	172	190	192	223	259
63000	200	15.0	123	308	207	209	212	261	308

• RTPS Encrypt

Sample Size (Bytes)	Ave (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	25	1.2	23	62	24	25	28	54	62
64	25	1.3	24	94	25	25	29	56	94
128	25	1.2	23	85	25	25	29	55	85
256	25	1.5	24	68	25	26	33	55	68
512	26	1.8	24	68	26	27	35	56	68
1024	28	15.5	26	4796	27	30	41	60	4796
2048	31	4.9	28	2077	30	33	45	65	2077
4096	36	9.7	34	4401	35	39	51	68	4401
8192	50	12.8	46	4986	48	56	80	116	4986
16384	107	31.3	60	230	99	148	173	187	230
32768	182	25.3	94	1308	195	196	200	270	1308
63000	212	16.9	136	1322	219	221	227	736	1322

• RTPS Sign with Original Auth, Data Encrypt

Sample Size (Bytes)	Ave (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	30	5.7	29	3098	30	30	34	60	3098
64	30	4.1	29	2164	30	31	35	60	2164
128	30	5.6	29	3055	30	31	35	60	3055
256	31	1.4	29	70	31	31	36	61	70
512	31	1.7	30	77	31	32	40	62	77
1024	34	2.2	32	89	33	35	44	64	89
2048	36	22.9	34	4736	36	38	46	74	4736
4096	43	3.2	40	98	42	43	62	74	98
8192	56	6.7	53	2050	55	58	82	115	2050
16384	117	31.7	69	213	116	163	185	196	213
32768	185	21.2	98	245	187	208	211	238	245
63000	229	16.5	153	1318	236	239	247	310	1318

• RTPS Sign, Submessage Encrypt with Original Auth, Data Encrypt

Sample Size (Bytes)	Ave (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	34	1.5	32	77	34	35	40	65	77
64	34	1.4	33	75	34	35	40	65	75
128	35	1.5	33	78	34	36	40	65	78
256	35	1.5	33	78	35	36	41	66	78
512	36	1.5	34	77	35	37	42	67	77
1024	37	6.1	35	2964	37	38	45	69	2964
2048	41	20.4	39	4707	40	42	49	75	4707
4096	49	4.8	45	118	48	50	73	92	118
8192	64	19.3	60	5067	62	64	105	125	5067
16384	128	34.1	79	218	127	176	194	208	218
32768	202	21.9	118	264	205	225	229	257	264
63000	258	19.1	182	1185	267	270	277	319	1185

• RTPS Sign, Submessage Encrypt

Sample Size (Bytes)	Ave (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	28	1.3	27	75	28	29	33	59	75
64	28	6.6	27	3703	28	29	33	58	3703
128	29	1.4	28	74	29	30	35	60	74
256	30	1.6	28	71	29	31	37	60	71
512	30	14.5	28	4740	30	31	39	61	4740
1024	32	7.1	30	3624	31	34	43	67	3624
2048	35	3.3	32	74	34	38	60	67	74
4096	41	32.9	38	4705	40	42	52	484	4705
8192	54	4.4	51	141	53	56	78	112	141
16384	116	32.4	68	216	115	163	182	195	216
32768	195	24.0	102	1372	206	208	213	280	1372
63000	225	17.0	151	276	235	238	244	272	276

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The script used to execute the tests can be found here:

#!/bin/bash
filename=$0
script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)

export datasizes="32 64 128 256 512 1024 2048 4096 8192 16384 32768 63000"
export datasizes_extended="${datasizes} 100000 500000 1048576 1548576 4194304 10485760"

export domain="2"
export exec_time=20
export num_reps=1
export instance_number=100000
export core=0
export run_batching_tests="1"
export run_no_batching_tests="1"

# We will use some colors to improve visibility of errors and info messages.
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
LIGHTBLUE='\033[0;36m'
NC='\033[0m'
INFO_TAG="${GREEN}[INFO][$role]:${NC}"
WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
ERROR_TAG="${RED}[ERROR][$role]:${NC}"

################################################################################

function disable_colors() {
    export RED=""
    export GREEN=""
    export YELLOW=""
    export NC=""
    export BLUE=""
    export LIGHTBLUE=""
    export INFO_TAG="${GREEN}[INFO][$role]:${NC}"
    export WARNING_TAG="${YELLOW}[WARNING][$role]:${NC}"
    export ERROR_TAG="${RED}[ERROR][$role]:${NC}"
}

function change_domain() {
    if [[ "$domain" == "1" ]]; then
        export domain="2"
    else
        export domain="1"
    fi
}

# Function to process and append lines to the output file
function append_to_output_file() {
    local line=${1?line required}; readonly line
    local append=$(echo "${2?append required}" | tr -s ' '); readonly append
    local file=${3?file required}; readonly file
    if [[ "${line: -1}" == $'\n' ]]; then
        line="${line::-1}"
    fi
    echo "${line}${append}" >> "$file"
}

# Usage: execute_test <keyed/unkeyed> <rel/be> <datasizes> <batchSize>
function execute_test() {

    local keyed_unkeyed=$1
    local rel_be=$2
    local datasizes_test=$3
    local other_args=$4
    local name_suffix=$5

    local commands_string_test=$commands_string
    local tag=""

    if [[ "${keyed_unkeyed}" == "keyed" ]]; then
        commands_string_test="${commands_string_test} -keyed -instances $instance_number"
        tag="[${YELLOW}${transport}${NC}|${BLUE}K${NC}|"
    else
        tag="[${YELLOW}${transport}${NC}|${LIGHTBLUE}UK${NC}|"
    fi

    if [[ "${rel_be}" == "be" ]]; then
        commands_string_test="${commands_string_test} -bestEffort"
        tag="${tag}${YELLOW}BE${NC}]"
    else
        tag="${tag}${RED}REL${NC}]"
    fi

    # If batch_size is set, we will add it to the command line of the publisher side.
    if [[ "$batch_size" != "" && "$role" == "pub" ]]; then
        commands_string_test="${commands_string_test} -batchSize $batch_size"
    fi

    tag="${tag}[${LIGHTBLUE}${lat_thr}${NC}]"

    local output_file=$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}.csv

    if [[ "$role" == "pub" ]]; then
        echo -e "${YELLOW}[TEST]: $keyed_unkeyed, $rel_be, Is a no-batching test = $no_batching_tests. ${NC}"
    fi

    if [[ "$thread_cpu_affinity" != "" ]]; then
        commands_string_test="${commands_string_test} -threadCPUAffinity $thread_cpu_affinity"
    fi

    local pre_command_string=""

    if [[ "$thread_priorities" != "" ]]; then
        echo -e "${WARNING_TAG} Thread priorities enabled, this requires using sudo"
        # When using sudo, the LD_LIBRARY_PATH is emptied, hence we need to set it again in the command itself
        export LD_LIBRARY_PATH_COPIED="$LD_LIBRARY_PATH"
        pre_command_string="sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH_COPIED "
        commands_string_test="${commands_string_test} -threadPriorities $thread_priorities"
    fi

    if [[ "$pin_memory" != "" ]]; then
        commands_string_test="${commands_string_test} -pinMemory"
    fi

    if [[ "$no_taskset" == "" && "$thread_cpu_affinity" == "" && "$LANGUAGE" != "java" && "$LANGUAGE" != "cs" ]]; then
        pre_command_string="$pre_command_string taskset -c $core"
    fi

    if [[ "$LANGUAGE" == "python" ]]; then
        pre_command_string="$pre_command_string python3 "
    fi

    if [[ "$DOCKER" == "1" ]]; then
        pre_command_string="$pre_command_string docker run --net=host -v /home/perfuser/rti_license_connextpro.dat:/opt/rti.com/rti_connext_dds-7.3.0/rti_license.dat rticom/perftest:7.3.0-EAR "
        executable=""
    fi

    # Get the aprox time this will take:
    total_tests=$((`wc -w <<< "$datasizes_test"` * num_reps))
    total_time=$((total_tests * exec_time))

    touch $output_file
    local no_headers=""
    local current_test=0
    for index in $(seq 1 ${num_reps}); do
        for DATALEN in ${datasizes_test}; do
            current_test=$((current_test + 1))

            if [[ ! -s $output_file ]]; then
                echo -e "${INFO_TAG} Output file is empty, filling the header."
                no_headers=""
            else
                echo -e "${INFO_TAG} Output file is not empty."
                no_headers=" -noOutputHeaders"
            fi

            export command="$pre_command_string $executable -domain $domain -dataLen $DATALEN $commands_string_test $other_args $no_headers"
            if [[ "$role" == "pub" ]]; then
                echo -e "Test ${tag} (${current_test}/${total_tests}) -- Total time = ${total_time}s"
                echo -e ${BLUE}$command${NC}
            else
                echo -e ${LIGHTBLUE}$command${NC}
            fi

            # In certain cases we need to wait a bit before running the test, this is
            # because the previous test might not be finished on the other side yet, or because the
            # discovery mechanism does not work like in connext DDS.
            if [[ "$LANGUAGE" == "cs" && "$role" == "pub" ]]; then
                sleep 3
            fi
            if [[ "$raw" == "1" && "$role" == "sub" ]]; then
                sleep 5
            fi

            # Gather netstat info before running the test
            if [[ "${get_netstat_info}" == "1" ]]; then
                echo -e "${INFO_TAG} Getting netstat info before"
                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt
            fi

            # Execute the command and capture the output
            if [[ $LANGUAGE == "c++98" || $LANGUAGE == "" ]]; then
                echo -e "${INFO_TAG} Using C++98, using -outputFile option"
                touch ${output_file}_tmp_line
                chmod 777 ${output_file}_tmp_line
                eval $command -outputFile ${output_file}_tmp_line
            else
                eval $command > ${output_file}_tmp_line
            fi

            number_of_lines=$(wc -l < ${output_file}_tmp_line)
            echo -e "${INFO_TAG} Size of the output text: $number_of_lines lines"

            # Check if this is the first test for this output file.
            local is_first_test=0
            if [[ ! -s $output_file ]]; then
                is_first_test=1
            fi
            local expected_lines=$((is_first_test+1))

            if [[ $number_of_lines -ne ${expected_lines} ]]; then
                echo -e "${WARNING_TAG} The output text should have ${expected_lines} lines, but it has ${number_of_lines}."
                echo -e "${RED} The content is: \n\"\"\""
                cat ${output_file}_tmp_line
                echo -e "\"\"\""
                echo -e "Not adding it to the output file. ${NC}"
            else
                if [[ $number_of_lines -gt 1 ]]; then
                    header_line=$(head -n 1 "${output_file}_tmp_line")
                    echo -e "${INFO_TAG} Header line is: $header_line"
                fi
                result_line=$(tail -n 1 "${output_file}_tmp_line")
                echo -e "${INFO_TAG} Result line is: $result_line"

                # If this is the first test, we need to add the header line to the output file
                if [[ $is_first_test -eq 1 ]]; then
                    append_to_output_file "$header_line" ", command-line" $output_file
                fi

                command=$(echo $command | sed 's/,/:comma:/g')
                # Remove the `"` characters from the command line
                command=$(echo $command | sed 's/\"//g')

                # Append the result line to the output file
                append_to_output_file "$result_line" ", $command" $output_file
            fi

            # Always remove the temporary file
            rm -rf ${output_file}_tmp_line

            # Gather netstat info after running the test
            if [[ "${get_netstat_info}" == "1" ]]; then
                echo -e "${INFO_TAG} Getting netstat info after"
                netstat -s -u | grep -e "error" -e "packet" > $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt
                touch "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
                python3 $script_location/../../../tools/diff_netstat_output.py \
                    -n $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_after.txt \
                    -o $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_before.txt \
                    -d $DATALEN $no_header_netstat \
                    -csv >> "$output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat.csv"
                rm -rf $output_folder/${lat_thr}_${role}_${keyed_unkeyed}_${rel_be}${name_suffix}_netstat_*.txt
                no_header_netstat=" -nh"
            fi

            change_domain
        done
    done
}

################################################################################
# PARSE COMMAND LINE OPTIONS:

while [ "$1" != "" ]; do
    case $1 in
        --executable)
            executable=$2
            shift
            ;;
        --docker)
            DOCKER="1"
            ;;
        --output-folder)
            output_folder=$2
            shift
            ;;
        --role)
            export role=$2
            shift
            ;;
        --core)
            export core=$2
            shift
            ;;
        --assign-core-list | --thread-cpu-affinity)
            export thread_cpu_affinity=$2
            shift
            ;;
        --assign-thread-priorities | --thread-priorities)
            export thread_priorities=$2
            shift
            ;;
        --pin-memory)
            export pin_memory="1"
            ;;
        --test-kind)
            export lat_thr=$2
            shift
            ;;
        --interface1)
            export interface=$2
            shift
            ;;
        --interface2)
            export interface2=$2
            shift
            ;;
        --ip1)
            export ip1=$2
            shift
            ;;
        --ip2)
            export ip2=$2
            shift
            ;;
        --repetitions)
            export num_reps=$2
            shift
            ;;
        --domain)
            export domain=$2
            shift
            ;;
        --execution-time)
            export exec_time=$2
            shift
            ;;
        --transport)
            export transport=$2
            shift
            ;;
        --datalen)
            export datalen_input=$2
            shift
            ;;
        --file-suffix)
            export file_suffix=$2
            shift
            ;;
        --folder-suffix)
            export folder_suffix=$2
            shift
            ;;
        --sub-folder)
            sub_folder=$2
            shift
            ;;
        --executable-suffix)
            export executable_suffix=$2
            shift
            ;;
        --extra-arguments)
            export extra_arguments=$2
            shift
            ;;
        --extra-arguments-pub)
            export extra_arguments_pub=$2
            shift
            ;;
        --extra-arguments-sub)
            export extra_arguments_sub=$2
            shift
            ;;
        --skip-be)
            export skip_be_tests="1"
            ;;
        --skip-rel)
            export skip_rel_tests="1"
            ;;
        --skip-keyed)
            export skip_keyed_data="1"
            ;;
        --skip-large-data)
            export skip_large_data="1"
            ;;
        --large-data)
            export large_data="1"
            ;;
        --keyed)
            export skip_unkeyed="1"
            ;;
        --unkeyed)
            export skip_keyed_data="1"
            ;;
        --no-batching | --skip-batching)
            export run_batching_tests="0"
            export run_no_batching_tests="1"
            ;;
        --skip-no-batching)
            export run_batching_tests="1"
            export run_no_batching_tests="0"
            ;;
        --batch-size)
            export batch_size=$2
            shift
            ;;
        --reliable)
            export skip_be_tests="1"
            ;;
        --best-effort)
            export skip_rel_tests="1"
            ;;
        --security-gov)
            export security_only="$2"
            shift
            ;;
        --micro)
            export micro="1"
            ;;
        --cert)
            export cert="1"
            ;;
        --raw | --raw-transport)
            export raw="1"
            ;;
        --tss)
            export tss="1"
            ;;
        --no-colors)
            export NO_COLORS="1"
            ;;
        --language)
            export LANGUAGE=$2
            shift
            ;;
        --loss-rate)
            export loss_rate=$2
            shift
            ;;
        --get-netstat-info | --netstat)
            export get_netstat_info="1"
            ;;
        --reduced-data-sizes-set)
            export REDUCED_DATA_SIZES_SET="1"
            ;;
        --dont-change-tuned-settings)
            export dont_change_tuned_settings="1"
            ;;
        --no-taskset)
            export no_taskset=1
            ;;
        *)
            echo -e "unknown parameter \"$1\""
            exit 255
            ;;
    esac
    shift
done

if [[ "$NO_COLORS" == "1" ]]; then
    disable_colors
fi

export folder_base="$(dirname "${executable}")"/../../..

if [[ $LANGUAGE == "java"  || "$LANGUAGE" == "cs" ]]; then
    export folder_base="$(dirname "${executable}")"/../..
fi
if [[ $tss == "1" ]]; then
    export folder_base="$(dirname "${executable}")"/../../../../..
fi

if [[ "${executable_suffix}" != "" ]]; then
    export executable="${executable}${executable_suffix}"
fi

if [[ -n "${folder_suffix}" ]]; then
    export output_folder="${output_folder}${folder_suffix}"
fi

if [[ -n "${sub_folder}" ]]; then
    export output_folder="${output_folder}/${sub_folder}"
fi

echo -e "${INFO_TAG} Perftest executable is: $executable"
echo -e "${INFO_TAG} Output folder is: $output_folder"

################################################################################

if [[ "$LANGUAGE" == "python" ]]; then
    export skip_keyed_data="1"
    export skip_large_data="1"
    export skip_be_tests="1"
    export run_no_batching_tests="0"
fi

if [[ "${skip_large_data}" == "1" ]]; then
    export datasizes_extended=${datasizes}
elif [[ "${large_data}" == "1" ]]; then
    export datasizes=${datasizes_extended}
fi

if [[ "${datalen_input}" != "" ]]; then
    echo -e "${YELLOW}[TEST] Testing only for ${datalen_input}${NC}"
    export datasizes=${datalen_input}
    export datasizes_extended=${datalen_input}
    if [[ "${run_no_batching_tests}" == "1" && "${run_batching_tests}" == "0" ]]; then
        export skip_large_data="1"
    fi
else 
    if [[ "${REDUCED_DATA_SIZES_SET}" != "" ]]; then
        echo -e "${YELLOW}[TEST] Testing Reduced set of datasizes ${NC}"
        export datasizes="32 128 512 2048 8192 32768 63000"
        export datasizes_extended="${datasizes} 102400 1048576 10485760"
    fi
fi

if [[ "$role" != "pub" && "$role" != "sub" ]]; then
    echo -e "${ERROR_TAG} It must be either publisher or subscriber"
    exit 255
fi

if [[ "$lat_thr" != "thr" && "$lat_thr" != "lat" ]]; then
    echo -e "${ERROR_TAG} It must be either lat or thr"
    exit 255
fi

if [[ "${interface}" == "" ]]; then
    echo "Using default nics"
    export nic_publisher=${ip_machine_1}
    export nic_subscriber=${ip_machine_2}
elif [[ "${interface}" == "both" ]]; then
    export nic_publisher="enp1s0f0,eno1"
    export nic_subscriber="enp1s0f0,eno1"
    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"
    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"
else
    export nic_publisher=$interface
    echo -e "${INFO_TAG} Using nic_publisher: ${nic_publisher}"

    if [[ "${interface2}" == "" ]]; then
        export nic_subscriber=$interface
    else
        export nic_subscriber=$interface2
    fi
    echo -e "${INFO_TAG} Using nic_subscriber: ${nic_subscriber}"

    if [[ "${ip1}" != "" ]]; then
        export ip_publisher=$ip1
        echo "Using ip_publisher: ${ip_publisher}"
    fi

    if [[ "${ip2}" != "" ]]; then
        export ip_subscriber=$ip2
        echo "Using ip_subscriber: ${ip_subscriber}"
    fi

fi

export transport_string="-transport $transport"

if [[ "$transport" == "UDPv4" ]]; then

    export transport_string_pub="$transport_string -nic $nic_publisher"
    export transport_string_sub="$transport_string -nic $nic_subscriber"

    if [[ "$raw" == "1" ]]; then
        export transport_string_pub="$transport_string_pub -peer ${ip_subscriber}"
        export transport_string_sub="$transport_string_sub -peer ${ip_publisher}"
    fi

    if [[ "$micro" == "1" || "$cert" == "1" ]]; then
        export transport_string_pub="$transport_string_pub -peer _udp://${ip_subscriber}"
        export transport_string_sub="$transport_string_sub -peer _udp://${ip_publisher}"
    fi

elif [[ "$transport" == "TCP" ]]; then
    export transport_string_pub="$transport_string \
        -nic $nic_publisher \
        -peer 0@tcpv4_lan://${ip_subscriber}:7400"
    export transport_string_sub="$transport_string \
        -nic $nic_subscriber \
        -peer 0@tcpv4_lan://${ip_publisher}:7400"
elif [[ "$transport" == "TLS" ]]; then
    export transport_string_pub="$transport_string \
        -nic $nic_publisher \
        -peer tlsv4_lan://${ip_subscriber}:7400"
    export transport_string_sub="$transport_string \
        -nic $nic_subscriber \
        -peer tlsv4_lan://${ip_publisher}:7400"
elif [[ "$transport" == "UDPv4_WAN" ]]; then
    export transport_string_pub="$transport_string \
        -nic $nic_publisher \
        -transportPublicAddress $ip_publisher:7400"
    export transport_string_sub="$transport_string \
        -nic $nic_subscriber \
        -peer 0@udpv4_wan://${ip_publisher}:7400"
else
    export transport_string_pub="$transport_string"
    export transport_string_sub="$transport_string"
fi

################################################################################

export pub_string="-pub \
        ${transport_string_pub} \
        -noPrintIntervals \
        -executionTime $exec_time"

if [[ ${lat_thr} == "lat" ]]; then
    export pub_string="$pub_string \
        -latencyTest"
fi

export sub_string="-sub \
        ${transport_string_sub} \
        -noPrintIntervals"

if [[ "$role" == "pub" ]]; then
    echo -e "$INFO_TAG Publisher side running"
    export commands_string=${pub_string}
    export extra_arguments="${extra_arguments} ${extra_arguments_pub}"
else
    echo -e "$INFO_TAG Subscriber side running"
    export commands_string=${sub_string}
    export extra_arguments="${extra_arguments} ${extra_arguments_sub}"
fi

###############################################################################

if [[ "$dont_change_tuned_settings" != "1" ]]; then
    echo -e "${INFO_TAG} Executing: /set_${lat_thr}_mode.sh"
    sudo /set_${lat_thr}_mode.sh
    sleep 5
fi

echo -e "${INFO_TAG} Disabling any loss rate"
sudo tc qdisc add dev $nic_publisher root netem loss 0%
sudo tc qdisc del dev $nic_publisher root netem loss 0%

if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
    echo -e "${INFO_TAG} Setting loss rate to ${loss_rate}%"
    sudo tc qdisc add dev $nic_publisher root netem loss $loss_rate%
fi

cd $folder_base
echo -e "${INFO_TAG} Folder Base is: $PWD"
mkdir -p $output_folder

if [[ "${batch_size}" != "" ]]; then
    if [[ "${lat_thr}" == "thr" ]]; then
        if [[ "${batch_size}" -eq 0 ]]; then
            echo -e "${INFO_TAG} Batch size is set to 0"
            export run_batching_tests="0"
            export run_no_batching_tests="1"
        else
            echo -e "${INFO_TAG} Batch size is set to ${batch_size}"
            export run_batching_tests="1"
            export run_no_batching_tests="0"
        fi
    else
        echo -e "${INFO_TAG} Batch size is set to ${batch_size}. This value will be ignored for latency tests."
        unset batch_size
    fi
fi

# Tests that may use batching (when doing throughput tests). Also Latency tests.
if [[ ${run_batching_tests} == "1" ]]; then

    # UNKEYED
    if [[ "${skip_unkeyed}" == "" ]]; then

        # RELIABLE
        if [[ "${skip_rel_tests}" == "" ]]; then
            execute_test "unkeyed" "rel" "${datasizes_extended}" "${extra_arguments}" "$file_suffix"
        fi

        # BEST EFFORT
        if [[ "${skip_be_tests}" == "" ]]; then
            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
        fi
    fi

    # KEYED
    if [[ "${skip_keyed_data}" == "" ]]; then

        # RELIABLE
        if [[ "${skip_rel_tests}" == "" ]]; then
            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "$file_suffix"
        fi

        # BEST EFFORT
        if [[ "${skip_be_tests}" == "" ]]; then
            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "$file_suffix"
        fi
    fi

fi

# Tests that will not use batching
if [[ "${lat_thr}" == "thr" && "${run_no_batching_tests}" == "1" ]]; then

    if [[ "$role" == "pub" ]]; then
        export commands_string="${commands_string} -batchSize 0"
    fi

    # UNKEYED
    if [[ "${skip_unkeyed}" == "" ]]; then

        # RELIABLE
        if [[ "${skip_rel_tests}" == "" ]]; then
            execute_test "unkeyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
        fi

        # BEST EFFORT
        if [[ "${skip_be_tests}" == "" ]]; then
            execute_test "unkeyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
        fi
    fi

    # KEYED
    if [[ "${skip_keyed_data}" == "" ]]; then

        # RELIABLE
        if [[ "${skip_rel_tests}" == "" ]]; then
            execute_test "keyed" "rel" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
        fi

        # BEST EFFORT
        if [[ "${skip_be_tests}" == "" ]]; then
            execute_test "keyed" "be" "${datasizes}" "${extra_arguments}" "_noBatch${file_suffix}"
        fi
    fi

fi

if [[ "$role" == "pub" && "${loss_rate}" != "" ]]; then
    echo -e "${INFO_TAG} Disabling loss rate"
    sudo tc qdisc del dev $nic_publisher root netem loss $loss_rate%
fi

#!/bin/bash
filename=$0
script_location=$(cd "$(dirname "$filename")" || exit 255; pwd)

export input_params=$@

while [ "$1" != "" ]; do
    case $1 in
        --executable)
            executable=$2
            shift
            ;;
        --security-gov)
            export security_only=$2
            shift
            ;;
        --ci)
            export CI="1"
            ;;
        *)
            ;;
    esac
    shift
done

echo $security_only

export folder_base="$(dirname "${executable}")"/../../..
export PATH_TO_GOVERNANCE_FILES_FOLDER=$folder_base/resource/secure

if [[ "${CI}" == "" ]]; then

    if [[ "${security_only}" == "none" || "${security_only}" == "" ]]; then
        echo -e "[Calling base_script/script.sh] -- No Security"
        "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
             --skip-be --skip-no-batching \
            --file-suffix "_security_none"
        sleep 5;
    fi

    if [[ "${security_only}" == "rtps_sign" || "${security_only}" == "" ]]; then
        echo -e "[Calling base_script/script.sh] -- RTPS Sign"
        "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
             --skip-be --skip-no-batching \
            --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_RTPSSign.xml " \
            --file-suffix "_security_rtps_sign"
    fi

    if [[ "${security_only}" == "rtps_sign_submessage_encrypt" || "${security_only}" == "" ]]; then
        echo -e "[Calling base_script/script.sh] -- RTPS Sign, Submessage Encrypt"
        "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
             --skip-be --skip-no-batching \
            --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_SignEncryptSubmessage.xml " \
            --file-suffix "_security_rtps_sign_submessage_encrypt"
    fi

fi

if [[ "${security_only}" == "no_protection" || "${security_only}" == "" ]]; then
    echo -e "[Calling base_script/script.sh] -- No Protection"
    "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
         --skip-be --skip-no-batching \
        --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_.xml " \
        --file-suffix "_security_no_protection"
fi

if [[ "${security_only}" == "rtps_encrypt" || "${security_only}" == "" ]]; then
    echo -e "[Calling base_script/script.sh] -- RTPS Encrypt"
    "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
         --skip-be --skip-no-batching \
        --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_RTPSEncrypt.xml " \
        --file-suffix "_security_rtps_encrypt"
fi

if [[ "${security_only}" == "rtps_sign_submessage_encrypt_orig_data_encrypt" || "${security_only}" == "" ]]; then
    echo -e "[Calling base_script/script.sh] -- RTPS Sign, Submessage Encrypt with original auth, Data Encrypt"
    "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
         --skip-be --skip-no-batching \
        --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_RTPSSignEncryptSubmessageWithOrigAuthEncryptData.xml " \
        --file-suffix "_security_rtps_sign_submessage_encrypt_orig_data_encrypt"
fi

if [[ "${security_only}" == "rtps_sign_orig_data_encrypt" || "${security_only}" == "" ]]; then
    echo -e "[Calling base_script/script.sh] -- RTPS Sign with Original auth, Data Encrypt"
    "${script_location}/../base_script/script.sh" $input_params --transport UDPv4 \
         --skip-be --skip-no-batching \
        --extra-arguments "-secureGovernanceFile $PATH_TO_GOVERNANCE_FILES_FOLDER/signed_PerftestGovernance_RTPSSignWithOrigAuthEncryptData.xml " \
        --file-suffix "_security_rtps_sign_orig_data_encrypt"
    sleep 5;
fi

Security Profiles

To test different levels of security, we have selected a well-known set of configurations. These configurations have been defined in the Governance files used by RTI Perftest. With these configurations, we have tested the minimum latency and maximum throughput achievable in different scenarios. The scenarios are described below.

The profiles we have used are the following:

• Not using security libraries

In this scenario, RTI Security Plugins is not being used, therefore the performance is the same as what the Core Libraries provide in Unkeyed, UDPv4 10Gbps Network, C++98.

• No protection

In this scenario, Security Plugins are enabled but no protection is provided at any level. This, as well as the previous scenario, is used as a way to calibrate the impact of using Security Plugins even when no security measures are applied.

The governance profile used in this scenario is the following:

<dds>
    <domain_access_rules>
      <domain_rule>
        <domains>
          <id_range>
            <min>0</min>
          </id_range>
        </domains>
        <allow_unauthenticated_participants>TRUE</allow_unauthenticated_participants>
        <enable_join_access_control>FALSE</enable_join_access_control>
        <discovery_protection_kind>NONE</discovery_protection_kind>
        <liveliness_protection_kind>NONE</liveliness_protection_kind>
        <rtps_protection_kind>NONE</rtps_protection_kind>
        <topic_access_rules>
          <topic_rule>
            <topic_expression>*</topic_expression>
            <enable_discovery_protection>FALSE</enable_discovery_protection>
            <enable_read_access_control>FALSE</enable_read_access_control>
            <enable_write_access_control>FALSE</enable_write_access_control>
            <metadata_protection_kind>NONE</metadata_protection_kind>
            <data_protection_kind>NONE</data_protection_kind>
          </topic_rule>
        </topic_access_rules>
      </domain_rule>
    </domain_access_rules>
</dds>

• RTPS ‘Sign’

This scenario sets the rtps_protection_kind to SIGN. This configuration provides protection against outsiders at the lowest cost.

The governance profile used in this scenario is the following:

<dds>
    <domain_access_rules>
      <domain_rule>
        <domains>
          <id_range>
            <min>0</min>
          </id_range>
        </domains>
        <allow_unauthenticated_participants>false</allow_unauthenticated_participants>
        <enable_join_access_control>false</enable_join_access_control>
        <discovery_protection_kind>NONE</discovery_protection_kind>
        <liveliness_protection_kind>NONE</liveliness_protection_kind>
        <rtps_protection_kind>SIGN</rtps_protection_kind>
        <topic_access_rules>
          <topic_rule>
            <topic_expression>*</topic_expression>
            <enable_discovery_protection>false</enable_discovery_protection>
            <enable_liveliness_protection>false</enable_liveliness_protection>
            <enable_read_access_control>false</enable_read_access_control>
            <enable_write_access_control>false</enable_write_access_control>
            <metadata_protection_kind>NONE</metadata_protection_kind>
            <data_protection_kind>NONE</data_protection_kind>
          </topic_rule>
        </topic_access_rules>
      </domain_rule>
    </domain_access_rules>
</dds>

• RTPS ‘Encrypt’

This scenario sets the rtps_protection_kind to ENCRYPT. This configuration is similar to the protection TLS provides.

The governance profile used in this scenario is the following:

<dds>
    <domain_access_rules>
      <domain_rule>
        <domains>
          <id_range>
            <min>0</min>
          </id_range>
        </domains>
        <allow_unauthenticated_participants>false</allow_unauthenticated_participants>
        <enable_join_access_control>false</enable_join_access_control>
        <discovery_protection_kind>NONE</discovery_protection_kind>
        <liveliness_protection_kind>NONE</liveliness_protection_kind>
        <rtps_protection_kind>ENCRYPT</rtps_protection_kind>
        <topic_access_rules>
          <topic_rule>
            <topic_expression>*</topic_expression>
            <enable_discovery_protection>false</enable_discovery_protection>
            <enable_liveliness_protection>false</enable_liveliness_protection>
            <enable_read_access_control>false</enable_read_access_control>
            <enable_write_access_control>false</enable_write_access_control>
            <metadata_protection_kind>NONE</metadata_protection_kind>
            <data_protection_kind>NONE</data_protection_kind>
          </topic_rule>
        </topic_access_rules>
      </domain_rule>
    </domain_access_rules>
</dds>

• RTPS ‘Sign with Original Authentication’ and Data ‘Encrypt’

This scenario sets the rtps_protection_kind to SIGN_WITH_ORIGIN_AUTHENTICATION. It also sets the data_protection_kind to ENCRYPT. This configuration is the common choice for intra-domain protection and confidentiality.

The governance profile used in this scenario is the following:

<dds>
    <domain_access_rules>
      <domain_rule>
        <domains>
          <id_range>
            <min>0</min>
          </id_range>
        </domains>
        <allow_unauthenticated_participants>false</allow_unauthenticated_participants>
        <enable_join_access_control>false</enable_join_access_control>
        <discovery_protection_kind>NONE</discovery_protection_kind>
        <liveliness_protection_kind>NONE</liveliness_protection_kind>
        <rtps_protection_kind>SIGN_WITH_ORIGIN_AUTHENTICATION</rtps_protection_kind>
        <topic_access_rules>
          <topic_rule>
            <topic_expression>*</topic_expression>
            <enable_discovery_protection>false</enable_discovery_protection>
            <enable_liveliness_protection>false</enable_liveliness_protection>
            <enable_read_access_control>false</enable_read_access_control>
            <enable_write_access_control>false</enable_write_access_control>
            <metadata_protection_kind>NONE</metadata_protection_kind>
            <data_protection_kind>ENCRYPT</data_protection_kind>
          </topic_rule>
        </topic_access_rules>
      </domain_rule>
    </domain_access_rules>
</dds>

• RTPS ‘Sign,’ Submessage ‘Encrypt with Original Authentication,’ and Data ‘Encrypt’

This scenario sets the rtps_protection_kind to SIGN. It also sets the data_protection_kind to ENCRYPT and the metadata_protection_kind to ENCRYPT_WITH_ORIGIN_AUTHENTICATION. This configuration offers the most robust protection.

The governance profile used in this scenario is the following:

<?xml version="1.0" encoding="UTF-8"?>

<dds>
    <domain_access_rules>
      <domain_rule>
        <domains>
          <id_range>
            <min>0</min>
          </id_range>
        </domains>
        <allow_unauthenticated_participants>false</allow_unauthenticated_participants>
        <enable_join_access_control>false</enable_join_access_control>
        <discovery_protection_kind>NONE</discovery_protection_kind>
        <liveliness_protection_kind>NONE</liveliness_protection_kind>
        <rtps_protection_kind>SIGN</rtps_protection_kind>
        <topic_access_rules>
          <topic_rule>
            <topic_expression>*</topic_expression>
            <enable_discovery_protection>false</enable_discovery_protection>
            <enable_liveliness_protection>false</enable_liveliness_protection>
            <enable_read_access_control>false</enable_read_access_control>
            <enable_write_access_control>false</enable_write_access_control>
            <metadata_protection_kind>ENCRYPT_WITH_ORIGINAL_AUTHENTICATION</metadata_protection_kind>
            <data_protection_kind>ENCRYPT</data_protection_kind>
          </topic_rule>
        </topic_access_rules>
      </domain_rule>
    </domain_access_rules>
</dds>

• RTPS ‘Sign,’ Submessage ‘Encrypt’

This scenario sets the rtps_protection_kind to SIGN. It also sets the metadata_protection_kind to ENCRYPT. This configuration allows user data confidentiality (with insiders protection) while keeping Wireshark capabilities.

The governance profile used in this scenario is the following:

<?xml version="1.0" encoding="UTF-8"?>

<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="dds_security_governance.xsd">

    <domain_access_rules>
      <domain_rule>
        <domains>
          <id_range>
            <min>0</min>
          </id_range>
        </domains>
        <allow_unauthenticated_participants>false</allow_unauthenticated_participants>
        <enable_join_access_control>false</enable_join_access_control>
        <discovery_protection_kind>NONE</discovery_protection_kind>
        <liveliness_protection_kind>NONE</liveliness_protection_kind>
        <rtps_protection_kind>SIGN</rtps_protection_kind>
        <topic_access_rules>
          <topic_rule>
            <topic_expression>*</topic_expression>
            <enable_discovery_protection>false</enable_discovery_protection>
            <enable_liveliness_protection>false</enable_liveliness_protection>
            <enable_read_access_control>false</enable_read_access_control>
            <enable_write_access_control>false</enable_write_access_control>
            <metadata_protection_kind>ENCRYPT_WITH_ORIGIN_AUTHENTICATION</metadata_protection_kind>
            <data_protection_kind>ENCRYPT</data_protection_kind>
          </topic_rule>
        </topic_access_rules>
      </domain_rule>
    </domain_access_rules>
</dds>

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Dell R340 Servers (13 Units)
Processor: Intel Xeon E-2278G (3.4-5GHz, 8c/16t, 16MB cache, 2 memory channels @2666MHz)
RAM: 4x 16GB 2666MHz DIMM (64GB RAM)
HD: 480GB SATA SSD
NIC 1: Intel 710 dual port 10Gbps SFP
OS: Ubuntu 20.04 -- gcc 9.3.0

Switch

Dell 2048 -- 10Gbps switch (10Gbps and 1Gbps interfaces)