1.1.2.5. Network Capture

The Network Capture feature allows you to capture the traffic that one or more Participants send or receive. Each Participant will generate a pcap-based file containing its traffic that can then be opened with Wireshark. For more information, see Network Capture, in the RTI Connext DDS Core Libraries User’s Manual.

The use of Network Capture has some performance penalties, both in terms of latency and throughput, because it involves the additional effort of processing RTPS frames. This processing effort comes mainly from writing the RTPS frames into an output file according to a format (e.g., pcap).

To prevent a bottleneck while writing to the output file, Connext DDS queues the RTPS frames. Then, a separate thread is responsible for reading from the queue and writing to the output file. So, when Network Capture is running, an extra copy is involved. As a result, the effect of copying the RTPS frames increases with their lengths. It means that larger payloads will affect performance more.

Furthermore, depending on the options provided to Network Capture, there will be an additional performance impact due to parsing the RTPS frames. For example, if we set the parameters so that user data is removed, the capture file will be smaller in size but at a performance cost. The performance cost comes from parsing the RTPS frames looking for user data so that it can be excluded from the capture.

Network Capture can also parse and save the results of decrypting RTPS frames. If parsing the encrypted content is enabled, instead of having one buffer (the encrypted RTPS frame), we may have up to three: the encrypted RTPS frame, its decryption (that may contain encrypted submessages), and the decrypted submessages. Processing, copying, and writing more information to the output capture file affects performance.

When performance requirements are strict, all these aspects should be considered before running Network Capture. In any case, Network Capture should only be used for debugging purposes.

This section includes the performance results of capturing traffic for two DomainParticipants, one publishing data and another subscribing to it. As usual in RTI Perftest, the latency results come from the Publisher and the throughput results results come from the Subscriber.

Network Capture, Unkeyed, Best Effort, UDPv4 10Gbps, C++98

The graph below shows the one-way latency without load between a Publisher and a Subscriber running in two Linux nodes connected locally in a 10Gbps network.

Detailed Statistics

The following table contain the raw numbers presented by RTI Perftest. These numbers are the exact output with no further processing.

Sample Size (Bytes)	Avg (μs)	Std (μs)	Min (μs)	Max (μs)	50% (μs)	90% (μs)	99% (μs)	99.99% (μs)	99.9999% (μs)
32	23	0.7	21	121	23	23	24	33	121
64	23	0.8	21	135	23	23	24	33	135
128	23	0.6	22	121	23	23	24	34	121
256	24	0.8	22	188	23	24	25	34	188
512	24	0.9	23	484	24	25	26	33	484
1024	26	1.3	24	712	26	26	27	36	712
8192	46	3.1	43	737	46	46	47	60	737
63000	113	27.7	107	627	110	111	124	482	627

Perftest Scripts

To produce these tests, we executed RTI Perftest for C++98. The exact commands used can be found here:

Publisher Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export exec_time=30
export nic=172.16.0.1
export pub_string="-pub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -exec $exec_time \
        -noXML\
        -networkCapture \
        -latencyTest"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_be_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_pub_unkeyed_rel_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_pub_keyed_be_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_pub_keyed_rel_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 -datalen $DATALEN $pub_string"
    echo $command
    $command >> $my_file;
    sleep 3;
done

Subscriber Side

sudo /set_lat_mode.sh

echo EXECUTABLE IS $1
export executable=$1

echo OUTPUT PATH IS $2
export output_folder=$2

export nic=172.16.0.2
export sub_string="-sub \
        -transport UDPv4 \
        -nic $nic \
        -noPrint \
        -noOutputHeaders \
        -networkCapture \
        -noXML"

mkdir -p $output_folder

echo ">> UNKEYED BE"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_be_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> UNKEYED REL"
export my_file=$output_folder/lat_udpv4_sub_unkeyed_rel_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000 100000 500000 1048576 1548576 4194304 10485760; do
    export command="taskset -c 0 \
    $executable $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED BE"
export my_file=$output_folder/lat_udpv4_sub_keyed_be_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -best -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done
sleep 5;

echo ">> KEYED REL"
export my_file=$output_folder/lat_udpv4_sub_keyed_rel_networkCap.csv
touch $my_file
for DATALEN in 32 64 128 256 512 1024 8192 63000; do
    export command="taskset -c 0 \
    $executable -keyed -instances 100000 $sub_string -datalen $DATALEN"
    echo $command
    $command >> $my_file;
    sleep 10;
done

Test Hardware

The following hardware was used to perform these tests:

Linux Nodes

Processor: Intel® Xeon® E-2186G 3.8GHz, 12M cache, 6C/12T, turbo (95W)
RAM: 16GB 2666MT/s DDR4 ECC UDIMM
NIC 1: Intel X550 Dual Port 10GbE BASE-T Adapter, PCIe Full Height
NIC 2: Intel Ethernet I350 Dual Port 1GbE BASE-T Adapter, PCIe Low Profile
OS: Ubuntu 18.04 -- gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Switch

Dell Networking S4048T-ON, 48x 10GBASE-T and 6x 40GbE QSFP+ ports, IO to PSU air, 2x AC PSU, OS9

The graph below shows the expected throughput behavior when performing a 1-1 communication between two Linux nodes in a 10Gbps network.

Note

By default, RTI Perftest enables batching when performing a Maximum Throughput test. The batching feature allows sending more than one data sample per RTPS packet, improving network performance for small data sizes. See the RTI Connext DDS Core Libraries User’s Manual for more information on batching.

The batch maximum size is set by RTI Perftest to be 8192 bytes; after 8192 bytes, batching is not enabled.