Transport file-size message

7 posts / 0 new
Last post
Offline
Last seen: 12 years 3 months ago
Joined: 12/14/2011
Posts: 1
Transport file-size message

 

What would be the best way to send a large file (>5M) over DDS?  I am aware there are many other alternatives, e.g. FTP, HTTP, etc.  but I like to consider it from the DDS perspective.  Thanks in advance.

 

- P

Gerardo Pardo's picture
Offline
Last seen: 2 months 1 day ago
Joined: 06/02/2010
Posts: 601
RTI DDS will automatically fragment and re-assemble data that exceeds the maximum size that can be accommodated by the underlying transport, 
however the QoS configuration/policies can definitely have a big impact...
I am assuming you are using the default UDP transport. If so, these are the QoS policies I would recommend you  or are needed to send large data:
1) Configure your UDP transport to allow messages up to 64KB (this is the maximum UDP allows). This will allow DDS to break the data into larger fragments which will reduce the number of datagrams on the wire, ACKS, Heartbeats, etc. Therefore increasing efficiency.
2) Configure the UDP transport to use large socket buffers (not strictly necessary but it helps)
3) Configure your data-writer to be reliable. That way the loss of a single data fragment will not cause the whole data sample to be discarded.
4) Define a Flow Controller to be used by your DataWriter to control the shape of the data it puts on the network (maximum burst, max bytes per second, etc.). If this is not done the fragments will be written as fast a the writer can which might overload the network or the readers. Details on FlowControllers can be found in section 6.6 of the RTI DDS 45e User's Manual.

 

5) Configure the DDS DataWriter to be asynchronous. This will allow the infrastructure to use a separate thread to send the fragments relieving your application thread from doing the fragmentation and UDP sending work and then associate the DataWriter with the FlowController defined in step (4).

 

The easiest way to do these steps is to use the XML QoS profiles as shown below. Using XML to configure QoS is described in chapter 15 of the RTI DDS 45e User's Manual.

Note that I did not test this configuration so there might be some errors or missing items...

<qos_profile  name="MyProfile"> 
   <participant_qos>
        <property>
            <value>
                <!--UDP/IP Transport configuration -->
                <element>
                    <name>dds.transport.UDPv4.builtin.parent.message_size_max</name>
                    <value>65536</value>
                </element>
                <element>
                    <name>dds.transport.UDPv4.builtin.send_socket_buffer_size</name>
                    <value>1000000</value>
                </element>
                <element>
                    <name>dds.transport.UDPv4.builtin.recv_socket_buffer_size</name>
                    <value>2000000</value>
                </element>
                <!-- Definition of the flow controller See users' manual section 
                    6.6 FlowControllers -->
                <element>
                    <name>
                        dds.flow_controller.token_bucket.MyFlowController.scheduling_policy</name>
                    <value>DDS_RR_FLOW_CONTROLLER_SCHED_POLICY</value>
                </element>
                <element>
                    <name>dds.flow_controller.token_bucket.MyFlowController.token_bucket.period.sec</name>
                    <value>0</value>
                </element>
                <element>
                    <name>dds.flow_controller.token_bucket.MyFlowController.token_bucket.period.nanosec</name>
                    <value>10000000</value>
                </element>
                <element>
                    <name>dds.flow_controller.token_bucket.MyFlowController.token_bucket.max_tokens</name>
                    <value>100</value>
                </element>
                <element>
                    <name>dds.flow_controller.token_bucket.MyFlowController.token_bucket.tokens_added_per_period</name>
                    <value>40</value>
                </element>
                <element>
                    <name>dds.flow_controller.token_bucket.MyFlowController.token_bucket.tokens_leaked_per_period</name>
                    <value>0</value>
                </element>
                <element>
                    <name>dds.flow_controller.token_bucket.MyFlowController.token_bucket.bytes_per_token</name>
                    <value>66000</value>
                </element>
            </value>
        </property>
    </participant_qos>
    <datawriter_qos>
        <publish_mode>
            <kind>ASYNCHRONOUS_PUBLISH_MODE_QOS</kind>
            <flow_controller_name>DEFAULT_FLOW_CONTROLLER_NAME</flow_controller_name>
        </publish_mode>
        <reliability>
            <kind>RELIABLE_RELIABILITY_QOS</kind>
        </reliability>
    </datawriter_qos>  
</qos_profile>  

 

ken
ken's picture
Offline
Last seen: 3 months 6 days ago
Joined: 04/13/2011
Posts: 64

Hi,

Another important consideration when looking at this functionality is where the fragmentation occurs. In the current approach, a type is defined in IDL which can accommodate the largest file that is desired to be transferred (~5MB) and DDS will break this data into fragments for transfer over the transport(s). This does have ramifications at the reader/writer queue level however. By default, RTI will allocate 32 samples for the reader/writer queues. These are allocated at their maximal size leading to ~160MB of memory consumed on each the writing and reading applications.

An alternative approach would be to define a smaller IDL data type on the order of, say, 63KB and to do the fragmentation at the application layer. The 63KB was chosen to allow UDP transfers to occur without fragmentation (provided that the transport was tuned as above to allow 64KB max messages through). In this scenario, the IDL type would also contain extra meta data to account for the fragmentation. Perhaps a segment number and total segment count in addition to the filename and sequence of bytes payload. The advantage with this approach is that the queues are smaller and the DDS configuration is simplified as there are no flow controllers to configure. Of course, the application layer is more complex in that it has to perform the fragmentation and reconstruction of the files.

Thanks,
Ken

Offline
Last seen: 10 years 7 months ago
Joined: 08/05/2013
Posts: 1

I am trying to write an example to send large data>64kB using UDP. So I took one of the available examples in the CPP/Hello_idl and modified the QoS to include the ASYNCHRONOUS_PUBLISH_MODE and increased buffer sizes for send and receive sockets. I did omit defining Flow control since I am looking for a maximum burst.

Is it necessary to define shared memory too for sending large data > 64KB?

When I am trying to run publisher I am getting an error as below:

Sending data...
COMMENDSrWriterService_write:!fragment data. Not supported by this writer.
PRESPsWriter_flushBatchWithCursor:!srw->write
PRESPsWriter_writeBatchInternal:!error flushing batch
! Write error 1
COMMENDSrWriterService_write:!fragment data. Not supported by this writer.
PRESPsWriter_flushBatchWithCursor:!srw->write
PRESPsWriter_writeBatchInternal:!error flushing batch
! Write error 1

Any idea on what would be the root cause for this?

Any suggestions towards this would be helpful and really appreciated.

Thanks,

Ben

Offline
Last seen: 10 years 7 months ago
Joined: 07/12/2013
Posts: 5

Hi Ben,

Data fragmentation is not supported when batching is enabled. From the error message it looks like you have enabled batching for the DataWriter. 

Can you try disabling batching? Batching can be disabled by setting  DDS_DataWriterQos::batch::enable to DDS_BOOLEAN_FALSE. Do note that by defualt this value is set to DDS_BOOLEAN_FALSE.

Regards,

Roshan

Offline
Last seen: 6 years 3 weeks ago
Joined: 07/12/2012
Posts: 51

Does the flow controller settings in this example also apply  to other transports, specifically TCP ?

Thank you

Nico

Offline
Last seen: 8 years 9 months ago
Joined: 05/28/2015
Posts: 1

I am using the above qos to transmit large data, but I am getting an error "COMMENDSrWriterService_sendSyncRepairData:!write resend. Reliable large data requires asynchronous writer."

Any idea what this error means as my writer is "ASYNCHRONOUS_PUBLISH_MODE_QOS", and what may be the possible fix for this issue.

Thanks,

Kajal