Performance issues when decoding dynamic_data array in Python

7 posts / 0 new
Last post
Offline
Last seen: 1 year 9 months ago
Joined: 12/06/2021
Posts: 17
Performance issues when decoding dynamic_data array in Python

 

Hi everyone, I'm having some performance issues when trying to convert a dynamic_data array values to a list in python (see code below). Is there a better/more efficient way to do it? I'm using this rti python library: https://github.com/rticommunity/connextdds-pyhttps://community.rti.com/static/documentation/connext-dds/6.0.1/api/connext_dds/api_python/rti.html

array_info = []           
if dynamic_data.member_info(i).kind == dds.TypeKind.ARRAY_TYPE:               
  array_data = dynamic_data.get_value(i)               
    for j in range(array_data.member_count):                   
         array_info.append(array_data[j])
 
Thanks in advance.
Marc.Chiesa's picture
Offline
Last seen: 2 years 1 month ago
Joined: 07/24/2017
Posts: 32

Does the following give you any improvement in performance?

if dynamic_data.member_info(i).kind == dds.TypeKind.ARRAY_TYPE:               
  array_info = list(dynamic_data[i])               
 
 
Offline
Last seen: 1 year 9 months ago
Joined: 12/06/2021
Posts: 17

yes, it does help on reducing to 1/3 of the time. Thank you very much. Can this also be used with arrays of enums(dds.TypeKind.ENUMERATION_TYPE)?

Marc.Chiesa's picture
Offline
Last seen: 2 years 1 month ago
Joined: 07/24/2017
Posts: 32

Yes, but there are some performance implications for that case. The returned sequence contains DynamicData EnumMember objects, which allows you to do things like print the string representation of the value. If you just want the ordinals you can use  list(sample.get_uint32_values(i)) which will be faster

Offline
Last seen: 1 year 9 months ago
Joined: 12/06/2021
Posts: 17

Hey Marc, 

Thanks for your previous comment. I'm still not getting the performance I needed. I notice that calling dynamic_data.get_value(i) or array_data.member_info(i).kind take around 20 millisecond each. Is that the performance you normally get?

These two rti_dds calls are taking most of the time from my code. I attached the complete function so you can take a look at it, it is a recursive function. Maybe there is other places also to optimize.

I'm open to any suggestions.

Thanks in advance.

 

File Attachments: 
Marc.Chiesa's picture
Offline
Last seen: 2 years 1 month ago
Joined: 07/24/2017
Posts: 32

Accessing nested DynamicData structures can get expensive without loaning because it will create a copy. The following might yield a signficant increase in performance:

 
def decode_dynamic_data(dynamic_data: dds.DynamicData, decoded_data: list):

    for i in range(dynamic_data.member_count):

        # Decoding nested structs
        member_info = dynamic_data.member_info(i)
        dynamic_data_kind = member_info.kind
        if dynamic_data_kind == dds.TypeKind.STRUCTURE_TYPE:
            with dynamic_data.loan_value(i) as loan:
                decode_dynamic_data(loan.data, decoded_data)

        # Decoding arrays
        elif dynamic_data_kind == dds.TypeKind.ARRAY_TYPE:
            # Decoding simple types (int, float, octet)
            array_data_kind = member_info.element_kind
            if array_data_kind == dds.TypeKind.ENUMERATION_TYPE:
                decoded_data.extend(dynamic_data.get_int32_values(i))

            # Decoding array of structs
            elif array_data_kind == dds.TypeKind.STRUCTURE_TYPE:
                with dynamic_data.loan_value(i) as loaned_array:
                    array_data = loaned_array.data
                    for j in range(len(array_data)):
                        # Decoding nested structs
                        with array_data.loan_value(j) as loaned_elem:
                            decode_dynamic_data(loaned_elem.data, decoded_data)
                        
            else:
                decoded_data.extend(dynamic_data[i])

        # Decoding enums
        elif dynamic_data_kind == dds.TypeKind.ENUMERATION_TYPE:
            decoded_data.append(dynamic_data.get_int32(i))

        # Decoding simple type (int, float, octet)
        else:
            decoded_data.append(dynamic_data[i])
 
Offline
Last seen: 1 year 9 months ago
Joined: 12/06/2021
Posts: 17

thanks Marc, for taking the time to look at the code and modify it. It works pretty nice.