I have written code to traverse dynamic data to collect data (read only) of samples. The code yields the correct results, however, performance can be very slow. In particular, traversing samples with arrays of structures. Running valgrind, 70% of call time is dedicated to get_complex_member(). I have also tried bind_complex_member() which is 40% slower, and is counter to what I expected (eliminating the overhead of copying should be faster.)
Should bind_ be faster than get_complex_member()? Is there a best practices/performance tips with respect to dynamic data and arrays of structures?
We are running version 5.0.0 on linux/fedora.
Intuitively, bind_complex_member should be faster than get_complex_member because the later makes a copy. It is conceivable that if get_complex_member is called on large arrays of structures, a large percentage of execution time is spent in just copying data. Perhaps, that's what you observe in valgrind.
Do you mean with bind_complex_member the performance degrades by additional 40%? That's not expected, AFAIK.
Without the knowledge of your types and sequence lengths, what I could do is run a test I've that compares performance of dynamic_data with generated code for a relatively complex type. I should be able to run test with both bind- and get_complex_member. It would be a variant of the perf-test here. Note that the test relies on a library called RefleX that simplies handling dynamic data in C++.
The most important "best practice" while using dynamic data in performance sensitive apps is to ensure that members of dynamic data are populated in the order of increasing member-id and avoid modifications to variable-sized members. Please see the "Performance" section here.
What language are you using?
-Sumant
Sorry for the delay in posts. We are using the C++ API (version 5.0.0), and the bind_complex_member does degrade performance (total time) by 40%. We have two messages in particular that are very slow. One having an array of 160 structures, and the other an array of 320 structures. The structures contain a mix of primitives and simple nested structures themselves, approximately 35 attributes total.
What we have implemented is a query function that traverses the sample to extract a single field, so it is truly random. Also, each query is independent, starting at the root of the sample traversing down the tree according to the field specified (we use a dot notation: "structA.structB.field1".) Since each query is independent, I knew we would be taking a hit for re-traversing the structures for adjacent fields, a price to pay for random access. I don't have a handle on how much of a penalty we are taking for not accessing the fields in order.
I am still a bit baffled by the bind* performance. Looking at the Valgrind results, it appears there is a lot of overhead with the complex_member functions. Both the get_ and bind_ calls had 8-10 internal calls for each complex member traversed. Is there any simpler way to access the data for read only operations?