I am involved in development a data recorder for a large DDS-enabled system. The design we have settled on depends largely on the behavior of an undocumented interface in the RTI DDS software. That interface is:
DDS_DynamicData::loan_serialized_bufferI( const DDS_Octet *, DDS_UnsignedLong )
Currently this interface does what we need in that it acquires the address and size of recieved serialized data. Our concern is whether it is undocumented for a reason - that being it possibly may change or be removed in some future release of RTI software. Also, the underlying behavior of this interface is in question in terms of the stablility of the data being accessed and what other under-the-covers mysteries may be occurring that would invalidate its use. For instance: Can another thread grab this buffer and use it before we get to it?
The only reference to this interface we can find is buried in the #include files.
Thanks in advance for any help you can provide.
B. Paul Bockstege
Hello Paul,
Aplologies for the delayed reply. The function is undocumented (and has the trailing I) because it is considered an internal function. After your email we are discussing internally whether there is some other "public" way to accomplish what you are trying to do and if not whether we should expose this function as part if the public interface with the corresponding pubic documentation...
However the DynamicData API is evolving a bit as a result of the recenctly-adopted DDS Extensible Types specification.
So I think the best way forward would be if you could explain us a bit your use case for needing this function and we take a look if there is a good way to enable that capability as part of the new API going forward. Does this sound reasonable?
Regards,
Gerardo
Gerardo:
Thank you for your response, I also apologize for this delayed reply. (The unfavorable rump-to-reptile ratio is such that the main swamp draining mission is often pre-empted.)
Yes, your plan fo rthe 'best way forward' does sound reasonable. I am a bit leery of using undocumented (or 'internal') functions for the reasons I stated previously but mainly because these can be changed and thus invalidate any user software based on them. So, why do I/we need the serialized data buffer and size as provided by this function?
Our task is to record DDS data. One problem is that the data types (or rather the IDL from which the data types are generated) are changing and are likely to change in the future. Hence we use the dynamic data interfaces. The other problem is that we are faced with high data rates - or at least a rate of 125000 samples/second (of at least 80 bytes/sample) seems high to us. To get around the data rate issue as well as the different data types our design uses an interim file format that includes the type_code information from the discovery process in the file header and writing the serialized data as recieved using the loan_serialized_bufferI() function to access that data. This interim file is post-processed in a non-realtime enviroment to produce access optimized data files.
Results so far have been positive but the original question remains - can we depend on loan_seriallized_bufferI()?
Waiting to hear from you again:
B. Paul Bockstege
Hello Paul,
I understand. We had that exact internal use-case in our RTI Recording and Replay Service. What we ended up doing was create a custom TypeSupport called "SerializedDataTypeSupport" whose data representation contained a char array and implemented the 'deserialize' function as a copy of the data from the Serialized buffer into the char array. While this is something that can be done by anyone it is not trivial as it does require some understanding of the TypePlugin functions and moreover the TypePlugin API can also change between RTI releases...
I think the best way forward would be for us (RTI) to add this 'SerializedData' as a builtin type (just like String, KeyedString, Octets, and KeyedOctets are today) so that is documented and supported. I already entered this into our feature request database.
In the meantime I think that using the API you are using in DynamicData is the best solution you have available...
Gerardo
Gerado:
Thank you for the information. It is good to know that we are not alone in fighting this battle. Do you have an estimate as to when this "SerializedData" builtin type would be available?
Also, is it possible to acquire any usage information on loan_serialized_bufferI()? The main concerns I have are threading issues and buffer life-time. Is it thread safe? How long do I have before the system takes the buffer back? And what exactly does "loan" mean in this context?
Thanks again for your response.
B. Paul Bockstege
Hello Paul,
Unfortunately I cannot estimate at this point in time when the "SerializedData" builtin type would be available. While I can guess at the effort required the scheduling of this feature into a particular release of RTI DDS depends on several other factors and committments.
As to the usage information:
This function should really be called "get_underlying_serialized_data_pointer()" all it does is get the pointer to the buffer where the serialized data is stored. It does not do any "loaning" reference counting or multi-thread protection. There is no corresponding "return_loan" operation.
As a consequence life-cycle of the buffer is tied to the lifecycle of the DynamicData object itself. The contract for the DynamicData (or any other data sample you get from the DataReader) is the same:
Either way the management is deterministic and the application can control it.
Let me know if this is enough information or you need additional details.
Gerardo
Hello Gerardo
Its been a while but I'm back (but not as bpb3).
We have been quite successful with our DDS recording system using the loan_serialized_bufferI() method discussed in the previous messages.
However...the "bean counters" around here have determined that we did not perform sufficient unit-tests on our code. So now I am trying to come up with a way to inject data into a DDS_DynamicData type, stick that data in a DDS_DynamicDataSeq type, and then pass that off to the method that uses the loan_serialized_bufferI(). All this without a publisher please.
A mock DDS_DynamicData type would be useful. We have created mocks of other DDS types and this is possible due to the pure virtual nature of the base types. We have not been successful doing so with DDS_DynamicData. Also, getting the 'take' method to work on a mocked DDSDataReader does nothing but core dump, somewhere in the 'typeinfo name' world according to gdb.
Any suggestions?
Thanks in advance for any help you can give us.
B. Paul Bockstege