How to change type_code_max_serialized_length QoS parameter

18 posts / 0 new
Last post
Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24
How to change type_code_max_serialized_length QoS parameter

I'm having difficulty changing the type_code_max_serialized_length QoS parameter for sending a large TypeCode.  I belive the format for this parameter is:

<participant_qos>
    <resource_limits>
        <type_code_max_serialized_length>4096</type_code_max_serialized_length>
        <type_object_max_serialized_length>4096</type_object_max_serialized_length
    </resource_limits>
</participant_qos>\
 

I tried working with a dds::core::policy::ResourceLimits object but can't find type_code_max_serialized_lengthor type_object_max_serialized_lengh properties.  The only configurable properties I see are max_instances, max_samples, and max_samples_per_instance.

How would I update my domain participant QoS for type_code_max_serialized_length?  Quick C++ snippet would be helpful.

Offline
Last seen: 2 months 3 weeks ago
Joined: 10/22/2018
Posts: 70

I am able to modify that QoS with the following code:
dds::domain::qos::DomainParticipantQos qos;
qos.policy<rti::core::policy::DomainParticipantResourceLimits>().type_code_max_serialized_length(foo);

Can you give that a try?

 

Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24

Thanks!  This works, I must have been configuring the wrong policy type before.

Is there a way to figure out the proper length to allocate for my TypeCode?  i.e. how to find the max serialized length of a custom type I generated.

Offline
Last seen: 2 months 3 weeks ago
Joined: 10/22/2018
Posts: 70

I believe you need the function cdr_serialized_sample_max_size that returns the maximum serialized size in bytes for the data of a type.

r
Offline
Last seen: 19 hours 38 min ago
Joined: 06/17/2019
Posts: 27

samr showed you how to determine the length of individual samples. If you're trying to find the appropriate value for type_code_max_serialized_length and type_object_max_serialized_length you can use admin console to see what the current length of your types are.

Here's an example using shapesdemo to analyse the ShapeTypeExtended datatype

ac

Typecode is the older way to share your type information. If you do not need to support legacy versions of connext, you can disable your type_code_max_serialized_length (by setting it equal to 0) and only use typeobject. 

You can even disable typeobject from being shared during discovery if you want. However your readers will need to understand the datatype they're subscribing to, and they won't be able to use dynamic data if you disable sharing of all type information. 

If you disable all type information from being shared, readers and writers will only be able to verify and match using the datatype name, instead of being able to verify and match on the actual data type that is shared. 

Some additional details: https://community.rti.com/kb/when-do-i-need-send-typecode-or-typeobject

Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24

Some of my topics show a TypeCode/TypeObject Size like Ross's screenshot above.  However I can't see any info for my large topic causing issues:

When I try to subscribe in the Admin Console the "Create Subscription" popup shows an error "Unable to subscribe to the topic, because data type of the topic is unknown".  Nothing is selectable within the Data Type dropdown.

I was thinking maybe I could import my topic from XML into the Admin Console.  I haven't been able to yet because rtiddsgen fails to export the topic to an XML.  I think because I have several #include dependency IDL's?  rtiddsgen with the -convertToXml flag throws "Error executing cpp preprocessor".

Also started looking into cdr_serialized_sample_max_size but not sure how to access that property.

r
Offline
Last seen: 19 hours 38 min ago
Joined: 06/17/2019
Posts: 27

What is likely happening is your actual typecode/typeobject sizes for that larger datatype is more than your current settings for typecode/object...max_serialized_length

For example if your actual typeobject size is 16kb, but your type_object_max_serialized_length is set to 4k, only the datatypes that are 4k and under will be shared on the wire to admin console. This is why admin console is not able to dynamically subscribe to your larger datatype (the datatype information is not shared with AC because it is over the max_serialized_length setting)

One easy fix would be to set your typecode/object...max_serialized_length settings very large, which will allow them to be shared to admin console. Then view them in admin console to set a more sane maximum.

The max for type_code_max_serialized_length is 65535 (it defaults to 0/off since it often isn't needed unless you have legacy interop concerns) and the max for type_object_max_serialized_length is 2147483647

https://community.rti.com/static/documentation/connext-dds/6.0.1/doc/api/connext_dds/api_cpp/structDDS__DomainParticipantResourceLimitsQosPolicy.html#a55d8561093c4ab21484613a4b8de2999

Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24

I went ahead and updated the max_serialized_length's for typecode/object as discussed above.  I'm getting this dds::core::Error about a "wire_protocol".  Is that a QoS I configure separate from my application (i.e. via XML)?

[CREATE Participant] DDS_WireProtocolQosPolicy_is_consistentI:inconsistent QoS policy: port_base > 0
[CREATE Participant] DDS_DomainParticipantQos_is_consistentI:inconsistent QoS policy: wire_protocol
[CREATE Participant] DDS_DomainParticipantFactory_create_participant_disabledI:ERROR: Inconsistent QoS
terminate called after throwing an instance of 'dds::core::Error'
  what():  Failed to create DomainParticipant
Aborted (core dumped)

My code looks like this:

DDSMessageHandler::DDSMessageHandler(int domain_id) : participant(domain_id, qos), ...<other initializations>
{
   qos.policy<rti::core::policy::DomainParticipantResourceLimits>().type_code_max_serialized_length(65535);
    qos.policy<rti::core::policy::DomainParticipantResourceLimits>().type_object_max_serialized_length(2147483647);

}

 

r
Offline
Last seen: 19 hours 38 min ago
Joined: 06/17/2019
Posts: 27

When I try to max out the type_object_max_serialized_length I get some memory allocation errors. When I lower it to a more sane 10000000 I don't have problems. Can you try that?

I left type_code_max_serialized_length = 65535

But your error is different, and seems to have to do with port usage. Did this error just start occuring after setting typecode/object max serialzied lengths?
What domain ID and participant ID are you using? (these affect the port selected)

Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24

Tried lowering type_object_max_serialized_length to 10000000, same type_code_max_serialized_length 65535.  Same failure to create domain participant.

I'm specifying 0 for domain id.  Not specifying a participant id (so I think DDS automatically picks -1 for me per this page?)

Offline
Last seen: 2 months 3 weeks ago
Joined: 10/22/2018
Posts: 70

The C API DDS_TypeCode_get_type_object_serialized_size (https://community.rti.com/static/documentation/connext-dds/6.0.1/doc/api/connext_dds/api_c/group__DDSTypeCodeModule.html#ga80f8a71344cfff5ae947cc6e2df37c74) should provide you with the information you need to set that value correctly. Unfortunately it is missing from the C++ API, I have raised an bug internally to fix this, but in the meantime you can just call the C function from C++.

r
Offline
Last seen: 19 hours 38 min ago
Joined: 06/17/2019
Posts: 27

If you're still receiving the QoS port_base errors with and without the typecode/object_max_serialized_length configuration than you're probably hitting two different issues.

For the port_base issue, are you setting the WIRE_PROTOCOL QoS policy configuration to anything non-default?

 

Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24

In my app haven't changed any QoS policies besides the type_code/object_max_serialized_length.

Also haven't edited any QoS XML's (how RTI tools get configured on startup I think?).

r
Offline
Last seen: 19 hours 38 min ago
Joined: 06/17/2019
Posts: 27

I think you may not be correctly initializing your QoS before the domain participant is created. 

You have a couple of options, you could use the default QoS values and use the 'PARTICIPANT_QOS_DEFAULT' arg. Here is what it looks like using the default rtiddsgen code (underlined)

DomainParticipant *participant = NULL;
Publisher *publisher = NULL;
Topic *topic = NULL;
DataWriter *writer = NULL;
ShapeTypeExtendedDataWriter * ShapeTypeExtended_writer = NULL;
ShapeTypeExtended *instance = NULL;
ReturnCode_t retcode;
InstanceHandle_t instance_handle = HANDLE_NIL;
const char *type_name = NULL;
int count = 0;
Duration_t send_period = {4,0};
/* To customize participant QoS, use the configuration file USER_QOS_PROFILES.xml */
participant = TheParticipantFactory->create_participant(
  domainId, PARTICIPANT_QOS_DEFAULT,
  NULL /* listener */, STATUS_MASK_NONE);
if (participant == NULL) {
  printf("create_participant error\n");
  publisher_shutdown(participant);
  return -1;
}

Alternatively if you want to set your own QoS you should first 'get_default_participant_qos', make any changes, then create the participant with the changes you wanted. Here's an example from the manual:

DDS_DomainId_t domain_id = 10;
DDS_DomainParticipantQos participant_qos;1
// initialize participant_qos with default values
factory->get_default_participant_qos(participant_qos);
// make QoS changes here
participant_qos.wire_protocol.participant_id = 2;
// Create the participant with modified qos
DDSDomainParticipant* participant = factory->create_participant(
  domain_id, participant_qos, NULL, DDS_STATUS_MASK_NONE);
if (participant == NULL) {
  // ... error
}

 

Here is the relevant section of the manual: https://community.rti.com/static/documentation/connext-dds/6.0.1/doc/manuals/connext_dds/html_files/RTI_ConnextDDS_CoreLibraries_UsersManual/index.htm#UsersManual/Setting_DomainParticipant_QosPolicies.htm#9.3.7.1_Configuring_QoS_Settings_when_DomainParticipant_is_Created%3FTocPath%3DPart%25202%253A%2520Core%2520Concepts%7C9.%2520Working%2520with%2520DDS%2520Domains%7C9.3%2520DomainParticipants%7C9.3.7%2520Setting%2520DomainParticipant%2520QosPolicies%7C_____1

If that doesn't help can you provide more of your code on how you are configuring what you pass into the create_participant() call. 

 

Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24

Thanks for the examples.  Trying to add the new QoS changes but getting hung up on the DomainParticipantFactory.  Haven't used one as before I generally just create DomainParticipant's.  Code below doesn't work, maybe you point me to where I'm headed off track.  Don't think I'm instantiating the factory correctly.  I left out that wire_protocol.participant_id = 2 but guessing I'll need that to fix the wire_protocol error.  Would this also be the right place to configure type_code/object_max_serialized_length (as done below)?

   DDS_DomainParticipantQos participant_qos;
    // initialize participant_qos with default values
    DDS_DomainParticipantFactory* factory = DDS_DomainParticipantFactory::get_instance();

    participant = DDS_DomainParticipantFactory_create_participant(factory,
                                                                  domain_id,
                                                                  &participant_qos,
                                                                  NULL,
                                                                  DDS_STATUS_MASK_NONE);

    participant_qos.resource_limits.type_code_max_serialized_length(65535);
    participant_qos.resource_limits.type_object_max_serialized_length(10000);

 

r
Offline
Last seen: 19 hours 38 min ago
Joined: 06/17/2019
Posts: 27

I don't think you need to reference participant_qos (with the "&") in the argument to create_participant(). I also don't usually get an instance of the participant factory since there is a predefined macro "TheParticipantFactory" that you can use directly. 

Here's my entire process:

  • I used the ShapeType.idl to generate code using rtiddsgen. (any basic IDL will work for this purpose)
    C:\Program Files\rti_connext_dds-6.0.1\bin\rtiddsgen.bat ShapeType.idl -ppDisable -language C++ -namespace -create typefiles -create examplefiles -create makefiles -platform x64Win64VS2017
  • Open up the .sln in visual studio, and open ShapeType_publisher.cxx
  • Around line 100 (in publisher_main() just after "Duration_t send_period = {4,0};") I added:

DDS_DomainParticipantQos participant_qos;
TheParticipantFactory->get_default_participant_qos(participant_qos);
participant_qos.participant_name.name = "test-name";
participant_qos.participant_name.role_name = "test-role";

 

  • Then I modify the create_participant() call on the next line to swap out PARTICIPANT_QOS_DEFAULT with participant_qos

participant = TheParticipantFactory->create_participant(domainId, participant_qos, NULL /* listener */, STATUS_MASK_NONE);

 

Then I can take a look in admin console and easily see this QoS was applied correctly because the participant name and role names changed

If you are still having trouble can you try and replicate my steps?

edit: and you'd want to set the typecode/object size prior to creating the participant. The basic flow is get the qos from the factory, make changes, then create participants. 

 

Offline
Last seen: 2 months 3 weeks ago
Joined: 09/08/2020
Posts: 24

Made this change, now compiler is throwing errors on the topic definitions.  Several like:

In file included from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/domain/detail/DomainParticipant.hpp:26:0,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/domain/DomainParticipant.hpp:26,
                 from ../../missiontype.hpp:24,
                 from ../ControllerCentral/DDS/DDSMessageHandler.h:4,
                 from ../ControllerCentral/DDS/DDSMessageHandler.cpp:1:
../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/domain/TDomainParticipant.hpp: In constructor 'DDSMessageHandler::DDSMessageHandler(int)':
../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/domain/TDomainParticipant.hpp:127:5: error: 'dds::domain::TDomainParticipant<DELEGATE>::TDomainParticipant() [with DELEGATE = rti::domain::DomainParticipantImpl]' is private
../ControllerCentral/DDS/DDSMessageHandler.cpp:5:51: error: within this context
In file included from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/topic/Topic.hpp:26:0,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/pub/detail/DataWriter.hpp:26,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/pub/DataWriter.hpp:26,
                 from ../ControllerCentral/DDS/DDSMessageHandler.h:7,
                 from ../ControllerCentral/DDS/DDSMessageHandler.cpp:1:
../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/topic/TTopic.hpp:64:5: error: 'dds::topic::Topic<T, DELEGATE>::Topic() [with T = missiontype; DELEGATE = rti::topic::TopicImpl]' is private
../ControllerCentral/DDS/DDSMessageHandler.cpp:5:51: error: within this context

Similar errors for all the topic Writer, Reader, Publisher, Subscriber creations.

Also:

In file included from ../../../../rti_connext_dds-6.0.1/include/ndds/ndds_c.h:96:0,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/rti/core/detail/NativeSequence.hpp:27,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/rti/core/InstanceHandle.hpp:22,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/core/detail/InstanceHandle.hpp:27,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/core/InstanceHandle.hpp:23,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/core/status/TStatus.hpp:27,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/core/status/detail/Status.hpp:26,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/core/status/Status.hpp:26,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/core/TEntity.hpp:29,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/domain/TDomainParticipant.hpp:31,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/domain/detail/DomainParticipant.hpp:26,
                 from ../../../../rti_connext_dds-6.0.1/include/ndds/hpp/dds/domain/DomainParticipant.hpp:26,
                 from ../../missiontype.hpp:24,
                 from ../ControllerCentral/DDS/DDSMessageHandler.h:4,
                 from ../ControllerCentral/DDS/DDSMessageHandler.cpp:1:
../../../../rti_connext_dds-6.0.1/include/ndds/dds_c/dds_c_domain.h:1662:16: error: forward declaration of 'DDS_DomainParticipantFactory {aka struct DDS_DomainParticipantFactoryImpl}'

Am I missing a necessary #include file for Domain Participants?  I was able to compile before when we saw the wire protocol issue.

Code snippets:

DDSMessageHandler.cpp

#include "DDSMessageHandler.h"
using namespace dds::domain::qos;

DDSMessageHandler::DDSMessageHandler(int domain_id)
{
    // create participant_qos with needed QoS values
    DDS_DomainParticipantQos participant_qos;
    DDS_TheParticipantFactory->get_default_participant_qos(participant_qos);
    participant_qos.participant_name.name = "central-dds-controller";
    participant_qos.participant_name.role_name = "central-role";
    //participant_qos.resource_limits.type_code_max_serialized_length(65535);
    //participant_qos.resource_limits.type_object_max_serialized_length(10000);

    participant = DDS_TheParticipantFactory->create_participant(domain_id, participant_qos, NULL /* listener */, STATUS_MASK_NONE);

    missionPlanTopic(participant, "mission topic");
    publisher(participant);
    missionPlanWriter(publisher,missionPlanTopic);
    globalWaypointTopic(participant, "waypoint topic");
    subscriber(participant);
    globalWaypointReader(subscriber,globalWaypointTopic);
}

DDSMessageHandler.h

#include "missiontype.hpp"
#include "waypointtype.hpp"

#include <dds/pub/DataWriter.hpp>
#include <dds/sub/DataReader.hpp>
private:
    dds::domain::DomainParticipant participant;
    dds::domain::qos::DomainParticipantQos qos;

    dds::topic::Topic<missiontype> missionPlanTopic;
    dds::topic::Topic<waypointtype> globalWaypointTopic;

    dds::pub::Publisher publisher;
    dds::pub::DataWriter<missiontype> missionPlanWriter;

    dds::sub::Subscriber subscriber;
    dds::sub::DataReader<waypointtype> globalWaypointReader;

r
Offline
Last seen: 19 hours 38 min ago
Joined: 06/17/2019
Posts: 27

FYI for anyone else running across a similar issue, they changed the way their entities were initialized per: https://community.rti.com/best-practices/modern-c-api-don%E2%80%99t-declare-entities-pointers which fixed their issue