[SOLVED] how to change max_objects_per_thread in rticonnextdds_connector

5 posts / 0 new
Log in or register to post comments
Last post
Sat, 06/17/2017 - 15:52
#1
jqcheng
Offline
Last seen: 7 years 4 months ago
Joined: 06/17/2017
Posts: 2
[SOLVED] how to change max_objects_per_thread in rticonnextdds_connector

Hi,

 We are experimenting with the rticonnextdds_connector with node.js. Due to some legacy reason, we need more than 10 participants from different domains. However, node.js has a single process. On our 64-bit Ubuntu 14.04, the maximum number of participants was limited to 8.

   In order to increase the number of participants, we would have to change the resource_limits.max_objects_per_thread of DDS_DomainParticipantFactoryQos. The api to access DDSDomainParticipantFactory is not exposed in the connector.

  Is is possible to use c++ addon to access the DDSDomainParticipantFactory and change the resource limits?

  The following was one such atempt:

#include <node.h>
#include <v8.h>
#include <ndds/ndds_cpp.h>

using namespace v8;

Handle<Value> Method(const Arguments& args) {

  // The following is the dds code
  DDS_DomainParticipantFactoryQos factoryQoS;
  DDSTheParticipantFactory->get_qos(factoryQoS);
  /* increase max_objects_per_thread as needed.
   * The default is 1024 (512 for 4.4c and below). */
  factoryQoS.resource_limits.max_objects_per_thread = 4096;
  DDSTheParticipantFactory->set_qos(factoryQoS);


  HandleScope scope;
  return scope.Close(String::New("true"));
}

void Init(Handle<Object> exports) {
  exports->Set(String::NewSymbol("dds_resource_init"),
      FunctionTemplate::New(Method)->GetFunction());
}

NODE_MODULE(dds_resource_init, Init)

   In the javascript:

var dds_init = require('bindings')('dds_resource_init');

console.log('increase dds resource limits');

  was called before the 

var rti = require('rticonnextdds-connector');
 
var connector = new rti.Connector("myqos.xml");

The c++ addon was compiled with:

gyp info using node-gyp@0.10.10
gyp info using node@0.10.25 | linux | x64

  So far, such approch has not worked for us.

  Is this approach fundamentally wrong?

  Any suggestions will be appreciated!

 

 

 

Keywords:
Top
Mon, 06/26/2017 - 12:01
gianpiero
gianpiero's picture
Offline
Last seen: 10 months 2 weeks ago
Joined: 06/02/2010
Posts: 177

Hello Jiqi,

Thanks for using the RTI Connector! My first answer was going to be to set that property in XML:
 
<participant_factory_qos>
    <resource_limits>
        <max_objects_per_thread>12</max_objects_per_thread>
    </resource_limits>
</participant_factory_qos>
 
But I tried myself and it doesn't work: unfortunately our XML parser does not support that QoS. I already filed a request (CORE-8126) so that the core can be improved.
 
As a workaround I think what you are doing is very clever, but as far as I know the C++ api (that you are using to set the qos) and the C api (that connector uses) have different participant factory. If possible i would try to do exactly what you are doing but using the C api. Basically use 
 
 
The full documentation is here.  
 
Make sense? Let me know if that work. 
 
 
Best,
  Gianpiero
Top
Thu, 06/29/2017 - 11:55
gianpiero
gianpiero's picture
Offline
Last seen: 10 months 2 weeks ago
Joined: 06/02/2010
Posts: 177

Hello Jiqui, 

I was able to reproduce your issue and create a solution using your brilliant approach. Let me write down what I did so we have track of it. 

I created my addon in a file called init_factory.cc that i compile using the spec in binding.gyp. I then used the modifed ShapeExample.xml that you shared with me and the modified writer.js. 

You can find all the files here: https://gist.github.com/gianpiero/994bdc4c2b40480050e7fb31864ae98c 

I was able to run successfuly. Changing 

set factoryQoS.resource_limits.max_objects_per_thread = 1024 into init_factor.cc

leads to the erros:

[CREATE Participant]REDAWorkerFactory_createObjectPerWorker:!create objectPerWorker: worker factory's max. storage count reached (you may need to increase resource_limits.max_objects_per_thread value)
 
 
When I change 
 
set factoryQoS.resource_limits.max_objects_per_thread = 4096 into init_factor.cc
 
The errors are gone! 
 
The only things you have to be careful about is using the right version of the header files, for the right version of the librticonnector.so you are using.  To check the version of the library just run
 
strings librtiddsconnector.dylib | grep BUILD
Best,
  Gianpiero
Top
Fri, 07/07/2017 - 11:42
jqcheng
Offline
Last seen: 7 years 4 months ago
Joined: 06/17/2017
Posts: 2

Hi Gianpiero,

    The approach finally worked for me on Ubuntu 14.04. As stated before, I am using the default nodejs and node-gyp for Ubuntu 14.04. The nodejs and node-gyp are a bit outdated now.

gyp info using node-gyp@0.10.10
gyp info using node@0.10.25 | linux | x64

My compiler is gcc 4.8.4. The rticonnextdds-connector was upgraded to 0.3.0. And the rti dds was upgraded to 5.3.0. The DDS api was switched to C. The addon is as follows:

#include <node.h>
#include <v8.h>


extern "C" {
  #include "ndds/ndds_c.h"
}

using namespace v8;

Handle<Value> Method(const Arguments& args) {

    DDS_DomainParticipantFactoryQos *factoryQoS = (DDS_DomainParticipantFactoryQos*)malloc(sizeof(struct DDS_DomainParticipantFactoryQos));
    DDS_DomainParticipantFactoryQos_initialize(factoryQoS);
    DDS_DomainParticipantFactory* myfactory = DDS_DomainParticipantFactory_get_instance();
    if (myfactory==NULL) printf("Bad factory instance");
    DDS_DomainParticipantFactory_get_qos(myfactory, factoryQoS);
    printf("The previous resource limits %d\n",factoryQoS->resource_limits.max_objects_per_thread);
    factoryQoS->resource_limits.max_objects_per_thread = 4096;
    DDS_DomainParticipantFactory_set_qos(myfactory, factoryQoS);
    printf("The current resource limits %d\n",factoryQoS->resource_limits.max_objects_per_thread);
    DDS_DomainParticipantFactoryQos_finalize(factoryQoS);


  HandleScope scope;
  return scope.Close(String::New("true"));
}

void Init(Handle<Object> exports) {
  exports-&gt;Set(String::NewSymbol("dds_resource_init"),
      FunctionTemplate::New(Method)->GetFunction());
}

NODE_MODULE(dds_resource_init, Init)

The binding.gyp was modified as follows:

{
    "targets" : [
        {
            "target_name": "dds_resource_init",
            'include_dirs': [
              '/opt/rti_connext_dds-5.3.0/include/',
              '/opt/rti_connext_dds-5.3.0/include/ndds'
            ],
            'cflags_cc':[
               '-DRTI_UNIX',
               '-DRTI_LINUX',
               '-DRTI_64BIT',
               '-m64',
               '-O2'
            ],
            'link_settings': {
                    'ldflags': [
                      '-m64',
                      '-Wl,--no-as-needed'
                    ],
                    'libraries': [
                      '-ldl',
                      '-lnsl',
                      '-lm',
                      '-lpthread',
                      '-lrt',
                      '../node_modules/rticonnextdds-connector/lib/x64Linux2.6gcc4.4.5/librtiddsconnector.so',
                    ],
                  },
            "sources": [ "addon.cc" ]
        }
    ]
}

A final note is that the calling script must be located at correct directory level. Otherwise, the addon will have difficulties to load the shared lib.

Thanks for the support!

 

 

 

 

Top
Tue, 01/14/2020 - 01:06
samr
Offline
Last seen: 1 month 2 weeks ago
Joined: 10/22/2018
Posts: 91

For anybody else running into this issue. In the Connector 1.0.0 release we added a new API to both the Python Connector and JavaScript Connector which can be used to configure the max_objects_per_thread. In both cases, it can only be called before a Connector is created.

Top