More than 1 participant per process

6 posts / 0 new
Last post
Offline
Last seen: 2 months 1 week ago
Joined: 04/04/2015
Posts: 4
More than 1 participant per process

Hello -

I have a question about creating participants.   Can I create more than one participant in the same process ?     If yes, are there  any pitfalls to be aware of?

I am on  Linux ( RHEL 7 ) and using rti 5.3.1.

I plan to call this function to create the participants:

DDSDomainParticipantFactory::get_instance()->create_participant_with_profile(...)

 

jwillemsen's picture
Offline
Last seen: 3 months 1 week ago
Joined: 09/24/2013
Posts: 54

Yes, you can create multiple domain participants in the same process. Be aware that each domain participant uses additional resources like threads, so limit the amount of participants to the minimum, see also https://community.rti.com/kb/what-maximum-number-participants-domain

Howard's picture
Offline
Last seen: 2 days 14 hours ago
Joined: 11/29/2012
Posts: 130

So to add more to Johnny's accurate comments, besides threads, participants will also consume memory (both process and shared) and sockets.  Also the more participants that you have in a system, the more CPU and network bandwidth is taken up in the discovery process.

So, this all leads to...you have to have a good reason to have multiple participants in a process.  Mainly it's divided into 2 categories

1) your system design requires more than 1 DDS domain, and an application needs to communicate in multiple DDS domains

2) you have data streams of different priority and would like DDS to deal with the data at different priorty levels (you can set the priority of DDS's receive threads, but unfortunately, Connext only allows you to set a single value used by all receive threads in a Participant).

 

Offline
Last seen: 2 months 1 week ago
Joined: 04/04/2015
Posts: 4

Thanks for the response.    My use case is very close to item 2  ( See response from Howard ) and I will only have 2 participants per process.  But I have a followup quesiton.  What is the maximum number of processes I can have on a single 'node'  in a system?  If each process has 2 participants and I have 8 processes on a node, I have 16 participants.   I know these will likely communicate DDS over shared memory so I have set my linux kernel to allow the maximum amount of shared memory.   The reason I ask this question is I am having trouble getting discovery and/or subscription matches to take place  on a single node with this numbers.

Thanks.

KP Singh's picture
Offline
Last seen: 4 days 22 hours ago
Joined: 03/05/2020
Posts: 11

Dear ed_hourigan

DDS supports upto 120 DomainParticipants on a single DDS DomainId on a machine (OS instance) i.e Node. This limitation comes from the fact that each DomainParticipants allocates 2 ports (which must get unique port numbers within a single computer). Since by default a DDS DomainId reserves a range of 250 ports you have the limit of 120 participants per domain per computer.

You can use this Spreadsheet to compute the UDP ports used by RTI Connext DDS to see the ports that would be assigned for each DomainId and participantId.

As you described, that your environment has 8 processes with 2 domain participants (DP). hence total 16 DP which are within the limits that are supported by DDS.

Now, sensing your problem.

We have to look in how discovery proceeds. Default value of NDDS_DISCOVERY_PEERS is 239.255.0.1,127.0.0.1,builtin.shmem://.

All participants in the same domain share same multicast receive address. Hence, if multicast is enabled, then they should discover each other. In addition to the multicast receive port, each DP allocates two port (one for discovery and one for data) and they are unique based on domainparticipant index.

Now, we have to understand the peer descriptor format. When we say 127.0.0.1, it means 4@127.0.0.1. Hence, DP will send discovery data on ports for Domain Participant[0,1,2,3,4]

For more info, refer documentation https://community.rti.com/static/documentation/connext-dds/6.0.1/doc/manuals/connext_dds/html_files/RTI_ConnextDDS_CoreLibraries_UsersManual/index.htm#UsersManual/Peer_Descriptor_Format.htm#14.2.1_Peer_Descriptor_Format%3FTocPath%3DPart%25203%253A%2520Advanced%2520Concepts%7C14.%2520Discovery%7C14.2%2520Configuring%2520the%2520Peers%2520List%2520Used%2520in%2520Discovery%7C14.2.1%2520Peer%2520Descriptor%2520Format%7C_____0

 

Try setting NDDS_DISCOVERY_PEERS=239.255.0.1,15@127.0.0.1,15@builtin.shmem://

If you are opening adminconsole or other DDS entities on the same machine, increase the factor from 15 to higher value like 20 as each DP in that domain will use one set of ports.

Regards

KP Singh

 

Offline
Last seen: 2 months 1 week ago
Joined: 04/04/2015
Posts: 4

I will give this a try

Thanks!