App behavior when shared memory fails

3 posts / 0 new
Last post
Offline
Last seen: 6 years 5 months ago
Joined: 06/10/2014
Posts: 49
App behavior when shared memory fails

Hi -

I am running several DDS applications on a single Ubuntu server.  I have the initial peers set in the QOS profile as shared memory, builtin.udpv4://127.0.0.1 and multicast.  I was having trouble with one application being unable to receive samples from another application running on the same server.  I verified that the sample was being published using rtiadminconsole.

When I looked at the logs I saw the the subscribing application was logging this error:

NDDS_Transport_Shmem_send:failed to add data. shmem queue for port 0x1eea is full (received_message_count_max=64, receive_buffer_size=65536). Try to increase queue resource limits.

I ended up disabling shared memory but I am wondering why if shared memory failed, DDS would not publish the sample using the local ip address?  The subscribing application participant listed 3 unicast locators (the server has 2 nics):

[0]   SHMEM: 0.0.5.5.4.1.0.0.0.0.0.0.0.0.0.0

[1] UDPv4: 10.1.10.203, port:7925

[2] UDPv4: 10.1.0.11, port 7925

I thought a writer would send a sample to each locator that the participant advertises (if it is reachable).

Thank you,

Anne Fiore

 

Organization:
Gerardo Pardo's picture
Offline
Last seen: 1 day 13 hours ago
Joined: 06/02/2010
Posts: 601

Hello Anne Fiore,

Currently (as of Connext 5.2) if two DomainParticipants are running on the same computer (or VM instance) and they both have shared memory active, then Connext DDS will prefer the shared-memory transport for sending information between the two DomainParticipants. If one of the Participants is in another computer or it does not enable shared memory then Connext will use all the other locators when sending to that Participant, exactly as you said.

There is one exception when multicast is available. In this case Multicast is preferred over shared memory. 

So in a normal deployment when you have the receiving DomainParticipants both in the same computer and on separate computers relative to the sending DomainParticipant then the sending Participant will:

  • Send over SharedMemory to the participants in the same computer
  • Send over the announced IP addresses to the DomainPartiipants in  other computers. This is exactly as you said.

What version of Connext DDS are you using? It appears that someone may have modified the default settings for the SharedMemory transport, is that the case? If so you should note that 65536 is really a very small value for the receive_buffer_size.Typically we recommend receive_buffer_size values close to 1 MByte. This is in fact the out-of-the-box setting for Connext 5.2... Perhaps you meant the message_size_max instead? This one is set by default to 65507 which is close to the 64KB number you showed.

Gerardo

Offline
Last seen: 6 years 5 months ago
Joined: 06/10/2014
Posts: 49

Hello Gerardo,

Thank you for this information.  We are using Connext 5.2.  We do have these shared memory values in our participant QOS properties:

<element>
<name>dds.transport.shmem.builtin.parent.message_size_max</name>
<value>65536</value>
</element>
<element>
<name>dds.transport.shmem.builtin.receive_buffer_size</name>
<value>65536</value>
</element>

We just transitioned from Connext 5.1 to 5.2 and these are the values we were using with 5.1.  I removed the receive_buffer_size from the QOS so that the default is used and did not get a shared memory error so it seems that we were setting this to be too small.

Thanks for your help.

Anne Fiore