Failure to Create a Participant due to Insufficient Shared Memory Kernel Settings
If you are running RTI Connext DDS 5.2.0 or later, you may get this error message when starting your application:
[D0062|ENABLE]DDS_DomainParticipantPresentation_reserve_participant_index_entryports:!enable reserve participant index [D0062|ENABLE]DDS_DomainParticipant_reserve_participant_index_entryports:Unusable shared memory transport. For a more in-depth explanation of the possible problem and solution, please visit https://community.rti.com/kb/osx510.
If you are running RTI Connext DDS 5.1.0 or earlier, you may get this error message when starting your application:
[D0061|ENABLE]NDDS_Transport_Shmem_create_recvresource_rrEA:failed to initialize shared memory resource segment for key 0x405971 [D0061|ENABLE]NDDS_Transport_Shmem_create_recvresource_rrEA:failed to initialize shared memory resource segment for key 0x405973 [D0061|ENABLE]DDS_DomainParticipantPresentation_reserve_participant_indexI:!enable reserve participant index [D0061|ENABLE]DDS_DomainParticipant_enableI:Automatic participant index failed to initialize. PLEASE VERIFY CONSISTENT TRANSPORT / DISCOVERY CONFIGURATION. DDSDomainParticipant_impl::createI:ERROR: Failed to auto-enable entity DomainParticipantFactory_impl::create_participant():!create failure creating participant create_participant error
This error is caused by an insufficient number or size of shared memory segments allowed by the operating system.
As a result, the DomainParticipant is unable to allocate enough resources and calculate its participant index which causes the error. When this happens:
- In RTI Connext 5.2.0 and later, the participant will be successfully created as long as one of the installed transports is valid and usable.
- In RTI Connext 5.1.0 and earlier, however, the participant will fail to be created, even if the participant has another usable transport installed.
This problem occurs most frequently on OSX due to the default kernel settings that control the resources needed by the RTI Connext shared memory transport.
To address this issue, you can increase the shared memory resources of your machine either temporarily or permanently. Note that the settings in this solution are a recommendation.
If you have a small number of applications you can use more conservative settings if necessary.
In macOS Catalina and later:
1.) Change user to root and go into the directory /Library/LaunchDaemons/
2.) Create a file within /Library/LaunchDaemons/ named memory.plist and add the content below:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>shmemsetup</string> <key>UserName</key> <string>root</string> <key>GroupName</key> <string>wheel</string> <key>ProgramArguments</key> <array> <string>/usr/sbin/sysctl</string> <string>-w</string> <string>kern.sysv.shmmax=419430400</string> <string>kern.sysv.shmmin=1</string> <string>kern.sysv.shmmni=128</string> <string>kern.sysv.shmseg=1024</string> <string>kern.sysv.shmall=262144</string> </array> <key>KeepAlive</key> <false/> <key>RunAtLoad</key> <true/> </dict> </plist>
3.) From within the /Library/LaunchDaemons/ directory, run:
sudo launchctl load memory.plist
4.) Restart your computer
5.) Verify that the settings have taken effect, e.g. in the terminal run:
sysctl kern.sysv.shmmax
This should print:
kern.sysv.shmmax: 419430400
Before macOS Catalina:
To increase the settings temporarily, you can run the following commands as user root:
/usr/sbin/sysctl -w kern.sysv.shmmax=419430400 /usr/sbin/sysctl -w kern.sysv.shmmin=1 /usr/sbin/sysctl -w kern.sysv.shmmni=128 /usr/sbin/sysctl -w kern.sysv.shmseg=1024 /usr/sbin/sysctl -w kern.sysv.shmall=262144
To increase the settings permanently, you will need to edit or create the file /etc/sysctl.conf. Creating or editing this file will require root permissions. Either add to your existing /etc/sysctl.conf file or create /etc/sysctl.conf with the following lines:
kern.sysv.shmmax=419430400 kern.sysv.shmmin=1 kern.sysv.shmmni=128 kern.sysv.shmseg=1024 kern.sysv.shmall=262144
You will need to reboot the machine after modifying this file to have the changes take effect.
As a convenience, you can run the attached shell script, shmem_osx.sh (contained in shmem_osx.zip), to temporarily change the shared memory settings of your system. It will also ask you if you would like to make the changes permanent and will either create or append to /etc/sysctl.conf if you choose to have it do so. You must run the script as user root.
Comments
mrivera
Fri, 12/22/2023 - 11:34
Permalink
RE: Failure to Create a Participant Insufficient Shared Memory
You can also recieve this error if you're trying to run a large number of participants, for example if you are trying to run 100 participants on the same macOs node using shared memory to communicate with each other. The proposed values will not be enough. In that case we will need to re-increase the shared memory configuration and also the semaphores configuration. But why?
Well, each Connext DDS application that has shared memory enabled allocates 4 individual semaphores so for running 100 participants the number of semaphore identifiers (semmni) should be > 400 and the default value in macOs is so far away from that.
Same happens with the shared memory settings we will need to increase the max number of shared memory identifiers (shmmni) and max shared memory segments per process (shmseg) to allow all our participants to work properly.
To check your current shared memory settings in macOs run the following command:
ipcs -M && /usr/sbin/sysctl kern.sysv.semmni kern.sysv.semmns kern.sysv.semmnu kern.sysv.semmsl kern.sysv.semumeIt will return something like:
IPC status from <running system>shminfo:
shmmax: 4194304 (max shared memory segment size)
shmmin: 1 (min shared memory segment size)
shmmni: 32 (max number of shared memory identifiers)
shmseg: 8 (max shared memory segments per process)
shmall: 1024 (max amount of shared memory in pages)
kern.sysv.semmni: 8 (# of semaphore identifiers)kern.sysv.semmns: 128 (# of semaphores in system)
kern.sysv.semmnu: 0 (# of undo structures in system)
kern.sysv.semmsl: 87381 (max # of semaphores per id)
kern.sysv.semume: 10 (max # of undo entries per process)
(Even if the values may differ a little bit depending on your current os version and arch you're still not going to be able to run 100 participants with these settings)
To make our system capable to run 100 participants simoultaneously we will need to increase those default values as we made before:
1. Change user to root and go into the root directory then go into:
/Library/LaunchDaemons/2. Run the following commands :
launchctl unload memory.plistrm memory.plist
launchctl unload sysctl.plist
rm sysctl.plist
3. Reboot
4. Change user to root and go again into the directory
/Library/LaunchDaemons/5. Create the new files memory.plist and sysctl.plist :
sysctl.plist:
memory.plist:
6. Run the following commands:
sudo launchctl load memory.plistsudo launchctl load sysctl.plist
7. Reboot
8. Check the changes has been applied succesfully by running the previous command:
ipcs -M && /usr/sbin/sysctl kern.sysv.semmni kern.sysv.semmns kern.sysv.semmnu kern.sysv.semmsl kern.sysv.semumeAfter all the process it should return:
IPC status from <running system> shminfo:shmmax: 419430400 (max shared memory segment size)
shmmin: 1 (min shared memory segment size)
shmmni: 1024 (max number of shared memory identifiers)
shmseg: 16384 (max shared memory segments per process)
shmall: 262144 (max amount of shared memory in pages)
kern.sysv.semmni: 87381kern.sysv.semmns: 87381
kern.sysv.semmnu: 87381
kern.sysv.semmsl: 87381
kern.sysv.semume: 10
If not please check if you have disabled System Integrity Protection (SIP) which could be preventing those changes to be applied permanently on your system by running:
csrutil statusIMPORTANT: : Those changes should be done VERY carefully —incorrect editing of the files can affect your system performance!