Why do I get an OSSL_STORE API error and crash when creating a secure participant on Linux?

On Linux systems, if you are creating a DomainParticipant with Security Plugins enabled, and your application is linked to both TensorFlow and OpenSSL, you may see errors like this:

RTI_Security_Util_uriToStringOssl:OpenSSL function PEM_write_bio_X509 failed with error: error:0c00009b:ASN.1 encoding routines:OPENSSL_internal:MISSING_VALUE
RTI_Security_Util_uriToStringOssl:OpenSSL function PEM_write_bio_X509 failed with error: error:0900000c:PEM routines:OPENSSL_internal:ASN.1 encoding routines
RTI_Security_Util_uriToStringEx:!read file using OSSL_STORE API. Attempting to read it using BIO API. URI: /path/to/file.pem  
free(): invalid pointer 

The errors may be accompanied by a crash and "free(): invalid pointer".

The problem is that libcrypto.so (i.e. OpenSSL; a dependency of Security Plugins) and libtensorflow_framework.so.2 (i.e. a project of Google, completely unrelated to RTI) both use the same symbol names for certain functions that are hit during initialization of a DomainParticipant with Security enabled.

In particular, PEM_write_bio_X509 (from the error messages above) is present in both libcrypto.so and libtensorflow_framework.so.2. If libtensorflow_framework.so.2 is loaded first, the application on Linux may try to use the TensorFlow version of PEM_write_bio_X509, which is not correct. Security Plugins require the OpenSSL version of PEM_write_bio_X509. The crash can occur when the OpenSSL OPENSSL_sk_pop_free symbol is called after the TensorFlow PEM_write_bio_X509 call.

To fix the problem, there are different options, including:

1.) You can export LD_PRELOAD as shown below before running your app, to force libcrypto.so to take precendence, e.g.:

export LD_PRELOAD=/path/to/openssl-3.0.12/x64Linux4gcc7.3.0/debug/lib/libcrypto.so

By exporting LD_PRELOAD like this, you force libcrypto to be used.

The Security Plugins User's Manual discusses using LD_PRELOAD to overcome similar problems when the Operating System OpenSSL library is incorrectly being loaded instead of the RTI OpenSSL distribution. See here for more details.

2.) Alternatively, you can build TensorFlow with OpenSSL symbols hidden, which can also resolve the problem.