I have an application that is statically linking to routing service and some other rti libraries.
I have my build setup to statically link these libraries:
- librtiroutingservicez.a
- librticonnextmsgcppz.a
- librticonnextmsgcz.a
- librtidlcppz.a
- librtidlcz.a
- libnddscpp2z.a
These libraries are be dynamically linked:
- libnddsc.so
- libnddscpp.so
- libnddscore.so
Occasionally the application crashes with a "double free or corruption" error.
I saw from another question (https://community.rti.com/forum-topic/double-free-error-exiting-certain-applications) that this might be due to statically linking the rti libraries. I am changing the code to dynamically link all the rti libraries but was wondering if anyone has any more information on this. This crash (so far) has not been reproduceable but will happen occassionally so I'm not sure that changing the linking will actually fix the problem.
This is a backtrace of one core file. In frame 5 RTIOsapiHeap_freeMemoryInternal () is trying to free memory and I see the "double free" error at frame 3.
#0 0x00007f51cf6b7cc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
resultvar = 0
pid = 1425
selftid = 20425
#1 0x00007f51cf6bb0d8 in __GI_abort () at abort.c:89
save_stage = 2
act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0},
sa_mask = {__val = {0, 8391157485091964242, 139989047325480, 0, 0,
139989372582265, 139989033079280, 139988503065480,
139989369278200, 139985869078528, 139989033079280,
139985869078529, 0, 17179869185, 0, 0}}, sa_flags = 0,
sa_restorer = 0x7f519cff7440}
sigs = {__val = {32, 0 <repeats 15 times>}}
#2 0x00007f51cf6f4394 in __libc_message (do_abort=do_abort@entry=1,
fmt=fmt@entry=0x7f51cf802b28 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/posix/libc_fatal.c:175
ap = {{gp_offset = 40, fp_offset = 0,
overflow_arg_area = 0x7f519cff7130,
reg_save_area = 0x7f519cff70c0}}
fd = 2
on_2 = <optimized out>
list = <optimized out>
nlist = <optimized out> cp = <optimized out> written = <optimized out>
#3 0x00007f51cf70066e in malloc_printerr (ptr=<optimized out>,
str=0x7f51cf802c10 "double free or corruption (!prev)", action=1)
at malloc.c:4996
buf = "00007f51b01c6260"
cp = <optimized out>
#4 _int_free (av=<optimized out>, p=<optimized out>, have_lock=0)
at malloc.c:3840
size = <optimized out>
fb = <optimized out>
nextchunk = <optimized out>
nextsize = <optimized out>
nextinuse = <optimized out>
prevsize = <optimized out>
bck = <optimized out>
fwd = <optimized out>
errstr = <optimized out>
locked = <optimized out>
#5 0x00007f51d0d04622 in RTIOsapiHeap_freeMemoryInternal ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
#6 0x00007f51d0ce825e in REDAFastBufferPool_growEmptyPoolEA ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#7 0x00007f51d0ce8d5c in REDAFastBufferPool_getBuffer ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#8 0x00007f51d0cec081 in REDASkiplistDefaultAllocator_createSkiplistNode ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#9 0x00007f51d0cecc69 in REDASkiplistNode_new ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#10 0x00007f51d0ced827 in REDASkiplist_assertNodeEA ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#11 0x00000000005ceae9 in ROUTERDdsConnection_assertSubscriptionFilter ()
No symbol table info available.
#12 0x00000000005da75f in ROUTERDdsSubBuiltinReaderListener_onDataAvailable ()
No symbol table info available.
#13 0x00007f51d118e591 in DDS_DataReaderListener_forward_onDataAvailable ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddsc.so
No symbol table info available.
#14 0x00007f51d1196289 in DDS_DataReader_impl_forward_onDataAvailable ()
#15 0x00007f51d0a65a29 in PRESPsReader_addSample ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#16 0x00007f51d0974f79 in DISCEndpointDiscoveryPlugin_assertRemoteEndpoint ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#17 0x00007f51d097858b in DISCEndpointDiscoveryPlugin_assertRemoteReader ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#18 0x00007f51d098929d in DISCSimpleEndpointDiscoveryPlugin_subscriptionReaderListenerOnDataAvailable ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#19 0x00007f51d0a7cdb5 in PRESPsService_readerNotifyOfReaderQueueChanges ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#20 0x00007f51d0a7e9c0 in PRESPsService_readerSampleListenerOnNewData ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#21 0x00007f51d0bb4915 in COMMENDSrReaderService_onSubmessage ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
#22 0x00007f51d0bfe4f3 in MIGInterpreter_parse ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#23 0x00007f51d0b886b8 in COMMENDActiveFacadeReceiver_loop ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#24 0x00007f51d0d06346 in RTIOsapiThreadChild_onSpawned ()
from /opt/DocBoxInc/RTI/ndds.5.2.0/lib/x64Linux3gcc4.8.2/libnddscore.so
No symbol table info available.
#25 0x00007f51d049d182 in start_thread (arg=0x7f519cff9700)
at pthread_create.c:312
__res = <optimized out>
pd = 0x7f519cff9700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139988503074560,
702956678290169682, 1, 1, 139988503075264, 139988503074560,
-620829754110943406, -620714117499232430},
mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0},
data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#26 0x00007f51cf77b47d in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.
Thanks for any information.
Anne Fiore
Hi Anna,
Sorry for the late response. I was wondering if you were able to get to the bottom of the issue. Please let us know.
Gianpiero
Hi Gianpiero,
I ended up linking dynamically to the routing service libraries. I haven't seen this crash since I changed to dynamic linking but I was never able to find the exact cause of the crash. I suspect something my app was doing that was timing related.
Thanks for the followup.
Anne
Anne,
Thanks for letting me know!
Best,
Gianpiero