How can I prevent a Stack Overrun on my VxWorks system?

Note: Applies to RTI Data Distribution Service 4.0 and 4.1

If you notice unexpected behavior (for example, unintended functions getting called, object creation failing, etc.), check your current stack margin by running the command checkStack. If the margin is low, this is a possible explanation of the problem. 

The maximum stack usage during creating/enabling of a DomainParticipant is slightly more than 20KB in some cases. Since the default VxWorks shell task is typically configured with a stack size of 20KB, it is possible to run out of stack space and experience the side effects of stack overrun. 

One solution is to create/enable the DomainParticipant in a thread with a larger stack. On VxWorks, you may use the command  taskSpawn to spawn a new task with your desired priority, option, and stack size. 

For example, suppose your main entrypoint for RTI Data Distribution Service participant creation/enabling is:

void MyRTIddsMain(int arg);

Instead of executing this directly in the shell (or start-up shell), you can create a new function as follows:

void StartMyRTIddsMain() 
{ 
    taskSpawn("MyRTIddsMain", 100, 0, 30000, 
              (FUNCPTR)MyRTIddsMain, arg, 0,0,0,0,0,0,0,0, 0); 
}

Another solution is to increase the default stack size of the shell task by recompiling the kernel.