HOWTO Use DDS on Xenomai: Configure Xenomai and Run RTI Connext DDS

1. Introduction

RTI has recently released an experimental port of RTI Connext DDS to Xenomai 2.6.1 through its Community Portal. Xenomai is a real-time development framework that provides interface-agnostic hard real-time support to user-space applications in a Linux environment. The purpose of this document is to describe the installation and configuration of a Xenomai environment to develop applications using RTI Connext DDS.

We have chosen Ubuntu Linux 10.04 (with Linux 2.6.38 kernel) to illustrate that process, although the installation of Xenomai on different systems would be pretty similar. If you want to test RTI Connext with Xenomai in your environment without making changes to your system, you can download a bootable ISO with Xenomai and the RTI software here.

This HOWTO is organized as follows: first we describe how to build a customized kernel prepared to fit Xenomai's needs; then, we illustrate how to install the RTI Connext DDS target in your Xenomai environment; and finally, we create a "hello world" example to test the installation and describe the steps to start developing applications.

2. Set Up Xenomai Environment

2.1 Configure and Compile Kernel

The first thing you need to do when configuring a Xenomai Environment is to patch and compile a kernel. This requires the installation of some dependencies from the Ubuntu repository, such as a compiler and some development libraries.

sudo apt-get install build-essential libncurses5-dev kernel-package

Once you have installed all the dependencies, download the kernel sources to be patched. We will use a vanilla kernel from kernel.org:

wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.38.8.tar.bz2
sudo tar xvjf linux-2.6.38.8.tar.bz2 -C /usr/src/

Now, download and untar the latest Xenomai bundle.

wget http://download.gna.org/xenomai/stable/xenomai-2.6.1.tar.bz2
sudo tar xvjf xenomai-2.6.1.tar.bz2 -C /usr/src/

Among other things, the Xenomai bundle includes a script to patch the kernel with the Adeos-Ipipe patch. Although it also includes patches for some kernel versions, if you need a specific patch for the kernel you are using, you can find the whole list of patches here. Once you have found a patch for the kernel you downloaded, run the script:

cd /usr/src/xenomai-2.6.1/
sudo ./scripts/prepare-kernel.sh --linux=/usr/src/linux-2.6.38.8 --adeos=/usr/src/xenomai-2.6.1/ksrc/arch/x86/patches/adeos-ipipe-2.6.38.8-x86-2.11-01.patch --arch=x86

Now you will need to configure some options before compiling. You can find documentation on these options in the Xenomai Troubleshooting Guide as well as in the Xenomai README. We recommend you to start from the configuration options of the kernel you are currently running before running make menunconfig.

sudo cp /boot/config-`uname -r` /usr/src/linux-2.6.38.8/.config

cd /usr/src/linux-2.6.38.8
sudo make menuconfig

These are the options we recommend you to modify:

• Processor type and features
  • Select your Processor Family. It is important to take into account the features the processor implements later when compiling Xenomai. For instance, you cannot enable TSC when configuring Xenomai if your processor does not have a TSC.
  • Disable HPET Timer Support (no longer necessary with I-pipe for Linux 3.2)
  • Disable -fstack-protector buffer overflow detection (not necessary on x86_32)
• Power management and ACPI options
  • Disable APM
  • ACPI (Advanced Configuration and Power Interface)
    • ACPI (Advanced Configuration and Power Interface) Support
      • Disable processor.
  • CPU frequency scaling
    • Disable CPU frequency scaling
  • Device Drivers
    • Input device support
      • Miscellaneous devices
        • Disable PC Speaker (no longer necessary with I-pipe for Linux 3.2)

You can also download the configuration file we used in our Xenomai Live ISO here.

Once the kernel has been configured, we will use make-kpkg to compile it. This application generates.deb packages with the kernel that can be reused for other installations. In this case we are going to generate two .deb files: kernel-image and kernel-headers. make-kpkg can only be used in Debian "derivatives" (e.g., Debian, Ubuntu, and Linux Mint), but you can find instructions on how to compile Xenomai for other distributions here.

sudo fakeroot make-kpkg --initrd --append-to-version=-xenomai-2.6.1 kernel-image kernel-headers

Now, using the packages generated, you can install the customized kernel on your system.

sudo dpkg -i *.deb
sudo update-initramfs -c -k "2.6.38.8-xenomai-2.6.1"
sudo update-grub

To run real-time applications with your user, create a xenomai group and add the user to it.

sudo addgroup xenomai
sudo adduser <user_name> xenomai

Before booting the new kernel, we need to add some options to the grub command line, including Xenomai group ID. To get the Xenomai group ID run:

cat /etc/group | sed -nr "s/xenomai:.:([0-9]+):.*/\1/p"

Finally, edit /etc/defaults/grub to add the following arguments to boot the kernel:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash noapic xeno_nucleus.xenomai_gid=<xenomai_gid>"

and save the changes

sudo update-grub

2.2 Configure and Install Xenomai User Libraries

Once you have installed the new kernel and set up your system, you can install Xenomai and run some tests to validate your configuration. Using the configure script, you can adapt Xenomai installation to your environment. For instance, in our installation we enabled SMP support and SEP instructions for issuing syscalls, among other things. In the Xenomai Installation Guide you will find more information on the options you can pass to the configure script. To configure, compile, and install Xenomai run:

cd /usr/src/xenomai-2.6.1/
./configure --enable-shared --enable-smp --enable-posix-auto-mlockall --enable-x86-sep
make
sudo make install

Xenomai is now installed under /usr/xenomai. If you want to use Xenomai's dynamic libraries you will need to add /usr/xenomai/lib to your LD_LIBRARY_PATH. You can create add a new configuration file under /etc/ld.so.conf.d/ to ensure the lib directory is searched when starting the application.

# xenomai libs
/usr/xenomai/lib

If you create the file, run the following command to update the dynamic linker run-time bindings.

sudo ldconfig

Finally, you can run a latency test to see the installation. You will need to boot your new kernel before running the test if you have not rebooted yet. You should see something like this:

/usr/xenomai/bin/latency
== Sampling period: 100 us
== Test mode: periodic user-mode task
== All results in microseconds
warming up...
RTT|  00:00:01  (periodic user-mode task, 100 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|      1.615|      1.923|      9.846|       0|     0|      1.615|      9.846
RTD|      1.615|      1.923|      9.692|       0|     0|      1.615|      9.846
RTD|      1.538|      1.923|     10.230|       0|     0|      1.538|     10.230

You can find more information on the latency test here.

3. Install RTI Connext DDS on Xenomai

Now that your environment is configured, we are ready to install RTI Connext on it. You will need to download and install the Host bundle for i86Linux and the Xenomai target self-extracting installer. Assuming you want to install the distribution in /home/rti/rticonnext, run:

cd /home/rti/rticonnext/
tar xvzf /path/to/connext/RTI_Connext_DDS_Host-5.0.0-i86Linux.tar.gz

Now, set the NDDSHOME environment variable this directory:

export NDDSHOME=/home/rti/rticonnext/ndds.5.0.0

The RTI Connext DDS Xenomai target installer will use NDDSHOME to extract all the files to the installation directory. Now, run the self-extracting installer (make sure first it has execute permission):

./RTI_Connext_DDS_Target-5.0.0-i86Xenomai2.6.1gcc4.4.3.run
Verifying archive integrity... All good.
Uncompressing RTI Connext DDS 5.0.0 -- i86Xenomai2.6.1gcc4.4.3 Port ..................................
Installer extracting to temporary directory: /tmp/selfgz1949
Installing RTI Connext DDS 5.0.0
NDDSHOME set to: /home/rti/rticonnext/ndds.5.0.0
Enter absolute path to installation directory: [/home/rti/rticonnext/ndds.5.0.0]
Installation directory is: /home/test/test/rticonnext/ndds.5.0.0
Specified directory recognized as valid Connext NDDSHOME
Proceed to install? [Y/n] Y

Now, you are ready to create a hello world example using RTI Connext on Xenomai.

4. Hello World Example

To test the installation we will now run a simple "hello world" application using RTI Connext. First, create a new directory for your example and edit a new file called hello.idl. with the following text:

struct hello_message {
    string name; //@key
    long count;
};

Now, using rtiddsgen, create a C++ example:

$NDDSHOME/scripts/rtiddsgen -example i86Xenomai2.6.1gcc4.4.3 -language C++ hello.idl

As you can see, rtiddsgen has created a bunch of new files, including a makefile and two source files with the publisher and subscriber application. Edit hello_publisher.cxx and add some data to send, for instance:

        printf("Writing hello_message, count %d\n", count);

        /* Modify the data to be sent here */
        strcpy(instance->name, "Fernando");
        instance->count = count;

        retcode = hello_message_writer->write(*instance, instance_handle);
        if (retcode != DDS_RETCODE_OK) {
            printf("write error %d\n", retcode);
        }

        NDDSUtility::sleep(send_period);

Now, compile the example using the makefile generated by rtiddsgen. Note that the makefile assumes that Xenomai is installed under /usr/xenomai—the default installation directory.

make -f makefile_hello_i86Xenomai2.6.1gcc4.4.3

If you want to create your own makefiles or run GCC by hand, you may want to check out xeno-config. This script provides, among other things, the cflags and ldflags you will need to use for compiling. For instance, to get the cflags you need to compile an application using the POSIX skin run:

/usr/xenomai/bin/xeno-config --skin posix --cflags
-I/usr/xenomai/include -D_GNU_SOURCE -D_REENTRANT -D__XENO__ -I/usr/xenomai/include/posix

Finally, run the publisher and subscriber applications and check whether they are able to communicate.

./objs/i86Xenomai2.6.1gcc4.4.3/hello_publisher 
Writing hello_message, count 0
Writing hello_message, count 1
Writing hello_message, count 2
Writing hello_message, count 3

./objs/i86Xenomai2.6.1gcc4.4.3/hello_subscriber 
hello_message subscriber sleeping for 4 sec...

   name: "Fernando"
   count: 1
hello_message subscriber sleeping for 4 sec...

   name: "Fernando"
   count: 2
hello_message subscriber sleeping for 4 sec...

   name: "Fernando"
   count: 3

NOTE ON REAL-TIME THREADS

As Xenomai documentation points out: "to get your application threads to be really considered as real-time threads by Xenomai scheduler, you will have to get them to use the real-time scheduling policy (called SCHED_FIFO)". To ease this process, we have added a call to pthread_setschedparam() in the code that returns the first instance of the Domain Participant Factory. That is, we modify scheduling policy—and the priority—of the thread calling DDSTheParticipantFactory->create_participant() in case its scheduling policy is not SCHED_FIFO. By default, we set the priority of the thread to 60 for non-real-time, but we will keep the priority of real-time threads.

Note also the implications of the use of the SCHED_FIFO scheduling policy described in the Guide to Port POSIX Applications to Xenomai: "SCHED_FIFO scheduling policy means that the scheduler will run a thread with this scheduling policy as long as it is runnable, and no other thread of higher priority is runnable. Concretely, if a thread using the SCHED_FIFO policy runs an infinite loop, nothing else runs, your system is locked up".