5.8. Port Validation¶
5.8.1. Introduction¶
This section explains how to build and run the Connext Micro Port Validation and is organized as follows:
5.8.2. Overview¶
After porting Connext Micro, it is important to confirm that your code works as expected. For this, Connext Micro comes with a suite of tests that you compile and run to validate your port.
The tests only cover the functionality described in the porting instructions earlier in this chapter Porting RTI Connext Micro.
The tests are a subset of the tests RTI runs internally. They are just exported for your use. RTI does not support any changes to the tests. The tests are built with RTI’s internal unit testing framework, ‘UTEST’. Everything needed to run the tests is shipped along with the rest of Connext Micro. The directory layout is as follows:
RTIMEHOME/----+---- CMakeLists.txt
|
+---- include
|
+---- src
|
+---- resource
|
+---- lib
|
+---- build
|
+---- test --+-- test --+-- setting --- <UTEST-SRC>
|
+-- osapi --+-- common -- <OSAPI-TEST-SRC>
| |
| +-- test ---- <OSAPITester>
|
+-- netio --+-- autosar ---- <NETIO-TEST-SRC>
| |
| +-- common ----- <NETIO-TEST-SRC>
| |
| +-- test ------- <NETIOTester>
| |
| +-- udp ------- <NETIO-TEST-SRC>
|
+-- include --- test ---- <UTEST-HDR>
The test folder includes four sub-folders. The ‘test’ and ‘include’ folders contain the UTEST framework that is required to run the unit tests. ‘osapi’ and ‘netio’ both contain common folders (containing the test source), as well as test folders (containing the test files).
5.8.3. Building the Port Validation Tests¶
By default, the port validation tests are not built. We recommend that you review Building the Connext Micro Source, since the same rules and considerations apply when building the port validation tests.
If you will be using the ctest (CMake test driver program) set the domain ID used to run the tests using this environment variable in your terminal:
On Linux and macOS systems:
export RTIME_TEST_CONFIG_ID="<your domain ID #>"
On Windows systems:
set RTIME_TEST_CONFIG_ID="<your domain ID #>"
5.8.3.1. Building with rtime-make¶
Use the option --test when running ‘rtime-make’.
On Linux and macOS systems:
<RTIMEHOME>/resource/scripts/rtime-make --config Debug --target self \
--name i86Linux2.6gcc4.4.5 -G "Unix Makefiles" --build --test
On Windows systems:
<RTIMEHOME>\resource\scripts\rtime-make.bat --config Debug --target self \
--name i86Win32VS2010 -G "Visual Studio 10 2010" --build --test
Explanation of arguments:
--test: Build the port validation tests.--config Debug: Create a Debug build.--target <target>: The target for the source files to be built. See Building Connext Micro with rtime-make for information on specifying the target architecture. “self” indicates that the host machine is also the target and Connext Micro will be built with the options that CMake automatically determines for the local compiler.--name <name>: The name of the build. Use a descriptive name following the recommendations on naming in section Preparing for a Build. If--nameis not specified, the value for--targetwill be used as the name.--build: Build the generated project files.
5.8.3.2. Manually building with CMake¶
The process for building the port validation tests manually with CMake
is the same as building the Connext Micro libraries manually with CMake.
Follow the instructions in Manually Building with CMake.
To build the port validation tests, you just need to ensure that the
flag RTI_BUILD_UNITTESTS is set to true, so use
-DRTI\_BUILD\_UNITTESTS=true when invoking CMake.
5.8.3.3. Custom Build Environments¶
The preferred method to build Connext Micro is to use CMake. However, in some cases it may be more convenient, or even necessary, to use a custom build environment. Please refer to Custom Build Environments to learn how to import Connext Micro code.
Additionally, in order to build the port validation tests the following steps are needed:
Add compile-time definition ‘__autosar__’ (Only for AUTOSAR Systems).
Add compile-time definition ‘__freertos__’ (Only for FreeRTOS Systems).
Add the following include paths:
<RTIMEHOME>/test/include
<RTIMEHOME>/test/netio/autosar (Only for AUTOSAR Systems).
<RTIMEHOME>/test/netio/common
<RTIMEHOME>/test/netio/test
<RTIMEHOME>/test/netio/udp
<RTIMEHOME>/test/osapi/common
<RTIMEHOME>/test/osapi/test
Import all source files from the folder <RTIMEHOME>/test/test
To build the NETIO test, import all source files from the folder <RTIMEHOME>/test/netio
To build the OSPIA test, import all source files from the folder <RTIMEHOME>/test/osapi
As explained above, you need to build and run two images, one with NETIO tests and another one with OSAPI tests.
5.8.4. Running the Tests¶
5.8.4.1. Setting Up a Config File¶
Since both OSAPI and NETIO run system tests, a config file is required. A template file for the unit-test configuration can be found in:
<RTIMEHOME>/resource/test/test.cfg
The template looks like this:
property
{
netio.udp.allow_interface_multicast=1;
}
property user = "test"
{
netio.udp.allow_interface="lo";
netio.udp.allow_interface_address=0x7F000001;
netio.udp.allow_interface_netmask=0xffffff00;
netio.udp.multicast_if="lo";
osapi.system.my_hostname="my_hostname";
}
Update the fields to reflect:
Interface name, interface address and interface netmask
Multicast
5.8.4.2. Running the tests using a configuration file¶
For systems with CMake, after compiling the tests you can simply run this command:
`ctest`
For extended output, run:
`ctest -V`
You need to run this command from the CMake build directory, that would be <RTIMEHOME>/build/cmake/Debug|Release/<arch>.
Otherwise you can run the executables directly with the following commands:
./test/bin/<arch>/osapiTester(d) -id <domain id> -config "./resource/test/test.cfg" -user test
./test/bin/<arch>/netioTester(d) -id <domain id> -config "./resource/test/test.cfg" -user test
Note: The environment variable RTIME_TEST_CONFIG_ID is only used when
running the tests with ‘ctest’. When running the test executables
directly, use the parameter -id to indicate the domain ID.
5.8.4.2.1. Running the tests on platforms without a file system¶
On platforms without a file system, it is not possible to use a configuration file to run the port validation tests. In this case, the configuration can be passed as parameters to the test application, like this:
./test/bin/<arch>/osapiTester -id <domain id>
-property osapi.system.my\_hostname=<hostname>
-property netio.udp.allow\_interface=<Interface name>
-property netio.udp.allow\_interface\_address=<Interface IP address>
-property netio.udp.allow\_interface\_netmask=<Interface mask>
-property netio.udp.allow\_interface\_multicast=<1\|0>
-property netio.udp.multicast\_if=<Multicast Interface name>
./test/bin/<arch>/netioTester -id <domain id>
-property osapi.system.my\_hostname=<hostname>
-property netio.udp.allow\_interface=<Interface name>
-property netio.udp.allow\_interface\_address=<Interface IP address>
-property netio.udp.allow\_interface\_netmask=<Interface mask>
-property netio.udp.allow\_interface\_multicast=<1\|0>
-property netio.udp.multicast\_if=<Multicast Interface name>
5.8.4.3. Test Results¶
After running ‘ctest -V’, the output should be as follows:
test 1
Start 1: osapi
1: Test command: /Users/garrett/workspace/RTI/connextmicro/rti/build/release/connextmicro/2.4.14/source/unix/build/cmake/unix/lib/osapiTesterzd "-id" "67" "-config" "./resource/test/test.cfg"
1: Test timeout computed to be: 9.99988e+06
1: hostname is Foothill.local
1: property netio.udp.allow_interface_multicast already exists
1: time/interface .................................................: Passed
1: time/performance ...............................................: Passed
1: system/hostname ................................................: Passed
1: system/lua .....................................................: Passed
1: mutex/basic ....................................................: Passed
1: mutex/lua ......................................................: Passed
1: semaphore/basic ................................................: Passed
1: semaphore/thread ...............................................: Passed
1: semaphore/timeout ..............................................: Passed
1: semaphore/timeout_mt ...........................................: Passed
1: semaphore/lua ..................................................: Passed
1: thread/basic ...................................................: Passed
1: thread/advanced ................................................: Passed
1: thread/priority ................................................: Passed
1: thread/lua .....................................................: Passed
1: timer/1s .......................................................: Passed
1: timer/3s .......................................................: Passed
1: timer/MICRO-221 ................................................: Passed
1: timer/MICRO-240 ................................................: Passed
1: timer/MICRO-839 ................................................: Passed
1: timer/MICRO-1617 ...............................................: Passed
1: timer/sec_nsec .................................................: Passed
1: timer/lua ......................................................: Passed
1: process/pid_as_string ..........................................: Passed
1: process/getpid .................................................: Passed
1: osapi:TESTS COMPLETED
1/2 Test #1: osapi ............................ Passed 73.49 sec
test 2
Start 2: netio
2: Test command: /Users/garrett/workspace/RTI/connextmicro/rti/build/release/connextmicro/2.4.14/rti_me.2.0/source/unix/build/cmake/unix/lib/netioTesterzd "-id" "67" "-config" "./resource/test/unittest.cfg"
2: Test timeout computed to be: 9.99988e+06
2: hostname is Foothill.local
2: property netio.udp.allow_interface_multicast already exists
2: address/parser .................................................: Passed
2: address/resolver ...............................................: Passed
2: address/ifname .................................................: Passed
2: route/precondition .............................................: Passed
2: route/lua ......................................................: Passed
2: route/precondition .............................................: Passed
2: route/route ....................................................: Passed
2: route/default_mc_route .........................................: Passed
2: route/lua ......................................................: Passed
2: udp/route ......................................................: Passed
2: udp/iftable ....................................................: Passed
2: udp/unicast ....................................................: Passed
2: udp/multicast ..................................................: Passed
2: udp/multicast_reserve ..........................................: Passed
2: udp/nat ........................................................: Passed
2: udp/max_message_size ...........................................: Passed
2: udp/strrchr ....................................................: Passed
2: udp/lua ........................................................: Passed
2: packet/set_head_tail ...........................................: Passed
2: netio:TESTS COMPLETED
2/2 Test #2: netio ............................ Passed 80.75 sec
100% tests passed, 0 tests failed out of 2
When a test fails, the output will be as follows:
1: system/hostname ................................................: Failed (FAILURE: SystemTester.c:523 osapi.system.my_hostname not set)
2: udp/iftable ....................................................: Failed (FAILURE: UDPInterfaceTester.c:2397 netio.udp.allow_interface property not found)
If a test fails, the test execution stops and any following tests will not run. In the above example, you can see that the tests OSAPI ‘system/hostname’ and NETIO ‘udp/iftable’ failed.
5.8.4.4. Troubleshooting¶
If the tests fail on hostname and iftable such as:
1: system/hostname .....................................................: Failed (FAILURE: SystemTester.c:523 osapi.system.my_hostname not set)
2: udp/iftable .........................................................: Failed (FAILURE: UDPInterfaceTester.c:2397 netio.udp.allow_interface property not found)
then you have incorrectly declared your domain ID. Refer to Setting Up a Config File for more information.
5.8.5. Embedded Platforms¶
When developing for an embedded platform, you will commonly need to create an image with all the software: OS, BSP, middleware, user application, etc. In this situation, you must create static libraries only, instead of executables.
Two static libraries are generated, one with the OSAPI tests and another with the NETIO tests. These are osapiTesterz(d) and netioTesterz(d) (the d suffix indicates whether it is a debug library if present or a release library if not present).
You need to build two images, one using the OSAPI test library and another using the NETIO test library. We recommend building and running one release image using the release libraries and one debug image using the debug libraries.
There is a third static library, rti_me_testz(d), which contains the UTEST framework. This library is needed to build both the OSAPI and NETIO tests.
For example, to build NETIO tests, use the following libraries:
netioTesterz(d)
rti_me_testz(d)
librti_mez(d)
5.8.5.1. AUTOSAR Systems¶
Before continuing, you should become familiar with the configuration needed to run Connext Micro on an AUTOSAR system. We recommend that you review Connext Micro on AUTOSAR.
The file <RTIMEHOME>/test/include/test/test_autosar.h contains the string definitions with all the properties that are used to run the port validation tests. That is, the following definition:
#define DEVICE_ETH_IP_STR "0xc0000002"
can be used when the IP address configured on an AUTOSAR system is
192.0.0.2. In your build system, you need to define the properties that
do not match with your configuration. That is, you need to define
DEVICE_ETH_IP_STRin your build system to use a different IP
address.
You need to add compile-time definition ‘__autosar__’.
The specific configuration needed to run the port validation tests on an AUTOSAR system includes the following:
The default timer task period used to run the AUTOSAR port validation tests is 10 ms. If your timer task is configured with a different periodicity, define ‘TIMER_TASK_PERIOD_MS’ with that periodicity value, in ms.
The default IP address used by the AUTOSAR port validation tests is “0xc0000002”. If your AUTOSAR configuration uses a different IP address, define ‘DEVICE_ETH_IP_STR’ with the string representation of that IP address.
The default IP mask used by the AUTOSAR port validation tests is “0xffffff00”. If your AUTOSAR configuration uses a different IP mask, define ‘DEVICE_ETH_IP_STR’ with the string representation of that IP mask.
The AUTOSAR port validation tests need at least 140 KB of RAM to run. AUTOSAR system properties (OSAPI_SystemProperty) must be configured correctly with at least this amount of memory in the heap. You need to define variables ‘const uint32 heap_area_size[NUMBER_OF_HEAP_AREAS];’ and ’char* const heap_area[NUMBER_OF_HEAP_AREAS];’. It is also possible to define NUMBER_OF_HEAP_AREAS in your build system (default value is 2).
The AUTOSAR port validation tests use 2 semaphores. 2 timeout events, 2 give events and 2 alarms are needed. You need to define ‘RTIME_Semaphore_Give_Event’ with the ID of the first semaphore give event, ‘RTIME_Semaphore_Timeout_Event’ with the ID of the first semaphore timeout event, and ‘RTIME_Semaphore_Alarm’ with the ID of the first semaphore alarm.
The AUTOSAR port validation tests can be configured to either use resources (for single core) or spinlock (for multicore) synchronization. The default is to use resources. You need to define ‘RTIME_SYNC_TYPE’ with a different value in your build system.
If using resources synchronization: the AUTOSAR port validation tests use 26 AUTOSAR resources. You need to define ‘RTIME_Resource01’ with the ID of the first resource and ‘RTIME_Resource26’ with the ID of the last resource.
If using spinlock synchronization: you need to define ‘RTIME_Spinlock’ with the spinlock ID.
3 UDP sockets are created. The AUTOSAR configuration must allow that.
The AUTOSAR port validation tests use SocketOwner ID 1 to create sockets. If your AUTOSAR configuration uses a different SocketOwner, you need to define ‘RTIME_SOCK_OWNER_ID’ with the ID of the SocketOwner that can be used to create sockets.
The AUTOSAR port validation tests use ID 0 as the IP address identifier representing the local IP address and EthIf controller to bind the socket to. If your AUTOSAR configuration uses a different ID, you need to define ‘RTIME_LOCAL_ADDR_ID’ with the correct value.
The ‘UDP receive task’ and ‘UDP receive event’ are mandatory. Some tests use them while some others don’t. You need to define their IDs using the macros ‘NETIO_Autosar_udp_receive_task’ and ‘RTIME_UDP_Receive_Event’.
The AUTOSAR port does not provide and does not need a ‘OSAPI_Thread_sleep()’ function. But the AUTOSAR port validation tests do need that functionality. The implementation is based on an alarm and an event. OSAPI_Thread_sleep() sets an alarm with ID ‘RTIME_Sleep_Alarm’ and waits until the event with ID ‘RTIME_Sleep_Event’ is set.
An alarm must be configured to set an event when it expires. It is important that the alarm is triggered by a counter based on a 1 ms tick and when the alarm expires. You need to define in your build system the alarm ID and the event ID using ‘RTIME_Sleep_Alarm’ and ‘RTIME_Sleep_Event’.
A task with the name ‘Micro_UnitTests_Task’ must be configured in the AUTOSAR configuration. This is the task that runs the port validation tests. The task shall have at least 32 KB stack. The implementation of this task is provided by the AUTOSAR port validation tests.
Test results will be printed to the standard output used by ‘printf()’.
5.8.5.2. FreeRTOS Systems¶
Before continuing, you should become familiar with the configuration needed to run Connext Micro on a FreeRTOS system. We recommend that you review Building the Connext Micro Source for FreeRTOS.
The file <RTIMEHOME>/test/include/test/test_freertos.h contains the string definitions with all the properties that are used to run the port validation tests. That is, the following definition:
#define DEVICE_ETH_IP_STR "0xc0000002"
can be used when the IP address configured on a FreeRTOS system is
192.0.0.2. In your build system, you need to define the properties that
do not match with your configuration. That is, you need to define
DEVICE_ETH_IP_STRin your build system to use a different IP
address.
Some NETIO tests send UDP packet to the local IP address. For that reason it is necessary to set the following lwIP flag:
#define LWIP_NETIF_LOOPBACK 1
You need to add compile-time definition ‘__freertos__’.
We recommend that you create a separate thread and call the UTEST main function (UTEST_main()) for platforms without dynamic linking. This thread should have at least 32 KB of stack.
An example implementation of that thread is:
void UTEST_freertos_main(void *param)
{
/* Avoid compiler warning */
(void)param;
/* Wait until network is available */
#ifdef USE_DHCP
while(DHCP_state != DHCP_ADDRESS_ASSIGNED)
{
OSAPI_Thread_sleep(1000);
}
#else
vTaskDelay(5000 / portTICK_RATE_MS);
#endif
(void)UTEST_main();
vTaskDelete(NULL);
}
5.8.6. Porting UTEST¶
If you wrote a new Connext Micro port, you will also need to port the porting validation module. Most of the changes needed are only in the file <RTIMEHOME>/include/test/test_setting.h.
Check for a compiler flag that identifies your platform. For example, Linux would be __linux__. If your compiler does not provide such a flag, you can add a flag to your build system, i.e. my_platform.
As explained in Running the tests on platforms without a file system, you can pass the test configuration through a file or through a string. Write a new section in the file <RTIMEHOME>/include/test/test_setting.h where you configure this. After this comment at the beginning of the file:
/* If the platform has not been specified, attempt to determine it. */
Write a section like the following:
#if \_\_my_platform\_\_ #ifndef MYCOMPANY_MYPLATFORM #define MYCOMPANY_MYPLATFORM #endif /* MYCOMPANY_MYPLATFORM */ #include "test_myplatform.h" #define HAVE_CONFIG_FILE 0 #define HAVE_ARG_STRING 1 #ifdef HAVE_TEST_RESULTS_FILE #undef HAVE_TEST_RESULTS_FILE #endif #define HAVE_TEST_RESULTS_FILE 0 #endif /* \_\_my_platform\_\_ */
The file “test_myplatform.h” is optional. You can create it to add any definitions that are useful for your tests.
If your platform does not have a file system, the value of HAVE_CONFIG_FILE shall be 0 and the value of HAVE_ARG_STRING shall be 1. You also need to undef HAVE_TEST_RESULTS_FILE. If your platform has a file system, you might need to change the logic, but that is optional.
In the file <RTIMEHOME>/include/test/test_setting.h, define the maximum length of the system name, so the new platform is recognized by UTEST. For example:
#elif defined(RTI_AUTOSAR) #define UTEST_SYSTEM_NAME_MAX_LENGTH 255 #elif defined(MYCOMPANY_MYPLATFORM) #define UTEST_SYSTEM_NAME_MAX_LENGTH 255 #else #error "Unknown platform. Please port UT_System.c to this platform." #endif
The third and fourth lines are new. You can also include any platform header file in this new code.
If you have defined
HAVE_ARG_STRINGas 1, you need to provide the string that will be used as an argument. Create the file <RTIMEHOME>/include/test/test_myplatform.h with the following content:#define UTEST_ARG_STRING(argv0_) \ "-property netio.udp.allow_interface_multicast=1 " \ "-property netio.udp.allow_interface=eth0 " \ "-property netio.udp.allow_interface_address=" DEVICE_ETH_IP_STR " " \ "-property netio.udp.allow_interface_netmask=" DEVICE_MASK_STR " " \ "-property netio.udp.multicast_if=eth0 " \ "-property osapi.system.my_hostname=Myplatform-host " \ "-id 80 "
Depending on your platform, the property values might be different. For instance, if your platform doesn’t have multicast you will need to set
netio.udp.allow_interface_multicast=0.If your new platform supports dynamic linking, executable binaries with OSAPI and NETIO tests are generated when you build the port validation tests.
If your new platform does not support dynamic linking, only static libraries with OSAPI and NETIO tests are generated when you build the port validation tests. These libraries provide a UTEST_main() function. We recommend that you create a separate thread and call that UTEST_main() function from that thread. This thread should have at least 32 KB of stack.