Skip to content

Testing

Note

For software prerequistes required to build and run the tests, please see Prerequisites.md.

Integration tests

Just like examples, integration tests of the Open IoT SDK are generated from templates:

cmake -S templates -B tmp

This generates both examples (inside __codegen/examples/) and integration tests (inside __codegen/tests/). The only difference is that an example intends to show the use of a component in a simple and readable way, whereas an integration test exercises a component more rigorously but may not be a simple teaching example for users who are not already familiar with the component.

To build an integration test, for example the TF-M test for the Corstone-300 platform using the GNU Arm Embedded Toolchain:

cmake -S __codegen/tests/tf-m/corstone-300/ -B __build -GNinja --toolchain toolchains/toolchain-arm-none-eabi-gcc.cmake -D FETCHCONTENT_SOURCE_DIR_OPEN-IOT-SDK=.
cmake --build __build

Tip

The snippet above selects GNU Arm Embedded Toolchain. You can pass --toolchain toolchains/toolchain-armclang.cmake to use Arm Compiler for Embedded instead.

Also, you can build a different test instead of __codegen/tests/tf-m/corstone-300/.

After building a test, you can run the test using the following command:

ctest --test-dir __build --verbose

Each test is run using htrun which loads the test image onto the platform, starts the test application and checks if the test produces expected outputs. CTest, which is part of the CMake tools, invokes the htrun command with arguments specified by the add_test() call in each application's CMakeLists.txt.

Note

Just like integration tests, running of each example can also be automated with CTest using the same command.

Writing integration tests with pyedmgr and Pigweed RPC

We recommend writing new integration tests in Python using pytest with the libraries pyedmgr and Pigweed RPC. These libraries provide device lifecycle management and typed communication with test devices. See the tutorial in integration tests for details.

Contributing an integration test

To contribute an integration test to the SDK, you need to add it as a template. See the documentation for templates for details.

Unit tests

Additional source code has been added for Open IoT SDK components that required a shim layer. Shim layers are provided with a suit of unit tests that runs during CI checks. The unit test suite can also be run on developer's machines.

Steps:

  1. Configure unit test build with CMake:

    cmake -B __unit_test_build -GNinja -DENABLE_DEVELOPER_MODE=ON -DOPT_ENABLE_DOXYGEN=OFF -DOPT_ENABLE_SANITIZER_LEAK=OFF -DOPT_ENABLE_CLANG_TIDY=OFF -DOPT_ENABLE_COVERAGE=OFF

  2. Build and run unit tests:

    cmake --build __unit_test_build
ctest --test-dir __unit_test_build --verbose

    This will output the result of the tests and give you a percentage of tests passed. For any tests that fail, an output will be given to assist with debugging.

Contributing a unit test

To contribute a unit test to the Open IoT SDK, you need to put the test in a subdirectory named tests of the component being tested.

Example

The Open IoT SDK's components/mbedtls/tests/ contains a unit test for the Mbed TLS threading shim layer based on CMSIS-RTOS v2.

Note that its parent directory, components/mbedtls/CMakeLists.txt, only does add_subdirectory(tests) if the condition (BUILD_TESTING AND NOT CMAKE_CROSSCOMPILING) is satisfied, because the unit tests only work when tests are enabled (BUILD_TESTING) for the host machine (NOT CMAKE_CROSSCOMPILING).