FISITA World Congress 2021: A Practical Guide to Fuzz Testing Embedded Software in a CI Pipeline

Software, including open-source software, is transforming the automotive industry. The Zephyr Project RTOS aims to be the first open-source real time operating system to achieve functional safety certifications making it applicable for use in automotive embedded systems. In terms of connectivity, Zephyr supports among others Bluetooth, Wi-Fi, IP, Ethernet and CAN. With the recent developments of cybersecurity standards and regulations such as “ISO/SAE 21434 Cybersecurity Engineering” and “UN Regulation 155 Cybersecurity”, it becomes paramount for automotive organizations to consider employing various cybersecurity activities in the development process. For example, to detect unknown vulnerabilities in automotive software it is recommended to perform various types of testing such as fuzz testing or penetration testing. Fuzz testing is a powerful test approach since in contrast to penetration testing it can be automated, and it can be used to effectively detect unknown vulnerabilities in the target software. However, the fuzz testing activity is often performed in a manual manner in the automotive industry today. In this paper, we present a practical guide to building fuzz testing into a CI (continuous integration) pipeline. Using the Zephyr project as an example, we describe the various steps to build a fuzz testing process. These steps include identifying the target communication protocols to fuzz, defining a test strategy of when, what and how long to fuzz, executing fuzz testing on a continuous basis in an automated fashion, detecting exceptions on the target system, and managing the test results. These practical steps are described in detail to help guide automotive organizations to build fuzz testing into the CI pipeline for their own target systems. Building fuzz testing into the CI pipeline enables automotive organizations to perform fuzz testing on a continuous basis and in an automated fashion which reduces the manual effort required. As a result, it is possible for automotive organizations to detect and fix unknown vulnerabilities early in the development process which reduces the involved costs and overall improves the product quality. We further discuss the benefits and provide additional insights on future needs in more detail in the paper.