Practical Aspects of Building a Constrained Random Test Framework for Safety-critical Embedded Systems

Abstract

In the safety-critical embedded system industry, one of the key challenges is to demonstrate the robustness and dependability of the product prior to market release, which is typically done using various verification and validation (V&V) strategies. Directed verification testing is a common strategy that performs black-box testing at the system level; however, it only samples a small set of specific system behaviors and requires heavily manual effort. In this paper, we describe our experience and lessons learned of applying the concept of constrained random testing on safety-critical embedded systems as a complimentary testing methodology. Constrained random testing enables us to cover many more system behaviors through random input variations, random fault injections, and automatic output comparisons. Additionally, it can reduce manual effort and increase confidence on the dependability of both firmware and hardware.