DomainSweep: Input Domain Driven Falsification of Cyber-Physical Systems

Abstract

Modern Cyber-Physical Systems (CPS), tasked with complex control functions like autonomous driving, increasingly incorporate AI-enabled controllers based on deep neural networks (DNN). This growth necessitates robust safety measures and reliable protocols that ensure these systems function correctly. However, as these systems grow in complexity and scale, traditional verification methods become limited. Instead of attempting to prove that a system's properties are always correct, an alternative approach is to focus on identifying inputs that cause the system to violate a specified property. This technique is known as Falsification. Typically, falsification employs a black-box strategy, using search-based testing and heuristics to diminish some robustness metric of a system's property. Although these methods are flexible and reusable across various systems, they often do not yield optimal results due to their reliance on general heuristics. Contrariwise, white-box falsification methods offer precision but are limited because they are tailored to specific systems or architectures, shrinking their wider applicability. To address these challenges, we introduce DomainSweep, a novel black-box falsification tool that leverages input domain models to effectively falsify CPS properties. Utilizing a Variational Autoencoder (VAE), DomainSweep adopts a unique approach by exploring low-dimensional embedding of the inputs to guide the falsification process. Experimental evaluations with various encoding schemas and coverage strategies demonstrated that DomainSweep achieved a falsification success rate of 60.18\%, significantly outperforming Breach, a well-known black-box tool, and delivering competitive results compared to FalsifAI, a state-of-the-art white-box framework. This demonstrates DomainSweep's robust capability in system falsification and establishes a strong foundation for future work in this field.