Improving the Accuracy of Oracle Verdicts Through Automated Model Steering

Gay, Gregory
Rayadurgam, Sanjai
Heimdahl, Mats
2014

View/Download File

14ase.pdf (304.82 KB)

Persistent link to this item

https://hdl.handle.net/11299/217442
Statistics
View Statistics

Journal Title

Journal ISSN

Volume Title

Title

Improving the Accuracy of Oracle Verdicts Through Automated Model Steering

Authors

Published Date

2014

Publisher

Type

Report

Abstract

The oracle--a judge of the correctness of the system under test (SUT)--is a major component of the testing process. Specifying test oracles is challenging for some domains, such as real-time embedded systems, where small changes in timing or sensory input may cause large behavioral differences. Models of such systems, often built for analysis and simulation, are appealing for reuse as oracles. These models, however, typically represent an idealized system, abstracting away certain issues such as non-deterministic timing behavior and sensor noise. Thus, even with the same inputs, the model's behavior may fail to match an acceptable behavior of the SUT, leading to many false positives reported by the oracle. We propose an automated steering framework that can adjust the behavior of the model to better match the behavior of the SUT to reduce the rate of false positives. This model steering is limited by a set of constraints (defining acceptable differences in behavior) and is based on a search process attempting to minimize a dissimilarity metric. This framework allows non-deterministic, but bounded, behavior differences, while preventing future mismatches, by guiding the oracle--within limits--to match the execution of the SUT. Results show that steering significantly increases SUT-oracle conformance with minimal masking of real faults and, thus, has significant potential for reducing false positives and, consequently, development costs.

Keywords

Description

Associated research group: Critical Systems Research Group

Related to

Replaces

License

Collections

University of Minnesota Software Engineering Center (UMSEC) Publications

Series/Report Number

Funding information

Isbn identifier

Doi identifier

Previously Published Citation

29th ACM/IEEE International Conference on Automated Software Engineering (ASE'14). Vasteras, Sweden, September 2014.

Other identifiers

Suggested citation

Gay, Gregory; Rayadurgam, Sanjai; Heimdahl, Mats. (2014). Improving the Accuracy of Oracle Verdicts Through Automated Model Steering. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/217442.

Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.