Share
Save the reference
Title
On the Cost-Effectiveness of Composite Metamorphic Relations for Testing Deep Learning SystemsAuthor
Other institutions
https://katalogoa.mondragon.edu/janium-bin/janium_login_opac.pl?find&ficha_no=167748Version
Postprint
Rights
© 2022 ACMAccess
Embargoed accessPublisher’s version
https://doi.org/10.1145/3524846.3527335Published at
2022 IEEE/ACM 7th International Workshop on Metamorphic Testing (MET) 09 May. Pp. 42-47. IEEE, 2022Publisher
IEEEKeywords
Deep learning
Learning systems
Costs
Automation ... [+]
Learning systems
Costs
Automation ... [+]
Deep learning
Learning systems
Costs
Automation
Conferences
Autonomous vehicles
testing [-]
Learning systems
Costs
Automation
Conferences
Autonomous vehicles
testing [-]
Abstract
Deep Learning (DL) components are increasing their presence in mission and safety-critical systems, such as autonomous vehicles. The verification process of such systems needs to be rigorous, for whic ... [+]
Deep Learning (DL) components are increasing their presence in mission and safety-critical systems, such as autonomous vehicles. The verification process of such systems needs to be rigorous, for which automated solutions are paramount. To allow test automation, test oracles are necessary. In the context of DL systems, meta-morphic test oracles have found to be effective. However, such oracles require the execution of multiple tests, which makes testing more expensive. Metamorphic relation composition can reduce the cost of metamorphic testing. However, its effectiveness has found mixed answers. This paper reports the preliminary results of our study on measuring the cost-effectiveness of composite metamor-phic relations for testing DL systems. To this end, we empirically evaluate the cost-effectiveness of composite metamorphic relations within a DL model for object classification. Our results suggest that composite metamorphic relations reduce the failure revealing capability when compared to their component metamorphic relations. [-]
