Automating Test Oracle Generation in DevOps for Industrial Elevators

Abstract

Orona is a world-renowned elevators developer. During elevators' lives, their software continues to evolve, e.g., due to hardware obsolescence, requirements changes, vulnerabilities, and bug corrections. Such continuous evolution demands the continuous testing of industrial elevators with the minimum manual effort possible. To this end, we present a tool, whose core component is a domain-specific language (DSL) with which a user can specify test oracles at a higher level of abstraction and independent of a testing level. The DSL also supports specifying uncertainty-aware test oracles to test elevators under various uncertainties inherent in them. Finally, the DSL is also equipped with test oracle generation that generates test oracle code automatically at the different DevOps testing levels (i.e., Software and Hardware-in-the-Loop test levels, and in operation) to enable reuse of test oracles across these levels. We evaluated this DSL with an industrial elevators case study at Orona's site to specify and generate test oracles. The evaluation showed that the high expressiveness of the DSL permits the high-level definition of test oracles in our industrial context. Based on the industrial application, we discuss our experiences and lessons learned.