Title: An iterative metamorphic testing technique for web services and case studies

Authors: Chang-ai Sun; An Fu; Yiqiang Liu; Qing Wen; Zuoyi Wang; Peng Wu; Tsong Yueh Chen

Addresses: School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China; State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, 100190, China ' School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China ' School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China ' School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China ' School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China ' State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, 100190, China; University of Chinese Academy of Sciences, Beijing, 100049, China ' Department of Computer Science and Software Engineering, Swinburne University of Technology, Melbourne, Australia

Abstract: Metamorphic testing (MT) is an innovative approach to alleviating the oracle problem in software testing, which uses metamorphic relations of the program under test, instead of the test oracles, to verify its outputs. To alleviate the oracle problem of testing web services, we had previously proposed an MT framework for web services. In this paper, we further improve the efficiency and automation of this framework by leveraging metamorphic relations to iteratively generate test cases. We present a fixed-size iterative MT algorithm and implement it in the MT framework. We conduct three case studies to evaluate the fault detection effectiveness and efficiency of the proposed approach. Experimental results suggest that, compared with the conventional MT, iterative MT can achieve a comparable fault detection effectiveness, but with significantly fewer resources. Observations and limitations are summarised to provide new insights into the application of iterative MT.

Keywords: metamorphic testing; metamorphic relations; test case generation; web services.

DOI: 10.1504/IJWGS.2020.110945

International Journal of Web and Grid Services, 2020 Vol.16 No.4, pp.364 - 392

Received: 07 Aug 2019
Accepted: 22 Apr 2020
Published online: 02 Nov 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article