تولید آزمایش مبتنی بر جهش توسط یک راهبرد انتخاب مسیر

ABSTRACT Mutation based test case generation via a path selection strategy

Software testing establishes the main method for identifying errors while suggesting an acceptable level of confidence for the software under investigation. Assessing the quality of a given data set a coverage measure must be utilised. Such a measure, tries to determine the extent to which the occurrences of some software features have been successfully exercised. Different program features lead to different criteria and hence different testing requirements. Criteria usually require the execution with actual data, to exercise their characteristics such as statements, branches, decisions, paths etc. Thus, the use of coverage criteria can be also seen as a guide for selecting appropriate test cases to increase the level of the testing thoroughness. Mutation testing is a fault-based testing technique originally introduced by Hamlet [1] and DeMillo et al. [2]. Researchers have provided evidence that mutation testing forms a rather powerful testing criterion able to effectively detect more faults than most of its structural testing rivals [3,4]. Mutation testing induces syntactical alterations of the code under test with the aim of producing semantically different versions of the considered code. Each such mutated program version contains one simple syntactic change from the original source code. To assess the quality of the test cases, mutation requires the execution of the altered program versions with the goal of distinguishing them from the original one. A mutant is said to be killed if there exists such a test whereas, it is said to be equivalent if there is not. Assessing the testing quality is usually measured by a ratio of the killed over the totally introduced mutants. This constitutes a criterion usually referred to as the mutation testing criterion [5]. Generating test cases successfully, with respect to a predefined-targeted killing ratio of the introduced mutants forms the focus of this paper. The generation of test cases when performing mutation can be a very costly incident. To generate mutation adequate test cases, a tester must design test cases able to infect, execute and reveal all the introduced mutants. The term infect is used to express the case where mutant execution caused a discrepancy in the program state, at the altered program point by the introduced mutant. This can be achieved by iteratively producing and executing test cases with both the original and the mutated program versions, in order to determine the mutants killed. The determination is done by comparing the outcomes of the original and the mutated program versions. This process can be very expensive if performed in a non systematic way. Researchers usually believe that mutation expenses are considerably higher than what the other white box criteria require. This belief relies on the huge number of testing requirements that it introduces and on the observation that mutation usually needs more test cases than the other structural testing criteria [3]. To date, the effort involved when applying mutation has received scant attention in the literature. This lack of effort quantification in relation to the absence of practical solutions addressing the peculiarities of the mutation testing process have resulted in its reduced usage compared to the other structural criteria such as branches, data flow, etc. In the present paper, a path based approach for generating mutation based test cases is proposed and empirically evaluated. The experimental evaluation is performed with the aim of assessing the effort entailed by mutation testing during the test generation process. There are various ways in producing test cases. The one employed in the present study is based on selecting paths that the mutants lie on, and subsequently executing them with test data, thus utilising each one of the mutants in order to kill them. The drawbacks of such a strategy, as it has been reported in the literature, are attributed to the presence of infeasible paths [6,7]. The strategy adopted in this exercise, seeks to reduce the undesirable effects of infeasible paths during the testing process and hence reciprocally, also reduce the overall testing effort. Therefore, it can be argued that there is a direct relationship between the number of generated test paths and the effort involved for performing mutation. In fact, and without any loss of generality, the experiment assesses the relative cost-effective application of weak mutation [8,5] testing based on a proposed path selection method able to eliminate the undesirable effects of infeasible paths. The paper is organised as follows: In Section 2 the relevant to the conducted experiment notation and terminology is given along with some related to the area work. In Section 3 a path selection strategy which seeks to reduce the incidence of infeasible paths is proposed. In order to evaluate the strategy and to interpret its achievements, certain features of the prototype implementation are analysed and presented in Section 4. Additionally, in Section 5 the results of the application of the proposed strategy to derive mutation test cases to a set of program units, are presented. The important findings of the experimental study along with the nature of infeasible paths, together with certain characteristics of the proposed strategy, can justify pertinent comments about the effort associated with weak mutation testing in general. These are presented in Section 6. Finally, the conclusions are discussed in Section 7.