On How Characteristics that Hinder Test Data Generation Using Symbolic Execution Combine: An Analysis of the SF100 Benchmark

Abstract

Symbolic execution has made great strides in the last two decades. However, some characteristics of programs still pose significant challenges to test data generation techniques that use symbolic execution and constraint solving. These challenges present themselves in the form of path explosion, constraint complexity, dependency, and exception-dependent paths. The few studies conducted to date to examine the distribution of such characteristics only investigate them separately. We posit that understanding how the combination of these characteristics appear in real world software may provide a more in-depth knowledge of the limitations of current tools and test data generation techniques. Thus, we analyzed a benchmark comprising 100 open source Java programs to get some insight into the frequency of methods that have combinations of the studied characteristics and the distribution of methods whose path constraints share similar structures.