Debugging Program Executions with Graphical Models

When executing a program, its trace through the software (i.e. the executed code lines) can be recorded. Given traces of passing and failing test cases, the goal of this work is to predict the code line that is defective.

Rhino Data Set

Rhino is an open source java script engine. For our paper we got several faulty revisions from the Rhino source code repository and recorded the trace of failing and passing test cases. You can download our rhino data set here [1.5 GB zipped].

The zip file contains one directory per defecti revision (i.e. 210682). The directory contains the following files:

passing_*.ser: Traces of passing test cases (one file per test case)
failing_*.ser: Traces of failing test cases.
fixinfo.ser: The code lines that were defective (ground truth) and were modified for the follow up revision.

The contents of each file follows the pattern

<classname>"."<methodname>"("<signature":" <linenumber1>","<linenumber2>","<linenumber3>","...

Example:

org/mozilla/javascript/ObjToIntMap.get(Ljava/lang/Object;I)I: 152,153,154,155
org/mozilla/javascript/optimizer/Codegen.addByteCode(B)V: 3529,3530
org/mozilla/javascript/ObjToIntMap.findIndex(Ljava/lang/Object;)I: 258,259,260,261,262,263,292
org/mozilla/javascript/ObjToIntMap.findIndex(Ljava/lang/Object;)I: 258,259,260,261,262,263,264,265,268

If methods are multiple times, this is reflected in multiple lines with the same method. Note that different calls may result in different traces (compare third and forth example). Note that in java, the same method name may have different signatures. In our experiment setup we refered to those as different methods.

Note that the structure of the call tree ("which method called which method?") is not preserved in this data set.

AspectJ Data Set

AspectJ is a compiler for aspect oriented programming in Java. For our paper we got several faulty revisions from the AspectJ source code repository and recorded the trace of failing and passing test cases. For further information on how to obtain this dataset see [Dallmeier07].

Since the data set is too large to be hosted on our webserver (3.4 GB tar.gz), we provide only a few traces (118781, 30168, 61536, 72531). The full data set is obtainable on request.

The zip file contains one directory per defective revision (i.e. 87376). The directory contains the following files:

bug87376_validpresent.bin: Traces of passing test cases.
bug87376_predict.bin: Traces of failing test cases.

Each of these files are in binary format you can read a sequence of four-bytes-integers from. Each file contains several traces, where the first integer specifies the number of tuples n, and the next integer following the tuples specifies the number of tuples for the next trace.

The format is specified as follows:

(<n> (<c> <s> <e> <m>)^n)*

n is the number of 4-tuples for the current trace
c is the class id
s is the code line id (uniqe to the method)
e is the error code (1 for a defective code line, 0 otherwise)
m is the method id (globally unique)

This format preserves information about the call tree.

Cite As

Dietz, L., Dallmeier, V., Zeller, A., Scheffer, T., 2009. Localizing bugs in program executions with graphical model. In: Advances in Neural Information Processing Systems.
[pdf] [supplement]

@inproceedings{Dietz-Nips09,
    title = {Localizing Bugs in Program Executions with Graphical Model},
    author = {Dietz, Laura and Dallmeier, Valentin and Zeller, Andreas and Scheffer, Tobias},
    booktitle = {Advances in Neural Information Processing Systems},
    year = {2009}
}

References

[Dallmeier07] Dallmeier, V., Zimmermann, T., 2007. Extraction of bug localization benchmarks from history. In: ASE '07: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering. ACM, New York, NY, USA, pp. 433-436.