Realism is something we're concerned with! Unfortunately there isn't a well-defined way to say how realistic a synthetic bug is, so we had to come up with our o

Realism is something we're concerned with! Unfortunately there isn't a well-defined way to say how realistic a synthetic bug is, so we had to come up with our own. The ultimate test of whether the bugs are useful is whether they can be used to improve bug detectors, and that remains to be seen. We attempted in the paper to make an argument based, effectively, on whether the modified program looks like a normal program from the point of view of a bug-finding tool. From that perspective, the control flow in the program is unmodified, and the data flow is exactly the same as in the original program except for a small portion where data from the input file flows through a global variable to the crash site. We show that this section of data flow is usually short (in terms of number of instructions in a dynamic trace); we're also working now on extending it so that the data flow is more "natural" (i.e., it goes through return values and function parameters, rather than through global data). Additionally, we showed that the bugs are distributed throughout the execution lifetime of the programs we tested. I definitely agree that they do not look like bugs humans most often write. However, to be effective, we just need them to look enough like the kind of bugs we want to catch that bug-finding software will have to reason about the program in the same way.