2 ms·

>Or are bugs due to interpretations of the specification just easier to find? It could be. Another possible explanation could be that I am biased due to mostly

• pfdietz 5y ago
  That customers report front end bugs could also reflect how easy they are to trigger. I admit this could mean they are more important, even if they aren't more numerous. The relative frequency of different kinds of bugs can also change over time. Testing has a diminishing returns behavior. If front end bugs are easy to trigger, they will get expunged, and you'll be left with the harder to trigger bugs elsewhere in the compiler. I saw this in writing conformance tests for Common Lisp implementations: after implementations got the little direct tests right (function FOO implements its specified behavior), the remaining defects tended to be deeper and more subtle. Eventually random testing was churning out bizarre compiler interactions (but even that eventually burned through the reachable bugs.) It would be interesting to study various compiler testing approaches using mutation: introduce bugs to the compiler by mutating its code, then see what fraction the testing can detect. Fully mutating a compiler is likely infeasible, but one could create a random sample of mutations and make estimates.
  • moonchild 5y ago
    It's the pareto principle, I think: 80% of code that's out there will exercise 20% of the compiler's codepaths. It gets increasingly difficult to find meaningful code that tests those paths.