I would not be surprised if compilers were the most complex examples of that kind of program in the wild. Still, I would be surprised if automated testing of a

I would not be surprised if compilers were the most complex examples of that kind of program in the wild. Still, I would be surprised if automated testing of a compiler was as difficult as, say, test cases kernel-space driver that simulate hardware failure. (this is one example of something I'm currently dealing with) This is probably why compilers tend to have comprehensive test suites, and why drivers (and other I/O-heavy software) usually don't. To be clear: I'm not saying writing compilers is easy; I'm saying automated testing of compilers is conceptually not that hard to imagine, so it gets done. Whereas testing I/O and other external interaction involves ridiculous levels of yak shaving as it has almost no existing tool support, so it doesn't happen. Yet, most of the software we as humans interact with directly is exactly that I/O-heavy stuff.