3 ms·
The problem with relying solely on "real world makefiles" is that there is not a lot of variation amongst those that are publicly accessible. The vast majority
by emelski 17y ago
The problem with relying solely on "real world makefiles" is that there is not a lot of variation amongst those that are publicly accessible. The vast majority of open source projects that use make are autoconf based, which means the makefiles are all more-or-less the same, except that the list of source files and the project name change. Once you can successfully build one of these, you can probably build all of them.
There are a handful projects that don't follow this pattern, such as Mozilla, but once you've built those they cease to be useful for flushing out new bugs.
So, what do you do once you've built all the open source projects you can find, and you are still finding incompatibilities in your parser when you go to customers? Markov chain-based makefiles are a reasonable way to extend the breadth of our testing. Obviously it's not the only thing we do, but it is another valuable tool to add to the toolbox.
- kurtosis 17y agoOkay I understand where you're coming from. I'm really curious though, what were some of the bugs that showed up? Were they easy to fix or did they require adding a lot of special cases and patchwork to the code?
- emelski 17y agoThere were about a dozen issues uncovered using this technique. I would say the majority of them were what I would call "real" bugs, as opposed to "sure, that doesn't work but nobody cares". One example is that gmake allows $$ in variable names, as in "FOO$$BAR=abc"; at the time our parser did not handle this correctly. I did not have to create special cases to fix these bugs, at least not any more so than any other parser feature.