4 ms·

Recursive Make Considered Harmful

18 points by thealphanerd 13y ago

Nursie 13y ago
The last project I worked on used recursive make. I did a lot of work on that system to get the damn thing to build properly. However at the end it did work, and just as well as an include-based make we've used on previous projects. He mentions in the paper that dependency generation using gcc -MM, well you can do the same in your recursive system with roughly the same amount of pain. And (I do hate to say this) some of the other concerns are really rather inconsequential with the speed of modern computers. It's a very good paper and it provides good insight into the problem space, but I'm not sure it's an absolute. See also '"Considered Harmful" Essays considered harmful' - http://meyerweb.com/eric/comment/chech.html http://meyerweb.com/eric/comment/chech.html On balance I think include-based make is probably a better way of doing things, and you can still have your module level files, they just become included rather than executed. Of course what all this fussing about with dependency scripts really shows is that what most people try to do with make could be automated by other systems.
RyanZAG 13y ago
Surely maintaining a single build file for enormous code projects worked on by many different teams would cause more harm than good? The paper writes this issue off as 'same stuff in one or many files, same complexity', but obviously this isn't the case when multiple teams are involved who don't understand the work or requirements of other teams. The paper feels far too theoretical and is likely why nobody has taken much note and recursive makefiles are still the most used form for very large projects 10+ years after the paper.
- Nursie 13y ago
  You can end up with all the build logic in a central place but with the module level makefiles basically just being a list of files and targets. The only real difference between the proposed solution and a 'normal' recursive make is really that these are written as 'include' rather than full makefiles, and dependencies better tracked. You don't end up with a single file until make parses everything. >> obviously this isn't the case when multiple teams are involved who don't understand the work or requirements of other teams. Massive projects with multiple teams probably won't want to work like this. He's mostly looking at a single product level, most likely worked on by a single team, he's concerned about speeding up their build times. AFAICT.
snarfy 13y ago
Nobody wants to write Makefiles and recursive Makefiles are easier to write. If it's truly the same information in multiple recursive Makefiles as it would be in one monolithic Makefile, then there should be a tool that converts recursive makefiles into monolithic makefiles. Humans should not have to reason about the entire project scope at once.
- noselasd 13y ago
  There are many ways where you only need to care about the local scope, you don't need to use a single monolithic makefile. See e.g. https://github.com/dmoulding/boilermake https://github.com/dmoulding/boilermake It is harder than it should be though.
kephra 13y ago
I would call this nested Makefiles. Recursive Makefile is more what I'm doing with my MakeHereCMS, where every Makefile is a symlink to the same top level Makefile.
- noselasd 13y ago
  recursive make[files] is a well established term for a makefile calling make on separate makefiles in subdirectories.
shadowmint 13y ago
I honestly consider any project that is complex enough to require more than a single trivial makefile (eg. lua, say) to be unsuitable for using make at all. Everything else I've seen that tries is exactly as bad as this paper suggests. This 'modern makefiles' stuff from the android NDK is one of the worst offenders; those makefiles are almost without fail difficult to debug and problematic (just search SO for 'ndk', 'makefile' if you don't believe me), but because it's android, I've actually had people advocate we use that approach for other projects too. Craziness.
- Nursie 13y ago
  What alternatives do you use? I've tried a few but I find most of them to be annoyingly opaque.

18 points by thealphanerd 13y ago

The last project I worked on used recursive make. I did a lot of work on that system to get the damn thing to build properly. However at the end it did work, and just as well as an include-based make we've used on previous projects. He mentions in the paper that dependency generation using gcc -MM, well you can do the same in your recursive system with roughly the same amount of pain. And (I do hate to say this) some of the other concerns are really rather inconsequential with the speed of modern computers. It's a very good paper and it provides good insight into the problem space, but I'm not sure it's an absolute. See also '"Considered Harmful" Essays considered harmful' - http://meyerweb.com/eric/comment/chech.html http://meyerweb.com/eric/comment/chech.html On balance I think include-based make is probably a better way of doing things, and you can still have your module level files, they just become included rather than executed. Of course what all this fussing about with dependency scripts really shows is that what most people try to do with make could be automated by other systems.

RyanZAG 13y ago

Surely maintaining a single build file for enormous code projects worked on by many different teams would cause more harm than good? The paper writes this issue off as 'same stuff in one or many files, same complexity', but obviously this isn't the case when multiple teams are involved who don't understand the work or requirements of other teams. The paper feels far too theoretical and is likely why nobody has taken much note and recursive makefiles are still the most used form for very large projects 10+ years after the paper.

Nursie 13y ago
You can end up with all the build logic in a central place but with the module level makefiles basically just being a list of files and targets. The only real difference between the proposed solution and a 'normal' recursive make is really that these are written as 'include' rather than full makefiles, and dependencies better tracked. You don't end up with a single file until make parses everything. >> obviously this isn't the case when multiple teams are involved who don't understand the work or requirements of other teams. Massive projects with multiple teams probably won't want to work like this. He's mostly looking at a single product level, most likely worked on by a single team, he's concerned about speeding up their build times. AFAICT.

snarfy 13y ago

Nobody wants to write Makefiles and recursive Makefiles are easier to write. If it's truly the same information in multiple recursive Makefiles as it would be in one monolithic Makefile, then there should be a tool that converts recursive makefiles into monolithic makefiles. Humans should not have to reason about the entire project scope at once.

noselasd 13y ago
There are many ways where you only need to care about the local scope, you don't need to use a single monolithic makefile. See e.g. https://github.com/dmoulding/boilermake https://github.com/dmoulding/boilermake It is harder than it should be though.

kephra 13y ago

I would call this nested Makefiles. Recursive Makefile is more what I'm doing with my MakeHereCMS, where every Makefile is a symlink to the same top level Makefile.

noselasd 13y ago
recursive make[files] is a well established term for a makefile calling make on separate makefiles in subdirectories.

shadowmint 13y ago

I honestly consider any project that is complex enough to require more than a single trivial makefile (eg. lua, say) to be unsuitable for using make at all. Everything else I've seen that tries is exactly as bad as this paper suggests. This 'modern makefiles' stuff from the android NDK is one of the worst offenders; those makefiles are almost without fail difficult to debug and problematic (just search SO for 'ndk', 'makefile' if you don't believe me), but because it's android, I've actually had people advocate we use that approach for other projects too. Craziness.

Nursie 13y ago
What alternatives do you use? I've tried a few but I find most of them to be annoyingly opaque.