3 ms·

I've designed and shipped more than one multi-million line of code product that had parts in C, C++, Fortran, Python, Tcl, and even some assembly, plus the usua

by jgeada 3y ago

I've designed and shipped more than one multi-million line of code product that had parts in C, C++, Fortran, Python, Tcl, and even some assembly, plus the usual grab bag of open source dependencies. For none of those was anything more sophisticated than (admittedly fairly complex) hierarchical Makefile description to make it come together. I'll be the first to admit that I've not seen it at all, but color me suspicious that it is an inherent problem of builds of a multi-language product, rather than unknown/unnecessary complexity in the architecture of the system being built.

gavinhoward 3y ago
Recursive Make isn't the best. https://aegis.sourceforge.net/auug97.pdf https://aegis.sourceforge.net/auug97.pdf That said, if Rig gives you nothing new, please don't even try it! I'd rather you wait until you need it to try it; then you'll be more forgiving of the bugs. :)
- jgeada 3y ago
  There is something very fishy in that article: the order of traversal in a hierarchical make should be entirely expressed by dependency rules, not mere existence of a directory. Failure to describe dependencies is a flaw in the creation of the Makefile, not an inherent property of make itself. Specifically, this idiom: all: for dir in $(MODULES); do \ (cd $$dir; ${MAKE} all); \ done should be banished from everyone's Makefiles. make isn't a just way of cobbling together some collection arbitrary collection of scripts. Each of those modules should be a target with dependencies. The rule to make that module is the recursive make. But the order is determined by the explicit dependencies, not whatever random order glob gives you. ie, what you should have is: module1: (cd $module1_dir; make) module2: module1 (cd $module2_dir; make) etc etc etc Every single time in my career that I've had someone complain about a flaw in make, what I discover is a significant flaw in their description of the make rules. If it is not behaving correctly, you did not fully describe the DAG.
  - jcranmer 3y ago
    > Every single time in my career that I've had someone complain about a flaw in make, what I discover is a significant flaw in their description of the make rules. If it is not behaving correctly, you did not fully describe the DAG. You can look at the dependency graph in two ways. The first is, in order to build X, you first need to build Y and Z. These kinds of dependencies tend to be relatively easy to figure out. The second way is to ask, if Y changes, what needs to be rebuilt--and these kind of rebuild dependencies aren't going to be correct unless they're autogenerated, and getting autogenerated dependencies done correctly is challenging. (This basically reaches the point in the article that any system that doesn't support dynamic dependencies ends up hacking its way to support for dynamic dependencies). The other key point I will make is that, while it can be possible to describe the DAG you want with make, it tends to fight you every step in the way. If almost everyone is complaining about how hard it is to get the DAG correct in make, then maybe the problem isn't that everyone can't figure out how to fully describe the DAG, it's that make isn't a good option for specifying the DAG.
    - jason_s 3y ago
      file modtime-based dependencies is enough to kill any interest in make I might have had. When I update my working copy from version control to an earlier version to look at how things used to work, but the modtime is later, make does the wrong thing.

by jgeada 3y ago

gavinhoward 3y ago

Recursive Make isn't the best. https://aegis.sourceforge.net/auug97.pdf https://aegis.sourceforge.net/auug97.pdf That said, if Rig gives you nothing new, please don't even try it! I'd rather you wait until you need it to try it; then you'll be more forgiving of the bugs. :)

jgeada 3y ago
There is something very fishy in that article: the order of traversal in a hierarchical make should be entirely expressed by dependency rules, not mere existence of a directory. Failure to describe dependencies is a flaw in the creation of the Makefile, not an inherent property of make itself. Specifically, this idiom: all: for dir in $(MODULES); do \ (cd $$dir; ${MAKE} all); \ done should be banished from everyone's Makefiles. make isn't a just way of cobbling together some collection arbitrary collection of scripts. Each of those modules should be a target with dependencies. The rule to make that module is the recursive make. But the order is determined by the explicit dependencies, not whatever random order glob gives you. ie, what you should have is: module1: (cd $module1_dir; make) module2: module1 (cd $module2_dir; make) etc etc etc Every single time in my career that I've had someone complain about a flaw in make, what I discover is a significant flaw in their description of the make rules. If it is not behaving correctly, you did not fully describe the DAG.
- jcranmer 3y ago
  > Every single time in my career that I've had someone complain about a flaw in make, what I discover is a significant flaw in their description of the make rules. If it is not behaving correctly, you did not fully describe the DAG. You can look at the dependency graph in two ways. The first is, in order to build X, you first need to build Y and Z. These kinds of dependencies tend to be relatively easy to figure out. The second way is to ask, if Y changes, what needs to be rebuilt--and these kind of rebuild dependencies aren't going to be correct unless they're autogenerated, and getting autogenerated dependencies done correctly is challenging. (This basically reaches the point in the article that any system that doesn't support dynamic dependencies ends up hacking its way to support for dynamic dependencies). The other key point I will make is that, while it can be possible to describe the DAG you want with make, it tends to fight you every step in the way. If almost everyone is complaining about how hard it is to get the DAG correct in make, then maybe the problem isn't that everyone can't figure out how to fully describe the DAG, it's that make isn't a good option for specifying the DAG.
  - jason_s 3y ago
    file modtime-based dependencies is enough to kill any interest in make I might have had. When I update my working copy from version control to an earlier version to look at how things used to work, but the modtime is later, make does the wrong thing.