4 ms·

The general point the author is making sounds true but I am sceptical regarding any criticism against make. Building and/or handling dependencies was basically

by cjfd 6y ago

The general point the author is making sounds true but I am sceptical regarding any criticism against make. Building and/or handling dependencies was basically a solved problem as soon as make was invented and all of the new stuff in this area just seems plainly unneeded to me. Also, when people invent new build systems one can end up with projects where one part is built using one build system and another part is using another. Since things can depend on each other in arbitrarily complex ways because of code generation this will lead to build either too much or too little. In particular having a build system for a particular language is such a strange concept.

jnxx 6y ago
Make works pretty well, that's why it is used. But there are a few points it does not cover: - Recursive building is complicated. it is not really easy to compose a small unit into a larger project, so that one just needs to copy the sub-unit into a folder. - in some instances, it is necessary to type extra instances of "make clean", because the dependency analysis of make does not understand that dependencies have changed. - say you build a program which includes /usr/include/ncurses.h. Then, you do a system update which replaces your ncurses library. Or, you do a git pull which changes that header. Make will not rebuild your binary without "make clean". - files appearing or disappearing along the include path can change the result of a "make clean;make all", but are not detected by make's dependecy analysis. Here an explanation of D.J. Bernstein: http://cr.yp.to/redo/honest-nonfile.html http://cr.yp.to/redo/honest-nonfile.html - doing atomically rebuilds: http://cr.yp.to/redo/atomic.html http://cr.yp.to/redo/atomic.html, so that a new dependency is either there, and complete, or not there - parallel builds (which is related to the previous point, and also to the necessity to have complete dependency information).
- boomlinde 6y ago
  > in some instances, it is necessary to type extra instances of "make clean", because the dependency analysis of make does not understand that dependencies have changed. The only sense in which Make will analyze a dependency is that it will run rules for which the target does not exist or has a less recent mtime than any of its dependencies. When what you describe happens, it's not a failure of Make's dependency analysis, but your failure to specify the dependency. This is easier said than done with the Make+CC combo, but I attribute that problem to quirks of C and CC, not to Make's dead simple and basically infallible dependency analysis. The only way to cause Make's dependency analysis to "fail" is to modify the mtime of an already built target to be more recent than that of a dependency that would otherwise be more recent. That's only a failure in the sense that it might not desirable, it's still well defined behavior of Make. > say you build a program which includes /usr/include/ncurses.h. Then, you do a system update which replaces your ncurses library. Or, you do a git pull which changes that header. Make will not rebuild your binary without "make clean". If this happens, it's because you have a dependency on /usr/include/ncurses.h that you have not specified for the target. Again, easier said than done to specify the location of system wide header files that are maybe only resolved using pkg-config because you have no idea of their location on the users' systems, but that's not a problem that Make supposes to solve for C and the ecosystem of applications that exist to patch up its deficiencies. Even worse is for example if you've specified your dependency on the ncurses header, update the system and end up with a new ncurses header that has new includes of its own that you have not specified. The only way you'll ever know is by reading ncurses header file. If you don't, the next system upgrade might update those unspecified indirect dependencies and your Makefile will shrug because you haven't specified those as dependencies. Broken, but really not on the Make end. > files appearing or disappearing along the include path can change the result of a "make clean;make all", but are not detected by make's dependecy analysis. Here an explanation of D.J. Bernstein: http://cr.yp.to/redo/honest-nonfile.html http://cr.yp.to/redo/honest-nonfile.html If the developer in the scenario described had simply added "vis.h" as a dependency to the rule that depends on it, it would not have happened. Creating "vis.h" and adding it and running make again would have solved the problem. > doing atomically rebuilds: http://cr.yp.to/redo/atomic.html http://cr.yp.to/redo/atomic.html, so that a new dependency is either there, and complete, or not there target: generate_something ./generate_something -o target.tmp && mv target.tmp target > parallel builds GNU Make for example does parallel builds fine. Yes, it's necessary to have complete dependency information (which in Make is achieved only by specifying it) but that's a necessity of any build system that supposes to reliably track dependencies in every case. In general, your criticism seems aimed specifically against Make+CC. In that sense I completely agree that it's not a great combination and that there are probably solutions much better tailored for building C code that address its quirks specifically. Make works well however when you know what and where your dependencies are. C, when used to build software for n>1 frequently updated operating systems presents a header and library labyrinth that makes that a non-trivial problem.
  - saurik 6y ago
    Compilers like GNU cc (and clang, FWIW) have a flag you can pass to have it dump GNU make compatible dependency files, and you can have it include system header files (I turn this on); and in the other direction, there is a feature of make designed to fetch and include files if they exist and, if not, to actually consider it a target for the current build system to quickly build, mostly for these dependency output files. If you use GNU automake--which is not at all required (I do not tend to use it, for example) but is the intention of the ecosystem--this is all set up for you (though maybe without turning on the feature for system header files to be included?). If you are having issues with C code and make not knowing the dependencies, you are using the tools wrong.
- vimax 6y ago
  Make has no way by itself to know what headers a c file needs. It would need to parse c code to do that. C files typically compile independently and then get linked together, so there are no dependencies between c files the make needs to know about when compiling the object files. The only dependencies make knows about are what you tell it. Gcc does parse your c files and knows which headers are needed. It can print out a list of headers used to compile a c file to an object, and prints it out in make syntax. You can add a short auto-dependency rule that creates and includes your header dependencies automatically. Look up make auto-dependencies.
BiteCode_dev 6y ago
The problem with make, is that it's yet another DSL to learn. Every single tool in your toolbox could introduce a new one, and it leads to fatigue. Task runner? New DSL (make) Test runner? New DSL (robot) Deployement? New DSL (Ansible) Batch spawning? New DSL (tox) Etc Of course, dev in half of those DSL is a total pain, because, like with most DSL, the tooling sucks: terrible completion, linting, debugging or composing experience. So people write/leverage tools they can use with their favorite language. And why not? You have to install it anyway (make isn't default on windows, it's not even installed on vanilla Ubuntu!). When I need a make-like tool, I use doit (https://pydoit.org/ https://pydoit.org/). Why? It's in Python, the language of my projects. So I can use the same tooling, the same libs, the same install and testing procedures. And so can people contributing: no need to train them. Most devs don't know how to use make (most of them are on Windows after all). Using make adds zero benefit for me: doit just does what make does (ok, that sentence is hard to parse :)). But make adds an extra step of asking to install it while I can just slip doit among my other python project dependancies. I have to google the syntax. I can't use tab to complete names, of right click to get documentation. And if (actually when) I need to debug my make file, God have mercy. It's not against make. It just doesn't provide enough value to justify the cost of it.
- The_rationalist 6y ago
  I agree with this, personally I use gradle with kotlin script which is just the normal programming language but behaving as a REPL
- vimax 6y ago
  Make is an extremely simple syntax for declaring file dependencies and then shell agnostic for all of the build rule scripting. If you want to use python, put this in your makefile: SHELL=python .ONESHELL:
  - BiteCode_dev 6y ago
    I've never seen a simple make file that was not a toy exemple. Besides, the opposite is preferable: define the running conditions in $FAMILIAR_LANGUAGE, but then run system shell commands.

by cjfd 6y ago

jnxx 6y ago

Make works pretty well, that's why it is used. But there are a few points it does not cover: - Recursive building is complicated. it is not really easy to compose a small unit into a larger project, so that one just needs to copy the sub-unit into a folder. - in some instances, it is necessary to type extra instances of "make clean", because the dependency analysis of make does not understand that dependencies have changed. - say you build a program which includes /usr/include/ncurses.h. Then, you do a system update which replaces your ncurses library. Or, you do a git pull which changes that header. Make will not rebuild your binary without "make clean". - files appearing or disappearing along the include path can change the result of a "make clean;make all", but are not detected by make's dependecy analysis. Here an explanation of D.J. Bernstein: http://cr.yp.to/redo/honest-nonfile.html http://cr.yp.to/redo/honest-nonfile.html - doing atomically rebuilds: http://cr.yp.to/redo/atomic.html http://cr.yp.to/redo/atomic.html, so that a new dependency is either there, and complete, or not there - parallel builds (which is related to the previous point, and also to the necessity to have complete dependency information).

boomlinde 6y ago
> in some instances, it is necessary to type extra instances of "make clean", because the dependency analysis of make does not understand that dependencies have changed. The only sense in which Make will analyze a dependency is that it will run rules for which the target does not exist or has a less recent mtime than any of its dependencies. When what you describe happens, it's not a failure of Make's dependency analysis, but your failure to specify the dependency. This is easier said than done with the Make+CC combo, but I attribute that problem to quirks of C and CC, not to Make's dead simple and basically infallible dependency analysis. The only way to cause Make's dependency analysis to "fail" is to modify the mtime of an already built target to be more recent than that of a dependency that would otherwise be more recent. That's only a failure in the sense that it might not desirable, it's still well defined behavior of Make. > say you build a program which includes /usr/include/ncurses.h. Then, you do a system update which replaces your ncurses library. Or, you do a git pull which changes that header. Make will not rebuild your binary without "make clean". If this happens, it's because you have a dependency on /usr/include/ncurses.h that you have not specified for the target. Again, easier said than done to specify the location of system wide header files that are maybe only resolved using pkg-config because you have no idea of their location on the users' systems, but that's not a problem that Make supposes to solve for C and the ecosystem of applications that exist to patch up its deficiencies. Even worse is for example if you've specified your dependency on the ncurses header, update the system and end up with a new ncurses header that has new includes of its own that you have not specified. The only way you'll ever know is by reading ncurses header file. If you don't, the next system upgrade might update those unspecified indirect dependencies and your Makefile will shrug because you haven't specified those as dependencies. Broken, but really not on the Make end. > files appearing or disappearing along the include path can change the result of a "make clean;make all", but are not detected by make's dependecy analysis. Here an explanation of D.J. Bernstein: http://cr.yp.to/redo/honest-nonfile.html http://cr.yp.to/redo/honest-nonfile.html If the developer in the scenario described had simply added "vis.h" as a dependency to the rule that depends on it, it would not have happened. Creating "vis.h" and adding it and running make again would have solved the problem. > doing atomically rebuilds: http://cr.yp.to/redo/atomic.html http://cr.yp.to/redo/atomic.html, so that a new dependency is either there, and complete, or not there target: generate_something ./generate_something -o target.tmp && mv target.tmp target > parallel builds GNU Make for example does parallel builds fine. Yes, it's necessary to have complete dependency information (which in Make is achieved only by specifying it) but that's a necessity of any build system that supposes to reliably track dependencies in every case. In general, your criticism seems aimed specifically against Make+CC. In that sense I completely agree that it's not a great combination and that there are probably solutions much better tailored for building C code that address its quirks specifically. Make works well however when you know what and where your dependencies are. C, when used to build software for n>1 frequently updated operating systems presents a header and library labyrinth that makes that a non-trivial problem.
- saurik 6y ago
  Compilers like GNU cc (and clang, FWIW) have a flag you can pass to have it dump GNU make compatible dependency files, and you can have it include system header files (I turn this on); and in the other direction, there is a feature of make designed to fetch and include files if they exist and, if not, to actually consider it a target for the current build system to quickly build, mostly for these dependency output files. If you use GNU automake--which is not at all required (I do not tend to use it, for example) but is the intention of the ecosystem--this is all set up for you (though maybe without turning on the feature for system header files to be included?). If you are having issues with C code and make not knowing the dependencies, you are using the tools wrong.
vimax 6y ago
Make has no way by itself to know what headers a c file needs. It would need to parse c code to do that. C files typically compile independently and then get linked together, so there are no dependencies between c files the make needs to know about when compiling the object files. The only dependencies make knows about are what you tell it. Gcc does parse your c files and knows which headers are needed. It can print out a list of headers used to compile a c file to an object, and prints it out in make syntax. You can add a short auto-dependency rule that creates and includes your header dependencies automatically. Look up make auto-dependencies.

BiteCode_dev 6y ago

The problem with make, is that it's yet another DSL to learn. Every single tool in your toolbox could introduce a new one, and it leads to fatigue. Task runner? New DSL (make) Test runner? New DSL (robot) Deployement? New DSL (Ansible) Batch spawning? New DSL (tox) Etc Of course, dev in half of those DSL is a total pain, because, like with most DSL, the tooling sucks: terrible completion, linting, debugging or composing experience. So people write/leverage tools they can use with their favorite language. And why not? You have to install it anyway (make isn't default on windows, it's not even installed on vanilla Ubuntu!). When I need a make-like tool, I use doit (https://pydoit.org/ https://pydoit.org/). Why? It's in Python, the language of my projects. So I can use the same tooling, the same libs, the same install and testing procedures. And so can people contributing: no need to train them. Most devs don't know how to use make (most of them are on Windows after all). Using make adds zero benefit for me: doit just does what make does (ok, that sentence is hard to parse :)). But make adds an extra step of asking to install it while I can just slip doit among my other python project dependancies. I have to google the syntax. I can't use tab to complete names, of right click to get documentation. And if (actually when) I need to debug my make file, God have mercy. It's not against make. It just doesn't provide enough value to justify the cost of it.

The_rationalist 6y ago
I agree with this, personally I use gradle with kotlin script which is just the normal programming language but behaving as a REPL
vimax 6y ago
Make is an extremely simple syntax for declaring file dependencies and then shell agnostic for all of the build rule scripting. If you want to use python, put this in your makefile: SHELL=python .ONESHELL:
- BiteCode_dev 6y ago
  I've never seen a simple make file that was not a toy exemple. Besides, the opposite is preferable: define the running conditions in $FAMILIAR_LANGUAGE, but then run system shell commands.