4 ms·

If your makefile has so many targets, maybe you are doing too much and you need to use another tool such as autoconf/automake on top?

asveikau 6y ago
Autotools projects tend to have recursive makefiles. Look up "recursive make considered harmful" for a good explanation of why that's bad. Edit: https://pic.blog.plover.com/prog/lib/auug97.pdf https://pic.blog.plover.com/prog/lib/auug97.pdf Then, the actual benefit that autotools provides, which is portability for C code running on mostly Unix-like systems, is a pretty dead topic today. I am not talking about all the C bashing people do in a place like HN, what I mean here is that C99 and POSIX compliance is pretty universal now in a way it wasn't 20 years ago. And there are much fewer Unix-like systems in common use. (Linux has taken over most.) Nobody has to run a check for whether it's safe to include string.h or the return type of getpid() anymore.
- kkapelon 6y ago
  ok, makes sense. But then why would somebody have so many targets on a makefile? What are the common scenarios for having so many targets in a makefile if not for doing different installation options?
  - asveikau 6y ago
    That's like saying "why are there so many shell scripts?" Make is a generalized scripting environment. So in one project you might have a target that builds an app, a bunch of rules that generate manpages, another that puts it in /usr/local, another that code signs, another that uploads to some app store.
  - grive 6y ago
    In the systems I have seen: * Because the rules are generated. Building a subpart sometimes means invoking a generated rule. * Because make is used for other things: building binaries, libraries, examples, test suite, documentation -- then executing them, preparing a test environment, installing them, generating distribution packages, etc. * Because the system is a framework, and each rule relates to one of many plugins (Buildroot for example). * Because the system is just very large and complex (linux). Now to go back to automake. It is still very useful to have automated checks of compliance -- either about some extended warnings supported only by some compilers and not others (clang vs GCC), allowing extensions, static analysis, coverage, etc. That being said, automake is terrible. The two passes to generate the actual build systems means most people are actually reading generated makefile and debugging that. As was said by GP, it is also sub-optimal regarding use of make. Frankly, it is just much simpler to directly work with makefiles, or change build tool altogether (meson + ninja is nice). Autotools has always been a nightmare to work with in my experience. Keeping with make, I much prefer musl way for example, where a POSIX shell script examines the dependencies and configures a single makefile giving the project state, then the actual makefiles will consume it to operate properly.
    - cat199 6y ago
      > Frankly, it is just much simpler to directly work with makefiles agree. pmake (colloquially bsd make though it arrived late in the CSRG era) allows for the concept of a 'makefile library' that handles common things (e.g. '.include <bsd.prog.mk>' to build a program if the project directory is laid out appropriately) which I think would have been a much better approach for autotools instead of generating makefiles in multiple stages. Unfortunately the autotools approach is many peoples 1st exposure to anything make-like so they are left with a bad impression of makefile complexity. IMHO pmake also has a nicer and more flexible syntax extension over posix vs gnumake (e.g. can define a dynamic target in a for loop). Unfortunately pmake is not very well known outside of the BSD community.
    - sigjuice 6y ago
      Reading and debugging the generated makefile should not be the norm IMHO. For the same reason that is isn't usually required to read and debug generated assembly (from C), Java/Python bytecode etc.
      - jschwartzi 6y ago
        Sure, except that the language used to generate the makefile is substantially harder to read in a lot of cases. And it doesn't tell you important things such as how libtool paths are configured or what paths are included in the gcc commands. These aren't that important unless you're cross-compiling for a system that doesn't have the same libc, libgcc, or library set and for which you need to set a "sysroot." Different projects have different ways of doing this, and sometimes a project will use something that sets sysroot appropriately for libtool but then doesn't generate the --sysroot command for gcc. For historical reasons Cross Compilation against a different root filesystem for a different target architecture is somewhat of a second-class citizen in most automake/autotools build systems and you really do have to read the makefile output to figure out how they've configured their particular instance.
  - tom_ 6y ago
    A Makefile that's basically a tool with multiple entry points, like `git'? The one I've got for my current work project has 36 such targets. Some people might have an individual shell script for each one, but this is just what I prefer to do. Everything's conveniently in one file, and it's easy to share code between targets. The convention I use is for all entry points to be phony targets. Non-entry point phony targets have name starting with _. The helper I can then use is: @grep -E "^\.PHONY:" Makefile | grep -v ":default" | sed "s/.PHONY:/ /" | grep -v -E "^[[:space:]]*_" | sort (The two spaces that replace ".PHONY" are just indentation, so the list stands out a bit more visually when scrolling back.) The phony target named "default" is the default one, that runs the above command, so I don't need to see its name.
  - joshmarlow 6y ago
    I've recently started using Make for my Python project as a convenient way of running arbitrary code checks. I like having targets like the following: test - runs pytest with whatever config flags are needed coverage - run test and then check for minimum code coverage run - run the application with sane defaults mypy - does type checking (mypy) lint - run flake8 And I usually have `make all` setup to run everything (static analysis before tests, then code coverage checks). In another project, I built out some testing infrastructure for a Rust project (setting up a testing database and tearing it down at the end). I sometimes feel like I "should" use a dedicated tool for this, but Make is ubiquitous and easy enough to use for simple use cases like this.
  - jcranmer 6y ago
    In my experience, with very large software projects, the multiplicity of targets tends to be related to a combination of multi-stage builds (where you need to build some tools to generate files that are needed to compile the project), installation insanity (e.g., needing to build a large set of locales for upload to the FTP server), and testing (differentiating between multiple test suites!).
  - kbob 6y ago
    Make is not just a build system; it's also an interactive development tool. It's useful to be able to build parts or variants, run tools such as static analyzers, build and run unit tests, flash embedded devices, etc. Make can orchestrate all that without duplicating the effort to track source lists, dependencies, platforms, etc. Here's a typical example that happens to be my current hobby project. Take a look at the first block comment and the "help" target. https://github.com/kbob/MinimumViableSynth-5/blob/master/make/common.make https://github.com/kbob/MinimumViableSynth-5/blob/master/mak...
- dpedu 6y ago
  https://www.microsoft.com/en-us/research/wp-content/uploads/2016/03/hadrian.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
  - non-entity 6y ago
    Heh, when I went to search for what GP recommended, this apparent rebuttal was the first result.

asveikau 6y ago

Autotools projects tend to have recursive makefiles. Look up "recursive make considered harmful" for a good explanation of why that's bad. Edit: https://pic.blog.plover.com/prog/lib/auug97.pdf https://pic.blog.plover.com/prog/lib/auug97.pdf Then, the actual benefit that autotools provides, which is portability for C code running on mostly Unix-like systems, is a pretty dead topic today. I am not talking about all the C bashing people do in a place like HN, what I mean here is that C99 and POSIX compliance is pretty universal now in a way it wasn't 20 years ago. And there are much fewer Unix-like systems in common use. (Linux has taken over most.) Nobody has to run a check for whether it's safe to include string.h or the return type of getpid() anymore.

kkapelon 6y ago
ok, makes sense. But then why would somebody have so many targets on a makefile? What are the common scenarios for having so many targets in a makefile if not for doing different installation options?
- asveikau 6y ago
  That's like saying "why are there so many shell scripts?" Make is a generalized scripting environment. So in one project you might have a target that builds an app, a bunch of rules that generate manpages, another that puts it in /usr/local, another that code signs, another that uploads to some app store.
- grive 6y ago
  In the systems I have seen: * Because the rules are generated. Building a subpart sometimes means invoking a generated rule. * Because make is used for other things: building binaries, libraries, examples, test suite, documentation -- then executing them, preparing a test environment, installing them, generating distribution packages, etc. * Because the system is a framework, and each rule relates to one of many plugins (Buildroot for example). * Because the system is just very large and complex (linux). Now to go back to automake. It is still very useful to have automated checks of compliance -- either about some extended warnings supported only by some compilers and not others (clang vs GCC), allowing extensions, static analysis, coverage, etc. That being said, automake is terrible. The two passes to generate the actual build systems means most people are actually reading generated makefile and debugging that. As was said by GP, it is also sub-optimal regarding use of make. Frankly, it is just much simpler to directly work with makefiles, or change build tool altogether (meson + ninja is nice). Autotools has always been a nightmare to work with in my experience. Keeping with make, I much prefer musl way for example, where a POSIX shell script examines the dependencies and configures a single makefile giving the project state, then the actual makefiles will consume it to operate properly.
  - cat199 6y ago
    > Frankly, it is just much simpler to directly work with makefiles agree. pmake (colloquially bsd make though it arrived late in the CSRG era) allows for the concept of a 'makefile library' that handles common things (e.g. '.include <bsd.prog.mk>' to build a program if the project directory is laid out appropriately) which I think would have been a much better approach for autotools instead of generating makefiles in multiple stages. Unfortunately the autotools approach is many peoples 1st exposure to anything make-like so they are left with a bad impression of makefile complexity. IMHO pmake also has a nicer and more flexible syntax extension over posix vs gnumake (e.g. can define a dynamic target in a for loop). Unfortunately pmake is not very well known outside of the BSD community.
  - sigjuice 6y ago
    Reading and debugging the generated makefile should not be the norm IMHO. For the same reason that is isn't usually required to read and debug generated assembly (from C), Java/Python bytecode etc.
    - jschwartzi 6y ago
      Sure, except that the language used to generate the makefile is substantially harder to read in a lot of cases. And it doesn't tell you important things such as how libtool paths are configured or what paths are included in the gcc commands. These aren't that important unless you're cross-compiling for a system that doesn't have the same libc, libgcc, or library set and for which you need to set a "sysroot." Different projects have different ways of doing this, and sometimes a project will use something that sets sysroot appropriately for libtool but then doesn't generate the --sysroot command for gcc. For historical reasons Cross Compilation against a different root filesystem for a different target architecture is somewhat of a second-class citizen in most automake/autotools build systems and you really do have to read the makefile output to figure out how they've configured their particular instance.
- tom_ 6y ago
  A Makefile that's basically a tool with multiple entry points, like `git'? The one I've got for my current work project has 36 such targets. Some people might have an individual shell script for each one, but this is just what I prefer to do. Everything's conveniently in one file, and it's easy to share code between targets. The convention I use is for all entry points to be phony targets. Non-entry point phony targets have name starting with _. The helper I can then use is: @grep -E "^\.PHONY:" Makefile | grep -v ":default" | sed "s/.PHONY:/ /" | grep -v -E "^[[:space:]]*_" | sort (The two spaces that replace ".PHONY" are just indentation, so the list stands out a bit more visually when scrolling back.) The phony target named "default" is the default one, that runs the above command, so I don't need to see its name.
- joshmarlow 6y ago
  I've recently started using Make for my Python project as a convenient way of running arbitrary code checks. I like having targets like the following: test - runs pytest with whatever config flags are needed coverage - run test and then check for minimum code coverage run - run the application with sane defaults mypy - does type checking (mypy) lint - run flake8 And I usually have `make all` setup to run everything (static analysis before tests, then code coverage checks). In another project, I built out some testing infrastructure for a Rust project (setting up a testing database and tearing it down at the end). I sometimes feel like I "should" use a dedicated tool for this, but Make is ubiquitous and easy enough to use for simple use cases like this.
- jcranmer 6y ago
  In my experience, with very large software projects, the multiplicity of targets tends to be related to a combination of multi-stage builds (where you need to build some tools to generate files that are needed to compile the project), installation insanity (e.g., needing to build a large set of locales for upload to the FTP server), and testing (differentiating between multiple test suites!).
- kbob 6y ago
  Make is not just a build system; it's also an interactive development tool. It's useful to be able to build parts or variants, run tools such as static analyzers, build and run unit tests, flash embedded devices, etc. Make can orchestrate all that without duplicating the effort to track source lists, dependencies, platforms, etc. Here's a typical example that happens to be my current hobby project. Take a look at the first block comment and the "help" target. https://github.com/kbob/MinimumViableSynth-5/blob/master/make/common.make https://github.com/kbob/MinimumViableSynth-5/blob/master/mak...
dpedu 6y ago
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/03/hadrian.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
- non-entity 6y ago
  Heh, when I went to search for what GP recommended, this apparent rebuttal was the first result.