6 ms·

Quasi-tangent: I was once hit by a coworker doing first code review on a new repo with "hmm...I'm not familiar with using Makefiles as a project management tool

Quasi-tangent: I was once hit by a coworker doing first code review on a new repo with "hmm...I'm not familiar with using Makefiles as a project management tool. So something something [don't remember] we should replace that." It struck me as weird because I don't see `make` as a build tool so much as automating shell script snippets you would/could type at the command line. From that POV I conceptually see Makefiles as like a `bin/` folder full of little scripts, honestly the main advantage for me being every command runs with working dir being where the Makefile is; no need to write a `set -euo pipefail; HERE="$(dirname "$0")"` dingleberry at the top of every hypothetical `bin/` script. And having incremental rebuild is sugar on top when it's possible to write :) I don't see `make` as a build tool because I believe from experience that is where the road to hell begins. But it is convenient to have `make` call CMake/Cargo/pip/whatever. Plain `make` can build, `make test` can test, `make fmt` can auto-format... Even better if you alias `m` to `make` in your shell.

gregwebs 3y ago
Your coworker's experience is more principled: Make is a mediocre tool for executing commands. It wasn't ever designed for that. Although it is pretty common to see what you are mentioning in projects because it doesn't require installing a dependency. For a repo where an easy to install (single binary) dependency is a non-issue, consider using just. [1] You get `just -l` where you can see all the command available, the ability to use different languages, and overall simpler command writing. [1] https://github.com/casey/just https://github.com/casey/just
- MrOxiMoron 3y ago
  just is great, I add it everywhere, just test, just run, just fix, just shell. just works ;-)
- masspro 3y ago
  This looks really nice. But then should `j` alias to `just` or to `jobs`? Non-starter. /s
- sime2009 3y ago
  A similar tool is `task` https://taskfile.dev/ https://taskfile.dev/ . It is quite capable and also a single executable. I've grown to quite like it.
  - wirrbel 3y ago
    I prefer task over just, while I am not a huge fan of YAML, we now use it everywhere so it just makes sense to not learn yet another DSL for Just and just use YAML.
    - bravura 3y ago
      How do task and just compare to SCons? Although SCons is Python (which is a pro or con depending upon your perspective), it has strong dependency management. Or is the argument that dependency management is part of build, not general project maintenance?
      - wirrbel 3y ago
        Starting out from the blog post, it talks essentially about Make as it was intended, as a build system to compile programs with. Make maps this to the task of producing a file from input files, which are written down in the form of rules in a Makefile. a key ingredient from make is that it checks for timestamps on disk for the source files and updates targets only if the source files have been modified after the targets have been built. If you go a bit further down this route, you end up with build tools that generate the compilation rules for you in some form: These are Automake/CMake/Meson and SCons. I did use scons years ago and it was nice, but its definitely completely lost its market share. IIRC Scons does this without generating Makefiles. Task and Just are following a different route. The problem people have solved by using a "hack" in Makefiles (PHONY targets), so that you can easily run "sub-commands" in Make (make install_deps, etc). It would never occur to me to use Scons in that space. Btw. a third option is to use a shell script like the following (POSIX-shell compatible actually). sub_install_deps() { set -e -x # ... } sc=$1 case $sc in "" | "-h" | "--help") sub_help ;; *) shift "sub_${sc}" "$@" if [ $? = 127 ]; then echo "Error: '${sc}' is not a known sc." >&2 echo " Run '${prog_name} --help' for a list of known scs." >&2 exit 1 fi ;; esac
- hathawsh 3y ago
  With all due respect, I don't understand the first part of your comment. Make's core purpose is to execute commands, isn't it? How was it not designed to execute commands?
  - nine_k 3y ago
    No, no. Make's principal purpose is to put a set of files into a desired state. It can "make" a particular file by invoking a dependent graph of commands that produce that file from other files. It checks timestamps and only run steps where the resulting files are older than some of the (transitive) source files. You can invoke it by naming a named rule, not a file, but the logic will remain. If this is not what you want to be doing, and if your Makefile is full of .PHONY targets, you likely need a Justfile instead, or (worse) plain shell scripts.
    - KajMagnus 3y ago
      I also don't like Make for running commands. I was using Make for running commands, but it interprets all command line parameters as build targets, which was annoying. And Bash scripts in Makefiles aren't type safe, became messy. Nowadays I use Make only for building. And Deno + Typescript instead, for running commands & scripts. (Hadn't heard about Just — the scrips aren't type safe though?)
      - maleldil 3y ago
        > Hadn't heard about Just — the scrips aren't type safe though? Just runs code written in other languages. It's `sh` by default, but you can define anything (e.g. `python3 -c` to run Python scripts).
      - gregwebs 3y ago
        depends on how you define type-safe. Even in TypeScript you are still executing strings when running commands. You can turn those strings into variables and share them as a form of type safety- the type check will fail if you mis-type the variable name. Just supports that- you will get an error if a variable that is being inserted doesn't actually exist. The error will occur at startup before usage or can be checked ahead of time with just --check --fmt. So for quick usage there's not much benefit to TS, but over time in TS you could build up type safe interfaces to commands and obviously if you want to run TS scripts instead of shell scripts that's going to have more opportunities for strong typing then invoking TS from just.
- mort96 3y ago
  Honestly Make is a pretty decent tool for executing shell snippets plus some lacklustre dependency resolution stuff on top.
hathawsh 3y ago
I really like GNU Make because it has a hidden superpower: the "-j" parameter enables instant, easy parallelization. I have a project with many subprojects and I use "-j16" to invoke the same command in all of the subprojects, 16 at a time. It saves a lot of time and it works for all commands that don't touch other subprojects. Like you, I use Make as a front end to lower level build tools. It seems to fit that role well.
- ttyprintk 3y ago
  ‘-j -l $MAXLOAD’ is even more polite on shared systems. Avoids most runaways, but not all.
- maccard 3y ago
  I think this is a great example of tools being stuck in the 90's, actually. Make should be parallel by default. I get it, I do. I know there's all sorts of legacy reasons, but my 4k resolution screen with 12GB vram and 32 cores still runs make build serially, outpitx to an 80x24 window and is bottlenecked down by stdout flushing, all because we can't change defaults.
  - stouset 3y ago
    And annoyingly, I don’t think there’s a way to tell `make` that a particular `Makefile` should be built with multiple threads by default. So you have to specify `-j` every time.
    - maccard 3y ago
      Yep. If it was opt-in in a per-makefile way, it would be ok (or preferably opt-out). I think cmake got this absolutely right, fwiw. You define the version of cmake you're writing against, and they go to great pains to ensure that they preserve behaviour even in newer versions of cmake. As an example, we could have a special variable: make_ver = 4.5 And even running with a future version of make the behaviour would be that of 4.5. if it's not present, it defaults to the latest version before the versions were introduced. This would let make do things like accept spaces instead of tabs, be parallel by default, etc in future versions while presenting old behaviour.
t43562 3y ago
Make is declarative - that's roughly the point of it. You give it rules and dependencies and variables and it's supposed to work out what to do. If your build is extremely simple and fast then it cannot add value.

gregwebs 3y ago

Your coworker's experience is more principled: Make is a mediocre tool for executing commands. It wasn't ever designed for that. Although it is pretty common to see what you are mentioning in projects because it doesn't require installing a dependency. For a repo where an easy to install (single binary) dependency is a non-issue, consider using just. [1] You get `just -l` where you can see all the command available, the ability to use different languages, and overall simpler command writing. [1] https://github.com/casey/just https://github.com/casey/just

MrOxiMoron 3y ago
just is great, I add it everywhere, just test, just run, just fix, just shell. just works ;-)
masspro 3y ago
This looks really nice. But then should `j` alias to `just` or to `jobs`? Non-starter. /s
sime2009 3y ago
A similar tool is `task` https://taskfile.dev/ https://taskfile.dev/ . It is quite capable and also a single executable. I've grown to quite like it.
- wirrbel 3y ago
  I prefer task over just, while I am not a huge fan of YAML, we now use it everywhere so it just makes sense to not learn yet another DSL for Just and just use YAML.
  - bravura 3y ago
    How do task and just compare to SCons? Although SCons is Python (which is a pro or con depending upon your perspective), it has strong dependency management. Or is the argument that dependency management is part of build, not general project maintenance?
    - wirrbel 3y ago
      Starting out from the blog post, it talks essentially about Make as it was intended, as a build system to compile programs with. Make maps this to the task of producing a file from input files, which are written down in the form of rules in a Makefile. a key ingredient from make is that it checks for timestamps on disk for the source files and updates targets only if the source files have been modified after the targets have been built. If you go a bit further down this route, you end up with build tools that generate the compilation rules for you in some form: These are Automake/CMake/Meson and SCons. I did use scons years ago and it was nice, but its definitely completely lost its market share. IIRC Scons does this without generating Makefiles. Task and Just are following a different route. The problem people have solved by using a "hack" in Makefiles (PHONY targets), so that you can easily run "sub-commands" in Make (make install_deps, etc). It would never occur to me to use Scons in that space. Btw. a third option is to use a shell script like the following (POSIX-shell compatible actually). sub_install_deps() { set -e -x # ... } sc=$1 case $sc in "" | "-h" | "--help") sub_help ;; *) shift "sub_${sc}" "$@" if [ $? = 127 ]; then echo "Error: '${sc}' is not a known sc." >&2 echo " Run '${prog_name} --help' for a list of known scs." >&2 exit 1 fi ;; esac
hathawsh 3y ago
With all due respect, I don't understand the first part of your comment. Make's core purpose is to execute commands, isn't it? How was it not designed to execute commands?
- nine_k 3y ago
  No, no. Make's principal purpose is to put a set of files into a desired state. It can "make" a particular file by invoking a dependent graph of commands that produce that file from other files. It checks timestamps and only run steps where the resulting files are older than some of the (transitive) source files. You can invoke it by naming a named rule, not a file, but the logic will remain. If this is not what you want to be doing, and if your Makefile is full of .PHONY targets, you likely need a Justfile instead, or (worse) plain shell scripts.
  - KajMagnus 3y ago
    I also don't like Make for running commands. I was using Make for running commands, but it interprets all command line parameters as build targets, which was annoying. And Bash scripts in Makefiles aren't type safe, became messy. Nowadays I use Make only for building. And Deno + Typescript instead, for running commands & scripts. (Hadn't heard about Just — the scrips aren't type safe though?)
    - maleldil 3y ago
      > Hadn't heard about Just — the scrips aren't type safe though? Just runs code written in other languages. It's `sh` by default, but you can define anything (e.g. `python3 -c` to run Python scripts).
    - gregwebs 3y ago
      depends on how you define type-safe. Even in TypeScript you are still executing strings when running commands. You can turn those strings into variables and share them as a form of type safety- the type check will fail if you mis-type the variable name. Just supports that- you will get an error if a variable that is being inserted doesn't actually exist. The error will occur at startup before usage or can be checked ahead of time with just --check --fmt. So for quick usage there's not much benefit to TS, but over time in TS you could build up type safe interfaces to commands and obviously if you want to run TS scripts instead of shell scripts that's going to have more opportunities for strong typing then invoking TS from just.
mort96 3y ago
Honestly Make is a pretty decent tool for executing shell snippets plus some lacklustre dependency resolution stuff on top.

hathawsh 3y ago

I really like GNU Make because it has a hidden superpower: the "-j" parameter enables instant, easy parallelization. I have a project with many subprojects and I use "-j16" to invoke the same command in all of the subprojects, 16 at a time. It saves a lot of time and it works for all commands that don't touch other subprojects. Like you, I use Make as a front end to lower level build tools. It seems to fit that role well.

ttyprintk 3y ago
‘-j -l $MAXLOAD’ is even more polite on shared systems. Avoids most runaways, but not all.
maccard 3y ago
I think this is a great example of tools being stuck in the 90's, actually. Make should be parallel by default. I get it, I do. I know there's all sorts of legacy reasons, but my 4k resolution screen with 12GB vram and 32 cores still runs make build serially, outpitx to an 80x24 window and is bottlenecked down by stdout flushing, all because we can't change defaults.
- stouset 3y ago
  And annoyingly, I don’t think there’s a way to tell `make` that a particular `Makefile` should be built with multiple threads by default. So you have to specify `-j` every time.
  - maccard 3y ago
    Yep. If it was opt-in in a per-makefile way, it would be ok (or preferably opt-out). I think cmake got this absolutely right, fwiw. You define the version of cmake you're writing against, and they go to great pains to ensure that they preserve behaviour even in newer versions of cmake. As an example, we could have a special variable: make_ver = 4.5 And even running with a future version of make the behaviour would be that of 4.5. if it's not present, it defaults to the latest version before the versions were introduced. This would let make do things like accept spaces instead of tabs, be parallel by default, etc in future versions while presenting old behaviour.

t43562 3y ago

Make is declarative - that's roughly the point of it. You give it rules and dependencies and variables and it's supposed to work out what to do. If your build is extremely simple and fast then it cannot add value.