5 ms·

A couple make flags that are useful and probably not very well known: Output synchronization which makes `make` print stdout/stderr only once a target finishes

A couple make flags that are useful and probably not very well known: Output synchronization which makes `make` print stdout/stderr only once a target finishes. Otherwise it's typically interleaved and hard to follow: make --output-sync=recurse -j10 On busy / multi-user systems, the `-j` flag for jobs may not be best. Instead you can also limit parallelism based on load average: make -j10 --load-average=10 Randomizing the order in which targets are scheduled. This is useful for your CI to harden your Makefiles and see if you're missing dependencies between targets: make --shuffle # or --shuffle=seed/reverse

holsta 1y ago
> A couple make flags that are useful [..] But not portable. Please don't use them outside of your own non-distributable toy projects.
- stabbles 1y ago
  The guide is basically about GNU Make, and the flags are obviously just for end users to invoke make.
- nrclark 1y ago
  Agreed if you're looking at it through the lens of portable software that you plan to distribute. Automake generates portable Makefiles for a reason. But there's another huge category: people who are automating something that's not open-source. Maybe it stays within the walls of their company, where it's totally fine to say "build machines will always be Ubuntu" or whatever other environment their company prefers. GNU Make has a ton of powerful features, and it makes sense to take advantage of them if you know that GNU Make will always be the one you use.
- matheusmoreira 1y ago
  Portability is overrated. Better to make full use of one's tools. Restricting oneself to some "portable" subset of all features is pure masochism. GNU Make is feature rich and is itself portable. It's also free software, as in freedom. Just use it.
  - f1shy 1y ago
    Not only overrated, but also the source of extreme complex and gigantic pieces of software, which end being a nightmare to keep updated. Just like optimization, it has its place and time.
    - wahern 1y ago
      People are too quick to [ab]use GNU Make features. IME, learning how to make do with portable make constructs can help discipline oneself to avoid excessive complexity, especially when it comes to macro definitions where GNU Make's Lispy looping and eval constructs are heavily overused and quickly lead to obtuse, impenetrable code. POSIX pattern substitutions are quite powerful and often produce easier to read code than the GNU equivalent. I'm not sure if computed variable names/nested variable references are well-defined in POSIX (e.g. "$($(FOO))"), but they are widely supported nonetheless, and often more readable than $(eval ...). (They can also be used for portable conditional constructs; I wouldn't argue they're more readable, though I often find them so.) Some GNU Make constructs, like pattern rules, are indispensable in all but the simplest projects, but can also be overused. For some reason there's a strong urge to programmatically generate build rules. But like with SQL queries, going beyond the parameterization already built into the language can be counter productive. A good Makefile, like a good SQL query, should be easy to read on its face. Yes, it often means greater verbosity and even repetition, but that can be a benefit to be embraced (at least embraced more than is instinctively common). EDIT: Computed variable references are well-defined as of POSIX-2024, including (AFAICT) on the left-hand side of a definition. In the discussion it was shown the semantics were already supported by all extant implementations.
      - f1shy 1y ago
        Absolutely. But the target has nothing to do with “portability” but a more fundamental and important principle(s) of readability and maintainability.
        wahern 1y ago
        It's a matter of praxis. Targeting portable constructs is (IMO) a useful methodology for achieving the abstract goal. It doesn't have to be strict, but it provides a quantifiable, objective metric (i.e. amount of non-portable constructs employed) to help achieve an otherwise subjective goal. Otherwise you face an ocean of choices that can be overwhelming, especially if you're not very experienced in the problem space. It's like the common refrain with C++: most developers settle on a subset of C++ to minimize code complexity; but which subset? (They can vary widely, across projects and time.) In the case of Make, you can just pick the POSIX and/or de facto portable subset as your target, avoiding alot of choice paralysis/anxiety (though you still face it when deciding when to break out of that box to leverage GNU extensions).
- f1shy 1y ago
  Not every project has to be a multi-platform, multi-os, multi-language monster. It is perfectly fine to target a specific set of architecture, os, etc. And I find insulting and silly calling it a “toy project”
- deng 1y ago
  I will not restrict myself to an arcane subset of Make just because you refuse to type 'gmake' instead of 'make'. Parallel execution, pattern rules, order-only prerequisites, includes, not to mention the dozens of useful function like (not)dir, (pat)subst, info... There's a reason why most POSIX Makefiles nowadays are generated. It's not GNU's fault that POSIX is stale. EDIT: There's one exception, and that would be using Guile as an extension language, as that is often not available. However, thanks to conditionals (also not in POSIX, of course), it can be used optionally. I once sped up a Windows build by an order of magnitude by implementing certain things in Guile instead of calling shell (which is notoriously slow on Windows).
  - matheusmoreira 1y ago
    Completely agree. POSIX is irrelevant anyway. Every single unixlike has unique features that are vastly superior to whatever legacy happens to be standardized by POSIX. Avoiding their use leads to nothing but misery.
  - Tor3 1y ago
    Agreed. My company decided on using GNU Make on every platform we supported, which back then (last century) was a bunch of Unix variants, and Linux. That made it possible to write a simple and portable build system which could be used for everything we did, no hassle. And not difficult, because gmake was available basically everywhere, then just as now.
f1shy 1y ago
The one that I most use is -B for unconditional build all
davemp 1y ago
I’ve seen and had ‘make -j’ dos machines enough times that I consider it a bug.
- bayindirh 1y ago
  If "make -j" successfully drowns a machine, I can argue that the machine has no serious bottlenecks for the job. Because, make is generally I/O bound when run with high parallelism, and if you can't saturate your I/O bandwidth, that's a good thing in general. However, if "make -j" is saturates a machine, and this is unintentional, I'd assume PEBKAC, or "holding it wrong", in general.
  - davemp 1y ago
    The problem is ‘make -j’ spinning up 100s of C++ compilation jobs, using up all of the systems RAM+swap, and causing major instability. I get that the OS could mitigate this, but that’s often not an option in professional settings. The reality is that most of the time users are expecting ‘make -j $(N_PROC)’, get bit in the ass, and then the GNU maintainers say PEBKAC—wasting hundreds of hours of junior dev time.
    - deleted 1y ago
      [deleted]
    - deleted 1y ago
      [deleted]
    - bayindirh 1y ago
      I’ll kindly disagree on wasting junior developer time. Any person who’s using tools professionally should read (or at least skim) the manual of the said tool. Especially, if it’s something foundational to their all workflow. They are junior because they are inexperienced, but being junior is the best place to make mistakes and learn good habits. If somebody asks what is the most important thing I have learnt over the years, I’d say “read the manual and the logs”.
      - 1y ago
blueflow 1y ago
> and probably not very well known Maybe the make authors could compile a list of options somewhere and ship it with their program, so users could read them? Something like a text file or using some typesetting language. This would make that knowledge much more accessible.
- tux1968 1y ago
  make --help Will give you the command line options. And GNU make has decent documentation online for everything else: https://www.gnu.org/software/make/manual/html_node/index.html https://www.gnu.org/software/make/manual/html_node/index.htm...
- moefh 1y ago
  Not sure if you're being snarky, but the manual has a list of all options accepted by make: https://www.gnu.org/software/make/manual/html_node/Options-Summary.html https://www.gnu.org/software/make/manual/html_node/Options-S... (`make --help` will only print the most common options)
- gbacon 1y ago
  That's really not necessary. Everyone here knows where to find the manual, and we all also know it has a considerable volume of information.
monkeyelite 1y ago
> On busy / multi-user systems Can’t the OS scheduler handle it?

holsta 1y ago

> A couple make flags that are useful [..] But not portable. Please don't use them outside of your own non-distributable toy projects.

stabbles 1y ago
The guide is basically about GNU Make, and the flags are obviously just for end users to invoke make.
nrclark 1y ago
Agreed if you're looking at it through the lens of portable software that you plan to distribute. Automake generates portable Makefiles for a reason. But there's another huge category: people who are automating something that's not open-source. Maybe it stays within the walls of their company, where it's totally fine to say "build machines will always be Ubuntu" or whatever other environment their company prefers. GNU Make has a ton of powerful features, and it makes sense to take advantage of them if you know that GNU Make will always be the one you use.
matheusmoreira 1y ago
Portability is overrated. Better to make full use of one's tools. Restricting oneself to some "portable" subset of all features is pure masochism. GNU Make is feature rich and is itself portable. It's also free software, as in freedom. Just use it.
- f1shy 1y ago
  Not only overrated, but also the source of extreme complex and gigantic pieces of software, which end being a nightmare to keep updated. Just like optimization, it has its place and time.
  - wahern 1y ago
    People are too quick to [ab]use GNU Make features. IME, learning how to make do with portable make constructs can help discipline oneself to avoid excessive complexity, especially when it comes to macro definitions where GNU Make's Lispy looping and eval constructs are heavily overused and quickly lead to obtuse, impenetrable code. POSIX pattern substitutions are quite powerful and often produce easier to read code than the GNU equivalent. I'm not sure if computed variable names/nested variable references are well-defined in POSIX (e.g. "$($(FOO))"), but they are widely supported nonetheless, and often more readable than $(eval ...). (They can also be used for portable conditional constructs; I wouldn't argue they're more readable, though I often find them so.) Some GNU Make constructs, like pattern rules, are indispensable in all but the simplest projects, but can also be overused. For some reason there's a strong urge to programmatically generate build rules. But like with SQL queries, going beyond the parameterization already built into the language can be counter productive. A good Makefile, like a good SQL query, should be easy to read on its face. Yes, it often means greater verbosity and even repetition, but that can be a benefit to be embraced (at least embraced more than is instinctively common). EDIT: Computed variable references are well-defined as of POSIX-2024, including (AFAICT) on the left-hand side of a definition. In the discussion it was shown the semantics were already supported by all extant implementations.
    - f1shy 1y ago
      Absolutely. But the target has nothing to do with “portability” but a more fundamental and important principle(s) of readability and maintainability.
      - wahern 1y ago
        It's a matter of praxis. Targeting portable constructs is (IMO) a useful methodology for achieving the abstract goal. It doesn't have to be strict, but it provides a quantifiable, objective metric (i.e. amount of non-portable constructs employed) to help achieve an otherwise subjective goal. Otherwise you face an ocean of choices that can be overwhelming, especially if you're not very experienced in the problem space. It's like the common refrain with C++: most developers settle on a subset of C++ to minimize code complexity; but which subset? (They can vary widely, across projects and time.) In the case of Make, you can just pick the POSIX and/or de facto portable subset as your target, avoiding alot of choice paralysis/anxiety (though you still face it when deciding when to break out of that box to leverage GNU extensions).
f1shy 1y ago
Not every project has to be a multi-platform, multi-os, multi-language monster. It is perfectly fine to target a specific set of architecture, os, etc. And I find insulting and silly calling it a “toy project”
deng 1y ago
I will not restrict myself to an arcane subset of Make just because you refuse to type 'gmake' instead of 'make'. Parallel execution, pattern rules, order-only prerequisites, includes, not to mention the dozens of useful function like (not)dir, (pat)subst, info... There's a reason why most POSIX Makefiles nowadays are generated. It's not GNU's fault that POSIX is stale. EDIT: There's one exception, and that would be using Guile as an extension language, as that is often not available. However, thanks to conditionals (also not in POSIX, of course), it can be used optionally. I once sped up a Windows build by an order of magnitude by implementing certain things in Guile instead of calling shell (which is notoriously slow on Windows).
- matheusmoreira 1y ago
  Completely agree. POSIX is irrelevant anyway. Every single unixlike has unique features that are vastly superior to whatever legacy happens to be standardized by POSIX. Avoiding their use leads to nothing but misery.
- Tor3 1y ago
  Agreed. My company decided on using GNU Make on every platform we supported, which back then (last century) was a bunch of Unix variants, and Linux. That made it possible to write a simple and portable build system which could be used for everything we did, no hassle. And not difficult, because gmake was available basically everywhere, then just as now.

f1shy 1y ago

The one that I most use is -B for unconditional build all

davemp 1y ago

I’ve seen and had ‘make -j’ dos machines enough times that I consider it a bug.

bayindirh 1y ago
If "make -j" successfully drowns a machine, I can argue that the machine has no serious bottlenecks for the job. Because, make is generally I/O bound when run with high parallelism, and if you can't saturate your I/O bandwidth, that's a good thing in general. However, if "make -j" is saturates a machine, and this is unintentional, I'd assume PEBKAC, or "holding it wrong", in general.
- davemp 1y ago
  The problem is ‘make -j’ spinning up 100s of C++ compilation jobs, using up all of the systems RAM+swap, and causing major instability. I get that the OS could mitigate this, but that’s often not an option in professional settings. The reality is that most of the time users are expecting ‘make -j $(N_PROC)’, get bit in the ass, and then the GNU maintainers say PEBKAC—wasting hundreds of hours of junior dev time.
  - deleted 1y ago
    [deleted]
  - deleted 1y ago
    [deleted]
  - bayindirh 1y ago
    I’ll kindly disagree on wasting junior developer time. Any person who’s using tools professionally should read (or at least skim) the manual of the said tool. Especially, if it’s something foundational to their all workflow. They are junior because they are inexperienced, but being junior is the best place to make mistakes and learn good habits. If somebody asks what is the most important thing I have learnt over the years, I’d say “read the manual and the logs”.
    - 1y ago

blueflow 1y ago

> and probably not very well known Maybe the make authors could compile a list of options somewhere and ship it with their program, so users could read them? Something like a text file or using some typesetting language. This would make that knowledge much more accessible.

tux1968 1y ago
make --help Will give you the command line options. And GNU make has decent documentation online for everything else: https://www.gnu.org/software/make/manual/html_node/index.html https://www.gnu.org/software/make/manual/html_node/index.htm...
moefh 1y ago
Not sure if you're being snarky, but the manual has a list of all options accepted by make: https://www.gnu.org/software/make/manual/html_node/Options-Summary.html https://www.gnu.org/software/make/manual/html_node/Options-S... (`make --help` will only print the most common options)
gbacon 1y ago
That's really not necessary. Everyone here knows where to find the manual, and we all also know it has a considerable volume of information.

monkeyelite 1y ago

> On busy / multi-user systems Can’t the OS scheduler handle it?