5 ms·

GNU Make Standard Library

164 points by ingve 2y ago

shakna 2y ago
I wonder how thread-safe the memoisation and/or ALists are? Make's version of parralel processing is a very fun little thing, but has a few quirks around recursion limits that can bite you when going off the beaten path. [0] https://www.gnu.org/software/make/manual/html_node/Options_002fRecursion.html https://www.gnu.org/software/make/manual/html_node/Options_0...
jgrahamc 2y ago
I guess this is here because it's been 20 years and I blogged about it on Monday: https://blog.jgc.org/2025/02/twenty-years-of-gnu-make-standard.html https://blog.jgc.org/2025/02/twenty-years-of-gnu-make-standa...
t43562 2y ago
GNU make now has a load directive which lets you load up functions written in C -load my_module.o Your makefile can contain instructions to build my_module.o and they will be automatically triggered. For example you can create an equality test that works in an expression (ifeq can't be used as part of an expression obviously). For example FILENAME=file.$(if $(equals $(compression_level),0),tar,.tar.bz3) The C for this function (without includes and export directives): char * func_equals (const char *func_name, unsigned int argc, char **argv) { char *result = NULL; if (strcmp(argv[0], argv[1]) == 0) { result = gmk_alloc(strlen(argv[0]) + 1); /* not handling failure for simplicity */ strcpy(result, argv[0]); } return result; } This can be done with a macro but it's ugly and verbose. Macros also slow makefile parsing a lot and for a large build like e.g. an operating system this makes a big difference - it's a penalty you pay every time you run "make" even if you only changed 1 file. There are plenty of things you cannot do with macros too. $(shell) is a getout card but it drastically slows down large makefiles. Your module has a setup function which gets called when it's loaded and this adds the function into gmake: int equals_gmk_setup (const gmk_floc *flocp) { gmk_add_function ("equals", func_equals, 2, 2, GMK_FUNC_DEFAULT); return 1; } Things that are hard/slow to do with macros like arithmetic - comparing, adding and so on are even better candidates. A hash function is great for generating intermediate target names that aren't too long for the filesystem. My favorite one that I've done is embedding a python interpreter into make - this is very convenient as it's MUCH faster than running a process from $(shell) and it keeps state between uses which can be useful.
- bandrami 2y ago
  GNU Make also embeds GNU Guile, a criminally underused feature: https://www.gnu.org/software/make/manual/html_node/Guile-Integration.html https://www.gnu.org/software/make/manual/html_node/Guile-Int...
  - srik 2y ago
    TIL thank you!
    - bandrami 2y ago
      I sometimes wonder if we would even have autotools or cmake if people just knew about this one simple trick
      - o11c 2y ago
        Autotools is designed to solve one very important problem: how do you build the GNU tools in the first place if all you have is some obscure Unix from the 1980s. If you already have gnu make, gnu bash, gnu binutils, gnu coreutils, etc. installed then autotools is pointless. I have yet to find evidence of cmake solving a problem (or even having design), though I guess `ccmake` would be kind of cool if it weren't full of nonsense?
        t43562 2y ago
        I see Autotools as sort of cool if you're porting to a new platform - the tests find out what works and if that's not enough then you have to make some effort to get around it. If you're lucky, however, you put your autotooled code on some completely new OS/hardware and it just builds. Nowadays the proportion of people who are porting the code is probably much smaller but it's still a way of keeping code working on unix with various compiler, architecture variations. IMO cmake just includes windows more effectively - for autotools you'd probably be forced down the cygwin route. I find it a bit easier to work with but it's still a nightmare from hell sometimes.
        2y ago
p4bl0 2y ago
Okay who will be doing this year's Advent of Code in GNU Make?
- throwfgtpwd234 2y ago
  What we really need is a C to GNU Make transpiler written in C just so it can translate itself.
  - phtrivier 2y ago
    But what system would you to build that ?
    - throwfgtpwd234 2y ago
      GNU Make and GCC, of course.
throwfgtpwd234 2y ago
OP FYI: US copyright law doesn't recognize or require a range of years, only the date of first publication. Many organizations have decided to omit (the) year(s) altogether. https://blog.ntpsec.org/2020/02/15/copyright-year.html https://blog.ntpsec.org/2020/02/15/copyright-year.html
- jgrahamc 2y ago
  Good to know. I tend to use them as markers of "I started working on this in year XXXX and I last worked on it in year YYYY".
- nemoniac 2y ago
  Check the first comment on the linked page for counterpoint. Also note that copyright laws exist outside the US and may differ.
  - throwfgtpwd234 2y ago
    [flagged]
- mubou 2y ago
  Correct me if I'm wrong, but I don't think the copyright statement actually has any legal effect anymore (Berne Convention of 1886 / 1989 for the US) so it doesn't really matter if there's a year or not. I think it's just informative.
rkangel 2y ago
I have written some large build systems entirely in Make. More complex things tend to rely on templates, but you can build arbitrary things, with two main limitations: The error messages are awful, particularly if using templates. "Unexpected separator. Stop" is a classic, with no indication where in your 2k lines of Make it might be. You can't have file or folder names with spaces in (usually including any parent folder to where your code is checked out). A "list" in Make is a series of strings separated by spaces. There are various workarounds that people suggest, but none of them work consistently across platforms. You just have to avoid spaces. At least this is less bad since Windows stopped using "Documents and Settings" for "home" folders.
- dima55 2y ago
  GNU Make now has a debugger (`apt install remake`) that eases your first pain point a lot
  - bonzini 2y ago
    Even then the problem with doing complicated stuff in Make is that it's very hard to reproduce the environment that triggered the bug in the first place. I came to the conclusion that you need to treat all of the build system as a linear process, which in Make would mean for example not using "=" at all except to define functions, only use ":=". With this kind of discipline I never really needed a debugger, but really Make is not the right language to write complex logic. Personally I am a fan of (and contributor to) Meson. It's not perfect but it strikes a good balance between what is in the basic package and what you can do in your build script, and by keeping the build phases separate it really helps with keeping things understandable. The lack of functions can be annoying (and in general I wish it could use Starlark as the language), but it doesn't hurt if you follow the same principle and treat the build script as a data flow process, with each phase producing data structures for the next one. So I think it's generally a good principle to follow.
beardyw 2y ago
Make is just the epitome of software development: Started as a simple idea Required more sophistication Became something no one person really understands
- dima55 2y ago
  Did you try reading the manual?
  - RegW 2y ago
    Certainly not! That would be a faux pas.
  - high_na_euv 2y ago
    When software requires you to read manual, then it is strong hint that it has poor UI UX Like, you dont read web browser manual to use it, even when using advanced features like debugger or dev console (advanced in compare to non-computer person)
    - armitron 2y ago
      Make is a power tool and power tools require effort to fully understand and master (though the base case in Make is surprisingly simple). It also has great documentation (which is something newer generations either don't appreciate or don't care about).
      - high_na_euv 2y ago
        Web browser is 10 times more powerful tool than Make.
        Gracana 2y ago
        And if you want to use it as a build tool, you're going to have to crack open some books.
  - 2y ago
davemp 2y ago
I’ve written a fair amount of Makefiles and bounced off of cmake. Recently I’ve started using zig to build some C++ projects and will not be switching back. Having your build tool just be a library in a good general purpose language is the right move. Why use unergonomic hacks like the OP when you can use a sane language? My build.zig files have LSP completions and similar syntax (or the same) as what I’m building. I put make solidly in the pile of tools that are only still around because of network effects. We’d have switched to something better if it weren’t defacto installed by every distro and had to make justifications to sys admins to install alternatives.
- aeonik 2y ago
  Same reason I like Clojure tools.build. Any other way feels a little ridiculous to do it any other way.
  - deleted 2y ago
    [deleted]
- einpoklum 2y ago
  Well, many of us certainly aren't happy living with the annoying syntax and arcane customs of CMake. But - it does get the job done, and incorporates large amounts of arcane knowledge, multiplied over numerous platforms - so that you don't have to. Can I get the same from a Zig-based build system, or build system generator? Can you perhaps link to some comparative review of using zig for building, with the use of CMake?
  - davemp 2y ago
    I don’t have a review, but here’s a significant project using it in a complicated cross platform build with different systems dependencies: https://github.com/raysan5/raylib/blob/master/build.zig https://github.com/raysan5/raylib/blob/master/build.zig They also have a cmakelists.txt and pile of other cmake stuff to compare against: https://github.com/raysan5/raylib/tree/master/cmake https://github.com/raysan5/raylib/tree/master/cmake One of the nicer things is that if you’re working with less technical folks, they only need to download the zig binary because it bundles the entire toolchain. Real nice when on corporate windows systems.
EPWN3D 2y ago
Just ran across this last week and pulled it into my Makefiles. Really nice.
anthk 2y ago
git clone git://bitreich.org/english_knight You don't need more. I wish everyone used mk instead of bloated software. Unix grew up too much.
- ta8645 2y ago
  That is one seriously funky website; I will not recover from the experience soon. For anyone who doesn't want to clone the above repo, english_knight turns out to just be a template for idiomatic makefiles. I pasted the relevant file from the repo here: https://nopaste.net/english_knight https://nopaste.net/english_knight
  - anthk 2y ago
    Well, you should access bitreich over gopher :) gopher://bitreich.org Much better.
    - ta8645 2y ago
      You're right. That was less traumatic, but still fun, heh.
    - egberts1 2y ago
      Wut? No telnet? #headduck
bch 2y ago
> The GNU Make Standard Library (GMSL) is […] released under the BSD License. That’s mildly interesting.
AceJohnny2 2y ago
The GMSL is a godsend for those of us stuck on GNU Make 3.81 for... "reasons" ("reasons" being on macOS and limited to Xcode Command-Line Tools, which provides a number of GNU tools frozen to the last version that's GPLv2, before GPLv3. Incidentally, that's also why macOS switched to Zsh by default, because Apple was tired of backporting security fixes to their Bash 3.2.57 /rant)
- jcarrano 2y ago
  I can't even describe how much frustration this caused me when I worked at the RIOT project. macOS was by far the most problematic development platform to support. Even Windows was far friendlier since it has WSL which is officially supported.
  - AceJohnny2 2y ago
    In most situations (not mine), you can likely depend on MacPorts, Homebrew, or Nix to provide more up-to-date version of the tools you need that what Xcode provides. In that way, it's similar to WSL in that you can augment your environment to get the proper Unix tools you expect.
m463 2y ago
Why doesn't someone just build good syntax into make itself? just put some coded comment at the top of the file like "#makev2" and below have variables, arrays, lists, ignore tabs vs spaces, etc
remram 2y ago
If it's not distributed with the tool, it is just "a make library".
MayeulC 2y ago
One of my pet peeves with make is that it handles special characters in filenames (space, semicolon) very badly. Would this library help with that? Some time ago, I tried to convert a music library, and ended up bulk renaming the files before and after running my makefile.
elif 2y ago
Having dealt with building a single meson project I can safely say I am glad gnu does not seem to be continuing down that path.

164 points by ingve 2y ago

I wonder how thread-safe the memoisation and/or ALists are? Make's version of parralel processing is a very fun little thing, but has a few quirks around recursion limits that can bite you when going off the beaten path. [0] https://www.gnu.org/software/make/manual/html_node/Options_002fRecursion.html https://www.gnu.org/software/make/manual/html_node/Options_0...

jgrahamc 2y ago

I guess this is here because it's been 20 years and I blogged about it on Monday: https://blog.jgc.org/2025/02/twenty-years-of-gnu-make-standard.html https://blog.jgc.org/2025/02/twenty-years-of-gnu-make-standa...

t43562 2y ago

GNU make now has a load directive which lets you load up functions written in C -load my_module.o Your makefile can contain instructions to build my_module.o and they will be automatically triggered. For example you can create an equality test that works in an expression (ifeq can't be used as part of an expression obviously). For example FILENAME=file.$(if $(equals $(compression_level),0),tar,.tar.bz3) The C for this function (without includes and export directives): char * func_equals (const char *func_name, unsigned int argc, char **argv) { char *result = NULL; if (strcmp(argv[0], argv[1]) == 0) { result = gmk_alloc(strlen(argv[0]) + 1); /* not handling failure for simplicity */ strcpy(result, argv[0]); } return result; } This can be done with a macro but it's ugly and verbose. Macros also slow makefile parsing a lot and for a large build like e.g. an operating system this makes a big difference - it's a penalty you pay every time you run "make" even if you only changed 1 file. There are plenty of things you cannot do with macros too. $(shell) is a getout card but it drastically slows down large makefiles. Your module has a setup function which gets called when it's loaded and this adds the function into gmake: int equals_gmk_setup (const gmk_floc *flocp) { gmk_add_function ("equals", func_equals, 2, 2, GMK_FUNC_DEFAULT); return 1; } Things that are hard/slow to do with macros like arithmetic - comparing, adding and so on are even better candidates. A hash function is great for generating intermediate target names that aren't too long for the filesystem. My favorite one that I've done is embedding a python interpreter into make - this is very convenient as it's MUCH faster than running a process from $(shell) and it keeps state between uses which can be useful.

bandrami 2y ago
GNU Make also embeds GNU Guile, a criminally underused feature: https://www.gnu.org/software/make/manual/html_node/Guile-Integration.html https://www.gnu.org/software/make/manual/html_node/Guile-Int...
- srik 2y ago
  TIL thank you!
  - bandrami 2y ago
    I sometimes wonder if we would even have autotools or cmake if people just knew about this one simple trick
    - o11c 2y ago
      Autotools is designed to solve one very important problem: how do you build the GNU tools in the first place if all you have is some obscure Unix from the 1980s. If you already have gnu make, gnu bash, gnu binutils, gnu coreutils, etc. installed then autotools is pointless. I have yet to find evidence of cmake solving a problem (or even having design), though I guess `ccmake` would be kind of cool if it weren't full of nonsense?
      - t43562 2y ago
        I see Autotools as sort of cool if you're porting to a new platform - the tests find out what works and if that's not enough then you have to make some effort to get around it. If you're lucky, however, you put your autotooled code on some completely new OS/hardware and it just builds. Nowadays the proportion of people who are porting the code is probably much smaller but it's still a way of keeping code working on unix with various compiler, architecture variations. IMO cmake just includes windows more effectively - for autotools you'd probably be forced down the cygwin route. I find it a bit easier to work with but it's still a nightmare from hell sometimes.
      - 2y ago

p4bl0 2y ago

Okay who will be doing this year's Advent of Code in GNU Make?

throwfgtpwd234 2y ago
What we really need is a C to GNU Make transpiler written in C just so it can translate itself.
- phtrivier 2y ago
  But what system would you to build that ?
  - throwfgtpwd234 2y ago
    GNU Make and GCC, of course.

throwfgtpwd234 2y ago

OP FYI: US copyright law doesn't recognize or require a range of years, only the date of first publication. Many organizations have decided to omit (the) year(s) altogether. https://blog.ntpsec.org/2020/02/15/copyright-year.html https://blog.ntpsec.org/2020/02/15/copyright-year.html

jgrahamc 2y ago
Good to know. I tend to use them as markers of "I started working on this in year XXXX and I last worked on it in year YYYY".
nemoniac 2y ago
Check the first comment on the linked page for counterpoint. Also note that copyright laws exist outside the US and may differ.
- throwfgtpwd234 2y ago
  [flagged]
mubou 2y ago
Correct me if I'm wrong, but I don't think the copyright statement actually has any legal effect anymore (Berne Convention of 1886 / 1989 for the US) so it doesn't really matter if there's a year or not. I think it's just informative.

rkangel 2y ago

I have written some large build systems entirely in Make. More complex things tend to rely on templates, but you can build arbitrary things, with two main limitations: The error messages are awful, particularly if using templates. "Unexpected separator. Stop" is a classic, with no indication where in your 2k lines of Make it might be. You can't have file or folder names with spaces in (usually including any parent folder to where your code is checked out). A "list" in Make is a series of strings separated by spaces. There are various workarounds that people suggest, but none of them work consistently across platforms. You just have to avoid spaces. At least this is less bad since Windows stopped using "Documents and Settings" for "home" folders.

dima55 2y ago
GNU Make now has a debugger (`apt install remake`) that eases your first pain point a lot
- bonzini 2y ago
  Even then the problem with doing complicated stuff in Make is that it's very hard to reproduce the environment that triggered the bug in the first place. I came to the conclusion that you need to treat all of the build system as a linear process, which in Make would mean for example not using "=" at all except to define functions, only use ":=". With this kind of discipline I never really needed a debugger, but really Make is not the right language to write complex logic. Personally I am a fan of (and contributor to) Meson. It's not perfect but it strikes a good balance between what is in the basic package and what you can do in your build script, and by keeping the build phases separate it really helps with keeping things understandable. The lack of functions can be annoying (and in general I wish it could use Starlark as the language), but it doesn't hurt if you follow the same principle and treat the build script as a data flow process, with each phase producing data structures for the next one. So I think it's generally a good principle to follow.

beardyw 2y ago

Make is just the epitome of software development: Started as a simple idea Required more sophistication Became something no one person really understands

dima55 2y ago
Did you try reading the manual?
- RegW 2y ago
  Certainly not! That would be a faux pas.
- high_na_euv 2y ago
  When software requires you to read manual, then it is strong hint that it has poor UI UX Like, you dont read web browser manual to use it, even when using advanced features like debugger or dev console (advanced in compare to non-computer person)
  - armitron 2y ago
    Make is a power tool and power tools require effort to fully understand and master (though the base case in Make is surprisingly simple). It also has great documentation (which is something newer generations either don't appreciate or don't care about).
    - high_na_euv 2y ago
      Web browser is 10 times more powerful tool than Make.
      - Gracana 2y ago
        And if you want to use it as a build tool, you're going to have to crack open some books.
- 2y ago

davemp 2y ago

I’ve written a fair amount of Makefiles and bounced off of cmake. Recently I’ve started using zig to build some C++ projects and will not be switching back. Having your build tool just be a library in a good general purpose language is the right move. Why use unergonomic hacks like the OP when you can use a sane language? My build.zig files have LSP completions and similar syntax (or the same) as what I’m building. I put make solidly in the pile of tools that are only still around because of network effects. We’d have switched to something better if it weren’t defacto installed by every distro and had to make justifications to sys admins to install alternatives.

aeonik 2y ago
Same reason I like Clojure tools.build. Any other way feels a little ridiculous to do it any other way.
- deleted 2y ago
  [deleted]
einpoklum 2y ago
Well, many of us certainly aren't happy living with the annoying syntax and arcane customs of CMake. But - it does get the job done, and incorporates large amounts of arcane knowledge, multiplied over numerous platforms - so that you don't have to. Can I get the same from a Zig-based build system, or build system generator? Can you perhaps link to some comparative review of using zig for building, with the use of CMake?
- davemp 2y ago
  I don’t have a review, but here’s a significant project using it in a complicated cross platform build with different systems dependencies: https://github.com/raysan5/raylib/blob/master/build.zig https://github.com/raysan5/raylib/blob/master/build.zig They also have a cmakelists.txt and pile of other cmake stuff to compare against: https://github.com/raysan5/raylib/tree/master/cmake https://github.com/raysan5/raylib/tree/master/cmake One of the nicer things is that if you’re working with less technical folks, they only need to download the zig binary because it bundles the entire toolchain. Real nice when on corporate windows systems.

EPWN3D 2y ago

Just ran across this last week and pulled it into my Makefiles. Really nice.

anthk 2y ago

git clone git://bitreich.org/english_knight You don't need more. I wish everyone used mk instead of bloated software. Unix grew up too much.

ta8645 2y ago
That is one seriously funky website; I will not recover from the experience soon. For anyone who doesn't want to clone the above repo, english_knight turns out to just be a template for idiomatic makefiles. I pasted the relevant file from the repo here: https://nopaste.net/english_knight https://nopaste.net/english_knight
- anthk 2y ago
  Well, you should access bitreich over gopher :) gopher://bitreich.org Much better.
  - ta8645 2y ago
    You're right. That was less traumatic, but still fun, heh.
  - egberts1 2y ago
    Wut? No telnet? #headduck

bch 2y ago

> The GNU Make Standard Library (GMSL) is […] released under the BSD License. That’s mildly interesting.

AceJohnny2 2y ago

The GMSL is a godsend for those of us stuck on GNU Make 3.81 for... "reasons" ("reasons" being on macOS and limited to Xcode Command-Line Tools, which provides a number of GNU tools frozen to the last version that's GPLv2, before GPLv3. Incidentally, that's also why macOS switched to Zsh by default, because Apple was tired of backporting security fixes to their Bash 3.2.57 /rant)

jcarrano 2y ago
I can't even describe how much frustration this caused me when I worked at the RIOT project. macOS was by far the most problematic development platform to support. Even Windows was far friendlier since it has WSL which is officially supported.
- AceJohnny2 2y ago
  In most situations (not mine), you can likely depend on MacPorts, Homebrew, or Nix to provide more up-to-date version of the tools you need that what Xcode provides. In that way, it's similar to WSL in that you can augment your environment to get the proper Unix tools you expect.

m463 2y ago

Why doesn't someone just build good syntax into make itself? just put some coded comment at the top of the file like "#makev2" and below have variables, arrays, lists, ignore tabs vs spaces, etc

remram 2y ago

If it's not distributed with the tool, it is just "a make library".

MayeulC 2y ago

One of my pet peeves with make is that it handles special characters in filenames (space, semicolon) very badly. Would this library help with that? Some time ago, I tried to convert a music library, and ended up bulk renaming the files before and after running my makefile.

elif 2y ago

Having dealt with building a single meson project I can safely say I am glad gnu does not seem to be continuing down that path.