5 ms·

GNU Make Guile Integration

117 points by gebt 7y ago

bjoli 7y ago
I have been using it with great success, but there has been some bother with portability. I used it in ways that could have been done with make proper, but since I already know a lot of guile I reached for that.
- jdc 7y ago
  Which platform did Guile have trouble on?
  - bjoli 7y ago
    Not cross-platform, but cross-computer. It isn't available everywhere, and is quite a fuzz to enable elsewhere.
jacobush 7y ago
Oh, like MSBuild with Powershell scripts. Each evil in their own ways.
snizovtsev 7y ago
make -p -f/dev/null | grep FEATURES
- alex_stoddard 7y ago
  Thank you kindly for providing the way to actually do the documented: "You can determine if make contains support for Guile by examining the .FEATURES variable; it will contain the word guile if Guile support is available." Unix - the documentation is excellent if you already know how to do it...
  - sigjuice 7y ago
    Seriously? Is it really that hard to figure out how to look at values of variables in general?
magv 7y ago
I have used this in one of my projects, but in the end had to abandon it: the problem is that Guile itself requires GNU make during build, which creates a circular dependency, and thus building it with e.g. FreeBSD ports required manual intervention every time you updated it. This combined with the fact that this integration is only available since GNU make 4.0, and some Linux distributions still have older make, made it a hassle to setup. This should improve over time, of course. Still a great feature though. Without it there are e.g. manipulations on the file paths that you can not do at all, or can do but only as a hack. Case in point: "GNU Make Standard Library" (https://gmsl.sourceforge.io/ https://gmsl.sourceforge.io/) is a collections of such hacks; here's its definition of 'strlen': strlen = $(call assert_no_dollar,$0,$1)$(strip $(eval __temp := $(subst $(__gmsl_space),x,$1))$(foreach a,$(__gmsl_characters),$(eval __temp := $$(subst $$a,x,$(__temp))))$(eval __temp := $(subst x,x ,$(__temp)))$(words $(__temp)))
- LukeShu 7y ago
  > only available since GNU make 4.0, and some Linux distributions still have older make, made it a hassle to setup. This should improve over time, of course. That feels like an outdated statement, make 4.0 came out in 2013. The only non-EOL system I can think of off the top of my head that isn't shipping >=4.0 is macOS (because they won't upgrade to anything GPLv3). However, unfortunately there's a popular CI system (cough CircleCI cough) that's still using Ubuntu 14.04 by default, which has been EOL for almost a year now. (And others, such as Travis-CI, used it by default right up until it was EOL). Oh, if you appreciate gmsl, you might also appreciate a book by the same author ('jgrahamc), The GNU Make Book. Some of the things in gmsl are certainly hacks, but a lot of it is quite reasonable if you start reading Make macros as "lisp, but with a bunch of extra dollar-signs and commas".
  - magv 7y ago
    Scientific Linux 6.10 has make 3.81, CentOS 7.7 has 3.82. These long-term-support distros have old software by their nature, and sometimes you're just stuck with one of them unfortunately.
  - nemetroid 7y ago
deng 7y ago
Guile integration in Make is super useful. I've used it on Windows to replace expensive shell calls with Guile functions and achieved tremendous speed-up. Building Make with Guile-support on Windows seems to be pretty complicated, but luckily ezwinports provides a native build, and it works great.
nrclark 7y ago
GNU Make's Guile integration is such a cool feature. The one hiccup with it is that isn't included in the default GNU Make package on any system I've ever used (although it can be installed as make-guile in a few distros). I'm guessing it's because Guile is around the same installed size as Python, and it needs a lot of external dependencies - one of which (the garbage-collection library) is a little oddball. A tool as foundational as Make probably doesn't want to pull in a huge dependency tree if avoidable. If we want to see Guile support added to Make by default, we'll need to reduce or eliminate Guile's external dependencies and slim down the install.
- LukeShu 7y ago
  It's enabled in the `make` package on Arch Linux. My Guile is around 40MB; my Python is around 80MB--both gargantuan compared to the 1.5MB Make.
- aijony 7y ago
  Is it cheating if we count GNU Guix?
- dfox 7y ago
  What exactly is "oddball" about BDW GC?
  - antonvs 7y ago
    What isn't oddball about it? It's designed to work as a garbage-collected replacement for C's malloc. That alone is enough to establish its oddballness. Any large system that makes the mistake of using it soon discovers its limitations. Not that it's a bad system per se - it achieves its design goals, it's just that its design goals are, to put it mildly, oddball. Basically, it's designed to allow systems that really should be written in a garbage-collected language, to kinda-sorta get away without that - up to a point.
    - dfox 7y ago
      You should not think of it as malloc replacement but as an GC that can work with native code that does not use fixed stack and register layout. That you can use such a thing for code generated by C compiler is side effect of that. In fact, there are various garbage collected runtimes with bespoke GC's that use similar implementation strategies. Edit: and one notable thing on BDW GC is its non-oddballness when it comes to building and/or packaging it. Also it is almost completely written in portable-ish C (it expects sane memory layout and relies on bunch of technically unspecified, but expected behaviors).
nrclark 7y ago
For those of us stuck working with vanilla GNU Make, you can still use $(shell) to accomplish a lot of what the Guile integration provides. As an example, string-length can farmed out by something like: strlen = $(shell printf "$1" | wc -c) When combined with $(eval), you can use the $(shell) command to do some pretty complex stuff in your Makefile. I've used this for doing checksum-based dependencies on one oddball build. It's not as nice as working with Guile, but it gets the job done in a pinch. And since the Guile interface doesn't have direct hooks into the DAG or anything, it doesn't really have access to tools that you couldn't provide by calling $(eval) on the result of a shell script.
PuercoPop 7y ago
Can someone who has used this featured give an example of what they used it for and how it improved their build system?
- opk 7y ago
  With a bit of creativity, you can achieve most things with the existing GNU make functions. Often the result is far from readable and relies on side-effects like sorting removing duplicates. But at least you don't need to worry about make being built with guile support. Guile does give you access to most system calls. The most interesting thing I've done with this is have the gmake process create a fifo, fork and exec tee in the child process to save a copy of the build output. It doesn't entirely work if gmake re-execs itself after building included makefiles. I've also tried doing a loadable module in C - to output timing to benchmark changes to my Makefile. I have a non-recursive setup which can take a few seconds at startup. The guile scripts are mostly limited to running during the initial make stage where dependencies are all gathered. You can't avoid the shell forks during the build process. There are also limitations in the strings being passed back to gmake so it didn't help greatly with colourising output when I tried that.
geofft 7y ago
For those of you who don't have make-guile, your GNU make probably still supports loadable modules in C (or your favorite C-ABI-compatible language): https://www.gnu.org/software/make/manual/html_node/Loading-Objects.html https://www.gnu.org/software/make/manual/html_node/Loading-O... I've been using that at work recently.
bhk 7y ago
If anyone is interested in using a Scheme-like language with Make 3.81, take a look at: https://github.com/bhk/scam https://github.com/bhk/scam

117 points by gebt 7y ago

I have been using it with great success, but there has been some bother with portability. I used it in ways that could have been done with make proper, but since I already know a lot of guile I reached for that.

jdc 7y ago
Which platform did Guile have trouble on?
- bjoli 7y ago
  Not cross-platform, but cross-computer. It isn't available everywhere, and is quite a fuzz to enable elsewhere.

jacobush 7y ago

Oh, like MSBuild with Powershell scripts. Each evil in their own ways.

snizovtsev 7y ago

make -p -f/dev/null | grep FEATURES

alex_stoddard 7y ago
Thank you kindly for providing the way to actually do the documented: "You can determine if make contains support for Guile by examining the .FEATURES variable; it will contain the word guile if Guile support is available." Unix - the documentation is excellent if you already know how to do it...
- sigjuice 7y ago
  Seriously? Is it really that hard to figure out how to look at values of variables in general?

magv 7y ago

I have used this in one of my projects, but in the end had to abandon it: the problem is that Guile itself requires GNU make during build, which creates a circular dependency, and thus building it with e.g. FreeBSD ports required manual intervention every time you updated it. This combined with the fact that this integration is only available since GNU make 4.0, and some Linux distributions still have older make, made it a hassle to setup. This should improve over time, of course. Still a great feature though. Without it there are e.g. manipulations on the file paths that you can not do at all, or can do but only as a hack. Case in point: "GNU Make Standard Library" (https://gmsl.sourceforge.io/ https://gmsl.sourceforge.io/) is a collections of such hacks; here's its definition of 'strlen': strlen = $(call assert_no_dollar,$0,$1)$(strip $(eval __temp := $(subst $(__gmsl_space),x,$1))$(foreach a,$(__gmsl_characters),$(eval __temp := $$(subst $$a,x,$(__temp))))$(eval __temp := $(subst x,x ,$(__temp)))$(words $(__temp)))

LukeShu 7y ago
> only available since GNU make 4.0, and some Linux distributions still have older make, made it a hassle to setup. This should improve over time, of course. That feels like an outdated statement, make 4.0 came out in 2013. The only non-EOL system I can think of off the top of my head that isn't shipping >=4.0 is macOS (because they won't upgrade to anything GPLv3). However, unfortunately there's a popular CI system (cough CircleCI cough) that's still using Ubuntu 14.04 by default, which has been EOL for almost a year now. (And others, such as Travis-CI, used it by default right up until it was EOL). Oh, if you appreciate gmsl, you might also appreciate a book by the same author ('jgrahamc), The GNU Make Book. Some of the things in gmsl are certainly hacks, but a lot of it is quite reasonable if you start reading Make macros as "lisp, but with a bunch of extra dollar-signs and commas".
- magv 7y ago
  Scientific Linux 6.10 has make 3.81, CentOS 7.7 has 3.82. These long-term-support distros have old software by their nature, and sometimes you're just stuck with one of them unfortunately.
- nemetroid 7y ago

deng 7y ago

Guile integration in Make is super useful. I've used it on Windows to replace expensive shell calls with Guile functions and achieved tremendous speed-up. Building Make with Guile-support on Windows seems to be pretty complicated, but luckily ezwinports provides a native build, and it works great.

nrclark 7y ago

GNU Make's Guile integration is such a cool feature. The one hiccup with it is that isn't included in the default GNU Make package on any system I've ever used (although it can be installed as make-guile in a few distros). I'm guessing it's because Guile is around the same installed size as Python, and it needs a lot of external dependencies - one of which (the garbage-collection library) is a little oddball. A tool as foundational as Make probably doesn't want to pull in a huge dependency tree if avoidable. If we want to see Guile support added to Make by default, we'll need to reduce or eliminate Guile's external dependencies and slim down the install.

LukeShu 7y ago
It's enabled in the `make` package on Arch Linux. My Guile is around 40MB; my Python is around 80MB--both gargantuan compared to the 1.5MB Make.
aijony 7y ago
Is it cheating if we count GNU Guix?
dfox 7y ago
What exactly is "oddball" about BDW GC?
- antonvs 7y ago
  What isn't oddball about it? It's designed to work as a garbage-collected replacement for C's malloc. That alone is enough to establish its oddballness. Any large system that makes the mistake of using it soon discovers its limitations. Not that it's a bad system per se - it achieves its design goals, it's just that its design goals are, to put it mildly, oddball. Basically, it's designed to allow systems that really should be written in a garbage-collected language, to kinda-sorta get away without that - up to a point.
  - dfox 7y ago
    You should not think of it as malloc replacement but as an GC that can work with native code that does not use fixed stack and register layout. That you can use such a thing for code generated by C compiler is side effect of that. In fact, there are various garbage collected runtimes with bespoke GC's that use similar implementation strategies. Edit: and one notable thing on BDW GC is its non-oddballness when it comes to building and/or packaging it. Also it is almost completely written in portable-ish C (it expects sane memory layout and relies on bunch of technically unspecified, but expected behaviors).

nrclark 7y ago

For those of us stuck working with vanilla GNU Make, you can still use $(shell) to accomplish a lot of what the Guile integration provides. As an example, string-length can farmed out by something like: strlen = $(shell printf "$1" | wc -c) When combined with $(eval), you can use the $(shell) command to do some pretty complex stuff in your Makefile. I've used this for doing checksum-based dependencies on one oddball build. It's not as nice as working with Guile, but it gets the job done in a pinch. And since the Guile interface doesn't have direct hooks into the DAG or anything, it doesn't really have access to tools that you couldn't provide by calling $(eval) on the result of a shell script.

PuercoPop 7y ago

Can someone who has used this featured give an example of what they used it for and how it improved their build system?

opk 7y ago
With a bit of creativity, you can achieve most things with the existing GNU make functions. Often the result is far from readable and relies on side-effects like sorting removing duplicates. But at least you don't need to worry about make being built with guile support. Guile does give you access to most system calls. The most interesting thing I've done with this is have the gmake process create a fifo, fork and exec tee in the child process to save a copy of the build output. It doesn't entirely work if gmake re-execs itself after building included makefiles. I've also tried doing a loadable module in C - to output timing to benchmark changes to my Makefile. I have a non-recursive setup which can take a few seconds at startup. The guile scripts are mostly limited to running during the initial make stage where dependencies are all gathered. You can't avoid the shell forks during the build process. There are also limitations in the strings being passed back to gmake so it didn't help greatly with colourising output when I tried that.

geofft 7y ago

For those of you who don't have make-guile, your GNU make probably still supports loadable modules in C (or your favorite C-ABI-compatible language): https://www.gnu.org/software/make/manual/html_node/Loading-Objects.html https://www.gnu.org/software/make/manual/html_node/Loading-O... I've been using that at work recently.

bhk 7y ago

If anyone is interested in using a Scheme-like language with Make 3.81, take a look at: https://github.com/bhk/scam https://github.com/bhk/scam