9 ms·

A Tutorial on Portable Makefiles (2017)

130 points by hardwaresofton 4y ago

hardwaresofton 4y ago
Also related & interesting: - https://frippery.org/make/ https://frippery.org/make/ (inspired this post -- HN: https://news.ycombinator.com/item?id=32300356 https://news.ycombinator.com/item?id=32300356) - https://github.com/casey/just https://github.com/casey/just (Make with less footguns, written in Rust) [EDIT] A side-note -- I personally use Make with just all stacks I create with (front/back/infra/etc). I think it's quite undervalued but maybe because I've never dealt with an industrial-strength terrible Makefile setup that would turn me off to it.
- tn1 4y ago
  My personal favorite: https://makepp.sourceforge.net https://makepp.sourceforge.net It's make, but works in the way you expect. Plus, with some extensions you can add custom functions to really make your life easier. We use it at $dayjob; at first I didn't like it, but eventually I learned to stop worrying and love it.
  - hardwaresofton 4y ago
    Thanks for sharing this! Evidently the HTTPS link is dead -- HTTP only: http://makepp.sourceforge.net/ http://makepp.sourceforge.net/ Straight to the tutorial for people ready to burn a couple hours figuring out if they should uproot: http://makepp.sourceforge.net/2.1/makepp_tutorial.html http://makepp.sourceforge.net/2.1/makepp_tutorial.html
Borkdude 4y ago
Check out babashka tasks. It's a take on Make for Clojure. https://book.babashka.org/#tasks https://book.babashka.org/#tasks Everything you do in there except shelling out will be portable, and if not, you can dispatch using the `fs/windows?` predicate.
wokwokwok 4y ago
2017 (when this is from) seems to me that it was around the time there was this weird resurgence of people claiming that “modern makefiles” (aka. carefully handcrafted makefiles) were fine. …but, it was a fad, and it faded, because make is not fine, in any non-trivial incarnation. Yep, sure. A little script for your projects that only you care about and runs on your machine… go for. I’m fond of make for trivial purposes, it’s easy to remember, it’s widely installed. …but, make doesn’t scale; the only way to scale make is to generate your makefiles programmatically; once you do this, your makefiles are no longer makefiles, they’re arcane generated non-portable custom build scripts. Various tools do this; autotools, cmake, etc. Sure, those are tools which are can use if you want, but the elegance and joy of using make vanishes once you use them; you are no longer using make, you are now using a different tool. …so, I have mixed feelings about posts like this. On the one hand, that’s cool; but on the other, you’re really wasting your time learning this stuff because, realistically, you’ll never use it. By the time you need it, you’ll be using a meta tool that generates human-unreadable Makefiles for you. Related, extensive thread: https://news.ycombinator.com/item?id=5275313 https://news.ycombinator.com/item?id=5275313
- sjaak 4y ago
  > …but, make doesn’t scale; What about these projects? https://github.com/redis/redis/blob/unstable/src/Makefile https://github.com/redis/redis/blob/unstable/src/Makefile https://github.com/torvalds/linux/blob/master/Makefile https://github.com/torvalds/linux/blob/master/Makefile
  - wokwokwok 4y ago
    Yes, I can also write a big Makefile to hand compile my project; I can also compile it using shell script that explicitly executes the build steps. …but I dont, not because it’s impossible to do but because the effort in maintaining those is high. Come on, are you serious? Hand crafted makefiles? Have you actually tried to maintain one of those? That redis makefile has a explicitly coded set of object files. Can you imagine trying to maintain that for opencv? I’m just too damn lazy to waste my time maintaining that kind of stuff; it’s a pointless, thankless task… but hey, I’ll give it to you; there are people out there who prefer to do it that way. I posit, they spend more time maintaining their build system then they would prefer, but they keep make for ideological reasons, not practical ones.
fao_ 4y ago
Of course my makefiles are portable, they fit on any kind of usb drive
- deleted 4y ago
  [deleted]
apatheticonion 4y ago
What is a makefile in 2022? I use them as cross-platform task runners with commands like "make run-tests" "make run-dev-server". I use it for Go, Rust and even JavaScript (omitting npm scripts). "make" is an odd keyword for such tasks but there really isn't another simple cross-platform task runner I can use.
- busfahrer 4y ago
  I use "just" for this purpose, and I'm very happy with it: https://github.com/casey/just https://github.com/casey/just
  - apatheticonion 4y ago
    This is great, thank you for the recommendation
- baliex 4y ago
  Same here. `make test` beats `./gradlew clean test` in one project and `pytest -s ...` in another, etc.
- rsclarke 4y ago
  https://taskfile.dev/ https://taskfile.dev/ has been my go to for this.
- walthamstow 4y ago
  Yep we use them to run tasks on AWS CodeBuild and Lambda machines. Make, like vi, has the advantage of just always being there already on whatever machine you need it on.
- efxhoy 4y ago
  Me too, I usually have a make “up”, “services” and “test” that do docker “compose up —detach", “docker compose down —volumes” and “rm -rf” temp dirs and run tests with whatever language i’m using. Saves a free brain cycle when switching languages. Sometimes a “data” target for downloading raw data from the internet for stuff like GIS projects. I never work in C or other languages that actually require the dependency management that make supports.
einpoklum 4y ago
The article itself mentions scalability issues; but beyond this - there are often many complications in Makefile construction which result from idiosyncracies of libraries, utilities, platform settings, tool versions, etc. etc. I used to write Makefiles - basically in the vein of this post's suggestions; but for several years have switched to using CMake. I have not had reason to look back. Why would I want to write Makefiles directly? What's the benefit?
deleted 4y ago
[deleted]
Syzygies 4y ago
Recovering an ancient project sent me back down the "which build tool should I be using?" rabbit hole. Getting a script to grave dig 57 inconsistently styled K&R C files (!) really stresses the build cycle, more than writing new code. First approximation: No one's happy with any build tool, with good reason. Then I discovered Ninja. It's an assembly language for builds, advertised as a back end for higher level tools, but many people script Ninja themselves. For me, 15 lines of Ruby code plus some heredoc boilerplate spawned a 138 line build.ninja file. Back in the day, this computer algebra system used to take an hour to build. Make still needs a frustrating number of seconds. Ninja on all my cores is a correct fraction of a second. Ninja is exceptionally fast, and handles parallelism extraordinarily well. It scales; they use it for Google Chrome. Various tools such as CMake can target Ninja. It is unique for its design goals. Most importantly, looking at Ninja code is not a "Just kill me now!" experience. https://ninja-build.org/ https://ninja-build.org/
- grlass 4y ago
  +1 for ninja. I first encountered is using Apache TVM, which uses CMake. I was first of all using the default make backend, and it was taking a while. Then the docs said to try adding the `-GNinja` flag to Cmake to build with ninja. I was blown away by how much faster the compilation was, and now try to use it whenever possible.
  - faho 4y ago
    One issue is that make will default to only using one job (i.e. one cpu), and you need to pass `-j NUMBER` to make it use more, while Ninja is parallel by default. For my uses, I've not found `ninja` to be much faster than `make -j8` on an 8-core machine. Not that the defaults don't matter, of course.
    - Syzygies 4y ago
      I'm surely also in that range. It took us years to write this project, but 57 C files is a small project. Others find themselves in the range where the build decision-making itself can take seconds, with tools other than ninja. Ninja is pared down for speed; having to declare keywords before I used them made me feel young again. I had to write these build files over again, and I just couldn't stomach the idea of writing another makefile.
cassepipe 4y ago
I personally use Xmake and try to advertise it every time I get the chance : No DSL it's just Lua, dead simple yet featureful, and it is ninja fast, or at least claims to be I never bothered to check that out, it's fast enough for me. https://xmake.io/#/ https://xmake.io/#/
- 1ark 4y ago
  Also a convert to xmake. Documentation is a bit hairy, but solvable. The code is easy to read and contribute to.
deleted 4y ago
[deleted]
jpe90 4y ago
I recently learned more about Makefiles by creating one to help me distribute a recent hobby project, which uses Lua embedded in C. I wish I had seen this when I was working on it! I learned (stole) from the musl libc configure script and makefile instead. Makefiles aren't as easy to work with as alternatives in other language ecosystems, but I found they let me do a lot without the need for extra dependencies. It's easy to shoot yourself in the foot with a configure script but the power is hard to beat. Coincidentally the author of this blog post commented on one of my reddit posts and was kind enough to help me work out a number of kinks in the project. https://jeskin.net/blog/clp/ https://jeskin.net/blog/clp/
orthoxerox 4y ago
I've been using this makefile to compile my C programs on Linux and Windows for several years: https://gist.github.com/orthoxerox/5ce97c72b03becfb433d923cf94424ed https://gist.github.com/orthoxerox/5ce97c72b03becfb433d923cf... As long as the program is just a bunch of C (and header) files with no special treatment, it works great.
benji-york 4y ago
If you like Make, but would like some additional functionality (tracing, debugger, profiling, etc.), check out remake: https://remake.readthedocs.io/en/latest/index.html https://remake.readthedocs.io/en/latest/index.html remake is a friendly fork of GNU Make that keeps up to date with recent releases but adds many niceties.
floor_ 4y ago
I feel like I'm in an alternate timeline where you include one .cpp file in a unity build and you're done while other people in their timeline are generating massive systems to do the same thing but worse. How did this become the preferred way. Its bonkers.
- acuozzo 4y ago
  Does Unity build for the 65C02S? In all seriousness, some of us write code targeting several different microcontrollers and many vendors decided to roll their own toolchain instead of just adding a new GCC backend. Build systems more elaborate than Make don't really help when your build comes down to invoking four different C compilers twelve different ways.
synergy20 4y ago
if you do cross platform(win,mac,linux), cmake is a good choice if you do linux 98% of the time then make is good enough. ninja does parallel build as good as 'make -j N' to me. meson, bazel are also good, but I don't have huge projects to leverage. all in all, make is what I use these days, solid, easy to read, get the job done.

Also related & interesting: - https://frippery.org/make/ https://frippery.org/make/ (inspired this post -- HN: https://news.ycombinator.com/item?id=32300356 https://news.ycombinator.com/item?id=32300356) - https://github.com/casey/just https://github.com/casey/just (Make with less footguns, written in Rust) [EDIT] A side-note -- I personally use Make with just all stacks I create with (front/back/infra/etc). I think it's quite undervalued but maybe because I've never dealt with an industrial-strength terrible Makefile setup that would turn me off to it.

tn1 4y ago
My personal favorite: https://makepp.sourceforge.net https://makepp.sourceforge.net It's make, but works in the way you expect. Plus, with some extensions you can add custom functions to really make your life easier. We use it at $dayjob; at first I didn't like it, but eventually I learned to stop worrying and love it.
- hardwaresofton 4y ago
  Thanks for sharing this! Evidently the HTTPS link is dead -- HTTP only: http://makepp.sourceforge.net/ http://makepp.sourceforge.net/ Straight to the tutorial for people ready to burn a couple hours figuring out if they should uproot: http://makepp.sourceforge.net/2.1/makepp_tutorial.html http://makepp.sourceforge.net/2.1/makepp_tutorial.html

Borkdude 4y ago

Check out babashka tasks. It's a take on Make for Clojure. https://book.babashka.org/#tasks https://book.babashka.org/#tasks Everything you do in there except shelling out will be portable, and if not, you can dispatch using the `fs/windows?` predicate.

wokwokwok 4y ago

2017 (when this is from) seems to me that it was around the time there was this weird resurgence of people claiming that “modern makefiles” (aka. carefully handcrafted makefiles) were fine. …but, it was a fad, and it faded, because make is not fine, in any non-trivial incarnation. Yep, sure. A little script for your projects that only you care about and runs on your machine… go for. I’m fond of make for trivial purposes, it’s easy to remember, it’s widely installed. …but, make doesn’t scale; the only way to scale make is to generate your makefiles programmatically; once you do this, your makefiles are no longer makefiles, they’re arcane generated non-portable custom build scripts. Various tools do this; autotools, cmake, etc. Sure, those are tools which are can use if you want, but the elegance and joy of using make vanishes once you use them; you are no longer using make, you are now using a different tool. …so, I have mixed feelings about posts like this. On the one hand, that’s cool; but on the other, you’re really wasting your time learning this stuff because, realistically, you’ll never use it. By the time you need it, you’ll be using a meta tool that generates human-unreadable Makefiles for you. Related, extensive thread: https://news.ycombinator.com/item?id=5275313 https://news.ycombinator.com/item?id=5275313

sjaak 4y ago
> …but, make doesn’t scale; What about these projects? https://github.com/redis/redis/blob/unstable/src/Makefile https://github.com/redis/redis/blob/unstable/src/Makefile https://github.com/torvalds/linux/blob/master/Makefile https://github.com/torvalds/linux/blob/master/Makefile
- wokwokwok 4y ago
  Yes, I can also write a big Makefile to hand compile my project; I can also compile it using shell script that explicitly executes the build steps. …but I dont, not because it’s impossible to do but because the effort in maintaining those is high. Come on, are you serious? Hand crafted makefiles? Have you actually tried to maintain one of those? That redis makefile has a explicitly coded set of object files. Can you imagine trying to maintain that for opencv? I’m just too damn lazy to waste my time maintaining that kind of stuff; it’s a pointless, thankless task… but hey, I’ll give it to you; there are people out there who prefer to do it that way. I posit, they spend more time maintaining their build system then they would prefer, but they keep make for ideological reasons, not practical ones.

fao_ 4y ago

Of course my makefiles are portable, they fit on any kind of usb drive

deleted 4y ago
[deleted]

apatheticonion 4y ago

What is a makefile in 2022? I use them as cross-platform task runners with commands like "make run-tests" "make run-dev-server". I use it for Go, Rust and even JavaScript (omitting npm scripts). "make" is an odd keyword for such tasks but there really isn't another simple cross-platform task runner I can use.

busfahrer 4y ago
I use "just" for this purpose, and I'm very happy with it: https://github.com/casey/just https://github.com/casey/just
- apatheticonion 4y ago
  This is great, thank you for the recommendation
baliex 4y ago
Same here. `make test` beats `./gradlew clean test` in one project and `pytest -s ...` in another, etc.
rsclarke 4y ago
https://taskfile.dev/ https://taskfile.dev/ has been my go to for this.
walthamstow 4y ago
Yep we use them to run tasks on AWS CodeBuild and Lambda machines. Make, like vi, has the advantage of just always being there already on whatever machine you need it on.
efxhoy 4y ago
Me too, I usually have a make “up”, “services” and “test” that do docker “compose up —detach", “docker compose down —volumes” and “rm -rf” temp dirs and run tests with whatever language i’m using. Saves a free brain cycle when switching languages. Sometimes a “data” target for downloading raw data from the internet for stuff like GIS projects. I never work in C or other languages that actually require the dependency management that make supports.

einpoklum 4y ago

The article itself mentions scalability issues; but beyond this - there are often many complications in Makefile construction which result from idiosyncracies of libraries, utilities, platform settings, tool versions, etc. etc. I used to write Makefiles - basically in the vein of this post's suggestions; but for several years have switched to using CMake. I have not had reason to look back. Why would I want to write Makefiles directly? What's the benefit?

deleted 4y ago

[deleted]

Syzygies 4y ago

Recovering an ancient project sent me back down the "which build tool should I be using?" rabbit hole. Getting a script to grave dig 57 inconsistently styled K&R C files (!) really stresses the build cycle, more than writing new code. First approximation: No one's happy with any build tool, with good reason. Then I discovered Ninja. It's an assembly language for builds, advertised as a back end for higher level tools, but many people script Ninja themselves. For me, 15 lines of Ruby code plus some heredoc boilerplate spawned a 138 line build.ninja file. Back in the day, this computer algebra system used to take an hour to build. Make still needs a frustrating number of seconds. Ninja on all my cores is a correct fraction of a second. Ninja is exceptionally fast, and handles parallelism extraordinarily well. It scales; they use it for Google Chrome. Various tools such as CMake can target Ninja. It is unique for its design goals. Most importantly, looking at Ninja code is not a "Just kill me now!" experience. https://ninja-build.org/ https://ninja-build.org/

grlass 4y ago
+1 for ninja. I first encountered is using Apache TVM, which uses CMake. I was first of all using the default make backend, and it was taking a while. Then the docs said to try adding the `-GNinja` flag to Cmake to build with ninja. I was blown away by how much faster the compilation was, and now try to use it whenever possible.
- faho 4y ago
  One issue is that make will default to only using one job (i.e. one cpu), and you need to pass `-j NUMBER` to make it use more, while Ninja is parallel by default. For my uses, I've not found `ninja` to be much faster than `make -j8` on an 8-core machine. Not that the defaults don't matter, of course.
  - Syzygies 4y ago
    I'm surely also in that range. It took us years to write this project, but 57 C files is a small project. Others find themselves in the range where the build decision-making itself can take seconds, with tools other than ninja. Ninja is pared down for speed; having to declare keywords before I used them made me feel young again. I had to write these build files over again, and I just couldn't stomach the idea of writing another makefile.

cassepipe 4y ago

I personally use Xmake and try to advertise it every time I get the chance : No DSL it's just Lua, dead simple yet featureful, and it is ninja fast, or at least claims to be I never bothered to check that out, it's fast enough for me. https://xmake.io/#/ https://xmake.io/#/

1ark 4y ago
Also a convert to xmake. Documentation is a bit hairy, but solvable. The code is easy to read and contribute to.

deleted 4y ago

[deleted]

jpe90 4y ago

I recently learned more about Makefiles by creating one to help me distribute a recent hobby project, which uses Lua embedded in C. I wish I had seen this when I was working on it! I learned (stole) from the musl libc configure script and makefile instead. Makefiles aren't as easy to work with as alternatives in other language ecosystems, but I found they let me do a lot without the need for extra dependencies. It's easy to shoot yourself in the foot with a configure script but the power is hard to beat. Coincidentally the author of this blog post commented on one of my reddit posts and was kind enough to help me work out a number of kinks in the project. https://jeskin.net/blog/clp/ https://jeskin.net/blog/clp/

orthoxerox 4y ago

I've been using this makefile to compile my C programs on Linux and Windows for several years: https://gist.github.com/orthoxerox/5ce97c72b03becfb433d923cf94424ed https://gist.github.com/orthoxerox/5ce97c72b03becfb433d923cf... As long as the program is just a bunch of C (and header) files with no special treatment, it works great.

benji-york 4y ago

If you like Make, but would like some additional functionality (tracing, debugger, profiling, etc.), check out remake: https://remake.readthedocs.io/en/latest/index.html https://remake.readthedocs.io/en/latest/index.html remake is a friendly fork of GNU Make that keeps up to date with recent releases but adds many niceties.

floor_ 4y ago

I feel like I'm in an alternate timeline where you include one .cpp file in a unity build and you're done while other people in their timeline are generating massive systems to do the same thing but worse. How did this become the preferred way. Its bonkers.

acuozzo 4y ago
Does Unity build for the 65C02S? In all seriousness, some of us write code targeting several different microcontrollers and many vendors decided to roll their own toolchain instead of just adding a new GCC backend. Build systems more elaborate than Make don't really help when your build comes down to invoking four different C compilers twelve different ways.

synergy20 4y ago

if you do cross platform(win,mac,linux), cmake is a good choice if you do linux 98% of the time then make is good enough. ninja does parallel build as good as 'make -j N' to me. meson, bazel are also good, but I don't have huge projects to leverage. all in all, make is what I use these days, solid, easy to read, get the job done.