10 ms·

Learn Makefiles with the Tastiest Examples

170 points by ingve 4y ago

dijit 4y ago
Rant incoming; I really don’t like Make for the common use-cases it is being used for in the modern day, such as the Golang ecosystem or in many cases: creating docker files. Make, at its core, is a tool that is meant to help you structure your dependencies for incrementally building things, but people use it as a glorified task runner simply due to the fact that it’s such a common build tool that it is nearly always installed if you have a development toolchain on your machine. Go and docker already have incremental compilation built in, and docker doesn’t give definable artifacts so you can’t make other things depend on them either It is a powerful tool, but the syntax is hideous and has more jagged edges than bash does, and we aren’t even using it in a way that justifies this. Makes me so frustrated for some reason. Like everyone deciding to use a supercar to do farmwork.
- tyingq 4y ago
  Agree, though it can be handy for sort of non-standard dependency stuff. Like if you need to pull some thing with curl before a build, and you want "curl succeeded (exited zero)" to be a pre-requisite for that build. It's often less work to get a Makefile to do that than a script, or the make-like things that come with some other languages. Maybe not the best example, but if you have a project with several weird things like that, make is often easier.
- mongol 4y ago
  But is there not a "minimal Makefile" type of subset that you can use that appears really clean and tidy? My Makefiles are super basic and are basically task runners. Perhaps there are traps I have not fallen into but I value that I can count on them working on whatever distro I run in 5 years time.
  - assbuttbuttass 4y ago
    A shell script will also work in whatever distro in whatever years time
    - kevin_thibedeau 4y ago
      A distro running a Busybox userland doesn't necessarily support the same command options as one running GNU or BSD.
    - paulddraper 4y ago
      Yeah, but Make is parallel+incremental. And can do what a shell script can.
- LeBit 4y ago
  go-task is a very good task runner. Better than “just" IMO. I am now using go-task with every project. Right after creating the gitignore, I create the Taskfile. https://github.com/go-task/task https://github.com/go-task/task
  - leetrout 4y ago
    Second time that has been mentioned today. I wish it wasn't YAML.
    - fbdab103 4y ago
      You were not kidding, I am not loving that syntax.
  - dividedbyzero 4y ago
    > Better than “just" IMO. Could you elaborate on its advantages vs just?
- antipaul 4y ago
  That's a fine opinion, but I haven't found a more widely available tool to describe a project "recipe", where project is written in one or more, of many possible languages * Snakefile -> requires installation * bash -> macOS is now ZSH. (Is bash a better choice than Make?) * python -> what if I'm not using python? What else is as "accessible" as Makefile? EDIT: formatting
  - jjnoakes 4y ago
    The Makefile is going to be running external commands, no? So you still have the same problem (what are those external commands written in?) - whatever the answer to that question is, your task runner could be written in as well, probably. At least, that's how I prefer to do it. A project that's heavy JS? Use a JS task runner. Heavy python? Python task runner. Heavy shell? And so on.
    - alwillis 4y ago
      > A project that's heavy JS? Use a JS task runner. Trying to keep up with JS task runners for web projects is why I started using Make and haven’t looked back.
      - jjnoakes 3y ago
        Sure but if a developer already has a JS runtime and they don't have Make installed, you've added unnecessary friction to the development process. I hate JS as much as the next guy, but if I'm developing in JS, I use a JS task runner. Even a simple one. There's not much to "keep up with" IMO - they are quite easy to create and use, if the common/popular ones are missing features or move too fast for you.
    - taeric 4y ago
      I have seen few hells as bad as "just use the target runtime for build."
      - jjnoakes 3y ago
- Izkata 4y ago
  I use it as a task runner with dependencies in python projects. In particular, it checks requirements files against the virtualenv to see if anything needs to be installed/updated, then all the tasks depend on that. If you're up-to-date the task runs immediately, if you're not it'll update for you. > and docker doesn’t give definable artifacts so you can’t make other things depend on them either Similar to how old projects have "make configure" to do some initial setup before "make" actually builds the project, I've done stuff on occasion where something like "make check" would pull information out of a system and create timestamped files in a scratch/ directory. Then the normal "make" would compare those files to the codebase to see if the system needed to be updated. It is different from the first use I just mentioned, since you need two commands and it's not entirely automatic, but it's still simpler than checking each of the dependencies yourself.
  - silverwind 4y ago
    I too use Make to install `node_modules` and `.venv`. It is very convenient never having to ask oneself "Is my node_modules up to date?" after pulling or switching branches. I'll just let Make figure that out.
  - tpoacher 3y ago
    Make is great. But. In your specific case, you may also be interested to check out Luigi. https://luigi.readthedocs.io/en/stable/ https://luigi.readthedocs.io/en/stable/
- clintonc 4y ago
  True. I do, however, like having some of the amenities of a task runner without bringing in the "cruft" that's suited toward incremental builds. I like using Just (https://just.systems/ https://just.systems/) for those cases.
- evancordell 4y ago
  I have long held similar opinions on make, and I've recently started using mage[0] in more and more go projects and have been happy with the result. It's more task-oriented, the way people tend to write Makefiles with .PHONY rules, but it's all in go. It can be bootstrapped just with go too, and comes with some utilities to do make-like incremental builds if you need to. [0]: https://magefile.org/ https://magefile.org/
  - emptysongglass 4y ago
    How does it compare to Just?
    - evancordell 3y ago
      I looked at both and came away thinking mage was more convenient for go-only projects. Just looked good and I would probably pick it for something that wasn't go-only (if make didn't make sense instead).
  - tpoacher 4y ago
    Couldn't you have achieved this even more simply by using make with a go shell? Btw, from the linked page: > Makefiles are hard to read and hard to write. Mostly because makefiles are essentially fancy bash scripts with significant white space and additional make-related syntax. Wait what? What does bash have to do with anything? Mage may well be amazing, but it doesn't sound like this person knows make that well at all. Which makes me think they're simply trying to reinvent the wheel -- in 'go'.
    - evancordell 3y ago
      > Couldn't you have achieved this even more simply by using make with a go shell? Make is still really about file to file transformations, and `go` already wraps up all of the behavior one would normally use make for. Plus you need make + a shell + go, vs. mage where all that's needed is go. I can't speak for the author, but I assume they're reacting to how Makefiles tend to be used in go projects and not how make works generally.
- paulddraper 4y ago
  Hard disagree. GO_SRCS=$(shell find src -name '*.go' -not -name '*_test.go) GO_TESTS=$(shell find src -name '*_test.go') .PHONY: run docker-run: target/image.target docker run example .PHONY: integration-test integration-test: target/integration-test.target .PHONY: test test: target/test.target target/bin: $(GO_SRCS) mkdir -p $(@D) GOOS=linux-amd64 go build -ldflags='-s -w' -o $@ target/context.target: Dockerfile target/bin rm -fr $(@:.target=) mkdir -p $(@:.target=) cp Dockerfile target/bin $(@:.target=) target/image.target: target/context.target docker build -t example $(<:.target=) target/integration-test.target: target/image.target # TODO: test the docker image target/test.target: $(GO_SRCS) $(GO_TESTS) go test What is your better thing? (The answer is Bazel, but if you think Make is a "supercar".......)
  - rgoulter 4y ago
    Most (all?) of your Makefile targets don't represent actual files to 'make'. You're using make to run a bunch of tasks, including where tasks may depend on other tasks having been run before. A task-runner tool like `just` is better suited to this task. https://github.com/casey/just https://github.com/casey/just `just` has some nice UX improvements over `make`. (e.g. doesn't require soft tabs, can list recipes out of the box, recipes can take command line arguments, supports .env files, can be run from any subdirectory).
    - paulddraper 4y ago
      > Most (all?) of your Makefile targets don't represent actual files to 'make'. > You're using make to run a bunch of tasks, including where tasks may depend on other tasks having been run before. And the cool thing is....it works for both! It can represent an actual file (e.g. a compiled binary) or abstract result (e.g., a successful test).
      - deleted 4y ago
        [deleted]
    - therein 4y ago
- jimbob45 4y ago
  What’s your preferred alternative?
- zie 4y ago
  You are correct that Make is being abused because it's common. Me, I'm going to keep abusing Make vs bothering to find something supposedly better, that will then inject dependency startup problems I'd rather avoid. Yes, abusing Make the way I and many others do is not the ideal use case, but it works just fine. It's good enough at being a task manager.
- Groxx 4y ago
  Yeah. And it's wildly misunderstood too, so you get random people writing and running `make clean test` stuff that inherently disagrees with itself, which can do all kinds of nonsense if your system isn't normal/clean/running on Thursday. I do still use it for simple automation, because it's nice to have a language-agnostic way to do simple things. But once it grows beyond about a page of text it tends to become a real nightmare, and is nigh impossible for most people to help maintain... which is not at all helped by its absolute lack of clear best-practices or warnings when misconfigured, and poor meshing with many common systems (like source-modifying tools, e.g. gofmt). It doesn't help that every guide starts out with # so easy! thing: ./build thing When in reality you pretty much always need at least how this page ends, to be even slightly stable and maintainable: # The final build step. $(BUILD_DIR)/$(TARGET_EXEC): $(OBJS) $(CXX) $(OBJS) -o $@ $(LDFLAGS) # Build step for C source $(BUILD_DIR)/%.c.o: %.c mkdir -p $(dir $@) $(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@ # Build step for C++ source $(BUILD_DIR)/%.cpp.o: %.cpp mkdir -p $(dir $@) $(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $@ It's a horrifying bait-and-switch that a lot of people never fully learn their way through to the end.
  - paulddraper 4y ago
    FYI if you care to, you can prevent `make clean test` by looking at `$(MAKECMDGOALS)`.
    - Groxx 4y ago
      I have some copypasta that I stick in every makefile that has too many people abusing it blindly, yeah. It works but it's a real pain that there isn't a standard way to do it. Which is an immediate consequence of make being built to make things, not run tasks. So it goes.
    - radiospiel 4y ago
      What's the problem with "make clean test"? A test target would have a test-build prerequisite anyways, so - clean out build artefacts - rebuild test builds - run tests sounds very much ok to me.
- gigatexal 4y ago
  I take the other side of this and find that it being so useful its use as a task runner just means make is really awesome. Even when “misused” it’s super useful. That being said I really do like this article. I want to learn how to use make properly.
- r0l1 4y ago
  I agree. That's why we use the small grml tool as task/command runner in all our projects. https://github.com/desertbit/grml https://github.com/desertbit/grml
- magoon 4y ago
  I always include Makefiles as a way of documenting useful (short) tasks for developers that are onboarding to projects. It gives them confidence in their ability to pick things up quickly. Whether or not they want to actually use make is up to them, but it’s one more thing they can reference.
williamcotton 4y ago
Put this at the top of your Makefile for a node project: export PATH := node_modules/.bin:$(PATH)
alhirzel 4y ago
I end up using Makefiles for many projects that have even trivial stages in their execution (e.g. manual/one-off ETL pipelines). I do feel like I bastardize it sometimes as a convenient command runner or brain augmentation: sure makes it easy to remember how to run something if you can just type "make" or look at the top of the file, a cool intersection between "can get it working in 5 minutes" and "will be readable in 5 months". The Just command runner [1] is good, but is not as ubiquitous. [1]: https://github.com/casey/just https://github.com/casey/just
- francoismassot 4y ago
  I've got the same usage of Make and wanted to give Just a try. Which limitations did you identify with Just?
  - alhirzel 4y ago
    Pattern rules and not being on a zillion Linux boxes already. Maybe I just haven't seen a good idiom for pattern rules...
kstrauser 4y ago
I've started using justfiles (https://just.systems/man/en/ https://just.systems/man/en/) for everything I would've previously used Makefiles for that aren't directly related to compiling code. Make is a wonderful way to provide a standard set of entry points into working with repos. For instance, if you can standardize on `make setup`, `make test`, `make docker`, etc. on all of your organization's code, it's very easy for a new develop on a project to start contributing quickly. You could do that with shell scripts, but then you have to re-implement all the command line parsing that make gives you for free. And with justfiles, you can have something that looks and feels like make, but with a more pleasant syntax for non-compiling jobs. IMO, Just is to make as zsh/fish are to bash. It's not ubiquitously available, and it's not 100% backward compatible, but if you can convince your coworkers to adopt it as your standard, it makes life that much easier for everyone.
- roqi 4y ago
  > And with justfiles, you can have something that looks and feels like make, but with a more pleasant syntax for non-compiling jobs. Make is a part of UNIX, which ensures it is extremely stable and is available on all UNIX systems. You don't get that from justfiles. You get syntactic sugar, but all the best parts of Make are thrown out of the window.
  - kstrauser 4y ago
    OK, that's just not true. For starters, which make? GNU and BSD are common variants, and aren't 100% compatible. The overlap is large and useful, but does mean that you have to be aware of it if you're using multiple systems. For my uses, make offers nothing of value over just. I don't need any of the clever build dependency resolution stuff when I'm working in Python, and cargo handles everything better when I'm using Rust. Which isn't to say that make isn't enormously powerful and useful for other people, only that its extra power causes more work than reward for the ways I personally want to use it. For me, just is everything about make that I actually use, minus all the complicating features that get in my way.
    - roqi 4y ago
roqi 4y ago
Nit picking. From the article: > Popular C/C++ alternative build systems are SCons, CMake, Bazel, and Ninja. CMake is not a build system. It's a build system generator. A higher level abstraction over the problem of specifying projects, which greatly simplifies the work of creating and maintaining them. Once a CMake project is created, it is then used to create the actually build system. Originally it was mainly Makefile, but nowadays it also supports Ninja, Xcode, and Visual Studio. I'm sure that if there is any interest cmake could also be updated to generate other build systems.
- aidenn0 4y ago
  Arguably CMake is a build system; as a user you can build your project only by running CMake commands. The fact that it will run e.g. ninja or make under-the-hood can be ignored for many use-cases.
  - roqi 4y ago
    > Arguably CMake is a build system; as a user you can build your project only by running CMake commands. If you put together a shell script that calls some other program that runs a build, that doesn't make shell script a build system. > The fact that it will run e.g. ninja or make under-the-hood can be ignored for many use-cases. You can't, though. Some features are only available on projects generaded by cmake for specific build systems.
boerseth 4y ago
I recently used my blog as a way to learn how to use Make and Makefiles. I write posts in Markdown, and my Makefile generates HTML for each using Pandoc, then stitches it all together into one `index.html` file. The webpage behaves like a single-page-app, with a table-of-contents and everything, with just some CSS trickery. I know there are probably frameworks and tools that do this for me already, in ways both more flexible and robust than how I've done it, but I've really enjoyed the process of getting to this point. I started writing dumb shell scripts and struggling with weird cases and messy code, then learned more about Make and improved on things by a lot. In the end, it didn't take a lot of code, yet I feel like I can be really pleased with the result. I have come to really like Make, I have to say. It is probably used more often than it should, judging by some of the comments here. I've seen a few Makefiles where a substantial percentage of recipes are `.PHONY`, and maybe those are the cases where you should think about looking for a different task-runner? Not sure, but I enjoy Make anyway.
banku_brougham 4y ago
My favorite glorified taskrunner is https://just.systems/ https://just.systems/, using the `justfile` to formalize random task steps I do in local development.
antipaul 4y ago
This tutorial feels like its only for C++ (-like) coders, since it focuses on "(re)compiling" > Makefiles are used to help decide which parts of a large program need to be recompiled I've found uses for Makefiles, where users don't have a clue what "compiling" is. (Though other commenters may not necessarily appreciate such use cases!)
pimlottc 4y ago
When it said “tastiest”, I thought the example problem was going to be making a cake or something. Seems like a good tutorial regardless.
move-on-by 4y ago
Makefiles are great for what they were made for. Compiling a binary? You should consider using Make. I get frustrated when I start seeing tests and run commands hidden behind a make task. An example I saw recently was using a makefile to run docker compose. Simple enough, but it used a non-standard docker compose file, so it required the -f flag for every invocation. Which is fine since that’s handled for you by the makefile. Then it had docker compose run commands to invoke pytests. One key feature of running tests is passing arguments to limit test runs to specific packages/files/functions- but of course passing arguments down to a command isn’t supported by Make. All very convoluted and not the best tool for the job IMO. A simple bash script with “$@“ to pass my args to pytest is a lot more straightforward and allows pytest features to be used without needing to do anything extra.
- taeric 4y ago
  This is silly. Make was made for hierarchical tasks that can be done incrementally. This was largely about compiling things, sure. But, I'd wager all old make files have test targets. My favorite example in recentish times is http://ftp.cs.stanford.edu/pub/sgb/Makefile http://ftp.cs.stanford.edu/pub/sgb/Makefile. And yeah, has a test target. That docker compose and friend weren't good for building on top of is their "fault," as it were. Stability of software is not something that modern build tools really take to heart, sadly. Note that this is not "do not crash" stability. I cede that progress is progress, such that there may be better tools today. I'd hesitate to base anything on python's ecosystem, sadly. They jumped a hell of a shark in the past few years that will be tough to get trust back on.
- paulddraper 4y ago
  > but of course passing arguments down to a command isn’t supported by Make Sure it is. .PHONY: test test: pytest $TEST_FILTER make TEST_FILTER=example test
m463 4y ago
Make is extremely useful, but it is not simple or pretty. I honestly think instead of teaching make, it would be easier to refine it and then teach the simpler, more elegant make. honestly, there is SO much low-hanging fruit. - tabs vs spaces, indentation, the mess of continuing multiline statements - variables are inelegant, interpreted out-of-order, and = vs := - quoting maintains the mess of the rest of linux where you confuse a literal vs a special character - files, directories and targets (.PHONY) are confusing and lots more all the way down
- wanttocomment 4y ago
  Those are good complaints but what's far more valuable is: - Unix standard - documentation - established knowledge - existing code bases - stable and well tested
  - metaltyphoon 4y ago
    > Unix standard That’s enough? What about 1/2 - 1/3 of the devs on windows?
    - wanttocomment 4y ago
      Yes? Everyone benefits from a standard specification. Besides, most windows devs are using Ubuntu or cygwin. Or they just the ide for their environment and would never touch a CLI
woodruffw 4y ago
I've been using and writing Makefiles for over a decade and, ironically, my least favorite experiences with it are in the ecosystems (C and C++) it's ostensibly intended for. When using it for Python or multi-language projects (especially ones with lots of code generation or intermediate outputs), it's a joy: it's very satisfying to be able to write dependency patterns and relations and have the machine satisfy and parallelize them.
rubicks 4y ago
GNU Make is shockingly versatile and unreasonably effective. I have used Makefiles to build, check, and install projects in C, C++, golang, and python. I have abused Makefiles to generate websites, Docker images, PDFs (from LaTeX), and even macro-expanded yaml. It's old and weird and I love it.
- nequo 4y ago
  > It's old and weird and I love it. Likewise. And it has the added benefit of parallelizing those website and LaTeX builds according to the dependency graph by just adding “-j4” to the make call.
  - nephanth 4y ago
    What is there to parallelize in a latex build? Isn't it incremental?
    - porridgeandrice 4y ago
      I assume parallelizing latex builds of separate sections for their website. Like building out blogs/physics and blogs/cs from their latex equivalents in parallel.
    - nequo 4y ago
      Sometimes the LaTeX document contains many tables or figures that also need to be built. But those can be built in parallel.
yellowapple 4y ago
I resisted learning Make because of the clunkiness around it (Tab-indentation? C-specific features? Gross!), but after using it a few times now I've come to appreciate it. Most recent case was for Starbound mods. The macOS and Linux versions of Starbound don't ship with any built-in tools for uploading mods to the Steam Workshop (unlike Windows, which apparently has a dedicated GUI for it), and I didn't want to have to manually go through a bunch of SteamCMD steps every time I wanted to publish an update, so for my first real mod I put together a Makefile to automatically pack all the files together (using Starbound's included asset_packer CLI tool), populate a metadata.vdf for the Steam Workshop, and run the right steamcmd incantations to get everything up. Once that was reliably working, I condensed everything into a GitHub template repo (https://github.com/YellowApple/Starbound-Mod https://github.com/YellowApple/Starbound-Mod) and ended up with a pretty straightforward (albeit a bit clunky in spots) universal Makefile: MODNAME = StarboundMod ifeq "$(shell uname)" "Linux" PLATFORM = linux endif ifeq "$(shell uname)" "Darwin" PLATFORM = macos endif SB_PACKER = ../../$(PLATFORM)/asset_packer OUT = ./out SRC_DIRS = achievements ai animations behaviors biomes \ celestial cinematics codex collections \ cursors damage dialog dungeons effects \ humanoid interface items leveling liquids \ monsters music names npcs objects parallax \ particles plants player projectiles quests \ radiomessages recipes rendering scripts sfx \ ships sky spawntypes species stagehands stats \ tech tenants terrain tiles tilesets treasure \ vehicles versioning weather SRC_EXTS = config macros patch png wav STEAMCMD = steamcmd +login $(STEAMCMD_USER) STEAM_ID_PARSER = awk '/"publishedfileid"/{print $$2}' \ metadata.vdf | sed 's/"//g' $(OUT)/pkg/contents.pak: $(SRC_DIRS) $(SRC_EXTS) rm -rf $(OUT)/pkg mkdir -p $(OUT)/pkg $(SB_PACKER) $(OUT)/src $(OUT)/pkg/$(MODNAME).pak $(OUT)/src: rm -rf $(OUT)/src mkdir -p $(OUT)/src cp _metadata $(OUT)/src/ $(SRC_DIRS): FORCE $(OUT)/src @cp -rv $@ $(OUT)/src/$@ 2>/dev/null \ || echo 'No $@ folder present in mod root; skipping' $(SRC_EXTS): FORCE $(OUT)/src @cp -v *.$@ $(OUT)/src/ 2>/dev/null \ || echo 'No *.$@ files present in mod root; skipping' upload: $(OUT)/pkg/contents.pak mkdir -p $(OUT)/workshop cp $(OUT)/pkg/$(MODNAME).pak $(OUT)/workshop/contents.pak cp preview.jpg $(OUT)/preview.jpg sed 's,{{PWD}},$(PWD),g' <metadata.vdf.template >metadata.vdf $(STEAMCMD) +workshop_build_item $(PWD)/metadata.vdf +quit $(eval STEAM_ID=$(shell $(STEAM_ID_PARSER))) sed -i 's/"publishedfileid" "0"/"publishedfileid" "$(STEAM_ID)"/g' \ metadata.vdf.template clean: FORCE rm -rf $(OUT) rm -f metadata.vdf FORCE: All in all, it works well enough. Certainly nicer than trying to do it with shell scripts IMO, and I don't have to worry quite as much about whether someone else using my mod template (including my future self) is able to install some fancier build tool / task runner given how ubiquitous (GNU) Make tends to be.
nprateem 4y ago
If you're thinking of using it as a task runner, do yourself a favour and pick something else (eg just). Make has no end of gotchas it's a complete pain to work with.
tpoacher 4y ago
The make manual was one of the more useful and enjoyable technical reads I've done in a while (really!). I suspect that people going "bleh make" below have simply not read the manual. make is incredible and super-useful. much of what is used instead of make is simply reinventing the wheel. (badly).

170 points by ingve 4y ago

Rant incoming; I really don’t like Make for the common use-cases it is being used for in the modern day, such as the Golang ecosystem or in many cases: creating docker files. Make, at its core, is a tool that is meant to help you structure your dependencies for incrementally building things, but people use it as a glorified task runner simply due to the fact that it’s such a common build tool that it is nearly always installed if you have a development toolchain on your machine. Go and docker already have incremental compilation built in, and docker doesn’t give definable artifacts so you can’t make other things depend on them either It is a powerful tool, but the syntax is hideous and has more jagged edges than bash does, and we aren’t even using it in a way that justifies this. Makes me so frustrated for some reason. Like everyone deciding to use a supercar to do farmwork.

tyingq 4y ago
Agree, though it can be handy for sort of non-standard dependency stuff. Like if you need to pull some thing with curl before a build, and you want "curl succeeded (exited zero)" to be a pre-requisite for that build. It's often less work to get a Makefile to do that than a script, or the make-like things that come with some other languages. Maybe not the best example, but if you have a project with several weird things like that, make is often easier.
mongol 4y ago
But is there not a "minimal Makefile" type of subset that you can use that appears really clean and tidy? My Makefiles are super basic and are basically task runners. Perhaps there are traps I have not fallen into but I value that I can count on them working on whatever distro I run in 5 years time.
- assbuttbuttass 4y ago
  A shell script will also work in whatever distro in whatever years time
  - kevin_thibedeau 4y ago
    A distro running a Busybox userland doesn't necessarily support the same command options as one running GNU or BSD.
  - paulddraper 4y ago
    Yeah, but Make is parallel+incremental. And can do what a shell script can.
LeBit 4y ago
go-task is a very good task runner. Better than “just" IMO. I am now using go-task with every project. Right after creating the gitignore, I create the Taskfile. https://github.com/go-task/task https://github.com/go-task/task
- leetrout 4y ago
  Second time that has been mentioned today. I wish it wasn't YAML.
  - fbdab103 4y ago
    You were not kidding, I am not loving that syntax.
- dividedbyzero 4y ago
  > Better than “just" IMO. Could you elaborate on its advantages vs just?
antipaul 4y ago
That's a fine opinion, but I haven't found a more widely available tool to describe a project "recipe", where project is written in one or more, of many possible languages * Snakefile -> requires installation * bash -> macOS is now ZSH. (Is bash a better choice than Make?) * python -> what if I'm not using python? What else is as "accessible" as Makefile? EDIT: formatting
- jjnoakes 4y ago
  The Makefile is going to be running external commands, no? So you still have the same problem (what are those external commands written in?) - whatever the answer to that question is, your task runner could be written in as well, probably. At least, that's how I prefer to do it. A project that's heavy JS? Use a JS task runner. Heavy python? Python task runner. Heavy shell? And so on.
  - alwillis 4y ago
    > A project that's heavy JS? Use a JS task runner. Trying to keep up with JS task runners for web projects is why I started using Make and haven’t looked back.
    - jjnoakes 3y ago
      Sure but if a developer already has a JS runtime and they don't have Make installed, you've added unnecessary friction to the development process. I hate JS as much as the next guy, but if I'm developing in JS, I use a JS task runner. Even a simple one. There's not much to "keep up with" IMO - they are quite easy to create and use, if the common/popular ones are missing features or move too fast for you.
  - taeric 4y ago
    I have seen few hells as bad as "just use the target runtime for build."
    - jjnoakes 3y ago
Izkata 4y ago
I use it as a task runner with dependencies in python projects. In particular, it checks requirements files against the virtualenv to see if anything needs to be installed/updated, then all the tasks depend on that. If you're up-to-date the task runs immediately, if you're not it'll update for you. > and docker doesn’t give definable artifacts so you can’t make other things depend on them either Similar to how old projects have "make configure" to do some initial setup before "make" actually builds the project, I've done stuff on occasion where something like "make check" would pull information out of a system and create timestamped files in a scratch/ directory. Then the normal "make" would compare those files to the codebase to see if the system needed to be updated. It is different from the first use I just mentioned, since you need two commands and it's not entirely automatic, but it's still simpler than checking each of the dependencies yourself.
- silverwind 4y ago
  I too use Make to install `node_modules` and `.venv`. It is very convenient never having to ask oneself "Is my node_modules up to date?" after pulling or switching branches. I'll just let Make figure that out.
- tpoacher 3y ago
  Make is great. But. In your specific case, you may also be interested to check out Luigi. https://luigi.readthedocs.io/en/stable/ https://luigi.readthedocs.io/en/stable/
clintonc 4y ago
True. I do, however, like having some of the amenities of a task runner without bringing in the "cruft" that's suited toward incremental builds. I like using Just (https://just.systems/ https://just.systems/) for those cases.
evancordell 4y ago
I have long held similar opinions on make, and I've recently started using mage[0] in more and more go projects and have been happy with the result. It's more task-oriented, the way people tend to write Makefiles with .PHONY rules, but it's all in go. It can be bootstrapped just with go too, and comes with some utilities to do make-like incremental builds if you need to. [0]: https://magefile.org/ https://magefile.org/
- emptysongglass 4y ago
  How does it compare to Just?
  - evancordell 3y ago
    I looked at both and came away thinking mage was more convenient for go-only projects. Just looked good and I would probably pick it for something that wasn't go-only (if make didn't make sense instead).
- tpoacher 4y ago
  Couldn't you have achieved this even more simply by using make with a go shell? Btw, from the linked page: > Makefiles are hard to read and hard to write. Mostly because makefiles are essentially fancy bash scripts with significant white space and additional make-related syntax. Wait what? What does bash have to do with anything? Mage may well be amazing, but it doesn't sound like this person knows make that well at all. Which makes me think they're simply trying to reinvent the wheel -- in 'go'.
  - evancordell 3y ago
    > Couldn't you have achieved this even more simply by using make with a go shell? Make is still really about file to file transformations, and `go` already wraps up all of the behavior one would normally use make for. Plus you need make + a shell + go, vs. mage where all that's needed is go. I can't speak for the author, but I assume they're reacting to how Makefiles tend to be used in go projects and not how make works generally.
paulddraper 4y ago
Hard disagree. GO_SRCS=$(shell find src -name '*.go' -not -name '*_test.go) GO_TESTS=$(shell find src -name '*_test.go') .PHONY: run docker-run: target/image.target docker run example .PHONY: integration-test integration-test: target/integration-test.target .PHONY: test test: target/test.target target/bin: $(GO_SRCS) mkdir -p $(@D) GOOS=linux-amd64 go build -ldflags='-s -w' -o $@ target/context.target: Dockerfile target/bin rm -fr $(@:.target=) mkdir -p $(@:.target=) cp Dockerfile target/bin $(@:.target=) target/image.target: target/context.target docker build -t example $(<:.target=) target/integration-test.target: target/image.target # TODO: test the docker image target/test.target: $(GO_SRCS) $(GO_TESTS) go test What is your better thing? (The answer is Bazel, but if you think Make is a "supercar".......)
- rgoulter 4y ago
  Most (all?) of your Makefile targets don't represent actual files to 'make'. You're using make to run a bunch of tasks, including where tasks may depend on other tasks having been run before. A task-runner tool like `just` is better suited to this task. https://github.com/casey/just https://github.com/casey/just `just` has some nice UX improvements over `make`. (e.g. doesn't require soft tabs, can list recipes out of the box, recipes can take command line arguments, supports .env files, can be run from any subdirectory).
  - paulddraper 4y ago
    > Most (all?) of your Makefile targets don't represent actual files to 'make'. > You're using make to run a bunch of tasks, including where tasks may depend on other tasks having been run before. And the cool thing is....it works for both! It can represent an actual file (e.g. a compiled binary) or abstract result (e.g., a successful test).
    - deleted 4y ago
      [deleted]
  - therein 4y ago
jimbob45 4y ago
What’s your preferred alternative?
zie 4y ago
You are correct that Make is being abused because it's common. Me, I'm going to keep abusing Make vs bothering to find something supposedly better, that will then inject dependency startup problems I'd rather avoid. Yes, abusing Make the way I and many others do is not the ideal use case, but it works just fine. It's good enough at being a task manager.
Groxx 4y ago
Yeah. And it's wildly misunderstood too, so you get random people writing and running `make clean test` stuff that inherently disagrees with itself, which can do all kinds of nonsense if your system isn't normal/clean/running on Thursday. I do still use it for simple automation, because it's nice to have a language-agnostic way to do simple things. But once it grows beyond about a page of text it tends to become a real nightmare, and is nigh impossible for most people to help maintain... which is not at all helped by its absolute lack of clear best-practices or warnings when misconfigured, and poor meshing with many common systems (like source-modifying tools, e.g. gofmt). It doesn't help that every guide starts out with # so easy! thing: ./build thing When in reality you pretty much always need at least how this page ends, to be even slightly stable and maintainable: # The final build step. $(BUILD_DIR)/$(TARGET_EXEC): $(OBJS) $(CXX) $(OBJS) -o $@ $(LDFLAGS) # Build step for C source $(BUILD_DIR)/%.c.o: %.c mkdir -p $(dir $@) $(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@ # Build step for C++ source $(BUILD_DIR)/%.cpp.o: %.cpp mkdir -p $(dir $@) $(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $@ It's a horrifying bait-and-switch that a lot of people never fully learn their way through to the end.
- paulddraper 4y ago
  FYI if you care to, you can prevent `make clean test` by looking at `$(MAKECMDGOALS)`.
  - Groxx 4y ago
    I have some copypasta that I stick in every makefile that has too many people abusing it blindly, yeah. It works but it's a real pain that there isn't a standard way to do it. Which is an immediate consequence of make being built to make things, not run tasks. So it goes.
  - radiospiel 4y ago
    What's the problem with "make clean test"? A test target would have a test-build prerequisite anyways, so - clean out build artefacts - rebuild test builds - run tests sounds very much ok to me.
gigatexal 4y ago
I take the other side of this and find that it being so useful its use as a task runner just means make is really awesome. Even when “misused” it’s super useful. That being said I really do like this article. I want to learn how to use make properly.
r0l1 4y ago
I agree. That's why we use the small grml tool as task/command runner in all our projects. https://github.com/desertbit/grml https://github.com/desertbit/grml
magoon 4y ago
I always include Makefiles as a way of documenting useful (short) tasks for developers that are onboarding to projects. It gives them confidence in their ability to pick things up quickly. Whether or not they want to actually use make is up to them, but it’s one more thing they can reference.

williamcotton 4y ago

Put this at the top of your Makefile for a node project: export PATH := node_modules/.bin:$(PATH)

alhirzel 4y ago

I end up using Makefiles for many projects that have even trivial stages in their execution (e.g. manual/one-off ETL pipelines). I do feel like I bastardize it sometimes as a convenient command runner or brain augmentation: sure makes it easy to remember how to run something if you can just type "make" or look at the top of the file, a cool intersection between "can get it working in 5 minutes" and "will be readable in 5 months". The Just command runner [1] is good, but is not as ubiquitous. [1]: https://github.com/casey/just https://github.com/casey/just

francoismassot 4y ago
I've got the same usage of Make and wanted to give Just a try. Which limitations did you identify with Just?
- alhirzel 4y ago
  Pattern rules and not being on a zillion Linux boxes already. Maybe I just haven't seen a good idiom for pattern rules...

kstrauser 4y ago

I've started using justfiles (https://just.systems/man/en/ https://just.systems/man/en/) for everything I would've previously used Makefiles for that aren't directly related to compiling code. Make is a wonderful way to provide a standard set of entry points into working with repos. For instance, if you can standardize on `make setup`, `make test`, `make docker`, etc. on all of your organization's code, it's very easy for a new develop on a project to start contributing quickly. You could do that with shell scripts, but then you have to re-implement all the command line parsing that make gives you for free. And with justfiles, you can have something that looks and feels like make, but with a more pleasant syntax for non-compiling jobs. IMO, Just is to make as zsh/fish are to bash. It's not ubiquitously available, and it's not 100% backward compatible, but if you can convince your coworkers to adopt it as your standard, it makes life that much easier for everyone.

roqi 4y ago
> And with justfiles, you can have something that looks and feels like make, but with a more pleasant syntax for non-compiling jobs. Make is a part of UNIX, which ensures it is extremely stable and is available on all UNIX systems. You don't get that from justfiles. You get syntactic sugar, but all the best parts of Make are thrown out of the window.
- kstrauser 4y ago
  OK, that's just not true. For starters, which make? GNU and BSD are common variants, and aren't 100% compatible. The overlap is large and useful, but does mean that you have to be aware of it if you're using multiple systems. For my uses, make offers nothing of value over just. I don't need any of the clever build dependency resolution stuff when I'm working in Python, and cargo handles everything better when I'm using Rust. Which isn't to say that make isn't enormously powerful and useful for other people, only that its extra power causes more work than reward for the ways I personally want to use it. For me, just is everything about make that I actually use, minus all the complicating features that get in my way.
  - roqi 4y ago

roqi 4y ago

Nit picking. From the article: > Popular C/C++ alternative build systems are SCons, CMake, Bazel, and Ninja. CMake is not a build system. It's a build system generator. A higher level abstraction over the problem of specifying projects, which greatly simplifies the work of creating and maintaining them. Once a CMake project is created, it is then used to create the actually build system. Originally it was mainly Makefile, but nowadays it also supports Ninja, Xcode, and Visual Studio. I'm sure that if there is any interest cmake could also be updated to generate other build systems.

aidenn0 4y ago
Arguably CMake is a build system; as a user you can build your project only by running CMake commands. The fact that it will run e.g. ninja or make under-the-hood can be ignored for many use-cases.
- roqi 4y ago
  > Arguably CMake is a build system; as a user you can build your project only by running CMake commands. If you put together a shell script that calls some other program that runs a build, that doesn't make shell script a build system. > The fact that it will run e.g. ninja or make under-the-hood can be ignored for many use-cases. You can't, though. Some features are only available on projects generaded by cmake for specific build systems.

boerseth 4y ago

I recently used my blog as a way to learn how to use Make and Makefiles. I write posts in Markdown, and my Makefile generates HTML for each using Pandoc, then stitches it all together into one `index.html` file. The webpage behaves like a single-page-app, with a table-of-contents and everything, with just some CSS trickery. I know there are probably frameworks and tools that do this for me already, in ways both more flexible and robust than how I've done it, but I've really enjoyed the process of getting to this point. I started writing dumb shell scripts and struggling with weird cases and messy code, then learned more about Make and improved on things by a lot. In the end, it didn't take a lot of code, yet I feel like I can be really pleased with the result. I have come to really like Make, I have to say. It is probably used more often than it should, judging by some of the comments here. I've seen a few Makefiles where a substantial percentage of recipes are `.PHONY`, and maybe those are the cases where you should think about looking for a different task-runner? Not sure, but I enjoy Make anyway.

banku_brougham 4y ago

My favorite glorified taskrunner is https://just.systems/ https://just.systems/, using the `justfile` to formalize random task steps I do in local development.

antipaul 4y ago

This tutorial feels like its only for C++ (-like) coders, since it focuses on "(re)compiling" > Makefiles are used to help decide which parts of a large program need to be recompiled I've found uses for Makefiles, where users don't have a clue what "compiling" is. (Though other commenters may not necessarily appreciate such use cases!)

pimlottc 4y ago

When it said “tastiest”, I thought the example problem was going to be making a cake or something. Seems like a good tutorial regardless.

move-on-by 4y ago

Makefiles are great for what they were made for. Compiling a binary? You should consider using Make. I get frustrated when I start seeing tests and run commands hidden behind a make task. An example I saw recently was using a makefile to run docker compose. Simple enough, but it used a non-standard docker compose file, so it required the -f flag for every invocation. Which is fine since that’s handled for you by the makefile. Then it had docker compose run commands to invoke pytests. One key feature of running tests is passing arguments to limit test runs to specific packages/files/functions- but of course passing arguments down to a command isn’t supported by Make. All very convoluted and not the best tool for the job IMO. A simple bash script with “$@“ to pass my args to pytest is a lot more straightforward and allows pytest features to be used without needing to do anything extra.

taeric 4y ago
This is silly. Make was made for hierarchical tasks that can be done incrementally. This was largely about compiling things, sure. But, I'd wager all old make files have test targets. My favorite example in recentish times is http://ftp.cs.stanford.edu/pub/sgb/Makefile http://ftp.cs.stanford.edu/pub/sgb/Makefile. And yeah, has a test target. That docker compose and friend weren't good for building on top of is their "fault," as it were. Stability of software is not something that modern build tools really take to heart, sadly. Note that this is not "do not crash" stability. I cede that progress is progress, such that there may be better tools today. I'd hesitate to base anything on python's ecosystem, sadly. They jumped a hell of a shark in the past few years that will be tough to get trust back on.
paulddraper 4y ago
> but of course passing arguments down to a command isn’t supported by Make Sure it is. .PHONY: test test: pytest $TEST_FILTER make TEST_FILTER=example test

m463 4y ago

Make is extremely useful, but it is not simple or pretty. I honestly think instead of teaching make, it would be easier to refine it and then teach the simpler, more elegant make. honestly, there is SO much low-hanging fruit. - tabs vs spaces, indentation, the mess of continuing multiline statements - variables are inelegant, interpreted out-of-order, and = vs := - quoting maintains the mess of the rest of linux where you confuse a literal vs a special character - files, directories and targets (.PHONY) are confusing and lots more all the way down

wanttocomment 4y ago
Those are good complaints but what's far more valuable is: - Unix standard - documentation - established knowledge - existing code bases - stable and well tested
- metaltyphoon 4y ago
  > Unix standard That’s enough? What about 1/2 - 1/3 of the devs on windows?
  - wanttocomment 4y ago
    Yes? Everyone benefits from a standard specification. Besides, most windows devs are using Ubuntu or cygwin. Or they just the ide for their environment and would never touch a CLI

woodruffw 4y ago

I've been using and writing Makefiles for over a decade and, ironically, my least favorite experiences with it are in the ecosystems (C and C++) it's ostensibly intended for. When using it for Python or multi-language projects (especially ones with lots of code generation or intermediate outputs), it's a joy: it's very satisfying to be able to write dependency patterns and relations and have the machine satisfy and parallelize them.

rubicks 4y ago

GNU Make is shockingly versatile and unreasonably effective. I have used Makefiles to build, check, and install projects in C, C++, golang, and python. I have abused Makefiles to generate websites, Docker images, PDFs (from LaTeX), and even macro-expanded yaml. It's old and weird and I love it.

nequo 4y ago
> It's old and weird and I love it. Likewise. And it has the added benefit of parallelizing those website and LaTeX builds according to the dependency graph by just adding “-j4” to the make call.
- nephanth 4y ago
  What is there to parallelize in a latex build? Isn't it incremental?
  - porridgeandrice 4y ago
    I assume parallelizing latex builds of separate sections for their website. Like building out blogs/physics and blogs/cs from their latex equivalents in parallel.
  - nequo 4y ago
    Sometimes the LaTeX document contains many tables or figures that also need to be built. But those can be built in parallel.

yellowapple 4y ago

I resisted learning Make because of the clunkiness around it (Tab-indentation? C-specific features? Gross!), but after using it a few times now I've come to appreciate it. Most recent case was for Starbound mods. The macOS and Linux versions of Starbound don't ship with any built-in tools for uploading mods to the Steam Workshop (unlike Windows, which apparently has a dedicated GUI for it), and I didn't want to have to manually go through a bunch of SteamCMD steps every time I wanted to publish an update, so for my first real mod I put together a Makefile to automatically pack all the files together (using Starbound's included asset_packer CLI tool), populate a metadata.vdf for the Steam Workshop, and run the right steamcmd incantations to get everything up. Once that was reliably working, I condensed everything into a GitHub template repo (https://github.com/YellowApple/Starbound-Mod https://github.com/YellowApple/Starbound-Mod) and ended up with a pretty straightforward (albeit a bit clunky in spots) universal Makefile: MODNAME = StarboundMod ifeq "$(shell uname)" "Linux" PLATFORM = linux endif ifeq "$(shell uname)" "Darwin" PLATFORM = macos endif SB_PACKER = ../../$(PLATFORM)/asset_packer OUT = ./out SRC_DIRS = achievements ai animations behaviors biomes \ celestial cinematics codex collections \ cursors damage dialog dungeons effects \ humanoid interface items leveling liquids \ monsters music names npcs objects parallax \ particles plants player projectiles quests \ radiomessages recipes rendering scripts sfx \ ships sky spawntypes species stagehands stats \ tech tenants terrain tiles tilesets treasure \ vehicles versioning weather SRC_EXTS = config macros patch png wav STEAMCMD = steamcmd +login $(STEAMCMD_USER) STEAM_ID_PARSER = awk '/"publishedfileid"/{print $$2}' \ metadata.vdf | sed 's/"//g' $(OUT)/pkg/contents.pak: $(SRC_DIRS) $(SRC_EXTS) rm -rf $(OUT)/pkg mkdir -p $(OUT)/pkg $(SB_PACKER) $(OUT)/src $(OUT)/pkg/$(MODNAME).pak $(OUT)/src: rm -rf $(OUT)/src mkdir -p $(OUT)/src cp _metadata $(OUT)/src/ $(SRC_DIRS): FORCE $(OUT)/src @cp -rv $@ $(OUT)/src/$@ 2>/dev/null \ || echo 'No $@ folder present in mod root; skipping' $(SRC_EXTS): FORCE $(OUT)/src @cp -v *.$@ $(OUT)/src/ 2>/dev/null \ || echo 'No *.$@ files present in mod root; skipping' upload: $(OUT)/pkg/contents.pak mkdir -p $(OUT)/workshop cp $(OUT)/pkg/$(MODNAME).pak $(OUT)/workshop/contents.pak cp preview.jpg $(OUT)/preview.jpg sed 's,{{PWD}},$(PWD),g' <metadata.vdf.template >metadata.vdf $(STEAMCMD) +workshop_build_item $(PWD)/metadata.vdf +quit $(eval STEAM_ID=$(shell $(STEAM_ID_PARSER))) sed -i 's/"publishedfileid" "0"/"publishedfileid" "$(STEAM_ID)"/g' \ metadata.vdf.template clean: FORCE rm -rf $(OUT) rm -f metadata.vdf FORCE: All in all, it works well enough. Certainly nicer than trying to do it with shell scripts IMO, and I don't have to worry quite as much about whether someone else using my mod template (including my future self) is able to install some fancier build tool / task runner given how ubiquitous (GNU) Make tends to be.

nprateem 4y ago

If you're thinking of using it as a task runner, do yourself a favour and pick something else (eg just). Make has no end of gotchas it's a complete pain to work with.

tpoacher 4y ago

The make manual was one of the more useful and enjoyable technical reads I've done in a while (really!). I suspect that people going "bleh make" below have simply not read the manual. make is incredible and super-useful. much of what is used instead of make is simply reinventing the wheel. (badly).