3 ms·

I'm thinking before we build a CI/CI pipeline, make sure there is a Makefile. Why have they gone out of style?

by jas39 2y ago

ericyd 2y ago
Not sure but my guess is because they aren't a good fit for many languages. If you need a task runner then often languages will have a built in option or there are better alternatives than Make. If you need a build system then Make isn't a good fit for a lot of modern languages.
- SOLAR_FIELDS 2y ago
  I’ve been using earthly a lot lately and its general value prop is simple: it turns out that if Buildkit is your primary build tool that Make targets can almost always be represented as OCI image layers. The killer feature IMO is that its syntax is familiar enough to end users of both Make and Dockerfiles that engineers tend to be willing to onboard to it. A lot of these other solutions that use proprietary DSL’s struggle to cover every use case, and the implementations in turing complete typical language SDK approach often forces you into analysis paralysis if there is no existing pattern.
maccard 2y ago
If I want to build and test a golang app and push it to a container repository, what value does a makefile provide over go build && docker push? All the tools do their own dependency tracking already (unfortunately).
- drowsspa 2y ago
  Honestly I like it just to keep it as a command runner with the needed flags. Then in the unholy YAML there's just make build, make test, etc
  - pdimitar 2y ago
    The `just` tool is a better and much easier to understand command runner than `make`, however. Much less feature surface, too, which eliminates nasty surprises coming from the unnecessary complexity of `make`.
    - drowsspa 2y ago
      There are a pletora of tools better than Make. But it's a standard tool, everyone is familiar with it, you probably don't even have to install it.
      - pdimitar 2y ago
        `just` is 90% similar to `make` in syntax, only it has 100x less foot guns. :) Also I'll never understand the appeal of "not having to install a tool". We're not in the 1980s anymore when that was an actual chore. You run a command, the tool is there (including in CI/CD), boom, done. What am I missing here? The advantages you list are flimsy at best.
        maccard 2y ago
        Bootstrapping can be painful in some languages or frameworks. Not everyone is running containerised builds where there are ephemeral environments that you just install a tool (and pay the 30+ second cost per build to run apt-get update). There’s certainly value in having a front door entry point. But I think it should be a shell script, not a makefile.
        pdimitar 2y ago
        Yes to your last. Either sh/bash script or a precompiled Golang program. If installing a tool is really such a problem then having a precompiled strongly typed program doing various tasks should be a no-brainer. I started openly hating `make` because I re-learned its specifics and quirks several times over the course of 10-ish years and then figured that I want to learn stuff with a staying power in my brain. I don't use `make` every work day so eventually any quirks disappear -- that's how our brains work. So that's why I learned most of `just` and it hasn't betrayed me so far, not once. Though I did write a few Elixir and Golang programs for running various tasks in production environment, too.
- jas39 2y ago
  Maybe none, but at some point you may want to do other things at buildtime, such as generating an sqlite db or generate code stubs for protobuf. Having a universal, and highly refined, tool like Make will help developers without domain knowledge. It also does not exclude the use of other tool like Just and Docker. A Makefile is also an easy jump-off point for a build pipeline.
  - maccard 2y ago
    Then introduce it for those things. But a makefile to call go build, go test, docker push, Ecs update-services provides no value other than all of a sudden not working on windows without another tool.
    - 8n4vidtmkvmk 2y ago
      I use make on windows under WSL. I've mostly given up on getting everything to run under every OS. There's always some quirk.
- tom_ 2y ago
  You want to use GNU Make, and then you can ignore the Make dependency tracking. GNU Make is much easier when you only use so-called phony targets (consult the manual), which always execute without doing any dependency tracking. As for the advantage, a makefile will definitely perform both go build and docker push, rather than just (say) docker push, an ever-present risk if you have to rely on your fingers to type these things in, or rely on your eyes to check that you recalled the right command from the history. It will also explicitly tell you the build failed rather than relying on you to do echo $? or for the tools to have some obvious output in the error case. A shell script is also an option. Makefiles have some helpful extra features: by default, commands are echoed; by default, the build fails if any command exits with a non-0 exit code (please consult local Unix person for the details); a Makefile inherently has multiple entry points; and, a Makefile can also be easier to get working on Windows than a shell script, though if you can demand that people run it from Git Bash then maybe there's not much in this. If you're still not convinced: that's fine! This is not a sales pitch. (I've more recently switching to using a do-everything Python script for this stuff, which is more annoying when it comes to invoking commands but has obvious advantages in terms of how easy it is to add extra logic and checks and some UI creature comforts and so on.)
  - d0mine 2y ago
    fabfile (Fabric/invoke: https://www.pyinvoke.org https://www.pyinvoke.org) may be an alternative to .PHONY-heavy Makefile (shell commands glued together by Python code)
GrumpyCat42 2y ago
My struggle with Make and bash is that they're not very expressive - maybe that's something we want in our CIs, but I've always preferred writing an actual program in that program's native language for CI/CD, even if it has to shell out some commands every now and again.
- ttyprintk 2y ago
  I prefer that, too. I've heard (less-experienced) tech leads forbid Makefiles because they're not declarative enough compared to yaml.
  - a1o 2y ago
    If you stick with what is in common between ninja build and Makefiles, and comment any usage you do of what isn't, the file will mostly mention a series of inputs -> box -> outputs. What happens is make will dispatch in a way that the inputs are all satisfied. It works fine afaict, the only issue is make doesn't contain by itself the tools that it expects to be available in your environment, so you will still need something else to solve that.
    - 8n4vidtmkvmk 2y ago
      It's quirky, but you can sort of fake dependencies on tools in make too.
gwbas1c 2y ago
I touched make once in 1999, in school. The syntax was arcane, even by 1999 standards. > Why have they gone out of style? Because no modern toolchain uses make. Its syntax is so arcane that it's been replaced with various tools that are designed for the specific stack. Otherwise, more generic build systems use modern languages / markup.

I'm thinking before we build a CI/CI pipeline, make sure there is a Makefile. Why have they gone out of style?

by jas39 2y ago

I'm thinking before we build a CI/CI pipeline, make sure there is a Makefile. Why have they gone out of style?

Not sure but my guess is because they aren't a good fit for many languages. If you need a task runner then often languages will have a built in option or there are better alternatives than Make. If you need a build system then Make isn't a good fit for a lot of modern languages.

SOLAR_FIELDS 2y ago
I’ve been using earthly a lot lately and its general value prop is simple: it turns out that if Buildkit is your primary build tool that Make targets can almost always be represented as OCI image layers. The killer feature IMO is that its syntax is familiar enough to end users of both Make and Dockerfiles that engineers tend to be willing to onboard to it. A lot of these other solutions that use proprietary DSL’s struggle to cover every use case, and the implementations in turing complete typical language SDK approach often forces you into analysis paralysis if there is no existing pattern.

maccard 2y ago

If I want to build and test a golang app and push it to a container repository, what value does a makefile provide over go build && docker push? All the tools do their own dependency tracking already (unfortunately).

drowsspa 2y ago
Honestly I like it just to keep it as a command runner with the needed flags. Then in the unholy YAML there's just make build, make test, etc
- pdimitar 2y ago
  The `just` tool is a better and much easier to understand command runner than `make`, however. Much less feature surface, too, which eliminates nasty surprises coming from the unnecessary complexity of `make`.
  - drowsspa 2y ago
    There are a pletora of tools better than Make. But it's a standard tool, everyone is familiar with it, you probably don't even have to install it.
    - pdimitar 2y ago
      `just` is 90% similar to `make` in syntax, only it has 100x less foot guns. :) Also I'll never understand the appeal of "not having to install a tool". We're not in the 1980s anymore when that was an actual chore. You run a command, the tool is there (including in CI/CD), boom, done. What am I missing here? The advantages you list are flimsy at best.
      - maccard 2y ago
        Bootstrapping can be painful in some languages or frameworks. Not everyone is running containerised builds where there are ephemeral environments that you just install a tool (and pay the 30+ second cost per build to run apt-get update). There’s certainly value in having a front door entry point. But I think it should be a shell script, not a makefile.
        pdimitar 2y ago
        Yes to your last. Either sh/bash script or a precompiled Golang program. If installing a tool is really such a problem then having a precompiled strongly typed program doing various tasks should be a no-brainer. I started openly hating `make` because I re-learned its specifics and quirks several times over the course of 10-ish years and then figured that I want to learn stuff with a staying power in my brain. I don't use `make` every work day so eventually any quirks disappear -- that's how our brains work. So that's why I learned most of `just` and it hasn't betrayed me so far, not once. Though I did write a few Elixir and Golang programs for running various tasks in production environment, too.
jas39 2y ago
Maybe none, but at some point you may want to do other things at buildtime, such as generating an sqlite db or generate code stubs for protobuf. Having a universal, and highly refined, tool like Make will help developers without domain knowledge. It also does not exclude the use of other tool like Just and Docker. A Makefile is also an easy jump-off point for a build pipeline.
- maccard 2y ago
  Then introduce it for those things. But a makefile to call go build, go test, docker push, Ecs update-services provides no value other than all of a sudden not working on windows without another tool.
  - 8n4vidtmkvmk 2y ago
    I use make on windows under WSL. I've mostly given up on getting everything to run under every OS. There's always some quirk.
tom_ 2y ago
You want to use GNU Make, and then you can ignore the Make dependency tracking. GNU Make is much easier when you only use so-called phony targets (consult the manual), which always execute without doing any dependency tracking. As for the advantage, a makefile will definitely perform both go build and docker push, rather than just (say) docker push, an ever-present risk if you have to rely on your fingers to type these things in, or rely on your eyes to check that you recalled the right command from the history. It will also explicitly tell you the build failed rather than relying on you to do echo $? or for the tools to have some obvious output in the error case. A shell script is also an option. Makefiles have some helpful extra features: by default, commands are echoed; by default, the build fails if any command exits with a non-0 exit code (please consult local Unix person for the details); a Makefile inherently has multiple entry points; and, a Makefile can also be easier to get working on Windows than a shell script, though if you can demand that people run it from Git Bash then maybe there's not much in this. If you're still not convinced: that's fine! This is not a sales pitch. (I've more recently switching to using a do-everything Python script for this stuff, which is more annoying when it comes to invoking commands but has obvious advantages in terms of how easy it is to add extra logic and checks and some UI creature comforts and so on.)
- d0mine 2y ago
  fabfile (Fabric/invoke: https://www.pyinvoke.org https://www.pyinvoke.org) may be an alternative to .PHONY-heavy Makefile (shell commands glued together by Python code)

GrumpyCat42 2y ago

My struggle with Make and bash is that they're not very expressive - maybe that's something we want in our CIs, but I've always preferred writing an actual program in that program's native language for CI/CD, even if it has to shell out some commands every now and again.

ttyprintk 2y ago
I prefer that, too. I've heard (less-experienced) tech leads forbid Makefiles because they're not declarative enough compared to yaml.
- a1o 2y ago
  If you stick with what is in common between ninja build and Makefiles, and comment any usage you do of what isn't, the file will mostly mention a series of inputs -> box -> outputs. What happens is make will dispatch in a way that the inputs are all satisfied. It works fine afaict, the only issue is make doesn't contain by itself the tools that it expects to be available in your environment, so you will still need something else to solve that.
  - 8n4vidtmkvmk 2y ago
    It's quirky, but you can sort of fake dependencies on tools in make too.

gwbas1c 2y ago

I touched make once in 1999, in school. The syntax was arcane, even by 1999 standards. > Why have they gone out of style? Because no modern toolchain uses make. Its syntax is so arcane that it's been replaced with various tools that are designed for the specific stack. Otherwise, more generic build systems use modern languages / markup.