5 ms·

Ok, since we're sharing controversial opinions... If you're going to rely on a fancy shell anyway, why not just throw make out of the loop altogether? That is

by clarry 7y ago

Ok, since we're sharing controversial opinions... If you're going to rely on a fancy shell anyway, why not just throw make out of the loop altogether? That is, unless you're working on a big project where incremental builds really make a difference. (But IME, with a few exceptions, these are the projects that usually outgrow and abandon make anyway) You can run cc via shell via make, or you can just run cc via shell. In the latter case, there's one less program (with quirks) to fight with, and more flexibility to do stuff that you can't easily bend make to do.

lloeki 7y ago
Because the core proposition value of make is the dependency graph and incremental rebuild, which then trivially allows for parallelisation with -j<n>, and doing that with shell scripts is basically reimplementing that part of make, only badly. Instead of pushing for makefiles to become shell scripts for the part you mention, I usually implement such things as support scripts (in shell, python, whatever) - who often come in useful on their own - and call those from make targets. IOW using the best tool for each job, with make as the orchestrator.
- clarry 7y ago
  > Because the core proposition value of make is the dependency graph, which then trivially allows for parallelisation with -j<n> My core argument is that this rarely matters, and in the cases it does, it is often the case that the project's needs surpass what make provides anyway. Or they're using a very special version of make with very special supporting infrastructure, like bsd.mk. I'm compiling some 100k lines of legacy C (for a custom operating system) plus compat shims that make the thing run on Linux and my very very trivial build script (plain posix compatible shell) takes less than 5 seconds to run in debug mode. Optimized release builds are a bit slower, but this hardly matters in day to day development. It would be trivial to add some parallelism with xargs. Fwiw I can compile the same thing with CMake, and it's faster only sometimes. It often happens that a header changes and everything needs to be built anyway. And it often happens that the parallelism bites me back when one file gives an error which scrolls out of sight as a dozen other files are still being compiled in parallel. I've wasted far more time fighting subtly broken makefiles (that don't pick up some dependencies and thus fail to recompile the things that need to be recompiled), and build systems with fancy declarative languages that don't present an obvious way to do what you do in one line of shell when you don't need to worry about a dependency graph.
  - MaxBarraclough 7y ago
    Personally I see CMake as the best way to work with portable C/C++ these days, as it supports Linux/Mac/Windows, including library-detection and everything else a traditional autotools "./configure" script does. That said, I'm not a fan of the CMake scripting language, which is quirky and error-prone. It should be easy to write a new 'Find' module, but it's not. There should be officially documented design patterns to make the task trivial. There still aren't. CMake gets the job done, but it's surprisingly difficult to work with. </rant>
    - irishcoffee 7y ago
      I really want to like CMake, but having spent the past 5 years using qmake daily, CMake seems very voodoo-ish and archaic.
      - pferde 7y ago
        Too bad, Qt project itself is moving to CMake as its main build tool: https://bugreports.qt.io/browse/QTBUG-73351 https://bugreports.qt.io/browse/QTBUG-73351
        irishcoffee 7y ago
        I'm aware.
    - hvdijk 7y ago
      Not everything else. One particular thing cmake does not do well that bugs me quite a lot is supporting source code generators, especially in cross compilations, where those source code generators are themselves written in C or some other language requiring compilation. With Makefiles, whether with autotools or manually written ones, this is easy to set up with a few custom rules. With cmake, those generators need to be turned into standalone projects.
  - usefulcat 7y ago
    > I'm compiling some 100k lines of legacy C (for a custom operating system) plus compat shims that make the thing run on Linux and my very very trivial build script (plain posix compatible shell) takes less than 5 seconds to run in debug mode. That's why /you/ don't need make. But there are plenty of 100k line projects that would take much longer, like nearly anything written in c++ for example.
- jakewins 7y ago
  Author here: Exactly this. We keep a `bin/` in the root dir with scripts/programs for more involved steps. Make gives us the dependency graph, incremental rebuilds and parallelization.
- Sean1708 7y ago
  > Because the core proposition value of make is the dependency graph and incremental rebuild, which then trivially allows for parallelisation with -j<n>, In my personal experience 90% of Makefiles I come across not only don't take advantage of these, but actually have to actively employ workarounds to avoid these things. If you're taking advantage of the things make has to offer then by all means use make, but if all of your targets are .PHONY and you're having to turn off make features just so you can put all your bash scripts in one file then maybe just put your bash scripts in separate files.
- BeetleB 7y ago
  > Because the core proposition value of make is the dependency graph and incremental rebuild So you are in agreement with your parent? He's suggesting make for these use cases. The issue is that a lot of times people use Makefiles for putting convenient commands that have little to do with dependencies/incremental builds.
jorge-fundido 7y ago
> If you're going to rely on a fancy shell anyway, why not just throw make out of the loop altogether? I think can still add value in that scenario as the standard project entrypoint. Someone completely new to the project should be able to `make help`, `make install`, etc. Even if the make targets are simple wrappers to project specific tooling (npm, pip, sbt, etc). If the Makefile is kept simple, then users can treat it as a form of README and should be able to cherry-pick from it as they see fit. But yeah, I share your sentiment about quirky, non-portable Makefiles potentially being anti-value.
- clarry 7y ago
  I certainly like it when I encounter a new project, type make, and it just works. This is rare these days.. honestly at this point there's so much tooling already around that one can't really ever take that for granted. Almost always, I need to look for some readme or look around and figure out what build system is in use (sometimes it's a common system plus custom stuff so just seeing a name you recognize isn't automatically going to mean the standard invocation will work). In this case, it hardly matters whether I'm going to run make or ./build or make help or ./build help. A simple script, which I'm advocating for simple projects, can double as a form of README just as a Makefile can.
GordonS 7y ago
This came up just a few days ago[0], and someone suggested you could use either of these: #!/bin/sh set -e case "$1" in build) ;; run) ;; clean) ;; *) echo "unknown: $1"; exit 2 ;; esac OR #!/bin/sh set -e test $# -gt 0 build() { : } run() { : } clean() { : } "$@" Personally, I find makefiles to be ubiquitous and well-understood, and if I want to do anything complex, I can always put that complexity in a shell script that's called by a make target. [0] https://news.ycombinator.com/item?id=21735176#21754331 https://news.ycombinator.com/item?id=21735176#21754331
- jtaft 7y ago
  The second example is vulnerable to command injection. ./test.sh ls -la
  - admax88q 7y ago
    "vulnerable" It's a shell script as part of your build system. Surely you're not passing untrusted input to your command line are you?
    - jtaft 7y ago
      Someone might. It's always better not to introduce such code.
      - GordonS 7y ago
        Unless I'm missing something, this seems a bit silly. Why would I be any more likely to type `test.sh rm -rf /` that `rm -rm /`?
        jtaft 7y ago
        I think you're both right that the risk is low, assuming it's a build script that is to be run locally by a developer. Code tends to get copied and pasted, and can easily sneak into other programs. Programs are integrated in ways which weren't originally intended. It's not a secure coding pattern, and that's why I mentioned it. During security reviews, I would be focusing on more risky vulnerabilities, but I still review and flag findings in build scripts. I'm more concerned with build scripts downloading content over HTTP, or missing security compiler flags, but I digress.
simias 7y ago
Besides performance, I do enjoy the feedback of incremental builds. It's not a huge deal but it's not uncommon that I spot a problem just because the build doesn't behave in the way I expected. Out of the top of my head: - I forget to add a dependency to the Makefile, I modify said file and notice that `make` doesn't do anything, hinting that something is wrong. If the build system rebuilds everything every time I probably won't notice the problem immediately. - I edit code in a project but run make in the wrong location (because I'm working on two projects at one for instance). Again, make will tell me that there's nothing to do when I make changes. - I run the latest build and it behaves in an unexpected way, I'm thinking that maybe I haven't actually built the last version of the source so the code I'm looking at isn't what's being executed. If I run "make" and I see that it starts rebuilding some artifacts I know that I'm out of date and need to try again. If nothing happens then I need to stop looking for excuse and actually start debugging. - I pull code written by other people (or apply a patch, or unstash old changes). Running make will give me some insight on what parts of the codebase have been impacted. If I see that it's rebuilding something I didn't expect to change it'll probably catch my eye and get me to double check what happened. So I definitely wouldn't trade a Makefile for a shell script, especially since if you build system can easily be replaced by a simple script you'll probably be able to do just fine with a very basic Makefile.
Myrmornis 7y ago
It’s nice to use make to provide a uniform interface for diverse projects, including random personal projects. I.e. whatever project you’re in, in whatever language etc, you know what the semantics of `make build` and `make test` are going to be. Even if these recipes are trivial the uniform interface is a win.

by clarry 7y ago

lloeki 7y ago

Because the core proposition value of make is the dependency graph and incremental rebuild, which then trivially allows for parallelisation with -j<n>, and doing that with shell scripts is basically reimplementing that part of make, only badly. Instead of pushing for makefiles to become shell scripts for the part you mention, I usually implement such things as support scripts (in shell, python, whatever) - who often come in useful on their own - and call those from make targets. IOW using the best tool for each job, with make as the orchestrator.

clarry 7y ago
> Because the core proposition value of make is the dependency graph, which then trivially allows for parallelisation with -j<n> My core argument is that this rarely matters, and in the cases it does, it is often the case that the project's needs surpass what make provides anyway. Or they're using a very special version of make with very special supporting infrastructure, like bsd.mk. I'm compiling some 100k lines of legacy C (for a custom operating system) plus compat shims that make the thing run on Linux and my very very trivial build script (plain posix compatible shell) takes less than 5 seconds to run in debug mode. Optimized release builds are a bit slower, but this hardly matters in day to day development. It would be trivial to add some parallelism with xargs. Fwiw I can compile the same thing with CMake, and it's faster only sometimes. It often happens that a header changes and everything needs to be built anyway. And it often happens that the parallelism bites me back when one file gives an error which scrolls out of sight as a dozen other files are still being compiled in parallel. I've wasted far more time fighting subtly broken makefiles (that don't pick up some dependencies and thus fail to recompile the things that need to be recompiled), and build systems with fancy declarative languages that don't present an obvious way to do what you do in one line of shell when you don't need to worry about a dependency graph.
- MaxBarraclough 7y ago
  Personally I see CMake as the best way to work with portable C/C++ these days, as it supports Linux/Mac/Windows, including library-detection and everything else a traditional autotools "./configure" script does. That said, I'm not a fan of the CMake scripting language, which is quirky and error-prone. It should be easy to write a new 'Find' module, but it's not. There should be officially documented design patterns to make the task trivial. There still aren't. CMake gets the job done, but it's surprisingly difficult to work with. </rant>
  - irishcoffee 7y ago
    I really want to like CMake, but having spent the past 5 years using qmake daily, CMake seems very voodoo-ish and archaic.
    - pferde 7y ago
      Too bad, Qt project itself is moving to CMake as its main build tool: https://bugreports.qt.io/browse/QTBUG-73351 https://bugreports.qt.io/browse/QTBUG-73351
      - irishcoffee 7y ago
        I'm aware.
  - hvdijk 7y ago
    Not everything else. One particular thing cmake does not do well that bugs me quite a lot is supporting source code generators, especially in cross compilations, where those source code generators are themselves written in C or some other language requiring compilation. With Makefiles, whether with autotools or manually written ones, this is easy to set up with a few custom rules. With cmake, those generators need to be turned into standalone projects.
- usefulcat 7y ago
  > I'm compiling some 100k lines of legacy C (for a custom operating system) plus compat shims that make the thing run on Linux and my very very trivial build script (plain posix compatible shell) takes less than 5 seconds to run in debug mode. That's why /you/ don't need make. But there are plenty of 100k line projects that would take much longer, like nearly anything written in c++ for example.
jakewins 7y ago
Author here: Exactly this. We keep a `bin/` in the root dir with scripts/programs for more involved steps. Make gives us the dependency graph, incremental rebuilds and parallelization.
Sean1708 7y ago
> Because the core proposition value of make is the dependency graph and incremental rebuild, which then trivially allows for parallelisation with -j<n>, In my personal experience 90% of Makefiles I come across not only don't take advantage of these, but actually have to actively employ workarounds to avoid these things. If you're taking advantage of the things make has to offer then by all means use make, but if all of your targets are .PHONY and you're having to turn off make features just so you can put all your bash scripts in one file then maybe just put your bash scripts in separate files.
BeetleB 7y ago
> Because the core proposition value of make is the dependency graph and incremental rebuild So you are in agreement with your parent? He's suggesting make for these use cases. The issue is that a lot of times people use Makefiles for putting convenient commands that have little to do with dependencies/incremental builds.

jorge-fundido 7y ago

> If you're going to rely on a fancy shell anyway, why not just throw make out of the loop altogether? I think can still add value in that scenario as the standard project entrypoint. Someone completely new to the project should be able to `make help`, `make install`, etc. Even if the make targets are simple wrappers to project specific tooling (npm, pip, sbt, etc). If the Makefile is kept simple, then users can treat it as a form of README and should be able to cherry-pick from it as they see fit. But yeah, I share your sentiment about quirky, non-portable Makefiles potentially being anti-value.

clarry 7y ago
I certainly like it when I encounter a new project, type make, and it just works. This is rare these days.. honestly at this point there's so much tooling already around that one can't really ever take that for granted. Almost always, I need to look for some readme or look around and figure out what build system is in use (sometimes it's a common system plus custom stuff so just seeing a name you recognize isn't automatically going to mean the standard invocation will work). In this case, it hardly matters whether I'm going to run make or ./build or make help or ./build help. A simple script, which I'm advocating for simple projects, can double as a form of README just as a Makefile can.

GordonS 7y ago

This came up just a few days ago[0], and someone suggested you could use either of these: #!/bin/sh set -e case "$1" in build) ;; run) ;; clean) ;; *) echo "unknown: $1"; exit 2 ;; esac OR #!/bin/sh set -e test $# -gt 0 build() { : } run() { : } clean() { : } "$@" Personally, I find makefiles to be ubiquitous and well-understood, and if I want to do anything complex, I can always put that complexity in a shell script that's called by a make target. [0] https://news.ycombinator.com/item?id=21735176#21754331 https://news.ycombinator.com/item?id=21735176#21754331

jtaft 7y ago
The second example is vulnerable to command injection. ./test.sh ls -la
- admax88q 7y ago
  "vulnerable" It's a shell script as part of your build system. Surely you're not passing untrusted input to your command line are you?
  - jtaft 7y ago
    Someone might. It's always better not to introduce such code.
    - GordonS 7y ago
      Unless I'm missing something, this seems a bit silly. Why would I be any more likely to type `test.sh rm -rf /` that `rm -rm /`?
      - jtaft 7y ago
        I think you're both right that the risk is low, assuming it's a build script that is to be run locally by a developer. Code tends to get copied and pasted, and can easily sneak into other programs. Programs are integrated in ways which weren't originally intended. It's not a secure coding pattern, and that's why I mentioned it. During security reviews, I would be focusing on more risky vulnerabilities, but I still review and flag findings in build scripts. I'm more concerned with build scripts downloading content over HTTP, or missing security compiler flags, but I digress.

simias 7y ago

Besides performance, I do enjoy the feedback of incremental builds. It's not a huge deal but it's not uncommon that I spot a problem just because the build doesn't behave in the way I expected. Out of the top of my head: - I forget to add a dependency to the Makefile, I modify said file and notice that `make` doesn't do anything, hinting that something is wrong. If the build system rebuilds everything every time I probably won't notice the problem immediately. - I edit code in a project but run make in the wrong location (because I'm working on two projects at one for instance). Again, make will tell me that there's nothing to do when I make changes. - I run the latest build and it behaves in an unexpected way, I'm thinking that maybe I haven't actually built the last version of the source so the code I'm looking at isn't what's being executed. If I run "make" and I see that it starts rebuilding some artifacts I know that I'm out of date and need to try again. If nothing happens then I need to stop looking for excuse and actually start debugging. - I pull code written by other people (or apply a patch, or unstash old changes). Running make will give me some insight on what parts of the codebase have been impacted. If I see that it's rebuilding something I didn't expect to change it'll probably catch my eye and get me to double check what happened. So I definitely wouldn't trade a Makefile for a shell script, especially since if you build system can easily be replaced by a simple script you'll probably be able to do just fine with a very basic Makefile.

Myrmornis 7y ago

It’s nice to use make to provide a uniform interface for diverse projects, including random personal projects. I.e. whatever project you’re in, in whatever language etc, you know what the semantics of `make build` and `make test` are going to be. Even if these recipes are trivial the uniform interface is a win.