7 ms·

Make vs. Tup (2016)

35 points by edjroot 8y ago

bhengaij 8y ago
There was a good post yesterday about redo. I'm all for good build systems and I have used make quite a bit but the problem in moving to a better build system is that makefiles are so convoluted and hard to reason about that nobody wants to be blamed for breaking the build by migrating. I wish there were a testing system for builds. Specify updating what should change what and check timestamps (to begin with) of created files.
- AstralStorm 8y ago
  That's just a start. Tup is much less flexible and forces certain conventions on the build system. It is also much more rigid about introducing dependencies during the build.
- geezerjay 8y ago
  > but the problem in moving to a better build system is that makefiles are so convoluted and hard to reason about Nowadays Makefiles are largely autogenerated by the build system. How many people are actually editing makefiles by hand in non-pet projects?
  - bhengaij 8y ago
    Pond meet frog
  - cat199 8y ago
    > How many people are actually editing makefiles by hand in non-pet projects? people that don't want makefiles so convoluted and hard to reason about that they can't edit them by hand.. i suggest taking a look at any of the BSD build systems and seeing what sane use of make can look like (PMake and not GNUMake; imho PMake's language makes this doable; GNUMake's language makes autotools and other mess-generating 'helpers' required) http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/share/mk/bsd.prog.mk https://svnweb.freebsd.org/base/stable/12/share/mk/
    - geezerjay 8y ago
      > people that don't want makefiles so convoluted and hard to reason about that they can't edit them by hand.. You've missed the point. The point was that nowadays editing or even looking at a makefile is far from standard practice, because makefiles are autogenerated by the build system and are closer to temp files than to project files. Who wastes their time reasoning about a temp file that just works?
  - jschwartzi 8y ago
    I wrote a build system for an embedded system with two processors and lots of code sharing across three platforms in Make because no other tool would let me program the whole DAG in the same language. The makefiles generated themselves.
breatheoften 8y ago
Anyone aware of an npm library that wraps tup to provide automatic dependency graph construction from objects changing on filesystem? I'd could use a javascript api that enabled automatically detecting filesystem change dependency graph associated with each of a given set of javascript function calls ... Perhaps with simplifying assumption that all function parameters are serializable or maybe even that the functions being tracked are only allowed to operate on strings that represents filesystem path's ... I'd love for the functionality implemented within tup of running a command and automatically doing the low-level kernel hacking necessary to track all the filesystem objects read/written to be abstracted into an application-level library ...
sjmulder 8y ago
What I've seen of Tup is good but it doesn't add enough to convince me to switch away from make which is already everywhere. CMake and similar tools don't attract me because they're not language independent. The best thing about make (and Tup too) is that it lets you express dependencies and ways to satisfy them in terms of other tools. Now what make is not good at is being ./configure. I'd like to see a more elegant autoconf-like tool to detect things about the environment and generate configure.{h,mk}.
- AstralStorm 8y ago
  CMake actually is language independent, but it so happens that only C, C++, Assembly, Fortran are first class citizens. Java and Ada are second class. Anything else, nobody wrote it. You might find a package to find the compiler and dependencies perhaps. It is very much possible to add other languages.
- haolez 8y ago
  I’ve used tup in a previous project. I was expecting issues that I couldn’t predict before getting my hands dirty, but it was actually a very smooth transition. For complex C and C++ projects, I can recommend tup. I’ve also used it for Latex documents.
- JNRowe 8y ago
  premake¹ feels like an elegant autoconf alternative to me, and can be combined with various build systems including make. The syntax is far nicer than cmake too, but frankly that bar isn’t very high. Sadly, it isn’t used by all that many projects² which holds me back from wanting to depend on it. I have played with it in a few personal projects though, and have been impressed. 1. https://premake.github.io/ https://premake.github.io/ 2. https://github.com/premake/premake-core/wiki/Who-Uses-Premake https://github.com/premake/premake-core/wiki/Who-Uses-Premak...
JNRowe 8y ago
I found the Build Systems a la Carte¹ paper, and NDM’s companion write up² to be good comparison of build systems in a more general manner. Caveat: It may skewed having been written by heavy Haskell hitters, and an author of Shake. I’ll note that I didn’t notice any bias, but maybe only because it aligned with mine ;) 1. https://www.microsoft.com/en-us/research/publication/build-systems-la-carte/ https://www.microsoft.com/en-us/research/publication/build-s... 2. http://neilmitchell.blogspot.com/2018/07/inside-paper-build-systems-la-carte.html http://neilmitchell.blogspot.com/2018/07/inside-paper-build-...
dig1 8y ago
This reminds me on 'Make vs. X' trend some years ago, where Make would be bashed how is slow, not extensible, had weird syntax and incompatible between implementations. So we got alternatives like Aegis[1], Scons[2], A-A-P[3] or Jam[4] instead, which were much faster and flashier on paper. But guess what, Make is still kicking, GNU Make got a bunch of new goodies (Guile scripting or loadable modules to name a few) and all those alternatives are pretty much dead now. What I see now is that we are repeating the same mistakes only with flashier tools (node, python3, rust). Although I always preferred Jam[4], I'm pretty happy with GNU Make now. Not perfect, does the job well and if I ever hit some weird platforms, I can always 'extend' myself to Autotools[5]. Funny thing, I'm even using Make to run Ansible scripts or compile Java/Clojure code and works like a charm. [1] http://aegis.sourceforge.net/ http://aegis.sourceforge.net/ [2] https://scons.org/ https://scons.org/ [3] http://www.a-a-p.org/ http://www.a-a-p.org/ [4] https://en.wikipedia.org/wiki/Perforce_Jam https://en.wikipedia.org/wiki/Perforce_Jam [5] https://en.wikipedia.org/wiki/GNU_Build_System https://en.wikipedia.org/wiki/GNU_Build_System
- heavenlyhash 8y ago
  I suspect a large amount of the issue is sheer availability. There's an odd proclivity of developers of build tools to assume that their favorite runtime (be it python for scons, etc) is already available or trivial to nicely install. And that's just... not true. The build tool itself is the one place in the ecosystem of building software that we have the least room for manual dependency handling and manual setup, because there's nowhere left to punt the hard part. Make isn't better at this either, but it's been so widely packaged and is so nearly-default available (it's remarkably hard to get a working desktop system without some transitive dependency having gotten make installed for you) that it defacto gets a free pass on this. My hypothesis is that these newer build tools would have a much better shot at reaching adoption if they were well-packaged enough that a single tarball (with zero external deps) or a single-line install script (think: "gradlew", though even that was cheating by assuming an existing jvm) could bootstrap them. Most don't seem to have invested that effort. And it shows.
  - geezerjay 8y ago
aidenn0 8y ago
My two favorite make alternatives make HN in two consecutive days. I like redo because with make you need to know two languages, with redo you know just one, the interface is simple and you get automatic dependency on build rules for free. Also the implementation is so much simpler than make, and yet I've not run into a feature I miss from make. I like tup because it is very opinionated and forces you to write your build scripts in a reasonable manner, while also being very fast and very correct. It might be possible to write a tup file that incorrectly handles dependencies, but you'd have to work hard to do so.
- majewsky 8y ago
  Do you have a link to the redo story? I can't seem to find it on hckrnews.com
  - ksherlock 8y ago
    https://news.ycombinator.com/item?id=18480433 https://news.ycombinator.com/item?id=18480433 perhaps
    - aidenn0 8y ago
      That's the one
      - JdeBP 8y ago
        Don't forget https://news.ycombinator.com/item?id=18473744 https://news.ycombinator.com/item?id=18473744 and https://news.ycombinator.com/item?id=18405227 https://news.ycombinator.com/item?id=18405227 .
jp57 8y ago
tl;dr: for the vast majority of projects, tup does not outperform make.
- geezerjay 8y ago
  That was my take as well, and I honestly don't get the downvotes. More importantly, in large projects the bulk of the comptational budget is spent actually compiling source files. It's hard to believe that picking which file neess to be compiled next takes more time than actually compiling it.

Make vs. Tup (2016)

35 points by edjroot 8y ago

There was a good post yesterday about redo. I'm all for good build systems and I have used make quite a bit but the problem in moving to a better build system is that makefiles are so convoluted and hard to reason about that nobody wants to be blamed for breaking the build by migrating. I wish there were a testing system for builds. Specify updating what should change what and check timestamps (to begin with) of created files.

AstralStorm 8y ago
That's just a start. Tup is much less flexible and forces certain conventions on the build system. It is also much more rigid about introducing dependencies during the build.
geezerjay 8y ago
> but the problem in moving to a better build system is that makefiles are so convoluted and hard to reason about Nowadays Makefiles are largely autogenerated by the build system. How many people are actually editing makefiles by hand in non-pet projects?
- bhengaij 8y ago
  Pond meet frog
- cat199 8y ago
  > How many people are actually editing makefiles by hand in non-pet projects? people that don't want makefiles so convoluted and hard to reason about that they can't edit them by hand.. i suggest taking a look at any of the BSD build systems and seeing what sane use of make can look like (PMake and not GNUMake; imho PMake's language makes this doable; GNUMake's language makes autotools and other mess-generating 'helpers' required) http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/share/mk/bsd.prog.mk https://svnweb.freebsd.org/base/stable/12/share/mk/
  - geezerjay 8y ago
    > people that don't want makefiles so convoluted and hard to reason about that they can't edit them by hand.. You've missed the point. The point was that nowadays editing or even looking at a makefile is far from standard practice, because makefiles are autogenerated by the build system and are closer to temp files than to project files. Who wastes their time reasoning about a temp file that just works?
- jschwartzi 8y ago
  I wrote a build system for an embedded system with two processors and lots of code sharing across three platforms in Make because no other tool would let me program the whole DAG in the same language. The makefiles generated themselves.

breatheoften 8y ago

Anyone aware of an npm library that wraps tup to provide automatic dependency graph construction from objects changing on filesystem? I'd could use a javascript api that enabled automatically detecting filesystem change dependency graph associated with each of a given set of javascript function calls ... Perhaps with simplifying assumption that all function parameters are serializable or maybe even that the functions being tracked are only allowed to operate on strings that represents filesystem path's ... I'd love for the functionality implemented within tup of running a command and automatically doing the low-level kernel hacking necessary to track all the filesystem objects read/written to be abstracted into an application-level library ...

sjmulder 8y ago

What I've seen of Tup is good but it doesn't add enough to convince me to switch away from make which is already everywhere. CMake and similar tools don't attract me because they're not language independent. The best thing about make (and Tup too) is that it lets you express dependencies and ways to satisfy them in terms of other tools. Now what make is not good at is being ./configure. I'd like to see a more elegant autoconf-like tool to detect things about the environment and generate configure.{h,mk}.

AstralStorm 8y ago
CMake actually is language independent, but it so happens that only C, C++, Assembly, Fortran are first class citizens. Java and Ada are second class. Anything else, nobody wrote it. You might find a package to find the compiler and dependencies perhaps. It is very much possible to add other languages.
haolez 8y ago
I’ve used tup in a previous project. I was expecting issues that I couldn’t predict before getting my hands dirty, but it was actually a very smooth transition. For complex C and C++ projects, I can recommend tup. I’ve also used it for Latex documents.
JNRowe 8y ago
premake¹ feels like an elegant autoconf alternative to me, and can be combined with various build systems including make. The syntax is far nicer than cmake too, but frankly that bar isn’t very high. Sadly, it isn’t used by all that many projects² which holds me back from wanting to depend on it. I have played with it in a few personal projects though, and have been impressed. 1. https://premake.github.io/ https://premake.github.io/ 2. https://github.com/premake/premake-core/wiki/Who-Uses-Premake https://github.com/premake/premake-core/wiki/Who-Uses-Premak...

JNRowe 8y ago

I found the Build Systems a la Carte¹ paper, and NDM’s companion write up² to be good comparison of build systems in a more general manner. Caveat: It may skewed having been written by heavy Haskell hitters, and an author of Shake. I’ll note that I didn’t notice any bias, but maybe only because it aligned with mine ;) 1. https://www.microsoft.com/en-us/research/publication/build-systems-la-carte/ https://www.microsoft.com/en-us/research/publication/build-s... 2. http://neilmitchell.blogspot.com/2018/07/inside-paper-build-systems-la-carte.html http://neilmitchell.blogspot.com/2018/07/inside-paper-build-...

dig1 8y ago

This reminds me on 'Make vs. X' trend some years ago, where Make would be bashed how is slow, not extensible, had weird syntax and incompatible between implementations. So we got alternatives like Aegis[1], Scons[2], A-A-P[3] or Jam[4] instead, which were much faster and flashier on paper. But guess what, Make is still kicking, GNU Make got a bunch of new goodies (Guile scripting or loadable modules to name a few) and all those alternatives are pretty much dead now. What I see now is that we are repeating the same mistakes only with flashier tools (node, python3, rust). Although I always preferred Jam[4], I'm pretty happy with GNU Make now. Not perfect, does the job well and if I ever hit some weird platforms, I can always 'extend' myself to Autotools[5]. Funny thing, I'm even using Make to run Ansible scripts or compile Java/Clojure code and works like a charm. [1] http://aegis.sourceforge.net/ http://aegis.sourceforge.net/ [2] https://scons.org/ https://scons.org/ [3] http://www.a-a-p.org/ http://www.a-a-p.org/ [4] https://en.wikipedia.org/wiki/Perforce_Jam https://en.wikipedia.org/wiki/Perforce_Jam [5] https://en.wikipedia.org/wiki/GNU_Build_System https://en.wikipedia.org/wiki/GNU_Build_System

heavenlyhash 8y ago
I suspect a large amount of the issue is sheer availability. There's an odd proclivity of developers of build tools to assume that their favorite runtime (be it python for scons, etc) is already available or trivial to nicely install. And that's just... not true. The build tool itself is the one place in the ecosystem of building software that we have the least room for manual dependency handling and manual setup, because there's nowhere left to punt the hard part. Make isn't better at this either, but it's been so widely packaged and is so nearly-default available (it's remarkably hard to get a working desktop system without some transitive dependency having gotten make installed for you) that it defacto gets a free pass on this. My hypothesis is that these newer build tools would have a much better shot at reaching adoption if they were well-packaged enough that a single tarball (with zero external deps) or a single-line install script (think: "gradlew", though even that was cheating by assuming an existing jvm) could bootstrap them. Most don't seem to have invested that effort. And it shows.
- geezerjay 8y ago

aidenn0 8y ago

My two favorite make alternatives make HN in two consecutive days. I like redo because with make you need to know two languages, with redo you know just one, the interface is simple and you get automatic dependency on build rules for free. Also the implementation is so much simpler than make, and yet I've not run into a feature I miss from make. I like tup because it is very opinionated and forces you to write your build scripts in a reasonable manner, while also being very fast and very correct. It might be possible to write a tup file that incorrectly handles dependencies, but you'd have to work hard to do so.

majewsky 8y ago
Do you have a link to the redo story? I can't seem to find it on hckrnews.com
- ksherlock 8y ago
  https://news.ycombinator.com/item?id=18480433 https://news.ycombinator.com/item?id=18480433 perhaps
  - aidenn0 8y ago
    That's the one
    - JdeBP 8y ago
      Don't forget https://news.ycombinator.com/item?id=18473744 https://news.ycombinator.com/item?id=18473744 and https://news.ycombinator.com/item?id=18405227 https://news.ycombinator.com/item?id=18405227 .

jp57 8y ago

tl;dr: for the vast majority of projects, tup does not outperform make.

geezerjay 8y ago
That was my take as well, and I honestly don't get the downvotes. More importantly, in large projects the bulk of the comptational budget is spent actually compiling source files. It's hard to believe that picking which file neess to be compiled next takes more time than actually compiling it.