3 ms·

the last 10 years in build tools has felt like 1 step forward, two steps back. i like being able to write tasks in any language other than Makefile. however, it

the last 10 years in build tools has felt like 1 step forward, two steps back. i like being able to write tasks in any language other than Makefile. however, it seems like many of the new popular options (cake, grunt, etc.) don't do what, to me, is Make's real purpose: resolve dependencies and only rebuild what's necessary. new task runners have either eliminated or pigeonholed the (typically one-to-one in makeland) correspondence between tasks and files, meaning the build system can't be intelligent about what tasks to run and which to not. computers are fast enough that this doesn't often bother me anymore, but i've run across some huge Rakefiles that could benefit from a rewrite in Make.

chrismonsanto 12y ago
> however, it seems like many of the new popular options (cake, grunt, etc.) don't do what, to me, is Make's real purpose: resolve dependencies and only rebuild what's necessary. You might like tup[1]. Its killer feature is that it automatically determines file-based dependencies by tracking reads and writes (using a FUSE filesystem). It has an extreme emphasis on correct, repeatable builds, and is very fast. Other stuff: - does work in parallel, and will let you know if your build isn't parallel safe. (note it is NOT relying on your specification of dependencies: even if you manually specify dependencies, it will tell you if something's wrong based on what it actually observes your dependencies to be) - tracks changes to the build script and reruns if the commands change. - cleans up old output files automatically if build rules are removed. - lets you maintain multiple build variants (say for different architectures, configurations, etc) - autogenerates .gitignore files for your build output - very easy to get started, and "Just Works". - for advanced usage, it is scriptable in Lua. I've tried every build system out there. For Unix-y file-based build tasks, tup is, by far, the best. I don't know why it isn't more well known. [1]: http://gittup.org/tup/index.html http://gittup.org/tup/index.html
- Touche 12y ago
  Seconded on tup's greatness. Go through the example to see what all it's doing: http://gittup.org/tup/ex_a_first_tupfile.html http://gittup.org/tup/ex_a_first_tupfile.html
- Rusky 12y ago
  +1 for the link. I hadn't seen tup before, and I really like how it feels like make in its simplicity, but is more explicit about the graph inputs and outputs, cares more about the output (e.g. deleting old files), and watches the file system for changes.
- rkrzr 12y ago
  I was already sold on tup after reading the first paragraph comparing it to make[1]: "This page compares make to tup. This page is a little biased because tup is so fast. How fast? This one time a beam of light was flying through the vacuum of space at the speed of light and then tup went by and was like "Yo beam of light, you need a lift?" cuz tup was going so fast it thought the beam of light had a flat tire and was stuck. True story. Anyway, feel free to run your own comparisons if you don't believe me and my (true) story." [1]: http://gittup.org/tup/make_vs_tup.html http://gittup.org/tup/make_vs_tup.html
  - andrewflnr 12y ago
    My favorite is the "Tup vs Mordor" benchmark.
- malkia 12y ago
  I think MSBuild does something like this - there is a filetracker that tracks what files were read/written while running a tool, and writes that information in a file. I think you can even install your own file-notification-changes .dll to track changes your way (maybe file system that is not supported, or something else). Similar to what lsof, procmon (windows) do.
mhw 12y ago
I agree completely, and I think the blame rests with Ant and Java. Java's dependency management was painful enough to deal with in 'make' that Ant was built to support building Java projects. But in doing so the authors threw away the explicit file dependencies that made 'make' so powerful in the first place. Instead it got people to think in terms of a graph of 'tasks', each of which could either figure out its own dependency management, or more commonly ignore them completely. Most tools that followed seem to have gone down the 'graph of tasks' avenue, with 'graph of file dependencies' as an additional mechanism if you're lucky. The huge Rakefiles you've seen could possibly have simply benefited from a rewrite in Rake. Rake has 'file' tasks which implement the file dependencies of 'make' but for some reason most users of Rake seem to ignore them completely.
ChuckMcM 12y ago
It can be worse than that, one build system I looked at built everything every time. Why? Because "Computers are fast enough that trying to figure out exactly what needs to be rebuilt is an anachronism, this can rebuild everything in the time it took that crufty old system to figure out what it actually had to build." I've given up trying to educate folks, I just make a note to check in with them, 6 months to a year later, to see if they are still building everything.
retrogradeorbit 12y ago
Yes. And then combine that dependency tree resolution with make -j. Simple and powerful.

chrismonsanto 12y ago

> however, it seems like many of the new popular options (cake, grunt, etc.) don't do what, to me, is Make's real purpose: resolve dependencies and only rebuild what's necessary. You might like tup[1]. Its killer feature is that it automatically determines file-based dependencies by tracking reads and writes (using a FUSE filesystem). It has an extreme emphasis on correct, repeatable builds, and is very fast. Other stuff: - does work in parallel, and will let you know if your build isn't parallel safe. (note it is NOT relying on your specification of dependencies: even if you manually specify dependencies, it will tell you if something's wrong based on what it actually observes your dependencies to be) - tracks changes to the build script and reruns if the commands change. - cleans up old output files automatically if build rules are removed. - lets you maintain multiple build variants (say for different architectures, configurations, etc) - autogenerates .gitignore files for your build output - very easy to get started, and "Just Works". - for advanced usage, it is scriptable in Lua. I've tried every build system out there. For Unix-y file-based build tasks, tup is, by far, the best. I don't know why it isn't more well known. [1]: http://gittup.org/tup/index.html http://gittup.org/tup/index.html

Touche 12y ago
Seconded on tup's greatness. Go through the example to see what all it's doing: http://gittup.org/tup/ex_a_first_tupfile.html http://gittup.org/tup/ex_a_first_tupfile.html
Rusky 12y ago
+1 for the link. I hadn't seen tup before, and I really like how it feels like make in its simplicity, but is more explicit about the graph inputs and outputs, cares more about the output (e.g. deleting old files), and watches the file system for changes.
rkrzr 12y ago
I was already sold on tup after reading the first paragraph comparing it to make[1]: "This page compares make to tup. This page is a little biased because tup is so fast. How fast? This one time a beam of light was flying through the vacuum of space at the speed of light and then tup went by and was like "Yo beam of light, you need a lift?" cuz tup was going so fast it thought the beam of light had a flat tire and was stuck. True story. Anyway, feel free to run your own comparisons if you don't believe me and my (true) story." [1]: http://gittup.org/tup/make_vs_tup.html http://gittup.org/tup/make_vs_tup.html
- andrewflnr 12y ago
  My favorite is the "Tup vs Mordor" benchmark.
malkia 12y ago
I think MSBuild does something like this - there is a filetracker that tracks what files were read/written while running a tool, and writes that information in a file. I think you can even install your own file-notification-changes .dll to track changes your way (maybe file system that is not supported, or something else). Similar to what lsof, procmon (windows) do.

mhw 12y ago

I agree completely, and I think the blame rests with Ant and Java. Java's dependency management was painful enough to deal with in 'make' that Ant was built to support building Java projects. But in doing so the authors threw away the explicit file dependencies that made 'make' so powerful in the first place. Instead it got people to think in terms of a graph of 'tasks', each of which could either figure out its own dependency management, or more commonly ignore them completely. Most tools that followed seem to have gone down the 'graph of tasks' avenue, with 'graph of file dependencies' as an additional mechanism if you're lucky. The huge Rakefiles you've seen could possibly have simply benefited from a rewrite in Rake. Rake has 'file' tasks which implement the file dependencies of 'make' but for some reason most users of Rake seem to ignore them completely.

ChuckMcM 12y ago

It can be worse than that, one build system I looked at built everything every time. Why? Because "Computers are fast enough that trying to figure out exactly what needs to be rebuilt is an anachronism, this can rebuild everything in the time it took that crufty old system to figure out what it actually had to build." I've given up trying to educate folks, I just make a note to check in with them, 6 months to a year later, to see if they are still building everything.

retrogradeorbit 12y ago

Yes. And then combine that dependency tree resolution with make -j. Simple and powerful.