4 ms·

This is a fantastic paper that I come back to over the years. I remember talking about it with a coworker when it came out, and we were amazed how elegantly it

This is a fantastic paper that I come back to over the years. I remember talking about it with a coworker when it came out, and we were amazed how elegantly it made the point that Excel is a build system, and how it helps understand how all build systems fit together in their design space.

kryptiskt 3y ago
An exercise for the esoteric programmer would be to make a sheet that turns Excel into an actual build system that can drive the build of a typical C program.
ted_bunny 3y ago
I'm great at excel and terrible at programming. Can you break down what's meant here?
- klodolph 3y ago
  In Excel, every cell has some number of inputs (which are other cells), and a rule for calculating the cell’s value from its input. That is basically how software build systems work, except usually they use files instead of spreadsheet cells, and they run commands (like “compile this source code”, or “combine these seven files into one program”) instead of Excel formulas.
- ufo 3y ago
  In a build system, when you update a source code file, it should recompile the binaries that depend on it. Similarly, in excel when you update a cell in the spreadsheet, it should recompute the cells that refer to it. The paper is about thinking about build systems in a more modular way. Are dependencies specified statically or dynamically? What algorithm is used to decide what should get rebuilt after an update? What order do we rebuild things? What happens if we pick different combinations of algorithm+order?
  - dayjaby 3y ago
    Is that actually a thing? What algorithms do you mean? I thought everything that depends on a change gets rebuilt.
    - ufo 3y ago
      The devil is in the details. For example, Make decides to rebuild based on timestamp, while Bazel decides to recompute based of file hashes. Thus, if you touch a file but don't change its contents, Make might rebuild things that Bazel wouldn't. Another example in the paper is the INDIRECT function in Excel. This makes it so that Excel cannot compute the dependency graph before the build starts. Excel has ways to deal with this, but they might cause some cells to be recomputed even when they didn't actually need to be. The paper has a bunch of interesting examples like this, it's worth the read :) The problem of dynamic dependencies is particularly tricky. For example, suppose that you want to teach your makefile that your compilation depends on the source C file, but also on all the headers that are #included by that C file. And you want to actually look inside the C file to get that list of headers, not just copy paste the list into the makefile. There's more than one way to handle this.
theamk 3y ago
This seems mostly exercise in Haskell programming, and not really related to actual things that people look for in the build system. For example, paper talks a great deal about schedulers, but those are very minute implementation details. If bazel gets "suspending" scheduler instead of "restarting" one, this is a super major change in the paper's eyes, but no user is likely to notice (other than "hey it works 2% faster now, cool!"). Moreover, the paper's scheduler abstraction is too limited - there are a whole bunch of schedulers which are non-representative in it, for example imagine a scheduler which is aware of memory and CPU usage, and schedules the tasks to avoid out-of-memory condition (like recent bazel). The division on "cloud" vs "non-cloud" systems is also pretty weird... ok, bazel has cloud caching built-in but make requires external "ccache", but the resulting behaviors are pretty similar.

kryptiskt 3y ago

An exercise for the esoteric programmer would be to make a sheet that turns Excel into an actual build system that can drive the build of a typical C program.

ted_bunny 3y ago

I'm great at excel and terrible at programming. Can you break down what's meant here?

klodolph 3y ago
In Excel, every cell has some number of inputs (which are other cells), and a rule for calculating the cell’s value from its input. That is basically how software build systems work, except usually they use files instead of spreadsheet cells, and they run commands (like “compile this source code”, or “combine these seven files into one program”) instead of Excel formulas.
ufo 3y ago
In a build system, when you update a source code file, it should recompile the binaries that depend on it. Similarly, in excel when you update a cell in the spreadsheet, it should recompute the cells that refer to it. The paper is about thinking about build systems in a more modular way. Are dependencies specified statically or dynamically? What algorithm is used to decide what should get rebuilt after an update? What order do we rebuild things? What happens if we pick different combinations of algorithm+order?
- dayjaby 3y ago
  Is that actually a thing? What algorithms do you mean? I thought everything that depends on a change gets rebuilt.
  - ufo 3y ago
    The devil is in the details. For example, Make decides to rebuild based on timestamp, while Bazel decides to recompute based of file hashes. Thus, if you touch a file but don't change its contents, Make might rebuild things that Bazel wouldn't. Another example in the paper is the INDIRECT function in Excel. This makes it so that Excel cannot compute the dependency graph before the build starts. Excel has ways to deal with this, but they might cause some cells to be recomputed even when they didn't actually need to be. The paper has a bunch of interesting examples like this, it's worth the read :) The problem of dynamic dependencies is particularly tricky. For example, suppose that you want to teach your makefile that your compilation depends on the source C file, but also on all the headers that are #included by that C file. And you want to actually look inside the C file to get that list of headers, not just copy paste the list into the makefile. There's more than one way to handle this.

theamk 3y ago

This seems mostly exercise in Haskell programming, and not really related to actual things that people look for in the build system. For example, paper talks a great deal about schedulers, but those are very minute implementation details. If bazel gets "suspending" scheduler instead of "restarting" one, this is a super major change in the paper's eyes, but no user is likely to notice (other than "hey it works 2% faster now, cool!"). Moreover, the paper's scheduler abstraction is too limited - there are a whole bunch of schedulers which are non-representative in it, for example imagine a scheduler which is aware of memory and CPU usage, and schedules the tasks to avoid out-of-memory condition (like recent bazel). The division on "cloud" vs "non-cloud" systems is also pretty weird... ok, bazel has cloud caching built-in but make requires external "ccache", but the resulting behaviors are pretty similar.