6 ms·

Makefiles operate at a low level of abstraction. The Makefile abstraction is "run this command when the output files don't exist or are older than the input fi

Makefiles operate at a low level of abstraction. The Makefile abstraction is "run this command when the output files don't exist or are older than the input files." You manually specify every tool to be run and all of its arguments. These tools operate at a higher level of abstraction. The BUILD file abstraction is something like "I am declaring a C library with these sources and headers." The build system then turns that into a set of commands to run. This higher-level abstraction provides a bunch of compelling features: - Reliability: it's easy to write Makefiles that are fragile, for example one that works when run on a single CPU but fails when run with -j, because of undeclared dependencies. When the tool is in charge, it has more control and can sandbox each build step so that undeclared dependencies simply won't be available in the sandbox. - Reproducibility: similar to the previous point, it's easy to write a Makefile that produces incorrect results with an incremental rebuild. For example, unless you implement your own header scanning (or manually list your header dependencies) "make" won't know to rebuild a source file when a header changes. These tools can ensure that builds are reproducible and incremental rebuilds are always correct. Besides that, working at a higher level generally means writing less code to accomplish the same task, and the code you do write is closer to expressing your actual intent compared with a Makefile.

moonchild 6y ago
> The Makefile abstraction is "run this command when the output files don't exist or are older than the input files." You manually specify every tool to be run and all of its arguments. > The BUILD file abstraction is something like "I am declaring a C library with these sources and headers." This is wrong. Even ninja has generic rules. Here's an example of a minimal makefile: OBJ = src/a.o src/b.o src/c.o libfoo.so: $(OBJ) $(CC) -shared -o libfoo.so $(OBJ) > When the tool is in charge, it has more control and can sandbox each build step so that undeclared dependencies simply won't be available in the sandbox. There's no reason this shouldn't be possible with make; it just hasn't been implemented so. Do bazel/buck/please actually do this? As far as I know tup is the only tool that actually verifies inputs/outputs of rules, and it needs FUSE to do so. > For example, unless you implement your own header scanning (or manually list your header dependencies) "make" won't know to rebuild a source file when a header changes. At least with GNU make, it's very easy: CFLAGS += -MMD -include $(patsubst %.o,%.d,$(OBJ)) True, it's a bit of a footgun, but by no means difficult. Make has problems, but the ones you listed aren't they.
- haberman 6y ago
  > Here's an example of a minimal makefile: Your example does not contradict what I wrote. You manually specified the tool to be run ($CC) and all of the arguments to that tool. It's true that there is a level of indirection through the $CC variable, but you're still operating at the level of specifying a tool's command-line. > There's no reason this shouldn't be possible with make; it just hasn't been implemented so. Make is 44 years old. If it were an easy extension to the existing paradigm, there has been ample time to implement such an extension. > Do bazel/buck/please actually do this? As far as I know tup is the only tool that actually verifies inputs/outputs of rules, and it needs FUSE to do so. Bazel certainly has some amount of sandboxing, though I don't think it's quite as complete as what is available internally at Google with Blaze. I haven't used Buck or Please, so I can't speak for them. > True, it's a bit of a footgun, but by no means difficult. Well footguns aren't great. :) As just one example, any header that is conditionally included (behind an #ifdef) could cause this cache to be invalidated when CFLAGS change, but Make has no idea of this.
  - moonchild 6y ago
    > You manually specified the tool to be run ($CC) and all of the arguments to that tool Notably, I didn't specify the arguments to be used to compile the source files. But if you insist: linklibrary = $(CC) -shared -o $1 $2 libfoo.so: $(OBJ) $(call linklibrary,$@,$(OBJ)) Notably, linklibrary can be defined according the platform, or according to dynamically set variables, giving you the same level of flexibility as make. >> There's no reason [sandboxing] shouldn't be possible with make; it just hasn't been implemented so. > Make is 44 years old. If it were an easy extension to the existing paradigm, there has been ample time to implement such an extension. ‘Nobody's done it yet, ergo it's not possible or easy’ is not a valid argument. > just one example, any header that is conditionally included (behind an #ifdef) could cause this cache to be invalidated when CFLAGS change, but Make has no idea of this. Then you specify that the source files depend on the makefile.
    - joshuamorton 6y ago
      > ‘Nobody's done it yet, ergo it's not possible or easy’ is not a valid argument. Ultimately, make and co are text oriented, while bazel and co are object oriented. Hacking object oriented capabilities into a text-oriented language isn't particularly fruitful or ergonomic.
ketzo 6y ago
This was an extremely good explanation for a non-trivial subject. Props for written communication ability.
- haberman 6y ago
  Thanks, I appreciate it!

moonchild 6y ago

> The Makefile abstraction is "run this command when the output files don't exist or are older than the input files." You manually specify every tool to be run and all of its arguments. > The BUILD file abstraction is something like "I am declaring a C library with these sources and headers." This is wrong. Even ninja has generic rules. Here's an example of a minimal makefile: OBJ = src/a.o src/b.o src/c.o libfoo.so: $(OBJ) $(CC) -shared -o libfoo.so $(OBJ) > When the tool is in charge, it has more control and can sandbox each build step so that undeclared dependencies simply won't be available in the sandbox. There's no reason this shouldn't be possible with make; it just hasn't been implemented so. Do bazel/buck/please actually do this? As far as I know tup is the only tool that actually verifies inputs/outputs of rules, and it needs FUSE to do so. > For example, unless you implement your own header scanning (or manually list your header dependencies) "make" won't know to rebuild a source file when a header changes. At least with GNU make, it's very easy: CFLAGS += -MMD -include $(patsubst %.o,%.d,$(OBJ)) True, it's a bit of a footgun, but by no means difficult. Make has problems, but the ones you listed aren't they.

haberman 6y ago
> Here's an example of a minimal makefile: Your example does not contradict what I wrote. You manually specified the tool to be run ($CC) and all of the arguments to that tool. It's true that there is a level of indirection through the $CC variable, but you're still operating at the level of specifying a tool's command-line. > There's no reason this shouldn't be possible with make; it just hasn't been implemented so. Make is 44 years old. If it were an easy extension to the existing paradigm, there has been ample time to implement such an extension. > Do bazel/buck/please actually do this? As far as I know tup is the only tool that actually verifies inputs/outputs of rules, and it needs FUSE to do so. Bazel certainly has some amount of sandboxing, though I don't think it's quite as complete as what is available internally at Google with Blaze. I haven't used Buck or Please, so I can't speak for them. > True, it's a bit of a footgun, but by no means difficult. Well footguns aren't great. :) As just one example, any header that is conditionally included (behind an #ifdef) could cause this cache to be invalidated when CFLAGS change, but Make has no idea of this.
- moonchild 6y ago
  > You manually specified the tool to be run ($CC) and all of the arguments to that tool Notably, I didn't specify the arguments to be used to compile the source files. But if you insist: linklibrary = $(CC) -shared -o $1 $2 libfoo.so: $(OBJ) $(call linklibrary,$@,$(OBJ)) Notably, linklibrary can be defined according the platform, or according to dynamically set variables, giving you the same level of flexibility as make. >> There's no reason [sandboxing] shouldn't be possible with make; it just hasn't been implemented so. > Make is 44 years old. If it were an easy extension to the existing paradigm, there has been ample time to implement such an extension. ‘Nobody's done it yet, ergo it's not possible or easy’ is not a valid argument. > just one example, any header that is conditionally included (behind an #ifdef) could cause this cache to be invalidated when CFLAGS change, but Make has no idea of this. Then you specify that the source files depend on the makefile.
  - joshuamorton 6y ago
    > ‘Nobody's done it yet, ergo it's not possible or easy’ is not a valid argument. Ultimately, make and co are text oriented, while bazel and co are object oriented. Hacking object oriented capabilities into a text-oriented language isn't particularly fruitful or ergonomic.

ketzo 6y ago

This was an extremely good explanation for a non-trivial subject. Props for written communication ability.

haberman 6y ago
Thanks, I appreciate it!