3 ms·

I've been doing this with Makefiles for a while. It has always bugged me that I'm not "making" an artifact. Think I"m going to try this `run.sh` approach.

chubot 6y ago
Yeah, as mentioned in the posts, make targets that are verbs rather than nouns should really be .PHONY, but most people forget that. While the idea of having Make's dependency engine is nice in theory, it falls down for one-off automation in my experience. For a couple reasons: (1) Make's dependency model has some well known deficiencies. It doesn't play well with tools that produce two files. It doesn't play well with tools that produce a directory tree of "dynamic" filenames (not known when you write the Makefile) (2) Most makefiles have bugs, especially when you do make -j (parallel builds). It's basically like writing a C program with a bunch of threads racing on global variables -- your Make targets will often be racing on the same file, leading to non-deterministic bugs. ----- So IMO it's better to mostly stick with the sequential model of shell for this kind of project automation. But shell can invoke make! When you know your dependencies, invoke them from run.sh! And when you're REALLY sure it's correct, invoke make -j :) In other words shell is my default, and make is only for when I want to spend the effort to write dependencies -- which is quite difficult, because Make provides you virtually no help with that. Bugs in dependencies are common and hard to find. If you're trained to run "make clean", then that's a symptom of a bug in the build specification. Shell "gets shit done" without these types of bugs. Debugging a shell script is very easy compared with debugging a makefile. The remaining problems with shell will hopefully be fixed by https://oilshell.org/ https://oilshell.org/ :) I do want to add some dependency support, but I didn't get to it: Shell, Awk, and Make Should Be Combined http://www.oilshell.org/blog/2016/11/13.html http://www.oilshell.org/blog/2016/11/13.html Make is just a small elaboration on the shell model (concurrent processes and files), but unfortunately it's implemented as a completely separate tool that shells out to shell! facepalm
- fao_ 6y ago
  > (1) Make's dependency model has some well known deficiencies. It doesn't play well with tools that produce two files. It doesn't play well with tools that produce a directory tree of "dynamic" filenames (not known when you write the Makefile) These are covered by pattern matching and prerequisites, which were backported from mk(1) into GNU Make (Albeit they changed the syntax to make it incomprehensible). > (2) Most makefiles have bugs, especially when you do make -j (parallel builds). It's basically like writing a C program with a bunch of threads racing on global variables -- your Make targets will often be racing on the same file, leading to non-deterministic bugs. mk(1) makes it easier to avoid bugs and easier to comprehend what the makefile is doing, it also allows you to invoke programming languages for specific targets as a feature of the Makefile, and other goodies :)
  - chubot 6y ago
    I guess you're talking about the Plan 9 tool: https://9fans.github.io/plan9port/man/man1/mk.html https://9fans.github.io/plan9port/man/man1/mk.html The GNU make pattern rules are OK, but they don't handle some pretty simple scenarios I've encountered in practice. One was building the Oil shell blog, where the names of the posts are dynamic, and not hard-coded in the make file. The other one is build variants. Say I have a custom tool with a flag. I want to build: - two binaries: oil and opy - two variants, opt and dbg I want them to look like: _build/oil.opt _build/oil.dbg _build/opy.opt _build/opy.dbg The % pattern rules apparently can't handle this. You need some kind of Turing complete "metaprogramming". (CMake and autoconf offer that, but mostly for different reasons.) This is a real problem that came up in: https://github.com/oilshell/oil/blob/master/Makefile https://github.com/oilshell/oil/blob/master/Makefile I plan to switch to a generated Ninja script to solve some of these problems.

chubot 6y ago

Yeah, as mentioned in the posts, make targets that are verbs rather than nouns should really be .PHONY, but most people forget that. While the idea of having Make's dependency engine is nice in theory, it falls down for one-off automation in my experience. For a couple reasons: (1) Make's dependency model has some well known deficiencies. It doesn't play well with tools that produce two files. It doesn't play well with tools that produce a directory tree of "dynamic" filenames (not known when you write the Makefile) (2) Most makefiles have bugs, especially when you do make -j (parallel builds). It's basically like writing a C program with a bunch of threads racing on global variables -- your Make targets will often be racing on the same file, leading to non-deterministic bugs. ----- So IMO it's better to mostly stick with the sequential model of shell for this kind of project automation. But shell can invoke make! When you know your dependencies, invoke them from run.sh! And when you're REALLY sure it's correct, invoke make -j :) In other words shell is my default, and make is only for when I want to spend the effort to write dependencies -- which is quite difficult, because Make provides you virtually no help with that. Bugs in dependencies are common and hard to find. If you're trained to run "make clean", then that's a symptom of a bug in the build specification. Shell "gets shit done" without these types of bugs. Debugging a shell script is very easy compared with debugging a makefile. The remaining problems with shell will hopefully be fixed by https://oilshell.org/ https://oilshell.org/ :) I do want to add some dependency support, but I didn't get to it: Shell, Awk, and Make Should Be Combined http://www.oilshell.org/blog/2016/11/13.html http://www.oilshell.org/blog/2016/11/13.html Make is just a small elaboration on the shell model (concurrent processes and files), but unfortunately it's implemented as a completely separate tool that shells out to shell! facepalm

fao_ 6y ago
> (1) Make's dependency model has some well known deficiencies. It doesn't play well with tools that produce two files. It doesn't play well with tools that produce a directory tree of "dynamic" filenames (not known when you write the Makefile) These are covered by pattern matching and prerequisites, which were backported from mk(1) into GNU Make (Albeit they changed the syntax to make it incomprehensible). > (2) Most makefiles have bugs, especially when you do make -j (parallel builds). It's basically like writing a C program with a bunch of threads racing on global variables -- your Make targets will often be racing on the same file, leading to non-deterministic bugs. mk(1) makes it easier to avoid bugs and easier to comprehend what the makefile is doing, it also allows you to invoke programming languages for specific targets as a feature of the Makefile, and other goodies :)
- chubot 6y ago
  I guess you're talking about the Plan 9 tool: https://9fans.github.io/plan9port/man/man1/mk.html https://9fans.github.io/plan9port/man/man1/mk.html The GNU make pattern rules are OK, but they don't handle some pretty simple scenarios I've encountered in practice. One was building the Oil shell blog, where the names of the posts are dynamic, and not hard-coded in the make file. The other one is build variants. Say I have a custom tool with a flag. I want to build: - two binaries: oil and opy - two variants, opt and dbg I want them to look like: _build/oil.opt _build/oil.dbg _build/opy.opt _build/opy.dbg The % pattern rules apparently can't handle this. You need some kind of Turing complete "metaprogramming". (CMake and autoconf offer that, but mostly for different reasons.) This is a real problem that came up in: https://github.com/oilshell/oil/blob/master/Makefile https://github.com/oilshell/oil/blob/master/Makefile I plan to switch to a generated Ninja script to solve some of these problems.