4 ms·

It is so trivial, it takes you 10 minutes to learn. Less even, here is an attempt: foo.o: foo.c clang -c -o foo.o foo.c This builds a file called

by anyfoo 2y ago

It is so trivial, it takes you 10 minutes to learn. Less even, here is an attempt: foo.o: foo.c clang -c -o foo.o foo.c This builds a file called foo.o, if foo.c is newer than foo.o. Imagine that there is an identical bar.o as well, that builds it from bar.c. fooexec: foo.o bar.o clang -o fooexec foo.o bar.o This links together foo.o and bar.o into a file called fooexec, if at least one of foo.o or bar.o is newer (which in turn means that any of its respective dependencies changed; e.g. change bar.c, and bar.o and then fooexec will rebuild). Now type this into the shell: make fooexec Congratulations! You have a fully built fooexec. Now change bar.c, type it again, and like magic, you have a rebuilt bar.o and fooexec. foo.o will stay put, it’s up to date! But of course all the .c -> .o rules look the same, so instead of writing a new one for every file, you'd use simple wildcards, and now the whole, entire Makefile is just: %.o: %.c clang -c -o $@ $< fooexec: foo.o bar.o clang -o fooexec foo.o bar.o Do $@ and $< look arcane at first glance? Yes. Is what they do simple, though? Also yes. (If your complaint is that you don't know what those "clang" lines do, then you don't know how to use the C compiler, and will have the same problem when writing a python script.) It pains me if that is not immediately obvious. And it pains me even more that someone would want to write a boilerplate heavy, complex python program, that rebuilds everything all the time, instead of just learning this very simple concept.

IgorPartola 2y ago
This is an excellent explanation of make! The only thing I will add is the slightly more abstract thought that make is meant to build files out of other files based on timestamps of the source files. And the target files can themselves become source files, which means that you can build dependency graphs. Lastly, it has “caching” built in: if bar.o is newer than bar.c, don’t rebuild it. If it doesn’t exist or is older, do rebuild it. So when you change one file, you don’t recompile the entire damn tree. It also has “phony” targets that aren’t files. The canonical example of this is “clean” which basically just does something like rm -rf *.o fooexec This is what the JavaScript tooling people have been chasing for the past 15 years and can’t seem to grasp. Someone actually did build a make-based JS build system I believe but it never got popular because it wasn’t written in JavaScript.
- fuzztester 2y ago
  >Lastly, it has “caching” built in: A nitpick, but I don't think that should really be called caching, although it kind of is, in a looser sense. It's just comparing the file timestamps of the .o and .c files, and building the .o from the .c, if the .c is newer, or if the .o doesn't exist.
  - IgorPartola 2y ago
    I mean it’s caching using a key derivation scheme which uses file names rather than file contents. It might not be sophisticated caching but it is caching.
CJefferson 2y ago
Saying this is an 'incremental makefile' isn't really correct, as changes to header files aren't going to lead to rebuilding. So, either you need to manually keep references to which .h files you include in your Makefiles up to date, or start worrying about M / MM / MG / MP, and of course you'd like those to be re-run when you change your files, and suddenly your Makefile is an awful lot less simple. This is the main reason I stopped teaching Makefiles in intro to C courses -- students often lost entire afternoons to forgetting this stuff, and failing to realise their mistake. It really shouldn't be any person's job to keep this up to date, when the computer knows it.
- fuhsnn 2y ago
  Yes, this is quite important for C projects as it leads to subtle bugs. I also long for a way to let make detect a change of $CC (new build of the compiler itself) in the context of debugging a compiler against third party makefile projects.
  - anyfoo 2y ago
    Have a step that takes the SHA256 or similar of $CC, make that a dependency. You’d need slightly more advanced make features if you want to make it “pretty” (and not just, say, touch a file through some shell code comparison), but this is a slightly more advanced request. Or potentially just specify the path to $CC itself as dependency? Presumably, if the C compiler changes, the timestamp of its executable does too. (Bad if it goes backwards, though.)
    - yjftsjthsd-h 2y ago
      > Have a step that takes the SHA256 or similar of $CC, make that a dependency. Careful, one more step and we'll start recreating nix in Makefiles;) (Wait, could we? Should we?)
  - CJefferson 2y ago
    I often use a Python program called 'fabricate', which basically checks every file a command opens, and re-runs the command if any touched file changes. I love it, and I hoped this would become the future of how build-systems are built -- you could automatically parallelise, rebuild, make clean, everything, just by tracking which files are read, and written. It does add a bit of overhead, but I'd be willing to pay that for simplicity. Unfortunately, it doesn't seem to have caught on.
saghm 2y ago
> Do $@ and $< look arcane at first glance? Yes. Is what they do simple, though? And yet, I still don't actually know what `$@` and `$<` actually mean outside of your example. I only can assume that somehow that substitutes in `foo.o` and `foo.c` because you literally put the non-magic versions right below it (and above it a few paragraphs above as well), but I'm honestly not even positive that I'd run `make foo.o` rather than `make foo` with your example, and I feel fairly confident in guessing that this would _not_ compile into a binary called `fooexec` like the target you defined above. I don't have an idea whatsoever what I'd need to put in the target to be able to substitute in something like that because the fact that the first `%` somehow becomes `$@` and the second becomes `$<` doesn't follow any obvious pattern I can use to figure out what the third symbol would be, or even if there is one. The problem is that "simple" is not the same as "intuitive" or "discoverable". Yes, I could probably put in the effort to go and find out these things, but why would you expect someone to be motivated to do that by someone telling you that it's "trivial" and it "pains" them as they explain things in a way that doesn't actually clarify things in a way that helps me? If you actually want to try to make a strong case that this is something worth learning for people, repeatedly referring to it in ways like "very simple" and "so trivial" and "immediately obvious" is counterproductive. As an aside, I think it's also a bit of leap to assume that they're "rebuilding everything all the time". It looks to me like the build.py script is only needed a single time to bootstrap from the original compiler to this one, and rebuilding every single C file in the new compiler rather than using the artifacts built with the original one is kind of the whole point. After that's done once, I don't see why the new compiler couldn't be used via the Makefile. If you're complaining about Python build scripts in general rather than this specific one, I don't know why you assume they can't be written to check the timestamps on the files to see if a rebuild is necessary before doing so. That's exactly what `make` does, and hey, it's a very simple concept!
- mturmon 2y ago
  Since you’re asking, $@ is the “target” of the build, that is, what you are aiming to build (left of the colon in the rule). This is why it looks like a bullseye. And $< is of course the thing the target depends on. It’s the input, like standard Unix notation for stdin. Hopefully this helps?
- 2y ago
dzaima 2y ago
I've written a good amount of makefiles (and even one rather complex one.. before replacing it with a thing written in an actually sane language, bootstrapped by a stripped down version of the makefile), and I still mix up $@ $< $> or whatever else my brain comes up with. And this is coming from someone deep into array languages, which have entire sets of unicode glyphs. For how often one typically writes/looks at build scripts, remembering the specifics about makefiles is quite likely to not be worth the effort. Makefiles start becoming incredibly awful if you want to do anything outside of pure computations in the separate targets. Making the target directory requires either putting the mkdir call in each separate built thing (awfully redundant), introducing a whole another rule (and still adding it as a dependency on every rule that'll be outputting to said directory), doing some recursive make, or running it outside of rules (but putting it under a proper if statement is likely pain). That's a bunch of awful solutions to something that really really really should just be a single calls to mkdir in any sane system. Another horrifically awful thing is if you want some reused data across rules (e.g. pkg-config results, current OS/arch). You can put them as globals, but that'll make them be calculated always, even if no build target needs them, unless you put them under an if, which is again non-trivial to get the condition correctly for. This becomes super bad if you do recursive make stuff. More generally, makefiles start out simple, but doing anything that's not strictly simple is gonna be pain, at the very least requiring looking at docs of a thing you'll probably never use again, and debuggability is also very low (amusingly, '-p' and '--debug' break on recursive make, seemingly losing some variables; and '-n' requires copying out & running the recursive call to get to the actually important part; and afaict none of those help with figuring out why a thing didn't build). And you're completely SOL if you wanted to do things like reorder compilations based on how long they previously took, or want a nice display of currently-compiling things.
- anyfoo 2y ago
  Fair. But as said, in my experience, they remain useful and simple for a pretty long time. I usually always start out with a simple Makefile. Only sometimes do I start with or eventually switch to something else because of the projects’ needs, but for most stuff (even stuff where the project itself is complex, i.e. embedded kernel stuff where different units are pasted together and whatnot), relatively simple Makefiles do their job pretty well. I also have a lot of phony targets. It really enjoy just being able to type: make restore kernel reset upload To bring a device into a sane state, build the kernel, reset the device, and upload the built firmware.
peterfirefly 2y ago
This is an amazingly good 2-minute introduction to makefiles. It would have been better without the last two sentences (and most of your comments on the topic). --- Something skipped over by your introduction (and most guides to makefiles) which very quickly either makes makefiles less short or much more complicated is header files. C files often depend on one or more header files that aren't system headers (this C compiler certainly does). They could be added to each individual rule by hand: codegen/codegen.o: codegen/codegen.c codegen/codegen.h linked_list.h libc.h $(CC) -c -o $@ $< or they could be put into a variable: HEADERS = codegen/codegen.h parser/parser.h parser/goto.h linked_list.h libc.h codegen/codegen.o: codegen/codegen.c $(HEADERS) $(CC) -c -o $@ $< which causes too much code to be recompiled every time a header file is changed. Or we could do something really fancy and get only the right header files added as dependencies for each compilation unit. That requires using a feature of the gcc/clang compilers (their preprocessors, really) that causes them to output a small makefile snippet that lists the exact header dependencies for a compilation unit (the '-MD' option). These snippets can then be included in the makefile. To make this look relatively clean requires some fancy macro use that is waaaay beyond what people can learn in ten minutes. This is an extremely common use case and it is a bit of an embarrassment that GNU make doesn't handle it better.
- anyfoo 2y ago
  I did mention that in other comments, including the C preprocessor helping. make is completely language agnostic, so it cannot “guess” header dependencies. In make, you get the dependencies you specify (whether manual or generated), that’s what I wanted to show.
  - peterfirefly 2y ago
    Make makes the easy part easy but the rest quite hard -- which I think you agree with. It sure would be nice to have something like this: codegen/codegen.o: !codegen/codegen.c $(CC) -c $(CFLAGS) -o $@ $< '!' would run $(CC) with -MD to extract the header dependencies. Doing that back in 1995 would have increased the value and usability of GNU make immensely (and greatly reduced the need for all the other complexity it has).

by anyfoo 2y ago

IgorPartola 2y ago

This is an excellent explanation of make! The only thing I will add is the slightly more abstract thought that make is meant to build files out of other files based on timestamps of the source files. And the target files can themselves become source files, which means that you can build dependency graphs. Lastly, it has “caching” built in: if bar.o is newer than bar.c, don’t rebuild it. If it doesn’t exist or is older, do rebuild it. So when you change one file, you don’t recompile the entire damn tree. It also has “phony” targets that aren’t files. The canonical example of this is “clean” which basically just does something like rm -rf *.o fooexec This is what the JavaScript tooling people have been chasing for the past 15 years and can’t seem to grasp. Someone actually did build a make-based JS build system I believe but it never got popular because it wasn’t written in JavaScript.

fuzztester 2y ago
>Lastly, it has “caching” built in: A nitpick, but I don't think that should really be called caching, although it kind of is, in a looser sense. It's just comparing the file timestamps of the .o and .c files, and building the .o from the .c, if the .c is newer, or if the .o doesn't exist.
- IgorPartola 2y ago
  I mean it’s caching using a key derivation scheme which uses file names rather than file contents. It might not be sophisticated caching but it is caching.

CJefferson 2y ago

Saying this is an 'incremental makefile' isn't really correct, as changes to header files aren't going to lead to rebuilding. So, either you need to manually keep references to which .h files you include in your Makefiles up to date, or start worrying about M / MM / MG / MP, and of course you'd like those to be re-run when you change your files, and suddenly your Makefile is an awful lot less simple. This is the main reason I stopped teaching Makefiles in intro to C courses -- students often lost entire afternoons to forgetting this stuff, and failing to realise their mistake. It really shouldn't be any person's job to keep this up to date, when the computer knows it.

fuhsnn 2y ago
Yes, this is quite important for C projects as it leads to subtle bugs. I also long for a way to let make detect a change of $CC (new build of the compiler itself) in the context of debugging a compiler against third party makefile projects.
- anyfoo 2y ago
  Have a step that takes the SHA256 or similar of $CC, make that a dependency. You’d need slightly more advanced make features if you want to make it “pretty” (and not just, say, touch a file through some shell code comparison), but this is a slightly more advanced request. Or potentially just specify the path to $CC itself as dependency? Presumably, if the C compiler changes, the timestamp of its executable does too. (Bad if it goes backwards, though.)
  - yjftsjthsd-h 2y ago
    > Have a step that takes the SHA256 or similar of $CC, make that a dependency. Careful, one more step and we'll start recreating nix in Makefiles;) (Wait, could we? Should we?)
- CJefferson 2y ago
  I often use a Python program called 'fabricate', which basically checks every file a command opens, and re-runs the command if any touched file changes. I love it, and I hoped this would become the future of how build-systems are built -- you could automatically parallelise, rebuild, make clean, everything, just by tracking which files are read, and written. It does add a bit of overhead, but I'd be willing to pay that for simplicity. Unfortunately, it doesn't seem to have caught on.

saghm 2y ago

> Do $@ and $< look arcane at first glance? Yes. Is what they do simple, though? And yet, I still don't actually know what `$@` and `$<` actually mean outside of your example. I only can assume that somehow that substitutes in `foo.o` and `foo.c` because you literally put the non-magic versions right below it (and above it a few paragraphs above as well), but I'm honestly not even positive that I'd run `make foo.o` rather than `make foo` with your example, and I feel fairly confident in guessing that this would _not_ compile into a binary called `fooexec` like the target you defined above. I don't have an idea whatsoever what I'd need to put in the target to be able to substitute in something like that because the fact that the first `%` somehow becomes `$@` and the second becomes `$<` doesn't follow any obvious pattern I can use to figure out what the third symbol would be, or even if there is one. The problem is that "simple" is not the same as "intuitive" or "discoverable". Yes, I could probably put in the effort to go and find out these things, but why would you expect someone to be motivated to do that by someone telling you that it's "trivial" and it "pains" them as they explain things in a way that doesn't actually clarify things in a way that helps me? If you actually want to try to make a strong case that this is something worth learning for people, repeatedly referring to it in ways like "very simple" and "so trivial" and "immediately obvious" is counterproductive. As an aside, I think it's also a bit of leap to assume that they're "rebuilding everything all the time". It looks to me like the build.py script is only needed a single time to bootstrap from the original compiler to this one, and rebuilding every single C file in the new compiler rather than using the artifacts built with the original one is kind of the whole point. After that's done once, I don't see why the new compiler couldn't be used via the Makefile. If you're complaining about Python build scripts in general rather than this specific one, I don't know why you assume they can't be written to check the timestamps on the files to see if a rebuild is necessary before doing so. That's exactly what `make` does, and hey, it's a very simple concept!

mturmon 2y ago
Since you’re asking, $@ is the “target” of the build, that is, what you are aiming to build (left of the colon in the rule). This is why it looks like a bullseye. And $< is of course the thing the target depends on. It’s the input, like standard Unix notation for stdin. Hopefully this helps?
2y ago

dzaima 2y ago

I've written a good amount of makefiles (and even one rather complex one.. before replacing it with a thing written in an actually sane language, bootstrapped by a stripped down version of the makefile), and I still mix up $@ $< $> or whatever else my brain comes up with. And this is coming from someone deep into array languages, which have entire sets of unicode glyphs. For how often one typically writes/looks at build scripts, remembering the specifics about makefiles is quite likely to not be worth the effort. Makefiles start becoming incredibly awful if you want to do anything outside of pure computations in the separate targets. Making the target directory requires either putting the mkdir call in each separate built thing (awfully redundant), introducing a whole another rule (and still adding it as a dependency on every rule that'll be outputting to said directory), doing some recursive make, or running it outside of rules (but putting it under a proper if statement is likely pain). That's a bunch of awful solutions to something that really really really should just be a single calls to mkdir in any sane system. Another horrifically awful thing is if you want some reused data across rules (e.g. pkg-config results, current OS/arch). You can put them as globals, but that'll make them be calculated always, even if no build target needs them, unless you put them under an if, which is again non-trivial to get the condition correctly for. This becomes super bad if you do recursive make stuff. More generally, makefiles start out simple, but doing anything that's not strictly simple is gonna be pain, at the very least requiring looking at docs of a thing you'll probably never use again, and debuggability is also very low (amusingly, '-p' and '--debug' break on recursive make, seemingly losing some variables; and '-n' requires copying out & running the recursive call to get to the actually important part; and afaict none of those help with figuring out why a thing didn't build). And you're completely SOL if you wanted to do things like reorder compilations based on how long they previously took, or want a nice display of currently-compiling things.

anyfoo 2y ago
Fair. But as said, in my experience, they remain useful and simple for a pretty long time. I usually always start out with a simple Makefile. Only sometimes do I start with or eventually switch to something else because of the projects’ needs, but for most stuff (even stuff where the project itself is complex, i.e. embedded kernel stuff where different units are pasted together and whatnot), relatively simple Makefiles do their job pretty well. I also have a lot of phony targets. It really enjoy just being able to type: make restore kernel reset upload To bring a device into a sane state, build the kernel, reset the device, and upload the built firmware.

peterfirefly 2y ago

This is an amazingly good 2-minute introduction to makefiles. It would have been better without the last two sentences (and most of your comments on the topic). --- Something skipped over by your introduction (and most guides to makefiles) which very quickly either makes makefiles less short or much more complicated is header files. C files often depend on one or more header files that aren't system headers (this C compiler certainly does). They could be added to each individual rule by hand: codegen/codegen.o: codegen/codegen.c codegen/codegen.h linked_list.h libc.h $(CC) -c -o $@ $< or they could be put into a variable: HEADERS = codegen/codegen.h parser/parser.h parser/goto.h linked_list.h libc.h codegen/codegen.o: codegen/codegen.c $(HEADERS) $(CC) -c -o $@ $< which causes too much code to be recompiled every time a header file is changed. Or we could do something really fancy and get only the right header files added as dependencies for each compilation unit. That requires using a feature of the gcc/clang compilers (their preprocessors, really) that causes them to output a small makefile snippet that lists the exact header dependencies for a compilation unit (the '-MD' option). These snippets can then be included in the makefile. To make this look relatively clean requires some fancy macro use that is waaaay beyond what people can learn in ten minutes. This is an extremely common use case and it is a bit of an embarrassment that GNU make doesn't handle it better.

anyfoo 2y ago
I did mention that in other comments, including the C preprocessor helping. make is completely language agnostic, so it cannot “guess” header dependencies. In make, you get the dependencies you specify (whether manual or generated), that’s what I wanted to show.
- peterfirefly 2y ago
  Make makes the easy part easy but the rest quite hard -- which I think you agree with. It sure would be nice to have something like this: codegen/codegen.o: !codegen/codegen.c $(CC) -c $(CFLAGS) -o $@ $< '!' would run $(CC) with -MD to extract the header dependencies. Doing that back in 1995 would have increased the value and usability of GNU make immensely (and greatly reduced the need for all the other complexity it has).