4 ms·

I sometimes use GNU Make to fire off custom code generators, before the files are handed off to other parts of the toolchain which can have their own complicate

I sometimes use GNU Make to fire off custom code generators, before the files are handed off to other parts of the toolchain which can have their own complicated dependency management. This works quite well. The one annoying problem that I often encounter is that Make does not handle multiple targets (i.e. the code generator generates multiple files, e.g. 'file1.h', 'file1.cpp', 'file2.h', 'file2.cpp', 'test.cpp'). I usually end up inserting a bunch of .PHONY targets, which causes unnecessary evaluation of the dependency graph, but at least it works, instead of breaking in seemingly random ways. My other use of Makefiles is to capture small (< ~5 lines) of bash, python or such scripts for doing certain things within a directory. I find that to be more efficient than documenting that sort of info in a README.md file.

gumby 7y ago
Not sure what you mean by "Make does not handle multiple targets". You can definitely do `make foo bar` and it will run the recipes for both foo and bar. You can also write a recipe with multiple prerequisites (which could be the result of a variable expansion). Curious about the limitation; could be there's a way around it or that I never ran into it.
- aidenn0 7y ago
  The poster is talking about a single command that generates multiple outputs. e.g. a codegen tool that generates "foo.c" and "foo.h"
  - gumby 7y ago
    I'm still not understanding the problem (sorry for being obtuse). If you have a program called `makes-two-files` you could have a recipe: foo.c foo.h: predecessor_of_foo @makes-two=files foo There are other ways to do this in Make as well. I use Make extensively to drive CMake which makes all sorts of products, so my confusion comes from thinking this works fine (works for my purpose, obviously, but I'm interested in other possibilities).
    - aidenn0 7y ago
      Consider: foo.c foo.h: foo.in some-tool foo.in If you have some targets that depend on foo.c and other targets that depend on foo.h and you build in parallel, some-tool will run twice, possibly at the same time as something is reading the output, possibly causing problems. [edit] The workaround I've used is something like: foo.c: foo.in some-tool foo.in foo.h: foo.c true Note that the "true" command is important because GNU make will search a library of implicit rules if you have no recipes for a particular target. In general multiple output files are even harder with some make replacements, because most make replacements try to be smarter about build outputs. redo, for example cannot handle this, but tup can.
      - gumby 7y ago
        Ah, now I understand the problem, thanks! Indeed I've stayed away from -j because it's very hard to figure out how recipes may interact when you have a complex Makefile (and with a simple one -j rarely helps much). I like your fix; even without -j, if some-tool is expensive to run you don't want it run redundantly.
        aidenn0 7y ago
        I use make (or redo) with -j for parallelizing tasks all the time. It's a really great tool for that. If you specify your dependencies correctly, then -j will not break anything (other than out-of-memory when parallelizing beyond your memory resources). Many implementations of redo have an option that will randomize the order of building dependencies; it's a useful way for testing if you have your dependencies fully specified or not. For non one-off tasks where incremental builds are desired, I've been really liking tup. It will automatically catch many mistakes with dependencies for you up-front, saving much hair-pulling down the road. [edit] An example of what I use make/redo parallelization for in one-off scripts is bulk conversion of images or sound files; it's usually 1 file in 1 file out and rarely are the tools already heavily parallel.
  - Ambroisie 7y ago
    Isn't the solution to that problem simply to use stamp files You declare foo-stamp as a dependency of both foo.c and foo.h. foo-stamp itself is dependent on the file used to generate those foo.* files. The recipe for foo-stamp invokes the code generator which updates the files, then you `touch foo-stamp`at the end of the recipe.
    - pstuart 7y ago
      > Isn't the solution to that problem simply to use stamp files TIL a new make trick. Thanks!
- emmelaich 7y ago
  Not sure what the op has in mind precisely, but Makefiles for modern Fortran with modules is annoying. Do a search for "makefile fortran modules" or similar and you'll find a bunch of articles, including https://stackoverflow.com/questions/35234003/how-to-create-a-makefile-for-a-fortran-program-using-modules https://stackoverflow.com/questions/35234003/how-to-create-a... I tried helping a friend with some Fortran but eventually gave up due to this issue and GNU Fortran's insistence on using the standard % instead of "." for structure syntax.

gumby 7y ago

Not sure what you mean by "Make does not handle multiple targets". You can definitely do `make foo bar` and it will run the recipes for both foo and bar. You can also write a recipe with multiple prerequisites (which could be the result of a variable expansion). Curious about the limitation; could be there's a way around it or that I never ran into it.

aidenn0 7y ago
The poster is talking about a single command that generates multiple outputs. e.g. a codegen tool that generates "foo.c" and "foo.h"
- gumby 7y ago
  I'm still not understanding the problem (sorry for being obtuse). If you have a program called `makes-two-files` you could have a recipe: foo.c foo.h: predecessor_of_foo @makes-two=files foo There are other ways to do this in Make as well. I use Make extensively to drive CMake which makes all sorts of products, so my confusion comes from thinking this works fine (works for my purpose, obviously, but I'm interested in other possibilities).
  - aidenn0 7y ago
    Consider: foo.c foo.h: foo.in some-tool foo.in If you have some targets that depend on foo.c and other targets that depend on foo.h and you build in parallel, some-tool will run twice, possibly at the same time as something is reading the output, possibly causing problems. [edit] The workaround I've used is something like: foo.c: foo.in some-tool foo.in foo.h: foo.c true Note that the "true" command is important because GNU make will search a library of implicit rules if you have no recipes for a particular target. In general multiple output files are even harder with some make replacements, because most make replacements try to be smarter about build outputs. redo, for example cannot handle this, but tup can.
    - gumby 7y ago
      Ah, now I understand the problem, thanks! Indeed I've stayed away from -j because it's very hard to figure out how recipes may interact when you have a complex Makefile (and with a simple one -j rarely helps much). I like your fix; even without -j, if some-tool is expensive to run you don't want it run redundantly.
      - aidenn0 7y ago
        I use make (or redo) with -j for parallelizing tasks all the time. It's a really great tool for that. If you specify your dependencies correctly, then -j will not break anything (other than out-of-memory when parallelizing beyond your memory resources). Many implementations of redo have an option that will randomize the order of building dependencies; it's a useful way for testing if you have your dependencies fully specified or not. For non one-off tasks where incremental builds are desired, I've been really liking tup. It will automatically catch many mistakes with dependencies for you up-front, saving much hair-pulling down the road. [edit] An example of what I use make/redo parallelization for in one-off scripts is bulk conversion of images or sound files; it's usually 1 file in 1 file out and rarely are the tools already heavily parallel.
- Ambroisie 7y ago
  Isn't the solution to that problem simply to use stamp files You declare foo-stamp as a dependency of both foo.c and foo.h. foo-stamp itself is dependent on the file used to generate those foo.* files. The recipe for foo-stamp invokes the code generator which updates the files, then you `touch foo-stamp`at the end of the recipe.
  - pstuart 7y ago
    > Isn't the solution to that problem simply to use stamp files TIL a new make trick. Thanks!
emmelaich 7y ago
Not sure what the op has in mind precisely, but Makefiles for modern Fortran with modules is annoying. Do a search for "makefile fortran modules" or similar and you'll find a bunch of articles, including https://stackoverflow.com/questions/35234003/how-to-create-a-makefile-for-a-fortran-program-using-modules https://stackoverflow.com/questions/35234003/how-to-create-a... I tried helping a friend with some Fortran but eventually gave up due to this issue and GNU Fortran's insistence on using the standard % instead of "." for structure syntax.