7 ms·

Make is an absolutely wonderful, wonderful tool. Most of the common criticisms of Make are actually criticisms of autoconf, which I agree is a hideous tool. (O

Make is an absolutely wonderful, wonderful tool. Most of the common criticisms of Make are actually criticisms of autoconf, which I agree is a hideous tool. (On the other hand, autoconf is a tool intended to address a hideous problem, so perhaps that's inevitable). A lot of the more recent build tools I see are largely reinventions of Make. Make is very widely supported (most basic projects can make do[0] with portable Makefiles, though GNU Make is also available for most systems), and its syntax is actually very easy to grasp and manipulate[1]. [0] no pun intended [1] Most projects only need a very small subset of what Make has to offer, anyway, and that can be learned in a matter of minutes.

lifthrasiir 13y ago
Make is a great tool for the intended uses. Unfortunately, many uses of Make-like tools also require the unintended uses; my favorite example is the auto-dependencies, which is required by most compilation tasks and still tedious to get it right [1]. It also (mostly) lacks modern programmable interfaces, which led to Makefiles laden with hacks and mumbo-jumbos. At least for the compilation tasks, I see Make is clearly being outdated. [1] Something like http://mad-scientist.net/make/autodep.html http://mad-scientist.net/make/autodep.html
- haberman 13y ago
  The auto-dependency problem is a perfect example of how there is room for improvement in make-land. When something that basically everyone wants to do takes a long web page to describe a complicated method that sort of works, that is a compelling indicator that there must be a better way. Incidentally, I am working on a make-replacement that is intended to address this and similar needs. It's still in very early stages, but I think I'm onto something. The idea is to write a very low-level tool that leaves out most of Make's higher-level abstractions. In my tool there are no recipes, no implicit rules, no variables or variable substitutions, no conditionals, etc. The input to my tool is just the precise specifications of the commands you need to run, their inputs and outputs (so a precise dependency graph can be calculated), and with everything fully-expanded already. The idea, then, is that whatever higher-level abstractions you want (if any) you build into a higher-level tool. The higher-level tool just spits out a file describing the list of tasks. Then the higher-level tool can worry about the higher-level structure, policy, configuration, etc. of your project. So instead of writing things like implicit rules in Makefiles, you just write some code that explicitly generates tasks. For example, with Make, you might write an implicit rule like this: %.o : %.c $(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@ Then Make magically decides which output files match this implicit rule. My idea is that, instead of this, explicitly apply your rules to your inputs. Your build system could instead be a Ruby/Python/etc. script that looks something like this: for c_file, o_file in files: tasks.append(Task( target=o_file, source=c_file, command="gcc -c %s -o %s", c_file, o_file )) print(tasks) I think this is much more convenient than having to program in Make and learn its quirky abstractions. I have an elegant solution for the auto-dependency problem (unproven, but I think it's promising). The idea is that your dependency-calculating tasks are still just tasks, but the tool knows how to integrate the calculated dependencies back into the overall dependency graph. These dependency-generating tasks depend on the files they are generating dependencies for, so it is all part of the unified dependency graph. If you're interested, star my project and follow my blog (where I will make any announcements about it): https://github.com/haberman/taskforce https://github.com/haberman/taskforce http://blog.reverberate.org/ http://blog.reverberate.org/
  - llimllib 13y ago
    I did something similar with my python make-alike, pub[1], where you can have something like: def outname(f): return f[:-1] + "o" @file_rule("*.c", outname) def compile(f): #do the compiling here Here's an example of file_rule in use: https://github.com/llimllib/Newsite/blob/master/pub.py#L27 https://github.com/llimllib/Newsite/blob/master/pub.py#L27 [1]: https://github.com/llimllib/pub/ https://github.com/llimllib/pub/
  - sigil 13y ago
    You should check out djb's redo [1], specifically the implementation by apenwarr [2]. It's close to the separation of concerns you're looking for: programs written in your language of choice enumerate the dependency graph, an external tool collects them and executes its way through the dependency graph towards your targets. I agree with you that Make's pattern matching facilities are ugly and could be improved upon. I tried something of the sort [3] with redo, and it worked well, but felt too alien. From what I've seen at big companies with huge builds, the "dependency graph itself changed, now what do we rebuild" problem is the killer problem. I bet you've seen instances of that at Google. [1] http://cr.yp.to/redo.html http://cr.yp.to/redo.html [2] https://github.com/apenwarr/redo https://github.com/apenwarr/redo [3] https://news.ycombinator.com/item?id=7464047 https://news.ycombinator.com/item?id=7464047
    - a8da6b0c91d 13y ago
      See also remake. It's a bit simpler, I think. https://github.com/rocky/remake https://github.com/rocky/remake
  - Someone 13y ago
    "Your build system could instead be a Ruby/Python/etc. script" So, to compile a slightly complicated project, I would need Ruby for the main project, Python for a library it builds on, Haskell for another library it builds on, etc? In theory, that's not a new problem, but I think that, by removing the common language that make provides, it will become much more common in practice. At the extreme, it would only be natural, but disastrous, for the guys writing language X to use language X to write their make file. Result: you need to bootstrap language X through a zillion versions to port it to a new system, or mess around trying to build its parts by hand.
- m_mueller 13y ago
  Honestly I don't get why auto dependencies are even needed. IMO splitting up manually managed dependency rules into multiple, manually ordered makefiles and importing them in a master makefile, should work just fine. It is a lot of work to switch to from a managed environment, but if you start with this from the beginning it's not a big deal adding dependencies whenever they come up. As a developer I like having a nicely formatted dependency documentation, including comments, in one place instead of having to look at what autoconf & co are spitting out.
  - strager 13y ago
    So every time I add an import or #include, I need to update the dep graph manually? That may work for larger teams, but that would seriously hurt productivity for smaller projects. I think the main issue with approaches like yours is that there's no way to verify correctness, so it's easy to introduce bugs.
    - m_mueller 13y ago
      - small projects should also lead to small dependency graphs, so I don't really get why it's bad there. - I'm just not a big fan of automating the configuration of a build system - you build/put yet another machine around a machine. As long as it works - ok, but as soon as you run into unexpected behavior it will become a huge time sink to debug. When you update the Makefile contents manually you take ownership over everything and you get a deeper understanding on how everything works together, right down to the command parameters sent to compiler and linker. If you don't want to do that I'd suggest switching to something like cmake - if it has to 'magic', at least have the behavior specified in a coherent manner.
  - stormbrew 13y ago
    How does this lead to having dependency information in one place? It puts it in at least two. The file and the makefile. Not at all sure what autoconf has to do with anything, autodeps are a completely separate problem space.
    - 13y ago
zimbatm 13y ago
The GNU Make manual pages are really great. I use them all the time when I need to be remembered of some of the syntax or features. https://www.gnu.org/software/make/manual/make.html https://www.gnu.org/software/make/manual/make.html
sdegutis 13y ago
I was surprised when I learned `make` and how simple it mostly is. Unfortunately it eschews man-pages in favor of info-pages which were not as pleasant to learn how to use, just to learn how to use `make`.
- massysett 13y ago
  Any time you want to groan about a tool's use of info, see if the BSDs have a similar tool. Most GNU tools simply implement an ancient UNIX tool so it's not like GNU has a monopoly on them. So in this case see FreeBSD make: http://www.freebsd.org/cgi/man.cgi?query=make&sektion=1 http://www.freebsd.org/cgi/man.cgi?query=make&sektion=1 which points you to this: https://www.freebsd.org/doc/en/books/pmake/ https://www.freebsd.org/doc/en/books/pmake/ But really, if you just hate using info(1), always remember that GNU also puts the pages on the web in HTML multichunks, HTML one big page, and PDF, so there's no reason to dodge a GNU tool just because you hate info(1). https://www.gnu.org/software/make/manual/ https://www.gnu.org/software/make/manual/
  - mickeyp 13y ago
    Another option, if you're an Emacs user, is to read it in Emacs. Emacs's info reader is very good.
    - theOnliest 13y ago
      If someone's argument is that info is hard to learn, it's pretty unlikely they're also an Emacs user (the standalone info uses emacs keybindings by default).
      - asb 13y ago
        pinfo is an easier to use viewer for info files (it tries to adopt a lynx-like style).
  - dmckeon 13y ago
    Or put something like this into a shell script or function: info --subnodes "$@" 2>&1 | less and info(1) output will look a lot like man(1) output.
thwest 13y ago
For me the problem is the argument formatting and dependency chasing for clang-osx and msvc/intelcc-windows

lifthrasiir 13y ago

Make is a great tool for the intended uses. Unfortunately, many uses of Make-like tools also require the unintended uses; my favorite example is the auto-dependencies, which is required by most compilation tasks and still tedious to get it right [1]. It also (mostly) lacks modern programmable interfaces, which led to Makefiles laden with hacks and mumbo-jumbos. At least for the compilation tasks, I see Make is clearly being outdated. [1] Something like http://mad-scientist.net/make/autodep.html http://mad-scientist.net/make/autodep.html

haberman 13y ago
The auto-dependency problem is a perfect example of how there is room for improvement in make-land. When something that basically everyone wants to do takes a long web page to describe a complicated method that sort of works, that is a compelling indicator that there must be a better way. Incidentally, I am working on a make-replacement that is intended to address this and similar needs. It's still in very early stages, but I think I'm onto something. The idea is to write a very low-level tool that leaves out most of Make's higher-level abstractions. In my tool there are no recipes, no implicit rules, no variables or variable substitutions, no conditionals, etc. The input to my tool is just the precise specifications of the commands you need to run, their inputs and outputs (so a precise dependency graph can be calculated), and with everything fully-expanded already. The idea, then, is that whatever higher-level abstractions you want (if any) you build into a higher-level tool. The higher-level tool just spits out a file describing the list of tasks. Then the higher-level tool can worry about the higher-level structure, policy, configuration, etc. of your project. So instead of writing things like implicit rules in Makefiles, you just write some code that explicitly generates tasks. For example, with Make, you might write an implicit rule like this: %.o : %.c $(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@ Then Make magically decides which output files match this implicit rule. My idea is that, instead of this, explicitly apply your rules to your inputs. Your build system could instead be a Ruby/Python/etc. script that looks something like this: for c_file, o_file in files: tasks.append(Task( target=o_file, source=c_file, command="gcc -c %s -o %s", c_file, o_file )) print(tasks) I think this is much more convenient than having to program in Make and learn its quirky abstractions. I have an elegant solution for the auto-dependency problem (unproven, but I think it's promising). The idea is that your dependency-calculating tasks are still just tasks, but the tool knows how to integrate the calculated dependencies back into the overall dependency graph. These dependency-generating tasks depend on the files they are generating dependencies for, so it is all part of the unified dependency graph. If you're interested, star my project and follow my blog (where I will make any announcements about it): https://github.com/haberman/taskforce https://github.com/haberman/taskforce http://blog.reverberate.org/ http://blog.reverberate.org/
- llimllib 13y ago
  I did something similar with my python make-alike, pub[1], where you can have something like: def outname(f): return f[:-1] + "o" @file_rule("*.c", outname) def compile(f): #do the compiling here Here's an example of file_rule in use: https://github.com/llimllib/Newsite/blob/master/pub.py#L27 https://github.com/llimllib/Newsite/blob/master/pub.py#L27 [1]: https://github.com/llimllib/pub/ https://github.com/llimllib/pub/
- sigil 13y ago
  You should check out djb's redo [1], specifically the implementation by apenwarr [2]. It's close to the separation of concerns you're looking for: programs written in your language of choice enumerate the dependency graph, an external tool collects them and executes its way through the dependency graph towards your targets. I agree with you that Make's pattern matching facilities are ugly and could be improved upon. I tried something of the sort [3] with redo, and it worked well, but felt too alien. From what I've seen at big companies with huge builds, the "dependency graph itself changed, now what do we rebuild" problem is the killer problem. I bet you've seen instances of that at Google. [1] http://cr.yp.to/redo.html http://cr.yp.to/redo.html [2] https://github.com/apenwarr/redo https://github.com/apenwarr/redo [3] https://news.ycombinator.com/item?id=7464047 https://news.ycombinator.com/item?id=7464047
  - a8da6b0c91d 13y ago
    See also remake. It's a bit simpler, I think. https://github.com/rocky/remake https://github.com/rocky/remake
- Someone 13y ago
  "Your build system could instead be a Ruby/Python/etc. script" So, to compile a slightly complicated project, I would need Ruby for the main project, Python for a library it builds on, Haskell for another library it builds on, etc? In theory, that's not a new problem, but I think that, by removing the common language that make provides, it will become much more common in practice. At the extreme, it would only be natural, but disastrous, for the guys writing language X to use language X to write their make file. Result: you need to bootstrap language X through a zillion versions to port it to a new system, or mess around trying to build its parts by hand.
m_mueller 13y ago
Honestly I don't get why auto dependencies are even needed. IMO splitting up manually managed dependency rules into multiple, manually ordered makefiles and importing them in a master makefile, should work just fine. It is a lot of work to switch to from a managed environment, but if you start with this from the beginning it's not a big deal adding dependencies whenever they come up. As a developer I like having a nicely formatted dependency documentation, including comments, in one place instead of having to look at what autoconf & co are spitting out.
- strager 13y ago
  So every time I add an import or #include, I need to update the dep graph manually? That may work for larger teams, but that would seriously hurt productivity for smaller projects. I think the main issue with approaches like yours is that there's no way to verify correctness, so it's easy to introduce bugs.
  - m_mueller 13y ago
    - small projects should also lead to small dependency graphs, so I don't really get why it's bad there. - I'm just not a big fan of automating the configuration of a build system - you build/put yet another machine around a machine. As long as it works - ok, but as soon as you run into unexpected behavior it will become a huge time sink to debug. When you update the Makefile contents manually you take ownership over everything and you get a deeper understanding on how everything works together, right down to the command parameters sent to compiler and linker. If you don't want to do that I'd suggest switching to something like cmake - if it has to 'magic', at least have the behavior specified in a coherent manner.
- stormbrew 13y ago
  How does this lead to having dependency information in one place? It puts it in at least two. The file and the makefile. Not at all sure what autoconf has to do with anything, autodeps are a completely separate problem space.
  - 13y ago

zimbatm 13y ago

The GNU Make manual pages are really great. I use them all the time when I need to be remembered of some of the syntax or features. https://www.gnu.org/software/make/manual/make.html https://www.gnu.org/software/make/manual/make.html

sdegutis 13y ago

I was surprised when I learned `make` and how simple it mostly is. Unfortunately it eschews man-pages in favor of info-pages which were not as pleasant to learn how to use, just to learn how to use `make`.

massysett 13y ago
Any time you want to groan about a tool's use of info, see if the BSDs have a similar tool. Most GNU tools simply implement an ancient UNIX tool so it's not like GNU has a monopoly on them. So in this case see FreeBSD make: http://www.freebsd.org/cgi/man.cgi?query=make&sektion=1 http://www.freebsd.org/cgi/man.cgi?query=make&sektion=1 which points you to this: https://www.freebsd.org/doc/en/books/pmake/ https://www.freebsd.org/doc/en/books/pmake/ But really, if you just hate using info(1), always remember that GNU also puts the pages on the web in HTML multichunks, HTML one big page, and PDF, so there's no reason to dodge a GNU tool just because you hate info(1). https://www.gnu.org/software/make/manual/ https://www.gnu.org/software/make/manual/
- mickeyp 13y ago
  Another option, if you're an Emacs user, is to read it in Emacs. Emacs's info reader is very good.
  - theOnliest 13y ago
    If someone's argument is that info is hard to learn, it's pretty unlikely they're also an Emacs user (the standalone info uses emacs keybindings by default).
    - asb 13y ago
      pinfo is an easier to use viewer for info files (it tries to adopt a lynx-like style).
- dmckeon 13y ago
  Or put something like this into a shell script or function: info --subnodes "$@" 2>&1 | less and info(1) output will look a lot like man(1) output.

thwest 13y ago

For me the problem is the argument formatting and dependency chasing for clang-osx and msvc/intelcc-windows