10 ms·
Make is a great tool for the intended uses. Unfortunately, many uses of Make-like tools also require the unintended uses; my favorite example is the auto-depend
by lifthrasiir 13y ago
Make is a great tool for the intended uses. Unfortunately, many uses of Make-like tools also require the unintended uses; my favorite example is the auto-dependencies, which is required by most compilation tasks and still tedious to get it right [1]. It also (mostly) lacks modern programmable interfaces, which led to Makefiles laden with hacks and mumbo-jumbos. At least for the compilation tasks, I see Make is clearly being outdated.
[1] Something like http://mad-scientist.net/make/autodep.html http://mad-scientist.net/make/autodep.html
- haberman 13y agoThe auto-dependency problem is a perfect example of how there is room for improvement in make-land. When something that basically everyone wants to do takes a long web page to describe a complicated method that sort of works, that is a compelling indicator that there must be a better way. Incidentally, I am working on a make-replacement that is intended to address this and similar needs. It's still in very early stages, but I think I'm onto something. The idea is to write a very low-level tool that leaves out most of Make's higher-level abstractions. In my tool there are no recipes, no implicit rules, no variables or variable substitutions, no conditionals, etc. The input to my tool is just the precise specifications of the commands you need to run, their inputs and outputs (so a precise dependency graph can be calculated), and with everything fully-expanded already. The idea, then, is that whatever higher-level abstractions you want (if any) you build into a higher-level tool. The higher-level tool just spits out a file describing the list of tasks. Then the higher-level tool can worry about the higher-level structure, policy, configuration, etc. of your project. So instead of writing things like implicit rules in Makefiles, you just write some code that explicitly generates tasks. For example, with Make, you might write an implicit rule like this: %.o : %.c $(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@ Then Make magically decides which output files match this implicit rule. My idea is that, instead of this, explicitly apply your rules to your inputs. Your build system could instead be a Ruby/Python/etc. script that looks something like this: for c_file, o_file in files: tasks.append(Task( target=o_file, source=c_file, command="gcc -c %s -o %s", c_file, o_file )) print(tasks) I think this is much more convenient than having to program in Make and learn its quirky abstractions. I have an elegant solution for the auto-dependency problem (unproven, but I think it's promising). The idea is that your dependency-calculating tasks are still just tasks, but the tool knows how to integrate the calculated dependencies back into the overall dependency graph. These dependency-generating tasks depend on the files they are generating dependencies for, so it is all part of the unified dependency graph. If you're interested, star my project and follow my blog (where I will make any announcements about it): https://github.com/haberman/taskforce https://github.com/haberman/taskforce http://blog.reverberate.org/ http://blog.reverberate.org/
- llimllib 13y agoI did something similar with my python make-alike, pub[1], where you can have something like: def outname(f): return f[:-1] + "o" @file_rule("*.c", outname) def compile(f): #do the compiling here Here's an example of file_rule in use: https://github.com/llimllib/Newsite/blob/master/pub.py#L27 https://github.com/llimllib/Newsite/blob/master/pub.py#L27 [1]: https://github.com/llimllib/pub/ https://github.com/llimllib/pub/
- sigil 13y agoYou should check out djb's redo [1], specifically the implementation by apenwarr [2]. It's close to the separation of concerns you're looking for: programs written in your language of choice enumerate the dependency graph, an external tool collects them and executes its way through the dependency graph towards your targets. I agree with you that Make's pattern matching facilities are ugly and could be improved upon. I tried something of the sort [3] with redo, and it worked well, but felt too alien. From what I've seen at big companies with huge builds, the "dependency graph itself changed, now what do we rebuild" problem is the killer problem. I bet you've seen instances of that at Google. [1] http://cr.yp.to/redo.html http://cr.yp.to/redo.html [2] https://github.com/apenwarr/redo https://github.com/apenwarr/redo [3] https://news.ycombinator.com/item?id=7464047 https://news.ycombinator.com/item?id=7464047
- a8da6b0c91d 13y agoSee also remake. It's a bit simpler, I think. https://github.com/rocky/remake https://github.com/rocky/remake
- Someone 13y ago"Your build system could instead be a Ruby/Python/etc. script" So, to compile a slightly complicated project, I would need Ruby for the main project, Python for a library it builds on, Haskell for another library it builds on, etc? In theory, that's not a new problem, but I think that, by removing the common language that make provides, it will become much more common in practice. At the extreme, it would only be natural, but disastrous, for the guys writing language X to use language X to write their make file. Result: you need to bootstrap language X through a zillion versions to port it to a new system, or mess around trying to build its parts by hand.
- m_mueller 13y agoHonestly I don't get why auto dependencies are even needed. IMO splitting up manually managed dependency rules into multiple, manually ordered makefiles and importing them in a master makefile, should work just fine. It is a lot of work to switch to from a managed environment, but if you start with this from the beginning it's not a big deal adding dependencies whenever they come up. As a developer I like having a nicely formatted dependency documentation, including comments, in one place instead of having to look at what autoconf & co are spitting out.
- strager 13y agoSo every time I add an import or #include, I need to update the dep graph manually? That may work for larger teams, but that would seriously hurt productivity for smaller projects. I think the main issue with approaches like yours is that there's no way to verify correctness, so it's easy to introduce bugs.
- m_mueller 13y ago- small projects should also lead to small dependency graphs, so I don't really get why it's bad there. - I'm just not a big fan of automating the configuration of a build system - you build/put yet another machine around a machine. As long as it works - ok, but as soon as you run into unexpected behavior it will become a huge time sink to debug. When you update the Makefile contents manually you take ownership over everything and you get a deeper understanding on how everything works together, right down to the command parameters sent to compiler and linker. If you don't want to do that I'd suggest switching to something like cmake - if it has to 'magic', at least have the behavior specified in a coherent manner.
- stormbrew 13y agoHow does this lead to having dependency information in one place? It puts it in at least two. The file and the makefile. Not at all sure what autoconf has to do with anything, autodeps are a completely separate problem space.
- 13y ago