5 ms·
The auto-dependency problem is a perfect example of how there is room for improvement in make-land. When something that basically everyone wants to do takes a l
by haberman 13y ago
The auto-dependency problem is a perfect example of how there is room for improvement in make-land. When something that basically everyone wants to do takes a long web page to describe a complicated method that sort of works, that is a compelling indicator that there must be a better way.
Incidentally, I am working on a make-replacement that is intended to address this and similar needs. It's still in very early stages, but I think I'm onto something.
The idea is to write a very low-level tool that leaves out most of Make's higher-level abstractions. In my tool there are no recipes, no implicit rules, no variables or variable substitutions, no conditionals, etc. The input to my tool is just the precise specifications of the commands you need to run, their inputs and outputs (so a precise dependency graph can be calculated), and with everything fully-expanded already.
The idea, then, is that whatever higher-level abstractions you want (if any) you build into a higher-level tool. The higher-level tool just spits out a file describing the list of tasks. Then the higher-level tool can worry about the higher-level structure, policy, configuration, etc. of your project. So instead of writing things like implicit rules in Makefiles, you just write some code that explicitly generates tasks.
For example, with Make, you might write an implicit rule like this:
%.o : %.c
$(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@
Then Make magically decides which output files match this implicit rule. My idea is that, instead of this, explicitly apply your rules to your inputs. Your build system could instead be a Ruby/Python/etc. script that looks something like this:
for c_file, o_file in files:
tasks.append(Task(
target=o_file,
source=c_file,
command="gcc -c %s -o %s", c_file, o_file
))
print(tasks)
I think this is much more convenient than having to program in Make and learn its quirky abstractions.
I have an elegant solution for the auto-dependency problem (unproven, but I think it's promising). The idea is that your dependency-calculating tasks are still just tasks, but the tool knows how to integrate the calculated dependencies back into the overall dependency graph. These dependency-generating tasks depend on the files they are generating dependencies for, so it is all part of the unified dependency graph.
If you're interested, star my project and follow my blog (where I will make any announcements about it):
https://github.com/haberman/taskforce https://github.com/haberman/taskforce
http://blog.reverberate.org/ http://blog.reverberate.org/
- llimllib 13y agoI did something similar with my python make-alike, pub[1], where you can have something like: def outname(f): return f[:-1] + "o" @file_rule("*.c", outname) def compile(f): #do the compiling here Here's an example of file_rule in use: https://github.com/llimllib/Newsite/blob/master/pub.py#L27 https://github.com/llimllib/Newsite/blob/master/pub.py#L27 [1]: https://github.com/llimllib/pub/ https://github.com/llimllib/pub/
- sigil 13y agoYou should check out djb's redo [1], specifically the implementation by apenwarr [2]. It's close to the separation of concerns you're looking for: programs written in your language of choice enumerate the dependency graph, an external tool collects them and executes its way through the dependency graph towards your targets. I agree with you that Make's pattern matching facilities are ugly and could be improved upon. I tried something of the sort [3] with redo, and it worked well, but felt too alien. From what I've seen at big companies with huge builds, the "dependency graph itself changed, now what do we rebuild" problem is the killer problem. I bet you've seen instances of that at Google. [1] http://cr.yp.to/redo.html http://cr.yp.to/redo.html [2] https://github.com/apenwarr/redo https://github.com/apenwarr/redo [3] https://news.ycombinator.com/item?id=7464047 https://news.ycombinator.com/item?id=7464047
- a8da6b0c91d 13y agoSee also remake. It's a bit simpler, I think. https://github.com/rocky/remake https://github.com/rocky/remake
- Someone 13y ago"Your build system could instead be a Ruby/Python/etc. script" So, to compile a slightly complicated project, I would need Ruby for the main project, Python for a library it builds on, Haskell for another library it builds on, etc? In theory, that's not a new problem, but I think that, by removing the common language that make provides, it will become much more common in practice. At the extreme, it would only be natural, but disastrous, for the guys writing language X to use language X to write their make file. Result: you need to bootstrap language X through a zillion versions to port it to a new system, or mess around trying to build its parts by hand.