4 ms·
I wonder if anyone has ever generalized "make" so that it can operate on general dependencies and tactics rather than always on files? The sort of thing I mean
by rwmj 7y ago
I wonder if anyone has ever generalized "make" so that it can operate on general dependencies and tactics rather than always on files? The sort of thing I mean is that you'd be able to write:
url_exists(http://example.com/file): file
rsync file example.com:/html
I've had a few attempts at this (most recently in 2013: https://people.redhat.com/~rjones/goaljobs/ https://people.redhat.com/~rjones/goaljobs/) but never really arrived at anything satisfactory.
- q3k 7y agoThis has also been a holy grail for me. The closest I've come to this are custom starlark rules in Bazel.
- lima 7y agoThis. Bazel is as close as it gets.
- boris 7y agoWe did in build2[1]. Specifically, we have a notion of target type which may not be path or mtime-based. For example, one such target type is alias (something that is achieved in make with PHONY targets). So you could define, say, a url target type. Then you can implement a rule that knows how to perform a certain operation on certain target types (we've also generalized a notion of operation so in build2 test, install, dist are operations, not targets). [1] https://build2.org https://build2.org
- skore 7y agoWell, kind of. I'm using make to build a content graph, part of that is http:// http:// dependencies. The underlying concept is that you can define protocols (https would be one) and then teach my make pre-parser about building them. An example would be: myfile.js: https://code.jquery.com/jquery-3.4.1.js local.js uglifyjs $^ -o $@ The 'https://' https://' is rewritten to map to a local cache directory (so it's little more than a glorified variable, but still more intuitive since protocols are so familiar from our use of the web), so the above code is actually something like myfile.js: cache/https/code.jquery.com/jquery-3.4.1.js local.js uglifyjs $^ -o $@ and that in turn invokes a rule that makes a HEAD request to the URL - if that is outdated against the local copy, that gets renewed. So, in regular make code, something like this: cache/https/%: cache/https/%.head curl -Ls https://$* > $@ cache/https/%.head: cache/https/%.head.force rsync --checksum $< $@ cache/https/%.head.force: FORCE curl -Ls -I https://$* > $@ I could see a way to turn this around, though. The concept of "protocols" is currently only GET in my setup, but I do plan on supporting the idea of POST, eventually. So what you're asking for is that, I guess. I must say, though, that in my now three years of using (or abusing) make, "just make it another file" is surprisingly often the right next step. In the POST situation, it would simply be a local file that has direct or side-effects attached that do what you're expecting it to do.
- lloeki 7y ago> "just make it another file" is surprisingly often the right next step This way it also often turns out that the "fetch"/"cache" part gets naturally separated from the rest, and that's super useful to track down any permanent or transitory issue.
- skore 7y agoYes! It's basically a built-in stack trace! Permanently written straight to your disk! It does get unwieldy after a certain size, but that's what I'm building tooling for.
- rwmj 7y agoPerhaps the part I'm missing is how you are rewriting the https:// https:// URLs to the paths. Do you preprocess the Makefile?
- skore 7y agoYeah, I'm using a preprocessor to generate the makefiles from what is basically blueprints.
- nerdponx 7y agoThere are literally hundreds of build tools and workflow engines that replace or extend some or all of Make. Many of them are domain-specific (especially to bioinformatics or machine learning), many of them are not.
- rwmj 7y agoAny links to any of these tools?