3 ms·
The answer to this is "use Shake", which is similar to what you describe. The secrets are that you need a more abstract concept of "targets" or "dependencies" t
by aseipp 6y ago
The answer to this is "use Shake", which is similar to what you describe. The secrets are that you need a more abstract concept of "targets" or "dependencies" that aren't just files — so you can depend on environment variables, file contents, results of dynamic commands, etc — you need to be able to express "dynamic" rules where you can compute new dependencies for a rule "on the fly" while it's executing, and you need elbow grease to write it as an API. Shake also has had a lot of research done about it comparing it to other systems, is used very successfully, and has an array of nice features other systems don't, including .ninja file compatibility and build profile reports: https://shakebuild.com/ https://shakebuild.com/
Shake is written in Haskell as a library (so your build rules have to be written in Haskell too, as programs) but you could either wire up a language via the FFI to make some classes of build rules easily expressible in JavaScript/Lua, or in theory rewrite the core of Shake in another language too. The theory is all there, though.
My personal rule of thumb is: use a "specific" build system if possible, but use Shake the second any build system requires even a tiny bit of complexity (multi language builds, requires parsing or other "advanced" features, etc). As a bonus it's also very fast, within a few % of Ninja when parsing Ninja files, and sometimes faster than that. I was surprised when it came in as 2x as fast as Ninja on a large-ish C codebase at work, due to having picked a different "build schedule" that distributed compilations across cores better.
- amelius 6y ago> My personal rule of thumb is: use a "specific" build system if possible, but use Shake the second any build system requires even a tiny bit of complexity Systems evolve. You don't want to rewrite your build scripts the moment you cross the complexity line. Your mention of Shake is nice though. However, I'm not sure if it could gain widespread adoption, since it's based on Haskell which is not very popular unfortunately (and has a bad name when it comes to learning curve).
- cbsmith 6y agoShake is interesting. I've been using bazel with a degree of success. It seems to incorporate these same concerns and scales out well to large builds. I'm not sure if shake solves enough of a new set of problems to really take hold.
- moonchild 6y agoInteresting. That looks like—mostly—what I was aiming for. (Though I would prefer lisp to haskell for this application, mainly because of metaprogramming.) However, I'm somewhat suspicious of this: > A target is a file we want the build system to produce This is somewhat arbitrary, and shouldn't be mandated by the build system. You say 'the secrets are that you need a more abstract concept of "targets" or "dependencies" that aren't just files', but that doesn't seem to be corroborated? They do show examples of 'phony' targets, but why not simply eliminate the conflation of build targets and files entirely? Phony targets are just a holdover from make. ______________________________________________________ The other reason I would prefer lisp (over haskell) is the lack of 'compile time'. It looks like, with shake, you have to recompile the build file every time you change it? With lisp, loading a file will be pretty much instantaneous.