3 ms·

So he re-built Gradle, or Buck, or Bazel, or any number of roughly similar it-runs-a-dag build tools? Not sure I buy that his problem was real. Gradle's the onl

So he re-built Gradle, or Buck, or Bazel, or any number of roughly similar it-runs-a-dag build tools? Not sure I buy that his problem was real. Gradle's the only one I'm meaningfully familiar with and making custom commands over a dag is a breeze.

jstimpfle 9y ago
The "run the DAG" part is very small and simple (as you can verify). I don't think there is a good argument against "rebuilding" that. As I explained my motivation was primarily keeping my build description in my own datastructures (see the text database). IMO that's a very worthwhile goal. I then went on to see what it takes to NIH everything, and it turns out that detecting what files need to be rebuilt is hairy, especially considering transitive C includes (which requires a dependency cache). But there is no one solution to these problems, and I do think there is some value in controlling that logic. And if you get the concepts right, it can be done. I've divided the code into loosely coupled building blocks. That hopefully makes it easier to understand. It should also be possible to recombine these blocks to make different build systems.
- kimikimi 9y ago
  FWIW there's a scope threshold that many projects don't always have to cross that justifies using a "real" build tool - and it has to do with those "hairy" parts of file detection and automatically finding dependencies in the shared environment. If the project allows it, though, your dependencies can be shrunk down until there's no issue. This is far more the case with target languages that already understand modules (e.g. Rust or Python versus C or JS), or "one big framework" projects where you are relying on linking in the one framework and having it do all the heavy lifting on the build side of things. Actually customizing a build substantially beyond that, tends to be the case more often when you have something like a compiler within your own pipeline, like automatically transforming and clipping source art assets into optimized, UI-ready form. With those systems there is a lot to be said for a simple script that you can debug.
- sushisource 9y ago
  I suspect you could (continuing the gradle example) write gradle plugins that would accomplish what you want to do as far as using your own data structures goes. Seems to me though the majority of what's in that textual database is standard C compile/link stuff that'd be handled fine by https://guides.gradle.org/building-c-executables/ https://guides.gradle.org/building-c-executables/
  - jstimpfle 9y ago
    Yep - currently only my data structure generator is in the build. But I have plenty more to add. I have a pretty poor asset pipeline that I currently run manually. This stuff is very costly, so you want to have good control and manual intervention as far as possible. I also want to add more packaging workflows for a basic "component entity architecture" which means collecting information from various modules and reorganizing them into a more global database. These things are tricky and I'm pretty positive I simply want to do them myself.
crdoconnor 9y ago
To me, the fact that people seem to keep doing this suggests that these tools aren't all that great.

jstimpfle 9y ago

The "run the DAG" part is very small and simple (as you can verify). I don't think there is a good argument against "rebuilding" that. As I explained my motivation was primarily keeping my build description in my own datastructures (see the text database). IMO that's a very worthwhile goal. I then went on to see what it takes to NIH everything, and it turns out that detecting what files need to be rebuilt is hairy, especially considering transitive C includes (which requires a dependency cache). But there is no one solution to these problems, and I do think there is some value in controlling that logic. And if you get the concepts right, it can be done. I've divided the code into loosely coupled building blocks. That hopefully makes it easier to understand. It should also be possible to recombine these blocks to make different build systems.

kimikimi 9y ago
FWIW there's a scope threshold that many projects don't always have to cross that justifies using a "real" build tool - and it has to do with those "hairy" parts of file detection and automatically finding dependencies in the shared environment. If the project allows it, though, your dependencies can be shrunk down until there's no issue. This is far more the case with target languages that already understand modules (e.g. Rust or Python versus C or JS), or "one big framework" projects where you are relying on linking in the one framework and having it do all the heavy lifting on the build side of things. Actually customizing a build substantially beyond that, tends to be the case more often when you have something like a compiler within your own pipeline, like automatically transforming and clipping source art assets into optimized, UI-ready form. With those systems there is a lot to be said for a simple script that you can debug.
sushisource 9y ago
I suspect you could (continuing the gradle example) write gradle plugins that would accomplish what you want to do as far as using your own data structures goes. Seems to me though the majority of what's in that textual database is standard C compile/link stuff that'd be handled fine by https://guides.gradle.org/building-c-executables/ https://guides.gradle.org/building-c-executables/
- jstimpfle 9y ago
  Yep - currently only my data structure generator is in the build. But I have plenty more to add. I have a pretty poor asset pipeline that I currently run manually. This stuff is very costly, so you want to have good control and manual intervention as far as possible. I also want to add more packaging workflows for a basic "component entity architecture" which means collecting information from various modules and reorganizing them into a more global database. These things are tricky and I'm pretty positive I simply want to do them myself.

crdoconnor 9y ago

To me, the fact that people seem to keep doing this suggests that these tools aren't all that great.