3 ms·

> What you really want is a way for scripts to describe their dependencies, and then the runner figures out what order to run them in, and cache scripts that do

> What you really want is a way for scripts to describe their dependencies, and then the runner figures out what order to run them in, and cache scripts that don't need to be run because their inputs didn't change. DAG + content-addressing, final binary being the target and everything resolved from there. We could have some beautiful build system that just works and is fast, but seems it never magically appears by itself although it seems so elegant. Guess Nix/NixOS is the closest we've gotten so far, works well enough, missing concurrency and parallelism though.

dpe82 9mo ago
Google's build system Bazel is what you describe.
- mattnewton 9mo ago
  I should really give it another look, I usually ended up not adopting it for projects because of the boilerplate and high setup overheard, but those are both things that ai agents can usually be trusted with. Maybe the calculus has changed.
  - dpe82 9mo ago
    Yeah Bazel is a PITA to do yourself. I haven't tried but I'm confident Claude et al. would handle it just fine. By similar logic I recently adopted NixOS for a lot of my servers; Claude is perfectly happy to slog through the pain of Nix configs for me.
    - hxtk 9mo ago
      I’m not a fan of generative AI for the use case because it’s rote enough to do deterministically, but deterministic code generation is getting better and better. Gazelle, the BUILD file generator for Go, now supports plugins and several other languages have Gazelle plugins. I’ve used AI to generate BUILD file generators before, though. I had good luck getting it to write a script that would analyze a Java project with circular dependencies and aggregate the cycle participants into a single target.
  - hxtk 9mo ago
    It’s gotten easier of late because Bazel modules are nice and Gazelle has started support plugins so it can do build file generation for other languages. I don’t like generative AI for rote tasks like this, but I’ve had good luck using generative AI to write deterministic code generators that I can commit to a project and reuse.
spankalee 9mo ago
Wireit does both DAG and content-addressing. It figerprints the inputs and outputs of dependencies. And you run scripts externally with plain `npm run` commands. It's really beautiful.
sestep 9mo ago
Could you clarify what you mean about Nix missing concurrency and parallelism? I often run builds using nix-output-monitor and it definitely looks like things are running in parallel, although I could be mistaken.
- embedding-shape 9mo ago
  I meant as part of the language itself, not just the runtime or for specific parts. Say I'm processing 100 JSON files, it'd be great if I could fire that off wrapped in 'parallel' or whatever, similar to Clojure and others I guess.

dpe82 9mo ago

Google's build system Bazel is what you describe.

mattnewton 9mo ago
I should really give it another look, I usually ended up not adopting it for projects because of the boilerplate and high setup overheard, but those are both things that ai agents can usually be trusted with. Maybe the calculus has changed.
- dpe82 9mo ago
  Yeah Bazel is a PITA to do yourself. I haven't tried but I'm confident Claude et al. would handle it just fine. By similar logic I recently adopted NixOS for a lot of my servers; Claude is perfectly happy to slog through the pain of Nix configs for me.
  - hxtk 9mo ago
    I’m not a fan of generative AI for the use case because it’s rote enough to do deterministically, but deterministic code generation is getting better and better. Gazelle, the BUILD file generator for Go, now supports plugins and several other languages have Gazelle plugins. I’ve used AI to generate BUILD file generators before, though. I had good luck getting it to write a script that would analyze a Java project with circular dependencies and aggregate the cycle participants into a single target.
- hxtk 9mo ago
  It’s gotten easier of late because Bazel modules are nice and Gazelle has started support plugins so it can do build file generation for other languages. I don’t like generative AI for rote tasks like this, but I’ve had good luck using generative AI to write deterministic code generators that I can commit to a project and reuse.

spankalee 9mo ago

Wireit does both DAG and content-addressing. It figerprints the inputs and outputs of dependencies. And you run scripts externally with plain `npm run` commands. It's really beautiful.

sestep 9mo ago

Could you clarify what you mean about Nix missing concurrency and parallelism? I often run builds using nix-output-monitor and it definitely looks like things are running in parallel, although I could be mistaken.

embedding-shape 9mo ago
I meant as part of the language itself, not just the runtime or for specific parts. Say I'm processing 100 JSON files, it'd be great if I could fire that off wrapped in 'parallel' or whatever, similar to Clojure and others I guess.