8 ms·

I truly believe any build system that uses a general-purpose language by default is too powerful. It lets people do silly stuff too easily. Build systems (for p

by xxpor 4y ago

I truly believe any build system that uses a general-purpose language by default is too powerful. It lets people do silly stuff too easily. Build systems (for projects with a lot of different contributors) should be easy to understand, with few, if any, project specific concepts to learn. There can always be an escape hatch to python (see GN, for example), but 99% of the code should just be boring lists of files to build.

bogwog 4y ago
I agree, that’s also pretty much why Starlark exists. However, there are many cases where you do need complex build logic. Personally, I always go for declarative CMake first, then waf as soon as I find my CMakeLists looking like something other than just a list of files. I’ve considered before creating a simple declarative language to build simple projects like that with waf, but I don’t like the idea of maintaining my own DSL for such little benefit, when CMake works just fine, and everyone knows how to use it. I feel like I’d end up with my own little TempleOS if I decided to go down that rabbit hole.
pjmlp 4y ago
They are the bane of any DevOps/Build Engineer when trying to fix build issues.
DrBazza 4y ago
The problem with build systems are its users. For exactly the reason you say. For a man with a hammer every problem is a nail. Developers don’t think of build systems in the right way. If you’re doing something complex in your build it should surely be a build task in its own right.
baby 4y ago
I think I would agree as well. So I’m not sure how that makes me feel about nix.
- xxpor 4y ago
  Nix is different because no one's smart enough to figure how how to do silly things ;)
- pxc 4y ago
  Nix is Turing complete, but it's not a general purpose language. It is designed as a DSL for building software, and I think it's pretty nice for that. The Nickel rationale doc has some thoughts on why this might be the right call: https://github.com/tweag/nickel/blob/master/RATIONALE.md#turing-completeness https://github.com/tweag/nickel/blob/master/RATIONALE.md#tur... From my (limited) experience with another deliberately limited configuration DSL (CUE), I think more power in such DSLs will pan out better in the long run. Of course, it's not all one or the other: a powerful build DSL can still enforce useful discipline, and a Turing-complete language can still be thoughtfully designed around a special purpose. I think Nix demonstrates both pretty well, actually.
- tadfisher 4y ago
  Nix forces you to serialize every build step (what it calls a "derivation"), and moreover it isolates the build environment to only include things built with Nix or verified by hash. So while there is a lot of power, the only thing you can do with that power is produce derivations which themselves actually run the build. Contrast this with Gradle, which is currently digging itself out of a hole by forcing authors to declare all inputs and outputs of their tasks so it can serialize them, but you can literally do anything Java can throughout the entire process. This is the kind of Herculean task which is neatly sidestepped by tightly controlling the DSL environment (inputs/outputs) as does Nix.
sangnoir 4y ago
You cannot magick away complexity. Large systems (think thousands of teams with hundreds of commits per minute) require a way to express complexity. When all is said and done, you'll have a turing-complete build system anyway - so why not go with something readable
- xxpor 4y ago
  I seriously doubt there's a single repo on the planet that averages hundreds of commits per minute. That's completely unmanageable for any number of reasons.
  - chucknthem 4y ago
    It wouldn't surprise me at all if some large repos at Google or Facebook now get to that many, it's easy to do once you have robots committing code (usually configuration changes).
  - sangnoir 4y ago
    I didn't mean on average, but the build tool has to handle the worst case and I probably am understating the worst case. I'd bet there are a more than a few repos that do get (at least) hundreds of commits as a highwater mark. My guess is lots of engineers + mono-repo + looming code-freeze deadline can do that like clockwork. Edit: Robots too as sibling pointed out. A single human action may result in dozens of bot-generated commits
    - xxpor 4y ago
      IMO there's almost never a good reason to have automated commits in repos outside of two cases: 1) Automated refactoring 2) Automated merges when CI passes Configs that can be generated should just be generated by the build. But that's a different topic
      - rrdharan 4y ago
        There are at least two other hugely important use cases you missed: - automatic security / vendoring updates (e.g. https://github.com/renovatebot/renovate https://github.com/renovatebot/renovate) - automated cross-repo syncs, e.g. Google has processes and tools that bidirectionally sync pieces of Google3 with GitHub repos
bluGill 4y ago
I'd like to agree, but every significant project has something wierd that their build system doesn't have built in yet. So you need some way to extend it. Useful build systems end up supporting lots of hacks ' That said, the more your build systems makes easy without having to write code the better.
- joshuamorton 4y ago
  While this is true, this isn't a problem for your buck/bazel/pants-likes. Between genrules and custom rules you can do this with all the power you (usually) need.
deleted 4y ago
[deleted]

by xxpor 4y ago

bogwog 4y ago

I agree, that’s also pretty much why Starlark exists. However, there are many cases where you do need complex build logic. Personally, I always go for declarative CMake first, then waf as soon as I find my CMakeLists looking like something other than just a list of files. I’ve considered before creating a simple declarative language to build simple projects like that with waf, but I don’t like the idea of maintaining my own DSL for such little benefit, when CMake works just fine, and everyone knows how to use it. I feel like I’d end up with my own little TempleOS if I decided to go down that rabbit hole.

pjmlp 4y ago

They are the bane of any DevOps/Build Engineer when trying to fix build issues.

DrBazza 4y ago

The problem with build systems are its users. For exactly the reason you say. For a man with a hammer every problem is a nail. Developers don’t think of build systems in the right way. If you’re doing something complex in your build it should surely be a build task in its own right.

baby 4y ago

I think I would agree as well. So I’m not sure how that makes me feel about nix.

xxpor 4y ago
Nix is different because no one's smart enough to figure how how to do silly things ;)
pxc 4y ago
Nix is Turing complete, but it's not a general purpose language. It is designed as a DSL for building software, and I think it's pretty nice for that. The Nickel rationale doc has some thoughts on why this might be the right call: https://github.com/tweag/nickel/blob/master/RATIONALE.md#turing-completeness https://github.com/tweag/nickel/blob/master/RATIONALE.md#tur... From my (limited) experience with another deliberately limited configuration DSL (CUE), I think more power in such DSLs will pan out better in the long run. Of course, it's not all one or the other: a powerful build DSL can still enforce useful discipline, and a Turing-complete language can still be thoughtfully designed around a special purpose. I think Nix demonstrates both pretty well, actually.
tadfisher 4y ago
Nix forces you to serialize every build step (what it calls a "derivation"), and moreover it isolates the build environment to only include things built with Nix or verified by hash. So while there is a lot of power, the only thing you can do with that power is produce derivations which themselves actually run the build. Contrast this with Gradle, which is currently digging itself out of a hole by forcing authors to declare all inputs and outputs of their tasks so it can serialize them, but you can literally do anything Java can throughout the entire process. This is the kind of Herculean task which is neatly sidestepped by tightly controlling the DSL environment (inputs/outputs) as does Nix.

sangnoir 4y ago

You cannot magick away complexity. Large systems (think thousands of teams with hundreds of commits per minute) require a way to express complexity. When all is said and done, you'll have a turing-complete build system anyway - so why not go with something readable

xxpor 4y ago
I seriously doubt there's a single repo on the planet that averages hundreds of commits per minute. That's completely unmanageable for any number of reasons.
- chucknthem 4y ago
  It wouldn't surprise me at all if some large repos at Google or Facebook now get to that many, it's easy to do once you have robots committing code (usually configuration changes).
- sangnoir 4y ago
  I didn't mean on average, but the build tool has to handle the worst case and I probably am understating the worst case. I'd bet there are a more than a few repos that do get (at least) hundreds of commits as a highwater mark. My guess is lots of engineers + mono-repo + looming code-freeze deadline can do that like clockwork. Edit: Robots too as sibling pointed out. A single human action may result in dozens of bot-generated commits
  - xxpor 4y ago
    IMO there's almost never a good reason to have automated commits in repos outside of two cases: 1) Automated refactoring 2) Automated merges when CI passes Configs that can be generated should just be generated by the build. But that's a different topic
    - rrdharan 4y ago
      There are at least two other hugely important use cases you missed: - automatic security / vendoring updates (e.g. https://github.com/renovatebot/renovate https://github.com/renovatebot/renovate) - automated cross-repo syncs, e.g. Google has processes and tools that bidirectionally sync pieces of Google3 with GitHub repos

bluGill 4y ago

I'd like to agree, but every significant project has something wierd that their build system doesn't have built in yet. So you need some way to extend it. Useful build systems end up supporting lots of hacks ' That said, the more your build systems makes easy without having to write code the better.

joshuamorton 4y ago
While this is true, this isn't a problem for your buck/bazel/pants-likes. Between genrules and custom rules you can do this with all the power you (usually) need.

deleted 4y ago

[deleted]