3 ms·

>There are three implementations (Go, Rust, Java) There are so many people at Google just goofing around, lol. People just doing hobby stuff for $350k/year. T

>There are three implementations (Go, Rust, Java) There are so many people at Google just goofing around, lol. People just doing hobby stuff for $350k/year. There is no justification to implement this custom dialect of Python for a build system that drives everyone crazy 3 times. Reminds me of when it was revealed that 400 people were working on Fuschia — a hobby OS that only shipped on a single smart home device.

deleted 2y ago
[deleted]
ithkuil 2y ago
IIRC initially blaze was literally using python as a config layer. That made it too easy to write build scripts that were too slow and too hard to optimize and that negatively affected the overall build experience. An increase in edit/build/run cycle efficiency of thousands of employees justifies an investment of a few engineers. They could have invented a new bespoke DSL. Instead they choose to stick to a well known and familiar language and just limit its expressiveness to a subset that would be easier to optimize. I think that's quite reasonable
- kevindamm 2y ago
  I also remember blaze directly eval'ing Python pre-Skylark, but I think the bigger problem was build hermeticity. This was compounded by env-related issues making it hard to build Python hermetically in the first place, and the importance of cached object files and build outputs in managing the incredibly large monorepo that is Google's source tree. And that's all before considering the Py2->Py3 migration (which, fortunately, was tackled afterwards). I think a new bespoke DSL would have been a non-starter since so many build scripts had already been written by the time Skylark was being conceived.
zellyn 2y ago
One of my hobbies while at Google (2010-2015) was to watch the multiple failed attempts to get rid of Borg Config Language and actual Python in Blaze. It took a lot of work until they eventually succeeded. You're probably underestimating the rigor, the pain, and also the value involved in cleaning those things up: being able to cleanly operate programmatically at scale in the Google monorepo is extremely necessary. (This is also why I don't trust configuration languages built by people who _didn't_ observe the years of pain. Cue and jsonnet are notable projects that were able to incorporate a lot of lessons.)
- zellyn 2y ago
  Also, I think the Rust version was built as a side-project, then handed off to Facebook. With regards to Fuchsia… I used to think building a new OS from the ground up was madness. Now I think _not_ doing that eventually is madness. I'd be a lot happier if Zircon and Fuchsia were done in Rust though…
  - surajrmal 2y ago
    A good deal of fuchsia is done in rust (roughly 50% and increasing over time). People over emphasize the importance of zircon needing to be in rust. It's more important for new code to be rust than for existing code to be rewritten. Zircon isn't a very fast growing part of the entire OS.
    - zellyn 2y ago
      Fair. That study on writing only new things in Rust was surprising! https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html https://security.googleblog.com/2024/09/eliminating-memory-s... I do think for an OS/Kernel, it's worth having everything in a memory-safe language, and possibly worth formal verification too, if the very core of it is small enough…
- foooorsyth 2y ago
  I can appreciate the pain of actually getting things through in a large (100k+ people) organization. What do you consider the be the justification for three separate implementations of the same build config language? Genuine question. I am not doubting the need for the DSL itself.
  - jsnell 2y ago
    (Note that the Rust implementation is by Facebook, not by Google.) If you're looking to embed a scripting language in a Go program, having a embedded language implementation written in Java isn't very useful. And vice versa.
surajrmal 2y ago
I'm not sure. I understand what you are trying to say - do you think we should give up on OS diversity? Are you making fun of a company for actually investing in something ambitious which can benefit many if successful? While I'm not sure you are in the same crowd, I always think it's interesting that the HN hivemind tends to be upset with the browser monoculture, but doesn't bat an eye at OS monoculture. It feels like you're really just channeling feelings about the company rather than the projects.
surajrmal 2y ago
Of the three, only two are Google produced. The rust implementation is written by Facebook for use in their build system. The java implementation is the original but is pretty tied to the the bazel build system and not really very suitable to other uses. The go implementation is meant to be embeddable into a varying number of applications. An example application is https://github.com/shac-project/shac https://github.com/shac-project/shac Within Google, there a large number of similar tools which are written in go and harness the starlark language. While there are plenty of other options, I will say I think starlark is often a great choice.
deleted 2y ago
[deleted]

deleted 2y ago

[deleted]

ithkuil 2y ago

IIRC initially blaze was literally using python as a config layer. That made it too easy to write build scripts that were too slow and too hard to optimize and that negatively affected the overall build experience. An increase in edit/build/run cycle efficiency of thousands of employees justifies an investment of a few engineers. They could have invented a new bespoke DSL. Instead they choose to stick to a well known and familiar language and just limit its expressiveness to a subset that would be easier to optimize. I think that's quite reasonable

kevindamm 2y ago
I also remember blaze directly eval'ing Python pre-Skylark, but I think the bigger problem was build hermeticity. This was compounded by env-related issues making it hard to build Python hermetically in the first place, and the importance of cached object files and build outputs in managing the incredibly large monorepo that is Google's source tree. And that's all before considering the Py2->Py3 migration (which, fortunately, was tackled afterwards). I think a new bespoke DSL would have been a non-starter since so many build scripts had already been written by the time Skylark was being conceived.

zellyn 2y ago

One of my hobbies while at Google (2010-2015) was to watch the multiple failed attempts to get rid of Borg Config Language and actual Python in Blaze. It took a lot of work until they eventually succeeded. You're probably underestimating the rigor, the pain, and also the value involved in cleaning those things up: being able to cleanly operate programmatically at scale in the Google monorepo is extremely necessary. (This is also why I don't trust configuration languages built by people who _didn't_ observe the years of pain. Cue and jsonnet are notable projects that were able to incorporate a lot of lessons.)

zellyn 2y ago
Also, I think the Rust version was built as a side-project, then handed off to Facebook. With regards to Fuchsia… I used to think building a new OS from the ground up was madness. Now I think _not_ doing that eventually is madness. I'd be a lot happier if Zircon and Fuchsia were done in Rust though…
- surajrmal 2y ago
  A good deal of fuchsia is done in rust (roughly 50% and increasing over time). People over emphasize the importance of zircon needing to be in rust. It's more important for new code to be rust than for existing code to be rewritten. Zircon isn't a very fast growing part of the entire OS.
  - zellyn 2y ago
    Fair. That study on writing only new things in Rust was surprising! https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html https://security.googleblog.com/2024/09/eliminating-memory-s... I do think for an OS/Kernel, it's worth having everything in a memory-safe language, and possibly worth formal verification too, if the very core of it is small enough…
foooorsyth 2y ago
I can appreciate the pain of actually getting things through in a large (100k+ people) organization. What do you consider the be the justification for three separate implementations of the same build config language? Genuine question. I am not doubting the need for the DSL itself.
- jsnell 2y ago
  (Note that the Rust implementation is by Facebook, not by Google.) If you're looking to embed a scripting language in a Go program, having a embedded language implementation written in Java isn't very useful. And vice versa.

surajrmal 2y ago

I'm not sure. I understand what you are trying to say - do you think we should give up on OS diversity? Are you making fun of a company for actually investing in something ambitious which can benefit many if successful? While I'm not sure you are in the same crowd, I always think it's interesting that the HN hivemind tends to be upset with the browser monoculture, but doesn't bat an eye at OS monoculture. It feels like you're really just channeling feelings about the company rather than the projects.

surajrmal 2y ago

Of the three, only two are Google produced. The rust implementation is written by Facebook for use in their build system. The java implementation is the original but is pretty tied to the the bazel build system and not really very suitable to other uses. The go implementation is meant to be embeddable into a varying number of applications. An example application is https://github.com/shac-project/shac https://github.com/shac-project/shac Within Google, there a large number of similar tools which are written in go and harness the starlark language. While there are plenty of other options, I will say I think starlark is often a great choice.

deleted 2y ago

[deleted]