5 ms·

What's the motivation for using YAML instead of Starlark (Bazel, Buck) or something closer to Python (Pants, please.build)? Seems as though much of the other mo

What's the motivation for using YAML instead of Starlark (Bazel, Buck) or something closer to Python (Pants, please.build)? Seems as though much of the other monorepo tools have (kinda) standardized on this. As I understand it, the primary reason these build systems leverage these Python-variants is so that the build rules, toolchains, constraints, and build definitions can all be written in the same language (since build rules often require some programmatic behavior). Perhaps with a future vision of them being totally interoperable across build systems.

Denzel 4y ago
Not to mention using an actual language aids readability, extensibility, and static analysis, unlike a data exchange format. Starlark is a benefit, not a con, so YAML feels like a major step backwards. I'm generally happy with Blaze/Bazel, so I'm not necessarily in the target market for Moonrepo, I guess. EDIT: This isn't really competing with Blaze/Bazel either when I look at the execution model. It goes back to imperatively defined tasks instead of declaratively defined dependencies, which feels more spirtually aligned with Make than Blaze/Bazel.
- mileswjohnson 4y ago
  Yeah that's fair feedback. At this time, we consider ourselves more of a Bazel-lite than an actual Bazel replacement. Once we support more languages and features, this may change in the future.
  - Denzel 4y ago
    I'm not trying to be overly critical here, in fact, I want to share the type of empathetic feedback I'd hope to receive as a founder. Have you actually talked to Blaze/Bazel users to understand what frustrations (if any) they have with their current build system? Have these users asked for a Bazel-lite? If so, and you still want to position yourself as Bazel-lite, then you should include some of their direct feedback and write your messaging accordingly. As a daily Blaze/Bazel user, I don't have a desire for a Bazel-lite. I've worked at a midsize company that used Bazel, and I'm working at the company that created Blaze/Bazel. Disclaimer: Opinions expressed are my own; not representative of my employer.
    - ctvo 4y ago
      Let me give them feedback in the vein of what you're asking for. I want the following as a user: - Distributed cache of built artifacts. Work with my company's network topology and security requirements. Make this easy to setup, operate, and seamless for developers to opt into - Seamless monorepo support (multiple languages) That could be considered Bazel-lite. And yes, I'd take it over Bazel currently. If you don't work at Google, convincing your engineering organization into learning Bazel is almost always a non-starter. Who uses Bazel in the wild? Xooglers primarily. Their value proposition is sound.
      - mileswjohnson 4y ago
        Thank you. This is how we feel as well.
      - Denzel 4y ago
        Off the top of my head, some companies that successfully use Bazel: SpaceX [1], Datadog [2], Databricks [3], Stripe [4], Uber [5], etc. That's a good 10k+ engineers using Bazel successfully in the wild. There's more, of course. [1]: https://www.youtube.com/watch?v=t_3bckhV_YI https://www.youtube.com/watch?v=t_3bckhV_YI [2]: https://www.youtube.com/watch?v=H67uuwVO1tc https://www.youtube.com/watch?v=H67uuwVO1tc [3]: https://www.databricks.com/blog/2019/02/27/speedy-scala-builds-with-bazel-at-databricks.html https://www.databricks.com/blog/2019/02/27/speedy-scala-buil... [4]: https://stripe.com/blog/fast-secure-builds-choose-two https://stripe.com/blog/fast-secure-builds-choose-two [5]: https://www.uber.com/en-SE/blog/go-monorepo-bazel/ https://www.uber.com/en-SE/blog/go-monorepo-bazel/
        ctvo 4y ago
        No one has, to my knowledge, argued that Bazel isn't useful or worthwhile. My point was adopting Bazel is costly for an engineering organization, and you may not need all of its features. Bazel-lite may be enough. Large companies, with established platform / ops teams, who can support Bazel and push for its adoption within an engineering organization definitely exist. There's probably a few Xooglers working there who'd like to see it happen too.
- rcme 4y ago
  > Not to mention using an actual language aids readability, extensibility, and static analysis, unlike a data exchange format. Clearly, you've never used Gradle.
mileswjohnson 4y ago
We chose YAML for a few reasons. The first being that we wanted a format that is basically universally supported everywhere. This filtered down to JSON, TOML, and YAML. JSON is an awful configuration format, so that was a no go. TOML is pretty great, but is also not very ubiquitous. That left us with YAML. The second reason is we wanted something language agnostic and not proprietary. It also helps that many other tools, like GitHub actions, are written in YAML. And lastly, we wanted a format that developers are familiar with, and won't need to spend time learning. Requiring developers to understand a special syntax simply to define a task to run is something we want to avoid.
- randall 4y ago
  IDK. I've been using buck for a bit, and having python within grasp is pretty useful. I don't like yaml for a variety of reasons, but the biggest of which is I like my config being able to be generated... maybe that's an anti pattern? IDK. Would love to know more about your take on yaml vs anything more pythoney.
  - computershit 4y ago
    What makes yaml not generatable?
    - randall 4y ago
      Sorry, I meant more dynamic than generatable... like doing stuff in the build file to build something complex in a more simple way. It's really 'do you make config an artifact, or do you make it code' sort of argument.
  - mileswjohnson 4y ago
    I'm not very familiar with Buck. But I'm assuming it works the same way as Bazel's Starlark? I can see the benefits of "generating code" within the BUILD file, but honestly, we haven't required it yet, and have been able to do most things with explicit configuration. Our token syntax helps with some dynamic aspects of this. Maybe in the future we'll revisit this.
andrew_ 4y ago
I much prefer YAML rather than having to write more code for a build system.

Denzel 4y ago

Not to mention using an actual language aids readability, extensibility, and static analysis, unlike a data exchange format. Starlark is a benefit, not a con, so YAML feels like a major step backwards. I'm generally happy with Blaze/Bazel, so I'm not necessarily in the target market for Moonrepo, I guess. EDIT: This isn't really competing with Blaze/Bazel either when I look at the execution model. It goes back to imperatively defined tasks instead of declaratively defined dependencies, which feels more spirtually aligned with Make than Blaze/Bazel.

mileswjohnson 4y ago
Yeah that's fair feedback. At this time, we consider ourselves more of a Bazel-lite than an actual Bazel replacement. Once we support more languages and features, this may change in the future.
- Denzel 4y ago
  I'm not trying to be overly critical here, in fact, I want to share the type of empathetic feedback I'd hope to receive as a founder. Have you actually talked to Blaze/Bazel users to understand what frustrations (if any) they have with their current build system? Have these users asked for a Bazel-lite? If so, and you still want to position yourself as Bazel-lite, then you should include some of their direct feedback and write your messaging accordingly. As a daily Blaze/Bazel user, I don't have a desire for a Bazel-lite. I've worked at a midsize company that used Bazel, and I'm working at the company that created Blaze/Bazel. Disclaimer: Opinions expressed are my own; not representative of my employer.
  - ctvo 4y ago
    Let me give them feedback in the vein of what you're asking for. I want the following as a user: - Distributed cache of built artifacts. Work with my company's network topology and security requirements. Make this easy to setup, operate, and seamless for developers to opt into - Seamless monorepo support (multiple languages) That could be considered Bazel-lite. And yes, I'd take it over Bazel currently. If you don't work at Google, convincing your engineering organization into learning Bazel is almost always a non-starter. Who uses Bazel in the wild? Xooglers primarily. Their value proposition is sound.
    - mileswjohnson 4y ago
      Thank you. This is how we feel as well.
    - Denzel 4y ago
      Off the top of my head, some companies that successfully use Bazel: SpaceX [1], Datadog [2], Databricks [3], Stripe [4], Uber [5], etc. That's a good 10k+ engineers using Bazel successfully in the wild. There's more, of course. [1]: https://www.youtube.com/watch?v=t_3bckhV_YI https://www.youtube.com/watch?v=t_3bckhV_YI [2]: https://www.youtube.com/watch?v=H67uuwVO1tc https://www.youtube.com/watch?v=H67uuwVO1tc [3]: https://www.databricks.com/blog/2019/02/27/speedy-scala-builds-with-bazel-at-databricks.html https://www.databricks.com/blog/2019/02/27/speedy-scala-buil... [4]: https://stripe.com/blog/fast-secure-builds-choose-two https://stripe.com/blog/fast-secure-builds-choose-two [5]: https://www.uber.com/en-SE/blog/go-monorepo-bazel/ https://www.uber.com/en-SE/blog/go-monorepo-bazel/
      - ctvo 4y ago
        No one has, to my knowledge, argued that Bazel isn't useful or worthwhile. My point was adopting Bazel is costly for an engineering organization, and you may not need all of its features. Bazel-lite may be enough. Large companies, with established platform / ops teams, who can support Bazel and push for its adoption within an engineering organization definitely exist. There's probably a few Xooglers working there who'd like to see it happen too.
rcme 4y ago
> Not to mention using an actual language aids readability, extensibility, and static analysis, unlike a data exchange format. Clearly, you've never used Gradle.

mileswjohnson 4y ago

We chose YAML for a few reasons. The first being that we wanted a format that is basically universally supported everywhere. This filtered down to JSON, TOML, and YAML. JSON is an awful configuration format, so that was a no go. TOML is pretty great, but is also not very ubiquitous. That left us with YAML. The second reason is we wanted something language agnostic and not proprietary. It also helps that many other tools, like GitHub actions, are written in YAML. And lastly, we wanted a format that developers are familiar with, and won't need to spend time learning. Requiring developers to understand a special syntax simply to define a task to run is something we want to avoid.

randall 4y ago
IDK. I've been using buck for a bit, and having python within grasp is pretty useful. I don't like yaml for a variety of reasons, but the biggest of which is I like my config being able to be generated... maybe that's an anti pattern? IDK. Would love to know more about your take on yaml vs anything more pythoney.
- computershit 4y ago
  What makes yaml not generatable?
  - randall 4y ago
    Sorry, I meant more dynamic than generatable... like doing stuff in the build file to build something complex in a more simple way. It's really 'do you make config an artifact, or do you make it code' sort of argument.
- mileswjohnson 4y ago
  I'm not very familiar with Buck. But I'm assuming it works the same way as Bazel's Starlark? I can see the benefits of "generating code" within the BUILD file, but honestly, we haven't required it yet, and have been able to do most things with explicit configuration. Our token syntax helps with some dynamic aspects of this. Maybe in the future we'll revisit this.

andrew_ 4y ago

I much prefer YAML rather than having to write more code for a build system.