4 ms·

Have you explored building with Bazel? What you describe as a problem is roughly what Bazel solves: Polyglot complex builds from fully vendored deps. Just poin

Have you explored building with Bazel? What you describe as a problem is roughly what Bazel solves: Polyglot complex builds from fully vendored deps. Just pointing this out because I had a fair share of issues with Cargo and ultimately moved to Bazel and a bit later to BuildBuddy as CI. Since then my builds are reliable, run a lot faster and even stuff like cross compilation in a cluster works flawlessly. Obviously there is some complexity implied when moving to Bazel, but the bigger question is whether the capabilities of your current build solution keep up with the complexity of your requirements?

steveklabnik 2y ago
(Not Josh, not involved with illumos, do work at Oxide) For at least one project at Oxide we use buck2, which is conceptually in a similar place. I’d like to use it more but am still too new to wield it effectively. In general I would love to see more “here’s how to move to buck/bazel when you outgrow cargo) content.
- marvin-hansen 2y ago
  Here you go! https://github.com/bazelbuild/examples/tree/main/rust-examples https://github.com/bazelbuild/examples/tree/main/rust-exampl... I wrote all of those examples and contributed them back to Bazel because I've been there... Personally, I prefer the Bazel ecosystem by a wide margin over buck2. By technology alone, buck2 is better, but as my requirements were growing, I needed a lot more mature rule sets such as rules OCI to build and publish container images without Docker and buck2 simply doesn't have the ecosystem available to support complex builds beyond a certain level. It may get there one day.
  - steveklabnik 2y ago
    Thanks! I fully agree with the ecosystem comments; I’m just a Buck fan because it’s in Rust and I like the “no built in rules” concept, but it’s true that it’s much younger and seemingly less widely used. Regardless I should spend some time with Bazel.
    - marvin-hansen 2y ago
      Here is another thing worth sharing. Database integration tests on Bazel used to be difficult. Ultimately I've found an elegant solution: https://github.com/diesel-rs/diesel/blob/master/examples/postgres/custom_arrays/README.md https://github.com/diesel-rs/diesel/blob/master/examples/pos... The parallel testing with dangling transactions isn't Diesel or Postgres specific. You can do the same with pure SQL and any relational DB that supports transactions. For CI, BuildBuddy can spin up Docker in a remote execution host, then you write a custom util that tests if the DB container is already running and if not starts one, out that in a test and then let Bazel execute all integration tests in parallel. For some weird reasons, all tests have to be in one file per insolated remote execution host, so I created one per table. Incremental builds that compile, tests, build and publish images usually complete in about one minute. That's thanks to the 80 Core BuildBuddy cluster with remote cache. GitHub took about an hour back in April when the repo was half in size. There is real gain in terms of developer velocity.
  - hucker 2y ago
    Agreed, my company tried buck2 first when evaluating a cargo replacement, but bazel is just so much more mature at this point that it ended up being the natural choice. Thanks for your examples, they helped us get started :)
jclulow 2y ago
I'm not hugely excited about Bazel, or really any of the more complex build tools made either directly for, or in the image of, the large corporate monorepo tooling. Right now we have decades of accumulated make stuff, with some orchestrating shell. It's good in parts, and not in others. What I would ideally like is some bespoke software that produces a big Ninja file to build literally everything. A colleague at Oxide spent at least a few hours poking at a generator that might eventually suit us, and while it's not anywhere near finished I thought it was a promising beginning I'd like us to pursue eventually! https://github.com/oxidecomputer/eos https://github.com/oxidecomputer/eos

steveklabnik 2y ago

(Not Josh, not involved with illumos, do work at Oxide) For at least one project at Oxide we use buck2, which is conceptually in a similar place. I’d like to use it more but am still too new to wield it effectively. In general I would love to see more “here’s how to move to buck/bazel when you outgrow cargo) content.

marvin-hansen 2y ago
Here you go! https://github.com/bazelbuild/examples/tree/main/rust-examples https://github.com/bazelbuild/examples/tree/main/rust-exampl... I wrote all of those examples and contributed them back to Bazel because I've been there... Personally, I prefer the Bazel ecosystem by a wide margin over buck2. By technology alone, buck2 is better, but as my requirements were growing, I needed a lot more mature rule sets such as rules OCI to build and publish container images without Docker and buck2 simply doesn't have the ecosystem available to support complex builds beyond a certain level. It may get there one day.
- steveklabnik 2y ago
  Thanks! I fully agree with the ecosystem comments; I’m just a Buck fan because it’s in Rust and I like the “no built in rules” concept, but it’s true that it’s much younger and seemingly less widely used. Regardless I should spend some time with Bazel.
  - marvin-hansen 2y ago
    Here is another thing worth sharing. Database integration tests on Bazel used to be difficult. Ultimately I've found an elegant solution: https://github.com/diesel-rs/diesel/blob/master/examples/postgres/custom_arrays/README.md https://github.com/diesel-rs/diesel/blob/master/examples/pos... The parallel testing with dangling transactions isn't Diesel or Postgres specific. You can do the same with pure SQL and any relational DB that supports transactions. For CI, BuildBuddy can spin up Docker in a remote execution host, then you write a custom util that tests if the DB container is already running and if not starts one, out that in a test and then let Bazel execute all integration tests in parallel. For some weird reasons, all tests have to be in one file per insolated remote execution host, so I created one per table. Incremental builds that compile, tests, build and publish images usually complete in about one minute. That's thanks to the 80 Core BuildBuddy cluster with remote cache. GitHub took about an hour back in April when the repo was half in size. There is real gain in terms of developer velocity.
- hucker 2y ago
  Agreed, my company tried buck2 first when evaluating a cargo replacement, but bazel is just so much more mature at this point that it ended up being the natural choice. Thanks for your examples, they helped us get started :)

jclulow 2y ago

I'm not hugely excited about Bazel, or really any of the more complex build tools made either directly for, or in the image of, the large corporate monorepo tooling. Right now we have decades of accumulated make stuff, with some orchestrating shell. It's good in parts, and not in others. What I would ideally like is some bespoke software that produces a big Ninja file to build literally everything. A colleague at Oxide spent at least a few hours poking at a generator that might eventually suit us, and while it's not anywhere near finished I thought it was a promising beginning I'd like us to pursue eventually! https://github.com/oxidecomputer/eos https://github.com/oxidecomputer/eos