3 ms·

I'm part of the illumos core team and I'm quite keen to use Rust in the base of illumos, FWIW. There several contexts in which we could make good use of it, an

I'm part of the illumos core team and I'm quite keen to use Rust in the base of illumos, FWIW. There several contexts in which we could make good use of it, and the primary challenges in most cases is either one of release engineering or the resultant binary sizes. I'm confident we'll be able to work them out, it'll just take more exploration and effort! The rough areas where I think things would be useful, and probably also roughly the order in which I would seek to do them: * Rust in the tools we use to build the software. We have a fair amount of Perl and Python and shell script that goes into doing things like making sure the build products are correctly formed, preparing things for packaging, diffing the binary artefacts from two different builds, etc. It would be pretty easy to use regular Cargo-driven Rust development for these tools, to the extent that they don't represent shipped artefacts. * Rust to make C-compatible shared libraries. We ship a lot of libraries in the base system, and I could totally see redoing some of the parts that are harder to get right (e.g., things that poke at cryptographic keys, or do a lot of string parsing, or complex maths!) in Rust. I suspect we would _not_ want to use Cargo for these bits, and probably try to minimise the dependencies and keep tight control on the size of the output binaries etc. * Rust to make kernel modules. Kernel modules are pretty similar to C shared libraries, in that we expect them to have few dependencies and probably not be built through Cargo using crates.io and so on. * Rust to make executable programs like daemons and command-line commands. I think here the temptation to use more dependencies will increase, and so this will be the hardest thing to figure out. The other thing beyond those challenges is that we want the build to work completely offline, _and_ we don't want to needlessly unpack and "vendor" (to use the vernacular) a lot of external files. So we'd probably be looking at using some of the facilities Cargo is growing to use an offline local clone of parts of a repository, and some way to reliably populate that while not inhibiting the ease of development, etc. Nothing insurmountable, just a lot of effort, like most things that are worth doing!

marvin-hansen 2y ago
Have you explored building with Bazel? What you describe as a problem is roughly what Bazel solves: Polyglot complex builds from fully vendored deps. Just pointing this out because I had a fair share of issues with Cargo and ultimately moved to Bazel and a bit later to BuildBuddy as CI. Since then my builds are reliable, run a lot faster and even stuff like cross compilation in a cluster works flawlessly. Obviously there is some complexity implied when moving to Bazel, but the bigger question is whether the capabilities of your current build solution keep up with the complexity of your requirements?
- steveklabnik 2y ago
  (Not Josh, not involved with illumos, do work at Oxide) For at least one project at Oxide we use buck2, which is conceptually in a similar place. I’d like to use it more but am still too new to wield it effectively. In general I would love to see more “here’s how to move to buck/bazel when you outgrow cargo) content.
  - marvin-hansen 2y ago
    Here you go! https://github.com/bazelbuild/examples/tree/main/rust-examples https://github.com/bazelbuild/examples/tree/main/rust-exampl... I wrote all of those examples and contributed them back to Bazel because I've been there... Personally, I prefer the Bazel ecosystem by a wide margin over buck2. By technology alone, buck2 is better, but as my requirements were growing, I needed a lot more mature rule sets such as rules OCI to build and publish container images without Docker and buck2 simply doesn't have the ecosystem available to support complex builds beyond a certain level. It may get there one day.
    - steveklabnik 2y ago
      Thanks! I fully agree with the ecosystem comments; I’m just a Buck fan because it’s in Rust and I like the “no built in rules” concept, but it’s true that it’s much younger and seemingly less widely used. Regardless I should spend some time with Bazel.
      - marvin-hansen 2y ago
        Here is another thing worth sharing. Database integration tests on Bazel used to be difficult. Ultimately I've found an elegant solution: https://github.com/diesel-rs/diesel/blob/master/examples/postgres/custom_arrays/README.md https://github.com/diesel-rs/diesel/blob/master/examples/pos... The parallel testing with dangling transactions isn't Diesel or Postgres specific. You can do the same with pure SQL and any relational DB that supports transactions. For CI, BuildBuddy can spin up Docker in a remote execution host, then you write a custom util that tests if the DB container is already running and if not starts one, out that in a test and then let Bazel execute all integration tests in parallel. For some weird reasons, all tests have to be in one file per insolated remote execution host, so I created one per table. Incremental builds that compile, tests, build and publish images usually complete in about one minute. That's thanks to the 80 Core BuildBuddy cluster with remote cache. GitHub took about an hour back in April when the repo was half in size. There is real gain in terms of developer velocity.
deleted 2y ago
[deleted]

marvin-hansen 2y ago

Have you explored building with Bazel? What you describe as a problem is roughly what Bazel solves: Polyglot complex builds from fully vendored deps. Just pointing this out because I had a fair share of issues with Cargo and ultimately moved to Bazel and a bit later to BuildBuddy as CI. Since then my builds are reliable, run a lot faster and even stuff like cross compilation in a cluster works flawlessly. Obviously there is some complexity implied when moving to Bazel, but the bigger question is whether the capabilities of your current build solution keep up with the complexity of your requirements?

steveklabnik 2y ago
(Not Josh, not involved with illumos, do work at Oxide) For at least one project at Oxide we use buck2, which is conceptually in a similar place. I’d like to use it more but am still too new to wield it effectively. In general I would love to see more “here’s how to move to buck/bazel when you outgrow cargo) content.
- marvin-hansen 2y ago
  Here you go! https://github.com/bazelbuild/examples/tree/main/rust-examples https://github.com/bazelbuild/examples/tree/main/rust-exampl... I wrote all of those examples and contributed them back to Bazel because I've been there... Personally, I prefer the Bazel ecosystem by a wide margin over buck2. By technology alone, buck2 is better, but as my requirements were growing, I needed a lot more mature rule sets such as rules OCI to build and publish container images without Docker and buck2 simply doesn't have the ecosystem available to support complex builds beyond a certain level. It may get there one day.
  - steveklabnik 2y ago
    Thanks! I fully agree with the ecosystem comments; I’m just a Buck fan because it’s in Rust and I like the “no built in rules” concept, but it’s true that it’s much younger and seemingly less widely used. Regardless I should spend some time with Bazel.
    - marvin-hansen 2y ago
      Here is another thing worth sharing. Database integration tests on Bazel used to be difficult. Ultimately I've found an elegant solution: https://github.com/diesel-rs/diesel/blob/master/examples/postgres/custom_arrays/README.md https://github.com/diesel-rs/diesel/blob/master/examples/pos... The parallel testing with dangling transactions isn't Diesel or Postgres specific. You can do the same with pure SQL and any relational DB that supports transactions. For CI, BuildBuddy can spin up Docker in a remote execution host, then you write a custom util that tests if the DB container is already running and if not starts one, out that in a test and then let Bazel execute all integration tests in parallel. For some weird reasons, all tests have to be in one file per insolated remote execution host, so I created one per table. Incremental builds that compile, tests, build and publish images usually complete in about one minute. That's thanks to the 80 Core BuildBuddy cluster with remote cache. GitHub took about an hour back in April when the repo was half in size. There is real gain in terms of developer velocity.

deleted 2y ago

[deleted]