4 ms·

> The development model of illumos is different from Linux and thus there are no Rust drivers in upstream illumos yet. But that is to be expected for new things

> The development model of illumos is different from Linux and thus there are no Rust drivers in upstream illumos yet. But that is to be expected for new things. In our model, we take the time to mature new tech in a fork, and for rust, the Oxide fork has taken that role. In there, we have several drivers for the Oxide Networking stack that are in rust. Based on that some experience could be gained. Soft forks are also good, but doesn't illumos have a stable driver ABI, such that you could just make your drivers completely independently? I thought that was how it had ex. Nvidia drivers ( https://docs.openindiana.org/dev/graphics-stack/#nvidia https://docs.openindiana.org/dev/graphics-stack/#nvidia )

boricj 2y ago
Even if there is a stable ABI driver to the point where upstream is oblivious of any Rust stuff going on inside drivers, that doesn't solve the problem of fragmentation amongst forks (assuming upstreaming to Illumos is a goal).
- jclulow 2y ago
  I'm part of the illumos core team and I'm quite keen to use Rust in the base of illumos, FWIW. There several contexts in which we could make good use of it, and the primary challenges in most cases is either one of release engineering or the resultant binary sizes. I'm confident we'll be able to work them out, it'll just take more exploration and effort! The rough areas where I think things would be useful, and probably also roughly the order in which I would seek to do them: * Rust in the tools we use to build the software. We have a fair amount of Perl and Python and shell script that goes into doing things like making sure the build products are correctly formed, preparing things for packaging, diffing the binary artefacts from two different builds, etc. It would be pretty easy to use regular Cargo-driven Rust development for these tools, to the extent that they don't represent shipped artefacts. * Rust to make C-compatible shared libraries. We ship a lot of libraries in the base system, and I could totally see redoing some of the parts that are harder to get right (e.g., things that poke at cryptographic keys, or do a lot of string parsing, or complex maths!) in Rust. I suspect we would _not_ want to use Cargo for these bits, and probably try to minimise the dependencies and keep tight control on the size of the output binaries etc. * Rust to make kernel modules. Kernel modules are pretty similar to C shared libraries, in that we expect them to have few dependencies and probably not be built through Cargo using crates.io and so on. * Rust to make executable programs like daemons and command-line commands. I think here the temptation to use more dependencies will increase, and so this will be the hardest thing to figure out. The other thing beyond those challenges is that we want the build to work completely offline, _and_ we don't want to needlessly unpack and "vendor" (to use the vernacular) a lot of external files. So we'd probably be looking at using some of the facilities Cargo is growing to use an offline local clone of parts of a repository, and some way to reliably populate that while not inhibiting the ease of development, etc. Nothing insurmountable, just a lot of effort, like most things that are worth doing!
  - marvin-hansen 2y ago
    Have you explored building with Bazel? What you describe as a problem is roughly what Bazel solves: Polyglot complex builds from fully vendored deps. Just pointing this out because I had a fair share of issues with Cargo and ultimately moved to Bazel and a bit later to BuildBuddy as CI. Since then my builds are reliable, run a lot faster and even stuff like cross compilation in a cluster works flawlessly. Obviously there is some complexity implied when moving to Bazel, but the bigger question is whether the capabilities of your current build solution keep up with the complexity of your requirements?
    - steveklabnik 2y ago
      (Not Josh, not involved with illumos, do work at Oxide) For at least one project at Oxide we use buck2, which is conceptually in a similar place. I’d like to use it more but am still too new to wield it effectively. In general I would love to see more “here’s how to move to buck/bazel when you outgrow cargo) content.
      - marvin-hansen 2y ago
        Here you go! https://github.com/bazelbuild/examples/tree/main/rust-examples https://github.com/bazelbuild/examples/tree/main/rust-exampl... I wrote all of those examples and contributed them back to Bazel because I've been there... Personally, I prefer the Bazel ecosystem by a wide margin over buck2. By technology alone, buck2 is better, but as my requirements were growing, I needed a lot more mature rule sets such as rules OCI to build and publish container images without Docker and buck2 simply doesn't have the ecosystem available to support complex builds beyond a certain level. It may get there one day.
        steveklabnik 2y ago
        Thanks! I fully agree with the ecosystem comments; I’m just a Buck fan because it’s in Rust and I like the “no built in rules” concept, but it’s true that it’s much younger and seemingly less widely used. Regardless I should spend some time with Bazel.
cryptonector 2y ago
I don't know if Illumos follows the Sun model of "interface contracts", which allows for downstreams to get timely notice of upcoming backwards-incompatible changes to unstable interfaces. Of course, Illumos can choose to make some intra-kernel interfaces stable.
- jclulow 2y ago
  The answer is definitely not "no", but as far as I'm aware we haven't found any particular cases where that level of formalism and that specific style of arrangement would help. In an open source world where multiple shipping operating systems are based on (different snapshots of) the same core code and interfaces, it seems much more valuable to try and push straight to Committed interfaces (public, stable, documented) where we can. There are plenty of cases where people use Uncommitted interfaces today in things that they layer on top in appliances and so on, but mostly it's expected that those people are paying attention to what's being worked on and integrated. When folks pull new changes from illumos-gate into their release engineering branches or soft forks, they're generally doing that explicitly and testing that things still work for them. Also, unlike the distributions (who ship artefacts to users) it doesn't really make much sense for us to version illumos itself. This makes it a bit harder to reason about when it would be acceptable to renege on an interface contract, etc. In general we try to move things to Committed over time, and then generally try very hard never to break or remove those things after that point.

boricj 2y ago

Even if there is a stable ABI driver to the point where upstream is oblivious of any Rust stuff going on inside drivers, that doesn't solve the problem of fragmentation amongst forks (assuming upstreaming to Illumos is a goal).

jclulow 2y ago
I'm part of the illumos core team and I'm quite keen to use Rust in the base of illumos, FWIW. There several contexts in which we could make good use of it, and the primary challenges in most cases is either one of release engineering or the resultant binary sizes. I'm confident we'll be able to work them out, it'll just take more exploration and effort! The rough areas where I think things would be useful, and probably also roughly the order in which I would seek to do them: * Rust in the tools we use to build the software. We have a fair amount of Perl and Python and shell script that goes into doing things like making sure the build products are correctly formed, preparing things for packaging, diffing the binary artefacts from two different builds, etc. It would be pretty easy to use regular Cargo-driven Rust development for these tools, to the extent that they don't represent shipped artefacts. * Rust to make C-compatible shared libraries. We ship a lot of libraries in the base system, and I could totally see redoing some of the parts that are harder to get right (e.g., things that poke at cryptographic keys, or do a lot of string parsing, or complex maths!) in Rust. I suspect we would _not_ want to use Cargo for these bits, and probably try to minimise the dependencies and keep tight control on the size of the output binaries etc. * Rust to make kernel modules. Kernel modules are pretty similar to C shared libraries, in that we expect them to have few dependencies and probably not be built through Cargo using crates.io and so on. * Rust to make executable programs like daemons and command-line commands. I think here the temptation to use more dependencies will increase, and so this will be the hardest thing to figure out. The other thing beyond those challenges is that we want the build to work completely offline, _and_ we don't want to needlessly unpack and "vendor" (to use the vernacular) a lot of external files. So we'd probably be looking at using some of the facilities Cargo is growing to use an offline local clone of parts of a repository, and some way to reliably populate that while not inhibiting the ease of development, etc. Nothing insurmountable, just a lot of effort, like most things that are worth doing!
- marvin-hansen 2y ago
  Have you explored building with Bazel? What you describe as a problem is roughly what Bazel solves: Polyglot complex builds from fully vendored deps. Just pointing this out because I had a fair share of issues with Cargo and ultimately moved to Bazel and a bit later to BuildBuddy as CI. Since then my builds are reliable, run a lot faster and even stuff like cross compilation in a cluster works flawlessly. Obviously there is some complexity implied when moving to Bazel, but the bigger question is whether the capabilities of your current build solution keep up with the complexity of your requirements?
  - steveklabnik 2y ago
    (Not Josh, not involved with illumos, do work at Oxide) For at least one project at Oxide we use buck2, which is conceptually in a similar place. I’d like to use it more but am still too new to wield it effectively. In general I would love to see more “here’s how to move to buck/bazel when you outgrow cargo) content.
    - marvin-hansen 2y ago
      Here you go! https://github.com/bazelbuild/examples/tree/main/rust-examples https://github.com/bazelbuild/examples/tree/main/rust-exampl... I wrote all of those examples and contributed them back to Bazel because I've been there... Personally, I prefer the Bazel ecosystem by a wide margin over buck2. By technology alone, buck2 is better, but as my requirements were growing, I needed a lot more mature rule sets such as rules OCI to build and publish container images without Docker and buck2 simply doesn't have the ecosystem available to support complex builds beyond a certain level. It may get there one day.
      - steveklabnik 2y ago
        Thanks! I fully agree with the ecosystem comments; I’m just a Buck fan because it’s in Rust and I like the “no built in rules” concept, but it’s true that it’s much younger and seemingly less widely used. Regardless I should spend some time with Bazel.

cryptonector 2y ago

I don't know if Illumos follows the Sun model of "interface contracts", which allows for downstreams to get timely notice of upcoming backwards-incompatible changes to unstable interfaces. Of course, Illumos can choose to make some intra-kernel interfaces stable.

jclulow 2y ago
The answer is definitely not "no", but as far as I'm aware we haven't found any particular cases where that level of formalism and that specific style of arrangement would help. In an open source world where multiple shipping operating systems are based on (different snapshots of) the same core code and interfaces, it seems much more valuable to try and push straight to Committed interfaces (public, stable, documented) where we can. There are plenty of cases where people use Uncommitted interfaces today in things that they layer on top in appliances and so on, but mostly it's expected that those people are paying attention to what's being worked on and integrated. When folks pull new changes from illumos-gate into their release engineering branches or soft forks, they're generally doing that explicitly and testing that things still work for them. Also, unlike the distributions (who ship artefacts to users) it doesn't really make much sense for us to version illumos itself. This makes it a bit harder to reason about when it would be acceptable to renege on an interface contract, etc. In general we try to move things to Committed over time, and then generally try very hard never to break or remove those things after that point.