5 ms·

The main drawbacks to me are 1. No support for nested virtualization, so running a vm inside your vm is not available. This prevents use of projects such as ku

The main drawbacks to me are 1. No support for nested virtualization, so running a vm inside your vm is not available. This prevents use of projects such as kubevirt or firecracker on a Linux guest, and WSL2 on a Windows guest. 2. No GPU support If the base hypervisor was Linux, it would be way more capable for users it seems. I also wonder if internally Linux is used for development of the platform itself so they can create "virtual" racks to dogfood the product without full blown physical racks. With all that said, I do not know the roadmap and admittedly there are already quite a few existing platforms built on kvm, so as their hypervisor improves and becomes more capable it could potentially become strategic advantage.

steveklabnik 3y ago
> I also wonder if internally Linux is used for development of the platform itself Developers at Oxide work on whatever platform they'd like, as long as they can do their work. I will say I am in the minority as a Windows user though, most are on some form of Unix. > so they can create "virtual" racks to dogfood the product without full blown physical racks. So one of the reasons why Rust is such an advantage for us is its strong cross-platform support: you can run a simulated version of the control plane on Mac, Linux, and Illumos, without a physical rack. The non-simulated version must run on Helios. [1] That said we do have a rack in the office (literally named dogfood) that employees can use for various things if they wish. 1: https://github.com/oxidecomputer/omicron?tab=readme-ov-file#build-and-run https://github.com/oxidecomputer/omicron?tab=readme-ov-file#...
- moondev 3y ago
  Interesting thanks for the insight. > I will say I am in the minority as a Windows user though, most are on some form of Unix. Now i'm imagining Helios inside WSI - Windows Subsystem for illumos
  - steveklabnik 3y ago
    You're welcome. I will give you one more fun anecdote here: when I came to Oxide, nobody in my corner of the company was using Windows. And hubris and humility almost Just Worked: we had one build system issue that was using strings instead of the path APIs, but as soon as I fixed those, it all worked. bcantrill remarked that if you had gone back in time and told him long ago that some of his code would Just Work on Windows, he would have called you a liar, and it's one of the things that validates our decisions to go with Rust over C as the default language for development inside Oxide. > Now i'm imagining Helios inside WSI - Windows Subsystem for illumos That would be pretty funny, ha! IIRC something about simulated omicron doesn't work inside WSL, but since I don't work on it actively, I haven't bothered to try and patch that up. I think I tried one time, I don't remember specifically what the issue was, as I don't generally use WSL for development, so it's a bit foreign to me as well.
    - panick21_ 3y ago
      > that was using strings instead of the path API Man you can't let Brain live that one down can you? :)
      - steveklabnik 3y ago
        I didn't bother to git blame the code, I myself do this from time to time :)
  - yjftsjthsd-h 3y ago
    > Now i'm imagining Helios inside WSI - Windows Subsystem for illumos I mean... WSL2 is just hyperv with some integration glue, and illumos isn't Linux but unix is unix; that might well be doable.
- fragmede 3y ago
  How is Oxide for GPU-heavy workloads?
  - steveklabnik 3y ago
    There are no GPUs in the rack, so pretty bad, haha. We certainly understand that there's space in the market for a GPU-focused product, but that's a different one than the one we're starting the company off with. There's additional challenge with how we as a company desire openness, and GPUs are incredibly proprietary. We'll see what the future brings. Luckily for us many people still desire good old classic CPU compute.
    - NexRebular 3y ago
      Would pass-through to VM work? At $work I'm running SmartOS servers with GPU passing to a ubuntu bhyve for the occasional CUDA compute and it works wonderfully. Wonder if similar could be possible with Helios?
      - steveklabnik 3y ago
        The software interface isn't the problem: the problem is that there are no physical GPUs in the product. There's nothing to pass through.

steveklabnik 3y ago

> I also wonder if internally Linux is used for development of the platform itself Developers at Oxide work on whatever platform they'd like, as long as they can do their work. I will say I am in the minority as a Windows user though, most are on some form of Unix. > so they can create "virtual" racks to dogfood the product without full blown physical racks. So one of the reasons why Rust is such an advantage for us is its strong cross-platform support: you can run a simulated version of the control plane on Mac, Linux, and Illumos, without a physical rack. The non-simulated version must run on Helios. [1] That said we do have a rack in the office (literally named dogfood) that employees can use for various things if they wish. 1: https://github.com/oxidecomputer/omicron?tab=readme-ov-file#build-and-run https://github.com/oxidecomputer/omicron?tab=readme-ov-file#...

moondev 3y ago
Interesting thanks for the insight. > I will say I am in the minority as a Windows user though, most are on some form of Unix. Now i'm imagining Helios inside WSI - Windows Subsystem for illumos
- steveklabnik 3y ago
  You're welcome. I will give you one more fun anecdote here: when I came to Oxide, nobody in my corner of the company was using Windows. And hubris and humility almost Just Worked: we had one build system issue that was using strings instead of the path APIs, but as soon as I fixed those, it all worked. bcantrill remarked that if you had gone back in time and told him long ago that some of his code would Just Work on Windows, he would have called you a liar, and it's one of the things that validates our decisions to go with Rust over C as the default language for development inside Oxide. > Now i'm imagining Helios inside WSI - Windows Subsystem for illumos That would be pretty funny, ha! IIRC something about simulated omicron doesn't work inside WSL, but since I don't work on it actively, I haven't bothered to try and patch that up. I think I tried one time, I don't remember specifically what the issue was, as I don't generally use WSL for development, so it's a bit foreign to me as well.
  - panick21_ 3y ago
    > that was using strings instead of the path API Man you can't let Brain live that one down can you? :)
    - steveklabnik 3y ago
      I didn't bother to git blame the code, I myself do this from time to time :)
- yjftsjthsd-h 3y ago
  > Now i'm imagining Helios inside WSI - Windows Subsystem for illumos I mean... WSL2 is just hyperv with some integration glue, and illumos isn't Linux but unix is unix; that might well be doable.
fragmede 3y ago
How is Oxide for GPU-heavy workloads?
- steveklabnik 3y ago
  There are no GPUs in the rack, so pretty bad, haha. We certainly understand that there's space in the market for a GPU-focused product, but that's a different one than the one we're starting the company off with. There's additional challenge with how we as a company desire openness, and GPUs are incredibly proprietary. We'll see what the future brings. Luckily for us many people still desire good old classic CPU compute.
  - NexRebular 3y ago
    Would pass-through to VM work? At $work I'm running SmartOS servers with GPU passing to a ubuntu bhyve for the occasional CUDA compute and it works wonderfully. Wonder if similar could be possible with Helios?
    - steveklabnik 3y ago
      The software interface isn't the problem: the problem is that there are no physical GPUs in the product. There's nothing to pass through.