7 ms·

Can anyone explain to a non-server person what Oxide hopes to accomplish? Is it basically just a new server with its own OS that makes it more secure?

solmag 5y ago
I think this OS is intended to run in embedded context where there are significant memory constraints; read its description, no runtime allocations etc. I linked two speeches where he goes over this in a bit more detail, but I hope the presentation opens it up even more.
detaro 5y ago
Pretty much "let's redo datacenter hardware from the ground up for current requirements, cutting off legacy things we don't need anymore"
- jeffbee 5y ago
  But the BMC would be the #1 item on my list of "things I don't need any more". How do you come up with a scratch legacy-free universe that still includes BMCs?
  - panick21_ 5y ago
    That's exactly what they are doing. The are removing as many thing from the BMC as possible. It only contains a few things, it boots and hand over and allows for some remote control of the low level. That's it.
  - p_l 5y ago
    Because BMC is a term for function (which turns out to be very useful and important) not a specific technology (I like the tongue in cheek "totally not a BMC" used by some people from Oxide)
  - detaro 5y ago
    You don't need low-level remote management anymore? Or what specifically are you associating with the term "BMC"? (i.e. for me, "BMC" is "turn it off and on and force it to boot from network, remotely")
    - jeffbee 5y ago
      Correct. The larger my installation becomes, the less I care about the state of individual machines. The ability to promptly remediate a single broken machine becomes irrelevant at scale.
      - detaro 5y ago
        So if a machine behaves odd/goes away and its OS doesn't respond you don't want management plane to be able to redeploy it/run hardware checks/... automatically?
        jeffbee 5y ago
        If you put it that way you make it too simple. The question is whether I want a second, smaller computer inside my larger computer that may at any time corrupt memory, monkey with network frames, turn off the power, assert PROCHOT, or do a million other bad things. It's not just a tool with benefits. It has both benefits and risks, and in my experience the risks are not worth those benefits.
        detaro 5y ago
        but we are talking in the context of a project which specifically aims to do these things without the baggage of other platforms. And these things are BMC functions.
        p_l 5y ago
        So, you manage computers using crashcarts like in ugly old days of PC servers? I thought that had considerable issue with scaling...
p_l 5y ago
Their target market is essentially private cloud space combined with turnkey rack space - a pretty common on-premise setup where you order not individual servers, but complete racks that are supposed to be "plug and play" in your DC (in practice YMMV, spent a month fighting combination of manglement and Rackable mess). You can think in this case of the final product as pretty big hypervisor cluster that is delivered complete. I'll admit more than once I'd kill for that kind of product, and I suspect that the price/performance ratio might be actually pretty good. The operating system in this case is used for internal service processor bits (compare: Minix 3 on Intel ME, whatever was that proprietary RTOS on AMD's PSP, etc. etc) that help keep the whole thing running and ship shape.
- _alex_ 5y ago
  Bingo. My guess is this is for control plane microcontrollers
  - steveklabnik 5y ago
    It is primarily being used for the root of trust as well as our service processor, aka "totally not a BMC."
  - hinkley 5y ago
    Brian has also complained in interviews about how many microcontrollers are already on your motherboard and how few of them Linux really controls. It's all proprietary and god knows what's actually running in there (and how many bugs and security vulnerabilities they have). None of those are a great situation for multitenant scenarios. This doesn't have to be control plane only. It could also be IO subsystems.
    - p_l 5y ago
      That said, Oxide doesn't get as much control in hands of owner as Raptor, but Raptor doesn't provide high integration rack like that :<
- ksec 5y ago
znpy 5y ago
No, it's a rack level design with the target market being not companies that buy single servers and fill racks with them but hyperscalers that need to fill whole datacenters with servers. Basically there are a huge range of scale levels where having hardware on-prem make sense financially. Also, Bryan Cantrill has some sort of personal nitpick with modern servers basically being x86 PCs with a different form factor, and with the fact that in modern servers hardware and software do not cooperate at all (and in some occasion, hardware gets in the way of software).
- tw04 5y ago
  > but hyberscalers that need to fill whole datacenters with servers. I strongly doubt this is aimed at Amazon, Google, or Microsoft (hyperscalers). They all already have their own highly customized hardware and firmware. If that is their target I wish them luck. There’s no margin and a ton of competition in that space and as long as they’ve been working on this that feels like a pretty poor gamble. What I believe this is actually targeting is small enterprise and up. A company that has dozens to thousands of servers. They’re willing to pay a premium for an easier go to market.
  - znpy 5y ago
    > I strongly doubt this is aimed at Amazon, Google, or Microsoft (hyperscalers). Indeed, that is not aimed at those hyperscalers.
    - ksec 5y ago
      They are pretty much the "only" hyperscalers. The only two you could add is possibly Alibaba and Tencent Cloud.
      - znpy 5y ago
        Joyent?
        5y ago
zweifuss 5y ago
I haven't looked at Oxide in depth. Hubris seems to be about reducing the Attack Surface of a server by * decreasing the active codebase by at least three orders of magnitude * using no C-Code (Rust only?) * most code is kernel independent and not privileged (e.g. drivers, task management, crash recovery) Also: Administration is mostly done by rebooting components.
- detaro 5y ago
  Hubris is for system management components like the BMC, not for the main CPU.
panick21_ 5y ago
Just to be clear, this OS Hubris is for the service processor. Its an OS for firmware, not the main OS that will run on the CPU. However they will likely ship with a something derived from Illumos and bhyve hypervisor. You can then provision VM threw the API (and likely integrated with tools like terraform or whatever). You will likely not interact directly with Illumos. Its basically attempt to help people make running data-center easier.

solmag 5y ago

I think this OS is intended to run in embedded context where there are significant memory constraints; read its description, no runtime allocations etc. I linked two speeches where he goes over this in a bit more detail, but I hope the presentation opens it up even more.

detaro 5y ago

Pretty much "let's redo datacenter hardware from the ground up for current requirements, cutting off legacy things we don't need anymore"

jeffbee 5y ago
But the BMC would be the #1 item on my list of "things I don't need any more". How do you come up with a scratch legacy-free universe that still includes BMCs?
- panick21_ 5y ago
  That's exactly what they are doing. The are removing as many thing from the BMC as possible. It only contains a few things, it boots and hand over and allows for some remote control of the low level. That's it.
- p_l 5y ago
  Because BMC is a term for function (which turns out to be very useful and important) not a specific technology (I like the tongue in cheek "totally not a BMC" used by some people from Oxide)
- detaro 5y ago
  You don't need low-level remote management anymore? Or what specifically are you associating with the term "BMC"? (i.e. for me, "BMC" is "turn it off and on and force it to boot from network, remotely")
  - jeffbee 5y ago
    Correct. The larger my installation becomes, the less I care about the state of individual machines. The ability to promptly remediate a single broken machine becomes irrelevant at scale.
    - detaro 5y ago
      So if a machine behaves odd/goes away and its OS doesn't respond you don't want management plane to be able to redeploy it/run hardware checks/... automatically?
      - jeffbee 5y ago
        If you put it that way you make it too simple. The question is whether I want a second, smaller computer inside my larger computer that may at any time corrupt memory, monkey with network frames, turn off the power, assert PROCHOT, or do a million other bad things. It's not just a tool with benefits. It has both benefits and risks, and in my experience the risks are not worth those benefits.
        detaro 5y ago
        but we are talking in the context of a project which specifically aims to do these things without the baggage of other platforms. And these things are BMC functions.
        p_l 5y ago
        So, you manage computers using crashcarts like in ugly old days of PC servers? I thought that had considerable issue with scaling...

p_l 5y ago

Their target market is essentially private cloud space combined with turnkey rack space - a pretty common on-premise setup where you order not individual servers, but complete racks that are supposed to be "plug and play" in your DC (in practice YMMV, spent a month fighting combination of manglement and Rackable mess). You can think in this case of the final product as pretty big hypervisor cluster that is delivered complete. I'll admit more than once I'd kill for that kind of product, and I suspect that the price/performance ratio might be actually pretty good. The operating system in this case is used for internal service processor bits (compare: Minix 3 on Intel ME, whatever was that proprietary RTOS on AMD's PSP, etc. etc) that help keep the whole thing running and ship shape.

_alex_ 5y ago
Bingo. My guess is this is for control plane microcontrollers
- steveklabnik 5y ago
  It is primarily being used for the root of trust as well as our service processor, aka "totally not a BMC."
- hinkley 5y ago
  Brian has also complained in interviews about how many microcontrollers are already on your motherboard and how few of them Linux really controls. It's all proprietary and god knows what's actually running in there (and how many bugs and security vulnerabilities they have). None of those are a great situation for multitenant scenarios. This doesn't have to be control plane only. It could also be IO subsystems.
  - p_l 5y ago
    That said, Oxide doesn't get as much control in hands of owner as Raptor, but Raptor doesn't provide high integration rack like that :<
ksec 5y ago

znpy 5y ago

No, it's a rack level design with the target market being not companies that buy single servers and fill racks with them but hyperscalers that need to fill whole datacenters with servers. Basically there are a huge range of scale levels where having hardware on-prem make sense financially. Also, Bryan Cantrill has some sort of personal nitpick with modern servers basically being x86 PCs with a different form factor, and with the fact that in modern servers hardware and software do not cooperate at all (and in some occasion, hardware gets in the way of software).

tw04 5y ago
> but hyberscalers that need to fill whole datacenters with servers. I strongly doubt this is aimed at Amazon, Google, or Microsoft (hyperscalers). They all already have their own highly customized hardware and firmware. If that is their target I wish them luck. There’s no margin and a ton of competition in that space and as long as they’ve been working on this that feels like a pretty poor gamble. What I believe this is actually targeting is small enterprise and up. A company that has dozens to thousands of servers. They’re willing to pay a premium for an easier go to market.
- znpy 5y ago
  > I strongly doubt this is aimed at Amazon, Google, or Microsoft (hyperscalers). Indeed, that is not aimed at those hyperscalers.
  - ksec 5y ago
    They are pretty much the "only" hyperscalers. The only two you could add is possibly Alibaba and Tencent Cloud.
    - znpy 5y ago
      Joyent?
      - 5y ago

zweifuss 5y ago

I haven't looked at Oxide in depth. Hubris seems to be about reducing the Attack Surface of a server by * decreasing the active codebase by at least three orders of magnitude * using no C-Code (Rust only?) * most code is kernel independent and not privileged (e.g. drivers, task management, crash recovery) Also: Administration is mostly done by rebooting components.

detaro 5y ago
Hubris is for system management components like the BMC, not for the main CPU.

panick21_ 5y ago

Just to be clear, this OS Hubris is for the service processor. Its an OS for firmware, not the main OS that will run on the CPU. However they will likely ship with a something derived from Illumos and bhyve hypervisor. You can then provision VM threw the API (and likely integrated with tools like terraform or whatever). You will likely not interact directly with Illumos. Its basically attempt to help people make running data-center easier.