4 ms·

It is a rack of servers, but, every aspect of it is supposed to be engineered to include the full list of things that are needed to make a rack of servers a use

It is a rack of servers, but, every aspect of it is supposed to be engineered to include the full list of things that are needed to make a rack of servers a useful VM hosting setup. So it includes the networking, connection to the service processors which allow you to remotely access each server, other management things, etc. Once installed, you plug in the network connection(s) and add power, then boot up. Then you can add your VMs and start running them.

freeopinion 3y ago
Except that you cut out all the fat of traditional servers. These aren't COTS servers. Imagine if somebody re-engineered an entire rack in the same way they once introduced blade computers. Any part that can be moved from standalone to rackwide should be cut out of individual shelves. So I would expect things like serial buses, video, certain power aspects, etc to all be removed from the individual shelves. When you cut out all that fat, you can fit a lot more muscle in its place. Then you can arrange everything on a rack scale for airflow, power, redundancy, etc. The obvious downside is that you are locked into a single supplier for everything in the rack. Somebody please add a link here for Oxide's strategy to minimize this risk for their customers. (And feel free to correct any mistaken over-simplification.) Overall, they have seemed incredibly open to sharing in a way that allows others to play. That might just be because they would die if they didn't, but I think a lot of people are willing to accept that they are pretty all-around awesome folks trying to make the world better. And making money in the process is not a bad gig. We'll see which other vendors jump into the pool. I wish them great success. But I'm wondering why they haven't coordinated my shipping address for their first shipment. First one's free, right?
- throwawaylinux 3y ago
  > When you cut out all that fat, you can fit a lot more muscle in its place. Then you can arrange everything on a rack scale for airflow, power, redundancy, etc. This is not really true. Data center servers are highly optimized for density already. Like tens of thousands of hours on airflow CFD, tweaking cases and fans and internal layout, heatsink design, etc. chasing the 0.1%s. There are many significant tradeoffs to be made, but there is not a large factor improvement just sitting on the shelf due to a bunch of unused legacy leftovers.
  - goombanast 3y ago
    People have this idea that legacy code or hardware is some huge burden but due to continuous progress in scale the relative size of legacy code or hardware shrinks over time. There might be some exception (e.g. physical connectors) but otherwise you can pretty much include the entirety of a 1995 PC computer (chips and firmware, OS, etc.) inside a tiny, tiny package. The attack surface it leaves open is a different story.
    - bcantrill 3y ago
      So, I don't disagree in the abstract -- but in this case, the presence of these abstractions are functioning as real obstacles with respect to platform enablement, and we found it be a tremendous win to not only eliminate the abstractions entirely.[0] [0] https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-not-to-open-it-the-need-for-holistic-systems/ https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-...
  - __d 3y ago
    Let's imagine a 42U rack full of Dell or HPE servers. Each one has a couple of power supplies, console ports, a video port and a GPU to drive it, USB ports for keyboard, and mouse, etc. More to the point with Oxide, there's layer upon layer of software cruft as well: various levels of proprietary firmware blobs driving the boot process, and then the management stack that underlies a commodity PC platform OS. All of that is gone, replaced with a bottom-up redesign, of both the hardware and the software, with the specific purpose of running large scale modern application loads. This goes well beyond a blade server, and is a completely different beast than a rack of DL360s.
    - throwawaylinux 3y ago
      I know it might seem like it, but there's just not that much in it. All video, USB, "console ports" can be driven from a small chip using a couple of watts. Some do all that on their BMC SoC, even. And if your requirements include redundant power supplies, then that's what you get, and if they don't you get something without them. And the boot and runtime firmware might be complicated and have its own issues, but there are not large runtime performance overheads in that either that you can just "rewrite it all" and get a big speedup. Proof of the pudding will be in the eating I guess. Does Oxide talk about performance advantages at all, or have numbers?

freeopinion 3y ago

Except that you cut out all the fat of traditional servers. These aren't COTS servers. Imagine if somebody re-engineered an entire rack in the same way they once introduced blade computers. Any part that can be moved from standalone to rackwide should be cut out of individual shelves. So I would expect things like serial buses, video, certain power aspects, etc to all be removed from the individual shelves. When you cut out all that fat, you can fit a lot more muscle in its place. Then you can arrange everything on a rack scale for airflow, power, redundancy, etc. The obvious downside is that you are locked into a single supplier for everything in the rack. Somebody please add a link here for Oxide's strategy to minimize this risk for their customers. (And feel free to correct any mistaken over-simplification.) Overall, they have seemed incredibly open to sharing in a way that allows others to play. That might just be because they would die if they didn't, but I think a lot of people are willing to accept that they are pretty all-around awesome folks trying to make the world better. And making money in the process is not a bad gig. We'll see which other vendors jump into the pool. I wish them great success. But I'm wondering why they haven't coordinated my shipping address for their first shipment. First one's free, right?

throwawaylinux 3y ago
> When you cut out all that fat, you can fit a lot more muscle in its place. Then you can arrange everything on a rack scale for airflow, power, redundancy, etc. This is not really true. Data center servers are highly optimized for density already. Like tens of thousands of hours on airflow CFD, tweaking cases and fans and internal layout, heatsink design, etc. chasing the 0.1%s. There are many significant tradeoffs to be made, but there is not a large factor improvement just sitting on the shelf due to a bunch of unused legacy leftovers.
- goombanast 3y ago
  People have this idea that legacy code or hardware is some huge burden but due to continuous progress in scale the relative size of legacy code or hardware shrinks over time. There might be some exception (e.g. physical connectors) but otherwise you can pretty much include the entirety of a 1995 PC computer (chips and firmware, OS, etc.) inside a tiny, tiny package. The attack surface it leaves open is a different story.
  - bcantrill 3y ago
    So, I don't disagree in the abstract -- but in this case, the presence of these abstractions are functioning as real obstacles with respect to platform enablement, and we found it be a tremendous win to not only eliminate the abstractions entirely.[0] [0] https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-not-to-open-it-the-need-for-holistic-systems/ https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-...
- __d 3y ago
  Let's imagine a 42U rack full of Dell or HPE servers. Each one has a couple of power supplies, console ports, a video port and a GPU to drive it, USB ports for keyboard, and mouse, etc. More to the point with Oxide, there's layer upon layer of software cruft as well: various levels of proprietary firmware blobs driving the boot process, and then the management stack that underlies a commodity PC platform OS. All of that is gone, replaced with a bottom-up redesign, of both the hardware and the software, with the specific purpose of running large scale modern application loads. This goes well beyond a blade server, and is a completely different beast than a rack of DL360s.
  - throwawaylinux 3y ago
    I know it might seem like it, but there's just not that much in it. All video, USB, "console ports" can be driven from a small chip using a couple of watts. Some do all that on their BMC SoC, even. And if your requirements include redundant power supplies, then that's what you get, and if they don't you get something without them. And the boot and runtime firmware might be complicated and have its own issues, but there are not large runtime performance overheads in that either that you can just "rewrite it all" and get a big speedup. Proof of the pudding will be in the eating I guess. Does Oxide talk about performance advantages at all, or have numbers?