4 ms·

> > The power shelf distributes DC power up and down the rack via a bus bar. This eliminates the 70 total AC power supplies found in an equivalent legacy server

• walrus01 2y ago
  The whole thing with eliminating 70 discrete 1U server size AC-to-DC power supplies is nothing new. It's the same general concept as the power distribution unit in the center of an open compute platform rack design from 10+ years ago. Everyone who's doing serious datacenter stuff at scale knows that one of the absolute least efficient, labor intensive and cabling intensive/annoying ways of powering stuff is to have something like a 42U cabinet with 36 servers in it, each of them with dual power supplies, with power leads going to a pair of 208V 30A vertical PDUs in the rear of the cabinet. It gets ugly fast in terms of efficiency. The single point of failure isn't really a problem as long as the software is architected to be tolerant of the disappearance of an entire node (mapping to a single motherboard that is a single or dual cpu socket config with a ton of DDR4 on it).
  • formerly_proven 2y ago
    That’s one reason why 2U4N systems are kinda popular. 1/4 the cabling in legacy infrastructure.
  • jeffbee 2y ago
    PDUs are also very failure-prone and not worth the trouble.
• dralley 2y ago
  >This creates a single point of failure, trading robustness for efficiency. There's nothing wrong with that, but software/ops might have to accommodate by making the opposite tradeoff. I'll happily take a single high qualify power supply (which may have internal redundancy FWIW) over 70 much more cheaply made power supplies that stress other parts of my datacenter via sheer inefficiency, and also costs more in aggregate. Nobody drives down the highway with 10 spare tires for their SUV.
  • hn-throw 2y ago
    Let's say your high quality supply's yearly failure rate is 100 times less than the cheap ones The probability of at least a single failure is 1-(1-r)^70. This is quite high even w/out considering the higher quality of the one supply. The probability of all 70 going down is r^70 which is absurdly low. Let's say r = 0.05 or one failed supply every 20 in a year. 1-(1-r)^70 = 97% r^70 < 1E-91 The high quality supply has r = 0.0005, in between no failure and all failing. If you code can handle node failure, very many, cheaper supplies appears to be more robust. (Assuming uncorrelated events. YMMV)
    • lillecarl 2y ago
      Yeah but the failure rate of an analog piece of copper is pretty low, it'll keep being copper unless you do stupid things. You'll have multiple power supplies provide power on the same piece of copper
      • hn-throw 2y ago
        TL/DR, isnt there a single, shared, DC supply that supplies said piece of copper? Presumably connected to mains? Or are the running on SOFCs?
        • mycoliza 2y ago
          The big piece of copper is fed by redundant rectifiers. Each power shelf has six independent rectifiers which are 5+1 redundant if the rack is fully loaded with compute sleds, or 3+3 redundant if the rack is half-populated. Customers who want more redundancy can also have a second power shelf with six more rectifiers.
          • hn-throw 2y ago
            I'm going to assume this is on 3 phase power, but how is the ripple filtered?
• jsolson 2y ago
  The bus bar itself is an SPoF, but it's also just dumb copper. That doesn't mean that nothing can go wrong, but it's pretty far into the tail of the failure distribution. The power shelf that keeps the busbar fed will have multiple rectifiers, often with at least N+1 redundancy so that you can have a rectifier fail and swap it without the rack itself failing. Similar things apply to the battery shelves.
  • immibis 2y ago
    It's also plausible to have multiple power supplies feeding the same bus bar in parallel (if they're designed to support this) e.g. one at each end of a row.
    • eaasen 2y ago
      This is how our rack works (Oxide employee). In each power shelf, there are 6 power supplies and only 5 need to be functional to run at full load. If you want even more redundancy, you can use both power shelves with independent power feeds to each so even if you lose a feed, the rack still has 5+1 redundant power supplies.
• sidewndr46 2y ago
  This isn't even remotely close. Unless all 32 servers have redundant AC power feeds present, you've traded one single point of failure for another single point of failure. In the event that all 32 servers had redundant AC power feeds, you could just install a pair of redundant DC power feeds.
  • gruez 2y ago
    >Unless all 32 servers have redundant AC power feeds present, you've traded one single point of failure for another single point of failure. Is this not standard? I vaguely remember that rack severs typically have two PSUs for this reason.
    • glitchcrab 2y ago
      It's highly dependent on the individual server model and quite often how you spec it too. Most 1U Dell machines I worked with in the past only had a single slot for a PSU, whereas the beefier 2U (and above) machines generally came with 2 PSUs.
      • thfuran 2y ago
        But 2 PSUs plugged into the same AC supply still have a single point of failure.
        • glitchcrab 2y ago
          Which is why you have two separate PDUs in the rack which are fed by different power feeds and you connect the server's 2 PSUs to opposing PDUs.
          • growse 2y ago
            This works brilliantly, right up to the point where your A side fails, and every single server suddenly doubles their demand on B. Better have good capacity management so you don't go over 100% on B when that happens! (I've seen it happen and take a DC out).
• MisterTea 2y ago
  > This creates a single point of failure, Who told you there is only one PSU in the power shelf?
• sunshowers 2y ago
  Look very carefully at the picture of the rack at https://oxide.computer/ https://oxide.computer/ :) there are two power shelves in the middle, not one. We're absolutely aware of the tradeoffs here and have made quite considered decisions!