5 ms·

Skeptical of that. There's only so much you can do against the physics of moving electrons around at high speeds... "Bigger Fans" and "compute density" doesn't

Skeptical of that. There's only so much you can do against the physics of moving electrons around at high speeds... "Bigger Fans" and "compute density" doesn't change that

capital_guy 1y ago
I believe the fans are actually smaller. The rack is definitely quieter than other racks, but he says in the rear rack tour that it's quite hot. Check out these videos of it [0] - https://www.youtube.com/watch?v=dHbgjB0RQ1s https://www.youtube.com/watch?v=dHbgjB0RQ1s [1] - https://www.youtube.com/watch?v=lJmw9OICH-4 https://www.youtube.com/watch?v=lJmw9OICH-4
- steveklabnik 1y ago
  The fans are 80mm, traditionally 1U is roughly half that size.
sunshowers 1y ago
Commodity hardware doesn't quite tend to operate at a compute vs efficiency Pareto frontier — there's a lot of wasted energy that we've been able to optimize with our vertical integration. (I work at Oxide.)
pmichaud 1y ago
I don't have any particular knowledge about oxide's cooling, but think about how bloated and inefficient literally every part of the compute stack is from metal to seeing these words on a screen. If you imagine fixing every part of it to be efficient top to bottom, I think you'll agree that we're not even in the same galaxy as the physical limitations of moving electrons around at high speeds.
- kortilla 1y ago
  But the majority of heat is going to come from the CPU and this is a product to run arbitrary customer workloads. If the customers leave these things idle, then oxide is going to shine. But a busy rack is going to be dominated by CPU heat.
  - zozbot234 1y ago
    Their rack scale from-scratch redesign includes fans big enough that they've reportedly managed to cool CPU hardware that was actually designed for water-cooling, with no expectation for air cooling (though admittedly, they say they only achieved this just barely, and with a LOT of noise). That seems like something that's going to be objectively verifiable as a step up in efficiency.
  - throw0101c 1y ago
    According to Oxide Computer, they found that going from 20mm to 80mm fans dropped their chassis power usage (efficiency is to the cube of the radius): a rack full of 1U servers had 25% of its power going to the fans, and they were able to get down to 1.2%: * https://www.youtube.com/watch?v=hTJYY_Y1H9Q https://www.youtube.com/watch?v=hTJYY_Y1H9Q From their weblog post: > Compared to a popular rackmount server vendor, Oxide is able to fill our specialized racks with 32 AMD Milan sleds and highly-available network switches using less than 15kW per rack, doubling the compute density in a typical data center. With just 16 of the alternative 1U servers and equivalent network switches, over 16kW of power is required per rack, leading to only 1,024 CPU cores vs Oxide’s 2,048. * https://oxide.computer/blog/how-oxide-cuts-data-center-power-consumption-in-half https://oxide.computer/blog/how-oxide-cuts-data-center-power...
    - yencabulator 1y ago
      This is not any different than the "blade" form factor that was popular in the 90s. Shared power and cooling that was not constrained by the height of a 1U rack chassis, with larger fans. Hell, even Supermicro has blade-style chassis with 80mm fans. This is not novel. It's just plain old engineering, optimized to sell whole racks not individual servers or <=8U units, sprinkled with opinions about low-level firmware etc, with a bespoke OS and management stack.
      - deleted 1y ago
        [deleted]
      - throw0101c 1y ago
naikrovek 1y ago
maybe you're not familiar with just how stupidly written most code is. you're right that there are efficiency limits, but not once in my career have I ever seen anyone even attempt to write their code so that it is efficient to run, outside of gaming.
- menaerus 1y ago
  Anything can be improved. Or almost anything. The question is by how much, 5%, 10% or 300%. In this case, I am not really I understand the problem oxide is set to solve so I can't really comment precisely but to me it sounds that if we generally say that data center equipment is suffering from power budget and cooling issues then I don't see this as a problem that software can solve.
  - steveklabnik 1y ago
    > I don't see this as a problem that software can solve. Neither do we, which is why we are building both software and hardware, together. That's the only way to truly tackle thorny issues like these.

Skeptical of that. There's only so much you can do against the physics of moving electrons around at high speeds... "Bigger Fans" and "compute density" doesn't change that

capital_guy 1y ago

I believe the fans are actually smaller. The rack is definitely quieter than other racks, but he says in the rear rack tour that it's quite hot. Check out these videos of it [0] - https://www.youtube.com/watch?v=dHbgjB0RQ1s https://www.youtube.com/watch?v=dHbgjB0RQ1s [1] - https://www.youtube.com/watch?v=lJmw9OICH-4 https://www.youtube.com/watch?v=lJmw9OICH-4

steveklabnik 1y ago
The fans are 80mm, traditionally 1U is roughly half that size.

sunshowers 1y ago

Commodity hardware doesn't quite tend to operate at a compute vs efficiency Pareto frontier — there's a lot of wasted energy that we've been able to optimize with our vertical integration. (I work at Oxide.)

pmichaud 1y ago

I don't have any particular knowledge about oxide's cooling, but think about how bloated and inefficient literally every part of the compute stack is from metal to seeing these words on a screen. If you imagine fixing every part of it to be efficient top to bottom, I think you'll agree that we're not even in the same galaxy as the physical limitations of moving electrons around at high speeds.

kortilla 1y ago
But the majority of heat is going to come from the CPU and this is a product to run arbitrary customer workloads. If the customers leave these things idle, then oxide is going to shine. But a busy rack is going to be dominated by CPU heat.
- zozbot234 1y ago
  Their rack scale from-scratch redesign includes fans big enough that they've reportedly managed to cool CPU hardware that was actually designed for water-cooling, with no expectation for air cooling (though admittedly, they say they only achieved this just barely, and with a LOT of noise). That seems like something that's going to be objectively verifiable as a step up in efficiency.
- throw0101c 1y ago
  According to Oxide Computer, they found that going from 20mm to 80mm fans dropped their chassis power usage (efficiency is to the cube of the radius): a rack full of 1U servers had 25% of its power going to the fans, and they were able to get down to 1.2%: * https://www.youtube.com/watch?v=hTJYY_Y1H9Q https://www.youtube.com/watch?v=hTJYY_Y1H9Q From their weblog post: > Compared to a popular rackmount server vendor, Oxide is able to fill our specialized racks with 32 AMD Milan sleds and highly-available network switches using less than 15kW per rack, doubling the compute density in a typical data center. With just 16 of the alternative 1U servers and equivalent network switches, over 16kW of power is required per rack, leading to only 1,024 CPU cores vs Oxide’s 2,048. * https://oxide.computer/blog/how-oxide-cuts-data-center-power-consumption-in-half https://oxide.computer/blog/how-oxide-cuts-data-center-power...
  - yencabulator 1y ago
    This is not any different than the "blade" form factor that was popular in the 90s. Shared power and cooling that was not constrained by the height of a 1U rack chassis, with larger fans. Hell, even Supermicro has blade-style chassis with 80mm fans. This is not novel. It's just plain old engineering, optimized to sell whole racks not individual servers or <=8U units, sprinkled with opinions about low-level firmware etc, with a bespoke OS and management stack.
    - deleted 1y ago
      [deleted]
    - throw0101c 1y ago

naikrovek 1y ago

maybe you're not familiar with just how stupidly written most code is. you're right that there are efficiency limits, but not once in my career have I ever seen anyone even attempt to write their code so that it is efficient to run, outside of gaming.

menaerus 1y ago
Anything can be improved. Or almost anything. The question is by how much, 5%, 10% or 300%. In this case, I am not really I understand the problem oxide is set to solve so I can't really comment precisely but to me it sounds that if we generally say that data center equipment is suffering from power budget and cooling issues then I don't see this as a problem that software can solve.
- steveklabnik 1y ago
  > I don't see this as a problem that software can solve. Neither do we, which is why we are building both software and hardware, together. That's the only way to truly tackle thorny issues like these.