4 ms·

How do you deal with the higher power density of these servers that needs to be put at the ISP locations ? Don't they have some constraints for the open connect

by Aissen 4y ago

How do you deal with the higher power density of these servers that needs to be put at the ISP locations ? Don't they have some constraints for the open connect machines ?

kkielhofner 4y ago
The Dell R7525 chassis is available with dual 800w power supplies. General thinking for power supplies is that each power supply is connected to completely separate power distribution - independent cabling, battery backup, generators, and points of entry to the facility. In many cases it's also two different power grids. This is so that if one power source fails anywhere the load can move over to the other power supply without exceeding the power that can be delivered through a single power supply or trip a breaker anywhere. Under normal operating conditions each power supply is doing half the load. Additionally, the National Electric Code in the US specifies that continuous load should not exceed 80% of given circuit/breaker capacity. So with dual 800 power supplies at "max" 80% load that's "only" 640 watts for one of these 2U servers. For 208V power that's only 3 amps. High density (for sure) compared to the old days but not as ridiculous as it may seem.
- Aissen 4y ago
  You're right, it's not that much for 2U. But for this config, I think they'd probably go for the 1400W power supplies: - 16 x 25W SSDs - 2 x 225W CPUs - On top of that, add RAM, cooling, etc. Honestly, it's still manageable. I doubt they'd put 10 of those in a single rack (you'd need an ISP that would want serve 2.2M subscribers in peak from a single location, not necessarily desirable on their side); but if the site is getting full, you'd feel the (power) pressure (slowly) mounting.
  - kkielhofner 4y ago
    I didn't dig into the CPU config, etc but you're right they'd probably go for the 1400W power supplies which is 5.4 amps max at 208V. It's for an older config (based on the other specs) but the current Netflix OpenConnect docs call for 750w[0] which is more reasonable even for this hardware configuration because no one really wants to consistently run their power supplies at 80% (even in branch loss) for obvious reasons. They absolutely wouldn't want to concentrate them. The entire purpose is to reduce ISP network load and get as close to the customer eyeballs as possible. I don't have any experience with these but I imagine ISPs would install them at their peering "hubs" in major cities - in my experience the usual suspects like Chicago, NYC, Miami, etc. [0] - https://openconnect.zendesk.com/hc/en-us/articles/360034538352 https://openconnect.zendesk.com/hc/en-us/articles/3600345383...
Bluecobra 4y ago
Delivering power is not the problem, cooling is. You can load up a cabinet with four 60A PDU's (~50kW) but the challenge is to cool all that hardware you packed in the cabinet.
- Aissen 4y ago
  Yeah, I was including that in the budget (server fans), but technically you're correct, DC cooling is powered separately.

by Aissen 4y ago

How do you deal with the higher power density of these servers that needs to be put at the ISP locations ? Don't they have some constraints for the open connect machines ?

kkielhofner 4y ago

The Dell R7525 chassis is available with dual 800w power supplies. General thinking for power supplies is that each power supply is connected to completely separate power distribution - independent cabling, battery backup, generators, and points of entry to the facility. In many cases it's also two different power grids. This is so that if one power source fails anywhere the load can move over to the other power supply without exceeding the power that can be delivered through a single power supply or trip a breaker anywhere. Under normal operating conditions each power supply is doing half the load. Additionally, the National Electric Code in the US specifies that continuous load should not exceed 80% of given circuit/breaker capacity. So with dual 800 power supplies at "max" 80% load that's "only" 640 watts for one of these 2U servers. For 208V power that's only 3 amps. High density (for sure) compared to the old days but not as ridiculous as it may seem.

Aissen 4y ago
You're right, it's not that much for 2U. But for this config, I think they'd probably go for the 1400W power supplies: - 16 x 25W SSDs - 2 x 225W CPUs - On top of that, add RAM, cooling, etc. Honestly, it's still manageable. I doubt they'd put 10 of those in a single rack (you'd need an ISP that would want serve 2.2M subscribers in peak from a single location, not necessarily desirable on their side); but if the site is getting full, you'd feel the (power) pressure (slowly) mounting.
- kkielhofner 4y ago
  I didn't dig into the CPU config, etc but you're right they'd probably go for the 1400W power supplies which is 5.4 amps max at 208V. It's for an older config (based on the other specs) but the current Netflix OpenConnect docs call for 750w[0] which is more reasonable even for this hardware configuration because no one really wants to consistently run their power supplies at 80% (even in branch loss) for obvious reasons. They absolutely wouldn't want to concentrate them. The entire purpose is to reduce ISP network load and get as close to the customer eyeballs as possible. I don't have any experience with these but I imagine ISPs would install them at their peering "hubs" in major cities - in my experience the usual suspects like Chicago, NYC, Miami, etc. [0] - https://openconnect.zendesk.com/hc/en-us/articles/360034538352 https://openconnect.zendesk.com/hc/en-us/articles/3600345383...

Bluecobra 4y ago

Delivering power is not the problem, cooling is. You can load up a cabinet with four 60A PDU's (~50kW) but the challenge is to cool all that hardware you packed in the cabinet.

Aissen 4y ago
Yeah, I was including that in the budget (server fans), but technically you're correct, DC cooling is powered separately.