3 ms·

Yes, when you're buying enough hardware you treat the rack or row as a purchase unit. A rack is typically hardware plus switching for that rack, so just a few n

by caw 10y ago

Yes, when you're buying enough hardware you treat the rack or row as a purchase unit. A rack is typically hardware plus switching for that rack, so just a few network and power cables are needed to connect a rack to the rest of the infrastructure. If these companies replace a whole rack when a percentage of machines fail that's a business decision based on cost of hardware versus repair costs. Where I worked we had warranties, so the hardware would get fixed (or die in place for things past warranty) and the entire rack would get rolled out at one time, since we bought in rack increments of capacity and thus they all had the same warranty dates. Letting things die in place obviously depends on how much spare capacity you have in your DC, which could be site-specific within large companies.

webmaven 10y ago
> Letting things die in place obviously depends on how much spare capacity you have in your DC, by "capacity" do you mean room? I associate capacity in this context more with power & cooling (neither of which are consumed by disabled machines). Or do you mean that if the DC's capacity is constrained, letting systems die in place and only replacing "mostly dead" racks on a rolling basis keeps capacity utilization below a critical threshold?
- caw 10y ago
  Generally dead systems don't hurt the datacenter in terms of infrastructure utilization, as they don't use power and they don't use cooling, but they could prevent you from bringing in new compute capacity. So you have to figure out whether you can run both the old and slightly broken systems at the same time as the new and faster systems. Capacity could mean physical space, since if floor tiles are occupied by a rack you'll have to put it somewhere else. But even if you have physical space to put another rack somewhere, that needs connected to power and cooling. Depending on your cooling layout (hot boxing, cold boxing, vented floors, etc) you may not have appropriate cooling for a new rack, especially a newer, higher density rack. Or you have sufficient cooling overall, but not for that density in that location because all your high density stuff was planned to go in a designated area. Same thing with power -- you may not have enough power cables for the power distribution units within a rack (you'd typically have 2 so the redundant power supplies don't share a PDU as a point of failure), the cabling may not reach, the electrical panel could be maxed out on amperage or breaker slots, or you'd throw the load on each of the electrical phases too far out of balance (A previous manager of mine was an electrical engineer, I'm not too sure about the real-world technicalities of this other than he tried to keep everything balanced). Networking could also be a limiting factor. You could run out of switch ports or SFPs because you only planned for N connections per racks on however many racks, and now you want to keep hardware around longer.

by caw 10y ago

webmaven 10y ago

> Letting things die in place obviously depends on how much spare capacity you have in your DC, by "capacity" do you mean room? I associate capacity in this context more with power & cooling (neither of which are consumed by disabled machines). Or do you mean that if the DC's capacity is constrained, letting systems die in place and only replacing "mostly dead" racks on a rolling basis keeps capacity utilization below a critical threshold?

caw 10y ago
Generally dead systems don't hurt the datacenter in terms of infrastructure utilization, as they don't use power and they don't use cooling, but they could prevent you from bringing in new compute capacity. So you have to figure out whether you can run both the old and slightly broken systems at the same time as the new and faster systems. Capacity could mean physical space, since if floor tiles are occupied by a rack you'll have to put it somewhere else. But even if you have physical space to put another rack somewhere, that needs connected to power and cooling. Depending on your cooling layout (hot boxing, cold boxing, vented floors, etc) you may not have appropriate cooling for a new rack, especially a newer, higher density rack. Or you have sufficient cooling overall, but not for that density in that location because all your high density stuff was planned to go in a designated area. Same thing with power -- you may not have enough power cables for the power distribution units within a rack (you'd typically have 2 so the redundant power supplies don't share a PDU as a point of failure), the cabling may not reach, the electrical panel could be maxed out on amperage or breaker slots, or you'd throw the load on each of the electrical phases too far out of balance (A previous manager of mine was an electrical engineer, I'm not too sure about the real-world technicalities of this other than he tried to keep everything balanced). Networking could also be a limiting factor. You could run out of switch ports or SFPs because you only planned for N connections per racks on however many racks, and now you want to keep hardware around longer.