3 ms·

Add in salary and benefits for dedicated operations staff, cost of geographic redundancy (duplicate hardware, duplicate colo, travel costs and/or local staff),

by nknight 15y ago

Add in salary and benefits for dedicated operations staff, cost of geographic redundancy (duplicate hardware, duplicate colo, travel costs and/or local staff), periodic hardware upgrades, network infrastructure, logistical problems, shipping delays... A friend of mine is currently an operations manager at a company with thousands of physical servers. This is a guy who has spent most of his career in and around datacenters. He's convinced that it's no longer worth whatever "savings" might be seen in owning the hardware, the surrounding and sometimes hidden costs are just too high. Basically, he's already past the point where letting specialists deal with the hardware/provisioning is well worth the cost.

thwarted 15y ago
A friend of mine is currently an operations manager at a company with thousands of physical servers. This is a guy who has spent most of his career in and around datacenters. ... he's already past the point where letting specialists deal with the hardware/provisioning is well worth the cost. Isn't this friend of yours one of the "specialists", that some company has hired to provide these services? So he wants to subcontract the work? (and I say this as someone who is in an arguably similar situation and career history as your friend). Those of us in operations need to start managing and deploying our resources like cloud providers do in order to further increase the savings one can (admittedly potentially, if done right) see from keeping things in-house, and keep the control, customization, and choice that one gets from doing things in-house. It's all about scale. If you have thousands of machines and you're still buying hardware one machine at a time and don't have dedicated people just to do things like swap out failed hardware then you aren't scaling effectively.
- nknight 15y ago
  No, his responsibility is the operation of the service. The operation of the hardware it runs on is not the primary duty, it is a... side effect? Not sure how to put that. It's like saying a building manager should be a carpenter or a plumber. It might be an unsurprising de facto situation, but the primary duty is not to be a carpenter or plumber. I should mention I was briefly working for him at this company, but ran away screaming for reasons not really related to him or the hardware, so I'm fairly familiar with what the deal is. It's not just grunt-level operations, it's operations engineering, complete with in-depth troubleshooting of application problems. This is what Google calls SRE. And they're not buying systems one at a time, they buy them by the truckload from Rackable/SGI, and do have dedicated staff for the things you're talking about. That's the whole problem. The dedicated staff together with the other costs I mentioned end up being more than they would spend being a major customer of a dedicated "cloud" provider.

by nknight 15y ago

thwarted 15y ago

A friend of mine is currently an operations manager at a company with thousands of physical servers. This is a guy who has spent most of his career in and around datacenters. ... he's already past the point where letting specialists deal with the hardware/provisioning is well worth the cost. Isn't this friend of yours one of the "specialists", that some company has hired to provide these services? So he wants to subcontract the work? (and I say this as someone who is in an arguably similar situation and career history as your friend). Those of us in operations need to start managing and deploying our resources like cloud providers do in order to further increase the savings one can (admittedly potentially, if done right) see from keeping things in-house, and keep the control, customization, and choice that one gets from doing things in-house. It's all about scale. If you have thousands of machines and you're still buying hardware one machine at a time and don't have dedicated people just to do things like swap out failed hardware then you aren't scaling effectively.

nknight 15y ago
No, his responsibility is the operation of the service. The operation of the hardware it runs on is not the primary duty, it is a... side effect? Not sure how to put that. It's like saying a building manager should be a carpenter or a plumber. It might be an unsurprising de facto situation, but the primary duty is not to be a carpenter or plumber. I should mention I was briefly working for him at this company, but ran away screaming for reasons not really related to him or the hardware, so I'm fairly familiar with what the deal is. It's not just grunt-level operations, it's operations engineering, complete with in-depth troubleshooting of application problems. This is what Google calls SRE. And they're not buying systems one at a time, they buy them by the truckload from Rackable/SGI, and do have dedicated staff for the things you're talking about. That's the whole problem. The dedicated staff together with the other costs I mentioned end up being more than they would spend being a major customer of a dedicated "cloud" provider.