4 ms·

It's interesting that Netflix mentions the cost of infrustructure. At the moment, owning dedicated servers and coloing them yourself is orders of magnitude che

by asharp 15y ago

It's interesting that Netflix mentions the cost of infrustructure. At the moment, owning dedicated servers and coloing them yourself is orders of magnitude cheaper then a cloud provider. It's only "cheaper" if your cost of money is stupidly high, ie. you're a startup, or other factors dominate your TCO, ie. you're a startup. What is interesting is that supply side there is no reason for this to be so. Furthermore there are reasons to expect that the equilibrium price for cloud to be below the cost of dedicated hardware/colocation.

ChuckMcM 15y ago
This is spot on. At some point I expect these costs to flip. I've done the TCO computation a couple of ways and for OCP/Web2 type deployments its still cheaper to do your own deployment once you need more than 500 instances. Of course not to take away from Andy's point, if you're deploying the equivalent of Slashdot (which ran on like a dozen servers) then you're silly to spend a couple employee slots on keeping that running in a co-hosted scenario. And these days, given the power of a machine, a LOT of things can be done on a relatively small handful of machines.
- asharp 15y ago
  The point will probably be a few years off, at least for retail prices. Cloudsmithing is still a new art, and there are very few people outside of Amazon who can produce a functional cloud.
nknight 15y ago
Add in salary and benefits for dedicated operations staff, cost of geographic redundancy (duplicate hardware, duplicate colo, travel costs and/or local staff), periodic hardware upgrades, network infrastructure, logistical problems, shipping delays... A friend of mine is currently an operations manager at a company with thousands of physical servers. This is a guy who has spent most of his career in and around datacenters. He's convinced that it's no longer worth whatever "savings" might be seen in owning the hardware, the surrounding and sometimes hidden costs are just too high. Basically, he's already past the point where letting specialists deal with the hardware/provisioning is well worth the cost.
- thwarted 15y ago
  A friend of mine is currently an operations manager at a company with thousands of physical servers. This is a guy who has spent most of his career in and around datacenters. ... he's already past the point where letting specialists deal with the hardware/provisioning is well worth the cost. Isn't this friend of yours one of the "specialists", that some company has hired to provide these services? So he wants to subcontract the work? (and I say this as someone who is in an arguably similar situation and career history as your friend). Those of us in operations need to start managing and deploying our resources like cloud providers do in order to further increase the savings one can (admittedly potentially, if done right) see from keeping things in-house, and keep the control, customization, and choice that one gets from doing things in-house. It's all about scale. If you have thousands of machines and you're still buying hardware one machine at a time and don't have dedicated people just to do things like swap out failed hardware then you aren't scaling effectively.
  - nknight 15y ago
    No, his responsibility is the operation of the service. The operation of the hardware it runs on is not the primary duty, it is a... side effect? Not sure how to put that. It's like saying a building manager should be a carpenter or a plumber. It might be an unsurprising de facto situation, but the primary duty is not to be a carpenter or plumber. I should mention I was briefly working for him at this company, but ran away screaming for reasons not really related to him or the hardware, so I'm fairly familiar with what the deal is. It's not just grunt-level operations, it's operations engineering, complete with in-depth troubleshooting of application problems. This is what Google calls SRE. And they're not buying systems one at a time, they buy them by the truckload from Rackable/SGI, and do have dedicated staff for the things you're talking about. That's the whole problem. The dedicated staff together with the other costs I mentioned end up being more than they would spend being a major customer of a dedicated "cloud" provider.

by asharp 15y ago

ChuckMcM 15y ago

This is spot on. At some point I expect these costs to flip. I've done the TCO computation a couple of ways and for OCP/Web2 type deployments its still cheaper to do your own deployment once you need more than 500 instances. Of course not to take away from Andy's point, if you're deploying the equivalent of Slashdot (which ran on like a dozen servers) then you're silly to spend a couple employee slots on keeping that running in a co-hosted scenario. And these days, given the power of a machine, a LOT of things can be done on a relatively small handful of machines.

asharp 15y ago
The point will probably be a few years off, at least for retail prices. Cloudsmithing is still a new art, and there are very few people outside of Amazon who can produce a functional cloud.

nknight 15y ago

Add in salary and benefits for dedicated operations staff, cost of geographic redundancy (duplicate hardware, duplicate colo, travel costs and/or local staff), periodic hardware upgrades, network infrastructure, logistical problems, shipping delays... A friend of mine is currently an operations manager at a company with thousands of physical servers. This is a guy who has spent most of his career in and around datacenters. He's convinced that it's no longer worth whatever "savings" might be seen in owning the hardware, the surrounding and sometimes hidden costs are just too high. Basically, he's already past the point where letting specialists deal with the hardware/provisioning is well worth the cost.

thwarted 15y ago
A friend of mine is currently an operations manager at a company with thousands of physical servers. This is a guy who has spent most of his career in and around datacenters. ... he's already past the point where letting specialists deal with the hardware/provisioning is well worth the cost. Isn't this friend of yours one of the "specialists", that some company has hired to provide these services? So he wants to subcontract the work? (and I say this as someone who is in an arguably similar situation and career history as your friend). Those of us in operations need to start managing and deploying our resources like cloud providers do in order to further increase the savings one can (admittedly potentially, if done right) see from keeping things in-house, and keep the control, customization, and choice that one gets from doing things in-house. It's all about scale. If you have thousands of machines and you're still buying hardware one machine at a time and don't have dedicated people just to do things like swap out failed hardware then you aren't scaling effectively.
- nknight 15y ago
  No, his responsibility is the operation of the service. The operation of the hardware it runs on is not the primary duty, it is a... side effect? Not sure how to put that. It's like saying a building manager should be a carpenter or a plumber. It might be an unsurprising de facto situation, but the primary duty is not to be a carpenter or plumber. I should mention I was briefly working for him at this company, but ran away screaming for reasons not really related to him or the hardware, so I'm fairly familiar with what the deal is. It's not just grunt-level operations, it's operations engineering, complete with in-depth troubleshooting of application problems. This is what Google calls SRE. And they're not buying systems one at a time, they buy them by the truckload from Rackable/SGI, and do have dedicated staff for the things you're talking about. That's the whole problem. The dedicated staff together with the other costs I mentioned end up being more than they would spend being a major customer of a dedicated "cloud" provider.