3 ms·

The problem with onsite data centers is that it's rare for the total cost to be tracked effectively, they often don't include salaries for example. Additionall

The problem with onsite data centers is that it's rare for the total cost to be tracked effectively, they often don't include salaries for example. Additionally to say that building something outside of the cloud ensures you're cloud agnostic is too simplistic. There are many more shops that would like to move to the cloud to reduce their costs, but can't due to on-site lock-in. They then run into much more expensive problems around expanding datacenters, network or power limitations, or staffing to maintain physical machines which have real world MTBFs you have to do something about when they fail. I think the issue isn't as much lock-in as much as apps being designed poorly, reliant on a specific feature that's not necessary, but then means their app will only work in one cloud as a result. However in the cases where there is no way around that feature, then you're getting a huge benefit of an instant solution so you can focus on your product rather than infrastructure. And if that cloud does provide a single killer feature you need, then just run that feature there.

skuhn 10y ago
I agree that there is no one right answer for all cases. Companies have definitely implemented their own datacenters poorly and been trapped as a result. You can avoid that fate, but you have to hire right and plan right. In my experience, the difference in personnel between a service built in AWS versus one built in datacenters isn't wildly different. If you have a five person team for the former, you might need a six or seven person team for the latter. You aren't dealing with hardware and vendor problems in AWS necessarily, but it's also not as simple as 'just run your app and forget the rest' (at least once you start to scale up). Again though, this is if you're smart about how you build the datacenter. There are a lot of tricks of the trade and a bunch of gotchas that aren't necessarily apparent until you've experienced them for yourself. Hiring experience here is the key. While you don't necessarily want to limit yourself to the lowest common denominator of cloud providers or your own hardware, any time you make use of something unique you need to make a decision. Is it worth getting locked in? What will you do if that solution goes poof in the night? To use a real recent example: what do you do if AWS runs out of X1 instances and you really really need some more right now to do your work? It sucks to get into a situation where you have no control over the outcome and even money can't fix it, either in your own datacenter or in AWS.
- deleted 10y ago
  [deleted]
- xorgar831 10y ago
  You just end up wasting your time maintain relationships with hardware vendors, troubleshooting low level issue, upgrading firmware. It wastes a lot of people's time in the company, not just the team, and you're building one off tooling to work with some vendor's half broken management API. No thanks. :)
  - skuhn 10y ago
    Sure, that happens. If you can save millions of dollars a month and move your business from non-viable to viable, I think it's a good tradeoff. That's what I do for a living, so I see value in it. Not everyone should have to deal with it though. Also, AWS is not problem free. When low level issues (like a firmware problem) strike, which they sometimes will, you often cannot identify or fix the issue yourself. AWS instances still run on that same shitty hardware, and while your odds are better that Amazon figured out most of the issues before you encountered them, your odds are also worse that they tested your particular special use case. So you can wind up playing the 'debug by support ticket' game. Check everything on your end. Check your deployment times for correlations to the problem. Finally give up and reach out to AWS support. Insist that the issue isn't on your end. Find out that the issue can't be resolved until their deployment next week. It's all understandable given the scope of what Amazon is doing, but they will never be as incentivized as you are to solve your problem.
    - xorgar831 10y ago
      I don't think non-viable to viable is exactly what's going on however, at least in most cases. I think the costs are not being fully understood, the impact to the mindshare of the team, the quality of service and redundancy, the additional headcounts and quirky management code that someone's going to be toiling away on when a cloud provider can handle it at a much lower cost. Yes, you still have to open tickets, and yes, they still have problems that suck up a lot of time, however they spend the brunt of the time burning folks out fixing those issues, not you. If you talk to folks working with physical systems, that consumes their mindshare to the point that they're not focusing on their core product anymore.
dorfsmay 10y ago
But, similarly, a lot of people new to the clouds thinks that it runs itself and you need zero staff. You still need somebody to set it up for you!

skuhn 10y ago

I agree that there is no one right answer for all cases. Companies have definitely implemented their own datacenters poorly and been trapped as a result. You can avoid that fate, but you have to hire right and plan right. In my experience, the difference in personnel between a service built in AWS versus one built in datacenters isn't wildly different. If you have a five person team for the former, you might need a six or seven person team for the latter. You aren't dealing with hardware and vendor problems in AWS necessarily, but it's also not as simple as 'just run your app and forget the rest' (at least once you start to scale up). Again though, this is if you're smart about how you build the datacenter. There are a lot of tricks of the trade and a bunch of gotchas that aren't necessarily apparent until you've experienced them for yourself. Hiring experience here is the key. While you don't necessarily want to limit yourself to the lowest common denominator of cloud providers or your own hardware, any time you make use of something unique you need to make a decision. Is it worth getting locked in? What will you do if that solution goes poof in the night? To use a real recent example: what do you do if AWS runs out of X1 instances and you really really need some more right now to do your work? It sucks to get into a situation where you have no control over the outcome and even money can't fix it, either in your own datacenter or in AWS.

deleted 10y ago
[deleted]
xorgar831 10y ago
You just end up wasting your time maintain relationships with hardware vendors, troubleshooting low level issue, upgrading firmware. It wastes a lot of people's time in the company, not just the team, and you're building one off tooling to work with some vendor's half broken management API. No thanks. :)
- skuhn 10y ago
  Sure, that happens. If you can save millions of dollars a month and move your business from non-viable to viable, I think it's a good tradeoff. That's what I do for a living, so I see value in it. Not everyone should have to deal with it though. Also, AWS is not problem free. When low level issues (like a firmware problem) strike, which they sometimes will, you often cannot identify or fix the issue yourself. AWS instances still run on that same shitty hardware, and while your odds are better that Amazon figured out most of the issues before you encountered them, your odds are also worse that they tested your particular special use case. So you can wind up playing the 'debug by support ticket' game. Check everything on your end. Check your deployment times for correlations to the problem. Finally give up and reach out to AWS support. Insist that the issue isn't on your end. Find out that the issue can't be resolved until their deployment next week. It's all understandable given the scope of what Amazon is doing, but they will never be as incentivized as you are to solve your problem.
  - xorgar831 10y ago
    I don't think non-viable to viable is exactly what's going on however, at least in most cases. I think the costs are not being fully understood, the impact to the mindshare of the team, the quality of service and redundancy, the additional headcounts and quirky management code that someone's going to be toiling away on when a cloud provider can handle it at a much lower cost. Yes, you still have to open tickets, and yes, they still have problems that suck up a lot of time, however they spend the brunt of the time burning folks out fixing those issues, not you. If you talk to folks working with physical systems, that consumes their mindshare to the point that they're not focusing on their core product anymore.

dorfsmay 10y ago

But, similarly, a lot of people new to the clouds thinks that it runs itself and you need zero staff. You still need somebody to set it up for you!