9 ms·

The prevailing wisdom throughout the last couple of years was: “ditch your on-prem infrastructure and migrate to a major cloud provider” And its starting to s

by RONROC 5y ago

The prevailing wisdom throughout the last couple of years was: “ditch your on-prem infrastructure and migrate to a major cloud provider” And its starting to seem like it could be something like: “ditch your on-prem infrastructure and spin up your own managed cloud” This is probably untenable for larger orgs where convenience gets the blank check treatment, but for smaller operations that can’t realize that value at scale and are spooked by these outages, what are the alternatives?

paulryanrogers 5y ago
Spread the risk? Smaller on prem and cloud / rented bare metal?
- Spivak 5y ago
  Nah, it's actually better to concentrate the risk in this case. If your app depends on a few 3rd party services -- SendGrid, Twilio, Okta and they're all hosted on different infra then congrats! You're gonna have issues when any one of them are down, yayyy. Also the marketing benefit can't be downplayed. If your postmortem is "AWS was having issues" then your execs and customers just accept that as the cost of doing business because there's a built-in assumption that AWS, Azure, GCP are world class and any in-house team couldn't do it better.
  - aflag 5y ago
    > Also the marketing benefit can't be downplayed. If your postmortem is "AWS was having issues" then your execs and customers just accept that as the cost of doing business because there's a built-in assumption that AWS, Azure, GCP are world class and any in-house team couldn't do it better. In my experience, execs and customers don't treat an outage differently because AWS is at fault. Though the developers do often have the attitude that it's "someone else's problem", which can actually can make execs more worried than if the problem was well known and under the company's control.
Victerius 5y ago
I'm tempted to found a startup to help businesses migrate from cloud providers to on-prem infrastructure.
- datavirtue 5y ago
  Slinging some of that sweet Tanzu or Ranger?
deleted 5y ago
[deleted]
f6v 5y ago
Self-managed infrastructure doesn’t fail now?
- iso1631 5y ago
  Not at this rate. I remember we had a power outage in 2006, it actually took one of my services off air. Since then of course that has been rectified, and the loss of a building wouldn't impact on any of the critical, essential or important services I provide.
  - mbesto 5y ago
    > Not at this rate. Source? Has there ever been an industry wide survey that compares availability from "insert average colo/data center operations" with the cloud ones? And I'm not talking about "we have 12 SREs who are based in Cupertino and are all paid top dollar to support a colo"...I'm talking average.
    - jandrewrogers 5y ago
      I've worked at a few small companies over the years that had their own significant colos and/or data centers built on the cheap, and only a sysadmin or two to run them. Anecdotally, if the infrastructure is setup right, outages are very rare. Some of these were serving up massive loads at the time. I've done these build-outs a few times in my career and it isn't that hard to do, reliable software is more likely to be an issue. The only significant outage I remember is when the redundant power systems in one DC both failed at the same time for different reasons, which can happen. It is as if the software industry has collectively forgotten how to run basic data center operations. Something that used to be a blue collar skill is now treated like arcane magic.
      - ctvo 5y ago
        > It is as if the software industry has collectively forgotten how to run basic data center operations. Something that used to be a blue collar skill is now treated like arcane magic. It's not arcane magic. It's undifferentiated toil that requires hiring for a different skill set than tech companies generally want to hire for. Of course when you get to a certain size it may make sense to take on this cost. I want us to stop pretending individual actors lack the agency to make their own decisions, and they're all blind to how AWS is charging them a fortune for such simple things they can do themselves. You get value from AWS or you stop using AWS.
xyst 5y ago
“Hybrid and multi cloud” is the future. In other words, give us more fucking money.
TameAntelope 5y ago
I don't think it's reasonable to be spooked by these outages, and to think your resolution would be to leave AWS entirely. A much faster and more effective solution that doesn't have you trading cloud problems with on-prem problems (the power outage still happens, except now it's your team that has to handle it) would be to update your services to run in multiple AZs and multiple regions. Get out of AWS is you want, but don't get out of AWS because of outages. You should be able to mitigate this relatively easily.
ernsheong 5y ago
Google Cloud seems to be doing much better, at least recently. There's also Azure. AWS seems to have placed growth above everything else at customers' expense.

by RONROC 5y ago

paulryanrogers 5y ago

Spread the risk? Smaller on prem and cloud / rented bare metal?

Spivak 5y ago
Nah, it's actually better to concentrate the risk in this case. If your app depends on a few 3rd party services -- SendGrid, Twilio, Okta and they're all hosted on different infra then congrats! You're gonna have issues when any one of them are down, yayyy. Also the marketing benefit can't be downplayed. If your postmortem is "AWS was having issues" then your execs and customers just accept that as the cost of doing business because there's a built-in assumption that AWS, Azure, GCP are world class and any in-house team couldn't do it better.
- aflag 5y ago
  > Also the marketing benefit can't be downplayed. If your postmortem is "AWS was having issues" then your execs and customers just accept that as the cost of doing business because there's a built-in assumption that AWS, Azure, GCP are world class and any in-house team couldn't do it better. In my experience, execs and customers don't treat an outage differently because AWS is at fault. Though the developers do often have the attitude that it's "someone else's problem", which can actually can make execs more worried than if the problem was well known and under the company's control.

Victerius 5y ago

I'm tempted to found a startup to help businesses migrate from cloud providers to on-prem infrastructure.

datavirtue 5y ago
Slinging some of that sweet Tanzu or Ranger?

deleted 5y ago

[deleted]

f6v 5y ago

Self-managed infrastructure doesn’t fail now?

iso1631 5y ago
Not at this rate. I remember we had a power outage in 2006, it actually took one of my services off air. Since then of course that has been rectified, and the loss of a building wouldn't impact on any of the critical, essential or important services I provide.
- mbesto 5y ago
  > Not at this rate. Source? Has there ever been an industry wide survey that compares availability from "insert average colo/data center operations" with the cloud ones? And I'm not talking about "we have 12 SREs who are based in Cupertino and are all paid top dollar to support a colo"...I'm talking average.
  - jandrewrogers 5y ago
    I've worked at a few small companies over the years that had their own significant colos and/or data centers built on the cheap, and only a sysadmin or two to run them. Anecdotally, if the infrastructure is setup right, outages are very rare. Some of these were serving up massive loads at the time. I've done these build-outs a few times in my career and it isn't that hard to do, reliable software is more likely to be an issue. The only significant outage I remember is when the redundant power systems in one DC both failed at the same time for different reasons, which can happen. It is as if the software industry has collectively forgotten how to run basic data center operations. Something that used to be a blue collar skill is now treated like arcane magic.
    - ctvo 5y ago
      > It is as if the software industry has collectively forgotten how to run basic data center operations. Something that used to be a blue collar skill is now treated like arcane magic. It's not arcane magic. It's undifferentiated toil that requires hiring for a different skill set than tech companies generally want to hire for. Of course when you get to a certain size it may make sense to take on this cost. I want us to stop pretending individual actors lack the agency to make their own decisions, and they're all blind to how AWS is charging them a fortune for such simple things they can do themselves. You get value from AWS or you stop using AWS.

xyst 5y ago

“Hybrid and multi cloud” is the future. In other words, give us more fucking money.

TameAntelope 5y ago

I don't think it's reasonable to be spooked by these outages, and to think your resolution would be to leave AWS entirely. A much faster and more effective solution that doesn't have you trading cloud problems with on-prem problems (the power outage still happens, except now it's your team that has to handle it) would be to update your services to run in multiple AZs and multiple regions. Get out of AWS is you want, but don't get out of AWS because of outages. You should be able to mitigate this relatively easily.

ernsheong 5y ago

Google Cloud seems to be doing much better, at least recently. There's also Azure. AWS seems to have placed growth above everything else at customers' expense.