7 ms·

This seems like an insane stance to have, it's like saying businesses should ship their own stock, using their own drivers, and their in-house made cars and pla

This seems like an insane stance to have, it's like saying businesses should ship their own stock, using their own drivers, and their in-house made cars and planes and in-house trained pilots. Heck, why stop at having servers on-site? Cast your own silicon waffers, after all you don't want spectrum exploits. Because you are worst at it. If a specialist is this bad, and the market is fully open, then it's because the problem is hard. AWS has fewer outages in one zone alone than the best self-hosted institutions, your facebooks and petagons. In-house servers would lead to an insane amount of outage. And guess what? AWS (and all other IAAS providers) will beg you to use multiple region because of this. The team/person that has millions of dollars a day staked on a single AWS region is an idiot and could not be entrusted to order a gaming PC from newegg, let alone run an in-house datacenter. edit: I will add that AWS specifically is meh and I wouldn't use it myself, there's better IASS. But it's insanity to even imagine self-hosted is more reliable than using even the shittiest of IASS providers.

roody15 5y ago
Quick follow up. I once used a IASS provider (hyperstreet) that was terrible. Long story short provider ended closing shop and the owner of the company now sells real estate in California. Was a nightmare recovering data. Even when the service was operational was sub par. Just saying perhaps the “shittiest” providers may not be more reliable.
deleted 5y ago
[deleted]
jgwil2 5y ago
> In-house servers would lead to an insane amount of outage. That might be true, but the effects of any given outage would be felt much less widely. If Disney has an outage, I can just find a movie on Netflix to watch instead. But now if one provider goes down, it can take down everything. To me, the problem isn't the cloud per se, it's one player's dominance in the space. We've taken the inherently distributed structure of the internet and re-centralized it, losing some robustness along the way.
- dragonwriter 5y ago
  > That might be true, but the effects of any given outage would be felt much less widely. If my system has an hour of downtime every year and the dozen other systems it interacts with and depends on each have an hour of downtime every year, it can be better that those tend to be correlated rather than independent.
bb88 5y ago
Apple created their own silicon. Fedex uses its own pilots. The USPS uses it's own cars. If you're a company relying upon AWS for your business, is it okay if you're down for a day, or two while you wait for AWS to resolve it's issue?
- lostlogin 5y ago
  It’s bloody annoying when all I want to do is vacuum the floor and Roomba says nope, “active AWS incident”.
  - Grazester 5y ago
    If all you wanted to do was vacuum the floor you would not have gotten that particular vacuum cleaner. Clearly you wanted to do more than just vacuum the floor and something like this happening should be weighed with the purchase of the vacuum.
    - lostlogin 5y ago
      I’ll rephrase. I wanted the floor vacuumed and I didn’t want to do it.
  - bb88 5y ago
    When I bought my automated sprinkler system, I got one that would continue to work if the company or the cloud went belly up.
- teh_klev 5y ago
  > Apple created their own silicon Apple designed their own silicon, a third party manufactures and packages it for them.
  - bb88 5y ago
    Pedantic, -1.
    - 5y ago
kayson 5y ago
I think you're missing the point of the comment. It's not "don't use cloud". It's "be prepared for when cloud goes down". Because it will, despite many companies either thinking it won't, or not planning for it.
midasuni 5y ago
> AWS has fewer outages in one zone alone than the best self-hosted institutions, your facebooks and petagons. In-house servers would lead to an insane amount of outage. It’s had two in 13 months
imiric 5y ago
> This seems like an insane stance to have, it's like saying businesses should ship their own stock, using their own drivers, and their in-house made cars and planes and in-house trained pilots. > Heck, why stop at having servers on-site? Cast your own silicon waffers, after all you don't want spectrum exploits. That's an overblown argument. Nobody is saying that, but it's clear that businesses that maintain their own infrastructure would've avoided today's AWS' outage. So just avoiding a single level of abstraction would've kept your company running today. > Because you are worst at it. If a specialist is this bad, and the market is fully open, then it's because the problem is hard. The problem is hard mostly because of scale. If you're a small business running a few websites with a few million hits per month, it might be cheaper and easier to colocate a few servers and hire a few DevOps or old-school sysadmins to administer the infrastructure. The tooling is there, and is not much more difficult to manage than a hundred different AWS products. I'm actually more worried about the DevOps trend where engineers are trained purely on cloud infrastructure and don't understand low-level tooling these systems are built on. > AWS has fewer outages in one zone alone than the best self-hosted institutions, your facebooks and petagons. In-house servers would lead to an insane amount of outage. That's anecdotal and would depend on the capability of your DevOps team and your in-house / colocation facility. > And guess what? AWS (and all other IAAS providers) will beg you to use multiple region because of this. The team/person that has millions of dollars a day staked on a single AWS region is an idiot and could not be entrusted to order a gaming PC from newegg, let alone run an in-house datacenter. Oh great, so the solution is to put even more of our eggs in a single provider's basket? The real solution would be having failover to a different cloud provider, and the infrastructure changes needed for that are _far_ from trivial. Even with that, there's only 3 major cloud providers you can pick from. Again, colocation in a trusted datacenter would've avoided all of this.
- solveit 5y ago
  This feels like a discussion that could sorely use some numbers. What are good examples of >a small business running a few websites with a few million hits per month, it might be cheaper and easier to colocate a few servers and hire a few DevOps or old-school sysadmins to administer the infrastructure. and how often do they go down?
naikrovek 5y ago
> AWS (and all other IAAS providers) will beg you to use multiple region will they? because AWS still puts new stuff in us-east-1 before anywhere else, and there is often a LONG delay before those things go to other regions. there are many other examples of why people use us-east-1 so often, but it all boils down to this: AWS encourage everyone to use us-east-1 and discourage the use of other regions for the same reasons. if they want to change how and where people deploy, they should change how they encourage it's customers to deploy. my employer uses multi-region deployments where possible, and we can't do that anywhere nearly as much as we'd like because of limitations that AWS has chosen to have. so if cloud providers want to encourage multi-region adoption, they need to stop discouraging and outright preventing it, first.
- danielheath 5y ago
  It works really well imo. All the people who want to use new stuff at the expense of stability choose us-east-1; those who want stability at the expense of new stuff run multi-region (usually not in us-east-1 )
- WaxProlix 5y ago
  Most features roll out to IAD second, third, or fourth. PDX and CMH are good candidates for earlier feature rollout, and usually it's tested in a small region first. I use PDX (us-west-2) for almost everything these days. I also think that they've been making a lot of the default region dropdowns and such point to CMH (us-east-2) to get folks to migrate away from IAD. Your contention that they're encouraging people to use that region just don't ring true to me.
- mypalmike 5y ago
  This argument seems rather contrived. Which feature available in only one region for a very long time has specifically impacted you? And what was the solution?
- CRConrad 5y ago
  > AWS still puts new stuff in us-east-1 before anywhere else, and there is often a LONG delay before those things go to other regions. Come to think of it (far down the second page of comments): Why east? Amazon is still mainly in Seattle, right? And Silicon Valley is in California. So one would have thought the high-tech hub both of Amazon and of the USA in general is still in the west, not east. So why us-east-1 before anywhere else, and not us-west-1?
savant_penguin 5y ago
they usually beg you to use multiple availability zones though I'm not sure how many aws services are easy to spawn at multiple regions
- dragonwriter 5y ago
  > they usually beg you to use multiple availability zones though Doesn't help you if it what goes down is AWS global services on which you directly, or other AWS services, depend (which tend to be tied to US-east-1).
- mypalmike 5y ago
  Which ones are difficult to deploy in multiple regions?
  - savant_penguin 5y ago
    iirc elb+autoscaling
optiomal_isgood 5y ago
This is the right answer, I recall studying for the solutions architect professional certification and reading this countless times: outages will happen and you should plan for them by using multi-region if you care about downtime. It's not AWS fault here, it's the companies', which assume that it will never be down. In-house servers also have outages, it's a very naive assumption to think that it'd be all better if all of those services were using their own servers. Facebook doesn't use AWS and they were down for several hours a couple weeks ago, and that's because they have way better engineers than the average company, working on their infrastructure, exclusively.
qaq 5y ago
"AWS has fewer outages in one zone alone than the best self-hosted institutions" sure you just call an outage "increased error rate"
SkyPuncher 5y ago
In addition to fewer outages, _many_ products get a free pass on incidents because basically everyone is being impacted by outage.
johannes1234321 5y ago
the benefit of self hosting is, that you are up, while a your competitors are down. However if youbare on AWS many of your competitors are down while you are down, so they can't takeover your business.

roody15 5y ago

Quick follow up. I once used a IASS provider (hyperstreet) that was terrible. Long story short provider ended closing shop and the owner of the company now sells real estate in California. Was a nightmare recovering data. Even when the service was operational was sub par. Just saying perhaps the “shittiest” providers may not be more reliable.

deleted 5y ago

[deleted]

jgwil2 5y ago

> In-house servers would lead to an insane amount of outage. That might be true, but the effects of any given outage would be felt much less widely. If Disney has an outage, I can just find a movie on Netflix to watch instead. But now if one provider goes down, it can take down everything. To me, the problem isn't the cloud per se, it's one player's dominance in the space. We've taken the inherently distributed structure of the internet and re-centralized it, losing some robustness along the way.

dragonwriter 5y ago
> That might be true, but the effects of any given outage would be felt much less widely. If my system has an hour of downtime every year and the dozen other systems it interacts with and depends on each have an hour of downtime every year, it can be better that those tend to be correlated rather than independent.

bb88 5y ago

Apple created their own silicon. Fedex uses its own pilots. The USPS uses it's own cars. If you're a company relying upon AWS for your business, is it okay if you're down for a day, or two while you wait for AWS to resolve it's issue?

lostlogin 5y ago
It’s bloody annoying when all I want to do is vacuum the floor and Roomba says nope, “active AWS incident”.
- Grazester 5y ago
  If all you wanted to do was vacuum the floor you would not have gotten that particular vacuum cleaner. Clearly you wanted to do more than just vacuum the floor and something like this happening should be weighed with the purchase of the vacuum.
  - lostlogin 5y ago
    I’ll rephrase. I wanted the floor vacuumed and I didn’t want to do it.
- bb88 5y ago
  When I bought my automated sprinkler system, I got one that would continue to work if the company or the cloud went belly up.
teh_klev 5y ago
> Apple created their own silicon Apple designed their own silicon, a third party manufactures and packages it for them.
- bb88 5y ago
  Pedantic, -1.
  - 5y ago

kayson 5y ago

I think you're missing the point of the comment. It's not "don't use cloud". It's "be prepared for when cloud goes down". Because it will, despite many companies either thinking it won't, or not planning for it.

midasuni 5y ago

> AWS has fewer outages in one zone alone than the best self-hosted institutions, your facebooks and petagons. In-house servers would lead to an insane amount of outage. It’s had two in 13 months

imiric 5y ago

> This seems like an insane stance to have, it's like saying businesses should ship their own stock, using their own drivers, and their in-house made cars and planes and in-house trained pilots. > Heck, why stop at having servers on-site? Cast your own silicon waffers, after all you don't want spectrum exploits. That's an overblown argument. Nobody is saying that, but it's clear that businesses that maintain their own infrastructure would've avoided today's AWS' outage. So just avoiding a single level of abstraction would've kept your company running today. > Because you are worst at it. If a specialist is this bad, and the market is fully open, then it's because the problem is hard. The problem is hard mostly because of scale. If you're a small business running a few websites with a few million hits per month, it might be cheaper and easier to colocate a few servers and hire a few DevOps or old-school sysadmins to administer the infrastructure. The tooling is there, and is not much more difficult to manage than a hundred different AWS products. I'm actually more worried about the DevOps trend where engineers are trained purely on cloud infrastructure and don't understand low-level tooling these systems are built on. > AWS has fewer outages in one zone alone than the best self-hosted institutions, your facebooks and petagons. In-house servers would lead to an insane amount of outage. That's anecdotal and would depend on the capability of your DevOps team and your in-house / colocation facility. > And guess what? AWS (and all other IAAS providers) will beg you to use multiple region because of this. The team/person that has millions of dollars a day staked on a single AWS region is an idiot and could not be entrusted to order a gaming PC from newegg, let alone run an in-house datacenter. Oh great, so the solution is to put even more of our eggs in a single provider's basket? The real solution would be having failover to a different cloud provider, and the infrastructure changes needed for that are _far_ from trivial. Even with that, there's only 3 major cloud providers you can pick from. Again, colocation in a trusted datacenter would've avoided all of this.

solveit 5y ago
This feels like a discussion that could sorely use some numbers. What are good examples of >a small business running a few websites with a few million hits per month, it might be cheaper and easier to colocate a few servers and hire a few DevOps or old-school sysadmins to administer the infrastructure. and how often do they go down?

naikrovek 5y ago

> AWS (and all other IAAS providers) will beg you to use multiple region will they? because AWS still puts new stuff in us-east-1 before anywhere else, and there is often a LONG delay before those things go to other regions. there are many other examples of why people use us-east-1 so often, but it all boils down to this: AWS encourage everyone to use us-east-1 and discourage the use of other regions for the same reasons. if they want to change how and where people deploy, they should change how they encourage it's customers to deploy. my employer uses multi-region deployments where possible, and we can't do that anywhere nearly as much as we'd like because of limitations that AWS has chosen to have. so if cloud providers want to encourage multi-region adoption, they need to stop discouraging and outright preventing it, first.

danielheath 5y ago
It works really well imo. All the people who want to use new stuff at the expense of stability choose us-east-1; those who want stability at the expense of new stuff run multi-region (usually not in us-east-1 )
WaxProlix 5y ago
Most features roll out to IAD second, third, or fourth. PDX and CMH are good candidates for earlier feature rollout, and usually it's tested in a small region first. I use PDX (us-west-2) for almost everything these days. I also think that they've been making a lot of the default region dropdowns and such point to CMH (us-east-2) to get folks to migrate away from IAD. Your contention that they're encouraging people to use that region just don't ring true to me.
mypalmike 5y ago
This argument seems rather contrived. Which feature available in only one region for a very long time has specifically impacted you? And what was the solution?
CRConrad 5y ago
> AWS still puts new stuff in us-east-1 before anywhere else, and there is often a LONG delay before those things go to other regions. Come to think of it (far down the second page of comments): Why east? Amazon is still mainly in Seattle, right? And Silicon Valley is in California. So one would have thought the high-tech hub both of Amazon and of the USA in general is still in the west, not east. So why us-east-1 before anywhere else, and not us-west-1?

savant_penguin 5y ago

they usually beg you to use multiple availability zones though I'm not sure how many aws services are easy to spawn at multiple regions

dragonwriter 5y ago
> they usually beg you to use multiple availability zones though Doesn't help you if it what goes down is AWS global services on which you directly, or other AWS services, depend (which tend to be tied to US-east-1).
mypalmike 5y ago
Which ones are difficult to deploy in multiple regions?
- savant_penguin 5y ago
  iirc elb+autoscaling

optiomal_isgood 5y ago

This is the right answer, I recall studying for the solutions architect professional certification and reading this countless times: outages will happen and you should plan for them by using multi-region if you care about downtime. It's not AWS fault here, it's the companies', which assume that it will never be down. In-house servers also have outages, it's a very naive assumption to think that it'd be all better if all of those services were using their own servers. Facebook doesn't use AWS and they were down for several hours a couple weeks ago, and that's because they have way better engineers than the average company, working on their infrastructure, exclusively.

qaq 5y ago

"AWS has fewer outages in one zone alone than the best self-hosted institutions" sure you just call an outage "increased error rate"

SkyPuncher 5y ago

In addition to fewer outages, _many_ products get a free pass on incidents because basically everyone is being impacted by outage.

johannes1234321 5y ago

the benefit of self hosting is, that you are up, while a your competitors are down. However if youbare on AWS many of your competitors are down while you are down, so they can't takeover your business.