5 ms·

Seeing all these horror stories, I can't help but wonder when people are going to switch back to on prem, or at least some hybrid. It's pretty apparent we can'

Seeing all these horror stories, I can't help but wonder when people are going to switch back to on prem, or at least some hybrid. It's pretty apparent we can't trust cloud providers...

reeeeee 4y ago
I'm not so sure. Aren't we having a bit of negativity bias here, as we mostly hear about the negative events while there are probably countless successful integrations with cloud providers.
- oars 4y ago
  Exactly. This is truly negativity bias. Look at all the banks shifting their infra to the cloud. Haven't heard anything bad happen from that. And their uptime and business requirements are as stringent as anyone else, even further scrutinized by regulators and auditors.
- dustinmoris 4y ago
  Yes, if you'd compare all horror stories of people having huge downtimes because their on-prem system collapsed vs. people having an issue with their cloud provider then we'd very quickly realise again why the cloud has taken us by storm.
fxtentacle 4y ago
I'd guess that a large percentage of SMBs are running on-prem, so you don't hear about switching back because they avoided the cloud in the first place.
cbg0 4y ago
Expect to have issues with colo as well, unless you own the building.
12ian34 4y ago
I think it's a vocal minority. The vast majority of GCS customers won't have this problem. These kind of posts typically do very well on HN.
- vidarh 4y ago
  It doesn't take most to have this problem. It takes a reasonable proportion to fear they could have this problem. That said, I think cost is more likely to drive change here than fear over billing - most people pick lover cost over risk mitigation far more often than they'd like to admit.
nibbleshifter 4y ago
A number of orgs I've worked with in the last few years have gone from on prem to full cloud to hybrid, with plans to bring more stuff on-prem in the future.
quickthrower2 4y ago
No need for on prem (rolling your own cloud just means it might be you that fucks up, or your ISP anyway). If you use standard legos like VMs, simple functions, containers, k8s, postgresql etc. then you can have a replica of your site on another provider. Scale it down and keep the data in cold storage. Flick the on switch for an hour a month to check it all works.
unity1001 4y ago
People won't switch to on-prem because if Google and some other providers screw up, there still are many other cloud providers. For K8 they could have gone with anyone from Digitalocean to Linode. Actually, they could have already replicated their cluster to more than one provider to use as failover in such a case. (assuming that they were using K8)
vidarh 4y ago
The pendulum will swing over cost, not over outages like these, because the cost challenges are real, while these outages are rare. It won't go all back to on prem because cloud providers will slash their margins once they struggle to maintain growth, and doing so they'll diminish the advantages of on-prem/hybrid setups for a lot of people, and so we'll find some equilibrium or other. I'd expect more emphasis on hybrid between multiple clouds rather than cloud vs. on prem to happen first. E.g. layers to threat more parts of cloud services as commodities.
- marcinzm 4y ago
  Full on prem is a pretty bad for a large company because your company now depends on a single monopolistic service provider that is run as a cost center (ie: the department managing your on prem stuff). Eventually that leads to a very bad engineer experience which leads to a massive opportunity cost for any tech company. It works for a while and then over time becomes more and more dysfunctional. Hybrid at least includes competition with cloud providers which makes things less dysfunctional.
  - vidarh 4y ago
    Most places de facto depend on a single provider that becomes a cost center when using cloud providers too. While you can shop around, in effect it tends to get entrenched and get more and more dysfunctional as people try to exploit characteristics of your approvals process for using resources from your chosen provider. For a big company, you need a team to manage these resources - I've lost count of the number of recruiters who has contacted me just this year for roles involving management or architecture of cloud strategy etc. because they're building whole supporting organisations around a specific paradigm for hosting rather than building an organisation responsible for understanding and providing the best possible substrate for their applications. I'm all for hybrid solutions, but mostly because it means you usually end up being able to cut the cost of on prem, colocation or managed servers even more relative to cloud setups because you can increase the utilisation rate (e.g. run your base load on cheap Hetzner servers but being able to spin up EC2 instances on short notice to take spices or handle failures up to and including all of Hetzner falling off the face of the earth). Most larger managed providers today also offers cloud services and many also offers colo services, so you can often easily mix and match as long as you're conscious off egress pricing which is often the biggest barrier to such setups today (especially if the big cloud providers are in the mix, though it's better than it was). That said, I prefer not putting my eggs in one basket, and instead going the direction of putting in place layers to treat a multi-provider setup as one. In the past I've e.g. done zero downtime migrations between AWS->GCP->Hetzner that way, and also had systems split between actual on prem (as in a data room at our office), multiple colo's, dedicated servers at Hetzner, and a couple of VM providers where everything was transparent to our engineers (they didn't need to know on what continent a service ran unless doing performance optimisation or reliability engineering). We prepped to tie in AWS resources too, but it never become cost effective for that company to do so vs. the prices of the other providers used - it would have been if we had sudden extreme load spikes, but it was a business where traffic was linked closely to physical capacity at restaurants and nightclubs, and so traffic was very predictable.
theshrike79 4y ago
On-prem requires a good half-dozen people just for managing the servers and doing on-call rotation. For a 10-20 person company that's a huge expense with not that much ROI.

Seeing all these horror stories, I can't help but wonder when people are going to switch back to on prem, or at least some hybrid. It's pretty apparent we can't trust cloud providers...

reeeeee 4y ago

I'm not so sure. Aren't we having a bit of negativity bias here, as we mostly hear about the negative events while there are probably countless successful integrations with cloud providers.

oars 4y ago
Exactly. This is truly negativity bias. Look at all the banks shifting their infra to the cloud. Haven't heard anything bad happen from that. And their uptime and business requirements are as stringent as anyone else, even further scrutinized by regulators and auditors.
dustinmoris 4y ago
Yes, if you'd compare all horror stories of people having huge downtimes because their on-prem system collapsed vs. people having an issue with their cloud provider then we'd very quickly realise again why the cloud has taken us by storm.

fxtentacle 4y ago

I'd guess that a large percentage of SMBs are running on-prem, so you don't hear about switching back because they avoided the cloud in the first place.

cbg0 4y ago

Expect to have issues with colo as well, unless you own the building.

12ian34 4y ago

I think it's a vocal minority. The vast majority of GCS customers won't have this problem. These kind of posts typically do very well on HN.

vidarh 4y ago
It doesn't take most to have this problem. It takes a reasonable proportion to fear they could have this problem. That said, I think cost is more likely to drive change here than fear over billing - most people pick lover cost over risk mitigation far more often than they'd like to admit.

nibbleshifter 4y ago

A number of orgs I've worked with in the last few years have gone from on prem to full cloud to hybrid, with plans to bring more stuff on-prem in the future.

quickthrower2 4y ago

No need for on prem (rolling your own cloud just means it might be you that fucks up, or your ISP anyway). If you use standard legos like VMs, simple functions, containers, k8s, postgresql etc. then you can have a replica of your site on another provider. Scale it down and keep the data in cold storage. Flick the on switch for an hour a month to check it all works.

unity1001 4y ago

People won't switch to on-prem because if Google and some other providers screw up, there still are many other cloud providers. For K8 they could have gone with anyone from Digitalocean to Linode. Actually, they could have already replicated their cluster to more than one provider to use as failover in such a case. (assuming that they were using K8)

vidarh 4y ago

The pendulum will swing over cost, not over outages like these, because the cost challenges are real, while these outages are rare. It won't go all back to on prem because cloud providers will slash their margins once they struggle to maintain growth, and doing so they'll diminish the advantages of on-prem/hybrid setups for a lot of people, and so we'll find some equilibrium or other. I'd expect more emphasis on hybrid between multiple clouds rather than cloud vs. on prem to happen first. E.g. layers to threat more parts of cloud services as commodities.

marcinzm 4y ago
Full on prem is a pretty bad for a large company because your company now depends on a single monopolistic service provider that is run as a cost center (ie: the department managing your on prem stuff). Eventually that leads to a very bad engineer experience which leads to a massive opportunity cost for any tech company. It works for a while and then over time becomes more and more dysfunctional. Hybrid at least includes competition with cloud providers which makes things less dysfunctional.
- vidarh 4y ago
  Most places de facto depend on a single provider that becomes a cost center when using cloud providers too. While you can shop around, in effect it tends to get entrenched and get more and more dysfunctional as people try to exploit characteristics of your approvals process for using resources from your chosen provider. For a big company, you need a team to manage these resources - I've lost count of the number of recruiters who has contacted me just this year for roles involving management or architecture of cloud strategy etc. because they're building whole supporting organisations around a specific paradigm for hosting rather than building an organisation responsible for understanding and providing the best possible substrate for their applications. I'm all for hybrid solutions, but mostly because it means you usually end up being able to cut the cost of on prem, colocation or managed servers even more relative to cloud setups because you can increase the utilisation rate (e.g. run your base load on cheap Hetzner servers but being able to spin up EC2 instances on short notice to take spices or handle failures up to and including all of Hetzner falling off the face of the earth). Most larger managed providers today also offers cloud services and many also offers colo services, so you can often easily mix and match as long as you're conscious off egress pricing which is often the biggest barrier to such setups today (especially if the big cloud providers are in the mix, though it's better than it was). That said, I prefer not putting my eggs in one basket, and instead going the direction of putting in place layers to treat a multi-provider setup as one. In the past I've e.g. done zero downtime migrations between AWS->GCP->Hetzner that way, and also had systems split between actual on prem (as in a data room at our office), multiple colo's, dedicated servers at Hetzner, and a couple of VM providers where everything was transparent to our engineers (they didn't need to know on what continent a service ran unless doing performance optimisation or reliability engineering). We prepped to tie in AWS resources too, but it never become cost effective for that company to do so vs. the prices of the other providers used - it would have been if we had sudden extreme load spikes, but it was a business where traffic was linked closely to physical capacity at restaurants and nightclubs, and so traffic was very predictable.

theshrike79 4y ago

On-prem requires a good half-dozen people just for managing the servers and doing on-call rotation. For a 10-20 person company that's a huge expense with not that much ROI.