5 ms·

Don't forget the lies of cost savings that the Cloud providers have shoved down our industry's throats. We are paying out the nose for cloud services and we are

by jdoss 3y ago

Don't forget the lies of cost savings that the Cloud providers have shoved down our industry's throats. We are paying out the nose for cloud services and we are giving up all the rights to our data. It's a bad deal in the end. I have a bunch of friends that work at SaaS companies and their cloud spend for pretty basic deployments is in the many thousands of dollars a month. Most of their deployments could be handled by a half rack with beefy servers in a couple of datacenters for a fraction of the cost. I pay for a full rack myself and it costs me ~$1200 a month for space, power and bandwidth (10Gb pipe with a current 1Gb commit), and my hardware costs for everything in that rack were a one time cost of around $3000. I have 160 GHz of CPU and 141 GiB of memory for my workloads with a few servers that are not yet provisioned into my Nomad cluster. And before you say well there are costs involved with finding people that have the skills to do that kind of thing and time needed to set all of that up, yes that is true, but our industry has moved from one bucket to another one that is more expensive in the end with a bunch of downsides. I think there is a middle ground where you can use some cloud services and run the important stuff on hardware you own. The tooling to self-host your own stack in a rack of servers you own is light years better than it was 10 years ago and it keeps getting better. Tools like https://nebula.defined.net/docs/ https://nebula.defined.net/docs/ and https://github.com/poseidon/typhoon https://github.com/poseidon/typhoon for example enable you to use whatever providers you want and build a deployment can cost less, gives you more control over your data, while being agile enough to make changes when the team needs something new or different. I am excited for the next 10 years of progress and I'd expect we are going to see more companies self-hosting their deployments on bare metal.

ibejoeb 3y ago
When I switched from colo and managed datacenter services to AWS in its early days, I saved a ton of money. But all of that hardware was much more expensive back then, and the quality of OSS products wasn't quite as good, and all kinds of other temporal realities were at play. Now, it's flipped. Hardware is pretty cheap. There are high quality OSS products that have easy-to-manage, native HA capabilities. It's no longer impractical to operate a production-ready deployment for a few hundred dollars. I guess the early cloud folks saw this coming, and now they're reaping the rewards.
- briffle 3y ago
  we are still way cheaper than before at a previous company. No need to worry about maintaining/monitoring HVAC (plus emergency AC systems) generators, security systems, water intrusion alarms, fire suppression, backhaul links.. Secondary site for DR that has duplicates of everything else. Firewalls that can handle the actual IPSec traffic on those backhaul links. (large Junipers are not cheap) Crazy lead times on new systems (months?!), and needing to size everything to fit our peak load. (we have some machines that add 12x more CPU to the database servers, run their reports in an hour, then remove all those extra CPU's now.)
  - jdoss 3y ago
    It sounds like your workloads need that kind of flexibility and using a cloud provider makes total sense. My point is a lot of deployments most likely don't have those requirements (and never will) and there are way more cost effective means run them.
- jdoss 3y ago
  For sure. I was at a company in the mid 2000s and we had 50+ racks of servers and managing those back in the day was a very big lift. If we had the tooling we do now back then, we could have done it with a fraction of the cost. When I told some of my old coworkers I was going to get my own rack two years ago, there were some comments that I am a glutton for pain and suffering. Overall, it's been a great experience and cost savings on my end. I couldn't afford to run my workloads on any cloud provider. I also understand that I have a skill set that enables me to take advantage of running my own hardware, but with the advancements in tooling and cheaper hardware costs I believe the folks that don't have these skills have a better chance of doing the same. I am hopeful that datacenter providers will shift their business models in the future to enable easier access to getting space, power and bandwidth. That would help enable smaller teams to run their own hardware. The network/bandwidth part has gotten better. Almost all of the major datacenter providers offer some sort of software defined network service so you can get cheaper bandwidth and IP space than from the cloud providers. It is still more expensive in the long run than getting a bandwidth provider yourself. IPv4 space is still expensive but you can lease blocks pretty easily too.
icedchai 3y ago
I've seen companies host a ton of tiny web sites, each in their own AWS account, replicating the same infrastructure N times. Tons of overhead. Total costs is many times what it would cost to host on a few mid range servers in a colo. But, it's "secure" and "scalable". Actual utilization is single digit requests/second, summed across all the sites, on a good day.
- jdoss 3y ago
  The problem is every company wants to have Google level usage so they plan for it and it never happens. This leaves them with an over engineered solution that costs an insane amount of money. The cloud providers continue feed our industry that FUD so companies feel using their services is the best and only option.
lijok 3y ago
> And before you say well there are costs involved with finding people that have the skills to do that kind of thing and time needed to set all of that up, yes that is true, but our industry has moved from one bucket to another one that is more expensive in the end with a bunch of downsides You really glossed over this point as if it wasn’t one of the most important points of debate in the cloud vs colo debate.

by jdoss 3y ago

ibejoeb 3y ago

When I switched from colo and managed datacenter services to AWS in its early days, I saved a ton of money. But all of that hardware was much more expensive back then, and the quality of OSS products wasn't quite as good, and all kinds of other temporal realities were at play. Now, it's flipped. Hardware is pretty cheap. There are high quality OSS products that have easy-to-manage, native HA capabilities. It's no longer impractical to operate a production-ready deployment for a few hundred dollars. I guess the early cloud folks saw this coming, and now they're reaping the rewards.

briffle 3y ago
we are still way cheaper than before at a previous company. No need to worry about maintaining/monitoring HVAC (plus emergency AC systems) generators, security systems, water intrusion alarms, fire suppression, backhaul links.. Secondary site for DR that has duplicates of everything else. Firewalls that can handle the actual IPSec traffic on those backhaul links. (large Junipers are not cheap) Crazy lead times on new systems (months?!), and needing to size everything to fit our peak load. (we have some machines that add 12x more CPU to the database servers, run their reports in an hour, then remove all those extra CPU's now.)
- jdoss 3y ago
  It sounds like your workloads need that kind of flexibility and using a cloud provider makes total sense. My point is a lot of deployments most likely don't have those requirements (and never will) and there are way more cost effective means run them.
jdoss 3y ago
For sure. I was at a company in the mid 2000s and we had 50+ racks of servers and managing those back in the day was a very big lift. If we had the tooling we do now back then, we could have done it with a fraction of the cost. When I told some of my old coworkers I was going to get my own rack two years ago, there were some comments that I am a glutton for pain and suffering. Overall, it's been a great experience and cost savings on my end. I couldn't afford to run my workloads on any cloud provider. I also understand that I have a skill set that enables me to take advantage of running my own hardware, but with the advancements in tooling and cheaper hardware costs I believe the folks that don't have these skills have a better chance of doing the same. I am hopeful that datacenter providers will shift their business models in the future to enable easier access to getting space, power and bandwidth. That would help enable smaller teams to run their own hardware. The network/bandwidth part has gotten better. Almost all of the major datacenter providers offer some sort of software defined network service so you can get cheaper bandwidth and IP space than from the cloud providers. It is still more expensive in the long run than getting a bandwidth provider yourself. IPv4 space is still expensive but you can lease blocks pretty easily too.

icedchai 3y ago

I've seen companies host a ton of tiny web sites, each in their own AWS account, replicating the same infrastructure N times. Tons of overhead. Total costs is many times what it would cost to host on a few mid range servers in a colo. But, it's "secure" and "scalable". Actual utilization is single digit requests/second, summed across all the sites, on a good day.

jdoss 3y ago
The problem is every company wants to have Google level usage so they plan for it and it never happens. This leaves them with an over engineered solution that costs an insane amount of money. The cloud providers continue feed our industry that FUD so companies feel using their services is the best and only option.

lijok 3y ago

> And before you say well there are costs involved with finding people that have the skills to do that kind of thing and time needed to set all of that up, yes that is true, but our industry has moved from one bucket to another one that is more expensive in the end with a bunch of downsides You really glossed over this point as if it wasn’t one of the most important points of debate in the cloud vs colo debate.