4 ms·

MBA here. The cloud is not a good value, the cloud handicaps your organization by placing your most valuable resource - how you compute - in 3rd party hands. Do

MBA here. The cloud is not a good value, the cloud handicaps your organization by placing your most valuable resource - how you compute - in 3rd party hands. Doing the financial math, the cloud is absurd, it is a very bad value; if it were a better value, the cloud providers would not have such high profits. Doing the flexibility math, having one's own datacenter - even a rented colo cabinet - provides far more flexibility than working through a cloud provider's ticketing system. Doing the organization knowledge math, having in-house skills to create, modify, and completely customize one's backend infrastructure with the knowledge that you're numbers are pure - nothing but your jobs are running on your hardware - grants you the visibility to optimize effectively. I've done this: built a 17 server cluster, housed in a colo, for a high compute startup producing 3D avatars from photos automatically. The hardware for the cluster, including a Federal Reserve class hardware firewall, was just over $50K, with a month to month colo bill of $600. This was in contrast to $96K per month if the same configuration were at AWS. This was 7 years ago, so adjust the prices, but you get the exponential higher expense point.

mwcampbell 5y ago
On the other hand, what if your tiny company spends only four figures per month on AWS and has only a single developer who also acts as a sysadmin as needed? Is it even practical for a company operating at that scale to colocate servers in an environment that has anything like the private network and multiple per-region availability zones of AWS? Not to mention that, presumably, the developer would have to implement and test their own equivalent of AWS's auto scaling and multi-AZ database services. To me, AWS is worth the money.
- deleted 5y ago
  [deleted]
- ericd 5y ago
  We colo’d our own servers as a two person team. We didn’t have multiple availability zones, but we’d have the ability to fail over to leased machines if we needed to via automated builds and frequent offsite backups, and that was good enough for our needs. But we never needed to over the few years we ran this setup. This was after running on leased dedicated machines (which is a good first step so you can figure out what you actually need without putting out the money for hardware). A private network is ridiculously easy to set up. In our case, we had two switches, one public, one private. Every machine had 4 NICs. Just bind some to public addresses, some to private, plug in to appropriate switch. Unless your traffic is extremely bursty, auto scaling is probably more trouble than it’s worth. We were serving millions of users with three dual-socket machines. And if you really need it, you can try to handle base load with dedicated servers, and bursts with AWS infra. But at <$10k/mo on AWS, you’re not getting anywhere near the horsepower I’m talking about.
notreallyserio 5y ago
That $600 figure doesn't include the minimum in-house 5 person staff + 1 manager necessary to run a 24x7 high-9s operation that could run operations such as live-migrating workloads to standby systems in order to fix faulty hardware. Additionally that same $96k could be used to run your compute in multiple datacenters. Back of the napkin math suggests that it'd be a wash once you expand to only three DCs. I understand it was a startup and may not have needed high-uptime ops from day 1, but a lot of organizations do, and the cost is not that dissimilar.
- bsenftner 5y ago
  Yes, it was a startup, with a staff of two. Once we had the cluster up, self monitoring, and rotating on command it is literally a half day a week to maintain the entire setup. The hardware firewall was our best investment, as it load balanced while fending off significant DDoS attacks from hackers. It also is entirely required to use an experienced network engineer for such a task. Far too many people try to wing this, and that's where all the publicized horror stories arise.
sofixa 5y ago
> I've done this: built a 17 server cluster, housed in a colo, for a high compute startup producing 3D avatars from photos automatically. The hardware for the cluster, including a Federal Reserve class hardware firewall, was just over $50K, with a month to month colo bill of $600. This was in contrast to $96K per month if the same configuration were at AWS. You also have to include the human time spent setting up and maintaining those servers. And also the fact that you can probably use a managed service on request instead of 17 servers running constantly. Not even mentioning redundancy.
- ericd 5y ago
  You can afford far more humans than you need to maintain 17 servers for $95.5k/month in savings…
- bsenftner 5y ago
  Yes, the human time is an investment that pays back multiple times over. Why the core of a technology business would be in the hands of a 3rd party dismays me. Of course, those 17 servers composed dev, staging, and live, plus fallbacks, ultimately composing quite a few APIs and related services. They were all used for high compute, where a task would floor 32 cores for 3.5 seconds, and that was essentially what the cluster was for, beyond the accounting and tracking of its use.
  - rswail 5y ago
    > core of a technology business Most businesses are not technology businesses, they deliver goods or services to their customers where IT is part of their costs to deliver. Investing in IT people to run that bare metal is a huge expense that doesn't improve your goods or services directly. As for "where a task would floor 32 cores for 3.5 seconds"... means that your internal IT infra had to support that maximum requirement headroom 100% of the time, even though it may not be used for 99% of the time (depending on load patterns etc).
mrep 5y ago
Not sure how an MBA helps here, but if all you are doing is straight compute, then yes, cloud is a waste of money. The benefit of cloud comes when you have a bunch of different teams with different optimal solutions such as nosql, sql, queue services, data processing pipelines, data warehouses, object storage, CDNs...

mwcampbell 5y ago

On the other hand, what if your tiny company spends only four figures per month on AWS and has only a single developer who also acts as a sysadmin as needed? Is it even practical for a company operating at that scale to colocate servers in an environment that has anything like the private network and multiple per-region availability zones of AWS? Not to mention that, presumably, the developer would have to implement and test their own equivalent of AWS's auto scaling and multi-AZ database services. To me, AWS is worth the money.

deleted 5y ago
[deleted]
ericd 5y ago
We colo’d our own servers as a two person team. We didn’t have multiple availability zones, but we’d have the ability to fail over to leased machines if we needed to via automated builds and frequent offsite backups, and that was good enough for our needs. But we never needed to over the few years we ran this setup. This was after running on leased dedicated machines (which is a good first step so you can figure out what you actually need without putting out the money for hardware). A private network is ridiculously easy to set up. In our case, we had two switches, one public, one private. Every machine had 4 NICs. Just bind some to public addresses, some to private, plug in to appropriate switch. Unless your traffic is extremely bursty, auto scaling is probably more trouble than it’s worth. We were serving millions of users with three dual-socket machines. And if you really need it, you can try to handle base load with dedicated servers, and bursts with AWS infra. But at <$10k/mo on AWS, you’re not getting anywhere near the horsepower I’m talking about.

notreallyserio 5y ago

That $600 figure doesn't include the minimum in-house 5 person staff + 1 manager necessary to run a 24x7 high-9s operation that could run operations such as live-migrating workloads to standby systems in order to fix faulty hardware. Additionally that same $96k could be used to run your compute in multiple datacenters. Back of the napkin math suggests that it'd be a wash once you expand to only three DCs. I understand it was a startup and may not have needed high-uptime ops from day 1, but a lot of organizations do, and the cost is not that dissimilar.

bsenftner 5y ago
Yes, it was a startup, with a staff of two. Once we had the cluster up, self monitoring, and rotating on command it is literally a half day a week to maintain the entire setup. The hardware firewall was our best investment, as it load balanced while fending off significant DDoS attacks from hackers. It also is entirely required to use an experienced network engineer for such a task. Far too many people try to wing this, and that's where all the publicized horror stories arise.

sofixa 5y ago

> I've done this: built a 17 server cluster, housed in a colo, for a high compute startup producing 3D avatars from photos automatically. The hardware for the cluster, including a Federal Reserve class hardware firewall, was just over $50K, with a month to month colo bill of $600. This was in contrast to $96K per month if the same configuration were at AWS. You also have to include the human time spent setting up and maintaining those servers. And also the fact that you can probably use a managed service on request instead of 17 servers running constantly. Not even mentioning redundancy.

ericd 5y ago
You can afford far more humans than you need to maintain 17 servers for $95.5k/month in savings…
bsenftner 5y ago
Yes, the human time is an investment that pays back multiple times over. Why the core of a technology business would be in the hands of a 3rd party dismays me. Of course, those 17 servers composed dev, staging, and live, plus fallbacks, ultimately composing quite a few APIs and related services. They were all used for high compute, where a task would floor 32 cores for 3.5 seconds, and that was essentially what the cluster was for, beyond the accounting and tracking of its use.
- rswail 5y ago
  > core of a technology business Most businesses are not technology businesses, they deliver goods or services to their customers where IT is part of their costs to deliver. Investing in IT people to run that bare metal is a huge expense that doesn't improve your goods or services directly. As for "where a task would floor 32 cores for 3.5 seconds"... means that your internal IT infra had to support that maximum requirement headroom 100% of the time, even though it may not be used for 99% of the time (depending on load patterns etc).

mrep 5y ago

Not sure how an MBA helps here, but if all you are doing is straight compute, then yes, cloud is a waste of money. The benefit of cloud comes when you have a bunch of different teams with different optimal solutions such as nosql, sql, queue services, data processing pipelines, data warehouses, object storage, CDNs...