4 ms·

I did an extensive cost analysis of a 1MW datacenter vs Cloud 10 years or so ago, so this stale data but take it as an anecdote. Accounting for the lease, facil

by xfitm3 4y ago

I did an extensive cost analysis of a 1MW datacenter vs Cloud 10 years or so ago, so this stale data but take it as an anecdote. Accounting for the lease, facillity equipment (UPS, STS, CRACs, generator, fuel, raised flooring, cabinets, BMS, etc), compute equipment (PDUs, servers, spare parts), facilities and technical staff, the costs for AWS were substantially cheaper. The trick is you need skills, most of which are not in demand anymore and difficult to fill. If you pretend like you can hire and manage: the results are clear. There are opex/capex aspects which I don't understand, I amortized servers with an average server lifetime of 3 years. It's easier for most people to spend money with one vendor, though, and that's where AWS/GCP makes sense.

tluyben2 4y ago
It really depends on your demands; a lot of companies are running 1000s of trivial web applications and that’s it. Those can run on 10 year old servers which cost nothing at all anymore and never break; I have a few racks of them (HP dl360). We have not had issues in our colo for those 10 years; I didn’t even replace a drive and these are mostly spinning. Sure cloud is easier and if you need things that aws offers then go for it; we run well on a few $100/mo. Our most sold/popular product costs (I did this calc because of another ‘cloud is great’ HN post a few months back) less than $1/mo to host and maintain and has 50k+ active users. I like and use aws but it makes no sense in many cases; I would rob myself running most of our products on it because they don’t require remotely anything that it offers. I think the ‘make a unicorn’ mindset messed up people’s bottom line understanding; making 10k-100k$ more profit seems pathetic to companies that raised 100m$. Thing is; that is real money and I definitely rather have dinner with my colleagues than send it to Amazon.
faeriechangling 4y ago
Datacenter skills have NEVER been in more demand, datacenter is BOOMING and even the people who work in the AWS datacenter aren’t doomed to work there for eternity their skills forever captured by big tech. I honestly don’t know what you’re talking about, the skills are out there, they just don’t come cheap. Many businesses at a scale to run their own datacenters are also at a scale to have a relationship with local schools and managers will literally build pipeline of employees and maintain this pipeline for decades. The only staffing issue is these datacenter workers are just making so much money and getting so much work can you afford to hire them during this boom? Being able to avoid most of the costs and training and staffing you’re talking about has been a thing for a LONG time before the rise of cloud. There’s long been plenty of companies that will run the datacenter infrastructure for you or ever servers and let you hash out the details. People still made their own datacenters and continue to do so to this day. That is not why cloud has been so successful, the biggest thing cloud did to be successful was pioneer a new pay per use business model and the tools to enable it. Later on they created a bunch of new business innovations like EverythingAAS. If those innovations don’t come into play, AWS isn’t more attractive than Hetzner. I’ve built and decommissioned smaller datacenters for different businesses a bit under 1MW. I’ve seen datacenters make no sense, I’ve seen datacenters make complete sense.
- 988747 4y ago
  >> the people who work in the AWS datacenter aren’t doomed to work there for eternity their skills forever captured by big tech. I think it's kind of "golden handcuffs" type of situation - if you want these people to come and work for you you have to compete on salary with Amazon, and Bezos has much deeper pockets than you.
otabdeveloper4 4y ago
You don't need to build your own datacenter. Rent a server or a colocation rack in an existing datacenter. The result will still be orders of magnitude cheaper than AWS.
- xfitm3 4y ago
  Building your own has more margin, colos are expensive and they force you to buy things like blended bandwidth to increase their margins. Running your own dark fiber ring with dwdm and peering at exchanges is way better when your product is data.
goodpoint 4y ago
> I amortized servers with an average server lifetime of 3 years That's a mistake.
- eb0la 4y ago
  Why? after amortization they have 0 value in you books, which means your assets don't sink your balance sheet.
  - goodpoint 4y ago
    To calculate a meaningful yearly TCO you have to consider the real useful life of servers which is far longer than 3 years.
  - fanf2 4y ago
    The lifetime of a server is at least 5 years, which is the typical warranty length for server kit.
- xfitm3 4y ago
  We used SuperMicro servers and their TCO increased substantially after an average of 36 months. Keep in mind there are environmental issues such as CRAC failures, power stability, back EMF, etc which all harm components. It's very difficult to quantify the server lifetime, you need to be conservative in financial planning and that ended up with a median lifetime of 36MO. You can get deeper into the weeds and look at Zinc Whiskers and such but that's a niche you just have to generalize.
pclmulqdq 4y ago
3 years is a very short depreciation timeline for computers and associated equipment. That's how you ended up finding clouds being cheaper. Many companies who buy their infrastructure learn that they can run servers for 10+ years if they are properly cooled and maintained. Parts that wear out (fans and disks) are cheaply replaceable.

by xfitm3 4y ago

tluyben2 4y ago

It really depends on your demands; a lot of companies are running 1000s of trivial web applications and that’s it. Those can run on 10 year old servers which cost nothing at all anymore and never break; I have a few racks of them (HP dl360). We have not had issues in our colo for those 10 years; I didn’t even replace a drive and these are mostly spinning. Sure cloud is easier and if you need things that aws offers then go for it; we run well on a few $100/mo. Our most sold/popular product costs (I did this calc because of another ‘cloud is great’ HN post a few months back) less than $1/mo to host and maintain and has 50k+ active users. I like and use aws but it makes no sense in many cases; I would rob myself running most of our products on it because they don’t require remotely anything that it offers. I think the ‘make a unicorn’ mindset messed up people’s bottom line understanding; making 10k-100k$ more profit seems pathetic to companies that raised 100m$. Thing is; that is real money and I definitely rather have dinner with my colleagues than send it to Amazon.

faeriechangling 4y ago

Datacenter skills have NEVER been in more demand, datacenter is BOOMING and even the people who work in the AWS datacenter aren’t doomed to work there for eternity their skills forever captured by big tech. I honestly don’t know what you’re talking about, the skills are out there, they just don’t come cheap. Many businesses at a scale to run their own datacenters are also at a scale to have a relationship with local schools and managers will literally build pipeline of employees and maintain this pipeline for decades. The only staffing issue is these datacenter workers are just making so much money and getting so much work can you afford to hire them during this boom? Being able to avoid most of the costs and training and staffing you’re talking about has been a thing for a LONG time before the rise of cloud. There’s long been plenty of companies that will run the datacenter infrastructure for you or ever servers and let you hash out the details. People still made their own datacenters and continue to do so to this day. That is not why cloud has been so successful, the biggest thing cloud did to be successful was pioneer a new pay per use business model and the tools to enable it. Later on they created a bunch of new business innovations like EverythingAAS. If those innovations don’t come into play, AWS isn’t more attractive than Hetzner. I’ve built and decommissioned smaller datacenters for different businesses a bit under 1MW. I’ve seen datacenters make no sense, I’ve seen datacenters make complete sense.

988747 4y ago
>> the people who work in the AWS datacenter aren’t doomed to work there for eternity their skills forever captured by big tech. I think it's kind of "golden handcuffs" type of situation - if you want these people to come and work for you you have to compete on salary with Amazon, and Bezos has much deeper pockets than you.

otabdeveloper4 4y ago

You don't need to build your own datacenter. Rent a server or a colocation rack in an existing datacenter. The result will still be orders of magnitude cheaper than AWS.

xfitm3 4y ago
Building your own has more margin, colos are expensive and they force you to buy things like blended bandwidth to increase their margins. Running your own dark fiber ring with dwdm and peering at exchanges is way better when your product is data.

goodpoint 4y ago

> I amortized servers with an average server lifetime of 3 years That's a mistake.

eb0la 4y ago
Why? after amortization they have 0 value in you books, which means your assets don't sink your balance sheet.
- goodpoint 4y ago
  To calculate a meaningful yearly TCO you have to consider the real useful life of servers which is far longer than 3 years.
- fanf2 4y ago
  The lifetime of a server is at least 5 years, which is the typical warranty length for server kit.
xfitm3 4y ago
We used SuperMicro servers and their TCO increased substantially after an average of 36 months. Keep in mind there are environmental issues such as CRAC failures, power stability, back EMF, etc which all harm components. It's very difficult to quantify the server lifetime, you need to be conservative in financial planning and that ended up with a median lifetime of 36MO. You can get deeper into the weeds and look at Zinc Whiskers and such but that's a niche you just have to generalize.

pclmulqdq 4y ago

3 years is a very short depreciation timeline for computers and associated equipment. That's how you ended up finding clouds being cheaper. Many companies who buy their infrastructure learn that they can run servers for 10+ years if they are properly cooled and maintained. Parts that wear out (fans and disks) are cheaply replaceable.