3 ms·

Yes, exactly this. At what point do you have the time/money/confidence to invest goodness knows how much in a data centre with space to grow, to purchase an en

Yes, exactly this. At what point do you have the time/money/confidence to invest goodness knows how much in a data centre with space to grow, to purchase an enormous amount of capital to have it all installed etc. the building alone could eat that first years saving easily. How many people are now needed to fault-find bad hardware/software/networks, to be on call for any problems? How many calls out to the Electrician to fix some power issue? How much to setup and run a large air-con system for the data centre. Maybe not much in the US where aircon is common but much more expensive in Europe. The fact they could afford to do this over such a short time period speaks to having a decent amount of cash on-hand.

hamandcheese 4y ago
> At what point do you have the time/money/confidence to invest goodness knows how much in a data centre with space to grow, to purchase an enormous amount of capital to have it all installed etc. the building alone could eat that first years saving easily. Co-locating has no capital investment other than hardware, and is pretty cheap. A 40U rack of compute charged as equivalent ec2 instances has a retail price easily of hundreds of thousands, if not a million+ USD per year. Suppose each U has a $10k capital cost to make the numbers round, that is $400k in capital. All this to say is that I don’t think capital is as big a factor as you might think.
- boltzmann-brain 4y ago
  In my experience doing detailed projections of exactly something like this - with racks full of GPU compute power - the infrastructure has paid itself after one quarter, maybe two, depending on what volume you're at. There very rarely is a reason to use GPU compute on the cloud - and advantages start as quickly as with just one single GPU.
- Corrado 4y ago
  I feel that a lot of posts like this might be under-representing the true costs of running your own hardware. I was only tangentially associated with a large-ish operation and I can tell you that there are loads of things that take a lot of time but are often over looked. Things like detecting and replacing bad hardware. HDs don't last forever and when they go bad it's not fun; especially if you have to source some specific model for your 7 year old server platform. Understanding your licensing and warranties is another huge cost that people don't take into account. We used to spend hours and hours figuring out if we could replace systems and what it would cost us. Finally, you have to dispose of all that hardware when it gets too old or out of warranty. If you've never had to do that you probably have no idea how hard it is to do it correctly and so that it satisfies your SOC2 auditor. All of these things (plus more) really add up. It's not just purchasing the components and installing them in racks. The management is probably even more expensive than the hardware. And all of these problems go away with a cloud provider.
  - Melatonic 4y ago
    Your VAR should be doing a lot of that work for you - sure you need to understand licensing but if you have a good rep they should be providing you with up to date info on all of this. HDD replacement is trivial and some COLOs can even do it for you - and any good vendor will have a thing setup where your appliance automatically notifies them and a new drive is overnighted to your COLO. Sure - maybe a few times a year you drive out to swap a drive. If you do not have good monitoring setup then that is entirely fixable - there are many stellar solutions these days out there. Hardware is easier than ever.
  - hamandcheese 4y ago
    Right, I agree that the cost might be underestimated, but I was specifically talking about the capital expenditure required to “eject” from the cloud, not the ongoing operating costs.
jjav 4y ago
> fix some power issue > large air-con system You wouldn't usually jump from AWS to buying up real estate to build your own physical data center. A sensible first step is to rent a rack at a colocation facility. They handle power, cooling, redundancy, physical access for you.
- carlivar 4y ago
  I wonder if there's a whole new generation of engineers that don't realize this? After all, how would they know about co-location if only ever using the cloud? We already interview ops and network slanted candidates that don't know what ARP is (we've now added this as an early quick screen).

hamandcheese 4y ago

> At what point do you have the time/money/confidence to invest goodness knows how much in a data centre with space to grow, to purchase an enormous amount of capital to have it all installed etc. the building alone could eat that first years saving easily. Co-locating has no capital investment other than hardware, and is pretty cheap. A 40U rack of compute charged as equivalent ec2 instances has a retail price easily of hundreds of thousands, if not a million+ USD per year. Suppose each U has a $10k capital cost to make the numbers round, that is $400k in capital. All this to say is that I don’t think capital is as big a factor as you might think.

boltzmann-brain 4y ago
In my experience doing detailed projections of exactly something like this - with racks full of GPU compute power - the infrastructure has paid itself after one quarter, maybe two, depending on what volume you're at. There very rarely is a reason to use GPU compute on the cloud - and advantages start as quickly as with just one single GPU.
Corrado 4y ago
I feel that a lot of posts like this might be under-representing the true costs of running your own hardware. I was only tangentially associated with a large-ish operation and I can tell you that there are loads of things that take a lot of time but are often over looked. Things like detecting and replacing bad hardware. HDs don't last forever and when they go bad it's not fun; especially if you have to source some specific model for your 7 year old server platform. Understanding your licensing and warranties is another huge cost that people don't take into account. We used to spend hours and hours figuring out if we could replace systems and what it would cost us. Finally, you have to dispose of all that hardware when it gets too old or out of warranty. If you've never had to do that you probably have no idea how hard it is to do it correctly and so that it satisfies your SOC2 auditor. All of these things (plus more) really add up. It's not just purchasing the components and installing them in racks. The management is probably even more expensive than the hardware. And all of these problems go away with a cloud provider.
- Melatonic 4y ago
  Your VAR should be doing a lot of that work for you - sure you need to understand licensing but if you have a good rep they should be providing you with up to date info on all of this. HDD replacement is trivial and some COLOs can even do it for you - and any good vendor will have a thing setup where your appliance automatically notifies them and a new drive is overnighted to your COLO. Sure - maybe a few times a year you drive out to swap a drive. If you do not have good monitoring setup then that is entirely fixable - there are many stellar solutions these days out there. Hardware is easier than ever.
- hamandcheese 4y ago
  Right, I agree that the cost might be underestimated, but I was specifically talking about the capital expenditure required to “eject” from the cloud, not the ongoing operating costs.

jjav 4y ago

> fix some power issue > large air-con system You wouldn't usually jump from AWS to buying up real estate to build your own physical data center. A sensible first step is to rent a rack at a colocation facility. They handle power, cooling, redundancy, physical access for you.

carlivar 4y ago
I wonder if there's a whole new generation of engineers that don't realize this? After all, how would they know about co-location if only ever using the cloud? We already interview ops and network slanted candidates that don't know what ARP is (we've now added this as an early quick screen).