4 ms·

> We have to deal with them if something weird happens Check back in three years once you've had to deal with OS upgrades, power outages, running out of disk s

> We have to deal with them if something weird happens Check back in three years once you've had to deal with OS upgrades, power outages, running out of disk space, and one of them failing.

abuani 2y ago
Three years sounds like a reasonably good return on investment. At that point the teams either big enough to justify paying $$ for a hosted solution, or still cheaper to buy dedicated hardware.
- delfinom 2y ago
  You can also just rent dedicated servers from hetzner or ovh for far cheaper than AWS instances with more horsepower. More expensive than a dell mini in a closet. But you get datacenter Internet and power backup.
- re-thc 2y ago
  > justify paying $$ for a hosted solution, or still cheaper to buy dedicated hardware. Always cheaper if you have at least some demand. For the lazy fix even just get a few mac mini and it'd work fine.
delfinom 2y ago
It's a CI runner my dude. You just obliterate and reinstall the base system in what 10 minutes?
cr125rider 2y ago
Are any of those problems challenging to deal with? Even if it’s a few expensive man hours the ROI is still quite short.
barkingcat 2y ago
if the break even was 4-5 months, it means compared to going to cloud vendors, in 36 months, the self hosted solution would have saved 7.2 times the cost!! Even if within 3 years there needs to be OS upgrades, you can just e-cycle the old machines and buy brand new minipc's (with added benefit of increased performance) every time you need to upgrade, and you'd still be ahead!!
jjice 2y ago
We're about eight months in at the moment, so here are some current stats: - No OS upgrades (running Ubuntu 22 LTS), but regular update - We've had a power outage, but not during the day, so when they rebooted, they went right back to action. Same issue with a network outage in the office. - We have hit disk space issues due to not clearing all the data we needed to. A quick SSH and an rm or two did the job, plus another 30 minutes to an hour to update a cron to remove that portion of the cache I see what you mean for sure, and there will undoubtedly be more things that come up at some point, but with the savings, it would take quite a few engineering hours per month to lose the savings at this point. And I won't lie, when something goes wrong, it's a nice change of pace to deal with a server instead of code once and a while. Can't actually use that for calculations, but I don't hate when it happens so far.
rcleveng 2y ago
When a better ROI of 4-5 months, you could just buy a new one when any of those things happen, and still be net positive.
bravetraveler 2y ago
> Check back in three years once you've had to deal with... This being routine engineering, I can tell you right now how it will go. Let's start! > OS upgrades Distinction needed: update/upgrade. Entirely automatable, some may say table stakes. Decide to update in this job or another. Upgrades are a common problem for either hosting world. Can my software work in this new environment? Hence, CI. > power outages Set the firmware to return to last power state. Also buy a couple UPS' and a PDU. Fixed. A host and networks between going away isn't so easy. > Running out of disk space How is this unique for them to solve? Hosted workers/cloud servers have finite disks too. Clean your workspaces in your pipelines. That's all disposing of the hosted one does. > and one of them failing. Someone else in the thread said it best, these are CI runners man. Capacity is down slightly. Oh no. Several cheap copies were bought to address this. The marketing was so successful that we have to debate that yes, there is value in owning things you need/use.

> We have to deal with them if something weird happens Check back in three years once you've had to deal with OS upgrades, power outages, running out of disk space, and one of them failing.

abuani 2y ago

Three years sounds like a reasonably good return on investment. At that point the teams either big enough to justify paying $$ for a hosted solution, or still cheaper to buy dedicated hardware.

delfinom 2y ago
You can also just rent dedicated servers from hetzner or ovh for far cheaper than AWS instances with more horsepower. More expensive than a dell mini in a closet. But you get datacenter Internet and power backup.
re-thc 2y ago
> justify paying $$ for a hosted solution, or still cheaper to buy dedicated hardware. Always cheaper if you have at least some demand. For the lazy fix even just get a few mac mini and it'd work fine.

delfinom 2y ago

It's a CI runner my dude. You just obliterate and reinstall the base system in what 10 minutes?

cr125rider 2y ago

Are any of those problems challenging to deal with? Even if it’s a few expensive man hours the ROI is still quite short.

barkingcat 2y ago

if the break even was 4-5 months, it means compared to going to cloud vendors, in 36 months, the self hosted solution would have saved 7.2 times the cost!! Even if within 3 years there needs to be OS upgrades, you can just e-cycle the old machines and buy brand new minipc's (with added benefit of increased performance) every time you need to upgrade, and you'd still be ahead!!

jjice 2y ago

We're about eight months in at the moment, so here are some current stats: - No OS upgrades (running Ubuntu 22 LTS), but regular update - We've had a power outage, but not during the day, so when they rebooted, they went right back to action. Same issue with a network outage in the office. - We have hit disk space issues due to not clearing all the data we needed to. A quick SSH and an rm or two did the job, plus another 30 minutes to an hour to update a cron to remove that portion of the cache I see what you mean for sure, and there will undoubtedly be more things that come up at some point, but with the savings, it would take quite a few engineering hours per month to lose the savings at this point. And I won't lie, when something goes wrong, it's a nice change of pace to deal with a server instead of code once and a while. Can't actually use that for calculations, but I don't hate when it happens so far.

rcleveng 2y ago

When a better ROI of 4-5 months, you could just buy a new one when any of those things happen, and still be net positive.

bravetraveler 2y ago

> Check back in three years once you've had to deal with... This being routine engineering, I can tell you right now how it will go. Let's start! > OS upgrades Distinction needed: update/upgrade. Entirely automatable, some may say table stakes. Decide to update in this job or another. Upgrades are a common problem for either hosting world. Can my software work in this new environment? Hence, CI. > power outages Set the firmware to return to last power state. Also buy a couple UPS' and a PDU. Fixed. A host and networks between going away isn't so easy. > Running out of disk space How is this unique for them to solve? Hosted workers/cloud servers have finite disks too. Clean your workspaces in your pipelines. That's all disposing of the hosted one does. > and one of them failing. Someone else in the thread said it best, these are CI runners man. Capacity is down slightly. Oh no. Several cheap copies were bought to address this. The marketing was so successful that we have to debate that yes, there is value in owning things you need/use.