3 ms·

I wonder if it has to do with mismatches between -as-cattle and -as-pets philosophies and usecases. From the post, it looks like they were using k8s. It's hard

I wonder if it has to do with mismatches between -as-cattle and -as-pets philosophies and usecases. From the post, it looks like they were using k8s. It's hard to tell given the, you know, total data loss, but it looks like the instance had at max triple-digit users. In my experience running small community stuff like this, the -as-cattle tooling, even though I'm familiar with it from work, is way overkill and introduces far more brittleness than it prevents. My go-to for stuff like this is -as-pets: gimme a box, a daemon, some rc scripts and maybe a backup client. Admin complexity and burnout are huge problems for volunteer sysadmins, and IMO the burden of -as-cattle tooling unnecessarily exacerbates both. I've had an irc daemon running on the same metal for almost 15 years now, uninterrupted besides occasional OS upgrades and patches. There is basically nothing else happening on the host. It is dumb simple and just works. Some of that may be on devs, actually: distribution-as-docker-container at the expense of real packaging seems to be on the rise.

noobermin 3y ago
This sounds like a good explanation. As an outsider may be complexity seems unnecessary but it's needed for scale (an analogy is what is needed in a kitchen for a spaghetti meal to what is needed in a resturaunt to what is needed in a canned spaghetti factory), but problems can arise when you try to shoehorn methodologies and tools for scaled production to smaller systems I guess.
lamontcg 3y ago
> Some of that may be on devs, actually: distribution-as-docker-container at the expense of real packaging seems to be on the rise. You could just run docker or podman or whatever alone on a VM and start/stop containers entirely manually.
tetha 3y ago
At work, we're running a hybrid container/vm setup. We host the applications from our dev-teams as containers, since there is a lot of them and this allows them to do most of the work to integrate new apps with the stack. However, the actual data stores like postgres, glusterfs and such are simple VMs for a simple reason: It has less failure modes and many of the failure modes on a VM are less subtle and strange to deal with. And very few failure modes on a VM go straight to irrecoverable dataloss. In fact, the company I work at has deployed production like that for many years when our needs were simpler with just 1-2 applications. 1-4 application VMs provisioned by a config management, maybe a loadbalancer, maybe 1-2 DB VMs behind it. Worked like a charm until the new parent company started throwing loads of complexity at it.
sofixa 3y ago
> Some of that may be on devs, actually: distribution-as-docker-container at the expense of real packaging seems to be on the rise. Because it eliminates a whole host of "works on my computer" "which version of X dependency library do you have installed" problems nobody wants to deal with.

noobermin 3y ago

This sounds like a good explanation. As an outsider may be complexity seems unnecessary but it's needed for scale (an analogy is what is needed in a kitchen for a spaghetti meal to what is needed in a resturaunt to what is needed in a canned spaghetti factory), but problems can arise when you try to shoehorn methodologies and tools for scaled production to smaller systems I guess.

lamontcg 3y ago

> Some of that may be on devs, actually: distribution-as-docker-container at the expense of real packaging seems to be on the rise. You could just run docker or podman or whatever alone on a VM and start/stop containers entirely manually.

tetha 3y ago

At work, we're running a hybrid container/vm setup. We host the applications from our dev-teams as containers, since there is a lot of them and this allows them to do most of the work to integrate new apps with the stack. However, the actual data stores like postgres, glusterfs and such are simple VMs for a simple reason: It has less failure modes and many of the failure modes on a VM are less subtle and strange to deal with. And very few failure modes on a VM go straight to irrecoverable dataloss. In fact, the company I work at has deployed production like that for many years when our needs were simpler with just 1-2 applications. 1-4 application VMs provisioned by a config management, maybe a loadbalancer, maybe 1-2 DB VMs behind it. Worked like a charm until the new parent company started throwing loads of complexity at it.

sofixa 3y ago

> Some of that may be on devs, actually: distribution-as-docker-container at the expense of real packaging seems to be on the rise. Because it eliminates a whole host of "works on my computer" "which version of X dependency library do you have installed" problems nobody wants to deal with.