3 ms·

Self-hosting is a big operations problem, with few tools to automate it. Long ago, I had an associate tell me that he was having some success with setting up W

Self-hosting is a big operations problem, with few tools to automate it. Long ago, I had an associate tell me that he was having some success with setting up Wordpress sites for local political organizations. I said to him: "Oh, that's really neat! What are you doing to ensure that the sites stay up to date with security patches?" His response was completely unrelated to my question, which I figured was my answer and was why there are so many hacked sites out there. Anything I deploy needs to have an upgrade plan. Ideally, something that provides a package (either on distro or a repo the package provides), so "apt update" will resolve it. Docker can be a good way as well, Sentry does a pretty good job at this.

x0x0 4y ago
The entire discussion on the link obscures the fact that saas companies are providing a real service. Even if you don't want the product to be updated, staying abreast of security patches, external api changes, OS changes, client changes, browser changes, etc is real work. Self hosting requires the person hosting to do all the ktlo work.
KronisLV 4y ago
> Anything I deploy needs to have an upgrade plan. Ideally, something that provides a package (either on distro or a repo the package provides), so "apt update" will resolve it. Docker can be a good way as well, Sentry does a pretty good job at this. I wrote an article called "Never update anything": https://blog.kronis.dev/articles/never-update-anything https://blog.kronis.dev/articles/never-update-anything which in truth argued that while updates are necessary, they're also going to break things... a lot. And there isn't always going to be an upgrade path either (e.g. using AngularJS or Clusterpoint). In my experience, even containers break things surprisingly often: everything from GitLab, Nextcloud, OpenProject to even things breaking in regular server updates, like a Debian update breaking GRUB or another Debian install automatically launching exim4 which prevented my own mail server from working. Perhaps that's because of how we build and package software, because of the fact that we don't separate the runtime from the data enough (e.g. persistent directories) or that we make too many assumptions about the environments... Regardless, I can understand why some don't even update working but insecure software: because of the risk to turn it into secure but not working software.
- XCSme 4y ago
  I agree, I ran many WordPress sites, UXWizz dashboards or even my own tools written in Node.js, and they never ever broke by themselves in 10+ years. The only time there is an issue if I decide to update/change anything. In general, software that has once worked will always work. If I open my PSP (Playstation Portable) from 2005 it will still work as well as in the first day, all the games work the same and the boot time and interface are faster than most consoles nowadays. Why does it still work? Because it once worked and nothing changed.
- dicknuckle 4y ago
  Anecdote, I also had a container app fail, from a trusted provider that I assumed weighsy have plenty of testing in place, but maybe it was too synthetic. Sonarr from LinuxServer.io briefly changed to a dev build and boned the DB before switching back to a stable build.
  - KronisLV 4y ago
    You know, I thought that I'd actually provide a bit more information about my experiences as well, because while these are anecdotes, they are still something that happened and shouldn't be entirely ignored. These things do happen, sometimes because of the architecture and what's inside of the container in question. For example, that was largely the case with the GitLab Omnibus containers, because of them containing lots of dependencies, or the same with OpenProject. GitLab: https://blog.kronis.dev/everything%20is%20broken/gitlab-updates-are-broken https://blog.kronis.dev/everything%20is%20broken/gitlab-upda... OpenProject: https://blog.kronis.dev/everything%20is%20broken/openproject-updates-are-broken https://blog.kronis.dev/everything%20is%20broken/openproject... (I still think that both are good software, but not without their pain points, like updates in this case) Then again, even something like Nextcloud or comparatively simpler systems like Grav still run into issues with upgrades, sometimes because going across major versions is too much, other times because the containers have vendor dependencies in volumes/bind mounts, which cause a plethora of issues. Nextcloud: https://blog.kronis.dev/everything%20is%20broken/nextcloud-is-broken https://blog.kronis.dev/everything%20is%20broken/nextcloud-i... Grav: https://blog.kronis.dev/everything%20is%20broken/grav-is-broken https://blog.kronis.dev/everything%20is%20broken/grav-is-bro... But it's a little bit odd when ever the operating systems we use have similar issues. Like that mention of a Debian update breaking GRUB prevented the server from booting altogether and that other time the exim4 package getting launched prevented my mail server from starting because of the port being bound. Debian breaking GRUB: https://blog.kronis.dev/everything%20is%20broken/debian-and-grub-are-broken https://blog.kronis.dev/everything%20is%20broken/debian-and-... Debian launching exim4: https://blog.kronis.dev/everything%20is%20broken/debian-updates-are-broken https://blog.kronis.dev/everything%20is%20broken/debian-upda... Other times you get problems with upgrading across libraries, or language runtime versions, which I didn't mention in the original post, but which is no less of an issue. It's even harder than just regular package or software updates, because sometimes it might necessitate a major rewrite. Example of Java 8 to Java 11: https://blog.kronis.dev/everything%20is%20broken/upgrading-java-8-to-11-is-broken https://blog.kronis.dev/everything%20is%20broken/upgrading-j... That said, I'm not sure how good the idea of documenting every software breakage out there is, because the folder with screenshots and information about things that went wrong just seems to be getting longer and longer for me. But hey, maybe it will serve as an insight into what things were around 2020s some day, when hopefully we've figured out how to upgrade software more safely.
chillfox 4y ago
I have run a lot of WordPress sites, it's easy for a skilled admin to run it securely for a long time with barely any effort and unfortunately it's also easy for a user to make it insecure.
- linsomniac 4y ago
  Sure, and that was what I floated with that guy, but the impression I got from him is that he installed it and moved on. So, as you say, a skilled admin can do it, given some ongoing attention. Which is exactly what I'm talking about WRT operations.
  - XCSme 4y ago
    I consider myself a "skilled admin", yet I don't feel like I have to do much to keep a WordPress instance running. Now it even updates itself. I often just install WP, a theme, and maybe one or two plugins, and then it runs without issues forever.

x0x0 4y ago

The entire discussion on the link obscures the fact that saas companies are providing a real service. Even if you don't want the product to be updated, staying abreast of security patches, external api changes, OS changes, client changes, browser changes, etc is real work. Self hosting requires the person hosting to do all the ktlo work.

KronisLV 4y ago

> Anything I deploy needs to have an upgrade plan. Ideally, something that provides a package (either on distro or a repo the package provides), so "apt update" will resolve it. Docker can be a good way as well, Sentry does a pretty good job at this. I wrote an article called "Never update anything": https://blog.kronis.dev/articles/never-update-anything https://blog.kronis.dev/articles/never-update-anything which in truth argued that while updates are necessary, they're also going to break things... a lot. And there isn't always going to be an upgrade path either (e.g. using AngularJS or Clusterpoint). In my experience, even containers break things surprisingly often: everything from GitLab, Nextcloud, OpenProject to even things breaking in regular server updates, like a Debian update breaking GRUB or another Debian install automatically launching exim4 which prevented my own mail server from working. Perhaps that's because of how we build and package software, because of the fact that we don't separate the runtime from the data enough (e.g. persistent directories) or that we make too many assumptions about the environments... Regardless, I can understand why some don't even update working but insecure software: because of the risk to turn it into secure but not working software.

XCSme 4y ago
I agree, I ran many WordPress sites, UXWizz dashboards or even my own tools written in Node.js, and they never ever broke by themselves in 10+ years. The only time there is an issue if I decide to update/change anything. In general, software that has once worked will always work. If I open my PSP (Playstation Portable) from 2005 it will still work as well as in the first day, all the games work the same and the boot time and interface are faster than most consoles nowadays. Why does it still work? Because it once worked and nothing changed.
dicknuckle 4y ago
Anecdote, I also had a container app fail, from a trusted provider that I assumed weighsy have plenty of testing in place, but maybe it was too synthetic. Sonarr from LinuxServer.io briefly changed to a dev build and boned the DB before switching back to a stable build.
- KronisLV 4y ago
  You know, I thought that I'd actually provide a bit more information about my experiences as well, because while these are anecdotes, they are still something that happened and shouldn't be entirely ignored. These things do happen, sometimes because of the architecture and what's inside of the container in question. For example, that was largely the case with the GitLab Omnibus containers, because of them containing lots of dependencies, or the same with OpenProject. GitLab: https://blog.kronis.dev/everything%20is%20broken/gitlab-updates-are-broken https://blog.kronis.dev/everything%20is%20broken/gitlab-upda... OpenProject: https://blog.kronis.dev/everything%20is%20broken/openproject-updates-are-broken https://blog.kronis.dev/everything%20is%20broken/openproject... (I still think that both are good software, but not without their pain points, like updates in this case) Then again, even something like Nextcloud or comparatively simpler systems like Grav still run into issues with upgrades, sometimes because going across major versions is too much, other times because the containers have vendor dependencies in volumes/bind mounts, which cause a plethora of issues. Nextcloud: https://blog.kronis.dev/everything%20is%20broken/nextcloud-is-broken https://blog.kronis.dev/everything%20is%20broken/nextcloud-i... Grav: https://blog.kronis.dev/everything%20is%20broken/grav-is-broken https://blog.kronis.dev/everything%20is%20broken/grav-is-bro... But it's a little bit odd when ever the operating systems we use have similar issues. Like that mention of a Debian update breaking GRUB prevented the server from booting altogether and that other time the exim4 package getting launched prevented my mail server from starting because of the port being bound. Debian breaking GRUB: https://blog.kronis.dev/everything%20is%20broken/debian-and-grub-are-broken https://blog.kronis.dev/everything%20is%20broken/debian-and-... Debian launching exim4: https://blog.kronis.dev/everything%20is%20broken/debian-updates-are-broken https://blog.kronis.dev/everything%20is%20broken/debian-upda... Other times you get problems with upgrading across libraries, or language runtime versions, which I didn't mention in the original post, but which is no less of an issue. It's even harder than just regular package or software updates, because sometimes it might necessitate a major rewrite. Example of Java 8 to Java 11: https://blog.kronis.dev/everything%20is%20broken/upgrading-java-8-to-11-is-broken https://blog.kronis.dev/everything%20is%20broken/upgrading-j... That said, I'm not sure how good the idea of documenting every software breakage out there is, because the folder with screenshots and information about things that went wrong just seems to be getting longer and longer for me. But hey, maybe it will serve as an insight into what things were around 2020s some day, when hopefully we've figured out how to upgrade software more safely.

chillfox 4y ago

I have run a lot of WordPress sites, it's easy for a skilled admin to run it securely for a long time with barely any effort and unfortunately it's also easy for a user to make it insecure.

linsomniac 4y ago
Sure, and that was what I floated with that guy, but the impression I got from him is that he installed it and moved on. So, as you say, a skilled admin can do it, given some ongoing attention. Which is exactly what I'm talking about WRT operations.
- XCSme 4y ago
  I consider myself a "skilled admin", yet I don't feel like I have to do much to keep a WordPress instance running. Now it even updates itself. I often just install WP, a theme, and maybe one or two plugins, and then it runs without issues forever.