4 ms·

This is one of the reasons I groan every time there's a new selfhosting thing that uses Docker. It's easy to start because all the wiring is there and everyone

This is one of the reasons I groan every time there's a new selfhosting thing that uses Docker. It's easy to start because all the wiring is there and everyone already makes their apps available with a Dockerfile, but it's not a remotely good solution for selfhosting and people should stop trying to use it that way.

apitman 3y ago
Unfortunately Sandstorm's approach, ie rewrite all the apps to fit a specific ecosystem, hasn't taken off either. I think docker with a little extra tooling for managing updates and backups is likely to be the solution eventually.
- ocdtrekkie 3y ago
  The problem is the Docker solution will never ever work and yet people are wasting thousands of person-hours trying, over and over in fact in dozens of redundant non-solutions. Sure Sandstorm or something like it will take a lot more effort, but at least it's moving in the correct direction. Just because Docker selfhosting has "taken off" doesn't mean it isn't still a dead end. EDIT: Also, to be clear, Docker is also a "specific ecosystem". It's very popular, it doesn't make it any more general than anything else.
  - EraYaN 3y ago
    Why would backups and restores never work though? The only thing you need to do is save your volumes/mounts essentially? The rest is a bunch of yaml in some docker-compose file for example. How is that "will never ever work"?
    - ocdtrekkie 3y ago
      The issue is that there is a lot more to a home server than updates and backups, and even doing those correctly automatically is... pretty hard if you don't have very strong opinions about how apps need to support them. So to give you an idea, Sandstorm requires: An app always must recover from forced termination (the only way it shuts apps down!) and that a given app carry all of the code it needs to update its data from any previous version. If you don't do these things, your homelabber will inevitably be fixing stuff themselves in various apps because their server lost power or the app developer demanded some bespoke intermediate inane update steps, or you missed a few versions while your server was off. What percentage of apps made available via Docker meets those requirements? Probably a lot less than you'd think. (Most of them that do easily work on Sandstorm!) But if you want to make selfhosting work for non-sysadmins (or sysadmins who don't want to do a second job when they get home from their main job), you have to do these things. Another thing is that you basically have to have security right because your users and even your server admin probably aren't experts, and definitely aren't paying for an enterprise-class firewall. So a big part of Sandstorm's opinionated app design is around making it fundamentally impossible for app vulnerabilities to exist. Things like proxying everything through the platform, isolating individual documents into their own app instances, handling authentication and authorization, etc. all become core to making apps secure enough to not have to worry about them that much. This is especially important because a lot of selfhosted apps get abandoned and unmaintained, but selfhosters still want to use them. Of course, all these things fundamentally require changes to the app your random Docker container shipper can't insist on, and that's before we get even into performance concerns, which are key since selfhosters tend to use old or cheap hardware.
BrandoElFollito 3y ago
In only look at selfhosting things that provide a docker image. This is how it should be: the internals are hidden from your and you just care about the data and the configuration. If you want to use a monolith you will have conflicts sooner or later. Of you want to use multiple VMs you need to orchestrate them. If you know how to do the above you are good to go to learn docker which is ultimately much simpler.
wmf 3y ago
A more detailed explanation from ten years ago: https://sandstorm.io/news/2014-08-19-why-not-run-docker-apps https://sandstorm.io/news/2014-08-19-why-not-run-docker-apps
doubled112 3y ago
I think it's the easy start that's the problem. Docker and Docker Compose is pretty good for self-hosting, except you have to understand enough to realize that it's not as simple as it sounds. You can easily lose data you didn't know you had if you're not paying attention. "I'll just delete the container and restart" sounds like a great solution, but you just blew away your photos or whatever because you didn't know it created some volume for you. I have a btrfs volume as the root of all the apps. Each application is in an individual subvolume so they can be snapshotted at upgrade time. The docker-compose.yml file points each app's volume to a relative directory inside that subvolume. This way I can move them around individually, and all their data is right there. I can back them up pretty easily too. Works for me and my use case, but you could never expect it to be turn key, and you couldn't hand it off to somebody non-technical.

apitman 3y ago

Unfortunately Sandstorm's approach, ie rewrite all the apps to fit a specific ecosystem, hasn't taken off either. I think docker with a little extra tooling for managing updates and backups is likely to be the solution eventually.

ocdtrekkie 3y ago
The problem is the Docker solution will never ever work and yet people are wasting thousands of person-hours trying, over and over in fact in dozens of redundant non-solutions. Sure Sandstorm or something like it will take a lot more effort, but at least it's moving in the correct direction. Just because Docker selfhosting has "taken off" doesn't mean it isn't still a dead end. EDIT: Also, to be clear, Docker is also a "specific ecosystem". It's very popular, it doesn't make it any more general than anything else.
- EraYaN 3y ago
  Why would backups and restores never work though? The only thing you need to do is save your volumes/mounts essentially? The rest is a bunch of yaml in some docker-compose file for example. How is that "will never ever work"?
  - ocdtrekkie 3y ago
    The issue is that there is a lot more to a home server than updates and backups, and even doing those correctly automatically is... pretty hard if you don't have very strong opinions about how apps need to support them. So to give you an idea, Sandstorm requires: An app always must recover from forced termination (the only way it shuts apps down!) and that a given app carry all of the code it needs to update its data from any previous version. If you don't do these things, your homelabber will inevitably be fixing stuff themselves in various apps because their server lost power or the app developer demanded some bespoke intermediate inane update steps, or you missed a few versions while your server was off. What percentage of apps made available via Docker meets those requirements? Probably a lot less than you'd think. (Most of them that do easily work on Sandstorm!) But if you want to make selfhosting work for non-sysadmins (or sysadmins who don't want to do a second job when they get home from their main job), you have to do these things. Another thing is that you basically have to have security right because your users and even your server admin probably aren't experts, and definitely aren't paying for an enterprise-class firewall. So a big part of Sandstorm's opinionated app design is around making it fundamentally impossible for app vulnerabilities to exist. Things like proxying everything through the platform, isolating individual documents into their own app instances, handling authentication and authorization, etc. all become core to making apps secure enough to not have to worry about them that much. This is especially important because a lot of selfhosted apps get abandoned and unmaintained, but selfhosters still want to use them. Of course, all these things fundamentally require changes to the app your random Docker container shipper can't insist on, and that's before we get even into performance concerns, which are key since selfhosters tend to use old or cheap hardware.

BrandoElFollito 3y ago

In only look at selfhosting things that provide a docker image. This is how it should be: the internals are hidden from your and you just care about the data and the configuration. If you want to use a monolith you will have conflicts sooner or later. Of you want to use multiple VMs you need to orchestrate them. If you know how to do the above you are good to go to learn docker which is ultimately much simpler.

wmf 3y ago

A more detailed explanation from ten years ago: https://sandstorm.io/news/2014-08-19-why-not-run-docker-apps https://sandstorm.io/news/2014-08-19-why-not-run-docker-apps

doubled112 3y ago

I think it's the easy start that's the problem. Docker and Docker Compose is pretty good for self-hosting, except you have to understand enough to realize that it's not as simple as it sounds. You can easily lose data you didn't know you had if you're not paying attention. "I'll just delete the container and restart" sounds like a great solution, but you just blew away your photos or whatever because you didn't know it created some volume for you. I have a btrfs volume as the root of all the apps. Each application is in an individual subvolume so they can be snapshotted at upgrade time. The docker-compose.yml file points each app's volume to a relative directory inside that subvolume. This way I can move them around individually, and all their data is right there. I can back them up pretty easily too. Works for me and my use case, but you could never expect it to be turn key, and you couldn't hand it off to somebody non-technical.