3 ms·

I'm working on something similar, and the crux of that issue is configurability vs automation. I.e. it's very easy to make backups for a system that users can'

I'm working on something similar, and the crux of that issue is configurability vs automation. I.e. it's very easy to make backups for a system that users can't configure at all. You just ship the image with some rsync commands or something and done. Once you start letting people edit the config files, you get into a spot where now you basically need to be able to parse the config files to read file paths. That often means making version-specific parsers for configuration options that are introduced or removed in some versions, or have differing defaults (i.e. in 1.2 if they don't set "storage_path" it's at /x/, but in 1.3 if they don't set it, it defaults to /y/). That gets to be a lot of work. Then it gets even worse when the users can edit the Docker config for the images, because all bets are off at that point. The user could do all kinds of weird, fucky shit that infinitely loops naive backup scripts because of host volume mounts or have one of the mounts actually be part of NFS mount on the host with like 200ms of latency so backups just hang for forever and etc. It's just begging for an infinite series of bugs from people who did something weird with their config and ended up backing up their whole drive instead of the Docker folder, or removing their whole root drive because they mount their host FS root in the backups folder and it got deleted by the "old backup cleanup" script, or who knows what. At some point, it's easier to just make your own setup where you define the limitations of that setup than it is to use someone else's setup but have to find the limitations on your own.

apitman 3y ago
The one thing you have working on your favor is that the more likely someone is to want to customize, the more likely they are to understand they have to be responsible for backups and updates. Is your project public? I'm working on one as well and would love to see what you have cooking.
- everforward 3y ago
  It's not at the moment, but I wouldn't be opposed to chucking you some access if you wanted to look (or talk about joining, I'm solo at the moment). The rough gist is that it's for hosting dedicated servers for video games, with an emphasis on having an easier to use UI than most providers combined with a usable API/CLI for advanced users. Ie. Instead of editing config files as regular text files, I want to offer a view with drop downs for each config option, a description of what each value does. (with some hidden for orchestration reasons, like the port it listens on) Architecture-wise, it's a 3-tier app. HTTP API frontend talks to gRPC backend talks to Postgres. gRPC in that middle tier because I want to move the control plane for the agents into gRPC. Currently, the control plane consists of generating an Ansible inventory and executing it, and Ansible handles downloading the game if it doesn't exist, making dirs, SFTPing configs, install a Prometheus agent, etc. It's not anything I intend to hyperscale, but I've always been the dedicated server guy among my friends, so I figured I might as well make a little thing out of it. I'd be happy with a small ARR.
megous 3y ago
> Once you start letting people edit the config files, That way lies madness... What can be done instead is to provide your own unified (some schema validated) configuration options, and generate the app specific config files from your source of truth. Then you can know what the user can configure and how to back everything up (and how to do a lot of other things in automated fashion). And you also have a safe upgrade path if format of any underlaying config changes.
- everforward 3y ago
  You can, but I find it very tedious. O2M or M2M relationships become cumbersome. Not because they're hard to use, but just because there's so fucking many of them. I opted to store the app and version, and have a function that retrieves a config parser for it. I just didn't want to manage 300 tables just to get the schema to work. That's also partially because I'm using Ent in Go, so it generates a struct for each table. It'd make my auto-complete useless, and that's not a price I'm willing to pay on a personal project lol

apitman 3y ago

The one thing you have working on your favor is that the more likely someone is to want to customize, the more likely they are to understand they have to be responsible for backups and updates. Is your project public? I'm working on one as well and would love to see what you have cooking.

everforward 3y ago
It's not at the moment, but I wouldn't be opposed to chucking you some access if you wanted to look (or talk about joining, I'm solo at the moment). The rough gist is that it's for hosting dedicated servers for video games, with an emphasis on having an easier to use UI than most providers combined with a usable API/CLI for advanced users. Ie. Instead of editing config files as regular text files, I want to offer a view with drop downs for each config option, a description of what each value does. (with some hidden for orchestration reasons, like the port it listens on) Architecture-wise, it's a 3-tier app. HTTP API frontend talks to gRPC backend talks to Postgres. gRPC in that middle tier because I want to move the control plane for the agents into gRPC. Currently, the control plane consists of generating an Ansible inventory and executing it, and Ansible handles downloading the game if it doesn't exist, making dirs, SFTPing configs, install a Prometheus agent, etc. It's not anything I intend to hyperscale, but I've always been the dedicated server guy among my friends, so I figured I might as well make a little thing out of it. I'd be happy with a small ARR.

megous 3y ago

> Once you start letting people edit the config files, That way lies madness... What can be done instead is to provide your own unified (some schema validated) configuration options, and generate the app specific config files from your source of truth. Then you can know what the user can configure and how to back everything up (and how to do a lot of other things in automated fashion). And you also have a safe upgrade path if format of any underlaying config changes.

everforward 3y ago
You can, but I find it very tedious. O2M or M2M relationships become cumbersome. Not because they're hard to use, but just because there's so fucking many of them. I opted to store the app and version, and have a function that retrieves a config parser for it. I just didn't want to manage 300 tables just to get the schema to work. That's also partially because I'm using Ent in Go, so it generates a struct for each table. It'd make my auto-complete useless, and that's not a price I'm willing to pay on a personal project lol