5 ms·

Nah docker is excellent and far far preferable to the standard way of doing things for most people. All the traditional distribution methods have barely even f

Nah docker is excellent and far far preferable to the standard way of doing things for most people. All the traditional distribution methods have barely even figured out how to uninstall a piece of software. `apt remove` will often leave other things lying around. He complains about configuration being more complex because its not a file? Except it is, and its so much simpler to just have a compose file that tells you EXACTLY which files are used for configuration and where they are. This is still not easy in normal linux packages. You have to google or dig through a list of 10-15 places that may be used for config.. The other brilliant opinion is that docker makes the barrier to entry for packaging too low.. but the alternative is not those things being packaged well in apt or whatever, its them not being packaged at all.. this is not a win. Whats the alternative to running a big project like say Sourcegraph through a docker compose instance? You have to get set up ~10 services yourself including Redis, Postgres, some logging thing blah blah. I do not believe this is ever easier than `docker compose up -d`. And if you want to run it without docker, the docker images are basically a self-documenting system for how to do that. This is strictly a win over previous systems, where there generally just is no documentation for that. Personally docker lowers the activation energy to deploy something that I can now try/run so many complex pieces of software so easily. I run sourcegraph, nginx, postgres, redis on my machine easily. A week ago I wanted to learn more about data engineering - got a whole Apache Airflow cluster setup with a single compose file, took a few minutes. That would have been at least an hour of following some half-outdated deploy guide before so I just wouldn't have done it. --- The beginning of the post is most revealing though: > The fact that I have multiple times had to unpack flatpaks and modify them to fix dependencies reveals ... Still, these systems work reasonably well, well enough that they continue to proliferate... Basically you are okay with opening up flatpaks to modify them but not opening up docker images.. it just comes down to familiarity.

hinkley 3y ago
I just found about a half a gigabyte of files from programs I uninstalled two laptops ago on my machine, and files from a Steam game I got a refund for because it would crash after fifteen minutes. It’s frustrating.
angarg12 3y ago
I spent the last couple of days trying to set up some software waddling though open source repositories, fixing broken deps, pinning minor versions, browsing SO for obscure errors... I wish I had a Docker container instead.
ofrzeta 3y ago
> figured out how to uninstall a piece of software. `apt remove` will often leave other things lying around. What's the Docker way of uninstall? In most cases Docker packaged software uses some kind of volume or local mount to save data. Is there a way to remove these when you remove the container? What about networks? (besides running prune on all available entities)
- zaptheimpaler 3y ago
  You can `docker rm -v <container>` to remove a ccontainer + its volumes. For larger stuff typically I just read the docker-compose file with all of that listed in one place, so its pretty easy to just `docker rm` the volumes and networks. With apt I have no idea what files were added or modified and there's no simple "undo".
  - ofrzeta 3y ago
    I wasn't aware of '-v' so thanks for that. You can list package content with 'dpkg -L', although, granted, it doesn't cover files that were somehow created by the install script. Also 'apt purge' removes all files including config.
  - danielheath 3y ago
    It’s very nice, although I had a hard time getting the btrfs driver to actually remove the backing store a year or so back.
kbenson 3y ago
The thing I'm always interested in finding out is how people using lots of containers deal with security updates. Do they go into every image and run the updates for the base image it's based on? Does that just mean you're now subject to multiple OS security and patching and best practices? Do they think it doesn't matter and just get new images then they're published, which from what I can see is just when the overplayed software has an update and not the system anduvraries it relies on? When glibc or zlib have a security update, what do you do? For RHEL and derivatives it looks line quay.io tries to help with that, but what's the best practice for the rest of the community? We haven't really adopted containers at all at work yet, and to be honest this is at least part of the reason. It feels like we'd be doing all the patching we currently are, and then a bunch more. Sure, there are some gains, but if also feels like there's a lot more work involved there too?
- slekker 3y ago
  You can use tools like Snyk to scan images for vulnerabilities (even the Docker(tm) cli tool has one now), you can do things like failing your CI pipeline if there are critica vulnerabilities. You can also use Dependabot (and others) to update your images on a cronjob-like schedule. This is a solved problem
  - SkiFire13 3y ago
    This is a solution for the app developer, not for the users that might have to deal with older and unsupported apps.
  - Yasuraka 3y ago
    I highly advise going with Trivy over Snyk or, as it's still in beta at the moment, Docker Scout.
    - slekker 3y ago
      Hadn't heard of Trivy, but glad to see another great product from AquaSec :^)
- 3y ago
chillfox 3y ago
Exactly, Docker + Compose is the best way of running server software these days. I keep my compose files in source control and I have got a CI server building images for anything that doesn’t have first party images available. Updates are super easy as well, just update the pinned version at the top of the compose file (if not using latest), then ’docker-compose pull’ followed by ’docker-compose up -d’ The entire thing is so much more stable and easier to manage than the RedHat/Ubuntu/FreeBSD systems I used to manage. (I use Alpine Linux + ZFS for the host OS)
progval 3y ago
> Whats the alternative to running a big project like say Sourcegraph through a docker compose instance? You have to get set up ~10 services yourself including Redis, Postgres, some logging thing blah blah. I do not believe this is ever easier than `docker compose up -d`. Even with Docker, it's not as easy as `docker compose up -d`. You need to setup backups too. How do you backup all these containers? Do they even support it? Meanwhile I already have Postgres and Redis tuned, running, and backuped. I'd even have a warm Postgres replica if I could figure how to make that work. Then you need to monitor them for security updates. How are you notified your Sourcegraph container and its dependencies' containers need to be updated and restarted? If they used system packages, I'd already have this solved with Munin's APT plugin and/or Debian's unattended-upgrades. So you need to install Watchtower or equivalent. Which can't tell the difference between a security update and other updates, so you might have your software updated with breaking changes at any point. Alternatively, you can locate and subscribe to the RSS feed for the repository of every Dockerfile involved (or use one of these third-party services providing RSS feeds for Dockerhub) and hope they contain a changelog. If I installed from Debian I wouldn't need that because I'm already subscribed to debian-security-announce@lists.debian.org
- stavros 3y ago
  I don't back up containers, why would you? I back up the attached volumes, or, more often, their state is fully in a database, which isn't in a container (it doesn't need to be), so I just back the database up.
  - progval 3y ago
    I meant backing up volumes.
    - imtringued 3y ago
      Number one advice for using docker volumes: Don't use docker volumes. Just mount directories or files from the host file system. I have never had a situation where I didn't regret docker volumes but where I regretted host FS mounts. How do you backup a directory? You tell me, because I have to use a multi gigabyte software package that, mind you, is a pain to install since it consists of like two dozen packages.
bakuninsbart 3y ago
> A week ago I wanted to learn more about data engineering - got a whole Apache Airflow cluster setup with a single compose file, took a few minutes. Off-topic, but how did you like it? Tried it out a couple of years ago and felt like it overcomplicates things for probably 99% of use cases, and the overhead is huge.
- zaptheimpaler 3y ago
  Over-complicated, I agree. "This cluster could have been a Makefile" There probably are good reasons to use it when you have complex distributed DAGs though.
rigid 3y ago
> All the traditional distribution methods have barely even figured out how to uninstall a piece of software. `apt remove` will often leave other things lying around. Sounds like an issue that needs to be fixed instead of working around it. Also dependency hell. Few distros manage those hard problems nicely but they do exist.
bandrami 3y ago
Wait wait wait. Docker has two use cases and you're conflating them. The original use case is: Project Foo uses libqxt version 7. Project Bar uses libqxt version 8. They are incompatible so I'd need two development workstations (or later two LXC containers). This is slow and heavy on diskspace; docker solves that problem. This is a great use of docker. The second use case that it has morphed into is: I've decided stack management is hard and I can't tell my downstream which libraries they need because even I'm no longer sure. So I'll just bundle them all as this opaque container and distribute it that way and now literally nobody knows what versions of what software they are running in production. This is a very harmful use case of docker that is unfortunately nearly universal at this point.
- jbverschoor 3y ago
  > Project Foo uses libqxt version 7. Project Bar uses libqxt version 8 Pretty much fixed by asdf or any sane manager.. Also, two containers aren't slow at all, but rebuilding one completely from scratch is. The latter is a good use case. No big deal to include a commit/version number, and a list of packages / libraries used in a lock file.. The problems with docker I have are: no lock files to pin versions. No integration with other package management tools. For example, I want to install my dependencies as a step in Docker. I don't want to manually copy over the list of dependencies from my lockfiles (gemfile, package, etc). The reason why I'd want this is that it speeds up builds.
- zaptheimpaler 3y ago
  I don't understand how stack management without Docker is any better. As far as I can tell, the alternative without Docker is the same. A list of `apt install`s with no versions listed or no mention of dependencies at all. If you used a lockfile in a language package manager, you can use the same lockfile in the docker too.
  - bandrami 3y ago
    So, as an example, 20 years ago when we started a project the devs and I (the sysadmin) would sit down and have a meeting where they would talk about what libraries they wanted to use (think CPAN or PEAR) and I would tell them what versions were compatible with our servers and those are the versions they would use for the lifetime of the project. Docker came about because devs got tired of hearing "no" to upgrades and admins got tired of saying it. But in 2023 if there was a zero day (though we didn't call them that back then) for libqxt7.2 (but not 7.0 or 8.1) I could tell you immediately if we were impacted. In 2023 I can't even tell you how many versions of OpenSSL are on my laptop right now as I type.
ckastner 3y ago
> `apt remove` will often leave other things lying around. If you mean configuration files, then this is by design. `apt purge` removes those as well.
nucleardog 3y ago
> Whats the alternative to running a big project like say Sourcegraph through a docker compose instance? You have to get set up ~10 services yourself including Redis, Postgres, some logging thing blah blah. I do not believe this is ever easier than `docker compose up -d`. Maybe my recollection is just fuzzy, but it seems to me back in the day many projects just had fewer dependencies and more of them were optional. "For larger installations and/or better performance, here's where to configure your redis instance." Instead now you try and run someone's little self-hosted bookmark manager and it's "just" docker-compose up to spin up the backend API, frontend UI, Postgres, redis, elasticsearch, thumbor, and a KinD container that we use to dynamically run pods for scraping your bookmarked websites! I'd almost _rather_ that sort of setup be reserved for stuff where it's worth investing the time to set it up. All of this complexity is easier to get up this way, but that doesn't make it easier to properly manage or a _good_ way to do things. I'd much rather run _one_ instance of Postgres, set up proper backups _once_, perform upgrades _once_, etc. Even if I don't care about the hardware resource usage, I do care about my time. How do I point this at an external postgres instance? Unfortunately, the setup instructions for many services these days start _and end_ at `docker-compose up`. And this idea of "dockerfiles as documentation" should really die. There are often so many implicit assumptions baked into them as to make them a complete minefield for use as a reference. And unless you're going to dig into a thousand lines of bash scripts, they're not going to answer the questions you actually need answers to like "how do I change this configuration option?".
bayindirh 3y ago
> All the traditional distribution methods have barely even figured out how to uninstall a piece of software. The most traditional way is to compile under /usr/local/application_name, and symlinking to /usr/local/(s)bin. Remove the folder and the links, and you're done. > `apt remove` will often leave other things lying around. "remove" is designed to leave config and database files in place, assuming that you might want to install it later without losing data. apt has "purge" option for the last decade which removes anything and everything completely. > Whats the alternative to running a big project like say Sourcegraph through a docker compose instance? You have to get set up ~10 services yourself including Redis, Postgres, some logging thing blah blah. I do not believe this is ever easier than `docker compose up -d`. Install in a single or on a couple of pet servers. Configure them, forget them. If you fancy, spawn a couple of VMs, and snapshot them daily. While it takes a bit more time, for a single time job, I won't mind. Docker's biggest curse is it enables many bad practices and advertises as best practices. Why enable SSL on a service while I can add a SSL terminating container? Why tune a piece of software if I can spawn five more copies of that for scaling?, etc. Docker is nice when it's packaged and documented well, and good for packaging stable meta-utilities (e.g.: A documenting pipeline which runs daily and terminates), but using it as a silver bullet and accepting its bad practices as gold standards is wasting resources, space, time; and creating security problems at the same time. Basically you are okay with installing containers blindly but not installing services and learning apt purge. it just comes down to familiarity. Obligatory XKCD: https://xkcd.com/1988/ https://xkcd.com/1988/

hinkley 3y ago

I just found about a half a gigabyte of files from programs I uninstalled two laptops ago on my machine, and files from a Steam game I got a refund for because it would crash after fifteen minutes. It’s frustrating.

angarg12 3y ago

I spent the last couple of days trying to set up some software waddling though open source repositories, fixing broken deps, pinning minor versions, browsing SO for obscure errors... I wish I had a Docker container instead.

ofrzeta 3y ago

> figured out how to uninstall a piece of software. `apt remove` will often leave other things lying around. What's the Docker way of uninstall? In most cases Docker packaged software uses some kind of volume or local mount to save data. Is there a way to remove these when you remove the container? What about networks? (besides running prune on all available entities)

zaptheimpaler 3y ago
You can `docker rm -v <container>` to remove a ccontainer + its volumes. For larger stuff typically I just read the docker-compose file with all of that listed in one place, so its pretty easy to just `docker rm` the volumes and networks. With apt I have no idea what files were added or modified and there's no simple "undo".
- ofrzeta 3y ago
  I wasn't aware of '-v' so thanks for that. You can list package content with 'dpkg -L', although, granted, it doesn't cover files that were somehow created by the install script. Also 'apt purge' removes all files including config.
- danielheath 3y ago
  It’s very nice, although I had a hard time getting the btrfs driver to actually remove the backing store a year or so back.

kbenson 3y ago

The thing I'm always interested in finding out is how people using lots of containers deal with security updates. Do they go into every image and run the updates for the base image it's based on? Does that just mean you're now subject to multiple OS security and patching and best practices? Do they think it doesn't matter and just get new images then they're published, which from what I can see is just when the overplayed software has an update and not the system anduvraries it relies on? When glibc or zlib have a security update, what do you do? For RHEL and derivatives it looks line quay.io tries to help with that, but what's the best practice for the rest of the community? We haven't really adopted containers at all at work yet, and to be honest this is at least part of the reason. It feels like we'd be doing all the patching we currently are, and then a bunch more. Sure, there are some gains, but if also feels like there's a lot more work involved there too?

slekker 3y ago
You can use tools like Snyk to scan images for vulnerabilities (even the Docker(tm) cli tool has one now), you can do things like failing your CI pipeline if there are critica vulnerabilities. You can also use Dependabot (and others) to update your images on a cronjob-like schedule. This is a solved problem
- SkiFire13 3y ago
  This is a solution for the app developer, not for the users that might have to deal with older and unsupported apps.
- Yasuraka 3y ago
  I highly advise going with Trivy over Snyk or, as it's still in beta at the moment, Docker Scout.
  - slekker 3y ago
    Hadn't heard of Trivy, but glad to see another great product from AquaSec :^)
3y ago

chillfox 3y ago

Exactly, Docker + Compose is the best way of running server software these days. I keep my compose files in source control and I have got a CI server building images for anything that doesn’t have first party images available. Updates are super easy as well, just update the pinned version at the top of the compose file (if not using latest), then ’docker-compose pull’ followed by ’docker-compose up -d’ The entire thing is so much more stable and easier to manage than the RedHat/Ubuntu/FreeBSD systems I used to manage. (I use Alpine Linux + ZFS for the host OS)

progval 3y ago

> Whats the alternative to running a big project like say Sourcegraph through a docker compose instance? You have to get set up ~10 services yourself including Redis, Postgres, some logging thing blah blah. I do not believe this is ever easier than `docker compose up -d`. Even with Docker, it's not as easy as `docker compose up -d`. You need to setup backups too. How do you backup all these containers? Do they even support it? Meanwhile I already have Postgres and Redis tuned, running, and backuped. I'd even have a warm Postgres replica if I could figure how to make that work. Then you need to monitor them for security updates. How are you notified your Sourcegraph container and its dependencies' containers need to be updated and restarted? If they used system packages, I'd already have this solved with Munin's APT plugin and/or Debian's unattended-upgrades. So you need to install Watchtower or equivalent. Which can't tell the difference between a security update and other updates, so you might have your software updated with breaking changes at any point. Alternatively, you can locate and subscribe to the RSS feed for the repository of every Dockerfile involved (or use one of these third-party services providing RSS feeds for Dockerhub) and hope they contain a changelog. If I installed from Debian I wouldn't need that because I'm already subscribed to debian-security-announce@lists.debian.org

stavros 3y ago
I don't back up containers, why would you? I back up the attached volumes, or, more often, their state is fully in a database, which isn't in a container (it doesn't need to be), so I just back the database up.
- progval 3y ago
  I meant backing up volumes.
  - imtringued 3y ago
    Number one advice for using docker volumes: Don't use docker volumes. Just mount directories or files from the host file system. I have never had a situation where I didn't regret docker volumes but where I regretted host FS mounts. How do you backup a directory? You tell me, because I have to use a multi gigabyte software package that, mind you, is a pain to install since it consists of like two dozen packages.

bakuninsbart 3y ago

> A week ago I wanted to learn more about data engineering - got a whole Apache Airflow cluster setup with a single compose file, took a few minutes. Off-topic, but how did you like it? Tried it out a couple of years ago and felt like it overcomplicates things for probably 99% of use cases, and the overhead is huge.

zaptheimpaler 3y ago
Over-complicated, I agree. "This cluster could have been a Makefile" There probably are good reasons to use it when you have complex distributed DAGs though.

rigid 3y ago

> All the traditional distribution methods have barely even figured out how to uninstall a piece of software. `apt remove` will often leave other things lying around. Sounds like an issue that needs to be fixed instead of working around it. Also dependency hell. Few distros manage those hard problems nicely but they do exist.

bandrami 3y ago

Wait wait wait. Docker has two use cases and you're conflating them. The original use case is: Project Foo uses libqxt version 7. Project Bar uses libqxt version 8. They are incompatible so I'd need two development workstations (or later two LXC containers). This is slow and heavy on diskspace; docker solves that problem. This is a great use of docker. The second use case that it has morphed into is: I've decided stack management is hard and I can't tell my downstream which libraries they need because even I'm no longer sure. So I'll just bundle them all as this opaque container and distribute it that way and now literally nobody knows what versions of what software they are running in production. This is a very harmful use case of docker that is unfortunately nearly universal at this point.

jbverschoor 3y ago
> Project Foo uses libqxt version 7. Project Bar uses libqxt version 8 Pretty much fixed by asdf or any sane manager.. Also, two containers aren't slow at all, but rebuilding one completely from scratch is. The latter is a good use case. No big deal to include a commit/version number, and a list of packages / libraries used in a lock file.. The problems with docker I have are: no lock files to pin versions. No integration with other package management tools. For example, I want to install my dependencies as a step in Docker. I don't want to manually copy over the list of dependencies from my lockfiles (gemfile, package, etc). The reason why I'd want this is that it speeds up builds.
zaptheimpaler 3y ago
I don't understand how stack management without Docker is any better. As far as I can tell, the alternative without Docker is the same. A list of `apt install`s with no versions listed or no mention of dependencies at all. If you used a lockfile in a language package manager, you can use the same lockfile in the docker too.
- bandrami 3y ago
  So, as an example, 20 years ago when we started a project the devs and I (the sysadmin) would sit down and have a meeting where they would talk about what libraries they wanted to use (think CPAN or PEAR) and I would tell them what versions were compatible with our servers and those are the versions they would use for the lifetime of the project. Docker came about because devs got tired of hearing "no" to upgrades and admins got tired of saying it. But in 2023 if there was a zero day (though we didn't call them that back then) for libqxt7.2 (but not 7.0 or 8.1) I could tell you immediately if we were impacted. In 2023 I can't even tell you how many versions of OpenSSL are on my laptop right now as I type.

ckastner 3y ago

> `apt remove` will often leave other things lying around. If you mean configuration files, then this is by design. `apt purge` removes those as well.

nucleardog 3y ago

> Whats the alternative to running a big project like say Sourcegraph through a docker compose instance? You have to get set up ~10 services yourself including Redis, Postgres, some logging thing blah blah. I do not believe this is ever easier than `docker compose up -d`. Maybe my recollection is just fuzzy, but it seems to me back in the day many projects just had fewer dependencies and more of them were optional. "For larger installations and/or better performance, here's where to configure your redis instance." Instead now you try and run someone's little self-hosted bookmark manager and it's "just" docker-compose up to spin up the backend API, frontend UI, Postgres, redis, elasticsearch, thumbor, and a KinD container that we use to dynamically run pods for scraping your bookmarked websites! I'd almost _rather_ that sort of setup be reserved for stuff where it's worth investing the time to set it up. All of this complexity is easier to get up this way, but that doesn't make it easier to properly manage or a _good_ way to do things. I'd much rather run _one_ instance of Postgres, set up proper backups _once_, perform upgrades _once_, etc. Even if I don't care about the hardware resource usage, I do care about my time. How do I point this at an external postgres instance? Unfortunately, the setup instructions for many services these days start _and end_ at `docker-compose up`. And this idea of "dockerfiles as documentation" should really die. There are often so many implicit assumptions baked into them as to make them a complete minefield for use as a reference. And unless you're going to dig into a thousand lines of bash scripts, they're not going to answer the questions you actually need answers to like "how do I change this configuration option?".

bayindirh 3y ago

> All the traditional distribution methods have barely even figured out how to uninstall a piece of software. The most traditional way is to compile under /usr/local/application_name, and symlinking to /usr/local/(s)bin. Remove the folder and the links, and you're done. > `apt remove` will often leave other things lying around. "remove" is designed to leave config and database files in place, assuming that you might want to install it later without losing data. apt has "purge" option for the last decade which removes anything and everything completely. > Whats the alternative to running a big project like say Sourcegraph through a docker compose instance? You have to get set up ~10 services yourself including Redis, Postgres, some logging thing blah blah. I do not believe this is ever easier than `docker compose up -d`. Install in a single or on a couple of pet servers. Configure them, forget them. If you fancy, spawn a couple of VMs, and snapshot them daily. While it takes a bit more time, for a single time job, I won't mind. Docker's biggest curse is it enables many bad practices and advertises as best practices. Why enable SSL on a service while I can add a SSL terminating container? Why tune a piece of software if I can spawn five more copies of that for scaling?, etc. Docker is nice when it's packaged and documented well, and good for packaging stable meta-utilities (e.g.: A documenting pipeline which runs daily and terminates), but using it as a silver bullet and accepting its bad practices as gold standards is wasting resources, space, time; and creating security problems at the same time. Basically you are okay with installing containers blindly but not installing services and learning apt purge. it just comes down to familiarity. Obligatory XKCD: https://xkcd.com/1988/ https://xkcd.com/1988/