5 ms·

If you haven’t seen the benefits, you’re not in the business of deploying a variety of applications to servers. The fact that I don’t have to install dependenc

by 9dev 2y ago

If you haven’t seen the benefits, you’re not in the business of deploying a variety of applications to servers. The fact that I don’t have to install dependencies on a server, or set up third-party applications like PHP, Apache, Redis, and the myriad of other packages anymore, or manage config files in /etc, or handle upgrades of libc gracefully, or worry about rolling restarts and maintenance downtime… all of this was solvable before, but has become radically easier with containers. Packaging an application and its dependencies into a single, distributable artifact that can be passed around and used on all kinds of machines was a glorious success.

steve1977 2y ago
I’m aware of all of that, I’m just saying that this has not translated into more reliable and better software in the end, interestingly enough. As said, I’m not blaming Docker, at least not directly. It’s more that the whole “ecosystem” around it seems to have so many disadvantages that in the end overweigh the advantages of Docker.
- derefr 2y ago
  It has translated to reliable legacy software. You can snapshot a piece of software, together with its runtime environment, at the point when it's still possible to build it; and then you can continue to run that built OCI image, with low overhead, on modern hardware — even when building the image from scratch has long become impossible due to e.g. all the package archives that the image fetched from going offline. (And this enables some increasingly wondrous acts of software archaeology, due to people building OCI images not for preservation, but just for "use at the time" — and then just never purging them from whatever repository they've pushed them to. People are preserving historical software builds in a runnable state, completely by accident!) Before Docker, the nearest thing you could do to this was to package software as a VM image — and there was no standard for what "a VM image" was, so this wasn't a particularly portable/long-term solution. Often VM-image formats became unsupported faster than the software held in them did! But now, with OCI images, we're nearly to the point where we've e.g. convinced academic science to publish a paper's computational apparatus as an OCI image, so that it can be pulled 10 years later when attempting to replicate the paper.
  - steve1977 2y ago
    > You can snapshot a piece of software, together with its runtime environment, at the point when it's still possible to build it I think you’re onto part of the problem here. The thing is that you have to snapshot a lot of nowadays software together with its runtime environment. I mean, I can still run Windows software (for example) that is 10 years or older without that requirement.
    - 9dev 2y ago
      The price for that kind of backwards compatibility is a literal army of engineers working for a global megacorporation. Free software could not manage that, so having a pragmatic way to keep software running in isolated containers seems like a great solution to me.
      - steve1977 2y ago
        There’s an army of developers working on Linux as well, employed by companies like IBM and Oracle. I don’t see a huge difference to Microsoft here to be honest.
        sangnoir 2y ago
        You'd have a better time working with Windows 7 than a 2.x Linux kernel. I love Linux, but Microsoft has longer support Windows for its operating systems.
    - ahnick 2y ago
      What are you even talking about? Being able to run 10 year old software (on any OS) is orthogonal to being able to build a piece software whose dependencies are completely missing. Don't pretend like this doesn't happen on Windows.
      - steve1977 2y ago
        My point was that a lot of older software, especially desktop apps, did not have such wild dependencies. Therefore this was less of an issue. Today with Python and with JavaScript and its NPM hell it is of course.
- 9dev 2y ago
  Im not sure I would agree here: from my personal experience, the increasing containerisation has definitely nudged lots of large software projects to behave better; they don’t spew so many artifacts all over the filesystem anymore, for example, and increasingly adopt environment variables for configuration. Additionally, I think lots of projects became able to adopt better tooling faster, since the barrier to use container-based tools is lower. Just think of GitHub Actions, which suddenly enabled everyone and their mother to adopt CI pipelines. That simply wasn’t possible before, and has led to more software adopting static analysis and automated testing, I think.
  - steve1977 2y ago
    This might all be true, but has this actually resulted in better software for end users? More stability, faster delivery of useful features? That is my concern.
    - watermelon0 2y ago
      For SaaS, I'd say it definitely improved and sped up delivery of the software from development machine to CI to production environment. How this translates to actual end users, it's totally up to the developers/DevOps/etc. of each product. For self-hosted software, be it for business or personal use, it immensely simplified how a software package can be pulled, and run in isolated environment. Dependency hell is avoided, and you can easily create/start/stop/delete a specific software, without affecting the rest of the host machine.
- bzmrgonz 2y ago
  I think the disconnect is in viewing your trees and not viewing the forest. Sure you were a responsible disciplined tree engineer for your acres, but what about the rest of the forest? Can we at least agree that docker made plant husbandry easier for the masses world-wide??
PaulHoule 2y ago
Circa 2005 I was working at places where I was responsible for 80 and 300 web sites respectively using a large range of technologies. On my own account I had about 30 domain names. I had scripts that would automatically generate the Apache configuration to deploy a new site in less than 30 seconds. At that time I found that most web sites have just a few things to configure: often a database connection, the path to where files are, and maybe a cryptographic secret. If you are systematic about where you put your files and how you do your configuration running servers with a lot of sites is about as easy as falling off a log, not to mention running development, test, staging, prod and any other sites you need. I have a Python system now with gunicorn servers and celery workers that exists in three instances on my PC, because I am disciplined and everything is documented I could bring it up on another machine manually pretty quickly, probably more quickly than I could download 3GB worth of docker images over my ADSL connection. With a script it would be no contest. There also was a time I was building AMIs and even selling them on the AMZN marketplace and the formula was write a Java program that writes a shell script that an EC2 instance runs on boot, when it is done it sends a message through SQS to tell the Java program to shut down and image the new machine. If Docker is anything it is a system that turns 1 MB worth of I/O into 1 GB of I/O. I found Docker was slowing me down when I was using a gigabit connection, I found it basically impossible to do anything with it (like boot up an image) on a 2MB/sec ADSL connection, with my current pair of 20MB/s connections it is still horrifyingly slow. I like how the OP is concerned about I/O speed and bringing it up and I think it could be improved if there was a better cache system (e.g. Docker might even work on slow ADSL if it properly recovered from failed downloads) However I think Docker has a conflict between “dev” (where I’d say your build is slow if you ever perceive yourself to be waiting) and “ops” (where a 20 minute build is “internet time”) I think ops is often happy with Docker, some devs really seem to like it, but for some of us it is a way to make a 20 sec task a 20 minute task.
- cogman10 2y ago
  And I'm guessing with this system you had a standard version of python, apache, and everything else. I imagine that with this system if you wanted to update to the latest version of python, in involved a long process making sure those 80 or 300 websites didn't break because of some random undocumented breaking change. As for docker image size, really just depends on dev discipline for better or for worse. The nginx image, for example, adds about 1MB of data on top of the whatever you did with your website.
mnahkies 2y ago
Agreed, I remember having to vendor runtimes to my services because we couldn't risk upgrading the system installed versions with the number of things running on the box, which then led into horrible hacks with LD_PRELOAD to workaround a mixture of OS / glibc version's in the fleet. Adding another replica of anything was a pain. Now I don't have to care what OS the host is running, or what dependencies are installed, and adding replicas is either automatic or editing a number in a config file. Containerization and orchestration tools like k8s have made life so much easier.
nitwit005 2y ago
As you note, it was all solvable before. A lot of us were just forced to "switch" from VMs to Docker; Docker that still got deployed to a VM. And then we got forced to switch to podman as they didn't want to pay for Docker.
- 9dev 2y ago
  > As you note, it was all solvable before. Washing clothes was possible before people had a washing machine, too; I’m not sure they would want to go back to that, though. I was there in the VM time, and I had to set up appliances shipped as a VM instance. It was awful. The complexity around updates and hypervisors, and all that OS adjustment work just to get a runtime environment going, that just disappeared with Docker (if done right, I’ll give you that). Organisations manage to abuse technology all the time. Remember when Roy Fielding wrote about using HTTP sensibly to transfer state from one system to another? Suddenly everything had to be „RESTful“, which for most people just meant that you tried to use as many HTTP verbs as possible and performed awkward URL gymnastics to get speaking resource identifiers. Horrible. But all of this doesn’t mean REST is a bad idea of itself - it’s a wonderful one, in fact, and can make an API substantially easier to reason about.

by 9dev 2y ago

steve1977 2y ago

I’m aware of all of that, I’m just saying that this has not translated into more reliable and better software in the end, interestingly enough. As said, I’m not blaming Docker, at least not directly. It’s more that the whole “ecosystem” around it seems to have so many disadvantages that in the end overweigh the advantages of Docker.

derefr 2y ago
It has translated to reliable legacy software. You can snapshot a piece of software, together with its runtime environment, at the point when it's still possible to build it; and then you can continue to run that built OCI image, with low overhead, on modern hardware — even when building the image from scratch has long become impossible due to e.g. all the package archives that the image fetched from going offline. (And this enables some increasingly wondrous acts of software archaeology, due to people building OCI images not for preservation, but just for "use at the time" — and then just never purging them from whatever repository they've pushed them to. People are preserving historical software builds in a runnable state, completely by accident!) Before Docker, the nearest thing you could do to this was to package software as a VM image — and there was no standard for what "a VM image" was, so this wasn't a particularly portable/long-term solution. Often VM-image formats became unsupported faster than the software held in them did! But now, with OCI images, we're nearly to the point where we've e.g. convinced academic science to publish a paper's computational apparatus as an OCI image, so that it can be pulled 10 years later when attempting to replicate the paper.
- steve1977 2y ago
  > You can snapshot a piece of software, together with its runtime environment, at the point when it's still possible to build it I think you’re onto part of the problem here. The thing is that you have to snapshot a lot of nowadays software together with its runtime environment. I mean, I can still run Windows software (for example) that is 10 years or older without that requirement.
  - 9dev 2y ago
    The price for that kind of backwards compatibility is a literal army of engineers working for a global megacorporation. Free software could not manage that, so having a pragmatic way to keep software running in isolated containers seems like a great solution to me.
    - steve1977 2y ago
      There’s an army of developers working on Linux as well, employed by companies like IBM and Oracle. I don’t see a huge difference to Microsoft here to be honest.
      - sangnoir 2y ago
        You'd have a better time working with Windows 7 than a 2.x Linux kernel. I love Linux, but Microsoft has longer support Windows for its operating systems.
  - ahnick 2y ago
    What are you even talking about? Being able to run 10 year old software (on any OS) is orthogonal to being able to build a piece software whose dependencies are completely missing. Don't pretend like this doesn't happen on Windows.
    - steve1977 2y ago
      My point was that a lot of older software, especially desktop apps, did not have such wild dependencies. Therefore this was less of an issue. Today with Python and with JavaScript and its NPM hell it is of course.
9dev 2y ago
Im not sure I would agree here: from my personal experience, the increasing containerisation has definitely nudged lots of large software projects to behave better; they don’t spew so many artifacts all over the filesystem anymore, for example, and increasingly adopt environment variables for configuration. Additionally, I think lots of projects became able to adopt better tooling faster, since the barrier to use container-based tools is lower. Just think of GitHub Actions, which suddenly enabled everyone and their mother to adopt CI pipelines. That simply wasn’t possible before, and has led to more software adopting static analysis and automated testing, I think.
- steve1977 2y ago
  This might all be true, but has this actually resulted in better software for end users? More stability, faster delivery of useful features? That is my concern.
  - watermelon0 2y ago
    For SaaS, I'd say it definitely improved and sped up delivery of the software from development machine to CI to production environment. How this translates to actual end users, it's totally up to the developers/DevOps/etc. of each product. For self-hosted software, be it for business or personal use, it immensely simplified how a software package can be pulled, and run in isolated environment. Dependency hell is avoided, and you can easily create/start/stop/delete a specific software, without affecting the rest of the host machine.
bzmrgonz 2y ago
I think the disconnect is in viewing your trees and not viewing the forest. Sure you were a responsible disciplined tree engineer for your acres, but what about the rest of the forest? Can we at least agree that docker made plant husbandry easier for the masses world-wide??

PaulHoule 2y ago

Circa 2005 I was working at places where I was responsible for 80 and 300 web sites respectively using a large range of technologies. On my own account I had about 30 domain names. I had scripts that would automatically generate the Apache configuration to deploy a new site in less than 30 seconds. At that time I found that most web sites have just a few things to configure: often a database connection, the path to where files are, and maybe a cryptographic secret. If you are systematic about where you put your files and how you do your configuration running servers with a lot of sites is about as easy as falling off a log, not to mention running development, test, staging, prod and any other sites you need. I have a Python system now with gunicorn servers and celery workers that exists in three instances on my PC, because I am disciplined and everything is documented I could bring it up on another machine manually pretty quickly, probably more quickly than I could download 3GB worth of docker images over my ADSL connection. With a script it would be no contest. There also was a time I was building AMIs and even selling them on the AMZN marketplace and the formula was write a Java program that writes a shell script that an EC2 instance runs on boot, when it is done it sends a message through SQS to tell the Java program to shut down and image the new machine. If Docker is anything it is a system that turns 1 MB worth of I/O into 1 GB of I/O. I found Docker was slowing me down when I was using a gigabit connection, I found it basically impossible to do anything with it (like boot up an image) on a 2MB/sec ADSL connection, with my current pair of 20MB/s connections it is still horrifyingly slow. I like how the OP is concerned about I/O speed and bringing it up and I think it could be improved if there was a better cache system (e.g. Docker might even work on slow ADSL if it properly recovered from failed downloads) However I think Docker has a conflict between “dev” (where I’d say your build is slow if you ever perceive yourself to be waiting) and “ops” (where a 20 minute build is “internet time”) I think ops is often happy with Docker, some devs really seem to like it, but for some of us it is a way to make a 20 sec task a 20 minute task.

cogman10 2y ago
And I'm guessing with this system you had a standard version of python, apache, and everything else. I imagine that with this system if you wanted to update to the latest version of python, in involved a long process making sure those 80 or 300 websites didn't break because of some random undocumented breaking change. As for docker image size, really just depends on dev discipline for better or for worse. The nginx image, for example, adds about 1MB of data on top of the whatever you did with your website.

mnahkies 2y ago

Agreed, I remember having to vendor runtimes to my services because we couldn't risk upgrading the system installed versions with the number of things running on the box, which then led into horrible hacks with LD_PRELOAD to workaround a mixture of OS / glibc version's in the fleet. Adding another replica of anything was a pain. Now I don't have to care what OS the host is running, or what dependencies are installed, and adding replicas is either automatic or editing a number in a config file. Containerization and orchestration tools like k8s have made life so much easier.

nitwit005 2y ago

As you note, it was all solvable before. A lot of us were just forced to "switch" from VMs to Docker; Docker that still got deployed to a VM. And then we got forced to switch to podman as they didn't want to pay for Docker.

9dev 2y ago
> As you note, it was all solvable before. Washing clothes was possible before people had a washing machine, too; I’m not sure they would want to go back to that, though. I was there in the VM time, and I had to set up appliances shipped as a VM instance. It was awful. The complexity around updates and hypervisors, and all that OS adjustment work just to get a runtime environment going, that just disappeared with Docker (if done right, I’ll give you that). Organisations manage to abuse technology all the time. Remember when Roy Fielding wrote about using HTTP sensibly to transfer state from one system to another? Suddenly everything had to be „RESTful“, which for most people just meant that you tried to use as many HTTP verbs as possible and performed awkward URL gymnastics to get speaking resource identifiers. Horrible. But all of this doesn’t mean REST is a bad idea of itself - it’s a wonderful one, in fact, and can make an API substantially easier to reason about.