8 ms·

A full decade since we took the 'it works on my machine' excuse and turned it into the industry standard architecture ('then we'll just ship your machine to pro

A full decade since we took the 'it works on my machine' excuse and turned it into the industry standard architecture ('then we'll just ship your machine to production').

redhanuman 7mo ago
the real trick was making "ship your machine" sound like best practice and ten years later we r doing the same thing with ai "it works in my notebook" jst became "containerize the notebook and call it a pipeline" the abstraction always wins because fixing the actual problem is just too hard.
- goodpoint 7mo ago
  ...while completely forgetting about security
- zbentley 7mo ago
  > fixing the actual problem is just too hard. I think it’s laziness, not difficulty. That’s not meant to be snide or glib: I think gaining expertise in how to package and deploy non-containerized applications isn’t difficult or unattainable for most engineers; rather, it’s tedious and specialized work to gain that expertise, and Docker allowed much of the field to skip doing it. That’s not good or bad per se, but I do think it’s different from “pre-container deployment was hard”. Pre-container deployment was neglected and not widely recognized as a specialty that needed to be cultivated, so most shops sucked at it. That’s not the same as “hard”.
  - skydhash 7mo ago
    It's not even laziness or expertise. A lot of people are against learning conventions. They want their way, meaning what works on their computer. That's why they like the current scope of package managers, docker, flatpack,... They can do what they want in the sandbox provided however nonsensical and then ship the whole thing. And it will break if you look at it the wrong way.
- Bratmon 7mo ago
  I mean, walking through a door is easier than tearing down a wall, walking through it, and rebuilding the wall. That doesn't mean the latter is a good idea.
forrestthewoods 7mo ago
Linux user space is an abject disaster of a design. So so so bad. Docker should not need to exist. Running computer programs need not be so difficult.
- esafak 7mo ago
  Who does it right?
  - whateverboat 7mo ago
    Windows.
    - deleted 7mo ago
      [deleted]
  - jjmarr 7mo ago
    Nix and Guix. Good luck convincing people to switch!
    - abacate 7mo ago
      Trying to convince people usually makes any resistance worse. Using it, solving problems with it, and building a real community around it tend to make a much greater impact in the long run.
      - NortySpock 7mo ago
        Yeah, but if the problem you are solving is rare for most practitioners, effectively theoretical until it actually happens, then people won't switch until they get bit by that particular problem.
    - zbentley 7mo ago
      But they’re roughly the same paradigm as docker, right? My understanding of the Nix approach is that it’s still reproducing most of a user land/filesystem in a captive/separate/sandbox environment. Like, docker is using namespaces for more stuff, Nix has a heavier emphasis on reproducibility/determinism, but … they’re both still throwing in the towel on deploying directly on the underlying OS’s userland (unless you go all the way to nixOS) and shipping what amounts to a filesystem in a box, no?
chuckadams 7mo ago
It's the ultimate in static linking. Perhaps a question that should be asked is why that approach is so compelling?
- blackcatsec 7mo ago
  I question that as well, it's also why Go is extremely popular. Could it just be a pendulum swing back towards static linking? Wonder when some enterprising OSS dev will rebrand dynamic linking in the future...
  - jfjasdfuw 7mo ago
    CGO_ENABLED=0 is sigma tier. I don't care about glibc or compatibility with /etc/nsswitch.conf. look at the hack rust does because it uses libc: > pub unsafe fn set_var<K: AsRef<OsStr>, V: AsRef<OsStr>>(key: K, value: V)
    - jcgl 7mo ago
      > I don't care about glibc or compatibility with /etc/nsswitch.conf. So what do you do when you need to resolve system users? I sure hope you don't parse /etc/passwd, since plenty of users (me included) use other user databases (e.g. sssd or systemd-userdbd).
      - cyberax 7mo ago
        Most software doesn't need to resolve users. You also can always shell out to `id` if you need an occasional bit of metadata.
        jcgl 7mo ago
        That's a fair point, and shelling out to id is probably a good solution. I guess what bothers me is the software authors who don't think this through, leaving applications non-functional in these situations. At least with Go, if you do CGO_ENABLED=0, and you use the stdlib functions to resolve user information, you end up with parsed /etc/passwd instead of shelling out to id. The Go stdlib should maybe shell out to id instead, but it doesn't. And it's understandable that software developers use the stdlib functions without thinking all too much about it. But in the end, simply advocating for CGO_ENABLED=0 results in software that is broken around the edges.
avsm 7mo ago
(coauthor of the article here) Well, before Docker I used to work on Xen and that possible future of massive block devices assembled using Vagrant and Packer has thankfully been avoided... One thing that's hard to capture in the article -- but that permeated the early Dockercons -- is the (positive) disruption Docker had in how IT shops were run. Before that going to production was a giant effort, and 'shipping your filesystem' quickly was such a change in how people approached their work. We had so many people come up to us grateful that they could suddenly build services more quickly and get them into the hands of users without having to seek permission slips signed in triplicate. We're seeing the another seismic cultural shift now with coding agents, but I think Docker had a similar impact back then, and it was a really fun community spirit. Less so today with the giant hyperscalars all dominating, sadly, but I'll keep my fond memories :-)
- talkvoix 7mo ago
  Great point about coding agents! Back then, Docker gave us 'it works on my machine, let's ship the machine'. Now, AI agents are giving us 'I have no idea how this works, let's ship the prompt'. The early Docker community spirit really was legendary though—before every hyperscaler wrapped it in 7 layers of proprietary managed services. Thanks for the memories and the write-up!
  - avsm 7mo ago
    Thanks for the kind words! I've been prodding @justincormack to resurrect the single most fun OS unconference I've ever attended -- New Directions in Operating Systems (last held back in 2014). https://operatingsystems.io https://operatingsystems.io Some of those talks strangely make more sense today (e.g. Rump Kernels or unikernels + coding agents seems like a really good combination, as the agent could search all the way through the kernel layers as well).
  - supriyo-biswas 7mo ago
    It seems that your entire profile is LLM generated comments, would appreciate it if you'd stop. Thanks.
curt15 7mo ago
>'then we'll just ship your machine production' Minus the kernel of course. What is one to do for workloads requiring special kernel features or modules?
- avsm 7mo ago
  Those are global to the machine; generally not an issue and seccomp rules can filter out undesirable syscalls to other containers. But GPU kernel/userspace driver matching has been a huge headache; see https://cacm.acm.org/research/a-decade-of-docker-containers/#sec17 https://cacm.acm.org/research/a-decade-of-docker-containers/... in the article for how the CDI is (sort of) helping standardise this.
syncsynchalt 7mo ago
I see this take a lot but I'd argue what Docker did was to entice everyone to capture their build into a repeatable process (via a Dockerfile). "Ship your machine to production" isn't so bad when you have a ten-line script to recreate the machine at the push of a button.
- lioeters 7mo ago
  Exactly my feeling. Docker is "works on this machine" with an executable recipe to build the machine and the application. Newer better solutions like OCI-compliant tools will gradually replace Docker, but the paradigm shift has provided a lot of lasting value.
  - Gigachad 7mo ago
    Yeah docker codifies what the process to convert a base linux distro in to a working platform for the app actually is. Every company I've worked at that didn't use docker just has this tribal knowledge or an outdated wiki page on the steps you need to take to get something to work. Vs a dockerfile that exactly documents the process.
Skywalker13 7mo ago
Oh, thank you... I'm not alone... I'm so tired of seeing crappy containers with pseudo service management handled by Dockerfiles, used instead of proper and serious packaging like that of many venerable Linux distributions.
hwhshs 7mo ago
In 2002 I used to think why cant they package a website. These .doc installation instructions are insane! What a waste of someones time. I sort of had the problem in mind. Docker is the answer. Not clever enough to have inventer it. If I did I would probably have invented octopus deploy as I was a Microsoft/.NET guy.
siva7 7mo ago
I think it was all about efficient large-scale server machine reproducibility and not about making local development workflows easier. If it were i'm sure Docker would look much more friendly for that case but it still advanced to an industry-standard even for small software departments because everyone used it.

A full decade since we took the 'it works on my machine' excuse and turned it into the industry standard architecture ('then we'll just ship your machine to production').

redhanuman 7mo ago

the real trick was making "ship your machine" sound like best practice and ten years later we r doing the same thing with ai "it works in my notebook" jst became "containerize the notebook and call it a pipeline" the abstraction always wins because fixing the actual problem is just too hard.

goodpoint 7mo ago
...while completely forgetting about security
zbentley 7mo ago
> fixing the actual problem is just too hard. I think it’s laziness, not difficulty. That’s not meant to be snide or glib: I think gaining expertise in how to package and deploy non-containerized applications isn’t difficult or unattainable for most engineers; rather, it’s tedious and specialized work to gain that expertise, and Docker allowed much of the field to skip doing it. That’s not good or bad per se, but I do think it’s different from “pre-container deployment was hard”. Pre-container deployment was neglected and not widely recognized as a specialty that needed to be cultivated, so most shops sucked at it. That’s not the same as “hard”.
- skydhash 7mo ago
  It's not even laziness or expertise. A lot of people are against learning conventions. They want their way, meaning what works on their computer. That's why they like the current scope of package managers, docker, flatpack,... They can do what they want in the sandbox provided however nonsensical and then ship the whole thing. And it will break if you look at it the wrong way.
Bratmon 7mo ago
I mean, walking through a door is easier than tearing down a wall, walking through it, and rebuilding the wall. That doesn't mean the latter is a good idea.

forrestthewoods 7mo ago

Linux user space is an abject disaster of a design. So so so bad. Docker should not need to exist. Running computer programs need not be so difficult.

esafak 7mo ago
Who does it right?
- whateverboat 7mo ago
  Windows.
  - deleted 7mo ago
    [deleted]
- jjmarr 7mo ago
  Nix and Guix. Good luck convincing people to switch!
  - abacate 7mo ago
    Trying to convince people usually makes any resistance worse. Using it, solving problems with it, and building a real community around it tend to make a much greater impact in the long run.
    - NortySpock 7mo ago
      Yeah, but if the problem you are solving is rare for most practitioners, effectively theoretical until it actually happens, then people won't switch until they get bit by that particular problem.
  - zbentley 7mo ago
    But they’re roughly the same paradigm as docker, right? My understanding of the Nix approach is that it’s still reproducing most of a user land/filesystem in a captive/separate/sandbox environment. Like, docker is using namespaces for more stuff, Nix has a heavier emphasis on reproducibility/determinism, but … they’re both still throwing in the towel on deploying directly on the underlying OS’s userland (unless you go all the way to nixOS) and shipping what amounts to a filesystem in a box, no?

chuckadams 7mo ago

It's the ultimate in static linking. Perhaps a question that should be asked is why that approach is so compelling?

blackcatsec 7mo ago
I question that as well, it's also why Go is extremely popular. Could it just be a pendulum swing back towards static linking? Wonder when some enterprising OSS dev will rebrand dynamic linking in the future...
- jfjasdfuw 7mo ago
  CGO_ENABLED=0 is sigma tier. I don't care about glibc or compatibility with /etc/nsswitch.conf. look at the hack rust does because it uses libc: > pub unsafe fn set_var<K: AsRef<OsStr>, V: AsRef<OsStr>>(key: K, value: V)
  - jcgl 7mo ago
    > I don't care about glibc or compatibility with /etc/nsswitch.conf. So what do you do when you need to resolve system users? I sure hope you don't parse /etc/passwd, since plenty of users (me included) use other user databases (e.g. sssd or systemd-userdbd).
    - cyberax 7mo ago
      Most software doesn't need to resolve users. You also can always shell out to `id` if you need an occasional bit of metadata.
      - jcgl 7mo ago
        That's a fair point, and shelling out to id is probably a good solution. I guess what bothers me is the software authors who don't think this through, leaving applications non-functional in these situations. At least with Go, if you do CGO_ENABLED=0, and you use the stdlib functions to resolve user information, you end up with parsed /etc/passwd instead of shelling out to id. The Go stdlib should maybe shell out to id instead, but it doesn't. And it's understandable that software developers use the stdlib functions without thinking all too much about it. But in the end, simply advocating for CGO_ENABLED=0 results in software that is broken around the edges.

avsm 7mo ago

(coauthor of the article here) Well, before Docker I used to work on Xen and that possible future of massive block devices assembled using Vagrant and Packer has thankfully been avoided... One thing that's hard to capture in the article -- but that permeated the early Dockercons -- is the (positive) disruption Docker had in how IT shops were run. Before that going to production was a giant effort, and 'shipping your filesystem' quickly was such a change in how people approached their work. We had so many people come up to us grateful that they could suddenly build services more quickly and get them into the hands of users without having to seek permission slips signed in triplicate. We're seeing the another seismic cultural shift now with coding agents, but I think Docker had a similar impact back then, and it was a really fun community spirit. Less so today with the giant hyperscalars all dominating, sadly, but I'll keep my fond memories :-)

talkvoix 7mo ago
Great point about coding agents! Back then, Docker gave us 'it works on my machine, let's ship the machine'. Now, AI agents are giving us 'I have no idea how this works, let's ship the prompt'. The early Docker community spirit really was legendary though—before every hyperscaler wrapped it in 7 layers of proprietary managed services. Thanks for the memories and the write-up!
- avsm 7mo ago
  Thanks for the kind words! I've been prodding @justincormack to resurrect the single most fun OS unconference I've ever attended -- New Directions in Operating Systems (last held back in 2014). https://operatingsystems.io https://operatingsystems.io Some of those talks strangely make more sense today (e.g. Rump Kernels or unikernels + coding agents seems like a really good combination, as the agent could search all the way through the kernel layers as well).
- supriyo-biswas 7mo ago
  It seems that your entire profile is LLM generated comments, would appreciate it if you'd stop. Thanks.

curt15 7mo ago

>'then we'll just ship your machine production' Minus the kernel of course. What is one to do for workloads requiring special kernel features or modules?

avsm 7mo ago
Those are global to the machine; generally not an issue and seccomp rules can filter out undesirable syscalls to other containers. But GPU kernel/userspace driver matching has been a huge headache; see https://cacm.acm.org/research/a-decade-of-docker-containers/#sec17 https://cacm.acm.org/research/a-decade-of-docker-containers/... in the article for how the CDI is (sort of) helping standardise this.

syncsynchalt 7mo ago

I see this take a lot but I'd argue what Docker did was to entice everyone to capture their build into a repeatable process (via a Dockerfile). "Ship your machine to production" isn't so bad when you have a ten-line script to recreate the machine at the push of a button.

lioeters 7mo ago
Exactly my feeling. Docker is "works on this machine" with an executable recipe to build the machine and the application. Newer better solutions like OCI-compliant tools will gradually replace Docker, but the paradigm shift has provided a lot of lasting value.
- Gigachad 7mo ago
  Yeah docker codifies what the process to convert a base linux distro in to a working platform for the app actually is. Every company I've worked at that didn't use docker just has this tribal knowledge or an outdated wiki page on the steps you need to take to get something to work. Vs a dockerfile that exactly documents the process.

Skywalker13 7mo ago

Oh, thank you... I'm not alone... I'm so tired of seeing crappy containers with pseudo service management handled by Dockerfiles, used instead of proper and serious packaging like that of many venerable Linux distributions.

hwhshs 7mo ago

In 2002 I used to think why cant they package a website. These .doc installation instructions are insane! What a waste of someones time. I sort of had the problem in mind. Docker is the answer. Not clever enough to have inventer it. If I did I would probably have invented octopus deploy as I was a Microsoft/.NET guy.

siva7 7mo ago

I think it was all about efficient large-scale server machine reproducibility and not about making local development workflows easier. If it were i'm sure Docker would look much more friendly for that case but it still advanced to an industry-standard even for small software departments because everyone used it.