8 ms·

Red Hat to contribute container tech (Podman, bootc, ComposeFS...) to CNCF

272 points by twelvenmonkeys 2y ago

xer0x 2y ago
What took them so long?
- tecleandor 2y ago
  Waiting for IBM to buy them? (half joking)
  - cyberax 2y ago
    Or The Onion!
- nijave 2y ago
  My take was they sort of dug in said "Docker isn't made right for Linux, we're reinventing it" On Fedora w/ SELinux that led to quite a bit of compatibility issues for a while with lots of quirky things that didn't behave the same. I think their implementations have gotten pretty stable and improved in compatibility since then
  - 2OEH8eoCRo0 2y ago
    That's my take as well. Red Hat's design choices fit into Linux much more neatly. Docker has always been rubbish with late cgroups v2 support, punching holes in my firewall, no rootless, etc.
    - mmh0000 2y ago
      > punching holes in my firewall I teach various Linux training courses. One of which is Containers. It always shocks several people per-class how Docker just blatantly ignores and rewrites existing firewall rules. And there's no real option to prevent that unless you want to manually configure ALL network routing. For me personally, that was one of the big issues the pushed me over to Podman. Also, Docker's insistence on "forcing" and preventing the disabling of using the malware-ridden Docker Hub didn't help me appreciate their security practices.[] [] https://jfrog.com/blog/attacks-on-docker-with-millions-of-malicious-repositories-spread-malware-and-phishing-scams/ https://jfrog.com/blog/attacks-on-docker-with-millions-of-ma... https://www.infosecurity-magazine.com/news/malicious-containers-found-docker/ https://www.infosecurity-magazine.com/news/malicious-contain... https://www.bleepingcomputer.com/news/security/millions-of-docker-repos-found-pushing-malware-phishing-sites/ https://www.bleepingcomputer.com/news/security/millions-of-d... https://www.bleepingcomputer.com/news/security/docker-hub-repositories-hide-over-1-650-malicious-containers/ https://www.bleepingcomputer.com/news/security/docker-hub-re... https://sysdig.com/blog/analysis-of-supply-chain-attacks-through-public-docker-images/ https://sysdig.com/blog/analysis-of-supply-chain-attacks-thr... ... ETC ...
      - hughesjj 2y ago
        I want to switch to podman. What are the general gotchas and difficulties you could see in doing that for multi architecture+os builds/deployments?
      - xelamonster 2y ago
        You might just be convincing me to switch, I generally love docker and compose but the firewall thing still blows my mind and that there still just is not a solution. My workaround has been to bind all docker port forwards to localhost and only ever expose them externally via reverse proxy. Which is annoying because that means I can't run the reverse proxy itself in docker.
      - justinclift 2y ago
        > It always shocks several people per-class how Docker just blatantly ignores and rewrites existing firewall rules. Yeah. Many times I've mentioned that to people, and they just don't believe it's a thing which Docker does. Including here on HN. :/
NewJazz 2y ago
Does podman support docker compose files well? Devs love them for local environments.
- lytedev 2y ago
  I've been using podman-compose, yes.
- spockz 2y ago
  I’ve been using it on my Fedora server because I make myself. I think all functionality and syntax is covered. However, the user feedback and TUI of docker-compose is way nicer (interactive at least). Also, podman compose does seem to recreate containers that do not need to be recreated in more cases than I have noticed docker compose do.
  - jeppester 2y ago
    You can run the standalone version for docker-compose against podman. You just need to have the podman.socket systemd service running.
    - spockz 2y ago
      Yes indeed. I have that on other systems as well. But I try to keep both around to notice these kind of differences. (I’m working on tooling that relies on docker compose files so I like to see how it behaves in different setups.)
- jeppester 2y ago
  I use podman with docker-compose files for my day-to-day work; spinning up databases and other service dependencies for locally running or containerized webapps. podman-compose never worked very well for me, so I'm running with the podman.socket systemd service and the standalone version of docker-compose. That is however working flawlessly. What I really like about podman (and which to be fair docker might have since catched up on) is that rootless containers work so well. Gone are the days where bind-mounting a project folder into a container would mess with your file permissions. In my experience podman also feels easier and less invasive to install, although I can't say if the latter is really the case.
RcouF1uZ4gsC 2y ago
Reading about Keycloak and how long it is taking to patch critical vulnerabilities, I wonder is CNCF becoming how Apache was - where abandoned open source software goes to die.
- teepo 2y ago
  I think that CNCF has better handle on abandonware, plus really good observably. https://devstats.cncf.io/ https://devstats.cncf.io/
- pphysch 2y ago
  Hopefully the Keycloak thing will spur more competition. I looked at some alternatives and settled on Keycloak because it was "obviously" the mature and hardened solution. Well, clearly not.
  - LilBytes 2y ago
    Zitadel is worth a look, we're replacing Keycloak with it currently.
  - pratio 2y ago
    I've replaced keycloak with https://goauthentik.io/ https://goauthentik.io/ 2 years on, no complaints
- preisschild 2y ago
  I think being a CNCF project is better than not being one. It gives it more visibility and structure and thus is less likely to be abandoned. But sure, unfortunately if not enough different companies and individiuals are maintaining stuff it gets abandoned.
- caniszczyk 2y ago
  Last I checked, Keycloak has increased in activity since joining CNCF... https://keycloak.devstats.cncf.io/d/1/activity-repository-groups?orgId=1&from=now-2y&to=now https://keycloak.devstats.cncf.io/d/1/activity-repository-gr... CNCF has probably 20x the funding of the ASF and is a different organization that spends millions of dollars on security audits, events and more, you can read about it in our annual report: https://www.cncf.io/reports/cncf-annual-report-2023/ https://www.cncf.io/reports/cncf-annual-report-2023/ Also we actively remove/prune projects that aren't active... we will probably archive ~10 this year https://www.cncf.io/project-metrics/ https://www.cncf.io/project-metrics/
kuratkull 2y ago
Podman actually works really well. Out-of-the-box virtually-no-configuration-needed rootless containers. It's also usable via docker-compose with a single env variable. (podman-compose wasn't up to par for us) We've been using it for a couple of years running and managing hundreds of containers per server - no feeling of flakiness whatsoever. It's virtually zeroconf and even supports GPUs for those who need it. It's like docker but better, IMO. Hope it gets a popularity boost from CNCF. Rooting for it.
- jeppester 2y ago
  I completely agree and have had the same experience as you with docker-compose working better than the alternatives. Past versions of podman were flaky, but since version 4, which is now a couple of years old, I haven't had any issues whatsoever. I'd recommend anyone using containers on linux to try it out instead of installing docker out of habit.
- bombela 2y ago
  The IO through fuse-overlay is performance limiting though. It's almost half the speed as overlay directly for layers with many tiny files. Note that Linux allows you to mount overlay within a user namespace if you are root within the user namespace. In other words, if you are root within a container; even though it is not root on the host; Linux accepte ton mount overlay filesystems (most filesystems are not allowed). `man user_namespace`
  - nolist_policy 2y ago
    You may need to do podman system reset The Linux kernel only gained unprivileged overlay recently. Kernel fuse and fuse-overlay are incompatible so you need to wipe everything. You may need to set [storage] driver = "overlay" in storage conf as well. https://docs.podman.io/en/stable/markdown/podman-system-reset.1.html https://docs.podman.io/en/stable/markdown/podman-system-rese...
- dbacar 2y ago
  > docker-compose with a single env variable what is that env variable?
dbacar 2y ago
To all those interested in podman, this book by Daniel Walsh is a gem. Highly recommended and it is free. https://developers.redhat.com/e-books/podman-action https://developers.redhat.com/e-books/podman-action
- duckmysick 2y ago
  Important to note, this book is from early 2023 and supports Podman version 4.1. It's missing newer features like quadlets. https://www.redhat.com/en/blog/quadlet-podman https://www.redhat.com/en/blog/quadlet-podman
  - msgilligan 2y ago
    Yes. Definitely already needs a second edition. I would happily buy it.
- 5d41402abc4b 2y ago
  The font on that book is so awful. As someone with bad sight i have to strain my eyes a lot to read even a single sentence.
- neitsab 2y ago
  Let me present you a full 212-page RHEL 9 docs PDF on everything Podman, updated as of this month: https://docs.redhat.com/en-us/documentation/red_hat_enterprise_linux/9/pdf/building_running_and_managing_containers/Red_Hat_Enterprise_Linux-9-Building_running_and_managing_containers-en-US.pdf https://docs.redhat.com/en-us/documentation/red_hat_enterpri...
vbezhenar 2y ago
Is CNCF new Apache foundation? Looks like everyone dumps their stuff there. Does not look promising. Am I missing something? Probably RedHat paid salary to podman developers, but who will pay salary to them now?
- EdwardDiego 2y ago
  Strimzi is CNCF, Strimzi still has a full time team of devs in RH.
- anticorporate 2y ago
  I'm sure Red Hat will continue to pay Podman developers, just like they continue to pay developers for the other upstream projects that are hosted at CNCF (like Kubernetes). I'm we can all think of some projects "abandoned" to foundations through the years, but in general, I'd call getting core infrastructure out of the control of a single company and into a place with more transparent and democratic governance a good thing.
  - cryptos 2y ago
    Maybe open source foundations should be more selective with the projects. I'm thinking of Oracle dumping OpenOffice, although it was obvious that LibreOffice is the way to go.
- lysace 2y ago
  In the same way as how the Kubernetes ecosystem is the new Enterprise Java ecosystem? Often even from the same companies as back in the late 90s/00s. Look: I'm probably ignorant, but from the outside the similarities seem striking. Please explain why I'm wrong. I'm humble on this one.
  - elzbardico 2y ago
    Kubernetes is the new websphere. Yaml the new ANT.
- nonameiguess 2y ago
greatgib 2y ago
Usually, when big orgs like that dump their projects to such a foundation (like Apache), it is that they are about to drop investing in support it soon.
j1mc 2y ago
I think people are missing the contribution of bootc and composefs. This is a big part of what undergirds Red Hat's new 'image mode' means of deployment. They're using container-related tooling to deploy whole operating systems, and it's a large part of where they're headed. I write this to say, "This is not them dumping abandonware." To me, it's them putting these technologies under the supervision of a neutral third party to encourage adoption.
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- jauntywundrkind 2y ago
  Composefs has a totally crucial feature that sold me immediately, which is that if two containers use the same file, the kernel can serve that file from the same page cache. This means you can hypothetically launch a lot of containers, and if they share some layers/base images, the memory usage can still be quite reasonable. Nice. https://github.com/containers/composefs?tab=readme-ov-file#backing-store-shared-on-disk-and-in-page-cache https://github.com/containers/composefs?tab=readme-ov-file#b...
  - ChocolateGod 2y ago
    If you swap to a content-based object store for container images (like OSTree/Flatpak), you could potentially save a lot of disk space too because you'll no longer need to be careful about your Dockerfile layers. If two OCI images share the same file, they'll be de-duplicated on disk and only be downloaded once.
- deleted 2y ago
  [deleted]
gigatexal 2y ago
This is cool and all I just want to make sure podman and others are maintained and useful. I’m sure they will be it’s just that I use podman every day and depend on it. I could go back to docker but why?
philipwhiuk 2y ago
Hmm maybe worth switching from Docker Desktop to Podman Desktop...
deleted 2y ago
[deleted]

xer0x 2y ago

What took them so long?

tecleandor 2y ago
Waiting for IBM to buy them? (half joking)
- cyberax 2y ago
  Or The Onion!
nijave 2y ago
My take was they sort of dug in said "Docker isn't made right for Linux, we're reinventing it" On Fedora w/ SELinux that led to quite a bit of compatibility issues for a while with lots of quirky things that didn't behave the same. I think their implementations have gotten pretty stable and improved in compatibility since then
- 2OEH8eoCRo0 2y ago
  That's my take as well. Red Hat's design choices fit into Linux much more neatly. Docker has always been rubbish with late cgroups v2 support, punching holes in my firewall, no rootless, etc.
  - mmh0000 2y ago
    > punching holes in my firewall I teach various Linux training courses. One of which is Containers. It always shocks several people per-class how Docker just blatantly ignores and rewrites existing firewall rules. And there's no real option to prevent that unless you want to manually configure ALL network routing. For me personally, that was one of the big issues the pushed me over to Podman. Also, Docker's insistence on "forcing" and preventing the disabling of using the malware-ridden Docker Hub didn't help me appreciate their security practices.[] [] https://jfrog.com/blog/attacks-on-docker-with-millions-of-malicious-repositories-spread-malware-and-phishing-scams/ https://jfrog.com/blog/attacks-on-docker-with-millions-of-ma... https://www.infosecurity-magazine.com/news/malicious-containers-found-docker/ https://www.infosecurity-magazine.com/news/malicious-contain... https://www.bleepingcomputer.com/news/security/millions-of-docker-repos-found-pushing-malware-phishing-sites/ https://www.bleepingcomputer.com/news/security/millions-of-d... https://www.bleepingcomputer.com/news/security/docker-hub-repositories-hide-over-1-650-malicious-containers/ https://www.bleepingcomputer.com/news/security/docker-hub-re... https://sysdig.com/blog/analysis-of-supply-chain-attacks-through-public-docker-images/ https://sysdig.com/blog/analysis-of-supply-chain-attacks-thr... ... ETC ...
    - hughesjj 2y ago
      I want to switch to podman. What are the general gotchas and difficulties you could see in doing that for multi architecture+os builds/deployments?
    - xelamonster 2y ago
      You might just be convincing me to switch, I generally love docker and compose but the firewall thing still blows my mind and that there still just is not a solution. My workaround has been to bind all docker port forwards to localhost and only ever expose them externally via reverse proxy. Which is annoying because that means I can't run the reverse proxy itself in docker.
    - justinclift 2y ago
      > It always shocks several people per-class how Docker just blatantly ignores and rewrites existing firewall rules. Yeah. Many times I've mentioned that to people, and they just don't believe it's a thing which Docker does. Including here on HN. :/

NewJazz 2y ago

Does podman support docker compose files well? Devs love them for local environments.

lytedev 2y ago
I've been using podman-compose, yes.
spockz 2y ago
I’ve been using it on my Fedora server because I make myself. I think all functionality and syntax is covered. However, the user feedback and TUI of docker-compose is way nicer (interactive at least). Also, podman compose does seem to recreate containers that do not need to be recreated in more cases than I have noticed docker compose do.
- jeppester 2y ago
  You can run the standalone version for docker-compose against podman. You just need to have the podman.socket systemd service running.
  - spockz 2y ago
    Yes indeed. I have that on other systems as well. But I try to keep both around to notice these kind of differences. (I’m working on tooling that relies on docker compose files so I like to see how it behaves in different setups.)
jeppester 2y ago
I use podman with docker-compose files for my day-to-day work; spinning up databases and other service dependencies for locally running or containerized webapps. podman-compose never worked very well for me, so I'm running with the podman.socket systemd service and the standalone version of docker-compose. That is however working flawlessly. What I really like about podman (and which to be fair docker might have since catched up on) is that rootless containers work so well. Gone are the days where bind-mounting a project folder into a container would mess with your file permissions. In my experience podman also feels easier and less invasive to install, although I can't say if the latter is really the case.

RcouF1uZ4gsC 2y ago

Reading about Keycloak and how long it is taking to patch critical vulnerabilities, I wonder is CNCF becoming how Apache was - where abandoned open source software goes to die.

teepo 2y ago
I think that CNCF has better handle on abandonware, plus really good observably. https://devstats.cncf.io/ https://devstats.cncf.io/
pphysch 2y ago
Hopefully the Keycloak thing will spur more competition. I looked at some alternatives and settled on Keycloak because it was "obviously" the mature and hardened solution. Well, clearly not.
- LilBytes 2y ago
  Zitadel is worth a look, we're replacing Keycloak with it currently.
- pratio 2y ago
  I've replaced keycloak with https://goauthentik.io/ https://goauthentik.io/ 2 years on, no complaints
preisschild 2y ago
I think being a CNCF project is better than not being one. It gives it more visibility and structure and thus is less likely to be abandoned. But sure, unfortunately if not enough different companies and individiuals are maintaining stuff it gets abandoned.
caniszczyk 2y ago
Last I checked, Keycloak has increased in activity since joining CNCF... https://keycloak.devstats.cncf.io/d/1/activity-repository-groups?orgId=1&from=now-2y&to=now https://keycloak.devstats.cncf.io/d/1/activity-repository-gr... CNCF has probably 20x the funding of the ASF and is a different organization that spends millions of dollars on security audits, events and more, you can read about it in our annual report: https://www.cncf.io/reports/cncf-annual-report-2023/ https://www.cncf.io/reports/cncf-annual-report-2023/ Also we actively remove/prune projects that aren't active... we will probably archive ~10 this year https://www.cncf.io/project-metrics/ https://www.cncf.io/project-metrics/

kuratkull 2y ago

Podman actually works really well. Out-of-the-box virtually-no-configuration-needed rootless containers. It's also usable via docker-compose with a single env variable. (podman-compose wasn't up to par for us) We've been using it for a couple of years running and managing hundreds of containers per server - no feeling of flakiness whatsoever. It's virtually zeroconf and even supports GPUs for those who need it. It's like docker but better, IMO. Hope it gets a popularity boost from CNCF. Rooting for it.

jeppester 2y ago
I completely agree and have had the same experience as you with docker-compose working better than the alternatives. Past versions of podman were flaky, but since version 4, which is now a couple of years old, I haven't had any issues whatsoever. I'd recommend anyone using containers on linux to try it out instead of installing docker out of habit.
bombela 2y ago
The IO through fuse-overlay is performance limiting though. It's almost half the speed as overlay directly for layers with many tiny files. Note that Linux allows you to mount overlay within a user namespace if you are root within the user namespace. In other words, if you are root within a container; even though it is not root on the host; Linux accepte ton mount overlay filesystems (most filesystems are not allowed). `man user_namespace`
- nolist_policy 2y ago
  You may need to do podman system reset The Linux kernel only gained unprivileged overlay recently. Kernel fuse and fuse-overlay are incompatible so you need to wipe everything. You may need to set [storage] driver = "overlay" in storage conf as well. https://docs.podman.io/en/stable/markdown/podman-system-reset.1.html https://docs.podman.io/en/stable/markdown/podman-system-rese...
dbacar 2y ago
> docker-compose with a single env variable what is that env variable?

dbacar 2y ago

To all those interested in podman, this book by Daniel Walsh is a gem. Highly recommended and it is free. https://developers.redhat.com/e-books/podman-action https://developers.redhat.com/e-books/podman-action

duckmysick 2y ago
Important to note, this book is from early 2023 and supports Podman version 4.1. It's missing newer features like quadlets. https://www.redhat.com/en/blog/quadlet-podman https://www.redhat.com/en/blog/quadlet-podman
- msgilligan 2y ago
  Yes. Definitely already needs a second edition. I would happily buy it.
5d41402abc4b 2y ago
The font on that book is so awful. As someone with bad sight i have to strain my eyes a lot to read even a single sentence.
neitsab 2y ago
Let me present you a full 212-page RHEL 9 docs PDF on everything Podman, updated as of this month: https://docs.redhat.com/en-us/documentation/red_hat_enterprise_linux/9/pdf/building_running_and_managing_containers/Red_Hat_Enterprise_Linux-9-Building_running_and_managing_containers-en-US.pdf https://docs.redhat.com/en-us/documentation/red_hat_enterpri...

vbezhenar 2y ago

Is CNCF new Apache foundation? Looks like everyone dumps their stuff there. Does not look promising. Am I missing something? Probably RedHat paid salary to podman developers, but who will pay salary to them now?

EdwardDiego 2y ago
Strimzi is CNCF, Strimzi still has a full time team of devs in RH.
anticorporate 2y ago
I'm sure Red Hat will continue to pay Podman developers, just like they continue to pay developers for the other upstream projects that are hosted at CNCF (like Kubernetes). I'm we can all think of some projects "abandoned" to foundations through the years, but in general, I'd call getting core infrastructure out of the control of a single company and into a place with more transparent and democratic governance a good thing.
- cryptos 2y ago
  Maybe open source foundations should be more selective with the projects. I'm thinking of Oracle dumping OpenOffice, although it was obvious that LibreOffice is the way to go.
lysace 2y ago
In the same way as how the Kubernetes ecosystem is the new Enterprise Java ecosystem? Often even from the same companies as back in the late 90s/00s. Look: I'm probably ignorant, but from the outside the similarities seem striking. Please explain why I'm wrong. I'm humble on this one.
- elzbardico 2y ago
  Kubernetes is the new websphere. Yaml the new ANT.
nonameiguess 2y ago

greatgib 2y ago

Usually, when big orgs like that dump their projects to such a foundation (like Apache), it is that they are about to drop investing in support it soon.

j1mc 2y ago

I think people are missing the contribution of bootc and composefs. This is a big part of what undergirds Red Hat's new 'image mode' means of deployment. They're using container-related tooling to deploy whole operating systems, and it's a large part of where they're headed. I write this to say, "This is not them dumping abandonware." To me, it's them putting these technologies under the supervision of a neutral third party to encourage adoption.

deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
jauntywundrkind 2y ago
Composefs has a totally crucial feature that sold me immediately, which is that if two containers use the same file, the kernel can serve that file from the same page cache. This means you can hypothetically launch a lot of containers, and if they share some layers/base images, the memory usage can still be quite reasonable. Nice. https://github.com/containers/composefs?tab=readme-ov-file#backing-store-shared-on-disk-and-in-page-cache https://github.com/containers/composefs?tab=readme-ov-file#b...
- ChocolateGod 2y ago
  If you swap to a content-based object store for container images (like OSTree/Flatpak), you could potentially save a lot of disk space too because you'll no longer need to be careful about your Dockerfile layers. If two OCI images share the same file, they'll be de-duplicated on disk and only be downloaded once.
deleted 2y ago
[deleted]

gigatexal 2y ago

This is cool and all I just want to make sure podman and others are maintained and useful. I’m sure they will be it’s just that I use podman every day and depend on it. I could go back to docker but why?

philipwhiuk 2y ago

Hmm maybe worth switching from Docker Desktop to Podman Desktop...

deleted 2y ago

[deleted]