12 ms·

Using Podman, Compose and BuildKit

311 points by LaSombra 1y ago

cyprien_g 1y ago
I have never used Podman, but I hear about it very often. I currently use Colima (https://github.com/abiosoft/colima https://github.com/abiosoft/colima) and I'm very satisfied with it. I have tried to find a good comparison between the two, but I find it hard to have a clear opinion on which one is best for me.
- rockyj 1y ago
  Colima is related but separate, Colima runs a Linux VM seamlessly in MacOS so you can interact with Docker as if it was running on your host OS. Podman will also need a VM (coz there is no native Docker on MacOS).
  - dizhn 1y ago
    That sounded like podman needs docker. It's more of a mostly compatible separate product that does not need a daemon to run. Both probably lack things like cgroups (a Linux kernel really) etc on macos.
  - privatelypublic 1y ago
    Podman Desktop uses a VM on windows too.(WSL2)
- cmiles74 1y ago
  I use Podman, I wanted a Docker-like experience on MacOS or Windows. I use it to provide services during development (PostgreSQL, etc.) and working on image builds. It works pretty well. :-)
lloydjones 1y ago
I encountered this exact issue (no buildkit with Podman) months ago. I gave up and used Docker desktop, but I'm glad you didn't. Well done!
haolez 1y ago
But Docker can run rootless these days, right? Can anyone chime in with how well does this work?
- pxc 1y ago
  Actually setting up rootless Docker is pretty manual and kludgy. With Podman, rootless is pretty much effortless.
  - haolez 1y ago
    Wow. It seems really clunky indeed: https://docs.docker.com/engine/security/rootless/ https://docs.docker.com/engine/security/rootless/
    - WhyNotHugo 1y ago
      Check your distribution's documentation for the easy way.
- WhyNotHugo 1y ago
  Been using it without any issues for years. Very few distros ship the extras required for it. If yours doesn't, you'll have some extra manual steps to set it up.
jabl 1y ago
If you're not wedded to docker-compose, with podman you can instead use the podman kube support, which provides roughly docker-compose equivalent features using a subset of the Kubernetes pod deployment syntax. Additionally, podman has nice systemd integration for such kube services, you just need to write a short systemd config snippet and then you can manage the kube service just like any other systemd service. Altogether a very nice combination for deploying containerized services if you don't want to go the whole hog to something like Kubernetes.
- GCUMstlyHarmls 1y ago
  (I'm a big podman stan) Last I tried using the .kube files I ran into issues with specifying container networks (https://github.com/containers/podman/issues/12965 https://github.com/containers/podman/issues/12965). This is sort of "fixed" by using a Quadlet ".kube" but IMO that's a pretty weak solution and removes the "here's your compose file, run it" aspect. Recently (now that Deb13 is out with Podman 5) I have started transitioning to Podmans Quadlet files which have been quite smooth so far. As you say, its great to run things without all the overhead of kubernetes.
  - teekert 1y ago
    "...removes the "here's your compose file, run it" Claude recently hallucinated this for me: [Container] ComposeService=my-service Yaml=/path/to/your/podman-compose.yaml For a brief moment in time I was happy but then: Can you really use "ComposeService" in the systemd unit file? I can't find any reference to it You're absolutely right to question that - I made an error. There is no ComposeService directive in systemd or Quadlet. It would be a nice best of both worlds...
    - tkiolp4 1y ago
      It’s exhausting. As someone who doesn’t work with systemd, I would have a hard time using llms for this topic.
      - 1y ago
mstade 1y ago
I replaced my Docker usage entirely with OrbStack[1] a few months ago, and have had zero issues with it so far. Great product that I happily pay a license for. My usage is fairly basic though and I'm sure mileage varies, but for my basic web dev setup it's been perfect. [1]: https://orbstack.dev/ https://orbstack.dev/
- jFriedensreich 1y ago
  orbstack is just a vm provider for docker on mac, colima offers the same features without a ui and is a great open replacement but as neither supports podman both are not really relevant to the podman discussion.
  - nicce 1y ago
    > orbstack is just a vm provider for docker on mac ”just” is a big statement here. Performance between colima and OrbStack are from different planets. Apple just released their own runtime so that is also worth inspecting.
    - jFriedensreich 1y ago
      can you back that claim up? i see a huge difference between orbstack and docker desktop but colima and orbstack use afaik the same technology and the performance was near identical in my tests. (Though you need to change the colima settings to vz and virtiofs)
      - nicce 1y ago
        > (Though you need to change the colima settings to vz and virtiofs) I think I have just used the defaults. The difference was huge in regular use. E.g. simple test to upgrade OS packages and time that.
    - blablabla123 1y ago
      I haven't used OrbStack in a while but would you say Colima or OrbStack is faster? At least on Intel Mac Colima is for me way better than Docker. Also better than Podman in terms of compatibility, although I had to switch back to Docker Desktop since I need full compat.
gz09 1y ago
Unfortunately, it's quite a big mess (as the article indicates), which leads to a steep learning curve for someone who "just wants to build some images". And that's just half of it. Want to build an image on two native architectures (ARM64 and AMD64) and then make a multi-arch image out of them. Might blow someones mind on how complicated that is with 2025 docker technologies: https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners https://docs.docker.com/build/ci/github-actions/multi-platfo...
- figmert 1y ago
  If you use qemu, you don't need a complicated multi-node build system. Obviously performance becomes a factor then.
smjburton 1y ago
This is an interesting find OP and could help people transition from Docker to Podman (especially if they're used to deploying with Docker-Compose). I think the better long-term approach though is to use systemd user units for deployment, or the more modern approach of using Podman Quadlets. There's a bit of a learning curve, but these approaches are more native to the Podman platform, and learning how systemd services work is a great skill to have.
lukaslalinsky 1y ago
I was a huge fan of Podman, but I eventually gave up and use Docker Compose for local development. It's not worth fighting the system. However, for single server deployments, where I don't need Kubernetes, I now exclusively use Quadlets to run apps and I couldn't be happier. It's a much nicer experience that using typical Docker/Podman setup. It feels integrated into the system.
- debugnik 1y ago
  > I was a huge fan of Podman, but I eventually gave up and use Docker Compose You can mix them. I was using docker-compose with podman instead of docker before switching to quadlets. I still prefer the experience of compose files, but quadlets do integrate much better into systemd.
heavensteeth 1y ago
I tried Podman on my messing around VPS but quickly reverted to rootless Docker. The straw that broke the camels back was a bug in `podman compose` that funnily enough was fixed two hours ago[1]; if `service1` has a `depends_on` on `service2`, bringing down `service1` will unconditionally bring down `service2`, even if other services also depend on it. So if two separate services depend on a database, killing one of them will kill the database too. Another incompatibility with Docker I experienced was raised in 2020 and fixed a few months ago[2]; you couldn't pass URLs to `build:` to automatically pull and build images. The patch for this turned out to be a few lines long. I'm sure Podman will be great once all of these bugs are ironed out, but for me, it's not quite there yet. [1]: https://github.com/containers/podman-compose/pull/1283 https://github.com/containers/podman-compose/pull/1283 [2]: https://github.com/containers/podman-compose/issues/127 https://github.com/containers/podman-compose/issues/127
- bogwog 1y ago
  Podman compose is an attempt to court Docker users by porting over a bad idea. Instead of that, learn how to create "quadlets" and you'll never want to touch docker again. See: https://www.redhat.com/en/blog/quadlet-podman https://www.redhat.com/en/blog/quadlet-podman I recommend starting with .container files instead of .kube, unless you're already familiar with kubernetes.
  - c-hendricks 1y ago
    So for my set of DVR services, quadlets would have me replace a single compose.yml with 6 .container files, and manually create the network, and have to stop and start all of the services individually. Not sure I'm sold.
    - bogwog 1y ago
      Not sure what your compose file looks like, but my container files are tiny, flat, and trivial to maintain. > manually create the network There's no way for me to know what your requirements are, but often times if you just need your containers to talk to each other, all you need is an empty file with a unique name. So `touch MyDVRNetwork.network` to create it, and add `Network=MyDVRNetwork` to your containerfiles. > and have to stop and start all of the services individually. Nope, container files are essentially already systemd service files. If you add them to the correct folder and set up the dependencies, systemd will automatically start them in the correct order at boot time, restart them if they fail, etc. That's the best part of quadlet IMO. Literally set it and forget it, and the process works the same for rootless containers (you just need to add them to your user folder instead of the system-wide folder) It gets even more awesome when you combine them with something like Fedora CoreOS and Butane. With a few small text files, you can declaratively generate an OS image with all of your desired services ready to go. It is pure bliss.
zoobab 1y ago
I use a special feature of Docker Buildx with a multistage Dockerfile build, with the SDK in the first part, and an empty container in the second part to copy the build artifact (like firmware.bin), like explained here: https://docs.docker.com/build/building/export/ https://docs.docker.com/build/building/export/ Does Podman supports the same feature?
- mkesper 1y ago
  Yes, since 2021 they also support the same syntax natively: https://github.com/containers/buildah/pull/3133 https://github.com/containers/buildah/pull/3133 While also supporting building Dockerfiles with buildah: https://github.com/containers/buildah/blob/main/docs/tutorials/01-intro.md#using-containerfilesdockerfiles-with-buildah https://github.com/containers/buildah/blob/main/docs/tutoria...
rsyring 1y ago
I've wanted to try Podman for a couple years. But I keep bowing out because there are no official Ubuntu builds that I could find. Upstream seems content with that status quo. That's their prerogative, and I could build it myself, but it makes me concerned they don't really have multi-distro compatibility as a priority, which makes me hesitant to commit time to experimenting with it when Docker considers Ubuntu a first-class citizen.
- bityard 1y ago
  Huh? It's right there in the docs: https://podman.io/docs/installation#installing-on-linux https://podman.io/docs/installation#installing-on-linux apt-get -y install podman Podman doesn't produce their own binary distributions for ANY Linux distro. The only binary packages they provide are for Mac and Windows because those don't have a native package repository. Even if they did release their own Linux packages, bear in mind that Podman development is driven by and sponsored almost entirely by Red Hat. It's not really in their interest to pay their developers to maintain packages for Ubuntu, a direct competitor in the enterprise Linux space.
  - figmert 1y ago
    The problem is podman in the apt repos are very old. Last time I checked, Podman was on v5, where debian was on v2 something.
    - saint_yossarian 1y ago
      Must have been ages ago, even stable now has 5.4: https://packages.debian.org/search?searchon=names&suite=all&section=all&keywords=podman https://packages.debian.org/search?searchon=names&suite=all&...
      - wink 1y ago
        "even stable" - that was released 2w ago. ... Also yes, podman v4 on bookwarm was famously useless in many cases and because of either libc or kernel (iirc) you could not even install v5 effortlessly. I like Debian and I like podman but putting this as a usefule nice experience (up until trixie released) is just weird framing.
digikata 1y ago
On Linux I'm using colima with docker compose and buildx and it seems to work ok for my limited cases. On Mac it works ok to, but there are networking cases that Colima on mac doesn't handle - so orbstack for there
leemelone 1y ago
podman pissed me off when I tried it in it's early days. Is it better?
evantbyrne 1y ago
It's not clear from the article, but is this for local development or production deployments? Because it's worth noting that Swarm solves a lot of the limitations that Compose and Podman have for running containers in a production environment. Swarm runs well on singular vms and people with Docker experience can learn the ropes in a day.
tracker1 1y ago
I've mostly just stuck to Docker myself... that said, podman definitely has some curiousity from me, just haven't taken the time to learn. One enhancement suggestion to TFA, would be to take a hash of the compose file path, then have that as a prefix name in a temp directory for a hash of the compose file itself... if the hash changes, dump the .json and rebuild in the temp path... then do the bakah against that file. This would be an easy enough script to make.
deknos 1y ago
i am always missing podman(-compose) cp as docker(-compose) cp :( That was so convenient.
atothayu 1y ago
clean site

311 points by LaSombra 1y ago

I have never used Podman, but I hear about it very often. I currently use Colima (https://github.com/abiosoft/colima https://github.com/abiosoft/colima) and I'm very satisfied with it. I have tried to find a good comparison between the two, but I find it hard to have a clear opinion on which one is best for me.

rockyj 1y ago
Colima is related but separate, Colima runs a Linux VM seamlessly in MacOS so you can interact with Docker as if it was running on your host OS. Podman will also need a VM (coz there is no native Docker on MacOS).
- dizhn 1y ago
  That sounded like podman needs docker. It's more of a mostly compatible separate product that does not need a daemon to run. Both probably lack things like cgroups (a Linux kernel really) etc on macos.
- privatelypublic 1y ago
  Podman Desktop uses a VM on windows too.(WSL2)
cmiles74 1y ago
I use Podman, I wanted a Docker-like experience on MacOS or Windows. I use it to provide services during development (PostgreSQL, etc.) and working on image builds. It works pretty well. :-)

lloydjones 1y ago

I encountered this exact issue (no buildkit with Podman) months ago. I gave up and used Docker desktop, but I'm glad you didn't. Well done!

haolez 1y ago

But Docker can run rootless these days, right? Can anyone chime in with how well does this work?

pxc 1y ago
Actually setting up rootless Docker is pretty manual and kludgy. With Podman, rootless is pretty much effortless.
- haolez 1y ago
  Wow. It seems really clunky indeed: https://docs.docker.com/engine/security/rootless/ https://docs.docker.com/engine/security/rootless/
  - WhyNotHugo 1y ago
    Check your distribution's documentation for the easy way.
WhyNotHugo 1y ago
Been using it without any issues for years. Very few distros ship the extras required for it. If yours doesn't, you'll have some extra manual steps to set it up.

jabl 1y ago

If you're not wedded to docker-compose, with podman you can instead use the podman kube support, which provides roughly docker-compose equivalent features using a subset of the Kubernetes pod deployment syntax. Additionally, podman has nice systemd integration for such kube services, you just need to write a short systemd config snippet and then you can manage the kube service just like any other systemd service. Altogether a very nice combination for deploying containerized services if you don't want to go the whole hog to something like Kubernetes.

GCUMstlyHarmls 1y ago
(I'm a big podman stan) Last I tried using the .kube files I ran into issues with specifying container networks (https://github.com/containers/podman/issues/12965 https://github.com/containers/podman/issues/12965). This is sort of "fixed" by using a Quadlet ".kube" but IMO that's a pretty weak solution and removes the "here's your compose file, run it" aspect. Recently (now that Deb13 is out with Podman 5) I have started transitioning to Podmans Quadlet files which have been quite smooth so far. As you say, its great to run things without all the overhead of kubernetes.
- teekert 1y ago
  "...removes the "here's your compose file, run it" Claude recently hallucinated this for me: [Container] ComposeService=my-service Yaml=/path/to/your/podman-compose.yaml For a brief moment in time I was happy but then: Can you really use "ComposeService" in the systemd unit file? I can't find any reference to it You're absolutely right to question that - I made an error. There is no ComposeService directive in systemd or Quadlet. It would be a nice best of both worlds...
  - tkiolp4 1y ago
    It’s exhausting. As someone who doesn’t work with systemd, I would have a hard time using llms for this topic.
    - 1y ago

mstade 1y ago

I replaced my Docker usage entirely with OrbStack[1] a few months ago, and have had zero issues with it so far. Great product that I happily pay a license for. My usage is fairly basic though and I'm sure mileage varies, but for my basic web dev setup it's been perfect. [1]: https://orbstack.dev/ https://orbstack.dev/

jFriedensreich 1y ago
orbstack is just a vm provider for docker on mac, colima offers the same features without a ui and is a great open replacement but as neither supports podman both are not really relevant to the podman discussion.
- nicce 1y ago
  > orbstack is just a vm provider for docker on mac ”just” is a big statement here. Performance between colima and OrbStack are from different planets. Apple just released their own runtime so that is also worth inspecting.
  - jFriedensreich 1y ago
    can you back that claim up? i see a huge difference between orbstack and docker desktop but colima and orbstack use afaik the same technology and the performance was near identical in my tests. (Though you need to change the colima settings to vz and virtiofs)
    - nicce 1y ago
      > (Though you need to change the colima settings to vz and virtiofs) I think I have just used the defaults. The difference was huge in regular use. E.g. simple test to upgrade OS packages and time that.
  - blablabla123 1y ago
    I haven't used OrbStack in a while but would you say Colima or OrbStack is faster? At least on Intel Mac Colima is for me way better than Docker. Also better than Podman in terms of compatibility, although I had to switch back to Docker Desktop since I need full compat.

gz09 1y ago

Unfortunately, it's quite a big mess (as the article indicates), which leads to a steep learning curve for someone who "just wants to build some images". And that's just half of it. Want to build an image on two native architectures (ARM64 and AMD64) and then make a multi-arch image out of them. Might blow someones mind on how complicated that is with 2025 docker technologies: https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners https://docs.docker.com/build/ci/github-actions/multi-platfo...

figmert 1y ago
If you use qemu, you don't need a complicated multi-node build system. Obviously performance becomes a factor then.

smjburton 1y ago

This is an interesting find OP and could help people transition from Docker to Podman (especially if they're used to deploying with Docker-Compose). I think the better long-term approach though is to use systemd user units for deployment, or the more modern approach of using Podman Quadlets. There's a bit of a learning curve, but these approaches are more native to the Podman platform, and learning how systemd services work is a great skill to have.

lukaslalinsky 1y ago

I was a huge fan of Podman, but I eventually gave up and use Docker Compose for local development. It's not worth fighting the system. However, for single server deployments, where I don't need Kubernetes, I now exclusively use Quadlets to run apps and I couldn't be happier. It's a much nicer experience that using typical Docker/Podman setup. It feels integrated into the system.

debugnik 1y ago
> I was a huge fan of Podman, but I eventually gave up and use Docker Compose You can mix them. I was using docker-compose with podman instead of docker before switching to quadlets. I still prefer the experience of compose files, but quadlets do integrate much better into systemd.

heavensteeth 1y ago

I tried Podman on my messing around VPS but quickly reverted to rootless Docker. The straw that broke the camels back was a bug in `podman compose` that funnily enough was fixed two hours ago[1]; if `service1` has a `depends_on` on `service2`, bringing down `service1` will unconditionally bring down `service2`, even if other services also depend on it. So if two separate services depend on a database, killing one of them will kill the database too. Another incompatibility with Docker I experienced was raised in 2020 and fixed a few months ago[2]; you couldn't pass URLs to `build:` to automatically pull and build images. The patch for this turned out to be a few lines long. I'm sure Podman will be great once all of these bugs are ironed out, but for me, it's not quite there yet. [1]: https://github.com/containers/podman-compose/pull/1283 https://github.com/containers/podman-compose/pull/1283 [2]: https://github.com/containers/podman-compose/issues/127 https://github.com/containers/podman-compose/issues/127

bogwog 1y ago
Podman compose is an attempt to court Docker users by porting over a bad idea. Instead of that, learn how to create "quadlets" and you'll never want to touch docker again. See: https://www.redhat.com/en/blog/quadlet-podman https://www.redhat.com/en/blog/quadlet-podman I recommend starting with .container files instead of .kube, unless you're already familiar with kubernetes.
- c-hendricks 1y ago
  So for my set of DVR services, quadlets would have me replace a single compose.yml with 6 .container files, and manually create the network, and have to stop and start all of the services individually. Not sure I'm sold.
  - bogwog 1y ago
    Not sure what your compose file looks like, but my container files are tiny, flat, and trivial to maintain. > manually create the network There's no way for me to know what your requirements are, but often times if you just need your containers to talk to each other, all you need is an empty file with a unique name. So `touch MyDVRNetwork.network` to create it, and add `Network=MyDVRNetwork` to your containerfiles. > and have to stop and start all of the services individually. Nope, container files are essentially already systemd service files. If you add them to the correct folder and set up the dependencies, systemd will automatically start them in the correct order at boot time, restart them if they fail, etc. That's the best part of quadlet IMO. Literally set it and forget it, and the process works the same for rootless containers (you just need to add them to your user folder instead of the system-wide folder) It gets even more awesome when you combine them with something like Fedora CoreOS and Butane. With a few small text files, you can declaratively generate an OS image with all of your desired services ready to go. It is pure bliss.

zoobab 1y ago

I use a special feature of Docker Buildx with a multistage Dockerfile build, with the SDK in the first part, and an empty container in the second part to copy the build artifact (like firmware.bin), like explained here: https://docs.docker.com/build/building/export/ https://docs.docker.com/build/building/export/ Does Podman supports the same feature?

mkesper 1y ago
Yes, since 2021 they also support the same syntax natively: https://github.com/containers/buildah/pull/3133 https://github.com/containers/buildah/pull/3133 While also supporting building Dockerfiles with buildah: https://github.com/containers/buildah/blob/main/docs/tutorials/01-intro.md#using-containerfilesdockerfiles-with-buildah https://github.com/containers/buildah/blob/main/docs/tutoria...

rsyring 1y ago

I've wanted to try Podman for a couple years. But I keep bowing out because there are no official Ubuntu builds that I could find. Upstream seems content with that status quo. That's their prerogative, and I could build it myself, but it makes me concerned they don't really have multi-distro compatibility as a priority, which makes me hesitant to commit time to experimenting with it when Docker considers Ubuntu a first-class citizen.

bityard 1y ago
Huh? It's right there in the docs: https://podman.io/docs/installation#installing-on-linux https://podman.io/docs/installation#installing-on-linux apt-get -y install podman Podman doesn't produce their own binary distributions for ANY Linux distro. The only binary packages they provide are for Mac and Windows because those don't have a native package repository. Even if they did release their own Linux packages, bear in mind that Podman development is driven by and sponsored almost entirely by Red Hat. It's not really in their interest to pay their developers to maintain packages for Ubuntu, a direct competitor in the enterprise Linux space.
- figmert 1y ago
  The problem is podman in the apt repos are very old. Last time I checked, Podman was on v5, where debian was on v2 something.
  - saint_yossarian 1y ago
    Must have been ages ago, even stable now has 5.4: https://packages.debian.org/search?searchon=names&suite=all&section=all&keywords=podman https://packages.debian.org/search?searchon=names&suite=all&...
    - wink 1y ago
      "even stable" - that was released 2w ago. ... Also yes, podman v4 on bookwarm was famously useless in many cases and because of either libc or kernel (iirc) you could not even install v5 effortlessly. I like Debian and I like podman but putting this as a usefule nice experience (up until trixie released) is just weird framing.

digikata 1y ago

On Linux I'm using colima with docker compose and buildx and it seems to work ok for my limited cases. On Mac it works ok to, but there are networking cases that Colima on mac doesn't handle - so orbstack for there

leemelone 1y ago

podman pissed me off when I tried it in it's early days. Is it better?

evantbyrne 1y ago

It's not clear from the article, but is this for local development or production deployments? Because it's worth noting that Swarm solves a lot of the limitations that Compose and Podman have for running containers in a production environment. Swarm runs well on singular vms and people with Docker experience can learn the ropes in a day.

tracker1 1y ago

I've mostly just stuck to Docker myself... that said, podman definitely has some curiousity from me, just haven't taken the time to learn. One enhancement suggestion to TFA, would be to take a hash of the compose file path, then have that as a prefix name in a temp directory for a hash of the compose file itself... if the hash changes, dump the .json and rebuild in the temp path... then do the bakah against that file. This would be an easy enough script to make.

deknos 1y ago

i am always missing podman(-compose) cp as docker(-compose) cp :( That was so convenient.

atothayu 1y ago

clean site