5 ms·

If you're not wedded to docker-compose, with podman you can instead use the podman kube support, which provides roughly docker-compose equivalent features using

by jabl 1y ago

If you're not wedded to docker-compose, with podman you can instead use the podman kube support, which provides roughly docker-compose equivalent features using a subset of the Kubernetes pod deployment syntax. Additionally, podman has nice systemd integration for such kube services, you just need to write a short systemd config snippet and then you can manage the kube service just like any other systemd service. Altogether a very nice combination for deploying containerized services if you don't want to go the whole hog to something like Kubernetes.

GCUMstlyHarmls 1y ago
(I'm a big podman stan) Last I tried using the .kube files I ran into issues with specifying container networks (https://github.com/containers/podman/issues/12965 https://github.com/containers/podman/issues/12965). This is sort of "fixed" by using a Quadlet ".kube" but IMO that's a pretty weak solution and removes the "here's your compose file, run it" aspect. Recently (now that Deb13 is out with Podman 5) I have started transitioning to Podmans Quadlet files which have been quite smooth so far. As you say, its great to run things without all the overhead of kubernetes.
- teekert 1y ago
  "...removes the "here's your compose file, run it" Claude recently hallucinated this for me: [Container] ComposeService=my-service Yaml=/path/to/your/podman-compose.yaml For a brief moment in time I was happy but then: Can you really use "ComposeService" in the systemd unit file? I can't find any reference to it You're absolutely right to question that - I made an error. There is no ComposeService directive in systemd or Quadlet. It would be a nice best of both worlds...
  - tkiolp4 1y ago
    It’s exhausting. As someone who doesn’t work with systemd, I would have a hard time using llms for this topic.
    - teekert 1y ago
      I do a combi, sometimes even asking the LLM and starting a ddg search in parallel. It speeds me up. Sometimes the LLM is right, sometimes it's not. NP, I'll get it to work. One should never do anything that one does not understand, but I get to the understand faster as I can also ask more in depth follow up questions to the LLM.
      - lotharcable 1y ago
        For me LLM is just a rubber duck that talks back. It is very stupid and is usually wrong in some meaningful way, but it can help break logjams in my thinking. Giving me clues that might be missing. Sort of like how writing gibberish is sometimes effective for writers to break writer's block. It is also nice for generating boiler plate code for languages that I am not super familiar with. The biggest problems I have with current state of the art LLMs is that errors compound. Meaning that I only really get somewhat useful answers when starting out with the first few questions or the first couple times I ask it to review some code. The longer the session lasts the more la-la land answers I get. It is a game of odds. I expect that with systemd and quadlets it is going to particularly useless because there just isn't that many examples out there. It can only regurgitate what it is trained with so if something isn't widely used and checked into code bases it is trained on then it can't really do anything with it. Which is why it is nice for a lot of common coding tasks, because a lot of code is just same thing tens of thousands people did before for only slightly different contexts and is mostly boilerplate.
  - 0points 1y ago
    Many moons ago, the concept of chaos monkey [1] was concieved. A irrational part of deployment, meant to trigger corner cases and improve the product's stability. Today, people who out-source thinking to a LLM get the chaos monkey for free. The only problem seems to be that the LLM proponents are ahistoricists. 1: https://en.wikipedia.org/wiki/Chaos_engineering#Chaos_Monkey https://en.wikipedia.org/wiki/Chaos_engineering#Chaos_Monkey
- zamalek 1y ago
  (I'm a bigger podman stan) I agree about quadlets, amazing. Docker has one of the most severe cases of not-invented-here. All solutions require a combination of a new DSL, a new protocol, a new encryption scheme, a new daemon, or any combination there-of. People are sleeping on using buildah directly; which OP alluded to with Bakah (but fell short of just using it directly). Ever wish you could run multiple commands in a single layer? Buildah lets you do that. Ever wish you could loop or some other branching in a dockerfile? Buildah lets you do that. Why? Because they didn't invent something new, and so the equivalent of a dockerfile in buildah is just a script in whatever scripting language you want (probably sh, though). This will probably give you the general idea: https://www.mankier.com/1/buildah-from https://www.mankier.com/1/buildah-from I came across this when struggling and repeatedly failing to get multi-arch containers built in Circle CI a few gears ago. You don't have access to an arm64 docker context on their x86 machines, so you are forced to orchestrate that manually (unless your arm64 build is fast enough under qemu). Things begin to rapidly fall apart once you are off of the blessed Docker happy path because of their NIH obsession. That's when I discovered buildah and it made the whole thing a cinch.
  - tracker1 1y ago
    on the QEMU thing... the only time I tried to cross-build arm containers from an x86 server was using whatever servers Github Actions supports... the x86_64 build was pretty normal for the project, but the qemu/buildx/arm64 build was about the same speed as an 8mb Raspberry Pi 4 to build the same project... pretty disappointing.
  - moondev 1y ago
    Buildah is elite tooling. Enables you to build with devices and caps and kernel modules. Buildx acts like you should sign a waiver and really weak documentation if at all for what you are trying to do
  - mardifoufs 1y ago
    Isn't buildah and podman themselves a case of NIH too? ;) I mean, they work fine but I don't think that's an issue with docker either.
iTokio 1y ago
Isn’t that limited to a single node? How would you configure a cluster? I’m trying to explore lightweight alternatives to kubernetes, such as docker swarm, but I think that the options are limited if you must support clusters with equivalent of pods and services at least.
- gf000 1y ago
  In the off chance your search didn't expand to k3s, I can semi-recommend it. My setup is a bit clunky (having a Hetzner cloud instance as controller and a local server as a node throught Tailscale), from which I get an occasional strange error that k3s pods fail to resolve another pod's domain without me having to re-create the DNS resolver system pod, and that I so far failed at getting Velero backups to work with k3s's local storage providers, but otherwise it is pretty decent.
  - iTokio 1y ago
    K3s is light in terms of resources, but heavy in operational complexity, I’m not looking for a smaller version of kubernetes but for a simple way to run container backed services when you’re not google but a small company, something that has few moving parts but is very reliable and low maintenance.
    - esseph 1y ago
      I've been back and forth on this for a long time, but I've just decided at this point that I either settle for podman or docker on a single host, or go to Talos / k3s / k8s. There's a lot of tools there, a lot of inertia, and eventually it's likely that I will need to solve the problems that k8s does. YMMV
    - tracker1 1y ago
      It that's what you want to do, just get something running... what's wrong with docker swarm?
      - osn9363739 1y ago
discardedrefuse 1y ago
> you just need to write a short systemd config snippet and then you can manage the kube service just like any other systemd service. Just FYI, `podman generate systemd --files --name mypod` will create all the systemd service files for you. https://docs.podman.io/en/latest/markdown/podman-generate-systemd.1.html https://docs.podman.io/en/latest/markdown/podman-generate-sy...
- justusthane 1y ago
  `podman generate systemd` was created as a bandaid because it was so difficult to manually write systemd units. Quadlets now make it much easier to create the units by hand, and ‘ `podman generate systemd` is deprecated.
  - discardedrefuse 1y ago
    I appreciate the correction. Its been a while since I used podman + systemd. I will definitely be checking out quadlets next time.
    - jwrallie 1y ago
      Be sure to podlet for automating the creation of the quadlet config files!
    - lotharcable 1y ago
      yeah Quadlets are a pretty reasonable improvement. It was introduced in Podman 4.4 which is circa 2023. And it takes a while for podman to get up to date in non-Redhat related distributions. Like Debian Stable was stuck on 4.3 until Trixie release this month. So unless you are using Fedora and friends or something like Arch it is kinda hard time going for podman users. Which is unfortunate. Docker has a bit of a advantage here because they encourage you to use their packages, not the distribution's. Here is a example Quadlet configuration i use for syncthing that I run out of my home: [Unit] Description=syncthing After=default.target [Container] ContainerName=syncthing Image=docker://docker.io/syncthing/syncthing:latest Volume=/home/lothar/.syncthing:/var/syncthing:z Volume=/home/lothar:/var/home/:rslave Network=host Pull=always Environment=PUID=1000 Environment=PGID=1000 Environment=STGUIADDRESS='' UserNS=keep-id:uid=1000,gid=1000 SecurityLabelDisable=true [Install] WantedBy=multi-user.target default.target This then gets dropped into ~/.config/containers/systemd/syncthing.container And it is handled automatically. This configures the syncthing container to always get updated on each startup, bypasses the "rootless" networking by using host networking (rootless networking is limited and slow), and the default Sync dir ends up in ~/.syncthing where as I can add more sync'd directories to my real home directory by directing it to /var/home/ in the syncthing web ui. As you can see the arguments under "container" is just really capitalized versions of docker/podman arguments. Also if you like GUIs the podman desktop has support for helping to generating quadlets. Although I haven't tried it out yet.
3abiton 1y ago
I am curious performance wise about the performance difference between podman and incus. I found incus to be also extremly flexible.
- esseph 1y ago
  These seem like two very different stacks designed to solve quite different problems (incus v podman)
- seabrookmx 1y ago
  They both utilize all the linux c-group magic to containerize. So performance is roughly the same. Incus is an LXD fork, and focuses on "system" containers. You basically get a full distro, complete with systemd, sshd, etc. etc. so it is easy to replace a VM with one of these. podman and docker are focused on OCI containers which typically run a single application (think webserver, database, etc). I actually use them together. My host machine runs both docker and incus. Docker runs my home server utilities (syncthing, vaultwarden, etc) and Incus runs a system container with my development environment in it. I have nested c-groups enabled so that incus container actually runs another copy of docker _within itself_ for all my development needs (redis, postgres, etc). What's nice about this is that the development environment can easily be backed up, or completely nuked without affecting my host. I use VS Code remote SSH to develop in it. The host typically uses < 10GB RAM with all this stuff running.. about half what it did when I was using KVM instead of Incus.
  - shellwizard 1y ago
    If you use the non LTS branch of incus it supports OCI containers. Have you tried that instead of running docker inside of a LXC container?
    - seabrookmx 1y ago
      That feature might be able to replace my docker usage on the host, so I don't need it and incus side by side. Which would be pretty neat. Within the incus dev environment container though I'm pretty sure I want to keep docker, as I have a lot of tooling that expects it for better or worse (docker compose especially). It also doesn't appear incus integrates buildkit etc. so even if I used it here, I'd still need something else to _build_ OCI images.
osigurdson 1y ago
That is what I do as well. I'd rather not have to remember more than one way of doing things so 'podman play kube' allows me to use Kubernetes knowledge for local / smaller scale things as well.

by jabl 1y ago

GCUMstlyHarmls 1y ago

(I'm a big podman stan) Last I tried using the .kube files I ran into issues with specifying container networks (https://github.com/containers/podman/issues/12965 https://github.com/containers/podman/issues/12965). This is sort of "fixed" by using a Quadlet ".kube" but IMO that's a pretty weak solution and removes the "here's your compose file, run it" aspect. Recently (now that Deb13 is out with Podman 5) I have started transitioning to Podmans Quadlet files which have been quite smooth so far. As you say, its great to run things without all the overhead of kubernetes.

teekert 1y ago
"...removes the "here's your compose file, run it" Claude recently hallucinated this for me: [Container] ComposeService=my-service Yaml=/path/to/your/podman-compose.yaml For a brief moment in time I was happy but then: Can you really use "ComposeService" in the systemd unit file? I can't find any reference to it You're absolutely right to question that - I made an error. There is no ComposeService directive in systemd or Quadlet. It would be a nice best of both worlds...
- tkiolp4 1y ago
  It’s exhausting. As someone who doesn’t work with systemd, I would have a hard time using llms for this topic.
  - teekert 1y ago
    I do a combi, sometimes even asking the LLM and starting a ddg search in parallel. It speeds me up. Sometimes the LLM is right, sometimes it's not. NP, I'll get it to work. One should never do anything that one does not understand, but I get to the understand faster as I can also ask more in depth follow up questions to the LLM.
    - lotharcable 1y ago
      For me LLM is just a rubber duck that talks back. It is very stupid and is usually wrong in some meaningful way, but it can help break logjams in my thinking. Giving me clues that might be missing. Sort of like how writing gibberish is sometimes effective for writers to break writer's block. It is also nice for generating boiler plate code for languages that I am not super familiar with. The biggest problems I have with current state of the art LLMs is that errors compound. Meaning that I only really get somewhat useful answers when starting out with the first few questions or the first couple times I ask it to review some code. The longer the session lasts the more la-la land answers I get. It is a game of odds. I expect that with systemd and quadlets it is going to particularly useless because there just isn't that many examples out there. It can only regurgitate what it is trained with so if something isn't widely used and checked into code bases it is trained on then it can't really do anything with it. Which is why it is nice for a lot of common coding tasks, because a lot of code is just same thing tens of thousands people did before for only slightly different contexts and is mostly boilerplate.
- 0points 1y ago
  Many moons ago, the concept of chaos monkey [1] was concieved. A irrational part of deployment, meant to trigger corner cases and improve the product's stability. Today, people who out-source thinking to a LLM get the chaos monkey for free. The only problem seems to be that the LLM proponents are ahistoricists. 1: https://en.wikipedia.org/wiki/Chaos_engineering#Chaos_Monkey https://en.wikipedia.org/wiki/Chaos_engineering#Chaos_Monkey
zamalek 1y ago
(I'm a bigger podman stan) I agree about quadlets, amazing. Docker has one of the most severe cases of not-invented-here. All solutions require a combination of a new DSL, a new protocol, a new encryption scheme, a new daemon, or any combination there-of. People are sleeping on using buildah directly; which OP alluded to with Bakah (but fell short of just using it directly). Ever wish you could run multiple commands in a single layer? Buildah lets you do that. Ever wish you could loop or some other branching in a dockerfile? Buildah lets you do that. Why? Because they didn't invent something new, and so the equivalent of a dockerfile in buildah is just a script in whatever scripting language you want (probably sh, though). This will probably give you the general idea: https://www.mankier.com/1/buildah-from https://www.mankier.com/1/buildah-from I came across this when struggling and repeatedly failing to get multi-arch containers built in Circle CI a few gears ago. You don't have access to an arm64 docker context on their x86 machines, so you are forced to orchestrate that manually (unless your arm64 build is fast enough under qemu). Things begin to rapidly fall apart once you are off of the blessed Docker happy path because of their NIH obsession. That's when I discovered buildah and it made the whole thing a cinch.
- tracker1 1y ago
  on the QEMU thing... the only time I tried to cross-build arm containers from an x86 server was using whatever servers Github Actions supports... the x86_64 build was pretty normal for the project, but the qemu/buildx/arm64 build was about the same speed as an 8mb Raspberry Pi 4 to build the same project... pretty disappointing.
- moondev 1y ago
  Buildah is elite tooling. Enables you to build with devices and caps and kernel modules. Buildx acts like you should sign a waiver and really weak documentation if at all for what you are trying to do
- mardifoufs 1y ago
  Isn't buildah and podman themselves a case of NIH too? ;) I mean, they work fine but I don't think that's an issue with docker either.

iTokio 1y ago

Isn’t that limited to a single node? How would you configure a cluster? I’m trying to explore lightweight alternatives to kubernetes, such as docker swarm, but I think that the options are limited if you must support clusters with equivalent of pods and services at least.

gf000 1y ago
In the off chance your search didn't expand to k3s, I can semi-recommend it. My setup is a bit clunky (having a Hetzner cloud instance as controller and a local server as a node throught Tailscale), from which I get an occasional strange error that k3s pods fail to resolve another pod's domain without me having to re-create the DNS resolver system pod, and that I so far failed at getting Velero backups to work with k3s's local storage providers, but otherwise it is pretty decent.
- iTokio 1y ago
  K3s is light in terms of resources, but heavy in operational complexity, I’m not looking for a smaller version of kubernetes but for a simple way to run container backed services when you’re not google but a small company, something that has few moving parts but is very reliable and low maintenance.
  - esseph 1y ago
    I've been back and forth on this for a long time, but I've just decided at this point that I either settle for podman or docker on a single host, or go to Talos / k3s / k8s. There's a lot of tools there, a lot of inertia, and eventually it's likely that I will need to solve the problems that k8s does. YMMV
  - tracker1 1y ago
    It that's what you want to do, just get something running... what's wrong with docker swarm?
    - osn9363739 1y ago

discardedrefuse 1y ago

> you just need to write a short systemd config snippet and then you can manage the kube service just like any other systemd service. Just FYI, `podman generate systemd --files --name mypod` will create all the systemd service files for you. https://docs.podman.io/en/latest/markdown/podman-generate-systemd.1.html https://docs.podman.io/en/latest/markdown/podman-generate-sy...

justusthane 1y ago
`podman generate systemd` was created as a bandaid because it was so difficult to manually write systemd units. Quadlets now make it much easier to create the units by hand, and ‘ `podman generate systemd` is deprecated.
- discardedrefuse 1y ago
  I appreciate the correction. Its been a while since I used podman + systemd. I will definitely be checking out quadlets next time.
  - jwrallie 1y ago
    Be sure to podlet for automating the creation of the quadlet config files!
  - lotharcable 1y ago
    yeah Quadlets are a pretty reasonable improvement. It was introduced in Podman 4.4 which is circa 2023. And it takes a while for podman to get up to date in non-Redhat related distributions. Like Debian Stable was stuck on 4.3 until Trixie release this month. So unless you are using Fedora and friends or something like Arch it is kinda hard time going for podman users. Which is unfortunate. Docker has a bit of a advantage here because they encourage you to use their packages, not the distribution's. Here is a example Quadlet configuration i use for syncthing that I run out of my home: [Unit] Description=syncthing After=default.target [Container] ContainerName=syncthing Image=docker://docker.io/syncthing/syncthing:latest Volume=/home/lothar/.syncthing:/var/syncthing:z Volume=/home/lothar:/var/home/:rslave Network=host Pull=always Environment=PUID=1000 Environment=PGID=1000 Environment=STGUIADDRESS='' UserNS=keep-id:uid=1000,gid=1000 SecurityLabelDisable=true [Install] WantedBy=multi-user.target default.target This then gets dropped into ~/.config/containers/systemd/syncthing.container And it is handled automatically. This configures the syncthing container to always get updated on each startup, bypasses the "rootless" networking by using host networking (rootless networking is limited and slow), and the default Sync dir ends up in ~/.syncthing where as I can add more sync'd directories to my real home directory by directing it to /var/home/ in the syncthing web ui. As you can see the arguments under "container" is just really capitalized versions of docker/podman arguments. Also if you like GUIs the podman desktop has support for helping to generating quadlets. Although I haven't tried it out yet.

3abiton 1y ago

I am curious performance wise about the performance difference between podman and incus. I found incus to be also extremly flexible.

esseph 1y ago
These seem like two very different stacks designed to solve quite different problems (incus v podman)
seabrookmx 1y ago
They both utilize all the linux c-group magic to containerize. So performance is roughly the same. Incus is an LXD fork, and focuses on "system" containers. You basically get a full distro, complete with systemd, sshd, etc. etc. so it is easy to replace a VM with one of these. podman and docker are focused on OCI containers which typically run a single application (think webserver, database, etc). I actually use them together. My host machine runs both docker and incus. Docker runs my home server utilities (syncthing, vaultwarden, etc) and Incus runs a system container with my development environment in it. I have nested c-groups enabled so that incus container actually runs another copy of docker _within itself_ for all my development needs (redis, postgres, etc). What's nice about this is that the development environment can easily be backed up, or completely nuked without affecting my host. I use VS Code remote SSH to develop in it. The host typically uses < 10GB RAM with all this stuff running.. about half what it did when I was using KVM instead of Incus.
- shellwizard 1y ago
  If you use the non LTS branch of incus it supports OCI containers. Have you tried that instead of running docker inside of a LXC container?
  - seabrookmx 1y ago
    That feature might be able to replace my docker usage on the host, so I don't need it and incus side by side. Which would be pretty neat. Within the incus dev environment container though I'm pretty sure I want to keep docker, as I have a lot of tooling that expects it for better or worse (docker compose especially). It also doesn't appear incus integrates buildkit etc. so even if I used it here, I'd still need something else to _build_ OCI images.

osigurdson 1y ago

That is what I do as well. I'd rather not have to remember more than one way of doing things so 'podman play kube' allows me to use Kubernetes knowledge for local / smaller scale things as well.