6 ms·
I'm an enduser and I expect things to "just work". That seems to match a lot with Red Hat's Linux distros (i.e. RHEL): "Just use it, it's a defacto standard so
by upnrunning2140 6y ago
I'm an enduser and I expect things to "just work". That seems to match a lot with Red Hat's Linux distros (i.e. RHEL): "Just use it, it's a defacto standard so most third party software is tested on it, don't worry about underlying components as Red Hat has made sane choices for you."
That's the theory (and Red Hat's marketing). I wish I could just be using CentOS/RHEL and not worry about the choices Red Hat makes. But the shit is piling up (sorry for my harsh words, no offence intended).
So, let's containerise something simple on top of the "centos" container image, let's say postfix. Oh, postfix on centos needs systemd for logging. But systemd in a container is a nightmare. You can get systemd to work in a container if the host system is using containerd and you pass certain special files from the host to the container. So just use Podman instead of Docker, right? Podman has functionality to make systemd in a container work. OK, I can run my container on my fat workstation (a Centos system with Podman). But is it still portable? Can I just run it in Github Actions, AWS ECS, a Windows machine with Docker Desktop? Nope. It's not portable.
Let's put the systemd rant aside (I really like the systemd CLI UX, but the architecture (dbus, etc) seems to only serve one usecase properly (Desktop systems) and seems to be heavily wrong for the container usecase).
Let's rant about the architectural choices Red Hat is making for containers.
Before Red Hat started to get involved there was the open source, (now) cross-platform containerd that a lot of tools are building on top. It consumes the low level runtime runc and both provides an API for Kubernetes (CRI) and additional things. It is highly pluggable and therefore the "only container runtime you'll ever need". And it's purely community governed (CNCF).
What is Red Hat doing? Are they building their container tools on top of containerd? Nope. They create their own low level container runtime (crun), their own mid level container runtime (cri-o). But cri-o only covers what's needed for Kubernetes. So to be able to build container images, etc (the "working with containers on a single machine" usecase) additional tools are needed (podman, buildah, skopeo) and they have to implement the missing functionality themselves. So Red Hat's Open Shift (Kubernetes distribution) builds on an entirely different stack than Podman.
Fragmentation everywhere. Was that really necessary?
Why should I care as an enduser? Well, I can inspect and debug all tools that use containerd the same way (i.e. using the "ctr" tool). I can relatively easily even write my own tools that use containerd (via it's grpc api) and can access the resources (containers, images, etc) that these tools are managing to solve my special custom needs.
For the Red Hat stuff everything is different. I could work with cri-o using crictl and write custom tools using cri interface. For Podman I would need to use Podman's API. (Which has annoyingly entirely changed between v1 and v2).
ok, enough rant (I could go on with how hard it is to get an up to date version of podman on a supported RHEL system even with app streams, that UBI images don't provide podman/buildah/skopeo, etc etc etc).
Please don't mistake me: Podman, cri-o, crun, open shift are all amazing technologies. And it's great that they are mostly open source via upstream projects. But I wish this whole fragmentation hadn't happened. For me as an enduser it's nightmare.
- upnrunning2140 6y agoand yes, people have proven that it's possible to build something like podman or buildah on top of containerd / buildkit! No need to reinvent the wheel! https://github.com/rancher/k3c https://github.com/rancher/k3c https://github.com/genuinetools/img https://github.com/genuinetools/img
- freedomben 6y agoDisclaimer: I work for Red Hat, but I don't code or design any of the things you've mentioned here. Opinions are my own. I think I just gave you your first ever upvote on HN! Thanks for this post, it's great discussion. Regarding Postfix/systemd issues, you should take a look at ubi-init image. It has a minimal systemd in it to address problems like that[1][2]. Regarding crun, have you looked at the "Why another implementation" in the crun README.md by chance?[3] Or have you read the blog post about it that goes into a lot more detail?[4] I totally agree with the dislike toward fragmentation. I myself have been very critical of NIH and fragmentation when it happens. I do think there's an interesting case to be made for crun though. [1]: https://developers.redhat.com/products/rhel/ubi https://developers.redhat.com/products/rhel/ubi [2]: https://catalog.redhat.com/software/containers/ubi8/ubi-init/5c359b97d70cc534b3a378c8 https://catalog.redhat.com/software/containers/ubi8/ubi-init... [3]: https://github.com/containers/crun https://github.com/containers/crun [4]: https://www.redhat.com/sysadmin/introduction-crun https://www.redhat.com/sysadmin/introduction-crun
- upnrunning2140 6y agothanks for the response and the links! I will certainly have a look. (And again sorry for my rather harsh words - I have a lot of respect for all the hard work everyone at Red Hat is doing. I can only imagine that it's much harder to make technical decisions and build tools like Podman than it looks like. Thank you for not taking my rant as an offence! It's just my brutally raw user experience).
- rhatdan 6y agocontainerd is a daemon, which actually came out after CRI-O. But anyways the hole idea is Podman and Buildah was to get away from using root running daemons to launch containers. The important thing about containers is that we launch we all follow the standards. Podman, Docker, CRI-O, Buildah, Containerd all use the same OCI based images available at container registries, and all launch them the same time via OCI Runtimes. These runtimes can be runc, crun, kata, krun, gVisor ... There is no fragmentation on OCI, which is important. Think of these as been web content, you have lots of choice on viewing and using web content, firefox, chrome, safari, IE, curl, linx, wget... All of these tools (Podman, Buildah, Skopeo, CRI-O) that can interact with the OCI content and all have different strengths. Please do not add FUD about systemd. None of the new container engines require systemd, they all can run and support all images at Docker.io. The new container engines support systemd better then Docker did, but they do NOT require it.