13 ms·
Packaging Kubernetes for Debian
- gitweb 6y agoThe packaging nightmare is why I left Debian and Ubuntu for Arch. I now maintain multiple packages in the AUR with relative ease, and see how easily other maintainers manage multiple packages like Kubernetes without issue, but if it was using Deb packages it would be a full-time job.
- viraptor 6y agoI'm not sure this is a fair comparison. AUR has next to no rules. You could just as easily maintain a PPA. As described in the article, the package is available via Debian-testing.
- loopz 6y agoArch/Manjaro is often praised as simpler, more powerful etc., but there are also complaints about instability. Mint has several QA links: upstream, debian, Ubuntu, Mint. With Mint sanitizing the corporate influence, ie. snap being off by default. I don't need bleeding edge, just stable desktop. So I've found my optimum in Mint, though nothing is perfect. Vendoring is shifting focus back to individual projects though. K8s could almost replace an OS in many regards. It is made to be modular, but only within its own framework. So it is changing the way software is run, similarly to initd and systemd. Such systems as we've seen, tend to "take over" and not be made to be part of and integrate well from inside an OS. Vendoring is really old practice, but also makes testing and support simpler for upstream. We all know the dynamic approach tend to be more unchanging, or associated with several circles of dependency hells. It's just different end goals for software development and deployment, that affect how software is distributed. Kind of similar to touted goals of JEE of java fame, which also is too monolithic to integrate well within complex systems.
- gitweb 6y agoI encourage everyone to go evaluate building and submitting a package using the AUR vs Launchpad (PPA). I worked at Canonical and would never go back to Dpkg. I recommend users try it out themselves and see how much easier and cleaner the PKGBUILD approach is. AUR does have a process to rate, flag and comment on packages. It is also much easier to review the installation script using PKGBUILD. There are a lot of high quality packages in the AUR but you should review everything you install unofficially, just like installing packages from a PPA.
- konjin 6y agoThe packaging nightmare also means that when you stand up a system it won't need upgrades for a decade Leave an arch system without an upgrade for a month and you're playing Russian roulette.
- gravypod 6y agoIt will need upgrades for those decades you're just hoping someone remembers how to apply them 9years after they're I'm wide spread adoption.
- konjin 6y agoNo, it will need security updates. Which Debian stable provides for seven years currently.
- jchw 6y agoPeople say this all the time but it isn’t true. Occasionally a package will require manual intervention to update properly. The manual intervention steps are generally pretty easy and documented on the Arch homepage, so if you haven’t updated in a while and are concerned, all you really have to do is check the homepage. But I don’t even do that. I just pacman -Syu and check if it failed. For the past few years this has worked out just fine. Most of the packages, I don’t even have installed, so I hardly ever hit a manual intervention step. Of course, Debian doesn’t require manual intervention for most updates, but it also isn’t rolling release. If you want packages that are actually recent you have to sit on Sid, which is a lot less stable than Arch. And if you have a Debian machine that’s a couple years old and you want to upgrade it to the latest Debian version? Good luck. Some of that isn’t Debian’s fault, but whereas on Arch major, breaking package changes are rare but irregular, on Debian they tend to hit you simultaneously in one major upgrade. For home machines, I definitely prefer an occasional manual intervention and continuous fixes over major breakages every couple years where I often just give up and reinstall from scratch. That said I use Arch a lot less nowadays as I’ve moved onto NixOS. Nix is clearly headed somewhere new, though it remains to be seen if the complexity of the approach is actually maintainable. If I want to make an idiomatic Arch package, it’s usually easy: PKGBUILDs are intuitive and simple. Nix is much the same for most stuff, especially since most of the boilerplate for various build systems has been automated; though for a program as complex as Kubernetes its a nightmare, to be sure. For Dpkg though, it seems like so much complexity, and I don’t think I ever really had a good experience. And honestly, I have no idea how to make an RPM package anymore.
- barumi 6y ago> The packaging nightmare is why I left Debian and Ubuntu for Arch. I find this comment odd and highly dubious. There is absolutely nothing wrong with Debian's packaging system. At most, some individual packages could have been packaged differently, which is arguably a matter of personal taste. Normally, everything just works, and works very well. Either you shed some additional light on your personal struggles with Debian's packaging, which you did not do at all, or I have to scratch off your baseless assertion as a kin of someone on the internet shouting at clouds.
- jchw 6y agoIt is a recurring theme that whenever Arch proponents mention that they don't like Debian packaging, there is a frustrated reaction to it. But I don't understand what people want to hear; there's clearly a reason why people would choose Debian over Arch, Arch users say packaging is a big part of it, and it seems like nobody can accept that this is right. I can at least tell you that people don't choose Arch over Debian for the installation experience. :)
- bawolff 6y agoThe parent was a bit aggressive, but i think asking why the first poster likes arch packages better than debian is a fair question, since that's the interesting part.
- lloeki 6y ago> I can at least tell you that people don't choose Arch over Debian for the installation experience Well, I do (sample of one). The Arch installation has been streamlined a lot. Now it’s all about 0) boot the image (cd, usb, PXE), which is actually an Arch install a) creating your filesystem (pick you poison), and mounting it b) telling pacman to install base, base-devel, and a bootloader on the target fs c) installing and configuring the bootloader d) rebooting Done. d-i barely takes care of that for you, it’s “just” wrapping it behind a UI (which is sort of useful, sure saves one from reading docs, but has been an annoying abstraction/obfuscation/magic layer for me more often than not). The remainder (setting up X/Wayland/whatever is no different on Debian than on Arch, as d-i does not help much.
- csande17 6y agoSay what you will about Gentoo, but I've actually really liked making some small packages for personal use. It's the closest system I've seen to just writing down the steps you'd normally take to compile the software manually.
- 1_player 6y agoThat how Arch Linux packages work. Just a shell file with some metadata
- cbmuser 6y agoArch doesn’t have to support multiple architectures and releases. Also, Arch usually puts all files into a single binary package while Debian splits arch-dependent and arch-independent files into separate packages. Also, Debian separates packages into runtime and development libraries, another thing Arch doesn’t do either.
- gitweb 6y agoThis is incorrect
- Foxboron 6y agoHow is the previous statement incorrect?
- gitweb 6y ago> Also, Debian separates packages into runtime and development libraries, another thing Arch doesn’t do either. Arch definitely has separate packages for runtime and development libraries. It doesn't have as many, but they exist and can be found by simply searching for `-dev`. > Arch usually puts all files into a single binary package while Debian splits arch-dependent and arch-independent files into separate packages Arch has `any` and then architecture specific packages: https://wiki.archlinux.org/index.php/Arch_package_guidelines#Architectures https://wiki.archlinux.org/index.php/Arch_package_guidelines... > Arch doesn’t have to support multiple architectures and releases https://wiki.archlinux.org/index.php/32-bit_package_guidelines https://wiki.archlinux.org/index.php/32-bit_package_guidelin... There are also distros of Arch for ARM and 32bit, however if you're looking for a more integrated multi-arch PKGBUILD-comparable distro then Alpine is it and probably what I'll be migrating to at some point.
- Foxboron 6y ago> Arch definitely has separate packages for runtime and development libraries. It doesn't have as many, but they exist and can be found by simply searching for `-dev`. They are an exception to the rule, where the benefits outweights the negatives. It's been done to ensure we have smaller container images, or if the maintaine thinks it makes sense. But as a rule, we do not care while debian does. >Arch has `any` and then architecture specific packages: How is the `any` arch related to package splitting? >https://wiki.archlinux.org/index.php/32-bit_package_guidelin https://wiki.archlinux.org/index.php/32-bit_package_guidelin... > There are also distros of Arch for ARM and 32bit, however if you're looking for a more integrated multi-arch PKGBUILD-comparable distro then Alpine is it and probably what I'll be migrating to at some point. Those package guidelines exist, but are dated and not used by us, the packagers. Arch ARM and 32bit are also seperate distirbutions that isn't affiliated with the 64bit Arch Linux distribution.
- pushedx 6y agoKubernetes binaries are meant to be statically linked. I can understand the licensing concern, but if each of those libraries was packaged separately the experience of using Kubernetes on Debian would be markedly worse.
- tupputuppu 6y agoIn which way would it be worse?
- captn3m0 6y agoDoes anyone know of any packaging efforts to run microk8s without snap? I love microk8s, but running snap just for that one package feels like an overkill.
- dharmab 6y agoWhen we implemented Kubernetes at my employer, one of the things we made clear from the outset was that there is no LTS release of Kubernetes. Keeping k8s and friends up to date is at least one full time job (ideally a half-pizza team). This also heavily influenced our choice of OS away from ones that emphasized LTS package management and towards more recent kernels with better BPF and kTLS support. (We also intentionally used short-lived TLS certificates to prevent anyone from running a cluster longer than several months without updates. Worked great. There are workarounds of course but anyone aware of them is the kind of person who updates anyway.)
- polskibus 6y agoWhat distro did you end up with?
- zimbatm 6y agoThat's why k8s is not a good fit for early startups. It will consume too much of your time that is better spent developing a product that the public wants.
- AlphaSite 6y agoIn that case use a managed cluster and it’s less work.
- speedgoose 6y agoIt has about the same issues but because it's managed and because you are a small company, nobody cares about your issues and you can do nothing about it.
- viraptor 6y agoWhat's the driver for people who actually want an OS-provided k8s? What holds them back from using upstream releases instead? (Or given large enough team - building/patching themselves) I guess it also applies to Ubuntu/openstack to a lesser level. If you're planning to run those as a platform, it sounds like more pain then necessary.
- throwaway54235 6y agoI worked on one of the large commercial projects that offers OS-packaged k8s. The customers were screaming for it. They want a simpla and reliable way to deploy k8s without a team of specialist, and the guarantee to receive security updates for years without having to upgrade to a new version.
- loopz 6y agoFor many it could just be a convenience, assuming the distro way just works. With conflicting goals and approaches, you might end up worse off though. Ideally, you mostly want to spend time and effort on higher pursuits than managing or building your own platform.
- viraptor 6y ago> Ideally, you mostly want to spend time and effort on higher pursuits than managing or building your own platform. I've run openstack for a while and... realistically if you run a platform, you need to have people/time to manage it. I understand from conversations about k8s that is not that much different.
- loopz 6y agoThinking more about dev end and one man shops than enterprise production. With CI/CD the landscape is shifting once more though.
- Fumtumi 6y agoIf you use microk8s from ubuntu, you know that there is a team behind it who takes care of updating it, upgrading it and making it simple for you to do so. You also know, that your issue with that setup might get noticed by someone else using the same and will fix it for you. You get sane defaults and you get k8s optimized for the os. I run microk8s at home. Its simple, fast, easy and works. Not seeing any issue with it.
- jancsika 6y ago> There are other packages in Debian with vendored dependencies, though none, he acknowledged, have anywhere near the 200 found in his Kubernetes package. How many are vendored in Chromium/Firefox?
- kubanczyk 6y agoThe decision to share libraries between many apps has some benefit, especially on desktops. It also has costs. It's a decision you can make. The article uses so many convoluted phrasings and arguments to avoid saying that. A bureaucracy of *.deb.
- inshadows 6y agoAvoid saying what? That it has pros and cons? What a shocking revelation you made there! It's implied in few paragraphs. It's not that hard to infer if you take off your hat. Thou who read Corbet often know a thing having pros and cons is implied in his reasoning.
- rstuart4133 6y ago> Thou who read Corbet often know a thing having pros and cons is implied in his reasoning. His skill is the warts are laid bare, but not unkindly and often with humour.
- dsr_ 6y agoThere's another interpretation available: kubernetes is not a mature system, and that's why it's too much work to package properly. 200 library dependencies is not too many for Debian to handle, as long as they don't change very often and the work is useful for other packages. But Kubernetes' dependencies do change often, and there are not enough other Go systems around to spread the work around. This is not a stable system, therefore Debian can't ship it in a stable release.
- orf 6y agoI think there’s a difference between “stable” and “evolving”. Software with a much lower release cadence than k8s works well with Debian’s distribution model, but it falls down with frequent releases and a large dependency tree. I guess you’d see that with NodeJS applications as well - I don’t think it’s practical for Debian to re-invent NPM and try to distribute and ship the large and changing dependency trees you find in those projects. However that doesn’t mean the software is not _stable_, for the typical definition of stable.
- justinclift 6y agoIt kind of feels like the same problem OpenStack had, and failed due to. That being said, OpenStack is technically still around, so it's not completely "dead". ;)
- loxs 6y agoAlso I don't get it, what's the problem with shipping a statically linked binary? Yeah, it goes a bit against the debian way, but it will facilitate things like out-of-the-box init scripts etc.
- tremon 6y agoTracking security incidents. If every piece of software uses the system-wide copy of a vulnerable library, you only need to patch that library and the entire system is safe. For every embedded copy of that library, a separate patch and release must be made. That's a lot of management overhead.
- effie 6y agoLots of people present their observations and opinions, but this problem is not hard, and the technically best solution is obvious - Debian curbs their ambitions and just distribute the damn thing, with all the vendored stuff, as upstream intended. Some Debian people don't like it. Users don't care, as long as they can apt install it.
- debianUserNew 6y agoDebian Uses care. Otherwise they would use any other distro. The good the about the distro fragmentation is precisely that, If debian curbs their ambitions then it is no longer debian, and those that prefer the debian way over ubuntu or fedora or arch would get screwed in the process. It’s fine to not like it and not use it, but I do like projects that keep their ways.
- effie 6y agoWell some may care, I agree with that. But Debian already changed its ways in controversial shift to systemd that leave many users disappointed - pragmatism and modernisation efforts prevailed over traditional ways of composability, stability and predictability. Why shouldn't this happen for the policy of packages as well? The Debian way is hitting its limits and is ripe for some updates. The amount of work needed to maintain foreign hard to comprehend software has to be curbed to make it manageable in the future. If enough people say "yeah lets package it as is and be done with it" why shouldn't Debian go that way? It is natural to change and adapt. Big software with fast release cadency is going to get even more complex and harder to package each release.
- throwaway7281 6y agoNaive question: I package small and midsize Go projects as deb packages all the time and find it relatively easy. All you need is the binaries and some config. Cross-compilation is easy with Go, so what exactly is a problem with k8s? Isn't it just a bunch of executables for kubectl, kubelet and api-server?
- ibotty 6y agoThe problem is building and packaging (and versioning) all dependencies. It's a reasonable policy to not allow bundling which conflicts with usual Go developing.
- moondev 6y agokubeadm kubelet kubectl + socat and a few more standard utils. Where are the 200+ dependencies they mention?
- cuillevel3 6y agoBuild time dependencies. Normally distributions would put every library from the dependency tree in a package and build it. Then install all the dependencies on the build host and produce the Kubernetes package.
- moondev 6y agoGood to know, thanks for the clarification. I can see now I oversimplified the packaging process.