5 ms·
I agree very much about the accidental complexity of containers. Ignoring the runtime concerns (cgroups, namespaces, networking, etc.), the main problem seems
by jrockway 4y ago
I agree very much about the accidental complexity of containers. Ignoring the runtime concerns (cgroups, namespaces, networking, etc.), the main problem seems to have been "we can't figure out how to get Python code to production", and the solution was "just ship a disk image of some developer's workstation to production". To do this, they created a new file format ("OCI image format", though not called that at the time), a new protocol for exchanging files between computers ("OCI distribution spec"), a new way to run executables on Linux ("OCI runtime spec"), and various proprietary versions of Make, with a caching mechanism that isn't aware of the actual dependencies that go into your final product. The result is a staggering level of complexity, all to work around the fact that nobody even bothered trying to add a build system to Python.
Like the author, I tend to write software in a language that produces a single statically linked binary, so I don't need any of this distribution stuff. I don't need layers, I don't need a Make-alike, I don't need a build cache, but I still have to go out of my way to wrap the generated binary and push it to a super special server. Imagine a world where we just skipped all of this, and your k8s manifest looks like:
containers:
- name: foo-app
image:
- architecture: amd64
os: linux
binary: https://github.com/whatever/download/foo-app/foo-app-1.0.0-linux-amd64
checksum: sha256@1234567890abcdef
- architecture: arm64
os: linux
binary: https://github.com/whatever/download/foo-app/foo-app-1.0.0-linux-arm64
checksum: sha256@abdcef1234567890
Meanwhile, if you don't want to orchestrate your deployment and just want to run your app:
wget https://github.com/whatever/download/foo-app/foo-app-1.0.0-linux-amd64
chmod a+x foo-app-1.0.0-linux-amd64
./foo-app-1.0.0-linux-amd64
I dunno. I feel like, as an industry, we spent billions of dollars, founded brand new companies, created a new job title ("Devops", don't get me started), all so that we could avoid making Python output a single binary. I'm not sure that random chaos did the right thing, and you're right to be bitter about it.
- jen20 4y agoThis is effectively what Nomad permits - scheduling things that are not Docker containers.
- BwackNinja 4y agoYou /can/ package python applications into a single binary -- just a zip file prepended with a python shebang. You can do much the same with the base python libraries. No one does it, I presume, because the community is against vendoring dependencies that way, the same way they are against static binaries. https://bwackninja.github.io/2020/11/25/packaging-python.html https://bwackninja.github.io/2020/11/25/packaging-python.htm...
- c7DJTLrn 4y agoThat doesn't solve anything. You still need a Python interpreter. It's literally just another way to move code around.
- isignal 4y agoThat doesn’t work for python extensions used by the application. These extensions may include or link to other dynamic libraries.
- krageon 4y agoIf you have a packaging problem approaching any kind of complexity all python packaging is trash.