4 ms·

When I read people complaining about "Small Images" like mentioned here where the author seems to warn against it and even proposes making sure enough "tooling"

by kator 5y ago

When I read people complaining about "Small Images" like mentioned here where the author seems to warn against it and even proposes making sure enough "tooling" is in the container. I'm always a bit curious why more people don't talk about using nsenter in the rare situation they need to inspect stuff in a running container? For a production microservice container, why have a bunch of bloat in the container that increases the surface area to manage while also only actually being used in edge cases?

foxfluff 5y ago
How do you use nsenter efficiently for running more than one program? You might need dozens of tools (and all their dependencies) to diagnose and fix issues.
- yjftsjthsd-h 5y ago
  If I understand your question correctly, you just make sure that the first command is nsenter bash
Lhiw 5y ago
I aim for "small images" because downloading a full install of Ubuntu is rediculous on anything but gigabit fibre. But I don't aim for minimal images. Using alpine and installing what ever you like without worrying about all the bullshit to make caches stay empty still results in images only a few megs, don't need to do much better than that. Though I tend to agree, if you're talking about a compiled language. Static binary and there is little need for anything other than `from scratch`.
hvgk 5y ago
I shall look into nsenter. I have never actually used it. It may cover this use case. As for the status quo, I spend a lot of time extracting dumps from things in containers and debugging weird issues which is really really difficult without the ability to install any tools on the container at will. When done, we blast the container and let kubernetes reschedule a new one, disposing of all tools and state. On bloat, mine are sitting on top of a debian container layer, mostly because it’s a hell of a lot easier dealing with some dynamic libraries than fecking around with static linking stuff and copying dependencies into a minimalist container. Edit: There is overlap between these two things which nsenter may not be able to solve looking at what I am reading.
xuhu 5y ago
There is probably no nsenter equivalent for "kubectl exec thepod -- /bin/sh" if you can't ssh into the node.
- gizdan 5y ago
  Minimal containers shouldn't need that though. Your logs should tell you enough about the application it's running. K8s (get/describe) should tell you about the lifecycle of the pod. Lastly something like "dive" should tell you all you need about the image's file layout.
  - gouggoug 5y ago
    > Minimal containers shouldn't need that though. In theory. In practice this is only rarely the case.
- mook 5y ago
  For non-production clusters, you may be able to play with ephemeral containers† if turning on the feature gate seems reasonable. † https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/#ephemeral-container https://kubernetes.io/docs/tasks/debug-application-cluster/d...

by kator 5y ago

foxfluff 5y ago

How do you use nsenter efficiently for running more than one program? You might need dozens of tools (and all their dependencies) to diagnose and fix issues.

yjftsjthsd-h 5y ago
If I understand your question correctly, you just make sure that the first command is nsenter bash

Lhiw 5y ago

I aim for "small images" because downloading a full install of Ubuntu is rediculous on anything but gigabit fibre. But I don't aim for minimal images. Using alpine and installing what ever you like without worrying about all the bullshit to make caches stay empty still results in images only a few megs, don't need to do much better than that. Though I tend to agree, if you're talking about a compiled language. Static binary and there is little need for anything other than `from scratch`.

hvgk 5y ago

I shall look into nsenter. I have never actually used it. It may cover this use case. As for the status quo, I spend a lot of time extracting dumps from things in containers and debugging weird issues which is really really difficult without the ability to install any tools on the container at will. When done, we blast the container and let kubernetes reschedule a new one, disposing of all tools and state. On bloat, mine are sitting on top of a debian container layer, mostly because it’s a hell of a lot easier dealing with some dynamic libraries than fecking around with static linking stuff and copying dependencies into a minimalist container. Edit: There is overlap between these two things which nsenter may not be able to solve looking at what I am reading.

xuhu 5y ago

There is probably no nsenter equivalent for "kubectl exec thepod -- /bin/sh" if you can't ssh into the node.

gizdan 5y ago
Minimal containers shouldn't need that though. Your logs should tell you enough about the application it's running. K8s (get/describe) should tell you about the lifecycle of the pod. Lastly something like "dive" should tell you all you need about the image's file layout.
- gouggoug 5y ago
  > Minimal containers shouldn't need that though. In theory. In practice this is only rarely the case.
mook 5y ago
For non-production clusters, you may be able to play with ephemeral containers† if turning on the feature gate seems reasonable. † https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/#ephemeral-container https://kubernetes.io/docs/tasks/debug-application-cluster/d...