3 ms·

Great write-up, but, why not just use dumb-init? https://github.com/Yelp/dumb-init https://github.com/Yelp/dumb-init

morelisp 4y ago
Running dumb-init in a shell would not have helped. Running dumb-init (or tini or whatever, there's lots of these) when your application can handle SIGTERM itself is unnecessary. These are tools for when you don't control an executable in the stack and it doesn't do the right thing itself.
- richardwhiuk 4y ago
  You won't get a corefile if you are PID 1, which is a pretty good reason to use tini.
  - morelisp 4y ago
    While I suppose this is true, I have never needed to examine a core dump from a container, I don't believe our Kubernetes deployment can collect them at the moment (or at least I would need to ask our ops team where to put them / how to get them once dumped), and to a first approximation zero of the services I run containerized produce helpful core dumps in the first place. It's not even out of the question for us to have more RAM used by one process than free disk space. (I do have a lot of experience with helpful corefiles, it's just been totally orthogonal to my experience with "at-scale" backend deployment. Different languages, different dev/ops relations, different attitudes towards abends in general.)
- pm90 4y ago
  > when your application can handle SIGTERM itself is unnecessary. Thats a big “when”. Many applications don’t. e.g. they spawn child processes which they forget about
  - morelisp 4y ago
    Container inits don't magically get you clean shutdown if your application doesn't handle SIGTERM, it just gets you, let's say, fast shutdown. (And "fast", if you're not going to shutdown cleanly, might be much worse!) If you don't want connection issues you'll still need to write a SIGTERM handler, and then you don't need the container init anymore. And "many" is kind of a cop-out word; yes, I'm sure by absolute number it's a lot but: - If you're in e.g. Go, you spawn thousands of child green threads but it's trivial to route a SIGTERMable context into all of them and wait for only the relevant ones. - If you're in e.g. Python, you'll be running in an application server that handles it and you're probably not spawning many threads yourself because it's usually pointless. - If you're any piece of software written to run outside of containers, that's also what non-container environments have used for decades to handle smooth upgrades. - etc. for other common cases in the end, "spawning threads" is no excuse for not handling clean listener shutdown and seems to be more about how "Unix-like" the application is and not threading - e.g. old Java programs tend to be worst in my experience. I've used tini et al mostly when putting entire legacy environments (e.g. nginx+PHP+memcache+mysqld) into a single container with a hacky shell script around them because we needed them off bare metal yesterday but there's no dev resources to properly adapt the application. There's definitely "many" of those in the world, but it doesn't speak to picking tooling to support that by default.
- ghoward 4y ago
  Oi, thank you for mentioning tini. I'm working on an init named tini, but it appears krallin got there first. Will have to think of a new name for mine.
  - lostmsu 4y ago
    mint (as for min init)

Running dumb-init in a shell would not have helped. Running dumb-init (or tini or whatever, there's lots of these) when your application can handle SIGTERM itself is unnecessary. These are tools for when you don't control an executable in the stack and it doesn't do the right thing itself.

richardwhiuk 4y ago
You won't get a corefile if you are PID 1, which is a pretty good reason to use tini.
- morelisp 4y ago
  While I suppose this is true, I have never needed to examine a core dump from a container, I don't believe our Kubernetes deployment can collect them at the moment (or at least I would need to ask our ops team where to put them / how to get them once dumped), and to a first approximation zero of the services I run containerized produce helpful core dumps in the first place. It's not even out of the question for us to have more RAM used by one process than free disk space. (I do have a lot of experience with helpful corefiles, it's just been totally orthogonal to my experience with "at-scale" backend deployment. Different languages, different dev/ops relations, different attitudes towards abends in general.)
pm90 4y ago
> when your application can handle SIGTERM itself is unnecessary. Thats a big “when”. Many applications don’t. e.g. they spawn child processes which they forget about
- morelisp 4y ago
  Container inits don't magically get you clean shutdown if your application doesn't handle SIGTERM, it just gets you, let's say, fast shutdown. (And "fast", if you're not going to shutdown cleanly, might be much worse!) If you don't want connection issues you'll still need to write a SIGTERM handler, and then you don't need the container init anymore. And "many" is kind of a cop-out word; yes, I'm sure by absolute number it's a lot but: - If you're in e.g. Go, you spawn thousands of child green threads but it's trivial to route a SIGTERMable context into all of them and wait for only the relevant ones. - If you're in e.g. Python, you'll be running in an application server that handles it and you're probably not spawning many threads yourself because it's usually pointless. - If you're any piece of software written to run outside of containers, that's also what non-container environments have used for decades to handle smooth upgrades. - etc. for other common cases in the end, "spawning threads" is no excuse for not handling clean listener shutdown and seems to be more about how "Unix-like" the application is and not threading - e.g. old Java programs tend to be worst in my experience. I've used tini et al mostly when putting entire legacy environments (e.g. nginx+PHP+memcache+mysqld) into a single container with a hacky shell script around them because we needed them off bare metal yesterday but there's no dev resources to properly adapt the application. There's definitely "many" of those in the world, but it doesn't speak to picking tooling to support that by default.
ghoward 4y ago
Oi, thank you for mentioning tini. I'm working on an init named tini, but it appears krallin got there first. Will have to think of a new name for mine.
- lostmsu 4y ago
  mint (as for min init)