3 ms·

Yeah, sending SIGKILL to pid1 of a container will insta-kill the whole thing. But the network resource stuff does also include removing iptables rules, so it's

by cyphar 6y ago

Yeah, sending SIGKILL to pid1 of a container will insta-kill the whole thing. But the network resource stuff does also include removing iptables rules, so it's not something that you really want to be skipping. It feels like the network resources being cleaned up should be a background job rather than blocking the kill (after all, it's not critical to stopping the container and most resources will be freed by the network namespace dying). Also (if you're not already aware of this), you should note that if you use "docker exec" or share pid namespaces between containers, this trick will no longer work because your saved pid is not the pid1 of the namespace (runc has a "kill everything in the container" mode to work around this -- so I'd suggest using "runc kill SIGKILL" rather than doing it manually but it's probably not that important). Have you opened a bug report in Docker upstream to see if they can improve the situation (perhaps by putting networking cleanup jobs to a background goroutine)?

cbrewster 6y ago
Indeed, for our use-case we want an insta-kill so we can free up our global lock on the container. But the docker daemon will still get notified of the container death and run any sort of cleanup. This just gives us an opportunity to free up the global lock before getting stuck waiting on the cleanup. > Have you opened a bug report in Docker upstream to see if they can improve the situation (perhaps by putting networking cleanup jobs to a background goroutine)? I have not yet, but I plan to! We figured we'd work around it ourselves instead of waiting for a potential fix upstream.
- cyphar 6y ago
  Yup, I forgot to mention that dockerd will get the death event and clean up like it would if pid1 died normally. My comment was more about what a patch upstream would look like. And I'm glad to hear you plan on opening an upstream issue about it. Sadly it's quite common for folks to work around issues which we could've fixed upstream but were never told about, so glad that you're bucking that trend. :D
  - xorcist 6y ago
    That has historically not been reliable in any version of docker. A proper solution would probably require restructuring the whole thing.
  - alias_neo 6y ago
    Unfortunately, the trend is the way it is because historically Docker has demonstrated contempt towards the tickets people have raised and many issues people suffer from have never been fixed. Those of us who need to get things done have taken to just working around it and moving on. If this attitude has changed, it should be communicated publicly so people can reset and come back to Docker with such issues.

by cyphar 6y ago

cbrewster 6y ago

Indeed, for our use-case we want an insta-kill so we can free up our global lock on the container. But the docker daemon will still get notified of the container death and run any sort of cleanup. This just gives us an opportunity to free up the global lock before getting stuck waiting on the cleanup. > Have you opened a bug report in Docker upstream to see if they can improve the situation (perhaps by putting networking cleanup jobs to a background goroutine)? I have not yet, but I plan to! We figured we'd work around it ourselves instead of waiting for a potential fix upstream.

cyphar 6y ago
Yup, I forgot to mention that dockerd will get the death event and clean up like it would if pid1 died normally. My comment was more about what a patch upstream would look like. And I'm glad to hear you plan on opening an upstream issue about it. Sadly it's quite common for folks to work around issues which we could've fixed upstream but were never told about, so glad that you're bucking that trend. :D
- xorcist 6y ago
  That has historically not been reliable in any version of docker. A proper solution would probably require restructuring the whole thing.
- alias_neo 6y ago
  Unfortunately, the trend is the way it is because historically Docker has demonstrated contempt towards the tickets people have raised and many issues people suffer from have never been fixed. Those of us who need to get things done have taken to just working around it and moving on. If this attitude has changed, it should be communicated publicly so people can reset and come back to Docker with such issues.