6 ms·
If you are the one designing your applications, don't rely on SIGTERM and clean shutdowns. Design them so SIGKILL is the default way to stop them such that no d
by rdtsc 12y ago
If you are the one designing your applications, don't rely on SIGTERM and clean shutdowns. Design them so SIGKILL is the default way to stop them such that no data is corrupted and persistent state stays consistent. Then you don't have to rely on some rarely executed "recovery" code, or even worse take chances of what might happen if SIGTERM doesn't work.
Handle SIGTERM only if you get a blackbox-like application (a database that will break if you pull the power plug on it) and so on.
- GauntletWizard 12y agoI don't disagree with the intent (Make crash-resistant, journaling programs), but TERM is an incredibly useful tool. Lame-Ducking (when your program continues processing current requests but refuses new ones) is a great way to handle rollouts and other planned maintenance in a way that minimises or prevents user visibility.
- dimman 12y agoA proper init system will send you a SIGTERM before a normal reboot/shutdown, and also for stopping the program. I do agree that "power failure safety" is a good way to go, but ignoring to handle SIGTERM if there is a use for it is just ignorant.
- rdtsc 12y ago> I do agree that "power failure safety" is a good way to go, So you have write code to handle abrupt power shutoff? And test it, and make sure it doesn't break your data at rest or your overwall service. And you have to write to handle a paralle set of code to also handle SIGTERM. > , but ignoring to handle SIGTERM if there is a use for it is just ignorant. It seems having 2 sets of shutdown/failure code (one abrupt power failure, "one gentle" is the more ignorant path.
- hyperpape 12y agoI thought you couldn't handle SIGKILL, so how do you have two sets of code? SIGTERM just lets you do a bit more of the operations you've guaranteed are always safe.
- rdtsc 12y ago> I thought you couldn't handle SIGKILL The process that gets sent the SIGKILL can't handle the moment after it gets the SIGKILL. But, the rest of the system could handle the "killing" clenaup (other process and management of resource). That could be another process (a supervisor). Another process an another machine. Or it could be the same process when it gets restarted. First thing it always does is deal with its remnants and after-effects of when it was killed last. > SIGTERM just lets you do a bit more of the operations you've guaranteed are always safe. What does that mean? Can you expand a bit. I don't quite understnad "guaranteed are always safe". If the process that gets sent the SIGTERM catches it and does some something "clever". That clever part is never guaranteed. Because the power can cut out, SIGKILL and come unexpectedly (supervisor decides that SIGTERM handler took too long and send), etc.
- hyperpape 12y agoRight, but you engineered your application so that it only performs operations that can be safely interrupted by SIGKILL. The application deals with some kind of stream of data. You engineer it so that processing can be interrupted at any point, and on next startup, you're fine--your data is consistent. Now, once you have that system, I would think that handling SIGTERM is not so hard--you finish processing what's in flight already, while not trying to handle anything new. From your earlier work, you know that this can't corrupt data. Even if SIGKILL follows while you're trying to handle SIGTERM, you're no worse off than if you just had received SIGKILL in the first place. The downside seems to be that you now have to show that your system will reliably terminate if it receives SIGTERM.
- dimman 12y agoWhen you actually work with especially embedded devices, you know that there's a big difference. Powerloss/SIGKILL must not corrupt/break your data/application. That is _not_ the same thing as for a network application for instance being nice and sending goodbye messages, cleaning up resources like closing fd's etc upon a SIGTERM. So no you design for being robust and to handle SIGTERM if applicable. You can never know when you'll get killed, that's why your design have to be solid all the way through.
- khrf 12y agoIf you think of it from a developer's view, you'll come to conclusion that you can't create a program that doesn't need to cleanup things before termination. Unless it's very useless program that doesn't process any incoming data. That's why there is SIGTERM in operating systems.
- angersock 12y agoYou very much can--it depends on what you're doing. Supporting SIGTERM and certain other signals is just deciding how big a crater you want to make when the program goes down. That said, it's good practice to use things (write-ahead logging, idempotent operations, etc.) that at least try to mitigate the amount of debris.
- rdtsc 12y ago> That said, it's good practice to use things (write-ahead logging, idempotent operations, etc.) that at least try to mitigate the amount of debris. Yap that are some of the things I used. Append only mode for files. Code that handle roll-back to last consistent state that runs on every startup. External monitors and watchdog that know how to monitor and to extra cleanup is necessary.
- lomnakkus 12y agoThat's nonsense. All ACID databases handle the requirement that a SIGKILL doesn't corrupt data. There may be a non-trivial recovery cost if you experience an unexpected shutdown, but that's about it for any application that's serious about data integrity. EDIT: In fact they handle a much stronger requirement that a power failure doesn't corrupt data, but the point stands.
- vinceguidry 12y agoACID databases can do this because they have the powerful hand of math giving them superpowers. Most developers don't.
- 12y ago
- alanfranzoni 12y agoYour point of view is correct, from a purely theoretical point of view. But the real world doesn't always work this way. The recovery process after a SIGKILL may require manual intervention, or may leave some data in an inconsistent state. Not every software in the world has the amount of testing for hard kills as an ACID compliant RDBMS - I'd say all the contrary. So, I don't want to trigger bugs and lose time in complicate recoveries if I am absolutely not forced to do it. Let processes cleanup after themselves, and hit them hard only when and if needed.
- dap 12y agoI hear what you're saying, but the flip side is that all that software also won't work when you lose power or the system panics. I've found it much easier to just always use SIGKILL. That way, you're testing the recovery code all the time, especially during development when you restart, upgrade, or after a crash. You fix the bugs, and you have high confidence because you're exercising this path all the time.
- rdtsc 12y ago> from a purely theoretical point of view. But the real world doesn't always work this way. I have implemented practical running systems that work like that so it is practical not just academic papers. > But the real world doesn't always work this way. Sure, sometimes you don't have choice. You get a 3rd party library/service/db that will shit the bed if something pulls the plug on the machine. You can try to identify those parts and eliminate them or mitigate it as much as possible. But sometimes you can't. Then try to characterise their behavior. Maybe have some external cleanup logic if you care to ensure data doesn't get corrupted. And live with it. > Not every software in the world has the amount of testing for hard kills as an ACID compliant RDBMS - I'd say all the contrary. Well but it is something to strive for. If you write the software it benefits to understand what happens to the data. Knowing a bit about file syncing, what kernel does with dirty pages, how/if/when to use O_DIRECT, using append-only modes, a write-ahead log. Think about using checksums in your persistent data, take advantage of atomic file system operations (renames), etc. Well I certainly saw the benefit and the return from running a better, well tested, well designed service that operates that way. > Let processes cleanup after themselves, Think about that statement for a bit. How do you guarnatee processes "cleanup up after themselves". Say a junior tech trips over the power cord of the server, how does the process "clean up after itself"?