4 ms·

Just last week I was debugging an issue with a library that used SQLite internally, and I noticed that whenever I shut down, it would leave all these "something

by OskarS 6y ago

Just last week I was debugging an issue with a library that used SQLite internally, and I noticed that whenever I shut down, it would leave all these "something.db-shm" and "something.db-wal" files around in addition to the "something.db" file. My hunch was the library wasn't properly deinitializing SQLite, and indeed that was the case, if you call sqlite3_close() properly, these files are deleted (they're essentially journal files for SQLite). The library was essentially doing what you're suggesting, ending without doing the proper cleanup. Destruction, in general, is one of the most complex areas of software development in my opinion. You're right, blocking or throwing errors during destruction is terrible, but you really do have to genuinely clean things up when exiting, you can't leave messes behind like this. However, in addition to that: you can't ever count on destruction occuring, because it's entirely possible that your program crashes, the user kills the process, the computer runs out of battery (or someone janks the cable out) or some other random thing causes your process to exit prematurely. You really need a lot of nuance in this area, and I don't agree with "cleanup at shutdown is a code smell". But you're certainly right that you can go too far in the other direction as well: destruction has to be fast, non-blocking, error-free, and never required.

gpderetta 6y ago
> you can't ever count on destruction occuring, because it's entirely possible that your program crashes, the user kills the process, the computer runs out of battery (or someone janks the cable out) or some other random thing causes your process to exit prematurely. Well yes, that's why you can't wait for shutdown time to cleanup your mess. In this case, either the files need to survive process exit or they can be unlinked as soon as they are created. Another option is to have very simple supervisor process whose only job is to cleanup. For a DB like sqlite it does makes sense to have some amount of shutdown cleanup to close transactions and mark the log as stale and avoid any recovery on startup, but it should be considered an optimization and not be a requirement. To make this short, I do in fact agree with your nuanced view. Pragmatism always trumps dogmatism in software engineering.
- dataflow 6y ago
  > In this case, either the files need to survive process exit or they can be unlinked as soon as they are created. The latter literally isn't possible on Windows. And I'd argue it's a good thing because you shouldn't do that in this case, though either way you don't have that option anyway. The notion of having a supervisory process just to clean up your mess that you had every chance to clean up is just poor design. Supervisory processes are for when you can't do that and there's critical work left over, not for cleaning up random clutter you just didn't bother to try to handle at the correct time. Imagine if every program that used SQLite spawned an extra supervisory process? Now imagine if this was true for every program that needed temporary files as long as it was running? That's obviously just terrible. Your code needs to clean up after itself as much as it can, whether via RAII or otherwise, but you can't just throw out the notion of cleaning things up on shutdown (or any other point) altogether!
  - gpderetta 6y ago
    you do not have a chance to cleanup if you are application is killed anyway. So either don't make a mess, make sure that you can clean up no matter what or live with it.
    - dataflow 6y ago
      > you do not have a chance to cleanup if you are application is killed anyway. Then you live with it in that case. That doesn't mean you should litter when the application is terminating gracefully. You clean up on shutdown when you are able. I don't see what's contentious about this.
dataflow 6y ago
Yeah I'd argue these problems happen because people don't take advantage of RAII (maybe because they don't like it or their language doesn't support it), and then forget/neglect to do the proper cleanups at the correct time. You want to design things so that they encapsulate their correctness by design. For a thread, that means joining it on destruction (because that should happen in general). For a mutex, it means not unlocking it on destruction (because that signals another part of the code is actually incorrect). For lock guard, it means unlocking on destruction. For files, it means closing them on destruction. Etc. You can't just reduce everything to "always" or "never".

by OskarS 6y ago

gpderetta 6y ago

> you can't ever count on destruction occuring, because it's entirely possible that your program crashes, the user kills the process, the computer runs out of battery (or someone janks the cable out) or some other random thing causes your process to exit prematurely. Well yes, that's why you can't wait for shutdown time to cleanup your mess. In this case, either the files need to survive process exit or they can be unlinked as soon as they are created. Another option is to have very simple supervisor process whose only job is to cleanup. For a DB like sqlite it does makes sense to have some amount of shutdown cleanup to close transactions and mark the log as stale and avoid any recovery on startup, but it should be considered an optimization and not be a requirement. To make this short, I do in fact agree with your nuanced view. Pragmatism always trumps dogmatism in software engineering.

dataflow 6y ago
> In this case, either the files need to survive process exit or they can be unlinked as soon as they are created. The latter literally isn't possible on Windows. And I'd argue it's a good thing because you shouldn't do that in this case, though either way you don't have that option anyway. The notion of having a supervisory process just to clean up your mess that you had every chance to clean up is just poor design. Supervisory processes are for when you can't do that and there's critical work left over, not for cleaning up random clutter you just didn't bother to try to handle at the correct time. Imagine if every program that used SQLite spawned an extra supervisory process? Now imagine if this was true for every program that needed temporary files as long as it was running? That's obviously just terrible. Your code needs to clean up after itself as much as it can, whether via RAII or otherwise, but you can't just throw out the notion of cleaning things up on shutdown (or any other point) altogether!
- gpderetta 6y ago
  you do not have a chance to cleanup if you are application is killed anyway. So either don't make a mess, make sure that you can clean up no matter what or live with it.
  - dataflow 6y ago
    > you do not have a chance to cleanup if you are application is killed anyway. Then you live with it in that case. That doesn't mean you should litter when the application is terminating gracefully. You clean up on shutdown when you are able. I don't see what's contentious about this.

dataflow 6y ago

Yeah I'd argue these problems happen because people don't take advantage of RAII (maybe because they don't like it or their language doesn't support it), and then forget/neglect to do the proper cleanups at the correct time. You want to design things so that they encapsulate their correctness by design. For a thread, that means joining it on destruction (because that should happen in general). For a mutex, it means not unlocking it on destruction (because that signals another part of the code is actually incorrect). For lock guard, it means unlocking on destruction. For files, it means closing them on destruction. Etc. You can't just reduce everything to "always" or "never".