4 ms·

While this is great, this only cleans up temporary files on a clean exit (well, depending on the shell), and no trap can catch SIGKILL. I much prefer cleaning u

While this is great, this only cleans up temporary files on a clean exit (well, depending on the shell), and no trap can catch SIGKILL. I much prefer cleaning up my files _before_ I use them, like: ``` touch 'temp.txt' exec 3 < 'temp.txt' exec 4 > 'temp.txt' rm -f 'temp.txt' ``` Now I have file descriptors 3 and 4 available for reading and writing, respectively, and even if I pulled the plug out from my desktop the file will not be anywhere on my filesystem.

m1keil 3y ago
If I understand correctly, this works more like a pipe and not a file in the sense that you can only read what you wrote once, right? But a nice trick nonetheless.
- mplewis9z 3y ago
  > you can only read what you wrote once In the way I’ve written it, yes. But you can open as many file descriptors as you need before you `rm` the file, and they each will keep their own position in the file. You can’t do a dynamic number of reads through the file (at least without dumping the contents somewhere else first), but a fixed number greater than one is absolutely possible.
  - m1keil 3y ago
    Ah, nice. Thanks!
hkwerf 3y ago
Same here. I'm maintaining a script which generate a file, sets up a loopback device and dmcrypt, then mounts the resulting map. Every of these operations but the mount can be done in a way that the kernel will remove them once they're not needed anymore, i.e. unmounting the filesystem leads to the dm-crypt map being discarded, leads to the loopback device being removed and the space reserved by the file being freed. This even persists past the exit of the script.
mbwgh 3y ago
Can you use something similar for lock files? For instance, I have some script which looks like this: function cleanup() { pkill -P $$ # Kill child processes. rm -f myscript.lock } trap cleanup exit lockfile -r 0 myscript.lock # Try to create lock file once. if [[ $? != 0 ]]; then echo ":(" exit 1 fi # Work work... cleanup Like you said, the cleanup handler is not invoked if the script receives a SIGKILL. Since the lock file needs to be present in the FS (contrary to a tmp file), maybe this is some inherent limitation, but I would be curious if some better approach exists.
- mplewis9z 3y ago
  If you're only trying to coordinate among sub-processes of your top-level script (ones that are at some point forked from the root script and duplicate its file descriptors), there's probably a way to get an approach similar to mine working, though I'd have to think a bit about exactly how to do so. Probably something along the lines of using a FIFO/pipe file in a similar way that I use a normal file in the original snippet. If you're trying to coordinate multiple invocations of the same script, you're out of luck trying to adapt my approach - something has to manage a resource for multiple unrelated processes, and thus they're all going to need a stable identifier of some sort to access it. There might be something mildly better than a filesystem-based lockfile, but it's almost certainly not worth the complexity/unfamiliarity to other people who might use your code. You're probably best off putting a file in a dedicated temporary directory like `/tmp` and relying on out-of-band cleanup to catch things like SIGKILL that will fall through the cracks of your normal cleanup.
  - mbwgh 3y ago
    Yeah my use case is unfortunately the latter. Maybe the out-of-band cleanup you suggest might be something like a simple daemon which is invoked with `$$` and `path/to/myscript.lock`. Which would then poll for the presence of a process with the given PID and deletes the lock file.
- lionkor 3y ago
  I believe the EXIT trap is not clearly defined in POSIX, so you should also trap the other signals you care about (SIGINT, SIGTERM, ...)
hawski 3y ago
There is still a small window between touch and rm for the file to not be cleaned up. There is no easy way around it with standard shell and coreutils, but there could be a tool that would use memfd_create(2) and exec user's command passing the /dev/fd/XXX filename somehow. A simple case could work like this: $ memfile /usr/bin/echo /dev/fd/3 $

m1keil 3y ago

If I understand correctly, this works more like a pipe and not a file in the sense that you can only read what you wrote once, right? But a nice trick nonetheless.

mplewis9z 3y ago
> you can only read what you wrote once In the way I’ve written it, yes. But you can open as many file descriptors as you need before you `rm` the file, and they each will keep their own position in the file. You can’t do a dynamic number of reads through the file (at least without dumping the contents somewhere else first), but a fixed number greater than one is absolutely possible.
- m1keil 3y ago
  Ah, nice. Thanks!

hkwerf 3y ago

Same here. I'm maintaining a script which generate a file, sets up a loopback device and dmcrypt, then mounts the resulting map. Every of these operations but the mount can be done in a way that the kernel will remove them once they're not needed anymore, i.e. unmounting the filesystem leads to the dm-crypt map being discarded, leads to the loopback device being removed and the space reserved by the file being freed. This even persists past the exit of the script.

mbwgh 3y ago

Can you use something similar for lock files? For instance, I have some script which looks like this: function cleanup() { pkill -P $$ # Kill child processes. rm -f myscript.lock } trap cleanup exit lockfile -r 0 myscript.lock # Try to create lock file once. if [[ $? != 0 ]]; then echo ":(" exit 1 fi # Work work... cleanup Like you said, the cleanup handler is not invoked if the script receives a SIGKILL. Since the lock file needs to be present in the FS (contrary to a tmp file), maybe this is some inherent limitation, but I would be curious if some better approach exists.

mplewis9z 3y ago
If you're only trying to coordinate among sub-processes of your top-level script (ones that are at some point forked from the root script and duplicate its file descriptors), there's probably a way to get an approach similar to mine working, though I'd have to think a bit about exactly how to do so. Probably something along the lines of using a FIFO/pipe file in a similar way that I use a normal file in the original snippet. If you're trying to coordinate multiple invocations of the same script, you're out of luck trying to adapt my approach - something has to manage a resource for multiple unrelated processes, and thus they're all going to need a stable identifier of some sort to access it. There might be something mildly better than a filesystem-based lockfile, but it's almost certainly not worth the complexity/unfamiliarity to other people who might use your code. You're probably best off putting a file in a dedicated temporary directory like `/tmp` and relying on out-of-band cleanup to catch things like SIGKILL that will fall through the cracks of your normal cleanup.
- mbwgh 3y ago
  Yeah my use case is unfortunately the latter. Maybe the out-of-band cleanup you suggest might be something like a simple daemon which is invoked with `$$` and `path/to/myscript.lock`. Which would then poll for the presence of a process with the given PID and deletes the lock file.
lionkor 3y ago
I believe the EXIT trap is not clearly defined in POSIX, so you should also trap the other signals you care about (SIGINT, SIGTERM, ...)

hawski 3y ago

There is still a small window between touch and rm for the file to not be cleaned up. There is no easy way around it with standard shell and coreutils, but there could be a tool that would use memfd_create(2) and exec user's command passing the /dev/fd/XXX filename somehow. A simple case could work like this: $ memfile /usr/bin/echo /dev/fd/3 $