4 ms·

"...the flock command uses flock() and it's pretty much your best bet for locking in shell scripts." I find that mkdir is the easiest and most reliable lock be

by chasil 4y ago

"...the flock command uses flock() and it's pretty much your best bet for locking in shell scripts." I find that mkdir is the easiest and most reliable lock between shell scripts under my control, and I use this approach for several variants of inotify (incron, path units, and inotifywait). #!/bin/sh if mkdir /tmp/dirlock 2> /dev/null then trap 'rmdir /tmp/dirlock' EXIT (critical section) fi This also does not require a subshell, so it is more efficient as it avoids a fork() - unlike the flock shell utility. I don't try to do this over NFS, but mkdir is guaranteed to be atomic, and I have never seen this fail. It's also more secure to lock in a private directory rather than /tmp, unless you want the lock mechanism to be promiscuous for some reason.

josephcsible 4y ago
Unlike real locks, your approach has the downside that an unexpected exit (e.g., kill -9) will stay locked until someone manually intervenes.
- ASalazarMX 4y ago
  Sometimes that's precisely what you want, if the unfinished script should run again without being checked.
- chasil 4y ago
  This was actually a problem on dash, so I added several more signals (INT TERM etc.). Nothing saves you from kill -9, as we all know. I did add a reboot entry to cron (@reboot with Vixie cron) to remove the directory. I can say that this method has processed millions of Oracle archived logs with no errors.
  - bombcar 4y ago
    I wonder if the fact that you can delete a file and still access its contents as long as a file descriptor is still open would allow some dorty magic. Probably not because iirc you can create a file with the same name right after deletion even if a fd is open to the old one.
    - chasil 4y ago
      SQLite will open temporary files (for sorting and sundry functions), then immediately unlink() them. This is allowed behavior under POSIX. The NFS trick was documented in "Why NFS Sucks" by Olaf Kirch. "Some other cases are a bit stranger. One such case is the ability to write to an open unlinked file. POSIX says an application can open a file for reading and writing, unlink it, and continue to do I/O on it. The file is not supposed to go away until the last application closes it. "This is difficult to do over NFS, since traditionally, the NFS server has no concept of “open” files (this was added in NFSv4, however). So when a client removes a file, it will be gone for good, and the file handle is no longer valid— and attempt to read from or write to that file will result in a “Stale file handle” error. "The way NFS traditionally kludges around this is by doing what has been dubbed a “silly re- name.” When the NFS client notices during an unlink call that one or more applications still hold an open file descriptor to this file, it will not send a REMOVE call to the server. Instead, it will rename the file to some temporary file name, usually .nfsXXX where XXX is some hex number. This file will stay around until the last application closes its open file descriptor, and only then will the NFS client send the final REMOVE call to the server that gets rid of this renamed file. "This sounds like a rather smart sleight of hand, and it is—up to a point. First off, this does not work across different clients. But that should not come as a surprise given the lack of cache consistency." https://www.kernel.org/doc/ols/2006/ols2006v2-pages-59-72.pdf https://www.kernel.org/doc/ols/2006/ols2006v2-pages-59-72.pd...
  - i_am_a_peasant 4y ago
    > Nothing saves you from kill -9, as we all know. You making a syscall and getting the kernel stuck somewhere and having your process in a D state will keep you alive ;D.
  - bheadmaster 4y ago
    > Nothing saves you from kill -9, as we all know. True, but it doesn't have to be you who does the cleanup. You could create a background process and establish a temporary UNIX socket connection, so when the parent dies, the socket closes and the child process cleans up everything.
- deleted 4y ago
  [deleted]
jakeogh 4y ago
I spent some time formalizing mkdir/noclobber locks (with lots of help from #bash): https://github.com/jakeogh/commandlock https://github.com/jakeogh/commandlock Long term work in progress... needs a simple Makefile
siebenmann 4y ago
We used to use this sort of locking in (frequently running) system cron jobs and the like. Then these jobs started getting killed off by Linux OOM on some systems and we ran into the downsides of locks that don't automatically clear if something goes wrong, and switched to flock(1) based locks (fortunately on the local system, so we're not affected by NFS issues there). (I'm the author of the linked-to entry.)
- chasil 4y ago
  Switch the shell from bash to dash (which is the default on Ubuntu) if that is part of the memory problem. Dash is nearly 10x smaller, rumored to be 4x faster than bash. Otherwise, have you compiled any object code with -Os?
  - siebenmann 4y ago
    Oops, my fallible memory bit me. We weren't specifically running into OOM, but into strict overcommit (which we had turned on on some machines). OOM will only kill big things, so it would be weird for bash (as /bin/sh) or small Python programs to get killed off. But strict overcommit favors already running things (who've already claimed their memory) over newly started cron jobs. (You could criticize the shell for needing to do dynamic allocation in failure paths and so being exposed to malloc() failing, but this is a hard area and lots of code assumes it can malloc() on demand, some of it in the C library.)

by chasil 4y ago

josephcsible 4y ago

Unlike real locks, your approach has the downside that an unexpected exit (e.g., kill -9) will stay locked until someone manually intervenes.

ASalazarMX 4y ago
Sometimes that's precisely what you want, if the unfinished script should run again without being checked.
chasil 4y ago
This was actually a problem on dash, so I added several more signals (INT TERM etc.). Nothing saves you from kill -9, as we all know. I did add a reboot entry to cron (@reboot with Vixie cron) to remove the directory. I can say that this method has processed millions of Oracle archived logs with no errors.
- bombcar 4y ago
  I wonder if the fact that you can delete a file and still access its contents as long as a file descriptor is still open would allow some dorty magic. Probably not because iirc you can create a file with the same name right after deletion even if a fd is open to the old one.
  - chasil 4y ago
    SQLite will open temporary files (for sorting and sundry functions), then immediately unlink() them. This is allowed behavior under POSIX. The NFS trick was documented in "Why NFS Sucks" by Olaf Kirch. "Some other cases are a bit stranger. One such case is the ability to write to an open unlinked file. POSIX says an application can open a file for reading and writing, unlink it, and continue to do I/O on it. The file is not supposed to go away until the last application closes it. "This is difficult to do over NFS, since traditionally, the NFS server has no concept of “open” files (this was added in NFSv4, however). So when a client removes a file, it will be gone for good, and the file handle is no longer valid— and attempt to read from or write to that file will result in a “Stale file handle” error. "The way NFS traditionally kludges around this is by doing what has been dubbed a “silly re- name.” When the NFS client notices during an unlink call that one or more applications still hold an open file descriptor to this file, it will not send a REMOVE call to the server. Instead, it will rename the file to some temporary file name, usually .nfsXXX where XXX is some hex number. This file will stay around until the last application closes its open file descriptor, and only then will the NFS client send the final REMOVE call to the server that gets rid of this renamed file. "This sounds like a rather smart sleight of hand, and it is—up to a point. First off, this does not work across different clients. But that should not come as a surprise given the lack of cache consistency." https://www.kernel.org/doc/ols/2006/ols2006v2-pages-59-72.pdf https://www.kernel.org/doc/ols/2006/ols2006v2-pages-59-72.pd...
- i_am_a_peasant 4y ago
  > Nothing saves you from kill -9, as we all know. You making a syscall and getting the kernel stuck somewhere and having your process in a D state will keep you alive ;D.
- bheadmaster 4y ago
  > Nothing saves you from kill -9, as we all know. True, but it doesn't have to be you who does the cleanup. You could create a background process and establish a temporary UNIX socket connection, so when the parent dies, the socket closes and the child process cleans up everything.
deleted 4y ago
[deleted]

jakeogh 4y ago

I spent some time formalizing mkdir/noclobber locks (with lots of help from #bash): https://github.com/jakeogh/commandlock https://github.com/jakeogh/commandlock Long term work in progress... needs a simple Makefile

siebenmann 4y ago

We used to use this sort of locking in (frequently running) system cron jobs and the like. Then these jobs started getting killed off by Linux OOM on some systems and we ran into the downsides of locks that don't automatically clear if something goes wrong, and switched to flock(1) based locks (fortunately on the local system, so we're not affected by NFS issues there). (I'm the author of the linked-to entry.)

chasil 4y ago
Switch the shell from bash to dash (which is the default on Ubuntu) if that is part of the memory problem. Dash is nearly 10x smaller, rumored to be 4x faster than bash. Otherwise, have you compiled any object code with -Os?
- siebenmann 4y ago
  Oops, my fallible memory bit me. We weren't specifically running into OOM, but into strict overcommit (which we had turned on on some machines). OOM will only kill big things, so it would be weird for bash (as /bin/sh) or small Python programs to get killed off. But strict overcommit favors already running things (who've already claimed their memory) over newly started cron jobs. (You could criticize the shell for needing to do dynamic allocation in failure paths and so being exposed to malloc() failing, but this is a hard area and lots of code assumes it can malloc() on demand, some of it in the C library.)