3 ms·

What I would really want is just support for the common Windows-like case: Disallow opening the file by another process for reading or writing, paired with a fl

by Asooka 10y ago

What I would really want is just support for the common Windows-like case: Disallow opening the file by another process for reading or writing, paired with a flag you can pass to open(). Just these two: open("file1", O_CREAT | O_EXCL | O_LOCK_RW); open("file2", O_CREAT | O_EXCL | O_LOCK_W); First atomically creates and locks a file if it didn't already exist and forbids other processes to open it for either reading or writing. Second does the same, but permits other processes to open it for reading. If you want to release the lock, maybe you can do it with fcntl, or maybe you just open the file again and close the original descriptor. Locks are always tied to descriptors, not like epoll. You aren't required to combine the lock with creat|excl like in the example, you can open and lock an existing file as long as the lock won't conflict with another process's open fd. That is, if process A has opened file X for reading, process B can open file X with O_LOCK_W, after which no other process can open file X for writing, but process B can't open it with O_LOCK_RW. Multiple processes can have O_LOCK_W but only a single one can have O_LOCK_RW. If the file system can't support locking (NFS et. al) return -1, set errno to EINVAL. You can pair this with fcntl to take a lock with an optional timeout on an already open file. This covers the usual case of "I don't want someone else to mess with this file while I do things with it", while avoiding the explosion of intricate states that region locking brings, and lets you create-and-lock atomically a file.

jo909 10y ago
Why prevent others from reading the file? I guess mainly to avoid them reading mid-update in an inconsistent state. But preventing the read completely until you are done writing the file is arguably also not "consistent", the reader wants a file and gets an error instead. So now the reader has to either error out or implement some error handling, like waiting a bit which is rather impractical. But wouldn't it be nicer if he just got the "old" version of the file until you are finished with the update and "commit" the new file?
- Asooka 10y ago
  Yes, it would be quite nice. But this version is simpler, and you can implement your version by writing to a temporary file, then doing an atomic rename. My operating model assumes a user creating some kind of content using an application. E.g. Photoshop is running and the user is editing a file. Photoshop can very well write partial updates to the file, so it's important that nobody else write to it, or you end up with a corrupted file. Maybe just opening the file with no locking mode should always succeed though, or there can be an additional LOCK_NONE for that case (I don't want to participate in the locking scheme and yes I want to read whatever data is there). In that case you're saying "I just want to read what's there even if it is not consistent right now". The LOCK constants should probably also be named FORBID, in the sense that you're opening the file and FORBIDding the operation from happening while you have it open. So you can forbid writing (because inconsistent data is nonsensical for this case) or reading and writing (because you'll be updating the file). But I think the LOCK names are easier to use, since you're specifying the operation you'll be doing and can be understood analogous to read/write locks.
  - jo909 10y ago
    You can also "forbid" others from reading and writing your file if use a new temporary file they don't know about, and weakly enforce that by removing the file system entry with unlink while keeping the fd open, then link it again later. Part of the goal is IMHO not having to write out the entire file every time, only because you changed a few bytes. The atomic renaming can't do that, to my knowledge. I'm really only wondering about the forbidding reads part, and if that gains anything useful. For a process that knows about and participates in the locking process that's already possible, but mandatory file locks are also for processes that did not participate. Like for example cat, the backup process etc.

by Asooka 10y ago

jo909 10y ago

Why prevent others from reading the file? I guess mainly to avoid them reading mid-update in an inconsistent state. But preventing the read completely until you are done writing the file is arguably also not "consistent", the reader wants a file and gets an error instead. So now the reader has to either error out or implement some error handling, like waiting a bit which is rather impractical. But wouldn't it be nicer if he just got the "old" version of the file until you are finished with the update and "commit" the new file?

Asooka 10y ago
Yes, it would be quite nice. But this version is simpler, and you can implement your version by writing to a temporary file, then doing an atomic rename. My operating model assumes a user creating some kind of content using an application. E.g. Photoshop is running and the user is editing a file. Photoshop can very well write partial updates to the file, so it's important that nobody else write to it, or you end up with a corrupted file. Maybe just opening the file with no locking mode should always succeed though, or there can be an additional LOCK_NONE for that case (I don't want to participate in the locking scheme and yes I want to read whatever data is there). In that case you're saying "I just want to read what's there even if it is not consistent right now". The LOCK constants should probably also be named FORBID, in the sense that you're opening the file and FORBIDding the operation from happening while you have it open. So you can forbid writing (because inconsistent data is nonsensical for this case) or reading and writing (because you'll be updating the file). But I think the LOCK names are easier to use, since you're specifying the operation you'll be doing and can be understood analogous to read/write locks.
- jo909 10y ago
  You can also "forbid" others from reading and writing your file if use a new temporary file they don't know about, and weakly enforce that by removing the file system entry with unlink while keeping the fd open, then link it again later. Part of the goal is IMHO not having to write out the entire file every time, only because you changed a few bytes. The atomic renaming can't do that, to my knowledge. I'm really only wondering about the forbidding reads part, and if that gains anything useful. For a process that knows about and participates in the locking process that's already possible, but mandatory file locks are also for processes that did not participate. Like for example cat, the backup process etc.