7 ms·
I've wanted somewhat different file system semantics, but close to the POSIX model: - Unit files. The unit of consistency is the entire file. Files are opened
by Animats 6y ago
I've wanted somewhat different file system semantics, but close to the POSIX model:
- Unit files. The unit of consistency is the entire file. Files are opened for writing, written, and closed. Then then become openable by other programs. Overwriting a file replaces the file as a unit; other readers see the old file until the writer closes and the reader reopens. This is the default and the case for most files. On a crash, the file reverts to the previous good file, if any. Can be memory-mapped as read only. File replacement can be done now through non-portable renaming gyrations. It should just work.
- Log files. Append-only mode, enforced. Can't seek back and overwrite. On a crash, the file recovers to some recent good write, with a correct end of file position. It does not tail off into junk.
- Temp files. Not persistent over restarts. Not backed up. Can be memory-mapped as read/write. On a crash, disappears.
- Managed files. These are for databases. Async I/O available. May have additional locking functions. Separate completion returns for "accepted data" (caller can reuse write buffer) and "committed data" (safely stored), so the database knows when the data has been safely stored. Can be memory-mapped as read/write. Intended for use by programs which are very aware of the file system semantics. On a crash, "committed data" should be intact but data not yet committed may be lost.
- wahern 6y ago> Overwriting a file replaces the file as a unit; other readers see the old file until the writer closes and the reader reopens. This is the semantic rename provides, and the strength of that semantic is a common complaint from many in the anti-POSIX crowd. I suspect you knew that already, but the number of syscalls would be similar in either case except that the latter file would need to be visible before the rename. Linux does provide O_TMPFILE + AT_SYMLINK_FOLLOW which provides the nearly ideal behavior. (Nearly because it still requires /proc/self/fd access, AFAIU.) > Log files. Append-only mode, enforced Both chflags (BSD) and chattr (Linux) provide append-only modes attached to the file/inode (instead of the file descriptor or open file table entry)[1].[2] Neither command nor their options are defined by POSIX, but adding more requirements to POSIX filesystem conformance goes against the grain of prevailing sentiments. > Temp files. Not persistent over restarts. Not backed up. Can be memory-mapped as read/write. On a crash, disappears. Again, the semantics of unlinking deliberately provide this, though it does make it invisible in the normal namespace. But there's a tension between leveraging the namespace for implicit semantics (Plan 9) vs adding a multitude of options and modes attached to each particular file (Windows). It's also not uncommon for temporary filesystems like /tmp to be reformatted on boot if the backing store is persistent. That's one reason to use multiple filesystems for a Unix install. [1] Note that O_APPEND mode is attached to the open file table entry, not the file descriptor, so the mode is inherited across dup and interprocess descriptor passing. Interestingly, on Linux /proc/self/fd (also /dev/fd, which is a symlink) has the semantics of open, not dup, so file table entry state like O_APPEND is lost; whereas on BSD /dev/fd has the semantics of dup so O_APPEND is preserved/enforced. [2] Solaris lacks chattr and chflags, but chmod supports an append-only option.
- deleted 6y ago[deleted]
- dan-robertson 6y agoRenaming is commonly proposed for rewriting files but does not provide the guarantees people usually think it does. Posix renames are only atomic during normal operation. But here atomic merely means that other syscalls will not return results for some partially-renamed state. The operation does not need to be atomic in the case of a crash (where here “atomic” means “you can’t read the file system in some partially renamed state after a crash”, a different definition but one which people commonly assume when they hear that rename is atomic). I’m not aware of any normal Linux file system where renames are always atomic over a crash
- dmm 6y agoZFS is really well behaved. It never reorders metadata in an observable way so renames should truely be atomic. In the event of a crash you'll get the old or the new name.
- fiddlerwoaroof 6y agoWhat I’d prefer to atomicity is automatic file versioning such that a file descriptor opened at time t0 always points to the same contents, unless you refresh it. You’d also need CoW and/or writing patches rather than rewriting the whole file and a GC/compactor for cleaning up transient versions of a file. The workflows this would be enable is that, when vim wants syntax checking, it writes a transient version of the file to the fs without updating the “HEAD” pointer and then hands a fd to the syntax checker that points to that new transient version.
- a1369209993 6y agoYeah, the way you're 'supposed' (as near as I can tell) to do this is: read 'foo' write 'foo~' hardlink 'foo~' onto 'foo' unlink 'foo~' on crash: if 'foo' exists: it's the correct state for the file unlink 'foo~' continue otherwise: hardlink 'foo~' onto 'foo' unlink 'foo~' I'm not sure Posix actually requires link and unlink to be atomic enough on crash for this work, but it seems to be fine for any sane filesystem (where the fs itself doesn't fall apart in presence of crashes).
- nitrogen 6y agoFile replacement can be done now through non-portable renaming gyrations. I was under the impression that POSIX requires renames to be atomic, and have relied on that in embedded Linux systems with UBIFS. Are there exceptions among truly compliant filesystems and OSes, and/or is the atomicity guarantee not as strong as I thought?
- simias 6y agoI agree with you, for me the main limitation of this system is that you have to be sure that the temporary file exists in the same fs as the target, which in general involves creating the temporary file in the same directory. It works well, but I always feel icky creating temporary files in the middle of the filesystem (which may end up lingering if my program crashes before I could issue the rename syscall). But beyond that I'm not aware of any portability issue, except maybe with weird filesystems like NFS in some non-standard configuration that doesn't implement strict locking.
- wahern 6y agoIt's atomic but it might not be persistent in the event of a crash. If the metadata wasn't committed then upon remounting the old file might reappear, and its likelihood of reappearing is independent of any other operations on that filesystem. Though, traditionally rename was not only atomic but also preferentially ordered ahead of other metadata operations so subsequent renames of other files wouldn't be visible if the older rename appeared. EDIT: Actually, I think the issue I had in mind was that writes to the new file might not be committed before the rename, so if you do open + write + rename + close, on a crash the rename might persist even though the write didn't. You technical should use fdatasync if you want the write + rename to be ordered. See, e.g., https://lwn.net/Articles/322823/ https://lwn.net/Articles/322823/ But this is such unintuitive behavior that I think that even for ext4 the more reliable behavior is still the default.
- jacobsenscott 6y agoIsn't this true of any atomic operation in the event of a crash? You'll either see the old data or the new data when the system comes back up.
- Someone 6y ago”Log files. Append-only mode, enforced. Can't seek back and overwrite. On a crash, the file recovers to some recent good write, with a correct end of file position. It does not tail off into junk.“ As wahern says, append-only is available in some OSes. I am not aware of any that guarantees the “recovers to some recent good write” part, though. UNIX-like OSes won’t support it, as they don’t have a way to write more than one byte atomically (https://en.wikipedia.org/wiki/Write_(system_call) https://en.wikipedia.org/wiki/Write_(system_call): The write function returns the number of bytes successfully written into the array, which may at times be less than the specified nbytes)
- magicalhippo 6y ago> Log files. Append-only mode, enforced. Anyone know of any file system which allow partial initial blocks? Ie a file can appear less than a full block long by adjusting its length, any where there's an offset into the first block where the file starts? Then you could easily do size-constrained rolling logs by simply adjusting the starting point of the file.
- j16sdiz 6y agoReiserFS
- Hello71 6y ago> - Log files. Append-only mode, enforced. Can't seek back and overwrite. On a crash, the file recovers to some recent good write, with a correct end of file position. It does not tail off into junk. Linux already semi-supports this with chattr +a, which is filesystem-specific, plus filesystem-specific mount options (e.g. data=journal). unfortunately, implementing the latter is non-trivial to do on a per-file basis. > - Temp files. Not persistent over restarts. Not backed up. Can be memory-mapped as read/write. On a crash, disappears. O_TMPFILE exists on Linux, and I don't know of any reason why it couldn't be added to POSIX. > - Managed files. These are for databases. Async I/O available. May have additional locking functions. Separate completion returns for "accepted data" (caller can reuse write buffer) and "committed data" (safely stored), so the database knows when the data has been safely stored. Can be memory-mapped as read/write. Intended for use by programs which are very aware of the file system semantics. On a crash, "committed data" should be intact but data not yet committed may be lost. io_uring exists now. these features are Linux-specific, but my point is that these features do exist now already, there is no need to wish for them.
- LgWoodenBadger 6y agoOne feature that I’ve wished for many times is a transactional filesystem, for metadata and the actual contents itself.
- inopinatus 6y agoThere are have been attempts at "phase tree" filesystems with essentially transactional write semantics for data and metadata simultaneously. Tux2 was one such nascent attempt that ended (rightly or wrongly) due to patent conflict with Netapp's WAFL [1]. The interesting thing is, I just took a look at what I believe were the patents in conflict, and they appear to have expired in 2015 [2,3]. Combining the phase tree concept with, say, Merkle trees might open up very interesting possibilities for large-scale reliable file storage. [1] https://linux.slashdot.org/story/00/10/13/2117258/tux2-the-filesystem-that-would-be-king https://linux.slashdot.org/story/00/10/13/2117258/tux2-the-f... [2] https://patents.google.com/patent/US5819292 https://patents.google.com/patent/US5819292 [3] https://patents.google.com/patent/US5963962 https://patents.google.com/patent/US5963962
- skissane 6y agoWindows has transactional NTFS. It was never widely adopted, in part because it requires use of a whole different filesystem API (CreateFileTransacted instead of CreateFile, etc). I think, if it had been more seamless, it may have been more successful. Given it is rarely used, Microsoft has marked it as deprecated for potential removal in future Windows versions.
- asveikau 6y agoIirc it was implemented with a pretty seamless API at the undocumented ntdll layer, which let you set a current transaction on a handle then do API calls as usual. The issue is that while NT mostly reasons about files as handles, Win32 deals primarily with the name for many operations. So they duplicated every api that took a filename.
- skissane 6y agoDoesn't surprise me to learn that the NT-level API is nicer. The NT API is in general much better thought out than the Win32 API. It is a real pity that Microsoft chose to leave the former largely undocumented and tell people to use the later instead.
- wolfd 6y agoAlong those lines, one thing that has bothered me is that we can't prepend files, even with modern SSDs with fancy controllers. Those controllers are jumbling the contents of a continuous disk onto different chips, and different sections on those chips. For things like MP4 metadata that usually need to be near the beginning of a file for the MP4 to be considered streamable, it can be quite cumbersome. Being unable to expand a file from the beginning means that fancy MP4 writers write a bunch of blank space (perhaps guessing the necessary size of the metadata portion), then after the stream data is written, go back and hope that they allocated enough blank space to fit the metadata within. If the writer was wrong, either it has to put the metadata at the end of the file (preventing streaming), or has to rewrite the entire file with a bigger buffer. Perhaps it is a wish that requires too many layers to be changed to be possible, but I like to dream.
- teraflop 6y agoThis is already possible on Linux with certain filesystems (notably xfs and ext4), as long as the size of the data you want to prepend is an integer multiple of the filesystem block size. https://lwn.net/Articles/629965/ https://lwn.net/Articles/629965/
- alkonaut 6y agoSadly even though a system might have one or more of those features, few programs can assume a specific file system (and many not even a specific OS!). So you'd have code branch into two parts, one where this is managed by the file system, and the other where it is manaed in user code (e.g. write to temp file, rename etc). This is the curse of the lowest common denominator. Even innovation makes it worse (more code, not less).
- pkphilip 6y agoSounds similar to a version control system to me - except that the version control system stores deltas. But there is nothing preventing it from periodically storing snapshots. Snapshots in a journaled file system would serve the same purpose. Of course, journaled file systems could also handle the meta data snapshots
- JdeBP 6y agoThe characterization in the headlined article that append-only has been "lost" since Multics isn't strictly true. It has been poorly implemented, but write and append rights are two separate things in NFSv4 ACLs. * https://www.freebsd.org/cgi/man.cgi?query=setfacl&sektion=1 https://www.freebsd.org/cgi/man.cgi?query=setfacl&sektion=1 * https://wiki.freebsd.org/NFSv4_ACLs https://wiki.freebsd.org/NFSv4_ACLs