3 ms·

> it's worth pointing out that what you describe is not obvious, and one can easily miss that there are two ways of writing a file to disk. Yeah, there's a dic

by jolmg 7y ago

> it's worth pointing out that what you describe is not obvious, and one can easily miss that there are two ways of writing a file to disk. Yeah, there's a dichotomy in how files are handled as presented to the user and how it's actually done. For example, despite most UIs presenting files as being opened, they're almost always closed and only opened when the program needs to do something to them at any given instant. > Case in point, it took me 17 years of programming to discover that "writing new files under the same filename" is a thing It took me many years (though not 17) to discover the opposite, that all forms of overwriting without changing the inode was possible. For many years, I was under the impression that one could modify files to the same length or a longer one, but that the core syscall interface for files (the calls that one is typically introduced to when learning about working with files) didn't allow for the shrinking of files. I made the wrong justifying assumption that this API design was to help against disk fragmentation. I also thought file-clobbering meant replacing files. I later learned that truncate(2) exists and allows for the arbitrary shrinking of files, but even now, I'm not sure that it's possible to shrink files arbitrarily on all systems with a unix-like file syscall interface. For example, Plan9 doesn't seem to have a truncate syscall.[1] It seemed to me that the only way file editors could modify a file to be shorter was to write a new one and replace the old one with it. I suppose they could have also clobbered files, but it didn't seem to me that it held any advantage over writing elsewhere and renaming, at least not while I wrongfully assumed that clobbering caused the inode to change. If anything, resorting to clobbering increases the risk of data-loss if there's any error while writing. Anyway, my point is that what you call non-obvious was what I intuitively thought was the only way file editors could work portably, and that this was due to my late introduction to the perhaps-non-portable truncate(2). [1] http://aiju.de/plan_9/plan9-syscalls http://aiju.de/plan_9/plan9-syscalls

TeMPOraL 7y ago
Come to think of it, I started programming on Windows, and my first introduction to the filesystem operations was iostream in C++. Being a kid then, I haven't learned about syscalls or inodes until a lot, lot later. I always imagined the filesystem as a more sophisticated version of an array of pairs <startAddress, length>. Thus I assumed shortening a file is trivial (just decrement the length), but making a file longer may require copying it over to somewhere else, if there isn't $newLength bytes available after the $startAddress. I believed that defragmentation meant removing gaps caused by shortening or deleting files. (N.b. this is how simple memory allocators work.) So in the end, for a long while, I hadn't had an issue that would make me double-check those assumptions.
account42 7y ago
The traditional way to truncate an existing file is using O_TRUNC in the open call. From your link it looks like Plan 9 has that flag. > In addition, there are [...] values that can be ORed with the omode: OTRUNC says to truncate the file to zero length before opening it
- jolmg 7y ago
  That's what I meant by clobbering. The thing is that you can't use that flag to say "shorten it by half" to keep the first half, but you can with truncate(2). My point was that if you can't shorten it by an arbitrary amount to avoid the redundancy of writing the same beginning of the file, then there's really no advantage to it over writing a new file to replace the old one. In fact, I thought clobbering (i.e. O_TRUNC) was nearly equivalent, only with the additional risk of data loss if there's an error while writing.
  - account42 7y ago
    Creating a new file also replaces all file metadata (creation time, permissions, ACLs, links, ...) while O_TRUNC only erases the contents.

by jolmg 7y ago

TeMPOraL 7y ago

Come to think of it, I started programming on Windows, and my first introduction to the filesystem operations was iostream in C++. Being a kid then, I haven't learned about syscalls or inodes until a lot, lot later. I always imagined the filesystem as a more sophisticated version of an array of pairs <startAddress, length>. Thus I assumed shortening a file is trivial (just decrement the length), but making a file longer may require copying it over to somewhere else, if there isn't $newLength bytes available after the $startAddress. I believed that defragmentation meant removing gaps caused by shortening or deleting files. (N.b. this is how simple memory allocators work.) So in the end, for a long while, I hadn't had an issue that would make me double-check those assumptions.

account42 7y ago

The traditional way to truncate an existing file is using O_TRUNC in the open call. From your link it looks like Plan 9 has that flag. > In addition, there are [...] values that can be ORed with the omode: OTRUNC says to truncate the file to zero length before opening it

jolmg 7y ago
That's what I meant by clobbering. The thing is that you can't use that flag to say "shorten it by half" to keep the first half, but you can with truncate(2). My point was that if you can't shorten it by an arbitrary amount to avoid the redundancy of writing the same beginning of the file, then there's really no advantage to it over writing a new file to replace the old one. In fact, I thought clobbering (i.e. O_TRUNC) was nearly equivalent, only with the additional risk of data loss if there's an error while writing.
- account42 7y ago
  Creating a new file also replaces all file metadata (creation time, permissions, ACLs, links, ...) while O_TRUNC only erases the contents.