4 ms·

That's an unfortunately common misconception supported by the wording in POSIX, probably because no one wants to codify ill-defined practices. I remember back

That's an unfortunately common misconception supported by the wording in POSIX, probably because no one wants to codify ill-defined practices. I remember back in the 00s, people would refer to the implementation of the write syscall in some old BSD to explain why short writes are expected to only happen in actual error conditions; IIRC, local disk I/O was considered "fast" and thus the syscall only checked for signals during setup, and while returning to userspace. Around that time, I came to realise that Linux doesn't cut write(2) short due to signals. antirez https://news.ycombinator.com/item?id=3790154 https://news.ycombinator.com/item?id=3790154 confirms that was still the case in 2012. This expectation of no short write to local files outside real errors has spread to too many programs, so must be upheld by any useful POSIX-compatible OS (https://utcc.utoronto.ca/~cks/space/blog/unix/WritesNotShortOften https://utcc.utoronto.ca/~cks/space/blog/unix/WritesNotShort...). However, there are no hard rules for when short writes to files can and cannot happen (e.g., I'm pretty sure NFS is a common exception), and programs can be portable enough as long without knowing them. In contrast, it's easy to exceed reasonable limits for atomic pipe I/O, so it makes sense that POSIX would codify existing practices. Unfortunately, a vague agreement of "no shenanigans on regular file writes" doesn't fit anywhere in a specification, so people lacking the context end up thinking that only pipes can be expected to provide uninterrupted writes, while the reality is that only pipes broke writes regularly enough that it made sense to specify when and how that can happen.

mzs 6y ago
The person behind Boost.AFIO wrote a test and determined that ext4 is only atomic for one byte at a time. O_DIRECT with its perf/portability considerations was needed too. https://stackoverflow.com/a/35258623 https://stackoverflow.com/a/35258623
- pkhuong 6y ago
  That's for concurrent reads, where the "atomicity" guarantees as defined in POSIX don't mean what anyone expects.
  - mzs 6y ago
    Yes I misinterpreted that, thanks. I wonder if a later spec requires sector atomic writes because I always worry something will be optimized later while still meeting reqs. I thought this had happened for XFS on linux but the sector guarantee remained )(only the stronger SGI behavior was dropped to reduce latency) - or how now more C compiler optimizations trip-up people inadvertently hitting UB. It really does seem 256 bytes like how used in this HN post is safe on all common systems, but I'd still be afraid it could break unless I see it spelled-out in a spec and man page.

mzs 6y ago

The person behind Boost.AFIO wrote a test and determined that ext4 is only atomic for one byte at a time. O_DIRECT with its perf/portability considerations was needed too. https://stackoverflow.com/a/35258623 https://stackoverflow.com/a/35258623

pkhuong 6y ago
That's for concurrent reads, where the "atomicity" guarantees as defined in POSIX don't mean what anyone expects.
- mzs 6y ago
  Yes I misinterpreted that, thanks. I wonder if a later spec requires sector atomic writes because I always worry something will be optimized later while still meeting reqs. I thought this had happened for XFS on linux but the sector guarantee remained )(only the stronger SGI behavior was dropped to reduce latency) - or how now more C compiler optimizations trip-up people inadvertently hitting UB. It really does seem 256 bytes like how used in this HN post is safe on all common systems, but I'd still be afraid it could break unless I see it spelled-out in a spec and man page.