5 ms·
Note that the title is a little click-baity. Specifically, this article is about what is wrong with POSIX I/O in high-performance, exascale computing. This arti
by gh02t 9y ago
Note that the title is a little click-baity. Specifically, this article is about what is wrong with POSIX I/O in high-performance, exascale computing. This article is about how the POSIX I/O model breaks down in a pretty extreme use case, not that it's fundamentally broken.
- convolvatron 9y agothe atomic write business is pretty painful in any distributed setting, and only comes into play with concurrent writers with overlapping byte offset regions. I've always kind of struggled to imagine a case where an application would be correct with this guarantee and faulty otherwise. i guess some sort of cleverly designed structure with fixed length records where a compound mutation could be expressed in a contiguous byte range?
- jstimpfle 9y agoAren't these atomic writes only a reflection of the fact that data is written out as blocks?
- convolvatron 9y agowell, yeah, it wouldn't otherwise be necessary for the writes to be fully serialized (because blocks and the cache traditionally, but really anything) edit: its also worth noting that even in the traditional implementation since the block interface doesn't provide atomicity on faults this is kinda probabilistic anyways
- Derbasti 9y agoImagine two processes appending rows to a csv file. If writes were non-atomic, they might insert partial rows, or overwrite each other.
- convolvatron 9y agothe append case is actually different, it requires serializing the end of file pointer. in the write atomicity case we're asking the file system to order all the writes, and make sure the resulting file shows each of them being completed in their entirety before the next is applied, without any interleaving even if they refer to the same byte offsets in the file
- glennklockwood 9y agoThere are very few cases in practice where it's needed. The bigger issue is deciding what consistency semantics you _do_ want to have once you decide that strong isn't necessary. Every decision exposes another corner case where someone's application breaks.
- fnord123 9y agoI can see how it might seem click-baity but I am pretty sure it wasn't intentional or malicious. Nextplatform is a site about high-performance, exascale computing. If you're on the site then it's not click-baity since the article is exactly what you would expect. The issue is that if you're on a link aggregator like HN then you lose that context.
- gnufx 9y agoRegardless of the article, the issues apply long before you reach exascale, e.g. on typical university-scale systems. PVFS' design compromises on POSIX go back quite a long way, and MPI-IO is about 20(?) years old. We've been running non-POSIX-compliant NFS for decades too.
- glennklockwood 9y agoA more pedantic title would be "What's so bad about POSIX I/O for scalable performance on networked file systems". As others have said, it's an old problem that predates exascale by decades. I never asserted that POSIX is fundamentally broken; it's just that other communities with I/O issues at extreme scales decided to not even bother with the idea of POSIX and went straight for stateless, REST-like APIs and got atomicity through immutability. It's a pronounced problem in high-performance computing because that community evolved from a much more primitive era and is dragging decades of legacy applications (and, to some degree, thinking) behind it.
- gh02t 9y agoOh I agree with you fully, when I posted this comment there was nothing here. Just wanted to clarify a bit for people.