I don't know if I'm contributing anything, but to clarify by going back to basics (rather than advanced or non standard system calls), all writes (and reads) a

I don't know if I'm contributing anything, but to clarify by going back to basics (rather than advanced or non standard system calls), all writes (and reads) are individually atomic, so if you and I are writing to the same file at the same time (in real time), how our overwriting behaves is relatively well behaved and can be relied on, so there is at least that level of atomic blocking, and that's important because the kernel also buffers files transparently, in that our two processes will share the same buffers, so beyond atomicity in a shared piece of the file, we won't get blocked wrt each other. But your reads need to pick up my writes (atomicity!) so there isn't going to be buffered queuing of writes on top of the disk buffering. so if the kernel decides it needs to spend more time syncing dirty buffers to disk, it's not blocking our processes in the regard above, but the system could become net i/o bound. This is 1/2 the reason to make sure to use virtual memory, so plenty of a flexible amount of disk buffer will be available (the other half being to swap out resident but no longer referenced dirty pages, as from the initialization code in your app)