4 ms·
Using read()/write() to access shared files is a bad idea because you have no control over buffering. Just because you've written something to a file does not m
by rbehrends 15y ago
Using read()/write() to access shared files is a bad idea because you have no control over buffering. Just because you've written something to a file does not mean it actually has been written somewhere where other processes can see it when they try to.
However, using shared files for IPC is very practical (I've done it before), but you want to use mmap() in conjunction with at least memory barriers or mutexes/semaphores/condition variables that work between processes. If you just need memory barriers, then the atomic_ops lib (http://www.hpl.hp.com/research/linux/atomic_ops/ http://www.hpl.hp.com/research/linux/atomic_ops/, https://github.com/ivmai/libatomic_ops/ https://github.com/ivmai/libatomic_ops/) will include those. For more advanced interprocess synchronization, you can use named semaphores on Linux/OS X (sem_open() and friends); on Linux, mutexes also work in shared memory.
Also, channels created with pipe() are very efficient on modern UNIXes; if that type of IPC is all you need, you may not require anything else (I've benchmarked them myself vs. handwritten implementations, and they're difficult to beat if there's any measurable degree of contention).
Incidentally, despite the author's father having misgivings about shared memory models, there's nothing inherently problematic about them; it's the near complete lack of programming language support to make it safe that makes it so frustrating to use. If you have a language that does not allow you read/write access to data without first having acquired the corresponding read/write lock, it's not any more error prone than interprocess communication.
The other element that makes shared memory concurrency hard is again a programming language issue; the absence of efficient thread-local variables. If you have efficient and convenient thread-local storage, you have all the benefits of separate processes (you can just make all your data thread-local), but can gradually share those data structures that can safely be shared, and generally more easily than in a multi-process model (try keeping anything that contains pointers in an mmap()ed area without tearing your hair out).
- HerraBRE 15y agoIf you are using write(2)/read(2) on a local file, is there any other buffer that comes into play aside from the memory in buffer cache itself? You are correct that obviously we don't really know when the data will hit disk, but that doesn't matter because the readers and writers are all hitting the same in-memory cache.
- rbehrends 15y agoWell, technically, the POSIX standard specifies that "After a write() to a regular file has successfully returned: ... Any successful read() from each byte position in the file that was modified by that write shall return the data specified by the write() for that position until such byte positions are again modified." How well any given operating system implements this and whether it properly considers all the aspects of multiple processes writing/reading to the same file is anybody's guess. Linux /should/ do it correctly, but as I recall, SQLite had some related issues with shared files on OS X a few years ago.