3 ms·
TLB is not necessarily an issue as presumably you would be running the processes on different cores. The primary problem is that separate address spaces requir
by gpderetta 6y ago
TLB is not necessarily an issue as presumably you would be running the processes on different cores.
The primary problem is that separate address spaces requires messages to be serialized (although of course serialization can be trivial). With shared memory you can share (or transfer ownership of) data by simply sharing a pointer.
- ncmncm 6y agoTLB usage is an advantage as each process, pinned to a core, gets its own. TLB shootdowns vanish. Cache line ownership, similarly, does not pingpong. Sharing pointers causes fragility, as failure in one thread takes down the whole system. Transferring ownership would add coupling. But such objects can live in a shared-memory area. Pointers into shared regions do require some care. Ideally, only one process ever owns objects in each, most commonly just the ring buffer. With separate processes, they may come and go as needed. Yes, setting up shared memory for ring buffers is more work, but interactions establishing them are not fast-path. Decoupling via separate processes is a different architecture, but has real benefits to motivate it.
- gpderetta 6y agoOh, I agree, it is all tradeoffs. Losing the ability to cheaply share objects, including closures, is a big tradeoff though. BTW lock free queues and data structures are pretty much required for interprocess sharing, unless you really want to deal with robust mutexes (and you don't!)
- ncmncm 6y agoI do it. Without mutexes. I don't count single-writer ring buffers among the ordinary lock-free queues, because there is no competition for access, and cache lines never change ownership. So, none of the overheads show up. And, they are very easy to get right, so pose none of the risk of subtle heisenbugs. Being able to kill and add downstream clients at random is worth a lot of architectural rejiggery.