3 ms·

I also went this route and came to the very same conclusions. Cap'n proto for fast reading, SHM for shared data, simple short messaging, just everything in C.

by rurban 3y ago

I also went this route and came to the very same conclusions. Cap'n proto for fast reading, SHM for shared data, simple short messaging, just everything in C. My only problem is MacOS with its too small default SHM buffers, you need to enhance them. Most solutions need a reboot, but a simple setter is enough. Like sudo sysctl -w kern.sysv.shmmax=16777216

ygoldfeld 3y ago
Interesting! I'd best write this down. Current notes on macOS and Windows port work: https://github.com/Flow-IPC/ipc/issues/101 https://github.com/Flow-IPC/ipc/issues/101 (<= https://github.com/orgs/Flow-IPC/discussions/98 https://github.com/orgs/Flow-IPC/discussions/98) For macOS/ARM64, currently it looks to me like the apparent lack of /dev/shm equivalent (unless I messed up in searching for it) means the most significant amount of new work necessary to port it ... but you just mentioned a thing I did not know about. (SHM size/count limits definitely were a thing in Linux, though, indeed.) TY
- rurban 3y ago
  I wrote some shm API wrappers here: https://github.com/rurban/smart/blob/new/source/algos/include/shmids.h https://github.com/rurban/smart/blob/new/source/algos/includ... Never use /dev/shm directly unless Linux only.
  - ygoldfeld 3y ago
    Oh… Sys V SHM… not POSIX. Old-school.
abcd_f 3y ago
There's no reason to use (the very ancient) SHM API over mmap, not today. You can literally do everything with mmap that you can do with shm, without hitting OS caps, no performance penalty and with a code that's simpler.
- monocasa 3y ago
  Well except the sharing memory part. Linux added memfds but macos doesn't have that. I imagine on macos you'd have to use the mach APIs if you're avoiding shm.
pmalynin 3y ago
Tbh on macOS you should probably use xpc / Mach and that will probably let you do more than a generic ipc library. Of course caveat emptor it’s not portable
- jcelerier 3y ago
  I would really not trust XPC to be any relevant when you care about raw performance and latency. Whole OS is slow as molasses.

by rurban 3y ago

ygoldfeld 3y ago

Interesting! I'd best write this down. Current notes on macOS and Windows port work: https://github.com/Flow-IPC/ipc/issues/101 https://github.com/Flow-IPC/ipc/issues/101 (<= https://github.com/orgs/Flow-IPC/discussions/98 https://github.com/orgs/Flow-IPC/discussions/98) For macOS/ARM64, currently it looks to me like the apparent lack of /dev/shm equivalent (unless I messed up in searching for it) means the most significant amount of new work necessary to port it ... but you just mentioned a thing I did not know about. (SHM size/count limits definitely were a thing in Linux, though, indeed.) TY

rurban 3y ago
I wrote some shm API wrappers here: https://github.com/rurban/smart/blob/new/source/algos/include/shmids.h https://github.com/rurban/smart/blob/new/source/algos/includ... Never use /dev/shm directly unless Linux only.
- ygoldfeld 3y ago
  Oh… Sys V SHM… not POSIX. Old-school.

abcd_f 3y ago

There's no reason to use (the very ancient) SHM API over mmap, not today. You can literally do everything with mmap that you can do with shm, without hitting OS caps, no performance penalty and with a code that's simpler.

monocasa 3y ago
Well except the sharing memory part. Linux added memfds but macos doesn't have that. I imagine on macos you'd have to use the mach APIs if you're avoiding shm.

pmalynin 3y ago

Tbh on macOS you should probably use xpc / Mach and that will probably let you do more than a generic ipc library. Of course caveat emptor it’s not portable

jcelerier 3y ago
I would really not trust XPC to be any relevant when you care about raw performance and latency. Whole OS is slow as molasses.