7 ms·

I'm surprised that microkernel-like IPC still hasn't found its way into any *NIX system. The closest we've gotten is System V IPC, which has never really taken

I'm surprised that microkernel-like IPC still hasn't found its way into any *NIX system. The closest we've gotten is System V IPC, which has never really taken off. Is this just difficult to design well or are people genuinely okay with socket(AF_UNIX, SOCK_SEQPACKET, 0)?

monocasa 7y ago
It's difficult to tack on to an existing kernel syscall interface. You really want a capability based interface to keep most of the permission checks out of the data plane. And even then modern microkernel IPC goes to extreme lengths. For instance the L4s tend to do crazy stuff like not save all of the registers, but make sure to only clobber the registers that can't be arguments for the call itself. You might be able to tack that onto the front of the syscall interface sort of like how objc_msgsend works, but it'd be a huge pain.
saagarjha 7y ago
Mach?
Matthias247 7y ago
I don't think it's difficult to design an IPC mechanism. But I DO think it's difficult to design an IPC mechanism that everyone is happy with. Some people want only synchronous IPC, others also asynchronous (out of order responses), some want multicast, some want events without associated request, some want pub/sub, etc. If at the end people continue building their own IPC mechanisms on top of TCP/IP or unix domain sockets then the in-kernel mechanism will just be another thing to maintain. There had been some endaveurs to bring new IPC mechanisms into the Linux Kernel (AF_BUS, KDBUS, BUS1). I think those failed for similar reasons (although I'm not sure were BUS1 now is - the others are definitely discontinued).
- jacquesm 7y ago
  KISS: QnX MsgSend, MsgReceive, MsgReply. That's really all it takes. http://www.qnx.com/developers/docs/6.5.0/index.jsp?topic=%2Fcom.qnx.doc.neutrino_lib_ref%2Fm%2Fmsgsend.html http://www.qnx.com/developers/docs/6.5.0/index.jsp?topic=%2F... You can go all the way from the lowest level kernel uses to very high level application constructs with that. Re-inventing existing wheels badly is something the software world excels at.
  - als0 7y ago
    > That’s really all it takes. Not quite. Those functions need established connections, so you need ConnectAttach and ConnectDetach as well. But none of that is useful unless you can identify clients so you also need ConnectClientInfo. This isn’t a dig, having worked on a custom IPC system I found the QNX approach to be the best of all worlds.
    - jacquesm 7y ago
      Sure, but the essence of the actual transfer, the part where performance matters is send/receive/reply.
- zzzcpan 7y ago
  > But I DO think it's difficult to design an IPC mechanism that everyone is happy with. Some people want only synchronous IPC ... Every form of IPC can be implemented on top of asynchronous message passing. Interface is not the problem. The problem is high performance designs with all the batching, memory mapped buffers, no syscalls, etc.
  - naasking 7y ago
    > Every form of IPC can be implemented on top of asynchronous message passing Sure, if you want to introduce inherent DoS vulnerabilities into your IPC subsystem, not to mention slow down IPC so much that it's practically unusable. Many early microkernels were asynchronous, and synchronous microkernels like L4 beat them easily every time. Furthermore, synchronous IPC can be made immune to DoS which is inherent to async IPC: https://www.researchgate.net/publication/4015956_Vulnerabilities_in_synchronous_IPC_designs https://www.researchgate.net/publication/4015956_Vulnerabili...
    - zzzcpan 7y ago
      > Sure, if you want to introduce inherent DoS vulnerabilities into your IPC subsystem, not to mention slow down IPC so much that it's practically unusable. This is nonsense. There could be DoS vulnerabilities in implementations, but they are not inherent to async message passing.
      - naasking 7y ago
        Yes they are. Who owns the buffers needed to store the async messages? 1. If they're booked to the receiver, then clients can easily DoS receivers by flooding them with message. 2. If they're booked to the sender, then receivers can easily DoS senders by blocking indefinitely. 3. If they're booked to the kernel (which is most common for true async message passing, unfortunately), then senders or receivers can DoS the whole system by the above two mechanisms. And that's only the most basic analysis. I suggest you read the paper I linked and its references if you want a more in-depth analysis of IPC vulnerabilities and performance properties.
pjmlp 7y ago
Android has it since Project Treble, classic Linux drivers are deemed legacy on Treble architecture. They make use of Android IPC to communicate among themselves and the kernel. https://source.android.com/devices/architecture/hidl/binder-ipc https://source.android.com/devices/architecture/hidl/binder-... Now given the role of Linux on Android, and what is accessible to userspace, maybe we shouldn't anyway consider it an *NIX system.
tyingq 7y ago
Google's Fuchsia seems to have it, though I'm not sure how it's implemented. https://fuchsia.dev/fuchsia-src/reference/syscalls#channels https://fuchsia.dev/fuchsia-src/reference/syscalls#channels

monocasa 7y ago

It's difficult to tack on to an existing kernel syscall interface. You really want a capability based interface to keep most of the permission checks out of the data plane. And even then modern microkernel IPC goes to extreme lengths. For instance the L4s tend to do crazy stuff like not save all of the registers, but make sure to only clobber the registers that can't be arguments for the call itself. You might be able to tack that onto the front of the syscall interface sort of like how objc_msgsend works, but it'd be a huge pain.

saagarjha 7y ago

Mach?

Matthias247 7y ago

I don't think it's difficult to design an IPC mechanism. But I DO think it's difficult to design an IPC mechanism that everyone is happy with. Some people want only synchronous IPC, others also asynchronous (out of order responses), some want multicast, some want events without associated request, some want pub/sub, etc. If at the end people continue building their own IPC mechanisms on top of TCP/IP or unix domain sockets then the in-kernel mechanism will just be another thing to maintain. There had been some endaveurs to bring new IPC mechanisms into the Linux Kernel (AF_BUS, KDBUS, BUS1). I think those failed for similar reasons (although I'm not sure were BUS1 now is - the others are definitely discontinued).

jacquesm 7y ago
KISS: QnX MsgSend, MsgReceive, MsgReply. That's really all it takes. http://www.qnx.com/developers/docs/6.5.0/index.jsp?topic=%2Fcom.qnx.doc.neutrino_lib_ref%2Fm%2Fmsgsend.html http://www.qnx.com/developers/docs/6.5.0/index.jsp?topic=%2F... You can go all the way from the lowest level kernel uses to very high level application constructs with that. Re-inventing existing wheels badly is something the software world excels at.
- als0 7y ago
  > That’s really all it takes. Not quite. Those functions need established connections, so you need ConnectAttach and ConnectDetach as well. But none of that is useful unless you can identify clients so you also need ConnectClientInfo. This isn’t a dig, having worked on a custom IPC system I found the QNX approach to be the best of all worlds.
  - jacquesm 7y ago
    Sure, but the essence of the actual transfer, the part where performance matters is send/receive/reply.
zzzcpan 7y ago
> But I DO think it's difficult to design an IPC mechanism that everyone is happy with. Some people want only synchronous IPC ... Every form of IPC can be implemented on top of asynchronous message passing. Interface is not the problem. The problem is high performance designs with all the batching, memory mapped buffers, no syscalls, etc.
- naasking 7y ago
  > Every form of IPC can be implemented on top of asynchronous message passing Sure, if you want to introduce inherent DoS vulnerabilities into your IPC subsystem, not to mention slow down IPC so much that it's practically unusable. Many early microkernels were asynchronous, and synchronous microkernels like L4 beat them easily every time. Furthermore, synchronous IPC can be made immune to DoS which is inherent to async IPC: https://www.researchgate.net/publication/4015956_Vulnerabilities_in_synchronous_IPC_designs https://www.researchgate.net/publication/4015956_Vulnerabili...
  - zzzcpan 7y ago
    > Sure, if you want to introduce inherent DoS vulnerabilities into your IPC subsystem, not to mention slow down IPC so much that it's practically unusable. This is nonsense. There could be DoS vulnerabilities in implementations, but they are not inherent to async message passing.
    - naasking 7y ago
      Yes they are. Who owns the buffers needed to store the async messages? 1. If they're booked to the receiver, then clients can easily DoS receivers by flooding them with message. 2. If they're booked to the sender, then receivers can easily DoS senders by blocking indefinitely. 3. If they're booked to the kernel (which is most common for true async message passing, unfortunately), then senders or receivers can DoS the whole system by the above two mechanisms. And that's only the most basic analysis. I suggest you read the paper I linked and its references if you want a more in-depth analysis of IPC vulnerabilities and performance properties.

pjmlp 7y ago

Android has it since Project Treble, classic Linux drivers are deemed legacy on Treble architecture. They make use of Android IPC to communicate among themselves and the kernel. https://source.android.com/devices/architecture/hidl/binder-ipc https://source.android.com/devices/architecture/hidl/binder-... Now given the role of Linux on Android, and what is accessible to userspace, maybe we shouldn't anyway consider it an *NIX system.

tyingq 7y ago

Google's Fuchsia seems to have it, though I'm not sure how it's implemented. https://fuchsia.dev/fuchsia-src/reference/syscalls#channels https://fuchsia.dev/fuchsia-src/reference/syscalls#channels