8 ms·

Bus1 – Kernel Message Bus

CalChris 10y ago
A discussion of where Bus1 fits in the bestiary of Linux IPC proposals: Bus1: a new Linux interprocess communication proposal https://lwn.net/Articles/697191/ https://lwn.net/Articles/697191/
- agumonkey 10y ago
  Someone on twitter suggested that Apple libxpc should be used. I never heard of it but if someone did, I'll be happy to know more about it.
  - ktRolster 10y ago
    Here you go: https://developer.apple.com/library/content/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingXPCServices.html https://developer.apple.com/library/content/documentation/Ma... It is higher level, though, it is not something you would integrate into the kernel (like bus1 intends to be). OSX has decent IPC though, since it is based on the Mach research kernel from CMU, which was kind of based around the concept of messaging to begin with. Applescript has interesting IPC concepts as well.
solidsnack9000 10y ago
> Also in the case where there can be no causal relationship, we are guaranteed > a global order. In case two events happend concurrently, there can never be > any inconsistency in which occurred before the other. By way of example, > consider two peers sending one message each to two different peers, we are > guaranteed that both the recipient peers receive the two messages in the same > order, even though the order may be arbitrary. How could that possibly work?
- sciurus 10y ago
  See https://github.com/bus1/documentation/wiki/Message-ordering https://github.com/bus1/documentation/wiki/Message-ordering for details on message ordering.
- mfukar 10y ago
  With their "synchronize-local-clocks" approach, it doesn't. They are using Lamport's algorithm to synchronize the clocks. However, Lamport's approach creates a _partial_ ordering, and to make that a _total_ ordering you need some mechanism to break "ties". For instance, the PID, or whatever. The catch is that the relationship derived from this arbitrary tie-breaking mechanism has nothing to do with causality, and therefore the total order it imposes is only an artifact of the mechanism chosen. Finding a tie-breaking mechanism that corresponds to the sending events is, in Lamport's own words, "not trivial".
  - tomegun 10y ago
    Indeed that is how we break ties (not exactly the PID, but you get the idea). The reason this works is that the only time we can have a tie is if there can be no causality between the events. I.e., the two sending events happen concurrently: the two ioctl calls overlap in time, so there would be no way for one to have caused the other. What problem do you see with this?
    - mfukar 10y ago
      I foresee a problem where people confuse the wording of "total order on all messages" in the wiki to mean there is a "global total order" - in other words, that bus1 solves distributed systems and we can all go home - and building buggy systems on this assumption. I'm not saying the concept is flawed or the implementation buggy, or anything like that. PS. Neil Brown in the LWN article already conflates "global" and "total" order.
      - jasonwatkinspdx 10y ago
        I'd suggest reading the hybrid logical clocks paper.
        mfukar 10y ago
        I have - I'm actually working with it on a multi-version IPC provider (totally unrelated to bus1 & friends). Is it relevant here? I know they're the latest and greatest, but they're not without problems either.
ktRolster 10y ago
The API seems bad to me, the ioctl() is ridiculously overloaded (disconnect, connect, read, and write all go through ioctl(), for example). As an API, this doesn't pass Linus' "good taste" test. As a solution, it's more complicated than necessary. On messaging in general: this is yet another IPC api from RedHat. They really want to get an IPC api into the kernel over there. However, their proposals seem 'ignorant,' in the sense that plenty of IPC research and work has been done over the last half century (or more?), and their proposals seems focused on their own small use cases. They seem unaware of how others have tried to solve the IPC problem. On why Linus doesn't want to integrate it: no kernel team is going to want to have yet another poorly-designed IPC stack to maintain even though hardly anyone uses it. That's been done, and IPC in Unix is a mess as a result. Any sane kernel dev would be resistant to this, unless the new proposal is absolutely beautiful, and you can look at it and say, "yeah, that is really great." RedHat should go out, investigate the research that has been done, become experts on the topic. Learn everyone's IPC problems, not just they problems they have in their own insular community. Only then create an API that actually is in good taste. Then begin the political work of getting people to adopt it. Start with the BSD teams. Start with the hobbyist OS devs at osdev.org. When the whole world agrees that it is a good thing, then Linus will put it in the kernel too.
- pcwalton 10y ago
  I don't have much of an opinion on how good this particular proposal is, but it is important to have a sensible IPC system for Linux. I see a lot of resistance to this, with claims that Unix sockets and POSIX are good enough. As the maintainer of Rust's ipc-channel, which wraps all this stuff, I strongly disagree. The complexity of the Linux implementation of ipc-channel [1] has been absurd compared to the Mac implementation, which uses Mach [2]. Don't let the similar lines-of-code count fool you—90% of the bugs that I've seen go by have been in the POSIX backend, necessitating things like manual fragmentation to get around size limits in the kernel, weird structure alignment rules in the cmsg API, file descriptor limits, ENOBUFS stuff, etc. etc. By contrast, the Mach implementation has more or less "just worked". At this point I don't really care what the Linux kernel decides on, as long as it decides on something. kdbus or Binder or anything would have been fine; the people who are truly hurt by kernel politics are us developers. [1]: https://github.com/servo/ipc-channel/blob/master/src/platform/unix/mod.rs https://github.com/servo/ipc-channel/blob/master/src/platfor... [2]: https://github.com/servo/ipc-channel/blob/master/src/platform/macos/mod.rs https://github.com/servo/ipc-channel/blob/master/src/platfor...
ndesaulniers 10y ago
So, Binder? And man, when people call ioctl a bastardized syscall... looks like 9 syscalls in one!
- TD-Linux 10y ago
  ... that's pretty much the point of ioctl - a syscall multiplexer in the context of a fd. 9 is pretty low - check out /dev/cdrom or /dev/dri/* if you want to see a lot.
- dom0 10y ago
  ioctl is pretty much a generic syscall interface (other OSes have similar things). An awful lot of Linux subsystems export an awful lot of syscalls through it; probably easily a four or five figure number (literature typically claims that Linux has only a couple hundred syscalls).
jacquesm 10y ago
Previous thread on HN: https://news.ycombinator.com/item?id=10708177 https://news.ycombinator.com/item?id=10708177
Matthias247 10y ago
Although I have brought a lot of high level RPC/IPC systems into production I'm really having a hard time imagining where I could use this. As far as I understand it gives me a bit more high level features (security, multicast) compared to other IPC primitives (sockets, pipes, ...). However once I'm going to use this my application (or high-level IPC framework) will be locked to Linux and no longer be portable to other platforms. So for any applications that should be at least halfway portable I would prefer something that works on the more common primitives (most likely sockets) and build something more powerful in a cross-plattform way on top ofi it (like HTTP, grpc, Thrift, ...). A full featured low-level framework makes sense if I have a whole set of applications on top of it which is not intended to be portable and uses it exclusively. Something like the Android/iOS/... platform. But the current ones already have settled on their infrastructure (e.g. on Binder), and even if new ones come up there is a high possibility that they wouldn't like at least something in Bus1 and instead come up with their own solution.
ausjke 10y ago
how is this related to systemd(not just technically)? if anything.
ausjke 10y ago
how is this related to systemd(not just technically)? if anything.
jojo3000 10y ago
There was also a presentation on Bus1 on the systemd.conf 2016: https://www.youtube.com/watch?v=6zN0b6BfgLY https://www.youtube.com/watch?v=6zN0b6BfgLY

A discussion of where Bus1 fits in the bestiary of Linux IPC proposals: Bus1: a new Linux interprocess communication proposal https://lwn.net/Articles/697191/ https://lwn.net/Articles/697191/

agumonkey 10y ago
Someone on twitter suggested that Apple libxpc should be used. I never heard of it but if someone did, I'll be happy to know more about it.
- ktRolster 10y ago
  Here you go: https://developer.apple.com/library/content/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingXPCServices.html https://developer.apple.com/library/content/documentation/Ma... It is higher level, though, it is not something you would integrate into the kernel (like bus1 intends to be). OSX has decent IPC though, since it is based on the Mach research kernel from CMU, which was kind of based around the concept of messaging to begin with. Applescript has interesting IPC concepts as well.

solidsnack9000 10y ago

> Also in the case where there can be no causal relationship, we are guaranteed > a global order. In case two events happend concurrently, there can never be > any inconsistency in which occurred before the other. By way of example, > consider two peers sending one message each to two different peers, we are > guaranteed that both the recipient peers receive the two messages in the same > order, even though the order may be arbitrary. How could that possibly work?

sciurus 10y ago
See https://github.com/bus1/documentation/wiki/Message-ordering https://github.com/bus1/documentation/wiki/Message-ordering for details on message ordering.
mfukar 10y ago
With their "synchronize-local-clocks" approach, it doesn't. They are using Lamport's algorithm to synchronize the clocks. However, Lamport's approach creates a _partial_ ordering, and to make that a _total_ ordering you need some mechanism to break "ties". For instance, the PID, or whatever. The catch is that the relationship derived from this arbitrary tie-breaking mechanism has nothing to do with causality, and therefore the total order it imposes is only an artifact of the mechanism chosen. Finding a tie-breaking mechanism that corresponds to the sending events is, in Lamport's own words, "not trivial".
- tomegun 10y ago
  Indeed that is how we break ties (not exactly the PID, but you get the idea). The reason this works is that the only time we can have a tie is if there can be no causality between the events. I.e., the two sending events happen concurrently: the two ioctl calls overlap in time, so there would be no way for one to have caused the other. What problem do you see with this?
  - mfukar 10y ago
    I foresee a problem where people confuse the wording of "total order on all messages" in the wiki to mean there is a "global total order" - in other words, that bus1 solves distributed systems and we can all go home - and building buggy systems on this assumption. I'm not saying the concept is flawed or the implementation buggy, or anything like that. PS. Neil Brown in the LWN article already conflates "global" and "total" order.
    - jasonwatkinspdx 10y ago
      I'd suggest reading the hybrid logical clocks paper.
      - mfukar 10y ago
        I have - I'm actually working with it on a multi-version IPC provider (totally unrelated to bus1 & friends). Is it relevant here? I know they're the latest and greatest, but they're not without problems either.

ktRolster 10y ago

The API seems bad to me, the ioctl() is ridiculously overloaded (disconnect, connect, read, and write all go through ioctl(), for example). As an API, this doesn't pass Linus' "good taste" test. As a solution, it's more complicated than necessary. On messaging in general: this is yet another IPC api from RedHat. They really want to get an IPC api into the kernel over there. However, their proposals seem 'ignorant,' in the sense that plenty of IPC research and work has been done over the last half century (or more?), and their proposals seems focused on their own small use cases. They seem unaware of how others have tried to solve the IPC problem. On why Linus doesn't want to integrate it: no kernel team is going to want to have yet another poorly-designed IPC stack to maintain even though hardly anyone uses it. That's been done, and IPC in Unix is a mess as a result. Any sane kernel dev would be resistant to this, unless the new proposal is absolutely beautiful, and you can look at it and say, "yeah, that is really great." RedHat should go out, investigate the research that has been done, become experts on the topic. Learn everyone's IPC problems, not just they problems they have in their own insular community. Only then create an API that actually is in good taste. Then begin the political work of getting people to adopt it. Start with the BSD teams. Start with the hobbyist OS devs at osdev.org. When the whole world agrees that it is a good thing, then Linus will put it in the kernel too.

pcwalton 10y ago
I don't have much of an opinion on how good this particular proposal is, but it is important to have a sensible IPC system for Linux. I see a lot of resistance to this, with claims that Unix sockets and POSIX are good enough. As the maintainer of Rust's ipc-channel, which wraps all this stuff, I strongly disagree. The complexity of the Linux implementation of ipc-channel [1] has been absurd compared to the Mac implementation, which uses Mach [2]. Don't let the similar lines-of-code count fool you—90% of the bugs that I've seen go by have been in the POSIX backend, necessitating things like manual fragmentation to get around size limits in the kernel, weird structure alignment rules in the cmsg API, file descriptor limits, ENOBUFS stuff, etc. etc. By contrast, the Mach implementation has more or less "just worked". At this point I don't really care what the Linux kernel decides on, as long as it decides on something. kdbus or Binder or anything would have been fine; the people who are truly hurt by kernel politics are us developers. [1]: https://github.com/servo/ipc-channel/blob/master/src/platform/unix/mod.rs https://github.com/servo/ipc-channel/blob/master/src/platfor... [2]: https://github.com/servo/ipc-channel/blob/master/src/platform/macos/mod.rs https://github.com/servo/ipc-channel/blob/master/src/platfor...

ndesaulniers 10y ago

So, Binder? And man, when people call ioctl a bastardized syscall... looks like 9 syscalls in one!

TD-Linux 10y ago
... that's pretty much the point of ioctl - a syscall multiplexer in the context of a fd. 9 is pretty low - check out /dev/cdrom or /dev/dri/* if you want to see a lot.
dom0 10y ago
ioctl is pretty much a generic syscall interface (other OSes have similar things). An awful lot of Linux subsystems export an awful lot of syscalls through it; probably easily a four or five figure number (literature typically claims that Linux has only a couple hundred syscalls).

jacquesm 10y ago

Previous thread on HN: https://news.ycombinator.com/item?id=10708177 https://news.ycombinator.com/item?id=10708177

Matthias247 10y ago

Although I have brought a lot of high level RPC/IPC systems into production I'm really having a hard time imagining where I could use this. As far as I understand it gives me a bit more high level features (security, multicast) compared to other IPC primitives (sockets, pipes, ...). However once I'm going to use this my application (or high-level IPC framework) will be locked to Linux and no longer be portable to other platforms. So for any applications that should be at least halfway portable I would prefer something that works on the more common primitives (most likely sockets) and build something more powerful in a cross-plattform way on top ofi it (like HTTP, grpc, Thrift, ...). A full featured low-level framework makes sense if I have a whole set of applications on top of it which is not intended to be portable and uses it exclusively. Something like the Android/iOS/... platform. But the current ones already have settled on their infrastructure (e.g. on Binder), and even if new ones come up there is a high possibility that they wouldn't like at least something in Bus1 and instead come up with their own solution.

ausjke 10y ago

how is this related to systemd(not just technically)? if anything.

ausjke 10y ago

how is this related to systemd(not just technically)? if anything.

jojo3000 10y ago

There was also a presentation on Bus1 on the systemd.conf 2016: https://www.youtube.com/watch?v=6zN0b6BfgLY https://www.youtube.com/watch?v=6zN0b6BfgLY