15 ms·
Bus1: a new Linux interprocess communication proposal
- digi_owl 10y agoAnd the cascade keeps cascading...
- chme 10y agoIts just so much easier to add new functionality to the kernel than to remove old ones.
- digi_owl 10y agoBecause then they don't have to deal with Torvalds "we don't break userspace" stance. Especially as the devs involved seems to hate maintaining API compatibility...
- deleted 10y ago[deleted]
- nixos 10y agoCan someone please explain what's wrong with the classical UNIX sockets and pipes that they need to invent and (re)invent dbus, bonobo, now bus1?
- rodgerd 10y ago> what's wrong with the classical UNIX sockets Well, for starters they're a bolt-on that doesn't fit with the actual classic Unix philosophy.
- cyphar 10y agoYou couldn't send credentials nor file descriptors to other processes without UNIX sockets. I would argue that allowing for that kind of decoupling of permissions and file access is very much in the spirit of the UNIX philosophy. Quite frankly, there are some problems you simply wouldn't be able to solve without them (in runC, the underlying runtime that Docker wraps, we use them quite extensively).
- gens 10y agoYou can send fd's through UDS and they do expose a lot of info on the process on the other side. http://www.thomasstover.com/uds.html http://www.thomasstover.com/uds.html
- amscanne 10y agoAFAIK, there's no good way to "broadcast" messages without bouncing through a user space co-ordinator. The current primitives just don't match this kind of need. Presumably, this inefficiency could be a bottleneck for some cases and eliminating the extra step might help, plus a kernel implementation could be simpler (no extra daemons, etc.)
- bitwize 10y agoI wrote a proof of concept that does multicast using the file system and inotify. A bit hacky and there are drawbacks, but it works.
- thwarted 10y agoHow's it perform on tmpfs or ramdisk?
- wahern 10y ago1) They want better performance over the existing solutions. For better or worse applications are [arguably] abusing the heck out of D-Bus. Using regular files might help with huge messages, but it's a lot of overheard for many small messages. 2) They want the ability to insert a broker that manages permissions among the peers, but have the kernel enforce those permissions on a per message basis so the broker doesn't become a bottleneck. That's not easy to with the filesystem. I mean, strictly speaking it's theoretically possible using the file system considering that in practice the only real broker would be systemd with a fixed set of policies, so theoretically the traditional Unix model of user, group, and supplementary groups would suffice by assigning users and processes into the proper groups, and binding message files into protected group-readable directories. But for whatever pretense, people want the solution to implement a system of opaque, ephemeral identifiers/capabilities. It sort of reminds me of SMF, launchd, systemd, and similar changes. For example, I was installing Solaris 11.3 on a VM yesterday and was trying to figure out how to set the hostname. You used to set it by storing the hostname to /etc/nodename. Now it's stored in some opaque tree data structure deep in the bowels of SMF, which you set with some long and impenetrable command-line utility invocation. You can still set /etc/nodename; if no hostname is configured then SMF will initialize the hostname from /etc/nodename and delete the file. It boggles the mind.
- Pyxl101 10y agoThe design goals of bus1 are described here: https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-July/003047.html https://lists.linuxfoundation.org/pipermail/ksummit-discuss/... Some call-outs for you: > * efficiently support n-to-n communication. * be well-suited for both unicast and multicast messages. * guarantee a global message order. * avoid any in-kernel buffering and rather transfer data directly from a sender into the receiver's mappable queue (single-copy). The downside of the alternatives that you mention is that they're not available as kernel features in Linux. Bus1 is inspired by that previous work and aims to make a new attempt at inclusion in the kernel, which they believe is required to provide the features they're targeting: > A user-space implementation of bus1 (or even any bus-based IPC) was considered, but was found to have several seemingly unavoidable issues. > o To guarantee reliable, global message ordering including multicasts, as well as to provide reliable capabilities, a bus-broker is required. In other words, the current linux syscall API is not sufficient to implement the design as described above in an efficient way without a dedicated, trusted, privileged process that manages the bus and routes messages between the peers. (...)
- pjc50 10y agoGlobal message order sounds like the kind of commitment that becomes increasingly expensive in a multi core environment..
- grandinj 10y agohence the use of some nifty optimised clocking algorithms: https://github.com/bus1/bus1/wiki/Message-ordering https://github.com/bus1/bus1/wiki/Message-ordering
- tomegun 10y agoThe overhead due to the message ordering is constant, irrespective of the number of cores/peers. See the link posted above for an explanation.
- chubot 10y agoI'm very sympathetic to that point of view -- UNIX is a paradox where everybody uses it but most people don't really know how to use it. But this paper makes a good quantitative case for POSIX not meeting the needs of modern computing: https://news.ycombinator.com/item?id=11652609 https://news.ycombinator.com/item?id=11652609 A good example is that Linux distros, Android, and OS X all have their own non-POSIX IPC mechanisms. Applications are somehow voting with their feet and not using Unix domain sockets and pipes. It leaves me scratching my head because I don't think serialization is that hard a problem. But it seems that most people want IPC with structured data. Android is a bit of a special case because it's a single-language OS. "Traditional POSIX threading models, IPC interfaces, and file system access are being replaced by platform and vendor-specific APIs and frameworks such as Grand Central Dispatch [18], Binder [29], DBus [25], and SQLite [1]."
- pjmlp 10y ago> UNIX is a paradox where everybody uses it but most people don't really know how to use it. Specially when the majority nowadays has the idea that GNU/Linux == UNIX and never tried to write an actual portable UNIX application.
- saynsedit 10y agoEh? The majority of modern POSIXy software runs on at least Linux, Mac OS X, and FreeBSD. In fact, Mac OS X is the most widely deployed UNIX-like system ever. What you're a claiming to be true probably more applies to Mac OS X than Linux. I honestly haven't seen a recent package that was accidentally not portable in the long-term across the various modern UNIX-like systems because of confusion of Linux for POSIX.
- digi_owl 10y agoBecause the automotive industry wants to move from QNX to Linux, and wants to use dbus like they used QNX IPC beforehand. Thus we have projects like streaming video over dbus(?!). https://lwn.net/Articles/551969/ https://lwn.net/Articles/551969/
- cnvogel 10y ago> Large automotive companies have built huge systems on top of QNX > messages, creating large libraries used by their applications. > They would like to be able to use those libraries > on Linux, but often don't know that there is > a way to get the QNX message API for Linux. It is called SIMPL > and it works well. (from the LWN article)
- rwmj 10y agoIt's a shame that QNX is dying inside Blackberry. If there's one project I would love to see open sourced, it is QNX.
- hobo_mark 10y agoThey tried to open it for a short period, I remember getting the sources and playing with it: http://www.qnx.com/news/pr_2471_1.html http://www.qnx.com/news/pr_2471_1.html And now I notice it was almost ten years ago! Nevertheless, too little too late...
- cm3 10y agoAnd the reason this is more prevalent on QNX is that it's a microkernel based platform. Here, in Linux, there's the debate what should be allowed into the kernel, whereas that question is pretty much never asked for QNX. You also don't get discussions about circumventing the kernel network stack to avoid context switches, to name a prominent example. Microkernels also lend themselves better to multikernel designs where you aim to make better use of all the available cpu cores by reducing sharing.
- ndesaulniers 10y agoIn Binder's case, it makes RPC between different languages significantly easier. You can't pass file descriptors over sockets. You can't do RPC over pipes. Anonymous sockets can only be shared with children. Named sockets require write permission and a filesystem. SysV is prone to resource leaks. Binder has authentication, shared memory, reference counting, weak references, dead object notifications. The descriptors and authentication part is key, as it allows Android's trusted system_server to open a device or socket for an untrusted process such as an App if it has user permissions. The receiver of a binder request can check the requester's PID/GID and with confidence from the kernel. Most importantly, I don't think it's healthy to think of POSIX as being done. Some experimentation and discovery of better APIs and better systems should be encouraged. gah, I'm driving myself crazy; I swear I saw somewhere a diagram of the Taxonomy of Unix IPC Mechanisms, with most of them crossed out and replaced simply with binder, please help me find it!
- dozzie 10y ago> You can't pass file descriptors over sockets. Good that my code doesn't know this. It would stop working. > The descriptors and authentication part is key, [...] If by "authentication" you mean "checking PID/UID/GID", we already have both in unix sockets.
- CJefferson 10y agoHow do you pass a file descriptor through a socket? They just take bytes? If you mean "I send some bytes that describes the file to open, and keep my processes in sync manually", of course you can do that, but true passing would allow a number of cool things.
- pjmlp 10y agoThe UNIX War is dead, long live the GNU/Linux war. Ergo, POSIX wasn't the end.
- Philipp__ 10y agoBecause Linux people like to reinvent the wheel! :)
- the_why_of_y 10y agoAF_UNIX sockets don't do multi-cast. AF_INET sockets don't do FD passing or credentials. There was a patch to add an AF_BUS socket family some years ago that sounded like a pretty general solution but the network maintainer said "over my dead body". http://lwn.net/Articles/504970/ http://lwn.net/Articles/504970/
- CaliforniaKarl 10y agoThe link used here is a subscriber link, which means an existing subscriber decided to make this post available to the entire HN community during its non-free period. If you find this content useful, and you aren't already a LWN subscriber, you should consider becoming one! More info is here: https://lwn.net/op/FAQ.lwn#subs https://lwn.net/op/FAQ.lwn#subs Subscription prices are here: https://lwn.net/subscribe/Info https://lwn.net/subscribe/Info
- voltagex_ 10y agoI second this, just resubscribed. (LWN's financial situation is not as good as it has been in the past)
- stormbrew 10y agoIs there a reason this couldn't be done as a new socket type (initialized by socket()) instead of either a dedicated new system call and/or the device node they're doing now? I'm not sure it'd be important to do that instead, I'm just curious if there's an obvious rationale I'm missing.
- amluto 10y agoIt's not a particularly good fit for the socket API. In particular, communication is unidirectional at its most basic and uses something a lot like seL4's "reply capabilities". It's a very nice, if counterintuitive, design, and it can give an asymptotic memory usage reduction, but it means that sendto and recvfrom don't work the usual way.
- aseipp 10y agoI believe one of the original proposals in this design space was AF_BUS[1], which was actually in fact a new socket address family. But that design used domain sockets, which were extended to support multicast -- and it was rejected as making an already complex path more complicated, among other reasons. [1] https://lwn.net/Articles/504970/ https://lwn.net/Articles/504970/
- digi_owl 10y agoAnother alternative i have seen aired it to use TIPC. A protocol that has been in the kernel since 2.6. http://tipc.sourceforge.net/tipc_linux.shtml http://tipc.sourceforge.net/tipc_linux.shtml
- ahoka 10y agoTIPC is very good protocol, I was working on very robust systems built on it. Just a quick list of features, from Wikipedia: Location transparency of services in a network Auto-discovery mechanism Reliable transport Standard socket interface support Connectionless, connection-oriented and multicast messaging Subscription to network events
- f2f 10y ago"To create a new node, an application performs any of the various ioctl() operations that accept a handle, passing the special reserved handle number of "3". what a bunch of jokers :)
- Annatar 10y agoThat's the thing about Linux: they have literally not one, but multiple armies of "programmers", and yet even after 20+ years of an absolutely furious development pace, basic things like inter-process communication still do not work desirably. Startup/shutdown? Yep, you know it already, it's still in flux (SMF in illumos has been humming along for a decade now). Filesystems? Flux. Observability? Flux. libc, what libc, that comes from GNU, flux. libstdc++, yeah we just decided to implement a "version 3 of that", flux. Debugging format? You have to rip out your runtime linker in order to get DWARF 2 support (DWARF 2 has been available for 23 years now), so, flux. Virtualization? Wait, which one of the 56 not-quite-right virtualization solutions do I mean, because Linux people are flapping with that and nobody can get it right? Flux. The illumos kernel has been stable for decades now. Teams of five people here, three people there, four people over there, they roll up their sleeves, look at the issues the customers have had and are actually having now, professional engineers with formal education and insight, and in two years they have a working product which only needs a small fix here or there, and is tenable decades into the future. It goes to show that the number of programmers does not a quality, working product make. For example: it's 2016, and my Linux filesystems in Vertica suddenly go into read-only mode, nobody knows why yet. In 2016! I cannot believe I just typed that, but it's true. I've been running ZFS since 2006 both professionally and privately on Solaris and SmartOS at several large companies and I have never seen it go into read-only mode. The team which designed ZFS was only five people, not an army. Bryan Cantrill summed it up so profoundly when, upon showing SmartOS running Linux applications, he ran /native/bin/dtrace and proclaimed: I have clean water!!!
- chme 10y ago> Flux. And while all that is constantly changing, its still possible to compile very old user space software and run it in on a current kernel.
- deleted 10y ago[deleted]
- qwertyuiop924 10y agoNo. Nonononononononono. We were ALMOST FREE! Just get these high-level abstractions out of my effing kernel. They don't belong there. Linus said as much himself. The only reason kdbus, this thing's predecessor, got so much as a glance is that Linus trusts Greg KH. A trust I would now consider somewhat misplaced. This is yet another piece of crap that was tacked on by systemd nonsense. It's non-portable, and they want us to use it upstack. Have these people lost their effing minds? Do they think we don't care about compatability? Just because Lennart said "screw POSIX" doesn't mean you should do it.
- IshKebab 10y agoI agree. I can't see any real reason for this to be in the kernel. Is it faster? Does it need access to memory in some way you can't do in userspace? The only practical reason I can think is if it is in the kernel you can 100% rely on it being present and available. Userspace processes can be killed. So perhaps the real solution is to provide some kind of mechanism for userspace processes that act like kernel drivers, in that they are definitely running all the time, but actually run in userspace.
- qwertyuiop924 10y agoThat's just a daemon. The reason this is in the kernel is that DBus is slow (according to Linus, because it could have been written better by monkeys with typewriters), and Lennart wanted it in the kernel/systemd, and Greg KH agreed, and Linus trusts Greg.
- sjellis 10y agoThe announcement email from David Herrmann explains that they did consider a user-space implementation, and lists the reasons why they decided that it had to be in-kernel: https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-July/003047.html https://lists.linuxfoundation.org/pipermail/ksummit-discuss/...
- qwertyuiop924 10y ago