5 ms·

No. Nonononononononono. We were ALMOST FREE! Just get these high-level abstractions out of my effing kernel. They don't belong there. Linus said as much himself

No. Nonononononononono. We were ALMOST FREE! Just get these high-level abstractions out of my effing kernel. They don't belong there. Linus said as much himself. The only reason kdbus, this thing's predecessor, got so much as a glance is that Linus trusts Greg KH. A trust I would now consider somewhat misplaced. This is yet another piece of crap that was tacked on by systemd nonsense. It's non-portable, and they want us to use it upstack. Have these people lost their effing minds? Do they think we don't care about compatability? Just because Lennart said "screw POSIX" doesn't mean you should do it.

IshKebab 10y ago
I agree. I can't see any real reason for this to be in the kernel. Is it faster? Does it need access to memory in some way you can't do in userspace? The only practical reason I can think is if it is in the kernel you can 100% rely on it being present and available. Userspace processes can be killed. So perhaps the real solution is to provide some kind of mechanism for userspace processes that act like kernel drivers, in that they are definitely running all the time, but actually run in userspace.
- qwertyuiop924 10y ago
  That's just a daemon. The reason this is in the kernel is that DBus is slow (according to Linus, because it could have been written better by monkeys with typewriters), and Lennart wanted it in the kernel/systemd, and Greg KH agreed, and Linus trusts Greg.
  - sjellis 10y ago
    The announcement email from David Herrmann explains that they did consider a user-space implementation, and lists the reasons why they decided that it had to be in-kernel: https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-July/003047.html https://lists.linuxfoundation.org/pipermail/ksummit-discuss/...
    - qwertyuiop924 10y ago
      I was talking about KDbus, the predecessor to this.
    - saynsedit 10y ago
      The reasons are mostly performance reasons. As I said in my other post, IPC latency is way down in the priority list of things to optimize (relatively to network and disk latency). Unless the application is a HPC parallel computation that is IPC bound, Bus1 has no performance justification. If the main justification for a modern kernel IPC mechanism is performance, I'd much much much more trust a proposal coming out of the HPC community than the desktop experience community.
      - deno 10y ago
        You’re ignoring the fact that Bus1 is based on binder, not dbus, so performance (mostly power efficiency) apparently was an issue—on Android. So now you have two ends of the spectrum where this effort does matter, performance wise. This only leaves the middle (desktop), but there are other reasons why that evil desktop crowd wants IPC in kernel, however high on a ‘priority list’ they might be.
        saynsedit 10y ago
        My point still stands. I don't think it's very likely that the main power hog in a mobile phone is the IPC mechanism. Unless IPC is happening in a tight loop, on the order of millions if times per second, it's just not common enough to really make a dent. You have wifi, screen brightness, CPU-hungry background apps, etc. to fix first.
        deno 10y ago
        http://lwn.net/Articles/504984/ http://lwn.net/Articles/504984/ #Priority Inversion #Latency It’s not about tight loops. Also this idea that you have to fix wifi chip or screen before you can add anything to the kernel doesn’t seem to be founded in any kind of logic.
- lathiat 10y ago
  The two main reasons I know/care about (there may be more) are that it can be much faster (zero copy, reduced context switches) and also guarantee the message isn't changed while reading (by sealing the originating buffer). The last is something you need kernel help to enforce.
  - qwertyuiop924 10y ago
    Yeah, but is it worth the cost? I would say no.
  - saynsedit 10y ago
    There's no case for this usually marginal speed benefit. Most programs are dealing with vastly slower bottlenecks. Like network or disk latency. IPC latency is so far down the list of priorities in terms of things to optimize. Not to mention that regular user-kernel-user buffer copying is plenty fast now, especially since the buffers will likely be cache-resident. Zero-copy is mostly hype without performance numbers to back it up. Don't need this complex message sealing mechanism.
  - dooglius 10y ago
    I don't see anything described that can't be done in a userspace library through shared memory. Maybe security is a concern, where we don't trust some nodes that might be exploited not to be able to screw with the shared memory? But then zero-copy would be off the table for the same reasons, no?
- tomegun 10y ago
  See the ksummit announcement elsewhere in this thread. The list of issues is given there. We could get 95% of the way in userspace, but not all the way (the issues are by no means obvious or trivial, so take a look at the announcement).
the_why_of_y 10y ago
Exactly! They should just use the Android Binder IPC that was merged some years ago and was widely heralded as a pinnacle of open development, good design and especially security. (Getting Binder standardized in the next revision of POSIX is surely a mere formality.)
- qwertyuiop924 10y ago
  No, it's not. Binder was designed for Java, and the rest of the Unix family doesn't support it.
rwmj 10y ago
I thought the interesting part of kdbus was the "memory sealing" - you could take a shared memory buffer, write to it, then "seal" it (so no further changes were allowed) and share it with another process. I believe you cannot efficiently do that in userspace. That doesn't seem to be part of this proposal.
- bluetech 10y ago
  There is memfd_create(2) (see discussion under "File sealing" and the related fcntls).
- tomegun 10y ago
  memory sealing can be used by anything that can do fd passing, which bus1 does (as does UDS). That code is already upstream and was not really tied to kdbus.
tomegun 10y ago
What's high-level about bus1? This has nothing to do with systemd, and very little (apart from history) to do with kdbus. As to portability, this is a linux kernel module, but there is nothing really inherent to linux about it, so porting it to another kernel should not be a lot of work if that is interesting to anyone.
- qwertyuiop924 10y ago
  What's high level? They're providing a message bus. That's not an IPC primitive. That's very high-level. As for systemd, I gathered there was a connection to kdbus, which was Lennart's baby. Thusly, I gathered it needed init to cooperate, the same way kdbus did, and any non-bus1-aware init just wouldn't work with apps that used bus1. As for portability, what other OS will implement this muck? None of them. It's portable the same way systemd is: "apps that depend on us are portable, so long as every os copies all our stuff, and re-implements all our APIs" Forcing other OS's to implement your stuff isn't portability.
  - the_why_of_y 10y ago
    None of QNX IPC, Mach IPC, Minix3 IPC, Solaris Doors, NT LPC or Android Binder is portable, and all of them are used by low-level user-space on their respective platform. But if Linux developers add a new IPC mechanism for similar purposes, it must be useless "muck"?
    - qwertyuiop924 10y ago
      NT doesn't have to worry about compat. Neither do Android apps. QNX, Mach, and Minix3 are microkernels, and that IPC isn't used by high-level userland in the UNIX compatible systems. Doors is something I have similar objections to. But if bus1 is a successor to kdbus, it's not intended to be used by low-level userland, it's built for highlevel userspace. Like systemd, this is trying to coax high-level userland to be incompatible with other unixes. The fact that no unixes other than the originating one, if the orginator was a unix, has picked up any of these systems speaks volumes about why you shouldn't use it in highlevel userland.

IshKebab 10y ago

I agree. I can't see any real reason for this to be in the kernel. Is it faster? Does it need access to memory in some way you can't do in userspace? The only practical reason I can think is if it is in the kernel you can 100% rely on it being present and available. Userspace processes can be killed. So perhaps the real solution is to provide some kind of mechanism for userspace processes that act like kernel drivers, in that they are definitely running all the time, but actually run in userspace.

qwertyuiop924 10y ago
That's just a daemon. The reason this is in the kernel is that DBus is slow (according to Linus, because it could have been written better by monkeys with typewriters), and Lennart wanted it in the kernel/systemd, and Greg KH agreed, and Linus trusts Greg.
- sjellis 10y ago
  The announcement email from David Herrmann explains that they did consider a user-space implementation, and lists the reasons why they decided that it had to be in-kernel: https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-July/003047.html https://lists.linuxfoundation.org/pipermail/ksummit-discuss/...
  - qwertyuiop924 10y ago
    I was talking about KDbus, the predecessor to this.
  - saynsedit 10y ago
    The reasons are mostly performance reasons. As I said in my other post, IPC latency is way down in the priority list of things to optimize (relatively to network and disk latency). Unless the application is a HPC parallel computation that is IPC bound, Bus1 has no performance justification. If the main justification for a modern kernel IPC mechanism is performance, I'd much much much more trust a proposal coming out of the HPC community than the desktop experience community.
    - deno 10y ago
      You’re ignoring the fact that Bus1 is based on binder, not dbus, so performance (mostly power efficiency) apparently was an issue—on Android. So now you have two ends of the spectrum where this effort does matter, performance wise. This only leaves the middle (desktop), but there are other reasons why that evil desktop crowd wants IPC in kernel, however high on a ‘priority list’ they might be.
      - saynsedit 10y ago
        My point still stands. I don't think it's very likely that the main power hog in a mobile phone is the IPC mechanism. Unless IPC is happening in a tight loop, on the order of millions if times per second, it's just not common enough to really make a dent. You have wifi, screen brightness, CPU-hungry background apps, etc. to fix first.
        deno 10y ago
        http://lwn.net/Articles/504984/ http://lwn.net/Articles/504984/ #Priority Inversion #Latency It’s not about tight loops. Also this idea that you have to fix wifi chip or screen before you can add anything to the kernel doesn’t seem to be founded in any kind of logic.
lathiat 10y ago
The two main reasons I know/care about (there may be more) are that it can be much faster (zero copy, reduced context switches) and also guarantee the message isn't changed while reading (by sealing the originating buffer). The last is something you need kernel help to enforce.
- qwertyuiop924 10y ago
  Yeah, but is it worth the cost? I would say no.
- saynsedit 10y ago
  There's no case for this usually marginal speed benefit. Most programs are dealing with vastly slower bottlenecks. Like network or disk latency. IPC latency is so far down the list of priorities in terms of things to optimize. Not to mention that regular user-kernel-user buffer copying is plenty fast now, especially since the buffers will likely be cache-resident. Zero-copy is mostly hype without performance numbers to back it up. Don't need this complex message sealing mechanism.
- dooglius 10y ago
  I don't see anything described that can't be done in a userspace library through shared memory. Maybe security is a concern, where we don't trust some nodes that might be exploited not to be able to screw with the shared memory? But then zero-copy would be off the table for the same reasons, no?
tomegun 10y ago
See the ksummit announcement elsewhere in this thread. The list of issues is given there. We could get 95% of the way in userspace, but not all the way (the issues are by no means obvious or trivial, so take a look at the announcement).

the_why_of_y 10y ago

Exactly! They should just use the Android Binder IPC that was merged some years ago and was widely heralded as a pinnacle of open development, good design and especially security. (Getting Binder standardized in the next revision of POSIX is surely a mere formality.)

qwertyuiop924 10y ago
No, it's not. Binder was designed for Java, and the rest of the Unix family doesn't support it.

rwmj 10y ago

I thought the interesting part of kdbus was the "memory sealing" - you could take a shared memory buffer, write to it, then "seal" it (so no further changes were allowed) and share it with another process. I believe you cannot efficiently do that in userspace. That doesn't seem to be part of this proposal.

bluetech 10y ago
There is memfd_create(2) (see discussion under "File sealing" and the related fcntls).
tomegun 10y ago
memory sealing can be used by anything that can do fd passing, which bus1 does (as does UDS). That code is already upstream and was not really tied to kdbus.

tomegun 10y ago

What's high-level about bus1? This has nothing to do with systemd, and very little (apart from history) to do with kdbus. As to portability, this is a linux kernel module, but there is nothing really inherent to linux about it, so porting it to another kernel should not be a lot of work if that is interesting to anyone.

qwertyuiop924 10y ago
What's high level? They're providing a message bus. That's not an IPC primitive. That's very high-level. As for systemd, I gathered there was a connection to kdbus, which was Lennart's baby. Thusly, I gathered it needed init to cooperate, the same way kdbus did, and any non-bus1-aware init just wouldn't work with apps that used bus1. As for portability, what other OS will implement this muck? None of them. It's portable the same way systemd is: "apps that depend on us are portable, so long as every os copies all our stuff, and re-implements all our APIs" Forcing other OS's to implement your stuff isn't portability.
- the_why_of_y 10y ago
  None of QNX IPC, Mach IPC, Minix3 IPC, Solaris Doors, NT LPC or Android Binder is portable, and all of them are used by low-level user-space on their respective platform. But if Linux developers add a new IPC mechanism for similar purposes, it must be useless "muck"?
  - qwertyuiop924 10y ago
    NT doesn't have to worry about compat. Neither do Android apps. QNX, Mach, and Minix3 are microkernels, and that IPC isn't used by high-level userland in the UNIX compatible systems. Doors is something I have similar objections to. But if bus1 is a successor to kdbus, it's not intended to be used by low-level userland, it's built for highlevel userspace. Like systemd, this is trying to coax high-level userland to be incompatible with other unixes. The fact that no unixes other than the originating one, if the orginator was a unix, has picked up any of these systems speaks volumes about why you shouldn't use it in highlevel userland.