6 ms·

BUS1: A new Linux kernel IPC bus being made by systemd developers

23 points by vezzy-fnord 11y ago

api 11y ago
Is there some high-level document or blog post somewhere that outlines the vision of the systemd developers and why they are building these systems? So far I see a variety of projects (of IMHO varying quality) but no attempt at articulating a vision to tie it all together. What's the goal? What problem is all this stuff solving?
- CaptSpify 11y ago
  > What problem is all this stuff solving? RH's lack of having their hands in everything (only kinda kidding)
- viraptor 11y ago
  Have a look at systemd.conf presentations (and videos https://www.youtube.com/channel/UCvq_RgZp3kljp9X8Io9Z1DA https://www.youtube.com/channel/UCvq_RgZp3kljp9X8Io9Z1DA). There are various goals there. My interpretation would be: universal, global interfaces for all system things. Which is not a bad goal in itself.
  - vezzy-fnord 11y ago
    Which is not a bad goal in itself. It's worse than a bad goal. "Universal, global interfaces for all system things" is a goal that is cognitively meaningless.
    - viraptor 11y ago
      Not sure why cognitively meaningless. To some extent syscalls are a global, universal interface for all system things. They're just very low level, so they carry very little information. Dbus is an abstraction that allows actions with rich information and context, and makes it possible to define simple interfaces to potentially complex sets of actions. Systemd and related services fulfill those interfaces. Why do you think that's a bad goal?
      - vezzy-fnord 11y ago
        "All system things" is a meaningless statement. Syscalls can actually carry quite a lot of information (just look at signal semantics) and they're not "very low level". The second paragraph is more wind. I don't think it's a bad goal. You haven't even specified a goal.
        viraptor 11y ago
        I see what you mean, but I think it's a technicality. There's no good definition that I know of that wouldn't be called either insufficient or scope creep. Can you come up with one that includes interface to date/time, connected devices (with access control), virtual-or-physical screen resolution, packages, system and user services, etc. ? They're not even all administration actions. My best description here is "all system things" even if it's poorly defined - simply because I don't have a better one. As for syscalls - they can carry a lot of data. That doesn't make it rich, structured information. Sure, everything is just writing/reading bytes at the end of the day. But there's a reason we've got open() function and don't use syscalls directly - rich interfaces are nice. Syscalls are definitely low level (they're defined in the cpu specs, how much more low-level can you get?).
        vezzy-fnord 11y ago
        I wouldn't describe it as a technicality, but more of a carte blanche vacuity. What comprises "the system"? Just about everything. Hence "all system things" is really "all everything things". Such underspecified weasel wording is a chief reason for systemd's status at large. Even your list involves diverse interfaces ranging from real-time kernel primitives to end-user archives (packages), so again, "system things" is more of a justification for perpetual consumption than anything resembling a mission statement or goal. (Things that merely wrap syscalls would probably be part of a utilities package anyway, a la util-linux.)
        viraptor 11y ago
        Yes, it does include diverse interfaces. And you keep saying the idea of common interface to all (system or everything) things is a bad idea, but not why. So again, why is it bad idea to want a common bus that can talk to all of those things and have interfaces that actually describe high-level, meaningful actions on them? I can agree systemd situation is bad, but I don't think it's because of technical reasons. Leaving systemd out of it, what's bad about the idea that distributions, backend services, and client applications start using a common bus that has an interface for action "The system timezone is now XXX"? (and similar actions for network configuration, printers, sessions, and "all everything things")
- emmelaich 11y ago
  You should at least read "Rethinking PID 1" from 2010 http://0pointer.de/blog/projects/systemd.html Lennart has written an enormous amount of excellent articles about systemd.
shiftoutbox 11y ago
Great, let the bullshit pour in ! I am sure the new ipc layer will make you 100 gagillion btc a second and solve world hunger . Yadda yadda yadda , google uses it .
techdragon 11y ago
So I'll just say it. I'm switching to FreeBSD. I'm sick of this train wreck. I don't care what they are developing I'm just sick of how they are doing it. I don't want my operating system to have this sort of mid life crisis.
- thescribe 11y ago
  I've kind of been thinking the same thing, this feels like a takeover by the freedesktop people.
  - chris_wot 11y ago
    And yet it's not an official announcement, hasn't been discussed on any systemd related mailing lists, has no documentation and merely exists as a git repository. It's almost as if the programmers were experimenting with ideas that are alternatives to kdbus. I'm appalled. What were they thinking?
    - thescribe 11y ago
      You're very right. I let my distrust for freedeskop color my reaction.
      - chris_wot 11y ago
        That's ok. I was too sarcastic in that reply, sorry if I caused offence (I strive to modify my tendencies towards catty comments, but I'm a work in progress - When I see I've done it I'm making it a policy to acknowledge it and apologise). Could I ask a genuine question? Aside from systemd, why do you distrust freedesktop?
  - gnu8 11y ago
    I agree with that is more or less what it is, but I'm not sure if I have a problem with it since it delivers the features people expect from a desktop OS. If you stop and think about the UNIX model, it really does not deliver everything that a desktop OS needs. At the same time, there is plenty of room for traditional sysv or bsd style Linux distributions, particularly in the cloud/container space. It boggles the mind that Ubuntu is the most popular distribution in that area, considering how many of its features are utterly unused and unnecessary on a container.
thescribe 11y ago
A few times now I've seen announcements that systemd is consuming some other aspect of Linux, and every thread devolves into accusations that systemd is eating Linux, and the other side saying that whatever it is just happens to be in-scope for systemd because of how it interacts with some current aspect of systemd. My question is now: What isn't in scope for systemd?
- viraptor 11y ago
  The way it's described on https://wiki.freedesktop.org/www/Software/systemd/ https://wiki.freedesktop.org/www/Software/systemd/ almost everything is in scope. Everything that's not an end user application can be considered a basic building block of Linux.
  - pzone 11y ago
    That's a ridiculous claim. Systemd is not going to swallow SSL, libjpeg, or GTK. Clearly there are boundaries for it, even if they are a little fuzzy.
    - viraptor 11y ago
      We're talking services / apps, not libraries. systemd needs OpenSSL for example, so no, it's not going to swallow OpenSSL. (SSL itself is a protocol, I assume you mean libraries here) Then again, it could swallow sslproxy by providing sockets with transparent SSL to services which don't talk SSL natively.
- LukeShu 11y ago
  I think that's a mis-diagnosis. Systemd interacts with a many different external components. OK, that means that systemd developers are exposed to a bunch of different components, and work with them frequently. So, they find a pain-point, and they think to themselves "I could improve this." And, because they are competent developers, and it's free/open-source, they do. Sure, the reason the found a pain-point, and chose to contribute is because of their work systemd, but that contribution isn't in-scope for systemd, isn't associated with systemd. It's just them being good citizens and improving the ecosystem. Now, remember that a significant portion of the systemd developers are also kernel developers, and have been since before systemd existed. So it's not unlikely that because of systemd, they see a way that they think the kernel could be improved. And they're contributing to the kernel anyway, so why not? That said, many people object to the kind of design decisions made in systemd, and don't like that that those kind of decisions are spreading to the rest of the ecosystem. This isn't systemd taking over, this is the fact that certain prolific individuals, who are systemd developers, are also contributing to other projects; naturally making the same kind of decisions.
LukeShu 11y ago
The article is... premature. The only "source" seems to be the appearance of the "bus1" project on GitHub. The author observed that the primary 2 contributors to this project are Kay Sievers and David Herrmann, who also happen to be systemd developers. This project isn't linked to systemd in any way but that. The only other contributors are Harald Hoyer and Lars Uebernickel. Note that Kay Sievers was a Linux kernel contributor before systemd even existed (I don't know about the other 3, perhaps they were too). Kay, David, and Harald are all Red Hat employees, and Red Hat has mentions in the copyright statements, leading the author to suggest that this is an official Red Hat project. It could be, but it could also be that RH just owns the copyright because it's too similar to what they do at work, but it's cool because RH is fine with them open-sourcing everything. But also "notable" is that Lars is a Canonical employee, which receives no mention in the article.
digi_owl 11y ago
From what is in the github repo, i get the impression that this is not related to a RPC/IPC bus at all. Instead it seems to be a systemd analog for IoT/mobile. Some quick LKML searching etc brings up bus1 references in relation to ARM and devicetree.
- JdeBP 11y ago
  It's quite amusing to notice that almost no-one does what you did and actually looks at the original, rather than relying upon someone else, be it some pseudonymous person on the WWW or things like Phoronix, to do the analysis. The systemd people are probably immensely grateful for that, otherwise they'd be having to deal with stuff that would give the cranks a field day like ... * ... "Kay Sievers is writing a replacement for udev named /usr/bin/org.bus1.devices." * ... "Kay Sievers is writing a replacement for Dracut init named /usr/bin/org.bus1.rdinit." * ... "Kay Sievers is writing a replacement for gummiboot that hardcodes support for Microsoft's Boot Manager for Windows." (https://github.com/bus1/boot-efi/blob/c7f3a8a25acc838677b08d8a7dd4fd979cef9939/src/boot.c#l1050 https://github.com/bus1/boot-efi/blob/c7f3a8a25acc838677b08d...) * ... "David Herrmann is writing a replacement for kdbus named bus1." * ... "Kay Sievers is writing a replacement for systemd named /usr/bin/org.bus1.init." * ... "Lennart Poettering made no comment when asked about Herrmann's bus1 and kmscon, systemd-consoled having been quietly pulled from systemd in July 2015." (https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/8fQ9t9pmUuT https://plus.google.com/+LennartPoetteringTheOneAndOnly/post...) The simple fact is that it is an IPC mechanism, and it almost certainly indeed is an attempt to come up with a replacement for kdbus, given the latter's withdrawal a while back. It was Herrmann himself who pulled systemd-consoled, and he did so citing the fact that he was working on kdbus. (https://github.com/systemd/systemd/pull/747 https://github.com/systemd/systemd/pull/747) It's not shocking to find him working on a successor. The mechanism is currently a filesystem like devpts, containing a pseudo-file named 'bus' that client applications open and perform ioctl()s on. The caveat "currently" is because there's no design documentation there, and the thing is far from finished. There's no guarantee that that will be its final form. The replacement plug-and-play manager, EFI boot loader, and process #1 programs look like the majority of the project because the test infrastructure for using the new bus, essentially a very feature limited bootstrap system, has been written more quickly than the actual kernel IPC subsystem itself has been. It's fair to say that this is at roughly the same stage of development as SystemXVI is, albeit that SystemXVI does have design doco (https://github.com/ServiceManager/ServiceManager/blob/master/doc/DESIGN.md https://github.com/ServiceManager/ServiceManager/blob/master...).

23 points by vezzy-fnord 11y ago

api 11y ago

Is there some high-level document or blog post somewhere that outlines the vision of the systemd developers and why they are building these systems? So far I see a variety of projects (of IMHO varying quality) but no attempt at articulating a vision to tie it all together. What's the goal? What problem is all this stuff solving?

CaptSpify 11y ago
> What problem is all this stuff solving? RH's lack of having their hands in everything (only kinda kidding)
viraptor 11y ago
Have a look at systemd.conf presentations (and videos https://www.youtube.com/channel/UCvq_RgZp3kljp9X8Io9Z1DA https://www.youtube.com/channel/UCvq_RgZp3kljp9X8Io9Z1DA). There are various goals there. My interpretation would be: universal, global interfaces for all system things. Which is not a bad goal in itself.
- vezzy-fnord 11y ago
  Which is not a bad goal in itself. It's worse than a bad goal. "Universal, global interfaces for all system things" is a goal that is cognitively meaningless.
  - viraptor 11y ago
    Not sure why cognitively meaningless. To some extent syscalls are a global, universal interface for all system things. They're just very low level, so they carry very little information. Dbus is an abstraction that allows actions with rich information and context, and makes it possible to define simple interfaces to potentially complex sets of actions. Systemd and related services fulfill those interfaces. Why do you think that's a bad goal?
    - vezzy-fnord 11y ago
      "All system things" is a meaningless statement. Syscalls can actually carry quite a lot of information (just look at signal semantics) and they're not "very low level". The second paragraph is more wind. I don't think it's a bad goal. You haven't even specified a goal.
      - viraptor 11y ago
        I see what you mean, but I think it's a technicality. There's no good definition that I know of that wouldn't be called either insufficient or scope creep. Can you come up with one that includes interface to date/time, connected devices (with access control), virtual-or-physical screen resolution, packages, system and user services, etc. ? They're not even all administration actions. My best description here is "all system things" even if it's poorly defined - simply because I don't have a better one. As for syscalls - they can carry a lot of data. That doesn't make it rich, structured information. Sure, everything is just writing/reading bytes at the end of the day. But there's a reason we've got open() function and don't use syscalls directly - rich interfaces are nice. Syscalls are definitely low level (they're defined in the cpu specs, how much more low-level can you get?).
        vezzy-fnord 11y ago
        I wouldn't describe it as a technicality, but more of a carte blanche vacuity. What comprises "the system"? Just about everything. Hence "all system things" is really "all everything things". Such underspecified weasel wording is a chief reason for systemd's status at large. Even your list involves diverse interfaces ranging from real-time kernel primitives to end-user archives (packages), so again, "system things" is more of a justification for perpetual consumption than anything resembling a mission statement or goal. (Things that merely wrap syscalls would probably be part of a utilities package anyway, a la util-linux.)
        viraptor 11y ago
        Yes, it does include diverse interfaces. And you keep saying the idea of common interface to all (system or everything) things is a bad idea, but not why. So again, why is it bad idea to want a common bus that can talk to all of those things and have interfaces that actually describe high-level, meaningful actions on them? I can agree systemd situation is bad, but I don't think it's because of technical reasons. Leaving systemd out of it, what's bad about the idea that distributions, backend services, and client applications start using a common bus that has an interface for action "The system timezone is now XXX"? (and similar actions for network configuration, printers, sessions, and "all everything things")
emmelaich 11y ago
You should at least read "Rethinking PID 1" from 2010 http://0pointer.de/blog/projects/systemd.html Lennart has written an enormous amount of excellent articles about systemd.

shiftoutbox 11y ago

Great, let the bullshit pour in ! I am sure the new ipc layer will make you 100 gagillion btc a second and solve world hunger . Yadda yadda yadda , google uses it .

techdragon 11y ago

So I'll just say it. I'm switching to FreeBSD. I'm sick of this train wreck. I don't care what they are developing I'm just sick of how they are doing it. I don't want my operating system to have this sort of mid life crisis.

thescribe 11y ago
I've kind of been thinking the same thing, this feels like a takeover by the freedesktop people.
- chris_wot 11y ago
  And yet it's not an official announcement, hasn't been discussed on any systemd related mailing lists, has no documentation and merely exists as a git repository. It's almost as if the programmers were experimenting with ideas that are alternatives to kdbus. I'm appalled. What were they thinking?
  - thescribe 11y ago
    You're very right. I let my distrust for freedeskop color my reaction.
    - chris_wot 11y ago
      That's ok. I was too sarcastic in that reply, sorry if I caused offence (I strive to modify my tendencies towards catty comments, but I'm a work in progress - When I see I've done it I'm making it a policy to acknowledge it and apologise). Could I ask a genuine question? Aside from systemd, why do you distrust freedesktop?
- gnu8 11y ago
  I agree with that is more or less what it is, but I'm not sure if I have a problem with it since it delivers the features people expect from a desktop OS. If you stop and think about the UNIX model, it really does not deliver everything that a desktop OS needs. At the same time, there is plenty of room for traditional sysv or bsd style Linux distributions, particularly in the cloud/container space. It boggles the mind that Ubuntu is the most popular distribution in that area, considering how many of its features are utterly unused and unnecessary on a container.

thescribe 11y ago

A few times now I've seen announcements that systemd is consuming some other aspect of Linux, and every thread devolves into accusations that systemd is eating Linux, and the other side saying that whatever it is just happens to be in-scope for systemd because of how it interacts with some current aspect of systemd. My question is now: What isn't in scope for systemd?

viraptor 11y ago
The way it's described on https://wiki.freedesktop.org/www/Software/systemd/ https://wiki.freedesktop.org/www/Software/systemd/ almost everything is in scope. Everything that's not an end user application can be considered a basic building block of Linux.
- pzone 11y ago
  That's a ridiculous claim. Systemd is not going to swallow SSL, libjpeg, or GTK. Clearly there are boundaries for it, even if they are a little fuzzy.
  - viraptor 11y ago
    We're talking services / apps, not libraries. systemd needs OpenSSL for example, so no, it's not going to swallow OpenSSL. (SSL itself is a protocol, I assume you mean libraries here) Then again, it could swallow sslproxy by providing sockets with transparent SSL to services which don't talk SSL natively.
LukeShu 11y ago
I think that's a mis-diagnosis. Systemd interacts with a many different external components. OK, that means that systemd developers are exposed to a bunch of different components, and work with them frequently. So, they find a pain-point, and they think to themselves "I could improve this." And, because they are competent developers, and it's free/open-source, they do. Sure, the reason the found a pain-point, and chose to contribute is because of their work systemd, but that contribution isn't in-scope for systemd, isn't associated with systemd. It's just them being good citizens and improving the ecosystem. Now, remember that a significant portion of the systemd developers are also kernel developers, and have been since before systemd existed. So it's not unlikely that because of systemd, they see a way that they think the kernel could be improved. And they're contributing to the kernel anyway, so why not? That said, many people object to the kind of design decisions made in systemd, and don't like that that those kind of decisions are spreading to the rest of the ecosystem. This isn't systemd taking over, this is the fact that certain prolific individuals, who are systemd developers, are also contributing to other projects; naturally making the same kind of decisions.

LukeShu 11y ago

The article is... premature. The only "source" seems to be the appearance of the "bus1" project on GitHub. The author observed that the primary 2 contributors to this project are Kay Sievers and David Herrmann, who also happen to be systemd developers. This project isn't linked to systemd in any way but that. The only other contributors are Harald Hoyer and Lars Uebernickel. Note that Kay Sievers was a Linux kernel contributor before systemd even existed (I don't know about the other 3, perhaps they were too). Kay, David, and Harald are all Red Hat employees, and Red Hat has mentions in the copyright statements, leading the author to suggest that this is an official Red Hat project. It could be, but it could also be that RH just owns the copyright because it's too similar to what they do at work, but it's cool because RH is fine with them open-sourcing everything. But also "notable" is that Lars is a Canonical employee, which receives no mention in the article.

digi_owl 11y ago

From what is in the github repo, i get the impression that this is not related to a RPC/IPC bus at all. Instead it seems to be a systemd analog for IoT/mobile. Some quick LKML searching etc brings up bus1 references in relation to ARM and devicetree.

JdeBP 11y ago
It's quite amusing to notice that almost no-one does what you did and actually looks at the original, rather than relying upon someone else, be it some pseudonymous person on the WWW or things like Phoronix, to do the analysis. The systemd people are probably immensely grateful for that, otherwise they'd be having to deal with stuff that would give the cranks a field day like ... * ... "Kay Sievers is writing a replacement for udev named /usr/bin/org.bus1.devices." * ... "Kay Sievers is writing a replacement for Dracut init named /usr/bin/org.bus1.rdinit." * ... "Kay Sievers is writing a replacement for gummiboot that hardcodes support for Microsoft's Boot Manager for Windows." (https://github.com/bus1/boot-efi/blob/c7f3a8a25acc838677b08d8a7dd4fd979cef9939/src/boot.c#l1050 https://github.com/bus1/boot-efi/blob/c7f3a8a25acc838677b08d...) * ... "David Herrmann is writing a replacement for kdbus named bus1." * ... "Kay Sievers is writing a replacement for systemd named /usr/bin/org.bus1.init." * ... "Lennart Poettering made no comment when asked about Herrmann's bus1 and kmscon, systemd-consoled having been quietly pulled from systemd in July 2015." (https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/8fQ9t9pmUuT https://plus.google.com/+LennartPoetteringTheOneAndOnly/post...) The simple fact is that it is an IPC mechanism, and it almost certainly indeed is an attempt to come up with a replacement for kdbus, given the latter's withdrawal a while back. It was Herrmann himself who pulled systemd-consoled, and he did so citing the fact that he was working on kdbus. (https://github.com/systemd/systemd/pull/747 https://github.com/systemd/systemd/pull/747) It's not shocking to find him working on a successor. The mechanism is currently a filesystem like devpts, containing a pseudo-file named 'bus' that client applications open and perform ioctl()s on. The caveat "currently" is because there's no design documentation there, and the thing is far from finished. There's no guarantee that that will be its final form. The replacement plug-and-play manager, EFI boot loader, and process #1 programs look like the majority of the project because the test infrastructure for using the new bus, essentially a very feature limited bootstrap system, has been written more quickly than the actual kernel IPC subsystem itself has been. It's fair to say that this is at roughly the same stage of development as SystemXVI is, albeit that SystemXVI does have design doco (https://github.com/ServiceManager/ServiceManager/blob/master/doc/DESIGN.md https://github.com/ServiceManager/ServiceManager/blob/master...).