6 ms·

Lets take it a level up. I'm still not sure why we even need a message bus in there in the first place. The whole Linux boot, init, systemd pile is a Rube Goldb

by nbbnbb 2y ago

Lets take it a level up. I'm still not sure why we even need a message bus in there in the first place. The whole Linux boot, init, systemd pile is a Rube Goldberg machine that is very difficult to understand at the moment. The only reason I suspect most people aren't complaining is that our abstraction level is currently a lot higher (docker / kubernetes etc) and machines are mostly ephemeral cattle and few people go near it. As for JSON, please don't even start me on that. It is one of the worst serialisation decisions we ever made as a society. Poorly defined and unreliable primitive types, terrible schema support and expensive to parse. In a kernel, it does not belong! Give it a few months and varlink will be YAML over the top of that.

otabdeveloper4 2y ago
> I'm still not sure why we even need a message bus in there in the first place. Because traditional POSIX IPC mechanisms are absolute unworkable dogshit. > It is one of the worst serialisation decisions we ever made as a society. There isn't really any alternative. It's either JSON or "JSON but in binary". (Like CBOR.) Anything else is not interoperable.
- nbbnbb 2y ago
  This is a quite frankly ridiculous point. Most of that garb came from the HPC people who built loads of stuff on top of it in the first place. It's absolutely fine for this sort of stuff. It's sending the odd little thing here and there, not on a complex HPC cluster. As for JSON, are you really that short sighted that it's the only method of encoding something? Is "oh well it doesn't fit the primitive types, so just shove it in a string and add another layer of parsing" acceptable? Hell no.
  - otabdeveloper4 2y ago
    > ...it's the only method of encoding something? If you want something on the system level parsable by anything? Yes it is.
    - nbbnbb 2y ago
      protobufs / asn.1 / structs ... Edit: hell even XML is better than this!
      - baq 2y ago
        Thank goodness they didn’t pick YAML though.
        nbbnbb 2y ago
        Yet!
      - RandomThoughts3 2y ago
        Structs are a part of C semantic. They are not an ipc format. You can somewhat use them like one if you take a lot of precaution about how they are laid out in memory including padding and packing but it’s very brittle. Asn.1 is both quite complicated and not very efficient. They could certainly have gone with protobufs or another binary serialisation format but would it really be better than the current choice? I don’t think the issue they are trying to solve is related to serialisation anyway. Seems to me they are unhappy about the bus part not the message format part.
        dfox 2y ago
        ASN.1 BER/DER is more or less the same thing as CBOR. The perceived complexity of ASN.1 comes from the schema language and specifications written in the convoluted telco/ITU-T style (and well, the 80's type system that has ~8 times two different types for “human readable string”).
        RandomThoughts3 2y ago
        > The perceived complexity of ASN.1 comes from the schema language and specifications written in the convoluted telco/ITU-T style (and well, the 80's type system that has ~8 times two different types for “human readable string”). I can’t resist pointing that it’s basically a longer way of saying quite complicated and not very efficient.
- jchw 2y ago
  There are a world of serialization formats that can offer a similar interoperability story to JSON or JSON-but-binary formats. And sure, implementing them in every language that someone might be interested in using them in might require complication, but: - Whatever: people in more niche languages are pretty used to needing to do FFI for things like this anyhow. - Many of them already have a better ecosystem than D-Bus. e.g. interoperability between Protobuf and Cap'n'proto implementations is good. Protobuf in most (all?) runtimes supports dynamically reading a schema and parsing binary wire format with it, as well as code generation. You can also maintain backwards compatibility in these formats by following relatively simple rules that can be statically-enforced. - JSON and JSON-but-binary have some annoying downsides. I really don't think field names of composite types belong as part of the ABI. JSON-like formats also often have to try to deal with the fact that JSON doesn't strictly define all semantics. Some of them differ from JSON is subtle ways, so supporting both JSON and sorta-JSON can lead to nasty side-effects. Maybe most importantly, since we're not writing software that's speaking to web browsers, JSON isn't even particularly convenient to begin with. A lot of the software will be in C and Rust most likely. It helps a bit for scripting languages like Python, but I'm not convinced it's worth the downsides.
  - otabdeveloper4 2y ago
    Sorry, but bash isn't a "niche language" and it doesn't have an FFI story.
    - jchw 2y ago
      I don't know how to tell you this, but, you don't need to implement an RPC protocol in bash, nor do you need FFI. You can use CLI tools like `dbus-send`. I pray to God nothing meaningful is actually doing what you are insinuating in any serious environment.
      - otabdeveloper4 2y ago
        I'm trying to tell you that something that isn't straceable and greappable shouldn't belong in your system services stack.
jchw 2y ago
I think it doesn't have to be a message bus per-se, that design decision is mostly just because it's convenient. Even D-Bus can actually be used without the bus, if you really want to. D-Bus is just a bus so it can solve a bunch of different IPC problems at the same time. e.g. D-Bus can handle ACLs/permissions on RPC methods, D-Bus can handle multiple producers and multiple consumers, and so on. I think ultimately all that's really needed is IPC, and having a unified bus is just to allow for some additional use cases that are harder if you're only using UNIX domain sockets directly. If there are going to be systemd components that have IPC, then I'd argue they should probably use something standard rather than something bespoke. It's good to not re-invent the wheel. Not that I think Varlink is any better. It seems like at best it's probably a lateral move. I hope this does not go forward.
- nbbnbb 2y ago
  > If there are going to be systemd components that have IPC, then I'd argue they should probably use something standard rather than something bespoke. It's good to not re-invent the wheel. This is my point. My favourite DBus situation a number of years ago was a CentOS 7 box that reboot stopped working on with a cryptic DBus error that no one has ever seen before. I had to sync it, power cycle the node from the ILO card and cross my fingers. I really don't give a shit about this. I just wanted to run my jobs on the node, not turn into a sysadmin due to someone else's dubious architectural decisions.
  - jchw 2y ago
    Yes but the systemd developers don't want to implement their own protocols with e.g. ACL checking, and given some of their track record I kind of think you don't want them to, either. I'm pretty sure the error conditions would be even more bespoke if they "just" used UNIX domain sockets directly. Don't get me wrong, there's nothing particularly wrong with UNIX domain sockets, but there's no "standard" protocols for communicating over UDS.
    - amluto 2y ago
      This is systemd we’re talking about. A service manager that already mucks with mount namespaces. It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead… > Public varlink interfaces are registered system-wide by their well-known address, by default /run/org.varlink.resolver. The resolver translates a given varlink interface to the service address which provides this interface. …we have well known names, and sandboxing, or replacing a service for just one client, remains a mess. Sigh.
therein 2y ago
Yeah, how will number/float serialization go? Are we going to serialize them as strings and parse them? That abstraction isn't handled the same way across multiple languages.
poettering 2y ago
Varlink is not a message bus. Hence you should be happy?

by nbbnbb 2y ago

otabdeveloper4 2y ago

> I'm still not sure why we even need a message bus in there in the first place. Because traditional POSIX IPC mechanisms are absolute unworkable dogshit. > It is one of the worst serialisation decisions we ever made as a society. There isn't really any alternative. It's either JSON or "JSON but in binary". (Like CBOR.) Anything else is not interoperable.

nbbnbb 2y ago
This is a quite frankly ridiculous point. Most of that garb came from the HPC people who built loads of stuff on top of it in the first place. It's absolutely fine for this sort of stuff. It's sending the odd little thing here and there, not on a complex HPC cluster. As for JSON, are you really that short sighted that it's the only method of encoding something? Is "oh well it doesn't fit the primitive types, so just shove it in a string and add another layer of parsing" acceptable? Hell no.
- otabdeveloper4 2y ago
  > ...it's the only method of encoding something? If you want something on the system level parsable by anything? Yes it is.
  - nbbnbb 2y ago
    protobufs / asn.1 / structs ... Edit: hell even XML is better than this!
    - baq 2y ago
      Thank goodness they didn’t pick YAML though.
      - nbbnbb 2y ago
        Yet!
    - RandomThoughts3 2y ago
      Structs are a part of C semantic. They are not an ipc format. You can somewhat use them like one if you take a lot of precaution about how they are laid out in memory including padding and packing but it’s very brittle. Asn.1 is both quite complicated and not very efficient. They could certainly have gone with protobufs or another binary serialisation format but would it really be better than the current choice? I don’t think the issue they are trying to solve is related to serialisation anyway. Seems to me they are unhappy about the bus part not the message format part.
      - dfox 2y ago
        ASN.1 BER/DER is more or less the same thing as CBOR. The perceived complexity of ASN.1 comes from the schema language and specifications written in the convoluted telco/ITU-T style (and well, the 80's type system that has ~8 times two different types for “human readable string”).
        RandomThoughts3 2y ago
        > The perceived complexity of ASN.1 comes from the schema language and specifications written in the convoluted telco/ITU-T style (and well, the 80's type system that has ~8 times two different types for “human readable string”). I can’t resist pointing that it’s basically a longer way of saying quite complicated and not very efficient.
jchw 2y ago
There are a world of serialization formats that can offer a similar interoperability story to JSON or JSON-but-binary formats. And sure, implementing them in every language that someone might be interested in using them in might require complication, but: - Whatever: people in more niche languages are pretty used to needing to do FFI for things like this anyhow. - Many of them already have a better ecosystem than D-Bus. e.g. interoperability between Protobuf and Cap'n'proto implementations is good. Protobuf in most (all?) runtimes supports dynamically reading a schema and parsing binary wire format with it, as well as code generation. You can also maintain backwards compatibility in these formats by following relatively simple rules that can be statically-enforced. - JSON and JSON-but-binary have some annoying downsides. I really don't think field names of composite types belong as part of the ABI. JSON-like formats also often have to try to deal with the fact that JSON doesn't strictly define all semantics. Some of them differ from JSON is subtle ways, so supporting both JSON and sorta-JSON can lead to nasty side-effects. Maybe most importantly, since we're not writing software that's speaking to web browsers, JSON isn't even particularly convenient to begin with. A lot of the software will be in C and Rust most likely. It helps a bit for scripting languages like Python, but I'm not convinced it's worth the downsides.
- otabdeveloper4 2y ago
  Sorry, but bash isn't a "niche language" and it doesn't have an FFI story.
  - jchw 2y ago
    I don't know how to tell you this, but, you don't need to implement an RPC protocol in bash, nor do you need FFI. You can use CLI tools like `dbus-send`. I pray to God nothing meaningful is actually doing what you are insinuating in any serious environment.
    - otabdeveloper4 2y ago
      I'm trying to tell you that something that isn't straceable and greappable shouldn't belong in your system services stack.

jchw 2y ago

I think it doesn't have to be a message bus per-se, that design decision is mostly just because it's convenient. Even D-Bus can actually be used without the bus, if you really want to. D-Bus is just a bus so it can solve a bunch of different IPC problems at the same time. e.g. D-Bus can handle ACLs/permissions on RPC methods, D-Bus can handle multiple producers and multiple consumers, and so on. I think ultimately all that's really needed is IPC, and having a unified bus is just to allow for some additional use cases that are harder if you're only using UNIX domain sockets directly. If there are going to be systemd components that have IPC, then I'd argue they should probably use something standard rather than something bespoke. It's good to not re-invent the wheel. Not that I think Varlink is any better. It seems like at best it's probably a lateral move. I hope this does not go forward.

nbbnbb 2y ago
> If there are going to be systemd components that have IPC, then I'd argue they should probably use something standard rather than something bespoke. It's good to not re-invent the wheel. This is my point. My favourite DBus situation a number of years ago was a CentOS 7 box that reboot stopped working on with a cryptic DBus error that no one has ever seen before. I had to sync it, power cycle the node from the ILO card and cross my fingers. I really don't give a shit about this. I just wanted to run my jobs on the node, not turn into a sysadmin due to someone else's dubious architectural decisions.
- jchw 2y ago
  Yes but the systemd developers don't want to implement their own protocols with e.g. ACL checking, and given some of their track record I kind of think you don't want them to, either. I'm pretty sure the error conditions would be even more bespoke if they "just" used UNIX domain sockets directly. Don't get me wrong, there's nothing particularly wrong with UNIX domain sockets, but there's no "standard" protocols for communicating over UDS.
  - amluto 2y ago
    This is systemd we’re talking about. A service manager that already mucks with mount namespaces. It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead… > Public varlink interfaces are registered system-wide by their well-known address, by default /run/org.varlink.resolver. The resolver translates a given varlink interface to the service address which provides this interface. …we have well known names, and sandboxing, or replacing a service for just one client, remains a mess. Sigh.

therein 2y ago

Yeah, how will number/float serialization go? Are we going to serialize them as strings and parse them? That abstraction isn't handled the same way across multiple languages.

poettering 2y ago

Varlink is not a message bus. Hence you should be happy?