4 ms·

Yes but the systemd developers don't want to implement their own protocols with e.g. ACL checking, and given some of their track record I kind of think you don'

by jchw 2y ago

Yes but the systemd developers don't want to implement their own protocols with e.g. ACL checking, and given some of their track record I kind of think you don't want them to, either. I'm pretty sure the error conditions would be even more bespoke if they "just" used UNIX domain sockets directly. Don't get me wrong, there's nothing particularly wrong with UNIX domain sockets, but there's no "standard" protocols for communicating over UDS.

amluto 2y ago
This is systemd we’re talking about. A service manager that already mucks with mount namespaces. It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead… > Public varlink interfaces are registered system-wide by their well-known address, by default /run/org.varlink.resolver. The resolver translates a given varlink interface to the service address which provides this interface. …we have well known names, and sandboxing, or replacing a service for just one client, remains a mess. Sigh.
- guappa 2y ago
  Please, your trolling is not really welcome. > It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead… Can you link to your PR where you solved the problem?
nbbnbb 2y ago
Well there sort of is but people don't tend to know or use it. If it's within the same machine and architecture, which should be the case for an init system, then a fixed size struct can be written and read trivially.
- jchw 2y ago
  C structs are a terrible serialization format, since they are not a serialization format at all. Nothing guarantees that you will get consistent struct behavior on the same machine, but also, it only really solves the problem for C. For everything else, you have to duplicate the C structure exactly, including however it may vary per architecture (e.g. due to alignment.) And OK fine. It's not that bad, most C ABIs are able to work around this reasonably OK (not all of them but sure, let's just call it a skill issue.) But then what do you do when you want anything more complicated than a completely fixed-size type? Like for example... a string. Or an array. Now we can't just use a struct, a single request will need to be split into multiple different structures at the bare minimum. And plus, there's no real reason to limit this all to the same machine. Tunneling UNIX domain sockets over the network is perfectly reasonable behavior and most* SSH implementations these days support this. So I think scoping the interoperability to "same machine" is unnecessarily limiting, especially when it's not actually hard to write consistent de/serialization in any language. * At least the ones I can think of, like OpenSSH[1], Go's x/crypto/ssh[2], and libssh2[3]. [1]: https://www.openssh.com/txt/release-6.7 https://www.openssh.com/txt/release-6.7 [2]: https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix [3]: https://github.com/libssh2/libssh2/pull/945 https://github.com/libssh2/libssh2/pull/945
  - nbbnbb 2y ago
    Note within the domain of this problem was the point. Which means on the same machine, with the same architecture and both ends being C which is what the init system is written in. You are adding more problems that don't exist to the specification. As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing.
    - jchw 2y ago
      > You are adding more problems that don't exist to the specification. D-Bus does in fact already have support for remoting, and like I said, you can tunnel it today. I'm only suggesting it because I have in fact tunneled D-Bus over the network to call into systemd specifically, already! > As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing. OK. So... waste an entire page of memory for each string. And then we avoid all of that parsing, but the resulting code is horribly error-prone. And then it still doesn't work if you actually want really large strings, and it also doesn't do much to answer arrays of other things like structures. Can you maybe see why this is compelling to virtually nobody?
    - bsder 2y ago
      > As for strings, just shove a char[4096] in there. For the love of God, use a proper slice/fat pointer, please. Switching over to slices eliminates 90%+ of the issues with using C. Carrying around the base and the length eliminates a huge number of the overrun issues (especially if you don't store them consecutively). Splitting the base and the offset gives a huge amount of semantic information and makes serialization vastly easier.
      - jchw 2y ago
        Broadly, I agree with you. C strings were a mistake. The standard library is full of broken shit that nobody should use, and worse, due to the myriad of slightly different "safe" string library functions, (a completely different subset of which is supported on any given platform,) which all have different edge cases, many people are over-confident that their C string code is actually correct. But is it? Is it checking errors? Does your function ensure that the destination buffer is null-terminated when it fails? Are you sure you don't have any off-by-one issues anywhere? Correct as you may be though the argument here is that you should just write raw structs into Unix sockets. In this case you can't really use pointers. So, realistically, no slices either. In this context a fixed-size buffer is quite literally the only sensible thing you can do, but also, I think it's a great demonstration of why you absolutely shouldn't do this. That said, if we're willing to get rid of the constraint of using only one plain C struct, you could use offsets instead of pointers. Allocate some contiguous chunk of memory for your entire request, place struct/strings/etc. in it, and use relative offsets. Then on the receiver side you just need some fairly basic validation checks to ensure none of the offsets go out of bounds. But at that point, you've basically invented part of Cap'n'proto, which begs the question... Why not just use something like that instead. It's pretty much the entire reason they were invented. Oh well. Unfortunately the unforced errors of D-Bus seem like they will lead to an overcorrection in the other direction, turning the core of our operating system into something that I suspect nobody will love in the long term.

by jchw 2y ago

amluto 2y ago

This is systemd we’re talking about. A service manager that already mucks with mount namespaces. It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead… > Public varlink interfaces are registered system-wide by their well-known address, by default /run/org.varlink.resolver. The resolver translates a given varlink interface to the service address which provides this interface. …we have well known names, and sandboxing, or replacing a service for just one client, remains a mess. Sigh.

guappa 2y ago
Please, your trolling is not really welcome. > It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead… Can you link to your PR where you solved the problem?

nbbnbb 2y ago

Well there sort of is but people don't tend to know or use it. If it's within the same machine and architecture, which should be the case for an init system, then a fixed size struct can be written and read trivially.

jchw 2y ago
C structs are a terrible serialization format, since they are not a serialization format at all. Nothing guarantees that you will get consistent struct behavior on the same machine, but also, it only really solves the problem for C. For everything else, you have to duplicate the C structure exactly, including however it may vary per architecture (e.g. due to alignment.) And OK fine. It's not that bad, most C ABIs are able to work around this reasonably OK (not all of them but sure, let's just call it a skill issue.) But then what do you do when you want anything more complicated than a completely fixed-size type? Like for example... a string. Or an array. Now we can't just use a struct, a single request will need to be split into multiple different structures at the bare minimum. And plus, there's no real reason to limit this all to the same machine. Tunneling UNIX domain sockets over the network is perfectly reasonable behavior and most* SSH implementations these days support this. So I think scoping the interoperability to "same machine" is unnecessarily limiting, especially when it's not actually hard to write consistent de/serialization in any language. * At least the ones I can think of, like OpenSSH[1], Go's x/crypto/ssh[2], and libssh2[3]. [1]: https://www.openssh.com/txt/release-6.7 https://www.openssh.com/txt/release-6.7 [2]: https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix [3]: https://github.com/libssh2/libssh2/pull/945 https://github.com/libssh2/libssh2/pull/945
- nbbnbb 2y ago
  Note within the domain of this problem was the point. Which means on the same machine, with the same architecture and both ends being C which is what the init system is written in. You are adding more problems that don't exist to the specification. As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing.
  - jchw 2y ago
    > You are adding more problems that don't exist to the specification. D-Bus does in fact already have support for remoting, and like I said, you can tunnel it today. I'm only suggesting it because I have in fact tunneled D-Bus over the network to call into systemd specifically, already! > As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing. OK. So... waste an entire page of memory for each string. And then we avoid all of that parsing, but the resulting code is horribly error-prone. And then it still doesn't work if you actually want really large strings, and it also doesn't do much to answer arrays of other things like structures. Can you maybe see why this is compelling to virtually nobody?
  - bsder 2y ago
    > As for strings, just shove a char[4096] in there. For the love of God, use a proper slice/fat pointer, please. Switching over to slices eliminates 90%+ of the issues with using C. Carrying around the base and the length eliminates a huge number of the overrun issues (especially if you don't store them consecutively). Splitting the base and the offset gives a huge amount of semantic information and makes serialization vastly easier.
    - jchw 2y ago
      Broadly, I agree with you. C strings were a mistake. The standard library is full of broken shit that nobody should use, and worse, due to the myriad of slightly different "safe" string library functions, (a completely different subset of which is supported on any given platform,) which all have different edge cases, many people are over-confident that their C string code is actually correct. But is it? Is it checking errors? Does your function ensure that the destination buffer is null-terminated when it fails? Are you sure you don't have any off-by-one issues anywhere? Correct as you may be though the argument here is that you should just write raw structs into Unix sockets. In this case you can't really use pointers. So, realistically, no slices either. In this context a fixed-size buffer is quite literally the only sensible thing you can do, but also, I think it's a great demonstration of why you absolutely shouldn't do this. That said, if we're willing to get rid of the constraint of using only one plain C struct, you could use offsets instead of pointers. Allocate some contiguous chunk of memory for your entire request, place struct/strings/etc. in it, and use relative offsets. Then on the receiver side you just need some fairly basic validation checks to ensure none of the offsets go out of bounds. But at that point, you've basically invented part of Cap'n'proto, which begs the question... Why not just use something like that instead. It's pretty much the entire reason they were invented. Oh well. Unfortunately the unforced errors of D-Bus seem like they will lead to an overcorrection in the other direction, turning the core of our operating system into something that I suspect nobody will love in the long term.