3 ms·

Well there sort of is but people don't tend to know or use it. If it's within the same machine and architecture, which should be the case for an init system, th

by nbbnbb 2y ago

Well there sort of is but people don't tend to know or use it. If it's within the same machine and architecture, which should be the case for an init system, then a fixed size struct can be written and read trivially.

jchw 2y ago
C structs are a terrible serialization format, since they are not a serialization format at all. Nothing guarantees that you will get consistent struct behavior on the same machine, but also, it only really solves the problem for C. For everything else, you have to duplicate the C structure exactly, including however it may vary per architecture (e.g. due to alignment.) And OK fine. It's not that bad, most C ABIs are able to work around this reasonably OK (not all of them but sure, let's just call it a skill issue.) But then what do you do when you want anything more complicated than a completely fixed-size type? Like for example... a string. Or an array. Now we can't just use a struct, a single request will need to be split into multiple different structures at the bare minimum. And plus, there's no real reason to limit this all to the same machine. Tunneling UNIX domain sockets over the network is perfectly reasonable behavior and most* SSH implementations these days support this. So I think scoping the interoperability to "same machine" is unnecessarily limiting, especially when it's not actually hard to write consistent de/serialization in any language. * At least the ones I can think of, like OpenSSH[1], Go's x/crypto/ssh[2], and libssh2[3]. [1]: https://www.openssh.com/txt/release-6.7 https://www.openssh.com/txt/release-6.7 [2]: https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix [3]: https://github.com/libssh2/libssh2/pull/945 https://github.com/libssh2/libssh2/pull/945
- nbbnbb 2y ago
  Note within the domain of this problem was the point. Which means on the same machine, with the same architecture and both ends being C which is what the init system is written in. You are adding more problems that don't exist to the specification. As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing.
  - jchw 2y ago
    > You are adding more problems that don't exist to the specification. D-Bus does in fact already have support for remoting, and like I said, you can tunnel it today. I'm only suggesting it because I have in fact tunneled D-Bus over the network to call into systemd specifically, already! > As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing. OK. So... waste an entire page of memory for each string. And then we avoid all of that parsing, but the resulting code is horribly error-prone. And then it still doesn't work if you actually want really large strings, and it also doesn't do much to answer arrays of other things like structures. Can you maybe see why this is compelling to virtually nobody?
  - bsder 2y ago
    > As for strings, just shove a char[4096] in there. For the love of God, use a proper slice/fat pointer, please. Switching over to slices eliminates 90%+ of the issues with using C. Carrying around the base and the length eliminates a huge number of the overrun issues (especially if you don't store them consecutively). Splitting the base and the offset gives a huge amount of semantic information and makes serialization vastly easier.
    - jchw 2y ago
      Broadly, I agree with you. C strings were a mistake. The standard library is full of broken shit that nobody should use, and worse, due to the myriad of slightly different "safe" string library functions, (a completely different subset of which is supported on any given platform,) which all have different edge cases, many people are over-confident that their C string code is actually correct. But is it? Is it checking errors? Does your function ensure that the destination buffer is null-terminated when it fails? Are you sure you don't have any off-by-one issues anywhere? Correct as you may be though the argument here is that you should just write raw structs into Unix sockets. In this case you can't really use pointers. So, realistically, no slices either. In this context a fixed-size buffer is quite literally the only sensible thing you can do, but also, I think it's a great demonstration of why you absolutely shouldn't do this. That said, if we're willing to get rid of the constraint of using only one plain C struct, you could use offsets instead of pointers. Allocate some contiguous chunk of memory for your entire request, place struct/strings/etc. in it, and use relative offsets. Then on the receiver side you just need some fairly basic validation checks to ensure none of the offsets go out of bounds. But at that point, you've basically invented part of Cap'n'proto, which begs the question... Why not just use something like that instead. It's pretty much the entire reason they were invented. Oh well. Unfortunately the unforced errors of D-Bus seem like they will lead to an overcorrection in the other direction, turning the core of our operating system into something that I suspect nobody will love in the long term.
- cryptonector 2y ago
  BTW, Lustre's RPC system's serialization is very much based on C structs. It's receiver-makes-right to deal with endianness, for example. It's a pain, but it's also fast.
  - jchw 2y ago
    Making an RPC serialization system that is zero-overhead i.e. can use the same format on the wire as it does on disk is not a terrible idea. Capnp is the serialization format that I've been suggesting as a potential candidate and it is basically just taking the idea of C structs, dumping it into it's own schema language, and adding the bare minimum to get some protobuf-like semantics.
    - cryptonector 2y ago
      Well, Lustre RPC doesn't use on-disk data structures on the wire, though that is indeed an interesting idea. In Lustre RPC _control_ messages go over one channel and they're all a C structure(s) with sender encoding hints so the receiver can make it right, and any variable-length payloads go in separate chunks trailing the C structures. Whereas bulk _data_ is done with RDMA, and there's no C structures in sight for that. Capnp sounds about right for encoding rules. The way I'd do it: - target 64-bit architectures (32-bit senders have to do work to encode, but 64-bit senders don't) - assume C-style struct packing rules in the host language (but not #pragma packed) - use an arena allocator - transmit {archflags, base pointer, data} - receiver makes right: - swab if necessary - fix interior pointers - fail if there are pointers to anything outside the received data - convert to 32-bit if the receiver is 32-bit (That's roughly what Lustre RPC does.) As for syntax, I'd build an "ASN.2" that has a syntax that's parseable with LALR(1), dammit, and which is more like what today's devs are used to, but which is otherwise 100% equivalent to ASN.1.
      - jchw 2y ago
        Out of curiosity, why not use offsets instead of pointers? That's what capnp does. I assume offset calculation is going to be efficient on most platforms. This removes the need for fixing up pointers; instead you just need to check bounds.

by nbbnbb 2y ago

jchw 2y ago

C structs are a terrible serialization format, since they are not a serialization format at all. Nothing guarantees that you will get consistent struct behavior on the same machine, but also, it only really solves the problem for C. For everything else, you have to duplicate the C structure exactly, including however it may vary per architecture (e.g. due to alignment.) And OK fine. It's not that bad, most C ABIs are able to work around this reasonably OK (not all of them but sure, let's just call it a skill issue.) But then what do you do when you want anything more complicated than a completely fixed-size type? Like for example... a string. Or an array. Now we can't just use a struct, a single request will need to be split into multiple different structures at the bare minimum. And plus, there's no real reason to limit this all to the same machine. Tunneling UNIX domain sockets over the network is perfectly reasonable behavior and most* SSH implementations these days support this. So I think scoping the interoperability to "same machine" is unnecessarily limiting, especially when it's not actually hard to write consistent de/serialization in any language. * At least the ones I can think of, like OpenSSH[1], Go's x/crypto/ssh[2], and libssh2[3]. [1]: https://www.openssh.com/txt/release-6.7 https://www.openssh.com/txt/release-6.7 [2]: https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix [3]: https://github.com/libssh2/libssh2/pull/945 https://github.com/libssh2/libssh2/pull/945

nbbnbb 2y ago
Note within the domain of this problem was the point. Which means on the same machine, with the same architecture and both ends being C which is what the init system is written in. You are adding more problems that don't exist to the specification. As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing.
- jchw 2y ago
  > You are adding more problems that don't exist to the specification. D-Bus does in fact already have support for remoting, and like I said, you can tunnel it today. I'm only suggesting it because I have in fact tunneled D-Bus over the network to call into systemd specifically, already! > As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing. OK. So... waste an entire page of memory for each string. And then we avoid all of that parsing, but the resulting code is horribly error-prone. And then it still doesn't work if you actually want really large strings, and it also doesn't do much to answer arrays of other things like structures. Can you maybe see why this is compelling to virtually nobody?
- bsder 2y ago
  > As for strings, just shove a char[4096] in there. For the love of God, use a proper slice/fat pointer, please. Switching over to slices eliminates 90%+ of the issues with using C. Carrying around the base and the length eliminates a huge number of the overrun issues (especially if you don't store them consecutively). Splitting the base and the offset gives a huge amount of semantic information and makes serialization vastly easier.
  - jchw 2y ago
    Broadly, I agree with you. C strings were a mistake. The standard library is full of broken shit that nobody should use, and worse, due to the myriad of slightly different "safe" string library functions, (a completely different subset of which is supported on any given platform,) which all have different edge cases, many people are over-confident that their C string code is actually correct. But is it? Is it checking errors? Does your function ensure that the destination buffer is null-terminated when it fails? Are you sure you don't have any off-by-one issues anywhere? Correct as you may be though the argument here is that you should just write raw structs into Unix sockets. In this case you can't really use pointers. So, realistically, no slices either. In this context a fixed-size buffer is quite literally the only sensible thing you can do, but also, I think it's a great demonstration of why you absolutely shouldn't do this. That said, if we're willing to get rid of the constraint of using only one plain C struct, you could use offsets instead of pointers. Allocate some contiguous chunk of memory for your entire request, place struct/strings/etc. in it, and use relative offsets. Then on the receiver side you just need some fairly basic validation checks to ensure none of the offsets go out of bounds. But at that point, you've basically invented part of Cap'n'proto, which begs the question... Why not just use something like that instead. It's pretty much the entire reason they were invented. Oh well. Unfortunately the unforced errors of D-Bus seem like they will lead to an overcorrection in the other direction, turning the core of our operating system into something that I suspect nobody will love in the long term.
cryptonector 2y ago
BTW, Lustre's RPC system's serialization is very much based on C structs. It's receiver-makes-right to deal with endianness, for example. It's a pain, but it's also fast.
- jchw 2y ago
  Making an RPC serialization system that is zero-overhead i.e. can use the same format on the wire as it does on disk is not a terrible idea. Capnp is the serialization format that I've been suggesting as a potential candidate and it is basically just taking the idea of C structs, dumping it into it's own schema language, and adding the bare minimum to get some protobuf-like semantics.
  - cryptonector 2y ago
    Well, Lustre RPC doesn't use on-disk data structures on the wire, though that is indeed an interesting idea. In Lustre RPC _control_ messages go over one channel and they're all a C structure(s) with sender encoding hints so the receiver can make it right, and any variable-length payloads go in separate chunks trailing the C structures. Whereas bulk _data_ is done with RDMA, and there's no C structures in sight for that. Capnp sounds about right for encoding rules. The way I'd do it: - target 64-bit architectures (32-bit senders have to do work to encode, but 64-bit senders don't) - assume C-style struct packing rules in the host language (but not #pragma packed) - use an arena allocator - transmit {archflags, base pointer, data} - receiver makes right: - swab if necessary - fix interior pointers - fail if there are pointers to anything outside the received data - convert to 32-bit if the receiver is 32-bit (That's roughly what Lustre RPC does.) As for syntax, I'd build an "ASN.2" that has a syntax that's parseable with LALR(1), dammit, and which is more like what today's devs are used to, but which is otherwise 100% equivalent to ASN.1.
    - jchw 2y ago
      Out of curiosity, why not use offsets instead of pointers? That's what capnp does. I assume offset calculation is going to be efficient on most platforms. This removes the need for fixing up pointers; instead you just need to check bounds.