5 ms·

Varlink – A plain-text, type-safe, discoverable, self-documenting interface

60 points by 1_player 7y ago

ar-nelson 7y ago
So... it looks like this is an RPC protocol, similar to JSON-RPC, but the word "RPC" doesn't appear anywhere on the homepage or the FAQ. And it has an interface description language, similar to Protobuf. It seems like it might be a nice middle ground between Protobuf (strongly-typed, specified) and JSON-RPC (human-readable, doesn't need a special library). But requiring clients to use a new RPC format is still a tough sell. What are the main advantages of Varlink over more mainstream RPC formats?
- 1_player 7y ago
  I read about Varlink while taking a look at podman, which uses it as a remote interface [1]. I don't think it does anything _much_ different than protobuf or JSON-RPC but simply IMO tries to create a common interface to interact with software or daemons. Protobuf is by itself just a protocol and encoding format, while Varlink wants to create a generic network-independent RPC interface you can easily access from your shell. I'm not sure I understand what's the underlying transport protocol (HTTP? custom one?) but having JSON as encoding format is pretty nice, any shell script could talk with it and it's eas(ier) to inspect. It looks like a generalised version of DBus, without requiring a central daemon. I think it looks pretty cool. EDIT: also it seems to natively handle subscriptions/polling for changes. EDIT 2: the transport protocol is just read/writing raw JSON to the file descriptor/socket, see the strace snippet at the bottom of https://github.com/varlink/linux-varlink https://github.com/varlink/linux-varlink 1: https://github.com/containers/libpod/blob/master/cmd/podman/varlink/io.podman.varlink https://github.com/containers/libpod/blob/master/cmd/podman/...
  - juped 7y ago
    >having JSON as encoding format is pretty nice, any shell script could talk with it and it's eas(ier) to inspect Why do people say things like this? The only format unix shell can reliably understand is tab-separated value.
    - 1_player 7y ago
      The shell can't speak HTTP either, but all OSes ship with a command line HTTP client. My point is that in 2019 JSON is such a common format _most_ people have or should have a JSON client encoder/decoder installed in their /bin, though I'm not sure any OS ships with any by default. To be fair, I have jq installed and I have no idea how to use it.
      - Spivak 7y ago
        I really love the jq language but I totally agree that the complexity skyrockets once you peek at the 'advanced' features because this neat little query language turns into a full programming language with io, variables, modules, functions. Very similar to AWK. If you want something that is just a good query language then I would highly recommend JMESPath. It's a little harder to grok initially but the spec is extremely detailed, comes with examples, and there's libraries for all the 'popular' programming languages. A good CLI tool that wraps the Go library is https://github.com/jmespath/jp https://github.com/jmespath/jp
Spivak 7y ago
Since this seems to be a project by Red Hat developers I'm curious to see how this fits in with dbus which seems to have overlapping concerns.
tln 7y ago
This article gives some nice background. https://lwn.net/Articles/742675/ https://lwn.net/Articles/742675/ It would be interesting for the shell to open a varlink connection to commands to allow progress updates. I could also see shells command completion interface being greatly simplified by using varlink.
osdev 7y ago
What are the pros/cons of using a custom built language/parser such as this(or others like googles gRPC IDL, etc), versus simply creating a DSL(Domain Specific Language) using an existing language like Kotlin,Scala et? I feel that leveraging an existing language with good DSL support, along with an existing IDE (e.g. IntelliJ) would be a lot more pragmatic, easier to support and to potentially maintain.
- johnny22 7y ago
  it is going to be in C due to all the various bindings to Rust, Python, JS, Go, etc. Nobody is gonna want programs that currently rely on the existing dbus libraries to rely on a java VM.
armitron 7y ago
This is terrible, I hope it doesn't catch on. There is absolutely no reason for a text format here, especially one as naive and inefficient as this. Binary messages that include size and version _at the beginning_ would be the expected solution.

60 points by 1_player 7y ago

So... it looks like this is an RPC protocol, similar to JSON-RPC, but the word "RPC" doesn't appear anywhere on the homepage or the FAQ. And it has an interface description language, similar to Protobuf. It seems like it might be a nice middle ground between Protobuf (strongly-typed, specified) and JSON-RPC (human-readable, doesn't need a special library). But requiring clients to use a new RPC format is still a tough sell. What are the main advantages of Varlink over more mainstream RPC formats?

1_player 7y ago
I read about Varlink while taking a look at podman, which uses it as a remote interface [1]. I don't think it does anything _much_ different than protobuf or JSON-RPC but simply IMO tries to create a common interface to interact with software or daemons. Protobuf is by itself just a protocol and encoding format, while Varlink wants to create a generic network-independent RPC interface you can easily access from your shell. I'm not sure I understand what's the underlying transport protocol (HTTP? custom one?) but having JSON as encoding format is pretty nice, any shell script could talk with it and it's eas(ier) to inspect. It looks like a generalised version of DBus, without requiring a central daemon. I think it looks pretty cool. EDIT: also it seems to natively handle subscriptions/polling for changes. EDIT 2: the transport protocol is just read/writing raw JSON to the file descriptor/socket, see the strace snippet at the bottom of https://github.com/varlink/linux-varlink https://github.com/varlink/linux-varlink 1: https://github.com/containers/libpod/blob/master/cmd/podman/varlink/io.podman.varlink https://github.com/containers/libpod/blob/master/cmd/podman/...
- juped 7y ago
  >having JSON as encoding format is pretty nice, any shell script could talk with it and it's eas(ier) to inspect Why do people say things like this? The only format unix shell can reliably understand is tab-separated value.
  - 1_player 7y ago
    The shell can't speak HTTP either, but all OSes ship with a command line HTTP client. My point is that in 2019 JSON is such a common format _most_ people have or should have a JSON client encoder/decoder installed in their /bin, though I'm not sure any OS ships with any by default. To be fair, I have jq installed and I have no idea how to use it.
    - Spivak 7y ago
      I really love the jq language but I totally agree that the complexity skyrockets once you peek at the 'advanced' features because this neat little query language turns into a full programming language with io, variables, modules, functions. Very similar to AWK. If you want something that is just a good query language then I would highly recommend JMESPath. It's a little harder to grok initially but the spec is extremely detailed, comes with examples, and there's libraries for all the 'popular' programming languages. A good CLI tool that wraps the Go library is https://github.com/jmespath/jp https://github.com/jmespath/jp

Spivak 7y ago

Since this seems to be a project by Red Hat developers I'm curious to see how this fits in with dbus which seems to have overlapping concerns.

tln 7y ago

This article gives some nice background. https://lwn.net/Articles/742675/ https://lwn.net/Articles/742675/ It would be interesting for the shell to open a varlink connection to commands to allow progress updates. I could also see shells command completion interface being greatly simplified by using varlink.

osdev 7y ago

What are the pros/cons of using a custom built language/parser such as this(or others like googles gRPC IDL, etc), versus simply creating a DSL(Domain Specific Language) using an existing language like Kotlin,Scala et? I feel that leveraging an existing language with good DSL support, along with an existing IDE (e.g. IntelliJ) would be a lot more pragmatic, easier to support and to potentially maintain.

johnny22 7y ago
it is going to be in C due to all the various bindings to Rust, Python, JS, Go, etc. Nobody is gonna want programs that currently rely on the existing dbus libraries to rely on a java VM.

armitron 7y ago

This is terrible, I hope it doesn't catch on. There is absolutely no reason for a text format here, especially one as naive and inefficient as this. Binary messages that include size and version _at the beginning_ would be the expected solution.