4 ms·

Sigh. Varlink is designed for IPC, so the expected data rate is maybe hundreds of messages per second in the worst case. JSON can be parsed at 3 Gigabytes/sec

by theamk 2y ago

Sigh. Varlink is designed for IPC, so the expected data rate is maybe hundreds of messages per second in the worst case. JSON can be parsed at 3 Gigabytes/second [0]. Even unoptimized stdlib parsers in scripting languages get 50 MB/sec. That's more than enough, standard library support is much more important for a project like this. And if there is ever a reason to send tar archive or video data, there is always "upgrade" option to switch to raw TCP sockets, with no JSON overhead at all. [0] https://news.ycombinator.com/item?id=19214387 https://news.ycombinator.com/item?id=19214387 [1] https://github.com/TeskaLabs/cysimdjson https://github.com/TeskaLabs/cysimdjson

fanf2 2y ago
Hundreds of messages per second sounds like the performance I would expect from a 1980s computer. Why so incredibly slow?
- orf 2y ago
  Read the comment you’re replying to. In it OP says it’s not slow, and the “hundreds of messages per second” you are referring to is not regarding the performance but the expected rate. It’s possible for something to be fast, but only happen infrequently.
  - ptx 2y ago
    It just seems strange that the message rate is assumed to be in some very specific range where the performance of D-Bus is a problem (mentioned in the slides) yet the performance overhead of JSON is not a problem.
userbinator 2y ago
It doesn't matter what the rate is. Of all the things they could do, they decided to choose one of the worst possible formats for IPC data[1], maybe to pander to the web crowd. [1] Maybe XML would be a close competitor.
- Spivak 2y ago
  It's the lingua-franca serialization format that has bindings to every programming language already, doesn't require agreeing on the payload layout beforehand, is beyond overkill for the task being asked of it, and is human debuggable over the wire. I'm begging you to read the format it's replacing. https://dbus.freedesktop.org/doc/dbus-specification.html https://dbus.freedesktop.org/doc/dbus-specification.html Ctrl+f Type System. JSON is beautiful, elegant, and minimal by comparison. yyyyuua(yv) It's like the printf message protocol but worse.
  - quotemstr 2y ago
    JSON isn't simple. It's simplistic. It's too simple. By refusing to acknowledge the complexity of the real world at the protocol layer, varlink shifts the burden of dealing with the complexity from one transport layer to N ad hoc application layers, making us all worse off.
    - Spivak 2y ago
      Do you mean https://varlink.org/Interface-Definition https://varlink.org/Interface-Definition? JSON is just the marshaling format. All the language bindings don't need a fancy parser outside of json.loads. I wish the interface definitions themselves had been specified in JSON as well. Peep the Python interface parser: https://github.com/varlink/python/blob/master/varlink/scanner.py https://github.com/varlink/python/blob/master/varlink/scanne.... This is the quality you can expect for an ad-hoc format, which is to say, pretty bad. Making the hardest part farming out to a parsing library you already have and is a built-in in your language is a no-brainer choice.
    - foul 2y ago
      Being simplistic is a leitmotiv of the project, at least someone in the post-GNOME cabal stopped reinventing the wheel.
  - dathinab 2y ago
    > lingua-franca serialization format it isn't, fundamentally can't be as it has no support for binary blobs and base64 encoded strings are a pretty terrible solution for this and the space overhead of many small number can also be pretty bad and while a lot of IPC use cases are so little performance sensitive that JSON is more then good enough and large blobs are normally not send over messages (instead you e.g. send a fd) there are still some use-cases where you use IPC to listen to system events and you have a situation where a pretty insane amount of them is emitted, in which case JSON might come back and bite you (less due to marshaling performance and more due to the space overhead which depending on the kind of events can easily be 33%+). But what I do not know is how relevant such edge cases are. Probably the biggest issue might be IPC system mainly used on 64bit systems not being able to properly represent all 64 bit integers .... but probably that is fine too.
deleted 2y ago
[deleted]

by theamk 2y ago

fanf2 2y ago

Hundreds of messages per second sounds like the performance I would expect from a 1980s computer. Why so incredibly slow?

orf 2y ago
Read the comment you’re replying to. In it OP says it’s not slow, and the “hundreds of messages per second” you are referring to is not regarding the performance but the expected rate. It’s possible for something to be fast, but only happen infrequently.
- ptx 2y ago
  It just seems strange that the message rate is assumed to be in some very specific range where the performance of D-Bus is a problem (mentioned in the slides) yet the performance overhead of JSON is not a problem.

userbinator 2y ago

It doesn't matter what the rate is. Of all the things they could do, they decided to choose one of the worst possible formats for IPC data[1], maybe to pander to the web crowd. [1] Maybe XML would be a close competitor.

Spivak 2y ago
It's the lingua-franca serialization format that has bindings to every programming language already, doesn't require agreeing on the payload layout beforehand, is beyond overkill for the task being asked of it, and is human debuggable over the wire. I'm begging you to read the format it's replacing. https://dbus.freedesktop.org/doc/dbus-specification.html https://dbus.freedesktop.org/doc/dbus-specification.html Ctrl+f Type System. JSON is beautiful, elegant, and minimal by comparison. yyyyuua(yv) It's like the printf message protocol but worse.
- quotemstr 2y ago
  JSON isn't simple. It's simplistic. It's too simple. By refusing to acknowledge the complexity of the real world at the protocol layer, varlink shifts the burden of dealing with the complexity from one transport layer to N ad hoc application layers, making us all worse off.
  - Spivak 2y ago
    Do you mean https://varlink.org/Interface-Definition https://varlink.org/Interface-Definition? JSON is just the marshaling format. All the language bindings don't need a fancy parser outside of json.loads. I wish the interface definitions themselves had been specified in JSON as well. Peep the Python interface parser: https://github.com/varlink/python/blob/master/varlink/scanner.py https://github.com/varlink/python/blob/master/varlink/scanne.... This is the quality you can expect for an ad-hoc format, which is to say, pretty bad. Making the hardest part farming out to a parsing library you already have and is a built-in in your language is a no-brainer choice.
  - foul 2y ago
    Being simplistic is a leitmotiv of the project, at least someone in the post-GNOME cabal stopped reinventing the wheel.
- dathinab 2y ago
  > lingua-franca serialization format it isn't, fundamentally can't be as it has no support for binary blobs and base64 encoded strings are a pretty terrible solution for this and the space overhead of many small number can also be pretty bad and while a lot of IPC use cases are so little performance sensitive that JSON is more then good enough and large blobs are normally not send over messages (instead you e.g. send a fd) there are still some use-cases where you use IPC to listen to system events and you have a situation where a pretty insane amount of them is emitted, in which case JSON might come back and bite you (less due to marshaling performance and more due to the space overhead which depending on the kind of events can easily be 33%+). But what I do not know is how relevant such edge cases are. Probably the biggest issue might be IPC system mainly used on 64bit systems not being able to properly represent all 64 bit integers .... but probably that is fine too.

deleted 2y ago

[deleted]