7 ms·

From “Design Decisions”[1]: > JSON. The protocol for front-end / back-end communication, as well as between the back-end and plug-ins, is based on simple JSON

• saurik 8y ago
  ... claims the project whose contributor is here claiming that they are "struggling" with JSON performance. Yeah... "in the noise". LOL.
  • uasm 8y ago
    But... how else are the people that have never seen a byte array or had to flip endianness will be able to write plugins for my text editor?
  • e98cuenc 8y ago
    It seems they're getting parsing times 1,000x slower than any other parser, 10,000x slower than simdjson. The complaint is understandable, but ironic :)
    • raphlinus 8y ago
      These numbers are not quite right for a variety of reasons (performance measurement methodology is hard), but to do something more of an apples-to-apples comparison, it's about 50x slower than serde in Rust. That's still a lot, obviously.
  • lifthrasiir 8y ago
    You misread the rationale. He is arguing that, with all conditions same, the difference between binary formats and JSON would be in the noise. It is often the case that the object construction is more costly than the JSON parsing, and you can't fix that with binary formats. As a minimal and extremely non-scientific benchmark, I've constructed a simple fixed data structure that encodes to JSON (using Python `json` module) and simple binary formats (that would be an ideal case for Python `struct` module). Decoding the same simple value 1,000,000 times in CPython 3.6.4 took... Format Size Iters. Speed ------ ---- --------- ---------- JSON 28 205,000 5.75 MB/s Struct 6 2,400,000 14.4 MB/s Of course YMMV, but even the `struct` module was only 2--12 times (depending on what you care about) faster than the `json` module in this particular case. And this is really minimal, you need an (slow) interpreted code for more complex binary formats. Right, you can use PyPy for the JIT compilation or binary modules for sidestepping the interpreter overhead! The point is that, it of course matters, but not quite drastic improvements you'd imagine.
    • orbifold 8y ago
      This is a super flawed argument. Clearly flat buffers and even protocol buffers are faster to serialize and deserialize than json, regardless of what you benchmark in python.
      • rapsey 8y ago
        And for the amount of messages that are being sent, the speed difference is irrelevant. This is the same conclusion sqlite developers came to. They tested turning JSON column types to binary and the speed difference was not large enough to warrant maintaining that code so they kept the data in JSON.
        • IshKebab 8y ago
          If the speed different is irrelevant, why are they struggling with it?
          • rapsey 8y ago
            Because most implementations are reasonably efficient. Swift default one is apparently not.
      • lifthrasiir 8y ago
        So you claim that decoding flat buffers and protobuf is faster than decoding with `struct`? I'm pretty much aware of various flaws and even stated some, but I barely buy that claim without a separate benchmark (which I really welcome by the way). At least I fully understand what the `struct` module actually does under the hood---it sorta compiles to a list of fields and "interprets" the dead simple VM in C. Oh, of course I've used the precompiled `struct.Struct` for that reason (but it was only 20% faster). Anyway, this arrangement is typical for most schematic serialization formats in any language: a bunch of function calls for gluing the desired format, plus a set of well-optimized core functions (not necessarily written in C :-). Henceforth my justification that this is close to the "bare-bones" serialization format.
      • 8y ago
  • deleted 8y ago
    [deleted]
• nothrabannosir 8y ago
  So is it too slow or not?
  • pdpi 8y ago
    It's not a binary yes/no question. Given equally-high quality JSON and binary serdes, JSON is sufficiently fast. Raphlinus is saying that Swift's built-in deserialiser is obnoxiously slow.
    • coldtea 8y ago
      Any reason not to just use a third party Swift JSON library?
  • roblabla 8y ago
    Xi has multiple components written in multiple languages. In the rust core, json de/serialization is not a problem, but swift is lacking a similar high-performance library.
  • raphlinus 8y ago
    We actually do get 60fps, but JSON parsing on the Swift side takes more than its share of total CPU load, affecting power consumption among other things. So (partly to address the trolls elsewhere in the thread), the choice of JSON does not preclude fast implementation (as the existence of simdjson proves), but it does make it dependent on the language having a performant JSON implementation. I made the assumption that this would be the case, and for Swift it isn't.
    • dilap 8y ago
      At some point though, isn't it maybe easier just to use an inherently more efficient format than trying to rely on clever implementations to save you? I totally get json for public internet services where you want to have lots of consumers and using a more efficient format would be significant friction, but writing an editor frontend is a very large endeavor -- it seems like the extra work of adopting something more efficient than json (like flatbuffers or whatever) would really be in the noise.
• anderspitman 8y ago
  I'm going off topic at this point but I'd think for a native app the main advantages of a binary format would be the static typing and code generation that come from using an IDL.
  • greglindahl 8y ago
    You can have a binary format that's self-describing. It's important to understand all of the independent parts that go into a format.
  • wwright 8y ago
    Rust (the language the Xi core is developed in) has static typing for JSON, as well as other serialization formats: https://github.com/serde-rs/json https://github.com/serde-rs/json
    • anderspitman 8y ago
      I'm familiar with serde. It's an incredible project, but I wouldn't quite call it "static typing for JSON". You still have to unwrap the parse at some point. However, I will concede the point that if you have Rust on both sides then you'll get most of the benefits.