3 ms·
Good thought. If you are coding in C++ maybe you can use some sort of binary serialization thing. Even in other languages if json parsing is a bottleneck it can
by avg_dev 4y ago
Good thought. If you are coding in C++ maybe you can use some sort of binary serialization thing. Even in other languages if json parsing is a bottleneck it can possibly be optimized away through use of a binary wire format. That said vector operations available to programmers is always a welcome thing I’d say. And who knows how much production json parsing this library really does, it could be a ton.
I’m torn. I’ve worked at shops where we aim over time to reduce response time while serving business logic and using statistical models that get iterated on. Even there I haven’t seen a blatant need for non-JSON rpc. But I know my experience doesn’t mirror everyone’s. And I like seeing and learning about instruction sets. I’m currently taking a course in parallel computing and I just used a avx2 for the first time in a toy program to subtract one vector from another in a single instruction which while not particularly useful is a window into more interesting things and is still SIMD.
I think on the whole making json parsing for a large enough fraction of processors is probably a huge win for the environment. But who is parsing json in C++?
- ollien 4y ago> But who is parsing json in C++? Well, Facebook for one! Folly has lots of utilities for this (see folly::dynamic[1]). We make extensive use of this at my (non-Facebook) job. [1] https://github.com/facebook/folly/blob/master/folly/docs/Dynamic.md https://github.com/facebook/folly/blob/master/folly/docs/Dyn...
- refulgentis 4y agoProbably a lot of people. JSON is a pretty popular interchange format.
- neurostimulant 4y agoGTA Online used a very slow c++ json parser for years which struggles with parsing their json file (which has grown to 10MB over the years). The load time was pretty bad, in some cases it can be more than 15 minutes. Improving them actually cut down startup time by 70%. https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
- avg_dev 4y agoI just read this; I remember reading it a while back. It's a good article. Approachable and actually achieves its goal. Thanks for the link. > Original online mode load time: ~6m flat > Time with only duplication check patch: 4m 30s > Time with only JSON parser patch: 2m 50s > Time with both issues patched: 1m 50s > > (6 * 60 - (1 * 60+50)) / (6 * 60) = 69.4% load time improvement (nice!) I would have loved to see someone swap out the parser for simdjson, especially without the code. That would be amazing. Maybe the 1m50s can be beaten. Just thinking about that core heating up for 6m one can see that this sort of code improvement helps reduce electricity usage, heat generated, and so on. It's just less computation. I see now that people need to parse JSON in C++ as they would any other language, and someone has to do it, and it should be done as fast and correctly as possible. It's a different issue from defining a performant wire protocol with good developer ergonomics (which to be frank I am not sure why my brain even went there to begin with; I had protobuf on the brain). Makes sense.
- beached_whale 4y agoA lot. I know of a few people using my library are in finance doing some crypto currency stuff and the feeds are JSON and/or done via JSON-RPC I think. I didn't really get into the details. But even then, doing 2ish GB/s isn't that bad. But people seem to think of C++ as only systems language only ignoring the decades of desktop usage and existing software. Plus web services running are using small fractions of the resources that systems like node use. I know in some examples I have done where a C++ web service used about 3MB of ram, plus usage depending on size of the request, whereas Node started at 300MB. With cloud costs, that's a lot more bang for buck. But JSON is the lingua franca of networked I/O these days, need to interop