6 ms·
The fastest Go JSON parser I know of is https://github.com/buger/jsonparser https://github.com/buger/jsonparser. I've used that one in production quite success
by nieksand 8y ago
The fastest Go JSON parser I know of is https://github.com/buger/jsonparser https://github.com/buger/jsonparser. I've used that one in production quite successfully.
I don't see that in the comparative benchmarks for Gojay.
- mintcrate 8y agoThat project is not an encoder/decoder - it just returns values of keys. They are not comparable as the use cases are entirely different.
- nieksand 8y agoWhen it comes to parsing JSON, they are indeed not doing the same thing. Buger's parser is not unmarshalling into an annotated struct. But often that struct isn't your end goal in life, such as when your data is being transformed, dispatched on, or being moved into another yet data structure. In that case unmarshalling to an intermediate structure, while convenient, is also extra computational work. Buger's parser isn't the first thing I'd reach for, but if JSON parsing performance is an issue, ObjectEach() can be used in many places where you'd otherwise invoke json.Unmarshal.
- alex_dev 8y agoHence why the above poster said different use cases and why comparing between them is ridiculous. Using ObjectEach() for every field to reach parity with json.Unmarshal is unconscionable.
- latch 8y ago> I don't see that in the comparative benchmarks for Gojay. Look again, JasonParser is included in some, but not all, of the benchmarks. JasonParser [apparently] allocates no memory, which, to me, seems pretty compelling.
- nieksand 8y agoYou are right. I totally missed it in the first few tables. The lack of allocs for JsonParser was definitely a big win for my project. When processing at high rps (40k msg/sec), GC is near the top of all my performance profiles. Driving allocs down with Buger's parser was very helpful for getting the throughput up.
- Twirrim 8y agoThere's an important distinction between the decoding speed of documents and overall throughput. Wonder what can be done to realistically generate something approximating a real world benchmark to cover the throughput scenario.
- zerr 8y agoWhat's the project? Seems like a language with a GC is not a good choice, is it?
- nieksand 8y agoEssentially a gloried batcher / data transformer consuming from Kafka. Even with the GC overhead, I will still able to hit my performance goal of 40k msg/sec per AWS c3.4xl. In a vacuum I would have picked Rust over Go for this. But Go is widely used at my work, whereas I'm the only Rust guy.
- woolvalley 8y agoHow much of a difference you think rust would of made? Have you made something similar?
- nieksand 8y agoI've written some high req/sec projects in Rust, but nothing that would be apples-to-apples to that particular Go project. The memory access patterns are straight-forward enough that the remaining GC could definitely go away. If I had to make a wild, from the hip, and unsubstantiated guess, maybe a 2x to 5x performance multiple. That would include a comparable amount of time spent optimizing and would depend greatly on the quality of the Kafka crates for Rust (which I've never used).
- tomohawk 8y agoCheck the source to the benchmarks https://github.com/francoispqt/gojay#benchmark-results https://github.com/francoispqt/gojay#benchmark-results
- segmondy 8y agoIs it? Kubernete switched from the native go Json parser to codec for performance increase. http://ugorji.net/blog/go-codecgen http://ugorji.net/blog/go-codecgen