5 ms·
You are right. I totally missed it in the first few tables. The lack of allocs for JsonParser was definitely a big win for my project. When processing at hig
by nieksand 8y ago
You are right. I totally missed it in the first few tables.
The lack of allocs for JsonParser was definitely a big win for my project. When processing at high rps (40k msg/sec), GC is near the top of all my performance profiles. Driving allocs down with Buger's parser was very helpful for getting the throughput up.
- Twirrim 8y agoThere's an important distinction between the decoding speed of documents and overall throughput. Wonder what can be done to realistically generate something approximating a real world benchmark to cover the throughput scenario.
- zerr 8y agoWhat's the project? Seems like a language with a GC is not a good choice, is it?
- nieksand 8y agoEssentially a gloried batcher / data transformer consuming from Kafka. Even with the GC overhead, I will still able to hit my performance goal of 40k msg/sec per AWS c3.4xl. In a vacuum I would have picked Rust over Go for this. But Go is widely used at my work, whereas I'm the only Rust guy.
- woolvalley 8y agoHow much of a difference you think rust would of made? Have you made something similar?
- nieksand 8y agoI've written some high req/sec projects in Rust, but nothing that would be apples-to-apples to that particular Go project. The memory access patterns are straight-forward enough that the remaining GC could definitely go away. If I had to make a wild, from the hip, and unsubstantiated guess, maybe a 2x to 5x performance multiple. That would include a comparable amount of time spent optimizing and would depend greatly on the quality of the Kafka crates for Rust (which I've never used).