4 ms·
i wonder if it's better to FFI into this library when using node.js, rather than using JSON.parse()
by chii 6y ago
i wonder if it's better to FFI into this library when using node.js, rather than using JSON.parse()
- numlock86 6y agoI guess it depends in your use-case. Looks like this was primarily made for large JSON files and not the typical small JSON payloads you encounter with HTTP bodies and the like. On top of that JSON.parse() is pretty heavily optimized already. Profiling is key.
- lifthrasiir 6y agoJSON.parse() is required to return a concrete JS value [1]. Most, if not all, C/C++ JSON parsers designed for performance don't build the value unless requested. So whether to switch indeed depends on your use case, but for the different reason: if you use the majority of the parsed JSON your performance would be rather bound by object creation speed instead of parsing speed. [1] It might be the case that modern benchmark-obsessed JS engines defer JSON parsing in this case though, in which case you definitely should not switch to simdjson.
- rattray 6y agoThat is the case with this library as well: https://github.com/luizperes/simdjson_nodejs/issues/5 https://github.com/luizperes/simdjson_nodejs/issues/5
- Legogris 6y agoThe FFI overhead in nodejs is significant. We have a project where a while back, after profiling, the majority of the CPU time was spent in a couple of small hotspots doing parsing and object construction in Nodejs. I broke those out into a Rust library that was >100x faster (IIRC) in synthetic benchmarks with the same complexity. Plugging it in with FFI into the Nodejs app and it actually performed slightly worse due to FFI overhead and translation. So for large documents; could be worth it. For lots of small objects; probably not. You'd have to try on real-world data for your use-case to know. Also https://github.com/luizperes/simdjson_nodejs/issues/5 https://github.com/luizperes/simdjson_nodejs/issues/5
- dgb23 6y ago> I broke those out into a Rust library that was >100x faster (IIRC) in synthetic benchmarks with the same complexity. Plugging it in with FFI into the Nodejs app and it actually performed slightly worse due to FFI overhead and translation. This is interesting. Have you tried to optimize for reducing the overhead? I can imagine that this can be hard/complex and require a refactoring of the consumer in some scenarios.