2 ms·
rust_serialization_benchmark: https://github.com/djkoloski/rust_serialization_benchmark https://github.com/djkoloski/rust_serialization_benchmark apache/arrow
by westurner 3y ago
rust_serialization_benchmark:
https://github.com/djkoloski/rust_serialization_benchmark https://github.com/djkoloski/rust_serialization_benchmark
apache/arrow-rs: https://github.com/apache/arrow-rs https://github.com/apache/arrow-rs
From https://arrow.apache.org/faq/ https://arrow.apache.org/faq/ :
> How does Arrow relate to Flatbuffers?
> Flatbuffers is a low-level building block for binary data serialization. It is not adapted to the representation of large, structured, homogenous data, and does not sit at the right abstraction layer for data analysis tasks.
> Arrow is a data layer aimed directly at the needs of data analysis, providing a comprehensive collection of data types required to analytics, built-in support for “null” values (representing missing data), and an expanding toolbox of I/O and computing facilities.
> The Arrow file format does use Flatbuffers under the hood to serialize schemas and other metadata needed to implement the Arrow binary IPC protocol, but the Arrow data format uses its own representation for optimal access and computation
- fbdab103 3y agoAnnoying how they mix µs and ms inside the same timing column. Makes it harder to visually assess.
- taintegral 3y agoMixing milliseconds down to picoseconds made the columns too wide. All of the benchmark data is also available as JSON in the benchmark_results directory if you wanted to drop it into sheets or some other analysis.
- coderedart 3y agoThe star of that benchmark is actually nanoserde. 1. outperforms a bunch of binary formats (as well as all other text formats) 2. has zero dependencies with fast compile times (even the derive macros are custom and don't use syn family crates as build-time deps)