3 ms·
Is there any serialization format that is more "friendly" to memory allocators while still being human-readable and -writable?
by nerdponx 4y ago
Is there any serialization format that is more "friendly" to memory allocators while still being human-readable and -writable?
- deleted 4y ago[deleted]
- hexane360 4y agoIt's not about the format per se, but more about the fact that you're parsing an unknown/fully general structure in that format. APIs like Rust's serde can help to avoid excessive allocation when you have JSON in a known schema.
- adgjlsfhk1 4y agotext isn't readable without software. why should we expect binary data formats to be?
- oever 4y agoIf a file is so large that the processor spends a lot of time parsing, then the file is too large to be conveniently edited by a person. For large files it is best to use a binary format that can be read quickly without parsing or allocation. https://rkyv.org/ https://rkyv.org/ is an example. Being 'friendly' is not why JSON is popular. JSON is popular because the decoder is included the web browser.
- nerdponx 4y agoThanks, and good point about dataset size. I appreciate the thorough "shootout" benchmarks provided by the authors as well!