3 ms·
> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this for
by nine_k 2y ago
> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.
That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.
- 90s_dev 2y agoWe are always reinventing wheels. If we didn't, they'd all still be made of wood.
- Zolomon 2y agoThey mention this in the article.
- spockz 2y agoIt reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?
- cstrahan 2y agoNo, one example of why being variable sized integers. See https://protobuf.dev/programming-guides/encoding/ https://protobuf.dev/programming-guides/encoding/
- carterschonwald 2y agoOne thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.